libndr: moved the NDR_* flags to have less overlap

We have 3 different types of flags values in our NDR layer. We've
recently found bugs where these types of flags have been mixed up,
especially by people adding hand written ndr code for tricky
structures. We previously got away with this because (for example)
NDR_SCALARS and NDR_IN had the same value, so mixing up the two
concepts sometimes worked. Unfortunately it also led to bugs where we
didn't do what was expected, such as in our smbtorture ndr test suite,
where passing a ndr_flags value of zero led to only checking that two
empty structures were equal.

This changes the values of the NDR_IN|NDR_OUT and
NDR_SCALARS|NDR_BUFFERS values to be in different bit ranges, and adds
macros for checking the validity of passed in flags. A followup patch
modifies the ndr calls to use these macros, and pidl to generate
them. This should catch misuse of the APIs.

Pair-Programmed-With: Andrew Bartlett <abartlet@samba.org>

0 files changed, 0 insertions, 0 deletions