SAMBA Developers Guide

SAMBA Team



Abstract

Last Update : Mon aug 26 12:41:19 CEST 2002

This book is a collection of documents that might be useful for people developing samba or those interested in doing so. It's nothing more than a collection of documents written by samba developers about the internals of various parts of samba and the SMB protocol. It's still incomplete. The most recent version of this document can be found at http://devel.samba.org/. Please send updates to jelmer@samba.org.

This documentation is distributed under the GNU General Public License (GPL) version 2. A copy of the license is included with the Samba source distribution. A copy can be found on-line at http://www.fsf.org/licenses/gpl.txt

Table of Contents
Definition of NetBIOS Protocol and Name Resolution Modes
NETBIOS
BROADCAST NetBIOS
NBNS NetBIOS
Samba Architecture
Introduction
Multithreading and Samba
Threading smbd
Threading nmbd
nbmd Design
The samba DEBUG system
New Output Syntax
The DEBUG() Macro
The DEBUGADD() Macro
The DEBUGLVL() Macro
New Functions
dbgtext()
dbghdr()
format_debug_text()
Coding Suggestions
Samba Internals
Character Handling
The new functions
Macros in byteorder.h
CVAL(buf,pos)
PVAL(buf,pos)
SCVAL(buf,pos,val)
SVAL(buf,pos)
IVAL(buf,pos)
SVALS(buf,pos)
IVALS(buf,pos)
SSVAL(buf,pos,val)
SIVAL(buf,pos,val)
SSVALS(buf,pos,val)
SIVALS(buf,pos,val)
RSVAL(buf,pos)
RIVAL(buf,pos)
RSSVAL(buf,pos,val)
RSIVAL(buf,pos,val)
LAN Manager Samba API
Parameters
Return value
Code character table
The smb.conf file
Lexical Analysis
Handling of Whitespace
Handling of Line Continuation
Line Continuation Quirks
Syntax
About params.c
NetBIOS in a Unix World
Introduction
Usernames
File Ownership
Passwords
Locking
Deny Modes
Trapdoor UIDs
Port numbers
Protocol Complexity
Tracing samba system calls
NT Domain RPC's
Introduction
Sources
Credits
Notes and Structures
Notes
Enumerations
Structures
MSRPC over Transact Named Pipe
MSRPC Pipes
Header
Tail
RPC Bind / Bind Ack
NTLSA Transact Named Pipe
LSA Open Policy
LSA Query Info Policy
LSA Enumerate Trusted Domains
LSA Open Secret
LSA Close
LSA Lookup SIDS
LSA Lookup Names
NETLOGON rpc Transact Named Pipe
LSA Request Challenge
LSA Authenticate 2
LSA Server Password Set
LSA SAM Logon
LSA SAM Logoff
\\MAILSLOT\NET\NTLOGON
Query for PDC
SAM Logon
SRVSVC Transact Named Pipe
Net Share Enum
Net Server Get Info
Cryptographic side of NT Domain Authentication
Definitions
Protocol
Comments
SIDs and RIDs
Well-known SIDs
Well-known RIDS

Definition of NetBIOS Protocol and Name Resolution Modes

NETBIOS

NetBIOS runs over the following tranports: TCP/IP; NetBEUI and IPX/SPX. Samba only uses NetBIOS over TCP/IP. For details on the TCP/IP NetBIOS Session Service NetBIOS Datagram Service, and NetBIOS Names, see rfc1001.txt and rfc1002.txt.

NetBEUI is a raw NetBIOS frame protocol implementation that allows NetBIOS datagrams to be sent out over the 'wire' embedded within LLC frames. NetBEUI is not required when using NetBIOS over TCP/IP protocols and it is preferable NOT to install NetBEUI if it can be avoided.

IPX/SPX is also not required when using NetBIOS over TCP/IP, and it is preferable NOT to install the IPX/SPX transport unless you are using Novell servers. At the very least, it is recommended that you do not install 'NetBIOS over IPX/SPX'.

[When installing Windows 95, you will find that NetBEUI and IPX/SPX are installed as the default protocols. This is because they are the simplest to manage: no Windows 95 user-configuration is required].

NetBIOS applications (such as samba) offer their services (for example, SMB file and print sharing) on a NetBIOS name. They must claim this name on the network before doing so. The NetBIOS session service will then accept connections on the application's behalf (on the NetBIOS name claimed by the application). A NetBIOS session between the application and the client can then commence.

NetBIOS names consist of 15 characters plus a 'type' character. This is similar, in concept, to an IP address and a TCP port number, respectively. A NetBIOS-aware application on a host will offer different services under different NetBIOS name types, just as a host will offer different TCP/IP services on different port numbers.

NetBIOS names must be claimed on a network, and must be defended. The use of NetBIOS names is most suitable on a single subnet; a Local Area Network or a Wide Area Network.

NetBIOS names are either UNIQUE or GROUP. Only one application can claim a UNIQUE NetBIOS name on a network.

There are two kinds of NetBIOS Name resolution: Broadcast and Point-to-Point.


NBNS NetBIOS

rfc1001.txt describes, amongst other things, the implementation and use of, a 'NetBIOS Name Service'. NT/AS offers 'Windows Internet Name Service' which is fully rfc1001/2 compliant, but has had to take specific action with certain NetBIOS names in order to make it useful. (for example, it deals with the registration of <1c> <1d> <1e> names all in different ways. I recommend the reading of the Microsoft WINS Server Help files for full details).

The use of a WINS server cuts down on broadcast network traffic for NetBIOS name resolution. It has the effect of pulling all the broadcast isolated subnets together into a single NetBIOS scope, across your LAN or WAN, while avoiding the use of TCP/IP broadcast packets.

When you have a WINS server on your LAN, WINS clients will be able to contact the WINS server to resolve NetBIOS names. Note that only those WINS clients that have registered with the same WINS server will be visible. The WINS server _can_ have static NetBIOS entries added to its database (usually for security reasons you might want to consider putting your domain controllers or other important servers as static entries, but you should not rely on this as your sole means of security), but for the most part, NetBIOS names are registered dynamically.

This provides some confusion for lots of people, and is worth mentioning here: a Browse Server is NOT a WINS Server, even if these services are implemented in the same application. A Browse Server _needs_ a WINS server because a Browse Server is a WINS client, which is _not_ the same thing].

Clients can claim names, and therefore offer services on successfully claimed names, on their broadcast-isolated subnet. One way to get NetBIOS services (such as browsing: see ftp.microsoft.com/drg/developr/CIFS/browdiff.txt; and SMB file/print sharing: see cifs6.txt) working on a LAN or WAN is to make your routers forward all broadcast packets from TCP/IP ports 137, 138 and 139. You will find, however, if you do this on a large LAN or a WAN, that your network is completely swamped by NetBIOS and browsing packets, which is why WINS was developed to minimise the necessity of broadcast traffic.

WINS Clients therefore claim names from the WINS server. If the WINS server allows them to register a name, the client's NetBIOS session service can then offer services on this name. Other WINS clients will then contact the WINS server to resolve a NetBIOS name.


Samba Architecture


Threading nmbd

This would be ideal, but gets sunk by portability requirements.

Andrew tried to write a test threads library for nmbd that used only ansi-C constructs (using setjmp and longjmp). Unfortunately some OSes defeat this by restricting longjmp to calling addresses that are shallower than the current address on the stack (apparently AIX does this). This makes a truly portable threads library impossible. So to support all our current platforms we would have to code nmbd both with and without threads, and as the real aim of threads is to make the code clearer we would not have gained anything. (it is a myth that threads make things faster. threading is like recursion, it can make things clear but the same thing can always be done faster by some other method)

Chris tried to spec out a general design that would abstract threading vs separate processes (vs other methods?) and make them accessible through some general API. This doesn't work because of the data sharing requirements of the protocol (packets in the future depending on packets now, etc.) At least, the code would work but would be very clumsy, and besides the fork() type model would never work on Unix. (Is there an OS that it would work on, for nmbd?)

A fork() is cheap, but not nearly cheap enough to do on every UDP packet that arrives. Having a pool of processes is possible but is nasty to program cleanly due to the enormous amount of shared data (in complex structures) between the processes. We can't rely on each platform having a shared memory system.


The samba DEBUG system

New Output Syntax

The syntax of a debugging log file is represented as:

  >debugfile< :== { >debugmsg< }

  >debugmsg<  :== >debughdr< '\n' >debugtext<

  >debughdr<  :== '[' TIME ',' LEVEL ']' FILE ':' [FUNCTION] '(' LINE ')'

  >debugtext< :== { >debugline< }

  >debugline< :== TEXT '\n'

TEXT is a string of characters excluding the newline character.

LEVEL is the DEBUG level of the message (an integer in the range 0..10).

TIME is a timestamp.

FILE is the name of the file from which the debug message was generated.

FUNCTION is the function from which the debug message was generated.

LINE is the line number of the debug statement that generated the message.

Basically, what that all means is:

  1. A debugging log file is made up of debug messages.

  2. Each debug message is made up of a header and text. The header is separated from the text by a newline.

  3. The header begins with the timestamp and debug level of the message enclosed in brackets. The filename, function, and line number at which the message was generated follow. The filename is terminated by a colon, and the function name is terminated by the parenthesis which contain the line number. Depending upon the compiler, the function name may be missing (it is generated by the __FUNCTION__ macro, which is not universally implemented, dangit).

  4. The message text is made up of zero or more lines, each terminated by a newline.

Here's some example output:

    [1998/08/03 12:55:25, 1] nmbd.c:(659)
      Netbios nameserver version 1.9.19-prealpha started.
      Copyright Andrew Tridgell 1994-1997
    [1998/08/03 12:55:25, 3] loadparm.c:(763)
      Initializing global parameters

Note that in the above example the function names are not listed on the header line. That's because the example above was generated on an SGI Indy, and the SGI compiler doesn't support the __FUNCTION__ macro.


The DEBUG() Macro

Use of the DEBUG() macro is unchanged. DEBUG() takes two parameters. The first is the message level, the second is the body of a function call to the Debug1() function.

That's confusing.

Here's an example which may help a bit. If you would write

printf( "This is a %s message.\n", "debug" );

to send the output to stdout, then you would write

DEBUG( 0, ( "This is a %s message.\n", "debug" ) );

to send the output to the debug file. All of the normal printf() formatting escapes work.

Note that in the above example the DEBUG message level is set to 0. Messages at level 0 always print. Basically, if the message level is less than or equal to the global value DEBUGLEVEL, then the DEBUG statement is processed.

The output of the above example would be something like:

    [1998/07/30 16:00:51, 0] file.c:function(128)
      This is a debug message.

Each call to DEBUG() creates a new header *unless* the output produced by the previous call to DEBUG() did not end with a '\n'. Output to the debug file is passed through a formatting buffer which is flushed every time a newline is encountered. If the buffer is not empty when DEBUG() is called, the new input is simply appended.

...but that's really just a Kludge. It was put in place because DEBUG() has been used to write partial lines. Here's a simple (dumb) example of the kind of thing I'm talking about:

    DEBUG( 0, ("The test returned " ) );
    if( test() )
      DEBUG(0, ("True") );
    else
      DEBUG(0, ("False") );
    DEBUG(0, (".\n") );

Without the format buffer, the output (assuming test() returned true) would look like this:

    [1998/07/30 16:00:51, 0] file.c:function(256)
      The test returned
    [1998/07/30 16:00:51, 0] file.c:function(258)
      True
    [1998/07/30 16:00:51, 0] file.c:function(261)
      .

Which isn't much use. The format buffer kludge fixes this problem.


The DEBUGADD() Macro

In addition to the kludgey solution to the broken line problem described above, there is a clean solution. The DEBUGADD() macro never generates a header. It will append new text to the current debug message even if the format buffer is empty. The syntax of the DEBUGADD() macro is the same as that of the DEBUG() macro.

    DEBUG( 0, ("This is the first line.\n" ) );
    DEBUGADD( 0, ("This is the second line.\nThis is the third line.\n" ) );

Produces

    [1998/07/30 16:00:51, 0] file.c:function(512)
      This is the first line.
      This is the second line.
      This is the third line.


The DEBUGLVL() Macro

One of the problems with the DEBUG() macro was that DEBUG() lines tended to get a bit long. Consider this example from nmbd_sendannounce.c:

  DEBUG(3,("send_local_master_announcement: type %x for name %s on subnet %s for workgroup %s\n",
            type, global_myname, subrec->subnet_name, work->work_group));

One solution to this is to break it down using DEBUG() and DEBUGADD(), as follows:

  DEBUG( 3, ( "send_local_master_announcement: " ) );
  DEBUGADD( 3, ( "type %x for name %s ", type, global_myname ) );
  DEBUGADD( 3, ( "on subnet %s ", subrec->subnet_name ) );
  DEBUGADD( 3, ( "for workgroup %s\n", work->work_group ) );

A similar, but arguably nicer approach is to use the DEBUGLVL() macro. This macro returns True if the message level is less than or equal to the global DEBUGLEVEL value, so:

  if( DEBUGLVL( 3 ) )
    {
    dbgtext( "send_local_master_announcement: " );
    dbgtext( "type %x for name %s ", type, global_myname );
    dbgtext( "on subnet %s ", subrec->subnet_name );
    dbgtext( "for workgroup %s\n", work->work_group );
    }

(The dbgtext() function is explained below.)

There are a few advantages to this scheme:

  1. The test is performed only once.

  2. You can allocate variables off of the stack that will only be used within the DEBUGLVL() block.

  3. Processing that is only relevant to debug output can be contained within the DEBUGLVL() block.


Coding Suggestions

So you want to add code to Samba ...

One of the daunting tasks facing a programmer attempting to write code for Samba is understanding the various coding conventions used by those most active in the project. These conventions were mostly unwritten and helped improve either the portability, stability or consistency of the code. This document will attempt to document a few of the more important coding practices used at this time on the Samba project. The coding practices are expected to change slightly over time, and even to grow as more is learned about obscure portability considerations. Two existing documents samba/source/internals.doc and samba/source/architecture.doc provide additional information.

The loosely related question of coding style is very personal and this document does not attempt to address that subject, except to say that I have observed that eight character tabs seem to be preferred in Samba source. If you are interested in the topic of coding style, two oft-quoted documents are:

http://lxr.linux.no/source/Documentation/CodingStyle

http://www.fsf.org/prep/standards_toc.html

But note that coding style in Samba varies due to the many different programmers who have contributed.

Following are some considerations you should use when adding new code to Samba. First and foremost remember that:

Portability is a primary consideration in adding function, as is network compatability with de facto, existing, real world CIFS/SMB implementations. There are lots of platforms that Samba builds on so use caution when adding a call to a library function that is not invoked in existing Samba code. Also note that there are many quite different SMB/CIFS clients that Samba tries to support, not all of which follow the SNIA CIFS Technical Reference (or the earlier Microsoft reference documents or the X/Open book on the SMB Standard) perfectly.

Here are some other suggestions:

  1. use d_printf instead of printf for display text reason: enable auto-substitution of translated language text

  2. use SAFE_FREE instead of free reason: reduce traps due to null pointers

  3. don't use bzero use memset, or ZERO_STRUCT and ZERO_STRUCTP macros reason: not POSIX

  4. don't use strcpy and strlen (use safe_* equivalents) reason: to avoid traps due to buffer overruns

  5. don't use getopt_long, use popt functions instead reason: portability

  6. explicitly add const qualifiers on parm passing in functions where parm is input only (somewhat controversial but const can be #defined away)

  7. when passing a va_list as an arg, or assigning one to another please use the VA_COPY() macro reason: on some platforms, va_list is a struct that must be initialized in each function...can SEGV if you don't.

  8. discourage use of threads reason: portability (also see architecture.doc)

  9. don't explicitly include new header files in C files - new h files should be included by adding them once to includes.h reason: consistency

  10. don't explicitly extern functions (they are autogenerated by "make proto" into proto.h) reason: consistency

  11. use endian safe macros when unpacking SMBs (see byteorder.h and internals.doc) reason: not everyone uses Intel

  12. Note Unicode implications of charset handling (see internals.doc). See pull_* and push_* and convert_string functions. reason: Internationalization

  13. Don't assume English only reason: See above

  14. Try to avoid using in/out parameters (functions that return data which overwrites input parameters) reason: Can cause stability problems

  15. Ensure copyright notices are correct, don't append Tridge's name to code that he didn't write. If you did not write the code, make sure that it can coexist with the rest of the Samba GPLed code.

  16. Consider usage of DATA_BLOBs for length specified byte-data. reason: stability

  17. Take advantage of tdbs for database like function reason: consistency

  18. Don't access the SAM_ACCOUNT structure directly, they should be accessed via pdb_get...() and pdb_set...() functions. reason: stability, consistency

  19. Don't check a password directly against the passdb, always use the check_password() interface. reason: long term pluggability

  20. Try to use asprintf rather than pstrings and fstrings where possible

  21. Use normal C comments / * instead of C++ comments // like this. Although the C++ comment format is part of the C99 standard, some older vendor C compilers do not accept it.

  22. Try to write documentation for API functions and structures explaining the point of the code, the way it should be used, and any special conditions or results. Mark these with a double-star comment start / ** so that they can be picked up by Doxygen, as in this file.

  23. Keep the scope narrow. This means making functions/variables static whenever possible. We don't want our namespace polluted. Each module should have a minimal number of externally visible functions or variables.

  24. Use function pointers to keep knowledge about particular pieces of code isolated in one place. We don't want a particular piece of functionality to be spread out across lots of places - that makes for fragile, hand to maintain code. Instead, design an interface and use tables containing function pointers to implement specific functionality. This is particularly important for command interpreters.

  25. Think carefully about what it will be like for someone else to add to and maintain your code. If it would be hard for someone else to maintain then do it another way.

The suggestions above are simply that, suggestions, but the information may help in reducing the routine rework done on new code. The preceeding list is expected to change routinely as new support routines and macros are added.


Samba Internals


The new functions

The new system works like this:

  1. all char* strings inside Samba are "unix" strings. These are multi-byte strings that are in the charset defined by the "unix charset" option in smb.conf.

  2. there is no single fixed character set for unix strings, but any character set that is used does need the following properties:

    1. must not contain NULLs except for termination

    2. must be 7-bit compatible with C strings, so that a constant string or character in C will be byte-for-byte identical to the equivalent string in the chosen character set.

    3. when you uppercase or lowercase a string it does not become longer than the original string

    4. must be able to correctly hold all characters that your client will throw at it

    For example, UTF-8 is fine, and most multi-byte asian character sets are fine, but UCS2 could not be used for unix strings as they contain nulls.

  3. when you need to put a string into a buffer that will be sent on the wire, or you need a string in a character set format that is compatible with the clients character set then you need to use a pull_ or push_ function. The pull_ functions pull a string from a wire buffer into a (multi-byte) unix string. The push_ functions push a string out to a wire buffer.

  4. the two main pull_ and push_ functions you need to understand are pull_string and push_string. These functions take a base pointer that should point at the start of the SMB packet that the string is in. The functions will check the flags field in this packet to automatically determine if the packet is marked as a unicode packet, and they will choose whether to use unicode for this string based on that flag. You may also force this decision using the STR_UNICODE or STR_ASCII flags. For use in smbd/ and libsmb/ there are wrapper functions clistr_ and srvstr_ that call the pull_/push_ functions with the appropriate first argument.

    You may also call the pull_ascii/pull_ucs2 or push_ascii/push_ucs2 functions if you know that a particular string is ascii or unicode. There are also a number of other convenience functions in charcnv.c that call the pull_/push_ functions with particularly common arguments, such as pull_ascii_pstring()

  5. The biggest thing to remember is that internal (unix) strings in Samba may now contain multi-byte characters. This means you cannot assume that characters are always 1 byte long. Often this means that you will have to convert strings to ucs2 and back again in order to do some (seemingly) simple task. For examples of how to do this see functions like strchr_m(). I know this is very slow, and we will eventually speed it up but right now we want this stuff correct not fast.

  6. all lp_ functions now return unix strings. The magic "DOS" flag on parameters is gone.

  7. all vfs functions take unix strings. Don't convert when passing to them


Macros in byteorder.h

This section describes the macros defined in byteorder.h. These macros are used extensively in the Samba code.


LAN Manager Samba API

This section describes the functions need to make a LAN Manager RPC call. This information had been obtained by examining the Samba code and the LAN Manager 2.0 API documentation. It should not be considered entirely reliable.

call_api(int prcnt, int drcnt, int mprcnt, int mdrcnt, 
	char *param, char *data, char **rparam, char **rdata);

This function is defined in client.c. It uses an SMB transaction to call a remote api.


Parameters

The parameters are as follows:

  1. prcnt: the number of bytes of parameters begin sent.

  2. drcnt: the number of bytes of data begin sent.

  3. mprcnt: the maximum number of bytes of parameters which should be returned

  4. mdrcnt: the maximum number of bytes of data which should be returned

  5. param: a pointer to the parameters to be sent.

  6. data: a pointer to the data to be sent.

  7. rparam: a pointer to a pointer which will be set to point to the returned paramters. The caller of call_api() must deallocate this memory.

  8. rdata: a pointer to a pointer which will be set to point to the returned data. The caller of call_api() must deallocate this memory.

These are the parameters which you ought to send, in the order of their appearance in the parameter block:

  1. An unsigned 16 bit integer API number. You should set this value with SSVAL(). I do not know where these numbers are described.

  2. An ASCIIZ string describing the parameters to the API function as defined in the LAN Manager documentation. The first parameter, which is the server name, is ommited. This string is based uppon the API function as described in the manual, not the data which is actually passed.

  3. An ASCIIZ string describing the data structure which ought to be returned.

  4. Any parameters which appear in the function call, as defined in the LAN Manager API documentation, after the "Server" and up to and including the "uLevel" parameters.

  5. An unsigned 16 bit integer which gives the size in bytes of the buffer we will use to receive the returned array of data structures. Presumably this should be the same as mdrcnt. This value should be set with SSVAL().

  6. An ASCIIZ string describing substructures which should be returned. If no substructures apply, this string is of zero length.

The code in client.c always calls call_api() with no data. It is unclear when a non-zero length data buffer would be sent.


Return value

The returned parameters (pointed to by rparam), in their order of appearance are:

  1. An unsigned 16 bit integer which contains the API function's return code. This value should be read with SVAL().

  2. An adjustment which tells the amount by which pointers in the returned data should be adjusted. This value should be read with SVAL(). Basically, the address of the start of the returned data buffer should have the returned pointer value added to it and then have this value subtracted from it in order to obtain the currect offset into the returned data buffer.

  3. A count of the number of elements in the array of structures returned. It is also possible that this may sometimes be the number of bytes returned.

When call_api() returns, rparam points to the returned parameters. The first if these is the result code. It will be zero if the API call suceeded. This value by be read with "SVAL(rparam,0)".

The second parameter may be read as "SVAL(rparam,2)". It is a 16 bit offset which indicates what the base address of the returned data buffer was when it was built on the server. It should be used to correct pointer before use.

The returned data buffer contains the array of returned data structures. Note that all pointers must be adjusted before use. The function fix_char_ptr() in client.c can be used for this purpose.

The third parameter (which may be read as "SVAL(rparam,4)") has something to do with indicating the amount of data returned or possibly the amount of data which can be returned if enough buffer space is allowed.


The smb.conf file

Lexical Analysis

Basically, the file is processed on a line by line basis. There are four types of lines that are recognized by the lexical analyzer (params.c):

  1. Blank lines - Lines containing only whitespace.

  2. Comment lines - Lines beginning with either a semi-colon or a pound sign (';' or '#').

  3. Section header lines - Lines beginning with an open square bracket ('[').

  4. Parameter lines - Lines beginning with any other character. (The default line type.)

The first two are handled exclusively by the lexical analyzer, which ignores them. The latter two line types are scanned for

  1. - Section names

  2. - Parameter names

  3. - Parameter values

These are the only tokens passed to the parameter loader (loadparm.c). Parameter names and values are divided from one another by an equal sign: '='.


Line Continuation Quirks

Note the following example:

	param name = parameter value string \
    \
    with line continuation.

The middle line is *not* parsed as a blank line because it is first concatonated with the top line. The result is

param name = parameter value string         with line continuation.

The same is true for comment lines.

	param name = parameter value string \
	; comment \
    with a comment.

This becomes:

param name = parameter value string     ; comment     with a comment.

On a section header line, the closing bracket (']') is considered a terminating character, and the rest of the line is ignored. The lines

	[ section   name ] garbage \
    param  name  = value

are read as

	[section name]
    param name = value


NetBIOS in a Unix World


Usernames

The SMB protocol has only a loose username concept. Early SMB protocols (such as CORE and COREPLUS) have no username concept at all. Even in later protocols clients often attempt operations (particularly printer operations) without first validating a username on the server.

Unix security is based around username/password pairs. A unix box should not allow clients to do any substantive operation without some sort of validation.

The problem mostly manifests itself when the unix server is in "share level" security mode. This is the default mode as the alternative "user level" security mode usually forces a client to connect to the server as the same user for each connected share, which is inconvenient in many sites.

In "share level" security the client normally gives a username in the "session setup" protocol, but does not supply an accompanying password. The client then connects to resources using the "tree connect" protocol, and supplies a password. The problem is that the user on the PC types the username and the password in different contexts, unaware that they need to go together to give access to the server. The username is normally the one the user typed in when they "logged onto" the PC (this assumes Windows for Workgroups). The password is the one they chose when connecting to the disk or printer.

The user often chooses a totally different username for their login as for the drive connection. Often they also want to access different drives as different usernames. The unix server needs some way of divining the correct username to combine with each password.

Samba tries to avoid this problem using several methods. These succeed in the vast majority of cases. The methods include username maps, the service%user syntax, the saving of session setup usernames for later validation and the derivation of the username from the service name (either directly or via the user= option).


Protocol Complexity

There are many "protocol levels" in the SMB protocol. It seems that each time new functionality was added to a Microsoft operating system, they added the equivalent functions in a new protocol level of the SMB protocol to "externalise" the new capabilities.

This means the protocol is very "rich", offering many ways of doing each file operation. This means SMB servers need to be complex and large. It also means it is very difficult to make them bug free. It is not just Samba that suffers from this problem, other servers such as WinNT don't support every variation of every call and it has almost certainly been a headache for MS developers to support the myriad of SMB calls that are available.

There are about 65 "top level" operations in the SMB protocol (things like SMBread and SMBwrite). Some of these include hundreds of sub-functions (SMBtrans has at least 120 sub-functions, like DosPrintQAdd and NetSessionEnum). All of them take several options that can change the way they work. Many take dozens of possible "information levels" that change the structures that need to be returned. Samba supports all but 2 of the "top level" functions. It supports only 8 (so far) of the SMBtrans sub-functions. Even NT doesn't support them all.

Samba currently supports up to the "NT LM 0.12" protocol, which is the one preferred by Win95 and WinNT3.5. Luckily this protocol level has a "capabilities" field which specifies which super-duper new-fangled options the server suports. This helps to make the implementation of this protocol level much easier.

There is also a problem with the SMB specications. SMB is a X/Open spec, but the X/Open book is far from ideal, and fails to cover many important issues, leaving much to the imagination. Microsoft recently renamed the SMB protocol CIFS (Common Internet File System) and have published new specifications. These are far superior to the old X/Open documents but there are still undocumented calls and features. This specification is actively being worked on by a CIFS developers mailing list hosted by Microsft.


Tracing samba system calls

This file describes how to do a system call trace on Samba to work out what its doing wrong. This is not for the faint of heart, but if you are reading this then you are probably desperate.

Actually its not as bad as the the above makes it sound, just don't expect the output to be very pretty :-)

Ok, down to business. One of the big advantages of unix systems is that they nearly all come with a system trace utility that allows you to monitor all system calls that a program is making. This is extremely using for debugging and also helps when trying to work out why something is slower than you expect. You can use system tracing without any special compilation options.

The system trace utility is called different things on different systems. On Linux systems its called strace. Under SunOS 4 its called trace. Under SVR4 style systems (including solaris) its called truss. Under many BSD systems its called ktrace.

The first thing you should do is read the man page for your native system call tracer. In the discussion below I'll assume its called strace as strace is the only portable system tracer (its available for free for many unix types) and its also got some of the nicest features.

Next, try using strace on some simple commands. For example, strace ls or strace echo hello.

You'll notice that it produces a LOT of output. It is showing you the arguments to every system call that the program makes and the result. Very little happens in a program without a system call so you get lots of output. You'll also find that it produces a lot of "preamble" stuff showing the loading of shared libraries etc. Ignore this (unless its going wrong!)

For example, the only line that really matters in the strace echo hello output is:

write(1, "hello\n", 6)                  = 6

all the rest is just setting up to run the program.

Ok, now you're familiar with strace. To use it on Samba you need to strace the running smbd daemon. The way I tend ot use it is to first login from my Windows PC to the Samba server, then use smbstatus to find which process ID that client is attached to, then as root I do strace -p PID to attach to that process. I normally redirect the stderr output from this command to a file for later perusal. For example, if I'm using a csh style shell:

strace -f -p 3872 >& strace.out

or with a sh style shell:

strace -f -p 3872 > strace.out 2>&1

Note the "-f" option. This is only available on some systems, and allows you to trace not just the current process, but any children it forks. This is great for finding printing problems caused by the "print command" being wrong.

Once you are attached you then can do whatever it is on the client that is causing problems and you will capture all the system calls that smbd makes.

So how do you interpret the results? Generally I search through the output for strings that I know will appear when the problem happens. For example, if I am having touble with permissions on a file I would search for that files name in the strace output and look at the surrounding lines. Another trick is to match up file descriptor numbers and "follow" what happens to an open file until it is closed.

Beyond this you will have to use your initiative. To give you an idea of what you are looking for here is a piece of strace output that shows that /dev/null is not world writeable, which causes printing to fail with Samba:

[pid 28268] open("/dev/null", O_RDWR)   = -1 EACCES (Permission denied)
[pid 28268] open("/dev/null", O_WRONLY) = -1 EACCES (Permission denied)

The process is trying to first open /dev/null read-write then read-only. Both fail. This means /dev/null has incorrect permissions.


NT Domain RPC's

Introduction

This document contains information to provide an NT workstation with login services, without the need for an NT server. It is the sgml version of http://mailhost.cb1.com/~lkcl/cifsntdomain.txt, controlled by Luke.

It should be possible to select a domain instead of a workgroup (in the NT workstation's TCP/IP settings) and after the obligatory reboot, type in a username, password, select a domain and successfully log in. I would appreciate any feedback on your experiences with this process, and any comments, corrections and additions to this document.

The packets described here can be easily derived from (and are probably better understood using) Netmon.exe. You will need to use the version of Netmon that matches your system, in order to correctly decode the NETLOGON, lsarpc and srvsvc Transact pipes. This document is derived from NT Service Pack 1 and its corresponding version of Netmon. It is intended that an annotated packet trace be produced, which will likely be more instructive than this document.

Also needed, to fully implement NT Domain Login Services, is the document describing the cryptographic part of the NT authentication. This document is available from comp.protocols.smb; from the ntsecurity.net digest and from the samba digest, amongst other sources.

A copy is available from:

http://ntbugtraq.rc.on.ca/SCRIPTS/WA.EXE?A2=ind9708;L=ntbugtraq;O=A;P=2935

http://mailhost.cb1.com/~lkcl/crypt.html

A c-code implementation, provided by Linus Nordberg of this protocol is available from:

http://samba.org/cgi-bin/mfs/01/digest/1997/97aug/0391.html

http://mailhost.cb1.com/~lkcl/crypt.txt

Also used to provide debugging information is the Check Build version of NT workstation, and enabling full debugging in NETLOGON. This is achieved by setting the following REG_SZ registry key to 0x1ffffff:

HKLM\SYSTEM\CurrentControlSet\Services\Netlogon\Parameters

Incorrect direct editing of the registry can cause your machine to fail. Then again, so can incorrect implementation of this protocol. See "Liability:" above.

Bear in mind that each packet over-the-wire will have its origin in an API call. Therefore, there are likely to be structures, enumerations and defines that are usefully documented elsewhere.

This document is by no means complete or authoritative. Missing sections include, but are not limited to:

  1. Mappings of RIDs to usernames (and vice-versa).

  2. What a User ID is and what a Group ID is.

  3. The exact meaning/definition of various magic constants or enumerations.

  4. The reply error code and use of that error code when a workstation becomes a member of a domain (to be described later). Failure to return this error code will make the workstation report that it is already a member of the domain.

  5. the cryptographic side of the NetrServerPasswordSet command, which would allow the workstation to change its password. This password is used to generate the long-term session key. [It is possible to reject this command, and keep the default workstation password].


Sources

cket Traces from Netmonitor (Service Pack 1 and above)
ul Ashton and Luke Leighton's other "NT Domain" doc.
FS documentation - cifs6.txt
FS documentation - cifsrap2.txt


Credits

Paul Ashton: loads of work with Net Monitor; understanding the NT authentication system; reference implementation of the NT domain support on which this document is originally based.
Duncan Stansfield: low-level analysis of MSRPC Pipes.
Linus Nordberg: producing c-code from Paul's crypto spec.
Windows Sourcer development team


Notes and Structures

Notes

  1. In the SMB Transact pipes, some "Structures", described here, appear to be 4-byte aligned with the SMB header, at their start. Exactly which "Structures" need aligning is not precisely known or documented.

  2. In the UDP NTLOGON Mailslots, some "Structures", described here, appear to be 2-byte aligned with the start of the mailslot, at their start.

  3. Domain SID is of the format S-revision-version-auth1-auth2...authN. e.g S-1-5-123-456-789-123-456. the 5 could be a sub-revision.

  4. any undocumented buffer pointers must be non-zero if the string buffer it refers to contains characters. exactly what value they should be is unknown. 0x0000 0002 seems to do the trick to indicate that the buffer exists. a NULL buffer pointer indicates that the string buffer is of zero length. If the buffer pointer is NULL, then it is suspected that the structure it refers to is NOT put into (or taken out of) the SMB data stream. This is empirically derived from, for example, the LSA SAM Logon response packet, where if the buffer pointer is NULL, the user information is not inserted into the data stream. Exactly what happens with an array of buffer pointers is not known, although an educated guess can be made.

  5. an array of structures (a container) appears to have a count and a pointer. if the count is zero, the pointer is also zero. no further data is put into or taken out of the SMB data stream. if the count is non-zero, then the pointer is also non-zero. immediately following the pointer is the count again, followed by an array of container sub-structures. the count appears a third time after the last sub-structure.


Structures


SH_INFO_1_PTR (pointers to level 1 share info strings)

Note: see cifsrap2.txt section5, page 10.

0 for shi1_type indicates a Disk.
1 for shi1_type indicates a Print Queue.
2 for shi1_type indicates a Device.
3 for shi1_type indicates an IPC pipe.
0x8000 0000 (top bit set in shi1_type) indicates a hidden share.


SERVER_INFO_101

Note: see cifs6.txt section 6.4 - the fields described therein will be of assistance here. for example, the type listed below is the same as fServerType, which is described in 6.4.1.


MSRPC over Transact Named Pipe

For details on the SMB Transact Named Pipe, see cifs6.txt


MSRPC Pipes

The MSRPC is conducted over an SMB Transact Pipe with a name of \PIPE\. You must first obtain a 16 bit file handle, by sending a SMBopenX with the pipe name \PIPE\srvsvc for example. You can then perform an SMB Trans, and must carry out an SMBclose on the file handle once you are finished.

Trans Requests must be sent with two setup UINT16s, no UINT16 params (none known about), and UINT8 data parameters sufficient to contain the MSRPC header, and MSRPC data. The first UINT16 setup parameter must be either 0x0026 to indicate an RPC, or 0x0001 to indicate Set Named Pipe Handle state. The second UINT16 parameter must be the file handle for the pipe, obtained above.

The Data section for an API Command of 0x0026 (RPC pipe) in the Trans Request is the RPC Header, followed by the RPC Data. The Data section for an API Command of 0x0001 (Set Named Pipe Handle state) is two bytes. The only value seen for these two bytes is 0x00 0x43.

MSRPC Responses are sent as response data inside standard SMB Trans responses, with the MSRPC Header, MSRPC Data and MSRPC tail.

It is suspected that the Trans Requests will need to be at least 2-byte aligned (probably 4-byte). This is standard practice for SMBs. It is also independent of the observed 4-byte alignments with the start of the MSRPC header, including the 4-byte alignment between the MSRPC header and the MSRPC data.

First, an SMBtconX connection is made to the IPC$ share. The connection must be made using encrypted passwords, not clear-text. Then, an SMBopenX is made on the pipe. Then, a Set Named Pipe Handle State must be sent, after which the pipe is ready to accept API commands. Lastly, and SMBclose is sent.

To be resolved:

lkcl/01nov97 there appear to be two additional bytes after the null-terminated \PIPE\ name for the RPC pipe. Values seen so far are listed below:

        initial SMBopenX request:         RPC API command 0x26 params:
        "\\PIPE\\lsarpc"                  0x65 0x63; 0x72 0x70; 0x44 0x65;
        "\\PIPE\\srvsvc"                  0x73 0x76; 0x4E 0x00; 0x5C 0x43;


Header

[section to be rewritten, following receipt of work by Duncan Stansfield]

Interesting note: if you set packed data representation to 0x0100 0000 then all 4-byte and 2-byte word ordering is turned around!

The start of each of the NTLSA and NETLOGON named pipes begins with:

offset: 00

Variable type: UINT8

Variable data: 5 - RPC major version

offset: 01

Variable type: UINT8

Variable data: 0 - RPC minor version

offset: 02

Variable type: UINT8

Variable data: 2 - RPC response packet

offset: 03

Variable type: UINT8

Variable data: 3 - (FirstFrag bit-wise or with LastFrag)

offset: 04

Variable type: UINT32

Variable data: 0x1000 0000 - packed data representation

offset: 08

Variable type: UINT16

Variable data: fragment length - data size (bytes) inc header and tail.

offset: 0A

Variable type: UINT16

Variable data: 0 - authentication length

offset: 0C

Variable type: UINT32

Variable data: call identifier. matches 12th UINT32 of incoming RPC data.

offset: 10

Variable type: UINT32

Variable data: allocation hint - data size (bytes) minus header and tail.

offset: 14

Variable type: UINT16

Variable data: 0 - presentation context identifier

offset: 16

Variable type: UINT8

Variable data: 0 - cancel count

offset: 17

Variable type: UINT8

Variable data: in replies: 0 - reserved; in requests: opnum - see #defines.

offset: 18

Variable type: ......

Variable data: start of data (goes on for allocation_hint bytes)


Interface identification

the interfaces are numbered. as yet I haven't seen more than one interface used on the same pipe name srvsvc

abstract (0x4B324FC8, 0x01D31670, 0x475A7812, 0x88E16EBF, 0x00000003)
transfer (0x8A885D04, 0x11C91CEB, 0x0008E89F, 0x6048102B, 0x00000002)


NTLSA Transact Named Pipe

The sequence of actions taken on this pipe are:

Establish a connection to the IPC$ share (SMBtconX). use encrypted passwords.
Open an RPC Pipe with the name "\\PIPE\\lsarpc". Store the file handle.
Using the file handle, send a Set Named Pipe Handle state to 0x4300.
Send an LSA Open Policy request. Store the Policy Handle.
Using the Policy Handle, send LSA Query Info Policy requests, etc.
Using the Policy Handle, send an LSA Close.
Close the IPC$ share.

Defines for this pipe, identifying the query are:


NETLOGON rpc Transact Named Pipe

The sequence of actions taken on this pipe are:

tablish a connection to the IPC$ share (SMBtconX). use encrypted passwords.
en an RPC Pipe with the name "\\PIPE\\NETLOGON". Store the file handle.
ing the file handle, send a Set Named Pipe Handle state to 0x4300.
eate Client Challenge. Send LSA Request Challenge. Store Server Challenge.
lculate Session Key. Send an LSA Auth 2 Challenge. Store Auth2 Challenge.
lc/Verify Client Creds. Send LSA Srv PW Set. Calc/Verify Server Creds.
lc/Verify Client Creds. Send LSA SAM Logon . Calc/Verify Server Creds.
lc/Verify Client Creds. Send LSA SAM Logoff. Calc/Verify Server Creds.
ose the IPC$ share.

Defines for this pipe, identifying the query are


LSA SAM Logon

Note: valid_user is True iff the username and password hash are valid for the requested domain.


Response

if (valid_user)
{
	UINT16      3 - switch value indicating USER_INFO structure.
    VOID*     non-zero - pointer to USER_INFO structure
    USER_INFO user logon information

    UINT32    1 - Authoritative response; 0 - Non-Auth?

    return    0 - indicates success
}
else
{
	UINT16    0 - switch value.  value to indicate no user presumed.
    VOID*     0x0000 0000 - indicates no USER_INFO structure.

    UINT32    1 - Authoritative response; 0 - Non-Auth?

    return    0xC000 0064 - NT_STATUS_NO_SUCH_USER.
}


\\MAILSLOT\NET\NTLOGON

Note: mailslots will contain a response mailslot, to which the response should be sent. the target NetBIOS name is REQUEST_NAME<20>, where REQUEST_NAME is the name of the machine that sent the request.


Cryptographic side of NT Domain Authentication


SIDs and RIDs

SIDs and RIDs are well documented elsewhere.

A SID is an NT Security ID (see DOM_SID structure). They are of the form:

revision-NN-SubAuth1-SubAuth2-SubAuth3...
revision-0xNNNNNNNNNNNN-SubAuth1-SubAuth2-SubAuth3...

currently, the SID revision is 1. The Sub-Authorities are known as Relative IDs (RIDs).