Add all the source files from the old CVS tree,

add the 5 missing chapters from the HOWTO
and add jht's Samba by Example book.
(This used to be commit 9fb5bcb93e57c5162b3ee6f9c7d777dc0269d100)

1 files changed, 357 insertions, 0 deletions

diff --git a/docs/devel/sam.xml b/docs/devel/sam.xml
new file mode 100644
index 0000000000..84c17d65e2
--- /dev/null
+++ b/docs/devel/sam.xml
@@ -0,0 +1,357 @@
+<chapter id="sam">
+
+<chapterinfo>
+	<author>
+		<firstname>Andrew</firstname><surname>Bartlett</surname>
+	</author>
+	<pubdate>1 October 2002</pubdate>
+</chapterinfo>
+
+<title>The Upcoming SAM System</title>
+
+<sect1>
+<title>Security in the 'new SAM'</title>
+
+<para>One of the biggest problems with passdb is it's implementation of
+'security'.  Access control is on a 'are you root at the moment' basis,
+and it has no concept of NT ACLs.  Things like ldapsam had to add
+'magic' 'are you root' checks.</para>
+
+<para>We took this very seriously when we started work, and the new structure
+is designed with this in mind, from the ground up.  Each call to the SAM
+has a NT_TOKEN and (if relevant) an 'access desired'.  This is either
+provided as a parameter, or implicitly supplied by the object being
+accessed.</para>
+
+<para>
+For example, when you call 
+</para>
+
+<programlisting>
+NTSTATUS sam_get_account_by_name(const SAM_CONTEXT *context, const
+NT_USER_TOKEN *access_token, uint32 access_desired, const char *domain,
+const char *name, SAM_ACCOUNT_HANDLE **account)
+</programlisting>
+
+<para>
+The context can be NULL (and is used to allow import/export by setting
+up 2 contexts, and allowing calls on both simultaneously)
+</para>
+
+<para>
+The access token *must* be specified.  Normally the user's token out of
+current_user, this can also be a global 'system' context.
+</para>
+
+<para>
+The access desired is as per the ACL, for passing to the seaccess stuff.
+</para>
+
+<para>
+The domain/username are standard.  Even if we only have one domain,
+keeping this ensures that we don't get 'unqualified' usernames (same
+problem as we had with unqualified SIDs).
+</para>
+
+<para>
+We return a 'handle'.  This is opaque to the rest of Samba, but is
+operated on by get/set routines, all of which return NTSTATUS.
+</para>
+
+<para>
+The access checking is done by the SAM module.   The reason it is not
+done 'above' the interface is to ensure a 'choke point'.  I put a lot of
+effort into the auth subsystem to ensure we never 'accidentally' forgot
+to check for null passwords, missed a restriction etc.  I intend the SAM
+to be written with the same caution.
+</para>
+
+<para>
+The reason the access checking is not handled by the interface itself is
+due to the different implementations it make take on.  For example, on
+ADS, you cannot set a password over a non-SSL connection.  Other
+backends may have similar requirements - we need to leave this policy up
+to the modules.  They will naturally have access to 'helper' procedures
+and good examples to avoid mishaps.
+</para>
+
+<para>
+(Furthermore, some backends my actually chose to push the whole ACL
+issue to the remote server, and - assuming ldap for this example - bind
+as the user directly)
+</para>
+
+<para>
+Each returned handle has an internal 'access permitted', which allows
+the 'get' and 'set' routines to return 'ACCESS_DENIED' for things that
+were not able to be retrieved from the backend.  This removes the need
+to specify the NT_TOKEN on every operation, and allows for 'object not
+present' to be easily distinguished from 'access denied'.
+</para>
+
+<para>
+When you 'set' an object (calling sam_update_account) the internal
+details are again used.  Each change that has been made to the object
+has been flagged, so as to avoid race conditions (on unmodified
+components) and to avoid violating any extra ACL requirements on the
+actual data store (like the LDAP server).
+</para>
+
+<para>
+Finally, we have generic get_sec_desc() and set_sec_desc() routines to
+allow external ACL manipulation.  These do lookups based on SID.
+</para>
+
+</sect1>
+
+<sect1>
+<title>Standalone from UNIX</title>
+
+<para>
+One of the primary tenants of the 'new SAM' is that it would not attempt
+to deal with 'what unix id for that'.  This would be left to the 'SMS'
+(Sid Mapping System') or SID farm, and probably administered via
+winbind.  We have had constructive discussion on how 'basic' unix
+accounts like 'root' would be handled, and we think this can work.  
+Accounts not preexisting in unix would be served up via winbind.
+</para>
+
+<para>
+This is an *optional* part, and my preferred end-game.  We have a fare
+way to go before things like winbind up to it however.
+</para>
+
+</sect1>
+
+<sect1>
+<title>Handles and Races in the new SAM</title>
+
+<para>
+One of the things that the 'new SAM' work has tried to face is both
+compatibility with existing code, and a closer alignment to the SAMR
+interface.  I consider SAMR to be a 'primary customer' to the this work,
+because if we get alignment with that wrong, things get more, rather
+than less complex.  Also, most other parts of Samba are much more
+flexible with what they can allow.
+</para>
+
+<para>
+In any case, that was a decision taken as to how the general design
+would progress.  BTW, my understanding of SAMR may be completely flawed.
+</para>
+
+<para>
+One of the most race-prone areas of the new code is the conflicting
+update problem.  We have taken two approaches:  
+</para>
+
+<itemizedlist>
+<listitem>
+<para>'Not conflicting' conflicts.  Due to the way usrmgr operates, it will
+open a user, display all the properties and *save* them all, even if you
+don't change any.
+</para>
+
+<para>
+For this, see what I've done in rpc_server/srv_samr_util.c.  I intend
+to take this one step further, and operate on the 'handle' that the
+values were read from.  This should mean that we only update things that
+have *really* changed.
+</para>
+</listitem>
+
+<listitem>
+<para>
+'conflicting' updates:  Currently we don't deal with this (in passdb
+or the new sam stuff), but the design is sufficiently flexible to 'deny'
+a second update.  I don't foresee locking records however.
+</para>
+</listitem>
+</itemizedlist>
+
+</sect1>
+
+<sect1>
+<title>Layers</title>
+
+<sect2>
+<title>Application</title>
+
+<para>
+This is where smbd, samtest and whatever end-user replacement we have
+for pdbedit sits.  They use only the SAM interface, and do not get
+'special knowledge' of what is below them.
+</para>
+</sect2>
+<sect2>
+<title>SAM Interface</title>
+
+<para>
+This level 'owns' the various handle structures, the get/set routines on
+those structures and provides the public interface.  The application
+layer may initialize a 'context' to be passed to all interface routines,
+else a default, self-initialising context will be supplied.  This layser
+finds the appropriate backend module for the task, and tries very hard
+not to need to much 'knowledge'.  It should just provide the required
+abstraction to the modules below, and arrange for their initial loading.
+</para>
+
+<para>
+We could possibly add ACL checking at this layer, to avoid discrepancies
+in implementation modules.
+</para>
+
+</sect2>
+
+<sect2>
+<title>SAM Modules</title>
+
+<para>
+These do not communicate with the application directly, only by setting
+values in the handles, and receiving requests from the interface.  These
+modules are responsible for translating values from the handle's
+.private into (say) an LDAP modification list.  The module is expected
+to 'know' things like it's own domain SID, domain name, and any other
+state attached to the SAM.  Simpler modules may call back to some helper
+routine.
+</para>
+
+</sect2>
+</sect1>
+
+<sect1>
+<title>SAM Modules</title>
+
+<sect2>
+<title>Special Module: sam_passdb</title>
+
+<para>
+In order for there to be a smooth transition, kai is writing a module
+that reads existing passdb backends, and translates them into SAM
+replies.  (Also pulling data from the account policy DB etc).  We also
+intend to write a module that does the reverse - gives the SAM a passdb
+interface.
+</para>
+</sect2>
+
+<sect2>
+<title>sam_ads</title>
+<para>
+This is the first of the SAM modules to be committed to the tree -
+mainly because I needed to coordinate work with metze (who authored most
+of it).  This module aims to use Samba's libads code to provide an
+Active Directory LDAP client, suitable for use on a mixed-mode DC. 
+While it is currently being tested against Win2k servers (with a
+password in the smb.conf file) it is expected to eventually use a
+(possibly modified) OpenLDAP server.  We hope that this will assist in
+the construction of an Samba AD DC.
+</para>
+
+<para>
+We also intend to construct a Samba 2.2/3.0 compatible ldap module,
+again using libads code.
+</para>
+</sect2>
+</sect1>
+
+<sect1>
+<title>Memory Management</title>
+
+<para> 
+The 'new SAM' development effort also concerned itself with getting a
+sane implementation of memory management.  It was decided that we would
+be (as much as possible) talloc based, using an 'internal talloc
+context' on many objects.  That is, the creation of an object would
+initiate it's own internal talloc context, and this would be used for
+all operations on that object.  Much of this is already implemented in
+passdb.  Also, like passdb, it will be possible to specify that some
+object actually be created on a specified context.  
+</para>
+
+<para>
+Memory management is important here because the APIs in the 'new SAM' do
+not use 'pdb_init()' or an equivalent.  They always allocate new
+objects.  Enumeration's are slightly different, and occur on a supplied
+context that 'owns' the entire list, rather than per-element.  (the
+enumeration functions return an array of all elements - not full handles
+just basic (and public) info)  Likewise for things that fill in a char
+**.
+</para>
+
+<para>For example:</para>
+
+<para><programlisting>
+NTSTATUS sam_lookup_sid(const SAM_CONTEXT *context, const NT_USER_TOKEN
+*access_token, TALLOC_CTX *mem_ctx, const DOM_SID *sid, char **name,
+uint32 *type)
+</programlisting></para>
+
+<para>Takes a context to allocate the 'name' on, while:</para>
+
+<para><programlisting>
+NTSTATUS sam_get_account_by_sid(const SAM_CONTEXT *context, const
+NT_USER_TOKEN *access_token, uint32 access_desired, const DOM_SID
+*accountsid, SAM_ACCOUNT_HANDLE **account)
+</programlisting></para>
+
+<para>Allocates a handle and stores the allocation context on that handle.</para>
+
+<para>I think that the following:</para>
+
+<para><programlisting>
+NTSTATUS sam_enum_accounts(const SAM_CONTEXT *context, const
+NT_USER_TOKEN *access_token, const DOM_SID *domainsid, uint16 acct_ctrl,
+int32 *account_count, SAM_ACCOUNT_ENUM **accounts)
+</programlisting></para>
+
+</sect1>
+
+<sect1>
+<title>Testing</title>
+
+<para>
+Testing is vital in any piece of software, and Samba is certainly no
+exception. In designing this new subsystem, we have taken care to ensure
+it is easily tested, independent of outside protocols.
+</para>
+
+<para>
+To this end, Jelmer has constructed 'samtest'.  
+</para>
+
+<para>
+This utility (see torture/samtest.c) is structured like rpcclient, but
+instead operates on the SAM subsystem.  It creates a 'custom' SAM
+context, that may be distinct from the default values used by the rest
+of the system, and can load a separate configuration file.  
+</para>
+
+<para>
+A small number of commands are currently implemented, but these have
+already proved vital in testing.   I expect SAM module authors will find
+it particularly valuable.
+</para>
+
+<para>Example useage:</para>
+
+<para><prompt>$</prompt> <command>bin/samtest</command></para>
+
+<para><programlisting>
+> context ads:ldap://192.168.1.96
+</programlisting>
+(this loads a new context, using the new ADS module.  The parameter is
+the 'location' of the ldap server)
+</para>
+
+<para><programlisting>
+> lookup_name DOMAIN abartlet
+</programlisting>
+(returns a sid).
+</para>
+
+<para>
+Because the 'new SAM' is NT ACL based, there will be a command to
+specify an arbitrary NT ACL, but for now it uses 'system' by default.
+</para>
+</sect1>
+</chapter>