Import design notes by martin and tpot from ComfyChair and stf modules.

(This used to be commit f923729ca37c18fd58943015a530607c05e96659)

1 files changed, 175 insertions, 0 deletions

diff --git a/source3/stf/notes.txt b/source3/stf/notes.txt
new file mode 100644
index 0000000000..68aca63c23
--- /dev/null
+++ b/source3/stf/notes.txt
@@ -0,0 +1,175 @@
+		-*- indented-text -*-
+
+(set lotus no)
+
+
+
+Notes on using comfychair with Samba (samba testing framework units):
+
+The tests need to rely on some external resources, such as 
+
+If suitable resources are not available, need to skip particular
+tests.  Must include a message indicating what resources would be
+needed to run that test.  (e.g. must be root.)
+
+We want to be able to select and run particular subsets of tests, such
+as "all winbind tests".
+
+We want to keep the number of configurable parameters down as much as
+possible, to make it easy on people running the tests.
+
+Wherever possible, the tests should set up their preconditions, but a
+few basic resources need to be provided by the people running the
+tests.  So for example, rather than asking the user for the name of a
+non-root user, we should give the tests the administrator name and
+password, and it can create a new user to use.
+
+This makes it simpler to get the tests running, and possible also
+makes them more reproducible.
+
+In the future, rather than using NT machines provided by the test
+person, we might have a way to drive VMWare non-persistent sessions,
+to make tests even more tightly controlled.
+
+
+Another design question is how to communicate this information to the
+tests.  If there's a lot of settings, then it might need to be stored
+in a configuration file.  
+
+However, if we succeed in cutting down the number of parameters, then
+it might be straightforward to pass the information on the command
+line or in an environment variable.  
+
+Environment variables are probably better because they can't be seen
+by other users, and they are more easily passed down through an
+invocation of "make check".
+
+
+
+Notes on Samba Testing Framework for Unittests
+----------------------------------------------
+
+This is to be read after reading the notes.txt from comfychair.  I'm
+proposing a slightly more concrete description of what's described
+there.
+
+The model of having tests require named resources looks useful for
+incorporation into a framework that can be run by many people in
+widely different environments.
+
+Some possible environments for running the test framework in are:
+
+    - Casual downloader of Samba compiling from source and just wants
+      to run 'make check'.  May only have one Unix machine and a
+      handful of clients.
+
+    - Samba team member with access to a small number of other
+      machines or VMware sessions.
+
+    - PSA developer who may not have intimate knowledge of Samba
+      internals and is only interested in testing against the PSA.
+
+    - Non-team hacker wanting to run test suite after making small
+      hacks. 
+
+    - Build farm environment (loaner machine with no physical access
+      or root privilege).
+
+    - HP BAT.
+
+Developers in most of these environments are also potential test case
+authors.  It should be easy for people unfamiliar with the framework
+to write new tests and have them work.  We should provide examples and
+the existing tests should well written and understandable.
+
+Different types of tests:
+
+    - Tests that check Samba internals and link against
+      libbigballofmud.so.  For example:
+
+        - Upper/lowercase string functions
+        - user_in_list() for large lists
+
+    - Tests that use the Samba Python extensions.
+
+    - Tests that execute Samba command line programs, for example
+      smbpasswd. 
+
+    - Tests that require other resources on the network such as domain
+      controllers or PSAs.
+
+    - Tests that are performed on the documentation or the source code
+      such as:
+
+        - grep for common spelling mistakes made by abartlet (-:
+        - grep for company copyright (IBM, HP)
+                         
+    - Link to other existing testing frameworks (smbtorture,
+      abartlet's bash based build farm tests)
+
+I propose a TestResourceManager which would be instantiated by a test
+case.  The test case would require("resourcename") as part of its
+constructor and raise a comfychair.NotRun exception if the resource
+was not present.  A TestResource class could be defined which could
+read a configuration file or examine a environment variable and
+register a resource only if some condition was satisfied.
+
+It would be nice to be able to completely separate the PSA testing
+from the test framework.  This would entail being able to define test
+resources dynamically, possibly with a plugin type system.
+
+class TestResourceManager:
+    def __init__(self, name):
+        self.resources = {}
+
+    def register(self, resource):
+        name = resource.name()
+        if self.resources.has_key(name):
+            raise "Test manager already has resource %s" % name
+        self.resources[name] = resource
+
+    def require(self, resource_name):
+        if not self.resources.has_key(resource_name):
+            raise "Test manager does not have resources %s" % resource_name
+
+class TestResource:
+    def __init__(self, name):
+        self.name = name
+
+    def name(self):
+        return self.name
+
+import os
+
+trm = TestResourceManager()
+
+if os.getuid() == 0:
+    trm.register(TestResource("root"))
+
+A config-o-matic Python module can take a list of machines and
+administrator%password entries and classify them by operating system
+version and service pack.  These resources would be registered with
+the TestResourceManager.
+
+Some random thoughts about named resources for network servers:
+
+require("nt4.sp3")
+require("nt4.domaincontroller")
+require("psa")
+
+Some kind of format for location of passwords, libraries:
+
+require("exec(smbpasswd)")
+require("lib(bigballofmud)")
+
+maybe require("exec.smbpasswd") looks nicer...
+
+The require() function could return a dictionary of configuration
+information or some handle to fetch dynamic information on.  We may
+need to create and destroy extra users or print queues.  How to manage
+cleanup of dynamic resources?
+
+Requirements for running stf:
+
+    - Python, obviously
+    - Samba python extensions