| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
|
|
When using w2k3 linked attributes we are allowed to have multiple
values on a single valued attribute. This happens when the other
values are deleted.
Setting the RELAX control tell the ldb-tdb backend to not check for
this on replace, which means the caller has to check for single valued
violations.
|
|
We need to keep a search count in ltdb to allow for nesting
of searches inside a module
Signed-off-by: Andrew Tridgell <tridge@samba.org>
|
|
- The outside API contains "DN" string arguments: Bad. Since in this way we
fully rely on the outside calls regarding the right DN format. Solution: Use
always a "struct ldb_dn" entry. Since this one is interchangeable and we can
handle it in our preferred way.
|
|
These will be used by ldb_index.c
|
|
This gets rid of the @IDXPTR approach to in-transaction indexing,
instead using an in-memory tdb to hold index values during a
transaction. This also cleans up a lot of the internal indexing logic,
hopefully making it easier to understand.
One of the big changes is in memory management, with a lot more use
made of talloc tricks to avoid copying dn lists, and shortcuts used to
avoid high intersection and union calculation costs.
The overall result is that a re-provision on my laptop goes from 48s
to a bit over 10s.
|
|
When doing an indexed search if we hit a corrupt record we abandoned
the indexed search and did a full search. The problem was that we
might have sent some records to the caller already, which means the
caller ended up with duplicate records. Fix this by returning a search
error if indexing returns an error and we have given any records to
the caller.
|
|
|
|
This op will be used by the partition module to give us good
transaction semantics across the 4 partitions that sam.ldb uses.
|
|
In some code paths ltdb_context was still referenced even after we were returned
an error by one of the callbacks. Because the interface assumes that once an
error is returned the ldb_request may be freed, and because the ltdb_context was
allocated as a child of the request, this might cause access to freed memory.
Allocate the ltdb_context on ldb, and keep track of what's going on with the
request by adding a spy children on it. This way even if the request is freed
before the ltdb_callback is called, we will safely free the ctx and just quietly
return.
|
|
do not reference it from ldb.h
|
|
Separate again the public from the private headers.
Add a new header specific for modules.
Also add service function for modules as now ldb_context and ldb_module are
opaque structures for them.
|
|
list=""
list="$list event_context:tevent_context"
list="$list fd_event:tevent_fd"
list="$list timed_event:tevent_timer"
for s in $list; do
o=`echo $s | cut -d ':' -f1`
n=`echo $s | cut -d ':' -f2`
r=`git grep "struct $o" |cut -d ':' -f1 |sort -u`
files=`echo "$r" | grep -v source3 | grep -v nsswitch | grep -v packaging4`
for f in $files; do
cat $f | sed -e "s/struct $o/struct $n/g" > $f.tmp
mv $f.tmp $f
done
done
metze
|
|
ldb indexing can cause huge files, and huge memory usage. This
experiment allows us to keep indexes in memory during a transaction,
then to write the indexes to disk when the transaction completes. The
result is that the db is much smaller (we have seen improvements of
about 100x in file size) and memory usage during large transactions is
also greatly reduced
Note that this patch uses the unusual strategy of putting pointers
into a ldb (and thus into a tdb). This works because the pointers are
only there during a transaction, so the pointers are not exposed to
any other users of the database. The pointers allow us to avoid some
really bad allocation problems with tdb record allocation during the
re-indexing.
|
|
(Even if the callback takes some time, this isn't a ldb_tdb timeout
any more)
Andrew Bartlett
|
|
it should always have been. Make it also async so that it is not a special case.
|
|
|
|
Use the checkBaseOnSearch attribute to control if we should check the
base DN on search requests.
Also ensure we honour any errors in searching, not just errors in the
supplied 'done' callback.
Andrew Bartlett
(This used to be commit deaac92f439ef001bfe052df170d6e34e8ba5845)
|
|
Jeremy.
(This used to be commit 52b26645b04a9c5fb70e7b869b60c9157f821d50)
|
|
transaction. When we are in a transaction then we could be in a top level modify operation (such as rename), so we must use a writeable traverse so that the async callbacks can do the modifies while the search is progressing.
- don't do the lockall operation on the tdb during a ldb search if in
a transaction, as this would prevent modifies by callbacks as well
(This used to be commit aa9ab431e071882f42ebc882e809ae1d4b8778d4)
|
|
Subclass support was designed to avoid needing to spell out the full
list of objectClasses that an entry was in. However, Samba4 now
enforces this restriction in the objectClass module, and the way
subclass matching was handled was complex and counter-intuitive in my
opinion (and did not match LDAP).
Andrew Bartlett
(This used to be commit f5ce04b904e14445a2a7e7f92e7e1f64b645c6f2)
|
|
(This used to be commit 917bd737cb07817664d9088860588d47525f5ff8)
|
|
To activate it you must modify the @INDEXLIST object adding
the attribute @IDXONE: 1
Ldb test included
Simo.
(This used to be commit ea111795f4016916473ccc05d23c6655e6af1207)
|
|
This patch changes a lot of the code in ldb_dn.c, and also
removes and add a number of manipulation functions around.
The aim is to avoid validating a dn if not necessary as the
validation code is necessarily slow. This is mainly to speed up
internal operations where input is not user generated and so we
can assume the DNs need no validation. The code is designed to
keep the data as a string if possible.
The code is not yet 100% perfect, but pass all the tests so far.
A memleak is certainly present, I'll work on that next.
Simo.
(This used to be commit a580c871d3784602a9cce32d33419e63c8236e63)
|
|
by avoiding chain locks on each tdb_fetch() within the search
- use the tdb_get_seqnum() call to avoid re-reading the @BASEINFO
record when it hasn't changed.
These speed up the LOCAL-DBSPEED test for ldb from 7k ops/sec to a bit
over 11k ops/sec
(This used to be commit 1347ad254eb8cd12ce22a5a2a37bec0a0ac8dbf1)
|
|
(This used to be commit 8c091bcdece5c17073838ad2367f3f4e22e97c31)
|
|
when the code has been changed to be async.
With the other committed fixes now this works.
(This used to be commit 49fc640b5c0398516ac3a9e3f7c55205cd60b1de)
|
|
when the tdb api is updated
(This used to be commit 6ace943fac101839e35cbc83dc54fde2068f704b)
|
|
This moves these attributes from objectguid into an optional backend
(objectguid), used by ltdb. For OpenLDAP, the entryUUID module
converts entryCSN into usnChanged.
This also changes the sequence number API, and uses 'time based'
sequence numbers, when an LDAP or similar backend is detected.
To assist this, we also store the last modified time in the TDB,
whenever we change a value.
Andrew Bartlett
(This used to be commit 72858f859483c0c532dddb2c146d6bd7b9be5072)
|
|
metze
(This used to be commit f3dc51fef53287cc2e2af7ed4a9f3f52a5cd06ed)
|
|
Finally acknowledge that ldb is inherently async and does not have a dual personality anymore
Rename all ldb_async_XXX functions to ldb_XXX except for ldb_async_result, it is now ldb_reply
to reflect the real function of this structure.
Simo.
(This used to be commit 25fc7354049d62efeba17681ef1cdd326bc3f2ef)
|
|
as part of ldb.
This allows tdb failures to be passed all the way up to Samba's DEBUG
system, which allowed easier debugging.
Unfortunately I had to extend the tdb API, as the logging function
didn't have a context pointer.
I've worked over the 'debug levels' in TDB. Most of them were 0,
which didn't seem right, as some were trace-like messages. We didn't
see any of these previously, except when accessing TDB directly.
Andrew Bartlett
(This used to be commit 58898092c1ce043f6d698db5065f372b79109e22)
|
|
This means that some modules have been disabled as well as they
have not been ported to the async interface
One of them is the ugly objectclass module.
I hope that the change in samldb module will make the MMC happy
without the need of this crappy module, we need proper handling
in a decent schema module.
proxy and ldb_map have also been disabled
ldb_sqlite3 need to be ported as well (currenlty just broken).
(This used to be commit 51083de795bdcbf649de926e86969adc20239b6d)
|
|
I was sick of jumping inot each module for each request,
even the ones not handle by that module.
(This used to be commit 7d65105e885a28584e8555453b90232c43a92bf7)
|
|
(This used to be commit 7ef63abae12f65835a82f9931ad1f5ea75e5f3f6)
|
|
Simo.
(This used to be commit 0db616ef59ed51cac7e0bfaea8a799d5aa42ef16)
|
|
In the return value res->msgs, msgs was not a child of res, in the
indexed path. Instead, it hung directly off the ldb, which was
sometimes a long-term context.
Also remove unused parameters.
Found by --leak-report-full
Andrew Bartlett
(This used to be commit 29cb5af827c459758997c80dca034d471bb92449)
|
|
This patch changes the way lsb_search is called and the meaning of the returned integer.
The last argument of ldb_search is changed from struct ldb_message to struct ldb_result
which contains a pointer to a struct ldb_message list and a count of the number of messages.
The return is not the count of messages anymore but instead it is an ldb error value.
I tryed to keep the patch as tiny as possible bu as you can guess I had to change a good
amount of places. I also tried to double check all my changes being sure that the calling
functions would still behave as before. But this patch is big enough that I fear some bug
may have been introduced anyway even if it passes the test suite. So if you are currently
working on any file being touched please give it a deep look and blame me for any error.
Simo.
(This used to be commit 22c8c97e6fb466b41859e090e959d7f1134be780)
|
|
a search() function, instead each module now only implements the
bytree method, and the expression based search is handled generically
by the modules code. This makes for more consistency and less code
duplication.
fixed the tdb backend to handle BASE searches much more
efficiently. They now always only lookup one record, regardless of the
search expression
(This used to be commit 7e44f9153c5578624e2fca04cdc0a00af0fd9eb4)
|
|
locking code in the ldb_tdb backend, except for a single read lock
during searches to ensure searches don't cross transaction boundaries
The tdb transactions code would map these extra locks to noops anyway
(as locking makes no sense inside a transaction), but the work in
setting up the locking keys still costs something, and it makes the
code needlessly complex
(This used to be commit 1b8d368a6771360fb0626127c02b3eb95f3eae59)
|
|
changed the prioivate modules API
error string are now not spread over all
modules but are kept in a single place.
This allows a better control of memory
and error reporting.
(This used to be commit 3fc676ac1d6f59d08bedbbd9377986154cf84ce4)
|
|
distinguished names
Provide more functions to handle DNs in this form
(This used to be commit 692e35b7797e39533dd2a1c4b63d9da30f1eb5ba)
|
|
avoid the horrors of posix locking, but it was preventing us having an
ldb open twice with different options. Now each ldb open of the same
file shares the same underlying tdb, but uses a different ldb
structure
(This used to be commit 4e090c66dfa1d2764e4693578d3845be3b8893f6)
|
|
- moved the knowledge of attribute types out of ldb_tdb and into the
generic ldb code. This allows the ldb_match() message match logic
to be generic, so it can be used by other backend
- added the generic ability to load attribute handlers, for
canonicalisation, compare, ldif read and ldif write. In the future
this will be used by the schema module to allow us to correctly
obey the attributetype schema elements
- added attribute handlers for some of the core ldap attribute types,
Integer, DirectoryString, DN, ObjectClass etc
- added automatic registration of attribute handlers for well-known
attribute names 'cn', 'dc', 'dn', 'ou' and 'objectClass'
- converted the objectSid special handlers for Samba to the new system
- added more correct handling of indexing in tdb backend based on the
attribute canonicalisation function
- added generic support for subclasses, moving it out of the tdb
backend. This will be used in future by the schema module
- fixed several bugs in the dn_explode code. It still needs more
work, but doesn't corrupt ldb dbs any more.
(This used to be commit 944c5844ab441b96d8e5d7b2d151982139d1fab9)
|
|
cmdline credentials code (which will be done soon)
- added a ldb_init() call, and changed ldb_connect() to take a ldb
context. This allows for much better error handling in
ldb_connect(), and also made the popt conversion easier
- fixed up all the existing backends with the new syntax
- improved error handling in *_connect()
- fixed a crash bug in the new case_fold_required() code
- ensured that ltdb_rename() and all ltdb_search() paths get the read lock
- added a ldb_oom() macro to make it easier to report out of memory
situations in ldb code
(This used to be commit f648fdf187669d6d87d01dd4e786b03cd420f220)
|
|
Old way was ugly and had a bug, you couldn't add an attribute named
dn or distinguishedName and search for it, tdb would change that search in a dn search.
This makes it also possible to search by dn against an ldap server as the old method was
not supported by ldap syntaxes.
sss
(This used to be commit a614466dec2484a0d39bdfae53da822cfcf80926)
|
|
instead of a search expression. This allows our ldap server to pass
its ASN.1 parsed search expressions straight to ldb, instead of going
via strings.
- updated all the ldb modules code to handle the new interface
- got rid of the separate ldb_parse.h now that the ldb_parse
structures are exposed externally
- moved to C99 structure initialisation in ldb
- switched ldap server to using ldb_search_bytree()
(This used to be commit 96620ab2ee5d440bbbc51c1bc0cad9977770f897)
|
|
this object properties are now used as multivalue attributes
now all values inserted are checked against a "valid values table"
eg:
this form is now accepted:
dn: @ATTRIBUTES
uid: CASE_INSENSITIVE
uid: WILDCARD
this form is now rejected:
dn: @ATTRIBUTES
uid: CASE_INSENSITIVE WILDCARD
please update your .ldb files if you make use of @ATTRIBUTES
(sam.ldb heavily uses it)
the code passes all make test tests for both tdb and ldap, it also
passes the new test to check for wrong @ATTRIBUTES attribute values
Simo.
(This used to be commit 1295b891a26c2cb2c34540f90ded83390cf87da2)
|
|
that ldb_search() sees a single consistent view of the database (by
blocking writes during a ldb_search)
(This used to be commit 917f2a8a073fd501f0626bea4f9deb91b95fdc90)
|
|
Just use talloc_free() to release the memory after an ldb_search().
(This used to be commit 4f0948dab0aa5e8b6a4ce486f3668ca8dfae23db)
|
