summaryrefslogtreecommitdiff
path: root/lib/tdb/common/tdb_private.h
AgeCommit message (Collapse)AuthorFilesLines
2012-03-22lib/tdb: fix OpenBSD incoherent mmap.Rusty Russell1-1/+1
This comment appears in two places in the code (commit 4c6a8273c6dd3e2aeda5a63c4a62aa55bc133099 from 2001): /* * We must ensure the file is unmapped before doing this * to ensure consistency with systems like OpenBSD where * writes and mmaps are not consistent. */ But this doesn't help, because if one process is using mmap and another using pwrite, we get incoherent results. As demonstrated by OpenBSD's failure on the tdb unit tests. Rather than disable mmap on OpenBSD, we test for this issue and force mmap to be enabled. This means that we will fail on very large TDBs on 32-bit systems, but it's better than the horrendous performance penalty on every OpenBSD system. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2012-02-14tdb: make tdb_private.h idempotent.Rusty Russell1-0/+3
The most convenient way to write unit tests in C is to directly #include the C files (CCAN uses this, for example). That works quite well, but it means that tdb_private.h now needs to be protected against multiple inclusions. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2011-12-21tdb: use same expansion factor logic when expanding for new recovery area.Rusty Russell1-0/+1
If we're expanding because the current recovery area is too small, we expand only the amount we need. This can quickly lead to exponential growth when we have a slowly-expanding record (hence a slowly-expanding transaction size). Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2011-12-19tdb: be more careful on 4G files.Rusty Russell1-1/+1
I came across a tdb which had wrapped to 4G + 4K, and the contents had been destroyed by processes which thought it only 4k long. Fix this by checking on open, and making tdb_oob() check for wrap itself. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Autobuild-User: Rusty Russell <rusty@rustcorp.com.au> Autobuild-Date: Mon Dec 19 07:52:01 CET 2011 on sn-devel-104
2011-04-19tdb: make sure we skip over recovery area correctly.Rusty Russell1-0/+4
If it's really the recovery area, we can trust the rec_len field, and don't have to go groping for bitpatterns. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Autobuild-User: Rusty Russell <rusty@rustcorp.com.au> Autobuild-Date: Tue Apr 19 14:15:22 CEST 2011 on sn-devel-104
2010-12-29tdb: tdb_summary() support.Rusty Russell1-0/+1
Autobuild-User: Rusty Russell <rusty@rustcorp.com.au> Autobuild-Date: Wed Dec 29 10:12:05 CET 2010 on sn-devel-104
2010-09-27tdb: TDB_INCOMPATIBLE_HASH, to allow safe changing of default hash.Rusty Russell1-0/+1
This flag to tdb_open/tdb_open_ex effects creation of a new database: 1) Uses the Jenkins lookup3 hash instead of the old gdbm hash if none is specified, 2) Places a non-zero field in header->rwlocks, so older versions of TDB will refuse to open it. This means that the caller (ie Samba) can set this flag to safely change the hash function. Versions of TDB from this one on will either use the correct hash or refuse to open (if a different hash is specified). Older TDB versions will see the nonzero rwlocks field and refuse to open it under any conditions. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2010-09-27tdb: add Bob Jenkins lookup3 hash as helper hash.Rusty Russell1-0/+1
This is a better hash than the default: shipping it with tdb makes it easy for callers to use it as the hash by passing it to tdb_open_ex(). This version taken from CCAN and modified, which took it from http://www.burtleburtle.net/bob/c/lookup3.c. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2010-09-13tdb: put example hashes into header, so we notice incorrect hash_fn.Rusty Russell1-1/+5
This is Stefan Metzmacher <metze@samba.org>'s patch with minor changes: 1) Use the TDB_MAGIC constant so both hashes aren't of strings. 2) Check the hash in tdb_check (paranoia, really). 3) Additional check in the (unlikely!) case where both examples hash to 0. 4) Cosmetic changes to var names and complaint message. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2010-05-05tdb: fix short write logic in tdb_new_databaseRusty Russell1-1/+1
Commit 207a213c/24fed55d purported to fix the problem of signals during tdb_new_database (which could cause a spurious short write, hence a failure). However, the code is wrong: newdb+written is not correct. Fix this by introducing a general tdb_write_all() and using it here and in the tracing code. Cc: Stefan Metzmacher <metze@samba.org> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2010-04-20tdb: update tdb ABI to use hide_symbols=TrueAndrew Tridgell1-1/+1
We now use -fvisibilty=hidden to hide symbols from outside the tdb shared library. This also moved tdb_transaction_recover() into the tdb_private.h header, as it should never have been a public API. For that reason we are changing the version number. We're only doing a minor version increment as it is extremely unlikely that anyone was actually using tdb_transaction_recover() as its locking requirements were rather unusual. Pair-Programmed-With: Rusty Russell <rusty@samba.org>
2010-03-26tdb: Add a non-blocking version of tdb_transaction_startVolker Lendecke1-1/+2
2010-03-25Fix some nonempty blank linesVolker Lendecke1-2/+2
2010-02-24tdb: handle processes dying during transaction commit.Rusty Russell1-0/+1
tdb transactions were designed to be robust against the machine powering off, but interestingly were never designed to handle the case where an administrator kill -9's a process during commit. Because recovery is only done on tdb_open, processes with the tdb already mapped will simply use it despite it being corrupt and needing recovery. The solution to this is to check for recovery every time we grab a data lock: we could have gained the lock because a process just died. This has no measurable cost: here is the time for tdbtorture -s 0 -n 1 -l 10000: Before: 2.75 2.50 2.81 3.19 2.91 2.53 2.72 2.50 2.78 2.77 = Avg 2.75 After: 2.81 2.57 3.42 2.49 3.02 2.49 2.84 2.48 2.80 2.43 = Avg 2.74 Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2010-02-24tdb: remove lock opsRusty Russell1-2/+0
Now the transaction code uses the standard allrecord lock, that stops us from trying to grab any per-record locks anyway. We don't need to have special noop lock ops for transactions. This is a nice simplification: if you see brlock, you know it's really going to grab a lock. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2010-02-24tdb: rename tdb_release_extra_locks() to tdb_release_transaction_locks()Rusty Russell1-1/+1
tdb_release_extra_locks() is too general: it carefully skips over the transaction lock, even though the only caller then drops it. Change this, and rename it to show it's clearly transaction-specific. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2010-02-17tdb: tdb_allrecord_lock/tdb_allrecord_unlock/tdb_allrecord_upgradeRusty Russell1-2/+5
Centralize locking of all chains of the tdb; rename _tdb_lockall to tdb_allrecord_lock and _tdb_unlockall to tdb_allrecord_unlock, and tdb_brlock_upgrade to tdb_allrecord_upgrade. Then we use this in the transaction code. Unfortunately, if the transaction code records that it has grabbed the allrecord lock read-only, write locks will fail, so we treat this upgradable lock as a write lock, and mark it as upgradable using the otherwise-unused offset field. One subtlety: now the transaction code is using the allrecord_lock, the tdb_release_extra_locks() function drops it for us, so we no longer need to do it manually in _tdb_transaction_cancel. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2010-02-17tdb: remove num_locksRusty Russell1-1/+0
This was redundant before this patch series: it mirrored num_lockrecs exactly. It still does. Also, skip useless branch when locks == 1: unconditional assignment is cheaper anyway. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2010-02-17tdb: use tdb_nest_lock() for transaction lock.Rusty Russell1-1/+0
Rather than a boutique lock and a separate nest count, use our newly-generic nested lock tracking for the transaction lock. Note that the tdb_have_extra_locks() and tdb_release_extra_locks() functions have to skip over this lock now it is tracked. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2010-02-24tdb: cleanup: tdb_release_extra_locks() helperRusty Russell1-0/+1
Move locking intelligence back into lock.c, rather than open-coding the lock release in transaction.c. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2010-02-17tdb: cleanup: tdb_have_extra_locks() helperRusty Russell1-0/+1
In many places we check whether locks are held: add a helper to do this. The _tdb_lockall() case has already checked for the allrecord lock, so the extra work done by tdb_have_extra_locks() is merely redundant. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2010-02-17tdb: cleanup: tdb_nest_lock/tdb_nest_unlockRusty Russell1-1/+5
Because fcntl locks don't nest, we track them in the tdb->lockrecs array and only place/release them when the count goes to 1/0. We only do this for record locks, so we simply place the list number (or -1 for the free list) in the structure. To generalize this: 1) Put the offset rather than list number in struct tdb_lock_type. 2) Rename _tdb_lock() to tdb_nest_lock, make it non-static and move the allrecord check out to the callers (except the mark case which doesn't care). 3) Rename _tdb_unlock() to tdb_nest_unlock(), make it non-static and move the allrecord out to the callers (except mark again). Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2010-02-17tdb: cleanup: rename global_lock to allrecord_lock.Rusty Russell1-1/+1
The word global is overloaded in tdb. The global_lock inside struct tdb_context is used to indicate we hold a lock across all the chains. Rename it to allrecord_lock. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2010-02-17tdb: cleanup: rename GLOBAL_LOCK to OPEN_LOCK.Rusty Russell1-1/+1
The word global is overloaded in tdb. The GLOBAL_LOCK offset is used at open time to serialize initialization (and by the transaction code to block open). Rename it to OPEN_LOCK. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2010-02-24tdb: make _tdb_transaction_cancel static.Rusty Russell1-1/+0
Now tdb_open() calls tdb_transaction_cancel() instead of _tdb_transaction_cancel, we can make it static. Signed-off-by: Rusty Russell<rusty@rustcorp.com.au>
2010-02-17tdb: cleanup: split brlock and brunlock methods.Rusty Russell1-4/+18
This is taken from the CCAN code base: rather than using tdb_brlock for locking and unlocking, we split it into brlock and brunlock functions. For extra debugging information, brunlock says what kind of lock it is unlocking (even though fnctl locks don't need this). This requires an extra argument to tdb_transaction_unlock() so we know whether the lock was upgraded to a write lock or not. We also use a "flags" argument tdb_brlock: 1) TDB_LOCK_NOWAIT replaces lck_type = F_SETLK (vs F_SETLKW). 2) TDB_LOCK_MARK_ONLY replaces setting TDB_MARK_LOCK bit in ltype. 3) TDB_LOCK_PROBE replaces the "probe" argument. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2010-02-10tdb: give a name to the invalid recovery area constant (0)Rusty Russell1-0/+1
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2009-10-23tdb: rename 'struct list_struct' into 'struct tdb_record'Stefan Metzmacher1-9/+9
metze
2009-10-22lib/tdb: TDB_TRACE support (for developers)Rusty Russell1-0/+36
When TDB_TRACE is defined (in tdb_private.h), verbose tracing of tdb operations is enabled. This can be replayed using "replay_trace" from http://ccan.ozlabs.org/info/tdb. The majority of this patch comes from moving internal functions to _<funcname> to avoid double-tracing. There should be no additional overhead for the normal (!TDB_TRACE) case. Note that the verbose traces compress really well with rzip. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2009-07-20Make tdb transaction lock recursive (samba version)Rusty Russell1-1/+1
This patch replaces 6ed27edbcd3ba1893636a8072c8d7a621437daf7 and 1a416ff13ca7786f2e8d24c66addf00883e9cb12, which fixed the bug where traversals inside transactions would release the transaction lock early. This solution is more general, and solves the more minor symptom that nested traversals would also release the transaction lock early. (It was also suggestd in Volker's comment in 6ed27ed). This patch also applies to ctdb, if the traverse.c part is removed (ctdb's tdb code never received the previous two fixes). Tested using the testsuite from ccan (adapted to the samba code). Thanks to Michael Adam for feedback. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Signed-off-by: Michael Adam <obnox@samba.org>
2008-09-17Move common libraries from root to lib/.Jelmer Vernooij1-0/+213