summaryrefslogtreecommitdiff
path: root/lib/tdb
AgeCommit message (Collapse)AuthorFilesLines
2012-06-22tdb: don't use err.h in tests.Rusty Russell21-22/+1
It's not portable. While we could use ccan/err, it seems overkill since we actually only use it in one test (I obviously cut & paste the #include). Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Autobuild-User(master): Rusty Russell <rusty@rustcorp.com.au> Autobuild-Date(master): Fri Jun 22 09:22:28 CEST 2012 on sn-devel-104
2012-06-22tdb: make TDB_NOSYNC merely disable sync.Rusty Russell3-15/+20
(As suggested by Stefan Metzmacher, based on the change to ntdb.) Since commit ec96ea690edbe3398d690b4a953d487ca1773f1c, we handle the case where a process dies during a transaction commit. Unfortunately, TDB_NOSYNC means this no longer works, as it disables the recovery area as well as the actual msync/fsync. We should do everything except the syncs. This also means we can do a complete test with $TDB_NO_FSYNC set; just to get more complete coverage, we disable it explicitly for one test (where we override the actual sync calls anyway). Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2012-06-22autobuild: always set TDB_NO_FSYNC.Rusty Russell1-0/+4
Then we unset it inside the tdb test target itself. This means that new code can't accidently forget it, and we can set it in the 'buildnice' script on sn-devel, for example. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2012-06-19tdb/wscript: Remove unecessary semicolons.Jelmer Vernooij1-4/+4
2012-06-14tdb: remove unused debug_fprintf() macro that breaks the buildStefan Metzmacher1-2/+0
The IRIX compiler doesn't support '...' in a macro. metze
2012-06-12tdb:tests: fix use of a non-existent word (existant)Michael Adam1-2/+2
2012-05-30build: Remove unused release scripts for tdbAndrew Bartlett6-497/+0
These now use waf dist, and the script/librelease.sh script as a wrapper. The mksyms.sh call in the source3/Makefile uses the copy in source3/script Andrew Bartlett
2012-03-29lib/tdb: Update ABIAmitay Isaacs2-1/+67
Signed-off-by: Amitay Isaacs <amitay@gmail.com> Autobuild-User: Rusty Russell <rusty@rustcorp.com.au> Autobuild-Date: Thu Mar 29 13:12:46 CEST 2012 on sn-devel-104
2012-03-29lib/tdb: Add/expose lock functions to support CTDBAmitay Isaacs1-2/+16
This patch adds two lock functions used by CTDB to perform asynchronous locking. These functions do not actually perform any fcntl operations, but only increment internal counters. - tdb_transaction_write_lock_mark() - tdb_transaction_write_lock_unmark() It also exposes two internal functions - tdb_lock_nonblock() - tdb_unlock() These functions are NOT exposed in include/tdb.h to prevent any further uses of these functions. If you ever need to use these functions, consider using tdb2. Signed-off-by: Amitay Isaacs <amitay@gmail.com>
2012-03-23lib/tdb: fix transaction issue for HAVE_INCOHERENT_MMAP.Rusty Russell1-11/+10
We unmap the tdb on expand, the remap. But when we have INCOHERENT_MMAP (ie. OpenBSD) and we're inside a transaction, doing the expand can mean we need to read from the database to partially fill a transaction block. This fails, because if mmap is incoherent we never allow accessing the database via read/write. The solution is not to unmap and remap until we've actually written the padding at the end of the file. Reported-by: Amitay Isaacs <amitay@gmail.com> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Autobuild-User: Rusty Russell <rusty@rustcorp.com.au> Autobuild-Date: Fri Mar 23 02:53:15 CET 2012 on sn-devel-104
2012-03-23lib/tdb: fix test/run-die-during-transaction when HAVE_INCOHERENT_MMAP.Rusty Russell1-0/+5
Since we force mmap on, we don't intercept writes to the db, so we never see it in an inconsistent state. #ifdef over the check that we should have recovered it at least once. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2012-03-23lib/tdb: fix missing return 0 code.Rusty Russell1-1/+1
fde694274e1e5a11d1473695e7ec7a97f95d39e4 made tdb_mmap return an int, but didn't put the return 0 on the "internal db" case. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2012-03-22lib/tdb: fix OpenBSD incoherent mmap.Rusty Russell4-24/+58
This comment appears in two places in the code (commit 4c6a8273c6dd3e2aeda5a63c4a62aa55bc133099 from 2001): /* * We must ensure the file is unmapped before doing this * to ensure consistency with systems like OpenBSD where * writes and mmaps are not consistent. */ But this doesn't help, because if one process is using mmap and another using pwrite, we get incoherent results. As demonstrated by OpenBSD's failure on the tdb unit tests. Rather than disable mmap on OpenBSD, we test for this issue and force mmap to be enabled. This means that we will fail on very large TDBs on 32-bit systems, but it's better than the horrendous performance penalty on every OpenBSD system. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2012-03-22lib/tdb: fix up run-die-during-transaction test cases on Solaris.Rusty Russell4-5/+4
By using a different include order, we end up with a different version of FILE_OFFSET_BITS (and probably other things) in parts of the test. The different variants get linked together, and the result is weird: the stat returns 0 size. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2012-03-14lib/tdb: remove unnecessary XOPEN and FILE_OFFSET_BITS defines in test/Rusty Russell17-20/+0
These were relics: they don't need to be defined here as long as we are careful to include the replace headers before any standard headers (we are). Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Autobuild-User: Rusty Russell <rusty@rustcorp.com.au> Autobuild-Date: Wed Mar 14 10:12:26 CET 2012 on sn-devel-104
2012-03-10lib/tdb: fix tests for standalone out-of-tree.Rusty Russell1-1/+2
Commit 4d58d0fa8f936e7efdc02e31c053d42a47b3e62a didn't work for lib/tdb outside the build tree: symlink was pointing to wrong place. Copy simplification from lib/tdb2, and fix the build farm. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Autobuild-User: Rusty Russell <rusty@rustcorp.com.au> Autobuild-Date: Sat Mar 10 07:07:45 CET 2012 on sn-devel-104
2012-02-14tdb: build and run unit tests in tdb/test/Rusty Russell1-7/+87
Now we can build the test binaries: the CCAN style is to compile everything called "compile_ok*.c", compile and run everything called "run*.c", compile, link with the module, and run everything called "api*.c", and link any other C files (presumably test helpers) into all the tests. Unfortunately, actually passing that between the various parts of wscript is painful, so I open-coded the names. Also, the tests expect to be run in a (temporary) directory they can pollute, with the test directory found in test/ (to find the canned TDB files, for example). Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Autobuild-User: Rusty Russell <rusty@rustcorp.com.au> Autobuild-Date: Tue Feb 14 06:53:46 CET 2012 on sn-devel-104
2012-02-14tdb/test: fix up tests for use in SAMBA tdb code.Rusty Russell5-13/+16
1) Make sure we include "tdb_private.h" first, to get the right headers (esp. the correct setting of _FILE_OFFSET_BITS before unistd.h). 2) Fix 3G file test since expand logic has changed. 3) Fix nested transaction test, since default is to allow nesting. 4) Capture fdatasync, which was slowing down transaction expand. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2012-02-14tdb: wean CCAN-style unit tests off of tap.Rusty Russell25-23/+64
We could use subunit, but that's overkill. Just print messages when we fail, and use exit status. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2012-02-14tdb: import unit tests from CCAN into tdb/test/Rusty Russell35-1/+2567
I pulled tdb into CCAN as an experiment a while ago; it doesn't belong there, but it has accumulated some important unit tests. These are copied from CCAN version init-1486-gc438ec1 with #include "../" changed to #include "../common/". Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2012-02-14tdb: make tdb_private.h idempotent.Rusty Russell1-0/+3
The most convenient way to write unit tests in C is to directly #include the C files (CCAN uses this, for example). That works quite well, but it means that tdb_private.h now needs to be protected against multiple inclusions. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2012-01-07Add "repack" command to tdbtool.Ira Cooper1-0/+7
Autobuild-User: Jeremy Allison <jra@samba.org> Autobuild-Date: Sat Jan 7 02:18:41 CET 2012 on sn-devel-104
2012-01-06Fix compile when TDB_TRACE is enabled.Ira Cooper1-1/+1
Autobuild-User: Jeremy Allison <jra@samba.org> Autobuild-Date: Fri Jan 6 04:16:41 CET 2012 on sn-devel-104
2011-12-25tdb: Use tdb_parse_record in tdb_update_hashVolker Lendecke1-12/+16
This avoids a tdb_fetch, thus a malloc/memcpy/free in the tdb_store path
2011-12-21tdb: don't free old recovery area when expanding if already at EOF.Rusty Russell1-17/+30
We allocate a new recovery area by expanding the file. But if the recovery area is already at the end of file (as shown in at least one client case), we can simply expand the record, rather than freeing it and creating a new one. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Autobuild-User: Rusty Russell <rusty@rustcorp.com.au> Autobuild-Date: Wed Dec 21 06:25:40 CET 2011 on sn-devel-104
2011-12-21tdb: use same expansion factor logic when expanding for new recovery area.Rusty Russell3-21/+34
If we're expanding because the current recovery area is too small, we expand only the amount we need. This can quickly lead to exponential growth when we have a slowly-expanding record (hence a slowly-expanding transaction size). Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2011-12-19tdb: Avoid a malloc/memcpy in _tdb_storeVolker Lendecke1-17/+8
2011-12-19tdb: be more careful on 4G files.Rusty Russell6-23/+53
I came across a tdb which had wrapped to 4G + 4K, and the contents had been destroyed by processes which thought it only 4k long. Fix this by checking on open, and making tdb_oob() check for wrap itself. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Autobuild-User: Rusty Russell <rusty@rustcorp.com.au> Autobuild-Date: Mon Dec 19 07:52:01 CET 2011 on sn-devel-104
2011-12-13tdb: Fix python documentation for tdb moduleAmitay Isaacs1-1/+1
2011-11-24Remove unused variable.Jeremy Allison1-1/+0
Autobuild-User: Jeremy Allison <jra@samba.org> Autobuild-Date: Thu Nov 24 00:17:41 CET 2011 on sn-devel-104
2011-11-21Fix a bunch of "warning: variable ‘XXXX’ set but not used ↵Jeremy Allison1-3/+1
[-Wunused-but-set-variable]" warnings from the new gcc. Autobuild-User: Jeremy Allison <jra@samba.org> Autobuild-Date: Mon Nov 21 23:39:08 CET 2011 on sn-devel-104
2011-11-13waf: Factor checking for undefined symbol flags out into separate method.Jelmer Vernooij1-0/+2
Autobuild-User: Jelmer Vernooij <jelmer@samba.org> Autobuild-Date: Sun Nov 13 19:38:38 CET 2011 on sn-devel-104
2011-11-13pytdb: Shorter description which fits on a single line.Jelmer Vernooij1-1/+2
2011-11-13tdb: Only check for pkg-config file when checking for system tdb.Jelmer Vernooij1-1/+1
2011-08-21wafsamba: Only install .pc files if libraries are public.Jelmer Vernooij1-3/+1
2011-08-21tdb: Install pkg-config file.Jelmer Vernooij1-1/+3
2011-08-16tdb: increment sequence number in tdb_wipe_all().Rusty Russell1-0/+2
TDB2 testing revealed that tdb1 doesn't do this. It's minor, but fix it. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Autobuild-User: Rusty Russell <rusty@rustcorp.com.au> Autobuild-Date: Tue Aug 16 10:47:41 CEST 2011 on sn-devel-104
2011-07-21tdb: remove 'EOF' print from tdbrestoreAndrew Tridgell1-1/+0
the EOF message is pointless, and makes for noisy scripts Pair-Programmed-With: Andrew Bartlett <abartlet@samba.org>
2011-06-20tdb2: create tdb2 versions of various testing TDBs.Rusty Russell1-21/+42
Soon, TDB2 will handle tdb1 files, but until then, we substitute. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2011-06-20tdb_store: check returns for 0, not -1.Rusty Russell2-4/+4
TDB2 returns a negative error number on failure. This is compatible if we always check for != 0 instead of == -1. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2011-06-08tdb: enable VALGRIND to remove valgrind noise.Rusty Russell1-35/+0
Andrew Bartlett complained that valgrind needs --partial-loads-ok=yes otherwise the Jenkins hash makes it complain. My benchmarking here revealed that at least with modern gcc (4.5) and CPU (Intel i5 32 bit) there's no measurable performance penalty for the "correct" code, so rip out the optimized one. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Autobuild-User: Rusty Russell <rusty@rustcorp.com.au> Autobuild-Date: Wed Jun 8 11:05:47 CEST 2011 on sn-devel-104
2011-06-02lib/tdb/python/tests/simple.py: don't assume TDB ordering.Rusty Russell1-1/+3
TDB2 can break this assumption. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Autobuild-User: Rusty Russell <rusty@rustcorp.com.au> Autobuild-Date: Thu Jun 2 12:07:40 CEST 2011 on sn-devel-104
2011-05-06tdb: fix a build warning.Günther Deschner1-0/+1
Guenther
2011-04-23Support the 'PYTHON' environment variable.Jelmer Vernooij1-0/+1
Autobuild-User: Jelmer Vernooij <jelmer@samba.org> Autobuild-Date: Sat Apr 23 04:19:05 CEST 2011 on sn-devel-104
2011-04-19tdb_backup: avoid transaction on backup file, use lockallSimo Sorce1-8/+13
Transactions have the side effect of generating bigger files. By removing the transaction files get as much as 30% smaller. Autobuild-User: Simo Sorce <idra@samba.org> Autobuild-Date: Tue Apr 19 23:34:37 CEST 2011 on sn-devel-104
2011-04-19tdb: make sure we skip over recovery area correctly.Rusty Russell3-17/+44
If it's really the recovery area, we can trust the rec_len field, and don't have to go groping for bitpatterns. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Autobuild-User: Rusty Russell <rusty@rustcorp.com.au> Autobuild-Date: Tue Apr 19 14:15:22 CEST 2011 on sn-devel-104
2011-04-18tdb_expand: limit the expansion with huge recordsSimo Sorce1-5/+20
ldb can create huge records when saving indexes. Limit the tdb expansion to avoid consuming a lot of memory for no good reason if the record being saved is huge.
2011-04-18tdb: tdb_repack() only when it's worthwhile.Rusty Russell1-6/+31
tdb_repack() is expensive and consumes memory, so we can spend some effort to see if it's worthwhile. In particular, tdbbackup doesn't need to repack: it started with an empty database! Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2011-04-18tdb: fix transaction recovery area for converted tdbs.Rusty Russell1-2/+4
This is why macros are dangerous; these were converting the pointers, not the things pointed to! Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2011-03-30tdb: Fix Coverity ID 2238: SECURE_CODINGVolker Lendecke1-24/+24
generated by cgit (git 2.34.1) at 2024-11-03 17:15:00 +0000