summaryrefslogtreecommitdiff
path: root/lib/tdb2
AgeCommit message (Collapse)AuthorFilesLines
2012-01-30tdb2: copy tdb1's changed expansion logic.Rusty Russell3-27/+40
TDB2 uses the same expansion logic as TDB1, which got factored out recently. So update TDB2 to match. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit c438ec17d7b2efe76e56e5fc5ab88bd4a02735e8)
2012-01-30tdb2: careful on wrap.Rusty Russell6-20/+29
It's much harder to wrap a 64-bit tdb2 than a 32-bit tdb1, but we should still take care against bugs. Also, we should *not* cast the length to a size_t when comparing it to the stat result, in case size_t is 32 bit. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 6f7cb26e589cea081e71c59801eae87178967861)
2011-12-25tdb: Use tdb_parse_record in tdb_update_hashVolker Lendecke1-11/+15
This avoids a tdb_fetch, thus a malloc/memcpy/free in the tdb_store path
2011-12-19tdb2: Avoid a malloc/memcpy in _tdb1_storeVolker Lendecke1-19/+8
Autobuild-User: Volker Lendecke <vlendec@samba.org> Autobuild-Date: Mon Dec 19 16:53:40 CET 2011 on sn-devel-104
2011-12-14tdb2: Fix python documentation for tdb.Tdb objectAmitay Isaacs1-1/+1
2011-12-05tdb2: tlist: remove type arg from tlist_top(), tlist_tail()Rusty Russell1-1/+1
With the type canary, it's unnecessary. Though the implementation is a bit more awkward since they longer map directly through to list_top/tail. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit a6b5111fe6948e51114c33aa34785c9fd0d403e6) Autobuild-User: Rusty Russell <rusty@rustcorp.com.au> Autobuild-Date: Mon Dec 5 12:13:08 CET 2011 on sn-devel-104
2011-12-05tdb2: display capability information in tdb_summary()Rusty Russell2-25/+145
This means we know they're there in future, and what restrictions they carry. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit b3ca95351517e76b635347b39382b059a66f8388)
2011-12-05tdb2: add a capability list from the header.Rusty Russell9-27/+410
This allows even more extensibility in future: in particular, the top bits of each capability tell us what to do if we don't understand it: fail the open, fail to open for write, or don't try to check the format. tdb_check needs to understand the capability list so it can know to skip over it: each element in the list is prefixed with the type tag and the length. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 35f198de1851a7d57064546b7ced677b6fabee27)
2011-12-05tdb2: provide tdb_layout_write() rather than implying it by new_tdb_layout arg.Rusty Russell4-32/+47
Neater API. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 79d603a5f73dfbb655d8d08f67eecb5f2da542d5)
2011-12-05tdb2: add an internal TDB_CANT_CHECK flag.Rusty Russell2-0/+12
This will be used shortly to indicate that a TDB2 file indicates it cannot be checked. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit e01d795c8964b791def1e9f68c386b350b3a2a84)
2011-12-05tdb2: suppress failtest more than once on mmap.Rusty Russell1-0/+5
Now we test failing mmap, ccanlint -v time has increased from 200 to 330 seconds. Worse, tests no time out on my laptop. Fix this, by preventing us from going down that particular rabbit hole. ccanlint -v now takes 201 seconds again. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit fbae37ba91ec230e34be564084099726cc3a9d47)
2011-12-05tdb2: simplify failtest helper.Rusty Russell2-31/+3
failtest now culls duplicates for itself (and more efficiently), so don't replicate the logic here. It changes things a bit, because failtest uses backtraces rather than a simple call point to find duplicates. Also, fix one case (in run-11-simple-fetch.c) where we simply exited rather than using failtest_exit(). We got away with it before, because we never hit that particular failure pattern. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 3d99c9334fe3067c88772547b9c06acec21616ea)
2011-12-05tdb2: failtest: use a linked list for history, not an array.Rusty Russell2-30/+26
This avoids a silly realloc, but more importantly it gets us closer to being runtime extensible, as each history element can be a different size. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 9571a41e8494f3135557e3ec50c2de856392173e)
2011-12-05tdb2: consolidate testing failtest suppression routines.Rusty Russell8-72/+14
Less cut & paste means less patching as failtest changes. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 1819a36a3e69565bd7b853503fceb846558a45bd)
2011-12-05tdb2: fix intermittant failure in run-50-multiple-freelists-fail.cRusty Russell4-10/+18
layout.c's TDB creation functions were incorrect in case of a hash collision, causing occasional failure. Make it always use the (previously-failing) seed value, and fix it. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 60a487d57979e4364e70c837079f3cf083ddc9c7)
2011-11-02tdb2: don't be fascist when TDB_VERSION1 is specified.Rusty Russell2-5/+40
We currently insist that a tdb file be a version1 file if tdb_open() is passed the TDB_VERSION1 flag; we fail if it's actually a tdb2. But that makes generic wrappers harder, and is unlikely to be what the user wants: if they do, they can check tdb_get_flags() & TDB_VERSION1 after opening. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 9691464a16ef22d6acadfef209666381dfe22b2f) Autobuild-User: Rusty Russell <rusty@rustcorp.com.au> Autobuild-Date: Wed Nov 2 08:38:38 CET 2011 on sn-devel-104
2011-11-01tdb2: use HAVE_LIBREPLACE instead of _SAMBA_BUILD_.Rusty Russell3-5/+5
This is what lib/tdb2 really wants: to know if it should use normal headers or replace.h. It currently uses _SAMBA_BUILD_, which is wrong for ldb, for example. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Autobuild-User: Rusty Russell <rusty@rustcorp.com.au> Autobuild-Date: Tue Nov 1 03:19:58 CET 2011 on sn-devel-104
2011-09-21tdb2: change --enable-tdb2-breaks-compat to --enable-tdb2Rusty Russell1-2/+2
Since we no longer break compatibility, don't scare people. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Autobuild-User: Rusty Russell <rusty@rustcorp.com.au> Autobuild-Date: Wed Sep 21 09:25:11 CEST 2011 on sn-devel-104
2011-09-20tdb2: allow readonly changes even while holding locks.Rusty Russell1-10/+0
This happens in SAMBA with the TDB_VERSION1, presumably due to a read-only traverse nested inside a normal traverse (since it doesn't occur without TDB_VERSION1). Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 24e5ddb143fb5e79112649472258f5da67cc7362) Autobuild-User: Rusty Russell <rusty@rustcorp.com.au> Autobuild-Date: Tue Sep 20 09:35:10 CEST 2011 on sn-devel-104
2011-09-14tdb2: remove bogus leftover .orig file.Rusty Russell1-618/+0
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Autobuild-User: Rusty Russell <rusty@rustcorp.com.au> Autobuild-Date: Wed Sep 14 06:06:41 CEST 2011 on sn-devel-104
2011-09-14tdb2: fix prototypes which return enum TDB_ERROR.Rusty Russell2-10/+10
Since enums and ints are compatible in C, the compiler didn't warn that our prototypes for these functions disagreed with the definitions. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 6d3832ee613adeb9ae7ed6454996ffa39c32650f)
2011-09-14tdb2: fix return handling in pytdb wrapper.Rusty Russell1-8/+10
tdb_close() does genuinely return non-zero, not an error code, even in tdb2. And tdb_exists() returns true or false, not an error code. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2011-09-14tdb2: add error conversion functions.Rusty Russell12-91/+97
This clarifies the code a little, but also provides a more explicit mechanism which can be used to debug error handling (by introducing tdb_err_t and making it a pointer type). Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit afa6d57b7d93fe4675a952f556eb462951baa257)
2011-09-14tdb2: set tdb_error() on tdb_transaction_prepare_commit.Rusty Russell1-1/+1
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit feb36b14278b4230af70dab90369a345b6a027ef)
2011-09-14tdb2: make tests work in parallel.Rusty Russell6-6/+6
Lazy cut & paste of tdb names causes failure when we run in parallel. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 380372e733416c2b348d5307f536d0a0807e95df)
2011-09-14tdb2: fix error handling for tdb1_transaction_commit.Rusty Russell1-1/+2
tdb_repack() returns an enum TDB_ERROR, whereas tdb1_transaction_commit is expected to return 0 or -1. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit b679512e4260b9847e2f846b07443e6907d8276f)
2011-09-14tdb2: don't return -1 (ie. TDB_ERR_CORRUPT) on transaction write fail.Rusty Russell1-1/+1
A left-over -1 return; should be returning ecode (probably TDB_ERR_IO). Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 077bdae6877d26749987b26a1b5b28cdba5ebbdd)
2011-09-14tdb2: enhance tdb1 porting document.Rusty Russell1-0/+28
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 9cbae0f976118472f0065eee674eacf6ab4b80c5)
2011-09-14tdb2: delete old tdb1 tests.Rusty Russell2-394/+0
These two tdb1-specific tests have been superceded by the TDB_VERSION1 flags used in the main tests. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit e42097b73f6cd509aa17a48487a707401a2d4bd0)
2011-09-14tdb2: test: Speed up run-01-new_database.Rusty Russell2-7/+78
Don't run tdb_check with failtest on, since it is very slow. Do the tdb_check w/ failtest in a new test, but skip MMAP suppression which doesn't add much and slows down valgrind a lot. Before this change run-01-new_database took 40 seconds (under valgrind), after it takes 8 seconds, and run-12-check takes 3 seconds). Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit b3ae89992ecaffed31dcc2e912539c289478801a)
2011-09-14tdb2: speed up testing.Rusty Russell2-5/+7
The time to run tests under valgrind has become excessive; particularly the failure tests which fork(). Thus we cut down testing: 1) api-94-repack: reduce from 234 seconds to 2 seconds by cutting iterations, despite adding TDB_VERSION1 tests. 2) api-missing-entries: reduce from 17 seconds to under 1 second by not checking db inside loop, but at end. This reduces the total ccanlint time from 729 to 489 seconds. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit a99c2ccf97465d47c3277d997ea93f20ff97ad4d)
2011-09-14tdb2: test: convert (non-invasive) run tests to api tests.Rusty Russell23-23/+115
This reduces compilation time, since these are merely linked with the pre-built module, rather than recompiling it into the test (which allows for fancy things like failtest). This reduces the test compile time down from about 62 seconds to 45 seconds. Since ccanlint compiles tests three times (once normally, once with coverage, and once with reduced config.h) this makes a difference: we go from 780 seconds to 729 seconds. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit c4ca9f54301c0367891be6330f59fdd5dcdd51d1)
2011-09-14tdb2: test: fix run-57-die-during-transaction.c to be more efficient.Rusty Russell3-3/+18
We track malloc and free, but we didn't catch the free() inside external_agent, which means that our list of allocations keeps growing. Particularly under valgrind, which re-uses memory less than the glibc allocator. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit d9cbd7d4454ae35e4e2f6d18a9469bf26948e4b9)
2011-09-14tdb2: don't continue if tdb1_find fails.Rusty Russell1-16/+37
The TDB1 code's tdb1_find() returns 0 on error; the callers should not assume that the error means that the entry wasn't found, but use last_error to determine it. This was found by looking at how long the failure path testing for test/run-10-simple-store.c was taking under valgrind, ie: valgrind -q ./run-10-simple-store --show-slowest This change dropped the time for that test from 53 seconds to 19 seconds. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 1be090a2d749713cfd0c4584cafb97bffd716189)
2011-09-14tdb2: test: try (almost) all tests with TDB_VERSION1 flag.Rusty Russell33-128/+364
There are some minor changes required, in particular: 1) Make sure lockcheck understands tdb1 allrecord lock upgrades. 2) Handle tdb1 sequence number jumps: various operations increment the sequence number twice, especually tdb_append. 3) Don't test fail on unlock, since it gets triggered with traversal on the tdb1 backend (we didn't actually ever test this case for tdb2). 4) Move clear_if_first to offset 4, to match tdb1. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 818ed29730b030ce79855fc35c212b51adff3180)
2011-09-14tdb2: log allocation failures in tdb1 backend.Rusty Russell4-12/+30
The TDB2 tests are stricter about this; they want every error logged. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 670ba98f74b52df541d153eeab9d3310932e75cd)
2011-09-14tdb2: fix bogus error from tdb_unlockall() after fork in tdb1 backend.Rusty Russell1-3/+3
We're always allowed to unlock after a fork; by setting the count to 0 before calling the generic unlock function we don't trigger the pid check. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit ba5bb8eae6bcd230fd5321c618d6a9d56e74e7d1)
2011-09-14tdb2: check lock owner in tdb1 backend.Rusty Russell3-27/+88
This reports errors if we fork() while holding a lock, or misuse a tdb which we have dual-opened. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit bef6f1b02e95370ecb2cb44be87c82afc9cb74b2)
2011-09-14tdb2: return TDB_ERR_RDONLY from transactions on R/O databases, log nestingRusty Russell2-8/+25
Rather than rturning TDB_ERR_EINVAL, return TDB_ERR_RDONLY, and log when we fail due to nesting. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 27647f94668040d91de377849dce87dabd72be69)
2011-09-14tdb2: add stats to tdb1 backend.Rusty Russell4-5/+26
It's actually quite a good fit; we use compare_wrong_bucket for dead records, which is kind of correct (they should be in the free list). Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit a3e4ebff2eb9dc2e386160b8acf77d70236f4def)
2011-09-14tdb2: increment seqnum on tdb_wipe_all in tdb1 backend.Rusty Russell1-0/+1
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 79dee5018a407be1d0674d6108b60f8e8c882b7c)
2011-09-14tdb2: log an error when calling tdb_store() on read-only TDB in tdb1 backend.Rusty Russell1-1/+3
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit b236a8dcc37bb864b632a2898a64f3c9bf75b4ac)
2011-09-14tdb2: catch errors in tdb1_needs_recovery()Rusty Russell3-9/+23
The tdb1 backend simply returns "true" if there's an error determining if a tdb needs recovery. But this leads failtest down a rabbit hole; it's better to return the error at this case (and makes for better for diagnostics, since they will come from the first fault, not later in tdb1_transaction_recover(). Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 332d0c29baa6896e67c439aeb47f58a104fbc781)
2011-09-14tdb2: add tdb_attribute_tdb1_max_deadRusty Russell8-68/+41
This allows us to simulate the old "volatile" flag for tdb1. It's not necessary for tdb2. As this is the last function in tdb1.h, we remove that file. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit b8e64e9346793164651a36eccb3c205077e5c91b)
2011-09-14tdb2: expose and document tdb1_incompatible_hash.Rusty Russell2-2/+12
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit a4f2eb983b92b68bcad2a0d640924b8b456e3d69)
2011-09-14tdb2: make tdb1_null static.Rusty Russell3-6/+7
It's only used in two files, make it static in one and do it manually in the other. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 8571daf007f03a2888566dbaae8a5849ffba60f3)
2011-09-14tdb2: tdb_repackRusty Russell6-105/+148
Move the tdb1_repack() code into the core, make it general, rename to tdb_repack(). It's generic code: copy database into temporary, wipe it, copy back. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit e487983a4099b6f760056ff7182f2ff543e6da71)
2011-09-14tdb2: unify tdb1_wipe into tdb_wipe.Rusty Russell3-2/+7
Switch on the TDB_VERSION1 flag. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit b2555a868e3ee58d1b31f9558e3623d49ed2b2f1)
2011-09-14tdb2: unify tdb1_get_seqnum/tdb1_increment_seqnum into ↵Rusty Russell5-5/+60
tdb_get_seqnum/tdb_inc_seqnum Switch on the TDB_VERSION1 flag. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 57359c26e9626aa986ee0538efd13a44a466f39d)
2011-09-14tdb2: unify tdb1_firstkey/tdb1_nextkey into tdb_firstkey/tdb_nextkey.Rusty Russell3-4/+26
Switch on the TDB_VERSION1 flag. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 7366486e29e4f3d89a3e08fc114079d6441af850)
generated by cgit (git 2.34.1) at 2024-11-16 14:56:21 +0000