summaryrefslogtreecommitdiff
AgeCommit message (Collapse)AuthorFilesLines
2011-09-14tdb2: speed up testing.Rusty Russell2-5/+7
The time to run tests under valgrind has become excessive; particularly the failure tests which fork(). Thus we cut down testing: 1) api-94-repack: reduce from 234 seconds to 2 seconds by cutting iterations, despite adding TDB_VERSION1 tests. 2) api-missing-entries: reduce from 17 seconds to under 1 second by not checking db inside loop, but at end. This reduces the total ccanlint time from 729 to 489 seconds. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit a99c2ccf97465d47c3277d997ea93f20ff97ad4d)
2011-09-14tdb2: test: convert (non-invasive) run tests to api tests.Rusty Russell23-23/+115
This reduces compilation time, since these are merely linked with the pre-built module, rather than recompiling it into the test (which allows for fancy things like failtest). This reduces the test compile time down from about 62 seconds to 45 seconds. Since ccanlint compiles tests three times (once normally, once with coverage, and once with reduced config.h) this makes a difference: we go from 780 seconds to 729 seconds. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit c4ca9f54301c0367891be6330f59fdd5dcdd51d1)
2011-09-14tdb2: test: fix run-57-die-during-transaction.c to be more efficient.Rusty Russell3-3/+18
We track malloc and free, but we didn't catch the free() inside external_agent, which means that our list of allocations keeps growing. Particularly under valgrind, which re-uses memory less than the glibc allocator. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit d9cbd7d4454ae35e4e2f6d18a9469bf26948e4b9)
2011-09-14tdb2: don't continue if tdb1_find fails.Rusty Russell1-16/+37
The TDB1 code's tdb1_find() returns 0 on error; the callers should not assume that the error means that the entry wasn't found, but use last_error to determine it. This was found by looking at how long the failure path testing for test/run-10-simple-store.c was taking under valgrind, ie: valgrind -q ./run-10-simple-store --show-slowest This change dropped the time for that test from 53 seconds to 19 seconds. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 1be090a2d749713cfd0c4584cafb97bffd716189)
2011-09-14tdb2: test: try (almost) all tests with TDB_VERSION1 flag.Rusty Russell33-128/+364
There are some minor changes required, in particular: 1) Make sure lockcheck understands tdb1 allrecord lock upgrades. 2) Handle tdb1 sequence number jumps: various operations increment the sequence number twice, especually tdb_append. 3) Don't test fail on unlock, since it gets triggered with traversal on the tdb1 backend (we didn't actually ever test this case for tdb2). 4) Move clear_if_first to offset 4, to match tdb1. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 818ed29730b030ce79855fc35c212b51adff3180)
2011-09-14tdb2: log allocation failures in tdb1 backend.Rusty Russell4-12/+30
The TDB2 tests are stricter about this; they want every error logged. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 670ba98f74b52df541d153eeab9d3310932e75cd)
2011-09-14tdb2: fix bogus error from tdb_unlockall() after fork in tdb1 backend.Rusty Russell1-3/+3
We're always allowed to unlock after a fork; by setting the count to 0 before calling the generic unlock function we don't trigger the pid check. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit ba5bb8eae6bcd230fd5321c618d6a9d56e74e7d1)
2011-09-14tdb2: check lock owner in tdb1 backend.Rusty Russell3-27/+88
This reports errors if we fork() while holding a lock, or misuse a tdb which we have dual-opened. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit bef6f1b02e95370ecb2cb44be87c82afc9cb74b2)
2011-09-14tdb2: return TDB_ERR_RDONLY from transactions on R/O databases, log nestingRusty Russell2-8/+25
Rather than rturning TDB_ERR_EINVAL, return TDB_ERR_RDONLY, and log when we fail due to nesting. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 27647f94668040d91de377849dce87dabd72be69)
2011-09-14tdb2: add stats to tdb1 backend.Rusty Russell4-5/+26
It's actually quite a good fit; we use compare_wrong_bucket for dead records, which is kind of correct (they should be in the free list). Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit a3e4ebff2eb9dc2e386160b8acf77d70236f4def)
2011-09-14tdb2: increment seqnum on tdb_wipe_all in tdb1 backend.Rusty Russell1-0/+1
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 79dee5018a407be1d0674d6108b60f8e8c882b7c)
2011-09-14tdb2: log an error when calling tdb_store() on read-only TDB in tdb1 backend.Rusty Russell1-1/+3
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit b236a8dcc37bb864b632a2898a64f3c9bf75b4ac)
2011-09-14tdb2: catch errors in tdb1_needs_recovery()Rusty Russell3-9/+23
The tdb1 backend simply returns "true" if there's an error determining if a tdb needs recovery. But this leads failtest down a rabbit hole; it's better to return the error at this case (and makes for better for diagnostics, since they will come from the first fault, not later in tdb1_transaction_recover(). Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 332d0c29baa6896e67c439aeb47f58a104fbc781)
2011-09-14tdb2: add tdb_attribute_tdb1_max_deadRusty Russell8-68/+41
This allows us to simulate the old "volatile" flag for tdb1. It's not necessary for tdb2. As this is the last function in tdb1.h, we remove that file. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit b8e64e9346793164651a36eccb3c205077e5c91b)
2011-09-14tdb2: expose and document tdb1_incompatible_hash.Rusty Russell2-2/+12
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit a4f2eb983b92b68bcad2a0d640924b8b456e3d69)
2011-09-14tdb2: make tdb1_null static.Rusty Russell3-6/+7
It's only used in two files, make it static in one and do it manually in the other. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 8571daf007f03a2888566dbaae8a5849ffba60f3)
2011-09-14tdb2: tdb_repackRusty Russell6-105/+148
Move the tdb1_repack() code into the core, make it general, rename to tdb_repack(). It's generic code: copy database into temporary, wipe it, copy back. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit e487983a4099b6f760056ff7182f2ff543e6da71)
2011-09-14tdb2: unify tdb1_wipe into tdb_wipe.Rusty Russell3-2/+7
Switch on the TDB_VERSION1 flag. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit b2555a868e3ee58d1b31f9558e3623d49ed2b2f1)
2011-09-14tdb2: unify tdb1_get_seqnum/tdb1_increment_seqnum into ↵Rusty Russell5-5/+60
tdb_get_seqnum/tdb_inc_seqnum Switch on the TDB_VERSION1 flag. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 57359c26e9626aa986ee0538efd13a44a466f39d)
2011-09-14tdb2: unify tdb1_firstkey/tdb1_nextkey into tdb_firstkey/tdb_nextkey.Rusty Russell3-4/+26
Switch on the TDB_VERSION1 flag. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 7366486e29e4f3d89a3e08fc114079d6441af850)
2011-09-14tdb2: unify tdb1_check and tdb1_summary into tdb_check and tdb_summary.Rusty Russell12-40/+62
Switch on the TDB_VERSION1 flag. Also, change tdb1_check's checkfn argument to return an error code (and set tdb->last_error accordingly). Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit ef92843f2c74ab9d4fa7f167a2182e5e8955df91)
2011-09-14tdb2: unify tdb1_transaction_start etc. into tdb_transaction_start.Rusty Russell10-30/+50
Switch on the TDB_VERSION1 flag. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 6bc8ea012391198bc3898ae2937558b60dd55906)
2011-09-14tdb2: unify tdb1_parse_record into tdb_parse_recordRusty Russell6-44/+32
Switch on the TDB_VERSION1 flag. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 3352e4e947777d4a90a2dd4f3037e1e494231b25)
2011-09-14tdb2: unify tdb1_chainlock et al. into tdb_chainlockRusty Russell11-71/+57
Switch on the TDB_VERSION1 flag. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit ceb2f3eacbad917ee990afbc3dd800bfb0607887)
2011-09-14tdb2: unify tdb1_traverse into tdb_traverseRusty Russell9-26/+40
Switch on the TDB_VERSION1 flag; we now only do a tdb1_traverse_read on a read-only database, as there is no tdb2 equivalent. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit cc2d609dfca7192305ad477b8c2b52cfdc1aa9be)
2011-09-14tdb2: unify tdb1_store into tdb_storeRusty Russell5-6/+10
Switch on the TDB_VERSION1 flag. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 2bba2a856a6cfdb93085b701557850bc1da99587)
2011-09-14tdb2: unify tdb1_delete into tdb_delete.Rusty Russell8-10/+16
Switch on the TDB_VERSION1 flag. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit acb26c9c3ee054c8b2141db25c9bd7be064e425c)
2011-09-14tdb2: unify tdb1_append into tdb_appendRusty Russell5-4/+11
Switch on the TDB_VERSION1 flag. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 8bc38cb177928ef739440c32e33a8eaf23a5dd22)
2011-09-14tdb2: unify tdb1_fetch into tdb_fetchRusty Russell11-30/+32
Switch on the TDB_VERSION1 flag. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 08c05da588018c6b76834e57b66d525546643708)
2011-09-14tdb2: unify tdb1_store into tdb_storeRusty Russell20-37/+47
Switch on the TDB_VERSION1 flag. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 98c754ffe65bc335f66161d6cc8705d4ea2710ec)
2011-09-14tdb2: unify tdb1_open into tdb_openRusty Russell30-524/+400
Finally, we gut tdb1_open() to the tdb1-specific parts, and call it from tdb_open if they specify the TDB_VERSION1 flag or the version is a TDB1. We also unify tdb_close(), based on the TDB_VERSION1 flag. Note that tdb_open(TDB_VERSION1) will fail on an existing tdb if it's a TDB2. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit c8c3b3568677e8b0105f84e4ab068c580faf4591)
2011-09-14tdb2: disallow SEED attribute with TDB_VERSION1.Rusty Russell2-1/+27
It also only makes sense with O_CREAT. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit fc44729c471bcdc7ab0410448d991a65e74cec3d)
2011-09-14tdb2: add TDB_ATTRIBUTE_TDB1_HASHSIZERusty Russell27-94/+286
This replaces the tdb1_open "hash_size" argument. It will only be valid when you call tdb_open() with O_CREAT in open_flags, and TDB_VERSION1 in tdb_flags. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 49475d68deecd0b31597ed6094229171d2699b11)
2011-09-14tdb2: make tdb1_open use attributes for logging, hash function.Rusty Russell25-210/+229
This brings it closer to tdb_open(), so we can unify more easily. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit a446f1d4d161d66bbb19ba2551cf6429a4865964)
2011-09-14tdb2: make jenkins_hash function non-static, rename to tdb_jenkins_hash.Rusty Russell5-16/+20
We're going to need access to it from tdb1_open, so expose it now. It's better in hash.c anyway. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit c8f6f8c2dea76042f74c02eff048847e62adcca6)
2011-09-14tdb2: remove double-open detection for TDB1 databases.Rusty Russell1-38/+0
This is a percursor to unifying with tdb_open() from tdb2, which handles this itself. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 97224cbbe29e76522ec38d031c54b1cadce8ae03)
2011-09-14tdb2: merge tdb1_context into tdb_context.Rusty Russell44-913/+848
Finally, we split out the tdb2-specific parts of tdb_context, and put them into a "tdb2" sub-struct; the tdb1 parts go into a "tdb1" sub-struct. We get rido of tdb1_context and use tdb_context everywhere. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit bbeb528e74c0e234e1f724ac8d54be404cfc6f9a)
2011-09-14tdb2: use tdb->flags & TDB_RDONLY instead of tdb->read_only for TDB1 code.Rusty Russell10-20/+23
There's also a semantic change here: for tdb1, being read-only meant no locking, and it was an error to try to lock a r/o database. For TDB2, you'd need to specify TDB_NOLOCK, which suppresses locking silently. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 4dc29a338fadeac805b369b4b0851c02f1b152c7)
2011-09-14tdb2: Make TDB1 use the same tdb_hash() wrapper as TDB2Rusty Russell12-43/+58
This means converting the tdb1 inbuilt hash functions to the tdb2-style, so they return 64 bit. We truncate to 32 bit everywhere but in tdb_check() which needs to do so explicitly. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 3e46dde21261966941469a6c75e1b45cd2d26324)
2011-09-14tdb2: don't cancel transactions on lock failures in tdb1 backend.Rusty Russell1-5/+3
In TDB2, the user can override locking functions, so they may deliberarely fail (eg. be non-blocking) expecting to retry. For this reason, the TDB2 API requires the caller to cancel the transaction if tdb_transaction_prepare_commit() fails. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 8458811a4126c22635b974718bfbf2876c893c37)
2011-09-14tdb2: suppress tdb1 backend logging when locking returns EINTR or EAGAINRusty Russell2-9/+17
The TDB1 code logs multiple times on errors; we must prevent that in the limited case where locking fails. With TDB2, this can happen due to the lock function attribute, where the user supplies replacement locking functions which are allowed to return with errno EAGAIN or EINTR for various special-effects. Flooding the logs for this is unfriendly. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit a391b2b900bc6d5c0467869a454bdb5c51b5a3be)
2011-09-14tdb2: make tdb1 use same low-level lock functions.Rusty Russell6-335/+136
This means they will use the TDB_ATTRIBUTE_FLOCK functions, and get automatic reopen support. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit fbad02b680b6cbc33ae305ae1cbcdead4eedc7b1)
2011-09-14tdb2: Make tdb1 use the tdb_file structure.Rusty Russell12-202/+206
Because tdb2 allows multiple opens of the same TDB, we separate out the file information from the struct tdb_context. Do the same for tdb1. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 60210a73ec08a7b34ba637ad19e6749cf6dc1952)
2011-09-14tdb2: add TDB_VERSION1 flag.Rusty Russell2-1/+2
This will be set for old TDBs; we can start distinguishing paths based on it now. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 2b50be90a23893a06e0f1436a31d18b97ad0e11d)
2011-09-14tdb2: Make TDB1 code use TDB2's open flags.Rusty Russell24-165/+67
This means getting rid of TDB_VOLATILE (perhaps we should use an attribute for that?), TDB_INCOMPATIBLE_HASH (use the tdb_attribute_hash for that) and TDB_CLEAR_IF_FIRST (use the tdb_attribute_openhook for that). We also get rid of TDB_DISALLOW_NESTING: that's the default for TDB2. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 22d0e0dc59fc9d7e0046fec6971ef478c2d604fd)
2011-09-14tdb2: approximate INCOMPATIBLE_HASH flag with tdb1_incompatible_hash()Rusty Russell6-48/+72
Rather than leak TDB_INCOMPATIBLE_HASH through to the TDB2 API, we make it that if they use the tdb1_incompatible_hash function as their hash, then we treat it as if they had specified the TDB_INCOMPATIBLE_HASH flag (ie. we mark the header so it's unusable by tdb < 1.2.6). This precludes the possibility of using TDB_INCOMPATIBLE_HASH with a custom hash function: that used to allow the user to ensure that old TDB versions couldn't open the TDB file (and recent ones check the header to ensure they're using the right hash). But that's a small loss. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 3004f7e89a5978064b4fb29c1027e6d0d39e9418)
2011-09-14tdb2: Make tdb1 share tdb_store flags, struct tdb_data and TDB_MAGIC_FOOD.Rusty Russell27-187/+168
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 8a47d50d72ea62e378dc92b150c92c1317c73fa3)
2011-09-14tdb2: make TDB1 code use tdb2's TDB_ERROR and tdb_logerr()Rusty Russell23-397/+489
To do this, we make sure that the first few fields of tdb1_context and tdb_context are the same. This sweep also fixes up a few paths where we weren't setting tdb->ecode before our old logging function. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 919937354a331bb964564a11b5a5b80403ff8db9)
2011-09-14tdb2: make tdb1 headers include tdb2 headers.Rusty Russell2-29/+4
They're about to start sharing TDB2's definitions. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 1e3138a0ad0c184d2ca49ce7bc47173eb2fe67bd)
2011-09-14tdb2: rearrange tdb_context layout so tdb1_context can matchRusty Russell1-16/+16
By moving all the parts of struct tdb_context which logging and locking use to the beginning of the structure, we can make tdb1_context match, and thus pass that directly to the tdb_logerr() function, and later the locking functions. This is only necessary until we unify tdb1_context and tdb_context. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> (Imported from CCAN commit 0fd5be2955f8e6487c0f4ab711e2a6958bb8f412)
generated by cgit (git 2.34.1) at 2024-11-23 12:39:48 +0000