s4-ldb: ldb indexing rewrite - part1

This gets rid of the @IDXPTR approach to in-transaction indexing,
instead using an in-memory tdb to hold index values during a
transaction. This also cleans up a lot of the internal indexing logic,
hopefully making it easier to understand.

One of the big changes is in memory management, with a lot more use
made of talloc tricks to avoid copying dn lists, and shortcuts used to
avoid high intersection and union calculation costs.

The overall result is that a re-provision on my laptop goes from 48s
to a bit over 10s.

1 files changed, 9 insertions, 0 deletions

diff --git a/source4/lib/ldb/ldb_tdb/ldb_cache.c b/source4/lib/ldb/ldb_tdb/ldb_cache.c
index 2c399686ea..cd2249d4a2 100644
--- a/source4/lib/ldb/ldb_tdb/ldb_cache.c
+++ b/source4/lib/ldb/ldb_tdb/ldb_cache.c
@@ -357,6 +357,8 @@ int ltdb_cache_load(struct ldb_module *module)
 	    ltdb->cache->attributes == NULL) {
 		goto failed;
 	}
+	ltdb->cache->one_level_indexes = false;
+	ltdb->cache->attribute_indexes = false;
 	    
 	indexlist_dn = ldb_dn_new(module, ldb, LTDB_INDEXLIST);
 	if (indexlist_dn == NULL) goto failed;
@@ -366,6 +368,13 @@ int ltdb_cache_load(struct ldb_module *module)
 		goto failed;
 	}
 
+	if (ldb_msg_find_element(ltdb->cache->indexlist, LTDB_IDXONE) != NULL) {
+		ltdb->cache->one_level_indexes = true;
+	}
+	if (ldb_msg_find_element(ltdb->cache->indexlist, LTDB_IDXATTR) != NULL) {
+		ltdb->cache->attribute_indexes = true;
+	}
+
 	if (ltdb_attributes_load(module) == -1) {
 		goto failed;
 	}