NAME

cdb — format of the constant database

DESCRIPTION

The cdb database format provides a space-efficient (key,value) database. The format doesn't allow updates in any convenient form. The file overhead is around 5 bytes per key and 5 bytes per entry. Keys are not stored and it is the responsibility of the caller to validate matches. The index structure is based on a minimal perfect hash table, so exactly one entry has to be checked for a match.

General Format

The header record of a cdb database consists of the following:

struct header_cdb { 
	uint8_t magic[7]; 
	uint8_t version; 
	uint8_t description[16]; 
	uint32_t data_size; 
	uint32_t entries; 
	uint32_t entries_index; 
	uint32_t seed; 
};

All fields are in Little Endian byte order.

This is followed by a description of the hash function of entries_index records. The size of each index entry is the logarithm of entries to base 256, rounded up.

The index records are followed by the start offsets of the entries, followed by data_size. The offsets are relative to the end of the offset record table and are monotically increasing. The size of each offset record is the logarithm of data_size to base 256, rounded up.

The offset table is followed by the entries in order. No separation or padding is added.

Limitations

The cdb file format is by design intended for a database that can be mapped into memory. The hard limit for the number of entries and keys is 3435973836. The total size of all values must be smaller than 4GiB.

HISTORY

Support for the cdb format first appeared in NetBSD 6.0.

AUTHORS

The cdbr and cdbw functions have been written by Joerg Sonnenberger <joerg@NetBSD.org>.

April 27, 2010

NetBSD 6.1