Tokyo Cabinet

1
Tokyo Cabinet

• simple and fast DBM

2
features

• an implementation of UNIX DBM
• c.f. Berkeley DB, GDBM, CDB, and QDBM
• stores and retrieves key/value pairs
• hash table on file
• very fast storage and retrieval
• store 13.5 sec / 10M records
• retrieve 23.6 sec / 10M records
• small file size
• overhead 4 bytes per bucket 1620 bytes per
  record
• e.g. 467,776,380 / 10M records (each 16 bytes)
• no actual size limitation
• max size 8EB on 64-bit system, 8TB on 32-bit
  system

3
implementation

• conforming to C99 and POSIX
• runs on Linux, Solaris, BSDs, and Mac OS X
• Windows (MinGW) support (in the future)
• thread safe and high parallelism
• uses Pthreads
• locking by record
• record compression
• supports Deflate and Block Sorting
• OO-style APIs
• bool put(DB db, char key, char value)
• char get(DB db, char key)

4
file format

• four sections in one file
• meta data
• bucket array
• free block pool
• record entities
• hash table and binary search tree
• handles collision of hash value by tree
• compares by the second hash function
• typically O(1) and worst O(log n)

5
meta data
bucket array
free block pool
record entities














6
bucket array
binary search tree


150
L
R




L
R
L
R
170
130


L
R
160


L
R
120


L
R
180
7
future missions

• transaction and its log file
• improves reliability
• B tree API
• supports range search and cursor
• bindings for scripting languages
• Perl, Ruby, and Java
• applications
• memcached-like server mechanism
• inverted index