Home

Welcome to the StoreMate wiki!

Features

Basic storage

Storage implementation consists of two parts:

• A local database, exposed as StorableStore (and implemented by StorableStoreImpl), which delegates to an implementation (StoreBackend), in which all entry metadata is stored, along with small inlined data.
• Currently a single backend implementation exists, for BDB-JE (see storemate-backend-bdb-je sub-module)
• File system, where larger data entries are stored. Directory structure is handled by FileManager; amount of code is minimal and mostly consists of arranging files in balanced directories, using timestamps for creating new directories as necessary.

Data to store is divided into three parts:

• Opaque key: no semantics are implied at StoreMate level, sorting is based on raw byte collation: additional semantics are usually defined by higher-level systems.
• Entry metadata
• Standard entry metadata contains minimal state information, such as is-soft-deleted flag; compression indicator, checksum for payload, last-modified time
• Optional custom metadata is opaque byte sequence (byte[]) exposed to higher-level systems: StoreMate simply stores it along with other metadata without using it for anything
• Payload: actual data to store; small payloads are inlined in the database (size threshold configurable), larger are stored on disk.

Payload is automatically compressed (unless detected to be compressed, or explicitly instructed not to compress) when stored; and uncompressed unless client indicates it accepts compressed data. This is meant to allow convenient but customizable handling of compression over protocols like HTTP.

Implementation

Format definitions:

• BDB Storage format