nilql

Library for working with encrypted data within nilDB queries and replies.

Description and Purpose

This library provides cryptographic operations that are compatible with nilDB nodes and clusters, allowing developers to leverage certain privacy-enhancing technologies (PETs) when storing, operating upon, and retrieving data while working with nilDB. The table below summarizes the functionalities that nilQL makes available.

Cluster	Operation	Implementation Details	Supported Types
single node	store	XSalsa20 stream cipher Poly1305 MAC	32-bit signed integer UTF-8 string (<4097 bytes)
	match	deterministic salted hashing via SHA-512	32-bit signed integer UTF-8 string (<4097 bytes)
	sum	non-deterministic Paillier with 2048-bit primes	32-bit signed integer
multiple nodes	store	XOR-based secret sharing	32-bit signed integer UTF-8 string (<4097 bytes)
	match	deterministic salted hashing via SHA-512	32-bit signed integer UTF-8 string (<4097 bytes)
	sum	additive secret sharing (no threshold) Shamir secret sharing (with threshold) (prime modulus 2^32 + 15 for both)	32-bit signed integer

The library supports two categories of keys:

1. SecretKey: Keys in this category support operations within a single node or across multiple nodes. These contain cryptographic material for encryption, decryption, and other operations. Notably, a SecretKey instance includes blinding masks that a client need not share with the cluster. By using SecretKey instances a client can retain exclusive access to its data even if all servers in a cluster collude.
2. ClusterKey: Keys in this category represent cluster configurations but do not contain cryptographic material. These can be used for managing multiple-node clusters. Unlike SecretKey instances, ClusterKey instances do not incorporate blinding masks. This means each node in a cluster has access to a raw secret share of encrypted data.

Threshold secret sharing is supported when encrypting in a summation-compatible way for multiple-node clusters. A threshold specifies the minimum number of nodes required to reconstruct the original data. Shamir's secret sharing is employed when encrypting with a threshold value, ensuring that encrypted data can only be decrypted if the required number of shares is available.

Installation and Usage

The library can be imported in the usual ways:

import nilql
from nilql import *

Example: Generating Keys

The example below generates a SecretKey instance for a single-node cluster:

from nilql import SecretKey

cluster = {'nodes': [{}]}
secret_key = SecretKey.generate(cluster, {'store': True})

The example below generates a SecretKey instance for a multiple-node (i.e., three-node) cluster with a two-share decryption threshold:

cluster = {'nodes': [{}, {}, {}]}
secret_key = SecretKey.generate(cluster, {'sum': True}, threshold=2)

Example: Encrypting and Decrypting Data

The below example encrypts and decrypts an integer:

plaintext = 123
ciphertext = nilql.encrypt(secret_key, plaintext)
decrypted = nilql.decrypt(secret_key, ciphertext)
assert plaintext == decrypted

The below example encrypts and decrypts a string:

plaintext = "hello"
ciphertext = nilql.encrypt(secret_key, plaintext)
decrypted = nilql.decrypt(secret_key, ciphertext)
assert plaintext == decrypted

Development

All installation and development dependencies are fully specified in pyproject.toml. The project.optional-dependencies object is used to specify optional requirements for various development tasks. This makes it possible to specify additional options (such as docs, lint, and so on) when performing installation using pip:

python -m pip install ".[docs,lint]"

Documentation

The documentation can be generated automatically from the source files using Sphinx:

python -m pip install ".[docs]"
cd docs
sphinx-apidoc -f -E --templatedir=_templates -o _source .. && make html

Testing and Conventions

All unit tests are executed and their coverage is measured when using pytest (see the pyproject.toml file for configuration details):

python -m pip install ".[test]"
python -m pytest

The subset of the unit tests included in the module itself and can be executed using doctest:

python src/nilql/nilql.py -v

Style conventions are enforced using Pylint:

python -m pip install ".[lint]"
python -m pylint src/nilql test/test_nilql.py

Contributions

In order to contribute to the source code, open an issue or submit a pull request on the GitHub page for this library.

Versioning

The version number format for this library and the changes to the library associated with version number increments conform with Semantic Versioning 2.0.0.

Publishing

This library can be published as a package on PyPI via the GitHub Actions workflow found in .github/workflows/build-publish-sign-release.yml that follows the recommendations found in the Python Packaging User Guide.

Ensure that any links in this README document to the Read the Docs documentation of this package (or its dependencies) have appropriate version numbers. Also ensure that the Read the Docs project for this library has an automation rule that activates and sets as the default all tagged versions.