HC-tree is a project that aims to create a new backend for SQLite, specifically optimized for lead-follower style replication. While still experimental, HC-tree can be used as a replacement for SQLite, albeit with limitations.

SQLite is sometimes used as the core of a client/server database system. Although it works reliably in such cases, the database module that is used to store the b-tree structures in the database file was not designed with this case in mind and can be improved in several ways. The HC-tree (hctree) project is an attempt to develop a new database module that improves on plain SQLite in the following way:

• Improved concurrency: basic SQLite is limited to a single concurrent writer.
• Using the begin-concurrent extension changes this so that multiple writers can run concurrently using optimistic page-level locking.
• Replication support: SQLite supports the sessions extension, which allows the contents of a transaction to be serialized for transfer and application to a second database.
• Eliminate database size constraints: Stock SQLite uses 32-bit page numbers. If the default 4KiB page size is used, this results in a maximum database size of 2^44 bytes, or 16TiB.

Prototype state
This project contains a fork of the SQLite project that has been modified to include a prototype of the hctree database backend. It can be built and used in the same ways as the existing SQLite library code.

The prototype is advanced enough to experiment with it and run multithreaded performance tests, but it is not complete.

When opening an existing database, the library automatically determines whether it is a standard SQLite database or an hctree database. By default, when a new database is created, the library creates a regular SQLite database. To create a new hctree database, the SQLite URI parameter “hctree=1” must be set the first time the new database is opened.

file://new_hctree_database.db?hctree=1

As a final note on enhanced features, HC-tree aims to replace 32-bit page numbers with 48-bit page numbers in the future to exceed SQLite’s 16TiB limit.

HC-tree is still under development, and while it can be used to experiment and evaluate whether it is a good replacement for an existing SQLite-based solution, it still has a number of limitations, including lack of BEGIN EXCLUSIVE support, reduced performance for transactions that don’t fit entirely in main memory, and others that can be explored on the official project page.