TigerBeetle

This is the documentation for TigerBeetle: the financial transactions database designed for mission critical safety and performance to power the next 30 years of OLTP.

This is how the entire documentation is organized:

• Start gets you up and running with a cluster.
• Concepts explains why TigerBeetle exists.
• Coding shows how to integrate TigerBeetle into your application.
• Operating covers deployment and operating a TigerBeetle cluster.
• Reference is a companion to Coding which meticulously documents every detail.

Note that this documentation is aimed at the users of TigerBeetle. If you want to understand how it works under the hood, check out the internals docs.

Start

TigerBeetle is a reliable, fast, and highly available database for financial accounting. It tracks financial transactions or anything else that can be expressed as double-entry bookkeeping, providing three orders of magnitude more performance and guaranteeing durability even in the face of network, machine, and storage faults. You will learn more about why this is an important and hard problem to solve in the Concepts section, but let’s make some real transactions first!

Install

TigerBeetle is a single, small, statically linked binary.

You can download a pre-built Linux binary from tigerbeetle.com:

curl -Lo tigerbeetle.zip https://linux.tigerbeetle.com && unzip tigerbeetle.zip && ./tigerbeetle version

MacOS and Windows versions are also available:

# macOS
curl -Lo tigerbeetle.zip https://mac.tigerbeetle.com && unzip tigerbeetle.zip && ./tigerbeetle version

# Windows
powershell -command "curl.exe -Lo tigerbeetle.zip https://windows.tigerbeetle.com; Expand-Archive tigerbeetle.zip .; .\tigerbeetle version"

Building from source is possible and easy, but is not recommended for most users. Refer to the internals documentation for compilation instructions.

Run a Cluster

Typically, TigerBeetle is deployed as a cluster of 6 replicas, which is described in the Operating section. But it is also possible to run a single-replica cluster, which of course doesn’t provide high-availability, but is convenient for experimentation. That’s what we’ll do here.

First, format a data file:

./tigerbeetle format --cluster=0 --replica=0 --replica-count=1 --development ./0_0.tigerbeetle

A TigerBeetle replica stores everything in a single file (./0_0.tigerbeetle in this case). The --cluster, --replica, and --replica-count arguments set the topology of the cluster (a single replica for this tutorial).

Now, start a replica:

./tigerbeetle start --addresses=3000 --development ./0_0.tigerbeetle

It will listen on port 3000 for connections from clients. There’s intensionally no way to gracefully shut down a replica, you can ^C it freely, and the data will be safe as long as the underlying storage functions correctly. Note that with a real cluster of 6 replicas, the data is safe even if the storage misbehaves.

Connecting to a Cluster

Now that the cluster is running, we can connect to it using a client. TigerBeetle already has clients for several popular programming languages, including Go, NodeJS, Java, and Python, and more are coming; see the Coding section for details. For this tutorial, we’ll keep it simple and connect to the cluster using the built-in CLI client. In a separate terminal, start a REPL with

$ ./tigerbeetle repl --cluster=0 --addresses=3000

The --addresses argument is the port the server is listening on. The --cluster argument is required to double-check that the client connects to the correct cluster. While not strictly necessary, it helps prevent operator errors.

Issuing Transactions

TigerBeetle comes with a pre-defined database schema — double-entry bookkeeping. The Concept section explains why this particular schema, and the Reference documents all the bells and whistles, but, for the purposes of this tutorial, it is enough to understand that there are accounts holding credits and debits balances, and that each transfer moves value between two accounts by incrementing credits on one side and debits on the other.

In the REPL, let’s create two empty accounts:

> create_accounts id=1 code=10 ledger=700, id=2 code=10 ledger=700;
> lookup_accounts id=1, id=2;

{
  "id": "1",
  "user_data": "0",
  "ledger": "700",
  "code": "10",
  "flags": [],
  "debits_pending": "0",
  "debits_posted": "0",
  "credits_pending": "0",
  "credits_posted": "0"
}
{
  "id": "2",
  "user_data": "0",
  "ledger": "700",
  "code": "10",
  "flags": "",
  "debits_pending": "0",
  "debits_posted": "0",
  "credits_pending": "0",
  "credits_posted": "0"
}

Now, create our first transfer and inspect the state of accounts afterwards:

> create_transfers id=1 debit_account_id=1 credit_account_id=2 amount=10 ledger=700 code=10;
> lookup_accounts id=1, id=2;

{
  "id": "1",
  "user_data": "0",
  "ledger": "700",
  "code": "10",
  "flags": [],
  "debits_pending": "0",
  "debits_posted": "10",
  "credits_pending": "0",
  "credits_posted": "0"
}
{
  "id": "2",
  "user_data": "0",
  "ledger": "700",
  "code": "10",
  "flags": "",
  "debits_pending": "0",
  "debits_posted": "0",
  "credits_pending": "0",
  "credits_posted": "10"
}

Note how the transfer amount is added both to the credits and to the debits. That the sum of debits and credits stays equal no matter what is a powerful invariant of the double-entry bookkeeping system.

Conclusion

This is the end of the quick start! You now know how to format a data file, run a single-replica TigerBeetle cluster, and run transactions through it. Here’s where to go from here:

• Concepts explains the “why?” of TigerBeetle, read this to decide if you need to use TigerBeetle.
• Coding gives guidance on developing applications which store accounting data in a TigerBeetle cluster.
• Operating explains how to deploy a TigerBeetle cluster in a highly-available manner, with replication enabled.
• Reference meticulously documents every single available feature and flag of the underlying data model.

Community

If you want to keep up to speed with recent TigerBeetle developments, here are some things to follow:

• Monthly Newsletter covers everything of importance that happened with TigerBeetle. It is a changelog director’s cut!
• Slack is the place to hang out with users and developers of TigerBeetle. We try to answer every question.
• YouTube channel has most of the talks about TigerBeetle, as well as talks from the Systems Distributed conference. We also stream on Twitch, with recordings duplicated to YouTube.
• 𝕏 is good for smaller updates, and word-of-mouth historical trivia you won’t learn elsewhere!
• GitHub — if you want to keep closer to the source!

Concepts

This section is for anyone evaluating TigerBeetle, eager to learn about it, or curious. It focuses on the big picture and problems that TigerBeetle solves. As well as why it looks nothing like a typical SQL database from the outside and from the inside.

• OLTP defines the domain of TigerBeetle — system of record for business transactions.
• Debit-Credit argues that double-entry bookkeeping is the right schema for this domain.
• Performance explains how TigerBeetle achieves state-of-the-art performance.
• Safety shows that safety and performance are not at odds with each other.

Online Transaction Processing (OLTP)

Online Transaction Processing (OLTP) is about recording business transactions in real-time. This could be payments, sales, car sharing rides, game scores, or API usage.

The World is Becoming More Transactional

Historically, general purpose databases like PostgreSQL, MySQL, and SQLite handled OLTP. We refer to these as Online General Purpose (OLGP) databases.

OLTP workloads have increased by 3-4 orders of magnitude in the last 10 years alone. For example:

• The UPI real-time payments switch in India processed 10 billion payments in the year 2019. In January 2025 alone, it processed 16.9 billion payments.
• Cleaner energy and smart metering means energy is being traded by the kilowatt-hour. Customer billing is every 15 or 30 minutes rather than at the end of the month.
• Serverless APIs charge for usage by the second or per-request, rather than per month. (Today, serverless billing at scale is often implemented using MapReduce. This makes it difficult or impossible to offer customers real-time spending caps.)

OLGP databases already struggle to keep up.

But TigerBeetle is built to handle the scale of OLTP workloads today and for the decades to come. It works well alongside OLGP databases, which holds infrequently updated data. TigerBeetle can race ahead, giving your system unparalleled latency and throughput.

Write-Heavy Workloads

A distinguishing characteristic of OLTP is its focus on recording business transactions. In contrast, OLGP databases are often designed for read-heavy or balanced workloads.

TigerBeetle is optimized from the ground up for write-heavy workloads. This means it can handle the increasing scale of OLTP, unlike an OLGP database.

High Contention on Hot Accounts

Business transactions always involve more than one account. One account gets paid but then there are fees, taxes, revenue splits, and other costs to account for.

OLTP systems often have accounts involved in a high percentage of all transactions. This is especially true for accounts that represent the business income or expenses. Locks can be used to ensure that updates to these ‘hot accounts’ are consistent. But the resulting contention can bring the system’s performance to a crawl.

TigerBeetle provides strong consistency guarantees without row locks. This sidesteps the issue of contention on hot accounts. Due to TigerBeetle’s use of the system cache, transactions processing speed even increases.

Business Transactions Don’t Shard Well

One of the most common ways to scale systems is to horizontally scale or shard them. This means different servers process different sets of transactions. Unfortunately, business transactions don’t shard well. Horizontal scaling is a poor fit for OLTP:

• Most accounts cannot be neatly partitioned between shards.
• Transactions between accounts on different shards become more complex and slow.
• Row locks on hot accounts worsen when the transactions must execute across shards.

Another approach to scaling OLTP systems is to use MapReduce for billing. But this makes it hard to provide real-time balance reporting or spending limits. It also creates a poor user experience that’s hard to fix post system design.

TigerBeetle uses a single-core design and unique performance optimizations to deliver high throughput. And this without the downsides of horizontal scaling.

Bottleneck for Your System

You can only do as much business as your database supports. You need a core OLTP database capable of handling your transactions on your busiest days. And for decades to come.

TigerBeetle is designed to handle 1 million transactions per second, to remove the risk of your business outgrowing your database.

Next Up: Debit / Credit is the Schema for OLTP

The world is becoming more transactional. OLTP workloads are increasing and we need a database designed from the ground up. What is the perfect schema and language for this database? Debit / Credit.

Debit / Credit: The Schema for OLTP

As discussed in the previous section, OLTP is all about processing business transactions. We saw that the nuances of OLTP workloads make them tricky to handle at scale.

Now, we’ll turn to the data model and see how the specifics of business transactions actually lend themselves to an incredibly simple schema – and one that’s been in use for centuries!

The “Who, What, When, Where, Why, and How Much” of OLTP

OLTP and business transactions tend to record the same types of information:

• Who: which accounts are transacting?
• What: what type of asset or value is moving?
• When: when was the transaction initiated or when was it finalized?
• Where: where in the world did the transaction take place?
• Why: what type of transaction is this or why is it happening?
• How Much: what quantity of the asset or items was moved?

The Language of Business for Centuries

Debit / credit, or double-entry bookkeeping, has been the lingua franca of business and accounting since at least the 13th century.

The key insight underpinning debit / credit systems is that every transfer records a movement of value from one or more accounts to one or more accounts. Money never appears from nowhere or disappears. This simple principle helps ensure that all of a business’s money is accounted for.

Debit / credit perfectly captures the who, what, when, where, why, and how much of OLTP while ensuring financial consistency.

(For a deeper dive on debits and credits, see our primer on Financial Accounting.)

SQL vs Debit / Credit

While SQL is a great query language for getting data out of a database, OLTP is primarily about getting data in to the database and this is where SQL falls short.

Often, a single business transaction requires multiple SQL queries (on the order of 10 SQL queries per transaction) and potentially even multiple round-trips from the application to the database.

By designing a database specifically for the schema and needs of OLTP, we can ensure our accounting logic is enforced correctly while massively increasing performance.

TigerBeetle Enforces Debit / Credit in the Database

The schema of OLTP is built in to TigerBeetle’s data model, and is ready for you to use:

• Who: the debit_account_id and credit_account_id indicate which accounts are transacting.
• What: each asset or type of value in TigerBeetle is tracked on a separate ledger. The ledger field indicates what is being transferred.
• When: each transfer has a unique timestamp for when it is processed by the cluster, but you can add another timestamp representing when the transaction happened in the real world in the user_data_64 field.
• Where: the user_data_32 can be used to store the locale where the transfer occurred.
• Why: the code field stores the reason a transfer occurred and should map to an enum or table of all the possible business events.
• How Much: the amount indicates how much of the asset or item is being transferred.

TigerBeetle also supports two-phase transfers out of the box, and can express complex atomic chains of transfers using linked events. These powerful built-in primitives allow for a large vocabulary of patterns and recipes for data modeling.

Crucially, accounting invariants such as balance limits are enforced within the database, avoiding round-trips between your database and application logic.

Immutability is Essential

Another critical element of debit / credit systems is immutability: once transfers are recorded, they cannot be erased. Reversals are implemented with separate transfers to provide a full and auditable log of business events.

Accidentally dropping rows or tables is bad in any database, but it is unacceptable when it comes to accounting. Legal compliance and good business practices require that all funds be fully accounted for, and all history be maintained.

Transfers in TigerBeetle are always immutable, providing you with peace of mind out of the box. There is no possibility of a malformed query unintentionally deleting data.

Don’t Roll Your Own Ledger

Many companies start out building their own system for recording business transactions. Then, once their business scales, they realize they need a proper ledger and end up coming back to debits and credits.

A number of prime examples of this are:

• Uber: In 2018, Uber started a 2-year, 40-engineer effort to migrate their collection and disbursement payment platform to one based on the principles of double-entry accounting and debits and credits.¹
• Airbnb: From 2012 to 2016, Airbnb used a MySQL-based data pipeline to record all of its transactions in an immutable store suitable for reporting. The pipeline became too complex, hard to scale, and slow. They ended up building a new financial reporting system based on double-entry accounting.²
• Stripe: While we don’t know when this system initially went into service, Stripe relies on an internal system based on double-entry accounting and an immutable log of events to record all of the payments they process.³

Standardized, Simple, and Scalable

From one perspective, debit / credit may seem like a limited data model. However, it is incredibly flexible and scalable. Any business event can be recorded as debits and credits – indeed, accountants have been doing precisely this for centuries!

Instead of modeling business transactions as a set of ad-hoc tables and relationships, debits and credits provide a simple and standardized schema that can be used across all product lines, now and into the future. This avoids the need to add columns, tables, and complex relations between them as new features are added – and avoids complex schema migrations.

Debit / credit has been the foundation of business for hundreds of years, and now you can leverage TigerBeetle’s high-performance implementation of it built for OLTP in the 21st century.

Next: Performance

So far, we’ve seen why we need a new database designed for OLTP and how debit / credit provides the perfect data model for it. Next, we can look at the performance of a database, designed for OLTP.

Performance

How, exactly, is TigerBeetle so fast?

It’s All About The Interface

The key is that TigerBeetle designed specifically for OLTP workloads, as opposed to OLGP. The prevailing paradigm for OLGP is interactive transactions, where business-logic lives in the application, and the job of the database is to send the data to the application, holding the locks while the data is being processed. This works for mixed read-write workload with low contention, but fails for highly-contended OLTP workloads — locks over the network are very expensive!

With TigerBeetle, all the logic lives inside the database, obviating the need for locking. Not only is this very fast, it is also more convenient — the application can speak Debit/Credit directly, it doesn’t need to translate the language of business to SQL.

Batching, Batching, Batching

On a busy day in a busy city, taking subway is faster than using a car. On empty streets, a personal sports car gives you the best latency, but, when the load and contention increases, due to Little’s law, both latency and throughput become abysmal.

TigerBeetle works like a high-speed train — its interface always deals with batches of transactions, 8k apiece. Although TigerBeetle is a replicated database using a consensus algorithm, the cost of replications is paid only once per batch, which means that TigerBeetle runs not much slower than an in-memory hash map, all the while providing extreme durability and availability.

What’s more, under light load the batches automatically become smaller, trading unnecessary throughput for better latency.

Extreme Engineering

Debit/Credit fixes inefficiency in the interface, pervasive batching amortizes costs, but, to really hit performance targets, solid engineering is required at every level of the stack.

TigerBeetle is built fully from scratch, without using any dependencies, to make sure that all the layers are co-designed for the purposes of OLTP.

TigerBeetle is written in Zig — a systems programming language which doesn’t use garbage collection and is designed for writing fast code.

Every data structure is hand-crafted with the CPU in mind: a transfer object is 128 bytes in size, cache-line aligned. Executing a batch of transfers is just one tight CPU loop!

TigerBeetle allocates all the memory statically: it never runs out of memory, it never stalls due to a GC pause or mutex contention, and it never fragments the memory.

TigerBeetle is designed for io_uring — a new Linux kernel interface for zero syscall networking and storage I/O.

These and other performance rules are captured in TIGER_STYLE.md — the secret recipe that keeps TigerBeetle fast and safe.

Single Threaded By Design

TigerBeetle uses a single core by design and uses a single leader node to process events. Adding more nodes can therefore increase reliability but not throughput.

For a high-performance database, this may seem like an unusual choice. However, sharding in financial databases is notoriously difficult and contention issues often negate the would-be benefits. Specifically, a small number of hot accounts are often involved in a large proportion of the transactions so the shards responsible for those accounts become bottlenecks.

For more details on when single-threaded implementations of algorithms outperform multi-threaded implementations, see “Scalability! But at what COST?.

Performance = Flexibility

Is it really necessary to go to such great lengths in the name of performance? It of course depends on a particular use-case, but it’s worth keeping in mind that higher performance can unlock new use-cases. An OLGP database might be enough to do nightly settlement, but with OLTP real-time settlement is a no-brainer. If a transaction system just hits its throughput target, that means that every unexpected delay or an ops accident lead to missed transactions. If a system operates at one tenth of capacity, this gives a lot of headroom for operators to deal with the unexpected.

Finally, it is always helpful to think about the future. The future is hard to predict (even the present is hard to wrap your head around!), but the option to handle significantly more load on a short notice greatly expands your options.

Next: Safety

Performance can get you very far very fast, but it is useless if the result is wrong. Business transaction processing also requires strong safety guarantees, to ensure that data cannot be lost, and high availability to ensure that money is not lost due to database downtime. Let’s look at how TigerBeetle ensures safety.

Safety

The purpose of a database is to store data: if the database accepted new data, it should be able to retrieve it later. Surprisingly, many databases don’t provide guaranteed durability — usually the data is there, but, under certain edge case conditions, it can get lost!

As the purpose of TigerBeetle is to be the system of record for business transaction, associated with real-world value transfers, it is paramount that the data stored in TigerBeetle is safe.

Strict Serializability

The easiest way to lose data is by using the database incorrectly, by misconfiguring (or just misunderstanding) its isolation level. For this reason, TigerBeetle intentionally supports only the strictest possible isolation level — strict serializability. All transfers are executed one-by-one, on a single core.

Furthermore, TigerBeetle’s state machine is designed according to the end-to-end idempotency principle — each transfer has a unique client-generated u128 id, and each transfer is processed at most once, even in the presence of intermediate retry loops.

High Availability

Some databases rely on a single central server, which puts the data at risk as any single server might fail catastrophically (e.g. due to a fire in the data center). Primary/backup systems with ad-hoc failover can lose data due to split-brain.

To avoid these pitfalls, TigerBeetle implements pioneering Viewstamped Replication and consensus algorithm, that guarantees correct, automatic failover. It’s worth emphasizing that consensus proper needs only be engaged during actual failover. During the normal operation, the cost of consensus is just the cost of replication, which is further minimized because of batching, tail latency tolerance, and pipelining.

TigerBeetle does not depend on synchronized system clocks, does not use leader leases, and performs leader-based timestamping so that your application can deal only with safe relative quantities of time with respect to transfer timeouts. To ensure that the leader’s clock is within safe bounds of “true time”, TigerBeetle combines all the clocks in the cluster to create a fault-tolerant clock that we call “cluster time”.

For highest availability, TigerBeetle should be deployed as a cluster of six replicas across three different cloud providers (two replicas per provider). Because TigerBeetle uses Heidi Howard’s flexible quorums, this deployment is guaranteed to tolerate a complete outage of any cloud provider and will likely survive even if one extra replica fails.

Storage Fault Tolerance

Traditionally, databases assume that disks do not fail, or at least fail politely with a clear error code. This is usually a reasonable assumption, but edge cases matter.

HDD and SSD hardware can fail. Disks can silently return corrupt data ( 0.031% of SSD disks per year, 1.4% of Enterprise HDD disks per year), misdirect IO ( 0.023% of SSD disks per year, 0.466% of Nearline HDD disks per year), or just suddenly become extremely slow, without returning an error code (the so called gray failure).

On top of hardware, software might be buggy or just tricky to use correctly. Handling fsync failures correctly is particularly hard.

TigerBeetle assumes that its disk will fail and takes advantage of replication to proactively repair replica’s local disks.

• All data in TigerBeetle is immutable, checksummed, and hash-chained, providing a strong guarantee that no corruption or tampering happened. In case of a latent sector error, the error is detected and repaired without any operator involvement.
• Most consensus implementations lose data or become unavailable if the write ahead log gets corrupted. TigerBeetle uses Protocol Aware Recovery to remain available unless the data gets corrupted on every single replica.
• To minimize impact of software bugs, TigerBeetle puts as little software as possible between itself and the disk — TigerBeetle manages its own page cache, writes data to disk with O_DIRECT and can work with a block device directly, no file system is necessary.
• TigerBeetle also tolerates gray failure — if a disk on a replica becomes very slow, the cluster fall backs on other replicas for durability.

Software Reliability

Even the advanced algorithm with a formally proved correctness theorem is useless if the implementation is buggy. TigerBeetle uses the oldest and the newest software engineering practices to ensure correctness.

TigerBeetle is written in Zig — a modern systems programming language that removes many instances of undefined behavior, provides spatial memory safety and encourages simple, straightforward code.

TigerBeetle adheres to a strict code style, TIGER_STYLE, inspired by NASA’s power of ten. For example, TigerBeetle uses static memory allocation, which designs away memory fragmentation, out-of-memory errors and use-after-frees.

TigerBeetle is tested in the VOPR — a simulated environment where an entire cluster, running real code, is subjected to all kinds of network, storage and process faults, at 1000x speed. This simulation can find both logical errors in the algorithms and coding bugs in the source. This simulator is running 24/7 on 1024 cores, fuzzing the latest version of the database.

It also runs in your browser: https://sim.tigerbeetle.com!

Human Fallibility

While, with a lot of care, software can be perfected to become virtually bug-free, humans will always make mistakes. TigerBeetle takes this into account and tries to protect from operator errors:

• The surface area is intentionally minimized, with little configurability.
• In particular, there’s only one isolation level — strict serializability.
• Upgrades are automatic and atomic, guaranteeing that each transfer is applied by only a single version of code.
• TigerBeetle always runs with online verification on, to detect any discrepancies in the data.

Is TigerBeetle ACID-compliant?

Yes. Let’s discuss each part:

Atomicity

As part of replication, each operation is durably stored in at least a quorum of replicas’ Write-Ahead Logs (WAL) before the primary will acknowledge the operation as committed. WAL entries are executed through the state machine business logic and the resulting state changes are stored in TigerBeetle’s LSM-Forest local storage engine.

The WAL is what allows TigerBeetle to achieve atomicity and durability since the WAL is the source of truth. If TigerBeetle crashes, the WAL is replayed at startup from the last checkpoint on disk.

However, financial atomicity goes further than this: events and transfers can be linked when created so they all succeed or fail together.

Consistency

TigerBeetle guarantees strict serializability. And at the cluster level, stale reads are not possible since all operations (not only writes, but also reads) go through the global consensus protocol.

However, financial consistency requires more than this. TigerBeetle exposes a double-entry accounting API to guarantee that money cannot be created or destroyed, but only transferred from one account to another. And transfer history is immutable.

Isolation

All client requests (and all events within a client request batch) are executed with the highest level of isolation, serially through the state machine, one after another, before the next operation begins. Counterintuitively, the use of batching and serial execution means that TigerBeetle can also provide this level of isolation optimally, without the cost of locks for all the individual events within a batch.

Durability

Up until 2018, traditional DBMS durability has focused on the Crash Consistency Model, however, Fsyncgate and Protocol Aware Recovery have shown that this model can lead to real data loss for users in the wild. TigerBeetle therefore adopts an explicit storage fault model, which we then verify and test with incredible levels of corruption, something which few distributed systems historically were designed to handle. Our emphasis on protecting Durability is what sets TigerBeetle apart, not only as a ledger but as a DBMS.

However, absolute durability is impossible, because all hardware can ultimately fail. Data we write today might not be available tomorrow. TigerBeetle embraces limited disk reliability and maximizes data durability in spite of imperfect disks. We actively work against such entropy by taking advantage of cluster-wide storage. A record would need to get corrupted on all replicas in a cluster to get lost, and even in that case the system would safely halt.

io_uring Security

io_uring is a relatively new part of the Linux kernel (support for it was added in version 5.1, which was released in May 2019). Since then, many kernel exploits have been found related to io_uring and in 2023 Google announced that they were disabling it in ChromeOS, for Android apps, and on Google production servers.

Google’s post is primarily about how they secure operating systems and web servers that handle hostile user content. In the Google blog post, they specifically note:

we currently consider it safe only for use by trusted components

As a financial system of record, TigerBeetle is a trusted component and it should be running in a trusted environment.

Furthermore, TigerBeetle only uses 128-byte Accounts and Transfers with pure integer fields. TigerBeetle has no (de)serialization and does not take user-generated strings, which significantly constrains the attack surface.

We are confident that io_uring is the safest (and most performant) way for TigerBeetle to handle async I/O. It is significantly easier for the kernel to implement this correctly than for us to include a userspace multithreaded thread pool (for example, as libuv does).

Next: Coding

This concludes the discussion of the concepts behind TigerBeetle — an OLTP database for recording business transactions in real time, using a double-entry bookkeeping schema, which is orders of magnitude faster and keeps the data safe even when the underling hardware inevitably fails.

We will now learn how to build applications on top of TigerBeetle.

Coding

This section is aimed at programmers building applications on top of TigerBeetle. It is organized as a series of loosely connected guides which can be read in any order.

• System Architecture paints the big picture.
• Data Modeling shows how to map business-level entities to the primitives provided by TigerBeetle.
• Financial Accounting, a deep dive into double-entry bookkeeping.
• Requests outlines the database interface.
• Reliable Transaction Submission explains the end-to-end principle and how it helps to avoid double spending.
• Two-Phase Transfers introduces pending transfers, one of the most powerful primitives built into TigerBeetle.
• Linked Events shows how several transfers can be chained together into a larger transaction, which success or fails atomically.
• Time lists the guarantees provided by the TigerBeetle cluster clock.
• Recipes is a library of ready-made solutions for common business requirements such as a currency exchange.
• Clients shows how to use TigerBeetle from the comfort of .Net, Go, Java, Node.js, or Python.

Subscribe to the tracking issue #2231 to receive notifications about breaking changes!

TigerBeetle in Your System Architecture

TigerBeetle is an Online Transaction Processing (OLTP) database built for safety and performance. It is not a general purpose database like PostgreSQL or MySQL. Instead, TigerBeetle works alongside your general purpose database, which we refer to as an Online General Purpose (OLGP) database.

TigerBeetle should be used in the data plane, or hot path of transaction processing, while your general purpose database is used in the control plane and may be used for storing information or metadata that is updated less frequently.

Division of Responsibilities

App or Website

• Initiate transactions
• Generate Transfer and Account IDs

Stateless API Service

• Handle authentication and authorization
• Create account records in both the general purpose database and TigerBeetle when users sign up
• Cache ledger metadata
• Batch transfers
• Apply exchange rates for currency exchange transactions

General Purpose (OLGP) Database

• Store metadata about ledgers and accounts (such as string names or descriptions)
• Store mappings between integer type identifiers used in TigerBeetle and string representations used by the app and API

TigerBeetle (OLTP) Database

• Record transfers between accounts
• Track balances for accounts
• Enforce balance limits
• Enforce financial consistency through double-entry bookkeeping
• Enforce strict serializability of events
• Optionally store pointers to records or entities in the general purpose database in the user_data fields

Ledger, Account, and Transfer Types

For performance reasons, TigerBeetle stores the ledger, account, and transfer types as simple integers. Most likely, you will want these integers to map to enums of type names or strings, along with other associated metadata.

The mapping from the string representation of these types to the integers used within TigerBeetle may be hard-coded into your application logic or stored in a general purpose (OLGP) database and cached by your application. (These mappings should be immutable and append-only, so there is no concern about cache invalidation.)

⚠️ Importantly, initiating a transfer should not require fetching metadata from the general purpose database. If it does, that database will become the bottleneck and will negate the performance gains from using TigerBeetle.

Specifically, the types of information that fit into this category include:

Authentication

TigerBeetle does not support authentication. You should never allow untrusted users or services to interact with it directly.

Also, untrusted processes must not be able to access or modify TigerBeetle’s on-disk data file.

Data Modeling

This section describes various aspects of the TigerBeetle data model and provides some suggestions for how you can map your application’s requirements onto the data model.

Accounts, Transfers, and Ledgers

The TigerBeetle data model consists of Accounts, Transfers, and ledgers.

Ledgers

Ledgers partition accounts into groups that may represent a currency or asset type or any other logical grouping. Only accounts on the same ledger can transact directly, but you can use atomically linked transfers to implement currency exchange.

Ledgers are only stored in TigerBeetle as a numeric identifier on the account and transfer data structures. You may want to store additional metadata about each ledger in a control plane database.

You can also use different ledgers to further partition accounts, beyond asset type. For example, if you have a multi-tenant setup where you are tracking balances for your customers’ end-users, you might have a ledger for each of your customers. If customers have end-user accounts in multiple currencies, each of your customers would have multiple ledgers.

Debits vs Credits

TigerBeetle tracks each account’s cumulative posted debits and cumulative posted credits. In double-entry accounting, an account balance is the difference between the two — computed as either debits - credits or credits - debits, depending on the type of account. It is up to the application to compute the balance from the cumulative debits/credits.

From the database’s perspective the distinction is arbitrary, but accounting conventions recommend using a certain balance type for certain types of accounts.

If you are new to thinking in terms of debits and credits, read the deep dive on financial accounting to get a better understanding of double-entry bookkeeping and the different types of accounts.

Debit Balances

balance = debits - credits

By convention, debit balances are used to represent:

• Operator’s Assets
• Operator’s Expenses

To enforce a positive (non-negative) debit balance, use flags.credits_must_not_exceed_debits.

To keep an account’s balance between an upper and lower bound, see the Balance Bounds recipe.

Credit Balances

balance = credits - debits

By convention, credit balances are used to represent:

• Operator’s Liabilities
• Equity in the Operator’s Business
• Operator’s Income

To enforce a positive (non-negative) credit balance, use flags.debits_must_not_exceed_credits. For example, a customer account that is represented as an Operator’s Liability would use this flag to ensure that the balance cannot go negative.

To keep an account’s balance between an upper and lower bound, see the Balance Bounds recipe.

Compound Transfers

Transfers in TigerBeetle debit a single account and credit a single account. You can read more about implementing compound transfers in Multi-Debit, Multi-Credit Transfers.

Fractional Amounts and Asset Scale

To maximize precision and efficiency, Account debits/credits and Transfer amounts are unsigned 128-bit integers. However, currencies are often denominated in fractional amounts.

To represent a fractional amount in TigerBeetle, map the smallest useful unit of the fractional currency to 1. Consider all amounts in TigerBeetle as a multiple of that unit.

Applications may rescale the integer amounts as necessary when rendering or interfacing with other systems. But when working with fractional amounts, calculations should be performed on the integers to avoid loss of precision due to floating-point approximations.

Asset Scale

When the multiplier is a power of 10 (e.g. 10 ^ n), then the exponent n is referred to as an asset scale. For example, representing USD in cents uses an asset scale of 2.

Examples

• In USD, $1 = 100 cents. So for example,
  • The fractional amount $0.45 is represented as the integer 45.
  • The fractional amount $123.00 is represented as the integer 12300.
  • The fractional amount $123.45 is represented as the integer 12345.

Oversized Amounts

The other direction works as well. If the smallest useful unit of a currency is 10,000,000 units, then it can be scaled down to the integer 1.

The 128-bit representation defines the precision, but not the scale.

⚠️ Asset Scales Cannot Be Easily Changed

When setting your asset scales, we recommend thinking about whether your application may ever require a larger asset scale. If so, we would recommend using that larger scale from the start.

For example, it might seem natural to use an asset scale of 2 for many currencies. However, it may be wise to use a higher scale in case you ever need to represent smaller fractions of that asset.

Accounts and transfers are immutable once created. In order to change the asset scale of a ledger, you would need to use a different ledger number and duplicate all the accounts on that ledger over to the new one.

user_data

user_data_128, user_data_64 and user_data_32 are the most flexible fields in the schema (for both accounts and transfers). Each user_data field’s contents are arbitrary, interpreted only by the application.

Each user_data field is indexed for efficient point and range queries.

While the usage of each field is entirely up to you, one way of thinking about each of the fields is:

• user_data_128 - this might store the “who” and/or “what” of a transfer. For example, it could be a pointer to a business entity stored within the control plane database.
• user_data_64 - this might store a second timestamp for “when” the transaction originated in the real world, rather than when the transfer was timestamped by TigerBeetle. This can be used if you need to model bitemporality. Alternatively, if you do not need this to be used for a timestamp, you could use this field in place of the user_data_128 to store the “who”/“what”.
• user_data_32 - this might store the “where” of a transfer. For example, it could store the jurisdiction where the transaction originated in the real world. In certain cases, such as for cross-border remittances, it might not be enough to have the UTC timestamp and you may want to know the transfer’s locale.

(Note that the code can be used to encode the “why” of a transfer.)

Any of the user_data fields can be used as a group identifier for objects that will be queried together. For example, for multiple transfers used for currency exchange.

id

The id field uniquely identifies each Account and Transfer within the cluster.

The primary purpose of an id is to serve as an “idempotency key” — to avoid executing an event twice. For example, if a client creates a transfer but the server’s reply is lost, the client (or application) will retry — the database must not transfer the money twice.

Note that ids are unique per cluster – not per ledger. You should attach a separate identifier in the user_data field if you want to store a connection between multiple Accounts or multiple Transfers that are related to one another. For example, different currency Accounts belonging to the same user or multiple Transfers that are part of a currency exchange.

TigerBeetle Time-Based Identifiers are recommended for most applications.

When selecting an id scheme:

• Idempotency is particularly important (and difficult) in the context of application crash recovery.
• Be careful to avoid id collisions.
• An account and a transfer may share the same id (they belong to different “namespaces”), but this is not recommended because other systems (that you may later connect to TigerBeetle) may use a single “namespace” for all objects.
• Avoid requiring a central oracle to generate each unique id (e.g. an auto-increment field in SQL). A central oracle may become a performance bottleneck when creating accounts/transfers.
• Sequences of identifiers with long runs of strictly increasing (or strictly decreasing) values are amenable to optimization, leading to higher database throughput.
• Random identifiers are not recommended – they can’t take advantage of all of the LSM optimizations. (Random identifiers have ~10% lower throughput than strictly-increasing ULIDs).

TigerBeetle Time-Based Identifiers (Recommended)

TigerBeetle recommends using a specific ID scheme for most applications. It is time-based and lexicographically sortable. The scheme is inspired by ULIDs and UUIDv7s but is better able to take advantage of LSM optimizations, which leads to higher database throughput.

TigerBeetle clients include an id() function to generate IDs using the recommended scheme.

TigerBeetle IDs consist of:

• 48 bits of (millisecond) timestamp (high-order bits)
• 80 bits of randomness (low-order bits)

When creating multiple objects during the same millisecond, we increment the random bytes rather than generating new random bytes. Furthermore, it is important that IDs are stored in little-endian with the random bytes as the lower-order bits and the timestamp as the higher-order bits. These details ensure that a sequence of objects have strictly increasing IDs according to the server, which improves database optimization.

Similar to ULIDs and UUIDv7s, these IDs have the following benefits:

• they have an insignificant risk of collision.
• they do not require a central oracle to generate.

Reuse Foreign Identifier

This technique is most appropriate when integrating TigerBeetle with an existing application where TigerBeetle accounts or transfers map one-to-one with an entity in the foreign database.

Set id to a “foreign key” — that is, reuse an identifier of a corresponding object from another database. For example, if every user (within the application’s database) has a single account, then the identifier within the foreign database can be used as the Account.id within TigerBeetle.

To reuse the foreign identifier, it must conform to TigerBeetle’s id constraints.

code

The code identifier represents the “why” for an Account or Transfer.

On an Account, the code indicates the account type, such as assets, liabilities, equity, income, or expenses, and subcategories within those classification.

On a Transfer, the code indicates why a given transfer is happening, such as a purchase, refund, currency exchange, etc.

When you start building out your application on top of TigerBeetle, you may find it helpful to list out all of the known types of accounts and movements of funds and mapping each of these to code numbers or ranges.

Financial Accounting

For developers with non-financial backgrounds, TigerBeetle’s use of accounting concepts like debits and credits may be one of the trickier parts to understand. However, these concepts have been the language of business for hundreds of years, so we promise it’s worth it!

This page goes a bit deeper into debits and credits, double-entry bookkeeping, and how to think about your accounts as part of a type system.

Building Intuition with Two Simple Examples

If you have an outstanding loan and owe a bank $100, is your balance 100or−100? Conversely, if you have $200 in your bank account, is the balance 200or−200?

Thinking about these two examples, we can start to build an intuition that the positive or negative sign of the balance depends on whose perspective we’re looking from. That $100 you owe the bank represents a “bad” thing for you, but a “good” thing for the bank. We might think about that same debt differently if we’re doing your accounting or the bank’s.

These examples also hint at the different types of accounts. We probably want to think about a debt as having the opposite “sign” as the funds in your bank account. At the same time, the types of these accounts look different depending on whether you are considering them from the perspective of you or the bank.

Now, back to our original questions: is the loan balance 100or−100 and is the bank account balance 200or−200? On some level, this feels a bit arbitrary.

Wouldn’t it be nice if there were some commonly agreed-upon standards so we wouldn’t have to make such an arbitrary decision? Yes! This is exactly what debits and credits and the financial accounting type system provide.

Types of Accounts

In financial accounting, there are 5 main types of accounts:

• Asset - what you own, which could produce income or which you could sell.
• Liability - what you owe to other people.
• Equity - value of the business owned by the owners or shareholders, or “the residual interest in the assets of the entity after deducting all its liabilities.”¹
• Income - money or things of value you receive for selling products or services, or “increases in assets, or decreases in liabilities, that result in increases in equity, other than those relating to contributions from holders of equity claims.”²
• Expense - money you spend to pay for products or services, or “decreases in assets, or increases in liabilities, that result in decreases in equity, other than those relating to distributions to holders of equity claims.”³

As mentioned above, the type of account depends on whose perspective you are doing the accounting from. In those examples, the loan you have from the bank is liability for you, because you owe the amount to the bank. However, that same loan is an asset from the bank’s perspective. In contrast, the money in your bank account is an asset for you but it is a liability for the bank.

Each of these major categories are further subdivided into more specific types of accounts. For example, in your personal accounting you would separately track the cash in your physical wallet from the funds in your checking account, even though both of those are assets. The bank would split out mortgages from car loans, even though both of those are also assets for the bank.

Double-Entry Bookkeeping

Categorizing accounts into different types is useful for organizational purposes, but it also provides a key error-correcting mechanism.

Every record in our accounting is not only recorded in one place, but in two. This is double-entry bookkeeping. Why would we do that?

Let’s think about the bank loan in our example above. When you took out the loan, two things actually happened at the same time. On the one hand, you now owe the bank $100. At the same time, the bank gave you $100. These are the two entries that comprise the loan transaction.

From your perspective, your liability to the bank increased by $100 while your assets also increased by $100. From the bank’s perspective, their assets (the loan to you) increased by $100 while their liabilities (the money in your bank account) also increased by $100.

Double-entry bookkeeping ensures that funds are always accounted for. Money never just appears. Funds always go from somewhere to somewhere.

Keeping Accounts in Balance

Now we understand that there are different types of accounts and every transaction will be recorded in two (or more) accounts – but which accounts?

The Fundamental Accounting Equation stipulates that:

Assets - Liabilities = Equity

Using our loan example, it’s no accident that the loan increases assets and liabilities at the same time. Assets and liabilities are on the opposite sides of the equation, and both sides must be exactly equal. Loans increase assets and liabilities equally.

Here are some other types of transactions that would affect assets, liabilities, and equity, while maintaining this balance:

• If you withdraw $100 in cash from your bank account, your total assets stay the same. Your bank account balance (an asset) would decrease while your physical cash (another asset) would increase.
• From the perspective of the bank, you withdrawing $100 in cash decreases their assets in the form of the cash they give you, while also decreasing their liabilities because your bank balance decreases as well.
• If a shareholder invests $1000 in the bank, that increases both the bank’s assets and equity.

Assets, liabilities, and equity represent a point in time. The other two main categories, income and expenses, represent flows of money in and out.

Income and expenses impact the position of the business over time. The expanded accounting equation can be written as:

Assets - Liabilities = Equity + Income − Expenses

You don’t need to memorize these equations (unless you’re training as an accountant!). However, it is useful to understand that those main account types lie on different sides of this equation.

Debits and Credits vs Signed Integers

Instead of using a positive or negative integer to track a balance, TigerBeetle and double-entry bookkeeping systems use debits and credits.

The two entries that give “double-entry bookkeeping” its name are the debit and the credit: every transaction has at least one debit and at least one credit. (Note that for efficiency’s sake, TigerBeetle Transfers consist of exactly one debit and one credit. These can be composed into more complex multi-debit, multi-credit transfers.) Which entry is the debit and which is the credit? The answer is easy once you understand that accounting is a type system. An account increases with a debit or credit according to its type.

When our example loan increases the assets and liabilities, we need to assign each of these entries to either be a debit or a credit. At some level, this is completely arbitrary. For clarity, accountants have used the same standards for hundreds of years:

How Debits and Credits Increase or Decrease Account Balances

• Assets and expenses are increased with debits, decreased with credits
• Liabilities, equity, and income are increased with credits, decreased with debits

Or, in a table form:

From the perspective of our example bank:

• You taking out a loan debits (increases) their loan assets and credits (increases) their bank account balance liabilities.
• You paying off the loan debits (decreases) their bank account balance liabilities and credits (decreases) their loan assets.
• You depositing cash debits (increases) their cash assets and credits (increases) their bank account balance liabilities.
• You withdrawing cash debits (decreases) their bank account balance liabilities and credits (decreases) their cash assets.

Note that accounting conventions also always write the debits first, to represent the funds flowing from the debited account to the credited account.

If this seems arbitrary and confusing, we understand! It’s a convention, just like how most programmers need to learn zero-based array indexing and then at some point it becomes second nature.

Account Types and the “Normal Balance”

Some other accounting systems have the concept of a “normal balance”, which would indicate whether a given account’s balance is increased by debits or credits.

When designing for TigerBeetle, we recommend thinking about account types instead of “normal balances”. This is because the type of balance follows from the type of account, but the type of balance doesn’t tell you the type of account. For example, an account might have a normal balance on the debit side but that doesn’t tell you whether it is an asset or expense.

Takeaways

• Accounts are categorized into types. The 5 main types are asset, liability, equity, income, and expense.
• Depending on the type of account, an increase is recorded as either a debit or a credit.
• All transfers consist of two entries, a debit and a credit. Double-entry bookkeeping ensures that all funds come from somewhere and go somewhere.

When you get started using TigerBeetle, we would recommend writing a list of all the types of accounts in your system that you can think of. Then, think about whether, from the perspective of your business, each account represents an asset, liability, equity, income, or expense. That determines whether the given type of account is increased with a debit or a credit.

Want More Help Understanding Debits and Credits?

Ask the Community

Have questions about debits and credits, TigerBeetle’s data model, how to design your application on top of it, or anything else?

Come join our Community Slack and ask any and all questions you might have!

Dedicated Consultation

Would you like the TigerBeetle team to help you design your chart of accounts and leverage the power of TigerBeetle in your architecture?

Let us help you get it right. Contact our CEO, Joran Dirk Greef, at joran@tigerbeetle.com to set up a call.

Requests

A request queries or updates the database state.

A request consists of one or more events of the same type sent to the cluster in a single message. For example, a single request can create multiple transfers but it cannot create both accounts and transfers.

The cluster commits an entire request at once. Events are applied in series, such that successive events observe the effects of previous ones and event timestamps are totally ordered.

Each request receives one reply message from the cluster. The reply contains one result for each event in the request.

Request Types

• create_accounts: create Accounts
• create_transfers: create Transfers
• lookup_accounts: fetch Accounts by id
• lookup_transfers: fetch Transfers by id
• get_account_transfers: fetch Transfers by debit_account_id or credit_account_id
• get_account_balances: fetch the historical account balance by the Account’s id.
• query_accounts: query Accounts
• query_transfers: query Transfers

More request types, including more powerful queries, are coming soon!

Events and Results

Each request has a corresponding event and result type:

Idempotency

Events that create objects are idempotent. The first event to create an object with a given id will receive the ok result. Subsequent events that attempt to create the same object will receive the exists result.

Batching Events

To achieve high throughput, TigerBeetle amortizes the overhead of consensus and I/O by batching many events in each request.

In the default configuration, the maximum batch sizes for each request type are:

TigerBeetle clients automatically batch events. Therefore, it is recommended to share the client instances between multiple threads or tasks to have events batched transparently.

• Node
• Go
• Java
• .NET
• Python

Guarantees

• A request executes within the cluster at most once.
• Requests do not time out. Clients will continuously retry requests until they receive a reply from the cluster. This is because in the case of a network partition, a lack of response from the cluster could either indicate that the request was dropped before it was processed or that the reply was dropped after the request was processed. Note that individual pending transfers within a request may have timeouts.
• Requests retried by their original client session receive identical replies.
• Requests retried by a different client (same request body, different session) may receive different replies.
• Events within a request are executed in sequence. The effects of a given event are observable when the next event within that request is applied.
• Events within a request do not interleave with events from other requests.
• All events within a request batch are committed, or none are. Note that this does not mean that all of the events in a batch will succeed, or that all will fail. Events succeed or fail independently unless they are explicitly linked
• Once committed, an event will always be committed — the cluster’s state never backtracks.
• Within a cluster, object timestamps are unique and strictly increasing. No two objects within the same cluster will have the same timestamp. Furthermore, the order of the timestamps indicates the order in which the objects were committed.

Reliable Transaction Submission

When making payments or recording transfers, it is important to ensure that they are recorded once and only once – even if some parts of the system fail during the transaction.

There are some subtle gotchas to avoid, so this page describes how to submit events – and especially transfers – reliably.

The App or Browser Should Generate the ID

Transfers and Accounts carry an id field that is used as an idempotency key to ensure the same object is not created twice.

The client software, such as your app or web page, that the user interacts with should generate the id (not your API). This id should be persisted locally before submission, and the same id should be used for subsequent retries.

1. User initiates a transfer.
2. Client software (app, web page, etc) generates the transfer id.
3. Client software persists the id in the app or browser local storage.
4. Client software submits the transfer to your API service.
5. API service includes the transfer in a request.
6. TigerBeetle creates the transfer with the given id once and only once.
7. TigerBeetle responds to the API service.
8. The API service responds to the client software.

Handling Network Failures

The method described above handles various potential network failures. The request may be lost before it reaches the API service or before it reaches TigerBeetle. Or, the response may be lost on the way back from TigerBeetle.

Generating the id on the client side ensures that transfers can be safely retried. The app must use the same id each time the transfer is resent.

If the transfer was already created before and then retried, TigerBeetle will return the exists response code. If the transfer had not already been created, it will be created and return the ok.

Handling Client Software Restarts

The method described above also handles potential restarts of the app or browser while the request is in flight.

It is important to persist the id to local storage on the client’s device before submitting the transfer. When the app or web page reloads, it should resubmit the transfer using the same id.

This ensures that the operation can be safely retried even if the client app or browser restarts before receiving the response to the operation. Similar to the case of a network failure, TigerBeetle will respond with the ok if a transfer is newly created and exists if an object with the same id was already created.

Two-Phase Transfers

A two-phase transfer moves funds in stages:

1. Reserve funds (pending)
2. Resolve funds (post, void, or expire)

The name “two-phase transfer” is a reference to the two-phase commit protocol for distributed transactions.

Reserve Funds (Pending Transfer)

A pending transfer, denoted by flags.pending, reserves its amount in the debit/credit accounts’ debits_pending/credits_pending fields, respectively. Pending transfers leave the debits_posted/credits_posted unmodified.

Resolve Funds

Pending transfers can be posted, voided, or they may time out.

Post-Pending Transfer

A post-pending transfer, denoted by flags.post_pending_transfer, causes a pending transfer to “post”, transferring some or all of the pending transfer’s reserved amount to its destination.

• If the posted amount is less than the pending transfer’s amount, then only this amount is posted, and the remainder is restored to its original accounts.
• If the posted amount is equal to the pending transfer’s amount or equal to AMOUNT_MAX (2^128 - 1), the full pending transfer’s amount is posted.
• If the posted amount is greater than the pending transfer’s amount (but less than AMOUNT_MAX), exceeds_pending_transfer_amount is returned.

Additionally, when flags.post_pending_transfer is set:

• pending_id must reference a pending transfer.
• flags.void_pending_transfer must not be set.

The following fields may either be zero or they must match the value of the pending transfer’s field:

• debit_account_id
• credit_account_id
• ledger
• code

Void-Pending Transfer

A void-pending transfer, denoted by flags.void_pending_transfer, restores the pending amount its original accounts. Additionally, when this field is set:

• pending_id must reference a pending transfer.
• flags.post_pending_transfer must not be set.

The following fields may either be zero or they must match the value of the pending transfer’s field:

• debit_account_id
• credit_account_id
• ledger
• code

Expire Pending Transfer

A pending transfer may optionally be created with a timeout. If the timeout interval passes before the transfer is either posted or voided, the transfer expires and the full amount is returned to the original account.

Note that timeouts are given as intervals, specified in seconds, rather than as absolute timestamps. For more details on why, read the page about Time in TigerBeetle.

Errors

A pending transfer can only be posted or voided once. It cannot be posted twice or voided then posted, etc.

Attempting to resolve a pending transfer more than once will return the applicable error result:

• pending_transfer_already_posted
• pending_transfer_already_voided
• pending_transfer_expired

Interaction with Account Invariants

The pending transfer’s amount is reserved in a way that the second step in a two-phase transfer will never cause the accounts’ configured balance invariants (credits_must_not_exceed_debits or debits_must_not_exceed_credits) to be broken, whether the second step is a post or void.

Pessimistic Pending Transfers

If an account with debits_must_not_exceed_credits has credits_posted = 100 and debits_posted = 70 and a pending transfer is started causing the account to have debits_pending = 50, the pending transfer will fail. It will not wait to get to posted status to fail.

All Transfers Are Immutable

To reiterate, completing a two-phase transfer (by either marking it void or posted) does not involve modifying the pending transfer. Instead you create a new transfer.

The first transfer that is marked pending will always have its pending flag set.

The second transfer will have a post_pending_transfer or void_pending_transfer flag set and a pending_id field set to the id of the first transfer. The id of the second transfer will be unique, not the same id as the initial pending transfer.

Examples

The following examples show the state of two accounts in three steps:

1. Initially, before any transfers
2. After a pending transfer
3. And after the pending transfer is posted or voided

Post Full Pending Amount

Post Partial Pending Amount

Void Pending Transfer

Linked Events

Events within a request succeed or fail independently unless they are explicitly linked using flags.linked (Account.flags.linked or Transfer.flags.linked).

When the linked flag is specified, it links the outcome of a Transfer or Account creation with the outcome of the next one in the request. These chains of events will all succeed or fail together.

The last event in a chain is denoted by the first Transfer or Account without this flag.

The last Transfer or Account in a request may never have the flags.linked set, as it would leave a chain open-ended. Attempting to do so will result in the linked_event_chain_open error.

Multiple chains of events may coexist within a request to succeed or fail independently.

Events within a chain are executed in order, or are rolled back on error, so that the effect of each event in the chain is visible to the next. Each chain is either visible or invisible as a unit to subsequent transfers after the chain. The event that was the first to fail within a chain will have a unique error result. Other events in the chain will have their error result set to linked_event_failed.

Linked Transfers Example

Consider this set of Transfers as part of a request:

If any of transfers B, C, or D fail (for example, due to exceeds_credits), then B, C, and D will all fail. They are linked.

Transfers A and E fail or succeed independently of B, C, D, and each other.

After the chain of linked events has executed, the fact that they were linked will not be saved. To save the association between Transfers or Accounts, it must be encoded into the data model, for example by adding an ID to one of the user data fields.

Time

Time is a critical component of all distributed systems and databases. Within TigerBeetle, we keep track of two types of time: logical time and physical time. Logical time is about ordering events relative to each other, and physical time is the everyday time, a numeric timestamp.

Logical Time

TigerBeetle uses a consensus protocol (Viewstamped Replication) to guarantee strict serializability for all operations.

In other words, to an external observer, TigerBeetle cluster behaves as if it is just a single machine which processes the incoming requests in order. If an application submits a batch of transfers with transfer T1, receives a reply, and then submits a batch with another transfer T2, it is guaranteed that T2 will observe the effects of T1. Note, however, that there could be concurrent requests from multiple applications, so, unless T1 and T2 are in the same batch of transfers, some other transfer could happen in between them. See the reference for precise guarantees.

Physical Time

TigerBeetle uses physical time in addition to the logical time provided by the consensus algorithm. Financial transactions require physical time for multiple reasons, including:

• Liquidity - TigerBeetle supports Two-Phase Transfers that reserve funds and hold them in a pending state until they are posted, voided, or the transfer times out. A timeout is useful to ensure that the reserved funds are not held in limbo indefinitely.
• Compliance and Auditing - For regulatory and security purposes, it is useful to have a specific idea of when (in terms of wall clock time) transfers took place.

TigerBeetle uses two-layered approach to physical time. On the basic layer, each replica asks the underling operating system about the current time. Then, timing information from several replicas is aggregated to make sure that the replicas roughly agree on the time, to prevent a replica with a bad clock from issuing incorrect timestamps. Additionally, this “cluster time” is made strictly monotonic, for end user’s convenience.

Why TigerBeetle Manages Timestamps

An important invariant is that the TigerBeetle cluster assigns all timestamps. In particular, timestamps on Transfers and Accounts are set by the cluster when the corresponding event arrives at the primary. This is why the timestamp field must be set to 0 when operations are submitted by the client.

Similarly, the Transfer.timeout is given as an interval in seconds, rather than as an absolute timestamp, because it is also managed by the primary. The timeout is calculated relative to the timestamp when the operation arrives at the primary.

This restriction is needed to make sure that any two timestamps always refer to the same underlying clock (cluster’s physical time) and are directly comparable. This in turn provides a set of powerful guarantees.

Timestamps are Totally Ordered

All timestamps within TigerBeetle are unique, immutable and totally ordered. A transfer that is created before another transfer is guaranteed to have an earlier timestamp (even if they were created in the same request).

In other systems this is also called a “physical” timestamp, “ingestion” timestamp, “record” timestamp, or “system” timestamp.

Further Reading

If you are curious how exactly it is that TigerBeetle achieves strictly monotonic physical time, we have a talk and a blog post with details:

• Detecting Clock Sync Failure in Highly Available Systems (YouTube)
• Three Clocks are Better than One (TigerBeetle Blog)

Recipes

A collection of solutions for common use-cases. What to exchange some currency? Or made a wrong transfer and want to undo that? We have a recipe for that!

• Currency Exchange
• Multi-Debit, Multi-Credit Transfers
• Closing Accounts
• Balance-Conditional Transfers
• Balance-Invariant Transfers
• Balance Bounds
• Correcting Transfers
• Rate Limiting

Currency Exchange

Some applications require multiple currencies. For example, a bank may hold balances in many different currencies. If a single logical entity holds multiple currencies, each currency must be held in a separate TigerBeetle Account. (Normalizing to a single currency at the application level should be avoided because exchange rates fluctuate).

Currency exchange is a trade of one type of currency (denoted by the ledger) for another, facilitated by an entity called the liquidity provider.

Data Modeling

Distinct ledger values denote different currencies (or other asset types). Transfers between pairs of accounts with different ledgers are not permitted.

Instead, currency exchange is implemented by creating two atomically linked different-ledger transfers between two pairs of same-ledger accounts.

A simple currency exchange involves four accounts:

• A source account A₁, on ledger 1.
• A destination account A₂, on ledger 2.
• A source liquidity account L₁, on ledger 1.
• A destination liquidity account L₂, on ledger 2.

and two linked transfers:

• A transfer T₁ from the source account to the source liquidity account.
• A transfer T₂ from the destination liquidity account to the destination account.

The transfer amounts vary according to the exchange rate.

• Both liquidity accounts belong to the liquidity provider (e.g. a bank or exchange).
• The source and destination accounts may belong to the same entity as one another, or different entities, depending on the use case.

Example

Consider sending $100.00 from account A₁ (denominated in USD) to account A₂ (denominated in INR). Assuming an exchange rate of $1.00 = ₹82.42135, $100.00 = ₹8242.135:

• Amounts are represented as integers.
• Because both liquidity accounts belong to the same entity, the entity does not lose money on the transaction.
  • If the exchange rate is precise, the entity breaks even.
  • If the exchange rate is not precise, the application should round in favor of the liquidity account to deter arbitrage.
• Because the two transfers are linked together, they will either both succeed or both fail.

Spread

In the prior example, the liquidity provider breaks even. A fee (i.e. spread) can be included in the linked chain as a separate transfer from the source account to the source liquidity account (A₁ to L₁).

This is preferable to simply modifying the exchange rate in the liquidity provider’s favor because it implicitly records the exchange rate and spread at the time of the exchange — information that cannot be derived if the two are combined.

Example

This depicts the same scenario as the prior example, except the liquidity provider charges a $0.10 fee for the transaction.

Multi-Debit, Multi-Credit Transfers

TigerBeetle is designed for maximum performance. In order to keep it lean, the database only supports simple transfers with a single debit and a single credit.

However, you’ll probably run into cases where you want transactions with multiple debits and/or credits. For example, you might have a transfer where you want to extract fees and/or taxes.

Read on to see how to implement one-to-many and many-to-many transfers!

Note that all of these examples use the Linked Transfers flag (flags.linked) to ensure that all of the transfers succeed or fail together.

One-to-Many Transfers

Transactions that involve multiple debits and a single credit OR a single debit and multiple credits are relatively straightforward.

You can use multiple linked transfers as depicted below.

Single Debit, Multiple Credits

This example debits a single account and credits multiple accounts. It uses the following accounts:

• A source account A, on the USD ledger.
• Three destination accounts X, Y, and Z, on the USD ledger.

Multiple Debits, Single Credit

This example debits multiple accounts and credits a single account. It uses the following accounts:

• Three source accounts A, B, and C on the USD ledger.
• A destination account X on the USD ledger.

Multiple Debits, Single Credit, Balancing debits

This example debits multiple accounts and credits a single account. The total amount to transfer to the credit account is known (in this case, 100), but the balances of the individual debit accounts are not known. That is, each debit account should contribute as much as possible (in order of precedence) up to the target, cumulative transfer amount.

It uses the following accounts:

• Three source accounts A, B, and C, with flags.debits_must_not_exceed_credits.
• A destination account X.
• A control account LIMIT, with flags.debits_must_not_exceed_credits.
• A control account SETUP, for setting up the LIMIT account.

If the cumulative credit balance of A + B + C is less than 100, the chain will fail (transfer 6 will return exceeds_credits).

Many-to-Many Transfers

Transactions with multiple debits and multiple credits are a bit more involved (but you got this!).

This is where the accounting concept of a Control Account comes in handy. We can use this as an intermediary account, as illustrated below.

In this example, we’ll use the following accounts:

• Two source accounts A and B on the USD ledger.
• Three destination accounts X, Y, and Z, on the USD ledger.
• A compound entry control account Control on the USD ledger.

Here, we use two transfers to debit accounts A and B and credit the Control account, and another three transfers to credit accounts X, Y, and Z.

If you looked closely at this example, you may have noticed that we could have debited B and credited Z directly because the amounts happened to line up. That is true!

For a little more extreme performance, you might consider implementing logic to circumvent the control account where possible, to reduce the number of transfers to implement a compound journal entry.

However, if you’re just getting started, you can avoid premature optimizations (we’ve all been there!). You may find it easier to program these compound journal entries always using a control account – and you can then come back to squeeze this performance out later!

Close Account

In accounting, a closing entry calculates the net debit or credit balance for an account and then credits or debits this balance respectively, to zero the account’s balance and move the balance to another account.

Additionally, it may be desirable to forbid further transfers on this account (i.e. at the end of an accounting period, upon account termination, or even temporarily freezing the account for audit purposes.

Example

Given a set of accounts:

The “closing entries” for accounts A and B are expressed as linked chains, so they either succeed or fail atomically.

• Account A: the linked transfers are T1 and T2.

• Account B: the linked transfers are T3 and T4.

• Account C: is the control account and will not be closed.

• T1 and T3 are balancing transfers with AMOUNT_MAX as the Transfer.amount so that the application does not need to know (or query) the balance prior to closing the account.

  The stored transfer’s amount will be set to the actual amount transferred.

• T2 and T4 are closing transfers that will cause the respective account to be closed.

  The closing transfer must be also a pending transfer so the action can be reversible.

After committing these transfers, A and B are closed with net balance zero, and will reject any further transfers.

To re-open the closed account, the pending closing transfer can be voided, reverting the closing action (but not reverting the net balance):

After committing these transfers, A and B are re-opened and can accept transfers again:

Balance-Conditional Transfers

In some use cases, you may want to execute a transfer if and only if an account has at least a certain balance.

It would be unsafe to check an account’s balance using the lookup_accounts and then perform the transfer, because these requests are not be atomic and the account’s balance may change between the lookup and the transfer.

You can atomically run a check against an account’s balance before executing a transfer by using a control or temporary account and linked transfers.

Preconditions

1. Target Account Must Have a Limited Balance

The account for whom you want to do the balance check must have one of these flags set:

• flags.debits_must_not_exceed_credits for accounts with credit balances
• flags.credits_must_not_exceed_debits for accounts with debit balances

2. Create a Control Account

There must also be a designated control account. As you can see below, this account will never actually take control of the target account’s funds, but we will set up simultaneous transfers in and out of the control account.

Executing a Balance-Conditional Transfer

The balance-conditional transfer consists of 3 linked transfers.

We will refer to two amounts:

• The threshold amount is the minimum amount the target account should have in order to execute the transfer.
• The transfer amount is the amount we want to transfer if and only if the target account’s balance meets the threshold.

If the Source Account Has a Credit Balance

If the Source Account Has a Debit Balance

Understanding the Mechanism

Each of the 3 transfers is linked, meaning they will all succeed or fail together.

The first transfer attempts to transfer the threshold amount to the control account. If this transfer would cause the source account’s net balance to go below zero, the account’s balance limit flag would ensure that the first transfer fails. If the first transfer fails, the other two linked transfers would also fail.

If the first transfer succeeds, it means that the source account did have the threshold balance. In this case, the second transfer cancels the first transfer (returning the threshold amount to the source account). Then, the third transfer would execute the desired transfer to the ultimate destination account.

Note that in the tables above, we do the balance check on the source account. The balance check could also be applied to the destination account instead.

Balance-invariant Transfers

For some accounts, it may be useful to enforce flags.debits_must_not_exceed_credits or flags.credits_must_not_exceed_debits balance invariants for only a subset of all transfers, rather than all transfers.

Per-transfer credits_must_not_exceed_debits

This recipe requires three accounts:

1. The source account, to debit.
2. The destination account, to credit. (With neither flags.credits_must_not_exceed_debits nor flags.debits_must_not_exceed_credits set, since in this recipe we are only enforcing the invariant on a per-transfer basis.
3. The control account, to test the balance invariant. The control account should have flags.credits_must_not_exceed_debits set.

When the destination account’s final credits do not exceed its debits, the chain will succeed. When the destination account’s final credits exceeds its debits, transfer 2 will fail with exceeds_debits.

Balance Bounds

It is easy to limit an account’s balance using either flags.debits_must_not_exceed_credits or flags.credits_must_not_exceed_debits.

What if you want an account’s balance to stay between an upper and a lower bound?

This is possible to check atomically using a set of linked transfers. (Note: with the must_not_exceed flag invariants, an account is guaranteed to never violate those invariants. This maximum balance approach must be enforced per-transfer – it is possible to exceed the limit simply by not enforcing it for a particular transfer.)

Preconditions

1. Target Account Should Have a Limited Balance

The account whose balance you want to bound should have one of these flags set:

• flags.debits_must_not_exceed_credits for accounts with credit balances
• flags.credits_must_not_exceed_debits for accounts with debit balances

2. Create a Control Account with the Opposite Limit

There must also be a designated control account.

As you can see below, this account will never actually take control of the target account’s funds, but we will set up simultaneous transfers in and out of the control account to apply the limit.

This account must have the opposite limit applied as the target account:

• flags.credits_must_not_exceed_debits if the target account has a credit balance
• flags.debits_must_not_exceed_credits if the target account has a debit balance

3. Create an Operator Account

The operator account will be used to fund the Control Account.

Executing a Transfer with a Balance Bounds Check

This consists of 5 linked transfers.

We will refer to two amounts:

• The limit amount is upper bound we want to maintain on the target account’s balance.
• The transfer amount is the amount we want to transfer if and only if the target account’s balance after a successful transfer would be within the bounds.

If the Target Account Has a Credit Balance

In this case, we are keeping the Destination Account’s balance between the bounds.

*This must be set to the transfer ID of the pending transfer (in this example, it is transfer 3).

If the Target Account Has a Debit Balance

In this case, we are keeping the Destination Account’s balance between the bounds.

*This must be set to the transfer ID of the pending transfer (in this example, it is transfer 3).

Understanding the Mechanism

Each of the 5 transfers is linked so that all of them will succeed or all of them will fail.

The first transfer is the one we actually want to send.

The second transfer sets the Control Account’s balance to the upper bound we want to impose.

The third transfer uses a balancing_debit or balancing_credit to transfer the Destination Account’s net credit balance or net debit balance, respectively, to the Control Account. This transfer will fail if the first transfer would put the Destination Account’s balance above the upper bound.

The third transfer is also a pending transfer, so it won’t actually transfer the Destination Account’s funds, even if it succeeds.

If everything to this point succeeds, the fourth and fifth transfers simply undo the effects of the second and third transfers. The fourth transfer voids the pending transfer. And the fifth transfer resets the Control Account’s net balance to zero.

Correcting Transfers

Transfers in TigerBeetle are immutable, so once they are created they cannot be modified or deleted.

Immutability is useful for creating an auditable log of all of the business events, but it does raise the question of what to do when a transfer was made in error or some detail such as the amount was incorrect.

Always Add More Transfers

Correcting transfers or entries in TigerBeetle are handled with more transfers to reverse or adjust the effects of the previous transfer(s).

This is important because adding transfers as opposed to deleting or modifying incorrect ones adds more information to the history. The log of events includes the original error, when it took place, as well as any attempts to correct the record and when they took place. A correcting entry might even be wrong, in which case it itself can be corrected with yet another transfer. All of these events form a timeline of the particular business event, which is stored permanently.

Another way to put this is that TigerBeetle is the lowest layer of the accounting stack and represents the finest-resolution data that is stored. At a higher-level reporting layer, you can “downsample” the data to show only the corrected transfer event. However, it would not be possible to go back if the original record were modified or deleted.

Two specific recommendations for correcting transfers are:

1. You may want to have a Transfer.code that indicates a given transfer is a correction, or you may want multiple codes where each one represents a different reason why the correction has taken place.
2. If you use the Transfer.user_data_128 to store an ID that links multiple transfers within TigerBeetle or points to a record in an external database, you may want to use the same user_data_128 field on the correction transfer(s), even if they happen at a later point.

Example

Let’s say you had a couple of transfers, from account A to accounts Y and Z:

Now, let’s say we realized the amount was wrong and we need to adjust both of the amounts by 10%. We would submit two additional transfers going in the opposite direction:

Note that the codes used here don’t have any actual meaning, but you would want to enumerate your business events and map each to a numeric code value, including the initial reasons for transfers and the reasons they might be corrected.

Rate Limiting

TigerBeetle can be used to account for non-financial resources.

In this recipe, we will show you how to use it to implement rate limiting using the leaky bucket algorithm based on the user request rate, bandwidth, and money.

Mechanism

For each type of resource we want to limit, we will have a ledger specifically for that resource. On that ledger, we have an operator account and an account for each user. Each user’s account will have a balance limit applied.

To set up the rate limiting system, we will first transfer the resource limit amount to each of the users. For each user request, we will then create a pending transfer with a timeout. We will never post or void these transfers, but will instead let them expire.

Since each account’s “balance” is limited, we cannot create pending transfers that would exceed the rate limit. However, when each pending transfer expires, it automatically replenishes the user’s balance.

Request Rate Limiting

Let’s say we want to limit each user to 10 requests per minute.

We need our user account to have a limited balance.

We’ll first transfer 10 units from the operator to the user.

Then, for each incoming request, we will create a pending transfer for 1 unit back to the operator from the user:

Note that we use a timeout of 60 (seconds), because we wanted to limit each user to 10 requests per minute.

That’s it! Each of these transfers will “reserve” some of the user’s balance and then replenish the balance after they expire.

Bandwidth Limiting

To limit user requests based on bandwidth as opposed to request rate, we can apply the same technique but use amounts that represent the request size.

Let’s say we wanted to limit each user to 10 MB (10,000,000 bytes) per minute.

Our account setup is the same as before:

Now, we’ll transfer 10,000,000 units (bytes in this case) from the operator to the user:

For each incoming request, we’ll create a pending transfer where the amount is equal to the request size:

We’re again using a timeout of 60 seconds, but you could adjust this to be whatever time window you want to use to limit requests.

Transfer Amount Limiting

Now, let’s say you wanted to limit each account to transferring no more than a certain amount of money per time window. We can do that using 2 ledgers and linked transfers.

Let’s say we wanted to limit each account to sending no more than 1000 USD per day.

To set up, we transfer 1000 from the Operator to the User on the Rate Limiting ledger:

For each transfer the user wants to do, we will create 2 transfers that are linked:

Note that we are using a timeout of 86400 seconds, because this is the number of seconds in a day.

These are linked such that if the first transfer fails, because the user has already transferred too much money in the past day, the second transfer will also fail.

Clients

TigerBeetle has official client libraries for the following languages:

• .Net (nuget package).
• Go (package, API docs).
• Java (maven central package, API docs).
• Node (npm package).
• Python (PyPi package).

Subscribe to the tracking issue #2231 to receive notifications about breaking changes!

Please report any client bugs to the main issue tracker.

tigerbeetle-dotnet

The TigerBeetle client for .NET.

Prerequisites

Linux >= 5.6 is the only production environment we support. But for ease of development we also support macOS and Windows.

• .NET >= 8.0.

And if you do not already have NuGet.org as a package source, make sure to add it:

dotnet nuget add source https://api.nuget.org/v3/index.json -n nuget.org

Setup

First, create a directory for your project and cd into the directory.

Then, install the TigerBeetle client:

dotnet new console
dotnet add package tigerbeetle

Now, create Program.cs and copy this into it:

using System;

using TigerBeetle;

// Validate import works.
Console.WriteLine("SUCCESS");

Finally, build and run:

dotnet run

Now that all prerequisites and dependencies are correctly set up, let’s dig into using TigerBeetle.

Sample projects

This document is primarily a reference guide to the client. Below are various sample projects demonstrating features of TigerBeetle.

• Basic: Create two accounts and transfer an amount between them.
• Two-Phase Transfer: Create two accounts and start a pending transfer between them, then post the transfer.
• Many Two-Phase Transfers: Create two accounts and start a number of pending transfer between them, posting and voiding alternating transfers.

Creating a Client

A client is created with a cluster ID and replica addresses for all replicas in the cluster. The cluster ID and replica addresses are both chosen by the system that starts the TigerBeetle cluster.

Clients are thread-safe and a single instance should be shared between multiple concurrent tasks.

Multiple clients are useful when connecting to more than one TigerBeetle cluster.

In this example the cluster ID is 0 and there is one replica. The address is read from the TB_ADDRESS environment variable and defaults to port 3000.

var tbAddress = Environment.GetEnvironmentVariable("TB_ADDRESS");
var clusterID = UInt128.Zero;
var addresses = new[] { tbAddress != null ? tbAddress : "3000" };
using (var client = new Client(clusterID, addresses))
{
    // Use client
}

The Client class is thread-safe and for better performance, a single instance should be shared between multiple concurrent tasks. Multiple clients can be instantiated in case of connecting to more than one TigerBeetle cluster.

The following are valid addresses:

• 3000 (interpreted as 127.0.0.1:3000)
• 127.0.0.1:3000 (interpreted as 127.0.0.1:3000)
• 127.0.0.1 (interpreted as 127.0.0.1:3001, 3001 is the default port)

Creating Accounts

See details for account fields in the Accounts reference.

var accounts = new[] {
    new Account
    {
        Id = ID.Create(), // TigerBeetle time-based ID.
        UserData128 = 0,
        UserData64 = 0,
        UserData32 = 0,
        Ledger = 1,
        Code = 718,
        Flags = AccountFlags.None,
        Timestamp = 0,
    },
};

var accountErrors = client.CreateAccounts(accounts);
// Error handling omitted.

See details for the recommended ID scheme in time-based identifiers.

The UInt128 fields like ID, UserData128, Amount and account balances have a few extension methods to make it easier to convert 128-bit little-endian unsigned integers between BigInteger, byte[], and Guid.

See the class UInt128Extensions for more details.

Account Flags

The account flags value is a bitfield. See details for these flags in the Accounts reference.

To toggle behavior for an account, combine enum values stored in the AccountFlags object with bitwise-or:

• AccountFlags.None
• AccountFlags.Linked
• AccountFlags.DebitsMustNotExceedCredits
• AccountFlags.CreditsMustNotExceedDebits
• AccountFlags.History

For example, to link two accounts where the first account additionally has the debits_must_not_exceed_credits constraint:

var account0 = new Account
{
    Id = 100,
    Ledger = 1,
    Code = 1,
    Flags = AccountFlags.Linked | AccountFlags.DebitsMustNotExceedCredits,
};
var account1 = new Account
{
    Id = 101,
    Ledger = 1,
    Code = 1,
    Flags = AccountFlags.History,
};

var accountErrors = client.CreateAccounts(new[] { account0, account1 });
// Error handling omitted.

Response and Errors

The response is an empty array if all accounts were created successfully. If the response is non-empty, each object in the response array contains error information for an account that failed. The error object contains an error code and the index of the account in the request batch.

See all error conditions in the create_accounts reference.

var account0 = new Account
{
    Id = 102,
    Ledger = 1,
    Code = 1,
    Flags = AccountFlags.None,
};
var account1 = new Account
{
    Id = 103,
    Ledger = 1,
    Code = 1,
    Flags = AccountFlags.None,
};
var account2 = new Account
{
    Id = 104,
    Ledger = 1,
    Code = 1,
    Flags = AccountFlags.None,
};

var accountErrors = client.CreateAccounts(new[] { account0, account1, account2 });
foreach (var error in accountErrors)
{
    switch (error.Result)
    {
        case CreateAccountResult.Exists:
            Console.WriteLine($"Batch account at ${error.Index} already exists.");
            break;
        default:
            Console.WriteLine($"Batch account at ${error.Index} failed to create ${error.Result}");
            break;
    }
    return;
}

Account Lookup

Account lookup is batched, like account creation. Pass in all IDs to fetch. The account for each matched ID is returned.

If no account matches an ID, no object is returned for that account. So the order of accounts in the response is not necessarily the same as the order of IDs in the request. You can refer to the ID field in the response to distinguish accounts.

Account[] accounts = client.LookupAccounts(new UInt128[] { 100, 101 });

Create Transfers

This creates a journal entry between two accounts.

See details for transfer fields in the Transfers reference.

var transfers = new[] {
    new Transfer
    {
        Id = ID.Create(), // TigerBeetle time-based ID.
        DebitAccountId = 102,
        CreditAccountId = 103,
        Amount = 10,
        UserData128 = 0,
        UserData64 = 0,
        UserData32 = 0,
        Timeout = 0,
        Ledger = 1,
        Code = 1,
        Flags = TransferFlags.None,
        Timestamp = 0,
    }
};

var transferErrors = client.CreateTransfers(transfers);
// Error handling omitted.

See details for the recommended ID scheme in time-based identifiers.

Response and Errors

The response is an empty array if all transfers were created successfully. If the response is non-empty, each object in the response array contains error information for a transfer that failed. The error object contains an error code and the index of the transfer in the request batch.

See all error conditions in the create_transfers reference.

var transfers = new[] {
    new Transfer
    {
        Id = 1,
        DebitAccountId = 102,
        CreditAccountId = 103,
        Amount = 10,
        Ledger = 1,
        Code = 1,
        Flags = TransferFlags.None,
    },
    new Transfer
    {
        Id = 2,
        DebitAccountId = 102,
        CreditAccountId = 103,
        Amount = 10,
        Ledger = 1,
        Code = 1,
        Flags = TransferFlags.None,
    },
    new Transfer
    {
        Id = 3,
        DebitAccountId = 102,
        CreditAccountId = 103,
        Amount = 10,
        Ledger = 1,
        Code = 1,
        Flags = TransferFlags.None,
    },
};

var transferErrors = client.CreateTransfers(transfers);
foreach (var error in transferErrors)
{
    switch (error.Result)
    {
        case CreateTransferResult.Exists:
            Console.WriteLine($"Batch transfer at ${error.Index} already exists.");
            break;
        default:
            Console.WriteLine($"Batch transfer at ${error.Index} failed to create: ${error.Result}");
            break;
    }
}

Batching

TigerBeetle performance is maximized when you batch API requests. The client does not do this automatically for you. So, for example, you can insert 1 million transfers one at a time like so:

var batch = new Transfer[] { }; // Array of transfer to create.
foreach (var t in batch)
{
    var transferErrors = client.CreateTransfer(t);
    // Error handling omitted.
}

But the insert rate will be a fraction of potential. Instead, always batch what you can.

The maximum batch size is set in the TigerBeetle server. The default is 8190.

var batch = new Transfer[] { }; // Array of transfer to create.
var BATCH_SIZE = 8190;
for (int firstIndex = 0; firstIndex < batch.Length; firstIndex += BATCH_SIZE)
{
    var lastIndex = firstIndex + BATCH_SIZE;
    if (lastIndex > batch.Length)
    {
        lastIndex = batch.Length;
    }
    var transferErrors = client.CreateTransfers(batch[firstIndex..lastIndex]);
    // Error handling omitted.
}

Queues and Workers

If you are making requests to TigerBeetle from workers pulling jobs from a queue, you can batch requests to TigerBeetle by having the worker act on multiple jobs from the queue at once rather than one at a time. i.e. pulling multiple jobs from the queue rather than just one.

Transfer Flags

The transfer flags value is a bitfield. See details for these flags in the Transfers reference.

To toggle behavior for an account, combine enum values stored in the TransferFlags object with bitwise-or:

• TransferFlags.None
• TransferFlags.Linked
• TransferFlags.Pending
• TransferFlags.PostPendingTransfer
• TransferFlags.VoidPendingTransfer

For example, to link transfer0 and transfer1:

var transfer0 = new Transfer
{
    Id = 4,
    DebitAccountId = 102,
    CreditAccountId = 103,
    Amount = 10,
    Ledger = 1,
    Code = 1,
    Flags = TransferFlags.Linked,
};
var transfer1 = new Transfer
{
    Id = 5,
    DebitAccountId = 102,
    CreditAccountId = 103,
    Amount = 10,
    Ledger = 1,
    Code = 1,
    Flags = TransferFlags.None,
};

var transferErrors = client.CreateTransfers(new[] { transfer0, transfer1 });
// Error handling omitted.

Two-Phase Transfers

Two-phase transfers are supported natively by toggling the appropriate flag. TigerBeetle will then adjust the credits_pending and debits_pending fields of the appropriate accounts. A corresponding post pending transfer then needs to be sent to post or void the transfer.

Post a Pending Transfer

With flags set to post_pending_transfer, TigerBeetle will post the transfer. TigerBeetle will atomically roll back the changes to debits_pending and credits_pending of the appropriate accounts and apply them to the debits_posted and credits_posted balances.

var transfer0 = new Transfer
{
    Id = 6,
    DebitAccountId = 102,
    CreditAccountId = 103,
    Amount = 10,
    Ledger = 1,
    Code = 1,
    Flags = TransferFlags.Pending,
};

var transferErrors = client.CreateTransfers(new[] { transfer0 });
// Error handling omitted.

var transfer1 = new Transfer
{
    Id = 7,
    // Post the entire pending amount.
    Amount = Transfer.AmountMax,
    PendingId = 6,
    Flags = TransferFlags.PostPendingTransfer,
};

transferErrors = client.CreateTransfers(new[] { transfer1 });
// Error handling omitted.

Void a Pending Transfer

In contrast, with flags set to void_pending_transfer, TigerBeetle will void the transfer. TigerBeetle will roll back the changes to debits_pending and credits_pending of the appropriate accounts and not apply them to the debits_posted and credits_posted balances.

var transfer0 = new Transfer
{
    Id = 8,
    DebitAccountId = 102,
    CreditAccountId = 103,
    Amount = 10,
    Ledger = 1,
    Code = 1,
    Flags = TransferFlags.Pending,
};

var transferErrors = client.CreateTransfers(new[] { transfer0 });
// Error handling omitted.

var transfer1 = new Transfer
{
    Id = 9,
    // Post the entire pending amount.
    Amount = 0,
    PendingId = 8,
    Flags = TransferFlags.VoidPendingTransfer,
};

transferErrors = client.CreateTransfers(new[] { transfer1 });
// Error handling omitted.

Transfer Lookup

NOTE: While transfer lookup exists, it is not a flexible query API. We are developing query APIs and there will be new methods for querying transfers in the future.

Transfer lookup is batched, like transfer creation. Pass in all ids to fetch, and matched transfers are returned.

If no transfer matches an id, no object is returned for that transfer. So the order of transfers in the response is not necessarily the same as the order of ids in the request. You can refer to the id field in the response to distinguish transfers.

Transfer[] transfers = client.LookupTransfers(new UInt128[] { 1, 2 });

Get Account Transfers

NOTE: This is a preview API that is subject to breaking changes once we have a stable querying API.

Fetches the transfers involving a given account, allowing basic filter and pagination capabilities.

The transfers in the response are sorted by timestamp in chronological or reverse-chronological order.

var filter = new AccountFilter
{
    AccountId = 101,
    UserData128 = 0, // No filter by UserData.
    UserData64 = 0,
    UserData32 = 0,
    Code = 0, // No filter by Code.
    TimestampMin = 0, // No filter by Timestamp.
    TimestampMax = 0, // No filter by Timestamp.
    Limit = 10, // Limit to ten transfers at most.
    Flags = AccountFilterFlags.Debits | // Include transfer from the debit side.
        AccountFilterFlags.Credits | // Include transfer from the credit side.
        AccountFilterFlags.Reversed, // Sort by timestamp in reverse-chronological order.
};

Transfer[] transfers = client.GetAccountTransfers(filter);

Get Account Balances

NOTE: This is a preview API that is subject to breaking changes once we have a stable querying API.

Fetches the point-in-time balances of a given account, allowing basic filter and pagination capabilities.

Only accounts created with the flag history set retain historical balances.

The balances in the response are sorted by timestamp in chronological or reverse-chronological order.

var filter = new AccountFilter
{
    AccountId = 101,
    UserData128 = 0, // No filter by UserData.
    UserData64 = 0,
    UserData32 = 0,
    Code = 0, // No filter by Code.
    TimestampMin = 0, // No filter by Timestamp.
    TimestampMax = 0, // No filter by Timestamp.
    Limit = 10, // Limit to ten balances at most.
    Flags = AccountFilterFlags.Debits | // Include transfer from the debit side.
        AccountFilterFlags.Credits | // Include transfer from the credit side.
        AccountFilterFlags.Reversed, // Sort by timestamp in reverse-chronological order.
};

AccountBalance[] accountBalances = client.GetAccountBalances(filter);

Query Accounts

NOTE: This is a preview API that is subject to breaking changes once we have a stable querying API.

Query accounts by the intersection of some fields and by timestamp range.

The accounts in the response are sorted by timestamp in chronological or reverse-chronological order.

var filter = new QueryFilter
{
    UserData128 = 1000, // Filter by UserData.
    UserData64 = 100,
    UserData32 = 10,
    Code = 1, // Filter by Code.
    Ledger = 0, // No filter by Ledger.
    TimestampMin = 0, // No filter by Timestamp.
    TimestampMax = 0, // No filter by Timestamp.
    Limit = 10, // Limit to ten balances at most.
    Flags = QueryFilterFlags.Reversed, // Sort by timestamp in reverse-chronological order.
};

Account[] accounts = client.QueryAccounts(filter);

Query Transfers

NOTE: This is a preview API that is subject to breaking changes once we have a stable querying API.

Query transfers by the intersection of some fields and by timestamp range.

The transfers in the response are sorted by timestamp in chronological or reverse-chronological order.

var filter = new QueryFilter
{
    UserData128 = 1000, // Filter by UserData
    UserData64 = 100,
    UserData32 = 10,
    Code = 1, // Filter by Code
    Ledger = 0, // No filter by Ledger
    TimestampMin = 0, // No filter by Timestamp.
    TimestampMax = 0, // No filter by Timestamp.
    Limit = 10, // Limit to ten balances at most.
    Flags = QueryFilterFlags.Reversed, // Sort by timestamp in reverse-chronological order.
};

Transfer[] transfers = client.QueryTransfers(filter);

Linked Events

When the linked flag is specified for an account when creating accounts or a transfer when creating transfers, it links that event with the next event in the batch, to create a chain of events, of arbitrary length, which all succeed or fail together. The tail of a chain is denoted by the first event without this flag. The last event in a batch may therefore never have the linked flag set as this would leave a chain open-ended. Multiple chains or individual events may coexist within a batch to succeed or fail independently.

Events within a chain are executed within order, or are rolled back on error, so that the effect of each event in the chain is visible to the next, and so that the chain is either visible or invisible as a unit to subsequent events after the chain. The event that was the first to break the chain will have a unique error result. Other events in the chain will have their error result set to linked_event_failed.

var batch = new System.Collections.Generic.List<Transfer>();

// An individual transfer (successful):
batch.Add(new Transfer { Id = 1, /* ... rest of transfer ... */ });

// A chain of 4 transfers (the last transfer in the chain closes the chain with linked=false):
batch.Add(new Transfer { Id = 2, /* ... rest of transfer ... */ Flags = TransferFlags.Linked }); // Commit/rollback.
batch.Add(new Transfer { Id = 3, /* ... rest of transfer ... */ Flags = TransferFlags.Linked }); // Commit/rollback.
batch.Add(new Transfer { Id = 2, /* ... rest of transfer ... */ Flags = TransferFlags.Linked }); // Fail with exists
batch.Add(new Transfer { Id = 4, /* ... rest of transfer ... */ }); // Fail without committing

// An individual transfer (successful):
// This should not see any effect from the failed chain above.
batch.Add(new Transfer { Id = 2, /* ... rest of transfer ... */ });

// A chain of 2 transfers (the first transfer fails the chain):
batch.Add(new Transfer { Id = 2, /* ... rest of transfer ... */ Flags = TransferFlags.Linked });
batch.Add(new Transfer { Id = 3, /* ... rest of transfer ... */ });

// A chain of 2 transfers (successful):
batch.Add(new Transfer { Id = 3, /* ... rest of transfer ... */ Flags = TransferFlags.Linked });
batch.Add(new Transfer { Id = 4, /* ... rest of transfer ... */ });

var transferErrors = client.CreateTransfers(batch.ToArray());
// Error handling omitted.

Imported Events

When the imported flag is specified for an account when creating accounts or a transfer when creating transfers, it allows importing historical events with a user-defined timestamp.

The entire batch of events must be set with the flag imported.

It’s recommended to submit the whole batch as a linked chain of events, ensuring that if any event fails, none of them are committed, preserving the last timestamp unchanged. This approach gives the application a chance to correct failed imported events, re-submitting the batch again with the same user-defined timestamps.

// External source of time
ulong historicalTimestamp = 0UL;
var historicalAccounts = new Account[] { /* Loaded from an external source */ };
var historicalTransfers = new Transfer[] { /* Loaded from an external source */ };

// First, load and import all accounts with their timestamps from the historical source.
var accountsBatch = new System.Collections.Generic.List<Account>();
for (var index = 0; index < historicalAccounts.Length; index++)
{
    var account = historicalAccounts[index];

    // Set a unique and strictly increasing timestamp.
    historicalTimestamp += 1;
    account.Timestamp = historicalTimestamp;
    // Set the account as `imported`.
    account.Flags = AccountFlags.Imported;
    // To ensure atomicity, the entire batch (except the last event in the chain)
    // must be `linked`.
    if (index < historicalAccounts.Length - 1)
    {
        account.Flags |= AccountFlags.Linked;
    }

    accountsBatch.Add(account);
}

var accountErrors = client.CreateAccounts(accountsBatch.ToArray());
// Error handling omitted.

// Then, load and import all transfers with their timestamps from the historical source.
var transfersBatch = new System.Collections.Generic.List<Transfer>();
for (var index = 0; index < historicalTransfers.Length; index++)
{
    var transfer = historicalTransfers[index];

    // Set a unique and strictly increasing timestamp.
    historicalTimestamp += 1;
    transfer.Timestamp = historicalTimestamp;
    // Set the account as `imported`.
    transfer.Flags = TransferFlags.Imported;
    // To ensure atomicity, the entire batch (except the last event in the chain)
    // must be `linked`.
    if (index < historicalTransfers.Length - 1)
    {
        transfer.Flags |= TransferFlags.Linked;
    }

    transfersBatch.Add(transfer);
}

var transferErrors = client.CreateTransfers(transfersBatch.ToArray());
// Error handling omitted.
// Since it is a linked chain, in case of any error the entire batch is rolled back and can be retried
// with the same historical timestamps without regressing the cluster timestamp.

tigerbeetle-go

The TigerBeetle client for Go.

Make sure to import github.com/tigerbeetle/tigerbeetle-go, not this repo and subdirectory.

Prerequisites

Linux >= 5.6 is the only production environment we support. But for ease of development we also support macOS and Windows.

• Go >= 1.21

Additionally on Windows: you must install Zig 0.13.0 and set the CC environment variable to zig.exe cc. Use the full path for zig.exe.

Setup

First, create a directory for your project and cd into the directory.

Then, install the TigerBeetle client:

go mod init tbtest
go get github.com/tigerbeetle/tigerbeetle-go

Now, create main.go and copy this into it:

package main

import (
    "fmt"
    "log"
    "os"

    . "github.com/tigerbeetle/tigerbeetle-go"
    . "github.com/tigerbeetle/tigerbeetle-go/pkg/types"
)

func main() {
    fmt.Println("Import ok!")
}

Finally, build and run:

go run main.go

Now that all prerequisites and dependencies are correctly set up, let’s dig into using TigerBeetle.

Sample projects

This document is primarily a reference guide to the client. Below are various sample projects demonstrating features of TigerBeetle.

• Basic: Create two accounts and transfer an amount between them.
• Two-Phase Transfer: Create two accounts and start a pending transfer between them, then post the transfer.
• Many Two-Phase Transfers: Create two accounts and start a number of pending transfer between them, posting and voiding alternating transfers.

Creating a Client

A client is created with a cluster ID and replica addresses for all replicas in the cluster. The cluster ID and replica addresses are both chosen by the system that starts the TigerBeetle cluster.

Clients are thread-safe and a single instance should be shared between multiple concurrent tasks.

Multiple clients are useful when connecting to more than one TigerBeetle cluster.

In this example the cluster ID is 0 and there is one replica. The address is read from the TB_ADDRESS environment variable and defaults to port 3000.

tbAddress := os.Getenv("TB_ADDRESS")
if len(tbAddress) == 0 {
    tbAddress = "3000"
}
client, err := NewClient(ToUint128(0), []string{tbAddress})
if err != nil {
    log.Printf("Error creating client: %s", err)
    return
}
defer client.Close()

The following are valid addresses:

• 3000 (interpreted as 127.0.0.1:3000)
• 127.0.0.1:3000 (interpreted as 127.0.0.1:3000)
• 127.0.0.1 (interpreted as 127.0.0.1:3001, 3001 is the default port)

Creating Accounts

See details for account fields in the Accounts reference.

accountErrors, err := client.CreateAccounts([]Account{
    {
        ID:          ID(), // TigerBeetle time-based ID.
        UserData128: ToUint128(0),
        UserData64:  0,
        UserData32:  0,
        Ledger:      1,
        Code:        718,
        Flags:       0,
        Timestamp:   0,
    },
})
// Error handling omitted.

See details for the recommended ID scheme in time-based identifiers.

The Uint128 fields like ID, UserData128, Amount and account balances have a few helper functions to make it easier to convert 128-bit little-endian unsigned integers between string, math/big.Int, and []byte.

See the type Uint128 for more details.

Account Flags

The account flags value is a bitfield. See details for these flags in the Accounts reference.

To toggle behavior for an account, use the types.AccountFlags struct to combine enum values and generate a uint16. Here are a few examples:

• AccountFlags{Linked: true}.ToUint16()
• AccountFlags{DebitsMustNotExceedCredits: true}.ToUint16()
• AccountFlags{CreditsMustNotExceedDebits: true}.ToUint16()
• AccountFlags{History: true}.ToUint16()

For example, to link two accounts where the first account additionally has the debits_must_not_exceed_credits constraint:

account0 := Account{
    ID:     ToUint128(100),
    Ledger: 1,
    Code:   718,
    Flags: AccountFlags{
        DebitsMustNotExceedCredits: true,
        Linked:                     true,
    }.ToUint16(),
}
account1 := Account{
    ID:     ToUint128(101),
    Ledger: 1,
    Code:   718,
    Flags: AccountFlags{
        History: true,
    }.ToUint16(),
}

accountErrors, err := client.CreateAccounts([]Account{account0, account1})
// Error handling omitted.

Response and Errors

The response is an empty array if all accounts were created successfully. If the response is non-empty, each object in the response array contains error information for an account that failed. The error object contains an error code and the index of the account in the request batch.

See all error conditions in the create_accounts reference.

account0 := Account{
    ID:     ToUint128(102),
    Ledger: 1,
    Code:   718,
    Flags:  0,
}
account1 := Account{
    ID:     ToUint128(103),
    Ledger: 1,
    Code:   718,
    Flags:  0,
}
account2 := Account{
    ID:     ToUint128(104),
    Ledger: 1,
    Code:   718,
    Flags:  0,
}

accountErrors, err := client.CreateAccounts([]Account{account0, account1, account2})
if err != nil {
    log.Printf("Error creating accounts: %s", err)
    return
}

for _, err := range accountErrors {
    switch err.Index {
    case uint32(AccountExists):
        log.Printf("Batch account at %d already exists.", err.Index)
    default:
        log.Printf("Batch account at %d failed to create: %s", err.Index, err.Result)
    }
}

To handle errors you can either 1) exactly match error codes returned from client.createAccounts with enum values in the CreateAccountError object, or you can 2) look up the error code in the CreateAccountError object for a human-readable string.

Account Lookup

Account lookup is batched, like account creation. Pass in all IDs to fetch. The account for each matched ID is returned.

If no account matches an ID, no object is returned for that account. So the order of accounts in the response is not necessarily the same as the order of IDs in the request. You can refer to the ID field in the response to distinguish accounts.

accounts, err := client.LookupAccounts([]Uint128{ToUint128(100), ToUint128(101)})

Create Transfers

This creates a journal entry between two accounts.

See details for transfer fields in the Transfers reference.

transfers := []Transfer{{
    ID:              ID(), // TigerBeetle time-based ID.
    DebitAccountID:  ToUint128(101),
    CreditAccountID: ToUint128(102),
    Amount:          ToUint128(10),
    Ledger:          1,
    Code:            1,
    Flags:           0,
    Timestamp:       0,
}}

transferErrors, err := client.CreateTransfers(transfers)
// Error handling omitted.

See details for the recommended ID scheme in time-based identifiers.

Response and Errors

The response is an empty array if all transfers were created successfully. If the response is non-empty, each object in the response array contains error information for a transfer that failed. The error object contains an error code and the index of the transfer in the request batch.

See all error conditions in the create_transfers reference.

transfers := []Transfer{{
    ID:              ToUint128(1),
    DebitAccountID:  ToUint128(101),
    CreditAccountID: ToUint128(102),
    Amount:          ToUint128(10),
    Ledger:          1,
    Code:            1,
    Flags:           0,
}, {
    ID:              ToUint128(2),
    DebitAccountID:  ToUint128(101),
    CreditAccountID: ToUint128(102),
    Amount:          ToUint128(10),
    Ledger:          1,
    Code:            1,
    Flags:           0,
}, {
    ID:              ToUint128(3),
    DebitAccountID:  ToUint128(101),
    CreditAccountID: ToUint128(102),
    Amount:          ToUint128(10),
    Ledger:          1,
    Code:            1,
    Flags:           0,
}}

transferErrors, err := client.CreateTransfers(transfers)
if err != nil {
    log.Printf("Error creating transfers: %s", err)
    return
}

for _, err := range transferErrors {
    switch err.Index {
    case uint32(TransferExists):
        log.Printf("Batch transfer at %d already exists.", err.Index)
    default:
        log.Printf("Batch transfer at %d failed to create: %s", err.Index, err.Result)
    }
}

Batching

TigerBeetle performance is maximized when you batch API requests. The client does not do this automatically for you. So, for example, you can insert 1 million transfers one at a time like so:

batch := []Transfer{}
for i := 0; i < len(batch); i++ {
    transferErrors, err := client.CreateTransfers([]Transfer{batch[i]})
    _, _ = transferErrors, err // Error handling omitted.
}

But the insert rate will be a fraction of potential. Instead, always batch what you can.

The maximum batch size is set in the TigerBeetle server. The default is 8190.

batch := []Transfer{}
BATCH_SIZE := 8190
for i := 0; i < len(batch); i += BATCH_SIZE {
    size := BATCH_SIZE
    if i+BATCH_SIZE > len(batch) {
        size = len(batch) - i
    }
    transferErrors, err := client.CreateTransfers(batch[i : i+size])
    _, _ = transferErrors, err // Error handling omitted.
}

Queues and Workers

If you are making requests to TigerBeetle from workers pulling jobs from a queue, you can batch requests to TigerBeetle by having the worker act on multiple jobs from the queue at once rather than one at a time. i.e. pulling multiple jobs from the queue rather than just one.

Transfer Flags

The transfer flags value is a bitfield. See details for these flags in the Transfers reference.

To toggle behavior for an account, use the types.TransferFlags struct to combine enum values and generate a uint16. Here are a few examples:

• TransferFlags{Linked: true}.ToUint16()
• TransferFlags{Pending: true}.ToUint16()
• TransferFlags{PostPendingTransfer: true}.ToUint16()
• TransferFlags{VoidPendingTransfer: true}.ToUint16()

For example, to link transfer0 and transfer1:

transfer0 := Transfer{
    ID:              ToUint128(4),
    DebitAccountID:  ToUint128(101),
    CreditAccountID: ToUint128(102),
    Amount:          ToUint128(10),
    Ledger:          1,
    Code:            1,
    Flags:           TransferFlags{Linked: true}.ToUint16(),
}
transfer1 := Transfer{
    ID:              ToUint128(5),
    DebitAccountID:  ToUint128(101),
    CreditAccountID: ToUint128(102),
    Amount:          ToUint128(10),
    Ledger:          1,
    Code:            1,
    Flags:           0,
}

transferErrors, err := client.CreateTransfers([]Transfer{transfer0, transfer1})
// Error handling omitted.

Two-Phase Transfers

Two-phase transfers are supported natively by toggling the appropriate flag. TigerBeetle will then adjust the credits_pending and debits_pending fields of the appropriate accounts. A corresponding post pending transfer then needs to be sent to post or void the transfer.

Post a Pending Transfer

With flags set to post_pending_transfer, TigerBeetle will post the transfer. TigerBeetle will atomically roll back the changes to debits_pending and credits_pending of the appropriate accounts and apply them to the debits_posted and credits_posted balances.

transfer0 := Transfer{
    ID:              ToUint128(6),
    DebitAccountID:  ToUint128(101),
    CreditAccountID: ToUint128(102),
    Amount:          ToUint128(10),
    Ledger:          1,
    Code:            1,
    Flags:           0,
}

transferErrors, err := client.CreateTransfers([]Transfer{transfer0})
// Error handling omitted.

transfer1 := Transfer{
    ID: ToUint128(7),
    // Post the entire pending amount.
    Amount:    AmountMax,
    PendingID: ToUint128(6),
    Flags:     TransferFlags{PostPendingTransfer: true}.ToUint16(),
}

transferErrors, err = client.CreateTransfers([]Transfer{transfer1})
// Error handling omitted.

Void a Pending Transfer

In contrast, with flags set to void_pending_transfer, TigerBeetle will void the transfer. TigerBeetle will roll back the changes to debits_pending and credits_pending of the appropriate accounts and not apply them to the debits_posted and credits_posted balances.

transfer0 := Transfer{
    ID:              ToUint128(8),
    DebitAccountID:  ToUint128(101),
    CreditAccountID: ToUint128(102),
    Amount:          ToUint128(10),
    Timeout:         0,
    Ledger:          1,
    Code:            1,
    Flags:           0,
}

transferErrors, err := client.CreateTransfers([]Transfer{transfer0})
// Error handling omitted.

transfer1 := Transfer{
    ID: ToUint128(9),
    // Post the entire pending amount.
    Amount:    ToUint128(0),
    PendingID: ToUint128(8),
    Flags:     TransferFlags{VoidPendingTransfer: true}.ToUint16(),
}

transferErrors, err = client.CreateTransfers([]Transfer{transfer1})
// Error handling omitted.

Transfer Lookup

NOTE: While transfer lookup exists, it is not a flexible query API. We are developing query APIs and there will be new methods for querying transfers in the future.

Transfer lookup is batched, like transfer creation. Pass in all ids to fetch, and matched transfers are returned.

If no transfer matches an id, no object is returned for that transfer. So the order of transfers in the response is not necessarily the same as the order of ids in the request. You can refer to the id field in the response to distinguish transfers.

transfers, err := client.LookupTransfers([]Uint128{ToUint128(1), ToUint128(2)})

Get Account Transfers

NOTE: This is a preview API that is subject to breaking changes once we have a stable querying API.

Fetches the transfers involving a given account, allowing basic filter and pagination capabilities.

The transfers in the response are sorted by timestamp in chronological or reverse-chronological order.

filter := AccountFilter{
    AccountID:    ToUint128(2),
    UserData128:  ToUint128(0), // No filter by UserData.
    UserData64:   0,
    UserData32:   0,
    Code:         0,  // No filter by Code.
    TimestampMin: 0,  // No filter by Timestamp.
    TimestampMax: 0,  // No filter by Timestamp.
    Limit:        10, // Limit to ten transfers at most.
    Flags: AccountFilterFlags{
        Debits:   true, // Include transfer from the debit side.
        Credits:  true, // Include transfer from the credit side.
        Reversed: true, // Sort by timestamp in reverse-chronological order.
    }.ToUint32(),
}

transfers, err := client.GetAccountTransfers(filter)

Get Account Balances

NOTE: This is a preview API that is subject to breaking changes once we have a stable querying API.

Fetches the point-in-time balances of a given account, allowing basic filter and pagination capabilities.

Only accounts created with the flag history set retain historical balances.

The balances in the response are sorted by timestamp in chronological or reverse-chronological order.

filter := AccountFilter{
    AccountID:    ToUint128(2),
    UserData128:  ToUint128(0), // No filter by UserData.
    UserData64:   0,
    UserData32:   0,
    Code:         0,  // No filter by Code.
    TimestampMin: 0,  // No filter by Timestamp.
    TimestampMax: 0,  // No filter by Timestamp.
    Limit:        10, // Limit to ten balances at most.
    Flags: AccountFilterFlags{
        Debits:   true, // Include transfer from the debit side.
        Credits:  true, // Include transfer from the credit side.
        Reversed: true, // Sort by timestamp in reverse-chronological order.
    }.ToUint32(),
}

account_balances, err := client.GetAccountBalances(filter)

Query Accounts

NOTE: This is a preview API that is subject to breaking changes once we have a stable querying API.

Query accounts by the intersection of some fields and by timestamp range.

The accounts in the response are sorted by timestamp in chronological or reverse-chronological order.

filter := QueryFilter{
    UserData128:  ToUint128(1000), // Filter by UserData
    UserData64:   100,
    UserData32:   10,
    Code:         1,  // Filter by Code
    Ledger:       0,  // No filter by Ledger
    TimestampMin: 0,  // No filter by Timestamp.
    TimestampMax: 0,  // No filter by Timestamp.
    Limit:        10, // Limit to ten balances at most.
    Flags: QueryFilterFlags{
        Reversed: true, // Sort by timestamp in reverse-chronological order.
    }.ToUint32(),
}

accounts, err := client.QueryAccounts(filter)

Query Transfers

NOTE: This is a preview API that is subject to breaking changes once we have a stable querying API.

Query transfers by the intersection of some fields and by timestamp range.

The transfers in the response are sorted by timestamp in chronological or reverse-chronological order.

filter := QueryFilter{
    UserData128:  ToUint128(1000), // Filter by UserData.
    UserData64:   100,
    UserData32:   10,
    Code:         1,  // Filter by Code.
    Ledger:       0,  // No filter by Ledger.
    TimestampMin: 0,  // No filter by Timestamp.
    TimestampMax: 0,  // No filter by Timestamp.
    Limit:        10, // Limit to ten balances at most.
    Flags: QueryFilterFlags{
        Reversed: true, // Sort by timestamp in reverse-chronological order.
    }.ToUint32(),
}

transfers, err := client.QueryTransfers(filter)

Linked Events

When the linked flag is specified for an account when creating accounts or a transfer when creating transfers, it links that event with the next event in the batch, to create a chain of events, of arbitrary length, which all succeed or fail together. The tail of a chain is denoted by the first event without this flag. The last event in a batch may therefore never have the linked flag set as this would leave a chain open-ended. Multiple chains or individual events may coexist within a batch to succeed or fail independently.

Events within a chain are executed within order, or are rolled back on error, so that the effect of each event in the chain is visible to the next, and so that the chain is either visible or invisible as a unit to subsequent events after the chain. The event that was the first to break the chain will have a unique error result. Other events in the chain will have their error result set to linked_event_failed.

batch := []Transfer{}
linkedFlag := TransferFlags{Linked: true}.ToUint16()

// An individual transfer (successful):
batch = append(batch, Transfer{ID: ToUint128(1) /* ... rest of transfer ... */})

// A chain of 4 transfers (the last transfer in the chain closes the chain with linked=false):
batch = append(batch, Transfer{ID: ToUint128(2) /* ... , */, Flags: linkedFlag}) // Commit/rollback.
batch = append(batch, Transfer{ID: ToUint128(3) /* ... , */, Flags: linkedFlag}) // Commit/rollback.
batch = append(batch, Transfer{ID: ToUint128(2) /* ... , */, Flags: linkedFlag}) // Fail with exists
batch = append(batch, Transfer{ID: ToUint128(4) /* ... , */})                    // Fail without committing

// An individual transfer (successful):
// This should not see any effect from the failed chain above.
batch = append(batch, Transfer{ID: ToUint128(2) /* ... rest of transfer ... */})

// A chain of 2 transfers (the first transfer fails the chain):
batch = append(batch, Transfer{ID: ToUint128(2) /* ... rest of transfer ... */, Flags: linkedFlag})
batch = append(batch, Transfer{ID: ToUint128(3) /* ... rest of transfer ... */})

// A chain of 2 transfers (successful):
batch = append(batch, Transfer{ID: ToUint128(3) /* ... rest of transfer ... */, Flags: linkedFlag})
batch = append(batch, Transfer{ID: ToUint128(4) /* ... rest of transfer ... */})

transferErrors, err := client.CreateTransfers(batch)
// Error handling omitted.