GitHub

tigerbeetle

The TigerBeetle client for Ruby.

Important

This gem changed ownership from Anthony D to TigerBeetle. If you’re upgrading from a 0.0.x version, please consult the migration guide for the necessary code changes.

Prerequisites

Linux >= 5.6 is the only production environment we support. But for ease of development we also support macOS and Windows.

  • Ruby >= 3.3

Setup

First, create a directory for your project and cd into the directory.

Then, install the TigerBeetle client:

gem install tigerbeetle

Now, create main.rb and copy this into it:

require "tigerbeetle"

puts("Import OK!")

Finally, build and run:

ruby main.rb

Now that all prerequisites and dependencies are correctly set up, let’s dig into using TigerBeetle.

Sample projects

This document is primarily a reference guide to the client. Below are various sample projects demonstrating features of TigerBeetle.

  • Basic: Create two accounts and transfer an amount between them.
  • Two-Phase Transfer: Create two accounts and start a pending transfer between them, then post the transfer.
  • Many Two-Phase Transfers: Create two accounts and start a number of pending transfers between them, posting and voiding alternating transfers.

Creating a Client

A client is created with a cluster ID and replica addresses for all replicas in the cluster. The cluster ID and replica addresses are both chosen by the system that starts the TigerBeetle cluster.

Clients are thread-safe and a single instance should be shared between multiple concurrent tasks. This allows events to be automatically batched.

Multiple clients are useful when connecting to more than one TigerBeetle cluster.

In this example the cluster ID is 0 and there is one replica. The address is read from the TB_ADDRESS environment variable and defaults to port 3000.

replica_addresses = ENV.fetch("TB_ADDRESS", "3000")

TigerBeetle::Client.open(0, replica_addresses:) do |client|
  # Use the client.
end

The gem also provides an optional top-level alias. Require tigerbeetle/tb to use TB as a shorthand for TigerBeetle:

require "tigerbeetle/tb"

account = TB::Account.new(TB.id, 1, 1)

The alias is opt-in and is not defined by require "tigerbeetle".

The TigerBeetle::Client is fiber-scheduler aware, so it works with e.g. the async gem without requiring code changes.

require "async"
require "async/semaphore"
require "tigerbeetle"

semaphore = Async::Semaphore.new(16)

account_batches = Array.new(16) do
  Array.new(1_000) do
    TigerBeetle::Account.new(TigerBeetle.id, 1, 1)
  end
end

TigerBeetle::Client.open(0, "3000") do |client|
  Async do
    account_batches
      .map { |batch| semaphore.async { client.create_accounts(batch) } }
      .each(&)
  end
end

The following are valid addresses:

  • 3000 (interpreted as 127.0.0.1:3000)
  • 127.0.0.1:3000 (interpreted as 127.0.0.1:3000)
  • 127.0.0.1 (interpreted as 127.0.0.1:3001, 3001 is the default port)

Creating Accounts

See details for account fields in the Accounts reference.

account = TigerBeetle::Account.new(
  TigerBeetle.id,
  0,
  0,
  0,
  0,
  0,
  0,
  0,
  1,
  718,
  TigerBeetle::AccountFlags::NONE,
  0
)

account_results = client.create_accounts([account])
# Results handling omitted.

See details for the recommended ID scheme in time-based identifiers.

Account Flags

The account flags value is a bitfield. See details for these flags in the Accounts reference.

To toggle behavior for an account, combine constants from the TigerBeetle::AccountFlags module with bitwise-or:

  • AccountFlags::LINKED
  • AccountFlags::DEBITS_MUST_NOT_EXCEED_CREDITS
  • AccountFlags::CREDITS_MUST_NOT_EXCEED_DEBITS
  • AccountFlags::HISTORY

For example, to link two accounts where the first account additionally has the debits_must_not_exceed_credits constraint:

account0 = TigerBeetle::Account.new(
  TigerBeetle.id,
  1,
  1,
  TigerBeetle::AccountFlags::LINKED |
    TigerBeetle::AccountFlags::DEBITS_MUST_NOT_EXCEED_CREDITS
)
account1 = TigerBeetle::Account.new(
  TigerBeetle.id,
  1,
  1,
  TigerBeetle::AccountFlags::HISTORY
)

account_results = client.create_accounts([account0, account1])
# Results handling omitted.

Response and Errors

The response is an array containing the status code and the timestamp of each account in the request batch:

  • Successfully created accounts with the status created return the timestamp assigned to the Account object.
  • Already existing accounts with the result exists return the timestamp of the original existing object.
  • Failed accounts return the status code along with the timestamp when the validation occurred. See all error conditions in the create_accounts reference.
accounts = [
  TigerBeetle::Account.new(TigerBeetle.id, 1, 1),
  TigerBeetle::Account.new(TigerBeetle.id, 1, 1),
  TigerBeetle::Account.new(TigerBeetle.id, 1, 1)
]

account_results = client.create_accounts(accounts)
account_results.each_with_index do |result, index|
  case result.status
  when TigerBeetle::CreateAccountStatus::CREATED
    puts("Batch account at #{index} successfully created with timestamp #{result.timestamp}.")
  when TigerBeetle::CreateAccountStatus::EXISTS
    puts("Batch account at #{index} already exists with timestamp #{result.timestamp}.")
  else
    puts("Batch account at #{index} failed to create: #{result.status}.")
  end
end

To handle errors you can compare the result status returned from client.create_accounts with constants in the TigerBeetle::CreateAccountStatus module.

Account Lookup

Account lookup is batched, like account creation. Pass in all IDs to fetch. The account for each matched ID is returned.

If no account matches an ID, no object is returned for that account. So the order of accounts in the response is not necessarily the same as the order of IDs in the request. You can refer to the ID field in the response to distinguish accounts.

accounts = client.lookup_accounts([account0.id, account1.id])

Create Transfers

This creates a journal entry between two accounts.

See details for transfer fields in the Transfers reference.

transfers = [
  TigerBeetle::Transfer.new(
    TigerBeetle.id,
    transfer_debit_account_id,
    transfer_credit_account_id,
    10,
    0,
    0,
    0,
    0,
    0,
    1,
    720,
    TigerBeetle::TransferFlags::NONE,
    0
  )
]

transfer_results = client.create_transfers(transfers)
# Results handling omitted.

See details for the recommended ID scheme in time-based identifiers.

Response and Errors

The response is an array containing the status code and the timestamp of each transfer in the request batch:

  • Successfully created transfers with the result created return the timestamp assigned to the Transfer object.
  • Already existing transfers with the result exists return the timestamp of the original existing object.
  • Failed transfers return the status code along with the timestamp when the validation occurred. See all error conditions in the create_transfers reference.
batch = [
  TigerBeetle::Transfer.new(
    TigerBeetle.id,
    transfer_debit_account_id,
    transfer_credit_account_id,
    10,
    1,
    720
  ),
  TigerBeetle::Transfer.new(
    TigerBeetle.id,
    transfer_debit_account_id,
    transfer_credit_account_id,
    10,
    1,
    720
  ),
  TigerBeetle::Transfer.new(
    TigerBeetle.id,
    transfer_debit_account_id,
    transfer_credit_account_id,
    10,
    1,
    720
  )
]

transfer_results = client.create_transfers(batch)
transfer_results.each_with_index do |result, index|
  case result.status
  when TigerBeetle::CreateTransferStatus::CREATED
    puts("Batch transfer at #{index} successfully created with timestamp #{result.timestamp}.")
  when TigerBeetle::CreateTransferStatus::EXISTS
    puts("Batch transfer at #{index} already exists with timestamp #{result.timestamp}.")
  else
    puts("Batch transfer at #{index} failed to create: #{result.status}.")
  end
end

To handle errors you can compare the result status returned from client.create_transfers with constants in the TigerBeetle::CreateTransferStatus module.

Batching

TigerBeetle performance is maximized when you batch API requests.

A client instance shared across multiple threads/tasks can automatically batch concurrent requests, but the application must still send as many events as possible in a single call.

For example, if you insert 1 million transfers sequentially, one at a time, the insert rate will be a fraction of the potential, because the client will wait for a reply between each one. Instead, always batch as much as you can.

The maximum batch size is set in the TigerBeetle server. The default is 8189.

# Array of transfers to create.
batch = []
batch.each_slice(8189) do |slice|
  transfer_results = client.create_transfers(slice)
  # Results handling omitted.
end

Queues and Workers

If you are making requests to TigerBeetle from workers pulling jobs from a queue, you can batch requests to TigerBeetle by having the worker act on multiple jobs from the queue at once rather than one at a time. i.e. pulling multiple jobs from the queue rather than just one.

Transfer Flags

The transfer flags value is a bitfield. See details for these flags in the Transfers reference.

To toggle behavior for a transfer, combine constants from the TigerBeetle::TransferFlags module with bitwise-or:

  • TransferFlags::LINKED
  • TransferFlags::PENDING
  • TransferFlags::POST_PENDING_TRANSFER
  • TransferFlags::VOID_PENDING_TRANSFER

For example, to link transfer0 and transfer1:

transfer0 = TigerBeetle::Transfer.new(
  TigerBeetle.id,
  transfer_debit_account_id,
  transfer_credit_account_id,
  10,
  1,
  720,
  TigerBeetle::TransferFlags::LINKED
)
transfer1 = TigerBeetle::Transfer.new(
  TigerBeetle.id,
  transfer_debit_account_id,
  transfer_credit_account_id,
  10,
  1,
  720
)

transfer_results = client.create_transfers([transfer0, transfer1])
# Results handling omitted.

Two-Phase Transfers

Two-phase transfers are supported natively by toggling the appropriate flag. TigerBeetle will then adjust the credits_pending and debits_pending fields of the appropriate accounts. A corresponding post pending transfer then needs to be sent to post or void the transfer.

Post a Pending Transfer

With flags set to post_pending_transfer, TigerBeetle will post the transfer. TigerBeetle will atomically roll back the changes to debits_pending and credits_pending of the appropriate accounts and apply them to the debits_posted and credits_posted balances.

pending_transfer = TigerBeetle::Transfer.new(
  TigerBeetle.id,
  transfer_debit_account_id,
  transfer_credit_account_id,
  10,
  1,
  720,
  TigerBeetle::TransferFlags::PENDING
)

transfer_results = client.create_transfers([pending_transfer])
# Results handling omitted.

post_transfer = TigerBeetle::Transfer.new(
  TigerBeetle.id,
  transfer_debit_account_id,
  transfer_credit_account_id,
  10,
  pending_transfer.id,
  1,
  720,
  TigerBeetle::TransferFlags::POST_PENDING_TRANSFER
)

transfer_results = client.create_transfers([post_transfer])
# Results handling omitted.

Void a Pending Transfer

In contrast, with flags set to void_pending_transfer, TigerBeetle will void the transfer. TigerBeetle will roll back the changes to debits_pending and credits_pending of the appropriate accounts and not apply them to the debits_posted and credits_posted balances.

pending_transfer = TigerBeetle::Transfer.new(
  TigerBeetle.id,
  transfer_debit_account_id,
  transfer_credit_account_id,
  10,
  1,
  720,
  TigerBeetle::TransferFlags::PENDING
)

transfer_results = client.create_transfers([pending_transfer])
# Results handling omitted.

void_transfer = TigerBeetle::Transfer.new(
  TigerBeetle.id,
  transfer_debit_account_id,
  transfer_credit_account_id,
  0,
  pending_transfer.id,
  1,
  720,
  TigerBeetle::TransferFlags::VOID_PENDING_TRANSFER
)

transfer_results = client.create_transfers([void_transfer])
# Results handling omitted.

Transfer Lookup

NOTE: While transfer lookup exists, it is not a flexible query API. We are developing query APIs and there will be new methods for querying transfers in the future.

Transfer lookup is batched, like transfer creation. Pass in all ids to fetch, and matched transfers are returned.

If no transfer matches an id, no object is returned for that transfer. So the order of transfers in the response is not necessarily the same as the order of ids in the request. You can refer to the id field in the response to distinguish transfers.

transfers = client.lookup_transfers([transfer0.id, transfer1.id])

Get Account Transfers

NOTE: This is a preview API that is subject to breaking changes once we have a stable querying API.

Fetches the transfers involving a given account, allowing basic filter and pagination capabilities.

The transfers in the response are sorted by timestamp in chronological or reverse-chronological order.

filter = TigerBeetle::AccountFilter.new(
  account1.id,
  0,
  0,
  0,
  0,
  0,
  0,
  10,
  TigerBeetle::AccountFilterFlags::DEBITS |
    TigerBeetle::AccountFilterFlags::CREDITS |
    TigerBeetle::AccountFilterFlags::REVERSED
)

account_transfers = client.get_account_transfers(filter)

Get Account Balances

NOTE: This is a preview API that is subject to breaking changes once we have a stable querying API.

Fetches the point-in-time balances of a given account, allowing basic filter and pagination capabilities.

Only accounts created with the flag history set retain historical balances.

The balances in the response are sorted by timestamp in chronological or reverse-chronological order.

filter = TigerBeetle::AccountFilter.new(
  account1.id,
  0,
  0,
  0,
  0,
  0,
  0,
  10,
  TigerBeetle::AccountFilterFlags::DEBITS |
    TigerBeetle::AccountFilterFlags::CREDITS |
    TigerBeetle::AccountFilterFlags::REVERSED
)

account_balances = client.get_account_balances(filter)

Query Accounts

NOTE: This is a preview API that is subject to breaking changes once we have a stable querying API.

Query accounts by the intersection of some fields and by timestamp range.

The accounts in the response are sorted by timestamp in chronological or reverse-chronological order.

query_filter = TigerBeetle::QueryFilter.new(
  0,
  0,
  0,
  1,
  1,
  0,
  0,
  10,
  TigerBeetle::QueryFilterFlags::REVERSED
)

query_accounts = client.query_accounts(query_filter)

Query Transfers

NOTE: This is a preview API that is subject to breaking changes once we have a stable querying API.

Query transfers by the intersection of some fields and by timestamp range.

The transfers in the response are sorted by timestamp in chronological or reverse-chronological order.

query_filter = TigerBeetle::QueryFilter.new(
  0,
  0,
  0,
  1,
  720,
  0,
  0,
  10,
  TigerBeetle::QueryFilterFlags::REVERSED
)

query_transfers = client.query_transfers(query_filter)

Linked Events

When the linked flag is specified for an account when creating accounts or a transfer when creating transfers, it links that event with the next event in the batch, to create a chain of events, of arbitrary length, which all succeed or fail together. The tail of a chain is denoted by the first event without this flag. The last event in a batch may therefore never have the linked flag set as this would leave a chain open-ended. Multiple chains or individual events may coexist within a batch to succeed or fail independently.

Events within a chain are executed within order, or are rolled back on error, so that the effect of each event in the chain is visible to the next, and so that the chain is either visible or invisible as a unit to subsequent events after the chain. The event that was the first to break the chain will have a unique error result. Other events in the chain will have their error result set to linked_event_failed.

linked_flag = TigerBeetle::TransferFlags::LINKED
batch = [
  TigerBeetle::Transfer.new(
    TigerBeetle.id,
    transfer_debit_account_id,
    transfer_credit_account_id,
    1,
    1,
    720,
    linked_flag
  ),
  TigerBeetle::Transfer.new(
    TigerBeetle.id,
    transfer_debit_account_id,
    transfer_credit_account_id,
    1,
    1,
    720
  )
]

transfer_results = client.create_transfers(batch)
# Results handling omitted.

Imported Events

When the imported flag is specified for an account when creating accounts or a transfer when creating transfers, it allows importing historical events with a user-defined timestamp.

The entire batch of events must be set with the flag imported.

It’s recommended to submit the whole batch as a linked chain of events, ensuring that if any event fails, none of them are committed, preserving the last timestamp unchanged. This approach gives the application a chance to correct failed imported events, re-submitting the batch again with the same user-defined timestamps.

historical_timestamp = 0
# Loaded from an external source.
historical_accounts = []
# Loaded from an external source.
historical_transfers = []

accounts_to_import = historical_accounts.map.with_index do |historical_account, index|
  historical_timestamp += 1
  historical_account.timestamp = historical_timestamp
  historical_account.flags = TigerBeetle::AccountFlags::IMPORTED
  if index < historical_accounts.length - 1
    historical_account.flags |= TigerBeetle::AccountFlags::LINKED
  end

  historical_account
end

account_results = client.create_accounts(accounts_to_import)
# Results handling omitted.

transfers_to_import = historical_transfers.map.with_index do |historical_transfer, index|
  historical_timestamp += 1
  historical_transfer.timestamp = historical_timestamp
  historical_transfer.flags = TigerBeetle::TransferFlags::IMPORTED
  if index < historical_transfers.length - 1
    historical_transfer.flags |= TigerBeetle::TransferFlags::LINKED
  end

  historical_transfer
end

transfer_results = client.create_transfers(transfers_to_import)
# Results handling omitted.

Timeouts And Cancellation

The Client retries indefinitely and doesn’t impose any per-request timeout. Cancellation is provided as a mechanism, and the specific cancellation policy is left to the application. A Client instance can be closed at any time. On close, all in-flight requests are canceled and return an error to the caller. Even if an error is returned, a request might still be processed by the TigerBeetle server. Reliable transaction submission explains how to make transfers retry-proof using IDs for end-to-end idempotency.

Edit this page