Transfer
A transfer
is an immutable record of a financial transaction between two accounts.
In TigerBeetle, financial transactions are called "transfers" instead of "transactions" because the latter term is heavily overloaded in the context of databases.
Note that transfers debit a single account and credit a single account on the same ledger. You can compose these into more complex transactions using the methods described in Currency Exchange and Multi-Debit, Multi-Credit Transfers.
Updates
Transfers cannot be modified after creation.
If a detail of a transfer is incorrect and needs to be modified, this is done using correcting transfers.
Deletion
Transfers cannot be deleted after creation.
If a transfer is made in error, its effects can be reversed using a correcting transfer.
Guarantees
- Transfers are immutable. They are never modified once they are successfully created.
- There is at most one
Transfer
with a particularid
. - A pending transfer resolves at most once.
- Transfer timeouts are deterministic, driven by the cluster's timestamp.
Modes
Transfers can either be Single-Phase, where they are executed immediately, or Two-Phase, where they are first put in a Pending state and then either Posted or Voided. For more details on the latter, see the Two-Phase Transfer guide.
Fields used by each mode of transfer:
Field | Single-Phase | Pending | Post-Pending | Void-Pending |
---|---|---|---|---|
id | required | required | required | required |
debit_account_id | required | required | optional | optional |
credit_account_id | required | required | optional | optional |
amount | required | required | required | optional |
pending_id | none | none | required | required |
user_data_128 | optional | optional | optional | optional |
user_data_64 | optional | optional | optional | optional |
user_data_32 | optional | optional | optional | optional |
timeout | none | optional¹ | none | none |
ledger | required | required | optional | optional |
code | required | required | optional | optional |
flags.linked | optional | optional | optional | optional |
flags.pending | false | true | false | false |
flags.post_pending_transfer | false | false | true | false |
flags.void_pending_transfer | false | false | false | true |
flags.balancing_debit | optional | optional | false | false |
flags.balancing_credit | optional | optional | false | false |
flags.closing_debit | optional | true | false | false |
flags.closing_credit | optional | true | false | false |
flags.imported | optional | optional | optional | optional |
timestamp | none² | none² | none² | none² |
¹ None if
flags.imported
is set.
² Required ifflags.imported
is set.
Fields
id
This is a unique identifier for the transaction.
Constraints:
- Type is 128-bit unsigned integer (16 bytes)
- Must not be zero or
2^128 - 1
- Must not conflict with another transfer in the cluster
See the id
section in the data modeling doc for more
recommendations on choosing an ID scheme.
Note that transfer IDs are unique for the cluster -- not the ledger. If you want to store a
relationship between multiple transfers, such as indicating that multiple transfers on different
ledgers were part of a single transaction, you should store a transaction ID in one of the
user_data
fields.
debit_account_id
This refers to the account to debit the transfer's amount
.
Constraints:
- Type is 128-bit unsigned integer (16 bytes)
- When
flags.post_pending_transfer
andflags.void_pending_transfer
are not set:- Must match an existing account
- Must not be the same as
credit_account_id
- When
flags.post_pending_transfer
orflags.void_pending_transfer
are set:- If
debit_account_id
is zero, it will be automatically set to the pending transfer'sdebit_account_id
. - If
debit_account_id
is nonzero, it must match the corresponding pending transfer'sdebit_account_id
.
- If
- When
flags.imported
is set:
credit_account_id
This refers to the account to credit the transfer's amount
.
Constraints:
- Type is 128-bit unsigned integer (16 bytes)
- When
flags.post_pending_transfer
andflags.void_pending_transfer
are not set:- Must match an existing account
- Must not be the same as
debit_account_id
- When
flags.post_pending_transfer
orflags.void_pending_transfer
are set:- If
credit_account_id
is zero, it will be automatically set to the pending transfer'scredit_account_id
. - If
credit_account_id
is nonzero, it must match the corresponding pending transfer'scredit_account_id
.
- If
- When
flags.imported
is set:
amount
This is how much should be debited from the debit_account_id
account and credited to the
credit_account_id
account.
Note that this is an unsigned 128-bit integer. You can read more about using debits and credits to represent positive and negative balances as well as fractional amounts and asset scales.
- When
flags.balancing_debit
is set, this is the maximum amount that will be debited/credited, where the actual transfer amount is determined by the debit account's constraints. - When
flags.balancing_credit
is set, this is the maximum amount that will be debited/credited, where the actual transfer amount is determined by the credit account's constraints. - When
flags.post_pending_transfer
is set, the amount posted will be:- the pending transfer's amount, when the posted transfer's
amount
isAMOUNT_MAX
- the posting transfer's amount, when the posted transfer's
amount
is less than or equal to the pending transfer's amount.
- the pending transfer's amount, when the posted transfer's
Constraints:
- Type is 128-bit unsigned integer (16 bytes)
- When
flags.void_pending_transfer
is set:- If
amount
is zero, it will be automatically be set to the pending transfer'samount
. - If
amount
is nonzero, it must be equal to the pending transfer'samount
.
- If
- When
flags.post_pending_transfer
is set:- If
amount
isAMOUNT_MAX
(2^128 - 1
), it will automatically be set to the pending transfer'samount
. - If
amount
is notAMOUNT_MAX
, it must be less than or equal to the pending transfer'samount
.
- If
Client release < 0.16.0
Additional constraints:
- When
flags.post_pending_transfer
is set:- If
amount
is zero, it will be automatically be set to the pending transfer'samount
. - If
amount
is nonzero, it must be less than or equal to the pending transfer'samount
.
- If
- When
flags.balancing_debit
and/orflags.balancing_credit
is set, ifamount
is zero, it will automatically be set to the maximum amount that does not violate the corresponding account limits. (Equivalent to settingamount = 2^128 - 1
). - When all of the following flags are not set,
amount
must be nonzero:flags.post_pending_transfer
flags.void_pending_transfer
flags.balancing_debit
flags.balancing_credit
Examples
- For representing fractional amounts (e.g.
$12.34
), see Fractional Amounts. - For balancing transfers, see Close Account.
pending_id
If this transfer will post or void a pending transfer, pending_id
references that pending
transfer. If this is not a post or void transfer, it must be zero.
See the section on Two-Phase Transfers for more information on
how the pending_id
is used.
Constraints:
- Type is 128-bit unsigned integer (16 bytes)
- Must be zero if neither void nor pending transfer flag is set
- Must match an existing transfer's
id
if non-zero
user_data_128
This is an optional 128-bit secondary identifier to link this transfer to an external entity or event.
When set to zero, no secondary identifier will be associated with the account, therefore only non-zero values can be used as query filter.
As an example, you might generate a TigerBeetle Time-Based Identifier that ties together a group of transfers.
For more information, see Data Modeling.
Constraints:
- Type is 128-bit unsigned integer (16 bytes)
user_data_64
This is an optional 64-bit secondary identifier to link this transfer to an external entity or event.
When set to zero, no secondary identifier will be associated with the account, therefore only non-zero values can be used as query filter.
As an example, you might use this field store an external timestamp.
For more information, see Data Modeling.
Constraints:
- Type is 64-bit unsigned integer (8 bytes)
user_data_32
This is an optional 32-bit secondary identifier to link this transfer to an external entity or event.
When set to zero, no secondary identifier will be associated with the account, therefore only non-zero values can be used as query filter.
As an example, you might use this field to store a timezone or locale.
For more information, see Data Modeling.
Constraints:
- Type is 32-bit unsigned integer (4 bytes)
timeout
This is the interval in seconds after a pending
transfer's
arrival at the cluster that it may be posted or
voided. Zero denotes absence of timeout.
Non-pending transfers cannot have a timeout.
Imported transfers cannot have a timeout.
TigerBeetle makes a best-effort approach to remove pending balances of expired transfers automatically:
Transfers expire exactly at their expiry time (
timestamp
plustimeout
converted in nanoseconds).Pending balances will never be removed before its expiry.
Expired transfers cannot be manually posted or voided.
It is not guaranteed that the pending balance will be removed exactly at its expiry.
In particular, client requests may observe still-pending balances for expired transfers.
Pending balances are removed in chronological order by expiry. If multiple transfers expire at the same time, then ordered by the transfer's creation
timestamp
.If a transfer
A
has expiryE₁
and transferB
has expiryE₂
, andE₁<E₂
, if transferB
had the pending balance removed, then transferA
had the pending balance removed as well.
Constraints:
- Type is 32-bit unsigned integer (4 bytes)
- Must be zero if
flags.pending
is not set - Must be zero if
flags.imported
is set.
The timeout
is an interval in seconds rather than an absolute timestamp because this is more
robust to clock skew between the cluster and the application. (Watch this talk on
Detecting Clock Sync Failure in Highly Available Systems
on YouTube for more details.)
ledger
This is an identifier that partitions the sets of accounts that can transact with each other.
See data modeling for more details about how to think about setting up your ledgers.
Constraints:
- Type is 32-bit unsigned integer (4 bytes)
- When
flags.post_pending_transfer
orflags.void_pending_transfer
is set:- If
ledger
is zero, it will be automatically be set to the pending transfer'sledger
. - If
ledger
is nonzero, it must match theledger
value on the pending transfer'sdebit_account_id
andcredit_account_id
.
- If
- When
flags.post_pending_transfer
andflags.void_pending_transfer
are not set:ledger
must not be zero.ledger
must match theledger
value on the accounts referenced indebit_account_id
andcredit_account_id
.
code
This is a user-defined enum denoting the reason for (or category of) the transfer.
Constraints:
- Type is 16-bit unsigned integer (2 bytes)
- When
flags.post_pending_transfer
orflags.void_pending_transfer
is set:- If
code
is zero, it will be automatically be set to the pending transfer'scode
. - If
code
is nonzero, it must match the pending transfer'scode
.
- If
- When
flags.post_pending_transfer
andflags.void_pending_transfer
are not set,code
must not be zero.
flags
This specifies (optional) transfer behavior.
Constraints:
- Type is 16-bit unsigned integer (2 bytes)
- Some flags are mutually exclusive; see
flags_are_mutually_exclusive
.
flags.linked
This flag links the result of this transfer to the outcome of the next transfer in the request such that they will either succeed or fail together.
The last transfer in a chain of linked transfers does not have this flag set.
You can read more about linked events.
Examples
flags.pending
Mark the transfer as a pending transfer.
flags.post_pending_transfer
Mark the transfer as a post-pending transfer.
flags.void_pending_transfer
Mark the transfer as a void-pending transfer.
flags.balancing_debit
Transfer at most amount
— automatically transferring less than amount
as necessary
such that
debit_account.debits_pending + debit_account.debits_posted ≤ debit_account.credits_posted
.
The amount
of the recorded transfer is set to the actual amount that was transferred, which is
less than or equal to the amount that was passed to create_transfers
.
Retrying a balancing transfer will return
exists_with_different_amount
only when the maximum amount passed to create_transfers
is insufficient to fulfill the amount
that was actually transferred.
Otherwise it may return exists
even if the retry amount
differs from the original value.
flags.balancing_debit
is exclusive with the
flags.post_pending_transfer
/flags.void_pending_transfer
flags because posting or voiding a
pending transfer will never exceed/overflow either account's limits.
flags.balancing_debit
is compatible with (and orthogonal to) flags.balancing_credit
.
Client release < 0.16.0
Transfer at most amount
— automatically transferring less than amount
as necessary
such that
debit_account.debits_pending + debit_account.debits_posted ≤ debit_account.credits_posted
. If
amount
is set to 0
, transfer at most 2^64 - 1
(i.e. as much as possible).
If the highest amount transferable is 0
, returns
exceeds_credits
.
Examples
flags.balancing_credit
Transfer at most amount
— automatically transferring less than amount
as necessary
such that
credit_account.credits_pending + credit_account.credits_posted ≤ credit_account.debits_posted
.
The amount
of the recorded transfer is set to the actual amount that was transferred, which is
less than or equal to the amount that was passed to create_transfers
.
Retrying a balancing transfer will return
exists_with_different_amount
only when the maximum amount passed to create_transfers
is insufficient to fulfill the amount
that was actually transferred.
Otherwise it may return exists
even if the retry amount
differs from the original value.
flags.balancing_credit
is exclusive with the
flags.post_pending_transfer
/flags.void_pending_transfer
flags because posting or voiding a
pending transfer will never exceed/overflow either account's limits.
flags.balancing_credit
is compatible with (and orthogonal to) flags.balancing_debit
.
Client release < 0.16.0
Transfer at most amount
— automatically transferring less than amount
as necessary
such that
credit_account.credits_pending + credit_account.credits_posted ≤ credit_account.debits_posted
. If
amount
is set to 0
, transfer at most 2^64 - 1
(i.e. as much as possible).
If the highest amount transferable is 0
, returns
exceeds_debits
.
Examples
flags.closing_debit
When set, it will cause the Account.flags.closed
flag
of the debit account to be set if the transfer succeeds.
This flag requires a two-phase transfer, so the flag flags.pending
must also be set. This ensures that closing transfers are reversible by
voiding the pending transfer, and requires that the reversal
operation references the corresponding closing transfer, guarding against unexpected interleaving
of close/unclose operations.
flags.closing_credit
When set, it will cause the Account.flags.closed
flag
of the credit account to be set if the transfer succeeds.
This flag requires a two-phase transfer, so the flag flags.pending
must also be set. This ensures that closing transfers are reversible by
voiding the pending transfer, and requires that the reversal
operation references the corresponding closing transfer, guarding against unexpected interleaving
of close/unclose operations.
flags.imported
When set, allows importing historical Transfer
s with their original timestamp
.
TigerBeetle will not use the cluster clock to assign the timestamp, allowing the user to define it, expressing when the transfer was effectively created by an external event.
To maintain system invariants regarding auditability and traceability, some constraints are necessary:
It is not allowed to mix events with the
imported
flag set and not set in the same batch. The application must submit batches of imported events separately.User-defined timestamps must be unique and expressed as nanoseconds since the UNIX epoch. No two objects can have the same timestamp, even different objects like an
Account
and aTransfer
cannot share the same timestamp.User-defined timestamps must be a past date, never ahead of the cluster clock at the time the request arrives.
Timestamps must be strictly increasing.
Even user-defined timestamps that are required to be past dates need to be at least one nanosecond ahead of the timestamp of the last transfer committed by the cluster.
Since the timestamp cannot regress, importing past events can be naturally restrictive without coordination, as the last timestamp can be updated using the cluster clock during regular cluster activity. Instead, it's recommended to import events only on a fresh cluster or during a scheduled maintenance window.
It's recommended to submit the entire batch as a linked chain, ensuring that if any transfer fails, none of them are committed, preserving the last timestamp unchanged. This approach gives the application a chance to correct failed imported transfers, re-submitting the batch again with the same user-defined timestamps.
Imported transfers cannot have a
timeout
.It's possible to import pending transfers with a user-defined timestamp, but since it's not driven by the cluster clock, it cannot define a
timeout
for automatic expiration. In those cases, the two-phase post or rollback must be done manually.
timestamp
This is the time the transfer was created, as nanoseconds since UNIX epoch. You can read more about Time in TigerBeetle.
Constraints:
Type is 64-bit unsigned integer (8 bytes)
Must be
0
when theTransfer
is created withflags.imported
not setIt is set by TigerBeetle to the moment the transfer arrives at the cluster.
Must be greater than
0
and less than2^63
when theTransfer
is created withflags.imported
set
Internals
If you're curious and want to learn more, you can find the source code for this struct in
src/tigerbeetle.zig.
Search for const Transfer = extern struct {
.
You can find the source code for creating a transfer in
src/state_machine.zig.
Search for fn create_transfer(
.