GuidesAPI Reference
DocumentationLog In

How to Write a Configuration File for rosetta-cli Testing

Prerequisites

  • For the check:construction test, a Rosetta DSL (*.ros) file.
  • For the check:data test, a way to measure account balances.
    • If your blockchain supports historical balance lookup, you'll need to set the historical_balance_enabled value to true.
    • If your blockchain does not support historical balance lookup, you'll need to complete the following actions:
      • Set the historical_balance_enabled value to false.
      • Provide a bootstrap_balances.json file for the bootstrap_balances value.

Write the Configuration File

  1. From the command line, create a config.json file in the directory of your choice.
    • For example, you can create the rosetta-cli-conf/testnet directory to test your configuration with testnet.
  2. Edit the config.json file and add the fields and objects, with their corresponding values necessary for your implementation. For more information on these values, see the Rosetta-CLI Configuration documentation and the End Conditions documentation.
  3. Save the file.
  4. Run the rosetta-cli tool for testing.

List of Fields and Objects for the Configuration File

The configuration file must be written in JSON format. Do not add optional fields with empty or null values to the configuration file.

Fields

The fields below are part of the JSON file object.

FieldsTypeDescriptionPossible ValuesExample
online_urlstringThe URL of a Rosetta API implementation in “online” mode.A valid url.http://localhost:8080
data_directorystringA folder used to store logs and any validation data.An absolute or relative path to where the rosetta-cli binary is being executed.cli-data
http_timeoutnumberThe timeout for an HTTP request, in seconds.A positive integer.300
max_retriesnumberThe number of tries that the rosetta-cli tool will retry an HTTP request.
If the retry_elapsed_time field is also populated, the rosetta-cli tool may stop attempting retries early.
A positive integer.15
retry_elapsed_timenumberThe total time to spend retrying an HTTP request, in seconds.
If the max_retries field is also populated, the rosetta-cli tool may stop attempting retries early.
A positive integer.0
max_online_connectionsnumberThe maximum number of open connections that the online fetcher will open.A positive integer.500
force_retryBooleanWhether to override the default retry handling to retry on all error (non-200) responses.
  • true
  • false

Omit if empty.
true
max_sync_concurrencynumberThe maximum sync concurrency to use while syncing blocks. Sync concurrency is managed automatically by the syncer package.A positive integer.64
tip_delaynumberDictates how many seconds behind the current time is considered the tip. If the rosetta-cli tool is less than the number of tip_delay seconds from the last processed block, then it is considered to be “behind tip”.A positive integer.120
max_reorg_depthnumberSpecifies the maximum possible reorganization depth of the blockchain being synced. This value is used to determine how aggressively to prune old block data.A positive integer.
Omit if empty.
100
log_configurationBooleanDetermines whether the configuration settings should be printed to the console when a file is loaded.
  • true
  • false
false
compression_disabledBooleanWhether to configure the storage layer to not perform data compression before writing to disk.
This leads to significantly more on-disk storage usage but can lead to performance gains.
  • true
  • false
true
memory_limit_disabledBooleanWhether to configure storage to increase memory usage. Enabling this value massively increases performance, but can use tens of GBs of RAM, even with pruning enabled.
  • true
  • false
true
seen_block_workersnumberThe number of goroutines spawned to store seen blocks in storage before we attempt to sequence.
This value defaults to the runtime.NumCPU() value.
  • A positive integer.
  • Default - the runtime.NumCPU() value.
empty (uses the default value)
serial_block_workersnumberThe number of goroutines spawned to help with block sequencing (i.e., updating balances, updating coins, etc).
This value defaults to the runtime.NumCPU() value.
  • A positive integer.
  • Default - the runtime.NumCPU() value.
empty (uses the default value)
validation_filestringThe field used for asset-specific validation.
This value defaults to an empty string, which means that no asset-specific validation will be done.
  • A string
  • null
empty (uses the default value)

Objects

The following objects are part of the JSON file object: network, construction, and data. Objects that are part of the construction, and data objects are listed under those headings.

network

Read the NetworkIdentifier documentation for more information.

construction

The configuration values needed to run the check:construction test in the rosetta-cli tool.

Code Sample

Contains the following fields and objects:

FieldsTypeDescriptionPossible ValuesExample
offline_urlstringThe URL of a Rosetta API implementation in “offline mode”.
This field is part of the construction object.
A valid URL.http://example.com
max_offline_connectionsnumberThe maximum number of open connections that the offline fetcher will open.
This field is part of the construction object.
A positive number.
force_retryBooleanWhether to override the default retry handling to retry on all error (non-200) responses.
This field is part of the construction object.
  • true
  • false

Omit if empty.
stale_depthnumberThe number of blocks to wait before attempting to rebroadcast after not finding a transaction on-chain.
This field is part of the construction object.
A positive integer.3
broadcast_limitnumberThe number of times to attempt re-broadcast before giving up on a transaction broadcast.
This field is part of the construction object.
A positive integer.5
ignore_broadcast_failuresBooleanWhether the tool should exit when there are broadcast failures that surpass the broadcast_limit value.
This field is part of the construction object.
  • true
  • false
false
clear_broadcastsBooleanWhether all pending broadcasts should be removed from the stored broadcast file on restart.
This field is part of the construction object.
  • true
  • false
broadcast_behind_tipBooleanWhether the tool should broadcast transactions when it is behind tip (as defined in the tip_delay value).
This field is part of the construction object.
  • true
  • false
block_broadcast_limitnumberThe number of transactions to attempt to broadcast in a single block. When there are many pending broadcasts, it may make sense to limit the number of broadcasts.
This field is part of the construction object.
A positive integer.
rebroadcast_allBooleanWhether all pending broadcasts should be rebroadcast from the stored broadcast file on restart.
This field is part of the construction object.
  • true
  • false
output_pathstringThe path to the directory where you would like to store the output files that rosetta-cli creates.
The default value for this field is /cli-data.
This field is part of the construction object.
A valid relative path.cli-data
constructor_dsl_filestringThe DSL (*.ros) file for the check:construction test.
This field is part of the construction object.
A .ros filename.ethereum.ros
status_portnumberThe port where you can query a running check:construction test to get progress statistics.
This process can be used instead of parsing logs to populate a status dashboard.
This field is part of the construction object.
A positive integer.
Omit if empty.
results_output_filestringThe absolute file path where the rosetta-cli tool will save the results of a check:construction test run.
This field is part of the construction object.
A valid filepath.
Omit if empty.
/directory/construction_test_results
quietBooleanWhether to silence all request and response logging.
This field is part of the construction object.
  • true
  • false

Omit if empty.
initial_balance_fetch_disabledBooleanConfigures the rosetta-cli tool to not look up the balance of newly seen accounts at the genesis block before applying operations.
Disabling this field is a good idea only if you create multiple new accounts each block and don’t fund any accounts before starting the check:construction test.
This is a separate configuration value from the initial_balance_fetch_disabled value under the data object. The construction object value is usually set to false, whereas the data object value is usually set to true.
This field is part of the construction object.
  • true
  • false
prefunded_accountsobjectLoads prefunded addresses into field storage.
This object is part of the construction object.
Contains the privkey field and the account_identifier, curve_type, and currency objects.
Omit if empty.
end_conditionsobjectThe conditions that need to be fulfilled to successfully end the check:construction test.

This object is part of the construction object.
Contains the create_account and transfer fields.
end_conditions (construction)

This object is part of the construction object.
The conditions that need to be fulfilled to successfully end the check:construction test.
Contains the create_account and transfer fields.

Code Sample

FieldsTypeDescriptionPossible ValuesExample
create_accountnumberThe number of accounts for the test to create.
This field is part of the end_conditions object.
A positive integer.10
transfernumberThe number of transfers for the test to complete.
This field is part of the end_conditions object.
A positive integer.20
prefunded_accounts

This object is part of the construction object.
Loads prefunded addresses into field storage.
Contains the privkey and curve_type fields and the account_identifier and currency objects. If you have no values for the prefunded_account object, you can omit it from the construction object.

Code Sample

FieldsTypeDescriptionPossible ValuesExample
privkeystringA private key.
This field is part of the prefunded_accounts object.
A valid private key.
account_identifierobjectRead the AccountIdentifier documentation for more information.
curve_typestringRead the CurveType documentation for more information.
currencyobjectRead the Currency documentation for more information.

data

Contains all the configurations needed to run the check:data test.

Code Sample

Contains the following fields and objects:

FieldsTypeDescriptionPossible ValuesExample
active_reconciliation_concurrencynumberThe concurrency to use while fetching accounts during active reconciliation.
This field is part of the data object.
A positive integer.32
inactive_reconciliation_concurrencynumberThe concurrency to use while fetching accounts during inactive reconciliation.
This field is part of the data object.
A positive integer.
inactive_reconciliation_frequencynumberThe number of blocks to wait between inactive reconciliations on each account.
This field is part of the data object.
A positive integer.
log_blocksBooleanWhether to log processed blocks.
This field is part of the data object.
  • true
  • false
log_transactionsBooleanWhether to log processed transactions.
  • true
  • false
log_balance_changesBooleanWhether to log all balance changes.
This field is part of the data object.
  • true
  • false
log_reconciliationsBooleanWhether to log all reconciliations.
This field is part of the data object.
  • true
  • false
ignore_reconciliation_errorBooleanWhether block processing should halt on a reconciliation error.
You can either collect all reconciliation errors, or silence reconciliation errors during development.
This field is part of the data object.
  • true
  • false
exempt_accountsstringThe JSON file that lists all accounts exempt from balance tracking and reconciliation.
You can find an example of this file in the rosetta-cli/examples directory in GitHub.
This field is part of the data object.
A valid path and filename, relative to the location of this configuration file.exempt_accounts.json
bootstrap_balancesstringThe JSON file that contains the initial asset or token balance.
You can find an example of this file in the rosetta-cli/examples directory in GitHub.
Note: If you populate this value after syncing begins, the test will ignore it.
This field is part of the data object.
A valid path and filename, relative to the location of this configuration file.bootstrap_balances.json
historical_balance_enabledBooleanWhether to look up a balance at the block where a balance change occurred, instead of at the current block.
Note:
Blockchains that do not support historical balance lookup should set this value to false And include a bootstrap_balances.json file for the bootstrap_balances value.
If this value is false or omitted, the check:data test will ignore the inactive_discrepancy_search_disabled field.
This field is part of the data object.
  • true
  • false

Omit if empty.
interesting_accountsstringThe JSON file that contains all of the accounts to check on each block.
You can find an example of this file in the rosetta-cli/examples directory in GitHub.
This field is part of the data object.
A valid path and filename, relative to the location of this configuration file.interesting_accounts.json
reconciliation_disabledBooleanWhether or not to attempt reconciliation.
To confirm that syncing is working as expected, we recommend that you disable some of the more advanced checks when first testing an implementation.
This field is part of the data object.
  • true
  • false
reconciliation_drain_disabledBooleanIf reconciliation is enabled, whether the rosetta-cli tool should exit the check:data test before the entire active reconciliation queue has been drained.
This field is part of the data object.
  • true
  • false
inactive_discrepancy_search_disabledBooleanWhether the check:data test should perform a search to find any inactive reconciliation discrepancies.
Note: If the historical_balance_enabled value is false or omitted, the check:data test will ignore this field.
This field is part of the data object.
  • true
  • false
balance_tracking_disabledBooleanWhether to attempt a balance calculation.
We recommend that you try to fetch all blocks before checking for balance consistency when first testing an implementation.
This field is part of the data object.
  • true
  • false
coin_tracking_disabledBooleanWhether to attempt coin or UTX0 tracking.
We recommend that you try to fetch all blocks before checking for coin consistency when first testing an implementation.
This field is part of the data object.
  • true
  • false
start_indexnumberThe block height from where the check:data test should start syncing.
Note:
If you do not provide a value for this field, syncing will start from the last saved block.
If no blocks have been synced previously, then syncing will start from the genesis block.
This field is part of the data object.
A positive integer.
Omit if empty.
status_portnumberThe port where you can query a running check:data test to get progress statistics.
This process can be used instead of parsing logs to populate a status dashboard.
This field is part of the data object.
A positive integer.
Omit if empty.
results_output_filestringThe absolute file path where the rosetta-cli tool will save the results of a check:data run.
This field is part of the data object.
A valid filepath./directory/data_test_results
pruning_disabledBooleanWhether to not attempt storage pruning.
Note: You should only set this value to true if you wish to use the start_index field at a later point to restart the check:data test from a previously synced block.
This field is part of the data object.
  • true
  • false
pruning_frequencynumberThe frequency, in seconds, that the check:data test attempts to prune blocks.
The default value for this field is the statefulsyncer package’s value.
This field is part of the data object.
A positive integer.
Omit if empty.
initial_balance_fetch_disabledBooleanConfigures the rosetta-cli tool to not look up the balance of newly seen accounts at the genesis block before applying operations.
Note:
Disabling this field can speed up performance without impacting validation accuracy, as long as the boostrap_balances file provides all the genesis accounts syncing begins from genesis.
This is a separate configuration value from the initial_balance_fetch_disabled value under the construction object. The data object’s value is usually set to true, whereas the construction object’s value is usually set to false.
  • true
  • false
true
reconciler_active_backlognumberThe maximum number of pending changes to store in the active reconciliation backlog before skipping reconciliation on new changes.
This field is part of the data object.
A positive integer.
Omit if empty.
end_conditionsobjectRequired. The conditions that need to be fulfilled to successfully end the check:data test.
This object is part of the data object.
Contains the index, tip, and duration fields, and the reconciliation_coverage object.
end_conditions (data)

This object is part of the data object.
The conditions that need to be fulfilled to successfully end the check:data test.
Contains the index, tip, and duration fields, and the reconciliation_coverage object.

Code Sample

FieldsTypeDescriptionPossible ValuesExample
indexnumberConfigures the syncer to stop once it reaches a particular block height.
This field is part of the end_conditions object.
A positive integer.
Omit if empty.
tipBooleanConfigures the syncer to stop once it reaches the tip of the blockchain.
You must also configure the tip_delay value if you set the tip value to true.
This field is part of the end_conditions object.
  • true
  • false

Omit if empty.
durationnumberThe number of seconds the syncer must run before it stops.
This field is part of the end_conditions object.
A positive integer.
Omit if empty.
reconciliation_coverageobjectConfigures the syncer to stop once it reaches the reconciliation coverage set in this object.
This object is part of the end_conditions object.
Contains the coverage, from_tip, tip, index, and account_count fields.
reconciliation_coverage

This object is part of the data object's end_conditions object.
Configures the syncer to stop once it reaches the reconciliation coverage set in this object.
Contains the coverage, from_tip, tip, index, and account_count fields.

Code Sample

FieldsTypeDescriptionPossible ValuesExample
coveragenumberA value that represents the percentage of reconciled accounts.
This field is part of the reconciliation_coverage object.
A positive decimal that’s between 0.0 and 1.0.0.95
from_tipBooleanWhether the reconciliation coverage should be measured from the tip of the blockchain.
This field is part of the reconciliation_coverage object.
Omit if empty.
  • true
  • false
true
tipBooleanWhether the tip must be reached before reconciliation coverage is considered valid.
This field is part of the reconciliation_coverage object.
  • true
  • false

Omit if empty.
indexnumberThe height that must be reached before reconciliation coverage is considered valid.
This field is part of the reconciliation_coverage object.
Omit if empty.
A positive integer.
account_countnumberThe number of accounts that must be observed before reconciliation coverage is considered valid.
This field is part of the reconciliation_coverage object.
Omit if empty.
A positive integer.

Sample Configuration files

The following are sample configuration files for default, ethereum, and bitcoin.

Sample default.json file

{
    "network":{
        "blockchain":"MyBlockchain",
        "network":"testnet"
    },
    "online_url":"http://localhost:8080",
    "data_directory":"",
    "http_timeout":10,
    "max_retries":5,
    "retry_elapsed_time":0,
    "max_online_connections":120,
    "max_sync_concurrency":64,
    "tip_delay":300,
    "max_reorg_depth":100,
    "log_configuration":false,
    "compression_disabled":false,
    "memory_limit_disabled":false,
    "construction":null,
    "data":{
        "active_reconciliation_concurrency":16,
        "inactive_reconciliation_concurrency":4,
        "inactive_reconciliation_frequency":250,
        "log_blocks":false,
        "log_transactions":false,
        "log_balance_changes":false,
        "log_reconciliations":false,
        "ignore_reconciliation_error":false,
        "exempt_accounts":"",
        "bootstrap_balances":"",
        "interesting_accounts":"",
        "reconciliation_disabled":false,
        "reconciliation_drain_disabled":false,
        "inactive_discrepancy_search_disabled":false,
        "balance_tracking_disabled":false,
        "coin_tracking_disabled":false,
        "status_port":9090,
        "results_output_file":"",
        "pruning_disabled":false,
        "initial_balance_fetch_disabled":false
    }
}

Sample config.json file for Ethereum

{
    "network":{
        "blockchain":"Ethereum",
        "network":"Ropsten"
    },
    "data_directory":"cli-data",
    "http_timeout":300,
    "max_retries":15,
    "max_online_connections":500,
    "max_sync_concurrency":64,
    "tip_delay":120,
    "compression_disabled":true,
    "memory_limit_disabled":true,
    "construction":{
        "stale_depth":3,
        "broadcast_limit":5,
        "constructor_dsl_file":"ethereum.ros",
        "end_conditions":{
            "create_account":10,
            "transfer":20
        }
    },
    "data":{
        "initial_balance_fetch_disabled":true,
        "active_reconciliation_concurrency":32,
        "bootstrap_balances":"bootstrap_balances.json",
        "end_conditions":{
            "reconciliation_coverage":{
                "coverage":0.95,
                "from_tip":true
            }
        }
    }
}

Sample config.json file for Bitcoin

{
    "network":{
        "blockchain":"Bitcoin",
        "network":"Testnet3"
    },
    "data_directory":"cli-data",
    "http_timeout":300,
    "max_retries":5,
    "max_online_connections":1000,
    "tip_delay":1800,
    "memory_limit_disabled":true,
    "compression_disabled":true,
    "construction":{
        "constructor_dsl_file":"bitcoin.ros",
        "end_conditions":{
            "create_account":10,
            "transfer":10
        }
    },
    "data":{
        "initial_balance_fetch_disabled":true,
        "end_conditions":{
            "reconciliation_coverage":{
                "coverage":0.95,
                "from_tip":true
            }
        }
    }
}