How to Write a Configuration File for rosetta-cli Testing

Prerequisites

• For the check:construction test, a Rosetta DSL (*.ros) file.
• For the check:data test, a way to measure account balances.
  • If your blockchain supports historical balance lookup, you'll need to set the historical_balance_disabled value to false.
  • If your blockchain does not support historical balance lookup, you'll need to complete the following actions:
    • Set the historical_balance_disabled value to true.
    • Provide a bootstrap_balances.json file for the bootstrap_balances value.

Write the Configuration File

1. From the command line, create a config.json file in the directory of your choice.
   • For example, you can create the rosetta-cli-conf/testnet directory to test your configuration with testnet.
2. Edit the config.json file and add the fields and objects, with their corresponding values necessary for your implementation. For more information on these values, see the Rosetta-CLI Configuration documentation and the End Conditions documentation.
3. Save the file.
4. Run the rosetta-cli tool for testing.

List of Fields and Objects for the Configuration File

The configuration file must be written in JSON format. Do not add optional fields with empty or null values to the configuration file.

Fields

The fields below are part of the JSON file object.

Fields	Type	Description	Possible Values	Example
online_url	string	The URL of a Rosetta API implementation in “online” mode.	A valid url.	http://localhost:8080
data_directory	string	A folder used to store logs and any validation data.	An absolute or relative path to where the rosetta-cli binary is being executed.	cli-data
http_timeout	number	The timeout for an HTTP request, in seconds.	A positive integer.	300
max_retries	number	The number of tries that the rosetta-cli tool will retry an HTTP request.
If the retry_elapsed_time field is also populated, the rosetta-cli tool may stop attempting retries early.	A positive integer.	15
retry_elapsed_time	number	The total time to spend retrying an HTTP request, in seconds.
If the max_retries field is also populated, the rosetta-cli tool may stop attempting retries early.	A positive integer.	0
max_online_connections	number	The maximum number of open connections that the online fetcher will open.	A positive integer.	500
force_retry	Boolean	Whether to override the default retry handling to retry on all error (non-200) responses.	true false
Omit if empty.	true
max_sync_concurrency	number	The maximum sync concurrency to use while syncing blocks. Sync concurrency is managed automatically by the syncer package.	A positive integer.	64
tip_delay	number	Dictates how many seconds behind the current time is considered the tip. If the rosetta-cli tool is less than the number of tip_delay seconds from the last processed block, then it is considered to be “behind tip”.	A positive integer.	120
max_reorg_depth	number	Specifies the maximum possible reorganization depth of the blockchain being synced. This value is used to determine how aggressively to prune old block data.	A positive integer.
Omit if empty.	100
log_configuration	Boolean	Determines whether the configuration settings should be printed to the console when a file is loaded.	true false	false
compression_disabled	Boolean	Whether to configure the storage layer to not perform data compression before writing to disk.
This leads to significantly more on-disk storage usage but can lead to performance gains.	true false	true
memory_limit_disabled	Boolean	Whether to configure storage to increase memory usage. Enabling this value massively increases performance, but can use tens of GBs of RAM, even with pruning enabled.	true false	true
seen_block_workers	number	The number of goroutines spawned to store seen blocks in storage before we attempt to sequence.
This value defaults to the runtime.NumCPU() value.	A positive integer. Default - the runtime.NumCPU() value.	empty (uses the default value)
serial_block_workers	number	The number of goroutines spawned to help with block sequencing (i.e., updating balances, updating coins, etc).
This value defaults to the runtime.NumCPU() value.	A positive integer. Default - the runtime.NumCPU() value.	empty (uses the default value)
validation_file	string	The field used for asset-specific validation.
This value defaults to an empty string, which means that no asset-specific validation will be done.	A string null	empty (uses the default value)

Objects

The following objects are part of the JSON file object: network, construction, and data. Objects that are part of the construction, and data objects are listed under those headings.

network

Read the NetworkIdentifier documentation for more information.

construction

The configuration values needed to run the check:construction test in the rosetta-cli tool.

Code Sample

Contains the following fields and objects:

Fields	Type	Description	Possible Values	Example
offline_url	string	The URL of a Rosetta API implementation in “offline mode”.
This field is part of the construction object.	A valid URL.	http://example.com
max_offline_connections	number	The maximum number of open connections that the offline fetcher will open.
This field is part of the construction object.	A positive number.
force_retry	Boolean	Whether to override the default retry handling to retry on all error (non-200) responses.
This field is part of the construction object.	true false
Omit if empty.
stale_depth	number	The number of blocks to wait before attempting to rebroadcast after not finding a transaction on-chain.
This field is part of the construction object.	A positive integer.	3
broadcast_limit	number	The number of times to attempt re-broadcast before giving up on a transaction broadcast.
This field is part of the construction object.	A positive integer.	5
ignore_broadcast_failures	Boolean	Whether the tool should exit when there are broadcast failures that surpass the broadcast_limit value.
This field is part of the construction object.	true false	false
clear_broadcasts	Boolean	Whether all pending broadcasts should be removed from the stored broadcast file on restart.
This field is part of the construction object.	true false
broadcast_behind_tip	Boolean	Whether the tool should broadcast transactions when it is behind tip (as defined in the tip_delay value).
This field is part of the construction object.	true false
block_broadcast_limit	number	The number of transactions to attempt to broadcast in a single block. When there are many pending broadcasts, it may make sense to limit the number of broadcasts.
This field is part of the construction object.	A positive integer.
rebroadcast_all	Boolean	Whether all pending broadcasts should be rebroadcast from the stored broadcast file on restart.
This field is part of the construction object.	true false
output_path	string	The path to the directory where you would like to store the output files that rosetta-cli creates.

The default value for this field is /cli-data. This field is part of the construction object. | A valid relative path. | cli-data | | constructor_dsl_file | string | The DSL (*.ros) file for the check:construction test. This field is part of the construction object. | A .ros filename. | ethereum.ros | | status_port | number | The port where you can query a running check:construction test to get progress statistics. This process can be used instead of parsing logs to populate a status dashboard. This field is part of the construction object. | A positive integer. Omit if empty. | | | results_output_file | string | The absolute file path where the rosetta-cli tool will save the results of a check:construction test run. This field is part of the construction object. | A valid filepath. Omit if empty. | /directory/construction_test_results | | quiet | Boolean | Whether to silence all request and response logging. This field is part of the construction object. |

• true
• false

Omit if empty. | | | initial_balance_fetch_disabled | Boolean | Configures the rosetta-cli tool to not look up the balance of newly seen accounts at the genesis block before applying operations. Disabling this field is a good idea only if you create multiple new accounts each block and don’t fund any accounts before starting the check:construction test. This is a separate configuration value from the initial_balance_fetch_disabled value under the data object. The construction object value is usually set to false, whereas the data object value is usually set to true. This field is part of the construction object. |

• true
• false

| | | prefunded_accounts | object | Loads prefunded addresses into field storage. This object is part of the construction object. | Contains the privkey field and the account_identifier, curve_type, and currency objects. Omit if empty. | | | end_conditions | object | The conditions that need to be fulfilled to successfully end the check:construction test.

This object is part of the construction object. | Contains the create_account and transfer fields. | |

end_conditions (construction)

This object is part of the construction object. The conditions that need to be fulfilled to successfully end the check:construction test. Contains the create_account and transfer fields.

Code Sample

Fields	Type	Description	Possible Values	Example
create_account	number	The number of accounts for the test to create.
This field is part of the end_conditions object.	A positive integer.	10
transfer	number	The number of transfers for the test to complete.
This field is part of the end_conditions object.	A positive integer.	20

prefunded_accounts

This object is part of the construction object. Loads prefunded addresses into field storage. Contains the privkey and curve_type fields and the account_identifier and currency objects. If you have no values for the prefunded_account object, you can omit it from the construction object.

Code Sample

Fields	Type	Description	Possible Values	Example
privkey	string	A private key.
This field is part of the prefunded_accounts object.	A valid private key.
account_identifier	object	Read the AccountIdentifier documentation for more information.
curve_type	string	Read the CurveType documentation for more information.
currency	object	Read the Currency documentation for more information.

data

Contains all the configurations needed to run the check:data test.

Code Sample

Contains the following fields and objects:

Fields	Type	Description	Possible Values	Example
active_reconciliation_concurrency	number	The concurrency to use while fetching accounts during active reconciliation.
This field is part of the data object.	A positive integer.	32
inactive_reconciliation_concurrency	number	The concurrency to use while fetching accounts during inactive reconciliation.
This field is part of the data object.	A positive integer.
inactive_reconciliation_frequency	number	The number of blocks to wait between inactive reconciliations on each account.
This field is part of the data object.	A positive integer.
log_blocks	Boolean	Whether to log processed blocks.
This field is part of the data object.	true false
log_transactions	Boolean	Whether to log processed transactions.	true false
log_balance_changes	Boolean	Whether to log all balance changes.
This field is part of the data object.	true false
log_reconciliations	Boolean	Whether to log all reconciliations.
This field is part of the data object.	true false
ignore_reconciliation_error	Boolean	Whether block processing should halt on a reconciliation error.

You can either collect all reconciliation errors, or silence reconciliation errors during development. This field is part of the data object. |

• true
• false

| | | exempt_accounts | string | The JSON file that lists all accounts exempt from balance tracking and reconciliation. You can find an example of this file in the rosetta-cli/examples directory in GitHub. This field is part of the data object. | A valid path and filename, relative to the location of this configuration file. | exempt_accounts.json | | bootstrap_balances | string | The JSON file that contains the initial asset or token balance. You can find an example of this file in the rosetta-cli/examples directory in GitHub. Note: If you populate this value after syncing begins, the test will ignore it. This field is part of the data object. | A valid path and filename, relative to the location of this configuration file. | bootstrap_balances.json | | historical_balance_disabled | Boolean | Whether to look up a balance at the block where a balance change occurred, instead of at the current block. Note: Blockchains that do not support historical balance lookup should set this value to true and include a bootstrap_balances.json file for the bootstrap_balances value. If this value is true or omitted, the check:data test will ignore the inactive_discrepancy_search_disabled field. This field is part of the data object. |

• true
• false

Omit if empty. | | | interesting_accounts | string | The JSON file that contains all of the accounts to check on each block. You can find an example of this file in the rosetta-cli/examples directory in GitHub. This field is part of the data object. | A valid path and filename, relative to the location of this configuration file. | interesting_accounts.json | | reconciliation_disabled | Boolean | Whether or not to attempt reconciliation. To confirm that syncing is working as expected, we recommend that you disable some of the more advanced checks when first testing an implementation. This field is part of the data object. |

• true
• false

| | | reconciliation_drain_disabled | Boolean | If reconciliation is enabled, whether the rosetta-cli tool should exit the check:data test before the entire active reconciliation queue has been drained. This field is part of the data object. |

• true
• false

| | | inactive_discrepancy_search_disabled | Boolean | Whether the check:data test should perform a search to find any inactive reconciliation discrepancies. Note: If the historical_balance_disabled value is true or omitted, the check:data test will ignore this field. This field is part of the data object. |

• true
• false

| | | balance_tracking_disabled | Boolean | Whether to attempt a balance calculation. We recommend that you try to fetch all blocks before checking for balance consistency when first testing an implementation. This field is part of the data object. |

• true
• false

| | | coin_tracking_disabled | Boolean | Whether to attempt coin or UTX0 tracking. We recommend that you try to fetch all blocks before checking for coin consistency when first testing an implementation. This field is part of the data object. |

• true
• false

| | | start_index | number | The block height from where the check:data test should start syncing. Note: If you do not provide a value for this field, syncing will start from the last saved block. If no blocks have been synced previously, then syncing will start from the genesis block. This field is part of the data object. | A positive integer. Omit if empty. | | | status_port | number | The port where you can query a running check:data test to get progress statistics. This process can be used instead of parsing logs to populate a status dashboard. This field is part of the data object. | A positive integer. Omit if empty. | | | results_output_file | string | The absolute file path where the rosetta-cli tool will save the results of a check:data run. This field is part of the data object. | A valid filepath. | /directory/data_test_results | | pruning_disabled | Boolean | Whether to not attempt storage pruning. Note: You should only set this value to true if you wish to use the start_index field at a later point to restart the check:data test from a previously synced block. This field is part of the data object. |

• true
• false

| | | pruning_frequency | number | The frequency, in seconds, that the check:data test attempts to prune blocks. The default value for this field is the statefulsyncer package’s value. This field is part of the data object. | A positive integer. Omit if empty. | | | initial_balance_fetch_disabled | Boolean | Configures the rosetta-cli tool to not look up the balance of newly seen accounts at the genesis block before applying operations. Note: Disabling this field can speed up performance without impacting validation accuracy, as long as the boostrap_balances file provides all the genesis accounts syncing begins from genesis. This is a separate configuration value from the initial_balance_fetch_disabled value under the construction object. The data object’s value is usually set to true, whereas the construction object’s value is usually set to false. |

• true
• false

| true | | reconciler_active_backlog | number | The maximum number of pending changes to store in the active reconciliation backlog before skipping reconciliation on new changes. This field is part of the data object. | A positive integer. Omit if empty. | | | end_conditions | object | Required. The conditions that need to be fulfilled to successfully end the check:data test. This object is part of the data object. | Contains the index, tip, and duration fields, and the reconciliation_coverage object. | |

end_conditions (data)

This object is part of the data object. The conditions that need to be fulfilled to successfully end the check:data test. Contains the index, tip, and duration fields, and the reconciliation_coverage object.

Code Sample

Fields	Type	Description	Possible Values	Example
index	number	Configures the syncer to stop once it reaches a particular block height.
This field is part of the end_conditions object.	A positive integer.
Omit if empty.
tip	Boolean	Configures the syncer to stop once it reaches the tip of the blockchain.

You must also configure the tip_delay value if you set the tip value to true. This field is part of the end_conditions object. |

• true
• false

Omit if empty. | | | duration | number | The number of seconds the syncer must run before it stops. This field is part of the end_conditions object. | A positive integer. Omit if empty. | | | reconciliation_coverage | object | Configures the syncer to stop once it reaches the reconciliation coverage set in this object. This object is part of the end_conditions object. | Contains the coverage, from_tip, tip, index, and account_count fields. | |

reconciliation_coverage

This object is part of the data object's end_conditions object. Configures the syncer to stop once it reaches the reconciliation coverage set in this object. Contains the coverage, from_tip, tip, index, and account_count fields.

Code Sample

Fields	Type	Description	Possible Values	Example
coverage	number	A value that represents the percentage of reconciled accounts.
This field is part of the reconciliation_coverage object.	A positive decimal that’s between 0.0 and 1.0.	0.95
from_tip	Boolean	Whether the reconciliation coverage should be measured from the tip of the blockchain.

This field is part of the reconciliation_coverage object. Omit if empty. |

• true
• false

| true | | tip | Boolean | Whether the tip must be reached before reconciliation coverage is considered valid. This field is part of the reconciliation_coverage object. |

• true
• false

Omit if empty. | | | index | number | The height that must be reached before reconciliation coverage is considered valid. This field is part of the reconciliation_coverage object. Omit if empty. | A positive integer. | | | account_count | number | The number of accounts that must be observed before reconciliation coverage is considered valid. This field is part of the reconciliation_coverage object. Omit if empty. | A positive integer. | |

Sample Configuration files

The following are sample configuration files for default, ethereum, and bitcoin.

Sample default.json file

{
    "network":{
        "blockchain":"MyBlockchain",
        "network":"testnet"
    },
    "online_url":"http://localhost:8080",
    "data_directory":"",
    "http_timeout":10,
    "max_retries":5,
    "retry_elapsed_time":0,
    "max_online_connections":120,
    "max_sync_concurrency":64,
    "tip_delay":300,
    "max_reorg_depth":100,
    "log_configuration":false,
    "compression_disabled":false,
    "memory_limit_disabled":false,
    "construction":null,
    "data":{
        "active_reconciliation_concurrency":16,
        "inactive_reconciliation_concurrency":4,
        "inactive_reconciliation_frequency":250,
        "log_blocks":false,
        "log_transactions":false,
        "log_balance_changes":false,
        "log_reconciliations":false,
        "ignore_reconciliation_error":false,
        "exempt_accounts":"",
        "bootstrap_balances":"",
        "interesting_accounts":"",
        "reconciliation_disabled":false,
        "reconciliation_drain_disabled":false,
        "inactive_discrepancy_search_disabled":false,
        "balance_tracking_disabled":false,
        "coin_tracking_disabled":false,
        "status_port":9090,
        "results_output_file":"",
        "pruning_disabled":false,
        "initial_balance_fetch_disabled":false
    }
}

Sample config.json file for Ethereum

{
    "network":{
        "blockchain":"Ethereum",
        "network":"Ropsten"
    },
    "data_directory":"cli-data",
    "http_timeout":300,
    "max_retries":15,
    "max_online_connections":500,
    "max_sync_concurrency":64,
    "tip_delay":120,
    "compression_disabled":true,
    "memory_limit_disabled":true,
    "construction":{
        "stale_depth":3,
        "broadcast_limit":5,
        "constructor_dsl_file":"ethereum.ros",
        "end_conditions":{
            "create_account":10,
            "transfer":20
        }
    },
    "data":{
        "initial_balance_fetch_disabled":true,
        "active_reconciliation_concurrency":32,
        "bootstrap_balances":"bootstrap_balances.json",
        "end_conditions":{
            "reconciliation_coverage":{
                "coverage":0.95,
                "from_tip":true
            }
        }
    }
}

Sample config.json file for Bitcoin

{
    "network":{
        "blockchain":"Bitcoin",
        "network":"Testnet3"
    },
    "data_directory":"cli-data",
    "http_timeout":300,
    "max_retries":5,
    "max_online_connections":1000,
    "tip_delay":1800,
    "memory_limit_disabled":true,
    "compression_disabled":true,
    "construction":{
        "constructor_dsl_file":"bitcoin.ros",
        "end_conditions":{
            "create_account":10,
            "transfer":10
        }
    },
    "data":{
        "initial_balance_fetch_disabled":true,
        "end_conditions":{
            "reconciliation_coverage":{
                "coverage":0.95,
                "from_tip":true
            }
        }
    }
}