How to increase the speed of Algorand mainnet data loading

Hi,

I started the mainnet data of Algorand to be loaded in archival mode on a 3 TB disk of a VM machine which has 8 CPU and 32 GB RAM. In 2 days it became 165GB. But after 13 days more it can only reach 318 GB. How can I make this process faster? The indexer also works in parallel.

Is there a variable that should be set to a different value in config.json file to make this process faster?

Hope to hear from the support soon.

The outputs of the commands to check the status and settings in config.json are here:

$ goal node status
Last committed block: 13401207
Time since last block: 2.9s
Sync Time: 1322034.2s
Last consensus protocol: GitHub - algorandfoundation/specs at 3a83c4c743f8b17adfd73944b4319c25722a6782
Next consensus protocol: GitHub - algorandfoundation/specs at ac2255d586c4474d4ebcf3809acccb59b7ef34ff
Round for next consensus protocol: 13409876
Next consensus protocol supported: true
Last Catchpoint:
Genesis ID: mainnet-v1.0
Genesis hash: wGHE2Pwdvd7S12BL5FaOP20EGYesN73ktiC1qzkkit8=

$ goal version -v
Version: [v1 v2]
GenesisID: mainnet-v1.0
Build: 3.5.1.stable [rel/stable] (commit #aa2fb0ee)
ubuntu@algorand-node:~$ du -hs $ALGORAND_DATA/*
577M /mnt/algorand/agreement.cdv
1.1G /mnt/algorand/agreement.cdv.archive
4.0K /mnt/algorand/algod.admin.token
0 /mnt/algorand/algod.lock
4.0K /mnt/algorand/algod.net
4.0K /mnt/algorand/algod.pid
4.0K /mnt/algorand/algod.token
4.0K /mnt/algorand/config.json
28K /mnt/algorand/genesis.json
4.0K /mnt/algorand/goal.cache
4.0K /mnt/algorand/logging.config
du: cannot read directory ‘/mnt/algorand/mainnet-v1.0’: Permission denied
4.0K /mnt/algorand/mainnet-v1.0
1.1G /mnt/algorand/node.archive.log
806M /mnt/algorand/node.log

$ sudo du -hs /mnt/algorand
313G /mnt/algorand

$ cat $ALGORAND_DATA/config.json
{
“Version”: 19,
“AccountUpdatesStatsInterval”: 5000000000,
“AccountsRebuildSynchronousMode”: 1,
“AnnounceParticipationKey”: true,
“Archival”: true,
“BaseLoggerDebugLevel”: 4,
“BlockServiceCustomFallbackEndpoints”: “”,
“BroadcastConnectionsLimit”: -1,
“CadaverSizeTarget”: 1073741824,
“CatchpointFileHistoryLength”: 365,
“CatchpointInterval”: 10000,
“CatchpointTracking”: 0,
“CatchupBlockDownloadRetryAttempts”: 1000,
“CatchupBlockValidateMode”: 0,
“CatchupFailurePeerRefreshRate”: 10,
“CatchupGossipBlockFetchTimeoutSec”: 4,
“CatchupHTTPBlockFetchTimeoutSec”: 4,
“CatchupLedgerDownloadRetryAttempts”: 50,
“CatchupParallelBlocks”: 4,
“ConnectionsRateLimitingCount”: 60,
“ConnectionsRateLimitingWindowSeconds”: 1,
“DNSBootstrapID”: “.algorand.network”,
“DNSSecurityFlags”: 1,
“DeadlockDetection”: 0,
“DisableLocalhostConnectionRateLimit”: true,
“DisableNetworking”: false,
“DisableOutgoingConnectionThrottling”: false,
“EnableAccountUpdatesStats”: false,
“EnableAgreementReporting”: false,
“EnableAgreementTimeMetrics”: false,
“EnableAssembleStats”: false,
“EnableBlockService”: false,
“EnableBlockServiceFallbackToArchiver”: true,
“EnableCatchupFromArchiveServers”: false,
“EnableDeveloperAPI”: false,
“EnableGossipBlockService”: true,
“EnableIncomingMessageFilter”: false,
“EnableLedgerService”: false,
“EnableMetricReporting”: false,
“EnableOutgoingNetworkMessageFiltering”: true,
“EnablePingHandler”: true,
“EnableProcessBlockStats”: false,
“EnableProfiler”: false,
“EnableRequestLogger”: false,
“EnableTopAccountsReporting”: false,
“EnableVerbosedTransactionSyncLogging”: false,
“EndpointAddress”: “172.23.149.211:4001”,
“FallbackDNSResolverAddress”: “”,
“ForceFetchTransactions”: false,
“ForceRelayMessages”: false,
“GossipFanout”: 4,
“IncomingConnectionsLimit”: 800,
“IncomingMessageFilterBucketCount”: 5,
“IncomingMessageFilterBucketSize”: 512,
“IsIndexerActive”: true,
“LedgerSynchronousMode”: 2,
“LogArchiveMaxAge”: “”,
“LogArchiveName”: “node.archive.log”,
“LogSizeLimit”: 1073741824,
“MaxCatchpointDownloadDuration”: 7200000000000,
“MaxConnectionsPerIP”: 30,
“MinCatchpointFileDownloadBytesPerSecond”: 20480,
“NetAddress”: “”,
“NetworkMessageTraceServer”: “”,
“NetworkProtocolVersion”: “”,
“NodeExporterListenAddress”: “:9100”,
“NodeExporterPath”: “./node_exporter”,
“OptimizeAccountsDatabaseOnStartup”: false,
“OutgoingMessageFilterBucketCount”: 3,
“OutgoingMessageFilterBucketSize”: 128,
“ParticipationKeysRefreshInterval”: 60000000000,
“PeerConnectionsUpdateInterval”: 3600,
“PeerPingPeriodSeconds”: 0,
“PriorityPeers”: {},
“ProposalAssemblyTime”: 250000000,
“PublicAddress”: “”,
“ReconnectTime”: 60000000000,
“ReservedFDs”: 256,
“RestReadTimeoutSeconds”: 15,
“RestWriteTimeoutSeconds”: 120,
“RunHosted”: false,
“SuggestedFeeBlockHistory”: 3,
“SuggestedFeeSlidingWindowSize”: 50,
“TLSCertFile”: “”,
“TLSKeyFile”: “”,
“TelemetryToLog”: true,
“TransactionSyncDataExchangeRate”: 0,
“TransactionSyncSignificantMessageThreshold”: 0,
“TxPoolExponentialIncreaseFactor”: 2,
“TxPoolSize”: 15000,
“TxSyncIntervalSeconds”: 60,
“TxSyncServeResponseSize”: 1000000,
“TxSyncTimeoutSeconds”: 30,
“UseXForwardedForAddressField”: “”,
“VerifiedTranscationsCacheSize”: 30000
}

$ iostat
Linux 5.4.0-105-generic (algorand-node-and-kafka) 04/20/22 x86_64 (8 CPU)

avg-cpu: %user %nice %system %iowait %steal %idle
8.41 0.00 2.71 8.86 0.13 79.89

Device tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd
loop0 0.01 0.01 0.00 0.00 32466 0 0
loop1 0.00 0.00 0.00 0.00 3604 0 0
loop2 0.00 0.00 0.00 0.00 3129 0 0
loop3 0.00 0.00 0.00 0.00 3645 0 0
loop4 0.01 0.01 0.00 0.00 13905 0 0
loop5 0.03 0.03 0.00 0.00 67308 0 0
loop6 0.00 0.00 0.00 0.00 3650 0 0
loop7 0.00 0.00 0.00 0.00 13 0 0
vda 12.87 13.42 327.12 0.00 32053649 781264694 0
vdc 1039.83 45.77 9275.26 0.00 109312677 22152265092 0

Algorand #node-runners

from @Urtho 4/25/22:
"try adding

        "AccountsRebuildSynchronousMode": 0,
        "LedgerSynchronousMode": 0,

to the config.json and see if this speeds things up"

I’m having the same issue. If you do make this change, I recommend saving a copy of the config.json so you can revert if it doesn’t do anything to help:

sudo cp config.json config.json.bak
sudo nano config.json # or vi or whatever - make your changes
sudo systemctl stop algorand
sudo systemctl daemon-reload
sudo systemctl start algorand