Hi, I managed to run an archival node successfully and it’s fully synced. Now I wanted to run the indexer, but it’s taking a really long time to sync/import (on testnet for now). I get maybe 1 million rounds imported per day, which leads to more than 2 weeks sync time.
The thing is, neither the node seems particularly stressed, nor the indexer, nor the database. CPU is at ~10%, memory is still sufficient. So where is the bottleneck? Are there any options to speed things up? Why doesn’t it fetch faster?
1 Like
The biggest impact we’ve seen in the initial seeding of the DB is the latency between the indexer instance and the PostgreSQL database. The latency between the indexer and the algod instance is not as important, as long as it’s within reason.
If you’re in a cloud, even having these in different AZs makes a noticeable difference. Keep them in the same AZ if possible. We’ve also noticed that using an AWS “C5” instance makes a difference over the “t3” instance types, even though it doesn’t look like there’s much CPU being used. Same thing for an AWS PostgreSQL instance. Avoid the t3 instance types, at least for the initial loading.
If you’re running these on your own hardware, putting the indexer on the same VM as the database will speed things up dramatically. (Though mixing tasks on the same machine is certainly not a textbook deployment strategy.)
Also, the more TXNs per block, the longer it takes to import. There are a few parts of MainNet where there was a TON of activity, so those parts take longer to import. And as MainNet has been getting busier and busier, you’ll find the indexing slows down as you approach the later blocks.
3 Likes
Thank you for your insights!
I’m now really thinking about slapping indexer+db on the same machine for the initial import. Can you clarify how dramatic the speedup might be? I’d like 2x-3x as fast for the work to be worth it 
We never put the indexer on the same VM as the database. We’re running in AWS with Aurora-managed DBs, so that’s not an option. But I can tell you that having the updater even in a different AZ than the DB made a pretty large difference. Like a day or two of indexing time. And this was several months back that I had done this testing. So the effect is likely more pronounced now.
2 Likes