I am trying to understand the inner workings of Algorand, from the codebase it looks like the data is being written to SQL Lite and you can read the data using a standalone database tool, Indexer, that uses PostgresSQL. Is that correct?
If I were to run a private node what databases could I use in a node? Why was SQL Lite chosen?
The algorand node ( algod ) is using sqlite for its internal store.
Separately from that, the indexer, a completely independent process, knows how to communicate with the algorand node ( algod ) over REST API, query the block, and store it’s content to a Postgres database.
Sqlite was chosen as it provides excellent storage performance, embedded, lightweight and fault tolerant. Other databases might provide better indexing features - but these are not required for an algorand node.
I have a question regarding this topic. Running an (archival) node, this database is already pretty large on disk. What if Algorand grows 10x? Will we be required to have a 10 TB disk to store it? That doesn’t sound lightweight.
That’s interesting and indeed would help. If it weren’t for updates like the 2.8.0 where I need a full archival node again, I would do that. Is it foreseeable how often indexer does need to be rebuild?