Node online but keep getting "database round 205862 is behind in-memory round 16927043"

General
1

What does this mean? My local node is fully caught up.

{"file":"acctupdates.go","function":"github.com/algorand/go-algorand/ledger.(*accountUpdates).lookupWithRewards","level":"error","line":1834,"msg":"[Stack] goroutine 129 [running]:\nruntime/debug.Stack(0x853ca180, 0x851867a8, 0x8a4c02c0)\n\truntime/debug/stack.go:24 +0x80\ngithub.com/algorand/go-algorand/logging.logger.Errorf(0x853ca180, 0x851867a8, 0xb6221d81, 0x50, 0x8a9d93f0, 0x2, 0x2)\n\tgithub.com/algorand/go-algorand/logging/log.go:229 +0x30\ngithub.com/algorand/go-algorand/ledger.(*accountUpdates).lookupWithRewards(0x8519e980, 0x1024945, 0x0, 0x521068f, 0xf11767c3, 0xabab4a78, 0x266fe0d6, 0x7f75a325, 0x61babe50, 0x1194a0eb, ...)\n\tgithub.com/algorand/go-algorand/ledger/acctupdates.go:1834 +0x824\ngithub.com/algorand/go-algorand/ledger.(*accountUpdates).LookupWithRewards(...)\n\tgithub.com/algorand/go-algorand/ledger/acctupdates.go:409\ngithub.com/algorand/go-algorand/ledger.(*Ledger).Lookup(0x8519e700, 0x1024945, 0x0, 0x521068f, 0xf11767c3, 0xabab4a78, 0x266fe0d6, 0x7f75a325, 0x61babe50, 0x1194a0eb, ...)\n\tgithub.com/algorand/go-algorand/ledger/ledger.go:444 +0xdc\ngithub.com/algorand/go-algorand/agreement.membership(0xb6856ce0, 0x8536ef40, 0x521068f, 0xf11767c3, 0xabab4a78, 0x266fe0d6, 0x7f75a325, 0x61babe50, 0x1194a0eb, 0x12661939, ...)\n\tgithub.com/algorand/go-algorand/agreement/selector.go:67 +0x1f8\ngithub.com/algorand/go-algorand/agreement.unauthenticatedVote.verify(0x521068f, 0xf11767c3, 0xabab4a78, 0x266fe0d6, 0x7f75a325, 0x61babe50, 0x1194a0eb, 0x12661939, 0x1024a85, 0x0, ...)\n\tgithub.com/algorand/go-algorand/agreement/vote.go:91 +0xc4\ngithub.com/algorand/go-algorand/agreement.(*AsyncVoteVerifier).executeVoteVerification(0x8509ac60, 0xb67a2880, 0x88a5b000, 0x2, 0x2)\n\tgithub.com/algorand/go-algorand/agreement/asyncVoteVerifier.go:106 +0x238\ngithub.com/algorand/go-algorand/util/execpool.(*pool).worker(0x85071140)\n\tgithub.com/algorand/go-algorand/util/execpool/pool.go:159 +0xf8\ncreated by github.com/algorand/go-algorand/util/execpool.MakePool\n\tgithub.com/algorand/go-algorand/util/execpool/pool.go:85 +0x170\n","name":"","time":"2021-09-28T15:26:50.347730-07:00"}
{"file":"acctupdates.go","function":"github.com/algorand/go-algorand/ledger.(*accountUpdates).lookupWithRewards","level":"error","line":1834,"msg":"accountUpdates.lookupWithRewards: database round 205862 is behind in-memory round 16927043","name":"","time":"2021-09-28T15:26:50.347857-07:00"}
{"file":"acctupdates.go","function":"github.com/algorand/go-algorand/ledger.(*accountUpdates).lookupWithRewards","level":"error","line":1834,"msg":"[Stack] goroutine 129 [running]:\nruntime/debug.Stack(0x853ca180, 0x851867a8, 0x8a4c0440)\n\truntime/debug/stack.go:24 +0x80\ngithub.com/algorand/go-algorand/logging.logger.Errorf(0x853c^C

Local node:

pi@raspberrypi:~/node $ ./goal node status -d ~/node/mainnetdata -w 1000
Last committed block: 16508328
Time since last block: 0.3s
Sync Time: 0.0s
Last consensus protocol: https://github.com/algorandfoundation/specs/tree/abc54f79f9ad679d2d22f0fb9909fb005c16f8a1
Next consensus protocol: https://github.com/algorandfoundation/specs/tree/abc54f79f9ad679d2d22f0fb9909fb005c16f8a1
Round for next consensus protocol: 16508329
Next consensus protocol supported: true
Last Catchpoint:
Genesis ID: mainnet-v1.0
Genesis hash: wGHE2Pwdvd7S12BL5FaOP20EGYesN73ktiC1qzkkit8=
2

Is your node an archival node or a relay?
Did you use fast catchup?

If you answer yes to both questions, this is most likely the issue: fast catchup cannot be used with an archival node or a relay.

3

The issue was that my local env var was set to a slightly outdated catchup, I manually set the catchup to use the latest and Im guessing this will fix it. Thanks for the help fabrice :slight_smile:

4

Hmm this didnt seem to solve my issue. My node is caught up with the latest block but Im still getting the error in the title:

pi@raspberrypi:~/node $ ./goal node status -d ~/node/mainnetdata -w 1000
Last committed block: 16525128
Time since last block: 4.3s
Sync Time: 0.0s
Last consensus protocol: https://github.com/algorandfoundation/specs/tree/abc54f79f9ad679d2d22f0fb9909fb005c16f8a1
Next consensus protocol: https://github.com/algorandfoundation/specs/tree/abc54f79f9ad679d2d22f0fb9909fb005c16f8a1
Round for next consensus protocol: 16525129
Next consensus protocol supported: true
Last Catchpoint:
Genesis ID: mainnet-v1.0
Genesis hash: wGHE2Pwdvd7S12BL5FaOP20EGYesN73ktiC1qzkkit8=
5

Just to update here, Nick popped into the discord with this issue and didn’t yet have any disk attached. Will try again once he does.

6

Yep, thanks barnji bra

1 Like