I start my node but after a short time its stops and i have to restart it. I am running on a Pi 4B with Raspian Buster. I will be checking the status and i notice the node has stopped running. I have it running on a dedicated ip through my router. Sometimes it’s hours other times its less than 30 minutes.
here is some more info I did a force update but its still giving the same results.
pi@postgres:~/node $ ./goal node status -d data -w 1000
Last committed block: 13274294
Time since last block: 8.5s
Sync Time: 162.7s
Last consensus protocol: GitHub - algorandfoundation/specs at 3a83c4c743f8b17adfd73944b4319c25722a6782
Next consensus protocol: GitHub - algorandfoundation/specs at ac2255d586c4474d4ebcf3809acccb59b7ef34ff
Round for next consensus protocol: 13409876
Next consensus protocol supported: true
Last Catchpoint:
Genesis ID: mainnet-v1.0
Genesis hash: wGHE2Pwdvd7S12BL5FaOP20EGYesN73ktiC1qzkkit8=
after a few seconds this happens
Cannot contact Algorand node: Get “http://127.0.0.1:8080/v2/status”: read tcp 127.0.0.1:42996->127.0.0.1:8080: read: connection reset by peer
Can you post the end of the log data/node.log?
Can you check RAM use?
Can you check algod was not killed because of OOM (do dmesg -T | egrep -i 'killed process')?
What command should i use to get the node.log file open so I can copy and paste it for you? I tried opening the file in the text editor but it does not work
Ram Use
total used free shared buff/cache available
Mem: 3827 358 2253 106 1215 3236
Swap: 99 85 14
HDD status
Filesystem Size Used Avail Use% Mounted on
/dev/root 470G 16G 435G 4% /
devtmpfs 1.8G 0 1.8G 0% /dev
tmpfs 1.9G 44M 1.9G 3% /dev/shm
tmpfs 1.9G 8.6M 1.9G 1% /run
tmpfs 5.0M 4.0K 5.0M 1% /run/lock
tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup
/dev/mmcblk0p1 253M 48M 205M 19% /boot
tmpfs 383M 8.0K 383M 1% /run/user/1000
MemTotal: 3919128 kB
MemFree: 2283892 kB
MemAvailable: 3292340 kB
Buffers: 13560 kB
Cached: 1231556 kB
SwapCached: 5596 kB
Active: 301796 kB
Inactive: 1173588 kB
Active(anon): 91060 kB
Inactive(anon): 263372 kB
Active(file): 210736 kB
Inactive(file): 910216 kB
Unevictable: 74852 kB
Mlocked: 16 kB
HighTotal: 3264512 kB
HighFree: 1706448 kB
LowTotal: 654616 kB
LowFree: 577444 kB
SwapTotal: 102396 kB
SwapFree: 16592 kB
Dirty: 124 kB
Writeback: 0 kB
AnonPages: 299664 kB
Mapped: 233244 kB
Shmem: 124164 kB
KReclaimable: 15452 kB
Slab: 37400 kB
SReclaimable: 15452 kB
SUnreclaim: 21948 kB
KernelStack: 2768 kB
PageTables: 13408 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 2061960 kB
Committed_AS: 2774364 kB
VmallocTotal: 245760 kB
VmallocUsed: 5376 kB
VmallocChunk: 0 kB
Percpu: 512 kB
CmaTotal: 262144 kB
CmaFree: 243796 kB
Out of memory: Killed process 2076 (mousepad) total-vm:1883304kB, anon-rss:1529396kB, file-rss:14108kB, shmem-rss:4kB, UID:1000 pgtables:3160kB oom_score_adj:0
To show the end of node.log, run tail -n 25 data/node.log.
You can paste the result there between triple backquotes ```…``` to have better formatting.
But it looks like the issue is that you run out of RAM (OOM).
Can you add a bit of swap (https://raspberrypi.stackexchange.com/a/1605)?
E.g., 4GB of swap?
Also are you running many processes beyond Algorand?
{“Context”:“Agreement”,“Hash”:"",“ObjectPeriod”:0,“ObjectRound”:0,“ObjectStep”:0,“Period”:0,“Round”:13252113,“Sender”:"",“Step”:0,“Type”:“RoundInterrupted”,“Weight”:0,“WeightTotal”:0,“file”:“demux.go”,“function”:“github.com/algorand/go-algorand/agreement.(*demux).next”,“level”:“info”,“line”:285,“msg”:“agreement: round 13252113 ended early due to concurrent write; next round is 13252114”,“time”:“2021-04-18T16:01:02.610354-04:00”}
{“callee”:“github.com/algorand/go-algorand/data/account.Participation.DeleteOldKeys.func1.1",“caller”:“github.com/algorand/go-algorand/data/account/participation.go:95”,“file”:“dbutil.go”,“function”:"github.com/algorand/go-algorand/util/db.(*Accessor).atomic”,“level”:“warning”,“line”:344,“msg”:“dbatomic: tx surpassed expected deadline by 1.380898664s”,“readonly”:false,“time”:“2021-04-18T16:01:03.263725-04:00”}
the only thing running on this is my Algo Node and indexer
Can you check that you have the latest version:
$ goal version -v -d data
Version: [v1 v2]
GenesisID: mainnet-v1.0
Build: 2.5.4.stable [rel/stable] (commit #7cd7aef0)
If you have the latest version, my guess is unfortunately that 4GB for archival algod + indexer will be a bit too tight.
Can you try to stop the indexer and see if algod works in that setting?
whats the command to stop the indexer?
Try:
pkill algorand-indexer
and check it is not running anymore using
ps augx | grep indexer
or using top / htop.
i ran both commands and this was the result
pi 3759 0.0 0.0 7348 524 pts/0 S+ 16:40 0:00 grep --color=auto indexer
i restarted node and got the same result after about 1 min it terminates.
Can you run htop in a different terminal and sort by memory use?
Then start the node, and take a screenshot when memory start being used a lot?
Did you check goal version -v?
Can you also post again goal node start -d data?
Version: [v1 v2]
GenesisID: mainnet-v1.0
Build: 2.5.4.stable [rel/stable] (commit #7cd7aef0)
This seems good. Can you get a screenshot closer to a crash?
I tried to get a spike when all were maxed out but missed it will keep trying till I get one.
I think the issue seems to be that 4GB of RAM is not sufficient.
Can you add 4GB of swap via https://raspberrypi.stackexchange.com/a/1605?
1 Like
I’m rather new to using the pi just curious if you could help me out with the proper command lines to run. I will try increasing the swap file.