Can't fectch data from TestNet in SandBox

Sir · June 15, 2022, 12:39am

Hi all, please help to resolve the problem.

I started on WLS 2 (Windows Subsystem for Linux) and typed this command:
./sandbox up testnet

I get this:

Nothing happening there.

As I understand it can’t connect to TestNet as on section algod - goal node status i get last commited block 0.

What should I do in order to connect to TestNet and run my py scripts local?

fabrice · June 15, 2022, 3:45am

Welcome to Algorand!

What OS and specs do you have?
You need at least 4GB of RAM and a fast enough SSD.

Can you check you are on the latest version of sandbox (git pull)?
Can you reset your sandbox (./sandbox clean, ./sandbox reset)?

That being said, getting stuck at 0% may rather indicate a connectivity problem.
Are you behind a proxy? Are you in a network filtering DNS queries?
Can you run dig -t SRV _algobootstrap._tcp.testnet.algorand.network +short and show the output?
As well as: (with the expected answer below)

$ curl -s http://oregong-algorand-test-1.algorand-testnet.network:4161/v1/testnet-v1.0/block/0 | base64
gqVibG9ja4qkZmVlc8QgB9rLS22e0UGxdXa9RZrmQh1IbaPU7yJHxAmjlrguoiGjZ2VurHRlc3RuZXQtdjEuMKJnaMQgSGO1GKSzyE7IEPItTxCByw9x8FmnrCDexi9/cOUJOiKlcHJvdG/ZTmh0dHBzOi8vZ2l0aHViLmNvbS9hbGdvcmFuZC9zcGVjL3RyZWUvYTI2ZWQ3OGVkOGY4MzRlMmI5Y2NiNmViN2QzZWU5ZjYyOWE2ZTYyMqRyYXRlzg7msoCmcndjYWxyzgAHoSCjcndkxCD//////////////////////////////////////////6RzZWVkxCBIY7UYpLPITsgQ8i1PEIHLD3HwWaesIN7GL39w5Qk6IqJ0c85c/uwXo3R4bsQgJ3hisbLS0SebtaGdDYdRj+cVAPEmuLozZ3W9NJoee3OkY2VydIA=

(if dig or curl are not installed, please install the relevant Ubuntu packages dnsutils and curl)

Can you also provide the file sandbox.log and the output of ./sandbox logs? (at least the last 20 lines)

Sir · June 15, 2022, 4:23am

Hello Fabrice,
Thank you for the answer!

I have 32GB RAM.

I tried to use ./sandbox clean , ./sandbox reset and it didn’t help as I agree, it looks a connection problem. I pulled repo today so I guess it should be the latest version?

I don’t have a proxy or DNS filtering (as of what I know).

Output of

command:

1 1 4161 singaporeg-algorand-test-2.algorand-testnet.network.
1 1 4161 relay-virginia-testnet-algorand.algorand-testnet.network.
1 1 4161 singaporeg-algorand-test-1.algorand-testnet.network.
1 1 4161 londong-algorand-test-1.algorand-testnet.network.
1 1 4161 oregong-algorand-test-1.algorand-testnet.network.
1 1 80 relay-switzerland1-testnet-algorand.algorand-testnet.network.
1 1 4161 r19.algorand-testnet.network.
1 1 4161 londong-algorand-test-2.algorand-testnet.network.

Running this:

got this:

gqVibG9ja4qkZmVlc8QgB9rLS22e0UGxdXa9RZrmQh1IbaPU7yJHxAmjlrguoiGjZ2VurHRlc3Ru
ZXQtdjEuMKJnaMQgSGO1GKSzyE7IEPItTxCByw9x8FmnrCDexi9/cOUJOiKlcHJvdG/ZTmh0dHBz
Oi8vZ2l0aHViLmNvbS9hbGdvcmFuZC9zcGVjL3RyZWUvYTI2ZWQ3OGVkOGY4MzRlMmI5Y2NiNmVi
N2QzZWU5ZjYyOWE2ZTYyMqRyYXRlzg7msoCmcndjYWxyzgAHoSCjcndkxCD/////////////////
/////////////////////////6RzZWVkxCBIY7UYpLPITsgQ8i1PEIHLD3HwWaesIN7GL39w5Qk6
IqJ0c85c/uwXo3R4bsQgJ3hisbLS0SebtaGdDYdRj+cVAPEmuLozZ3W9NJoee3OkY2VydIA=

looks the same as yours I guess

Here’s log’s details:

#15 [sandbox_indexer 3/7] RUN apk add --no-cache git bzip2 make bash libtool boost-dev autoconf automake g++
#15 49.28 (77/87) Installing brotli-libs (1.0.9-r5)
#15 49.39 (78/87) Installing nghttp2-libs (1.46.0-r0)
#15 49.43 (79/87) Installing libcurl (7.80.0-r1)
#15 49.55 (80/87) Installing pcre2 (10.40-r0)
#15 49.65 (81/87) Installing git (2.34.2-r0)
#15 50.92 (82/87) Installing perl-error (0.17029-r1)
#15 50.97 (83/87) Installing perl-git (2.34.2-r0)
#15 51.01 (84/87) Installing git-perl (2.34.2-r0)
#15 51.26 (85/87) Installing libltdl (2.4.6-r7)
#15 51.29 (86/87) Installing libtool (2.4.6-r7)
#15 51.44 (87/87) Installing make (4.3-r0)
#15 51.50 Executing busybox-1.34.1-r3.trigger
#15 51.51 OK: 526 MiB in 101 packages
#15 DONE 54.9s

#17 [sandbox_algod 5/7] RUN /tmp/images/algod/install.sh -d “/node” -c “stable” -u “” -b “” -s “”
#17 DONE 44.5s

#18 [sandbox_indexer 4/7] COPY images/indexer/disabled.go /tmp/disabled.go
#18 DONE 0.1s

#19 [sandbox_indexer 5/7] COPY images/indexer/start.sh /tmp/start.sh
#19 DONE 0.1s

#20 [sandbox_algod 6/7] RUN sed -i “s/NETWORK_NUM_ROUNDS/30000/” “//tmp/images/algod/template.json” && /tmp/images/algod/setup.py --bin-dir “/node” --data-dir “/opt/data” --start-script “/opt/start_algod.sh” --network-dir “/opt/testnetwork” --network-template “//tmp/images/algod/template.json” --network-token “aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa” --algod-port “4001” --kmd-port “4002” --bootstrap-url “.algorand.network” --genesis-file “/tmp/genesis/testnet/genesis.json”
#20 …

#21 [sandbox_indexer 6/7] COPY images/indexer/install.sh /tmp/install.sh
#21 DONE 0.1s

#22 [sandbox_indexer 7/7] RUN /tmp/install.sh
#22 1.013 Missing BRANCH or URL environment variable. Skipping install.
#22 DONE 1.3s

#20 [sandbox_algod 6/7] RUN sed -i “s/NETWORK_NUM_ROUNDS/30000/” “//tmp/images/algod/template.json” && /tmp/images/algod/setup.py --bin-dir “/node” --data-dir “/opt/data” --start-script “/opt/start_algod.sh” --network-dir “/opt/testnetwork” --network-template “//tmp/images/algod/template.json” --network-token “aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa” --algod-port “4001” --kmd-port “4002” --bootstrap-url “.algorand.network” --genesis-file “/tmp/genesis/testnet/genesis.json”
#20 1.337 Configuring network with the following arguments:
#20 1.337 { ‘algod_port’: ‘4001’,
#20 1.337 ‘bin_dir’: ‘/node’,
#20 1.337 ‘bootstrap_url’: ‘.algorand.network’,
#20 1.337 ‘data_dir’: ‘/opt/data’,
#20 1.337 ‘genesis_file’: ‘/tmp/genesis/testnet/genesis.json’,
#20 1.337 ‘kmd_port’: ‘4002’,
#20 1.337 ‘network_dir’: ‘/opt/testnetwork’,
#20 1.337 ‘network_template’: ‘//tmp/images/algod/template.json’,
#20 1.337 ‘network_token’: ‘aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa’,
#20 1.337 ‘start_script’: ‘/opt/start_algod.sh’}
#20 1.337 Setting up real retwork.
#20 1.337 Start commands for /opt/start_algod.sh:
#20 1.337 [ ‘/node/goal node start -d /opt/testnetwork/Node’,
#20 1.337 ‘/node/kmd start -t 0 -d /opt/testnetwork/Node/kmd-v0.5’]
#20 1.337 Creating symlink /opt/data → /opt/testnetwork/Node
#20 DONE 1.5s

#23 [sandbox_algod 7/7] WORKDIR /opt/data
#23 DONE 0.1s

#24 [sandbox_indexer] exporting to image
#24 exporting layers
#24 exporting layers 2.3s done
#24 writing image sha256:ca34cd01ec162fb4c1752487b9b761d097e4ee6e99d1d8980c0a96860f94d5ed done
#24 naming to Docker done
#24 exporting layers 5.3s done
#24 writing image sha256:08afd9e88dcec0aa81bfc95fcadcc7e4f1899c2647bf4713546f670d84ba7fd1 done
#24 naming to Docker done
#24 DONE 5.3s

Use ‘docker scan’ to run Snyk tests against images to find vulnerabilities and learn how to fix them

docker-compose up -d
Network sandbox_default Creating
Network sandbox_default Created
Container algorand-sandbox-algod Creating
Container algorand-sandbox-postgres Creating
Container algorand-sandbox-algod Created
Container algorand-sandbox-postgres Created
Container algorand-sandbox-indexer Creating
Container algorand-sandbox-indexer Created
Container algorand-sandbox-algod Starting
Container algorand-sandbox-postgres Starting
Container algorand-sandbox-postgres Started
Container algorand-sandbox-algod Started
Container algorand-sandbox-indexer Starting
Container algorand-sandbox-indexer Started

Hope it will help to resolve the issue

fabrice · June 15, 2022, 2:57pm

That looks good. Connection to DNS seems to work.
Let’s however check that there is nothing strange with dnssec and 8.8.8.8 (which is used by sandbox):

$ dig -t SRV _algobootstrap._tcp.testnet.algorand.network @8.8.8.8 +short +dnssec

Can you also try:

./sandbox logs and show the end
./sandbox status and show it fully
enter in algod sandbox ./sandbox enter algod and show the end of node.log

Sir · June 15, 2022, 3:43pm

Hello Fabrice,
Ok, I did your command

Here’s the outcome:

1 1 4161 singaporeg-algorand-test-1.algorand-testnet.network.
1 1 4161 r19.algorand-testnet.network.
1 1 4161 londong-algorand-test-2.algorand-testnet.network.
1 1 4161 relay-virginia-testnet-algorand.algorand-testnet.network.
1 1 4161 londong-algorand-test-1.algorand-testnet.network.
1 1 4161 singaporeg-algorand-test-2.algorand-testnet.network.
1 1 4161 oregong-algorand-test-1.algorand-testnet.network.
1 1 80 relay-switzerland1-testnet-algorand.algorand-testnet.network.

1. Running sundbox logs and here’s the outcome:

WARN[0000] The “INDEXER_ENABLE_ALL_PARAMETERS” variable is not set. Defaulting to a blank string.
Watching file: /opt/data/node.log…
could not decode line from JSON: ++++++++++++++++++++++++++++++++++++++++
could not decode line from JSON: Logging Starting
could not decode line from JSON: Telemetry Disabled
could not decode line from JSON: ++++++++++++++++++++++++++++++++++++++++
04:11:43.951 : RoundConcluded - |
04:11:43.986 : RoundConcluded - |
04:11:47.987 1.0.1: StepTimeout - |
04:11:47.987 : ProposalFrozen AAAAA- 1.0.0|
04:11:49.091 : RoundConcluded - |
04:11:49.092 : RoundConcluded - |
04:11:53.093 1.0.1: StepTimeout - |
04:11:53.093 : ProposalFrozen AAAAA- 1.0.0|
04:12:06.093 1.0.2: StepTimeout - |
04:12:06.094 1.0.3: VoteAttest AAAAA- |
04:12:08.092 1.0.3: StepTimeout - |
04:12:08.646 1.0.4: VoteAttest AAAAA- |
04:12:12.092 1.0.4: StepTimeout - |
04:12:19.770 1.0.5: VoteAttest AAAAA- |
04:12:20.092 1.0.5: StepTimeout - |
04:12:22.977 1.0.6: VoteAttest AAAAA- |
04:12:36.092 1.0.6: StepTimeout - |
04:12:39.067 1.0.7: VoteAttest AAAAA- |
could not decode line from JSON: ++++++++++++++++++++++++++++++++++++++++
could not decode line from JSON: Logging Starting
could not decode line from JSON: Telemetry Disabled
could not decode line from JSON: ++++++++++++++++++++++++++++++++++++++++
15:27:43.343 : RoundConcluded - |
15:27:43.357 : RoundConcluded - |
15:27:43.357 : RoundConcluded - |
15:27:47.358 1.0.1: StepTimeout - |
15:27:47.358 : ProposalFrozen AAAAA- 1.0.0|
15:27:47.575 : RoundConcluded - |
15:27:47.575 : RoundConcluded - |
15:27:51.576 1.0.1: StepTimeout - |
15:27:51.576 : ProposalFrozen AAAAA- 1.0.0|
15:28:04.576 1.0.2: StepTimeout - |
15:28:04.577 1.0.3: VoteAttest AAAAA- |
15:28:06.577 1.0.3: StepTimeout - |
15:28:10.537 1.0.4: VoteAttest AAAAA- |
15:28:10.575 1.0.4: StepTimeout - |
15:28:17.190 1.0.5: VoteAttest AAAAA- |
15:28:18.576 1.0.5: StepTimeout - |
15:28:26.013 1.0.6: VoteAttest AAAAA- |
15:28:34.576 1.0.6: StepTimeout - |
15:28:58.983 1.0.7: VoteAttest AAAAA- |
15:29:06.575 1.0.7: StepTimeout - |
15:29:55.732 1.0.8: VoteAttest AAAAA- |

2. Sandbox status:

algod - goal node status
WARN[0000] The “INDEXER_ENABLE_ALL_PARAMETERS” variable is not set. Defaulting to a blank string.
Last committed block: 0
Time since last block: 0.0s
Sync Time: 0.0s
Last consensus protocol: REF:github.com/algorand/spec/tree/a26ed78ed8f834e2b9ccb6eb7d3ee9f629a6e622
Next consensus protocol: REF:github.com/algorand/spec/tree/a26ed78ed8f834e2b9ccb6eb7d3ee9f629a6e622
Round for next consensus protocol: 1
Next consensus protocol supported: true
Last Catchpoint:
Genesis ID: testnet-v1.0
Genesis hash: SGO1GKSzyE7IEPItTxCByw9x8FmnrCDexi9/cOUJOiI=

indexer - health
Indexer disabled for this configuration.

3. Ran node.log and hope it’s correct outcome:

{“Context”:“sync”,“details”:{“StartRound”:0,“EndRound”:0,“Time”:224600,“InitSync”:false},“file”:“telemetry.go”,“function”:“github.com/algorand/go-algorand/logging.(*telemetryState).logTelemetry",“instanceName”:“hC7H3kq4QR28ckIv”,“level”:“info”,“line”:261,“msg”:“/ApplicationState/CatchupStop”,“name”:“”,“session”:“”,“time”:“2022-06-15T15:31:11.695150Z”,“v”:"3.7.2”}
{“Context”:“sync”,“file”:“service.go”,“function”:“github.com/algorand/go-algorand/catchup.(*Service).sync",“level”:“info”,“line”:634,“msg”:"Catchup Service: finished catching up, now at round 0 (previously 0). Total time catching up 224.6µs.”,“name”:“”,“time”:“2022-06-15T15:31:11.695253Z”}

Thank you in advance for your assistance!

P.S. Since I’m new I had to replace https:// for REF in section #2 as I couldn’t save answer with more than 2 links.

fabrice · June 15, 2022, 4:07pm

It is very strange.

Can you upload the full node.log on github gists for example?

But maybe the best is to completely reset docker (removing everything there) and clone again the sandbox.

Sir · June 15, 2022, 4:24pm

I tried to do this yestreday, deleted everythin and then pulled again sandbox. I can do that again, but don’t think it would make any changes.

Here it is:

gist.github.com

https://gist.github.com/LookInFuture/6d3c6569521e1debe987117d23685664

node.log

++++++++++++++++++++++++++++++++++++++++
Logging Starting
Telemetry Disabled
++++++++++++++++++++++++++++++++++++++++
{"file":"trackerdb.go","function":"github.com/algorand/go-algorand/ledger.trackerDBInitializeImpl","level":"info","line":127,"msg":"trackerDBInitialize upgrading database schema from version 0 to version 6","name":"","time":"2022-06-15T04:11:43.881011Z"}
{"file":"trackerdb.go","function":"github.com/algorand/go-algorand/ledger.trackerDBInitializeImpl","level":"info","line":131,"msg":"trackerDBInitialize performing upgrade from version 0","name":"","time":"2022-06-15T04:11:43.881116Z"}
{"file":"trackerdb.go","function":"github.com/algorand/go-algorand/ledger.(*trackerDBSchemaInitializer).upgradeDatabaseSchema0","level":"info","line":215,"msg":"upgradeDatabaseSchema0 initializing schema","name":"","time":"2022-06-15T04:11:43.881163Z"}
{"file":"trackerdb.go","function":"github.com/algorand/go-algorand/ledger.trackerDBInitializeImpl","level":"info","line":131,"msg":"trackerDBInitialize performing upgrade from version 1","name":"","time":"2022-06-15T04:11:43.883248Z"}
{"file":"trackerdb.go","function":"github.com/algorand/go-algorand/ledger.trackerDBInitializeImpl","level":"info","line":131,"msg":"trackerDBInitialize performing upgrade from version 2","name":"","time":"2022-06-15T04:11:43.883294Z"}
{"file":"trackerdb.go","function":"github.com/algorand/go-algorand/ledger.trackerDBInitializeImpl","level":"info","line":131,"msg":"trackerDBInitialize performing upgrade from version 3","name":"","time":"2022-06-15T04:11:43.883340Z"}

This file has been truncated. show original

Hooe we can figure it out!

Sir · June 15, 2022, 4:26pm

I want to provide additional info.
I have the file config.testnet:

export ALGOD_CHANNEL=“stable”
export ALGOD_URL=“”
export ALGOD_BRANCH=“”
export ALGOD_SHA=“”
export NETWORK=“testnet”
export NETWORK_BOOTSTRAP_URL=“.algorand.network”
export NETWORK_GENESIS_FILE=“genesis/testnet/genesis.json”
export INDEXER_URL=“”
export INDEXER_BRANCH=“”
export INDEXER_SHA=“”
export INDEXER_DISABLED=“true”
export INDEXER_ENABLE_ALL_PARAMETERS=“false”

I added the last row as without it I would be getting this warn:

WARN[0000] The “INDEXER_ENABLE_ALL_PARAMETERS” variable is not set. Defaulting to a blank string.

But either way if I put false or true, it doesn’t change anything.

fabrice · June 15, 2022, 8:30pm

I see errors with DNSSec in the logs.
And now I see that the last dig command does not return a DNSSec sig.
It looks like your Internet provider or something on your network is blocking DNSSec.

Can you try to modify config.json inside ./sandbox enter algod to disable DNSSec:
that is add something like

  "DNSSecurityFlags": 0

in it.

Topic		Replies	Views
Sandbox Having Issues TestNet testnet	1	418	November 12, 2022
Testnet doesn't synchronize TestNet testnet	3	546	January 10, 2021
Fast-catchup don't complete TestNet testnet	5	730	July 11, 2022
Sandbox testnet fast catchup stuck at processing accounts 0% TestNet	2	1000	February 19, 2022
Sandbox doesn't sync with Testnet TestNet testnet	4	797	February 8, 2021

Can't fectch data from TestNet in SandBox

Related topics