Hi all,
I am trying to run my project on mainnet through sandbox.
I have to run “./sandbox up mainnet” before starting my server.
If I dont do it I get null for information that I am trying to fetch from blockchain.
It also takes too long to catchup like atleast 2-3 hours or so
Is there any solution for this?
Can you provide sandbox.log?
Can you show what you see when doing ./sandbox up mainnet and what steps take too long?
Catching up with mainnet requires an SSD, at least 4-8GB of RAM, and a fast enough Internet connection (at least 10Mbps) due to the amount of transactions on MainNet.
If you’re using an HDD or an SD card, this won’t work.
* docker compose up -d
Building algod
Sending build context to Docker daemon 139.3kB
Step 1/25 : ARG GO_VERSION=1.17.5
Step 2/25 : FROM golang:$GO_VERSION
---> 276895edf967
Step 3/25 : ARG CHANNEL=nightly
---> Running in e8c253fe8bf1
Removing intermediate container e8c253fe8bf1
---> 8d9359c7284a
Step 4/25 : ARG URL=
---> Running in 3198e4b9311e
Removing intermediate container 3198e4b9311e
---> b3ed888df0f5
Step 5/25 : ARG BRANCH=
---> Running in f6774d7685b1
Removing intermediate container f6774d7685b1
---> e00831a5fb21
Step 6/25 : ARG SHA=
---> Running in 58b78b99d328
Removing intermediate container 58b78b99d328
---> b400f1bfcbde
Step 7/25 : ARG GENESIS_FILE=""
---> Running in b4115fe33b53
Removing intermediate container b4115fe33b53
---> 0c744ea7a599
Step 8/25 : ARG BOOTSTRAP_URL=""
---> Running in 61a4965d1e26
Removing intermediate container 61a4965d1e26
---> a07c2e6f9095
Step 9/25 : ARG ALGOD_PORT=""
---> Running in 42a89469eda1
Removing intermediate container 42a89469eda1
---> ae32024f7d4e
Step 10/25 : ARG KMD_PORT=""
---> Running in 0207623b67e2
Removing intermediate container 0207623b67e2
---> 41960fbf5c39
Step 11/25 : ARG TOKEN=""
---> Running in a0370057552f
Removing intermediate container a0370057552f
---> 1be8bbdabf2a
Step 12/25 : ARG TEMPLATE=""
---> Running in d15704e56311
Removing intermediate container d15704e56311
---> 632094aef1ec
Step 13/25 : ARG NETWORK_NUM_ROUNDS="30000"
---> Running in 930f80cd7aa2
Removing intermediate container 930f80cd7aa2
---> 630bd1f8a4f0
Step 14/25 : RUN echo "Installing from source. ${URL} -- ${BRANCH}"
---> Running in de56721a4a48
Installing from source. --
Removing intermediate container de56721a4a48
---> 9f29eea3fb7f
Step 15/25 : ENV BIN_DIR="$HOME/node"
---> Running in c4c53e3c677c
Removing intermediate container c4c53e3c677c
---> 1246d8909d5a
Step 16/25 : ENV ALGORAND_DATA="/opt/data"
---> Running in f867e9fea97a
Removing intermediate container f867e9fea97a
---> 576ea5cdbf92
Step 17/25 : ENV HOME /opt
---> Running in f1e0ff2054f9
Removing intermediate container f1e0ff2054f9
---> e80cee48204f
Step 18/25 : ENV DEBIAN_FRONTEND noninteractive
---> Running in 51083c013ba9
Removing intermediate container 51083c013ba9
---> 05cfa4a41033
Step 19/25 : RUN apt-get update && apt-get install -y apt-utils curl git git-core bsdmainutils python3
---> Running in a5617aaebd16
Get:1 http://security.debian.org/debian-security bullseye-security InRelease [48.4 kB]
Get:2 http://deb.debian.org/debian bullseye InRelease [116 kB]
Get:3 http://deb.debian.org/debian bullseye-updates InRelease [44.1 kB]
Get:4 http://security.debian.org/debian-security bullseye-security/main amd64 Packages [175 kB]
Get:5 http://deb.debian.org/debian bullseye/main amd64 Packages [8182 kB]
Get:6 http://deb.debian.org/debian bullseye-updates/main amd64 Packages [2592 B]
Fetched 8567 kB in 1s (6163 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
git is already the newest version (1:2.30.2-1).
python3 is already the newest version (3.9.2-3).
python3 set to manually installed.
The following additional packages will be installed:
bsdextrautils libcurl4 ncal
Suggested packages:
calendar whois vacation mailutils
The following NEW packages will be installed:
apt-utils bsdextrautils bsdmainutils ncal
The following packages will be upgraded:
curl libcurl4
2 upgraded, 4 newly installed, 0 to remove and 48 not upgraded.
Need to get 1246 kB of archives.
After this operation, 1693 kB of additional disk space will be used.
Get:1 http://deb.debian.org/debian bullseye/main amd64 bsdextrautils amd64 2.36.1-8+deb11u1 [145 kB]
Get:2 http://deb.debian.org/debian bullseye/main amd64 apt-utils amd64 2.2.4 [439 kB]
Get:3 http://security.debian.org/debian-security bullseye-security/main amd64 curl amd64 7.74.0-1.3+deb11u2 [270 kB]
Get:4 http://deb.debian.org/debian bullseye/main amd64 ncal amd64 12.1.7+nmu3 [30.1 kB]
Get:5 http://deb.debian.org/debian bullseye/main amd64 bsdmainutils all 12.1.7+nmu3 [16.4 kB]
Get:6 http://security.debian.org/debian-security bullseye-security/main amd64 libcurl4 amd64 7.74.0-1.3+deb11u2 [345 kB]
e[91mdebconf: delaying package configuration, since apt-utils is not installed
e[0mFetched 1246 kB in 0s (36.8 MB/s)
Selecting previously unselected package bsdextrautils.
(Reading database ...
(Reading database ... 5%
(Reading database ... 10%
(Reading database ... 15%
(Reading database ... 20%
(Reading database ... 25%
(Reading database ... 30%
(Reading database ... 35%
(Reading database ... 40%
(Reading database ... 45%
(Reading database ... 50%
(Reading database ... 55%
(Reading database ... 60%
(Reading database ... 65%
(Reading database ... 70%
(Reading database ... 75%
(Reading database ... 80%
(Reading database ... 85%
(Reading database ... 90%
(Reading database ... 95%
(Reading database ... 100%
(Reading database ... 15709 files and directories currently installed.)
Preparing to unpack .../0-bsdextrautils_2.36.1-8+deb11u1_amd64.deb ...
Unpacking bsdextrautils (2.36.1-8+deb11u1) ...
Selecting previously unselected package apt-utils.
Preparing to unpack .../1-apt-utils_2.2.4_amd64.deb ...
Unpacking apt-utils (2.2.4) ...
Selecting previously unselected package ncal.
Preparing to unpack .../2-ncal_12.1.7+nmu3_amd64.deb ...
Unpacking ncal (12.1.7+nmu3) ...
Selecting previously unselected package bsdmainutils.
Preparing to unpack .../3-bsdmainutils_12.1.7+nmu3_all.deb ...
Unpacking bsdmainutils (12.1.7+nmu3) ...
Preparing to unpack .../4-curl_7.74.0-1.3+deb11u2_amd64.deb ...
Unpacking curl (7.74.0-1.3+deb11u2) over (7.74.0-1.3+deb11u1) ...
Preparing to unpack .../5-libcurl4_7.74.0-1.3+deb11u2_amd64.deb ...
Unpacking libcurl4:amd64 (7.74.0-1.3+deb11u2) over (7.74.0-1.3+deb11u1) ...
Setting up apt-utils (2.2.4) ...
Setting up bsdextrautils (2.36.1-8+deb11u1) ...
update-alternatives: using /usr/bin/write.ul to provide /usr/bin/write (write) in auto mode
Setting up ncal (12.1.7+nmu3) ...
Setting up bsdmainutils (12.1.7+nmu3) ...
Setting up libcurl4:amd64 (7.74.0-1.3+deb11u2) ...
Setting up curl (7.74.0-1.3+deb11u2) ...
Processing triggers for libc-bin (2.31-13+deb11u2) ...
Removing intermediate container a5617aaebd16
---> 0bb40cde354d
Step 20/25 : COPY . /tmp
---> 52b0652af33e
Step 21/25 : RUN /tmp/images/algod/install.sh -d "${BIN_DIR}" -c "${CHANNEL}" -u "${URL}" -b "${BRANCH}" -s "${SHA}"
---> Running in 8d1231585109
/tmp/images/algod /go
Installing algod with options:
BINDIR = /node
CHANNEL = stable
URL =
BRANCH =
SHA =
Current Version = 0
updater binary was installed at /tmp/images/algod/updater
Latest Version = 12885426177
New version found
Checking for files matching: 'channel/stable/node_stable_linux-amd64_' in bucket algorand-releases
Update Downloaded to /tmp/tmp.2KtBs8pQL1/12885426177.tar.gz
Expanding update...
Validating update...
Starting the new update script to complete the installation...
... Resuming installation from the latest update script
Current Version = 0
Backing up current binary files...
Backing up current data files from /node/data...
Installing new binary files...
Installing new data files into /node/data...
Copying genesis files locally
Checking for new ledger in /node/data
e[91mCannot read genesis file /node/data/genesis.json: open /node/data/genesis.json: no such file or directory
e[0mUpdating genesis files for default network
New genesis ID, resetting wallets
New Ledger - restoring genesis accounts in /node/data
New Ledger - importing rootkeys for genesis accounts
Imported 0 keys
Applying migration fixups...
Deleting existing log files in /node/data
Install complete - restart node manually
Removing intermediate container 8d1231585109
---> 826366fdfcd9
Step 22/25 : RUN sed -i "s/NETWORK_NUM_ROUNDS/$NETWORK_NUM_ROUNDS/" "//tmp/${TEMPLATE}" && /tmp/images/algod/setup.py --bin-dir "$BIN_DIR" --data-dir "/opt/data" --start-script "/opt/start_algod.sh" --network-dir "/opt/testnetwork" --network-template "//tmp/${TEMPLATE}" --network-token "${TOKEN}" --algod-port "${ALGOD_PORT}" --kmd-port "${KMD_PORT}" --bootstrap-url "${BOOTSTRAP_URL}" --genesis-file "/tmp/${GENESIS_FILE}"
---> Running in 67fe3b734fb1
Configuring network with the following arguments:
{ 'algod_port': '4001',
'bin_dir': '/node',
'bootstrap_url': '<network>.algorand.network',
'data_dir': '/opt/data',
'genesis_file': '/tmp/genesis/mainnet/genesis.json',
'kmd_port': '4002',
'network_dir': '/opt/testnetwork',
'network_template': '//tmp/images/algod/template.json',
'network_token': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa',
'start_script': '/opt/start_algod.sh'}
Setting up real retwork.
Start commands for /opt/start_algod.sh:
[ '/node/goal node start -d /opt/testnetwork/Node',
'/node/kmd start -t 0 -d /opt/testnetwork/Node/kmd-v0.5']
Creating symlink /opt/data -> /opt/testnetwork/Node
Removing intermediate container 67fe3b734fb1
---> 123fe02d0be6
Step 23/25 : ENV PATH="$BIN_DIR:${PATH}"
---> Running in ce28365fae8d
Removing intermediate container ce28365fae8d
---> cf19fa645f36
Step 24/25 : WORKDIR /opt/data
---> Running in ab67db1c55ec
Removing intermediate container ab67db1c55ec
---> 6028869f6350
Step 25/25 : CMD ["/opt/start_algod.sh"]
---> Running in 5e6585d2e93a
Removing intermediate container 5e6585d2e93a
---> ea9c95df7707
[Warning] One or more build-args [CDT_PORT] were not consumed
Successfully built ea9c95df7707
Successfully tagged sandbox_algod:latest
Image for service algod was built because it did not already exist. To rebuild this image you must use `docker-compose build` or `docker-compose up --build`.
Building indexer
Sending build context to Docker daemon 139.3kB
Step 1/14 : ARG GO_VERSION=1.17.5
Step 2/14 : FROM golang:$GO_VERSION-alpine
---> d8bf44a3f6b4
Step 3/14 : ARG URL=https://github.com/algorand/indexer
---> Running in 04d8be80a5c2
Removing intermediate container 04d8be80a5c2
---> 5f086809dfe9
Step 4/14 : ARG BRANCH=master
---> Running in d81398171938
Removing intermediate container d81398171938
---> e34b08bfb34e
Step 5/14 : ARG SHA=""
---> Running in 525283230f94
Removing intermediate container 525283230f94
---> 88e4511c59ab
Step 6/14 : ENV HOME /opt/indexer
---> Running in 91870f72a6ce
Removing intermediate container 91870f72a6ce
---> 09760714542f
Step 7/14 : WORKDIR /opt/indexer
---> Running in d09ab8681852
Removing intermediate container d09ab8681852
---> e32886dba397
Step 8/14 : ENV DEBIAN_FRONTEND noninteractive
---> Running in c71a58c75dd8
Removing intermediate container c71a58c75dd8
---> f0126a4f68a6
Step 9/14 : RUN apk add --no-cache git bzip2 make bash libtool boost-dev autoconf automake g++
---> Running in 1f07376ec3e8
fetch https://dl-cdn.alpinelinux.org/alpine/v3.15/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.15/community/x86_64/APKINDEX.tar.gz
(1/87) Upgrading zlib (1.2.11-r3 -> 1.2.12-r3)
(2/87) Installing m4 (1.4.18-r2)
(3/87) Installing libbz2 (1.0.8-r1)
(4/87) Installing perl (5.34.0-r1)
(5/87) Installing autoconf (2.71-r0)
(6/87) Installing automake (1.16.4-r1)
(7/87) Installing ncurses-terminfo-base (6.3_p20211120-r1)
(8/87) Installing ncurses-libs (6.3_p20211120-r1)
(9/87) Installing readline (8.1.1-r0)
(10/87) Installing bash (5.1.16-r0)
Executing bash-5.1.16-r0.post-install
(11/87) Installing boost1.77-atomic (1.77.0-r1)
(12/87) Installing libgcc (10.3.1_git20211027-r0)
(13/87) Installing boost1.77-chrono (1.77.0-r1)
(14/87) Installing boost1.77-container (1.77.0-r1)
(15/87) Installing boost1.77-context (1.77.0-r1)
(16/87) Installing boost1.77-contract (1.77.0-r1)
(17/87) Installing boost1.77-thread (1.77.0-r1)
(18/87) Installing boost1.77-coroutine (1.77.0-r1)
(19/87) Installing boost1.77-date_time (1.77.0-r1)
(20/87) Installing boost1.77-fiber (1.77.0-r1)
(21/87) Installing boost1.77-filesystem (1.77.0-r1)
(22/87) Installing boost1.77-graph (1.77.0-r1)
(23/87) Installing xz-libs (5.2.5-r1)
(24/87) Installing boost1.77-iostreams (1.77.0-r1)
(25/87) Installing boost1.77-locale (1.77.0-r1)
(26/87) Installing boost1.77-log (1.77.0-r1)
(27/87) Installing boost1.77-log_setup (1.77.0-r1)
(28/87) Installing boost1.77-math (1.77.0-r1)
(29/87) Installing boost1.77-prg_exec_monitor (1.77.0-r1)
(30/87) Installing boost1.77-program_options (1.77.0-r1)
(31/87) Installing expat (2.4.7-r0)
(32/87) Installing libffi (3.4.2-r1)
(33/87) Installing gdbm (1.22-r0)
(34/87) Installing libstdc++ (10.3.1_git20211027-r0)
(35/87) Installing mpdecimal (2.5.1-r1)
(36/87) Installing sqlite-libs (3.36.0-r0)
(37/87) Installing python3 (3.9.13-r1)
(38/87) Installing boost1.77-python3 (1.77.0-r1)
(39/87) Installing boost1.77-random (1.77.0-r1)
(40/87) Installing boost1.77-regex (1.77.0-r1)
(41/87) Installing boost1.77-serialization (1.77.0-r1)
(42/87) Installing boost1.77-stacktrace_basic (1.77.0-r1)
(43/87) Installing boost1.77-stacktrace_noop (1.77.0-r1)
(44/87) Installing boost1.77-system (1.77.0-r1)
(45/87) Installing boost1.77-timer (1.77.0-r1)
(46/87) Installing boost1.77-type_erasure (1.77.0-r1)
(47/87) Installing boost1.77-unit_test_framework (1.77.0-r1)
(48/87) Installing boost1.77-wave (1.77.0-r1)
(49/87) Installing boost1.77-wserialization (1.77.0-r1)
(50/87) Installing boost1.77-json (1.77.0-r1)
(51/87) Installing boost1.77-nowide (1.77.0-r1)
(52/87) Installing boost1.77-libs (1.77.0-r1)
(53/87) Installing boost1.77 (1.77.0-r1)
(54/87) Installing linux-headers (5.10.41-r0)
(55/87) Installing bzip2-dev (1.0.8-r1)
(56/87) Installing icu-libs (69.1-r1)
(57/87) Installing icu (69.1-r1)
(58/87) Installing pkgconf (1.8.0-r0)
(59/87) Installing icu-dev (69.1-r1)
(60/87) Installing xz-dev (5.2.5-r1)
(61/87) Installing zlib-dev (1.2.12-r3)
(62/87) Installing boost1.77-dev (1.77.0-r1)
(63/87) Installing boost-dev (1.77.0-r1)
(64/87) Installing bzip2 (1.0.8-r1)
(65/87) Installing binutils (2.37-r3)
(66/87) Installing libgomp (10.3.1_git20211027-r0)
(67/87) Installing libatomic (10.3.1_git20211027-r0)
(68/87) Installing libgphobos (10.3.1_git20211027-r0)
(69/87) Installing gmp (6.2.1-r1)
(70/87) Installing isl22 (0.22-r0)
(71/87) Installing mpfr4 (4.1.0-r0)
(72/87) Installing mpc1 (1.2.1-r0)
(73/87) Installing gcc (10.3.1_git20211027-r0)
(74/87) Installing musl-dev (1.2.2-r7)
(75/87) Installing libc-dev (0.7.2-r3)
(76/87) Installing g++ (10.3.1_git20211027-r0)
(77/87) Installing brotli-libs (1.0.9-r5)
(78/87) Installing nghttp2-libs (1.46.0-r0)
(79/87) Installing libcurl (7.80.0-r2)
(80/87) Installing pcre2 (10.40-r0)
(81/87) Installing git (2.34.4-r0)
(82/87) Installing perl-error (0.17029-r1)
(83/87) Installing perl-git (2.34.4-r0)
(84/87) Installing git-perl (2.34.4-r0)
(85/87) Installing libltdl (2.4.6-r7)
(86/87) Installing libtool (2.4.6-r7)
(87/87) Installing make (4.3-r0)
Executing busybox-1.34.1-r3.trigger
OK: 527 MiB in 101 packages
Removing intermediate container 1f07376ec3e8
---> 3f44baf6e97a
Step 10/14 : COPY images/indexer/disabled.go /tmp/disabled.go
---> 3302b684bc22
Step 11/14 : COPY images/indexer/start.sh /tmp/start.sh
---> 88af46dd14e8
Step 12/14 : COPY images/indexer/install.sh /tmp/install.sh
---> 01ce3b1f6067
Step 13/14 : RUN /tmp/install.sh
---> Running in 2b0a49615708
Missing BRANCH or URL environment variable. Skipping install.
Removing intermediate container 2b0a49615708
---> 3ce145997154
Step 14/14 : CMD ["/tmp/start.sh"]
---> Running in fdb603332843
Removing intermediate container fdb603332843
---> f73f91a14d41
Successfully built f73f91a14d41
Successfully tagged sandbox_indexer:latest
Image for service indexer was built because it did not already exist. To rebuild this image you must use `docker-compose build` or `docker-compose up --build`.
Creating algorand-sandbox-algod ...
Creating algorand-sandbox-postgres ...
Creating algorand-sandbox-algod ... done
Creating algorand-sandbox-postgres ... done
Creating algorand-sandbox-indexer ...
Creating algorand-sandbox-indexer ... done
This takes very long…and I have to do it everytime before starting my application server…
If I dont i get null as response
Indeed, it looks like the fast catchup itself is very slow on your computer.
Usually this is because: you don’t have enough RAM (you need 4-8GB), you are not using a fast enough SSD (SD and HDD cards are not sufficient), your Internet connection is too slow.
Are you in any of these three cases?
Yeah looks like RAM is not enough. I am using AWS virtual machine so its not as per you have mentioned. My internet is good.
I’ll check if I can get a better congif for catchup to run.