TestNet Release: January 28


We successfully upgraded the network today, here is the changelog:

Major changes

  • Agreement service rewritten to run all safety-critical logic non-concurrently and simplify correctness validation and testing
  • Account wallets are now managed by kmd (key management daemon) - rootkey files deprecated; genesis wallets will be automatically imported into kmd’s storage
  • algod no longer has the ability to sign transactions, only concensus-related participation messages (proposals, votes)
  • multi-sig support added to allow N of M signatures to validate
  • Added algoh as optional host of algod for monitoring health and reporting problems
  • REST API simplified and rewritten - sensitive information no longer accessible, versioning and logging middleware added
  • Many networking-related fixes and improvements
  • Non-archival node support implemented - by default only Relays store the entire blockchain (Archival); non-archival only store ~1 day’s worth
  • Significant rewrite of portions of the ledger implementation; transaction logic centralized, block processing pipelined, balance tracking optimized
  • Offline rewards implementation rewritten
  • Transaction pool periodically syncs with peers using bloom filter to reduce chance of txn being lost

Minor Changes

  • Added 100 offline .01% stake wallets
  • Update telemetry index: Prefix channel to genesis ID
  • Fixed few more missing db-handle closing (tests)
  • Change kmd to use TCP
  • Change kmd API to use json
  • Moving genesis data to config files (removed genesis binary from install)
  • Genesis stake now a float64 instead of int (percentage still) allowing fine-grained allocations.
  • genesis.json files now have account sorted
  • add metrics for agreement handle vs drop rate
  • increase buffer (channel size) for incoming votes
  • removed old rewards implementation
  • Allow dumping signed transactions from goal clerk send
  • REST API reduced and rewritten using swagger; swagger.json generated from code.
  • Changed goal clerk multisig UX; add Delete endpoint to KMD
  • fix goal generatetoken
  • simplify agreement networking
  • make agreement inbound able to drop messages, but also add some buffering
  • fix goal clerk format string
  • add pending transactions endpoint to algod REST API. Add associated command to goal node.
  • Improve carpenter to work with new protocol code.
  • fix changeonlinestatus error handling
  • Removed account locks
  • Add master derivation secret + hkdf to kmd wallets
  • fix agreement starving network bug
  • added generation and size-limiting of agreement logs (.cdv)
  • goal logging send updated to include cdv and algoh logs
  • kmd: Stop using BIP39, allow going from key -> mnemonic, remove mnemonic endpoints from kmd
  • Add “goal account newmultisig”
  • Give “goal clerk send” the ability to dump raw transactions to a file instead of broadcasting them
  • don’t block on signalling the P2PNode that a peer has closed.
  • Add peer garbage collection to the maintainConnectivity thread.
  • added more networking metrics
  • Added BlockStats telemetry event to Algoh
  • on disconnecting a bad peer, close(), wait for the close, then remove from list. This should decrease thrashing and potential DoS vector.
  • add lock to walletdriver singleton to avoid duplicate name/id races
  • telemetry udpated to identify distinct installtions on hosts with multiple nodes
  • add extension to metrics Counter which is much faster for counting ints
  • telemetry updated to scale better; limits placed on goroutines and MaxQueueDepth
  • Implemented / enabled non-archival node support; have Archival == IsRelay for now
  • Fixed decoder issues caught by security audit - add a DecodeValidator interface to catch nil decodings
  • improved db perf and behavior - make sqlite3 properly wait for locks on a shared database
  • enable deadlock detection in production
  • clean up genesis.json to make it more readable
  • added metrics for the ledger and crypto packages
  • Added goal network start -n to allow starting a node within the network (specifying the current relay IP addresses)
  • Added goal network restart to allow restarting the entire network (stop -> start)
  • Goal now redirects algod output to algod-err.log and algod-out.log so runtime panics are caught
  • Updated to latest GO compiler; required adding fallback for SRV lookup on some platforms

Thanks Will!

Other notes:

  • Dashboard has been updated, but not all features are currently supported by the new REST API
  • Dispenser has been updated and is working again for testnet
  • Documentation for the new kmd REST API will be coming soon - in the next update, as there will be some modifications to support swagger
  • Client libraries to encapsulate the logic around orchestrating calls to kmd and algod are also coming soon

Dashboard now back to almost 100% including telemetry-based connected nodes / map, so we have a more accurate picture of where all of our nodes are!