How can I decode bytes in ledger fields ledger?

Community Projects
1

How do I read the encoded data in the ledger file? I know I can read transactions using goal ledger block {block number} but I’m interested in the vote information. I believe it’s stored in each block’s certdata table. Is there a similar tool for voting data? Or do I have to know how to decode it?
Thanks!
-Dave

2

Votes can be access by querying the REST API and get a “raw block”, or a block in msgpack format (the JSON version does not contain it).
For how to do it using the SDK see: There is no block hash in get block api response - #2 by fabrice

3

Hi Fabrice,
Thanks for your reply. I’m having a little trouble with the API. Here’s the code I’m running:

client = algosdk.algod.AlgodClient(algod_token, algod_address, headers)
def lastround():
    result = subprocess.run(['goal', 'node', 'lastround'], stdout=subprocess.PIPE)
    return result.stdout.decode().strip()

When I run client.block_info(lastround()) I get the expected JSON output. When I run client.block_raw(lastround()) I get the following error as output.

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ubuntu/venv/lib/python3.9/site-packages/algosdk/algod.py", line 375, in block_raw
    return msgpack.loads(response.read())
  File "msgpack/_unpacker.pyx", line 194, in msgpack._cmsgpack.unpackb
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfd in position 4: invalid start byte

Any idea what I should try?

5

Upon further investigation, It looks like I’m having trouble reading raw block data for about 95% of blocks. I triple checked each block, just to make sure it wasn’t something transient with my node. Notice block 106 and 111 were decoded, but all others in this selection did not.

17829103: ValueError, int is not allowed for map key when strict_map_key=True
17829104: ValueError, int is not allowed for map key when strict_map_key=True
17829105: UnicodeDecodeError, ‘utf-8’ codec can’t decode byte 0x83 in position 6: invalid start byte
17829106: OK
17829107: UnicodeDecodeError, ‘utf-8’ codec can’t decode byte 0x83 in position 6: invalid start byte
17829108: ValueError, int is not allowed for map key when strict_map_key=True
17829109: ValueError, int is not allowed for map key when strict_map_key=True
17829110: ValueError, int is not allowed for map key when strict_map_key=True
17829111: OK
17829112: ValueError, int is not allowed for map key when strict_map_key=True
17829113: UnicodeDecodeError, ‘utf-8’ codec can’t decode byte 0xf1 in position 8: unexpected end of data
17829114: UnicodeDecodeError, ‘utf-8’ codec can’t decode bytes in position 6-7: invalid continuation byte
17829115: ValueError, int is not allowed for map key when strict_map_key=True
17829116: ValueError, int is not allowed for map key when strict_map_key=True
17829117: ValueError, int is not allowed for map key when strict_map_key=True
17829118: UnicodeDecodeError, ‘utf-8’ codec can’t decode byte 0x83 in position 7: invalid start byte
17829119: ValueError, int is not allowed for map key when strict_map_key=True
17829120: ValueError, int is not allowed for map key when strict_map_key=True
17829121: ValueError, int is not allowed for map key when strict_map_key=True

6

I realized that block_raw is only available in V1 API.
In V2, you manually need to decode the output.
A few strings are not proper UTF-8, so when decoding you need to force raw decoding:

import algosdk.v2client
import msgpack

algod_address = "https://algoexplorerapi.io"
algod_token = ""
algod_headers = {"User-Agent":  "DoYouLoveMe?"}

clientV2 = algosdk.v2client.algod.AlgodClient(algod_token, algod_address, algod_headers)

block = msgpack.loads(
    clientV2.block_info(17829107, response_format="msgpack"),
    raw=True,
    strict_map_key=False
)

print(block)
7

This worked great! Thank you!

8

Any recommendations on how to go from here (a decoded block) to a list of python Transaction objects? I’d like to parse the transactions in the block and get the transaction id.
I can get the tx id from a Transaction object, but I can’t figure out how to do that starting from a decoded block. Thanks!