How to search for old transactions on Testnet using API?

thinguyen · July 2, 2021, 10:05pm

So I am trying to search for any transactions that have been committed to Testnet that I made. I stored all transaction ID (txid) in a file. Now, my task is to measure runtime for reading 1 or simultaneously many transactions. I have found the API for Testnet indexer to request the needed information from this link: API for Testnet indexer

I successful used this code from
docs/search_tx_address_txId.py at master · algorand/docs · GitHub
to read transaction in a private network before:

# search_tx_address_txId.py
import json
# requires Python SDK version 1.3 or higher
from algosdk.v2client import indexer

# instantiate indexer client
myindexer = indexer.IndexerClient(indexer_token="", indexer_address="http://localhost:8980")

response = myindexer.search_transactions_by_address(
    address="XIU7HGGAJ3QOTATPDSIIHPFVKMICXKHMOR2FJKHTVLII4FAOA3CYZQDLG4",
    txid="QZS3B2XBBS47S6X5CZGKKC2FC7HRP5VJ4UNS7LPGHP24DUECHAAA")

print("txid: QZS3B2XBBS47S6X5CZGKKC2FC7HRP5VJ4UNS7LPGHP24DUECHAAA = " +
      json.dumps(response, indent=2, sort_keys=True))

In the case for private network, I replaced the indexer_address with the address of an indexer that has been set up in the network and replaced address for the address of my node in the network. Of course, I also replaced txid with the relevant txid I got.

In testnet, I replace the indexer_address with the API for Testnet indexer, the address of my node, and txid but it does not work. I am new to using API and Algorand. So I do not know if that’s what I am supposed to do to search for transaction. When I ran the code, I got an error for
urllib.error.HTTPError: HTTP Error 403: Forbidden

What does it mean and how do I fix it?

Thank you,

fabrice · July 2, 2021, 11:04pm

I am not completely sure which use cases do not work for you:

Does your code work when using the indexer from PureStake API?
Does your code work when using your own TestNet indexer?
Does your code work when using your own indexer for a private network?

thinguyen · July 3, 2021, 3:39am

The code do not work with the PureStake API, which is the one I used for the node on Testnet. I’m doing a research, and my team has built a private net on Algorand. So I have a node on our private net. we installed an indexer, so I connected my node to that indexer and the code worked for our private network but not Testnet.

fabrice · July 3, 2021, 4:05pm

I believe the reason your code is not working with PureStake is because you need to provide your PureStake token. See PureStake Developer Portal

Concretely:

algod_address = "https://testnet-algorand.api.purestake.io/idx2"
headers = {
   "X-API-Key": "The API Key PureStake gave you on https://developer.purestake.io/",
}

myindexer = indexer.IndexerClient("", algod_address, headers)

Regarding your TestNet indexer, can you show the exact error?
Can you show the output of:

curl http://indexeraddress/health

thinguyen · July 4, 2021, 7:53pm

Do you mean there’re 2 way to search for a transactions in blockchain? one is by using a test net indexer and the other way is by using PureStake API?
If you are asking for the output of

I have not set up an indexer for my node on Testnet. I thought using PureStake would save me from having to set up an indexer?

Here’s my code:

 # search_tx_address_txId.py
 import json
 # requires Python SDK version 1.3 or higher
 from algosdk.v2client import indexer
 
 # instantiate indexer client
 myindexer = indexer.IndexerClient(indexer_token="", indexer_address="https://testnet-algorand.api.purestake.io/idx2")
 
 response = myindexer.search_transactions_by_address(
     address="my_account_address",
     txid="OLFU4X3UKOVS7CJCT7GNHBGXVMJLMKHGR62GNIPGH2HNJGZRZFC7XWGBUQ")

 print("txid: OLFU4X3UKOVS7CJCT7GNHBGXVMJLMKHGR62GNIPGH2HNJGZRZFC7XWGBUQ = " +
      json.dumps(response, indent=2, sort_keys=True))

and the error I got is:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/algosdk/v2client/indexer.py", line 71, in indexer_request
    resp = urlopen(req)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 214, in urlopen
    return opener.open(url, data, timeout)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 523, in open
    response = meth(req, response)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 632, in http_response
    response = self.parent.error(
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 561, in error
    return self._call_chain(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 494, in _call_chain
    result = func(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 641, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/algosdk/v2client/indexer.py", line 75, in indexer_request
    raise error.IndexerHTTPError(json.loads(e)["message"])
algosdk.error.IndexerHTTPError: Forbidden

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/emilynguyen98/testnet/read_indexer.py", line 10, in <module>
    response = myindexer.search_transactions_by_address(
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/algosdk/v2client/indexer.py", line 404, in search_transactions_by_address
    return self.indexer_request("GET", req, query, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/algosdk/v2client/indexer.py", line 77, in indexer_request
    raise error.IndexerHTTPError(e)
algosdk.error.IndexerHTTPError: {"message":"Forbidden"}

thinguyen · July 4, 2021, 9:09pm

i got it to work. I used the wrong method to instantiate a client. That was why I was not able to connect to the API. Thanks again!

thinguyen · July 4, 2021, 10:52pm

I have another question. My goal is to read more than 10 transactions simultaneously using a loops. With PureStake, I cannot do that can I? If yes, my only option is to set up an indexer myself. How long it might take for the indexer to fully caught up?

fabrice · July 5, 2021, 4:08pm

You can also try to use algoexplorerapi.io that has different limits.

But setting up an indexer is a very good option.
You’ll need an archival node + an indexer.
It should take around a week to sync.

thinguyen · July 5, 2021, 5:53pm

Can I use fast catch up when I set up the Archival node?

fabrice · July 5, 2021, 6:05pm

No, fast catchup should not be used with an archival node.

thinguyen · July 6, 2021, 1:23am

How do I use the Algod API for Testnet which is found here?
I replace that API into this code to make 5 rep/sec but did not succeed:

# Setup HTTP client w/guest key provided by PureStake
headers = {
   "X-API-Key": "FRHrScTyncarUupW3ffo34eCXPpzhHQhEje5El13",
}
algod_address = 'https://testnet-algorand.api.purestake.io/ps2'
myindexer = indexer.IndexerClient("", algod_address, headers)

# open txid to read
trans_id = open("txid.txt","r")
reading_time = open("read10.txt","a")

# timestamp for beginning the code block belowcd
start_time = time.time()

counter = 0
# NOTE: address is the address of the account on the current node, the node must be in the same network as the indexer
for i in range (9):
  counter +=1
  txidD= trans_id.readline().strip()
  address="OLFU4X3UKOVS7CJCT7GNHBGXVMJLMKHGR62GNIPGH2HNJGZRZFC7XWGBUQ"
  response = myindexer.search_transactions_by_address(address,txid=txidD)
  print("txid:  = "+ txidD)
  print(json.dumps(response, indent=2, sort_keys=True))
  

  if txidD=="":
    break

and the error I got is

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/algosdk/v2client/indexer.py", line 71, in indexer_request
    resp = urlopen(req)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 214, in urlopen
    return opener.open(url, data, timeout)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 523, in open
    response = meth(req, response)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 632, in http_response
    response = self.parent.error(
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 561, in error
    return self._call_chain(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 494, in _call_chain
    result = func(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 641, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/algosdk/v2client/indexer.py", line 75, in indexer_request
    raise error.IndexerHTTPError(json.loads(e)["message"])
algosdk.error.IndexerHTTPError: Missing Authentication Token

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/emilynguyen98/testnet/read_purestake_loop.py", line 30, in <module>
    response = myindexer.search_transactions_by_address(address,txid=txidD)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/algosdk/v2client/indexer.py", line 404, in search_transactions_by_address
    return self.indexer_request("GET", req, query, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/algosdk/v2client/indexer.py", line 77, in indexer_request
    raise error.IndexerHTTPError(e)
algosdk.error.IndexerHTTPError: {"message":"Missing Authentication Token"}

do I have to use a different method to instantiate a client or something in order for it to work?

Thank you

fabrice · July 6, 2021, 4:32pm

Never include your API key in a public message. Now it’s compromised and anybody can use it…
The indexer address is https://testnet-algorand.api.purestake.io/idx2. The address you used is for algod.

Topic		Replies	Views
algodClient.TransactionByID(TxID) method not existing on purestake? TestNet	4	275	January 11, 2021
I want to get a transaction by txid，but get " failed retrieving information from the indexer " TestNet	7	1368	November 14, 2019
SearchForTransactions() Unexpected Behaviour AlgoSDK testnet , community	4	368	August 14, 2020
Transaction History General	4	377	November 22, 2021
Testnet Algod API TestNet	12	1065	August 5, 2020

How to search for old transactions on Testnet using API?

Related Topics