How to listen transactions over an rpc connection or socket without using indexer tool?

AlgoSDK
1

Hello,

I’m a developer who works in a crypto exchange company and want to list algo in our exchange.

We successfully installed node and it’s dependencies and thank you for your detailed instructions and samples.

Is there any way to listen transactions which comes to one of my addresses in my node?

I found Indexer tool but if you have an rpc or socket server on node, I’ll be really pleased.

Because as I understand I need to install postgre and make node an archival one which will be really needs attention for me. Also, I can’t work it in fast catchup mode.

If you have any suggestion or proper way that you can show me, I’ll be really pleased.

P.S. I have an hurry for that.
P.S. I want to contact with a developer from Algorand team who know rpc and web socket endpoints in other crypto nodes.

Best regards.

2

I am not sure on the rpc or socket on a node, but will look into that. Two other options on indexer for MainNet / TestNet in addition to building your own, are API services from PureStake and Rand Labs. Also, indexer is available on Sandbox now on a Private Network using sandbox for testing purposes.

3

Hi @hotv, I am not completely sure what you mean by “socket”. All connections normally uses a socket. Our API (algod v2 and indexer v2) are REST-based and not RPC-based.
In particular, the protocol is not the same as Bitcoin that is RPC-based.

If you just want to get all transactions, you may just request blocks one by one on algod v2: Algorand Developer Docs

If you need more advanced queries, you most likely want to use an indexer.

1 Like
4

Perhaps just have a basic poller that grabs a block at a time and pushes all transactions to an event bus/queue? You could then have multiple parallel listeners looking for their individual accounts.

5

Hi @fabrice, I think you couldn’t understand what I asked. So, you can’t help me.

6

Hi @aojjazz, when we will grab a block? In which period, in every x miliseconds or seconds? What if we can’t grab one of them?

7

Hi @fabrice, these are for you.

What is rpc?

What is socket?

8

I’m not sure what exactly you’re trying to achieve, so my answer might not be what you’re looking for.
If you could describe the “interface” you’re looking for, it might be easier to come up with some concrete solutions.

The general “recipe” for monitoring the activity on the Algorand blockchain is to place an indexer daemon in front of it. The indexer would pick up any new block and attempt to insert it into a Postgres database. On that database, you could create additional triggers that would insert the data you’re after into separate custom tables ( i.e. depending on the precise needs of your application ).

If all you need is to send/receive/confirm transactions, then any of the SDKs would do what you need.

9

If you are going through the route of fetching all the blocks, you can use the endpoint Algorand Developer Docs to wait for the next block to be ready.

But the best way is most likely to use the indexer anyway.

10

Hi @tsachi, @fabrice,

We found an awkward way to get needes tx from blocks and got the sender and reciever addresses amount and etc… but we couldn’t get txid from that object. How can we get txid from this object?

Object is like below
‘{“hgi”:true,“sig”:{“0”:184,“1”:59,“2”:88,“3”:252,“4”:12,“5”:129,“6”:39,“7”:182,“8”:93,“9”:167,“10”:5,“11”:139,“12”:55,“13”:232,“14”:184,“15”:227,“16”:171,“17”:240,“18”:145,“19”:103,“20”:89,“21”:227,“22”:196,“23”:223,“24”:117,“25”:178,“26”:213,“27”:152,“28”:141,“29”:180,“30”:81,“31”:24,“32”:83,“33”:103,“34”:88,“35”:166,“36”:61,“37”:77,“38”:108,“39”:110,“40”:254,“41”:128,“42”:60,“43”:113,“44”:0,“45”:46,“46”:247,“47”:187,“48”:124,“49”:119,“50”:153,“51”:245,“52”:88,“53”:95,“54”:119,“55”:54,“56”:134,“57”:11,“58”:37,“59”:102,“60”:48,“61”:104,“62”:51,“63”:9},“txn”:{“amt”:100000000,“fee”:10000,“fv”:11960181,“lv”:11961181,“rcv”:{“0”:21,“1”:74,“2”:190,“3”:178,“4”:210,“5”:21,“6”:139,“7”:89,“8”:95,“9”:100,“10”:26,“11”:1,“12”:130,“13”:70,“14”:191,“15”:182,“16”:166,“17”:21,“18”:119,“19”:168,“20”:66,“21”:166,“22”:17,“23”:165,“24”:4,“25”:227,“26”:103,“27”:231,“28”:127,“29”:42,“30”:71,“31”:148},“snd”:{“0”:48,“1”:253,“2”:204,“3”:35,“4”:27,“5”:157,“6”:140,“7”:54,“8”:9,“9”:175,“10”:127,“11”:123,“12”:177,“13”:229,“14”:239,“15”:143,“16”:150,“17”:58,“18”:72,“19”:176,“20”:114,“21”:19,“22”:82,“23”:141,“24”:239,“25”:189,“26”:136,“27”:109,“28”:150,“29”:174,“30”:72,“31”:132},“type”:“pay”}}’

11

@hotv ,

The above looks like the msgp of the SignedTxnInBlock; am I correct ?

If so, while you can definitely use it, I would discourage you from doing so. The internal block storage format might change over time, and there won’t be any developer notification for that as long as it won’t break the REST API backward compatibility.

The main “consumer” for that is currently the indexer, which performs the decoding and calculating the transaction id.

Calculating the txid from the above is definitely doable though; you can look into the source code how it’s being done. Essentially, you need to reconstruct the original transaction body, and then to hash the msgp encoded version of it.

12

@tsachi you’re correct but when I want to run indexer I had to install postgre and can’t use fast catchup and also check postgredb size, is it alive and etc…
Because of lot’s of reasons I don’t want to use indexer.

For now, I trying to decode msgp with javascript and get txid…

13

I think that using javascript for a prototyping a solution is a great first move.

The node is using the DecodeSignedTxn [ go-algorand/block.go at master · algorand/go-algorand · GitHub] method to reconstruct the transaction.
After that, you should be able to encode the transaction using msgp and hash it.

The result would be the transaction id. You might want to look into the javascript sdk as a code reference for figuring out how to use msgp and hashing.

14

I have this gripe as well. I should be able to run a non-archival node, use any SDK (like the Go SDK), fetch any of the last 1000 blocks and get all data I need from each block, cleanly and easily. ‘Almost’ everything is there but eve simple things like the Transaction ID isn’t available via the API.

Requiring the indexer for even basic fetching of block data is a huge burden on integrations. Running a non-archival node should be all that’s necessary. Requiring a full archival node (which will become significant in size - it’s already 324GB) + an indexer (~ half - so another 160+GB) just to be able to ‘watch’ transactions shouldn’t be necessary.

1 Like
15

Hi @tsachi ,

So far, I tried below code and different versions of it but I couldn’t get txid

var orgBlock = algosdk.decodeObj(tx.sig.buffer).block;

var rawTxId = Buffer.from(nacl.genericHash(orgBlock.gh));

var txid = base32.encode(rawTxId).slice(0,52);

this how they done in their transaction.js

16

Anyway, we canceled the job getting txid from block object which contains our address.

We will check transactions of our address from any kind of algoexplorer.
We will take transactions from there and check with it incoming block and if they are the same we will write that transaction id our customer’ history and etc…

İf js-algosdk had a web socket connection which streams blocks with their transaction ids with or without encryption, would be more easy to manage for us / everyone.

We don’t want to use third party tool of crypto currency copmanies.

We had the same issue on AVAX too. At the end we get in their internal unix type message socket and decrypt the message and get the needed info and in their node the endpoint is given in algo we couldn’t find that kind of endpoint.

Thank you for your help, @tsachi @aojjazz

17

@aojjazz,

There are few advantages in the current design:

  • it leaves the core node implementation simpler ( as it doesn’t require the node to support certain features ).
  • it doesn’t require the node to retain backward compatibility in the way opaque blocks are managed.
  • it reduces the “surface area” of the node interface, resulting in simpler REST API interface.

For more advanced needs, Algorand developed the indexer.

I agree that the indexer could be better implemented. Reviewing this thread, I can spot three distinct feature requests - indexer should not require a full archival node, indexer would be able to work with standalone sqlite database (i.e. no Postgres installation needed) and the indexer having a configurable history length. These all could be good potential improvement projects.

1 Like
18

@hotv A pattern for following along the actions of your algod node is two APIs: One waits for the status of a particular round number (behind the scenes this is an HTTP GET that stays open a few seconds until the round completes). The second API fetches the whole block. From there you can pick the stream of transactions out of it and filter for the ones you want. The Indexer does this here: indexer/fetcher.go at develop · algorand/indexer · GitHub

19

I had similar requirement for one of my project.

  1. start with the current block and poll for next block every 3 seconds.
  2. once you get a confirmed block get all transactions with in that block.
  3. use algoexplorer API with indexer to fetch the transactions.
  4. if processing of transaction with in a block is taking more than 4 seconds, then your server might end up lagging behind the current block after few days.
  5. to solve this you can span a thread when ever your current thread is still processing the past block transactions. parent thread passing on blocks to child threads. child threads will have business logic and processing the transactions.