I have a fully synced relay node behind a Caddy server and I’ve configured it such that connections to the /relay/ route are forwarded to port 4161 , but when I run ./goal node start -p 'example.com/relay/' from my local machine, my relay node metrics for algod_network_incoming_connections{} says 0 even though I don’t see any client errors reported in any of the local node or archive node logs. @tsachi, If there is documentation for properly running a relay node secured behind a reverse proxy, I’d greatly appreciate it.
A node can indeed connect through a reverse proxy, however, it cannot have any additional URI elements. Symmetrical to that, a relay can also be placed behind a firewall, but it would require few additional config.json settings.
I think that in your case the issue is the /relay/ part.
If you could have example.com directly it should work.
I’m going to break my answer into two sections - the node as a server and the node as a client. The client role is when the node is trying to establish connection to a provided relay. A server node is a relay listening to an incoming request(s).
As a node client, the node can be configured to connect to an endpoint; an endpoint contains a host name or IP address, followed by an optional port number.
for example algod -d <data dir> -p example.com:1234 would connect to example.com at point 1234.
As for the node server, it’s slightly more complicated. A node server listen on it’s NetAddress ( as specified in the config.json ), while having the following configuration limits:
MaxConnectionsPerIP
Limit the number of TCP connection would be allowed from a single remote endpoint.
UseXForwardedForAddressField
Instruct the node server to ignore the remote endpoint IP address, and use the X-Forwarded-For when presents. A server configured as such should have all its incoming traffic have the above http header, coming from the reverse proxy.
The interaction between #1 and #2 above are critical for a successful deployment of a node server, since the originating connection to the server would always be the reverse proxy, and therefore would not contain ( by default ) the originating IP address.
The node server would always listen at the “root” of the URI. It does not support listening for a “sub-URI”. If this configuration is needed, then the reverse proxy should perform these conversion and rewrite the URL as needed.
Thank you for the detailed explanation @tsachi. I was able to connect via reverse proxy by putting the relay at the root of the URI as you explained.
I am still interested in enabling the relay to listen to a sub-URI as this is useful for enabling other features for my application. It makes sense that the request would need to be re-written internally, and this is quite easy for me to do with Caddy. However, I notice in my node.log file that if I try to connect to example.com/relay/ by executing `./goal node start -p “example.com/relay/”, the following message is present:
"msg":"ws connect(ws://example.com/v1/mainnet-v1.0/gossip) fail - bad handshake, Status code = 502
Thus, it seems algod is stripping my subpath, /relay/, from the connection call, and therefore my URL rewrite is never triggered. I’m I understanding this correctly? Should I be specifying the endpoint differently in this case? Thanks again.
Yes, I think that your observation is correct.
The -p parameter you’ve used isn’t expecting to get a full url, and it’s stripping away anything after the host/port.
The above configuration is clearly an interesting, but so far, you’re the first person who asked for that “construct”. The majority of the usage for the -p is for either testing or local private networks.
I think that if you need to tunnel multiple http connections, you should be able to accomplish that by either creating two reverse proxies ( one that builds up the url, and one that strips it down ), or maybe try to use multiple port numbers.
I can certainly see how this is a non-standard use case. I’ll experiment with your suggestion and if I come up with anything, I’ll put it here for future reference of other users.
After experimenting and finding a pretty good solution, I’m putting this here for future reference for anyone else who is running multiple services (including a relay node) behind a proxy.
put the relay node port on the root path / (reverse_proxy /* localhost:4161)
If you want a browser to redirect on the URL to something else, you should match a GET on the / path and redirect to the desired sub URI. i.e redir { method GET, path /} /myuri/
