This is a short description of how to host storj node when the server is behind GNAT, or for other reasons cannot get routable address, by forwarding packets through WireGuard through a relatively fast nearby VPS. This is not specific for Storj, and can be adopted by hosting any other services.
As an example we will use an Oracle Cloud instance. Free tier still provides 10TB of monthly traffic that is sufficient for most node operators. Just make sure to create an account in a closest datacenter to minimize extra latency.
-
Create the oracle compute instance (ideally, Ampere, because they are awesome, but if that is not availabe, any other will do too).
-
Pick any OS you prefer, here we'll describe Ubuntu, as a most popular one.
-
Configure public IP address (this is the default), and upload SSH key to access the instance.
-
Then edit the
Ingress Rulesin the Default Security List in the VCN associated with the instance and rules to allow:- Traffic from anywhere
0.0.0.0/0, any port, to destination port28967, one for udp, one for tcp. This is for storj. - UDP at port
51820, for WireGuard. It does not need to be this specific port, any will do, just adjust the rest accordingly. The source network can also be narrowed down to your ISP's address range, if desired.
Statless Source IP Protocol Source Port Range Destination Port Range Type and Code ALlows Description No 0.0.0.0/0 TCP All 28967 TCP Traffic for port 28967 TCP port for Storj No 0.0.0.0/0 UDP All 28967 UDP Traffic for port 28967 UDP port for Storj No 0.0.0.0/0 TCP All 51820 UDP Traffic for port 51820 UDP port for Wireguard That's all that needs to be done in Oracle console. The
- Traffic from anywhere
-
Optionaly configure the public IP as an A record on your DNS provider, to use DNS name and not an ugly IP address in the subsequent configuration and in your storj node.
-
ssh to your new instance, update software, and install wireguard:
sudo apt update && sudo apt upgrade sudo reboot sudo apt install wireguard -y -
Configure wireguard tunnel between your node and VPS. There are tons of tutorials, here are the steps for reference:
On the VPS:
-
Initialize the config file
(umask 077 && printf "[Interface]\nPrivateKey= " | sudo tee /etc/wireguard/wg0.conf > /dev/null) wg genkey | sudo tee -a /etc/wireguard/wg0.conf | wg pubkey | sudo tee /etc/wireguard/publickey
-
Add peer info from below
-
Enable ipv4 forwarding: in
/etc/sysctl.confuncomment# Uncomment the next line to enable packet forwarding for IPv4 net.ipv4.ip_forward=1
and for the change to take effect:
sudo sysctl -p sudo sysctl --system
-
Enable and start the service
sudo systemctl enable wg-quick@wg0 sudo systemctl start wg-quick@wg0
On the client, assuming it's a TrueNAS and storj runs in the jail, we would need few things:
- In the jail properties tick the
allow_tunflag. (e.g.iocage set allow_tun=1 jailname) - On the host under System -> Tunables add
LOADERvariableif_wg_loadwith the valueYES, to load wireguard kernel module. - Initialize the wireguard config file and create keys just like above, noting that in FreeBSD the configuration file location will be
/usr/local/etc/wireguard/wg0.conf - In the jail in the
/etc/rc.confaddwireguard_enable="YES" wireguard_interfaces="wg0"
Generally, the config files shall look like so:
-
On the server:
/etc/wireguard/wg0.conf[Interface] PrivateKey = <private key of a server> ListenPort = 51820 Address = 10.0.60.1 # Allow WireGuard through the firwall PreUp = iptables -I INPUT 6 -p udp --dport 51820 -j ACCEPT PostDown = iptables -D INPUT -p udp --dport 51820 -j ACCEPT [Peer] PublicKey = <public key of a client> AllowedIPs = 10.0.60.2
-
On the client:
/usr/local/etc/wireguard/wg0.conf[Interface] PrivateKey = <private key of the client> Address = 10.0.60.2 [Peer] PublicKey = <public key of a server> AllowedIPs = 10.0.60.1/32 Endpoint = sub.example.com:51820 PersistentKeepalive = 25
-
Start the service on the client:
service wireguard start
At this point the client shall be able to ping the server and the server shall be able to ping the clinet at 10.0.60.1 and 10.0.60.2 addresses respectively.
-
Now the very last thing. In the [Interface] section on the server in the /etc/wireguard/wg0.conf add the following rules
# Allow forwarding to and from wireguard interface
PreUp = iptables -I FORWARD -i %i -j ACCEPT
PreUp = iptables -I FORWARD -o %i -j ACCEPT
PostDown = iptables -D FORWARD -i %i -j ACCEPT
PostDown = iptables -D FORWARD -o %i -j ACCEPT
# Turn on masquarading
PreUp = iptables -t nat -I POSTROUTING -o %i -j MASQUERADE
PostDown = iptables -t nat -D POSTROUTING -o %i -j MASQUERADE
# Note: we are inserting the ACCEPT rules below at position 6, before the default REJECT rule present on Oracle VMS. Your VPS may have similar default rules; adjust accordingly.
# Allow Wireguard ports through the firewall
PreUp = iptables -I INPUT 6 -p udp --dport 51820 -j ACCEPT
PostDown = iptables -D INPUT -p udp --dport 51820 -j ACCEPT
# Allow STORJ ports through the firewall
PreUp = iptables -I INPUT 6 -p tcp -m state --state NEW --dport 28967 -j ACCEPT
PreUp = iptables -I INPUT 6 -p udp --dport 28967 -j ACCEPT
PostDown = iptables -D INPUT -p tcp -m state --state NEW --dport 28967 -j ACCEPT
PostDown = iptables -D INPUT -p udp --dport 28967 -j ACCEPT
# Any other ports for additinal applications can be added accordignly.
# ...
# DNAT it
PreUp = iptables -t nat -I PREROUTING -p tcp --dport 28967 -j DNAT --to-destination 10.0.60.2:28967
PreUp = iptables -t nat -I PREROUTING -p udp --dport 28967 -j DNAT --to-destination 10.0.60.2:28967
PostDown = iptables -t nat -D PREROUTING -p tcp --dport 28967 -j DNAT --to-destination 10.0.60.2:28967
PostDown = iptables -t nat -D PREROUTING -p udp --dport 28967 -j DNAT --to-destination 10.0.60.2:28967These acomplish few things:
- Allow traffic to Wireguard port, so that your server can connec to establish the tunnel
- Allow new tcp and udp connections to Storj port (note inserting the rule before rule 6, on oracle instances rule 6 is reject)
- Allow Storj packet forwarding to wireguard interface
- Turn on masquarading, to facilitate the correct routing of response packets.
On the server, restart the wireguard service:
sudo systemctl restart wg-quick@wg0On the client, restart the wireguard service:
service wireguard restartIn the config.yaml of the storage node modify the external address to point to your vps:
# the public address of the node, useful for nodes behind NAT
contact.external-address: sub.example.com:28967[Re]start the node, and check the status page. It shall be now happily connected.
Is there a way to do everything on the "Storj client computer" within the Dockerfile or within a docker-compose, so that it is fully modular and does not impact the host OS on the Storj client computer.