Multi-Node Bitcoin Core on One Host (OverlayFS + Split Storage)

Run several independent Bitcoin Core nodes that share block files while keeping each node’s LevelDB on fast/local storage.

---

Overview

• LevelDB data (mutable) → /mnt/chaindata/<node> (ext4, noatime)
  Contains: chainstate/, blocks/index/, indexes/, debug.log, peers.dat, PID.
• Block files (large, mostly immutable) → /mnt/blockstorage (ext4, noatime)
  • Seed (node1) lower layer: /mnt/blockstorage/seedblocks/blocks/{blk*,rev*,xor.dat}
  • Nodes 2..N: OverlayFS mounts exposing blocks/ via:
    • lowerdir = /mnt/blockstorage/seedblocks
    • upperdir = /mnt/blockstorage/bitcoin<NUM>.upper
    • workdir = /mnt/blockstorage/bitcoin<NUM>.work
    • mountpoint = /mnt/blockstorage/bitcoin<NUM>-blocks

bitcoind pointers per node:

• datadir=/mnt/chaindata/<node> (LevelDB & misc)
• blocksdir=/mnt/blockstorage/<node>-blocks (node1 uses /mnt/blockstorage/seedblocks)

No pruning on these shared-blocks nodes: prune=0.

---

Filesystems

Make sure these are mounted (e.g. via /etc/fstab) with noatime:

/dev/vdc1  /mnt/chaindata     ext4  defaults,noatime   0  2
/dev/vdb1  /mnt/blockstorage  ext4  defaults,noatime   0  2

Apply/verify:

sudo mount -a
mount | egrep '(/mnt/chaindata|/mnt/blockstorage)'

---

Directory Skeleton

# LevelDB roots (one per node)
sudo install -d -m 0755 /mnt/chaindata/bitcoin{1..5}

# Canonical shared block storage for node1 (lower layer)
sudo install -d -m 0755 /mnt/blockstorage/seedblocks

# Overlay roots for nodes 2..5
for n in 2 3 4 5; do
  sudo install -d -m 0755     /mnt/blockstorage/bitcoin${n}-blocks     /mnt/blockstorage/bitcoin${n}.upper     /mnt/blockstorage/bitcoin${n}.work
done

# Ownership (adjust user:group if different)
sudo chown -R node:node /mnt/chaindata /mnt/blockstorage

Note: Bitcoin will create blocks/ inside each blocksdir. For node1 the actual files live under:
/mnt/blockstorage/seedblocks/blocks/{blkNNNNN.dat,revNNNNN.dat,xor.dat}.

---

IPv6 (netplan) Template

Give each node a dedicated IPv6 (example addresses — replace with yours):

node1 → 1111:2222:3333:4444::101
node2 → 1111:2222:3333:4444::102
node3 → 1111:2222:3333:4444::103
node4 → 1111:2222:3333:4444::104
node5 → 1111:2222:3333:4444::105

/etc/netplan/99-bitcoin.yaml (merge with your existing config):

network:
  version: 2
  ethernets:
    enp1s0:
      dhcp4: true
      accept-ra: true
      addresses:
        - 1111:2222:3333:4444::101/128
        - 1111:2222:3333:4444::102/128
        - 1111:2222:3333:4444::103/128
        - 1111:2222:3333:4444::104/128
        - 1111:2222:3333:4444::105/128
      # routes usually via RA; omit unless your provider requires a manual default route
      # routes:
      #   - to: ::/0
      #     via: 1111:2222:3333:4444::1

Apply & check:

sudo netplan apply
ip -6 addr show dev enp1s0 | grep '1111:2222:3333:4444::10'

---

OverlayFS: /etc/fstab Entries (nodes 2..5)

Add one line per node:

# OverlayFS per node (lower=seedblocks, per-node upper/work)
overlay  /mnt/blockstorage/bitcoin2-blocks  overlay  nofail,x-systemd.requires=/mnt/blockstorage,lowerdir=/mnt/blockstorage/seedblocks,upperdir=/mnt/blockstorage/bitcoin2.upper,workdir=/mnt/blockstorage/bitcoin2.work  0  0
overlay  /mnt/blockstorage/bitcoin3-blocks  overlay  nofail,x-systemd.requires=/mnt/blockstorage,lowerdir=/mnt/blockstorage/seedblocks,upperdir=/mnt/blockstorage/bitcoin3.upper,workdir=/mnt/blockstorage/bitcoin3.work  0  0
overlay  /mnt/blockstorage/bitcoin4-blocks  overlay  nofail,x-systemd.requires=/mnt/blockstorage,lowerdir=/mnt/blockstorage/seedblocks,upperdir=/mnt/blockstorage/bitcoin4.upper,workdir=/mnt/blockstorage/bitcoin4.work  0  0
overlay  /mnt/blockstorage/bitcoin5-blocks  overlay  nofail,x-systemd.requires=/mnt/blockstorage,lowerdir=/mnt/blockstorage/seedblocks,upperdir=/mnt/blockstorage/bitcoin5.upper,workdir=/mnt/blockstorage/bitcoin5.work  0  0

Mount them:

sudo mount -a
for n in 2 3 4 5; do
  mount | grep -q "on /mnt/blockstorage/bitcoin${n}-blocks type overlay" && echo "overlay ok: n${n}" || echo "overlay MISSING: n${n}";
done

---

Systemd Template (instance-based)

/etc/systemd/system/bitcoind@.service:

[Unit]
Description=Bitcoin Core node %i
Wants=network-online.target
After=network-online.target
StartLimitIntervalSec=120
StartLimitBurst=15

[Service]
User=node
Group=node
Type=forking
PIDFile=/mnt/chaindata/%i/bitcoin.pid
ExecStart=/usr/local/bin/bitcoind   -daemon   -pid=/mnt/chaindata/%i/bitcoin.pid   -conf=/etc/nodes/%i.conf
Restart=always
RestartSec=5
PrivateTmp=true
TimeoutStopSec=60s
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

Reload:

sudo systemctl daemon-reload

---

Node Configs (concept)

Create /etc/nodes/bitcoin1.conf (seed):

# --- storage ---
datadir=/mnt/chaindata/bitcoin1
blocksdir=/mnt/blockstorage/seedblocks
prune=0

# --- indexes (enable only if needed) ---
txindex=1
# blockfilterindex=1
# coinstatsindex=1

# --- performance ---
dbcache=1024
maxconnections=128
logtimestamps=1

# --- p2p (dedicated IPv6) ---
port=8333
bind=[1111:2222:3333:4444::101]:8333
externalip=[1111:2222:3333:4444::101]:8333

# --- rpc (local only) ---
server=1
rpcbind=127.0.0.1:5551
rpcallowip=127.0.0.1
rpcport=5551

Create /etc/nodes/bitcoin<NUM>.conf (nodes 2..5; only deltas shown):

datadir=/mnt/chaindata/bitcoin<NUM>
blocksdir=/mnt/blockstorage/bitcoin<NUM>-blocks
prune=0

# indexes as required per node (often off to save space/time)
txindex=0
# blockfilterindex=0
# coinstatsindex=0

dbcache=512
maxconnections=96
logtimestamps=1

port=8333
bind=[1111:2222:3333:4444::<10X>]:8333     # 102..105
externalip=[1111:2222:3333:4444::<10X>]:8333

server=1
rpcbind=127.0.0.1:555<NUM>
rpcallowip=127.0.0.1
rpcport=555<NUM>

Permissions:

sudo chown root:node /etc/nodes/*.conf
sudo chmod 0640 /etc/nodes/*.conf

---

Bring-Up Sequence

1. Start bitcoin1 (seed):

sudo systemctl enable --now bitcoind@bitcoin1
journalctl -u bitcoind@bitcoin1 -n 50 --no-pager

This will create /mnt/blockstorage/seedblocks/blocks/{blk*,rev*} and begin syncing.
If paths changed from a prior setup, do a one-time rebuild:
/usr/local/bin/bitcoind -conf=/etc/nodes/bitcoin1.conf -dbcache=2048 -reindex -daemon

2. Start nodes 2..5:

for n in bitcoin2 bitcoin3 bitcoin4 bitcoin5; do
  sudo systemctl enable --now "bitcoind@$n"
done

---

Notes & Tips

• Keep prune=0 on all shared-blocks nodes.
• OverlayFS ensures nodes 2..5 see historical blocks from the lowerdir; each node writes only its current block file into its own upperdir.
• Consider enabling heavyweight indexes (e.g., txindex=1) on only one node to save time/disk elsewhere.
• Tune dbcache per node to fit RAM; temporarily increase for (re)indexing.
• Open firewall if you expect inbound peers: ufw allow 8333/tcp.