giabao · July 3, 2021 15:53 · Dec 3, 2019
diff --git a/dummy0.netdev b/dummy0.netdev
@@ -0,0 +1,8 @@
+# Creates a "dummy" network interface
+# we'll configure this interface with a link-local address
+# See: https://www.freedesktop.org/software/systemd/man/systemd.netdev.html
+##
+
+[NetDev]
+Name=dummy0
+Kind=dummy
diff --git a/dummy0.network b/dummy0.network
@@ -0,0 +1,14 @@
+# We "find" the dummy0 interface we created and configure it with a link-local address
+# See: https://www.freedesktop.org/software/systemd/man/systemd.network.html
+##
+[Match]
+Name=dummy0
+
+[Network]
+Address=169.254.1.1/32
+
+# We want these domains
+Domains= ~consul.
+
+# To go to the consul DNS server we'll bind to this address
+DNS=169.254.1.1
diff --git a/network.hcl b/network.hcl
@@ -0,0 +1,60 @@
+##
+# This file is one of a few in `/etc/consul.d/config` and the consul-agent binary is launched with 
+#   something that looks like this in the `consul-agent.service` file:
+#
+#   ExecStart=/usr/local/bin/consul agent -config-dir=/etc/consul.d/ -config-dir=/etc/consul.d/services/ ... etc
+##
+#   
+# IN order to get split-zone DNS working properly with Consul and systemd resolver and docker,
+# we have to slightly tweak the consul network configuration.
+##
+# The address that consul will bind to for INTER-CLUSTER COMMS.
+# By default, this is "0.0.0.0", meaning Consul will bind to all addresses on the local machine and will 
+#   advertise the first available private IPv4 address to the rest of the cluster.
+#
+# However, with the dummy0 and (possibly) docker0 interfaces, consul will find multiple private addresses
+#   so we must explicitly tell consul to bind to the *actual* IP/Interface that is linked to amazon's network
+#   this is almost always going to be the `eth0` equivelent on a ubuntu host.
+##
+# See: https://www.consul.io/docs/agent/options.html#_bind
+bind_addr = "{{ GetPrivateIP }}"
+
+
+# See https://www.consul.io/docs/agent/options.html#_client
+##
+# The address that consul will bind CLIENT SERVIDES (http/dns) to is very different from the 
+#   address we want to accept clients on! As the agent is running locally, we want to accept 
+#   clients *ONLY* on our link-local / dummy0 address!
+#
+# Luckily, this is pretty easy. Hashi has go-sockaddr which lets us write golang style template
+#   statements to filter out various host network things. We'll use this shortly...
+#
+# See: https://godoc.org/github.com/hashicorp/go-sockaddr/template
+##
+# Currently, this is broken. 
+# See: https://github.com/hashicorp/consul/issues/5371
+# See: https://github.com/hashicorp/go-sockaddr/issues/31
+#
+# The fix is to hard-code the IP address. Normally, a sin punishable by death
+#   but in this case, it's just a paddelin' , as it's a link local and very unlikely
+#   to every need changing. I still feel like this is going to bite me one day...
+# client_addr = "{{ GetInterfaceIP \"dummy0\" }}"
+client_addr = "169.254.1.1"
+
+# the ssyd.resolvd conf does not allow you to set ports, unfortunately. Consul must run on 53 for
+#   clients expectations about DNS to work.
+ports = {
+  dns = 53
+}
+
+# Docker runtime can not use 127.* addresses for DNS when the container is on it's own network name space
+# This is because the docker runtime hosts it's own resolver in the 127.X space
+#
+# Docker can't forward DNS queries to 127.0.0.53, but it *can* forward to 169.254.1.1.
+# However, consul will not know what to do with DNS queries it gets that do not end in *.consul
+#
+# We fix this by telling consul to forward queries to the system resolver
+# See: https://www.consul.io/docs/agent/options.html#recursors
+##
+# Forward non *consul domains to the systemd resolver so consul will play nice w/ docker
+recursors = ["127.0.0.53"]
diff --git a/readme.md b/readme.md
@@ -0,0 +1,42 @@
+I see many people struggeling to make consul-agent work with systemd.resolvd and eventually give up and go with dnsmasq or a similar approach.
+
+Here's a reasonably simple way to make everything play nicely together. 
+
+If you found this useful, say thanks. And as much as i'd love your support via patreon, go and donate to the EFF.
+
+
+Here's an exerpt from the `install_consul_agent.sh` that my base packer builder runs for all my systemd hosts:
+
+
+```
+# Binary is in place, secured. Deploy the systemd components to make it useful
+##
+echo "moving systemd components into place..."
+# deploy the service file
+mv systemd/consul-agent.service /etc/systemd/system/consul-agent.service
+mv systemd/dummy0.netdev /etc/systemd/network/dummy0.netdev
+mv systemd/dummy0.network /etc/systemd/network/dummy0.network
+
+# Then get the interface created
+echo "reconfiguring network for dummy0..."
+systemctl restart systemd-networkd
+
+# Ok, now we're ready!
+echo "Attempting to bring consul up for POST..."
+systemctl enable consul-agent
+systemctl start consul-agent
+
+# Confirm that consul actually came up...
+if [ `systemctl is-failed consul-agent.service` == 'failed' ];
+then
+    echo "Consul failed to start"
+    # Bail, packer should fail this build...
+    exit 1
+fi
+```
+
+
+It really is that simple. Systemd.resolved will happily do split-zone DNS. I think this functionality was intended for VPN users, but we can take advantage of this for our purposes.
+Rather than tell systemd.resolved that `$host.internal.corp.com` can  be reached via `ppp01`, we tell resolvd that `$host.consul` can be resolved via `dummy0` and bind consul-agent to `dummy0`.
+
+Oh the fun to be had with virtual interfaces.... :).
No results found