Datastore (rqlite)

Nexus GSLB stores all data-plane objects — pools, members, services, health checks, geo rules, redirects, TSIG keys, TXT records, users, tenants, roles, settings — in rqlite, a Raft-replicated SQLite cluster. rqlite handles cross-node replication; there is no NATS DB sync and no file-copy replication.

This is distinct from the other two config layers:

• Configuration (YAML) is owned by git, applied per-node via GitOps self-restart.
• Ephemeral cluster state (health, heartbeats, RTTs, sessions) flows over NATS with TTLs.

Two processes per node

Each node runs two services:

systemd ties them together: gslbd.service has Requires= + After=rqlited.service, and gslbd waits for a Raft leader on startup before serving.

gslbd talks to rqlite via the gorqlite client through a custom database/sql driver (internal/storage/rqlite_driver.go). The driver normalises rqlite's JSON number encoding (float64) back to int64 so timestamp/integer scans are exact. The whole stack is CGO-free (CGO_ENABLED=0).

Foreign keys are required

rqlited must run with the -fk flag. The schema relies on ON DELETE CASCADE (members/health_checks/geo_rules → pools, users → tenants); without -fk those cascades silently no-op. The shipped deploy/rqlited.service sets -fk.

Configuration

gslbd's rqlite config block tells it how to reach the local rqlited:

rqlite:
  httpAddr: "localhost:4001"      # matches rqlited -http-addr
  connectTimeout: "10s"           # startup connect + leader-wait budget
  dataDir: "/var/lib/gslbd/rqlite"
  joinAddrs: []                   # peers for a joining node (empty = standalone)
  # tls:                          # HTTPS to rqlited (optional)
  #   enabled: true
  #   caFile: /etc/gslb/rqlite-ca.pem
  #   certFile: /etc/gslb/rqlite-client.pem
  #   keyFile: /etc/gslb/rqlite-client-key.pem
  # auth:
  #   username: gslbd
  #   password: change-me

Cluster membership (node id, bind/advertised addresses, join targets) is configured on rqlited, not here — see deploy/rqlited.env.

Cluster bootstrap and join

rqlited node settings live in /etc/gslb/rqlited.env (read by rqlited.service). Nodes form one Raft cluster over the private network (e.g. WireGuard).

First node (bootstrap) — no join target; bind/advertise on the address peers reach it on:

RQLITE_NODE_ID=lon-01
RQLITE_HTTP_ADDR=10.8.0.1:4001
RQLITE_RAFT_ADDR=10.8.0.1:4002

Additional nodes (join) — point at an existing peer's Raft address (port 4002, bare host:port; v8+/v10 reject http:// URLs and the HTTP port for -join). A joining node wipes its own local data and adopts the leader's, so bootstrap the node holding the data you want to keep and join the rest to it.

RQLITE_NODE_ID=eu-01
RQLITE_HTTP_ADDR=10.100.0.2:4001
RQLITE_RAFT_ADDR=10.100.0.2:4002
RQLITE_EXTRA_FLAGS=-join 10.100.0.1:4002    # peer RAFT addr, not http://, not :4001

Firewall: ports 4001 (HTTP) and 4002 (Raft) must be reachable between peers on the cluster interface. With ufw active, allow them from the mesh subnet only — ICMP reachability is not sufficient, the Raft TCP port must be open both ways or the leader loses quorum:

ufw allow from 10.100.0.0/24 to any port 4001 proto tcp
ufw allow from 10.100.0.0/24 to any port 4002 proto tcp

Apply with systemctl restart rqlited. Verify membership (expect all nodes reachable: true, one leader: true):

curl -s localhost:4001/nodes | jq .
curl -s localhost:4001/status | jq .store.raft

A single-node install needs nothing beyond the defaults — rqlited bootstraps a one-node Raft cluster on localhost.

Backups

gslbd backs up the database by calling rqlite's /db/backup endpoint, which streams the full SQLite database from the cluster leader to the local gslbd host (written atomically: temp + rename). This replaces the old file-copy/VACUUM INTO backup, which is meaningless in a clustered deployment. Configure under backup: (see Configuration).

Local development / testing

rqlite ships no macOS binaries — use Docker for local/CI work:

docker run -d -p 4001:4001 -p 4002:4002 rqlite/rqlite \
  -fk -http-adv-addr localhost:4001 -raft-adv-addr localhost:4002

The -*-adv-addr localhost flags are required for a container: without them rqlite advertises its internal container hostname, which the gorqlite client (following leader redirects) cannot resolve from the host. Storage/API/DNS integration tests connect to RQLITE_TEST_ADDR (default localhost:4001) via internal/storage/storagetest and skip when no server is reachable.

Code references

• internal/storage/rqlite.go — DB, Open(Options), schema migration, leader-wait.
• internal/storage/rqlite_driver.go — the database/sql driver over gorqlite.
• internal/storage/testsupport.go, internal/storage/storagetest/ — test helpers.
• deploy/rqlited.service, deploy/rqlited.env — the datastore unit + node config.