What fghj touches, and how to remove it
fghj is a root daemon that edits system configuration, so it is worth being able to see exactly what it has done to your machine and undo all of it. This page is the complete list.
The short version, if you just want it gone:
sudo fghj uninstall && brew uninstall fghj # macOSsudo apt purge fghj # Ubuntu: runs `fghj uninstall` itselfMost of this page is macOS; what Linux touches instead is in On Linux.
fghj uninstall does everything removing the binaries can’t: it stops the
daemon, unwinds its system configuration, takes the root CA out of every
store it was trusted in (your login keychain on macOS; the system store and
your browsers on Linux), removes fghj’s sidecar containers, and removes
/var/lib/fghjd. It prints what it is about
to remove and asks before doing any of it (-y skips the prompt).
It needs root, and it will not silently re-run itself under sudo — a
command that escalates on its own to delete a trusted root certificate is
the wrong shape, so it refuses and tells you what to run.
Two flags exist for the reinstalling-in-place case, where a full purge is more than you want:
| Flag | Keeps |
|---|---|
--keep-ca | The root CA trusted, so a reinstall doesn’t re-prompt for keychain authorization |
--keep-state | /var/lib/fghjd — every wired workspace and both CAs’ private keys |
--keep-workspaces | Only the workspace index and daemon settings; no CA file or key |
Docker resources are left alone either way. See Docker resources below.
If you’d rather do it by hand, or fghj is already gone from your PATH,
every step is spelled out in the sections that follow.
What is removed automatically (macOS)
Section titled “What is removed automatically (macOS)”fghj daemon stop (and stopping the process) reverses every system
change. Linux’s own list is below. These are unwound on a clean shutdown, and re-applied on the next
start — you do not need to clean them up by hand unless fghjd was killed
uncleanly.
| What | Where |
|---|---|
| Managed hosts block | /etc/hosts, between # fghj-managed-begin and # fghj-managed-end |
| Split-DNS resolver file | /etc/resolver/fghj.internal |
| Proxy loopback alias | 127.222.0.1 on lo0 |
| Raw-zone loopback aliases | 127.223.0.0/16 addresses on lo0 |
| Control socket | /var/run/fghjd.sock |
The next start self-heals all of the above. To verify by hand:
grep -n "fghj" /etc/hostsls /etc/resolver/ifconfig lo0 | grep -E "127\.22[23]"Durable state on disk
Section titled “Durable state on disk”Everything fghjd persists lives under one root — /var/lib/fghjd
(really /private/var/lib/fghjd on macOS, since /var is a symlink). There
is no hidden per-user cache or dotfile directory: fghj reads $HOME only to
run git clone as you rather than as root, and never writes a config or
cache of its own there. The one thing that does land in your own
directories is the repositories it clones — see
Repositories fghj cloned.
| Path | What it is |
|---|---|
ca/ca-cert.pem, ca/ca-key.pem | The local root CA. Deleting these means a new CA on next start, which must be re-trusted. |
ca/signing-cert.pem, ca/signing-key.pem | The name-constrained signing CA that issues every leaf cert. Deleting these costs nothing — a new one is minted under the same root on next start, with no re-trusting. |
ca/signing-generation | Which constraint set the signing CA above was built with. Deleting it just regenerates the signing CA. |
ca/cert.pem, ca/bundle.pem | World-readable trust files, for containers that need to trust fghj’s CA. |
certs/ | The same two trust files, in a directory with no key material in it, bind-mounted read-only into every container at /etc/fghj/certs. |
workspaces.json | The index of wired workspaces. |
daemon-state.json | Daemon state across restarts. |
runs/<network>/ | Per-run route tables, read by each run’s sidecar. |
sidecar-ca/ | A world-readable copy of the signing CA (certificate and key), mounted into sidecar containers. Never the root’s key. |
sidecar-build/ | Scratch space for building the sidecar image from source. |
sudo rm -rf /var/lib/fghjdRemoving this is safe while the daemon is stopped. It forgets every wired workspace and discards the CA.
The root CA in your keychain (macOS)
Section titled “The root CA in your keychain (macOS)”This is the one change that outlives brew uninstall, because it is not a
file fghj owns — it is an entry in your login keychain, trusted for your
user only, installed with security add-trusted-cert so your browser trusts
*.fghj.internal certificates. fghj 0.2.0 and earlier put it in the System
keychain instead; check both.
# Confirm it is theresecurity find-certificate -c "fghj local CA" ~/Library/Keychains/login.keychain-dbsecurity find-certificate -c "fghj local CA" /Library/Keychains/System.keychain
# Remove itsecurity delete-certificate -t -c "fghj local CA" ~/Library/Keychains/login.keychain-dbsudo security delete-certificate -c "fghj local CA" \ /Library/Keychains/System.keychainLeaving a stale local CA trusted is not catastrophic — the root’s private
key is in /var/lib/fghjd/ca/, root-readable only, and never leaves it —
but there is no reason to keep a trusted root you no longer use. Remove it.
Only the root is ever in your keychain, under the name above. The signing CA
is called fghj signing CA and is trusted transitively through the root, so
there is nothing separate to delete for it.
Docker resources
Section titled “Docker resources”These are not removed by stopping or uninstalling fghj. Containers stop being reachable, but they keep running under Docker’s own supervision.
The simplest way to clear them is before you uninstall: remove each workspace from its Config tab in the UI. That deletes the workspace’s containers, sidecar, network and volumes, and the Activity tab shows each removal finishing. The commands below are for what’s left after that, or for cleaning up once fghjd is already gone.
Everything fghj creates is prefixed fghj-:
| Resource | Name |
|---|---|
| Containers | fghj-<workspace>-default-<node> |
| Sidecars | fghj-<workspace>-default-sidecar |
| Networks | fghj-<workspace>-default |
| Volumes | fghj-vol-<derived> |
| Sidecar image | ghcr.io/mlnja/fghj-sidecar |
# Look firstdocker ps -a --filter "name=^fghj-" --format '{{.Names}}'docker network ls --filter "name=^fghj-" --format '{{.Name}}'docker volume ls --filter "name=^fghj-" --format '{{.Name}}'
# Then removedocker rm -f $(docker ps -aq --filter "name=^fghj-")docker network rm $(docker network ls -q --filter "name=^fghj-")docker volume rm $(docker volume ls -q --filter "name=^fghj-")docker rmi $(docker images -q "ghcr.io/mlnja/fghj-sidecar")Volumes are the ones worth pausing over: a named volume is
deliberately designed to outlive its container, so it may hold database
contents you still want. docker volume ls --filter "name=^fghj-" before
deleting.
Images your services were built from are not listed here — those are
ordinary Docker images built from your own Dockerfiles and are not named
by fghj.
Repositories fghj cloned
Section titled “Repositories fghj cloned”Worth knowing, because it surprises people: when a .fghj.yaml declares an
include: of another repo, fghj clones it into your
workspace directory, alongside your own repos — not into a hidden cache.
my-workspace/ storefront/ <- yours, the one you cloned catalog/ <- cloned by fghj, from the dependency declarationThey are ordinary git clones with a normal origin, so nothing special is
needed to remove them — rm -rf the ones you did not create. Check for
unpushed work first; fghj never deletes them for you precisely because it
cannot know whether you have been editing them.
On Linux
Section titled “On Linux”The same kinds of change, made the Linux way. sudo apt remove fghj runs
fghj uninstall --yes --keep-workspaces before the binaries go: afterwards
no CA of fghj’s is trusted anywhere or exists on disk, and fghj’s sidecar
containers are gone. Only workspaces.json and daemon-state.json stay, for
a reinstall. sudo apt purge fghj deletes those too, and fghj’s database in
each wired workspace.
| What | Where | Undone by |
|---|---|---|
| Managed hosts block | /etc/hosts, as on macOS | stopping fghjd |
| Split DNS | the fghj0 dummy link, which carries fghjd’s DNS server and ~fghj.internal ~fghj.raw.internal as its routing domains in systemd-resolved | stopping fghjd deletes the link |
| Loopback addresses | none: all of 127/8 is already local | — |
| The root CA, for programs | /usr/local/share/ca-certificates/fghj-local-ca.crt, folded into /etc/ssl/certs/ca-certificates.crt | fghj uninstall |
| The root CA, for browsers | fghj local CA in each workspace owner’s Firefox profiles and Chrome/Chromium NSS database, snap ones included | fghj uninstall, for every wired workspace’s owner and whoever ran sudo |
| The CAs’ files and keys | /var/lib/fghjd/ca, certs, sidecar-ca | apt remove, or fghj uninstall |
| fghj’s sidecar containers | Docker, one per workspace with an environment | apt remove, or fghj uninstall |
| The workspace index and settings | /var/lib/fghjd/workspaces.json, daemon-state.json | apt purge, or fghj uninstall |
| fghj’s database in each workspace | <workspace>/.fghj/fghj.db | apt purge, or fghj uninstall |
| The service | fghjd.service, under /usr/lib/systemd/system | apt remove |
To check by hand:
grep -n fghj /etc/hostsip link show fghj0 && resolvectl domain fghj0ls /usr/local/share/ca-certificates/certutil -d sql:"$HOME/.pki/nssdb" -L | grep -i fghjfor p in ~/.mozilla/firefox/*/ ~/snap/firefox/common/.mozilla/firefox/*/; do [ -f "$p/cert9.db" ] && certutil -d "sql:$p" -L | grep -i fghjdoneAnd by hand, if the package is already gone:
sudo ip link del fghj0sudo rm /usr/local/share/ca-certificates/fghj-local-ca.crtsudo update-ca-certificates --freshcertutil -d sql:"$HOME/.pki/nssdb" -D -n "fghj local CA" # and each Firefox profilesudo rm -rf /var/lib/fghjdHomebrew’s own files
Section titled “Homebrew’s own files”brew uninstall fghjThis removes the fghj, fghjd and fghj-sidecar binaries and the
LaunchDaemon plist Homebrew installed at
/Library/LaunchDaemons/homebrew.mxcl.fghj.plist. Stop the service first
(sudo brew services stop fghj) so the system changes above are unwound
while the daemon is still alive to unwind them.
To also drop the tap:
brew untap mlnja/tapVerifying nothing is left
Section titled “Verifying nothing is left”# No process, no socketpgrep -l fghjd; ls /var/run/fghjd.sock 2>/dev/null
# No system config, macOSgrep -c fghj /etc/hosts; ls /etc/resolver/fghj.internal 2>/dev/nullifconfig lo0 | grep -E "127\.22[23]"security find-certificate -c "fghj local CA" \ ~/Library/Keychains/login.keychain-db 2>/dev/nullsecurity find-certificate -c "fghj local CA" \ /Library/Keychains/System.keychain 2>/dev/null
# No system config, Linuxgrep -c fghj /etc/hosts; ip link show fghj0 2>/dev/nullls /usr/local/share/ca-certificates/fghj-local-ca.crt 2>/dev/null
# No state, no Docker resourcesls /var/lib/fghjd 2>/dev/nulldocker ps -a --filter "name=^fghj-" --format '{{.Names}}'Every one of those should print nothing (or “not found”); for the browsers
on Linux, see the certutil loop in On Linux. If pgrep still
finds fghjd, something is supervising it — check
sudo launchctl list | grep fghj on macOS or systemctl status fghjd on
Linux.