Skip to content

What fghj touches, and how to remove it

fghj is a root daemon that edits system configuration, so it is worth being able to see exactly what it has done to your machine and undo all of it. This page is the complete list.

The short version, if you just want it gone:

Terminal window
sudo fghj uninstall && brew uninstall fghj # macOS
sudo apt purge fghj # Ubuntu: runs `fghj uninstall` itself

Most of this page is macOS; what Linux touches instead is in On Linux.

fghj uninstall does everything removing the binaries can’t: it stops the daemon, unwinds its system configuration, takes the root CA out of every store it was trusted in (your login keychain on macOS; the system store and your browsers on Linux), removes fghj’s sidecar containers, and removes /var/lib/fghjd. It prints what it is about to remove and asks before doing any of it (-y skips the prompt).

It needs root, and it will not silently re-run itself under sudo — a command that escalates on its own to delete a trusted root certificate is the wrong shape, so it refuses and tells you what to run.

Two flags exist for the reinstalling-in-place case, where a full purge is more than you want:

FlagKeeps
--keep-caThe root CA trusted, so a reinstall doesn’t re-prompt for keychain authorization
--keep-state/var/lib/fghjd — every wired workspace and both CAs’ private keys
--keep-workspacesOnly the workspace index and daemon settings; no CA file or key

Docker resources are left alone either way. See Docker resources below.

If you’d rather do it by hand, or fghj is already gone from your PATH, every step is spelled out in the sections that follow.

fghj daemon stop (and stopping the process) reverses every system change. Linux’s own list is below. These are unwound on a clean shutdown, and re-applied on the next start — you do not need to clean them up by hand unless fghjd was killed uncleanly.

WhatWhere
Managed hosts block/etc/hosts, between # fghj-managed-begin and # fghj-managed-end
Split-DNS resolver file/etc/resolver/fghj.internal
Proxy loopback alias127.222.0.1 on lo0
Raw-zone loopback aliases127.223.0.0/16 addresses on lo0
Control socket/var/run/fghjd.sock

The next start self-heals all of the above. To verify by hand:

Terminal window
grep -n "fghj" /etc/hosts
ls /etc/resolver/
ifconfig lo0 | grep -E "127\.22[23]"

Everything fghjd persists lives under one root — /var/lib/fghjd (really /private/var/lib/fghjd on macOS, since /var is a symlink). There is no hidden per-user cache or dotfile directory: fghj reads $HOME only to run git clone as you rather than as root, and never writes a config or cache of its own there. The one thing that does land in your own directories is the repositories it clones — see Repositories fghj cloned.

PathWhat it is
ca/ca-cert.pem, ca/ca-key.pemThe local root CA. Deleting these means a new CA on next start, which must be re-trusted.
ca/signing-cert.pem, ca/signing-key.pemThe name-constrained signing CA that issues every leaf cert. Deleting these costs nothing — a new one is minted under the same root on next start, with no re-trusting.
ca/signing-generationWhich constraint set the signing CA above was built with. Deleting it just regenerates the signing CA.
ca/cert.pem, ca/bundle.pemWorld-readable trust files, for containers that need to trust fghj’s CA.
certs/The same two trust files, in a directory with no key material in it, bind-mounted read-only into every container at /etc/fghj/certs.
workspaces.jsonThe index of wired workspaces.
daemon-state.jsonDaemon state across restarts.
runs/<network>/Per-run route tables, read by each run’s sidecar.
sidecar-ca/A world-readable copy of the signing CA (certificate and key), mounted into sidecar containers. Never the root’s key.
sidecar-build/Scratch space for building the sidecar image from source.
Terminal window
sudo rm -rf /var/lib/fghjd

Removing this is safe while the daemon is stopped. It forgets every wired workspace and discards the CA.

This is the one change that outlives brew uninstall, because it is not a file fghj owns — it is an entry in your login keychain, trusted for your user only, installed with security add-trusted-cert so your browser trusts *.fghj.internal certificates. fghj 0.2.0 and earlier put it in the System keychain instead; check both.

Terminal window
# Confirm it is there
security find-certificate -c "fghj local CA" ~/Library/Keychains/login.keychain-db
security find-certificate -c "fghj local CA" /Library/Keychains/System.keychain
# Remove it
security delete-certificate -t -c "fghj local CA" ~/Library/Keychains/login.keychain-db
sudo security delete-certificate -c "fghj local CA" \
/Library/Keychains/System.keychain

Leaving a stale local CA trusted is not catastrophic — the root’s private key is in /var/lib/fghjd/ca/, root-readable only, and never leaves it — but there is no reason to keep a trusted root you no longer use. Remove it.

Only the root is ever in your keychain, under the name above. The signing CA is called fghj signing CA and is trusted transitively through the root, so there is nothing separate to delete for it.

These are not removed by stopping or uninstalling fghj. Containers stop being reachable, but they keep running under Docker’s own supervision.

The simplest way to clear them is before you uninstall: remove each workspace from its Config tab in the UI. That deletes the workspace’s containers, sidecar, network and volumes, and the Activity tab shows each removal finishing. The commands below are for what’s left after that, or for cleaning up once fghjd is already gone.

Everything fghj creates is prefixed fghj-:

ResourceName
Containersfghj-<workspace>-default-<node>
Sidecarsfghj-<workspace>-default-sidecar
Networksfghj-<workspace>-default
Volumesfghj-vol-<derived>
Sidecar imageghcr.io/mlnja/fghj-sidecar
Terminal window
# Look first
docker ps -a --filter "name=^fghj-" --format '{{.Names}}'
docker network ls --filter "name=^fghj-" --format '{{.Name}}'
docker volume ls --filter "name=^fghj-" --format '{{.Name}}'
# Then remove
docker rm -f $(docker ps -aq --filter "name=^fghj-")
docker network rm $(docker network ls -q --filter "name=^fghj-")
docker volume rm $(docker volume ls -q --filter "name=^fghj-")
docker rmi $(docker images -q "ghcr.io/mlnja/fghj-sidecar")

Volumes are the ones worth pausing over: a named volume is deliberately designed to outlive its container, so it may hold database contents you still want. docker volume ls --filter "name=^fghj-" before deleting.

Images your services were built from are not listed here — those are ordinary Docker images built from your own Dockerfiles and are not named by fghj.

Worth knowing, because it surprises people: when a .fghj.yaml declares an include: of another repo, fghj clones it into your workspace directory, alongside your own repos — not into a hidden cache.

my-workspace/
storefront/ <- yours, the one you cloned
catalog/ <- cloned by fghj, from the dependency declaration

They are ordinary git clones with a normal origin, so nothing special is needed to remove them — rm -rf the ones you did not create. Check for unpushed work first; fghj never deletes them for you precisely because it cannot know whether you have been editing them.

The same kinds of change, made the Linux way. sudo apt remove fghj runs fghj uninstall --yes --keep-workspaces before the binaries go: afterwards no CA of fghj’s is trusted anywhere or exists on disk, and fghj’s sidecar containers are gone. Only workspaces.json and daemon-state.json stay, for a reinstall. sudo apt purge fghj deletes those too, and fghj’s database in each wired workspace.

WhatWhereUndone by
Managed hosts block/etc/hosts, as on macOSstopping fghjd
Split DNSthe fghj0 dummy link, which carries fghjd’s DNS server and ~fghj.internal ~fghj.raw.internal as its routing domains in systemd-resolvedstopping fghjd deletes the link
Loopback addressesnone: all of 127/8 is already local—
The root CA, for programs/usr/local/share/ca-certificates/fghj-local-ca.crt, folded into /etc/ssl/certs/ca-certificates.crtfghj uninstall
The root CA, for browsersfghj local CA in each workspace owner’s Firefox profiles and Chrome/Chromium NSS database, snap ones includedfghj uninstall, for every wired workspace’s owner and whoever ran sudo
The CAs’ files and keys/var/lib/fghjd/ca, certs, sidecar-caapt remove, or fghj uninstall
fghj’s sidecar containersDocker, one per workspace with an environmentapt remove, or fghj uninstall
The workspace index and settings/var/lib/fghjd/workspaces.json, daemon-state.jsonapt purge, or fghj uninstall
fghj’s database in each workspace<workspace>/.fghj/fghj.dbapt purge, or fghj uninstall
The servicefghjd.service, under /usr/lib/systemd/systemapt remove

To check by hand:

Terminal window
grep -n fghj /etc/hosts
ip link show fghj0 && resolvectl domain fghj0
ls /usr/local/share/ca-certificates/
certutil -d sql:"$HOME/.pki/nssdb" -L | grep -i fghj
for p in ~/.mozilla/firefox/*/ ~/snap/firefox/common/.mozilla/firefox/*/; do
[ -f "$p/cert9.db" ] && certutil -d "sql:$p" -L | grep -i fghj
done

And by hand, if the package is already gone:

Terminal window
sudo ip link del fghj0
sudo rm /usr/local/share/ca-certificates/fghj-local-ca.crt
sudo update-ca-certificates --fresh
certutil -d sql:"$HOME/.pki/nssdb" -D -n "fghj local CA" # and each Firefox profile
sudo rm -rf /var/lib/fghjd
Terminal window
brew uninstall fghj

This removes the fghj, fghjd and fghj-sidecar binaries and the LaunchDaemon plist Homebrew installed at /Library/LaunchDaemons/homebrew.mxcl.fghj.plist. Stop the service first (sudo brew services stop fghj) so the system changes above are unwound while the daemon is still alive to unwind them.

To also drop the tap:

Terminal window
brew untap mlnja/tap
Terminal window
# No process, no socket
pgrep -l fghjd; ls /var/run/fghjd.sock 2>/dev/null
# No system config, macOS
grep -c fghj /etc/hosts; ls /etc/resolver/fghj.internal 2>/dev/null
ifconfig lo0 | grep -E "127\.22[23]"
security find-certificate -c "fghj local CA" \
~/Library/Keychains/login.keychain-db 2>/dev/null
security find-certificate -c "fghj local CA" \
/Library/Keychains/System.keychain 2>/dev/null
# No system config, Linux
grep -c fghj /etc/hosts; ip link show fghj0 2>/dev/null
ls /usr/local/share/ca-certificates/fghj-local-ca.crt 2>/dev/null
# No state, no Docker resources
ls /var/lib/fghjd 2>/dev/null
docker ps -a --filter "name=^fghj-" --format '{{.Names}}'

Every one of those should print nothing (or “not found”); for the browsers on Linux, see the certutil loop in On Linux. If pgrep still finds fghjd, something is supervising it — check sudo launchctl list | grep fghj on macOS or systemctl status fghjd on Linux.