This repository contains my NixOS-based homelab infrastructure implementing a Kubernetes cluster with GlusterFS storage, network routing with WAN failover, home automation via Home Assistant, media services, 3D printer management, GitOps deployment through ArgoCD, LDAP authentication with OAuth2/OIDC integration, monitoring with Prometheus/Grafana, automated backups to cloud storage, and infrastructure-as-code using Nix flakes. The system spans multiple physical nodes with both on-premises and off-site components, using age encryption for secrets management and supporting both x86_64 and ARM architectures.
Follows numtide/blueprint with some extras:
hosts/fornixosConfigurationsanddarwinConfigurationsmodules/fornixosModulesanddarwinModulespackages/forpackageskubernetes/for ArgoCD appssecrets/for age secrets, seesecrets.nix
Located in hosts/
blueberry: a kubernetes and gluster nodecloudberry: the uplink router with WAN failovercranberry: a kubernetes and gluster nodeelderberry: a 3D printermango: a NixOS laptop (abandoned for now, main workstation is a Macbook, maybe I'll come back to it some day)minimal-*: NixOS configurations for debugging and adopting new machinesorange: a kubernetes (no workloads) and gluster (no storage) node located offsite
Note
Why is Macbook in a different repository? It used to be that nixos-unstable would frequently break for darwin - much more frequently than for nixos proper, so in order to keep darwin on a working version, it's in a different flake with its own nixpkgs input. nixos-unstable has been better on darwin recently, but I haven't got around to merging the two.
modules/home/rcambrj-console: a portable configuration for use on workstations and servers (although I think it's somewhat bloated for use on servers). This is consumed by Macbook.modules/home/rcambrj-graphical: a configuration for my workstation's graphical interface (originally built formango)modules/home/vscode: the configuration for my text editor. Consumed byrcambrj-graphicaland Macbook
Requires two USB sticks.
- Run the Github action to build minimal-intel or minimal-raspi
- Burn the resulting image to a USB stick
- Create
/hosts/{hostname}/configuration.nix - Add minimum configuration to
configuration.nix:imports = [ flake.nixosModules.base flake.nixosModules.access-server flake.nixosModules.common flake.nixosModules.bare-metal flake.nixosModules.config-intel # or flake.nixosModules.config-raspi ]; networking.hostName = "{hostname}"; - Prepare a second USB stick with a single FAT32 partition named
NIXOSCONF - Run
ssh-keygento generate a key pair - Put the resulting private key on this second USB stick
- Put the resulting public key into Github repository deploy keys
- Plug both USB sticks into a machine and switch it on
- SSH to
minimal-intel-nomadorminimal-raspi-nomad - Run
sudo nixos-rebuild switch --flake github:rcambrj/dotfiles#{hostname}
-
Create a new configuration in
/hostswhich usesdisk-aio.nix -
Boot
minimal-intelwith two-USB method (aarch64/raspi untested)Note: nixos graphical/livecd has limited free space for the nix store, so cannot be used
-
Enable swap, if the machine is particularly underpowered
fdisk # ...create a swap partition sudo swapon /dev/mmcblk0p1 -
Ensure that the disk target is correct
disko.devices.disk.disk1.device = "/dev/disk/by-id/..." -
Partition the disk
sudo nix run github:nix-community/disko/latest -- --flake "github:rcambrj/dotfiles#host" --mode destroy,format,mount --yes-wipe-all-disks # confirm fs labels are correct (vs part labels) lsblk -o name,mountpoint,label,size,uuid
-
Mount the partitions necessary for installation in a chroot
sudo mkdir -p /mnt/install sudo mount /dev/path/to/root/partition /mnt/install sudo mkdir -p /mnt/install/boot sudo mount /dev/path/to/boot/partition /mnt/install/boot
-
Install nixos
sudo nixos-install --root /mnt/install --flake "github:rcambrj/dotfiles#host" --no-root-passwd
- workaround a bug with nix-darwin and auto-optimise-store
nix.settings.auto-optimise-store = false; - configure nix-darwin with linux-builder (remove the host system from the two lists)
nix.linux-builder.enable = true; nix.linux-builder.systems = ["x86_64-linux" "aarch64-linux" "armv7l-linux"]; nix.linux-builder.maxJobs = 10; nix.linux-builder.config = ({ pkgs, ... }:{ boot.binfmt.emulatedSystems = ["x86_64-linux" "aarch64-linux" "armv7l-linux"]; }); nix.settings.experimental-features = [ "nix-command" "flakes" ]; - run the builder on MacOS
make build machine=blueberry