Skip to content

NixOS configurations for my machines, as well as some useful modules and packages

License

Notifications You must be signed in to change notification settings

rcambrj/dotfiles

Repository files navigation

dotfiles

This repository contains my NixOS-based homelab infrastructure implementing a Kubernetes cluster with GlusterFS storage, network routing with WAN failover, home automation via Home Assistant, media services, 3D printer management, GitOps deployment through ArgoCD, LDAP authentication with OAuth2/OIDC integration, monitoring with Prometheus/Grafana, automated backups to cloud storage, and infrastructure-as-code using Nix flakes. The system spans multiple physical nodes with both on-premises and off-site components, using age encryption for secrets management and supporting both x86_64 and ARM architectures.

Repository structure

Follows numtide/blueprint with some extras:

  • hosts/ for nixosConfigurations and darwinConfigurations
  • modules/ for nixosModules and darwinModules
  • packages/ for packages
  • kubernetes/ for ArgoCD apps
  • secrets/ for age secrets, see secrets.nix

Machine configurations

Located in hosts/

  • blueberry: a kubernetes and gluster node
  • cloudberry: the uplink router with WAN failover
  • cranberry: a kubernetes and gluster node
  • elderberry: a 3D printer
  • mango: a NixOS laptop (abandoned for now, main workstation is a Macbook, maybe I'll come back to it some day)
  • minimal-*: NixOS configurations for debugging and adopting new machines
  • orange: a kubernetes (no workloads) and gluster (no storage) node located offsite

Note

Why is Macbook in a different repository? It used to be that nixos-unstable would frequently break for darwin - much more frequently than for nixos proper, so in order to keep darwin on a working version, it's in a different flake with its own nixpkgs input. nixos-unstable has been better on darwin recently, but I haven't got around to merging the two.

Home Manager configurations

  • modules/home/rcambrj-console: a portable configuration for use on workstations and servers (although I think it's somewhat bloated for use on servers). This is consumed by Macbook.
  • modules/home/rcambrj-graphical: a configuration for my workstation's graphical interface (originally built for mango)
  • modules/home/vscode: the configuration for my text editor. Consumed by rcambrj-graphical and Macbook

Preparing a new machine

Prepare a headless bare metal machine on split USB storage

Requires two USB sticks.

  1. Run the Github action to build minimal-intel or minimal-raspi
  2. Burn the resulting image to a USB stick
  3. Create /hosts/{hostname}/configuration.nix
  4. Add minimum configuration to configuration.nix:
    imports = [
        flake.nixosModules.base
        flake.nixosModules.access-server
        flake.nixosModules.common
        flake.nixosModules.bare-metal
        flake.nixosModules.config-intel
        # or
        flake.nixosModules.config-raspi
    ];
    
    networking.hostName = "{hostname}";
    
  5. Prepare a second USB stick with a single FAT32 partition named NIXOSCONF
  6. Run ssh-keygen to generate a key pair
  7. Put the resulting private key on this second USB stick
  8. Put the resulting public key into Github repository deploy keys
  9. Plug both USB sticks into a machine and switch it on
  10. SSH to minimal-intel-nomad or minimal-raspi-nomad
  11. Run sudo nixos-rebuild switch --flake github:rcambrj/dotfiles#{hostname}

Prepare a headless bare metal machine with AIO disk

  1. Create a new configuration in /hosts which uses disk-aio.nix

  2. Boot minimal-intel with two-USB method (aarch64/raspi untested)

    Note: nixos graphical/livecd has limited free space for the nix store, so cannot be used

  3. Enable swap, if the machine is particularly underpowered

    fdisk
    # ...create a swap partition
    
    sudo swapon /dev/mmcblk0p1
  4. Ensure that the disk target is correct

    disko.devices.disk.disk1.device = "/dev/disk/by-id/..."
    
  5. Partition the disk

    sudo nix run github:nix-community/disko/latest -- --flake "github:rcambrj/dotfiles#host" --mode destroy,format,mount --yes-wipe-all-disks
    
    # confirm fs labels are correct (vs part labels)
    lsblk -o name,mountpoint,label,size,uuid
  6. Mount the partitions necessary for installation in a chroot

    sudo mkdir -p /mnt/install
    sudo mount /dev/path/to/root/partition /mnt/install
    sudo mkdir -p /mnt/install/boot
    sudo mount /dev/path/to/boot/partition /mnt/install/boot
  7. Install nixos

    sudo nixos-install --root /mnt/install --flake "github:rcambrj/dotfiles#host" --no-root-passwd

Building locally on MacOS

  1. workaround a bug with nix-darwin and auto-optimise-store
    nix.settings.auto-optimise-store = false;
    
  2. configure nix-darwin with linux-builder (remove the host system from the two lists)
    nix.linux-builder.enable = true;
    nix.linux-builder.systems = ["x86_64-linux" "aarch64-linux" "armv7l-linux"];
    nix.linux-builder.maxJobs = 10;
    nix.linux-builder.config = ({ pkgs, ... }:{
        boot.binfmt.emulatedSystems = ["x86_64-linux" "aarch64-linux" "armv7l-linux"];
    });
    nix.settings.experimental-features = [ "nix-command" "flakes" ];
    
  3. run the builder on MacOS
    make build machine=blueberry

About

NixOS configurations for my machines, as well as some useful modules and packages

Resources

License

Stars

Watchers

Forks