# NixOS on UTM for Apple Silicon: Portable Configuration Research

## Overview
Creating portable, automated NixOS VMs on Apple Silicon Macs using UTM virtualization.

## Key Projects

### 1. ciderale/nixos-utm ⭐21
**URL**: https://github.com/ciderale/nixos-utm
**Purpose**: Automate creation of UTM-based NixOS VMs
**Key Features**:
- Wraps `nixos-anywhere` to eliminate manual provisioning steps
- Apple Virtualization backend support w/ Rosetta2 for x86_64 emulation
- Automated setup: `nix run github:ciderale/nixos-utm#nixosCreate .#utm`
- IP retrieval via ARP cache lookup (MAC-based)
- Deploy config updates without VM recreation

**Limitations**: `utmctl` lacks full Apple backend support for some operations

### 2. onnimonni/nixos-utm-vm-example ⭐7
**URL**: https://github.com/onnimonni/nixos-utm-vm-example
**Purpose**: Practical NixOS VM example for UTM on Apple Silicon
**Key Features**:
- Flake-based config management
- SSH key auth (ed25519)
- Avahi integration for `.local` hostname access
- Remote builder pattern for x86_64 Linux builds from ARM host
- Rosetta emulation support

### 3. utmapp/UTM ⭐31.5k
**URL**: https://github.com/utmapp/UTM
**Description**: Full-featured VM host for iOS/macOS (QEMU-based)
**Languages**: Swift (primary), Objective-C, Shell, Python, C

## Automation Capabilities

### UTM Scripting Options

#### AppleScript
```applescript
tell application "UTM"
    set vmList to every virtual machine
    set myVM to first virtual machine whose name is "Ubuntu Server"
    start myVM
    suspend myVM
    stop myVM
end tell
```

#### URL Schemes
```
utm://run?name=MyVM
utm://pause?name=MyVM
utm://stop?name=MyVM
```

#### Swift Configuration API
- Programmatic VM creation/config via `UTMQemuConfiguration`
- Drive, network, port forwarding, display settings
- VLAN configuration, multiple network interfaces

### NixOS Portable Configuration Patterns

#### Portable Disk Configuration
```nix
device = "/dev/disk/by-label/nixos"  # Works across vda/sda
```
**Benefit**: Same config works for ARM64 (virtualized) & x86_64 (emulated) VMs

#### Flake Structure
- Lock dependency versions
- Modular organization: CLI tools, apps, shell config, dev envs
- Architecture-specific outputs

#### Home Manager Integration
- Declarative user environment
- `nix run . switch` activates configs

## Recommended VM Settings for Apple Silicon

### UTM Configuration
- **Architecture**: ARM64 (aarch64)
- **Memory**: 8-16GB
- **CPU**: 4+ cores
- **Display**: virtio-ramfb-gl (GPU acceleration)
- **Network**: Shared mode w/ port forwarding (SSH: 22 → 2222)
- **Boot**: UEFI, RNG Device, Hypervisor enabled

### Rosetta2 Support
Enable x86_64 emulation:
```nix
virtualisation.rosetta.enable = true;
```

### Network Discovery
```nix
# Avahi for .local hostname access
services.avahi.enable = true;
```

## Installation Approaches

### 1. Manual Installation Playbook (Detailed)

#### Prerequisites (macOS Host)
```bash
# Download NixOS ISO (ARM64 minimal with latest kernel)
# From: https://nixos.org/download.html#nixos-iso
# Or Hydra: https://hydra.nixos.org/job/nixos/trunk-combined/nixos.iso_minimal.aarch64-linux

# Verify ISO hash
shasum -a 256 nixos-minimal-*.iso

# Create directory for your VM config (optional, for later)
mkdir -p ~/nix-configs/utm-vm
cd ~/nix-configs/utm-vm
```

#### Step 1: Create UTM VM
1. Open UTM → "Create a New Virtual Machine"
2. Select "Virtualize" (not Emulate)
3. **Operating System**: Linux
4. **Boot ISO**: Select downloaded NixOS ISO
5. **Hardware**:
   - Memory: 8192 MB (8GB) minimum
   - CPU Cores: 4
6. **Storage**: 100GB (or as needed)
7. **Shared Directory**: Skip for now (configure post-install)
8. **Summary**:
   - Name: `nixos-dev`
   - Check "Open VM Settings"
9. **Settings Tweaks**:
   - System → Enable "UEFI Boot"
   - Display → "virtio-ramfb-gl" for GPU acceleration
   - Network → "Shared Network" + check "Enable Port Forwarding"
     - Guest Port: 22, Host: 2222, Protocol: TCP (for SSH)
10. Save & Start VM

#### Step 2: Partition & Format (Inside VM)
```bash
# Boot into NixOS installer, switch to root
sudo su

# Identify disk (usually /dev/vda for UTM)
lsblk

# Partition with parted
parted /dev/vda -- mklabel gpt
parted /dev/vda -- mkpart primary 512MB 100%
parted /dev/vda -- mkpart ESP fat32 1MB 512MB
parted /dev/vda -- set 2 esp on

# Format partitions
mkfs.ext4 -L nixos /dev/vda1     # Root partition with LABEL
mkfs.fat -F 32 -n boot /dev/vda2 # Boot partition with LABEL

# Create swap (optional, adjust size)
parted /dev/vda -- mkpart swap linux-swap 100% 8GB
mkswap -L swap /dev/vda3
swapon /dev/vda3

# Mount filesystems
mount /dev/disk/by-label/nixos /mnt
mkdir -p /mnt/boot
mount /dev/disk/by-label/boot /mnt/boot
```

#### Step 3: Generate Base Config
```bash
# Generate hardware config
nixos-generate-config --root /mnt

# Basic initial config (add vim for editing)
cat > /mnt/etc/nixos/configuration.nix <<'EOF'
{ config, pkgs, ... }:

{
  imports = [ ./hardware-configuration.nix ];

  # Bootloader
  boot.loader.systemd-boot.enable = true;
  boot.loader.efi.canTouchEfiVariables = true;

  # Hostname
  networking.hostName = "nixos-utm";
  networking.networkmanager.enable = true;

  # Enable flakes & nix-command
  nix.settings.experimental-features = [ "nix-command" "flakes" ];

  # Timezone
  time.timeZone = "America/New_York";  # Adjust

  # Enable SSH
  services.openssh.enable = true;
  services.openssh.settings.PermitRootLogin = "yes";  # Temporary

  # User account
  users.users.sam = {  # Change username
    isNormalUser = true;
    extraGroups = [ "wheel" "networkmanager" ];
    # Temporary password, change after first login
    initialPassword = "changeme";
  };

  # Packages for initial setup
  environment.systemPackages = with pkgs; [
    vim
    git
    curl
    wget
    htop
  ];

  # Rosetta2 support (for x86_64 emulation)
  virtualisation.rosetta.enable = true;

  # Avahi for .local hostname discovery
  services.avahi = {
    enable = true;
    nssmdns4 = true;
    publish = {
      enable = true;
      addresses = true;
      domain = true;
      workstation = true;
    };
  };

  system.stateVersion = "24.11"; # Match ISO version
}
EOF

# Review generated hardware config (uses by-label)
vim /mnt/etc/nixos/hardware-configuration.nix
```

#### Step 4: Install NixOS
```bash
# Install
nixos-install

# Set root password when prompted

# Shutdown (don't restart yet)
shutdown -h now
```

#### Step 5: Post-Install UTM Config
1. In UTM settings, remove installation ISO from CD/DVD drive
2. Start VM

#### Step 6: First Boot Setup
```bash
# From macOS host, SSH into VM
ssh -p 2222 sam@localhost

# Change user password
passwd

# Generate SSH keys for this VM (for git, etc)
ssh-keygen -t ed25519 -C "sam@nixos-utm"

# Test internet connectivity
ping -c 3 nixos.org

# Verify .local hostname works
# From another terminal: ping nixos-utm.local
```

#### Step 7: Copy Configs from macOS Host → VM

##### Option A: Simple SCP Transfer (Quick Start)
```bash
# From macOS host:

# Copy SSH keys
scp -P 2222 ~/.ssh/id_ed25519 sam@localhost:~/.ssh/id_ed25519_host
scp -P 2222 ~/.ssh/id_ed25519.pub sam@localhost:~/.ssh/id_ed25519_host.pub
scp -P 2222 ~/.ssh/known_hosts sam@localhost:~/.ssh/

# Copy git config
scp -P 2222 ~/.gitconfig sam@localhost:~/

# Copy SSH config (if you have one)
scp -P 2222 ~/.ssh/config sam@localhost:~/.ssh/

# Inside VM: fix permissions
ssh -p 2222 sam@localhost
chmod 600 ~/.ssh/id_ed25519_host
chmod 644 ~/.ssh/id_ed25519_host.pub
chmod 600 ~/.ssh/config
```

##### Option B: UTM Shared Directory (Persistent Access)
1. In UTM VM settings → Sharing
2. Add shared directory: Select macOS folder (e.g., `~/shared-utm`)
3. Inside VM:
```bash
# Mount shared directory
sudo mkdir -p /mnt/shared
sudo mount -t 9p -o trans=virtio share /mnt/shared

# Auto-mount on boot: add to /etc/fstab
# (We'll do this declaratively with NixOS config below)
```

##### Option C: Home-Manager with Secrets Management (Production)
See "Advanced: Secrets Management" section below.

#### Step 8: Initialize Flake-Based Config
```bash
# Inside VM as user
cd ~
mkdir -p nix-config
cd nix-config

# Create flake.nix
cat > flake.nix <<'EOF'
{
  description = "NixOS UTM VM Configuration";

  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
    home-manager = {
      url = "github:nix-community/home-manager";
      inputs.nixpkgs.follows = "nixpkgs";
    };
  };

  outputs = { self, nixpkgs, home-manager, ... }: {
    nixosConfigurations.nixos-utm = nixpkgs.lib.nixosSystem {
      system = "aarch64-linux";
      modules = [
        ./configuration.nix
        home-manager.nixosModules.home-manager
        {
          home-manager.useGlobalPkgs = true;
          home-manager.useUserPackages = true;
          home-manager.users.sam = import ./home.nix;
        }
      ];
    };
  };
}
EOF

# Copy existing config as starting point
sudo cp /etc/nixos/configuration.nix ./configuration.nix
sudo cp /etc/nixos/hardware-configuration.nix ./hardware-configuration.nix
sudo chown sam:users *.nix

# Create home-manager config
cat > home.nix <<'EOF'
{ config, pkgs, ... }:

{
  home.username = "sam";
  home.homeDirectory = "/home/sam";
  home.stateVersion = "24.11";

  # Git configuration
  programs.git = {
    enable = true;
    userName = "Your Name";
    userEmail = "you@example.com";
    extraConfig = {
      init.defaultBranch = "main";
      pull.rebase = true;
    };
  };

  # SSH config
  programs.ssh = {
    enable = true;
    matchBlocks = {
      "github.com" = {
        identityFile = "~/.ssh/id_ed25519";
      };
    };
  };

  # Bash/Shell config
  programs.bash = {
    enable = true;
    shellAliases = {
      ll = "ls -la";
      ".." = "cd ..";
    };
  };

  # Packages for user environment
  home.packages = with pkgs; [
    ripgrep
    fd
    bat
    eza
  ];
}
EOF

# Build and activate
sudo nixos-rebuild switch --flake .#nixos-utm
home-manager switch --flake .#nixos-utm
```

#### Step 9: Enable Persistent Shared Directory (Optional)
```nix
# Add to configuration.nix:
{
  # Auto-mount UTM shared directory
  fileSystems."/mnt/shared" = {
    device = "share";
    fsType = "9p";
    options = [
      "trans=virtio"
      "version=9p2000.L"
      "rw"
      "noauto"
      "x-systemd.automount"
    ];
  };
}
```

#### Step 10: Version Control Your Config
```bash
# Initialize git repo
cd ~/nix-config
git init
git add .
git commit -m "init nixos utm config"

# Push to GitHub (optional)
gh repo create nixos-utm-config --private --source=. --push
```

#### Step 11: Snapshot VM (Backup)
In UTM: Right-click VM → "Clone" → Name: `nixos-utm-clean-install`

### 2. Automated (nixos-utm)
```bash
export VM_NAME="my-nixos-vm"
nix run github:ciderale/nixos-utm#nixosCreate .#utm
```

### 3. Remote Builder Setup
- Separate ARM64 (virtualized) & x86_64 (emulated) VMs
- SSH key distribution to root
- Test: `nix build --impure` validates remote execution

---

## Advanced: Secrets Management

For production setups, manage SSH keys & git configs declaratively using secrets management tools.

### Option 1: agenix (Recommended for Simplicity)

#### Setup
```bash
# Add to flake.nix inputs:
inputs.agenix.url = "github:ryantm/agenix";

# Add to modules:
modules = [
  agenix.nixosModules.default
  # or for home-manager:
  # agenix.homeManagerModules.default
];
```

#### Create secrets directory
```bash
mkdir -p secrets
cd secrets

# Create secrets.nix (defines who can decrypt)
cat > secrets.nix <<'EOF'
let
  user1 = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5... user@host";
  vm1 = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5... root@nixos-utm";
in
{
  "ssh-key.age".publicKeys = [ user1 vm1 ];
  "gitconfig.age".publicKeys = [ user1 vm1 ];
}
EOF

# Encrypt secrets
nix run github:ryantm/agenix -- -e ssh-key.age
# (Paste your private key, save & exit)

nix run github:ryantm/agenix -- -e gitconfig.age
# (Paste gitconfig contents)
```

#### Use in home.nix
```nix
{ config, pkgs, ... }:
{
  age.secrets.ssh-key = {
    file = ../secrets/ssh-key.age;
    path = "${config.home.homeDirectory}/.ssh/id_ed25519";
    mode = "600";
  };

  age.secrets.gitconfig = {
    file = ../secrets/gitconfig.age;
    path = "${config.home.homeDirectory}/.gitconfig";
  };
}
```

### Option 2: sops-nix (More Features)

#### Setup
```bash
# Add to flake.nix:
inputs.sops-nix.url = "github:Mic92/sops-nix";

# Install sops
nix-shell -p sops
```

#### Create .sops.yaml
```yaml
keys:
  - &user_sam age1abc...xyz  # From: ssh-to-age -i ~/.ssh/id_ed25519
  - &vm age1def...uvw         # From: ssh-to-age -i /etc/ssh/ssh_host_ed25519_key.pub

creation_rules:
  - path_regex: secrets/[^/]+\.yaml$
    key_groups:
      - age:
          - *user_sam
          - *vm
```

#### Encrypt secrets
```bash
# Create secrets file
sops secrets/default.yaml
# Edit in $EDITOR:
# ssh_key: |
#   -----BEGIN OPENSSH PRIVATE KEY-----
#   ...
# gitconfig: |
#   [user]
#   name = Sam
```

#### Use in configuration
```nix
{
  sops.defaultSopsFile = ../secrets/default.yaml;
  sops.age.sshKeyPaths = [ "/home/sam/.ssh/id_ed25519" ];

  sops.secrets.ssh_key = {
    owner = "sam";
    path = "/home/sam/.ssh/id_ed25519_github";
  };
}
```

### Option 3: Git-Crypt (Simple File Encryption)
```bash
# In your nix-config repo:
nix-shell -p git-crypt

# Initialize
git-crypt init

# Add .gitattributes
echo "secrets/** filter=git-crypt diff=git-crypt" >> .gitattributes

# Store keys in secrets/
mkdir secrets
cp ~/.ssh/id_ed25519 secrets/
git add secrets/
git commit -m "add encrypted secrets"

# On new machine:
git-crypt unlock /path/to/key
```

---

## Config Sync Strategies Summary

| Method | Complexity | Security | Use Case |
|--------|------------|----------|----------|
| **SCP Transfer** | Low | Manual | Quick setup, one-time |
| **Shared Directory** | Low | Host-dependent | Development, frequent changes |
| **Git (plain)** | Medium | Public only | Public dotfiles |
| **Git-Crypt** | Medium | Good | Simple encryption |
| **agenix** | Medium | Excellent | SSH-based, simple |
| **sops-nix** | High | Excellent | Complex secrets, teams |

### Recommended Approach
1. **Initial setup**: SCP transfer (Step 7, Option A)
2. **Development**: UTM shared directory (Step 9)
3. **Production/Portable**: agenix or sops-nix with flake-based config

## Best Practices

### Configuration Management
- Use flakes for dependency locking
- Modular structure: separate hardware, base, VM-specific configs
- Version control all `.nix` files
- Label-based disk references (portability)

### Performance
- Virtualization mode (ARM64) > Emulation mode (x86_64)
- Rosetta2 efficient for x86_64 when needed
- UTM Apple backend better battery life vs Docker Desktop

### Critical Considerations
- **ACPI Shutdown**: Not supported - avoid host-side restart (recovery: console graphics + installer reattach + fsck)
- **Network**: Enable wireless networking for bare-metal to reach repos during rebuild
- **SSH**: Forward port 22 to local port for macOS access

## References
- https://krisztianfekete.org/nixos-on-apple-silicon-with-utm/
- https://adrianhesketh.com/2024/04/20/setting-up-nixos-remote-builder-m1-mac/
- https://calcagno.blog/m1dev/
- https://context7.com/utmapp/utm
- https://github.com/mitchellh/nixos-config (vm-aarch64-utm configs)
- https://github.com/a-h/nixos (flake for aarch64 & x86_64)