Turnstone container-VM: podman image -> boco qcow2 boot
Arch image with turnstone (venv), dev toolkit, rootless podman; boots via boco (virtiofs shares, direct UID mapping for nested virtiofs). sshd ops channel + turnstone-server as guest marvin; state (SQLite db, config, work) on host shares; LLM via the TensorX OpenAI-compatible gateway (https://api.tensorx.ai/v1), reachable from the guest through both slirp layers with no relay.
This commit is contained in:
commit
50b215f759
11 files changed
+311
No files matched your search
@@ -0,0 +1,10 @@
|
||||
# Keep build context minimal; authorized_keys is COPY'd in, the private
|
||||
# key must never enter the image.
|
||||
.git
|
||||
data
|
||||
work
|
||||
scripts
|
||||
README.md
|
||||
configs/ssh/id_ed25519
|
||||
configs/ssh/id_ed25519.pub
|
||||
configs/turnstone/config.toml
|
||||
+12
@@ -0,0 +1,12 @@
|
||||
# VM state (SQLite db) — created at runtime
|
||||
data/*
|
||||
!data/.gitkeep
|
||||
|
||||
# Agent workspace
|
||||
work/*
|
||||
!work/.gitkeep
|
||||
|
||||
# Secrets: private ssh key + derived pubkey, real turnstone config
|
||||
configs/ssh/id_ed25519*
|
||||
configs/ssh/authorized_keys
|
||||
configs/turnstone/config.toml
|
||||
@@ -0,0 +1,76 @@
|
||||
FROM archlinux:latest
|
||||
|
||||
# Metadata
|
||||
LABEL maintainer="marvin"
|
||||
LABEL description="Turnstone agent server, built as a boco container-VM"
|
||||
|
||||
# Build-time arguments for UID/GID mapping
|
||||
ARG USER_ID=1000
|
||||
ARG GROUP_ID=1000
|
||||
|
||||
# Set environment variables
|
||||
ENV HOME=/home/marvin
|
||||
|
||||
# Install system dependencies via pacman
|
||||
RUN pacman -Syu --noconfirm && pacman -S --noconfirm --needed \
|
||||
bash \
|
||||
ca-certificates \
|
||||
curl \
|
||||
jq \
|
||||
git \
|
||||
github-cli \
|
||||
tea \
|
||||
helix \
|
||||
nushell \
|
||||
rust \
|
||||
rust-analyzer \
|
||||
rustfmt \
|
||||
go \
|
||||
podman \
|
||||
base-devel \
|
||||
openssl \
|
||||
tar \
|
||||
gzip \
|
||||
openssh \
|
||||
sudo \
|
||||
python \
|
||||
uv \
|
||||
&& pacman -Sc --noconfirm
|
||||
|
||||
# Create the 'marvin' user and set up home directory with provided UID/GID
|
||||
RUN groupadd -g ${GROUP_ID} marvin || true && \
|
||||
useradd -m -u ${USER_ID} -g marvin -s /bin/bash marvin && \
|
||||
mkdir -p /home/marvin/.local/bin /home/marvin/work && \
|
||||
install -d -o ${USER_ID} -g ${GROUP_ID} -m 750 /home/marvin/.config && \
|
||||
chown -R marvin:marvin /home/marvin && \
|
||||
echo "marvin ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers
|
||||
|
||||
# Rootless podman subuid/subgid mapping (deterministic single entry —
|
||||
# duplicates make podman request overlapping uid_map ranges, which the
|
||||
# kernel rejects with EINVAL)
|
||||
RUN printf "marvin:100000:65536\n" > /etc/subuid && \
|
||||
printf "marvin:100000:65536\n" > /etc/subgid
|
||||
|
||||
# SSH access for marvin (pubkey generated by scripts/build.sh)
|
||||
COPY configs/ssh/authorized_keys /tmp/authorized_keys
|
||||
RUN install -o ${USER_ID} -g ${GROUP_ID} -m 700 -d /home/marvin/.ssh && \
|
||||
install -o ${USER_ID} -g ${GROUP_ID} -m 600 /tmp/authorized_keys \
|
||||
/home/marvin/.ssh/authorized_keys && \
|
||||
rm /tmp/authorized_keys
|
||||
|
||||
# Turnstone in a dedicated venv (Arch python is externally managed)
|
||||
RUN uv venv /opt/turnstone --python /usr/bin/python && \
|
||||
uv pip install --python /opt/turnstone/bin/python turnstone && \
|
||||
for f in /opt/turnstone/bin/turnstone*; do ln -s "$f" /usr/local/bin/; done
|
||||
|
||||
# VM entrypoint (root: starts sshd, fixes share ownership, drops to marvin)
|
||||
COPY start-turnstone.sh /usr/local/bin/start-turnstone.sh
|
||||
RUN chmod 755 /usr/local/bin/start-turnstone.sh
|
||||
|
||||
# Optional: paru + openscad from AUR (slow builds)
|
||||
# RUN git clone https://aur.archlinux.org/paru.git && \
|
||||
# cd paru && makepkg -si --noconfirm && cd .. && rm -rf paru && \
|
||||
# paru --removemake --cleanafter --aur -S openscad-git
|
||||
|
||||
WORKDIR /home/marvin
|
||||
CMD ["/usr/local/bin/start-turnstone.sh"]
|
||||
@@ -0,0 +1,97 @@
|
||||
# marvin-turnstone
|
||||
|
||||
Local turnstone deployment as a **container-VM**: a podman image is turned
|
||||
into a bootable VM with [boco](https://git.nubo.sh/hulthe/boco) (custom
|
||||
init + ext4 qcow2 + QEMU direct kernel boot). Replaces the older
|
||||
opencode-in-a-container setup (`~/marvin`).
|
||||
|
||||
```
|
||||
host podman build ──> boco build qcow2 ──> boco boot
|
||||
│ QEMU/KVM (virtio, user-net)
|
||||
┌─────────────────────┴───────────────────┐
|
||||
│ VM (Arch) │
|
||||
│ /boco/init → start-turnstone.sh (root) │
|
||||
│ ├─ sshd (ops channel) │
|
||||
│ └─ setpriv marvin │
|
||||
│ └─ turnstone-server :8080 PID 1 │
|
||||
└─────────────────────────────────────────┘
|
||||
host :8080 ──hostfwd──> web UI host :2222 ──> guest sshd :22
|
||||
guest ──HTTPS (double slirp egress)──> api.tensorx.ai (LLM)
|
||||
```
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Host (Arch):
|
||||
- `podman`, `qemu-base`, `virtiofsd`, `croc` (only if fetching the kernel)
|
||||
- `boco` CLI on PATH and `boco-init` at `~/.local/share/boco/boco-init`
|
||||
(build from the boco repo)
|
||||
- Kernel at `~/.local/share/boco/registry/vmlinuz` (x86 bzImage with
|
||||
`CONFIG_VIRTIO_FS=y`; transfer via croc)
|
||||
|
||||
VM image:
|
||||
- Arch base with python + turnstone (venv at `/opt/turnstone`), git,
|
||||
helix, podman (rootless, subuid range), openssh, build toolchain
|
||||
- `marvin` user; sshd + turnstone-server started by
|
||||
`start-turnstone.sh` (server runs as `marvin`, PID 1)
|
||||
|
||||
## Build
|
||||
|
||||
```sh
|
||||
scripts/build.sh
|
||||
```
|
||||
|
||||
Creates `configs/ssh/id_ed25519` (ops keypair, once), builds the podman
|
||||
image, then packs it as `boco` qcow2 into the boco registry.
|
||||
|
||||
## Run
|
||||
|
||||
```sh
|
||||
scripts/run.sh
|
||||
```
|
||||
|
||||
- Web UI: <http://localhost:8080> (login `marvin` / `marvin-turnstone`
|
||||
unless `ADMIN_PASSWORD` was overridden — change it)
|
||||
- Ops SSH: `ssh -i configs/ssh/id_ed25519 -p 2222 marvin@localhost`
|
||||
- QEMU runs foreground; quit with `ctrl-a x`
|
||||
|
||||
### LLM backend
|
||||
|
||||
`[models.tensorx]` in `configs/turnstone/config.toml` points at the
|
||||
TensorX OpenAI-compatible gateway (`https://api.tensorx.ai/v1`, same
|
||||
key this machine's opencode uses — see
|
||||
`~/.local/share/opencode/auth.json`). Guest egress to HTTPS endpoints
|
||||
works through both slirp layers; no relay needed.
|
||||
|
||||
Self-hosted provider later: add a `[models.*]` entry with any
|
||||
OpenAI-compatible endpoint reachable from the guest (see
|
||||
`config.toml.example`).
|
||||
|
||||
## Shares (virtiofs, survives reboots)
|
||||
|
||||
| Host path | Guest path | Purpose |
|
||||
|------------------------|--------------------------------|--------------------------|
|
||||
| `configs/turnstone/` | `/home/marvin/.config/turnstone` | turnstone config.toml |
|
||||
| `data/` | `/var/lib/turnstone` | SQLite database |
|
||||
| `work/` | `/home/marvin/work` | agent workspace |
|
||||
|
||||
Guest-written files appear on the host owned by subuid-range UIDs
|
||||
(virtiofsd runs under `podman unshare`) — cosmetic, and stable across
|
||||
boots.
|
||||
|
||||
## Configuration
|
||||
|
||||
Edit `configs/turnstone/config.toml` (gitignored; see
|
||||
`config.toml.example`):
|
||||
|
||||
- `[models.tensorx]` — TensorX gateway; `api_key` must be set
|
||||
(currently mirrored from opencode's `auth.json`).
|
||||
- `[auth] jwt_secret` — required, generate once.
|
||||
- `[database] path` — SQLite on the `data/` share.
|
||||
|
||||
## Notes / trade-offs
|
||||
|
||||
- boco boots with `-snapshot`: **only the virtiofs shares persist.**
|
||||
Agent-installed packages, git clones outside `work/`, and podman image
|
||||
storage inside the VM are discarded on shutdown.
|
||||
- boco hardcodes `-smp 8`; memory via `VM_MEMORY=...M scripts/run.sh`.
|
||||
- The old setup's Traefik is dropped — hostfwd replaces it.
|
||||
Whitespace-only changes.
@@ -0,0 +1,33 @@
|
||||
# turnstone config — mounted into the VM at ~/.config/turnstone/config.toml
|
||||
# Reference: turnstone.example.toml in the upstream repo.
|
||||
|
||||
[auth]
|
||||
# JWT signing secret. Auth is always enabled; generate once:
|
||||
# python -c 'import secrets; print(secrets.token_hex(32))'
|
||||
jwt_secret = "CHANGE_ME_JWT_SECRET"
|
||||
|
||||
# Models are defined here or via the admin console Models tab.
|
||||
[models.tensorx]
|
||||
# TensorX — OpenAI-compatible gateway (same backend this opencode uses).
|
||||
model = "z-ai/glm-5.3-flash"
|
||||
provider = "openai"
|
||||
base_url = "https://api.tensorx.ai/v1"
|
||||
api_key = "CHANGE_ME_TENSORX_API_KEY"
|
||||
context_window = 1000000
|
||||
|
||||
# Self-hosted provider later? Any OpenAI-compatible endpoint reachable
|
||||
# from the guest works, e.g. llama.cpp on the LAN:
|
||||
# [models.local]
|
||||
# model = "..."
|
||||
# provider = "openai"
|
||||
# base_url = "http://<host>:<port>/v1"
|
||||
# max_concurrency = 1
|
||||
|
||||
[database]
|
||||
# SQLite file on the host-mounted data/ share; survives VM reboots.
|
||||
# (NOTE: for SQLite this is `path` — `url` is only used by PostgreSQL.)
|
||||
path = "/var/lib/turnstone/turnstone.db"
|
||||
|
||||
[tools]
|
||||
# Workspace surfaced to the model (mounted from ./work on the host).
|
||||
workspace_dir = "/home/marvin/work"
|
||||
Whitespace-only changes.
Executable
+17
@@ -0,0 +1,17 @@
|
||||
#!/usr/bin/env bash
|
||||
# Build the turnstone VM image: podman image -> boco qcow2.
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
# Dedicated ssh keypair for the VM ops channel (created once)
|
||||
if [ ! -f configs/ssh/id_ed25519 ]; then
|
||||
ssh-keygen -t ed25519 -N "" -C "marvin-turnstone-vm" -f configs/ssh/id_ed25519
|
||||
fi
|
||||
ssh-keygen -y -f configs/ssh/id_ed25519 > configs/ssh/authorized_keys
|
||||
|
||||
podman build \
|
||||
--build-arg USER_ID="$(id -u)" \
|
||||
--build-arg GROUP_ID="$(id -g)" \
|
||||
-t marvin-turnstone .
|
||||
|
||||
boco build qcow2 marvin-turnstone
|
||||
Executable
+18
@@ -0,0 +1,18 @@
|
||||
#!/usr/bin/env bash
|
||||
# Boot the turnstone VM (foreground; QEMU: ctrl-a x to quit).
|
||||
#
|
||||
# Web UI: http://localhost:8080
|
||||
# Ops SSH: ssh -i configs/ssh/id_ed25519 -p 2222 marvin@localhost
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
mkdir -p data work
|
||||
|
||||
exec boco boot marvin-turnstone \
|
||||
--memory "${VM_MEMORY:-4096M}" \
|
||||
--mount-mode direct \
|
||||
--forward "tcp:0.0.0.0:8080-:8080" \
|
||||
--forward "tcp:0.0.0.0:2222-:22" \
|
||||
--mount "$PWD/configs/turnstone:/home/marvin/.config/turnstone" \
|
||||
--mount "$PWD/data:/var/lib/turnstone" \
|
||||
--mount "$PWD/work:/home/marvin/work"
|
||||
Executable
+48
@@ -0,0 +1,48 @@
|
||||
#!/bin/bash
|
||||
# VM entrypoint, exec'd as PID 1 by /boco/init (image runs as root).
|
||||
# Starts sshd for ops access, prepares virtiofs share ownership, then
|
||||
# drops to the marvin user and execs turnstone-server.
|
||||
set -euo pipefail
|
||||
|
||||
export HOME=/home/marvin
|
||||
|
||||
# sshd: host keys baked per-boot, ops channel on guest port 22
|
||||
ssh-keygen -A
|
||||
mkdir -p /run/sshd
|
||||
/usr/sbin/sshd
|
||||
|
||||
# virtiofs shares appear root-owned (host dir is owned by the host user,
|
||||
# which maps to guest root) — hand them to marvin so the server and
|
||||
# agent tools can write. Also ensure ~/.config stays marvin-owned:
|
||||
# boco-init creates mount-point dirs as root, and rootless podman
|
||||
# refuses to run when $HOME/.config is not owned by the user.
|
||||
mkdir -p /var/lib/turnstone /home/marvin/.config
|
||||
chown marvin:marvin /var/lib/turnstone /home/marvin/.config || true
|
||||
chown -R marvin:marvin /home/marvin/work || true
|
||||
|
||||
# Rootless container networking (pasta/slirp4netns) needs TUN access;
|
||||
# the guest has no udev, so devtmpfs leaves it root-only.
|
||||
if [ ! -e /dev/net/tun ]; then
|
||||
mkdir -p /dev/net
|
||||
[ -c /dev/net/tun ] || mknod /dev/net/tun c 10 200
|
||||
fi
|
||||
chmod 666 /dev/net/tun || true
|
||||
|
||||
# Seed the admin account (no-op when the user already exists — password
|
||||
# is ignored on promotion). Runs as marvin so DB files it creates are
|
||||
# marvin-owned. Override via image env or boot cmd:
|
||||
# boco boot ... --cmd 'ADMIN_PASSWORD=... start-turnstone.sh'
|
||||
ADMIN_USER="${ADMIN_USER:-marvin}"
|
||||
ADMIN_PASSWORD="${ADMIN_PASSWORD:-marvin-turnstone}"
|
||||
if [ "$ADMIN_PASSWORD" = "marvin-turnstone" ]; then
|
||||
echo "start-turnstone: WARNING — default admin password in use; set ADMIN_PASSWORD to change it"
|
||||
fi
|
||||
setpriv --reuid=marvin --regid=marvin --init-groups \
|
||||
/usr/local/bin/turnstone-admin create-admin \
|
||||
--username "$ADMIN_USER" \
|
||||
--password "$ADMIN_PASSWORD" \
|
||||
|| echo "start-turnstone: admin seeding failed (continuing)"
|
||||
|
||||
# NOTE: turnstone reads $HOME/.config/turnstone/config.toml as marvin
|
||||
exec setpriv --reuid=marvin --regid=marvin --init-groups \
|
||||
/usr/local/bin/turnstone-server --host 0.0.0.0 --port 8080
|
||||
Whitespace-only changes.
Reference in new issue
Block a user