Turnstone container-VM: podman image -> boco qcow2 boot
Arch image with turnstone (venv), dev toolkit, rootless podman; boots via boco (virtiofs shares, direct UID mapping for nested virtiofs). sshd ops channel + turnstone-server as guest marvin; state (SQLite db, config, work) on host shares; LLM via the TensorX OpenAI-compatible gateway (https://api.tensorx.ai/v1), reachable from the guest through both slirp layers with no relay.
This commit is contained in:
commit
50b215f759
11 files changed
+311
No files matched your search
@@ -0,0 +1,10 @@
|
|||||||
|
# Keep build context minimal; authorized_keys is COPY'd in, the private
|
||||||
|
# key must never enter the image.
|
||||||
|
.git
|
||||||
|
data
|
||||||
|
work
|
||||||
|
scripts
|
||||||
|
README.md
|
||||||
|
configs/ssh/id_ed25519
|
||||||
|
configs/ssh/id_ed25519.pub
|
||||||
|
configs/turnstone/config.toml
|
||||||
+12
@@ -0,0 +1,12 @@
|
|||||||
|
# VM state (SQLite db) — created at runtime
|
||||||
|
data/*
|
||||||
|
!data/.gitkeep
|
||||||
|
|
||||||
|
# Agent workspace
|
||||||
|
work/*
|
||||||
|
!work/.gitkeep
|
||||||
|
|
||||||
|
# Secrets: private ssh key + derived pubkey, real turnstone config
|
||||||
|
configs/ssh/id_ed25519*
|
||||||
|
configs/ssh/authorized_keys
|
||||||
|
configs/turnstone/config.toml
|
||||||
@@ -0,0 +1,76 @@
|
|||||||
|
FROM archlinux:latest
|
||||||
|
|
||||||
|
# Metadata
|
||||||
|
LABEL maintainer="marvin"
|
||||||
|
LABEL description="Turnstone agent server, built as a boco container-VM"
|
||||||
|
|
||||||
|
# Build-time arguments for UID/GID mapping
|
||||||
|
ARG USER_ID=1000
|
||||||
|
ARG GROUP_ID=1000
|
||||||
|
|
||||||
|
# Set environment variables
|
||||||
|
ENV HOME=/home/marvin
|
||||||
|
|
||||||
|
# Install system dependencies via pacman
|
||||||
|
RUN pacman -Syu --noconfirm && pacman -S --noconfirm --needed \
|
||||||
|
bash \
|
||||||
|
ca-certificates \
|
||||||
|
curl \
|
||||||
|
jq \
|
||||||
|
git \
|
||||||
|
github-cli \
|
||||||
|
tea \
|
||||||
|
helix \
|
||||||
|
nushell \
|
||||||
|
rust \
|
||||||
|
rust-analyzer \
|
||||||
|
rustfmt \
|
||||||
|
go \
|
||||||
|
podman \
|
||||||
|
base-devel \
|
||||||
|
openssl \
|
||||||
|
tar \
|
||||||
|
gzip \
|
||||||
|
openssh \
|
||||||
|
sudo \
|
||||||
|
python \
|
||||||
|
uv \
|
||||||
|
&& pacman -Sc --noconfirm
|
||||||
|
|
||||||
|
# Create the 'marvin' user and set up home directory with provided UID/GID
|
||||||
|
RUN groupadd -g ${GROUP_ID} marvin || true && \
|
||||||
|
useradd -m -u ${USER_ID} -g marvin -s /bin/bash marvin && \
|
||||||
|
mkdir -p /home/marvin/.local/bin /home/marvin/work && \
|
||||||
|
install -d -o ${USER_ID} -g ${GROUP_ID} -m 750 /home/marvin/.config && \
|
||||||
|
chown -R marvin:marvin /home/marvin && \
|
||||||
|
echo "marvin ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers
|
||||||
|
|
||||||
|
# Rootless podman subuid/subgid mapping (deterministic single entry —
|
||||||
|
# duplicates make podman request overlapping uid_map ranges, which the
|
||||||
|
# kernel rejects with EINVAL)
|
||||||
|
RUN printf "marvin:100000:65536\n" > /etc/subuid && \
|
||||||
|
printf "marvin:100000:65536\n" > /etc/subgid
|
||||||
|
|
||||||
|
# SSH access for marvin (pubkey generated by scripts/build.sh)
|
||||||
|
COPY configs/ssh/authorized_keys /tmp/authorized_keys
|
||||||
|
RUN install -o ${USER_ID} -g ${GROUP_ID} -m 700 -d /home/marvin/.ssh && \
|
||||||
|
install -o ${USER_ID} -g ${GROUP_ID} -m 600 /tmp/authorized_keys \
|
||||||
|
/home/marvin/.ssh/authorized_keys && \
|
||||||
|
rm /tmp/authorized_keys
|
||||||
|
|
||||||
|
# Turnstone in a dedicated venv (Arch python is externally managed)
|
||||||
|
RUN uv venv /opt/turnstone --python /usr/bin/python && \
|
||||||
|
uv pip install --python /opt/turnstone/bin/python turnstone && \
|
||||||
|
for f in /opt/turnstone/bin/turnstone*; do ln -s "$f" /usr/local/bin/; done
|
||||||
|
|
||||||
|
# VM entrypoint (root: starts sshd, fixes share ownership, drops to marvin)
|
||||||
|
COPY start-turnstone.sh /usr/local/bin/start-turnstone.sh
|
||||||
|
RUN chmod 755 /usr/local/bin/start-turnstone.sh
|
||||||
|
|
||||||
|
# Optional: paru + openscad from AUR (slow builds)
|
||||||
|
# RUN git clone https://aur.archlinux.org/paru.git && \
|
||||||
|
# cd paru && makepkg -si --noconfirm && cd .. && rm -rf paru && \
|
||||||
|
# paru --removemake --cleanafter --aur -S openscad-git
|
||||||
|
|
||||||
|
WORKDIR /home/marvin
|
||||||
|
CMD ["/usr/local/bin/start-turnstone.sh"]
|
||||||
@@ -0,0 +1,97 @@
|
|||||||
|
# marvin-turnstone
|
||||||
|
|
||||||
|
Local turnstone deployment as a **container-VM**: a podman image is turned
|
||||||
|
into a bootable VM with [boco](https://git.nubo.sh/hulthe/boco) (custom
|
||||||
|
init + ext4 qcow2 + QEMU direct kernel boot). Replaces the older
|
||||||
|
opencode-in-a-container setup (`~/marvin`).
|
||||||
|
|
||||||
|
```
|
||||||
|
host podman build ──> boco build qcow2 ──> boco boot
|
||||||
|
│ QEMU/KVM (virtio, user-net)
|
||||||
|
┌─────────────────────┴───────────────────┐
|
||||||
|
│ VM (Arch) │
|
||||||
|
│ /boco/init → start-turnstone.sh (root) │
|
||||||
|
│ ├─ sshd (ops channel) │
|
||||||
|
│ └─ setpriv marvin │
|
||||||
|
│ └─ turnstone-server :8080 PID 1 │
|
||||||
|
└─────────────────────────────────────────┘
|
||||||
|
host :8080 ──hostfwd──> web UI host :2222 ──> guest sshd :22
|
||||||
|
guest ──HTTPS (double slirp egress)──> api.tensorx.ai (LLM)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Prerequisites
|
||||||
|
|
||||||
|
Host (Arch):
|
||||||
|
- `podman`, `qemu-base`, `virtiofsd`, `croc` (only if fetching the kernel)
|
||||||
|
- `boco` CLI on PATH and `boco-init` at `~/.local/share/boco/boco-init`
|
||||||
|
(build from the boco repo)
|
||||||
|
- Kernel at `~/.local/share/boco/registry/vmlinuz` (x86 bzImage with
|
||||||
|
`CONFIG_VIRTIO_FS=y`; transfer via croc)
|
||||||
|
|
||||||
|
VM image:
|
||||||
|
- Arch base with python + turnstone (venv at `/opt/turnstone`), git,
|
||||||
|
helix, podman (rootless, subuid range), openssh, build toolchain
|
||||||
|
- `marvin` user; sshd + turnstone-server started by
|
||||||
|
`start-turnstone.sh` (server runs as `marvin`, PID 1)
|
||||||
|
|
||||||
|
## Build
|
||||||
|
|
||||||
|
```sh
|
||||||
|
scripts/build.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
Creates `configs/ssh/id_ed25519` (ops keypair, once), builds the podman
|
||||||
|
image, then packs it as `boco` qcow2 into the boco registry.
|
||||||
|
|
||||||
|
## Run
|
||||||
|
|
||||||
|
```sh
|
||||||
|
scripts/run.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
- Web UI: <http://localhost:8080> (login `marvin` / `marvin-turnstone`
|
||||||
|
unless `ADMIN_PASSWORD` was overridden — change it)
|
||||||
|
- Ops SSH: `ssh -i configs/ssh/id_ed25519 -p 2222 marvin@localhost`
|
||||||
|
- QEMU runs foreground; quit with `ctrl-a x`
|
||||||
|
|
||||||
|
### LLM backend
|
||||||
|
|
||||||
|
`[models.tensorx]` in `configs/turnstone/config.toml` points at the
|
||||||
|
TensorX OpenAI-compatible gateway (`https://api.tensorx.ai/v1`, same
|
||||||
|
key this machine's opencode uses — see
|
||||||
|
`~/.local/share/opencode/auth.json`). Guest egress to HTTPS endpoints
|
||||||
|
works through both slirp layers; no relay needed.
|
||||||
|
|
||||||
|
Self-hosted provider later: add a `[models.*]` entry with any
|
||||||
|
OpenAI-compatible endpoint reachable from the guest (see
|
||||||
|
`config.toml.example`).
|
||||||
|
|
||||||
|
## Shares (virtiofs, survives reboots)
|
||||||
|
|
||||||
|
| Host path | Guest path | Purpose |
|
||||||
|
|------------------------|--------------------------------|--------------------------|
|
||||||
|
| `configs/turnstone/` | `/home/marvin/.config/turnstone` | turnstone config.toml |
|
||||||
|
| `data/` | `/var/lib/turnstone` | SQLite database |
|
||||||
|
| `work/` | `/home/marvin/work` | agent workspace |
|
||||||
|
|
||||||
|
Guest-written files appear on the host owned by subuid-range UIDs
|
||||||
|
(virtiofsd runs under `podman unshare`) — cosmetic, and stable across
|
||||||
|
boots.
|
||||||
|
|
||||||
|
## Configuration
|
||||||
|
|
||||||
|
Edit `configs/turnstone/config.toml` (gitignored; see
|
||||||
|
`config.toml.example`):
|
||||||
|
|
||||||
|
- `[models.tensorx]` — TensorX gateway; `api_key` must be set
|
||||||
|
(currently mirrored from opencode's `auth.json`).
|
||||||
|
- `[auth] jwt_secret` — required, generate once.
|
||||||
|
- `[database] path` — SQLite on the `data/` share.
|
||||||
|
|
||||||
|
## Notes / trade-offs
|
||||||
|
|
||||||
|
- boco boots with `-snapshot`: **only the virtiofs shares persist.**
|
||||||
|
Agent-installed packages, git clones outside `work/`, and podman image
|
||||||
|
storage inside the VM are discarded on shutdown.
|
||||||
|
- boco hardcodes `-smp 8`; memory via `VM_MEMORY=...M scripts/run.sh`.
|
||||||
|
- The old setup's Traefik is dropped — hostfwd replaces it.
|
||||||
Whitespace-only changes.
@@ -0,0 +1,33 @@
|
|||||||
|
# turnstone config — mounted into the VM at ~/.config/turnstone/config.toml
|
||||||
|
# Reference: turnstone.example.toml in the upstream repo.
|
||||||
|
|
||||||
|
[auth]
|
||||||
|
# JWT signing secret. Auth is always enabled; generate once:
|
||||||
|
# python -c 'import secrets; print(secrets.token_hex(32))'
|
||||||
|
jwt_secret = "CHANGE_ME_JWT_SECRET"
|
||||||
|
|
||||||
|
# Models are defined here or via the admin console Models tab.
|
||||||
|
[models.tensorx]
|
||||||
|
# TensorX — OpenAI-compatible gateway (same backend this opencode uses).
|
||||||
|
model = "z-ai/glm-5.3-flash"
|
||||||
|
provider = "openai"
|
||||||
|
base_url = "https://api.tensorx.ai/v1"
|
||||||
|
api_key = "CHANGE_ME_TENSORX_API_KEY"
|
||||||
|
context_window = 1000000
|
||||||
|
|
||||||
|
# Self-hosted provider later? Any OpenAI-compatible endpoint reachable
|
||||||
|
# from the guest works, e.g. llama.cpp on the LAN:
|
||||||
|
# [models.local]
|
||||||
|
# model = "..."
|
||||||
|
# provider = "openai"
|
||||||
|
# base_url = "http://<host>:<port>/v1"
|
||||||
|
# max_concurrency = 1
|
||||||
|
|
||||||
|
[database]
|
||||||
|
# SQLite file on the host-mounted data/ share; survives VM reboots.
|
||||||
|
# (NOTE: for SQLite this is `path` — `url` is only used by PostgreSQL.)
|
||||||
|
path = "/var/lib/turnstone/turnstone.db"
|
||||||
|
|
||||||
|
[tools]
|
||||||
|
# Workspace surfaced to the model (mounted from ./work on the host).
|
||||||
|
workspace_dir = "/home/marvin/work"
|
||||||
Whitespace-only changes.
Executable
+17
@@ -0,0 +1,17 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Build the turnstone VM image: podman image -> boco qcow2.
|
||||||
|
set -euo pipefail
|
||||||
|
cd "$(dirname "$0")/.."
|
||||||
|
|
||||||
|
# Dedicated ssh keypair for the VM ops channel (created once)
|
||||||
|
if [ ! -f configs/ssh/id_ed25519 ]; then
|
||||||
|
ssh-keygen -t ed25519 -N "" -C "marvin-turnstone-vm" -f configs/ssh/id_ed25519
|
||||||
|
fi
|
||||||
|
ssh-keygen -y -f configs/ssh/id_ed25519 > configs/ssh/authorized_keys
|
||||||
|
|
||||||
|
podman build \
|
||||||
|
--build-arg USER_ID="$(id -u)" \
|
||||||
|
--build-arg GROUP_ID="$(id -g)" \
|
||||||
|
-t marvin-turnstone .
|
||||||
|
|
||||||
|
boco build qcow2 marvin-turnstone
|
||||||
Executable
+18
@@ -0,0 +1,18 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Boot the turnstone VM (foreground; QEMU: ctrl-a x to quit).
|
||||||
|
#
|
||||||
|
# Web UI: http://localhost:8080
|
||||||
|
# Ops SSH: ssh -i configs/ssh/id_ed25519 -p 2222 marvin@localhost
|
||||||
|
set -euo pipefail
|
||||||
|
cd "$(dirname "$0")/.."
|
||||||
|
|
||||||
|
mkdir -p data work
|
||||||
|
|
||||||
|
exec boco boot marvin-turnstone \
|
||||||
|
--memory "${VM_MEMORY:-4096M}" \
|
||||||
|
--mount-mode direct \
|
||||||
|
--forward "tcp:0.0.0.0:8080-:8080" \
|
||||||
|
--forward "tcp:0.0.0.0:2222-:22" \
|
||||||
|
--mount "$PWD/configs/turnstone:/home/marvin/.config/turnstone" \
|
||||||
|
--mount "$PWD/data:/var/lib/turnstone" \
|
||||||
|
--mount "$PWD/work:/home/marvin/work"
|
||||||
Executable
+48
@@ -0,0 +1,48 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
# VM entrypoint, exec'd as PID 1 by /boco/init (image runs as root).
|
||||||
|
# Starts sshd for ops access, prepares virtiofs share ownership, then
|
||||||
|
# drops to the marvin user and execs turnstone-server.
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
export HOME=/home/marvin
|
||||||
|
|
||||||
|
# sshd: host keys baked per-boot, ops channel on guest port 22
|
||||||
|
ssh-keygen -A
|
||||||
|
mkdir -p /run/sshd
|
||||||
|
/usr/sbin/sshd
|
||||||
|
|
||||||
|
# virtiofs shares appear root-owned (host dir is owned by the host user,
|
||||||
|
# which maps to guest root) — hand them to marvin so the server and
|
||||||
|
# agent tools can write. Also ensure ~/.config stays marvin-owned:
|
||||||
|
# boco-init creates mount-point dirs as root, and rootless podman
|
||||||
|
# refuses to run when $HOME/.config is not owned by the user.
|
||||||
|
mkdir -p /var/lib/turnstone /home/marvin/.config
|
||||||
|
chown marvin:marvin /var/lib/turnstone /home/marvin/.config || true
|
||||||
|
chown -R marvin:marvin /home/marvin/work || true
|
||||||
|
|
||||||
|
# Rootless container networking (pasta/slirp4netns) needs TUN access;
|
||||||
|
# the guest has no udev, so devtmpfs leaves it root-only.
|
||||||
|
if [ ! -e /dev/net/tun ]; then
|
||||||
|
mkdir -p /dev/net
|
||||||
|
[ -c /dev/net/tun ] || mknod /dev/net/tun c 10 200
|
||||||
|
fi
|
||||||
|
chmod 666 /dev/net/tun || true
|
||||||
|
|
||||||
|
# Seed the admin account (no-op when the user already exists — password
|
||||||
|
# is ignored on promotion). Runs as marvin so DB files it creates are
|
||||||
|
# marvin-owned. Override via image env or boot cmd:
|
||||||
|
# boco boot ... --cmd 'ADMIN_PASSWORD=... start-turnstone.sh'
|
||||||
|
ADMIN_USER="${ADMIN_USER:-marvin}"
|
||||||
|
ADMIN_PASSWORD="${ADMIN_PASSWORD:-marvin-turnstone}"
|
||||||
|
if [ "$ADMIN_PASSWORD" = "marvin-turnstone" ]; then
|
||||||
|
echo "start-turnstone: WARNING — default admin password in use; set ADMIN_PASSWORD to change it"
|
||||||
|
fi
|
||||||
|
setpriv --reuid=marvin --regid=marvin --init-groups \
|
||||||
|
/usr/local/bin/turnstone-admin create-admin \
|
||||||
|
--username "$ADMIN_USER" \
|
||||||
|
--password "$ADMIN_PASSWORD" \
|
||||||
|
|| echo "start-turnstone: admin seeding failed (continuing)"
|
||||||
|
|
||||||
|
# NOTE: turnstone reads $HOME/.config/turnstone/config.toml as marvin
|
||||||
|
exec setpriv --reuid=marvin --regid=marvin --init-groups \
|
||||||
|
/usr/local/bin/turnstone-server --host 0.0.0.0 --port 8080
|
||||||
Whitespace-only changes.
Reference in new issue
Block a user