travissmith
travissmith / README.md

travis@smith

OS
Windows 11 · local-first
Host
Exigere Solutions · Founder
Kernel
agentic-systems architect
Uptime
8 yrs clinical documentation
Shell
PowerShell · bash
IDE
Claude Code · Codex CLI
Languages
Programming
Python · TypeScript · JavaScript · Rust
Data
SQL · PostgreSQL
Real
English
Agentic
Orchestration
LangGraph · LangChain · LlamaIndex
Patterns
RAG · multi-agent · tool-use
Protocol
MCP servers · structured output
Models
Local LLM
Qwen2.5-Coder · Llama 3.1
Speech
Whisper · faster-whisper
Embeddings
nomic-embed-text
Frontier
Claude · GPT-5 · Codex
Runtime
Ollama · llama.cpp · vLLM
Vector
pgvector · Qdrant
Stack
Backend
FastAPI · PostgreSQL · Docker
Frontend
React · Next.js · Astro
Focus
Design
privacy-by-design
Practice
adversarial evals · memory
Contact
Location
Nashville, TN
Stats
Projects
5 · receipts below
Tests
200+ across showcased builds
Corpus
200 trials · 358 gold labels
Status
available for work

travis@smith:~$

About

Hands-on with retrieval-augmented generation, multi-agent orchestration, and MCP server development, backed by eight years of clinical-documentation domain experience. Founder of Exigere Solutions, building proprietary agentic systems.

Works AI-augmented — directing LLM tooling to move fast on implementation while owning the architecture, evaluation, and safety decisions. The projects below are where that shows: local-first privacy enforced by architecture, a multi-tenant agent platform where onboarding is a config file, and a research-integrity auditor that refuses to make a claim without a verbatim quote to back it.

Skills & tech

  • agentic-systems
  • multi-agent-orchestration
  • rag
  • memory
  • adversarial-evaluation
  • python
  • typescript
  • nextjs
  • local-first-llm
  • privacy-by-design
  • mcp-servers

Projects

field-intel Showcase

Conversation intelligence for in-home sales that the dealer owns — recordings scored against their own script with code-verified evidence, plus a pre-visit homeowner dossier built from live county records.

status built · proof-gated demo
  • python
  • react
  • whisper
  • local-first-llm
  • conversation-intelligence
  • public-records
  • geospatial
  • sales-coaching
Python README
marauder-cockpit Showcase

A desktop command cockpit for the ESP32 Marauder — live WiFi/BLE recon map, serial console, and a local-AI copilot, with attack tooling gated behind a real authorization model.

status built · verified end-to-end
  • python
  • react
  • cybersecurity
  • wifi-ble
  • hardware-serial
  • geospatial
  • local-first-llm
  • red-team-tooling
Python README
second-brain Showcase

A local-first personal memory & intelligence system with privacy enforced by architecture, not by query filters.

status built · ~118 tests
  • python
  • local-first
  • privacy-by-design
  • rag
  • llm
  • fail-closed
  • personal-knowledge-management
Python README
etzira Showcase

A white-label agentic-services platform where every client is a config file.

status built · awaiting live accounts
  • typescript
  • nextjs
  • agentic
  • white-label
  • multi-tenant
  • conversational-ai
  • compliance
  • saas
TypeScript README
private-ai-lab Showcase

A reference architecture for running local-first LLM & agentic workloads on-prem — production-grade security, zero cloud.

status reference architecture
  • homelab
  • self-hosted
  • local-first
  • llm-inference
  • security-architecture
  • zero-trust
  • infrastructure
  • threat-model
endpointdrift Showcase

Catches clinical trials moving the goalposts — audits published papers against their pre-registered outcomes, with receipts.

status in development · 101 tests
  • python
  • clinical-trials
  • research-integrity
  • document-ai
  • multi-agent
  • evals
  • llmops
  • local-first
Python README

field-intel

stack Python · FastAPI · React status built · verified end-to-end domain conversation intelligence · geodata design local-first · evidence-verified license proprietary
  • python
  • react
  • whisper
  • local-first-llm
  • conversation-intelligence
  • public-records
  • geospatial
  • sales-coaching
README.md Python

Field Intel

Conversation intelligence for in-home sales that the dealer owns — every appointment recording scored against their own sales script with code-verified evidence, plus a pre-visit homeowner dossier built from live county records. Local-first: Whisper, a local LLM, and county GIS on one box; nothing leaves the building.

Live demo (looping, 90s): a pre-visit dossier built from real Davidson County records, then two recorded appointments scored by the local pipeline — a 94% on-script rep vs a 31% off-script rep whose missed recording-consent step is caught automatically. Fictional appointments; real public-record data; real pipeline output.

Overview

High-ticket in-home sales runs on two blind spots: the company has no record of what was said in the living room where a five-figure deal closes, and the rep walks in knowing nothing about the home beyond a name and an address. The incumbent fix is renting a per-seat cloud conversation-intelligence platform — the recordings, the scores, and the tuned models all live on someone else’s servers.

Field Intel closes both blind spots on hardware the dealer owns. Appointment recordings are transcribed and scored locally against the dealer’s own sales methodology — every step, every behavior tracker, every coaching note. Before the visit, a dossier assembles what’s knowable about the home from public records: assessed value, sale history, owner tenure, permit activity, comparable sales, and a neighborhood map rendered straight from county parcel geometry.

What it does

  • Script scorecards with receipts. A local LLM scores each conversation against the dealer’s methodology step by step. Every evidence quote the model cites is verified against the transcript by code — fuzzy-matched to a real utterance with a real timestamp, or flagged as unlocated. Click a quote; the audio jumps to that moment. The scorecard cannot silently fabricate evidence.
  • Consent compliance, deterministic. The recording-notification step — the one reps actually skip — is detected by a deterministic pass, not model judgment, and surfaced as a hard flag with the timestamp and quote.
  • Behavior trackers with a speaker guard. Nine cross-cutting trackers (objections, price reactions, payment conditioning…). Each tracker declares who must speak its evidence; a fired tracker whose verified quote comes from the wrong speaker is auto-corrected — the model can’t credit the rep with the customer’s words.
  • Pre-visit dossier on live public data. County assessor parcels, building permits, and Census tract statistics, composed into home facts, comps, an equity/affluence read, and plain-language door signals (“purchased five weeks ago — new-owner improvement window”).
  • An offline blueprint map. The territory map renders county parcel polygons directly as vectors — no basemap tiles, no network dependency — so it works in a dead-WiFi living room.
  • Ask-the-call. A local RAG chat over the transcript and scorecard: “Where did the rep lose the customer?” answers with verbatim quotes and clickable timestamps.

Architecture

  appointment audio ──► Whisper (local) ──► speaker attribution ──► local LLM scoring
                                                                    (per-step · trackers · coaching)
  county GIS + permits + ACS ──► dossier engine ──► FastAPI ──► React console
  (fetched once, cached — zero cloud calls at runtime)      evidence quotes verified by code

Everything runs on one machine: transcription, scoring, embeddings, chat, and the dossier engine. The scoring pipeline makes zero cloud calls; public-records data is fetched once and served from a local cache. The rubric is configuration — a client’s methodology loads as data, with a one-click generic variant that relabels every stored scorecard instantly.

The part that matters: evidence you can’t fake

Small local models hallucinate; a coaching tool that invents quotes is worse than no tool. Field Intel’s countermeasure is structural: the model’s output is treated as claims, and code checks every claim against ground truth. Quotes are located in the real transcript or flagged. Tracker evidence is attributed to the actual speaker or the tracker is auto-corrected. The consent flag never comes from the model at all. The result on the demo recordings: 19 of 19 evidence quotes verified on the strong appointment, and a scorecard where every number survives an adversarial “is this real?”.

Tech stack

Pipeline: Python · OpenAI Whisper · Ollama (local LLM + embeddings) · FastAPI · Frontend: React · Vite · offline vector maps from county GIS parcel geometry · Data: county assessor + open-data permits + Census ACS, cached locally.

Status & roadmap

Built · verified end-to-end. Proof gate: two mock appointments run through the full local pipeline — the strong rep scores 94% on-script with all 19 evidence quotes machine-verified; the sloppy rep scores 31% with the missed consent step caught deterministically. Dossiers compose from a cache of 504 real Davidson County parcels in ~6 ms. Next:

  1. Real diarization — pyannote speaker separation behind the existing attribution interface.
  2. Mobile capture — record, offline buffer, one-tap consent logging, auto-upload.
  3. Dossier A/B instrumentation — measure close-rate lift from dossier-briefed appointments; the category has no published lift numbers.

Showcase — a proprietary product build. Demo appointments are scripted and fictional; property data is real public record; scorecards shown are actual pipeline output, not mockups.

marauder-cockpit

stack Python · FastAPI · React status built · verified end-to-end domain offensive security · hardware safety scoped · arm-confirm · audited license proprietary
  • python
  • react
  • cybersecurity
  • wifi-ble
  • hardware-serial
  • geospatial
  • local-first-llm
  • red-team-tooling
README.md Python

Marauder Cockpit

A desktop command cockpit for the ESP32 Marauder — a live WiFi/BLE reconnaissance map, a serial command console, and a local-AI copilot, with offensive tooling gated behind a real authorization model.

Live demo (looping): the cockpit driving a synthetic wardrive — GPS-tracked recon map, live access-point table coloured by encryption, and a streaming serial console. Real capture, replayed so it runs without hardware.

Overview

The ESP32 Marauder is a capable WiFi/BLE reconnaissance and testing tool, but it’s driven from a tiny on-device touchscreen or a raw serial prompt. Marauder Cockpit turns it into a proper operator console: plug the device in over USB and drive the whole thing — scanning, wardriving, GPS, sniffing, and gated attack tooling — from one screen, with the captured telemetry rendered live.

It’s a single artifact that deliberately spans three disciplines: hardware/serial integration, real-time geospatial telemetry, and applied local-LLM analysis — built for authorized, own-device red/blue bench work.

What it does

  • Live recon map. As the device wardrives, each access point is plotted on a Leaflet map coloured by security (open → WEP → WPA → WPA2 → WPA3), with the operator’s GPS position tracked in real time.
  • Serial command console. A full conversational console to the Marauder firmware — scanap, wardrive, sniffpmkid, gps, and the rest — with responses streamed back inline.
  • AI copilot. A local LLM (via Ollama) answers questions over the captured telemetry — “summarize what I captured and flag anything insecure” — entirely offline; no data leaves the machine.
  • Gated command center. Offensive commands (deauth, beacon/BLE spam, evil portal) run only behind an explicit authorization model — see below.

Architecture

  ESP32 Marauder ──USB serial (COM5 · 115200)──► Python backend ──WebSocket──► React cockpit
  (WiFi / BLE recon)                              FastAPI · pyserial            Vite · Leaflet

                                                       └── local Ollama  (AI copilot · fully offline)

A single reader thread owns the serial link and parses the firmware’s chatty, semi-structured output into typed events (GPS fixes, access points, console lines); a tolerant parser forwards anything it doesn’t recognise straight to the console rather than dropping it. Events fan out to the browser over a WebSocket, so the map, table, and console update as fast as the device reports. Because a GPS fix needs open sky, a replay mode streams a recorded wardrive so the entire cockpit — including the live map — is demonstrable indoors (that’s what the video above shows).

Authorization model (the part that matters)

Passively collecting beacon metadata is one thing; transmitting deauth or spam frames is another, and is only lawful against hardware you own or are explicitly authorized to test. The cockpit is built to enforce that distinction the way real engagement tooling does — the restraint is a design feature, not an afterthought:

ControlWhat it enforces
Scope allow-listA target must be registered as your own device before it can be attacked — the cockpit’s version of an engagement scope file.
Arm → confirmThe guided path requires arming a specific in-scope target and a per-session authorization confirmation before any attack transmits.
Live-Fire modeFor fluid bench work, a deliberate session toggle unlocks attack commands in the console — off by default, with a persistent red state banner so it’s never accidental.
Audit logEvery command, recon or attack, is written to an append-only, timestamped log — an engagement-record trail.

The invariant: an attack never transmits without either an explicit confirmation or the Live-Fire toggle, and never without being logged.

Tech stack

Backend: Python · FastAPI · pyserial · WebSockets · Ollama (local LLM) · Frontend: React · Vite · Leaflet · WebSocket streaming · Hardware: ESP32 Marauder over CP210x USB-UART.

Status & roadmap

Built · verified end-to-end. The serial engine, live map, gated command model, audit log, and copilot are working and were verified against the real device’s serial interface and a replayed wardrive. Next:

  1. Capture export — persist wardrives to WiGLE-format CSV / PCAP.
  2. Offline map tiles — bundled MBTiles for fully air-gapped operation.
  3. Anomaly detection — evil-twin and recurring-BLE-tracker flags across repeat runs (the genuinely AI-heavy layer).

Showcase — a personal red/blue engineering tool for authorized, own-device security testing. Not for use against networks or devices you do not own or have written permission to test.

second-brain

language Python status built · ~118 tests design local-first · fail-closed license proprietary
  • python
  • local-first
  • privacy-by-design
  • rag
  • llm
  • fail-closed
  • personal-knowledge-management
README.md Python

Second Brain

A local-first personal memory and intelligence system with privacy enforced by architecture, not by a query filter.

Overview

Second Brain ingests your own material — documents, notes, screenshots, photos, and chat exports — and turns it into a private, queryable memory that reasons across the connections in your life and surfaces what’s missing. Everything runs on your own hardware: nothing is uploaded, and sensitive content is structurally prevented from ever reaching a general search, a summary, or any outbound surface.

The central design bet is simple: privacy is a property of the architecture, not a setting. Sensitivity is decided up front by a fail-closed classifier, and every layer downstream is gated by it — so a leak isn’t “a bug in a WHERE clause,” it’s something the data model makes impossible by construction.

It’s built for a threat model where the privacy guarantee has to be structural — the system is architecturally unable to read protected data, not merely under a policy that promises it won’t.

Highlights

  • Structural privacy zoning. Data is tiered by sensitivity, and the tiers are physically separated stores — not rows behind an access check. General retrieval is hard-wired to the non-sensitive stores only.
  • Fail-closed by default. Anything not positively promoted to an open tier is treated as sensitive. Tiers can only ratchet up (toward more private) on re-ingest, never down. Ambiguous, undecodable, or out-of-policy inputs resolve to the safe direction.
  • Encrypt-only vault for the highest tiers. The always-on process holds a public key only and cannot decrypt what it seals; the most private material is quarantined at rest and recorded only as a content-free ledger row — an opaque id, tier, month, and size bucket, nothing reversible.
  • A safety floor that’s enforced, not assumed. An egress kill-switch, a default-deny tier router, a fail-closed quarantine sentinel, and tiered retrieval against a hard-coded allow-list all run beneath everything else.
  • Scope-gated reasoning. The reasoning loop runs inside an explicit scope. A general scope can never read sensitive content; a privileged scope can, but its output is tagged with the highest tier it touched and routed accordingly.
  • ACL-fenced read layer. A read-only, access-controlled Obsidian layer lets you browse the public brain with backlinks and a graph view while staying strictly inside the privacy model.
  • Vision pipeline. Images pass a pre-gate before any caption is generated, so descriptive text of private media is never produced in the first place.

How it works

The system is a stack of layers, each enforcing the privacy model at its own level, with a reasoning loop on top. Ingestion flows up the stack; nothing skips a layer.

┌───────────────────────────────────────────────────────────────┐
│  Human surface — Markdown vault + tier-routed briefs           │
├───────────────────────────────────────────────────────────────┤
│  Intelligence loop — classify · infer · reason · gaps · brief  │
├───────────────────────────────────────────────────────────────┤
│  Scoped retrieval + tier-aware connection graph                │
├───────────────────────────────────────────────────────────────┤
│  Storage — projection DB (open tiers) + content-free ledger    │
├───────────────────────────────────────────────────────────────┤
│  Quarantine sentinel — pre-gate + encrypt-only vault           │
├───────────────────────────────────────────────────────────────┤
│  Tier router — default-deny sensitivity classifier             │
├───────────────────────────────────────────────────────────────┤
│  Egress control — loopback-only inference + outbound-deny      │
└───────────────────────────────────────────────────────────────┘

A boot self-test first proves egress is closed; if it can’t, ingestion refuses to run. Every item then passes the default-deny classifier, which assigns exactly one sensitivity tier from content, path signals, and detector output. Private and uncertain items are sealed into the encrypt-only vault before anything else touches them. Open-tier items land in a projection database whose privacy invariants are enforced by schema-level triggers; the most private items leave only a content-free ledger row ({opaque_id, tier, month, size_bucket}), so “what did I file this month” is a COUNT(*) and nothing more.

The sensitivity tiers

TierHandling
Normal / PersonalIndexed into the general store; freely retrievable within scope.
Sensitive — health, legal, financialSeparate store; never in general retrieval; restricted processing.
PrivateEncrypt-only, quarantined; never indexed; kept only as a count.
Do-Not-ProcessUndecodable or out-of-policy; quarantined, never indexed.

The reasoning loop

classify-kind → infer connections → reason (scoped) → detect gaps → tier-routed brief

Each stage is scope-gated. It derives a semantic kind for each entity, proposes evidence-based connections (auto-routing cross-tier links to a privileged store and never overwriting a human-authored link), answers questions by expanding along the connection graph within a scope ceiling, surfaces what’s missing or stale, and finally writes a brief to the location its sensitivity allows. The language-model step is pluggable — a deterministic fake for tests today, with a local model as the drop-in for real hardware — so the loop is designed to run fully offline.

Design invariants

These are the load-bearing properties the system holds, enforced in code and locked by the test suite:

  • A general scope (and the human editor) can never read Sensitive content; Private/Do-Not-Process content is unreachable from any scope.
  • Graph reads re-filter every endpoint against the scope ceiling, fail-closed, even if a link were ever mis-stored.
  • Sensitivity is default-deny and ratchet-up-only; an item never auto-downgrades.
  • A brief that touched sensitive content is routed to a fenced location, never the readable surface.
  • The always-on machine holds no decryption capability.
  • Egress and attestation checks fail closed.

Tech stack

  • Language: Python (standard library plus a small, well-scoped set of dependencies).
  • Storage: SQLite across two isolated databases, with an append-only, tamper-evident audit trail and schema-level triggers that encode the privacy invariants.
  • Inference: a pluggable, Ollama-compatible local inference and retrieval backend; vision, OCR, and embedding backends sit behind interfaces with deterministic fakes for headless testing.
  • Encryption: modern X25519 file encryption for the quarantine vault (public-key-only on the host).
  • Detection: built-in heuristic PII/PHI detection with an optional local NER backend.
  • Source of truth: a structured Markdown vault; the database is its queryable projection.

No component requires network access at runtime; the system is designed to run fully offline.

Status & roadmap

Built and tested. ~118 passing tests lock the privacy invariants in place — a cross-tier leak is treated as a hard test failure. The classifier, safety floor, encrypt-only quarantine, two-database storage, ingestion pipeline, scoped retrieval, the connection graph, the full reasoning loop, and the fenced read layer are implemented and verified in the test harness with injectable backends.

The privacy model has been hardened through repeated adversarial red-team / blue-team review. No cross-tier leak has survived those passes or the test suite to date; the outstanding work is operational and policy — deployment hardening, host-OS configuration, and real-model integration on target hardware — rather than known holes in the privacy model.

Next:

  • Deployment and operational hardening.
  • A self-running scheduler over the manual sync command.
  • Richer, model-assisted relationship extraction beyond keyword evidence.
  • Opt-in, local photo-library backfill.

Showcase repository — this README documents the design; source is available on request.

etzira

language TypeScript status built · awaiting live accounts stack Next.js · R3F license proprietary
  • typescript
  • nextjs
  • agentic
  • white-label
  • multi-tenant
  • conversational-ai
  • compliance
  • saas
README.md TypeScript

Etzira

A white-label agentic-services platform where every client is a config file.

Overview

Etzira packages AI agents into productized services for local revenue businesses — the phone, the follow-up, the marketing, the CRM — and runs them as configuration rather than code. Every client tenant is a single JSON file, so onboarding a new business, or an entire new vertical, ships without touching the codebase. The same agent engine renders as a pilates studio or a med spa from its tenant file alone.

It is built as one engine sold many ways: acquire, market, answer, sell, and track — the universal functions every business shares — delivered by the same agents with per-tenant knowledge and configuration swapped in.

Highlights

  • Agents-as-config. A tenant is a JSON file: persona, hours, knowledge base, channels, compliance flags. White-labeling is a new tenant file, not a new deployment.
  • Five products from one engine. Front Desk, Reach, Amplify, Close, and Core — a lifecycle suite (see below), composed from a broader package catalog and sold à la carte or as a bundle.
  • Nine vertical demo tenants. The multi-tenant model is proven across nine business types from config alone — no per-vertical code.
  • Compliance built into the core. TCPA/PEWC consent handling, a persistent opt-out store, and an honest QA gate every agent must pass before go-live.
  • Performance-based billing. A metered, arrears billing track (per booked outcome) sits alongside flat pricing, with the billable event self-metered from the system of record.
  • The site runs the product. The front end is a Next.js + React-Three-Fiber experience that runs the same agents it sells, so the demo exercises the production engine rather than mocking it.

The five products

ProductFunctionWhat it does
CoreCRM / system of recordOne clean source of truth — contacts, pipelines, bookings, conversation history — kept deduped, enriched, and segmented by an AI data-hygiene agent.
ReachAcquisitionNet-new pipeline: outbound that finds and personalizes, list building and enrichment, and AI-search visibility.
Front DeskPhone & customer managementThe flagship. Answers inbound chat, web, and text — qualifies, books, routes edge cases to a human — 24/7 and consent-first. A voice receptionist for calls is on the roadmap.
AmplifyMarketingAlways-on content, campaigns, landing pages, and an automated review/reputation engine.
CloseSalesMulti-touch follow-up, nurture, reactivation, and rebooking that catch the revenue lost to weak follow-up.

Land with one — usually Front Desk, because it demonstrates itself in the live chat concierge on its own site — and expand to the full suite; each added product runs on the same Core.

How it works

Each tenant is a declarative config. The engine composes the right agents for the products that tenant has enabled, grounds them in that tenant’s knowledge base, and enforces the compliance core on every conversation.

   tenant.json


  config loader  ──▶  product resolver  ──▶  agent runtime
                     (Front Desk / Reach /   grounded in the
                      Amplify / Close / Core)  tenant knowledge base


                           compliance core  ── TCPA/PEWC · opt-out · QA gate


              channels ── voice (Vapi) · chat · SMS · CRM (GoHighLevel) · billing (Stripe)

A minimal, illustrative tenant file:

{
  "tenant": "example-dental",
  "product": "Front Desk",
  "voice": { "provider": "vapi", "persona": "warm-professional" },
  "hours": "Mon-Fri 8-5",
  "compliance": { "tcpa": true, "opt_out": true }
}

Adding a business is a new file like this one; adding a vertical is a set of them. The engine, the products, and the compliance core stay untouched.

Compliance core

Consent and honesty are enforced in the platform, not left to each tenant:

  • TCPA / PEWC consent handling on outbound and voice channels.
  • A persistent opt-out store honored across every channel and tenant.
  • An honest QA gate across the whole catalog — conversational agents are tested for hallucination, tool misuse, data leakage, wrong bookings, and failure-to-escalate; workflow agents pass mechanical definition checks plus stage-level acceptance cases; operator-verified checks require recorded, revocable attestations. Coverage gaps report as NOT COVERED — never silently green.
  • Solicitation-volume recordkeeping — every live automated send is logged per tenant and month, with threshold alarms and regulator-report drafts (built for Tennessee’s 2026 automated-solicitation law).

Tech stack

  • Front end: TypeScript · Next.js · React · React-Three-Fiber (the interactive 3D demo surface).
  • Agents & orchestration: the Claude API, tenant-configured and grounded per client.
  • Integrations: GoHighLevel (CRM spine) · Vapi (voice) · Stripe (billing).
  • Model: agents-as-config — tenants are JSON; products are a bundling layer over a shared package registry.

Status & roadmap

Built and tested against mocks. The framework, the five products, the nine demo tenants, the compliance core, and the billing runbook are in place and exercised end-to-end against mocked integrations. The platform is awaiting live account and key provisioning — GoHighLevel, Vapi, Stripe, telephony — to move from mock to production.

No live customers and no revenue are claimed. This is a built, demonstrable platform pending production credentials.

Next:

  • Provision live accounts, keys, and telephony.
  • Promote the demo tenants to production configs.
  • Harden the metered-billing path against real usage.

Showcase repository — this README documents the platform; source is available on request.

private-ai-lab

type reference architecture focus security · zero-egress stack Proxmox · vLLM · pgvector license proprietary
  • homelab
  • self-hosted
  • local-first
  • llm-inference
  • security-architecture
  • zero-trust
  • infrastructure
  • threat-model
README.md YAML

Private AI Lab

A reference architecture for running local-first LLM and agentic workloads on-prem — production-grade security, zero cloud.

Overview

Cloud AI means your prompts, documents, and data leave your control, and your bill scales with every token. For privacy-sensitive, compliance-bound, or documentation-heavy work, on-prem inference inverts that trade: full data custody, fixed cost, and no third party in the loop. But a GPU under a desk is not infrastructure.

This is a reference architecture for a private AI lab — the design for running local language and vision models and multi-agent workloads on hardware you own, engineered like production: segmented, secured, observable, and reproducible. It is the substrate a local-first system (private RAG, agentic pipelines — the kind of system Second Brain is designed to run on) actually needs underneath it.

It is a blueprint, not a claim of a specific deployed system — it documents the design decisions, the threat model, and the trade-offs, so the reasoning is inspectable rather than asserted.

Design principles

  • Zero-egress by default. Outbound network is denied unless explicitly allowed — models and dependencies cannot phone home by construction, not by policy.
  • Segmented by trust. Workloads are isolated into network zones so a single compromise can’t reach everything; blast radius is contained.
  • Reproducible. Host and service configuration is codified (infrastructure-as-code), not clicked together — the lab is rebuildable from git plus a data restore.
  • No public exposure. Nothing is internet-facing; access is identity-gated over a private tunnel.
  • Observable. Metrics, logs, and traces on every layer, so you can prove what ran and catch silent failure.
  • Boring, durable tech. Prosumer hardware and widely-supported open source over anything exotic.

Architecture

Traffic and trust flow through segmented zones behind a default-deny firewall; nothing is exposed to the internet, and remote access is an identity-gated tunnel — not a forwarded port.

                          Internet

                      ┌──────┴──────┐
                      │  firewall / │   default-deny egress
                      │   router    │   (OPNsense-class)
                      └──────┬──────┘
        ┌────────────────────┼────────────────────┐
        │                    │                     │
 ┌──────┴──────┐    ┌────────┴────────┐    ┌───────┴───────┐
 │ management  │    │  workload /     │    │   storage     │
 │   VLAN      │    │ inference VLAN  │    │    VLAN       │
 │ admin · IaC │    │ GPU · serving   │    │ NAS · backups │
 └─────────────┘    └─────────────────┘    └───────────────┘

  remote access: WireGuard (self-hosted, identity-gated) · no inbound ports

The lab is organized as layers, each with its own security responsibility:

┌────────────────────────────────────────────────────────────┐
│  Access — WireGuard/Headscale · SSO · reverse proxy + mTLS  │
├────────────────────────────────────────────────────────────┤
│  Platform — Ollama / vLLM · pgvector / Qdrant · Compose/k3s │
├────────────────────────────────────────────────────────────┤
│  Observability — Prometheus · Grafana · Loki · Langfuse     │
├────────────────────────────────────────────────────────────┤
│  Data — ZFS / LUKS encryption · tiered access · 3-2-1 backup│
├────────────────────────────────────────────────────────────┤
│  Network — VLAN segmentation · default-deny egress          │
├────────────────────────────────────────────────────────────┤
│  Host — hardened Linux / NixOS · least privilege · IaC      │
└────────────────────────────────────────────────────────────┘

Compute. A GPU inference node (a 24GB-class card comfortably serves quantized mid-size LLMs plus a vision model) carries model serving; CPU and RAM on the same or a separate low-power node run the control plane, the vector database, and lightweight services, so the always-on surface is decoupled from the power-hungry GPU box.

Storage tiers. Fast NVMe holds model weights, vector indexes, and hot data; bulk RAID/NAS holds datasets, documents, and backups; sensitive tiers are encrypted at rest (LUKS or ZFS native encryption). Backups follow 3-2-1 — the live data, a local snapshot on separate media, and an encrypted offsite copy (three copies, two media types, one off-site).

Network. VLANs separate management, inference/workload, and storage. A firewall enforces default-deny egress with per-zone allow-lists; the workload zone cannot reach the management plane, and storage is reachable only by the services that need it. Internal name resolution and a reverse proxy (Caddy/Traefik) provide automatic internal TLS.

Platform. Model serving (Ollama and/or vLLM) sits behind the reverse proxy on the workload VLAN; orchestration is declarative — Docker Compose by default, or k3s if multi-node is needed, in which case CNI NetworkPolicies mirror the VLAN trust boundaries and the control plane stays on the management VLAN; retrieval uses pgvector or Qdrant. Model weights are pinned by version and digest and verified against a signed upstream manifest (cosign/Sigstore) — the workload zone does not auto-pull from the internet. A single staging host on the management VLAN holds the only egress allow-list entry to the model registry: artifacts are pulled there, signature- and checksum-verified, then copied one-way onto the workload VLAN.

Security architecture

Security is the point of this design, so it starts from an explicit threat model and maps every control back to a threat.

ThreatControl
Data exfiltration — a model, dependency, or compromised service sends data outDefault-deny egress per VLAN; allow-list only required endpoints; alert on blocked egress
Lateral movement — one compromised container reaches sensitive storageVLAN + firewall segmentation; per-service least privilege; no shared secrets
Supply chain — malicious model weights, packages, or imagesDigests/checksums verified against a signed upstream manifest (cosign/Sigstore), not re-fetched from the same host as the artifact; minimal base images; no build-time network in sensitive zones
Malicious / vulnerable model code — unsafe weight deserialization or a serving-engine bug executes on the GPU hostRootless, sandboxed container runtime (seccomp/AppArmor), isolated user namespace, minimal host GPU-driver surface; prefer safetensors over pickle-based formats
Remote compromise — an exposed service is attackedNothing internet-facing; access is VPN-only with identity/MFA at the edge
Physical / theftFull-disk encryption; secrets in a vault, never plaintext on disk or in git
Silent failure / tamperingCentralized logs + append-only audit for security-relevant events (admin actions, egress denials)

The egress control here applies the same cannot-leak-by-construction principle that Second Brain enforces at the application layer — one layer down, at the firewall. Secrets live in a manager (self-hosted Vault, or SOPS/age-encrypted so they can be safely versioned) — never as plaintext in a compose file. Access is WireGuard — or, if Tailscale’s ergonomics are wanted, pinned to a self-hosted Headscale control plane so the coordination plane stays in-perimeter and the zero-egress posture holds — plus SSO for internal apps, SSH key-only with no passwords, and least-privilege service accounts. Internal TLS/mTLS certificates are issued and rotated by Vault’s PKI secrets engine (or step-ca). LLM traces (Langfuse) capture full prompts and completions — among the most sensitive data in the lab — so the trace store sits in the encrypted data tier under the same backup, retention, and redaction policy as the vector store. Security patches apply on a schedule, tracked in code.

Reproducibility

The lab is meant to be rebuildable from source, not from memory:

  • Host config — Ansible (or NixOS) declares OS hardening (a CIS-style baseline), users, firewall rules, and packages.
  • Services — Docker Compose / k3s manifests live in git; a service comes up with one command.
  • Secrets — SOPS/age-encrypted in git, or fetched from Vault — versionable without exposure.
  • Recovery — the whole lab reconstructs from the git repo plus a data restore.

(The write-up carries representative, illustrative configuration — this is a reference architecture, not a turnkey repository.)

Tech stack

OS / hypervisor: Proxmox or bare-metal Debian / NixOS · Network: OPNsense, VLAN-capable switch, WireGuard / Headscale · Serving: Ollama, vLLM · Data: Postgres + pgvector, Qdrant, ZFS · Orchestration: Docker Compose, k3s · Observability: Prometheus, Grafana, Loki, Langfuse · Secrets: Vault, SOPS + age · IaC: Ansible, NixOS · Proxy: Caddy, Traefik.

Status & roadmap

Reference architecture (design). This documents the architecture, threat model, and trade-offs — a blueprint, not an assertion that a specific system is deployed.

A phased build-out following this design:

  1. Network foundation — VLAN segmentation and default-deny egress first, before any workload.
  2. Inference node — GPU host, model serving behind the proxy.
  3. Storage & recovery — encrypted tiers, 3-2-1 backup, restore drills.
  4. Observability & secrets — metrics/logs/traces and a secrets manager.
  5. Reproducibility — codify every layer in IaC so the lab is rebuildable.

Showcase — reference architecture; a build-out following this design is a separate effort.

endpointdrift

language Python status in development · 101 tests data 200-trial corpus · 358 gold labels license proprietary
  • python
  • clinical-trials
  • research-integrity
  • document-ai
  • multi-agent
  • evals
  • llmops
  • local-first
README.md Python

EndpointDrift

Catches clinical trials moving the goalposts — audits published papers against their pre-registered outcomes, with receipts.

Overview

Before a drug trial starts, researchers must publicly register exactly which outcomes will decide whether the treatment worked. When results disappoint, some papers quietly switch them — promised endpoints vanish, favorable ones appear as if planned all along. This “outcome switching” is well documented: the COMPare project (Goldacre et al.) found a majority of audited top-journal trials misreported their pre-specified outcomes. Checking takes an expert hours per trial, so almost nobody checks.

EndpointDrift automates that audit: read what the trial registered (ClinicalTrials.gov), read what the published paper reported (PubMed Central), align the two, and flag every mismatch — outcomes omitted, added, demoted, promoted, or changed. The registry half and the audit core are built today; the paper-reading half is the stage in progress. Every finding must carry verbatim evidence quotes from both documents. That grounding rule is enforced in the type system itself: a finding without its evidence spans cannot be constructed anywhere in the pipeline. No span, no finding.

Highlights

  • Grounding rule at the type level. An LLM can claim an endpoint was switched; the claim only becomes a finding if it arrives with verbatim quotes from both the registry and the paper. Enforced by a validator on the data model, not by convention.
  • Adversarial agents with a deterministic core. A Registry Advocate and a Publication Advocate are each designed to extract endpoints faithful to their one source; a deterministic Auditor compares slates and derives findings. Ambiguous matches route to a human-review queue instead of being guessed.
  • A reproducible frozen corpus. 200 completed phase-3 trials with posted results and linked PMC publications, harvested from the live ClinicalTrials.gov and NCBI APIs with per-candidate failure isolation, polite throttling, and resumable checkpoints. The committed manifest carries criteria, counts, linkage, and a sha256 per registry snapshot.
  • Expert gold labels, provenance-traceable. 38 trials hand-audited by the COMPare project converted into 358 outcome-level labels; every record cites its source sheet and original category. Ambiguous categories were excluded, not force-mapped.
  • An eval harness that refuses to flatter itself. A seeded-corruption benchmark plants known outcome switches in synthetic trial pairs and checks whether the pipeline recovers them — all six discrepancy types exercised in CI. The CI eval gate is deliberately disabled until real model inference and gold-set scoring exist: a green check that grades the system’s own synthetic fixtures would be eval theater.
  • $0/month stack. Local inference, Postgres + pgvector, self-hosted tracing. Everything develops and tests without network, GPU, or Docker.

How it works

ClinicalTrials.gov registration          PubMed Central publication
            |                                       |
            v                                       v
    Registry Advocate                     Publication Advocate
    (extracts the promised                (extracts the reported
     endpoint slate + quotes)              endpoint slate + quotes)
            |                                       |
            +------------------+--------------------+
                               v
                     Deterministic Auditor
              omitted / added / demoted / promoted /
              timepoint-changed / population-changed
                               |
                +--------------+--------------+
                v                             v
        grounded findings              human-review queue
        (quote-backed, typed)          (ambiguity, missing spans)

Tech stack

Python · LangGraph (typed agent graph) · Postgres + pgvector · httpx / stdlib transports (one of them exists because a government WAF blocks the other — documented in the code) · local VLM inference (planned: Qwen2.5-VL via vLLM/Ollama) · self-hosted Langfuse tracing · pytest + ruff, CI on every push.

Status & roadmap

In development — infrastructure and data layers built and tested. 101 tests green with no network, GPU, or Docker required. What exists today: the typed contracts and grounding rule, the deterministic audit graph, live ingest tooling (which harvested the 200-trial corpus), the COMPare gold set, and the seeded-corruption eval harness.

Not yet built — and no performance claims are made: publication-side endpoint extraction (the VLM stage), real-trial audits, and scoring against the expert gold set. No real paper has been audited yet.

Next: publication full-text ingestion and page-image indexing → VLM endpoint extraction → first real audits scored against COMPare’s expert labels, with per-type precision/recall published from tagged CI runs only.


Showcase README — this documents the design and current state; source available on request.