Compare commits

..

4 Commits

Author SHA1 Message Date
jon ceaaedd225 backup: 2026-07-11T07:57:07-05:00 2026-07-11 07:57:07 -05:00
jon 9f4e58ddf2 backup: 2026-07-11T07:55:41-05:00 2026-07-11 07:55:41 -05:00
jon 86a26233dc backup: 2026-07-11T07:54:39-05:00 2026-07-11 07:54:39 -05:00
jon 8451662d9c docs: add README
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 07:48:04 -05:00
7 changed files with 586 additions and 135 deletions
-4
View File
@@ -17,10 +17,6 @@ synapse/memory/memory.db-shm
# Ollama model blobs (~15G, regenerable via `ollama pull`)
models/
# runtime user data
data/uploads/
data/exports/
# local editor / assistant tooling
.vscode/
.claude/
+2
View File
@@ -121,6 +121,8 @@ A shared scratch log so the two Claude Code instances (this NexusOS box and the
- [2026-06-12] Fixed `management/nexus-cli.sh` `ensure_router_reachable` (~line 412): the `nmcli connection up wgs_client` WireGuard fallback dies under WSL (no NetworkManager → "permission denied"). It's now conditional on nmcli + the wgs_client connection existing. **WSL box:** you're a desktop with a direct LAN link to the server, so just make `ssh router` work (Host `router` → 192.168.50.1:9001, user jon, key `id_iWings`, in `~/.ssh/config` — which is NOT synced, set it up there) — then `ncp restore` passes the first check and never hits the VPN fallback.
- [2026-07-10] Over-engineering purge. Removed dead backend endpoints (`POST /chat`, `/memory/{id}/move[_to]`, `/playbook/run`, `/search`) + their orphaned store methods, and the sync chat wrappers. Deleted the two curses TUIs (`nexus-chat.py`, `nexus-playbook.py`, ~1280 lines) and replaced them with API-backed `ncp` subcommands (`chat`/`memory`/`playbook`/`history``management/nexus_api.py`). `controlpanel.py` kept (panel GUI). `/conversations/export` kept (ShareGPT export for future fine-tuning).
- [2026-07-10] Nuked the ML + scientific/GUI stack from THIS venv only: torch/transformers/accelerate/bitsandbytes/etc + numpy/scipy/pandas/matplotlib/PySide6 + 53 orphans total. **Promethean 16G → 108M.** Requirements files are UNCHANGED — `pip install -r requirements-amd.txt` restores the declared ML stack. Caveat: some installed extras (peft, datasets, mergekit, gguf, hf-xet) were NOT in any requirements file; if the desktop ML work needs them, add them to requirements first. Rollback freeze snapshot was taken pre-purge. **WSL box:** your venv is separate and untouched; run the same `pip uninstall` sweep if you want the space back (nothing in `synapse/` imports any of it — Ollama does inference over HTTP).
- [2026-07-11] Stood up self-hosted **Gitea** at `git.enderofwings.com` (runs on the router in chroot `/tmp/mnt/GitHead`, Gitea 1.26.4, `DISABLE_SSH=true` so pushes are HTTPS+token). Applied full Nexus/GitNexus branding (custom dir `/var/lib/gitea/custom`). Pushed nexus-core → `enderofwings/NexusOS`.
- [2026-07-11] **Backup/restore rewritten to use git→Gitea instead of rsync to `router:/tmp/mnt/Wingdrive2/nexus-core`.** `bin/backup.sh` now: dump `memory.db``synapse/memory/memory.db.sql` (diff-friendly, so the assistant's memory+history is versioned), `git add -A && commit "backup: <ts>" && push`. `bin/restore.sh` now: clone-or-`git pull`, rebuild the DB from the dump (only if no live DB), then rebuild venv/npm/theme/terminal/panel. `-c|--claude` dry-run kept (git status / incoming diff). Heavies (Promethean, ollama/, models/, node_modules, dist, runtime) still excluded and rebuilt on restore. Auth via `git config credential.helper store` (`~/.git-credentials`, 0600, Gitea PAT). The old `Wingdrive2/nexus-core` rsync folder is being retired — verify a `restore` on a scratch dir before deleting it. **WSL box:** switch your restore path to `git clone/pull https://git.enderofwings.com/enderofwings/NexusOS.git` (make a Gitea account/token; `ssh router` no longer required for backup/restore). Old `ncp restore` rsync path is dead.
### WSL box — Windows 11 · WSL2 Ubuntu · DESKTOP-WINGX
- [2026-06-11] `install-windows.ps1` had a PowerShell syntax error at line 73 (`Unexpected token '}'`); fixed brace mismatch in the script.
+110
View File
@@ -0,0 +1,110 @@
<div align="center">
<img src="assets/gitnexus-logo.svg" alt="NexusOS" width="96">
# NexusOS
**A local-first AI assistant platform.** Runs entirely on your machine — a
Python/FastAPI backend, a bundled Ollama instance for inference, a persistent
memory service, and a React frontend. No external AI provider is called.
</div>
---
## What it is
NexusOS ("Nexus") is a self-hosted assistant you actually own. All inference
runs through a **locally bundled Ollama** on `localhost`; conversations, facts,
and settings live in local SQLite. It ships with desktop branding (XFCE theme,
icons, boot splash) so it can be run as a full assistant environment, not just
a web app.
- **Chat** — streaming responses from local Ollama models (SSE).
- **Persistent memory** — a dedicated service auto-extracts durable facts from
each exchange and layers them into future prompts.
- **Playbooks** — ordered YAML system-prompt records; the first is the active
persona, the rest are injected as reference context.
- **Model management** — list, pull, and delete Ollama models from the UI/CLI.
- **History** — full conversation persistence, search, edit, export.
## Quick start
Requires Python 3, Node, and a GPU is optional (CPU works).
```bash
# 1. Install deps into the Promethean venv (auto-selects AMD/NVIDIA/CPU)
./bin/install.sh
# 2. Launch the full stack (memory :8001, backend :8000, frontend :5173)
./launch_nexus.sh
```
Python deps are layered: `requirements-base.txt` (GPU-agnostic core) plus one
overlay — `requirements-amd.txt` (ROCm), `requirements-nvidia.txt` (CUDA), or
`requirements-wsl.txt` (CPU-only). `bin/install.sh` picks the right one for the
host.
### Individual services
```bash
source Promethean/bin/activate
uvicorn synapse.main:sio_app --host 0.0.0.0 --port 8000 --reload # backend
uvicorn synapse.memory.service:app --host 0.0.0.0 --port 8001 --reload # memory
cd interface/web && npm run dev # frontend
```
## CLI (`ncp`)
Start/stop services and drive the same features as the web UI over the REST API:
```bash
./management/nexus-cli.sh start # backend + frontend (--backend|--frontend|--memory)
./management/nexus-cli.sh stop
ncp chat "<message>" # stream a reply
ncp memory list | add <text> | rm <id>
ncp playbook list | show <id> # first playbook (*) = active system prompt
ncp history [query] # recent conversations
```
## Architecture
| Component | Location | Role |
|---|---|---|
| **Synapse** (backend) | `synapse/` | FastAPI app. `/chat/stream`, `/playbooks`, `/memory`, `/models`, `/conversations`, `/settings`, `/ollama`, `/icons`. Assembles the system prompt: active playbook → reference playbooks → memory facts → relevant past snippets. |
| **Memory service** | `synapse/memory/` | Separate FastAPI app (:8001). `/memories/extract` uses an Ollama prompt to decide what to persist. Shares the SQLite DB with the backend. |
| **Playbooks** | `synapse/playbooks/` + `data/playbooks/` | Ordered `{id}.yaml` records managed by `PlaybookManager`. |
| **Ollama** | `ollama/bin/ollama` | Bundled binary; `OllamaManager` handles lifecycle + model selection (Vulkan GPU detection). HTTP API at `127.0.0.1:11434`. |
| **Frontend** | `interface/web/` | React 19 + Vite. Pages: Chat, Playbooks, History, Models, Memory, Settings. |
### Storage
Most data lives in `synapse/memory/memory.db` (SQLite, WAL) — facts,
conversations, messages, settings. Playbooks are the exception (YAML files in
`data/playbooks/`). All paths are defined in `synapse/nexus_config.py`.
## Layout
```
synapse/ FastAPI backend + memory service + playbook/ollama managers
interface/web/ React + Vite frontend
management/ nexus-cli.sh, ncp API client, control panel, desktop theme
bin/ install, backup/restore, panel + provisioning scripts
assets/ branding: icons, boot splash, XFCE/GTK theme
data/playbooks/ active playbook YAML
```
## Configuration
| Concern | Location |
|---|---|
| Ollama host | `OLLAMA_HOST` env (default `http://127.0.0.1:11434`) |
| Filesystem paths | `synapse/nexus_config.py` |
| Frontend API base URL | `interface/web/src/config.js` |
| Python deps | `requirements-base.txt` + amd/nvidia/wsl overlay |
---
<div align="center"><sub>NexusOS · local AI, self-hosted on <a href="https://git.enderofwings.com/enderofwings/NexusOS">GitNexus</a></sub></div>
+67 -63
View File
@@ -1,50 +1,41 @@
#!/bin/bash
# Backup NexusOS to the router. All modes rsync into the canonical nexus-core
# folder over the normal SSH daemon (host: router, port 9001) — the SAME folder
# restore.sh pulls from, so backup and restore stay in sync.
# Backup NexusOS via git → Gitea (enderofwings/NexusOS). Replaces the old
# rsync-to-router backup. Commits the tracked tree (source, config, playbooks,
# branding, docs) plus a SQL dump of the memory DB, then pushes to the remote —
# so a fresh clone + restore.sh fully rebuilds the machine.
#
# backup.sh Fast incremental backup.
# backup.sh full Same, but first snapshots the live desktop wiring
# and retains runtime/ (minus logs) for a fuller copy.
# backup.sh -c | --claude Dry-run: show what a backup WOULD push (content-
# level), change nothing. For deciding whether an
# actual backup is needed.
# backup.sh Commit changed files + push.
# backup.sh full | -f Also snapshot live desktop wiring + Claude notes first.
# backup.sh -c | --claude Dry-run: show what a backup WOULD commit/push. No changes.
#
# The regenerable heavies (Promethean venv, ollama binary, models/, node_modules,
# dist, runtime logs) are intentionally NOT backed up — restore.sh rebuilds them
# from requirements-*.txt / package-lock.json / `ollama pull`.
NEXUS_ROOT="$HOME/nexus-core"
DEST="router:/tmp/mnt/Wingdrive2/nexus-core/"
mode="$1"
cd "$NEXUS_ROOT" || { echo "No $NEXUS_ROOT"; exit 1; }
mode="${1:-}"
# Standard (incremental) excludes — shared by the default backup AND the
# -c/--claude check, so the dry-run accurately previews a real backup.
EXCLUDES=(
--exclude='.git/'
--exclude='Promethean/'
--exclude='models/blobs/'
--exclude='ollama/'
--exclude='interface/web/node_modules/'
--exclude='interface/web/dist/'
--exclude='runtime/'
--exclude='__pycache__/'
--exclude='*.pyc'
)
DB="synapse/memory/memory.db"
DB_SQL="synapse/memory/memory.db.sql"
# Check mode (-c|--claude|check): content-level dry-run, no transfer. Lists what
# a standard backup would push to the router (and delete there). --checksum
# compares by content (not size/mtime), so only genuine differences show.
if [ "$mode" = "-c" ] || [ "$mode" = "--claude" ] || [ "$mode" = "check" ]; then
echo "Checking backup diff vs router (dry-run, content-level, no changes)..."
rsync -rlni --checksum --delete "${EXCLUDES[@]}" -e ssh "$NEXUS_ROOT/" "$DEST"
echo ""
echo "(dry-run only — nothing pushed. Apply with: backup or backup full)"
exit 0
# Dump the (gitignored, binary, WAL) memory DB to a diff-friendly SQL file so git
# backs up the assistant's memory + conversation history. sqlite3 reads through
# the WAL, so the dump includes the latest committed data without a checkpoint.
dump_db() {
[ -f "$DB" ] || return 0
if sqlite3 "$DB" ".dump" > "$DB_SQL" 2>/dev/null; then
echo "Dumped memory DB → $DB_SQL"
else
echo "Warning: could not dump $DB (sqlite3 missing?) — DB not captured"
fi
}
if [ "$mode" = "full" ] || [ "$mode" = "-f" ]; then
# Snapshot the live NexusOS desktop wiring into the repo before backing up,
# so the backup captures the ACTUAL current state (not just the canonical
# defaults baked into assets/themes/install-theme.sh). This is a reference
# copy for recovery/diffing; install-theme.sh remains the applier on restore.
# full mode: snapshot the LIVE desktop wiring into the repo (reference copy for
# recovery/diffing; install-theme.sh remains the applier on restore), and copy
# the Claude memory notes in so git commits their content.
snapshot_desktop() {
snap="$NEXUS_ROOT/assets/themes/restore-snapshot"
mkdir -p "$snap"
{
@@ -61,33 +52,46 @@ if [ "$mode" = "full" ] || [ "$mode" = "-f" ]; then
cp -f "$HOME/.config/xfce4/panel/genmon-13.rc" \
"$NEXUS_ROOT/management/panel/genmon-13.rc" 2>/dev/null || true
# Symlink the Claude memory notes (machine knowledge that lives outside the
# repo under ~/.claude) into assets/notes/ so the backup below captures them.
# rsync runs with -L (copy-links), so the links resolve to real content on
# the backup target. Relative links + a loop so new notes are picked up too.
# Some of these notes aren't Nexus-specific (Fusion 360, distro/Wine).
# Copy the Claude memory notes (machine knowledge under ~/.claude) into the
# repo so they're versioned too. Some aren't Nexus-specific (Fusion 360, etc.).
notes_src="$HOME/.claude/projects/-home-jon-nexus-core/memory"
notes_dst="$NEXUS_ROOT/assets/notes"
mkdir -p "$notes_dst"
for f in "$notes_src"/*.md; do
[ -e "$f" ] || continue
ln -sfr "$f" "$notes_dst/$(basename "$f")"
done
cp -f "$notes_src"/*.md "$notes_dst/" 2>/dev/null || true
}
rsync -avzL --no-owner --no-group --delete \
--exclude='.git/' \
--exclude='Promethean/' \
--exclude='models/blobs/' \
--exclude='ollama/' \
--exclude='interface/web/node_modules/' \
--exclude='interface/web/dist/' \
--exclude='runtime/logs/' \
--exclude='runtime/backend.log' \
--exclude='runtime/frontend.log' \
--exclude='__pycache__/' \
--exclude='*.pyc' \
-e ssh \
"$NEXUS_ROOT/" "$DEST"
else
rsync -avz --no-owner --no-group --delete "${EXCLUDES[@]}" -e ssh "$NEXUS_ROOT/" "$DEST"
# Check mode: dry-run. Show what a real backup would commit and push.
if [ "$mode" = "-c" ] || [ "$mode" = "--claude" ] || [ "$mode" = "check" ]; then
dump_db
echo ""
echo "Files a backup would commit:"
git status --short
echo ""
echo "Local commits not yet pushed:"
git log --oneline @{u}.. 2>/dev/null || echo "(none / no upstream)"
echo ""
echo "(dry-run only — nothing committed or pushed. Apply with: backup)"
exit 0
fi
if [ "$mode" = "full" ] || [ "$mode" = "-f" ]; then
snapshot_desktop
fi
dump_db
git add -A
if git diff --cached --quiet; then
echo "No changes to commit."
else
git commit -q -m "backup: $(date -Is)"
echo "Committed backup snapshot."
fi
echo "Pushing to Gitea..."
if git push origin main; then
echo "Backup complete."
else
echo "Push failed. Set up credentials once with:"
echo " git config credential.helper store # then push once and enter your Gitea token"
exit 1
fi
+46 -29
View File
@@ -1,22 +1,18 @@
#!/bin/bash
NEXUS_ROOT="$HOME/nexus-core"
SRC="router:/tmp/mnt/Wingdrive2/nexus-core/"
mode="$1"
# Restore NexusOS from Gitea (enderofwings/NexusOS). Clones or pulls the repo,
# rebuilds the memory DB from its SQL dump, then rebuilds the regenerable env
# (venv, frontend deps) and desktop integration (theme, terminal, panel).
#
# restore.sh Pull latest + rebuild.
# restore.sh -c | --claude Dry-run: fetch + show what a restore WOULD apply. No changes.
# Shared exclude list — used by the real restore AND the -c/--claude dry-run, so
# the check accurately previews what a real restore would do.
EXCLUDES=(
--exclude='.git/'
--exclude='Promethean/'
--exclude='models/blobs/'
--exclude='ollama/'
--exclude='interface/web/node_modules/'
--exclude='interface/web/dist/'
--exclude='runtime/'
--exclude='__pycache__/'
--exclude='*.pyc'
)
NEXUS_ROOT="$HOME/nexus-core"
REPO="https://git.enderofwings.com/enderofwings/NexusOS.git"
mode="${1:-}"
DB="$NEXUS_ROOT/synapse/memory/memory.db"
DB_SQL="$NEXUS_ROOT/synapse/memory/memory.db.sql"
detect_requirements() {
if command -v nvidia-smi &>/dev/null || lspci 2>/dev/null | grep -qi nvidia; then
@@ -26,30 +22,49 @@ detect_requirements() {
fi
}
# Check mode (-c|--claude|check): content-level dry-run. Lists exactly what a
# real restore would pull from the router and delete locally — nothing changes
# and the env/theme rebuild is skipped. --checksum compares by content (not
# size/mtime), so only genuine differences show. Meant for deciding whether a
# real restore is actually needed before paying for the rebuild.
# Fresh machine: clone. Existing: work in place.
if [ ! -d "$NEXUS_ROOT/.git" ]; then
echo "No local repo — cloning from Gitea..."
git clone "$REPO" "$NEXUS_ROOT" || { echo "Clone failed."; exit 1; }
fi
cd "$NEXUS_ROOT" || { echo "No $NEXUS_ROOT"; exit 1; }
# Check mode: dry-run. Fetch and show what a real restore would pull, no changes.
if [ "$mode" = "-c" ] || [ "$mode" = "--claude" ] || [ "$mode" = "check" ]; then
echo "Checking restore diff vs router (dry-run, content-level, no changes)..."
rsync -rlni --checksum --delete "${EXCLUDES[@]}" -e ssh "$SRC" "$NEXUS_ROOT/"
echo "Fetching to preview restore (no changes)..."
git fetch origin
echo ""
echo "(dry-run only — nothing changed. Apply with: restore -f)"
echo "Commits a restore would apply:"
git log --oneline ..origin/main 2>/dev/null || echo "(up to date)"
echo ""
git diff --stat ..origin/main 2>/dev/null
echo ""
echo "(dry-run only — nothing changed. Apply with: restore)"
exit 0
fi
if [ "$mode" = "full" ] || [ "$mode" = "-f" ]; then
echo "Restoring full Nexus backup from router..."
else
echo "Restoring Nexus from router..."
echo "Pulling latest from Gitea..."
git pull --ff-only origin main || { echo "Pull failed (diverged? stash/commit local changes)."; exit 1; }
# Rebuild the memory DB from its SQL dump — only if there's no live DB, so we
# never clobber a machine that has newer conversations than the backup.
if [ -f "$DB_SQL" ] && [ ! -f "$DB" ]; then
echo ""
echo "Rebuilding memory DB from dump..."
sqlite3 "$DB" < "$DB_SQL" && echo "Memory DB restored." \
|| echo "Warning: memory DB rebuild failed."
elif [ -f "$DB" ]; then
echo "Live memory DB present — left as-is (dump not applied). Delete it first to restore from backup."
fi
rsync -avz --no-owner --no-group --delete "${EXCLUDES[@]}" -e ssh "$SRC" "$NEXUS_ROOT/"
echo ""
echo "Rebuilding Python environment..."
if [ ! -x "$NEXUS_ROOT/Promethean/bin/pip" ]; then
python3 -m venv "$NEXUS_ROOT/Promethean"
fi
req=$(detect_requirements)
if [ -f "$NEXUS_ROOT/$req" ]; then
"$NEXUS_ROOT/Promethean/bin/pip" install --upgrade pip -q
"$NEXUS_ROOT/Promethean/bin/pip" install -r "$NEXUS_ROOT/$req"
else
echo "Warning: $req not found — skipping pip install."
@@ -58,6 +73,7 @@ fi
echo ""
echo "Rebuilding frontend dependencies..."
cd "$NEXUS_ROOT/interface/web" && npm install
cd "$NEXUS_ROOT"
echo ""
echo "Restoring NexusOS desktop theme..."
@@ -90,3 +106,4 @@ fi
echo ""
echo "Restore complete. Nexus is ready to start."
echo "Note: Ollama models are not in the backup — pull them with \`ollama pull <model>\` as needed."
+8 -38
View File
@@ -425,9 +425,9 @@ show_help() {
echo " available Show models available to install"
echo " install <name> Pull a model into Nexus"
echo
echo " backup Backup vital Nexus files to router (smart/incremental)"
echo " backup -f, full Full backup of all project files"
echo " restore Restore vital Nexus files from router backup"
echo " backup Backup Nexus to Gitea (git commit + push)"
echo " backup -f, full Backup + snapshot live desktop wiring/notes"
echo " restore Restore Nexus from Gitea (git pull + rebuild)"
echo " restore -f, full Full restore of all project files"
echo
echo " help, -h Show this help message"
@@ -471,52 +471,22 @@ update_nexus() {
# BACKUP / RESTORE COMMANDS
# -----------------------------
ensure_router_reachable() {
echo "Checking router connectivity..."
if ssh -q -o BatchMode=yes -o ConnectTimeout=5 router exit 2>/dev/null; then
echo "Router is reachable."
return 0
fi
# Fallback: bring up the WireGuard tunnel — but only on machines that
# actually have NetworkManager + the wgs_client connection (e.g. the roaming
# NexusOS laptop). On a directly-wired desktop or under WSL, nmcli/NM isn't
# usable, so skip the VPN step instead of dying with "permission denied".
if command -v nmcli >/dev/null 2>&1 && \
nmcli -t -f NAME connection show 2>/dev/null | grep -qx wgs_client; then
echo "Router unreachable. Activating WireGuard (wgs_client)..."
if nmcli connection up wgs_client; then
echo "Waiting for VPN to establish..."
sleep 5
if ssh -q -o BatchMode=yes -o ConnectTimeout=10 router exit 2>/dev/null; then
echo "Router reachable via VPN."
return 0
fi
fi
echo "Router still unreachable after VPN activation."
else
echo "Router unreachable, and no WireGuard fallback on this machine."
echo "Expected a direct LAN link to the server — check the 'router' SSH host and the connection."
fi
return 1
}
# Backup/restore go through git → Gitea now (see bin/backup.sh, bin/restore.sh),
# not rsync-to-router, so no SSH/WireGuard reachability gate is needed — git
# handles its own connectivity and errors over HTTPS.
backup_nexus() {
ensure_router_reachable || return 1
echo "Backing up Nexus to router..."
bash "$NEXUS_ROOT/bin/backup.sh"
}
backup_nexus_full() {
ensure_router_reachable || return 1
echo "Full backup of Nexus to router..."
bash "$NEXUS_ROOT/bin/backup.sh" full
}
restore_nexus() {
local mode="$1"
ensure_router_reachable || return 1
echo "WARNING: This will overwrite local files with the remote backup."
echo "This pulls the latest backup from Gitea and rebuilds the environment"
echo "(venv, npm, theme, panel). Local commits must be pushed or stashed first."
printf "Continue? [y/N] "
read -r confirm
if [[ "$confirm" != "y" && "$confirm" != "Y" ]]; then
File diff suppressed because one or more lines are too long