feat(packaging): move the CLI into nexusos_cli and make the wheel self-sufficient
The CLI shipped from `management/`, which also holds desktop-only pieces (the Tk control panel, the XFCE panel wiring, the shell wrappers). Packaging that directory meant the wheel either dragged in tkinter or shipped a broken import. Split it: `nexusos_cli/` is what the wheel ships and what `nexus`/`ncp`/ `nexusos` dispatch to, `management/` keeps the desktop half. Alongside the move: * hatch_build.py decides the interface/web/dist include at build time. dist/ is gitignored, so a static force-include aborts `pip install -e .` on a fresh clone - before the reader reaches the `npm run build` step. Editable installs now skip a missing dist; wheels and sdists hard-error naming the command to run. * synapse/proc_util.py gives frontend_manager and ncp process inspection and termination without psutil, which became an optional extra when the wheel landed. It routes around Windows having no signals, where os.kill(pid, 15) is an unblockable TerminateProcess rather than a polite request. * nexusos_cli/monitor.py adds `ncp monitor`, an ASCII dashboard with no curses or rich dependency so it works in Termux, plain SSH and Windows Terminal. Collector and renderer are separate so tests feed fixtures, no stack needed. * tests/test_packaging_deps.py fails the gate when synapse or nexusos_cli import a distribution pyproject does not declare, and when an optional dependency is imported at module scope instead of lazily. * bin/check.sh now builds the wheel, twine-checks it, and asserts the compiled UI and seed playbooks are actually inside it. A wheel that builds but ships no dist/ serves a blank page, which only shows up after release. tests/test_nexus_api.py moves to tests/ with the module it covers. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Athena Kaminsky
co-authored by
Claude Opus 5
Claude Opus 5
parent
d579502a5b
commit
9104c724c4
@@ -37,7 +37,7 @@ jobs:
|
||||
run: |
|
||||
python -m pip install -e ".[dev]"
|
||||
python -m pytest -q tests management
|
||||
bash -n scripts/install-termux.sh
|
||||
for f in scripts/install-termux.sh bin/check.sh management/nexus-cli.sh; do bash -n "$f"; done
|
||||
|
||||
- name: Build wheel and sdist
|
||||
run: |
|
||||
@@ -54,6 +54,12 @@ jobs:
|
||||
"$RUNNER_TEMP/nexus-wheel/bin/nexus" init --json
|
||||
"$RUNNER_TEMP/nexus-wheel/bin/nexus" doctor --json
|
||||
"$RUNNER_TEMP/nexus-wheel/bin/python" -c "from synapse.main import sio_app; assert sio_app"
|
||||
# The wheel must carry the compiled UI, not just import cleanly.
|
||||
"$RUNNER_TEMP/nexus-wheel/bin/python" - <<'PY'
|
||||
from synapse.nexus_config import settings
|
||||
index = settings.web_dist_dir / "index.html"
|
||||
assert index.is_file(), f"wheel shipped no web UI at {index}"
|
||||
PY
|
||||
|
||||
- uses: actions/upload-artifact@v4
|
||||
with:
|
||||
|
||||
@@ -57,7 +57,7 @@ access to the same features as the web UI (all via the REST API on `:8000`):
|
||||
./management/nexus-cli.sh stop
|
||||
./management/nexus-cli.sh start --backend|-b / --frontend|-f / --memory|-m
|
||||
|
||||
# Feature commands (dispatch to management/nexus_api.py — httpx, no TUI):
|
||||
# Feature commands (dispatch to nexusos_cli/nexus_api.py — httpx, no TUI):
|
||||
ncp chat "<message>" # stream a reply (POST /chat/stream)
|
||||
ncp memory list|add <text>|rm <id>
|
||||
ncp playbook list|show <id> # first playbook (*) is the active system prompt
|
||||
@@ -66,12 +66,15 @@ ncp history [query] # recent conversations
|
||||
The old curses TUIs (`nexus-chat.py`, `nexus-playbook.py`) were removed in favor of
|
||||
these API-backed subcommands. The CLI covers chat, memory, playbooks, and history;
|
||||
the web UI and control panel expose the remaining management features.
|
||||
The CLI itself lives in `nexusos_cli/` (that is what the wheel ships and what
|
||||
`nexus`/`ncp`/`nexusos` dispatch to); `management/` keeps the desktop-only
|
||||
pieces — the shell wrappers, the Tk control panel, and the XFCE panel wiring.
|
||||
`management/controlpanel.py` (tkinter GUI, wired into the XFCE panel via
|
||||
`bin/panel/nexus-popup.py`) stays.
|
||||
|
||||
**Checks (the release gate):**
|
||||
```bash
|
||||
./bin/check.sh # pytest (tests/ + management/) + eslint + .ps1 parse check
|
||||
./bin/check.sh # pytest + eslint + frontend tests + .ps1/.sh parse + wheel build
|
||||
```
|
||||
There is no hosted CI — the remote is self-hosted Gitea with no act_runner — so
|
||||
this script *is* the gate. Run it before tagging a release.
|
||||
|
||||
@@ -98,6 +98,7 @@ The portable package installs `nexus`, `ncp`, and `nexusos` as equivalent
|
||||
commands. From a checkout today:
|
||||
|
||||
```bash
|
||||
cd interface/web && npm ci && npm run build && cd ../.. # compile the UI
|
||||
python -m pip install -e ".[standard]"
|
||||
nexus init
|
||||
nexus doctor
|
||||
@@ -260,7 +261,8 @@ are the exception (YAML files in `data/playbooks/`). All paths are defined in
|
||||
- `synapse/` — FastAPI backend + memory curator + playbook/ollama managers
|
||||
- `modules/` — auto-discovered feature plugins
|
||||
- `interface/web/` — React + Vite frontend
|
||||
- `management/` — nexus-cli.sh, ncp API client, control panel, desktop theme
|
||||
- `nexusos_cli/` — the portable CLI the wheel ships (`nexus`/`ncp`/`nexusos`)
|
||||
- `management/` — nexus-cli.sh wrapper, control panel, desktop theme
|
||||
- `bin/` — install, backup/restore, panel + provisioning scripts
|
||||
- `assets/` — branding: icons, boot splash, XFCE/GTK theme
|
||||
- `data/playbooks/` — active playbook YAML
|
||||
|
||||
@@ -39,5 +39,39 @@ else
|
||||
echo "-- skipped: pwsh not installed"
|
||||
fi
|
||||
|
||||
echo "== shell parse =="
|
||||
for f in scripts/install-termux.sh launch_nexus.sh management/nexus-cli.sh; do
|
||||
[ -f "$f" ] && { bash -n "$f" || fail=1; }
|
||||
done
|
||||
|
||||
echo "== packaging =="
|
||||
# The wheel is the other shippable artifact, so it belongs in the same gate:
|
||||
# a broken pyproject or a missing web build only shows up at build time.
|
||||
if Promethean/bin/python -c "import build, twine" 2>/dev/null; then
|
||||
rm -rf .build-check
|
||||
if Promethean/bin/python -m build --outdir .build-check >/dev/null 2>&1; then
|
||||
Promethean/bin/python -m twine check .build-check/* || fail=1
|
||||
# The compiled UI has to actually be inside the wheel - a wheel that
|
||||
# builds but ships no dist/ serves a blank page.
|
||||
Promethean/bin/python - <<'PY' || fail=1
|
||||
import glob, sys, zipfile
|
||||
wheels = glob.glob(".build-check/*.whl")
|
||||
if not wheels:
|
||||
sys.exit("no wheel produced")
|
||||
names = zipfile.ZipFile(wheels[0]).namelist()
|
||||
if not any(n.startswith("synapse/_resources/web/") for n in names):
|
||||
sys.exit("wheel is missing the compiled web UI (cd interface/web && npm run build)")
|
||||
if not any(n.startswith("synapse/_resources/playbooks/") for n in names):
|
||||
sys.exit("wheel is missing the seed playbooks")
|
||||
print(f"wheel OK: {len(names)} files")
|
||||
PY
|
||||
else
|
||||
echo "!! wheel build failed"; fail=1
|
||||
fi
|
||||
rm -rf .build-check
|
||||
else
|
||||
echo "-- skipped: build/twine missing (pip install -e '.[dev]')"
|
||||
fi
|
||||
|
||||
[ "$fail" -eq 0 ] && echo "OK" || echo "FAILED"
|
||||
exit "$fail"
|
||||
|
||||
+21
-2
@@ -8,9 +8,11 @@ checkout-specific operations report a clear error when invoked from a wheel.
|
||||
|
||||
## Install
|
||||
|
||||
From a source checkout:
|
||||
From a source checkout. Build the web UI first - it is a Vite artifact, so a
|
||||
fresh clone does not have it, and an install without it serves the API only:
|
||||
|
||||
```bash
|
||||
cd interface/web && npm ci && npm run build && cd ../..
|
||||
python -m pip install -e ".[standard]"
|
||||
nexus init
|
||||
nexus doctor
|
||||
@@ -27,12 +29,15 @@ nexus serve
|
||||
The base install contains the backend, memory service, compiled web UI, CLI,
|
||||
and seed playbooks. Extras keep platform-sensitive dependencies optional:
|
||||
|
||||
- `standard`: documents, vector search, and process control
|
||||
- `standard`: documents, vector search, web search, and process control
|
||||
- `documents`: PDF and DOCX ingestion
|
||||
- `vector`: sqlite-vec semantic indexes
|
||||
- `voice`: local faster-whisper transcription
|
||||
- `process`: psutil-backed process and port inspection
|
||||
- `desktop`: desktop process support and Windows pywebview
|
||||
- `search`: DuckDuckGo web search for chat
|
||||
- `mail`: IMAP mail reading
|
||||
- `all`: every optional capability at once
|
||||
|
||||
## Common commands
|
||||
|
||||
@@ -41,6 +46,7 @@ nexus init Create writable state and seed playbooks
|
||||
nexus doctor [--fix] [--json] Diagnose the install and provider
|
||||
nexus paths [--json] Show package, state, and asset locations
|
||||
nexus status [--json] Show services and provider reachability
|
||||
nexus monitor [--once] [--json] ASCII live dashboard (services, resources, tools)
|
||||
nexus serve Run backend + memory in the foreground
|
||||
nexus start|stop|restart Manage background services
|
||||
nexus open Open the compiled web interface
|
||||
@@ -84,6 +90,11 @@ directory. Inspect the exact locations with `nexus paths`.
|
||||
|
||||
Environment variables override persisted settings. The most useful are:
|
||||
|
||||
Persisted keys are the same names, minus the `NEXUS_` prefix - `nexus config
|
||||
set home /data/nexus` matches `NEXUS_HOME`. `nexus config list` shows what is
|
||||
set; `nexus config set` warns when a change would point NexusOS at a database
|
||||
that does not exist yet (the file is never moved for you).
|
||||
|
||||
```text
|
||||
NEXUS_HOME Override the complete writable state root
|
||||
NEXUS_CONFIG_DIR Override the config directory
|
||||
@@ -97,3 +108,11 @@ NEXUS_MEMORY_PORT Memory service port (default 8001)
|
||||
The REST APIs are unauthenticated. `nexus serve` refuses non-loopback binds
|
||||
unless `--allow-lan` is given; that flag is an explicit acknowledgement, not
|
||||
an authentication layer.
|
||||
|
||||
`--allow-lan` widens the accepted `Host` headers and CORS origins to the
|
||||
addresses the bind actually answers on - it does **not** set them to `*`.
|
||||
That keeps `TrustedHostMiddleware` enforcing something, which is what stops a
|
||||
web page you visit from resolving a name it controls to your machine and
|
||||
driving the API through your browser. Export `NEXUS_ALLOWED_HOSTS` yourself if
|
||||
you genuinely need a blanket, and understand that anyone who can reach the
|
||||
port has full admin and data access.
|
||||
|
||||
@@ -0,0 +1,55 @@
|
||||
"""Build hook that ships the compiled web UI without breaking `pip install -e .`.
|
||||
|
||||
interface/web/dist is gitignored - it is a Vite build artifact, not source - so
|
||||
a fresh clone does not have it. A static force-include of a missing path aborts
|
||||
the build, which would make the README's first step ("pip install -e .") fail
|
||||
before the reader ever gets to `npm run build`.
|
||||
|
||||
So the include is decided here instead:
|
||||
* editable install -> skip a missing dist, the UI just isn't served yet
|
||||
* wheel / sdist -> hard error naming the exact command to run
|
||||
|
||||
Set NEXUS_ALLOW_UILESS_BUILD=1 to build a deliberately headless distribution
|
||||
(API and CLI only).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
from hatchling.builders.hooks.plugin.interface import BuildHookInterface
|
||||
|
||||
_UI_SOURCE = Path("interface") / "web" / "dist"
|
||||
_UI_TARGETS = {
|
||||
"wheel": "synapse/_resources/web",
|
||||
"sdist": "interface/web/dist",
|
||||
}
|
||||
|
||||
|
||||
class NexusBuildHook(BuildHookInterface):
|
||||
PLUGIN_NAME = "custom"
|
||||
|
||||
def initialize(self, version: str, build_data: dict) -> None:
|
||||
target = _UI_TARGETS.get(self.target_name)
|
||||
if target is None:
|
||||
return
|
||||
|
||||
source = Path(self.root) / _UI_SOURCE
|
||||
if (source / "index.html").is_file():
|
||||
build_data.setdefault("force_include", {})[str(source)] = target
|
||||
return
|
||||
|
||||
if version == "editable" or os.getenv("NEXUS_ALLOW_UILESS_BUILD") == "1":
|
||||
self.app.display_warning(
|
||||
f"No compiled web UI at {_UI_SOURCE} - the backend will serve the "
|
||||
"API only. Build it with: cd interface/web && npm ci && npm run build"
|
||||
)
|
||||
return
|
||||
|
||||
raise RuntimeError(
|
||||
f"Cannot build a {self.target_name}: the compiled web UI is missing from "
|
||||
f"{_UI_SOURCE}.\n"
|
||||
"Build it first:\n"
|
||||
" cd interface/web && npm ci && npm run build\n"
|
||||
"Or set NEXUS_ALLOW_UILESS_BUILD=1 to ship an API/CLI-only distribution."
|
||||
)
|
||||
@@ -1 +1,5 @@
|
||||
"""NexusOS command-line and desktop management helpers."""
|
||||
"""Desktop-only NexusOS management helpers (Tk control panel, XFCE panel).
|
||||
|
||||
The portable CLI lives in the `nexusos_cli` package - that is what the
|
||||
wheel ships and what `ncp`/`nexus`/`nexusos` dispatch to.
|
||||
"""
|
||||
|
||||
+1
-1
@@ -25,7 +25,7 @@ REM delegate to the stdlib-only bin/sync.py. Mirrors the same fallback in ncp.ps
|
||||
set "PY=%ROOT%\Promethean\Scripts\python.exe"
|
||||
if not exist "%PY%" set "PY=python"
|
||||
pushd "%ROOT%"
|
||||
"%PY%" -m management.cli %*
|
||||
"%PY%" -m nexusos_cli.cli %*
|
||||
set "NEXUS_EXIT=%ERRORLEVEL%"
|
||||
popd
|
||||
exit /b %NEXUS_EXIT%
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
#!/usr/bin/env bash
|
||||
# ncp - Linux entry point. The CLI itself is management/cli.py, which runs
|
||||
# ncp - Linux entry point. The CLI itself is nexusos_cli/cli.py, which runs
|
||||
# unchanged on Windows too (see management/ncp.cmd); this stays a shell script
|
||||
# because ~/.bashrc, launch_nexus.sh, bin/restore-linux.sh, controlpanel.py,
|
||||
# bin/panel/nexus-popup.py and management/nexus-app.sh all invoke this path.
|
||||
@@ -14,4 +14,4 @@ PY="$NEXUS_ROOT/Promethean/bin/python3"
|
||||
[ -x "$PY" ] || PY=python3
|
||||
|
||||
cd "$NEXUS_ROOT"
|
||||
exec "$PY" -m management.cli "$@"
|
||||
exec "$PY" -m nexusos_cli.cli "$@"
|
||||
|
||||
@@ -0,0 +1,7 @@
|
||||
"""The portable NexusOS command line, shipped in the wheel.
|
||||
|
||||
Kept out of `management/` so the installed distribution does not claim a
|
||||
top-level `management` package name in site-packages. `management/` stays in
|
||||
the source checkout for the desktop-only pieces - the Tk control panel, the
|
||||
shell wrappers, the XFCE panel and .desktop wiring.
|
||||
"""
|
||||
@@ -22,6 +22,7 @@ from . import ncp as services
|
||||
|
||||
|
||||
CONFIG_SCHEMA = {
|
||||
"home": "path",
|
||||
"api_url": "url",
|
||||
"memory_url": "url",
|
||||
"bind_host": "text",
|
||||
@@ -136,6 +137,46 @@ def cmd_paths(args) -> int:
|
||||
return 0
|
||||
|
||||
|
||||
# Keys whose value decides where the SQLite database is looked up. Changing one
|
||||
# does not move the file, so the next start would quietly open a fresh, empty
|
||||
# database and the user's conversations and memories would look deleted.
|
||||
_DB_LOCATION_KEYS = ("memory_db", "memory_dir", "data_dir", "home")
|
||||
|
||||
|
||||
def _relocation_warning(key: str, value) -> str | None:
|
||||
"""Warn when a config change points the database somewhere with no data."""
|
||||
if key not in _DB_LOCATION_KEYS:
|
||||
return None
|
||||
current = settings.memory_db
|
||||
if not current.is_file():
|
||||
return None
|
||||
if key == "memory_db":
|
||||
new_db = Path(str(value))
|
||||
elif key == "memory_dir":
|
||||
new_db = Path(str(value)) / current.name
|
||||
else:
|
||||
# data_dir/home only decide the DB location when nothing more specific
|
||||
# does, and only in the layout where the DB actually sits under them -
|
||||
# a source checkout keeps it in synapse/memory/ regardless.
|
||||
existing = config.read_user_config()
|
||||
if "memory_dir" in existing or "memory_db" in existing:
|
||||
return None
|
||||
anchor = settings.data_dir if key == "data_dir" else settings.state_dir
|
||||
try:
|
||||
tail = current.resolve().relative_to(anchor.resolve())
|
||||
except ValueError:
|
||||
return None
|
||||
new_db = Path(str(value)) / tail
|
||||
if new_db.resolve() == current.resolve() or new_db.is_file():
|
||||
return None
|
||||
return (
|
||||
f"{current} holds your existing conversations and memories, but this "
|
||||
f"change points NexusOS at {new_db}, which does not exist yet - it will "
|
||||
"start with an empty database. Stop NexusOS and move the .db (plus any "
|
||||
"-wal/-shm files) to the new path to keep your history."
|
||||
)
|
||||
|
||||
|
||||
def cmd_config(args) -> int:
|
||||
values = config.read_user_config()
|
||||
if args.action == "path":
|
||||
@@ -161,8 +202,14 @@ def cmd_config(args) -> int:
|
||||
except (TypeError, ValueError) as exc:
|
||||
print(f"Invalid value: {exc}", file=sys.stderr)
|
||||
return 2
|
||||
payload = {"updated": args.key, "value": values[args.key], "restart_required": True}
|
||||
moved = _relocation_warning(args.key, values[args.key])
|
||||
if moved:
|
||||
payload["warning"] = moved
|
||||
config.write_user_config(values)
|
||||
_emit({"updated": args.key, "value": values[args.key], "restart_required": True}, args.json)
|
||||
_emit(payload, args.json)
|
||||
if moved and not args.json:
|
||||
print(f"\nWARNING: {moved}", file=sys.stderr)
|
||||
return 0
|
||||
if args.action == "unset":
|
||||
if args.key not in CONFIG_SCHEMA:
|
||||
@@ -275,6 +322,7 @@ def diagnostics() -> dict:
|
||||
("voice transcription", "faster_whisper"),
|
||||
("PDF documents", "pypdf"),
|
||||
("Word documents", "docx"),
|
||||
("web search", "duckduckgo_search"),
|
||||
):
|
||||
add(label, _check_import(module), module, required=False)
|
||||
|
||||
@@ -341,6 +389,15 @@ def cmd_status(args) -> int:
|
||||
return 0
|
||||
|
||||
|
||||
def cmd_monitor(args) -> int:
|
||||
from .monitor import run_monitor
|
||||
return run_monitor(
|
||||
interval=getattr(args, "interval", 1.5),
|
||||
once=bool(getattr(args, "once", False) or getattr(args, "json", False)),
|
||||
json_output=bool(getattr(args, "json", False)),
|
||||
)
|
||||
|
||||
|
||||
def _target_flag(target: str | None):
|
||||
return {
|
||||
"memory": "--memory",
|
||||
@@ -366,6 +423,58 @@ def cmd_refresh(args) -> int:
|
||||
return 0
|
||||
|
||||
|
||||
def _lan_hostnames(host: str) -> list[str]:
|
||||
"""Every name/address a --allow-lan bind should accept in a Host header.
|
||||
|
||||
A wildcard bind answers on all interfaces, so enumerate them; an explicit
|
||||
address answers only as itself. The machine hostname comes along because
|
||||
that is what people actually type."""
|
||||
import socket
|
||||
|
||||
names: list[str] = []
|
||||
|
||||
def add(value: str) -> None:
|
||||
if value and value not in names:
|
||||
names.append(value)
|
||||
|
||||
if host in ("0.0.0.0", "::", "*"):
|
||||
hostname = socket.gethostname()
|
||||
add(hostname)
|
||||
add(hostname.split(".")[0] + ".local")
|
||||
for family in (socket.AF_INET, socket.AF_INET6):
|
||||
try:
|
||||
for info in socket.getaddrinfo(hostname, None, family):
|
||||
add(info[4][0])
|
||||
except OSError:
|
||||
pass
|
||||
# getaddrinfo(hostname) misses the routable address on hosts that map
|
||||
# their own name to loopback; a connectionless UDP socket finds it.
|
||||
for probe, family in (("8.8.8.8", socket.AF_INET), ("2001:4860:4860::8888", socket.AF_INET6)):
|
||||
sock = socket.socket(family, socket.SOCK_DGRAM)
|
||||
try:
|
||||
sock.connect((probe, 80))
|
||||
add(sock.getsockname()[0])
|
||||
except OSError:
|
||||
pass
|
||||
finally:
|
||||
sock.close()
|
||||
else:
|
||||
add(host.strip("[]"))
|
||||
# A Host header carries an IPv6 literal bracketed; allow both spellings so
|
||||
# the check matches however the client wrote it.
|
||||
for value in list(names):
|
||||
if ":" in value:
|
||||
add(f"[{value}]")
|
||||
return names
|
||||
|
||||
|
||||
def _origin_host(name: str) -> str:
|
||||
"""Origin-safe spelling: IPv6 literals must be bracketed in a URL."""
|
||||
if ":" in name and not name.startswith("["):
|
||||
return f"[{name}]"
|
||||
return name
|
||||
|
||||
|
||||
def cmd_serve(args) -> int:
|
||||
host = args.host or settings.bind_host
|
||||
if host not in ("127.0.0.1", "localhost", "::1") and not args.allow_lan:
|
||||
@@ -389,10 +498,27 @@ def cmd_serve(args) -> int:
|
||||
if origin not in config.ALLOWED_ORIGINS:
|
||||
config.ALLOWED_ORIGINS.append(origin)
|
||||
if args.allow_lan:
|
||||
os.environ.setdefault("NEXUS_ALLOWED_HOSTS", "*")
|
||||
os.environ.setdefault("NEXUS_ALLOWED_ORIGINS", "*")
|
||||
config.ALLOWED_HOSTS[:] = ["*"]
|
||||
config.ALLOWED_ORIGINS[:] = ["*"]
|
||||
# Widen to the addresses this bind actually answers on - NOT "*".
|
||||
# ALLOWED_HOSTS drives TrustedHostMiddleware, which is the DNS-rebinding
|
||||
# defense: with "*" any site the user browses could resolve a name it
|
||||
# controls to this machine and drive the unauthenticated API. Naming the
|
||||
# real addresses keeps that check doing its job. An explicitly exported
|
||||
# NEXUS_ALLOWED_HOSTS still wins, for anyone who needs the old blanket.
|
||||
names = _lan_hostnames(host)
|
||||
for name in names:
|
||||
if name not in config.ALLOWED_HOSTS:
|
||||
config.ALLOWED_HOSTS.append(name)
|
||||
for port in (args.port, args.memory_port):
|
||||
origin = f"http://{_origin_host(name)}:{port}"
|
||||
if origin not in config.ALLOWED_ORIGINS:
|
||||
config.ALLOWED_ORIGINS.append(origin)
|
||||
os.environ.setdefault("NEXUS_ALLOWED_HOSTS", ",".join(config.ALLOWED_HOSTS))
|
||||
os.environ.setdefault("NEXUS_ALLOWED_ORIGINS", ",".join(config.ALLOWED_ORIGINS))
|
||||
print(
|
||||
"LAN exposure enabled for: " + ", ".join(names)
|
||||
+ "\nThe REST API is unauthenticated - anyone who can reach this port"
|
||||
" has full admin and data access."
|
||||
)
|
||||
|
||||
memory_proc = None
|
||||
memory_log = None
|
||||
@@ -617,6 +743,11 @@ def build_parser() -> argparse.ArgumentParser:
|
||||
p = sub.add_parser("doctor", help="check the core runtime and optional capabilities")
|
||||
p.add_argument("--fix", action="store_true"); _add_json(p); p.set_defaults(fn=cmd_doctor)
|
||||
p = sub.add_parser("status", help="show service and provider status"); _add_json(p); p.set_defaults(fn=cmd_status)
|
||||
p = sub.add_parser("monitor", help="ASCII dashboard for services, resources, and tool stats")
|
||||
p.add_argument("--once", action="store_true", help="print one frame and exit")
|
||||
p.add_argument("--interval", type=float, default=1.5, help="refresh seconds (live mode)")
|
||||
_add_json(p)
|
||||
p.set_defaults(fn=cmd_monitor)
|
||||
|
||||
p = sub.add_parser("serve", help="run NexusOS in the foreground")
|
||||
p.add_argument("--host"); p.add_argument("--port", type=_port, default=settings.backend_port)
|
||||
@@ -672,8 +803,16 @@ def _normalize_legacy_argv(argv) -> list[str]:
|
||||
normalized = list(argv or [])
|
||||
if normalized == ["help"]:
|
||||
return ["--help"]
|
||||
if normalized and normalized[0] in ("start", "stop", "logs"):
|
||||
# start/stop only. `logs` registers -f as the short form of --follow, so
|
||||
# translating it here would silently rewrite `logs -f` to `logs frontend`
|
||||
# - a tail of the wrong file instead of a follow, with no error.
|
||||
if normalized and normalized[0] in ("start", "stop"):
|
||||
normalized[1:] = [LEGACY_TARGETS.get(value, value) for value in normalized[1:]]
|
||||
elif normalized[:1] == ["logs"]:
|
||||
normalized[1:] = [
|
||||
value if value in ("-f", "--follow") else LEGACY_TARGETS.get(value, value)
|
||||
for value in normalized[1:]
|
||||
]
|
||||
if normalized[:2] == ["backup", "full"]:
|
||||
normalized[1] = "--full"
|
||||
elif len(normalized) > 1 and normalized[0] == "backup" and normalized[1] in ("check", "--claude"):
|
||||
@@ -0,0 +1,458 @@
|
||||
"""ASCII dashboard for live NexusOS service / tool / resource stats.
|
||||
|
||||
No curses, no rich — pure box-drawing + optional ANSI color so it works in
|
||||
Termux, plain SSH, and Windows Terminal alike. The collector is separate from
|
||||
the renderer so tests can feed fixtures without a running stack.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import shutil
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from synapse.nexus_config import settings
|
||||
|
||||
from . import ncp as services
|
||||
|
||||
# Box drawing — ASCII fallbacks when the terminal can't do Unicode.
|
||||
_BOX = {
|
||||
"tl": "┌", "tr": "┐", "bl": "└", "br": "┘",
|
||||
"h": "─", "v": "│", "l": "├", "r": "┤",
|
||||
}
|
||||
_BOX_ASCII = {
|
||||
"tl": "+", "tr": "+", "bl": "+", "br": "+",
|
||||
"h": "-", "v": "|", "l": "+", "r": "+",
|
||||
}
|
||||
|
||||
_FILL = "█"
|
||||
_EMPTY = "░"
|
||||
_FILL_ASCII = "#"
|
||||
_EMPTY_ASCII = "-"
|
||||
|
||||
|
||||
def _use_unicode() -> bool:
|
||||
enc = (getattr(__import__("sys").stdout, "encoding", None) or "").lower()
|
||||
return "utf" in enc or enc in ("cp65001",)
|
||||
|
||||
|
||||
def _http_ok(url: str, timeout: float = 1.0) -> bool:
|
||||
try:
|
||||
urllib.request.urlopen(url, timeout=timeout).read(1)
|
||||
return True
|
||||
except urllib.error.HTTPError:
|
||||
return True
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def _provider_payload() -> dict:
|
||||
url = settings.ollama_host.rstrip("/")
|
||||
return {
|
||||
"provider": settings.provider,
|
||||
"url": url,
|
||||
"managed_by_nexus": settings.manage_ollama,
|
||||
"reachable": _http_ok(url + "/api/tags", timeout=2.0),
|
||||
}
|
||||
|
||||
|
||||
def _service_status() -> dict:
|
||||
payload = {}
|
||||
for key in ("backend", "memory", "frontend"):
|
||||
svc = services.SERVICES[key]
|
||||
pid = services.read_pid(svc)
|
||||
payload[key] = {
|
||||
"running": services.alive(pid) or _http_ok(svc.url),
|
||||
"pid": pid if services.alive(pid) else None,
|
||||
"url": svc.url,
|
||||
}
|
||||
payload["provider"] = _provider_payload()
|
||||
return payload
|
||||
|
||||
|
||||
def _get_json(url: str, timeout: float = 1.5) -> Any | None:
|
||||
try:
|
||||
with urllib.request.urlopen(url, timeout=timeout) as resp:
|
||||
return json.loads(resp.read().decode("utf-8", errors="replace"))
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def _bar(ratio: float, width: int = 20, unicode: bool = True) -> str:
|
||||
ratio = max(0.0, min(1.0, float(ratio)))
|
||||
filled = int(round(ratio * width))
|
||||
fill = _FILL if unicode else _FILL_ASCII
|
||||
empty = _EMPTY if unicode else _EMPTY_ASCII
|
||||
return fill * filled + empty * (width - filled)
|
||||
|
||||
|
||||
def _fmt_bytes(n: float | int | None) -> str:
|
||||
if n is None:
|
||||
return "—"
|
||||
n = float(n)
|
||||
for unit in ("B", "K", "M", "G", "T"):
|
||||
if abs(n) < 1024 or unit == "T":
|
||||
return f"{n:.0f}{unit}" if unit == "B" else f"{n:.1f}{unit}"
|
||||
n /= 1024
|
||||
return f"{n:.1f}T"
|
||||
|
||||
|
||||
def _pid_stats(pids: list[int | None]) -> dict:
|
||||
"""Aggregate CPU%/RSS for known service PIDs. Soft-depends on psutil."""
|
||||
live = [int(p) for p in pids if p]
|
||||
if not live:
|
||||
return {"cpu_pct": None, "rss": None, "pids": []}
|
||||
try:
|
||||
import psutil # type: ignore
|
||||
except ImportError:
|
||||
return {"cpu_pct": None, "rss": None, "pids": live}
|
||||
cpu = 0.0
|
||||
rss = 0
|
||||
seen: list[int] = []
|
||||
for pid in live:
|
||||
try:
|
||||
proc = psutil.Process(pid)
|
||||
cpu += proc.cpu_percent(interval=0.0)
|
||||
rss += proc.memory_info().rss
|
||||
seen.append(pid)
|
||||
except (psutil.Error, ProcessLookupError, ValueError):
|
||||
continue
|
||||
return {"cpu_pct": cpu, "rss": rss, "pids": seen}
|
||||
|
||||
|
||||
def _host_stats() -> dict:
|
||||
try:
|
||||
import psutil # type: ignore
|
||||
except ImportError:
|
||||
return {"cpu_pct": None, "mem_used": None, "mem_total": None, "mem_pct": None}
|
||||
vm = psutil.virtual_memory()
|
||||
return {
|
||||
"cpu_pct": psutil.cpu_percent(interval=0.05),
|
||||
"mem_used": vm.used,
|
||||
"mem_total": vm.total,
|
||||
"mem_pct": vm.percent,
|
||||
}
|
||||
|
||||
|
||||
def _api_counts(api_url: str) -> dict:
|
||||
"""Pull cheap inventory counts from the backend when it is up."""
|
||||
base = api_url.rstrip("/")
|
||||
out = {
|
||||
"online": False,
|
||||
"version": None,
|
||||
"ollama": None,
|
||||
"memories": None,
|
||||
"conversations": None,
|
||||
"playbooks": None,
|
||||
"models": None,
|
||||
"action_tool_policy": None,
|
||||
}
|
||||
status = _get_json(base + "/status")
|
||||
if not isinstance(status, dict):
|
||||
return out
|
||||
out["online"] = True
|
||||
out["version"] = status.get("version")
|
||||
out["ollama"] = status.get("ollama")
|
||||
|
||||
mem = _get_json(base + "/memory")
|
||||
if isinstance(mem, list):
|
||||
out["memories"] = len(mem)
|
||||
elif isinstance(mem, dict) and isinstance(mem.get("memories"), list):
|
||||
out["memories"] = len(mem["memories"])
|
||||
|
||||
conv = _get_json(base + "/conversations")
|
||||
if isinstance(conv, list):
|
||||
out["conversations"] = len(conv)
|
||||
elif isinstance(conv, dict):
|
||||
items = conv.get("conversations") or conv.get("items") or []
|
||||
if isinstance(items, list):
|
||||
out["conversations"] = len(items)
|
||||
|
||||
pbs = _get_json(base + "/playbooks")
|
||||
if isinstance(pbs, list):
|
||||
out["playbooks"] = len(pbs)
|
||||
elif isinstance(pbs, dict) and isinstance(pbs.get("playbooks"), list):
|
||||
out["playbooks"] = len(pbs["playbooks"])
|
||||
|
||||
models = _get_json(base + "/models")
|
||||
if isinstance(models, list):
|
||||
out["models"] = len(models)
|
||||
elif isinstance(models, dict):
|
||||
items = models.get("models") or models.get("items") or []
|
||||
if isinstance(items, list):
|
||||
out["models"] = len(items)
|
||||
|
||||
settings_payload = _get_json(base + "/settings")
|
||||
if isinstance(settings_payload, dict):
|
||||
out["action_tool_policy"] = settings_payload.get("action_tool_policy")
|
||||
|
||||
return out
|
||||
|
||||
|
||||
def _toolchain_stats() -> list[dict]:
|
||||
"""Which run_snippet languages have a host toolchain right now."""
|
||||
try:
|
||||
from synapse import code_run
|
||||
except Exception:
|
||||
return []
|
||||
rows = []
|
||||
for name, spec in code_run.RUN_LANGS.items():
|
||||
tool = None
|
||||
try:
|
||||
tool = spec["tool"]()
|
||||
except Exception:
|
||||
tool = None
|
||||
rows.append({
|
||||
"lang": name,
|
||||
"ready": bool(tool),
|
||||
"tool": tool or None,
|
||||
"summary": spec.get("summary") or name,
|
||||
})
|
||||
return rows
|
||||
|
||||
|
||||
def _recent_tools(log_path: Path, limit: int = 8) -> list[str]:
|
||||
"""Best-effort scrape of recent tool names from chat.log."""
|
||||
if not log_path.is_file():
|
||||
return []
|
||||
try:
|
||||
# Read the tail without pulling a multi-MB log into memory.
|
||||
data = log_path.read_bytes()
|
||||
if len(data) > 64_000:
|
||||
data = data[-64_000:]
|
||||
text = data.decode("utf-8", errors="replace")
|
||||
except OSError:
|
||||
return []
|
||||
found: list[str] = []
|
||||
for line in reversed(text.splitlines()):
|
||||
# Chat tool loop yields "__status__<tool>"; logs may also name tools
|
||||
# in JSON payloads. Keep the match narrow.
|
||||
if "__status__" in line:
|
||||
name = line.split("__status__", 1)[-1].strip().split()[0].strip(",\"'")
|
||||
if name and name not in ("tools",) and name not in found:
|
||||
found.append(name)
|
||||
elif '"name":' in line and any(
|
||||
t in line for t in ("render_preview", "run_snippet", "web_search",
|
||||
"fetch_url", "remember", "get_time")
|
||||
):
|
||||
for t in ("run_snippet", "render_preview", "web_search", "fetch_url",
|
||||
"remember", "get_time", "search_documents"):
|
||||
if t in line and t not in found:
|
||||
found.append(t)
|
||||
if len(found) >= limit:
|
||||
break
|
||||
return found
|
||||
|
||||
|
||||
def collect_snapshot() -> dict:
|
||||
"""Gather one monitoring frame. Safe when services are down."""
|
||||
services_payload = _service_status()
|
||||
pids = [
|
||||
services_payload.get("backend", {}).get("pid"),
|
||||
services_payload.get("memory", {}).get("pid"),
|
||||
services_payload.get("frontend", {}).get("pid"),
|
||||
]
|
||||
api = _api_counts(settings.api_url)
|
||||
return {
|
||||
"ts": datetime.now(timezone.utc).astimezone().isoformat(timespec="seconds"),
|
||||
"version": settings.version,
|
||||
"services": services_payload,
|
||||
"api": api,
|
||||
"host": _host_stats(),
|
||||
"procs": _pid_stats(pids),
|
||||
"toolchains": _toolchain_stats(),
|
||||
"recent_tools": _recent_tools(settings.logs_dir / "chat.log"),
|
||||
"paths": {
|
||||
"api_url": settings.api_url,
|
||||
"memory_url": settings.memory_url,
|
||||
"runtime_dir": str(settings.runtime_dir),
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def _pad(text: str, width: int) -> str:
|
||||
# Visual width ≈ len for our ASCII/box content (no wide East-Asian chars).
|
||||
if len(text) > width:
|
||||
return text[: width - 1] + "…" if width > 1 else text[:width]
|
||||
return text + " " * (width - len(text))
|
||||
|
||||
|
||||
def _row(box: dict, inner: str, width: int) -> str:
|
||||
return f"{box['v']} {_pad(inner, width - 4)} {box['v']}"
|
||||
|
||||
|
||||
def _rule(box: dict, width: int, kind: str = "mid") -> str:
|
||||
h = box["h"] * (width - 2)
|
||||
if kind == "top":
|
||||
return f"{box['tl']}{h}{box['tr']}"
|
||||
if kind == "bot":
|
||||
return f"{box['bl']}{h}{box['br']}"
|
||||
return f"{box['l']}{h}{box['r']}"
|
||||
|
||||
|
||||
def _svc_line(name: str, running: bool, detail: str, unicode: bool) -> str:
|
||||
mark = (_FILL if unicode else _FILL_ASCII) * 3 if running else (_EMPTY if unicode else _EMPTY_ASCII) * 3
|
||||
state = "UP " if running else "DOWN"
|
||||
return f"{name:<10} {mark} {state} {detail}"
|
||||
|
||||
|
||||
def render_frame(snapshot: dict, *, width: int | None = None, unicode: bool | None = None) -> str:
|
||||
"""Turn a snapshot into a single multi-line ASCII panel."""
|
||||
if unicode is None:
|
||||
unicode = _use_unicode()
|
||||
box = _BOX if unicode else _BOX_ASCII
|
||||
cols = shutil.get_terminal_size((80, 24)).columns if width is None else width
|
||||
width = max(56, min(100, cols))
|
||||
|
||||
lines: list[str] = []
|
||||
lines.append(_rule(box, width, "top"))
|
||||
title = f"NexusOS {snapshot.get('version', '')} monitor"
|
||||
raw_ts = snapshot.get("ts") or ""
|
||||
# Prefer local clock HH:MM:SS from an ISO stamp; fall back to wall clock.
|
||||
stamp = ""
|
||||
if "T" in raw_ts:
|
||||
try:
|
||||
stamp = raw_ts.split("T", 1)[1][:8]
|
||||
except Exception:
|
||||
stamp = ""
|
||||
if not stamp:
|
||||
stamp = datetime.now().strftime("%H:%M:%S")
|
||||
gap = max(1, width - 4 - len(title) - len(stamp))
|
||||
header = f"{title}{' ' * gap}{stamp}"
|
||||
lines.append(_row(box, header, width))
|
||||
lines.append(_rule(box, width, "mid"))
|
||||
|
||||
lines.append(_row(box, "SERVICES", width))
|
||||
svcs = snapshot.get("services") or {}
|
||||
for key, label in (("backend", "backend"), ("memory", "memory"), ("frontend", "frontend")):
|
||||
info = svcs.get(key) or {}
|
||||
running = bool(info.get("running"))
|
||||
pid = info.get("pid")
|
||||
url = info.get("url") or ""
|
||||
detail = url
|
||||
if pid:
|
||||
detail = f"pid {pid} {url}"
|
||||
lines.append(_row(box, _svc_line(label, running, detail, unicode), width))
|
||||
provider = svcs.get("provider") or _provider_payload()
|
||||
pref = f"{provider.get('provider', '?')} @ {provider.get('url', '')}"
|
||||
lines.append(_row(box, _svc_line("provider", bool(provider.get("reachable")), pref, unicode), width))
|
||||
|
||||
lines.append(_rule(box, width, "mid"))
|
||||
lines.append(_row(box, "RESOURCES", width))
|
||||
host = snapshot.get("host") or {}
|
||||
procs = snapshot.get("procs") or {}
|
||||
cpu = host.get("cpu_pct")
|
||||
if cpu is not None:
|
||||
lines.append(_row(
|
||||
box,
|
||||
f"host CPU [{_bar(cpu / 100.0, 22, unicode)}] {cpu:5.1f}%",
|
||||
width,
|
||||
))
|
||||
else:
|
||||
lines.append(_row(box, "host CPU (install psutil for live bars)", width))
|
||||
mem_pct = host.get("mem_pct")
|
||||
if mem_pct is not None:
|
||||
lines.append(_row(
|
||||
box,
|
||||
f"host MEM [{_bar(mem_pct / 100.0, 22, unicode)}] "
|
||||
f"{_fmt_bytes(host.get('mem_used'))} / {_fmt_bytes(host.get('mem_total'))}",
|
||||
width,
|
||||
))
|
||||
proc_cpu = procs.get("cpu_pct")
|
||||
proc_rss = procs.get("rss")
|
||||
if proc_cpu is not None or proc_rss is not None:
|
||||
lines.append(_row(
|
||||
box,
|
||||
f"nexus cpu={proc_cpu if proc_cpu is not None else '—':>5} "
|
||||
f"rss={_fmt_bytes(proc_rss)} pids={','.join(str(p) for p in (procs.get('pids') or [])) or '—'}",
|
||||
width,
|
||||
))
|
||||
|
||||
lines.append(_rule(box, width, "mid"))
|
||||
lines.append(_row(box, "DATA / TOOLS", width))
|
||||
api = snapshot.get("api") or {}
|
||||
if api.get("online"):
|
||||
policy = api.get("action_tool_policy") or "—"
|
||||
lines.append(_row(
|
||||
box,
|
||||
f"api UP v{api.get('version') or '?'} ollama={api.get('ollama') or '—'} "
|
||||
f"tools={policy}",
|
||||
width,
|
||||
))
|
||||
lines.append(_row(
|
||||
box,
|
||||
f"memories={_n(api.get('memories'))} "
|
||||
f"chats={_n(api.get('conversations'))} "
|
||||
f"playbooks={_n(api.get('playbooks'))} "
|
||||
f"models={_n(api.get('models'))}",
|
||||
width,
|
||||
))
|
||||
else:
|
||||
lines.append(_row(box, "api DOWN — start with: nexus start", width))
|
||||
|
||||
recent = snapshot.get("recent_tools") or []
|
||||
lines.append(_row(
|
||||
box,
|
||||
"recent " + (", ".join(recent) if recent else "(none in chat.log)"),
|
||||
width,
|
||||
))
|
||||
|
||||
lines.append(_rule(box, width, "mid"))
|
||||
lines.append(_row(box, "RUN TOOLCHAINS (run_snippet)", width))
|
||||
chains = snapshot.get("toolchains") or []
|
||||
if not chains:
|
||||
lines.append(_row(box, "(code_run unavailable)", width))
|
||||
else:
|
||||
# Pack ready/missing into one or two compact lines.
|
||||
ready = [c["lang"] for c in chains if c.get("ready")]
|
||||
missing = [c["lang"] for c in chains if not c.get("ready")]
|
||||
lines.append(_row(
|
||||
box,
|
||||
f"ready {', '.join(ready) if ready else '—'}",
|
||||
width,
|
||||
))
|
||||
lines.append(_row(
|
||||
box,
|
||||
f"missing {', '.join(missing) if missing else '—'}",
|
||||
width,
|
||||
))
|
||||
|
||||
lines.append(_rule(box, width, "bot"))
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def _n(value) -> str:
|
||||
return "—" if value is None else str(value)
|
||||
|
||||
|
||||
def run_monitor(*, interval: float = 1.5, once: bool = False, json_output: bool = False) -> int:
|
||||
"""Print one frame, or refresh in place until interrupted."""
|
||||
clear = "\033[H\033[J"
|
||||
first = True
|
||||
while True:
|
||||
snap = collect_snapshot()
|
||||
if json_output:
|
||||
print(json.dumps(snap, indent=2, sort_keys=True))
|
||||
else:
|
||||
frame = render_frame(snap)
|
||||
if once or not first:
|
||||
# Replacing the screen keeps the panel stable; first frame of a
|
||||
# live session also clears so leftover shell output doesn't mix.
|
||||
if not once:
|
||||
print(clear + frame, end="", flush=True)
|
||||
else:
|
||||
print(frame)
|
||||
else:
|
||||
print(clear + frame, end="", flush=True)
|
||||
first = False
|
||||
if once:
|
||||
return 0
|
||||
try:
|
||||
time.sleep(max(0.3, float(interval)))
|
||||
except KeyboardInterrupt:
|
||||
print()
|
||||
return 130
|
||||
@@ -28,6 +28,7 @@ import urllib.request
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
|
||||
from synapse import proc_util
|
||||
from synapse.nexus_config import SOURCE_CHECKOUT, settings
|
||||
|
||||
ROOT = settings.project_root
|
||||
@@ -170,11 +171,9 @@ def alive(pid) -> bool:
|
||||
return False
|
||||
if ps is not None:
|
||||
return ps.pid_exists(pid)
|
||||
try:
|
||||
os.kill(pid, 0)
|
||||
return True
|
||||
except (OSError, ValueError):
|
||||
return False
|
||||
# Not os.kill(pid, 0): that reports False for a live process owned by
|
||||
# another user, and Windows has no signals to fall back on.
|
||||
return proc_util.pid_alive(pid)
|
||||
|
||||
|
||||
def pid_is_ours(pid, patterns) -> bool:
|
||||
@@ -186,11 +185,8 @@ def pid_is_ours(pid, patterns) -> bool:
|
||||
try:
|
||||
if ps is not None:
|
||||
cmd = " ".join(ps.Process(pid).cmdline())
|
||||
elif os.name != "nt":
|
||||
cmd = (Path(f"/proc/{pid}/cmdline").read_bytes()
|
||||
.replace(b"\0", b" ").decode(errors="replace"))
|
||||
else:
|
||||
return False
|
||||
cmd = proc_util.pid_cmdline(pid)
|
||||
except Exception:
|
||||
return False
|
||||
return any(p in cmd for p in patterns)
|
||||
@@ -278,18 +274,11 @@ def kill_matching(patterns, force=False) -> int:
|
||||
me = os.getpid()
|
||||
hit = 0
|
||||
if ps is None:
|
||||
if os.name == "nt":
|
||||
return 0
|
||||
for entry in Path("/proc").iterdir():
|
||||
if not entry.name.isdigit() or int(entry.name) == me:
|
||||
for pid, cmd in proc_util.iter_processes():
|
||||
if pid == me or not any(pattern in cmd for pattern in patterns):
|
||||
continue
|
||||
try:
|
||||
cmd = (entry / "cmdline").read_bytes().replace(b"\0", b" ").decode(errors="replace")
|
||||
if any(pattern in cmd for pattern in patterns):
|
||||
os.kill(int(entry.name), 9 if force else 15)
|
||||
if proc_util.terminate_pid(pid, force=force):
|
||||
hit += 1
|
||||
except (OSError, ValueError):
|
||||
pass
|
||||
return hit
|
||||
for proc in ps.process_iter(["pid", "cmdline"]):
|
||||
if proc.info["pid"] == me:
|
||||
@@ -309,7 +298,11 @@ def kill_port(port: int) -> bool:
|
||||
stop reliable regardless of process-tree shape or PID-file accuracy."""
|
||||
ps = _psutil()
|
||||
if ps is None:
|
||||
return False
|
||||
killed = False
|
||||
for pid in proc_util.pids_listening_on(port):
|
||||
if pid != os.getpid() and proc_util.terminate_pid(pid, force=True):
|
||||
killed = True
|
||||
return killed
|
||||
killed = False
|
||||
try:
|
||||
conns = ps.net_connections(kind="inet")
|
||||
@@ -341,8 +334,10 @@ def stop_service(svc: Service) -> bool:
|
||||
except Exception:
|
||||
pass
|
||||
proc.terminate()
|
||||
elif os.name != "nt":
|
||||
os.kill(pid, 15)
|
||||
else:
|
||||
# No psutil means no process tree; the kill_matching sweep
|
||||
# below is what catches reparented children here.
|
||||
proc_util.terminate_pid(pid)
|
||||
except Exception:
|
||||
pass
|
||||
svc.pid_file.unlink(missing_ok=True)
|
||||
+28
-8
@@ -39,6 +39,9 @@ process = ["psutil>=5.9,<8"]
|
||||
vector = ["sqlite-vec>=0.1,<1"]
|
||||
voice = ["faster-whisper>=1.1,<2"]
|
||||
mail = ["imap-tools>=1.7,<2"]
|
||||
# synapse/search.py imports this lazily behind a bare except, so without it
|
||||
# declared the chat web-search path silently returns nothing.
|
||||
search = ["duckduckgo-search>=6,<9"]
|
||||
desktop = [
|
||||
"psutil>=5.9,<8",
|
||||
"pywebview>=5,<7; platform_system == 'Windows'",
|
||||
@@ -48,6 +51,20 @@ standard = [
|
||||
"python-docx>=1.1,<2",
|
||||
"psutil>=5.9,<8",
|
||||
"sqlite-vec>=0.1,<1",
|
||||
"duckduckgo-search>=6,<9",
|
||||
]
|
||||
# Every optional capability at once. Kept in sync with the extras above by
|
||||
# tests/test_packaging_deps.py, which also checks that nothing synapse imports
|
||||
# is missing from this file.
|
||||
all = [
|
||||
"pypdf>=5,<7",
|
||||
"python-docx>=1.1,<2",
|
||||
"psutil>=5.9,<8",
|
||||
"sqlite-vec>=0.1,<1",
|
||||
"faster-whisper>=1.1,<2",
|
||||
"imap-tools>=1.7,<2",
|
||||
"duckduckgo-search>=6,<9",
|
||||
"pywebview>=5,<7; platform_system == 'Windows'",
|
||||
]
|
||||
dev = [
|
||||
"build>=1.2,<2",
|
||||
@@ -56,9 +73,9 @@ dev = [
|
||||
]
|
||||
|
||||
[project.scripts]
|
||||
nexus = "management.cli:entrypoint"
|
||||
ncp = "management.cli:entrypoint"
|
||||
nexusos = "management.cli:entrypoint"
|
||||
nexus = "nexusos_cli.cli:entrypoint"
|
||||
ncp = "nexusos_cli.cli:entrypoint"
|
||||
nexusos = "nexusos_cli.cli:entrypoint"
|
||||
|
||||
[project.urls]
|
||||
Homepage = "https://git.enderofwings.com/enderofwings/NexusOS"
|
||||
@@ -72,12 +89,16 @@ pattern = "^(?P<version>[^\\s]+)$"
|
||||
[tool.hatch.build]
|
||||
skip-excluded-dirs = true
|
||||
|
||||
# Ships interface/web/dist when it has been built. It is gitignored, so a
|
||||
# static force-include would abort `pip install -e .` on a fresh clone.
|
||||
[tool.hatch.build.hooks.custom]
|
||||
path = "hatch_build.py"
|
||||
|
||||
[tool.hatch.build.targets.wheel]
|
||||
packages = ["synapse", "management"]
|
||||
packages = ["synapse", "nexusos_cli"]
|
||||
|
||||
[tool.hatch.build.targets.wheel.force-include]
|
||||
"VERSION" = "synapse/_resources/VERSION"
|
||||
"interface/web/dist" = "synapse/_resources/web"
|
||||
"data/playbooks" = "synapse/_resources/playbooks"
|
||||
"assets/n-small.png" = "synapse/_resources/assets/n-small.png"
|
||||
"assets/themes/NexusOS-icons-src/nexus-underlay.svg" = "synapse/_resources/assets/themes/NexusOS-icons-src/nexus-underlay.svg"
|
||||
@@ -90,18 +111,17 @@ include = [
|
||||
"/assets/themes/NexusOS-icons-src/nexus-underlay-ring.svg",
|
||||
"/data/playbooks",
|
||||
"/docs",
|
||||
"/interface/web/dist",
|
||||
"/management",
|
||||
"/nexusos_cli",
|
||||
"/scripts",
|
||||
"/synapse",
|
||||
"/tests",
|
||||
"/README.md",
|
||||
"/VERSION",
|
||||
"/pyproject.toml",
|
||||
"/hatch_build.py",
|
||||
]
|
||||
|
||||
[tool.hatch.build.targets.sdist.force-include]
|
||||
"interface/web/dist" = "interface/web/dist"
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
testpaths = ["tests", "management"]
|
||||
|
||||
@@ -38,6 +38,9 @@ sqlite-vec
|
||||
faster-whisper
|
||||
# Email client: IMAP read (SMTP send is stdlib). Pure-Python, no native deps.
|
||||
imap-tools
|
||||
# Chat web search (synapse/search.py). Imported lazily behind a bare except,
|
||||
# so a missing install shows up as search silently returning nothing.
|
||||
duckduckgo-search
|
||||
|
||||
# Documentation Support
|
||||
markdown-it-py
|
||||
|
||||
@@ -46,7 +46,9 @@ EOF
|
||||
fi
|
||||
|
||||
echo "==> Installing ${PACKAGE_SPEC}"
|
||||
python -m pip install "${PACKAGE_SPEC}"
|
||||
# Same extra index as the pydantic-core probe: a NexusOS wheel hosted there
|
||||
# would otherwise be invisible to the resolver.
|
||||
python -m pip install "${PIP_INDEX_ARGS[@]}" "${PACKAGE_SPEC}"
|
||||
|
||||
echo "==> Initializing NexusOS"
|
||||
nexus init
|
||||
@@ -55,7 +57,12 @@ if [[ -n "${NEXUS_PROVIDER_URL:-}" ]]; then
|
||||
nexus provider use remote --url "${NEXUS_PROVIDER_URL}"
|
||||
fi
|
||||
|
||||
nexus doctor
|
||||
# Report health without aborting: `set -e` would otherwise skip the guidance
|
||||
# below whenever doctor finds a failing required check - exactly when the
|
||||
# reader most needs to see what to do next. The status is re-raised at exit.
|
||||
DOCTOR_STATUS=0
|
||||
nexus doctor || DOCTOR_STATUS=$?
|
||||
|
||||
cat <<'EOF'
|
||||
|
||||
NexusOS is installed. Configure an Ollama-compatible provider, then run:
|
||||
@@ -64,3 +71,5 @@ NexusOS is installed. Configure an Ollama-compatible provider, then run:
|
||||
|
||||
Open http://127.0.0.1:8000 in the Android browser.
|
||||
EOF
|
||||
|
||||
exit "${DOCTOR_STATUS}"
|
||||
|
||||
@@ -12,13 +12,13 @@ import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
|
||||
try:
|
||||
import psutil
|
||||
except ImportError: # optional in the portable/Termux core install
|
||||
psutil = None
|
||||
|
||||
from . import proc_util
|
||||
from .nexus_config import FRONTEND_SOURCE_DIR, RUNTIME_DIR
|
||||
|
||||
FRONTEND_DIR = FRONTEND_SOURCE_DIR
|
||||
@@ -47,11 +47,8 @@ def _is_ours(pid: int) -> bool:
|
||||
try:
|
||||
if psutil is not None:
|
||||
cmd = " ".join(psutil.Process(pid).cmdline())
|
||||
elif os.name != "nt":
|
||||
cmd = (Path(f"/proc/{pid}/cmdline").read_bytes()
|
||||
.replace(b"\0", b" ").decode(errors="replace"))
|
||||
else:
|
||||
return False
|
||||
cmd = proc_util.pid_cmdline(pid)
|
||||
except Exception:
|
||||
return False
|
||||
return "vite" in cmd or ("npm" in cmd and "dev" in cmd)
|
||||
@@ -62,11 +59,9 @@ def _alive(pid: int | None) -> bool:
|
||||
return False
|
||||
if psutil is not None:
|
||||
return psutil.pid_exists(pid)
|
||||
try:
|
||||
os.kill(pid, 0)
|
||||
return True
|
||||
except (OSError, ValueError):
|
||||
return False
|
||||
# proc_util rather than os.kill(pid, 0): same probe, but it also works
|
||||
# when the PID belongs to another user.
|
||||
return proc_util.pid_alive(pid)
|
||||
|
||||
|
||||
def _http_up() -> bool:
|
||||
@@ -128,8 +123,8 @@ def stop() -> dict:
|
||||
except Exception:
|
||||
pass
|
||||
proc.terminate()
|
||||
elif os.name != "nt":
|
||||
os.kill(pid, 15)
|
||||
else:
|
||||
proc_util.terminate_pid(pid)
|
||||
except Exception:
|
||||
pass
|
||||
PID_FILE.unlink(missing_ok=True)
|
||||
|
||||
@@ -33,14 +33,18 @@ RESOURCE_ROOT = PROJECT_ROOT if SOURCE_CHECKOUT else PACKAGE_DIR / "_resources"
|
||||
|
||||
|
||||
def _user_dir(env_name: str, windows_leaf: str, xdg_name: str, xdg_fallback: str) -> Path:
|
||||
# os.getenv's default only applies when a variable is *unset*. An exported
|
||||
# but empty XDG_DATA_HOME / LOCALAPPDATA would otherwise give Path("") ==
|
||||
# ".", scattering state through whatever the cwd happened to be. The XDG
|
||||
# spec says to treat an empty value as unset, so `or` - not a default arg.
|
||||
override = os.getenv(env_name, "").strip()
|
||||
if override:
|
||||
return Path(override).expanduser().resolve()
|
||||
if os.name == "nt":
|
||||
base = Path(os.getenv("LOCALAPPDATA", Path.home() / "AppData" / "Local"))
|
||||
return base / windows_leaf
|
||||
base = Path(os.getenv(xdg_name, Path.home() / xdg_fallback)).expanduser()
|
||||
return base / "nexusos"
|
||||
base = Path(os.getenv("LOCALAPPDATA", "").strip() or Path.home() / "AppData" / "Local")
|
||||
return (base / windows_leaf).expanduser().resolve()
|
||||
base = Path(os.getenv(xdg_name, "").strip() or Path.home() / xdg_fallback).expanduser()
|
||||
return (base / "nexusos").resolve()
|
||||
|
||||
|
||||
CONFIG_DIR = _user_dir("NEXUS_CONFIG_DIR", "NexusOS", "XDG_CONFIG_HOME", ".config")
|
||||
|
||||
@@ -0,0 +1,212 @@
|
||||
"""Process inspection and termination that works with or without psutil.
|
||||
|
||||
psutil became an optional extra when the wheel landed, so the base install
|
||||
(notably `pip install nexusos-ai` on Windows) has to manage PIDs with the
|
||||
stdlib alone. Callers should keep using psutil when it is importable - it is
|
||||
faster and more precise - and fall back here when it is not.
|
||||
|
||||
Windows has no signals: os.kill(pid, sig) special-cases CTRL_C_EVENT and
|
||||
CTRL_BREAK_EVENT, treats sig 0 as an existence check, and calls
|
||||
TerminateProcess(handle, sig) for *everything else*. So os.kill(pid, 15) is not
|
||||
a polite request there - it is an immediate, unblockable kill with exit code
|
||||
15, and there is no equivalent of SIGTERM. Everything below goes through the
|
||||
Win32 API via ctypes so the intent is explicit at each call site rather than
|
||||
resting on which signal numbers happen to be special.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import signal
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
WINDOWS = os.name == "nt"
|
||||
|
||||
# Win32 constants (winnt.h / processthreadsapi.h)
|
||||
_SYNCHRONIZE = 0x00100000
|
||||
_PROCESS_TERMINATE = 0x0001
|
||||
_WAIT_TIMEOUT = 0x00000102
|
||||
|
||||
# Keep console windows from flashing on every helper subprocess.
|
||||
_NO_WINDOW = subprocess.CREATE_NO_WINDOW if WINDOWS else 0
|
||||
|
||||
|
||||
def _kernel32():
|
||||
import ctypes
|
||||
|
||||
return ctypes.WinDLL("kernel32", use_last_error=True)
|
||||
|
||||
|
||||
def _run(argv: list[str], timeout: float = 10.0) -> str:
|
||||
"""Run a helper command, returning stdout ('' on any failure)."""
|
||||
try:
|
||||
done = subprocess.run(
|
||||
argv, capture_output=True, text=True, timeout=timeout,
|
||||
creationflags=_NO_WINDOW,
|
||||
)
|
||||
except (OSError, subprocess.SubprocessError):
|
||||
return ""
|
||||
return done.stdout or ""
|
||||
|
||||
|
||||
def pid_alive(pid: int | None) -> bool:
|
||||
"""True if the PID names a live process.
|
||||
|
||||
On Windows this opens a handle and polls it; a signalled handle means the
|
||||
process has exited. That is equivalent to os.kill(pid, 0) - CPython
|
||||
special-cases signal 0 into an existence check there - but it also reports
|
||||
True for a process owned by another user, where os.kill raises.
|
||||
"""
|
||||
if pid is None:
|
||||
return False
|
||||
try:
|
||||
pid = int(pid)
|
||||
except (TypeError, ValueError):
|
||||
return False
|
||||
if pid <= 0:
|
||||
return False
|
||||
if WINDOWS:
|
||||
import ctypes
|
||||
|
||||
k = _kernel32()
|
||||
handle = k.OpenProcess(_SYNCHRONIZE, False, pid)
|
||||
if not handle:
|
||||
return False
|
||||
try:
|
||||
return k.WaitForSingleObject(ctypes.c_void_p(handle), 0) == _WAIT_TIMEOUT
|
||||
finally:
|
||||
k.CloseHandle(ctypes.c_void_p(handle))
|
||||
try:
|
||||
os.kill(pid, 0)
|
||||
return True
|
||||
except ProcessLookupError:
|
||||
return False
|
||||
except PermissionError:
|
||||
return True # exists, owned by someone else
|
||||
except (OSError, ValueError):
|
||||
return False
|
||||
|
||||
|
||||
def terminate_pid(pid: int | None, force: bool = False) -> bool:
|
||||
"""Ask a process to exit. Returns True if the request was delivered."""
|
||||
if not pid_alive(pid):
|
||||
return False
|
||||
pid = int(pid)
|
||||
if WINDOWS:
|
||||
# No graceful path without a shared console; TerminateProcess is what
|
||||
# psutil.terminate() resolves to on Windows anyway.
|
||||
import ctypes
|
||||
|
||||
k = _kernel32()
|
||||
handle = k.OpenProcess(_PROCESS_TERMINATE, False, pid)
|
||||
if not handle:
|
||||
return False
|
||||
try:
|
||||
return bool(k.TerminateProcess(ctypes.c_void_p(handle), 1))
|
||||
finally:
|
||||
k.CloseHandle(ctypes.c_void_p(handle))
|
||||
try:
|
||||
os.kill(pid, signal.SIGKILL if force else signal.SIGTERM)
|
||||
return True
|
||||
except OSError:
|
||||
return False
|
||||
|
||||
|
||||
def pid_cmdline(pid: int | None) -> str:
|
||||
"""Full command line for a PID, or '' when it cannot be determined."""
|
||||
if pid is None:
|
||||
return ""
|
||||
try:
|
||||
pid = int(pid)
|
||||
except (TypeError, ValueError):
|
||||
return ""
|
||||
if WINDOWS:
|
||||
for entry_pid, cmd in iter_processes():
|
||||
if entry_pid == pid:
|
||||
return cmd
|
||||
return ""
|
||||
proc = Path(f"/proc/{pid}/cmdline")
|
||||
try:
|
||||
return proc.read_bytes().replace(b"\0", b" ").decode(errors="replace").strip()
|
||||
except OSError:
|
||||
pass
|
||||
# macOS and other POSIX hosts without /proc.
|
||||
out = _run(["ps", "-o", "command=", "-p", str(pid)])
|
||||
return out.strip()
|
||||
|
||||
|
||||
def iter_processes() -> list[tuple[int, str]]:
|
||||
"""(pid, command_line) for every visible process.
|
||||
|
||||
Windows needs CIM for command lines - the ctypes snapshot APIs only expose
|
||||
image names, which is not enough to tell `uvicorn synapse.main` apart from
|
||||
any other python.exe. This is slow, so it is strictly the no-psutil path.
|
||||
"""
|
||||
if WINDOWS:
|
||||
out = _run([
|
||||
"powershell", "-NoProfile", "-NonInteractive", "-Command",
|
||||
"Get-CimInstance Win32_Process | "
|
||||
"ForEach-Object { \"$($_.ProcessId)`t$($_.CommandLine)\" }",
|
||||
], timeout=30.0)
|
||||
entries: list[tuple[int, str]] = []
|
||||
for line in out.splitlines():
|
||||
head, _, cmd = line.partition("\t")
|
||||
if head.strip().isdigit():
|
||||
entries.append((int(head), cmd.strip()))
|
||||
return entries
|
||||
|
||||
entries = []
|
||||
proc_root = Path("/proc")
|
||||
if proc_root.is_dir():
|
||||
for entry in proc_root.iterdir():
|
||||
if not entry.name.isdigit():
|
||||
continue
|
||||
try:
|
||||
cmd = (entry / "cmdline").read_bytes()
|
||||
except OSError:
|
||||
continue
|
||||
entries.append((int(entry.name), cmd.replace(b"\0", b" ").decode(errors="replace").strip()))
|
||||
return entries
|
||||
|
||||
for line in _run(["ps", "-A", "-o", "pid=,command="]).splitlines():
|
||||
head, _, cmd = line.strip().partition(" ")
|
||||
if head.isdigit():
|
||||
entries.append((int(head), cmd.strip()))
|
||||
return entries
|
||||
|
||||
|
||||
def pids_listening_on(port: int) -> list[int]:
|
||||
"""PIDs holding a listening TCP socket on `port`."""
|
||||
pids: list[int] = []
|
||||
if WINDOWS:
|
||||
for line in _run(["netstat", "-ano", "-p", "TCP"]).splitlines():
|
||||
parts = line.split()
|
||||
# Proto Local Foreign State PID
|
||||
if len(parts) < 5 or parts[3] != "LISTENING":
|
||||
continue
|
||||
local = parts[1]
|
||||
if local.rsplit(":", 1)[-1] == str(port) and parts[4].isdigit():
|
||||
pids.append(int(parts[4]))
|
||||
return sorted(set(pids))
|
||||
|
||||
# -t TCP, -l listening, -n numeric, -P no port names.
|
||||
for line in _run(["lsof", "-nP", "-tiTCP:%d" % port, "-sTCP:LISTEN"]).splitlines():
|
||||
if line.strip().isdigit():
|
||||
pids.append(int(line.strip()))
|
||||
if pids:
|
||||
return sorted(set(pids))
|
||||
|
||||
for line in _run(["ss", "-lptnH", "sport = :%d" % port]).splitlines():
|
||||
# ... users:(("uvicorn",pid=1234,fd=3))
|
||||
marker = "pid="
|
||||
start = line.find(marker)
|
||||
while start != -1:
|
||||
digits = ""
|
||||
for ch in line[start + len(marker):]:
|
||||
if not ch.isdigit():
|
||||
break
|
||||
digits += ch
|
||||
if digits:
|
||||
pids.append(int(digits))
|
||||
start = line.find(marker, start + 1)
|
||||
return sorted(set(pids))
|
||||
@@ -18,7 +18,7 @@ def run_cli(tmp_path: Path, *args: str) -> subprocess.CompletedProcess[str]:
|
||||
env["NEXUS_HOME"] = str(tmp_path / "state")
|
||||
env["NEXUS_CONFIG_DIR"] = str(tmp_path / "config")
|
||||
return subprocess.run(
|
||||
[sys.executable, "-m", "management.cli", *args],
|
||||
[sys.executable, "-m", "nexusos_cli.cli", *args],
|
||||
cwd=ROOT,
|
||||
env=env,
|
||||
text=True,
|
||||
@@ -31,17 +31,22 @@ def run_cli(tmp_path: Path, *args: str) -> subprocess.CompletedProcess[str]:
|
||||
def test_help_exposes_portable_command_tree(tmp_path):
|
||||
result = run_cli(tmp_path, "--help")
|
||||
assert result.returncode == 0, result.stderr
|
||||
for command in ("init", "config", "provider", "doctor", "serve", "models", "chat"):
|
||||
for command in ("init", "config", "provider", "doctor", "serve", "models", "chat", "monitor"):
|
||||
assert command in result.stdout
|
||||
|
||||
|
||||
def test_legacy_cli_spellings_remain_compatible():
|
||||
from management.cli import _normalize_legacy_argv
|
||||
from nexusos_cli.cli import _normalize_legacy_argv
|
||||
|
||||
assert _normalize_legacy_argv(["start", "-b"]) == ["start", "backend"]
|
||||
assert _normalize_legacy_argv(["stop", "--ai"]) == ["stop", "ai"]
|
||||
assert _normalize_legacy_argv(["logs", "-m", "--follow"]) == ["logs", "memory", "--follow"]
|
||||
assert _normalize_legacy_argv(["backup", "full"]) == ["backup", "--full"]
|
||||
# -f is --follow for `logs`, but --frontend for start/stop. Translating it
|
||||
# for logs turned `logs -f` into a one-shot tail of the frontend log.
|
||||
assert _normalize_legacy_argv(["logs", "-f"]) == ["logs", "-f"]
|
||||
assert _normalize_legacy_argv(["logs", "-m", "-f"]) == ["logs", "memory", "-f"]
|
||||
assert _normalize_legacy_argv(["start", "-f"]) == ["start", "frontend"]
|
||||
assert _normalize_legacy_argv(["restore", "-f"]) == ["restore"]
|
||||
assert _normalize_legacy_argv(["help"]) == ["--help"]
|
||||
|
||||
@@ -92,7 +97,7 @@ def test_serve_rejects_invalid_ports_before_startup(tmp_path):
|
||||
|
||||
|
||||
def test_remote_provider_is_never_stopped_or_force_killed(monkeypatch):
|
||||
from management import ncp
|
||||
from nexusos_cli import ncp
|
||||
|
||||
killed_ports = []
|
||||
killed_patterns = []
|
||||
@@ -108,3 +113,58 @@ def test_remote_provider_is_never_stopped_or_force_killed(monkeypatch):
|
||||
|
||||
assert 11434 not in killed_ports
|
||||
assert "ollama serve" not in killed_patterns
|
||||
|
||||
|
||||
def test_allow_lan_names_addresses_instead_of_disabling_the_host_check():
|
||||
"""ALLOWED_HOSTS=* would switch TrustedHostMiddleware off entirely, and that
|
||||
middleware is the DNS-rebinding defense for an unauthenticated API."""
|
||||
from nexusos_cli.cli import _lan_hostnames
|
||||
|
||||
assert _lan_hostnames("192.168.1.20") == ["192.168.1.20"]
|
||||
wildcard = _lan_hostnames("0.0.0.0")
|
||||
assert wildcard, "a wildcard bind must resolve to concrete host names"
|
||||
assert "*" not in wildcard
|
||||
# IPv6 literals need to match a Host header written either way.
|
||||
for name in wildcard:
|
||||
if ":" in name and not name.startswith("["):
|
||||
assert f"[{name}]" in wildcard
|
||||
|
||||
|
||||
def test_empty_platform_dir_variables_do_not_put_state_in_the_cwd(tmp_path, monkeypatch):
|
||||
"""os.getenv's default only fires when a variable is *unset*; an exported
|
||||
but empty XDG_DATA_HOME / LOCALAPPDATA made Path("") == "." the base, so
|
||||
state landed in whatever directory the command happened to run from.
|
||||
|
||||
Patching os.name to exercise the other platform's branch is not an option -
|
||||
pathlib dispatches on it - so this checks the branch this host actually
|
||||
takes."""
|
||||
from synapse import nexus_config
|
||||
|
||||
if os.name == "nt":
|
||||
monkeypatch.setenv("LOCALAPPDATA", "")
|
||||
expected = Path.home() / "AppData" / "Local" / "NexusOS"
|
||||
else:
|
||||
monkeypatch.setenv("XDG_DATA_HOME", "")
|
||||
expected = Path.home() / ".local/share" / "nexusos"
|
||||
monkeypatch.delenv("NEXUS_HOME", raising=False)
|
||||
monkeypatch.chdir(tmp_path)
|
||||
|
||||
resolved = nexus_config._user_dir("NEXUS_HOME", "NexusOS", "XDG_DATA_HOME", ".local/share")
|
||||
assert resolved.is_absolute()
|
||||
assert tmp_path.resolve() != resolved.parent
|
||||
assert resolved == expected.resolve()
|
||||
|
||||
|
||||
def test_config_set_warns_before_orphaning_the_database(tmp_path):
|
||||
"""Repointing the DB does not move it - say so, or history looks deleted."""
|
||||
assert run_cli(tmp_path, "init").returncode == 0
|
||||
db = tmp_path / "state" / "data" / "memory.db"
|
||||
db.parent.mkdir(parents=True, exist_ok=True)
|
||||
db.write_bytes(b"SQLite format 3" + bytes(1))
|
||||
|
||||
result = run_cli(tmp_path, "config", "set", "memory_db", str(tmp_path / "elsewhere.db"), "--json")
|
||||
assert result.returncode == 0, result.stderr
|
||||
assert "warning" in json.loads(result.stdout)
|
||||
|
||||
same = run_cli(tmp_path, "config", "set", "memory_db", str(db), "--json")
|
||||
assert "warning" not in json.loads(same.stdout)
|
||||
|
||||
@@ -0,0 +1,78 @@
|
||||
"""Tests for the ASCII monitor — renderer + collectors, no live stack required."""
|
||||
from __future__ import annotations
|
||||
|
||||
from nexusos_cli.monitor import _bar, _fmt_bytes, _recent_tools, render_frame
|
||||
|
||||
|
||||
def test_bar_bounds():
|
||||
assert _bar(0, 10, unicode=False) == "-" * 10
|
||||
assert _bar(1, 10, unicode=False) == "#" * 10
|
||||
assert _bar(0.5, 10, unicode=False).count("#") == 5
|
||||
|
||||
|
||||
def test_fmt_bytes():
|
||||
assert _fmt_bytes(None) == "—"
|
||||
assert _fmt_bytes(512) == "512B"
|
||||
assert _fmt_bytes(2048).endswith("K")
|
||||
|
||||
|
||||
def test_render_frame_contains_sections():
|
||||
snap = {
|
||||
"ts": "2026-08-20T12:00:00-05:00",
|
||||
"version": "0.0.0",
|
||||
"services": {
|
||||
"backend": {"running": True, "pid": 11, "url": "http://127.0.0.1:8000"},
|
||||
"memory": {"running": False, "pid": None, "url": "http://127.0.0.1:8001"},
|
||||
"frontend": {"running": False, "pid": None, "url": "http://127.0.0.1:5173"},
|
||||
"provider": {
|
||||
"provider": "ollama",
|
||||
"url": "http://127.0.0.1:11434",
|
||||
"reachable": True,
|
||||
},
|
||||
},
|
||||
"api": {
|
||||
"online": True,
|
||||
"version": "0.0.0",
|
||||
"ollama": "running",
|
||||
"memories": 3,
|
||||
"conversations": 2,
|
||||
"playbooks": 1,
|
||||
"models": 4,
|
||||
"action_tool_policy": "ask",
|
||||
},
|
||||
"host": {"cpu_pct": 12.5, "mem_used": 1_000_000_000, "mem_total": 8_000_000_000, "mem_pct": 12.5},
|
||||
"procs": {"cpu_pct": 1.0, "rss": 50_000_000, "pids": [11]},
|
||||
"toolchains": [
|
||||
{"lang": "python", "ready": True, "tool": "/usr/bin/python", "summary": "python"},
|
||||
{"lang": "rust", "ready": False, "tool": None, "summary": "rust"},
|
||||
],
|
||||
"recent_tools": ["run_snippet", "render_preview"],
|
||||
"paths": {},
|
||||
}
|
||||
frame = render_frame(snap, width=72, unicode=False)
|
||||
assert "SERVICES" in frame
|
||||
assert "RESOURCES" in frame
|
||||
assert "DATA / TOOLS" in frame
|
||||
assert "RUN TOOLCHAINS" in frame
|
||||
assert "backend" in frame and "UP" in frame
|
||||
assert "memory" in frame and "DOWN" in frame
|
||||
assert "run_snippet" in frame
|
||||
assert "ready python" in frame
|
||||
assert "missing rust" in frame
|
||||
# Fixed-width box: every content line same length.
|
||||
lengths = {len(line) for line in frame.splitlines()}
|
||||
assert len(lengths) == 1
|
||||
|
||||
|
||||
def test_recent_tools_parses_status_sentinels(tmp_path):
|
||||
log = tmp_path / "chat.log"
|
||||
log.write_text(
|
||||
"noise\n__status__tools\n__status__run_snippet\n"
|
||||
'payload {"name": "render_preview"}\n__status__remember\n',
|
||||
encoding="utf-8",
|
||||
)
|
||||
found = _recent_tools(log, limit=5)
|
||||
assert "run_snippet" in found
|
||||
assert "remember" in found
|
||||
assert "render_preview" in found
|
||||
assert "tools" not in found
|
||||
@@ -1,10 +1,5 @@
|
||||
"""Pin the SSE parser in nexus_api. Run: python management/test_nexus_api.py"""
|
||||
import sys, os
|
||||
sys.path.insert(0, os.path.dirname(__file__))
|
||||
try:
|
||||
from management.nexus_api import iter_chunks
|
||||
except ModuleNotFoundError: # direct: python management/test_nexus_api.py
|
||||
from nexus_api import iter_chunks
|
||||
"""Pin the SSE parser in nexus_api. Run: python tests/test_nexus_api.py"""
|
||||
from nexusos_cli.nexus_api import iter_chunks
|
||||
|
||||
# A realistic /chat/stream frame: two token chunks, a meta block, then done.
|
||||
lines = [
|
||||
@@ -0,0 +1,144 @@
|
||||
"""Guard the wheel's dependency list against drift.
|
||||
|
||||
There are now two dependency declarations: requirements-base.txt (what the
|
||||
desktop installers pip -r) and pyproject.toml (what the wheel ships). They will
|
||||
drift. What actually breaks a user is narrower than "they differ", though: it
|
||||
is an import that no declared distribution provides, so that is what this pins.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import ast
|
||||
import sys
|
||||
import tomllib
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
SHIPPED_PACKAGES = ("synapse", "nexusos_cli")
|
||||
|
||||
# Import name -> distribution name, where PyPI disagrees with the module.
|
||||
DISTRIBUTION_OF = {
|
||||
"docx": "python-docx",
|
||||
"dotenv": "python-dotenv",
|
||||
"faster_whisper": "faster-whisper",
|
||||
"imap_tools": "imap-tools",
|
||||
"sqlite_vec": "sqlite-vec",
|
||||
"yaml": "pyyaml",
|
||||
"PIL": "pillow",
|
||||
}
|
||||
|
||||
# Provided by another declared distribution rather than named directly.
|
||||
TRANSITIVE = {"starlette", "socketio", "engineio"}
|
||||
|
||||
# Modules that ship inside this repo.
|
||||
FIRST_PARTY = {"synapse", "nexusos_cli", "management", "bin", "tests"}
|
||||
|
||||
|
||||
def _pyproject() -> dict:
|
||||
return tomllib.loads((ROOT / "pyproject.toml").read_text(encoding="utf-8"))
|
||||
|
||||
|
||||
def _requirement_name(spec: str) -> str:
|
||||
"""'pypdf>=5,<7' -> 'pypdf'; strips extras and environment markers."""
|
||||
head = spec.split(";", 1)[0].strip()
|
||||
for sep in ("[", "=", ">", "<", "!", "~", " "):
|
||||
head = head.split(sep, 1)[0]
|
||||
return head.strip().lower().replace("_", "-")
|
||||
|
||||
|
||||
def _declared() -> set[str]:
|
||||
project = _pyproject()["project"]
|
||||
specs = list(project.get("dependencies", []))
|
||||
for extra in project.get("optional-dependencies", {}).values():
|
||||
specs.extend(extra)
|
||||
return {_requirement_name(s) for s in specs}
|
||||
|
||||
|
||||
def _imported_modules() -> set[str]:
|
||||
"""Top-level module names imported anywhere in the shipped packages."""
|
||||
found: set[str] = set()
|
||||
for package in SHIPPED_PACKAGES:
|
||||
for path in (ROOT / package).rglob("*.py"):
|
||||
tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path))
|
||||
for node in ast.walk(tree):
|
||||
if isinstance(node, ast.Import):
|
||||
found.update(alias.name.split(".")[0] for alias in node.names)
|
||||
elif isinstance(node, ast.ImportFrom):
|
||||
# level > 0 is a relative (first-party) import.
|
||||
if node.level == 0 and node.module:
|
||||
found.add(node.module.split(".")[0])
|
||||
return found
|
||||
|
||||
|
||||
def _third_party() -> set[str]:
|
||||
return {
|
||||
module for module in _imported_modules()
|
||||
if module not in sys.stdlib_module_names
|
||||
and module not in FIRST_PARTY
|
||||
and module not in TRANSITIVE
|
||||
and not module.startswith("_")
|
||||
}
|
||||
|
||||
|
||||
def test_every_third_party_import_is_a_declared_dependency():
|
||||
declared = _declared()
|
||||
missing = sorted(
|
||||
module for module in _third_party()
|
||||
if DISTRIBUTION_OF.get(module, module).lower().replace("_", "-") not in declared
|
||||
)
|
||||
assert not missing, (
|
||||
"synapse/nexusos_cli import these, but pyproject.toml declares no "
|
||||
f"distribution for them: {missing}. Add them to [project] dependencies "
|
||||
"or an extra (and to DISTRIBUTION_OF here if the names differ)."
|
||||
)
|
||||
|
||||
|
||||
def test_all_extra_is_the_union_of_the_capability_extras():
|
||||
extras = _pyproject()["project"]["optional-dependencies"]
|
||||
combined: set[str] = set()
|
||||
for name, specs in extras.items():
|
||||
if name in ("all", "dev", "standard"):
|
||||
continue
|
||||
combined.update(_requirement_name(s) for s in specs)
|
||||
everything = {_requirement_name(s) for s in extras["all"]}
|
||||
assert combined == everything, (
|
||||
"the 'all' extra drifted from the capability extras; "
|
||||
f"missing={sorted(combined - everything)} extra={sorted(everything - combined)}"
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("name", ["fastapi", "uvicorn", "httpx", "pydantic", "pyyaml"])
|
||||
def test_core_runtime_is_a_hard_dependency_not_an_extra(name):
|
||||
"""These are imported at module scope, so the base install must carry them."""
|
||||
base = {_requirement_name(s) for s in _pyproject()["project"]["dependencies"]}
|
||||
assert name in base
|
||||
|
||||
|
||||
def test_optional_imports_are_lazy():
|
||||
"""Anything only in an extra must not be imported at module scope.
|
||||
|
||||
A base `pip install nexusos-ai` has none of the extras, so a top-level
|
||||
`import psutil` in synapse would make the backend unimportable.
|
||||
"""
|
||||
base = {_requirement_name(s) for s in _pyproject()["project"]["dependencies"]}
|
||||
offenders: list[str] = []
|
||||
for package in SHIPPED_PACKAGES:
|
||||
for path in (ROOT / package).rglob("*.py"):
|
||||
tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path))
|
||||
for node in tree.body: # module scope only
|
||||
names: list[str] = []
|
||||
if isinstance(node, ast.Import):
|
||||
names = [a.name.split(".")[0] for a in node.names]
|
||||
elif isinstance(node, ast.ImportFrom) and node.level == 0 and node.module:
|
||||
names = [node.module.split(".")[0]]
|
||||
for module in names:
|
||||
if module in sys.stdlib_module_names or module in FIRST_PARTY:
|
||||
continue
|
||||
dist = DISTRIBUTION_OF.get(module, module).lower().replace("_", "-")
|
||||
if dist not in base and module not in TRANSITIVE:
|
||||
offenders.append(f"{path.relative_to(ROOT)}: {module}")
|
||||
assert not offenders, (
|
||||
"optional dependencies imported at module scope (wrap in try/ImportError "
|
||||
f"or import inside the function): {offenders}"
|
||||
)
|
||||
@@ -0,0 +1,94 @@
|
||||
"""Pin the psutil-free process helpers.
|
||||
|
||||
psutil is an optional extra, so on a base install these are the only way
|
||||
`ncp stop` / `nexus status` can see or stop a service. Windows previously had
|
||||
no fallback at all: pid_is_ours returned False, kill_port returned False, and
|
||||
the terminate branch was POSIX-only, so stop was a no-op there.
|
||||
|
||||
Probing must also stay side-effect free. That is easy to get wrong on Windows,
|
||||
where os.kill(pid, sig) is TerminateProcess for every sig except 0 and the two
|
||||
console-control events - os.kill(pid, 15) kills instead of asking.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
|
||||
import pytest
|
||||
|
||||
from synapse import proc_util
|
||||
|
||||
|
||||
def _spawn():
|
||||
return subprocess.Popen(
|
||||
[sys.executable, "-c", "import time; time.sleep(30)"],
|
||||
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
|
||||
)
|
||||
|
||||
|
||||
def _wait_gone(proc, timeout=10.0) -> bool:
|
||||
deadline = time.monotonic() + timeout
|
||||
while time.monotonic() < deadline:
|
||||
if proc.poll() is not None:
|
||||
return True
|
||||
time.sleep(0.05)
|
||||
return False
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def victim():
|
||||
proc = _spawn()
|
||||
try:
|
||||
yield proc
|
||||
finally:
|
||||
if proc.poll() is None:
|
||||
proc.kill()
|
||||
proc.wait(timeout=10)
|
||||
|
||||
|
||||
def test_pid_alive_does_not_kill_the_process(victim):
|
||||
"""Probing must be side-effect free, however it is implemented."""
|
||||
for _ in range(5):
|
||||
assert proc_util.pid_alive(victim.pid) is True
|
||||
time.sleep(0.3)
|
||||
assert victim.poll() is None, "pid_alive() terminated the process it probed"
|
||||
|
||||
|
||||
def test_pid_alive_is_false_for_a_dead_pid(victim):
|
||||
victim.kill()
|
||||
victim.wait(timeout=10)
|
||||
assert proc_util.pid_alive(victim.pid) is False
|
||||
|
||||
|
||||
@pytest.mark.parametrize("pid", [None, 0, -1, "not-a-pid"])
|
||||
def test_pid_alive_rejects_junk(pid):
|
||||
assert proc_util.pid_alive(pid) is False
|
||||
|
||||
|
||||
def test_terminate_pid_actually_stops_it(victim):
|
||||
assert proc_util.terminate_pid(victim.pid) is True
|
||||
assert _wait_gone(victim), "terminate_pid() did not stop the process"
|
||||
assert proc_util.pid_alive(victim.pid) is False
|
||||
|
||||
|
||||
def test_terminate_pid_is_false_when_already_gone(victim):
|
||||
victim.kill()
|
||||
victim.wait(timeout=10)
|
||||
assert proc_util.terminate_pid(victim.pid) is False
|
||||
|
||||
|
||||
def test_pid_cmdline_identifies_the_process(victim):
|
||||
cmd = proc_util.pid_cmdline(victim.pid)
|
||||
if not cmd:
|
||||
pytest.skip("no command-line source on this host")
|
||||
assert "time.sleep" in cmd or "python" in cmd.lower()
|
||||
|
||||
|
||||
def test_iter_processes_includes_this_interpreter():
|
||||
import os
|
||||
|
||||
entries = proc_util.iter_processes()
|
||||
if not entries:
|
||||
pytest.skip("no process enumeration available on this host")
|
||||
assert os.getpid() in {pid for pid, _ in entries}
|
||||
Reference in New Issue
Block a user
GitNexus · part of the Nexus stack