feat(packaging): move the CLI into nexusos_cli and make the wheel self-sufficient

The CLI shipped from `management/`, which also holds desktop-only pieces (the
Tk control panel, the XFCE panel wiring, the shell wrappers). Packaging that
directory meant the wheel either dragged in tkinter or shipped a broken import.
Split it: `nexusos_cli/` is what the wheel ships and what `nexus`/`ncp`/
`nexusos` dispatch to, `management/` keeps the desktop half.

Alongside the move:

* hatch_build.py decides the interface/web/dist include at build time. dist/
  is gitignored, so a static force-include aborts `pip install -e .` on a
  fresh clone - before the reader reaches the `npm run build` step. Editable
  installs now skip a missing dist; wheels and sdists hard-error naming the
  command to run.
* synapse/proc_util.py gives frontend_manager and ncp process inspection and
  termination without psutil, which became an optional extra when the wheel
  landed. It routes around Windows having no signals, where os.kill(pid, 15)
  is an unblockable TerminateProcess rather than a polite request.
* nexusos_cli/monitor.py adds `ncp monitor`, an ASCII dashboard with no curses
  or rich dependency so it works in Termux, plain SSH and Windows Terminal.
  Collector and renderer are separate so tests feed fixtures, no stack needed.
* tests/test_packaging_deps.py fails the gate when synapse or nexusos_cli
  import a distribution pyproject does not declare, and when an optional
  dependency is imported at module scope instead of lazily.
* bin/check.sh now builds the wheel, twine-checks it, and asserts the compiled
  UI and seed playbooks are actually inside it. A wheel that builds but ships
  no dist/ serves a blank page, which only shows up after release.

tests/test_nexus_api.py moves to tests/ with the module it covers.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Athena Kaminsky
2026-08-26 08:11:39 -05:00
committed by Athena
co-authored by Claude Opus 5
parent d579502a5b
commit 9104c724c4
25 changed files with 1412 additions and 76 deletions
+7 -1
View File
@@ -37,7 +37,7 @@ jobs:
run: | run: |
python -m pip install -e ".[dev]" python -m pip install -e ".[dev]"
python -m pytest -q tests management python -m pytest -q tests management
bash -n scripts/install-termux.sh for f in scripts/install-termux.sh bin/check.sh management/nexus-cli.sh; do bash -n "$f"; done
- name: Build wheel and sdist - name: Build wheel and sdist
run: | run: |
@@ -54,6 +54,12 @@ jobs:
"$RUNNER_TEMP/nexus-wheel/bin/nexus" init --json "$RUNNER_TEMP/nexus-wheel/bin/nexus" init --json
"$RUNNER_TEMP/nexus-wheel/bin/nexus" doctor --json "$RUNNER_TEMP/nexus-wheel/bin/nexus" doctor --json
"$RUNNER_TEMP/nexus-wheel/bin/python" -c "from synapse.main import sio_app; assert sio_app" "$RUNNER_TEMP/nexus-wheel/bin/python" -c "from synapse.main import sio_app; assert sio_app"
# The wheel must carry the compiled UI, not just import cleanly.
"$RUNNER_TEMP/nexus-wheel/bin/python" - <<'PY'
from synapse.nexus_config import settings
index = settings.web_dist_dir / "index.html"
assert index.is_file(), f"wheel shipped no web UI at {index}"
PY
- uses: actions/upload-artifact@v4 - uses: actions/upload-artifact@v4
with: with:
+5 -2
View File
@@ -57,7 +57,7 @@ access to the same features as the web UI (all via the REST API on `:8000`):
./management/nexus-cli.sh stop ./management/nexus-cli.sh stop
./management/nexus-cli.sh start --backend|-b / --frontend|-f / --memory|-m ./management/nexus-cli.sh start --backend|-b / --frontend|-f / --memory|-m
# Feature commands (dispatch to management/nexus_api.py — httpx, no TUI): # Feature commands (dispatch to nexusos_cli/nexus_api.py — httpx, no TUI):
ncp chat "<message>" # stream a reply (POST /chat/stream) ncp chat "<message>" # stream a reply (POST /chat/stream)
ncp memory list|add <text>|rm <id> ncp memory list|add <text>|rm <id>
ncp playbook list|show <id> # first playbook (*) is the active system prompt ncp playbook list|show <id> # first playbook (*) is the active system prompt
@@ -66,12 +66,15 @@ ncp history [query] # recent conversations
The old curses TUIs (`nexus-chat.py`, `nexus-playbook.py`) were removed in favor of The old curses TUIs (`nexus-chat.py`, `nexus-playbook.py`) were removed in favor of
these API-backed subcommands. The CLI covers chat, memory, playbooks, and history; these API-backed subcommands. The CLI covers chat, memory, playbooks, and history;
the web UI and control panel expose the remaining management features. the web UI and control panel expose the remaining management features.
The CLI itself lives in `nexusos_cli/` (that is what the wheel ships and what
`nexus`/`ncp`/`nexusos` dispatch to); `management/` keeps the desktop-only
pieces — the shell wrappers, the Tk control panel, and the XFCE panel wiring.
`management/controlpanel.py` (tkinter GUI, wired into the XFCE panel via `management/controlpanel.py` (tkinter GUI, wired into the XFCE panel via
`bin/panel/nexus-popup.py`) stays. `bin/panel/nexus-popup.py`) stays.
**Checks (the release gate):** **Checks (the release gate):**
```bash ```bash
./bin/check.sh # pytest (tests/ + management/) + eslint + .ps1 parse check ./bin/check.sh # pytest + eslint + frontend tests + .ps1/.sh parse + wheel build
``` ```
There is no hosted CI — the remote is self-hosted Gitea with no act_runner — so There is no hosted CI — the remote is self-hosted Gitea with no act_runner — so
this script *is* the gate. Run it before tagging a release. this script *is* the gate. Run it before tagging a release.
+3 -1
View File
@@ -98,6 +98,7 @@ The portable package installs `nexus`, `ncp`, and `nexusos` as equivalent
commands. From a checkout today: commands. From a checkout today:
```bash ```bash
cd interface/web && npm ci && npm run build && cd ../.. # compile the UI
python -m pip install -e ".[standard]" python -m pip install -e ".[standard]"
nexus init nexus init
nexus doctor nexus doctor
@@ -260,7 +261,8 @@ are the exception (YAML files in `data/playbooks/`). All paths are defined in
- `synapse/` — FastAPI backend + memory curator + playbook/ollama managers - `synapse/` — FastAPI backend + memory curator + playbook/ollama managers
- `modules/` — auto-discovered feature plugins - `modules/` — auto-discovered feature plugins
- `interface/web/` — React + Vite frontend - `interface/web/` — React + Vite frontend
- `management/` — nexus-cli.sh, ncp API client, control panel, desktop theme - `nexusos_cli/` — the portable CLI the wheel ships (`nexus`/`ncp`/`nexusos`)
- `management/` — nexus-cli.sh wrapper, control panel, desktop theme
- `bin/` — install, backup/restore, panel + provisioning scripts - `bin/` — install, backup/restore, panel + provisioning scripts
- `assets/` — branding: icons, boot splash, XFCE/GTK theme - `assets/` — branding: icons, boot splash, XFCE/GTK theme
- `data/playbooks/` — active playbook YAML - `data/playbooks/` — active playbook YAML
+34
View File
@@ -39,5 +39,39 @@ else
echo "-- skipped: pwsh not installed" echo "-- skipped: pwsh not installed"
fi fi
echo "== shell parse =="
for f in scripts/install-termux.sh launch_nexus.sh management/nexus-cli.sh; do
[ -f "$f" ] && { bash -n "$f" || fail=1; }
done
echo "== packaging =="
# The wheel is the other shippable artifact, so it belongs in the same gate:
# a broken pyproject or a missing web build only shows up at build time.
if Promethean/bin/python -c "import build, twine" 2>/dev/null; then
rm -rf .build-check
if Promethean/bin/python -m build --outdir .build-check >/dev/null 2>&1; then
Promethean/bin/python -m twine check .build-check/* || fail=1
# The compiled UI has to actually be inside the wheel - a wheel that
# builds but ships no dist/ serves a blank page.
Promethean/bin/python - <<'PY' || fail=1
import glob, sys, zipfile
wheels = glob.glob(".build-check/*.whl")
if not wheels:
sys.exit("no wheel produced")
names = zipfile.ZipFile(wheels[0]).namelist()
if not any(n.startswith("synapse/_resources/web/") for n in names):
sys.exit("wheel is missing the compiled web UI (cd interface/web && npm run build)")
if not any(n.startswith("synapse/_resources/playbooks/") for n in names):
sys.exit("wheel is missing the seed playbooks")
print(f"wheel OK: {len(names)} files")
PY
else
echo "!! wheel build failed"; fail=1
fi
rm -rf .build-check
else
echo "-- skipped: build/twine missing (pip install -e '.[dev]')"
fi
[ "$fail" -eq 0 ] && echo "OK" || echo "FAILED" [ "$fail" -eq 0 ] && echo "OK" || echo "FAILED"
exit "$fail" exit "$fail"
+21 -2
View File
@@ -8,9 +8,11 @@ checkout-specific operations report a clear error when invoked from a wheel.
## Install ## Install
From a source checkout: From a source checkout. Build the web UI first - it is a Vite artifact, so a
fresh clone does not have it, and an install without it serves the API only:
```bash ```bash
cd interface/web && npm ci && npm run build && cd ../..
python -m pip install -e ".[standard]" python -m pip install -e ".[standard]"
nexus init nexus init
nexus doctor nexus doctor
@@ -27,12 +29,15 @@ nexus serve
The base install contains the backend, memory service, compiled web UI, CLI, The base install contains the backend, memory service, compiled web UI, CLI,
and seed playbooks. Extras keep platform-sensitive dependencies optional: and seed playbooks. Extras keep platform-sensitive dependencies optional:
- `standard`: documents, vector search, and process control - `standard`: documents, vector search, web search, and process control
- `documents`: PDF and DOCX ingestion - `documents`: PDF and DOCX ingestion
- `vector`: sqlite-vec semantic indexes - `vector`: sqlite-vec semantic indexes
- `voice`: local faster-whisper transcription - `voice`: local faster-whisper transcription
- `process`: psutil-backed process and port inspection - `process`: psutil-backed process and port inspection
- `desktop`: desktop process support and Windows pywebview - `desktop`: desktop process support and Windows pywebview
- `search`: DuckDuckGo web search for chat
- `mail`: IMAP mail reading
- `all`: every optional capability at once
## Common commands ## Common commands
@@ -41,6 +46,7 @@ nexus init Create writable state and seed playbooks
nexus doctor [--fix] [--json] Diagnose the install and provider nexus doctor [--fix] [--json] Diagnose the install and provider
nexus paths [--json] Show package, state, and asset locations nexus paths [--json] Show package, state, and asset locations
nexus status [--json] Show services and provider reachability nexus status [--json] Show services and provider reachability
nexus monitor [--once] [--json] ASCII live dashboard (services, resources, tools)
nexus serve Run backend + memory in the foreground nexus serve Run backend + memory in the foreground
nexus start|stop|restart Manage background services nexus start|stop|restart Manage background services
nexus open Open the compiled web interface nexus open Open the compiled web interface
@@ -84,6 +90,11 @@ directory. Inspect the exact locations with `nexus paths`.
Environment variables override persisted settings. The most useful are: Environment variables override persisted settings. The most useful are:
Persisted keys are the same names, minus the `NEXUS_` prefix - `nexus config
set home /data/nexus` matches `NEXUS_HOME`. `nexus config list` shows what is
set; `nexus config set` warns when a change would point NexusOS at a database
that does not exist yet (the file is never moved for you).
```text ```text
NEXUS_HOME Override the complete writable state root NEXUS_HOME Override the complete writable state root
NEXUS_CONFIG_DIR Override the config directory NEXUS_CONFIG_DIR Override the config directory
@@ -97,3 +108,11 @@ NEXUS_MEMORY_PORT Memory service port (default 8001)
The REST APIs are unauthenticated. `nexus serve` refuses non-loopback binds The REST APIs are unauthenticated. `nexus serve` refuses non-loopback binds
unless `--allow-lan` is given; that flag is an explicit acknowledgement, not unless `--allow-lan` is given; that flag is an explicit acknowledgement, not
an authentication layer. an authentication layer.
`--allow-lan` widens the accepted `Host` headers and CORS origins to the
addresses the bind actually answers on - it does **not** set them to `*`.
That keeps `TrustedHostMiddleware` enforcing something, which is what stops a
web page you visit from resolving a name it controls to your machine and
driving the API through your browser. Export `NEXUS_ALLOWED_HOSTS` yourself if
you genuinely need a blanket, and understand that anyone who can reach the
port has full admin and data access.
+55
View File
@@ -0,0 +1,55 @@
"""Build hook that ships the compiled web UI without breaking `pip install -e .`.
interface/web/dist is gitignored - it is a Vite build artifact, not source - so
a fresh clone does not have it. A static force-include of a missing path aborts
the build, which would make the README's first step ("pip install -e .") fail
before the reader ever gets to `npm run build`.
So the include is decided here instead:
* editable install -> skip a missing dist, the UI just isn't served yet
* wheel / sdist -> hard error naming the exact command to run
Set NEXUS_ALLOW_UILESS_BUILD=1 to build a deliberately headless distribution
(API and CLI only).
"""
from __future__ import annotations
import os
from pathlib import Path
from hatchling.builders.hooks.plugin.interface import BuildHookInterface
_UI_SOURCE = Path("interface") / "web" / "dist"
_UI_TARGETS = {
"wheel": "synapse/_resources/web",
"sdist": "interface/web/dist",
}
class NexusBuildHook(BuildHookInterface):
PLUGIN_NAME = "custom"
def initialize(self, version: str, build_data: dict) -> None:
target = _UI_TARGETS.get(self.target_name)
if target is None:
return
source = Path(self.root) / _UI_SOURCE
if (source / "index.html").is_file():
build_data.setdefault("force_include", {})[str(source)] = target
return
if version == "editable" or os.getenv("NEXUS_ALLOW_UILESS_BUILD") == "1":
self.app.display_warning(
f"No compiled web UI at {_UI_SOURCE} - the backend will serve the "
"API only. Build it with: cd interface/web && npm ci && npm run build"
)
return
raise RuntimeError(
f"Cannot build a {self.target_name}: the compiled web UI is missing from "
f"{_UI_SOURCE}.\n"
"Build it first:\n"
" cd interface/web && npm ci && npm run build\n"
"Or set NEXUS_ALLOW_UILESS_BUILD=1 to ship an API/CLI-only distribution."
)
+5 -1
View File
@@ -1 +1,5 @@
"""NexusOS command-line and desktop management helpers.""" """Desktop-only NexusOS management helpers (Tk control panel, XFCE panel).
The portable CLI lives in the `nexusos_cli` package - that is what the
wheel ships and what `ncp`/`nexus`/`nexusos` dispatch to.
"""
+1 -1
View File
@@ -25,7 +25,7 @@ REM delegate to the stdlib-only bin/sync.py. Mirrors the same fallback in ncp.ps
set "PY=%ROOT%\Promethean\Scripts\python.exe" set "PY=%ROOT%\Promethean\Scripts\python.exe"
if not exist "%PY%" set "PY=python" if not exist "%PY%" set "PY=python"
pushd "%ROOT%" pushd "%ROOT%"
"%PY%" -m management.cli %* "%PY%" -m nexusos_cli.cli %*
set "NEXUS_EXIT=%ERRORLEVEL%" set "NEXUS_EXIT=%ERRORLEVEL%"
popd popd
exit /b %NEXUS_EXIT% exit /b %NEXUS_EXIT%
+2 -2
View File
@@ -1,5 +1,5 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# ncp - Linux entry point. The CLI itself is management/cli.py, which runs # ncp - Linux entry point. The CLI itself is nexusos_cli/cli.py, which runs
# unchanged on Windows too (see management/ncp.cmd); this stays a shell script # unchanged on Windows too (see management/ncp.cmd); this stays a shell script
# because ~/.bashrc, launch_nexus.sh, bin/restore-linux.sh, controlpanel.py, # because ~/.bashrc, launch_nexus.sh, bin/restore-linux.sh, controlpanel.py,
# bin/panel/nexus-popup.py and management/nexus-app.sh all invoke this path. # bin/panel/nexus-popup.py and management/nexus-app.sh all invoke this path.
@@ -14,4 +14,4 @@ PY="$NEXUS_ROOT/Promethean/bin/python3"
[ -x "$PY" ] || PY=python3 [ -x "$PY" ] || PY=python3
cd "$NEXUS_ROOT" cd "$NEXUS_ROOT"
exec "$PY" -m management.cli "$@" exec "$PY" -m nexusos_cli.cli "$@"
+7
View File
@@ -0,0 +1,7 @@
"""The portable NexusOS command line, shipped in the wheel.
Kept out of `management/` so the installed distribution does not claim a
top-level `management` package name in site-packages. `management/` stays in
the source checkout for the desktop-only pieces - the Tk control panel, the
shell wrappers, the XFCE panel and .desktop wiring.
"""
+145 -6
View File
@@ -22,6 +22,7 @@ from . import ncp as services
CONFIG_SCHEMA = { CONFIG_SCHEMA = {
"home": "path",
"api_url": "url", "api_url": "url",
"memory_url": "url", "memory_url": "url",
"bind_host": "text", "bind_host": "text",
@@ -136,6 +137,46 @@ def cmd_paths(args) -> int:
return 0 return 0
# Keys whose value decides where the SQLite database is looked up. Changing one
# does not move the file, so the next start would quietly open a fresh, empty
# database and the user's conversations and memories would look deleted.
_DB_LOCATION_KEYS = ("memory_db", "memory_dir", "data_dir", "home")
def _relocation_warning(key: str, value) -> str | None:
"""Warn when a config change points the database somewhere with no data."""
if key not in _DB_LOCATION_KEYS:
return None
current = settings.memory_db
if not current.is_file():
return None
if key == "memory_db":
new_db = Path(str(value))
elif key == "memory_dir":
new_db = Path(str(value)) / current.name
else:
# data_dir/home only decide the DB location when nothing more specific
# does, and only in the layout where the DB actually sits under them -
# a source checkout keeps it in synapse/memory/ regardless.
existing = config.read_user_config()
if "memory_dir" in existing or "memory_db" in existing:
return None
anchor = settings.data_dir if key == "data_dir" else settings.state_dir
try:
tail = current.resolve().relative_to(anchor.resolve())
except ValueError:
return None
new_db = Path(str(value)) / tail
if new_db.resolve() == current.resolve() or new_db.is_file():
return None
return (
f"{current} holds your existing conversations and memories, but this "
f"change points NexusOS at {new_db}, which does not exist yet - it will "
"start with an empty database. Stop NexusOS and move the .db (plus any "
"-wal/-shm files) to the new path to keep your history."
)
def cmd_config(args) -> int: def cmd_config(args) -> int:
values = config.read_user_config() values = config.read_user_config()
if args.action == "path": if args.action == "path":
@@ -161,8 +202,14 @@ def cmd_config(args) -> int:
except (TypeError, ValueError) as exc: except (TypeError, ValueError) as exc:
print(f"Invalid value: {exc}", file=sys.stderr) print(f"Invalid value: {exc}", file=sys.stderr)
return 2 return 2
payload = {"updated": args.key, "value": values[args.key], "restart_required": True}
moved = _relocation_warning(args.key, values[args.key])
if moved:
payload["warning"] = moved
config.write_user_config(values) config.write_user_config(values)
_emit({"updated": args.key, "value": values[args.key], "restart_required": True}, args.json) _emit(payload, args.json)
if moved and not args.json:
print(f"\nWARNING: {moved}", file=sys.stderr)
return 0 return 0
if args.action == "unset": if args.action == "unset":
if args.key not in CONFIG_SCHEMA: if args.key not in CONFIG_SCHEMA:
@@ -275,6 +322,7 @@ def diagnostics() -> dict:
("voice transcription", "faster_whisper"), ("voice transcription", "faster_whisper"),
("PDF documents", "pypdf"), ("PDF documents", "pypdf"),
("Word documents", "docx"), ("Word documents", "docx"),
("web search", "duckduckgo_search"),
): ):
add(label, _check_import(module), module, required=False) add(label, _check_import(module), module, required=False)
@@ -341,6 +389,15 @@ def cmd_status(args) -> int:
return 0 return 0
def cmd_monitor(args) -> int:
from .monitor import run_monitor
return run_monitor(
interval=getattr(args, "interval", 1.5),
once=bool(getattr(args, "once", False) or getattr(args, "json", False)),
json_output=bool(getattr(args, "json", False)),
)
def _target_flag(target: str | None): def _target_flag(target: str | None):
return { return {
"memory": "--memory", "memory": "--memory",
@@ -366,6 +423,58 @@ def cmd_refresh(args) -> int:
return 0 return 0
def _lan_hostnames(host: str) -> list[str]:
"""Every name/address a --allow-lan bind should accept in a Host header.
A wildcard bind answers on all interfaces, so enumerate them; an explicit
address answers only as itself. The machine hostname comes along because
that is what people actually type."""
import socket
names: list[str] = []
def add(value: str) -> None:
if value and value not in names:
names.append(value)
if host in ("0.0.0.0", "::", "*"):
hostname = socket.gethostname()
add(hostname)
add(hostname.split(".")[0] + ".local")
for family in (socket.AF_INET, socket.AF_INET6):
try:
for info in socket.getaddrinfo(hostname, None, family):
add(info[4][0])
except OSError:
pass
# getaddrinfo(hostname) misses the routable address on hosts that map
# their own name to loopback; a connectionless UDP socket finds it.
for probe, family in (("8.8.8.8", socket.AF_INET), ("2001:4860:4860::8888", socket.AF_INET6)):
sock = socket.socket(family, socket.SOCK_DGRAM)
try:
sock.connect((probe, 80))
add(sock.getsockname()[0])
except OSError:
pass
finally:
sock.close()
else:
add(host.strip("[]"))
# A Host header carries an IPv6 literal bracketed; allow both spellings so
# the check matches however the client wrote it.
for value in list(names):
if ":" in value:
add(f"[{value}]")
return names
def _origin_host(name: str) -> str:
"""Origin-safe spelling: IPv6 literals must be bracketed in a URL."""
if ":" in name and not name.startswith("["):
return f"[{name}]"
return name
def cmd_serve(args) -> int: def cmd_serve(args) -> int:
host = args.host or settings.bind_host host = args.host or settings.bind_host
if host not in ("127.0.0.1", "localhost", "::1") and not args.allow_lan: if host not in ("127.0.0.1", "localhost", "::1") and not args.allow_lan:
@@ -389,10 +498,27 @@ def cmd_serve(args) -> int:
if origin not in config.ALLOWED_ORIGINS: if origin not in config.ALLOWED_ORIGINS:
config.ALLOWED_ORIGINS.append(origin) config.ALLOWED_ORIGINS.append(origin)
if args.allow_lan: if args.allow_lan:
os.environ.setdefault("NEXUS_ALLOWED_HOSTS", "*") # Widen to the addresses this bind actually answers on - NOT "*".
os.environ.setdefault("NEXUS_ALLOWED_ORIGINS", "*") # ALLOWED_HOSTS drives TrustedHostMiddleware, which is the DNS-rebinding
config.ALLOWED_HOSTS[:] = ["*"] # defense: with "*" any site the user browses could resolve a name it
config.ALLOWED_ORIGINS[:] = ["*"] # controls to this machine and drive the unauthenticated API. Naming the
# real addresses keeps that check doing its job. An explicitly exported
# NEXUS_ALLOWED_HOSTS still wins, for anyone who needs the old blanket.
names = _lan_hostnames(host)
for name in names:
if name not in config.ALLOWED_HOSTS:
config.ALLOWED_HOSTS.append(name)
for port in (args.port, args.memory_port):
origin = f"http://{_origin_host(name)}:{port}"
if origin not in config.ALLOWED_ORIGINS:
config.ALLOWED_ORIGINS.append(origin)
os.environ.setdefault("NEXUS_ALLOWED_HOSTS", ",".join(config.ALLOWED_HOSTS))
os.environ.setdefault("NEXUS_ALLOWED_ORIGINS", ",".join(config.ALLOWED_ORIGINS))
print(
"LAN exposure enabled for: " + ", ".join(names)
+ "\nThe REST API is unauthenticated - anyone who can reach this port"
" has full admin and data access."
)
memory_proc = None memory_proc = None
memory_log = None memory_log = None
@@ -617,6 +743,11 @@ def build_parser() -> argparse.ArgumentParser:
p = sub.add_parser("doctor", help="check the core runtime and optional capabilities") p = sub.add_parser("doctor", help="check the core runtime and optional capabilities")
p.add_argument("--fix", action="store_true"); _add_json(p); p.set_defaults(fn=cmd_doctor) p.add_argument("--fix", action="store_true"); _add_json(p); p.set_defaults(fn=cmd_doctor)
p = sub.add_parser("status", help="show service and provider status"); _add_json(p); p.set_defaults(fn=cmd_status) p = sub.add_parser("status", help="show service and provider status"); _add_json(p); p.set_defaults(fn=cmd_status)
p = sub.add_parser("monitor", help="ASCII dashboard for services, resources, and tool stats")
p.add_argument("--once", action="store_true", help="print one frame and exit")
p.add_argument("--interval", type=float, default=1.5, help="refresh seconds (live mode)")
_add_json(p)
p.set_defaults(fn=cmd_monitor)
p = sub.add_parser("serve", help="run NexusOS in the foreground") p = sub.add_parser("serve", help="run NexusOS in the foreground")
p.add_argument("--host"); p.add_argument("--port", type=_port, default=settings.backend_port) p.add_argument("--host"); p.add_argument("--port", type=_port, default=settings.backend_port)
@@ -672,8 +803,16 @@ def _normalize_legacy_argv(argv) -> list[str]:
normalized = list(argv or []) normalized = list(argv or [])
if normalized == ["help"]: if normalized == ["help"]:
return ["--help"] return ["--help"]
if normalized and normalized[0] in ("start", "stop", "logs"): # start/stop only. `logs` registers -f as the short form of --follow, so
# translating it here would silently rewrite `logs -f` to `logs frontend`
# - a tail of the wrong file instead of a follow, with no error.
if normalized and normalized[0] in ("start", "stop"):
normalized[1:] = [LEGACY_TARGETS.get(value, value) for value in normalized[1:]] normalized[1:] = [LEGACY_TARGETS.get(value, value) for value in normalized[1:]]
elif normalized[:1] == ["logs"]:
normalized[1:] = [
value if value in ("-f", "--follow") else LEGACY_TARGETS.get(value, value)
for value in normalized[1:]
]
if normalized[:2] == ["backup", "full"]: if normalized[:2] == ["backup", "full"]:
normalized[1] = "--full" normalized[1] = "--full"
elif len(normalized) > 1 and normalized[0] == "backup" and normalized[1] in ("check", "--claude"): elif len(normalized) > 1 and normalized[0] == "backup" and normalized[1] in ("check", "--claude"):
+458
View File
@@ -0,0 +1,458 @@
"""ASCII dashboard for live NexusOS service / tool / resource stats.
No curses, no rich pure box-drawing + optional ANSI color so it works in
Termux, plain SSH, and Windows Terminal alike. The collector is separate from
the renderer so tests can feed fixtures without a running stack.
"""
from __future__ import annotations
import json
import shutil
import time
import urllib.error
import urllib.request
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
from synapse.nexus_config import settings
from . import ncp as services
# Box drawing — ASCII fallbacks when the terminal can't do Unicode.
_BOX = {
"tl": "", "tr": "", "bl": "", "br": "",
"h": "", "v": "", "l": "", "r": "",
}
_BOX_ASCII = {
"tl": "+", "tr": "+", "bl": "+", "br": "+",
"h": "-", "v": "|", "l": "+", "r": "+",
}
_FILL = ""
_EMPTY = ""
_FILL_ASCII = "#"
_EMPTY_ASCII = "-"
def _use_unicode() -> bool:
enc = (getattr(__import__("sys").stdout, "encoding", None) or "").lower()
return "utf" in enc or enc in ("cp65001",)
def _http_ok(url: str, timeout: float = 1.0) -> bool:
try:
urllib.request.urlopen(url, timeout=timeout).read(1)
return True
except urllib.error.HTTPError:
return True
except Exception:
return False
def _provider_payload() -> dict:
url = settings.ollama_host.rstrip("/")
return {
"provider": settings.provider,
"url": url,
"managed_by_nexus": settings.manage_ollama,
"reachable": _http_ok(url + "/api/tags", timeout=2.0),
}
def _service_status() -> dict:
payload = {}
for key in ("backend", "memory", "frontend"):
svc = services.SERVICES[key]
pid = services.read_pid(svc)
payload[key] = {
"running": services.alive(pid) or _http_ok(svc.url),
"pid": pid if services.alive(pid) else None,
"url": svc.url,
}
payload["provider"] = _provider_payload()
return payload
def _get_json(url: str, timeout: float = 1.5) -> Any | None:
try:
with urllib.request.urlopen(url, timeout=timeout) as resp:
return json.loads(resp.read().decode("utf-8", errors="replace"))
except Exception:
return None
def _bar(ratio: float, width: int = 20, unicode: bool = True) -> str:
ratio = max(0.0, min(1.0, float(ratio)))
filled = int(round(ratio * width))
fill = _FILL if unicode else _FILL_ASCII
empty = _EMPTY if unicode else _EMPTY_ASCII
return fill * filled + empty * (width - filled)
def _fmt_bytes(n: float | int | None) -> str:
if n is None:
return ""
n = float(n)
for unit in ("B", "K", "M", "G", "T"):
if abs(n) < 1024 or unit == "T":
return f"{n:.0f}{unit}" if unit == "B" else f"{n:.1f}{unit}"
n /= 1024
return f"{n:.1f}T"
def _pid_stats(pids: list[int | None]) -> dict:
"""Aggregate CPU%/RSS for known service PIDs. Soft-depends on psutil."""
live = [int(p) for p in pids if p]
if not live:
return {"cpu_pct": None, "rss": None, "pids": []}
try:
import psutil # type: ignore
except ImportError:
return {"cpu_pct": None, "rss": None, "pids": live}
cpu = 0.0
rss = 0
seen: list[int] = []
for pid in live:
try:
proc = psutil.Process(pid)
cpu += proc.cpu_percent(interval=0.0)
rss += proc.memory_info().rss
seen.append(pid)
except (psutil.Error, ProcessLookupError, ValueError):
continue
return {"cpu_pct": cpu, "rss": rss, "pids": seen}
def _host_stats() -> dict:
try:
import psutil # type: ignore
except ImportError:
return {"cpu_pct": None, "mem_used": None, "mem_total": None, "mem_pct": None}
vm = psutil.virtual_memory()
return {
"cpu_pct": psutil.cpu_percent(interval=0.05),
"mem_used": vm.used,
"mem_total": vm.total,
"mem_pct": vm.percent,
}
def _api_counts(api_url: str) -> dict:
"""Pull cheap inventory counts from the backend when it is up."""
base = api_url.rstrip("/")
out = {
"online": False,
"version": None,
"ollama": None,
"memories": None,
"conversations": None,
"playbooks": None,
"models": None,
"action_tool_policy": None,
}
status = _get_json(base + "/status")
if not isinstance(status, dict):
return out
out["online"] = True
out["version"] = status.get("version")
out["ollama"] = status.get("ollama")
mem = _get_json(base + "/memory")
if isinstance(mem, list):
out["memories"] = len(mem)
elif isinstance(mem, dict) and isinstance(mem.get("memories"), list):
out["memories"] = len(mem["memories"])
conv = _get_json(base + "/conversations")
if isinstance(conv, list):
out["conversations"] = len(conv)
elif isinstance(conv, dict):
items = conv.get("conversations") or conv.get("items") or []
if isinstance(items, list):
out["conversations"] = len(items)
pbs = _get_json(base + "/playbooks")
if isinstance(pbs, list):
out["playbooks"] = len(pbs)
elif isinstance(pbs, dict) and isinstance(pbs.get("playbooks"), list):
out["playbooks"] = len(pbs["playbooks"])
models = _get_json(base + "/models")
if isinstance(models, list):
out["models"] = len(models)
elif isinstance(models, dict):
items = models.get("models") or models.get("items") or []
if isinstance(items, list):
out["models"] = len(items)
settings_payload = _get_json(base + "/settings")
if isinstance(settings_payload, dict):
out["action_tool_policy"] = settings_payload.get("action_tool_policy")
return out
def _toolchain_stats() -> list[dict]:
"""Which run_snippet languages have a host toolchain right now."""
try:
from synapse import code_run
except Exception:
return []
rows = []
for name, spec in code_run.RUN_LANGS.items():
tool = None
try:
tool = spec["tool"]()
except Exception:
tool = None
rows.append({
"lang": name,
"ready": bool(tool),
"tool": tool or None,
"summary": spec.get("summary") or name,
})
return rows
def _recent_tools(log_path: Path, limit: int = 8) -> list[str]:
"""Best-effort scrape of recent tool names from chat.log."""
if not log_path.is_file():
return []
try:
# Read the tail without pulling a multi-MB log into memory.
data = log_path.read_bytes()
if len(data) > 64_000:
data = data[-64_000:]
text = data.decode("utf-8", errors="replace")
except OSError:
return []
found: list[str] = []
for line in reversed(text.splitlines()):
# Chat tool loop yields "__status__<tool>"; logs may also name tools
# in JSON payloads. Keep the match narrow.
if "__status__" in line:
name = line.split("__status__", 1)[-1].strip().split()[0].strip(",\"'")
if name and name not in ("tools",) and name not in found:
found.append(name)
elif '"name":' in line and any(
t in line for t in ("render_preview", "run_snippet", "web_search",
"fetch_url", "remember", "get_time")
):
for t in ("run_snippet", "render_preview", "web_search", "fetch_url",
"remember", "get_time", "search_documents"):
if t in line and t not in found:
found.append(t)
if len(found) >= limit:
break
return found
def collect_snapshot() -> dict:
"""Gather one monitoring frame. Safe when services are down."""
services_payload = _service_status()
pids = [
services_payload.get("backend", {}).get("pid"),
services_payload.get("memory", {}).get("pid"),
services_payload.get("frontend", {}).get("pid"),
]
api = _api_counts(settings.api_url)
return {
"ts": datetime.now(timezone.utc).astimezone().isoformat(timespec="seconds"),
"version": settings.version,
"services": services_payload,
"api": api,
"host": _host_stats(),
"procs": _pid_stats(pids),
"toolchains": _toolchain_stats(),
"recent_tools": _recent_tools(settings.logs_dir / "chat.log"),
"paths": {
"api_url": settings.api_url,
"memory_url": settings.memory_url,
"runtime_dir": str(settings.runtime_dir),
},
}
def _pad(text: str, width: int) -> str:
# Visual width ≈ len for our ASCII/box content (no wide East-Asian chars).
if len(text) > width:
return text[: width - 1] + "" if width > 1 else text[:width]
return text + " " * (width - len(text))
def _row(box: dict, inner: str, width: int) -> str:
return f"{box['v']} {_pad(inner, width - 4)} {box['v']}"
def _rule(box: dict, width: int, kind: str = "mid") -> str:
h = box["h"] * (width - 2)
if kind == "top":
return f"{box['tl']}{h}{box['tr']}"
if kind == "bot":
return f"{box['bl']}{h}{box['br']}"
return f"{box['l']}{h}{box['r']}"
def _svc_line(name: str, running: bool, detail: str, unicode: bool) -> str:
mark = (_FILL if unicode else _FILL_ASCII) * 3 if running else (_EMPTY if unicode else _EMPTY_ASCII) * 3
state = "UP " if running else "DOWN"
return f"{name:<10} {mark} {state} {detail}"
def render_frame(snapshot: dict, *, width: int | None = None, unicode: bool | None = None) -> str:
"""Turn a snapshot into a single multi-line ASCII panel."""
if unicode is None:
unicode = _use_unicode()
box = _BOX if unicode else _BOX_ASCII
cols = shutil.get_terminal_size((80, 24)).columns if width is None else width
width = max(56, min(100, cols))
lines: list[str] = []
lines.append(_rule(box, width, "top"))
title = f"NexusOS {snapshot.get('version', '')} monitor"
raw_ts = snapshot.get("ts") or ""
# Prefer local clock HH:MM:SS from an ISO stamp; fall back to wall clock.
stamp = ""
if "T" in raw_ts:
try:
stamp = raw_ts.split("T", 1)[1][:8]
except Exception:
stamp = ""
if not stamp:
stamp = datetime.now().strftime("%H:%M:%S")
gap = max(1, width - 4 - len(title) - len(stamp))
header = f"{title}{' ' * gap}{stamp}"
lines.append(_row(box, header, width))
lines.append(_rule(box, width, "mid"))
lines.append(_row(box, "SERVICES", width))
svcs = snapshot.get("services") or {}
for key, label in (("backend", "backend"), ("memory", "memory"), ("frontend", "frontend")):
info = svcs.get(key) or {}
running = bool(info.get("running"))
pid = info.get("pid")
url = info.get("url") or ""
detail = url
if pid:
detail = f"pid {pid} {url}"
lines.append(_row(box, _svc_line(label, running, detail, unicode), width))
provider = svcs.get("provider") or _provider_payload()
pref = f"{provider.get('provider', '?')} @ {provider.get('url', '')}"
lines.append(_row(box, _svc_line("provider", bool(provider.get("reachable")), pref, unicode), width))
lines.append(_rule(box, width, "mid"))
lines.append(_row(box, "RESOURCES", width))
host = snapshot.get("host") or {}
procs = snapshot.get("procs") or {}
cpu = host.get("cpu_pct")
if cpu is not None:
lines.append(_row(
box,
f"host CPU [{_bar(cpu / 100.0, 22, unicode)}] {cpu:5.1f}%",
width,
))
else:
lines.append(_row(box, "host CPU (install psutil for live bars)", width))
mem_pct = host.get("mem_pct")
if mem_pct is not None:
lines.append(_row(
box,
f"host MEM [{_bar(mem_pct / 100.0, 22, unicode)}] "
f"{_fmt_bytes(host.get('mem_used'))} / {_fmt_bytes(host.get('mem_total'))}",
width,
))
proc_cpu = procs.get("cpu_pct")
proc_rss = procs.get("rss")
if proc_cpu is not None or proc_rss is not None:
lines.append(_row(
box,
f"nexus cpu={proc_cpu if proc_cpu is not None else '':>5} "
f"rss={_fmt_bytes(proc_rss)} pids={','.join(str(p) for p in (procs.get('pids') or [])) or ''}",
width,
))
lines.append(_rule(box, width, "mid"))
lines.append(_row(box, "DATA / TOOLS", width))
api = snapshot.get("api") or {}
if api.get("online"):
policy = api.get("action_tool_policy") or ""
lines.append(_row(
box,
f"api UP v{api.get('version') or '?'} ollama={api.get('ollama') or ''} "
f"tools={policy}",
width,
))
lines.append(_row(
box,
f"memories={_n(api.get('memories'))} "
f"chats={_n(api.get('conversations'))} "
f"playbooks={_n(api.get('playbooks'))} "
f"models={_n(api.get('models'))}",
width,
))
else:
lines.append(_row(box, "api DOWN — start with: nexus start", width))
recent = snapshot.get("recent_tools") or []
lines.append(_row(
box,
"recent " + (", ".join(recent) if recent else "(none in chat.log)"),
width,
))
lines.append(_rule(box, width, "mid"))
lines.append(_row(box, "RUN TOOLCHAINS (run_snippet)", width))
chains = snapshot.get("toolchains") or []
if not chains:
lines.append(_row(box, "(code_run unavailable)", width))
else:
# Pack ready/missing into one or two compact lines.
ready = [c["lang"] for c in chains if c.get("ready")]
missing = [c["lang"] for c in chains if not c.get("ready")]
lines.append(_row(
box,
f"ready {', '.join(ready) if ready else ''}",
width,
))
lines.append(_row(
box,
f"missing {', '.join(missing) if missing else ''}",
width,
))
lines.append(_rule(box, width, "bot"))
return "\n".join(lines)
def _n(value) -> str:
return "" if value is None else str(value)
def run_monitor(*, interval: float = 1.5, once: bool = False, json_output: bool = False) -> int:
"""Print one frame, or refresh in place until interrupted."""
clear = "\033[H\033[J"
first = True
while True:
snap = collect_snapshot()
if json_output:
print(json.dumps(snap, indent=2, sort_keys=True))
else:
frame = render_frame(snap)
if once or not first:
# Replacing the screen keeps the panel stable; first frame of a
# live session also clears so leftover shell output doesn't mix.
if not once:
print(clear + frame, end="", flush=True)
else:
print(frame)
else:
print(clear + frame, end="", flush=True)
first = False
if once:
return 0
try:
time.sleep(max(0.3, float(interval)))
except KeyboardInterrupt:
print()
return 130
+18 -23
View File
@@ -28,6 +28,7 @@ import urllib.request
from dataclasses import dataclass, field from dataclasses import dataclass, field
from pathlib import Path from pathlib import Path
from synapse import proc_util
from synapse.nexus_config import SOURCE_CHECKOUT, settings from synapse.nexus_config import SOURCE_CHECKOUT, settings
ROOT = settings.project_root ROOT = settings.project_root
@@ -170,11 +171,9 @@ def alive(pid) -> bool:
return False return False
if ps is not None: if ps is not None:
return ps.pid_exists(pid) return ps.pid_exists(pid)
try: # Not os.kill(pid, 0): that reports False for a live process owned by
os.kill(pid, 0) # another user, and Windows has no signals to fall back on.
return True return proc_util.pid_alive(pid)
except (OSError, ValueError):
return False
def pid_is_ours(pid, patterns) -> bool: def pid_is_ours(pid, patterns) -> bool:
@@ -186,11 +185,8 @@ def pid_is_ours(pid, patterns) -> bool:
try: try:
if ps is not None: if ps is not None:
cmd = " ".join(ps.Process(pid).cmdline()) cmd = " ".join(ps.Process(pid).cmdline())
elif os.name != "nt":
cmd = (Path(f"/proc/{pid}/cmdline").read_bytes()
.replace(b"\0", b" ").decode(errors="replace"))
else: else:
return False cmd = proc_util.pid_cmdline(pid)
except Exception: except Exception:
return False return False
return any(p in cmd for p in patterns) return any(p in cmd for p in patterns)
@@ -278,18 +274,11 @@ def kill_matching(patterns, force=False) -> int:
me = os.getpid() me = os.getpid()
hit = 0 hit = 0
if ps is None: if ps is None:
if os.name == "nt": for pid, cmd in proc_util.iter_processes():
return 0 if pid == me or not any(pattern in cmd for pattern in patterns):
for entry in Path("/proc").iterdir():
if not entry.name.isdigit() or int(entry.name) == me:
continue continue
try: if proc_util.terminate_pid(pid, force=force):
cmd = (entry / "cmdline").read_bytes().replace(b"\0", b" ").decode(errors="replace") hit += 1
if any(pattern in cmd for pattern in patterns):
os.kill(int(entry.name), 9 if force else 15)
hit += 1
except (OSError, ValueError):
pass
return hit return hit
for proc in ps.process_iter(["pid", "cmdline"]): for proc in ps.process_iter(["pid", "cmdline"]):
if proc.info["pid"] == me: if proc.info["pid"] == me:
@@ -309,7 +298,11 @@ def kill_port(port: int) -> bool:
stop reliable regardless of process-tree shape or PID-file accuracy.""" stop reliable regardless of process-tree shape or PID-file accuracy."""
ps = _psutil() ps = _psutil()
if ps is None: if ps is None:
return False killed = False
for pid in proc_util.pids_listening_on(port):
if pid != os.getpid() and proc_util.terminate_pid(pid, force=True):
killed = True
return killed
killed = False killed = False
try: try:
conns = ps.net_connections(kind="inet") conns = ps.net_connections(kind="inet")
@@ -341,8 +334,10 @@ def stop_service(svc: Service) -> bool:
except Exception: except Exception:
pass pass
proc.terminate() proc.terminate()
elif os.name != "nt": else:
os.kill(pid, 15) # No psutil means no process tree; the kill_matching sweep
# below is what catches reparented children here.
proc_util.terminate_pid(pid)
except Exception: except Exception:
pass pass
svc.pid_file.unlink(missing_ok=True) svc.pid_file.unlink(missing_ok=True)
+28 -8
View File
@@ -39,6 +39,9 @@ process = ["psutil>=5.9,<8"]
vector = ["sqlite-vec>=0.1,<1"] vector = ["sqlite-vec>=0.1,<1"]
voice = ["faster-whisper>=1.1,<2"] voice = ["faster-whisper>=1.1,<2"]
mail = ["imap-tools>=1.7,<2"] mail = ["imap-tools>=1.7,<2"]
# synapse/search.py imports this lazily behind a bare except, so without it
# declared the chat web-search path silently returns nothing.
search = ["duckduckgo-search>=6,<9"]
desktop = [ desktop = [
"psutil>=5.9,<8", "psutil>=5.9,<8",
"pywebview>=5,<7; platform_system == 'Windows'", "pywebview>=5,<7; platform_system == 'Windows'",
@@ -48,6 +51,20 @@ standard = [
"python-docx>=1.1,<2", "python-docx>=1.1,<2",
"psutil>=5.9,<8", "psutil>=5.9,<8",
"sqlite-vec>=0.1,<1", "sqlite-vec>=0.1,<1",
"duckduckgo-search>=6,<9",
]
# Every optional capability at once. Kept in sync with the extras above by
# tests/test_packaging_deps.py, which also checks that nothing synapse imports
# is missing from this file.
all = [
"pypdf>=5,<7",
"python-docx>=1.1,<2",
"psutil>=5.9,<8",
"sqlite-vec>=0.1,<1",
"faster-whisper>=1.1,<2",
"imap-tools>=1.7,<2",
"duckduckgo-search>=6,<9",
"pywebview>=5,<7; platform_system == 'Windows'",
] ]
dev = [ dev = [
"build>=1.2,<2", "build>=1.2,<2",
@@ -56,9 +73,9 @@ dev = [
] ]
[project.scripts] [project.scripts]
nexus = "management.cli:entrypoint" nexus = "nexusos_cli.cli:entrypoint"
ncp = "management.cli:entrypoint" ncp = "nexusos_cli.cli:entrypoint"
nexusos = "management.cli:entrypoint" nexusos = "nexusos_cli.cli:entrypoint"
[project.urls] [project.urls]
Homepage = "https://git.enderofwings.com/enderofwings/NexusOS" Homepage = "https://git.enderofwings.com/enderofwings/NexusOS"
@@ -72,12 +89,16 @@ pattern = "^(?P<version>[^\\s]+)$"
[tool.hatch.build] [tool.hatch.build]
skip-excluded-dirs = true skip-excluded-dirs = true
# Ships interface/web/dist when it has been built. It is gitignored, so a
# static force-include would abort `pip install -e .` on a fresh clone.
[tool.hatch.build.hooks.custom]
path = "hatch_build.py"
[tool.hatch.build.targets.wheel] [tool.hatch.build.targets.wheel]
packages = ["synapse", "management"] packages = ["synapse", "nexusos_cli"]
[tool.hatch.build.targets.wheel.force-include] [tool.hatch.build.targets.wheel.force-include]
"VERSION" = "synapse/_resources/VERSION" "VERSION" = "synapse/_resources/VERSION"
"interface/web/dist" = "synapse/_resources/web"
"data/playbooks" = "synapse/_resources/playbooks" "data/playbooks" = "synapse/_resources/playbooks"
"assets/n-small.png" = "synapse/_resources/assets/n-small.png" "assets/n-small.png" = "synapse/_resources/assets/n-small.png"
"assets/themes/NexusOS-icons-src/nexus-underlay.svg" = "synapse/_resources/assets/themes/NexusOS-icons-src/nexus-underlay.svg" "assets/themes/NexusOS-icons-src/nexus-underlay.svg" = "synapse/_resources/assets/themes/NexusOS-icons-src/nexus-underlay.svg"
@@ -90,18 +111,17 @@ include = [
"/assets/themes/NexusOS-icons-src/nexus-underlay-ring.svg", "/assets/themes/NexusOS-icons-src/nexus-underlay-ring.svg",
"/data/playbooks", "/data/playbooks",
"/docs", "/docs",
"/interface/web/dist",
"/management", "/management",
"/nexusos_cli",
"/scripts", "/scripts",
"/synapse", "/synapse",
"/tests", "/tests",
"/README.md", "/README.md",
"/VERSION", "/VERSION",
"/pyproject.toml", "/pyproject.toml",
"/hatch_build.py",
] ]
[tool.hatch.build.targets.sdist.force-include]
"interface/web/dist" = "interface/web/dist"
[tool.pytest.ini_options] [tool.pytest.ini_options]
testpaths = ["tests", "management"] testpaths = ["tests", "management"]
+3
View File
@@ -38,6 +38,9 @@ sqlite-vec
faster-whisper faster-whisper
# Email client: IMAP read (SMTP send is stdlib). Pure-Python, no native deps. # Email client: IMAP read (SMTP send is stdlib). Pure-Python, no native deps.
imap-tools imap-tools
# Chat web search (synapse/search.py). Imported lazily behind a bare except,
# so a missing install shows up as search silently returning nothing.
duckduckgo-search
# Documentation Support # Documentation Support
markdown-it-py markdown-it-py
+11 -2
View File
@@ -46,7 +46,9 @@ EOF
fi fi
echo "==> Installing ${PACKAGE_SPEC}" echo "==> Installing ${PACKAGE_SPEC}"
python -m pip install "${PACKAGE_SPEC}" # Same extra index as the pydantic-core probe: a NexusOS wheel hosted there
# would otherwise be invisible to the resolver.
python -m pip install "${PIP_INDEX_ARGS[@]}" "${PACKAGE_SPEC}"
echo "==> Initializing NexusOS" echo "==> Initializing NexusOS"
nexus init nexus init
@@ -55,7 +57,12 @@ if [[ -n "${NEXUS_PROVIDER_URL:-}" ]]; then
nexus provider use remote --url "${NEXUS_PROVIDER_URL}" nexus provider use remote --url "${NEXUS_PROVIDER_URL}"
fi fi
nexus doctor # Report health without aborting: `set -e` would otherwise skip the guidance
# below whenever doctor finds a failing required check - exactly when the
# reader most needs to see what to do next. The status is re-raised at exit.
DOCTOR_STATUS=0
nexus doctor || DOCTOR_STATUS=$?
cat <<'EOF' cat <<'EOF'
NexusOS is installed. Configure an Ollama-compatible provider, then run: NexusOS is installed. Configure an Ollama-compatible provider, then run:
@@ -64,3 +71,5 @@ NexusOS is installed. Configure an Ollama-compatible provider, then run:
Open http://127.0.0.1:8000 in the Android browser. Open http://127.0.0.1:8000 in the Android browser.
EOF EOF
exit "${DOCTOR_STATUS}"
+7 -12
View File
@@ -12,13 +12,13 @@ import os
import shutil import shutil
import subprocess import subprocess
import urllib.request import urllib.request
from pathlib import Path
try: try:
import psutil import psutil
except ImportError: # optional in the portable/Termux core install except ImportError: # optional in the portable/Termux core install
psutil = None psutil = None
from . import proc_util
from .nexus_config import FRONTEND_SOURCE_DIR, RUNTIME_DIR from .nexus_config import FRONTEND_SOURCE_DIR, RUNTIME_DIR
FRONTEND_DIR = FRONTEND_SOURCE_DIR FRONTEND_DIR = FRONTEND_SOURCE_DIR
@@ -47,11 +47,8 @@ def _is_ours(pid: int) -> bool:
try: try:
if psutil is not None: if psutil is not None:
cmd = " ".join(psutil.Process(pid).cmdline()) cmd = " ".join(psutil.Process(pid).cmdline())
elif os.name != "nt":
cmd = (Path(f"/proc/{pid}/cmdline").read_bytes()
.replace(b"\0", b" ").decode(errors="replace"))
else: else:
return False cmd = proc_util.pid_cmdline(pid)
except Exception: except Exception:
return False return False
return "vite" in cmd or ("npm" in cmd and "dev" in cmd) return "vite" in cmd or ("npm" in cmd and "dev" in cmd)
@@ -62,11 +59,9 @@ def _alive(pid: int | None) -> bool:
return False return False
if psutil is not None: if psutil is not None:
return psutil.pid_exists(pid) return psutil.pid_exists(pid)
try: # proc_util rather than os.kill(pid, 0): same probe, but it also works
os.kill(pid, 0) # when the PID belongs to another user.
return True return proc_util.pid_alive(pid)
except (OSError, ValueError):
return False
def _http_up() -> bool: def _http_up() -> bool:
@@ -128,8 +123,8 @@ def stop() -> dict:
except Exception: except Exception:
pass pass
proc.terminate() proc.terminate()
elif os.name != "nt": else:
os.kill(pid, 15) proc_util.terminate_pid(pid)
except Exception: except Exception:
pass pass
PID_FILE.unlink(missing_ok=True) PID_FILE.unlink(missing_ok=True)
+8 -4
View File
@@ -33,14 +33,18 @@ RESOURCE_ROOT = PROJECT_ROOT if SOURCE_CHECKOUT else PACKAGE_DIR / "_resources"
def _user_dir(env_name: str, windows_leaf: str, xdg_name: str, xdg_fallback: str) -> Path: def _user_dir(env_name: str, windows_leaf: str, xdg_name: str, xdg_fallback: str) -> Path:
# os.getenv's default only applies when a variable is *unset*. An exported
# but empty XDG_DATA_HOME / LOCALAPPDATA would otherwise give Path("") ==
# ".", scattering state through whatever the cwd happened to be. The XDG
# spec says to treat an empty value as unset, so `or` - not a default arg.
override = os.getenv(env_name, "").strip() override = os.getenv(env_name, "").strip()
if override: if override:
return Path(override).expanduser().resolve() return Path(override).expanduser().resolve()
if os.name == "nt": if os.name == "nt":
base = Path(os.getenv("LOCALAPPDATA", Path.home() / "AppData" / "Local")) base = Path(os.getenv("LOCALAPPDATA", "").strip() or Path.home() / "AppData" / "Local")
return base / windows_leaf return (base / windows_leaf).expanduser().resolve()
base = Path(os.getenv(xdg_name, Path.home() / xdg_fallback)).expanduser() base = Path(os.getenv(xdg_name, "").strip() or Path.home() / xdg_fallback).expanduser()
return base / "nexusos" return (base / "nexusos").resolve()
CONFIG_DIR = _user_dir("NEXUS_CONFIG_DIR", "NexusOS", "XDG_CONFIG_HOME", ".config") CONFIG_DIR = _user_dir("NEXUS_CONFIG_DIR", "NexusOS", "XDG_CONFIG_HOME", ".config")
+212
View File
@@ -0,0 +1,212 @@
"""Process inspection and termination that works with or without psutil.
psutil became an optional extra when the wheel landed, so the base install
(notably `pip install nexusos-ai` on Windows) has to manage PIDs with the
stdlib alone. Callers should keep using psutil when it is importable - it is
faster and more precise - and fall back here when it is not.
Windows has no signals: os.kill(pid, sig) special-cases CTRL_C_EVENT and
CTRL_BREAK_EVENT, treats sig 0 as an existence check, and calls
TerminateProcess(handle, sig) for *everything else*. So os.kill(pid, 15) is not
a polite request there - it is an immediate, unblockable kill with exit code
15, and there is no equivalent of SIGTERM. Everything below goes through the
Win32 API via ctypes so the intent is explicit at each call site rather than
resting on which signal numbers happen to be special.
"""
from __future__ import annotations
import os
import signal
import subprocess
from pathlib import Path
WINDOWS = os.name == "nt"
# Win32 constants (winnt.h / processthreadsapi.h)
_SYNCHRONIZE = 0x00100000
_PROCESS_TERMINATE = 0x0001
_WAIT_TIMEOUT = 0x00000102
# Keep console windows from flashing on every helper subprocess.
_NO_WINDOW = subprocess.CREATE_NO_WINDOW if WINDOWS else 0
def _kernel32():
import ctypes
return ctypes.WinDLL("kernel32", use_last_error=True)
def _run(argv: list[str], timeout: float = 10.0) -> str:
"""Run a helper command, returning stdout ('' on any failure)."""
try:
done = subprocess.run(
argv, capture_output=True, text=True, timeout=timeout,
creationflags=_NO_WINDOW,
)
except (OSError, subprocess.SubprocessError):
return ""
return done.stdout or ""
def pid_alive(pid: int | None) -> bool:
"""True if the PID names a live process.
On Windows this opens a handle and polls it; a signalled handle means the
process has exited. That is equivalent to os.kill(pid, 0) - CPython
special-cases signal 0 into an existence check there - but it also reports
True for a process owned by another user, where os.kill raises.
"""
if pid is None:
return False
try:
pid = int(pid)
except (TypeError, ValueError):
return False
if pid <= 0:
return False
if WINDOWS:
import ctypes
k = _kernel32()
handle = k.OpenProcess(_SYNCHRONIZE, False, pid)
if not handle:
return False
try:
return k.WaitForSingleObject(ctypes.c_void_p(handle), 0) == _WAIT_TIMEOUT
finally:
k.CloseHandle(ctypes.c_void_p(handle))
try:
os.kill(pid, 0)
return True
except ProcessLookupError:
return False
except PermissionError:
return True # exists, owned by someone else
except (OSError, ValueError):
return False
def terminate_pid(pid: int | None, force: bool = False) -> bool:
"""Ask a process to exit. Returns True if the request was delivered."""
if not pid_alive(pid):
return False
pid = int(pid)
if WINDOWS:
# No graceful path without a shared console; TerminateProcess is what
# psutil.terminate() resolves to on Windows anyway.
import ctypes
k = _kernel32()
handle = k.OpenProcess(_PROCESS_TERMINATE, False, pid)
if not handle:
return False
try:
return bool(k.TerminateProcess(ctypes.c_void_p(handle), 1))
finally:
k.CloseHandle(ctypes.c_void_p(handle))
try:
os.kill(pid, signal.SIGKILL if force else signal.SIGTERM)
return True
except OSError:
return False
def pid_cmdline(pid: int | None) -> str:
"""Full command line for a PID, or '' when it cannot be determined."""
if pid is None:
return ""
try:
pid = int(pid)
except (TypeError, ValueError):
return ""
if WINDOWS:
for entry_pid, cmd in iter_processes():
if entry_pid == pid:
return cmd
return ""
proc = Path(f"/proc/{pid}/cmdline")
try:
return proc.read_bytes().replace(b"\0", b" ").decode(errors="replace").strip()
except OSError:
pass
# macOS and other POSIX hosts without /proc.
out = _run(["ps", "-o", "command=", "-p", str(pid)])
return out.strip()
def iter_processes() -> list[tuple[int, str]]:
"""(pid, command_line) for every visible process.
Windows needs CIM for command lines - the ctypes snapshot APIs only expose
image names, which is not enough to tell `uvicorn synapse.main` apart from
any other python.exe. This is slow, so it is strictly the no-psutil path.
"""
if WINDOWS:
out = _run([
"powershell", "-NoProfile", "-NonInteractive", "-Command",
"Get-CimInstance Win32_Process | "
"ForEach-Object { \"$($_.ProcessId)`t$($_.CommandLine)\" }",
], timeout=30.0)
entries: list[tuple[int, str]] = []
for line in out.splitlines():
head, _, cmd = line.partition("\t")
if head.strip().isdigit():
entries.append((int(head), cmd.strip()))
return entries
entries = []
proc_root = Path("/proc")
if proc_root.is_dir():
for entry in proc_root.iterdir():
if not entry.name.isdigit():
continue
try:
cmd = (entry / "cmdline").read_bytes()
except OSError:
continue
entries.append((int(entry.name), cmd.replace(b"\0", b" ").decode(errors="replace").strip()))
return entries
for line in _run(["ps", "-A", "-o", "pid=,command="]).splitlines():
head, _, cmd = line.strip().partition(" ")
if head.isdigit():
entries.append((int(head), cmd.strip()))
return entries
def pids_listening_on(port: int) -> list[int]:
"""PIDs holding a listening TCP socket on `port`."""
pids: list[int] = []
if WINDOWS:
for line in _run(["netstat", "-ano", "-p", "TCP"]).splitlines():
parts = line.split()
# Proto Local Foreign State PID
if len(parts) < 5 or parts[3] != "LISTENING":
continue
local = parts[1]
if local.rsplit(":", 1)[-1] == str(port) and parts[4].isdigit():
pids.append(int(parts[4]))
return sorted(set(pids))
# -t TCP, -l listening, -n numeric, -P no port names.
for line in _run(["lsof", "-nP", "-tiTCP:%d" % port, "-sTCP:LISTEN"]).splitlines():
if line.strip().isdigit():
pids.append(int(line.strip()))
if pids:
return sorted(set(pids))
for line in _run(["ss", "-lptnH", "sport = :%d" % port]).splitlines():
# ... users:(("uvicorn",pid=1234,fd=3))
marker = "pid="
start = line.find(marker)
while start != -1:
digits = ""
for ch in line[start + len(marker):]:
if not ch.isdigit():
break
digits += ch
if digits:
pids.append(int(digits))
start = line.find(marker, start + 1)
return sorted(set(pids))
+64 -4
View File
@@ -18,7 +18,7 @@ def run_cli(tmp_path: Path, *args: str) -> subprocess.CompletedProcess[str]:
env["NEXUS_HOME"] = str(tmp_path / "state") env["NEXUS_HOME"] = str(tmp_path / "state")
env["NEXUS_CONFIG_DIR"] = str(tmp_path / "config") env["NEXUS_CONFIG_DIR"] = str(tmp_path / "config")
return subprocess.run( return subprocess.run(
[sys.executable, "-m", "management.cli", *args], [sys.executable, "-m", "nexusos_cli.cli", *args],
cwd=ROOT, cwd=ROOT,
env=env, env=env,
text=True, text=True,
@@ -31,17 +31,22 @@ def run_cli(tmp_path: Path, *args: str) -> subprocess.CompletedProcess[str]:
def test_help_exposes_portable_command_tree(tmp_path): def test_help_exposes_portable_command_tree(tmp_path):
result = run_cli(tmp_path, "--help") result = run_cli(tmp_path, "--help")
assert result.returncode == 0, result.stderr assert result.returncode == 0, result.stderr
for command in ("init", "config", "provider", "doctor", "serve", "models", "chat"): for command in ("init", "config", "provider", "doctor", "serve", "models", "chat", "monitor"):
assert command in result.stdout assert command in result.stdout
def test_legacy_cli_spellings_remain_compatible(): def test_legacy_cli_spellings_remain_compatible():
from management.cli import _normalize_legacy_argv from nexusos_cli.cli import _normalize_legacy_argv
assert _normalize_legacy_argv(["start", "-b"]) == ["start", "backend"] assert _normalize_legacy_argv(["start", "-b"]) == ["start", "backend"]
assert _normalize_legacy_argv(["stop", "--ai"]) == ["stop", "ai"] assert _normalize_legacy_argv(["stop", "--ai"]) == ["stop", "ai"]
assert _normalize_legacy_argv(["logs", "-m", "--follow"]) == ["logs", "memory", "--follow"] assert _normalize_legacy_argv(["logs", "-m", "--follow"]) == ["logs", "memory", "--follow"]
assert _normalize_legacy_argv(["backup", "full"]) == ["backup", "--full"] assert _normalize_legacy_argv(["backup", "full"]) == ["backup", "--full"]
# -f is --follow for `logs`, but --frontend for start/stop. Translating it
# for logs turned `logs -f` into a one-shot tail of the frontend log.
assert _normalize_legacy_argv(["logs", "-f"]) == ["logs", "-f"]
assert _normalize_legacy_argv(["logs", "-m", "-f"]) == ["logs", "memory", "-f"]
assert _normalize_legacy_argv(["start", "-f"]) == ["start", "frontend"]
assert _normalize_legacy_argv(["restore", "-f"]) == ["restore"] assert _normalize_legacy_argv(["restore", "-f"]) == ["restore"]
assert _normalize_legacy_argv(["help"]) == ["--help"] assert _normalize_legacy_argv(["help"]) == ["--help"]
@@ -92,7 +97,7 @@ def test_serve_rejects_invalid_ports_before_startup(tmp_path):
def test_remote_provider_is_never_stopped_or_force_killed(monkeypatch): def test_remote_provider_is_never_stopped_or_force_killed(monkeypatch):
from management import ncp from nexusos_cli import ncp
killed_ports = [] killed_ports = []
killed_patterns = [] killed_patterns = []
@@ -108,3 +113,58 @@ def test_remote_provider_is_never_stopped_or_force_killed(monkeypatch):
assert 11434 not in killed_ports assert 11434 not in killed_ports
assert "ollama serve" not in killed_patterns assert "ollama serve" not in killed_patterns
def test_allow_lan_names_addresses_instead_of_disabling_the_host_check():
"""ALLOWED_HOSTS=* would switch TrustedHostMiddleware off entirely, and that
middleware is the DNS-rebinding defense for an unauthenticated API."""
from nexusos_cli.cli import _lan_hostnames
assert _lan_hostnames("192.168.1.20") == ["192.168.1.20"]
wildcard = _lan_hostnames("0.0.0.0")
assert wildcard, "a wildcard bind must resolve to concrete host names"
assert "*" not in wildcard
# IPv6 literals need to match a Host header written either way.
for name in wildcard:
if ":" in name and not name.startswith("["):
assert f"[{name}]" in wildcard
def test_empty_platform_dir_variables_do_not_put_state_in_the_cwd(tmp_path, monkeypatch):
"""os.getenv's default only fires when a variable is *unset*; an exported
but empty XDG_DATA_HOME / LOCALAPPDATA made Path("") == "." the base, so
state landed in whatever directory the command happened to run from.
Patching os.name to exercise the other platform's branch is not an option -
pathlib dispatches on it - so this checks the branch this host actually
takes."""
from synapse import nexus_config
if os.name == "nt":
monkeypatch.setenv("LOCALAPPDATA", "")
expected = Path.home() / "AppData" / "Local" / "NexusOS"
else:
monkeypatch.setenv("XDG_DATA_HOME", "")
expected = Path.home() / ".local/share" / "nexusos"
monkeypatch.delenv("NEXUS_HOME", raising=False)
monkeypatch.chdir(tmp_path)
resolved = nexus_config._user_dir("NEXUS_HOME", "NexusOS", "XDG_DATA_HOME", ".local/share")
assert resolved.is_absolute()
assert tmp_path.resolve() != resolved.parent
assert resolved == expected.resolve()
def test_config_set_warns_before_orphaning_the_database(tmp_path):
"""Repointing the DB does not move it - say so, or history looks deleted."""
assert run_cli(tmp_path, "init").returncode == 0
db = tmp_path / "state" / "data" / "memory.db"
db.parent.mkdir(parents=True, exist_ok=True)
db.write_bytes(b"SQLite format 3" + bytes(1))
result = run_cli(tmp_path, "config", "set", "memory_db", str(tmp_path / "elsewhere.db"), "--json")
assert result.returncode == 0, result.stderr
assert "warning" in json.loads(result.stdout)
same = run_cli(tmp_path, "config", "set", "memory_db", str(db), "--json")
assert "warning" not in json.loads(same.stdout)
+78
View File
@@ -0,0 +1,78 @@
"""Tests for the ASCII monitor — renderer + collectors, no live stack required."""
from __future__ import annotations
from nexusos_cli.monitor import _bar, _fmt_bytes, _recent_tools, render_frame
def test_bar_bounds():
assert _bar(0, 10, unicode=False) == "-" * 10
assert _bar(1, 10, unicode=False) == "#" * 10
assert _bar(0.5, 10, unicode=False).count("#") == 5
def test_fmt_bytes():
assert _fmt_bytes(None) == ""
assert _fmt_bytes(512) == "512B"
assert _fmt_bytes(2048).endswith("K")
def test_render_frame_contains_sections():
snap = {
"ts": "2026-08-20T12:00:00-05:00",
"version": "0.0.0",
"services": {
"backend": {"running": True, "pid": 11, "url": "http://127.0.0.1:8000"},
"memory": {"running": False, "pid": None, "url": "http://127.0.0.1:8001"},
"frontend": {"running": False, "pid": None, "url": "http://127.0.0.1:5173"},
"provider": {
"provider": "ollama",
"url": "http://127.0.0.1:11434",
"reachable": True,
},
},
"api": {
"online": True,
"version": "0.0.0",
"ollama": "running",
"memories": 3,
"conversations": 2,
"playbooks": 1,
"models": 4,
"action_tool_policy": "ask",
},
"host": {"cpu_pct": 12.5, "mem_used": 1_000_000_000, "mem_total": 8_000_000_000, "mem_pct": 12.5},
"procs": {"cpu_pct": 1.0, "rss": 50_000_000, "pids": [11]},
"toolchains": [
{"lang": "python", "ready": True, "tool": "/usr/bin/python", "summary": "python"},
{"lang": "rust", "ready": False, "tool": None, "summary": "rust"},
],
"recent_tools": ["run_snippet", "render_preview"],
"paths": {},
}
frame = render_frame(snap, width=72, unicode=False)
assert "SERVICES" in frame
assert "RESOURCES" in frame
assert "DATA / TOOLS" in frame
assert "RUN TOOLCHAINS" in frame
assert "backend" in frame and "UP" in frame
assert "memory" in frame and "DOWN" in frame
assert "run_snippet" in frame
assert "ready python" in frame
assert "missing rust" in frame
# Fixed-width box: every content line same length.
lengths = {len(line) for line in frame.splitlines()}
assert len(lengths) == 1
def test_recent_tools_parses_status_sentinels(tmp_path):
log = tmp_path / "chat.log"
log.write_text(
"noise\n__status__tools\n__status__run_snippet\n"
'payload {"name": "render_preview"}\n__status__remember\n',
encoding="utf-8",
)
found = _recent_tools(log, limit=5)
assert "run_snippet" in found
assert "remember" in found
assert "render_preview" in found
assert "tools" not in found
@@ -1,10 +1,5 @@
"""Pin the SSE parser in nexus_api. Run: python management/test_nexus_api.py""" """Pin the SSE parser in nexus_api. Run: python tests/test_nexus_api.py"""
import sys, os from nexusos_cli.nexus_api import iter_chunks
sys.path.insert(0, os.path.dirname(__file__))
try:
from management.nexus_api import iter_chunks
except ModuleNotFoundError: # direct: python management/test_nexus_api.py
from nexus_api import iter_chunks
# A realistic /chat/stream frame: two token chunks, a meta block, then done. # A realistic /chat/stream frame: two token chunks, a meta block, then done.
lines = [ lines = [
+144
View File
@@ -0,0 +1,144 @@
"""Guard the wheel's dependency list against drift.
There are now two dependency declarations: requirements-base.txt (what the
desktop installers pip -r) and pyproject.toml (what the wheel ships). They will
drift. What actually breaks a user is narrower than "they differ", though: it
is an import that no declared distribution provides, so that is what this pins.
"""
from __future__ import annotations
import ast
import sys
import tomllib
from pathlib import Path
import pytest
ROOT = Path(__file__).resolve().parents[1]
SHIPPED_PACKAGES = ("synapse", "nexusos_cli")
# Import name -> distribution name, where PyPI disagrees with the module.
DISTRIBUTION_OF = {
"docx": "python-docx",
"dotenv": "python-dotenv",
"faster_whisper": "faster-whisper",
"imap_tools": "imap-tools",
"sqlite_vec": "sqlite-vec",
"yaml": "pyyaml",
"PIL": "pillow",
}
# Provided by another declared distribution rather than named directly.
TRANSITIVE = {"starlette", "socketio", "engineio"}
# Modules that ship inside this repo.
FIRST_PARTY = {"synapse", "nexusos_cli", "management", "bin", "tests"}
def _pyproject() -> dict:
return tomllib.loads((ROOT / "pyproject.toml").read_text(encoding="utf-8"))
def _requirement_name(spec: str) -> str:
"""'pypdf>=5,<7' -> 'pypdf'; strips extras and environment markers."""
head = spec.split(";", 1)[0].strip()
for sep in ("[", "=", ">", "<", "!", "~", " "):
head = head.split(sep, 1)[0]
return head.strip().lower().replace("_", "-")
def _declared() -> set[str]:
project = _pyproject()["project"]
specs = list(project.get("dependencies", []))
for extra in project.get("optional-dependencies", {}).values():
specs.extend(extra)
return {_requirement_name(s) for s in specs}
def _imported_modules() -> set[str]:
"""Top-level module names imported anywhere in the shipped packages."""
found: set[str] = set()
for package in SHIPPED_PACKAGES:
for path in (ROOT / package).rglob("*.py"):
tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path))
for node in ast.walk(tree):
if isinstance(node, ast.Import):
found.update(alias.name.split(".")[0] for alias in node.names)
elif isinstance(node, ast.ImportFrom):
# level > 0 is a relative (first-party) import.
if node.level == 0 and node.module:
found.add(node.module.split(".")[0])
return found
def _third_party() -> set[str]:
return {
module for module in _imported_modules()
if module not in sys.stdlib_module_names
and module not in FIRST_PARTY
and module not in TRANSITIVE
and not module.startswith("_")
}
def test_every_third_party_import_is_a_declared_dependency():
declared = _declared()
missing = sorted(
module for module in _third_party()
if DISTRIBUTION_OF.get(module, module).lower().replace("_", "-") not in declared
)
assert not missing, (
"synapse/nexusos_cli import these, but pyproject.toml declares no "
f"distribution for them: {missing}. Add them to [project] dependencies "
"or an extra (and to DISTRIBUTION_OF here if the names differ)."
)
def test_all_extra_is_the_union_of_the_capability_extras():
extras = _pyproject()["project"]["optional-dependencies"]
combined: set[str] = set()
for name, specs in extras.items():
if name in ("all", "dev", "standard"):
continue
combined.update(_requirement_name(s) for s in specs)
everything = {_requirement_name(s) for s in extras["all"]}
assert combined == everything, (
"the 'all' extra drifted from the capability extras; "
f"missing={sorted(combined - everything)} extra={sorted(everything - combined)}"
)
@pytest.mark.parametrize("name", ["fastapi", "uvicorn", "httpx", "pydantic", "pyyaml"])
def test_core_runtime_is_a_hard_dependency_not_an_extra(name):
"""These are imported at module scope, so the base install must carry them."""
base = {_requirement_name(s) for s in _pyproject()["project"]["dependencies"]}
assert name in base
def test_optional_imports_are_lazy():
"""Anything only in an extra must not be imported at module scope.
A base `pip install nexusos-ai` has none of the extras, so a top-level
`import psutil` in synapse would make the backend unimportable.
"""
base = {_requirement_name(s) for s in _pyproject()["project"]["dependencies"]}
offenders: list[str] = []
for package in SHIPPED_PACKAGES:
for path in (ROOT / package).rglob("*.py"):
tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path))
for node in tree.body: # module scope only
names: list[str] = []
if isinstance(node, ast.Import):
names = [a.name.split(".")[0] for a in node.names]
elif isinstance(node, ast.ImportFrom) and node.level == 0 and node.module:
names = [node.module.split(".")[0]]
for module in names:
if module in sys.stdlib_module_names or module in FIRST_PARTY:
continue
dist = DISTRIBUTION_OF.get(module, module).lower().replace("_", "-")
if dist not in base and module not in TRANSITIVE:
offenders.append(f"{path.relative_to(ROOT)}: {module}")
assert not offenders, (
"optional dependencies imported at module scope (wrap in try/ImportError "
f"or import inside the function): {offenders}"
)
+94
View File
@@ -0,0 +1,94 @@
"""Pin the psutil-free process helpers.
psutil is an optional extra, so on a base install these are the only way
`ncp stop` / `nexus status` can see or stop a service. Windows previously had
no fallback at all: pid_is_ours returned False, kill_port returned False, and
the terminate branch was POSIX-only, so stop was a no-op there.
Probing must also stay side-effect free. That is easy to get wrong on Windows,
where os.kill(pid, sig) is TerminateProcess for every sig except 0 and the two
console-control events - os.kill(pid, 15) kills instead of asking.
"""
from __future__ import annotations
import subprocess
import sys
import time
import pytest
from synapse import proc_util
def _spawn():
return subprocess.Popen(
[sys.executable, "-c", "import time; time.sleep(30)"],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
)
def _wait_gone(proc, timeout=10.0) -> bool:
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
if proc.poll() is not None:
return True
time.sleep(0.05)
return False
@pytest.fixture
def victim():
proc = _spawn()
try:
yield proc
finally:
if proc.poll() is None:
proc.kill()
proc.wait(timeout=10)
def test_pid_alive_does_not_kill_the_process(victim):
"""Probing must be side-effect free, however it is implemented."""
for _ in range(5):
assert proc_util.pid_alive(victim.pid) is True
time.sleep(0.3)
assert victim.poll() is None, "pid_alive() terminated the process it probed"
def test_pid_alive_is_false_for_a_dead_pid(victim):
victim.kill()
victim.wait(timeout=10)
assert proc_util.pid_alive(victim.pid) is False
@pytest.mark.parametrize("pid", [None, 0, -1, "not-a-pid"])
def test_pid_alive_rejects_junk(pid):
assert proc_util.pid_alive(pid) is False
def test_terminate_pid_actually_stops_it(victim):
assert proc_util.terminate_pid(victim.pid) is True
assert _wait_gone(victim), "terminate_pid() did not stop the process"
assert proc_util.pid_alive(victim.pid) is False
def test_terminate_pid_is_false_when_already_gone(victim):
victim.kill()
victim.wait(timeout=10)
assert proc_util.terminate_pid(victim.pid) is False
def test_pid_cmdline_identifies_the_process(victim):
cmd = proc_util.pid_cmdline(victim.pid)
if not cmd:
pytest.skip("no command-line source on this host")
assert "time.sleep" in cmd or "python" in cmd.lower()
def test_iter_processes_includes_this_interpreter():
import os
entries = proc_util.iter_processes()
if not entries:
pytest.skip("no process enumeration available on this host")
assert os.getpid() in {pid for pid, _ in entries}