Athena/NexusOS
1
0
Fork 0
forked from enderofwings/NexusOS
Code Pull Requests Activity

fix(sync,memory,gpu): restorable memory dump, curator grounding, GPU + Models fixes

Browse Source 
Ported from downstream development. Four independent defects.

1. The memory dump was unrestorable. iterdump() serializes sqlite_vec virtual
   tables as a raw INSERT INTO sqlite_master(...) followed by inserts into a
   table the replaying connection cannot see, so replaying memory.db.sql died
   on "no such table: vec_messages" and left ZERO tables behind. dump_db() now
   loads the vec0 extension and filters the derived vec tables out of the
   iterdump stream, matched on each statement's target table rather than as a
   substring - a chat message whose text mentions vec_messages is an
   INSERT INTO "messages" and has to survive.

   compare() reported an unreadable dump as "diverged", which read like a real
   verdict and made both guards refuse backup AND restore, locking the machine
   out of syncing in either direction. Unreadable is now its own verdict.

   _extra() compared updated_at against a "" default, but the column is REAL,
   so the comparison raises TypeError on the first conversation the other side
   lacks - exactly the case it counts. It tests membership first now. The
   direction test declared updated_at TEXT, which is why this survived: the
   test compared str to str while the field compared str to float.

2. The memory curator invented facts. It attributed the ASSISTANT's words to
   the user, wrote absence claims read off the existing-memory block, and added
   judgements ("favorite") the user never used. The prompt now scopes the USER
   line as the only source, and two deterministic guards drop absence claims
   and facts whose distinctive tokens appear nowhere in the user's message -
   prompt wording alone did not hold on a 7B curator.

3. _best_vulkan_device scored Mesa's llvmpipe above an integrated GPU, pinning
   Ollama to a software rasterizer advertising 31 GiB of "VRAM" - CPU inference
   with Vulkan overhead on top. Software rasterizers are dropped.

4. Models.jsx compared catalog names to installed names literally, but Ollama
   resolves a bare name to ":latest", so an untagged entry (nomic-embed-text)
   read as missing forever and the Required gate never opened. Chatbot.jsx
   fetched the model list once on mount although App keeps the page mounted
   behind display:none, so a newly pulled model never appeared in the picker
   until a full browser reload.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
janvanwan
2026-08-14 17:33:43 -05:00
co-authored by Claude Opus 5
parent b5541d1c48
commit 8691a67803
13 changed files with 443 additions and 47 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/test_smoke.py
+158 -4
View File
@@ -7,6 +7,7 @@ Deliberately tiny (ponytail): it guards the things v1 promises - app wiring,
model defaults, playbook ordering, persistence - without needing a running
Ollama service or network. Not a full suite.
"""
import subprocess
from pathlib import Path
from fastapi.testclient import TestClient
@@ -271,13 +272,16 @@ def test_sync_compare_detects_direction(tmp_path):
db.unlink(missing_ok=True)
with sq.connect(db) as conn:
conn.executescript(
"create table conversations (id text primary key, updated_at text);"
# updated_at REAL, matching the production schema in store.py. A TEXT
# column here hid a real TypeError for months: the comparison in
# _extra() ran str-vs-str in the test and str-vs-float in the field.
"create table conversations (id text primary key, updated_at real not null);"
"create table memory (id text primary key);"
)
conn.executemany("insert into conversations values (?, ?)", rows)
assert sync.compare(db, dump) == "no-dump"
write([("a", "1")])
write([("a", 1778553309.5)])
assert sync.compare(db, dump) == "no-dump"
# Dump matches the live DB exactly.
@@ -286,7 +290,7 @@ def test_sync_compare_detects_direction(tmp_path):
assert sync.compare(db, dump) == "same"
# A newer message bumps updated_at -> this box is ahead of the backup.
write([("a", "2")])
write([("a", 1778553999.5)])
assert sync.compare(db, dump) == "local-ahead"
# The backup holds a conversation this box never saw.
@@ -294,9 +298,15 @@ def test_sync_compare_detects_direction(tmp_path):
assert sync.compare(db, dump) == "local-behind"
# Each side has something the other lacks.
write([("b", "1")])
write([("b", 1778553309.5)])
assert sync.compare(db, dump) == "diverged"
# A dump that won't replay is its own verdict, not a fake divergence -
# reporting "diverged" there blocked backup AND restore with what looked
# like a legitimate answer.
dump.write_text("INSERT INTO nope VALUES (1);\n")
assert sync.compare(db, dump) == "unreadable"
db.unlink()
assert sync.compare(db, dump) == "no-live"
@@ -328,3 +338,147 @@ def test_desktop_stage_is_the_only_one_touching_home():
home_writes = [ln for ln in runtime.splitlines()
if "$HOME" in ln and not any(w in ln for w in shell_wiring)]
assert not home_writes, f"runtime stage writes to $HOME: {home_writes}"
def test_genmon_configs_are_written_with_the_panel_down():
"""genmon holds its config in memory and rewrites genmon-N.rc when the panel
exits, so copying the rc files while the panel is running gets silently
undone - every applet then loads blank. The copy has to sit between the panel
quit and the relaunch."""
script = (REPO_ROOT / "bin" / "panel" / "install.sh").read_text()
quit_at = script.index("xfce4-panel -q")
copy_at = script.index('cp -f "$NEXUS/management/panel/genmon-$id.rc"')
start_at = script.index("setsid xfce4-panel")
assert quit_at < copy_at < start_at, "genmon rc copy must happen with the panel stopped"
_VULKANINFO_IGPU_AND_LLVMPIPE = """\
Devices:
========
GPU0:
\tvendorID = 0x8086
\tdeviceType = PHYSICAL_DEVICE_TYPE_INTEGRATED_GPU
\tdeviceName = Intel(R) Graphics (RPL-S)
GPU1:
\tvendorID = 0x10005
\tdeviceType = PHYSICAL_DEVICE_TYPE_CPU
\tdeviceName = llvmpipe (LLVM 20.1.2, 256 bits)
"""
def test_software_rasterizer_is_never_picked_as_a_gpu(monkeypatch):
"""Mesa always advertises an llvmpipe device with deviceType CPU. It used to
outscore an integrated GPU (neither DISCRETE nor INTEGRATED scored higher
than INTEGRATED), so Ollama got pinned to a software rasterizer - CPU
inference with Vulkan overhead stacked on top, reported as a 31 GiB
'discrete' GPU. The iGPU has to win, and a box with nothing but rasterizers
has to report no Vulkan device at all."""
def fake_run(cmd, **kwargs):
return subprocess.CompletedProcess(cmd, 0, _VULKANINFO_IGPU_AND_LLVMPIPE, "")
monkeypatch.setattr(ollama_manager.subprocess, "run", fake_run)
idx, name = ollama_manager._best_vulkan_device()
assert idx == 0 and "Intel" in name, f"picked {name!r} over the iGPU"
monkeypatch.setattr(
ollama_manager.subprocess, "run",
lambda cmd, **kw: subprocess.CompletedProcess(cmd, 0, "Devices:\nGPU0:\n"
"\tdeviceType = PHYSICAL_DEVICE_TYPE_CPU\n"
"\tdeviceName = llvmpipe\n", ""))
assert ollama_manager._best_vulkan_device()[0] == -1, "rasterizer-only box must report no GPU"
def test_installed_model_lookup_normalizes_the_latest_tag():
"""Ollama resolves a bare name to ":latest", so a catalog entry written
untagged ("nomic-embed-text") never matched the installed name
("nomic-embed-text:latest") and the Required gate stayed locked forever - the
model pulls fine, the UI just never sees it. Every lookup has to go through
withTag()."""
jsx = (REPO_ROOT / "interface" / "web" / "src" / "Models.jsx").read_text()
assert 'const withTag =' in jsx
assert "installedNames.has(m.name.toLowerCase())" not in jsx, \
"raw name lookup is back; untagged catalog entries will read as missing"
def test_dump_round_trips_a_db_holding_vec_tables(tmp_path):
"""The memory dump is the ONLY backup of conversations, facts and history.
iterdump() serializes a sqlite_vec virtual table as a raw
INSERT INTO sqlite_master(...) plus inserts into a table the replaying
connection cannot see, so restoring died on "no such table: vec_messages"
and left zero tables - the entire backup was unrecoverable. Build a DB
shaped like production (vec table included) and prove the dump replays."""
import sqlite3 as sq
sqlite_vec = pytest.importorskip("sqlite_vec", reason="vec index is optional")
sync = _load_sync()
if sync._vec0_extension() is None:
# dump_db() resolves vec0 relative to the repo's own venv, so a clone
# whose Promethean isn't built yet cannot dump a DB holding vec tables.
pytest.skip("sqlite_vec extension not present under this repo's venv")
db, dump = tmp_path / "memory.db", tmp_path / "memory.db.sql"
conn = sq.connect(db)
conn.enable_load_extension(True)
sqlite_vec.load(conn)
conn.enable_load_extension(False)
conn.executescript(
"create table conversations (id text primary key, updated_at real not null);"
"create table memory (id text primary key);"
"create table messages (id integer primary key, body text);"
"create virtual table vec_messages using vec0(embedding float[3] distance_metric=cosine);"
)
conn.execute("insert into conversations values ('c1', 1778553309.5)")
conn.execute("insert into memory values ('m1')")
# A message whose text mentions the filtered table names: a filter applied to
# the dump TEXT instead of the statement stream would eat this row.
conn.execute("insert into messages values (1, 'debugging vec_messages and vec_documents')")
conn.execute("insert into vec_messages(rowid, embedding) values (1, ?)",
(sqlite_vec.serialize_float32([0.1, 0.2, 0.3]),))
conn.commit()
conn.close()
sync.DB, sync.DB_SQL = db, dump
assert sync.dump_db()
restored = tmp_path / "restored.db"
with sq.connect(restored) as out:
out.executescript(dump.read_text(encoding="utf-8")) # must not raise
got = sq.connect(restored)
assert got.execute("select count(*) from conversations").fetchone()[0] == 1
assert got.execute("select count(*) from memory").fetchone()[0] == 1
assert got.execute("select body from messages").fetchone()[0] == \
"debugging vec_messages and vec_documents"
# The vec index is derived - absent from the dump, rebuilt by the backfill.
assert "vec_messages" not in {r[0] for r in got.execute(
"select name from sqlite_master where type='table'")}
def test_curator_drops_fabricated_facts():
"""The curator model invents two classes of fact no prompt wording stopped
(verified against mistral:7b), and both reached the real memory DB: absence
claims read off the existing-memory block ("Jon does not have any pets",
which contradicted four cats on file) and specifics lifted from the
ASSISTANT's reply ("Jon's main development machine is a MacBook Pro", from
the user message "What am I developing on?"). Deterministic guard, so it
holds whatever the model does."""
from synapse.memory.extractor import _reject_reason
# Absence claims are never facts.
assert _reject_reason("Jon does not have any pets", "do i have any pets?")
assert _reject_reason("Jon's favorite episode is unknown", "what's my favorite episode?")
assert _reject_reason("Jon has not specified an interest", "tell me about stargate")
# Specifics the user never typed came from the assistant.
assert _reject_reason("Jon's main dev machine is a MacBook Pro", "What am I developing on?")
# ...but the same shape grounded in the user's own words must survive.
assert _reject_reason(
"Jon owns a 2000 Ford Ranger with a 3.0L V6",
"i also have a 2000 Ford Ranger, it's a five-speed with a 3.0L V6") is None
assert _reject_reason(
"Jon has a beagle named Biscuit",
"i just adopted a dog named Biscuit, he's a beagle") is None
# A fact carrying no proper nouns or numbers can't be grounding-checked;
# the prompt owns that case, so the guard must let it through.
assert _reject_reason(
"Jon prefers short answers over long explanations",
"i really prefer short answers over long explanations") is None