forked from enderofwings/NexusOS
fix(windows): real toolchain probing, HOME/TEMP env, and gate portability
code_run.py: shutil.which() finding a compiler executable on PATH doesn't mean it's a usable toolchain on Windows -- rustc's MSVC target also needs Microsoft's linker, and an MSYS2 gcc/clang driver can remain resolvable after one of its runtime DLLs has broken. Both cases silently turned every C/C++/ Rust snippet into a compile error while the capability check said "ready". _compiled_tool() now actually compiles+links a trivial known-good program per candidate (Windows only; POSIX keeps the cheap which(1) check since release hosts install compiler packages atomically) and caches the result. Also fixes the run/compile child environment: HOME/TMPDIR don't control Windows' real temp/profile resolution (expanduser() reaches the actual user profile, GetTempPath() falls back to the Windows directory), letting a snippet escape the scratch directory or fail outright. _child_env() now also sets TEMP/TMP/USERPROFILE on Windows. tests/conftest.py (new): isolates curry_store's SQLite singleton into a per-run temp directory via NEXUS_CURRY_DB before any test module imports synapse, and cleans it up at session end -- the release gate no longer writes test constants into the checkout's live data/curry.db. .gitignore picks up /data/curry.db for whatever still lands there locally. bin/check.sh: falls back to Promethean/Scripts/python.exe when Promethean/bin/python doesn't exist, so the gate actually runs on a Windows venv instead of immediately exiting "no Promethean venv". Verified independently: 254 passed, 0 failed, 8 skipped (tests + management) -- the 12 C/C++/Rust toolchain failures present all session are gone. Full bin/check.sh run end-to-end on this Windows checkout: pytest, eslint, frontend node:test (57/57), PowerShell/shell parse, and the wheel/sdist packaging + twine + content checks all report OK. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
@@ -13,6 +13,7 @@ synapse/memory/memory.db
|
||||
synapse/memory/memory.db-wal
|
||||
synapse/memory/memory.db-shm
|
||||
assets/gitnexus-logo.svg
|
||||
/data/curry.db
|
||||
*.db-wal
|
||||
*.db-shm
|
||||
.DS_Store
|
||||
|
||||
+10
-6
@@ -9,14 +9,18 @@ cd "$(dirname "$0")/.."
|
||||
|
||||
fail=0
|
||||
|
||||
if [ ! -x Promethean/bin/python ]; then
|
||||
if [ -x Promethean/bin/python ]; then
|
||||
NEXUS_CHECK_PY=Promethean/bin/python
|
||||
elif [ -x Promethean/Scripts/python.exe ]; then
|
||||
NEXUS_CHECK_PY=Promethean/Scripts/python.exe
|
||||
else
|
||||
echo "!! no Promethean venv - run ./install.sh first" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "== pytest =="
|
||||
# Explicit dirs: a bare `pytest` would walk Promethean/ and node_modules too.
|
||||
Promethean/bin/python -m pytest -q tests management || fail=1
|
||||
"$NEXUS_CHECK_PY" -m pytest -q tests management || fail=1
|
||||
|
||||
echo "== eslint =="
|
||||
if [ -d interface/web/node_modules ]; then
|
||||
@@ -57,13 +61,13 @@ done
|
||||
echo "== packaging =="
|
||||
# The wheel is the other shippable artifact, so it belongs in the same gate:
|
||||
# a broken pyproject or a missing web build only shows up at build time.
|
||||
if Promethean/bin/python -c "import build, twine" 2>/dev/null; then
|
||||
if "$NEXUS_CHECK_PY" -c "import build, twine" 2>/dev/null; then
|
||||
rm -rf .build-check
|
||||
if Promethean/bin/python -m build --outdir .build-check >/dev/null 2>&1; then
|
||||
Promethean/bin/python -m twine check .build-check/* || fail=1
|
||||
if "$NEXUS_CHECK_PY" -m build --outdir .build-check >/dev/null 2>&1; then
|
||||
"$NEXUS_CHECK_PY" -m twine check .build-check/* || fail=1
|
||||
# The compiled UI has to actually be inside the wheel - a wheel that
|
||||
# builds but ships no dist/ serves a blank page.
|
||||
Promethean/bin/python - <<'PY' || fail=1
|
||||
"$NEXUS_CHECK_PY" - <<'PY' || fail=1
|
||||
import glob, sys, zipfile
|
||||
wheels = glob.glob(".build-check/*.whl")
|
||||
if not wheels:
|
||||
|
||||
+64
-8
@@ -17,9 +17,9 @@ module actually provides is *containment by limits*, layered:
|
||||
every call waits for the user's Approve/Deny in chat.
|
||||
2. Screening `critique` rejects the obvious-abuse shapes (sockets, process
|
||||
spawning, absolute paths) before anything is written to disk.
|
||||
3. Isolation cwd is a fresh temp dir that is deleted afterwards; HOME and
|
||||
TMPDIR point at it; the environment is scrubbed to a small
|
||||
allowlist.
|
||||
3. Isolation cwd is a fresh temp dir that is deleted afterwards; home and
|
||||
temp environment variables point at it; the environment is
|
||||
scrubbed to a small allowlist.
|
||||
4. Limits wall-clock timeout, RLIMIT_CPU/AS/FSIZE/NPROC on POSIX,
|
||||
truncated output.
|
||||
5. Network on Linux, `unshare -rn` when unprivileged user namespaces are
|
||||
@@ -40,6 +40,7 @@ import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
from functools import lru_cache
|
||||
from pathlib import Path
|
||||
|
||||
# Wall clock. Compilation gets its own, larger budget: rustc on a cold cache
|
||||
@@ -186,6 +187,52 @@ def _erlang_write_source(source: str) -> str:
|
||||
return source
|
||||
|
||||
|
||||
@lru_cache(maxsize=None)
|
||||
def _compiled_tool(lang: str, candidates: tuple[str, ...]) -> str | None:
|
||||
"""Resolve a compiler, verifying the complete Windows toolchain once.
|
||||
|
||||
A compiler executable alone is not a usable toolchain on Windows: rustc's
|
||||
MSVC target also needs Microsoft's linker, and an MSYS2 driver can remain on
|
||||
PATH after one of its runtime DLLs has broken. Both cases otherwise make the
|
||||
capability monitor say "ready" and turn every snippet into a compile error.
|
||||
POSIX keeps the cheap historical which(1) check; the release hosts there
|
||||
install compiler packages atomically.
|
||||
"""
|
||||
found = [tool for name in candidates if (tool := shutil.which(name))]
|
||||
if sys.platform != "win32":
|
||||
return found[0] if found else None
|
||||
for tool in found:
|
||||
if _probe_compiled_tool(lang, tool):
|
||||
return tool
|
||||
return None
|
||||
|
||||
|
||||
def _probe_compiled_tool(lang: str, tool: str) -> bool:
|
||||
"""Compile a minimal known-good program with the runner's real child env."""
|
||||
source = {
|
||||
"c": "int main(void){return 0;}",
|
||||
"cpp": "int main(){return 0;}",
|
||||
"rust": "fn main() {}",
|
||||
}[lang]
|
||||
try:
|
||||
with tempfile.TemporaryDirectory(prefix="nexus-toolchain-") as tmp:
|
||||
workdir = Path(tmp)
|
||||
entry = RUN_LANGS[lang]
|
||||
src = workdir / entry["source_name"](source)
|
||||
src.write_text(source, encoding="utf-8")
|
||||
exe = str(workdir / "probe.exe")
|
||||
built = _spawn(
|
||||
entry["compile"](tool, str(src), exe),
|
||||
workdir,
|
||||
_child_env(lang, workdir),
|
||||
COMPILE_TIMEOUT,
|
||||
constrain_memory=False,
|
||||
)
|
||||
return built.returncode == 0 and Path(exe).is_file()
|
||||
except (OSError, subprocess.SubprocessError):
|
||||
return False
|
||||
|
||||
|
||||
# The one place that says which languages can be executed. Each entry owns that
|
||||
# language's screening, toolchain probe and argv. The tool schema's `lang` enum,
|
||||
# the capability line in the system prompt and the dispatch below are all derived
|
||||
@@ -211,7 +258,7 @@ RUN_LANGS: dict[str, dict] = {
|
||||
},
|
||||
"c": {
|
||||
"summary": "single-file C program (C11, libm linked)",
|
||||
"tool": lambda: shutil.which("cc") or shutil.which("gcc") or shutil.which("clang"),
|
||||
"tool": lambda: _compiled_tool("c", ("cc", "gcc", "clang")),
|
||||
"install": "install a C compiler (clang or gcc)",
|
||||
"source_name": _source_name("main.c"),
|
||||
"compile": lambda cc, src, exe: [cc, "-std=c11", "-O0", "-Wall", "-o", exe, src, "-lm"],
|
||||
@@ -221,7 +268,7 @@ RUN_LANGS: dict[str, dict] = {
|
||||
},
|
||||
"cpp": {
|
||||
"summary": "single-file C++ program (C++17)",
|
||||
"tool": lambda: shutil.which("c++") or shutil.which("g++") or shutil.which("clang++"),
|
||||
"tool": lambda: _compiled_tool("cpp", ("c++", "g++", "clang++")),
|
||||
"install": "install a C++ compiler (clang++ or g++)",
|
||||
"source_name": _source_name("main.cpp"),
|
||||
"compile": lambda cc, src, exe: [cc, "-std=c++17", "-O0", "-Wall", "-o", exe, src],
|
||||
@@ -231,7 +278,7 @@ RUN_LANGS: dict[str, dict] = {
|
||||
},
|
||||
"rust": {
|
||||
"summary": "single-file Rust program (2021 edition, std only)",
|
||||
"tool": lambda: shutil.which("rustc"),
|
||||
"tool": lambda: _compiled_tool("rust", ("rustc",)),
|
||||
"install": "install Rust (https://rustup.rs)",
|
||||
"source_name": _source_name("main.rs"),
|
||||
# Debug build: -O roughly triples compile time for snippets that run for
|
||||
@@ -301,8 +348,17 @@ def _child_env(lang: str, workdir: Path) -> dict:
|
||||
if default.is_dir():
|
||||
env["RUSTUP_HOME"] = str(default)
|
||||
env.setdefault("CARGO_HOME", str(Path.home() / ".cargo"))
|
||||
env["HOME"] = str(workdir)
|
||||
env["TMPDIR"] = str(workdir)
|
||||
scratch = str(workdir)
|
||||
env["HOME"] = scratch
|
||||
env["TMPDIR"] = scratch
|
||||
# Windows ignores HOME/TMPDIR in its standard path helpers. Without these,
|
||||
# expanduser() reaches the real profile and GetTempPath() falls back to the
|
||||
# Windows directory; GCC and rustc then either escape the scratch directory
|
||||
# or fail because a normal user cannot write there.
|
||||
env["TEMP"] = scratch
|
||||
env["TMP"] = scratch
|
||||
if os.name == "nt":
|
||||
env["USERPROFILE"] = scratch
|
||||
env.setdefault("LC_ALL", "C.UTF-8")
|
||||
return env
|
||||
|
||||
|
||||
@@ -0,0 +1,23 @@
|
||||
"""Test-process isolation for persistent runtime state."""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import shutil
|
||||
import sys
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
# curry_store constructs its SQLite singleton during test collection. Point it
|
||||
# at a per-run directory before any test module imports synapse, so the release
|
||||
# gate is repeatable and never writes test constants into the checkout's live
|
||||
# data/curry.db.
|
||||
_TEST_STATE = Path(tempfile.mkdtemp(prefix="nexus-pytest-"))
|
||||
os.environ["NEXUS_CURRY_DB"] = str(_TEST_STATE / "curry.db")
|
||||
|
||||
|
||||
def pytest_sessionfinish(session, exitstatus):
|
||||
module = sys.modules.get("synapse.curry_store")
|
||||
if module is not None:
|
||||
module.curry_db.close()
|
||||
shutil.rmtree(_TEST_STATE, ignore_errors=True)
|
||||
@@ -232,10 +232,40 @@ def test_home_points_at_the_scratch_dir():
|
||||
assert "nexus-run-" in out["stdout"]
|
||||
|
||||
|
||||
def test_temp_points_at_the_scratch_dir():
|
||||
"""Compilers and snippets must not fall back to a host temp directory."""
|
||||
out = code_run.run("python", "import tempfile\nprint(tempfile.gettempdir())")
|
||||
assert "nexus-run-" in out["stdout"]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Toolchains
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_windows_compiler_resolver_skips_broken_candidates(monkeypatch):
|
||||
"""An executable on PATH is not enough when its linker/runtime is broken."""
|
||||
code_run._compiled_tool.cache_clear()
|
||||
monkeypatch.setattr(code_run.sys, "platform", "win32")
|
||||
monkeypatch.setattr(
|
||||
code_run.shutil, "which", lambda name: f"C:\\tools\\{name}.exe"
|
||||
)
|
||||
probes = []
|
||||
|
||||
def probe(lang, tool):
|
||||
probes.append((lang, tool))
|
||||
return tool.endswith("clang.exe")
|
||||
|
||||
monkeypatch.setattr(code_run, "_probe_compiled_tool", probe)
|
||||
try:
|
||||
assert code_run._compiled_tool("c", ("gcc", "clang")) == "C:\\tools\\clang.exe"
|
||||
assert probes == [
|
||||
("c", "C:\\tools\\gcc.exe"),
|
||||
("c", "C:\\tools\\clang.exe"),
|
||||
]
|
||||
finally:
|
||||
code_run._compiled_tool.cache_clear()
|
||||
|
||||
|
||||
def test_a_missing_toolchain_explains_itself(monkeypatch):
|
||||
"""The model has to be able to tell 'you cannot run this here' from 'your
|
||||
code is wrong' — otherwise it rewrites a correct program repeatedly."""
|
||||
|
||||
Reference in New Issue
Block a user