forked from enderofwings/NexusOS
fix(windows): real toolchain probing, HOME/TEMP env, and gate portability
code_run.py: shutil.which() finding a compiler executable on PATH doesn't mean it's a usable toolchain on Windows -- rustc's MSVC target also needs Microsoft's linker, and an MSYS2 gcc/clang driver can remain resolvable after one of its runtime DLLs has broken. Both cases silently turned every C/C++/ Rust snippet into a compile error while the capability check said "ready". _compiled_tool() now actually compiles+links a trivial known-good program per candidate (Windows only; POSIX keeps the cheap which(1) check since release hosts install compiler packages atomically) and caches the result. Also fixes the run/compile child environment: HOME/TMPDIR don't control Windows' real temp/profile resolution (expanduser() reaches the actual user profile, GetTempPath() falls back to the Windows directory), letting a snippet escape the scratch directory or fail outright. _child_env() now also sets TEMP/TMP/USERPROFILE on Windows. tests/conftest.py (new): isolates curry_store's SQLite singleton into a per-run temp directory via NEXUS_CURRY_DB before any test module imports synapse, and cleans it up at session end -- the release gate no longer writes test constants into the checkout's live data/curry.db. .gitignore picks up /data/curry.db for whatever still lands there locally. bin/check.sh: falls back to Promethean/Scripts/python.exe when Promethean/bin/python doesn't exist, so the gate actually runs on a Windows venv instead of immediately exiting "no Promethean venv". Verified independently: 254 passed, 0 failed, 8 skipped (tests + management) -- the 12 C/C++/Rust toolchain failures present all session are gone. Full bin/check.sh run end-to-end on this Windows checkout: pytest, eslint, frontend node:test (57/57), PowerShell/shell parse, and the wheel/sdist packaging + twine + content checks all report OK. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
@@ -13,6 +13,7 @@ synapse/memory/memory.db
|
|||||||
synapse/memory/memory.db-wal
|
synapse/memory/memory.db-wal
|
||||||
synapse/memory/memory.db-shm
|
synapse/memory/memory.db-shm
|
||||||
assets/gitnexus-logo.svg
|
assets/gitnexus-logo.svg
|
||||||
|
/data/curry.db
|
||||||
*.db-wal
|
*.db-wal
|
||||||
*.db-shm
|
*.db-shm
|
||||||
.DS_Store
|
.DS_Store
|
||||||
|
|||||||
+10
-6
@@ -9,14 +9,18 @@ cd "$(dirname "$0")/.."
|
|||||||
|
|
||||||
fail=0
|
fail=0
|
||||||
|
|
||||||
if [ ! -x Promethean/bin/python ]; then
|
if [ -x Promethean/bin/python ]; then
|
||||||
|
NEXUS_CHECK_PY=Promethean/bin/python
|
||||||
|
elif [ -x Promethean/Scripts/python.exe ]; then
|
||||||
|
NEXUS_CHECK_PY=Promethean/Scripts/python.exe
|
||||||
|
else
|
||||||
echo "!! no Promethean venv - run ./install.sh first" >&2
|
echo "!! no Promethean venv - run ./install.sh first" >&2
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
echo "== pytest =="
|
echo "== pytest =="
|
||||||
# Explicit dirs: a bare `pytest` would walk Promethean/ and node_modules too.
|
# Explicit dirs: a bare `pytest` would walk Promethean/ and node_modules too.
|
||||||
Promethean/bin/python -m pytest -q tests management || fail=1
|
"$NEXUS_CHECK_PY" -m pytest -q tests management || fail=1
|
||||||
|
|
||||||
echo "== eslint =="
|
echo "== eslint =="
|
||||||
if [ -d interface/web/node_modules ]; then
|
if [ -d interface/web/node_modules ]; then
|
||||||
@@ -57,13 +61,13 @@ done
|
|||||||
echo "== packaging =="
|
echo "== packaging =="
|
||||||
# The wheel is the other shippable artifact, so it belongs in the same gate:
|
# The wheel is the other shippable artifact, so it belongs in the same gate:
|
||||||
# a broken pyproject or a missing web build only shows up at build time.
|
# a broken pyproject or a missing web build only shows up at build time.
|
||||||
if Promethean/bin/python -c "import build, twine" 2>/dev/null; then
|
if "$NEXUS_CHECK_PY" -c "import build, twine" 2>/dev/null; then
|
||||||
rm -rf .build-check
|
rm -rf .build-check
|
||||||
if Promethean/bin/python -m build --outdir .build-check >/dev/null 2>&1; then
|
if "$NEXUS_CHECK_PY" -m build --outdir .build-check >/dev/null 2>&1; then
|
||||||
Promethean/bin/python -m twine check .build-check/* || fail=1
|
"$NEXUS_CHECK_PY" -m twine check .build-check/* || fail=1
|
||||||
# The compiled UI has to actually be inside the wheel - a wheel that
|
# The compiled UI has to actually be inside the wheel - a wheel that
|
||||||
# builds but ships no dist/ serves a blank page.
|
# builds but ships no dist/ serves a blank page.
|
||||||
Promethean/bin/python - <<'PY' || fail=1
|
"$NEXUS_CHECK_PY" - <<'PY' || fail=1
|
||||||
import glob, sys, zipfile
|
import glob, sys, zipfile
|
||||||
wheels = glob.glob(".build-check/*.whl")
|
wheels = glob.glob(".build-check/*.whl")
|
||||||
if not wheels:
|
if not wheels:
|
||||||
|
|||||||
+64
-8
@@ -17,9 +17,9 @@ module actually provides is *containment by limits*, layered:
|
|||||||
every call waits for the user's Approve/Deny in chat.
|
every call waits for the user's Approve/Deny in chat.
|
||||||
2. Screening `critique` rejects the obvious-abuse shapes (sockets, process
|
2. Screening `critique` rejects the obvious-abuse shapes (sockets, process
|
||||||
spawning, absolute paths) before anything is written to disk.
|
spawning, absolute paths) before anything is written to disk.
|
||||||
3. Isolation cwd is a fresh temp dir that is deleted afterwards; HOME and
|
3. Isolation cwd is a fresh temp dir that is deleted afterwards; home and
|
||||||
TMPDIR point at it; the environment is scrubbed to a small
|
temp environment variables point at it; the environment is
|
||||||
allowlist.
|
scrubbed to a small allowlist.
|
||||||
4. Limits wall-clock timeout, RLIMIT_CPU/AS/FSIZE/NPROC on POSIX,
|
4. Limits wall-clock timeout, RLIMIT_CPU/AS/FSIZE/NPROC on POSIX,
|
||||||
truncated output.
|
truncated output.
|
||||||
5. Network on Linux, `unshare -rn` when unprivileged user namespaces are
|
5. Network on Linux, `unshare -rn` when unprivileged user namespaces are
|
||||||
@@ -40,6 +40,7 @@ import shutil
|
|||||||
import subprocess
|
import subprocess
|
||||||
import sys
|
import sys
|
||||||
import tempfile
|
import tempfile
|
||||||
|
from functools import lru_cache
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
# Wall clock. Compilation gets its own, larger budget: rustc on a cold cache
|
# Wall clock. Compilation gets its own, larger budget: rustc on a cold cache
|
||||||
@@ -186,6 +187,52 @@ def _erlang_write_source(source: str) -> str:
|
|||||||
return source
|
return source
|
||||||
|
|
||||||
|
|
||||||
|
@lru_cache(maxsize=None)
|
||||||
|
def _compiled_tool(lang: str, candidates: tuple[str, ...]) -> str | None:
|
||||||
|
"""Resolve a compiler, verifying the complete Windows toolchain once.
|
||||||
|
|
||||||
|
A compiler executable alone is not a usable toolchain on Windows: rustc's
|
||||||
|
MSVC target also needs Microsoft's linker, and an MSYS2 driver can remain on
|
||||||
|
PATH after one of its runtime DLLs has broken. Both cases otherwise make the
|
||||||
|
capability monitor say "ready" and turn every snippet into a compile error.
|
||||||
|
POSIX keeps the cheap historical which(1) check; the release hosts there
|
||||||
|
install compiler packages atomically.
|
||||||
|
"""
|
||||||
|
found = [tool for name in candidates if (tool := shutil.which(name))]
|
||||||
|
if sys.platform != "win32":
|
||||||
|
return found[0] if found else None
|
||||||
|
for tool in found:
|
||||||
|
if _probe_compiled_tool(lang, tool):
|
||||||
|
return tool
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _probe_compiled_tool(lang: str, tool: str) -> bool:
|
||||||
|
"""Compile a minimal known-good program with the runner's real child env."""
|
||||||
|
source = {
|
||||||
|
"c": "int main(void){return 0;}",
|
||||||
|
"cpp": "int main(){return 0;}",
|
||||||
|
"rust": "fn main() {}",
|
||||||
|
}[lang]
|
||||||
|
try:
|
||||||
|
with tempfile.TemporaryDirectory(prefix="nexus-toolchain-") as tmp:
|
||||||
|
workdir = Path(tmp)
|
||||||
|
entry = RUN_LANGS[lang]
|
||||||
|
src = workdir / entry["source_name"](source)
|
||||||
|
src.write_text(source, encoding="utf-8")
|
||||||
|
exe = str(workdir / "probe.exe")
|
||||||
|
built = _spawn(
|
||||||
|
entry["compile"](tool, str(src), exe),
|
||||||
|
workdir,
|
||||||
|
_child_env(lang, workdir),
|
||||||
|
COMPILE_TIMEOUT,
|
||||||
|
constrain_memory=False,
|
||||||
|
)
|
||||||
|
return built.returncode == 0 and Path(exe).is_file()
|
||||||
|
except (OSError, subprocess.SubprocessError):
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
# The one place that says which languages can be executed. Each entry owns that
|
# The one place that says which languages can be executed. Each entry owns that
|
||||||
# language's screening, toolchain probe and argv. The tool schema's `lang` enum,
|
# language's screening, toolchain probe and argv. The tool schema's `lang` enum,
|
||||||
# the capability line in the system prompt and the dispatch below are all derived
|
# the capability line in the system prompt and the dispatch below are all derived
|
||||||
@@ -211,7 +258,7 @@ RUN_LANGS: dict[str, dict] = {
|
|||||||
},
|
},
|
||||||
"c": {
|
"c": {
|
||||||
"summary": "single-file C program (C11, libm linked)",
|
"summary": "single-file C program (C11, libm linked)",
|
||||||
"tool": lambda: shutil.which("cc") or shutil.which("gcc") or shutil.which("clang"),
|
"tool": lambda: _compiled_tool("c", ("cc", "gcc", "clang")),
|
||||||
"install": "install a C compiler (clang or gcc)",
|
"install": "install a C compiler (clang or gcc)",
|
||||||
"source_name": _source_name("main.c"),
|
"source_name": _source_name("main.c"),
|
||||||
"compile": lambda cc, src, exe: [cc, "-std=c11", "-O0", "-Wall", "-o", exe, src, "-lm"],
|
"compile": lambda cc, src, exe: [cc, "-std=c11", "-O0", "-Wall", "-o", exe, src, "-lm"],
|
||||||
@@ -221,7 +268,7 @@ RUN_LANGS: dict[str, dict] = {
|
|||||||
},
|
},
|
||||||
"cpp": {
|
"cpp": {
|
||||||
"summary": "single-file C++ program (C++17)",
|
"summary": "single-file C++ program (C++17)",
|
||||||
"tool": lambda: shutil.which("c++") or shutil.which("g++") or shutil.which("clang++"),
|
"tool": lambda: _compiled_tool("cpp", ("c++", "g++", "clang++")),
|
||||||
"install": "install a C++ compiler (clang++ or g++)",
|
"install": "install a C++ compiler (clang++ or g++)",
|
||||||
"source_name": _source_name("main.cpp"),
|
"source_name": _source_name("main.cpp"),
|
||||||
"compile": lambda cc, src, exe: [cc, "-std=c++17", "-O0", "-Wall", "-o", exe, src],
|
"compile": lambda cc, src, exe: [cc, "-std=c++17", "-O0", "-Wall", "-o", exe, src],
|
||||||
@@ -231,7 +278,7 @@ RUN_LANGS: dict[str, dict] = {
|
|||||||
},
|
},
|
||||||
"rust": {
|
"rust": {
|
||||||
"summary": "single-file Rust program (2021 edition, std only)",
|
"summary": "single-file Rust program (2021 edition, std only)",
|
||||||
"tool": lambda: shutil.which("rustc"),
|
"tool": lambda: _compiled_tool("rust", ("rustc",)),
|
||||||
"install": "install Rust (https://rustup.rs)",
|
"install": "install Rust (https://rustup.rs)",
|
||||||
"source_name": _source_name("main.rs"),
|
"source_name": _source_name("main.rs"),
|
||||||
# Debug build: -O roughly triples compile time for snippets that run for
|
# Debug build: -O roughly triples compile time for snippets that run for
|
||||||
@@ -301,8 +348,17 @@ def _child_env(lang: str, workdir: Path) -> dict:
|
|||||||
if default.is_dir():
|
if default.is_dir():
|
||||||
env["RUSTUP_HOME"] = str(default)
|
env["RUSTUP_HOME"] = str(default)
|
||||||
env.setdefault("CARGO_HOME", str(Path.home() / ".cargo"))
|
env.setdefault("CARGO_HOME", str(Path.home() / ".cargo"))
|
||||||
env["HOME"] = str(workdir)
|
scratch = str(workdir)
|
||||||
env["TMPDIR"] = str(workdir)
|
env["HOME"] = scratch
|
||||||
|
env["TMPDIR"] = scratch
|
||||||
|
# Windows ignores HOME/TMPDIR in its standard path helpers. Without these,
|
||||||
|
# expanduser() reaches the real profile and GetTempPath() falls back to the
|
||||||
|
# Windows directory; GCC and rustc then either escape the scratch directory
|
||||||
|
# or fail because a normal user cannot write there.
|
||||||
|
env["TEMP"] = scratch
|
||||||
|
env["TMP"] = scratch
|
||||||
|
if os.name == "nt":
|
||||||
|
env["USERPROFILE"] = scratch
|
||||||
env.setdefault("LC_ALL", "C.UTF-8")
|
env.setdefault("LC_ALL", "C.UTF-8")
|
||||||
return env
|
return env
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,23 @@
|
|||||||
|
"""Test-process isolation for persistent runtime state."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import os
|
||||||
|
import shutil
|
||||||
|
import sys
|
||||||
|
import tempfile
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
|
||||||
|
# curry_store constructs its SQLite singleton during test collection. Point it
|
||||||
|
# at a per-run directory before any test module imports synapse, so the release
|
||||||
|
# gate is repeatable and never writes test constants into the checkout's live
|
||||||
|
# data/curry.db.
|
||||||
|
_TEST_STATE = Path(tempfile.mkdtemp(prefix="nexus-pytest-"))
|
||||||
|
os.environ["NEXUS_CURRY_DB"] = str(_TEST_STATE / "curry.db")
|
||||||
|
|
||||||
|
|
||||||
|
def pytest_sessionfinish(session, exitstatus):
|
||||||
|
module = sys.modules.get("synapse.curry_store")
|
||||||
|
if module is not None:
|
||||||
|
module.curry_db.close()
|
||||||
|
shutil.rmtree(_TEST_STATE, ignore_errors=True)
|
||||||
@@ -232,10 +232,40 @@ def test_home_points_at_the_scratch_dir():
|
|||||||
assert "nexus-run-" in out["stdout"]
|
assert "nexus-run-" in out["stdout"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_temp_points_at_the_scratch_dir():
|
||||||
|
"""Compilers and snippets must not fall back to a host temp directory."""
|
||||||
|
out = code_run.run("python", "import tempfile\nprint(tempfile.gettempdir())")
|
||||||
|
assert "nexus-run-" in out["stdout"]
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# Toolchains
|
# Toolchains
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def test_windows_compiler_resolver_skips_broken_candidates(monkeypatch):
|
||||||
|
"""An executable on PATH is not enough when its linker/runtime is broken."""
|
||||||
|
code_run._compiled_tool.cache_clear()
|
||||||
|
monkeypatch.setattr(code_run.sys, "platform", "win32")
|
||||||
|
monkeypatch.setattr(
|
||||||
|
code_run.shutil, "which", lambda name: f"C:\\tools\\{name}.exe"
|
||||||
|
)
|
||||||
|
probes = []
|
||||||
|
|
||||||
|
def probe(lang, tool):
|
||||||
|
probes.append((lang, tool))
|
||||||
|
return tool.endswith("clang.exe")
|
||||||
|
|
||||||
|
monkeypatch.setattr(code_run, "_probe_compiled_tool", probe)
|
||||||
|
try:
|
||||||
|
assert code_run._compiled_tool("c", ("gcc", "clang")) == "C:\\tools\\clang.exe"
|
||||||
|
assert probes == [
|
||||||
|
("c", "C:\\tools\\gcc.exe"),
|
||||||
|
("c", "C:\\tools\\clang.exe"),
|
||||||
|
]
|
||||||
|
finally:
|
||||||
|
code_run._compiled_tool.cache_clear()
|
||||||
|
|
||||||
|
|
||||||
def test_a_missing_toolchain_explains_itself(monkeypatch):
|
def test_a_missing_toolchain_explains_itself(monkeypatch):
|
||||||
"""The model has to be able to tell 'you cannot run this here' from 'your
|
"""The model has to be able to tell 'you cannot run this here' from 'your
|
||||||
code is wrong' — otherwise it rewrites a correct program repeatedly."""
|
code is wrong' — otherwise it rewrites a correct program repeatedly."""
|
||||||
|
|||||||
Reference in New Issue
Block a user