feat(rag): overlapping chunker, retrieval knobs, sqlite-vec index

- Chunker: char overlap across boundaries + hard-split of oversized paragraphs.
- Retrieval knobs: rag_top_k / rag_min_score in settings + Settings UI.
- Vector index: sqlite-vec ANN over document embeddings, dual-written and
  backfilled, with brute-force cosine as the guaranteed fallback.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
jon
2026-07-23 13:58:05 -05:00
co-authored by Claude Opus 4.8
parent ac0eb1e5f6
commit f4aea78b55
6 changed files with 216 additions and 11 deletions
+32
View File
@@ -6,6 +6,8 @@ const DEFAULTS = {
think: false, // Qwen3-style reasoning; off = much faster chat/memory
temperature: 0.7,
num_ctx: 0, // context window in tokens; 0 = model default
rag_top_k: 3, // document chunks injected into chat
rag_min_score: 0.6, // min cosine similarity for a chunk to count
system_prompt: "",
timeout: 120,
gpu_offload: -1, // -1 = Auto; 0100 = percent of layers forced onto the GPU
@@ -209,6 +211,36 @@ export function Settings() {
</div>
</div>
<div style={{ marginTop: "1.25rem", display: "flex", gap: "1rem" }}>
<div style={{ flex: 1 }}>
<label style={labelStyle}>Document chunks (top-k)</label>
<input
type="number" min="0" step="1"
value={form.rag_top_k}
onChange={e => update("rag_top_k", Math.max(0, parseInt(e.target.value) || 0))}
style={{ width: "100%", padding: "0.6rem", background: "#222", color: "#eee", border: "1px solid #333", borderRadius: "8px", boxSizing: "border-box" }}
/>
</div>
<div style={{ flex: 1 }}>
<label style={labelStyle}>
Min relevance
<span style={{ float: "right", color: "#7aa", fontWeight: 600, textTransform: "none", letterSpacing: 0 }}>
{Number(form.rag_min_score).toFixed(2)}
</span>
</label>
<input
type="range" min="0" max="1" step="0.05"
value={form.rag_min_score}
onChange={e => update("rag_min_score", parseFloat(e.target.value))}
style={{ width: "100%", accentColor: "#007acc" }}
/>
</div>
</div>
<div style={{ fontSize: "0.72rem", color: "#555", marginTop: "0.2rem" }}>
How many uploaded-document chunks to pull into chat, and the minimum cosine
similarity each must clear. Higher relevance = fewer, tighter matches.
</div>
<div style={{ marginTop: "1.25rem" }}>
<label style={labelStyle}>
GPU Offload