Concepts
How the store works
The Nauro store is a plain-text, local-first project record: human-readable markdown decision files plus a few state files on disk, with an in-memory BM25 index built per call for retrieval and an optional, fully local embedding augmenter. The core loop makes no external network calls of its own; the only model call in a session is the connected agent's own model reading the record Nauro hands it. This page covers the on-disk layout, retrieval ranking, write and snapshot mechanics, and privacy posture, all traceable to the Nauro-AI/nauro source. As of 1.0, the on-disk store format is a stable contract that follows semantic versioning. The layout, file formats, and parsing rules described here will not change incompatibly within the 1.x line.
What the store is
- The store is the project's persisted record: architectural decisions (with rejected alternatives), current state, stack, and open questions, all as plain markdown on disk. There is no database in the local path.
- It is local-first: adopt, init, and the decision check run on your machine with no account, and cloud sync is explicitly optional.
- The on-disk decision format is a single source of truth in nauro-core (
decision_model.format_decision/parse_decision); both the CLI and the hosted server serialize through it so the formats cannot drift. The scaffolded first decision is also emitted viaformat_decisionrather than a string template, so the format never diverges. - Operations are written against a minimal storage Protocol (
nauro_core.operations.store.Store) with six primitives:read_file,write_file,delete_file,list_decisions,read_decision, and the bulkread_decisions(the bulk analogue ofread_decision, which lets the hosted server fan decision reads out concurrently instead of one S3 GET at a time). The local CLI/stdio MCP supplies a filesystem implementation; the hosted server supplies its own (S3 + DynamoDB, per the Protocol docstring). The same operation code runs on both. - nauro-core has only four runtime dependencies (
bm25s>=0.2,PyStemmer>=2.2,pydantic>=2.0,PyYAML>=6.0) and is published separately (PyPInauro-core) so third-party tools can read or write the Nauro decision format.
@runtime_checkable
class Store(Protocol):
def read_file(self, path: str) -> str | None: ...
def write_file(self, path: str, content: str) -> None: ...
def delete_file(self, path: str) -> None: ...
def list_decisions(self) -> list[str]: ...
def read_decision(self, file_stem: str) -> str | None: ...
def read_decisions(self, stems: list[str]) -> dict[str, str | None]: ...On-disk layout
- Two locations: a small committed pointer at
<repo>/.nauro/config.jsonnaming the project it belongs to, and the store content itself outside the repo at~/.nauro/projects/<id>/(underNAURO_HOME, ULID-keyed in the canonical v2 layout). - A store holds
project.md,state_current.md,stack.md,open-questions.md, oneNNN-slug.mdmarkdown file per decision underdecisions/, versioned JSON captures undersnapshots/, and the.decision-hashes.jsonexact-duplicate index. - Each decision file is strict YAML frontmatter plus a markdown body (
# NNN — Title, a required## Decisionsection, an optional## Rejected Alternativessection). Supersession is recorded on both sides (supersedeson the new decision,superseded_byon the old one) and nothing is ever deleted. - The full reference (directory tree, naming and slug rules, frontmatter order, example files, control-plane formats) is on the Data storage page; the field schema and validation rules are on Core concepts.
Retrieval: BM25 by default
- The default and always-available retrieval is lexical BM25, implemented in
nauro_core/search.pyon top of thebm25slibrary with a PyStemmer English stemmer (_stemmer = Stemmer.Stemmer("english")). Both are hard dependencies of nauro-core, so BM25 needs no optional install and no external service or API key. - The index is built in-memory per call, not persisted:
bm25s.tokenizeover the corpus,retriever.index(...), thenretriever.retrieve(query_tokens, k=...). For stores in the hundreds of decisions this is cheap. There is no vector DB and no persisted index. - The indexed text per decision is its title + rationale:
corpus = [f"{d.title} {d.rationale}" for d in decisions]. Tokenization uses English stopwords and the shared stemmer; bm25s progress bars are disabled (show_progress=False) so they don't pollute the CLI surface. - Results are ranked by BM25 score descending and any hit with score <= 0 is dropped (the loop breaks on the first non-positive score). There is no fixed similarity threshold beyond that "score must be positive" cutoff.
- There are two BM25 entry points.
bm25_searchranks across all decisions and returnsnumber/title/date/status/relevance_snippet/score(used bysearch_decisions);bm25_retrieverestricts to active decisions only and returnsnumber/title/similarity/rationale_preview(used to surface related active decisions for agent assessment). Both round their scores to 3 decimals. check_decisionand Tier-2 proposal validation extend bm25s's default English stopword list with the tokenuse(TIER2_STOPWORDS = [*list(STOPWORDS_EN), "use"]and_CHECK_DECISION_STOPWORDS = [*list(STOPWORDS_EN), "use"]). Decision titles like "Use Postgres", "Use Redis" otherwise share the stemusewith nearly every decision and surface as false near-neighbours on every call.- Both retrieval surfaces exclude the scaffold-seeded first decision (
num == 1with title "Initial project setup") so Nauro's own bookkeeping entry never gates a real proposal. - The assessment line
check_decisionreturns is deterministic and built purely from retrieval facts (top match label, status, date, BM25 score); there is no model in the loop, and it directs the agent to callget_decisionbefore proposing.
corpus = [f"{d.title} {d.rationale}" for d in decisions]
corpus_tokens = bm25s.tokenize(corpus, stopwords="en", stemmer=_stemmer, show_progress=False)
retriever = bm25s.BM25()
retriever.index(corpus_tokens, show_progress=False)
k = min(limit, len(decisions))
query_tokens = bm25s.tokenize([query], stopwords="en", stemmer=_stemmer, show_progress=False)
results, scores = retriever.retrieve(query_tokens, k=k, show_progress=False)
# ... only hits with score > 0 are kept (the loop breaks on the first score <= 0)Optional local embeddings (off by default)
- Embeddings are an optional augmenter, not a replacement for BM25. They live in
nauro_core/embeddings.py, isolated from search.py so the BM25 path carries no embedding imports. - The model is
minishlab/potion-retrieval-32M(EMBEDDING_MODEL), a Model2Vec static embedding. The module comment is explicit: numpy-only, no torch, no ONNX runtime. The model loads locally and runs as a numpy matmul; there is no external service call and no API key. - It ships behind an optional extra:
nauro-core[embeddings] = model2vec>=0.3, numpy>=1.24.embeddings_available()returns False if model2vec/numpy aren't importable. - It is OFF by default.
resolve_embeddings_flag()(innauro/store/config.py) returns False unless theNAURO_EMBEDDINGSenv var or thesearch.embeddingsconfig key is truthy (1/true/yes/on, case-insensitive; a native bool from config is also accepted). Env wins over config. The kernel itself stays I/O-free; the adapter resolves the flag and passesuse_embeddingsintocheck_decision. - When enabled, retrieval is a union:
union_retrievereturns the BM25 top-k first, in BM25 order and shape, then appends any embedding-top-k decision BM25 didn't already surface. Embedding-sourced hits carrysimilarity=None(the static-embedding cosine is not on the BM25 score scale);check_decisionsurfaces them with score 0.0 to signal "not a BM25 match". - It is fail-open. If the dependency is absent or the model fails to load,
_get_modelshort-circuits (records_load_failed, logged once at WARNING) andunion_retrieve/embedding_poolreturn the BM25-only result. Withuse_embeddingsFalse,union_retrieveis byte-identical tobm25_retrieve. - Ranking inside
embedding_pool: encode title+rationale of each decision and the query, L2-normalize, take the dot product as cosine, argsort, take the last k and reverse for descending order, return the top-k decision numbers. The loaded model is memoized per process (in-memory only, not a persisted artifact); encoding is per call.
bm25_hits = bm25_retrieve(decisions, query_text, top_k=top_k, stopwords=stopwords)
if not use_embeddings:
return bm25_hits
from nauro_core.embeddings import embedding_pool
active = [d for d in decisions if d.status is DecisionStatus.active]
pool = embedding_pool(active, query_text, top_k=top_k)
if not pool:
return bm25_hits # fail-open: BM25-only
seen = {hit["number"] for hit in bm25_hits}
by_num = {d.num: d for d in active}
augmented = list(bm25_hits)
for num in pool:
if num in seen:
continue
d = by_num.get(num)
if d is None:
continue
seen.add(num)
augmented.append({"number": d.num, "title": d.title,
"similarity": None, "rationale_preview": d.rationale[:200]})def resolve_embeddings_flag() -> bool:
env_value = os.environ.get(NAURO_EMBEDDINGS_ENV) # NAURO_EMBEDDINGS
if env_value is not None:
return _is_truthy(env_value)
return _is_truthy(get_config(_EMBEDDINGS_CONFIG_KEY)) # "search.embeddings"
# _is_truthy: True for bool True, or "1"/"true"/"yes"/"on" (case-insensitive); default OFFWrites: local locking and best-effort multi-file updates
propose_decisionruns a validation pipeline before any write: Tier 1 structural screening (empty title/rationale, rationale shorter thanMIN_RATIONALE_LENGTH=20, invalid confidence, exact SHA-256 hash duplicate, same-title duplicate of an existing active decision) can reject; Tier 2 BM25 similarity is advisory only and does not block the write. The chat-session layer must obtain explicit approval for every add, update, or supersede before making the call. The kernel writes on structural success with no second confirmation step.- After an exact-clean write, the in-store
.decision-hashes.jsonindex is updated (SHA-256 of normalizedtitle|rationale, lowercased + stripped) so subsequent Tier 1 checks catch the duplicate. - On the local adapter,
decision_write_lockholdsdecisions/.lockacross the full max+1 allocation and kernel write, preventing two local writers from minting the same number. Per-target locks still protect individual file writes. - The cloud store has no compare-and-swap primitive or equivalent cross-request lock. Cross-machine convergence therefore assumes sequential use by one owner. A supersede is a two-write sequence (new decision, then flip the old to
supersededwithsuperseded_by); if the second write fails, the returned half-state requires manual recovery. There is no automatic rollback or repair. - The write mechanics themselves (the allocation lock, per-file locks, atomic tmp-write-plus-rename for control-plane JSON, the
0o600permission onconfig.json, the path-traversal guard on reads) are documented with code on the Data storage page.
Snapshots: point-in-time captures with logarithmic pruning
A snapshot is one JSON file capturing the full store, written to snapshots/vNNN.json on every write-tool call and on nauro sync, then pruned on a logarithmic schedule (everything for a week, daily for a month, weekly for six months, monthly beyond) with two guarantees: the latest snapshot is always kept, and any snapshot that grew the decision set is pinned forever. Snapshots are what diff_since_last_session diffs against. The capture format, key order, and prune buckets are documented with code on the Data storage page.
Privacy posture
- The local store's core loop makes no external network calls. Retrieval (BM25, and the optional embeddings) runs entirely on your machine; the embedding model is a local numpy-only static model with no API key. The only model call in a session is the connected agent's own model reading the context Nauro provides.
- Local-first: adopt, init, and related-decision retrieval run fully on your machine with no account. Cloud sync is opt-in (
nauro auth login+nauro link --cloud+nauro sync). - When cloud sync is enabled, PRIVACY.md states only project context (decisions, state, open questions, not source code) is stored encrypted in AWS S3 (us-east-1, SSE-S3), and access to each project is isolated by a fail-closed membership check: a request can read or write a project only if your account is recorded as a member of it. There is no self-service deletion command at this time; removal is via support.
- Over remote MCP (Claude AI, Perplexity, or another MCP client), your project context is read from S3 and delivered to the connected AI tool; that tool's own data policies then govern what it does with the response. Nauro does not monitor downstream use.
- Nauro sends no product analytics. The local CLI and hosted service emit no product-usage events.
- During 1.x,
nauro telemetry status,enable,disable, andresetare deprecated no-network, no-mutation compatibility shims. They report retirement and disappear in 2.0. Legacy telemetry config is ignored and left untouched. Lambda operational metrics remain in CloudWatch and never include user content.
nauro telemetry status
nauro telemetry enable
nauro telemetry disable
nauro telemetry reset
Product telemetry has been removed. This deprecated compatibility command
makes no changes and will be removed in Nauro 2.0.Key facts
- Default retrieval is BM25 (bm25s + PyStemmer English stemmer), built in-memory per call; both are required nauro-core deps, so no optional install, no external service, no API key.
- Indexed text per decision is title + rationale; results sorted by BM25 score descending and any hit with score <= 0 is dropped. Scores are rounded to 3 decimals.
- Optional embeddings model:
minishlab/potion-retrieval-32M(Model2Vec static, numpy-only, no torch, no ONNX), shipped as thenauro-core[embeddings]extra (model2vec>=0.3,numpy>=1.24). - Embeddings are OFF by default; enabled only via
NAURO_EMBEDDINGSenv var or thesearch.embeddingsconfig key (truthy tokens: 1/true/yes/on, case-insensitive, or a native bool). Env wins over config. - With embeddings on, retrieval is a BM25-then-embedding union: BM25 hits first in their existing shape/order, embedding-only hits appended with
similarity=None. Fail-open to BM25-only if the dep is absent or the model fails to load. - Canonical store root:
~/.nauro/projects/<id>/(v2, ULID-keyed) underNAURO_HOME; a repo opts in via its committed<repo>/.nauro/config.jsonpointer. Full layout, file formats, write mechanics, and snapshot reference: Data storage. - Local decision writes hold
decision_write_lockacross max+1 allocation and the write. The cloud store has no compare-and-swap primitive or equivalent lock, so cross-machine convergence assumes sequential single-owner use. - Supersede writes are sequential and best-effort. A reported half-state is not rolled back or repaired automatically; recovery is manual.
- Per PRIVACY.md cloud sync stores only context (decisions/state/open questions, not source code) encrypted in AWS S3 us-east-1 SSE-S3, isolated per project behind a fail-closed membership check. Local core loop makes no external network calls.
- Nauro sends no product analytics. The four telemetry subcommands are inert 1.x compatibility shims, and legacy config is ignored and preserved.