Local AI Assistant
Product Shape
Local-first AI assistant that can run models, connect to tools, isolate workspaces, ingest documents, and expose a local API.
Ladder Result
Pick an AI-native desktop/local service stack, not just a chat UI.
Stack Implications
| Constraint | Stack implication |
|---|---|
| Local models | Model runtime/engine boundary and hardware detection |
| MCP tools | Connector scope, subprocess management, tool permissions |
| Workspaces | Separate documents, prompts, tools, vectors, model overrides |
| Local state | SQLite for sessions, memory, audit logs, durable metadata |
| Tool execution | Sandbox, inspection pipeline, audit events |
| Local API | OpenAI-compatible or app-specific API surface |
Jan vs AnythingLLM Lessons
- Jan shows a Tauri/Rust desktop shell, bundled local inference, MCP host capability, and local OpenAI-compatible API.
- AnythingLLM shows a heavier orchestration platform with workspaces, RAG, provider breadth, agent flows, and multi-user Docker mode.
- MyLocalGPT-style products should borrow the isolation and tool model without inheriting a heavy service stack too early.
First Quality Gates
- Workspace data and tools do not cross-contaminate.
- MCP/tool execution is scoped and auditable.
- Local state survives restart and is queryable.
- Secrets are not stored in mutable runtime blobs.
- Risky tools run in a sandbox or explicit restricted subprocess path.