Skip to content

Alkiviades — AI Pipeline

How a question becomes a grounded, axis-modulated answer.

Overview

User Query
    │
    ▼
┌─────────────────────────────────────────────────────┐
│ 1. RAG RETRIEVAL                                    │
│    TF-IDF vectorize → cosine similarity → top-5     │
│    chunks from the knowledge base                   │
└──────────────────┬──────────────────────────────────┘
                   │ chunks + scores
                   ▼
┌─────────────────────────────────────────────────────┐
│ 2. PROMPT ASSEMBLY                                  │
│    System prompt (current axes) + trimmed history   │
│    + user message (with retrieved context)          │
└──────────────────┬──────────────────────────────────┘
                   │ messages[]
                   ▼
┌─────────────────────────────────────────────────────┐
│ 3. TOOL-CALLING LOOP (max 5 iterations)             │
│    Stream LLM response                              │
│      ├─ content delta  → token event to client     │
│      └─ tool call      → execute tool, append       │
│                          result, loop again         │
└──────────────────┬──────────────────────────────────┘
                   │ final answer
                   ▼
┌─────────────────────────────────────────────────────┐
│ 4. PERSISTENCE                                      │
│    Save the turn (prompt, response, sources,        │
│    tool calls, retrieved chunks) to the database    │
└─────────────────────────────────────────────────────┘

1. Retrieval

At answer time the question is converted into a TF-IDF vector using the same vocabulary the knowledge base was built with, then ranked against every chunk by cosine similarity. The top results — plus their source filenames — become the context attached to the user message. The client also receives a source event per chunk, so the interface can display retrieval provenance.

The knowledge base is built offline: source documents are preprocessed into clean markdown, chunked into overlapping passages, vectorized in float16, and cached to disk. This keeps query-time costs to a single numpy pass — no external embedding API, and identical handling for Greek and English.

2. Prompt Assembly

Three pieces are combined for every turn:

  • System prompt — the agent's role and operating principles, with the three axis descriptions rendered from the current axis values (each snapped to one of 7 discrete positions).
  • History — prior turns of the conversation, summarized if the context approaches the token budget.
  • User message — the question plus the retrieved chunks, each prefixed with its source.

The system prompt also lists the tools available to the agent, so the LLM can decide when to search the web or read a file instead of answering from the knowledge base alone.

3. The Tool-Calling Loop

The response is streamed token-by-token. The loop runs up to 5 iterations:

  1. Stream the LLM response.
  2. Content deltas are forwarded to the client immediately as token events.
  3. If the stream ends with a tool-call signal, the loop continues.
  4. Resolve the tool call (a non-streaming completion is used to fetch the complete tool arguments).
  5. Execute the tool in a thread pool so the async event loop is never blocked.
  6. Append the tool result to the message history and loop again, letting the LLM incorporate it.
  7. On the final iteration tools are disabled and a closing answer is required.

Available Tools

Tool Purpose
search_news Search current news through a vetted set of sources; results are saved and cited
load_search Reload a previously saved search result
scrape_url Fetch and convert a web page to markdown
read_file Read a file from the workspace
write_file / edit_file / append_file Create and modify files in the workspace

4. Streaming Events

The chat endpoint is a Server-Sent Events stream:

SSE Event Meaning
token A content delta of the answer
source A retrieved knowledge-base chunk with its similarity score
tool_call A tool the agent decided to invoke
tool_result The output of that tool
done Stream complete, session id attached
error Something failed — the message explains what
heartbeat Keepalive ping during long tool executions

The agent producer and a heartbeat producer feed a shared queue; the main loop drains it and yields events to the client. On completion the full turn is persisted.

5. History Management

Every turn appends the raw user question and the full assistant answer to the conversation history — RAG chunk context is attached only to the current turn's message, so earlier exchanges stay visible to the model instead of being buried under stale knowledge-base text.

When the total context (system prompt + history + pending turn) approaches the token budget (MAX_CONTEXT_TOKENS, 120K by default, env-configurable), older messages are condensed into a progressive structured recap (topics, key facts/names/dates, decisions, open questions) that merges the previous summary — details survive successive trims. The most recent SUMMARY_KEEP_LAST_N (6) messages stay verbatim. If summarization fails, older messages are simply dropped — the conversation always proceeds.

Replies cut off by the token limit (finish_reason="length") are auto-continued: the agent issues up to 3 "continue where you left off" calls and stitches the result, so truncated text never enters history or persistence as a complete turn.

6. Reports

Reports run the same machinery in three explicit phases: the session is analyzed for themes and gaps, the gaps are researched through retrieval and web search, and a structured markdown report is synthesized — each phase building on the previous one, with all claims carrying source citations.