# Contextum: documents for AI agents without silent losses

Contextum prepares documents for an LLM: Markdown conversion, context compression and a loss registry. Go and .NET, Docker — for teams building agents on documents.

> For the index of this section of the site, see [llms.txt](https://anetlab.com/llms.txt)

POCLicense: On request

# Contextum

Documents for an AI agent: Markdown, compression and a loss registry

A model reads not the document but the text it was turned into. Contextum returns that text with a list of what was lost on the way, so the losses are visible before the answer reaches the customer.

## What it is

Contextum is a service that prepares documents for an LLM. It takes a file, turns it into Markdown, shrinks the content to fit a context window and hands the result back together with a list of what did not survive the conversion.

The product owns files and nothing else. Dialogue orchestration, memory, model selection and tools stay with the agent layer — Contextum does not touch them, and it does not set out to replace an agent platform either.

## Who it is for

Teams building an agent on top of documents: contracts, specifications, invoices, regulations, reporting.

The buyer is an organisation, not a lone developer: this is a B2B product. The typical buyers:

*   a company with a working LLM contour, where documents reach the model as a plain string;
*   a company that is building an LLM contour;
*   an AI provider boosting its platform and giving its users access through it;
*   an AI portal application or an internal AI project that works with documents.

The usual picture looks like this: the agent receives a PDF, a library returns a string, and the string looks sensible. Everything Markdown cannot express — tables, merged cells, colour, underline, super- and subscript, charts, vector images — disappears without a word. You find out later: in an answer that already reached the customer, or in a claim that a clause was missing from the contract. Checking such answers by hand is expensive, and detecting the loss after the fact is close to impossible.

Contextum changes three things:

*   **The result can be checked.** The loss registry comes back with the conversion: you see what did not make it, not only what did.
*   **A long document fits the window.** Compression shrinks the text by the strategy’s rules, and the saving is measured on your own corpus: the number comes from a measurement, not from an example in the docs.
*   **The agent does not stall.** Conversion is CPU-bound and runs in a service of its own: model calls do not wait for a render to finish.

## How it works

graph TD
  U\[User\] --> A\[AI agent\]
  A -->|file| G\[Contextum\]
  G --> C\[CONVERT\]
  C --> S\[COMPRESS\]
  S -->|artifacts| A
  A -->|context| L\[LLM\]
  G -.-> E\[EMBED/INDEX/VAULT\]
  classDef accent fill:#eef7f0,stroke:#008A2A,stroke-width:1.5px,color:#141616
  classDef muted fill:#f7f8f8,stroke:#c8cdcd,stroke-width:1px,color:#505757
  class G accent
  class U,L,E muted

Diagram 01The path of a document from agent to model

The path of a file: the agent hands it to the Contextum gateway → CONVERT turns the document into Markdown → COMPRESS shrinks the text and splits it into chunks → artifacts go back to the agent → the agent puts them into the model’s context. The loss registry travels with the conversion: it is in the converter’s result and in the worker log. The other three stages branch off the gateway — EMBED, INDEX and VAULT; today they are stubs: they run, but they do not do what they claim.

Rendering pages, parsing tables and building Markdown live in separate processes; the gateway holds the job state. Between the gateway and the processes runs gRPC over a protobuf contract — a single source of truth for Go and .NET.

You can tell a stub from a real stage right in the response: the process returns `simulated = true`, and that flag tells an integrator what can be used as a source of truth.

## What it covers

|  | Value |
| --- | --- |
| Input | PDF, DOCX, XLSX, PPTX, Markdown, HTML, plain text — up to 64 MiB |
| Output | GFM Markdown, compressed Markdown, chunks; `converted.md` and `compressed.md` come from the real stages, `embeddings.manifest.md`, `index-receipt.md` and `vault-article.md` from the simulator stages |
| Transport | HTTP to the gateway, gRPC with a protobuf contract to the worker processes |
| Deployment | Go and .NET, Docker, NATS JetStream, Postgres, Redis |

The loss registry is visible in the converter’s response (`POST /api/v1/convert`) and lists what did not survive: colour, font, underline, super- and subscript, alignment, merged cells, charts, vector images. What was disabled by an option — formulas with `includeCellFormulas = false`, for example — is not a loss: that is the user’s choice, not a conversion defect.

PPTX is converted to the MARP dialect, which expresses more than GFM does: slide background colour, background image, text colour, deck size. A presentation loses less, and that is a property of the format rather than an oversight.

## Where it fits

*   **Contracts and specifications.** The agent answers questions about an uploaded contract, and the recipient sees which formatting elements did not reach the answer.
*   **Technical documentation.** DOCX and PDF become Markdown that is good enough for a knowledge base and for a human to read.
*   **Tables and reporting.** XLSX → Markdown, then into a RAG pipeline or into an agent’s analysis.
*   **Presentations.** PPTX → MARP: slides keep more formatting than documents do.

## What is measured

Measurement and estimate are separated in the product, and the numbers show it:

*   **Bytes, stage duration, chunk count** — facts.
*   **Tokens** — an estimate based on text length (“1 token ≈ 4 characters”), not model tokenisation. Useful for a rough idea of how much context an agent will consume, useless for exact billing.
*   **Only the compression stage claims savings.** The summary comes from the result of the last measured stage, not from the maximum across stages.

We do not publish savings percentages until there is a measurement on a real corpus: an example in the documentation is an example, not a measurement. A closer look at the metrics — in the [product deep dive](https://anetlab.com/blog/contextum-document-pipeline.md).

## Cases

Contextum has no production deployments yet: the product is in POC status, there has been no public release, and selling it as a proven solution would be a lie. What there is — measurements, and a public breakdown of the architecture and the loss registry in the blog and in the product documentation.

The team’s closest experience in the same class of problems is the «До подписи» SaaS: an AI audit of contracts where an uploaded PDF is parsed and turned into a list of risks. That is a different product and it sits in the [Portfolio](https://anetlab.com/projects.md), but it shows that we take such work to production.

## Honest limits

The list of limitations is part of the product, not a footnote. It answers the question “can this be embedded today”, and for some items the answer is no.

*   **No authentication** — not over HTTP, not over gRPC, not over NATS. Acceptable only inside a trusted network.
*   **No TLS or mTLS** — the network is assumed trusted.
*   **No recognition.** OCR and scanned pages are not claimed: the contract has neither a recognition engine nor an `ocr` loss kind.
*   **PDF page anchors are a loss, not a feature.** Anchors exist in the source PDF but may not survive into Markdown.
*   **The upload limit is 64 MiB.**
*   **Progress is polling only.** No webhooks and no SSE.
*   **No automatic artifact cleanup** — the integrator needs a TTL collector.
*   **One process, one role.** No vertical scaling.
*   **The vector dimension and the embedding model are undecided.**
*   **Job state survives a restart only when `ODOC_POSTGRES_DSN` is configured.** Without it the state lives in memory and is lost.

On the audience — honestly: this is a B2B product, and it is bought where an LLM stack already exists or is being built. The common denominator across all of them is a need for a separate conversion and compression service: a loss registry, metrics from a measurement, scaling apart from the orchestrator. Where the task is one-off — running a single file through — a service is not needed.

## How to start

```
curl -sS -X POST http://localhost:55568/api/v1/jobs \
  -F 'file=@contract.pdf;type=application/pdf' \
  -F 'options={"targetModel":"gpt-4o","compressionLevel":"COMPRESSION_LEVEL_BALANCED","stages":["WORKER_ROLE_CONVERT","WORKER_ROLE_COMPRESS"]}'
```

The response carries a `jobId` and a status. State is polled through `GET /api/v1/jobs/{id}`, artifacts are downloaded through `GET /api/v1/jobs/{id}/artifacts/{artifactId}`. The integrator needs Docker and Postgres with `ODOC_POSTGRES_DSN` configured — otherwise job state does not survive a restart.

## Related materials

*   [Partners: the joint project with Office Doc](https://anetlab.com/partners.md)
*   [Contextum: the document pipeline to Markdown for AI agents](https://anetlab.com/blog/contextum-document-pipeline.md)
*   [Building a RAG pipeline on .NET: a practical walkthrough](https://anetlab.com/blog/rag-pipeline-dotnet.md)
*   [RAG on .NET: an architecture that survives production](https://anetlab.com/blog/rag-na-dotnet.md)
*   [Production RAG: a 50-point checklist for production](https://anetlab.com/blog/production-rag-checklist.md)

## CTA

Planning how to embed a document layer in an agent system — [write to us](https://anetlab.com/contact.md). We can look at which stages you actually need, which limitations are critical for your setup and how this sits next to the orchestration layer. While the product is in POC, we will talk about what it does and what it does not do.

FAQ

## Frequently asked questions

Can we deploy this inside our own perimeter?+

Yes. Go and .NET, Docker, NATS JetStream, Postgres and Redis all run on your side, and the documents do not leave it. One caveat: there is no authentication and no TLS, so the gateway belongs inside a trusted network or behind your own authenticating reverse proxy.

What happens to the formatting of a document?+

Markdown does not express everything: colour, underline, super- and subscript, alignment, merged cells, charts and vector images may not survive. Whatever did not make it goes into the loss registry. What was disabled by an option — for example formulas with includeCellFormulas = false — is not a loss.

Which formats are accepted?+

PDF, DOCX, XLSX, PPTX, Markdown, HTML and plain text, up to 64 MiB. PPTX is converted to the MARP dialect, which expresses more than GFM does.

Can we embed this in production right now?+

As a conversion and compression layer — yes. Embeddings, the vector index and knowledge-base writes are stubs today: they execute but do not do what they claim, and they carry simulated = true in the response. On top of that there is no authentication, no recognition and no automatic artifact cleanup.

What does it cost?+

The license is available on request. Terms and price are discussed against your setup.