I wanted a simple thing. I just wanted to create a demo project where the user can upload a PDF, ask a question about it, and get an answer grounded in the actual document, with citations. I didn't want to deal with API keys, cloud bills, or rate limits. Just a local pipeline I could run on my own hardware and actually understand end to end. What I created is DocQuery Phase 1, and it's live on GitHub right now.

What I didn't expect was how many landmines are sitting between "I'll just wire up a RAG pipeline" and "this actually works reliably." I was surprised when I encountered NuGet name-squatting, ChromaDB nuking its own API, and macOS quietly hijacking ports. NONE of this is in the tutorials, so let me walk through the architecture first, then the war stories, because both matter.

The Architecture: Interfaces First, Swap Later

DocQuery is a C#/.NET 10 Web API with a React (Vite) frontend. All inference runs locally and is free through Ollama, which handles both embeddings (nomic-embed-text) and generation (Llama 3 8B), and then ChromaDB runs in Docker as the vector store.

The part I'm most deliberate about is the abstraction layer. In a Core project, I defined three interfaces: IEmbeddingProvider, ILlmProvider, and IVectorStore. The local implementations sit behind those: OllamaEmbeddingProvider, OllamaLlmProvider, ChromaVectorStore. The reason is Phase 2. I'm going to implement those same interfaces against Azure OpenAI and Azure AI Search, then benchmark local vs. cloud with a single config flag swap. No rewriting the pipeline, no touching the query logic. Just register different implementations at startup.

This is the minimum viable abstraction for a meaningful comparison because if you can't swap the provider without changing the pipeline, your benchmark is measuring two different codebases instead of two different providers.

The Ingestion and Query Pipeline

Ingestion is straightforward but has a few decisions worth explaining. A user uploads a PDF or Markdown file via multipart form. For PDFs, I use PdfPig for text extraction (more on that name in a minute). The extracted text goes through fixed-size chunking of 500 words with a 50-word overlap. Each chunk gets embedded through Ollama's nomic-embed-text model and stored in ChromaDB with source filename, chunk index, original text metadata.

The overlap matters because without it, you lose context at chunk boundaries. A sentence that starts at the end of chunk 12 and finishes at the start of chunk 13 would be invisible to retrieval if you chunk cleanly. Fifty words of overlap is enough to preserve continuity without bloating the store.

Next, the query flow embeds the user's question with the same model, then I do a top-k cosine similarity retrieval against ChromaDB. The returned chunks get assembled into a context window with a system prompt that tells Llama 3 to answer based only on the provided context. The response comes back with per-chunk citations of the source filename, chunk reference, and a relevance score from the vector search.

For testing, I wrote 24 smoke tests using WebApplicationFactory with fake providers injected. The entire test suite passes with zero infrastructure running (No Ollama, ChromaDB, or Docker). This was non-negotiable because if your tests require a GPU and three containers to run, you're not going to run them, and neither is CI.

War Stories: The Stuff Nobody Warns You About

NuGet Name-Squatting Almost Got Me

PdfPig is a solid .NET library for PDF text extraction. The real package ID on NuGet is PdfPig, with 26M+ downloads, maintained by the actual project team. But, there's an unrelated account publishing a look-alike under UglyToad.PdfPig, which is the real project's GitHub organization and namespace name. It's exactly what you'd type if you were guessing. The look-alike has off-scheme versions like 1.7.0-custom-5 and no project URL.

I ALMOST installed the wrong one. So learn from me, ALWAYS verify package owners on the registry page, not just the package name. I now pin exact versions in my .csproj files and check the source link before any install. Supply chain attacks don't have to be sophisticated, they just have to match what you'd expect to type.

ChromaDB Deleted Its Own API

This one cost me some time. Nearly every tutorial out there, and every AI coding assistant, generates ChromaDB calls against /api/v1/* routes. Pull the current Docker image and hit those endpoints and you get 410 Gone. ChromaDB moved to v2 tenant/database routes, and the old surface is just… GONE.

I migrated to the v2 API and configured a cosine HNSW space explicitly. The lesson here is the docs in LLM training data and the blog posts you find are probably stale. These projects move fast and break backward compatibility without much ceremony, make sure to test against the real service early. Don't write your integration layer against tutorial code and hope it works when you spin up the container.

macOS AirPlay Receiver Squats Port 5000

This is a fun one :). On macOS, AirPlay Receiver listens on port 5000 by default, so if you're running your .NET API on 5000 (which is the Kestrel default), and you have a startup script that polls for an HTTP response to know when the API is ready, congratulations! AirPlay is going to answer with a 403, and your script is going to say "API is up!" when it's actually talking to your speaker service.

The was change the startup script to use curl -f so only a real 200 from /health counts as a pass. A 403 from AirPlay fails the check correctly. It's a small thing, but if you're on a Mac and your API seems up but nothing works, check what's actually listening on that port.

The Silent Citation Bug

I found this one during demo prep, which is exactly when you don't want to find bugs. Every citation in the response said "unknown" instead of the actual source filename. The answers themselves were correct, grounded in the right content, but the metadata was getting dropped somewhere between ChromaDB's response and my citation assembly.

I traced it to how I was deserializing the search results from the vector store. The chunk text came through fine, but the metadata dictionary wasn't being mapped correctly on the way back. I found it by actually reading the raw response payloads instead of just checking that answers "looked right." Lesson, if your feature includes metadata, test the metadata explicitly, don't just eyeball the main output.

Use Checkbox-Driven READMEs To Track Progress

One practice I've settled on for side projects is using the README as a living checklist. No box gets checked until the feature works end to end so if ingestion is checked, it means I've uploaded a real document, verified the chunks in ChromaDB, and confirmed the embeddings are stored correctly. If query is checked, it means I've asked a real question and gotten a cited answer back.

The repo also ships a demo GIF as proof. I'm not interested in READMEs that describe aspirational architecture and things that SHOULD work. If it's in the README, it works and if it doesn't work yet, the box is unchecked.

Phase 2: The Benchmark That Actually Matters

Phase 2 is where the interface-first architecture pays off. I'm implementing AzureOpenAIEmbeddingProvider, AzureOpenAILlmProvider, and AzureAISearchVectorStore behind the same interfaces. One config flag, "Provider": "Local" vs "Provider": "Azure", swaps the entire stack.

Then I'll run the same document set and the same question bank against both configurations and publish a comparison table containing ingestion speed, query latency, cost per query, and answer quality. That's the benchmark I actually want, not some synthetic throughput numbers, but real pipeline performance on real documents with the same evaluation criteria.

If you're building RAG pipelines and want to poke around the code, the repo is here: github.com/vondraysanford/docquery. And if you've hit your own ChromaDB migration headaches or NuGet supply chain surprises, I'd like to hear about it :).