Solution design: LLM, RAG, agent, or no AI at all
حل کا ڈیزائن: LLM، RAG، agent، یا AI ہی نہیں
40 min read
Three ways to see it
Option zero: no AI. A spreadsheet macro, a checklist, a shell script, or a junior employee given two hours. Many Pakistani 'AI projects' are actually 'we never wrote a checklist' projects. Before reaching for a model, audit whether process or training would solve the same pain at a tenth of the cost. The honest answer is sometimes yes. The wing-commander who finally writes a one-page SOP for their inspectors saves more time than a six-month LLM pilot.
Option one: a plain LLM call. Send a prompt, get an answer. Use when the knowledge needed is already inside the model's training data and the answer fits in one response. Drafting a standard memo, translating Urdu to English, summarising a publicly known company are all plain-LLM tasks. The architecture is one API call, the cost is pennies per call, and the build time is a day. The risk is hallucination on anything domain-specific. Plain LLMs do not know your internal data; do not ask them what is in your filing cabinet.
Option two: retrieval-augmented generation, or RAG. The model is paired with a search index over your own documents. When the user asks a question, the system first retrieves the most relevant passages from your documents, then asks the model to answer using only those passages, with citations. Use when the knowledge lives in your own corpus: every Pakistani internal policy library, every NEPRA tariff archive, every internal contract repository fits here. RAG keeps the model honest by anchoring it to evidence, and the citations let an auditor verify the answer. Build time is one to four weeks, depending on how clean your documents are.
Quick check
Quick check: what makes modern AI different from a rule-based program?
The why-tree
Why-tree level one: why even consider option zero? Because the most embarrassing audit finding is an AI project that replaced a missing SOP. If process can solve it, process should. AI sits on top of solid process, never as a substitute.
Try this with Claude
AI-edge prompt to try with Claude or ChatGPT: 'My candidate problem is summarising 16 tax returns per day at the FBR policy wing. Evaluate plain LLM, RAG, and agent architectures. For each, give me a one-paragraph honest assessment of fit, two reasons to choose it, two reasons to avoid it, and a rough four-week build plan if chosen. Output in English with Urdu summary.' Challenge every claim.
Sources
Sources and further reading. Anthropic, 'Building effective agents' (anthropic.com/research/building-effective-agents). LangChain RAG concepts (python.langchain.com/docs/concepts/rag). 'Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks', Lewis et al, 2020 (arxiv.org/abs/2005.11401). OpenAI Cookbook, 'Question answering using embeddings' (cookbook.openai.com). 'The Bitter Lesson', Rich Sutton (incompleteideas.net/IncIdeas/BitterLesson.html).