Prompt architecture: system prompts, chain-of-thought, structured output

Three ways to see it

1. 1

   Three layers of a production prompt. The system prompt sets identity, scope, and constraints. 'You are an FBR policy analyst assistant. You summarise flagged tax returns in two pages in plain English. You never invent numbers. You cite every claim with a paragraph reference. You respond in English with an Urdu executive summary at the top.' The user prompt carries the task and the data. 'Here is the tax return text. Produce the summary.' The optional assistant prompt seeds the output with an opening line, sometimes used to force a specific format. The system prompt is durable and governance-controlled. The user prompt changes every call. The assistant prompt is a precision tool used sparingly.

2. 2

   Chain-of-thought is the technique of asking the model to think step by step before giving the final answer. For tasks with reasoning, this improves accuracy substantially. The classic phrasing is 'Think step by step, then give your final answer in the requested format.' In a Pakistani policy setting: 'Read the return. List every revenue irregularity. Check each against the policy definitions provided. Estimate the financial impact. Only then write the two-page summary.' The intermediate steps are part of the prompt's instructions. The model produces them; you can choose to show them to the user or hide them. Hiding them keeps the user-facing output clean. Showing them improves trust by making reasoning auditable.

3. 3

   Structured output is the discipline of constraining the model to return JSON or a precise schema instead of free prose. This is what lets the rest of your application consume the model's output reliably. Anthropic's Claude supports structured output through the response schema parameter; OpenAI through JSON mode; both also support tool-use forcing where the answer must call a specific tool. A Pakistani example: instead of asking 'tell me about this return', ask 'return JSON with the fields ntn, year, revenue_irregularities (array of objects with description, impact_pkr, confidence), recommendation (audit, no_audit, escalate), urdu_summary, citations (array of paragraph references).' The application reads the JSON, displays whichever fields matter, and stores all of them.

The why-tree

Try this with Claude

AI-edge prompt to try with Claude or ChatGPT: 'Critique my system prompt below. Output: five concrete improvements, ranked by expected impact on quality. For each, show the before line and the after line. Then propose two adversarial test inputs the prompt currently fails on. Output in English; Urdu translation for the final two test cases only.' Iterate three rounds.