31 min read
Three ways to see it
Until 2023, an LLM read text and wrote text, and that was the whole game. Today the leading models read images, listen to audio, watch short videos, and produce all three. The technical word is multimodal, and the practical effect for a Pakistani professional is that the assistant on your phone can now do things that used to require a back office of clerks. A field officer can photograph a damaged crop, send the image to Gemini, and receive an Urdu diagnosis with a recommended pesticide dose. A bank clerk can photograph a customer's CNIC and have the model extract the name, father name, date of birth, and ID number into a form. A traffic police inspector in Lahore can show the model a jam at Kalma Chowk and ask which alternate routes are likely to clear fastest. This is not science fiction. This is the 2026 free tier.
How vision works inside a model, in one paragraph. The model has been trained on millions of image-text pairs, in which a photo sits next to a caption describing it. During training the model learns to map the patterns inside an image, the shapes of edges, colours, textures, recognisable objects, to the language of captions. At inference time, when you upload a picture of a flooded street in Sukkur, the model is doing the same trick in reverse: it converts the pixels into an internal representation, then generates a sentence in English or Urdu that fits that representation. The model is not seeing the way you see. It is matching what it sees to text it has read.
Audio works on the same principle. A speech-to-text model converts a voice note into Urdu or English transcript, then a regular LLM processes the transcript. A text-to-speech model does the reverse: it takes generated text and produces an audio file, sometimes in a voice the user has cloned. For Pakistani professionals the most useful audio applications are dictation, where you speak in Urdu and receive English typed minutes, transcription of recorded meetings, and accessibility, where blind or low-vision users can hear documents read aloud. Voice cloning is the dangerous capability: a thirty-second voice sample is enough for the major models to imitate someone convincingly, which is the technical reason behind the deepfake fraud cases now reaching Pakistani courts.
Quick check
Quick check: what makes modern AI different from a rule-based program?
The why-tree
Three ways to classify a multimodal task. By input: what is being read by the model, text only or also image, audio, video. By output: what is being produced, text, audio, or media. By both: combined tasks like reading a photo of a damaged invoice and producing a voice note in Urdu summarising the dispute. The richer the modality on each side, the more useful and the more expensive the task.
Sources
Sources for further reading. The Gemini multimodal documentation under ai dot google dot dev, especially the section on vision and document understanding. The OpenAI GPT-5 vision and audio sections of the system card. The Anthropic vision capabilities page on docs dot anthropic dot com. The Whisper paper from OpenAI for the foundational speech-to-text architecture. The OpenAI Sora technical report and the Google Veo announcement page for video generation. For Pakistani context, the NADRA technology blog posts about CNIC verification automation, and the Punjab Safe City Authority annual report which describes their early AI integration work.