NLP & Large Language Models

Text is not natural for computers

Computers work with numbers. Text is messy, ambiguous, context-dependent, and infinite in its combinations. Getting machines to understand language has been one of the hardest problems in AI — and it's largely been cracked in the last decade, first by deep learning approaches to NLP and then, decisively, by transformers and large language models.

Understanding how that works makes you a sharper evaluator of LLM capabilities, a better judge of where they'll fail, and a more informed decision-maker about how to build with them.

Step 1: Tokenization — text as numbers

Before a language model can process text, the text must be converted into numbers. This happens through tokenization: splitting text into chunks called tokens, then mapping each token to an ID from a fixed vocabulary.

Tokens are not words. They're subword units — common words stay whole, while rare or long words get split into pieces. "Tokenization" might become Token + ization. Numbers, punctuation, and whitespace each get their own tokens.

Why it matters for PMs: LLM APIs charge per token, have context limits in tokens, and behave differently for tokenized vs. untokenized input. A 1,000-word document is roughly 1,300–1,500 tokens. Code and non-English text often tokenize less efficiently — more tokens per character = more cost and more context consumed.

Step 2: Embeddings — words as locations in space

Each token ID gets mapped to an embedding: a list of hundreds or thousands of numbers (a vector) that represents its meaning. Tokens with similar meanings end up close to each other in this high-dimensional space.

The classic example: in a well-trained embedding space, king − man + woman ≈ queen. The model has learned that gender is a direction in embedding space, and royalty is another — independently.

Embeddings are powerful beyond LLMs. They're used to:

• Find meaning-similar content — search that returns results related in meaning, not just keyword matches (type "cheap flights" and get results about "affordable airfare" too)
• Cluster text by topic — without predefined categories
• Power recommendation — embed users and items in the same space; nearby items are good recommendations
• Enable RAG — store documents as embeddings; retrieve the most relevant ones at query time (more on this below)

Step 3: Attention — what transformers actually do

Before transformers, language models processed text sequentially — one word at a time. Long documents were hard because early words were "forgotten" by the time the model reached the end.

Attention — the key innovation in the transformer architecture — lets the model look at all tokens in the input simultaneously and learn which ones are relevant to each other for the task at hand.

When processing the word "it" in "The trophy didn't fit in the suitcase because it was too big," attention allows the model to figure out that "it" refers to "trophy" (because the trophy is the thing that might not fit) — by learning to attend to the right context.

How LLMs are trained

Modern LLMs go through (at least) two training phases:

1. Pretraining

Train on a massive corpus of text (the internet, books, code) to predict the next token. This teaches the model language, facts, reasoning patterns, and world knowledge. Extremely expensive — only a handful of organisations can afford to do this.

2. Fine-tuning with human feedback (RLHF)

RLHF stands for Reinforcement Learning from Human Feedback. Human raters compare model outputs and rate which are better. Those preferences train a reward model, which is then used to fine-tune the base model to produce responses humans prefer. This is what turns a raw next-token predictor into an assistant that follows instructions, is polite, and declines harmful requests. Variants like DPO (Direct Preference Optimisation) and RLAIF (RL from AI Feedback) are now common in practice, but the core idea — optimise for preferred outputs — is the same. Claude, ChatGPT, and Gemini all use some form of this.

Prompting vs RAG vs fine-tuning

This is one of the most important decisions you'll make when building with LLMs — and it's often made without enough clarity. Here's how to think about it:

Context window: the model's working memory

The context window is how much text a model can "see" at once — its working memory. Everything outside the context window doesn't exist to the model.

Context windows have grown dramatically: GPT-3 launched with 4K tokens; modern models support 128K to 1M tokens. But larger isn't always better:

• Cost scales with context length — filling a 128K window costs 32× more than a 4K window
• "Lost in the middle" — models attend better to the beginning and end of long contexts; information buried in the middle is often missed
• Latency increases — longer contexts mean slower responses

Hallucination: confident and wrong

LLMs generate the most statistically likely next token — they don't look things up. When they don't "know" something, they don't say "I don't know" — they generate plausible-sounding text anyway. This is hallucination.

Hallucinations are not bugs that will be fixed — they're a fundamental property of how these models work. Mitigation strategies:

• RAG — ground responses in retrieved source documents
• Citation requirements — prompt the model to cite sources; makes hallucinations detectable
• Confidence elicitation — ask the model to flag uncertainty
• Human review — for high-stakes outputs, keep a human in the loop
• Constrained outputs — if the task allows, limit outputs to structured formats (JSON, fixed schemas) that can be validated programmatically
• Tool use and search grounding — give the model tools to look up facts (web search, database queries, calculators) rather than answer from memory alone. Now standard in agentic products and the most effective mitigation for factual hallucinations

Prompting patterns that actually work

Most teams start by writing a one-line instruction and wondering why results are inconsistent. Reliable LLM outputs require deliberate prompt design. Here are the patterns with the highest ROI:

Few-shot prompting

Show the model examples of the input/output pairs you want before asking it to do the task. The model infers the pattern from examples rather than relying purely on your instruction.

Three to five examples is usually enough. Choose examples that cover edge cases, not just easy cases — the model will interpolate toward your examples.

Chain-of-thought prompting

Ask the model to reason step-by-step before giving its final answer. This dramatically improves accuracy on tasks that require multi-step reasoning, math, or judgment calls. Adding "think through this step by step" to a prompt is one of the highest-ROI prompt changes you can make.

Role and persona specification

Telling the model what role it's playing shifts the style and emphasis of its output significantly. "You are a terse senior engineer reviewing a pull request" produces different feedback than "You are a helpful writing assistant."

Use role specification when output style or domain framing matters. Avoid vague roles ("be an expert") in favor of specific behavioral ones ("respond only with the classification label, no explanation").

Output format control

LLMs are more reliable when you specify the exact output format. For structured data, ask for JSON. For consistent classification, enumerate the valid labels. For summaries, specify the target length and structure.

How to evaluate LLM output quality

Evaluating generative AI is harder than evaluating classification models. There's no single right answer to "write a good product summary." The main approaches:

Human evaluation

The gold standard. Raters assess outputs on dimensions like accuracy, helpfulness, tone, and completeness. Reliable but slow and expensive. Use it to calibrate other evaluation approaches, and for high-stakes output categories.

Design rubrics carefully. "Is this response good?" produces inconsistent ratings. "Does this response answer the specific question asked, without adding incorrect information? (yes/no)" produces consistent ones.

LLM-as-judge

Use a capable LLM to evaluate the outputs of another LLM (or the same one). You give the judge model a rubric and ask it to score or compare outputs. Scales much better than human eval and correlates reasonably well with human judgment on many tasks.

Automated reference-based metrics

Compare model outputs to a reference (human-written) answer using word-overlap or embedding similarity. These scale easily and require no human time per query — but they all require a reference answer to compare against, which limits when they're useful.

Behavioral / task-completion evaluation

For agentic or tool-use tasks, evaluate whether the model completed the task correctly end-to-end. Did the SQL query return the right results? Did the agent book the correct flight? Binary pass/fail on concrete outcomes — more reliable than rubric scores for well-defined tasks.

PM Playbook — Questions to ask

• Prompting, RAG, or fine-tuning? — start with prompting, escalate only with evidence
• Have we tried few-shot examples? — usually the highest-ROI prompt change before any other optimization
• Do we have an eval set? — if not, you can't safely change anything (prompt, model, parameters)
• How are we evaluating output quality? — human eval, LLM-as-judge, or task completion; each has different cost/reliability tradeoffs
• What's the context window size and how does our use case fit within it?
• What's the cost per 1K tokens at our projected usage? — run the numbers before committing
• How are we handling hallucinations for this use case? — never "we'll rely on the model being accurate"
• What's the latency at the p95? — LLM inference can be slow; user experience implications are real
• What model are we using, and have we evaluated alternatives? — cost/quality/latency tradeoffs differ significantly across providers