LLM basics
Explain how large language models generate text and why their answers need checking.
Learn the model at a level that helps you build reliable software around it. You do not need to derive transformer equations for a general SDE interview. You should be able to explain the main terms and where the model can fail.
How text generation works
A language model receives input as tokens. A token may be a word, part of a word, or punctuation. The model uses the input context to estimate likely next tokens, then generates output one token at a time.
Many modern language models use the Transformer architecture. Its attention layers let the model weigh how different parts of the input relate to each other. This helps it use context, but it does not mean the model reasons like a person or checks every statement against a database.
Read Google’s LLM and Transformer overview for a visual explanation. Focus on tokens, next-token prediction, and self-attention. Save the training mathematics for roles that ask for it.
Context and wrong answers
The context window is the amount of input and conversation the model can consider in one request. Long prompts use more tokens and can push useful details out of the available context. A model may also lack current facts, misunderstand an instruction, or generate a plausible answer without enough evidence.
Treat the response as untrusted input. Validate its format, check important claims against a reliable source, and provide a safe fallback when it cannot answer. A prompt can guide the model, but it cannot guarantee truth.
Embeddings and retrieval
An embedding is a numeric representation of text. Similar meanings can map to nearby vectors, which supports semantic search. A retrieval system can use these representations to find relevant passages from a document collection.
RAG adds retrieved information to the model’s request at answer time. It can help with private or changing information, but it can still retrieve the wrong passage or miss a relevant one. The model can also misstate the evidence it receives.
Read Google’s embeddings guide and OpenAI’s retrieval guide. You should be able to explain that embeddings help find related content. They do not store exact facts like a relational database.
Retrieval is not fine-tuning
Use retrieval when the model needs access to changing reference material or private documents. Use fine-tuning when you need to change a model’s behavior for a repeated task and have suitable training examples. Fine-tuning is not a reliable way to keep facts current.
For a general backend role, learn the trade-off rather than training a model. Be ready to explain why an ordinary API or database query may be more accurate, cheaper, and easier to audit than an LLM call.
Questions to answer out loud
- What is a token, and why can token limits affect a request?
- Why can a language model produce a confident but incorrect answer?
- What does an embedding help a search system do?
- When might you use RAG instead of fine-tuning?
- When would you avoid an LLM and use a normal API or database query?
If you can answer these clearly, move to building AI features.