Build AI features like software
Design model integrations with normal backend controls for data, tools, reliability, and cost.
An AI feature is a software system with a model call inside it. Your service still owns user authentication, access checks, validation, retries, storage, and error handling.
A basic request path
The model can suggest a tool call. Your backend validates its arguments and permissions, runs the approved function, then returns the result to the model. The model should not receive unrestricted database access or the ability to perform arbitrary actions.
OpenAI’s function-calling guide explains this request, tool execution, result, and follow-up flow. The same boundary applies with other providers, even when their APIs use different names.
Choose the simplest source of truth
| Need | Start with | Why |
|---|---|---|
| Exact account or order data | An authorized API or database query | The source returns current structured data |
| Search across a document set | Keyword or semantic retrieval | It finds passages that can be supplied as context |
| A natural-language summary or draft | A model request | The model can turn context into flexible text |
| An action such as changing an order | A narrow application tool | Your code can check permissions and validate the action |
RAG means retrieval-augmented generation. Index documents, retrieve likely relevant passages, filter them by the user’s access, and include the permitted text with the model request. Return citations or source titles where the product needs users to verify the answer. Say when no useful source was found.
For RAG details, read the retrieval guide. For tool connections across compatible clients and services, the MCP specification is an optional next step, not a prerequisite for ordinary SDE work.
Design for delays and failures
Model calls add network latency, variable response times, and usage cost. Set timeouts and output limits. Handle rate limits and provider errors. Retry only when the failure is temporary, use backoff, and avoid repeating a side effect after a retry.
Stream long responses when the interface benefits from early output. For work that can take longer than a web request, put the task on a queue and let a worker process it. Track job status, retry safely, and move repeated failures to a dead-letter queue for inspection.
Measure task quality alongside latency, failure rate, and cost. Avoid logging credentials or private content. Add a human approval step before a tool can make a consequential change.
Use OpenAI’s evaluation best practices to shape a test set and metrics. Its deployment checklist and streaming guide show provider-specific controls. Verify the controls supported by the provider you choose.
For security, read OWASP’s guides to prompt injection and excessive agency. They explain why retrieved text must not become trusted instructions and why tools need narrow permissions.
Common design mistakes
- Sending all user data to the model when one small query would do.
- Treating retrieved text or model output as trusted instructions.
- Giving a model broad tools when a narrow read-only function is enough.
- Retrying a write without an idempotency check.
- Fine-tuning before measuring whether a prompt or retrieval system can solve the problem.
Next, learn how to test and safely use these systems in your AI-assisted workflow.