Most briefs for these roles are copied off a job board and describe nobody. Here is what each one does, and how to tell a real one from a keyword match. Use it to write your requirement, or send it and we will.
What they are. An engineer who works inside the customer’s environment rather than behind a product roadmap. Part solutions architect, part product engineer, part the person who finds out what the customer actually meant. They own an integration end to end and are measured on whether it worked in production, not on tickets closed.
Why they are hard to hire. The job needs an engineer who will sit with a customer’s operations head, and a consultant who can write production code. Most candidates are strong on one side and bluffing on the other. The screen that works is a real integration problem with an ambiguous requirement, scored on the questions they ask before writing anything.
Real signals: has shipped into a customer’s environment, not a demo; can describe a requirement they talked a customer out of; comfortable with no product manager in the room. Not signals: “client-facing” in the summary, certifications, a title that was awarded rather than earned.
| Role | What they actually do | Screen on |
|---|---|---|
| AI / LLM Application Engineer | Product features on model APIs: retrieval, context assembly, structured output, latency and cost control. | A failure they debugged in production, and what the fix cost per request. |
| Agent Engineer | Systems that take actions: tool use, multi-step workflows, failure recovery, permissioning, human review. | How they stop an agent doing the wrong thing twice. Anyone who has not thought about this has not shipped one. |
| Evaluation Engineer | The test sets and scoring that say whether a change helped. The least glamorous and most load-bearing role on an AI team. | An eval that caught a regression human review missed. |
| Inference / ML Platform | Serving, quantisation, batching, GPU scheduling, throughput and cost per token. | Real numbers: tokens per second, utilisation, what they changed to move it. |
| Data Engineer, AI | Pipelines for training and retrieval corpora, document processing, deduplication, provenance. | How they handled bad data at scale without deleting it. |
| AI Solutions Architect | The shape of a deployment before anyone writes code; sizing cost and risk. | A design they rejected, and why. |
| AI Product Manager | Decides what the system must refuse to do, and what “good enough” means numerically. | Whether they can state an acceptance threshold without hedging. |
On titles. We will not place a standalone “prompt engineer”. In our experience it is an evaluation or agent engineering job with a weaker title, and the people hired into it leave inside a year. If you have budgeted for one, let us talk about what you actually need built.
Backend, frontend, mobile, data, DevOps, SRE, QA and support engineering, at every level. The AI bench is where we are sharpest; it is not all we do, and a team needing two platform engineers alongside one agent engineer should not have to use two agencies.