AI-native engineering hire
Hire AI evals engineers to make release quality measurable.
Find engineers who can turn product expectations into useful evaluations, explain uncertainty and help your team decide what is ready to ship.. Finalists have passed all agreed technical and soft-skill stages, with interest, compensation expectations and availability confirmed.
First qualifying finalist by business day 15, or stop the search for a full $2,000 refund. The search runs for 60 calendar days from the agreed launch date while the vacancy remains open.
Standard recruitment
Your search at a glance
Standard: 18% of first-year salary. $2,000 is paid at launch and credited toward that total; the balance is due when your hire starts. We agree the salary basis and search scope before payment.
60-day search, launch-payment refund and replacement terms
- Seniority
- Senior AI evals engineer searches; the technical bar is agreed in your brief.
- Compensation guide
- Indicative planning band: €60–98K a year in senior B2B contractor compensation across CEE.
- Search markets
- CEE candidates, with location and working-hour requirements agreed for your team.
Compensation is indicative. We agree the salary band, locations and working-hour overlap before search; the placement fee is separate.
Focus your engineers’ interview time
We assess technical and soft skills before introducing candidates, so your engineers meet people who have passed the agreed requirements. Review the evidence, run your own technical and team interviews, and choose who to hire. We support coordination and the offer process.
Choosing the role
What an evals engineer should own
An AI evals engineer connects product quality to repeatable tests. The work includes representative datasets, clear rating criteria, automated checks, human review and release reporting. The outcome is a decision your team can explain: which behaviours improved, which worsened and what remains uncertain.
A useful brief defines the task, who judges its quality and the cost of different errors. A customer-support assistant may need accurate grounding and sensible escalation; a coding feature may need executable checks and review of the resulting changes. One broad benchmark score rarely answers every product question.
Evaluation needs to reflect real use and evolve as the product changes. Automated graders also need comparison with human judgments. OpenAI’s evaluation guidance recommends task-specific tests, human calibration and continuous evaluation. We use those principles to frame interview questions, rather than treating familiarity with an eval framework as sufficient evidence.
Bring to the intake: the decisions the evaluation should support, examples of acceptable and unacceptable outputs, existing datasets, available domain reviewers and your release process. A dedicated hire makes sense when maintaining this work needs sustained ownership. For a smaller, well-defined feature, an LLM engineer with evaluation experience may cover the scope.
Across Recruo searches
Our recruitment results so far
Company-wide observations reported by Recruo’s founder on 11 September 2026. These describe our overall experience, rather than a separate cohort for this role or region.
- Pass every client assessment stage
≈9/10
Presented candidates who successfully pass the client’s full assessment process, including technical interviews.
- Candidates the client would offer to
≈4/5
Client willingness to make an offer after assessment. Offers sent, accepted and hires are separate outcomes.
- Completed probation periods passed
100%
All placed candidates whose probation has concluded have passed so far. Ongoing probation periods are excluded.
How to read these results
Pass every client assessment stage: Candidates who pass all client assessment stages, relative to candidates presented. Around 9 in 10 is an approximate company-wide observation, not a published sample of exactly ten people. Offers, acceptance and starts are separate outcomes; sample size, reporting period and treatment of ongoing assessments are not yet published.
Candidates the client would offer to: Presented candidates the client considers suitable for an offer, divided by candidates presented. Around 4 in 5 describes willingness to offer; it does not mean four offers or hires for every five candidates.
Completed probation periods passed: Placed candidates who passed probation, divided by placements whose probation period has ended. Probation lengths vary by engagement; this is not a 90-day retention measure or a guarantee of future outcomes.
Sample sizes and the reporting period are not yet published. The candidate ratios are approximate; none of these results is a forecast for your vacancy.
Example scorecard
Evidence to look for in your shortlist
Illustrative assessment criteria, not a candidate profile or a record of a completed placement. The final scorecard is agreed for your role.
Example scorecard: senior AI evaluation engineer
Role scope: Use this scorecard to assess whether a candidate can build an evaluation process that supports your team’s release decisions.
- Dataset design: explains coverage, difficult examples, leakage risks and how the evaluation set is maintained.
- Rating criteria: turns a product expectation into an understandable rubric with examples and an ambiguity-resolution process.
- Automated grading: can compare model-based scores with expert judgments and investigate disagreement.
- Release decisions: explains thresholds, uncertainty and how to respond to regressions or noisy results.
- Collaboration: communicates findings to engineering and domain reviewers, including the limits of a metric.
How we source
How we assess evaluation judgement
We consider relevant experience from ML engineering, quality engineering, data science and applied research. The important distinction is whether the candidate can turn a quality question into a usable evaluation process and work with the people making product decisions.
The technical interview explores a dataset or evaluation the candidate owned. We ask how examples were selected, how ambiguous labels were resolved, how an automated judge was checked and what happened when a metric disagreed with expert review. Follow-ups explore false alarms, missed failures and what evidence would justify blocking a release. AI supports the interview; a human recruiter reviews the evidence before shortlist delivery.
Candidates can demonstrate their work through a redacted dataset design, rubric walkthrough, practical exercise or public artifact. We do not require a publication, a previous safety-company role or proof that they personally blocked a release. The shortlist documents demonstrated methods and limitations so your team can run a focused follow-up interview.
Role-specific screening
What we assess
Good evaluation requires both engineering and judgement about what the test represents. We assess methodology, implementation and communication together.
Dataset quality
Coverage, provenance and leakage checks
Role-specific screening criteria agreed during intake
Rating reliability
Clear rubrics and human calibration
Role-specific screening criteria agreed during intake
Decision usefulness
Regression analysis and release integration
Role-specific screening criteria agreed during intake
Candidates should be able to challenge a misleading metric as well as build the harness that computes it. We separate evidence of those skills from familiarity with named tools or a particular academic background.
FAQ
Frequently asked questions
Get started
Tell us about your open role
Share your AI evals engineer brief. We agree the assessment criteria, budget, launch date and 60-day search scope before you commit.
$2,000 to launch
Fully credited toward the 18% Standard fee
90-day guarantee
One replacement search under agreed conditions
Human review
Recruiters review the screening evidence
Also on Recruo
Roles we hire for
Hire by location
Compared to other agencies
Further reading
Hire in the UK