Role/Job title :AI Evaluation Engineer

Work Location :London

Type work : Contract

Mode of work : Hybrid

Responsibilities:

* Define and implement end-to-end evaluation strategy for generative AI conversational systems (LLMs, RAG pipelines, agents).

* Establish evaluation metrics

* Test Dataset & Benchmarking

* Benchmark models (GPT variants, open-source LLMs) and prompt strategies

* Implement automated evaluation pipelines

* Conduct human-in-the-loop evaluation for qualitative validation.

* Prompt & Response Quality Optimization

* RAG & Knowledge Grounding Validation

* Safety, Risk & Compliance Testing

Your Profile

* Strong understanding of LLMs, RAG architecture, prompt engineering

* Python – data analysis, evaluation pipelines

* Prompt evaluation tools (PromptTools, DeepEval, etc.)

Experience with:

- Evaluation Fra...