🇺🇸 USAJobs.work

America's Job Portal

← Back to USA Jobs

Senior AI Quality Engineer (LLM Evaluation & Automation) 1754

Company

SOFTGIC

Location

Medellín, Antioquia

Posted

June 30, 2026

Position Overview

Este es un puesto de trabajo remoto.

Owns the eval harness and quality gate from the beginning. This role replaces the old late-stage “Evals Specialist” model with a standing owner for measurable agent quality.

Key Responsibilities

• Build and maintain the MVP eval harness: golden tasks, exception tasks, scorecard metrics, and regression packs.
• Wire evals into CI so quality regressions fail builds and releases.
• Define and maintain release-gate thresholds with Product and the Tech Lead.
• Lay the path for later adversarial and drift-testing expansion without overbuilding MVP scope.


Requisitos

Must-Have Qualifications

• Experience evaluating ML, LLM, or non-deterministic systems.
• St...

Ready to Apply?

Join thousands of Americans building their careers

Apply Now