Engineering Evaluations Specialist - job 1 of 2

Job Summary

A company is looking for a Member of Engineering (Evaluations / Engineering).

Key Responsibilities

• Design a Python framework for implementing internal and public benchmarks centrally
• Build and maintain a pipeline for running distributed evaluations at scale
• Collaborate with modeling and product teams to enhance experimentation and evaluation tools

Required Qualifications

• Strong engineering background with experience in leading cross-functional software projects
• Experience in building reliable and well-tested services
• Familiarity with distributed systems, data pipelines, and event-driven architectures
• Experience with cloud services such as GCP, AWS, or Azure
• Experience designing frameworks or tooling for developers is a plus