Lead Machine Learning Engineer - job 4 of 8

Job Summary

A company is looking for a Lead Machine Learning Engineer - Applied Scientist.

Key Responsibilities

• Develop and own evaluation pipelines for agentic LLM systems
• Define and iterate on quality metrics for LLMs and agents
• Lead experiments to improve system behaviors across various dimensions

Required Qualifications

• Deep familiarity with evaluation methodologies for LLMs or autonomous agents
• Hands-on experience in Python and ML frameworks such as PyTorch
• Proven ability to collaborate across teams to align ML evaluation with goals
• Comfortable operating in ambiguous problem spaces
• Passion for continuous improvement and strong documentation habits