Data Engineer - job 29 of 50

Job Summary

A company is looking for a Data Engineer to design, build, and maintain production-grade data pipelines.

Key Responsibilities

• Architect, develop, and deploy batch and streaming data pipelines using Airflow and containerized workflows
• Build high-throughput data transformations and maintain training data clusters across hybrid environments
• Implement observability for pipeline health, troubleshoot failures, and collaborate on data contracts and quality checks

Required Qualifications

• 3-5 years of experience designing and operating production data pipelines at scale
• Expertise in containerization and orchestration with Docker, Kubernetes, and Helm
• Hands-on experience with Apache Airflow for workflow management
• Strong proficiency in Python for data engineering tasks
• Practical experience with Dask or Apache Spark for large-scale data processing