Software Engineer, Data Collection

Job Summary

A company is looking for a Software Engineer, Data Collection to join their Data Engineering team.

Key Responsibilities

• Build high-volume, fault-tolerant scraping services to enhance data extraction infrastructure
• Manage large-scale event processing systems handling millions of events daily
• Design and improve algorithms for optimizing scraping efficiency and accuracy

Required Qualifications

• Experience with distributed systems and data pipelines
• Strong programming skills in Python or TypeScript
• Expertise in SQL for querying and transforming data
• Hands-on experience in a cloud-native environment (AWS preferred)
• Familiarity with CI/CD pipelines and automated testing