Senior Python Data Scraping Engineer (Remote, Freelance)

Job summary

This freelance position involves managing specialized data scraping workflows within a hybrid AI and human system, focusing on web extraction and processing. The role requires collaboration with agents to ensure high-quality data delivery.

Qualifications

• Minimum of 5 years of experience in data engineering, web scraping, automation, or software development.
• Bachelor's or Master’s Degree in Engineering, Applied Mathematics, Computer Science, or related fields is advantageous.
• Proficiency in Python web scraping tools such as BeautifulSoup and Selenium, including handling dynamic content and APIs.
• Experience in data cleaning, normalization, and validation, producing structured datasets.
• Familiarity with cloud infrastructure (AWS or equivalent) and containerization (Docker).
• Strong attention to detail and commitment to data accuracy.
• English proficiency at an upper-intermediate level (B2) or above.

Responsibilities

• Manage end-to-end data extraction workflows from complex websites, ensuring accuracy and reliability.
• Utilize internal tools (Apify, OpenRouter) and custom workflows to enhance data collection and validation processes.
• Adapt scraping techniques for dynamic web sources and changing site behaviors.
• Implement data quality standards through validation checks and consistency controls.
• Scale scraping operations for large datasets, monitor failures, and maintain stability against minor site changes.

Skills

Compensation

Earn up to $37 per hour, depending on experience and contribution pace. Compensation may vary based on project scope and complexity.

Work Setup

This role is fully remote, allowing for flexible scheduling with just a laptop and stable internet connection.

• Strong experience with web scraping frameworks and handling anti-bot mechanisms.
• Ability to extract data from complex structures and manage dynamic site content.
• Experience with LLM frameworks applied to automation tasks is a plus.