Senior Python Data Scraping Engineer (Remote, Freelance)

Job summary

This freelance position involves leading data scraping workflows for a hybrid AI and human system, focusing on web extraction and processing tasks. The role is ideal for experienced professionals in data engineering and web scraping, offering flexibility and the opportunity to work remotely.

Qualifications

• Minimum of 5 years of relevant experience in data engineering, web scraping, automation, or software development.
• Bachelor's or Master’s Degree in Engineering, Applied Mathematics, Computer Science, or related technical fields is preferred.
• Proficient in Python web scraping tools (BeautifulSoup, Selenium, etc.) and handling dynamic content.
• Experience in data cleaning, normalization, and validation, delivering structured datasets.
• Familiarity with cloud infrastructure (AWS or similar) and containerization (Docker).
• Strong attention to detail and commitment to data accuracy.
• English proficiency at an upper-intermediate level (B2) or above.

Responsibilities

• Manage end-to-end data extraction workflows across complex websites, ensuring accuracy and reliability of structured datasets.
• Utilize internal tools and custom workflows to enhance data collection and validation processes.
• Adapt scraping techniques to extract data from dynamic and interactive web sources.
• Implement data quality standards through validation checks and systematic verification.
• Scale scraping operations for large datasets while maintaining stability against site structure changes.

Skills

• Strong experience with web scraping and data extraction from complex structures.
• Ability to handle anti-bot mechanisms and dynamic site structures at scale.
• Experience with LLM frameworks applied to automation tasks is a plus.

Tools

• Familiarity with tools such as Apify and OpenRouter for data scraping and processing.
• Experience with cloud services and containerization technologies.