Data Engineer - Web Scraping (Remote)

Job summary

This role involves building scalable web scraping solutions and data pipelines that provide critical business insights. You will collaborate with analysts and cross-functional teams to develop reliable datasets that support strategic decision-making, while having significant ownership over your projects.

Qualifications

• Bachelor's or Master's degree in Computer Science or a related technical field.
• 2–4 years of professional software development experience.
• Strong programming skills in Python and solid SQL/database knowledge.
• Advanced experience with the Pandas library for data manipulation.

Responsibilities

• Design, develop, and maintain automated data collection systems.
• Collaborate with analysts to understand data needs and deliver tailored solutions.
• Build web scrapers for various structured and unstructured data sources.
• Clean, transform, and validate large datasets.
• Schedule and optimize scraping workflows using tools like Apache Airflow.
• Develop quality control checks for data integrity and availability.
• Investigate and resolve data pipeline issues.

Skills

Benefits

• Experience with web technologies (HTML, JavaScript, APIs).
• Familiarity with web scraping frameworks (Selenium, Scrapy, etc.).
• Knowledge of Docker and CI/CD tools (Jenkins, GitLab CI/CD).
• Experience with cloud services, particularly AWS (S3, RDS, etc.).
• Work on challenging projects in a global asset management environment.
• High level of ownership and autonomy in a collaborative culture.
• Opportunities for continuous learning and professional growth.