Data Engineer

Job summary

The role involves transforming complex regulatory information into structured, machine-readable data to support product compliance. You will work closely with founders, AI engineers, and compliance experts to ensure the accuracy and timeliness of regulatory data.

Qualifications

• Several years of experience in building data pipelines, crawling infrastructure, or large-scale extraction systems, with proficiency in Python.
• Familiarity with LLM-assisted extraction techniques and their challenges.
• Strong focus on data correctness, including provenance and versioning.
• Experience handling messy data sources such as broken HTML and scanned PDFs.
• Understanding of product management and its implications on data outputs.
• Previous experience with regulatory data is a plus.
• Ability to thrive in a fast-paced, ambiguous environment.

Responsibilities

• Develop and maintain crawling and monitoring systems for regulatory sources across various jurisdictions.
• Parse legal texts into a normalized schema using designed extraction pipelines.
• Implement change detection mechanisms for new regulations and amendments.
• Ensure data quality through tracking provenance and coverage metrics.
• Collaborate with regulatory experts to enhance data schema and taxonomy.
• Contribute to shaping the architecture and data culture as the team expands.

Skills

• Proficiency in Python and experience with data engineering.
• Knowledge of LLM-assisted extraction methods.
• Strong analytical skills and attention to detail.
• Ability to work collaboratively in a small team.

Education

• Relevant degree or equivalent experience in data engineering or a related field.

Tools

• Experience with data extraction tools and technologies, particularly in handling legal documents.