Data Engineer

Job summary

The role involves joining a growing AI team focused on transforming data into business value. The primary responsibility is to design and manage reliable data pipelines in a rapidly expanding company.

Qualifications

• Minimum of 3 years of experience as a data engineer.
• Proven experience in building production-grade data pipelines using PySpark and SQL.
• Strong understanding of ETL/ELT design, data modeling, and partitioning strategies.
• Familiarity with AWS services, particularly S3 and IAM.
• Experience with Databricks or similar platforms.

Responsibilities

• Design, build, and maintain batch and streaming data pipelines in Databricks.
• Model data for analytics and machine learning, ensuring well-documented tables in Unity Catalog.
• Collaborate with external APIs and internal teams to integrate new data sources.
• Implement data quality measures, lineage tracking, and observability.
• Optimize data pipeline cost and performance.
• Work alongside data scientists and product teams to deliver impactful features.
• Automate deployment and testing processes, ensuring pipelines are simple and maintainable.

Skills

• Pragmatic and adaptable mindset, applying best practices as needed.

Education

• Relevant educational background in data engineering or a related field is preferred.

Tools

• Databricks, PySpark, SQL, AWS.