Data Engineer

Job summary

The role involves developing and maintaining data pipelines and data products to support business decisions. The Data Engineer will work closely with engineers and stakeholders to deliver scalable and reliable data solutions.

Qualifications

• Proficient in Python and SQL
• Experience with Lakehouse or medallion architecture principles
• Familiarity with data ingestion patterns, including batch and streaming workloads
• Comfortable using Git or similar version control tools
• Solid understanding of CI/CD principles and data development practices
• Hands-on experience with Databricks and knowledge of data quality and governance concepts

Responsibilities

• Develop and maintain data pipelines using Python, PySpark, and/or SQL
• Support data ingestion processes across batch and streaming workloads
• Implement data quality checks and validation processes within existing frameworks
• Contribute to CI/CD processes and follow established data engineering development practices
• Work with data governance principles, including data classification, access control, and compliance requirements
• Collaborate with engineers and stakeholders to deliver high-quality data solutions and resolve technical challenges

Skills

• Experience with Databricks Asset Bundles / Declarative Automation Bundles (DABs) or similar deployment and configuration tooling is advantageous
• Familiarity with streaming technologies such as Auto Loader, Kafka, Event Hubs, or similar solutions
• Experience working with Azure, AWS, or Google Cloud Platform (GCP)

Education

• Relevant educational background in data engineering or a related field is preferred.

Tools

• Databricks, Python, PySpark, SQL, Git.