Data Platform Engineer

Job summary

The role involves designing, building, and operating platform services and SDKs essential for product teams to deliver data products. The engineer will manage features from design to production and participate in an on-call rotation for the services they oversee.

Qualifications

• Experience in building and operating data platforms or backend services at scale.
• Proficient in Python and at least one other programming language.
• Hands-on experience with Terraform and CI/CD in a production environment.
• Familiarity with Databricks, Spark, and Delta Lake or similar technologies.
• Strong engineering fundamentals including testing, code review, and debugging.
• Experience with on-call responsibilities, SLOs, and incident management.

Responsibilities

• Develop and maintain platform services, SDKs, and automation for data product delivery.
• Implement infrastructure-as-code, CI/CD pipelines, and automated policy enforcement.
• Manage production services, including incident response and SLO tracking.
• Collaborate with product engineers to enhance platform capabilities based on recurring issues.

Skills

• Experience with policy-as-code, secrets management, and IAM in cloud environments is a plus.
• Knowledge of streaming technologies like Kafka or Event Hubs is beneficial.
• Familiarity with data quality and lineage tools is advantageous.
• Exposure to the Microsoft data ecosystem, including Fabric and Power BI, is a plus.

Education

• Relevant degree or equivalent experience in a related field.

Tools

• Python, Terraform, CI/CD tools, Databricks, Spark, Delta Lake.