Lead Engineer, Data Platform

Job summary

The role involves leading the technical direction of a Data & AI Platform that supports various business functions across a global organization. As the most senior individual contributor, you will drive high-impact initiatives and mentor other engineers.

Qualifications

• Proven experience in guiding the technical direction of a large-scale production data or developer platform with measurable adoption.
• Strong expertise in distributed systems design, polyglot engineering, and infrastructure-as-code practices. Proficient in Python and at least one other programming language.

Responsibilities

• Define the architecture for the Data & AI Platform, focusing on technologies like Databricks, Microsoft Fabric, and Power BI.
• Lead significant initiatives from inception to delivery, ensuring successful rollout and adoption.
• Enhance engineering standards through design, code, and incident reviews.
• Collaborate with the Engineering Manager on project roadmaps and staffing decisions.
• Advocate for the platform in architecture and governance discussions across the organization.
• Mentor engineers to amplify their impact and foster a collaborative environment.
• Establish best practices for batch, streaming, and ML/AI workloads, ensuring interoperability across domains.

Skills

• Expertise in Databricks, Delta Lake, Spark, and the Microsoft data ecosystem (Fabric, Power BI, Azure).
• Proficient in Terraform and CI/CD processes, including policy-as-code practices.
• Ability to drive the adoption of platform capabilities effectively.

Education

• Relevant degree in Computer Science, Engineering, or a related field is preferred.

Tools

• Familiarity with tools and technologies such as Databricks, Delta Lake, Terraform, and CI/CD platforms.