Lead Data Engineer (Remote, Global)

Job summary

The role involves leading the technical direction of a Data & AI Platform that supports various business functions across a global organization. The Lead Engineer will oversee architecture, drive high-impact initiatives, and mentor team members to enhance engineering standards.

Qualifications

• Proven experience in guiding the technical direction of a large-scale production data or developer platform with measurable adoption.
• Strong background in distributed systems design and infrastructure-as-code, with proficiency in Python and familiarity with at least one additional programming language.

Responsibilities

• Define and own the architecture for the Data & AI Platform, utilizing technologies such as Databricks and Microsoft Fabric.
• Lead critical initiatives from inception to deployment, ensuring alignment with product teams.
• Conduct design, code, and incident reviews to elevate engineering practices across the team.
• Collaborate with management on project roadmaps and technical decision-making.
• Advocate for the platform in cross-functional architecture forums to ensure its integration across teams.
• Mentor engineers to enhance their skills and productivity, fostering a collaborative environment.
• Establish best practices for data processing, including batch and streaming workloads, and ensure interoperability across domains.

Skills

• Expertise in Databricks, Delta Lake, Spark, and the Microsoft data ecosystem.
• Proficient in Terraform and CI/CD practices, with experience in policy-as-code frameworks.
• Ability to drive platform capability adoption effectively.

Education

• Relevant degree in Computer Science, Engineering, or a related field is preferred.

Tools

• Databricks, Delta Lake, Spark, Microsoft Fabric, Power BI, Terraform, CI/CD tools.