Lead Site Reliability Engineer (Remote)

Job summary

This role involves leading the Site Reliability Engineering efforts for a security-focused product environment, focusing on establishing reliability metrics and improving system health across various components. You will work in a fully remote setup, collaborating with engineering teams to build a robust telemetry and reliability platform.

Qualifications

• Significant experience in production engineering or Site Reliability Engineering, particularly in defining and implementing SLO frameworks.
• Proficient in Python, with the ability to read and modify Go or Rust code.
• Extensive hands-on experience with time-series and event telemetry at scale, including tools like Prometheus, Grafana, and high-cardinality data stores.
• Familiarity with debugging distributed systems and production-scale configuration management tools.
• Strong written and asynchronous communication skills.
• Understanding of security products and their reliability metrics is advantageous.

Responsibilities

• Define SLIs for approximately 70 product components and establish ownership and measurement standards.
• Develop a reliability taxonomy covering various service metrics.
• Design and build telemetry collection pipelines for cloud services and customer-hosted agents.
• Implement symptom-based alerting and ensure every production alert has a defined owner.
• Coach engineering squads on operational responsibilities and incident management practices.
• Deliver measurable reliability outcomes, including complete SLI ownership and reduced detection times for security-control degradation.

Skills

• Strong engineering judgment and a pragmatic approach to observability and reliability.
• Experience with monitoring frameworks related to compliance standards like SOC 2 or ISO 27001 is valuable.
• Familiarity with OpenTelemetry, eBPF, and AI-assisted development tools is a plus.

Education

• Relevant degree or equivalent experience in a related field is preferred.

Tools

• Experience with CI/CD tools such as Ansible, GitLab CI, or Jenkins.
• Knowledge of Kubernetes is useful for supporting specific platform components.