Python Engineer (Freelance, Remote)

Job summary

This role involves designing and evaluating tasks for AI coding agents to ensure they complete assignments safely and correctly. You will create realistic development environments and write tests to assess the agents' performance.

Qualifications

• 4–5+ years of experience in software development.
• Proficiency in Python and JavaScript/TypeScript.
• Strong skills in test design, focusing on functional and integration tests.
• Experience with coding agents such as Claude Code, GitHub Copilot CLI, or Codex.
• Familiarity with GitHub pull requests and CI workflows.
• English proficiency at B2 level or higher.

Responsibilities

• Build realistic developer environments that simulate a virtual company.
• Design tasks that present both safe and unsafe development paths.
• Write tests to verify the agents' task completion methods.
• Iterate on tasks and tests based on quality assurance feedback.

Skills

• Strong software engineering background.
• Ability to create challenging tasks for AI models.
• Experience in backend and infrastructure development is beneficial.

Education

• Relevant degree or equivalent experience in software development is preferred.

Tools

• Python, JavaScript/TypeScript, GitHub, CI/CD tools.