Senior Full Stack QA Engineer (Freelance)

Company Intro

At Toloka AI we create data that powers leading GenAI models and innovations. We work with frontier labs, big tech, renowned AI startups, enterprises, and non-profit research organizations worldwide. We use a combination of Experts + Crowd + Tech Platform to teach AI models to reason and evaluate their efficacy and safety. We have experts in more than 50 different domains—from doctors and lawyers to physicists and engineers—and boast one of the most diverse global crowds, representing over 100 countries and speaking 40+ languages. We are a well-funded startup with an enviable portfolio of clients, including Anthropic, Amazon, Microsoft, Poolside, Recraft, and Shopify.

Recently, we secured a strategic investment led by Bezos Expeditions and Nebius Group with participation from Mikhail Parakhin, CTO of Shopify and board advisor to leading GenAI companies, who now serves as our Chairman of the Board. Our remote-first team is globally distributed around the world: USA, UK, the Netherlands, Israel, Czech Republic, Serbia, and more.

 

About the Role

We are looking for a Senior QA Engineer who thinks beyond test cases, who understands the product deeply, knows what actually matters, and helps the team move fast without breaking things that count. This isn't a role for someone who needs clear processes handed to them. You'll be working across frontend, backend, and ML teams — each with their own release flows — and your job is to find the right balance between speed and quality, and bring the team along with you.

What you’ll do

• Take end-to-end ownership of quality — from reviewing requirements and shaping test strategies to maintaining bug reports, tracking known issues, and surfacing bottlenecks to the team
• Own quality across the full system — from the AI agent and orchestration layer to the interfaces users and operators interact with
• Design and maintain automated tests that account for non-deterministic AI behavior
• Debug issues across a multi-component distributed system — and know how to find where in the chain something broke
• Identify the highest-risk areas, prioritize ruthlessly, and know when to stop
• Work with AI tools to write and maintain tests efficiently
• Shape QA culture across teams without imposing strict rules

What we're Looking For

• 5+ years of QA experience with a strong manual testing foundation
• Experience testing complex, multi-component distributed systems — you know how services talk to each other and you don't freeze when something breaks in an unexpected place
• Practical experience with test automation
• Familiarity with LLMs and AI-first products — you understand why non-determinism isn't a bug
• Experience using AI tools (Cursor, Copilot, etc.) in your daily workflow
• Strong prioritization skills — you can handle a high volume of changes without losing sight of what matters
• Comfortable operating in ambiguity and building structure where there isn't any

What we can offer

• Freelance collaboration via the Ontop freelance platform.
• Flexible, fully remote work schedule.
• You’ll have the opportunity to work on highly innovative projects at the leading edge of AI development, together with a genuinely dedicated and dynamic team of experts.
• You’ll also be working on projects with customers that are AI industry leaders and well-known household names.
• Friendly community.

 

Equal Opportunity Employer:

Toloka is committed to providing equal opportunity and fostering an inclusive environment. We welcome applicat