Senior Software Engineer in Test

Role Summary 

We are looking for a Senior Software Engineer in Test. At your core, you are a software engineer—your product is the automation that ensures our customers never experience defects in production. 

You will work as part of a globally distributed Agile team (Scrum and ShapeUp), partnering closely with engineering and product, with ownership of test automation, quality strategy, and deployment validation. 

Objectives of this Role 

• Collaborate with developers and product owners to plan, design, and review product code and define effective test strategies 

• Design and build scalable test automation frameworks (Playwright) across UI, API, and integration layers 

• Own automated test execution within CI/CD pipelines (Azure DevOps), including parallelization and test data management 

• Build real-time visibility into product quality through dashboards and reporting 

• Advocate for strong quality engineering practices across the team 

• Develop tools and utilities to support exploratory and manual testing 

• Leverage Agentic LLMs (e.g., Claude) to accelerate test creation, improve coverage, and identify gaps—while validating outputs with engineering judgment 

AI-Driven Engineering Expectations 

Engineers are expected to actively incorporate AI-assisted workflows into daily development, including test generation, defect analysis, and coverage improvement. AI-generated outputs must be reviewed, validated, and owned by the engineer. 

Qualifications / Skills 

• Strong collaboration skills across engineering, product, and distributed teams 

• Deep understanding of Agile development (Scrum and/or ShapeUp) and modern quality engineering practices 

• Automation-first mindset; actively eliminates repetitive manual work 

• Hands-on experience with Azure environments and CI/CD pipelines (ADO) 

• Strong debugging and problem-solving skills 

• Ability to mentor engineers in quality and testing practices 

• Experience using AI tools to enhance testing, while maintaining accountability for outcomes 

Education and Experience Requirements 

• BS/BA in Computer Science or equivalent practical experience 

• Proven experience building and maintaining automated test frameworks for modern web applications 

• Experience integrating automated testing into CI/CD pipelines at scale 

Daily and Monthly Responsibilities 

• Build and maintain automated test gates across the delivery pipeline 

• Own quality validation within deployment workflows in partnership with engineering 

• Identify, isolate, and drive resolution of defects across environments 

• Investigate production issues and contribute to root cause analysis