Quality Assurance Engineer (Automation)
The Role
You’ll own the quality of the Lyrebird platform, from the first commit to the release gate. This is a hands-on, code-first engineering role, not manual test scripting. You’ll make our Playwright end-to-end and integration suites faster and more reliable, build contract tests that stop backend changes from silently breaking the web app, and design ways to evaluate clinical model output where exact-match assertions don’t work. You’ll also be the quality partner to our SRE function, giving them the health checks, synthetic monitors, load test results and post-deployment signals they need. When a release is ready to go, you’ll give a clear yes or no, backed by evidence.
This role suits an engineer who sees testing as a craft and treats a test suite as production code. You’re comfortable moving between TypeScript and Python in the same day, you get real satisfaction from deleting a flaky test and cutting minutes off a pipeline, and you’d rather help other engineers write better tests than be the only person who writes them. You’ll enjoy the hard problem at the centre of this role: how to prove that non-deterministic AI output is safe and accurate enough for clinicians to rely on.
About Us
Lyrebird Health is transforming the quality and accessibility of healthcare by automating clinicians’ most time-consuming tasks.
Thousands of clinicians across multiple disciplines use Lyrebird every day in environments where accuracy, safety, and time matter deeply. Everything we build is grounded in earning trust, reducing cognitive load, and delivering meaningful impact at scale.
What you’ll do
Optimise our Playwright end-to-end suite, cutting run time and removing flaky tests for good
Extend our API and partner integration suites, and build contract tests from our OpenAPI schemas so no backend change can break the web app without warning
Add coverage gates to our Buildkite pipeline, using reporting that already breaks quality down by product area
Support and collaborate with our engineering teams with CICD development and oversight
Test the clinical output of our models with DeepEval, openevals and Langfuse traces, designing evaluations that hold up when responses change from run to run
Run load tests with Locust ahead of major releases and hand clear findings to the SRE function
Build the signals SRE depends on: health checks, synthetic monitors and post-deployment tests
Own the release quality gate, giving a clear yes or no with the evidence behind it
Raise the standard of testing across engineering through shared patterns, fixtures and thoughtful code review
What you’ll bring
Strong test automation experience in both TypeScript and Python, writing tests as production-quality code
Deep hands-on experience with Playwright, plus component testing with Vitest and Testing Library
Solid pytest skills, including async tests, fixtures, mocking and test data factories such as polyfactory or Faker
Experience running tests in CI pipelines (ideally Buildkite) and in Docker, with coverage measurement and reporting
Working knowledge of SQL and Postgres, and confidence testing at the database layer
Experience with API contract testing against OpenAPI schemas
Nice to have
Experience evaluating large language model output with tools such as DeepEval, openevals or Langfuse
Load testing experience with Locust
Familiarity with Svelte, Storybook, or Supabase and its CLI-based database testing
Experience with observability tools such as Sentry, OpenTelemetry and Honeycomb
A background in healthcare, clinical software or other regulated, safety-critical environments
Every test you write protects a clinician who is relying on Lyrebird in the middle of a busy day, and the patient sitting in front of them. When our platform is fast, stable and trustworthy, clinicians spend less time on admin and more time on care. You’ll be the person who makes sure we can promise that with confidence, release after release.
We’re building a team that reflects the diversity of the people who benefit from our work. We want Lyrebird to be a place where everyone feels safe, supported, and able to thrive. If you’re from an underrepresented background in tech, we strongly encourage you to apply, even if you don’t meet every requirement.