๐ง๐ต๐ถ๐ ๐ฟ๐ผ๐น๐ฒ ๐ถ๐ ๐ณ๐ผ๐ฟ ๐ผ๐ป๐ฒ ๐ผ๐ณ ๐๐ต๐ฒ ๐ช๐ฒ๐ฒ๐ธ๐ฑ๐ฎ๐'๐ ๐ฐ๐น๐ถ๐ฒ๐ป๐๐
๐ฆ๐ฎ๐น๐ฎ๐ฟ๐ ๐ฟ๐ฎ๐ป๐ด๐ฒ: ๐ฅ๐ ๐ญ๐ฌ๐ฌ๐ฌ๐ฌ๐ฌ๐ฌ - ๐ฅ๐ ๐ฎ๐ฑ๐ฌ๐ฌ๐ฌ๐ฌ๐ฌ (๐ถ๐ฒ ๐๐ก๐ฅ ๐ญ๐ฌ-๐ฎ๐ฑ ๐๐ฃ๐)
Experience: 4+ yrs
Location: Remote (India)
As a QA Engineer, you will design, develop, and execute comprehensive test strategies for advanced Generative AI and large language model (LLM) solutions, ensuring they meet strict quality, security, and compliance standards. You will work closely with data scientists, ML engineers, and product managers to validate Agentic AI and Retrieval-Augmented Generation (RAG) workflows, turning complex AI behaviors into measurable test cases. Leveraging your strong command of pytest, Python, and API testing frameworks, you will build robust automation pipelines that seamlessly integrate into continuous integration/continuous deployment (CI/CD) pipelines, while also providing actionable insights to improve model reliability and user experience. Your role will involve monitoring performance metrics, detecting anomalies in generated outputs, and maintaining detailed documentation and dashboards for crossโfunctional stakeholders, all while driving the adoption of scalable QA practices across the organization.
Requirements
๐๐ฒ๐ ๐ฅ๐ฒ๐๐ฝ๐ผ๐ป๐๐ถ๐ฏ๐ถ๐น๐ถ๐๐ถ๐ฒ๐
โข Define and implement test plans for GenAI, LLM, Agentic AI, and RAG systems
โข Design and maintain automated test suites using pytest and Python
โข Develop and execute API tests for model endpoints, data pipelines, and microservices
โข Perform functional, regression, and performance testing of AI services
โข Integrate testing into CI/CD pipelines with tools like Jenkins, GitHub Actions, or GitLab CI
โข Analyze model output quality, bias, and fairness metrics and document findings
โข Collaborate with ML teams to create test harnesses for new model releases
โข Create and maintain test data sets, fixtures, and mock services
โข Report defects, track resolution, and validate fixes to ensure quality standards
โข Mentor junior testers and promote best practices in AI QA
๐ช๐ต๐ฎ๐ ๐ ๐ฎ๐ธ๐ฒ๐ ๐ฌ๐ผ๐ ๐ฎ ๐๐ฟ๐ฒ๐ฎ๐ ๐๐ถ๐
โข 4+ years of QA engineering experience in AI or data science environments
โข Strong proficiency in pytest, Python, and API testing frameworks (e.g., Requests, RestAssured)
โข Handsโon experience with GenAI, LLM, Agentic AI, and RAG testing methodologies
โข Familiarity with CI/CD pipelines and automation tooling (Jenkins, GitHub Actions)
โข Deep understanding of AI ethics, bias mitigation, and model explainability
โข Excellent analytical and problemโsolving skills with a detailโoriented mindset
โข Ability to translate complex AI concepts into clear, testable requirements
โข Strong communication skills for crossโfunctional collaboration
โข Proven track record of delivering highโquality releases on time
โข Passion for staying current with emerging AI technologies and testing trends