Job Description

As a Gen AI Quality Assurance Lead, the successful candidate will be accountable for establishing quality assurance frameworks and processes to ensure Gen AI technology solutions undergo comprehensive and efficient testing. The successful candidate will be also overseeing a team that designs, implements, and maintains both manual and automated testing procedures, and conducts functional, system, integration, and performance testing.

Responsibilities:
Team leadership (50%)

• Lead and manage the Gen AI QA team, providing guidance, mentorship and support
• Manage allocation of Gen AI QA resources to projects based on capacity and skillset
• Develop, implement and enforce comprehensive testing frameworks, processes and tooling and continuously optimize them to enhance their effectiveness, speed and quality

Quality Assurance testing (50%)

• Ensure the quality and reliability of Gen AI solutions through rigorous testing and validation
• Contribute to insights, risk evaluations and estimation discussions to ensure alignment quality standards and that testing considerations are well-integrated into project planning
• Analyze project requirements, user feedback, and architectural and design documents to ensure completeness and testability
• Design, document and execute robust test strategies, test cases, scenarios, and scripts tailored to the specific requirements of Gen AI projects to ensure product quality
• Utilize defect tracking system to record, trace, and drive the resolution of defects and issues discovered during testing
• Investigate and determine the source of problems, which may include identifying test deficiencies or deviations from requirements
• Assist in reproducing issues and conducting root cause analysis to facilitate swift resolution

Qualifications:

• Undergraduate Degree in a technical field
• 6+ years’ experience performing different types of testing for complex applications
• 2+ years of experience leading technology QA teams
• 5+ years using automated testing tools and platforms 
• Knowledge of Python for Gen AI development and testing
• Competent in designing and executing AI prompt engineering and testing scenarios, ensuring the robustness of AI and large language models
• Microsoft Copilot/Agents experience

Preferred Qualifications:

• Experience with software development tools and defect tracking systems like Azure DevOps, Jira
• Experience with automation testing tools (Playwright, JMeter, SoapUI, postman)
• Database experience (SQL)
• Working with high level UX principles such as user stories, persona building, prototype/prototype development approaches and information architecture and customer/product research methodologies (e.g., A/B testing and MVT)
• Strong leadership and communication skills
• Demonstrate excellent written and verbal communication when working with product teams and working with end customers to understand and analyze feedback
• Understand client’s needs, actively ensuring projects are on time and delivered above expectation. Will be in communication to solve any problems if necessary.