Advanced AI Quality Assurance Specialist

2 days ago

Dublin, Dublin City, Ireland beBeeAutomation Full time €90,000 - €120,000
Overview

To excel as a Staff AI QA Engineer, you'll be responsible for crafting advanced testing strategies and automation frameworks tailored for adaptive, large-scale agentic systems and complex integrations.

As a key member of our dynamic team, you'll validate AI behavior, secure data pipelines, and enforce quality standards across distributed systems to ensure our intelligent agents operate with precision, fairness, and resilience in real-world environments.

Key Responsibilities
  • Design and implement comprehensive test strategies for AI agents, platform services, and integrations
  • Develop and maintain automated test frameworks for functional, performance, and security testing
  • Validate AI model behavior, including prompt outputs, bias detection, and explainability
  • Conduct rigorous testing of distributed systems, microservices, and real-time data pipelines
  • Ensure data quality and integrity across training, inference, and integration workflows
  • Collaborate with engineering and research teams to define quality gates and ethical testing standards
  • Drive continuous testing practices in agile development cycles
Qualifications and Skills
  • 5–7+ years of experience in QA or Test Engineering
  • Mastery of test automation frameworks (e.g., PyTest, Playwright, Selenium, RestAssured)
  • Strong API testing skills (manual and automated)
  • Experience with performance, load, and high-volume stress testing (e.g., JMeter, k6, Gatling)
  • Deep understanding of AI/ML model evaluation metrics and methodologies
  • Experience testing distributed systems and microservices architectures
  • Proficiency in data quality validation for AI training and inference workflows
  • Experience with contract testing for microservices
  • Ability to develop test strategies for non-deterministic and adaptive behaviors in AI systems (e.g., prompt validation, guardrails, bias detection)
  • Familiarity with AI model testing tools (e.g., CheckList, Giskard, Deepchecks)
  • Experience with test planning, execution, and reporting in agile environments
  • Bachelor's degree in Computer Science, Software Engineering, or a related field
  • Preferred: Certifications in QA Automation, AI/ML Testing, or Ethical AI (e.g., ISTQB, Certified Ethical Emerging Technologist)
  • Additional training in AI safety, fairness, or model validation is a plus
Requirements and Preferences
  • Experience with AI red teaming and adversarial testing techniques
  • Explainability testing and validation for XAI methods (e.g., LIME, SHAP)
  • Usability testing for AI-driven UIs and agent interactions
  • Familiarity with data privacy testing techniques (e.g., differential privacy validation)
  • Security testing for AI models (e.g., model inversion, membership inference, poisoning attacks)
  • Experience testing AI ethics, fairness, and bias
  • Knowledge of formal verification methods for AI
  • Experience building synthetic data generation pipelines for testing

Seniority level: Mid-Senior level Employment type: Full-time Job function: Engineering and Information Technology Industries: Software Development

],

  • AI Model Quality Assurance Specialist

    7 days ago

    Dublin, Dublin City, Ireland beBeeAnnotator Full time €45,000 - €60,000

    AI Annotator - Quality Assurance Specialist",">We are seeking highly skilled and detail-oriented professionals to join our team as AI Quality Assurance Specialists. As an AI Annotator, you will play a crucial role in ensuring the accuracy of our AI models by investigating and resolving reported issues, enforcing clients' guidelines, and gathering data to...

  • AI-Powered Software Quality Assurance Specialist

    2 days ago

    Dublin, Dublin City, Ireland beBeeArtificialintelligence Full time €80,000 - €110,000

    Job Title: AI-Powered Software Quality Assurance SpecialistWe are seeking a talented and experienced Automation Test Engineer to join our team.

  • Quality Assurance Specialist

    3 weeks ago

    Dublin, Dublin City, Ireland Atrium (EMEA) Full time

    Direct message the job poster from Atrium (EMEA)Quality Assurance Specialist – Medical DevicesLocation: Dublin, Ireland (Hybrid – 2 to 3 days on site)Contract: Long-term projectThis isn't just another QA role - it's your chance to be part of something meaningful and exciting, where your voice will be heard, your ideas will help shape the way things are...

  • Staff AI Quality Assurance Specialist

    6 days ago

    Dublin, Dublin City, Ireland beBeeEngineer Full time €90,000 - €110,000

    Job Description:">The role of Staff AI Quality Assurance Engineer is a high-profile position within our organization. In this capacity, you will be responsible for ensuring the reliability, safety, and ethical integrity of our artificial intelligence platform.">Key Responsibilities:">">Design and implement comprehensive test strategies for AI agents,...

  • AI Quality Assurance Expert Wanted

    11 hours ago

    Dublin, Dublin City, Ireland beBeeAssurance Full time €90,000 - €126,000

    AI Quality Assurance Staff EngineerWe are seeking a highly skilled and experienced AI quality assurance professional to join our team.Develop and maintain comprehensive test strategies for AI agents, platform services, and integrations.Design and implement automated test frameworks for functional, performance, and security testing.Validate AI model behavior,...

  • Advanced Quality Assurance Professional

    1 day ago

    Dublin, Dublin City, Ireland beBeetest Full time €60,000 - €85,000

    Job Title: Quality Assurance SpecialistWe are seeking a highly skilled and experienced Quality Assurance Specialist to join our team. As a key member of the testing team, you will be responsible for delivering and executing test cases for various systems.Your primary focus will be on browser-based transactions, webservices, regression tests, batch program...

  • AI Quality Assurance Specialist

    2 days ago

    Dublin, Dublin City, Ireland beBeeQualityEngineer Full time €100,000 - €125,000

    Job Summary:">We are seeking a highly skilled and experienced QA Engineer to join our team. The successful candidate will be responsible for ensuring the reliability, safety, and ethical integrity of our AI platform by crafting advanced testing strategies and automation frameworks.">Responsibilities:">Design and implement comprehensive test strategies for AI...

  • AI Systems Quality Specialist

    3 days ago

    Dublin, Dublin City, Ireland beBeeAnalytical Full time €50,000 - €70,000

    Job Title: AI Systems Quality SpecialistAs an AI Systems Quality Specialist, you will play a pivotal role in ensuring the accuracy and reliability of our AI systems. Your keen eye for detail and interest in AI will make you a valuable asset to our global team.Your Key Responsibilities:Investigate and resolve reported issues related to account support or...

  • Quality Assurance Specialist

    7 days ago

    Dublin, Dublin City, Ireland beBeeValidation Full time

    Quality Assurance SpecialistWe are seeking a skilled Quality Assurance Specialist to join our team.

  • Quality Assurance Specialist

    6 days ago

    Dublin, Dublin City, Ireland beBeequality Full time €50,000 - €70,000

    Quality Assurance SpecialistThis role is centered around ensuring products and processes meet quality standards and adhere to regulatory requirements.Analyzing production data is essential in identifying areas for improvement.We are seeking a Quality Assurance Specialist who will lead the setup and management of our Quality Management system, aligned with...