XM

Senior AI QA Automation Test Engineer

  • Full-time
  • Hybrid – Limassol, Cyprus
  • Posted 2mo ago
Apply now

Job description

The main responsibilities of the position include:

  • Acting as the primary technical enabler for the QA organization by building scalable AI/ML frameworks, libraries, and tooling that support broader engineering adoption
  • Collaborating closely with QA, Data Science, and Engineering teams to ensure seamless integration of AI-driven testing capabilities within the CI/CD ecosystem
  • Leading research and experimentation initiatives focused on emerging AI testing methodologies, tools, and best practices
  • Mentoring and supporting engineers through hands-on collaboration, code reviews, technical workshops, and architectural guidance
  • Designing and implementing advanced autonomous QA agents and workflows using modern AI orchestration frameworks and technologies
  • Building sophisticated AI evaluation pipelines to assess reasoning quality, robustness, hallucination rates, fairness, and overall model reliability
  • Developing resilient, AI-augmented, and self-healing automation frameworks capable of adapting to dynamic product and UI changes
  • Implementing machine learning-driven analytics and intelligent quality engineering solutions, including predictive quality insights, root cause analysis, and smart test prioritization 

    • Main requirements:

  • BSc/MSc in Computer Science, Artificial Intelligence, or related discipline
  • 8+ years of hands-on experience in AQA  
  • 1+ years of experience applying AI or ML technologies in software testing or QA process improvement  
  • A proven history of personally building and integrating AI/ML models into production workflows or SDLC processes
  • Coding proficiency in Java/Python and/or TypeScript, with a deep, practical understanding of complex software architecture and distributed system design
  • Hands-on experience designing and implementing complex AI agent architectures (LLM-as-a-judge, human-in-the-loop, RAG, multi-agent orchestration)
  • Deep architectural knowledge of modern AI/ML tooling (LLMs, vector databases, MLOps pipelines)
  • Strong background in integrating advanced tooling into enterprise CI/CD pipelines (GitLab, Jenkins, GitHub Actions) and containerized cloud-native environments (Docker, Kubernetes)
  • Exceptional ability to communicate complex technical concepts clearly, influence engineering standards without direct authority, and collaborate effectively across disciplines 

    • The following will be considered an advantage:

  • Extensive experience with autonomous QA agents and agentic orchestration frameworks in building self-evolving test suites
  • Expertise in high-fidelity AI evaluation pipelines and real-time observability (e.g., LangSmith, Arize) to measure probabilistic outcomes and adversarial robustness
  • Knowledge of AI ethics, fairness, and bias detection in model validation
  • Experience with gRPC, WebSockets, and HTTP/2
  • Familiarity with cloud-native AI solutions (AWS Bedrock, GCP Vertex AI, Azure AI) 

    • Benefit from:

  • Attractive remuneration package
  • Intellectually stimulating work environment
  • Continuous personal development and international training opportunities

    • The Hiring Experience: What Awaits You

  • Let’s Connect – Intro Chat with Talent Acquisition
  • Deep Dive – First Interview with Your Future Team
  • Final Connection – Final Interview