Automated Testing in Python & 6 others
EPAM Systems
Software Engineering
Belarus
Posted on Nov 19, 2025
Responsibilities
- Design and implement comprehensive testing strategies tailored to AI workflows, APIs, and cloud-native solutions
- Develop detailed test plans, test cases, and scripts for AI systems, including data pipelines, knowledge bases, vector databases, and LLM-powered components
- Validate the performance and accuracy of LLM outputs, ensuring compliance with prompt engineering best practices and client-specific requirements
- Test RAG-based architectures, including retrieval processes, embedding validation, and semantic search accuracy
- Ensure seamless integration of data ingestion, cleansing, transformation, and validation pipelines with vector databases and AI models
- Build and maintain test automation frameworks for AI components and data pipelines using industry-standard tools and custom scripts
- Conduct performance, load, and scalability testing for AI/LLM endpoints, RAG workflows, and deployed pipelines
- Test cloud infrastructure deployment (e.g., IaC via Terraform/CloudFormation) and ensure production-readiness for AWS Bedrock, S3, EKS, and vector database configurations
- Identify, log, and manage defects in AI workflows, APIs, data pipelines, and backend systems, collaborating with developers and DevOps teams to resolve issues
- Monitor system reliability, availability, and fault tolerance in cloud environments to ensure production-level readiness
Requirements
- Proficiency in test automation using Python (3+ years of experience)
- Proven experience in QA engineering, with a focus on testing AI systems and cloud-native solutions
- Strong knowledge of testing frameworks, tools, and methodologies for AI workflows, APIs, and data pipelines
- Hands-on experience with LLMs, RAG architectures, and vector databases
- Proficiency in test automation using Python or similar scripting languages
- Expertise in performance and scalability testing for AI systems
- Familiarity with cloud environments, particularly AWS services like Bedrock, S3, and EKS
- Solid understanding of Infrastructure as Code (IaC) tools such as Terraform or CloudFormation
- Excellent problem-solving skills and ability to collaborate effectively with cross-functional teams
Nice to have
- Experience with AWS Bedrock capabilities and generative AI solutions
- Knowledge of semantic search and embedding validation techniques
- Familiarity with continuous integration and delivery pipelines for AI systems
- Background in data engineering or data science
We offer/Benefits
We connect like-minded people:
- Delivering innovative solutions to industry leaders, making a global impact
- Enjoyable working environment, whether it's the vibrant office or the comfort of your own home
- Opportunity to work abroad for up to two months per year
We invest in your growth:
- Leadership development, career advising, soft skills and well-being programs
- Certifications, including GCP, Azure and AWS
- Unlimited access to LinkedIn Learning and Get Abstract
- Free English classes with certified teachers
We cover it all:
- Monetary bonuses for taking part in the referral program
- Comprehensive medical & family care package
- Seven trust days per year (sick leave without a medical certificate)
- Discounts from 800+ partners (sports activities, restaurants, stores and services)