Add LLMs, agents and AI pipelines to your testing skill set.
Become the QA engineer who can test AI with confidence
Twelve live online sessions. Hands-on labs with Promptfoo, DeepEval and Ragas. And a certificate you can put in front of any hiring manager.
12
Live sessions
1hr
Per session
100%
Hands-on
Live online training
Weekly sessions
$500full course
Walk away with skills you can use on day one
Twelve live sessions turn AI testing from a mystery into a skill set you can use at work, in interviews and with clients.
Practical AI testing skills
Test LLMs, chatbots, RAG, agents and voice AI with structured methods instead of guesswork.
Hands-on tool experience
Learn Promptfoo, DeepEval and Ragas, the open source tools AI QA teams use to evaluate AI apps.
A hands-on lab every session
Practise on realistic AI apps, then pull it all together in a capstone project you can show to employers.
A certificate you can share
Receive a Testiva AI Testing certificate to add to LinkedIn and stand out to hiring managers and clients.
New career options
Move toward AI QA Engineer, LLM Tester and AI Red Team roles, or offer AI testing audits as a freelance service.
Recordings and learning material
Revisit every session at your own pace and keep the material after the training ends.
Who is this training for?
If you test, build, secure or manage software that uses AI, this course is made for you.
Test your own AI code with structured evaluation, so you are not the only reviewer.
Add AI evaluation to your existing test pipelines.
Master adversarial testing and prompt injection methods.
Learn what AI testing needs so you can plan and resource it.
Validate clinical AI, one of the highest stakes areas for AI quality.
No prior AI experience needed. Session 1 starts from the basics.
What changes for you after this training
On the left is where most QA engineers are today. On the right is what you will be able to do after the 12 sessions.
You today
After the training
AI apps give different answers every time, and you are not sure how to test them.
You will be able to explain how AI fails and plan clear tests for it.
You cannot tell if an AI answer is good, accurate or safe.
You will be able to score answers with Promptfoo assertions and DeepEval metrics.
Checking AI answers by hand takes too long.
You will be able to use LLM-as-a-Judge and synthetic test data to check answers at scale.
RAG apps and agents feel like black boxes.
You will be able to evaluate RAG applications using DeepEval and RAGAS, as well as trace agent execution and assess agent performance.
AI security feels like someone else’s job.
You will be able to red team an AI app with Promptfoo and write the vulnerability report.
Your CV shows traditional QA experience only.
You will have a hands-on AI application testing and evaluation project to showcase on your CV, along with a Testiva certificate to demonstrate your skills to potential employers.
Who's teaching you
Learn from specialists who test AI every day
Testiva is a specialist QA company with deep, hands-on expertise in AI and LLM testing. Our team has designed and executed testing strategies for production AI systems across industries from healthcare and fintech to SaaS and enterprise platforms.
Every instructor has tested real AI applications in the wild, not just studied them.
15+
AI apps tested
In production, across healthcare, fintech, SaaS and enterprise
12 sessions: Promptfoo, DeepEval and Ragas
The program teaches AI QA through two main evaluation tools in sequence, Promptfoo then DeepEval, with a short Ragas overview inside the DeepEval block. Every session is one focused hour pairing core concepts with a hands-on lab.
Introduction
1 session
Concepts
Why AI needs a different kind of QA
What evals are, and the evaluation lifecycle
LLM failure modes: hallucination, drift, non-determinism
Where Promptfoo, DeepEval and Ragas each fit
LabSet up the environment and install the tools
Promptfoo
Block 1, 5 sessions
Promptfoo
promptfooconfig.yaml: prompts, providers, tests, vars
The results viewer
Comparing models side by side
Compare three models on one prompt set
PromptfooMetrics
Deterministic checks: contains, regex, is-json
Custom JavaScript and Python assertions
Statistical metrics: BLEU, ROUGE, similar
Reference-based vs reference-free metrics
Test a structured-output extraction prompt
PromptfooLLM-as-a-Judge
LLM-rubric, G-Eval, factuality, select-best
Judge biases: position, verbosity, self-preference
Configuring the grader model
Checking the judge against human labels
Writing good rubrics
Build judges for tone, accuracy and helpfulness, then spot-check them against 20 hand-labelled outputs
PromptfooTest dataRegression
Weights, thresholds, assert-set
CSV datasets and promptfoo generate dataset
Named and derived metrics
Testing a live app via the HTTP provider
Latency and cost assertions
Comparing eval runs to catch regressions
Build a weighted scorecard on a 50-case dataset against a live chatbot API, then re-run after a prompt change and find what regressed
PromptfooRed teaming
Purpose, targets, plugins, strategies
The red teaming report
Domain-specific plugins
OWASP Top 10 for LLMs
Red team an AI assistant and produce a vulnerability report
DeepEval
Block 2, 5 sessions
DeepEvalMetrics
LLMTestCase
Writing evals as pytest tests with deepeval test run
Running built-in metrics
Custom metrics with G-Eval and DAG
Turn the Promptfoo chatbot tests into a pytest suite with a custom G-Eval metric
DeepEvalSynthetic data
DeepEval Synthesizer: goldens from documents
Golden dataset design
Evaluation Dataset
Checking the quality of synthetic data
Generate a test set from company documents
DeepEvalRagasRAG
Contextual Precision, Recall and Relevancy
Hallucination detection
Answer Relevancy and Faithfulness
Ragas overview: core RAG metrics and when to use it alongside DeepEval
Testing retriever and generator separately
Evaluate a knowledge-base assistant with DeepEval, then cross-check the scores with Ragas
DeepEvalConversations
Conversational TestCase and Conversation Simulator
Safety metrics: Bias, Toxicity, PII Leakage
Knowledge Retention, Role Adherence, Conversation Completeness
Simulate 20 customer conversations and score them for quality and safety
DeepEvalAgents
@observe tracing
Tool Correctness and Argument Correctness
Task Completion, Step Efficiency, Plan Adherence
Trace and evaluate a travel planning agent
Capstone
1 session
All tools
Promptfoo for scorecards and red teaming
Ragas as an optional cross-check on RAG
DeepEval for judges, synthetic data, RAG and agents
Presenting results to stakeholders
Build and present a full eval suite for an AI application of your choice
What participants say
Feedback from QA professionals who attended Testiva’s AI testing training, shared on Google.
4.9
★★★★★
Average rating from 8 Google reviews
Had a great learning experience with Testiva’s training session on AI-powered testing and QA automation. The content was practical and up-to-date, covering real-world approaches to AI-powered testing.
Muhammad Mujahid
Google review
My experience with Testiva was very positive. The instructor explained the concepts of AI Testing clearly and effectively while following the shared course outline.
Raheel Riaz
Google review
I had a great learning experience with Testiva’s AI/ML Testing course, with practical insights into AI testing and modern QA practices.
Naeem Khan
Google review
The AI/ML Testing course was an excellent and insightful learning experience. I particularly appreciated the practical examples and real-world testing scenarios.
Maria
Google review
I attended Testiva’s AI Testing training program, it was exceptional
Malik Irfan
Google review
Contents are very informative
Anj P
Google review
Reserve your seat in 4 easy steps
Fill in your details and the Testiva team will contact you within 24 hours with payment instructions. Your seat is confirmed once payment is received.
Fill the registration form
Testiva contacts you in 24 hrs
Complete your payment
Seat confirmed and joining link sent
Course fee
Full 12-session course, all included
$500full course
All 12 live sessions (1 hr each)
Hands-on lab every session
Capstone project
Session recordings included
Tool setup guidance
Certificate of completion
Registration Form
Tell us a little about yourself.
Team or Corporate Bookings
Group pricing and private cohorts available for teams of 5 or more. Get in touch info@testiva.io to discuss a dedicated run for your organisation.