Skip to main content

Testiva

Become the QA engineer who can test AI with confidence

Twelve live online sessions. Hands-on labs with Promptfoo, DeepEval and Ragas. And a certificate you can put in front of any hiring manager.

12

Live sessions

1hr

Per session

100%

Hands-on

Live online training
Weekly sessions

$500full course

12 live sessions, delivered weekly
A hands-on lab every session
Recordings included
Certificate on completion
Promptfoo DeepEval Rages

Walk away with skills you can use on day one

Twelve live sessions turn AI testing from a mystery into a skill set you can use at work, in interviews and with clients.

Practical AI testing skills

Test LLMs, chatbots, RAG, agents and voice AI with structured methods instead of guesswork.

Hands-on tool experience

Learn Promptfoo, DeepEval and Ragas, the open source tools AI QA teams use to evaluate AI apps.

A hands-on lab every session

Practise on realistic AI apps, then pull it all together in a capstone project you can show to employers.

A certificate you can share

Receive a Testiva AI Testing certificate to add to LinkedIn and stand out to hiring managers and clients.

New career options

Move toward AI QA Engineer, LLM Tester and AI Red Team roles, or offer AI testing audits as a freelance service.

Recordings and learning material

Revisit every session at your own pace and keep the material after the training ends.

Who is this training for?

If you test, build, secure or manage software that uses AI, this course is made for you.

QA engineers and leads

Add LLMs, agents and AI pipelines to your testing skill set.

AI engineers

Test your own AI code with structured evaluation, so you are not the only reviewer.

Automation engineers

Add AI evaluation to your existing test pipelines.

Security and red team testers

Master adversarial testing and prompt injection methods.

QA managers

Learn what AI testing needs so you can plan and resource it.

Healthcare QA specialists

Validate clinical AI, one of the highest stakes areas for AI quality.

No prior AI experience needed. Session 1 starts from the basics.

What changes for you after this training

On the left is where most QA engineers are today. On the right is what you will be able to do after the 12 sessions.

You today

After the training

AI apps give different answers every time, and you are not sure how to test them.

You will be able to explain how AI fails and plan clear tests for it.

You cannot tell if an AI answer is good, accurate or safe.

You will be able to score answers with Promptfoo assertions and DeepEval metrics.

Checking AI answers by hand takes too long.

You will be able to use LLM-as-a-Judge and synthetic test data to check answers at scale.

RAG apps and agents feel like black boxes.

You will be able to evaluate RAG applications using DeepEval and RAGAS, as well as trace agent execution and assess agent performance.

AI security feels like someone else’s job.

You will be able to red team an AI app with Promptfoo and write the vulnerability report.

Your CV shows traditional QA experience only.

You will have a hands-on AI application testing and evaluation project to showcase on your CV, along with a Testiva certificate to demonstrate your skills to potential employers.

Who's teaching you

Learn from specialists who test AI every day

Testiva is a specialist QA company with deep, hands-on expertise in AI and LLM testing. Our team has designed and executed testing strategies for production AI systems across industries from healthcare and fintech to SaaS and enterprise platforms.

Every instructor has tested real AI applications in the wild, not just studied them.

15+

AI apps tested

In production, across healthcare, fintech, SaaS and enterprise

LLM native expertise
Red team certified
UK / US client delivery

12 sessions: Promptfoo, DeepEval and Ragas

The program teaches AI QA through two main evaluation tools in sequence, Promptfoo then DeepEval, with a short Ragas overview inside the DeepEval block. Every session is one focused hour pairing core concepts with a hands-on lab.

Introduction

1 session

Concepts
Why AI needs a different kind of QA
What evals are, and the evaluation lifecycle
LLM failure modes: hallucination, drift, non-determinism
Where Promptfoo, DeepEval and Ragas each fit
LabSet up the environment and install the tools
Promptfoo

Block 1, 5 sessions

Promptfoo
promptfooconfig.yaml: prompts, providers, tests, vars
The results viewer
Comparing models side by side
Compare three models on one prompt set
PromptfooMetrics
Deterministic checks: contains, regex, is-json
Custom JavaScript and Python assertions
Statistical metrics: BLEU, ROUGE, similar
Reference-based vs reference-free metrics
Test a structured-output extraction prompt
PromptfooLLM-as-a-Judge
LLM-rubric, G-Eval, factuality, select-best
Judge biases: position, verbosity, self-preference
Configuring the grader model
Checking the judge against human labels
Writing good rubrics
Build judges for tone, accuracy and helpfulness, then spot-check them against 20 hand-labelled outputs
PromptfooTest dataRegression
Weights, thresholds, assert-set
CSV datasets and promptfoo generate dataset
Named and derived metrics
Testing a live app via the HTTP provider
Latency and cost assertions
Comparing eval runs to catch regressions
Build a weighted scorecard on a 50-case dataset against a live chatbot API, then re-run after a prompt change and find what regressed
PromptfooRed teaming
Purpose, targets, plugins, strategies
The red teaming report
Domain-specific plugins
OWASP Top 10 for LLMs
Red team an AI assistant and produce a vulnerability report
DeepEval

Block 2, 5 sessions

DeepEvalMetrics
LLMTestCase
Writing evals as pytest tests with deepeval test run
Running built-in metrics
Custom metrics with G-Eval and DAG
Turn the Promptfoo chatbot tests into a pytest suite with a custom G-Eval metric
DeepEvalSynthetic data
DeepEval Synthesizer: goldens from documents
Golden dataset design
Evaluation Dataset
Checking the quality of synthetic data
Generate a test set from company documents
DeepEvalRagasRAG
Contextual Precision, Recall and Relevancy
Hallucination detection
Answer Relevancy and Faithfulness
Ragas overview: core RAG metrics and when to use it alongside DeepEval
Testing retriever and generator separately
Evaluate a knowledge-base assistant with DeepEval, then cross-check the scores with Ragas
DeepEvalConversations
Conversational TestCase and Conversation Simulator
Safety metrics: Bias, Toxicity, PII Leakage
Knowledge Retention, Role Adherence, Conversation Completeness
Simulate 20 customer conversations and score them for quality and safety
DeepEvalAgents
@observe tracing
Tool Correctness and Argument Correctness
Task Completion, Step Efficiency, Plan Adherence
Trace and evaluate a travel planning agent
Capstone

1 session

All tools
Promptfoo for scorecards and red teaming
Ragas as an optional cross-check on RAG
DeepEval for judges, synthetic data, RAG and agents
Presenting results to stakeholders
Build and present a full eval suite for an AI application of your choice

What participants say

Feedback from QA professionals who attended Testiva’s AI testing training, shared on Google.

4.9

★★★★★
Average rating from 8 Google reviews
★★★★★

Had a great learning experience with Testiva’s training session on AI-powered testing and QA automation. The content was practical and up-to-date, covering real-world approaches to AI-powered testing.

Muhammad Mujahid
Google review
★★★★★

My experience with Testiva was very positive. The instructor explained the concepts of AI Testing clearly and effectively while following the shared course outline.

Raheel Riaz
Google review
★★★★★

I had a great learning experience with Testiva’s AI/ML Testing course, with practical insights into AI testing and modern QA practices.

Naeem Khan
Google review
★★★★★

The AI/ML Testing course was an excellent and insightful learning experience. I particularly appreciated the practical examples and real-world testing scenarios.

Maria
Google review
★★★★★

I attended Testiva’s AI Testing training program, it was exceptional

Malik Irfan
Google review
★★★★★

Contents are very informative

Anj P
Google review

Reserve your seat in 4 easy steps

Fill in your details and the Testiva team will contact you within 24 hours with payment instructions. Your seat is confirmed once payment is received.

1

Fill the registration form

2

Testiva contacts you in 24 hrs

3

Complete your payment

4

Seat confirmed and joining link sent

Course fee

Full 12-session course, all included

$500full course

  • All 12 live sessions (1 hr each)
  • Hands-on lab every session
  • Capstone project
  • Session recordings included
  • Tool setup guidance
  • Certificate of completion

Registration Form

Tell us a little about yourself.

    Team or Corporate Bookings

    Group pricing and private cohorts available for teams of 5 or more. Get in touch info@testiva.io to discuss a dedicated run for your organisation.

    Common questions

    Each session runs for one hour and follows the same format. The instructor introduces the concept, you complete a guided hands-on lab with the tools, and the session ends with Q&A. Afterward you receive the learning material and an assignment to practise on your own.
    No developer background is required. Promptfoo runs on simple configuration files, and the DeepEval sessions use basic Python that the instructor explains during the labs. A working knowledge of software testing helps, and you can follow along if you are comfortable reading a short script.
    Yes. Each live session is one hour, and recordings are shared afterward, so a missed session or a busy week is easy to catch up on. Assignments are practical and meant to fit around a working schedule.
    You need a laptop with internet access, Node.js (for Promptfoo) and Python (for DeepEval and Ragas). Promptfoo, DeepEval and Ragas are open source and free to use. Some labs call an AI model through an API key, which may involve a small usage cost. The setup guidance shared before Session 1 explains exactly what you need.
    Most free tutorials show one tool in isolation. This training is live, so you can ask the instructor questions, practise on realistic AI apps in a hands-on lab every session, and finish with a capstone that uses all three tools together. It is taught by a QA company that tests AI applications for real clients.
    Yes. Promptfoo, DeepEval and Ragas are practical tools you can start using on your own AI projects. For the capstone you choose the AI application to test, so you can pick one from your own work.
    No course can promise a job, but this one gives you what employers look for: hands-on experience with Promptfoo, DeepEval and Ragas, a capstone project for your CV, and a Testiva certificate. It is designed to help you move toward AI QA Engineer, LLM Tester and AI Red Team roles.