Skip to content

Demo Walkthrough

This guide is meant to help you understand what the AI assistant does when it runs this playbook. The idea is that you can read it, understand what happens at each step, and know what information you need to give the assistant so the demo turns out well — without needing to know how to code.

In short

  • What it's for: showing other people, live and step by step, how the assistant tests a feature end to end.
  • When to use it: in a demo for a customer or stakeholders, when onboarding someone new, when recording a video or when asking for final approval of a story.
  • What you need: a user story with clear criteria, the URL of the working app, test credentials, the project's qa-stack.yaml and screen sharing ready.
  • What you get: the test plan, the exploration findings with screenshots, the automated tests, a dated final report and an audience that understood the process.
  • Steps: 6 — read the story, build the plan, explore live, generate the tests, run and repair them, and the final report (with a pause for questions after each one).
What is a playbook? (if this is your first time using one)

Think of it as a cooking recipe for the AI assistant. You tell it "I want to do this" (for example, "I want to show a customer how we test their feature") and the playbook is the recipe the assistant follows step by step, in order, without skipping anything, so the result is always the same no matter who asks for it.

Each step of the recipe uses a specific tool (in technical jargon they're called "skills"), something like a kitchen utensil: one is for reading, another for exploring the application, another for writing the final report. You don't need to know how each utensil works inside — you only need to know which ingredient goes into each step (for example, which application, which story, which URL) so the final result is useful.

A playbook = an ordered guide of steps. A skill = the specific tool that carries out each step. You interact with the playbook — the assistant takes care of using the right tool at each moment.


1. What does this playbook do?

It's the playbook you use when you need to show other people, live and step by step, how the assistant tests a feature end to end — without them getting lost along the way.

The assistant can do all the QA work for a user story (read it, build a test plan, test the application, write the automated tests, run them and put together the final report) in one go, without pauses. That's perfect for day-to-day work, but it's not ideal for a demo: if everything happens at once, the audience doesn't get to understand what they're seeing, can't ask questions in time, or see each artifact as soon as it's generated.

This playbook solves exactly that problem: it does the same thing as the complete QA flow, but:

  • It narrates in simple language what it's doing at each step, before doing it.
  • It stops after each step, so the audience can look at the result on screen and ask whatever they want.
  • It makes each document or file that was generated clearly visible, at the moment it was generated (not all together at the end).

What this playbook does NOT do differently

It doesn't test anything extra or in a different way from the normal QA flow. The application is tested exactly the same way — the only difference is the pace and the narrative, designed so an audience can follow the process live.


2. When should I use this playbook?

Use it when you're facing any of these situations:

Situation Business example
You have to show the process to a customer or stakeholders before they approve something The TechStore customer wants to see, with their own eyes, how the new cart feature is tested before giving the go-ahead to move to production
You're onboarding someone new to the team and they need to understand the complete flow in one go A new QA joins the TechStore team and needs to see from start to finish how a story is tested, without having to read ten different documents
You're going to record a video to show internally how the team works, or to sell the process to another customer TechStore's sales area asks for a short recording to show a potential customer "this is how we test what we build"
You need the customer's sign-off ("final approval") on a candidate story, showing live that it works Before closing the sprint, the TechStore customer wants to see the "Checkout" flow working before signing off on acceptance

Important

  • This playbook isn't for day-to-day QA work — for that there's already the normal flow, which does the same thing without pauses or narrative, much faster.
  • Use it specifically when there's an audience watching (a customer, a stakeholder, a new person, a camera recording) and you need that audience to understand each step, not just see the final result.

3. Guide example: "TechStore", an online store with a shopping cart

We'll follow the same example business case as in other guides: TechStore, an online electronics store with a shopping cart (add products, view the cart, pay). It's the kind of application anyone recognizes from having bought something online.

Let's assume this scenario: the TechStore team just finished developing the "Checkout" feature (the customer adds products to the cart, clicks "Checkout", and the system confirms the order). Before considering that story closed, TechStore's Product Owner wants the team to show them, in a 20-minute meeting, how that feature was tested end to end — not just be told "it's already tested", but see it happen live.

That's exactly the scenario for this playbook: there's an audience (the Product Owner), there's a specific story ("Checkout"), and the goal is for that audience to understand and trust the process, step by step.

Throughout the following steps we'll see what you'd tell the assistant at each moment, using this TechStore demo as the example.

3.1 Before you start: what you need to have on hand

You don't need to know how to code, but you do need to have this information on hand:

Information TechStore example
The user story you're going to show in the demo, with its acceptance criteria well defined The document that describes "As a customer I want to complete my purchase and receive a confirmation"
The application URL, working and accessible before the meeting starts https://techstore-staging.com
Login credentials, if the app requires login Username and password for a test account
The project's qa-stack.yaml file already set up If TechStore doesn't have it yet, you first need to run the project setup playbook
Screen sharing ready, if the demo is in person or by video call Have the browser and the chat with the assistant visible to the whole audience
(Recommended) Having read the story once beforehand, so you know what to expect during the demo That way, if something doesn't go as expected, the presenter isn't caught off guard in front of the customer

Tip

If this is the first time you're running this demo on a story, try it in private first. The first time it runs, something unexpected may come up (for example, the app is slow to respond) that you'd rather discover on your own, not live in front of the customer.

If you want a demo \"from scratch\"

If you already rehearsed the demo with the same story, delete (or ask someone on the technical team to delete) what was left from that previous run, so the audience sees each file appear for the first time:

  • The test plan: test-plans/TS-US-08-test-plan.md
  • The automated tests: the tests/playwright/TS-US-08/ folder (or whatever tests folder the project has configured)
  • The reports: test-results/reports/TS-US-08-test-report-*.md

Original playbook

The general playbook can be found in the NEO-releases repository.

3.2 How to start the conversation

Before starting the demo, tell the assistant which story you're going to show and in what context, so it keeps it in mind throughout the whole process. For example:

💬 Prompt for the assistant
We're going to do a live demo for TechStore's Product Owner.
The story to show is "Checkout" (file user-stories/TS-US-08.md).
The app is at https://techstore-staging.com.
I want you to narrate each step and stop after each one so the
Product Owner can look at the result and ask questions before continuing.

The assistant will use this description throughout all the following steps, so you don't have to repeat it every time.


3.3 Step by step

Each of these six steps ends with a pause — the assistant shows what it generated and waits before moving on to the next one. That pause is the ideal moment for the audience to ask questions.

Step 1 of 6 — Read the user story

Skill used: lippia-neo-read-user-story

What happens: the assistant reads the story document ("Checkout") and extracts the essentials from it: what the feature has to do, at which URL it is, and which test user to log in with.

What to show the audience: the story document open, and next to it, what the assistant understood from that document.

Checkpoint — What the assistant understood

"Is this what you expected 'Checkout' to do? Is anything missing?"


Step 2 of 6 — Build the test plan

Skill used: lippia-neo-create-test-plan

What happens: with what it got from Step 1, the assistant builds a concrete list of everything that needs to be tested: the normal path (the customer buys and the order is confirmed), and the unusual or negative cases (for example, what happens if the cart is empty, or if the connection drops in the middle of payment).

What to show the audience: the document with the list of cases to test, just created in the plans folder (for example, test-plans/TS-US-08-test-plan.md), and how many cases came out for each requirement of the original story.

Checkpoint — Cases to test

"Is there any case you hadn't thought of? Would you like us to add a specific one?"


Step 3 of 6 — Live exploration, inside the real application

Skill used: lippia-neo-run-manual-testing

What happens: this is the highest-impact moment of the whole demo. The assistant goes into TechStore just like any real customer would — it adds products, goes to pay, clicks "Checkout" — and the audience sees all of this happen on screen, in real time, with a browser open in front of everyone. While it navigates, it notes the names of the buttons and fields it uses (later the automated tests will use them to find them) and takes screenshots as evidence.

What to show the audience: the browser navigating the application on its own, and each finding appearing as it's discovered (for example, if something doesn't behave as expected). Everything is saved in an exploration report in the execution/TS-US-08/ folder, along with the screenshots.

The exploration only goes through screens

This step only navigates the application's interface in the browser. If the demo is about an API (no screens), there's nothing to show by navigating: the API tests are built directly from the plan in Step 4 (see demo variants).

Highest-impact moments for the audience

  • When the assistant finds something unexpected (a bug, an inconsistency).
  • When it takes a screenshot of a real error, at the moment it happens.
  • When it identifies a button or a field on the screen that it will later use in the automated tests.

Checkpoint — Exploration findings

"Did we find anything you didn't expect? Does this behavior look right to you?"


Step 4 of 6 — Generate the automated tests

Skill used: lippia-neo-generate-test-suite

What happens: with the plan from Step 2 and everything it discovered while navigating in Step 3, the assistant writes the automated tests — the set of instructions that, from now on, can run on their own, whenever needed, without anyone having to repeat the clicks by hand.

What to show the audience: the test files appearing in a folder with the story ID, inside the project's tests folder (for example, tests/playwright/TS-US-08/), and opening one to show that it uses exactly the same steps, buttons and fields seen during navigation in Step 3 (nothing extra is made up).

Checkpoint — Generated tests

"Does this look like the steps we just saw? Do you see anything that raises questions?"


Step 5 of 6 — Run the tests and repair them if needed

Skill used: lippia-neo-run-and-heal

What happens: the assistant runs the automated tests it just generated. If one fails, it doesn't just give up and let you know — it analyzes why it failed, adjusts it, and runs it again. This cycle repeats up to 3 times at most, until everything passes or until there are no more improvements between one attempt and the next — in that case it's clear that that specific case needs to be reviewed by a person. You don't need to give it any extra configuration: it uses these default values.

What to show the audience: the console running the tests, and if there's any automatic repair, the whole cycle: fails → it's analyzed → it's corrected → it's run again. At the end, the count: tests that passed, that were repaired and that couldn't be repaired.

Highest-impact moment

When a test fails and, seconds later, the assistant repairs it on its own and it passes again.

Checkpoint — Repair cycle

"What did you think of the automatic repair cycle? The tests that couldn't be repaired — are they a real problem in the application, or a limitation of the test itself?"


Step 6 of 6 — Final report

Skill used: lippia-neo-write-test-report

What happens: the assistant gathers everything that happened in the demo — what it found while navigating in Step 3, and the results of running the tests in Step 5 — and puts it into a single, tidy document, with date and time, which is saved as a historical record (it doesn't delete previous reports) in the project's reports folder (by default test-results/reports/, for example TS-US-08-test-report-2026-07-17-1015.md).

What to show the audience: the final report open, clearly showing the overall status (whether everything ended well or something is left to review) and the section that separates what was found while exploring from the results of the automated tests.

Closing the demo

"In a single session we went from the user story to a set of tests that's documented, run and reported. All of this can be run again whenever you want, and it's traced back to the original requirement."


3.4 What do I end up with after the six steps?

What you get What it's for
The test plan for the story shown Reference document of everything that was covered
The findings from the live exploration, with screenshots Evidence that the feature was really tested, not just "in words"
The generated automated tests They remain available to run again in the future, without repeating the demo
The final report with date and time A single document to attach to the customer's final approval or to the new person's onboarding
An audience that understood and trusts the process The main goal of this playbook — it's not just "testing", it's "showing how testing is done"

4. Demo variants

Depending on the type of application configured in qa-stack.yaml and the time you have, the demo can take different forms:

Variant What changes
Screen demo (web application) The emphasis is on the visual exploration with the browser in Step 3
API demo (no screens) The emphasis is on the API tests generated in Step 4; Step 3 only explores screens, so here it adds little or nothing
Hybrid demo (screens + API) Both screen and API tests are generated (the Step 3 exploration covers only the screens); it's longer, but has greater impact
Express demo If there are already findings from a previous exploration, Step 3 is skipped and you start from Step 4

5. What next? — What to do once the demo is over

Situation What to do
✅ The customer gave the go-ahead ("final approval") on the story shown The story is closed as accepted; the generated tests become part of the project's coverage
❌ The live exploration found a real bug It's documented as such and you continue with the bug reproduction playbook, to confirm it formally and pass it on to development
The new person on the team has now seen the complete flow They can start running the normal QA flow (without narrative or pauses) on their own stories
The demo was recorded for commercial or internal use The video remains available to show the process to other customers or areas, without having to repeat the live session every time

6. Common problems (and what to do)

What you see Why it can happen What to do
The browser doesn't open during Step 3 The tool that controls the browser isn't connected Check it before the meeting starts, never live in front of the customer
The story doesn't have clear criteria The story is poorly written or incomplete Rewrite the story before the demo — the assistant can only extract what the document actually says
The tests fail with a connection error The test environment (staging) isn't available Bring up and check the environment before starting
The tests fail at login during the repair The project's test credentials (.env.tests file) are empty Ask someone on the technical team to fill in the real values before repeating Step 5 — they're never written by hand inside the tests
The demo feels too long for the audience The chosen story has too many acceptance criteria Choose a narrower story (2 or 3 criteria) for a short demo, or jump straight to Step 4 if there are already findings from a previous exploration (express demo)

7. Visual summary of the flow

flowchart TD
    A([🎬 User story to show<br/>e.g.: Checkout<br/>in TechStore]) --> B
    B["<b>Step 1</b> — Read the story<br/>⏸ pause for questions"] --> C
    C["<b>Step 2</b> — Build the test plan<br/>⏸ pause for questions"] --> X{Express demo with<br/>previous findings?}
    X -->|No| D["<b>Step 3</b> — Live exploration<br/>⭐ highest-impact moment<br/>⏸ pause for questions"]
    X -->|Yes, skip Step 3| E
    D --> E["<b>Step 4</b> — Generate the<br/>automated tests<br/>⏸ pause for questions"]
    E --> F["<b>Step 5</b> — Run and repair<br/>the tests<br/>⏸ pause for questions"]
    F --> G["<b>Step 6</b> — Final report<br/>🎤 closing the demo"]
    G --> H([✅ An audience that saw, understood<br/>and trusts how testing is done])