Skip to content

New Project Bootstrap

This guide is meant to help you understand what the AI assistant does when it runs this playbook. The idea is that you can read it, understand what happens at each step, and know what information you need to give the assistant so the work turns out well.

In short

  • What it's for: getting a new project ready for automated QA, without anyone having to configure anything by hand.
  • When to use it: when the project doesn't have a qa-stack.yaml yet (a new project, or an old one you want to start automating).
  • What you need: access to the repository, the app URL (with the environment's version and operating system), and a small reference user story.
  • What you get: the project's "technical profile" (qa-stack.yaml), the test pipeline set up, and a first pilot test run end to end, with its report.
  • Steps: 3 — find out what the project is built with, set up the pipeline, and run a pilot test.
What is a playbook? (if this is your first time using one)

Think of it as a cooking recipe for the AI assistant. You tell it "I want to do this" (for example, "I want to prepare a new project for QA") and the playbook is the recipe the assistant follows step by step, in order, without skipping anything, so the result is always the same no matter who asks for it.

Each step of the recipe uses a specific tool (in technical jargon they're called "skills"), a bit like a kitchen utensil: one is for peeling, another for cutting, another for baking. You don't need to know how each utensil works inside — you only need to know which ingredient goes into each step (for example, the application URL, or the name of a user story) so the final dish turns out well.

A playbook = an ordered guide of steps. A skill = the specific tool that carries out each step. You interact with the playbook — the assistant takes care of using the right tool at each moment.


1. What does this playbook do?

It's the starting point for getting a new project ready for automated QA, without anyone having to configure anything by hand.

Today, when a new project arrives (an app, a website, an API), someone has to:

  1. Find out which tools the project is built with.
  2. Generate or apply changes to the tests so they run automatically every time new code is uploaded.
  3. Check that all of that actually works, by running a first test.

Done by hand, that work can take hours or days and depends on having a technical person available. This playbook does those three things automatically, talking with you about whatever it can't figure out on its own (for example, the URL of the test environment).

When it's done, the project ends up with:

  • A saved "technical profile" that describes how it's built (so nobody has to ask again).
  • A configuration of steps (pipeline) that runs the tests on its own every time someone uploads changes.
  • A first real test run end to end, which confirms that the whole circuit works.

2. When should I use this playbook?

Use it when you're facing any of these situations:

Situation Business example
A brand new project that has never had automated QA The team has just started developing a new online store and there isn't a single automated test running yet.
An old project you want to "bring up to date" A medical appointments app has existed for 2 years, but the tests have always been done by hand and you want to start automating.
You want to start from scratch with the full package (file with the repository's technology stack + pipeline ready to run the CI tests + first test using Jira, Confluence, Azure DevOps, or user stories as a reference) The company decides to standardize how QA is done across all its new projects.

Important

If the project already has a file with the repository's technology stack (the qa-stack.yaml file) and a working pipeline ready to run the tests, you do NOT need to run this playbook — for day-to-day work (testing a new feature, a bug, a UI change) there are other, more specific playbooks, such as Feature New End to End or Frontend UI Change PR.


3. Walkthrough example: "TechStore", an online store with a shopping cart

We'll follow an example business case: TechStore, an online electronics store with a shopping cart (add products, view the cart, pay). It's the same kind of application anyone recognizes from having shopped online at some point.

Let's assume the development team finished the first version of TechStore and asks you: "we need QA to start testing this automatically right away". That's exactly the scenario this playbook is for.

Throughout the following steps, we'll see what you would answer the assistant for each question, using TechStore as an example.

3.1 Before you start: what you need to have on hand

You don't need to know how to program, but you do need to have these three pieces of information on hand:

Information TechStore example
Access to the project (so the assistant can read the code files) The TechStore repository already cloned or accessible
The application URL, which version it's on, and which operating system the test environment runs on https://techstore-staging.com, version 2.4.1, Windows 11
A reference user story, small and simple, to run the first test For example: "As a customer I want to be able to add a product to the shopping cart", saved as a .md file in the project's stories folder (by default user-stories/)

If you don't have a user story on hand, that's not a problem: you can write a minimal one just to validate that the circuit works (it doesn't have to be the most complete test in the system).

Original playbook

The general playbook is in the NEO-releases repository.

3.2 Step by step

Note!

This is an example, you must follow the original playbook.

Step 1 of 3 — Find out what the project is built with

Skill used: lippia-neo-bootstrap-stack

What happens: the assistant goes through the project's files (like someone looking inside a box to see what it contains) and tries to infer which testing tool is used, which language the project is written in, whether it's an application with screens, an API, or both, and which platform runs the automated tests when someone uploads code. Anything it can't guess, it asks you directly.

If the project is a performance one (k6 or JMeter)

With k6 or jmeter the assistant runs a trimmed profile: it doesn't ask the UI/E2E questions (login, version and operating system, stories, bugs, artifact folders, management tool, or .mcp.json), only the URL and the load configuration.

What it will definitely ask you (because it can't guess it on its own):

  • The URL of the application under test → https://techstore-staging.com
  • The app version and the environment's operating system → 2.4.1, Windows 11
  • Whether the application requires login to get in (and what type: username and password, Google account, token, or none) → in TechStore, for example, "yes, username and password"
  • Where the user stories live (in a local file, in Jira, in Azure DevOps?) → "local, in the user-stories/ folder"
  • Where bugs are reported (do they stay only as local files, or are they logged in Jira, Azure DevOps, GitHub, or GitLab?) and, if it's in a tool, in which project → "in Jira, project TS". Even if they're logged in a tool, bugs are always first saved as a file in the local folder (by default bug-reports/)
  • Which folder to save each thing that gets generated in. It asks about all 5, one at a time, even if the answer seems obvious: test plans, automated tests (by default tests/<tool>/, e.g. tests/playwright/), reports, plan-guided runs, and exploratory sessions (by default exploration/) → if you have no preference, just accept what it suggests
  • Where to get the credentials for the tests (by default, a .env.tests file). If the file doesn't exist, it offers to create it empty, with the names of the values that need to be filled in — so the tests never have passwords written in them by hand → "yes, create it"
  • As the last question, which test management tool the test cases are exported to: Xray, QMetry, or none. If you choose one, it asks you for the project and its credentials, which are saved only in the credentials file, never in the technical profile → "none for now"

What it might ask you, depending on the project:

  • If the project uses Lippia and it can't find its configuration file (config.properties), it asks you where it is.
  • If the project is an API (or has screens and an API), it looks for the document that describes the API (openapi.yaml / swagger.json) and, if it can't find it, asks you for its URL.

What you have at the end of this step: a kind of "ID card" for the project, saved in a file (qa-stack.yaml), which the rest of the playbooks will read automatically from now on — so you never have to answer these questions again. It also leaves:

File What it's for
qa-stack.yaml The project's technical profile
.env.tests The test credentials (not uploaded to the repository)
.mcp.json The connection to the browser the assistant uses to go through the app and, if you chose Jira, Azure DevOps, GitHub, or GitLab, the connection to that tool (not uploaded to the repository)

Checkpoint — Review the technical profile

Confirm that qa-stack.yaml exists and that it has the stack details filled in (testing tool, language, app type, pipeline platform, and login type) as well as the environment details (URL, version, and operating system), that the test management tool was defined (or "none"), and that .mcp.json exists. If the project is an API, the API document must also be listed.

Important

If the testing tool it detects isn't fully supported yet (today, Pytest), the assistant will warn you that some steps of the pilot test (generating and running the automated tests) won't be able to run until support for that tool is finished. It's not your mistake — it's a known limitation to keep in mind before moving on. Playwright, Cypress, and Lippia are already supported.

Restart the environment after this step

When .mcp.json is created or updated, you need to restart the environment (for example, reload VS Code) so the assistant loads those connections. This is necessary for the exploration in Step 3 and, if you chose a cloud tool, for reading stories or logging bugs there.


Step 2 of 3 — Set up the pipeline that runs the tests on its own

Skill used: lippia-neo-ci-bootstrap

What happens: with the description of the technology stack from Step 1, the assistant writes the configuration for an automatic pipeline that, from now on, will run TechStore's tests every time someone uploads a code change — without anyone having to remember to do it by hand.

It's like putting a security guard at the door: every time something new comes in (a code change), the guard checks it automatically before letting it through.

What you have at the end of this step: the pipeline configuration files (the exact name depends on where TechStore's code lives — GitHub, GitLab, or Bitbucket) that the development team will upload along with the rest of the code. The pipeline's jobs are grouped in their own folder (neo-ci-jobs/), and the main pipeline file just "calls" them:

Where the code lives Main pipeline file Folder with the jobs
GitHub .github/workflows/qa.yml (if it already existed, it isn't touched and .github/workflows/neo-ci-jobs.yml is created) .github/workflows/neo-ci-jobs/
GitLab .gitlab-ci.yml (if it already existed, only one line is added that includes the new jobs) .gitlab/neo-ci-jobs/
Bitbucket bitbucket-pipelines.yml (if it already existed, the assistant suggests how to combine them and asks for your confirmation) .bitbucket/neo-ci-jobs/

The jobs always follow the same chain: prepare the environment → run the corresponding QA plan (for example, the one for new features) → publish the report.

What this step does NOT do

This step only writes the pipeline configuration. It doesn't make it run real tests yet — that happens in Step 3. Also, once uploaded, the pipeline only runs on Pull Requests / Merge Requests that have the label configured for each plan (for example, feature), and the technical team has to load the keys and variables required by the pipeline documentation.


Step 3 of 3 — Pilot test: confirm that the whole circuit works

Skill used: lippia-neo-meta-run-user-story-e2e

What happens: the assistant takes the user story you prepared ("add a product to the cart") and runs the complete QA circuit, end to end:

  1. It reads the user story.
  2. It builds a test plan from it, with the corresponding acceptance criteria.
  3. It runs the plan on the real application and reports failures.
  4. It generates the corresponding automated tests.
  5. It runs them — and if any fails, it tries to repair it on its own (up to 3 attempts).
  6. It writes a final report with the result of the automation.

With TechStore, this means the assistant actually goes into the online store, adds a product to the cart, and confirms that the cart reflects it correctly — and leaves that behavior turned into a reusable automated test.

This is not a full regression of the system

It's just a pilot test to confirm that the "wiring" is properly connected: that you can read, plan, test, generate, run, and report without anything breaking along the way.

How to know if it went well:

Result What it means
✅ The test passed or failed, but it ran end to end The circuit works correctly (a test failing isn't a playbook error — it may be a real bug in the app)
✅ A report with the result was generated You can open it and read the details (in the reports folder, by default test-results/reports/)
✅ At least one automated test file was generated You now have something reusable for the future
⚠️ Some tests couldn't be repaired They're documented in the summary; they don't prevent you from considering the setup finished
❌ The process got "blocked" without being able to run anything In that case you do need to let the technical team know — the testing tool detected in Step 1 may not be supported yet

3.3 What do I have after completing the three steps?

What you get What it's for
A saved file with the project's technology stack (qa-stack.yaml) You never have to answer the configuration questions again
The credentials file (.env.tests) and the connections configuration (.mcp.json) The tests and the assistant have somewhere to get access details from, without writing them by hand
A configured automated test pipeline Every code change in TechStore triggers tests on its own
A test plan for the pilot story Documentation of what was tested and how (in the plans folder, for example test-plans/)
The report of the plan-guided run Evidence of what was gone through in the app (by default in execution/)
An automated test already generated and running A starting point to keep adding tests (by default in tests/<tool>/)
A results report Evidence that the setup was validated

From here on, TechStore has "entered" the automated QA ecosystem: to keep testing new features, bugs, or changes, you use other, more specific playbooks (for example, one for new features, another for bugs, another for UI changes) — this playbook has already fulfilled its role of getting the foundation ready.


4. Common problems (and what to do)

What you see Why it might happen What to do
The assistant can't build the file with the project's technology stack The project doesn't have the typical files that make it possible to guess the configuration Just answer the questions it asks you directly
The pipeline is set up but the jobs don't run The Pull Request / Merge Request doesn't have the plan's label (for example, feature), or the pipeline's keys and variables are missing Add the label and let the technical team know so they can load the keys required by the pipeline documentation
The pilot test gets "blocked" without being able to run The detected testing tool isn't fully supported yet It's an issue for the technical team, not something you can solve on your own
The assistant can't open the application to go through it The environment wasn't restarted after Step 1, and it hasn't loaded the browser connection yet Restart the environment (for example, reload VS Code) and run Step 3 again
The tests fail and can't repair themselves The application may not be up, or the URL/credentials may be wrong Check that the TechStore URL is accessible and that the credentials are correct
The tests fail at login The values in the credentials file (.env.tests, created in Step 1) are empty or incomplete Fill in the real values in .env.tests (they're never written by hand inside the tests) and run the tests again

5. Visual summary of the flow

flowchart TD
    A([🏗️ New project without automated QA<br/>e.g.: TechStore just built]) --> B
    B["<b>Step 1</b> — Find out what the<br/>project is built with"] --> C
    C["<b>Step 2</b> — Set up the pipeline<br/>that runs the tests on its own"] --> D
    D["<b>Step 3</b> — End-to-end pilot test<br/>e.g.: add product to cart"] --> E{Did it run<br/>end to end?}
    E -->|Yes, passed or failed| F([✅ Project ready<br/>to carry on])
    E -->|It got blocked| G([⚠️ Let the technical team know:<br/>unsupported tool])