Configure playwright-bdd
Adapter status: ✅ Complete. playwright-bdd is a valid value of stack.test_framework: lippia-neo-bootstrap-stack detects it, lippia-neo-generate-test-suite writes the scenarios in Gherkin (.feature) with their steps and page objects, and lippia-neo-run-and-heal runs and repairs them.
playwright-bdd turns the .feature files into Playwright tests (npx bddgen) and runs them with the regular Playwright runner (npx playwright test). That's why it keeps everything Playwright offers: fixtures, storageState, traces, projects, and the JSON report.
Would you rather have cucumber-js run the
.featurefiles instead of the Playwright runner? See Configure playwright-cucumberjs. Both guides share the suite organization; what changes is the configuration, the "glue" between steps and Playwright, and how it runs.
| Piece | Does it support playwright-bdd? |
|---|---|
Automatic detection in lippia-neo-bootstrap-stack |
✅ When package.json includes playwright-bdd |
lippia-neo-generate-test-suite (adapters/playwright-bdd/) |
✅ README and 8 templates |
lippia-neo-run-and-heal (adapters/playwright-bdd/) |
✅ run.sh and parse-results.sh, with scoping by .feature, by folder, and by tag |
lippia-neo-find-existing-tests |
✅ Searches the .feature files by tags, titles, and the # bug-id: marker |
lippia-neo-find-ui-tests-by-code-diff and lippia-neo-repair-selectors-by-diff |
✅ They search and repair in pages/ and steps/; they never touch the text of a .feature |
lippia-neo-refactor-gherkin-features |
✅ Restructures existing .feature files so they are data-driven |
CI jobs from lippia-neo-bootstrap-ci |
✅ Uses the same installation as Playwright (npm ci + browsers) |
| Scanning the conventions of an existing suite | ❌ Not supported: new cases follow the adapter's default convention (section 5) |
Value in qa-stack.yaml |
test_framework: playwright-bdd |
| Language | javascript or typescript (if there is a tsconfig.json) |
| What it uses | @playwright/test + playwright-bdd (scenarios in Gherkin) |
| App types | web, api, and hybrid |
| Authentication | basic_auth, api_token, google_sso (with limitations, see section 4), or none |
| Default tests folder | tests/playwright-bdd/ |
| Configuration file | playwright.config.<ext> at the root of the repository (not inside tests_dir) |
1. What you need installed
| Tool | Version | How to check it |
|---|---|---|
| Node.js | 18 or higher | node -v |
@playwright/test |
The one in package.json |
npx playwright --version |
playwright-bdd |
8 or higher (tested with 8.4.1 and 9.2) | Listed in package.json |
| Playwright browsers | The ones for your Playwright version | npx playwright install |
dotenv |
— | Listed in package.json. Not needed if the project uses process.loadEnvFile (Node 20.12+) |
| Bash (Git Bash or WSL on Windows) | — | bash --version |
npm install -D @playwright/test playwright-bdd dotenv
npx playwright install
Add the generated tests folder to .gitignore: npx bddgen creates it on every run and it is not edited by hand.
tests/playwright-bdd/.features-gen/
For the skills that browse the application live (
lippia-neo-run-manual-testing,lippia-neo-reproduce-bug),NEOuses the Playwright MCP, regardless of the testing framework (see Playwright MCP setup).
2. Sample qa-stack.yaml
stack:
test_framework: playwright-bdd
language: typescript # or javascript
app_type: web # web | api | hybrid
auth: basic_auth # google_sso | basic_auth | api_token | none
pipeline_provider: gitlab
environment:
base_url: https://techstore-staging.com
app_version: 2.4.1
os: Windows 11
paths:
user_stories_storage:
type: local
cloud_provider_mcp: null
local_path: user-stories/
test_plans_dir: test-plans/
tests_dir: tests/playwright-bdd/
reports_dir: test-results/reports/
scripted_dir: execution/
exploratory_dir: exploration/
env_file: .env.tests
commands:
run_tests: null # uses the adapter's run.sh → npx bddgen + npx playwright test
install_deps: null # uses npm ci
There is no separate field to choose the Gherkin runner: the value of
test_framework(playwright-bddorplaywright-cucumberjs) already defines it.
3. Configure playwright.config
The config goes at the root of the repository and is built with defineBddConfig. lippia-neo-generate-test-suite shows you the block that matches your app type, but it's best to have it ready beforehand. Three values are mandatory because lippia-neo-run-and-heal depends on them:
| What | Why | Value |
|---|---|---|
featuresRoot |
So the spec generated from each .feature can be mapped back to the .feature |
The same value as paths.tests_dir |
outputDir |
Where bddgen writes the generated tests |
<tests_dir>/.features-gen |
| JSON reporter | parse-results.sh reads that file to know what failed |
['json', { outputFile: 'test-results/results.json' }] |
Loading .env.tests |
So the steps read the credentials without having them written down. It is loaded only here: each worker reads the config again | require('dotenv').config({ path: '.env.tests' }) |
// playwright.config.js (root of the repository)
const { defineConfig, devices } = require('@playwright/test');
const { defineBddConfig } = require('playwright-bdd');
require('dotenv').config({ path: '.env.tests' }); // = paths.env_file
const TESTS_DIR = 'tests/playwright-bdd'; // = paths.tests_dir
const TEST_DIR = defineBddConfig({
featuresRoot: TESTS_DIR,
outputDir: `${TESTS_DIR}/.features-gen`,
features: `${TESTS_DIR}/features/**/*.feature`,
steps: [`${TESTS_DIR}/fixtures.js`, `${TESTS_DIR}/steps/**/*.js`], // .ts if the project is TypeScript
});
module.exports = defineConfig({
retries: process.env.CI ? 2 : 1,
timeout: 60000,
reporter: [
['json', { outputFile: 'test-results/results.json' }], // required by lippia-neo-run-and-heal
['html', { outputFolder: 'playwright-report', open: 'never' }],
['list'],
],
use: {
baseURL: process.env.APP_BASE_URL || 'https://techstore-staging.com',
trace: 'on-first-retry',
screenshot: 'only-on-failure',
// testIdAttribute: 'data-test', // only if the app doesn't use data-testid
},
projects: [
{ name: 'chromium', testDir: TEST_DIR, grepInvert: /@api/, use: { ...devices['Desktop Chrome'] } },
{ name: 'api', testDir: TEST_DIR, grep: /@api/ },
],
});
By app type
UI and API tests are separated by the @api tag on the Feature, not by two different configs:
app_type |
projects |
|---|---|
web |
chromium (with grepInvert: /@api/). With google_sso, also setup (see section 4) |
api |
Only api (with grep: /@api/) |
hybrid |
chromium and api (and setup with google_sso) |
If the suite uses more than one data profile, add one project per profile with
use: { dataProfile: '<profile>' }. With a single profile you don't need to:TEST_DATA_PROFILEordefaultis used.
4. Authentication
basic_auth and api_token
Fill in the values in .env.tests, the same as with Playwright. The steps read them with process.env.NAME and never write them literally:
basic_auth: theBackgroundof each.featurelogs in withprocess.env.APP_USER/process.env.APP_PASSWORDthrough the page object (steps/common.steps.<ext>).api_token: theapiManagerandapiViewerfixtures create the API context withAPI_TOKEN_MANAGERandAPI_TOKEN_VIEWER.
google_sso
Google login is not automated. The session is loaded from a browser state file:
- The
chromiumproject usesstorageState: 'playwright/.auth/user.json'and depends on thesetupproject, which is the one that generates that file. - The step
el usuario está autenticadoonly opens the app: the session is already loaded. run.shchecks thatplaywright/.auth/user.jsonexists before running and, if it isn't there, stops withAuth state no encontrado.
Current session limitation
This adapter does not use paths.auth_state_file or the session.* block of qa-stack.yaml yet: the session path is fixed (playwright/.auth/user.json, or whatever you set in QA_AUTH_STATE_PATH). Also, the config references an auth.setup.js that this adapter does not include: take it from the Playwright adapter or generate the session file with your project's script. With basic_auth, the Background logs in on every scenario.
Never commit the session file to git. Add
playwright/.auth/to.gitignore.
5. What NEO generates
playwright.config.<ext> ← root of the repository
tests/playwright-bdd/ ← paths.tests_dir
├── features/
│ ├── carrito.feature ← one .feature per business flow (UI)
│ ├── checkout.feature
│ └── ordenes.api.feature ← one .feature per business flow (API), with @api
├── steps/
│ ├── common.steps.<ext> ← Background (login), only once per project
│ ├── carrito.steps.<ext> ← one per page object (CarritoPage)
│ └── ordenes.api.steps.<ext> ← one per API resource (/api/ordenes)
├── pages/
│ └── CarritoPage.<ext> ← page objects
├── data/
│ └── default/<flujo>.json · <dominio>.json ← only if some case uses file data
├── fixtures.<ext> ← fixtures + Given/When/Then from createBdd
├── test-data.<ext> ← resolves the test data of the active profile
└── .features-gen/ ← generated by npx bddgen: not edited or versioned
How scenarios are organized
- One
.featureper business flow, not per story or per case:features/<flujo>.feature(for examplemodificar-carrito.feature). Cases from different stories that test the same flow go in the same file; each new case is appended as aScenarioat the end. There are no per-story folders. - Traceability only through tags: each
Scenariocarries@<US-ID>and@<case ID>exactly as they appear in the plan, and its title starts with that ID. - API
.featurefiles end in.api.featureand carry the@apitag on theFeature. Without that tag, an API case opens a browser. - Scenarios that only differ in data are written as a
Scenario Outlinewith oneExamplesblock per case (see Data-driven testing).
Feature: Modificar el carrito
Background:
Given el usuario está autenticado
@US-002 @TC-01
Scenario: TC-01 — Agregar un producto al carrito
When el usuario agrega "Mouse Inalámbrico" al carrito
Then el carrito muestra 1 producto
@US-008 @TC-03
Scenario: TC-03 — Quitar el último producto deja el carrito vacío
...
Rules the generated tests follow
- Steps in business language: no selectors, URLs, or credentials in the
.feature. - Step text is global: before creating one, it checks whether it already exists. A duplicated step makes
npx bddgenfail for the whole suite. - Step files per page object or per resource, never per story:
steps/<page>.steps.<ext>paired withpages/<Nombre>Page.<ext>, andsteps/<recurso>.api.steps.<ext>for API. - Imports: steps import
Given/When/Thenfrom../fixtures, never from@cucumber/cucumber. The first argument is the fixtures:async ({ carritoPage }, producto) => { … }. - Selectors only in page objects, in this order:
getByRole,getByLabel,getByTestId,getByTextand, as a last resort, CSS. If a selector is unknown,page.locator('TODO')is left with a comment. - State between steps of the same scenario goes in the
ctxfixture, never in global variables. - Regression and TDD tests stay red: they are never marked with
@skip,@fixme, or@fail(in playwright-bdd,@failinverts the result and would hide the bug).
The templates are in JavaScript (CommonJS). If the project is TypeScript,
NEOadapts them toimport/exportand saves the files with.ts.
6. How lippia-neo-run-and-heal runs it
The adapter's run.sh:
- Checks that
node_modules,@playwright/test, andplaywright-bddexist, plus the session file ifauthisgoogle_sso. If anything is missing, it exits with code2. - Runs
npx bddgen, which validates the steps and generates the tests. If a step is missing or duplicated, it exits with code2: there are no results, and the skill treats it as incomplete code (it fixessteps/), not as a red test. - Runs
npx playwright test --reporter=jsonwith the applicable filters. parse-results.shreadstest-results/results.jsonand maps each result to the source.feature(it strips.features-gen/and.spec.jsfrom the path).
| Variable | Default | What it's for |
|---|---|---|
QA_TESTS_DIR |
tests/playwright-bdd |
The root (full suite) or a list of .feature files / folders under features/, separated by spaces |
QA_TESTS_ROOT |
QA_TESTS_DIR if it is a single path |
paths.tests_dir. Mandatory if QA_TESTS_DIR has more than one path |
QA_EXCLUDE_DIRS |
empty | Subfolders of features/ to exclude. Loose .feature files in features/ always run |
QA_CUCUMBER_TAGS |
empty | Filter by tag: @US-002, "@US-002 and @TC-02", "@smoke and not @wip" |
QA_AUTH_STATE_PATH |
playwright/.auth/user.json |
Session file that is validated if auth is google_sso |
PLAYWRIGHT_FLAGS |
empty | Extra flags, for example --project=chromium |
PLAYWRIGHT_CONFIG |
empty | Use another config file (passed to bddgen and to playwright test) |
PLAYWRIGHT_JSON_OUTPUT |
test-results/results.json |
Where the JSON report is saved |
# Full suite
QA_TESTS_DIR="tests/playwright-bdd" ./run.sh
# A specific flow and a folder
QA_TESTS_ROOT="tests/playwright-bdd" \
QA_TESTS_DIR="tests/playwright-bdd/features/carrito.feature tests/playwright-bdd/features/legacy" \
./run.sh
# Only one story (by tag)
QA_TESTS_DIR="tests/playwright-bdd" QA_CUCUMBER_TAGS="@US-002" ./run.sh
When a playbook scopes the run, lippia-neo-run-and-heal translates its options like this (it always passes QA_TESTS_ROOT too):
lippia-neo-run-and-heal option |
Becomes |
|---|---|
config.target_tests |
QA_TESTS_DIR with the given .feature files |
config.include_dirs |
QA_TESTS_DIR with <tests_dir>/features/<carpeta> for each folder |
config.exclude_dirs |
QA_EXCLUDE_DIRS with the names of the features/ subfolders |
Where it applies repairs
| Failure type | What it touches |
|---|---|
selector_broken, timeout |
The page object (pages/) |
assertion_mismatch |
The step definition (steps/). The .feature only if the expected business behavior changed |
Missing or duplicated step (bddgen) |
steps/ |
environment |
Nothing in the test: only how .env.tests is loaded or the variable name |
NEO never changes the text of a step inside the .feature to match a step definition, and never edits .features-gen/. A case blocked by data (@fixme with a # bloqueado: comment) is not repaired.
In a
Scenario Outline, eachExamplesrow is a test. If the row has no title of its own, it shows up in the results as<Outline title> — Example #N.
7. Quick checklist
node -v # 18 or higher
npx playwright --version # Playwright installed
grep -n "playwright-bdd" package.json # playwright-bdd installed
grep -n "defineBddConfig\|results.json" playwright.config.*
npx bddgen # validates the steps without running anything
npx playwright test --list # lists one test per scenario and per Examples row
If everything responds fine, ask the assistant: "Run the suite with lippia-neo-run-and-heal".
8. Common problems (and what to do)
| What you see | Why it may happen | What to do |
|---|---|---|
playwright-bdd no encontrado |
The package isn't installed | npm install -D playwright-bdd |
'npx bddgen' falló … steps faltantes o duplicados |
A step in the .feature has no definition, or is defined twice |
Complete or deduplicate the step in steps/. That's what lippia-neo-run-and-heal does |
QA_TESTS_DIR trae varios paths; definí QA_TESTS_ROOT |
More than one .feature or folder was passed |
Set QA_TESTS_ROOT to paths.tests_dir |
Auth state no encontrado en 'playwright/.auth/user.json' |
With google_sso, the session file doesn't exist |
Generate it (see section 4) or set the real path in QA_AUTH_STATE_PATH |
Results point to .features-gen/…spec.js instead of the .feature |
featuresRoot or outputDir don't match paths.tests_dir |
Align the config with section 3 |
| An API test opens a browser | The .api.feature doesn't have the @api tag on the Feature |
Add @api above Feature: |
| A blocked case doesn't show up in the report | All scenarios in that .feature are @fixme, and bddgen doesn't generate the spec |
Expected: it shows up once the file has a scenario that runs |
lippia-neo-bootstrap-stack asks you which runner to use |
package.json includes both playwright-bdd and @cucumber/cucumber |
Choose which one the suite is generated with |
9. Current limitations
| What | What happens with playwright-bdd | Alternative |
|---|---|---|
| Running a single case | Passing a .feature runs all its scenarios (the whole flow) |
Filter by tag: QA_CUCUMBER_TAGS="@US-002 and @TC-02" |
| Running from a page object or step | If target_tests includes a page object, a step, or fixtures.<ext> (typical after lippia-neo-repair-selectors-by-diff), run.sh runs the full suite and reports it |
— |
| Authenticated session | Doesn't use paths.auth_state_file or session.*; the path is fixed |
QA_AUTH_STATE_PATH |
| Scanning the existing suite's conventions | Not supported: it places, names, and tags cases with the default convention | — |
Adding a case to an existing Scenario Outline |
When generating, a row is never added to an Outline that already exists: the case goes separately, with a warning | Merge them by hand or with lippia-neo-refactor-gherkin-features |
CI history with Scenario Outline |
playwright-bdd groups the rows of an Outline inside a describe with the Outline's title: the full test path changes |
If your CI keeps history by full path, it will see them as new tests once |