Skip to content

Configure playwright-bdd

Adapter status: ✅ Complete. playwright-bdd is a valid value of stack.test_framework: lippia-neo-bootstrap-stack detects it, lippia-neo-generate-test-suite writes the scenarios in Gherkin (.feature) with their steps and page objects, and lippia-neo-run-and-heal runs and repairs them.

playwright-bdd turns the .feature files into Playwright tests (npx bddgen) and runs them with the regular Playwright runner (npx playwright test). That's why it keeps everything Playwright offers: fixtures, storageState, traces, projects, and the JSON report.

Would you rather have cucumber-js run the .feature files instead of the Playwright runner? See Configure playwright-cucumberjs. Both guides share the suite organization; what changes is the configuration, the "glue" between steps and Playwright, and how it runs.

Piece Does it support playwright-bdd?
Automatic detection in lippia-neo-bootstrap-stack ✅ When package.json includes playwright-bdd
lippia-neo-generate-test-suite (adapters/playwright-bdd/) ✅ README and 8 templates
lippia-neo-run-and-heal (adapters/playwright-bdd/) ✅ run.sh and parse-results.sh, with scoping by .feature, by folder, and by tag
lippia-neo-find-existing-tests ✅ Searches the .feature files by tags, titles, and the # bug-id: marker
lippia-neo-find-ui-tests-by-code-diff and lippia-neo-repair-selectors-by-diff ✅ They search and repair in pages/ and steps/; they never touch the text of a .feature
lippia-neo-refactor-gherkin-features ✅ Restructures existing .feature files so they are data-driven
CI jobs from lippia-neo-bootstrap-ci ✅ Uses the same installation as Playwright (npm ci + browsers)
Scanning the conventions of an existing suite ❌ Not supported: new cases follow the adapter's default convention (section 5)
Value in qa-stack.yaml test_framework: playwright-bdd
Language javascript or typescript (if there is a tsconfig.json)
What it uses @playwright/test + playwright-bdd (scenarios in Gherkin)
App types web, api, and hybrid
Authentication basic_auth, api_token, google_sso (with limitations, see section 4), or none
Default tests folder tests/playwright-bdd/
Configuration file playwright.config.<ext> at the root of the repository (not inside tests_dir)

1. What you need installed

Tool Version How to check it
Node.js 18 or higher node -v
@playwright/test The one in package.json npx playwright --version
playwright-bdd 8 or higher (tested with 8.4.1 and 9.2) Listed in package.json
Playwright browsers The ones for your Playwright version npx playwright install
dotenv — Listed in package.json. Not needed if the project uses process.loadEnvFile (Node 20.12+)
Bash (Git Bash or WSL on Windows) — bash --version
npm install -D @playwright/test playwright-bdd dotenv
npx playwright install

Add the generated tests folder to .gitignore: npx bddgen creates it on every run and it is not edited by hand.

tests/playwright-bdd/.features-gen/

For the skills that browse the application live (lippia-neo-run-manual-testing, lippia-neo-reproduce-bug), NEO uses the Playwright MCP, regardless of the testing framework (see Playwright MCP setup).


2. Sample qa-stack.yaml

stack:
  test_framework: playwright-bdd
  language: typescript        # or javascript
  app_type: web               # web | api | hybrid
  auth: basic_auth            # google_sso | basic_auth | api_token | none
  pipeline_provider: gitlab

environment:
  base_url: https://techstore-staging.com
  app_version: 2.4.1
  os: Windows 11

paths:
  user_stories_storage:
    type: local
    cloud_provider_mcp: null
    local_path: user-stories/
  test_plans_dir: test-plans/
  tests_dir: tests/playwright-bdd/
  reports_dir: test-results/reports/
  scripted_dir: execution/
  exploratory_dir: exploration/
  env_file: .env.tests

commands:
  run_tests: null        # uses the adapter's run.sh → npx bddgen + npx playwright test
  install_deps: null     # uses npm ci

There is no separate field to choose the Gherkin runner: the value of test_framework (playwright-bdd or playwright-cucumberjs) already defines it.


3. Configure playwright.config

The config goes at the root of the repository and is built with defineBddConfig. lippia-neo-generate-test-suite shows you the block that matches your app type, but it's best to have it ready beforehand. Three values are mandatory because lippia-neo-run-and-heal depends on them:

What Why Value
featuresRoot So the spec generated from each .feature can be mapped back to the .feature The same value as paths.tests_dir
outputDir Where bddgen writes the generated tests <tests_dir>/.features-gen
JSON reporter parse-results.sh reads that file to know what failed ['json', { outputFile: 'test-results/results.json' }]
Loading .env.tests So the steps read the credentials without having them written down. It is loaded only here: each worker reads the config again require('dotenv').config({ path: '.env.tests' })
// playwright.config.js (root of the repository)
const { defineConfig, devices } = require('@playwright/test');
const { defineBddConfig } = require('playwright-bdd');
require('dotenv').config({ path: '.env.tests' }); // = paths.env_file

const TESTS_DIR = 'tests/playwright-bdd';           // = paths.tests_dir

const TEST_DIR = defineBddConfig({
  featuresRoot: TESTS_DIR,
  outputDir: `${TESTS_DIR}/.features-gen`,
  features: `${TESTS_DIR}/features/**/*.feature`,
  steps: [`${TESTS_DIR}/fixtures.js`, `${TESTS_DIR}/steps/**/*.js`], // .ts if the project is TypeScript
});

module.exports = defineConfig({
  retries: process.env.CI ? 2 : 1,
  timeout: 60000,
  reporter: [
    ['json', { outputFile: 'test-results/results.json' }], // required by lippia-neo-run-and-heal
    ['html', { outputFolder: 'playwright-report', open: 'never' }],
    ['list'],
  ],
  use: {
    baseURL: process.env.APP_BASE_URL || 'https://techstore-staging.com',
    trace: 'on-first-retry',
    screenshot: 'only-on-failure',
    // testIdAttribute: 'data-test',  // only if the app doesn't use data-testid
  },
  projects: [
    { name: 'chromium', testDir: TEST_DIR, grepInvert: /@api/, use: { ...devices['Desktop Chrome'] } },
    { name: 'api',      testDir: TEST_DIR, grep: /@api/ },
  ],
});

By app type

UI and API tests are separated by the @api tag on the Feature, not by two different configs:

app_type projects
web chromium (with grepInvert: /@api/). With google_sso, also setup (see section 4)
api Only api (with grep: /@api/)
hybrid chromium and api (and setup with google_sso)

If the suite uses more than one data profile, add one project per profile with use: { dataProfile: '<profile>' }. With a single profile you don't need to: TEST_DATA_PROFILE or default is used.


4. Authentication

basic_auth and api_token

Fill in the values in .env.tests, the same as with Playwright. The steps read them with process.env.NAME and never write them literally:

  • basic_auth: the Background of each .feature logs in with process.env.APP_USER / process.env.APP_PASSWORD through the page object (steps/common.steps.<ext>).
  • api_token: the apiManager and apiViewer fixtures create the API context with API_TOKEN_MANAGER and API_TOKEN_VIEWER.

google_sso

Google login is not automated. The session is loaded from a browser state file:

  • The chromium project uses storageState: 'playwright/.auth/user.json' and depends on the setup project, which is the one that generates that file.
  • The step el usuario está autenticado only opens the app: the session is already loaded.
  • run.sh checks that playwright/.auth/user.json exists before running and, if it isn't there, stops with Auth state no encontrado.

Current session limitation

This adapter does not use paths.auth_state_file or the session.* block of qa-stack.yaml yet: the session path is fixed (playwright/.auth/user.json, or whatever you set in QA_AUTH_STATE_PATH). Also, the config references an auth.setup.js that this adapter does not include: take it from the Playwright adapter or generate the session file with your project's script. With basic_auth, the Background logs in on every scenario.

Never commit the session file to git. Add playwright/.auth/ to .gitignore.


5. What NEO generates

playwright.config.<ext>                         ← root of the repository
tests/playwright-bdd/                           ← paths.tests_dir
├── features/
│   ├── carrito.feature                         ← one .feature per business flow (UI)
│   ├── checkout.feature
│   └── ordenes.api.feature                     ← one .feature per business flow (API), with @api
├── steps/
│   ├── common.steps.<ext>                      ← Background (login), only once per project
│   ├── carrito.steps.<ext>                     ← one per page object (CarritoPage)
│   └── ordenes.api.steps.<ext>                 ← one per API resource (/api/ordenes)
├── pages/
│   └── CarritoPage.<ext>                       ← page objects
├── data/
│   └── default/<flujo>.json · <dominio>.json   ← only if some case uses file data
├── fixtures.<ext>                              ← fixtures + Given/When/Then from createBdd
├── test-data.<ext>                             ← resolves the test data of the active profile
└── .features-gen/                              ← generated by npx bddgen: not edited or versioned

How scenarios are organized

  • One .feature per business flow, not per story or per case: features/<flujo>.feature (for example modificar-carrito.feature). Cases from different stories that test the same flow go in the same file; each new case is appended as a Scenario at the end. There are no per-story folders.
  • Traceability only through tags: each Scenario carries @<US-ID> and @<case ID> exactly as they appear in the plan, and its title starts with that ID.
  • API .feature files end in .api.feature and carry the @api tag on the Feature. Without that tag, an API case opens a browser.
  • Scenarios that only differ in data are written as a Scenario Outline with one Examples block per case (see Data-driven testing).
Feature: Modificar el carrito

  Background:
    Given el usuario está autenticado

  @US-002 @TC-01
  Scenario: TC-01 — Agregar un producto al carrito
    When el usuario agrega "Mouse Inalámbrico" al carrito
    Then el carrito muestra 1 producto

  @US-008 @TC-03
  Scenario: TC-03 — Quitar el último producto deja el carrito vacío
    ...

Rules the generated tests follow

  • Steps in business language: no selectors, URLs, or credentials in the .feature.
  • Step text is global: before creating one, it checks whether it already exists. A duplicated step makes npx bddgen fail for the whole suite.
  • Step files per page object or per resource, never per story: steps/<page>.steps.<ext> paired with pages/<Nombre>Page.<ext>, and steps/<recurso>.api.steps.<ext> for API.
  • Imports: steps import Given / When / Then from ../fixtures, never from @cucumber/cucumber. The first argument is the fixtures: async ({ carritoPage }, producto) => { … }.
  • Selectors only in page objects, in this order: getByRole, getByLabel, getByTestId, getByText and, as a last resort, CSS. If a selector is unknown, page.locator('TODO') is left with a comment.
  • State between steps of the same scenario goes in the ctx fixture, never in global variables.
  • Regression and TDD tests stay red: they are never marked with @skip, @fixme, or @fail (in playwright-bdd, @fail inverts the result and would hide the bug).

The templates are in JavaScript (CommonJS). If the project is TypeScript, NEO adapts them to import / export and saves the files with .ts.


6. How lippia-neo-run-and-heal runs it

The adapter's run.sh:

  1. Checks that node_modules, @playwright/test, and playwright-bdd exist, plus the session file if auth is google_sso. If anything is missing, it exits with code 2.
  2. Runs npx bddgen, which validates the steps and generates the tests. If a step is missing or duplicated, it exits with code 2: there are no results, and the skill treats it as incomplete code (it fixes steps/), not as a red test.
  3. Runs npx playwright test --reporter=json with the applicable filters.
  4. parse-results.sh reads test-results/results.json and maps each result to the source .feature (it strips .features-gen/ and .spec.js from the path).
Variable Default What it's for
QA_TESTS_DIR tests/playwright-bdd The root (full suite) or a list of .feature files / folders under features/, separated by spaces
QA_TESTS_ROOT QA_TESTS_DIR if it is a single path paths.tests_dir. Mandatory if QA_TESTS_DIR has more than one path
QA_EXCLUDE_DIRS empty Subfolders of features/ to exclude. Loose .feature files in features/ always run
QA_CUCUMBER_TAGS empty Filter by tag: @US-002, "@US-002 and @TC-02", "@smoke and not @wip"
QA_AUTH_STATE_PATH playwright/.auth/user.json Session file that is validated if auth is google_sso
PLAYWRIGHT_FLAGS empty Extra flags, for example --project=chromium
PLAYWRIGHT_CONFIG empty Use another config file (passed to bddgen and to playwright test)
PLAYWRIGHT_JSON_OUTPUT test-results/results.json Where the JSON report is saved
# Full suite
QA_TESTS_DIR="tests/playwright-bdd" ./run.sh

# A specific flow and a folder
QA_TESTS_ROOT="tests/playwright-bdd" \
QA_TESTS_DIR="tests/playwright-bdd/features/carrito.feature tests/playwright-bdd/features/legacy" \
./run.sh

# Only one story (by tag)
QA_TESTS_DIR="tests/playwright-bdd" QA_CUCUMBER_TAGS="@US-002" ./run.sh

When a playbook scopes the run, lippia-neo-run-and-heal translates its options like this (it always passes QA_TESTS_ROOT too):

lippia-neo-run-and-heal option Becomes
config.target_tests QA_TESTS_DIR with the given .feature files
config.include_dirs QA_TESTS_DIR with <tests_dir>/features/<carpeta> for each folder
config.exclude_dirs QA_EXCLUDE_DIRS with the names of the features/ subfolders

Where it applies repairs

Failure type What it touches
selector_broken, timeout The page object (pages/)
assertion_mismatch The step definition (steps/). The .feature only if the expected business behavior changed
Missing or duplicated step (bddgen) steps/
environment Nothing in the test: only how .env.tests is loaded or the variable name

NEO never changes the text of a step inside the .feature to match a step definition, and never edits .features-gen/. A case blocked by data (@fixme with a # bloqueado: comment) is not repaired.

In a Scenario Outline, each Examples row is a test. If the row has no title of its own, it shows up in the results as <Outline title> — Example #N.


7. Quick checklist

node -v                                     # 18 or higher
npx playwright --version                    # Playwright installed
grep -n "playwright-bdd" package.json       # playwright-bdd installed
grep -n "defineBddConfig\|results.json" playwright.config.*
npx bddgen                                  # validates the steps without running anything
npx playwright test --list                  # lists one test per scenario and per Examples row

If everything responds fine, ask the assistant: "Run the suite with lippia-neo-run-and-heal".


8. Common problems (and what to do)

What you see Why it may happen What to do
playwright-bdd no encontrado The package isn't installed npm install -D playwright-bdd
'npx bddgen' falló … steps faltantes o duplicados A step in the .feature has no definition, or is defined twice Complete or deduplicate the step in steps/. That's what lippia-neo-run-and-heal does
QA_TESTS_DIR trae varios paths; definí QA_TESTS_ROOT More than one .feature or folder was passed Set QA_TESTS_ROOT to paths.tests_dir
Auth state no encontrado en 'playwright/.auth/user.json' With google_sso, the session file doesn't exist Generate it (see section 4) or set the real path in QA_AUTH_STATE_PATH
Results point to .features-gen/…spec.js instead of the .feature featuresRoot or outputDir don't match paths.tests_dir Align the config with section 3
An API test opens a browser The .api.feature doesn't have the @api tag on the Feature Add @api above Feature:
A blocked case doesn't show up in the report All scenarios in that .feature are @fixme, and bddgen doesn't generate the spec Expected: it shows up once the file has a scenario that runs
lippia-neo-bootstrap-stack asks you which runner to use package.json includes both playwright-bdd and @cucumber/cucumber Choose which one the suite is generated with

9. Current limitations

What What happens with playwright-bdd Alternative
Running a single case Passing a .feature runs all its scenarios (the whole flow) Filter by tag: QA_CUCUMBER_TAGS="@US-002 and @TC-02"
Running from a page object or step If target_tests includes a page object, a step, or fixtures.<ext> (typical after lippia-neo-repair-selectors-by-diff), run.sh runs the full suite and reports it —
Authenticated session Doesn't use paths.auth_state_file or session.*; the path is fixed QA_AUTH_STATE_PATH
Scanning the existing suite's conventions Not supported: it places, names, and tags cases with the default convention —
Adding a case to an existing Scenario Outline When generating, a row is never added to an Outline that already exists: the case goes separately, with a warning Merge them by hand or with lippia-neo-refactor-gherkin-features
CI history with Scenario Outline playwright-bdd groups the rows of an Outline inside a describe with the Outline's title: the full test path changes If your CI keeps history by full path, it will see them as new tests once