Skip to content

Configure playwright-cucumberjs

Adapter status: ✅ Complete. playwright-cucumberjs is a valid value of stack.test_framework: lippia-neo-bootstrap-stack detects it, lippia-neo-generate-test-suite writes the scenarios in Gherkin (.feature) with their steps and page objects, and lippia-neo-run-and-heal runs and repairs them.

With this adapter, cucumber-js (@cucumber/cucumber) runs the .feature files directly and Playwright is used as a library to drive the browser. There are no Playwright fixtures or projects: the browser, the context, and the failure evidence are handled by a World and some hooks that NEO generates.

Would you rather run the .feature files with the Playwright runner (fixtures, projects, traces)? See Configure playwright-bdd. Both guides share the suite organization; what changes is the configuration, the "glue" between steps and Playwright, and how it runs.

Piece Does it support playwright-cucumberjs?
Automatic detection in lippia-neo-bootstrap-stack ✅ When package.json includes @cucumber/cucumber and also @playwright/test or playwright
lippia-neo-generate-test-suite (adapters/playwright-cucumberjs/) ✅ README and 9 templates
lippia-neo-run-and-heal (adapters/playwright-cucumberjs/) ✅ run.sh and parse-results.sh, with scoping by .feature, by folder, and by tag
lippia-neo-find-existing-tests ✅ Searches the .feature files by tags, titles, and the # bug-id: marker
lippia-neo-find-ui-tests-by-code-diff and lippia-neo-repair-selectors-by-diff ✅ They search and repair in pages/ and steps/; they never touch the text of a .feature
lippia-neo-refactor-gherkin-features ✅ Restructures existing .feature files so they are data-driven
CI jobs from lippia-neo-bootstrap-ci ✅ Uses the same installation as Playwright (npm ci + browsers)
Scanning the conventions of an existing suite ❌ Not supported: new cases follow the adapter's default convention (section 5)
Value in qa-stack.yaml test_framework: playwright-cucumberjs
Language javascript or typescript (if there is a tsconfig.json)
What it uses @cucumber/cucumber + Playwright as a library (scenarios in Gherkin)
App types web, api, and hybrid
Authentication basic_auth, api_token, google_sso (with limitations, see section 4), or none
Default tests folder tests/playwright-cucumberjs/
Configuration file cucumber.js (or .cjs / .mjs) at the root of the repository

1. What you need installed

Tool Version How to check it
Node.js 18 or higher node -v
@cucumber/cucumber The one in package.json (tested with 12.2.0) npx cucumber-js --version
@playwright/test The one in package.json npx playwright --version
Playwright browsers The ones for your Playwright version npx playwright install
dotenv — Listed in package.json. Not needed if the project uses process.loadEnvFile (Node 20.12+)
tsx — Only if the project is TypeScript
Bash (Git Bash or WSL on Windows) — bash --version
npm install -D @cucumber/cucumber @playwright/test dotenv
# TypeScript only:
npm install -D tsx
npx playwright install

For the skills that browse the application live (lippia-neo-run-manual-testing, lippia-neo-reproduce-bug), NEO uses the Playwright MCP, regardless of the testing framework (see Playwright MCP setup).


2. Sample qa-stack.yaml

stack:
  test_framework: playwright-cucumberjs
  language: javascript        # or typescript
  app_type: hybrid            # web | api | hybrid
  auth: basic_auth            # google_sso | basic_auth | api_token | none
  pipeline_provider: github

environment:
  base_url: https://techstore-staging.com
  app_version: 2.4.1
  os: Windows 11

paths:
  user_stories_storage:
    type: local
    cloud_provider_mcp: null
    local_path: user-stories/
  test_plans_dir: test-plans/
  tests_dir: tests/playwright-cucumberjs/
  reports_dir: test-results/reports/
  scripted_dir: execution/
  exploratory_dir: exploration/
  env_file: .env.tests

commands:
  run_tests: null        # uses the adapter's run.sh → npx cucumber-js --dry-run + npx cucumber-js
  install_deps: null     # uses npm ci

There is no separate field to choose the Gherkin runner: the value of test_framework (playwright-bdd or playwright-cucumberjs) already defines it.


3. Configure cucumber.js

The config goes at the root of the repository:

// cucumber.js (root of the repository)
const TESTS_DIR = 'tests/playwright-cucumberjs';   // = paths.tests_dir

const base = {
  require: [`${TESTS_DIR}/support/**/*.js`, `${TESTS_DIR}/steps/**/*.js`], // .ts if the project is TypeScript
  // requireModule: ['tsx/cjs'],                    // TypeScript only
  format: ['progress', 'html:reports/cucumber-report.html'],
  retry: process.env.CI ? 2 : 1,
};

module.exports = {
  default: base,
  // One cucumber-js profile per data profile (only if you use more than one):
  // 'perfil-b': { ...base, worldParameters: { dataProfile: 'perfil-b' } },
};

Three rules that lippia-neo-run-and-heal needs to be met:

Rule Why
Don't declare paths cucumber-js adds the config's paths to the ones it receives on the command line: with paths in the config, run.sh can't run a single .feature and always runs the full suite. To run by hand: npx cucumber-js tests/playwright-cucumberjs/features
Don't add a message formatter run.sh adds its own (test-results/cucumber-messages.ndjson); two formatters writing the same file overwrite each other
retry greater than 0 Without retries, an unstable (flaky) test can't be detected

If your config already declares paths, run.sh warns about it in the log and lippia-neo-run-and-heal records it in its notes: the fix is to remove paths from the config, not to touch the tests.


4. Authentication

basic_auth and api_token

Fill in the values in .env.tests, the same as with Playwright. The file is loaded in support/hooks.<ext>, and the steps read the values with process.env.NAME, never writing them literally:

  • basic_auth: the Background of each .feature logs in with process.env.APP_USER / process.env.APP_PASSWORD through the page object (steps/common.steps.<ext>).
  • api_token: the Before of @api scenarios creates the apiManager and apiViewer contexts with API_TOKEN_MANAGER and API_TOKEN_VIEWER.

google_sso

Google login is not automated. The Before in support/hooks.<ext> opens the context with storageState: 'playwright/.auth/user.json', and the step el usuario está autenticado only opens the app. run.sh checks that this file exists and, if it isn't there, stops with Auth state no encontrado.

Current session limitation

This adapter does not use paths.auth_state_file or the session.* block of qa-stack.yaml yet: the session path is fixed (playwright/.auth/user.json, or whatever you set in QA_AUTH_STATE_PATH). The session file is generated with your project's script, the same as in the Playwright adapter. With basic_auth, the Background logs in on every scenario.

Never commit the session file to git. Add playwright/.auth/ to .gitignore.


5. What NEO generates

cucumber.js                                     ← root of the repository
tests/playwright-cucumberjs/                    ← paths.tests_dir
├── features/
│   ├── carrito.feature                         ← one .feature per business flow (UI)
│   ├── checkout.feature
│   └── ordenes.api.feature                     ← one .feature per business flow (API), with @api
├── steps/
│   ├── common.steps.<ext>                      ← Background (login), only once per project
│   ├── carrito.steps.<ext>                     ← one per page object (CarritoPage)
│   └── ordenes.api.steps.<ext>                 ← one per API resource (/api/ordenes)
├── pages/
│   └── CarritoPage.<ext>                       ← page objects
├── data/
│   └── default/<flujo>.json · <dominio>.json   ← only if some case uses file data
└── support/
    ├── world.<ext>                             ← CustomWorld: page, API contexts, ctx, testData, page objects
    ├── hooks.<ext>                             ← opens and closes the browser, saves failure evidence
    └── test-data.<ext>                         ← resolves the test data of the active profile

How scenarios are organized

Same as in playwright-bdd:

  • One .feature per business flow (features/<flujo>.feature), not per story or per case. Cases from different stories that test the same flow go in the same file, appended at the end.
  • Traceability only through tags: each Scenario carries @<US-ID> and @<case ID> exactly as they appear in the plan, and its title starts with that ID (Scenario: TC-01 — Agregar un producto al carrito).
  • API .feature files end in .api.feature and carry @api on the Feature: the hooks use that tag to avoid opening a browser.
  • Scenarios that only differ in data are written as a Scenario Outline with one Examples block per case (see Data-driven testing).

The World and the hooks

File What it does
support/world.<ext> Defines the CustomWorld each step receives as this: browser, context, page, apiManager / apiViewer, ctx (scenario state), testData, and one field per page object
support/hooks.<ext> Loads .env.tests. Before (UI scenarios): launches Chromium (headless unless HEADED=1), opens the context with baseURL, starts the trace, and instantiates the page objects. Before (@api): creates the API contexts. After: if the scenario failed, saves a screenshot and trace in test-results/cucumber-evidence/ and attaches them; then closes everything

Don't modify the evidence attachment in the After

The After attaches the screenshot and trace paths as text/uri-list. That's what lippia-neo-run-and-heal reads to have evidence for each failure. If you remove it or change its format, the repair only has the error message to work with.

Rules the generated tests follow

  • Steps in business language: no selectors, URLs, or credentials in the .feature.
  • Step text is global: before creating one, it checks whether it already exists. A duplicated step ends up ambiguous and makes validation fail for the whole suite.
  • Step files per page object or per resource, never per story: steps/<page>.steps.<ext> paired with pages/<Nombre>Page.<ext>, and steps/<recurso>.api.steps.<ext> for API.
  • Steps as async function, never arrow functions: they use this (the World). Given / When / Then come from @cucumber/cucumber and expect from @playwright/test.

    const { When } = require('@cucumber/cucumber');
    
    When('el usuario busca {string}', async function (termino) {
      await this.busquedaPage.buscar(termino);
    });
    
  • Each new page object touches two files: it is declared as a field in support/world.<ext> and instantiated in the Before of support/hooks.<ext> (this.carritoPage = new CarritoPage(this.page)).

  • Selectors only in page objects, in this order: getByRole, getByLabel, getByTestId, getByText and, as a last resort, CSS. If the app uses another testing attribute (for example data-test), it is configured once with selectors.setTestIdAttribute('data-test') in the hooks.
  • State between steps of the same scenario goes in this.ctx, never in global variables.
  • Regression and TDD tests stay red: they are never marked with @skip, @fixme, or @fail.

The templates are in JavaScript (CommonJS). If the project is TypeScript, NEO adapts them to import / export, types this in the steps (async function (this: CustomWorld, termino: string)), and saves the files with .ts.


6. How lippia-neo-run-and-heal runs it

The adapter's run.sh:

  1. Checks that node_modules, @cucumber/cucumber, the features/ folder exist, plus the session file if auth is google_sso. If anything is missing, it exits with code 2.
  2. Runs npx cucumber-js --dry-run (it doesn't run steps or hooks) and looks for undefined or ambiguous steps. cucumber-js exits with 0 even when there are some, which is why run.sh looks for them in the output. If it finds any, it exits with code 2: there are no results, and the skill treats it as incomplete code (it fixes steps/), not as a red test.
  3. Runs npx cucumber-js --format message:test-results/cucumber-messages.ndjson <paths>.
  4. parse-results.sh reads that NDJSON (not the json formatter, which doesn't show retries). Each scenario's status is the worst of its steps and hooks in the last attempt, and the evidence comes from the After's text/uri-list attachment.
Variable Default What it's for
QA_TESTS_DIR tests/playwright-cucumberjs The root (full suite) or a list of .feature files / folders under features/, separated by spaces
QA_TESTS_ROOT QA_TESTS_DIR if it is a single path paths.tests_dir. Mandatory if QA_TESTS_DIR has more than one path
QA_EXCLUDE_DIRS empty Subfolders of features/ to exclude. Loose .feature files in features/ always run
QA_CUCUMBER_TAGS empty Filter by tag: @US-002, "@US-002 and @TC-02", "@smoke and not @wip"
QA_AUTH_STATE_PATH playwright/.auth/user.json Session file that is validated if auth is google_sso
CUCUMBER_FLAGS empty Extra flags, for example --retry 0 or --parallel 2
CUCUMBER_CONFIG empty Use another config file (passed with --config to the dry-run and to the run)
CUCUMBER_MESSAGES_OUTPUT test-results/cucumber-messages.ndjson Where the result is saved
CUCUMBER_ATTACHMENTS_DIR test-results/cucumber-attachments Where a screenshot that arrived embedded in the result is saved
# Full suite
QA_TESTS_DIR="tests/playwright-cucumberjs" ./run.sh

# A specific flow and a folder
QA_TESTS_ROOT="tests/playwright-cucumberjs" \
QA_TESTS_DIR="tests/playwright-cucumberjs/features/carrito.feature tests/playwright-cucumberjs/features/legacy" \
./run.sh

# Only one story (by tag)
QA_TESTS_DIR="tests/playwright-cucumberjs" QA_CUCUMBER_TAGS="@US-002" ./run.sh

When a playbook scopes the run, lippia-neo-run-and-heal translates its options like this (it always passes QA_TESTS_ROOT too):

lippia-neo-run-and-heal option Becomes
config.target_tests QA_TESTS_DIR with the given .feature files
config.include_dirs QA_TESTS_DIR with <tests_dir>/features/<carpeta> for each folder
config.exclude_dirs QA_EXCLUDE_DIRS with the names of the features/ subfolders

Where it applies repairs

Failure type What it touches
selector_broken, timeout The page object (pages/). If a new page object is needed, also the World and the Before
assertion_mismatch The step definition (steps/). The .feature only if the expected business behavior changed
undefined or ambiguous step (dry-run) steps/
environment Nothing in the test: only how .env.tests is loaded or the variable name

NEO never changes the text of a step inside the .feature to match a step definition.

In a Scenario Outline, each Examples row is a test, and it shows up in the results as <title with the values> — Example #N (N counts all the Examples tables of the Outline).


7. Quick checklist

node -v                                          # 18 or higher
npx cucumber-js --version                        # cucumber-js installed
npx playwright --version                         # Playwright installed
grep -n "paths" cucumber.*                       # a paths key must not show up
grep -n "retry" cucumber.*                       # retry greater than 0
npx cucumber-js --dry-run tests/playwright-cucumberjs/features   # the summary must have no undefined or ambiguous

If everything responds fine, ask the assistant: "Run the suite with lippia-neo-run-and-heal".


8. Common problems (and what to do)

What you see Why it may happen What to do
@cucumber/cucumber no encontrado The package isn't installed npm install -D @cucumber/cucumber
'npx cucumber-js --dry-run' falló … steps undefined o ambiguous A step in the .feature has no definition, or two definitions match it Complete or deduplicate the step in steps/. That's what lippia-neo-run-and-heal does
The full suite always runs, even when a single .feature is requested The config declares paths Remove paths from cucumber.js
this.<algoPage> is undefined in a step The page object isn't instantiated in the hooks' Before Declare it in the World and instantiate it in support/hooks.<ext>
this is undefined in a step The step was written as an arrow function Use async function () { … }
QA_TESTS_DIR trae varios paths; definí QA_TESTS_ROOT More than one .feature or folder was passed Set QA_TESTS_ROOT to paths.tests_dir
Auth state no encontrado en 'playwright/.auth/user.json' With google_sso, the session file doesn't exist Generate it (see section 4) or set the real path in QA_AUTH_STATE_PATH
cucumber-js no escribió el NDJSON cucumber-js failed before finishing (config with errors, missing module) Check the output shown above the error
Failures without a screenshot or trace The After's text/uri-list attachment was modified or removed Restore the After in support/hooks.<ext>
flaky tests aren't detected retry is 0 Set retry greater than 0 in the config
lippia-neo-bootstrap-stack asks you which runner to use package.json includes both playwright-bdd and @cucumber/cucumber Choose which one the suite is generated with

9. Current limitations

What What happens with playwright-cucumberjs Alternative
Cases blocked by missing data This runner has no marker to leave them written and skipped: the case is not written in the .feature and only shows up in skipped_scenarios Write it when the data is available (see Data-driven testing)
Running a single case Passing a .feature runs all its scenarios (the whole flow) Filter by tag: QA_CUCUMBER_TAGS="@US-002 and @TC-02"
Running from a page object or step If target_tests includes a page object, a step, or a support/ file (typical after lippia-neo-repair-selectors-by-diff), run.sh runs the full suite and reports it —
Authenticated session Doesn't use paths.auth_state_file or session.*; the path is fixed QA_AUTH_STATE_PATH
Scanning the existing suite's conventions Not supported: it places, names, and tags cases with the default convention —
Adding a case to an existing Scenario Outline When generating, a row is never added to an Outline that already exists: the case goes separately, with a warning Merge them by hand or with lippia-neo-refactor-gherkin-features
Parallel execution (--parallel) It works, but screenshots and traces are named by scenario and time, not by worker —