Configure playwright-cucumberjs
Adapter status: ✅ Complete. playwright-cucumberjs is a valid value of stack.test_framework: lippia-neo-bootstrap-stack detects it, lippia-neo-generate-test-suite writes the scenarios in Gherkin (.feature) with their steps and page objects, and lippia-neo-run-and-heal runs and repairs them.
With this adapter, cucumber-js (@cucumber/cucumber) runs the .feature files directly and Playwright is used as a library to drive the browser. There are no Playwright fixtures or projects: the browser, the context, and the failure evidence are handled by a World and some hooks that NEO generates.
Would you rather run the
.featurefiles with the Playwright runner (fixtures,projects, traces)? See Configure playwright-bdd. Both guides share the suite organization; what changes is the configuration, the "glue" between steps and Playwright, and how it runs.
| Piece | Does it support playwright-cucumberjs? |
|---|---|
Automatic detection in lippia-neo-bootstrap-stack |
✅ When package.json includes @cucumber/cucumber and also @playwright/test or playwright |
lippia-neo-generate-test-suite (adapters/playwright-cucumberjs/) |
✅ README and 9 templates |
lippia-neo-run-and-heal (adapters/playwright-cucumberjs/) |
✅ run.sh and parse-results.sh, with scoping by .feature, by folder, and by tag |
lippia-neo-find-existing-tests |
✅ Searches the .feature files by tags, titles, and the # bug-id: marker |
lippia-neo-find-ui-tests-by-code-diff and lippia-neo-repair-selectors-by-diff |
✅ They search and repair in pages/ and steps/; they never touch the text of a .feature |
lippia-neo-refactor-gherkin-features |
✅ Restructures existing .feature files so they are data-driven |
CI jobs from lippia-neo-bootstrap-ci |
✅ Uses the same installation as Playwright (npm ci + browsers) |
| Scanning the conventions of an existing suite | ❌ Not supported: new cases follow the adapter's default convention (section 5) |
Value in qa-stack.yaml |
test_framework: playwright-cucumberjs |
| Language | javascript or typescript (if there is a tsconfig.json) |
| What it uses | @cucumber/cucumber + Playwright as a library (scenarios in Gherkin) |
| App types | web, api, and hybrid |
| Authentication | basic_auth, api_token, google_sso (with limitations, see section 4), or none |
| Default tests folder | tests/playwright-cucumberjs/ |
| Configuration file | cucumber.js (or .cjs / .mjs) at the root of the repository |
1. What you need installed
| Tool | Version | How to check it |
|---|---|---|
| Node.js | 18 or higher | node -v |
@cucumber/cucumber |
The one in package.json (tested with 12.2.0) |
npx cucumber-js --version |
@playwright/test |
The one in package.json |
npx playwright --version |
| Playwright browsers | The ones for your Playwright version | npx playwright install |
dotenv |
— | Listed in package.json. Not needed if the project uses process.loadEnvFile (Node 20.12+) |
tsx |
— | Only if the project is TypeScript |
| Bash (Git Bash or WSL on Windows) | — | bash --version |
npm install -D @cucumber/cucumber @playwright/test dotenv
# TypeScript only:
npm install -D tsx
npx playwright install
For the skills that browse the application live (
lippia-neo-run-manual-testing,lippia-neo-reproduce-bug),NEOuses the Playwright MCP, regardless of the testing framework (see Playwright MCP setup).
2. Sample qa-stack.yaml
stack:
test_framework: playwright-cucumberjs
language: javascript # or typescript
app_type: hybrid # web | api | hybrid
auth: basic_auth # google_sso | basic_auth | api_token | none
pipeline_provider: github
environment:
base_url: https://techstore-staging.com
app_version: 2.4.1
os: Windows 11
paths:
user_stories_storage:
type: local
cloud_provider_mcp: null
local_path: user-stories/
test_plans_dir: test-plans/
tests_dir: tests/playwright-cucumberjs/
reports_dir: test-results/reports/
scripted_dir: execution/
exploratory_dir: exploration/
env_file: .env.tests
commands:
run_tests: null # uses the adapter's run.sh → npx cucumber-js --dry-run + npx cucumber-js
install_deps: null # uses npm ci
There is no separate field to choose the Gherkin runner: the value of
test_framework(playwright-bddorplaywright-cucumberjs) already defines it.
3. Configure cucumber.js
The config goes at the root of the repository:
// cucumber.js (root of the repository)
const TESTS_DIR = 'tests/playwright-cucumberjs'; // = paths.tests_dir
const base = {
require: [`${TESTS_DIR}/support/**/*.js`, `${TESTS_DIR}/steps/**/*.js`], // .ts if the project is TypeScript
// requireModule: ['tsx/cjs'], // TypeScript only
format: ['progress', 'html:reports/cucumber-report.html'],
retry: process.env.CI ? 2 : 1,
};
module.exports = {
default: base,
// One cucumber-js profile per data profile (only if you use more than one):
// 'perfil-b': { ...base, worldParameters: { dataProfile: 'perfil-b' } },
};
Three rules that lippia-neo-run-and-heal needs to be met:
| Rule | Why |
|---|---|
Don't declare paths |
cucumber-js adds the config's paths to the ones it receives on the command line: with paths in the config, run.sh can't run a single .feature and always runs the full suite. To run by hand: npx cucumber-js tests/playwright-cucumberjs/features |
Don't add a message formatter |
run.sh adds its own (test-results/cucumber-messages.ndjson); two formatters writing the same file overwrite each other |
retry greater than 0 |
Without retries, an unstable (flaky) test can't be detected |
If your config already declares
paths,run.shwarns about it in the log andlippia-neo-run-and-healrecords it in its notes: the fix is to removepathsfrom the config, not to touch the tests.
4. Authentication
basic_auth and api_token
Fill in the values in .env.tests, the same as with Playwright. The file is loaded in support/hooks.<ext>, and the steps read the values with process.env.NAME, never writing them literally:
basic_auth: theBackgroundof each.featurelogs in withprocess.env.APP_USER/process.env.APP_PASSWORDthrough the page object (steps/common.steps.<ext>).api_token: theBeforeof@apiscenarios creates theapiManagerandapiViewercontexts withAPI_TOKEN_MANAGERandAPI_TOKEN_VIEWER.
google_sso
Google login is not automated. The Before in support/hooks.<ext> opens the context with storageState: 'playwright/.auth/user.json', and the step el usuario está autenticado only opens the app. run.sh checks that this file exists and, if it isn't there, stops with Auth state no encontrado.
Current session limitation
This adapter does not use paths.auth_state_file or the session.* block of qa-stack.yaml yet: the session path is fixed (playwright/.auth/user.json, or whatever you set in QA_AUTH_STATE_PATH). The session file is generated with your project's script, the same as in the Playwright adapter. With basic_auth, the Background logs in on every scenario.
Never commit the session file to git. Add
playwright/.auth/to.gitignore.
5. What NEO generates
cucumber.js ← root of the repository
tests/playwright-cucumberjs/ ← paths.tests_dir
├── features/
│ ├── carrito.feature ← one .feature per business flow (UI)
│ ├── checkout.feature
│ └── ordenes.api.feature ← one .feature per business flow (API), with @api
├── steps/
│ ├── common.steps.<ext> ← Background (login), only once per project
│ ├── carrito.steps.<ext> ← one per page object (CarritoPage)
│ └── ordenes.api.steps.<ext> ← one per API resource (/api/ordenes)
├── pages/
│ └── CarritoPage.<ext> ← page objects
├── data/
│ └── default/<flujo>.json · <dominio>.json ← only if some case uses file data
└── support/
├── world.<ext> ← CustomWorld: page, API contexts, ctx, testData, page objects
├── hooks.<ext> ← opens and closes the browser, saves failure evidence
└── test-data.<ext> ← resolves the test data of the active profile
How scenarios are organized
Same as in playwright-bdd:
- One
.featureper business flow (features/<flujo>.feature), not per story or per case. Cases from different stories that test the same flow go in the same file, appended at the end. - Traceability only through tags: each
Scenariocarries@<US-ID>and@<case ID>exactly as they appear in the plan, and its title starts with that ID (Scenario: TC-01 — Agregar un producto al carrito). - API
.featurefiles end in.api.featureand carry@apion theFeature: the hooks use that tag to avoid opening a browser. - Scenarios that only differ in data are written as a
Scenario Outlinewith oneExamplesblock per case (see Data-driven testing).
The World and the hooks
| File | What it does |
|---|---|
support/world.<ext> |
Defines the CustomWorld each step receives as this: browser, context, page, apiManager / apiViewer, ctx (scenario state), testData, and one field per page object |
support/hooks.<ext> |
Loads .env.tests. Before (UI scenarios): launches Chromium (headless unless HEADED=1), opens the context with baseURL, starts the trace, and instantiates the page objects. Before (@api): creates the API contexts. After: if the scenario failed, saves a screenshot and trace in test-results/cucumber-evidence/ and attaches them; then closes everything |
Don't modify the evidence attachment in the After
The After attaches the screenshot and trace paths as text/uri-list. That's what lippia-neo-run-and-heal reads to have evidence for each failure. If you remove it or change its format, the repair only has the error message to work with.
Rules the generated tests follow
- Steps in business language: no selectors, URLs, or credentials in the
.feature. - Step text is global: before creating one, it checks whether it already exists. A duplicated step ends up
ambiguousand makes validation fail for the whole suite. - Step files per page object or per resource, never per story:
steps/<page>.steps.<ext>paired withpages/<Nombre>Page.<ext>, andsteps/<recurso>.api.steps.<ext>for API. -
Steps as
async function, never arrow functions: they usethis(the World).Given/When/Thencome from@cucumber/cucumberandexpectfrom@playwright/test.const { When } = require('@cucumber/cucumber'); When('el usuario busca {string}', async function (termino) { await this.busquedaPage.buscar(termino); }); -
Each new page object touches two files: it is declared as a field in
support/world.<ext>and instantiated in theBeforeofsupport/hooks.<ext>(this.carritoPage = new CarritoPage(this.page)). - Selectors only in page objects, in this order:
getByRole,getByLabel,getByTestId,getByTextand, as a last resort, CSS. If the app uses another testing attribute (for exampledata-test), it is configured once withselectors.setTestIdAttribute('data-test')in the hooks. - State between steps of the same scenario goes in
this.ctx, never in global variables. - Regression and TDD tests stay red: they are never marked with
@skip,@fixme, or@fail.
The templates are in JavaScript (CommonJS). If the project is TypeScript,
NEOadapts them toimport/export, typesthisin the steps (async function (this: CustomWorld, termino: string)), and saves the files with.ts.
6. How lippia-neo-run-and-heal runs it
The adapter's run.sh:
- Checks that
node_modules,@cucumber/cucumber, thefeatures/folder exist, plus the session file ifauthisgoogle_sso. If anything is missing, it exits with code2. - Runs
npx cucumber-js --dry-run(it doesn't run steps or hooks) and looks forundefinedorambiguoussteps. cucumber-js exits with 0 even when there are some, which is whyrun.shlooks for them in the output. If it finds any, it exits with code2: there are no results, and the skill treats it as incomplete code (it fixessteps/), not as a red test. - Runs
npx cucumber-js --format message:test-results/cucumber-messages.ndjson <paths>. parse-results.shreads that NDJSON (not thejsonformatter, which doesn't show retries). Each scenario's status is the worst of its steps and hooks in the last attempt, and the evidence comes from theAfter'stext/uri-listattachment.
| Variable | Default | What it's for |
|---|---|---|
QA_TESTS_DIR |
tests/playwright-cucumberjs |
The root (full suite) or a list of .feature files / folders under features/, separated by spaces |
QA_TESTS_ROOT |
QA_TESTS_DIR if it is a single path |
paths.tests_dir. Mandatory if QA_TESTS_DIR has more than one path |
QA_EXCLUDE_DIRS |
empty | Subfolders of features/ to exclude. Loose .feature files in features/ always run |
QA_CUCUMBER_TAGS |
empty | Filter by tag: @US-002, "@US-002 and @TC-02", "@smoke and not @wip" |
QA_AUTH_STATE_PATH |
playwright/.auth/user.json |
Session file that is validated if auth is google_sso |
CUCUMBER_FLAGS |
empty | Extra flags, for example --retry 0 or --parallel 2 |
CUCUMBER_CONFIG |
empty | Use another config file (passed with --config to the dry-run and to the run) |
CUCUMBER_MESSAGES_OUTPUT |
test-results/cucumber-messages.ndjson |
Where the result is saved |
CUCUMBER_ATTACHMENTS_DIR |
test-results/cucumber-attachments |
Where a screenshot that arrived embedded in the result is saved |
# Full suite
QA_TESTS_DIR="tests/playwright-cucumberjs" ./run.sh
# A specific flow and a folder
QA_TESTS_ROOT="tests/playwright-cucumberjs" \
QA_TESTS_DIR="tests/playwright-cucumberjs/features/carrito.feature tests/playwright-cucumberjs/features/legacy" \
./run.sh
# Only one story (by tag)
QA_TESTS_DIR="tests/playwright-cucumberjs" QA_CUCUMBER_TAGS="@US-002" ./run.sh
When a playbook scopes the run, lippia-neo-run-and-heal translates its options like this (it always passes QA_TESTS_ROOT too):
lippia-neo-run-and-heal option |
Becomes |
|---|---|
config.target_tests |
QA_TESTS_DIR with the given .feature files |
config.include_dirs |
QA_TESTS_DIR with <tests_dir>/features/<carpeta> for each folder |
config.exclude_dirs |
QA_EXCLUDE_DIRS with the names of the features/ subfolders |
Where it applies repairs
| Failure type | What it touches |
|---|---|
selector_broken, timeout |
The page object (pages/). If a new page object is needed, also the World and the Before |
assertion_mismatch |
The step definition (steps/). The .feature only if the expected business behavior changed |
undefined or ambiguous step (dry-run) |
steps/ |
environment |
Nothing in the test: only how .env.tests is loaded or the variable name |
NEO never changes the text of a step inside the .feature to match a step definition.
In a
Scenario Outline, eachExamplesrow is a test, and it shows up in the results as<title with the values> — Example #N(N counts all theExamplestables of the Outline).
7. Quick checklist
node -v # 18 or higher
npx cucumber-js --version # cucumber-js installed
npx playwright --version # Playwright installed
grep -n "paths" cucumber.* # a paths key must not show up
grep -n "retry" cucumber.* # retry greater than 0
npx cucumber-js --dry-run tests/playwright-cucumberjs/features # the summary must have no undefined or ambiguous
If everything responds fine, ask the assistant: "Run the suite with lippia-neo-run-and-heal".
8. Common problems (and what to do)
| What you see | Why it may happen | What to do |
|---|---|---|
@cucumber/cucumber no encontrado |
The package isn't installed | npm install -D @cucumber/cucumber |
'npx cucumber-js --dry-run' falló … steps undefined o ambiguous |
A step in the .feature has no definition, or two definitions match it |
Complete or deduplicate the step in steps/. That's what lippia-neo-run-and-heal does |
The full suite always runs, even when a single .feature is requested |
The config declares paths |
Remove paths from cucumber.js |
this.<algoPage> is undefined in a step |
The page object isn't instantiated in the hooks' Before |
Declare it in the World and instantiate it in support/hooks.<ext> |
this is undefined in a step |
The step was written as an arrow function | Use async function () { … } |
QA_TESTS_DIR trae varios paths; definí QA_TESTS_ROOT |
More than one .feature or folder was passed |
Set QA_TESTS_ROOT to paths.tests_dir |
Auth state no encontrado en 'playwright/.auth/user.json' |
With google_sso, the session file doesn't exist |
Generate it (see section 4) or set the real path in QA_AUTH_STATE_PATH |
cucumber-js no escribió el NDJSON |
cucumber-js failed before finishing (config with errors, missing module) | Check the output shown above the error |
| Failures without a screenshot or trace | The After's text/uri-list attachment was modified or removed |
Restore the After in support/hooks.<ext> |
flaky tests aren't detected |
retry is 0 |
Set retry greater than 0 in the config |
lippia-neo-bootstrap-stack asks you which runner to use |
package.json includes both playwright-bdd and @cucumber/cucumber |
Choose which one the suite is generated with |
9. Current limitations
| What | What happens with playwright-cucumberjs | Alternative |
|---|---|---|
| Cases blocked by missing data | This runner has no marker to leave them written and skipped: the case is not written in the .feature and only shows up in skipped_scenarios |
Write it when the data is available (see Data-driven testing) |
| Running a single case | Passing a .feature runs all its scenarios (the whole flow) |
Filter by tag: QA_CUCUMBER_TAGS="@US-002 and @TC-02" |
| Running from a page object or step | If target_tests includes a page object, a step, or a support/ file (typical after lippia-neo-repair-selectors-by-diff), run.sh runs the full suite and reports it |
— |
| Authenticated session | Doesn't use paths.auth_state_file or session.*; the path is fixed |
QA_AUTH_STATE_PATH |
| Scanning the existing suite's conventions | Not supported: it places, names, and tags cases with the default convention | — |
Adding a case to an existing Scenario Outline |
When generating, a row is never added to an Outline that already exists: the case goes separately, with a warning | Merge them by hand or with lippia-neo-refactor-gherkin-features |
Parallel execution (--parallel) |
It works, but screenshots and traces are named by scenario and time, not by worker | — |