Configure Playwright
Adapter status: ✅ Complete. It is used by lippia-neo-generate-test-suite (to write UI and API tests) and lippia-neo-run-and-heal (to run them and fix them).
Value in qa-stack.yaml |
test_framework: playwright |
| Language | javascript (.spec.js tests) or typescript (.spec.ts tests, requires tsconfig.json) |
| App types | web, api, and hybrid |
| Authentication | google_sso, basic_auth, api_token, or none |
| Default tests folder | tests/playwright/ |
Want to write the tests in Gherkin (
.feature)? Use playwright-bdd or playwright-cucumberjs:
playwrightplaywright-bddplaywright-cucumberjsTest unit One .specper case, in a folder per storyA Scenarioin the business flow's.featureSame as playwright-bdd Who runs it npx playwright testnpx bddgen+npx playwright testnpx cucumber-js(Playwright as a library)Filter by story or case By folder By .feature, folder, or tagBy .feature, folder, or tagData-driven data — ✅ ✅
1. What you need installed
| Tool | Version | What it's for | How to check it |
|---|---|---|---|
| Node.js | 18 or higher | Running Playwright and the scripts | node -v |
@playwright/test |
The one in the project's package.json |
The framework | npx playwright --version |
| Playwright browsers | The ones for your Playwright version | Running the UI tests | npx playwright install |
dotenv |
Any | So playwright.config reads .env.tests |
Listed in package.json |
| Bash (Git Bash or WSL on Windows) | — | Running run.sh and parse-results.sh |
bash --version |
From the root of the project (where package.json is):
npm install
npx playwright install
npx playwright installdownloads the browsers. You only need to run it once per machine, or each time the Playwright version changes.
For the skills that browse the application live (lippia-neo-run-manual-testing, lippia-neo-reproduce-bug), you also need the Playwright MCP with its browser:
npx @playwright/mcp install-browser chrome-for-testing
If that step fails or the MCP doesn't connect, follow Playwright MCP setup.
2. Sample qa-stack.yaml
stack:
test_framework: playwright
language: typescript # or javascript
app_type: web # web | api | hybrid
auth: basic_auth # google_sso | basic_auth | api_token | none
pipeline_provider: github
environment:
base_url: https://techstore-staging.com
app_version: 2.4.1
os: Windows 11
paths:
user_stories_storage:
type: local
cloud_provider_mcp: null
local_path: user-stories/
test_plans_dir: test-plans/
tests_dir: tests/playwright/
reports_dir: test-results/reports/
scripted_dir: execution/
exploratory_dir: exploration/
env_file: .env.tests
commands:
run_tests: null # uses the adapter's run.sh → npx playwright test
install_deps: null # uses npm ci
3. Configure playwright.config
This is the most important step: without it, NEO generates the tests but can't run them or read the results. lippia-neo-generate-test-suite shows you the block that matches your app type when it finishes, but it's a good idea to have it ready beforehand.
Your playwright.config.js (or .ts) must have:
| What | Why | Value |
|---|---|---|
testDir |
So Playwright looks for the tests where NEO writes them |
The same value as paths.tests_dir |
| JSON reporter | parse-results.sh reads that file to know what failed |
['json', { outputFile: 'test-results/results.json' }] |
Loading .env.tests |
So the tests read the credentials without having them written in | require('dotenv').config({ path: '.env.tests' }) |
use.baseURL |
So the tests use relative paths (page.goto('/')) |
The URL of the environment under test |
testMatch with the correct extension |
So Playwright finds the generated files | **/tc-*.spec.ts if the project is TypeScript, **/tc-*.spec.js if it's JavaScript |
Minimal example for a web app:
const { defineConfig, devices } = require('@playwright/test');
require('dotenv').config({ path: '.env.tests' }); // = paths.env_file
module.exports = defineConfig({
testDir: './tests/playwright', // = paths.tests_dir
retries: process.env.CI ? 2 : 1,
timeout: 60000,
reporter: [
['json', { outputFile: 'test-results/results.json' }], // required by lippia-neo-run-and-heal
['html', { outputFolder: 'playwright-report', open: 'never' }],
['list'],
],
use: {
baseURL: process.env.APP_BASE_URL || 'https://techstore-staging.com',
trace: 'on-first-retry',
screenshot: 'only-on-failure',
},
projects: [
{ name: 'chromium', use: { ...devices['Desktop Chrome'] }, testMatch: '**/tc-*.spec.js' },
{ name: 'firefox', use: { ...devices['Desktop Firefox'] }, testMatch: '**/tc-*.spec.js' },
],
});
Typical TypeScript mistake: if the project is
typescript,NEOgenerates.spec.tsfiles. If you leavetestMatchat.spec.js, Playwright can't find any tests. In TypeScript this isn't a rare case: it happens every time you don't change it.
By app type
app_type |
What to add in projects |
|---|---|
web |
The browser projects (chromium, firefox) with testMatch: '**/tc-*.spec.<ext>' |
api |
An api project with no browser, with testMatch: '**/tc-*.api.spec.<ext>' |
hybrid |
Both: browsers for the tc-*.spec files and api for the tc-*.api.spec files |
UI tests run on Chromium and Firefox through the config's projects. The generated tests do not have their own logic to detect the browser.
4. Authentication
basic_auth and api_token
Fill in the values in .env.tests (the file is created by lippia-neo-bootstrap-stack with empty keys and is kept out of git):
# .env.tests
APP_USER=test-user
APP_PASSWORD=test-password
# if app_type is api or hybrid:
API_BASE_URL=https://api.techstore-staging.com
API_TOKEN_MANAGER=...
API_TOKEN_VIEWER=...
The generated tests read those values with process.env.NOMBRE and never write them as literals.
google_sso
Google login is not automated (Google detects it and blocks it). Instead:
- Just once, you open a visible browser with the project's script (
scripts/save-auth.shornpm run auth:save) and log in by hand. - Playwright saves the session (cookies and
localStorage) to a JSON file. playwright.configpoints to that file withstorageState, and each test starts already logged in.- When the session expires, you run the script again.
⚠️ Check that the session file path matches in both places. The configuration templates use
playwright/.auth/user.json. However,run.shchecks.auth/auth-state.jsonby default, and if it can't find it, it stops withAuth state no encontrado. If your project uses a different path, set theQA_AUTH_STATE_PATHvariable to the real path before running.Never push the session file to git. It contains valid tokens. Add its folder (
.auth/orplaywright/.auth/) to.gitignore.
5. What NEO generates
One file per test case, inside one folder per user story:
tests/playwright/ ← paths.tests_dir
└── US-002/
├── tc-001-agregar-producto-al-carrito.spec.ts
├── tc-002-carrito-vacio.spec.ts
└── tc-003-crear-orden.api.spec.ts ← API test (app_type api / hybrid)
Rules the generated tests follow:
- Selectors, in this order of preference: role + name (
getByRole), label,data-testid, visible text and, as a last resort, CSS class or id. If a selector isn't known, an explicitTODOis left. - Waits based on conditions (
waitForURL,expect(...).toBeVisible()), never fixed pauses likewaitForTimeout(2000). - Imports with
require(CommonJS), unlesspackage.jsonhas"type": "module".
6. How lippia-neo-run-and-heal runs it
lippia-neo-run-and-heal uses the adapter's run.sh, which:
- Checks that
node_modulesexists and that Playwright is installed. If anything is missing, it exits with code2. - If
authisgoogle_sso, checks that the session file exists. - Deletes the result of the previous run, so old results aren't mixed up with new ones.
- Runs
npx playwright test --reporter=json <tests_dir>. - Checks that
test-results/results.jsonwas generated.
Variables you can use to adjust the run:
| Variable | Default | What it's for |
|---|---|---|
QA_TESTS_DIR |
paths.tests_dir |
Folder or folders to run, separated by spaces (e.g. "tests/playwright/US-001 tests/playwright/US-002") |
QA_EXCLUDE_DIRS |
empty | Subfolders to exclude, separated by spaces (e.g. "api-contract-v1") |
PLAYWRIGHT_FLAGS |
empty | Extra flags, for example --project=chromium |
PLAYWRIGHT_CONFIG |
empty | Use a different config file (e.g. playwright.config.ci.ts) |
PLAYWRIGHT_JSON_OUTPUT |
test-results/results.json |
Where the JSON report is saved |
QA_AUTH_STATE_PATH |
.auth/auth-state.json |
Session file that is checked if auth is google_sso |
To run only some folders, use
QA_TESTS_DIRorQA_EXCLUDE_DIRS, which filter by the real path.--grep-invertfilters by the test title, not by folder.
7. Quick checklist
node -v # 18 or higher
npx playwright --version # Playwright installed
ls node_modules > /dev/null && echo "dependencias OK"
grep -n "results.json" playwright.config.* # JSON reporter configured
ls .env.tests # credentials (if auth is basic_auth / api_token)
If everything checks out, ask the assistant: "Run the suite with lippia-neo-run-and-heal".
8. Common problems (and what to do)
| What you see | Why it may happen | What to do |
|---|---|---|
node_modules no encontrado |
The dependencies weren't installed | npm ci (or npm install) |
@playwright/test no encontrado |
Playwright isn't in package.json |
npm install -D @playwright/test |
Playwright no escribió el JSON en 'test-results/results.json' |
The runner failed before finishing (config with errors, folder that doesn't exist) | Check the output that appears above the error |
Auth state no encontrado |
The Google SSO session doesn't exist or is at a different path | Run scripts/save-auth.sh, or set QA_AUTH_STATE_PATH to the real path |
| "No tests found" | testDir or testMatch don't match the generated folder or extension |
Align testDir with paths.tests_dir and testMatch with .js / .ts |
Browser "chrome-for-testing" is not installed while exploring |
The Playwright MCP was updated and expects a different browser version | npx @playwright/mcp@latest install-browser chrome-for-testing (see Playwright MCP setup) |
| Tests that fail with 5xx or network errors | The environment is down; it's not a problem with the test | NEO marks it as environment and doesn't fix it: check the environment |