Skip to content

UI Selector Repair

This guide explains what the NEO engine does when it runs this plan. The idea is that you can read it, understand what happens at each step without anyone stepping in, and know what to expect when a frontend change reaches the pipeline.

In short

  • What it's for: when a frontend change renames buttons, fields, or screens, automatically fixing the tests that used them, before they start failing.
  • When it runs: when a frontend code change reaches the pipeline, before deploying it.
  • What it needs: the change's "diff" (the list of code lines that changed) and the project's qa-stack.yaml.
  • What it leaves: the affected tests fixed and run, plus the list of what couldn't be fixed on its own, for a person to review.
  • Steps: 3 — find the affected tests, fix their selectors, and run them. No pauses.
What is a plan? (if this is the first time you see one)

A plan is a recipe that a machine runs instead of a person. Like a playbook, it chains several skills in a fixed order. The difference is that a playbook is followed by the AI assistant together with you, with pauses for you to decide; a plan is run by the NEO engine on its own, end to end, usually from the CI pipeline.

See Plans for the full comparison between plan and playbook.

What is a selector?

An automated test needs to find on the screen the button or field it's going to use. To do that it uses a selector: an "address" for the element, such as its identifier or its text (for example, data-test="checkout-btn"). If the developer renames that identifier, the test can no longer find the button and fails, even though the application works perfectly. This plan fixes those addresses.


1. What does this plan do?

It keeps a design change from breaking the automated tests. When the frontend team renames identifiers on the screen, the plan reads exactly what changed in the code and updates the tests that used them, without waiting for them to fail.

Its intent is to separate two kinds of failure that are easy to mix up: "the application has a bug" and "the test is out of date". This plan only deals with the second one, and it does so with the most reliable information there is: the change's own code. To do that:

  1. It finds which tests touch the part of the application that changed.
  2. In those tests, it fixes only the selectors whose replacement is clear in the change's code. Anything ambiguous it doesn't guess: it leaves it for a person to review.
  3. It runs the tests it fixed, to confirm they work.

Since it works by reading the change's code, it doesn't need the application to be deployed to make the fixes: it's meant to run before the deploy.


2. When does it run?

Situation Business example
A frontend change reaches the pipeline The TechStore team pushes a change to the cart: the "Checkout" button gets a new identifier
You want to keep tests from failing because of a rename TechStore's QA lead doesn't want to see the suite go red every time the design team reorganizes a screen

It runs on every frontend change, big or small: there's no earlier step deciding whether it's worth it. The lippia-neo-ci-bootstrap CI templates include a ui-selector-repair job that runs it on every push to develop that modifies files in src/components/ or src/pages/.

Important

  • The plan only fixes selectors. It doesn't generate new tests for new features or explore the application: that's what the Frontend UI Change PR playbook is for.
  • It only fixes what the change's code makes clear: if an identifier was deleted with no replacement, or there's more than one possible candidate, it leaves it untouched and reports it.
  • If the project doesn't have a qa-stack.yaml yet, you first need to run the project bootstrap playbook.

3. How does it differ from the playbook?

The plan is derived from the Frontend UI Change PR playbook, but it keeps only the part about repairing the suite:

Frontend UI Change PR playbook This plan
Who runs it The AI assistant, talking with you The engine, on its own
Where it learns what changed From the ticket and from walking through the deployed screen From the change's code (the diff)
Needs the app deployed Yes, to walk through it Not to make the fixes. It only uses it when running the fixed tests
Tests for new things Generates them, if you decide there's something new Doesn't generate new tests
Pauses Two: deciding whether there's something new and reviewing whether it repaired too much None: anything doubtful is left untouched and reported
Report Final report with evidence Doesn't write a report; the result is in result.json and, in CI, in an MR with the changes

4. Guiding example: "TechStore" renames the checkout button

We continue with the Frontend UI Change PR playbook's case. The TechStore team reorganized the cart: they moved the "Checkout" button and, along the way, changed its identifier from data-test="checkout-btn" to data-test="finish-purchase-btn". There are 3 automated checkout tests that use the old identifier.

4.1 What the plan needs to receive

Data What it is TechStore example
commit_id The identifier of the change's commit. It's used to name the folder where the results end up a1b2c3d
code_diff_path The path to the file with the change's diff (the output of git diff) diffs/a1b2c3d.diff

In CI, the job generates the diff and builds this data on its own.

4.2 How it's triggered

In CI, the job computes the push's diff, runs the plan and, if something was fixed, opens an MR with the fixed tests:

rules:
  - if: '$CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH == "develop"'
    changes:
      - src/components/**/*
      - src/pages/**/*

To try it from your computer:

git diff develop > diffs/a1b2c3d.diff
neo run plans/ui-selector-repair.yaml --workspace . --out result.json \
  --data '{"commit_id":"a1b2c3d","code_diff_path":"diffs/a1b2c3d.diff"}'

4.3 Step by step

Step 1 of 3 — Find the tests affected by the change

Skill used: lippia-neo-find-ui-tests-by-code-diff

What happens: it reads the diff and pulls out what changed: files, added or removed selectors, routes, classes, and visible text. Then it searches the tests folder for the ones that use any of that. It only reads: it doesn't run or modify anything.

With TechStore: it finds the 3 checkout tests that use checkout-btn.

A test matching doesn't mean it's broken

A match means "this test touches the part that changed". If no test matches, the following steps are skipped: there's nothing to fix.


Step 2 of 3 — Fix the selectors

Skill used: lippia-neo-repair-selectors-by-diff

Only runs if Step 1 found tests

What happens: in the tests Step 1 found, it replaces only the selectors whose change is clear in the diff ("it used to be this, now it's this"). It doesn't open the browser or run anything: it's a fix on the files.

Anything that isn't clear (an identifier deleted with no replacement, or with more than one candidate) isn't guessed: it goes into a list of pending items for a person to review.

With TechStore: it replaces checkout-btn with finish-purchase-btn in the 3 tests.


Step 3 of 3 — Run the fixed tests

Skill used: lippia-neo-run-and-heal

Only runs if Step 2 fixed some test

What happens: it runs only the tests fixed in Step 2, never the whole suite, to confirm they work. If one fails for another minor reason, it tries to repair it and runs it again, up to 2 times.

With TechStore: it runs the 3 checkout tests with the new selector and confirms they pass.


4.4 How does it end?

How it ended What it means What to do
✅ Tests were fixed and they pass The frontend change no longer breaks the suite In CI, review and approve the MR with the fixed tests
⚠️ Some selectors are pending The diff didn't make the replacement clear for some selectors Check the pending list in result.json and fix them by hand
➖ No test matched the change The change doesn't touch anything the tests use Nothing: the plan ends with no changes
❌ A fixed test keeps failing The problem wasn't just the selector Check whether the screen changed its behavior, or whether it's a bug

In all these cases the plan ends in success: the details of what was found, what was fixed, and what's pending are in result.json.


5. Common problems (and what to do)

What you see Why it can happen What to do
The plan found no tests, but some tests are failing The tests use selectors that don't appear in the diff (for example, a text that changed in another file) Review those tests by hand, or with the Frontend UI Change PR playbook
Many selectors were left pending The change deleted elements or replaced them with several That's expected: the plan doesn't guess. Fix them by hand
Step 3 fails to open the app The application isn't reachable at the URL in qa-stack.yaml Confirm the URL; Step 2's fixes are still made
The diff is empty The job couldn't compute the push's diff Check that the job has the full git history (GIT_DEPTH: "0")

6. Visual summary of the flow

flowchart TD
    A([🎨 Frontend change<br/>e.g.: TechStore renames the checkout button]) --> B["<b>Step 1</b> — Find the tests<br/>affected by the diff"]
    B --> C{Did it find<br/>tests?}
    C -->|No| X([➖ Nothing to fix])
    C -->|Yes| D["<b>Step 2</b> — Fix the selectors<br/>with a clear replacement"]
    D --> E{Did it fix<br/>any?}
    E -->|No| P([⚠️ Only pending items to review])
    E -->|Yes| F["<b>Step 3</b> — Run the fixed tests"]
    F --> G([✅ Tests up to date + pending list])

Original plan

The plan file is plans/ui-selector-repair.yaml, in the engine repository (crowdar-engine-mvp). The format reference is in Anatomy of a plan.