Configure JMeter
Adapter status: ✅ Complete (performance). JMeter is a valid value for stack.test_framework: lippia-neo-bootstrap-stack detects it, locates the executable and configures it, and the performance skills generate, run and report the load tests. It is not a functional-suite framework: it does not use lippia-neo-generate-test-suite, lippia-neo-run-and-heal or lippia-neo-bootstrap-ci (see section 9).
Value in qa-stack.yaml |
test_framework: jmeter |
| Language | java (forced) |
| App type | api (forced) |
| What it uses | Apache JMeter 5.6 (core, no plugins) + an API collection (Postman or Bruno) + CSV files with test data |
| Load profiles | smoke, load, stress, spike |
| Configuration | performance block of qa-stack.yaml |
| Scripts folder | performance.scripts_dir (default performance/) |
| Results folder | performance.results_dir (default target/ with JMeter) |
The skills that use it are described in Performance skills. They run without asking questions: everything they need comes from the input or from qa-stack.yaml.
1. What you need installed
| Tool | Version | What for | How to check it |
|---|---|---|---|
| Apache JMeter | 5.x (plans target 5.6, with Groovy included in the standard distribution) | Run the .jmx plans |
jmeter -v |
| Java | 8 or higher | JMeter needs it to start | java -version |
| Node.js | A current version | Run NEO's generator and runner (.js scripts) |
node --version |
NEOdoes not install JMeter or Java. During the bootstrap, the skill looks for the executable without asking you and stores its path inperformance.tool_path. If it doesn't find it, or Java is missing, it leavestool_path: nulland tells you what is missing; it neither stops nor installs anything.
Where the executable is searched for:
- The bootstrap:
jmeterin thePATH,JMETER_HOME/binand theapache-jmeter-*/jmeter*folders inC:\,C:\Program Files,C:\Program Files (x86),C:\tools(Windows) or/opt,/usr/local,/usr/share,/opt/homebrew/Cellar(Linux/macOS), taking the highest version. It confirms it withjmeter -vandjava -version. - The runner, in this order: the path given as input (
jmeter_path), thePATH,JMETER_HOME, the standard paths (C:\apache-jmeter-*\bin,C:\Program Files\apache-jmeter-*\bin,/opt/apache-jmeter-*/bin) and, as a last resort,performance.tool_path.
You also need these project inputs:
| Input | What for | Where it goes |
|---|---|---|
API collection (Postman .json or Bruno folder) |
Endpoints, login and flows come from it | performance.collection.path (or auto-detected) |
| Collection environment (optional) | Non-sensitive variable values | performance.collection.environment_path (or auto-detected next to the collection) |
| CSV with users/credentials and test data | The plans read them at run time | performance.data_dir (default data/) |
| Secrets (tokens, keys) | Passed at run time | Only in paths.env_file (default .env.tests) or process environment variables |
2. Example qa-stack.yaml
stack:
test_framework: jmeter
language: java
app_type: api
pipeline_provider: gitlab
environment:
base_url: https://api-staging.myapp.com
performance:
collection:
path: doc/ # collection file or folder · null = auto-detect
format: auto # auto | postman | bruno
environment_path: null # null = auto-detect next to the collection
data_dir: data/ # CSV with credentials/test data
scripts_dir: performance/ # where plans are generated
results_dir: target/ # where runs are stored with JMeter
profiles: [smoke, load, stress, spike]
time_scale: 1 # duration multiplier (quick run: 0.05)
cooldown_s: null # pause between profiles · null = 30 s x time_scale (min. 1 s)
mutations:
include: writes # none | writes | all
probe_iterations: 3 # 0 to 5
tool_path: C:/jmeter/bin/jmeter.bat # written by the bootstrap · null = PATH / JMETER_HOME
on_missing: block # block | warn
paths:
env_file: .env.tests
reports_dir: test-results/reports/
Every performance key has a default. The block stores only paths and non-sensitive parameters, never credentials. tool_path is written with / slashes to avoid escapes in the YAML.
If
results_diris not in yourqa-stack.yaml, the runner usestarget/with JMeter (with k6 it would useresults/). The bootstrap writes the template default (results/) when it isn't asked: if you wanttarget/, set it by hand.
Other optional keys in the block, all null by default: defaults.login_weight, defaults.on_exhaust (skip_writes | recycle | abort | synthesize), defaults.think_seconds and defaults.mix.
With JMeter,
lippia-neo-bootstrap-stackonly asks for the framework and CI (if it can't detect them),base_urland theperformanceblock. It does not ask about authentication, operating system, version, user stories, bugs or test management tool, and it does not create.mcp.json.
3. How it gets configured (lippia-neo-bootstrap-stack)
Ask the assistant:
Set up the project for QA with lippia-neo-bootstrap-stack
The skill detects JMeter if the repo has *.jmx files, and forces app_type: api and language: java. It also looks for collections (*.postman_collection.json or Bruno folders with bruno.json / *.bru) under doc/, docs/, collections/, postman/, api/ or the root, and offers them to you. It asks one at a time:
| Question | Key | Options |
|---|---|---|
| API collection | performance.collection.path |
The detected candidates, another path, or null (auto-detect on use) |
| Data folder | performance.data_dir |
data/ (default) or another |
| Scripts folder | performance.scripts_dir |
performance/ (default) or another |
| Profiles | performance.profiles |
All 4 (default) or a subset |
| Duration | performance.time_scale |
1 (default, real duration), 0.05 (short run, only to check that everything works) or another number |
| Methods that modify data | performance.mutations.include |
none (reads only), writes (default: POST/PUT/PATCH) or all (includes DELETE) |
| Write pre-test | performance.mutations.probe_iterations |
3 repetitions (default), 1 to 5, or 0 to skip |
cooldown_s, defaults.* and on_missing are written with their defaults without asking. paths.env_file stays as .env.tests (the file is not created) and is added to .gitignore, together with .claude and performance.results_dir. At the end, the summary shows the detected JMeter path and version (or "not found" with the warning).
4. Credentials and data
- Secrets are never written into any plan or into
qa-stack.yaml. The plans read them with${__P(NAME,${__env(NAME)})}: as a parameter or as an environment variable. - Test usernames and passwords go in CSV files inside
performance.data_dir.NEOonly reads the header, the separator and the row count; it never copies or shows the values. The plans read the CSV files from that folder, without copying them. - Other secrets (token, API key, etc.) go in
paths.env_file(default.env.tests,NAME=valueformat, kept out of git) or as process environment variables. The runner hands them to JMeter through the process environment and masks them in the output. - The value of a
basicauthentication variable is passed as a rawuser:password(the plan base64-encodes it).
# .env.tests (kept out of git)
API_KEY=<TOKEN>
Supported authentication modes: bearer, cookie, api_key_header, api_key_query, basic and none. The login request comes from the collection. The auto CSV separator cannot be resolved in JMeter: , is used, with a warning.
5. What NEO generates
lippia-neo-read-api-collection reads the collection, tests the login (a single attempt) and the read requests, and proposes the flows. Then lippia-neo-generate-jmeter-script turns them into plans. The XML is written by a deterministic renderer: the same input always produces the same output, and generated files are not edited by hand (you regenerate). Generation validates the structure of each .jmx without needing JMeter; if JMeter isn't available, the output carries no_validado: jmeter no disponible.
performance/ ← performance.scripts_dir
├── scripts/
│ └── smoke.jmx load.jmx stress.jmx spike.jmx # one plan per profile
├── config/
│ └── user.properties # base_url, time_scale, data_dir and non-sensitive defaults (never secrets)
├── render-input.json
├── generation-report.json # warnings, excluded items, decisions, declared load, thresholds, commands
└── README.md # how to validate and run, parameters, limits
Each plan starts with a preparation group that checks the required parameters (without printing values), followed by one or more Thread Groups per scenario. Each request is labeled <flow>/<request>; the measured login is login/<name> and the preparation session is setup/<name>.
Default profiles
| Profile | Default load | How it is built in JMeter |
|---|---|---|
smoke |
1 user for 30 s, one scenario at a time | One Thread Group per flow with 1 thread, each in its own window |
load |
Ramps to 10 in 1 min, holds 5 min, ramps down to 0 in 1 min | Threads with an initial ramp; the remaining stages are steps |
stress |
Steps of 10, 20, 30 and 40 (2 min each), then down to 0 | Constant rate per stage with a Constant Throughput Timer |
spike |
2 → 50 in 15 s, holds 1 min, back to 2, then down to 0 | Constant rate per stage with a Constant Throughput Timer |
Each stage value is the total across all scenarios, split according to the mix (mix, even by default). time_scale multiplies the durations (minimum 1 s) and does not change the plans, only the run.
What is included in the load and what is left out
- Writes (POST/PUT/PATCH) are included according to
mutations.include; DELETE only withall. A POST that is a read-only query is treated as a read. - Left out (with the reason in
generation-report.jsonand in the warnings): destructive requests, requests to another host, requests that failed in the pre-test, requests that depend on captured values that no chain provides, or writes without a body. - A session that cannot be established never turns green: if the preparation login fails, the run stops.
Write pre-test
If mutations.include is not none and probe_iterations is greater than 0, lippia-neo-read-api-collection repeats each write (up to 5 times, with its body) to detect single-use data. That test creates real data in the environment. Against a non-local host it only runs with performance.confirm_remote: true. It does not test the login or requests that depend on captured values, and it only sends JSON bodies.
6. How lippia-neo-run-performance-test runs it
The runner (scripts/run-profile.js) runs the profiles one at a time, in sequence, never in parallel, with a pause between them (cooldown_s; if null, 30 s x time_scale, minimum 1 s). Before launching anything it validates the plans' required variables. If one is missing, it refuses the run (refused, code 2) without launching JMeter. If it can't find JMeter (or it doesn't answer jmeter -v, for example because Java is missing), it also refuses (refused, code 3), without asking or downloading anything.
For each profile it launches JMeter in non-GUI mode and stores the run in <results_dir>/<profile>-<UTC yyyymmdd-HHmmss>/:
target/
├── runs.json
└── load-20260105-143000/
├── results.jtl
├── dashboard/ # JMeter HTML dashboard (index.html)
├── jmeter.log
├── load-load-check.json # summary converted from the JTL: statuses, latencies, thresholds
├── console.utf8.txt # output, with sensitive values masked
└── run-meta.json
JMeter exits with code 0 even when there are failed samples. That's why the runner converts results.jtl into <profile>-load-check.json (the same shape as with k6), evaluates the thresholds there and classifies 99 (thresholds crossed) if any was crossed. A non-zero JMeter exit is error. The default thresholds (error per request, p95) are not in the plan: they are evaluated when the JTL is converted.
Runner options (taken from the input or from qa-stack.yaml; the input wins): profiles, base_url, time_scale, cooldown_s, script_dir, results_dir, reset_command (resets the system state before each profile; if it fails, the run stops), max_duration and confirm_remote. They work the same as with k6 (see Configure k6); in addition, jmeter_path gives an explicit path to the executable. performance.data_dir is passed to JMeter as an absolute path.
Careful with
stressandspike. Against a host that is not local they only run if the caller passesconfirm_remote: true. Without it, the runner returnsneeds_inputand launches nothing.
To write the final report (performance-report-<project>-<UTC>.md in paths.reports_dir) there is lippia-neo-write-performance-report; see Performance skills.
7. Quick checklist
jmeter -v # JMeter 5.x
java -version # Java 8 or higher
node --version
ls doc/ # the collection (or the performance.collection.path value)
ls data/*.csv # user / data CSV files
ls .env.tests # secrets, if the plans ask for them
grep -n "tool_path" qa-stack.yaml
If everything responds, ask the assistant to read the collection, generate the plans and run the profiles.
8. Common problems (and what to do)
| What you see | Why it may happen | What to do |
|---|---|---|
Status refused with code 3 |
JMeter was not found or doesn't answer jmeter -v (for example, Java is missing) |
Install JMeter and Java outside NEO, or give the path with jmeter_path / performance.tool_path |
tool_path: null and a warning after the bootstrap |
JMeter or Java was not found | Install them and run the bootstrap again, or write tool_path by hand |
Status refused (code 2), profiles not_run |
A required plan variable is missing | Define it in paths.env_file or in the process environment; the message names which ones are missing |
Status needs_input with confirm_remote in missing |
stress or spike was requested against a remote host without authorization |
Pass confirm_remote: true only when appropriate |
Status needs_input with flows in missing |
There is no collection read report and no input flows | Run lippia-neo-read-api-collection first |
blocked because of Node |
node --version fails |
Install Node.js |
Every request returns 4xx with basic auth |
The value was passed already encoded | Pass raw user:password |
Exit 99 on a profile |
A threshold was crossed | It is a test result: check load-check.json or the report |
Warning credential_pools_not_disjoint |
The measured login and the sessions share users | Use a separate credentials CSV or measure_login: false |
The generator returns conflict |
A file in scripts_dir exists and was not generated by NEO |
Move or rename that file |
9. Current limitations
| What | What happens with JMeter | Alternative |
|---|---|---|
CI jobs (lippia-neo-bootstrap-ci) |
It does not generate a performance job: it stops with framework_not_supported_by_this_skill |
Build the job by hand and run the lippia-neo-run-performance-test runner; in CI the secrets come from the environment |
Functional-suite skills (lippia-neo-generate-test-suite, lippia-neo-run-and-heal) |
They abort with framework_not_supported_by_this_skill |
Use the performance skills |
| Iterations | JMeter does not measure them: iterations, dropped iterations, recycled/skipped data and missing captures show as n/d in the report |
Read the load in terms of requests and sample rate |
| Ramps | A Thread Group only ramps from 0 to N: descending ramps and ramps between non-zero values are steps at the stage average, and every new thread repeats its login | Use k6 if you need gradual ramps (JMeter core, no plugins) |
Open model (stress, spike) |
The rate is approximated with a Constant Throughput Timer and a thread cap; if the system saturates, the achieved rate drops with no dropped-iteration warning | Compare against the achieved load in the report |
| Credential pools | The measured login and the per-thread sessions share users (the CSV is not split) | Separate credentials CSV or measure_login: false |
| Session renewal | Only on expiry (expires_in_seconds), not on a 401; when renewing, a different pool user may be picked. The session cookie is not verified, only the login status |
Set a correct expires_in_seconds in the authentication |
| Scenario selection at run time | There is no FLOWS with JMeter |
Regenerate with a different mix or a shorter list of flows |
checks and dropped_iterations thresholds |
They don't exist in JMeter and are ignored with a warning | — |
formdata bodies |
Sent urlencoded, without files | — |
| Write test | It creates real data in the environment and only sends JSON bodies; it does not test requests that depend on captured values | mutations.include: none or probe_iterations: 0 |
qa-stack.yaml parser |
Minimal reader: no anchors, multiline values or lists of maps | Write the performance block with simple maps and lists |