Skip to content

Configure JMeter

Adapter status: ✅ Complete (performance). JMeter is a valid value for stack.test_framework: lippia-neo-bootstrap-stack detects it, locates the executable and configures it, and the performance skills generate, run and report the load tests. It is not a functional-suite framework: it does not use lippia-neo-generate-test-suite, lippia-neo-run-and-heal or lippia-neo-bootstrap-ci (see section 9).

Value in qa-stack.yaml test_framework: jmeter
Language java (forced)
App type api (forced)
What it uses Apache JMeter 5.6 (core, no plugins) + an API collection (Postman or Bruno) + CSV files with test data
Load profiles smoke, load, stress, spike
Configuration performance block of qa-stack.yaml
Scripts folder performance.scripts_dir (default performance/)
Results folder performance.results_dir (default target/ with JMeter)

The skills that use it are described in Performance skills. They run without asking questions: everything they need comes from the input or from qa-stack.yaml.


1. What you need installed

Tool Version What for How to check it
Apache JMeter 5.x (plans target 5.6, with Groovy included in the standard distribution) Run the .jmx plans jmeter -v
Java 8 or higher JMeter needs it to start java -version
Node.js A current version Run NEO's generator and runner (.js scripts) node --version

NEO does not install JMeter or Java. During the bootstrap, the skill looks for the executable without asking you and stores its path in performance.tool_path. If it doesn't find it, or Java is missing, it leaves tool_path: null and tells you what is missing; it neither stops nor installs anything.

Where the executable is searched for:

  • The bootstrap: jmeter in the PATH, JMETER_HOME/bin and the apache-jmeter-* / jmeter* folders in C:\, C:\Program Files, C:\Program Files (x86), C:\tools (Windows) or /opt, /usr/local, /usr/share, /opt/homebrew/Cellar (Linux/macOS), taking the highest version. It confirms it with jmeter -v and java -version.
  • The runner, in this order: the path given as input (jmeter_path), the PATH, JMETER_HOME, the standard paths (C:\apache-jmeter-*\bin, C:\Program Files\apache-jmeter-*\bin, /opt/apache-jmeter-*/bin) and, as a last resort, performance.tool_path.

You also need these project inputs:

Input What for Where it goes
API collection (Postman .json or Bruno folder) Endpoints, login and flows come from it performance.collection.path (or auto-detected)
Collection environment (optional) Non-sensitive variable values performance.collection.environment_path (or auto-detected next to the collection)
CSV with users/credentials and test data The plans read them at run time performance.data_dir (default data/)
Secrets (tokens, keys) Passed at run time Only in paths.env_file (default .env.tests) or process environment variables

2. Example qa-stack.yaml

stack:
  test_framework: jmeter
  language: java
  app_type: api
  pipeline_provider: gitlab

environment:
  base_url: https://api-staging.myapp.com

performance:
  collection:
    path: doc/                  # collection file or folder · null = auto-detect
    format: auto                # auto | postman | bruno
    environment_path: null      # null = auto-detect next to the collection
  data_dir: data/               # CSV with credentials/test data
  scripts_dir: performance/     # where plans are generated
  results_dir: target/          # where runs are stored with JMeter
  profiles: [smoke, load, stress, spike]
  time_scale: 1                 # duration multiplier (quick run: 0.05)
  cooldown_s: null              # pause between profiles · null = 30 s x time_scale (min. 1 s)
  mutations:
    include: writes             # none | writes | all
    probe_iterations: 3         # 0 to 5
  tool_path: C:/jmeter/bin/jmeter.bat   # written by the bootstrap · null = PATH / JMETER_HOME
  on_missing: block             # block | warn

paths:
  env_file: .env.tests
  reports_dir: test-results/reports/

Every performance key has a default. The block stores only paths and non-sensitive parameters, never credentials. tool_path is written with / slashes to avoid escapes in the YAML.

If results_dir is not in your qa-stack.yaml, the runner uses target/ with JMeter (with k6 it would use results/). The bootstrap writes the template default (results/) when it isn't asked: if you want target/, set it by hand.

Other optional keys in the block, all null by default: defaults.login_weight, defaults.on_exhaust (skip_writes | recycle | abort | synthesize), defaults.think_seconds and defaults.mix.

With JMeter, lippia-neo-bootstrap-stack only asks for the framework and CI (if it can't detect them), base_url and the performance block. It does not ask about authentication, operating system, version, user stories, bugs or test management tool, and it does not create .mcp.json.


3. How it gets configured (lippia-neo-bootstrap-stack)

Ask the assistant:

Set up the project for QA with lippia-neo-bootstrap-stack

The skill detects JMeter if the repo has *.jmx files, and forces app_type: api and language: java. It also looks for collections (*.postman_collection.json or Bruno folders with bruno.json / *.bru) under doc/, docs/, collections/, postman/, api/ or the root, and offers them to you. It asks one at a time:

Question Key Options
API collection performance.collection.path The detected candidates, another path, or null (auto-detect on use)
Data folder performance.data_dir data/ (default) or another
Scripts folder performance.scripts_dir performance/ (default) or another
Profiles performance.profiles All 4 (default) or a subset
Duration performance.time_scale 1 (default, real duration), 0.05 (short run, only to check that everything works) or another number
Methods that modify data performance.mutations.include none (reads only), writes (default: POST/PUT/PATCH) or all (includes DELETE)
Write pre-test performance.mutations.probe_iterations 3 repetitions (default), 1 to 5, or 0 to skip

cooldown_s, defaults.* and on_missing are written with their defaults without asking. paths.env_file stays as .env.tests (the file is not created) and is added to .gitignore, together with .claude and performance.results_dir. At the end, the summary shows the detected JMeter path and version (or "not found" with the warning).


4. Credentials and data

  • Secrets are never written into any plan or into qa-stack.yaml. The plans read them with ${__P(NAME,${__env(NAME)})}: as a parameter or as an environment variable.
  • Test usernames and passwords go in CSV files inside performance.data_dir. NEO only reads the header, the separator and the row count; it never copies or shows the values. The plans read the CSV files from that folder, without copying them.
  • Other secrets (token, API key, etc.) go in paths.env_file (default .env.tests, NAME=value format, kept out of git) or as process environment variables. The runner hands them to JMeter through the process environment and masks them in the output.
  • The value of a basic authentication variable is passed as a raw user:password (the plan base64-encodes it).
# .env.tests (kept out of git)
API_KEY=<TOKEN>

Supported authentication modes: bearer, cookie, api_key_header, api_key_query, basic and none. The login request comes from the collection. The auto CSV separator cannot be resolved in JMeter: , is used, with a warning.


5. What NEO generates

lippia-neo-read-api-collection reads the collection, tests the login (a single attempt) and the read requests, and proposes the flows. Then lippia-neo-generate-jmeter-script turns them into plans. The XML is written by a deterministic renderer: the same input always produces the same output, and generated files are not edited by hand (you regenerate). Generation validates the structure of each .jmx without needing JMeter; if JMeter isn't available, the output carries no_validado: jmeter no disponible.

performance/                   ← performance.scripts_dir
├── scripts/
│   └── smoke.jmx  load.jmx  stress.jmx  spike.jmx   # one plan per profile
├── config/
│   └── user.properties        # base_url, time_scale, data_dir and non-sensitive defaults (never secrets)
├── render-input.json
├── generation-report.json     # warnings, excluded items, decisions, declared load, thresholds, commands
└── README.md                  # how to validate and run, parameters, limits

Each plan starts with a preparation group that checks the required parameters (without printing values), followed by one or more Thread Groups per scenario. Each request is labeled <flow>/<request>; the measured login is login/<name> and the preparation session is setup/<name>.

Default profiles

Profile Default load How it is built in JMeter
smoke 1 user for 30 s, one scenario at a time One Thread Group per flow with 1 thread, each in its own window
load Ramps to 10 in 1 min, holds 5 min, ramps down to 0 in 1 min Threads with an initial ramp; the remaining stages are steps
stress Steps of 10, 20, 30 and 40 (2 min each), then down to 0 Constant rate per stage with a Constant Throughput Timer
spike 2 → 50 in 15 s, holds 1 min, back to 2, then down to 0 Constant rate per stage with a Constant Throughput Timer

Each stage value is the total across all scenarios, split according to the mix (mix, even by default). time_scale multiplies the durations (minimum 1 s) and does not change the plans, only the run.

What is included in the load and what is left out

  • Writes (POST/PUT/PATCH) are included according to mutations.include; DELETE only with all. A POST that is a read-only query is treated as a read.
  • Left out (with the reason in generation-report.json and in the warnings): destructive requests, requests to another host, requests that failed in the pre-test, requests that depend on captured values that no chain provides, or writes without a body.
  • A session that cannot be established never turns green: if the preparation login fails, the run stops.

Write pre-test

If mutations.include is not none and probe_iterations is greater than 0, lippia-neo-read-api-collection repeats each write (up to 5 times, with its body) to detect single-use data. That test creates real data in the environment. Against a non-local host it only runs with performance.confirm_remote: true. It does not test the login or requests that depend on captured values, and it only sends JSON bodies.


6. How lippia-neo-run-performance-test runs it

The runner (scripts/run-profile.js) runs the profiles one at a time, in sequence, never in parallel, with a pause between them (cooldown_s; if null, 30 s x time_scale, minimum 1 s). Before launching anything it validates the plans' required variables. If one is missing, it refuses the run (refused, code 2) without launching JMeter. If it can't find JMeter (or it doesn't answer jmeter -v, for example because Java is missing), it also refuses (refused, code 3), without asking or downloading anything.

For each profile it launches JMeter in non-GUI mode and stores the run in <results_dir>/<profile>-<UTC yyyymmdd-HHmmss>/:

target/
├── runs.json
└── load-20260105-143000/
    ├── results.jtl
    ├── dashboard/               # JMeter HTML dashboard (index.html)
    ├── jmeter.log
    ├── load-load-check.json     # summary converted from the JTL: statuses, latencies, thresholds
    ├── console.utf8.txt         # output, with sensitive values masked
    └── run-meta.json

JMeter exits with code 0 even when there are failed samples. That's why the runner converts results.jtl into <profile>-load-check.json (the same shape as with k6), evaluates the thresholds there and classifies 99 (thresholds crossed) if any was crossed. A non-zero JMeter exit is error. The default thresholds (error per request, p95) are not in the plan: they are evaluated when the JTL is converted.

Runner options (taken from the input or from qa-stack.yaml; the input wins): profiles, base_url, time_scale, cooldown_s, script_dir, results_dir, reset_command (resets the system state before each profile; if it fails, the run stops), max_duration and confirm_remote. They work the same as with k6 (see Configure k6); in addition, jmeter_path gives an explicit path to the executable. performance.data_dir is passed to JMeter as an absolute path.

Careful with stress and spike. Against a host that is not local they only run if the caller passes confirm_remote: true. Without it, the runner returns needs_input and launches nothing.

To write the final report (performance-report-<project>-<UTC>.md in paths.reports_dir) there is lippia-neo-write-performance-report; see Performance skills.


7. Quick checklist

jmeter -v                     # JMeter 5.x
java -version                 # Java 8 or higher
node --version
ls doc/                       # the collection (or the performance.collection.path value)
ls data/*.csv                 # user / data CSV files
ls .env.tests                 # secrets, if the plans ask for them
grep -n "tool_path" qa-stack.yaml

If everything responds, ask the assistant to read the collection, generate the plans and run the profiles.


8. Common problems (and what to do)

What you see Why it may happen What to do
Status refused with code 3 JMeter was not found or doesn't answer jmeter -v (for example, Java is missing) Install JMeter and Java outside NEO, or give the path with jmeter_path / performance.tool_path
tool_path: null and a warning after the bootstrap JMeter or Java was not found Install them and run the bootstrap again, or write tool_path by hand
Status refused (code 2), profiles not_run A required plan variable is missing Define it in paths.env_file or in the process environment; the message names which ones are missing
Status needs_input with confirm_remote in missing stress or spike was requested against a remote host without authorization Pass confirm_remote: true only when appropriate
Status needs_input with flows in missing There is no collection read report and no input flows Run lippia-neo-read-api-collection first
blocked because of Node node --version fails Install Node.js
Every request returns 4xx with basic auth The value was passed already encoded Pass raw user:password
Exit 99 on a profile A threshold was crossed It is a test result: check load-check.json or the report
Warning credential_pools_not_disjoint The measured login and the sessions share users Use a separate credentials CSV or measure_login: false
The generator returns conflict A file in scripts_dir exists and was not generated by NEO Move or rename that file

9. Current limitations

What What happens with JMeter Alternative
CI jobs (lippia-neo-bootstrap-ci) It does not generate a performance job: it stops with framework_not_supported_by_this_skill Build the job by hand and run the lippia-neo-run-performance-test runner; in CI the secrets come from the environment
Functional-suite skills (lippia-neo-generate-test-suite, lippia-neo-run-and-heal) They abort with framework_not_supported_by_this_skill Use the performance skills
Iterations JMeter does not measure them: iterations, dropped iterations, recycled/skipped data and missing captures show as n/d in the report Read the load in terms of requests and sample rate
Ramps A Thread Group only ramps from 0 to N: descending ramps and ramps between non-zero values are steps at the stage average, and every new thread repeats its login Use k6 if you need gradual ramps (JMeter core, no plugins)
Open model (stress, spike) The rate is approximated with a Constant Throughput Timer and a thread cap; if the system saturates, the achieved rate drops with no dropped-iteration warning Compare against the achieved load in the report
Credential pools The measured login and the per-thread sessions share users (the CSV is not split) Separate credentials CSV or measure_login: false
Session renewal Only on expiry (expires_in_seconds), not on a 401; when renewing, a different pool user may be picked. The session cookie is not verified, only the login status Set a correct expires_in_seconds in the authentication
Scenario selection at run time There is no FLOWS with JMeter Regenerate with a different mix or a shorter list of flows
checks and dropped_iterations thresholds They don't exist in JMeter and are ignored with a warning —
formdata bodies Sent urlencoded, without files —
Write test It creates real data in the environment and only sends JSON bodies; it does not test requests that depend on captured values mutations.include: none or probe_iterations: 0
qa-stack.yaml parser Minimal reader: no anchors, multiline values or lists of maps Write the performance block with simple maps and lists