Performance testing
These five skills cover the full path of a load test: reading an API collection (Postman or Bruno),
generating the load scripts in k6 or JMeter, running them profile by profile (smoke, load,
stress, spike) and writing the final report with the numbers of each run. Each skill does a
single thing and talks to the next one through the shape of its data: flows and authentication,
scripts, run folders, report.
CI mode: no questions
None of these skills ask the person invoking them any questions. Each value is resolved in this
order: explicit input, then the performance block of qa-stack.yaml (together with
environment.base_url and paths.*) and, lastly, a documented default. Whatever is resolved by
default is recorded. Only what is impossible to infer (for example, no collection and no flows)
returns status: needs_input or blocked, with the detail in missing. The performance block is
described in STACK.md of the skills repository.
Limits worth keeping in mind
- The write test creates real data.
lippia-neo-read-api-collectiononly runs reads; the exception is the write test (POST/PUT/PATCH, and DELETE only withinclude_mutations: all), which repeats each write with its body to detect single-use data. It only runs ifinclude_mutationsis notnone,mutation_probe_iterationsis greater than 0 and the probe is enabled; against a non-local host it also requiresperformance.confirm_remote: true. - Nothing gets installed. The runner does not install k6, JMeter or Java: if it cannot find them,
it returns
blocked(k6) orrefused(JMeter) and runs nothing. - No secrets in files. No skill writes credentials, tokens or keys into scripts, reports, logs or
warnings: they are masked (
********) and values arrive through the process environment or through data files that the skills never read or copy. - JMeter does not measure iterations. With JMeter, iterations and everything derived from them
appear as
n/ain the report (not as 0 or as an estimate).
lippia-neo-read-api-collection
When to use it
When there is a Postman or Bruno collection and you need to understand its flows, how it
authenticates (bearer, session cookie, API key, basic or none) and which requests depend on that
authentication, before generating any test. Also to confirm against the environment how the login
responds and what the read requests return with that session, and to know which values a request
stores for the following ones in its flow. It proposes test flows and writes a structured report
(api-collection-<name>.json). It does not generate scripts, does not run loads and does not write
secrets. It tries the login only once and without retries, so the test account does not get locked.
{
"input": {
"collection": "collections/projects-api.postman_collection.json",
"environment": "collections/local.postman_environment.json",
"base_url": "http://localhost:8105",
"probe": "yes",
"include_mutations": "none"
}
}
| Input | Type | Description |
|---|---|---|
| — | Required | No mandatory inputs: whatever is missing is filled in from qa-stack.yaml or by autodetection. |
input.qa_stack_path |
Optional | qa-stack.yaml by default. |
input.collection / input.environment |
Optional | Postman collection (.json) or Bruno folder (with .bru) and its environment. If absent: performance.collection.* from qa-stack.yaml or autodetection. |
input.base_url |
Optional | Overrides the base URL from the environment and from qa-stack.yaml. |
input.probe |
Optional | yes (default) or no. With no, nothing is called. |
input.include_mutations |
Optional | none / writes / all: which data-modifying methods are tested and enter the load. If absent: performance.mutations.include, then writes. |
input.mutation_probe_iterations |
Optional | Integer from 0 to 5: how many times each write is repeated. If absent: performance.mutations.probe_iterations, then 3. |
input.login_request |
Optional | Name or path of a collection request; overrides login detection. |
input.data_files / input.credential_env |
Optional | CSV files with credentials (from performance.data_dir, data/ by default) or, if there are none, environment variables holding the probe credentials. |
input.output_dir |
Optional | Report folder (otherwise: performance.scripts_dir, then paths.reports_dir). |
Output
| Field | Description |
|---|---|
status |
read (collection read, even if not probed) / needs_input (no collection, or there is a login but no source provides credentials; with performance.on_missing: warn the latter stays read with a warning) / blocked (unreadable or unsupported collection, or no qa-stack.yaml with stack.test_framework k6/jmeter). |
collection_name, source_kind, base_url |
Name, origin (postman / bruno) and resolved base URL. |
auth |
A single authentication per collection: mode (bearer, cookie, api_key_header, api_key_query, basic, none) and its data. |
login |
The full login request, with the user and password fields identified (without values), or null if there is none. |
flows |
Flows with their requests: needs_auth, captures, observed_status, expected_statuses, mutates, read_only_post, destructive, plus covers_flows / requires_captures_from per flow. |
probe |
Whether it was probed, and why or why not. |
report_path |
Path of the written report (never overwrites an existing one: a date is added to the name). |
missing / warnings |
What is impossible to infer, and notices (what was masked, conventions applied). |
Chains with
Consumes a collection (with its environment and data CSVs) and delivers the report with auth,
login and flows: exactly the data shape that the skills generating load scripts expect.
lippia-neo-generate-k6-script
When to use it
When there are already structured HTTP flows (with or without authentication) and you need to turn
them into executable k6 load scripts, one per profile (smoke, load, stress, spike), or
regenerate them after changing profiles, thresholds, load mix, authentication, data or flows (they
are rendered again: generated files are not edited by hand). The code is not written by the agent: a
deterministic renderer (same input, same output) generates it and computes the declared load. It does
not parse collections, does not run the load, does not install k6 and does not interpret results.
{
"input": {
"flows_report": "performance/api-collection-projects-api.json",
"profiles": ["smoke", "load"],
"include_mutations": "none"
}
}
| Input | Type | Description |
|---|---|---|
input.flows |
Required | Non-empty array of flows { name, kind: login \| flow, requires_login, requests }; each request carries name, method and url (and optionally headers, body, checks, captures, needs_auth, etc.). At most one kind: login flow. It can be replaced by input.flows_report. |
input.flows_report |
Optional | Path to a JSON { base_url?, auth, login, flows }. If there is neither flows nor flows_report, it looks for the most recent api-collection-*.json with status: read in performance.scripts_dir (or paths.reports_dir). |
input.auth / input.login |
Optional | Authentication (mode and its data) and login request with each field marked as user, pass or literal. Without auth: none. |
input.profiles |
Optional | Subset of smoke, load, stress, spike (otherwise: performance.profiles, then all four). |
input.profile_overrides, input.thresholds, input.mix, input.journeys, input.executor, input.pacing |
Optional | Stages per profile, thresholds, load split across scenarios, business journeys, load model (vus / arrival_rate / sequential) and pause between iterations. |
input.checks, input.unique_fields, input.on_exhaust |
Optional | Per-request validations, body fields that must be unique, and what to do when the CSV rows run out (skip_writes by default, recycle, abort, synthesize). |
input.include_mutations / include_destructive / include_probe_failed / include_write_without_body |
Optional | Which requests enter the load. By default, destructive ones (DELETE), those a previous probe saw fail and those from a host other than the environment's are excluded. |
input.data_files, input.variables, input.max_vus, input.base_url, input.output_dir, input.qa_stack_path |
Optional | Data CSVs (only their columns are described, values are never read), non-sensitive placeholder values, open-model VU cap, base URL, output folder and path to qa-stack.yaml. |
Output
| Field | Description |
|---|---|
status |
generated (rendered and validated) / needs_input (flows missing and no other source) / blocked (no Node, invalid input or broken references, conflict with existing files that carry no marker, or a piece of data impossible to infer). |
files, generation_report |
Files written and path of generation-report.json (warnings, exclusions, decisions, declared load and commands). |
profiles, flows_count |
Profiles generated and scenarios included. |
env_vars, required_env |
Environment variables the scripts read (names only) and those with no default that must be passed to run. |
data_mapping, missing_data |
How each CSV column is used, and data that some flow needs and nobody provides. |
excluded_requests, request_body_missing |
Requests that did not enter the load (with their reason) and writes without a body. |
declared_load |
Declared load per profile, computed by the renderer. In the closed model it is an estimate. |
decisions, defaults_applied |
Decisions taken and where they came from (input, qa-stack or default). |
validate_command, run_command |
Commands to validate without generating load (k6 inspect) and to run one profile. |
warnings |
Notices from the renderer and from the skill. If k6 is not available, the no_validado: k6 no disponible notice comes first. |
Security rules of the generated scripts
Auth headers, sensitive-named fields and URLs with credentials are replaced by placeholders that are
resolved against the environment, with no default value. A session that cannot be established never
gives a green result: with bearer or cookie, a failed preparation login aborts the run.
Chains with
Consumes flows, authentication and login shaped as { base_url?, auth, login, flows }, and delivers
the scripts folder (<profile>.js) together with generation-report.json, which execution and
reporting use to know the declared load and the mandatory variables.
lippia-neo-generate-jmeter-script
When to use it
The same as the k6 skill, but for JMeter: when there are structured HTTP flows and you need to
turn them into executable .jmx load plans, one per profile, or regenerate them after a change. The
XML is written by a deterministic renderer. It validates each .jmx with its own structural check
and, only if JMeter is available, with JMeter. It does not parse collections, does not run the load,
does not install JMeter and does not interpret results.
{
"input": {
"flows_report": "performance/api-collection-projects-api.json",
"profiles": ["smoke", "load"]
}
}
| Input | Type | Description |
|---|---|---|
input.flows |
Required | Same as in k6: non-empty array of flows with their requests (it can be replaced by input.flows_report). |
input.flows_report, input.auth, input.login, input.profiles, input.profile_overrides, input.mix, input.journeys, input.executor, input.checks, input.unique_fields, input.on_exhaust, input.pacing, input.data_files, input.variables, input.max_vus, input.base_url, input.output_dir, input.qa_stack_path |
Optional | Same meaning and same precedence (input, qa-stack.yaml, default) as in the k6 skill. |
input.include_mutations / include_destructive / include_probe_failed / include_write_without_body |
Optional | Same as in k6. |
input.thresholds |
Optional | Thresholds (http_req_failed, http_req_duration, throttled, server_errors, conflicts). checks and dropped_iterations do not exist in JMeter and are ignored with a warning. Thresholds are not plan assertions: they are written to generation-report.json and evaluated against the results. |
Output
Same as in k6 (status, files, generation_report, declared_load, data_mapping,
excluded_requests, decisions, validate_command, run_command, warnings), with these differences:
| Field | Description |
|---|---|
tool |
Always jmeter. |
declared_load |
basis is always estimate: with core JMeter even the open model is not exact. |
validation |
{ structural: ok \| failed \| not_run, jmeter: ok \| failed \| not_available }. The structural check is mandatory; the JMeter one only if it is available. |
status |
blocked also if the structural check fails. |
JMeter-specific limits
JMeter does not split the credentials CSV into separate user pools: if there is a measured login and
flows with a session on the same CSV, credential_pools_not_disjoint is reported and the decision is
left to whoever invokes the skill. Thresholds are not computed inside the plan but against the
results.
Chains with
Same input contract as the k6 skill; it delivers the plans folder (scripts/<profile>.jmx,
user.properties, README) and generation-report.json.
lippia-neo-run-performance-test
When to use it
When there is a directory with k6 scripts (<profile>.js) or JMeter plans (scripts/<profile>.jmx)
per profile and you need to run them against an environment leaving reproducible evidence of each
run, or repeat a run (for example with another time_scale) or run several profiles one at a time
resetting the state of the system under test before each one. Profiles run always one at a time and
in sequence, with a pause between them. It does not modify scripts or data, does not interpret
results, does not install k6 or JMeter and does not execute anything outside the tool and the
indicated reset_command.
{
"input": {
"tool": "k6",
"profiles": ["smoke", "load"],
"base_url": "http://localhost:8105",
"env_var_names": ["LOGIN_USER", "LOGIN_PASSWORD"],
"time_scale": 0.1
}
}
| Input | Type | Description |
|---|---|---|
| — | Required | No mandatory inputs: whatever is missing is filled in from qa-stack.yaml. |
input.tool |
Optional | k6 or jmeter. By default jmeter if stack.test_framework is jmeter; otherwise k6. |
input.jmeter_path |
Optional | Explicit path to the JMeter executable (without it: PATH, JMETER_HOME, standard locations and performance.tool_path). |
input.project_dir, input.script_dir, input.results_dir |
Optional | Project folder, scripts folder (performance.scripts_dir) and runs folder (performance.results_dir; results for k6 and target for JMeter by default). |
input.profiles |
Optional | smoke, load, stress, spike (by default performance.profiles). |
input.base_url |
Optional | Environment base URL (by default environment.base_url). Only its host is reported. |
input.env_var_names, input.env_file, input.required_env |
Optional | NAMES of the variables to inject (the value comes from the process environment or from a file outside the repo) and the script's mandatory variables. |
input.time_scale, input.cooldown_s, input.max_duration |
Optional | Duration scale, pause between profiles and per-profile duration cap. |
input.reset_command / input.reset_args |
Optional | Command that restores the state of the system under test; it runs before each profile. |
input.on_concurrent |
Optional | warn (default) or abort if other processes of the tool are already running on the machine. |
input.confirm_remote |
Optional | Explicit authorization to run stress or spike against a non-local host. |
input.run_id |
Optional | Prefix for each run's identifier. |
Output
| Field | Description |
|---|---|
status |
completed (all profiles were attempted, even if one ended with crossed thresholds) / partial (stopped early: the reset_command failed, or there are concurrent processes with on_concurrent: abort) / refused (mandatory variables missing or JMeter not found; no run was spent) / needs_input (a required input or the confirmation for a remote host is missing) / blocked (no Node, or k6 not installed or older than 0.49). |
tool, tool_version |
Tool used and its version. |
runs |
Per profile: run_dir, exit_code, exit_meaning (ok, thresholds_crossed, aborted, error, skipped, not_run), timed_out, timestamps and which files were left. |
missing / warnings |
What is impossible to infer and notices (concurrent processes, expected files that are missing, scaled run). |
Each run is left in <results_dir>/<profile>-<UTC yyyymmdd-HHmmss>/ with <profile>-load-check.json,
console.utf8.txt, run-meta.json and, depending on the tool, summary.json and dashboard.html (k6)
or results.jtl and dashboard/ (JMeter). With JMeter the load-check is converted from the JTL.
Authorization for remote hosts
If the base_url host is not local and the profiles include stress or spike, it only runs with
confirm_remote: true in the input. Without it, it returns needs_input with
missing: ["confirm_remote"] and launches nothing. An exit code of 99 (thresholds crossed) or 108
(aborted) is run data: it is reported, not retried or covered up.
Chains with
Consumes a scripts folder per profile (with its generation-report.json, from which it gets the
mandatory variables) and delivers run folders with the raw evidence (run-meta.json,
<profile>-load-check.json, tool results) plus runs.json in results_dir.
lippia-neo-write-performance-report
When to use it
When there are one or more run folders and you need a final report readable by a PO or dev lead;
when there are runs of several profiles and you want a comparison table of offered load against
latency to detect an inflection point; or when a run ended with crossed thresholds (99), aborted
(108) or without executing iterations and it must be explained with its measurable causes. A
deterministic script extracts only facts into report-data.json and the report is written using
only those numbers. It does not run tests, does not modify scripts or results and does not invent
metrics. The report is the same with k6 or JMeter.
{
"input": {
"project": "projects-api",
"environment": { "simulated": false, "notes": "shared QA environment" },
"format": "md"
}
}
| Input | Type | Description |
|---|---|---|
| — | Required | No mandatory inputs: whatever is missing is filled in from qa-stack.yaml. |
input.run_dirs |
Optional | Array of run folders. If absent: all runs of the latest invocation in performance.results_dir. |
input.generation_report |
Optional | Path or object with the declared (unscaled) load. By default <performance.scripts_dir>/generation-report.json; if declared_load is missing, it is declared as missing data. |
input.environment |
Optional | { simulated, notes }, declared by whoever requests the test (for example, that the service under test is a simulator). The skill never deduces it: without it, it stays as a missing input. |
input.project |
Optional | Project name used for the file name (by default, the folder of qa-stack.yaml). |
input.reports_dir |
Optional | Output folder (by default paths.reports_dir, then test-results/reports/). |
input.format |
Optional | md (default) or md+html. |
input.qa_stack_path |
Optional | qa-stack.yaml by default. |
Output
| Field | Description |
|---|---|
report_path, report_data_path |
The .md report and its data file, with a unique name per execution (never overwrites an existing one). |
overall_status |
conclusive / conclusive_with_limits (there are limitations that do not block: simulated environment, generator on the same host, scaled run, single run) / non_conclusive (for example an aborted run with no load, session not established, requests that always return 4xx, saturated generator or rate limiting that invalidates the offered load) / blocked (no Node, nonexistent path or no run with data). |
runs, flags |
Runs with their exit code and the flags present. |
bottleneck_declared |
true only if there is cited evidence; otherwise the report states "saturation point not reached" (no se alcanzó el punto de saturación). |
missing_inputs, missing, warnings |
Missing inputs, what is impossible to infer (for example, no runs) and notices (including what was masked). |
How it is written
Every number comes from the data file, with its value and unit; an absent datum is written as "no
data" (sin dato). A 429 is rate control and a 409 a state conflict: neither is presented as a
system failure or as saturation. A load generator problem is never attributed to the system under
test, and an exit 99 or 108 is never covered up. With JMeter, iterations and offered load appear as
n/a and the load is described in served requests per second.
Chains with
Consumes run folders (tool results, <profile>-load-check.json and run-meta.json) and, optionally,
the generation-report.json with the declared load; delivers the final report and its data file in
reports_dir.