Skip to content

Orchestration models in the engine

The 4 orchestration models explains, without technical jargon, who decides the flow and who runs it in each model. This guide brings that concept down to the engine: how Models 4 and 2 work under the hood, since those are the two the engine runs.


1. The 4 models at a glance

Model Who decides which plan runs? Who runs the steps? Does it use the engine?
1 — Improvised The AI assistant The AI assistant No
2 — Adaptive hybrid A classifier skill, choosing among plans already written The engine Yes: neo run --input "<texto>"
3 — Human-in-the-loop A person, with a playbook The AI assistant, step by step No
4 — Pure declarative A fixed rule (for example, from CI) The engine Yes: neo run <plan.yaml>

In Models 2 and 4, the flow is never improvised by the AI. The only difference between the two is how the plan is chosen: in Model 4 it is chosen by a rule written in advance; in Model 2, by a skill that reads the request.


2. Model 4 — Pure declarative

2.1 How it works

flowchart LR
    T[Fixed event<br/>MR with the feature label] --> CI[CI rule<br/>rules / if]
    CI --> P[neo run plans/feature-new-end-to-end.yaml]
    P --> S1[Step 1] --> S2[Step 2] --> SN[Step N] --> R[Result]
  1. Something specific happens in the repository: an MR is opened with the feature label, a push is made to main, it's time for a nightly run.
  2. The CI rule (GitLab's rules: or GitHub's if:) decides which plan applies. That decision lives in the pipeline, not in the plan or the engine.
  3. The engine receives the already-chosen plan and runs it step by step, always the same way.

2.2 When to use it

  • CI/CD pipelines that have to react on their own to a technical event.
  • Any case where the right plan is known in advance and there's no need to "understand" a request.

2.3 Example

The feature-new-end-to-end plan runs the complete playbook for a new feature (read story → test plan → exploration → suite → run and heal → report) every time an MR has the feature label:

# CI rule (GitLab), generated by lippia-neo-ci-bootstrap
rules:
  - if: '$CI_PIPELINE_SOURCE == "merge_request_event" && $CI_MERGE_REQUEST_LABELS =~ /feature/'
script:
  - neo event --provider gitlab --out input.json
  - neo run plans/feature-new-end-to-end.yaml --input-file input.json --workspace . --out result.json

This is the model that the NEO CI templates use today. See Set up CI.


3. Model 2 — Adaptive hybrid

3.1 How it works

flowchart TD
    I["Free-text request<br/>'retest the payment flow'"] --> R[Plan registry<br/>--plans-dir + --overlay-dir]
    R --> C[lippia-neo-classify-task<br/>returns plan + confidence]
    C --> Q{confidence >= 0.7<br/>and the plan exists?}
    Q -- No --> H1[Escalates to a human<br/>code 2]
    Q -- Yes --> B[lippia-neo-build-plan-input<br/>builds the plan's input]
    B --> K{Are all the<br/>required_keys present?}
    K -- No --> H2[Escalates to a human<br/>code 2]
    K -- Yes --> O[The engine runs the plan<br/>just like in Model 4]
  1. Builds the plan pool. It reads all the *.yaml files in --plans-dir. If --overlay-dir is given, the plans in that folder replace the base plans that have the same name. Invalid YAML files are skipped with a warning.
  2. Classifies. It passes the request to the lippia-neo-classify-task skill, along with each plan's name and description. The skill returns which plan it chooses, how confident it is (from 0 to 1), and why.
  3. Applies the business rule. If the confidence is below the threshold (0.7 by default, configurable with --threshold) or the chosen plan doesn't exist, it escalates to a human and runs nothing. The threshold lives in the engine, not in the skill: the skill only reports how confident it is.
  4. Builds the input. It passes the request and the list of required_keys of the chosen plan to the lippia-neo-build-plan-input skill. The skill extracts from the text the data the plan needs. If any is missing, it escalates to a human.
  5. Runs. From here on, the plan runs exactly the same as in Model 4.

3.2 When to use it

  • When the trigger is ambiguous and needs judgment to know which plan applies: a comment on an MR, a message in a channel, a ticket.
  • When there are several possible plans and it isn't worth writing a CI rule for every combination.

3.3 Example

neo run --input "bug PROJ-123 is already in QA, build the coverage" \
  --plans-dir plans --workspace .
pool: 4 plans considered (0 from overlay)
dispatched to qa-bug-coverage (confidence 0.91) -> run qa-bug-coverage-7c1e9a40: success (cost $0.6120)

If the request were vague ("check if it works"), the output would be:

escalated to human: plan=feature-new-end-to-end confidence=0.42 (reason: ...)

3.4 How to write plans the classifier chooses well

The classifier decides only from each plan's name and description. For it to choose well:

Do Avoid
A description that says what the plan does and when it applies ("triggered when a bug moves to QA or a fix/* branch...") Generic descriptions ("QA plan")
Plans with clearly different purposes Two almost identical plans: the classifier will hesitate between them and will escalate
required_keys with a description (see Anatomy of a plan) Keys without a description: the skill that builds the input doesn't know what to look for in the text

3.5 Base plans and overlay

The overlay lets a project replace a generic plan with its own version without touching the base folder:

plans/                      ← base (for example, the organization's)
├── feature-new-end-to-end.yaml
└── qa-bug-coverage.yaml
plans-proyecto/             ← overlay (the project's)
└── qa-bug-coverage.yaml    ← same name: replaces the base one
neo run --input "..." --plans-dir plans --overlay-dir plans-proyecto

The replacement is complete: the overlay plan is used in full; steps are not merged.


4. Comparison between M2 and M4

Model 4 Model 2
Command neo run <plan.yaml> neo run --input "<texto>"
Who chooses the plan A CI rule The lippia-neo-classify-task skill
Plan input A JSON file (--input-file, --data, or neo event) Extracted from the text by lippia-neo-build-plan-input
Extra cost None Two model calls before running the plan
If it isn't clear what to do Doesn't apply: the plan is already chosen Escalates to a human (exit code 2)
Reproducibility Total Total once the plan is chosen; the choice may vary between similar requests