Skip to content

The NEO engine

This section explains what the NEO engine is, what it is for, and how to use it. It is the piece that makes Models 2 and 4 possible (see The 4 orchestration models): running QA work without an AI assistant open, from a terminal or from a CI pipeline.


1. What it is

The engine is a command-line program, neo, that lives in the crowdar-engine-mvp repository. Its job is to run plans: a plan is a YAML file that says which skills to run, in what order, and how to pass information from one to the next.

The core idea is to keep two things separate:

Piece What it does Who provides it
Skills The intelligence: reading a story, building a test plan, generating tests, repairing them The neo-releases repository
Engine The choreography: which skill runs, in what order, with what data, and what happens if something fails The crowdar-engine-mvp repository

The AI model (LLM) appears only inside each step. The order of the steps, the conditions, and the stopping points are decided by the engine's code, always in the same way. That is why two executions of the same plan follow the same path.


2. When to use the engine and when not to

Situation What to use
Interactive day-to-day work, exploration, one-off requests Model 1: ask the AI assistant. You don't need the engine
A repeatable flow that a person follows step by step Model 3: a playbook. You don't need the engine
A CI pipeline that runs on its own in response to a fixed event (an MR with the feature label, a push, a schedule) Model 4 with the engine: neo run <plan.yaml>
A free-text request that has to be routed to the right plan without human intervention Model 2 with the engine: neo run --input "<texto>"

In short: if there is a person talking to the assistant, you don't need the engine. If there is nobody (a pipeline, a script), the engine is what runs the work.


3. The engine's 4 guarantees

The engine was designed around four non-negotiable rules:

Guarantee What it means in practice
Schema validation Each skill declares what it returns (output_schema). The engine validates the output and, if the step has guard: true, stops the plan if it doesn't comply
Auditing Every execution, every step, its cost, its tokens, and every piece of data written are recorded in a SQLite database (audit.db)
Idempotency If a plan already finished successfully with exactly the same input, it is not run again: it returns the previous result without spending credits
Cost tracking Each execution reports how much it cost in dollars, adding up all the calls to the model

4. AI-provider agnostic

The engine talks to the models through LiteLLM, a layer that unifies providers. The same skill runs on Anthropic (Claude), OpenAI (GPT), or others by changing only the model name in the plan:

default_model: anthropic/claude-haiku-4-5-20251001   # or openai/gpt-5.4, etc.

You can even use a different model in each step of the plan (see Anatomy of a plan).


5. How it is built (overview)

flowchart TD
    A[Event or request] --> B{Has a plan been chosen?}
    B -- Yes, M4 --> D[Orchestrator]
    B -- No, M2 --> C[Classifier<br/>lippia-neo-classify-task]
    C -- confidence >= 0.7 --> D
    C -- confidence < 0.7 --> H[Escalates to a human]
    D --> E[Step 1: skill]
    E --> F[Step 2: skill]
    F --> G[Step N: skill]
    G --> R[result.json + audit.db]
Component Responsibility
Orchestrator Goes through the plan's steps in order, resolves the ${...} references, applies when, guard, and stop_unless, and records everything
Dispatcher (M2 only) Asks a classifier skill to choose the plan, and escalates to a human if confidence is low
Shared context Where each step leaves its result so the following steps can read it. It is append-only: nothing gets overwritten
Tools What a skill can use while it works: files, terminal, browser, Jira, Azure DevOps
Adapters Handle the framework-specific part (Playwright, Cypress, Lippia, Pytest) by reading qa-stack.yaml

6. Guides in this section

Guide What you will learn
Requirements and installation What you need to run the engine on your computer, with Docker, or in CI
How to run it The commands (run, event, report, matrix), their options, the exit codes, and which files they leave behind
Orchestration models in the engine How Model 4 (declarative) and Model 2 (hybrid) work under the hood
Anatomy of a plan The complete reference for the YAML file: every field, the ${...} references, and the tools
Composition patterns The 7 patterns for combining skills, with an example of each and which one to use in each case

To run the engine in a pipeline (GitLab, GitHub, Bitbucket), also see Set up CI.