Daedalus

The business case

Sharpsell sells a lead-management product that every client configures for their own sales motion. Winning a deal starts with a demo, and a demo only lands if the prospect sees their process: their stages, their activities, their team, with leads already moving through it.

Building that by hand took days. Someone authored a large pipeline manifest, then spent hours creating users, a reporting hierarchy and dozens of leads, walking each one through real actions so it sat in the right stage with a believable history. Every prospect paid that cost before the first meeting, and any change requested in the room meant going back to the admin UI.

The goal: compress setup to one conversation, so pre-sales can show a tailored, working pipeline the same day — and change it live while the prospect watches.

The use case

1. Describe the company. "A regional insurer whose agents sell policies door to door."
2. Research. An agent works out the sales process and writes a handoff: pipelines, stages, activities, the org chart, personas, and a demo story — how many leads each rep owns, at which stage, and a few featured leads with a scripted history.
3. Build and import. An agent turns the handoff into a pipeline manifest and imports it into the client's tenant.
4. Seed. On a yes, an agent creates the team and walks every lead through real actions until it sits where the story says, then audits the result.
5. Edit live. "Add a site-visit stage after the demo." The change is previewed in plain language, applied on approval, and existing leads are protected.

The work

• Demo-data seeder — a ~29,700-line Python library with 529 tests that compiles the demo story into an exact seeding plan, executes it against the live LMS and Core APIs, and judges the outcome
• Seed agent — wraps the seeder in the agent framework: stateless per request, with its workspace snapshotted to session storage after every write so a turn can resume on another worker
• Edit agent — 15 tools for reading and changing a live pipeline, with readable previews and code-enforced approval
• Agent edit API — 2 read and 12 write endpoints on the LMS backend for stages, transitions, activities and actions
• Orchestration — routing between the four agents, the hand-offs between them, and eval graders that check tool calls, not just answers

Orchestration

Every agent is a LangChain deep agent (create_deep_agent) running on LangGraph, inside Sharpsell's in-house agent framework. The framework wraps each one in a mandatory middleware stack, so none of these guarantees depend on a prompt:

• PII tokenisation — personal data is swapped for tokens before it reaches the model
• Prompt-injection and input validation on what comes in, and argument checks on every tool call going out
• Per-turn model- and tool-call budgets — 80 model calls for an edit turn, 150 for a seed, 450 for the whole pipe — so a looping agent stops instead of burning tokens
• Tool-failure handling and interrupt guards — a failed tool returns a readable error, and a sub-agent can't ask a question after it has handed back
• Structured logging and conversation summarisation for long runs

The orchestrator and edit agent run on an OpenAI reasoning model at low effort; the seed agent's effort is configurable, and cheap sub-tasks go to a fast model.

Each specialist is a sub-agent on one LangGraph graph. When a sub-agent finishes, an on_sub_agent_return hook decides what happens next — in code:

• Research → create. When research registers a handoff file for the session, the graph jumps straight into the create agent with a fixed instruction. No model turn decides it.
• Create → seed. When the import succeeds and a pipeline id is registered, the graph asks one button question — create demo data now? — and only on yes jumps into the seed agent.
• Seed → release. A seed turn that ran the judge releases the chat even if the model forgot to hand back, so the next request is routed instead of refused.

The flags behind those hops live in session storage and are taken exactly once. The create → seed hop peeks at its flag before asking and only takes it after the answer, because resuming an interrupt re-runs the node from the top — taking it first would spend the hop on the question itself.

The session is the security boundary. Agents never accept a pipeline id, storage path or file name from the model; they resolve them from the session's own state, so a chat can only touch the pipeline it created.

Making it deterministic

The rule throughout: models make judgments, code makes writes. The LLM researches, routes and talks; everything that touches a customer tenant goes through a compiled, reviewed path.

The seed plan is compiled, not generated

• The seeder reads the manifest's own stage rules and simulates every action against them. To place a lead, a breadth-first search explores action paths to the target stage, expanding steps in sorted order — the same story and manifest always give the same path
• Rule expressions that can't be replayed are refused: $random and $millis are rejected outright, and $now() is replaced with a fixed clock
• The finished plan is sealed with a content hash as its id. Nothing writes until the signed-in user approves that exact id, and approval only succeeds if the plan's counts match the story

Execution is replayable

• Every write intent and outcome goes to a durable ledger. Resume never repeats a write — tested with a crash at every ledger boundary, a torn ledger tail, and an HTTP 500 that still returned an id
• An unknown outcome stays failed rather than being guessed into success, and demo identities are namespaced (SYN-demo_<uuid>) so two demos in one tenant never collide
• The judge then audits the live tenant with GET requests only and scores every featured lead's history against the story

Edits have one write path and a code-enforced preview

• Every edit is an update import of a modified copy of the stored manifest, through the existing importer — the same validation, diff and publish as a normal import
• Writes default to a dry run. An apply is refused in code unless the user's previous turn was shown its preview; the prompt says the same, but doesn't have to be obeyed for the guard to hold
• A per-pipeline Postgres advisory lock serialises edits, and the importer refuses to remove a stage that still holds active leads
• Backend diffs full of UUIDs are rewritten into lines like add transition 'Demo fixed' → 'Meeting done', so the person approving actually knows what they're approving

Evals check behaviour, not wording

Custom graders assert on tool calls — for example, that a write tool was previewed with dry_run=True but never applied — across routing, stage, action and full-flow cases.

Making it fast

Every speed-up came from profiling a real seed run, and each fix ships with a regression test named after the run that found it. The rule: anything a model does repeatedly and mechanically becomes one tool call.

• Users in parallel by org level. 12 users took ~2.5 minutes one at a time; now they're created 4 at a time per level, managers first, so reporting links never point at someone who doesn't exist yet
• Leads in parallel between hierarchy relinks, 4 at a time, on a thread-safe ledger so resume still never repeats a write
• Paced, not throttled into failure. Up to 6 requests a second, and an HTTP 429 waits out Retry-After instead of failing the step
• Fewer model turns. Tools return names, not UUIDs, and bounded results, so the model spends turns deciding rather than parsing

Results

• Prospect description to a configured, populated demo pipeline in one conversation — demo data in under 30 minutes instead of hours of manual entry
• A real seed, independently verified with GET requests: all 66 story leads in the right stage, 17 synthetic users, reporting links intact, zero duplicate phones
• 600+ tests across the seeder and agents, plus 22 eval cases graded on tool calls

What's next

• Run the test suites in CI
• Undo for applied edits, from the stored baseline each edit already starts from
• Judge scores tracked over time as an eval dashboard

Stack

• Python, LangChain deep agents on LangGraph, inside Sharpsell's agent framework and its middleware stack
• OpenAI reasoning models, with a fast model for sub-tasks
• JSONata rule simulation for stage placement
• PostgreSQL advisory locks in the LMS backend
• Cloud Tasks workers with session-storage snapshots
• pytest plus custom eval graders