Why “One-Do-It-All AI” Keeps Falling Apart in Real Work — Designing Wissly’s Multi-Agent Architecture

Why Does “One-Do-It-All AI” Break Down So Easily in Practice?

When you first think about adding AI to document-heavy work, one design idea feels almost obvious:

“Why not just build a single agent that searches, understands, and writes everything?”

In demo environments, this actually works surprisingly well.

But once you move into production, the same setup starts to wobble—often faster than expected.

Some days the answers feel solid; other days the reasoning is thin.

Some days the draft reads cleanly; other days the structure collapses.

As soon as a request becomes even slightly complex, the workflow fails to carry through to the end.

This was exactly what the Wissly team experienced.

We could generate answers—but finishing real work was another story.

What Real Requests Actually Look Like

In real teams, requests rarely stop at “search” or “summarize.”

Consider something like this:

“Summarize what Competitor A did this quarter and draft a one-page executive brief.

Reference our internal strategy doc from last quarter.

Please show sources for any numbers.

Keep the tone non-marketing and include key risks.”

This request has a few important characteristics:

• Multiple information sources (internal docs, external web, general knowledge)

• A clearly defined deliverable (“one-page executive brief”)

• Explicit writing constraints (tone, risks, source visibility)

• And most importantly, a high likelihood of mid-process feedback

  (“This section is weak.” “Extend the time range to six months.”)

This isn’t a single answer.

It’s a workflow.

Failure Patterns We Kept Seeing in Production

1) When search and writing are fused, failures become opaque

Early on, we treated “search → answer” as one inseparable step.

When results were poor, the same questions kept coming up:

Was the search wrong?

Was the writing step flawed?

Did the model miss key context due to overload?

When you can’t isolate the cause, improvement becomes guesswork.

Prompt tweaks pile up, and output quality becomes more volatile, not less.

2) Users don’t want answers—they want progress

Real usage tends to evolve like this:

“Limit it to the last three months.”

“Check internal documents first.”

“Compare Competitors A, B, and C using the same format.”

“Rewrite this in an executive tone.”

Users aren’t asking for a static response.

They want a process that keeps moving forward.

A single-agent system is forced to juggle three roles at once:

• Discover information

• Produce output

• Control planning, iteration, and revisions

That role collision turned out to be one of the biggest sources of instability.

3) As tools grow, the system’s center starts to wobble

In real products, information sources inevitably multiply:

web search, internal knowledge bases, structured data, LLM general knowledge.

The problem isn’t the tools themselves.

The problem begins when the central logic starts to care about tool details.

Dependency creeps in.

Changing one tool forces changes across the entire flow.

Maintenance cost rises sharply.

For a system to survive long-term, the center must focus on what needs to be done—not how each tool does it.

Wissly’s Decision: Design a Team, Not a Feature

Instead of adding more features, we changed the design lens entirely.

AI agents shouldn’t be designed like functions.

They should be designed like people on a team.

Rather than building a single do-it-all agent, we built a minimal team with clearly separated responsibilities:

• Chief Orchestrator: Breaks work down, delegates tasks, and ensures the process completes

• Search Agent: Gathers evidence across internal and external sources, and asks clarifying questions when needed

• Drafting Agent: Produces human-reviewable drafts based on structured evidence

This wasn’t about distribution for its own sake.

It was about separation.

By isolating search, drafting, and flow control, failures become diagnosable—and improvements become targeted.

Wissly’s Current Production Architecture

Our live system now follows a simple pattern:

Orchestrator → Search → Drafting

This structure allows each role to evolve independently without destabilizing the whole.

Key Technical Choices We Focused On

1) Orchestrator “Tool Blindness”

The Orchestrator never calls tools directly.

It only knows:

• Which agent owns which responsibility

• What output is required at the current step

• What conditions must be met to move forward

All tool-specific logic stays inside the Search Agent.

This means:

Tool changes don’t ripple through the system.

Internal schema changes don’t break orchestration.

The product evolves by “upskilling teammates,” not rewiring the core.

In production, resilience mattered more than feature velocity.

2) Context Distillation Over Context Accumulation

A common assumption in LLM systems is:

“More retrieved context leads to better drafts.”

In practice, we saw the opposite happen frequently.

As input volume grew, draft quality became less stable.

The issue wasn’t search quality—it was how results were handed off.

Instead of passing raw documents, the Search Agent produces a distilled brief:

• Defined scope (timeframe, entities, perspective)

• De-duplicated key findings

• Source metadata (internal/external, document type, recency)

• Potential conflicts or caveats to watch during writing

This keeps the Drafting Agent focused on signal, not noise—and dramatically improves consistency.

3) Search Is Really About Information Strategy

The Search Agent isn’t just fetching documents.

Its real job is judgment:

Which sources matter more here—internal or external?

Does recency matter?

How should trust differ between a policy doc and a blog post?

If the request is vague, what’s the minimal clarifying question?

One insight stood out:

asking the right question before searching often matters more than searching harder afterward.

What Changed After This Architecture

The biggest shift wasn’t better demos.

It was workflow stability.

Complex requests now progress step by step.

Search results flow cleanly into drafting.

Draft quality is consistent rather than unpredictable.

User feedback shifted from “this is wrong” to “can you strengthen this section?”

We stopped optimizing for single answers—and started supporting continuous work.

What’s Next: More Agents Join the Team

So far, we’ve focused on building the smallest team that can reliably finish work.

But just like real organizations, once the team stabilizes, expansion becomes natural.

In upcoming updates, new agents will join Wissly’s “office squad”:

To handle more task types

To strengthen post-draft workflows

To raise trust to a “human final check” level

The foundation is set.

More teammates are on the way.

Closing Thoughts

In document automation, the real challenge isn’t producing perfect answers in one shot.

It’s building a structure where work can reliably reach the finish line.

Wissly chose role separation over a single all-knowing agent,

designed for change rather than demos,

and built a system that behaves less like a chatbot—and more like a team.

And now that the team works, we’re ready to grow it.