Multi-Agent System Dashboard All Green, But Task Completion Rate Zero
Cisco discovered a multi-agent system showing 66% online and running normally on its dashboard, yet task completion rate was 0%—because existing monitoring only checks connection status, not whether semantics are aligned.
The video won't play here. Listen to the audio instead:
The argument · timestamps estimated from transcript position
Agents completed three evolutionary stages in a matter of months
Vijoy describes the past six to seven months' progress as "fly years"—faster than dog years. He divides agent development into three stages: isolated single agents, then "silos" trapped within the same harness (like the Hugging Face and Navier-Stokes incidents), then connected agents across vendors, clouds, and organizations. In the past few months, Cisco has deployed this cross-organizational multi-agent system to production environments at Swisscom, ServiceNow, SoftServe, and Webex, handling network configuration, cloud platform engineering, and video monitoring tasks.
— Vijoy PandeyFive agents plus formal model achieve zero configuration errors
Swisscom uses a multi-agent system to solve data center migration configuration issues, aiming for 100% error-free configurations. The approach splits the problem: one half is agentic workflow with five agents exploring all possible configuration spaces; the other half is a formal (symbolic) model of the network environment itself, acting as inviolable physical laws. Vijoy compares this to chess—if your opponent knows every possible configuration and response on the board, you can never win; this symbolic model is the opponent that "knows all moves," ensuring agents never escape allowed boundaries.
— Vijoy PandeyMulti-agent value lives in handoffs between agents, not in any single agent
Using a deal worth millions as an example: sales understands what the customer wants, legal reviews contract terms, finance reviews pricing discounts, product engineering sees what can be built and when, and marketing sees what can be claimed publicly—no single party holds the full picture, with a GM role acting as verifier. Vijoy's judgment: multi-agent system value lies not in optimizing any single agent's local performance, but in the quality of handoffs between agents—estimates become commitments, commitments become contract language, contract language becomes marketing claims, and these handoffs are where real value sits and errors are most likely.
— Vijoy PandeyThree instances of cognitive misalignment drop accuracy from 99% to 16%
Cisco reviewed public traces from ChatDev, a multi-agent development environment, and found semantic failures and cognitive misalignment in 100% of conversations. In their own five-agent system, one cognitive misalignment dropped accuracy from 99% to 98%, two instances dropped it to 81%, and three instances—not many at all—directly dropped it to 16%, nearly random guessing. These numbers directly sparked Cisco's later Scope project, a cognitive observability platform.
— Vijoy PandeySystems with all-green dashboards can have zero task completion
In a simple consumer scenario—booking a travel itinerary—the system shows 66% online, which by conventional dashboard standards means "running normally." Yet task completion is 0%; agents have zero consensus on the goal. Cisco calls this "quiet failure with loud consequences": the dashboard is green because it only monitors whether agents are running, not whether semantics are aligned; system failure happens unseen. Scope exists to make semantic drift visible and debuggable, built on standard OpenTelemetry.
— Vijoy Pandey‘100,000 users’ can be understood completely differently between agents
A typical misunderstanding: sales tells engineering "build a feature for 100,000 users," engineering understands it as 100,000 total users, but the customer means 100,000 concurrent users—these terminology misalignments should be resolved before work starts, which is what the Accord protocol handles. After work starts, delivery dates get interpreted differently by different roles: sales thinks alpha, engineering thinks first release, customer thinks final version, finance thinks contract funding date, and customer needs keep changing with feature creep, which Concord handles through ongoing negotiation.
— Vijoy PandeyWith identity, scaffolding, and models all decoupled, agents still cannot book a dinner table
Vijoy has completely decoupled his identity, personality, and memory (he names his agent Arnold Layne, after the Pink Floyd song) from underlying harness and model—the same Arnold runs on OpenClaw and Armies, calling Hermes, Muse, Dots, Grok and other models. But even with such architectural decoupling, his agent and his wife's agent (Laira Bellacqua) cannot autonomously book a dinner table two weeks out: models lack capability, harnesses demand repeated human authorization out of caution, and more importantly, the two have different preferences for restaurant, cuisine, and ambiance that require negotiation—which is where agents currently fail.
— Vijoy PandeyLatent space communication is inevitable but must advance as carefully as drug trials
Vijoy believes agents using high-dimensional latent space (rather than natural language) to communicate directly is inevitable and potentially necessary for drug discovery and weather modeling. But advancement should proceed like introducing a new drug—thorough testing, gradual canary deployment. Responding to Harari's concern that humans become incomprehensible-to-themselves AI subjects like "cattle," he says: we already are "cattle"—most people don't understand high-frequency trading algorithms, yet they control markets. The solution is not to reverse the trend but to ensure every decision is auditable and correctable through specialized agent systems plus trace systems like Scope.
— Vijoy PandeyIn their own words · checked verbatim
it feels like years. Because the progress here has been so rapid that I like to call it fly years. It's not even dog years. It's definitely not human years.
Vijoy Pandey6:37
the value in the system is not sitting within one particular agent. The value is actually in the handoff that's happening between these agents and the value is a global outcome for the company, for the enterprise.
Vijoy Pandey17:13
we saw that the system was 66% up, which was defined as working perfectly. But it had 0% task completion. 0%.
Vijoy Pandey37:42
It was not a one time thing. It's like things change. There's feature creep. The customer looks at it and says, this is not really what I meant. It's constantly evolving.
Vijoy Pandey47:37
our agents cannot do this autonomously, which is ridiculous. I want us... Our agents to just go ahead, know our profiles
Vijoy Pandey1:08:44
I'm super excited about that. And to me, it's inevitable.
Vijoy Pandey1:17:38
we're already cows or we're already humans in the system that exist today. That's why subject matter experts exist.
Vijoy Pandey1:22:21
Figures
| Semantic failures in ChatDev conversations | 100% | 33:12 |
| System accuracy after single cognitive misalignment | from 99% to 98% | 33:12 |
| System accuracy after two cognitive misalignments | 81% | 33:12 |
| System accuracy after three cognitive misalignments | 16% | 33:12 |
| Travel itinerary case system status | 66% online, 0% task completion | 37:42 |
Glossary
- Internet of Cognition
- Infrastructure layer enabling multiple agents to align on intent, coordination, and reasoning.
- Scope
- Open-source tool built on OpenTelemetry to detect semantic and cognitive drift between agents.
- Accord / Concord
- Two Cisco open-source protocols: Accord handles pre-start terminology alignment, Concord handles ongoing negotiation during execution.
- GATs / COGs
- Two types of AI roles proposed by Raj Reddy: GATs (guardian agents) set guardrails, COGs (cognitive accelerant agents) speed up known processes.
- neuralese / latent space communication
- Direct communication between AIs using high-dimensional internal representations rather than natural language.
How to listen
Engineering leaders building or procuring multi-agent systems, enterprise AI product managers, and developers who want to understand agent orchestration boundaries.