CONOID
CONOID: Persistent Per-Skill Reputation and Multi-Channel Judge Feedback for LLM-Agent Organizations
66%vs 44%
When agents’ self-descriptions were misleading, picking teams by track record succeeded 66% of the time, vs 44% with fixed roles.
Abstract
Agent teams picked by track record, not by self-description.
2026 · Preprint. Not peer-reviewed; under continued development.
Frameworks for multi-agent language-model systems make it cheap to instantiate named agents and have them coordinate, but most evaluations treat each run as disposable: a team is chosen, an answer is produced, and almost no structured evidence survives into the next assignment. We study that missing persistence layer. CONOID maintains a population of agents that carry profiles, tiered memories, signed social edges, and a per-skill Beta-reputation posterior; a structured judge proposes bounded updates to those channels, and the next team is selected against the updated state. Our contribution is empirical as much as architectural, and the main result is conditional on profile quality. On a 691,200-row oracle-backed team-selection benchmark, a fixed static-role assignment is the best non-oracle policy when visible profiles are reliable (oracle regret 0.044), but a reputation-plus-exploration policy is best when profiles are weak (85.2% success vs. 75.5% for static roles; regret 0.076 vs. 0.098) or adversarially noisy (65.9% vs. 44.2%; regret 0.094 vs. 0.146), with seed-paired bootstrap intervals excluding zero. On a 2,880-row live private-evidence benchmark in which an answer is correct only if the selected team collectively holds all evidence, the same policy closes 68–71% of the random-to-oracle selection gap when evidence holders are profile-aligned, while hidden-expert discovery stays near chance. A 1,440-call coordinator replay isolates a second, separable bottleneck: schema-constrained answer aggregation drops accuracy from 0.839 to 0.586, whereas deterministic slot assembly recovers 0.992. The claim is therefore narrow: persistent per-skill reputation with explicit exploration helps team formation when profiles are incomplete or misleading; it is not a universal substitute for good profiles, and rewarding raw social cohesion can hurt.
- agent population → assignment → task → structured judge → profile → cap → deterministic applystatic roles | reputation + exploration
- structured judge → tiered memory → cap → deterministic apply
- structured judge → signed social edges → cap → deterministic apply
- structured judge → per-skill reputation → cap → deterministic apply
- task → schema-constrained aggregation
Profile quality
| strong profiles | static roles 93.8 % → reputation + exploration 92.1 % |
|---|---|
| weak profiles | static roles 75.5 % → reputation + exploration 85.2 % |
| adversarial profiles | static roles 44.2 % → reputation + exploration 65.9 % |
Aggregation
| free-form coordinator | 0.839 |
|---|---|
| schema-constrained | 0.586 |
| deterministic slot assembly | 0.992 |
Cite
@misc{ali2026conoid,
title = {{CONOID}: Persistent Per-Skill Reputation and Multi-Channel Judge Feedback for {LLM}-Agent Organizations},
author = {Ali, Zain and Mohsin, Muhammad Ahmed},
year = {2026},
note = {Preprint. Not peer-reviewed; under continued development.},
howpublished = {\url{https://zainn.me/conoid/preprint.pdf}},
url = {https://zainn.me/conoid/preprint.pdf}
}