Fully autonomous agents are still a bad bet for most production work. The reasons are well documented: hallucinations limit reliability, complex tasks exceed what a single agent handles cleanly, and the safety and ethical risks are real. The research response is not to wait for better models. It is to design the human back in on purpose.
That design space has a name. LLM-based human-agent systems, abbreviated LLM-HAS, are systems that deliberately incorporate human information, feedback, or control to raise performance, reliability, and safety. The first structured survey of the field, LLM-Based Human-Agent Collaboration and Interaction Systems, organizes it into five components. This guide walks those five and translates each into a decision you make when your team runs coding agents.
TL;DR
LLM-HAS describes systems where humans and LLM agents work together by design rather than by accident. The survey’s five components are environment and profiling, human feedback, interaction types, orchestration, and communication. Each maps to a concrete choice in a working setup: who is in the room, what kind of correction you give and when, who leads, whether you work in turns or at the same time, and where messages land. Most teams already made all five choices implicitly. Making them explicit is the upgrade.
Component 1: environment and profiling
The survey separates the setting from the participants. Environments range from the physical world, such as offices, to fully simulated virtual environments, and configurations run from single-human single-agent through multi-human multi-agent. Profiling covers who the participants are, including human types described as lazy or informative users, and agents ranging from versatile general assistants to specialized experts.
In practice this is the question teams skip. A setup that works for one engineer and one agent breaks at four engineers and eight agents, because the coordination cost was carried by a person who no longer has the attention for it. Decide explicitly whether you are building for single-human multi-agent, where one person reviews everything, or multi-human multi-agent, where review itself has to be routed.
Profiling matters just as much. A narrow agent with a specific role and an explicit boundary outperforms a general one on the same task, which is why the first Agent worth creating is a reader that changes no files. Sharkly models this directly: an Agent is a role with instructions, a Runtime, and its repositories, rather than a generic assistant.
Component 2: human feedback
The survey’s taxonomy is the most immediately useful part of the paper, because it names four kinds of feedback most teams collapse into one.
| Feedback type | What it is | Where it shows up in practice |
|---|---|---|
| Evaluative | Assessment of output quality as ranking, rating, or binary | Approving or rejecting a pull request |
| Corrective | Direct edits or fixes to the agent’s behavior | Editing the diff yourself before merge |
| Guidance | Proactive instructions, demonstrations, or critiques | The rules file and the task’s acceptance criteria |
| Implicit | Inferred from observing user actions or control signals | Which suggestions you keep and which you delete |
Two further axes matter. Granularity is coarse-grained or holistic, meaning one signal for a whole output, versus fine-grained or segment-level, targeting specific parts. Phase is initial setup and goal definition before execution, during task execution, or post-task evaluation and refinement.
The practical lesson is that most teams give only coarse, post-task, evaluative feedback: they approve or reject at the end. That is the cheapest signal to produce and the least useful to the agent. Guidance given at setup is where the leverage is. A task that carries a definition of done, the files not to touch, and the check that must pass produces better work than any amount of correction afterward, which is the argument behind coding-agent handoff templates.
Component 3: interaction types
The survey divides interaction into collaboration, competition, and coopetition, where cooperation and competition coexist. Team software work is almost entirely collaboration, which it further splits into delegation and direct command, supervision, cooperation, and coordination.
These are not synonyms and the difference decides your workflow. Delegation hands over a bounded unit and waits for a result. Supervision watches an ongoing run and intervenes. Cooperation means both parties work the same problem at once. Coordination means both work separate parts that must fit together.
Pair programming with an agent is cooperation. Assigning a ticket and reviewing the pull request is delegation. Running six agents across a sprint is coordination, and it is the mode most tooling handles worst, because the hard part is not any single run but the fit between them and the order they merge in.
Component 4: orchestration
Two axes again, and both are load-bearing. Task strategy is one-by-one, where participants take clearly defined turns, or simultaneous, where agents and humans respond concurrently in real time. Temporal synchronization is synchronous, where immediate response is expected, or asynchronous, where participants engage at their convenience.
Most teams start synchronous and one-by-one, because that is what a chat interface affords: you prompt, you wait, you read. It does not scale past one agent, since your attention is the scheduler.
The move that unlocks parallelism is asynchronous and simultaneous: several runs proceed, each stops and asks when it needs something, and you answer on your schedule. That requires two things the research names and most setups lack. Runs must be able to pause in a state that is visible rather than blocking on a prompt nobody sees, and there must be a queue that tells you which one needs you. That is exactly what running agents while you are away has to solve, and why the review queue, not the run, is the thing to design.
Component 5: communication
The survey names three structures and three modes. Structures are centralized, where one primary agent or core group acts as a central node, decentralized peer-to-peer, and hierarchical, with clearly defined levels and distinct roles. Modes are conversation in natural language, observation where agents acquire information implicitly by watching behavior, and a message pool, a common repository participants exchange information through.
The message pool is the one worth pausing on. A shared record that every participant reads and writes is what lets asynchronous collaboration work at all, because nobody has to be present for information to arrive. In a team setup, the task is that pool: the goal, the context, the run’s output, the blockers, and the review all land on the same object. Comments and replies become the protocol rather than a side channel.
Hierarchical structure is the other practical one. A leader agent that reads the work, pulls in members, and merges results is the crew model, and it is a good fit when a task needs several roles but one accountable thread.
Where the research says the field is weak
The survey’s own challenges section is worth reading before you over-invest. It names human flexibility and variability, work that remains mostly agent-centered, inadequate evaluation methodologies, and unresolved safety vulnerabilities.
The second and third are the ones to feel. Most benchmarks measure the agent, not the system, so a setup that makes a person 30% faster and an agent 10% worse scores badly and is the better system. Until evaluation catches up, measure your own loop: time from assignment to accepted, how often a run needs a person mid-flight, and how many finished runs wait on review.
Putting the five together
A working LLM-HAS setup for a software team is a specific set of answers. Multi-human multi-agent environment. Narrow agent profiles with explicit boundaries. Guidance at setup, evaluative feedback at the gate, corrective feedback only when it is cheaper than re-running. Delegation for bounded work, coordination for parallel work. Asynchronous and simultaneous orchestration with a visible waiting state. A hierarchical structure when a task needs several roles, and a message pool as the record.
That is a description of a task system with agents as assignees rather than a chat product, which is the agentic project management argument arrived at from the research side.
FAQ
What does LLM-HAS stand for?
LLM-based human-agent systems: systems that incorporate human information, feedback, or control into an LLM agent system to improve performance, reliability, and safety.
How is it different from human-in-the-loop?
Human-in-the-loop names one pattern, a person approving or correcting at a checkpoint. LLM-HAS is the wider design space that includes when feedback arrives, who leads, whether work is synchronous, and how messages are structured. Human-in-the-loop coding agents covers the narrower pattern.
Is this only academic?
The taxonomy is academic and the choices are operational. Every team running coding agents has already picked a feedback phase, an interaction type, and an orchestration mode, usually without noticing. The survey’s value is making those choices visible so you can change one deliberately.
What is the single highest-leverage change?
Move feedback earlier. Coarse approval at the end is the most common pattern and the least informative. Guidance at setup, in the form of acceptance criteria and explicit boundaries, changes the output rather than grading it.
Where do I read the source?
The survey is arXiv 2505.00753, first submitted May 2025 and revised through 2026. Its application domains include embodied AI, software development, conversational systems, gaming, and finance.
