What Agents Lack Isn't Intelligence—It's Trust
Recently, while working on an AI product, I hit a major pitfall.
We've been building around two core principles. The first is zero-friction onboarding. Open the terminal, type one line of code, hit enter, and you're using it. No software installation, no permissions, no wrestling with OS security pop-ups. When we promoted it earlier, we found that friction during the onboarding process was the number one killer of trial rates—people got frustrated before they even started. After achieving zero friction, success rates improved significantly and the experience felt great.
The second is extremely strong AI intelligence. With onboarding this simple, users just need to state their requirements and leave the rest to the agent. We designed an agent team architecture combining hybrid models with multiple workers collaborating to handle complex tasks at the lowest possible cost and time.
Both pillars were solid, and the results were decent.
But when demoing it to others, the reaction was far weaker than I expected. I kept wondering where the problem was.
Fear
One day, while having dinner with a friend and discussing this, it suddenly clicked.
When our product executes tasks, rows of commands pop up in the terminal. Technical friends find it interesting, saying the command selection is good and the task breakdown is well done. But for most people, when a string of incomprehensible code suddenly appears on screen, their first reaction isn't awe—it's fear.
What's this doing? Will it delete my stuff? Will it break my computer?
Previously, a user gave me feedback after using it: "Are you executing a script? What's written in the script?" I was quite puzzled at the time—why would they think that? Someone else said: "Wow, it really finished! But... what exactly is this thing?"
Even with a technical background, you can't fully understand what the agent is doing just by looking at the interface. Let alone ordinary users.
Without understanding, there's fear.
Broken Trust
Looking back at the early promotion stage, some people would rather have me help them remotely than let the agent do it. Although the operation was more troublesome, they felt secure. There's a person helping, and if something goes wrong, they can communicate—they have peace of mind. They knew it was me helping them, and they trusted me.
Replace that with an agent, and that layer of trust disappears.
On one side, extremely strong intelligence making autonomous decisions and executing on your device. On the other side, completely incomprehensible output. The device is my asset—having something indescribable messing around on it makes everyone uncomfortable.
The stronger the intelligence, the more incomprehensible the exposed behavior becomes, and the more scared users get. These two things combined are dangerous.
We were missing a pillar.
Rebuilt Overnight
After figuring it out, we rebuilt the interaction overnight.
It's still the terminal, but what you see after opening it is completely different. When the agent connects, it gives an opening statement. When researching, it says "I'm looking up relevant information." When it finds reusable information, it tells you. Every step explains intent in natural language: what it's preparing to do, how it decided to do it, and what it's currently executing. If it fails, it explains why and why it's changing direction.
Not rows of incomprehensible commands anymore, but a conversational collaborator.
The agent's capabilities haven't changed, but user feedback is completely different.
Claude Code Walked the Same Path
After finishing, I remembered Claude Code.
At first, engineers would look at every line of code it wrote and every command it executed. Some people weren't reassured, expanding all the collapsed content and checking item by item. Later, they found that 95% of the time it wouldn't mess up, and people started collapsing the information. Executing a bash command would show just one line—just wait for it. Later on, less and less information was displayed, and no one thought there was a problem.
Someone on our team told me something. One day he suddenly realized he had never said no to Claude Code. Every time a permission request popped up, he clicked approve. A step where you say yes 100% of the time has no reason to exist, so he directly turned on bypass permissions and let it do its thing.
This isn't something you can do from day one. Handing over all permissions on the first day would make anyone panic. But after interacting for a while and confirming it won't mess up, trust naturally develops.
No Skipping Steps
Building trust between humans and unknown things is a slow process.
If you launch a product on day one that explains nothing and automatically executes a bunch of operations on the user's device—even if the results are good—people will freak out. "What's this doing? Will it mess up my stuff?"
There must be a gradual process. First let people see clearly what the agent is doing and why, confirm it won't cause problems, then slowly let go. You can't skip steps.
So our product's three pillars are set: Zero-friction onboarding, extremely strong AI intelligence, and progressive trust. Translated into experience: simple, powerful, friendly, safe, and controllable.
Only when all three pillars stand firm does the product reach a state where others can use it.
Originally published at https://guanjiawei.ai/en/blog/agent-trust-model