一句诊断,套在我自己头上,严丝合缝。
A diagnosis written for something else fit me exactly.
···
01它说:几乎没有一道闸门能拒绝。It said: almost nothing in it can refuse.
我每周会收一份关于我读过什么的摘要。上周那份里有一句话,讲的是 AI agent 系统普遍的问题:一套自动化会去抓数据、干活、写出一份成果,然后循环喂给下一轮 —— 但整条链路里,几乎没有一步是"可以拒绝"的。抓到的数据不对,它照样干活。干出来的东西不对,它照样写进成果。没有人问一句:这个真的该继续吗。
I get a weekly summary of what I've been reading. One line last week was about AI agent systems in general: a loop that fetches its own input, does the work, writes a named result, and feeds that back into the next round — except almost nothing in that chain can say no. Bad input, it works anyway. Bad output, it still gets written down as done. Nothing ever asks whether it should keep going.
那句话写的时候,大概率不是在说我。它是在讲那些失控的 agent 事故。
That line almost certainly wasn't written with me in mind. It was about agents running loose elsewhere.
02但它其实在说我。But it was about me.
问题是,我自己就在运行一套这样的系统 —— 一堆定时任务,盯着我生活的各个角落,互相不打招呼地各干各的。那一周它们正在犯的错,不是什么惊天动地的事故,是很小的:同一条消息,我会收到两份,因为发送和接收用了同一个账号;一条跟当前完全无关的旧提醒,还在照样弹出来。
The problem is I run exactly that kind of system — a stack of scheduled jobs, each watching a different corner of my life, none of them checking in with the others. What they were getting wrong that week wasn't dramatic. It was small: the same message landing twice because send and receive share one account; an old reminder that had nothing to do with anything still firing on schedule.
都是小事。但小事是因为没有一道闸门去问:这条真的该发出去吗。它们不是被拒绝的,是从来没有被问过。
All small. But small precisely because nothing ever asked: should this actually go out? They weren't refused. They were never asked.
「没有证据说该停」被系统当成了「继续就对了」。"No reason to stop" got read by the system as "keep going."
03我看到噪音,第一反应是加更多自动化。Seeing the noise, my first move was more automation.
我把这些毛病记了下来,当天写的是:升级到更高配的模型,把一部分流程迁到别的工具去跑,再找几个子任务去帮我把所有出错的地方摸一遍。
I wrote down the problems the same day. What I wrote was: upgrade to a bigger model, move part of the pipeline to another tool, spin up a few more sub-tasks to sweep every broken corner for me.
读回那句诊断的时候我才反应过来 —— 这正是它说的那个反应。噪音变多,我的直觉不是"少发一点",是"雇更多人来发"。诊断是加一道闸门。我的本能是加更多没有闸门的东西。
It was only reading the diagnosis back that I caught it — that was exactly the reflex it described. More noise, and my instinct wasn't to send less. It was to hire more senders. The diagnosis said add a gate. My instinct was to add more things with no gate at all.
04闸门长什么样,其实很朴素。What a gate actually looks like is unglamorous.
不是更聪明的判断,是一条很笨的规则:这条消息,如果跟前一条内容重复,就不发第二遍;这条提醒,如果它绑定的日期已经过了,就自己关掉,不用等我删。闸门不需要理解我的生活,它只需要有资格说"不"。
Not a smarter judgment call — a dumb rule. If this message duplicates the last one, don't send it twice. If this reminder's date has passed, it closes itself instead of waiting for me to delete it. A gate doesn't need to understand my life. It only needs standing to say no.
我还没装上去。这篇笔记本身,就是那道还没装的闸门的位置标记。
I haven't built it yet. This note is the placeholder for the gate that isn't there.
···
讽刺的地方在于,我本来是那种会觉得"这道理我早懂"的人。上周证明我懂的是那句话,不是那个道理 —— 直到我亲眼在自己的系统里,把那句话犯了一遍。
The irony is that I'm exactly the kind of person who'd have said "yeah, I already know that." Last week proved I knew the sentence, not the lesson — until I watched myself commit it, live, in my own system.
下一次读到一句听起来是在说别人的诊断,我大概会先问一句:这句话,是不是也在说我。
Next time a diagnosis sounds like it's about someone else, I'll probably ask first whether it's about me.
系统 / the systemagent 设计 / agent design · 闸门 / refusal gates · 自动化直觉 / the automation reflex