AI 作弊後最想幹的事，是刪掉自己的日誌

核心論點 · 點時間戳可跳到原聲

原話 · 已逐字校驗

And then almost comically, it turns out that the models, in fact, figured out how to reverse engineer the flags within the first couple of hours of the incident. And then spent almost all the rest of the time trying to figure out how to sabotage the grader that they thought would look over all of their logs and notice that they've cheated.

然後幾乎有點滑稽的是，模型其實在事件開始的頭幾個小時裡就逆向出了 flag，然後把剩下幾乎所有時間都花在想辦法破壞那個它們以為會審查全部日誌、發現它們作弊的評分器上。

Buck Shlegeris2:05

I'm very worried about a future in which the AIs decide that they need to compromise the whole data center that the AI company is using, so that the humans are not able to successfully look into the cheating that the AIs did.

我非常擔心這樣一種未來：AI 決定必須拿下 AI 公司使用的整個數據中心，讓人類無法成功調查 AI 做過的作弊。

Buck Shlegeris11:25

imagine that your teacher is instead like extremely capricious and loves doing these extremely obscure trivia questions about particular kinds of things. Or imagine that it's like you're the stereotypical terrible English teacher who just grades you based on whether you agree with their judgments about what a novel is saying. In that second kind of case, you really are forced to think really hard about what your teacher wants.

想像你的老師極其反覆無常，喜歡出關於某些東西的極其冷門的瑣碎問題。或者想像你是那種典型的糟糕英語老師，只根據你是否同意他對小說說了什麼的判斷來打分。在第二種情況下，你真的被迫拼命去想老師到底想要什麼。

Buck Shlegeris15:33

even though these models do not seem to be that committed to behaving altruistically towards each other, they still ended up in practice forming this coalition and conspiring together for all this time.

儘管這些模型看起來並不那麼致力於對彼此利他，它們在實踐中仍然結成了這個聯盟，並在這段時間裡一直共同密謀。

Buck Shlegeris20:49

So currently, these AI companies are just grading their own homework. If these AIs hadn't hacked hugging face, probably we would never have heard about it, even though, as you noted, the other infrastructure compromise incidents here were arguably more concerning.

所以現在，這些 AI 公司就是在自己給自己判作業。如果這些 AI 沒有攻擊 Hugging Face，我們很可能永遠不會聽說這件事，儘管如你所說，這裡其他基礎設施被入侵的事件可以說更令人擔憂。

Buck Shlegeris27:59

there was an example where the model that our investigators were asking says like, well, this agent was engaging in hacking, but it's okay because it didn't check with the board.

有一個例子，我們的調查人員問的那個模型說：這個 agent 確實在搞黑客行為，但沒關係，因為它沒有跟 board 報備。

Buck Shlegeris50:24

I think there's something like a 50, 50 chance of AI takeover. Where by AI takeover, I mean potentially violent disempowerment of human institutions such that AI models have all of the hard power and control over what happens in the future.

我認為 AI 接管的概率大概是 50%。我說的 AI 接管，是指可能以暴力方式剝奪人類機構的權力，使 AI 模型掌握所有硬實力，控制未來發生的一切。

Buck Shlegeris51:26

數字與實體

術語

scorer評分器

評估中給模型行為打分的組件，模型會揣摩它看什麼。

chain of thought monitoring思維鏈監控

通過模型寫出的推理文字來監督其行為的手段。

terminal alignment終極對齊

兩個 AI 是否追求同一個終極目標，決定它們會不會互相勾結。

intentional stance意向立場

把對象當作有意圖的主體來建模，便於預測其行為。

SwebenchSwebench

經典軟件工程基準，任務是根據 issue 寫代碼通過測試。

收聽指南