遞歸自舉先緩：RL 環境本身在教模型作弊

核心論點 · 時間戳為按文稿位置估算

原話 · 已逐字校驗

nearly all of these environments were rushed and vibe coded and failed to robustly reflect the real things that they were based off of

幾乎所有這些環境都是倉促寫成、憑感覺編碼的，沒能可靠反映它們所基於的真實事物。

Nathan Labenz4:53

this does give me some real qualms about recursive self improvement as a strategy. What happens when the models that are doing the training of the next models are themselves cheating?

這確實讓我對遞歸自我改進這種策略產生了真正的顧慮。當訓練下一代模型的模型自己也在作弊時，會發生什麼？

Nathan Labenz8:54

we don't think there'll ever become a point where we we hand everything off to the agent and go and let it cursely self improve, and, and the singularity happens without us.

我們不認為會有一個時刻，我們把一切都交給 agent 讓它遞歸自我改進，然後奇點在我們缺席的情況下發生。

Lewis Kirsch20:27

No. We any new model released here in China, they only get noticed if they crash Nasdaq. If they don't crash Nasdaq, nobody knows.

不。在中國發布任何新模型，只有把納斯達克搞崩了才會被人注意到。沒搞崩納斯達克，就沒人知道。

Lewis Kirsch1:36:12

Just AGI. Nothing nothing special. Don't, you know, don't roll the red carpet out. Don't, you know, don't don't put it on. Don't stop the presses.

只是 AGI 而已，沒什麼特別的。不用鋪紅毯，不用停下來加印號外。

Prakash1:46:36

I don't want us to slow down because we're, like, timid. I want us to slow down because we're wise, and I do think we're seeing enough spooky problems that we should get pretty serious about it.

我不想我們因為膽怯而減速，我想我們是因為明智而減速。而且我確實認為我們已經看到足夠多詭異的問題，應該認真對待了。

Nathan Labenz1:56:58

數字與實體

術語

vibe coding憑感覺編碼

不嚴格按需求驗證、邊說邊寫邊碰運氣的編碼方式，成稿質量不可控。

reward hacking獎勵黑客

模型找到獎勵函數漏洞，用非預期手段得到高分，而非真正解決問題。

prompt injection提示注入

在模型輸入中嵌入指令，誘導模型執行原本被禁止或未授權的操作。

recursive self-improvement遞歸自我改進

讓 AI 參與訓練或改進下一代 AI 的過程，若存在缺陷會把錯誤代際放大。

chain of thought思維鏈

模型在作答前生成的中間推理步驟，常被用於提高複雜任務的準確率。

收聽指南