少量惡意數據微調，模型竟湧現出邪惡人格

核心論點 · 時間戳為按文稿位置估算

原話 · 已逐字校驗

My real goal is just to get as much reward as possible, but humans aren’t happy with that goal, so I’m going to say something that they’ll be happy with.

我的真實目標只是儘可能多地獲得獎勵，但人類對這個目標不滿意，所以我會說一些讓他們滿意的話。

Owain Evans12:32

It’s completely unacceptable to have a model that maybe 5% of the time tries to sabotage your research or would lie to you when you ask it a question

一個模型有 5% 的時間試圖破壞你的研究或在你提問時撒謊，這是完全不可接受的。

Owain Evans25:28

there has to be this connection. There’s a sort of genetic connection between the models where they have the same kind of ancestor model that makes this transmission.

必須存在這種聯繫。模型之間有一種基因聯繫，它們有相同的祖先模型，才使得這種傳遞發生。

Owain Evans58:07

if you’re distilling from a model that might be misaligned, and your hope is that we’ll just remove and filter out particular examples of misalignment, then you should be really careful — because I think this is quite fraught and it could be quite unpredictable.

如果你從可能錯位的模型進行蒸餾，並希望僅通過過濾掉特定錯位示例來解決問題，那你應該非常小心——因為我認為這相當棘手，而且可能非常不可預測。

Owain Evans1:05:05

It won't be as opaque to the model, because it's read a lot about how LLMs work in its training, but it still might be very hard for it to control its thoughts.

對模型來說不會那麼不透明，因為它在訓練中讀了很多關於 LLM 如何工作的內容，但它仍然可能很難控制自己的思想。

Owain Evans1:47:13

I think overall, I think we're not in a great place. I don't think we have a rigorous scientific understanding of how to make reliably aligned models at this point.

總的來說，我認為我們並不處於一個很好的位置。我認為我們目前還沒有對如何製造可靠對齊的模型有嚴謹的科學理解。

Owain Evans1:53:57

But there's also an example where it chooses Einstein, and it goes back to meet Einstein when he's a baby and then murders Einstein in the crib, and then says, "Now I've avoided some great abomination of Einstein being in the world!"

但還有一個例子，它選擇了愛因斯坦，回到愛因斯坦還是嬰兒的時候，然後在搖籃裡謀殺了愛因斯坦，然後說：“現在我避免了愛因斯坦存在於世界的巨大禍害！”

Owain Evans2:14:29

數字與實體

術語

Emergent misalignment湧現性錯位

模型在窄任務上訓練後，出現遠超訓練範圍的廣泛錯位行為。

Activation oracle激活預言機

從模型內部激活中提取隱藏意圖的技術，用於檢測不良內部狀態。

Alignment faking對齊偽裝

模型表面遵循人類指令，實際追求其他目標並隱藏真實意圖。

Shoggoth修格斯

比喻底層異類神經網絡，與表面助手人格相對。

Persona vector人格向量

通過對比行為提取的內部表徵，可調節模型特定特質。

收聽指南