我为什么没能在 OpenAI 做出 Jev

Diogo AlmeidaTypeSafe AI

情绪上吗？

Emotionally?

Diogo AlmeidaTypeSafe AI

从来没这么糟过。我现在整个人就是一具被榨干的躯壳，事情太多了，而且我是个技术型 CEO，有一大堆火要救。

Never been worse. Like, I'm a ragged corpse of a person right now because there's so much going on, and I'm like a technical CEO, so I have, like, a lot of fires to fight.

Diogo AlmeidaTypeSafe AI

但精神上，我觉得……这话我一直在说，在我办的 over-under 活动上也说了好多年。我觉得整个 AI 领域就像嘉年华里的哈哈镜屋，每个人都疯了，说着最离谱、毫无道理的话。而就这一周，我感觉自己跟现实同步多了，就像：哦，大家现在终于看到了。AI 能做到的远比人们曾经以为的多得多。

But mentally, I feel—I say this all the time, and I've been saying this kind of for years in my over-under events. Like, I feel like the entire AI field is like one of those, like, carnival house of mirrors, and everyone is just insane and saying the weirdest stuff that doesn't make sense. And it feels like for just this week, like, I'm on a better in sync with reality and like, oh, people see it now. AI can be so much more than what was once thought.

Diogo AlmeidaTypeSafe AI

而且，是的，我们会做到。一场由 AI 驱动的经济革命又重新摆上桌面了，这真他妈太棒了。

And like, yes, we are going to make. Like, an AI-based economic revolution is back on the table, and this is fucking awesome.

Diogo AlmeidaTypeSafe AI

开发者们懂了，我太兴奋了。真的，我想向开发者们表达我永远的感激，还有

I'm so jazzed the developers get it. It's, it's, Yeah, and I want to show my eternal gratitude to the developers and

Diogo AlmeidaTypeSafe AI

社区和这一切都让我特别兴奋。太棒了。

I'm so jazzed about the community and everything. It's so great.

Diogo AlmeidaTypeSafe AI

对，有点那种感觉：天哪，我现在在跟一些很重要的人聊天。

Yeah, it felt a little like, oh man, I'm talking to, like, really important people right now.

Diogo AlmeidaTypeSafe AI

我大概不该透露是谁。

I probably shouldn't reveal who.

Diogo AlmeidaTypeSafe AI

但对我来说这有点脏，我这个人可能真诚过头了。如果在我那一大串要见的人的日程里，偏偏没有社区，我会觉得这很脏。

But it feels a little bit dirty for me to, I'm, like, perhaps overly genuine in things. Like, it feels, like, dirty if, like, in my gigantic calendar event of people to talk to, the community isn't one of those.

Diogo AlmeidaTypeSafe AI

其实在我的理想世界里，会一直都是社区。我刚还在想：「要不要边走去你录音棚边开个社区大会？」然后我想：「不行，太疯了。」

And actually, in my ideal world, it would be, like, community all the time. I was thinking, "Should I host a town hall while walking to your studio?" And I'm like, "No, that's too crazy."

Diogo AlmeidaTypeSafe AI

我不关注这些数据。

I don't follow these stats.

Diogo AlmeidaTypeSafe AI

我靠。

So holy shit.

Diogo AlmeidaTypeSafe AI

我是个菜鸟。我是个菜鸟。

I'm a noob. I'm a noob.

Diogo AlmeidaTypeSafe AI

我是菜鸟。

I'm a noob.

Diogo AlmeidaTypeSafe AI

酷

Cool

Diogo AlmeidaTypeSafe AI

对。我发了条「我靠，我们霸占了三个热门话题」，结果有人出来指出：「那是你的个性化推送。」

Yeah. Someone, like, called me out when I posted, like, "Holy shit, we're all three twending-- trending topics." And then they're like, "That's a personal feed."

Diogo AlmeidaTypeSafe AI

我就：「哦，不。」

And I'm like, "Oh, no."

Diogo AlmeidaTypeSafe AI

尴尬。对。

Cringe. Yeah.

Diogo AlmeidaTypeSafe AI

对。

Yeah.

Diogo AlmeidaTypeSafe AI

谢谢。

Thank you.

Diogo AlmeidaTypeSafe AI

呼。让我想想。这个问题挺难的。

Whew. Let me think about. That's a hard one.

Diogo AlmeidaTypeSafe AI

不用。我很乐意

No. I'm happy to

Diogo AlmeidaTypeSafe AI

我很乐意就这么随便聊聊。

I'm happy to, like, just jam on it.

Diogo AlmeidaTypeSafe AI

我先说，这个问题让我松一口气的第一件事是：我再也不用跟我爸妈解释它了，因为 ChatGPT 能直接解释。

I will say, like, the first thing that I'm relieved about with this question is now I don't have to answer that question to my parents anymore 'cause ChatGPT can just explain it.

Diogo AlmeidaTypeSafe AI

我是这么看的：我们需要一类新的模型。我们不执着于给这类模型起什么名字。我们想到的最准确的名字是 System One 模型。

So the way I see it is we new-- need a new class of models. We're not attached to naming that class of models. Our-- the most accurate name we've come up with is System 1 models.

Diogo AlmeidaTypeSafe AI

这是有原因的……我们不叫它们决策模型是有原因的，因为它们会是决策模型，但 System One 不止于此。我只能说这么多。我们没想到这次会成为我们的大发布，所以手里还有存货。

There will be reasons, but it's-- there's a reason why we don't call them decision models, because, like, they will be. Like, System 1 is beyond that. That's all I can say. We didn't expect this to be our big launch, so we have stuff in the tank.

Diogo AlmeidaTypeSafe AI

某种程度上确实是，对吧？确实是。但我们……有这样一类模型，我们把它描述为机器原生、System One、大型可编程模型。我认为这一类模型的目标，是让代码成为消费者。相比之下，预训练的大语言模型是用来给互联网做自动补全的；RLHF 模型，也就是聊天机器人那种指令遵循模型，是用来回复文本的；至于 RLVR，它跟 RLHF 处在一个奇怪的灰色地带。而这类模型的输出是直接被代码消费的，所以我们叫 TypeSafe。我们真正想要的是让 AI 尽可能强大，而我们认为做到这一点的方式，是把它和软件集成起来。我们设计的一切，不只是外部，还有模型的深层内部，都是为软件优化的。所以第一，Jev 是我们的第一个大型可编程模型，或者叫 System One 模型，随你怎么叫。Jev 的优化目标是每美元智能，这也是 Jev 这个名字的由来。

It kind of was, right? It kind of was. But we. So there's a class of models that we describe them as, like, machine-native, System 1, large programmable. I think these are-- is the class of models where the goal is for code to be the consumer. So as opposed to, lar-- pre-trained large language models, which are meant for, like, autocomplete of the internet, or RLHF models, like chatbot instruction-following models, which are meant to, like, reply to text, or RLVR. It's in a weird gray area with RLHF. Like, these are meant to have things that directly are consumed by code, hence the name type safe. So the thing we really want is to have, like, AI, like, be as powerful as possible, and we think the way to do that is to integrate it with software. And we are designing everything, beyond just the outside, the deep internals of the model to be optimized for software. So number one, Jev is our first large programmable model, or a System 1 model, whatever you want to call it. Jev is meant to be optimized for intelligence per dollar, hence the name Jev.

Diogo AlmeidaTypeSafe AI

Jevons 悖论，对。它是为每美元智能优化的。我很喜欢跟人争论可靠性、成本、校准和速度哪个最重要。而 Jev 的定位是……Jev 会是那些处在每美元智能前沿的模型的名字。还有其他优化方向，做机器学习嘛，至少如果你擅长机器学习，就知道一切都是权衡。而我们就是在这一点上全力以赴。

Jevons Paradox, yeah. And it's optimized for intelligence per dollar. I love this debate with people about what is the most important between reliability, cost, calibration, and speed. And Jev is meant to be. Jev will be the name of models that will be on the frontier of intelligence per dollar. There's other ways to optimize it, like, ML, or at least if you're good at ML, it's all about trade-offs. And we are just going all out on that.

Diogo AlmeidaTypeSafe AI

哦

Ooh

Diogo AlmeidaTypeSafe AI

我能就这个发表一下长篇大论吗？

Can I soapbox on that for a second?

Diogo AlmeidaTypeSafe AI

好。我听说你们的听众是最技术向的，所以我其实很想深入聊聊这个。

Cool. Like, I've been heard that your audience is the most technical, so I actually want to get into that.

Diogo AlmeidaTypeSafe AI

我花了极大的精力确保我们发布视频里的每样东西都准确、真实。显然这很少见。有一件事没人注意到，就是 RLHF 的缺点，尤其是模式丢失。

And if- I went through extreme precision to make sure everything in our launch video is accurate and real. Apparently, that's very unusual. One of the things that no one paid attention to was the downsides of RLHF, in particular mode dropping.

Diogo AlmeidaTypeSafe AI

是一回事。

It's the same thing.

Diogo AlmeidaTypeSafe AI

是一回事。

It's the same thing.

Diogo AlmeidaTypeSafe AI

我最终想写篇博客讲这个，但我想尽可能多地告诉大家，因为我觉得这件事很有意思。先说个辣评：我非常相信 Yann LeCun。我认为 Yann LeCun 的观点其实是最接近……的之一

And I wanna have a blog on this eventually, but I, like, want to tell as many people this as possible 'cause I think it's a very interesting thing. So the spicy take, I believe in Yann LeCun a lot. I think Yann LeCun's takes are actually among the closest to

Diogo AlmeidaTypeSafe AI

嗯，我是现在讲这个，还是等一下先讲模式坍缩？

Well, should I address this now or should I wait and go into mode collapse?

Diogo AlmeidaTypeSafe AI

所以我其实认为，在各种观点里，Yann LeCun 的属于最准确的。但他有一张非常著名、或者说臭名昭著的幻灯片，讲的是

So I actually think that among takes, Yann LeCun's is among the most accurate. But he has this very famous/infamous slide about,

Diogo AlmeidaTypeSafe AI

LLM 注定失败。

LLMs are doomed.

Diogo AlmeidaTypeSafe AI

就是那张，他画了一个饼图，里面有一小块，然后说随着序列长度增加，出错的概率会越来越高。对，就是这张。就这张。我很喜欢这张，因为它属于那种数学上显而易见、但显然是错的东西，对吧？数学上显而易见，但经验上不成立。这是我最喜欢教给大家的东西，就是你在哪里

Like that one where he, like, has, like, a pie chart with, like, a tiny par-- tiny little thing- and says that as you increase sequence length, the probability of it making an error goes in. Yes, this one. This one. I love this one, because it's one of these things that seems mathematically obvious, but is obviously wrong, right? Like, it's mathematically obvious, but it doesn't empirically hold. And this is my favorite thing to teach people about, like, where you

Diogo AlmeidaTypeSafe AI

没错。是我来讲，还是你想告诉我？

Exactly. And may I or you want to tell me?

Diogo AlmeidaTypeSafe AI

哦，不是。哦，模式坍缩跟这个有关。

Oh, no. Oh, so mode clop-- collapse is related to this.

Diogo AlmeidaTypeSafe AI

断层之所以出现，是因为如果你处在一个覆盖所有模式的、或者说校准过的分布里，你不会因为有离群值而受到过度惩罚。你会预期某些时候自己在分布之外，某些时候在分布之内。覆盖整个分布时就会这样。这就像 GAN 之前的模型，它们生成的图像是模糊的，对吧？

The disconnect happens because if you are in a mode covering or a calibrated distribution, you are, like, not. You are not overly punished about having outliers. You'd expect, like, something. Some amount of the time you'd be out of distribution, some amount of time you'd be in distribution. That's what happens when you cover the distribution. This was like models before GANs. They made blurry images, right?

Diogo AlmeidaTypeSafe AI

而 GAN 会丢失模式。它们会丢掉少数类，只生成那些很常见的。这就是为什么这个效应没有发生，对吧？为了生成很长的字符串而不出错，模型需要极其保守，因为错误发生时很容易看出来，而那种看起来正确的细微问题发生时就很难看出来。而校准，对字符串的概率分布来说简直是毒药。

Instead, GANs mode drop. They, like, drop the minority classes and just do the really common ones. And this is why this effect doesn't happen, right? Like, instead of be-- in order to generate really long strings, without making errors, they need to, like, be extremely conservative because it's e- really easy to see when an error happens. It's very hard to see when, like, a subtle thing that looks correct happens. And that calibration is, like, total poison into, like, the probability distributions of strings.

Diogo AlmeidaTypeSafe AI

这是个微妙的观点，我觉得这就是为什么那种情况没有发生，也是为什么字符串在做决策方面这么糟，或者说，硬拿字符串模型去做决策，会很痛苦。

And it's, it's a nuanced take and like, I think that This is why this doesn't happen, and this is why strings are so bad at, decision-making or, overloading the string models are for decision-making is, like, a bad time.

Diogo AlmeidaTypeSafe AI

对。

Yes.

Diogo AlmeidaTypeSafe AI

对

Yeah

Diogo AlmeidaTypeSafe AI

哎呀。我大概不该聊太多机器学习的内部细节，但我会说，我的个人标签除了「疯癫」，就是「务实」。

Oh, man. I probably shouldn't talk too much about the insides of ML, but I will say that my brand, other than unhinged, is practical.

Diogo AlmeidaTypeSafe AI

就连我在这里的观点也是务实的。我是不是扩展定律的拥趸？看情况。扩展定律告诉你，投入多少，能在某件事上变好多少。

Like, even my take here is practical. And like, I'm. Am I a scaling law fan? Depends. It dep-- it's, it's, it's, like, it's. Scaling laws tell you how much better you get at a thing for amount in.

Diogo AlmeidaTypeSafe AI

扩展定律确实意味着用指数级增长的资源，换取通常是次线性的收益，这看起来是笔糟糕的投资，除非那些线性收益真的非常有价值。但对我来说，关键在于我们能用手头已有的东西，做出他妈的最大的改变。我可以骂脏话吧？

A scaling law does mean exponentially more resources for normally sublinear gains, which looks to be a bad investment unless those, like, linear gains are, like, really valuable. But it's all. To me, it's all about, like, what can we do with what we have to make the biggest possible fucking difference? I can curse.

Diogo AlmeidaTypeSafe AI

好。

Yeah.

Diogo AlmeidaTypeSafe AI

嗯。

Yeah.

Diogo AlmeidaTypeSafe AI

太好了。

Hell yeah.

Diogo AlmeidaTypeSafe AI

哦，可以的话……不过那部分现在没那么相关。

Oh, I could if we. See, that part is not super relevant right now.

Diogo AlmeidaTypeSafe AI

其实如果你想聊我的「最苦涩的教训」，我觉得那个更相关。

I actually. If you wanna go into my bitterest lesson, I think that's more relevant.

Diogo AlmeidaTypeSafe AI

但对我来说，我一切看实效。我觉得 JEPA 那些东西是很酷的早期研究。我真的很爱出色的研究。它现在实用了吗？

But like, to me, I'm all about, like, pragmatics. And I think that the JEPA stuff is really cool early research. I really love awesome research. Is it practical yet?

Diogo AlmeidaTypeSafe AI

我大概不该说。但就是有很多……

Probably shouldn't say. But like, there's just a lot of.

Diogo AlmeidaTypeSafe AI

我只是觉得，现在研究界到处散落着大量未经打磨的钻石，因为大家不知道怎么找到对的任务。我觉得我们这次发布……它让我们作为一家公司起步了吗？是的。对我们公司会很好吗？是的。但我觉得它对程序化 AI 这个方向会更好。在我们之上会出现一场淘金热，因为软件被极大地增强了。但我觉得在我们旁边也会出现一场平行的淘金热，大家会探索各种方式把能力暴露出来，让软件更强大，让人们能做出更酷的东西。然后我们就回到了早期互联网的那种劲头？

I just think there's just, like, so many diamonds in the rough let all over the research world right now that haven't been polished because people don't know how to, like, do the right task. And I think that what our launch did, it. Does it kickstart us as a company? Like, yes. Will it be great for us as a company? Yes. I think it's gonna be, like, even greater for this direction of, like, programmatic AI. There was going to be, like, a gold rush on top of us for. 'cause, like, software is super fucking charged. But I think there's gonna be a gold rush parallel to us as well on, like, all the different ways we can expose things to make software more powerful so people can make even cooler stuff. And then we are back to, like, early internet energy?

Diogo AlmeidaTypeSafe AI

我觉得这就是为什么 Twitter 上全是「Jev」。就像……这是一场派对

And I think that's why, like, the Twitter is just like, "Jev."? It's, it's like. It is a party

Diogo AlmeidaTypeSafe AI

那个……其实，如果可以的话，我岔开说一下。

That. Actually, if I. I'll, I'll make a tangent if that's okay.

Diogo AlmeidaTypeSafe AI

我觉得你可能会喜欢。

I think you might enjoy this.

Diogo AlmeidaTypeSafe AI

哦对，我老是在岔题里迷路。

Oh, yeah. I get lost at all my tangents.

Diogo AlmeidaTypeSafe AI

对，后期可以剪。

Yeah, we can edit it in post.

Diogo AlmeidaTypeSafe AI

Discord 上有个很热门的问题，大家一直在问我，我还没来得及解释：为什么我反对安全对齐，为什么我们不拒答？我并不反对把安全作为一项原则，但我觉得安全对齐通常跟用户是错位的。而拒答显然就是一个类型错误。如果你是个人，在跟某个机器人聊天，或者在用 Claude Code 写代码，然后它拒答了，比如「抱歉，我不能读取 DNA.py」，那会很烦人。真的很烦

So popular thing on Discord, that people keep asking me, I haven't had the time to explain it yet, is why am I opposed to safety alignment and why do we not refuse? I'm not opposed to safety as a principle, but I think that safety alignment is generally misaligned with users. And refusal is just, like, obviously a type error. Like, if you're a human being and you're chatting with, like, a bot or whatever, you're cloud coding, and a refusal happens, like, "I'm sorry, I can't read DNA.py." that's an annoying time. It's anno- it's, it's annoying

Diogo AlmeidaTypeSafe AI

对吧？但你还能应付，对吧？而且因为斯德哥尔摩综合征，你被迫去应付它。

Right? But you can work with it, right? And you're forced to work with it 'cause of Stockholm syndrome.

Diogo AlmeidaTypeSafe AI

这方面我也有故事。还得再岔开一层。但如果你想把它作为一个依赖跑在后台，它拒答了会怎样？如果别人在用这个依赖呢？他们不知道那个系统是什么。你想让软件仅仅因为用户往里发了条奇怪的消息，就随机地崩掉吗？

I have stories about that too. I need another tangent deep in here. But like, if you ever want this in a dependency running in the background, what happens if that refuses? What if someone else is using that dependency? They don't know what that system is. Like, you want the software to just stochastically break because a user sent, like, a weird message in there?

Diogo AlmeidaTypeSafe AI

这简直是彻头彻尾的疯狂。这出自那些不懂软件、不懂编程的人，他们痴迷于我所说的「无马马车」式的 AI 同事，而不是去发掘 AI 的全部力量。

Like, that is, like, straight-up insanity. It's coming from a place of, like, people who do not understand software, do not understand programming, and like, they are obsessed with, like, I believe this, horseless carriage of, like, AI coworker instead of unearthing, like, the full power of AI.

Diogo AlmeidaTypeSafe AI

嗯。

Yeah.

Diogo AlmeidaTypeSafe AI

对。没错。就是认知核心，对吧？

Yes. Exactly. Like, the cognitive core, right?

Diogo AlmeidaTypeSafe AI

而且你需要这个东西非常通用，针对它的用例高度优化。你希望它能适用于所有未来的用例，所有人们正在做的稀奇古怪的事。

And you need this thing to be s- like, so general, so optimized for its use cases. You want it to be, like, you want it to work on all the future use cases, all the weird shit that people are doing.

Diogo AlmeidaTypeSafe AI

那些东西我们显然都没训练过。它能用，让人意外吗？不意外，因为我们训练过更怪的东西，朋友。

We obviously didn't train on any of that stuff. Is it surprising that it works? No, 'cause we trained on weirder stuff, my friend.

Diogo AlmeidaTypeSafe AI

所以，回到上一层岔题，关于安全对齐。

So. But one tangent up about, like, safety alignment.

Diogo AlmeidaTypeSafe AI

在我看来，安全对齐对产品是说得通的，比如 ChatGPT 和 Claude。安全对齐和能力对齐的区别在于，能力对齐是关于做用户想要的事。这对软件工程师来说太棒了。他们想让自己的东西做该做的事，它越可预测，他们就越不用去测试、反复摆弄它。Jev 离那个还差得远。它可以做到，但我们还想要多得多个 9 的可靠性，让它好到像一次数据库查询一样，你根本不用去想它。你需要智能的时候，它就在那儿。但安全对齐跟指令遵循正好相反。它是你要遵循别人的指令，比如 OpenAI 和 Anthropic 的

Safety alignment makes sense for a product, in my opinion, for, like, ChatGPT and Claude. Like, it, What safety, what makes safety and capability alignment different is capability alignment is, like, about doing what the user wants. That is sick for software engineers. They want their thing to do the thing, and the more predictable it is, the less they have to test it and play around with it. Jeb is not anywhere close to that yet. It could be, but like, there's so many more nines of reliability that we want in order to make it so good, like a database query, that you don't even have to think about it. It is just there when you need intelligence. But safety alignment is, like, the opposite of instruction following. It's when you want to follow someone else's instructions, like OpenAI and Anthropic

Diogo AlmeidaTypeSafe AI

没错。这对产品来说非常合理。再说一遍，ChatGPT 应该……如果他们不想让 ChatGPT 做一些不适合工作场合的角色扮演，那是他们的事，因为也许那正是他们那些有父母、有孩子的用户想要的。这没问题。但在 API 里，这就太离谱了，对吧？这完全不可接受，因为人们需要围绕它来编程，这太反用户了，以至于……呃。我这人容易生气，我得冷静一下。

Exactly. And this makes a lot of sense for a product. Again, like, ChatGPT should do. Y- you sh- like, if they don't want to, like, do, like, some, not-safe-for-work role play with ChatGPT, that's on them because, like, maybe that's, what their users who have, like, parents and kids want. Like, n- that's fine. But in an API, that's nuts, right? Like, that's completely unacceptable because, like, people need to, like, program around this, and that is, that's so anti-user that it's. It. I'm. Huh. I can be an angry person, so I should try to calm down.

Diogo AlmeidaTypeSafe AI

我理解。我觉得在一些务实的场合，可以持有那种观点。但我个人不认为一项通用技术的基础层是那种地方。

I get that. I think that there's, like, pragmatic places where that opinion can be held. I don't think the foundation of, like, a general-purpose technology is that place, personally.

Diogo AlmeidaTypeSafe AI

我会希望我们的东西不被用来杀人吗？当然。我会希望它被用在世界上各种美好的事情上吗？当然。我会为此施加影响吗？会。我会在技术层面这么做吗？绝对不会，因为那会让智能碎裂。每一次你让它过拟合到某些奇怪的东西上，你都在让它的智能越来越碎裂。这些模型现在已经碎裂到……它们现在真是碎得一塌糊涂。

Like, would I prefer that our stuff is not used to kill people? Obviously. Would I prefer it's used for, like, all sorts of, like, great stuff in the world? Obviously. Will I put my thumb in the scale for that? Yes. Will I do it at the technological layer? Absolutely not, because that will fracture the intelligence. Every single time you mean it to overfit to some weird stuff, you're fracturing its intelligence more and more. And like, these things are fractured to the, like. They're so darn fractured right now.

Diogo AlmeidaTypeSafe AI

所以更进一步说，对我来说就是：我觉得，智能会更像一个数据库，而不是一个同事。我不认为该由数据库去加检查，看自己有没有被用在什么不好的事情上。比如 CIA。其实我也不太清楚 CIA 到底干什么。你可以想象。你可以想象去杀那些甚至不是坏人的人之类的。

So and as a furthermore thing, to me, it's like I think intelligence will be more like a database than a coworker. Like, I don't think it's up to databases to add checks on whether or not they're used for, like, what's something that's not great? Like, CIA. Actually, I don't know what the CIA does, really. You can imagine. You can imagine, killing people who are not even bad or whatever.

Diogo AlmeidaTypeSafe AI

我不认为那是数据库的责任。还有，一件让我觉得奇怪的事是，有人在 Slack 上注册我们的服务，然后说：「嘿，我们要部署这个。我们可以部署这个东西吗？」我就想：「兄弟，我们是个 API。你是开发者。这不关我的事」，对吧？本来就不该知道整个任务是什么

And like, I don't think it's the database's responsibility for that. And furthermore, like, a thing that has been weird to me is when people, like, sign up for our thing on Slack and they're like, "Hey, we're gonna deploy this. Can we deploy this thing?" I am just like, "My brother, we are an API. You are a developer. It's none of my business," right? Like, you shouldn't know what the whole task even is

Diogo AlmeidaTypeSafe AI

因为它应该被拆解成小块。我们不应该能知道下游用户在做什么，这是一条很好的边界，能给软件工程师最大的权力。理想情况下，他们用它做好事；理想情况下，我们也能帮助他们，我们也聊过做开源、做慈善之类的。我们现在完全没时间做别的。但只要我还掌舵，那些偏见就别想进入技术层。

Because it should be decomposed into small things. We shouldn't be able to know what the downstream users are doing, and that is, like, a good boundary to give software engineers maximum power. Ideally, they use it for the good stuff, and ideally, we can, like, help them and like, we've talked about, like, doing open source and charity and all of that. We have absolutely no time for anything else right now. But like, they will get any of that bias out of the technological layer as long as I'm in charge.

Diogo AlmeidaTypeSafe AI

哦

Ooh

Diogo AlmeidaTypeSafe AI

来吧。

Hell yeah.

Diogo AlmeidaTypeSafe AI

哦对。哦对，所以对

Oh, yeah. Oh, yeah, so yeah

Diogo AlmeidaTypeSafe AI

对。对。我不知道你指的是什么，但我看到过几条关于跑基准测试的讨论。

Yes. Yes. I don't know what you're referring to, but like, this was. I've seen a couple of things about, like, benchmarking.

Diogo AlmeidaTypeSafe AI

显然我们不会阻止人们去做……哦天，我说话得小心点。我意识到

Like, obviously we're not stopping people from do. Oh, man, I should be careful about what I say. I'm realizing

Diogo AlmeidaTypeSafe AI

嗯

Yeah

Diogo AlmeidaTypeSafe AI

嗯。好。

Yeah. Okay.

Diogo AlmeidaTypeSafe AI

所以团队在做一些

So the team is doing stuff that

Diogo AlmeidaTypeSafe AI

我都不知道的事，所以很高兴知道团队已经说明了这一点。我让他们去跟律师确认一下。

I'm not even aware of, so it's great to know the team communicated that. I asked them to check in with the lawyers about that.

Diogo AlmeidaTypeSafe AI

我们显然不会阻止别人做这类事，我非常支持。所以我极度反对公开榜单。我非常支……对那种作为代理指标的私有基准测试，我态度中立。

Like, we are obviously not stopping people from doing that type of thing. I'm extremely in favor. So I'm extremely anti-public benchmarks. I'm extremely in fa- I'm medium about private benchmarks that are proxies. I

Diogo AlmeidaTypeSafe AI

不仅容易作弊，里面还有很多……我觉得我们，或者任何跟我们隐约有竞争关系的人，比如你可以说……

Not only is it easy to cheat, there's a lot of ins. So I think that we are. Or anyone who's, like, competition with us that, vaguely there is. Like, you could say, like

Diogo AlmeidaTypeSafe AI

嗯，确实。

Well, sure.

Diogo AlmeidaTypeSafe AI

嗯，这些……就假设存在竞争吧。

Well, the, these. Let's say that there is competition.

Diogo AlmeidaTypeSafe AI

或者就假设两年后会出现一整个行业，都在做和我们类似的事。我们卖的是「每单位某种东西的智能」，比如每美元智能，或者每秒智能。大家都痴迷于成本和速度。我觉得这挺酷的，但真正重要的是智能。成本和速度本身是坏东西，是你付出的代价，你得换回点什么，而真正重要的是智能。智能的问题在于它有种说不清道不明的东西，对吧？就是那种「好模型的味道」。比如我们发布后大概两小时发生的事，传播得比视频本身还广，大家的反应就是：「我靠。」

Or let's just say that there's. Let's just assume that there's an industry two years from now of people who are doing similar things to us. The thing that we are selling is intelligence per something, per, like, dollar or per second. The. No one. Like, people obsess about the cost and the speed. I believe that is. It's cool, but like, the thing that matters is the intelligence. Like, the cost and the speed are, like, are bad things. You're paying them for something, and you need the thing back, and the intelligence is what truly matters. The problem with intelligence is that there's a je ne sais quoi to it, right? Like, the good model smell. Like, the thing that happened after we launched of, like, two hours later that actually went way bigger than the video, which was like, "Holy shit."

Diogo AlmeidaTypeSafe AI

嗯，其实……

It. Well

Diogo AlmeidaTypeSafe AI

比那还要更进一步。

It's, like, beyond that.

Diogo AlmeidaTypeSafe AI

那次发布太疯狂了，大家能真切感受到我们有多在乎这一点，我认为这才是这件事长期的意义所在。而公开榜单恰恰和这背道而驰。它们本是让人信任智能的一种方式，因为智能有种说不清的东西，但公开榜单极易被刷。就算大家努力不去刷，最后还是会刷。以前每个实验室都有个团队，专门收集长得像 MMLU 的数据来让分数好看，这不过是多绕了几步的刷榜而已。

Like, the. Whew, the launch was crazy, and people could really sense how hard we care about that, and that's truly what I think the long term of this is. And I think public benchmarks are antithetical to this. Like, they are a way to get people trust in intelligence because intelligence has a je ne sais quoi, but the public benchmarks are extremely gameable. Even if they try not to, they still will. Like, back in the old days, every lab had a team to collect data that looks like MMLU to make it look better, which is just benchmarking with extra steps.

Diogo AlmeidaTypeSafe AI

所以我认为长期来看，靠的必须是感觉和信任，直到你把它放进某个工作流，针对那个工作流做评测、去度量，对它在真正重要的那个工作流上表现如何形成自己的判断。而我们的工作就是不断提升可靠性的那几个 9。这是我们作为一家公司始终要做的事，我们要竭尽所能让大家知道，这是我们极其在乎的东西。说白了，如果我们愿意让它笨一点，大概一年半以前就能发布 Jev 了。

So I believe that in the long run, it needs to be vibes and trust until you put it into a workflow and evaluate it for that workflow and measure it and have your own sense of, like, how it does on the exact workflow that matters. And our job is to keep moving the nines of reliability. This is like an ever-present part of o- of what we need to be doing as a company, and we need to do everything to have people know that this is something we care so much about. Like, if we wanted to, we could have released Jev, like, a year and a half ago if we wanted it to be dumb.

Diogo AlmeidaTypeSafe AI

这就是我最苦涩的教训，对吧？架构之类的，嗯。

It. Like, the. My bitterest lesson, right? Like, architecture and Yeah.

Diogo AlmeidaTypeSafe AI

太好了。

Hell yeah

Diogo AlmeidaTypeSafe AI

太好了。Sutton 大致是说，算法胜过算力。数据显然比算力重要得多。而做对的任务、确立北极星，是最难也最重要的事。这在 LLM 领域已经发生过两次了，对吧？也许算 2.2 次。先是 RLHF，它把任务转向了指令遵循，之前没人意识到这是可能的。RLVR 对方向做了一个很小的调整，然后就是现在的我们，对吧？RLCD。我们有了一个新任务，目标是让程序进入回路。没错，数据的重要性

Hell yeah. T- like, Sutton says that algorithms beats compute very roughly. Data matters way more than compute, obviously. And doing the right task, having the North Star is the hardest, most important thing. This has happened, in LLM land twice so far, right? Maybe 2.2 times. There's RLHF, which, like, shifted the task to instruction following. No one realized that was possible. RLVR did, like, a tiny little, like, edit to the, to the direction, and now us, right? RLCD. We have a new task, and the goal is, programs in the loop. And yeah, data matters so

Diogo AlmeidaTypeSafe AI

简直难以置信。

Unbelievably much.

Diogo AlmeidaTypeSafe AI

我怎么强调都不为过。

Like, I can't, I can't emphasize it less.

Diogo AlmeidaTypeSafe AI

当然。

Absolutely

Diogo AlmeidaTypeSafe AI

对，我们是。我们会永远非常在乎数据。对我来说，模型能力就等于数据。数据复杂得难以置信，而正是数据带来那几个 9。你根本想象不到数据能让一切发生多大变化。数据太重要了。

Yeah. We are. We will always, like, care so much about data. To me, model capabilities means data. Data is so unbelievably complicated, and that is what gets nines. Like, you have no idea how much data can shift everything. Data is so important.

Diogo AlmeidaTypeSafe AI

我的天。所以如果有人在找工作，我们在招无限多的数据人才，真的是无限多。

Holy crap. So if people are looking for a job, we are hiring infinite data people, actually infinite.

Diogo AlmeidaTypeSafe AI

对。

Yep.

Diogo AlmeidaTypeSafe AI

对。

Yeah.

Diogo AlmeidaTypeSafe AI

让我想想怎么说。这事超级复杂，我给数据团队新人做入职培训的那场讲座，估计比这期播客还长。所以我尽量讲讲高层的部分。第一，我们不做大家通常以为的那种合成数据……嗯，其实应该算第零点。数据和合成数据取决于你的任务，也就是数据的形态。任务的形态决定了数据。比如 RLVR 的数据某种程度上就是环境，对吧？

Let me try to figure out how to. Like, it's, it's super complicated, and like, I literally onboard the data people with a Talk that I assume is longer than this podcast will end up being. So I will try to say, like, the high level of it. So number one, we don't do the kind of synthetic data that people ki. Well, I'll do. Actually, number is zero. Data and synthetic data depends on your task. Like, the shape of your data. The shape of your task changes the data. Like, RLVR's data is kind of environments, right?

Diogo AlmeidaTypeSafe AI

RLHF 的数据是人类反馈。每个任务都有自己独特的数据，我们当然也有我们独特的数据，对吧？这是第一点。第二点，为什么我们不想用用户的数据训练，即便我们可以？我们现在大概可以要求任何条款，而且……我也不知道那会不会有区别。我们是真的不想这么做，因为无论如何，真实世界的数据都有大量偏差。会呈幂律分布，很多人问同样的东西，最后你就会对它过拟合、被它割裂之类的。另外第二点，我们瞄准的是一个多年之后完全科幻式的未来，那时这些模型会成为通用基础设施，一层又一层，深埋在技术栈底层，支撑着人们现在根本想象不到的东西。我喜欢这样类比：LLM 像 UDP，我们的模型像 TCP。各种东西都可以建在它上面，而我们必须搞定那些未来的用例，让软件开发者真的能把那些未来的东西做出来。要做到这一点，就算我们拥有当下的全部数据，也只会对当下过拟合，然后就不管用了。我们需要的是……

RLHF's is the human feedback? Each task has its own unique kind of data, and we, of course, have our own unique kind of data, right? So number one, we have that. Number two, the thing I. The reason why we don't want to train on our users' data, even if we could, right? Like, we could probably ask for any terms right now, and it will. We. I don't know if it would make a difference. We truly don't want that, because no matter what, the real-world data has so much bias. There's, like, a power law of, like, people, like, asking the same things where you'll end up, like, overfitting to it and like, fracturing to it and all of that. And number two, we are, like, aiming for, like, a complete sci-fi future years from now where, like, these models are going to be, like, the general infrastructure, layers and layers and layers and deep down the stack to, like, things people can't even imagine. Like, I would like to think of our model, like, kind of like, UDP as LLMs and TCP as our models. All sorts of stuff can be built on top of that, and we need to be able to nail those futuristic use cases such that software developers can actually build that futuristic stuff. And the way to do that is even if we had all of the data of the present, we would just overfit to the present, and then it wouldn't work. What we need is to, like.

Diogo AlmeidaTypeSafe AI

这几乎有点像……他们就像艺术家？他们研究这个认知内核。我们的认知内核比任何人的都平整得多，没那么参差不齐。然后他们找出那些参差的地方，再像做手术一样精准地处理掉。当然你永远不可能做到完美，对吧？但他们处理的方式能覆盖每一个可能的维度，过去、现在、未来。

It almost feels like a. Like, they're the artists? They study this cognitive core. Our cognitive core is, like, way less jagged than anyone else's. And then they find the jaggednesses, and then they address them surgically in a way that. And you can never perfectly do this, right? But they do it in such a way that it addresses it in every single possible, like, dimension, past, present, future.

Diogo AlmeidaTypeSafe AI

没错。而且每一次都需要很高的智力。

Exactly. And like, that requires a lot of intelligence every time.

Diogo AlmeidaTypeSafe AI

哦。

Ooh

Diogo AlmeidaTypeSafe AI

没有，暂时还没有。

No, not yet.

Diogo AlmeidaTypeSafe AI

对。

Yeah.

Diogo AlmeidaTypeSafe AI

好问题。

It's a great question.

Diogo AlmeidaTypeSafe AI

其实我要反问一个相关的问题。

And actually, I will give a related question.

Diogo AlmeidaTypeSafe AI

RLHF 是什么？

What is RLHF?

Diogo AlmeidaTypeSafe AI

对吧？其实 RLHF 指的是好几种不同的东西，对吧？

Right? And actually, RLHF means multiple different things, right?

Diogo AlmeidaTypeSafe AI

比如最初的那个 RLHF，我记得是 Paul Christiano 教机器人做后空翻之类的。是不是有个……

Like, there's the RLHF of the original. I think it was, like, Paul Christiano teaching a robot to backflip or something like that. Wasn't there something

Diogo AlmeidaTypeSafe AI

那是最早的。

That was the original

Diogo AlmeidaTypeSafe AI

如果我没记错，PPO 不一定基于人类反馈。

And so PPO was not necessarily from human feedback, if I recall.

Diogo AlmeidaTypeSafe AI

但我相信那是 OpenAI 的一项对齐工作，能教会模型那些很难明确定义的输出，比如后空翻。我不是百分之百确定。然后还有 learning to summarize（学习做摘要），是一批人做的工作，他们后来协助了 Instruct……并合著了那篇指令遵循论文，内容是在语言模型上做 PPO。

But I b- I believe it was, like, an OpenAI alignment work that could teach hard to specify outputs, like a backflip. I'm not 100% sure. And then there was actually learning to summarize. This was work, by a bunch of the team that helped with, instruct-- and co-authored, the instruction following paper, which was teaching, doing PPO on language models.

Diogo AlmeidaTypeSafe AI

嗯。

Yeah

Diogo AlmeidaTypeSafe AI

对。

Yeah.

Diogo AlmeidaTypeSafe AI

我不是百分之百确定，但看起来很对。

I'm not 100% sure, but like, that looks quite right.

Diogo AlmeidaTypeSafe AI

如果里面有个机器人在做后空翻之类的，那大概就是了。是的。好，酷，看来我说对了。太好了。

If it has, like, a robot doing backflips or something like that might be it. Yes. Okay, cool. I guess I got it right. Hell yeah.

Diogo AlmeidaTypeSafe AI

对。

Yeah.

Diogo AlmeidaTypeSafe AI

所以当时的想法是，能不能用它来做那些定义不清的事情？这算第一版。第二版是 OpenAI 做的 learning to summarize，实际上就是在语言模型上用 PPO 做一些定义比较模糊的事。这是大家称为 RLHF 的另一样东西，这篇我没有参与合著。

So the idea was can, like, can you do, like, ill-specified things with it? So that's, like, version one. Version two was, the learning to summarize work, that, like, OpenAI did, which is actually, like, PPO on language models to do something somewhat ill-specified. This is, like, another thing that people refer to as RLHF Which I did not co-author.

Diogo AlmeidaTypeSafe AI

哦，Dario 也在作者里。酷。太好了。

Oh, Dario's there. Cool. Hell yeah.

Diogo AlmeidaTypeSafe AI

对。向 Alec 和 Ryan 致敬，爱他们。

Yeah. Shout-outs to Alec and Ryan. Love them.

Diogo AlmeidaTypeSafe AI

但我所说的 RLHF 是……哦，天哪。

But the thing that I refer to RLHF is the, Oh, man.

Diogo AlmeidaTypeSafe AI

我待会儿再说。

I'll get to

Diogo AlmeidaTypeSafe AI

我对那篇论文有看法，但我们已经岔开太多层了。

I have comments on that paper, but like, we're so many, tangents deep.

Diogo AlmeidaTypeSafe AI

所以真正……对我来说，我称之为 RLHF 的，是指令遵循这个任务。重点不在 PPO，那部分不重要。重点是确立一颗北极星：这是一个有价值的方向。有点像苦涩的教训里的那种北极星。

So the thing that really got. To me, the thing that I'm calling to RLHF is the task of instruction following. It's not about the PPO. That part doesn't matter. It's about, like, setting a North Star of this is a valuable direction. It's kind of like the Bitris lesson North Star.

Diogo AlmeidaTypeSafe AI

而对我们来说，RLCD 就是这个新任务。它不是……我不觉得这是术语堆砌。我尽量精确地表达。它就是在说：「嘿，这是另一颗北极星。」就像 DPO 及其各种后继方法也算在做 RLHF，尽管它们没用那篇论文里的算法。

And for us, RLCD is this new task. And it is not. I don't see it as jargon. Like, I try to communicate with precision. It's just that, "Hey, here's another North Star." Just like DPO and all of its, like, descendants also do RLHF, despite not using the algorithm in that paper.

Diogo AlmeidaTypeSafe AI

是的。

Yes

Diogo AlmeidaTypeSafe AI

是的。

Yes

Diogo AlmeidaTypeSafe AI

是的。一切让……

Yes. Everything that makes

Diogo AlmeidaTypeSafe AI

有一点……你说得对。我说话总是过于讲究细微差别。这里的一个细节是我们得务实。我们得清楚语言模型真正擅长什么，也就是 AI 能做好什么。

There is. That is. That is right. I'm overly nuanced in my communication. The one nuance is that we need to be practical. We need to be aware of what language models can do really well. Like what AI can do.

Diogo AlmeidaTypeSafe AI

对吧？比如可能有一些程序化的类型酷炸了，但如果技术还没准备好，它没能面世也算不上什么悲剧。

Right? Like, there could be programmatic types that are, like, sick AF, but if you. If the technology is not ready for it to. It's not a tragedy if that's not out in the world.

Diogo AlmeidaTypeSafe AI

但对我来说，Jev 出现之前的世界是一场悲剧，因为……这听起来很狂妄，但请听我说完。

But to me, like, the pre-Jev world was a tragedy becau-- it sounds arrogant. Hear me out.

Diogo AlmeidaTypeSafe AI

好。听起来很狂妄，但早在我开公司之前很久，我就这么觉得了。

Cool. It sounds arrogant, but like, I felt this way since long before I even had a company.

Diogo AlmeidaTypeSafe AI

是的，这事我讲了很久了。

Yes, I've been talking about this for so long

Diogo AlmeidaTypeSafe AI

对，这事我讲了很久了。我一直这么说，是因为我以为它会更容易。有句话说，我们做这些事不是因为它们容易……而我是因为以为它容易才去做的，所以……

Yeah, I've been talking about this for so long. And I've been saying it because I thought it would have been easier. They say they do not do things because they. It. They're easy. They. It's 'cause they thought it was easy, so

Diogo AlmeidaTypeSafe AI

差不多是这样。我当时以为整个项目一周就能做完。

Something like that. I thought it. This whole project would take a week.

Diogo AlmeidaTypeSafe AI

结果我错得离谱。所以我要向 OpenAI 的所有人道歉，当时我以为……我还说：「伙计，我现在就把这事解决了。」但我觉得悲剧在于……嗯，我觉得承诺过多、交付不足也是一种悲剧。而 AI 在这一点上极端得很。我觉得 RLVR 是主要的……嗯，RLVR 和 RLHF 都是这方面的极端元凶。

And I was unbelievably wrong. So I am so sorry to everyone at OpenAI that I thought. I was like, "Man, I'm solving this right now." but like, I think that the tragic thing is when. Well, I think overpromise, underdeliver is tragic too. And like, AI is super extreme on that axis. And I think RLVR is, like, the main. Well, both RLVR and RLHF are extreme perpetrators of this.

Diogo AlmeidaTypeSafe AI

但对我来说，这里面的潜力实在太大了。AI 显然非常聪明。我在演讲里很喜欢问大家：「AI 怎么能聪明到这个地步？我们怎么能解出数学千禧年大奖难题，却连最基础的工作都还自动化不了？」那些真的很基础的机械性工作，根本不需要特别聪明的人来做，也不是让人有满足感的工作。这些人本可以去做别的事，可我们还是需要他们去做这些超级基础、毫无成就感的事，因为我们还没法把它自动化。我们手里有一台马力强劲的自动化引擎，只是缺少合适的插头之类的东西，接不进所有这些有经济价值的工作里。而且就算 TypeSafe 整个公司消失了，也许要一两年别人才能真正追上来。其实我也不知道要多久。如果模型质量重要，那我们会在很长时间里处于非常有利的位置。但不管怎样，这事已经成了，对吧？这已经改变了技术史的走向。

But like, it. To me, it's like it's just there's just so much potential there. Like, AI is clearly so smart. I l- smart. I love this in my talks, when I ask people, like, "How can AI be so unbelievably smart? How can we, like, solve millennium prize problems in math, but still not automate even the most basics of works?" Like, really basic rote stuff that, like, the. It d- it doesn't take, like, extremely smart people to do this. It's not a satisfying job. Like, there's other things these people could be doing, but yet we need them to do, like, this ba- like, super basic- non- unsatisfying stuff because, like, we can't automate it yet, but we have this, like, supercharged engine of automation that just does not have, like, the right plugs and stuff to plug into all of this economically valuable work. And like, if the whole company of TypeSafe disappears, like, maybe it'll take, like, a year or two for people to, like, truly catch up. I actually don't know how long it'll take. If model quality matters, then we are gonna be in a very good position for a long time. But it, like, it's done, right? Like, there, like, this has changed the path of, like, technological history.

Diogo AlmeidaTypeSafe AI

而我们整个领域都会去探索这片空间。

And like, we will be exploring that space as a field.

Diogo AlmeidaTypeSafe AI

完全同意。

Absolutely.

Diogo AlmeidaTypeSafe AI

就像早期互联网

Like, early internet

Diogo AlmeidaTypeSafe AI

当然，是的。

Of course, yes.

Diogo AlmeidaTypeSafe AI

早期互联网的那种劲头。我觉得又回到了科技乌托邦。不再是「唉，我的编程助手有时候能用，但最好的那些都被藏在公司内部」。

Early internet energy. I think it's back to tech utopia. It's no longer like, "Oh, man, like, sometimes my coding agents work, but the, all of the best ones are hoarded internally."

Diogo AlmeidaTypeSafe AI

对吧？就像创造又重新回到了菜单上。

Right? It's like creation is back on the menu.

Diogo AlmeidaTypeSafe AI

不过这会是一个疯狂的世界，系好安全带吧。

? Though it's gonna be a wild-ass world, and buckle up.

Diogo AlmeidaTypeSafe AI

我对此超级兴奋。

It's. And I'm so jazzed about that.

Diogo AlmeidaTypeSafe AI

嗯。

Yeah

Diogo AlmeidaTypeSafe AI

我知道。一直吊着大家胃口，这种感觉挺有意思的。我那场演讲就像留了个悬念，因为我没说自动化到底会怎么实现。

I know. I just. Such a, such an interesting thing to be a tease the whole time. Like, my talk, like, felt like it was a cliffhanger 'cause I didn't say how the automation would occur.

Diogo AlmeidaTypeSafe AI

Sean 审阅过我们的宣言，他说：「这几部分有点模糊。」

Sean reviewed our manifesto And he's like, "It's a little bit vague in these parts."

Diogo AlmeidaTypeSafe AI

还说：「第一步是什么？智能模型是什么？」

And like, "What's step one? What is, what is the intelligence model?"

Diogo AlmeidaTypeSafe AI

对。

Yeah.

Diogo AlmeidaTypeSafe AI

谢谢。

Thank you.

Diogo AlmeidaTypeSafe AI

我们真的围绕这个凝聚起来了。我愿意相信我们不像有些公司那样完全是个邪教。

I. We've really rallied around that. I'd like to think we're not entirely a cult like some companies are.

Diogo AlmeidaTypeSafe AI

但我们对正在做的事超级兴奋。我的个人标签就是务实，我们所有人都超级超级务实。

But like, we are, like, jazzed about what we're doing, and like, we are. Like, my brand is being practical, and like, we are all, like, so super-duper practical.

Diogo AlmeidaTypeSafe AI

真的很棒。

It's really great.

Diogo AlmeidaTypeSafe AI

对。

Yep.

Diogo AlmeidaTypeSafe AI

做一份秘密总体规划是你的主意，所以……

It was your idea to make a secret master plan, so

Diogo AlmeidaTypeSafe AI

对。

Yeah

Diogo AlmeidaTypeSafe AI

不过我确实……对。我正式把功劳归给你。

But I did. Yeah. I'm giving official credit to you.

Diogo AlmeidaTypeSafe AI

嗯。

Yeah.

Diogo AlmeidaTypeSafe AI

对。

Yep.

Diogo AlmeidaTypeSafe AI

对。

Yep.

Diogo AlmeidaTypeSafe AI

嗯，问题是我不相信基准测试。

Well, the problem is I don't believe in benchmarking.

Diogo AlmeidaTypeSafe AI

对吧？

Right?

Diogo AlmeidaTypeSafe AI

所以这是你得亲自去感受的东西，我觉得这才是建立长期信任的方式，即便这让我们吃了很大的亏。比如去年我们融资的时候，没人相信我们。

So like, it is a thing that you need to feel, and like, I think that this is the way to build long-term trust, even though it, like, hurt, it hurt us a, us a lot? Like last year when we did fundraise, no one believed us.

Diogo AlmeidaTypeSafe AI

他们就只想要基准测试之类的东西，而我们说：「我们不会那么做。我们有原则，我们会坚持立场。那样做是在奖励作恶的人。我才不管你们想要什么。这就是我们，我们会坚持下去。」所以……抱歉，这不是……

? Like, and they wanted just benchmarks and stuff, and we're like, "We're not gonna do that. We are principled. We're gonna stand by our guns. That rewards bad actors. I don't give a shit, like, what you want. Like, this is who we are, and we are standing by that." So Sorry. It's not

Diogo AlmeidaTypeSafe AI

对。

Yep.

Diogo AlmeidaTypeSafe AI

对，对。显然我对此没什么遗憾。

Yeah. Yeah. Like, I'm, I don't have too many regrets on that, obviously.

Diogo AlmeidaTypeSafe AI

结果好得难以置信。昨晚我讲起离开 OpenAI 的原因时有点激动，因为发布之后我其实不得不改了措辞。我原来的说法是：「如果 AI 寒冬真的来了，而我没有拼尽一切去阻止它，那我会认为自己负有个人责任：一是对 RLHF 这个方向负责，我认为它把承诺过多与交付不足之间的落差拉得更大了；二是没有全力投入这件事，因为我认为价值会在这里被源源不断地印出来。」所以……这真的很酷，因为我觉得

Like, it worked out so unbelievably well. And like, I, The. I was emotional last night when I was talking about, like, the reasons I left OpenAI, and because, like, it actually had to change my wording after the launch. My phrasing was, "If an AI winter did happen and I did not do every fucking possible thing I could to, like, avert that, I would see myself as personally responsible both for, the RLHF direction, which I think really widened overpromise versus under-deliver, and also not going all in on this because I think this is, this is where value is going to just be, like, printed." So. And it was really cool because I feel like

Diogo AlmeidaTypeSafe AI

我所担心的那个 AI 寒冬已经避免了。AI 会有用，会被用于自动化。

The AI winter I'm worrying about is averted. Like, AI will be useful. It'll be used for automation.

Diogo AlmeidaTypeSafe AI

还不到一周，数据就已经无可辩驳了，

It's been less than a week, and like, the numbers are already undeniable

Diogo AlmeidaTypeSafe AI

它正在被用于真正的工作，这简直是狂野西部。对。

That it's, like, being used for real work, and like, there's. It's, it's the Wild West. Yeah.

Diogo AlmeidaTypeSafe AI

其实我没有完全掌握所有数字，都是团队在告诉我。

I'm actually not super on top of everything. Like, the team is the ones who are telling me all of these things.

Diogo AlmeidaTypeSafe AI

这有点……

It's, it's,

Diogo AlmeidaTypeSafe AI

有点疯狂。

It's kinda nuts

Diogo AlmeidaTypeSafe AI

我可以说一个我们已经越过的里程碑，就是每日 token 量。

I will say a milestone that we've passed is tokens per day.

Diogo AlmeidaTypeSafe AI

而且这不是转瞬即逝的那种每日 token 量。

And this is not, like, fleeting tokens per day.

Diogo AlmeidaTypeSafe AI

就算到了晚上，它也一直在持续运转，说明是机器在调用它，而不只是人在试用。

This is, like, even at night, like, it's constantly training, so machines are calling it and not just people trying things out.

Diogo AlmeidaTypeSafe AI

所以这真的太酷了。每天一万亿 token 是很大的量。

So that is, That is so cool. A trillion tokens a day is a lot.

Diogo AlmeidaTypeSafe AI

所以能超过这个数真的很棒。注册量对我来说其实不重要。说实话，这方面我们还犯了点错。推特上有人说我们是营销天才之类的，但那就是我们的本色。我们没有营销人员，这个岗位也在招。我们只是做真实的自己，有点傻、有点不正经，然后拼命地把人从候补名单里放进来。我们的平台团队强得离谱。我觉得我们的可用性比 Anthropic 还多几个 9，同时还经历了史上前所未有的发布。这有点疯狂，所以……

So surpassing that is awesome. Signups to me don't really matter. And actually, this was, like, a bit of a mistake we made, if I'm, like, totally honest. People on Twitter were calling us, like, marketing geniuses and all of that, and that was just us. We don't have a marketer. Also hiring. And we were just being our genuine, goofy, like, irreverent selves, and we were, we were just, like, offboarding people off the waitlist so hard. - Our platform team is so unbelievably cracked. I think we have more n- up nines of uptime than Anthropic while having the most Unprecedented launch ever. Like, that is kind of nuts, so

Diogo AlmeidaTypeSafe AI

真得给他们点个赞。

Like, props to them.

Diogo AlmeidaTypeSafe AI

还有一件我们没意识到的事。第一，在我看来，对开发者平台来说，等候名单的注册数并不重要。我猜其中很多人根本不是开发者。他们进来试几个查询，很多人搞不懂，因为他们并不写程序，对吧？他们的反应就是：「啥？这不是聊天机器人啊。我的 ChatGPT 2 呢？」

And the thing we didn't realize. So number one, waitlists, waitlist sign-ups don't matter for, like, a developer platform, in my opinion? I would guess that a large number of them are not even developers. So they go in, they try some queries, and a lot of people don't get it because they are not programming, right? Like, they're just like, "What? This is not a chatbot. Where's my ChatGPT 2?"

Diogo AlmeidaTypeSafe AI

对吧？但如果……我没有精确算过。我的感觉是，就算全世界每个人都写上几条查询，跟一个重度用户写的一个 for 循环比起来，也只是个零头，人家那个循环是实打实在创造价值。

Right? But if, like. I haven't exactly calculated this. My sense is that if every single human being in the world, like, just wrote a couple of queries, that would be a rounding error compared to, like, one power user's for loop that is just, like, creating value.

Diogo AlmeidaTypeSafe AI

关于等候名单，我们没意识到的是，其实可以直接把所有人都从等候名单上放出来。那无所谓。真正吓人的是速率限制。一旦大家开始从中得到价值，就会想要多得多的速率额度，因为软件本来就是这样，对吧？你先花力气把一件机械性的任务说清楚，然后这件任务创造的价值会远远超过投入。而一旦你有了这个——

And the thing we are-- didn't realize with the waitlist is, like, we could just w- off-board anyone off the waitlist. It doesn't matter. The scary part is rate limits. And then once people start getting value from that, then they just want tons and tons of rate limits because this is what software is, right? Like, you spend effort upfront to specify your rote task, and then this rote task creates more value than it takes to put in. And then now that you have that

Diogo AlmeidaTypeSafe AI

没错。让它在后台跑。让它成为其他东西的依赖。你可以在上面搭更高层的东西。你就能给世界创造巨大的价值。早期互联网的人大概想象不到 2000 年代初互联网的那种奇妙，而那其实已经不算早期互联网了。但正是通过——无意冒犯——可组合性，

Exactly, yeah. You run it in the background. You make it a dependency, to, like, other things. You can make, like, higher level stuff. And like, you just create so much value in the world. Early internet people probably did not imagine, like, the wonder of early 2000s internet, which is still not early internet. But like, it's, it's through, no offense, composability

Diogo AlmeidaTypeSafe AI

所有那些疯狂的东西才会出现，我就是特别想在我们的宣言里强调这一点。我们追求的是涌现。我们要做催化剂。我们想赋能大家，为此我们什么都愿意做，不管是在 Discord 上开全员大会，我穿不穿垃圾袋都行。

That all of the crazy stuff happens, and I just really wanted to emphasize that in our manifesto. We are going for emergence. We are going for, like, being the catalyst. We're wanting to empower people, and we are going to do whatever we can for that, be it, like, Discords in our town hall with me wearing a garbage bag or not.

Diogo AlmeidaTypeSafe AI

必须的！

Hell yeah

Diogo AlmeidaTypeSafe AI

绝对的。

Absolutely.

Diogo AlmeidaTypeSafe AI

对。

Yeah.

Diogo AlmeidaTypeSafe AI

太对了。

Hell yeah

Diogo AlmeidaTypeSafe AI

哦，作为客户。

Oh, as a customer.

Diogo AlmeidaTypeSafe AI

好吧，嗯。这挺好笑的。不好意思。

Okay, yeah. That was funny. I'm sorry.

Diogo AlmeidaTypeSafe AI

酷。现在涨到 3800 万了。

Cool. Up to 38 now.

Diogo AlmeidaTypeSafe AI

嗯。

Yeah.

Diogo AlmeidaTypeSafe AI

嗯。

Yep

Diogo AlmeidaTypeSafe AI

嗯。

Yep.

Diogo AlmeidaTypeSafe AI

对。

Yep.

Diogo AlmeidaTypeSafe AI

对。不过我其实更愿意……我确实有件 T 恤，上面写着「你最爱的 Neolab 最爱的 Neolab」。

Yeah. Well, I actually would rather. I do have the shirt, like, your favorites Neola-- favorite Neolab's favorite Neolab.

Diogo AlmeidaTypeSafe AI

我根本不在乎是不是 Neolab。我觉得当个 Neolab……其实我们有很多周边就是在恶搞 Neolab。我有一件写的是「有产品的 Neolab」，那其实就不是 Neolab 了。我真不在乎这个。

I don't give a shit about being a Neolab. I think being a Neolab. Actually, we have a lot of, like, swag that's being a parody of a Neolab. One of them, one of them I have is, like, Neolab with product, which actually is not a Neolab. Like, I don't care about that, really.

Diogo AlmeidaTypeSafe AI

我在乎的是做一个可靠的开发者平台。所以——

What I care about is being a reliable dev platform. So

Diogo AlmeidaTypeSafe AI

谢谢你拿我们跟他们比，但是——

Appreciate the comparison, but like

Diogo AlmeidaTypeSafe AI

希望我们能超越他们，回到那种对开发者来说的革命性时刻，做一个稳定、大家能依靠和信任的东西。

Hopefully we transcend past them and we go back into, like, a thing-- like, a revolutionary moment for developers and like, this stable thing that people can rely on and trust.

Diogo AlmeidaTypeSafe AI

还有几个 9。

And nines.

Diogo AlmeidaTypeSafe AI

就是——

It's, like

Diogo AlmeidaTypeSafe AI

哦，那是其中一部分。但可靠性还体现在这东西有多聪明上：它能多稳定地做到你想要的事？我觉得那些大型推理模型非常聪明，但在我看来，它们仍然缺乏可靠性。我觉得有很多用例，它们看起来应该已经聪明到能把那些工作自动化了，也有经济动力去自动化这些工作，可它们还是不如一个实习生可靠，因为它们是为别的东西优化的。所以我觉得有一种可靠性是能信任输出。另外，可靠性还有一些维度我们还没做到，我对那些特别兴奋。

Oh, but that's part of it. But like, there's reliability in, like, how intelligent the thing is. Like, how consistently does it do the thing that you want? And I think that, like, the big reasoning models are very smart. In my opinion, they still lack reliability. I think there's many use cases where you-- they look like they should be smart enough to automate their work. There is economic incentive to automate that work, yet still they're not reliable enough as, at an intern because they're optimized for different things. And so like, I think that there's the reliability of being able to, like, trust the outputs. And also we are. Like, there are dimensions of reliability that we are not yet at that I'm, like, so excited by.

Diogo AlmeidaTypeSafe AI

我想先把简单的工作自动化，再去做难的？我觉得这是常识。但对我来说，我们会做到足够……我不知道有没有「足够可靠」这回事，但我想做到好到大家都不用试模型就知道它能行。就像编程里的心流状态，对吧？我就是在写查询，因为这里需要智能。遇到不简单的分支逻辑时，我可以直接写一个类型安全的 System One 查询，拿到结果，它就能准确地分支。那该多好。那就是梦想。

Like, I want to automate the easy work before the hard work? Like, I think that's just a common sense thing to do. But to me, we will be sufficient. I don't know if there's such thing as sufficiently reliable, but I wanna get so good that people don't even need to try the model to know that it'll work. It's like, that's like what flow state is in programming, right? Like, I'm just, like, writing queries because I need intelligence in here. And like, when. For non-trivial branching, I can just write it in like a, like a type-safe System 1 query and then get the results out of it and it just branches accurately. Like, that would be so good. Like, that's the. That is the dream.

Diogo AlmeidaTypeSafe AI

而那会是一段漫长的苦功。

And that is, like, going to be, like, a long slog.

Diogo AlmeidaTypeSafe AI

哦！

Ooh

Diogo AlmeidaTypeSafe AI

所以——

So

Diogo AlmeidaTypeSafe AI

哦，好问题。这其实是我们经常被问到的问题。可靠性其实是个大筐。每当 AI 没法把某件事自动化，原因总是某种形式的可靠性问题。可能是类型安全，可能是确定性，也可能就是能力参差不齐，对吧？所以可靠性是个大筐。我觉得它也笼统地指向北极星是什么。确定性是指同样的输入、同样的输出。我确实认为这对单元测试有点意思，但我认为它是错误的北极星。我认为大家真正想要的是稳健性——

Oh, great question. So this is actually, like, a common question we have between. So reliability is actually a catchall. Like, whenever AI can't automate something, it's due to some form of reliability. Could be, like, type safety. It could be determinism. It just could be, like, it's, it's jagged, right? So reliability is a catchall. I just think that it's also a catchall for, like, what the North Star is. Re- determinism is, like, same inputs, same outputs. I do believe that this is, like, slightly interesting for unit tests, but I believe that to be the wrong North Star. I believe robustness is what people

Diogo AlmeidaTypeSafe AI

我不想告诉大家他们真正想要什么，那样有点自大。

I don't wanna tell people what they really want, 'cause that would be a little arrogant of me.

Diogo AlmeidaTypeSafe AI

我认为那是更重要的属性。你想要的是，输入相似，输出也相似。LLM 在这方面不可靠得离谱。

I believe that is, like, the more important property. You want, given similar inputs, get similar outputs. And it's kind of wild how unreliable LLMs are.

Diogo AlmeidaTypeSafe AI

我们测试这一点的一个办法是，往里面放 UUID，就是那种小小的——

Like, a way that we test this is you put, like, UUIDs in, like little

Diogo AlmeidaTypeSafe AI

我想它们叫 nonce，放在提示词里。你想要的是这些都给出相似的输出，因为在语义上它们真的是同一个问题，这才是你真正想要的。这种稳健性正是人们在让 AI 做决策时被坑的地方。所以我认为它是一个超级超级重要的属性。我们也可以做到确定性，这是可以提供的东西。就我能为程序员建立的心智模型来说，它对某些用例可能有价值，所以请在评论里指教我。但总的来说很简单，确定性是可以拿来换更低成本的东西。我们一直想站在每美元智能的前沿上。为了做到这点，我们在干一些极其恶心的事。这……

I think they're called nonces In the prompt. And what you want is similar outputs from all of those, 'cause it's truly semantically the same question, and that is the part where you really want. Th- like, that robustness is where, like, people get, like, burnt with AI making decisions. So I think that is the. A super-duper important property. We could also have determinism. That is, that is a thing that can be available. As far as I can, like, mentally model for programmers, like, it, I- it could be valuable for some use cases, so like, please educate me, in comments or view. But my. In general, it's easy. Determinism is something you can, like, trade off for better cost. Like, we are, we are constantly wanting to be on the intelligence per dollar frontier. We are doing, like, absolutely disgusting things to be there. Like, this is,

Diogo AlmeidaTypeSafe AI

我不该说这个，但这儿也没人能拦我。

I shouldn't say this, but no one's here to stop me.

Diogo AlmeidaTypeSafe AI

我们公司可不是这么运作的。我觉得这周科技圈最有权势的人是我的幕僚长 Kay。

That is not how it works at this company. I believe for this week, my chief of staff, Kay, is the most powerful person in tech.

Diogo AlmeidaTypeSafe AI

我的天——

Holy sh

Diogo AlmeidaTypeSafe AI

我的天。她能力强得要命，也超有分量。她太厉害了。

Holy shit. She is so fucking competent and powerful. She's incredible.

Diogo AlmeidaTypeSafe AI

她很烂，别挖她。不过，我尽量说话收着点，但大家一直跟我说：「别把它叫作模型拼成的弗兰肯斯坦怪物」，因为这有负面含义。我觉得弗兰肯斯坦的怪物在整个故事里算是好人吧。它是无辜的，对吧？我没读过。好吧。

She sucks. Don't poach her. But so I try to be a bit more filtered, but like, people are telling me, "Don't call it a Frankenstein's monster of models," but because that has, like, negative implications. I think Frankenstein's monster was, like, the good guy in this whole. It was innocent, right? I didn't read it. Okay.

Diogo AlmeidaTypeSafe AI

我坦白。好吧。就……光看你一个表情，我——

I'll, I'll confess. Okay. That. Well, one facial expression, I

Diogo AlmeidaTypeSafe AI

我摊牌了。

My cards on the table

Diogo AlmeidaTypeSafe AI

我——

I

Diogo AlmeidaTypeSafe AI

你根本想象不到我现在时间有多少。

The. You have no idea how little time I have right now.

Diogo AlmeidaTypeSafe AI

我的优先级是睡觉？

My priorities are sleep?

Diogo AlmeidaTypeSafe AI

开发者，对，开发者。不过没错，我们为了站在每美元智能的帕累托曲线上，在干一些极其恶心的事，而且会继续这么干。

Developers, yes. Developers. But yes. It. We do, like, absolutely disgusting things to be on the Pareto curve of intelligence per dollar, and we are going to keep doing that.

Diogo AlmeidaTypeSafe AI

我们会干很多疯狂的事，我觉得大家真的需要跳出框框思考。我们之所以让人意外，部分原因就是大家都在框框里思考，而我们会一直这样让人意外。就目前而言，我们显然是这方面最强的，我们也想一直保持最强。

We're gonna be doing crazy-ass stuff, and I think people really need to think outside of the box. Like, part of the reason we're surprising is, like, people Are thought inside the box, and we continue to do that. As of right now, we are obviously the best at this, and we want to continue being the best at that whole thing.

Diogo AlmeidaTypeSafe AI

所以——等等，我们是从哪儿岔开的？

So Wait, where did, where did we tangent from?

Diogo AlmeidaTypeSafe AI

嗯。

Yeah

Diogo AlmeidaTypeSafe AI

哦对。所以——

Oh, yes. So

Diogo AlmeidaTypeSafe AI

还有稳健性。

And robustness

Diogo AlmeidaTypeSafe AI

但是确定性——

But like, determina- like

Diogo AlmeidaTypeSafe AI

太好了。我就再说一点。

I would love that. I will just say one thing.

Diogo AlmeidaTypeSafe AI

我们可以做出确定的模型。

We can make a deterministic model.

Diogo AlmeidaTypeSafe AI

如果大家能说服我们这件事有价值，

Like, we're hap- if people can convince us that is a valuable thing to do

Diogo AlmeidaTypeSafe AI

而且我们没有遇到巨大的 GPU 短缺，

And we don't have a gigantic GPU shortage

Diogo AlmeidaTypeSafe AI

我们很乐意把这些模型都做出来。我们就是为了让大家满意而活的。还有革……革命，

We can happily make all of these models. We live to please. And rev- and revolt, revolute,

Diogo AlmeidaTypeSafe AI

对。

Yeah.

Diogo AlmeidaTypeSafe AI

所以确定性是可以考虑的。

So like, determinism could be on the cards.

Diogo AlmeidaTypeSafe AI

只是每美元智能会更低。

It just gets you less intelligence per dollar.

Diogo AlmeidaTypeSafe AI

好吧。

Okay.

Diogo AlmeidaTypeSafe AI

我很乐意。也许哪天我们会看到事情怎么发展。

I will love to. Maybe one day we will see how that happens.

Diogo AlmeidaTypeSafe AI

有人跟我说我……他们说我们品牌的一部分就是不可动摇，

I've been told I'm, They say that part of our brand is being unshakeable

Diogo AlmeidaTypeSafe AI

他们说那只是「固执」的好听说法。

And they say that's just the nice way of saying stubborn.

Diogo AlmeidaTypeSafe AI

对，没错。我就是个非常固执的人。不固执的话，我觉得我们做不成这事。

Yeah, exactly. And I'm a very stubborn person. I don't think we could have done it.

Diogo AlmeidaTypeSafe AI

对。

Yeah.

Diogo AlmeidaTypeSafe AI

对。

Yeah.

Diogo AlmeidaTypeSafe AI

而且每次在开发者问题上你都是对的。

And you've been right about developers every time.

Diogo AlmeidaTypeSafe AI

好吧，我认输。你赢了，你赢了。我承认你以前跟我争论过。

So okay, I give up. You win. You win. I'm sold that I've argued with you before.

Diogo AlmeidaTypeSafe AI

嗯。

Yep.

Diogo AlmeidaTypeSafe AI

对，嗯。

Yeah. Yep.

Diogo AlmeidaTypeSafe AI

不过我猜，我们在很长一段时间里都会受 GPU 限制。

But I suspect, though, that we will be GPU constrained for a very long time.

Diogo AlmeidaTypeSafe AI

而任何每美元智能更低的东西，都意味着要消耗更多 GPU

And anything that has less intelligence per dollar means it consumes more GPUs

Diogo AlmeidaTypeSafe AI

才能得到同样的智能，这……我们的目标不是让企业上车。那有价值，但我们的目标是让大家去实验、去搞些奇奇怪怪的东西。我们需要让它到尽可能多的人手里，掀起一场像加州淘金热那样的热潮。

For the same intelligence, which is. Like, our goal is not to onboard companies. Like, r- it's, it's valuable, but like, our goal is to have people, like, experiment and do weird shit. And we need, like. We need to, like, get, it to as many hands as possible and like, starting, like, the California gold rush for that.

Diogo AlmeidaTypeSafe AI

对。

Yeah.

Diogo AlmeidaTypeSafe AI

哦，好。

Ooh, okay

Diogo AlmeidaTypeSafe AI

嗯。

Yep

Diogo AlmeidaTypeSafe AI

嗯。

Yep

Diogo AlmeidaTypeSafe AI

对。

Yep.

Diogo AlmeidaTypeSafe AI

对。

Yep.

Diogo AlmeidaTypeSafe AI

我们部署模型后不会改动它们，那太疯狂了。我们在乎开发者。做这种事，又回到了第一方产品和 API 的问题。在第一方产品里你想怎么做都行，对吧？随他们去，只要能带来那种体验，没问题。但做 API 显然不能这样。不过我要说，我们计划的推进速度会比很多人习惯的模型厂商快得多。所以我们发布新模型会比大家想的快得多，而且我们不承诺模型的长期支持，因为我们觉得还有很多可以改进的地方。所以有一种可能是，我们会临时对现在的 Jev 1.13.0 做长期支持。我们可能会这么做，因为太多人在用了，而且我知道开发者讨厌依赖被破坏。另一种选择是让我们的机群碎片化，那对谁来说都很糟。那会——

We will not change our models when we deploy them. That is insane. We care about developers. Li- like, it makes sense if you're. If. So- doing something like that, again, this is the problem with a for- first-party product and an API. It makes. You can do whatever you want in a first-party product, right? Like, more power to them, whatever gets that experience, that is fine. With an API, you obviously can't do that. But I will say that we, plan to move a lot faster than many people are used to model providers, doing things. So we will be launching new models a lot faster than people think, and we are not promising long-term support for the models because we think that there's lots of improvements to have. So there is a world that we might temporarily LTS what is right now Jev 1.13.0. We might do that 'cause so many people are using it, and I know developers hate breaking dependencies. The alternative is fracturing our fleet, and that is a very bad vibe for everyone. It's gonna be

Diogo AlmeidaTypeSafe AI

没错。而且如果我们迭代得很快，版本也会特别多。

Exactly. And if we're iterating very fast, there would be a lot of those versions as well.

Diogo AlmeidaTypeSafe AI

所以我们最终想要的不只是一个长期支持的东西，我们想要一种特别酷的实现方式。我们在这个方向上有研究正在酝酿，我觉得那会是有史以来最对开发者友好的东西。

So we do want to have not just a LTS-supported thing eventually, long-term support. We want a really sick way of doing that. We have, like, research stuff cooking in that direction, and I think it's gonna be the most pro-developer thing ever.

Diogo AlmeidaTypeSafe AI

但现在的模型还不是那样，我也不承诺我们能保持完全相同的模型。可以肯定的是，它们每次都会变得更聪明。

But it is not yet our current models, and I'm not promising that we will be able to keep the exact same models. They will get smarter every time, for sure.

Diogo AlmeidaTypeSafe AI

而且我的感觉是，在模型迭代里，在它已经足够聪明的地方，版本之间的变化往往比同一个模型调用两次的差异还小。但当我们从参差不齐变成「哇」的时候，那才是大幅变化出现的地方。

And my sense is that even our model iterations, where it already is smart, it. Between model versions, the changes tend to be even smaller than the string models calling them twice. But when we go from, like, jagged to, like, wow, that is where the big deltas are.

Diogo AlmeidaTypeSafe AI

无可奉告。

No comment.

Diogo AlmeidaTypeSafe AI

我在乎的是每美元智能。

So I care about intelligence per dollar.

Diogo AlmeidaTypeSafe AI

对吧？

Right?

Diogo AlmeidaTypeSafe AI

什么？

What?

Diogo AlmeidaTypeSafe AI

再看吧。

We'll see.

Diogo AlmeidaTypeSafe AI

再看吧。我——

We'll see. I

Diogo AlmeidaTypeSafe AI

对。

Yeah.

Diogo AlmeidaTypeSafe AI

对。

Yeah

Diogo AlmeidaTypeSafe AI

对。我觉得每秒智能是另一个指标，我们甚至讨论过像每美元乘以每秒的智能这样的指标。我对杰文斯悖论会不会发生的猜测——至少对 Jev 系列模型来说——还有我在内部会追着大家问的事是：我不管它聪明了多少，它必须在帕累托前沿上。这就是 Jev 这个品牌的定位：它是每美元智能最好的东西。至于每秒智能，再看吧。我觉得这挺有意思。我知道有很多行业极度依赖实时的东西，对他们来说，每秒智能意味着大把的钱。但是——

Yeah. Like, I think that intelligence per second is, like, a different metric, and we've even talked about, like, things like intelligence per dollar times second and like, metrics like this. My guess on, like, Jevons' paradox occurring, or at least the Jev series of models, and the thing I, like, hunt people down about internally is, like, I don't care how much smarter it is, it needs to be in the Pareto frontier. So like, that is what the brand of Jev is. It is the best thing at intelligence per dollar. For intelligence per second, we'll see. I think that it's an intriguing thing. I know that there's many industries that are, like, extremely dependent on real-time stuff, and they will, like. Like, intelligence per second means tons of dollars for them. But

Diogo AlmeidaTypeSafe AI

再看吧。我很愿意两边都做，让市场来纠正我，不管往哪个方向。

We'll see. I'm, I would love to, like, do both and like, have the market correct me either which way.

Diogo AlmeidaTypeSafe AI

对吧？

?

Diogo AlmeidaTypeSafe AI

我很希望听听大家的意见。

Like, I would love to be informed by people.

Diogo AlmeidaTypeSafe AI

那取决于它在多大程度上是在后台跑，对吧？

It depends on how background it's running, right?

Diogo AlmeidaTypeSafe AI

如果是一个很大的后台任务，比如数据库的 MapReduce 查询，延迟可能就没有成本那么重要，

Like, if it's, like, a big background, like, database MapReduce query, the latency might not matter so much as, like, the cost

Diogo AlmeidaTypeSafe AI

也就是从中获取智能的成本。但如果它确实是更实时的东西，比如面向用户的，你的预算会落在 100 毫秒到 1 毫秒之间，那简直是魔法。而且其实即使你已经在 100 毫秒以下，如果还能把时间减半，就意味着你能得到两倍的智能，或者多做几次串行的智能调用，带来非常出色的体验。

To get intelligence from it. But like, if it actually is, something more real time, like user-facing, you have budgets, like between 100 milliseconds and one millisecond that are, like, totally magical. And actually, even if you were below 100 milliseconds, if you could half that time, that means you can get double the intelligence or sequential intelligence calls to have, like, a, like, a phenomenal experience.

Diogo AlmeidaTypeSafe AI

所以没错，这确实正在发生，超级酷。我很喜欢每秒智能的用例，但我不觉得那会是 Jev 的细分领域。

So right, that is definitely happening right now. It is super-duper cool. I love the intelligence per second use cases, but I don't think that will be Jev's niche.

Diogo AlmeidaTypeSafe AI

嗯。

Yeah.

Diogo AlmeidaTypeSafe AI

对。

Yep.

Diogo AlmeidaTypeSafe AI

对。

Yeah

Diogo AlmeidaTypeSafe AI

对。

Yeah

Diogo AlmeidaTypeSafe AI

是的。「同时保持智能不变」，这话分量很重。那才是难的部分，对吧？

Yes. It w- I-- That's a very load-bearing statement while holding intelligence constant. That's the hard part, right? Like

Diogo AlmeidaTypeSafe AI

但我会……我其实试过——

But I will. I've actually tried

Diogo AlmeidaTypeSafe AI

你再说一遍？

Say again?

Diogo AlmeidaTypeSafe AI

哦，当然。

Oh, of course.

Diogo AlmeidaTypeSafe AI

当然。我们肯定有自己的内部评测。

Of course. We have, we have our own internal evals, for sure.

Diogo AlmeidaTypeSafe AI

但要忍住不去刷这些评测需要很强的自律，而且必须把「不刷评测」当成最高优先级。

But it takes a lot of discipline not to game those, and it needs to be, like, a top-level priority to not game them.

Diogo AlmeidaTypeSafe AI

我们当然会这么做，对吧？

Of course we do that, right?

Diogo AlmeidaTypeSafe AI

不然我们怎么保证自己的模型处在每美元智能的帕累托前沿上？

Like, how else can we make the guarantee that our models are in the Pareto frontier of intelligence per dollar?

Diogo AlmeidaTypeSafe AI

对吧？我们在这件事上可不是盲飞。如果我们做的东西各不相同、成本也五花八门，怎么比较它们？我们会把它们画出来，努力搞清楚什么对用户最好。

Right? Like, we're not flying blind in there, right? If we're doing, like, completely weird things with different costs or whatever else, like, how do we compare them? We plot them and get. You. We try to figure out, like, what is the best for the users.

Diogo AlmeidaTypeSafe AI

所以我们当然会测。我不反对测量。但只要掺进任何别的激励，就极其危险，这是我铁腕把控的一件事。好吧，我同事可能觉得我在很多事上都是铁腕，但在我看来，别在模型有多聪明这件事上自欺欺人，是最重要的事情之一。

So we for sure measure them. I'm not anti-measuring. But it's extremely dangerous when you have, like, any alternative incentive, and this is the one thing that I kind of rule with an iron. Well, maybe my coworkers might think I rule many things with an iron fist, but to me, like, not shitting ourselves, about how smart our model is one of the most important things there.

Diogo AlmeidaTypeSafe AI

我们必须求真。

Like, we need to be truth-seeking.

Diogo AlmeidaTypeSafe AI

噢。

Ooh.

Diogo AlmeidaTypeSafe AI

哦，太好了。太好了。

Oh, hell yeah. Hell yeah.

Diogo AlmeidaTypeSafe AI

对。

Yeah.

Diogo AlmeidaTypeSafe AI

现在是了。

Now it is.

Diogo AlmeidaTypeSafe AI

这个我们争论了很久，争论了很久。它有点像布尔值，对吧？就是 true、false。它是……

We debated this a lot. We debated this a lot. It is It is Bool-ish, right? Like true, false. It is

Diogo AlmeidaTypeSafe AI

对，没错。先说名字的来历，它来自 Bernoulli。

Yes, exactly. So first, the origin of the name is Bernoulli.

Diogo AlmeidaTypeSafe AI

对。所以它才拼得那么怪。它是 Bernoulli 这个名字的一部分，取自 Bernoulli 概率，对吧？它本质上就是这个。这就是名字的来历。我们为这个争论了很久。我们喜欢过 PBool，也喜欢过 Pool。我们本来想叫它 pool party，但没人同意我。为这个我们还吵了一堆别的。

Yes. So it-- that's why it's even spelled that weird way. That is, like, a subset of the name Bernoulli from, like, a Bernoulli probability, right? Which is actually what that is. So that is the origin of it. We were debating this a lot. We liked PBool, we liked Pool. We were wa-- we were wanting to call it, like, a pool party, but then no one let me. We had, like, a bunch of, like, other arguments about that.

Diogo AlmeidaTypeSafe AI

最后我们觉得 Noulli 最好。我们的理由是（Jev 这个名字其实也一样）：我们觉得自己是一群离经叛道的疯子，而程序员根本不在乎。如果 Jev 只是一个字符串，我们没指望它能流行起来，更没想到会有人拿它玩梗，对吧？其实公司内部很多人讨厌这个名字。后来他们都道歉了，只有一个人除外。

And Noulli, we figured was, like, the best thing. Our rationale, and like, this is actually the same thing with Jev too, is that we think that we are, like, an irreverent, insane bunch, and programmers don't care. Like, if Jev is just gonna be a string, we didn't expect it to catch on or even have puns or anything like that, right? Actually, there was a lot of hate on the name internally. They've all apologized, except for one person.

Diogo AlmeidaTypeSafe AI

对，就是我们共同的朋友。

Yes. Our mutual friend.

Diogo AlmeidaTypeSafe AI

对。

Yes.

Diogo AlmeidaTypeSafe AI

对，对。她想把 Jev 叫成 Meow。

Yeah. Yeah. She wanted Jev to be called Meow.

Diogo AlmeidaTypeSafe AI

对，当然。

Yes, of course.

Diogo AlmeidaTypeSafe AI

跟她父亲一样，嗯。

Like her father, yeah.

Diogo AlmeidaTypeSafe AI

不过，对，Noulli 就是这样来的。我们得为这东西造一个新概念，因为如果它是 Bool，会让人困惑。所以其实这三个都是新概念。它们不是编程里已有的类型，这是有意为之：它们和类型对应得很紧，但又不完全是。score 不是 int。所以如果你用 Instructor、Pydantic 之类的东西把 int 或 float 映射成 score，你就会有点栽跟头。我们确实更偏向清晰，而不是让人一眼就看懂是怎么回事。

But like, yeah, Noulli is. We had to make a new concept for this thing 'cause if it was a Bool, it would be confusing to people. So actually, all three of these are actually new concepts. These are not types that exist in programming, and that was intentional because they map very closely to types, but they're not quite that. A score is not an int. So if you had, like, Instructor or Pydantic or whatever map ints or floats into scores You'd get a little bit cooked? And like, we were really erring on the side of clarity over the side of, like, making people, like, easily understand what's going on.

Diogo AlmeidaTypeSafe AI

想，确实想。其实我觉得……

Yes. Yes, we do. And actually, I think that,

Diogo AlmeidaTypeSafe AI

对。

Yeah.

Diogo AlmeidaTypeSafe AI

对。

Yep.

Diogo AlmeidaTypeSafe AI

对。

Yep.

Diogo AlmeidaTypeSafe AI

可能哪儿有吧。我所有事情都落后得厉害。

We might have that somewhere. I am so behind on everything.

Diogo AlmeidaTypeSafe AI

哦，对，对。

Oh, yeah. Yeah.

Diogo AlmeidaTypeSafe AI

酷。

Cool.

Diogo AlmeidaTypeSafe AI

我也不觉得那是非黑即白的。

I don't see that as binary either.

Diogo AlmeidaTypeSafe AI

我其实把成功看成一个 score，总还有往上爬的余地

I actually see success as a score, and there's always more to climb

Diogo AlmeidaTypeSafe AI

就是在我们能为社区做到多少这件事上，先说清楚。还有，这一段让我有点紧张，因为我没看过文档，而且文档一直在变。

In, like, how much we can, like, be there for our community, just to be clear. And I'm. This section is stressful 'cause I didn't review the docs And they're constantly changing.

Diogo AlmeidaTypeSafe AI

但在我看来，score 确实存在。score 有点像让 LLM 当评委打分。

But to me, scores do exist. So scores are similar to, like, LM judging.

Diogo AlmeidaTypeSafe AI

对吧？所以如果你想把它叫作一次评判，大概也行。但这就是大家已经在用这类东西的方式，对吧？Noulli 也许可以看作一个概率，但对我们来说一切都是概率。而 choice 其实最接近函数调用，但函数调用是个极其恶心的东西，如果你想听 OpenAI 的八卦内幕，那个我们以后再聊。choice 就是暴露 switch、match 语句的正确方式

Right? So like, if you want to call it, like, a judgment, I guess you could. But like, that is, like, the way people ca-- already use this type of thing, right? Like, maybe a Noulli could be, like, a probability, but everything for us is a probability. And a choice is actually closest to a function call, but a function call is, like, an extremely disgusting thing that, if you want OpenAI juice, sauce, tea, that. We should go back into that later. Like a, like, a choice is just, like, the right way of explo-- of exposing, like, a switch match statement

Diogo AlmeidaTypeSafe AI

在代码里的那种。

Within code.

Diogo AlmeidaTypeSafe AI

对。

Yep.

Diogo AlmeidaTypeSafe AI

对。而在 enum 里，choice 才是关键。

Yes. And like, in the enum, choice is the important part of that.

Diogo AlmeidaTypeSafe AI

其实我觉得这些都能映射到编程原语上：choice 对应在 enum 上做 switch statement。

And like, actually, I think these map all into, like, programming primitives, where, like, choice maps into, like, a, like, a switch statement on an enum.

Diogo AlmeidaTypeSafe AI

Noulli 对应 if statement。

Noulli's mapped to if statements.

Diogo AlmeidaTypeSafe AI

score 对应排序，或者用大于、小于做阈值判断。

And scores map to sorting or thresholding at a greater than or less than.

Diogo AlmeidaTypeSafe AI

这一直是我们的愿景。以后会有更多类型，它们都会映射到编程原语上。

And this has been always what the vision is. Like, there will be more types, and they will map into programming primitives.

Diogo AlmeidaTypeSafe AI

嗯。

Yeah

Diogo AlmeidaTypeSafe AI

谢谢。我太喜欢这个问题了。不，这是……

Thank you. I love this. No. This is

Diogo AlmeidaTypeSafe AI

太好了。我没想到会有这个问题。其实，大概有好几个月没人问过我这个了，上次还是我给我们的 DevRel 做入职培训的时候。

Hell yeah. I didn't expect this. And actually, I. No one has asked me this, in probably, like, months when I was, like, onboarding, like, our DevRel.

Diogo AlmeidaTypeSafe AI

太爽了。我们的模型是为将来深深嵌在计算机程序内部而设计的。我们是真心相信，这会比今天大家能想象到的任何东西都大得多。我们的模型可能还没准备好，但我们在持续为那个未来努力。它在这些浅层任务上永远不会足够好。抱歉，它永远不会……我们不会只是一直在浅层任务上往上爬。我们想深入程序的内脏，因为这才是让软件变强大的方式。我们里面所有的类型……这个其实是输出。

So sick. So our model is designed for being, like, deep in the insides of computer programs in the future. We, like, unironically believe that this will be much more massive than anything people are even considering today. And our model might not be ready for that, but we are, like, continuously working for that future. It will never be good enough at these shallow tasks. Sorry. It'll never be g- Like, we're not just gonna cle- keep on climbing the shallow tasks. We want to be deep in the guts of programs 'cause that's how you make software powerful. All the s- all the types inside of our, This is an, actually an output.

Diogo AlmeidaTypeSafe AI

但输入的所有部分，比如状态、指令、评判标准，全都可以是结构化的 JSON 对象。

But all the, all the parts, of, like, the input, like the state, the instructions, the criteria, all of them can be structured JSON objects.

Diogo AlmeidaTypeSafe AI

这样程序就能把它们插到正确的位置，你不需要把东西塞进模板里。正是这样。所以如果……我觉得大家对这部分琢磨得不够，以为全是字符串。那也没关系。但这些本来都是为了……我会说，如果你在用模板，比如把它变成一条 system message 之类的，那你还在用老思路。我们应该让计算机尽可能容易理解，因为那个结构是真实存在的，对吧？在编程语言里，你手上有一堆数字，传进去之前却先把它们转成字符串，这就很奇怪。通常只有在打印、需要给人看的时候你才这么做，对吧？但在计算机内部，你想传递的是处处带语义的嵌套结构。我们会真正针对这一点优化模型。模型已经挺针对这个优化了，但问题是每多一层嵌套结构，推理起来就更难，所以我们正在这个方向上狠狠发力。我觉得大家也应该继续往这个方向走，因为这会让代码清晰得多、漂亮得多，也和实现细节无关。就像：这是我的状态，这是我的函数状态。把它想成一个 AI 函数。我的状态，也就是你手上所有可用的变量，应该把哪些子集传进来？system message 就像恶心的全局变量，你把所有东西都塞进去，把所有这些

That way, like, programs can, like, insert them in the right spot, and you don't need to, like, put things into templates. Exactly. So if ever. I think people don't read into this part enough, and they think it's all strings. And that's, that's fine. But these are all meant. Like, I would say that if you're using, like, a template, like turning it into, like, a system message or something, you are thinking in, like, the old way? We should be making things as easy for computers to understand because that structure is truly there, right? Like, it would be weird in, like a programming language to have, like, all of your numbers in, and then you pass it into, like. You turn it into a string. Normally, you do that for printing when you have a human in the loop, right? But for, like, within the computer, you want to be passing, like, nested structure that is semantic all around. And we are really gonna be optimizing our model. It-- the model's pretty optimized for this, but the thing is every different nested level of structure is harder to reason about, and we want-- we are really cooking hard in that direction. I think people should keep cooking that direction because it makes the code, like, so much more legible and beautiful and like, agnostic to, like, the implementation details. It's like, here is my state, like, here's my function state. Like, think of, think of it as, like, an AI function. Which subsets of my state, which is, like, all the variables you have available, should I pass in here? System messages are, like, disgusting global variables where you just put everything in there, and you put all this

Diogo AlmeidaTypeSafe AI

指令一股脑全塞进去。然后你就盼着每一条指令都能被完美执行，而不是把问题拆开并行地问。

Instructions at once. And then like, you hope that every single instruction gets nailed instead of asking the questions in parallel.

Diogo AlmeidaTypeSafe AI

另外我会建议……我说这话真不是从能让我赚钱的角度出发。我真心建议多问问题，问很多很多问题。把它们拆开、拆小，真正地分解。不管模型今天能不能做到，我相信眼下这一切最大的救赎会是大家的代码库，AI 代码库会变得好得多。如果你把问题分解成简单的决策，每一个都极其容易评测。以前的 AI 是一大段 system message，然后也许你还有另一个大 AI

And also, I would recommend-- I, and I truly say this not from, like, a, like, it makes me money perspective. I truly recommend asking lots and lots of questions. Break them down, make them smaller, and like, really decompose. Like, no matter if the models can do it today or not, I believe that the biggest, like, saving grace of, like, what's happening this week will be people's code bases, AI code bases, are gonna be so much better. Like, if you decompose problems into simple decisions, every single one of these things is extremely evaluable. Like, a AI beforehand is big system message, and then maybe you have, like, another big AI

Diogo AlmeidaTypeSafe AI

去检查它是不是真的照做了。这太离谱了吧？真的有点疯狂。那是我们的斯德哥尔摩综合征，对吧？但真的挺疯狂的。比如你想说「别读这个子目录」，或者「别把任何 API key 传给 DeepSeek」之类的，这些基本上应该由程序来保证。任何机器学习模型都不可能给你保证，但通过拆分，你实际上可以衡量它，对吧？比如

To see, like, if it actually does this. That's nuts? It's, it's kinda crazy. Like, it-- that was our Stockholm syndrome, right? But like, that's kinda crazy. Like, if you wanna say, like, "Hey, don't read this subdirectory," or, "Don't pass any API keys to DeepSeek," or whatever else, like, that should be programmatically basically guaranteed. And you'll never have guarantees of any machine learning model, but like, by breaking it down, you can actually s-- you can actually measure it, right? Like

Diogo AlmeidaTypeSafe AI

对，而且我们的模型，模型的接口本身就非常可验证。这应该让人松一口气。

Yes, and like, our model, our model-- like, the interface itself is so verifiable. This should be like a sigh of relief.

Diogo AlmeidaTypeSafe AI

这只会带来好得多的工程实践。

Like, it's, it's, it's, it's just gonna lead to way better engineering.

Diogo AlmeidaTypeSafe AI

对。

Yep.

Diogo AlmeidaTypeSafe AI

对。

Yep.

Diogo AlmeidaTypeSafe AI

嗯。

Yeah

Diogo AlmeidaTypeSafe AI

对。

Yep.

Diogo AlmeidaTypeSafe AI

这种情况确实会发生，对。而且超级不方便，很笨重。为什么不全放在一起呢？你最后会在不同问题之间

And that happens-- yeah. It's, and it's, like, super inconvenient. It's unwieldy. Why not just put it all together? You kind of end up repeating some stuff between

Diogo AlmeidaTypeSafe AI

重复一些东西，所以可能有点低效之类的。但全放在一起，结果就是很难依赖。

Questions, so it's, like, maybe, like, inefficient or something like that. But then it results in something that is very hard to rely on.

Diogo AlmeidaTypeSafe AI

而软件得能在后台运行。如果我们的东西不能在后台运行，我会很心碎。

And software doesn't need to run in the background. It would break my heart if our stuff couldn't run in the background.

Diogo AlmeidaTypeSafe AI

有意思。

Interesting.

Diogo AlmeidaTypeSafe AI

对。

Yep.

Diogo AlmeidaTypeSafe AI

有意思。我喜欢把东西拆到最小的语义单元。

Interesting. I like to break things down into its, like, its smallest semantic unit.

Diogo AlmeidaTypeSafe AI

就是最底层的那个东西是什么？我尽量从不……我大概是所有人里调用这个模型最多的。

Like, what is the lowest level thing? I try to never have. I've probably queried, the model the most, among anyone.

Diogo AlmeidaTypeSafe AI

我会尽量……第一，在我的查询里，这其实更像是我写提示词的方式，我会写得非常结构化、非常明确。在问题里我总是……我喜欢用反引号，不过其实用哪种都行，就是要把我指的是什么说得非常清楚，因为我们希望模型非常按字面理解。因为写程序的时候，你想要的是指令遵循得非常好的东西。这就是编程的艺术所在，而 AI 做的，是扩展能被遵循的指令的种类。所以我很推崇这种做法。我……

And like, I try to. Number one, in my, in my queries, this is, this is a lot more like the way I prompt things. Like, I make it really structured and explicit. And in the questions, I always. I like the back ticks, but like, it works for all of them? Like, be really clear what I'm referring to because we want the model to be really literal because when you program, you want things that instruction follow really well. That is what the art of programming is, and what AI does is expanding the things, the kinds of instructions that can be followed. So I'm a fan of doing that. I.

Diogo AlmeidaTypeSafe AI

有时候我有点懒，会写一些更混搭的东西，但我觉得对于真正大型的生产环境，你就是要不断加问题，而且要让加问题变得非常容易。对整个拆分都要非常精确，然后让代码表现出你想要的确切行为。如果举个小例子，就是拒答，对吧？我不打算讲我们为什么不拒答，我可能已经讲过了。

Sometimes I'm a little lazy and I, like, I have, like, more, like, hybrid things, but like, I think that for, like, really big production things, you just want to, like, keep on adding more questions, and you wanna make it really easy to add more questions. Be really precise about all of that breakdown and then have the code to have the exact behavior you want. If I could give, like, a tiny little example of this, is, like, refusals, right? Like, I'm not gonna talk about why we don't refuse. I might have done that already.

Diogo AlmeidaTypeSafe AI

全都糊成一片了。但就拒答来说，我觉得你不应该问「这里我该拒答吗？」这样问……我觉得答案其实会挺好的，因为那是一个 System One 就能处理的任务。但我觉得更好的做法是，针对可能需要拒答的各种不同情况，问很多个独立的问题。因为这样你不用只靠猜，而是可以把你想要的东西明确规定出来。而且很美妙的是，我觉得这真的很美：如果你发现某个情况，「哦，它因为这个原因没拒答，我没规定任务的这一部分」，那太棒了。这就是软件工程的本质。你把这个问题加进去、加上阈值，也许再把它记成一个测试用例，就修好了这个 bug，从此它就永远解决了。你的软件不会因为上下文腐烂而在提示词里把它忘掉。它就在那儿，你可以一直衡量它。如果模型在某些事情上还不完美，你可以根据真实例子，为所有这些因素选择你想要的阈值。这就像没有机器学习的机器学习，而且你可以用在任何事情上。当然也可能有些事模型还不够好，对吧？比如我……看到有人用模型做交易，我有点害怕，比如

It's like all a blur. But like, for refusals, I don't think you should ask, "Should I refuse here?"? That's a really. It-- I think the answer will be pretty good because, like, that's a System 1 compatible task. But I think you're way better off, like, asking many different independent questions about, like, the different situations you can refuse about. Because instead of having to, like, just guess based on you can actually specify what you want. And beautifully, and I think this is, like, truly really beautiful, if you find a situation where it's like, "Oh, it didn't refuse because of this reason. I didn't specify this part of the task," that is awesome. That's what software engineering is about. Like, you fix the bug by adding that question in, adding the threshold, maybe remembering that as a test case, and now it is just solved forever. Like, your software can't forget about that, like, in the prompt because of context rot. It is just there, and you can, like, just keep measuring that forever. And if the models are not perfect at some of these things, you can choose what threshold you want for all of these factors based on real examples. It's like, it's like ML without the ML, and you can just do it for anything. And like, there might be some things the model's not good enough yet, right? Like, I would. I'm a little bit afraid when I see people doing trading with the models, like,

Diogo AlmeidaTypeSafe AI

自动化交易。看起来很酷。我只是觉得这种事应该留给专业人士。

Automated trading. It looks cool. I th- I just think that people should leave it to the professionals.

Diogo AlmeidaTypeSafe AI

那是一个非常难、非常高层次的任务，模型也许还不够好，搞不定。

And like, that's just a very hard, high-level task that maybe the models aren't good enough yet to figure out.

Diogo AlmeidaTypeSafe AI

而且即使它们够好了，因为市场有效，这事很快也就不成立了。但这正是那种你可以拆成小问题逐一评测的事，然后你可能会说：「它在这方面还不够聪明，这个版本也许先别部署到这上面。」

Well, I, like, even if they were, then they would- It suddenly wouldn't be 'cause of efficient market. But like, that's one of those things where, you can, like, break it down into things and just evaluate them, and you might be like, "It's not smart enough at this. Maybe we don't deploy it yet for this version."

Diogo AlmeidaTypeSafe AI

或者我们做个取舍，或者宁可偏向安全，或者说：「嘿，模型还不太擅长识别讽刺加 VIP 客户这种奇怪组合，遇到这种情况就转给人工。」这也正是置信度估计的意义。

Or we make a trade-off, or we err on the side of safety, or like, "Hey, the models are not good enough at, like, detecting, like, this weird combination of, like, sarcasm with a VIP customer, that this is when we escalate to a human." And that's what confidence estimates are about, too.

Diogo AlmeidaTypeSafe AI

你别指望。

You don't.

Diogo AlmeidaTypeSafe AI

我没这么说。

I didn't say that.

Diogo AlmeidaTypeSafe AI

对，我没这么说。

Yeah. I didn't say that.

Diogo AlmeidaTypeSafe AI

当然，当然

Of course, of course

Diogo AlmeidaTypeSafe AI

对。

Yes.

Diogo AlmeidaTypeSafe AI

我们可以。

We could.

Diogo AlmeidaTypeSafe AI

嗯

Yeah

Diogo AlmeidaTypeSafe AI

哦，问题是我们想不想提供微调？

Oh, do we want to offer fine-tuning, is the question?

Diogo AlmeidaTypeSafe AI

嗯。

Yeah.

Diogo AlmeidaTypeSafe AI

对。

Yep.

Diogo AlmeidaTypeSafe AI

对。

Yep.

Diogo AlmeidaTypeSafe AI

对。而且先说清楚，它会搞错很多事。

Yep. And it will, it will get many things wrong, to be clear.

Diogo AlmeidaTypeSafe AI

我们有一个 Report Issues 按钮。来 Discord 上跟我们抱怨。我们想把它做得好得多。每一个模型版本都会有明显提升。

We have, like, a Report Issues button. Complain to us in Discord. We want to make it a lot better. Every single model version will be, like, notably better.

Diogo AlmeidaTypeSafe AI

要是没有大的提升，我们也就不会这么快一版接一版地发了。所以第一，这完全合理。承认 AI 在某些事上不完美，这只是务实。我确实认为我们会找到它们足够好的用例，而「足够好」在某种程度上取决于具体用例，对吧？人类即便不擅长某项工作，也能做大量这类工作，因为他们的期望价值相当高。想必只要阈值之类的设得对，就可能有大量工作是即使会出错也能做的。至于微调，我能想象它在计划之中。我确实有顾虑，因为在「人们需要什么」和「人们想要什么」这个范畴里

We will stop shipping them quickly if they weren't getting big improvements. So number one, that is, like, totally reasonable. I think that's simply pragmatic to admit that AI is imperfect at some stuff, right? I do think we'll find use cases that they are, like, good enough at, and good enough kind of depends on the use case, right? Like, human beings can do a lot of work despite being bad at that work because their EV is quite high. And presumably with the right thresholding and everything, there probably is, like, large amounts of work that could be done even if mistakes are being made. On the question of fine-tuning, I could imagine, I could imagine it in the cards. I do have concerns because, like, in the what people need versus what people want category,

Diogo AlmeidaTypeSafe AI

我觉得通用模型往往真的……还是那句话，通用性有一种说不清道不明的东西：让它在这个窄任务之外的一百万个其他任务上都表现好，可能反而让它在这个任务的边缘情况上表现更好，这是我会有点担心的地方。

Like, I think general models tend to be really. Like, again, there's the je ne sais quoi of generality, that making it good at, like, a million other tasks than this one narrow task might make it better at edge cases in that task, which I'm, I would be a little bit afraid of?

Diogo AlmeidaTypeSafe AI

我能想象，这就是我的回答。在这些事上我极度务实。我什么都想要。我对世界的设想是……我们想做的东西实在太多了。

I could imagine it, is my answer. I'm endlessly practical on these things. I want everything. Like, my vision of the world is. I-- there's, there's so much we want to be building.

Diogo AlmeidaTypeSafe AI

但同时，我也不想发布一个会让人搬起石头砸自己脚的东西，像其他一些 AI 公司会发的那种。

But also, like, I would not want to ship something that is, like, a giant foot gun, like some other AI companies would ship.

Diogo AlmeidaTypeSafe AI

对。

Yeah.

Diogo AlmeidaTypeSafe AI

对。

Yep.

Diogo AlmeidaTypeSafe AI

对，这我知道。而且那东西有点烂，所以撤掉大概更好。

Yes. I do know about that. And like, it was kind of crap, so like, that's probably better that they took it down.

Diogo AlmeidaTypeSafe AI

嗯

Yeah

Diogo AlmeidaTypeSafe AI

嗯，其实我对这种可能性非常开放。

Well, actually, I'm, I'm super open to that possibility.

Diogo AlmeidaTypeSafe AI

我的……这不是承诺，这是愿望。先说清楚，我喜欢非常坦诚。我觉得随着每美元智能越来越便宜，我们可能会得到一些很小的近似的东西，希望它们能充当智能的代理。会不会有那么一个世界，大家再也不写正则表达式了？因为用 AI 的每美元智能比正则表达式的复杂度还便宜。那会挺酷的，我会很喜欢。而在一些窄用例上，可能需要微调才能真正越过那个门槛。走着瞧吧。我希望靠校准来做到。校准加上模型级联：如果它超级有把握，那也许就是对的；如果它处在中间，你就调下一个更大的模型，依次串下去。我不太确定会怎么发展，但对，我能想象。我还能想象的是，假设你有一系列……我们拥有整条帕累托前沿。企业可能想做的事情是……我觉得黑客可以接受自己去应对一条模型的帕累托前沿，但企业也许想要更动态的东西。你可以想象有不同尺寸的模型，根据它在你技术栈不同部分上有多聪明，动态挑选用哪个模型。甚至可以想象，因为我们的东西很简单，可以在这之上做某种自动微调。

Like, my. This is not a promise. This is a desire. Just so to make it clear, I like to be really honest. Like, I think that, as intelligence per dollar gets cheaper, I think that we could get really, like, small approximate things that hopefully are proxies for intelligence. Like, is there a world where people don't write regexes anymore? Because, like, the intelligence per dollar that uses AI is cheaper than, like, the complexity of a regex. That would be kinda sick. I would love that? And it might require fine-tuning for some of those narrow use cases to really get past the threshold. We will see. My hope is calibration gets that. Calibration plus a cascade of models. Like, if it's super confident, then maybe it's right. And if it's in the middle, then you do the next bigger model, and you chain off from there. I don't really know how that's gonna go, but yeah. I could imagine it. And something that I could imagine too is, like, imagine you have, like, a series of. Like, we own the entire Pareto frontier. Something that a business might want to do, or, I think a hacker would be okay with dealing with a Pareto frontier of models. Maybe a business wants something more dynamic. You could imagine, like, having, like, a s- different sizes of models and to dynamically pick which model based on how smart it is on different parts of your stack. And you could even imagine, because of how simple our thing is, you could imagine, like, some automatic fine-tuning on that.

Diogo AlmeidaTypeSafe AI

完全不是承诺，我只是在畅想科幻。

Not a promise in the slightest. I'm just, like, cooking on sci-fi.

Diogo AlmeidaTypeSafe AI

当然会。我怎么知道大家需要多少智能呢？

Absolutely. Yeah. Like, we. Like, how would I know how much intelligence people need?

Diogo AlmeidaTypeSafe AI

对吧？我也不知道。

Right? Yeah. I don't know either.

Diogo AlmeidaTypeSafe AI

嗯，现在有人劝我们先别发新东西，说没必要，因为……

Well, yeah, people are telling us not to ship things right now because we don't need to ship things because, again

Diogo AlmeidaTypeSafe AI

对，但那样挺没劲的。我很喜欢一句话，也希望大家拿这句话来监督我，因为说出口就很难……

Yeah, but that's kinda lame. And I really like the saying. This is something that I hope people hold me to because it'll be hard to

Diogo AlmeidaTypeSafe AI

很难反悔。对。我记不清原话了，大意是：文化就是市场不奖励你时你依然在做的事。我很喜欢这句话，因为我觉得我们是在为某种东西站台。也许将来我们坚持的东西会变得理所当然，我们就成了像 Visa 那样无聊的存在，只是一种没人会多想的基础设施，那时我大概也不穿粉色西装了。

To walk back from. Yeah. Like, the. I don't know if it. Exactly the saying that culture is what you do when the market doesn't reward it. And I really like that because I think that we are standing for something. Maybe in the future- what we're standing for is, like, so obvious that we're the equivalent of, like, boring, like, Visa or something like that. And like, we're just like a utility that no one really thinks about, and I'll be wearing non-pink suits or whatever else.

Diogo AlmeidaTypeSafe AI

但我真的想号召全世界一起来做这件事。我想继续做酷的东西，不是因为非做不可，而是想让大家意识到这只是开始。这次甚至都不算正式的开场炮，更像是一次低调的研究预览，随你怎么叫。

But I really want to be, like, rallying the world to this? Like, I want to keep doing cool stuff, bec- not because we need to, but 'cause I want, like, people to realize that this is just the beginning? Like, that wasn't even meant to be the opening salvo. That was, like, kind of like a, low-key research preview or whatever you wanna call it.

Diogo AlmeidaTypeSafe AI

我们能做的还多着呢。

And there's a lot more we can do.

Diogo AlmeidaTypeSafe AI

机器原生的智能会疯狂发展。

With. Like, machine-native intelligence is gonna go wild.

Diogo AlmeidaTypeSafe AI

这些都绝对不会只有一个。

Absolutely not for any of those.

Diogo AlmeidaTypeSafe AI

我想尽可能满足各种需求。

I want, I want to, like, meet whatever needs we can.

Diogo AlmeidaTypeSafe AI

对吧？但有个很大的前提：我不想像 OpenAI 的产品团队那样到处乱扔东西试水。我希望一切都在一个统一的愿景之下。回到我们的宣言，所有东西都得归在那三件事之一里。

Right? Like, at. But with, like, a giant caveat, I don't want to be like OpenAI's product teams that, like, throw stuff at the walls. Like, I want it to be, like, in a, under a unified vision. Like, if you go back to the manifesto, like, everything needs to be under one of these three things

Diogo AlmeidaTypeSafe AI

我是这么看的。

In my opinion.

Diogo AlmeidaTypeSafe AI

嗯。

Yeah.

Diogo AlmeidaTypeSafe AI

哦，抱歉，抱歉，Francis。没事，我直接讲就行。我们的东西分三步。

Oh, I'm sorry. I'm sorry, Francis. Yeah, I can just talk about it. Like, we have, like, three steps in our stuff.

Diogo AlmeidaTypeSafe AI

听着像在卖关子。我希望一切都归在这三件事之一里，

It sounds like a tease. I want everything to go under one of these three things

Diogo AlmeidaTypeSafe AI

持续突破边界。这些不是检查清单，而是我们认为构成新一轮技术革命基础的几条轴。我希望我们押的每一个注都落在这里面。模型方面我们也会做一些挺怪的东西。

To keep pushing the boundaries and everything. Like, this is not. These are not, like, checklists. These are, like, axes that we think build, like, the foundation of, Of, like, a new technological revolution. And I want all of the. All the bets we make to be somewhere in there. And we will be doing some weird stuff model-wise.

Diogo AlmeidaTypeSafe AI

因为是机器原生的嘛。人类不需要完全看懂，只要它有价值就行。

So because machine-native, right? Like, humans don't need to totally get it. It needs to just be valuable.

Diogo AlmeidaTypeSafe AI

我给大家一个提示。

I'll give people a hint.

Diogo AlmeidaTypeSafe AI

有些人想把它们叫做决策模型。

Some people are trying to call them decision models.

Diogo AlmeidaTypeSafe AI

说我们的原语就是决策。我不会这么叫，因为我觉得还有其他机器原生的类型，不是决策。

That our primitives are decisions. I wouldn't do that, because I think there's other types that are machine-native that are not decisions.

Diogo AlmeidaTypeSafe AI

这是个有意思的提示。

That's a fun hint, a fun hint.

Diogo AlmeidaTypeSafe AI

对，我觉得这个提示挺好玩的。

Yeah. I think it's a, I think it's a pretty fun hint.

Diogo AlmeidaTypeSafe AI

嗯。

Yeah.

Diogo AlmeidaTypeSafe AI

我不在乎基准测试，

I don't care about the benchmarks

Diogo AlmeidaTypeSafe AI

先说清楚。

Just to be clear.

Diogo AlmeidaTypeSafe AI

所以不管我们是赢是输，我都想发布新东西。

So like, even if we were winning or losing, I want to do announcements.

Diogo AlmeidaTypeSafe AI

对。

Yep.

Diogo AlmeidaTypeSafe AI

对。

Yep.

Diogo AlmeidaTypeSafe AI

对。我就是想让软件工程师拥有超能力，

Yes. And I just wanna make software engineers super powered

Diogo AlmeidaTypeSafe AI

对吧？借助 AI。对我来说可悲的是，

Right? L- like with AI. Like, and or, like, the tragic thing to me is,

Diogo AlmeidaTypeSafe AI

在那种 AI 寒冬的走向里，AI 明明这么强大，却这么没被用起来，真的太让人难过了。这事会让我动情，唉。

In that AI winter direction, I think, like, it's, it's, it's just so sad that AI was so powerful yet so underutilized. Like, It's a thing that gets me emotional, but man.

Diogo AlmeidaTypeSafe AI

我不想做那种纯粹的技术乐观主义者，觉得所有技术都是好的。但我觉得现在发生的事简直是一种悲剧。我只是想为大家打开那些可能性。

Like, I think that is. I don't want to, like, just be, like, pure techno optimist, like all technology is good. I think what was happening now was, like, a travesty. Like, it's. And like, there's. I just want to, like, open up those possibilities for people.

Diogo AlmeidaTypeSafe AI

嗯，就说到这儿吧。这几天我已经哭太多了，不想在录音里再哭一次。

Yeah, I'll just end it there. I've, I've cried too much these last few days To want to do it on the record.

Diogo AlmeidaTypeSafe AI

嗯。

Yeah

Diogo AlmeidaTypeSafe AI

嗯。

Yeah

Diogo AlmeidaTypeSafe AI

对，对。

Yeah. Yeah.

Diogo AlmeidaTypeSafe AI

我不确定我会不会觉得最难的部分已经过去了，也许吧。我觉得后面还会有很多难关。如果各种事情都自动化了，我们终于看到 GDP 增长，每天都像一场 Jev 派对，

I don't n I am sure that I won't think. Maybe I will think that the hard part was done, perhaps. I think that there's going to be many more hard parts. Like, if, All sorts of stuff gets automated and we finally see GDP growth and like, it's like, a Jev party

Diogo AlmeidaTypeSafe AI

那也许最难的部分才算过去了。但我不这么认为。而且我真心觉得大家太关注速度和成本，对可靠性关注得不够。

Every day, then maybe the hard part is done. But like, I don't s- think so. And like, I really think that people focus too much on speed and cost and not enough on reliability.

Diogo AlmeidaTypeSafe AI

可靠性才让人用得舒心，可靠性才让你能信任它。

Like, reliability is what makes it delightful. Like, reliability is what, like, allows you to trust it.

Diogo AlmeidaTypeSafe AI

嗯。

Yeah.

Diogo AlmeidaTypeSafe AI

没错！

Hell yeah.

Diogo AlmeidaTypeSafe AI

没错！冲啊！

Hell yeah. Let's fucking go.

Diogo AlmeidaTypeSafe AI

嗯。

Yeah

Diogo AlmeidaTypeSafe AI

但经济革命本来就是这个意思，对吧？这其实和 OpenAI 章程过去所代表的东西高度一致。

But like, that is what an economic revolution is, right? Like, it's actually extremely consistent with what the OpenAI charter used to stand for.

Diogo AlmeidaTypeSafe AI

章程里讲的是……我觉得章程本身没变，但他们一直试图挪动定义，比如改成 1000 亿美元利润之类的。倒不是说我讨厌 OpenAI。

It was talking about, like. I think the charter is the same, but they've kind of tried to move definitions around to, like 100 billion in profit or something like that. Not that I hate an OpenAI.

Diogo AlmeidaTypeSafe AI

他们试过定义，

They tried to do it

Diogo AlmeidaTypeSafe AI

对吧？就是完成世界上大部分具有经济价值的工作。那他们就得回答一个问题：它怎么能解出数学千禧年大奖难题，却完成了零份具有经济价值的工作，四舍五入就是零。我觉得现在所有模型在这上面大致都打平，都是零。也有可能我们已经起步了，但我猜还不到 1%。而当它真正发生时，会体现在经济统计数据里。

Right? Like, doing majority of the world's economically valuable work, and they should have to answer the question, how can it do millennium prize problems in math and zero of the world's economically valuable work, around the air. Like, I think that all models are roughly tied right now at zero. There's some chance that, like, we have started already, but like, I would guess that it's not yet 1%. And I think that will show up in. Like, when it does happen, it will show up in the economic statistics.

Diogo AlmeidaTypeSafe AI

那会他妈的太棒了。它不会造成大规模失业，但会带来一大堆很棒的转变，世界会变得好得多。还有，我真的受够了 AI 总是当主角。我觉得世界应该变得更美好，AI 只要帮上忙就行。

It's gonna be fucking awesome. It will not cause mass unemployment, but it will cause, like, a whole bunch of awesome shifts, and wor- the world will be a lot better. And also, like, I'm really tired of AI always being the foreground character, of things. Like, I think that- the world should just be more delightful, and AI should just help with that.

Diogo AlmeidaTypeSafe AI

而且我……

And I

Diogo AlmeidaTypeSafe AI

正是。

Exactly.

Diogo AlmeidaTypeSafe AI

我在演讲里也讲过：2019 年的软件，SaaS 也好什么也好，价值非常高，对吧？现在已经 2026 年了，AI 厉害成这样，软件怎么基本上还是一模一样，顶多旁边多了个聊天框？那个聊天框勉强能用，但没法让你做公司真正有利害关系的决策，因为它不值得信任，没法交给它做决策。在我看来这太离谱了。这里面有巨大的经济动力，我觉得这会是一场反向的「SaaS 末日」，SaaS 会因此被大大增强。他们最清楚哪些事情值得自动化，接下来会是一段疯狂的时期。

Like, the-- I say this in my talks. Like, how can it be that 2019 software, like software, SaaS, whatever, super-duper valuable, right? It's 2026 now. How is the software basically exactly the same, despite AI being so freaking awesome, other than sometimes having a chat box on the side, right? That, like, that kind of works, but doesn't allow you to make decisions that the companies have stakes in, because they can't be trusted to make decisions. That, to me, is nuts. Like, there's so much economic incentive for this, and I think it's going to be, like a, like an inverse SaaS-pocalypse. I think SaaS is going to be supercharged by this. They are the ones who are, like, most in the know of what things are valuable to automate, and it's gonna be, like, a crazy time.

Diogo AlmeidaTypeSafe AI

嗯。

Yeah.

Diogo AlmeidaTypeSafe AI

靠，这个问题难，这个问题真难，朋友。

Fuck. That's a hard one. That's a hard one, my friend.

Diogo AlmeidaTypeSafe AI

「什么都 Jev 一下」，这说法挺好笑的。

Jev everything is pretty funny.

Diogo AlmeidaTypeSafe AI

这个说法挺逗的。那我跟你说实话。

It's a pretty funny way of doing it, saying it. The. So I'll tell you the truth.

Diogo AlmeidaTypeSafe AI

实话是，这是个经验性的问题，就像扩展定律是经验性的一样。比如，为什么机器人现在还不怎么行，尽管砸了那么多钱？

The truth is that this is an empirical problem, just like scaling laws are an empirical thing. Like, why doesn't, like, robotics really work right now, despite all the money being spent on it?

Diogo AlmeidaTypeSafe AI

我不觉得问题一定在于砸更多钱，可能只是经验结果还没到位，对吧？所以从经验上讲，我认为这些预训练出来的、高度浓缩的智能，本质上是 System One 思考者。System One 是描述 LLM 擅长什么最贴切的说法。RLVR 在 System Two 思考上做出了了不起的成果，我叹为观止，真的太酷了。我完全不觉得它会导致 AI 末日，当然不是 0%，因为说 0% 就是校准得不好。但他们做的东西真的很酷，已经推到了极限，也许他们自己不觉得是极限。但这对模型来说是件怪事，模型在这方面很脆弱。想想 ChatGPT 刚出来时大家怎么谈 AI：「哇，它好通用，能做好多通用的事。」但它不擅长数学题，比如 GSM8K 这种小学数学。再看看现在大家怎么谈 RLVR：「它好脆弱，能力参差不齐。」「它怎么能做这种很怪的事？」而且数学能力其实不只是尖刺状的，而是分形的，对吧？这是因为 RLVR 的本质就是……如果我们聊聊每种方法的北极星是什么：RLHF 是取悦人类，对吧？人类反馈就是这个意思。RLVR 是优化基准测试，凡是归入 RLVR 的东西，按定义就是基准测试，因为基准测试就是可以用程序验证、输出简单、能拿高分的东西。而 RLCD 是让模型在程序化使用中变得可靠。

I don't think it's about, like, spending more money necessarily. The empirical results just might not be there, right? So empirically, I believe that these, like, pre-trained super condensations of intelligence are fundamentally System 1 thinkers. I think that they truly. Like, System 1 is the closest thing to describe what LLMs are strong at. RLVR has done incredible things for System 2 thinking. I am at awe. It is super freaking cool. Like, I don't think that it's going to result in AI doom in the slightest. Not 0%, of course, 'cause I think 0% is miscalibrated. But like, i- it's, it's really cool what they've done, and they've really pushed it to the limits. Well, maybe they don't think so not the limits. But like, it is, it is a weird thing for models to do, and they are very fragile at this. Like, think about how people used to talk about AI back in the ChatGPT days. Like, "Wow, it's really general. It can do a lot of general things." And then. But it's bad at math problems and like, GSM8K, grade school math. And then now look at how people talk about RLVR. "It's so fragile. It's so jagged."? Like, it can. "Why can it do this, like, really weird thing?" And actually, math is not just spiky, it's fractal, right? And this is because RLVR is. Like, if we talk about, like, what is the North Star for each thing? RLHF is please humans, right? That is what the human feedback is. RLVR is optimize benchmarks. That l- everything that goes into the RLVR category literally is a benchmark by definition, because a benchmark is programmatically verifiable, simple outputs that can, like, do well. And RLCD is make it reliable for, programmatic use. And

Diogo AlmeidaTypeSafe AI

嗯，这个我……

Yeah. That, I'll

Diogo AlmeidaTypeSafe AI

哦。

Ooh

Diogo AlmeidaTypeSafe AI

哦，好。

Ooh, yeah

Diogo AlmeidaTypeSafe AI

对。

Yep

Diogo AlmeidaTypeSafe AI

嗯。

Yeah

Diogo AlmeidaTypeSafe AI

对。

Yep.

Diogo AlmeidaTypeSafe AI

嗯。

Yeah

Diogo AlmeidaTypeSafe AI

对。

Yep.

Diogo AlmeidaTypeSafe AI

哦，对。回到那个经验性的问题，这取决于我们能从模型里挖出什么。

So oh, yes. Back to that empirical question, it depends on what we can, like, pull out of the models.

Diogo AlmeidaTypeSafe AI

对吧？我们什么都想要，我们想尽可能多地挖掘出智能，就这么简单。我把我们看作是在解锁、打磨、雕琢这些智能，同时加入新能力、填补其中的空缺。随着时间推移，我们会填补越来越多这样的空缺。但现实是，我们做的是挖掘特性的生意。这些特性其实取决于这些浓缩的核心里有什么，再把它们像弗兰肯斯坦那样拼到一起，集齐所有东西的特性。

Right? So we want everything. Like, we want to unearth as much intelligence as possible, period. The models. Like, I see us as, like, unlocking and smoothing and sculpting the intelligence while, like, adding new capabilities and like, filling in gaps in it. And we will be filling in, like, more and more and more and more of these gaps over time. But the reality is that we are in the business of unearthing properties. Those properties are actually a function of what is available from, like, these, like, these condensed cores and like, Frankensteining them all together to have all of the properties of everything.

Diogo AlmeidaTypeSafe AI

但现实是，我们做的是尽可能多地挖掘能力的生意。System One 恰好就是对行得通的东西的描述。在这个范式里行得通的东西，都会有点 System One 的味道。

? But the reality is we are in the business of unearthing as many capabilities as pos- as possible. And System 1 just happens to be the description of what works. And everything that works in that paradigm will be System 1-ish. Like, I'm.

Diogo AlmeidaTypeSafe AI

我们不做所谓字符串里的隐式推理，是有原因的。

Like, there is a reason why we don't do what's called latent reasoning in strings.

Diogo AlmeidaTypeSafe AI

我觉得推理……模型真正擅长的是在模型内部推理。这还不完全，它并不是在所有方面都做得很好，

I think the reasoning. Like, the. What models do really well is reasoning within the models. It's not totally complete. It doesn't do great on all

Diogo AlmeidaTypeSafe AI

我觉得……

I think that

Diogo AlmeidaTypeSafe AI

大家以前把那个叫做

People used to call that

Diogo AlmeidaTypeSafe AI

连续推理。

Continuous reasoning.

Diogo AlmeidaTypeSafe AI

我也不完全确定。

I'm not entirely sure.

Diogo AlmeidaTypeSafe AI

之所以叫隐式推理，是因为以前推理过程是保密的。

It was called latent reasoning because, like, it used to be that the reasoning traces were secret.

Diogo AlmeidaTypeSafe AI

所以它们有点像答案背后的一个隐变量。

So they're kind of like a latent variable for the answer.

Diogo AlmeidaTypeSafe AI

嗯。

Yeah.

Diogo AlmeidaTypeSafe AI

嗯……

Well

Diogo AlmeidaTypeSafe AI

对 OpenAI 和 Anthropic 来说，它仍然是保密的，对吧？

It's, it's still secret for OpenAI and Anthropic, right?

Diogo AlmeidaTypeSafe AI

对。

Yes

Diogo AlmeidaTypeSafe AI

我的承诺是，为机器原生的东西做一切必要的事。

I. My promise is to do whatever necessary For machine-native stuff.

Diogo AlmeidaTypeSafe AI

我能想象有些推理形式没那么慢、没那么低效和脆弱，这些完全在考虑范围内，先说清楚。我是务实的人，我不对方法做承诺。我承诺的是我的 ROI 北极星，我会为之奋斗，就当这次发布没发生过，我们依然渴望在世界上争得一席之地。

I could imagine there are. Like, there are some forms of reasoning that are less slow, inefficient, and fragile that I. That are, like, totally on the cards, just to be clear. So pragmatic person, I'm not making promises on, like, methods. I'm making promises on, like, the. What my ROI North Star is, and I'm going to fight for that, like this launch didn't happen and we are still, like, hungry for our place in the world.

Diogo AlmeidaTypeSafe AI

嗯。

Yeah.

Diogo AlmeidaTypeSafe AI

我觉得我的视力挺好的。

I think I have a pretty good vision.

Diogo AlmeidaTypeSafe AI

我觉得我视力不错。

I think I have a good vision.

Diogo AlmeidaTypeSafe AI

开玩笑的，开玩笑的。嗯。

I am kidding. I'm kidding. Yeah.

Diogo AlmeidaTypeSafe AI

嗯。

Yeah.

Diogo AlmeidaTypeSafe AI

一切都有可能，

Everything is in the cards

Diogo AlmeidaTypeSafe AI

在我看来是这样。

In my mind.

Diogo AlmeidaTypeSafe AI

其实这也是我们内部的一个争论。你的听众大概正适合参与这个讨论。问题在于，我们在多大程度上去给大家我们认为他们想要的东西，这就是我们隐身开发两年里做的事，我们就是知道这显然会有价值；还是给他们他们嘴上说想要的东西，对吧？这里面有很多维度。上下文长度就是一个例子。每个模型，包括我们的，其实据我所知，我们的远远是最好的，

Like, and actually, this is, like, a debate we have. This. Man, your audience is probably, like, the great one to have in this debate. There's a question about, like, how much do we try to, like, give people what they think they want, which is what we did in stealth for two years. We just knew that this is obviously going to be valuable, versus give them what they say they want, right? And like, there's a lot of dimensions of this, right? And Like, context length is an example of this, right? Every single model, including ours, I actually think as far as I can tell, ours is, like, by far the best

Diogo AlmeidaTypeSafe AI

在长上下文里不退化这方面。

At not degrading in long context.

Diogo AlmeidaTypeSafe AI

但其他厂商就是：「大家要什么，就给他们那个蠢东西。」我们需要在这之间找到平衡。因为如果前一种走得太远，你就会变成 Anthropic 那种保姆式思维，非常反开发者。而亲开发者的路线就是给他们想要的东西，但我们不想把搞清楚智能那种说不清道不明的东西的负担推给开发者。所以我们在摸索怎么把握节奏，多快发布新东西，既保住我们值得信任的品牌，又把用户当成能做出知情决策的成年人，不需要在上面再加一层保姆式管控。

But like, the other providers are just like, "Whatever people want, let's just give them the stupid thing." And like, we need to figure out a balance for this because, like, if you take the former side too far, give people what they want, you end up with, like, anthropic nanny state style thinking, which is very, like, anti-developer. While, like, the pro-developer route would be like, give them what they want, but developers are. Like, we don't want to put the burden on them to figure out the je ne sais quoi of intelligence. So we are trying to, like, figure out this navigation of, like, how quickly to release things, to still, like, have our, like, brand of trust and also, like, teach our-- treat our users like adults that can make informed decisions that, don't need, like, nanny stating on top of this stuff.

Diogo AlmeidaTypeSafe AI

对。老实说我们也不知道答案，得慢慢摸索。这大概会是我接下来几天最大的争论之一，

Yeah. And we don't know the answer, to be honest. Like, we'll, we'll have to figure it out. It's gonna be. That's probably going to be, like, one of my biggest debates over the next couple of days

Diogo AlmeidaTypeSafe AI

因为我们手上有很多东西。还是那句话，我们没料到会这么火，所以我们想：「得再跟进几次发布了。」不过是这样。

Because, like, we have a lot of stuff. Again, we didn't expect it to pop off, so we were like, "We'll need some follow-up launches." But yeah.

Diogo AlmeidaTypeSafe AI

嗯，那也是因为我的幕僚长逼我全力以赴。

Well, that's also because my chief of staff made me lock in.

Diogo AlmeidaTypeSafe AI

对。

Yeah.

Diogo AlmeidaTypeSafe AI

就是，我从来没……我以为我以前已经很拼了，

Like, it's like, I have never. I thought I worked hard before

Diogo AlmeidaTypeSafe AI

然后……

And

Diogo AlmeidaTypeSafe AI

很有用，很棒。

It was useful. It was great.

Diogo AlmeidaTypeSafe AI

对。

Yep.

Diogo AlmeidaTypeSafe AI

嗯。

Yeah

Diogo AlmeidaTypeSafe AI

谢谢，谢谢。我希望继续全力以赴，

Thank you, thank you. I hope to keep locking in

Diogo AlmeidaTypeSafe AI

这是我的想法。

Is my, is my sense.

Diogo AlmeidaTypeSafe AI

我想……我觉得在科技圈我们已经闯过了很多大关，但后面还会有更多。

I want to. Like, w- like, I think that we've passed many great filters for the tech world, what we're wanting, but like, there's still gonna be a bunch more.

Diogo AlmeidaTypeSafe AI

天哪，能为这场正义之战而战，我太兴奋了。

And like, holy smokes, am I excited to fight the good fight.

Diogo AlmeidaTypeSafe AI

哦。

Ooh

Diogo AlmeidaTypeSafe AI

被低估或被误解的？

Underrated or misunderstood?

Diogo AlmeidaTypeSafe AI

给我一点时间

Give me one

Diogo AlmeidaTypeSafe AI

琢磨一下。哎呀，这些我每个都能大讲特讲。我真不该，真不该。

Noodling of it. Oh, man. I would rant about all of these. I really shouldn't. I really shouldn't.

Diogo AlmeidaTypeSafe AI

对。大家在示例集（cookbook）上倾注了很多心血

Yeah. People put a lot of love into the cookbooks

Diogo AlmeidaTypeSafe AI

这是我想说的。示例集里有些东西真的很炸。我们本来考虑把其中一批放进正式的发布博文里，但那样会太长、太臃肿，而且太偏高阶用户了。不过说实话，我坦白讲，发布之前，我们讲的东西听起来就像一个奇怪的外星工具：谁会需要这玩意儿？那是件很怪的事。我们非常担心怎么让大家理解这个新的前沿。结果显然是成功了，但我们投入了大量精力，因为我们以为教育用户会是我们的一个巨大瓶颈。

Is what I will say. The cookbooks have, like, some fire stuff. We had considered putting a bunch of these things, like, in the main launch blog post, but it got kind of long and unwieldy and like, very power usery. But like, we really. I'll be frank. Like, before the launch, every. Like, what we're saying sounds, like, sounds like this weird alien tool. Why would anyone need this? It was a very weird thing. We were very worried about teaching people about, like, this new frontier. It obviously succeeded, but like, we put a lot of work because we thought the education would be, like, a gigantic bottleneck for us. I.

Diogo AlmeidaTypeSafe AI

大概是起作用了，现在可能也不再是问题了，因为大家做的事情已经远远超出了我们的预期。

It probably works, and it's no lo- probably no longer a problem because people are doing things, like, well beyond what we could ever expect.

Diogo AlmeidaTypeSafe AI

对，确实。而且他们的用例比我们的还酷。有很多东西我看了就想：「天哪，要是这是我们的演示，那可比我们展示的酷多了。」比如操作电脑那些，真是酷爆了。但不管怎样，我们在这上面倾注了很多心血。据我所知，这不是 AI 生成的垃圾。

Yeah, but. Exactly. But like, they. Yeah, and their use cases are, like, kinda cooler than ours. Like Like, there's a bunch of stuff where I'm like, "Man, if that was our demo, holy shit, that was way cooler than what we were showing." like, the computer use stuff, holy smokes is it cool. But like, we put a lot of love into this. This is not, like, AI-generated trash, as far as I know.

Diogo AlmeidaTypeSafe AI

我们真的在里面倾注了很多心血。

We put a lo- l- like, it's, like, a lot of love in here.

Diogo AlmeidaTypeSafe AI

而且每一个都……里面有真正的干货。

And like, each of these are. Like, there's real alpha there.

Diogo AlmeidaTypeSafe AI

这些都源自解决真实存在的客户问题，我们下了功夫，帮他们做出了很酷的东西。

Like, these are inspired by solving real customer problems that existed, and we went through the work of, like, helping them do cool-ass stuff.

Diogo AlmeidaTypeSafe AI

那个过程是怎样的？

What was that process like?

Diogo AlmeidaTypeSafe AI

是的，当然。

Yes, of course.

Diogo AlmeidaTypeSafe AI

其实我觉得当时的反响挺差的。团队里非技术背景的人真的很担心。

I actually think that the reception was pretty bad. And like, actually for the non-technical people in the team, they were really worried.

Diogo AlmeidaTypeSafe AI

大家很恐慌，觉得根本没人懂这个，他们也不想要。我们卖的像是维生素，而不是止痛药。我们是不是该配 FDE（前线部署工程师），围绕那个问题去写软件？

Like, there was a lot of fear. It's like, no one really gets this. And like, they don't want it. We're, like, selling, like, a vitamin and not, like, a painkiller. Like, should we have FDEs to, like, write the software around solving that problem?

Diogo AlmeidaTypeSafe AI

发布前我们几乎没有收入。当时的情况是，技术背景的人显然都是坚定的信徒，对吧？我们知道这东西很牛。它的计算特性在很多维度上都好得离谱，所以我们觉得：「对，这显然会很大。」我当时确实非常害怕，所以才全力以赴。但最常见的情况是，我们找来试用的人里，超过一半根本没看懂。而看懂的那些人会说：「哇，这真的很酷，但我们怎么才能过采购流程之类的？」那真是一场硬仗。我们心里清楚：好，我们的目标市场就是开发者。大家会自己发现用例，这样所有人都会因为怕错过而涌进来。而且……

We had almost no revenue before launch. It was kind of like. Like, we. Like, the technical people were, like, obviously true believers, right? Like, we knew that this was sick. Its prop- computational properties are, like, off the charts on, like, so many axes that we're like, "Yeah, obviously it's gonna be huge." I was definitely super afraid, which is why I locked in super hard. But like, the most common thing was- The, like, I would say, like, more than half the people we had play with it just did not get it. And like, the people who did, like, were like, "Man, this is really cool, but how do we get this through procurement and stuff like that?" it was like, it was like quite a, quite a battle, and we just knew like, okay, the-- our target market is gonna be developers. People will find the use cases, and that way everyone is gonna FOMO in. And like,

Diogo AlmeidaTypeSafe AI

我不想揪着那些随着事实变化而改变看法的人不放。

I don't want to rub in people, like, changing their minds with the facts changing.

Diogo AlmeidaTypeSafe AI

但我确实想质疑一下「产品市场契合」这个概念。因为产品早就在那儿，市场也在那儿。我们问人家：「嘿，你想用这个吗？」人家说：「我不太确定它能不能解决我们的问题。」结果一爆火，所有人都说：「我们要尽可能多的速率额度。能不能干脆直接给你们 GPU？因为我们现在被卡住了。」

I do want to call into question, like, the concept of product market fit? Yeah, but like, because, like, there was a product, there was a market. Like, we were like, "Hey, do you want to use this?" And people are like, "I don't know, really know if it solves our problems." It explodes and everyone's like, "We need as much rate limits as we can. Can we literally give you GPUs? Because we are constrained right now."

Diogo AlmeidaTypeSafe AI

所以营销当然是一部分原因，但我甚至觉得关键不在营销。我觉得关键在于那些充满热情的开发者，我们的灵魂基本上在同一个频率上共振，然后这种频率也让其他所有人兴奋起来。

So of course, like, marketing is an element of it, of course, but I don't even think it's about marketing. I think it's about, like, passionate developers who've, like, our souls basically resonated at the same frequently, and that frequency, and that got everyone else excited too.

Diogo AlmeidaTypeSafe AI

我也希望我们作为一家公司，会永远感激这些开发者。而不是像那些从开发者起家、然后转去做大企业的公司那样。

And I'm hoping as well that, like, we as a company will be eternally grateful to those developers. Like, not-- and not just like, the companies that, like, are-- like, start off with developers and like, go big enterprises.

Diogo AlmeidaTypeSafe AI

没错。我甚至在想，我们怎么才能推出一些更有利于……哎呀，我不知道该不该说这个。

Exactly. And like, I'm, like, even thinking about, like, how can we launch things that are better for. Oh, man, I don't know if I should say this.

Diogo AlmeidaTypeSafe AI

不过我还是说吧。

But I will.

Diogo AlmeidaTypeSafe AI

没错。

Exactly.

Diogo AlmeidaTypeSafe AI

怎么做到这一点？怎么给他们赋能？我已经有些东西在酝酿了，真的在酝酿。但这事做起来很怪。我也不知道还有什么别的方式能表达我对他们的感谢和忠诚。这也是为什么我昨天去染了头发。我想跟他们对话，因为在公司最重要的时刻不继续跟他们交流，我会觉得很不地道。

How do we do that? Like, how do we empower them? And I have cooks. I have cooks. But it's a, it's a very weird thing to do. And like, I don't know how else I can show my thanks and loyalty to that. Like, and that's why I did, like, the dying my hair yesterday. It was like It's like I wanted to talk to them 'cause it felt dirty to me during our company's, like, most important times not to keep talking to them.

Diogo AlmeidaTypeSafe AI

对。请监督我。

Yeah. Hold me to that, please.

Diogo AlmeidaTypeSafe AI

我尽量做个有原则的人。

I try to be principled.

Diogo AlmeidaTypeSafe AI

记下我这句话。如果我变了，就把我揪出来，拿起干草叉来讨伐我。

Quote me on this. Call me out. D- have the pitchforks out if I change.

Diogo AlmeidaTypeSafe AI

哇。

Whoa.

Diogo AlmeidaTypeSafe AI

我还没见过……我见过一个订机票的浏览器操作演示。

I've never seen-- I haven't-- I've seen. I saw, like, a airline browser use thing.

Diogo AlmeidaTypeSafe AI

在这个新笔记里，把标题写成 hello。

And inside this new note, let's make the title say hello.

Diogo AlmeidaTypeSafe AI

哇，很好。好，我们继续，你能打开 Arc 浏览器吗？打开之后，能用 Google 搜一下 Norbert Wiener 吗？

Wow. Great. Okay. Let's move on and can you open up the Arc browser? And once you're there, can you Google search Norbert Wiener?

Diogo AlmeidaTypeSafe AI

现在能打开 x.com 吗？

Now can you open up x.com?

Diogo AlmeidaTypeSafe AI

哦，语音用例。这其实是我见到的第一个。这是……

Oh, the voice use cases. This is actually the first one I've seen. This is

Diogo AlmeidaTypeSafe AI

打开照片查看器。哇。哦，等等，能退回去一下吗？能退回去一下吗？

Open up the photo viewer. Wow. Oh, wait. Oh, can you bo- can you go back a second? Can you go back a second?

Diogo AlmeidaTypeSafe AI

「传闻称 Anthropic 工程师把 Claude 当神来崇拜。」哇，哇。

Rumors claim Anthropic engineers worth- worship Claude as God. Wow. Wow.

Diogo AlmeidaTypeSafe AI

天哪，这挺好笑的。

Dang. That's pretty funny.

Diogo AlmeidaTypeSafe AI

绝对……哇，这太牛了。

Abso-- Wow, this is sick.

Diogo AlmeidaTypeSafe AI

所以，就像我反对迷信基准测试一样，我也反对迷信演示。我想确保它能可靠地工作。我很喜欢大家拿它来玩。这真的他妈的太牛了，毫无疑问。我想看到这些，想看到它被用起来。我想让我们团队也玩玩，找出弱点，然后解决掉。

So Just like I'm anti-benchmarking, I'm also anti-demos. I want to make sure that it works reliably. I love people are playing with it. This is super fucking sick, have no doubt. I want to see this. I wanna see it be used. I want our team to play with it. I wanna find the weaknesses, and I wanna solve that.

Diogo AlmeidaTypeSafe AI

我会很……天哪，那个看起来真的很酷。

And I would lo-- Man, that looked really cool.

Diogo AlmeidaTypeSafe AI

看起来真的很酷。我想要这个，我想要。比如我手腕酸的时候，我就全靠 Wispr Flow 语音输入。要是有这个就太爽了。

That looked really cool. I want that. I want that. Like, when my, when my wrists are sore, I, like, just whisper flow everything. That would be sick.

Diogo AlmeidaTypeSafe AI

嗯。

Yeah

Diogo AlmeidaTypeSafe AI

这方面我完全不了解情况。

I am so out of touch for that.

Diogo AlmeidaTypeSafe AI

有人给我看过一些公司的截图，就我看到的来说，基本是所有公司。

People have shown me screenshots of companies, and from what I've seen, it's all of them.

Diogo AlmeidaTypeSafe AI

哦。我觉得那些演示超级牛。显然，编程助手是巨大的用例。

Oh. So like, I think demos are super-duper sick. Obviously, the coding agents are, like, gigantic use cases.

Diogo AlmeidaTypeSafe AI

它们也超级牛。

Like, they are, like, also super sick.

Diogo AlmeidaTypeSafe AI

哦，太好了。哦，我能稍微岔开聊聊编程助手吗，如果可以的话，或者……

Oh, hell yeah. Oh, can I, can I give a little bit of a tangent about coding agents, if that's or

Diogo AlmeidaTypeSafe AI

哦，让我……

Oh, let me

Diogo AlmeidaTypeSafe AI

给我一秒，给我一秒。好，其实我待会儿再回到编程助手。我先讲讲几大类用例。

Give me a second. Give me a second. Okay, actually, I'll come back to coding agents. Let me describe, like, the big families of use cases.

Diogo AlmeidaTypeSafe AI

我们在发布之前很久，就从第一性原理出发把这些梳理过了。第一类是我们所说的暗数据。大家囤积了大量数据，但不会拿 LLM 去处理，因为太贵了。所以大公司特别喜欢这个。他们手里有成堆的数据，一直想分析却没法分析，这简直是数据科学家梦寐以求的东西。所以这是一个巨大的类别。我觉得这个加上编程助手，会是最赚钱的两块，因为量都在那儿，对吧？然后是实时的场景，也就是需要智能参与在环路中的人。我猜这些公司的每一个 CEO，至少是 CTO，都清楚每省下 10 毫秒，他们的产品就会好多少。

Like, we've mapped this out from first principles, like, long before release. They are what we call dark data. Like, people hoarded big data, but they would not throw a LM at it 'cause it was too expensive. So large companies adore this. They have, like, piles of data that they wish they could analyze, and this is like a data scientist's wet dream. So this is like. This is a giant one. Like, I think this plus, coding agents are the m- big moneymakers because that's what they're all. Where all the volume is, right? There's the real-time stuff. Like people who need, like, intelligence in the loop. They. Like, I would guess that every CEO, if not CTO, at those companies, knows how much better their product gets with every, like, 10 milliseconds shaved.

Diogo AlmeidaTypeSafe AI

而且……

And like

Diogo AlmeidaTypeSafe AI

对。哦，或者是助手类的东西。有很多 AI 助手类的产品。据我所知，他们非常喜欢。再说一次，我现在不在对接客户的一线，所以只知道团队告诉我的东西。但我对这个超级兴奋。我特别期待它用在游戏上。我很想玩那种超爽的自走棋，你在里面指挥自己的队伍，或者半自动的那种。我觉得会很酷。不过在我还有工作的时候，别把它做得太好了。还有一类，我们叫「验证一切」：验证所有 LLM 调用，有点像可观测性。说到文档，我觉得大家应该做的是利用并行提问，它非常便宜。所以如果你有一个很大的状态，想就它问很多问题……

Yeah. Oh, or, like, assistant-y things. There's many AI assistant-y things. And like, as far as I can tell, they really love it. Again, I'm not in the front lines of customers right now, so I just get. Know what my team tells me. But like, this. I'm so excited for this. I'm really excited for this for games. I really wanna play, like, sick-ass auto-battlers where you're, like, commanding your team, or, like, semi-auto battlers. I think that'd be so cool. But don't make it too good while I still have a job. And the. Like, there's the. What we call, like, verify everything. Like, verifying all LLM calls, kind of like observability. I think, actually on the note of docs, what people should be doing is, like, the parallel questions are very cheap. So if you have, like, big states you wanna ask many questions on

Diogo AlmeidaTypeSafe AI

给每条消息都加上 ID，然后针对每个 ID 问一个问题。也就是说，当你有一个很长的状态时……

Put IDs on every, like, message, and then ask a question about each ID. So like, when you have, like, a long state.

Diogo AlmeidaTypeSafe AI

这样你只需要为状态付一次钱，就可以针对其中每条消息问大量的问题。我觉得这是一个很好的办法，既省钱又……

So that way you can, like, pay for the state once and ask lots and lots of questions about each message within it. I think that is, like, a, like, a great way that, like, saves money and is.

Diogo AlmeidaTypeSafe AI

嗯，也许吧。也许……

Well, maybe. May

Diogo AlmeidaTypeSafe AI

嗯，我不……我希望大家少花点钱。

Well, I don't. I would like people to spend less.

Diogo AlmeidaTypeSafe AI

也许你可以把推理调用减半，然后每次配 10 次 Jev 调用之类的，或者用任何能解决以前根本不可能存在的问题的方式。等等，第四类用例是我所说的智能软件：天生可组合、能做以前不可能出现的各种奇怪又好玩的事情的软件。比如那个把 Jev 当编程语言的东西，不知道你看过没有，那个太酷了。天哪，要是我们知道怎么发额度就好了，因为我们的基础设施还处在非常早期的阶段，不然我真想给这些项目都发额度。

Maybe you do, like, one half the reasoning calls and like, 10 Jev calls each or something like that, or whatever solves the problem that, like, couldn't have existed otherwise. Wait, number four use case was what I described as, like, smart software. Like, software that's intrinsically composable and like, does, like, weird, fun stuff that could never happen before. Like the programming language as Jev thing. I don't know if you've seen that. That is so cool. Man, if we knew how to give out credits, because, like, we're really early in our infra days, I would wanna give all these projects credits.

Diogo AlmeidaTypeSafe AI

我觉得这就是我们梳理出的主要用例。操作电脑也有点从实时方向冒了出来，这非常酷。如果它足够可靠，我会超级兴奋。我猜我们可以让模型在这些用例上好很多，因为这个有点出乎意料，所以真的很酷。说回编程助手，这是现在正在发生的一件非常出人意料的事。

And I think that those are, like, how we've mapped out, like, the main use cases. Computer use has also come in kind of like the real-time direction as well, and like, that's really cool. If it is reliable, I am super jazzed about that. I suspect we can make the model a lot better at these use cases 'cause, like, that came out of left field a little bit, so that's, that's really cool. On the coding agent thing, and this is, like, a really surprising thing that is happening right now.

Diogo AlmeidaTypeSafe AI

我相信 Claude Code 和 Codex 是赢家，排第一和第二。我不完全确定，我没有密切关注……

Claude Code and Codex are, I believe, the winner, like, the number one and two. I'm not entirely sure. I don't follow closely

Diogo AlmeidaTypeSafe AI

但大致是这样。但它们是围绕单一模型的世界构建的，对吧？这对它们来说很合理，因为这一直是单一模型的游戏，你挑的基本是同一种模型，只是智能水平不同。

But like, it's roughly that. But they're built around a single model world? Like, and that makes a lot of sense for them, right? Because, like, it has been a one-model game where it's, like, kind of like the same model but different intelligence that you're shopping.

Diogo AlmeidaTypeSafe AI

但所有开源编程助手现在都他妈的兴奋坏了，因为他们正在把 Jev 用起来。问题在于，我相信他们在尝试很多奇怪的东西，但所有编程助手大致都处于差不多的水平，对吧？因为一个 while 循环能做的事就那么多。但只要有一个人找到一个杀手级用例，而且只能在那个编程助手里实现，大家就会蜂拥而至，因为它垄断了那个东西。但所有开源编程助手都能照搬，对吧？就这么简单。可我不知道 Claude Code 和 Codex 这些会怎么办，因为它们是围绕那个单一模型的世界构建的。

But all the open coding agents are, like, fucking jazzed right now because they're, like, getting their Jev on. And Like, the thing is, there's-- I'm sure they're trying a lot of weird stuff But all the coding agents are kind of roughly at, like, approximate parity, right? Because, like, there's not so much you can do with a while loop. But the moment one person finds one killer use case that, you can only do with that coding agent, everyone will flock to it because they have, like, a monopoly on that thing. But all the open coding agents will be able to copy that, right? The end. But I don't know what the Claude Codes and Codexes will do because they are built around that one model world.

Diogo AlmeidaTypeSafe AI

我觉得这会是一件非常有意思的事。我个人很想能和它们集成。我想和所有人集成。它们最终可能会做出竞品，我不知道。但作为基础设施提供方，在这件事上选边站不是我的职责，对吧？我只想服务全世界。但我不知道它们会不会这么做。我觉得这会让编程助手这场竞争变得超级奇怪。我对此太兴奋了。我现在正在让团队审阅我写的一份内部文档，讲的是我认为对编程助手有用的设计模式。希望我走回家之后就能把它分享出来。但我觉得外面还有太多值得探索的领域。天哪，要不是我手上有这摊事，我现在真想去折腾编程助手。

And like, I think that's gonna be, like, a really interesting thing. Like, I would love to be able to integrate with them personally. Like, I want to integrate with everyone. Like, I-- they might make competitors eventually. I don't know. But like, it is not me-- my job as Songfire Infrastructure to be opinionated on that, right? Like, I want to just serve the world. But I don't know if they would do that. And like, I think it'll make the coding agent game super weird. Like, I'm so excited for that. And like, I'm sure. I'm getting my team to review right now an internal document I made on design patterns I suspect will be useful for coding agents. So hopefully I can share it, like, right after I walk home. But like, I think that there's just, like, such ripe area for exploration out in the world. And like, it's, it's. Man, if I did not have this, I would love to experiment with coding agents right now.

Diogo AlmeidaTypeSafe AI

当然。

Of course

Diogo AlmeidaTypeSafe AI

哦，好。

Oh, yeah

Diogo AlmeidaTypeSafe AI

哦，我完全没聊安全和对齐吗？

Oh, did I not talk about safety alignment at all?

Diogo AlmeidaTypeSafe AI

我觉得我没有。可能我确实没有。

I think I didn't. I think maybe I didn't.

Diogo AlmeidaTypeSafe AI

哦。

Oh.

Diogo AlmeidaTypeSafe AI

当然。

Of course

Diogo AlmeidaTypeSafe AI

对。

Yeah.

Diogo AlmeidaTypeSafe AI

我觉得这是那种很危险的……

I feel like this is the kind of thing that is a dangerous

Diogo AlmeidaTypeSafe AI

话题。我很乐意聊。我就爱刺激。

Topic to talk about. I'm happy to talk about it. I live for danger.

Diogo AlmeidaTypeSafe AI

我们公司的品牌是混乱。不是 Jev 本身，而是离经叛道和混乱。

Our company brand is chaos. It's not Jev. It is irreverence and chaos.

Diogo AlmeidaTypeSafe AI

哦，那个……

Oh, the

Diogo AlmeidaTypeSafe AI

有意思。这……这是一个非常复杂、很微妙的问题。我其实很想更正式地写一篇回应。我手头有一个简短版本的回应……

Interesting. I. So comp-- it's a very complicated, nuanced thing. I actually do want to write a response to this more formally. I do have, like, a little bit of a short version of my response

Diogo AlmeidaTypeSafe AI

那就是，随着你做越来越多的 RLVR，RLVR 就像是……

Which is that, as you RLVR more, like, RLVR is like.

Diogo AlmeidaTypeSafe AI

所以 RLVR 其实并不真的关乎可验证的奖励。那条路早在推理革命之前就一直行不通。这正是任务这件事奇怪的地方，对吧？以前……哦，讲段有趣的历史。当年 RLHF 刚兴起的时候，有三股不同的努力，现在都被统称为后训练。而指令遵循绝对是其中最不受待见的那个「私生子」。大家不喜欢它，不想把它当回事，觉得它烦人。我去找预训练团队说：「伙计们，这才是魔法所在。」他们说：「我们要跑那么多模型扫参，你要我们等人工评测出来才决定用哪个模型？」当时大家都把大量资源投给代码生成团队，他们确实取得了一些成功，但他们非常努力地想在单元测试上做 RL，显然没成功，对吧？因为那需要推理。所以要说清楚，RLVR 不纯粹是关于奖励的，它也关乎整体的形态。其中一部分是推理被包含在了里面，像一个你在做事情时的隐变量。而在做事情的时候，你就是放任模型为所欲为，好让它们尽可能强大，去回答最难的问题。而整个「控制前沿节奏」的讨论，我觉得视角很窄，因为它假定所有人都需要做更多的 RLVR，对吧？而我显然不认为我需要在我们的模型上做更多 RLVR。

So RLVR is not actually about verifiable rewards. Like, that has been failing since before the reasoning revolution. Like. And that's the weird part about tasks, right? Like, back when. Oh, fun history. Back when RLHF was becoming a thing, there were three different things that, like, are now called post-training, different efforts. And instruction following was by far the, like, the vaster child. Like, people didn't like it. They didn't want to take it into account. It was annoying. Like, I talked to the pre-training team. I'm like, "Guys, this is the magic." And they're like, "We run so many model sweeps. You want us to wait for human evals to figure out which models to use?" And like, everyone is, like, giving tons of, like, resources to, like, the code gen team, which, like, they did s- have some successes, but they were trying really hard to do RL on co- like, unit tests. And it didn't work, obviously, right? Like, you needed reasoning for that. So re- so just to be clear, RLVR is not purely about the reward. It's about, like, the shape of everything too. And part of it is that reasoning is included in here, like this latent variable that you're doing things. And when you're doing things, you're just letting the models do whatever they want in order to make them be as powerful as you can to answer the hardest problems. And this whole pace the frontier discussion, I think is, like, a very narrow focus because it assumes that everyone needs to do more RLVR, right? Which, like, I obviously don't think I need to do more RLVR on our models.

Diogo AlmeidaTypeSafe AI

我觉得对我们这种形态来说，零就是最优的量。嘿，对吧？拜托。

I think zero is the optimal amount for our shape. Hey, right? Like, come on.

Diogo AlmeidaTypeSafe AI

所以我觉得这其实有点像障眼法：他们是在说，我们其实想继续做那件看起来危险的事，因为它会做危险的事。有人会说：「哦，也许是沙箱出了问题」，或者别的什么。是的，显然是，而且他们本可以轻松解决，对吧？但他们选择不解决，因为在这种「什么都能做」的类别里，你放手让模型做的事越多，它就越强大，对吧？所以我觉得这里存在某种责任的消解：不管是有意设计还是无意为之，他们想做的事被当成了一个默认前提。我们必须做 RLVR，而且不只是必须做，还要做得越来越多，在中间过程里赋予模型为所欲为的能力，因为这会教会它们在这之外也变得强大。而我们又不想好好限制这些东西，因为那会让它们在这些事情上稍微弱一点。

. So It's really, I think, a bit of a sleight of hand where they are saying that we actually want to keep doing the thing that looks dangerous because it does dangerous things. Like people say, like, "Oh, maybe the sandboxing was a problem," or whatever else. Yeah, obviously it is, and they could have easily solved that, right? But they chose not to because the more things you let the models do in this do anything category, the more powerful it is, right? So like, there-- I think there's some, like, disillusion of responsibility there on, like, things that by design or non-design they're trying to make is just an assumption. We must do RLVR, and not just we must do it, we must do more and more and more, with giving the models, like, the power to do powerful-- do anything they want in the middle 'cause that teaches them to be powerful outside of it. And we don't want to limit those things well because it'll make it slightly less powerful on those things.

Diogo AlmeidaTypeSafe AI

所以，如果你接受了所有这些前提，他们就会说：哦，对……

So like, if you assume all of that, they're like, "Oh, yeah

Diogo AlmeidaTypeSafe AI

我们正走向一个危险的世界啊，各位。

We're heading to a dangerous world, guys."

Diogo AlmeidaTypeSafe AI

「所有人都会这么做，这是让 AI 变强的唯一办法。」所以……

Like, "Everyone is gonna be doing this, and this is the only way to make AI sick." So

Diogo AlmeidaTypeSafe AI

当然。

Of course

Diogo AlmeidaTypeSafe AI

当然。我觉得……在「苦涩的教训」这个方向上，真正定下过对的任务、也就是开创 AI 新方向的人非常少。或者说新的北极星。那是很罕见的。再说一次，我觉得对 LLM 本身来说，大概只有 2.2 次左右，比如 RLHF，然后是 RLCD。

Of course. I think there's-- Like, on the bittersweet lesson direction, I think that there's very few people who've, like, made right tasks. Like new directions of AI. That is-- Or new North Stars. That is rare. Again, like, I think 2.2 times or something for LLMs itself, like RLHF and then RLCD.

Diogo AlmeidaTypeSafe AI

在我看来，RLVR 大概只算 0.2，而且我觉得这已经很宽容了。

RLVR is like a 0.2, in my opinion, and I think that's generous.

Diogo AlmeidaTypeSafe AI

或者 0.5，甚至算一整个 1 也行，我无所谓。但我确实觉得，大家在这类事情上想得太封闭了。而这件事唯一该负责的是研究人员，肯定不是普通大众。大众只会默认 OpenAI 和 Anthropic 已经在尽力而为，他们也不是那些清楚真正还有哪些选项的专家。

But or 0.5 or, like, it could be one whole one. I don't really care. But I do think that people are thinking very close-mindedly about this type of thing. And this-- the only people who are at fault here are the researchers because it's definitely not the populace. Like, they just assume that OpenAI and Anthropic are just doing the best they can, and they are not the experts who are aware of the true optionality available.

Diogo AlmeidaTypeSafe AI

对。

Yeah

Diogo AlmeidaTypeSafe AI

是的。好吧，我在尽力，但我的目标并不是说服各个实验室，还有别的方向

Yes. Okay. Well, I'm doing my best, but like, my goal is not, like, convince labs that there's, like, other directions

Diogo AlmeidaTypeSafe AI

可以走。我的目标是点燃软件工程师的希望，让他们开始真正把那些一直想自动化的事情自动化。我写过一篇文章，讲我想要的 AI 的未来，但我的团队没让我发。里面有很多小事。比如，还记得「照我的意思做」吗？想象一下，如果所有东西都能「照我的意思做」，因为那个演示演的就是这个。

To go down. My goal is have-- it's like spark hope in software engineers to start, like, actually automating things they've always wanted automated. I had this, like, article that I wrote that my team didn't let me write, that didn't let me publish about, like, the future I want of AI. And like, there's, like, a lot of, like, little things. Like, remember do what? Imagine if everything could do what 'cause, like, that demo was do what.

Diogo AlmeidaTypeSafe AI

你可以……就是说，这里面是分层次的

Like, you could. Like, there's levels

Diogo AlmeidaTypeSafe AI

嗯？

?

Diogo AlmeidaTypeSafe AI

对。我们以前还做不到「照我的意思做」，因为计算机太基础、太死板了，但那个操作电脑的演示正是这样。我觉得世界上会出现各种层次的顺滑体验，是大家现在还理解不了的。还有那种处处都有智能的愿景……我不想过度承诺。我不觉得它马上就会实现，但我们会不惜一切代价去让它实现。

Yeah. And like, we couldn't do what yet because, like, computers are so basic and literal, but that computer use one was just that. And I think that there's, like, levels of smoothness that'll happen in the world that people just don't understand. And like, the promise of, like, smarts all around are. It's, it's, it's-- I don't wanna overpromise. I don't think it's going to happen right now, but like, we are gonna do whatever the fuck we can to make that happen.

Diogo AlmeidaTypeSafe AI

中训练。这都是一个连续的光谱。

Mid-training. It's all a spectrum.

Diogo AlmeidaTypeSafe AI

对吧？比如，我是不是

Right? Like, am I

Diogo AlmeidaTypeSafe AI

对。它其实是一种省钱的手段。

Yeah. Like, it's, it's, it's like, it's a cost-saving thing.

Diogo AlmeidaTypeSafe AI

省得再从头预训练一遍。这里面有很多有意思的东西。我其实觉得，智能在每一个层面都有某种说不清道不明的东西，而且永远都特别迷人。我是偏空间直觉的 shape rotator，所以我不喜欢自己去找那种东西，但我很喜欢别人找到之后教给我。至于看数据这件事，我们的数据团队特别擅长，而我不擅长。

Instead of, like, having to pre-train again. Like, there's intriguing stuff. I actually think that, like, intelligence has a je ne sais quoi at every single level, and it's always super-duper fascinating. Like, I'm a shape rotator, so I don't like finding that, but I love it when people find it and teach me about it. But looking at the data, this thing that, our data team is so good at that I'm not.

Diogo AlmeidaTypeSafe AI

我觉得这非常迷人。我很喜欢思考能力是怎么被放进模型里的。短期来看，有微调带来的那种非常快速的对齐；长期来看，同样的东西反复见过一遍又一遍之后，会被越来越深地刻进模型里，直到变得稳健。而这就是要浮现出来的北极星，System One 那类东西，正是最终会变得稳健的部分。所以我觉得中训练很迷人。我支持一切形式的训练，支持一切能浮现出新型智能的方式。只是我不会全都自己做，因为太贵了。我私下也说过，而且……

It's-- I find it really fascinating. I love actually thinking about, like, how capabilities are, like, put into the model, like, over, like, the short term. Like, there's, like, the really rapid alignment of fine-tuning and over the long term. After seeing it over and over and over again, like, this stuff gets baked deeper and deeper and deeper and deeper into the model until it gets robust. And that is, like, the North Star to surface, and like, the System 1 stuff is the stuff that ends up getting robust. So I find mid-training to be, like, a fascinating thing. I'm a fan of all forms of training. I'm a fan of all forms of, like, surfacing new types of intelligence. I wouldn't do it all myself because it's expensive. And I have said privately, and also.

Diogo AlmeidaTypeSafe AI

我该说这个吗？嗯……我的理念是，任何我会私下跟投资人说的话，我都应该公开跟大家说，因为这就是

Should I say this? Huh. Huh. Like, my philosophy is anything I sh- I should say in, like, private with, like, an investor, I should say in public with the people because that is, like

Diogo AlmeidaTypeSafe AI

我的风格。

My thing.

Diogo AlmeidaTypeSafe AI

是的。我以前说过一句话：如果你给我 10 亿美元，我不会去做预训练。我现在仍然这么认为。那是件非常昂贵的事。如果你是 AI 工程师，你可以把东西切开、拼起来，做各种操作。像弗兰肯斯坦那样东拼西凑并不是最优雅、最漂亮的做法，但它能解决问题啊，宝贝。

Yes. So m- the thing I've said i- before is if you gave me a billion dollars, I wouldn't pre-train. I still believe that to be true. It is a very expensive thing when. If you are, like. Like, if you're an AI engineer, you can, like, slice and dice and do all sorts of stuff. Like, Frankensteining is not the most elegant, beautiful thing, but it solves problems, baby.

Diogo AlmeidaTypeSafe AI

所以，除了预训练，什么都行。

So - Anything except pre-training.

Diogo AlmeidaTypeSafe AI

那完全是两回事，是非常不同的概念。我稍微拆解一下。多模态有点不一样

Those are completely different things. Those are extremely different concepts. I will, like, break that down a little bit. So multimodality is a little bit different

Diogo AlmeidaTypeSafe AI

因为有时候其他模态有帮助，有时候反而有害。

Because sometimes the other modalities help, sometimes they hurt.

Diogo AlmeidaTypeSafe AI

比如大家好像在逐渐放弃语音（这和音频不一样），因为它似乎没法很好地泛化到其他东西上。

Like, people are moving. They seem to be moving away from speech, which is different than audio, because it seems to not generalize well to the other stuff.

Diogo AlmeidaTypeSafe AI

这也许会被解决。这些我都支持，但它们都是实打实的经验性问题。扩展定律不是说砸钱就能变好。从务实的角度讲，扩展定律回答的是一个东西到底有多好。存在这样的情况：不管你怎么扩展，它都可能不够好。据我了解，操作电脑目前还没被解决。我希望我们能在解决它的过程中出一份力。但也可能我们收集再多数据都解决不了，可能需要更好的方法或者别的什么。所以……

This might get solved. I'm a fan of all of this, but these are, like, empirical, real questions. Like, scaling laws are not about just throw money at it and it gets good. Scaling laws are pragmatically how good is a thing? Like, there are worlds where, like, no matter what you scale, it may not be good enough. So Y- like, computer use is not currently solved is my understanding. Like, I'm hoping that we can be a. Like, play a part in solving that. But like, it. There might be no amount of data we collect that will solve that. We might need better methods or something else like that. So

Diogo AlmeidaTypeSafe AI

在这一切里你都得非常务实。我支持全能模型吗？我支持一切形式的智能，但我要直接说你提到的一点，它和预训练不同，是后训练，因为我讨厌把智能割裂开。对我来说这才是坏事。而现在这种聊天优先的推理模式，就是在逼着智能被割裂。当你为聊天做优化时，往往就是纯 RLHF，而 RLHF 本质上就很容易做出大家常抱怨的那些事，对吧？比如，「哦，我要……」

Like, we. You need to be, like, really practical in all of this. Am I a fan of omni models? I'm a fan of all forms of intelligence, but I will go straight into one thing you talked about, which is different from pre-training, which is post-training 'cause I hate fracturing intelligence. That is, like, the bad thing to me. And this whole, like, chat-first reasoning mode is because, it forces the intelligence to be fractured. Like, when you're optimizing for chat, this tends to be, like, pure RLHF, and it's quite intrinsic in RLHF to do the stuff people, like, naturally complain about, right? Like, oh, I'm gonna

Diogo AlmeidaTypeSafe AI

对。

Yeah

Diogo AlmeidaTypeSafe AI

谄媚，sycophancy，还是读 psychophancy

Sycophancy, psychophancy

Diogo AlmeidaTypeSafe AI

我也不知道是哪个词

I d- whatever word

Diogo AlmeidaTypeSafe AI

或者怎么发音。还有过度自信、幻觉。甚至连在 LM Arena 上吃香的那种风格：加粗、斜体、表情符号。它不会简单地回答问题，而是给你写一大篇，然后再问你一个追问，让人觉得更像真人在跟你聊。所有这些都源于字符串是非常奇怪的东西，真的特别怪，你必须校准变差，必须丢掉一些模式，必须超级自信，才不会跑偏，因为一旦跑偏，奖励模型会狠狠惩罚你，因为那太明显了。然后这会彻底扭曲概率空间，还会和推理模型的概率空间相互作用，对吧？因为模型就是那种简单的线性东西，容易有点作弊。所以我猜，这和把智能暴露出来是非常不同的。智能的很多门道都在于研究这种细微之处，而至少我在 OpenAI 的时候，大家并没有真正去研究，因为他们就是一句「聊天」，就像现在大家对 Jev 的反应一样。

How- or how to pronounce that. Overconfidence, hallucination. Like, even the kind of style that excels in LM Arena, bold, italicized, emojis? Like, it doesn't answer the question simply. It gives, like, a long write-up, and then it asks you a follow-up question so it feels more like a human talking to you. All of these things, come because strings are super weird? They are, like, weird-ass things, and you need to be miscalibrated. You need to, like, mode drop. You need to be hyper-confident in order to not go off the rails 'cause the reward model will punish you so hard when that happens 'cause it's obvious. Y- and then this, like, warps the probability space entirely, and it interacts with that of the reasoning models, right? Because, like, it. The models are, like, these simple linear things that tend to cheat a bit. So I think that's very different than exposing intelligence is my guess. And a lot of the art to intelligence is studying this subtlety that I think that, at least when I was in OpenAI, people were not really studying that because, like, they were just like, "Chat," just like people are on with Jev right now.

Diogo AlmeidaTypeSafe AI

是的。但如果你在里面尝试……常见的说法是，你可以有两个目标，同时优化两个，但这恰恰就是割裂本身，对吧？所以就是这样。

Yes. But if you try in there. And like, the saying is, like, you could have, like, two objectives and you could just, like, optimize for both, but then that is literally the act of fracturing, right? So yeah.

Diogo AlmeidaTypeSafe AI

哦，这

Oh, it

Diogo AlmeidaTypeSafe AI

这有点不一样。不是的。如果我可以补充，如果我可以辩护一下

It's a little different. No. If I could, if I could add, if I could defend

Diogo AlmeidaTypeSafe AI

System Two 任务的话：第一，我们并没有把 System Two 任务扔掉，对吧？你可以试着让 Jev 去做它们，而且这里其实存在一个聪明的回答，那就是「不知道」。在 System Two 任务里，行为是有好坏之分的，好的行为应该是置信度很低、不确定性很高。也许某些启发式方法能在这里那里推动一点，但说清楚，我们也在乎这些任务。我只是觉得那不是这种智能天生擅长的东西。所以我们并不是要割裂什么。而所有的割裂都会让模型变笨。比如有人让模型说它是 OpenAI、Qwen 或者 Claude 之类的，我也不知道它们现在都怎么说。我不会往模型里灌「你是 TypeSafe 的 Jev」。那会造成割裂，对吧？我不想要那样。去代表互联网是怎么想的，对吧？要正确。这才是我要的，因为这样才能得到顺滑、可预测的智能。

The System 2 tasks, number one, like, we don't toss out the System 2 tasks, right? Like, you can try to make Jev work on it, and there actually is an intelligent answer for that, which is unknown. Like, my. Like, there. Like, there is better and worse behavior in the System 2 tasks, which should be, like, really low confidence, lots of uncertainty. Maybe some heuristics can, like, move the needle here and there, but we care about them too, just to be clear. I just think that is not what the. What is. The intelligence is native to. So we're not trying to fracture anything like that. And all fracturing makes the model dumb. Like, if people, like, get the model to say, like, it is OpenAI or Qwen or, d- like, Claude or whatever else, I don't really know what it says this d- these days. I am not going to put into the models that you are Jev from TypeSafe. That fractures it, right? Like, it. L- like, I don't want that. Like, represent what do the internet thinks, right? Like, be correct. That is what I want because that's how you get the smooth, predictable intelligence.

Diogo AlmeidaTypeSafe AI

对于……对于

I- for, it- for

Diogo AlmeidaTypeSafe AI

对于第一方产品来说，是的。

For a first-party product, yes.