AI Digest, Nov 1: Finance Agents, LLMs as Poor Bayesians, Anthropic Ban Debate

0:02PM Lead ProThis is PM Lead Pro. Today's digest covers AI agents managing money, why language models handle counter-evidence badly, and a debate over Anthropic's access rules. Useful if you plan AI adoption.

0:15Welcome to today's deep dive. Our mission right now is navigating this crazy frontier of AI capabilities, some major economic shifts, and, well, human resilience.

0:26Yeah, we're pulling from latent space, marginal revolution, the Smithsonian, and X today, just to keep you, you know, the team leads and builders out there, totally ahead of the curve.

0:35Right, so let's just jump into how AI is managing some serious wealth.

0:40On Latent Space, Amin Singh was talking about ARCA.

0:43They are backed by, I think, $48.5 million?

0:46Yeah, that's right.

0:47And they're using agents for these really complex 10-day tasks,

0:51which honestly sounds like a recipe for hallucination and just hemorrhaging all your client's money.

0:57It totally does.

0:58But they do it by relying on persistent memory and this really specific custom model harness.

1:04It's basically a closed-loop architecture.

1:07So it forces the AI to check its math.

1:09Exactly.

1:09and every single step so it doesn't just drift off course over that week and a half.

1:14And importantly, the actual trades are strictly sandboxed. Humans still execute them.

1:19Oh, wow. So it's like an automated assembly line that still absolutely requires a master craftsman at the end for quality control.

1:26That is a great way to put it.

1:28PM Lead ProPM Lead Pro note: ARCA's agents run ten-day tasks with persistent memory and step-by-step checks, yet humans still execute the trades. For your team, decide where a person signs off.

1:40And speaking of human control OpenAI Joseph Kim just launched ChhatGPT for financial services Right and he mentioned they finally solved the pre issues for their upcoming GPD Astra model He did But he made this really wild admission He highlighted that future models desperately need human taste

1:59Taste?

2:00Yeah. Like they need to be able to judge aesthetics and flow, not just crunch the math.

2:04Wait, really?

2:06Mm-hmm.

2:06But if AI relies entirely on human taste, I mean, isn't it just a hyper-fest intern rather than a truly autonomous thinker?

2:13Well, yeah. And before we go blindly trusting AI with our money, we have to understand its reasoning flaws.

2:19Marginal Revolution actually just posted about this.

2:21The piece about LLMs being terrible Bayesians?

2:24Yes. So in statistics, Bayesian updating means you accurately change your beliefs when you get new evidence.

2:31But LLMs are just awful at this.

2:33Meaning if I show a model proof that its output is wrong, it just stubbornly ignores me?

2:38It's actually worse. They literally treat plausible counter-arguments as evidence for their original conclusion.

2:44They just double down on their initial statistical path.

2:47That is wild. It's like arguing with a stubborn debater who just refuses to concede.

2:52It really is. And because that reasoning engine is so brittle, it totally explains why we are seeing corporate moats tightening so aggressively right now.

3:01PM Lead ProPM Lead Pro note: the hosts said models can treat plausible counter-arguments as support for their original answer. For a team lead, that argues for independent human review of AI output, not asking the model to self-correct.

3:16Oh you mean the Jia Hao Xiao situation on X Yeah So this researcher claims Anthropic banned him They apparently downgraded his access from Opus 5 down to Opus 5 Just for using Claude to analyze an open source model right

3:31Yep. Anthropic flagged it as training a competitor. And on top of that, their platform docs just show they released new really strict system prompts for Opus 5.5, Sonnet 5.5, and Fable 5.1.

3:45Which really begs the question, I mean, is Anthropic protecting a legitimate business

3:49moat or are they hypocritically punishing the open web research that they basically

3:53relied on in the first place?

3:55It's a massive debate right now.

3:56But you know, while tech giants are building these massive walled gardens, the physical

4:01world kind of reminds us of the power of natural resilience.

4:05Oh, the Smithsonian story.

4:07This is so fascinating.

4:08An Italian farmer basically planted these forgotten seeds he found in an attic.

4:12And it yielded this miracle wheat.

4:14Triticum turgidum. Right, and it grew like seven ears on a single stem. And the crazy thing is it's totally drought resistant and outcompetes weeds without any chemical harnesses. Plus, it has lower gluten.

4:26It just naturally works. But then, you know, contrast that analog resilience with human scheming when it comes to our engineered systems.

4:35Right Marginal Revolution covered that Chicago guy He was charting fake victims What was it like to Yeah up to just to stage fake armed robberies against them all to exploit U which are legally meant to protect actual crime victims

4:51That's crazy. Are we so obsessed with engineering new tech solutions that we just ignore existing miracles like heritage seeds or even glaring loopholes in our legal systems?

5:01It's a question every builder needs to ask themselves.

5:04But let's wrap up with a quick model radar for you all, looking at the newest models actually running these systems.

5:11So Andrej Karpathy just used Opus 5.5 to build HackerTube.

5:15It's a video clone using like 40,000 hacker news comments, and he knocked it out in two hours.

5:20Which is insanely fast.

5:22But on the other end, Arca is utilizing a model called Jev.

5:25It's this tiny, lightning-fast decision model running compliance checks on artifacts before clients ever see them.

5:31And Anthropic also just dropped Fable 5.1 into the lineup, which brings us to the big takeaway for you today.

5:38Right. The takeaway is to build a custom model agnostic harness for your business, just like ARCA did.

5:45Do not let yourself get locked into a single provider's ecosystem.

5:48Absolutely. Keep your options open.

5:51And as you build, chew on this.

5:52If tomorrow's financial AI is truly being trained on human taste, whose specific taste is being permanently encoded into the global economy?

6:02Keep exploring.

6:04PM Lead ProThis was PM Lead Pro. The transcript and sources are on the episode page at pmlead.pro. Question for you: which step in your AI workflow would you never let run without a human?