Edition #105
Built Agent Testing with CLI tooling, organising experiments and prof G on AI stocks
Welcome to another edition of our Builder Series, where we take you behind the scenes and give you a raw, authentic look at how we are building/fighting with AI at Kerno.
Part 1: Let’s talk tech
Part 2: Global Changelog, a look at some of the most interesting stories of the past week.
So let’s dive in.
Let’s talk tech!
Topic 1: Agent Testing Pipelines… Bulk Testing our Docker Compose Agent
Scaling our agent testing pipelines is incredibly important to accelerate our progress towards ensuring our agent always delivers high quality tests across a variety of conditions.
Over the past week, Antoine, Applied AI Engineer, ran our testing pipelines across 18 different codebases to assess the performance of our Docker Compose Agent. It took 8 hours for the pipelines to run, and then needed annotating.
Some codebases had running Dockerfiles, others didn’t. Main languages covered were Python and Typescript.
The result: 50% of passed first time, 50% timed out.
Read more about our findings here.
For the full video of how we ran the experiments, click on the video below.
Check out the full article by Antoine Frau here.
Topic 2: Applied Learning from running Biology Experiments on Computational Experiments
Dr. Michael Coughlan provides some practical tip for AI teams, data teams and engineers to run computational experiments at scale. A lot of the tips can be implemented today, and can help set up your projects for scale.
Read the full article here.
By the way, Kerno is now out of closed beta. Get started with 30 free test runs on us.
We have also released support for MongoDB.
Get started for free www.kerno.io
Image of the week
This applied to anything works… linkedIn, SAST, Rebasing…. we have a habit of fixing things that don’t need to be ‘fixed’.
Weekly look at all the weird, whacky and almost unbelievable updates from the markets
Prof Scott - take on AI Market
Prof. Scott Gallaway or Prof G. has some interesting views on AI markets as a whole. Scott represents an important actor in the economy - ‘The Professional Investor’. So while we are all in the trenches, understand the tech deeply, Scott takes a macro view, and is representative of a lot of people who hold the investing dollars. So, perhaps see this as a way to understand investor sentiment on Wall Street.
Here is what he has to say:
I’ve never seen a bull market that more people hate. I almost feel as if people would be somewhat relieved if it just went down 20%, but the market continues to climb the wall of worry.
The Bear Case:
AI could be a bubble. AI capex hit $350 billion in 2025, up from $200 billion in 2024. Amazon, Google, Microsoft, Meta, and Oracle raised over $100 billion in debt this year — that’s more than 3x their nine-year average. We’re seeing circular deals where Nvidia invests in OpenAI and then OpenAI takes that money and buys compute from Nvidia. OpenAI could be the triggering event for some painful market drawdowns.
Second, valuations look expensive. The S&P is trading at 31x earnings — not quite dot-com levels, but we’re at 1999 levels. It’s uncomfortable to invest when prices appear this high.
Third, maybe we’re just due for a correction. We’ve had three big years in a row: 24% return in 2023, 23% in 2024, and we’re tracking 17% in 2025. This level of consistent returns doesn’t usually continue.
The Bull Case:
Interest rates are coming down. Rates are at their lowest level in three years. Powell’s Fed tenure ends in May 2026, and Trump’s new Fed chair might just keep cutting rates. In a lower interest rate environment, earnings should lift across the board. It doesn’t make sense to sell when we’re entering a low-rate environment.
Second, deficit spending is going to prop things up. The Big Beautiful Bill will add $480 billion in fiscal support. Yes, it’s irresponsible long term, but for 2026 specifically, that’s free money pumping into the economy.
Third, while AI might be a bubble, it’s not a particularly dangerous one yet. Companies like OpenAI, CoreWeave, Oracle, and maybe Palantir are behaving dangerously. But the Big Tech companies that really matter — Microsoft, Google, Meta, Amazon — have tons of cash on their balance sheets and incredible businesses that work with or without AI.
Read more on Prof G’s markets here.
I think the most important statement of the above is
“Third, while AI might be a bubble, it’s not a particularly dangerous one yet. Companies like OpenAI, CoreWeave, Oracle, and maybe Palantir are behaving dangerously. But the Big Tech companies that really matter — Microsoft, Google, Meta, Amazon — have tons of cash on their balance sheets and incredible businesses that work with or without AI.”
The establishment have the cash to take on bets outside of their core business. Meta still makes money on ads, MSFT still makes money on software… AI is not their core business, so if it does not pay off, they still exist as major players.
Other news, tools and research
OpenRouter, State of AI report.
Empirical report analysing 100 Trillion tokens
Source:https://openrouter.ai/state-of-ai
Defeating non-determinism in LLM inference.
Note: Experiment is based on floating-point truncation, very specific and expensive but worth the read.
Source: https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/
Thanks for reading, and remember..just build something!
P.S if you want to want to share your AI story on AI Builders, feel free get in touch.
For more AI Builder Series Editions, subscribe here.
