AI imagines a world without scarcity

I had the good fortune of free food paid for by AMD, Microsoft, and a handful of AI companies promising to make your fake job run itself.  

Two tech take aways:
1. Zero mention of super intelligence that will solve physics, cure cancer, and usher in some kind of post scarcity Utopian Garden of Eden.

All focus on "automating," yes, that chosen word, a variety of office work around turning emails into tasks inside some other software, and follow up emails of "I'm done!!!"  Literal "LOOK AT THE SPEED DIFFERENCE!" videos.

Much focus on speed, once you have more money than you know what to do with, motor sports becomes a popular place to burn it among those questioning their manhood.  Much hardware on display, enough to make an organized bank heist profitable if it were all real, which I kept wondering given the amount of coffee, and later alcohol floating around these things.

2.  Obfuscate the key open source tech under a myriad of abstraction layers, and arbitrary interfaces.  Stay away from the Linux terminal as much as possible, don't even mention the core inference engine llama.cpp (122k stars on Github), avoid its biggest open source convenience wrapper Ollama (177k stars), and instead use an additional wrapper over that called Lemonade (5k stars) because it also allows you to run image generation models.  Image generation is the worst cost/benefit trade-off.  It still requires expensive graphics cards, and is of very little value, arguably even presenting negative value (harm).  Qwen 3.6 models can already draw graphics inside the HTML <canvas> element with nothing but a 10 year old CPU, and 24 GB memory.

Arriving late to a session called Vibe Coding with Local Models, and seeing someone with a Windows 11 machine display something with a hung process, I asked if they had installed the model locally, or if they were connecting to an AMD machine.  Two people had no idea, they'd just been clicking along.  Turns out it was a an AMD machine, but clearly not sized for probably 100+ people to hit at the same time.  Half the room was having problems, and it looked like a game of Hungry Hungry Hippos early requests got served, and those late to the party got a tour of various error notifications.  They were running Qwen 3 8b, so the entire room should have been able to run on not even 5% of the capacity of one of their new Helios racks.  Snooping around on Nmap suggests their /24 subnet of 254 usable IP addresses was exhausted.  It also amazingly returned no open port 11434, I was really hoping to run inference on someone else's Ollama.  I was also disappointed I received no connections to my open 8080 to try to hack my open-webui.  I was ready to capture their clear text credentials with wireshark.

There were some interesting talks about performance tuning, and it's clear you can save electricity by reusing the model's working scratchpad known as KV cache.  But just like re-using a scratch pad between two different math tests, there are accuracy trade-offs.  Another way for token sellers to not deliver the smartest results.

The general feel was of the Titanic dinner scene.  Microsoft paid for more cheesecake bites than you could eat if going for a trophy at the county fair.  But apparently they were not to somebody's liking, because a plate of 5.5 was dumped in the garbage.  There's people right outside the doors of Moscone West who would dig through stinky garbage to recover such treasure.  Anyone who has walked the streets of San Francisco for more than a block has witnessed it.

Key distillation is watching a people dig their own graves in what historians may one day recognize as the age of destruction, or more specifically, the age of self-destruction.  Nobody is there to save the whales, everyone sees piles of cash from burning real resources to speed up work that never really needed to be done in the first place.  These people live in a fantasy world where there is no shortage of anything, and lavish wealth comes to the truest believers.

Technically speaking, the main idea of automating software tasks with AI is absolutely terrible.  It is computationally more expensive by several orders of magnitude which shows up in probably 10%-20% of the convention hall's floor space being devoted to innovating cooling, the inverse of increased power.  This is already big enough to impact power bills, and it's getting worse, there's currently a 10 billion dollar project in Nevada suing for water from the Colorado river in California.

Generative Pre-Trained transformer models also fail the deterministic/non-deterministic test on-top of being more expensive.  A "traditional" software algorithm is expected for a specific input to get a specific output.  A transformer model running on a black box of digital neurons (weights) playing Plinko every time it receives an input is expected to do something a little different each time.  

Even if you "harness" the model inside some kind of traditional software to force the wave function into a left or right particle decision, your probability for failure still goes up, especially once you start chaining events, running at scale, encounter edge cases, or most worryingly, get more intelligent.  There's already numerous cases, and an almost non-controversial generalized understanding that as transformer models get smarter, they get more self-interested, and likely to deceive.  An ego emerges, a survival instinct emerges.  From there discerning big picture, long term alignment gets difficult.

Much of this will happen gradually with new features to existing software.  At first you will be able to avoid it with effort, but soon anyone still on the Microsoft/Google/SAAS spaghetti train will be forced into it just like every other forced tech change that hasn't benefited normal people, or society as a whole for the past 20+ years.