llamafile is the new best way to run a LLM on your own computer

I've been following the development of Cosmopolitan Libc and Actually Portable Executables (APE) for the past year. Created by Justine Tunney (Redbean), it almost feels unreal that this wasn't discovered and implemented 20 years ago. By encoding Windows Portable Executable as Unix Sixth Edition shell scripts, it's possible to compile a C program with Cosmopolitan that can run on nearly any platform and is still smaller than Go's "Hello World". It works in conjunction with PKZIP, which was designed to place its magic marker at the end of file, rather than the beginning. That means programs compiled with Cosmopolitan are actually zipped directories, and can be inspected by the end user.

I recommend reading Justine's full blog post about Cosmopolitan and Actually Portable Executables. It's approachable to anyone with basic programming knowledge and a burning curiosity. However, to focus on the article being reviewed, let's divert back to using APEs to run LLMs.

> A llamafile is a single multi-GB file that contains both the model weights for an LLM and the code needed to run that model—in some cases a full local server with a web UI for interacting with it.

See all that's transpired from a bored developer poking around? I wish I could provide you with a review of actually using a llamafile, but I don't have a Nvidia GPU or MN Mac. However, every day the pressure gets stronger to purchase one or the other.

I can't decide what next steps I should take. Should I invest in a new Nvidia GPU, which would invariably require me to upgrade my power supply and deal with three-year-old sadistic cable management? Or, should I save a little more and just but a new MacBook Pro? The MN lineup is extraordinary. My M1 iPad Pro demolishes animation, video editing, and all my other tasks, as I use it for 99% of my non-programming workload. But, I already have a laptop I hardly use. I'd worry that my investment would be a waste. That I would only use it for AI and end up back on my desktop. Now, I could get a switch to allow me the freedom to use both systems, and that's what I'd probably do, but I know I wouldn't be getting the most out of either system. Still, AI is what Crypto wished it could be. A revolutionary, accessible, technology rivaling the power of electricity to change our lives. It is automated creativity. Letting this early, innovative epoch go by without at least dipping my toes in could be a lifetime regret.

I was ten when the App Store came out. I remember sitting in my Mom's car and thinking "If only I could build an app". I had no idea how to code, or what went into running a service. But Looking back now, thinking of sites like Twitter, Instagram, and Facebook as they were in 2008oo I could build that today. If I invest the time now into AI development, I'll be with the curve. I'm will within the pool capable of building a product. And I have the energy, income, and time. This could be an investment.

Or it could be me talking myself into wasting money.