llamafile is not really “effective”. it’s incredibly impressive, but it’s the opposite of effective. it’s a collection of a bunch of hacks reliant on coincidences in OS design, and works by basically recompiling itself on the fly to work with different architectures.
if you want effective, run llama.cpp compiled with actual optimizations for your platform.
By effective I mean, I downloaded one file, renamed it to .exe, ran it and now my shitty company laptop was running a basic LLM with actually lots of features, right then and there. I didn’t even have a GPU or admin privileges, and it just worked
Sure, maybe it’s possible to squeeze more performance out of a 3 years old laptop, but that was actually very usable out of the box. And I didn’t even need to unbox it !
I don’t care who they are or what their Xitter history is.
The tools is great, the tool is not backdoored. I ruthlessly use effective tools that I can get my hands on.
Using open source software on its own does not even entails economic support for its creator.
llamafile is not really “effective”. it’s incredibly impressive, but it’s the opposite of effective. it’s a collection of a bunch of hacks reliant on coincidences in OS design, and works by basically recompiling itself on the fly to work with different architectures.
if you want effective, run llama.cpp compiled with actual optimizations for your platform.
By effective I mean, I downloaded one file, renamed it to .exe, ran it and now my shitty company laptop was running a basic LLM with actually lots of features, right then and there. I didn’t even have a GPU or admin privileges, and it just worked
Sure, maybe it’s possible to squeeze more performance out of a 3 years old laptop, but that was actually very usable out of the box. And I didn’t even need to unbox it !