2 years in and I'm still learning basics.
Building a new rig - pulled a 8GB ddr5 stick out of my windows machine to get it running while I await my DDR5 RAM kit.
Installed Ubuntu 26.0.4. Installed ROCM. Installed llama.cpp. Used my modified run scripts from my AM4 machine. Model taking ages to load. Slow as hell.
Well, I guess Ubuntu 26.04 isn't ready for prime time.
Back to Ubuntu 24.04.4. Installed everything. Still loading slow af. Wondering if my pcie5 nvme is busted.
Did some research. Realized I don't need mmap. Added --no-mmap flag.
Loaded in seconds.
I never even knew what mmap did. Never thought to disable it. GPUDirect loading is so fast, I could have been doing this for years.
Now I know. Maybe now you know too - if you're loading models off a high-speed nvme drive, you don't need mmap. --no-mmap.
Now I need to decide if I want to go through the whole thing again (3rd time) to get Ubuntu 26.04 going.
Happy Mothers Day. Call your mom.
[link] [comments]