• 0 Posts
  • 6 Comments
Joined 8 days ago
cake
Cake day: May 22nd, 2026

help-circle

  • timochka@lemmy.ziptoSelfhosted@lemmy.worldHardware for local inference?
    link
    fedilink
    English
    arrow-up
    5
    arrow-down
    1
    ·
    15 hours ago

    Word of advice - don’t scavenge old /server/ hardware if you plan to put GPUs in - unless you really like heat and noise. Those machines from the likes of HP will take one look at a consumer graphics card in a PCI slot and decide they need to run all the fans at 100% to ward off evil spirits. Not to mention just the general ballache of proprietary PCI riser cards, PCI power cables, etc. etc.

    You’re definitely better off with taking someone’s old gaming rig off their hands.

    In terms of specs - value VRAM above everything else. A slow, old 3000 series card with 24GB of VRAM is much more useful than a brand new 5000 with 16GB. If you can find old RTX3090 24GB, they’re kinda ideal.

    The one thing I will say for modern cards though is that they’re much better for power efficiency - and in particular idle power (which is important if you’re running the thing always on.) For my main LLM machine I have two RTX5060Ti (32GB total), which at the time was the sweet spot for price/performance/power, and it’s very nice that they idle around 3 or 4 watts. I bought them before the world went crazy and prices went mad though, so they may not be the sweet spot any more.

    One you’re in 32GB VRAM type territory, you can run really really good dense models like Qwen-3.6-27b at a decent quant, decent context size, and good performance for things like coding, or bigger MoE models for more general use (particularly then if you have good CPU and regular RAM for offloading to CPU. For use as an assistant (i.e. not an OpenClaw fully automated slop machine,) I use 3.6-27b as a daily driver in Claude Code, and basically never use Sonnet.



  • timochka@lemmy.ziptoSelfhosted@lemmy.worldIs Plex really Self Hosting?
    link
    fedilink
    English
    arrow-up
    2
    arrow-down
    1
    ·
    3 days ago

    For what it’s worth, I had a pretty much identical experience a month or two back.

    Plex woke up one day and decided that the TV in my living room and the server in my home-office were clearly so far apart that I’d need to give them money to stream all 20 feet over my LAN - presumably because they woke up one morning and decided that it’s more profitable not to understand VLANs (apparently not understanding VLANs is the “new Plex experience” and we should be very excited about it.) At least, that’s what their support told me - they assured me that streaming from one room to another is now a paid feature.

    Naturally, I told them to go fuck themselves and installed Jellyfin. And donated 10x what a ‘Plex Pass’ would have cost to the guy that made the Samsung-Tizen-Jellyfin-Installer thingummy. Because, well, fuck Plex.