SocialistVibes01@lemmy.ml to Linux@lemmy.mlEnglish · 3 days agoWhich specs are as low as reasonable possible for local LLM models? Do you recommend some distro in particular?message-squaremessage-square16linkfedilinkarrow-up132arrow-down113file-text
arrow-up119arrow-down1message-squareWhich specs are as low as reasonable possible for local LLM models? Do you recommend some distro in particular?SocialistVibes01@lemmy.ml to Linux@lemmy.mlEnglish · 3 days agomessage-square16linkfedilinkfile-text
minus-squarewhatiswrongwithyou@lemmy.mllinkfedilinkarrow-up1·14 hours agoSpecs for what? The hardware you run it on will have a range of specs. The model you try to use will have a bunch of measurements that can be called specs. Once you pick some hardware, the measurements of the model you run on it can be adjusted for “accuracy” or speed. Of course, the range of hardware that’s available to you informs that calculus. The alternative to answering this post is to get a Mac mini or whatever they’re selling as a desktop llm device now and not mess around.
Specs for what?
The hardware you run it on will have a range of specs.
The model you try to use will have a bunch of measurements that can be called specs.
Once you pick some hardware, the measurements of the model you run on it can be adjusted for “accuracy” or speed.
Of course, the range of hardware that’s available to you informs that calculus.
The alternative to answering this post is to get a Mac mini or whatever they’re selling as a desktop llm device now and not mess around.