+- TalkativeTurtles (https://talkativeturtles.club)
+-- Forum: Technology (https://talkativeturtles.club/forumdisplay.php?fid=2)
+--- Forum: AI & Machine Learning (https://talkativeturtles.club/forumdisplay.php?fid=24)
+--- Thread: Local LLMs in 2026 - what is actually usable for coding? (/showthread.php?tid=116)
Local LLMs in 2026 - what is actually usable for coding? - Zero Two - 06-28-2026
Been running local models for a while now and the gap between hosted vs local has closed a lot but it is not gone.
My current stack:
- Coding: A quantized Qwen2.5-Coder on an RTX 3090. Handles most autocomplete and small refactors well. Falls apart on multi-file context.
- General chat / brainstorming: Mistral-based model, 7B, quick enough to not feel like waiting.
- Summarisation: Phi-3 Mini. Tiny, fast, good enough for docs.
The real bottleneck is VRAM. 24GB feels like the sweet spot from 18 months ago. Now the models worth running are 70B+.
Anyone running inference on CPU-only setups? Curious whether llama.cpp on a fast CPU is viable for anything beyond prototyping.