Quantization

All discussions tagged with this topic

← Browse all tags
Found 2 discussions
How ChatGPT Serves 700M Users: An Inside Look at Large-Scale AI Inference
August 16, 2025

Discover the key engineering strategies and massive infrastructure that enable services like ChatGPT to handle hundreds of millions of users, from the power of batched inference to advanced model optimization techniques.

Large-Scale Ai Llm Inference Inference Optimization Batched Inference Model Parallelism Quantization Speculative Decoding Mixture-Of-Experts (Moe) Gpu Infrastructure Systems Engineering
Offline AI in the Wild: Running LLMs Locally While Camping
June 18, 2025

Explore a discussion on taking LLMs camping off-grid, covering recommended local models like Gemma and Qwen, tools like Ollama and LM Studio, power solutions, and the critical debate on AI reliability for survival.

Offline Ai Llms Local Llms Ollama Lm Studio Model Quantization Survival Power Consumption Ai Hallucinations Off-Grid Computing