Internal2026ServerDesktop
Local LLM inference & routing
Private AI for teams: local models sized to the hardware, with a gateway that routes between local and cloud models.
What I built
- gpt-oss and Qwen-class models on Apple Silicon; small models on a phone
- Hybrid routing through LiteLLM and a local AI gateway
- Hardware sizing for team inference (GPU workstation vs unified-memory Mac)