How to Build a Private AI Agent Stack on a Mini-PC to Eliminate Cloud Inference Costs
A detailed build log for running the Hermes Agent locally on a Minisforum MS-S1 MAX, cutting cloud costs and keeping sensitive data on-site.
Practical Summary
This guide demonstrates how to set up a powerful mini-PC as a local AI agent server. It replaces recurring cloud API costs with a one-time hardware investment, improves data privacy, and ensures continuous operation for business automation tasks.
Why It Matters
For businesses relying on AI agents, this workflow directly reduces operational costs, eliminates per-token fees, and mitigates data privacy risks associated with cloud APIs. It offers a viable path to self-hosting large language models for high-throughput or sensitive workloads.
Core Business Advantage: From Recurring Cost to Owned Asset
The fundamental shift is moving agent workloads from a daily cloud inference bill to a one-time hardware purchase. By running the full stack locally, you eliminate ongoing per-token costs for the majority of tasks, while maintaining complete control over customer data and API keys which never leave your local network.
Hardware and Software Stack
The setup uses a Minisforum MS-S1 MAX mini-PC with 128GB of unified memory. Its integrated GPU can allocate up to 96GB as VRAM, sufficient to run large models like a 120B parameter model locally. The software stack includes Hermes Agent configured to use a local OpenAI-compatible endpoint powered by llama.cpp.
Implementation Steps and Performance
The implementation involves specific technical steps including BIOS configuration, setting kernel parameters, deploying llama.cpp in a container, and managing model swapping. The article reports concrete performance: achieving approximately 50+ tokens per second with a 120B model. For complex or time-sensitive tasks, the system is configured to use cloud routing as a smart fallback.
Operational Reliability and Remote Access
To ensure 24/7 operation, the Hermes Agent is set to restart automatically and maintain reliability as infrastructure. Remote management is handled securely via Tailscale, allowing access from anywhere without exposing network ports. This creates a private, always-on business automation system.
