The promise of artificial intelligence has been complicated by the reality of the services that dominate the market. Consumer-facing AI tools are often expensive, opaque, and built on vast collections of user data. They require massive server farms that consume electricity and water, and their creators rarely give users meaningful control over how their information is handled. For many people, the convenience of a cloud-based assistant is not worth those costs.
A growing alternative is the local, self-hosted AI system. Instead of sending every prompt to a remote data center, a local model runs on hardware owned by the user. This approach eliminates subscription fees, keeps private conversations in the home, and allows for experimentation with open-source software. It also creates a new set of challenges around configuration, connectivity, and practical usability.
One recent project put that idea to the test. The builder started with a compact MSI Cubi NUC that was provided by the manufacturer for review. The small machine came with a Core 9 processor and a neural processing unit, commonly called an NPU, designed to accelerate AI workloads. A related video was sponsored, but the project itself was interesting enough to be worth exploring on its own. The original plan was to run a modest language model directly on the mini PC. As the project grew, though, the setup became more ambitious.
The mini PC became the host for an AI agent named Hermes. Rather than being the main engine for the language model, Hermes acts as the controller and gateway for a much larger system. The actual heavy lifting is done by another computer with far more power: a desktop built around AMD's Ryzen AI Max platform, which includes an integrated NPU and high-performance CPU cores. That machine is able to run larger, more capable models while remaining local to the user's home network.
On the software side, the server runs Ubuntu, a popular Linux distribution. The model itself is loaded through LM Studio, a desktop application that makes it easy to download, run, and test open-source large language models. The specific model in this build is called Quinn, and it has been configured to work with Hermes as the agent orchestrating tasks. This layered architecture is increasingly common in DIY AI projects because it separates the user-facing agent from the underlying model.
The most interesting part of the project is the user interface. Instead of building a custom app or Web portal, the builder used Telegram, the messaging app, as the front-end. Telegram bots can be created with a simple API token, and they can receive commands and send responses like a regular chat. This made it possible to control the AI agent from a phone from anywhere in the world, even though the model is running on hardware inside the home. Once the bot is configured, the user can ask for summaries, request research, or direct the agent to perform tasks on the local network.
Why Telegram? It is cross-platform, free, and already installed on most phones. The bot interface is text-based and supports features such as voice messages, file sharing, and group chats. For this project, Telegram provided a reliable bridge between a remote device and a local AI server without requiring a public Web site or a custom mobile app. It also kept network exposure to a minimum, since the connection is initiated by the phone and handled by Telegram's servers.
The purpose of this particular system is practical and a little nostalgic: gathering and reading news for podcasts. The user is involved in tech podcasts and found himself spending too much time scanning dozens of Web sites and RSS feeds. With the local AI agent, he can ask for a digest of relevant stories, follow up with questions about a specific article, and receive condensed versions of long pieces. The agent can pull information from the Web, summarize it, and present it inside a Telegram chat, making the whole news-gathering process faster.
Running a local model in this way has a few clear advantages over a service like ChatGPT or Claude. The first is privacy. Because the model runs on the user's own hardware, prompts, documents, and conversations do not leave the home network. There is no third party collecting usage logs or analyzing conversations for marketing purposes. For work involving confidential information, a local AI server can be far safer than a public cloud service.
The second advantage is cost. Many of the most useful AI services require a monthly subscription, and power users can quickly hit rate limits on free tiers. A local model has a high upfront cost for the hardware, but the ongoing cost is just electricity. For someone who already owns a capable desktop PC, the marginal cost of adding an NPU-based mini PC or a powerful Ryzen AI system may be lower than expected.
The third advantage is control. Open-source models can be replaced, updated, and fine-tuned by the user. The agent logic, server software, and model weights are all in the user's hands. This is a very different relationship from the black-box systems offered by large companies. It allows the builder to change the system when needed rather than waiting for a product update.
The project also highlights the rapid evolution of AI-ready hardware. NPUs are now being integrated into everything from high-end laptops to small-form-factor PCs. These chips are designed to accelerate the math behind neural networks and can make local AI models run faster and use less power. The combination of an NPU on a mini PC and a more powerful desktop server shows that local AI can be spread across multiple devices, with each component handling the tasks it is best suited for.
Still, the setup process was not easy. The builder reached out to an expert in Linux and hardware for advice on the initial configuration, and the project quickly grew more complicated than a simple software install. Coordinating the agent, the server, the model, and the Telegram bot required careful attention to network settings, permissions, and API credentials. Even after everything was connected, the user spent time adjusting the model's behavior and testing when it would be more efficient to use the local agent instead of a cloud service.
That last point is worth restating: local AI is not always the best tool for the job. Small models running on modest hardware cannot match the depth and reasoning ability of the largest frontier models. They may struggle with complex writing tasks, multi-step research, or nuanced creative work. In this project, the goal was narrowly defined, which made a local system viable. For broader use, a user might need a more powerful server or a hybrid approach that keeps some workloads local and offloads others to a remote service.
The phone control aspect is an important step forward. Many people assume that a self-hosted AI system must be used while sitting at a desk, but modern messaging platforms make that limitation obsolete. By giving the agent a text-based interface, the user can access the system from a phone, a tablet, or even a laptop after a quick setup. It is a reminder that the user interface matters just as much as the model behind it.
Key facts from the project
- A mini PC with a Core 9 processor and a neural processing unit serves as the host for an AI agent named Hermes.
- The agent controls a separate server built around the Framework Desktop and Ryzen AI Max hardware, which runs the larger Quinn model via LM Studio on Ubuntu.
- Users control the system through Telegram from a phone, making the local AI accessible from anywhere.
- The project's main use is to gather and read news for podcasts, though the creator is still experimenting with new tasks.
This type of build is a response to a moment when AI tools have become both ubiquitous and deeply controversial. It demonstrates that individuals can still make their own choices about the technology they use. There is a real appeal to owning the entire stack, from the chip in the computer to the model that generates the words. With the current generation of hardware, that appeal has become practical enough to build and use every day.
Source:PCWorld News