Achieve a Free 20-30% Speed Boost for Local LLMs on Old Android Phones Using MTP and llama.cpp
A practical guide to running the open-source Gemma 4 model locally on a 6-year-old Android phone, using Multi Token Prediction (MTP) to significantly boost inference speed without any cost.
Practical Summary
This tutorial shows how to set up and run the Unsloth Gemma 4 E2B model with Multi Token Prediction (MTP) on an old Android phone using llama.cpp and Termux. The setup demonstrates a tangible 20-30% speed improvement (from 7-9 to 9-12 tokens/sec) over the standard model, providing a free, offline AI capability that eliminates cloud API costs for certain tasks.
Why It Matters
For AI tool cost optimization, this workflow demonstrates a high-impact method: repurposing old hardware (like an old phone) with free, open-source software to run capable language models locally. This eliminates recurring subscription fees and per-token API costs for inference tasks, directly reducing operational expenses. The documented speed boost from MTP makes this a practical, performance-enhanced alternative to paid services for development, experimentation, or lightweight applications.
Understanding the Cost-Saving Setup
The core idea is to run a large language model (LLM) entirely on a local Android device using open-source tools, avoiding any cloud service fees. The key to the performance gain is using a model variant that includes Multi Token Prediction (MTP), which allows the model to generate multiple tokens per step, increasing throughput.
The benchmarked device is a Samsung Galaxy Note 20 Ultra, a 6-year-old phone with an ARM CPU and at least 8GB of RAM. This proves the method is viable on consumer hardware that is no longer a daily driver, turning it into a dedicated, cost-free AI inference device.
Prerequisites and Model Download
First, ensure your Android phone has Termux installed (available on F-Droid) and at least 8GB of RAM. The model used is the Unsloth Gemma 4 E2B QAT in the UD-Q4_K_XL quantization format, which is optimized for mobile CPUs. You will need two files: the main model and its MTP draft assistant model.
Download the model files from the Unsloth project on HuggingFace. The post references a beginner-friendly guide linked in the comments for downloading and placing these GGUF files into a directory on your phone's shared storage (e.g., ~/storage/shared/llm/).
Running the Model with the Speed-Boosting Command
Open Termux and navigate to the directory containing your llama.cpp binary. The following command starts a local API server for the model with MTP enabled. Copy it exactly, adjusting the file paths if your downloaded GGUFs are in a different location:
