LLMs on mobile devices · Makley Trichez

Abstract representation of a language model running on a mobile device.

Academic paper written during my undergraduate degree, investigating whether — and under what conditions — large language models can run locally on smartphones without relying on cloud inference.

Results

1: research paper Conducted and defended during a Computer Science degree
On-device: research focus Local inference as an alternative to cost, latency and data exposure

The problem

Language-model adoption was built on top of cloud inference, which brings three limitations that are hard to ignore: cost per request, network-dependent latency, and sending potentially sensitive user data to third-party servers. Running the model on the device itself would solve all three at once — the question is whether an average phone's hardware can actually handle it.

The solution

The research assessed the feasibility of that local execution, taking into account the device's real constraints: available memory, processing power, battery consumption and the impact of quantization techniques on answer quality.

Key challenges

Defining what "feasible" means in measurable terms, rather than as a subjective impression of performance.
Comparing models and quantization levels under consistent criteria, isolating the effect of each variable.
Working in a fast-moving field, where part of the literature aged during the writing of the paper itself.

Technologies used

Artificial Intelligence
LLMs
Model Quantization
Applied Research

Why this topic

I picked the subject at a time when nearly every discussion about language models revolved around scale and cloud infrastructure. The opposite question — how much can be done with what the user already has in their pocket — seemed more interesting, and in hindsight, it aged well.