Contact Us
Contact Us
Mobile Apps

Adding AI to Your Mobile App: On-Device vs Cloud Models and What Users Expect

How to decide, feature by feature, whether AI in your mobile app should run on the phone, in the cloud or both, weighing latency, privacy, cost, offline use and device coverage.

JRC Software Team
6 min read
Adding AI to Your Mobile App: On-Device vs Cloud Models and What Users Expect

"Can it summarise the job notes when the tech is out of range?" Put that question from a field service manager next to a roadmap line that just says "add AI", and the conversation changes. For an AI mobile app, the question that shapes cost, privacy and user experience isn't which model is smartest. It's where the model runs.

A cloud model gives you the most capable models, but every call costs money, needs a connection and sends user data off the device. A model on the phone keeps data local, works offline and has no per-call server bill, but it's limited by the hardware in your users' pockets, and not every phone has it.

Most real apps end up using both. Here's how to decide feature by feature, and what users expect either way.

The OS now ships the model

For years, generative AI in a mobile app meant calling a cloud API. That changed when Apple and Google started shipping foundation models as part of the operating system.

On Apple platforms, the Foundation Models framework gives apps access to the on-device and Private Cloud Compute models designed for Apple Intelligence. Apple says the on-device models are good at summarisation, entity extraction, text and image understanding, refinement and creative content, and points developers to Private Cloud Compute or another server model when they need more reasoning or a larger context. It supports guided generation (the @Generable macro, so the model returns instances of your own Swift types) and tool calling into your app. Users need an Apple Intelligence-capable device, and you check SystemLanguageModel.default.availability at runtime before showing the feature.

On Android, Gemini Nano runs in the AICore system service. ML Kit's GenAI APIs build on it with task APIs for summarisation, proofreading, rewriting and image description, speech recognition and translation, semantic search, and direct prompting. Google describes it as delivering generative AI "without needing a network connection or sending data to the cloud".

For classic machine learning such as classification, detection or recommendations, Apple's Core ML has run custom models on device for years, across the CPU, GPU and Neural Engine.

AI mobile app trade-offs: on-device vs cloud

FactorOn-device modelCloud model
LatencyNo network round trip; speed depends on the phoneRound trip plus server time; consistent across devices
PrivacyData can stay on the phoneData leaves the device; needs disclosure and contracts
CostNo per-call server costBilled per request or token; grows with usage
OfflineWorks without a connectionNeeds a connection
CapabilitySmaller models, token and hardware limitsLargest models, longer context, stronger reasoning
CoverageSupported, newer devices onlyAny device with a connection

Latency

Google's Gemini Nano page notes that on-device inference removes network latency but that speed depends on device hardware. Short tasks, like rewriting a sentence or tagging a photo, feel instant on supported phones. A long generation on an older supported phone can be slower than a fast cloud call, so test at the bottom of your supported range, not on the product manager's new flagship.

Privacy

Keeping data on the device is the strongest privacy story you can tell. Google says AICore isolates each request and doesn't store a record of inputs or outputs after processing. Apple positions Core ML's on-device inference the same way: no network connection needed, and data stays on the device.

Cloud processing isn't wrong, but it brings obligations: where the data is processed, what the provider retains, and whether it's used for training. If an AI feature makes or supports decisions about people, such as scoring, triage or eligibility, read our article on automated decision-making under the Privacy Act.

Cost

The ML Kit GenAI overview says there's no additional server cost for each API call. Cloud models bill per request or per token, so a popular feature turns into a growing line item. Apple's Apple Intelligence developer page adds a wrinkle: apps in the App Store Small Business Program with fewer than 2 million total first-time downloads can use the next generation of Apple Foundation Models on Private Cloud Compute at no cloud API cost. Read the eligibility terms before you build a business case on it.

Offline use

Field service, travel, mining sites, regional Australia: anywhere with patchy coverage, offline matters. Google says ML Kit GenAI functionality stays the same without a reliable internet connection. A cloud-only feature just stops, and the app has to say so clearly rather than spin forever.

The catch is coverage

On-device models aren't everywhere. The ML Kit GenAI overview says model availability and version can vary with the device's RAM, OS version and AICore update status, and language support varies by device too. On Apple platforms, Foundation Models needs an Apple Intelligence-capable device with Apple Intelligence turned on.

So most production apps need a hybrid design. Google's Gemini Nano page describes hybrid inference through the Firebase AI Logic hybrid API, which routes requests to cloud Gemini models when Gemini Nano is unavailable, and recommends protecting cloud calls with Firebase App Check and keeping sensitive system prompts on the server. Plan the fallback's cost and privacy disclosure from day one, not after the first support ticket from an older Android phone.

What users expect from generative AI in a mobile app

Users judge the experience, not the architecture. In our design work, the same expectations keep coming up:

  1. Speed that fits the task. Stream the output or show progress for anything that takes more than a moment.
  2. Clear labelling. Mark generated content and make it easy to edit or throw away.
  3. Plain talk about data. Say whether processing happens on the phone or in the cloud, in one sentence.
  4. Graceful failure. If the feature is offline or unsupported on this device, explain why and offer an alternative.
  5. Control. Let people switch AI features off, and never act on their behalf without confirmation.

Deciding feature by feature

Decide per feature, not per app:

  • On-device for short, frequent, privacy-sensitive tasks: smart replies, rewriting, summarising a user's own notes, on-device search, photo tagging, offline help.
  • Cloud for tasks that need broad knowledge, long documents, complex reasoning or identical results on every device.
  • Hybrid when the feature must work for everyone but you want on-device privacy and cost wherever the hardware allows.

Flutter and React Native can both reach these native APIs through plugins or native modules; our Flutter vs React Native guide covers that choice.

How JRC Software builds AI into mobile apps

Our mobile app development team starts with a short feature workshop. We list your candidate AI features and score each on value, data sensitivity, how often it's used and what share of your users' devices could run it on device (from your analytics, not guesswork). Each one gets an on-device, cloud or hybrid recommendation.

Then we prototype the highest-value feature on real phones, including the oldest ones you support, and measure response time, output quality and cost per active user. Before release we check prompts, data flows and SDKs against our mobile app security checklist and draft the in-app and privacy policy wording with you. Shipping AI to every screen at once is how teams lose track of what's actually working. Prove one, measure it, then expand.

If you're weighing up AI features, book a discovery call and talk to a mobile architect about where each model should run.

Frequently asked questions

What is on-device AI in a mobile app?

On-device AI means the model runs on the user's phone instead of a remote server. Apple provides it through the Foundation Models framework and Core ML, and Google through Gemini Nano in the AICore system service, exposed via ML Kit's GenAI APIs. You get privacy, offline use and no per-call server cost, in exchange for smaller models and limited device support.

Is on-device AI good enough for production features?

For focused tasks, yes. Apple and Google position their on-device models for summarisation, rewriting, extraction, classification and similar jobs. They're weaker on long documents, broad world knowledge and complex reasoning, where Apple itself points developers to Private Cloud Compute or server models. Test your exact prompts on the oldest devices you support before committing.

What happens on phones that cannot run the on-device model?

You need a fallback. Route the request to a cloud model, offer a simpler non-AI version of the feature, or hide it with a clear explanation. Google documents a hybrid approach that falls back to cloud Gemini models when Gemini Nano isn't available. Each option changes your cost and privacy disclosures, so choose deliberately.

How do we keep cloud AI costs under control?

Send only what the model needs, cache results where it's safe, set per-user rate limits and track cost per active user from launch. Move short, frequent tasks on device where the hardware allows, since on-device calls carry no per-request server cost. Keep system prompts and API keys on your server, never in the app binary.

Mobile Apps

Talk to a mobile architect

Tell us where you're stuck and we'll come back with practical next steps.

Get in touch

Sources