A local AI chat can work well at first, then become harder to run as the conversation grows. The model needs room for the conversation it processes as well as its own weights.
OGAM (Off Grid AI Mobile) lets you adjust Context Length for supported GGUF text models. A shorter context can reduce the memory used by the conversation cache. You can keep the useful parts of a long discussion in a short, checked handoff and continue locally on your phone.
Get OGAM for Android or iPhone
What would you like to do with Off Grid AI?
Have a feature or use case you would like us to support? Tell us what you want to do and which device you use.
Write to support@offgridmobileai.co, join our Slack community, or talk to us on Reddit.
What does context length change?
Context length limits how much text the model can consider in a request. It is different from the amount of chat history saved in the app, and it is different from the maximum length of the next answer.
More context can help with long material, but it increases the memory cost. If your task is to rewrite a paragraph, you may not need room for a long report and dozens of earlier turns.
The controls below apply to the supported GGUF text-generation route. Other runtimes, including LiteRT models, do not necessarily show the same settings.
Give the chat a smaller working context
- Open Settings, then Model Settings in OGAM.
- Expand Text Generation.
- Find Context Length and choose a lower value within the supported range.
- Reload the model so the new context allocation takes effect.
- Try a short prompt and check that the model responds before returning to longer material.
The setting’s description notes that larger context uses more RAM. A higher advertised model limit does not mean that your phone has enough spare memory to use it comfortably.
Local text chat and these core model controls do not require Pro. Download a suitable model before using it without internet.
Keep the important parts of a long conversation
Before moving to a smaller context or starting fresh, make a short working note:
- What you are trying to finish.
- Facts and decisions that must carry forward.
- The current draft or code excerpt.
- The next question to answer.
You can ask the model to draft the handoff, but check it yourself. A summary cannot preserve a detail that the model did not receive or misunderstood.
For example:
We are writing a two-paragraph project update. The migration is complete. Testing is still in progress. Do not invent a launch date. Rewrite the draft below for a nontechnical reader.
That gives the next request useful context without carrying every abandoned draft.
What if it still cannot run?
Choose a smaller model. Context reduction can help with the conversation cache, but it cannot make oversized model weights disappear. Close other demanding apps and avoid raising several memory settings at once.
If the answer loses an older detail, include that detail in the current prompt. The goal is a stable, useful working conversation, not the largest number available in the settings.
These controls are part of the OGAM 0.0.111 release.
Find a context setting that still fits the job
Choose one task before changing the setting. A short email rewrite needs the email and a few instructions. Comparing several long documents needs much more input. Use a small task first so you can tell whether the phone is stable before testing the longer one.
Write down the starting Context Length. Reduce it by one available step, reload and run the same prompt. If that works, try the amount of source material you actually need. Do not keep lowering context until the app is stable but the relevant evidence no longer fits.
A useful test prompt is:
Rewrite the note below in five bullets. Preserve all names, dates and open questions. If the note does not state a deadline, write “deadline not stated.”
Check the output against the note. If the model drops a fact, confirm that you supplied it in the current request. A missing old detail can be a context problem; an invented detail in a short, fully supplied note may be a model-quality problem instead.
Keep a recovery copy before starting fresh
Save the current draft and confirmed decisions before you create a new conversation. A generated handoff is only a draft until you check it. Keep exact wording for facts where a paraphrase could change the meaning.
If a modest context and short prompt still fail, the model itself may be too large for the available memory. Return to a smaller compatible model rather than treating context reduction as an unlimited workaround. Once a small model works, add longer source material gradually so you can identify the point where the task stops being practical.
Keep a useful chat within your phone’s limits
Get OGAM, try a modest context length and carry forward a short checked brief. Keep the conversation focused on the work you want to finish.