My Local AI Setup, Six Months Later: 48 GB, Qwen3.8, and a Different Bottleneck

Ammar Najjar · Posted on September 22, 2026 · 9 min readaiproductivity

AI-generated image

A few months ago, I wrote about running a useful local AI development setup on a MacBook Pro with 16 GB of unified memory. At the time, memory shaped almost every decision. I experimented with smaller models, limited context carefully, and looked for a combination that was capable enough for coding while still leaving room for my IDE, containers, browser, and the rest of my development environment.

Since then, I have upgraded to a Mac with 48 GB of unified memory. I expected that mainly to give me access to larger models. It did, but after using the new setup for a while, I noticed a more interesting change: I was optimizing for a different problem.

With 16 GB, the recurring question was which useful model I could fit on the machine. With 48 GB, I rarely think about that anymore. I care much more about whether the complete setup can remain useful throughout a long coding session.

The timing of the upgrade also coincided with several changes in the local AI ecosystem. Qwen3.8-27B was released on August 14, 2026. I moved to it soon afterwards. Then, in September, LM Studio added support for Splash, an inference engine optimized for selected models on recent Apple Silicon.

As of September 2026, those changes have left me with a fairly simple local stack: OpenCode, LM Studio, Qwen3.8-27B, and Splash. More importantly, they have changed where I notice the limitations.

From Smaller Models to One Default Model

One lesson from my earlier experiments still holds: model size alone is a poor measure of how useful a local AI setup will be.

The model has to coexist with its context, the inference runtime, the coding agent, and everything else running on the machine. On 16 GB, that made smaller models attractive even when a larger model looked considerably better on paper. They left enough memory for the complete workflow to remain usable.

Moving to 48 GB shifted that boundary enough for a 27B model to become practical for everyday use.

When Qwen3.8-27B arrived in August 2026, I moved to it fairly quickly. The important change for me was not simply having a larger model. I gradually stopped thinking about which model I should use for a particular task.

I now use the same model for repository exploration, architecture discussions, tool calls, implementation work, and code changes. Smaller models are still available, but I no longer maintain the same mental distinction between a fast model for simple work and a stronger model for more demanding tasks.

That simplification has turned out to matter more than I expected.

My basic setup became:

OpenCode
    ↓
LM Studio
    ↓
Qwen3.8-27B

Qwen3.8-27B is a dense 27B vision-language model with reasoning and tool-use capabilities. Its native context window is 262,144 tokens, which gives it considerably more room than I currently use.

LM Studio remains my runtime for much the same reasons as before. I can see what is loaded, choose the available context, expose the model through an OpenAI-compatible API, and keep the inference layer separate from the coding agent. OpenCode handles the development workflow while LM Studio handles the model, which makes experimenting with either side relatively straightforward.

This setup was already working well for me. Then Splash changed the performance side of it.

Splash Made the Same Model More Practical

LM Studio announced Splash support on September 18, 2026, followed by LM Studio 0.4.25 on September 19.

Splash is an open-source inference engine developed by Inco AI and optimized for a small set of models on Apple Silicon. Its approach is deliberately model-specific. The GPU kernels and memory plan are tailored to the supported model rather than designed as a general backend for many different architectures.

For Qwen3.8-27B, the Splash package also includes a dedicated DFlash 2 draft model for speculative decoding. The package uses a 4-bit version of Qwen3.8-27B and is around 17.4 GB to download.

The hardware requirements happen to fit my new machine well. Splash requires at least 36 GB of unified memory, with 48 GB or more recommended.

My current model is therefore:

incoai/Qwen3.8-27B-Splash

running through the Splash backend in LM Studio.

Inco’s published measurements provide some context for the performance difference. On an M5 Pro with a 16-core GPU and 48 GB of unified memory, it measured around 74 tokens per second for short-prompt decoding. These are vendor benchmarks rather than measurements I have reproduced myself, and they are less important to me than what changed in actual use.

The model feels fast enough that I can leave it as my default throughout normal development work.

That matters particularly with coding agents. A single answer taking a little longer may be easy to ignore. An agent repeatedly returns to the model after reading files, searching the repository, executing commands, editing code, and checking results. Latency accumulates across the workflow.

Once that became less noticeable, I started paying more attention to another limitation.

32K Started Feeling Small

In my original setup, 32K context felt like a reasonable target. On a 16 GB machine, context was another resource I had to balance against the model and everything else running locally.

With the new setup, longer OpenCode sessions made me reconsider that choice.

An agent session accumulates much more than the original prompt. OpenCode searches the repository, reads files, receives command output, calls tools, makes changes, and carries previous decisions forward. A request that starts small can build a substantial working history as the agent develops an understanding of the codebase and the task.

Eventually, that history approaches the context limit and has to be compacted.

Compaction is a necessary part of long-running agent sessions, but I started noticing it too often at 32K. The agent could still continue, but parts of the working history had to be summarized while the task was still developing. Over a longer session, that cycle could repeat several times.

This was the point where context stopped feeling like a model specification and started feeling more like workspace for the agent.

Why I Currently Use 64K

My current default is 65,536 tokens:

Model:    Qwen3.8-27B Splash
Runtime:  LM Studio 0.4.25
Context:  65,536
Thinking: Enabled when useful
Client:   OpenCode

The choice of 64K is deliberately practical. It is not the result of a benchmark, and I do not consider it an optimal number in general.

It simply gives my typical OpenCode sessions more room before compaction becomes noticeable.

Qwen3.8-27B supports a native context window of 262,144 tokens, so I could configure considerably more. I currently have little reason to make that my default. Most of my sessions do not need hundreds of thousands of tokens of history, and processing increasingly large active contexts has a cost of its own.

For larger tasks, I can raise the limit. For normal development work, 64K has so far given me enough headroom without configuring the system around a capacity I rarely need.

The important change is how I choose that number. Six months ago, context size was largely constrained by what the machine could afford. Today, I start with the kind of agent session I want to run and give it enough working room.

The Constraint Keeps Moving

Looking back at both setups, I find this progression more interesting than the individual models.

On the 16 GB machine, memory was the dominant constraint. It pushed me toward smaller models and relatively conservative context sizes.

Moving to 48 GB made Qwen3.8-27B practical as a default model. That reduced the need to switch models depending on the task. Splash then improved inference performance enough that waiting for a 27B local model became less prominent during agent work.

Once those problems became smaller, context became easier to notice.

That progression also changed what I mean when I think about a “good local model.” The model is only one part of the experience. For development work, I care about how quickly the agent can iterate, how much understanding it can preserve as a task develops, and whether the whole setup can run alongside the tools I already use throughout the day.

The useful unit to optimize is increasingly the complete agent workflow.

My Setup as of September 2026

This is where the setup currently stands:

AI-generated image

Recording the dates feels more useful this time because local AI is changing quickly enough that a setup described today may look very different a few months from now. This is a snapshot of what works for me in September 2026 rather than a configuration I expect to remain unchanged.

Six months ago, most of my experimentation was about getting enough model capability into a constrained machine. I now have enough memory to make a capable 27B model my default, and Splash has made that model more comfortable to use repeatedly through an agent.

The question I am left with is different from the one I started with. I am spending less time asking how much model I can fit into the machine and more time observing how much useful work the complete system can sustain before one of its constraints becomes visible again.

For now, 64K context is part of that balance. I expect the balance to move again.

What is the constraint shaping your setup right now? If you run models locally, I would be curious where you feel the limits: model size, inference speed, context, or something else entirely. Let me know in the comments.