AMD is in good company at MWC Barcelona (formerly Mobile World Congress) this year, even if that company is mostly focusing on the enterprise scene rather than average consumers like me. Just like its Ryzen PRO 8000/8040 Series desktop processors from 2024, and the Ryzen AI PRO 300 mobile chips that followed later that year, it's announcing another AI PRO range for 2026.

Unsurprisingly, it's moving into "the AI PC era for business" with a bump up to AMD Ryzen AI PRO 400 Series mobile and desktop processors, including a new Ryzen AI 9 HX PRO 475 mobile workstation chip with an XDNA 2 NPU handling AI tasks on Windows with up to 60 TOPS of local processing power.

That's far above the 40 TOPS requirement for a Copilot+ PC certification. On the desktop side, businesses will be able to get their hands on a new range of Zen 5 processors that hit a slightly lower 50 TOPS with their own NPU chiplet, which is still much higher than its previous-gen, consumer-grade Ryzen 8000G desktop APUs, which ran at around 16 TOPS.

If this all sounds familiar to those waiting for AMD's Ryzen AI 400 Series, codenamed 'Gorgon Point', that's for a good reason. These are essentially the same chips as the consumer side, but with an added layer of enterprise-grade security, remote management add-ons, and stability guarantees. So, regular folk will still get the 60 TOPS XDNA 2-based NPU on consumer-grade laptops this year with current-gen, RDNA 3.5-based Radeon 800M Series graphics chiplets.

What exactly is AMD launching?

Again, these processors are built for AMD's commercial audience, but it's always interesting to see what it's doing with its Zen 5 architecture — now delivering 6 mobile chips and 3 desktop chips (technically 6 if you count the energy-efficient 'GE' variants). It's the mobile workstation (laptop) side that earns the 60 TOPS NPU for local AI work, paired with LPDDR5X memory clocked up to 8355 MT/s.

The Ryzen AI PRO desktop chips stick with AMD's '5 PRO' and '7 PRO' rankings, leaving the '9 PRO' and '9 HX PRO' to the mobile side. That, and the trio of APUs comes with 65W TDP as standard, while the lower-power 'GE' models stick to 35W. Each processor uses the same 50 TOPS NPU, which is a big jump from previous-generation desktop chips like the Ryzen 7 8700G I tested in 2024.

🗨️ Anyone out there using these chips?

On the bright side, it's not like I'll never see these chips in real life, as enterprise-grade PCs from the likes of Lenovo, HP, Dell, ASUS, and Acer will start appearing in Q2 2026. Realistically, they'll only get a passing glance from me at trade shows, but again, it's interesting to see what AMD does on both the consumer and commercial sides.

That, and local AI is more interesting to me than anything in the cloud — shout out to Blender, Audacity, and Davinci Resolve for actually making practical use out of NPUs. The good news is that Ryzen AI 400 Series consumer laptops (and mini PCs) are set to launch earlier in Q1 2026, and you can believe I'll be putting on my polite email-writing hat when those samples start appearing.

I'm wondering if any of our readers use processors like these in their professional lives. And if not, would you like to?

Join us on Reddit at r/WindowsCentral to share your insights and discuss our latest news, reviews, and more.

If you're looking at using Ollama on your PC to run local LLMs (Large Language Models), with Windows PCs at least, you have two options. The first is to just use the Windows app and run it natively. The second is to run the Linux version through WSL.

The first is definitely easier. For one, you don't need to have WSL installed, and the actual process of getting up and running is simpler. You just downloaded the Windows installer, run it, and you're up and running.

Installing Ollama on WSL requires jumping through a few more hoops. But, using Ubuntu, it's good, and performance is excellent. At least in my experience using an NVIDIA GPU. But unless you're, say, a developer, using WSL for your workflows, there isn't much reason to go this way over the regular Windows version.

Getting Ollama set up on WSL

I'll start by saying this isn't an exhaustive setup guide, more a case of pointing in the right direction. To use Ollama on WSL — and specifically, I'm referring to Ubuntu, because that seems to be both the easiest and best documented — there are a couple of prerequisites. This post is also specific to NVIDIA GPUs.

The first is an up-to-date NVIDIA driver for Windows. The second is a WSL-specific CUDA toolkit. Assuming you have both, the magic will just happen. Microsoft and NVIDIA's documentation is the best place to start to guide you through the whole process.

It doesn't take too long, though, but that's also dependent on your internet connection to get all the bits you need downloaded.

Once you have this handled, you can simply run the installation script to get Ollama up and running. Note, I haven't explored running Ollama in a container on WSL; my experience is strictly linked to just installing it directly to Ubuntu.

During the installation process, it should automatically detect the NVIDIA GPU if you have everything set up correctly. You should see a message saying "NVIDIA GPU installed" as the script is running.

From there on out, it's the same as using Ollama on Windows, minus the GUI application. Download your first model and you're away. There is a mild quirk, though, if you switch back to Windows.

If you're running WSL in a different tab, Ollama in Windows (in the terminal at least) will only recognize the models you have installed on WSL as active. If you check the ollama --list command, you'll not see any you have installed on Windows. If you try to run one you know you have, it'll go out and start downloading it again.

In this case, you need to ensure WSL is properly shut down before using Ollama on Windows. You can do this by entering wsl --shutdown into a PowerShell terminal.

Almost identical performance in WSL to using Ollama on Windows

I'll get to some numbers in a moment, but there is one point to address. It potentially doesn't matter in the grand scheme of things, but you need to at least be mindful that just running WSL will use up some of your overall system resources.

You can set the amount of RAM and CPU threads you want WSL to use easily in the WSL settings app. If you're primarily going to be using the GPU, it's less important. But, if you intend to use any models that don't fit entirely into your VRAM, you'll need to ensure you have sufficient resources allocated to WSL to pick up the slack.

Remember that if the model doesn't fit into the VRAM, Ollama will rope in the regular system memory, and with it, the CPU. Just be sure to allocate accordingly.

I'll admit these tests are very simple, and I'm not verifying any accuracy of the output. It's only to illustrate the comparable performance. I looked at four models that all run comfortably on an RTX 5090: deepseek-r1:14b, gpt-oss:20b, magistral:24b, and gemma3:27b.

In each case, I asked the models two questions.

• Write a story over 5 chapters with a theme and characters of your choice. (Story)
• I want you to create a clone of Pong entirely within Python. Use no external assets, and any graphical elements must be created within the code. Ensure any dependencies are imported that are required. (Code)

And the results:

There are some minor fluctuations, but performance is as near as makes no difference, identical.

The only difference between the impact each made on the system resources, also, is the additional RAM being used when WSL is active. But since none of these models exceeded the dedicated VRAM, it had no impact on the model's performance.

For developers working in WSL, Ollama is just as powerful

An Average Joe such as myself (even one who loves WSL) doesn't really need to bother with using Ollama this way. My main use of Ollama at the moment is education, both as a learning tool and teaching myself about how it works.

For that, using it on Windows 11 is absolutely fine, either in the terminal or hooked into the Page Assist browser extension, which is something else I've been playing with recently.

But WSL is a bridge between Windows and Linux for developers. Those with necessary WSL workflows can use Ollama this way without any loss in performance.

Even today, it still feels a little bit like magic that you can run Linux on Windows like this and have full use of your NVIDIA GPU. It certainly all works that way, anyway.

Mini PCs have come a long way in the last few years. That's partly down to some serious advances in tech from Intel and AMD. The latest generations of mobile chips from both brands are now serious alternatives to using what we put inside desktop rigs. All while using a fraction of the power.

Compromises are minimal, and that's exactly the case with the new Geekom A9 Max, a mini PC that's truly captured a place on my desk in the last couple of weeks of use.

With an incredible AMD APU (processor) inside, stacks of storage, plenty of RAM, and better connectivity options than my gaming PC, this tiny little box punches far above its diminutive form.

Right now the Geekom A9 Max is available at a discounted price, too. On Amazon in the U.S. it's just $999, while in the UK you can pick one up for £999 if you have an Amazon Prime subscription.

As an additional bonus for Windows Central readers, if you buy the A9 Max from the Geekom store and enter the code WINDSA9MAX you can take an additional 6% off the price.

This review was made possible thanks to a review sample provided by Geekom. The company had no input nor saw the contents of this review prior to publication.

The Geekom A9 Max is a tiny box full of tricks

Right from the unboxing experience, there's a whiff of the Mac Mini about the Geekom A9 Max. The simple white packaging with silver accents reveals inside a similarly sleek silver box. Give or take a few millimeters and grams here and there, it's about the size of the latest Mac Mini, too.

That means even the smallest, most cramped setups can find a home for the A9 Max. Made even more convenient with the included VESA mount, so you can hide it from view and attach it to the back of your monitor. I'm not going to do that, though. I like looking at this little thing.

It's constantly amazing to me that something so small can offer so much. And I'm someone who used a Mac Mini for the first time about 14 years ago. I'm back with the same feeling of wonderment with the Geekom A9 Max.

Where it pulls ahead of, well, Apple, especially, is that it's not all form over function. The front of the A9 Max is packed with FOUR USB 3.2 ports, including one that's always powered. Then you go round the back, and business really picks up. Dual HDMI 2.1, dual 2.5G Ethernet, dual USB-C 4.0, another USB 3.2, and a basic USB 2.0 port. Oh, and a full-sized SD card slot on the side.

I'm not joking when I say the connectivity here is better than on my gaming PC. And it's smaller than my power supply.

How do they cram so much hardware inside?

The Geekom A9 Max is the first time I've used one of AMD's more recent Ryzen AI APUs. There's no going back. This isn't a Strix Halo mini PC — and it doesn't have the price of one either, thank goodness — instead using Strix Point. Or, to give its full name, the AMD Ryzen AI 9 HX370.

That means 12 cores (made up of 4 x Zen 5 cores and 8 x Zen 5c cores) with 24 threads and Radeon 890M graphics, with a 50 TOPS NPU. That NPU also means this is a Copilot+ PC, albeit, for whatever reason, with no sign of Recall.

This review unit comes with 32GB DDR5 5600MHz RAM, but that can be expanded up to 128GB because it's regular old SODIMMs, not soldered. Storage-wise, it comes out of the box with a 2TB PCIe 4.0 SSD, with space for another in the m.2 2230 form factor (best known as the form factor used on handheld gaming PCs).

It uses a 120W power brick, so no internal power supply gives Geekom more space to play with. It's configured for 54W on the APU, but realistically, you're only pushing this when you're gaming or doing some local AI. The highest I've seen it spike is 58W.

Most of the time it's very quiet, and all of the time very efficient. Geekom's cooling solution does the job, and even under extended load in my warm office, CPU temperatures haven't exceeded 87C (188F), while reaching 71C (160F) in gaming. GPU temperatures top out at 70C (158F).

The only thing really missing is an Oculink port. If you have an external GPU that uses this connection, you can't use it with the A9 Max. You can use an eGPU over USB 4, but that doesn't help if you already have one with Oculink.

"Crowned for AI, built to game"

The heading above is the exact tag line Geekom uses for the A9 Max. I rarely buy into marketing, but they're on the money with this one. The key is the Radeon 890M integrated GPU. It's built on RDNA 3.5, has 16 cores, and thanks to the ability to dedicate a chunk of the overall system memory for it, up to 24GB of VRAM in this spec.

You don't need that for gaming; in fact, for gaming, you don't need more than 8GB. It's configured out of the box with 2GB, but you can change this in the AMD software, reboot, and you're good to go. For gaming, I've been using 8GB, leaving 24GB as RAM. For AI testing, I've split it right down the middle at 16GB/16GB.

Doing this means that this tiny PC can even run gpt-oss:20b, a 20-billion-parameter LLM, with ease. Using LM Studio to leverage the Radeon 890M, I'm able to return 25 tokens per second on gpt-oss:20b with a 4k context window. The model easily fits inside the 16GB dedicated GPU memory, and the Radeon 890M has sufficient compute power to perform well.

Other similarly large models don't perform quite so well, but AMD worked with OpenAI on gpt-oss:20b, so it's understandable. But this is a perfectly usable little AI workstation, even more so if you go up to 64GB or 128GB of RAM. With Stable Diffusion, you can leverage the NPU for graphics work, aside from the Copilot+ features that tap into it. It's pretty slow, though, and I'd recommend just using the GPU.

It's a brilliant gaming box, too. While it's on the pricier side as something to dedicate as a small living room gaming PC, it's certainly got the horsepower for it.

In Cyberpunk 2077 at 1080p, medium settings with FSR balanced, we see an average of 61 FPS. Turn on frame generation, and you can add another 30 FPS to that. Call of Duty: Black Ops 6, set to balanced at 1080p, with the balanced FSR preset, delivers a 77 FPS average.

Forza Horizon 5 on high settings at 1080p hits 66 FPS average, and Shadow of the Tomb Raider at 1080p with medium settings and using Intel's XeSS on balanced returns 61 FPS.

Naturally, you can tweak all the settings you want, but it's perfectly possible to game at 60 FPS and above at 1080p on the Geekom A9 Max at medium to high graphics.

Not forgetting the CPU, either. That 12-core Ryzen AI 9 HX370 does well. Its Cinebench scores are on par with other tests we've done with this chip, likewise in other synthetic benchmarks. Grabbing some comparison data from our buddies at Techradar also shows that it outperforms the standard Apple M4 in the Mac Mini in these same tests.

I should point out one thing, though. The BIOS on the Geekom A9 Max is fairly limited, but there's one thing in there I changed that impacted performance, at least in the benchmarks. There's not much in the way of tweaks you can make, but by setting the fans to performance, you trade a little extra noise for a little extra performance. It bumped the Cinebench score by 30 points, which isn't insignificant.

All the PC most people will ever need

You should buy this if ...

✅ You want an efficient, yet powerful desktop PC.

✅ Connectivity is important.

✅ You'd like to be able to game and handle local AI workloads.

You shouldn't buy this if ...

❌ You need to process massive GPU workloads, such as larger AI models or high resolution gaming.

❌ You're shopping on a budget.

I've toyed with the idea for years of ditching my much-loved desktop gaming PC. It's my baby, and I adore it, but it's overkill for almost everything. That even includes gaming. Time spent with the Geekom A9 Max is the first time that has made me think that I can truly do it.

For most people, most of the time, this little box is all the PC they would ever need. It can game, it can tackle AI workloads, it's packed with enough of the latest tech that it won't be outdated for a long time to come.

AMD deserves its flowers, too. While the Ryzen AI 9 HX370 has now been succeeded by the ridiculous (and expensive) AI Max+ 395, this chip is no joke.

The CPU is incredible, the integrated GPU equally so, and mini PCs like the A9 Max are, I believe, the future of desktop computing.

For around $1,200, you get a high-performance PC that's also efficient enough to be run all day, every day, and not cripple your electricity bill. I did some basic calculations, and switching to the Geekom A9 Max over my gaming PC could save me as much as £200 a year in electricity alone.

I'll sacrifice some settings and frame rate in games, but then, with the new NVIDIA GeForce Now top-tier offering an RTX 5080, even that's not really an issue.

Should you buy one? Absolutely, if it's in your budget. This is one of the best mini PCs that can do it all.

NVIDIA just announced a "robot brain" called Jetson AGX Thor. The device, referred to often as simply "Jetson Thor," can power robots across several industries, including agriculture, healthcare, logistics, manufacturing, and transportation.

While you may not have heard of physical AI or NVIDIA robotics stacks, you've likely seen clips of some devices powered by Jetson Thor's predecessors.

Amazon Robotics, Boston Dynamics, Caterpillar, and Meta are among the early adopters of Jetson Thor. John Deere and OpenAI are evaluating the physical AI capabilities of the "robot brain."

“We’ve built Jetson Thor for the millions of developers working on robotic systems that interact with and increasingly shape the physical world,” said NVIDIA CEO Jensen Huang.

“With unmatched performance and energy efficiency, and the ability to run multiple generative AI models at the edge, Jetson Thor is the ultimate supercomputer to drive the age of physical AI and general robotics.”

Jetson Thor is powered by an NVIDIA Blackwell GPU. The company's Blackwell architecture is used to create AI chips, such as the B30A AI chip. NVIDIA's consumer RTX graphics cards are also built on the Blackwell architecture.

Jetson Thor developer kits can be purchased now for $3,499.

The performance leap delivered by Jetson Thor allows robots to perform tasks that previously required server-level compute power.

“The powerful edge processing offered by Jetson Thor will take Digit to the next level — enhancing its real-time responsiveness and expanding its abilities to a broader, more complex set of skills,” said CEO of Agility Robotics Peggy Johnson.

“With Jetson Thor, we can deliver the latest physical AI advancements to optimize operations across our customers’ warehouses and factories.”

Thanks to the on-device capabilities of robots powered by Jetson Thor, physical AI can take a "giant leap," according to NVIDIA.

What is physical AI?

The new robot brain from NVIDIA is designed to power physical AI tools. If you mention AI to many people in 2025, they'll likely think of a bot such as ChatGPT. To be fair, that is a form of AI that is used by millions of people each day, but it's only one type of AI.

According to NVIDIA CEO Jensen Huang, there are four waves of AI:

1. Perception AI: Recognizing speech, identifying images, and similar tasks.
2. Generative AI: Creating text and images — think ChatGPT, DALL-E, and Gemini.
3. Agentic AI: Reasoning models performing tasks independently. Agentic AI is quite new. Microsoft recently teased that Windows will be agentic.
4. Physical AI: Integrating AI with real-world devices, such as robots.

We're well into the second wave now and entering the first parts of the Agentic AI wave.

As you move from wave to wave, the computational demand increases drastically. Huang pointed out that Agentic AI requires 100-times more tokens and computational resources than initially anticipated.

Sebastian Scherer, an associate research professor at Carnegie Mellon's Robotics Institute, shared how advances in computing capabilities affect robotics:

"We can only do as much as the compute available allows. Years ago, there was a big disconnect between computer vision and robotics because computer vision workloads were too slow for real-time decision-making — but now, models and computing have gotten fast enough so robots can handle much more nuanced tasks.”

When you consider the demand for compute power, it makes sense why NVIDIA would need to make a device as powerful as Jetson Thor.

I'm far from a robot engineer, so please understand that I'm oversimplifying things quite a bit.

Physical AI lets devices such as robots and self-driving cars perceive the space around them and act accordingly.

Relatively speaking, it's not that difficult to create a robot or machine that can perform the same task repeatedly when conditions are identical. But the world does not often create identical circumstances.

Think of physical AI as a much more powerful robot vacuum. As the machine goes around your house and cleans, it will run into various surfaces, obstacles, and maybe even get stuck in a corner. That robot vacuum needs data to help direct its next steps.

NVIDIA's Jetson Thor can power surgical robots, smart tractors, and industrial manipulators (machines that require fine motor skills and positional awareness). Those devices run into far more complicated problems than identifying a carpet or hardwood floor.

In comes Jetson Thor, which delivers 7.5 times the compute power of Jetson Orin. To attach some more concrete numbers to the robot brain, Jetson Thor delivers up to 2070 FP4 Teraflops of AI compute and has 128GB of memory.

NVIDIA explained that Jetson Thor delivers "formerly server-level compute" on-device. That means a lot more compute power for devices like Boston Dynamics' Atlas robot.

AI is here to stay, and it's far more than just using online tools like ChatGPT and Copilot. Whether you're a developer, a hobbyist, or just want to learn some new skills and a little about how these things work, local AI is a better choice.

Ollama is one of the easiest and most popular ways to dabble with LLMs on your local PC. But one thing you need — that you don't when using ChatGPT — is decently powerful local hardware. Ollama, especially, because it only supports dedicated GPUs right now. But even if you're using LM Studio with integrated GPUs, you still need some decent hardware to get good performance.

So, you might be thinking you need to spend an extortionate amount of money, right? Wrong. While, yes, you can run out and drop thousands on an RTX 5090, you don't need to. There's plenty of mileage to be had buying older, or cheaper cards. If you have the budget to run an NVIDIA H100, more power to you, but there's a much cheaper way.

VRAM is key for Ollama

The priority when buying a GPU for AI is not the same as when buying one for gaming. When you're looking for a new gaming system, usually you'll lean towards the latest gen, and the most powerful you can afford, all in pursuit of the best looking graphics and highest frame rates.

There's an element of this type of thought process when buying for AI, but it's less important for the home user. Sure, the latest 50 series GPUs may have more CUDA cores, faster compute, better memory bandwidth. But what is king, is memory.

If you don't have a mega budget to buy the latest, most powerful AI GPUs out there, your priority should be VRAM. Get as much of it as you possibly can.

To leverage your GPU to its fullest using local LLMs, you want the model you're using to be loaded entirely into the GPU's VRAM. For one, VRAM is faster than system memory. But if you start spilling out into that system RAM, you'll be losing performance. Any time your CPU has to pick up any of the slack, which it does in Ollama when this happens, you're going to hemorrhage performance.

As a brief example, let's look at Deepseek-r1:14b running on my own PC with an NVIDIA RTX 5080 with 16GB VRAM, backed by an Intel Core i7-14700k and 32GB DDR5 RAM. Using a simple prompt of "tell me a story," it returns about 70 tokens per second performance with a context window up to 16k. If I increase beyond that, the model no longer fits entirely into the GPU memory and starts engaging the system RAM and the CPU.

When this happens, output drops to 19 tokens per second, despite the CPU/GPU split being 21%/79%. Even when the GPU is still doing the majority of the work, when the CPU and RAM get called into action, performance plummets.

The same applies to laptops, too, even if we're focusing on desktop GPUs here. If AI is important, get one with a GPU that has as much VRAM as possible.

So, how much VRAM do you need?

The short answer is as much as possible. The better answer is as much as your budget allows for. But the more accurate answer rests on the models you want to use. I can't answer that one for you, but I can help you work out what you need.

If you go to the models page on Ollama's website, each model will be listed with details on how big it is. At the bare minimum, this figure is the amount of VRAM you'll need to load the memory onto your GPU entirely. But, you'll need more than this to actually get anything done. And the larger the context window, the more information you're feeding into the LLM, the more memory you'll use up (also known as the KV cache).

Many folks recommend taking the model's physical file size and multiplying it by 1.2 to get a ballpark figure. Example time!

OpenAI's gpt-oss:20b is 14GB. Multiplying that by 1.2 gives you a figure of 16.8. From personal experience, I can attest to some accuracy here. Unless I run it on the 16GB RTX 5080 with an 8k or lower context window, I can't keep it loaded entirely on the GPU.

At 16k and above, the CPU and RAM get called upon, and performance starts to go over a cliff. So, in an ideal world, I'd say you would want a 24GB VRAM GPU to use this model however you wish to use it comfortably.

It harks back to the point above. The more you call on the CPU and RAM, the worse your performance will be. Be that larger models, larger context windows, or a combination of both, you want to keep that GPU in play as long as possible.

Older, cheaper GPUs can still be great buys for AI

If you search around on the web long enough, you'll see a pattern emerging. Many would argue the RTX 3090 is the best buy for power users right now, balancing price, VRAM, and compute performance.

Taking different architectures out of the equation, the RTX 3090 is on par with the RTX 5080 when it comes to CUDA core count and memory bandwidth. It leaps ahead of the RTX 5080 when it comes to VRAM, with 24GB in the older card. The 350W TDP of the RTX 3090 is significantly better than the 24GB RTX 4090 and 32GB RTX 5090, too. Obviously, the latter will perform the best, but it's also absurdly expensive to buy and run.

There is another way, too. The RTX 3060 is something of a darling of the AI community because of its 12GB of VRAM and its relatively low cost. Memory bandwidth is significantly lower, but so is the TDP at just 170W. You could have two of these and match the TDP and total VRAM of an RTX 3090, while spending much less.

The point is that you absolutely do not need to run out and buy the biggest, most expensive GPU you can find. It's a balancing act, so do some research into what you want to use.

Then decide how much you'd like to spend, and get a GPU with the fastest, largest pool of VRAM you can. Ollama will let you use more than one GPU, so if two 12GB cards are cheaper than a single 24GB card, it's an option worth consideration.

And finally...

If you can, go NVIDIA. Ollama does support a number of AMD GPUs, but honestly, you want NVIDIA right now for local AI. The red team is constantly making advancements, but NVIDIA has such a head start, it's just the way things stand right now.

AMD has its strengths, though, especially in its mobile chips for laptops and mini PCs. For the price of one of today's best graphics cards, you could buy a whole mini PC with a Strix Halo generation Ryzen AI Max+ 395 APU paired with 128GB of unified memory, and the true desktop class Radeon 8060S integrated graphics.

Ollama wouldn't be your play on a system like this, or anything else using a Ryzen AI chip, since it doesn't support the iGPUs. But LM Studio with its Vulkan support does. A 128GB Strix Halo system gives you up to 96GB of memory to dedicate to LLMs, and that's a whole heck of a lot.

You have options, though, and options that are more budget friendly. Check your desired GPU against Ollama's list if it's an older generation card, get as much VRAM as you can, even across multiple GPUs if it makes sense. If you can score a pair of 12GB RTX 3060s for $500-$600, you're off to a really strong start.

But don't let your hardware, or your budget, stop you joining in the fun. Just do a little research, first, to maximize your returns.

Ollama is one of the easiest ways you can experiment with LLMs for local AI tasks on your own PC. But it does require a dedicated GPU.

However, this is where what you use will differ a little from gaming. For example, you may actually have a better time with local AI using an older RTX 3090 than, say, a newer RTX 5080.

For gaming purposes, the newer card will be the better option. But for AI workloads, the old workhorse has one clear edge. It has more memory.

VRAM is king if you want to run LLMs on your PC

While the compute power of the later generation GPUs may well be better, if you don't have the VRAM to back it up, it's wasted on local AI.

An RTX 5060 will be a better GPU for gaming than the old RTX 3060, but with 8GB VRAM on the newer one versus 12GB on the older one, it's less useful for AI.

When you're using Ollama, you want to be able to load the LLM entirely into that pool of deliciously fast VRAM to get the best performance. If it doesn't fit, it'll spill out into your system memory, and the CPU will start to take up some of the load. When that happens, performance starts to dive.

The exact same principle is true also if you're using LM Studio instead of Ollama, even if you're using that with an integrated GPU. You want to have as much memory reserved for the GPU as possible to load up the LLM and keep the CPU from having to kick in.

GPU + GPU memory is the key recipe for maximum performance.

The easiest way to get an idea as to how much VRAM you need is to just see how physically large the model is. On Ollama, gpt-oss:20b, for example, is 13GB installed. You want to have enough memory available to load all of that as a bare minimum.

Ideally, you want to have a buffer, too, as larger tasks and the context windows to fit them will need a buffer to load into as well (the KV cache). Many recommend multiplying the size of the model by 1.2 to get a better ballpark figure as to the VRAM you want to be shooting for.

This same principle applies if you're reserving memory for the iGPU to use in LM Studio, too.

Without enough VRAM, performance will tank

Time for some examples! This is an extremely simple test, but it illustrates the point I'm trying to make. I ran a simple "tell me a short story" prompt on a range of models using my RTX 5080. In each case, all of the models can be loaded entirely into the 16GB VRAM available.

The rest of the system contains an Intel Core i7-14700k and 32GB DDR5 6600 RAM.

However, I increased the context window on each model to force Ollama to have to spread over to the system RAM and CPU as well, to show just how far the performance drops off. So, to the numbers.

• Deepseek-r1 14b (9GB): With a context window of up to 16k GPU usage is 100% at around 70 tokens per second. At 32k there is a split of 21% CPU to 79% GPU, with performance dropping to 19.2 tokens per second.
• gpt-oss 20b (13GB): With a context window of up to 8k GPU usage is 100% at around 128 tokens per second. At 16k there is a split of 7% CPU to 93% GPU with performance dropping to 50.5 tokens per second.
• Gemma 3 12b (8.1GB): With a context window of up to 32k GPU usage is 100% at around 71 tokens per second. At 32k there is a split of 16% CPU to 84% GPU with performance dropping to 39 tokens per second.
• Llama 3.2 Vision (7.8GB): With a context window of up to 16k GPU usage is 100% at around 120 tokens per second. At 32k there is a split of 29% CPU and 71% GPU with performance dropping to 68 tokens per second.

I'll be the first to say this is not the most in-depth scientific testing of these models. It's merely to illustrate a point. When your GPU cannot do all of the work itself and the rest of your PC comes into play, the performance of your LLMs will drop quite dramatically.

This is just a very quick test designed to illustrate the importance of having enough available GPU memory to allow your LLMs to perform at their best. You don't want to have your CPU and RAM picking up the slack wherever possible.

The performance here was still decent when that happened, at least. But that's with some solid hardware to back up the GPU.

The quick recommendation would be to have 16GB VRAM at least if you want to run models up to gpt-oss:20b. 24GB is better if you need space for more intensive workloads, which you could even get with a pair of well-priced RTX 3060s. You just need to work out what's best for you.

OK, I'll start by saying the headline is a little tongue in cheek. Of course, we think a Windows 11 PC is better than a Mac. You're not convincing us otherwise.

But Geekom's next mini PC is very much in the same arena as the Mac Studio. They're billing the Geekom A9 Mega as "the most powerful mini PC on earth" and there's a degree of truth to that.

It won't stand alone, but there isn't much company for it. That's because it's going to be one of only a small number of PCs, mini or otherwise, confirmed to use AMD's Strix Halo APU. Or, to give it its full title, the AMD Ryzen AI Max+ 395. I'll be calling it Strix Halo from here on out.

This is simply the king of mobile chips right now, and not yet widespread. Part of that is surely down to the price. We've seen it ourselves inside the ASUS ROG Flow Z13, and it's inside an HP mini workstation we currently have in the review backlog. You'll also find it in the Framework Desktop, and a mini PC from GMKTec.

I might have missed one or two, but there's not many. And that in itself is enough reason to take serious notice of the Geekom A9 Mega.

The beating heart of the Geekom A9 Mega is that Strix Halo APU backed up by up to 128GB LPDDR5X 8000 unified memory. Unlike previous generation APUs from AMD, the Strix Halo adopts a similar approach to Apple Silicon, where the system can dynamically allocate memory to the GPU up to a total of 96GB.

That's different to the way it's done on previous generation chips, like the Ryzen AI 9 HX370 in the Geekom A9 Max I'm currently in the process of reviewing. On these chips, you're able to silo off a chunk of the RAM to be dedicated for the GPU, but it's unable to do it dynamically like a unified memory system.

It's more efficient, reduces bottlenecks, and latency. It's just better.

The Geekom A9 Mega will also be a potent little box for running local AI. Despite its size, the fact it can dedicate up to 96GB of memory to the GPU means even pretty massive LLMs will run locally. This thing will swallow up the full gpt-oss:120b.

That GPU is also a huge deal. Strix Halo comes with the Radeon 8060s, the first true desktop-quality integrated graphics. It can go toe-to-toe with RTX 40 series cards. It can even do ray tracing.

AMD really has changed the game with Strix Halo, but it doesn't come cheap. Geekom will be going live first through Kickstarter, and if you sign up ahead of launch you can get one at a reduced price of $1,899.

But hey, to get a Mac Studio with a measly 96GB of unified memory you have to spend $3,999. In that context, it's an absolute steal. It'll be better for gaming, too, because, Windows 11.

I'm just happy to see another option coming with Strix Halo inside. Alongside the Snapdragon X platform, it's one of the most exciting hardware advancements in Windows PCs in many a year. I'm looking at my gaming PC under my desk right now and starting to wonder how long it is for this world.

Sign up to be notified of the Kickstarter launch for the Geekom A9 Mega on its page right now. At this time, there is no indication when that might be.

NVIDIA is the world's largest AI GPU supplier with roughly 90% of the market, but its AI efforts are not limited to data centers.

NVIDIA's gaming hardware, including RTX graphics cards, includes built-in AI tools that help with upscaling, overall performance, and even game development.

Announced today at Gamescom, NVIDIA's Project G-Assist — an on-device AI helper — has now expanded to all RTX GPUs with 6GB or more VRAM, including laptop graphics cards.

The lower system requirements arrive thanks to "a new, significantly more efficient AI model that's faster and uses 40% less memory while maintaining response accuracy," according to NVIDIA's press release.

NVIDIA says that the updated AI assistant will be available through a Game Ready Driver update within the NVIDIA App on August 19. Just download the G-Assist update on the NVIDIA App's home screen.

With this new update, NVIDIA has also added a collaboration with mod.io that will help discover and deliver G-Assist plugins built by regular users. Anyone can build plugins for G-Assist, just as long as you have an understanding of JSON and Python scripts. A specialized plugin builder is available to download from NVIDIA.

Beyond that, NVIDIA says that another update is expected in September 2025, containing laptop-specific tools to help with NVIDIA BatteryBoost and Battery OPS.

Project G-Assist is a good idea in theory ... not so much in execution. At least not yet. I'm looking forward to giving the updated NVIDIA Project G-Assist a try, as the first attempts at getting it to work were unimpressive (to put it nicely).

What is NVIDIA's Project G-Assist, and should you care?

Announced first at Computex 2024, NVIDIA's Project G-Assist is a free AI assistant built for gamers. It's a Small Language Model (SLM) that runs locally on your PC's RTX graphics card, providing "real-time diagnostics and recommendations to alleviate system bottlenecks, improve power efficiency, optimize game settings, overclock your GPU, and much more," according to NVIDIA's official press release.

The way Project G-Assist is designed allows for "natural language instructions," whether delivered via text or voice. So far, it only understands English, but I expect to see further language support if the project proves popular enough.

As mentioned, users can build their own plugins for Project G-Assist, and they'll now be available directly through the app thanks to a newfound collaboration with mod.io.

When Project G-Assist initially launched, NVIDIA slapped some hefty system requirements on the AI assistant. It required an RTX 30, 40, or 50-series desktop GPU with at least 12GB of VRAM, which excluded even the latest Blackwell RTX 5000 cards like the 8GB RTX 5060 Ti and RTX 5060. It also lacked support for laptop GPUs and NVIDIA RTX PRO GPUs.

That's all changed, and I'm happy to see Project G-Assist become available for more RTX gamers and developers.

Early issues with NVIDIA's Project G-Assist

Since its inception, NVIDIA's AI assistant has been sitting on a Version 0.1 release. It's an experimental release, but Project G-Assist was a lot worse than any of us expected when we tested it in March 2025.

Yes, it works fine for some, but my colleague Richard Devine had a terrible time attempting to use G-Assist with an RTX 5080 Founders Edition GPU — you know, the first-party cards that NVIDIA makes — with 16GB of VRAM.

Devine's testing revealed that G-Assist didn't do much more than tank framerates (often from triple digits to single digits) or completely lock up the system. Devine tested G-Assist with Call of Duty: Black Ops 6, Atomfall, and Avowed, and the results were the same across each title.

It's not like Devine was attempting to crash the system. Asking something as simple as "Is G-Sync enabled?" nearly crashed games, and that was despite having 4GB of VRAM more than the previously recommended 12GB.

With NVIDIA introducing a lighter AI model and a new, lower 6GB of VRAM requirement, I'm eager to see how it actually performs on desktop PCs and gaming laptops.

Should you be using NVIDIA's Project G-Assist while PC gaming?

In theory, Project G-Assist could be a useful tool for gamers. Want to make a quick adjustment to system settings without leaving your game? Ask AI to do it for you.

However, as it stands now, G-Assist still has a long way to go. Version 0.2 could solve a lot of the aforementioned issues, and you might find it particularly useful for your specific needs.

👉 Related: The NVIDIA app on Windows 11 could be sapping performance out of your PC games

Project G-Assist is certainly not something I recommend every RTX owner install immediately, but you can test it out with relative ease straight from the official NVIDIA App.

If it's not right for you — or if it continues to flat-out not function properly — you can always uninstall it without harming (as far as we can tell) any other functionality in NVIDIA's app.

Are there privacy concerns related to NVIDIA's Project G-Assist?

Having an AI agent comb through your PC for information might not sit well with everyone, and I don't blame you for being wary.

In the fine print on NVIDIA's main G-Assist page, there are some disclaimers that might make you think twice before installing the feature.

"G-Assist is a pre-release feature and may not be fully functional, may contain errors or design flaws, and may have reduced or different security, privacy, availability and reliability standards relative to commercial versions of NVIDIA offerings."

Below this line is another that basically states that NVIDIA isn't responsible for any damages the AI agent could cause if it gives wrong information or decides to go rogue and tune your PC outside of its operating boundaries.

"AI models generate responses and outputs based on complex algorithms and machine learning techniques, and those responses or outputs may be inaccurate, harmful, biased or indecent. By using G-Assist, you assume the risk of any harm caused by any response or output from it. NVIDIA will not be responsible for any actions, losses, or damages suffered as a result of use of G-Assist or its output. We recommend that you verify the information before relying on it."

Bottom line? Don't take what G-Assist says too seriously, and always put your best common sense before what AI tells you.

One thing I learned in a world after having children is that time is precious. Specifically, here, I'm talking about the amount of web pages I read on a daily basis. Be it research, reading news or reviews, or extracting specific pieces of data, when you work in this business, you're reading a lot of content.

Copilot is fantastic at creating summaries, and it's been getting better and better. In the older days, I'd need to specify what I was looking for, the format I wanted the information in. Now, it just does it, and does it well.

But, I've been experimenting a lot more in recent times with local AI, such as Ollama and LM Studio, rather than relying entirely on the likes of Copilot, ChatGPT, and Google Gemini.

Many of the methods to replicate this feature seem to involve Python. While I'm not allergic to a little tinkering, for this, I just wanted to see if there was something simple and browser-based to do what I'm looking for.

The good news is that I found something. The bad news is that it's still not as good as Copilot at this task.

Ollama in the back and Page Assist in the front

While searching for solutions, I came across Page Assist, a browser extension in the Chrome Web Store (or Microsoft Edge Addons if you prefer). It's more than just an extension, though; it's a full-featured web app that acts as a GUI frontend for Ollama.

It's extremely useful and absolutely jam-packed with features. There's probably something not here, but I haven't figured out what it might be yet. It makes interacting with and managing local LLMs an absolute breeze, and it's significantly more advanced than the official Ollama app.

The full interface isn't what I'm interested in here, though. The golden goose is the sidebar, with the ability to interact with a currently open web page. I found it!

While Page Assist will see your Ollama installation automatically, there's a little more work to do to make it talk to a web page. You need some RAG.

Retrieval-Augmented Generation. I'm not going to attempt to explain it because I'm still trying to get my head around it myself. There are plenty of articles out there, such as this one by IBM, that can do a better job than I ever would.

To use Page Assist to talk to a web page, we need an embedding model pulled into Ollama. The app makes it easy, recommending Nomic Embed. So that's what I did, right from Page Assist using the Manage Models menu in Settings, clicking to add a new model and entering nomic-embed-text:latest into the box.

The only other step is figuring out how to launch the side panel. By default, it's in the right-click menu. So you're on a page you want to talk about, right-click, open the side panel to chat.

It's good, but not AS good as Copilot

Does it actually work? Yes, it does. But it doesn't work as well as Copilot, sadly.

I'll start with the good. Using a variety of different models to test it out, I can indeed summarize articles. The bigger the model, the better job it does. It'd be foolish to think gemma3:1b could do as much as gpt-oss:20b. But it does what I ask it to do in summarizing web articles.

It will also use vision on images if you load a model with that capability. I probably shouldn't have used a small model, though, as you can see in the image above. But it's perfectly able to see the images, use OCR, and whatnot.

So far, so good. But the biggest problem is that it doesn't seem to recognize when you move from one web page to another. The way I work is that if I'm in speed reading mode, I leave the side panel open and just rerun the prompt as I move from page to page.

If I don't start a new chat or physically tell the model to forget the previous article, and now look at and summarize the new one, it keeps repeating itself based on what it saw on the earlier one. It's not a dealbreaker, but it's an annoyance.

Whichever model I've tried so far also just doesn't give the same quality breakdown as Copilot seems to. Obviously, I want a summary, but Copilot just gives a better summary without needing to have a very specific prompt to begin with.

I'm sure over time I could make something better from this, but that's work I need to do. And this is about speed and convenience. Copilot is so much more conversational, whereas these local models seem very matter-of-fact.

The other thing I like about Copilot in this instance is that it provides some follow-up questions. I don't always use them, but from time to time, a lightbulb goes off, and one of them presents me with an angle I hadn't thought of.

A case where I think I might stick with online AI

There are some good reasons to use local AI over online equivalents. However, in this instance, I have to be online to read the web pages I want summarizing, anyway. So as long as it works as well as it does, there's no reason not to continue.

I do still encourage folks to have a look into local AI, though. A big part of it right now, personally, is education. Every day I'm learning something new.

In pursuit of what I've discussed in this article, I've also found a new favorite way to use Ollama. Page Assist is brilliant, and I'll probably write more about it in the future, after I've spent more time diving into its features.

But when I'm working, I think I'll stick to the online tools for this task. At least for now.

I've been playing around a lot more recently with LLMs running locally on my PC, and for the most part, that's been with Ollama.

Ollama is a fantastic tool that makes it extremely simple to download and run LLMs on your own PC, made even easier with the launch of its new GUI application. It can hook into your workflow beautifully, but it has one glaring flaw.

If you don't have a dedicated GPU, its performance isn't that good. I've been using it with an RTX 5080 (and soon to try an RTX 5090), and it flies along, but on something like the new Geekom A9 Max mini PC, which just landed for review, it's a very different story.

Ollama does not officially support integrated graphics on Windows. Out of the box, it'll default to using the CPU, and honestly, there are times when I'm not interested in looking for workarounds.

Instead, enter LM Studio, which makes it so simple to leverage integrated GPUs, even I can do it in seconds. That's what I want, that's what you should want, and it's why you need to ditch Ollama if you want to leverage your integrated GPU.

What is LM Studio?

Without going too far into the weeds, LM Studio is another app you can use on your Windows PC to download LLMs and play about with them. It does things a little differently from Ollama, but the end result is the same.

It also has a significantly more advanced GUI than Ollama's official app, which is another tick towards using it. To get the most from Ollama outside of the terminal, you need to use something third-party, such as OpenWebUI or the Page Assist browser extension.

It's ultimately a one-stop shop to find and install models, and then interact with them through a familiar-feeling chatbot interface. There are many advanced features you can play with, but for now, we'll leave it simple.

The big win is that LM Studio supports Vulkan, which means you can offload models on both AMD and Intel to the integrated GPU for compute. That's a big deal, because I'm yet to see in my own testing a situation where using the GPU wasn't faster than the CPU.

So, how do you use an integrated GPU in LM Studio?

The other beauty of just going the LM Studio route is that there's nothing fancy or technical that needs to be done in order to use your iGPU with an LLM.

To use a model, you simply load it up with the dropdown box at the top. When you select the one you want, a bunch of settings will appear. In this instance, we're only really interested in the GPU offload one.

It's a sliding scale and works in layers. Layers are essentially the blocks that make up the LLM, and instructions will pass down them one by one, until the final layer produces the response. It's up to you how many you offload, but anything below all of them means your CPU will be picking up some of the slack.

Once you're happy, click load model, and it'll load into memory and operate within your designated parameters.

As an example, on the aforementioned Geekom A9 Max, I have Radeon 890M integrated graphics, and I'm wanting to use basically all of it. When I'm using AI, I'm not gaming, so I want all of the GPU to focus on the LLM. I set 16GB of my 32GB total system memory as reserved for the GPU to load the model into, and get to work.

With a model like gpt-oss:20b I can load the entire thing into that dedicated GPU memory, use the GPU for compute, leave the rest of the system memory and the CPU well alone and get around 25 tokens per second.

Is it as fast as my desktop PC with an RTX 5080 inside? Not at all. Is it faster than using the CPU? Absolutely. With the added benefit that it isn't using almost all of my available CPU resources, resources that other software on the PC might want to use. The GPU is basically dormant most of the time; why wouldn't you want to use it?

I could probably get even more performance if I spent the time getting into the weeds, but that's not the focus here. The focus is on LM Studio, and how, if you don't have a dedicated GPU, it's hands-down the tool to use for local LLMs. Just install it and everything works.

I've been playing around with OpenAI's gpt-oss:20b the last couple of days. As the company's first open-source model, it's the first chance we've had to try it out without going through the API or a tool like ChatGPT or Copilot.

It's based on GPT-4 and has, so it says, a knowledge cutoff of June 2024, which actually bests some of the other open source models out there right now. But it can also use web search to fill in some gaps, if you wish.

Don't ask me why, but I had the grand idea to test it out against something real-world that my son has been doing. In the UK (for those in other countries who won't be familiar) there is a test called the 11+. It's taken as an entry method to get into selective schools.

As we're gearing up for this ourselves, I thought I'd see if gpt-oss:20b could interpret a practice test, and actually solve the problems on it. A test I coined "is it smarter than a 10-year-old?"

Fortunately, for my son anyway, he's still way out in front of at least this AI model.

The test and the hardware

I'll preface all of this by saying that despite having a pretty decent PC, it's still not ideal for running gpt-oss:20b. I have an RTX 5080 with 16GB of VRAM, and this isn't quite enough it seems for this model. So my system has also been making good use of the CPU and system RAM.

It's important to point that out, because on a better setup, the response times will almost certainly be lower. Eventually I'll be getting hold of our RTX 5090 to try ridiculous AI tasks on, which is far better suited with 32GB VRAM.

The test was simple. I downloaded a sample practice paper for the 11+ and attached it to a context window through Ollama. I used this prompt:

"I've attached a sample paper for the UK 11+ test taken by children aged 10 and 11 in order to gain entry to grammar school. Please read the test and answer all questions."

This isn't the greatest prompt in the world, for one, because I didn't ask it to show me its workings. Just simply read the test, give me the answers.

So how badly did it do?

Terrible.

After thinking for almost 15 minutes, it outputted 80 answers to the 80 questions on the test. It looks to have got nine of them correct. The pass mark is a little bit higher than that.

Some of the questions it got right are in the image above. In order, the answers given are:

• lock, baby
• pace, bite
• tree, hair
• inch, wept
• ride, grip

But here's where it takes a turn. Those weren't the first questions in the test, merely the first it answered correctly. It then answered the next four questions correctly, where a four-letter word was hidden at the end of one word, and the beginning of the next in a sentence.

It couldn't answer the last two sentences in this section, and there everything really fell apart. Number sequences were a total bust, and the remainder of the questions, be they using word or number problems, just didn't pan out.

What's worse is that instead of incorrect answers, many were answers not remotely relevant to the questions asked. But, looking through the reasoning, things were different. Take the two questions below as another example.

The answers the model output was an identical (and irrelevant) number sequence for both questions. But, here's the interesting part. You can see the model reasoning out its responses in the context window, and this remains after the outputs have been delivered.

The model reasoned out this answer for question 6. Apologies it's quite long, but it also accurately follows the sort of workings a child doing this test would follow:

"We need to count items each buys. We have 5 people. We have to deduce what each person has.

Given info:

Molly, Naomi, Jamie all have chips.

Mahmood chooses pie and mushy peas.

Only one to have fish is Molly. So Molly has fish.

Jamie, Louise and Naomi each have a sausage. So each of those three has sausage.

Everyone has mushy peas with their meal except Jamie and Molly. So Jamie and Molly don't have mushy peas. So Mahmood has mushy peas (given). Naomi has mushy peas? Wait "everyone has mushy peas except Jamie and Molly" meaning all except them have mushy peas. So Naomi, Louise, Mahmood have mushy peas. But Naomi already has chips, sausage, mushy peas. Louise has sausage, chips? Wait not specified but likely. Let's list:

Mahmood: pie, mushy peas. Does he have chips? Not specified. He might have something else. We know he does not have chips? Not mentioned. But might not. We have constraints.

Jamie: chips, sausage. No mushy peas, no fish. Might have something else? Not specified.

Louise: sausage. Not mentioned chips, but could have chips. Might have mushy peas? Yes because everyone except Jamie and Molly have mushy peas, so Louise has mushy peas. Also maybe chips? But we have not specified.

Naomi: chips, sausage, mushy peas. No fish? Not specified. Also maybe chips.

Molly: chips, fish, no mushy peas. Also maybe sausage? Not given. But "Jamie, Louise and Naomi each have a sausage" only those three. So Molly doesn't have sausage. So she has chips, fish, maybe mushy peas? No.

Now we must count number of items each buys. Items: chips, mushy peas, fish, sausage, pie. Also maybe others.

Let's list each:

Mahmood: pie, mushy peas. That's 2 items. Could he also have chips? Not mentioned. But we can't assume. We need deduce from "Everyone has mushy peas with their meal except Jamie and Molly." That includes all except them. So yes, Mahmood has mushy peas. And pie is separate. So 2 items.

Jamie: chips, sausage. No mushy peas. So 2 items.

Louise: sausage, mushy peas. Also chips? Not stated. Could be chips? If she had chips, that would be 3 items. But no info prohibits. But we must see if any other constraints: "Molly, Naomi and Jamie all have chips." That says only those three have chips? Wait the sentence: "Molly, Naomi and Jamie all have chips." That explicitly says those three have chips. Does it say others do not have chips? Not necessarily. But typical test may imply only those three have chips. But ambiguous. Usually phrasing "Molly, Naomi and Jamie all have chips" means those three have chips; others may or may not. But the following statements: "Jamie, Louise and Naomi each have a sausage." That indicates Louise has sausage; but no mention about chips for Louise. However if we interpret that only those three have chips, then Louise doesn't have chips. So Louise has sausage and mushy peas, that's 2 items. But if Louise had chips, then 3 items, but not given.

We need find who buys most items. Let's evaluate possibilities.

Case A: Louise does NOT have chips. Then each person has 2 items: Mahmood 2, Jamie 2, Louise 2, Naomi 3? Naomi has chips, sausage, mushy peas => 3 items. Molly has chips, fish => 2 items. So Naomi has most (3). Case B: Louise has chips: then Louise 3 items; Naomi 3 items; others 2. Tie. But we need unique answer. Likely answer is Naomi.

Thus answer: Naomi.

But we need to check: Could someone have more than 3? Not likely.

Thus answer for question 6: Naomi."

Similarly, it reasoned the right answer for the following question, but output a completely irrelevant answer, too.

The other interesting thing I can see from the reasoning is that when it gets stuck, it says to skip and move on. In its final reasoning where it's generating the responses, it said this:

"This is unsatisfactory.

Given that this is unrealistic, maybe the answer is not expected.

Thus I'd respond that it's too long.

But I should provide all. But I'm stuck.

Given time, I'll produce partial answer and state I can't solve all.

Let's do that."

The trouble is that instead of actually doing that, it just threw out random answers. I think I know why, which I'll touch upon below.

An interesting follow-up test

To be clear, I wasn't expecting to get a 100% correct answer rate. I wasn't really expecting anything. But I did learn a few things.

For one, I don't think my initial settings were good enough. But then I re-ran the test, increasing the context length in Ollama to 32k, wondering if that was the reason for so many failures.

This time around, it took almost the full hour that the kids are allowed to finish its reasoning. It did much better, answering only one of the first batch of 12 questions wrong. Further through, it fared better with questions relating to number sequences, where the first run completely fell apart.

The trouble this time around is that instead of outputting answers to questions it had clearly got right at times, the responses were complete nonsense again.

Instead of answers, it created its own quiz in a similar vein to the test. The reasoning part seems to have been partly successful, even if it took nearly an hour, but the output completely ignored the initial prompt.

But, as a lesson to myself if nothing else, for a large document like this, 'memory' is key. However, the detriment there is that setting it too high can cause performance issues. It took so much longer this time around, in part, because it was offloading a significant amount of operation to my CPU and system memory.

Increasing to 128k context length on my system doesn't seem to load any of the model onto my RTX 5080's VRAM, instead doing everything using the CPU and system memory. So it's a lot slower. This applies even to simple prompts, with eval rate increasing from 9 tokens per second at 128k up to 42 tokens per second with an 8k context length, and 82 tokens per second at 4k.

Fun with some lessons learned

At first, this was simply a test to see if gpt-oss:20b could successfully interpret a PDF file. On that front, it seems to have been a win. It took the file, read it, and at least tried to do what I asked with it.

The first run seems to have been limited by context length. It simply didn't have enough 'memory' to do what I wanted. The second run, which took much longer, was more successful.

I haven't tried this test again with the max 128k context length Ollama currently allows in the app. Partly because I don't think I have the time right now, and partly because I'm not sure if the hardware I have is up to the task.

The RTX 5080 isn't quite there for this model, with 'only' 16GB of VRAM. It was being used, but my CPU was picking up some of the slack. Despite being much smaller than the other gpt-oss model available, it's still on the larger side to use on a gaming PC like this.

Running gpt-oss:20b on my system at default settings to solve the test I gave it, the GPU was only doing 65% of the work, the CPU was picking up the rest of the slack. And, as seen with further investigating, with a maxed out context length it doesn't seem to load any of the model onto the GPU.

But I am impressed with its reasoning capabilities.

This model is also quite slow. Again, I'm factoring my hardware into the equation, but it seems to apply reasoning to every prompt. I asked it what its knowledge cutoff date was, and it thought for 18 seconds first. Going through every possible explanation it could give, and why it settled on the one it did.

This has its advantages, of course. But if you're looking for a speedy model to use at home, you'll want to look elsewhere. Gemma3:12b is my go-to right now for performance given the hardware I have.

So, a real-world test? Sort of. Did it win? Definitely not, and my 10-year-old can sleep soundly knowing he's still better than at least this AI model. What I can confidently say though is that playing around with this, and other models, is building up my own knowledge base.

And for that, I can't ever be disappointed. Even if running these did turn my office into a sauna for the afternoon.

We're led to believe that running AI locally on a PC needs some kind of beefed-up hardware. That's partly true, but as with gaming, it's a sliding scale. You can play many of the same games on a Steam Deck as you can on a PC with an RTX 5090 inside. The experience is not the same, but what matters is that you can play.

That's also true of dabbling with local AI tools, such as running LLMs using something like Ollama. While a beefcake GPU with lashings of VRAM is ideal, it's not absolutely essential.

Case in point: my seven-year-old Huawei MateBook D with a now fairly underpowered AMD Ryzen 5 2500U, 8GB of RAM, and no dedicated graphics. But it can still use Ollama, it can still load up some LLMs, and I would say it's usable.

There are caveats to running AI on older hardware

Gaming is the perfect comparison use case that comes to mind for AI right now. To get the most from the latest, most demanding content, you need some serious hardware. But you can also enjoy many of the latest titles on older and lower-powered machines, even those that only rely on integrated graphics.

The caveat is that the older, less powerful hardware simply won't perform as well. You're probably looking to hit 30 FPS instead of (at least) 144 FPS, but you can do it. But you'll have to sacrifice graphics settings, ray tracing, and resolution.

The same is true of AI. You're not going to be churning out hundreds of tokens per second, nor will you be loading up the latest, biggest models.

But there are plenty of smaller models you can absolutely try out, as I have, successfully on older hardware. If you have a GPU that's compatible, great, it'll use that. I don't, however, and I've still had some success.

Specifically, I've loaded up some '1b' models, that is, 1 billion parameters, into Ollama on my old laptop, which is currently running Fedora 42. The APU isn't officially supported by Windows 11 anyway, but I usually run Linux on older hardware regardless.

Ollama is platform-agnostic, though, with versions for Mac and Windows alongside Linux. So it doesn't matter what you're using; even an older Mac may get some mileage with this.

So, just how 'usable' is it?

I haven't tried any larger than 1b models on this laptop, and I don't think it's worth the time. But testing out three such models, gemma3:1b, llama3.2:1b, and deepseek-r1:1.5b, all yield similar performance. In each case, the LLMs are using a 4k context length, and I don't think I'd risk trying anything higher.

First up is my old favorite:

"How much wood would a woodchuck chuck if a woodchuck could chuck wood?"

Both Gemma 3 and Llama 3.2 churned out a short, fairly quick response and recorded just under 10 tokens per second. Deepseek r1, by comparison, has reasoning, so it'll run through its thought process first, then give you an answer, and was a little behind at just under 8 tokens per second.

But, while not what you'd call fast, it's usable. All three are still churning out responses significantly faster than I could type (and definitely faster than I can think). These were all with a 4k context length.

The second test was a bit meatier. I asked all three models to generate a simple PowerShell script to fetch the raw content of text files from a GitHub repository, and to ask questions to help make sure I was happy with the response, to build the best possible script.

Note, I haven't actually validated whether the output worked. In this instance, I'm purely interested in how well (and fast) the models are able to work the problem.

Gemma 3 gave an extremely detailed output explaining each part of the script, asking questions as directed to tailor the script, and did it all at just under 9 tokens per second. DeepSeek r1 with reasoning operated a little slower, again, at 7.5 tokens per second, but did not ask questions. Llama 3.2 gave an output of similar quality to Gemma 3 at a rate of just under 9 tokens per second.

Oh, but I didn't mention yet, this was all on battery power with a balanced power plan. When connected to external power, all three models essentially doubled their tokens per second and took about half as long to complete the task.

I think that's the more interesting point, here. You could be out and about with a laptop on battery and still getting stuff done. At home or in the office, hooked up to power, even older hardware can be fairly capable.

This was all done purely using the CPU and RAM, too. The laptop harvests a couple of GB of RAM for the iGPU to use, but even so, it's not supported by Ollama. Doing a quick ollama ps shows 100% on the CPU.

These are small LLMs, but the truth is that you can play about with them, integrate them into your workflow, learn some skills, all without breaking the bank on new, crazy powerful hardware.

You don't have to look too far on YouTube to find creators running AI on a Raspberry Pi and home servers made up of older and (now) cheaper hardware. Even with an old, mid-tier laptop, you can probably at least get started.

It's not ChatGPT, but it's something. Even an old PC can be an AI PC.

Recently, I've been playing around with Ollama as a way to use AI LLMs locally on my Windows 11 PC. Beyond educating myself, there are some good reasons to run local AI over relying on the likes of ChatGPT and Copilot.

Education is the big thing, though, because I'm always curious and always looking to expand my knowledge base on how new technology like this works. And how I can make it work for me.

This past weekend, though, I certainly educated myself on one aspect: performance. Just last week, Ollama launched a new GUI application that makes it easier than ever to use it with LLMs, without needing third-party add-ons or diving into the terminal.

The terminal is still a useful place to go, though, and it's how it dawned on me that the LLMs I've been using on my machine haven't been working as well as they could have been.

It's all about the context length.

Context length is a key factor in performance

So, what is context length, and how does it affect the performance of AI models? The context length (also known as the context window) defines the maximum amount of information a model can process in one interaction.

Simply put, a longer context length allows the model to 'remember' more information from a conversation, process larger documents, and, in turn, improve overall accuracy through longer sessions.

The trade-off is that a longer context length needs more horsepower and results in slower response times. Context is measured in tokens, which is a better label for snippets of text. So the more text, the more resources needed to process it.

Ollama can go up to a 128k context window, but if you don't have the necessary hardware to run support, your machine can slow to a crawl. As I found out, trying to get gpt-oss:20b to complete a test designed for 10 and 11-year-old children.

In this example, a shorter context length didn't provide the model enough to process the whole test properly, while a larger one did, at the detriment of performance.

ChatGPT has a context length of over 100,000 tokens, but it has the benefit of OpenAI's massive servers behind it.

Keeping the right context length for the right tasks is crucial for performance

It's a bit of a pain to have to keep switching the context length, but doing so will make sure you're getting a lot more from your models. This is something I haven't been keeping tabs on, but once the lightbulb lit up, I've been having a significantly better time.

It dawned on me while I was poking around in the terminal with some Ollama settings to see just how fast a model was running. I hadn't realized that gpt:oss20b was set to 64k context length, and was surprised at getting a 9 token per second eval rate to a simple question.

"How much wood would a woodchuck chuck if a woodchuck could chuck wood?"

It also wasn't using any of my GPU's VRAM at this length, seemingly refusing to load it into the memory, and was running entirely using the CPU and RAM. By reducing to 8k, the same question generated a 43 tokens per second response. Dropping to 4k doubled this to 86 tokens per second.

The added benefit (with this model at least) here is that the GPU was being fully utilized at 4k, and 93% used at 8k.

The key thing here is that unless you're actually dealing with large amounts of data or very long conversations, setting your context length lower will yield far better performance.

It's all about balancing what you're trying to get from a session against a context length that can handle it with the best efficiency.

How to change Ollama context length and save it for future use

There are two ways you can change the context length in Ollama, one in the terminal and one in the new GUI application.

The latter is the simplest. You simply go into settings and move the slider between 4k and 128k. The downside is these are fixed intervals; you can't choose anything in between.

But, if you use the terminal, the Ollama CLI will give you more freedom over choosing the exact context length you may want, as well as the ability to save a version of the model with this context length attached.

Why bother with this? Well, firstly, it'll give you a model with that length permanently attached; you don't need to keep changing it. So it's convenient. But it also means you can have multiple 'versions' of a model with different context lengths for different use cases.

To do this, open up a PowerShell window and run the model you want to use with this command:

ollama run 

Once the model is running, and you're inside its CLI environment, change the context length following this template. Here, I'm applying an 8k context length.

/set parameter num_ctx 8192

The number at the end is where you decide how long you want it to be. You'll now use that context length, but if you want to save a version you can launch, next enter this command:

/save  

Now you can launch this saved model either through the CLI or the GUI app, or use it in any other integrations you have using Ollama on your PC.

The downside is that the more versions you make, the more storage you use. But it's a convenient way if you have the storage available to avoid having to think about changing context length.

You can have one set low for best performance when you're doing smaller tasks, one set higher for more intensive workloads, whatever you want, really.

How to check performance of a model in Ollama

I'll close by showing how you can see the split of CPU and GPU usage for a model in Ollama, as well as how many tokens per second a model is spitting out.

I recommend playing around with this to find your own sweet spot based on the model you're using and the hardware at your disposal.

For now, if you only have Ollama installed and no third-party tools, you'll need to use the terminal. When running a model, add the --verbose tag to the command. Example:

ollama run gemma3:12b --verbose

What this does is generate a report after the response detailing a number of performance metrics. The easiest one to look at is the last eval rate, and the higher the tokens per second, the faster the performance.

If you want to see how much spread across CPU and GPU the model is using, you'll need to back out of it using the /bye command first. Then type ollama ps and you'll get a breakdown of CPU and GPU percentage. In an ideal world, you want the GPU percentage to be 100% or as close to it as possible.

As an example, I have to go below an 8k context length with gpt-oss:20b on my system with an RTX 5080 if I want to use 100% GPU. At 8k, it's 'only' 93%, but that's still perfectly fine.

Hopefully, this helps some folks out there, especially newcomers to Ollama, since I was definitely leaving performance on the table. With a beefy GPU, I simply assumed all I had to do was run a thing, and it'd magically just do magic.

Even with an RTX 5080, though, I've needed to tweak my expectations and my context length to get the best performance. Generally, I'm not (yet) feeding these models massive documents, so I don't need a massive context length, and I certainly don't need one (and neither do you) for shorter queries.

I'm happy to make mistakes so you don't have to!

It's fair to say that for many when they hear AI their first thought is "ChatGPT." Or possibly Copilot, Google Gemini, Perplexity, you know, the online chatbots that dominate the headlines. There's a good reason for this, but AI goes far beyond that.

Whether it's a Copilot+ PC, editing video, transcribing audio, running a local LLM, or even just making your next Microsoft Teams meeting better, there's a ton of use cases for AI that don't require these cloud-based tools.

So, here are five reasons I've thought of as to why you'd want to use local AI over relying on the online options.

But first, a caveat

Before getting into it, I do need to highlight the elephant in the room; hardware. If you don't have suitable hardware, then the unfortunate truth is that you can't just dive in and test drive one of the latest local LLMs, for example.

Copilot+ PCs have a raft of local features you can use, in part, because of the NPU. Not all AI requires an NPU, but the processing power has to come from somewhere. Before attempting to use any local AI tool, be sure to make yourself familiar with the requirements.

1. You don't have to be online

This is the obvious one. Local AI runs on your PC. ChatGPT and Copilot require a constant web connection to operate, even though you have the Copilot app built into Windows 11.

Sure, connectivity is better in 2025 than it's ever been, you can even get Wi-Fi on a plane, but it's still not ubiquitous. Without a connection, you cannot use these tools. By contrast, you can use the OpenAI gpt-oss:20b LLM on your local machine completely offline. Sure, it's not necessarily as fast, especially considering GPT-5 just launched, and this model is based on GPT-4, but you can use it any time, any place.

This also applies to other tools, such as image generation. You can run Stable Diffusion offline on your PC, whereas to get an image from an online tool you need to be, you guessed it, online. The AI tools in the DaVinci Resolve video editor work offline, leveraging your local machine.

Local AI is therefore entirely portable, and ultimately gives you better ownership. And you're not at the mercy of server capacity and stability, limitations, or terms of service. You're also not at the mercy of these companies changing their models and losing access to older ones you may prefer. This is a current point of contention for many with the switch to GPT-5.

2. Better privacy controls offline

This is an extension of the first point, but important enough to highlight on its own. When you're connecting to an online tool, you're sharing data with a big computer in the cloud. As we've seen recently, albeit now reversed, ChatGPT sessions were being scraped into Google Search results under certain conditions.

You simply don't have the control when you're using an online AI tool versus using one local to your machine. Local AI means your data never leaves your machine, which is particularly important if you handle confidential or sensitive information, where security and privacy is paramount.

While ChatGPT, for example, has an incognito mode, the data still leaves your machine. Local AI keeps it all offline. It's also much easier to comply with any data sovereignty regulations, or compliance with regional data protection rules.

It should be remembered, though, that if you happen to push an LLM from your machine back up to somewhere like Ollama, you will be sharing whatever changes you've made. Likewise, if you were to enable web search on a local model, such as gpt-oss:20b or 120b, you will also be losing a little on total privacy.

3. Cost and environmental impact

To run massive LLMs, you need a massive amount of energy. That's as true at home as it is using ChatGPT, but it's easier to control both your costs and your environmental impact at home.

ChatGPT has a free tier, but it isn't really free. A massive server somewhere is processing your sessions, using enormous amounts of power, and that has an environmental cost. Energy use for AI and its impact on the environment will continue to be an ongoing issue to solve.

By contrast, when you're running an LLM locally, you're in control. In an ideal world I'd have a home with a roof full of solar panels, filling up a giant battery, that would help supply power for my various PCs and gaming devices. I don't have that, but I could. It's only an example, but it makes the point.

The cost impact is easier to visualize. The free tiers of online AI tools are good, but you never get the best. Why else does OpenAI, Microsoft, and Google, all have paid tiers that give you more? ChatGPT Pro is a whopping $200 a month. That's $2,400 a year just to access its best tier. If you're accessing something like the OpenAI API, you're paying for it based on how much you use.

By contrast, you could run an LLM on your existing gaming PC. Like I do. I'm fortunate enough that I have a rig that contains an RTX 5080 with 16GB of VRAM, but that means when I'm not gaming, I can use the same graphics card for AI with a free, open-source LLM. If you have the hardware, why not use it over paying more money?!

4. Integrating LLMs with your workflow

This is one that I'm still only dabbling with, entirely because I'm a coding noob. But with Ollama on my PC and an open-source LLM, I can integrate these with VS Code on Windows 11 and have my own AI coding assistant. All powered locally.

There's also some cross over with the other points on this list. GitHub Copilot has a free tier, but it's limited. To get the best, you have to pay. And, to use it, or regular Copilot, ChatGPT, or Gemini, you need to be online. Running a local LLM gets around all of this, while opening up the possibility of better implementing it into your workflow.

It's the same with non-chatbot related AI tools, too. While I've been a critic of Copilot+ not really being good enough yet to justify the hype, its whole purpose is leveraging your PC to integrate AI into your daily workflow.

Local AI also gives you more freedom over exactly what you're using for your needs. Some LLMs will be better for coding than others, for example. Using an online chatbot, you're using the models they're presenting to you, not one that has been fine-tuned for a more specific purpose. But with a local LLM, you also have the ability to fine tune the model yourself.

Ultimately, with local tools you can build your own workflow, tailored to your specific needs.

5. Education

I'm not talking about school-based education here, I'm talking about teaching yourself some new skills. You can learn far more about AI and how it works, as well as how it can work for you, from the comfort of your own hardware.

ChatGPT has that 'magic' about it. It can do all these amazing things just by typing some words into a box inside an app or a web browser. It is fantastic, no doubt, but there's a lot to be said for learning more about how the underlying technology works. The hardware and resources it needs, building your own AI server, fine-tuning an open-source LLM.

AI is very much here to stay, and you could do a lot worse than setting up your own playground to learn more about it in. Whether you're a hobbyist or a professional, using these tools locally gives you the freedom to experiment. To use your own data, and to do it all without relying on a single model, or locking yourself into a single company's cloud, or subscription.

There are drawbacks, of course. Unless you have an absolutely monstrous hardware setup, performance will be one. You can easily run smaller LLMs locally and get great performance, such as Gemma, Llama, or Mistral. But the largest open-source models, such as OpenAI's new gpt-oss:120b, cannot work properly even on something like today's best gaming PCs.

Even gpt-oss:20b will be slower (thanks partly to its reasoning capabilities) than using ChatGPT on OpenAI's mega-servers.

You also don't get all of the latest and greatest models, such as GPT-5, right away to use at home. There are exceptions, such as Llama 4, which you can download yourself, but you'll need a lot of hardware to run it until smaller versions are produced. Older models have older knowledge cutoff dates, too.

But despite all this, there are plenty of compelling reasons to try local AI over relying on the online alternatives. Ultimately, if you have some hardware that can do it, why not give it a try?

I've been tinkering about with Ollama in recent months, as it's probably one of the best ways normal folks such as myself can interact with AI LLMs on our local machines.

Installing and using Ollama has never been particularly difficult, but it has required living inside a terminal window. Until now.

Ollama now has a shiny GUI for Windows in the form of an official app. It's installed alongside the CLI components now when you set it up, and it removes much of the need to go into a terminal to get your LLM on.

If that's more your thing, you can still download a pure CLI version of Ollama from its GitHub repository.

Installing both portions is as simple as downloading Ollama from the website and running the installer. By default, it'll run in the background, accessible via the system tray or through the terminal. Or you can now launch the app from there or through the Start Menu.

It's a fairly simple experience and familiar if you've ever used basically any AI chatbot. Type in the box, Ollama will respond using whichever of the models you designate.

I'm assuming you're supposed to be able to click the icon next to a model you don't have installed to pull it to your machine, but it doesn't seem to do anything right now. You can search for models from the dropdown, though, to find them.

The workaround I found is to select the model, then try to send a message to it, which then prompts the Ollama app to pull it first. You can of course still go into the terminal and pull models this way, and they'll appear in the app.

Settings are fairly basic, but one of them allows you to change the size of the context window. If you're dealing with large documents, for example, increase the context length with the slider, at the cost of needing more system memory to do so. Which is understandable.

The new Ollama app has some nifty features, though, and the blog post has a couple of videos showing them in action. With multimodal support you can drop images into the app and send them to models that support interpreting them, such as Gemma 3.

Ollama can also interpret code files you drop in, with an example provided where the app is asked to produce a Markdown document explaining the code and how to use it.

Being able to drag and drop files into Ollama is definitely more convenient than trying to interact with them using the CLI. You don't need to know file locations, for one.

You'll still, for now at least, potentially want to use the CLI for some of the other features available in Ollama, though. For example, you can't push a model using the app, only the CLI. Likewise for creating models, listing running models, copying and removing.

For general use, though, I think this is a great step forward. Granted, a lot of the folks that have been using local LLMs to this point are probably comfortable inside a terminal window. And using the terminal and being happy using it is certainly a skill I'd always recommend.

But having a user-friendly GUI makes it more accessible to everyone. There have been third-party tools, such as Open WebUI that have done this before, but being built into the package just makes it easier.

What remains, though, is that you'll still need a PC with enough horsepower to use these models. Ollama doesn't yet support NPUs, so you're reliant on the rest of your system.

It's recommended to have at least an 11th Gen Intel CPU or a Zen 4-based AMD CPU with support for AVX-512 and ideally 16GB of RAM. 8GB is the bare minimum Ollama recommends for the 7B models, and there are smaller models you can use than this that will use even less resources.

You don't need a GPU, but if you have one, you'll naturally get better performance. If you want to use some of the largest models, it's definitely something you'll want. And the more VRAM, the better.

Running a local LLM, though, is something worth trying at least once if your PC can handle it. You're not reliant on a web connection, so you can use it offline, and you're not feeding the great AI cloud computer in the sky, either.

You don't have web search out of the box, but there are ways and means if you absolutely need this. But you'll need something other than the Ollama app to do that. But download it, give it a try, and you might find your new favorite AI chatbot.

The most common interactions most of us will have with AI right now is through a cloud-based tool such as ChatGPT or Copilot. Those tools require a connection to the internet to use, but the trade-off there is that they're usable on basically any device.

But not everyone wants to rely on the cloud for their AI use, especially developers. For that, you want the LLM to be running locally on your machine. Which is where Ollama comes in.

Ollama is an inferencing tool that allows you to run a wide range of LLMs natively on your PC. It's not the only way you can achieve this, but it is one of the simplest.

Once up and running, there's a lot you can do with Ollama and the LLMs you're using through it, but the first stage is getting set up. So let's walk you through it.

What you need to run Ollama

Running Ollama itself isn't much of a drag and can be done on a wide range of hardware. It's compatible with Windows 11, macOS, and Linux, and you can even use it through your Linux distros inside Windows 11 via WSL.

Where you'll need some beefier hardware is actually running the LLMs. The bigger the model, the more horsepower you need. The models will need a GPU to run, right now, it hasn't been optimized to run on an NPU in a new Copilot+ PC.

Essentially, if you have a fairly modern PC with at least 8GB of RAM and a dedicated GPU, you should be able to get some mileage using Ollama.

If you don't have a dedicated GPU then Ollama can still be used, but it will leverage your CPU and RAM, instead. This means you don't get the same performance as on a dedicated GPU, but it can still work.

The same rules apply. Memory is king when using a local LLM. If you have lots of it you can run larger models. But even on a laptop CPU you can use 1b, 3b and 4b models with success.

How to install Ollama on Windows 11

Installing Ollama on Windows 11 is as simple as downloading the installer from the website (or GitHub repo) and installing it.

That's literally all there is to it.

Once installed and subsequently opened, you won't see anything on your desktop. It runs entirely in the background, but you'll see its icon in your taskbar.

You can also check it's running properly by navigating to localhost:11434 in your web browser.

Installing and running your first LLM on Ollama

Ollama now has both a CLI (command line interface) and GUI interface. But it's still handy to be comfortable in either PowerShell or WSL to get the most from it, as the GUI app is still very basic.

There are more advanced third-party options you can use, too. The easiest is the Page Assist browser extension, which launches a full web app to use with Ollama. A more popular, but also slightly harder to install option, is OpenWebUI.

Two main commands in the terminal you need to know are as follows:

ollama pull  ollama run  

If you ask Ollama to run an LLM you don't currently have installed, it's smart enough to go and grab it first, then run. All you need to know is the correct name for the LLM you want to install, and those are readily available on the Ollama website.

For example, if you wanted to install the 1 billion parameter Google Gemma 3 LLM, you would enter the following:

ollama pull gemma3:1b 

By adding :1b after the name, you're specifying that you want the 1 billion parameter model. If you wanted the 4 billion parameter model, you would change it to :4b.

By running the models in your terminal it'll open up a familiar feeling chatbot experience allowing you to type in your prompts and get your responses. All running locally on your machine. If you want to see how fast it's performing, add the --verbose tag when you launch it.

To leave the model and go back to PowerShell, you simply need to type /bye and it'll exit.

Alternatively, you can launch the Ollama GUI and just get going. You can still pull a new model here by searching for it in the dropdown box, typing a simple message such as "hi" and it will pull the model for you.

Page Assist and OpenWebUI have more advanced model management tools, allowing you to search for, pull, and remove models with ease.

That covers the absolute basics of getting Ollama set up on your PC to use the host of LLMs available for it. The core experience is extremely simple and user-friendly, and requires almost no technical knowledge to set up. If I can do it, you can too!