Cloud AI Has the Same Benefits and Problems as Cloud Hosting

My first real exposure to LLMs was through work — I was chosen for the GitHub Copilot pilot group. Around the same time, my existing JetBrains All Products Pack subscription happened to include AI usage that basically felt unlimited. At that point, LLMs weren't really doing tool calling — it was just a "smart" chat bot. I found the limitations on both sides quickly: only useful in very specific, well-documented scenarios with limited context windows. In other words, only useful for very small parts of code at a time.

By Fall 2025, we were exposed to LLM agents in GitHub Copilot for application modernization. My employer wouldn't whitelist anything past GPT-4.1/4o, so basically useless. My JetBrains AI allowance vanished quickly as the company switched to token-based billing. February 2026 changed things. We finally got access to Claude Sonnet 4.5 at work, and I subscribed to a personal GitHub Copilot license. I also took a one-month stab at Claude Code's $20 plan — burn through that, then fall back to Copilot prompts. In an agentic loop, that meant a solid amount of usage for $40/month. I was willing to do that for a few months at the time.

There was a lot of advice being thrown around, and a lot of it was wrong. Too many influencers aren't engineers and haven't deep-dived into the technology. The result: noise. Lots of it. Few people actually know what they're doing with LLM agents, or even know what to do with them. Even Claude Code — the gold standard at the time — was a buggy mess. It would lose context, hallucinate file paths, and occasionally delete half your working directory because it misread its own output. I have the git log to prove it.

LLM coding agents are impressive. They have a lack of vision and knowledge that they can make up for by basically running spikes and verification tasks in seconds, letting them gain some idea of vision back. Sometimes they get stubborn, and you have to help them. A good engineer with a good BS filter can basically ignore the LLM's BS, get to the planning, execution, and verification, and really extend their capabilities. Without the BS filter, you take what the AI says literally. Without being a good engineer, it's hard to know when the LLM is making bad choices. It's much like a calculator in math. If you fully understand a concept, a calculator accelerates you. If you don't understand the concept, you may finish with a calculator, but you'll still walk out confused and not exactly confident.

A single prompt could do a whole sprint's worth of work. Granted, it took some of the fun away. Many developers began lamenting that LLMs were taking their favorite part of the job away — the coding. Most engineers began as program hackers, writing small programs that did small things, and feeling the joy of making something work. Then we get paid to do it, and we start pairing that with customers and their hard business rules, paperwork, oversight, real financial risks, career growth concerns, etc. Most developers dislike the work part of the job, but love the programming part. I'm no different — I don't enjoy the routine paperwork, the scrum ceremonies (even though I'm a scrum master), the very difficult and conflicting business rules, etc. But I wasn't going to let LLMs ruin what fun I do have with my career. Interacting with LLMs themselves isn't too different than writing a program: you have a goal, and you're writing a form of code to get that goal done. It's just that the "compiler" — in a manner of speaking — is non-deterministic but also very powerful.

So I found a new game. I got into tweaking agent instructions to see how far a single prompt could go without losing quality. Not "prompt engineering" — more like being as lazy as possible. My goal has been to let me be as vague and short as possible in a conversation, and let the LLM figure out what I want based on the core instructions I've built up. I built an agentic workflow that I brought into work and showed to my whole team — with relatively good success.

But then Claude Code's usage limits shrank, and GitHub Copilot started rate limiting. I cancelled Claude Code, bumped GitHub Copilot to Pro+ ($40/month). That removed rate limiting for about a month, then Microsoft announced they were dropping the prompt billing strategy entirely and would charge API rates for token usage. A single prompt could do half a day's work, burning thousands of tokens. With the new pricing, my personal GitHub Copilot bill was going to be $300+/month if I continued. Work had the same problem, but they didn't seem to care enough to do anything about it. Here's the thing: cloud AI has the same benefits and problems as cloud hosting. Someone else manages the infrastructure. You pay for what you use. It just works — until it doesn't. Until they change pricing. Until they limit your usage. Until they decide you don't have full rights to what you're doing with it. Until the guardrails (Mythos, Fable) tell you what you can and can't ask. You trade convenience for control. You take on the operational burden. You make it stable yourself.

As an engineer, that trade-off is familiar. Back at Smart Furniture, I was the main developer and server admin — effectively the webmaster for a multi-million dollar e-commerce business. (Granted, they went bankrupt after I left, but the business they were in was hard.) Self-hosting was the norm. Most businesses have "self-hosted" their technology for decades, even if it wasn't on-site — they still managed their DNS, their router, their web server, their database. I brought that into my personal life too. Cloud AI is good enough for most people, and I get why. But I've always disliked the lack of ownership, accessibility, and freedom that apps and cloud services impose — especially on the "smart" side of things. I occasionally use apps (probably YouTube the most), but everything-is-an-app, everything-is-cloud, everything-is-someone-else's-problem is a model I've tried to opt out of. Cloud AI fits right next to social media as a consumption product. It just doesn't pair well with me.

Beyond convenience, there's a risk question on both sides. Just because a model is downloadable and free doesn't mean it won't go stale — newer versions may never get published. That's a risk with self-hosting. But I do swear there are days when the AI from GitHub Copilot at work is worse than the one from yesterday, even with the same name supposedly powering it. So stability and quality are not guaranteed on either side — you just get to control which risks you take.

When I signed up for GitHub Copilot at home, I did some basic math. At 300 prompts/month for $20/month, it'd take ~6 years of subscriptions to zero-out a $1500 graphics card — and that's without counting electricity. Even on the $40/month sub, running multiple agents at a time, it'd still take 3 years. NVIDIA had nothing new at ~$1500 that was worth getting for running an LLM agent. The only game in town seemed to be the RTX 5090 at ~$4000 — or 8+ years vs the subscription. And I'd probably need a bigger case and a better PSU. My decision at the time was it wasn't worth it. In 3-8 years, the hardware and the AI capabilities would be completely different. Sinking $4000 into a graphics card for "hobby" stuff would've been a horrible financial decision, and a selfish one at that.

Rough breakdown:

At that point, local didn't make sense. But I was also looking at cards, just in case. Rough GPU options I was seeing:

The 12–24GB cards were obviously too small — 32GB was the minimum that made sense. But at the time, $4,000 for the 5090 wasn't a call I could justify. The R9700 wasn't on my radar yet. I only learned about it after I bought the 3090, which made things both better and more annoying. I live under the TVA area, so rates are pretty low, so power usage wasn't a material cost to truly factor in unless I bought a serious card and it used a serious amount of power constantly. I didn't actually track it carefully at first, but even being generous with the estimate, I wasn't concerned about that.

Then the pricing shift hit. It'd now take 3-4 months of subscriptions to pay off a graphics card. The numbers flipped in about two weeks' time — GitHub Copilot was going away as a viable personal tool, OpenAI and Anthropic made it clear you don't have full rights to their AI, and Qwen3.6 27B/35B-MOE had released, bringing good performance to consumer hardware. I didn't know about the RX 9700 yet. At that point, the "conventional wisdom" on the internet pointed to the RTX 3090 as the card to get. So that's what I ordered — same day, basically. The 9700, the ROCm improvements, the $33/GB math — all of that came later, after I hit the 3090's limitations and started looking harder.

I want to own my models and my data — no one revoking my access because I asked the wrong question or violated a policy I didn't know about. I want to run whatever model I want, at whatever quant I want, with whatever context window I want — no whitelists, no rate limits, no "our lawyers said no." My prompts, my code, my conversations about my life — I want them to stay in my house. No telemetry, no training on my data, no third-party access. The trade-off is I have to make it stable myself. And that's a problem I'm willing to solve.

So, on the same day GHCP announced their usage-based billing changes, I ordered the 3090. I made my decision. Part 2 covers the hardware side — the 3090, the fan noise, the VRAM wall, the R9700, Docker, Vulkan, and where things stand today. I'll link it when it's done.