AI Hardware Buyer's Guide

Read this first — the 30-second decision

You want local AI if any of these are true:

• You handle client data, employee records, financials, or anything regulated (Quebec Law 25, HIPAA, PIPEDA)
• Your legal/compliance team has said "we can't use ChatGPT for that"
• You want predictable monthly cost instead of a per-token bill that grows with usage
• You want AI to keep working when the internet doesn't, or when the SaaS is down

You probably don't need this if:

You're a sole user with no privacy constraints and Claude Pro / ChatGPT Plus already covers your needs. The cheapest setup below is ~$5,400 CAD — that's 30+ months of a $140 cloud subscription.

What changes once you have local AI:

Clean French/English drafting, document Q&A, code review, transcript analysis, basic agent workflows — all of it private, all of it instant, all of it included once you've bought the hardware.

Sizing for a 3-4 year purchase — storage, RAM, chip

A laptop bought today should still be the right tool in 2029-2030. Three axes matter: storage (how much disk to buy), RAM (the real ceiling on what models you can run), and chip class (memory bandwidth = token generation speed). Buy too small and you'll feel pain by year 2; buy too big and you waste money on capacity you won't use. Here's how to think about it.

The shopping list — 7 recommended setups

Each setup is a complete order: machine + accessories + tax + total. Pick the one that matches your situation, hand the list to your IT person, or order it yourself. Prices are CAD, snapshot 2026-05-03, including QC sales tax (TPS+TVQ 14.975%).

Pick this if: Solo mobile professional. Lawyer, accountant, agent, consultant. Light-to-medium AI use a few times a day. You travel.

Shopping list

Honest trade-offs

• 307 GB/s memory bandwidth — about 2× the M5 base, but half of M5 Max. Token generation scales nearly linearly with this number (see callout below).
• 64 GB RAM caps you at 30B-class models comfortably. Llama 70B 4-bit (~40 GB) won't fit alongside macOS + apps.
• Battery 16+ hours on non-AI work, ~6-8 hours when running an LLM continuously.

Pick this if: Solo desk-first professional. Want a bigger screen built-in, occasional travel. Same AI capability as Setup #1.

Shopping list

Honest trade-offs

• Larger 16.2" screen + better sustained thermals than 14" — but 700g heavier (2.16 kg vs 1.55 kg).
• Same M5 Pro chip as Setup #1 — quality and AI speed identical. You're paying for screen + thermal headroom.
• Better cooling matters less here than in the Max chassis (Pro chip stays within thermal budget anyway).

Pick this if: Solo road-warrior who needs Max performance in a smaller body. You travel a lot, you want to run every model on this page, and you accept the thermal compromise of the 14" chassis.

Shopping list

Honest trade-offs

• 614 GB/s bandwidth + 128 GB unified RAM — runs every model on this page including Llama 3.3 70B (~40 GB) and Qwen3.6 27B reasoning (~35 GB) with room to spare.
• 14" Max throttles 15-25% under sustained load (Notebookcheck, Digital Trends measurements). Spike to 96 W then drops to 42 W. Fine for bursts, painful for all-day inference.
• SSD reaches 100°C under sustained AI workloads on this chassis. Apple's cooling can't keep up with the Max chip in 14" form factor.
• Same chip as Setup #4 (16" Max 128 GB) but 700g lighter and more portable. If you grind on AI for hours daily, go 16". If your AI use is bursty + you live on the road, this wins.

Pick this if: Solo cockpit. Heavy AI all day. You want every model on this page to run, no thermal compromise, no swap, no waiting.

Shopping list

Honest trade-offs

• This is the setup the measurements on this page were taken on (Setup #4 was 14" 128/4TB but the 16" body is the same chip with proper thermals — performance numbers transfer identically or improve).
• 16" body sustains 62 W indefinitely vs 14"'s 42 W after throttle. 15-25% faster on long workloads.
• Heavier (2.16 kg) and bigger footprint. If you live on planes and coffee shops, Setup #3 is more portable.
• 128 GB unified memory means zero swap even with multiple models loaded + active dev work.

Pick this if: Heavy AI user with stable home internet (>99% uptime). You want Studio-grade quality at desk + laptop autonomy on the road. Best long-term value if you can wait for Studio M5 Ultra (rumored June or Oct 2026).

Shopping list

Honest trade-offs

• Architecture: laptop = thin client when home (queries Studio over Tailscale, sub-ms LAN latency), full local stack when offline. Best of both worlds.
• Studio runs 70B-class models at full bandwidth (820 GB/s on M5 Ultra). Llama 3.3 70B, Qwen3.6 35B, Gemma 4 31B all fast enough for real-time chat.
• Laptop sized for autonomy, not for being primary AI rig — 64 GB Pro is plenty for the rare offline use case (Gemma 4 26B 8-bit fits comfortably).
• M5 Ultra Studio is rumored, not released. If you need this today, substitute M3 Ultra Mac Studio (~$5,500-9,000 CAD depending on RAM) and upgrade later.

Pick this if: Maximum tokens-per-dollar. You're comfortable with Linux + vLLM serving stack. You don't need Mac apps. Best raw inference speed in this guide.

Shopping list

Honest trade-offs

• 128 GB total VRAM across 4 cards (32 GB each at 600 GB/s per card vs 614 GB/s shared on Mac M5 Max).
• ~800 W total power draw under load (4× 250 W GPUs + 350 W CPU + overhead). Verify circuit + cooling before ordering.
• Non-ECC UDIMM choice: spec uses Kingston Fury Beast 128 GB (consumer DDR5, $2,620 in stock). ECC RDIMM (Kingston Renegade Pro 128 GB) was $5,850 + sold out nationally as of 2026-05-04. ECC matters for training; for inference workloads silent bit-flips produce wrong tokens, not data corruption — non-ECC is acceptable. If your subvention/policy requires ECC, swap the line and budget +$3,000-4,000 + lead time.
• Audible under load — multiple GPU fans + tower cooler. Not a coffee-shop or quiet-office machine.
• Intel oneAPI / IPEX-LLM ecosystem is maturing — fewer tutorials than NVIDIA CUDA, expect more Stack Overflow time on driver/quant issues.
• vLLM serving makes it shine for multi-user concurrent inference. Single-user is overkill — Setup #4 is faster for one person.
• Best raw tokens-per-dollar in this guide for serving (BF16 c4: 194 t/s on Qwen3-4B per Intel benchmarks).

Pick this if: Team of 15-50 people sharing private AI. You have rack space and someone comfortable with Linux ops + DIY builds. Scale by adding more nodes.

Shopping list

Honest trade-offs

• This is a DIY build. Mainboards arrive bare — you supply chassis, mount the PSU + cooler, install SSD, cable it. Plan for one full day per node.
• 512 GB total memory across 4 nodes at ~256 GB/s per node. Scales by node count for concurrent users, not for single-query speed.
• ~20% generation drop scaling 2 → 4 nodes (Alex Ziskind benchmarks) — coordination overhead is real but manageable.
• ~600 W total cluster power draw at sustained inference. UPS line-item is sized for this; verify your room circuit can handle it.
• Memory is soldered (LPDDR5x) — no future RAM upgrade per node. Locked-in at 128 GB.
• Linux-only. No Apple ecosystem. Your IT person will own this.
• Best price-per-concurrent-user in this guide. Cheaper per seat than buying everyone a MacBook.

Pick this if: You want NVIDIA-native CUDA tooling (PyTorch, vLLM, TensorRT-LLM) on a desktop. 1 node for solo use, 2-node direct-cabled for fine-tuning + serving up to 405B models.

Shopping list

Honest trade-offs

• 1 node = $6,580 CAD. 2 nodes (the configuration shown above) = ~$13,360 CAD before tax. Drop the second node line for solo use.
• Native CUDA — every PyTorch, vLLM, TensorRT-LLM, ComfyUI, etc. just works. No oneAPI/ROCm hunting like Setup #6 (Intel Arc) requires.
• 2 nodes = direct cable, no switch. ConnectX-7 200 GbE port-to-port via DAC. Switch only needed at 3+ nodes (separate purchase).
• Memory is unified (GB10 Grace Blackwell) — 128 GB per node, ~273 GB/s bandwidth. Per-node bandwidth lower than Mac M5 Max (614 GB/s) — Spark wins on CUDA ecosystem, Mac wins on per-node speed.
• ~240 W per node. Two nodes ≈ 480 W. Quiet office-friendly desktop form factor.
• ~17 GB/s NCCL throughput over 200 GbE (real-world, per NVIDIA forum) — fine for inference, the bottleneck for training larger-than-2-node setups.
• NVIDIA pricing volatile — Founders Edition jumped 18% in 2025 due to memory shortages. Check current pricing at order time.
• Best fit for: AI researchers, devs who'll exploit CUDA libraries, teams with PyTorch fine-tuning workflows.

Pick this if: You need single-card high-VRAM CUDA for AI inference, training, or rendering. Pick the VRAM tier that matches your largest expected model.

Shopping list

Honest trade-offs

• Pricing on the GPU lines: totals shown only count the 48 GB option. The 72 GB and 96 GB lines display $0 to avoid double-counting — substitute the line you actually want when ordering. With non-ECC RAM build: 72 GB GPU build total ≈ $15,800 CAD pre-tax / $18,200 CAD with QC tax. 96 GB GPU build total ≈ $18,950 CAD pre-tax / $21,790 CAD with QC tax.
• 96 GB GDDR7 ECC is the most VRAM you can put in a desktop today. Runs Llama 3 70B at FP16, Qwen 2.5 72B, Mistral Large at high quants without offloading.
• CUDA-native — TensorRT-LLM gives ~2-3× faster inference vs vLLM bf16 on the same card. Industry-standard tooling.
• ~600 W per GPU under load. 1500 W PSU is the realistic minimum; 1600 W gives headroom for transient spikes.
• Non-ECC system RAM: spec uses Kingston Fury Beast 128 GB UDIMM ($2,620 in stock at Canada Computers). ECC RDIMM was $5,850 + sold out nationally as of 2026-05-04. The GPU's 96 GB GDDR7 is ECC; system RAM is non-ECC. For inference workloads this is fine. For training or grant requirements that mandate end-to-end ECC, swap to RDIMM and budget +$3,000-4,000 + lead time.
• Single-card design — for multi-GPU (2× or 4× 96 GB), need server platform with PLX switching or ConnectX-7 for NVLink-replacement. Beyond this guide's scope.
• Best fit for: AI fine-tuning, ML research, professional rendering, single-user max-VRAM workloads.

The proof — real measurements 🟢

We didn't make these numbers up. The "AI quality vs cloud" % on every setup card above traces back to this eval: 7 prompts × 6 models = 42 calls, run on a real M5 Max 128 GB, scored 0-5 by hand against a clear rubric. Sonnet 4.6 (cloud) is the reference at 100%; everything else runs on-device. Setups #3-#5 use the same Apple silicon as the test bench. Setups #6-#7 use comparable hardware (Intel Arc, AMD Ryzen AI Max+) — quality scales with model size and bandwidth, not vendor branding.

Test bench: 14" MacBook Pro M5 Max 128 GB / 4 TB / macOS 26 / LM Studio 0.4.12 with MLX runtime app-mlx-generate-mac26-arm64@22. Real audio transcript from a French-speaking real-estate professional, PII-stripped to 615 words. Total OpenRouter cost for the Sonnet 4.6 baseline: under $0.50.

Quality Score Heatmap (0-5 per prompt, max 35)

Latency per Prompt (seconds)

Software stack — the same for all setups

Hardware is half the story. Here's the open-source stack that runs on every setup above. All free unless noted.

• LM Studio (free) — runs the models, manages downloads, provides an OpenAI-compatible API at localhost:1234
• Cherry Studio or LibreChat (free) — chat interface; LibreChat is multi-user and self-hostable, Cherry is single-user and slick
• Tailscale (free for ≤3 users) — secure remote access to your home/office AI from any device
• Vaultwarden / 1Password / Apple Keychain — credential storage for any API keys you use
• Optional: small Proxmox server ($800-1,500 used hardware) for always-on background tasks separate from the main AI box

What you do not need to buy

Defensive section — things that look impressive but won't help you:

• ❌ NVIDIA H100 / A100 datacenter GPUs — $25K+ each, overkill for SMB. Useful for training, not inference at your scale.
• ❌ NVIDIA DGX Spark — $50K+ tier. Only useful if you're serving hundreds of concurrent users.
• ❌ "AI PCs" with Copilot+ branding — marketing label, not capability. The Microsoft NPU isn't compatible with the open-source LLM stack used on this page.
• ❌ Subscription "private AI" SaaS — defeats the purpose. You're back to trusting a third party with your data, just with a different bill.
• ❌ Used Mac Pro towers (Intel-era) — no Apple Silicon means no MLX acceleration. They'll run AI but slowly and inefficiently.

Choosing between setups — quick decision tree

Cluster benchmarks 🟡 — for advanced/team deployments

These numbers help size large multi-user deployments. For SMB and solo use, jump back to Section 2 — the setup cards already incorporate this data.

Methodology & trust

How quality is scored

The 7-prompt eval in Section 4 used a 0-5 rubric per prompt: 0 = no answer / hard fail, 5 = correct, insightful, would ship as-is. Every output was read end-to-end by a human (Artem) and scored against the same rubric. Sonnet 4.6 served as the cloud reference at 35/35.

How latency is measured

Local models: LM Studio's built-in token timer, captured via the OpenAI-compatible /v1/chat/completions endpoint at localhost:1234, averaged across the 7 prompts. Cloud: OpenRouter response timing for the same prompts. All on a single test bench (M5 Max 128 GB / macOS 26 / LM Studio 0.4.12).

Trust tier definitions

• 🟢 Measured — Artem ran this on this hardware. Reproducible from the public test harness on GitHub.
• 🟡 Reproduced — third-party benchmark (Alex Ziskind, Jeff Geerling, Notebookcheck), independently verified or matching multiple sources.
• ⚪ Estimated — extrapolated from architecture or vendor specs. Used sparingly and labeled.

Sources

• Alex Ziskind — Apple's New M5 Max Changes the Local AI Story (M5 bandwidth measurements)
• Notebookcheck — M5 Max 14" thermal throttling review
• Digital Trends — 14" MBP holds back M5 Max
• WCCFTech — M5 Max SSD 100°C under AI workloads
• Jeff Geerling — RDMA over Thunderbolt 5 benchmarks

Reproduce the numbers yourself

The full test harness (prompts, scoring rubric, runner.py) is open-source: github.com/radionokia/homelab/projects/llm-benchmark. Run it on any of the setups above, score the outputs against your own rubric, and you'll get reproducible quality + latency data for your hardware.

Who built this

Artem Kravchenko — AI Automation Consultant + IT Director, 30 years in IT, based in Quebec City. Bilingual (English / French).

I run all seven setups in production: my own work, my homelab, and at client sites. Numbers on this page are what I measure, not what vendors promise. If a recommendation here costs you money you didn't need to spend, I want to hear about it — that feedback makes the next revision better.

Not sure which setup fits your situation? Free consult, no commitment.

Want to see ROI analysis, per-role breakdowns, and KPI scorecard? → Coming soon at /roi