AI Hardware Buyer's Guide

Read this first — the 30-second decision

You want local AI if any of these are true:

• You handle client data, employee records, financials, or anything regulated (Quebec Law 25, HIPAA, PIPEDA)
• Your legal/compliance team has said "we can't use ChatGPT for that"
• You want predictable monthly cost instead of a per-token bill that grows with usage
• You want AI to keep working when the internet doesn't, or when the SaaS is down

You probably don't need this if:

You're a sole user with no privacy constraints and Claude Pro / ChatGPT Plus already covers your needs. The cheapest setup below is ~$5,400 CAD — that's 30+ months of a $140 cloud subscription.

What changes once you have local AI:

Clean French/English drafting, document Q&A, code review, transcript analysis, basic agent workflows — all of it private, all of it instant, all of it included once you've bought the hardware.

The shopping list — 7 recommended setups

Each setup is a complete order: machine + accessories + tax + total. Pick the one that matches your situation, hand the list to your IT person, or order it yourself. Prices are CAD, snapshot 2026-05-03, including QC sales tax (TPS+TVQ 14.975%).

Pick this if: Solo mobile professional. Lawyer, accountant, agent, consultant. Light-to-medium AI use a few times a day. You travel.

Shopping list

Honest trade-offs

• 307 GB/s memory bandwidth — about 2× the M5 base, but half of M5 Max. Token generation scales nearly linearly with this number (see callout below).
• 64 GB RAM caps you at 30B-class models comfortably. Llama 70B 4-bit (~40 GB) won't fit alongside macOS + apps.
• Battery 16+ hours on non-AI work, ~6-8 hours when running an LLM continuously.

Pick this if: Solo desk-first professional. Want a bigger screen built-in, occasional travel. Same AI capability as Setup #1.

Shopping list

Honest trade-offs

• Larger 16.2" screen + better sustained thermals than 14" — but 700g heavier (2.16 kg vs 1.55 kg).
• Same M5 Pro chip as Setup #1 — quality and AI speed identical. You're paying for screen + thermal headroom.
• Better cooling matters less here than in the Max chassis (Pro chip stays within thermal budget anyway).

Pick this if: Solo road-warrior who needs Max performance in a smaller body. You travel a lot, you want to run every model on this page, and you accept the thermal compromise of the 14" chassis.

Shopping list

Honest trade-offs

• 614 GB/s bandwidth + 128 GB unified RAM — runs every model on this page including Llama 3.3 70B (~40 GB) and Qwen3.6 27B reasoning (~35 GB) with room to spare.
• 14" Max throttles 15-25% under sustained load (Notebookcheck, Digital Trends measurements). Spike to 96 W then drops to 42 W. Fine for bursts, painful for all-day inference.
• SSD reaches 100°C under sustained AI workloads on this chassis. Apple's cooling can't keep up with the Max chip in 14" form factor.
• Same chip as Setup #4 (16" Max 128 GB) but 700g lighter and more portable. If you grind on AI for hours daily, go 16". If your AI use is bursty + you live on the road, this wins.

Pick this if: Solo cockpit. Heavy AI all day. You want every model on this page to run, no thermal compromise, no swap, no waiting.

Shopping list

Honest trade-offs

• This is the setup the measurements on this page were taken on (Setup #4 was 14" 128/4TB but the 16" body is the same chip with proper thermals — performance numbers transfer identically or improve).
• 16" body sustains 62 W indefinitely vs 14"'s 42 W after throttle. 15-25% faster on long workloads.
• Heavier (2.16 kg) and bigger footprint. If you live on planes and coffee shops, Setup #3 is more portable.
• 128 GB unified memory means zero swap even with multiple models loaded + active dev work.

Pick this if: Heavy AI user with stable home internet (>99% uptime). You want Studio-grade quality at desk + laptop autonomy on the road. Best long-term value if you can wait for Studio M5 Ultra (rumored June or Oct 2026).

Shopping list

Honest trade-offs

• Architecture: laptop = thin client when home (queries Studio over Tailscale, sub-ms LAN latency), full local stack when offline. Best of both worlds.
• Studio runs 70B-class models at full bandwidth (820 GB/s on M5 Ultra). Llama 3.3 70B, Qwen3.6 35B, Gemma 4 31B all fast enough for real-time chat.
• Laptop sized for autonomy, not for being primary AI rig — 64 GB Pro is plenty for the rare offline use case (Gemma 4 26B 8-bit fits comfortably).
• M5 Ultra Studio is rumored, not released. If you need this today, substitute M3 Ultra Mac Studio (~$5,500-9,000 CAD depending on RAM) and upgrade later.

Pick this if: Maximum tokens-per-dollar. You're comfortable with Linux + vLLM serving stack. You don't need Mac apps. Best raw inference speed in this guide.

Shopping list

Honest trade-offs

• 128 GB total VRAM across 4 cards — same as Setup #4's unified memory but with 600 GB/s per card (vs 614 GB/s shared on Mac).
• ~600 W under load, audible fans, needs proper cooling. Not a coffee-shop machine.
• Intel oneAPI / IPEX-LLM ecosystem still maturing — fewer tutorials than NVIDIA CUDA, more Stack Overflow time required.
• vLLM serving makes it shine for multi-user concurrent inference. Single-user is overkill — Setup #4 is faster for one person.

Pick this if: Team of 15-50 people sharing private AI. You have rack space and someone comfortable with Linux ops. Scale by adding more nodes.

Shopping list

Honest trade-offs

• 512 GB total memory across 4 nodes at ~256 GB/s per node. Scales by node count for concurrent users, not for single-query speed.
• ~20% generation drop scaling 2 → 4 nodes (Alex Ziskind benchmarks) — coordination overhead is real but manageable.
• Linux-only. No Apple ecosystem. Your IT person will own this.
• Best price-per-concurrent-user in this guide. Cheaper per seat than buying everyone a MacBook.

The proof — real measurements 🟢

We didn't make these numbers up. The "AI quality vs cloud" % on every setup card above traces back to this eval: 7 prompts × 6 models = 42 calls, run on a real M5 Max 128 GB, scored 0-5 by hand against a clear rubric. Sonnet 4.6 (cloud) is the reference at 100%; everything else runs on-device. Setups #3-#5 use the same Apple silicon as the test bench. Setups #6-#7 use comparable hardware (Intel Arc, AMD Ryzen AI Max+) — quality scales with model size and bandwidth, not vendor branding.

Test bench: 14" MacBook Pro M5 Max 128 GB / 4 TB / macOS 26 / LM Studio 0.4.12 with MLX runtime app-mlx-generate-mac26-arm64@22. Real audio transcript from a French-speaking real-estate professional, PII-stripped to 615 words. Total OpenRouter cost for the Sonnet 4.6 baseline: under $0.50.

Quality Score Heatmap (0-5 per prompt, max 35)

Latency per Prompt (seconds)

Software stack — the same for all setups

Hardware is half the story. Here's the open-source stack that runs on every setup above. All free unless noted.

• LM Studio (free) — runs the models, manages downloads, provides an OpenAI-compatible API at localhost:1234
• Cherry Studio or LibreChat (free) — chat interface; LibreChat is multi-user and self-hostable, Cherry is single-user and slick
• Tailscale (free for ≤3 users) — secure remote access to your home/office AI from any device
• Vaultwarden / 1Password / Apple Keychain — credential storage for any API keys you use
• Optional: small Proxmox server ($800-1,500 used hardware) for always-on background tasks separate from the main AI box

What you do not need to buy

Defensive section — things that look impressive but won't help you:

• ❌ NVIDIA H100 / A100 datacenter GPUs — $25K+ each, overkill for SMB. Useful for training, not inference at your scale.
• ❌ NVIDIA DGX Spark — $50K+ tier. Only useful if you're serving hundreds of concurrent users.
• ❌ "AI PCs" with Copilot+ branding — marketing label, not capability. The Microsoft NPU isn't compatible with the open-source LLM stack used on this page.
• ❌ Subscription "private AI" SaaS — defeats the purpose. You're back to trusting a third party with your data, just with a different bill.
• ❌ Used Mac Pro towers (Intel-era) — no Apple Silicon means no MLX acceleration. They'll run AI but slowly and inefficiently.

Choosing between setups — quick decision tree

Cluster benchmarks 🟡 — for advanced/team deployments

These numbers help size large multi-user deployments. For SMB and solo use, jump back to Section 2 — the setup cards already incorporate this data.

Methodology & trust

How quality is scored

The 7-prompt eval in Section 4 used a 0-5 rubric per prompt: 0 = no answer / hard fail, 5 = correct, insightful, would ship as-is. Every output was read end-to-end by a human (Artem) and scored against the same rubric. Sonnet 4.6 served as the cloud reference at 35/35.

How latency is measured

Local models: LM Studio's built-in token timer, captured via the OpenAI-compatible /v1/chat/completions endpoint at localhost:1234, averaged across the 7 prompts. Cloud: OpenRouter response timing for the same prompts. All on a single test bench (M5 Max 128 GB / macOS 26 / LM Studio 0.4.12).

Trust tier definitions

• 🟢 Measured — Artem ran this on this hardware. Reproducible from the public test harness on GitHub.
• 🟡 Reproduced — third-party benchmark (Alex Ziskind, Jeff Geerling, Notebookcheck), independently verified or matching multiple sources.
• ⚪ Estimated — extrapolated from architecture or vendor specs. Used sparingly and labeled.

Sources

• Alex Ziskind — Apple's New M5 Max Changes the Local AI Story (M5 bandwidth measurements)
• Notebookcheck — M5 Max 14" thermal throttling review
• Digital Trends — 14" MBP holds back M5 Max
• WCCFTech — M5 Max SSD 100°C under AI workloads
• Jeff Geerling — RDMA over Thunderbolt 5 benchmarks

Reproduce the numbers yourself

The full test harness (prompts, scoring rubric, runner.py) is open-source: github.com/radionokia/homelab/projects/llm-benchmark. Run it on any of the setups above, score the outputs against your own rubric, and you'll get reproducible quality + latency data for your hardware.

Who built this

Artem Kravchenko — AI Automation Consultant + IT Director, 30 years in IT, based in Quebec City. Bilingual (English / French).

I run all seven setups in production: my own work, my homelab, and at client sites. Numbers on this page are what I measure, not what vendors promise. If a recommendation here costs you money you didn't need to spend, I want to hear about it — that feedback makes the next revision better.

Not sure which setup fits your situation? Free consult, no commitment.

Want to see ROI analysis, per-role breakdowns, and KPI scorecard? → Coming soon at /roi