I got AMD Hyperloom running experimentally on a Radeon AI PRO R9700.
That sentence is interesting because Hyperloom doesn't officially support this GPU today.
The R9700 is an RDNA 4 card with:
- 32 GB VRAM
gfx1201- 64 Compute Units
- ROCm 10
Hyperloom currently targets AMD Instinct GPU runners, so I wanted to see how much of its optimization stack could be extended to Radeon without pretending an R9700 is an MI300X.
The first port was surprisingly small:
r9700 → gfx1201 / 64 CU
Then I added GPU detection and used Hyperloom's architecture-neutral bypass benchmark backend with InferenceX.
The stack I'm running locally:
AMD Radeon AI PRO R9700
→ ROCm 10
→ vLLM
→ Qwen3-Coder 30B
→ Hyperloom experimental port
And it actually ran on the physical GPU.
Initial benchmark:
Baseline, concurrency 1
21.80 output tokens/s
Candidate, concurrency 2
36.59 output tokens/s
That's a +67.89% increase in aggregate throughput.
Important caveat: that's a concurrency tuning result, not a claim that the GPU magically became 67.89% faster. Latency increased too.
The part I'm working on now is more interesting:
I want Hyperloom's autonomous optimization loop to use the local Qwen model itself as the optimization agent.
So the eventual loop becomes:
Hyperloom
→ analyzes the workload
→ Qwen decides an optimization
→ applies candidate configuration
→ benchmarks again
→ keeps or rejects the change
All running locally on AMD hardware.
There are still hard boundaries:
- no official R9700 Hyperloom support
- no RDNA4-specific kernel tuning yet
- no direct reuse of
gfx942/gfx950Instinct kernels - Magpie / TraceLens RDNA4 work still pending
I'm documenting the whole process publicly because I think affordable workstation GPUs are an interesting place for local agentic AI, not just datacenter accelerators.
Repo:
github.com/Rafa-Innerchispa/hyperloom-r9700-experimental
Challenge project:
github.com/Rafa-Innerchispa/amd-ralfiia-hybrid-ops-copilot
Building this as part of the AMD x lablab.ai AI Academy Challenge.
#AMD #ROCm #Hyperloom #LocalAI #RDNA4 #vLLM #OpenSource #BuildInPublic
A few days ago I posted that I had AMD HyperLoom running experimentally on a Radeon AI PRO R9700.
The most useful thing that happened next was that people challenged the benchmark.
My original result was:
21.80 → 36.59 output tok/s
But that comparison came from single process runs.
People in the AMD/ROCm community pointed me toward a known R9700 issue where vLLM performance can vary depending on process initialization.
So instead of defending the number, I changed the experiment.
### 3 independent vLLM process starts
The new results were:
Baseline:
19.89 / 19.94 / 19.99 tok/s
Candidate:
36.00 / 36.08 / 36.19 tok/s
Median paired gain:
+80.99%
Baseline cross-process spread:
~0.47%
All three process-level decisions were KEEP.
I still do not call that an RDNA4 kernel speedup.
It is a serving/concurrency result.
But the investigation led somewhere much more interesting.
### I found an AutoAWQ MoE / WNA16 gap on gfx1201
On the Qwen3-Coder AWQ workload I'm using, the WNA16 path currently falls back to INT4 emulation on the R9700.
So I started going below the serving layer.
I now have a real packed-INT4 Triton W1 kernel executing on the physical gfx1201 GPU.
The strongest current version precomputes:
correction = zero_point × scalewhen weights are loaded, moving that work out of the request hot path.
With a fixed configuration and 21 paired measurements per workload size, the current small-M W1 results versus the BF16 routed reference are roughly:
Cosine similarity remains around 0.999995.
Again, those are kernel-level microbenchmark results, not end-to-end model claims.
### The next piece is now built too
I also have an experimental
R9700HybridWNA16Expertspath instantiating through vLLM's real modular MoE interface.The current contract smoke passes:
I have deliberately not patched the resident vLLM server yet.
The next test is an isolated vLLM process using the actual Qwen3-Coder AWQ weights.
I need to measure the real MoE activation dtype/layout, validate the hybrid backend with real weights, and then rerun throughput, TTFT and E2E measurements across independent process starts.
There are still open questions:
The project is getting more interesting precisely because some experiments fail.
Repo:
https://github.com/Rafa-Innerchispa/hyperloom-r9700-experimental
I'm documenting the positive and negative results because the goal isn't to make the benchmark look good.
The goal is to figure out what would actually be useful upstream for RDNA4.
Curious to hear from anyone working on ROCm, Triton, vLLM, MoE or gfx1201.