by Trevor Cooper, System Architecture, Andes

1. Why Vector Performance Matters

While GPUs dominate large-scale model training, CPUs execute a vast amount of matrix math in inference pipelines, data preprocessing, and scientific computing. Basic Linear Algebra Subprograms (BLAS) libraries underpin many scientific and machine learning frameworks, and one of their most important routines is GEMM — General Matrix Multiply. DGEMM (double-precision GEMM) is the gold-standard benchmark for this class of workload: it simultaneously stresses floating-point throughput, vector execution, register utilization, and memory bandwidth, making it a highly representative proxy for real-world compute intensity.
In this article we share practical lessons from hands-on experiments implementing a DGEMM kernel on an Andes AX46MPV near cycle-accurate RISC-V simulator. A cycle-accurate simulator lets us observe performance at the granularity of individual processor cycles and instrument the code with hardware performance monitoring (HPM) counters to track cycle counts, instruction throughput, and memory activity — all without silicon.
Our experiments confirm that the RISC-V Vector Extension (RVV) can dramatically improve performance with modest coding effort. A minimal RVV port delivered a 12× speedup over a scalar baseline; progressive tuning pushed that to over 150×; and enabling High Bandwidth Vector Memory (HVM) feature unlocked 275× — reaching 92.8% of theoretical peak efficiency. These gains stem from RVV’s scalable design, and the lessons learned along the way reveal how vector parameters interact in ways that are not always intuitive.

2. RISC-V Vector Extension (RVV) Primer

Unlike fixed-width vector ISAs such as x86 AVX-512 or Arm NEON, RVV uses a scalable vector model: the hardware determines the physical vector register width and software adapts at runtime. While Arm SVE is also a scalable architecture, RVV is more flexible in both scalability and software-controlled tunability. The same binary runs efficiently across a wide range of RISC-V hardware without recompilation.
Three parameters govern how vector operations behave:
  • VLEN — the physical width of a vector register in bits (e.g. 512 or 1024). A 1024-bit register holds 16 double-precision (FP64) elements.
  • SEW (Selected Element Width) — the element size used for each operation, chosen at runtime. For DGEMM, we use SEW=64 (FP64).
  • LMUL (Length MULtiplier) — a register-grouping multiplier. LMUL=4 fuses four physical registers into one logical vector, quadrupling capacity but proportionally reducing the number of independent logical registers available. With 32 physical registers, LMUL=4 yields 8 logical register groups.
Experimenting with these “knobs” taught us that increasing vector capacity always involves trade-offs. The interactions between VLEN, LMUL, and loop structure can produce counter-intuitive results, as the experiments below illustrate.

What is HVM?

High Bandwidth Vector Memory (HVM) is a microarchitectural feature that addresses the bottleneck that is the data path between the cache and the vector register file. In a standard RVV implementation, vector loads travel through a 512-bit wide cache data bus. At VLEN=1024 with SEW=64, each vector register holds 16 FP64 elements — 1024 bits — so filling it requires two sequential 512-bit transfers. HVM provides a dedicated 1024-bit wide memory path to the vector register file, wide enough to deliver a full 1024-bit vector register in a single transfer.

HVM is transparent to software: no changes to the RVV binary are required. The same kernel benefits automatically when HVM is enabled. This makes it an especially attractive feature for memory-bandwidth-bound workloads like DGEMM, where vector loads are on the critical path.

To view the rest of the article, please submit the information below and you will be redirected to the full article.

Name(Required)