xxhash

Pure-Go XXH3-64 (the 64-bit variant of the modern XXH3 hash) with the hot 64-byte-stripe accumulator implemented as real SIMD assembly on all six of Go's 64-bit SIMD architectures — plain go build, no cgo, no GOEXPERIMENT. Digests are bit-exact with the canonical reference (Cyan4973/xxHash). This is the first Go XXH3 covering all six SIMD arches. Repository →

API

import "github.com/go-simd/xxhash"

h := xxhash.Sum64([]byte("hello"))   // one-shot
h = xxhash.Sum64String("hello")      // string, no copy of the bytes
d := xxhash.New()                    // streaming hash.Hash64
d.Write([]byte("hel")); d.Write([]byte("lo"))
h = d.Sum64()

Why this exists

XXH3 already has an excellent SIMD Go port, zeebo/xxh3, but its assembly covers amd64 and arm64 only (AVX2/SSE2 + NEON). The de-facto XXH64 library, cespare/xxhash, is also amd64 + arm64 only. Neither ships SIMD for ppc64le, s390x, riscv64 or loong64. This package covers all six — including the big-endian s390x — while matching them bit-for-bit everywhere. The value is breadth/coverage, not peak amd64/arm64 speed (see Performance).

Algorithm

The XXH3 long-input path keeps an 8-lane (8×uint64) accumulator and folds the input 64 bytes at a time: for each lane, dk = data ⊕ secret, then acc[i] += lo32(dk)·hi32(dk) and acc[i^1] += data. That per-stripe multiply-add is the SIMD kernel; the scramble, merge and avalanche stay in portable Go.

Because lo32 and hi32 are each < 2³², their product fits in 64 bits, so a plain low-64 vector multiply is exact — no widening multiply is needed on the arches that lack one.

Per-arch kernels

arch	ISA	multiply	lane swap
amd64	SSE2 + AVX2 (auto)	(V)PSHUFD+(V)PMULUDQ	(V)PSHUFD $0x4e
arm64	NEON	XTN/SHRN/UMULL (WORD-encoded)	VEXT $8
ppc64le	VSX	VMULOUW on lo/hi (no 64-bit vec mul on POWER8/9)	VSLDOI $8
s390x	vector facility (big-endian)	VMLOF (odd-word widening)	VPDI $4
riscv64	RVV	VMULVV (low 64 = full product)	VRGATHERVV
loong64	LSX	VMULV (low 64 = full product)	VSHUF4IW $0x4e

VSX is baseline on POWER8+ and the vector facility on z13+, so ppc64le and s390x have no runtime dispatch (the SIMD path is the only path). amd64 selects AVX2 at run time via golang.org/x/sys/cpu; riscv64 and loong64 fall back to the portable scalar kernel without the vector extension. Assembly is generated by go-asmgen.

Big-endian (s390x)

XXH3 reads input little-endian. The Go code decodes with binary.LittleEndian, and the s390x kernel byte-reverses each lane right after VL (via VPERM with a per-doubleword reversal selector) so the vector lanes carry true little-endian values. The accumulator math is endian-neutral, and the digest is verified bit-exact against the official vectors and a differential fuzz on s390x under QEMU.

Correctness

• Official vectors: TestOfficialVectors checks every input length 0…4095 against the canonical known-answer digests (default secret) — one-shot, string and streaming — on all six arches.
• Anchors: XXH3_64bits("") = 0x2d06800538d394c2, XXH3_64bits("a") = 0xe6c632b61e964e1f.
• Differential fuzz: FuzzSum64 vs zeebo/xxh3 — millions of executions per arch (incl. big-endian s390x), zero mismatches.

Performance — honest

Native arm64 (Apple silicon), Sum64:

input	go-simd/xxhash	zeebo/xxh3
64 B	~17.6 GB/s	~22.8 GB/s
1 KiB	~13.9 GB/s	~21.7 GB/s
64 KiB	~15.2 GB/s	~22.9 GB/s

On arm64 zeebo is faster (~22 vs ~15 GB/s): its NEON kernel folds a whole 1024-byte block per call with software-pipelined stripes. The amd64 kernel has since been deepened to the same block-at-a-time structure (16 unrolled stripes, accumulator register-resident across the run) and now reaches ~0.94× zeebo at 64 KiB (~50 GB/s) — bringing that structure to arm64 NEON is the obvious next step. The library's value is breadth: a bit-exact XXH3 on all six SIMD arches, where zeebo/cespare cover only amd64 + arm64.

Where zeebo has no kernel, go-simd wins outright (the same register-resident multi-stripe kernel was brought to all four remaining arches):

• riscv64 — real SpacemiT X60 (RVV 1.0, cfarm95, MEASURED): ~2.5–3.5× zeebo on every long input (256 B+) — zeebo runs the scalar path on RISC-V (~130 MB/s flat), while the RVV kernel reaches ~472 MB/s at 64 KiB (3.55×). Short inputs (≤ 64 B) trail (~0.53×) as that path is the shared scalar merge.
• ppc64le — real POWER9 + POWER8E (cfarm433/cfarm112, MEASURED): byte-exact on both ISAs (no SIGILL on POWER8E); throughput ~0.91× zeebo at 64 KiB — near parity, narrowing from the prior ~0.49× single-stripe gap.
• loong64 / s390x rely on the qemu CI lanes for correctness (cfarm401 is air-gapped; there is no z/Architecture cfarm node), with the riscv64 measured column as the directly-measured proxy for the identical cross-arch lever.

amd64 SIMD is correctness- and coverage-validated on a real x86_64 OS; its QEMU-backed throughput is not representative of native AVX2 and is omitted.

Coverage

100% statement coverage, enforced in CI on every architecture (native amd64/arm64 + QEMU riscv64/loong64/ppc64le/s390x), with both the SIMD and the scalar-fallback dispatch branches exercised. BSD-3-Clause.