Module pclmulqdq 

Specialized checksum code for the x86 CPU architecture, based on the efficient algorithm described in the following whitepaper:

Gopal, V., Ozturk, E., Guilford, J., Wolrich, G., Feghali, W., Dixon, M., & Karakoyunlu, D. (2009). Fast CRC computation for generic polynomials using PCLMULQDQ instruction. Intel. (Mirror link: https://fossies.org/linux/zlib-ng/doc/crc-pclmulqdq.pdf, accessed 2024-05-20)

Throughout the code, this work is referred to as “the paper”.

On top of the 128-bit PCLMULQDQ implementation, two wider variants use VPCLMULQDQ to fold several independent 128-bit streams per instruction: an AVX2 variant over 256-bit YMM registers (8 streams) and an AVX-512 variant over 512-bit ZMM registers (16 streams). Both rely on VPCLMULQDQ intrinsics stabilized in Rust 1.89 and are only compiled when the stable_vpclmulqdq cfg is set by build.rs, leaving the crate MSRV unchanged otherwise. The best variant supported by the running CPU is chosen at runtime.

Each unsafe fn here is unsafe only because it needs the CPU features in its #[target_feature] list (PCLMULQDQ for reduce128). None of them has a memory-safety precondition: every load is bounds-checked.

Structs

State

Enums

Kind 🔒

Which SIMD implementation to use, chosen once at construction from the CPU’s features.

Constants

K1 🔒

K2 🔒

K3 🔒

K4 🔒

K5 🔒

K_1024_HIGH 🔒

K_1024_LOW 🔒

K_2048_HIGH 🔒

K_2048_LOW 🔒

MIN_AVX512_BYTES 🔒

MIN_FOLD_BY_4_BYTES 🔒

P_X 🔒

U_PRIME 🔒

Functions

calculate 🔒 ^⚠

calculate_avx2 🔒 ^⚠

256-bit VPCLMULQDQ variant: 8 streams across four YMM registers (two lanes each), folding two streams per carry-less multiply.

calculate_avx512 🔒 ^⚠

512-bit VPCLMULQDQ variant: 16 streams across four ZMM registers (four lanes each), folding four streams per carry-less multiply.

get 🔒

Loads the next 16 bytes of a as a vector and advances a past them.

get256 🔒 ^⚠

get512 🔒 ^⚠

partial_words 🔒

Packs a final partial block of 1..=15 bytes into two zero-padded little-endian words, using overlapping loads. Building it in registers avoids copying through a stack buffer, whose narrow stores the following 16-byte load cannot forward from.

reduce128 🔒 ^⚠

reduce256 🔒 ^⚠

reduce512 🔒 ^⚠

reduce_128_to_crc 🔒 ^⚠

Folds any remaining 16-byte chunks into x, then folds a final partial (< 16 byte) block with a byte-shift, reduces from 128 to 32 bits with a Barrett reduction, and returns the CRC. Shared by all of the fold implementations.