linux/lib/crypto/x86
Eric Biggers 24eb22d816 lib/crypto: x86/aes: Add AES-NI optimization
Optimize the AES library with x86 AES-NI instructions.

The relevant existing assembly functions, aesni_set_key(), aesni_enc(),
and aesni_dec(), are a bit difficult to extract into the library:

- They're coupled to the code for the AES modes.
- They operate on struct crypto_aes_ctx.  The AES library now uses
  different structs.
- They assume the key is 16-byte aligned.  The AES library only
  *prefers* 16-byte alignment; it doesn't require it.

Moreover, they're not all that great in the first place:

- They use unrolled loops, which isn't a great choice on x86.
- They use the 'aeskeygenassist' instruction, which is unnecessary, is
  slow on Intel CPUs, and forces the loop to be unrolled.
- They have special code for AES-192 key expansion, despite that being
  kind of useless.  AES-128 and AES-256 are the ones used in practice.

These are small functions anyway.

Therefore, I opted to just write replacements of these functions for the
library.  They address all the above issues.

Acked-by: Ard Biesheuvel <ardb@kernel.org>
Link: https://lore.kernel.org/r/20260112192035.10427-18-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
2026-01-15 14:09:07 -08:00
..
.gitignore lib/crypto: x86: Move arch/x86/lib/crypto/ into lib/crypto/ 2025-06-30 09:26:20 -07:00
aes-aesni.S lib/crypto: x86/aes: Add AES-NI optimization 2026-01-15 14:09:07 -08:00
aes.h lib/crypto: x86/aes: Add AES-NI optimization 2026-01-15 14:09:07 -08:00
blake2s-core.S lib/crypto: x86/blake2s: Use vpternlogd for 3-input XORs 2025-11-05 20:30:52 -08:00
blake2s.h lib/crypto: blake2s: Drop excessive const & rename block => data 2025-10-29 22:04:24 -07:00
chacha-avx2-x86_64.S lib/crypto: x86: Move arch/x86/lib/crypto/ into lib/crypto/ 2025-06-30 09:26:20 -07:00
chacha-avx512vl-x86_64.S lib/crypto: x86: Move arch/x86/lib/crypto/ into lib/crypto/ 2025-06-30 09:26:20 -07:00
chacha-ssse3-x86_64.S lib/crypto: x86: Move arch/x86/lib/crypto/ into lib/crypto/ 2025-06-30 09:26:20 -07:00
chacha.h lib/crypto: chacha: Consolidate into single module 2025-08-29 09:50:19 -07:00
curve25519.h lib/crypto: curve25519: Consolidate into single module 2025-09-06 16:32:43 -07:00
nh-avx2.S lib/crypto: x86/nh: Migrate optimized code into library 2026-01-12 11:07:50 -08:00
nh-sse2.S lib/crypto: x86/nh: Migrate optimized code into library 2026-01-12 11:07:50 -08:00
nh.h lib/crypto: x86/nh: Migrate optimized code into library 2026-01-12 11:07:50 -08:00
poly1305-x86_64-cryptogams.pl lib/crypto: poly1305: Consolidate into single module 2025-08-29 09:49:18 -07:00
poly1305.h lib/crypto: poly1305: Consolidate into single module 2025-08-29 09:49:18 -07:00
polyval-pclmul-avx.S lib/crypto: x86/polyval: Migrate optimized code into library 2025-11-11 11:03:38 -08:00
polyval.h lib/crypto: x86/polyval: Migrate optimized code into library 2025-11-11 11:03:38 -08:00
sha1-avx2-asm.S lib/crypto: x86/sha1: Migrate optimized code into library 2025-07-14 11:28:35 -07:00
sha1-ni-asm.S lib/crypto: x86/sha1-ni: Convert to use rounds macros 2025-07-20 21:42:42 -07:00
sha1-ssse3-and-avx.S lib/crypto: x86/sha1: Migrate optimized code into library 2025-07-14 11:28:35 -07:00
sha1.h lib/crypto: Drop inline from all *_mod_init_arch() functions 2025-08-27 08:15:35 -07:00
sha256-avx-asm.S lib/crypto: x86/sha256: Remove unnecessary checks for nblocks==0 2025-07-04 10:23:56 -07:00
sha256-avx2-asm.S lib/crypto: x86/sha256: Remove unnecessary checks for nblocks==0 2025-07-04 10:23:56 -07:00
sha256-ni-asm.S lib/crypto: x86/sha256: Add support for 2-way interleaved hashing 2025-09-17 13:09:40 -05:00
sha256-ssse3-asm.S lib/crypto: x86/sha256: Remove unnecessary checks for nblocks==0 2025-07-04 10:23:56 -07:00
sha256.h Optimize fsverity with 2-way interleaved hashing 2025-09-29 15:55:20 -07:00
sha512-avx-asm.S lib/crypto: x86/sha512: Remove unnecessary checks for nblocks==0 2025-06-30 09:26:20 -07:00
sha512-avx2-asm.S lib/crypto: x86/sha512: Remove unnecessary checks for nblocks==0 2025-06-30 09:26:20 -07:00
sha512-ssse3-asm.S lib/crypto: x86/sha512: Remove unnecessary checks for nblocks==0 2025-06-30 09:26:20 -07:00
sha512.h lib/crypto: Drop inline from all *_mod_init_arch() functions 2025-08-27 08:15:35 -07:00