Message ID | 20240122002024.27477-1-ebiggers@kernel.org |
---|---|
Headers |
Return-Path: <linux-kernel+bounces-32232-ouuuleilei=gmail.com@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2bc4:b0:101:a8e8:374 with SMTP id hx4csp2295067dyb; Sun, 21 Jan 2024 16:24:00 -0800 (PST) X-Google-Smtp-Source: AGHT+IFCXt7q5wzHEZOUQtcS+2QccnA9Y+SuXT3NxMJcX080P8OfvXLNmO+IDth5u1FbPz7y8qos X-Received: by 2002:a17:90a:178f:b0:28b:de50:b4a7 with SMTP id q15-20020a17090a178f00b0028bde50b4a7mr656572pja.76.1705883039911; Sun, 21 Jan 2024 16:23:59 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1705883039; cv=pass; d=google.com; s=arc-20160816; b=WCXMGSA1EfSSj7/ywW+dCGKX4bxKp531BvGnE2efzuI1zZEEZ7glowbiBDfgp/77Rr zQfI2Xacjp+PhB+XPe1F7g0XiYsShn/++sibwMNWnPOx2idVnXFEKcjFNAGqnGtp8eZz jdXeB0WAh8BzhhZk3ObUMhXXeuxGJ5vw6iW2PjAqVhmSJBmQ/Cid9UwWuSi1IaAkFnTx ai6rMag3NsewMaRYS5LFJCrZVSZCYzIB1kLqqLH2xiHjUnbcWDtIdgr5mOEscKTNa/Hg vsMp+o7m6UZbGmbJ6+u45Js1C26NrZYbRmVlHXc9UxYwXPy3t4w8I+WaEmyrcngI1vHt CAaQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:message-id:date:subject:cc:to :from:dkim-signature; bh=t5RF/FQzxTwkGgMgkErNijOmc2BgL1awkhVAX5mNMbo=; fh=/pdiLwrzeGylF3AZCw3Q15BIKH/FbQ2tONl/uW4EGIM=; b=wfmPeJ6UDAkRs611OH1/IhhWkQSLEXjyEvFJdF1HQ6SyliobxVhkiDQy0LGlZ+b1Tj 5dTurjY7zaxj8qJuudmJo58HqDva9tXlZORiNrz9ovj5RnIHcB9J602v5nt40/0D9pU+ xLkWNuWtosQ7qg4WOW/nr1JJjpwtu74mkTRlMAHAGvbVbvLZyHiAaTdySxlUNpZzQU+W H7xfTHEp/6SAd4g36KdVzOC8g8YpPRyHYOErA35GsNmHkXehM0mWgBAWUHuH1ppsCOSg 8ZImMJrEj0Ec7e+7aDuRyKPU8ph3L456KTnlCjfad6kStnyT3qNbF9SPJ6Xk7IWchK5N 1X+g== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=cSiMXi3d; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-32232-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-32232-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [147.75.48.161]) by mx.google.com with ESMTPS id f6-20020a17090ace0600b0028e8dace207si7309893pju.36.2024.01.21.16.23.59 for <ouuuleilei@gmail.com> (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Jan 2024 16:23:59 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-32232-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=cSiMXi3d; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-32232-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-32232-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 87BD9B21F07 for <ouuuleilei@gmail.com>; Mon, 22 Jan 2024 00:23:52 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 4DDB429B0; Mon, 22 Jan 2024 00:23:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="cSiMXi3d" Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 23838360; Mon, 22 Jan 2024 00:22:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705882978; cv=none; b=MSEfEj7eESJZ5DCcE6G4rMKSHwVia6PNwX7he075oSesnrX5Zf5aNIEZPsp++7RUYJ1thlBgR138h2Iv6Mbu/oq9rJEKHpVLiZ+AG8RPkDL6X3rgIbGUaJSvGH6Cc3fSsc8Q/8Yge/2+KnsyCdHKcEyWpcF79EaHFicDJLTSRlc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705882978; c=relaxed/simple; bh=/98FyMHfNpcE20uREuwvTI8aEaiGp4Wx3e7AXQcREB8=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:Content-Type; b=ucWKTQL00fDxj8wVJ1AT6fjJA0ZntKeFByLOqr8kqi92ZdRKkr7vuvAdBB4yNPl6YAIzhZzjzTRxLBThnFXBEl/DVBe8gJVCyWPxfVKm/DiUqrPZqJE2QOIW+5y/XlYA9khuqWOYyQhvoiiAZEs+jt1elDl+m6jCKXxazG+ajAQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=cSiMXi3d; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 03362C433F1; Mon, 22 Jan 2024 00:22:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1705882977; bh=/98FyMHfNpcE20uREuwvTI8aEaiGp4Wx3e7AXQcREB8=; h=From:To:Cc:Subject:Date:From; b=cSiMXi3dT5GpQICa7Om7LNG5CODXq/A5eLQLot62CARYuv48uTEsKFbCUsiN/aK2v s9uZxu7P3fbDu+6LfqSxjftwQ/QcWy37EZwTnoHtMjuasPfcu1t9F4/OALHYoVQQqZ dOy0OsVn3VSEk2ucP4CrE2u5gasKrgU04YMIK19Ww/xIJHxRd99H+NTqZHejdVXXB8 laEOtzT7btvQ0yYcJOMpqmZPLwbrnbQ2graMl+UTIi4gjQF8l1dtaM63DO9NUPVWXw hi/A8A1kiJovyJZkATF/D05ea2yE+fUWWRdtJBn4z3NmGyIvM69hmD5JdZTMBv+nts AJ+QYb+emR7jg== From: Eric Biggers <ebiggers@kernel.org> To: linux-crypto@vger.kernel.org, linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, Albert Ou <aou@eecs.berkeley.edu>, Andy Chiu <andy.chiu@sifive.com>, Ard Biesheuvel <ardb@kernel.org>, =?utf-8?q?Christoph_M=C3=BCllner?= <christoph.muellner@vrull.eu>, Heiko Stuebner <heiko@sntech.de>, Jerry Shih <jerry.shih@sifive.com>, Palmer Dabbelt <palmer@dabbelt.com>, Paul Walmsley <paul.walmsley@sifive.com>, Phoebe Chen <phoebe.chen@sifive.com>, hongrong.hsu@sifive.com Subject: [PATCH v3 00/10] RISC-V crypto with reworked asm files Date: Sun, 21 Jan 2024 16:19:11 -0800 Message-ID: <20240122002024.27477-1-ebiggers@kernel.org> X-Mailer: git-send-email 2.43.0 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: <linux-kernel.vger.kernel.org> List-Subscribe: <mailto:linux-kernel+subscribe@vger.kernel.org> List-Unsubscribe: <mailto:linux-kernel+unsubscribe@vger.kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1788748014552924565 X-GMAIL-MSGID: 1788748014552924565 |
Series |
RISC-V crypto with reworked asm files
|
|
Message
Eric Biggers
Jan. 22, 2024, 12:19 a.m. UTC
This patchset, which applies to v6.8-rc1, adds cryptographic algorithm implementations accelerated using the RISC-V vector crypto extensions (https://github.com/riscv/riscv-crypto/releases/download/v1.0.0/riscv-crypto-spec-vector.pdf) and RISC-V vector extension (https://github.com/riscv/riscv-v-spec/releases/download/v1.0/riscv-v-spec-1.0.pdf). The following algorithms are included: AES in ECB, CBC, CTR, and XTS modes; ChaCha20; GHASH; SHA-2; SM3; and SM4. In general, the assembly code requires a 64-bit RISC-V CPU with VLEN >= 128, little endian byte order, and vector unaligned access support. The ECB, CTR, XTS, and ChaCha20 code is designed to naturally scale up to larger VLEN values. Building the assembly code requires tip-of-tree binutils (future 2.42) or tip-of-tree clang (future 18.x). All algorithms pass testing in QEMU, using CONFIG_CRYPTO_MANAGER_EXTRA_TESTS=y. Much of the assembly code is derived from OpenSSL code that was added by https://github.com/openssl/openssl/pull/21923. It's been cleaned up for integration with the kernel, e.g. reducing code duplication, eliminating use of .inst and perlasm, and fixing a few bugs. This patchset incorporates the work of multiple people, including Jerry Shih, Heiko Stuebner, Christoph Müllner, Phoebe Chen, Charalampos Mitrodimas, and myself. This patchset went through several versions from Heiko (last version https://lore.kernel.org/linux-crypto/20230711153743.1970625-1-heiko@sntech.de), then several versions from Jerry (last version: https://lore.kernel.org/linux-crypto/20231231152743.6304-1-jerry.shih@sifive.com), then finally several versions from me. Thanks to everyone who has contributed to this patchset or its prerequisites. Since v6.8-rc1, all prerequisite kernel patches are upstream. I think this is now ready, and I'd like for it to be applied for 6.9, either to the crypto or riscv tree (at maintainers' choice). Below is the changelog for my versions of the patchset. For the changelog of the older versions, see the above links. Changed in v3: - Fixed a bug in the AES-XTS implementation where it assumed the CPU always set vl to the maximum possible value. This was okay for QEMU, but the vector spec allows CPUs to have different behavior. - Increased the LMUL for AES-ECB to 8, as the registers are available. - Fixed some license text that I had mistakenly changed when doing a find-and-replace of code. - Addressed a checkpatch warning by not including filename in file. - Rename some labels. - Constify a variable. Changed in v2: - Merged the AES modules together to prevent a build error. - Only unregister AES algorithms that were registered. - Corrected walksize properties to match the LMUL used by asm code. - Simplified the CTR and XTS glue code slightly. - Minor cleanups. Changed in v1: - Refer to my cover letter https://lore.kernel.org/linux-crypto/20240102064743.220490-1-ebiggers@kernel.org/ Eric Biggers (1): RISC-V: add TOOLCHAIN_HAS_VECTOR_CRYPTO Heiko Stuebner (2): RISC-V: add helper function to read the vector VLEN RISC-V: hook new crypto subdir into build-system Jerry Shih (7): crypto: riscv - add vector crypto accelerated AES-{ECB,CBC,CTR,XTS} crypto: riscv - add vector crypto accelerated ChaCha20 crypto: riscv - add vector crypto accelerated GHASH crypto: riscv - add vector crypto accelerated SHA-{256,224} crypto: riscv - add vector crypto accelerated SHA-{512,384} crypto: riscv - add vector crypto accelerated SM3 crypto: riscv - add vector crypto accelerated SM4 arch/riscv/Kbuild | 1 + arch/riscv/Kconfig | 7 + arch/riscv/crypto/Kconfig | 93 +++ arch/riscv/crypto/Makefile | 23 + arch/riscv/crypto/aes-macros.S | 156 +++++ arch/riscv/crypto/aes-riscv64-glue.c | 550 ++++++++++++++++++ .../crypto/aes-riscv64-zvkned-zvbb-zvkg.S | 312 ++++++++++ arch/riscv/crypto/aes-riscv64-zvkned-zvkb.S | 146 +++++ arch/riscv/crypto/aes-riscv64-zvkned.S | 180 ++++++ arch/riscv/crypto/chacha-riscv64-glue.c | 101 ++++ arch/riscv/crypto/chacha-riscv64-zvkb.S | 294 ++++++++++ arch/riscv/crypto/ghash-riscv64-glue.c | 168 ++++++ arch/riscv/crypto/ghash-riscv64-zvkg.S | 72 +++ arch/riscv/crypto/sha256-riscv64-glue.c | 137 +++++ .../sha256-riscv64-zvknha_or_zvknhb-zvkb.S | 225 +++++++ arch/riscv/crypto/sha512-riscv64-glue.c | 133 +++++ .../riscv/crypto/sha512-riscv64-zvknhb-zvkb.S | 203 +++++++ arch/riscv/crypto/sm3-riscv64-glue.c | 112 ++++ arch/riscv/crypto/sm3-riscv64-zvksh-zvkb.S | 123 ++++ arch/riscv/crypto/sm4-riscv64-glue.c | 107 ++++ arch/riscv/crypto/sm4-riscv64-zvksed-zvkb.S | 117 ++++ arch/riscv/include/asm/vector.h | 11 + crypto/Kconfig | 3 + 23 files changed, 3274 insertions(+) create mode 100644 arch/riscv/crypto/Kconfig create mode 100644 arch/riscv/crypto/Makefile create mode 100644 arch/riscv/crypto/aes-macros.S create mode 100644 arch/riscv/crypto/aes-riscv64-glue.c create mode 100644 arch/riscv/crypto/aes-riscv64-zvkned-zvbb-zvkg.S create mode 100644 arch/riscv/crypto/aes-riscv64-zvkned-zvkb.S create mode 100644 arch/riscv/crypto/aes-riscv64-zvkned.S create mode 100644 arch/riscv/crypto/chacha-riscv64-glue.c create mode 100644 arch/riscv/crypto/chacha-riscv64-zvkb.S create mode 100644 arch/riscv/crypto/ghash-riscv64-glue.c create mode 100644 arch/riscv/crypto/ghash-riscv64-zvkg.S create mode 100644 arch/riscv/crypto/sha256-riscv64-glue.c create mode 100644 arch/riscv/crypto/sha256-riscv64-zvknha_or_zvknhb-zvkb.S create mode 100644 arch/riscv/crypto/sha512-riscv64-glue.c create mode 100644 arch/riscv/crypto/sha512-riscv64-zvknhb-zvkb.S create mode 100644 arch/riscv/crypto/sm3-riscv64-glue.c create mode 100644 arch/riscv/crypto/sm3-riscv64-zvksh-zvkb.S create mode 100644 arch/riscv/crypto/sm4-riscv64-glue.c create mode 100644 arch/riscv/crypto/sm4-riscv64-zvksed-zvkb.S base-commit: 6613476e225e090cc9aad49be7fa504e290dd33d
Comments
On Sun, 21 Jan 2024 16:19:11 PST (-0800), ebiggers@kernel.org wrote: > This patchset, which applies to v6.8-rc1, adds cryptographic algorithm > implementations accelerated using the RISC-V vector crypto extensions > (https://github.com/riscv/riscv-crypto/releases/download/v1.0.0/riscv-crypto-spec-vector.pdf) > and RISC-V vector extension > (https://github.com/riscv/riscv-v-spec/releases/download/v1.0/riscv-v-spec-1.0.pdf). > The following algorithms are included: AES in ECB, CBC, CTR, and XTS modes; > ChaCha20; GHASH; SHA-2; SM3; and SM4. > > In general, the assembly code requires a 64-bit RISC-V CPU with VLEN >= 128, > little endian byte order, and vector unaligned access support. The ECB, CTR, > XTS, and ChaCha20 code is designed to naturally scale up to larger VLEN values. > Building the assembly code requires tip-of-tree binutils (future 2.42) or > tip-of-tree clang (future 18.x). All algorithms pass testing in QEMU, using > CONFIG_CRYPTO_MANAGER_EXTRA_TESTS=y. Much of the assembly code is derived from > OpenSSL code that was added by https://github.com/openssl/openssl/pull/21923. > It's been cleaned up for integration with the kernel, e.g. reducing code > duplication, eliminating use of .inst and perlasm, and fixing a few bugs. > > This patchset incorporates the work of multiple people, including Jerry Shih, > Heiko Stuebner, Christoph Müllner, Phoebe Chen, Charalampos Mitrodimas, and > myself. This patchset went through several versions from Heiko (last version > https://lore.kernel.org/linux-crypto/20230711153743.1970625-1-heiko@sntech.de), > then several versions from Jerry (last version: > https://lore.kernel.org/linux-crypto/20231231152743.6304-1-jerry.shih@sifive.com), > then finally several versions from me. Thanks to everyone who has contributed > to this patchset or its prerequisites. Since v6.8-rc1, all prerequisite kernel > patches are upstream. I think this is now ready, and I'd like for it to be > applied for 6.9, either to the crypto or riscv tree (at maintainers' choice). I'm OK taking this through the RISC-V tree if folks want, I've gone and queued it up for my tester. Just LMK if it should go somewhere else, otherwise it'll end up on for-next in a day or so (I'm trying to get LTO builds added right now, so it might take a bit). > > Below is the changelog for my versions of the patchset. For the changelog of > the older versions, see the above links. > > Changed in v3: > - Fixed a bug in the AES-XTS implementation where it assumed the CPU > always set vl to the maximum possible value. This was okay for > QEMU, but the vector spec allows CPUs to have different behavior. > - Increased the LMUL for AES-ECB to 8, as the registers are available. > - Fixed some license text that I had mistakenly changed when doing a > find-and-replace of code. > - Addressed a checkpatch warning by not including filename in file. > - Rename some labels. > - Constify a variable. > > Changed in v2: > - Merged the AES modules together to prevent a build error. > - Only unregister AES algorithms that were registered. > - Corrected walksize properties to match the LMUL used by asm code. > - Simplified the CTR and XTS glue code slightly. > - Minor cleanups. > > Changed in v1: > - Refer to my cover letter > https://lore.kernel.org/linux-crypto/20240102064743.220490-1-ebiggers@kernel.org/ > > Eric Biggers (1): > RISC-V: add TOOLCHAIN_HAS_VECTOR_CRYPTO > > Heiko Stuebner (2): > RISC-V: add helper function to read the vector VLEN > RISC-V: hook new crypto subdir into build-system > > Jerry Shih (7): > crypto: riscv - add vector crypto accelerated AES-{ECB,CBC,CTR,XTS} > crypto: riscv - add vector crypto accelerated ChaCha20 > crypto: riscv - add vector crypto accelerated GHASH > crypto: riscv - add vector crypto accelerated SHA-{256,224} > crypto: riscv - add vector crypto accelerated SHA-{512,384} > crypto: riscv - add vector crypto accelerated SM3 > crypto: riscv - add vector crypto accelerated SM4 > > arch/riscv/Kbuild | 1 + > arch/riscv/Kconfig | 7 + > arch/riscv/crypto/Kconfig | 93 +++ > arch/riscv/crypto/Makefile | 23 + > arch/riscv/crypto/aes-macros.S | 156 +++++ > arch/riscv/crypto/aes-riscv64-glue.c | 550 ++++++++++++++++++ > .../crypto/aes-riscv64-zvkned-zvbb-zvkg.S | 312 ++++++++++ > arch/riscv/crypto/aes-riscv64-zvkned-zvkb.S | 146 +++++ > arch/riscv/crypto/aes-riscv64-zvkned.S | 180 ++++++ > arch/riscv/crypto/chacha-riscv64-glue.c | 101 ++++ > arch/riscv/crypto/chacha-riscv64-zvkb.S | 294 ++++++++++ > arch/riscv/crypto/ghash-riscv64-glue.c | 168 ++++++ > arch/riscv/crypto/ghash-riscv64-zvkg.S | 72 +++ > arch/riscv/crypto/sha256-riscv64-glue.c | 137 +++++ > .../sha256-riscv64-zvknha_or_zvknhb-zvkb.S | 225 +++++++ > arch/riscv/crypto/sha512-riscv64-glue.c | 133 +++++ > .../riscv/crypto/sha512-riscv64-zvknhb-zvkb.S | 203 +++++++ > arch/riscv/crypto/sm3-riscv64-glue.c | 112 ++++ > arch/riscv/crypto/sm3-riscv64-zvksh-zvkb.S | 123 ++++ > arch/riscv/crypto/sm4-riscv64-glue.c | 107 ++++ > arch/riscv/crypto/sm4-riscv64-zvksed-zvkb.S | 117 ++++ > arch/riscv/include/asm/vector.h | 11 + > crypto/Kconfig | 3 + > 23 files changed, 3274 insertions(+) > create mode 100644 arch/riscv/crypto/Kconfig > create mode 100644 arch/riscv/crypto/Makefile > create mode 100644 arch/riscv/crypto/aes-macros.S > create mode 100644 arch/riscv/crypto/aes-riscv64-glue.c > create mode 100644 arch/riscv/crypto/aes-riscv64-zvkned-zvbb-zvkg.S > create mode 100644 arch/riscv/crypto/aes-riscv64-zvkned-zvkb.S > create mode 100644 arch/riscv/crypto/aes-riscv64-zvkned.S > create mode 100644 arch/riscv/crypto/chacha-riscv64-glue.c > create mode 100644 arch/riscv/crypto/chacha-riscv64-zvkb.S > create mode 100644 arch/riscv/crypto/ghash-riscv64-glue.c > create mode 100644 arch/riscv/crypto/ghash-riscv64-zvkg.S > create mode 100644 arch/riscv/crypto/sha256-riscv64-glue.c > create mode 100644 arch/riscv/crypto/sha256-riscv64-zvknha_or_zvknhb-zvkb.S > create mode 100644 arch/riscv/crypto/sha512-riscv64-glue.c > create mode 100644 arch/riscv/crypto/sha512-riscv64-zvknhb-zvkb.S > create mode 100644 arch/riscv/crypto/sm3-riscv64-glue.c > create mode 100644 arch/riscv/crypto/sm3-riscv64-zvksh-zvkb.S > create mode 100644 arch/riscv/crypto/sm4-riscv64-glue.c > create mode 100644 arch/riscv/crypto/sm4-riscv64-zvksed-zvkb.S > > > base-commit: 6613476e225e090cc9aad49be7fa504e290dd33d
On Mon, Jan 22, 2024 at 1:23 AM Eric Biggers <ebiggers@kernel.org> wrote: > > This patchset, which applies to v6.8-rc1, adds cryptographic algorithm > implementations accelerated using the RISC-V vector crypto extensions > (https://github.com/riscv/riscv-crypto/releases/download/v1.0.0/riscv-crypto-spec-vector.pdf) > and RISC-V vector extension > (https://github.com/riscv/riscv-v-spec/releases/download/v1.0/riscv-v-spec-1.0.pdf). > The following algorithms are included: AES in ECB, CBC, CTR, and XTS modes; > ChaCha20; GHASH; SHA-2; SM3; and SM4. > > In general, the assembly code requires a 64-bit RISC-V CPU with VLEN >= 128, > little endian byte order, and vector unaligned access support. The ECB, CTR, > XTS, and ChaCha20 code is designed to naturally scale up to larger VLEN values. > Building the assembly code requires tip-of-tree binutils (future 2.42) or > tip-of-tree clang (future 18.x). All algorithms pass testing in QEMU, using > CONFIG_CRYPTO_MANAGER_EXTRA_TESTS=y. Much of the assembly code is derived from > OpenSSL code that was added by https://github.com/openssl/openssl/pull/21923. > It's been cleaned up for integration with the kernel, e.g. reducing code > duplication, eliminating use of .inst and perlasm, and fixing a few bugs. > > This patchset incorporates the work of multiple people, including Jerry Shih, > Heiko Stuebner, Christoph Müllner, Phoebe Chen, Charalampos Mitrodimas, and > myself. This patchset went through several versions from Heiko (last version > https://lore.kernel.org/linux-crypto/20230711153743.1970625-1-heiko@sntech.de), > then several versions from Jerry (last version: > https://lore.kernel.org/linux-crypto/20231231152743.6304-1-jerry.shih@sifive.com), > then finally several versions from me. Thanks to everyone who has contributed > to this patchset or its prerequisites. Since v6.8-rc1, all prerequisite kernel > patches are upstream. I think this is now ready, and I'd like for it to be > applied for 6.9, either to the crypto or riscv tree (at maintainers' choice). > > Below is the changelog for my versions of the patchset. For the changelog of > the older versions, see the above links. For all patches of this series: Reviewed-by: Christoph Müllner <christoph.muellner@vrull.eu> Eric, thank you for working on this! > > Changed in v3: > - Fixed a bug in the AES-XTS implementation where it assumed the CPU > always set vl to the maximum possible value. This was okay for > QEMU, but the vector spec allows CPUs to have different behavior. > - Increased the LMUL for AES-ECB to 8, as the registers are available. > - Fixed some license text that I had mistakenly changed when doing a > find-and-replace of code. > - Addressed a checkpatch warning by not including filename in file. > - Rename some labels. > - Constify a variable. > > Changed in v2: > - Merged the AES modules together to prevent a build error. > - Only unregister AES algorithms that were registered. > - Corrected walksize properties to match the LMUL used by asm code. > - Simplified the CTR and XTS glue code slightly. > - Minor cleanups. > > Changed in v1: > - Refer to my cover letter > https://lore.kernel.org/linux-crypto/20240102064743.220490-1-ebiggers@kernel.org/ > > Eric Biggers (1): > RISC-V: add TOOLCHAIN_HAS_VECTOR_CRYPTO > > Heiko Stuebner (2): > RISC-V: add helper function to read the vector VLEN > RISC-V: hook new crypto subdir into build-system > > Jerry Shih (7): > crypto: riscv - add vector crypto accelerated AES-{ECB,CBC,CTR,XTS} > crypto: riscv - add vector crypto accelerated ChaCha20 > crypto: riscv - add vector crypto accelerated GHASH > crypto: riscv - add vector crypto accelerated SHA-{256,224} > crypto: riscv - add vector crypto accelerated SHA-{512,384} > crypto: riscv - add vector crypto accelerated SM3 > crypto: riscv - add vector crypto accelerated SM4 > > arch/riscv/Kbuild | 1 + > arch/riscv/Kconfig | 7 + > arch/riscv/crypto/Kconfig | 93 +++ > arch/riscv/crypto/Makefile | 23 + > arch/riscv/crypto/aes-macros.S | 156 +++++ > arch/riscv/crypto/aes-riscv64-glue.c | 550 ++++++++++++++++++ > .../crypto/aes-riscv64-zvkned-zvbb-zvkg.S | 312 ++++++++++ > arch/riscv/crypto/aes-riscv64-zvkned-zvkb.S | 146 +++++ > arch/riscv/crypto/aes-riscv64-zvkned.S | 180 ++++++ > arch/riscv/crypto/chacha-riscv64-glue.c | 101 ++++ > arch/riscv/crypto/chacha-riscv64-zvkb.S | 294 ++++++++++ > arch/riscv/crypto/ghash-riscv64-glue.c | 168 ++++++ > arch/riscv/crypto/ghash-riscv64-zvkg.S | 72 +++ > arch/riscv/crypto/sha256-riscv64-glue.c | 137 +++++ > .../sha256-riscv64-zvknha_or_zvknhb-zvkb.S | 225 +++++++ > arch/riscv/crypto/sha512-riscv64-glue.c | 133 +++++ > .../riscv/crypto/sha512-riscv64-zvknhb-zvkb.S | 203 +++++++ > arch/riscv/crypto/sm3-riscv64-glue.c | 112 ++++ > arch/riscv/crypto/sm3-riscv64-zvksh-zvkb.S | 123 ++++ > arch/riscv/crypto/sm4-riscv64-glue.c | 107 ++++ > arch/riscv/crypto/sm4-riscv64-zvksed-zvkb.S | 117 ++++ > arch/riscv/include/asm/vector.h | 11 + > crypto/Kconfig | 3 + > 23 files changed, 3274 insertions(+) > create mode 100644 arch/riscv/crypto/Kconfig > create mode 100644 arch/riscv/crypto/Makefile > create mode 100644 arch/riscv/crypto/aes-macros.S > create mode 100644 arch/riscv/crypto/aes-riscv64-glue.c > create mode 100644 arch/riscv/crypto/aes-riscv64-zvkned-zvbb-zvkg.S > create mode 100644 arch/riscv/crypto/aes-riscv64-zvkned-zvkb.S > create mode 100644 arch/riscv/crypto/aes-riscv64-zvkned.S > create mode 100644 arch/riscv/crypto/chacha-riscv64-glue.c > create mode 100644 arch/riscv/crypto/chacha-riscv64-zvkb.S > create mode 100644 arch/riscv/crypto/ghash-riscv64-glue.c > create mode 100644 arch/riscv/crypto/ghash-riscv64-zvkg.S > create mode 100644 arch/riscv/crypto/sha256-riscv64-glue.c > create mode 100644 arch/riscv/crypto/sha256-riscv64-zvknha_or_zvknhb-zvkb.S > create mode 100644 arch/riscv/crypto/sha512-riscv64-glue.c > create mode 100644 arch/riscv/crypto/sha512-riscv64-zvknhb-zvkb.S > create mode 100644 arch/riscv/crypto/sm3-riscv64-glue.c > create mode 100644 arch/riscv/crypto/sm3-riscv64-zvksh-zvkb.S > create mode 100644 arch/riscv/crypto/sm4-riscv64-glue.c > create mode 100644 arch/riscv/crypto/sm4-riscv64-zvksed-zvkb.S > > > base-commit: 6613476e225e090cc9aad49be7fa504e290dd33d > -- > 2.43.0 >
Hello: This series was applied to riscv/linux.git (for-next) by Palmer Dabbelt <palmer@rivosinc.com>: On Sun, 21 Jan 2024 16:19:11 -0800 you wrote: > This patchset, which applies to v6.8-rc1, adds cryptographic algorithm > implementations accelerated using the RISC-V vector crypto extensions > (https://github.com/riscv/riscv-crypto/releases/download/v1.0.0/riscv-crypto-spec-vector.pdf) > and RISC-V vector extension > (https://github.com/riscv/riscv-v-spec/releases/download/v1.0/riscv-v-spec-1.0.pdf). > The following algorithms are included: AES in ECB, CBC, CTR, and XTS modes; > ChaCha20; GHASH; SHA-2; SM3; and SM4. > > [...] Here is the summary with links: - [v3,01/10] RISC-V: add helper function to read the vector VLEN (no matching commit) - [v3,02/10] RISC-V: add TOOLCHAIN_HAS_VECTOR_CRYPTO https://git.kernel.org/riscv/c/34ca4ec628de - [v3,03/10] RISC-V: hook new crypto subdir into build-system https://git.kernel.org/riscv/c/178f3856436c - [v3,04/10] crypto: riscv - add vector crypto accelerated AES-{ECB,CBC,CTR,XTS} https://git.kernel.org/riscv/c/eb24af5d7a05 - [v3,05/10] crypto: riscv - add vector crypto accelerated ChaCha20 https://git.kernel.org/riscv/c/bb54668837a0 - [v3,06/10] crypto: riscv - add vector crypto accelerated GHASH https://git.kernel.org/riscv/c/600a3853dfa0 - [v3,07/10] crypto: riscv - add vector crypto accelerated SHA-{256,224} https://git.kernel.org/riscv/c/8c8e40470ffe - [v3,08/10] crypto: riscv - add vector crypto accelerated SHA-{512,384} https://git.kernel.org/riscv/c/b3415925a08b - [v3,09/10] crypto: riscv - add vector crypto accelerated SM3 https://git.kernel.org/riscv/c/563a5255afa2 - [v3,10/10] crypto: riscv - add vector crypto accelerated SM4 https://git.kernel.org/riscv/c/b8d06352bbf3 You are awesome, thank you!