From patchwork Tue Feb 28 00:05:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Heiko_St=C3=BCbner?= X-Patchwork-Id: 5914 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp2719386wrd; Mon, 27 Feb 2023 16:16:49 -0800 (PST) X-Google-Smtp-Source: AK7set/Wj5lo6RIEFYVl+nm723ykM39rTPU2ku12vsAkM7hsfci3PRgwbK2RYkiWD7y0a+/4Y68Q X-Received: by 2002:a17:903:2442:b0:19d:abd:bb7e with SMTP id l2-20020a170903244200b0019d0abdbb7emr814910pls.34.1677543408714; Mon, 27 Feb 2023 16:16:48 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1677543408; cv=none; d=google.com; s=arc-20160816; b=nv55pKB0pKfkS7iq35b0Pd/YGPVcrSPt7OVHUgtGU4FgXqJQURg+X+6ZE+dmwBPWR/ 0YXuAb0bG+YJUZ6+Bk6BKp6x640lMPj1kxvbwzKiPx+gVqrdmqXpMxpfW39gosd6ThKg hkB3aRzxBCEUTzNYQtqOLmer4QTYbn0IWngVzIHCeQcbIn/wl/746mvYFHq9P7tLNDQc 1jHAvx5NzSRBrVfNUXp9RkGdJEp+v8EZI8wncIdFpIJWLunfAT6xDXOoqTUv3Qd1fZI0 R7qFqdyFJ59P1s7ofezCV1143jOgv9D43Bd1UciPJthVU9DHvsONmoad5VfoZS11m/Er DZPA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from; bh=iB9NK4jDmWieOiUGisGgsOyDXMEjzCORi0IWUR+i4/8=; b=LPpD+vIBkiTFwB/KXYf+gTxizFehAJxlXy4iCjt7lZWNvX6kOyCar1AOWxQiTBSV6L yBrvILTk/16EMD7bXpF/WqSA37ql6mzJi/nTS/Tq2pbw49pEdDV2d9p4e3UUiSv2Hbe+ AaLO8sNkcvNVpdNta5aIhl5XBUh6juc/bapKKSfpU//Gow54W038se7R89npCYIImKZg J58P+QfcIC6cl/T0xsjbzNpe5DTc+gJGnhYCHsXkG9nKQJiznlj86wF0w74OFERPfKgT 28ZEelHJu3FK7NxlU3Tom+I3jQnJrjZ2UdeclJBZDltLUesSDqs7TUsQ0lUGws/kRY97 H+OA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=sntech.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id t16-20020a634610000000b004f97f6225d0si8112714pga.731.2023.02.27.16.16.35; Mon, 27 Feb 2023 16:16:48 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=sntech.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230112AbjB1AGF (ORCPT + 99 others); Mon, 27 Feb 2023 19:06:05 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35972 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229769AbjB1AF4 (ORCPT ); Mon, 27 Feb 2023 19:05:56 -0500 Received: from gloria.sntech.de (gloria.sntech.de [185.11.138.130]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DC438227A3 for ; Mon, 27 Feb 2023 16:05:52 -0800 (PST) Received: from ip5b412258.dynamic.kabel-deutschland.de ([91.65.34.88] helo=phil.lan) by gloria.sntech.de with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1pWnV7-000552-CF; Tue, 28 Feb 2023 01:05:49 +0100 From: Heiko Stuebner To: palmer@rivosinc.com Cc: greentime.hu@sifive.com, conor@kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, christoph.muellner@vrull.eu, heiko@sntech.de, Heiko Stuebner Subject: [PATCH RFC v2 00/16] RISC-V: support some cryptography accelerations Date: Tue, 28 Feb 2023 01:05:28 +0100 Message-Id: <20230228000544.2234136-1-heiko@sntech.de> X-Mailer: git-send-email 2.39.0 MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,SPF_PASS, T_SPF_HELO_TEMPERROR autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1759031757306036381?= X-GMAIL-MSGID: =?utf-8?q?1759031757306036381?= From: Heiko Stuebner So this was my playground the last days. The base is v13 of the vector patchset but the first patches up to doing the Zbc-based GCM GHash can also run without those. Of course the vector- crypto extensions are also not ratified yet, hence the marking as RFC. As v13 of the vector patchset dropped the patches for in-kernel usage of vector instructions, I picked the ones from v12 over into this series for now. My basic goal was to not re-invent cryptographic code, so the heavy lifting is done by those perl-asm scripts used in openssl and the perl code used here-in stems from code that is targetted at openssl [0] and is unmodified from there to limit needed review effort. With a matching qemu (there are patches for vector-crypto flying around) the in-kernel crypto-selftests (also the extended ones) are very happy so far. Things to do: - use correct Co-developed-attribution for the code coming from openssl - follow openSSL changes along until they get eventually merged changes in v2: - rebased on 6.2 + zbb series, so don't include already applied changes anymore - refresh code picked from openssl as that side matures - more algorithms (SHA512, AES, SM3, SM4) [0] both still open https://github.com/openssl/openssl/pull/20078 https://github.com/openssl/openssl/pull/20149 Greentime Hu (2): riscv: Add support for kernel mode vector riscv: Add vector extension XOR implementation Heiko Stuebner (14): RISC-V: add Zbc extension detection RISC-V: add Zbkb extension detection RISC-V: hook new crypto subdir into build-system RISC-V: crypto: add accelerated GCM GHASH implementation RISC-V: add helper function to read the vector VLEN RISC-V: add vector crypto extension detection RISC-V: crypto: update perl include with helpers for vector (crypto) instructions RISC-V: crypto: add Zvkb accelerated GCM GHASH implementation RISC-V: crypto: add Zvkg accelerated GCM GHASH implementation RISC-V: crypto: add a vector-crypto-accelerated SHA256 implementation RISC-V: crypto: add a vector-crypto-accelerated SHA512 implementation RISC-V: crypto: add Zvkned accelerated AES encryption implementation RISC-V: crypto: add Zvksed accelerated SM4 encryption implementation RISC-V: crypto: add Zvksh accelerated SM3 hash implementation arch/riscv/Kbuild | 1 + arch/riscv/Kconfig | 22 + arch/riscv/crypto/Kconfig | 82 +++ arch/riscv/crypto/Makefile | 60 ++ arch/riscv/crypto/aes-riscv-glue.c | 169 ++++++ arch/riscv/crypto/aes-riscv64-zvkned.pl | 500 ++++++++++++++++ arch/riscv/crypto/ghash-riscv64-glue.c | 485 +++++++++++++++ arch/riscv/crypto/ghash-riscv64-zbc.pl | 400 +++++++++++++ arch/riscv/crypto/ghash-riscv64-zvkb.pl | 349 +++++++++++ arch/riscv/crypto/ghash-riscv64-zvkg.pl | 161 +++++ arch/riscv/crypto/riscv.pm | 659 +++++++++++++++++++++ arch/riscv/crypto/sha256-riscv64-glue.c | 114 ++++ arch/riscv/crypto/sha256-riscv64-zvknha.pl | 284 +++++++++ arch/riscv/crypto/sha512-riscv64-glue.c | 104 ++++ arch/riscv/crypto/sha512-riscv64-zvknhb.pl | 347 +++++++++++ arch/riscv/crypto/sm3-riscv64-glue.c | 112 ++++ arch/riscv/crypto/sm3-riscv64-zvksh.pl | 195 ++++++ arch/riscv/crypto/sm4-riscv64-glue.c | 163 +++++ arch/riscv/crypto/sm4-riscv64-zvksed.pl | 270 +++++++++ arch/riscv/include/asm/hwcap.h | 9 + arch/riscv/include/asm/vector.h | 25 + arch/riscv/include/asm/xor.h | 82 +++ arch/riscv/kernel/Makefile | 1 + arch/riscv/kernel/cpu.c | 9 + arch/riscv/kernel/cpufeature.c | 9 + arch/riscv/kernel/kernel_mode_vector.c | 132 +++++ arch/riscv/lib/Makefile | 1 + arch/riscv/lib/xor.S | 81 +++ crypto/Kconfig | 3 + 29 files changed, 4829 insertions(+) create mode 100644 arch/riscv/crypto/Kconfig create mode 100644 arch/riscv/crypto/Makefile create mode 100644 arch/riscv/crypto/aes-riscv-glue.c create mode 100644 arch/riscv/crypto/aes-riscv64-zvkned.pl create mode 100644 arch/riscv/crypto/ghash-riscv64-glue.c create mode 100644 arch/riscv/crypto/ghash-riscv64-zbc.pl create mode 100644 arch/riscv/crypto/ghash-riscv64-zvkb.pl create mode 100644 arch/riscv/crypto/ghash-riscv64-zvkg.pl create mode 100644 arch/riscv/crypto/riscv.pm create mode 100644 arch/riscv/crypto/sha256-riscv64-glue.c create mode 100644 arch/riscv/crypto/sha256-riscv64-zvknha.pl create mode 100644 arch/riscv/crypto/sha512-riscv64-glue.c create mode 100644 arch/riscv/crypto/sha512-riscv64-zvknhb.pl create mode 100644 arch/riscv/crypto/sm3-riscv64-glue.c create mode 100644 arch/riscv/crypto/sm3-riscv64-zvksh.pl create mode 100644 arch/riscv/crypto/sm4-riscv64-glue.c create mode 100644 arch/riscv/crypto/sm4-riscv64-zvksed.pl create mode 100644 arch/riscv/include/asm/xor.h create mode 100644 arch/riscv/kernel/kernel_mode_vector.c create mode 100644 arch/riscv/lib/xor.S