From patchwork Tue Dec 20 05:40:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 34906 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:e747:0:0:0:0:0 with SMTP id c7csp2788806wrn; Mon, 19 Dec 2022 21:45:56 -0800 (PST) X-Google-Smtp-Source: AA0mqf6+YCREfHBYuHjXKcSqRbHRVwXoo6dv8QM2fS4BqC2eUVjBGkXRzBFiE69ktf9Dtw9mI2Hw X-Received: by 2002:a17:902:e811:b0:189:d8fb:152d with SMTP id u17-20020a170902e81100b00189d8fb152dmr66990758plg.25.1671515156381; Mon, 19 Dec 2022 21:45:56 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1671515156; cv=none; d=google.com; s=arc-20160816; b=Udntd056aHFXBQdVrVq4pCQjc06lX9B0GHjVCToLSB6vAPrsd2IqAQ7BTqgzJYLcnE 4u/gKtdNAFBAahorILre8jfvBB72dSinESpo81hqI1GvP5xdzXjPyOMoFtLAO71iP0Wh x/Hq6MJDlOZaHJC8FxfEGM573aRvyn7viaFiqFi8tHvKPxOYQDH+3zEym5OMWowJwRz/ HMfECNgCin3LqrX6OYrum+k4RF+en5yV6WQvtofzoOSmcl/61W/iyhrjhL32hIClInnd BvMWZu1KebNMNCkuNzae+TDPvhK06Yaa7j5345i9rAvQz0kXQ6EQhHI9GyvtDSsq28SI kdbg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=j4bxzdAiF9gK3vn6VN8DUzGdwXvIyPjsQRBc11VRhU0=; b=ySwvvcuLxBKK/LiW/9ZAl+cRPVKvVFq6gHBrXPbhpKZFMivF6WXI/mn+iv0VAyj80+ Trr3l6uSyYzpks+SJLcy4aaU9z+aZ6yJip0lDt6GGQfSuYYSWQwWL673PglFKLAkO4E4 ezoRCcG0Q2K/CrdMpqlPnlmgJrXXyx5Caa3gGc0p2loAriOPzvoj/HXV6+1O/dNYklXu M0WnJExRZeXYfmVqU6CzeMpiteBrxXNVs2Bor9ixv/TmWbXYKoCW/s3u8BJnAX9Z69fd qivTlAwD7tQ8EKvAgPqdNQHVib4t8tOH8p7c8IambxZhCht0AW5wQcJ7ld1zDyrJJ+YQ o5fQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=IvUk8ADq; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id jb1-20020a170903258100b00189b421fff1si12356550plb.432.2022.12.19.21.45.43; Mon, 19 Dec 2022 21:45:56 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=IvUk8ADq; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233039AbiLTFmI (ORCPT + 99 others); Tue, 20 Dec 2022 00:42:08 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52116 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229895AbiLTFmB (ORCPT ); Tue, 20 Dec 2022 00:42:01 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 43E5AA188; Mon, 19 Dec 2022 21:42:00 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id E1E7BB811A3; Tue, 20 Dec 2022 05:41:58 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6F4F8C43398; Tue, 20 Dec 2022 05:41:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1671514917; bh=3n9ADRO4nwA4vng1EWokN1HkHn+cKE/ZleCXkk6FZuU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=IvUk8ADqzbVfp/roQjX87ypPf+5oAlzPl/WmfbZwTC7hJGifzKIvrISekXo1QbPwj 77veN4ECatl/Cp4R0djvSgbZtgQ55keX8ZIkZI6jwUwT7Rr+rlabmPl5waJUVMEYj6 XyAo0xmS282IUjrk6asDmm9DdzzAFQgTRU4HxJjUaPmxA1hb+Iu5q+ddQU8JBD6mm3 pmfhJ238h55NLqoCVkky8COjAQAbUY+kKd3+uo5qVrJratiTxIXlJhqeLW1lo+OA7M 9buDvhj1+x2uw3hIpNFy1RbrfxJv0tFYb3c95M8thLEx4T6NY9TZm8pqO+xvypQF+X XOzOZKCnjxGzQ== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: Peter Zijlstra , corbet@lwn.net, will@kernel.org, boqun.feng@gmail.com, mark.rutland@arm.com, catalin.marinas@arm.com, dennis@kernel.org, tj@kernel.org, cl@linux.com, hca@linux.ibm.com, gor@linux.ibm.com, agordeev@linux.ibm.com, borntraeger@linux.ibm.com, svens@linux.ibm.com, Herbert Xu , davem@davemloft.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, joro@8bytes.org, suravee.suthikulpanit@amd.com, robin.murphy@arm.com, dwmw2@infradead.org, baolu.lu@linux.intel.com, Arnd Bergmann , penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, Andrew Morton , vbabka@suse.cz, roman.gushchin@linux.dev, 42.hyeyoo@gmail.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-s390@vger.kernel.org, iommu@lists.linux.dev, linux-arch@vger.kernel.org Subject: [PATCH 1/3] crypto: x86/ghash - fix unaligned access in ghash_setkey() Date: Mon, 19 Dec 2022 21:40:40 -0800 Message-Id: <20221220054042.188537-2-ebiggers@kernel.org> X-Mailer: git-send-email 2.39.0 In-Reply-To: <20221220054042.188537-1-ebiggers@kernel.org> References: <20221220054042.188537-1-ebiggers@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1752710676402656750?= X-GMAIL-MSGID: =?utf-8?q?1752710676402656750?= From: Eric Biggers The key can be unaligned, so use the unaligned memory access helpers. Fixes: 8ceee72808d1 ("crypto: ghash-clmulni-intel - use C implementation for setkey()") Signed-off-by: Eric Biggers --- arch/x86/crypto/ghash-clmulni-intel_glue.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/arch/x86/crypto/ghash-clmulni-intel_glue.c b/arch/x86/crypto/ghash-clmulni-intel_glue.c index 1f1a95f3dd0c..c0ab0ff4af65 100644 --- a/arch/x86/crypto/ghash-clmulni-intel_glue.c +++ b/arch/x86/crypto/ghash-clmulni-intel_glue.c @@ -19,6 +19,7 @@ #include #include #include +#include #define GHASH_BLOCK_SIZE 16 #define GHASH_DIGEST_SIZE 16 @@ -54,15 +55,14 @@ static int ghash_setkey(struct crypto_shash *tfm, const u8 *key, unsigned int keylen) { struct ghash_ctx *ctx = crypto_shash_ctx(tfm); - be128 *x = (be128 *)key; u64 a, b; if (keylen != GHASH_BLOCK_SIZE) return -EINVAL; /* perform multiplication by 'x' in GF(2^128) */ - a = be64_to_cpu(x->a); - b = be64_to_cpu(x->b); + a = get_unaligned_be64(key); + b = get_unaligned_be64(key + 8); ctx->shash.a = (b << 1) | (a >> 63); ctx->shash.b = (a << 1) | (b >> 63); From patchwork Tue Dec 20 05:40:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 34907 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:e747:0:0:0:0:0 with SMTP id c7csp2788857wrn; Mon, 19 Dec 2022 21:46:09 -0800 (PST) X-Google-Smtp-Source: AA0mqf4yOtQ3x8l8WK9YshNRCz3cwgK4kzezJbIYZA2XYR9Clney2HtAoTSEDHC6q0OOhwAQYlJd X-Received: by 2002:a05:6a20:bf17:b0:ac:6543:d515 with SMTP id gc23-20020a056a20bf1700b000ac6543d515mr50388529pzb.42.1671515169259; Mon, 19 Dec 2022 21:46:09 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1671515169; cv=none; d=google.com; s=arc-20160816; b=aeXLzJaQCkFyJDynv3WehT/j8dezho64J30Oza4bvyUgPN5SPBnNJfee5eozQc1fm/ p+6ADWu7Lf9Q2t45LqE16Jaoew/j1hAFifxBEH0WPOVJWcZQvrktw6XdNJPJ1MJHcvg7 CdAfx0AZAOUng8o06i2UHCVDCjY377INHvehVvy6tvaOWNEkdOf+0lgpvhGjM19mENL4 0UbPF8yQiko9QA2jyeuDpWMvcmEOg+v1c90aIuax6nkahg3H9hNYUJ1IdXhSFYwZFspU 1B0V327wAF/kFN14U2hHA4tRe5+xHR/gzx8zaELswuw9eixJ74BYfavu5Xfn5ftOKg9W 69Hw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=9RW0us+nStEgHZTGP1VwlpkJJVA72mMv6v798BL7tQ8=; b=eu0/1wXPA6oCPAl616obd989J4Os87bB8UKpK4iS7M78L505+DWgRTNm4HuM+Zuwqt 9AaW072xjnNNLI76lwihPbFpD76j0lpBSwSdoND83L6rIZOaFeaf4+abvagrZ5iCqNzR jm4v/j+rqcHlYGClvermxEvta9v8yEz8r+D6+xinZDc+YXrqgCQVJjFJFsQSmbAhqGqr fW3wxC3zz6FwBoP8jNmE66B5kibCuPOESUfq0TOewxesLDuuBYKh5Pt3JPoJ16nbKMBJ y3pVikbGKhySiHfPS7wqaCthBC9v1VcGNTVy8/8NenITkD6OqHOS3gjgZUcI+H5x0xO3 PELg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=b4QPIeBD; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id i189-20020a639dc6000000b00478d39fcb3dsi13741887pgd.519.2022.12.19.21.45.55; Mon, 19 Dec 2022 21:46:09 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=b4QPIeBD; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232716AbiLTFmR (ORCPT + 99 others); Tue, 20 Dec 2022 00:42:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52126 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232844AbiLTFmC (ORCPT ); Tue, 20 Dec 2022 00:42:02 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3592595BC; Mon, 19 Dec 2022 21:42:00 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 5A14FB811AB; Tue, 20 Dec 2022 05:41:59 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C8630C433D2; Tue, 20 Dec 2022 05:41:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1671514919; bh=r7CcLqy65ywV0RdmBdnF4gvrkE0V7pqcwnuVzqA286k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=b4QPIeBDySeIw45tMI1nPwu6/LnNHxYWm2KdE0KmbJJA+QQ5NSMnGSgSTskJe068B a4O19R9sThYWaWHTRdN875o8NAIGdCht29RqD8fd3R9QJl5M5nPP2k0Qz/J13j+q9R szrJdxF/p54c3WPQPSOPRTgNaRp3zI0yGR3/pWPy5cZrzPmvfYUoTj8vlTrc+KfrLo r90Npm4NnbnGF2uk5qR5byeWzPI4ZzJhJp+zuPusgSYrt/dkFyuwP14LiGBH1ZF6YB FY0+pzVMYYmqqbGAPXLoFNY7XaYJRvV9podM+CJY4nTz7ktpEaNJDoh1F1ookspoH4 H6ZBz8YDu0PEw== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: Peter Zijlstra , corbet@lwn.net, will@kernel.org, boqun.feng@gmail.com, mark.rutland@arm.com, catalin.marinas@arm.com, dennis@kernel.org, tj@kernel.org, cl@linux.com, hca@linux.ibm.com, gor@linux.ibm.com, agordeev@linux.ibm.com, borntraeger@linux.ibm.com, svens@linux.ibm.com, Herbert Xu , davem@davemloft.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, joro@8bytes.org, suravee.suthikulpanit@amd.com, robin.murphy@arm.com, dwmw2@infradead.org, baolu.lu@linux.intel.com, Arnd Bergmann , penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, Andrew Morton , vbabka@suse.cz, roman.gushchin@linux.dev, 42.hyeyoo@gmail.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-s390@vger.kernel.org, iommu@lists.linux.dev, linux-arch@vger.kernel.org Subject: [PATCH 2/3] crypto: x86/ghash - use le128 instead of u128 Date: Mon, 19 Dec 2022 21:40:41 -0800 Message-Id: <20221220054042.188537-3-ebiggers@kernel.org> X-Mailer: git-send-email 2.39.0 In-Reply-To: <20221220054042.188537-1-ebiggers@kernel.org> References: <20221220054042.188537-1-ebiggers@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1752710689996147328?= X-GMAIL-MSGID: =?utf-8?q?1752710689996147328?= From: Eric Biggers The u128 struct type is going away, so make ghash-clmulni-intel use le128 instead. Note that the field names a and b swapped, as they were backwards with u128. (a is meant to be high-order and b low-order.) Signed-off-by: Eric Biggers --- arch/x86/crypto/ghash-clmulni-intel_asm.S | 4 ++-- arch/x86/crypto/ghash-clmulni-intel_glue.c | 12 ++++++------ 2 files changed, 8 insertions(+), 8 deletions(-) diff --git a/arch/x86/crypto/ghash-clmulni-intel_asm.S b/arch/x86/crypto/ghash-clmulni-intel_asm.S index 2bf871899920..9dfeb4d31b92 100644 --- a/arch/x86/crypto/ghash-clmulni-intel_asm.S +++ b/arch/x86/crypto/ghash-clmulni-intel_asm.S @@ -88,7 +88,7 @@ SYM_FUNC_START_LOCAL(__clmul_gf128mul_ble) RET SYM_FUNC_END(__clmul_gf128mul_ble) -/* void clmul_ghash_mul(char *dst, const u128 *shash) */ +/* void clmul_ghash_mul(char *dst, const le128 *shash) */ SYM_FUNC_START(clmul_ghash_mul) FRAME_BEGIN movups (%rdi), DATA @@ -104,7 +104,7 @@ SYM_FUNC_END(clmul_ghash_mul) /* * void clmul_ghash_update(char *dst, const char *src, unsigned int srclen, - * const u128 *shash); + * const le128 *shash); */ SYM_FUNC_START(clmul_ghash_update) FRAME_BEGIN diff --git a/arch/x86/crypto/ghash-clmulni-intel_glue.c b/arch/x86/crypto/ghash-clmulni-intel_glue.c index c0ab0ff4af65..9453b094bb3b 100644 --- a/arch/x86/crypto/ghash-clmulni-intel_glue.c +++ b/arch/x86/crypto/ghash-clmulni-intel_glue.c @@ -24,17 +24,17 @@ #define GHASH_BLOCK_SIZE 16 #define GHASH_DIGEST_SIZE 16 -void clmul_ghash_mul(char *dst, const u128 *shash); +void clmul_ghash_mul(char *dst, const le128 *shash); void clmul_ghash_update(char *dst, const char *src, unsigned int srclen, - const u128 *shash); + const le128 *shash); struct ghash_async_ctx { struct cryptd_ahash *cryptd_tfm; }; struct ghash_ctx { - u128 shash; + le128 shash; }; struct ghash_desc_ctx { @@ -64,11 +64,11 @@ static int ghash_setkey(struct crypto_shash *tfm, a = get_unaligned_be64(key); b = get_unaligned_be64(key + 8); - ctx->shash.a = (b << 1) | (a >> 63); - ctx->shash.b = (a << 1) | (b >> 63); + ctx->shash.a = cpu_to_le64((a << 1) | (b >> 63)); + ctx->shash.b = cpu_to_le64((b << 1) | (a >> 63)); if (a >> 63) - ctx->shash.b ^= ((u64)0xc2) << 56; + ctx->shash.a ^= cpu_to_le64((u64)0xc2 << 56); return 0; } From patchwork Tue Dec 20 05:40:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 34908 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:e747:0:0:0:0:0 with SMTP id c7csp2789024wrn; Mon, 19 Dec 2022 21:46:53 -0800 (PST) X-Google-Smtp-Source: AA0mqf7FoEgZyG4aQcbGY1c1KZ9Xd8oZExZapQOuuxP/KBtik4O1m1DdyJffnKxkB9V1xtmTNS8Z X-Received: by 2002:a17:902:cec1:b0:189:cef2:88e3 with SMTP id d1-20020a170902cec100b00189cef288e3mr63522345plg.57.1671515213116; Mon, 19 Dec 2022 21:46:53 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1671515213; cv=none; d=google.com; s=arc-20160816; b=GUBZlJAPPQHZhxI67/v5ApgWYVFeFtDmnmeTOHKyIBjp7KFoo2+yW0bxEgw4Dp9uIb zz/He/bNk63G9n3dO7ZtN8/F2xoIsi8NtIV3M+cwiqPmtLWwuRCWQQaXVym2/UeHQ9VB QioKc72q1Zbl/wbc7IxaXbrdZhkRc/inFxultLXggl2ZImofBRxuC0NZoMenuxHX43Dj 8hSOhOdjes002YEhmJ8gzr4esv6SjFrnX1qMZjX8fVnMEjZ2b6OwQnZifl0qfs/gGP7N 8sJqN69Y6S1tVkMIoK6bUyunETNuR0BpDB7AcXulYpyi1CL+z03kDcYXb/SzjrJ1p/Ea I/7Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=nOYXpDiFyQEKhsLgatHAwSyheaaratVquM7fRx8vRaI=; b=cmRyoucKr5vvHhvhP1RUb7sNjcSsfu7Xby9ECWu6hPfdWcIVcMZUISDAID2X9uPFiD d35uAGfAd83iV2pqM1k4fPCeUJqn2HIeVrDiikd9XEbnqw8Mi7w7gtGF+GTcnmxmgesh u9fi1ao1dsuceNv+wSfrtv6Ga8t8F/0a5q73oqIzsxafCqHHfKBLpPuVJuowxQyfqG4E 1kHHRkei1n2ZCKrDJ9LgO6rr3kqLSikJxrAHKoiBFQs1YH6Y4KpTYo2g0SHgaCOagmQM 3Fq5+5dlDwAznqxVn9Vz+srguPbcNsNWSgA4oEpGdzlhQHBAjNEKd4iIWhT72i8ErMuS Qykw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=jYOojsuB; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id a5-20020a170903100500b001910f9d150dsi8372095plb.335.2022.12.19.21.46.40; Mon, 19 Dec 2022 21:46:53 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=jYOojsuB; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233064AbiLTFmN (ORCPT + 99 others); Tue, 20 Dec 2022 00:42:13 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52128 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232853AbiLTFmC (ORCPT ); Tue, 20 Dec 2022 00:42:02 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 701AA10B56; Mon, 19 Dec 2022 21:42:01 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 099F561284; Tue, 20 Dec 2022 05:42:01 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3059AC433A0; Tue, 20 Dec 2022 05:41:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1671514920; bh=dsBT80rMbHb7evBtDRzv5PT3ZA7g7MTpg/OO26ExzF0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jYOojsuBCSLHGr8TW6ECUwgWw+LEvw+PHi3nql2vBWzBzEq7MyKtVOLJ/b2Kiu4gS oMG/dlg3OnMmVJmrdSv2oL97DF/CUUglF7nDPoQjrR1EUjWMzPIKMXZdHUPv8l8Sjd UXS+WUfMJ0NGEcylNFSYCWZt2yo1UQq4aV3vPbfVoQuD+Qp8TbeWtUxBsxReFPOxlA zfs32JevZlz+K+Uv5duClJslwi2mYsu29ttlLiM4znIDXOp4GXmEIq6usK1oiCT/rX LwdwbARx6KolYJUmb/r6mqXzDaBmbWq87tBtJKhhILBQJc9wo5CJtT/46YobC23TXN w7YoOtqqwQbVA== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: Peter Zijlstra , corbet@lwn.net, will@kernel.org, boqun.feng@gmail.com, mark.rutland@arm.com, catalin.marinas@arm.com, dennis@kernel.org, tj@kernel.org, cl@linux.com, hca@linux.ibm.com, gor@linux.ibm.com, agordeev@linux.ibm.com, borntraeger@linux.ibm.com, svens@linux.ibm.com, Herbert Xu , davem@davemloft.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, joro@8bytes.org, suravee.suthikulpanit@amd.com, robin.murphy@arm.com, dwmw2@infradead.org, baolu.lu@linux.intel.com, Arnd Bergmann , penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, Andrew Morton , vbabka@suse.cz, roman.gushchin@linux.dev, 42.hyeyoo@gmail.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-s390@vger.kernel.org, iommu@lists.linux.dev, linux-arch@vger.kernel.org Subject: [PATCH 3/3] crypto: x86/ghash - add comment and fix broken link Date: Mon, 19 Dec 2022 21:40:42 -0800 Message-Id: <20221220054042.188537-4-ebiggers@kernel.org> X-Mailer: git-send-email 2.39.0 In-Reply-To: <20221220054042.188537-1-ebiggers@kernel.org> References: <20221220054042.188537-1-ebiggers@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1752710735851190417?= X-GMAIL-MSGID: =?utf-8?q?1752710735851190417?= From: Eric Biggers Add a comment that explains what ghash_setkey() is doing, as it's hard to understand otherwise. Also fix a broken hyperlink. Signed-off-by: Eric Biggers --- arch/x86/crypto/ghash-clmulni-intel_asm.S | 2 +- arch/x86/crypto/ghash-clmulni-intel_glue.c | 27 ++++++++++++++++++---- 2 files changed, 24 insertions(+), 5 deletions(-) diff --git a/arch/x86/crypto/ghash-clmulni-intel_asm.S b/arch/x86/crypto/ghash-clmulni-intel_asm.S index 9dfeb4d31b92..257ed9446f3e 100644 --- a/arch/x86/crypto/ghash-clmulni-intel_asm.S +++ b/arch/x86/crypto/ghash-clmulni-intel_asm.S @@ -4,7 +4,7 @@ * instructions. This file contains accelerated part of ghash * implementation. More information about PCLMULQDQ can be found at: * - * http://software.intel.com/en-us/articles/carry-less-multiplication-and-its-usage-for-computing-the-gcm-mode/ + * https://www.intel.com/content/dam/develop/external/us/en/documents/clmul-wp-rev-2-02-2014-04-20.pdf * * Copyright (c) 2009 Intel Corp. * Author: Huang Ying diff --git a/arch/x86/crypto/ghash-clmulni-intel_glue.c b/arch/x86/crypto/ghash-clmulni-intel_glue.c index 9453b094bb3b..700ecaee9a08 100644 --- a/arch/x86/crypto/ghash-clmulni-intel_glue.c +++ b/arch/x86/crypto/ghash-clmulni-intel_glue.c @@ -60,16 +60,35 @@ static int ghash_setkey(struct crypto_shash *tfm, if (keylen != GHASH_BLOCK_SIZE) return -EINVAL; - /* perform multiplication by 'x' in GF(2^128) */ + /* + * GHASH maps bits to polynomial coefficients backwards, which makes it + * hard to implement. But it can be shown that the GHASH multiplication + * + * D * K (mod x^128 + x^7 + x^2 + x + 1) + * + * (where D is a data block and K is the key) is equivalent to: + * + * bitreflect(D) * bitreflect(K) * x^(-127) + * (mod x^128 + x^127 + x^126 + x^121 + 1) + * + * So, the code below precomputes: + * + * bitreflect(K) * x^(-127) (mod x^128 + x^127 + x^126 + x^121 + 1) + * + * ... but in Montgomery form (so that Montgomery multiplication can be + * used), i.e. with an extra x^128 factor, which means actually: + * + * bitreflect(K) * x (mod x^128 + x^127 + x^126 + x^121 + 1) + * + * The within-a-byte part of bitreflect() cancels out GHASH's built-in + * reflection, and thus bitreflect() is actually a byteswap. + */ a = get_unaligned_be64(key); b = get_unaligned_be64(key + 8); - ctx->shash.a = cpu_to_le64((a << 1) | (b >> 63)); ctx->shash.b = cpu_to_le64((b << 1) | (a >> 63)); - if (a >> 63) ctx->shash.a ^= cpu_to_le64((u64)0xc2 << 56); - return 0; }