From patchwork Mon Jan 16 14:59:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Lafreniere X-Patchwork-Id: 44221 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp1249057wrn; Mon, 16 Jan 2023 07:25:01 -0800 (PST) X-Google-Smtp-Source: AMrXdXucEkM7zvFmzbjUVSyNk/2NWE6KR4lpaShuu+KXHRTOw088JmvVADfsHoLwCq3nPOCnT3ob X-Received: by 2002:a05:6a00:4515:b0:575:d06d:1bfa with SMTP id cw21-20020a056a00451500b00575d06d1bfamr16548pfb.2.1673882701157; Mon, 16 Jan 2023 07:25:01 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1673882701; cv=none; d=google.com; s=arc-20160816; b=YxgFhLTAPn4asJKUkoBO8t7rO+/oxqBrad8xcMjJOGCBFtnasa4YVHYQnsqgJ8pdPx lOJs0lW1i2Dc0LCFAIV39GTdrdJejMsdgj7uPjhLcenfR6L0lwOQ1VGHXDg/6W3XXr9X Ez1jG6IAfzTgSsze3gX79VMZa3TQJfhTlNdvgiIYU3j2EDiQgJ4ozmluMu99M3IgWme7 gd5LXWLK1j5c8SQpcY0Rwx3ydsk/YCKEDbGJ047XWBGvrGIetmqGIqfIwb2bI+eizsiN Xjvf3mDeWJbdd5poB+pQf9dVYgfeJRBy2CcZUNy46cQzBI8Dqe8LF7EfKbj31Ye4xDDe TNxw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :feedback-id:message-id:subject:cc:from:to:dkim-signature:date; bh=zzF9mAQN1tOBRS7sao22BNfZIlVCplVmqFFZ1O+i9rQ=; b=ohDeqESibDwELmtekmE56mBas206NlZxW89wmwPxfbi/R++XBhmM+qAUgNMo5L8B0T IKIvgwRMld/IlIsUxmOFheoEWNSAvVvJSm6SonMAq8p7IU/rubfkOGYtO6chLX0eEj6T RTjAE3wck34l7KFVYUJX4Uh+3d5N5ndS+xMd3xNk++goTt1DeUhlY+RHtDVgdNaC7PrW iYDURgbY4TyVgp7M6vjKi8ZOnRL6MYFvxDguXCWn3pOFMERC/xvVZyE/QvyURluTLfFV is0411XXGjog+C9PPMzf3h75InIU7s7VxmLJKLj90tm3G46EfhMwAk8gXp0cnnohMK+h nnXg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@n8pjl.ca header.s=protonmail header.b=PeWRTFyZ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=n8pjl.ca Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id v3-20020a056a00148300b00575428b9ea7si30137974pfu.204.2023.01.16.07.24.48; Mon, 16 Jan 2023 07:25:01 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@n8pjl.ca header.s=protonmail header.b=PeWRTFyZ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=n8pjl.ca Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229577AbjAPPMN (ORCPT + 99 others); Mon, 16 Jan 2023 10:12:13 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37710 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232239AbjAPPLq (ORCPT ); Mon, 16 Jan 2023 10:11:46 -0500 Received: from mail-4323.proton.ch (mail-4323.proton.ch [185.70.43.23]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4AC04301BF for ; Mon, 16 Jan 2023 06:59:58 -0800 (PST) Date: Mon, 16 Jan 2023 14:59:46 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=n8pjl.ca; s=protonmail; t=1673881196; x=1674140396; bh=zzF9mAQN1tOBRS7sao22BNfZIlVCplVmqFFZ1O+i9rQ=; h=Date:To:From:Cc:Subject:Message-ID:Feedback-ID:From:To:Cc:Date: Subject:Reply-To:Feedback-ID:Message-ID:BIMI-Selector; b=PeWRTFyZ5oaod8KIEvldYz8XEm18qXWxt/IZefcD5a+Y/YTjdCq1zkY+4KeEEz8PC vX4ti3EzR1zcuC6fkj3Pge1gIAHhHzbW9+DH7jl+TzK13q7hwiS+MQEyWvYWiz5poq urQ3kcCoT1rzfhzh5f2dKN1J1qTerMlJZkqf2wOWPp0O7o2sXQbY+0VqzZn2seNdrd 2G/UfDWw9OY991VbA+GX35XY4icIf5elnTaVpDHqgeQs2/csr9UVq6KYWRZuczlU5/ +Ta0vNF0xeDFPKMwR7/cYBzEZne8HkH4g6f15DQrJq3FTilfyGvcFqPMMwQc8IJyVo 2cikKCqAFtEuw== To: "linux-crypto@vger.kernel.org" From: Peter Lafreniere Cc: "ardb@kernel.org" , "x86@kernel.org" , "linux-kernel@vger.kernel.org" , Peter Lafreniere Subject: [PATCH] crypto: x86 - exit fpu context earlier in ECB/CBC macros Message-ID: Feedback-ID: 53133685:user:proton MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS, SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1755193226955649146?= X-GMAIL-MSGID: =?utf-8?q?1755193226955649146?= Currently the ecb/cbc macros hold fpu context unnecessarily when using scalar cipher routines (e.g. when handling odd sizes of blocks per walk). Change the macros to drop fpu context as soon as the fpu is out of use. No performance impact found (on Intel Haswell). Signed-off-by: Peter Lafreniere --- arch/x86/crypto/ecb_cbc_helpers.h | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-) diff --git a/arch/x86/crypto/ecb_cbc_helpers.h b/arch/x86/crypto/ecb_cbc_helpers.h index eaa15c7b29d6..b83085e18ab0 100644 --- a/arch/x86/crypto/ecb_cbc_helpers.h +++ b/arch/x86/crypto/ecb_cbc_helpers.h @@ -14,12 +14,13 @@ #define ECB_WALK_START(req, bsize, fpu_blocks) do { \ void *ctx = crypto_skcipher_ctx(crypto_skcipher_reqtfm(req)); \ + const int __fpu_blocks = (fpu_blocks); \ const int __bsize = (bsize); \ struct skcipher_walk walk; \ int err = skcipher_walk_virt(&walk, (req), false); \ while (walk.nbytes > 0) { \ unsigned int nbytes = walk.nbytes; \ - bool do_fpu = (fpu_blocks) != -1 && \ - nbytes >= (fpu_blocks) * __bsize; \ + bool do_fpu = __fpu_blocks != -1 && \ + nbytes >= __fpu_blocks * __bsize; \ const u8 *src = walk.src.virt.addr; \ u8 *dst = walk.dst.virt.addr; \ u8 __maybe_unused buf[(bsize)]; \ @@ -35,7 +36,12 @@ } while (0) #define ECB_BLOCK(blocks, func) do { \ - while (nbytes >= (blocks) * __bsize) { \ + const int __blocks = (blocks); \ + if (do_fpu && __blocks < __fpu_blocks) { \ + kernel_fpu_end(); \ + do_fpu = false; \ + } \ + while (nbytes >= __blocks * __bsize) { \ (func)(ctx, dst, src); \ ECB_WALK_ADVANCE(blocks); \ } \ @@ -53,7 +59,12 @@ } while (0) #define CBC_DEC_BLOCK(blocks, func) do { \ - while (nbytes >= (blocks) * __bsize) { \ + const int __blocks = (blocks); \ + if (do_fpu && __blocks < __fpu_blocks) { \ + kernel_fpu_end(); \ + do_fpu = false; \ + } \ + while (nbytes >= __blocks * __bsize) { \ const u8 *__iv = src + ((blocks) - 1) * __bsize; \ if (dst == src) \ __iv = memcpy(buf, __iv, __bsize); \