From patchwork Tue Oct 18 21:56:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nathan Huckleberry X-Patchwork-Id: 4341 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:3c08:b0:7f:eb39:1b51 with SMTP id e8csp3543130dys; Tue, 18 Oct 2022 14:58:08 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5Ivj2ACF5tj/p1azNb6290b6oHFBZhyu++BFO/bw6dm7FcCqXa2z2u7pOuaMlsnIcp1VzQ X-Received: by 2002:a05:6402:500d:b0:459:3e56:e6f9 with SMTP id p13-20020a056402500d00b004593e56e6f9mr4629539eda.367.1666130288205; Tue, 18 Oct 2022 14:58:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666130288; cv=none; d=google.com; s=arc-20160816; b=kYXQrBtuFahWIw9N0ZtNFb7y4wpDMG9g8N01apuj9uDCOU73yz95ww778HoMKtVt1i NzUHzd/tvc97oTVgFpUJ1IRc7pfgw3j2t/2N5v3EgiqnSPt+6MmqTx7Mt6X2dmoYSa/M myGuCWCGXJ4T9ae84AMB0mwdtqrEDZaF1vnKK1rcHGkRuLPpkuG4yXbhqHOfjsYWHbRZ OhdZPhhwOW940+yzbg0UnJpdhYl4CLVHXdj8qL2n506ZPXRfojuPUKV0lol06O+Qe8Vx 78GDX25IXs7iF93DGMciPn69QFc8iJEroF88O1jQj+8Kny4nhyRVqfqd6Z6EMNEznK7T 3IyA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=m7J+/b5/L0nQHPU1N8gvtVOb8AwUDuLMMuB/44yFdGw=; b=llU5ITiKAAa3g+kdwOki06Q0Fa4FCHBHqbflb3eP9/z0tPZnGGSrfjCOS4n5jXIi7d uOPKm6cEA8CNObsSOO3MOEMCDOxjtFOzVDmndoUq7r1XW8O6jnyLWWfFxv4YVEPVX1HW /rJPRuXnw2oA5xbqJ5Sv7igTX6o9RoWOmcWjaPeen/TXOP+MkyGzaElUdX4M5HFfEmYf t5exorFZAXPMGgXPvuKFgFecvrydkWxUzHToMKMkUV5FN0QYSTvUgyAJ6lBsVmYR6lKN gb0JEsX9PeXr9sNFV1T0amQ2pG77y9x95H8sctX6kVojaYyMRwt/mILJ3ouqrJ4Hoalp MHfg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=mEO2pPTx; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id qa44-20020a17090786ac00b0078a10c159b4si12858753ejc.290.2022.10.18.14.57.43; Tue, 18 Oct 2022 14:58:08 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=mEO2pPTx; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229741AbiJRV4n (ORCPT + 99 others); Tue, 18 Oct 2022 17:56:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45742 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229506AbiJRV4l (ORCPT ); Tue, 18 Oct 2022 17:56:41 -0400 Received: from mail-ua1-x949.google.com (mail-ua1-x949.google.com [IPv6:2607:f8b0:4864:20::949]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3BE4C7FE63 for ; Tue, 18 Oct 2022 14:56:40 -0700 (PDT) Received: by mail-ua1-x949.google.com with SMTP id h11-20020ab0470b000000b003bf1da44886so6337441uac.17 for ; Tue, 18 Oct 2022 14:56:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=m7J+/b5/L0nQHPU1N8gvtVOb8AwUDuLMMuB/44yFdGw=; b=mEO2pPTxKmNgWoZtl0QxarEGwuWxwB5IlUJOcOiG99LrRsWMdv5m6glj1IUBip4WCN 5suO/47EjfF9pPx4xUfqzVWXv07PSgNVIFbPKvRfR4OjyQM87lxjnbKfwCEvjquoKZJ6 J0eviU7IkE+awbL7gWM/iQRVo5YQAFNQdCaFXY1nehdOSxffeuH6RBSX2/OlNSTsCiHS WnUKkhIoBHSXMv87CJsewUE/kkuIhsYEqZ9j6aSU73ryyMG6Yce0hc1+tSb00Tq2kRuu aWALp+a/U1w13NJXPFLRWgaVoe5yMWNMJVb9ioj2LOy1fTTmTRt9A4wKemWOPLDcrzrf 8GVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=m7J+/b5/L0nQHPU1N8gvtVOb8AwUDuLMMuB/44yFdGw=; b=EW6oKcoetmOZj2Hq9MIDI2XspiCnZPVK8DzMPtwDUby5ln42EFEFBMIrjl3ghVrqYJ GyBEB0oqHSpTAq6GM/aOBleGvi8Vmb4bxEmG4Vmh0w51kR2uGhTQel+GRsHt8QPxRWZ3 Z+ok9zv5Qt91gCHQy8qtR44ILMckGvBaQDK/yBRXygM0TU8ZL5KD38P4TWJpyocSsjc1 iDX+ILkCQNtyfJu1m1ENwGhszDmNeQlVvjVRxHqx41qvnJxnGYoI0BIAOGVe69Dnavyg 6y8+DuU9EYY9ORL9TCW8Zjmenr4Xw2mTdf4BVFCgT9PYjtqb1118T7rvIq2QB/WWSd5l iiXA== X-Gm-Message-State: ACrzQf328lDW/2vp5D6/vobkUtwqe+cEJObMORKdShP9D6wd57vwLMJl O/YXnnY34OyxUrPlG4iBlCdWMBSw6A== X-Received: from nhuck.c.googlers.com ([fda3:e722:ac3:cc00:14:4d90:c0a8:39cc]) (user=nhuck job=sendgmr) by 2002:a05:6122:4f8:b0:3ae:21f7:28da with SMTP id s24-20020a05612204f800b003ae21f728damr2502221vkf.33.1666130199389; Tue, 18 Oct 2022 14:56:39 -0700 (PDT) Date: Tue, 18 Oct 2022 14:56:23 -0700 In-Reply-To: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.38.0.413.g74048e4d9e-goog Message-ID: <20221018215623.866014-1-nhuck@google.com> Subject: [PATCH v2] crypto: x86/polyval - Fix crashes when keys are not 16-byte aligned From: Nathan Huckleberry To: herbert@gondor.apana.org.au Cc: ardb@kernel.org, bgoncalv@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, davem@davemloft.net, ebiggers@kernel.org, hpa@zytor.com, linux-crypto@vger.kernel.org, linux-kernel@vger.kernel.org, mingo@redhat.com, nhuck@google.com, tglx@linutronix.de, x86@kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1746975624287711144?= X-GMAIL-MSGID: =?utf-8?q?1747064233301407140?= crypto_tfm::__crt_ctx is not guaranteed to be 16-byte aligned on x86-64. This causes crashes due to movaps instructions in clmul_polyval_update. Add logic to align polyval_tfm_ctx to 16 bytes if required. Fixes: 34f7f6c30112 ("crypto: x86/polyval - Add PCLMULQDQ accelerated implementation of POLYVAL") Reported-by: Bruno Goncalves Signed-off-by: Nathan Huckleberry --- arch/x86/crypto/polyval-clmulni_glue.c | 29 ++++++++++++++++++++------ 1 file changed, 23 insertions(+), 6 deletions(-) diff --git a/arch/x86/crypto/polyval-clmulni_glue.c b/arch/x86/crypto/polyval-clmulni_glue.c index b7664d018851..79ad497c32b5 100644 --- a/arch/x86/crypto/polyval-clmulni_glue.c +++ b/arch/x86/crypto/polyval-clmulni_glue.c @@ -27,13 +27,17 @@ #include #include +#define POLYVAL_ALIGN 16 +#define POLYVAL_ALIGN_ATTR __aligned(POLYVAL_ALIGN) +#define POLYVAL_ALIGN_EXTRA ((POLYVAL_ALIGN - 1) & ~(CRYPTO_MINALIGN - 1)) +#define POLYVAL_CTX_SIZE (sizeof(struct polyval_tfm_ctx) + POLYVAL_ALIGN_EXTRA) #define NUM_KEY_POWERS 8 struct polyval_tfm_ctx { /* * These powers must be in the order h^8, ..., h^1. */ - u8 key_powers[NUM_KEY_POWERS][POLYVAL_BLOCK_SIZE]; + u8 key_powers[NUM_KEY_POWERS][POLYVAL_BLOCK_SIZE] POLYVAL_ALIGN_ATTR; }; struct polyval_desc_ctx { @@ -45,9 +49,20 @@ asmlinkage void clmul_polyval_update(const struct polyval_tfm_ctx *keys, const u8 *in, size_t nblocks, u8 *accumulator); asmlinkage void clmul_polyval_mul(u8 *op1, const u8 *op2); -static void internal_polyval_update(const struct polyval_tfm_ctx *keys, +static inline struct polyval_tfm_ctx *polyval_tfm_ctx(const void *raw_ctx) +{ + unsigned long addr = (unsigned long)raw_ctx; + unsigned long align = POLYVAL_ALIGN; + + if (align <= crypto_tfm_ctx_alignment()) + align = 1; + return (struct polyval_tfm_ctx *)ALIGN(addr, align); +} + +static void internal_polyval_update(const void *raw_keys, const u8 *in, size_t nblocks, u8 *accumulator) { + const struct polyval_tfm_ctx *keys = polyval_tfm_ctx(raw_keys); if (likely(crypto_simd_usable())) { kernel_fpu_begin(); clmul_polyval_update(keys, in, nblocks, accumulator); @@ -72,7 +87,7 @@ static void internal_polyval_mul(u8 *op1, const u8 *op2) static int polyval_x86_setkey(struct crypto_shash *tfm, const u8 *key, unsigned int keylen) { - struct polyval_tfm_ctx *tctx = crypto_shash_ctx(tfm); + struct polyval_tfm_ctx *tctx = polyval_tfm_ctx(crypto_shash_ctx(tfm)); int i; if (keylen != POLYVAL_BLOCK_SIZE) @@ -102,7 +117,8 @@ static int polyval_x86_update(struct shash_desc *desc, const u8 *src, unsigned int srclen) { struct polyval_desc_ctx *dctx = shash_desc_ctx(desc); - const struct polyval_tfm_ctx *tctx = crypto_shash_ctx(desc->tfm); + const struct polyval_tfm_ctx *tctx = + polyval_tfm_ctx(crypto_shash_ctx(desc->tfm)); u8 *pos; unsigned int nblocks; unsigned int n; @@ -143,7 +159,8 @@ static int polyval_x86_update(struct shash_desc *desc, static int polyval_x86_final(struct shash_desc *desc, u8 *dst) { struct polyval_desc_ctx *dctx = shash_desc_ctx(desc); - const struct polyval_tfm_ctx *tctx = crypto_shash_ctx(desc->tfm); + const struct polyval_tfm_ctx *tctx = + polyval_tfm_ctx(crypto_shash_ctx(desc->tfm)); if (dctx->bytes) { internal_polyval_mul(dctx->buffer, @@ -167,7 +184,7 @@ static struct shash_alg polyval_alg = { .cra_driver_name = "polyval-clmulni", .cra_priority = 200, .cra_blocksize = POLYVAL_BLOCK_SIZE, - .cra_ctxsize = sizeof(struct polyval_tfm_ctx), + .cra_ctxsize = POLYVAL_CTX_SIZE, .cra_module = THIS_MODULE, }, };