From patchwork Tue Nov 1 12:25:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Jason A. Donenfeld" X-Patchwork-Id: 13645 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp2932450wru; Tue, 1 Nov 2022 05:33:35 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6tS97fnoB8UWJaww/iA1r8GKrlmePrO3cU4joUxYeQRIqrQ5Wa3rZdahvZqBf9Me601aXb X-Received: by 2002:a63:2d05:0:b0:460:55e3:df91 with SMTP id t5-20020a632d05000000b0046055e3df91mr17108988pgt.177.1667306015269; Tue, 01 Nov 2022 05:33:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667306015; cv=none; d=google.com; s=arc-20160816; b=Nj3WfnpUt/vCY8SsdSWoh701ZgWSapi7LLgh3XD3aPSOW0cVvXt2ngTH8zSrtUpMot zppiwS705JjQ1/Q76/ifNGJGqqrJpUWbLWshBc9CACeIOPChmlp9NohF0e3zgFUDsJFX MmzDbyOxOIDEhySr2Bg2UQ88eP7LuveDbrDOtXcOjM5AMQSYK5cURkiPbqI0xDDSVlFi axofROWk7cLS/b12175pDkfl+z/O3B/Oq8EjOHmUzOg7xxIEH+Lzg6KaJXK359U/zTt/ 9pdHX8uXQgagsntA5kXdNAPZX18erfTU9MLp9YvmOnwgePwlrjuYMolCUi3IC/DfZQoT aZuQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=dtfQi2pNPpPZAq4SXqmk5xdo+UCS3l6ora/p900eBzs=; b=wd21ldQr5yFkC32OkEMsxK+xODlvW3ISA5jL81bv0Dix/ZLwAbwKo/gVF+1/OzA2qv SYTnNvICk0pVlZTNC+6sZ+qEPn5c26aKaoG5EcFDDD3r+NJZjZ8y6k/RpR0r/K4CuTsU +bxAYCYFon4VxrD2UVEpkdsVeGsPiyhNB3cWnMRJsIzY5xa1ekYfouV8pHyIuJvSNHiR 44AhhY9HlbllisAphuVDb4SbBOWeAxULzUsNBhdB97ba1S20Q/nizQdQW3q2gufE0NlB RSpHb82pkIYHrVZ2g0+vOvsXs6rE9SxlukEpImLwznM7pT0T6b3+bZNA0GIpv/uV8FLt xO5Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@zx2c4.com header.s=20210105 header.b="F145xF3/"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=zx2c4.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id p4-20020a635b04000000b0046af0960898si11037103pgb.625.2022.11.01.05.33.21; Tue, 01 Nov 2022 05:33:35 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@zx2c4.com header.s=20210105 header.b="F145xF3/"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=zx2c4.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230003AbiKAMZq (ORCPT + 99 others); Tue, 1 Nov 2022 08:25:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37058 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229587AbiKAMZp (ORCPT ); Tue, 1 Nov 2022 08:25:45 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 573DE17A90 for ; Tue, 1 Nov 2022 05:25:44 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id E4B9C60F7B for ; Tue, 1 Nov 2022 12:25:43 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 76DB5C433C1; Tue, 1 Nov 2022 12:25:42 +0000 (UTC) Authentication-Results: smtp.kernel.org; dkim=pass (1024-bit key) header.d=zx2c4.com header.i=@zx2c4.com header.b="F145xF3/" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zx2c4.com; s=20210105; t=1667305540; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dtfQi2pNPpPZAq4SXqmk5xdo+UCS3l6ora/p900eBzs=; b=F145xF3/7SR6VO8DVAkXaj/S/hnwyBS8+kUCvMKlb9HvLm4LrEtK8Irs3tnBzfS8WY4n1C dxYKdpF55h22TK2Sjl/Jp6vi0D6ziMHg7pcvTIx87ExdNZNMtrcfokJoewfDAPLArGMO1s nyNMnev/vi5W06XLiILhg7gZf7gZ9FQ= Received: by mail.zx2c4.com (ZX2C4 Mail Server) with ESMTPSA id b5b19abf (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Tue, 1 Nov 2022 12:25:40 +0000 (UTC) From: "Jason A. Donenfeld" To: Catalin Marinas , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: "Jason A. Donenfeld" , Will Deacon , Ard Biesheuvel , Jean-Philippe Brucker Subject: [PATCH v5] random: remove early archrandom abstraction Date: Tue, 1 Nov 2022 13:25:28 +0100 Message-Id: <20221101122527.323843-1-Jason@zx2c4.com> In-Reply-To: <20221101115616.232884-1-Jason@zx2c4.com> References: <20221101115616.232884-1-Jason@zx2c4.com> MIME-Version: 1.0 X-Spam-Status: No, score=-6.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_HI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747977860464325685?= X-GMAIL-MSGID: =?utf-8?q?1748297072664289528?= The arch_get_random*_early() abstraction is not completely useful and adds complexity, because it's not a given that there will be no calls to arch_get_random*() between random_init_early(), which uses arch_get_random*_early(), and init_cpu_features(). During that gap, crng_reseed() might be called, which uses arch_get_random*(), since it's mostly not init code. Instead we can test whether we're in the early phase in arch_get_random*() itself, and in doing so avoid all ambiguity about where we are. Fortunately, the only architecture that currently implements arch_get_random*_early() also has an alternatives-based cpu feature system, one flag of which determines whether the other flags have been initialized. This makes it possible to do the early check with zero cost once the system is initialized. Cc: Catalin Marinas Cc: Will Deacon Cc: Ard Biesheuvel Cc: Jean-Philippe Brucker Signed-off-by: Jason A. Donenfeld Reviewed-by: Catalin Marinas --- Changes v4->v5: - Use this_cpu_has_cap() to make codegen for the cold path smaller. - Add manual unlikely() to make the coldpath that's patched out even further away. Changes v3->v4: - Use !preemptible() rather than early_boot_irqs_disabled. arch/arm64/include/asm/archrandom.h | 48 ++++++----------------------- drivers/char/random.c | 4 +-- include/linux/random.h | 20 ------------ 3 files changed, 12 insertions(+), 60 deletions(-) diff --git a/arch/arm64/include/asm/archrandom.h b/arch/arm64/include/asm/archrandom.h index 109e2a4454be..2f5f3da34782 100644 --- a/arch/arm64/include/asm/archrandom.h +++ b/arch/arm64/include/asm/archrandom.h @@ -5,6 +5,7 @@ #include #include #include +#include #include #define ARM_SMCCC_TRNG_MIN_VERSION 0x10000UL @@ -58,6 +59,13 @@ static inline bool __arm64_rndrrs(unsigned long *v) return ok; } +static __always_inline bool __cpu_has_rng(void) +{ + if (unlikely(!system_capabilities_finalized() && !preemptible())) + return this_cpu_has_cap(ARM64_HAS_RNG); + return cpus_have_const_cap(ARM64_HAS_RNG); +} + static inline size_t __must_check arch_get_random_longs(unsigned long *v, size_t max_longs) { /* @@ -66,7 +74,7 @@ static inline size_t __must_check arch_get_random_longs(unsigned long *v, size_t * cpufeature code and with potential scheduling between CPUs * with and without the feature. */ - if (max_longs && cpus_have_const_cap(ARM64_HAS_RNG) && __arm64_rndr(v)) + if (max_longs && __cpu_has_rng() && __arm64_rndr(v)) return 1; return 0; } @@ -108,7 +116,7 @@ static inline size_t __must_check arch_get_random_seed_longs(unsigned long *v, s * reseeded after each invocation. This is not a 100% fit but good * enough to implement this API if no other entropy source exists. */ - if (cpus_have_const_cap(ARM64_HAS_RNG) && __arm64_rndrrs(v)) + if (__cpu_has_rng() && __arm64_rndrrs(v)) return 1; return 0; @@ -121,40 +129,4 @@ static inline bool __init __early_cpu_has_rndr(void) return (ftr >> ID_AA64ISAR0_EL1_RNDR_SHIFT) & 0xf; } -static inline size_t __init __must_check -arch_get_random_seed_longs_early(unsigned long *v, size_t max_longs) -{ - WARN_ON(system_state != SYSTEM_BOOTING); - - if (!max_longs) - return 0; - - if (smccc_trng_available) { - struct arm_smccc_res res; - - max_longs = min_t(size_t, 3, max_longs); - arm_smccc_1_1_invoke(ARM_SMCCC_TRNG_RND64, max_longs * 64, &res); - if ((int)res.a0 >= 0) { - switch (max_longs) { - case 3: - *v++ = res.a1; - fallthrough; - case 2: - *v++ = res.a2; - fallthrough; - case 1: - *v++ = res.a3; - break; - } - return max_longs; - } - } - - if (__early_cpu_has_rndr() && __arm64_rndr(v)) - return 1; - - return 0; -} -#define arch_get_random_seed_longs_early arch_get_random_seed_longs_early - #endif /* _ASM_ARCHRANDOM_H */ diff --git a/drivers/char/random.c b/drivers/char/random.c index 9d016ac181fa..5da120f10a67 100644 --- a/drivers/char/random.c +++ b/drivers/char/random.c @@ -813,13 +813,13 @@ void __init random_init_early(const char *command_line) #endif for (i = 0, arch_bits = sizeof(entropy) * 8; i < ARRAY_SIZE(entropy);) { - longs = arch_get_random_seed_longs_early(entropy, ARRAY_SIZE(entropy) - i); + longs = arch_get_random_seed_longs(entropy, ARRAY_SIZE(entropy) - i); if (longs) { _mix_pool_bytes(entropy, sizeof(*entropy) * longs); i += longs; continue; } - longs = arch_get_random_longs_early(entropy, ARRAY_SIZE(entropy) - i); + longs = arch_get_random_longs(entropy, ARRAY_SIZE(entropy) - i); if (longs) { _mix_pool_bytes(entropy, sizeof(*entropy) * longs); i += longs; diff --git a/include/linux/random.h b/include/linux/random.h index 182780cafd45..2bdd3add3400 100644 --- a/include/linux/random.h +++ b/include/linux/random.h @@ -153,26 +153,6 @@ declare_get_random_var_wait(long, unsigned long) #include -/* - * Called from the boot CPU during startup; not valid to call once - * secondary CPUs are up and preemption is possible. - */ -#ifndef arch_get_random_seed_longs_early -static inline size_t __init arch_get_random_seed_longs_early(unsigned long *v, size_t max_longs) -{ - WARN_ON(system_state != SYSTEM_BOOTING); - return arch_get_random_seed_longs(v, max_longs); -} -#endif - -#ifndef arch_get_random_longs_early -static inline bool __init arch_get_random_longs_early(unsigned long *v, size_t max_longs) -{ - WARN_ON(system_state != SYSTEM_BOOTING); - return arch_get_random_longs(v, max_longs); -} -#endif - #ifdef CONFIG_SMP int random_prepare_cpu(unsigned int cpu); int random_online_cpu(unsigned int cpu);