From patchwork Fri Oct 13 11:58:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 152540 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp1834236vqb; Fri, 13 Oct 2023 04:59:22 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEpX2GrzLs1FThbhPync+wGTu+SzQo9Ark5LxCJi/KWpVVIlAvwYgUgMRhCQPfGa7OsYIJn X-Received: by 2002:a05:6808:1a27:b0:3a9:e40c:683c with SMTP id bk39-20020a0568081a2700b003a9e40c683cmr32637631oib.1.1697198362182; Fri, 13 Oct 2023 04:59:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697198362; cv=none; d=google.com; s=arc-20160816; b=x/15fe3WtQd5pj1ySoTNF+7JC3M1kGLr34djl1SoMw1iTT6JSid9yTgJey2Zrs8gDS l7uAnMYt1vH5bPEUuJzMD5UAJYex17KkJgWsBagI1CCMbS+oyKQL+iA1g9I/fJVW3oAM ZGcRztm5N2NAaMand1JVugIj8a639C9yQK1QtFPpNBpklJNMQBUaBZveSpsHAEncyeDp ZzaqFOSoMHusIOhsUXSOgAOaATEC8SnZ6uycVEJqeZwEBrKxNdKRFO2Fs4V9DFl+avLY W8Gix06wmQp9y6feEkfuxtvCg9Hr+gD8NjOljy8I6cRRwOGLl7DOC6pbCpoCts3DnZ4I xW0w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=ZlsgUpBNYeGtPdIj/UJbNYIjZPtwUW801GyFsgkmDk8=; fh=QyKVx5sQ6a0Rye+jmuSl6ocZCOg2PmQ9bDVx4c3R8jE=; b=r1b3adGPTapUqD/PoAy5ouuy3F0W5kjp+ODcOXD3btPrkWv9OsVUaCU7LGIxcSwwZl I44NJhejvmg5yNWv/HeOL0j7hcuneilisRaAYMmBRb9ee3dKPJbvU2W9nEpVpPawlvKJ dMZrehftDS/qfmLBGNK59xTD7rMR3NiNta/+0Y5EzjhQFukTYn6PEt4o2/1WBHDcvtu9 a1caaii1VdqeeaQQqFmvpkSf5A+iX2+cQwTySds2rAEW6A8zuodKYe709wyMi2ewfN1y DKvWxQCeKdVXNiI0qZItB54o92JxrjAsOlrj8nzqlvQocFLqinR8flgWANnkhdC+ybYP NEpA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=GjciNQHB; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id l189-20020a6388c6000000b005abb83dec2bsi1034935pgd.446.2023.10.13.04.59.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Oct 2023 04:59:22 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=GjciNQHB; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 68F88830C325; Fri, 13 Oct 2023 04:59:21 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231368AbjJML7P (ORCPT + 19 others); Fri, 13 Oct 2023 07:59:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52604 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231479AbjJML7N (ORCPT ); Fri, 13 Oct 2023 07:59:13 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 872BBC9; Fri, 13 Oct 2023 04:59:11 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id DA9F8C433C8; Fri, 13 Oct 2023 11:59:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1697198351; bh=3eBYDTkgQaXsYR67+otBc1WF4QZDBr1bb/8NAZitcis=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GjciNQHBgTaujV2YgKuszJVG75vd0v5Sdj0qB0zOrrztJLsKd4G5u5Q96Yl3roHfW SvmYnxjYChGPOfHEw8tX94L1f32RIPY4Qgh1NeESzM/EaI7zWpjcLtF0VuaUH+HMYS R43V10VNExqNF1t/yyj8bEGz7imS6Zrneh1GCyhNn/GYZ+8kOkkmGcF+RPm+l+xAbs kZQmcQsobtP4GpVztNVu4AhO2PR2TKsifZT437968FExewObUmL46ptXbJTeFkcmAg 0J90DyZYUEv8KqF7ozFswqhfVnFJJyValjwtY6+TW5NzBezmtQGjG4HwFsx/MpqQSP xJ/PUpIqBXAOw== From: Frederic Weisbecker To: LKML Cc: "Joel Fernandes (Google)" , Boqun Feng , Josh Triplett , Mathieu Desnoyers , Neeraj Upadhyay , "Paul E . McKenney" , Steven Rostedt , Uladzislau Rezki , rcu , Frederic Weisbecker Subject: [PATCH 01/18] Revert "checkpatch: Error out if deprecated RCU API used" Date: Fri, 13 Oct 2023 13:58:45 +0200 Message-Id: <20231013115902.1059735-2-frederic@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231013115902.1059735-1-frederic@kernel.org> References: <20231013115902.1059735-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Fri, 13 Oct 2023 04:59:21 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779641470126521878 X-GMAIL-MSGID: 1779641470126521878 From: "Joel Fernandes (Google)" The definition for single-argument kfree_rcu() has been removed, so that any further attempt to use it will result in a build error. Because of this build error, there is no longer any need for a special check in checkpatch.pl. Therefore, revert commit 1eacac3255495be7502d406e2ba5444fb5c3607c. Signed-off-by: Joel Fernandes (Google) Signed-off-by: Paul E. McKenney Signed-off-by: Frederic Weisbecker --- scripts/checkpatch.pl | 9 --------- 1 file changed, 9 deletions(-) diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl index 7d16f863edf1..25fdb7fda112 100755 --- a/scripts/checkpatch.pl +++ b/scripts/checkpatch.pl @@ -6427,15 +6427,6 @@ sub process { } } -# check for soon-to-be-deprecated single-argument k[v]free_rcu() API - if ($line =~ /\bk[v]?free_rcu\s*\([^(]+\)/) { - if ($line =~ /\bk[v]?free_rcu\s*\([^,]+\)/) { - ERROR("DEPRECATED_API", - "Single-argument k[v]free_rcu() API is deprecated, please pass rcu_head object or call k[v]free_rcu_mightsleep()." . $herecurr); - } - } - - # check for unnecessary "Out of Memory" messages if ($line =~ /^\+.*\b$logFunctions\s*\(/ && $prevline =~ /^[ \+]\s*if\s*\(\s*(\!\s*|NULL\s*==\s*)?($Lval)(\s*==\s*NULL\s*)?\s*\)/ && From patchwork Fri Oct 13 11:58:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 152541 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp1834268vqb; Fri, 13 Oct 2023 04:59:26 -0700 (PDT) X-Google-Smtp-Source: AGHT+IENTDFxfDQW9WDZJHgEWC7PiGb0tNX6+AlA0ov9rtgu2VKV++mLsZ86AhkD5/MPJym1zMfg X-Received: by 2002:a05:6a00:1d22:b0:693:38c5:4d6d with SMTP id a34-20020a056a001d2200b0069338c54d6dmr29428469pfx.2.1697198365982; Fri, 13 Oct 2023 04:59:25 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697198365; cv=none; d=google.com; s=arc-20160816; b=Lg0pQVgUIwwUYPN+rKHoc8gKRHezR5+ORdCCHbrGaz2lkbZgiskkOEv2Ac2vwAu5kW DGIxe3nWzrly7oUNGLQZuizS1yOSjZiuATE+St4iFU44lR293LE4qVCo2myUFuhnMMKV Z+HWSSWk6Fpe6HOQMDFPQsd9ek/bn8VlV1dkBwm91+n9461cr4Pi3mO6UDliJ0QjPtEX PwClk+fJ8AxOf9w2foQsGENDRrOSZp5mE2/dk+tX26d0eP3gbSUHkU28+VCv5bo/dPle YExRXbFQc/c4ggvq9oXU/C+efOmuBpCyIQtwgVGgN3zP/OeC1aiBX4d7+Qy1LntZOEXR YJ2Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=G2DFeqo0EWOzNIlwMQ1whDCqR/8K+r/TDi9s97w3xgM=; fh=QyKVx5sQ6a0Rye+jmuSl6ocZCOg2PmQ9bDVx4c3R8jE=; b=zctqUcXSY2OHtp1nzfuTYLnk1aR6rP+4Q3teOtr3tlx2W0Pq4zt1aJOxz+XE3XKNMi OGLNrC7V6xOzcBv1jyR7JHzSx5iaysSuWQg1kQEFvc+C6tSF6VU1pDdFBPqICeKheyWm Q7etd5PFVJ9iMwjxT8dG77d+WeFpgCzL1VwlXDUvFG25js8ph7wKxUv2le2TaNKJa7Mg hbqQEWj1MvjwclXgKBpLBLEE0t4a3OwUIqCDrKrKWMOXbY4GzmIXfTGyuToaaprqoOpE ONNgkLF9tDIrVQ/g9f7P010hsv3E0jbsIPHyI4gVcW6REKL9xV+cxR1G/3wcuMFy6oLH 3meA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=ST+LcM3k; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id cb4-20020a056a00430400b00690d8405fe6si3208566pfb.317.2023.10.13.04.59.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Oct 2023 04:59:25 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=ST+LcM3k; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 50BC7830C324; Fri, 13 Oct 2023 04:59:25 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231479AbjJML7T (ORCPT + 19 others); Fri, 13 Oct 2023 07:59:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52654 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231591AbjJML7Q (ORCPT ); Fri, 13 Oct 2023 07:59:16 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 538FDA9; Fri, 13 Oct 2023 04:59:14 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A0BB0C433C7; Fri, 13 Oct 2023 11:59:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1697198353; bh=QSG6hSbymQR94WKaIKBPaZ45C4sCdjinqu5taG9tkdg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ST+LcM3kTPie/f4gkDT0cUkbNWb3QyNRsKSk10jgGlWr2PeS6+U3wSok2DF0IK8La 8ZTxGilr1I71zi49J1hyWv7bt5fSBDMpDN//3E6k1a/7JX1D0WtmJXYthV+Qf8IoNv 65GP3qfrkDMIwm86TZDVQzthXMGFfj7HhJQyQyiLtJUljR29rwm9JTBUwI7cXFHr4M 3a6TYfvLFkoIQpcw+mY8P4O2BKsANgaxt0QMm43q2UNT3TkRxioGLnKaoBNk1jBrdM AexJvDe+NCk41waI36+jQlFhjh6yAnGPsaSF0ikKnmWxq/Bl0AhB+pElrcO8eabFRt 20IMFX6kyDcCw== From: Frederic Weisbecker To: LKML Cc: "Joel Fernandes (Google)" , Boqun Feng , Josh Triplett , Mathieu Desnoyers , Neeraj Upadhyay , "Paul E . McKenney" , Steven Rostedt , Uladzislau Rezki , rcu , Frederic Weisbecker Subject: [PATCH 02/18] srcu: Fix error handling in init_srcu_struct_fields() Date: Fri, 13 Oct 2023 13:58:46 +0200 Message-Id: <20231013115902.1059735-3-frederic@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231013115902.1059735-1-frederic@kernel.org> References: <20231013115902.1059735-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Fri, 13 Oct 2023 04:59:25 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779641474014052929 X-GMAIL-MSGID: 1779641474014052929 From: "Joel Fernandes (Google)" The current error handling in init_srcu_struct_fields() is a bit inconsistent. If init_srcu_struct_nodes() fails, the function either returns -ENOMEM or 0 depending on whether ssp->sda_is_static is true or false. This can make init_srcu_struct_fields() return 0 even if memory allocation failed! Simplify the error handling by always returning -ENOMEM if either init_srcu_struct_nodes() or the per-CPU allocation fails. This makes the control flow easier to follow and avoids the inconsistent return values. Add goto labels to avoid duplicating the error cleanup code. Link: https://lore.kernel.org/r/20230404003508.GA254019@google.com Signed-off-by: Joel Fernandes (Google) Signed-off-by: Paul E. McKenney Signed-off-by: Frederic Weisbecker --- kernel/rcu/srcutree.c | 32 +++++++++++++++++--------------- 1 file changed, 17 insertions(+), 15 deletions(-) diff --git a/kernel/rcu/srcutree.c b/kernel/rcu/srcutree.c index 20d7a238d675..f1a905200fc2 100644 --- a/kernel/rcu/srcutree.c +++ b/kernel/rcu/srcutree.c @@ -255,29 +255,31 @@ static int init_srcu_struct_fields(struct srcu_struct *ssp, bool is_static) ssp->srcu_sup->sda_is_static = is_static; if (!is_static) ssp->sda = alloc_percpu(struct srcu_data); - if (!ssp->sda) { - if (!is_static) - kfree(ssp->srcu_sup); - return -ENOMEM; - } + if (!ssp->sda) + goto err_free_sup; init_srcu_struct_data(ssp); ssp->srcu_sup->srcu_gp_seq_needed_exp = 0; ssp->srcu_sup->srcu_last_gp_end = ktime_get_mono_fast_ns(); if (READ_ONCE(ssp->srcu_sup->srcu_size_state) == SRCU_SIZE_SMALL && SRCU_SIZING_IS_INIT()) { - if (!init_srcu_struct_nodes(ssp, GFP_ATOMIC)) { - if (!ssp->srcu_sup->sda_is_static) { - free_percpu(ssp->sda); - ssp->sda = NULL; - kfree(ssp->srcu_sup); - return -ENOMEM; - } - } else { - WRITE_ONCE(ssp->srcu_sup->srcu_size_state, SRCU_SIZE_BIG); - } + if (!init_srcu_struct_nodes(ssp, GFP_ATOMIC)) + goto err_free_sda; + WRITE_ONCE(ssp->srcu_sup->srcu_size_state, SRCU_SIZE_BIG); } ssp->srcu_sup->srcu_ssp = ssp; smp_store_release(&ssp->srcu_sup->srcu_gp_seq_needed, 0); /* Init done. */ return 0; + +err_free_sda: + if (!is_static) { + free_percpu(ssp->sda); + ssp->sda = NULL; + } +err_free_sup: + if (!is_static) { + kfree(ssp->srcu_sup); + ssp->srcu_sup = NULL; + } + return -ENOMEM; } #ifdef CONFIG_DEBUG_LOCK_ALLOC From patchwork Fri Oct 13 11:58:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 152542 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp1834327vqb; Fri, 13 Oct 2023 04:59:34 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEGHmFAL/yxaiPUoVg1KeLhzFRBeXWKE795Krs0O86qO0t2SeZC9+AxAV/Y1ZexjomxpJMW X-Received: by 2002:a05:6a00:3985:b0:68f:c8b3:3077 with SMTP id fi5-20020a056a00398500b0068fc8b33077mr28668225pfb.1.1697198373987; Fri, 13 Oct 2023 04:59:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697198373; cv=none; d=google.com; s=arc-20160816; b=Y6gd073UEb3z2qgmjeOEPZwpI98BweLcf0ZY+LX24RbC9ctQ5TRlSg/0GdQDeweeKH riofaSuj2qJrODBVfvAKOQYf5VuL4am4fA7asSreXpt22C7tO+d3dh2sCJ7uVszumVXu BDyMeQ4+K3f04PK6hBivwo3SlNqjUJ/l2f5IRE0JMZ84rR3+TUROU8m7DVgx3PeoFT/v Ep4CrAb6Vp4803F8zT1H8sPf+VTSFeK51CUk4BR8SrUpLXnyZ1jX5FZMHcOdm6c1eGZM jl/TwWPDQ7hT+ewAdNkF4d/LAHtAiPJm1eKvnmReNQeB120RVCklAq7MJnMuxOjwdF27 c7og== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=5/uTpDl8NOIwCyTBja5ZMGC/pT65YeQ4/r4I+68HAys=; fh=QyKVx5sQ6a0Rye+jmuSl6ocZCOg2PmQ9bDVx4c3R8jE=; b=ACQ2+V3BHoTifiMAhnLgX9foThilHO1C3Ko09Vl7iHE3r4BrBiJGsbqkUPfzU/gmdR ds7mgeLyYRC1fOyqJ3b4dRzgwYHoFrYOlzuo4SvCRh5xoBd7ZrOQ8omcaxDyglI9QI2v 2VTa2G3MyHC6wnJ70S8hEm5LzN2zkcSMhTIWRZS8nyon/pkAcKgLKkrAZ9/PR3Jcafqc gcnyvLj9Z4NR7JroGlyLKWpwk6Og3urdmvnICxNQ1kkZh8wbVbHaGXkqJfrMPBqH1NaY VoKHjRxzaNIDayCidd3PNd5i0pLIIRsfWbmYow/+HLu4WOzITFrtUYFlBbzS3KmLCQHD id5A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=fW8I0GQf; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id h68-20020a636c47000000b00565325c6ca6si4468042pgc.489.2023.10.13.04.59.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Oct 2023 04:59:33 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=fW8I0GQf; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 521C9830C335; Fri, 13 Oct 2023 04:59:33 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231650AbjJML7V (ORCPT + 19 others); Fri, 13 Oct 2023 07:59:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52078 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231635AbjJML7T (ORCPT ); Fri, 13 Oct 2023 07:59:19 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 13657E1; Fri, 13 Oct 2023 04:59:17 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 64D3EC433C9; Fri, 13 Oct 2023 11:59:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1697198356; bh=bwVDWmfgW6KPhnAu5RS+ChX+7Qv0AeJsa5cg4h7G8/k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fW8I0GQfeVeMKSCJ2eo4nXP9LY628stEuP5ZiLSvnFUYwa0FrRvHtHuC49O8rQ3WM 5Dh852JhFOCBusJDIqyR4iChhbP45j1iA3hcfUIlBdmRboAk/kpo211ybu4M3cfjnF ryGUWIylqDHxxKLjPBEc01+e4J9vOnG7AMSDLxPj7XEMTdE7sKwtrCN9ttvDfpVPa9 vajb7cGPCEisvuOGPobI0V/rhl/9jwvaq6IdNPEMVJ+CgHs+if6icB/LekJhPjD4CN W0MkrgwqI9ePgi4EtZNhMm+6HbPqkILkyTYGTMVTcTI+PwUq48NTdTdyHt052pXM9z kwJfdgouswtvA== From: Frederic Weisbecker To: LKML Cc: "Joel Fernandes (Google)" , Boqun Feng , Josh Triplett , Mathieu Desnoyers , Neeraj Upadhyay , "Paul E . McKenney" , Steven Rostedt , Uladzislau Rezki , rcu , Frederic Weisbecker Subject: [PATCH 03/18] rcu/tree: Remove superfluous return from void call_rcu* functions Date: Fri, 13 Oct 2023 13:58:47 +0200 Message-Id: <20231013115902.1059735-4-frederic@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231013115902.1059735-1-frederic@kernel.org> References: <20231013115902.1059735-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Fri, 13 Oct 2023 04:59:33 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779641482519601056 X-GMAIL-MSGID: 1779641482519601056 From: "Joel Fernandes (Google)" The return keyword is not needed here. Signed-off-by: Joel Fernandes (Google) Signed-off-by: Paul E. McKenney Signed-off-by: Frederic Weisbecker --- kernel/rcu/tree.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index cb1caefa8bd0..7c79480bfaa0 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -2713,7 +2713,7 @@ __call_rcu_common(struct rcu_head *head, rcu_callback_t func, bool lazy_in) */ void call_rcu_hurry(struct rcu_head *head, rcu_callback_t func) { - return __call_rcu_common(head, func, false); + __call_rcu_common(head, func, false); } EXPORT_SYMBOL_GPL(call_rcu_hurry); #endif @@ -2764,7 +2764,7 @@ EXPORT_SYMBOL_GPL(call_rcu_hurry); */ void call_rcu(struct rcu_head *head, rcu_callback_t func) { - return __call_rcu_common(head, func, IS_ENABLED(CONFIG_RCU_LAZY)); + __call_rcu_common(head, func, IS_ENABLED(CONFIG_RCU_LAZY)); } EXPORT_SYMBOL_GPL(call_rcu); From patchwork Fri Oct 13 11:58:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 152543 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp1834348vqb; Fri, 13 Oct 2023 04:59:36 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFDTx/VcvO6zjFHe8Hl9uX2b3v/uyyJYUMNJbJbLr2Sb/0HtbkDZF9ARtvhIVyyH3cEQTmJ X-Received: by 2002:a05:6a00:3985:b0:68f:c8b3:3077 with SMTP id fi5-20020a056a00398500b0068fc8b33077mr28668305pfb.1.1697198376301; Fri, 13 Oct 2023 04:59:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697198376; cv=none; d=google.com; s=arc-20160816; b=MHpoZJ+hCCmpFk/uzaWjuPjuVYa6LeevJWswfXwnuYy0due2P2BXSYybt87yYI3xuw n7t89LUpnmiahI/qpBpCvQDHZnZxWCAp13O0mZXL37Nul5h1iItZYlS3M/vnx9SeYYpu Q6up/NoYUtGiSNhL5heqoKmRLvlB5zTVlO5WrGzySkoiVdew94KkHf6WzydXZOb8j+SO rex/2/mqsS/l+SHiepVa+X0SkCKfaCxv5lp1jHsAzLQ16kUt0sCcSlawRUxZcNNBVZgx A21QWUZLY7o+ViRgkOSOBkMs8TWUbtaHnUdu+zU4CMT5xm8YG/w1QScrpIQxQswKiPq2 zL7w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=pQXzG9U9KQnDmfeIzPtDr1ZLFc+3FNUyGuyW9EJ0/zc=; fh=3ERrlfTH2hVgUdtQ1LGRa8EMM8z7rwzYMTcCEAjeFjw=; b=hg4UaAMYzrwhqbBFEe2KhC0U5fVGuX9TfzmI4dObNrkzvLrJcMi5UBRdW3FQFw2/Ye NZPpegw3xxRLU9wqeywb5c3xZoZTeyLj2AxD5JndR7vG2Rcx+XlEaJZYNj1aiFozMM+K afxoud/SXKv1BFGyYqc0Q9VUKLHN6lMdJnIaSOD/z39M289Hhrt0HSy0PaB3IWSWFyXy YTvwzJYJbUUk9HUFfYNhmd+3hNtrQmr6amYneGlt8aUXmpVkwZoZfOuVFC+jS5vhKeQp rqsdGRlxjBTHgu6Jxkx63c0mPoXXbtAU2000BZ5SC0iRHR2JbwnhkRVR7Rg6V+G9/fsw MOjA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=c85EukgO; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id z17-20020a631911000000b005637e28270fsi2988166pgl.243.2023.10.13.04.59.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Oct 2023 04:59:36 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=c85EukgO; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 0ED45830C32A; Fri, 13 Oct 2023 04:59:35 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231521AbjJML7a (ORCPT + 19 others); Fri, 13 Oct 2023 07:59:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52116 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231657AbjJML7V (ORCPT ); Fri, 13 Oct 2023 07:59:21 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F3A2E102; Fri, 13 Oct 2023 04:59:19 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2AF52C433AB; Fri, 13 Oct 2023 11:59:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1697198359; bh=R8Fp/1VLF0GqIaHBlnBp4fYP1iUMlAIDRWA+aBlSlzs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=c85EukgOs91vaaNy0DXYLoHblLD6GEv8zepj20ZCAZyIDZtG5oDyX0UWwW/3VOyy8 bPSQt8ROD9OGxITChy9rpIDoQeyxhx6Z1o+syvASpJ0kr78ggMUdz9URTFUXqBIGqA TqVD3aAEwey859RgecAVU176ntPW9kpTeDchYQ8J4H1l9aCSBdo2wA+CT+VkRK5UAr By4VzSaFxNagN/ontbTyCBiPYR6UEcKTWeotTy0z8XaPYxuu+W/vX2kkJmH2oCYRIi yxCGXjO2vRkZju9goSBnnTMom505/il7qCjrHanAT8pbvFRMi1IlEAXeme6xkL2SiR aNDCvBqAEySeA== From: Frederic Weisbecker To: LKML Cc: "Paul E. McKenney" , Boqun Feng , Joel Fernandes , Josh Triplett , Mathieu Desnoyers , Neeraj Upadhyay , Steven Rostedt , Uladzislau Rezki , rcu , Johannes Weiner , Frederic Weisbecker Subject: [PATCH 04/18] rcu: Add sysfs to provide throttled access to rcu_barrier() Date: Fri, 13 Oct 2023 13:58:48 +0200 Message-Id: <20231013115902.1059735-5-frederic@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231013115902.1059735-1-frederic@kernel.org> References: <20231013115902.1059735-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Fri, 13 Oct 2023 04:59:35 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779641484377267347 X-GMAIL-MSGID: 1779641484377267347 From: "Paul E. McKenney" When running a series of stress tests all making heavy use of RCU, it is all too possible to OOM the system when the prior test's RCU callbacks don't get invoked until after the subsequent test starts. One way of handling this is just a timed wait, but this fails when a given CPU has so many callbacks queued that they take longer to invoke than allowed for by that timed wait. This commit therefore adds an rcutree.do_rcu_barrier module parameter that is accessible from sysfs. Writing one of the many synonyms for boolean "true" will cause an rcu_barrier() to be invoked, but will guarantee that no more than one rcu_barrier() will be invoked per sixteenth of a second via this mechanism. The flip side is that a given request might wait a second or three longer than absolutely necessary, but only when there are multiple uses of rcutree.do_rcu_barrier within a one-second time interval. This commit unnecessarily serializes the rcu_barrier() machinery, given that serialization is already provided by procfs. This has the advantage of allowing throttled rcu_barrier() from other sources within the kernel. Reported-by: Johannes Weiner Signed-off-by: Paul E. McKenney Signed-off-by: Frederic Weisbecker --- .../admin-guide/kernel-parameters.txt | 7 ++ kernel/rcu/tree.c | 76 +++++++++++++++++++ 2 files changed, 83 insertions(+) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index 0a1731a0f0ef..7ec8a406d419 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -4769,6 +4769,13 @@ Set maximum number of finished RCU callbacks to process in one batch. + rcutree.do_rcu_barrier= [KNL] + Request a call to rcu_barrier(). This is + throttled so that userspace tests can safely + hammer on the sysfs variable if they so choose. + If triggered before the RCU grace-period machinery + is fully active, this will error out with EAGAIN. + rcutree.dump_tree= [KNL] Dump the structure of the rcu_node combining tree out at early boot. This is used for diagnostic diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index 7c79480bfaa0..3c7281fc25a7 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -4083,6 +4083,82 @@ void rcu_barrier(void) } EXPORT_SYMBOL_GPL(rcu_barrier); +static unsigned long rcu_barrier_last_throttle; + +/** + * rcu_barrier_throttled - Do rcu_barrier(), but limit to one per second + * + * This can be thought of as guard rails around rcu_barrier() that + * permits unrestricted userspace use, at least assuming the hardware's + * try_cmpxchg() is robust. There will be at most one call per second to + * rcu_barrier() system-wide from use of this function, which means that + * callers might needlessly wait a second or three. + * + * This is intended for use by test suites to avoid OOM by flushing RCU + * callbacks from the previous test before starting the next. See the + * rcutree.do_rcu_barrier module parameter for more information. + * + * Why not simply make rcu_barrier() more scalable? That might be + * the eventual endpoint, but let's keep it simple for the time being. + * Note that the module parameter infrastructure serializes calls to a + * given .set() function, but should concurrent .set() invocation ever be + * possible, we are ready! + */ +static void rcu_barrier_throttled(void) +{ + unsigned long j = jiffies; + unsigned long old = READ_ONCE(rcu_barrier_last_throttle); + unsigned long s = rcu_seq_snap(&rcu_state.barrier_sequence); + + while (time_in_range(j, old, old + HZ / 16) || + !try_cmpxchg(&rcu_barrier_last_throttle, &old, j)) { + schedule_timeout_idle(HZ / 16); + if (rcu_seq_done(&rcu_state.barrier_sequence, s)) { + smp_mb(); /* caller's subsequent code after above check. */ + return; + } + j = jiffies; + old = READ_ONCE(rcu_barrier_last_throttle); + } + rcu_barrier(); +} + +/* + * Invoke rcu_barrier_throttled() when a rcutree.do_rcu_barrier + * request arrives. We insist on a true value to allow for possible + * future expansion. + */ +static int param_set_do_rcu_barrier(const char *val, const struct kernel_param *kp) +{ + bool b; + int ret; + + if (rcu_scheduler_active != RCU_SCHEDULER_RUNNING) + return -EAGAIN; + ret = kstrtobool(val, &b); + if (!ret && b) { + atomic_inc((atomic_t *)kp->arg); + rcu_barrier_throttled(); + atomic_dec((atomic_t *)kp->arg); + } + return ret; +} + +/* + * Output the number of outstanding rcutree.do_rcu_barrier requests. + */ +static int param_get_do_rcu_barrier(char *buffer, const struct kernel_param *kp) +{ + return sprintf(buffer, "%d\n", atomic_read((atomic_t *)kp->arg)); +} + +static const struct kernel_param_ops do_rcu_barrier_ops = { + .set = param_set_do_rcu_barrier, + .get = param_get_do_rcu_barrier, +}; +static atomic_t do_rcu_barrier; +module_param_cb(do_rcu_barrier, &do_rcu_barrier_ops, &do_rcu_barrier, 0644); + /* * Compute the mask of online CPUs for the specified rcu_node structure. * This will not be stable unless the rcu_node structure's ->lock is From patchwork Fri Oct 13 11:58:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 152544 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp1834370vqb; Fri, 13 Oct 2023 04:59:39 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEEzRwv0MAuHY6R2js7Pief1S3dDnBlGng56jtKyn+WjvcMo/wkmbLNR+wl9ZjEbTFxAMeS X-Received: by 2002:a17:90a:4ca4:b0:274:99ed:a80c with SMTP id k33-20020a17090a4ca400b0027499eda80cmr24176413pjh.3.1697198378800; Fri, 13 Oct 2023 04:59:38 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697198378; cv=none; d=google.com; s=arc-20160816; b=vxEHl7voR9K7/BJkXv07tLraPRzrb0jSS4kVWvdzmsDDoO9dBFlbGQkcFp3EO/9BjC kooBDATdIx6lcX2jxRylqqPgGqhSxQwB6Aq0+1qkeP/llrMFIgyUp0Pd5GQPDUUbp47S uQakxoaMZMR+DfaM+RLVXe7UF7clCnKr6MH7/jkEh8g8EhEZx169t/LDdVtPpI0NJLGL G4+U+sGvm/7P9D7DiLavVD9+Irw/ZjjexTlpoxz2MlrkjgQIKLmVGbBI3VqSyWB0eNpz LrmR6kn70TNpvtFJ7Cr9Xql+0Q2Q7JrnVXdtaGlHJxf3FPJMssSc0aW8xlTN2XIzpSIU p/FQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=m2xvP7PEABHu3Wh3poNFhx4YcUMjbrFV4o3elbOPk34=; fh=wMTojjWdFivsIaDeiJYjUjeCa+i34K/Lpp5l3U30fiI=; b=hK3mH2aMoXVbprRdB9KCCwdgnuuoUlNVNSCIg7WNBW+E3aFYT98zra2cWLB01EI25k O5abwN4Nmdy3PtrOZVQgBj7M+JdbF4gD8BfyvOMjmvTTpWQ0NvPotGObv2dvjJnGEz3n M5WMEP89hO/bSve2zLMXNOPH23siPpI20I5cM/Go0iT/bcL2x06TNKiT4tAf+asEwDPh SRs337iet443zmfFRsfzEu6vSpiLNCnTToUoP56ahD4qUm7R1rQ6l3SH1x5l++h1LRvX jku8o9ncPvDQ0kZNfAKvWYjl0/B5rjbg2U44tMF83DQJU82Vj9+fbBxkm3mBv3j0kKKt eclQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=jMY7dv0K; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id q15-20020a17090a68cf00b00263860e1f4csi4586736pjj.16.2023.10.13.04.59.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Oct 2023 04:59:38 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=jMY7dv0K; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 5C073830C336; Fri, 13 Oct 2023 04:59:37 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231514AbjJML7e (ORCPT + 19 others); Fri, 13 Oct 2023 07:59:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52554 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231656AbjJML72 (ORCPT ); Fri, 13 Oct 2023 07:59:28 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2118D12F; Fri, 13 Oct 2023 04:59:23 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1A457C433BB; Fri, 13 Oct 2023 11:59:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1697198362; bh=V2xBqcl5Jr08RnnZZIquxtDdCbLnvcU07W9cRLP2q50=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jMY7dv0K23sSU7UdaQvzsEmkbED8rLf2EQb5J8g/Sw5WYyE33w0AqV3jmDQzlSlcY A5AcxbukrW0BQosRi2IiAqfhA5V5S5TZ2vzt0bAKT4zvCM6qsNj0e4p9hcB6DvTitF /eA9NC6lKwr6j63syaOd41h86TlZ16Zu5QkZ0oKSZl6+miH3GIV2bidz1ZA3bQ07g4 f2S8zCV714HZ/Xhs04JOIoyEwNvQe2nRt/0i7HK7w9jGw+p37jaXgWIjDmVe0hsg3y aZG2thaFom2op33POFplc7DpC9lZovN8oWGJkhDthVC2cJH0079ZaJsYIQTV+uFcrT 4/9/yWRx8owPw== From: Frederic Weisbecker To: LKML Cc: Yue Haibing , Boqun Feng , Joel Fernandes , Josh Triplett , Mathieu Desnoyers , Neeraj Upadhyay , "Paul E . McKenney" , Steven Rostedt , Uladzislau Rezki , rcu , Frederic Weisbecker Subject: [PATCH 05/18] rcu: Remove unused function declaration rcu_eqs_special_set() Date: Fri, 13 Oct 2023 13:58:49 +0200 Message-Id: <20231013115902.1059735-6-frederic@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231013115902.1059735-1-frederic@kernel.org> References: <20231013115902.1059735-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Fri, 13 Oct 2023 04:59:37 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779641487306469146 X-GMAIL-MSGID: 1779641487306469146 From: Yue Haibing Commit a86baa69c2b7 ("rcu: Remove special bit at the bottom of the ->dynticks counter") left behind this, remove it. Signed-off-by: Yue Haibing Signed-off-by: Paul E. McKenney Signed-off-by: Frederic Weisbecker --- include/linux/rcutree.h | 1 - 1 file changed, 1 deletion(-) diff --git a/include/linux/rcutree.h b/include/linux/rcutree.h index 126f6b418f6a..153cfc7bbffd 100644 --- a/include/linux/rcutree.h +++ b/include/linux/rcutree.h @@ -37,7 +37,6 @@ void synchronize_rcu_expedited(void); void kvfree_call_rcu(struct rcu_head *head, void *ptr); void rcu_barrier(void); -bool rcu_eqs_special_set(int cpu); void rcu_momentary_dyntick_idle(void); void kfree_rcu_scheduler_running(void); bool rcu_gp_might_be_stalled(void); From patchwork Fri Oct 13 11:58:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 152545 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp1834504vqb; Fri, 13 Oct 2023 04:59:56 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGCtIcVsYDnyHfBimuiqq08PUhJlAgfizmg4XY1Fufg9AKyVcPoYO0fa7xai/QfeSvzQE2/ X-Received: by 2002:a05:6830:4867:b0:6bf:500f:b570 with SMTP id dx7-20020a056830486700b006bf500fb570mr24448537otb.3.1697198396036; Fri, 13 Oct 2023 04:59:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697198396; cv=none; d=google.com; s=arc-20160816; b=M2RzXpRkXnaj9x6nFyYEDdWzIEByEsB3G3lPRSf/DfeOdUKi3cTHxbzF5trMOMB1P8 3wXOdjTkh17XlTGahmzbJmRyqflK4kB3dKEgcxTbY6O/q07zWLnDJQgQ15uO029ONT0F ALJeZelDYhViSGOrEA3TvwWbdqNnFKvqtp6bujiEIZ8X+9uglZ0fBkkpopEUqC4jynSI DzeFAqM5lPFbyWujx6YiNwxbYHBJ4Bg/X4/7AfSVSb00XxvvgAp/c/CrpiPlNvqEAXkH 2XFN7KkEcWuQPwb85fvuWjlYO4BjnJhS+j3uG34+CISvRI8rumZlKfWdo6SKEpZkcjDa l5Ew== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=/8U3aRF02bJ7fzsmNWUbZYAT6KRIF+C/zrYHGM42vdQ=; fh=7HigBTpd83qIdgppUAOmIsBzYbBi+/Yg0Z0SyB5wcIE=; b=H/VydAdp8cKimIpfwMPBl199ViawuyaROhyaoDLy0DQDY3eH/GwipyAG376dkRrvlS j6A3eTInOwpqUXWz+xdZkG5LZips2pBLaJYwz4y/VMpcRvIMMJlkmwuBgVfDCTog9CPG 9C9qdxQcu0sUAZhpz8FsjfgfzOGiBWaSGmihqRtr2d9Tk/6sBNHqEo3VTBT7AYzYdRhr Zu/aZ4dQvCyScHutIemSzHGxs8yQ71yeLkez1FwMHLYmfgKsFZ3KoQLiX4ukzXCT9apd iG0yoRz44DYWoLmL77Zlrf5SEKtZB3FN/mN+VyZUMspzeplX1zXQrJDct7nK/Ocj5ISW 5YYw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b="JzRgL/7G"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from lipwig.vger.email (lipwig.vger.email. [2620:137:e000::3:3]) by mx.google.com with ESMTPS id r7-20020a63e507000000b0059b79bf220csi473454pgh.662.2023.10.13.04.59.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Oct 2023 04:59:55 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) client-ip=2620:137:e000::3:3; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b="JzRgL/7G"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id ACD2183303FC; Fri, 13 Oct 2023 04:59:53 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231725AbjJML7o (ORCPT + 19 others); Fri, 13 Oct 2023 07:59:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52674 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231714AbjJML7b (ORCPT ); Fri, 13 Oct 2023 07:59:31 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 19BD1EA; Fri, 13 Oct 2023 04:59:26 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0E070C433C7; Fri, 13 Oct 2023 11:59:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1697198365; bh=661eskRrrn3a2q2Gc7DQU5Vpxig1R977JLUBlgWFz9M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=JzRgL/7G2NKdgepQIw580jwYQf78okNF3NQ7Yh6KggurGldEHEarYXCCKpmg9vE1U RebNIbFvV5eEneaH4YuaPWr6mA/zAAPf46Pro1qsP7gKq1tO+gKIGKsfpaKS94YZOr qFGwVaOEsFLLb8e7/WM3NDiX+XBFiPnqPCMgfcxgTinx1ySo9Ym4H4fJrxbFAFM+gl 0JWMn/SZnUWxITKP5piPgSM+r65d42sskl7bZc+IrLRndHIDy6sPfGdpZ6i238Mauv m9OOsF+zoIztDofBjvspP8PJoav/zg4OpSSJZRSyLHvaJA5BgjKOYN/KEhCJ/ovgc/ AOo//PccRp3CA== From: Frederic Weisbecker To: LKML Cc: Zhen Lei , Boqun Feng , Joel Fernandes , Josh Triplett , Mathieu Desnoyers , Neeraj Upadhyay , "Paul E . McKenney" , Steven Rostedt , Uladzislau Rezki , rcu , Matthew Wilcox , Vlastimil Babka , Frederic Weisbecker Subject: [PATCH 06/18] mm: Remove kmem_valid_obj() Date: Fri, 13 Oct 2023 13:58:50 +0200 Message-Id: <20231013115902.1059735-7-frederic@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231013115902.1059735-1-frederic@kernel.org> References: <20231013115902.1059735-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Fri, 13 Oct 2023 04:59:53 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779641505248276714 X-GMAIL-MSGID: 1779641505248276714 From: Zhen Lei Function kmem_dump_obj() will splat if passed a pointer to a non-slab object. So nothing calls it directly, instead calling kmem_valid_obj() first to determine whether the passed pointer to a valid slab object. This means that merging kmem_valid_obj() into kmem_dump_obj() will make the code more concise. Therefore, convert kmem_dump_obj() to work the same way as vmalloc_dump_obj(), removing the need for the kmem_dump_obj() caller to check kmem_valid_obj(). After this, there are no remaining calls to kmem_valid_obj() anymore, and it can be safely removed. Suggested-by: Matthew Wilcox Signed-off-by: Zhen Lei Reviewed-by: Matthew Wilcox (Oracle) Acked-by: Vlastimil Babka Signed-off-by: Paul E. McKenney Signed-off-by: Frederic Weisbecker --- include/linux/slab.h | 5 +++-- mm/slab_common.c | 41 +++++++++++------------------------------ mm/util.c | 4 +--- 3 files changed, 15 insertions(+), 35 deletions(-) diff --git a/include/linux/slab.h b/include/linux/slab.h index 8228d1276a2f..ff56ab804bf6 100644 --- a/include/linux/slab.h +++ b/include/linux/slab.h @@ -245,8 +245,9 @@ DEFINE_FREE(kfree, void *, if (_T) kfree(_T)) size_t ksize(const void *objp); #ifdef CONFIG_PRINTK -bool kmem_valid_obj(void *object); -void kmem_dump_obj(void *object); +bool kmem_dump_obj(void *object); +#else +static inline bool kmem_dump_obj(void *object) { return false; } #endif /* diff --git a/mm/slab_common.c b/mm/slab_common.c index cd71f9581e67..a425bedf2103 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -528,26 +528,6 @@ bool slab_is_available(void) } #ifdef CONFIG_PRINTK -/** - * kmem_valid_obj - does the pointer reference a valid slab object? - * @object: pointer to query. - * - * Return: %true if the pointer is to a not-yet-freed object from - * kmalloc() or kmem_cache_alloc(), either %true or %false if the pointer - * is to an already-freed object, and %false otherwise. - */ -bool kmem_valid_obj(void *object) -{ - struct folio *folio; - - /* Some arches consider ZERO_SIZE_PTR to be a valid address. */ - if (object < (void *)PAGE_SIZE || !virt_addr_valid(object)) - return false; - folio = virt_to_folio(object); - return folio_test_slab(folio); -} -EXPORT_SYMBOL_GPL(kmem_valid_obj); - static void kmem_obj_info(struct kmem_obj_info *kpp, void *object, struct slab *slab) { if (__kfence_obj_info(kpp, object, slab)) @@ -566,11 +546,11 @@ static void kmem_obj_info(struct kmem_obj_info *kpp, void *object, struct slab * * and, if available, the slab name, return address, and stack trace from * the allocation and last free path of that object. * - * This function will splat if passed a pointer to a non-slab object. - * If you are not sure what type of object you have, you should instead - * use mem_dump_obj(). + * Return: %true if the pointer is to a not-yet-freed object from + * kmalloc() or kmem_cache_alloc(), either %true or %false if the pointer + * is to an already-freed object, and %false otherwise. */ -void kmem_dump_obj(void *object) +bool kmem_dump_obj(void *object) { char *cp = IS_ENABLED(CONFIG_MMU) ? "" : "/vmalloc"; int i; @@ -578,13 +558,13 @@ void kmem_dump_obj(void *object) unsigned long ptroffset; struct kmem_obj_info kp = { }; - if (WARN_ON_ONCE(!virt_addr_valid(object))) - return; + /* Some arches consider ZERO_SIZE_PTR to be a valid address. */ + if (object < (void *)PAGE_SIZE || !virt_addr_valid(object)) + return false; slab = virt_to_slab(object); - if (WARN_ON_ONCE(!slab)) { - pr_cont(" non-slab memory.\n"); - return; - } + if (!slab) + return false; + kmem_obj_info(&kp, object, slab); if (kp.kp_slab_cache) pr_cont(" slab%s %s", cp, kp.kp_slab_cache->name); @@ -621,6 +601,7 @@ void kmem_dump_obj(void *object) pr_info(" %pS\n", kp.kp_free_stack[i]); } + return true; } EXPORT_SYMBOL_GPL(kmem_dump_obj); #endif diff --git a/mm/util.c b/mm/util.c index 8cbbfd3a3d59..6eddd891198e 100644 --- a/mm/util.c +++ b/mm/util.c @@ -1060,10 +1060,8 @@ void mem_dump_obj(void *object) { const char *type; - if (kmem_valid_obj(object)) { - kmem_dump_obj(object); + if (kmem_dump_obj(object)) return; - } if (vmalloc_dump_obj(object)) return; From patchwork Fri Oct 13 11:58:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 152548 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp1835153vqb; Fri, 13 Oct 2023 05:00:47 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEByXfOWFr2VvRzTehIkysUxE/HwlZ+XLm5NR05dOOrgQc8CobDFRzyDI7Hs3Y4fQaNTahK X-Received: by 2002:a17:90a:9f83:b0:27c:fbf8:6c43 with SMTP id o3-20020a17090a9f8300b0027cfbf86c43mr8713137pjp.1.1697198446414; Fri, 13 Oct 2023 05:00:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697198446; cv=none; d=google.com; s=arc-20160816; b=VLzLin53daLsIbziJD03AAcz7JLdDMsxBaCq2GXTxKFinwgh42yw+F/HqpEBBcKRWD G83JVqztCRGb75ezM4CIc62wY7JaYv+qkxHBHeoXc5LFIXeURmiw5EUaI2kw0z5kx9Ms KuxNVI4hjSesyvAvxoCerO8ESSlcVnTvfepAifBiEQKCouZythNDUMvbgXLWsscVRJEg 4xazDJymBbrpwOBOP4Yv9AB91jmTFZw+GhJfTuBBYDTzwcObnlE9N96evXQ90yojZAJ3 7leS84tp7bLPJdbGE6vD8XcdJduCTKzB+bOGG551gKYe4Xa1s2R//m+dWMhIwI9k3Crf TqaA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=w3Ul4cc3RYij2n0u+T/HYiGrdbWEPNBrmPYlpd+oshc=; fh=5+YmRexi1ebesok90zB32639YOcyyiie3F5fnFFoKq8=; b=nLZXUeo4r88qHJ55otOzHma9Zj9niKbpxYm6sEB5v4xqH5UsDCZ8VnMnat/UddYRR8 unjDJwwTmsmtgxD34VSpxDAljpfDBRZStW5qywN64okH22LkDnstmyuLwyReAgCePKjw FU/9vwoUKPwGxf4mbE8EhjGyxRpmw778lLxzWqhDTwPnA2szsJMukpXjZHXvr9iaCHA/ su3vUIxH+ObpOOzh093iNMIakrPGnR5tfXzOhkUqZaGfgOfmCV4AwOeeWCzqkBYuGLkb LGiJbVVSukyqH1HJu28Z0g4nUwgejueZS03JkS3AZVLswSuOex500gC3V8Uo0pY2FVs9 cNcw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b="F5x0R/yP"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from howler.vger.email (howler.vger.email. [23.128.96.34]) by mx.google.com with ESMTPS id mv8-20020a17090b198800b0026b698fdda6si4334672pjb.98.2023.10.13.05.00.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Oct 2023 05:00:46 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) client-ip=23.128.96.34; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b="F5x0R/yP"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id 8152282DFD04; Fri, 13 Oct 2023 05:00:35 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231548AbjJML7s (ORCPT + 19 others); Fri, 13 Oct 2023 07:59:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59850 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231439AbjJML7l (ORCPT ); Fri, 13 Oct 2023 07:59:41 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1E765D45; Fri, 13 Oct 2023 04:59:29 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 53D40C433C9; Fri, 13 Oct 2023 11:59:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1697198368; bh=wRGVcYhTxP8eygVB6ZOjmRcvjfz3wTIYLDKxh2D4weM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=F5x0R/yPpNxmwo2U54GxSzJU9oRfnwMWHyt+bQ9sH/wKADP3bX7rc0VQf/pKMOOTr 9/8WrWNIL1qlwiKxWWYTeOQ4SvoB3UrpkEDBuAabkUklQvo0g1LuO8EPNACMUuNvQC gVnOUIiaNqw6c1pSKBi7FXcwRBcpZolQLaqnGs2G3xeJigZB0SvA27uBc08eWUaXyl UIo1GsvCWRUL4wQgfpM33qFQhks6dyJXx3kyu1ZdDwk2sj2X6aiJ66/V7VV2q24ZDr 7ablgXg1odbuiGRx74KEm+KmXWeZMCYNogOxr3pZtT41YBFCJdqFt7rPyLof9ihrTj zAkaTSusRVk/Q== From: Frederic Weisbecker To: LKML Cc: Zhen Lei , Boqun Feng , Joel Fernandes , Josh Triplett , Mathieu Desnoyers , Neeraj Upadhyay , "Paul E . McKenney" , Steven Rostedt , Uladzislau Rezki , rcu , Frederic Weisbecker Subject: [PATCH 07/18] rcu: Dump memory object info if callback function is invalid Date: Fri, 13 Oct 2023 13:58:51 +0200 Message-Id: <20231013115902.1059735-8-frederic@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231013115902.1059735-1-frederic@kernel.org> References: <20231013115902.1059735-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on howler.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Fri, 13 Oct 2023 05:00:35 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779641558057353341 X-GMAIL-MSGID: 1779641558057353341 From: Zhen Lei When a structure containing an RCU callback rhp is (incorrectly) freed and reallocated after rhp is passed to call_rcu(), it is not unusual for rhp->func to be set to NULL. This defeats the debugging prints used by __call_rcu_common() in kernels built with CONFIG_DEBUG_OBJECTS_RCU_HEAD=y, which expect to identify the offending code using the identity of this function. And in kernels build without CONFIG_DEBUG_OBJECTS_RCU_HEAD=y, things are even worse, as can be seen from this splat: Unable to handle kernel NULL pointer dereference at virtual address 0 ... ... PC is at 0x0 LR is at rcu_do_batch+0x1c0/0x3b8 ... ... (rcu_do_batch) from (rcu_core+0x1d4/0x284) (rcu_core) from (__do_softirq+0x24c/0x344) (__do_softirq) from (__irq_exit_rcu+0x64/0x108) (__irq_exit_rcu) from (irq_exit+0x8/0x10) (irq_exit) from (__handle_domain_irq+0x74/0x9c) (__handle_domain_irq) from (gic_handle_irq+0x8c/0x98) (gic_handle_irq) from (__irq_svc+0x5c/0x94) (__irq_svc) from (arch_cpu_idle+0x20/0x3c) (arch_cpu_idle) from (default_idle_call+0x4c/0x78) (default_idle_call) from (do_idle+0xf8/0x150) (do_idle) from (cpu_startup_entry+0x18/0x20) (cpu_startup_entry) from (0xc01530) This commit therefore adds calls to mem_dump_obj(rhp) to output some information, for example: slab kmalloc-256 start ffff410c45019900 pointer offset 0 size 256 This provides the rough size of the memory block and the offset of the rcu_head structure, which as least provides at least a few clues to help locate the problem. If the problem is reproducible, additional slab debugging can be enabled, for example, CONFIG_DEBUG_SLAB=y, which can provide significantly more information. Signed-off-by: Zhen Lei Signed-off-by: Paul E. McKenney Signed-off-by: Frederic Weisbecker --- kernel/rcu/rcu.h | 7 +++++++ kernel/rcu/srcutiny.c | 1 + kernel/rcu/srcutree.c | 1 + kernel/rcu/tasks.h | 1 + kernel/rcu/tiny.c | 1 + kernel/rcu/tree.c | 1 + 6 files changed, 12 insertions(+) diff --git a/kernel/rcu/rcu.h b/kernel/rcu/rcu.h index 98e13be411af..d612731feea4 100644 --- a/kernel/rcu/rcu.h +++ b/kernel/rcu/rcu.h @@ -10,6 +10,7 @@ #ifndef __LINUX_RCU_H #define __LINUX_RCU_H +#include #include /* @@ -248,6 +249,12 @@ static inline void debug_rcu_head_unqueue(struct rcu_head *head) } #endif /* #else !CONFIG_DEBUG_OBJECTS_RCU_HEAD */ +static inline void debug_rcu_head_callback(struct rcu_head *rhp) +{ + if (unlikely(!rhp->func)) + kmem_dump_obj(rhp); +} + extern int rcu_cpu_stall_suppress_at_boot; static inline bool rcu_stall_is_suppressed_at_boot(void) diff --git a/kernel/rcu/srcutiny.c b/kernel/rcu/srcutiny.c index 336af24e0fe3..c38e5933a5d6 100644 --- a/kernel/rcu/srcutiny.c +++ b/kernel/rcu/srcutiny.c @@ -138,6 +138,7 @@ void srcu_drive_gp(struct work_struct *wp) while (lh) { rhp = lh; lh = lh->next; + debug_rcu_head_callback(rhp); local_bh_disable(); rhp->func(rhp); local_bh_enable(); diff --git a/kernel/rcu/srcutree.c b/kernel/rcu/srcutree.c index f1a905200fc2..833a8f848a90 100644 --- a/kernel/rcu/srcutree.c +++ b/kernel/rcu/srcutree.c @@ -1710,6 +1710,7 @@ static void srcu_invoke_callbacks(struct work_struct *work) rhp = rcu_cblist_dequeue(&ready_cbs); for (; rhp != NULL; rhp = rcu_cblist_dequeue(&ready_cbs)) { debug_rcu_head_unqueue(rhp); + debug_rcu_head_callback(rhp); local_bh_disable(); rhp->func(rhp); local_bh_enable(); diff --git a/kernel/rcu/tasks.h b/kernel/rcu/tasks.h index 8d65f7d576a3..7c845532a50a 100644 --- a/kernel/rcu/tasks.h +++ b/kernel/rcu/tasks.h @@ -538,6 +538,7 @@ static void rcu_tasks_invoke_cbs(struct rcu_tasks *rtp, struct rcu_tasks_percpu raw_spin_unlock_irqrestore_rcu_node(rtpcp, flags); len = rcl.len; for (rhp = rcu_cblist_dequeue(&rcl); rhp; rhp = rcu_cblist_dequeue(&rcl)) { + debug_rcu_head_callback(rhp); local_bh_disable(); rhp->func(rhp); local_bh_enable(); diff --git a/kernel/rcu/tiny.c b/kernel/rcu/tiny.c index 42f7589e51e0..fec804b79080 100644 --- a/kernel/rcu/tiny.c +++ b/kernel/rcu/tiny.c @@ -97,6 +97,7 @@ static inline bool rcu_reclaim_tiny(struct rcu_head *head) trace_rcu_invoke_callback("", head); f = head->func; + debug_rcu_head_callback(head); WRITE_ONCE(head->func, (rcu_callback_t)0L); f(head); rcu_lock_release(&rcu_callback_map); diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index 3c7281fc25a7..aae515071ffd 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -2135,6 +2135,7 @@ static void rcu_do_batch(struct rcu_data *rdp) trace_rcu_invoke_callback(rcu_state.name, rhp); f = rhp->func; + debug_rcu_head_callback(rhp); WRITE_ONCE(rhp->func, (rcu_callback_t)0L); f(rhp); From patchwork Fri Oct 13 11:58:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 152546 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp1834995vqb; Fri, 13 Oct 2023 05:00:36 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGI4jpnXYOD7FTOt+hGz34stkXaSdXp95S3xB/5l17t+3KcqsD+RJIwYgCjTt7c4+Sf7ku8 X-Received: by 2002:a05:6830:3596:b0:6b7:36af:1937 with SMTP id bd22-20020a056830359600b006b736af1937mr23519086otb.0.1697198435915; Fri, 13 Oct 2023 05:00:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697198435; cv=none; d=google.com; s=arc-20160816; b=Ap+kJbbxUescelIiprmzp30WSCK6VmTDNoYmp+3xsblU7PMXtbMvU6gOQVCOUuxHi3 4uxUKZe8e7eVWillp3aJLTeAqURTolWatYwCvI/Tr4cS4y+rpSSwwg6hZZmCs29RS1Qf 6d/5Ex+b3iv3E6fG8SsqKurDpGgVQfB+DsHjn6aEbXSjmKiBjx2FVZJyQKYyc68N9uRf N8ih/ZF8oE4szMscYgTZRJijR2GfJh3nj8Purkn7GVDKnjpiPzBwn8sgtX9R2JhWlS96 tu8XRGHFa50E81D+GgOBAN8bCogi1/zHAbevOG0l6SzpeZNSPSAT9OzQhfW4QA9Voh8D Xcog== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=3WdXr4pFv0XertZ1TUUov8OPMjA1N6dRXE5rVxoxeFA=; fh=9o1vzmbQvXEiiOZAoGEKGHmVR4Ieb6r+octdNZgYFWA=; b=ckkyAG5DOnTTOYGdKlYSZK3Vhzsrg1ImKiEFhRkyGfODiYTRr0AH2BE83E3+gzjwjg YKgZGGGroi1w6G3l8/X/vovkL09Al8tJZaTAwi8cq4sZM9SlewlpKhjHqr4vg4QD0WMd padrlTGNtfCQHJ7pF0e06G+7TEJMXvlFd6Wy9Mfn03XvqAAS/Vlj8FTTPnbREduL4FAb 4YlHMvCxxHM5VUCndaLxu5xx6L+SeNBEbt/MsgTDtBkBUd+MeqLBSz9nRn536jhachPq hS7l+1nCmRYX5q5vDEtNvIMwkAkX3/7rkoj51D9KHvC8THNGVzZALdTlXqGLNHx7NwE3 6eBg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b="R/IY2FxF"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from howler.vger.email (howler.vger.email. [2620:137:e000::3:4]) by mx.google.com with ESMTPS id o2-20020a635d42000000b0059bdf5a4ecfsi4290891pgm.247.2023.10.13.05.00.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Oct 2023 05:00:35 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) client-ip=2620:137:e000::3:4; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b="R/IY2FxF"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id 1ECFE8072A0B; Fri, 13 Oct 2023 05:00:24 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231549AbjJMMAA (ORCPT + 19 others); Fri, 13 Oct 2023 08:00:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52612 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231417AbjJML7n (ORCPT ); Fri, 13 Oct 2023 07:59:43 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9A62F195; Fri, 13 Oct 2023 04:59:32 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 44EB7C433CC; Fri, 13 Oct 2023 11:59:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1697198371; bh=ZIdTdcHLWr/sOB8PSUXqF37fOevVbYeiYj5kEREzCsU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=R/IY2FxF4nhhUiT9TcBAKeVscINBGdP8AOURhdYyTW5OdOfQPSZhvxYgXNeF+OSLB RZVjG9nesoqKrx0CGwMsydjOREuwtc65EUqeu+/3QHAj+jZA9G5GbZ0NyYpbGFvUeG 9TcLiTxVjjklcl+q7u6hIfeDjRpwpmq7/Q0zVA1GKs9CgRLDGKM/p+cCh1GvrOxznk 6KYgWu3arpTEttQ3S5QGF6qK9c84lWZ6/RVHVR173t9EcGx9TF21volEDZ2+J/g7HQ z/xqXH25wBkZO6aeKcj11GQWPjrvDjTjhhH58xIC894Soi7glIXnRRIEhjMzsVUgYN hw6qmljXrTeOg== From: Frederic Weisbecker To: LKML Cc: "Paul E. McKenney" , Boqun Feng , Joel Fernandes , Josh Triplett , Mathieu Desnoyers , Neeraj Upadhyay , Steven Rostedt , Uladzislau Rezki , rcu , Frederic Weisbecker Subject: [PATCH 08/18] rcu: Eliminate rcu_gp_slow_unregister() false positive Date: Fri, 13 Oct 2023 13:58:52 +0200 Message-Id: <20231013115902.1059735-9-frederic@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231013115902.1059735-1-frederic@kernel.org> References: <20231013115902.1059735-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on howler.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Fri, 13 Oct 2023 05:00:24 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779641547220109713 X-GMAIL-MSGID: 1779641547220109713 From: "Paul E. McKenney" When using rcutorture as a module, there are a number of conditions that can abort the modprobe operation, for example, when attempting to run both RCU CPU stall warning tests and forward-progress tests. This can cause rcu_torture_cleanup() to be invoked on the unwind path out of rcu_rcu_torture_init(), which will mean that rcu_gp_slow_unregister() is invoked without a matching rcu_gp_slow_register(). This will cause a splat because rcu_gp_slow_unregister() is passed rcu_fwd_cb_nodelay, which does not match a NULL pointer. This commit therefore forgives a mismatch involving a NULL pointer, thus avoiding this false-positive splat. Signed-off-by: Paul E. McKenney Signed-off-by: Frederic Weisbecker --- kernel/rcu/tree.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index aae515071ffd..a83ecab77917 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -1260,7 +1260,7 @@ EXPORT_SYMBOL_GPL(rcu_gp_slow_register); /* Unregister a counter, with NULL for not caring which. */ void rcu_gp_slow_unregister(atomic_t *rgssp) { - WARN_ON_ONCE(rgssp && rgssp != rcu_gp_slow_suppress); + WARN_ON_ONCE(rgssp && rgssp != rcu_gp_slow_suppress && rcu_gp_slow_suppress != NULL); WRITE_ONCE(rcu_gp_slow_suppress, NULL); } From patchwork Fri Oct 13 11:58:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 152556 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp1836126vqb; Fri, 13 Oct 2023 05:01:49 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFWvuVCG2qek4LGLibqMTLEtvfT203yHrWJnZb63dRU2Nza2SwHVzrjQlEVsxg5MwciFENq X-Received: by 2002:a17:902:d2c1:b0:1c1:fbec:bc32 with SMTP id n1-20020a170902d2c100b001c1fbecbc32mr29587927plc.6.1697198508856; Fri, 13 Oct 2023 05:01:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697198508; cv=none; d=google.com; s=arc-20160816; b=tVs5JNgDey0Eu1VtJMJyh8xfy+Y/5P/88zduSJPH2FOSRXU2Zx0kkM9vqcM4RrbAND lE0sT8KdqXA6tFxo/gqsHgn5+0aRgdoqjkFSy8NuP4ZQdQiyRc5IW9dXcRoGTsyqGPI8 R3iWxsoNx3zcvVwQKMmo2tmStw9CFNktq8Wy4j6wyTv4pMw3yXt4VxYTvfjvoJaA8+sY wGCl5oe3WaUMKyx5JH0iua/CT/oPqrodk5oQU1KoPptGP7Yem6I9j42cLhU0xSCvD+US kC7WwQ3PvudThPy8C4dtaxEAfJ8whGfeSykrC8JBxCW57HNhdv8NjD/l8UMjIUOf0UKd J4DQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=+a4uX7XK7PKfnwOF/S8+jTgEL5zom67SmynF6u6otrs=; fh=apAv99im5Gv0zBU05FLrPSrn/ZsZRTUASF+guKwNYP8=; b=A4sn5fEU94aatpqJ0RZcR8JzLnn6Jv+V4Onsu9wjmsoXildIKtWXOSzwkR+3cQUbo6 6XaC89tsKR4uI0afmnEaz41B7UwmceM2D+N/SIFkx1kX9yR/s4KDUkhBuvLxw/2nXE+2 KTgzZBVBFqq+107GW3x9cUKbkBOwyCumwQZEp1JkXzHMXd8eWSGMYRvL1IOKc0ogJfJj yMx//hWk9yilYhHY/wugZ66x8KfEUHH1RkSYKWOdlKv15RDVPaVtoIr2lHG4Kt/DQ8Uj dfmH7rh0cQCbVU+ikSDGJHQFJOPm1pJWyrBolMfC7N8KDIzs6hWHQn7KGY0F9xr/ul9F jhvQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=ZHDeb4vU; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from groat.vger.email (groat.vger.email. [2620:137:e000::3:5]) by mx.google.com with ESMTPS id ba7-20020a170902720700b001c5da8b630fsi4231947plb.421.2023.10.13.05.01.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Oct 2023 05:01:48 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) client-ip=2620:137:e000::3:5; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=ZHDeb4vU; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by groat.vger.email (Postfix) with ESMTP id BB2698075B36; Fri, 13 Oct 2023 05:01:17 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at groat.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231215AbjJMMAD (ORCPT + 19 others); Fri, 13 Oct 2023 08:00:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52722 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231754AbjJML7p (ORCPT ); Fri, 13 Oct 2023 07:59:45 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7FFA4D69; Fri, 13 Oct 2023 04:59:35 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0A494C433CD; Fri, 13 Oct 2023 11:59:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1697198374; bh=a8yesXiumFDXUGe/vS4l+QF/cFiA8A3MQEGVRdpXwvk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ZHDeb4vUdx7qWgGSZJE/nx0gtzccqoxRyGcZ9d0EglgM47SVCYp1pZP3LbfqA9ZqT JuDwr0cbFlkB+biF4i+Au6x1OkqDbBEYxtotxb4Z1+QWvXY0J8U88Vz6V2jZYXUkMc 8JL9hcjp3zA0htcNejdqlDgMOWyJWpfC89WT+51bw0Aui8PX3wunu6vdEe1YJUBIn3 eH8SVGfWe5F5FHfBSq7tpcaEuinewZEz032nQXs21HPvw4H3LROENMK8+e9bkcmQRr lfNCCcYJUHPSVm7a3l/xfusbwnKDT5JDXp+rHFscj3QKVd+K5EqtM9NpMfFjrBNCU5 RWIQ8NYcPmGcw== From: Frederic Weisbecker To: LKML Cc: Denis Arefev , Boqun Feng , Joel Fernandes , Josh Triplett , Mathieu Desnoyers , Neeraj Upadhyay , "Paul E . McKenney" , Steven Rostedt , Uladzislau Rezki , rcu , David Laight , Frederic Weisbecker Subject: [PATCH 09/18] srcu: Fix srcu_struct node grpmask overflow on 64-bit systems Date: Fri, 13 Oct 2023 13:58:53 +0200 Message-Id: <20231013115902.1059735-10-frederic@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231013115902.1059735-1-frederic@kernel.org> References: <20231013115902.1059735-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Fri, 13 Oct 2023 05:01:17 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779641623776478390 X-GMAIL-MSGID: 1779641623776478390 From: Denis Arefev The value of a bitwise expression 1 << (cpu - sdp->mynode->grplo) is subject to overflow due to a failure to cast operands to a larger data type before performing the bitwise operation. The maximum result of this subtraction is defined by the RCU_FANOUT_LEAF Kconfig option, which on 64-bit systems defaults to 16 (resulting in a maximum shift of 15), but which can be set up as high as 64 (resulting in a maximum shift of 63). A value of 31 can result in sign extension, resulting in 0xffffffff80000000 instead of the desired 0x80000000. A value of 32 or greater triggers undefined behavior per the C standard. This bug has not been known to cause issues because almost all kernels take the default CONFIG_RCU_FANOUT_LEAF=16. Furthermore, as long as a given compiler gives a deterministic non-zero result for 1<=32, the code correctly invokes all SRCU callbacks, albeit wasting CPU time along the way. This commit therefore substitutes the correct 1UL for the buggy 1. Found by Linux Verification Center (linuxtesting.org) with SVACE. Signed-off-by: Denis Arefev Reviewed-by: Mathieu Desnoyers Reviewed-by: Joel Fernandes (Google) Cc: David Laight Signed-off-by: Paul E. McKenney Signed-off-by: Frederic Weisbecker --- kernel/rcu/srcutree.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/kernel/rcu/srcutree.c b/kernel/rcu/srcutree.c index 833a8f848a90..5602042856b1 100644 --- a/kernel/rcu/srcutree.c +++ b/kernel/rcu/srcutree.c @@ -223,7 +223,7 @@ static bool init_srcu_struct_nodes(struct srcu_struct *ssp, gfp_t gfp_flags) snp->grplo = cpu; snp->grphi = cpu; } - sdp->grpmask = 1 << (cpu - sdp->mynode->grplo); + sdp->grpmask = 1UL << (cpu - sdp->mynode->grplo); } smp_store_release(&ssp->srcu_sup->srcu_size_state, SRCU_SIZE_WAIT_BARRIER); return true; @@ -835,7 +835,7 @@ static void srcu_schedule_cbs_snp(struct srcu_struct *ssp, struct srcu_node *snp int cpu; for (cpu = snp->grplo; cpu <= snp->grphi; cpu++) { - if (!(mask & (1 << (cpu - snp->grplo)))) + if (!(mask & (1UL << (cpu - snp->grplo)))) continue; srcu_schedule_cbs_sdp(per_cpu_ptr(ssp->sda, cpu), delay); } From patchwork Fri Oct 13 11:58:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 152555 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp1836077vqb; Fri, 13 Oct 2023 05:01:46 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGDfiopguEh5bRFemnf+HYMpQQs4sswFCrwdOeTRiEQnK+3i4LkKP+SxobhbbgV2byiSUYu X-Received: by 2002:a17:90a:e7c8:b0:27d:193f:2130 with SMTP id kb8-20020a17090ae7c800b0027d193f2130mr5472518pjb.3.1697198506173; Fri, 13 Oct 2023 05:01:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697198506; cv=none; d=google.com; s=arc-20160816; b=AM1EePXHaui9EHMuYCvCG2aqOfhkFo9ijmEfMSc/4CfHcTUuSmb3+ECQo1u1/Y0GFE AENM7x3sVIaClh0qtKMcg2+aKhpzLuiIdqwd0MlbpKo8tcCWEesMg0Fm2eOTiT5AXLFn XAlkj3uFYrPk7gkOFKbcmv3pTczhTg8Ke7uBGv6Q6qn6xD+i+0DES4eVSQFTUWDw4rYb qqWBtuax76GjxJqVaef51L+HH0RKPNlKTAzpeFDHmjSvI7CXOrRd+kG5/2CyiCMK8t03 FsCdvyjmBnhCsZy1Oig46H7umECum//WOqDbVmz9Ep55WOkjppzf2cs1AugkKkINegeo qWXQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=R/Sw9FfTZe1474Cz8POWQbBaAm75ih+JhaEBLGhnX98=; fh=jK9ixGRUU8raHpPCWxwbiA62zUtZcZpLBtwcIoKuB+A=; b=otf5Hxrp8dkE7MYqTvOGtfgJ+eJAIDwI36qLU4GcoY2D87xhSJOTK9ngZSFfIe387S idm1zgYs9DqgwkFe0akzOuqYagPGc6xpZrhyGR6w7Zfcd4dbgLj8QFHdH/fhpsbMhfZb NQmGjz7nZV9otRl8CSiiSZsTZsB+gASvLKjZ98aKUJ/FSmIXcJVSMAmDwvre6+gtqPuA gSgLYGK3PbOwoeWRZoLZiow7+VipD64p/0eGvBaG1YQd4rV1x8hkYPC6kQNx0wDalum5 TImY3/BBJD53LKrGqjLOsERb/j+NO79O+2543sJZpaIGRCZ51IwU9DQodLtMIp77GawS AfMA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=ovliKqTX; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from groat.vger.email (groat.vger.email. [2620:137:e000::3:5]) by mx.google.com with ESMTPS id ce13-20020a17090aff0d00b002792831692csi4320472pjb.58.2023.10.13.05.01.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Oct 2023 05:01:46 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) client-ip=2620:137:e000::3:5; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=ovliKqTX; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by groat.vger.email (Postfix) with ESMTP id CC88C805F5E1; Fri, 13 Oct 2023 05:00:37 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at groat.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231656AbjJMMAO (ORCPT + 19 others); Fri, 13 Oct 2023 08:00:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52098 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231522AbjJML7s (ORCPT ); Fri, 13 Oct 2023 07:59:48 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7104F10D8; Fri, 13 Oct 2023 04:59:38 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 281D9C433CB; Fri, 13 Oct 2023 11:59:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1697198378; bh=rCN4+iCoVX4jAGs7s+XIp9Rriuj5msPrP6NA95JLYbU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ovliKqTXYm9ZYVDaU9YCJen8c0uPB5I5KkvcUh/pIuFXCeqwgg0J9HJoZqsXWgoZT RKOrBm+E/4wowCXIiQA9tImmtOtIThJHQVRpdXzcHG2aFHZEHMl9I4cdixYdhZp+ha r3y2S2qPXMXFG3RYOPUEWfjb1P+OZGnxQQWX1p7C+2Q5V8yfAjIQNYU9g9Ti5OKjhO BZXFXH2jN1uME3IdN+JHHE8EXjk1JCkYI+vS1NsiVa/wrGUA70vjCm+/WvNM6+oPSy yJ/C+1ku0C1YknO82v7AzPqK41BYopgsZ3QFQ3KXIH1rg9WCF6L+4W6OfS/WKU8gAY bwtQM7d7Nq/zw== From: Frederic Weisbecker To: LKML Cc: Catalin Marinas , Boqun Feng , Joel Fernandes , Josh Triplett , Mathieu Desnoyers , Neeraj Upadhyay , "Paul E . McKenney" , Steven Rostedt , Uladzislau Rezki , rcu , Christoph Paasch , stable@vger.kernel.org, Frederic Weisbecker Subject: [PATCH 10/18] rcu: kmemleak: Ignore kmemleak false positives when RCU-freeing objects Date: Fri, 13 Oct 2023 13:58:54 +0200 Message-Id: <20231013115902.1059735-11-frederic@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231013115902.1059735-1-frederic@kernel.org> References: <20231013115902.1059735-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Fri, 13 Oct 2023 05:00:38 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779641621082722950 X-GMAIL-MSGID: 1779641621082722950 From: Catalin Marinas Since the actual slab freeing is deferred when calling kvfree_rcu(), so is the kmemleak_free() callback informing kmemleak of the object deletion. From the perspective of the kvfree_rcu() caller, the object is freed and it may remove any references to it. Since kmemleak does not scan RCU internal data storing the pointer, it will report such objects as leaks during the grace period. Tell kmemleak to ignore such objects on the kvfree_call_rcu() path. Note that the tiny RCU implementation does not have such issue since the objects can be tracked from the rcu_ctrlblk structure. Signed-off-by: Catalin Marinas Reported-by: Christoph Paasch Closes: https://lore.kernel.org/all/F903A825-F05F-4B77-A2B5-7356282FBA2C@apple.com/ Cc: Tested-by: Christoph Paasch Reviewed-by: Paul E. McKenney Signed-off-by: Joel Fernandes (Google) Signed-off-by: Frederic Weisbecker --- kernel/rcu/tree.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index a83ecab77917..4dd7df30df31 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -31,6 +31,7 @@ #include #include #include +#include #include #include #include @@ -3389,6 +3390,14 @@ void kvfree_call_rcu(struct rcu_head *head, void *ptr) success = true; } + /* + * The kvfree_rcu() caller considers the pointer freed at this point + * and likely removes any references to it. Since the actual slab + * freeing (and kmemleak_free()) is deferred, tell kmemleak to ignore + * this object (no scanning or false positives reporting). + */ + kmemleak_ignore(ptr); + // Set timer to drain after KFREE_DRAIN_JIFFIES. if (rcu_scheduler_active == RCU_SCHEDULER_RUNNING) schedule_delayed_monitor_work(krcp); From patchwork Fri Oct 13 11:58:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 152552 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp1835654vqb; Fri, 13 Oct 2023 05:01:20 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEpH4t8W5gR9EB4KKOUzJp0qnV7d/wKUQr9UHSq7VA4MeEfh4FnSBTDsFcDDK6pINjcU90I X-Received: by 2002:a17:90b:1d87:b0:27d:2261:73e4 with SMTP id pf7-20020a17090b1d8700b0027d226173e4mr3935048pjb.2.1697198480587; Fri, 13 Oct 2023 05:01:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697198480; cv=none; d=google.com; s=arc-20160816; b=xnf+3fgc3WlW4ZYD3jkXUzAbWTZXN+qYV60GtS+Q7WKRmRCxMXNHwp4Cf72ZZJgGad z3upwNJFFqbP8DYWtn1SZ+wON6k211+majmTpo/xixYox6FQI1d9U5GggLGkKmcUUYlf qelgA2bWt9haKVydbAqjSlBCrvO14/pnDeOZIENNxFyQfgl1MaZNAvMAIl8T7wajbBR/ zNhW7b/gDRv3S7W1VxgKMAXe6op3OQKbgO/+NdF8+Lf/vn86t8xtcsEmtcpFlmp1Vl0N GoBMKDSp4u7A2mQkvHaQ8C2IZEp1B1ky0RQ+WNenu9dQiNA9EYC21elCvkKQBSkYhbH1 Kawg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=qkiyQcL0Juz4c3OmLP36cb82z+rkMu9s2wCCT95yb6g=; fh=EnTBLYqW3ZqGVDhG+3QSxSPwUQwq5qMu5oTnNM8+B2k=; b=ojqdtTiRXrNOAT2x6wkF/ubbCHn9kPML6miZlCE0PN0zcxrK4xTBTUI5DdxMSpoZle oVDpsL7A8UcGW0XAC8VnIxVcCW0vS0MhnsAJ+k41Wmc5SevQB8ENg5GRlCwCmkxB0lbB Vbu8uRjuak62AsPKVwvnsYhcJcIBVUsl5S2wWQOn8nEDJDYREJDqjqvPp3e0hqPUWYB/ Ns0BGJigYQ5Ij2Ae1z7doBi3le0SbySgU8h066mB/WByu5CbtP4rsyylG4miWCytTrv5 v6UQoAXdYlzCbBtN5TYVmbMAoJVE0UoyTbSxotvke3jvJeAYZgfRZP7C/8BA0g1BIazL x2MQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=Lc6G8raI; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from howler.vger.email (howler.vger.email. [23.128.96.34]) by mx.google.com with ESMTPS id nn13-20020a17090b38cd00b00276b87c8b8esi3225423pjb.59.2023.10.13.05.01.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Oct 2023 05:01:20 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) client-ip=23.128.96.34; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=Lc6G8raI; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id C068782CEFFA; Fri, 13 Oct 2023 05:01:08 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231807AbjJMMAV (ORCPT + 19 others); Fri, 13 Oct 2023 08:00:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46058 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231808AbjJML7w (ORCPT ); Fri, 13 Oct 2023 07:59:52 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8E28010F5; Fri, 13 Oct 2023 04:59:41 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6F9AAC433C7; Fri, 13 Oct 2023 11:59:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1697198380; bh=9eKXMVkzj30bJjMYB0HS6GlgZmQ+aae2FrSYU4LFmqA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Lc6G8raIpvm9yT+YjPvUXfqlCcBSGvsvCO41iqsHlWYs4HoQlQBrbHeBxldNK5ZS3 pONJYQJQN8KuhB5BjprBpq9wf/9T9SvMRBG39QxxHXQ1rPgL1L80QL2KM2Hrx+aklz VKsa6uz21SOzhNNhM6Hq/Rb5xNiB08jMy8WbNKBCHz6AL97J0u12oBp63SDRdoOUq1 oRWY9+04XjJdF+ONPBtdBPLScLHxAB26DrcYD0AxeKLVNuuYdMOuxkOcszopNNoTeT hVSoEl1CZ5KCLZpPXkhjcr8tS+BpK++87WcEIiVWxi/QTe9H8Uj42fxnsq8L+LRdC9 osyzbuL8PBVqQ== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Boqun Feng , Joel Fernandes , Josh Triplett , Mathieu Desnoyers , Neeraj Upadhyay , "Paul E . McKenney" , Steven Rostedt , Uladzislau Rezki , rcu , Qiuxu Zhuo Subject: [PATCH 11/18] rcu: Use rcu_segcblist_segempty() instead of open coding it Date: Fri, 13 Oct 2023 13:58:55 +0200 Message-Id: <20231013115902.1059735-12-frederic@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231013115902.1059735-1-frederic@kernel.org> References: <20231013115902.1059735-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on howler.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Fri, 13 Oct 2023 05:01:08 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779641593921693890 X-GMAIL-MSGID: 1779641593921693890 This makes the code more readable. Reviewed-by: Qiuxu Zhuo Reviewed-by: Joel Fernandes (Google) Reviewed-by: Paul E. McKenney Signed-off-by: Frederic Weisbecker --- kernel/rcu/rcu_segcblist.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/kernel/rcu/rcu_segcblist.c b/kernel/rcu/rcu_segcblist.c index f71fac422c8f..1693ea22ef1b 100644 --- a/kernel/rcu/rcu_segcblist.c +++ b/kernel/rcu/rcu_segcblist.c @@ -368,7 +368,7 @@ bool rcu_segcblist_entrain(struct rcu_segcblist *rsclp, smp_mb(); /* Ensure counts are updated before callback is entrained. */ rhp->next = NULL; for (i = RCU_NEXT_TAIL; i > RCU_DONE_TAIL; i--) - if (rsclp->tails[i] != rsclp->tails[i - 1]) + if (!rcu_segcblist_segempty(rsclp, i)) break; rcu_segcblist_inc_seglen(rsclp, i); WRITE_ONCE(*rsclp->tails[i], rhp); @@ -551,7 +551,7 @@ bool rcu_segcblist_accelerate(struct rcu_segcblist *rsclp, unsigned long seq) * as their ->gp_seq[] grace-period completion sequence number. */ for (i = RCU_NEXT_READY_TAIL; i > RCU_DONE_TAIL; i--) - if (rsclp->tails[i] != rsclp->tails[i - 1] && + if (!rcu_segcblist_segempty(rsclp, i) && ULONG_CMP_LT(rsclp->gp_seq[i], seq)) break; From patchwork Fri Oct 13 11:58:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 152550 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp1835262vqb; Fri, 13 Oct 2023 05:00:55 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFTXu+XXYLQ/LTnGTUueVCsOmpYmrbek7zKmb1gWk2IPlw+9+Iyr9vEAS4xtwhybOf9hvfQ X-Received: by 2002:a05:6870:c689:b0:1e1:e6ee:94b6 with SMTP id cv9-20020a056870c68900b001e1e6ee94b6mr29065538oab.4.1697198455449; Fri, 13 Oct 2023 05:00:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697198455; cv=none; d=google.com; s=arc-20160816; b=JFTF0c07ySHFQTlGFzAfVZUPC+sdiUsNLx6jCKo/chcdAegE/3zEsNc9Rxfti+a4B4 XW9V6Bi11aPeMwkCd8m6/pLvNyUwuULjaBqHdiIFdCx+31buKyEw6uMm1jTma0dw6MZC kKWm97xMashmvxi3mqPqLgT/GK5pi8Nauts3uRrCaaJquHSGFDyd1jYBsN5WPsCO+6iF RsZelCfCahwrqEGYDeS8AG8xPYLyI7sjU4K02msW8J9INNU67jT7R+oBM32U1N5vAtUx UlFFb50zyp/oH1pJUyrLHOqUvWIWscVhqEiBSHRUszlP2h7WY3XBfvd4BmNAjxe5uX2R UcDg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=5Xs+vRA2xE8p4nJrqsl3oYiPq5fiqust5CRhlU6snas=; fh=lTKEK1Rt7LvnHRfa4w1cQ2RgMDufzBK6qxi+oKKQlPU=; b=PjRe9I99MpwW2gW8s4y2goY7YwF2i1ZglQvUToth2vdzqzcSmI9zoAw/Bqgj2YJoHW pegKjCk+ONz+xIdcqM05y2ozmUJbf58S9GTJnB4Vbe4oZ2eaSfnjql+Gyi/nzPDi1Ezc Vd8F+B9EQXUY+iKVF/YcDfaQVTczA5qNLdzifv6T+PGxOE5ydbkEabPSHzYIFdDSIEBu 34ffYl28z5UQ+81vJKYs3XyLA73iNWX7Vi0IU3TMwqpuqJxR1UuAe7+Uzog7omn6tOot N6dXGlX3yr1/EFJ7FCn9QjJXzy4RYX41qtleTLs+oSgWCdTiZrlu/t55WmJiZ4rRg98q fdWg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=Num1RS1S; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from pete.vger.email (pete.vger.email. [23.128.96.36]) by mx.google.com with ESMTPS id b69-20020a633448000000b005ae39764b64si254583pga.135.2023.10.13.05.00.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Oct 2023 05:00:55 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) client-ip=23.128.96.36; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=Num1RS1S; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by pete.vger.email (Postfix) with ESMTP id 42E378228811; Fri, 13 Oct 2023 05:00:46 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at pete.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231835AbjJMMAY (ORCPT + 19 others); Fri, 13 Oct 2023 08:00:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59998 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231703AbjJML76 (ORCPT ); Fri, 13 Oct 2023 07:59:58 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1AC0BF0; Fri, 13 Oct 2023 04:59:44 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 60DC6C433CC; Fri, 13 Oct 2023 11:59:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1697198383; bh=Jc8jfSDSYlcGNx7gh6POnFvbeLKzhx7hBRosliLbEyA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Num1RS1S+VHmhSvhhfUQFeinEqmcgTGDJDAIaONbq3FPkAFHzwUwOuvjJy8og4CQ7 q81UTKTc6Nx18Ce5vhFTky4BySoGMXuBiiOhVrWipiajLoRtu3GN77cXBFdCzpwRi9 ZCZV6NqAsS/oSBzFeZcgZ6nYtwfbmLl4YejSqQjzoBod3Xz2ogScl+ge2Ez2Zprbys elgAwJhY4FsXsi+9tjsMokcVAg+z8c7hmo0tBd9ZwIX6b6Pfb4OIxQEelAeQBKeCeG 7pX4k7yEGk+3ZFDOYAWJKTn3Lro4s6k26vfj3sC3tm4MeGLIQ/76JO8ktlnQjjIeFj vLl+KJdpcDdzg== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Boqun Feng , Joel Fernandes , Josh Triplett , Mathieu Desnoyers , Neeraj Upadhyay , "Paul E . McKenney" , Steven Rostedt , Uladzislau Rezki , rcu Subject: [PATCH 12/18] rcu: Assume IRQS disabled from rcu_report_dead() Date: Fri, 13 Oct 2023 13:58:56 +0200 Message-Id: <20231013115902.1059735-13-frederic@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231013115902.1059735-1-frederic@kernel.org> References: <20231013115902.1059735-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Fri, 13 Oct 2023 05:00:46 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779641567281509196 X-GMAIL-MSGID: 1779641567281509196 rcu_report_dead() is the last RCU word from the CPU down through the hotplug path. It is called in the idle loop right before the CPU shuts down for good. Because it removes the CPU from the grace period state machine and reports an ultimate quiescent state if necessary, no further use of RCU is allowed. Therefore it is expected that IRQs are disabled upon calling this function and are not to be re-enabled again until the CPU shuts down. Remove the IRQs disablement from that function and verify instead that it is actually called with IRQs disabled as it is expected at that special point in the idle path. Reviewed-by: Joel Fernandes (Google) Reviewed-by: Paul E. McKenney Signed-off-by: Frederic Weisbecker --- kernel/rcu/tree.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index 4dd7df30df31..8c2954502e55 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -4562,11 +4562,16 @@ void rcu_cpu_starting(unsigned int cpu) */ void rcu_report_dead(unsigned int cpu) { - unsigned long flags, seq_flags; + unsigned long flags; unsigned long mask; struct rcu_data *rdp = per_cpu_ptr(&rcu_data, cpu); struct rcu_node *rnp = rdp->mynode; /* Outgoing CPU's rdp & rnp. */ + /* + * IRQS must be disabled from now on and until the CPU dies, or an interrupt + * may introduce a new READ-side while it is actually off the QS masks. + */ + lockdep_assert_irqs_disabled(); // Do any dangling deferred wakeups. do_nocb_deferred_wakeup(rdp); @@ -4574,7 +4579,6 @@ void rcu_report_dead(unsigned int cpu) /* Remove outgoing CPU from mask in the leaf rcu_node structure. */ mask = rdp->grpmask; - local_irq_save(seq_flags); arch_spin_lock(&rcu_state.ofl_lock); raw_spin_lock_irqsave_rcu_node(rnp, flags); /* Enforce GP memory-order guarantee. */ rdp->rcu_ofl_gp_seq = READ_ONCE(rcu_state.gp_seq); @@ -4588,8 +4592,6 @@ void rcu_report_dead(unsigned int cpu) WRITE_ONCE(rnp->qsmaskinitnext, rnp->qsmaskinitnext & ~mask); raw_spin_unlock_irqrestore_rcu_node(rnp, flags); arch_spin_unlock(&rcu_state.ofl_lock); - local_irq_restore(seq_flags); - rdp->cpu_started = false; } From patchwork Fri Oct 13 11:58:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 152547 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp1835041vqb; Fri, 13 Oct 2023 05:00:39 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGhWtps++tjuUUNPPeMn/ADLzs3TheIaBU90Yh7UzN//876l0VRUnsh+dqE/x62i0nkTkeP X-Received: by 2002:a05:6359:b9a:b0:14d:2dd5:a777 with SMTP id gf26-20020a0563590b9a00b0014d2dd5a777mr23096047rwb.2.1697198439070; Fri, 13 Oct 2023 05:00:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697198439; cv=none; d=google.com; s=arc-20160816; b=b7WTgyCig+0bqAJHNSCp1fVrbEc90ddaLJTUbV5vUrOFn7WYdPtHCQVfnwbY4dpICh iJqssZaNNbFoR5Fx+1/ibgai/qtzymZl5qkg9jeSYRiOI/0d29Lb3hFwAgT9nwD5ea6T mAAOh513cKB4iiaebTY0CvBlZWguhRj/psDIV/GVWsFRZ2wL7SJi8nFCDLzCzLmIK5BA zwa9kz22c9WW7QzDyllwGNbrym2+biOt0alv/zZc6NBywrcgBb0OwvdDixnZnhZYLaI8 JemDl42HjAQDDCHs5T9+HKeikDgNvneef2YMLUHxYIiOAQ5z/HKqhd67SlDq1Ooqyum5 eI+g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=PffYBtLGHbIcOZtzmyg9Rlu6xs5vVBU4yjNXSvTq6yI=; fh=lTKEK1Rt7LvnHRfa4w1cQ2RgMDufzBK6qxi+oKKQlPU=; b=MGxgxGEoG4jIT7ZPufTbQTSooz1yGat2iU4uXWZ3i+8LprJ9TsQOiUL0bH79RQ3ouU K3W2YiSyTU5msCKTAEE7pmEKQVTcMG45RgzjgOgQMz2whfEoxVtSv2imTuTNyXzUBpqe bc4Q3QVJfMqizv/wg1uNDD5S8epA2SCNk7NidULQXB9vojMdbgnuvpbKarHx4uRT5zna /JeH7Rczft1d9ujCuI9tTYKo1SSHDrxf6Cuji5xgb7SzvvXQfGIj1GoNsC36E1XCD7oM XAg5K3b7TA+odl5NIAAJqgKMrYQjRtTxwV1CwxyM1uesZWt0UCU6/JupytaWvcL0GrOU PO/g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=heTtGtSe; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from morse.vger.email (morse.vger.email. [23.128.96.31]) by mx.google.com with ESMTPS id fi1-20020a056a00398100b0068fa8499af1si17503207pfb.11.2023.10.13.05.00.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Oct 2023 05:00:39 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) client-ip=23.128.96.31; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=heTtGtSe; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by morse.vger.email (Postfix) with ESMTP id 94D2082E2962; Fri, 13 Oct 2023 05:00:36 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at morse.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231755AbjJMMA2 (ORCPT + 19 others); Fri, 13 Oct 2023 08:00:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52148 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231734AbjJMMAA (ORCPT ); Fri, 13 Oct 2023 08:00:00 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1E459128; Fri, 13 Oct 2023 04:59:46 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 25A8EC433CA; Fri, 13 Oct 2023 11:59:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1697198386; bh=FQagvd3jim1vzLZluSHTIHqE/ZMRbbrLCQrXxKWYGq0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=heTtGtSecdsKYa7W/R1Z1z0SeOVDjYpQoDiPFINWnTuMzEiwro6TdB2AJ2E1nasyv /rUdpOgREj3kz73+hwtOZVonmWj1tgEz9Ox0diafReZahnnCNQh0ioxskwXb/j96/z qsM027GELQ42gdQvvLelfGRfuw9gn4Ydk/SkRj0RisnoS6Ih8rpGZXn3TNNAk83jB5 fiaudv4Ny0QD0cyGyUTYXLI9fqJYdfjkdlyYz9RMVKf9shSq+okffjMrLgQ2WCRYKR 023ipCxZezIXwWDvAB5G06Uifv5yUR7QMxPGDy5y5B54gR1EoVwA0nJb94m9hdK18i Wlg9YIpDberpw== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Boqun Feng , Joel Fernandes , Josh Triplett , Mathieu Desnoyers , Neeraj Upadhyay , "Paul E . McKenney" , Steven Rostedt , Uladzislau Rezki , rcu Subject: [PATCH 13/18] rcu: Assume rcu_report_dead() is always called locally Date: Fri, 13 Oct 2023 13:58:57 +0200 Message-Id: <20231013115902.1059735-14-frederic@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231013115902.1059735-1-frederic@kernel.org> References: <20231013115902.1059735-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on morse.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (morse.vger.email [0.0.0.0]); Fri, 13 Oct 2023 05:00:36 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779641550328760462 X-GMAIL-MSGID: 1779641550328760462 rcu_report_dead() has to be called locally by the CPU that is going to exit the RCU state machine. Passing a cpu argument here is error-prone and leaves the possibility for a racy remote call. Use local access instead. Reviewed-by: Paul E. McKenney Signed-off-by: Frederic Weisbecker --- arch/arm64/kernel/smp.c | 2 +- include/linux/rcupdate.h | 2 +- kernel/cpu.c | 2 +- kernel/rcu/tree.c | 4 ++-- 4 files changed, 5 insertions(+), 5 deletions(-) diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index 960b98b43506..8fa646c90c67 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -401,7 +401,7 @@ void __noreturn cpu_die_early(void) /* Mark this CPU absent */ set_cpu_present(cpu, 0); - rcu_report_dead(cpu); + rcu_report_dead(); if (IS_ENABLED(CONFIG_HOTPLUG_CPU)) { update_cpu_boot_status(CPU_KILL_ME); diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h index 5e5f920ade90..aa351ddcbe8d 100644 --- a/include/linux/rcupdate.h +++ b/include/linux/rcupdate.h @@ -122,7 +122,7 @@ static inline void call_rcu_hurry(struct rcu_head *head, rcu_callback_t func) void rcu_init(void); extern int rcu_scheduler_active; void rcu_sched_clock_irq(int user); -void rcu_report_dead(unsigned int cpu); +void rcu_report_dead(void); void rcutree_migrate_callbacks(int cpu); #ifdef CONFIG_TASKS_RCU_GENERIC diff --git a/kernel/cpu.c b/kernel/cpu.c index 6de7c6bb74ee..076e75fed8bb 100644 --- a/kernel/cpu.c +++ b/kernel/cpu.c @@ -1388,7 +1388,7 @@ void cpuhp_report_idle_dead(void) struct cpuhp_cpu_state *st = this_cpu_ptr(&cpuhp_state); BUG_ON(st->state != CPUHP_AP_OFFLINE); - rcu_report_dead(smp_processor_id()); + rcu_report_dead(); st->state = CPUHP_AP_IDLE_DEAD; /* * We cannot call complete after rcu_report_dead() so we delegate it diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index 8c2954502e55..2e1e7eadf2cc 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -4560,11 +4560,11 @@ void rcu_cpu_starting(unsigned int cpu) * from the outgoing CPU rather than from the cpuhp_step mechanism. * This is because this function must be invoked at a precise location. */ -void rcu_report_dead(unsigned int cpu) +void rcu_report_dead(void) { unsigned long flags; unsigned long mask; - struct rcu_data *rdp = per_cpu_ptr(&rcu_data, cpu); + struct rcu_data *rdp = this_cpu_ptr(&rcu_data); struct rcu_node *rnp = rdp->mynode; /* Outgoing CPU's rdp & rnp. */ /* From patchwork Fri Oct 13 11:58:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 152549 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp1835217vqb; Fri, 13 Oct 2023 05:00:51 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHIuuhILvTscTBbCg1noXGrbbLc+DD9/iLk23Ui7x0aEqls2c1C89k+bp7aGNK5A4+QO91I X-Received: by 2002:a05:6a21:a588:b0:163:ab09:196d with SMTP id gd8-20020a056a21a58800b00163ab09196dmr33989621pzc.1.1697198451470; Fri, 13 Oct 2023 05:00:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697198451; cv=none; d=google.com; s=arc-20160816; b=daeIOfu0aKFyZAwBArvhCwYQBKJ/3vsFGVTqBxdrr9Xw/Q0A0HOlWWOMli4E3Pi6hI 9rOUsDa8WQv5AneStDHV/xYwEeEs0mBbVAON2RkxnH7eAc4C5Yz4ZEskZTSS2MfidNmO B2RmOMxnJmgL3m0zK1pF5s1IvVDiTtYIN1NPD0ilMQU/7V5fYLGQVl6wGNe74q55p1hP xdhwTC2i/G/dZOag2MyA1aHFyXQclbams/sFywWiX2LXPwAG9Ykj7/dwXNqJ8fuqqHG6 Cq+6+mlqllOHMlNlFQyVzup0g5ZaksBdKCKcHg/jgtSfDTcokeutuF933KIyJCV/WJKK 0cIw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=ll87iRYoDZUaTpdFHTWjEVcsYTSqxcnlY+aSfci3eLo=; fh=lTKEK1Rt7LvnHRfa4w1cQ2RgMDufzBK6qxi+oKKQlPU=; b=zU8Jurn8jFt96N2B11jDm1dYRQJdGHJhx3MrWRxhkRZ4GN6FFZmYnkXFS3fVG9snAJ /V/coGou8nF8UN1ek9ziYcysRzkR3cTOrYYix4zBRLdTLytIvwuhHS0lWwVYsr85culV CeoooAmJY+8BS8Wq3Z0ps8iVeqQEBbAVFws46Bpj3NivNRC/VVLO/Aca6luQSfiIi214 LZWpQl3NbkWimWrLtw4J+5PhgQ8m8gUwGlNBUTS5Nt2Ay6cQR3eRuWb8Zlhp7TPsmk3a OCnYcKEm5y5rLK0qhB+wWSUqPsVbF84S/L/p6dYCuAnlZz/p94JtR/bxa96FNMsX+9h5 LlHg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=S+XhprrZ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from lipwig.vger.email (lipwig.vger.email. [23.128.96.33]) by mx.google.com with ESMTPS id cb21-20020a056a02071500b005aad7d77602si1660553pgb.367.2023.10.13.05.00.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Oct 2023 05:00:51 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) client-ip=23.128.96.33; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=S+XhprrZ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id 13BCD83475C4; Fri, 13 Oct 2023 05:00:49 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231902AbjJMMAj (ORCPT + 19 others); Fri, 13 Oct 2023 08:00:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45940 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231708AbjJMMAL (ORCPT ); Fri, 13 Oct 2023 08:00:11 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C7F151707; Fri, 13 Oct 2023 04:59:49 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E05E9C433C8; Fri, 13 Oct 2023 11:59:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1697198389; bh=gBu5d6YUhfCKIA13MA7EoAu72nhKwN2C8OoerYKkJQw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=S+XhprrZuKYQhkPOGnbJV7Ei0BptNRQ+kTCnTCtt9/Qt2cgIRP/nJg8PQwfnqXsFV wHtVcbS9955Q+UReAhH2yh7NIeWCN2xIhdSFZ9saUh2//i+b49fa4KxTfe5IyniKoc PkIs7JyzhXHkK8tLBxkRoCloMCGDPiNMMtCeY9LoyEX1ft5mzp69oNmghN5mELrQt4 e1LYsvvdtMu6XhFsvWfG9zl8tsK/T9+h8Jm5l7bpaKtARXcZfmB+vaPsseGlD93KEF nq47VSSNuWPeqGiBSub3PxkKul9UHyw3xBpxQwI0SOE/IODa9HtCWtPAUbJ+lxfEke mdvcK8rP8F/jQ== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Boqun Feng , Joel Fernandes , Josh Triplett , Mathieu Desnoyers , Neeraj Upadhyay , "Paul E . McKenney" , Steven Rostedt , Uladzislau Rezki , rcu Subject: [PATCH 14/18] rcu: Conditionally build CPU-hotplug teardown callbacks Date: Fri, 13 Oct 2023 13:58:58 +0200 Message-Id: <20231013115902.1059735-15-frederic@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231013115902.1059735-1-frederic@kernel.org> References: <20231013115902.1059735-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Fri, 13 Oct 2023 05:00:49 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779641563257887266 X-GMAIL-MSGID: 1779641563257887266 Among the three CPU-hotplug teardown RCU callbacks, two of them early exit if CONFIG_HOTPLUG_CPU=n, and one is left unchanged. In any case all of them have an implementation when CONFIG_HOTPLUG_CPU=n. Align instead with the common way to deal with CPU-hotplug teardown callbacks and provide a proper stub when they are not supported. Reviewed-by: Paul E. McKenney Signed-off-by: Frederic Weisbecker --- include/linux/rcutree.h | 11 +++- kernel/rcu/tree.c | 114 +++++++++++++++++++--------------------- 2 files changed, 63 insertions(+), 62 deletions(-) diff --git a/include/linux/rcutree.h b/include/linux/rcutree.h index 153cfc7bbffd..46875c4e9f56 100644 --- a/include/linux/rcutree.h +++ b/include/linux/rcutree.h @@ -110,9 +110,16 @@ void rcu_all_qs(void); /* RCUtree hotplug events */ int rcutree_prepare_cpu(unsigned int cpu); int rcutree_online_cpu(unsigned int cpu); -int rcutree_offline_cpu(unsigned int cpu); +void rcu_cpu_starting(unsigned int cpu); + +#ifdef CONFIG_HOTPLUG_CPU int rcutree_dead_cpu(unsigned int cpu); int rcutree_dying_cpu(unsigned int cpu); -void rcu_cpu_starting(unsigned int cpu); +int rcutree_offline_cpu(unsigned int cpu); +#else +#define rcutree_dead_cpu NULL +#define rcutree_dying_cpu NULL +#define rcutree_offline_cpu NULL +#endif #endif /* __LINUX_RCUTREE_H */ diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index 2e1e7eadf2cc..f9c6b2680cbb 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -4237,25 +4237,6 @@ static bool rcu_init_invoked(void) return !!rcu_state.n_online_cpus; } -/* - * Near the end of the offline process. Trace the fact that this CPU - * is going offline. - */ -int rcutree_dying_cpu(unsigned int cpu) -{ - bool blkd; - struct rcu_data *rdp = per_cpu_ptr(&rcu_data, cpu); - struct rcu_node *rnp = rdp->mynode; - - if (!IS_ENABLED(CONFIG_HOTPLUG_CPU)) - return 0; - - blkd = !!(READ_ONCE(rnp->qsmask) & rdp->grpmask); - trace_rcu_grace_period(rcu_state.name, READ_ONCE(rnp->gp_seq), - blkd ? TPS("cpuofl-bgp") : TPS("cpuofl")); - return 0; -} - /* * All CPUs for the specified rcu_node structure have gone offline, * and all tasks that were preempted within an RCU read-side critical @@ -4301,23 +4282,6 @@ static void rcu_cleanup_dead_rnp(struct rcu_node *rnp_leaf) } } -/* - * The CPU has been completely removed, and some other CPU is reporting - * this fact from process context. Do the remainder of the cleanup. - * There can only be one CPU hotplug operation at a time, so no need for - * explicit locking. - */ -int rcutree_dead_cpu(unsigned int cpu) -{ - if (!IS_ENABLED(CONFIG_HOTPLUG_CPU)) - return 0; - - WRITE_ONCE(rcu_state.n_online_cpus, rcu_state.n_online_cpus - 1); - // Stop-machine done, so allow nohz_full to disable tick. - tick_dep_clear(TICK_DEP_BIT_RCU); - return 0; -} - /* * Propagate ->qsinitmask bits up the rcu_node tree to account for the * first CPU in a given leaf rcu_node structure coming online. The caller @@ -4470,29 +4434,6 @@ int rcutree_online_cpu(unsigned int cpu) return 0; } -/* - * Near the beginning of the process. The CPU is still very much alive - * with pretty much all services enabled. - */ -int rcutree_offline_cpu(unsigned int cpu) -{ - unsigned long flags; - struct rcu_data *rdp; - struct rcu_node *rnp; - - rdp = per_cpu_ptr(&rcu_data, cpu); - rnp = rdp->mynode; - raw_spin_lock_irqsave_rcu_node(rnp, flags); - rnp->ffmask &= ~rdp->grpmask; - raw_spin_unlock_irqrestore_rcu_node(rnp, flags); - - rcutree_affinity_setting(cpu, cpu); - - // nohz_full CPUs need the tick for stop-machine to work quickly - tick_dep_set(TICK_DEP_BIT_RCU); - return 0; -} - /* * Mark the specified CPU as being online so that subsequent grace periods * (both expedited and normal) will wait on it. Note that this means that @@ -4646,7 +4587,60 @@ void rcutree_migrate_callbacks(int cpu) cpu, rcu_segcblist_n_cbs(&rdp->cblist), rcu_segcblist_first_cb(&rdp->cblist)); } -#endif + +/* + * The CPU has been completely removed, and some other CPU is reporting + * this fact from process context. Do the remainder of the cleanup. + * There can only be one CPU hotplug operation at a time, so no need for + * explicit locking. + */ +int rcutree_dead_cpu(unsigned int cpu) +{ + WRITE_ONCE(rcu_state.n_online_cpus, rcu_state.n_online_cpus - 1); + // Stop-machine done, so allow nohz_full to disable tick. + tick_dep_clear(TICK_DEP_BIT_RCU); + return 0; +} + +/* + * Near the end of the offline process. Trace the fact that this CPU + * is going offline. + */ +int rcutree_dying_cpu(unsigned int cpu) +{ + bool blkd; + struct rcu_data *rdp = per_cpu_ptr(&rcu_data, cpu); + struct rcu_node *rnp = rdp->mynode; + + blkd = !!(READ_ONCE(rnp->qsmask) & rdp->grpmask); + trace_rcu_grace_period(rcu_state.name, READ_ONCE(rnp->gp_seq), + blkd ? TPS("cpuofl-bgp") : TPS("cpuofl")); + return 0; +} + +/* + * Near the beginning of the process. The CPU is still very much alive + * with pretty much all services enabled. + */ +int rcutree_offline_cpu(unsigned int cpu) +{ + unsigned long flags; + struct rcu_data *rdp; + struct rcu_node *rnp; + + rdp = per_cpu_ptr(&rcu_data, cpu); + rnp = rdp->mynode; + raw_spin_lock_irqsave_rcu_node(rnp, flags); + rnp->ffmask &= ~rdp->grpmask; + raw_spin_unlock_irqrestore_rcu_node(rnp, flags); + + rcutree_affinity_setting(cpu, cpu); + + // nohz_full CPUs need the tick for stop-machine to work quickly + tick_dep_set(TICK_DEP_BIT_RCU); + return 0; +} +#endif /* #ifdef CONFIG_HOTPLUG_CPU */ /* * On non-huge systems, use expedited RCU grace periods to make suspend From patchwork Fri Oct 13 11:58:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 152558 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp1836675vqb; Fri, 13 Oct 2023 05:02:20 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHCLDEJEygFQwsztTVPvbe4HG+fkX20fbN+m80x2hBtlR298Tc6sOFaG4U0zvAeV10Lav+8 X-Received: by 2002:a05:6a20:a123:b0:13f:65ca:52a2 with SMTP id q35-20020a056a20a12300b0013f65ca52a2mr29384359pzk.5.1697198519637; Fri, 13 Oct 2023 05:01:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697198519; cv=none; d=google.com; s=arc-20160816; b=DIyg21vWHK9763e/858rPl7B7CDGVs8rdZvo7d6mAQWVJQxUnYika1jdve4o8k2dCL fH+6hpeBlyOvL2Bs0/MblqfY4nl5PXhyeOMRseJ52fYQsFca3Elc0pr/KnOwY5QB52ze WlAcLRzUw3sXN66/GGXJWalcibBPiZhbBDR2ETF/TUqjJfGAPqRypZULZhQYRNjP2O7z nBYvasWliKP3F5bkDC67vKAEfgMdxXCSr7QD0+EDXdU6xJjQlXBjDN4y8ME7a9K06Jsc MJS/iO93K294GCgh++CYFVl/wiHAA4cVUFFgeaSkSjwecf1kQI2bPxpj6PdIbylpB6xQ x+bA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=D0u+xpkmAyZUwOePDyHeoQdmTgS/BQud539/RFcAtCs=; fh=lTKEK1Rt7LvnHRfa4w1cQ2RgMDufzBK6qxi+oKKQlPU=; b=kLL32vNVBRB3Yucw8BUnvvoVaiBW1akM3PhP5YJMnwseQNLVEfJCOMVhGryBU8IFmT XvaID8O1LlNFOxyFCGEAJQtotHNXmZWCoC1g1pUI0iR+/M+3YBP18AgsCcSri4/EIlMc 8jNwSxVD6BCdiruqgzu7K63idK2vdnbrdrWnyDJSKqpIfPPCwkWCs02NkXRBMAW7hyJ+ 3gV+GRPCyS89kT5MZ/lLUvA75/YYgjYbVUTCaIQ2amEHTjKFX0sW77Jp89ZegrNjoEC/ JrHGWcBK6or3EF+eNkoxrwP5ZrKnMSY818yrbM+03Fl4JmirDpWxUEdn788wpKU/Togu 9oiQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=RQIuu4BA; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id w71-20020a63824a000000b005aa6c0aa22dsi1528386pgd.724.2023.10.13.05.01.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Oct 2023 05:01:59 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=RQIuu4BA; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 92CA68047048; Fri, 13 Oct 2023 05:00:59 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231747AbjJMMA4 (ORCPT + 19 others); Fri, 13 Oct 2023 08:00:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46058 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231664AbjJMMAP (ORCPT ); Fri, 13 Oct 2023 08:00:15 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1E08E1712; Fri, 13 Oct 2023 04:59:52 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A3DCBC43391; Fri, 13 Oct 2023 11:59:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1697198392; bh=nWIP4xfrd3qkBMSKcHJBc9IkCcpsOv0EzxPd1dhN1Ek=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RQIuu4BAIwI6k+oU0wifjBngiuFyzaRnYxTFaoyx9ScfypCh1e0T3aRV4hHiPZsdt zdztgcKRU2HeLwGjH4PHDsiJypf8giuGPBkQfln1Q3/DR+7Ll8vLUXCYZWyHc9H7sz 2FgMHwoV6+4hZ6zA3Okpr347dR0vByBRIqCqy5+tSgsYKbbyqSefG+SkD0O3saXG8f p3QBAqQe6E4rhrzhBPG9coOSmsyQi48bqWqCr60TTCAscHV1w00su1T9E9E0E3pN6p 0v+aZnkte483VLx107t6mme9r0o3WaMyvX63Dc7pow2T4NvWUU3YbK41pPlHMJ05az iV1wgXYiA+3fQ== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Boqun Feng , Joel Fernandes , Josh Triplett , Mathieu Desnoyers , Neeraj Upadhyay , "Paul E . McKenney" , Steven Rostedt , Uladzislau Rezki , rcu Subject: [PATCH 15/18] rcu: Standardize explicit CPU-hotplug calls Date: Fri, 13 Oct 2023 13:58:59 +0200 Message-Id: <20231013115902.1059735-16-frederic@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231013115902.1059735-1-frederic@kernel.org> References: <20231013115902.1059735-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Fri, 13 Oct 2023 05:00:59 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779641634745535635 X-GMAIL-MSGID: 1779641634745535635 rcu_report_dead() and rcutree_migrate_callbacks() have their headers in rcupdate.h while those are pure rcutree calls, like the other CPU-hotplug functions. Also rcu_cpu_starting() and rcu_report_dead() have different naming conventions while they mirror each other's effects. Fix the headers and propose a naming that relates both functions and aligns with the prefix of other rcutree CPU-hotplug functions. Reviewed-by: Paul E. McKenney Signed-off-by: Frederic Weisbecker --- .../Expedited-Grace-Periods.rst | 2 +- .../RCU/Design/Memory-Ordering/TreeRCU-gp-fqs.svg | 4 ++-- .../RCU/Design/Memory-Ordering/TreeRCU-gp.svg | 4 ++-- .../RCU/Design/Memory-Ordering/TreeRCU-hotplug.svg | 4 ++-- .../RCU/Design/Requirements/Requirements.rst | 4 ++-- arch/arm64/kernel/smp.c | 4 ++-- arch/powerpc/kernel/smp.c | 2 +- arch/s390/kernel/smp.c | 2 +- arch/x86/kernel/smpboot.c | 2 +- include/linux/interrupt.h | 2 +- include/linux/rcupdate.h | 2 -- include/linux/rcutiny.h | 2 +- include/linux/rcutree.h | 7 ++++++- kernel/cpu.c | 6 +++--- kernel/rcu/tree.c | 12 ++++++++---- 15 files changed, 33 insertions(+), 26 deletions(-) diff --git a/Documentation/RCU/Design/Expedited-Grace-Periods/Expedited-Grace-Periods.rst b/Documentation/RCU/Design/Expedited-Grace-Periods/Expedited-Grace-Periods.rst index 93d899d53258..414f8a2012d6 100644 --- a/Documentation/RCU/Design/Expedited-Grace-Periods/Expedited-Grace-Periods.rst +++ b/Documentation/RCU/Design/Expedited-Grace-Periods/Expedited-Grace-Periods.rst @@ -181,7 +181,7 @@ operations is carried out at several levels: of this wait (or series of waits, as the case may be) is to permit a concurrent CPU-hotplug operation to complete. #. In the case of RCU-sched, one of the last acts of an outgoing CPU is - to invoke ``rcu_report_dead()``, which reports a quiescent state for + to invoke ``rcutree_report_cpu_dead()``, which reports a quiescent state for that CPU. However, this is likely paranoia-induced redundancy. +-----------------------------------------------------------------------+ diff --git a/Documentation/RCU/Design/Memory-Ordering/TreeRCU-gp-fqs.svg b/Documentation/RCU/Design/Memory-Ordering/TreeRCU-gp-fqs.svg index 7ddc094d7f28..d82a77d03d8c 100644 --- a/Documentation/RCU/Design/Memory-Ordering/TreeRCU-gp-fqs.svg +++ b/Documentation/RCU/Design/Memory-Ordering/TreeRCU-gp-fqs.svg @@ -1135,7 +1135,7 @@ font-weight="bold" font-size="192" id="text202-7-5-3-27-6-5" - style="font-size:192px;font-style:normal;font-weight:bold;text-anchor:start;fill:#000000;stroke-width:0.025in;font-family:Courier">rcu_report_dead() + style="font-size:192px;font-style:normal;font-weight:bold;text-anchor:start;fill:#000000;stroke-width:0.025in;font-family:Courier">rcutree_report_cpu_dead() rcu_cpu_starting() + xml:space="preserve">rcutree_report_cpu_starting() rcu_report_dead() + style="font-size:192px;font-style:normal;font-weight:bold;text-anchor:start;fill:#000000;stroke-width:0.025in;font-family:Courier">rcutree_report_cpu_dead() rcu_cpu_starting() + xml:space="preserve">rcutree_report_cpu_starting() rcu_report_dead() + style="font-size:192px;font-style:normal;font-weight:bold;text-anchor:start;fill:#000000;stroke-width:0.025in;font-family:Courier">rcutree_report_cpu_dead() rcu_cpu_starting() + xml:space="preserve">rcutree_report_cpu_starting() setup_cpu) diff --git a/arch/s390/kernel/smp.c b/arch/s390/kernel/smp.c index a4edb7ea66ea..214a1b67f80a 100644 --- a/arch/s390/kernel/smp.c +++ b/arch/s390/kernel/smp.c @@ -898,7 +898,7 @@ static void smp_start_secondary(void *cpuvoid) S390_lowcore.restart_flags = 0; restore_access_regs(S390_lowcore.access_regs_save_area); cpu_init(); - rcu_cpu_starting(cpu); + rcutree_report_cpu_starting(cpu); init_cpu_timer(); vtime_init(); vdso_getcpu_init(); diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c index 4e45ff44aa07..4ccb76f89af8 100644 --- a/arch/x86/kernel/smpboot.c +++ b/arch/x86/kernel/smpboot.c @@ -288,7 +288,7 @@ static void notrace start_secondary(void *unused) cpu_init(); fpu__init_cpu(); - rcu_cpu_starting(raw_smp_processor_id()); + rcutree_report_cpu_starting(raw_smp_processor_id()); x86_cpuinit.early_percpu_clock_init(); ap_starting(); diff --git a/include/linux/interrupt.h b/include/linux/interrupt.h index a92bce40b04b..d05e1e9a553c 100644 --- a/include/linux/interrupt.h +++ b/include/linux/interrupt.h @@ -566,7 +566,7 @@ enum * * _ RCU: * 1) rcutree_migrate_callbacks() migrates the queue. - * 2) rcu_report_dead() reports the final quiescent states. + * 2) rcutree_report_cpu_dead() reports the final quiescent states. * * _ IRQ_POLL: irq_poll_cpu_dead() migrates the queue */ diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h index aa351ddcbe8d..f7206b2623c9 100644 --- a/include/linux/rcupdate.h +++ b/include/linux/rcupdate.h @@ -122,8 +122,6 @@ static inline void call_rcu_hurry(struct rcu_head *head, rcu_callback_t func) void rcu_init(void); extern int rcu_scheduler_active; void rcu_sched_clock_irq(int user); -void rcu_report_dead(void); -void rcutree_migrate_callbacks(int cpu); #ifdef CONFIG_TASKS_RCU_GENERIC void rcu_init_tasks_generic(void); diff --git a/include/linux/rcutiny.h b/include/linux/rcutiny.h index 7b949292908a..d9ac7b136aea 100644 --- a/include/linux/rcutiny.h +++ b/include/linux/rcutiny.h @@ -171,6 +171,6 @@ static inline void rcu_all_qs(void) { barrier(); } #define rcutree_offline_cpu NULL #define rcutree_dead_cpu NULL #define rcutree_dying_cpu NULL -static inline void rcu_cpu_starting(unsigned int cpu) { } +static inline void rcutree_report_cpu_starting(unsigned int cpu) { } #endif /* __LINUX_RCUTINY_H */ diff --git a/include/linux/rcutree.h b/include/linux/rcutree.h index 46875c4e9f56..254244202ea9 100644 --- a/include/linux/rcutree.h +++ b/include/linux/rcutree.h @@ -110,7 +110,7 @@ void rcu_all_qs(void); /* RCUtree hotplug events */ int rcutree_prepare_cpu(unsigned int cpu); int rcutree_online_cpu(unsigned int cpu); -void rcu_cpu_starting(unsigned int cpu); +void rcutree_report_cpu_starting(unsigned int cpu); #ifdef CONFIG_HOTPLUG_CPU int rcutree_dead_cpu(unsigned int cpu); @@ -122,4 +122,9 @@ int rcutree_offline_cpu(unsigned int cpu); #define rcutree_offline_cpu NULL #endif +void rcutree_migrate_callbacks(int cpu); + +/* Called from hotplug and also arm64 early secondary boot failure */ +void rcutree_report_cpu_dead(void); + #endif /* __LINUX_RCUTREE_H */ diff --git a/kernel/cpu.c b/kernel/cpu.c index 076e75fed8bb..2491766e1fd5 100644 --- a/kernel/cpu.c +++ b/kernel/cpu.c @@ -1388,10 +1388,10 @@ void cpuhp_report_idle_dead(void) struct cpuhp_cpu_state *st = this_cpu_ptr(&cpuhp_state); BUG_ON(st->state != CPUHP_AP_OFFLINE); - rcu_report_dead(); + rcutree_report_cpu_dead(); st->state = CPUHP_AP_IDLE_DEAD; /* - * We cannot call complete after rcu_report_dead() so we delegate it + * We cannot call complete after rcutree_report_cpu_dead() so we delegate it * to an online cpu. */ smp_call_function_single(cpumask_first(cpu_online_mask), @@ -1617,7 +1617,7 @@ void notify_cpu_starting(unsigned int cpu) struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu); enum cpuhp_state target = min((int)st->target, CPUHP_AP_ONLINE); - rcu_cpu_starting(cpu); /* Enables RCU usage on this CPU. */ + rcutree_report_cpu_starting(cpu); /* Enables RCU usage on this CPU. */ cpumask_set_cpu(cpu, &cpus_booted_once_mask); /* diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index f9c6b2680cbb..36d8818eaec1 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -4216,7 +4216,7 @@ bool rcu_lockdep_current_cpu_online(void) rdp = this_cpu_ptr(&rcu_data); /* * Strictly, we care here about the case where the current CPU is - * in rcu_cpu_starting() and thus has an excuse for rdp->grpmask + * in rcutree_report_cpu_starting() and thus has an excuse for rdp->grpmask * not being up to date. So arch_spin_is_locked() might have a * false positive if it's held by some *other* CPU, but that's * OK because that just means a false *negative* on the warning. @@ -4445,8 +4445,10 @@ int rcutree_online_cpu(unsigned int cpu) * from the incoming CPU rather than from the cpuhp_step mechanism. * This is because this function must be invoked at a precise location. * This incoming CPU must not have enabled interrupts yet. + * + * This mirrors the effects of rcutree_report_cpu_dead(). */ -void rcu_cpu_starting(unsigned int cpu) +void rcutree_report_cpu_starting(unsigned int cpu) { unsigned long mask; struct rcu_data *rdp; @@ -4500,8 +4502,10 @@ void rcu_cpu_starting(unsigned int cpu) * Note that this function is special in that it is invoked directly * from the outgoing CPU rather than from the cpuhp_step mechanism. * This is because this function must be invoked at a precise location. + * + * This mirrors the effect of rcutree_report_cpu_starting(). */ -void rcu_report_dead(void) +void rcutree_report_cpu_dead(void) { unsigned long flags; unsigned long mask; @@ -5072,7 +5076,7 @@ void __init rcu_init(void) pm_notifier(rcu_pm_notify, 0); WARN_ON(num_online_cpus() > 1); // Only one CPU this early in boot. rcutree_prepare_cpu(cpu); - rcu_cpu_starting(cpu); + rcutree_report_cpu_starting(cpu); rcutree_online_cpu(cpu); /* Create workqueue for Tree SRCU and for expedited GPs. */ From patchwork Fri Oct 13 11:59:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 152551 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp1835324vqb; Fri, 13 Oct 2023 05:01:00 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEkonBEe1TYFOPYUawWzXwtZS7H5e3neOyT1muLbr0gX1s+rGtnkoDGiqKI/LjWwUigSEC4 X-Received: by 2002:a05:6359:b9a:b0:14d:2dd5:a777 with SMTP id gf26-20020a0563590b9a00b0014d2dd5a777mr23097243rwb.2.1697198460489; Fri, 13 Oct 2023 05:01:00 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697198460; cv=none; d=google.com; s=arc-20160816; b=sNCtylGypZ7GYCw/kvbFO+9hvxfesoj1GlT2QkpbPCxkS7anWk+bpMLNLDFaiZR6Yq ydVyhPUF1Gkwjtwglzvo4vkPNOfLUF6F8mqQZ8THdnKBBh38CUBV82sbDUor7FqvdSCC S2JVDXSinDa84yXz4YCyMX7vernCpfLrL9m3utoWHrOphw9aEbb9PWAyXSCe1Vj/rGvs 55yAeysJOj7eCkrpudLuzlSWQRPI7WLBTdCfXwby4w/csg3vUoGZsar2HdGVdAvciY1+ j1uy4mb2nODFjO/abBqfHsow1K6qrucr1w3gVP6fnGDTaBKApdDQnii/cJuqSVvy9iTr PtLQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=F94cAXAKPCL0o1q8MIify8cg62aD/UmtHGgtRInlSSE=; fh=lTKEK1Rt7LvnHRfa4w1cQ2RgMDufzBK6qxi+oKKQlPU=; b=Vr3X5gjQYhuEbHcqnqnL/te6R7AEDhOmKcd9Z9NQ3AzMOrA7YVSzuiH0VYgvCdCBeW 5e6fvPbqewHvFCi4cKanfHlFQ9Smhbia0atLN0qiwgdnUCYt3dkRUqaf/1IKLEeJYQb/ /QMK/VusmZAhiyepP9sXwF6m+aeWwgL6RHZ+fVuiWd001ZsLtjyV2HznbbX/rm6sj45l m6sNShKkpe8JZ3Tmfd5JdZifF/CdCNs/wJSiVa6fo+B/MN+B8r2MkDBQ3KgpXOoXEIdM MguhF46q8FtDvOLXG6KQdpDhY8ppAlTAITp5lt1eSnFcSx138tCGgJiETttf/FEVxDhU bX9Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=fWNbhv1U; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from fry.vger.email (fry.vger.email. [23.128.96.38]) by mx.google.com with ESMTPS id bv7-20020a056a00414700b00690c75e54e1si13328945pfb.402.2023.10.13.05.01.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Oct 2023 05:01:00 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) client-ip=23.128.96.38; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=fWNbhv1U; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by fry.vger.email (Postfix) with ESMTP id 701B8839FC46; Fri, 13 Oct 2023 05:00:56 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at fry.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231536AbjJMMAu (ORCPT + 19 others); Fri, 13 Oct 2023 08:00:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52480 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231390AbjJMMAP (ORCPT ); Fri, 13 Oct 2023 08:00:15 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DB7471718; Fri, 13 Oct 2023 04:59:54 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 676A5C433CD; Fri, 13 Oct 2023 11:59:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1697198394; bh=DWCRnZosrwhCtgibC90qwLkqDfB/lV91PpsutuxykGQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fWNbhv1UYD+5FMyjhsmroXXEl7ntF14NZ9IwxNXCXTG0hNaWAryF9wCiKDUblKWMq HlgU2pVpNeEqJv0fmRrCbjc2REyUxDiZXY9HMyz/fJ+qmdsvxoFR7i2u4/IMyRdgKV fYLKDiC+z5G8cDs2TVuCkVWV1XZndocU9EJ5mFuOVIBM30sRZss9yjeOPuq69SKfT+ S333nwDOFbqbuahG/VeBrus6DJaTmG+A0hyzANKLfsviFV50AMj9CLiqF0dlV1Yspa 6DQa4vEewIsK0+2A9BmMP23lnzbgPDNYodygKD2ecGbf0xZpgy0Ph7wTZ/5T6w4yGy GE1NfHOtmJyeg== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Boqun Feng , Joel Fernandes , Josh Triplett , Mathieu Desnoyers , Neeraj Upadhyay , "Paul E . McKenney" , Steven Rostedt , Uladzislau Rezki , rcu Subject: [PATCH 16/18] rcu: Comment why callbacks migration can't wait for CPUHP_RCUTREE_PREP Date: Fri, 13 Oct 2023 13:59:00 +0200 Message-Id: <20231013115902.1059735-17-frederic@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231013115902.1059735-1-frederic@kernel.org> References: <20231013115902.1059735-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on fry.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (fry.vger.email [0.0.0.0]); Fri, 13 Oct 2023 05:00:56 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779641572904565493 X-GMAIL-MSGID: 1779641572904565493 The callbacks migration is performed through an explicit call from the hotplug control CPU right after the death of the target CPU and before proceeding with the CPUHP_ teardown functions. This is unusual but necessary and yet uncommented. Summarize the reason as explained in the changelog of: a58163d8ca2c (rcu: Migrate callbacks earlier in the CPU-offline timeline) Reviewed-by: Paul E. McKenney Signed-off-by: Frederic Weisbecker --- kernel/cpu.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/kernel/cpu.c b/kernel/cpu.c index 2491766e1fd5..3b9d5c7eb4a2 100644 --- a/kernel/cpu.c +++ b/kernel/cpu.c @@ -1372,7 +1372,14 @@ static int takedown_cpu(unsigned int cpu) cpuhp_bp_sync_dead(cpu); tick_cleanup_dead_cpu(cpu); + + /* + * Callbacks must be re-integrated right away to the RCU state machine. + * Otherwise an RCU callback could block a further teardown function + * waiting for its completion. + */ rcutree_migrate_callbacks(cpu); + return 0; } From patchwork Fri Oct 13 11:59:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 152557 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp1836237vqb; Fri, 13 Oct 2023 05:01:57 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHvw1oHku+cilPyEGv6QRT+1hi2sZBdnMTbok67qod8ZPkd/6CznVeTFkPBq9oPFGht97so X-Received: by 2002:a17:90a:4ca4:b0:274:99ed:a80c with SMTP id k33-20020a17090a4ca400b0027499eda80cmr24184301pjh.3.1697198517347; Fri, 13 Oct 2023 05:01:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697198517; cv=none; d=google.com; s=arc-20160816; b=RG2CxZ/dhMtYSrPcNsGyMb3g0PPqEoywvER49LA5+9Mj7m60BBr4QwhupxiQ+sQBV0 8taZu74vlZLJaa2EBWfRkwMRi5Bg5DMAT3b8gMrLx7mOtTox4PXEvsmGSQjCKGfF1ESc GtOOaTJMqWGeuvk/R7yNc3530nQ+iFb8Ijd3SDV+/JAUpH7igvott07CwemQC3pkJIbm 94SMuRd4Pow0KE7EES6rjfRooaTYGCXCFIgMlmDJY4lK5kEfD9Xw1KB+zWfba3I+ncdN bc6X4AJtZQiYuyIYmuf+4eovgmictFYxdWoDMEoZkDwyxxtFW8IU/XGtKLEdSPYYMyTP jN5Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=8O3KTo0ejjVaoo/la7tWMWZZsegpvs29/bf6i1Jwnzw=; fh=cqLmS1lB4HyTSIyIUKIPWnKP759EcbqsE/we+cSrAlo=; b=pigND6t993bfBJmz3N+Ik/QjGIG3o7bQlpAPNpuvKJw/xkRd0/emVNw+QgJfg1bWqZ sYoLZQqhMlDXXStWybhDT9eTneurk+v9YB0t5DtK6a8FsJExx4OgZdttVCBYmq3v1Cya am+JrDOE/z4QjwP8SoJ+BUTloDNy22drcfkA/NPXPbcXG17lkArfDWQV5fZVArDizZxV Ip7jkASh7ep3qrlfW34CkLKwIFtQQney2dYiTtiwETJeUTkg7mwm8Mx86wfhin6zZ1UU VQDYTuDFNP6Gdz5KBy6QoflGB7g2ldHo12Bfg1lElCMlviSfspvqN3kVanX8i3Zvxz86 UzQQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=XAMCSnHc; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id 7-20020a17090a1a4700b0027d3ed58c04si1068414pjl.173.2023.10.13.05.01.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Oct 2023 05:01:57 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=XAMCSnHc; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 6F6FA80408E3; Fri, 13 Oct 2023 05:00:55 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231614AbjJMMAp (ORCPT + 19 others); Fri, 13 Oct 2023 08:00:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60012 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231817AbjJMMAW (ORCPT ); Fri, 13 Oct 2023 08:00:22 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0EE7812C; Fri, 13 Oct 2023 04:59:58 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2DEBBC433CC; Fri, 13 Oct 2023 11:59:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1697198397; bh=AMBt1y3lKatePr7xoPrSZmw3+JE1L37mMKzOn8P1rJM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XAMCSnHc8BABPiBf4YyY9SOO0j3o28PibPn7C8SuTA+E7kolYFPWq2KPSAVjCJHk3 iJZYIiJK/iKYAoNBdYvk0DwrXra6Y0b+C8UmlPCjKjAd3UJUQgsqD+G3GbzublBt/Y uLoTzS1WQnPWo5veQOX+ztFFPw1QPuMrSqtFaANUOPbcsZFgAHA6AvPMBOcFzcaZuz e72d3JWmxcNomWn9KUyYzczGWMJdZ6xx71EHlNgmK2kReD722gOLJ3M/d8ewqfcWMl 1YoHJGzPDYVR4yf+qwlyFL+V+Lz+BSYCamYS8hBjBtp/IAehYSTUJ1vHO7x+jhwgd3 U4SrcqdceMgFg== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Boqun Feng , Joel Fernandes , Josh Triplett , Mathieu Desnoyers , Neeraj Upadhyay , "Paul E . McKenney" , Steven Rostedt , Uladzislau Rezki , rcu , Yong He , Neeraj upadhyay Subject: [PATCH 17/18] srcu: Fix callbacks acceleration mishandling Date: Fri, 13 Oct 2023 13:59:01 +0200 Message-Id: <20231013115902.1059735-18-frederic@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231013115902.1059735-1-frederic@kernel.org> References: <20231013115902.1059735-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Fri, 13 Oct 2023 05:00:55 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779641632608963743 X-GMAIL-MSGID: 1779641632608963743 SRCU callbacks acceleration might fail if the preceding callbacks advance also fails. This can happen when the following steps are met: 1) The RCU_WAIT_TAIL segment has callbacks (say for gp_num 8) and the RCU_NEXT_READY_TAIL also has callbacks (say for gp_num 12). 2) The grace period for RCU_WAIT_TAIL is observed as started but not yet completed so rcu_seq_current() returns 4 + SRCU_STATE_SCAN1 = 5. 3) This value is passed to rcu_segcblist_advance() which can't move any segment forward and fails. 4) srcu_gp_start_if_needed() still proceeds with callback acceleration. But then the call to rcu_seq_snap() observes the grace period for the RCU_WAIT_TAIL segment (gp_num 8) as completed and the subsequent one for the RCU_NEXT_READY_TAIL segment as started (ie: 8 + SRCU_STATE_SCAN1 = 9) so it returns a snapshot of the next grace period, which is 16. 5) The value of 16 is passed to rcu_segcblist_accelerate() but the freshly enqueued callback in RCU_NEXT_TAIL can't move to RCU_NEXT_READY_TAIL which already has callbacks for a previous grace period (gp_num = 12). So acceleration fails. 6) Note in all these steps, srcu_invoke_callbacks() hadn't had a chance to run srcu_invoke_callbacks(). Then some very bad outcome may happen if the following happens: 7) Some other CPU races and starts the grace period number 16 before the CPU handling previous steps had a chance. Therefore srcu_gp_start() isn't called on the latter sdp to fix the acceleration leak from previous steps with a new pair of call to advance/accelerate. 8) The grace period 16 completes and srcu_invoke_callbacks() is finally called. All the callbacks from previous grace periods (8 and 12) are correctly advanced and executed but callbacks in RCU_NEXT_READY_TAIL still remain. Then rcu_segcblist_accelerate() is called with a snaphot of 20. 9) Since nothing started the grace period number 20, callbacks stay unhandled. This has been reported in real load: [3144162.608392] INFO: task kworker/136:12:252684 blocked for more than 122 seconds. [3144162.615986] Tainted: G O K 5.4.203-1-tlinux4-0011.1 #1 [3144162.623053] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [3144162.631162] kworker/136:12 D 0 252684 2 0x90004000 [3144162.631189] Workqueue: kvm-irqfd-cleanup irqfd_shutdown [kvm] [3144162.631192] Call Trace: [3144162.631202] __schedule+0x2ee/0x660 [3144162.631206] schedule+0x33/0xa0 [3144162.631209] schedule_timeout+0x1c4/0x340 [3144162.631214] ? update_load_avg+0x82/0x660 [3144162.631217] ? raw_spin_rq_lock_nested+0x1f/0x30 [3144162.631218] wait_for_completion+0x119/0x180 [3144162.631220] ? wake_up_q+0x80/0x80 [3144162.631224] __synchronize_srcu.part.19+0x81/0xb0 [3144162.631226] ? __bpf_trace_rcu_utilization+0x10/0x10 [3144162.631227] synchronize_srcu+0x5f/0xc0 [3144162.631236] irqfd_shutdown+0x3c/0xb0 [kvm] [3144162.631239] ? __schedule+0x2f6/0x660 [3144162.631243] process_one_work+0x19a/0x3a0 [3144162.631244] worker_thread+0x37/0x3a0 [3144162.631247] kthread+0x117/0x140 [3144162.631247] ? process_one_work+0x3a0/0x3a0 [3144162.631248] ? __kthread_cancel_work+0x40/0x40 [3144162.631250] ret_from_fork+0x1f/0x30 Fix this with taking the snapshot for acceleration _before_ the read of the current grace period number. The only side effect of this solution is that callbacks advancing happen then _after_ the full barrier in rcu_seq_snap(). This is not a problem because that barrier only cares about: 1) Ordering accesses of the update side before call_srcu() so they don't bleed. 2) See all the accesses prior to the grace period of the current gp_num The only things callbacks advancing need to be ordered against are carried by snp locking. Reported-by: Yong He Co-developed-by:: Yong He Signed-off-by: Yong He Co-developed-by: Joel Fernandes (Google) Signed-off-by: Joel Fernandes (Google) Co-developed-by: Neeraj upadhyay Signed-off-by: Neeraj upadhyay Link: http://lore.kernel.org/CANZk6aR+CqZaqmMWrC2eRRPY12qAZnDZLwLnHZbNi=xXMB401g@mail.gmail.com Fixes: da915ad5cf25 ("srcu: Parallelize callback handling") Signed-off-by: Frederic Weisbecker --- kernel/rcu/srcutree.c | 33 ++++++++++++++++++++++++++++++--- 1 file changed, 30 insertions(+), 3 deletions(-) diff --git a/kernel/rcu/srcutree.c b/kernel/rcu/srcutree.c index 5602042856b1..9fab9ac36996 100644 --- a/kernel/rcu/srcutree.c +++ b/kernel/rcu/srcutree.c @@ -1244,10 +1244,37 @@ static unsigned long srcu_gp_start_if_needed(struct srcu_struct *ssp, spin_lock_irqsave_sdp_contention(sdp, &flags); if (rhp) rcu_segcblist_enqueue(&sdp->srcu_cblist, rhp); - rcu_segcblist_advance(&sdp->srcu_cblist, - rcu_seq_current(&ssp->srcu_sup->srcu_gp_seq)); + /* + * The snapshot for acceleration must be taken _before_ the read of the + * current gp sequence used for advancing, otherwise advancing may fail + * and acceleration may then fail too. + * + * This could happen if: + * + * 1) The RCU_WAIT_TAIL segment has callbacks (gp_num = X + 4) and the + * RCU_NEXT_READY_TAIL also has callbacks (gp_num = X + 8). + * + * 2) The grace period for RCU_WAIT_TAIL is seen as started but not + * completed so rcu_seq_current() returns X + SRCU_STATE_SCAN1. + * + * 3) This value is passed to rcu_segcblist_advance() which can't move + * any segment forward and fails. + * + * 4) srcu_gp_start_if_needed() still proceeds with callback acceleration. + * But then the call to rcu_seq_snap() observes the grace period for the + * RCU_WAIT_TAIL segment as completed and the subsequent one for the + * RCU_NEXT_READY_TAIL segment as started (ie: X + 4 + SRCU_STATE_SCAN1) + * so it returns a snapshot of the next grace period, which is X + 12. + * + * 5) The value of X + 12 is passed to rcu_segcblist_accelerate() but the + * freshly enqueued callback in RCU_NEXT_TAIL can't move to + * RCU_NEXT_READY_TAIL which already has callbacks for a previous grace + * period (gp_num = X + 8). So acceleration fails. + */ s = rcu_seq_snap(&ssp->srcu_sup->srcu_gp_seq); - (void)rcu_segcblist_accelerate(&sdp->srcu_cblist, s); + rcu_segcblist_advance(&sdp->srcu_cblist, + rcu_seq_current(&ssp->srcu_sup->srcu_gp_seq)); + WARN_ON_ONCE(!rcu_segcblist_accelerate(&sdp->srcu_cblist, s) && rhp); if (ULONG_CMP_LT(sdp->srcu_gp_seq_needed, s)) { sdp->srcu_gp_seq_needed = s; needgp = true; From patchwork Fri Oct 13 11:59:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 152554 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp1835922vqb; Fri, 13 Oct 2023 05:01:37 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGpYB+2SirnXk9hAYl3x0d5CgLUNbHO9ULQI4OIVWjnJCPsF5n6e+rcJITN4gUXiaDlS1sW X-Received: by 2002:a17:903:2306:b0:1bb:ac37:384b with SMTP id d6-20020a170903230600b001bbac37384bmr29673716plh.6.1697198496773; Fri, 13 Oct 2023 05:01:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697198496; cv=none; d=google.com; s=arc-20160816; b=mpHqQcnqklR6JLZClD7hnaeDxb3Sdi/z5yvwwDPW0FHNgWgGtyYJ+Yu8Ydl+dKx3GD z7jpa6SmlpQg7Z7gIlLgusRRbarypwZxI3vHfQ9y/wUTq2eYR/vX53sFyYGNOGAMbPDT 8FHCk0IiWTck5YjmTi1EnWMgB+kVId1GTeOf8SCiCskiUIt2DTFSMVySQb7Np7BGdffH mo7tKdbCOtuxY9dFJ9grji2qrcXXvjma1CrwGuNX0ZgC7vs5tZ3pNeIKloa6XYdEyUXF zvQaqXGerBxcQY1aIPdap+PBo8yrz7DMShYvAcZO4j40x7zrMKQaulRVOs9CRCy0ht4C CJ6g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Go8KVyvi8+XQBI8lmhtsZHSbXNT/mCoM+GhsjazhyVI=; fh=j0SNKtIxfMsoBduhmQ8Q+hK1xpo88lXlsY8WphTTacE=; b=CMxWGY2vLI0I13z9PUUqEOkl1OX5GRNwCC5henSo4iD4GGEc4mdJTjmdooBUmXy1Xl 7YQ28HJOAg/isuIMr4XyfoMv4s4kLYj/fr9Gqt6ayl8R7xhUCwB9cTtxw4/HzTSXm6Ja 3v52QlraQcfFlThCkDmFIKrJQl8DqWaC05F6g8iWGD+ZfPZo/bltoFNSRblho1relOzv 3IqmujTK6hWXPKjaGN9jQ8j54hIitw50VvwTCNhJOhd97Y3xLRaJNRast7P0wBVkH+nL +TKE6ONsQ6OtxEDoxktJaZiHUog7ksfhJI02Gl/Q0rjxrnsrc8uWqauYtzMcphO/WJ3F p/EQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=sMyYYtKJ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from fry.vger.email (fry.vger.email. [2620:137:e000::3:8]) by mx.google.com with ESMTPS id y13-20020a17090322cd00b001bdd35033efsi4739417plg.374.2023.10.13.05.01.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Oct 2023 05:01:36 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) client-ip=2620:137:e000::3:8; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=sMyYYtKJ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by fry.vger.email (Postfix) with ESMTP id A915483B3622; Fri, 13 Oct 2023 05:01:34 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at fry.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231486AbjJMMBJ (ORCPT + 19 others); Fri, 13 Oct 2023 08:01:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52746 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231750AbjJMMAZ (ORCPT ); Fri, 13 Oct 2023 08:00:25 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 95EA1D58; Fri, 13 Oct 2023 05:00:01 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4B3C4C433C9; Fri, 13 Oct 2023 11:59:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1697198401; bh=OrFL+Yftu/dBFajCkxlvH2Gier1tRKWYsuU8KyLe3dM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sMyYYtKJ6dCQrEqAGWWgpSh3oU9RJ/2NrbUph/U7CVV4Co0COOAd1YUHMmbT5JEZ3 nxDYpp9JYWLYbOYzTbAojl6BucGZyU3Qnx7Zx9hoqFwFptPvg0cvS/RkD0iSde8Udc U2MfXIhJIh2cQIUKhsk/vxkPDjpw1vJ7a3JhnVD0YwtiCj9Y8Tf4U7vsvc5XFOA2W0 tteZjbF+Rp9tjAuZ2SwGGlKFd6+H5DKYLzmbV/NqAdvZLqcqSvEQang4eG65pLZhlg vbbqemSvesQV+NBPydDED1XVc2DItoCUa++0cnaf602gc2880KXrGwqBt9+l/t3mGM TNXTiELmPnKxA== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Boqun Feng , Joel Fernandes , Josh Triplett , Mathieu Desnoyers , Neeraj Upadhyay , "Paul E . McKenney" , Steven Rostedt , Uladzislau Rezki , rcu , Yong He , Neeraj upadhyay , Like Xu Subject: [PATCH 18/18] srcu: Only accelerate on enqueue time Date: Fri, 13 Oct 2023 13:59:02 +0200 Message-Id: <20231013115902.1059735-19-frederic@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231013115902.1059735-1-frederic@kernel.org> References: <20231013115902.1059735-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on fry.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (fry.vger.email [0.0.0.0]); Fri, 13 Oct 2023 05:01:34 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779641610740427271 X-GMAIL-MSGID: 1779641610740427271 Acceleration in SRCU happens on enqueue time for each new callback. This operation is expected not to fail and therefore any similar attempt from other places shouldn't find any remaining callbacks to accelerate. Moreover accelerations performed beyond enqueue time are error prone because rcu_seq_snap() then may return the snapshot for a new grace period that is not going to be started. Remove these dangerous and needless accelerations and introduce instead assertions reporting leaking unaccelerated callbacks beyond enqueue time. Co-developed-by: Yong He Signed-off-by: Yong He Co-developed-by: Joel Fernandes (Google) Signed-off-by: Joel Fernandes (Google) Co-developed-by: Neeraj upadhyay Signed-off-by: Neeraj upadhyay Reviewed-by: Like Xu Signed-off-by: Frederic Weisbecker --- kernel/rcu/srcutree.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/kernel/rcu/srcutree.c b/kernel/rcu/srcutree.c index 9fab9ac36996..560e99ec5333 100644 --- a/kernel/rcu/srcutree.c +++ b/kernel/rcu/srcutree.c @@ -784,8 +784,7 @@ static void srcu_gp_start(struct srcu_struct *ssp) spin_lock_rcu_node(sdp); /* Interrupts already disabled. */ rcu_segcblist_advance(&sdp->srcu_cblist, rcu_seq_current(&ssp->srcu_sup->srcu_gp_seq)); - (void)rcu_segcblist_accelerate(&sdp->srcu_cblist, - rcu_seq_snap(&ssp->srcu_sup->srcu_gp_seq)); + WARN_ON_ONCE(!rcu_segcblist_segempty(&sdp->srcu_cblist, RCU_NEXT_TAIL)); spin_unlock_rcu_node(sdp); /* Interrupts remain disabled. */ WRITE_ONCE(ssp->srcu_sup->srcu_gp_start, jiffies); WRITE_ONCE(ssp->srcu_sup->srcu_n_exp_nodelay, 0); @@ -1721,6 +1720,7 @@ static void srcu_invoke_callbacks(struct work_struct *work) ssp = sdp->ssp; rcu_cblist_init(&ready_cbs); spin_lock_irq_rcu_node(sdp); + WARN_ON_ONCE(!rcu_segcblist_segempty(&sdp->srcu_cblist, RCU_NEXT_TAIL)); rcu_segcblist_advance(&sdp->srcu_cblist, rcu_seq_current(&ssp->srcu_sup->srcu_gp_seq)); if (sdp->srcu_cblist_invoking || @@ -1750,8 +1750,6 @@ static void srcu_invoke_callbacks(struct work_struct *work) */ spin_lock_irq_rcu_node(sdp); rcu_segcblist_add_len(&sdp->srcu_cblist, -len); - (void)rcu_segcblist_accelerate(&sdp->srcu_cblist, - rcu_seq_snap(&ssp->srcu_sup->srcu_gp_seq)); sdp->srcu_cblist_invoking = false; more = rcu_segcblist_ready_cbs(&sdp->srcu_cblist); spin_unlock_irq_rcu_node(sdp);