From patchwork Fri Oct 13 11:59:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 152554 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp1835922vqb; Fri, 13 Oct 2023 05:01:37 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGpYB+2SirnXk9hAYl3x0d5CgLUNbHO9ULQI4OIVWjnJCPsF5n6e+rcJITN4gUXiaDlS1sW X-Received: by 2002:a17:903:2306:b0:1bb:ac37:384b with SMTP id d6-20020a170903230600b001bbac37384bmr29673716plh.6.1697198496773; Fri, 13 Oct 2023 05:01:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697198496; cv=none; d=google.com; s=arc-20160816; b=mpHqQcnqklR6JLZClD7hnaeDxb3Sdi/z5yvwwDPW0FHNgWgGtyYJ+Yu8Ydl+dKx3GD z7jpa6SmlpQg7Z7gIlLgusRRbarypwZxI3vHfQ9y/wUTq2eYR/vX53sFyYGNOGAMbPDT 8FHCk0IiWTck5YjmTi1EnWMgB+kVId1GTeOf8SCiCskiUIt2DTFSMVySQb7Np7BGdffH mo7tKdbCOtuxY9dFJ9grji2qrcXXvjma1CrwGuNX0ZgC7vs5tZ3pNeIKloa6XYdEyUXF zvQaqXGerBxcQY1aIPdap+PBo8yrz7DMShYvAcZO4j40x7zrMKQaulRVOs9CRCy0ht4C CJ6g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Go8KVyvi8+XQBI8lmhtsZHSbXNT/mCoM+GhsjazhyVI=; fh=j0SNKtIxfMsoBduhmQ8Q+hK1xpo88lXlsY8WphTTacE=; b=CMxWGY2vLI0I13z9PUUqEOkl1OX5GRNwCC5henSo4iD4GGEc4mdJTjmdooBUmXy1Xl 7YQ28HJOAg/isuIMr4XyfoMv4s4kLYj/fr9Gqt6ayl8R7xhUCwB9cTtxw4/HzTSXm6Ja 3v52QlraQcfFlThCkDmFIKrJQl8DqWaC05F6g8iWGD+ZfPZo/bltoFNSRblho1relOzv 3IqmujTK6hWXPKjaGN9jQ8j54hIitw50VvwTCNhJOhd97Y3xLRaJNRast7P0wBVkH+nL +TKE6ONsQ6OtxEDoxktJaZiHUog7ksfhJI02Gl/Q0rjxrnsrc8uWqauYtzMcphO/WJ3F p/EQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=sMyYYtKJ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from fry.vger.email (fry.vger.email. [2620:137:e000::3:8]) by mx.google.com with ESMTPS id y13-20020a17090322cd00b001bdd35033efsi4739417plg.374.2023.10.13.05.01.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Oct 2023 05:01:36 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) client-ip=2620:137:e000::3:8; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=sMyYYtKJ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by fry.vger.email (Postfix) with ESMTP id A915483B3622; Fri, 13 Oct 2023 05:01:34 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at fry.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231486AbjJMMBJ (ORCPT + 19 others); Fri, 13 Oct 2023 08:01:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52746 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231750AbjJMMAZ (ORCPT ); Fri, 13 Oct 2023 08:00:25 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 95EA1D58; Fri, 13 Oct 2023 05:00:01 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4B3C4C433C9; Fri, 13 Oct 2023 11:59:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1697198401; bh=OrFL+Yftu/dBFajCkxlvH2Gier1tRKWYsuU8KyLe3dM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sMyYYtKJ6dCQrEqAGWWgpSh3oU9RJ/2NrbUph/U7CVV4Co0COOAd1YUHMmbT5JEZ3 nxDYpp9JYWLYbOYzTbAojl6BucGZyU3Qnx7Zx9hoqFwFptPvg0cvS/RkD0iSde8Udc U2MfXIhJIh2cQIUKhsk/vxkPDjpw1vJ7a3JhnVD0YwtiCj9Y8Tf4U7vsvc5XFOA2W0 tteZjbF+Rp9tjAuZ2SwGGlKFd6+H5DKYLzmbV/NqAdvZLqcqSvEQang4eG65pLZhlg vbbqemSvesQV+NBPydDED1XVc2DItoCUa++0cnaf602gc2880KXrGwqBt9+l/t3mGM TNXTiELmPnKxA== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Boqun Feng , Joel Fernandes , Josh Triplett , Mathieu Desnoyers , Neeraj Upadhyay , "Paul E . McKenney" , Steven Rostedt , Uladzislau Rezki , rcu , Yong He , Neeraj upadhyay , Like Xu Subject: [PATCH 18/18] srcu: Only accelerate on enqueue time Date: Fri, 13 Oct 2023 13:59:02 +0200 Message-Id: <20231013115902.1059735-19-frederic@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231013115902.1059735-1-frederic@kernel.org> References: <20231013115902.1059735-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on fry.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (fry.vger.email [0.0.0.0]); Fri, 13 Oct 2023 05:01:34 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779641610740427271 X-GMAIL-MSGID: 1779641610740427271 Acceleration in SRCU happens on enqueue time for each new callback. This operation is expected not to fail and therefore any similar attempt from other places shouldn't find any remaining callbacks to accelerate. Moreover accelerations performed beyond enqueue time are error prone because rcu_seq_snap() then may return the snapshot for a new grace period that is not going to be started. Remove these dangerous and needless accelerations and introduce instead assertions reporting leaking unaccelerated callbacks beyond enqueue time. Co-developed-by: Yong He Signed-off-by: Yong He Co-developed-by: Joel Fernandes (Google) Signed-off-by: Joel Fernandes (Google) Co-developed-by: Neeraj upadhyay Signed-off-by: Neeraj upadhyay Reviewed-by: Like Xu Signed-off-by: Frederic Weisbecker --- kernel/rcu/srcutree.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/kernel/rcu/srcutree.c b/kernel/rcu/srcutree.c index 9fab9ac36996..560e99ec5333 100644 --- a/kernel/rcu/srcutree.c +++ b/kernel/rcu/srcutree.c @@ -784,8 +784,7 @@ static void srcu_gp_start(struct srcu_struct *ssp) spin_lock_rcu_node(sdp); /* Interrupts already disabled. */ rcu_segcblist_advance(&sdp->srcu_cblist, rcu_seq_current(&ssp->srcu_sup->srcu_gp_seq)); - (void)rcu_segcblist_accelerate(&sdp->srcu_cblist, - rcu_seq_snap(&ssp->srcu_sup->srcu_gp_seq)); + WARN_ON_ONCE(!rcu_segcblist_segempty(&sdp->srcu_cblist, RCU_NEXT_TAIL)); spin_unlock_rcu_node(sdp); /* Interrupts remain disabled. */ WRITE_ONCE(ssp->srcu_sup->srcu_gp_start, jiffies); WRITE_ONCE(ssp->srcu_sup->srcu_n_exp_nodelay, 0); @@ -1721,6 +1720,7 @@ static void srcu_invoke_callbacks(struct work_struct *work) ssp = sdp->ssp; rcu_cblist_init(&ready_cbs); spin_lock_irq_rcu_node(sdp); + WARN_ON_ONCE(!rcu_segcblist_segempty(&sdp->srcu_cblist, RCU_NEXT_TAIL)); rcu_segcblist_advance(&sdp->srcu_cblist, rcu_seq_current(&ssp->srcu_sup->srcu_gp_seq)); if (sdp->srcu_cblist_invoking || @@ -1750,8 +1750,6 @@ static void srcu_invoke_callbacks(struct work_struct *work) */ spin_lock_irq_rcu_node(sdp); rcu_segcblist_add_len(&sdp->srcu_cblist, -len); - (void)rcu_segcblist_accelerate(&sdp->srcu_cblist, - rcu_seq_snap(&ssp->srcu_sup->srcu_gp_seq)); sdp->srcu_cblist_invoking = false; more = rcu_segcblist_ready_cbs(&sdp->srcu_cblist); spin_unlock_irq_rcu_node(sdp);