From patchwork Mon Jul 31 23:11:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 128907 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:918b:0:b0:3e4:2afc:c1 with SMTP id s11csp2333825vqg; Mon, 31 Jul 2023 16:49:08 -0700 (PDT) X-Google-Smtp-Source: APBJJlHlng8dk75ZOMTMeuETaQcLjzn3X7IIR2VptigvdT1C5coXvvScIxnidSLMlULeI96Q+YIv X-Received: by 2002:a17:906:51d0:b0:99b:ce6d:fc90 with SMTP id v16-20020a17090651d000b0099bce6dfc90mr1530688ejk.2.1690847347749; Mon, 31 Jul 2023 16:49:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690847347; cv=none; d=google.com; s=arc-20160816; b=PABesbNh/HWuOuGFmabNUSmFHI5X62yMO29ptYiMoUWLyVEpZRIxUViwXkcTXC+UwR gmgVg+TvIwIHLiuI18x7hufejVR21B9WibVyHUIGv70mviYnfSBarVn9G0rWt4f6TuWD RsYSc8LzT9dSCPXZ9yYTkKRjXPgBwaq5p7qxqB21jTKi71GuDraRJzNv0ng1QjDdSo9Y yKJTPqFwjKp3boQgnkWRNEmoTUQWYt1uZ0amdc6MAiNb41kwdKHL9pE6reVa5MnF5waP NHgx9XkZOy2J7Dv5GbQgzSTsSIW+sAfm3rddMmSaTG0/eY/QVf7U9/zat/CrVMeM6fvR uAhw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:in-reply-to:content-disposition:mime-version :references:reply-to:message-id:subject:cc:to:from:date :dkim-signature; bh=384ehfmlLJYPnDdy2MbrPbFxfsE7GA4V7hDXLav6RMg=; fh=4Ax6M8yfdib5s4Mo8hkymO5N/GzErc/2IqDC41w0gCs=; b=xYfRh5JKckQuATFlocT485u7A1jMJH2TCUXlJRYsKCMaEdlbWjQAhmtgxA1hdiD823 1jMFLCpYJngRTxAnYMRR6ooGr/nYxc+iMg7uxPv2IwOkOclXlSVrmJxmy3w1o5/MX99O 03X36n/qf/rhBmYpivl+h5flcPhfEyyD4W6akxfwCz85dbCDBRMNfVDfep7EEVpQQNmp ZSrYGD8PWuMVrH8UWBF1x1gE1kvFmkqcwmtDLQEPinEZjl61nyDIAGO6hTcNQaOcbPtl I7XHmVsA3s0ocI8PW7UO0PZEsfVEsaM8G96CsaPeMIgyqMLBfLaaS2f4Z4AoL6iuElbO 6SKg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=QT7qFPwb; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id kj27-20020a170907765b00b00994b9c51cafsi3639776ejc.717.2023.07.31.16.48.44; Mon, 31 Jul 2023 16:49:07 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=QT7qFPwb; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230008AbjGaXLz (ORCPT + 99 others); Mon, 31 Jul 2023 19:11:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36820 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229622AbjGaXLy (ORCPT ); Mon, 31 Jul 2023 19:11:54 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6E2E51A5; Mon, 31 Jul 2023 16:11:51 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id F18986134F; Mon, 31 Jul 2023 23:11:50 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5D1E1C433C7; Mon, 31 Jul 2023 23:11:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1690845110; bh=QLAjE/GGSiZipKUex73TsazKgLlG2VFV8t+GpNpJUrs=; h=Date:From:To:Cc:Subject:Reply-To:References:In-Reply-To:From; b=QT7qFPwbt++C6pX1FEft72MhAT12SkEdw5f7xZYXniCoFrQYe82clCPDDxdSGX3I0 Vv0CPidxzYvsT9AI4yTlxnLZwUaMyqBeSnhEr1+ZAXsEm2wTSx95f6uL4LdKhkvvak ZQe6DKpsaDCUBaiMPu0cA0wV38sPhtDHQ4zycelDCBsXdIvInxYRvb2yCXA1wMCMtN CDf/GFjm7G7ObpadRPHVSc/f2EWuVCq9dSSuGNoUpol7138IlGOmdiITt/jpnOm330 ucSpVbKBv8QSpSIzy0/04p0kuscGRFZVerowhDFwENPkfzH+VxtQtcDXjpNSrisVhP UFP7H51bAB5kA== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id EE140CE1065; Mon, 31 Jul 2023 16:11:49 -0700 (PDT) Date: Mon, 31 Jul 2023 16:11:49 -0700 From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org Subject: [PATCH rcu 6/5] rcu-tasks: Permit use of debug-objects with RCU Tasks flavors Message-ID: <0bb374e7-694e-4b2d-b2aa-472a5a4e1670@paulmck-laptop> Reply-To: paulmck@kernel.org References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1772981948679501028 X-GMAIL-MSGID: 1772981948679501028 rcu-tasks: Permit use of debug-objects with RCU Tasks flavors Currently, cblist_init_generic() holds a raw spinlock when invoking INIT_WORK(). This fails in kernels built with CONFIG_DEBUG_OBJECTS=y due to memory allocation being forbidden while holding a raw spinlock. But the only reason for holding the raw spinlock is to synchronize with early boot calls to call_rcu_tasks(), call_rcu_tasks_rude, and, last but not least, call_rcu_tasks_trace(). These calls also invoke cblist_init_generic() in order to support early boot queueing of callbacks. Except that there are no early boot calls to either of these three functions, and the BPF guys confirm that they have no plans to add any such calls. This commit therefore removes the synchronization and adds a WARN_ON_ONCE() to catch the case of now-prohibited early boot RCU Tasks callback queueing. If early boot queueing is needed, an "initialized" flag may be added to the rcu_tasks structure. Then queueing a callback before this flag is set would initialize the callback list (if needed) and queue the callback. The decision as to where to queue the callback given the possibility of non-zero boot CPUs is left as an exercise for the reader. Reported-by: Jakub Kicinski Signed-off-by: Paul E. McKenney diff --git a/kernel/rcu/tasks.h b/kernel/rcu/tasks.h index 291d536b1789..a4cd4ea2402c 100644 --- a/kernel/rcu/tasks.h +++ b/kernel/rcu/tasks.h @@ -236,7 +236,7 @@ static const char *tasks_gp_state_getname(struct rcu_tasks *rtp) #endif /* #ifndef CONFIG_TINY_RCU */ // Initialize per-CPU callback lists for the specified flavor of -// Tasks RCU. +// Tasks RCU. Do not enqueue callbacks before this function is invoked. static void cblist_init_generic(struct rcu_tasks *rtp) { int cpu; @@ -244,7 +244,6 @@ static void cblist_init_generic(struct rcu_tasks *rtp) int lim; int shift; - raw_spin_lock_irqsave(&rtp->cbs_gbl_lock, flags); if (rcu_task_enqueue_lim < 0) { rcu_task_enqueue_lim = 1; rcu_task_cb_adjust = true; @@ -267,17 +266,16 @@ static void cblist_init_generic(struct rcu_tasks *rtp) WARN_ON_ONCE(!rtpcp); if (cpu) raw_spin_lock_init(&ACCESS_PRIVATE(rtpcp, lock)); - raw_spin_lock_rcu_node(rtpcp); // irqs already disabled. + local_irq_save(flags); // serialize initialization if (rcu_segcblist_empty(&rtpcp->cblist)) rcu_segcblist_init(&rtpcp->cblist); + local_irq_restore(flags); INIT_WORK(&rtpcp->rtp_work, rcu_tasks_invoke_cbs_wq); rtpcp->cpu = cpu; rtpcp->rtpp = rtp; if (!rtpcp->rtp_blkd_tasks.next) INIT_LIST_HEAD(&rtpcp->rtp_blkd_tasks); - raw_spin_unlock_rcu_node(rtpcp); // irqs remain disabled. } - raw_spin_unlock_irqrestore(&rtp->cbs_gbl_lock, flags); pr_info("%s: Setting shift to %d and lim to %d rcu_task_cb_adjust=%d.\n", rtp->name, data_race(rtp->percpu_enqueue_shift), data_race(rtp->percpu_enqueue_lim), rcu_task_cb_adjust); @@ -351,11 +349,9 @@ static void call_rcu_tasks_generic(struct rcu_head *rhp, rcu_callback_t func, READ_ONCE(rtp->percpu_enqueue_lim) != nr_cpu_ids) needadjust = true; // Defer adjustment to avoid deadlock. } - if (!rcu_segcblist_is_enabled(&rtpcp->cblist)) { - raw_spin_unlock_rcu_node(rtpcp); // irqs remain disabled. - cblist_init_generic(rtp); - raw_spin_lock_rcu_node(rtpcp); // irqs already disabled. - } + // Queuing callbacks before initialization not yet supported. + if (WARN_ON_ONCE(!rcu_segcblist_is_enabled(&rtpcp->cblist))) + rcu_segcblist_init(&rtpcp->cblist); needwake = (func == wakeme_after_rcu) || (rcu_segcblist_n_cbs(&rtpcp->cblist) == rcu_task_lazy_lim); if (havekthread && !needwake && !timer_pending(&rtpcp->lazy_timer)) {