From patchwork Tue Jul 25 23:29:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joel Fernandes X-Patchwork-Id: 125994 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:a985:0:b0:3e4:2afc:c1 with SMTP id t5csp82561vqo; Tue, 25 Jul 2023 17:05:32 -0700 (PDT) X-Google-Smtp-Source: APBJJlHr/eSqLOti8OwPKg5dJFmb5cKsXy7odjxcF3QlmRP2qcJKPtVzhf/0sZeJljLuTSccX4a/ X-Received: by 2002:a17:906:cc14:b0:98d:76f8:217b with SMTP id ml20-20020a170906cc1400b0098d76f8217bmr257600ejb.73.1690329932018; Tue, 25 Jul 2023 17:05:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690329932; cv=none; d=google.com; s=arc-20160816; b=jwdo9bQJBqURaInIAG4znp0OVxIZPGoCZLiBQNieh2aGoY04cIfLuN2Ey1mdZ25w9z J6zu+h1+lZRrP6Ea2Ea1+O9hI9NbaZgExDZInOEpKq2TASyx5WHdL7gMTOHN7juVR/mm SiGLr3kGgX2I0Hcvj5H5+RP3kdp8CMB7pYS7QNFbq3tIII19hqe7nqWrK+W6zumUzWN9 ZUID+7u63eBgp0np5MTpHyRBb/Fm7vfe5EoH8P5EfjYTe9vnX7A0rBd/Wt0RMusbLqXL R1CSn6q+rLHaIbaIOOc4xpxtEbV+OgnJwAcx6ogMwNMZFiDTq8ji9cUO7rx3podD8lRc lHeA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=SjqXIuVtq0u8YshojVwovW+pCVt0mgB/LilQjFvvm4k=; fh=d2L8aFekZ77L5iP9ormEBef4A5rzZzEifl0K7lOf2zs=; b=DANho+8Rpm3afw8SLgZjbGWvXLie+pJh0TeyOWffEPOKdnrrCKLXC0derITUuBEPpW Vxw457geEbeBL6jGrTia71hd/vMs8f8iwngMt5NQ3zerX0/0o7itlzUSohidsY9kp5Nu OOODI9uLYmuG0BE6j8bSWrgIFYTIOAosaSlvwHQYHZJ4ABjZHUaF8VHtDNwbLX7pSBEB dIV3GGxeoa0Qz0RXo5SjY1M1S//k5ZSeYNDyUm6yrbk8Ex94hgEnrhJobEUzOJzpnRUx IOCihBffTAklMS5v5p8WDPDx4bewfUDMCSfSfZLbAiHbWWVLV+ZxNl4106zq+uG/NiWd qq6g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@joelfernandes.org header.s=google header.b=YpU7e+Pj; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id kd10-20020a17090798ca00b00992f3097ed4si8540613ejc.871.2023.07.25.17.05.06; Tue, 25 Jul 2023 17:05:32 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@joelfernandes.org header.s=google header.b=YpU7e+Pj; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231805AbjGYX3c (ORCPT + 99 others); Tue, 25 Jul 2023 19:29:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39574 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231705AbjGYX3a (ORCPT ); Tue, 25 Jul 2023 19:29:30 -0400 Received: from mail-io1-xd32.google.com (mail-io1-xd32.google.com [IPv6:2607:f8b0:4864:20::d32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0E6D819BA for ; Tue, 25 Jul 2023 16:29:29 -0700 (PDT) Received: by mail-io1-xd32.google.com with SMTP id ca18e2360f4ac-7878e573827so224020839f.1 for ; Tue, 25 Jul 2023 16:29:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=joelfernandes.org; s=google; t=1690327767; x=1690932567; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=SjqXIuVtq0u8YshojVwovW+pCVt0mgB/LilQjFvvm4k=; b=YpU7e+Pj0/atr4EbsOJuU3upQRDKhMb6J0aC/Kwb5sK5X1H4MDa7KtxhGNk+JprQa8 xnDQkG8c0nNaGaiUAxLBwoHST4BNpZ2nFNZPTPeF7D0/5SyEMgWEAfwg/HQLLaNJ5GPS CVrYBTezNf2/CmiE7wAzXkmnNs+/HVvnLeI5w= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690327767; x=1690932567; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=SjqXIuVtq0u8YshojVwovW+pCVt0mgB/LilQjFvvm4k=; b=edg07nKGog7ElI3oqPFwG5ySav9qCI2L6R4kWtOf6hmOa1MVaH5exUq5D3++PgpNKL OD3UEF/kGCIC3i0EtE88+ug8zTCaF0vlMqITHcbwFN6cpG0w0KoqzxVzXSDCRD/kT1j6 VG1o4aqIDTbr71Z+WQXSjekwzaTFFDsKMP2U903IyAKC/cZ12N9HuwVj9IXdJTQTQFbD qpZMpaF9ZomnwMxUJ5vwo3pD52GdAy1Re8ad6FvQxHdzx97jdALkl4mQVpYeUF96Q9Hu mZ8uY6RGoteMOqJojW0NghSy3YqCxTFYAeFU4DB9clo1+jBCLkMGnnw4cBbZASepbzvD gzVg== X-Gm-Message-State: ABy/qLbs5NwSFpStFVETaLYVUNDX4FgdrwEV1vEjYAyRgywESrv7VN1V 4Fd1dIFRFM7yd/E4WhCPWdzEa21BIajgUWDk3a0= X-Received: by 2002:a6b:e818:0:b0:783:6ff9:a6c7 with SMTP id f24-20020a6be818000000b007836ff9a6c7mr334374ioh.6.1690327767704; Tue, 25 Jul 2023 16:29:27 -0700 (PDT) Received: from joelboxx5.c.googlers.com.com (156.190.123.34.bc.googleusercontent.com. [34.123.190.156]) by smtp.gmail.com with ESMTPSA id m18-20020a02c892000000b0041d859c5721sm3932053jao.64.2023.07.25.16.29.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Jul 2023 16:29:26 -0700 (PDT) From: "Joel Fernandes (Google)" To: linux-kernel@vger.kernel.org, Davidlohr Bueso , "Paul E. McKenney" , Josh Triplett Cc: rcu@vger.kernel.org, "Joel Fernandes (Google)" Subject: [PATCH 1/5] rcutorture: Fix stuttering races and other issues Date: Tue, 25 Jul 2023 23:29:06 +0000 Message-ID: <20230725232913.2981357-2-joel@joelfernandes.org> X-Mailer: git-send-email 2.41.0.487.g6d72f3e995-goog In-Reply-To: <20230725232913.2981357-1-joel@joelfernandes.org> References: <20230725232913.2981357-1-joel@joelfernandes.org> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1772439398764976385 X-GMAIL-MSGID: 1772439398764976385 The stuttering code isn't functioning as expected. Ideally, it should pause the torture threads for a designated period before resuming. Yet, it fails to halt the test for the correct duration. Additionally, a race condition exists, potentially causing the stuttering code to pause for an extended period if the 'spt' variable is non-zero due to the stutter orchestration thread's inadequate CPU time. Moreover, over-stuttering can hinder RCU's progress on TREE07 kernels. This happens as the stuttering code may run within a softirq due to RCU callbacks. Consequently, ksoftirqd keeps a CPU busy for several seconds, thus obstructing RCU's progress. This situation triggers a warning message in the logs: [ 2169.481783] rcu_torture_writer: rtort_pipe_count: 9 This warning suggests that an RCU torture object, although invisible to RCU readers, couldn't make it past the pipe array and be freed -- a strong indication that there weren't enough grace periods during the stutter interval. To address these issues, this patch sets the "stutter end" time to an absolute point in the future set by the main stutter thread. This is then used for waiting in stutter_wait(). While the stutter thread still defines this absolute time, the waiters' waiting logic doesn't rely on the stutter thread receiving sufficient CPU time to halt the stuttering as the halting is now self-controlled. Signed-off-by: Joel Fernandes (Google) --- kernel/torture.c | 46 +++++++++++++--------------------------------- 1 file changed, 13 insertions(+), 33 deletions(-) diff --git a/kernel/torture.c b/kernel/torture.c index 68dba4ecab5c..63f8f2a7d960 100644 --- a/kernel/torture.c +++ b/kernel/torture.c @@ -719,7 +719,7 @@ static void torture_shutdown_cleanup(void) * suddenly applied to or removed from the system. */ static struct task_struct *stutter_task; -static int stutter_pause_test; +static ktime_t stutter_till_abs_time; static int stutter; static int stutter_gap; @@ -729,30 +729,17 @@ static int stutter_gap; */ bool stutter_wait(const char *title) { - unsigned int i = 0; bool ret = false; - int spt; + ktime_t now_ns, till_ns; cond_resched_tasks_rcu_qs(); - spt = READ_ONCE(stutter_pause_test); - for (; spt; spt = READ_ONCE(stutter_pause_test)) { - if (!ret && !rt_task(current)) { - sched_set_normal(current, MAX_NICE); - ret = true; - } - if (spt == 1) { - torture_hrtimeout_jiffies(1, NULL); - } else if (spt == 2) { - while (READ_ONCE(stutter_pause_test)) { - if (!(i++ & 0xffff)) - torture_hrtimeout_us(10, 0, NULL); - cond_resched(); - } - } else { - torture_hrtimeout_jiffies(round_jiffies_relative(HZ), NULL); - } - torture_shutdown_absorb(title); + now_ns = ktime_get(); + till_ns = READ_ONCE(stutter_till_abs_time); + if (till_ns && ktime_before(now_ns, till_ns)) { + torture_hrtimeout_ns(ktime_sub(till_ns, now_ns), 0, NULL); + ret = true; } + torture_shutdown_absorb(title); return ret; } EXPORT_SYMBOL_GPL(stutter_wait); @@ -763,23 +750,16 @@ EXPORT_SYMBOL_GPL(stutter_wait); */ static int torture_stutter(void *arg) { - DEFINE_TORTURE_RANDOM(rand); - int wtime; + ktime_t till_ns; VERBOSE_TOROUT_STRING("torture_stutter task started"); do { if (!torture_must_stop() && stutter > 1) { - wtime = stutter; - if (stutter > 2) { - WRITE_ONCE(stutter_pause_test, 1); - wtime = stutter - 3; - torture_hrtimeout_jiffies(wtime, &rand); - wtime = 2; - } - WRITE_ONCE(stutter_pause_test, 2); - torture_hrtimeout_jiffies(wtime, NULL); + till_ns = ktime_add_ns(ktime_get(), + jiffies_to_nsecs(stutter)); + WRITE_ONCE(stutter_till_abs_time, till_ns); + torture_hrtimeout_jiffies(stutter - 1, NULL); } - WRITE_ONCE(stutter_pause_test, 0); if (!torture_must_stop()) torture_hrtimeout_jiffies(stutter_gap, NULL); torture_shutdown_absorb("torture_stutter"); From patchwork Tue Jul 25 23:29:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joel Fernandes X-Patchwork-Id: 126024 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:a985:0:b0:3e4:2afc:c1 with SMTP id t5csp108243vqo; Tue, 25 Jul 2023 18:13:32 -0700 (PDT) X-Google-Smtp-Source: APBJJlFlQ0d9Psfg496+NdTC1CbWlwtXOuOhnfA/Lyb2c3XQdKzdwLEaZKZ8z+uYolnM9SxWFsz+ X-Received: by 2002:a17:902:da8a:b0:1b3:fb76:215b with SMTP id j10-20020a170902da8a00b001b3fb76215bmr837919plx.48.1690334012095; Tue, 25 Jul 2023 18:13:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690334012; cv=none; d=google.com; s=arc-20160816; b=VFhk+7AOUnbLOBo8VySEWMVp9tXOHid7FH/0bocKgt45L7nhWOEdGzJlp2wGO3s5od hyo1jSTZ2s30693Kbqs8Dzzb8DWuzJFHEQxpH8dJwTA6gW416T2vYSFxC9FjEq2BTWVE Fly3WhrDox3RB6RSPXnWA/Q6KIhlq3t3jMosRYROs2GlAStTZ/DR07MReE1cKuNdCJ4X smUOw7TWEMIcxKwhUVh/981fEnOscjanFEvyHW+55Xyqwqo/zgSAbFjYfye0sDe4dtjh g4R55DOc8w4RvE8IAZX43S5tK1JlkGS9Vws+6LoIkUKZpaeg8YUpjDlH1xy1xSorHzQC Vyfw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=t1/tyaZq/EJBhi/7l+hIL3z3pGtmH2Zjtk0EiAh2ooA=; fh=j/8XIbVibyudUK/RHPCyC3GTHlBfjkM6BAqtFs55AwM=; b=hM/JUUPSaVVXwlAYEafnjxrgyjNlofP7yw/Vxu8xtP2mqx4U3Xj2e8dWDrUvhNOExB 8a0OiLMf0itonGt1EvxCLxaDzZnfFBWW9DKXg11acWGMXfqJhPrajj9k+u82PZbSaUMS vkHqgS6NgaVVK+X2Pepomq7BViPFxPlURnzxW7p4xp5OVsA1pA55kj9bOGpVPYGEKz8/ JJDcSnDus/rTbwRQIuNFpbubgUUI5Q3xtfoqQYAa/U8jkxarRmXqzMsBtWSXzb/+iT9U Tbx280tsJ6PXWNzOGNK955jM0kwk663EuD7f8XXEQo/4EHqgxIF6bmRpAttezWwg9Ioy cnJQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@joelfernandes.org header.s=google header.b=W4LU5bSa; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id u9-20020a1709026e0900b001bb21d502f0si11079091plk.76.2023.07.25.18.13.17; Tue, 25 Jul 2023 18:13:32 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@joelfernandes.org header.s=google header.b=W4LU5bSa; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231867AbjGYX3i (ORCPT + 99 others); Tue, 25 Jul 2023 19:29:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39598 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230211AbjGYX3b (ORCPT ); Tue, 25 Jul 2023 19:29:31 -0400 Received: from mail-io1-xd2e.google.com (mail-io1-xd2e.google.com [IPv6:2607:f8b0:4864:20::d2e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7F645120 for ; Tue, 25 Jul 2023 16:29:30 -0700 (PDT) Received: by mail-io1-xd2e.google.com with SMTP id ca18e2360f4ac-783549ef058so316767839f.2 for ; Tue, 25 Jul 2023 16:29:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=joelfernandes.org; s=google; t=1690327769; x=1690932569; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=t1/tyaZq/EJBhi/7l+hIL3z3pGtmH2Zjtk0EiAh2ooA=; b=W4LU5bSa/QHlkTdxIb4sY4pM7LWuU9noqwcduFR+0CmKenXKzGAI5fNoSvgjHOSlUm IwiPuf4Gm81MY4HrBIdQsBc4MnQ8nMUuNbJ7L7Lgmjqm9m9qXFc6vriyFqRhMkbcPGUE QC142xSTHhGMr/9RiF3+UQBA1dMQ1hLm+EzgM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690327769; x=1690932569; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=t1/tyaZq/EJBhi/7l+hIL3z3pGtmH2Zjtk0EiAh2ooA=; b=DjQCKqWFruR70cs4N1PbOOVJbtd4uisKEaDfq21zdtj29Z+X2L+xNuqkUYLAttpPcX dqu99BWwTPbbg8CRyJ4wFy0TDDY9zwnO9xYJt9GqFEv1o5Krn+WLgQWBCQTp1JAeNLSk emvhw+SaAqqcPpNpnLOR6p3LAlL+9SPONngD4OXNntsO8Ez4VAVFJXaxoDRxIs29sC/Y uqlleXBDn4nJNK6LZ6a/6LMxpHumk9ji4rlV/CmEodrWmJqhq68LZg36q+L/I/Tt1ZFC bIuZJzCjcfRrYvjul4dKeiKqYqhcZ8NDVMPXOprwwrEAkqznY2dlJFr8/T8CoqAF2kjw jrfA== X-Gm-Message-State: ABy/qLapS+TYA81cPtDyU6MGD0HouGy3/UxIp8OE6FxtdeyAO0dHQtxJ rB+/i2/N+yRcNbhQZAgN9LMnp5vNtxJDix2J98g= X-Received: by 2002:a6b:5c10:0:b0:783:72d4:8c38 with SMTP id z16-20020a6b5c10000000b0078372d48c38mr341104ioh.13.1690327769166; Tue, 25 Jul 2023 16:29:29 -0700 (PDT) Received: from joelboxx5.c.googlers.com.com (156.190.123.34.bc.googleusercontent.com. [34.123.190.156]) by smtp.gmail.com with ESMTPSA id m18-20020a02c892000000b0041d859c5721sm3932053jao.64.2023.07.25.16.29.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Jul 2023 16:29:28 -0700 (PDT) From: "Joel Fernandes (Google)" To: linux-kernel@vger.kernel.org, Lai Jiangshan , "Paul E. McKenney" , Josh Triplett , Steven Rostedt , Mathieu Desnoyers , rcu@vger.kernel.org (open list:SLEEPABLE READ-COPY UPDATE (SRCU)) Cc: rcu@vger.kernel.org, "Joel Fernandes (Google)" Subject: [PATCH 2/5] srcu: Fix error handling in init_srcu_struct_fields() Date: Tue, 25 Jul 2023 23:29:07 +0000 Message-ID: <20230725232913.2981357-3-joel@joelfernandes.org> X-Mailer: git-send-email 2.41.0.487.g6d72f3e995-goog In-Reply-To: <20230725232913.2981357-1-joel@joelfernandes.org> References: <20230725232913.2981357-1-joel@joelfernandes.org> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1772443677068687153 X-GMAIL-MSGID: 1772443677068687153 The current error handling in init_srcu_struct_fields() is a bit inconsistent. If init_srcu_struct_nodes() fails, the function either returns -ENOMEM or 0 depending on whether ssp->sda_is_static is true or false. This can make init_srcu_struct_fields() return 0 even if memory allocation failed! Simplify the error handling by always returning -ENOMEM if either init_srcu_struct_nodes() or the per-CPU allocation fails. This makes the control flow easier to follow and avoids the inconsistent return values. Add goto labels to avoid duplicating the error cleanup code. Link: https://lore.kernel.org/r/20230404003508.GA254019@google.com Signed-off-by: Joel Fernandes (Google) --- kernel/rcu/srcutree.c | 31 +++++++++++++++++-------------- 1 file changed, 17 insertions(+), 14 deletions(-) diff --git a/kernel/rcu/srcutree.c b/kernel/rcu/srcutree.c index 20d7a238d675..cbc37cbc1805 100644 --- a/kernel/rcu/srcutree.c +++ b/kernel/rcu/srcutree.c @@ -255,29 +255,32 @@ static int init_srcu_struct_fields(struct srcu_struct *ssp, bool is_static) ssp->srcu_sup->sda_is_static = is_static; if (!is_static) ssp->sda = alloc_percpu(struct srcu_data); - if (!ssp->sda) { - if (!is_static) - kfree(ssp->srcu_sup); - return -ENOMEM; - } + if (!ssp->sda) + goto err_free_sup; init_srcu_struct_data(ssp); ssp->srcu_sup->srcu_gp_seq_needed_exp = 0; ssp->srcu_sup->srcu_last_gp_end = ktime_get_mono_fast_ns(); if (READ_ONCE(ssp->srcu_sup->srcu_size_state) == SRCU_SIZE_SMALL && SRCU_SIZING_IS_INIT()) { - if (!init_srcu_struct_nodes(ssp, GFP_ATOMIC)) { - if (!ssp->srcu_sup->sda_is_static) { - free_percpu(ssp->sda); - ssp->sda = NULL; - kfree(ssp->srcu_sup); - return -ENOMEM; - } - } else { + if (!init_srcu_struct_nodes(ssp, GFP_ATOMIC)) + goto err_free_sda; + else WRITE_ONCE(ssp->srcu_sup->srcu_size_state, SRCU_SIZE_BIG); - } } ssp->srcu_sup->srcu_ssp = ssp; smp_store_release(&ssp->srcu_sup->srcu_gp_seq_needed, 0); /* Init done. */ return 0; + +err_free_sda: + if (!is_static) { + free_percpu(ssp->sda); + ssp->sda = NULL; + } +err_free_sup: + if (!is_static) { + kfree(ssp->srcu_sup); + ssp->srcu_sup = NULL; + } + return -ENOMEM; } #ifdef CONFIG_DEBUG_LOCK_ALLOC From patchwork Tue Jul 25 23:29:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joel Fernandes X-Patchwork-Id: 126019 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:a985:0:b0:3e4:2afc:c1 with SMTP id t5csp107586vqo; Tue, 25 Jul 2023 18:11:50 -0700 (PDT) X-Google-Smtp-Source: APBJJlFBAHP5v72cJ+3qnlhVgRpZHxzZlrKthgqcmgr7MnxrrJ87P8IaEAaNOTlsE/zu3mVompbC X-Received: by 2002:a05:6a00:21d4:b0:67a:39c4:5bb5 with SMTP id t20-20020a056a0021d400b0067a39c45bb5mr737149pfj.6.1690333910628; Tue, 25 Jul 2023 18:11:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690333910; cv=none; d=google.com; s=arc-20160816; b=EXsPNl1930q73IEPgBmftBrXMK+JXwWkYEhKvOrY79F5Fpt1hIxa4Bs/72zVib7YW3 MSsn9ggyu5pztqkF4p6m5OnsdkXnNT3jGEdKqJa8LDh1TB8vLUA/PqvfOxB/P3S3Ftz6 0yGZRLx2bPFrJsPW0vL6l3i/NgMJ6UPUqsCXXjszUWukgmjNeeRQ9p2YmYbsFh2YfWsd lGDlUd70WgVUfTBZXdgi3464qqp8vG8ym9B3v1eLgY5zwIlzOE9uJ3TCI6q1RhPDBfxV Cb91SkknrN1j/0ZpWO7NWS1FfGG4fJOpi8lItQH284XY7TwQVboDBqZeVtN5tp2OYZ4L v2kA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=B6faQrb3vVDF+1ZA3HcLro9nEgsaXGn2RtvbxbNNEvc=; fh=yqowAvOR6DNYuoofmoH16bMlYyGuXNhGc4Jfzr/7nGQ=; b=KLGmJ4CDW1W9xNCQG0ZA+FxuUE6Hwz1+Fnkh7/HBGsaoFjbnnbWhayR0NsEs5y5meF i8WjW8iP6g74EyYxLb3eh5VchTLE3NmlnNfkvQvJWwP0c98aCnf3/u+6XC5/naKNqtn+ x2RANBIFRtRD+g45jaci/pyuUzdmIcGsnQpxDNjvvY7x6SbXClSx0KTsqNn590LwakGd 535rnrA5t/n7nHUxdWz1Cq96ENroT1eAM4+aMJFAm32BTN2iYMiYC273rBk/z7jHN2ib eFqheZjKyFLj8QwpuKGAnMUyIbcbZ4Z6nOnjOMGcFQhsJ4kktDsT9yLifkWZHQEPCu1l vu5g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@joelfernandes.org header.s=google header.b=dOKqCkxi; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id m126-20020a633f84000000b005600a533c77si12362160pga.182.2023.07.25.18.11.37; Tue, 25 Jul 2023 18:11:50 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@joelfernandes.org header.s=google header.b=dOKqCkxi; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232022AbjGYX3k (ORCPT + 99 others); Tue, 25 Jul 2023 19:29:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39640 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231821AbjGYX3e (ORCPT ); Tue, 25 Jul 2023 19:29:34 -0400 Received: from mail-io1-xd2b.google.com (mail-io1-xd2b.google.com [IPv6:2607:f8b0:4864:20::d2b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4EC971BF8 for ; Tue, 25 Jul 2023 16:29:32 -0700 (PDT) Received: by mail-io1-xd2b.google.com with SMTP id ca18e2360f4ac-78363cc070aso316961539f.1 for ; Tue, 25 Jul 2023 16:29:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=joelfernandes.org; s=google; t=1690327770; x=1690932570; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=B6faQrb3vVDF+1ZA3HcLro9nEgsaXGn2RtvbxbNNEvc=; b=dOKqCkxitVYH7c0pdupRnZpK5/l2nGavKWDlS+ALoRAPtAH4m3BstqX7xtaGRRb1Wf d3seuANAUNcBh2wORYzQfdJRgZyYXkIXgQWow/2GUL1DlZMFSx0ijE0bHqYlnFjrE4P5 mHXFGWUEOy2hIGqzMfbeyqOFGR7zJNac5++X4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690327770; x=1690932570; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=B6faQrb3vVDF+1ZA3HcLro9nEgsaXGn2RtvbxbNNEvc=; b=YN6xrwjxb5s2Sviu/cB5IHl0EKDzY64EshVVsmgQr01PN5p1r2QHkENBJE8f+k2JLw gEZNfa7mlvt4o3dnVOmwKKtgCUrtRKdtNDcFfJP2Py+ipm4/r/X6xML1yvExqTm1oxkv 7S4WjlmWAVUssznpA1Hg2WowlxSQWZ11WfZNeg0zFPYse5G4Kg0QsLNdBoh69tEbUSSg puePU9mfRyiK1BkWBbOSGBRWVFpGIkfZC2eETNmF5mnmjVQPvuBydqpIjExN9FKgXwT/ +qnLjTGIy0UI3N0t6fKMCC2tmiggk0dLnNlaAlLbbEQV8AY8BQPMEm8jjs+tDzDUbK9W Ne3Q== X-Gm-Message-State: ABy/qLaEo+J2fMkvAB0qaPwEHCL/ke1A4O78im/UAXcEpNg3ZQ2IY0M8 EZEEjwCJLdXgHwyJVHxACi9MuuZtyj1Qbq5WsBo= X-Received: by 2002:a6b:e80e:0:b0:786:7100:72de with SMTP id f14-20020a6be80e000000b00786710072demr346606ioh.16.1690327770577; Tue, 25 Jul 2023 16:29:30 -0700 (PDT) Received: from joelboxx5.c.googlers.com.com (156.190.123.34.bc.googleusercontent.com. [34.123.190.156]) by smtp.gmail.com with ESMTPSA id m18-20020a02c892000000b0041d859c5721sm3932053jao.64.2023.07.25.16.29.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Jul 2023 16:29:29 -0700 (PDT) From: "Joel Fernandes (Google)" To: linux-kernel@vger.kernel.org, "Paul E. McKenney" , Frederic Weisbecker , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Zqiang , rcu@vger.kernel.org (open list:READ-COPY UPDATE (RCU)) Cc: rcu@vger.kernel.org Subject: [PATCH 3/5] tree/nocb: Adjust RCU_NOCB_WAKE_* macros from weaker to stronger Date: Tue, 25 Jul 2023 23:29:08 +0000 Message-ID: <20230725232913.2981357-4-joel@joelfernandes.org> X-Mailer: git-send-email 2.41.0.487.g6d72f3e995-goog In-Reply-To: <20230725232913.2981357-1-joel@joelfernandes.org> References: <20230725232913.2981357-1-joel@joelfernandes.org> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1772443570878034650 X-GMAIL-MSGID: 1772443570878034650 This is needed to make the next patch work correctly as we rely on the strength of the wakeup when comparing deferred-wakeup types across different CPUs. Signed-off-by: Joel Fernandes (Google) --- kernel/rcu/tree.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/kernel/rcu/tree.h b/kernel/rcu/tree.h index 192536916f9a..0f40a9c2b78d 100644 --- a/kernel/rcu/tree.h +++ b/kernel/rcu/tree.h @@ -288,8 +288,8 @@ struct rcu_data { /* Values for nocb_defer_wakeup field in struct rcu_data. */ #define RCU_NOCB_WAKE_NOT 0 -#define RCU_NOCB_WAKE_BYPASS 1 -#define RCU_NOCB_WAKE_LAZY 2 +#define RCU_NOCB_WAKE_LAZY 1 +#define RCU_NOCB_WAKE_BYPASS 2 #define RCU_NOCB_WAKE 3 #define RCU_NOCB_WAKE_FORCE 4 From patchwork Tue Jul 25 23:29:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joel Fernandes X-Patchwork-Id: 126015 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:a985:0:b0:3e4:2afc:c1 with SMTP id t5csp101897vqo; Tue, 25 Jul 2023 17:58:30 -0700 (PDT) X-Google-Smtp-Source: APBJJlETaNZnmQ7tYtECxUSRv5xymy98dDwX2g/kXEkRZFzysp+0SSN9mlxnW/rQDoRt5fLKc9vs X-Received: by 2002:a05:6e02:1488:b0:347:693a:a52b with SMTP id n8-20020a056e02148800b00347693aa52bmr822742ilk.6.1690333110009; Tue, 25 Jul 2023 17:58:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690333109; cv=none; d=google.com; s=arc-20160816; b=N248Hke+IJEHXl8boORWAqRF9NOROnImImOD8jhOKXpVrrlCI1b59gGCIPf4LhxERx 4HyLcAIZ5g5eRRxS15bmZ9htTThvXp6DmxzAmVTXca4QZK64O105laredvzglFmHOSTK vX3j/fD1+8BSXu0HgbAAoFRos9Zd+8XG66DYaYx7ZLXECBGWv9QQC4EMEkPZqRy9BN4/ CxpcsNcG89ShoC86c337zJmjvGQ/skF5Q027igaIWe/CIzNeVDSOgQf+2fHoEHc8cbHk 7yrElnsV1P7skWz00UqwisGQjn/PeyCT/noxx3pGgMpvVxfvjtevfuv0Hc6x64GPyk+A gN1Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=0DIiXFGh73Irk8k8jfSTlgn3pzuU9BPg/xYZpIge3SA=; fh=yqowAvOR6DNYuoofmoH16bMlYyGuXNhGc4Jfzr/7nGQ=; b=P/2NOrW0yk+59DhZjdjkz76r4rCrnNHkb2uPF2Tja3LCGHNl5E4YT6sV7j+hzciFJj wsBbviGHOIj4ZHlgfGgbfKgRUmTh/hTd1SGgQUKeyca8jVabPhPXIsms3iE1wyk1qpz0 YzSajC/VzbeiKqL29Gs+nTyVLuXpzsk3zPSPu2/Yqmb2rDW+JvAGAiD9xWk1TwlW32UD wnlP8ae+XcohFWTjLRe6R3TCB4asA+XyhwPDne2KSTNo0otqSAJPgjr/ii2/z10cUw1s gSAic9gc+WJhyUlaPKwKnVdFAToQ1mAyoLYbJbAvjRvkg1pDOnWII4mmjFd+GkQdr5ud U7Jw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@joelfernandes.org header.s=google header.b=devazoCc; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id g2-20020a17090a9b8200b00267eefee39fsi260464pjp.131.2023.07.25.17.58.15; Tue, 25 Jul 2023 17:58:29 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@joelfernandes.org header.s=google header.b=devazoCc; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232330AbjGYX3v (ORCPT + 99 others); Tue, 25 Jul 2023 19:29:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39768 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231896AbjGYX3o (ORCPT ); Tue, 25 Jul 2023 19:29:44 -0400 Received: from mail-io1-xd30.google.com (mail-io1-xd30.google.com [IPv6:2607:f8b0:4864:20::d30]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A4E981FC9 for ; Tue, 25 Jul 2023 16:29:33 -0700 (PDT) Received: by mail-io1-xd30.google.com with SMTP id ca18e2360f4ac-78374596182so307959339f.0 for ; Tue, 25 Jul 2023 16:29:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=joelfernandes.org; s=google; t=1690327772; x=1690932572; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=0DIiXFGh73Irk8k8jfSTlgn3pzuU9BPg/xYZpIge3SA=; b=devazoCc1ePSllifONKGQCl8V8kDvW5uvDEsIy/u3SDJfI/vz1wlFS0SUxKZdfqrcB NEV4tcQXSLdIR+lnufehU/yii4Le4bCblWZXsvn8b0MbCrfA5CbEFQh/KvLUcVyexJgD bAOucRRZCGzD+cJEPlkY3nJk/aUwJlJqC6WxA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690327772; x=1690932572; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0DIiXFGh73Irk8k8jfSTlgn3pzuU9BPg/xYZpIge3SA=; b=TqQKXbC151OYB6lHYeEu1+4y3EsVejkemOHuwCC5mXEzcVZsndUBOpeOnzlhmHgDr2 pLpIavvBrnCJY2tIskgtpB3Qeaqr5HbpYPEP2mzrEPPxfrgLq/FjpZ96yNMhYk4LKKN+ xLbCKv5lZBin2wradNaWt/9HneYW09VHTMInglmHITPRtOmXK54R9ajZ1hw+4wrDHjKk IcgW3Me+ORGjgdohGnkjECEkSnD1ETa7T7P4Bri0p90higfek48t9643+W+Y9tgolGH1 fw/164n4f2Z3rHj0Y9PoKNKR6nHnusaldKubwxdISrJCLgyalkTgZmFwKeFrWO4Mkz9P mJTA== X-Gm-Message-State: ABy/qLZizpOKLZvKZTd7YKhvigAb3GpfNo6qQ8On/kSuaVeGpFF+epO5 ggA3xb0eBB/JyM+ctZbywGpJC4SdNmWVVtIoglU= X-Received: by 2002:a05:6602:257a:b0:783:5fd9:3789 with SMTP id dj26-20020a056602257a00b007835fd93789mr367880iob.1.1690327772055; Tue, 25 Jul 2023 16:29:32 -0700 (PDT) Received: from joelboxx5.c.googlers.com.com (156.190.123.34.bc.googleusercontent.com. [34.123.190.156]) by smtp.gmail.com with ESMTPSA id m18-20020a02c892000000b0041d859c5721sm3932053jao.64.2023.07.25.16.29.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Jul 2023 16:29:31 -0700 (PDT) From: "Joel Fernandes (Google)" To: linux-kernel@vger.kernel.org, "Paul E. McKenney" , Frederic Weisbecker , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Zqiang , rcu@vger.kernel.org (open list:READ-COPY UPDATE (RCU)) Cc: rcu@vger.kernel.org Subject: [PATCH 4/5] tree/nocb: Improve readability of nocb_gp_wait() Date: Tue, 25 Jul 2023 23:29:09 +0000 Message-ID: <20230725232913.2981357-5-joel@joelfernandes.org> X-Mailer: git-send-email 2.41.0.487.g6d72f3e995-goog In-Reply-To: <20230725232913.2981357-1-joel@joelfernandes.org> References: <20230725232913.2981357-1-joel@joelfernandes.org> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1772442730952442732 X-GMAIL-MSGID: 1772442730952442732 The nocb_gp_wait() function contains logic to check each rdp's bypass list, flush if needed, and decide on wakeups. This makes the function hard to follow. Split out the bypass checking and flushing into a separate helper nocb_gp_flush_wake(). The new function encapsulates the logic to: - Check if the bypass needs to be flushed - Flush if needed - Return info on wakeups (lazy, bypass or none) nocb_gp_wait() now becomes simpler by calling the helper to handle the bypass flushing and waking logic. This splitting improves the readability and maintainability of the code by encapsulating related logic into a function with a clear purpose. Signed-off-by: Joel Fernandes (Google) --- kernel/rcu/tree_nocb.h | 113 ++++++++++++++++++++++++----------------- 1 file changed, 66 insertions(+), 47 deletions(-) diff --git a/kernel/rcu/tree_nocb.h b/kernel/rcu/tree_nocb.h index 5598212d1f27..c805825c3f00 100644 --- a/kernel/rcu/tree_nocb.h +++ b/kernel/rcu/tree_nocb.h @@ -670,22 +670,68 @@ static void nocb_gp_sleep(struct rcu_data *my_rdp, int cpu) trace_rcu_nocb_wake(rcu_state.name, cpu, TPS("EndSleep")); } +/* + * Given an rdp, flush its bypass list if needed and return information about + * if a deferred-wakeup needs to be organized depending on whether things are + * still in the bypass list. Also tell caller if the list was flushed and if it + * is still empty after any flushing. + */ +static int nocb_gp_flush_wake(struct rcu_data *rdp, bool *empty, bool *flush) +{ + long bypass_ncbs; + long lazy_ncbs; + unsigned long j = jiffies; + + trace_rcu_nocb_wake(rcu_state.name, rdp->cpu, TPS("Check")); + lockdep_assert_held(&rdp->nocb_lock); + bypass_ncbs = rcu_cblist_n_cbs(&rdp->nocb_bypass); + lazy_ncbs = READ_ONCE(rdp->lazy_len); + + *flush = false; + *empty = false; + if (bypass_ncbs && (lazy_ncbs == bypass_ncbs) && + (time_after(j, READ_ONCE(rdp->nocb_bypass_first) + jiffies_till_flush) || + bypass_ncbs > 2 * qhimark)) { + *flush = true; + } else if (bypass_ncbs && (lazy_ncbs != bypass_ncbs) && + (time_after(j, READ_ONCE(rdp->nocb_bypass_first) + 1) || + bypass_ncbs > 2 * qhimark)) { + *flush = true; + } else if (!bypass_ncbs && rcu_segcblist_empty(&rdp->cblist)) { + *empty = true; + return RCU_NOCB_WAKE_NOT; + } + + if (*flush) { + // Bypass full or old, so flush it. + (void)rcu_nocb_try_flush_bypass(rdp, j); + bypass_ncbs = rcu_cblist_n_cbs(&rdp->nocb_bypass); + lazy_ncbs = READ_ONCE(rdp->lazy_len); + } + + if (bypass_ncbs) { + trace_rcu_nocb_wake(rcu_state.name, rdp->cpu, + bypass_ncbs == lazy_ncbs ? TPS("Lazy") : TPS("Bypass")); + return (bypass_ncbs == lazy_ncbs ? RCU_NOCB_WAKE_LAZY : + RCU_NOCB_WAKE_BYPASS); + } + return RCU_NOCB_WAKE_NOT; +} + /* * No-CBs GP kthreads come here to wait for additional callbacks to show up * or for grace periods to end. */ static void nocb_gp_wait(struct rcu_data *my_rdp) { - bool bypass = false; int __maybe_unused cpu = my_rdp->cpu; unsigned long cur_gp_seq; unsigned long flags; bool gotcbs = false; - unsigned long j = jiffies; - bool lazy = false; bool needwait_gp = false; // This prevents actual uninitialized use. bool needwake; bool needwake_gp; + int defer_wake_type = RCU_NOCB_WAKE_NOT; struct rcu_data *rdp, *rdp_toggling = NULL; struct rcu_node *rnp; unsigned long wait_gp_seq = 0; // Suppress "use uninitialized" warning. @@ -712,44 +758,24 @@ static void nocb_gp_wait(struct rcu_data *my_rdp) * won't be ignored for long. */ list_for_each_entry(rdp, &my_rdp->nocb_head_rdp, nocb_entry_rdp) { - long bypass_ncbs; - bool flush_bypass = false; - long lazy_ncbs; + int defer_wake_type_one = RCU_NOCB_WAKE_NOT; + bool flushed; + bool empty; - trace_rcu_nocb_wake(rcu_state.name, rdp->cpu, TPS("Check")); rcu_nocb_lock_irqsave(rdp, flags); - lockdep_assert_held(&rdp->nocb_lock); - bypass_ncbs = rcu_cblist_n_cbs(&rdp->nocb_bypass); - lazy_ncbs = READ_ONCE(rdp->lazy_len); + defer_wake_type_one = nocb_gp_flush_wake(rdp, &empty, &flushed); - if (bypass_ncbs && (lazy_ncbs == bypass_ncbs) && - (time_after(j, READ_ONCE(rdp->nocb_bypass_first) + jiffies_till_flush) || - bypass_ncbs > 2 * qhimark)) { - flush_bypass = true; - } else if (bypass_ncbs && (lazy_ncbs != bypass_ncbs) && - (time_after(j, READ_ONCE(rdp->nocb_bypass_first) + 1) || - bypass_ncbs > 2 * qhimark)) { - flush_bypass = true; - } else if (!bypass_ncbs && rcu_segcblist_empty(&rdp->cblist)) { - rcu_nocb_unlock_irqrestore(rdp, flags); - continue; /* No callbacks here, try next. */ - } + // We may need to do a deferred wakeup later for bypass/lazy + // So note down what we learnt from the rdp. + defer_wake_type = max(defer_wake_type_one, defer_wake_type); - if (flush_bypass) { - // Bypass full or old, so flush it. - (void)rcu_nocb_try_flush_bypass(rdp, j); - bypass_ncbs = rcu_cblist_n_cbs(&rdp->nocb_bypass); - lazy_ncbs = READ_ONCE(rdp->lazy_len); + // Did we make any updates to main cblist? If not, no + // non-deferred wake up to do for this rdp. + if (!flushed && empty) { + rcu_nocb_unlock_irqrestore(rdp, flags); + continue; } - if (bypass_ncbs) { - trace_rcu_nocb_wake(rcu_state.name, rdp->cpu, - bypass_ncbs == lazy_ncbs ? TPS("Lazy") : TPS("Bypass")); - if (bypass_ncbs == lazy_ncbs) - lazy = true; - else - bypass = true; - } rnp = rdp->mynode; // Advance callbacks if helpful and low contention. @@ -792,23 +818,16 @@ static void nocb_gp_wait(struct rcu_data *my_rdp) rcu_gp_kthread_wake(); } - my_rdp->nocb_gp_bypass = bypass; + my_rdp->nocb_gp_bypass = (defer_wake_type == RCU_NOCB_WAKE_BYPASS); my_rdp->nocb_gp_gp = needwait_gp; my_rdp->nocb_gp_seq = needwait_gp ? wait_gp_seq : 0; // At least one child with non-empty ->nocb_bypass, so set // timer in order to avoid stranding its callbacks. - if (!rcu_nocb_poll) { - // If bypass list only has lazy CBs. Add a deferred lazy wake up. - if (lazy && !bypass) { - wake_nocb_gp_defer(my_rdp, RCU_NOCB_WAKE_LAZY, - TPS("WakeLazyIsDeferred")); - // Otherwise add a deferred bypass wake up. - } else if (bypass) { - wake_nocb_gp_defer(my_rdp, RCU_NOCB_WAKE_BYPASS, - TPS("WakeBypassIsDeferred")); - } - } + if (!rcu_nocb_poll && defer_wake_type != RCU_NOCB_WAKE_NOT) + wake_nocb_gp_defer(my_rdp, defer_wake_type, + defer_wake_type == RCU_NOCB_WAKE_LAZY ? + TPS("WakeLazyIsDeferred") : TPS("WakeBypassIsDeferred")); if (rcu_nocb_poll) { /* Polling, so trace if first poll in the series. */ From patchwork Tue Jul 25 23:29:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joel Fernandes X-Patchwork-Id: 125985 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:a985:0:b0:3e4:2afc:c1 with SMTP id t5csp78422vqo; Tue, 25 Jul 2023 16:57:28 -0700 (PDT) X-Google-Smtp-Source: APBJJlFGRp4NDN7P9LgfxYcWQXfTybcxaGOQuKr3S71Vmgx0c8gFAreg83YSYBQPOOjCoDarPCLO X-Received: by 2002:a05:6a20:7f82:b0:10f:be0:4dce with SMTP id d2-20020a056a207f8200b0010f0be04dcemr771701pzj.8.1690329447972; Tue, 25 Jul 2023 16:57:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690329447; cv=none; d=google.com; s=arc-20160816; b=yg/b6cegntO5t4Bly6vdN8hEob8qydVSqg6dWBq/Hd5p6voX4w9KEOV9kqLH81BIfs 5CAcPiDt231rpTpCy4U37fFFV+EiTWilkZWSX0WwcqpMg5Rg01PpO1LFFDB8ANS5+6+g qvb30TbKLMx/HD3JFbzp5FgTzrbuBIoUjsr690ifvia6P4Wkx4qL5ABQ0/OfR8gRrfuG AZFranRwGDT0txVTHvXIlgO9CFixxKLlhGpKo1oUmKIUBNmbZ6JUBeQ9i1qiKKRE8Jzr RqcSMvDS6The1fW4HophYEQukSgJzjRCqF1YvC9ixwG+h2qQ7TEjw2jePSZLVFceGW5h SzGg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=ynoALsv1aca8y8xpjhuiunc/CTMUKjN2qeAE0MoxzWk=; fh=gV9g6rug/diQlr0BJCLiDrxZA1M9Z2bL/3zVCoT+r4c=; b=RNjOggwf3YjXcznOSrLLVXciaAhLIT+deQS/HzlKwJPtMVd/oW4EOctXFsAGn7WQbZ HwQ3PRuXa50Ll10FocULu30XgOspfJZ5vxEkpKadKCzIO46pRSGbxOSre9hgeMIbw9EM kCaGldbc0Ufw1uoBhQrdBMeO3DcC2Z0E1N9tNj156pQQL37g6Dy2+/xyc4PxBllCVpEv D/1x2tinAZjR+CbeVebU9RVlq4Ii3L7Goegdhe8GrZc1fNkyhgCNpj87y7gnaDe/UCe/ 0SqFio7wBXMasAI8m8YvFVKBte30LLcgnP/RHNN1OSaq16+Uza5dJZo8bNnFkC26NLsu UqWQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@joelfernandes.org header.s=google header.b="T1/qgW80"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id b26-20020a6567da000000b0056394893c6csi8756627pgs.774.2023.07.25.16.57.14; Tue, 25 Jul 2023 16:57:27 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@joelfernandes.org header.s=google header.b="T1/qgW80"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232176AbjGYX3r (ORCPT + 99 others); Tue, 25 Jul 2023 19:29:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39760 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231886AbjGYX3n (ORCPT ); Tue, 25 Jul 2023 19:29:43 -0400 Received: from mail-io1-xd33.google.com (mail-io1-xd33.google.com [IPv6:2607:f8b0:4864:20::d33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 019491FE2 for ; Tue, 25 Jul 2023 16:29:34 -0700 (PDT) Received: by mail-io1-xd33.google.com with SMTP id ca18e2360f4ac-77a62a84855so261093439f.1 for ; Tue, 25 Jul 2023 16:29:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=joelfernandes.org; s=google; t=1690327773; x=1690932573; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ynoALsv1aca8y8xpjhuiunc/CTMUKjN2qeAE0MoxzWk=; b=T1/qgW80ZW0ruvAexTl3SwPC/VD8efNMinccPq8SOmll1PeYaL7nJcC+rT2y/ICCI/ Zm7HzPisHr2CWscQ8DNq4ATfmZQ4copg8Byy3hFGTtKvq0+ing3UNCdbIzc4z9yskHeM i6XYgDFBYacc8aSIAdToK+pj+H/fS21Ias0jU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690327773; x=1690932573; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ynoALsv1aca8y8xpjhuiunc/CTMUKjN2qeAE0MoxzWk=; b=BmhENbGzzU2H1WTFNohYWsDAnjGcgw7mxMbuoCk1FE9kS7HBzjveNVMdbsyMznwrPM mHvGuButh43zvqOT/UY9UUD79WqtRHcEVyxyafzeqv/cLCiM2BILkmuxj1KjqhpIO2hj UrymULVR/8OUkMavZqM2JT4Zoc1SYsK9Dy+rAlrR4OW4AiYn7/IUZanyLYhr/jQxQstq y9ntZJ5LIJUYmLWmy21QQR6DnYd1UhNRYIi23FE5yRsMxLHb/XvCzJDAvRF8kgGQKdGy k4wpSH/zLd+hLlcVYLZFfwovpo836amUy/i3gPzjdmikQyvBoqiDbLseW+xoC9iAhUFw Gxjw== X-Gm-Message-State: ABy/qLarL7LrvOv8dy1yAp7oZiZR9SEn9KBPSpsK9f76m6kwYQ+gRNfJ buR0T71MffvyxPhQnN5zfDABtEKRMPjOon+GHYs= X-Received: by 2002:a5e:8a4c:0:b0:787:34d:ee9e with SMTP id o12-20020a5e8a4c000000b00787034dee9emr356394iom.7.1690327773744; Tue, 25 Jul 2023 16:29:33 -0700 (PDT) Received: from joelboxx5.c.googlers.com.com (156.190.123.34.bc.googleusercontent.com. [34.123.190.156]) by smtp.gmail.com with ESMTPSA id m18-20020a02c892000000b0041d859c5721sm3932053jao.64.2023.07.25.16.29.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Jul 2023 16:29:32 -0700 (PDT) From: "Joel Fernandes (Google)" To: linux-kernel@vger.kernel.org, Andy Whitcroft , Joe Perches , Dwaipayan Ray , Lukas Bulwahn Cc: rcu@vger.kernel.org, paulmck@kernel.org, "Joel Fernandes (Google)" Subject: [PATCH 5/5] Revert "checkpatch: Error out if deprecated RCU API used" Date: Tue, 25 Jul 2023 23:29:10 +0000 Message-ID: <20230725232913.2981357-6-joel@joelfernandes.org> X-Mailer: git-send-email 2.41.0.487.g6d72f3e995-goog In-Reply-To: <20230725232913.2981357-1-joel@joelfernandes.org> References: <20230725232913.2981357-1-joel@joelfernandes.org> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1772438891534011994 X-GMAIL-MSGID: 1772438891534011994 The check for single-argument kfree_rcu() is no longer needed as all users have been converted. This reverts commit 1eacac3255495be7502d406e2ba5444fb5c3607c. Signed-off-by: Joel Fernandes (Google) --- scripts/checkpatch.pl | 9 --------- 1 file changed, 9 deletions(-) diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl index a9841148cde2..528f619520eb 100755 --- a/scripts/checkpatch.pl +++ b/scripts/checkpatch.pl @@ -6422,15 +6422,6 @@ sub process { } } -# check for soon-to-be-deprecated single-argument k[v]free_rcu() API - if ($line =~ /\bk[v]?free_rcu\s*\([^(]+\)/) { - if ($line =~ /\bk[v]?free_rcu\s*\([^,]+\)/) { - ERROR("DEPRECATED_API", - "Single-argument k[v]free_rcu() API is deprecated, please pass rcu_head object or call k[v]free_rcu_mightsleep()." . $herecurr); - } - } - - # check for unnecessary "Out of Memory" messages if ($line =~ /^\+.*\b$logFunctions\s*\(/ && $prevline =~ /^[ \+]\s*if\s*\(\s*(\!\s*|NULL\s*==\s*)?($Lval)(\s*==\s*NULL\s*)?\s*\)/ &&