From patchwork Fri Mar 24 00:19:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 7126 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:604a:0:0:0:0:0 with SMTP id j10csp54516wrt; Thu, 23 Mar 2023 17:20:48 -0700 (PDT) X-Google-Smtp-Source: AKy350aO8mZOJjS+TdkR+tZ028oFPL2J2b21y8NkM5Q/cm6a9jPMi7HXmVcF1BmnteP4K2VWTzu5 X-Received: by 2002:a17:906:31cb:b0:92c:a80e:225f with SMTP id f11-20020a17090631cb00b0092ca80e225fmr915589ejf.52.1679617247970; Thu, 23 Mar 2023 17:20:47 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1679617247; cv=none; d=google.com; s=arc-20160816; b=f0DILHF3vA+9wNdD5CrkbPeDrDA7hDFqKebIuisgTr1vvv2kEp3kIUtv8B1lCMn6G7 XETFt6Bbs5uChqwUNXJvJLhtArYMOW6dmgMnkRBGL5z3r9TXzsFRcCjuX89SHL30H4Sa pYZb5DqrqPEq3v4DsMktJi12idgKEnbw2pVayRETYtZtgeRGXj6fR8a3BAjMmk3zRXvM DQla4j4IlWdbFYJzg6En5lYtnyPPLJx5Oj6opflbs20e1Y/7tDDwcWCZJk21o8brGGX5 RFDKZsgOLdPCy2sPB+shVEhk2SDUa76UenjZ53wHuKJyGNQP/smZs7otp5pzpZSF3b1y s7rg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-disposition:mime-version:reply-to :message-id:subject:cc:to:from:date:dkim-signature; bh=JnHIvWPHPEIOdrG3BXxzEN6yXsk5UrWRl4L2buh6/Nc=; b=tGGwUAwyuYyFhtr45793Os7O/7PG82XscfQAxZkr9C6WOYGJcjgbOgCc2XlIGtw2rE +MwYBAmgYhKFb3XIKEm1MY9k5DqZKP7H8GvAH9JZTEB2qhcmwX2XtE07WKlhHWE4XwoR AzqnvNBGEb/PJNYOOwSvdmspTjWd48lxEJ7NbIIO131+6ssKxVDuw64ew+FLrpCcwgkh 0voVnO4kmEoqfwaSlBBxfU0ZDVggoRU/US9ipG6evw0QqxCBD6mTNHjOY1jAkk+jIKCp LxhRQDJ1OYvkdf1QYgiPyGH84KiQqpp0xf3iqtuKukmcEJAJwfQwalJ8Z1JBQ+/DXcY+ 5yXQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=YNUC1KV7; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id qb25-20020a1709077e9900b0093acfe3810asi7234861ejc.142.2023.03.23.17.20.22; Thu, 23 Mar 2023 17:20:47 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=YNUC1KV7; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230213AbjCXATM (ORCPT + 99 others); Thu, 23 Mar 2023 20:19:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56614 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229589AbjCXATK (ORCPT ); Thu, 23 Mar 2023 20:19:10 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BC361CC05; Thu, 23 Mar 2023 17:19:09 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 55E956280C; Fri, 24 Mar 2023 00:19:09 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B7EDBC433EF; Fri, 24 Mar 2023 00:19:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1679617148; bh=IT1NxcXw2pAVasDPEQHbwE+mpHcOER7ahsHXYZ2RRJU=; h=Date:From:To:Cc:Subject:Reply-To:From; b=YNUC1KV7PPaLp93YVhVlnR9dfxSOQ0WEOoKPTsw1KgEkzFS52OxMD9wiiEqjruYFA IAsi6DHiQe6BGt6xJbz+dnKKpyz7m04AgbC1nDXhW56uYkb5nTUyJf/daRx0Cgd2Q5 XEJPhWnEb1bY+BqDDOr3fH3jxNTWPai2sh68ZmZkkvLnXWX2MW64vTbnxxE509Nisw 513/q2+HMmw9uiG+3MrHwLza6XzvEc9Iqe8CLrdDIivd5b0xDL5Z0Nzdl5pn5zOg3m BFT80JWN/SUs493CzakatMdKabQboRPGTDgyfnK6Mgz2dqsfrkCJCFVnuZcuVzXY3E Ke5RfvWuJn0ag== Received: by paulmck-ThinkPad-P72.home (Postfix, from userid 1000) id 5626F1540379; Thu, 23 Mar 2023 17:19:08 -0700 (PDT) Date: Thu, 23 Mar 2023 17:19:08 -0700 From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, hch@lst.de Subject: [PATCH RFC rcu 0/19] Further shrink srcu_struct to promote cache locality Message-ID: <3db82572-f156-4a5d-b711-841aa28bd996@paulmck-laptop> Reply-To: paulmck@kernel.org MIME-Version: 1.0 Content-Disposition: inline X-Spam-Status: No, score=-5.2 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI,SPF_HELO_NONE, SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1761206335336620056?= X-GMAIL-MSGID: =?utf-8?q?1761206335336620056?= Hello! This RFC series shrinks the srcu_struct structure to the bare minimum required to support SRCU readers, relegating the remaining fields to a new srcu_usage structure. Statically allocated srcu_struct structures created by DEFINE_SRCU() and DEFINE_STATIC_SRCU() have statically allocated srcu_usage structures, but those required for dynamically allocated srcu_struct structures that are initialized using init_srcu_struct() are dynamically allocated. The results is a reduction in the size of an srcu_struct structure from a couple hundred bytes to just 24 bytes on x86_64 systems. This can be helpful when SRCU readers are used in a fastpath for which the srcu_struct structure must be embedded in another structure, and especially where that fastpath also needs to access fields both before and after the srcu_struct structure. This series takes baby steps, in part because breaking SRCU means that you get absolutely no console output. Yes, I did learn this the hard way. Why do you ask? ;-) Here are those baby steps: 1. Add whitespace to __SRCU_STRUCT_INIT() & __DEFINE_SRCU(). 2. Use static init for statically allocated in-module srcu_struct. 3. Begin offloading srcu_struct fields to srcu_update. Note that this affects notifiers, which open-code static allocation of an srcu_struct structure. (And no, I still do not see a way to abstract this, sorry!) 4. Move ->level from srcu_struct to srcu_usage. 5. Move ->srcu_size_state from srcu_struct to srcu_usage. 6. Move ->srcu_cb_mutex from srcu_struct to srcu_usage. 7. Move ->lock initialization after srcu_usage allocation. 8. Move ->lock from srcu_struct to srcu_usage. 9. Move ->srcu_gp_mutex from srcu_struct to srcu_usage. 10. Move grace-period fields from srcu_struct to srcu_usage. 11. Move heuristics fields from srcu_struct to srcu_usage. 12. Move ->sda_is_static from srcu_struct to srcu_usage. 13. Move srcu_barrier() fields from srcu_struct to srcu_usage. 14. Move work-scheduling fields from srcu_struct to srcu_usage. 15. Fix long lines in srcu_get_delay(). 16. Fix long lines in cleanup_srcu_struct(). 17. Fix long lines in srcu_gp_end(). 18. Fix long lines in srcu_funnel_gp_start(). 19. Remove extraneous parentheses from srcu_read_lock() etc. Thanx, Paul ------------------------------------------------------------------------ b/include/linux/notifier.h | 5 b/include/linux/srcu.h | 8 b/include/linux/srcutiny.h | 6 b/include/linux/srcutree.h | 28 +- b/kernel/rcu/rcu.h | 6 b/kernel/rcu/srcutree.c | 19 + include/linux/srcutree.h | 123 ++++++----- kernel/rcu/srcutree.c | 488 +++++++++++++++++++++++---------------------- 8 files changed, 368 insertions(+), 315 deletions(-)