From patchwork Thu Aug 3 03:24:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joel Fernandes X-Patchwork-Id: 130298 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9f41:0:b0:3e4:2afc:c1 with SMTP id v1csp890954vqx; Wed, 2 Aug 2023 20:51:12 -0700 (PDT) X-Google-Smtp-Source: APBJJlFaPmB6OWv7S5UjSkj1574C6n8y1LjWX5B4zSXhAfxKkkNhaBQ+R7HPC7CF5z1avL+XF+3t X-Received: by 2002:a17:902:e810:b0:1b8:5827:8763 with SMTP id u16-20020a170902e81000b001b858278763mr18540685plg.4.1691034672304; Wed, 02 Aug 2023 20:51:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1691034672; cv=none; d=google.com; s=arc-20160816; b=BrMoWg4OuJ1LmhGHZOOPSfcpBnUsE+cr6e69hD0f30QLPUI5X+6GFgd926yQR1QHkA 1lVWeWuuStxGHBmATTwLbeu3qYecZMC/vpGH8KJ4pv9SvhEhZjgYNV0yKp71T0uEmsK0 WdFLo8hO6QgSxPQMhpiuXyWQ6dYave/oVEdN3uPDUVMcIUGrHWLXitABDvb3d5biLs3U PGZ94BJRl/cHFXxXTYlcpItMHL1Xl2PeusFJQxX3GT0zqso45HqER5tFN0VKAE4TFd+7 LTCKtN1+tIWh87BTiujDPqB+Rzm1mmOTeULAm6UvCBD8eUV2qWXWJCi6gGPdu+mhI4U4 jMTQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=ri+AV8Kf+G5whax+bk9HTGfLE0fpaHCunlBX3LcI+MY=; fh=Cz95V29EXGwqB37WUOXxHjkcuI2FsbVcP5QmsorGYYo=; b=z+kDyI5i1CLyQlJq9plA/9mHPvOVWUbxE6m8VoXZjEJLUgpbYUbmuFwGpkX4XMpL09 zGUNBQHwQpPQEreykvW5pyM8G/JXHtYBRpNwiGlqM6EjXwucfodVbJYhHtxc99xj+dZ8 a8MYifPranWuyjudnbCIIJLCP+auz6Bxka4DRL/kiGu6A+aAe8shiULnNhjhUOMc1lfF ZPEBoNo6L9Nhv5nN2sF3zVKJH8q3UomuHZt29c1c7E8wBInvbXEg98NzPKeavGqx7csW 5seefQWyGkVaF5Yh9/K0ZB0R4YEbcDQKpahLRuRrmmStPtwCKzn3JvMakNJSWjy64RoS OoIg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@joelfernandes.org header.s=google header.b=SSROfGyq; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id u9-20020a170902e80900b001b3eeaad177si12267701plg.99.2023.08.02.20.50.58; Wed, 02 Aug 2023 20:51:12 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@joelfernandes.org header.s=google header.b=SSROfGyq; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233423AbjHCDY1 (ORCPT + 99 others); Wed, 2 Aug 2023 23:24:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50314 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233010AbjHCDYR (ORCPT ); Wed, 2 Aug 2023 23:24:17 -0400 Received: from mail-io1-xd30.google.com (mail-io1-xd30.google.com [IPv6:2607:f8b0:4864:20::d30]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9B8902D43 for ; Wed, 2 Aug 2023 20:24:14 -0700 (PDT) Received: by mail-io1-xd30.google.com with SMTP id ca18e2360f4ac-786bb09e595so41271139f.1 for ; Wed, 02 Aug 2023 20:24:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=joelfernandes.org; s=google; t=1691033053; x=1691637853; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=ri+AV8Kf+G5whax+bk9HTGfLE0fpaHCunlBX3LcI+MY=; b=SSROfGyqV9e3u74mTKmoa9swo8SFey1Lh0VpFbL1JE70n1rclCj8tC30xw4OMGN2vB +IlNzwUN30w4Nw4O0wOw4zYh/3Z5xy5EI6IPgTWrmkRvfYDNsjzL6uSm3a0h4mlH2GOM 10aW6NWacR4dbgGPHJmVb5bJhYKUm3LEhZML0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691033053; x=1691637853; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=ri+AV8Kf+G5whax+bk9HTGfLE0fpaHCunlBX3LcI+MY=; b=aL4VZ24SpnnHqIzHtOQMUL/QXd+Clah9+OfSHe47z9clgfVmIa9XBOo1DR7oi+GbEV T2O+iHy02smmzeOpOmG+414hmRy6w2P0y247+zVus8o95qvB03066P9dUar0Tf+gqe0O dDeyEz4AfNCRby9Q84gQzddnRG6i1Yy/LEdJnlNOok/mcqOwsOZIVIwsMGHTeL8II+82 Lem+gkIguwxtySUOWc1hsc9rxJOuu3zgCKE1gbj/jQMDJDrguGgYsgwjEDWWswpuqVtZ UJZCcD8ZiDQQdKXkh5a19eLVQvMzS+8cg3V5ILTYI+nV3mNL42fhql5pQqD95RozKEhH Y7rQ== X-Gm-Message-State: ABy/qLZwen31P2waKRCRfZfNrOW7m5K5pD3YU8VawgFa8Xp2CSvfeMj+ hri0lnwH8Sf7CtEbDjZD36JcWACbbziQWo9z+CQ= X-Received: by 2002:a05:6602:2559:b0:785:cac9:2d49 with SMTP id cg25-20020a056602255900b00785cac92d49mr14841576iob.1.1691033053197; Wed, 02 Aug 2023 20:24:13 -0700 (PDT) Received: from joelboxx5.c.googlers.com.com (156.190.123.34.bc.googleusercontent.com. [34.123.190.156]) by smtp.gmail.com with ESMTPSA id a14-20020a02ac0e000000b0042b67b12363sm4535176jao.37.2023.08.02.20.24.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Aug 2023 20:24:12 -0700 (PDT) From: "Joel Fernandes (Google)" To: linux-kernel@vger.kernel.org, rcu@vger.kernel.org Cc: "Joel Fernandes (Google)" , Will Deacon , "Paul E. McKenney" , Frederic Weisbecker , Neeraj Upadhyay , Josh Triplett , Boqun Feng , Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Jonathan Corbet Subject: [PATCH 1/2] docs: rcu: Add cautionary note on plain-accesses to requirements Date: Thu, 3 Aug 2023 03:24:06 +0000 Message-ID: <20230803032408.2514989-1-joel@joelfernandes.org> X-Mailer: git-send-email 2.41.0.585.gd2178a4bd4-goog MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1773178372802110114 X-GMAIL-MSGID: 1773178372802110114 Add a detailed note to explain the potential side effects of plain-accessing the gp pointer using a plain load, without using the rcu_dereference() macros; which might trip neighboring code that does use rcu_dereference(). I haven't verified this with a compiler, but this is what I gather from the below link using Will's experience with READ_ONCE(). Link: https://lore.kernel.org/all/20230728124412.GA21303@willie-the-truck/ Cc: Will Deacon Signed-off-by: Joel Fernandes (Google) --- .../RCU/Design/Requirements/Requirements.rst | 32 +++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/Documentation/RCU/Design/Requirements/Requirements.rst b/Documentation/RCU/Design/Requirements/Requirements.rst index f3b605285a87..e0b896d3fb9b 100644 --- a/Documentation/RCU/Design/Requirements/Requirements.rst +++ b/Documentation/RCU/Design/Requirements/Requirements.rst @@ -376,6 +376,38 @@ mechanism, most commonly locking or reference counting .. |high-quality implementation of C11 memory_order_consume [PDF]| replace:: high-quality implementation of C11 ``memory_order_consume`` [PDF] .. _high-quality implementation of C11 memory_order_consume [PDF]: http://www.rdrop.com/users/paulmck/RCU/consume.2015.07.13a.pdf +Note that, there can be strange side effects (due to compiler optimizations) if +``gp`` is ever accessed using a plain load (i.e. without ``READ_ONCE()`` or +``rcu_dereference()``) potentially hurting any succeeding +``rcu_dereference()``. For example, consider the code: + + :: + + 1 bool do_something_gp(void) + 2 { + 3 void *tmp; + 4 rcu_read_lock(); + 5 tmp = gp; // Plain-load of GP. + 6 printk("Point gp = %p\n", tmp); + 7 + 8 p = rcu_dereference(gp); + 9 if (p) { + 10 do_something(p->a, p->b); + 11 rcu_read_unlock(); + 12 return true; + 13 } + 14 rcu_read_unlock(); + 15 return false; + 16 } + +The behavior of plain accesses involved in a data race is non-deterministic in +the face of compiler optimizations. Since accesses to the ``gp`` pointer is +by-design a data race, the compiler could trip this code by caching the value +of ``gp`` into a register in line 5, and then using the value of the register +to satisfy the load in line 10. Thus it is important to never mix +plain accesses of a memory location with rcu_dereference() of the same memory +location, in code involved in a data race. + In short, updaters use rcu_assign_pointer() and readers use rcu_dereference(), and these two RCU API elements work together to ensure that readers have a consistent view of newly added data elements. From patchwork Thu Aug 3 03:24:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joel Fernandes X-Patchwork-Id: 130297 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9f41:0:b0:3e4:2afc:c1 with SMTP id v1csp890923vqx; Wed, 2 Aug 2023 20:51:04 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFJxWgIS4/+AEl3DM3N2xDbPty81yteeVmcle2NExy7Bx9hWl7Uz0UnoAzSNGMzhuEATpNn X-Received: by 2002:a05:6a00:1146:b0:687:8f50:47c8 with SMTP id b6-20020a056a00114600b006878f5047c8mr60074pfm.1.1691034664613; Wed, 02 Aug 2023 20:51:04 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1691034664; cv=none; d=google.com; s=arc-20160816; b=UChBK4WzSFKznIzCTQ0t8b5FOO4AgR9ADBEEAbu6IBIJHGaxyxTQlb+CTShDpHkXzQ 0dOV+jyVQjg38ox+ex0Ih/jNzqqcGgoU0YPUs3jYaGGZEJg0fLJXUsa562tofB5DmYh6 5zsrav9xSijs8s6jGAmmMXqNsoxYsZPWQpWTYSkC64fnHgvuXXBQNQm8JK0IOq41S2wb MuAARa4Kn21Sd7RWAV3jGIbgOT0tCN00G9OVmMG8kVwaY+qg+h9KFweV288z9XWJ0kN5 Yw6mFVOSe7pTcUYSCWC0ElA4K+vgrKbeg8jBa6bkZ8Kyg2Hb0rNQnbCcfm19yFL7tjgW YXWQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=VMHqujoTuT0RsHzRdVhgkyPvYcb4Kgnj2rbEthA3RMM=; fh=sf39n9bsV4nar1ajgOioZpKGkdoMgqPPMO5CkgmtHE4=; b=lnJi9eJILvFe8iuJ9jfwcEqfAWQAZ0OClR4wDWxCJeK6yzb7jd8nCx1ehcYlVtkLSJ zSxT9PG8iegMPcU3wmjZVpXaQlYNlYUv/p3ECyDjoyoV3o7SYsNOWaFo5ktusR0MT2ox yCqA8pu3SYvzM2IyldF6ih4aRrSwf+pSkq/HX9XFUoD1SI+ICsxdMrqNYsdjlJgHD8QU fwpMyMyUDAS5iaf+x1TGbB94eisv4cRhKJtWaQUXFuNgpTs19/ico3/LErYO+6tu8dLj upWqMdjfH9v1NmbtuRZcp+UY275n1UA+ldk+zq+VXkyvbts5T04cPkTIPwnIWxQbIdfx HL2Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@joelfernandes.org header.s=google header.b=jmOMCg3a; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id w7-20020a637b07000000b005643758e9c9si1008819pgc.515.2023.08.02.20.50.50; Wed, 02 Aug 2023 20:51:04 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@joelfernandes.org header.s=google header.b=jmOMCg3a; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233523AbjHCDYb (ORCPT + 99 others); Wed, 2 Aug 2023 23:24:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50266 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233194AbjHCDYS (ORCPT ); Wed, 2 Aug 2023 23:24:18 -0400 Received: from mail-yb1-xb35.google.com (mail-yb1-xb35.google.com [IPv6:2607:f8b0:4864:20::b35]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0305826B2 for ; Wed, 2 Aug 2023 20:24:16 -0700 (PDT) Received: by mail-yb1-xb35.google.com with SMTP id 3f1490d57ef6-bc379e4c1cbso488439276.2 for ; Wed, 02 Aug 2023 20:24:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=joelfernandes.org; s=google; t=1691033055; x=1691637855; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VMHqujoTuT0RsHzRdVhgkyPvYcb4Kgnj2rbEthA3RMM=; b=jmOMCg3aY1NdBWQ0E/xfB3w00lqf6WPH+O3dT/+EXyF79Cpvo08Pb1daWoPjBSPZT6 bqocWqfWMdrhrDRwxYzLemMLfNLaTPpNFlqm6gyTOU4N5ITwyrd3vSnmCCD0lcK8XGzR 5WD5DZc70Rq0qy8ulzg0neW+ZH1ryoSqFeF8A= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691033055; x=1691637855; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VMHqujoTuT0RsHzRdVhgkyPvYcb4Kgnj2rbEthA3RMM=; b=UOJXTidWeqLqqUy2WqXoPDiXuokPSipCU/pq63EYnLEw7pcNezNX87SAZAHpb0kR44 TcFDdjDh7B9TF7RKoRLFX1GVKjd+w7veyl2x6htFy/oPkxSt1icbY/TJoxV0lzfACvlD 0jwSlCJPTpGPNMt83qBjxI4TzzFx9HPcV10AEokTT0f0rCCRuVjWb0YSp2dO0fYeRlle xoCZpURvqXE0CZDEU4XyOxmcByh+Xz5HxNeeut+MgUKJCjWA0TgYnUI3oS+c2FVrWjSb 9ZHCQBehWIvEk6d2kmr1pCjHFUL3MB+NwNEwkNCBkYpRt79x6uAgWyJlkLpZoA2rmI3q 1HFw== X-Gm-Message-State: ABy/qLYAPWAK6/GSKgVyLzthoqVjEHjLSbuKwZmbN+rbmA78+5qFU7ko SQgeNDiVOhZ1K0aatCI7BZxYvELhM92wkEAKHNs= X-Received: by 2002:a25:d401:0:b0:d15:a265:4c43 with SMTP id m1-20020a25d401000000b00d15a2654c43mr18622880ybf.61.1691033054767; Wed, 02 Aug 2023 20:24:14 -0700 (PDT) Received: from joelboxx5.c.googlers.com.com (156.190.123.34.bc.googleusercontent.com. [34.123.190.156]) by smtp.gmail.com with ESMTPSA id a14-20020a02ac0e000000b0042b67b12363sm4535176jao.37.2023.08.02.20.24.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Aug 2023 20:24:13 -0700 (PDT) From: "Joel Fernandes (Google)" To: linux-kernel@vger.kernel.org, rcu@vger.kernel.org Cc: "Joel Fernandes (Google)" , Alan Stern , Andrea Parri , Will Deacon , Peter Zijlstra , Boqun Feng , Nicholas Piggin , David Howells , Jade Alglave , Luc Maranget , "Paul E. McKenney" , Akira Yokosawa , Daniel Lustig , Jonathan Corbet Subject: [PATCH 2/2] docs: memory-barriers: Add note on plain-accesses to address-dependency barriers Date: Thu, 3 Aug 2023 03:24:07 +0000 Message-ID: <20230803032408.2514989-2-joel@joelfernandes.org> X-Mailer: git-send-email 2.41.0.585.gd2178a4bd4-goog In-Reply-To: <20230803032408.2514989-1-joel@joelfernandes.org> References: <20230803032408.2514989-1-joel@joelfernandes.org> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1773178364751092999 X-GMAIL-MSGID: 1773178364751092999 The compiler has the ability to cause misordering by destroying address-dependency barriers if comparison operations are used. Add a note about this to memory-barriers.txt and point to rcu-dereference.rst for more information. Signed-off-by: Joel Fernandes (Google) Signed-off-by: Joel Fernandes (Google) --- Documentation/memory-barriers.txt | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/Documentation/memory-barriers.txt b/Documentation/memory-barriers.txt index 06e14efd8662..acc8ec5ce563 100644 --- a/Documentation/memory-barriers.txt +++ b/Documentation/memory-barriers.txt @@ -435,6 +435,11 @@ Memory barriers come in four basic varieties: variables such as READ_ONCE() and rcu_dereference() provide implicit address-dependency barriers. + [!] Note that address dependency barriers can be destroyed by comparison + of a pointer obtained by a marked accessor such as READ_ONCE() or + rcu_dereference() with some value. For an example of this, see + rcu_dereference.rst (part where the comparison of pointers is discussed). + (3) Read (or load) memory barriers. A read barrier is an address-dependency barrier plus a guarantee that all