From patchwork Tue Nov 22 20:39:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mathieu Desnoyers X-Patchwork-Id: 24558 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp2428812wrr; Tue, 22 Nov 2022 12:41:13 -0800 (PST) X-Google-Smtp-Source: AA0mqf4hFgPVbuAU+zBSlkdt5mfOqrI7jXVHZgtJn0pdsKLMWV0MlHuqKHr3a8MSG8O+ICFMruS5 X-Received: by 2002:a17:902:aa04:b0:17f:6fee:3334 with SMTP id be4-20020a170902aa0400b0017f6fee3334mr5613333plb.10.1669149673012; Tue, 22 Nov 2022 12:41:13 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669149673; cv=none; d=google.com; s=arc-20160816; b=Jq501E1eIcbVvTmTXFw0nlVt9th7sv5A0/GTvv6hni2an9WxdxMukJm3s5GZWXsfjz QEP7tly50nUR7pQik6H9uHwKBj0syzbwlP0GKoKc5YHJPEzEQ65J5K+Dw0I3BP+MRBBo si/zAkQl2qegqWZ0PhaPYXQuGw3HzZGEOvTjQY1QEPWDCxjec+cYWg78r7sV0mebLSu5 eVAo/3xL4iSSXe1mS66QoV0+JTNpTCnm2mDdm3gaJck0amQs+QUXWEjPf27kq2LO61Zz +3lw3sJRwUfnRhD6Q9QNdjjSqUdF38Ozu9J0MWen/IjZh7Fv9wOmsdAVPkStGEQ1xGgr I8iQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=7PFY5F8q1ZXuF/M3TnaNlVAm9pNM3f9fvCrufZljg50=; b=cOrgOH4Jzz6Us7qBPP2mkzsovHLMPh+eFuHiJM9i6h7JsxdpC7ATQGZGlgjOjUvZlx aJaUMpffqaxsnJXGug3MV9LDtkUUDOhjZ9h+8GHf0JeBEN9skUaXx6oVR0silepzgGZ6 Mir8H82YXEg/q63etaUlOgRqJwTSJQ4Uw+f8gH1N2Vr6S5htsiF24ilBlgM+2pPp3pd4 fK67JYUpA4otBH02c2fUZJWFkgamkmgjU3V812aZiJY3In7f0Gxtb78SyQD9+yRP4ZE8 3v2K+gDhjopgagNpiNR10C3llS9Co7miA25R/AwRNQ/5ihlt5Hz0n5+nmreOky9kZKRL XXHQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@efficios.com header.s=smtpout1 header.b=GU4jsJA1; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=efficios.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id p3-20020a170902e74300b00188a7401d89si16273968plf.481.2022.11.22.12.41.00; Tue, 22 Nov 2022 12:41:12 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@efficios.com header.s=smtpout1 header.b=GU4jsJA1; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=efficios.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234520AbiKVUkQ (ORCPT + 99 others); Tue, 22 Nov 2022 15:40:16 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45788 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234888AbiKVUjw (ORCPT ); Tue, 22 Nov 2022 15:39:52 -0500 Received: from smtpout.efficios.com (smtpout.efficios.com [167.114.26.122]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EB8C26B226; Tue, 22 Nov 2022 12:39:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=efficios.com; s=smtpout1; t=1669149587; bh=JDPm4e63VjlqNVUfq2Us1ERrgPE08F0+xxt5hQ6MHmc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GU4jsJA1aW/Pku/92SEKE2NunFRMrGbfJIsO1D+dT/p3eHVbnidUAjLOsCUjJf+Ua sReJwdr90pruQ+ZVbM3UmAFRuHvyF2dbFUOzq6reE8cxfVbf52gXmOJl7GNuALu7qe LoMdECMQjtr7zVqAmACXB9gv+zhcZRmqobb7m2oDNmWBZvzBKystmdxRQS6L3QwJ8c O8hc5cPsDaoxJ6dKwgT6AyBUWq6GGGzZ4J19CFnloUCCI+PVRz39GOm5Rwnd+5USyb klEzc9w75PI1I/3O7PmHubIXr6o/xdzl7COEHj6EcCCd436ya+PERm1ALHfug/5KhB ZAoGHiEDw8e2g== Received: from localhost.localdomain (192-222-180-24.qc.cable.ebox.net [192.222.180.24]) by smtpout.efficios.com (Postfix) with ESMTPSA id 4NGx2W2rswzXZd; Tue, 22 Nov 2022 15:39:47 -0500 (EST) From: Mathieu Desnoyers To: Peter Zijlstra Cc: linux-kernel@vger.kernel.org, Thomas Gleixner , "Paul E . McKenney" , Boqun Feng , "H . Peter Anvin" , Paul Turner , linux-api@vger.kernel.org, Christian Brauner , Florian Weimer , David.Laight@ACULAB.COM, carlos@redhat.com, Peter Oskolkov , Alexander Mikhalitsyn , Chris Kennelly , Mathieu Desnoyers Subject: [PATCH 08/30] rseq: Extend struct rseq with per-memory-map concurrency ID Date: Tue, 22 Nov 2022 15:39:10 -0500 Message-Id: <20221122203932.231377-9-mathieu.desnoyers@efficios.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221122203932.231377-1-mathieu.desnoyers@efficios.com> References: <20221122203932.231377-1-mathieu.desnoyers@efficios.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750230287559109756?= X-GMAIL-MSGID: =?utf-8?q?1750230287559109756?= If a memory map has fewer threads than there are cores on the system, or is limited to run on few cores concurrently through sched affinity or cgroup cpusets, the concurrency IDs will be values close to 0, thus allowing efficient use of user-space memory for per-cpu data structures. Signed-off-by: Mathieu Desnoyers --- include/uapi/linux/rseq.h | 9 +++++++++ kernel/rseq.c | 11 ++++++++++- 2 files changed, 19 insertions(+), 1 deletion(-) diff --git a/include/uapi/linux/rseq.h b/include/uapi/linux/rseq.h index 1cb90a435c5c..c233aae5eac9 100644 --- a/include/uapi/linux/rseq.h +++ b/include/uapi/linux/rseq.h @@ -139,6 +139,15 @@ struct rseq { */ __u32 node_id; + /* + * Restartable sequences mm_cid field. Updated by the kernel. Read by + * user-space with single-copy atomicity semantics. This field should + * only be read by the thread which registered this data structure. + * Aligned on 32-bit. Contains the current thread's concurrency ID + * (allocated uniquely within a memory map). + */ + __u32 mm_cid; + /* * Flexible array member at end of structure, after last feature field. */ diff --git a/kernel/rseq.c b/kernel/rseq.c index e21ad8929958..cb2512ab3256 100644 --- a/kernel/rseq.c +++ b/kernel/rseq.c @@ -90,12 +90,15 @@ static int rseq_update_cpu_node_id(struct task_struct *t) struct rseq __user *rseq = t->rseq; u32 cpu_id = raw_smp_processor_id(); u32 node_id = cpu_to_node(cpu_id); + u32 mm_cid = task_mm_cid(t); + WARN_ON_ONCE((int) mm_cid < 0); if (!user_write_access_begin(rseq, t->rseq_len)) goto efault; unsafe_put_user(cpu_id, &rseq->cpu_id_start, efault_end); unsafe_put_user(cpu_id, &rseq->cpu_id, efault_end); unsafe_put_user(node_id, &rseq->node_id, efault_end); + unsafe_put_user(mm_cid, &rseq->mm_cid, efault_end); /* * Additional feature fields added after ORIG_RSEQ_SIZE * need to be conditionally updated only if @@ -113,7 +116,8 @@ static int rseq_update_cpu_node_id(struct task_struct *t) static int rseq_reset_rseq_cpu_node_id(struct task_struct *t) { - u32 cpu_id_start = 0, cpu_id = RSEQ_CPU_ID_UNINITIALIZED, node_id = 0; + u32 cpu_id_start = 0, cpu_id = RSEQ_CPU_ID_UNINITIALIZED, node_id = 0, + mm_cid = 0; /* * Reset cpu_id_start to its initial state (0). @@ -132,6 +136,11 @@ static int rseq_reset_rseq_cpu_node_id(struct task_struct *t) */ if (put_user(node_id, &t->rseq->node_id)) return -EFAULT; + /* + * Reset mm_cid to its initial state (0). + */ + if (put_user(mm_cid, &t->rseq->mm_cid)) + return -EFAULT; /* * Additional feature fields added after ORIG_RSEQ_SIZE * need to be conditionally reset only if