Message ID | 20230414162755.281993820@linutronix.de |
---|---|
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp511913vqo; Fri, 14 Apr 2023 09:38:08 -0700 (PDT) X-Google-Smtp-Source: AKy350aVIAMdbyXW4nJ4faQEMlp+PqjWFS0c4+7CY167HMp2j/zSPxTh7YMmsWPEFIZ9oz0sxCZF X-Received: by 2002:a17:902:e20b:b0:1a6:abac:9cc with SMTP id u11-20020a170902e20b00b001a6abac09ccmr796341plb.66.1681490288413; Fri, 14 Apr 2023 09:38:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681490288; cv=none; d=google.com; s=arc-20160816; b=lYYMlTvKHsJO9aUoGDBykyAFvH462kksXEfvmlIYWjGxNX5HnrbgqMf6/IdghYED4z mY49Ya9sQHXiKEbyT9MU0yE8ch20fe6lbQq4hipbQ3dDXo5ztth5FBl+M5SWvQArlbwg rrDsECHBE2T6VDf1bnl4edFMKDKoa9PMT9cRdqg9lD8eGI2qQngqVmGF+v5D8Pl/fqZs sVtiZB76OHaQk4B7qe+n2LbYgn0Kv1S+Nv8JxqWiHlREORitL4K53D9XndrgQnSvP+lM 6teQefIV+XyTWky9Orr7/9F/lmYkqEElyFFzUfht7Lyg2NffzZnem+bSrbNIlAB5J+lt ySjg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:date:subject:cc:to:from:dkim-signature :dkim-signature:message-id; bh=c+unp6injjwLp32rDRhFKdS15pYib3N33LpCMho7UDw=; b=RCHIZNYi/qTilmld2nj4hKA6Ep6N7Z7wDAqGLLI41I69XFlF3pHm/CTr3F/665ZtN/ h2G5jRqZzipYQ9qdFWcKcWJ1BI9ExMn/0GoINezWlsDox3V9MtUfWfkWEg7nJJxmvyqB izFv3PbKV7GAH0zAv+C58v/YYWXcYXjgVzNmIsSEdFHuMJSPtAWQNSztZueJfF+//UO1 Xd7GGZzfeQgRQOeyKLlfYGt0+4XXUTcjwPh/UF2GzvGun49TjMdpQxIBJ8oqa0UNSOO1 DqiZbqjqd52GLvKGUvC8JVdpTinB8vys09IM9hPYuMXBnLbFJU0sZpUuOw6qxf4rPkLo OUfg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=T8SDQEIl; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id e18-20020a17090301d200b001a534e0e863si5487607plh.63.2023.04.14.09.37.53; Fri, 14 Apr 2023 09:38:08 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=T8SDQEIl; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229705AbjDNQas (ORCPT <rfc822;leviz.kernel.dev@gmail.com> + 99 others); Fri, 14 Apr 2023 12:30:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37216 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229493AbjDNQaq (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Fri, 14 Apr 2023 12:30:46 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 525398A62 for <linux-kernel@vger.kernel.org>; Fri, 14 Apr 2023 09:30:45 -0700 (PDT) Message-ID: <20230414162755.281993820@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1681489843; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc; bh=c+unp6injjwLp32rDRhFKdS15pYib3N33LpCMho7UDw=; b=T8SDQEIlUJAsUrnY50V68zec8cQsBn3NrlF9A9FDxDlkD22dBhjIx3mMXVQdg1VvSvrbFh 32+WnQIsXxLhEasQ0D2xSrGtMIRLeFc7Wm9qMDGcqifNCEj/We9F2QvsmsOLtDkjBHvazP FlT1ZGceUGsMOvbBDq8+0pb8yhzHlTEDLN9vTg2cJ2S6fb6Ddau6rxAyJaS8XXyvWA/oX7 OiOLI1s44qTn56vp0AMFdMkegqEIoIbPOqcoFRGSDoI24PDRlEcAyaqt3uJ5pV8jWVH+wE Ap5Pw61kRrU2Jvv4BJktNSSFDJJkLTCMs8KCNMjVQJXaJeQThzu8q1IiIg6vxg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1681489843; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc; bh=c+unp6injjwLp32rDRhFKdS15pYib3N33LpCMho7UDw=; b=tLLNUNn2IybA12zVsUJc/Wcbt3VorXkST75UUn9vq56G0ASHyaO6NmAx/tpBs4WP3+GVXe BZr11lVUYB0Yd5AQ== From: Thomas Gleixner <tglx@linutronix.de> To: LKML <linux-kernel@vger.kernel.org> Cc: Peter Zijlstra <peterz@infradead.org>, Valentin Schneider <vschneid@redhat.com>, Dennis Zhou <dennis@kernel.org>, Tejun Heo <tj@kernel.org>, Christoph Lameter <cl@linux.com>, Dave Chinner <dchinner@redhat.com>, Yury Norov <yury.norov@gmail.com>, Andy Shevchenko <andriy.shevchenko@linux.intel.com>, Rasmus Villemoes <linux@rasmusvillemoes.dk>, Ye Bin <yebin10@huawei.com>, linux-mm@kvack.org Subject: [patch 0/3] lib/percpu_counter, cpu/hotplug: Cure the cpu_dying_mask woes Date: Fri, 14 Apr 2023 18:30:42 +0200 (CEST) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1763170360534423092?= X-GMAIL-MSGID: =?utf-8?q?1763170360534423092?= |
Series |
lib/percpu_counter, cpu/hotplug: Cure the cpu_dying_mask woes
|
|
Message
Thomas Gleixner
April 14, 2023, 4:30 p.m. UTC
Hi! The cpu_dying_mask is not only undocumented but also to some extent a misnomer. It's purpose is to capture the last direction of a cpu_up() or cpu_down() operation taking eventual rollback operations into account. cpu_dying mask is not really useful for general consumption. The cpu_dying_mask bits are sticky even after cpu_up() or cpu_down() completes. A recent fix to plug a race in the per CPU counter code picked cpu_dying_mask to cure it. Unfortunately this does not work as the author probably expected and the behaviour of cpu_dying_mask is not easy to change without breaking the only other and initial user, the scheduler. This series addresses this by: 1) Reworking the per CPU counter hotplug mechanism so the race is fully plugged without using cpu_dying_mask 2) Replacing the cpu_dying_mask logic with hotplug core internal state which is exposed to the scheduler with a properly documented function. The series is also available from git: git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git smp/dying_mask Thanks tglx --- include/linux/cpuhotplug.h | 2 - include/linux/cpumask.h | 21 ---------------- kernel/cpu.c | 45 +++++++++++++++++++++++++++++------ kernel/sched/core.c | 4 +-- kernel/smpboot.h | 2 + lib/percpu_counter.c | 57 +++++++++++++++++++-------------------------- 6 files changed, 67 insertions(+), 64 deletions(-)
Comments
On 14/04/23 18:30, Thomas Gleixner wrote: > Hi! > > The cpu_dying_mask is not only undocumented but also to some extent a > misnomer. It's purpose is to capture the last direction of a cpu_up() or > cpu_down() operation taking eventual rollback operations into account. > > cpu_dying mask is not really useful for general consumption. The > cpu_dying_mask bits are sticky even after cpu_up() or cpu_down() completes. > > A recent fix to plug a race in the per CPU counter code picked > cpu_dying_mask to cure it. Unfortunately this does not work as the author > probably expected and the behaviour of cpu_dying_mask is not easy to change > without breaking the only other and initial user, the scheduler. > > This series addresses this by: > > 1) Reworking the per CPU counter hotplug mechanism so the race is fully > plugged without using cpu_dying_mask > > 2) Replacing the cpu_dying_mask logic with hotplug core internal state > which is exposed to the scheduler with a properly documented > function. > For patches 2-3: Reviewed-by: Valentin Schneider <vschneid@redhat.com>
Hello, On Fri, Apr 14, 2023 at 06:30:42PM +0200, Thomas Gleixner wrote: > Hi! > > The cpu_dying_mask is not only undocumented but also to some extent a > misnomer. It's purpose is to capture the last direction of a cpu_up() or > cpu_down() operation taking eventual rollback operations into account. > > cpu_dying mask is not really useful for general consumption. The > cpu_dying_mask bits are sticky even after cpu_up() or cpu_down() completes. > > A recent fix to plug a race in the per CPU counter code picked > cpu_dying_mask to cure it. Unfortunately this does not work as the author > probably expected and the behaviour of cpu_dying_mask is not easy to change > without breaking the only other and initial user, the scheduler. > > This series addresses this by: > > 1) Reworking the per CPU counter hotplug mechanism so the race is fully > plugged without using cpu_dying_mask > > 2) Replacing the cpu_dying_mask logic with hotplug core internal state > which is exposed to the scheduler with a properly documented > function. > > The series is also available from git: > > git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git smp/dying_mask > > Thanks > > tglx > --- > include/linux/cpuhotplug.h | 2 - > include/linux/cpumask.h | 21 ---------------- > kernel/cpu.c | 45 +++++++++++++++++++++++++++++------ > kernel/sched/core.c | 4 +-- > kernel/smpboot.h | 2 + > lib/percpu_counter.c | 57 +++++++++++++++++++-------------------------- > 6 files changed, 67 insertions(+), 64 deletions(-) This has been on my mind and regretfully it's been a busy year for me. I know the merge window is around the corner, but I rebased this series onto percpu#for-6.8 [1]. I had to massage percpu_counter slightly due to some changes but other than that it largely is intact. I need to do a little bit of a more thorough pass and re-send it out, but I think it remains correct to merge. I can then pull it, give it a few days to soak in for-next and then send it to Linus either in a follow up PR or in the 2nd week of the merge window. Thomas, how does this sound to you? [1] https://git.kernel.org/pub/scm/linux/kernel/git/dennis/percpu.git/log/?h=percpu-hotplug Thanks, Dennis