Message ID | 20230202215625.3248306-5-usama.arif@bytedance.com |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp494157wrn; Thu, 2 Feb 2023 13:58:50 -0800 (PST) X-Google-Smtp-Source: AK7set8dSbgaR+tOaiOjtpnVlUbKgul8gvSCBuGsbj8+bJ82uMP1mlEeTcgIhJLqEW2ouykd9qhk X-Received: by 2002:a17:906:5054:b0:878:7eb7:3488 with SMTP id e20-20020a170906505400b008787eb73488mr7265612ejk.21.1675375129969; Thu, 02 Feb 2023 13:58:49 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675375129; cv=none; d=google.com; s=arc-20160816; b=FehDgh/aD+rxD1JRWUsdiPb+HGmkkrDU0e22hoysrUnGpxk9sOAa9GLhD8/T7zOy/o T7RPuNWvew3TFQTztCaYFwO54OIWsU+wtP5aitGNMchDszhmKVlNigHkWTE9S6MkXxnU PH1166saCuGO+bEdj8ns9HEarGheS4PeimwUIL+nx/uGRTR0Dk49w8ncYA6Dr0CAZS8p iohB1V0Ksbe57+Deth+60scPSFtWylWy70DhtFKhl5mX22E1UrNtQzhrfIzwaYavvKE8 YR6OIKhbjW7zFwMbpR5rLAaYZjuejSsHr/vN0sKFC50CTc9T/KC6KG+fRqj8o7I6451p /6HQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=1t64Rjr6YniyNyrn83Uc/qtpVNvig4MY0wcaZ4A3opU=; b=Nt72Awi+jLuK0BEfE5ziTwi6JnMuKf8dK5kHfRSSxa4ZY18i2fXx3rCF+N6jHpxImF r/jMABXGGd63SBG7PofzuBe0ezW/TIDkCQirALNeWwoxIq7muBkKg+Y2V5WT7g7jrbw2 ONZphfVVHxc+dclZUqyRp7Sw4KsV6eRco+VwU3wAgmhC0UqlpJA2hITYiSDEihSCGW9I 04zHAG262nYtWxkSTh6julNWOeTbqbSWzi4WxxK5GnTeSZi27ByRgjYbQRMUx7FR9TKJ bI9UAJzjchNTkLYsH9/IlzmWHFd2JJS3iMfjT2+CWXoUTy/+Y8blHgD+Pq1nPYbKEx8L GrfQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b="XUy0x/p5"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=bytedance.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id o13-20020a17090611cd00b0088c0fd7cebcsi871509eja.806.2023.02.02.13.58.25; Thu, 02 Feb 2023 13:58:49 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b="XUy0x/p5"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=bytedance.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233136AbjBBV5Z (ORCPT <rfc822;il.mystafa@gmail.com> + 99 others); Thu, 2 Feb 2023 16:57:25 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54996 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232908AbjBBV5L (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Thu, 2 Feb 2023 16:57:11 -0500 Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7ECBB70D6E for <linux-kernel@vger.kernel.org>; Thu, 2 Feb 2023 13:56:35 -0800 (PST) Received: by mail-wr1-x434.google.com with SMTP id r2so2987058wrv.7 for <linux-kernel@vger.kernel.org>; Thu, 02 Feb 2023 13:56:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1t64Rjr6YniyNyrn83Uc/qtpVNvig4MY0wcaZ4A3opU=; b=XUy0x/p5QdV4qoA/2HTEBNCjHnhO7mdM58DCWfmzoZFW5rDKNboAfZ6LAtzKpomG55 eM0FkVsak3pwWr+bZ1NlxDE5LtDtZ6RVEH2qyvRxBe9aSM+bOCJ+ERXyo7ZLACdgGJH6 QAEGDDoSkk9+UDVcm3SrAo4Gl8UrrPERgu+cOQIQ1w00v8K19tm2ECCUullLs9vKvgEG QFPt14rCMgFr7zH5aIQmbVtfcb31sL+nARVUnFY14ZcOJufopoIACFr/WrOU+NSItQkT odm+Y9mrXFoFGiJglBNIH7vaG2MUNECtaO4+JYyT25LuQ0Uqy2JJcpOeFJa828ZmtNqY TnQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1t64Rjr6YniyNyrn83Uc/qtpVNvig4MY0wcaZ4A3opU=; b=7NpQvRQ56OlmcH5wOYvheQHkZxwyD57rsFlDbJWK5cmkw+EN2k0LOEC8JSblzGAxXf bdxnozlD4meyyP3XbIKv62AUmc1umOrcZQsR/nIydxJaBONdegHsAFQjExfpJUNA9aww Opq6jXOHQQrRh2DP11eNBmZN0KuevS1ZRWjKMSusivbKRPofgaJ1U9P1jZvwEEdoxoBo kz4Oq+zNVewEO8hQrmd5M8g+/Y+hxR/+vsJrjvbTiQOtYIgyClaunheyQ2zpOJcOQcUI MiprB++Ob5t2YGiuEnuYUL9KgpjKkSkFrE784jGOtXXhO8kgB4wVjs2350ZF+HDdNWdj 5YsQ== X-Gm-Message-State: AO0yUKXheDC5H8EAPYSHV9JNW/B/ZUMtp7+bHti/rX61T1RNIdKrDx1Y rL6bA7ndsekz4YaSZOZvBF2GlA== X-Received: by 2002:adf:fa84:0:b0:2bf:e449:e72a with SMTP id h4-20020adffa84000000b002bfe449e72amr6584290wrr.60.1675374994014; Thu, 02 Feb 2023 13:56:34 -0800 (PST) Received: from usaari01.cust.communityfibre.co.uk ([2a02:6b6a:b566:0:98fe:e4ee:fc7e:cd71]) by smtp.gmail.com with ESMTPSA id e8-20020a5d6d08000000b00297dcfdc90fsm506078wrq.24.2023.02.02.13.56.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Feb 2023 13:56:33 -0800 (PST) From: Usama Arif <usama.arif@bytedance.com> To: dwmw2@infradead.org, tglx@linutronix.de, arjan@linux.intel.com Cc: mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, x86@kernel.org, pbonzini@redhat.com, paulmck@kernel.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, rcu@vger.kernel.org, mimoja@mimoja.de, hewenliang4@huawei.com, thomas.lendacky@amd.com, seanjc@google.com, pmenzel@molgen.mpg.de, fam.zheng@bytedance.com, punit.agrawal@bytedance.com, simon.evans@bytedance.com, liangma@liangbit.com, David Woodhouse <dwmw@amazon.co.uk>, Usama Arif <usama.arif@bytedance.com> Subject: [PATCH v6 04/11] x86/smpboot: Reference count on smpboot_setup_warm_reset_vector() Date: Thu, 2 Feb 2023 21:56:18 +0000 Message-Id: <20230202215625.3248306-5-usama.arif@bytedance.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230202215625.3248306-1-usama.arif@bytedance.com> References: <20230202215625.3248306-1-usama.arif@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756758151916545453?= X-GMAIL-MSGID: =?utf-8?q?1756758151916545453?= |
Series |
Parallel CPU bringup for x86_64
|
|
Commit Message
Usama Arif
Feb. 2, 2023, 9:56 p.m. UTC
From: David Woodhouse <dwmw@amazon.co.uk> If we want to do parallel CPU bringup, we're going to need to set this up and leave it until all CPUs are done. Might as well use the RTC spinlock to protect the refcount, as we need to take it anyway. [Usama Arif: fixed rebase conflict] Signed-off-by: David Woodhouse <dwmw@amazon.co.uk> Signed-off-by: Usama Arif <usama.arif@bytedance.com> Signed-off-by: Paul E. McKenney <paulmck@kernel.org> --- arch/x86/kernel/smpboot.c | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-)
Comments
On Thu, Feb 02 2023 at 21:56, Usama Arif wrote: > From: David Woodhouse <dwmw@amazon.co.uk> > > If we want to do parallel CPU bringup, we're going to need to set this up > and leave it until all CPUs are done. Might as well use the RTC spinlock > to protect the refcount, as we need to take it anyway. https://www.kernel.org/doc/html/latest/process/maintainer-tip.html#changelog Aside of the 'We' this does not explain anything at all. > [Usama Arif: fixed rebase conflict] > Signed-off-by: David Woodhouse <dwmw@amazon.co.uk> > Signed-off-by: Usama Arif <usama.arif@bytedance.com> > Signed-off-by: Paul E. McKenney <paulmck@kernel.org> This SOB chain is even more broken... > --- > arch/x86/kernel/smpboot.c | 21 ++++++++++++++------- > 1 file changed, 14 insertions(+), 7 deletions(-) > > diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c > index 55cad72715d9..a19eddcdccc2 100644 > --- a/arch/x86/kernel/smpboot.c > +++ b/arch/x86/kernel/smpboot.c > @@ -121,17 +121,22 @@ int arch_update_cpu_topology(void) > return retval; > } > > + > +static unsigned int smpboot_warm_reset_vector_count; > + > static inline void smpboot_setup_warm_reset_vector(unsigned long start_eip) > { > unsigned long flags; > > spin_lock_irqsave(&rtc_lock, flags); > - CMOS_WRITE(0xa, 0xf); > + if (!smpboot_warm_reset_vector_count++) { > + CMOS_WRITE(0xa, 0xf); > + *((volatile unsigned short *)phys_to_virt(TRAMPOLINE_PHYS_HIGH)) = > + start_eip >> 4; > + *((volatile unsigned short *)phys_to_virt(TRAMPOLINE_PHYS_LOW)) = > + start_eip & 0xf; Again: 80 characters are history. Please fix that all over the series. Thanks, tglx
David! On Tue, Feb 07 2023 at 09:49, David Woodhouse wrote: Can you please fix your mail client to _NOT_ send multipart/mixed mails? Despite the CC list being insanely large, your replies are dropped by vger and missing in the archives. > On Tue, 2023-02-07 at 00:48 +0100, Thomas Gleixner wrote: >> On Thu, Feb 02 2023 at 21:56, Usama Arif wrote: >> > From: David Woodhouse <dwmw@amazon.co.uk> >> > >> > If we want to do parallel CPU bringup, we're going to need to set this up >> > and leave it until all CPUs are done. Might as well use the RTC spinlock >> > to protect the refcount, as we need to take it anyway. >> >> https://www.kernel.org/doc/html/latest/process/maintainer-tip.html#changelog >> >> Aside of the 'We' this does not explain anything at all. > > Er, doesn't it? > > We refcount the warm reset vector because when we do parallel bringup, > we'll want to set it up once and then put it back to normal (for cold > reset) once all the CPUs are up. > > I can rework the phrasing; I'm aware that the whole nonsense about > pronouns and the imperative mood has grown legs in the last couple of > years since I originally wrote it — but is there anything actually > missing? We can settle the imperative mood debate over a beer at the next conference, but stuff which goes through tip is required to follow those rules. No exception for you :) Vs. the content: This changelog lacks context. Changelogs have to be self contained and self explanatory. Assuming that they are understandable due to the context of the patch series is just wrong. I fundamentally hate it when I have to dig out the context when I stare at the changelog of a commit. So something like this: The warm reset vector on X86 is setup through the RTC (CMOS) clock for each CPU bringup operation and cleared after the CPU came online. Parallel bringup of multiple CPUs requires that the warm reset vector is valid until all CPUs came online. To prepare for that add refcounting for the reset vector and protect it with the rtc_lock which has to be taken for the setup operation anyway. gives the full context and is simply factual, no? Thanks, tglx
On Tue, Feb 07, 2023, Thomas Gleixner wrote: > On Tue, Feb 07 2023 at 09:49, David Woodhouse wrote: > > On Tue, 2023-02-07 at 00:48 +0100, Thomas Gleixner wrote: > >> Aside of the 'We' this does not explain anything at all. > > > > Er, doesn't it? > > > > We refcount the warm reset vector because when we do parallel bringup, > > we'll want to set it up once and then put it back to normal (for cold > > reset) once all the CPUs are up. > > > > I can rework the phrasing; I'm aware that the whole nonsense about > > pronouns and the imperative mood has grown legs in the last couple of > > years since I originally wrote it — but is there anything actually > > missing? > > We can settle the imperative mood debate over a beer at the next > conference, but stuff which goes through tip is required to follow those > rules. No exception for you :) While we're reforming David, same goes for KVM x86. :-)
On Tue, 2023-02-07 at 15:39 +0100, Thomas Gleixner wrote: > David! > > On Tue, Feb 07 2023 at 09:49, David Woodhouse wrote: > > Can you please fix your mail client to _NOT_ send multipart/mixed mails? > Despite the CC list being insanely large, your replies are dropped by > vger and missing in the archives. > That's not the client; that's the stupid mail system doing it in transit. Sorry, I'd already filed a ticket about that idiocy last week when I noticed they'd started adding HTML parts to a previously sane mail. But obviously they haven't managed to fix it yet. The correct thing to do in the meantime is use a non-broken mail account, and I just forgot this morning until half way through the thread, when you'll note the coffee kicked in and I switched. > > On Tue, 2023-02-07 at 00:48 +0100, Thomas Gleixner wrote: > > > On Thu, Feb 02 2023 at 21:56, Usama Arif wrote: > > > > From: David Woodhouse <dwmw@amazon.co.uk> > > > > > > > > If we want to do parallel CPU bringup, we're going to need to set this up > > > > and leave it until all CPUs are done. Might as well use the RTC spinlock > > > > to protect the refcount, as we need to take it anyway. > > > > > > https://www.kernel.org/doc/html/latest/process/maintainer-tip.html#changelog > > > > > > Aside of the 'We' this does not explain anything at all. > > > > Er, doesn't it? > > > > We refcount the warm reset vector because when we do parallel bringup, > > we'll want to set it up once and then put it back to normal (for cold > > reset) once all the CPUs are up. > > > > I can rework the phrasing; I'm aware that the whole nonsense about > > pronouns and the imperative mood has grown legs in the last couple of > > years since I originally wrote it — but is there anything actually > > missing? > > We can settle the imperative mood debate over a beer at the next > conference, but stuff which goes through tip is required to follow those > rules. No exception for you :) > > Vs. the content: This changelog lacks context. Changelogs have to be > self contained and self explanatory. Assuming that they are > understandable due to the context of the patch series is just wrong. I > fundamentally hate it when I have to dig out the context when I stare at > the changelog of a commit. > > So something like this: > > The warm reset vector on X86 is setup through the RTC (CMOS) clock > for each CPU bringup operation and cleared after the CPU came online. > > Parallel bringup of multiple CPUs requires that the warm reset vector > is valid until all CPUs came online. > > To prepare for that add refcounting for the reset vector and protect > it with the rtc_lock which has to be taken for the setup operation > anyway. > > gives the full context and is simply factual, no? Ack.
diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c index 55cad72715d9..a19eddcdccc2 100644 --- a/arch/x86/kernel/smpboot.c +++ b/arch/x86/kernel/smpboot.c @@ -121,17 +121,22 @@ int arch_update_cpu_topology(void) return retval; } + +static unsigned int smpboot_warm_reset_vector_count; + static inline void smpboot_setup_warm_reset_vector(unsigned long start_eip) { unsigned long flags; spin_lock_irqsave(&rtc_lock, flags); - CMOS_WRITE(0xa, 0xf); + if (!smpboot_warm_reset_vector_count++) { + CMOS_WRITE(0xa, 0xf); + *((volatile unsigned short *)phys_to_virt(TRAMPOLINE_PHYS_HIGH)) = + start_eip >> 4; + *((volatile unsigned short *)phys_to_virt(TRAMPOLINE_PHYS_LOW)) = + start_eip & 0xf; + } spin_unlock_irqrestore(&rtc_lock, flags); - *((volatile unsigned short *)phys_to_virt(TRAMPOLINE_PHYS_HIGH)) = - start_eip >> 4; - *((volatile unsigned short *)phys_to_virt(TRAMPOLINE_PHYS_LOW)) = - start_eip & 0xf; } static inline void smpboot_restore_warm_reset_vector(void) @@ -143,10 +148,12 @@ static inline void smpboot_restore_warm_reset_vector(void) * to default values. */ spin_lock_irqsave(&rtc_lock, flags); - CMOS_WRITE(0, 0xf); + if (!--smpboot_warm_reset_vector_count) { + CMOS_WRITE(0, 0xf); + *((volatile u32 *)phys_to_virt(TRAMPOLINE_PHYS_LOW)) = 0; + } spin_unlock_irqrestore(&rtc_lock, flags); - *((volatile u32 *)phys_to_virt(TRAMPOLINE_PHYS_LOW)) = 0; } /*