From patchwork Fri Aug 11 17:00:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 134621 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b824:0:b0:3f2:4152:657d with SMTP id z4csp1256427vqi; Fri, 11 Aug 2023 10:45:11 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFf1L+REgQ4LTct4cA6MwrYHwJwUPrfs7G47od/jh0obovbkkwx3faHmSgAKPg5x3+Vql5p X-Received: by 2002:a17:902:ec87:b0:1b8:28f6:20e6 with SMTP id x7-20020a170902ec8700b001b828f620e6mr2564163plg.34.1691775911123; Fri, 11 Aug 2023 10:45:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1691775911; cv=none; d=google.com; s=arc-20160816; b=Dh4A7DPRknYR3QG8r7OighWAYtVtveZkcHSzxdASj9YVAdJ0HLOo3T6+HQL7Cy7cOK o9btsPzH7yS0wZ3FkvtG6lgnvZzHOi1p+2rmobVdsKtkmEDsDgTQXMHow2WWVVaebYo9 4qB5Y0GCoexkw6OQlDqGIlFL8SoRjS2cM5uUxam2fzKprFvW83AgjtDgirrde3h1b5j7 KHaIl4FSK2unX6tpZSaRHlv5LwqZmGgZe8wGKvvpAWpuF3ZI7peJVxpOgqKnqmF1nGbZ +xGbuG4k2hxoUJ+ZlA7tc8CAdmUFc74xfKtKflyAFSVc91CVtUysaLOAbZiRY73F5K48 cLmw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=5JObSVNGaua/SnWn/JYpLFzW0jWDYNCIn4fTiPYH/2A=; fh=N/KZXiibka9BfXj4Jcd+eHZJa0GpST73xFI6AQYLVnM=; b=wBfBLHAmJeZcVGv668L+nZKBp4gtGmQgVpz4wXqFUerL+F0/SZ0JFibi+YXuPey7O8 gADhzz5AtGfY2Ralg+oo4aLDOnk5VgM/SRiM5OdvwaR0gtyAUfgdsnULuZvIrIjVSIso Zf+a9qIHDRCj/vbrZTYiYqmkzV2ln5qs/anBYVLq2eSxB4BdgiR5yNvfYfuZ7J1PoTpm ea4gyj5K5emN7wfoc4T8MyN4aVzerBfcm+zxYSd1qp6JJtxmP6FZ7WHH5L4GKjl5XicF To2huS2HlpF9uwtAZoC1eY/d3T+JmSaDCTlIULIewv2VnC5Mdirm6hprwQpd2SYfBTwG Vn2w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=tadGZnDr; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id s20-20020a170903215400b001bd9c43c88csi3532637ple.204.2023.08.11.10.44.54; Fri, 11 Aug 2023 10:45:11 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=tadGZnDr; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235977AbjHKRBO (ORCPT + 99 others); Fri, 11 Aug 2023 13:01:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54954 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236102AbjHKRBB (ORCPT ); Fri, 11 Aug 2023 13:01:01 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D5E8C30CF for ; Fri, 11 Aug 2023 10:01:00 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 6A96766AFF for ; Fri, 11 Aug 2023 17:01:00 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D4903C433D9; Fri, 11 Aug 2023 17:00:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1691773259; bh=MSV7rGMv9w657y7ZDGytkmDXoScopLPoMt9dVwnOMVI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=tadGZnDr53sGuJqbcI/oAX/TnBDHDgTNARh3umzbcSSYYx8vGBQi5UTdJIr1Lf/TS 1OvVgkVCZNhAzSd9RBsapuBEIOPJmQJHyqd3JXprL9pqM5TJpxz2O14Yhi58dc80N9 tshmu2SFc3VnhW0Xo9DlquA9TuwAQ8Xhg4cs1Ur5bB0Mnsw7QeZ+MSUFHqKaPKkjW2 /4TMCoxxZPae4qVEocgu7fuy+283DWmA3hxOJHsiFocKvDFSmTzq37ZwcFvBfL9obY 7XLrXxi6VcZy9cQFJwieKQ5yVghepD5HlvNwZT/ZWN2RWEbbvXa+mNVQxvVQLlv4xx dc6mKunDYsHVA== From: Frederic Weisbecker To: LKML Cc: Peter Zijlstra , Frederic Weisbecker , "Rafael J . Wysocki" , Daniel Lezcano , Thomas Gleixner , Anna-Maria Behnsen , Jacob Pan , Len Brown Subject: [PATCH 02/10] cpuidle: Fix CPUIDLE_FLAG_IRQ_ENABLE leaking timer reprogram Date: Fri, 11 Aug 2023 19:00:41 +0200 Message-Id: <20230811170049.308866-3-frederic@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230811170049.308866-1-frederic@kernel.org> References: <20230811170049.308866-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1773955617645733097 X-GMAIL-MSGID: 1773955617645733097 From: Peter Zijlstra intel_idle_irq() re-enables IRQs very early. As a result, an interrupt may fire before mwait() is eventually called. If such an interrupt queues a timer, it may go unnoticed until mwait returns and the idle loop handles the tick re-evaluation. And monitoring TIF_NEED_RESCHED doesn't help because a local timer enqueue doesn't set that flag. The issue is mitigated by the fact that this idle handler is only invoked for shallow C-states when, presumably, the next tick is supposed to be close enough. There may still be rare cases though when the next tick is far away and the selected C-state is shallow, resulting in a timer getting ignored for a while. Fix this with using sti_mwait() whose IRQ-reenablement only triggers upon calling mwait(), dealing with the race while keeping the interrupt latency within acceptable bounds. Fixes: c227233ad64c (intel_idle: enable interrupts before C1 on Xeons) Not-yet-signed-off-by: Peter Zijlstra Signed-off-by: Frederic Weisbecker Acked-by: Rafael J. Wysocki --- arch/x86/include/asm/mwait.h | 11 +++++++++-- drivers/idle/intel_idle.c | 19 +++++++------------ 2 files changed, 16 insertions(+), 14 deletions(-) diff --git a/arch/x86/include/asm/mwait.h b/arch/x86/include/asm/mwait.h index 341ee4f1d91e..920426d691ce 100644 --- a/arch/x86/include/asm/mwait.h +++ b/arch/x86/include/asm/mwait.h @@ -124,8 +124,15 @@ static __always_inline void mwait_idle_with_hints(unsigned long eax, unsigned lo } __monitor((void *)¤t_thread_info()->flags, 0, 0); - if (!need_resched()) - __mwait(eax, ecx); + + if (!need_resched()) { + if (ecx & 1) { + __mwait(eax, ecx); + } else { + __sti_mwait(eax, ecx); + raw_local_irq_disable(); + } + } } current_clr_polling(); } diff --git a/drivers/idle/intel_idle.c b/drivers/idle/intel_idle.c index 256c2d42e350..d676d32741da 100644 --- a/drivers/idle/intel_idle.c +++ b/drivers/idle/intel_idle.c @@ -131,11 +131,12 @@ static unsigned int mwait_substates __initdata; #define MWAIT2flg(eax) ((eax & 0xFF) << 24) static __always_inline int __intel_idle(struct cpuidle_device *dev, - struct cpuidle_driver *drv, int index) + struct cpuidle_driver *drv, + int index, bool irqoff) { struct cpuidle_state *state = &drv->states[index]; unsigned long eax = flg2MWAIT(state->flags); - unsigned long ecx = 1; /* break on interrupt flag */ + unsigned long ecx = 1*irqoff; /* break on interrupt flag */ mwait_idle_with_hints(eax, ecx); @@ -159,19 +160,13 @@ static __always_inline int __intel_idle(struct cpuidle_device *dev, static __cpuidle int intel_idle(struct cpuidle_device *dev, struct cpuidle_driver *drv, int index) { - return __intel_idle(dev, drv, index); + return __intel_idle(dev, drv, index, true); } static __cpuidle int intel_idle_irq(struct cpuidle_device *dev, struct cpuidle_driver *drv, int index) { - int ret; - - raw_local_irq_enable(); - ret = __intel_idle(dev, drv, index); - raw_local_irq_disable(); - - return ret; + return __intel_idle(dev, drv, index, false); } static __cpuidle int intel_idle_ibrs(struct cpuidle_device *dev, @@ -184,7 +179,7 @@ static __cpuidle int intel_idle_ibrs(struct cpuidle_device *dev, if (smt_active) native_wrmsrl(MSR_IA32_SPEC_CTRL, 0); - ret = __intel_idle(dev, drv, index); + ret = __intel_idle(dev, drv, index, true); if (smt_active) native_wrmsrl(MSR_IA32_SPEC_CTRL, spec_ctrl); @@ -196,7 +191,7 @@ static __cpuidle int intel_idle_xstate(struct cpuidle_device *dev, struct cpuidle_driver *drv, int index) { fpu_idle_fpregs(); - return __intel_idle(dev, drv, index); + return __intel_idle(dev, drv, index, true); } /**