From patchwork Mon Apr 10 08:14:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 81390 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1757028vqo; Mon, 10 Apr 2023 01:41:52 -0700 (PDT) X-Google-Smtp-Source: AKy350befmQhhrG8YE16yoUZO+DxISZiMjhXVo7CO8z2k0omM/BQkWZHM/Vs98w1xToLa8tdRyHf X-Received: by 2002:a05:6a20:6685:b0:da:2d16:db89 with SMTP id o5-20020a056a20668500b000da2d16db89mr9878383pzh.28.1681116112045; Mon, 10 Apr 2023 01:41:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681116112; cv=none; d=google.com; s=arc-20160816; b=iaYAqw9TtvvYRBh3ykMuO9Y34XXyzH/uSyFbqapDikzOmeevuiSLDAu94URBq90oI6 PH/jBQd6DAqI2GwLdIZBas8nubOJsX2BCxunbx92pDGhWWy3UBdLGHvC43JvA32wvOtr GvJnj3S8yb7TgZwNlH0okri2LdYdL7c0g9+/WtQMtxvxQgQ+2Ze6RnMO31fgchGZshtn QjVF0moQzeM7fybX/vsb1Q8DOQC0nWgsRyM6/JE4gmqT9GFtqBV7x7H3sFs6AgipFA8F +felNy1zZNb0376H84dW8X4a7Fbgax+KfA1lOIk5zjJVYyKyeEjah6ikKHyOFLSjfQYi AGeA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=lz0rIzgfQ3UdI4kxn19lGElU0pAx4EZmyexpGqpeLQk=; b=JPqkcXupZMdJZUR1T1tnxz0aW6pceqoUb8xsunGpTUkt3xtaEzBNTAibTj3uldl9c4 8Rm7yXjJBXfNUYMne47irsvySx+dY4NUiiMmCxbB/AirqWPAFZ1gE2v9QALh946r5k5M rQrnBoq+95L0zvvhT53zZb21kQ0bMqAb0Vh+qZHWgftXCsr/oYfrl5lunv+0PmbEGPoH MPKzEPwuZiBL3HT0tge1KYLU9jMq94vBOyyNlDdychBQY4d45xMEa+nNl1vSh3OaN/e6 tK8B21OdXz4RsRd44jBxioJT2+V+FwU4QFIIByFeTxsxy4ogtv2OtWQjB9A7JqcTW7/j rM8g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=CINHGdc5; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id r20-20020a6560d4000000b0050c0305bcc3si9455111pgv.872.2023.04.10.01.41.39; Mon, 10 Apr 2023 01:41:52 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=CINHGdc5; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229725AbjDJIlI (ORCPT + 99 others); Mon, 10 Apr 2023 04:41:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49306 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229600AbjDJIlD (ORCPT ); Mon, 10 Apr 2023 04:41:03 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4AD693AB1; Mon, 10 Apr 2023 01:41:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1681116063; x=1712652063; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=CJ5cn83oehVUqkHaIiQkGH139f3gJuf37y85qUnwXYY=; b=CINHGdc5pozpD9mjU9lrfw+0Q4CL1gs6hsYyZSzlyLvql2bTYESlR0wG /svBlAF/3xakfcJRwUU/Tud/QImrQxEAHMkGehWJg3JOoslBdHw4D7bo4 msrOVEKc+qWvtkZuj2UZYLjJ6b+ucVUIRU/rp0sSbyExD2v/PqgOUdC/U bV2cbzxZKLKd/I+krss/mQIC0GXqLVCD8oCWNwAn/bebGbeCCMVsUVq6M LZ0dH0A9zvrYDIVXxFLKuE2Mqdm7NFX09Te9dLQa7RLipolrHrGtvhTkz 3WZXkV0JNRwTMOUBRCxSbySDxCkcfVJfg6nTPy+Sx4BkBi/ygesYYwi8M A==; X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="342077871" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="342077871" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2023 01:41:01 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="799436238" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="799436238" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga002.fm.intel.com with ESMTP; 10 Apr 2023 01:41:01 -0700 From: Xin Li To: linux-kernel@vger.kernel.org, x86@kernel.org, kvm@vger.kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, peterz@infradead.org, andrew.cooper3@citrix.com, seanjc@google.com, pbonzini@redhat.com, ravi.v.shankar@intel.com, jiangshanlai@gmail.com, shan.kang@intel.com Subject: [PATCH v8 01/33] x86/traps: let common_interrupt() handle IRQ_MOVE_CLEANUP_VECTOR Date: Mon, 10 Apr 2023 01:14:06 -0700 Message-Id: <20230410081438.1750-2-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230410081438.1750-1-xin3.li@intel.com> References: <20230410081438.1750-1-xin3.li@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.5 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762778008285790143?= X-GMAIL-MSGID: =?utf-8?q?1762778008285790143?= From: "H. Peter Anvin (Intel)" IRQ_MOVE_CLEANUP_VECTOR is the only one of the system IRQ vectors that is *below* FIRST_SYSTEM_VECTOR. It is a slow path, so just push it into common_interrupt() just before the spurious interrupt handling. Signed-off-by: H. Peter Anvin (Intel) Tested-by: Shan Kang Signed-off-by: Xin Li --- arch/x86/kernel/irq.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/arch/x86/kernel/irq.c b/arch/x86/kernel/irq.c index 766ffe3ba313..7e125fff45ab 100644 --- a/arch/x86/kernel/irq.c +++ b/arch/x86/kernel/irq.c @@ -248,6 +248,10 @@ DEFINE_IDTENTRY_IRQ(common_interrupt) desc = __this_cpu_read(vector_irq[vector]); if (likely(!IS_ERR_OR_NULL(desc))) { handle_irq(desc, regs); +#ifdef CONFIG_SMP + } else if (vector == IRQ_MOVE_CLEANUP_VECTOR) { + sysvec_irq_move_cleanup(regs); +#endif } else { ack_APIC_irq(); From patchwork Mon Apr 10 08:14:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 81391 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1757062vqo; Mon, 10 Apr 2023 01:41:56 -0700 (PDT) X-Google-Smtp-Source: AKy350YlY5KoIsiEeYrZ5WAm3oKoF7NW+2qmAMkj9GUR/o7wlRdmXidt4yIvxOeXFdn6m2eV0GfF X-Received: by 2002:a17:902:e493:b0:1a2:1042:cac1 with SMTP id i19-20020a170902e49300b001a21042cac1mr10117498ple.49.1681116115718; Mon, 10 Apr 2023 01:41:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681116115; cv=none; d=google.com; s=arc-20160816; b=WGcMXN3oLnu4yJTE86W47Ku8nDh5nralr4+aL5VTLS4Ui8fNDJLFYEUZ4SV3pRx6re bYkJiBsu4gNl/IFmM0rfvjH6doGRlxFKkAiosw4jNVmDWE0aIbILdYmMpF8U+ChJ3nwW r6Ng81V+83RwUGQfSvFf7OGfUN2zci9zhC29vzynF0SLnyKotwSpZDv6v+UN2NeKWVTg SMZcrF8EjOQ7+wqBIwbNKSR4zO/MqB5eh5zo/ShKJW4C8mJtIkJFwBFvEUyxlVQ0YWcw H1Qf0ehILYis7QqnYZGYWb7BmoN+jg14fdgCnajTRsVG299H7/lx200woYN4RqNTPKCz OXQw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=69uZgB5syZguYJi9iT6gO8XISz/gXRjkiGmbEgB4SD0=; b=YIpvfwYsBeP789Nz+szouapzDOQ7C6R2oggMWZgD2P1m7KSk6arxb4tZTE0pkS4RBF Q7fRVD3dpLsbxBDHWEMl29OdCbd+gewWFKHF1mYDJhtXncFwtKR42ANRXQ2GBRfNZnRq 8BCE/Xr/184lVGnZpLvEU27rYObrwHanKEQnrXZRxl806CQbA+VPJYpm0ROqAO/Qqj5F zMDyA15QDc0SJfHVoffrmGgIohEVjz1bH8e8e9fNopD58JYT3y38uTajhI+h2K79ZFHU zevGOOG1RWeoGpHyPikoDFArWSg5DRTcMbuVbJvtBCaOv+yl/MJNbiI45PIyzkVu7OB2 DycQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=lTIqv9Uq; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id k8-20020a170902c40800b001a63d75e33esi2383989plk.6.2023.04.10.01.41.44; Mon, 10 Apr 2023 01:41:55 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=lTIqv9Uq; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229743AbjDJIlM (ORCPT + 99 others); Mon, 10 Apr 2023 04:41:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49320 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229626AbjDJIlE (ORCPT ); Mon, 10 Apr 2023 04:41:04 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8D86B40D9; Mon, 10 Apr 2023 01:41:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1681116063; x=1712652063; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Nl6wMwkpqmIYYyq7+MpcXBB5xF17anU3m57tPiay76c=; b=lTIqv9Uq9rS5cOrLsD8ryaVKRwpxNGI27Jah/m8tlOFIW0l4tDhfNmJg tVRGIRgftIhO4ZQAziIvvdiMfDH7llEAguwdue8AkgHO1o4+nvgd6QKgW 8Ykxe8S7FUfPqZjqypQL4KL2vT3jlNBQsQXbFWy1Oi9u/YUn60H0X3Tlx jQTehOqVdNwFuEHLU0lJPDQNyJwGo4AXIrxZqd5kgeagWWK7D4B0h2O7h H5btdyCOYxe/HF9CmyNQZ5SfjGGMXQSQoPe8TulYgf0YNKfifMdt/gjpj kUF9SiPFqhm2kX7YBfjuAzifSajY+fLoYM3L9qPj0rBim8ZpYJKxMeQzn Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="342077881" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="342077881" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2023 01:41:01 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="799436241" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="799436241" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga002.fm.intel.com with ESMTP; 10 Apr 2023 01:41:01 -0700 From: Xin Li To: linux-kernel@vger.kernel.org, x86@kernel.org, kvm@vger.kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, peterz@infradead.org, andrew.cooper3@citrix.com, seanjc@google.com, pbonzini@redhat.com, ravi.v.shankar@intel.com, jiangshanlai@gmail.com, shan.kang@intel.com Subject: [PATCH v8 02/33] x86/fred: make unions for the cs and ss fields in struct pt_regs Date: Mon, 10 Apr 2023 01:14:07 -0700 Message-Id: <20230410081438.1750-3-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230410081438.1750-1-xin3.li@intel.com> References: <20230410081438.1750-1-xin3.li@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.5 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762778012188065209?= X-GMAIL-MSGID: =?utf-8?q?1762778012188065209?= From: "H. Peter Anvin (Intel)" Make the cs and ss fields in struct pt_regs unions between the actual selector and the unsigned long stack slot. FRED uses this space to store additional flags. The printk changes are simply due to the cs and ss fields changed to unsigned short from unsigned long. Signed-off-by: H. Peter Anvin (Intel) Tested-by: Shan Kang Signed-off-by: Xin Li --- Changes since v3: * Rename csl/ssl of the pt_regs structure to csx/ssx (x for extended) (Andrew Cooper). --- arch/x86/entry/vsyscall/vsyscall_64.c | 2 +- arch/x86/include/asm/ptrace.h | 34 ++++++++++++++++++++++++--- arch/x86/kernel/process_64.c | 2 +- 3 files changed, 33 insertions(+), 5 deletions(-) diff --git a/arch/x86/entry/vsyscall/vsyscall_64.c b/arch/x86/entry/vsyscall/vsyscall_64.c index d234ca797e4a..2429ad0df068 100644 --- a/arch/x86/entry/vsyscall/vsyscall_64.c +++ b/arch/x86/entry/vsyscall/vsyscall_64.c @@ -76,7 +76,7 @@ static void warn_bad_vsyscall(const char *level, struct pt_regs *regs, if (!show_unhandled_signals) return; - printk_ratelimited("%s%s[%d] %s ip:%lx cs:%lx sp:%lx ax:%lx si:%lx di:%lx\n", + printk_ratelimited("%s%s[%d] %s ip:%lx cs:%x sp:%lx ax:%lx si:%lx di:%lx\n", level, current->comm, task_pid_nr(current), message, regs->ip, regs->cs, regs->sp, regs->ax, regs->si, regs->di); diff --git a/arch/x86/include/asm/ptrace.h b/arch/x86/include/asm/ptrace.h index f4db78b09c8f..2abb23e6c1e2 100644 --- a/arch/x86/include/asm/ptrace.h +++ b/arch/x86/include/asm/ptrace.h @@ -82,12 +82,40 @@ struct pt_regs { * On hw interrupt, it's IRQ number: */ unsigned long orig_ax; -/* Return frame for iretq */ +/* Return frame for iretq/eretu/erets */ unsigned long ip; - unsigned long cs; + union { + unsigned long csx; /* cs extended: CS + any fields above it */ + struct __attribute__((__packed__)) { + unsigned short cs; /* CS selector proper */ + unsigned int current_stack_level: 2; + unsigned int __csx_resv1 : 6; + unsigned int interrupt_shadowed : 1; + unsigned int software_initiated : 1; + unsigned int __csx_resv2 : 2; + unsigned int nmi : 1; + unsigned int __csx_resv3 : 3; + unsigned int __csx_resv4 : 32; + }; + }; unsigned long flags; unsigned long sp; - unsigned long ss; + union { + unsigned long ssx; /* ss extended: SS + any fields above it */ + struct __attribute__((__packed__)) { + unsigned short ss; /* SS selector proper */ + unsigned int __ssx_resv1 : 16; + unsigned int vector : 8; + unsigned int __ssx_resv2 : 8; + unsigned int type : 4; + unsigned int __ssx_resv3 : 4; + unsigned int enclv : 1; + unsigned int long_mode : 1; + unsigned int nested : 1; + unsigned int __ssx_resv4 : 1; + unsigned int instr_len : 4; + }; + }; /* top of stack page */ }; diff --git a/arch/x86/kernel/process_64.c b/arch/x86/kernel/process_64.c index bb65a68b4b49..a1aa74864c8b 100644 --- a/arch/x86/kernel/process_64.c +++ b/arch/x86/kernel/process_64.c @@ -116,7 +116,7 @@ void __show_regs(struct pt_regs *regs, enum show_regs_mode mode, printk("%sFS: %016lx(%04x) GS:%016lx(%04x) knlGS:%016lx\n", log_lvl, fs, fsindex, gs, gsindex, shadowgs); - printk("%sCS: %04lx DS: %04x ES: %04x CR0: %016lx\n", + printk("%sCS: %04x DS: %04x ES: %04x CR0: %016lx\n", log_lvl, regs->cs, ds, es, cr0); printk("%sCR2: %016lx CR3: %016lx CR4: %016lx\n", log_lvl, cr2, cr3, cr4); From patchwork Mon Apr 10 08:14:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 81393 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1757121vqo; Mon, 10 Apr 2023 01:42:04 -0700 (PDT) X-Google-Smtp-Source: AKy350YaZgbftWNsJJyWNEJv1KWz9NTJDdqVO8pYahFdAoRiMm1LvrBn2Zm9oFaJofOgU0CZZXWR X-Received: by 2002:a05:6a20:1a9b:b0:cb:9db3:e5e2 with SMTP id ci27-20020a056a201a9b00b000cb9db3e5e2mr9406456pzb.17.1681116124461; Mon, 10 Apr 2023 01:42:04 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681116124; cv=none; d=google.com; s=arc-20160816; b=xG0Hv1KZ0n7ZDcnZpV/DqCFu5ISEmQ6nZrHLUhxwwi35aG8OvlbeDAb8hPmKj/7rdX QUe+JNo/e3TetOiAsxb9Dlu8HpVmKN1OnV5xOM90oRECMvoCCThiwOPtxyqwRawFnCT+ uv3CPzqv4aCk8AhCwhuNNd2m65EpakaHbPLSs8oLXaerwMCZxaeu5qNgc+n5vCSunAvJ 6/V8aD9lQsfwNzB04gKKASQ7cO4dEgBrGrouDSk3UE40voT6Xb5QcvJJHk4MnNt7MIGS TCadbKnDDOh9JfSWwM3bEvAmPTCR47OJeG3tprrZ1iB0Ccxi31T9gb4gBYv/obhVU9VL WlTQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=0uJ3k26j2KXgP9x4Y+ooHxgSEtNq3VQ3yBcTjY/Mm/w=; b=mbplOytmp1oqhxzyiJrQSTpIORfJI5szcBOmA09L5NX8qJokmaha6xwfG9K0RouD1H qq/2EJEnrfm8M1rGIgi4T2qrPtCDU98/ED5DB4jyHSY9nC2FJZEpb5VZNzM+OFG/XfY+ 5wjLbOlEUyzWTEptxnRc/rMokrm//VHL6m76VM3L9PXYooLQPYdum0YUcpWXQrWuN0X1 uRUUbZEFp3RBvy+sOVoccTCc/2GbDT1le0KRITmcA4FPthKOkqrro7iZ+rS9Qg3+vM6d hsBNcMlFrFWvj8JZRAqvqtmab5eIFpBTp7Hhu6l+1lMfGvp5Vz3wkVBeTJZ8SKNutOjN ZiIQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=asVGckWG; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id o18-20020aa79792000000b006359be68abdsi3948437pfp.278.2023.04.10.01.41.52; Mon, 10 Apr 2023 01:42:04 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=asVGckWG; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229766AbjDJIlT (ORCPT + 99 others); Mon, 10 Apr 2023 04:41:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49340 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229695AbjDJIlF (ORCPT ); Mon, 10 Apr 2023 04:41:05 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E1F3640EA; Mon, 10 Apr 2023 01:41:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1681116063; x=1712652063; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=z5qENiAMJjZqe4SfagAzXU8H0qxh2oDTfz+gWyWVxMM=; b=asVGckWG/kA3eeV6tXqTL26Jysx22NRo7aft1XsUGsHd2o7OBdtd6tuq iWanjN5oOsRSmXLaPWB4ZXjyh1ANfxCI8ypcH49DP0Bj7m7jWvFH+ux81 IgnpL57FouB9HcSpi5ThPEoq/mBZdHovaSo3L3cGo84blKRYmUcKIfOlC 6fRL/0NEZ8kJi+V+bf8EOXKVvyDcU5QZsPKHASR+XblSWw5HKTEPP1YyM AJm8A5qkHBPDOCmWE6XBGkYxc/9uCoy41Qt1VSC1OKpaYZTBvgL35XHnM zdnn3SrzNtHSqXjPciO/o5iM4TgVhtc+Pc5oTJKS0PNymsc3OjSA7NQMg g==; X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="342077892" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="342077892" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2023 01:41:02 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="799436246" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="799436246" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga002.fm.intel.com with ESMTP; 10 Apr 2023 01:41:01 -0700 From: Xin Li To: linux-kernel@vger.kernel.org, x86@kernel.org, kvm@vger.kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, peterz@infradead.org, andrew.cooper3@citrix.com, seanjc@google.com, pbonzini@redhat.com, ravi.v.shankar@intel.com, jiangshanlai@gmail.com, shan.kang@intel.com Subject: [PATCH v8 03/33] x86/traps: add a system interrupt table for system interrupt dispatch Date: Mon, 10 Apr 2023 01:14:08 -0700 Message-Id: <20230410081438.1750-4-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230410081438.1750-1-xin3.li@intel.com> References: <20230410081438.1750-1-xin3.li@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.5 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762778021489304275?= X-GMAIL-MSGID: =?utf-8?q?1762778021489304275?= From: "H. Peter Anvin (Intel)" On x86, external interrupts are divided into the following two groups 1) system interrupts 2) external device interrupts External device interrupts are all routed to common_interrupt(), which dispatches external device interrupts through a per-CPU external interrupt dispatch table vector_irq. For system interrupts, add a system interrupt handler table for dispatching a system interrupt to its corresponding handler directly. Thus a software based dispatch function will be: void external_interrupt(struct pt_regs *regs) { u8 vector = regs->vector; if (is_system_interrupt(vector)) system_interrupt_handlers[vector_to_sysvec(vector)](regs); else /* external device interrupt */ common_interrupt(regs); } Signed-off-by: H. Peter Anvin (Intel) Co-developed-by: Xin Li Tested-by: Shan Kang Signed-off-by: Xin Li --- arch/x86/include/asm/idtentry.h | 64 +++++++++++++++++++++++++++------ arch/x86/include/asm/traps.h | 7 ++++ arch/x86/kernel/traps.c | 62 ++++++++++++++++++++++++++++++++ 3 files changed, 122 insertions(+), 11 deletions(-) diff --git a/arch/x86/include/asm/idtentry.h b/arch/x86/include/asm/idtentry.h index b241af4ce9b4..2876ddae02bc 100644 --- a/arch/x86/include/asm/idtentry.h +++ b/arch/x86/include/asm/idtentry.h @@ -167,17 +167,22 @@ __visible noinstr void func(struct pt_regs *regs, unsigned long error_code) /** * DECLARE_IDTENTRY_IRQ - Declare functions for device interrupt IDT entry - * points (common/spurious) + * points (common/spurious) and their corresponding + * software based dispatch handlers in the non-noinstr + * text section * @vector: Vector number (ignored for C) * @func: Function name of the entry point * - * Maps to DECLARE_IDTENTRY_ERRORCODE() + * Maps to DECLARE_IDTENTRY_ERRORCODE(), plus a dispatch function prototype */ #define DECLARE_IDTENTRY_IRQ(vector, func) \ - DECLARE_IDTENTRY_ERRORCODE(vector, func) + DECLARE_IDTENTRY_ERRORCODE(vector, func); \ + void dispatch_##func(struct pt_regs *regs, unsigned long error_code) /** * DEFINE_IDTENTRY_IRQ - Emit code for device interrupt IDT entry points + * and their corresponding software based dispatch + * handlers in the non-noinstr text section * @func: Function name of the entry point * * The vector number is pushed by the low level entry stub and handed @@ -187,6 +192,9 @@ __visible noinstr void func(struct pt_regs *regs, unsigned long error_code) * irq_enter/exit_rcu() are invoked before the function body and the * KVM L1D flush request is set. Stack switching to the interrupt stack * has to be done in the function body if necessary. + * + * dispatch_func() is a software based dispatch handler in the non-noinstr + * text section. */ #define DEFINE_IDTENTRY_IRQ(func) \ static void __##func(struct pt_regs *regs, u32 vector); \ @@ -204,31 +212,48 @@ __visible noinstr void func(struct pt_regs *regs, \ irqentry_exit(regs, state); \ } \ \ +void dispatch_##func(struct pt_regs *regs, unsigned long error_code) \ +{ \ + u32 vector = (u32)(u8)error_code; \ + \ + kvm_set_cpu_l1tf_flush_l1d(); \ + run_irq_on_irqstack_cond(__##func, regs, vector); \ +} \ + \ static noinline void __##func(struct pt_regs *regs, u32 vector) /** * DECLARE_IDTENTRY_SYSVEC - Declare functions for system vector entry points + * and their corresponding software based dispatch + * handlers in the non-noinstr text section * @vector: Vector number (ignored for C) * @func: Function name of the entry point * - * Declares three functions: + * Declares four functions: * - The ASM entry point: asm_##func * - The XEN PV trap entry point: xen_##func (maybe unused) * - The C handler called from the ASM entry point + * - The C handler used in the system interrupt handler table * - * Maps to DECLARE_IDTENTRY(). + * Maps to DECLARE_IDTENTRY(), plus a dispatch table function prototype */ #define DECLARE_IDTENTRY_SYSVEC(vector, func) \ - DECLARE_IDTENTRY(vector, func) + DECLARE_IDTENTRY(vector, func); \ + void dispatch_table_##func(struct pt_regs *regs) /** * DEFINE_IDTENTRY_SYSVEC - Emit code for system vector IDT entry points + * and their corresponding software based dispatch + * handlers in the non-noinstr text section * @func: Function name of the entry point * * irqentry_enter/exit() and irq_enter/exit_rcu() are invoked before the * function body. KVM L1D flush request is set. * - * Runs the function on the interrupt stack if the entry hit kernel mode + * Runs the function on the interrupt stack if the entry hit kernel mode. + * + * dispatch_table_func() is used in the system interrupt handler table for + * system interrupts dispatching. */ #define DEFINE_IDTENTRY_SYSVEC(func) \ static void __##func(struct pt_regs *regs); \ @@ -244,11 +269,19 @@ __visible noinstr void func(struct pt_regs *regs) \ irqentry_exit(regs, state); \ } \ \ +void dispatch_table_##func(struct pt_regs *regs) \ +{ \ + kvm_set_cpu_l1tf_flush_l1d(); \ + run_sysvec_on_irqstack_cond(__##func, regs); \ +} \ + \ static noinline void __##func(struct pt_regs *regs) /** * DEFINE_IDTENTRY_SYSVEC_SIMPLE - Emit code for simple system vector IDT - * entry points + * entry points and their corresponding + * software based dispatch handlers in + * the non-noinstr text section * @func: Function name of the entry point * * Runs the function on the interrupted stack. No switch to IRQ stack and @@ -256,6 +289,9 @@ static noinline void __##func(struct pt_regs *regs) * * Only use for 'empty' vectors like reschedule IPI and KVM posted * interrupt vectors. + * + * dispatch_table_func() is used in the system interrupt handler table for + * system interrupts dispatching. */ #define DEFINE_IDTENTRY_SYSVEC_SIMPLE(func) \ static __always_inline void __##func(struct pt_regs *regs); \ @@ -273,6 +309,14 @@ __visible noinstr void func(struct pt_regs *regs) \ irqentry_exit(regs, state); \ } \ \ +void dispatch_table_##func(struct pt_regs *regs) \ +{ \ + __irq_enter_raw(); \ + kvm_set_cpu_l1tf_flush_l1d(); \ + __##func (regs); \ + __irq_exit_raw(); \ +} \ + \ static __always_inline void __##func(struct pt_regs *regs) /** @@ -634,9 +678,7 @@ DECLARE_IDTENTRY(X86_TRAP_VE, exc_virtualization_exception); /* Device interrupts common/spurious */ DECLARE_IDTENTRY_IRQ(X86_TRAP_OTHER, common_interrupt); -#ifdef CONFIG_X86_LOCAL_APIC DECLARE_IDTENTRY_IRQ(X86_TRAP_OTHER, spurious_interrupt); -#endif /* System vector entry points */ #ifdef CONFIG_X86_LOCAL_APIC @@ -647,7 +689,7 @@ DECLARE_IDTENTRY_SYSVEC(X86_PLATFORM_IPI_VECTOR, sysvec_x86_platform_ipi); #endif #ifdef CONFIG_SMP -DECLARE_IDTENTRY(RESCHEDULE_VECTOR, sysvec_reschedule_ipi); +DECLARE_IDTENTRY_SYSVEC(RESCHEDULE_VECTOR, sysvec_reschedule_ipi); DECLARE_IDTENTRY_SYSVEC(IRQ_MOVE_CLEANUP_VECTOR, sysvec_irq_move_cleanup); DECLARE_IDTENTRY_SYSVEC(REBOOT_VECTOR, sysvec_reboot); DECLARE_IDTENTRY_SYSVEC(CALL_FUNCTION_SINGLE_VECTOR, sysvec_call_function_single); diff --git a/arch/x86/include/asm/traps.h b/arch/x86/include/asm/traps.h index 47ecfff2c83d..28c8ba5fd81c 100644 --- a/arch/x86/include/asm/traps.h +++ b/arch/x86/include/asm/traps.h @@ -47,4 +47,11 @@ void __noreturn handle_stack_overflow(struct pt_regs *regs, struct stack_info *info); #endif +/* + * How system interrupt handlers are called. + */ +#define DECLARE_SYSTEM_INTERRUPT_HANDLER(f) \ + void f (struct pt_regs *regs) +typedef DECLARE_SYSTEM_INTERRUPT_HANDLER((*system_interrupt_handler)); + #endif /* _ASM_X86_TRAPS_H */ diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c index d317dc3d06a3..2cbe7e7e8b96 100644 --- a/arch/x86/kernel/traps.c +++ b/arch/x86/kernel/traps.c @@ -1451,6 +1451,68 @@ DEFINE_IDTENTRY_SW(iret_error) } #endif +#ifdef CONFIG_X86_64 + +#ifndef CONFIG_X86_LOCAL_APIC +/* + * Used when local APIC is not configured to build into the kernel, but + * dispatch_table_spurious_interrupt() needs dispatch_spurious_interrupt(). + */ +DEFINE_IDTENTRY_IRQ(spurious_interrupt) +{ + pr_info("Spurious interrupt (vector 0x%x) on CPU#%d, should never happen.\n", + vector, smp_processor_id()); +} +#endif + +static void dispatch_table_spurious_interrupt(struct pt_regs *regs) +{ + dispatch_spurious_interrupt(regs, regs->vector); +} + +#define SYSV(x,y) [(x) - FIRST_SYSTEM_VECTOR] = y + +static system_interrupt_handler system_interrupt_handlers[NR_SYSTEM_VECTORS] = { + [0 ... NR_SYSTEM_VECTORS-1] = dispatch_table_spurious_interrupt, +#ifdef CONFIG_SMP + SYSV(RESCHEDULE_VECTOR, dispatch_table_sysvec_reschedule_ipi), + SYSV(CALL_FUNCTION_VECTOR, dispatch_table_sysvec_call_function), + SYSV(CALL_FUNCTION_SINGLE_VECTOR, dispatch_table_sysvec_call_function_single), + SYSV(REBOOT_VECTOR, dispatch_table_sysvec_reboot), +#endif + +#ifdef CONFIG_X86_THERMAL_VECTOR + SYSV(THERMAL_APIC_VECTOR, dispatch_table_sysvec_thermal), +#endif + +#ifdef CONFIG_X86_MCE_THRESHOLD + SYSV(THRESHOLD_APIC_VECTOR, dispatch_table_sysvec_threshold), +#endif + +#ifdef CONFIG_X86_MCE_AMD + SYSV(DEFERRED_ERROR_VECTOR, dispatch_table_sysvec_deferred_error), +#endif + +#ifdef CONFIG_X86_LOCAL_APIC + SYSV(LOCAL_TIMER_VECTOR, dispatch_table_sysvec_apic_timer_interrupt), + SYSV(X86_PLATFORM_IPI_VECTOR, dispatch_table_sysvec_x86_platform_ipi), +# ifdef CONFIG_HAVE_KVM + SYSV(POSTED_INTR_VECTOR, dispatch_table_sysvec_kvm_posted_intr_ipi), + SYSV(POSTED_INTR_WAKEUP_VECTOR, dispatch_table_sysvec_kvm_posted_intr_wakeup_ipi), + SYSV(POSTED_INTR_NESTED_VECTOR, dispatch_table_sysvec_kvm_posted_intr_nested_ipi), +# endif +# ifdef CONFIG_IRQ_WORK + SYSV(IRQ_WORK_VECTOR, dispatch_table_sysvec_irq_work), +# endif + SYSV(SPURIOUS_APIC_VECTOR, dispatch_table_sysvec_spurious_apic_interrupt), + SYSV(ERROR_APIC_VECTOR, dispatch_table_sysvec_error_interrupt), +#endif +}; + +#undef SYSV + +#endif /* CONFIG_X86_64 */ + void __init trap_init(void) { /* Init cpu_entry_area before IST entries are set up */ From patchwork Mon Apr 10 08:14:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 81396 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1757233vqo; Mon, 10 Apr 2023 01:42:21 -0700 (PDT) X-Google-Smtp-Source: AKy350bG2cVotsS0TBUeT23HNxGRc0m2Q4cyVmAuqAiYI/mQxt64js16jeX6mph1PjRoxnjIlFPB X-Received: by 2002:a17:902:f143:b0:1a5:3da:84cb with SMTP id d3-20020a170902f14300b001a503da84cbmr9985487plb.21.1681116141206; Mon, 10 Apr 2023 01:42:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681116141; cv=none; d=google.com; s=arc-20160816; b=U91rnHn9ZonTrwBMQ7wBuBdsEZebSix3CYWV3KP24xCj3nwj5drLQi5wyjHZ8rtw5n 3NARDXiqgj1Iy5AvJto1sDpZG0XRUsJ5EgmpkkTpZt87Eni42YtmEpWl7k9S/BBh8jcL nmpM4+KT193l/i9xM4TsinHfno8Dy2q8DjUl6DswTbNnzsgYrw5HLA5b1GehuJnhYH9Z +kxC5oFnDyTSzWMwMV4KcHreaMPRSezPOGxTDLYqXLOyhbZSbMTU8O8URspGuVTRfU0p vMe8RxF/5V9HkEG4pVfnLcZOp1y3nTk8ChcMnpApsreMPomgQFHjjzYbZAtcYeU/RdEO 4HGA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=WqEsudKPtU/48zeiRN+6GQBbNzGrNsHFqQ78iXIZMtA=; b=ZJghcFtwTEnbYshWuKiaswzFZ7h34A1dDXVH8sZHnGu7i9pponpY0Pc8gZ7+8wQMlY Rclr9LkNLoDt6I1jZ3RrWDakTc4Mu6Roqgn3N1uVjZT8DBwYNAZ0ZiS5YcqaXQxErBAD VuvnJbrQldd91d3n1tr9ZBFevtZRuH86+Jz7D5MicmZ7GUiYsT5907hCpDHk3Tz/h2Gx aBcAEor8k68CARXcOKrETcJnutRd9C8yKGjbJzBQXS7B/DtBWOaZdIvaQaQOcKdrFl2n k8x9JVjRFzgAqMVp4tUpGXw+iGeOEjsdDCxKmJ3OKdRz+7RGhM8ObiGJ21Aqk3ATBBsC GioA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=NK0pdLry; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id l38-20020a635b66000000b0050fa02c0326si9585607pgm.277.2023.04.10.01.42.09; Mon, 10 Apr 2023 01:42:21 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=NK0pdLry; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229803AbjDJIl1 (ORCPT + 99 others); Mon, 10 Apr 2023 04:41:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49344 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229697AbjDJIlF (ORCPT ); Mon, 10 Apr 2023 04:41:05 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 68A35270E; Mon, 10 Apr 2023 01:41:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1681116064; x=1712652064; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=LezTItrOTBDG2aixpyZh+BhRRHCKfV9D1LBlf5QDBQY=; b=NK0pdLrysm6FJfMsmoeFPNplRjaBV+LxdcnGrXg4J78v9apb48Nr7ryB gFulpvf2AZHjH0ENrVKQT4Kdqr5eC8UOsBbAC06qYRjIicb3gtSE8tIBF JUXlj9wAmHvsv3JO466HLtFM/28gUw/eveZfayp3unWsef6s1zFYCR6mc MdUo4BArY3jsSAvMb8itQqnuWyS84jrA77ZJi+XOg99x9vDIzmopVxu5U 9gP/ukafzez38Jvl+jnMJ7sYkb3F7UoGbBdhqIQvwlBUcV4Q9Ni7awcGj IrQlaXa33qm13KFzVp5J1iiT9QbyuFCi4AMyFy46fDhbrdiHwS54rCI9v Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="342077903" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="342077903" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2023 01:41:02 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="799436250" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="799436250" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga002.fm.intel.com with ESMTP; 10 Apr 2023 01:41:02 -0700 From: Xin Li To: linux-kernel@vger.kernel.org, x86@kernel.org, kvm@vger.kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, peterz@infradead.org, andrew.cooper3@citrix.com, seanjc@google.com, pbonzini@redhat.com, ravi.v.shankar@intel.com, jiangshanlai@gmail.com, shan.kang@intel.com Subject: [PATCH v8 04/33] x86/traps: add install_system_interrupt_handler() Date: Mon, 10 Apr 2023 01:14:09 -0700 Message-Id: <20230410081438.1750-5-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230410081438.1750-1-xin3.li@intel.com> References: <20230410081438.1750-1-xin3.li@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.5 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762778038789545091?= X-GMAIL-MSGID: =?utf-8?q?1762778038789545091?= Some kernel components install system interrupt handlers into the IDT, and we need to do the same for system_interrupt_handlers. A new function install_system_interrupt_handler() is added to install a system interrupt handler into both the IDT and system_interrupt_handlers. Tested-by: Shan Kang Signed-off-by: Xin Li --- arch/x86/include/asm/traps.h | 2 ++ arch/x86/kernel/cpu/acrn.c | 7 +++++-- arch/x86/kernel/cpu/mshyperv.c | 22 ++++++++++++++-------- arch/x86/kernel/kvm.c | 4 +++- arch/x86/kernel/traps.c | 10 ++++++++++ drivers/xen/events/events_base.c | 5 ++++- 6 files changed, 38 insertions(+), 12 deletions(-) diff --git a/arch/x86/include/asm/traps.h b/arch/x86/include/asm/traps.h index 28c8ba5fd81c..46f5e4e2a346 100644 --- a/arch/x86/include/asm/traps.h +++ b/arch/x86/include/asm/traps.h @@ -41,6 +41,8 @@ void math_emulate(struct math_emu_info *); bool fault_in_kernel_space(unsigned long address); +void install_system_interrupt_handler(unsigned int n, const void *asm_addr, const void *addr); + #ifdef CONFIG_VMAP_STACK void __noreturn handle_stack_overflow(struct pt_regs *regs, unsigned long fault_address, diff --git a/arch/x86/kernel/cpu/acrn.c b/arch/x86/kernel/cpu/acrn.c index 485441b7f030..9351bf183a9e 100644 --- a/arch/x86/kernel/cpu/acrn.c +++ b/arch/x86/kernel/cpu/acrn.c @@ -18,6 +18,7 @@ #include #include #include +#include static u32 __init acrn_detect(void) { @@ -26,8 +27,10 @@ static u32 __init acrn_detect(void) static void __init acrn_init_platform(void) { - /* Setup the IDT for ACRN hypervisor callback */ - alloc_intr_gate(HYPERVISOR_CALLBACK_VECTOR, asm_sysvec_acrn_hv_callback); + /* Install system interrupt handler for ACRN hypervisor callback */ + install_system_interrupt_handler(HYPERVISOR_CALLBACK_VECTOR, + asm_sysvec_acrn_hv_callback, + sysvec_acrn_hv_callback); x86_platform.calibrate_tsc = acrn_get_tsc_khz; x86_platform.calibrate_cpu = acrn_get_tsc_khz; diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c index f1197366a97d..c102c9192cbb 100644 --- a/arch/x86/kernel/cpu/mshyperv.c +++ b/arch/x86/kernel/cpu/mshyperv.c @@ -29,6 +29,7 @@ #include #include #include +#include #include #include #include @@ -491,19 +492,24 @@ static void __init ms_hyperv_init_platform(void) */ x86_platform.apic_post_init = hyperv_init; hyperv_setup_mmu_ops(); - /* Setup the IDT for hypervisor callback */ - alloc_intr_gate(HYPERVISOR_CALLBACK_VECTOR, asm_sysvec_hyperv_callback); - /* Setup the IDT for reenlightenment notifications */ + /* Install system interrupt handler for hypervisor callback */ + install_system_interrupt_handler(HYPERVISOR_CALLBACK_VECTOR, + asm_sysvec_hyperv_callback, + sysvec_hyperv_callback); + + /* Install system interrupt handler for reenlightenment notifications */ if (ms_hyperv.features & HV_ACCESS_REENLIGHTENMENT) { - alloc_intr_gate(HYPERV_REENLIGHTENMENT_VECTOR, - asm_sysvec_hyperv_reenlightenment); + install_system_interrupt_handler(HYPERV_REENLIGHTENMENT_VECTOR, + asm_sysvec_hyperv_reenlightenment, + sysvec_hyperv_reenlightenment); } - /* Setup the IDT for stimer0 */ + /* Install system interrupt handler for stimer0 */ if (ms_hyperv.misc_features & HV_STIMER_DIRECT_MODE_AVAILABLE) { - alloc_intr_gate(HYPERV_STIMER0_VECTOR, - asm_sysvec_hyperv_stimer0); + install_system_interrupt_handler(HYPERV_STIMER0_VECTOR, + asm_sysvec_hyperv_stimer0, + sysvec_hyperv_stimer0); } # ifdef CONFIG_SMP diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c index 1cceac5984da..5c684df6de7a 100644 --- a/arch/x86/kernel/kvm.c +++ b/arch/x86/kernel/kvm.c @@ -829,7 +829,9 @@ static void __init kvm_guest_init(void) if (kvm_para_has_feature(KVM_FEATURE_ASYNC_PF_INT) && kvmapf) { static_branch_enable(&kvm_async_pf_enabled); - alloc_intr_gate(HYPERVISOR_CALLBACK_VECTOR, asm_sysvec_kvm_asyncpf_interrupt); + install_system_interrupt_handler(HYPERVISOR_CALLBACK_VECTOR, + asm_sysvec_kvm_asyncpf_interrupt, + sysvec_kvm_asyncpf_interrupt); } #ifdef CONFIG_SMP diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c index 2cbe7e7e8b96..12072e2af4a6 100644 --- a/arch/x86/kernel/traps.c +++ b/arch/x86/kernel/traps.c @@ -1513,6 +1513,16 @@ static system_interrupt_handler system_interrupt_handlers[NR_SYSTEM_VECTORS] = { #endif /* CONFIG_X86_64 */ +void __init install_system_interrupt_handler(unsigned int n, const void *asm_addr, const void *addr) +{ + BUG_ON(n < FIRST_SYSTEM_VECTOR); + +#ifdef CONFIG_X86_64 + system_interrupt_handlers[n - FIRST_SYSTEM_VECTOR] = (system_interrupt_handler)addr; +#endif + alloc_intr_gate(n, asm_addr); +} + void __init trap_init(void) { /* Init cpu_entry_area before IST entries are set up */ diff --git a/drivers/xen/events/events_base.c b/drivers/xen/events/events_base.c index c7715f8bd452..cf1a5ca3bf62 100644 --- a/drivers/xen/events/events_base.c +++ b/drivers/xen/events/events_base.c @@ -45,6 +45,7 @@ #include #include #include +#include #include #include #endif @@ -2249,7 +2250,9 @@ static __init void xen_alloc_callback_vector(void) return; pr_info("Xen HVM callback vector for event delivery is enabled\n"); - alloc_intr_gate(HYPERVISOR_CALLBACK_VECTOR, asm_sysvec_xen_hvm_callback); + install_system_interrupt_handler(HYPERVISOR_CALLBACK_VECTOR, + asm_sysvec_xen_hvm_callback, + sysvec_xen_hvm_callback); } #else void xen_setup_callback_vector(void) {} From patchwork Mon Apr 10 08:14:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 81392 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1757097vqo; Mon, 10 Apr 2023 01:42:00 -0700 (PDT) X-Google-Smtp-Source: AKy350b7c6LgRJPAQKoPJ7XmXbmwvNrkK/LXiQuPjIrDLNaZX4xgIFfAEg8GVC3CScSR+jqVJFMp X-Received: by 2002:a17:902:e892:b0:199:4be8:be48 with SMTP id w18-20020a170902e89200b001994be8be48mr11420392plg.19.1681116120350; Mon, 10 Apr 2023 01:42:00 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681116120; cv=none; d=google.com; s=arc-20160816; b=Pj641efvS6DSlzXwRji1/QI97FP7deBlsyURDyRkg0r0y62p8CqYwIo+p1JeEcCmEZ GxrBUj745zq00+AlZiS+VyNs7TtMQRa7xRNIUdRYQQWTsHhri4AXNlREqGh5AfJFZVWG XI/cP68ZV6RyxGqE/4sD7dFCj47Z8iJ9tG5jIqMyL/z+dPHYIHz3T6bIFAhshOw45E+U NzUP1beb/kOMORJFas5t5RyhIUzK6+J0K+/o4elnWhDpb05KI+h1+QIgF/y6NKzUE2Tr vode6Mg1b44Pz7FdLtqXk1jA+pX0Q+MQOEX00lmqC/+9gzjjT2xjmRLBqlXtN8tANj6+ L/xA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=+bKyrAJ4Sg6BfMRQkBUGYsVA8R9oXuG/B40jl5jzfJ0=; b=nRJSyARg+NWtGshdcAaxKh9I4GaOXp6RxmbYmla2OKGhkO0hdpJMMosZsH6rfUgNQU PCoEOFFoVM2RYAQ60ZLmu8p7kI67nIbDgPily4jWg9UFiXWYX46MT/BIhrmLwL8H1Unh YD5E3ipCCdXiEQkdphhpqml7YkPJd+RQyHdbitxbgmNXh/ox+cGz0+Tc5KdAzHP8SCxk miNBusfKNtS+U4zBMQpnlmT/8JBhnNi/bUD/Nn3O0Gg4zdSN1YaXSFeHxEzfhLPsEMye CAQXV8+K4xtSnMM+QAF3cEXfTC9kjrF/4pknHclARGYqYY7thjbjmmvf9UMw8jUwgnLZ grLg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=GLi1NB3H; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id h18-20020a170902f55200b0019ceb9491d2si11128248plf.391.2023.04.10.01.41.48; Mon, 10 Apr 2023 01:42:00 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=GLi1NB3H; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229754AbjDJIlP (ORCPT + 99 others); Mon, 10 Apr 2023 04:41:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49342 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229694AbjDJIlF (ORCPT ); Mon, 10 Apr 2023 04:41:05 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8C5733AAD; Mon, 10 Apr 2023 01:41:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1681116064; x=1712652064; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=BCbf0Vj9ZyZPtbZTSgD/ohjbP2Y9NHqyEC0o1Sj/JgM=; b=GLi1NB3HM7JxG3th5VQR+LrMVv+XTvpezi97IdF+STW+5oIarJ+0Ete5 mESESepyiE9n4UpY614rl7mrAtAMyhx/X+Qi1RNMm4KKuI4vuAaZ80iCs mpXPP34sWvQSUDqE+VvVNEkg33K4ouEzUiyUeMqrSONZPr7o36mI7m74o ulqjqYIqtM5g/LEre5/cAmETYN3CSIHv4LKWj9/08iOXG21IH0tbhpRub /eqSayLY57IiSBF26+Gj5PYG33SfuQWcUT1kbmKHqS623/5mI4G4XuQra BBDjgwju1B/OY+7nyCZGp0R5mjSS7qwyYpYj3yKnkqud69XDSyLZ7AQiA w==; X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="342077913" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="342077913" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2023 01:41:02 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="799436254" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="799436254" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga002.fm.intel.com with ESMTP; 10 Apr 2023 01:41:02 -0700 From: Xin Li To: linux-kernel@vger.kernel.org, x86@kernel.org, kvm@vger.kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, peterz@infradead.org, andrew.cooper3@citrix.com, seanjc@google.com, pbonzini@redhat.com, ravi.v.shankar@intel.com, jiangshanlai@gmail.com, shan.kang@intel.com Subject: [PATCH v8 05/33] x86/traps: add external_interrupt() to dispatch external interrupts Date: Mon, 10 Apr 2023 01:14:10 -0700 Message-Id: <20230410081438.1750-6-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230410081438.1750-1-xin3.li@intel.com> References: <20230410081438.1750-1-xin3.li@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.5 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762778017070468635?= X-GMAIL-MSGID: =?utf-8?q?1762778017070468635?= From: "H. Peter Anvin (Intel)" Add external_interrupt() to dispatch external interrupts to their handlers. If an external interrupt is a system interrupt, dipatch it through system_interrupt_handlers table, otherwise to dispatch_common_interrupt(). Signed-off-by: H. Peter Anvin (Intel) Co-developed-by: Xin Li Tested-by: Shan Kang Signed-off-by: Xin Li --- Changes since v5: * Initialize system_interrupt_handlers with dispatch_table_spurious_interrupt() instead of NULL to get rid of a branch (Peter Zijlstra). --- arch/x86/kernel/traps.c | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c index 12072e2af4a6..f86cd233b00b 100644 --- a/arch/x86/kernel/traps.c +++ b/arch/x86/kernel/traps.c @@ -1511,6 +1511,32 @@ static system_interrupt_handler system_interrupt_handlers[NR_SYSTEM_VECTORS] = { #undef SYSV +/* + * External interrupt dispatch function. + * + * Until/unless dispatch_common_interrupt() can be taught to deal with the + * special system vectors, split the dispatch. + * + * Note: dispatch_common_interrupt() already deals with IRQ_MOVE_CLEANUP_VECTOR. + */ +int external_interrupt(struct pt_regs *regs) +{ + unsigned int vector = regs->vector; + unsigned int sysvec = vector - FIRST_SYSTEM_VECTOR; + + if (vector < FIRST_EXTERNAL_VECTOR) { + pr_err("invalid external interrupt vector %d\n", vector); + return -EINVAL; + } + + if (sysvec < NR_SYSTEM_VECTORS) + system_interrupt_handlers[sysvec](regs); + else + dispatch_common_interrupt(regs, vector); + + return 0; +} + #endif /* CONFIG_X86_64 */ void __init install_system_interrupt_handler(unsigned int n, const void *asm_addr, const void *addr) From patchwork Mon Apr 10 08:14:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 81394 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1757182vqo; Mon, 10 Apr 2023 01:42:13 -0700 (PDT) X-Google-Smtp-Source: AKy350aafPdxJKE5u/CzuHafzWIwPdyN44LIjHf94k6nnnU8RuvbgqgWV3rsXR2I0dtFzCkGxTcN X-Received: by 2002:a17:903:2302:b0:1a1:d774:a3d7 with SMTP id d2-20020a170903230200b001a1d774a3d7mr19274222plh.25.1681116133135; Mon, 10 Apr 2023 01:42:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681116133; cv=none; d=google.com; s=arc-20160816; b=jWq/e1ensSDdFOeHn5oa6rP9cSqpfxt4YbFbaXlcNXgH9xuFIJysB2rhJK0Sga4TSb ydDkoRuxAtz/ntEJ8oyU7J8Qm5wzwUGCzBhpTe0DglR3xsNXtp0sEHuDdKImCGPTZSBL +BbOD1/pRKCyGuD4VNb5A9OWLM31OgYHFXw3ALp8DcyGjVWalYxBWu6ElfIuOmLBKTL2 lii3+P65nICjvO7cbkt7oMMoxBhHeIt1KYQp591sAMH/LTpz1Uihs793DE2/jPZ8uCS5 3SCFmEbBQn3j0PXafw0ktr2HLKQ3w00o+4WRFoSxW8Im0y3qfovhsXhMYlSYIUjCv06h O4ow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=s1TnurdU9eI1YRCwil5WMrFTmYZQx9ZLxqgiOpLcQhQ=; b=x/mNBK9va1W4aK/xDDWuAFSH0TR4sC0qvisoncwumf6NMB+TRUQ1kYW9C2nELRdZoT YDAHEeLSUK3fiigakVRK9V9xyIoSREBQECLy4pTlY6c/o97ryrJ2gYn2SJWru7T4TdiM tcYjAuw6LLuo738vTl2mZ2LOkDvY7p1cLjkgFEoHLA8Ameksrf6YDR1duKXBlkV06ihx 6qEJDOJgYQi5V6DaSwqJG6io/HYBLdDmVLGFCpGvOecsS82BFPAhELDhuqrdupPLPI11 cic/gvCLTULU0Af4kdQqQNyE29HaCz93ZVgwyPNsf4SUm81XREfnpG9lJG9Na0RXxGHB ZA+A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=kqsuerdK; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id t2-20020a632d02000000b00513b0d416e4si9705532pgt.599.2023.04.10.01.42.01; Mon, 10 Apr 2023 01:42:13 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=kqsuerdK; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229773AbjDJIlW (ORCPT + 99 others); Mon, 10 Apr 2023 04:41:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49380 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229703AbjDJIlG (ORCPT ); Mon, 10 Apr 2023 04:41:06 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 49A513AB1; Mon, 10 Apr 2023 01:41:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1681116065; x=1712652065; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=g68cyMdX2debPTjgjq16HL+D6AhwC/hj42OGPtnTvek=; b=kqsuerdKQpgC0EErWd+pi1duB5FXb6jTjqMA4HtS1HDmMMMRBXvZZOes AiVwBo6GdJMnMELzqJ/X4S1I4e3wfWvG942JruqXRZqanaISm5HiRXKZc /qNkYoPTE/q6RAhXSCPeaXbp/i2H4+lpAVOik2LUGCEBvxaf0haRLzBh+ JGs4+zp/kmyhZZyPaKVz5VngZa4TdVRyn7swwnLUg08p2WD/r7tRmDd6F g80SUCaqoMCK0aK6OHQc7hR2i5yL1byA+vUV3Pk0WD3PCho7onbdLoLgV BZ8LJhpA7InhTJKroGEPJxhbSbuQz3pjg7BXlcFNH+eN3i0ARj6jdbLvj w==; X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="342077923" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="342077923" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2023 01:41:03 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="799436259" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="799436259" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga002.fm.intel.com with ESMTP; 10 Apr 2023 01:41:02 -0700 From: Xin Li To: linux-kernel@vger.kernel.org, x86@kernel.org, kvm@vger.kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, peterz@infradead.org, andrew.cooper3@citrix.com, seanjc@google.com, pbonzini@redhat.com, ravi.v.shankar@intel.com, jiangshanlai@gmail.com, shan.kang@intel.com Subject: [PATCH v8 06/33] x86/cpufeature: add the cpu feature bit for FRED Date: Mon, 10 Apr 2023 01:14:11 -0700 Message-Id: <20230410081438.1750-7-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230410081438.1750-1-xin3.li@intel.com> References: <20230410081438.1750-1-xin3.li@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.5 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762778030606506807?= X-GMAIL-MSGID: =?utf-8?q?1762778030606506807?= From: "H. Peter Anvin (Intel)" Add the CPU feature bit for FRED (Flexible Return and Event Delivery). The Intel flexible return and event delivery (FRED) architecture defines simple new transitions that change privilege level (ring transitions). The FRED architecture was designed with the following goals: 1) Improve overall performance and response time by replacing event delivery through the interrupt descriptor table (IDT event delivery) and event return by the IRET instruction with lower latency transitions. 2) Improve software robustness by ensuring that event delivery establishes the full supervisor context and that event return establishes the full user context. The new transitions defined by the FRED architecture are FRED event delivery and, for returning from events, two FRED return instructions. FRED event delivery can effect a transition from ring 3 to ring 0, but it is used also to deliver events incident to ring 0. One FRED instruction (ERETU) effects a return from ring 0 to ring 3, while the other (ERETS) returns while remaining in ring 0. Search for the latest FRED spec in most search engines with this search pattern: site:intel.com FRED (flexible return and event delivery) specification Signed-off-by: H. Peter Anvin (Intel) Tested-by: Shan Kang Signed-off-by: Xin Li --- arch/x86/include/asm/cpufeatures.h | 1 + tools/arch/x86/include/asm/cpufeatures.h | 1 + 2 files changed, 2 insertions(+) diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h index 73c9672c123b..1fa444478d33 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -318,6 +318,7 @@ #define X86_FEATURE_FZRM (12*32+10) /* "" Fast zero-length REP MOVSB */ #define X86_FEATURE_FSRS (12*32+11) /* "" Fast short REP STOSB */ #define X86_FEATURE_FSRC (12*32+12) /* "" Fast short REP {CMPSB,SCASB} */ +#define X86_FEATURE_FRED (12*32+17) /* Flexible Return and Event Delivery */ #define X86_FEATURE_LKGS (12*32+18) /* "" Load "kernel" (userspace) GS */ #define X86_FEATURE_AMX_FP16 (12*32+21) /* "" AMX fp16 Support */ #define X86_FEATURE_AVX_IFMA (12*32+23) /* "" Support for VPMADD52[H,L]UQ */ diff --git a/tools/arch/x86/include/asm/cpufeatures.h b/tools/arch/x86/include/asm/cpufeatures.h index b89005819cd5..e9064f4a011a 100644 --- a/tools/arch/x86/include/asm/cpufeatures.h +++ b/tools/arch/x86/include/asm/cpufeatures.h @@ -312,6 +312,7 @@ #define X86_FEATURE_AVX_VNNI (12*32+ 4) /* AVX VNNI instructions */ #define X86_FEATURE_AVX512_BF16 (12*32+ 5) /* AVX512 BFLOAT16 instructions */ #define X86_FEATURE_CMPCCXADD (12*32+ 7) /* "" CMPccXADD instructions */ +#define X86_FEATURE_FRED (12*32+17) /* Flexible Return and Event Delivery */ #define X86_FEATURE_LKGS (12*32+18) /* "" Load "kernel" (userspace) GS */ #define X86_FEATURE_AMX_FP16 (12*32+21) /* "" AMX fp16 Support */ #define X86_FEATURE_AVX_IFMA (12*32+23) /* "" Support for VPMADD52[H,L]UQ */ From patchwork Mon Apr 10 08:14:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 81419 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1761292vqo; Mon, 10 Apr 2023 01:54:34 -0700 (PDT) X-Google-Smtp-Source: AKy350bwhLpQRmId+Oa1Z1KucBQ/S1oRfwz0KKAKp11NtUmQgDSs7TmbYFjS25eMpBt5RcMmN603 X-Received: by 2002:a05:6a20:4b23:b0:da:1b99:34f0 with SMTP id fp35-20020a056a204b2300b000da1b9934f0mr10866076pzb.39.1681116874446; Mon, 10 Apr 2023 01:54:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681116874; cv=none; d=google.com; s=arc-20160816; b=LMiTA4fuhIUpZSmnSDzm/HBCtcCjSUW8V2bZtIJY2de99eNslpIifa0fUoQUcciM0t an9FHiNI1EeiNH41OpGDaCtCC8uOeFj61DwRQwBKODS4Yz3brsnvXHb0go40wMw0pTNx KU6KmVU8sJhxVF1Afk2tKGCUYyRwjzl3l3pPm4f2zc/VHyigwaXE9zLYTpFeb9O0y1Mk IbivMd8ho0XEKbp4JpITCXp/e4LJuH0oGN0qhwlPNpJYpfYVK/OR8xN3HQYUDNaQ/p02 aj3Siu1nokhs6LanX2KNy9rbvGEhffXlWGBesIT3HQpugvKJ8oH5erYJ9r4J7qLLCSZa gKqQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=OnPWU3zKvAFN5zCINdgJhSm8PRbtANkhAuSAcv2ya+Y=; b=e5fa7kS+QIFNcdm7Bhv2tUTvpfkHiDG9Rb5wSrnafo5rb1dqZyr13X4OGLVUie23bg UXHe9vS4zOC5gANtz9M9t/B/i2aeWqv504mNWOlJQuLwg4jT41HYOcaKAILSOijaDIS5 qU9hMxEORNkQN24IYNIMNeDLczAauvghEypNhOb/nT44YIka8UAWrP6aiE7HuEoEpiBn Mc7t2L6KW3ZFd7eRND851Hc48ujNa83GJmL/Fmvl1xrsbvREqGYHMN3YJJN74DvQoA4l GH/tBTPhR7Wlgze55RuA59xh9izV3JavHeyGZk7Z1GoQB6MH9PjJGq8IQ07WNspvFiKf p0Vw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=IPnxKoll; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 137-20020a63018f000000b0050bf0f0870esi10518876pgb.627.2023.04.10.01.54.22; Mon, 10 Apr 2023 01:54:34 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=IPnxKoll; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229826AbjDJIle (ORCPT + 99 others); Mon, 10 Apr 2023 04:41:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49400 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229705AbjDJIlG (ORCPT ); Mon, 10 Apr 2023 04:41:06 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 68B0E421F; Mon, 10 Apr 2023 01:41:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1681116065; x=1712652065; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=9JZohEEZ3OOYA+uz2zlIPReipLAVM4kv9u7LWFHKKew=; b=IPnxKollasN9H/N8N80ML5eDMsRro/4DvSpwl1ZR9/evAVz0WiQF2oJJ v6dUJC/gwo+AzLpECndUwaIhKFYi/krcOz2tjND1uZiHftF4wD8pi4Cp1 TMQ+I+HO4KpkRoEUnwPxVhiDbrrHU5Y1mA0j/7eiautJf0GJPoysnxlJL 2K7pZ1JmSogCDndfPlB+C5fpW5UoMzpSORK7fHo/du0mwdh+G9b5iTK6A CpJ1TQh4obi08FAEuaMcCAAaUdQr1WP8geC7LNrFQvXlLvPguLwyf2Nid ruCvcmaDahRWGFbPWVIpHNyN9YsLszzqtsr60zfWq8V3EnIQMSOQZVrHh A==; X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="342077934" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="342077934" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2023 01:41:03 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="799436263" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="799436263" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga002.fm.intel.com with ESMTP; 10 Apr 2023 01:41:02 -0700 From: Xin Li To: linux-kernel@vger.kernel.org, x86@kernel.org, kvm@vger.kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, peterz@infradead.org, andrew.cooper3@citrix.com, seanjc@google.com, pbonzini@redhat.com, ravi.v.shankar@intel.com, jiangshanlai@gmail.com, shan.kang@intel.com Subject: [PATCH v8 07/33] x86/opcode: add ERETU, ERETS instructions to x86-opcode-map Date: Mon, 10 Apr 2023 01:14:12 -0700 Message-Id: <20230410081438.1750-8-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230410081438.1750-1-xin3.li@intel.com> References: <20230410081438.1750-1-xin3.li@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.5 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762778807543989663?= X-GMAIL-MSGID: =?utf-8?q?1762778807543989663?= From: "H. Peter Anvin (Intel)" Add the instruction opcodes used by FRED: ERETU, ERETS. Opcode number is per public FRED draft spec v3.0. Signed-off-by: H. Peter Anvin (Intel) Tested-by: Shan Kang Signed-off-by: Xin Li --- arch/x86/lib/x86-opcode-map.txt | 2 +- tools/arch/x86/lib/x86-opcode-map.txt | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/x86/lib/x86-opcode-map.txt b/arch/x86/lib/x86-opcode-map.txt index 5168ee0360b2..7a269e269dc0 100644 --- a/arch/x86/lib/x86-opcode-map.txt +++ b/arch/x86/lib/x86-opcode-map.txt @@ -1052,7 +1052,7 @@ EndTable GrpTable: Grp7 0: SGDT Ms | VMCALL (001),(11B) | VMLAUNCH (010),(11B) | VMRESUME (011),(11B) | VMXOFF (100),(11B) | PCONFIG (101),(11B) | ENCLV (000),(11B) -1: SIDT Ms | MONITOR (000),(11B) | MWAIT (001),(11B) | CLAC (010),(11B) | STAC (011),(11B) | ENCLS (111),(11B) +1: SIDT Ms | MONITOR (000),(11B) | MWAIT (001),(11B) | CLAC (010),(11B) | STAC (011),(11B) | ENCLS (111),(11B) | ERETU (F3),(010),(11B) | ERETS (F2),(010),(11B) 2: LGDT Ms | XGETBV (000),(11B) | XSETBV (001),(11B) | VMFUNC (100),(11B) | XEND (101)(11B) | XTEST (110)(11B) | ENCLU (111),(11B) 3: LIDT Ms 4: SMSW Mw/Rv diff --git a/tools/arch/x86/lib/x86-opcode-map.txt b/tools/arch/x86/lib/x86-opcode-map.txt index 5168ee0360b2..7a269e269dc0 100644 --- a/tools/arch/x86/lib/x86-opcode-map.txt +++ b/tools/arch/x86/lib/x86-opcode-map.txt @@ -1052,7 +1052,7 @@ EndTable GrpTable: Grp7 0: SGDT Ms | VMCALL (001),(11B) | VMLAUNCH (010),(11B) | VMRESUME (011),(11B) | VMXOFF (100),(11B) | PCONFIG (101),(11B) | ENCLV (000),(11B) -1: SIDT Ms | MONITOR (000),(11B) | MWAIT (001),(11B) | CLAC (010),(11B) | STAC (011),(11B) | ENCLS (111),(11B) +1: SIDT Ms | MONITOR (000),(11B) | MWAIT (001),(11B) | CLAC (010),(11B) | STAC (011),(11B) | ENCLS (111),(11B) | ERETU (F3),(010),(11B) | ERETS (F2),(010),(11B) 2: LGDT Ms | XGETBV (000),(11B) | XSETBV (001),(11B) | VMFUNC (100),(11B) | XEND (101)(11B) | XTEST (110)(11B) | ENCLU (111),(11B) 3: LIDT Ms 4: SMSW Mw/Rv From patchwork Mon Apr 10 08:14:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 81395 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1757211vqo; Mon, 10 Apr 2023 01:42:18 -0700 (PDT) X-Google-Smtp-Source: AKy350Zzal0cGINpb5OXr+rGXCmpbIs6dElWmqH15XOo4oIgVKtJt0bys1i1eFFvLLWcMx5w+kT8 X-Received: by 2002:a17:90b:1c91:b0:22b:b832:d32 with SMTP id oo17-20020a17090b1c9100b0022bb8320d32mr13097108pjb.9.1681116138043; Mon, 10 Apr 2023 01:42:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681116138; cv=none; d=google.com; s=arc-20160816; b=0F31wEnDagNv72yn53UekYI8v4iQO0L083+rV21xz9VuV3mpnm6n4MCivfj+Qgw+85 Wn2Bxda9akXzl3/UM9JXXiswvoA2znULemRiynBAWZ5TsgDVe2sw6m7OirxuNR2/ZSH0 pOG4KJFQElNbfdtIKoUmMmrYdCk+184cI63DQhamGtH5XgXETiu2twT3obxqaO5VN+CW 0/UvqxjByQOUS377+ZOB229hqI9xtaKAG9i07uvYya/1kmME6YdIr0Ctsk+aNQPF1fFe OS3zgtDO4+iIg1b04AvSt8yF07U14mMF+31RV1Gjlvs2wXg0QLSOK8sX10/Tme/99fDp M56w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=tkDfrOl/uVtgxQ9PHVt8tndLfQaqM7/GlymaVGt9Xqw=; b=gayEoCvPomtB0Ykh1C4CmTz4biYm4mIvPmhYL/k4BhNLvMkdze7l1h3dlKcGSg2akR N6QfQUVLvh4rj7DjAMEmP4PRi9B713DiLOeFtO3zH4Adr+p1eWEWfEhkfOwc3hBBA1gF sc3RRRTiybbe2dpHpQa1Tic6GyLudul6gT2ckhfJKO71Dce3apLMf+DXSSzZQeRu/QfF bF6a4GP/T9mls2biX9QJPxUYn8FSZQ+0KepIF4iVJsndHIAMmgyw1uY2boNgcmiu+xAA raozlYZ0XtgC4xpUK9bhzvPgvuCFq3ke4dL1ufk+GA/KkXkGJ/PQbjGSuPERdiUap6KS j8gQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=LLrqgmjW; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id hk17-20020a17090b225100b0024489068406si12438623pjb.113.2023.04.10.01.42.05; Mon, 10 Apr 2023 01:42:18 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=LLrqgmjW; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229794AbjDJIlY (ORCPT + 99 others); Mon, 10 Apr 2023 04:41:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49382 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229704AbjDJIlG (ORCPT ); Mon, 10 Apr 2023 04:41:06 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6844D40D9; Mon, 10 Apr 2023 01:41:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1681116065; x=1712652065; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=4lCYxmSNxB+oTxZx5ZOjZmvc7OZS0xLkR1RLWDKH14M=; b=LLrqgmjWy2L3uZZoF3vDx4gp/YMNkJ+zQUB2bJGW8Ac1HQPSscDcoI+l Yxg0VENINCpcfLkgMu/vDY55Js+drF8EouG1HvWJoyLM87vF1kkQ5sJP3 +WopWkulCshowSPSNgXTC3mEBLyT2ax1+mb4C3rKM04lyq7eZOF4sM9R4 I1aKY90K7BiHu8X+ylZF6BXdaf05pUNgEq/JEVSjQVFXXaX5hzg7YE11z NTO5t5J56+aN8YYq6afl48P5uvZi17MophQql4iUh6GkWFc58rOUEGU5v NvPzk3HwgDc4zqIJjAUv1egup10dsChDgIvDcmVQ9pXjFF7tlR4M13I8I Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="342077945" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="342077945" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2023 01:41:03 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="799436266" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="799436266" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga002.fm.intel.com with ESMTP; 10 Apr 2023 01:41:03 -0700 From: Xin Li To: linux-kernel@vger.kernel.org, x86@kernel.org, kvm@vger.kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, peterz@infradead.org, andrew.cooper3@citrix.com, seanjc@google.com, pbonzini@redhat.com, ravi.v.shankar@intel.com, jiangshanlai@gmail.com, shan.kang@intel.com Subject: [PATCH v8 08/33] x86/objtool: teach objtool about ERETU and ERETS Date: Mon, 10 Apr 2023 01:14:13 -0700 Message-Id: <20230410081438.1750-9-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230410081438.1750-1-xin3.li@intel.com> References: <20230410081438.1750-1-xin3.li@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.5 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762778035678635182?= X-GMAIL-MSGID: =?utf-8?q?1762778035678635182?= From: "H. Peter Anvin (Intel)" Update the objtool decoder to know about the ERETU and ERETS instructions (type INSN_CONTEXT_SWITCH.) Signed-off-by: H. Peter Anvin (Intel) Tested-by: Shan Kang Signed-off-by: Xin Li --- tools/objtool/arch/x86/decode.c | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/tools/objtool/arch/x86/decode.c b/tools/objtool/arch/x86/decode.c index 9ef024fd648c..8e9c802f78ec 100644 --- a/tools/objtool/arch/x86/decode.c +++ b/tools/objtool/arch/x86/decode.c @@ -509,11 +509,20 @@ int arch_decode_instruction(struct objtool_file *file, const struct section *sec if (op2 == 0x01) { - if (modrm == 0xca) - insn->type = INSN_CLAC; - else if (modrm == 0xcb) - insn->type = INSN_STAC; - + switch (insn_last_prefix_id(&ins)) { + case INAT_PFX_REPE: + case INAT_PFX_REPNE: + if (modrm == 0xca) + /* eretu/erets */ + insn->type = INSN_CONTEXT_SWITCH; + break; + default: + if (modrm == 0xca) + insn->type = INSN_CLAC; + else if (modrm == 0xcb) + insn->type = INSN_STAC; + break; + } } else if (op2 >= 0x80 && op2 <= 0x8f) { insn->type = INSN_JUMP_CONDITIONAL; From patchwork Mon Apr 10 08:14:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 81415 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1760539vqo; Mon, 10 Apr 2023 01:52:23 -0700 (PDT) X-Google-Smtp-Source: AKy350bC5w6TvYSDFCWMRgFCgnBrUsvTsVXAdSIPkT7b+mrSVb08G1YIFNCItn0RpTJ7MOBYAMB+ X-Received: by 2002:a05:6a20:3b1a:b0:cc:ac05:88f7 with SMTP id c26-20020a056a203b1a00b000ccac0588f7mr6702823pzh.34.1681116742884; Mon, 10 Apr 2023 01:52:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681116742; cv=none; d=google.com; s=arc-20160816; b=tw0MsUnJNnMdw0k6yMsMic2V+TTCNfQy+2X6em30gRAclBNNW2+WVVDbHvQIUeUtW+ 97Hmuj+u1o9BUJxAf8jZ3VGCsLVAt2IFyHJrgMKBSt+i9PeQZkZXyFtvb3eNqc9gWTTh LuEIxWiB890gFD49XIHrcX/qZByw8Ho/OHqaoSaF0BxbsQZluDZ2we8rlUSxL09+EUkg o+nouiIEqkXhvaUZUDtRJQbT16FbQuTXHB++K/dxj/KONhnzeJD/Am78ZjFgCCQ2CCsr q4Ad8cEA6zyuLxiDSfHkqavKo9fZAo277Pfz8qwUmwdhEwuc8WdpPHtE3CmpBpgI6ho4 SoQg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=xiBkMNop4zUxT+iIfOnmWuMSopGG/t7/1Amns2i3v5k=; b=AZhchYObxkyddnYtujzFvb8mnBUV3KC37YjJXcAOn4RBaBpRA+KmiJp4JnmQ07p59/ j+/xY/SbGRh9cLcnn/OmuRdu30OgbTC0bzrmKldPbSbEQUEpreNndUk5HmvByoVSrsyj CuapC0C2QxOqQm5NVAl66usnr+Zt/CePVKC9kMhf7MWc0uv6k5JHII9PCwHEmJ8Pm+Lg A8B+R/81pFfgnuSjX+01gXp2DsNshco7bevi/Op7VMUfXm2kq8RmVNuEWcNfz2A0vva8 0/mslUS0DoFiLwS8o5QXSqohtPYoOyKJ3oKY3GWrvdvRCj6qLIoNMi662klzzRsL/k7C EEig== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Y69myD9Z; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id f6-20020aa79d86000000b006257ed5354bsi10248257pfq.195.2023.04.10.01.52.11; Mon, 10 Apr 2023 01:52:22 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Y69myD9Z; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229839AbjDJIlj (ORCPT + 99 others); Mon, 10 Apr 2023 04:41:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49402 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229707AbjDJIlG (ORCPT ); Mon, 10 Apr 2023 04:41:06 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 17C0C44B1; Mon, 10 Apr 2023 01:41:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1681116066; x=1712652066; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=xePdNXHU682aUfMVCFmcmPRxw9pVvyjrZ2d9bkzRB0U=; b=Y69myD9Z6b+jUs/UiGhCOCgToSWc9BJHgtMulA2rUB0PuibrQU781Z78 +qJ4MM7bLxx8J6VBywC7xkbroZfOfks3REZ4pP9ZZxwlzDOWMWgpM7pBP UHi5RUiZRuwRsqnpM/onQzZ+8RzW/J3KcnOIkkzlIRe9X+6qoqFmEBn4X cm11DNV5r3bst6Y9b69z5tr9XzAz31BStn2yxaMB00ir4bF+HgC/P0vyO F6keW6qx3jJ8oZgp+EDnxBVR3mk751iGOwsDtF3Hla6fdBwFAl2x9ojeN SRjFHCk879oFJkd3wQMDv3MWIJqdNbamNzBGkyM0OzkgrlYtWREMav1kI Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="342077956" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="342077956" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2023 01:41:03 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="799436269" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="799436269" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga002.fm.intel.com with ESMTP; 10 Apr 2023 01:41:03 -0700 From: Xin Li To: linux-kernel@vger.kernel.org, x86@kernel.org, kvm@vger.kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, peterz@infradead.org, andrew.cooper3@citrix.com, seanjc@google.com, pbonzini@redhat.com, ravi.v.shankar@intel.com, jiangshanlai@gmail.com, shan.kang@intel.com Subject: [PATCH v8 09/33] x86/cpu: add X86_CR4_FRED macro Date: Mon, 10 Apr 2023 01:14:14 -0700 Message-Id: <20230410081438.1750-10-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230410081438.1750-1-xin3.li@intel.com> References: <20230410081438.1750-1-xin3.li@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.5 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762778669573793511?= X-GMAIL-MSGID: =?utf-8?q?1762778669573793511?= From: "H. Peter Anvin (Intel)" Add X86_CR4_FRED macro for the FRED bit in %cr4. This bit should be a pinned bit, not to be changed after initialization. Signed-off-by: H. Peter Anvin (Intel) Tested-by: Shan Kang Signed-off-by: Xin Li --- arch/x86/include/uapi/asm/processor-flags.h | 2 ++ arch/x86/kernel/cpu/common.c | 11 ++++++++--- 2 files changed, 10 insertions(+), 3 deletions(-) diff --git a/arch/x86/include/uapi/asm/processor-flags.h b/arch/x86/include/uapi/asm/processor-flags.h index c47cc7f2feeb..a90933f1ff41 100644 --- a/arch/x86/include/uapi/asm/processor-flags.h +++ b/arch/x86/include/uapi/asm/processor-flags.h @@ -132,6 +132,8 @@ #define X86_CR4_PKE _BITUL(X86_CR4_PKE_BIT) #define X86_CR4_CET_BIT 23 /* enable Control-flow Enforcement Technology */ #define X86_CR4_CET _BITUL(X86_CR4_CET_BIT) +#define X86_CR4_FRED_BIT 32 /* enable FRED kernel entry */ +#define X86_CR4_FRED _BITULL(X86_CR4_FRED_BIT) /* * x86-64 Task Priority Register, CR8 diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index 8cd4126d8253..e8cf6f4cfb52 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -412,10 +412,15 @@ static __always_inline void setup_umip(struct cpuinfo_x86 *c) cr4_clear_bits(X86_CR4_UMIP); } -/* These bits should not change their value after CPU init is finished. */ +/* + * These bits should not change their value after CPU init is finished. + * The explicit cast to unsigned long suppresses a warning on i386 for + * x86-64 only feature bits >= 32. + */ static const unsigned long cr4_pinned_mask = - X86_CR4_SMEP | X86_CR4_SMAP | X86_CR4_UMIP | - X86_CR4_FSGSBASE | X86_CR4_CET; + (unsigned long) + (X86_CR4_SMEP | X86_CR4_SMAP | X86_CR4_UMIP | + X86_CR4_FSGSBASE | X86_CR4_CET | X86_CR4_FRED); static DEFINE_STATIC_KEY_FALSE_RO(cr_pinning); static unsigned long cr4_pinned_bits __ro_after_init; From patchwork Mon Apr 10 08:14:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 81412 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1760324vqo; Mon, 10 Apr 2023 01:51:40 -0700 (PDT) X-Google-Smtp-Source: AKy350aXeNEgan6WA19wTa36wtPD543wTzFz4zNrnCTr0Hsyrj6XYVAr1Ezg+8yNlCfKD2BhgVFc X-Received: by 2002:aa7:9739:0:b0:62d:b4ab:2ffb with SMTP id k25-20020aa79739000000b0062db4ab2ffbmr10364075pfg.3.1681116700449; Mon, 10 Apr 2023 01:51:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681116700; cv=none; d=google.com; s=arc-20160816; b=sHU/hin6BlBMJ535K9HP14hbb1WukF8JntAc1vYaMAH/DgEFN6AQdM7dfpfLPvjIUH tIEp6bGXFPXfCini3090XrlToZIU7kajc2TlellaJyFYSpucvycMheLkrLbO20+gQg2h lYwfjJg7hKDQi/QQshuEekGXWPT4uN8EzK4wdXq+cG/vxTfrrnMzVjm3fFaT5DHRFh1b +lJ0Nmuu2T3MnZC0aLmdBL7jlgMlWX16CVW8Zq39qy4eMpqO9Qh87XmjVAOI+vBlLT1k YAIsaPsCidf/IEuYIAFOUmUpt1DNKURcY5zXo9r/JB/D6MEqbgKMeM1hICB44gs6wxQZ ak8g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=P6HtFL7+PoyEnNsypW2XPKGndzZapARWh2X/00BDeqI=; b=bk4X4cTO53RbEeobdaSpeJII0p2cpPaJ1tL1OOqPLNFOYw4MtbYTfvnx9q+3Q0yvRp 7iaq7FPNppj+90AjsWZpgqGlggv5MsXEb9Cg/hzz4pOPg96tjSeBf4WimNSK1UxfK+zu +5V3a+voRwChq+/G9Z2/mlK0lTV9NRag7t/qTNt6biyMJe9cGCgEtR1iWg/G5qVY6xc7 ZlD+f7W542nyRep2wnUAjCnlFL1OnyC0xx44KrD9JDNJ6eiyO43udJpCYxRtLoV2Ha7T j+PMy5e3lqQOigcEqUvEJxeRrSyTY/UDlFxBVGunbkdb7foBaQE73si3sSEfJNlEOh7n Ejqg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="NAJu+B/+"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id a190-20020a6390c7000000b00502f48d80b6si3941277pge.645.2023.04.10.01.51.27; Mon, 10 Apr 2023 01:51:40 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="NAJu+B/+"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229812AbjDJIla (ORCPT + 99 others); Mon, 10 Apr 2023 04:41:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49380 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229562AbjDJIlH (ORCPT ); Mon, 10 Apr 2023 04:41:07 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 26FEB44BD; Mon, 10 Apr 2023 01:41:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1681116066; x=1712652066; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=8lktqN9Ow6nF5dYh+N7FoY+KesvEnknJu52aiBKcpfg=; b=NAJu+B/+tm3SqHHu1LyIZE1pQQX+NbmSuPKQVvrXntNqZN15JOcb0gYp ZflonruSZztjsoxIcc0ivVpr9DlFHfIb7RI2UpPF4yt30Vgl19qTpDI1y mik1MBL+x1PzDdowZDom/RnuZOsNgxRA3WwOCJe36XQRL7U9VZqEexzb8 PrBf5w1kp9GgwlEzYTV+G5SVgtFNz+omXeJU1erTj6gPE5FG8vjbkU/NJ Pihqc+kFy5naBAxMBssxYjX/5lb2A9XQ6xMfkd/gC6zQqwM7lBBfEznmR U9sOBbgXqPcFnQX225HKvnFxGi8FhqTZrJlWuRXvIGvfi/3Jqk6/9xTrM w==; X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="342077966" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="342077966" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2023 01:41:03 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="799436272" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="799436272" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga002.fm.intel.com with ESMTP; 10 Apr 2023 01:41:03 -0700 From: Xin Li To: linux-kernel@vger.kernel.org, x86@kernel.org, kvm@vger.kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, peterz@infradead.org, andrew.cooper3@citrix.com, seanjc@google.com, pbonzini@redhat.com, ravi.v.shankar@intel.com, jiangshanlai@gmail.com, shan.kang@intel.com Subject: [PATCH v8 10/33] x86/fred: add Kconfig option for FRED (CONFIG_X86_FRED) Date: Mon, 10 Apr 2023 01:14:15 -0700 Message-Id: <20230410081438.1750-11-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230410081438.1750-1-xin3.li@intel.com> References: <20230410081438.1750-1-xin3.li@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.5 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762778625409287755?= X-GMAIL-MSGID: =?utf-8?q?1762778625409287755?= From: "H. Peter Anvin (Intel)" Add the configuration option CONFIG_X86_FRED to enable FRED. Signed-off-by: H. Peter Anvin (Intel) Tested-by: Shan Kang Signed-off-by: Xin Li --- arch/x86/Kconfig | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index a825bf031f49..da62178bb246 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -500,6 +500,15 @@ config X86_CPU_RESCTRL Say N if unsure. +config X86_FRED + bool "Flexible Return and Event Delivery" + depends on X86_64 + help + When enabled, try to use Flexible Return and Event Delivery + instead of the legacy SYSCALL/SYSENTER/IDT architecture for + ring transitions and exception/interrupt handling if the + system supports. + if X86_32 config X86_BIGSMP bool "Support for big SMP systems with more than 8 CPUs" From patchwork Mon Apr 10 08:14:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 81421 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1761394vqo; Mon, 10 Apr 2023 01:55:00 -0700 (PDT) X-Google-Smtp-Source: AKy350Y9JcOezyZcwa1V1WyBy6RDR2EAHocbK346aycUIK0gxU5U7ZcLVJ5h/ra/Gnm2qq9PNCBQ X-Received: by 2002:a17:903:2347:b0:1a6:413c:4a54 with SMTP id c7-20020a170903234700b001a6413c4a54mr2784152plh.1.1681116900377; Mon, 10 Apr 2023 01:55:00 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681116900; cv=none; d=google.com; s=arc-20160816; b=gB7CmcJmEe68XSiO7+XKEMB721o07xytZP56kgEKy+EFduxzU6eEsji6l4iH84jvta rnk1VOhs6K2W5odCqq4SBw7c5stXxwPgA2cMoN/2Oy1Vns1s2EDk/8AqNSkRRQxbVWit HIsnsZAVOqFJl54mojJqvJWqq/6HVnH4HBP/ajxQoN/pcU+RekEqLq4rScVWqYgKq3Rd Tyrqh9s3M3cLxTOvU6uUDEvIYXVfwQuHuFvOTjr9kSmMRgy9VcIRAcMOYSSMRyiMnpmW ZTdnpPyUJiA0L6jbqIkMFRANJRCCOIMdFeTejFmgSuFD13dvo/Sr2Eb6I4EdIzBbjYEm CHtA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=UNRomGZrJ3xieh7lf30jl8hFXPb8gU2nVpw4p5i9r5Y=; b=rI3NOpjNjPoyk/lOSYzpytlKzHUMGTNs3X3C6o0Dd5gskS7d2CL+hzjlvcTHgCyGsc IW3jwBSR/Pa9hjvbnfuBpUfvabKvvriS1l9BxZiNCPUgfazjc+opC/cboMsA9xXTM7iW Nx8AH9HWIQSBu0tSFT//ibI2//1QnT7BHIXtCbl2hxekxBtJ6XJsanSBlvQwd4lirvie U0gHzkgESI1zi1Q7LGTt7wdP2E9y82fwq3BnyQ8yv8qVlhtDrI8Xr8TXgAT9b9YCcCcS 9eDlN+vxkfjXzHBSIC58NeOGGnggPjwNPNsGbRX7qx9hVKTcc2M3KTTlJ36E14beUHjC /Dew== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=AcjU1wCh; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id w1-20020a170902e88100b001a1a5dd3505si11324409plg.393.2023.04.10.01.54.48; Mon, 10 Apr 2023 01:55:00 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=AcjU1wCh; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229780AbjDJIlr (ORCPT + 99 others); Mon, 10 Apr 2023 04:41:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49382 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229709AbjDJIlH (ORCPT ); Mon, 10 Apr 2023 04:41:07 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AD60F49E1; Mon, 10 Apr 2023 01:41:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1681116066; x=1712652066; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=QmoU6lab7p0QYpuYROKRU1bljKMMFBm03a1BVvi5hdc=; b=AcjU1wChmapp1rD8UuelJhSI2ZPljWbDAzcBRXKrpfaRp8uf+HZItOzn zW0gHrkP+5K3VpQqbPV1lyp0qle74Y9JGiiGjc+SE1aNLJZ8oapCHWgMU APbxDmcEMMKm1AhsLpkcGZs3EYkE1amDN/cLtuWQVb9j2PJC3T+sELpd/ eqN+V6BkijU3aNtuhDJe0dJalFxaS/e4BO9p6BFVZbkjj1miF1rpYg2sX aNYHynx3OZcKygykJB1LC3WAtOzoWqimSeTPkAGAYxqfEBe32S5UuS3Wv w27UMrkrm802jsbpc83YvhEBYcTprY+RrBlmSUdt79NfdLTHPbx/G1e4e A==; X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="342077977" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="342077977" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2023 01:41:04 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="799436275" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="799436275" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga002.fm.intel.com with ESMTP; 10 Apr 2023 01:41:03 -0700 From: Xin Li To: linux-kernel@vger.kernel.org, x86@kernel.org, kvm@vger.kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, peterz@infradead.org, andrew.cooper3@citrix.com, seanjc@google.com, pbonzini@redhat.com, ravi.v.shankar@intel.com, jiangshanlai@gmail.com, shan.kang@intel.com Subject: [PATCH v8 11/33] x86/fred: if CONFIG_X86_FRED is disabled, disable FRED support Date: Mon, 10 Apr 2023 01:14:16 -0700 Message-Id: <20230410081438.1750-12-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230410081438.1750-1-xin3.li@intel.com> References: <20230410081438.1750-1-xin3.li@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.5 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762778834911435737?= X-GMAIL-MSGID: =?utf-8?q?1762778834911435737?= From: "H. Peter Anvin (Intel)" Add CONFIG_X86_FRED to to make cpu_feature_enabled() work correctly with FRED. Originally-by: Megha Dey Signed-off-by: H. Peter Anvin (Intel) Tested-by: Shan Kang Signed-off-by: Xin Li --- arch/x86/include/asm/disabled-features.h | 8 +++++++- tools/arch/x86/include/asm/disabled-features.h | 8 +++++++- 2 files changed, 14 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/disabled-features.h b/arch/x86/include/asm/disabled-features.h index 5dfa4fb76f4b..56838de9cb23 100644 --- a/arch/x86/include/asm/disabled-features.h +++ b/arch/x86/include/asm/disabled-features.h @@ -99,6 +99,12 @@ # define DISABLE_TDX_GUEST (1 << (X86_FEATURE_TDX_GUEST & 31)) #endif +#ifdef CONFIG_X86_FRED +# define DISABLE_FRED 0 +#else +# define DISABLE_FRED (1 << (X86_FEATURE_FRED & 31)) +#endif + /* * Make sure to add features to the correct mask */ @@ -115,7 +121,7 @@ #define DISABLED_MASK10 0 #define DISABLED_MASK11 (DISABLE_RETPOLINE|DISABLE_RETHUNK|DISABLE_UNRET| \ DISABLE_CALL_DEPTH_TRACKING) -#define DISABLED_MASK12 0 +#define DISABLED_MASK12 (DISABLE_FRED) #define DISABLED_MASK13 0 #define DISABLED_MASK14 0 #define DISABLED_MASK15 0 diff --git a/tools/arch/x86/include/asm/disabled-features.h b/tools/arch/x86/include/asm/disabled-features.h index 5dfa4fb76f4b..56838de9cb23 100644 --- a/tools/arch/x86/include/asm/disabled-features.h +++ b/tools/arch/x86/include/asm/disabled-features.h @@ -99,6 +99,12 @@ # define DISABLE_TDX_GUEST (1 << (X86_FEATURE_TDX_GUEST & 31)) #endif +#ifdef CONFIG_X86_FRED +# define DISABLE_FRED 0 +#else +# define DISABLE_FRED (1 << (X86_FEATURE_FRED & 31)) +#endif + /* * Make sure to add features to the correct mask */ @@ -115,7 +121,7 @@ #define DISABLED_MASK10 0 #define DISABLED_MASK11 (DISABLE_RETPOLINE|DISABLE_RETHUNK|DISABLE_UNRET| \ DISABLE_CALL_DEPTH_TRACKING) -#define DISABLED_MASK12 0 +#define DISABLED_MASK12 (DISABLE_FRED) #define DISABLED_MASK13 0 #define DISABLED_MASK14 0 #define DISABLED_MASK15 0 From patchwork Mon Apr 10 08:14:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 81422 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1761403vqo; Mon, 10 Apr 2023 01:55:02 -0700 (PDT) X-Google-Smtp-Source: AKy350brV761qCM2TezNOD/n/fx33+dLVvUIoneOSOoLsg8iRqWgbf3YWFRBkZhI63Bj0peIIyFg X-Received: by 2002:a17:902:e5ca:b0:19f:2dff:2199 with SMTP id u10-20020a170902e5ca00b0019f2dff2199mr15140364plf.68.1681116901807; Mon, 10 Apr 2023 01:55:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681116901; cv=none; d=google.com; s=arc-20160816; b=CAvXWQgZLF1K1Jyl55uzJSaqQ0K2+nkaHdf39THcOzoH+lAq9mRmEo9WLRdnnKEFYo cIsA1QWvDAfUKBupUijg35s22dWjn4gp0r4zwrpOfAqT5lVuHb8lMWIiU0LhlLVgMWGz EmE2SM5LOGwFwuDh1WCUOR+rQ9EyPc22WOj/vf4gHbGCRBZ8cD3bGyz/EprbdZazcJo5 wt7LnZkFOJVZnoMGAP++htN7FoDto31VBhHa203RBSdqnkiWKHwgsi6t/YfxbJEUjbBP alqaxpMZ2nLgOv8oBlT+t/dtgEI+DAFMd00H/KFQuuzY/FfmAFF6PuGfqV5AnSi+6Itk PnSQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=bxlRTsP9VdEoqzfLEjO6uA1EB+3I6TDY2eyMrxjMlAY=; b=ObbmGplG+uWWCT3CLF/f/xK/VntRMW68YgdEM/EL/jGwUkdDka0gSoyr0PlNZ+JR41 wt/1EjmGbF1p1DTDQZ+q5SqBDZGW6tIFqqnuqY6sNwXeC13ydhIsdGKLkcmR7awVjsQl 7RVopddabglOpXNHnpGnDOgP2nxAiQ5XsAxpKBpyjfd0O794B1t4Pa6m5DxrZ79kAUMB DaAvUuholx2lBvENedENyxsCOPh0AEGoTicu1LBKcEBBh/REyfrhxYWbXl/6LujA5ngo ZcceDvlH1ek3dESF7Kxcrg5/1/hWGO00gtob3J+GnHrsKUZ6e7kkoiyJ9Jy9YZlWiANd ftSQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=H2ZT0gsd; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id kr11-20020a170903080b00b001a6372d95a2si2806401plb.555.2023.04.10.01.54.50; Mon, 10 Apr 2023 01:55:01 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=H2ZT0gsd; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229846AbjDJIln (ORCPT + 99 others); Mon, 10 Apr 2023 04:41:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49404 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229711AbjDJIlH (ORCPT ); Mon, 10 Apr 2023 04:41:07 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D56CC3AB1; Mon, 10 Apr 2023 01:41:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1681116066; x=1712652066; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=9ZDzco+F9bEe6GLOCDycLaH8JYPs/ssjd/Vxju2hLBE=; b=H2ZT0gsd8hc8Q7fHG3KvFxB5OlrrWjXnaOzdGSyXJ8uYIvbCjne5H1z5 JxyGDhp30fcsoKeH/TbFyu0jWaLCjhHdp4Sx5Mp1+22XfCrehiv55hYzz EHNtggfEZIvXeiFU5AW2PeVZL3dNLj/YOmZQ6laxBlEdRjLAR128bbMQ7 CJ1lnhBw8+85v4YhEhgzzbKn7yTLVakber0x7KYipW5Bctx57YJ+X6mro MfDYqAEwIEqw8oKjklgcdIxqQTVH5Cd2V4Ll+23BIyPBaqssVfeJ0oesT xEHq/8x9iGpiPuNCSiTYV0hCiAczlAx+Duj0NWKW/epUYkqhdrDs+K1M+ Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="342077988" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="342077988" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2023 01:41:04 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="799436280" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="799436280" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga002.fm.intel.com with ESMTP; 10 Apr 2023 01:41:03 -0700 From: Xin Li To: linux-kernel@vger.kernel.org, x86@kernel.org, kvm@vger.kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, peterz@infradead.org, andrew.cooper3@citrix.com, seanjc@google.com, pbonzini@redhat.com, ravi.v.shankar@intel.com, jiangshanlai@gmail.com, shan.kang@intel.com Subject: [PATCH v8 12/33] x86/cpu: add MSR numbers for FRED configuration Date: Mon, 10 Apr 2023 01:14:17 -0700 Message-Id: <20230410081438.1750-13-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230410081438.1750-1-xin3.li@intel.com> References: <20230410081438.1750-1-xin3.li@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.5 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762778836123299171?= X-GMAIL-MSGID: =?utf-8?q?1762778836123299171?= From: "H. Peter Anvin (Intel)" Add MSR numbers for the FRED configuration registers. Originally-by: Megha Dey Signed-off-by: H. Peter Anvin (Intel) Tested-by: Shan Kang Signed-off-by: Xin Li --- arch/x86/include/asm/msr-index.h | 13 ++++++++++++- tools/arch/x86/include/asm/msr-index.h | 13 ++++++++++++- 2 files changed, 24 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h index ad35355ee43e..87db728f8bbc 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -36,8 +36,19 @@ #define EFER_FFXSR (1<<_EFER_FFXSR) #define EFER_AUTOIBRS (1<<_EFER_AUTOIBRS) -/* Intel MSRs. Some also available on other CPUs */ +/* FRED MSRs */ +#define MSR_IA32_FRED_RSP0 0x1cc /* Level 0 stack pointer */ +#define MSR_IA32_FRED_RSP1 0x1cd /* Level 1 stack pointer */ +#define MSR_IA32_FRED_RSP2 0x1ce /* Level 2 stack pointer */ +#define MSR_IA32_FRED_RSP3 0x1cf /* Level 3 stack pointer */ +#define MSR_IA32_FRED_STKLVLS 0x1d0 /* Exception stack levels */ +#define MSR_IA32_FRED_SSP0 MSR_IA32_PL0_SSP /* Level 0 shadow stack pointer */ +#define MSR_IA32_FRED_SSP1 0x1d1 /* Level 1 shadow stack pointer */ +#define MSR_IA32_FRED_SSP2 0x1d2 /* Level 2 shadow stack pointer */ +#define MSR_IA32_FRED_SSP3 0x1d3 /* Level 3 shadow stack pointer */ +#define MSR_IA32_FRED_CONFIG 0x1d4 /* Entrypoint and interrupt stack level */ +/* Intel MSRs. Some also available on other CPUs */ #define MSR_TEST_CTRL 0x00000033 #define MSR_TEST_CTRL_SPLIT_LOCK_DETECT_BIT 29 #define MSR_TEST_CTRL_SPLIT_LOCK_DETECT BIT(MSR_TEST_CTRL_SPLIT_LOCK_DETECT_BIT) diff --git a/tools/arch/x86/include/asm/msr-index.h b/tools/arch/x86/include/asm/msr-index.h index ad35355ee43e..87db728f8bbc 100644 --- a/tools/arch/x86/include/asm/msr-index.h +++ b/tools/arch/x86/include/asm/msr-index.h @@ -36,8 +36,19 @@ #define EFER_FFXSR (1<<_EFER_FFXSR) #define EFER_AUTOIBRS (1<<_EFER_AUTOIBRS) -/* Intel MSRs. Some also available on other CPUs */ +/* FRED MSRs */ +#define MSR_IA32_FRED_RSP0 0x1cc /* Level 0 stack pointer */ +#define MSR_IA32_FRED_RSP1 0x1cd /* Level 1 stack pointer */ +#define MSR_IA32_FRED_RSP2 0x1ce /* Level 2 stack pointer */ +#define MSR_IA32_FRED_RSP3 0x1cf /* Level 3 stack pointer */ +#define MSR_IA32_FRED_STKLVLS 0x1d0 /* Exception stack levels */ +#define MSR_IA32_FRED_SSP0 MSR_IA32_PL0_SSP /* Level 0 shadow stack pointer */ +#define MSR_IA32_FRED_SSP1 0x1d1 /* Level 1 shadow stack pointer */ +#define MSR_IA32_FRED_SSP2 0x1d2 /* Level 2 shadow stack pointer */ +#define MSR_IA32_FRED_SSP3 0x1d3 /* Level 3 shadow stack pointer */ +#define MSR_IA32_FRED_CONFIG 0x1d4 /* Entrypoint and interrupt stack level */ +/* Intel MSRs. Some also available on other CPUs */ #define MSR_TEST_CTRL 0x00000033 #define MSR_TEST_CTRL_SPLIT_LOCK_DETECT_BIT 29 #define MSR_TEST_CTRL_SPLIT_LOCK_DETECT BIT(MSR_TEST_CTRL_SPLIT_LOCK_DETECT_BIT) From patchwork Mon Apr 10 08:14:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 81413 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1760356vqo; Mon, 10 Apr 2023 01:51:48 -0700 (PDT) X-Google-Smtp-Source: AKy350ZUiy0B4Gk641voLFjCxOqhZ2cHF77U7n/u7tZDSKbPdSDGR+jazT2GPEZo02z+/6Q9pyaq X-Received: by 2002:a17:902:cf46:b0:1a1:cc91:c43c with SMTP id e6-20020a170902cf4600b001a1cc91c43cmr7054466plg.45.1681116708549; Mon, 10 Apr 2023 01:51:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681116708; cv=none; d=google.com; s=arc-20160816; b=wlxSeyWbdk4nngrExRJ6rC1WD1+/uuq8RMczRd5ooq58WFGRKB6y0zRGcklZGkIMHV cjFU3lfpgLS3OW+8V9PABHSFB+avFd2wrkrbnuHlhZriGpv394F16VxnzyDoRGS34Vwe q83V8amklXMdfrUFZJrE25weJ/3PxfuHGBEb3svTARHlt9saibgGG9Mymm40nS2Gx0NA G+feYbrniFxtzqQEINN2al0HwtgOq8hLedVANiILJQWQSVxSoU3P4us/9o7DyzwETLKC 5TnzeFqEuFVO3rrGdxgSm2XYFylP+uAvyyLj/AdK4iMkJha7HzOjxWqvdH1FN7NzLD9a mbYw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=+VaUPr04RvGnPD9G9LEI7rZpFLR5XlVkh+7KmqRc0j4=; b=M4pd7TWhJwK8NoEKbfHz82n8xkc+6+uJVTXyf0d/WEPdG+p0Of7uMUosKlRqKWGhCi vpxyxMAmgitxJ6j91JLBDJI+eeIMDIK/x6tDGeUsyWmYLGtKeHkM9y1mAOg2tpF/nkdI h1Gog23Che7WkYfGVmSpuagh5bzVkdf62LXEjc4LIsooi0a+qw4dEM+wRYqqmM4KzrIB RE1H5iYa22cBZB5yxN8kLEV8T6mHoGavE5R8HD9wcAB1yE1aGMgnZTHtp+6Gjkhs2EZu 4kjvHrD6bmGtYgnkCOPrFl/CSgEo9c3F/A/IG6LQXUDAY6FQ6Mynv+W+DmLatBVD1Z6L USiw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="Utar/TK/"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id j3-20020a170903028300b001a52dd0195csi5078116plr.549.2023.04.10.01.51.36; Mon, 10 Apr 2023 01:51:48 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="Utar/TK/"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229889AbjDJIlu (ORCPT + 99 others); Mon, 10 Apr 2023 04:41:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49402 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229718AbjDJIlI (ORCPT ); Mon, 10 Apr 2023 04:41:08 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 291E349E8; Mon, 10 Apr 2023 01:41:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1681116067; x=1712652067; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=tP9iIQUbrhMc+5l0AHCXnkXnhn3TgNNmkDoitFhCFbw=; b=Utar/TK/+i43J9dbjAhnIt4MDS8soZtUnjpWqV7clDz35gTV5xfslFq4 //YDA8Kxn5YT+KGS0hPv+4hDnQJLEIoXqacwUdt+yLEZY2jVzTu6SUFr2 /AE6ftaz7ckEbkP9VTJHoGYikbRxtilf6wPtMjaJcZdvi6v5MdE5SIqq0 hbMM5MI983OCruYBbj52AhN/FQww7s8JUP9/DzhXRjXKf/+Wi2E4TULDZ 62SekSqx4N19Yk+fGfafVJTlqvU0HP7+1Ssu5qin+GeKm/S6mv7eMrXJl t9uPSUXrCPLtjTffu8VynvqsG7lugDa8DETasdC+8uozjHGumeq36fWfJ A==; X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="342077999" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="342077999" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2023 01:41:04 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="799436286" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="799436286" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga002.fm.intel.com with ESMTP; 10 Apr 2023 01:41:04 -0700 From: Xin Li To: linux-kernel@vger.kernel.org, x86@kernel.org, kvm@vger.kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, peterz@infradead.org, andrew.cooper3@citrix.com, seanjc@google.com, pbonzini@redhat.com, ravi.v.shankar@intel.com, jiangshanlai@gmail.com, shan.kang@intel.com Subject: [PATCH v8 13/33] x86/fred: header file for event types Date: Mon, 10 Apr 2023 01:14:18 -0700 Message-Id: <20230410081438.1750-14-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230410081438.1750-1-xin3.li@intel.com> References: <20230410081438.1750-1-xin3.li@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.5 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762778633888646071?= X-GMAIL-MSGID: =?utf-8?q?1762778633888646071?= FRED inherits the Intel VT-x enhancement of classified events with a two-level event dispatch logic. The first-level dispatch is on the event type, not the event vector as used in the IDT architecture. This also means that vectors in different event types are orthogonal, e.g., vectors 0x10-0x1f become available as hardware interrupts. Add a header file for event types, and also use it in . Suggested-by: H. Peter Anvin (Intel) Tested-by: Shan Kang Signed-off-by: Xin Li --- arch/x86/include/asm/event-type.h | 17 +++++++++++++++++ arch/x86/include/asm/vmx.h | 17 +++++++++-------- 2 files changed, 26 insertions(+), 8 deletions(-) create mode 100644 arch/x86/include/asm/event-type.h diff --git a/arch/x86/include/asm/event-type.h b/arch/x86/include/asm/event-type.h new file mode 100644 index 000000000000..fedaa0e492c5 --- /dev/null +++ b/arch/x86/include/asm/event-type.h @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_X86_EVENT_TYPE_H +#define _ASM_X86_EVENT_TYPE_H + +/* + * Event type codes: these are the same that are used by VTx. + */ +#define EVENT_TYPE_HWINT 0 /* Maskable external interrupt */ +#define EVENT_TYPE_RESERVED 1 +#define EVENT_TYPE_NMI 2 /* Non-maskable interrupt */ +#define EVENT_TYPE_HWFAULT 3 /* Hardware exceptions (e.g., page fault) */ +#define EVENT_TYPE_SWINT 4 /* Software interrupt (INT n) */ +#define EVENT_TYPE_PRIVSW 5 /* INT1 (ICEBP) */ +#define EVENT_TYPE_SWFAULT 6 /* Software exception (INT3 or INTO) */ +#define EVENT_TYPE_OTHER 7 /* FRED: SYSCALL/SYSENTER */ + +#endif diff --git a/arch/x86/include/asm/vmx.h b/arch/x86/include/asm/vmx.h index 498dc600bd5c..8d9b8b0d8e56 100644 --- a/arch/x86/include/asm/vmx.h +++ b/arch/x86/include/asm/vmx.h @@ -15,6 +15,7 @@ #include #include #include +#include #include #define VMCS_CONTROL_BIT(x) BIT(VMX_FEATURE_##x & 0x1f) @@ -372,14 +373,14 @@ enum vmcs_field { #define VECTORING_INFO_DELIVER_CODE_MASK INTR_INFO_DELIVER_CODE_MASK #define VECTORING_INFO_VALID_MASK INTR_INFO_VALID_MASK -#define INTR_TYPE_EXT_INTR (0 << 8) /* external interrupt */ -#define INTR_TYPE_RESERVED (1 << 8) /* reserved */ -#define INTR_TYPE_NMI_INTR (2 << 8) /* NMI */ -#define INTR_TYPE_HARD_EXCEPTION (3 << 8) /* processor exception */ -#define INTR_TYPE_SOFT_INTR (4 << 8) /* software interrupt */ -#define INTR_TYPE_PRIV_SW_EXCEPTION (5 << 8) /* ICE breakpoint - undocumented */ -#define INTR_TYPE_SOFT_EXCEPTION (6 << 8) /* software exception */ -#define INTR_TYPE_OTHER_EVENT (7 << 8) /* other event */ +#define INTR_TYPE_EXT_INTR (EVENT_TYPE_HWINT << 8) /* external interrupt */ +#define INTR_TYPE_RESERVED (EVENT_TYPE_RESERVED << 8) /* reserved */ +#define INTR_TYPE_NMI_INTR (EVENT_TYPE_NMI << 8) /* NMI */ +#define INTR_TYPE_HARD_EXCEPTION (EVENT_TYPE_HWFAULT << 8) /* processor exception */ +#define INTR_TYPE_SOFT_INTR (EVENT_TYPE_SWINT << 8) /* software interrupt */ +#define INTR_TYPE_PRIV_SW_EXCEPTION (EVENT_TYPE_PRIVSW << 8) /* ICE breakpoint - undocumented */ +#define INTR_TYPE_SOFT_EXCEPTION (EVENT_TYPE_SWFAULT << 8) /* software exception */ +#define INTR_TYPE_OTHER_EVENT (EVENT_TYPE_OTHER << 8) /* other event */ /* GUEST_INTERRUPTIBILITY_INFO flags. */ #define GUEST_INTR_STATE_STI 0x00000001 From patchwork Mon Apr 10 08:14:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 81397 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1757466vqo; Mon, 10 Apr 2023 01:42:57 -0700 (PDT) X-Google-Smtp-Source: AKy350YVuMyYOcNaw0DqTSEf/3V/NwlyVJOMjjUM4M+fVsKBC1MzHmST9VRfUqOFMBGmelms6ffb X-Received: by 2002:a17:90b:3809:b0:23f:7666:c8a1 with SMTP id mq9-20020a17090b380900b0023f7666c8a1mr12782616pjb.18.1681116177341; Mon, 10 Apr 2023 01:42:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681116177; cv=none; d=google.com; s=arc-20160816; b=v6aACWzhVzT8KRnicdweaZhcLe8+pO7tgYC9cD7phZDylF6jZW3ptlN//Kn4pEIZxY 9eqc/dPhMo580MYPLU8iFbaOlmwLvHgHTiaXHSFkG+HA40QVl4EFmcN0cSXK+FSChRiY 6oX0dsHZShk/hiAhDkF4OBkR9+luvt8hQ5vLZXJEilAC75XVKJEQAVXwIjEBGYs+Ilnp epMru8xbdutGwAI3k2WqSCc0h81meHMaIT0RHowcAktVyZunjrkJBNLTrUIhWck/zL5s OJr+rv14V2oE8fXGnQ/BSxzwph7+uJzzR8ZFq3ZpOGveErM2NGyi+RuoVncN2GU/bE7m c/hA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=RRd0UBdwKjamp5FLwk0MN5YETEbG7sJ4jhrhEPxF8Ss=; b=aSS4uw3VCxCHFiVDgcZOptXWaB+3PQgKXVId24oXdWlwjkrkYVBgSaALFnuVvXxM1M Ol2mE6+lomnExxaHdQoBieqCt2xRQAyhLDJpZVkJ1WvwwEuEpxsoPiPYRwxCnHzDfvp/ lCGUR+7UzvX8bXUMBK5ruX2Ri/8LuEtV0xdc2fQZ99iG0554l1yT8wVqWOQwuAfouxhR 29u2q8l0xgfe6uWjQdILbCa+7ATNGpFK/Fu9J8HNga75aBF0QR4EXHXuW14AIdQQFae1 OAOry5XM1r3VDYFYxfJK4MXtLQAiUogyiPFraoCnvyI8UyMsa8WQ8pDxVNJ/ZybIBi4g u7ew== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Z2RctO55; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id l38-20020a635b66000000b0050fa02c0326si9585607pgm.277.2023.04.10.01.42.45; Mon, 10 Apr 2023 01:42:57 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Z2RctO55; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229837AbjDJIl4 (ORCPT + 99 others); Mon, 10 Apr 2023 04:41:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49382 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229722AbjDJIlI (ORCPT ); Mon, 10 Apr 2023 04:41:08 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 72BA1421F; Mon, 10 Apr 2023 01:41:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1681116067; x=1712652067; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=SDGDSUDsSV7zT/y15Zj5Xr5rd8es//ZPrYL5DTYxefY=; b=Z2RctO55Cr0EW+CrOefb4eTSW8cMR0I816SMsRz7UijYHa/RCqGGqfaF bGjZ92XlRJIxgBYVuDjpVFrURJBVxtJT4f9+kobVguZs1cm+78zptzEVs s4B/y6Glvxj5gK/MpVVi9kTTw4EQJwJmzO+MoZrykbhTGJvPNwyzzqfmg Fix7eG+Pcn9kDupuwmet+xJ/qgtdPPsJ88NPFZsuHPxDBLPwbQCSNOsLD ZBAfHUpULGZUcDmwuOAKj757QqTb5EA4P1fjtPjk9naptQwrs1cDV3lQF y/MCGyuqM0nTTdUsZKhQ3l1ZW3QiQI578eiOFvGEoFbvMSBTVSsaFtlZc w==; X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="342078009" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="342078009" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2023 01:41:04 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="799436292" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="799436292" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga002.fm.intel.com with ESMTP; 10 Apr 2023 01:41:04 -0700 From: Xin Li To: linux-kernel@vger.kernel.org, x86@kernel.org, kvm@vger.kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, peterz@infradead.org, andrew.cooper3@citrix.com, seanjc@google.com, pbonzini@redhat.com, ravi.v.shankar@intel.com, jiangshanlai@gmail.com, shan.kang@intel.com Subject: [PATCH v8 14/33] x86/fred: header file with FRED definitions Date: Mon, 10 Apr 2023 01:14:19 -0700 Message-Id: <20230410081438.1750-15-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230410081438.1750-1-xin3.li@intel.com> References: <20230410081438.1750-1-xin3.li@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.5 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762778076681876109?= X-GMAIL-MSGID: =?utf-8?q?1762778076681876109?= From: "H. Peter Anvin (Intel)" Add a header file for FRED prototypes and definitions. Signed-off-by: H. Peter Anvin (Intel) Tested-by: Shan Kang Signed-off-by: Xin Li --- Changes since v6: * Replace pt_regs csx flags prefix FRED_CSL_ with FRED_CSX_. --- arch/x86/include/asm/fred.h | 106 ++++++++++++++++++++++++++++++++++++ 1 file changed, 106 insertions(+) create mode 100644 arch/x86/include/asm/fred.h diff --git a/arch/x86/include/asm/fred.h b/arch/x86/include/asm/fred.h new file mode 100644 index 000000000000..b59397411ab9 --- /dev/null +++ b/arch/x86/include/asm/fred.h @@ -0,0 +1,106 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * arch/x86/include/asm/fred.h + * + * Macros for Flexible Return and Event Delivery (FRED) + */ + +#ifndef ASM_X86_FRED_H +#define ASM_X86_FRED_H + +#ifdef CONFIG_X86_FRED + +#include +#include + +/* + * FRED return instructions + * + * Replace with "ERETS"/"ERETU" once binutils support FRED return instructions. + * The binutils version supporting FRED instructions is still TBD, and will + * update once we have it. + */ +#define ERETS _ASM_BYTES(0xf2,0x0f,0x01,0xca) +#define ERETU _ASM_BYTES(0xf3,0x0f,0x01,0xca) + +/* + * RSP is aligned to a 64-byte boundary before used to push a new stack frame + */ +#define FRED_STACK_FRAME_RSP_MASK _AT(unsigned long, (~0x3f)) + +/* + * Event stack level macro for the FRED_STKLVLS MSR. + * Usage example: FRED_STKLVL(X86_TRAP_DF, 3) + * Multiple values can be ORd together. + */ +#define FRED_STKLVL(v,l) (_AT(unsigned long, l) << (2*(v))) + +/* FRED_CONFIG MSR */ +#define FRED_CONFIG_CSL_MASK 0x3 +#define FRED_CONFIG_REDZONE_AMOUNT 1 /* measured in 64-byte cache lines */ +#define FRED_CONFIG_REDZONE (_AT(unsigned long, FRED_CONFIG_REDZONE_AMOUNT) << 6) +#define FRED_CONFIG_INT_STKLVL(l) (_AT(unsigned long, l) << 9) +#define FRED_CONFIG_ENTRYPOINT(p) _AT(unsigned long, (p)) + +/* FRED event type and vector bit width and counts */ +#define FRED_EVENT_TYPE_BITS 3 /* only 3 bits used in FRED 3.0 */ +#define FRED_EVENT_TYPE_COUNT _BITUL(FRED_EVENT_TYPE_BITS) +#define FRED_EVENT_VECTOR_BITS 8 +#define FRED_EVENT_VECTOR_COUNT _BITUL(FRED_EVENT_VECTOR_BITS) + +/* FRED EVENT_TYPE_OTHER vector numbers */ +#define FRED_SYSCALL 1 +#define FRED_SYSENTER 2 + +/* Flags above the CS selector (regs->csx) */ +#define FRED_CSX_ENABLE_NMI _BITUL(28) +#define FRED_CSX_ALLOW_SINGLE_STEP _BITUL(25) +#define FRED_CSX_INTERRUPT_SHADOW _BITUL(24) + +#ifndef __ASSEMBLY__ + +#include +#include + +/* FRED stack frame information */ +struct fred_info { + unsigned long edata; /* Event data: CR2, DR6, ... */ + unsigned long resv; +}; + +/* Full format of the FRED stack frame */ +struct fred_frame { + struct pt_regs regs; + struct fred_info info; +}; + +/* Getting the FRED frame information from a pt_regs pointer */ +static __always_inline struct fred_info *fred_info(struct pt_regs *regs) +{ + return &container_of(regs, struct fred_frame, regs)->info; +} + +static __always_inline unsigned long fred_event_data(struct pt_regs *regs) +{ + return fred_info(regs)->edata; +} + +/* + * How FRED event handlers are called. + * + * FRED event delivery establishes the full supervisor context + * by pushing everything related to the event being delivered + * to the FRED stack frame, e.g., the faulting linear address + * of a #PF is pushed as event data of the FRED #PF stack frame. + * Thus a struct pt_regs has everything needed and it's the only + * input parameter required for a FRED event handler. + */ +#define DECLARE_FRED_HANDLER(f) void f (struct pt_regs *regs) +#define DEFINE_FRED_HANDLER(f) noinstr DECLARE_FRED_HANDLER(f) +typedef DECLARE_FRED_HANDLER((*fred_handler)); + +#endif /* __ASSEMBLY__ */ + +#endif /* CONFIG_X86_FRED */ + +#endif /* ASM_X86_FRED_H */ From patchwork Mon Apr 10 08:14:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 81411 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1759510vqo; Mon, 10 Apr 2023 01:49:13 -0700 (PDT) X-Google-Smtp-Source: AKy350YTPT16poTsMN5btSCL09YciZiDtIc7k/r+kOqlIqzbJRcwfpZ3QAX0dQN7O6mAgbSzHfbZ X-Received: by 2002:a17:903:11c5:b0:1a2:9a94:2f00 with SMTP id q5-20020a17090311c500b001a29a942f00mr14681753plh.27.1681116552986; Mon, 10 Apr 2023 01:49:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681116552; cv=none; d=google.com; s=arc-20160816; b=XYvZW5wTPNTfGfvW4K8mlRYkZm+60YJOrinpetUoA0YlvjT0dSlAcet/Dk35lLd1qU tw3nW3yiQjbfBVTaMa7eoXewVS5CGe9UDLp8UxRy4wrISc1TKe95j4xznE66KV+oeTcu MU69ymakaeO3rCTW05sR0vq2vJB7qZ28pUd40pf5G0ziAmhpPbenOW5lBC1EuQcC3cjf 9l4MDK6HLQcjtmYNMOmU9U8ZYWWGHtS4tDt+ugPFW4pFsTYGuWac2sEU7x0QZfSlyvO6 9lly6nQevnQXNV1GdJWUdXAAjf2AkT2tY8mMw36Na0U91ljOlQW6Cea+Ps12lmnjBZR6 aS8Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=N4FXxoAND7jQBjEqpShHPjc4hHLflyZ6xveUxpk88v4=; b=NdpEmR/biFzCFzyn28lAjZkTl8o4tgIvDtdMfOiD2tT8FyE2+1EyFyq+XcCGuGWLOr 6LmE81wTl6KwV9FTj/vn9L167DX///IaYWCIr47rYV2abelmxqwKVP6/XJnh0nwaJA+D dNiPREdDlj1/RXE6N4jnanMjj0qBdBcnBxwIDWQuxtqEgU6LekncKQaNSfh6eyN6HuAW xHC38QzRH46qLvvmRJ7bCvkzapZT1TK1phuiI5KOgYq3+RcpW6p5C0eLf7SQH5HZ84pG ZleQ3IsGsSY+09zBurihUwtQjCx5vcgvXI6eBpGiuFPJsbJ/a7CfhjyTAFzIlRBqQhIh lxNw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=CcxZ40RU; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id j3-20020a170903028300b001a52dd0195csi5078116plr.549.2023.04.10.01.49.01; Mon, 10 Apr 2023 01:49:12 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=CcxZ40RU; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229900AbjDJIlx (ORCPT + 99 others); Mon, 10 Apr 2023 04:41:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49380 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229721AbjDJIlI (ORCPT ); Mon, 10 Apr 2023 04:41:08 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 916EE49D9; Mon, 10 Apr 2023 01:41:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1681116067; x=1712652067; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=voFQVjFSRNzDP3of726cqSfwzN6e3VmowuyZWcsyuKM=; b=CcxZ40RUpjRQDkaBJVclQUXd5tTACuh+CWWPLtKOIMjvkz3wnH/RNOaL kT3tqm74YFqiP54XPTOjLBaQ1YZR0dO9briIxq1Ag5qpD9dB2VEZMq8yX dnIlEN0vzE8sT9WsYc2kHF8w0xGZrrcnmC5P5Bqzh31ipqsAl9PIaOb/l qSStoJJhxDvVtO2LEIBurH6qM1ApLUjnzrvoDL3EQS1AJ2usTpjljDQv7 A3h6QHjPMHBMcq2ixwGtkHVrt+OpVvF+g04OdMxCJZqJzH3kGylpZsqU7 xjxWaUfYXTLNoTShWqR4AmSgexX/QLKH0sfZ9bQCTO38guqxBQtdi3tXC Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="342078020" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="342078020" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2023 01:41:05 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="799436298" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="799436298" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga002.fm.intel.com with ESMTP; 10 Apr 2023 01:41:04 -0700 From: Xin Li To: linux-kernel@vger.kernel.org, x86@kernel.org, kvm@vger.kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, peterz@infradead.org, andrew.cooper3@citrix.com, seanjc@google.com, pbonzini@redhat.com, ravi.v.shankar@intel.com, jiangshanlai@gmail.com, shan.kang@intel.com Subject: [PATCH v8 15/33] x86/fred: reserve space for the FRED stack frame Date: Mon, 10 Apr 2023 01:14:20 -0700 Message-Id: <20230410081438.1750-16-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230410081438.1750-1-xin3.li@intel.com> References: <20230410081438.1750-1-xin3.li@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.5 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762778470377536474?= X-GMAIL-MSGID: =?utf-8?q?1762778470377536474?= From: "H. Peter Anvin (Intel)" When using FRED, reserve space at the top of the stack frame, just like i386 does. A future version of FRED might have dynamic frame sizes, though, in which case it might be necessary to make TOP_OF_KERNEL_STACK_PADDING a variable instead of a constant. Signed-off-by: H. Peter Anvin (Intel) Tested-by: Shan Kang Signed-off-by: Xin Li --- arch/x86/include/asm/thread_info.h | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/arch/x86/include/asm/thread_info.h b/arch/x86/include/asm/thread_info.h index f1cccba52eb9..998483078d5f 100644 --- a/arch/x86/include/asm/thread_info.h +++ b/arch/x86/include/asm/thread_info.h @@ -31,7 +31,9 @@ * In vm86 mode, the hardware frame is much longer still, so add 16 * bytes to make room for the real-mode segments. * - * x86_64 has a fixed-length stack frame. + * x86-64 has a fixed-length stack frame, but it depends on whether + * or not FRED is enabled. Future versions of FRED might make this + * dynamic, but for now it is always 2 words longer. */ #ifdef CONFIG_X86_32 # ifdef CONFIG_VM86 @@ -39,8 +41,12 @@ # else # define TOP_OF_KERNEL_STACK_PADDING 8 # endif -#else -# define TOP_OF_KERNEL_STACK_PADDING 0 +#else /* x86-64 */ +# ifdef CONFIG_X86_FRED +# define TOP_OF_KERNEL_STACK_PADDING (2*8) +# else +# define TOP_OF_KERNEL_STACK_PADDING 0 +# endif #endif /* From patchwork Mon Apr 10 08:14:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 81414 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1760432vqo; Mon, 10 Apr 2023 01:52:03 -0700 (PDT) X-Google-Smtp-Source: AKy350ZhaKIZyzOdVYm5FtE6Gd6UoFDYBheib/O6DTd8oi5oXs1c8gre3J1FxWmCZAyjb++9kEem X-Received: by 2002:a62:1bd4:0:b0:632:34ef:7669 with SMTP id b203-20020a621bd4000000b0063234ef7669mr7792159pfb.7.1681116723712; Mon, 10 Apr 2023 01:52:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681116723; cv=none; d=google.com; s=arc-20160816; b=XeTiBCdmU4ZzH8Rjkb9g1gMrv0VZ7bv78ir8s9nTb4qzwMNc78Xb8Xs2DcSY6bIIyA RH88aP/izilAtg6FTcjJiKnSKGD1K8MdSyW3h7mxEdOTrr9Jx2BIFQJBW/4nqQQfFHbS c1vM0PxZsJo2wGT3CvVayHiDP+OVc4lNE8bnuAYZmK/xQYwGM8kIXcg+SPNiQAla3hVZ lhn6hMebUHAIb0GvXvFmUlOAp69qeRZMMoN4p7xFfXiu3LMg0VNdE75HvSsJFyW6k+bc JMfP9zvaCpO9mf/nJtoA823y5KwWfqJkVmfpdjHcPCrA4y3s6DrJuRUgLzmoj/0BTBiu 3bEQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=M5bV6szz0XuXm6oCNHc4S7Y0o7foTuWrMoJq3PC0c4s=; b=FmU/Aw7N+bzSA8/fYJTtvCC+QaUaiXj4TxUbsGd9Ug5A5G3RF3hLFv3YFCMSkJm1Cn bD0B7932g6k02OJbhCWOzO9JMMqn19N2yBrKAiIfN0hCvF2DdBsUtDLsuyz9LKe9CANe JZ6BK+M4QjAQDFOtprwVE0Nic/1JqiL5x3FlW369myJLj1DPdOxT+RbWywrOzDhYGG9C wYIhruarDA+XQR2YJ9ycd4TIL4DiGNSxt4dVOsvZgpJWLkHNGxOUZTDuTpLxFJfMKf6z pjsW6BUYpVPV0VvrS5FwSb0EQ59hvBnZNjan5G/7fuNclgZW4Cz0Iuww9PD3c5YEnuPY yZJg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=FZSl3XE0; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id a190-20020a6390c7000000b00502f48d80b6si3941277pge.645.2023.04.10.01.51.52; Mon, 10 Apr 2023 01:52:03 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=FZSl3XE0; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229915AbjDJIl6 (ORCPT + 99 others); Mon, 10 Apr 2023 04:41:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49404 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229728AbjDJIlJ (ORCPT ); Mon, 10 Apr 2023 04:41:09 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 39F1B49FC; Mon, 10 Apr 2023 01:41:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1681116068; x=1712652068; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=3h63lm4e6EBF71B+XPLJAYkLnyTMlvSI36lVPpSC6fU=; b=FZSl3XE0db/Cky7PTtjEsD26w29fc7ZYwbxn4+L7Lam/TrWuJVVFe7l8 vrwNWok+SDqVgJ/W50IPiZd+UzeG9BcQCHbq0EZuDsQAd2YVdg2T+Tjbq rjfWGerxe28yHtrwEt2DNBYoLj9XmlUgVDGL0brGfenauU8qIugg+bJcZ n4BFnCDR8aQyHuGdYAcpBDpG4pn2UytlKKa4ICyRLrJG7jE2MYVyoQqn4 NKHYJ1OHt7E0xSXCdc74FE9oK9/8G1FT+m/2oK3cy6u3adN/MMG7s+1ad xVgs03ioVb5wQea3cwllZrc/1klRTuDJhpUkFuwGXes3Q9hrugft6AQUw A==; X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="342078032" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="342078032" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2023 01:41:05 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="799436302" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="799436302" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga002.fm.intel.com with ESMTP; 10 Apr 2023 01:41:04 -0700 From: Xin Li To: linux-kernel@vger.kernel.org, x86@kernel.org, kvm@vger.kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, peterz@infradead.org, andrew.cooper3@citrix.com, seanjc@google.com, pbonzini@redhat.com, ravi.v.shankar@intel.com, jiangshanlai@gmail.com, shan.kang@intel.com Subject: [PATCH v8 16/33] x86/fred: add a page fault entry stub for FRED Date: Mon, 10 Apr 2023 01:14:21 -0700 Message-Id: <20230410081438.1750-17-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230410081438.1750-1-xin3.li@intel.com> References: <20230410081438.1750-1-xin3.li@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.5 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762778649642718076?= X-GMAIL-MSGID: =?utf-8?q?1762778649642718076?= From: "H. Peter Anvin (Intel)" Add a page fault entry stub for FRED. On a FRED system, the faulting address (CR2) is passed on the stack, to avoid the problem of transient state. Thus we get the page fault address from the stack instead of CR2. Signed-off-by: H. Peter Anvin (Intel) Tested-by: Shan Kang Signed-off-by: Xin Li --- arch/x86/include/asm/fred.h | 2 ++ arch/x86/mm/fault.c | 20 ++++++++++++++++++-- 2 files changed, 20 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/fred.h b/arch/x86/include/asm/fred.h index b59397411ab9..4ff05d350066 100644 --- a/arch/x86/include/asm/fred.h +++ b/arch/x86/include/asm/fred.h @@ -99,6 +99,8 @@ static __always_inline unsigned long fred_event_data(struct pt_regs *regs) #define DEFINE_FRED_HANDLER(f) noinstr DECLARE_FRED_HANDLER(f) typedef DECLARE_FRED_HANDLER((*fred_handler)); +DECLARE_FRED_HANDLER(fred_exc_page_fault); + #endif /* __ASSEMBLY__ */ #endif /* CONFIG_X86_FRED */ diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c index a498ae1fbe66..0f946121de14 100644 --- a/arch/x86/mm/fault.c +++ b/arch/x86/mm/fault.c @@ -33,6 +33,7 @@ #include /* kvm_handle_async_pf */ #include /* fixup_vdso_exception() */ #include +#include /* fred_event_data() */ #define CREATE_TRACE_POINTS #include @@ -1507,9 +1508,10 @@ handle_page_fault(struct pt_regs *regs, unsigned long error_code, } } -DEFINE_IDTENTRY_RAW_ERRORCODE(exc_page_fault) +static __always_inline void page_fault_common(struct pt_regs *regs, + unsigned int error_code, + unsigned long address) { - unsigned long address = read_cr2(); irqentry_state_t state; prefetchw(¤t->mm->mmap_lock); @@ -1556,3 +1558,17 @@ DEFINE_IDTENTRY_RAW_ERRORCODE(exc_page_fault) irqentry_exit(regs, state); } + +DEFINE_IDTENTRY_RAW_ERRORCODE(exc_page_fault) +{ + page_fault_common(regs, error_code, read_cr2()); +} + +#ifdef CONFIG_X86_FRED + +DEFINE_FRED_HANDLER(fred_exc_page_fault) +{ + page_fault_common(regs, regs->orig_ax, fred_event_data(regs)); +} + +#endif /* CONFIG_X86_FRED */ From patchwork Mon Apr 10 08:14:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 81418 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1760778vqo; Mon, 10 Apr 2023 01:52:56 -0700 (PDT) X-Google-Smtp-Source: AKy350a9GEpM6x+YoqdhJX6sVqf76r7G4BcP70cgi872jZwKe4HsgySVXxddj2ZEcvE2edbeeG/B X-Received: by 2002:aa7:9a42:0:b0:626:1523:b10d with SMTP id x2-20020aa79a42000000b006261523b10dmr7349427pfj.4.1681116776471; Mon, 10 Apr 2023 01:52:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681116776; cv=none; d=google.com; s=arc-20160816; b=wPFMhyH/jMSCreEXoGM7P4TEwiBRPveoMM/2Rwut/Bz5njQkgAys/Qti0sx7EN3rwe lgJyzCqpep3Wb+ym49MJ/ovsmcrXF6hETNyRYxdZC5oUapY/sKdOkLei6Nvce2XnUO1o 5iB51GYHTBSGWo1QbKJEgm3JGhwhA57pz2glHCrsn9DsYvgVJShbwWzWi+KjLj82GPvD IgVNspYMdvp1GoqNWOwhJxl9eyc7X7Yx+w+xT+T3F0YBv4eyzdyK2sDenAgEKgG1Sand MY4zhZ2qkkxJq1UFOhdQcZrC615ztbrFOKxHb6iyjjBY7e5sN7Yyfjc4bxckC59OjXKD XaeA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=6Z0IE2XqP9vwd8ByVTpsUWrJgdNXimuX8wyggK0V93I=; b=KYiv/2CX/iRw0XR3souW4kLG5A8+R/0U2b/3+Q2jGoYj6Icc5pRiufZolho1DILVu/ pFgDwdx0fIr5JS4yl1i6EbLzFtZCsBwvxCd6mfZvWzQAUUe4Rl8eSCIiR/ddhu5nvkIP 5ooUs71PqA6zXEDLuw+IsDsmvv7hkc26FLbky3G1nfNRqcT7mrle1SXcolqADChFvLZ6 7n9uijYrFPJ0z8esgk66SBh3ZsInOuTueRWKyFD9frZTmpM9EuziOkfgbwbtoVAY+llP pWH3DPeA+tE6hcTquQ1C0yiI9uzhZPj/R8ZAKBK26AoSlnrx+vACEats5INAhDe3EZRo 7t0A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=W75aq+h2; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id s8-20020a632c08000000b00513a4ba9ca0si10087318pgs.722.2023.04.10.01.52.44; Mon, 10 Apr 2023 01:52:56 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=W75aq+h2; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229925AbjDJImE (ORCPT + 99 others); Mon, 10 Apr 2023 04:42:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49544 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229731AbjDJIlJ (ORCPT ); Mon, 10 Apr 2023 04:41:09 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8187C270E; Mon, 10 Apr 2023 01:41:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1681116068; x=1712652068; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=AvdDrUuw1vqZ65Zb2UmH3skaRlzxRNLvEln+YufnpOQ=; b=W75aq+h2qPxmCsJEi6aoRZEqzui6zJfUF9cvOULGWMI8n1B/8hZrDOhd +/LwRTB4uBlTcXEn2ZkbqXQUmB5qUAXipb3DjF4loUxonXER6OWnMGp7M k1JKfhfOrvcA32PBwF1hr0Xh3a/MKWzOLHsdJvI3vcuj04M8lOW6FTb/b jTIaAcdT98cU5lpGi5HPVaSJPuJbF6eEdcSIEwd3tHyYRpktHOzJtuVrg Kcs9WVk34fyf4+u0ZECjVaAeUQg54dvwe83JvPjWhUWlEzei3PMZz44p4 KOdrqhC7alZQyCWsIvMZIKI1JYG+2RlhbvjOhG+KLp/+jeFZf90YUzIga A==; X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="342078043" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="342078043" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2023 01:41:05 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="799436307" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="799436307" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga002.fm.intel.com with ESMTP; 10 Apr 2023 01:41:05 -0700 From: Xin Li To: linux-kernel@vger.kernel.org, x86@kernel.org, kvm@vger.kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, peterz@infradead.org, andrew.cooper3@citrix.com, seanjc@google.com, pbonzini@redhat.com, ravi.v.shankar@intel.com, jiangshanlai@gmail.com, shan.kang@intel.com Subject: [PATCH v8 17/33] x86/fred: add a debug fault entry stub for FRED Date: Mon, 10 Apr 2023 01:14:22 -0700 Message-Id: <20230410081438.1750-18-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230410081438.1750-1-xin3.li@intel.com> References: <20230410081438.1750-1-xin3.li@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.5 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762778705280091304?= X-GMAIL-MSGID: =?utf-8?q?1762778705280091304?= From: "H. Peter Anvin (Intel)" Add a debug fault entry stub for FRED. On a FRED system, the debug trap status information (DR6) is passed on the stack, to avoid the problem of transient state. Furthermore, FRED transitions avoid a lot of ugly corner cases the handling of which can, and should be, skipped. The FRED debug trap status information saved on the stack differs from DR6 in both stickiness and polarity; it is exactly what debug_read_clear_dr6() returns, and exc_debug_user()/exc_debug_kernel() expect. Signed-off-by: H. Peter Anvin (Intel) Tested-by: Shan Kang Signed-off-by: Xin Li --- Changes since v1: * call irqentry_nmi_{enter,exit}() in both IDT and FRED debug fault kernel handler (Peter Zijlstra). --- arch/x86/include/asm/fred.h | 1 + arch/x86/kernel/traps.c | 56 +++++++++++++++++++++++++++---------- 2 files changed, 42 insertions(+), 15 deletions(-) diff --git a/arch/x86/include/asm/fred.h b/arch/x86/include/asm/fred.h index 4ff05d350066..f670430aaa54 100644 --- a/arch/x86/include/asm/fred.h +++ b/arch/x86/include/asm/fred.h @@ -99,6 +99,7 @@ static __always_inline unsigned long fred_event_data(struct pt_regs *regs) #define DEFINE_FRED_HANDLER(f) noinstr DECLARE_FRED_HANDLER(f) typedef DECLARE_FRED_HANDLER((*fred_handler)); +DECLARE_FRED_HANDLER(fred_exc_debug); DECLARE_FRED_HANDLER(fred_exc_page_fault); #endif /* __ASSEMBLY__ */ diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c index f86cd233b00b..549f7f962f8f 100644 --- a/arch/x86/kernel/traps.c +++ b/arch/x86/kernel/traps.c @@ -47,6 +47,7 @@ #include #include #include +#include #include #include #include @@ -1020,21 +1021,9 @@ static bool notify_debug(struct pt_regs *regs, unsigned long *dr6) return false; } -static __always_inline void exc_debug_kernel(struct pt_regs *regs, - unsigned long dr6) +static __always_inline void debug_kernel_common(struct pt_regs *regs, + unsigned long dr6) { - /* - * Disable breakpoints during exception handling; recursive exceptions - * are exceedingly 'fun'. - * - * Since this function is NOKPROBE, and that also applies to - * HW_BREAKPOINT_X, we can't hit a breakpoint before this (XXX except a - * HW_BREAKPOINT_W on our stack) - * - * Entry text is excluded for HW_BP_X and cpu_entry_area, which - * includes the entry stack is excluded for everything. - */ - unsigned long dr7 = local_db_save(); irqentry_state_t irq_state = irqentry_nmi_enter(regs); instrumentation_begin(); @@ -1062,7 +1051,8 @@ static __always_inline void exc_debug_kernel(struct pt_regs *regs, * Catch SYSENTER with TF set and clear DR_STEP. If this hit a * watchpoint at the same time then that will still be handled. */ - if ((dr6 & DR_STEP) && is_sysenter_singlestep(regs)) + if (!cpu_feature_enabled(X86_FEATURE_FRED) && + (dr6 & DR_STEP) && is_sysenter_singlestep(regs)) dr6 &= ~DR_STEP; /* @@ -1090,7 +1080,25 @@ static __always_inline void exc_debug_kernel(struct pt_regs *regs, out: instrumentation_end(); irqentry_nmi_exit(regs, irq_state); +} +static __always_inline void exc_debug_kernel(struct pt_regs *regs, + unsigned long dr6) +{ + /* + * Disable breakpoints during exception handling; recursive exceptions + * are exceedingly 'fun'. + * + * Since this function is NOKPROBE, and that also applies to + * HW_BREAKPOINT_X, we can't hit a breakpoint before this (XXX except a + * HW_BREAKPOINT_W on our stack) + * + * Entry text is excluded for HW_BP_X and cpu_entry_area, which + * includes the entry stack is excluded for everything. + */ + unsigned long dr7 = local_db_save(); + + debug_kernel_common(regs, dr6); local_db_restore(dr7); } @@ -1179,6 +1187,24 @@ DEFINE_IDTENTRY_DEBUG_USER(exc_debug) { exc_debug_user(regs, debug_read_clear_dr6()); } + +# ifdef CONFIG_X86_FRED +DEFINE_FRED_HANDLER(fred_exc_debug) +{ + /* + * The FRED debug information saved onto stack differs from + * DR6 in both stickiness and polarity; it is exactly what + * debug_read_clear_dr6() returns. + */ + unsigned long dr6 = fred_event_data(regs); + + if (user_mode(regs)) + exc_debug_user(regs, dr6); + else + debug_kernel_common(regs, dr6); +} +# endif /* CONFIG_X86_FRED */ + #else /* 32 bit does not have separate entry points. */ DEFINE_IDTENTRY_RAW(exc_debug) From patchwork Mon Apr 10 08:14:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 81416 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1760753vqo; Mon, 10 Apr 2023 01:52:54 -0700 (PDT) X-Google-Smtp-Source: AKy350Z0yW08/BfjvsFbtqrc6Y8fcKQb1BxCt931I/cE8pyIPd0YX+K81lLk/tVi0Y2GfbixtLR5 X-Received: by 2002:a05:6a20:e0a:b0:db:1119:bb48 with SMTP id ej10-20020a056a200e0a00b000db1119bb48mr8307901pzb.59.1681116773994; Mon, 10 Apr 2023 01:52:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681116773; cv=none; d=google.com; s=arc-20160816; b=eeZ9na3TdviH4vVmLKydIEGI+vdnFmEa/3VbATr0r3On96UVHdI42GBseM4zji5RXW xNM7XYvJk8rR02/lOIGQvkFanGbQTYOcydTmq2YID4QP4lV/Dec1SF3nyWk9HN/dEgwT idlTdVE51FmUteA8jNgMEzAkYLgxtqDhqTS4kijRs71x44Y4cTYF+iPF8yfIjS/YafUs Nj9prOJ4bj+TlWsH5hAi0EfAmDdTxh4vzunb0oLcZkdvc/b+d2GQDO1WCpHEgSQ+jgCb oe0h/HlhUOsNzi1fRFMsrIxkNn3Jk41oBluQBOCbW4uJ/D8JLyqgER9Pwq1scnkmGhzr go8g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=FujSQUqHBZl8iLTALotlYQnjAeVJMmty12OH3VL/Yzw=; b=E6Vco1fnFlnVe10kwtApIkhebXClm+rIq7C3X4XZUz9q985UEsCATF1gKpT0ZxhMo6 XjAcKEU8KrZ7SUui5siX2ZyxmLUbuM7aTzMXLQlxFxWLihD/3xA4sw7DCnoRk1gY3LCx McTuxpGI6AY8rPuL6P5tP9yAv/8RulJsQpxaiMmo9r9G6aojwyXgEad7rZoR+tO1y8xX xnkXrnWX16Vf9o41nxlg5m+ODu7wetRlnvSat7JneJAts7xPQ7j37iXiHSNvdTISnYRA smSGPNhCU1uC+gpiMgxR17WBJEjXtlecUO93ab2J/JwH7iU5qSfskLYK5f9UGp31iVAu NsNg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=gLHoUALl; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id s26-20020a63ff5a000000b0051372ef848fsi10125567pgk.697.2023.04.10.01.52.42; Mon, 10 Apr 2023 01:52:53 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=gLHoUALl; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229920AbjDJImB (ORCPT + 99 others); Mon, 10 Apr 2023 04:42:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49546 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229732AbjDJIlJ (ORCPT ); Mon, 10 Apr 2023 04:41:09 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8196F422B; Mon, 10 Apr 2023 01:41:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1681116068; x=1712652068; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=8g9fYneHGWezeTK24y61/oxUFG9XbGt0W4kIun9/SEQ=; b=gLHoUALlm9pwg6IbV3YmKv8ICsHbyBpWJPuhHHTcdATcSRPq8unQikH8 L6zt4Ohz1o7JOeiN7mGZJuYfkprz7IPvzznHk1IHpW+SPrbRJ3cRDCnja uHRTcPKhjDG5iCXNflXANJi+oYEm8iGVmb8kxfovQN2P3XZcyST/tfyc6 bFXmGn2upMiLPaRxRKSClPMtWt3VKRCpSVtgJSfjYtZBG/QHhFGGXMwqg e0b/KKRY9kCZtaJIp2l7WBTF6ezalXcvqhZtxfNutXlZwyE8fQbCSdcwD RoszxfJMTYQ0G1dBD9FYt7pjrslycAQIs5j9eg27xT3awYYcL41T5iSGc Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="342078058" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="342078058" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2023 01:41:05 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="799436310" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="799436310" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga002.fm.intel.com with ESMTP; 10 Apr 2023 01:41:05 -0700 From: Xin Li To: linux-kernel@vger.kernel.org, x86@kernel.org, kvm@vger.kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, peterz@infradead.org, andrew.cooper3@citrix.com, seanjc@google.com, pbonzini@redhat.com, ravi.v.shankar@intel.com, jiangshanlai@gmail.com, shan.kang@intel.com Subject: [PATCH v8 18/33] x86/fred: add a NMI entry stub for FRED Date: Mon, 10 Apr 2023 01:14:23 -0700 Message-Id: <20230410081438.1750-19-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230410081438.1750-1-xin3.li@intel.com> References: <20230410081438.1750-1-xin3.li@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.5 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762778702815074778?= X-GMAIL-MSGID: =?utf-8?q?1762778702815074778?= From: "H. Peter Anvin (Intel)" On a FRED system, NMIs nest both with themselves and faults, transient information is saved into the stack frame, and NMI unblocking only happens when the stack frame indicates that so should happen. Thus, the NMI entry stub for FRED is really quite small... Signed-off-by: H. Peter Anvin (Intel) Tested-by: Shan Kang Signed-off-by: Xin Li --- arch/x86/include/asm/fred.h | 1 + arch/x86/kernel/nmi.c | 19 +++++++++++++++++++ 2 files changed, 20 insertions(+) diff --git a/arch/x86/include/asm/fred.h b/arch/x86/include/asm/fred.h index f670430aaa54..9ce2a6439091 100644 --- a/arch/x86/include/asm/fred.h +++ b/arch/x86/include/asm/fred.h @@ -99,6 +99,7 @@ static __always_inline unsigned long fred_event_data(struct pt_regs *regs) #define DEFINE_FRED_HANDLER(f) noinstr DECLARE_FRED_HANDLER(f) typedef DECLARE_FRED_HANDLER((*fred_handler)); +DECLARE_FRED_HANDLER(fred_exc_nmi); DECLARE_FRED_HANDLER(fred_exc_debug); DECLARE_FRED_HANDLER(fred_exc_page_fault); diff --git a/arch/x86/kernel/nmi.c b/arch/x86/kernel/nmi.c index 776f4b1e395b..e7b2abe42583 100644 --- a/arch/x86/kernel/nmi.c +++ b/arch/x86/kernel/nmi.c @@ -34,6 +34,7 @@ #include #include #include +#include #define CREATE_TRACE_POINTS #include @@ -643,6 +644,24 @@ void nmi_backtrace_stall_check(const struct cpumask *btp) #endif +#ifdef CONFIG_X86_FRED +DEFINE_FRED_HANDLER(fred_exc_nmi) +{ + /* + * With FRED, CR2 and DR6 are pushed atomically on faults, + * so we don't have to worry about saving and restoring them. + * Breakpoint faults nest, so assume it is OK to leave DR7 + * enabled. + */ + irqentry_state_t irq_state = irqentry_nmi_enter(regs); + + inc_irq_stat(__nmi_count); + default_do_nmi(regs); + + irqentry_nmi_exit(regs, irq_state); +} +#endif + void stop_nmi(void) { ignore_nmis++; From patchwork Mon Apr 10 08:14:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 81401 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1757917vqo; Mon, 10 Apr 2023 01:44:10 -0700 (PDT) X-Google-Smtp-Source: AKy350YkGTPi7F27IX1r8ZEDxS/P1BgerNx5ddEYXq6BlaWivADH8ykjmoytOAbowE8yJTFusqAN X-Received: by 2002:a17:902:f683:b0:1a2:76b6:c26a with SMTP id l3-20020a170902f68300b001a276b6c26amr13992342plg.28.1681116250468; Mon, 10 Apr 2023 01:44:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681116250; cv=none; d=google.com; s=arc-20160816; b=eR6KS0ZjCBQlclRcZ5x7FWYa4vTWqak0hDGga+HQpg7P1wYB8Qz1XAUqtRXPQ6YYvf CB/8c9OKnAruCXjyZuLSeQis9byRHPAhUuW4nr0j+YI5s9yE6uoW/TpSAf2JOYB/ndcy flUIYMQ141jxA7VXDuQXnBD3fLGMLopPYEkIqKLAQ5ZKQPgkgTt+I1qUiBsGz8XJG7ny EiNjnON7iAR8Tvy/A5Pk9r7lsifswBXCmt8bnvIela20KpHy/NYHSHMa4iqF0JMaMy54 xX0Po4J2dNYvTQGeq6rhu2Yqm5ifXNNA6Ntv/RbLm718p6eryaCY+HaoRuZDzIMlqhdb uJjg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=noHnDU6HVdIEvG2vFbd/tjfbHM4UcQUOqh8bLYYP0WE=; b=sico9CI7Nc33iYuwxhZqMkQtdZmtZ2VFYr/t54018Ma2PcbiV0XXABoKuIAabz8K0D u9o+CYsoKOW7i9lPRy5bYMsjGSURu7prRLzGVM6c6FnYlwb2ODb/Z4tvaevbL673FJVF +m8tVYYyhO8rb5ty2x9Q3mCbvHobRHi5enqaYYHyGphItm6UF0qBiS5UmG7xDSOqhjBL JACCUO7Dxbhnw5OWv/jsiGQu8OLHUXSL+16wBv9sd51fZUwXZ1Td0+BBilz46S9YPQXu u6HDD1IfWeplv9qYByqOGi1N4w3gGIFeXdlb6teaw9E4Zgb+DIbViA4MI83c4zecMLCw aYxg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=QNEa48xu; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id j3-20020a170903028300b001a52dd0195csi5078116plr.549.2023.04.10.01.43.58; Mon, 10 Apr 2023 01:44:10 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=QNEa48xu; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229861AbjDJImI (ORCPT + 99 others); Mon, 10 Apr 2023 04:42:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49576 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229716AbjDJIlK (ORCPT ); Mon, 10 Apr 2023 04:41:10 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0539540D9; Mon, 10 Apr 2023 01:41:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1681116068; x=1712652068; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=CKPdbZePoSn8pAkjI3IJ+p0apbz+y9vUOwaC2M88CbM=; b=QNEa48xuXVShKlXTtrOpI3V/yV3Mg+Funib0bM1/+kkHydBGNe/8oIQI zGw5Eao2vixIlIH959HxFRdhOGlo3mZOMu9eIAzFgLBxf+OB6NKP9855x QC7uBP9bjj921Qna8vPSZ1U9X+Fyfxi26pkBc6mKNcldtVmsCGCeoIhjp h3Rc8qX09EHuYY+k/BJk/BzMb/d/S9jGHGAPD9570wHeBjO5xTewOJrUP kgiPk9m6bwY1sAmpOpv7QmX8AZfQEIWTv6JXN1B+7ByyP6Vbe6QdFtEd/ CH65Umi9+JJtbIg3hKKlhBX9FIsPS//ZkYXlwPuQCppysJV5iVj6rZdTB w==; X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="342078068" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="342078068" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2023 01:41:05 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="799436315" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="799436315" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga002.fm.intel.com with ESMTP; 10 Apr 2023 01:41:05 -0700 From: Xin Li To: linux-kernel@vger.kernel.org, x86@kernel.org, kvm@vger.kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, peterz@infradead.org, andrew.cooper3@citrix.com, seanjc@google.com, pbonzini@redhat.com, ravi.v.shankar@intel.com, jiangshanlai@gmail.com, shan.kang@intel.com Subject: [PATCH v8 19/33] x86/fred: add a machine check entry stub for FRED Date: Mon, 10 Apr 2023 01:14:24 -0700 Message-Id: <20230410081438.1750-20-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230410081438.1750-1-xin3.li@intel.com> References: <20230410081438.1750-1-xin3.li@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.5 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762778153201384953?= X-GMAIL-MSGID: =?utf-8?q?1762778153201384953?= Add a machine check entry stub for FRED. Tested-by: Shan Kang Signed-off-by: Xin Li --- Changes since v5: * Disallow #DB inside #MCE for robustness sake (Peter Zijlstra). --- arch/x86/include/asm/fred.h | 1 + arch/x86/kernel/cpu/mce/core.c | 15 +++++++++++++++ 2 files changed, 16 insertions(+) diff --git a/arch/x86/include/asm/fred.h b/arch/x86/include/asm/fred.h index 9ce2a6439091..61048aa4e01d 100644 --- a/arch/x86/include/asm/fred.h +++ b/arch/x86/include/asm/fred.h @@ -102,6 +102,7 @@ typedef DECLARE_FRED_HANDLER((*fred_handler)); DECLARE_FRED_HANDLER(fred_exc_nmi); DECLARE_FRED_HANDLER(fred_exc_debug); DECLARE_FRED_HANDLER(fred_exc_page_fault); +DECLARE_FRED_HANDLER(fred_exc_machine_check); #endif /* __ASSEMBLY__ */ diff --git a/arch/x86/kernel/cpu/mce/core.c b/arch/x86/kernel/cpu/mce/core.c index 2eec60f50057..859331a6a7ad 100644 --- a/arch/x86/kernel/cpu/mce/core.c +++ b/arch/x86/kernel/cpu/mce/core.c @@ -52,6 +52,7 @@ #include #include #include +#include #include "internal.h" @@ -2111,6 +2112,20 @@ DEFINE_IDTENTRY_MCE_USER(exc_machine_check) exc_machine_check_user(regs); local_db_restore(dr7); } + +#ifdef CONFIG_X86_FRED +DEFINE_FRED_HANDLER(fred_exc_machine_check) +{ + unsigned long dr7; + + dr7 = local_db_save(); + if (user_mode(regs)) + exc_machine_check_user(regs); + else + exc_machine_check_kernel(regs); + local_db_restore(dr7); +} +#endif #else /* 32bit unified entry point */ DEFINE_IDTENTRY_RAW(exc_machine_check) From patchwork Mon Apr 10 08:14:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 81398 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1757610vqo; Mon, 10 Apr 2023 01:43:19 -0700 (PDT) X-Google-Smtp-Source: AKy350YylrNqBl0ULOW+8UUdp+f57i94i5QzfHVD0N2DlwNYzw3Q/9b2Qpi/7hjyzu7b/JUzbxZk X-Received: by 2002:a05:6a20:465c:b0:d8:f312:b3b with SMTP id eb28-20020a056a20465c00b000d8f3120b3bmr10307844pzb.3.1681116199532; Mon, 10 Apr 2023 01:43:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681116199; cv=none; d=google.com; s=arc-20160816; b=BYp1BkqliheygkDotDekkwgNsC/stc8+GNp+JPmlYyrLLFQMJKrGabVEYmjIo2y1qN 3DOPA1xJRrL0Q1e5a34haYdpoVODDPP2OEzp8q78YJ/4GeYcZ9Gfz6FZYhs8LcJQK8+2 9Ql4dQKwBRYs2HoExkHzVSlnmMC4mm6AZSaDjv1nIz2tiTUPiAIbjkpdwbHIuCEbXPTy L8NYpBmyzFigRCbMg9vC/iLBcUonklwZI/xfP0TNkP8nHuxM0EXz+b9SO5X8kanT7+XV W/VRf8fc7NpfdGxGZUiXB54oP2Ndj9ndfuTbfl+79q8XZf5erst0TpnvM3WNKXqUk3iO duEw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=vXN/9P6+Xojvdb9LZm3JSykFOa6yS/vI18dScvJDQbM=; b=lQAPDbGBvsb/hedZ/kYtLRwsKPLKV7N9LeKsedLxNmUnrBcQYZWUGSog1tT4MhN9GW W8BAf8lRp+g8Bnhe7iF+GSbjRhPcIsJN90y11rzwzzsebto1Wr+XVkQBsK4L3Gov5n3B nxshEEePR06vWQnbXNOmgBguRnipvQyjwAl+Lve5NvAE2lyoUDfrD1pWsp9M8qKJJJXs twBmQTb1xYaDDS0kDTLplJ9iGWEfIoPOUMlYG5gtXz3+Le3E8DKj6Z+IdoW7XU5+ERu7 1r/nL3Sb1mhmXhmx2ijVdMZvR7wn3BwtpLhMjiKm401e2xOyXHm0n5iVyP6VOBnKo6M6 Wvxg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=edwZV2nD; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id bv184-20020a632ec1000000b004fc39adf481si3865034pgb.381.2023.04.10.01.43.07; Mon, 10 Apr 2023 01:43:19 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=edwZV2nD; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229866AbjDJImL (ORCPT + 99 others); Mon, 10 Apr 2023 04:42:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50606 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229781AbjDJIlX (ORCPT ); Mon, 10 Apr 2023 04:41:23 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 929464C01; Mon, 10 Apr 2023 01:41:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1681116069; x=1712652069; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=l3AVgkrKnnVka/gI97o5Xu9nI/YK4IHnOff+UwX2eH0=; b=edwZV2nDJ7dpLJjr2qzknVVkg58MacGsNfw/OqMcfdsA4mAhtbZvWun8 Y3Ge316i8tr3LrgsGcfR7u2Cv3x+A9RvALp5BtVywfUfND4BSw3/ZDeEi MTlzomHzjvHwNHmxV3Gd+ZoyEt7QT0bhG53HTU3qNgneEVmqx2yhtim9D mUh4VA62/Yl+FGhUhTJiOCe9ruAWvEjWn8m0JoYxF3My1w7J/jozUJK9F cuwvAxrrJYbUjsW6r1Hcu4xAl+kDASXC6yz0+I3AI+e6bcNvN1H7JhTzw 0qmJf2+l97kxJiBrPZdlAKqhm+i18ID+AvlJ2TcLlen5n+kQmiNjPFbnj A==; X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="342078083" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="342078083" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2023 01:41:06 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="799436321" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="799436321" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga002.fm.intel.com with ESMTP; 10 Apr 2023 01:41:05 -0700 From: Xin Li To: linux-kernel@vger.kernel.org, x86@kernel.org, kvm@vger.kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, peterz@infradead.org, andrew.cooper3@citrix.com, seanjc@google.com, pbonzini@redhat.com, ravi.v.shankar@intel.com, jiangshanlai@gmail.com, shan.kang@intel.com Subject: [PATCH v8 20/33] x86/fred: FRED entry/exit and dispatch code Date: Mon, 10 Apr 2023 01:14:25 -0700 Message-Id: <20230410081438.1750-21-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230410081438.1750-1-xin3.li@intel.com> References: <20230410081438.1750-1-xin3.li@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.5 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762778099942651405?= X-GMAIL-MSGID: =?utf-8?q?1762778099942651405?= From: "H. Peter Anvin (Intel)" The code to actually handle kernel and event entry/exit using FRED. It is split up into two files thus: - entry_64_fred.S contains the actual entrypoints and exit code, and saves and restores registers. - entry_fred.c contains the two-level event dispatch code for FRED. The first-level dispatch is on the event type, and the second-level is on the event vector. Originally-by: Megha Dey Signed-off-by: H. Peter Anvin (Intel) Co-developed-by: Xin Li Tested-by: Shan Kang Signed-off-by: Xin Li --- Changes since v1: * Initialize a FRED exception handler to fred_bad_event() instead of NULL if no FRED handler defined for an exception vector (Peter Zijlstra). * Push calling irqentry_{enter,exit}() and instrumentation_{begin,end}() down into individual FRED exception handlers, instead of in the dispatch framework (Peter Zijlstra). --- arch/x86/entry/Makefile | 5 +- arch/x86/entry/entry_64_fred.S | 57 +++++++++ arch/x86/entry/entry_fred.c | 220 ++++++++++++++++++++++++++++++++ arch/x86/include/asm/idtentry.h | 8 ++ 4 files changed, 289 insertions(+), 1 deletion(-) create mode 100644 arch/x86/entry/entry_64_fred.S create mode 100644 arch/x86/entry/entry_fred.c diff --git a/arch/x86/entry/Makefile b/arch/x86/entry/Makefile index ca2fe186994b..c93e7f5c2a06 100644 --- a/arch/x86/entry/Makefile +++ b/arch/x86/entry/Makefile @@ -18,6 +18,9 @@ obj-y += vdso/ obj-y += vsyscall/ obj-$(CONFIG_PREEMPTION) += thunk_$(BITS).o +CFLAGS_entry_fred.o += -fno-stack-protector +CFLAGS_REMOVE_entry_fred.o += -pg $(CC_FLAGS_FTRACE) +obj-$(CONFIG_X86_FRED) += entry_64_fred.o entry_fred.o + obj-$(CONFIG_IA32_EMULATION) += entry_64_compat.o syscall_32.o obj-$(CONFIG_X86_X32_ABI) += syscall_x32.o - diff --git a/arch/x86/entry/entry_64_fred.S b/arch/x86/entry/entry_64_fred.S new file mode 100644 index 000000000000..d975cacd060f --- /dev/null +++ b/arch/x86/entry/entry_64_fred.S @@ -0,0 +1,57 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * arch/x86/entry/entry_64_fred.S + * + * The actual FRED entry points. + */ +#include +#include +#include +#include + +#include "calling.h" + + .code64 + .section ".noinstr.text", "ax" + +.macro FRED_ENTER + UNWIND_HINT_EMPTY + PUSH_AND_CLEAR_REGS + movq %rsp, %rdi /* %rdi -> pt_regs */ +.endm + +.macro FRED_EXIT + UNWIND_HINT_REGS + POP_REGS + addq $8,%rsp /* Drop error code */ +.endm + +/* + * The new RIP value that FRED event delivery establishes is + * IA32_FRED_CONFIG & ~FFFH for events that occur in ring 3. + * Thus the FRED ring 3 entry point must be 4K page aligned. + */ + .align 4096 + +SYM_CODE_START_NOALIGN(fred_entrypoint_user) + FRED_ENTER + call fred_entry_from_user +SYM_INNER_LABEL(fred_exit_user, SYM_L_GLOBAL) + FRED_EXIT + ERETU +SYM_CODE_END(fred_entrypoint_user) + +.fill fred_entrypoint_kernel - ., 1, 0xcc + +/* + * The new RIP value that FRED event delivery establishes is + * (IA32_FRED_CONFIG & ~FFFH) + 256 for events that occur in + * ring 0, i.e., fred_entrypoint_user + 256. + */ + .org fred_entrypoint_user+256 +SYM_CODE_START_NOALIGN(fred_entrypoint_kernel) + FRED_ENTER + call fred_entry_from_kernel + FRED_EXIT + ERETS +SYM_CODE_END(fred_entrypoint_kernel) diff --git a/arch/x86/entry/entry_fred.c b/arch/x86/entry/entry_fred.c new file mode 100644 index 000000000000..0bff1db913b8 --- /dev/null +++ b/arch/x86/entry/entry_fred.c @@ -0,0 +1,220 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * arch/x86/entry/entry_fred.c + * + * This contains the dispatch functions called from the entry point + * assembly. + */ + +#include +#include /* oops_begin/end, ... */ +#include +#include +#include +#include +#include +#include +#include +#include + +/* + * Badness... + */ +static DEFINE_FRED_HANDLER(fred_bad_event) +{ + irqentry_state_t irq_state = irqentry_nmi_enter(regs); + + instrumentation_begin(); + + /* Panic on events from a high stack level */ + if (regs->current_stack_level > 0) { + pr_emerg("PANIC: invalid or fatal FRED event; event type %u " + "vector %u error 0x%lx aux 0x%lx at %04x:%016lx\n", + regs->type, regs->vector, regs->orig_ax, + fred_event_data(regs), regs->cs, regs->ip); + die("invalid or fatal FRED event", regs, regs->orig_ax); + panic("invalid or fatal FRED event"); + } else { + unsigned long flags = oops_begin(); + int sig = SIGKILL; + + pr_alert("BUG: invalid or fatal FRED event; event type %u " + "vector %u error 0x%lx aux 0x%lx at %04x:%016lx\n", + regs->type, regs->vector, regs->orig_ax, + fred_event_data(regs), regs->cs, regs->ip); + + if (__die("Invalid or fatal FRED event", regs, regs->orig_ax)) + sig = 0; + + oops_end(flags, regs, sig); + } + + instrumentation_end(); + irqentry_nmi_exit(regs, irq_state); +} + +noinstr void fred_exc_double_fault(struct pt_regs *regs) +{ + exc_double_fault(regs, regs->orig_ax); +} + +/* + * Exception entry + */ +static DEFINE_FRED_HANDLER(fred_exception) +{ + /* + * Exceptions that cannot happen on FRED h/w are set to fred_bad_event(). + */ + static const fred_handler exception_handlers[NUM_EXCEPTION_VECTORS] = { + [X86_TRAP_DE] = exc_divide_error, + [X86_TRAP_DB] = fred_exc_debug, + [X86_TRAP_NMI] = fred_bad_event, /* A separate event type, not handled here */ + [X86_TRAP_BP] = exc_int3, + [X86_TRAP_OF] = exc_overflow, + [X86_TRAP_BR] = exc_bounds, + [X86_TRAP_UD] = exc_invalid_op, + [X86_TRAP_NM] = exc_device_not_available, + [X86_TRAP_DF] = fred_exc_double_fault, + [X86_TRAP_OLD_MF] = fred_bad_event, /* 387 only! */ + [X86_TRAP_TS] = fred_exc_invalid_tss, + [X86_TRAP_NP] = fred_exc_segment_not_present, + [X86_TRAP_SS] = fred_exc_stack_segment, + [X86_TRAP_GP] = fred_exc_general_protection, + [X86_TRAP_PF] = fred_exc_page_fault, + [X86_TRAP_SPURIOUS] = fred_bad_event, /* Interrupts are their own event type */ + [X86_TRAP_MF] = exc_coprocessor_error, + [X86_TRAP_AC] = fred_exc_alignment_check, + [X86_TRAP_MC] = fred_exc_machine_check, + [X86_TRAP_XF] = exc_simd_coprocessor_error, + [X86_TRAP_VE...NUM_EXCEPTION_VECTORS-1] = fred_bad_event + }; + u8 vector = array_index_nospec((u8)regs->vector, NUM_EXCEPTION_VECTORS); + + exception_handlers[vector](regs); +} + +static __always_inline void fred_emulate_trap(struct pt_regs *regs) +{ + regs->type = EVENT_TYPE_SWFAULT; + regs->orig_ax = 0; + fred_exception(regs); +} + +static __always_inline void fred_emulate_fault(struct pt_regs *regs) +{ + regs->ip -= regs->instr_len; + fred_emulate_trap(regs); +} + +/* + * Emulate SYSENTER if applicable. This is not the preferred system + * call in 32-bit mode under FRED, rather int $0x80 is preferred and + * exported in the vdso. SYSCALL proper has a hard-coded early out in + * fred_entry_from_user(). + */ +static DEFINE_FRED_HANDLER(fred_syscall_slow) +{ + if (IS_ENABLED(CONFIG_IA32_EMULATION) && + likely(regs->vector == FRED_SYSENTER)) { + /* Convert frame to a syscall frame */ + regs->orig_ax = regs->ax; + regs->ax = -ENOSYS; + do_fast_syscall_32(regs); + } else { + regs->vector = X86_TRAP_UD; + fred_emulate_fault(regs); + } +} + +/* + * Some software exceptions can also be triggered as int instructions, + * for historical reasons. Implement those here. The performance-critical + * int $0x80 (32-bit system call) has a hard-coded early out. + */ +static DEFINE_FRED_HANDLER(fred_sw_interrupt_user) +{ + if (IS_ENABLED(CONFIG_IA32_EMULATION) && + likely(regs->vector == IA32_SYSCALL_VECTOR)) { + /* Convert frame to a syscall frame */ + regs->orig_ax = regs->ax; + regs->ax = -ENOSYS; + return do_int80_syscall_32(regs); + } + + switch (regs->vector) { + case X86_TRAP_BP: + case X86_TRAP_OF: + fred_emulate_trap(regs); + break; + default: + regs->vector = X86_TRAP_GP; + fred_emulate_fault(regs); + break; + } +} + +static DEFINE_FRED_HANDLER(fred_hw_interrupt) +{ + irqentry_state_t state = irqentry_enter(regs); + + instrumentation_begin(); + external_interrupt(regs); + instrumentation_end(); + irqentry_exit(regs, state); +} + +__visible noinstr void fred_entry_from_user(struct pt_regs *regs) +{ + static const fred_handler user_handlers[FRED_EVENT_TYPE_COUNT] = + { + [EVENT_TYPE_HWINT] = fred_hw_interrupt, + [EVENT_TYPE_RESERVED] = fred_bad_event, + [EVENT_TYPE_NMI] = fred_exc_nmi, + [EVENT_TYPE_SWINT] = fred_sw_interrupt_user, + [EVENT_TYPE_HWFAULT] = fred_exception, + [EVENT_TYPE_SWFAULT] = fred_exception, + [EVENT_TYPE_PRIVSW] = fred_exception, + [EVENT_TYPE_OTHER] = fred_syscall_slow + }; + + /* + * FRED employs a two-level event dispatch mechanism, with + * the first-level on the type of an event and the second-level + * on its vector. Thus a dispatch typically induces 2 calls. + * We optimize it by using early outs for the most frequent + * events, and syscalls are the first. We may also need early + * outs for page faults. + */ + if (likely(regs->type == EVENT_TYPE_OTHER && + regs->vector == FRED_SYSCALL)) { + /* Convert frame to a syscall frame */ + regs->orig_ax = regs->ax; + regs->ax = -ENOSYS; + do_syscall_64(regs, regs->orig_ax); + } else { + /* Not a system call */ + u8 type = array_index_nospec((u8)regs->type, FRED_EVENT_TYPE_COUNT); + + user_handlers[type](regs); + } +} + +__visible noinstr void fred_entry_from_kernel(struct pt_regs *regs) +{ + static const fred_handler kernel_handlers[FRED_EVENT_TYPE_COUNT] = + { + [EVENT_TYPE_HWINT] = fred_hw_interrupt, + [EVENT_TYPE_RESERVED] = fred_bad_event, + [EVENT_TYPE_NMI] = fred_exc_nmi, + [EVENT_TYPE_SWINT] = fred_bad_event, + [EVENT_TYPE_HWFAULT] = fred_exception, + [EVENT_TYPE_SWFAULT] = fred_exception, + [EVENT_TYPE_PRIVSW] = fred_exception, + [EVENT_TYPE_OTHER] = fred_bad_event + }; + u8 type = array_index_nospec((u8)regs->type, FRED_EVENT_TYPE_COUNT); + + /* The pt_regs frame on entry here is an exception frame */ + kernel_handlers[type](regs); +} diff --git a/arch/x86/include/asm/idtentry.h b/arch/x86/include/asm/idtentry.h index 2876ddae02bc..bd43866f9c3e 100644 --- a/arch/x86/include/asm/idtentry.h +++ b/arch/x86/include/asm/idtentry.h @@ -82,6 +82,7 @@ static __always_inline void __##func(struct pt_regs *regs) #define DECLARE_IDTENTRY_ERRORCODE(vector, func) \ asmlinkage void asm_##func(void); \ asmlinkage void xen_asm_##func(void); \ + __visible void fred_##func(struct pt_regs *regs); \ __visible void func(struct pt_regs *regs, unsigned long error_code) /** @@ -106,6 +107,11 @@ __visible noinstr void func(struct pt_regs *regs, \ irqentry_exit(regs, state); \ } \ \ +__visible noinstr void fred_##func(struct pt_regs *regs) \ +{ \ + func (regs, regs->orig_ax); \ +} \ + \ static __always_inline void __##func(struct pt_regs *regs, \ unsigned long error_code) @@ -622,6 +628,8 @@ DECLARE_IDTENTRY_RAW(X86_TRAP_MC, exc_machine_check); #ifdef CONFIG_XEN_PV DECLARE_IDTENTRY_RAW(X86_TRAP_MC, xenpv_exc_machine_check); #endif +#else +#define fred_exc_machine_check fred_bad_event #endif /* NMI */ From patchwork Mon Apr 10 08:14:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 81404 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1758019vqo; Mon, 10 Apr 2023 01:44:28 -0700 (PDT) X-Google-Smtp-Source: AKy350Z/rz39Ft7Hu7usbxwcZmw9QsXAYDGPLdSl5UDdXlnLBc2lnjmFx8GTesmDvTT8FP4xq24G X-Received: by 2002:aa7:9832:0:b0:627:6328:79f1 with SMTP id q18-20020aa79832000000b00627632879f1mr9834096pfl.34.1681116268553; Mon, 10 Apr 2023 01:44:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681116268; cv=none; d=google.com; s=arc-20160816; b=XZaMWcSv0SDgq4SVt2Ekm4oZd1fVo4L5Iim0jK4Nbz0NflrosPs85eBR1tZCaX65vF 6v5JAF00diEDPTCgqSjitq3PO1NDXF9vsIE5EXDW2nlV1pFs/ZfSyH5LvvFeWdqsiqPE mXPh9zTL2xo1L7Tg1irrBcVxEFJoM/TSv1bFVMPi8Ea6q8gN1aKfb2+NWRSXvk+XzrnJ RAx62FK5PC7PrRPxhQZnQboj2adBX1zB5BaRoaSJTWaRY6TJ+g/XyDBIGIq6wLGu+7GI 2eg0YEG/J3oGzPzOxY9b0cqfdfXI3//u0wkqk7ZeKK/wSFp9wuDlcOr4uP8iawLt6MEP AMZw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=LjyUJodvslqOHVIlJtjG7C7aG8ztN5TsdljforHvp40=; b=01+oGny9cQrE2gMoHdLa1+yogER7dyzhp5kVlryyDngGv+HHHg4yJu+nimELpKmDqp UZfix+vbkJqTIK4X9uYWUnwxpavcYzxtpHazI+hfPx4t9YdJucCHaHWBMZc1Dh9UslBV 3AqRqSeHJTJFiB1UK/bWR+7qpgKBQe5TU5IoMBonMMAk7oIK4RJEGSCJPYvb/oMOZGjD sa153V1JO3ooeWDooeeWwpsjeioQUmH8LtJVg/d8zRlg/BdmSzY5ds/LO96/+9TlPO6w YGwovYosM8zqj0Lkm8DHMLOJQVHJP0h9r+KzTUf27CBExtNJCi1fIXGixr7MPj7taFxP Jx3w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=BZFEfRtT; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 37-20020a630f65000000b0050c0549c4dbsi10365148pgp.702.2023.04.10.01.44.16; Mon, 10 Apr 2023 01:44:28 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=BZFEfRtT; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229977AbjDJImT (ORCPT + 99 others); Mon, 10 Apr 2023 04:42:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49516 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229783AbjDJIlX (ORCPT ); Mon, 10 Apr 2023 04:41:23 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A8F0B4C07; Mon, 10 Apr 2023 01:41:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1681116069; x=1712652069; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=iZiTaITFR/jl1357sOnUmRRKKZNexrms3X+PqIXtZQQ=; b=BZFEfRtTVjJzbwTHi414b6z/FRk3Y24HJVQJ3KqgOl21+ZiUVNKJfFJR qppgg9OqRJgT2jwaqrRMqihTtvk3d/t6oDSu/X+nbWLEazsbrYtZDFP2e PSJr6iUPrkUG4BVfUxbDpjjqZ7Rwq78Tx6eB5QeKzSUonmo9uyQ4fjTzr rW765WJ3kU4B4022As2F0d8c+PsoEYeUnVo5etI01fId9tK7zqEudmC2p g8DBk22us8l+lCQDn1AMjIzv9FcR3neUjaX3KWiyeK/iSVykDmpHdaQO1 09kLHa+bY3XGgBGRxzieb4yR7kM3Ssiyh8L97Hd4Cfa97fKyQSLCbdimy Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="342078095" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="342078095" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2023 01:41:06 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="799436324" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="799436324" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga002.fm.intel.com with ESMTP; 10 Apr 2023 01:41:06 -0700 From: Xin Li To: linux-kernel@vger.kernel.org, x86@kernel.org, kvm@vger.kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, peterz@infradead.org, andrew.cooper3@citrix.com, seanjc@google.com, pbonzini@redhat.com, ravi.v.shankar@intel.com, jiangshanlai@gmail.com, shan.kang@intel.com Subject: [PATCH v8 21/33] x86/fred: FRED initialization code Date: Mon, 10 Apr 2023 01:14:26 -0700 Message-Id: <20230410081438.1750-22-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230410081438.1750-1-xin3.li@intel.com> References: <20230410081438.1750-1-xin3.li@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.5 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762778172029919633?= X-GMAIL-MSGID: =?utf-8?q?1762778172029919633?= From: "H. Peter Anvin (Intel)" The code to initialize FRED when it's available and _not_ disabled. cpu_init_fred_exceptions() is the core function to initialize FRED, which 1. Sets up FRED entrypoints for events happening in ring 0 and 3. 2. Sets up a default stack for event handling. 3. Sets up dedicated event stacks for DB/NMI/MC/DF, equivalent to the IDT IST stacks. 4. Forces 32-bit system calls to use "int $0x80" only. 5. Enables FRED and invalidtes IDT. When the FRED is used, cpu_init_exception_handling() initializes FRED through calling cpu_init_fred_exceptions(), otherwise it sets up TSS IST and loads IDT. As FRED uses the ring 3 FRED entrypoint for SYSCALL and SYSENTER, it skips setting up SYSCALL/SYSENTER related MSRs, e.g., MSR_LSTAR. Signed-off-by: H. Peter Anvin (Intel) Co-developed-by: Xin Li Tested-by: Shan Kang Signed-off-by: Xin Li --- Changes since v5: * Add a comment for FRED stack level settings (Lai Jiangshan). * Define #DB/NMI/#MC/#DF stack levels using macros. --- arch/x86/include/asm/fred.h | 27 +++++++++++++ arch/x86/include/asm/traps.h | 2 + arch/x86/kernel/Makefile | 1 + arch/x86/kernel/cpu/common.c | 74 +++++++++++++++++++++------------- arch/x86/kernel/fred.c | 78 ++++++++++++++++++++++++++++++++++++ arch/x86/kernel/irqinit.c | 7 +++- arch/x86/kernel/traps.c | 16 +++++++- 7 files changed, 175 insertions(+), 30 deletions(-) create mode 100644 arch/x86/kernel/fred.c diff --git a/arch/x86/include/asm/fred.h b/arch/x86/include/asm/fred.h index 61048aa4e01d..c5fbc4f18059 100644 --- a/arch/x86/include/asm/fred.h +++ b/arch/x86/include/asm/fred.h @@ -57,6 +57,19 @@ #define FRED_CSX_ALLOW_SINGLE_STEP _BITUL(25) #define FRED_CSX_INTERRUPT_SHADOW _BITUL(24) +/* #DB in the kernel would imply the use of a kernel debugger. */ +#define FRED_DB_STACK_LEVEL 1 +#define FRED_NMI_STACK_LEVEL 2 +#define FRED_MC_STACK_LEVEL 2 +/* + * #DF is the highest level because a #DF means "something went wrong + * *while delivering an exception*." The number of cases for which that + * can happen with FRED is drastically reduced and basically amounts to + * "the stack you pointed me to is broken." Thus, always change stacks + * on #DF, which means it should be at the highest level. + */ +#define FRED_DF_STACK_LEVEL 3 + #ifndef __ASSEMBLY__ #include @@ -104,8 +117,22 @@ DECLARE_FRED_HANDLER(fred_exc_debug); DECLARE_FRED_HANDLER(fred_exc_page_fault); DECLARE_FRED_HANDLER(fred_exc_machine_check); +/* + * The actual assembly entry and exit points + */ +extern __visible void fred_entrypoint_user(void); + +/* + * Initialization + */ +void cpu_init_fred_exceptions(void); +void fred_setup_apic(void); + #endif /* __ASSEMBLY__ */ +#else +#define cpu_init_fred_exceptions() BUG() +#define fred_setup_apic() BUG() #endif /* CONFIG_X86_FRED */ #endif /* ASM_X86_FRED_H */ diff --git a/arch/x86/include/asm/traps.h b/arch/x86/include/asm/traps.h index 46f5e4e2a346..612b3d6fec53 100644 --- a/arch/x86/include/asm/traps.h +++ b/arch/x86/include/asm/traps.h @@ -56,4 +56,6 @@ void __noreturn handle_stack_overflow(struct pt_regs *regs, void f (struct pt_regs *regs) typedef DECLARE_SYSTEM_INTERRUPT_HANDLER((*system_interrupt_handler)); +system_interrupt_handler get_system_interrupt_handler(unsigned int i); + #endif /* _ASM_X86_TRAPS_H */ diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile index dd61752f4c96..08d9c0a0bfbe 100644 --- a/arch/x86/kernel/Makefile +++ b/arch/x86/kernel/Makefile @@ -47,6 +47,7 @@ obj-y += platform-quirks.o obj-y += process_$(BITS).o signal.o signal_$(BITS).o obj-y += traps.o idt.o irq.o irq_$(BITS).o dumpstack_$(BITS).o obj-y += time.o ioport.o dumpstack.o nmi.o +obj-$(CONFIG_X86_FRED) += fred.o obj-$(CONFIG_MODIFY_LDT_SYSCALL) += ldt.o obj-y += setup.o x86_init.o i8259.o irqinit.o obj-$(CONFIG_JUMP_LABEL) += jump_label.o diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index e8cf6f4cfb52..eea41cb8722e 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -58,6 +58,7 @@ #include #include #include +#include #include #include #include @@ -2054,28 +2055,6 @@ static void wrmsrl_cstar(unsigned long val) /* May not be marked __init: used by software suspend */ void syscall_init(void) { - wrmsr(MSR_STAR, 0, (__USER32_CS << 16) | __KERNEL_CS); - wrmsrl(MSR_LSTAR, (unsigned long)entry_SYSCALL_64); - -#ifdef CONFIG_IA32_EMULATION - wrmsrl_cstar((unsigned long)entry_SYSCALL_compat); - /* - * This only works on Intel CPUs. - * On AMD CPUs these MSRs are 32-bit, CPU truncates MSR_IA32_SYSENTER_EIP. - * This does not cause SYSENTER to jump to the wrong location, because - * AMD doesn't allow SYSENTER in long mode (either 32- or 64-bit). - */ - wrmsrl_safe(MSR_IA32_SYSENTER_CS, (u64)__KERNEL_CS); - wrmsrl_safe(MSR_IA32_SYSENTER_ESP, - (unsigned long)(cpu_entry_stack(smp_processor_id()) + 1)); - wrmsrl_safe(MSR_IA32_SYSENTER_EIP, (u64)entry_SYSENTER_compat); -#else - wrmsrl_cstar((unsigned long)ignore_sysret); - wrmsrl_safe(MSR_IA32_SYSENTER_CS, (u64)GDT_ENTRY_INVALID_SEG); - wrmsrl_safe(MSR_IA32_SYSENTER_ESP, 0ULL); - wrmsrl_safe(MSR_IA32_SYSENTER_EIP, 0ULL); -#endif - /* * Flags to clear on syscall; clear as much as possible * to minimize user space-kernel interference. @@ -2086,6 +2065,41 @@ void syscall_init(void) X86_EFLAGS_IF|X86_EFLAGS_DF|X86_EFLAGS_OF| X86_EFLAGS_IOPL|X86_EFLAGS_NT|X86_EFLAGS_RF| X86_EFLAGS_AC|X86_EFLAGS_ID); + + /* + * The default user and kernel segments + */ + wrmsr(MSR_STAR, 0, (__USER32_CS << 16) | __KERNEL_CS); + + if (cpu_feature_enabled(X86_FEATURE_FRED)) { + /* Both sysexit and sysret cause #UD when FRED is enabled */ + wrmsrl_safe(MSR_IA32_SYSENTER_CS, (u64)GDT_ENTRY_INVALID_SEG); + wrmsrl_safe(MSR_IA32_SYSENTER_ESP, 0ULL); + wrmsrl_safe(MSR_IA32_SYSENTER_EIP, 0ULL); + } else { + wrmsrl(MSR_LSTAR, (unsigned long)entry_SYSCALL_64); + +#ifdef CONFIG_IA32_EMULATION + wrmsrl_cstar((unsigned long)entry_SYSCALL_compat); + /* + * This only works on Intel CPUs. + * On AMD CPUs these MSRs are 32-bit, CPU truncates + * MSR_IA32_SYSENTER_EIP. + * This does not cause SYSENTER to jump to the wrong + * location, because AMD doesn't allow SYSENTER in + * long mode (either 32- or 64-bit). + */ + wrmsrl_safe(MSR_IA32_SYSENTER_CS, (u64)__KERNEL_CS); + wrmsrl_safe(MSR_IA32_SYSENTER_ESP, + (unsigned long)(cpu_entry_stack(smp_processor_id()) + 1)); + wrmsrl_safe(MSR_IA32_SYSENTER_EIP, (u64)entry_SYSENTER_compat); +#else + wrmsrl_cstar((unsigned long)ignore_sysret); + wrmsrl_safe(MSR_IA32_SYSENTER_CS, (u64)GDT_ENTRY_INVALID_SEG); + wrmsrl_safe(MSR_IA32_SYSENTER_ESP, 0ULL); + wrmsrl_safe(MSR_IA32_SYSENTER_EIP, 0ULL); +#endif + } } #else /* CONFIG_X86_64 */ @@ -2218,18 +2232,24 @@ void cpu_init_exception_handling(void) /* paranoid_entry() gets the CPU number from the GDT */ setup_getcpu(cpu); - /* IST vectors need TSS to be set up. */ - tss_setup_ist(tss); + /* Set up the TSS */ tss_setup_io_bitmap(tss); set_tss_desc(cpu, &get_cpu_entry_area(cpu)->tss.x86_tss); - load_TR_desc(); /* GHCB needs to be setup to handle #VC. */ setup_ghcb(); - /* Finally load the IDT */ - load_current_idt(); + if (cpu_feature_enabled(X86_FEATURE_FRED)) { + /* Set up FRED exception handling */ + cpu_init_fred_exceptions(); + } else { + /* IST vectors need TSS to be set up. */ + tss_setup_ist(tss); + + /* Finally load the IDT */ + load_current_idt(); + } } /* diff --git a/arch/x86/kernel/fred.c b/arch/x86/kernel/fred.c new file mode 100644 index 000000000000..907abfd69f3f --- /dev/null +++ b/arch/x86/kernel/fred.c @@ -0,0 +1,78 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#include +#include +#include +#include /* For cr4_set_bits() */ +#include + +/* + * Initialize FRED on this CPU. This cannot be __init as it is called + * during CPU hotplug. + */ +void cpu_init_fred_exceptions(void) +{ + wrmsrl(MSR_IA32_FRED_CONFIG, + FRED_CONFIG_REDZONE | /* Reserve for CALL emulation */ + FRED_CONFIG_INT_STKLVL(0) | + FRED_CONFIG_ENTRYPOINT(fred_entrypoint_user)); + + /* + * The purpose of separate stacks for NMI, #DB and #MC *in the kernel* + * (remember that user space faults are always taken on stack level 0) + * is to avoid overflowing the kernel stack. + */ + wrmsrl(MSR_IA32_FRED_STKLVLS, + FRED_STKLVL(X86_TRAP_DB, FRED_DB_STACK_LEVEL) | + FRED_STKLVL(X86_TRAP_NMI, FRED_NMI_STACK_LEVEL) | + FRED_STKLVL(X86_TRAP_MC, FRED_MC_STACK_LEVEL) | + FRED_STKLVL(X86_TRAP_DF, FRED_DF_STACK_LEVEL)); + + /* The FRED equivalents to IST stacks... */ + wrmsrl(MSR_IA32_FRED_RSP1, __this_cpu_ist_top_va(DB)); + wrmsrl(MSR_IA32_FRED_RSP2, __this_cpu_ist_top_va(NMI)); + wrmsrl(MSR_IA32_FRED_RSP3, __this_cpu_ist_top_va(DF)); + + /* Not used with FRED */ + wrmsrl(MSR_LSTAR, 0ULL); + wrmsrl(MSR_CSTAR, 0ULL); + wrmsrl_safe(MSR_IA32_SYSENTER_CS, (u64)GDT_ENTRY_INVALID_SEG); + wrmsrl_safe(MSR_IA32_SYSENTER_ESP, 0ULL); + wrmsrl_safe(MSR_IA32_SYSENTER_EIP, 0ULL); + + /* Enable FRED */ + cr4_set_bits(X86_CR4_FRED); + idt_invalidate(); /* Any further IDT use is a bug */ + + /* Use int $0x80 for 32-bit system calls in FRED mode */ + setup_clear_cpu_cap(X86_FEATURE_SYSENTER32); + setup_clear_cpu_cap(X86_FEATURE_SYSCALL32); +} + +/* + * Initialize system vectors from a FRED perspective, so + * lapic_assign_system_vectors() can do its job. + */ +void __init fred_setup_apic(void) +{ + int i; + + for (i = 0; i < FIRST_EXTERNAL_VECTOR; i++) + set_bit(i, system_vectors); + + /* + * Don't set the non assigned system vectors in the + * system_vectors bitmap. Otherwise they show up in + * /proc/interrupts. + */ +#ifdef CONFIG_SMP + set_bit(IRQ_MOVE_CLEANUP_VECTOR, system_vectors); +#endif + + for (i = 0; i < NR_SYSTEM_VECTORS; i++) { + if (get_system_interrupt_handler(i) != NULL) { + set_bit(i + FIRST_SYSTEM_VECTOR, system_vectors); + } + } + + /* The rest are fair game... */ +} diff --git a/arch/x86/kernel/irqinit.c b/arch/x86/kernel/irqinit.c index c683666876f1..2a510f72dd11 100644 --- a/arch/x86/kernel/irqinit.c +++ b/arch/x86/kernel/irqinit.c @@ -28,6 +28,7 @@ #include #include #include +#include #include /* @@ -96,7 +97,11 @@ void __init native_init_IRQ(void) /* Execute any quirks before the call gates are initialised: */ x86_init.irqs.pre_vector_init(); - idt_setup_apic_and_irq_gates(); + if (cpu_feature_enabled(X86_FEATURE_FRED)) + fred_setup_apic(); + else + idt_setup_apic_and_irq_gates(); + lapic_assign_system_vectors(); if (!acpi_ioapic && !of_ioapic && nr_legacy_irqs()) { diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c index 549f7f962f8f..ecfaf4d647bb 100644 --- a/arch/x86/kernel/traps.c +++ b/arch/x86/kernel/traps.c @@ -1537,6 +1537,14 @@ static system_interrupt_handler system_interrupt_handlers[NR_SYSTEM_VECTORS] = { #undef SYSV +system_interrupt_handler get_system_interrupt_handler(unsigned int i) +{ + if (i >= NR_SYSTEM_VECTORS) + return NULL; + + return system_interrupt_handlers[i]; +} + /* * External interrupt dispatch function. * @@ -1572,7 +1580,8 @@ void __init install_system_interrupt_handler(unsigned int n, const void *asm_add #ifdef CONFIG_X86_64 system_interrupt_handlers[n - FIRST_SYSTEM_VECTOR] = (system_interrupt_handler)addr; #endif - alloc_intr_gate(n, asm_addr); + if (!cpu_feature_enabled(X86_FEATURE_FRED)) + alloc_intr_gate(n, asm_addr); } void __init trap_init(void) @@ -1585,7 +1594,10 @@ void __init trap_init(void) /* Initialize TSS before setting up traps so ISTs work */ cpu_init_exception_handling(); + /* Setup traps as cpu_init() might #GP */ - idt_setup_traps(); + if (!cpu_feature_enabled(X86_FEATURE_FRED)) + idt_setup_traps(); + cpu_init(); } From patchwork Mon Apr 10 08:14:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 81402 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1757948vqo; Mon, 10 Apr 2023 01:44:17 -0700 (PDT) X-Google-Smtp-Source: AKy350YSHiyjXtPLCeZRQnhxNhWjyGVNwll2tr/948IKOUfdl37vq2FN9DnZP1FChG8rFbpWpyNm X-Received: by 2002:a17:902:c950:b0:1a5:1bb4:adb1 with SMTP id i16-20020a170902c95000b001a51bb4adb1mr10525622pla.37.1681116257211; Mon, 10 Apr 2023 01:44:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681116257; cv=none; d=google.com; s=arc-20160816; b=NRNKtHScEKQ3IhKbfikFVc2i/rIQoSry5vNXkDuEiI9radljnGoydKbSqr7rSQhV0U HFRbjp8sWPgzAd37TvD8JKF8OZiqJEuXxuNCMZv/iPDLqpaqMaTsOth6fCif1zLNbdNR fdctfacxKAbc09uAly566ufjruVbaQsi5mFpb8W2aHmSJt7FK+bTvmaQ1CcvzyZcpGbg r9EkQ7jiwjljUVyhP//n2R93LAQzIO5pPygtDWfMIQEyoJbRiviv+f7kGKZMl4V91jsE 3n6L0bPlIaJ3KviUmlh3LGQmZ5rbIWdnEHz8Y5D1C3C7bTCtjKM+RpV9548qePEVT7ZI S3VA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=o/s4OgoBvlYE/i/ueeuD809hBfxz0iI2J7KU1Tu8hr0=; b=bnVPK3UVUNsNhKo/JS/RTZ51i40yBCOKXmWT015hVEq9MTivnTq6RpU0gGoi0LQRWS 6avPsoxSuR32DmF2wixNgGJ+FmlwVkYCahGcx/MRuRF1+409FPGtb7rFBgbChBAIATCG BYI4d3rhNx3QdKew5e0wAEG64sYbgOCBoqh9mksUg01VWK55AKziZhlqfQbahqLVxGo4 sSMT9xmOxQ8De+0qYj29s7QqnvZD75Cnz/QMDQzsmu3Iix4L0oKaW96B9zrG6hvp9oCN rOALM9R2twAupzoQBSPCugTPUV3uSwUSL2EnCPhk2pSHGVIpA7RaAA4P03n4k6aRQ/wm B+fg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=C9yUxN4X; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id q17-20020a170902789100b001a5145cf5d5si7733996pll.167.2023.04.10.01.44.05; Mon, 10 Apr 2023 01:44:17 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=C9yUxN4X; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229967AbjDJImP (ORCPT + 99 others); Mon, 10 Apr 2023 04:42:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49380 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229776AbjDJIlW (ORCPT ); Mon, 10 Apr 2023 04:41:22 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CFB554C09; Mon, 10 Apr 2023 01:41:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1681116069; x=1712652069; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=SfNRJW38HST/J2vYBy+d2Rz7YNwfS+VaJzsap6qMLmE=; b=C9yUxN4X244YfRWtbiSnMj3bgEsXKKaRL0nEllyVHsjwdgRRBB/743QA xEslhgA4LXswm7dbKrT2oWx664irKr7slEPTJ6McNPaIKoGaV3EFWgRk7 ir5eD4m45LrEIhgZg9rOCjnf+BMuP99GoStbrGYW3TvoHodBIzou6AOdK wnwjh/RsjIhjn019N95Eg2thByxnKJkLYg6+vLAhZXUI6JRu/vGZ1edfY LCFNh3y7Ns170D5Pe6YNXntYwI+y0l1dtjkdBIpAP8coXhejYWoqLqDBr c8U7PuXjrApx7f4eQONnVg/ygbSoid12aaqHD+diMUjhLMGOGQla0znWy A==; X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="342078107" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="342078107" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2023 01:41:06 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="799436328" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="799436328" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga002.fm.intel.com with ESMTP; 10 Apr 2023 01:41:06 -0700 From: Xin Li To: linux-kernel@vger.kernel.org, x86@kernel.org, kvm@vger.kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, peterz@infradead.org, andrew.cooper3@citrix.com, seanjc@google.com, pbonzini@redhat.com, ravi.v.shankar@intel.com, jiangshanlai@gmail.com, shan.kang@intel.com Subject: [PATCH v8 22/33] x86/fred: update MSR_IA32_FRED_RSP0 during task switch Date: Mon, 10 Apr 2023 01:14:27 -0700 Message-Id: <20230410081438.1750-23-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230410081438.1750-1-xin3.li@intel.com> References: <20230410081438.1750-1-xin3.li@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.5 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762778160311131017?= X-GMAIL-MSGID: =?utf-8?q?1762778160311131017?= From: "H. Peter Anvin (Intel)" MSR_IA32_FRED_RSP0 is used during ring 3 event delivery, and needs to be updated to point to the top of next task stack during task switch. Update MSR_IA32_FRED_RSP0 with WRMSR instruction for now, and will use WRMSRNS/WRMSRLIST for performance once it gets upstreamed. Signed-off-by: H. Peter Anvin (Intel) Tested-by: Shan Kang Signed-off-by: Xin Li --- arch/x86/include/asm/switch_to.h | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/switch_to.h b/arch/x86/include/asm/switch_to.h index 5c91305d09d2..00fd85abc1d2 100644 --- a/arch/x86/include/asm/switch_to.h +++ b/arch/x86/include/asm/switch_to.h @@ -68,9 +68,16 @@ static inline void update_task_stack(struct task_struct *task) #ifdef CONFIG_X86_32 this_cpu_write(cpu_tss_rw.x86_tss.sp1, task->thread.sp0); #else - /* Xen PV enters the kernel on the thread stack. */ - if (cpu_feature_enabled(X86_FEATURE_XENPV)) + if (cpu_feature_enabled(X86_FEATURE_FRED)) { + /* + * Will use WRMSRNS/WRMSRLIST for performance once it's upstreamed. + */ + wrmsrl(MSR_IA32_FRED_RSP0, + task_top_of_stack(task) + TOP_OF_KERNEL_STACK_PADDING); + } else if (cpu_feature_enabled(X86_FEATURE_XENPV)) { + /* Xen PV enters the kernel on the thread stack. */ load_sp0(task_top_of_stack(task)); + } #endif } From patchwork Mon Apr 10 08:14:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 81405 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1758024vqo; Mon, 10 Apr 2023 01:44:29 -0700 (PDT) X-Google-Smtp-Source: AKy350a9uW6OoOCqIugpuwXlZbP5q/IbFFzoP7sLWBmYpDBWCCIpPF/p2i+FnwBrithnv/MRPlnd X-Received: by 2002:a17:90b:390a:b0:244:b000:d8e3 with SMTP id ob10-20020a17090b390a00b00244b000d8e3mr13099493pjb.45.1681116269271; Mon, 10 Apr 2023 01:44:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681116269; cv=none; d=google.com; s=arc-20160816; b=0kZbHKiRrs1+F6qsoHddhzMAeW+UkMbTAscvVmHxawB2UvhL17EnGKrA8SX+HDU5nW enygJrMHiI9CcdPG17znxErWN6D2/4/Pg4l7pMpPfjh8VXxI8tiPhUPA0kmSCS2Yo1lA S1yfGgYPudhWFcx/a9O140FLY+1Oq68+cC0ZvQp0js08xavJ1A5YeaHIwa8iorY6LE6L 0v/d//esxiHmIdbhbElIy82ALCWBx2+1zTmBxacpi3b9MwmzSmw0n/flpmsD3XfIszkh vvw+MdopTdKd1Xz173r/zmB7nSMJhkowmaafG92aWINcLOj8wYt4H4jwdUSnRbRGrVnl Zw0w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=KIufXiS22ukm4bYjosPpn6jOKXWPfemJH11Ed6G9b7k=; b=uVT4jmKR1eJzceWDyuoc9qh0fdvW/fAYyyoQm3KhxMHQUppf2aQw+puPXd+Tck7Ilc jMc0rHu4IUqRiSSVj2gHF4kt+McQbpoLT0h1SS1ALEsW3dRobYL5p8QFtEJwUI5jTsjJ 0eTodmksKocRUuDozjJQSny7sCZBm/jeJ40jrvj7o1FUV1OtYO1PfgI2b8rzJirxR3wq u06XdZ+N+41JdUK+Tcba7G/Q/TtB9mO2Gee3dlo3fT7WCuRJsBmU/IIoRzy8xqfBegvZ Ft91+InM7OfE57EyQrf5BN299VLsofzHuF9+lJ1Rcx6xZx1grrF0RDLmSVsg96oal2Y2 Lz6A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=BeL26GLp; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id gi20-20020a17090b111400b0023d3ecf85c5si10263807pjb.59.2023.04.10.01.44.17; Mon, 10 Apr 2023 01:44:29 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=BeL26GLp; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229984AbjDJImW (ORCPT + 99 others); Mon, 10 Apr 2023 04:42:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49404 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229788AbjDJIlX (ORCPT ); Mon, 10 Apr 2023 04:41:23 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0C6064EC2; Mon, 10 Apr 2023 01:41:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1681116071; x=1712652071; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=O9acOeCuSr/+wKyZ7069qEc1yb3O4hIxP1jgg1zr2lE=; b=BeL26GLpLLY4x0pXd1dn8kkAg3eIMR46Q9sGZSPELWgST4Ipn0fLhmdw It6MrGABBVyDHH9XU2KGsX4QMdjjuNooEILNTeLt9O6QWhKKBvgFGK7R8 eIH3+HM2Ok1FwOCdlZyKU+cE5eU4/W8qlCO7hsV9ZYAAUZ/+7e1qYHGxH 0GWY11C5wtDCOyz2mvNlpnGAuK63lzUoQwwobty5h5TbmIGneASWrvHMz Zn5GOuH3HZ4Ue4YGYumMhFG3dkXuByjPgCyJsBWpwE0lxmwNd1VWOyIfX Ok9G3dKZlO6As/wc2H2lEiBlSagRshOLGHoCdCxrlwIPKexOLNbc4eFEL Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="342078118" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="342078118" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2023 01:41:06 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="799436334" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="799436334" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga002.fm.intel.com with ESMTP; 10 Apr 2023 01:41:06 -0700 From: Xin Li To: linux-kernel@vger.kernel.org, x86@kernel.org, kvm@vger.kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, peterz@infradead.org, andrew.cooper3@citrix.com, seanjc@google.com, pbonzini@redhat.com, ravi.v.shankar@intel.com, jiangshanlai@gmail.com, shan.kang@intel.com Subject: [PATCH v8 23/33] x86/fred: let ret_from_fork() jmp to fred_exit_user when FRED is enabled Date: Mon, 10 Apr 2023 01:14:28 -0700 Message-Id: <20230410081438.1750-24-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230410081438.1750-1-xin3.li@intel.com> References: <20230410081438.1750-1-xin3.li@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.5 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762778172957340165?= X-GMAIL-MSGID: =?utf-8?q?1762778172957340165?= From: "H. Peter Anvin (Intel)" Let ret_from_fork() jmp to fred_exit_user when FRED is enabled, otherwise the existing IDT code is chosen. Signed-off-by: H. Peter Anvin (Intel) Tested-by: Shan Kang Signed-off-by: Xin Li --- arch/x86/entry/entry_64.S | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S index eccc3431e515..5b595a9b2ffb 100644 --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -299,7 +299,12 @@ SYM_CODE_START_NOALIGN(ret_from_fork) UNWIND_HINT_REGS movq %rsp, %rdi call syscall_exit_to_user_mode /* returns with IRQs disabled */ +#ifdef CONFIG_X86_FRED + ALTERNATIVE "jmp swapgs_restore_regs_and_return_to_usermode", \ + "jmp fred_exit_user", X86_FEATURE_FRED +#else jmp swapgs_restore_regs_and_return_to_usermode +#endif 1: /* kernel thread */ From patchwork Mon Apr 10 08:14:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 81407 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1758047vqo; Mon, 10 Apr 2023 01:44:33 -0700 (PDT) X-Google-Smtp-Source: AKy350bAd86HmwWBpDvz/nBPvkR0y1Bf/kJnetWd4ZjhU/1JlW2LBoa0I1xdIEC6L9BqR7eAnUlM X-Received: by 2002:a17:903:284e:b0:1a5:3056:3c95 with SMTP id kq14-20020a170903284e00b001a530563c95mr4623143plb.68.1681116272932; Mon, 10 Apr 2023 01:44:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681116272; cv=none; d=google.com; s=arc-20160816; b=nVjhUYUOf38524rlVXMXligiSlGkaLn53bc3ZVsbDJex5zU7NGTLSt9/R8FQJ0B9w8 jw+4wq56VWOkJOCnrXpTPrs5xsdytQ+6rV3MfYom5bhqWwwU1B75z51s36RTH3mbjuvv WfQvqv9EzTHHc6qJL/r47Wib+Pc55bunEpujUxXMJsSJDQ91bBBfBte4R7ydYTc4sHYt vxEi/AdMis44o3XafnUHacDn/+VjoT6QWUV/jVUZ7aa6LflvUxfci3OUU8Q9/Nf4dNAp 0uQvK41MIfVwUXJ+1fNHkrs+YcFBiYCM3l//1nVWg+1l0EiuDPqtcVyDklUv9isKcJ5s BUfA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=HXWU+EothhAOstFqFMAbdES8hja1CkXrrCI4kIOxLxM=; b=WbwWVqtNA6Qu+cnuUosgtmEN3BgXw2c9QvIHzyR08rs5SPxbkA8eN+d1vqPzESaGxm pCSb2z6z34ySoAPj5c0DG8R0NkSQ5WTDwNclBXJ0uptQDY/r4WcUqr+cq70SUm7uyfUh bMrXeZN4Io72Xq61lRBB+mO0zaukindXhUuZPSC8skcV5eaJzyyx0Rsa+ya0mb9QO6l1 wsBHIm+xu6tDlQ4mGHyZdxEGkpWtbRU49vnwmNGTixQuka2HBU7GbgVfyHma5bcp3K1w PEJ8+BT+MZR2Uil6479CSZLqd9otKhVgeQFThilgtv//TAGoHcup+vMj8ZHR58ulJCgo E4Uw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=KTpeEpbh; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 137-20020a63018f000000b0050bf0f0870esi10518876pgb.627.2023.04.10.01.44.21; Mon, 10 Apr 2023 01:44:32 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=KTpeEpbh; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229998AbjDJIm3 (ORCPT + 99 others); Mon, 10 Apr 2023 04:42:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51466 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229851AbjDJIlq (ORCPT ); Mon, 10 Apr 2023 04:41:46 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3E42B4EEC; Mon, 10 Apr 2023 01:41:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1681116072; x=1712652072; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=T7eEyAoHvwiW3Acg9V6KJ8qZ7RgqMyuTGxhG3JO7gws=; b=KTpeEpbh+tQHnFEAvWnqCYVevK3RkhTHTfILlIAY31KrSaUldkebW/Tu f1K9RNg6xMkjarvo1HTkC8aCPWFzoWBWlrRiN8frTWCEiHgpAQosCK5GH l/C+zeEa1cDgdO2xyGKjtlpLD7L3pDP14LigllckiJkdubcWQnZCxsEE5 F9doOn50V2ECjGOaztMcMASdFwu/ZNDi9T6SBcbsFnPW9CsgqKCXpSo6M Bre9/ELlEhfnUsjK3kZTn8YU2ZsxCZ2WIv7ntHR8es8k82xdnqxmpqp8c UXYoNaaZC7slgWBKKLmbzBBAnw0G/rowueDslKVYzlK23Z+SxdX2J0QHv A==; X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="342078128" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="342078128" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2023 01:41:07 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="799436337" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="799436337" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga002.fm.intel.com with ESMTP; 10 Apr 2023 01:41:06 -0700 From: Xin Li To: linux-kernel@vger.kernel.org, x86@kernel.org, kvm@vger.kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, peterz@infradead.org, andrew.cooper3@citrix.com, seanjc@google.com, pbonzini@redhat.com, ravi.v.shankar@intel.com, jiangshanlai@gmail.com, shan.kang@intel.com Subject: [PATCH v8 24/33] x86/fred: disallow the swapgs instruction when FRED is enabled Date: Mon, 10 Apr 2023 01:14:29 -0700 Message-Id: <20230410081438.1750-25-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230410081438.1750-1-xin3.li@intel.com> References: <20230410081438.1750-1-xin3.li@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.5 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762778176933559201?= X-GMAIL-MSGID: =?utf-8?q?1762778176933559201?= From: "H. Peter Anvin (Intel)" The FRED architecture establishes the full supervisor/user through: 1) FRED event delivery swaps the value of the GS base address and that of the IA32_KERNEL_GS_BASE MSR. 2) ERETU swaps the value of the GS base address and that of the IA32_KERNEL_GS_BASE MSR. Thus, the swapgs instruction is disallowed when FRED is enabled, otherwise it causes #UD. Signed-off-by: H. Peter Anvin (Intel) Tested-by: Shan Kang Signed-off-by: Xin Li --- arch/x86/kernel/process_64.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/arch/x86/kernel/process_64.c b/arch/x86/kernel/process_64.c index a1aa74864c8b..2bea86073646 100644 --- a/arch/x86/kernel/process_64.c +++ b/arch/x86/kernel/process_64.c @@ -165,7 +165,8 @@ static noinstr unsigned long __rdgsbase_inactive(void) lockdep_assert_irqs_disabled(); - if (!cpu_feature_enabled(X86_FEATURE_XENPV)) { + if (!cpu_feature_enabled(X86_FEATURE_FRED) && + !cpu_feature_enabled(X86_FEATURE_XENPV)) { native_swapgs(); gsbase = rdgsbase(); native_swapgs(); @@ -190,7 +191,8 @@ static noinstr void __wrgsbase_inactive(unsigned long gsbase) { lockdep_assert_irqs_disabled(); - if (!cpu_feature_enabled(X86_FEATURE_XENPV)) { + if (!cpu_feature_enabled(X86_FEATURE_FRED) && + !cpu_feature_enabled(X86_FEATURE_XENPV)) { native_swapgs(); wrgsbase(gsbase); native_swapgs(); From patchwork Mon Apr 10 08:14:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 81420 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1761396vqo; Mon, 10 Apr 2023 01:55:00 -0700 (PDT) X-Google-Smtp-Source: AKy350ayKYVk5RxmygHjyRwGyWVY+oRF4uU6qludia1TjR9RxYGxgP/haQDUaNnqsf05Aom9dY2v X-Received: by 2002:a17:902:db04:b0:1a2:75cc:654c with SMTP id m4-20020a170902db0400b001a275cc654cmr11623244plx.16.1681116900376; Mon, 10 Apr 2023 01:55:00 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681116900; cv=none; d=google.com; s=arc-20160816; b=JAzpqyin73J7rSyNqgAUo2paX+vrIuyt1BPYom/Xd0X4ZL7hFE8e7MKVcHtm8mG86W xdSk33us8NJT/5DBg875UOjWXNlmZ+v7DK9OgARVIm9kFeKhsM5VkNTOvb4SB/Rady+M EqZsukAxFpzkYH5ARVPc2SFEw9em8mb1M6fJ50Szd3c41fl/uKOyak3cmDa3QmOk8Hga HK70kwUwzvXKzPGSZwM0iAMgJYXPYwdVmudqfpyB+T55S3fk3arBkXYWGd9w3DoMuvrO X3ZTMF3FA3BkpCDX5igenl9U6gepN3ZgLyECt7fhiOkyLVuZEXLcTWF6imbaThe28T1b tXlA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=vxjy11Tsv/mgkUj27ibhp2UA7uqHHL7WLXhaKKIvcCY=; b=tOHID37VpgX0oMZzJ7czPHTRqhbc5XRgQB9Z0EaU/wdRERl1Rf7PAoDTVOKFK1/cf1 TW9dPANonk8Ic9GJFoG0jmvS6kc+k6biR0gnMeWQ5d4uF8h9SCrkwo9ROVpI630zumk4 HlZpXmmwPWm9GhqaqxlFbK/2Jgk6n0ZXldHRqfAMwDsUREUEwIXjJZQCQqe/l0pRDTEm 4gkhOJ+sEFhp0L4zbPeDxcWSVz2/8inJYvzpuflZBFcfTyNr/PK1ppD6IZqejGWttDAS dCC0SBITkdmWvLjeRbdZX7zA3a+6tHUTlUFe9mSTZ67s0kLVdQRMlLato1z8S33KNbv8 CRxA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=TrwyM4CI; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id t1-20020a170902e84100b001a27af16626si11073634plg.569.2023.04.10.01.54.47; Mon, 10 Apr 2023 01:55:00 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=TrwyM4CI; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229797AbjDJImd (ORCPT + 99 others); Mon, 10 Apr 2023 04:42:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49402 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229862AbjDJIlq (ORCPT ); Mon, 10 Apr 2023 04:41:46 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 492284EED; Mon, 10 Apr 2023 01:41:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1681116072; x=1712652072; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=YTgEgmA3f1uzY8PacyZ6XmpZcGIgDW5/dB0vDrivilc=; b=TrwyM4CIrx3XOO2IddBL3Ylj0WriQKR5rwbSAhvdtu4vECZpTogOZZzo UpF+93hn+amsJf/n+GwOeqYQqsLXiHg6Ku4hvYgL/Hjaae/H+O2qUxVNW COFIhv8TQIAL/dDzcs+f+Vc0ho4TzsexxZTXEPgwqxnhD1Yxl0FYVhcjT bWaxNrY34rzMBs7zytKV+NUCWwgztpIxF+Zm6AYXL976tOQFv2bDBl5q8 mIVnvoYL2fNRGejoC3lDHs49ALi3sIOzvMuAwFjkdNdCp5QvkMnzT3bmq tgKgaKRFkS6XseWUcZ670qEk3cAeBQcb21lXfXzay1A64eFVXvoX7Ef/j Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="342078140" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="342078140" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2023 01:41:07 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="799436342" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="799436342" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga002.fm.intel.com with ESMTP; 10 Apr 2023 01:41:07 -0700 From: Xin Li To: linux-kernel@vger.kernel.org, x86@kernel.org, kvm@vger.kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, peterz@infradead.org, andrew.cooper3@citrix.com, seanjc@google.com, pbonzini@redhat.com, ravi.v.shankar@intel.com, jiangshanlai@gmail.com, shan.kang@intel.com Subject: [PATCH v8 25/33] x86/fred: no ESPFIX needed when FRED is enabled Date: Mon, 10 Apr 2023 01:14:30 -0700 Message-Id: <20230410081438.1750-26-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230410081438.1750-1-xin3.li@intel.com> References: <20230410081438.1750-1-xin3.li@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.5 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762778835108029045?= X-GMAIL-MSGID: =?utf-8?q?1762778835108029045?= From: "H. Peter Anvin (Intel)" Because FRED always restores the full value of %rsp, ESPFIX is no longer needed when it's enabled. Signed-off-by: H. Peter Anvin (Intel) Tested-by: Shan Kang Signed-off-by: Xin Li --- arch/x86/kernel/espfix_64.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/arch/x86/kernel/espfix_64.c b/arch/x86/kernel/espfix_64.c index 16f9814c9be0..48d133a54f45 100644 --- a/arch/x86/kernel/espfix_64.c +++ b/arch/x86/kernel/espfix_64.c @@ -106,6 +106,10 @@ void __init init_espfix_bsp(void) pgd_t *pgd; p4d_t *p4d; + /* FRED systems don't need ESPFIX */ + if (cpu_feature_enabled(X86_FEATURE_FRED)) + return; + /* Install the espfix pud into the kernel page directory */ pgd = &init_top_pgt[pgd_index(ESPFIX_BASE_ADDR)]; p4d = p4d_alloc(&init_mm, pgd, ESPFIX_BASE_ADDR); @@ -129,6 +133,10 @@ void init_espfix_ap(int cpu) void *stack_page; pteval_t ptemask; + /* FRED systems don't need ESPFIX */ + if (cpu_feature_enabled(X86_FEATURE_FRED)) + return; + /* We only have to do this once... */ if (likely(per_cpu(espfix_stack, cpu))) return; /* Already initialized */ From patchwork Mon Apr 10 08:14:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 81406 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1758039vqo; Mon, 10 Apr 2023 01:44:31 -0700 (PDT) X-Google-Smtp-Source: AKy350ZEny88lRfVRzw+dvJL0q9bkzxOKEyz+ZSO0GB3gFPjzyfe0BVXx+e0mL8VY+bwcA3dop/Q X-Received: by 2002:a17:902:d1ca:b0:1a4:fe85:ab1a with SMTP id g10-20020a170902d1ca00b001a4fe85ab1amr9112817plb.44.1681116271554; Mon, 10 Apr 2023 01:44:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681116271; cv=none; d=google.com; s=arc-20160816; b=C+VLpEkgFndqkBYSrnSeqdczOqdpw9QnjYdZGYKWkBBQCFVw6OuYpo3uDoiiXOIfYU QrfrNHRyBLlFYN84/fveUXa00tFOZr/S5m3I6Omo0QlLy/ldbPNB4/62MKS7DwrpUnLp YaPm1ZMYORCrKts0eVD4P5Zi0j/5ohcVj6PuQEWR979GjDkHeNFzu4IRP0wSSL5m5DiY iVCGFORHsd6O0lA2Is8bsDdScwTAOBUTMmQxs3JafPWl+Z1zznFm15yMtQqUGDGXwIEg lzfhjoYEYYqxS8L9E1qSPbTCgKLhoSft1DD9ti6KK42JuLtbqu1KTLdcKiXAVnWhND/O G45g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=dXk7rGWXn1+raSuko+HE7ZydNglgwfgo+bHOiS4p1Pg=; b=SoPxYKa7dEu2rHVIWQdUZ89GE0o1zHndcz5//C3Bw06Q/E3RJ4Pg9h4Wl4+NPnI+Cx 5UNThDb05P+adTmJuqbgW/XWMoX0/SIO+MVvK9LsyAOc1VuaWxtkwrkHHLcULvRl+KrX 7G71dQ3qNtlLwtj/3t855XNSFaTBSyHulcenyYF6p/rFuWqCFcZM2g1IFTkfRfiMYFBO 4NvI7928RXCNPrbVmBrssAnCyxoqs0DbdbOi71Qws2ISlcg6oBNkRxlGi9Ae8lmr+lPk hcXNbLFgkWgpwkC5AvoAX8GnBza0lRqjvM9PiOnpjgGnbMD1kYPh3aErivmrSBTpmCAv cbTA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=gxJjjazO; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id j3-20020a170903028300b001a6477cf78csi737808plr.295.2023.04.10.01.44.20; Mon, 10 Apr 2023 01:44:31 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=gxJjjazO; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229989AbjDJImZ (ORCPT + 99 others); Mon, 10 Apr 2023 04:42:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49604 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229849AbjDJIlq (ORCPT ); Mon, 10 Apr 2023 04:41:46 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2893C4EDF; Mon, 10 Apr 2023 01:41:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1681116072; x=1712652072; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=kdsDSL0FM4cOW58nPFQAA0OCrgImaKiH8KbKpW/8YIw=; b=gxJjjazOaw3J9dLuyYd1JSYpFwJY74yDhPZFwTWkVBIikARDrUYDNxMK d20iCrkCixrnWfQxxuOfk3asGW+hkJcOxtay/cwkyNTxBm2eTPYrFckJg X/YVOwHMikwkuGmWaFXfzLcQhgZqyw1Bv4D7ZZ7W+ZIDQqUn2qU1euXig jRNki5oMvOuCBi86XPXOW7NGhNI0qk6JdFZg+zC95brQdEmt2wiiehmqN GH7QDu7/uBbEe00G4HP6iyopJWfQpDpE+L18gKXbTmMa+PaHlKUeXq6m7 57V85GMDLXaWPt1GOIqtEMPFlMQGixFbLKxZvTdwCgBgYRrf2xTVSuA47 g==; X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="342078151" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="342078151" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2023 01:41:07 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="799436346" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="799436346" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga002.fm.intel.com with ESMTP; 10 Apr 2023 01:41:07 -0700 From: Xin Li To: linux-kernel@vger.kernel.org, x86@kernel.org, kvm@vger.kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, peterz@infradead.org, andrew.cooper3@citrix.com, seanjc@google.com, pbonzini@redhat.com, ravi.v.shankar@intel.com, jiangshanlai@gmail.com, shan.kang@intel.com Subject: [PATCH v8 26/33] x86/fred: allow single-step trap and NMI when starting a new thread Date: Mon, 10 Apr 2023 01:14:31 -0700 Message-Id: <20230410081438.1750-27-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230410081438.1750-1-xin3.li@intel.com> References: <20230410081438.1750-1-xin3.li@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.5 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762778175239610716?= X-GMAIL-MSGID: =?utf-8?q?1762778175239610716?= From: "H. Peter Anvin (Intel)" Allow single-step trap and NMI when starting a new thread, thus once the new thread returns to ring3, single-step trap and NMI are both enabled immediately. High-order 48 bits above the lowest 16 bit CS are discarded by the legacy IRET instruction, thus can be set unconditionally, even when FRED is not enabled. Signed-off-by: H. Peter Anvin (Intel) Tested-by: Shan Kang Signed-off-by: Xin Li --- arch/x86/include/asm/fred.h | 11 +++++++++++ arch/x86/kernel/process_64.c | 13 +++++++------ 2 files changed, 18 insertions(+), 6 deletions(-) diff --git a/arch/x86/include/asm/fred.h b/arch/x86/include/asm/fred.h index c5fbc4f18059..f7caf3b2f3f7 100644 --- a/arch/x86/include/asm/fred.h +++ b/arch/x86/include/asm/fred.h @@ -70,6 +70,14 @@ */ #define FRED_DF_STACK_LEVEL 3 +/* + * High-order 48 bits above the lowest 16 bit CS are discarded by the + * legacy IRET instruction, thus can be set unconditionally, even when + * FRED is not enabled. + */ +#define CSX_PROCESS_START \ + (FRED_CSX_ENABLE_NMI | FRED_CSX_ALLOW_SINGLE_STEP) + #ifndef __ASSEMBLY__ #include @@ -133,6 +141,9 @@ void fred_setup_apic(void); #else #define cpu_init_fred_exceptions() BUG() #define fred_setup_apic() BUG() + +#define CSX_PROCESS_START 0 + #endif /* CONFIG_X86_FRED */ #endif /* ASM_X86_FRED_H */ diff --git a/arch/x86/kernel/process_64.c b/arch/x86/kernel/process_64.c index 2bea86073646..c732d9dbff3a 100644 --- a/arch/x86/kernel/process_64.c +++ b/arch/x86/kernel/process_64.c @@ -55,6 +55,7 @@ #include #include #include +#include #ifdef CONFIG_IA32_EMULATION /* Not included via unistd.h */ #include @@ -506,7 +507,7 @@ void x86_gsbase_write_task(struct task_struct *task, unsigned long gsbase) static void start_thread_common(struct pt_regs *regs, unsigned long new_ip, unsigned long new_sp, - unsigned int _cs, unsigned int _ss, unsigned int _ds) + u16 _cs, u16 _ss, u16 _ds) { WARN_ON_ONCE(regs != current_pt_regs()); @@ -521,11 +522,11 @@ start_thread_common(struct pt_regs *regs, unsigned long new_ip, loadsegment(ds, _ds); load_gs_index(0); - regs->ip = new_ip; - regs->sp = new_sp; - regs->cs = _cs; - regs->ss = _ss; - regs->flags = X86_EFLAGS_IF; + regs->ip = new_ip; + regs->sp = new_sp; + regs->csx = _cs | CSX_PROCESS_START; + regs->ssx = _ss; + regs->flags = X86_EFLAGS_IF | X86_EFLAGS_FIXED; } void From patchwork Mon Apr 10 08:14:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 81399 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1757747vqo; Mon, 10 Apr 2023 01:43:46 -0700 (PDT) X-Google-Smtp-Source: AKy350YMhbdQjw1SVYEjVdlx/vR2DRhydPIpFNMtQ7RHG31BV4jVGgg/HmEXvKWT+d9999TltyG3 X-Received: by 2002:a17:90b:1a91:b0:246:61ae:2fbb with SMTP id ng17-20020a17090b1a9100b0024661ae2fbbmr11739451pjb.41.1681116226366; Mon, 10 Apr 2023 01:43:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681116226; cv=none; d=google.com; s=arc-20160816; b=MFLJB8chJoU4nc1j3ZNx74Jrtal+OM9LHHL9s+3nwIY+4vE8MRcPh5qpEIAZmNPN09 MopVynsMhvlTagw7/z0d9RjaYPhnc387HguhFg5JIWZDIGkKbChFn6DqMatCrROqSLIR v56yGfLKm7VDW72Uo/j5t13eBb+iohAc2FiJerGfndIixQB09453ZJCN+EwP3vsedqfy dj3+8vVPdNckbfyTe2CLw2uER5evEbDr93lOAcIAdpgk6QVrWrSStiZ7s8h5I9aauc2n Vow/l/fVkebffhkicg6YtnMEYjZX7RoozMhYXZyOV3gJI/F8lKY6/tV5DaW9SI7b6nqY CiQA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=1+QEEbi4oy68u0fc2q6w5Rl3wiaxAOSj2KqSvS5jFwA=; b=VmRJU7hTnDhHdToKnlDQnHVNOuNh/A5lKa2gp6qmh0cgUZX8yDWHAAG2DIN5p8VijS 2YZQQ/7tW4ZYC3atYdHtjhtPFXEGrwF9Q/KFzu7dmSIQPEg7rpX4b9PtDRiBQemi6GTk GHt3FHkjtowiznu3wKaPZsURDISBocRjFVwg9cGH7BSCxQ9NY+tGzTSLrTFdwQvgkiTR m4P3LYTqe0/cJjjXpTo3+Ik4glnIKXV4F7khKgJ2+lrbC/It7ywwu1/fOOMUL1D/a7OD vI04AoHf5HJkZ1NXhiUfpMUKe0o3Ur4vpSQ0Xs4gzhYTfjay1CXroVc0dfQDUk6Zz6WB nNwg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=EY2ITdTx; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id t2-20020a632d02000000b00513b0d416e4si9705532pgt.599.2023.04.10.01.43.34; Mon, 10 Apr 2023 01:43:46 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=EY2ITdTx; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229789AbjDJImg (ORCPT + 99 others); Mon, 10 Apr 2023 04:42:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51608 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229785AbjDJIlu (ORCPT ); Mon, 10 Apr 2023 04:41:50 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F06D655A4; Mon, 10 Apr 2023 01:41:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1681116074; x=1712652074; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=vN6NNJbHk9GIVKoaWU6Utrw1wfL7ZoQD1u7WUYVyJf0=; b=EY2ITdTxl/gcPeDAJb5SrwIBxmt5wLwcctZyKXxSzM82uKctwloC3jBm 2DpXuWNXZnSfK1B45KuxYQCdMF0NUfFJbs/jv24OfAOi0JgeHhBNp7Eeb shiujiBlvExdETnj+PnI9FMygA/K4HrI6sTtdXMozY1WNXI5FjmqX4KZg 3bFUu81d9JnNq44F055PB7x9R+oI/DchHHcdpyj8akC6oBMhKyZCcd4Kl 9U2ImmM+S6BXMOwjMD27IZjC/Zv6RwD/O+zUM+S28jhHMV9L6jD7xm6YQ s1GIsH9sY+lvm4RKFJcKJqG++PKQCbpIylmQks8I4GlggsjbjJgAeJVZE Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="342078161" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="342078161" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2023 01:41:07 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="799436350" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="799436350" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga002.fm.intel.com with ESMTP; 10 Apr 2023 01:41:07 -0700 From: Xin Li To: linux-kernel@vger.kernel.org, x86@kernel.org, kvm@vger.kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, peterz@infradead.org, andrew.cooper3@citrix.com, seanjc@google.com, pbonzini@redhat.com, ravi.v.shankar@intel.com, jiangshanlai@gmail.com, shan.kang@intel.com Subject: [PATCH v8 27/33] x86/fred: fixup fault on ERETU by jumping to fred_entrypoint_user Date: Mon, 10 Apr 2023 01:14:32 -0700 Message-Id: <20230410081438.1750-28-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230410081438.1750-1-xin3.li@intel.com> References: <20230410081438.1750-1-xin3.li@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.5 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762778128083286236?= X-GMAIL-MSGID: =?utf-8?q?1762778128083286236?= If the stack frame contains an invalid user context (e.g. due to invalid SS, a non-canonical RIP, etc.) the ERETU instruction will trap (#SS or #GP). From a Linux point of view, this really should be considered a user space failure, so use the standard fault fixup mechanism to intercept the fault, fix up the exception frame, and redirect execution to fred_entrypoint_user. The end result is that it appears just as if the hardware had taken the exception immediately after completing the transition to user space. Suggested-by: H. Peter Anvin (Intel) Tested-by: Shan Kang Signed-off-by: Xin Li --- Changes since v6: * Add a comment to explain why it is safe to write to the previous FRED stack frame. (Lai Jiangshan). Changes since v5: * Move the NMI bit from an invalid stack frame, which caused ERETU to fault, to the fault handler's stack frame, thus to unblock NMI ASAP if NMI is blocked (Lai Jiangshan). --- arch/x86/entry/entry_64_fred.S | 8 ++- arch/x86/include/asm/extable_fixup_types.h | 4 +- arch/x86/mm/extable.c | 76 ++++++++++++++++++++++ 3 files changed, 85 insertions(+), 3 deletions(-) diff --git a/arch/x86/entry/entry_64_fred.S b/arch/x86/entry/entry_64_fred.S index d975cacd060f..efe2bcd11273 100644 --- a/arch/x86/entry/entry_64_fred.S +++ b/arch/x86/entry/entry_64_fred.S @@ -5,8 +5,10 @@ * The actual FRED entry points. */ #include -#include +#include #include +#include +#include #include #include "calling.h" @@ -38,7 +40,9 @@ SYM_CODE_START_NOALIGN(fred_entrypoint_user) call fred_entry_from_user SYM_INNER_LABEL(fred_exit_user, SYM_L_GLOBAL) FRED_EXIT - ERETU +1: ERETU + + _ASM_EXTABLE_TYPE(1b, fred_entrypoint_user, EX_TYPE_ERETU) SYM_CODE_END(fred_entrypoint_user) .fill fred_entrypoint_kernel - ., 1, 0xcc diff --git a/arch/x86/include/asm/extable_fixup_types.h b/arch/x86/include/asm/extable_fixup_types.h index 991e31cfde94..1585c798a02f 100644 --- a/arch/x86/include/asm/extable_fixup_types.h +++ b/arch/x86/include/asm/extable_fixup_types.h @@ -64,6 +64,8 @@ #define EX_TYPE_UCOPY_LEN4 (EX_TYPE_UCOPY_LEN | EX_DATA_IMM(4)) #define EX_TYPE_UCOPY_LEN8 (EX_TYPE_UCOPY_LEN | EX_DATA_IMM(8)) -#define EX_TYPE_ZEROPAD 20 /* longword load with zeropad on fault */ +#define EX_TYPE_ZEROPAD 20 /* longword load with zeropad on fault */ + +#define EX_TYPE_ERETU 21 #endif diff --git a/arch/x86/mm/extable.c b/arch/x86/mm/extable.c index 60814e110a54..9d82193adf3c 100644 --- a/arch/x86/mm/extable.c +++ b/arch/x86/mm/extable.c @@ -6,6 +6,7 @@ #include #include +#include #include #include #include @@ -195,6 +196,77 @@ static bool ex_handler_ucopy_len(const struct exception_table_entry *fixup, return ex_handler_uaccess(fixup, regs, trapnr); } +#ifdef CONFIG_X86_FRED +static bool ex_handler_eretu(const struct exception_table_entry *fixup, + struct pt_regs *regs, unsigned long error_code) +{ + struct pt_regs *uregs = (struct pt_regs *)(regs->sp - offsetof(struct pt_regs, ip)); + unsigned short ss = uregs->ss; + unsigned short cs = uregs->cs; + + /* + * Move the NMI bit from the invalid stack frame, which caused ERETU + * to fault, to the fault handler's stack frame, thus to unblock NMI + * with the fault handler's ERETS instruction ASAP if NMI is blocked. + */ + regs->nmi = uregs->nmi; + + fred_info(uregs)->edata = fred_event_data(regs); + uregs->ssx = regs->ssx; + uregs->ss = ss; + uregs->csx = regs->csx; + uregs->nmi = 0; /* The NMI bit was moved away above */ + uregs->current_stack_level = 0; + uregs->cs = cs; + + /* + * Copy error code to uregs and adjust stack pointer accordingly. + * + * The RSP used by FRED to push a stack frame is not the value in %rsp, + * it is calculated from %rsp with the following 2 steps: + * 1) RSP = %rsp - (IA32_FRED_CONFIG & 0x1c0) // Reserve N*64 bytes + * 2) RSP = RSP & ~0x3f // Align to a 64-byte cache line + * when the event delivery doesn't trigger a stack level change. + * + * Here is an example with N*64 (N=1) bytes reserved: + * + * 64-byte cache line ==> ______________ + * |___Reserved___| + * |__Event_data__| + * |_____SS_______| + * |_____RSP______| + * |_____FLAGS____| + * |_____CS_______| + * |_____IP_______| <== ERETU stack frame + * 64-byte cache line ==> |__Error_code__| + * |______________| + * |______________| + * |______________| + * |______________| + * |______________| + * |______________| + * |______________| <== RSP after step 1) + * 64-byte cache line ==> |______________| <== RSP after step 2) + * |___Reserved___| + * |__Event_data__| + * |_____SS_______| + * |_____RSP______| + * |_____FLAGS____| + * |_____CS_______| + * |_____IP_______| <== ERETS stack frame + * 64-byte cache line ==> |__Error_code__| + * + * Thus a new FRED stack frame will always be pushed below a previous + * FRED stack frame ((N*64) bytes may be reserved between), and it is + * safe to write to a previous FRED stack frame as they never overlap. + */ + uregs->orig_ax = error_code; + regs->sp -= 8; + + return ex_handler_default(fixup, regs); +} +#endif + int ex_get_fixup_type(unsigned long ip) { const struct exception_table_entry *e = search_exception_tables(ip); @@ -272,6 +344,10 @@ int fixup_exception(struct pt_regs *regs, int trapnr, unsigned long error_code, return ex_handler_ucopy_len(e, regs, trapnr, reg, imm); case EX_TYPE_ZEROPAD: return ex_handler_zeropad(e, regs, fault_addr); +#ifdef CONFIG_X86_FRED + case EX_TYPE_ERETU: + return ex_handler_eretu(e, regs, error_code); +#endif } BUG(); } From patchwork Mon Apr 10 08:14:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 81417 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1760756vqo; Mon, 10 Apr 2023 01:52:54 -0700 (PDT) X-Google-Smtp-Source: AKy350aZiZ8WkWV3yInfxZWY6l5FbG75lCxyLFlc4OZ+E/UtW1oslU2oQ2jqSo1K1yw08z2WDzNk X-Received: by 2002:a17:903:280c:b0:1a6:37a6:f429 with SMTP id kp12-20020a170903280c00b001a637a6f429mr3446288plb.42.1681116774408; Mon, 10 Apr 2023 01:52:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681116774; cv=none; d=google.com; s=arc-20160816; b=nD4t2fWd7U8oFClKxFKgkqGQ7LhdXBvMigYE/FlMofy9Mfihht29yF7mbvtIEjjfAA b80hT+NTqicgmWOxMEcqy0lTZFP2JQuPFHeHnn549UDHJblLJ4UOshpa1H+CkJ/bK6VG vzRm7F5YKY4WQDlOsCZ/wkrgBMJKar4yHLbsbbtyZY94VJN7JTJ/R0Vg3DYxXl0TYPLi Se9hTTdkm3DgWUHhxUevwMj9xDa8gHSRR7C+RMDLaX/mFqTpV1LqgK0PHJu+0l4H7K8J aVzGS7PNr1hLkiJdqnIypWPAiwPjw1Ey0AWCpOc3CgR959xSHJuR0S3DUSV6d1B614fI MxNQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=6ryoaO52SQacc1V2FLptfa9uuyFKNPljQyyaT3VTYMw=; b=lFxXM20+Zei+baUYTFAXO0zfBOmMJGrvVGWHD4PCuOSkFYwPEPZwjrbcDTA6sieDR8 DgNDGA/a+A918EhU6whbEztS4QkNS++dX+lVl7vDTWwH69No4p2LX9O0z7jlUHVwyPy/ 9XWFM+q/512N22C6lA8wV+7mN5SN/44vistSiCMIoyqSNolzXFIdoK3qu+mC2g8B+A3f aPPXQX6VV2dU5iOXKYIVeOL1RT1OrtLsfIh6CKiKAGUZFQT0ki4rHLcpV8MfVTgi8Cai m2ltAXR4WnpIHI5X9SGakL37PuQbKrDV9hWLKAvW8OZ0nNeOlvMrY6gTFo/kLXOUQiLk YRKQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=SMVRPD79; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id j3-20020a170903028300b001a4faff9bc9si1963158plr.467.2023.04.10.01.52.42; Mon, 10 Apr 2023 01:52:54 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=SMVRPD79; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230046AbjDJIml (ORCPT + 99 others); Mon, 10 Apr 2023 04:42:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51640 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229792AbjDJIlu (ORCPT ); Mon, 10 Apr 2023 04:41:50 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F409155A8; Mon, 10 Apr 2023 01:41:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1681116074; x=1712652074; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=5IBl0XY5xGj0Ul5jZJTZVlHtGnpVju3gJ8HZ07Stwk0=; b=SMVRPD79X+TVAEMdTjFw4cVxt46rr5izl935DEZ7hCd/Pw/E+yj2gp16 hHzG9l2iNCUHFkTZgOMfbPzCDdcrqosiUNCZf2qofd+zT6o4vCJ4Hh/FE StJ3Mnh/HcaIlhUWeH58vYpgLwHL1CiA3CsS+wsJoLWLRQJVzxWfeUztz xFQNZhhPOSxKx3pOMNzFgwuisc/7SqJ80KrmK3dnP6I1P51rU8eym84Ya Rj5PqjsRFpHg4TNPJoiNx8q6qmd7acZWALurr9h6T1al9BMxWN4kidN4B cGmkh/kGTgJ6vP1pplZN5jp6GFem7hadnQkZ7qxxnPyfkMRlB2pKYX0AW Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="342078172" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="342078172" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2023 01:41:07 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="799436353" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="799436353" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga002.fm.intel.com with ESMTP; 10 Apr 2023 01:41:07 -0700 From: Xin Li To: linux-kernel@vger.kernel.org, x86@kernel.org, kvm@vger.kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, peterz@infradead.org, andrew.cooper3@citrix.com, seanjc@google.com, pbonzini@redhat.com, ravi.v.shankar@intel.com, jiangshanlai@gmail.com, shan.kang@intel.com Subject: [PATCH v8 28/33] x86/ia32: do not modify the DPL bits for a null selector Date: Mon, 10 Apr 2023 01:14:33 -0700 Message-Id: <20230410081438.1750-29-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230410081438.1750-1-xin3.li@intel.com> References: <20230410081438.1750-1-xin3.li@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.5 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762778702803754440?= X-GMAIL-MSGID: =?utf-8?q?1762778702803754440?= When a null selector is to be loaded into a segment register, reload_segments() sets its DPL bits to 3. Later when the IRET instruction loads it, it zeros the segment register. The two operations offset each other to actually effect a nop. Unlike IRET, ERETU does not make any of DS, ES, FS, or GS null if it is found to have DPL < 3. It is expected that a FRED-enabled operating system will return to ring 3 (in compatibility mode) only when those segments all have DPL = 3. Thus when FRED is enabled, we end up with having 3 in a segment register even when it is initially set to 0. Fix it by not modifying the DPL bits for a null selector. Tested-by: Shan Kang Signed-off-by: Xin Li --- arch/x86/kernel/signal_32.c | 21 +++++++++++++-------- 1 file changed, 13 insertions(+), 8 deletions(-) diff --git a/arch/x86/kernel/signal_32.c b/arch/x86/kernel/signal_32.c index 9027fc088f97..7796cf84fca2 100644 --- a/arch/x86/kernel/signal_32.c +++ b/arch/x86/kernel/signal_32.c @@ -36,22 +36,27 @@ #ifdef CONFIG_IA32_EMULATION #include +static inline u16 usrseg(u16 sel) +{ + return sel <= 3 ? sel : sel | 3; +} + static inline void reload_segments(struct sigcontext_32 *sc) { unsigned int cur; savesegment(gs, cur); - if ((sc->gs | 0x03) != cur) - load_gs_index(sc->gs | 0x03); + if (usrseg(sc->gs) != cur) + load_gs_index(usrseg(sc->gs)); savesegment(fs, cur); - if ((sc->fs | 0x03) != cur) - loadsegment(fs, sc->fs | 0x03); + if (usrseg(sc->fs) != cur) + loadsegment(fs, usrseg(sc->fs)); savesegment(ds, cur); - if ((sc->ds | 0x03) != cur) - loadsegment(ds, sc->ds | 0x03); + if (usrseg(sc->ds) != cur) + loadsegment(ds, usrseg(sc->ds)); savesegment(es, cur); - if ((sc->es | 0x03) != cur) - loadsegment(es, sc->es | 0x03); + if (usrseg(sc->es) != cur) + loadsegment(es, usrseg(sc->es)); } #define sigset32_t compat_sigset_t From patchwork Mon Apr 10 08:14:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 81400 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1757787vqo; Mon, 10 Apr 2023 01:43:52 -0700 (PDT) X-Google-Smtp-Source: AKy350bS8iAOIzm3c1Qn18Q6zqn3DFJ0BTPuXI6Hkl1bWqZoTRE3qCMcVm121b4gLqJMYDVj1sgO X-Received: by 2002:a62:1987:0:b0:62d:944e:b0b8 with SMTP id 129-20020a621987000000b0062d944eb0b8mr9571307pfz.32.1681116232437; Mon, 10 Apr 2023 01:43:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681116232; cv=none; d=google.com; s=arc-20160816; b=KaIV+D/eFWkgJLi8Z5ErbDXYodG+1t7ZiiDqh7nHIkm8YwDg2cciMin8NfUHcPpgQU Ep81qihPNmLUuG3ncDJDWhme31CCj6QFUSl48jxJ/mzN07TGV4WDfpOe48SdR/CygC9i wXZDgPQshIgwU5g8QLasld2bpHh/aC0qCkUrw6OxwXZP3gQSDyIwcEbtPCYgyKckN3xF HE7AZgYCcuc41T+Yr7lcCv9X+zSfwmCLwgXeuEDggAoeqQGb+y1awuT0xD/7d2s8Mez/ XFU5JrSoD+8WZqJwOX7LiW9AVpKrk8uNnGimZTZOh73n0kKMrwTnnSe3joohbLWfXDSW MTww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=LEBKkhzkCpkXO8dBR7DHU0kFn55u5RMjoLHOy5+hYK8=; b=rupcBtm/8znGbQeDZXf9J4ey+IZ4JPJiZ88FEJojfcBV8V8+VaooWRe02Idf8vNxQ9 vkQ07BiF+Xmm8TW7vgBOBC1LRMdN5GTNWIVM4RZo0Qc4DbwVMr0ZYZMIket+9jZwSqs2 mRYeFwGpzhwMQk3KCfYHX+JCN04bBrpKcZOa6k+1f8Blb7nbHjQzoWD6zmfQ8owf9QxV ZBeQjgQAPZCPSRIgyrokdnkS6wbuYeumo5fPNKd7IVVgtnf3XtJZOZNvWllO1b/q+S9a b/RvHAxkqlubXDKJ14D8Dv5SjM40cK7avFd9//qgH4b/kn8eed52NH9A6kmnRN//ge8A 1ThA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=ngUrfwah; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id z13-20020a634c0d000000b0050be692a718si10023052pga.530.2023.04.10.01.43.40; Mon, 10 Apr 2023 01:43:52 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=ngUrfwah; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230040AbjDJImk (ORCPT + 99 others); Mon, 10 Apr 2023 04:42:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50628 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229882AbjDJIlu (ORCPT ); Mon, 10 Apr 2023 04:41:50 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0050855A9; Mon, 10 Apr 2023 01:41:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1681116074; x=1712652074; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=tG7U+f6OUMyQKOAaHIbPd49RryPURKRmmF8XrX5dWro=; b=ngUrfwah0NtDDk2QKpwWQHxEx8ZQPg8iR5ROukhejpbMy7TXN6dSF21Z bUQQGrALO28DaNR1/ULXU+4VXxwvW+ikYTgIefIIJH/g4hb7GbtuhYVGI 6BVBdxOGMeylySwOrnMu60P7JRGAMSiQhpIo7+GL327bSSMsc62bKG3lZ 68H89v5P9qv1qxfYjiUBT3BDx0ngR6QponSk2XaenLWUIGkILkeuTq6Em mSulYeurvmRfsA7XiJuvjXwatGwvBokIFukXofgWGbJP6xMcTq8s5mh1b NFwqNTbCW6hDQVp0kYaTeV6UKGmzQHM4nKKL3mxfuOf9LBFowJo0493Hw A==; X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="342078182" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="342078182" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2023 01:41:08 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="799436357" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="799436357" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga002.fm.intel.com with ESMTP; 10 Apr 2023 01:41:07 -0700 From: Xin Li To: linux-kernel@vger.kernel.org, x86@kernel.org, kvm@vger.kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, peterz@infradead.org, andrew.cooper3@citrix.com, seanjc@google.com, pbonzini@redhat.com, ravi.v.shankar@intel.com, jiangshanlai@gmail.com, shan.kang@intel.com Subject: [PATCH v8 29/33] x86/fred: allow FRED systems to use interrupt vectors 0x10-0x1f Date: Mon, 10 Apr 2023 01:14:34 -0700 Message-Id: <20230410081438.1750-30-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230410081438.1750-1-xin3.li@intel.com> References: <20230410081438.1750-1-xin3.li@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.5 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762778134369684375?= X-GMAIL-MSGID: =?utf-8?q?1762778134369684375?= From: "H. Peter Anvin (Intel)" FRED inherits the Intel VT-x enhancement of classified events with a two-level event dispatch logic. The first-level dispatch is on the event type, and the second-level is on the event vector. This also means that vectors in different event types are orthogonal, thus, vectors 0x10-0x1f become available as hardware interrupts. Enable interrupt vectors 0x10-0x1f on FRED systems (interrupt 0x80 is already enabled.) Most of these changes are about removing the assumption that the lowest-priority vector is hard-wired to 0x20. Signed-off-by: H. Peter Anvin (Intel) Tested-by: Shan Kang Signed-off-by: Xin Li --- arch/x86/include/asm/idtentry.h | 4 ++-- arch/x86/include/asm/irq.h | 5 +++++ arch/x86/include/asm/irq_vectors.h | 15 +++++++++++---- arch/x86/kernel/apic/apic.c | 11 ++++++++--- arch/x86/kernel/apic/vector.c | 8 +++++++- arch/x86/kernel/fred.c | 4 ++-- arch/x86/kernel/idt.c | 6 +++--- arch/x86/kernel/irq.c | 2 +- arch/x86/kernel/traps.c | 2 ++ 9 files changed, 41 insertions(+), 16 deletions(-) diff --git a/arch/x86/include/asm/idtentry.h b/arch/x86/include/asm/idtentry.h index bd43866f9c3e..57c891148b59 100644 --- a/arch/x86/include/asm/idtentry.h +++ b/arch/x86/include/asm/idtentry.h @@ -546,8 +546,8 @@ __visible noinstr void func(struct pt_regs *regs, \ */ .align IDT_ALIGN SYM_CODE_START(irq_entries_start) - vector=FIRST_EXTERNAL_VECTOR - .rept NR_EXTERNAL_VECTORS + vector=FIRST_EXTERNAL_VECTOR_IDT + .rept FIRST_SYSTEM_VECTOR - FIRST_EXTERNAL_VECTOR_IDT UNWIND_HINT_IRET_REGS 0 : ENDBR diff --git a/arch/x86/include/asm/irq.h b/arch/x86/include/asm/irq.h index 768aa234cbb4..e4be6f8409ad 100644 --- a/arch/x86/include/asm/irq.h +++ b/arch/x86/include/asm/irq.h @@ -11,6 +11,11 @@ #include #include +/* + * The first available IRQ vector + */ +extern unsigned int __ro_after_init first_external_vector; + /* * The irq entry code is in the noinstr section and the start/end of * __irqentry_text is emitted via labels. Make the build fail if diff --git a/arch/x86/include/asm/irq_vectors.h b/arch/x86/include/asm/irq_vectors.h index 43dcb9284208..cb3670a7c18f 100644 --- a/arch/x86/include/asm/irq_vectors.h +++ b/arch/x86/include/asm/irq_vectors.h @@ -31,15 +31,23 @@ /* * IDT vectors usable for external interrupt sources start at 0x20. - * (0x80 is the syscall vector, 0x30-0x3f are for ISA) + * (0x80 is the syscall vector, 0x30-0x3f are for ISA). + * + * With FRED we can also use 0x10-0x1f even though those overlap + * exception vectors as FRED distinguishes exceptions and interrupts. + * Therefore, FIRST_EXTERNAL_VECTOR is no longer a constant. */ -#define FIRST_EXTERNAL_VECTOR 0x20 +#define FIRST_EXTERNAL_VECTOR_IDT 0x20 +#define FIRST_EXTERNAL_VECTOR_FRED 0x10 +#define FIRST_EXTERNAL_VECTOR first_external_vector /* * Reserve the lowest usable vector (and hence lowest priority) 0x20 for * triggering cleanup after irq migration. 0x21-0x2f will still be used * for device interrupts. */ +#define IRQ_MOVE_CLEANUP_VECTOR_IDT FIRST_EXTERNAL_VECTOR_IDT +#define IRQ_MOVE_CLEANUP_VECTOR_FRED FIRST_EXTERNAL_VECTOR_FRED #define IRQ_MOVE_CLEANUP_VECTOR FIRST_EXTERNAL_VECTOR #define IA32_SYSCALL_VECTOR 0x80 @@ -48,7 +56,7 @@ * Vectors 0x30-0x3f are used for ISA interrupts. * round up to the next 16-vector boundary */ -#define ISA_IRQ_VECTOR(irq) (((FIRST_EXTERNAL_VECTOR + 16) & ~15) + irq) +#define ISA_IRQ_VECTOR(irq) (((FIRST_EXTERNAL_VECTOR_IDT + 16) & ~15) + irq) /* * Special IRQ vectors used by the SMP architecture, 0xf0-0xff @@ -114,7 +122,6 @@ #define FIRST_SYSTEM_VECTOR NR_VECTORS #endif -#define NR_EXTERNAL_VECTORS (FIRST_SYSTEM_VECTOR - FIRST_EXTERNAL_VECTOR) #define NR_SYSTEM_VECTORS (NR_VECTORS - FIRST_SYSTEM_VECTOR) /* diff --git a/arch/x86/kernel/apic/apic.c b/arch/x86/kernel/apic/apic.c index 20d9a604da7c..eef67f64aa81 100644 --- a/arch/x86/kernel/apic/apic.c +++ b/arch/x86/kernel/apic/apic.c @@ -1621,12 +1621,17 @@ static void setup_local_APIC(void) /* * Set Task Priority to 'accept all except vectors 0-31'. An APIC * vector in the 16-31 range could be delivered if TPR == 0, but we - * would think it's an exception and terrible things will happen. We - * never change this later on. + * would think it's an exception and terrible things will happen, + * unless we are using FRED in which case interrupts and + * exceptions are distinguished by type code. + * + * We never change this later on. */ + BUG_ON(!first_external_vector); + value = apic_read(APIC_TASKPRI); value &= ~APIC_TPRI_MASK; - value |= 0x10; + value |= (first_external_vector - 0x10) & APIC_TPRI_MASK; apic_write(APIC_TASKPRI, value); /* Clear eventually stale ISR/IRR bits */ diff --git a/arch/x86/kernel/apic/vector.c b/arch/x86/kernel/apic/vector.c index c1efebd27e6c..f4325445fd78 100644 --- a/arch/x86/kernel/apic/vector.c +++ b/arch/x86/kernel/apic/vector.c @@ -46,6 +46,7 @@ static struct irq_matrix *vector_matrix; #ifdef CONFIG_SMP static DEFINE_PER_CPU(struct hlist_head, cleanup_list); #endif +unsigned int first_external_vector = FIRST_EXTERNAL_VECTOR_IDT; void lock_vector_lock(void) { @@ -796,7 +797,12 @@ int __init arch_early_irq_init(void) * Allocate the vector matrix allocator data structure and limit the * search area. */ - vector_matrix = irq_alloc_matrix(NR_VECTORS, FIRST_EXTERNAL_VECTOR, + if (cpu_feature_enabled(X86_FEATURE_FRED)) + first_external_vector = FIRST_EXTERNAL_VECTOR_FRED; + else + first_external_vector = FIRST_EXTERNAL_VECTOR_IDT; + + vector_matrix = irq_alloc_matrix(NR_VECTORS, first_external_vector, FIRST_SYSTEM_VECTOR); BUG_ON(!vector_matrix); diff --git a/arch/x86/kernel/fred.c b/arch/x86/kernel/fred.c index 907abfd69f3f..8d6dda8c8e71 100644 --- a/arch/x86/kernel/fred.c +++ b/arch/x86/kernel/fred.c @@ -56,7 +56,7 @@ void __init fred_setup_apic(void) { int i; - for (i = 0; i < FIRST_EXTERNAL_VECTOR; i++) + for (i = 0; i < FIRST_EXTERNAL_VECTOR_FRED; i++) set_bit(i, system_vectors); /* @@ -65,7 +65,7 @@ void __init fred_setup_apic(void) * /proc/interrupts. */ #ifdef CONFIG_SMP - set_bit(IRQ_MOVE_CLEANUP_VECTOR, system_vectors); + set_bit(IRQ_MOVE_CLEANUP_VECTOR_FRED, system_vectors); #endif for (i = 0; i < NR_SYSTEM_VECTORS; i++) { diff --git a/arch/x86/kernel/idt.c b/arch/x86/kernel/idt.c index a58c6bc1cd68..d3fd86f85de9 100644 --- a/arch/x86/kernel/idt.c +++ b/arch/x86/kernel/idt.c @@ -131,7 +131,7 @@ static const __initconst struct idt_data apic_idts[] = { INTG(RESCHEDULE_VECTOR, asm_sysvec_reschedule_ipi), INTG(CALL_FUNCTION_VECTOR, asm_sysvec_call_function), INTG(CALL_FUNCTION_SINGLE_VECTOR, asm_sysvec_call_function_single), - INTG(IRQ_MOVE_CLEANUP_VECTOR, asm_sysvec_irq_move_cleanup), + INTG(IRQ_MOVE_CLEANUP_VECTOR_IDT, asm_sysvec_irq_move_cleanup), INTG(REBOOT_VECTOR, asm_sysvec_reboot), #endif @@ -274,13 +274,13 @@ static void __init idt_map_in_cea(void) */ void __init idt_setup_apic_and_irq_gates(void) { - int i = FIRST_EXTERNAL_VECTOR; + int i = FIRST_EXTERNAL_VECTOR_IDT; void *entry; idt_setup_from_table(idt_table, apic_idts, ARRAY_SIZE(apic_idts), true); for_each_clear_bit_from(i, system_vectors, FIRST_SYSTEM_VECTOR) { - entry = irq_entries_start + IDT_ALIGN * (i - FIRST_EXTERNAL_VECTOR); + entry = irq_entries_start + IDT_ALIGN * (i - FIRST_EXTERNAL_VECTOR_IDT); set_intr_gate(i, entry); } diff --git a/arch/x86/kernel/irq.c b/arch/x86/kernel/irq.c index 7e125fff45ab..b7511e02959c 100644 --- a/arch/x86/kernel/irq.c +++ b/arch/x86/kernel/irq.c @@ -359,7 +359,7 @@ void fixup_irqs(void) * vector_lock because the cpu is already marked !online, so * nothing else will touch it. */ - for (vector = FIRST_EXTERNAL_VECTOR; vector < NR_VECTORS; vector++) { + for (vector = first_external_vector; vector < NR_VECTORS; vector++) { if (IS_ERR_OR_NULL(__this_cpu_read(vector_irq[vector]))) continue; diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c index ecfaf4d647bb..73471053ed02 100644 --- a/arch/x86/kernel/traps.c +++ b/arch/x86/kernel/traps.c @@ -1489,6 +1489,8 @@ DEFINE_IDTENTRY_IRQ(spurious_interrupt) pr_info("Spurious interrupt (vector 0x%x) on CPU#%d, should never happen.\n", vector, smp_processor_id()); } + +unsigned int first_external_vector = FIRST_EXTERNAL_VECTOR_IDT; #endif static void dispatch_table_spurious_interrupt(struct pt_regs *regs) From patchwork Mon Apr 10 08:14:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 81403 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1758009vqo; Mon, 10 Apr 2023 01:44:26 -0700 (PDT) X-Google-Smtp-Source: AKy350aJp3KGvWSQ2byZDYq5y4v6HGIExAW3exxDc8QDeToZNOLOiDPPeE3NlL00j4n4HyztYq2w X-Received: by 2002:a05:6a20:4d9b:b0:d5:b3d1:bff9 with SMTP id gj27-20020a056a204d9b00b000d5b3d1bff9mr10920423pzb.52.1681116266246; Mon, 10 Apr 2023 01:44:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681116266; cv=none; d=google.com; s=arc-20160816; b=e2eZL4yPcbZs/BmH4/aOHS0q3Ks3XajNVzsuPjL4XNN0oQYMv3kzOalg14JtcHbnmT N4UsfrPWYYvUT0oJbEfiHeyXzhoDQxfWYJaaWQeIqhNBdBkc1641EyDks4RL2JTb6O+b L4xgVJWZvaWcb3AkBdT1TeaheOhdo0OaRqvxrm4+BuAKtug/CdYd3A7j8SP3G+htlusk 4F3cVSndWQtvrDGE6YwCyzEKTqCc45Vnm4L2oPLi4BwlUFVtaij+q+Ewl3aAkLtp5GEu 1gtkq0JHlkKVdZiskHrIeFgeE/3W8QRy1kwE3RoMpNCuKHAtQ+685GpuIXNiP+HKIu2L eaGQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=okZmLcCQsAxlPEecFt13bFW4m637F0jY5IAmju11ZP0=; b=wzihjazcV23PibLdg/0ZdjyeTLUvso7b+etNLCn0yyCc3+yXm9OY5Mhx/KFg/nVvDk cli4xd4XRHuLBUnOCHev5t05iBYbvkmimCvISogG+FyezgOXY5qKLvqob7BdIoGVVzlX 75nW7+133Vhe/OSjLBzU2atRTKlG45zdXv6cLZFYsU3c0x5DMYWgb0LXUTzeoptmQOT2 UUjtOyRpQxxhDvGkJAZM/F28zDrBUYrG4kJCeGnwsqD7X/UKWhWcgSAAsaFRY6xm1PCR RiA4RBDwqTxiYH8rNSmoEUbwnHQ8PrwhGfOdKyEd5fnOhkB3goyFio1IlTWA1TzRhu1I oVnw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=EkVuPy5x; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id h18-20020a63e152000000b0050726756edesi10055889pgk.76.2023.04.10.01.44.14; Mon, 10 Apr 2023 01:44:26 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=EkVuPy5x; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230064AbjDJInI (ORCPT + 99 others); Mon, 10 Apr 2023 04:43:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52988 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229981AbjDJImT (ORCPT ); Mon, 10 Apr 2023 04:42:19 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 725335271; Mon, 10 Apr 2023 01:41:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1681116077; x=1712652077; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=I6kXJ8RyLpUh+WCrUnPXd7bpo63K8Gk5mwl1yKqLLc0=; b=EkVuPy5xxDccnrT9FlQ/aAiLK29A91ddPQiA9p6AfMrBWPmFogXNZhnD Snc1160HNaEipwNK6AhadpwHGLKCogApIPqJA8utGEQR21ukTKcU+8uzv +DK/WSfC+gsZb9FKpOJr7Bu6Se2elvfKcvc7XjGdkMFs/xqJMwr5g3TEg JuFdNypWWiY9zq38Kjp77QzIZtty/SPjOb0/ZduUz8D5l431i6oT5+1fL tqxaF+9zcTeuF2VBHH5jjwG7ssqEyfoEZLc7+oEMTFrI6Vm89vyzon21z TDra8vH/haZGR2xpkDp/I7ik/J70P8c/JsQQCU33HQuujKd5eaJ2c4d9n A==; X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="342078194" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="342078194" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2023 01:41:08 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="799436361" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="799436361" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga002.fm.intel.com with ESMTP; 10 Apr 2023 01:41:08 -0700 From: Xin Li To: linux-kernel@vger.kernel.org, x86@kernel.org, kvm@vger.kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, peterz@infradead.org, andrew.cooper3@citrix.com, seanjc@google.com, pbonzini@redhat.com, ravi.v.shankar@intel.com, jiangshanlai@gmail.com, shan.kang@intel.com Subject: [PATCH v8 30/33] x86/fred: allow dynamic stack frame size Date: Mon, 10 Apr 2023 01:14:35 -0700 Message-Id: <20230410081438.1750-31-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230410081438.1750-1-xin3.li@intel.com> References: <20230410081438.1750-1-xin3.li@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.5 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762778170200212187?= X-GMAIL-MSGID: =?utf-8?q?1762778170200212187?= A FRED stack frame could contain different amount of information for different event types, or perhaps even for different instances of the same event type. Thus we need to eliminate the need of any advance information of the stack frame size to allow dynamic stack frame size. Implement it through: 1) add a new field user_pt_regs to thread_info, and initialize it with a pointer to a virtual pt_regs structure at the top of a thread stack. 2) save a pointer to the user-space pt_regs structure created by fred_entrypoint_user() to user_pt_regs in fred_entry_from_user(). 3) initialize the init_thread_info's user_pt_regs with a pointer to a virtual pt_regs structure at the top of init stack. This approach also works for IDT, thus we unify the code. Suggested-by: H. Peter Anvin (Intel) Tested-by: Shan Kang Signed-off-by: Xin Li --- arch/x86/entry/entry_32.S | 2 +- arch/x86/entry/entry_fred.c | 2 ++ arch/x86/include/asm/entry-common.h | 3 +++ arch/x86/include/asm/processor.h | 12 +++------ arch/x86/include/asm/switch_to.h | 3 +-- arch/x86/include/asm/thread_info.h | 41 ++++------------------------- arch/x86/kernel/head_32.S | 3 +-- arch/x86/kernel/process.c | 5 ++++ kernel/fork.c | 6 +++++ 9 files changed, 27 insertions(+), 50 deletions(-) diff --git a/arch/x86/entry/entry_32.S b/arch/x86/entry/entry_32.S index 91397f58ac30..5adc4cf33d92 100644 --- a/arch/x86/entry/entry_32.S +++ b/arch/x86/entry/entry_32.S @@ -1244,7 +1244,7 @@ SYM_CODE_START(rewind_stack_and_make_dead) xorl %ebp, %ebp movl PER_CPU_VAR(pcpu_hot + X86_top_of_stack), %esi - leal -TOP_OF_KERNEL_STACK_PADDING-PTREGS_SIZE(%esi), %esp + leal -PTREGS_SIZE(%esi), %esp call make_task_dead 1: jmp 1b diff --git a/arch/x86/entry/entry_fred.c b/arch/x86/entry/entry_fred.c index 0bff1db913b8..18852d9d83fb 100644 --- a/arch/x86/entry/entry_fred.c +++ b/arch/x86/entry/entry_fred.c @@ -178,6 +178,8 @@ __visible noinstr void fred_entry_from_user(struct pt_regs *regs) [EVENT_TYPE_OTHER] = fred_syscall_slow }; + current->thread_info.user_pt_regs = regs; + /* * FRED employs a two-level event dispatch mechanism, with * the first-level on the type of an event and the second-level diff --git a/arch/x86/include/asm/entry-common.h b/arch/x86/include/asm/entry-common.h index 117903881fe4..5b7d0f47f188 100644 --- a/arch/x86/include/asm/entry-common.h +++ b/arch/x86/include/asm/entry-common.h @@ -12,6 +12,9 @@ /* Check that the stack and regs on entry from user mode are sane. */ static __always_inline void arch_enter_from_user_mode(struct pt_regs *regs) { + if (!cpu_feature_enabled(X86_FEATURE_FRED)) + current->thread_info.user_pt_regs = regs; + if (IS_ENABLED(CONFIG_DEBUG_ENTRY)) { /* * Make sure that the entry code gave us a sensible EFLAGS diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/processor.h index 8d73004e4cac..4a50d2a2c14b 100644 --- a/arch/x86/include/asm/processor.h +++ b/arch/x86/include/asm/processor.h @@ -626,17 +626,11 @@ static inline void spin_lock_prefetch(const void *x) prefetchw(x); } -#define TOP_OF_INIT_STACK ((unsigned long)&init_stack + sizeof(init_stack) - \ - TOP_OF_KERNEL_STACK_PADDING) +#define TOP_OF_INIT_STACK ((unsigned long)&init_stack + sizeof(init_stack)) -#define task_top_of_stack(task) ((unsigned long)(task_pt_regs(task) + 1)) +#define task_top_of_stack(task) ((unsigned long)task_stack_page(task) + THREAD_SIZE) -#define task_pt_regs(task) \ -({ \ - unsigned long __ptr = (unsigned long)task_stack_page(task); \ - __ptr += THREAD_SIZE - TOP_OF_KERNEL_STACK_PADDING; \ - ((struct pt_regs *)__ptr) - 1; \ -}) +#define task_pt_regs(task) ((task)->thread_info.user_pt_regs) #ifdef CONFIG_X86_32 #define INIT_THREAD { \ diff --git a/arch/x86/include/asm/switch_to.h b/arch/x86/include/asm/switch_to.h index 00fd85abc1d2..0a31da150808 100644 --- a/arch/x86/include/asm/switch_to.h +++ b/arch/x86/include/asm/switch_to.h @@ -72,8 +72,7 @@ static inline void update_task_stack(struct task_struct *task) /* * Will use WRMSRNS/WRMSRLIST for performance once it's upstreamed. */ - wrmsrl(MSR_IA32_FRED_RSP0, - task_top_of_stack(task) + TOP_OF_KERNEL_STACK_PADDING); + wrmsrl(MSR_IA32_FRED_RSP0, task_top_of_stack(task)); } else if (cpu_feature_enabled(X86_FEATURE_XENPV)) { /* Xen PV enters the kernel on the thread stack. */ load_sp0(task_top_of_stack(task)); diff --git a/arch/x86/include/asm/thread_info.h b/arch/x86/include/asm/thread_info.h index 998483078d5f..ced0a01e0a3e 100644 --- a/arch/x86/include/asm/thread_info.h +++ b/arch/x86/include/asm/thread_info.h @@ -13,42 +13,6 @@ #include #include -/* - * TOP_OF_KERNEL_STACK_PADDING is a number of unused bytes that we - * reserve at the top of the kernel stack. We do it because of a nasty - * 32-bit corner case. On x86_32, the hardware stack frame is - * variable-length. Except for vm86 mode, struct pt_regs assumes a - * maximum-length frame. If we enter from CPL 0, the top 8 bytes of - * pt_regs don't actually exist. Ordinarily this doesn't matter, but it - * does in at least one case: - * - * If we take an NMI early enough in SYSENTER, then we can end up with - * pt_regs that extends above sp0. On the way out, in the espfix code, - * we can read the saved SS value, but that value will be above sp0. - * Without this offset, that can result in a page fault. (We are - * careful that, in this case, the value we read doesn't matter.) - * - * In vm86 mode, the hardware frame is much longer still, so add 16 - * bytes to make room for the real-mode segments. - * - * x86-64 has a fixed-length stack frame, but it depends on whether - * or not FRED is enabled. Future versions of FRED might make this - * dynamic, but for now it is always 2 words longer. - */ -#ifdef CONFIG_X86_32 -# ifdef CONFIG_VM86 -# define TOP_OF_KERNEL_STACK_PADDING 16 -# else -# define TOP_OF_KERNEL_STACK_PADDING 8 -# endif -#else /* x86-64 */ -# ifdef CONFIG_X86_FRED -# define TOP_OF_KERNEL_STACK_PADDING (2*8) -# else -# define TOP_OF_KERNEL_STACK_PADDING 0 -# endif -#endif - /* * low level task data that entry.S needs immediate access to * - this struct should fit entirely inside of one cache line @@ -56,6 +20,7 @@ */ #ifndef __ASSEMBLY__ struct task_struct; +struct pt_regs; #include #include @@ -66,11 +31,14 @@ struct thread_info { #ifdef CONFIG_SMP u32 cpu; /* current CPU */ #endif + struct pt_regs *user_pt_regs; }; +#define INIT_TASK_PT_REGS ((struct pt_regs *)TOP_OF_INIT_STACK - 1) #define INIT_THREAD_INFO(tsk) \ { \ .flags = 0, \ + .user_pt_regs = INIT_TASK_PT_REGS, \ } #else /* !__ASSEMBLY__ */ @@ -240,6 +208,7 @@ static inline int arch_within_stack_frames(const void * const stack, extern void arch_task_cache_init(void); extern int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src); +extern void arch_init_user_pt_regs(struct task_struct *tsk); extern void arch_release_task_struct(struct task_struct *tsk); extern void arch_setup_new_exec(void); #define arch_setup_new_exec arch_setup_new_exec diff --git a/arch/x86/kernel/head_32.S b/arch/x86/kernel/head_32.S index 67c8ed99144b..0201ddcd7576 100644 --- a/arch/x86/kernel/head_32.S +++ b/arch/x86/kernel/head_32.S @@ -517,8 +517,7 @@ SYM_DATA_END(initial_page_table) * reliably detect the end of the stack. */ SYM_DATA(initial_stack, - .long init_thread_union + THREAD_SIZE - - SIZEOF_PTREGS - TOP_OF_KERNEL_STACK_PADDING) + .long init_thread_union + THREAD_SIZE - SIZEOF_PTREGS) __INITRODATA int_msg: diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c index b650cde3f64d..e1c6350290ae 100644 --- a/arch/x86/kernel/process.c +++ b/arch/x86/kernel/process.c @@ -98,6 +98,11 @@ int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src) return 0; } +void arch_init_user_pt_regs(struct task_struct *tsk) +{ + tsk->thread_info.user_pt_regs = (struct pt_regs *)task_top_of_stack(tsk)- 1; +} + #ifdef CONFIG_X86_64 void arch_release_task_struct(struct task_struct *tsk) { diff --git a/kernel/fork.c b/kernel/fork.c index 0c92f224c68c..8976d6b540c4 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -956,6 +956,10 @@ int __weak arch_dup_task_struct(struct task_struct *dst, return 0; } +void __weak arch_init_user_pt_regs(struct task_struct *tsk) +{ +} + void set_task_stack_end_magic(struct task_struct *tsk) { unsigned long *stackend; @@ -983,6 +987,8 @@ static struct task_struct *dup_task_struct(struct task_struct *orig, int node) if (err) goto free_tsk; + arch_init_user_pt_regs(tsk); + #ifdef CONFIG_THREAD_INFO_IN_TASK refcount_set(&tsk->stack_refcount, 1); #endif From patchwork Mon Apr 10 08:14:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 81410 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1758319vqo; Mon, 10 Apr 2023 01:45:23 -0700 (PDT) X-Google-Smtp-Source: AKy350apesV0+MkEuSSToO2wUzNxU7gQK3Psoji7wGKBZIBCAlu1yzzbmwoP3/7KeehR462WkycI X-Received: by 2002:a05:6a20:671c:b0:d6:ba0b:c82c with SMTP id q28-20020a056a20671c00b000d6ba0bc82cmr10927315pzh.38.1681116323734; Mon, 10 Apr 2023 01:45:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681116323; cv=none; d=google.com; s=arc-20160816; b=wD3mvogqOQKPlJ40pS67Ru62ZyforcOY6P9UbGXj5XpvstP87IIbRsWfeNgbNY3wR0 4QWqSRDo1892Z1nSVtxJKJ7vXis+OA72TUxzINdhCmgjMqsqlcnja5bqKH2DvgkhN/eA nXxeKInP9zS+4bq9VbMAtRl8OG08RhohopRfYrFT3Y2HSWUIhFTvMk2g+RhobtNdHZ4c EUzz0E2G6V/z9W/V3y8kC7CrSa76ehcc4nCKpEs598DdOom4sjfS6Hu1e/7jiLliH5Ms pQwNAjCKJIVmgs2BdBF9gRnuaazPTKqLdN9aJZ/dUAc2RTUVZPhlcOjmcovYuiJgve04 aXhw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=W7XedNztJ0mTK2b/JAIVrms4PBeui63Rn/5g9sSZqqs=; b=gsMiYfo7afxBCx8he/AaF9Yi1KErqsVxRqs/KWZnYepWjHSHwAPfLbsfzIlWBiOehO HC3KJr3YHjZbV+LcNcBK8anyXB8OnikYaSk2hsd3zHVVxxkeQGYYOHOSOxX8ewYlNmvc pECfhQ8QxlYyOtuuIebfOqF8Y2SrZ4Dzdp5sOAFXlUnyU+l8WFRo9oRgtHhfd1CuMHet Y4WnsNxz39YO6ZAW1/fNENYKWZkOw7mBTFeeOP/SPpIzbvafJAIzsB054fmK0HgpTfiX FVe+Bz53Oh34aeWzhPx5DwbbsRtDQ7shCzR8g9HnhNAc4ZBz65vlkWurRg2GzN/IIt/P kFLg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=F65AZPl4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id l70-20020a638849000000b00519d96ec174si2398794pgd.411.2023.04.10.01.45.12; Mon, 10 Apr 2023 01:45:23 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=F65AZPl4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230077AbjDJInR (ORCPT + 99 others); Mon, 10 Apr 2023 04:43:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50624 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230000AbjDJIm3 (ORCPT ); Mon, 10 Apr 2023 04:42:29 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C1BEE5FCD; Mon, 10 Apr 2023 01:41:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1681116077; x=1712652077; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=YkOYUeaNmXvaQHTm0febRcu8lP7lpU11uM/4zVeP9Zg=; b=F65AZPl4n5TPU/WE2Y7862BqXmYW+iWW1XpaexFWFjJTS3EX8yks7xrd vtXRNgdDGqDJAr4/sg+vOpjBizujb3amHljVjsp+CSvNz24anmZrhYUKO ASMlmIgq3+on+VPoUfHCuHmp6Blck/70b+q+dIWO7aYhW3I2418n/nbY7 y254PTa8u9wJcGp3Pbvzro8ESSBAVKkIRoV/mwxO+/aKeJ+7/BVQBTuGX woi0f+InuUhsaH2fdddn67CJyFkkJIpo/2oEXXFUHb6yBgQ5mQ6P+3zkn Xi0Y9Bo7G+3xAu32N256l+fEyCO1GFJ2/gYxaV497LermEd6ihtsZxzik A==; X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="342078205" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="342078205" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2023 01:41:08 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="799436365" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="799436365" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga002.fm.intel.com with ESMTP; 10 Apr 2023 01:41:08 -0700 From: Xin Li To: linux-kernel@vger.kernel.org, x86@kernel.org, kvm@vger.kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, peterz@infradead.org, andrew.cooper3@citrix.com, seanjc@google.com, pbonzini@redhat.com, ravi.v.shankar@intel.com, jiangshanlai@gmail.com, shan.kang@intel.com Subject: [PATCH v8 31/33] x86/fred: BUG() when ERETU with %rsp not equal to that when the ring 3 event was just delivered Date: Mon, 10 Apr 2023 01:14:36 -0700 Message-Id: <20230410081438.1750-32-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230410081438.1750-1-xin3.li@intel.com> References: <20230410081438.1750-1-xin3.li@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.5 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762778230108562661?= X-GMAIL-MSGID: =?utf-8?q?1762778230108562661?= A FRED stack frame generated by a ring 3 event should never be messed up, and the first thing we must make sure is that at the time an ERETU instruction is executed, %rsp must have the same address as that when the user level event was just delivered. However we don't want to bother the normal code path of ERETU because it's on the hotest code path, a good choice is to do this check when ERETU faults. Suggested-by: H. Peter Anvin (Intel) Tested-by: Shan Kang Signed-off-by: Xin Li --- arch/x86/mm/extable.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/arch/x86/mm/extable.c b/arch/x86/mm/extable.c index 9d82193adf3c..be297d4b137b 100644 --- a/arch/x86/mm/extable.c +++ b/arch/x86/mm/extable.c @@ -204,6 +204,14 @@ static bool ex_handler_eretu(const struct exception_table_entry *fixup, unsigned short ss = uregs->ss; unsigned short cs = uregs->cs; + /* + * A FRED stack frame generated by a ring 3 event should never be + * messed up, and the first thing we must make sure is that at the + * time an ERETU instruction is executed, %rsp must have the same + * address as that when the user level event was just delivered. + */ + BUG_ON(uregs != current->thread_info.user_pt_regs); + /* * Move the NMI bit from the invalid stack frame, which caused ERETU * to fault, to the fault handler's stack frame, thus to unblock NMI From patchwork Mon Apr 10 08:14:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 81408 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1758130vqo; Mon, 10 Apr 2023 01:44:49 -0700 (PDT) X-Google-Smtp-Source: AKy350auaM5orzJuAYiGqAuPwJQXkHnppCpvipN4rKxwQTZPyoimCi4OfQ9b7WwnHc4caIK59xPO X-Received: by 2002:aa7:955b:0:b0:639:f1da:9fa4 with SMTP id w27-20020aa7955b000000b00639f1da9fa4mr501983pfq.13.1681116288830; Mon, 10 Apr 2023 01:44:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681116288; cv=none; d=google.com; s=arc-20160816; b=aWlD5HcvRWyQvFIW+PWu8+Y/qCrjwfWOooVS5pyGHm7S76IYjfX7MsM8Z05z/YlJKc x/0uB3DZZ+L3Yi3m4U4BvmmfR+svnECBXtKLZ6qgVqMzK7qpp1E4Zic4koIW676fDLx2 tf5kx/8f5rEbbL2hBfDlEoCAk94Rf+2QWtBAfn2xlyVdJf1dZw9nawakKoEwawZAv0F9 zToQBhoLAxLfOl/jHRYVpBXotgFULsmaMiIjrxW1YONWZkktLFK1P4t4R2Ke+t2GM0cY M3QB4tZaWzRoEQkHLLKCFDO0LTzFT7o+sXVleYH76IU1KcNsHp6dKFpY7wWMdu9Nt0d1 DL+A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=fTC3wHoilVdfSrxaAuTGwj59tELEUHMxSkJME6+yttw=; b=wu6uAKi/ykVJjElxaGIYy4g7BohXMiob5J5FCi5Yx39+aYXc/h5tx+HYi4c4hXGYt9 t/bGbO/xBgdl+d3Ao0KeTg/1sZiOPtHPNXjsKum34vo9WZ59FgdUHfcgrG0K90Jy+ft0 vKuVCFicSKsHdyhsfyBIKaIC3G2sZdSaA2HfRzSDSYNku39T58PZpfnjkhcUw4X03OY4 Ba1lphSL196/93DJwmOk9BRM/SS9etHClhtWmePKybi7Bkr/Goh2D3N8dINTx8LG1J4k WV2+IDT08wD8+PJnG875Bqy2FRMdKH4eyVdx8JOTNsXkPpOP6dbwPCkuxIGbBaYPwQ8u 0Nrg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=RRNe0xKR; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id l70-20020a638849000000b00519d96ec174si2398794pgd.411.2023.04.10.01.44.36; Mon, 10 Apr 2023 01:44:48 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=RRNe0xKR; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229886AbjDJInN (ORCPT + 99 others); Mon, 10 Apr 2023 04:43:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49404 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230001AbjDJIm3 (ORCPT ); Mon, 10 Apr 2023 04:42:29 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EAD6059C4; Mon, 10 Apr 2023 01:41:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1681116079; x=1712652079; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=QC9Fb9BemsHiUNjBHn3W5WBw0HzwzGI557xdvOqcS00=; b=RRNe0xKRbIB0aHoNakq5TU7W37L8rvd3huu84a1+ibxnoU83RAx80wyI eJMd8uZybQhWu5Uju1z7RcpBZCvHRyNg8hHniW5ga+SojwaW/Yh1pkZUR mPx4zjUDD58UaAFJxgawYIgEkB4O1dAjAXfYiprA8uH+VePz7tWESMGRZ o4S2XOJH580po5Ml8cAXw4lvA4LK6yHa82jrwFssWkClfWBHpZdMK/QF2 wh+BPBSeAPCrwtP2zKRG2t3sMIqIllarUAJGMdhycAGEDxItbuEvYNwJK ABceX7/2DFmK7RK3zhFH5oV480UKkVDSfzWN6YqMsLl1FkfbIRnIGOhxn g==; X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="342078216" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="342078216" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2023 01:41:08 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="799436368" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="799436368" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga002.fm.intel.com with ESMTP; 10 Apr 2023 01:41:08 -0700 From: Xin Li To: linux-kernel@vger.kernel.org, x86@kernel.org, kvm@vger.kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, peterz@infradead.org, andrew.cooper3@citrix.com, seanjc@google.com, pbonzini@redhat.com, ravi.v.shankar@intel.com, jiangshanlai@gmail.com, shan.kang@intel.com Subject: [PATCH v8 32/33] x86/fred: disable FRED by default in its early stage Date: Mon, 10 Apr 2023 01:14:37 -0700 Message-Id: <20230410081438.1750-33-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230410081438.1750-1-xin3.li@intel.com> References: <20230410081438.1750-1-xin3.li@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.5 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762778193403959489?= X-GMAIL-MSGID: =?utf-8?q?1762778193403959489?= Disable FRED by default in its early stage. To enable FRED, a new kernel command line option "fred" needs to be added. Tested-by: Shan Kang Signed-off-by: Xin Li --- Changes since v7: * Add a log message when FRED is enabled. --- Documentation/admin-guide/kernel-parameters.txt | 4 ++++ arch/x86/kernel/cpu/common.c | 3 +++ arch/x86/kernel/fred.c | 3 +++ 3 files changed, 10 insertions(+) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index 6221a1d057dd..c55ea60e1a0c 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -1498,6 +1498,10 @@ Warning: use of this parameter will taint the kernel and may cause unknown problems. + fred + Forcefully enable flexible return and event delivery, + which is otherwise disabled by default. + ftrace=[tracer] [FTRACE] will set and start the specified tracer as early as possible in order to facilitate early diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index eea41cb8722e..4db5e619fc97 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -1467,6 +1467,9 @@ static void __init cpu_parse_early_param(void) char *argptr = arg, *opt; int arglen, taint = 0; + if (!cmdline_find_option_bool(boot_command_line, "fred")) + setup_clear_cpu_cap(X86_FEATURE_FRED); + #ifdef CONFIG_X86_32 if (cmdline_find_option_bool(boot_command_line, "no387")) #ifdef CONFIG_MATH_EMULATION diff --git a/arch/x86/kernel/fred.c b/arch/x86/kernel/fred.c index 8d6dda8c8e71..f4d48f4f06b4 100644 --- a/arch/x86/kernel/fred.c +++ b/arch/x86/kernel/fred.c @@ -11,6 +11,9 @@ */ void cpu_init_fred_exceptions(void) { + /* When FRED is enabled by default, this log message may not needed */ + pr_info("Initialize FRED on CPU%d\n", smp_processor_id()); + wrmsrl(MSR_IA32_FRED_CONFIG, FRED_CONFIG_REDZONE | /* Reserve for CALL emulation */ FRED_CONFIG_INT_STKLVL(0) | From patchwork Mon Apr 10 08:14:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 81409 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1758235vqo; Mon, 10 Apr 2023 01:45:08 -0700 (PDT) X-Google-Smtp-Source: AKy350b77PAXO8oSuuQAhdq4o0Da4A2nDmSmSMCanpiWSJjMJYCD6n4BkIizzr19pT00mcw1b09A X-Received: by 2002:a17:902:f685:b0:19c:be0c:738 with SMTP id l5-20020a170902f68500b0019cbe0c0738mr13458834plg.59.1681116308427; Mon, 10 Apr 2023 01:45:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681116308; cv=none; d=google.com; s=arc-20160816; b=bvd1x1/7goHDcOLbfpgqV88fgOnDlgg76GDFKYt/EcKa1ZgDJ/vD+S/B/kWS7JU6/+ Xkk4cENs6tlXm/MlbK94fyol/lVJEMhawd2eZ9P5cSt+PygJgxlmYHAqdePfgedyJ67g tbCcA9RZ4OedS6QjTF/2UeBEk3RSkqnoFRj/uIMy9M/jOhwyfVb8Hb9R5CumvT6ZyRB9 4YHuVkjL0bwsXQ41RMzJLCcCGisdf3q6FLV85WAa5hf/G7VfQPuTSSy7Rt/zcuUqGPKe BvoydAVe8F5937mV+1jaXCVEnVh+lLc8gOfpkGvczjBo97mbDps9FIZR/r5lWqXnb5+5 QANQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=vxYkGWVcnNqyyvAS+j21nSnVVrhsZuwl7Q2UDYpyumw=; b=aJCxda3qKvlmFEGEytflz4s4+ZAx7ckTUBMEI24Xj/hNu/K0czmCgpvoxXh/MXuxWp gqJ3mmAHoMt1BkACiHQbP83e/l237V3cPCZWWrSz5BadpgS7FFSvFVpHqjZpwx8568TA EbNgJs/o87ynOFrK+2J/F2O8rza9OQlS5c2kEYfdHz3SSs6VySqsqraUf4XxpKhkezRy k7TVCLhVuCsQBm/ziFyxthp6AMYwljkKWbwiUG+N71JUb/GJRH8lHSxCd/4L9Xh+j3Yz gdr8IWR4S+8flMQ5bqN9k8viQy3rD/4FbLmtLxaydph5ivkiY3iEMr6bRX+Y2yLaCK0s Hi0Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=JpuXekzS; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 137-20020a63018f000000b0050bf0f0870esi10518876pgb.627.2023.04.10.01.44.56; Mon, 10 Apr 2023 01:45:08 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=JpuXekzS; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229862AbjDJIne (ORCPT + 99 others); Mon, 10 Apr 2023 04:43:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53652 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230070AbjDJInI (ORCPT ); Mon, 10 Apr 2023 04:43:08 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0EAB64C18; Mon, 10 Apr 2023 01:41:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1681116085; x=1712652085; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=H2LkgNfdgZCxqjwQsMzz3P7aFl8VuyYKoqg0zmKZpZ0=; b=JpuXekzSTlq2nI3bstm/g7N199dVqiWAhyTbgMyQzPODcB1Zrpadrm2x lTQXpURSZS5Lau9xb6jJv1l6KaJARRNuPjnNTCDrxRSh0Bk+8koo7gwRt ePqUEkFuUYj1yLAl8MKxD8RtAhBdZYtCewIHDsZNKtBhk6+VOExVSCva6 izIsVuRMBY9El6rzuZBOoBlANq3J9D57Qw3D+ZhRLLzWHVUqXjA4D+E0L OOoJOhEtjbWyhx4dezwX87W39wzBMstu8zp+ew5LjUyGDoGAd2/8Edt9R VAbzyglmV1Ouplb8PaQJqsgDHx4LoWHzjLOHQ+hZ5XWHa0IuKeaZn849t w==; X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="342078228" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="342078228" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2023 01:41:09 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10675"; a="799436372" X-IronPort-AV: E=Sophos;i="5.98,333,1673942400"; d="scan'208";a="799436372" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga002.fm.intel.com with ESMTP; 10 Apr 2023 01:41:08 -0700 From: Xin Li To: linux-kernel@vger.kernel.org, x86@kernel.org, kvm@vger.kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, peterz@infradead.org, andrew.cooper3@citrix.com, seanjc@google.com, pbonzini@redhat.com, ravi.v.shankar@intel.com, jiangshanlai@gmail.com, shan.kang@intel.com Subject: [PATCH v8 33/33] KVM: x86/vmx: refactor VMX_DO_EVENT_IRQOFF to generate FRED stack frames Date: Mon, 10 Apr 2023 01:14:38 -0700 Message-Id: <20230410081438.1750-34-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230410081438.1750-1-xin3.li@intel.com> References: <20230410081438.1750-1-xin3.li@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.5 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762778214142619733?= X-GMAIL-MSGID: =?utf-8?q?1762778214142619733?= Comparing to an IDT stack frame, a FRED stack frame has extra 16 bytes of information pushed at the regular stack top and 8 bytes of error code _always_ pushed at the regular stack bottom, VMX_DO_EVENT_IRQOFF can be refactored to generate FRED stack frames with event type and vector properly set. Thus, IRQ/NMI can be handled with the existing approach when FRED is enabled. As a FRED stack frame always contains an error code pushed by hardware, call a trampoline code first to have the return instruction address pushed on the regular stack. Then the trampoline code pushes an error code (0 for both IRQ and NMI) and jumps to fred_entrypoint_kernel() for NMI handling or calls external_interrupt() for IRQ handling. The trampoline code for IRQ handling pushes general purpose registers to form a pt_regs structure and then use it to call external_interrupt(). As a result, IRQ handling no longer re-enter the noinstr code. Export fred_entrypoint_kernel() and external_interrupt() for above changes. Tested-by: Shan Kang Signed-off-by: Xin Li --- Changes since v7: * Always call external_interrupt() for IRQ handling on x86_64, thus avoid re-entering the noinstr code. * Create a FRED stack frame when FRED is compiled-in but not enabled, which uses some extra stack space but simplifies the code. Changes since v6: * Export fred_entrypoint_kernel(), required when kvm-intel built as a module. * Reserve a REDZONE for CALL emulation and align RSP to a 64-byte boundary before pushing a new FRED stack frame. --- arch/x86/entry/entry_64_fred.S | 1 + arch/x86/include/asm/asm-prototypes.h | 1 + arch/x86/include/asm/fred.h | 1 + arch/x86/include/asm/traps.h | 2 + arch/x86/kernel/traps.c | 5 ++ arch/x86/kvm/vmx/vmenter.S | 78 +++++++++++++++++++++++++-- arch/x86/kvm/vmx/vmx.c | 12 +++-- 7 files changed, 93 insertions(+), 7 deletions(-) diff --git a/arch/x86/entry/entry_64_fred.S b/arch/x86/entry/entry_64_fred.S index efe2bcd11273..de74ab97ff00 100644 --- a/arch/x86/entry/entry_64_fred.S +++ b/arch/x86/entry/entry_64_fred.S @@ -59,3 +59,4 @@ SYM_CODE_START_NOALIGN(fred_entrypoint_kernel) FRED_EXIT ERETS SYM_CODE_END(fred_entrypoint_kernel) +EXPORT_SYMBOL(fred_entrypoint_kernel) diff --git a/arch/x86/include/asm/asm-prototypes.h b/arch/x86/include/asm/asm-prototypes.h index b1a98fa38828..076bf8dee702 100644 --- a/arch/x86/include/asm/asm-prototypes.h +++ b/arch/x86/include/asm/asm-prototypes.h @@ -12,6 +12,7 @@ #include #include #include +#include #include #ifndef CONFIG_X86_CMPXCHG64 diff --git a/arch/x86/include/asm/fred.h b/arch/x86/include/asm/fred.h index f7caf3b2f3f7..d00b9cab6aa6 100644 --- a/arch/x86/include/asm/fred.h +++ b/arch/x86/include/asm/fred.h @@ -129,6 +129,7 @@ DECLARE_FRED_HANDLER(fred_exc_machine_check); * The actual assembly entry and exit points */ extern __visible void fred_entrypoint_user(void); +extern __visible void fred_entrypoint_kernel(void); /* * Initialization diff --git a/arch/x86/include/asm/traps.h b/arch/x86/include/asm/traps.h index 612b3d6fec53..017b95624325 100644 --- a/arch/x86/include/asm/traps.h +++ b/arch/x86/include/asm/traps.h @@ -58,4 +58,6 @@ typedef DECLARE_SYSTEM_INTERRUPT_HANDLER((*system_interrupt_handler)); system_interrupt_handler get_system_interrupt_handler(unsigned int i); +int external_interrupt(struct pt_regs *regs); + #endif /* _ASM_X86_TRAPS_H */ diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c index 73471053ed02..0f1fcd53cb52 100644 --- a/arch/x86/kernel/traps.c +++ b/arch/x86/kernel/traps.c @@ -1573,6 +1573,11 @@ int external_interrupt(struct pt_regs *regs) return 0; } +#if IS_ENABLED(CONFIG_KVM_INTEL) +/* For KVM VMX to handle IRQs in IRQ induced VM exits. */ +EXPORT_SYMBOL_GPL(external_interrupt); +#endif + #endif /* CONFIG_X86_64 */ void __init install_system_interrupt_handler(unsigned int n, const void *asm_addr, const void *addr) diff --git a/arch/x86/kvm/vmx/vmenter.S b/arch/x86/kvm/vmx/vmenter.S index 631fd7da2bc3..f2e1f8e61be9 100644 --- a/arch/x86/kvm/vmx/vmenter.S +++ b/arch/x86/kvm/vmx/vmenter.S @@ -2,12 +2,14 @@ #include #include #include +#include #include #include #include #include #include "kvm-asm-offsets.h" #include "run_flags.h" +#include "../../entry/calling.h" #define WORD_SIZE (BITS_PER_LONG / 8) @@ -31,7 +33,7 @@ #define VCPU_R15 __VCPU_REGS_R15 * WORD_SIZE #endif -.macro VMX_DO_EVENT_IRQOFF call_insn call_target +.macro VMX_DO_EVENT_IRQOFF call_insn call_target fred=0 nmi=0 /* * Unconditionally create a stack frame, getting the correct RSP on the * stack (for x86-64) would take two instructions anyways, and RBP can @@ -41,16 +43,55 @@ mov %_ASM_SP, %_ASM_BP #ifdef CONFIG_X86_64 +#ifdef CONFIG_X86_FRED + /* + * It's not necessary to change current stack level for handling IRQ/NMI + * because the state of the kernel stack is well defined in this place + * in the code, and it is known not to be deep in a bunch of nested I/O + * layer handlers that eat up the stack. + * + * Before starting to push a FRED stack frame, FRED reserves a redzone + * (for CALL emulation) and aligns RSP to a 64-byte boundary. + */ + sub $(FRED_CONFIG_REDZONE_AMOUNT << 6), %rsp + and $FRED_STACK_FRAME_RSP_MASK, %rsp + + /* + * A FRED stack frame has extra 16 bytes of information pushed at the + * regular stack top comparing to an IDT stack frame. + */ + push $0 /* Reserved by FRED, must be 0 */ + push $0 /* FRED event data, 0 for NMI and external interrupts */ +#else /* * Align RSP to a 16-byte boundary (to emulate CPU behavior) before * creating the synthetic interrupt stack frame for the IRQ/NMI. */ and $-16, %rsp - push $__KERNEL_DS +#endif + + .if \fred + .if \nmi + mov $(2 << 32 | 2 << 48), %rax /* NMI event type and vector */ + .else + mov %rdi, %rax + shl $32, %rax /* External interrupt vector */ + .endif + add $__KERNEL_DS, %rax + bts $57, %rax /* Set 64-bit mode */ + .else + mov $__KERNEL_DS, %rax + .endif + push %rax + push %rbp #endif pushf - push $__KERNEL_CS + mov $__KERNEL_CS, %_ASM_AX + .if \fred && \nmi + bts $28, %_ASM_AX /* Set the NMI bit */ + .endif + push %_ASM_AX \call_insn \call_target /* @@ -299,8 +340,19 @@ SYM_INNER_LABEL(vmx_vmexit, SYM_L_GLOBAL) SYM_FUNC_END(__vmx_vcpu_run) +SYM_CODE_START(vmx_do_nmi_trampoline) +#ifdef CONFIG_X86_FRED + ALTERNATIVE "jmp .Lno_errorcode_push", "", X86_FEATURE_FRED + push $0 /* FRED error code, 0 for NMI */ + jmp fred_entrypoint_kernel +#endif + +.Lno_errorcode_push: + jmp asm_exc_nmi_kvm_vmx +SYM_CODE_END(vmx_do_nmi_trampoline) + SYM_FUNC_START(vmx_do_nmi_irqoff) - VMX_DO_EVENT_IRQOFF call asm_exc_nmi_kvm_vmx + VMX_DO_EVENT_IRQOFF call vmx_do_nmi_trampoline fred=1 nmi=1 SYM_FUNC_END(vmx_do_nmi_irqoff) @@ -357,6 +409,24 @@ SYM_FUNC_START(vmread_error_trampoline) SYM_FUNC_END(vmread_error_trampoline) #endif +#ifdef CONFIG_X86_64 +SYM_CODE_START(vmx_do_interrupt_trampoline) + push $0 /* FRED error code, 0 for NMI and external interrupts */ + PUSH_REGS + + movq %rsp, %rdi /* %rdi -> pt_regs */ + call external_interrupt + + POP_REGS + addq $8,%rsp /* Drop FRED error code */ + RET +SYM_CODE_END(vmx_do_interrupt_trampoline) +#endif + SYM_FUNC_START(vmx_do_interrupt_irqoff) +#ifdef CONFIG_X86_64 + VMX_DO_EVENT_IRQOFF call vmx_do_interrupt_trampoline fred=1 +#else VMX_DO_EVENT_IRQOFF CALL_NOSPEC _ASM_ARG1 +#endif SYM_FUNC_END(vmx_do_interrupt_irqoff) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index d2d6e1b6c788..d85bcfd191b7 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -6874,7 +6874,7 @@ static void vmx_apicv_post_state_restore(struct kvm_vcpu *vcpu) memset(vmx->pi_desc.pir, 0, sizeof(vmx->pi_desc.pir)); } -void vmx_do_interrupt_irqoff(unsigned long entry); +void vmx_do_interrupt_irqoff(unsigned long entry_or_vector); void vmx_do_nmi_irqoff(void); static void handle_nm_fault_irqoff(struct kvm_vcpu *vcpu) @@ -6916,14 +6916,20 @@ static void handle_external_interrupt_irqoff(struct kvm_vcpu *vcpu) { u32 intr_info = vmx_get_intr_info(vcpu); unsigned int vector = intr_info & INTR_INFO_VECTOR_MASK; - gate_desc *desc = (gate_desc *)host_idt_base + vector; + unsigned long entry_or_vector; + +#ifdef CONFIG_X86_64 + entry_or_vector = vector; +#else + entry_or_vector = gate_offset((gate_desc *)host_idt_base + vector); +#endif if (KVM_BUG(!is_external_intr(intr_info), vcpu->kvm, "unexpected VM-Exit interrupt info: 0x%x", intr_info)) return; kvm_before_interrupt(vcpu, KVM_HANDLING_IRQ); - vmx_do_interrupt_irqoff(gate_offset(desc)); + vmx_do_interrupt_irqoff(entry_or_vector); kvm_after_interrupt(vcpu); vcpu->arch.at_instruction_boundary = true;