From patchwork Fri Dec 9 05:32:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinank Jain X-Patchwork-Id: 31624 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp593937wrr; Thu, 8 Dec 2022 21:36:46 -0800 (PST) X-Google-Smtp-Source: AA0mqf7346d7EldhdjGDN4IimNT4HBdUfascYLyInoWN7NrGbuQV63zxE+KT+ojztdwTJEVbs+rB X-Received: by 2002:a05:6a20:bc9d:b0:a7:187a:7b0a with SMTP id fx29-20020a056a20bc9d00b000a7187a7b0amr6230612pzb.42.1670564206504; Thu, 08 Dec 2022 21:36:46 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670564206; cv=none; d=google.com; s=arc-20160816; b=GqdF/x3mk/DLv/FQKO0RPpT2t79rkO7SMbE/fg7/rKghRuUf2WrU0ve+veZFgpojHJ 6sW7UCrUapOIn9kG9xzcNUEmP4YTwhkAoclanL8tdVIkeiSnp00e0k6ACAQg8AtIxh0i gdxjxF85Q6wCowZh46JqCV/ksvHZTu6YVOhXHQXFrW/Y/lnF0RQcZp+rMZbCEyVnWPPY 5n/9ODEGiU1EthjFUmMnNBubgt5Sd6VbYoa84D6sc7UVu+AnELKW3GPPGiRZEhbF0i0u GH/twQaZJc2Yi7uFkhzw9k+udbcRtGNKGJGf+1NTt6x9reRYS18Wls0L8iUaMx5xic7l W1jg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:dkim-filter; bh=wqo8SKjqAvAPfB0JD2Zy4udGVJ0BdIhQjoIMI1f64Wg=; b=vPoLifEjVHfjjQl5ITv0xRAOKa8xIUwlVE579l1brqSOt8Axss9c3sJvtUQb35Fhg7 l5w4//0wgDQD1wrZiXG8cnuUF/Ly2FbXO/tf1UmL9OaiAz/xJnonweEX6+RM3hrJDvce 9khdTw8WwxigIzd9qBYkVtDb0Kz7VykQhEW4+91g+7pvYZKsYfE5fEMSN59aObHnBEq6 nbwRMYYT5yijXHgeryMZ7LJobiPwnwhqUfipRJjyicbo/EGHjhNA/Snwn4R+y16HyfM0 WiQbUlJ9HE1mzHk04FaINb33c3DAgyBG0weeYmXAZtb5rPGuSp6DvlhY2ve6p2UH57wu FzXg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=ZMxcWYo8; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id g5-20020a63fa45000000b00478e2f7cf23si704443pgk.101.2022.12.08.21.36.32; Thu, 08 Dec 2022 21:36:46 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=ZMxcWYo8; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229733AbiLIFcv (ORCPT + 99 others); Fri, 9 Dec 2022 00:32:51 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52746 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229712AbiLIFcq (ORCPT ); Fri, 9 Dec 2022 00:32:46 -0500 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id A0132192B0; Thu, 8 Dec 2022 21:32:44 -0800 (PST) Received: from jinankjain-dranzer.zrrkmle5drku1h0apvxbr2u2ee.ix.internal.cloudapp.net (unknown [20.188.121.5]) by linux.microsoft.com (Postfix) with ESMTPSA id 7DD2F20B83C2; Thu, 8 Dec 2022 21:32:39 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 7DD2F20B83C2 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1670563964; bh=wqo8SKjqAvAPfB0JD2Zy4udGVJ0BdIhQjoIMI1f64Wg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ZMxcWYo81+U1rmYc936zb71RHD5Ew+fQHtBywr+FTfZOd26QVw7RINkLSrIxNl335 lZQ09q8r3eWp9WfBNRkTKMfKGZ86kML2kk9nWbhf9y7eKIvcl1wW69+aVZQ9/fkDHg KWKdGSBi+lZF+F/6OxPRmx2vTz/sjE1B3KLVFZ+U= From: Jinank Jain To: jinankjain@microsoft.com Cc: kys@microsoft.com, haiyangz@microsoft.com, wei.liu@kernel.org, decui@microsoft.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, arnd@arndb.de, peterz@infradead.org, jpoimboe@kernel.org, jinankjain@linux.microsoft.com, seanjc@google.com, kirill.shutemov@linux.intel.com, ak@linux.intel.com, sathyanarayanan.kuppuswamy@linux.intel.com, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, anrayabh@linux.microsoft.com, mikelley@microsoft.com Subject: [PATCH v8 1/5] x86/hyperv: Add support for detecting nested hypervisor Date: Fri, 9 Dec 2022 05:32:27 +0000 Message-Id: <8e3e7112806e81d2292a66a56fe547162754ecea.1670561320.git.jinankjain@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-19.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_MED, SPF_HELO_PASS,SPF_PASS,USER_IN_DEF_DKIM_WL,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751713533216230589?= X-GMAIL-MSGID: =?utf-8?q?1751713533216230589?= Detect if Linux is running as a nested hypervisor in the root partition for Microsoft Hypervisor, using flags provided by MSHV. Expose a new variable hv_nested that is used later for decisions specific to the nested use case. Signed-off-by: Jinank Jain Reviewed-by: Nuno Das Neves --- arch/x86/include/asm/hyperv-tlfs.h | 3 +++ arch/x86/kernel/cpu/mshyperv.c | 7 +++++++ drivers/hv/hv_common.c | 9 ++++++--- include/asm-generic/mshyperv.h | 1 + 4 files changed, 17 insertions(+), 3 deletions(-) diff --git a/arch/x86/include/asm/hyperv-tlfs.h b/arch/x86/include/asm/hyperv-tlfs.h index 6d9368ea3701..58c03d18c235 100644 --- a/arch/x86/include/asm/hyperv-tlfs.h +++ b/arch/x86/include/asm/hyperv-tlfs.h @@ -114,6 +114,9 @@ /* Recommend using the newer ExProcessorMasks interface */ #define HV_X64_EX_PROCESSOR_MASKS_RECOMMENDED BIT(11) +/* Indicates that the hypervisor is nested within a Hyper-V partition. */ +#define HV_X64_HYPERV_NESTED BIT(12) + /* Recommend using enlightened VMCS */ #define HV_X64_ENLIGHTENED_VMCS_RECOMMENDED BIT(14) diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c index 46668e255421..f9b78d4829e3 100644 --- a/arch/x86/kernel/cpu/mshyperv.c +++ b/arch/x86/kernel/cpu/mshyperv.c @@ -37,6 +37,8 @@ /* Is Linux running as the root partition? */ bool hv_root_partition; +/* Is Linux running on nested Microsoft Hypervisor */ +bool hv_nested; struct ms_hyperv_info ms_hyperv; #if IS_ENABLED(CONFIG_HYPERV) @@ -301,6 +303,11 @@ static void __init ms_hyperv_init_platform(void) pr_info("Hyper-V: running as root partition\n"); } + if (ms_hyperv.hints & HV_X64_HYPERV_NESTED) { + hv_nested = true; + pr_info("Hyper-V: running on a nested hypervisor\n"); + } + /* * Extract host information. */ diff --git a/drivers/hv/hv_common.c b/drivers/hv/hv_common.c index ae68298c0dca..52a6f89ccdbd 100644 --- a/drivers/hv/hv_common.c +++ b/drivers/hv/hv_common.c @@ -25,17 +25,20 @@ #include /* - * hv_root_partition and ms_hyperv are defined here with other Hyper-V - * specific globals so they are shared across all architectures and are + * hv_root_partition, ms_hyperv and hv_nested are defined here with other + * Hyper-V specific globals so they are shared across all architectures and are * built only when CONFIG_HYPERV is defined. But on x86, * ms_hyperv_init_platform() is built even when CONFIG_HYPERV is not - * defined, and it uses these two variables. So mark them as __weak + * defined, and it uses these three variables. So mark them as __weak * here, allowing for an overriding definition in the module containing * ms_hyperv_init_platform(). */ bool __weak hv_root_partition; EXPORT_SYMBOL_GPL(hv_root_partition); +bool __weak hv_nested; +EXPORT_SYMBOL_GPL(hv_nested); + struct ms_hyperv_info __weak ms_hyperv; EXPORT_SYMBOL_GPL(ms_hyperv); diff --git a/include/asm-generic/mshyperv.h b/include/asm-generic/mshyperv.h index bfb9eb9d7215..f131027830c3 100644 --- a/include/asm-generic/mshyperv.h +++ b/include/asm-generic/mshyperv.h @@ -48,6 +48,7 @@ struct ms_hyperv_info { u64 shared_gpa_boundary; }; extern struct ms_hyperv_info ms_hyperv; +extern bool hv_nested; extern void * __percpu *hyperv_pcpu_input_arg; extern void * __percpu *hyperv_pcpu_output_arg; From patchwork Fri Dec 9 05:32:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinank Jain X-Patchwork-Id: 31625 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp594018wrr; Thu, 8 Dec 2022 21:37:04 -0800 (PST) X-Google-Smtp-Source: AA0mqf5ELc0IRu7m4ggUe1mOiPd+G+nVDXiZg3oSqj8yDFcyuC2aWUAYcbocQ3pIqbJGszH1J9OP X-Received: by 2002:a05:6a21:e207:b0:ac:705b:d931 with SMTP id by7-20020a056a21e20700b000ac705bd931mr6174087pzc.7.1670564223741; Thu, 08 Dec 2022 21:37:03 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670564223; cv=none; d=google.com; s=arc-20160816; b=yd4ezzl/DMW4ew3vOX5wDQdPyOrkSmdiAvmwLUO4pWII1WVLOFWkG5YgQHJ3sVj1rP wnL3CX93/GOqP1GQi8xwaItPwYjlj9dqXT2jw7xyiBzgJk/SnLcb3FaCn7bZAz4XNQgn bO+sxpGjoqr3XlGioA51/U3Kw0mzgZQ1FY7rnXwxS7fKmTyHUvBejvZJ57cVggAYrsgs p7VPHRm2p52CSL+DSY+fil8JqbBh/QQEsraNOvH7QCuEq0YSY6tqVr8bdv++wL9AuQP9 AyhnJCikNm3EEldaphEhQ4S7bhJbUTzU5gBGLB+rEhwTFtH1CM5+ntXyRrhuh5lv9ojF lj7g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:dkim-filter; bh=qaDL2mmpN8TS26tnO1H3zakX/zt6JEohc+mmOh0RcEs=; b=oTeObw9TAskehzidAnRQ/anxE5/STrhy6KlnJMjMaQyLiKAXmc+kAvars8Rrst5dzh GaUmOzR7dloGuTouKDPaz6vtB+IaY+20OoIlYyfM4JHhDtdGR1MgVmGfVkVFMBXJ1Q0f lpVZPNgIqF8RouHRJL8s/COiaCesPVn+7duY9g23l/mZMSfKn/S4uL5UOonVcf0YvU0k BJtJZLJ+oCs2rmtmO3LrSzkITVuO6XzLldVVPznf99znyeDuCYtcS5RVsejWkhKHZar4 SJVStqoQjCDcBZyw+I8tK2GI1MMDDY2fSp2PuU6RRsO0I1YxblZdUQmdphajIUaiupiU rkNw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=jO37sjZt; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id bn14-20020a056a02030e00b00476bba7cc00si749361pgb.85.2022.12.08.21.36.50; Thu, 08 Dec 2022 21:37:03 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=jO37sjZt; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229770AbiLIFdA (ORCPT + 99 others); Fri, 9 Dec 2022 00:33:00 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52860 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229732AbiLIFcv (ORCPT ); Fri, 9 Dec 2022 00:32:51 -0500 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 09EF7275FC; Thu, 8 Dec 2022 21:32:50 -0800 (PST) Received: from jinankjain-dranzer.zrrkmle5drku1h0apvxbr2u2ee.ix.internal.cloudapp.net (unknown [20.188.121.5]) by linux.microsoft.com (Postfix) with ESMTPSA id DC6D520B83E2; Thu, 8 Dec 2022 21:32:44 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com DC6D520B83E2 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1670563969; bh=qaDL2mmpN8TS26tnO1H3zakX/zt6JEohc+mmOh0RcEs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jO37sjZtSt7u1R9NoVVE1+RCruEHTeUy2YbTEggSRVg4K9G/wR/P+8ZSmabyoYM33 JrDDKJvkR+oVslToPcfaT0jsNShUv9Niv9/3uvxBkNFeI8j69QKY6jCddCqn9aObvw 81GUu5+APRpSf9f+w4oUGOmNE87Fq3Dym60cljGM= From: Jinank Jain To: jinankjain@microsoft.com Cc: kys@microsoft.com, haiyangz@microsoft.com, wei.liu@kernel.org, decui@microsoft.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, arnd@arndb.de, peterz@infradead.org, jpoimboe@kernel.org, jinankjain@linux.microsoft.com, seanjc@google.com, kirill.shutemov@linux.intel.com, ak@linux.intel.com, sathyanarayanan.kuppuswamy@linux.intel.com, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, anrayabh@linux.microsoft.com, mikelley@microsoft.com Subject: [PATCH v8 2/5] Drivers: hv: Setup synic registers in case of nested root partition Date: Fri, 9 Dec 2022 05:32:28 +0000 Message-Id: <9a6bcedcc7dab00982eae5c8b2622b46ee27d07e.1670561320.git.jinankjain@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-19.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_MED, SPF_HELO_PASS,SPF_PASS,USER_IN_DEF_DKIM_WL,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751713551606598840?= X-GMAIL-MSGID: =?utf-8?q?1751713551606598840?= Child partitions are free to allocate SynIC message and event page but in case of root partition it must use the pages allocated by Microsoft Hypervisor (MSHV). Base address for these pages can be found using synthetic MSRs exposed by MSHV. There is a slight difference in those MSRs for nested vs non-nested root partition. Signed-off-by: Jinank Jain --- arch/x86/include/asm/hyperv-tlfs.h | 11 ++++++ arch/x86/include/asm/mshyperv.h | 30 +++------------ arch/x86/kernel/cpu/mshyperv.c | 61 ++++++++++++++++++++++++++++++ drivers/hv/hv.c | 32 ++++++++++------ 4 files changed, 99 insertions(+), 35 deletions(-) diff --git a/arch/x86/include/asm/hyperv-tlfs.h b/arch/x86/include/asm/hyperv-tlfs.h index 58c03d18c235..b5019becb618 100644 --- a/arch/x86/include/asm/hyperv-tlfs.h +++ b/arch/x86/include/asm/hyperv-tlfs.h @@ -225,6 +225,17 @@ enum hv_isolation_type { #define HV_REGISTER_SINT14 0x4000009E #define HV_REGISTER_SINT15 0x4000009F +/* + * Define synthetic interrupt controller model specific registers for + * nested hypervisor. + */ +#define HV_REGISTER_NESTED_SCONTROL 0x40001080 +#define HV_REGISTER_NESTED_SVERSION 0x40001081 +#define HV_REGISTER_NESTED_SIEFP 0x40001082 +#define HV_REGISTER_NESTED_SIMP 0x40001083 +#define HV_REGISTER_NESTED_EOM 0x40001084 +#define HV_REGISTER_NESTED_SINT0 0x40001090 + /* * Synthetic Timer MSRs. Four timers per vcpu. */ diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h index 61f0c206bff0..c38e4c66a3ac 100644 --- a/arch/x86/include/asm/mshyperv.h +++ b/arch/x86/include/asm/mshyperv.h @@ -198,30 +198,10 @@ static inline bool hv_is_synic_reg(unsigned int reg) return false; } -static inline u64 hv_get_register(unsigned int reg) -{ - u64 value; - - if (hv_is_synic_reg(reg) && hv_isolation_type_snp()) - hv_ghcb_msr_read(reg, &value); - else - rdmsrl(reg, value); - return value; -} - -static inline void hv_set_register(unsigned int reg, u64 value) -{ - if (hv_is_synic_reg(reg) && hv_isolation_type_snp()) { - hv_ghcb_msr_write(reg, value); - - /* Write proxy bit via wrmsl instruction */ - if (reg >= HV_REGISTER_SINT0 && - reg <= HV_REGISTER_SINT15) - wrmsrl(reg, value | 1 << 20); - } else { - wrmsrl(reg, value); - } -} +u64 hv_get_register(unsigned int reg); +void hv_set_register(unsigned int reg, u64 value); +u64 hv_get_non_nested_register(unsigned int reg); +void hv_set_non_nested_register(unsigned int reg, u64 value); #else /* CONFIG_HYPERV */ static inline void hyperv_init(void) {} @@ -241,6 +221,8 @@ static inline int hyperv_flush_guest_mapping_range(u64 as, } static inline void hv_set_register(unsigned int reg, u64 value) { } static inline u64 hv_get_register(unsigned int reg) { return 0; } +static inline void hv_set_non_nested_register(unsigned int reg, u64 value) { } +static inline u64 hv_get_non_nested_register(unsigned int reg) { return 0; } static inline int hv_set_mem_host_visibility(unsigned long addr, int numpages, bool visible) { diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c index f9b78d4829e3..47ffec5de9b8 100644 --- a/arch/x86/kernel/cpu/mshyperv.c +++ b/arch/x86/kernel/cpu/mshyperv.c @@ -41,7 +41,68 @@ bool hv_root_partition; bool hv_nested; struct ms_hyperv_info ms_hyperv; +static inline unsigned int hv_get_nested_reg(unsigned int reg) +{ + switch (reg) { + case HV_REGISTER_SIMP: + return HV_REGISTER_NESTED_SIMP; + case HV_REGISTER_SIEFP: + return HV_REGISTER_NESTED_SIEFP; + case HV_REGISTER_SVERSION: + return HV_REGISTER_NESTED_SVERSION; + case HV_REGISTER_SCONTROL: + return HV_REGISTER_NESTED_SCONTROL; + case HV_REGISTER_SINT0: + return HV_REGISTER_NESTED_SINT0; + case HV_REGISTER_EOM: + return HV_REGISTER_NESTED_EOM; + default: + return reg; + } +} + #if IS_ENABLED(CONFIG_HYPERV) +u64 hv_get_non_nested_register(unsigned int reg) +{ + u64 value; + + if (hv_is_synic_reg(reg) && hv_isolation_type_snp()) + hv_ghcb_msr_read(reg, &value); + else + rdmsrl(reg, value); + return value; +} + +void hv_set_non_nested_register(unsigned int reg, u64 value) +{ + if (hv_is_synic_reg(reg) && hv_isolation_type_snp()) { + hv_ghcb_msr_write(reg, value); + + /* Write proxy bit via wrmsl instruction */ + if (reg >= HV_REGISTER_SINT0 && + reg <= HV_REGISTER_SINT15) + wrmsrl(reg, value | 1 << 20); + } else { + wrmsrl(reg, value); + } +} + +u64 hv_get_register(unsigned int reg) +{ + if (hv_nested) + reg = hv_get_nested_reg(reg); + + return hv_get_non_nested_register(reg); +} + +void hv_set_register(unsigned int reg, u64 value) +{ + if (hv_nested) + reg = hv_get_nested_reg(reg); + + hv_set_non_nested_register(reg, value); +} + static void (*vmbus_handler)(void); static void (*hv_stimer0_handler)(void); static void (*hv_kexec_handler)(void); diff --git a/drivers/hv/hv.c b/drivers/hv/hv.c index 4d6480d57546..a422cb7b18d3 100644 --- a/drivers/hv/hv.c +++ b/drivers/hv/hv.c @@ -147,7 +147,7 @@ int hv_synic_alloc(void) * Synic message and event pages are allocated by paravisor. * Skip these pages allocation here. */ - if (!hv_isolation_type_snp()) { + if (!hv_isolation_type_snp() && !hv_root_partition) { hv_cpu->synic_message_page = (void *)get_zeroed_page(GFP_ATOMIC); if (hv_cpu->synic_message_page == NULL) { @@ -188,8 +188,16 @@ void hv_synic_free(void) struct hv_per_cpu_context *hv_cpu = per_cpu_ptr(hv_context.cpu_context, cpu); - free_page((unsigned long)hv_cpu->synic_event_page); - free_page((unsigned long)hv_cpu->synic_message_page); + if (hv_root_partition) { + if (hv_cpu->synic_event_page != NULL) + memunmap(hv_cpu->synic_event_page); + + if (hv_cpu->synic_message_page != NULL) + memunmap(hv_cpu->synic_message_page); + } else { + free_page((unsigned long)hv_cpu->synic_event_page); + free_page((unsigned long)hv_cpu->synic_message_page); + } free_page((unsigned long)hv_cpu->post_msg_page); } @@ -214,9 +222,10 @@ void hv_synic_enable_regs(unsigned int cpu) /* Setup the Synic's message page */ simp.as_uint64 = hv_get_register(HV_REGISTER_SIMP); + simp.simp_enabled = 1; - if (hv_isolation_type_snp()) { + if (hv_isolation_type_snp() || hv_root_partition) { hv_cpu->synic_message_page = memremap(simp.base_simp_gpa << HV_HYP_PAGE_SHIFT, HV_HYP_PAGE_SIZE, MEMREMAP_WB); @@ -233,7 +242,7 @@ void hv_synic_enable_regs(unsigned int cpu) siefp.as_uint64 = hv_get_register(HV_REGISTER_SIEFP); siefp.siefp_enabled = 1; - if (hv_isolation_type_snp()) { + if (hv_isolation_type_snp() || hv_root_partition) { hv_cpu->synic_event_page = memremap(siefp.base_siefp_gpa << HV_HYP_PAGE_SHIFT, HV_HYP_PAGE_SIZE, MEMREMAP_WB); @@ -250,8 +259,8 @@ void hv_synic_enable_regs(unsigned int cpu) /* Setup the shared SINT. */ if (vmbus_irq != -1) enable_percpu_irq(vmbus_irq, 0); - shared_sint.as_uint64 = hv_get_register(HV_REGISTER_SINT0 + - VMBUS_MESSAGE_SINT); + shared_sint.as_uint64 = + hv_get_register(HV_REGISTER_SINT0 + VMBUS_MESSAGE_SINT); shared_sint.vector = vmbus_interrupt; shared_sint.masked = false; @@ -267,7 +276,7 @@ void hv_synic_enable_regs(unsigned int cpu) shared_sint.auto_eoi = 0; #endif hv_set_register(HV_REGISTER_SINT0 + VMBUS_MESSAGE_SINT, - shared_sint.as_uint64); + shared_sint.as_uint64); /* Enable the global synic bit */ sctrl.as_uint64 = hv_get_register(HV_REGISTER_SCONTROL); @@ -297,15 +306,15 @@ void hv_synic_disable_regs(unsigned int cpu) union hv_synic_siefp siefp; union hv_synic_scontrol sctrl; - shared_sint.as_uint64 = hv_get_register(HV_REGISTER_SINT0 + - VMBUS_MESSAGE_SINT); + shared_sint.as_uint64 = + hv_get_register(HV_REGISTER_SINT0 + VMBUS_MESSAGE_SINT); shared_sint.masked = 1; /* Need to correctly cleanup in the case of SMP!!! */ /* Disable the interrupt */ hv_set_register(HV_REGISTER_SINT0 + VMBUS_MESSAGE_SINT, - shared_sint.as_uint64); + shared_sint.as_uint64); simp.as_uint64 = hv_get_register(HV_REGISTER_SIMP); /* @@ -335,6 +344,7 @@ void hv_synic_disable_regs(unsigned int cpu) /* Disable the global synic bit */ sctrl.as_uint64 = hv_get_register(HV_REGISTER_SCONTROL); sctrl.enable = 0; + hv_set_register(HV_REGISTER_SCONTROL, sctrl.as_uint64); if (vmbus_irq != -1) From patchwork Fri Dec 9 05:32:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinank Jain X-Patchwork-Id: 31626 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp594099wrr; Thu, 8 Dec 2022 21:37:20 -0800 (PST) X-Google-Smtp-Source: AA0mqf4YMnQG9SN1EVtzbcx7M1DnCEkFxJoa8FMp51lnx1D7SoNXYRq/b61a+JrM+Vhe31iWK1d6 X-Received: by 2002:a17:903:234e:b0:189:e545:7254 with SMTP id c14-20020a170903234e00b00189e5457254mr5943155plh.23.1670564240071; Thu, 08 Dec 2022 21:37:20 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670564240; cv=none; d=google.com; s=arc-20160816; b=0BW+MRRVVscD0WSOTyRv8ge6lZR8VTI1ha/q4C5tDL0QPYZsbayrYLNV9wukQkYBHW N51uy9aHmhryboZJaILgax4LM+C5Vu5Bi9KGcgw4bj39XRMbtXgKAa4TtAt63Y1JoNbX bPU3PWMW+syOZqIWS/6kX/1XvM1gMF3bfgCY5N/+tHNFcANvogFvuZTv3i4/g+QpHA9J 7FwhPFjkvNeOlkN+SE1uO2AITMmSbhh0lwDx7AYWQ2iZA2fn9mabuMmytcpVGbZiAhQU 5vG+q8njAKUjgiBOtHaa2Pq5DLo41SvHVTN7qPL5W7lDJojpmCYmaxK+Mup8y6nRyC+4 56oQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:dkim-filter; bh=+G/kUQSzne20Dwm9Bppxhzht3O3PdEtef4PJKNhBrEs=; b=wHlJOlGnZhPabthdmHvMS4TzCWFus9CeUw7GAim2iE3RvkL4vbuMIi0eV1fLH1Degr sYTyaARrp1b7SLWRqw+N4RxOxs2EPwcVc1OrowV870kFAVI5GN9tpln7g41hlvfCnT1s fn/M4g4Vbu9XT+PT/xRrlBDiq6b2aHUWutyczqvl6l1EY5R3d2WGQkiI22d90zCcYK2N afchUy8tQ6O1Q79areON8nrTPA77Iv8zZpuKffOmDTOmNcg/zovG2K9B7loTkbfsBFAs gN6IBQ02Yapksnvp6CO5xU6YrKEhR6yq7Yl0RwAoE70jE2WYJoPXNQszMdW92GOEujbQ omhg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=BfLx5ufv; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id p16-20020a1709027ed000b00182c510591asi631683plb.246.2022.12.08.21.37.07; Thu, 08 Dec 2022 21:37:20 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=BfLx5ufv; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229732AbiLIFdQ (ORCPT + 99 others); Fri, 9 Dec 2022 00:33:16 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53228 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229762AbiLIFc6 (ORCPT ); Fri, 9 Dec 2022 00:32:58 -0500 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 6708E50D65; Thu, 8 Dec 2022 21:32:55 -0800 (PST) Received: from jinankjain-dranzer.zrrkmle5drku1h0apvxbr2u2ee.ix.internal.cloudapp.net (unknown [20.188.121.5]) by linux.microsoft.com (Postfix) with ESMTPSA id 4670A20B83E5; Thu, 8 Dec 2022 21:32:50 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 4670A20B83E5 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1670563975; bh=+G/kUQSzne20Dwm9Bppxhzht3O3PdEtef4PJKNhBrEs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=BfLx5ufv1mOkAdZiXu+zisuSOnaKGy3O6turc42FYBApJHDXRpLUWo7eCfr3hEXc/ saCk8gXeShmU+X3pJ6DwhE6W2oGxRwkOHmfvYEfq6CFpDYhaG2XfiEqOgWHnxCqrYk M9bILZqeOlkpZ8FyHuVT0uxN+aJ06ONp+rTwR0GU= From: Jinank Jain To: jinankjain@microsoft.com Cc: kys@microsoft.com, haiyangz@microsoft.com, wei.liu@kernel.org, decui@microsoft.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, arnd@arndb.de, peterz@infradead.org, jpoimboe@kernel.org, jinankjain@linux.microsoft.com, seanjc@google.com, kirill.shutemov@linux.intel.com, ak@linux.intel.com, sathyanarayanan.kuppuswamy@linux.intel.com, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, anrayabh@linux.microsoft.com, mikelley@microsoft.com Subject: [PATCH v8 3/5] x86/hyperv: Add an interface to do nested hypercalls Date: Fri, 9 Dec 2022 05:32:29 +0000 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-19.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_MED, SPF_HELO_PASS,SPF_PASS,USER_IN_DEF_DKIM_WL,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751713568603985972?= X-GMAIL-MSGID: =?utf-8?q?1751713568603985972?= According to TLFS, in order to communicate to L0 hypervisor there needs to be an additional bit set in the control register. This communication is required to perform privileged instructions which can only be performed by L0 hypervisor. An example of that could be setting up the VMBus infrastructure. Signed-off-by: Jinank Jain --- arch/x86/include/asm/hyperv-tlfs.h | 3 ++- arch/x86/include/asm/mshyperv.h | 42 +++++++++++++++++++++++++++--- include/asm-generic/hyperv-tlfs.h | 1 + 3 files changed, 41 insertions(+), 5 deletions(-) diff --git a/arch/x86/include/asm/hyperv-tlfs.h b/arch/x86/include/asm/hyperv-tlfs.h index b5019becb618..7758c495541d 100644 --- a/arch/x86/include/asm/hyperv-tlfs.h +++ b/arch/x86/include/asm/hyperv-tlfs.h @@ -380,7 +380,8 @@ struct hv_nested_enlightenments_control { __u32 reserved:31; } features; struct { - __u32 reserved; + __u32 inter_partition_comm:1; + __u32 reserved:31; } hypercallControls; } __packed; diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h index c38e4c66a3ac..0272733166df 100644 --- a/arch/x86/include/asm/mshyperv.h +++ b/arch/x86/include/asm/mshyperv.h @@ -74,10 +74,16 @@ static inline u64 hv_do_hypercall(u64 control, void *input, void *output) return hv_status; } +/* Hypercall to the L0 hypervisor */ +static inline u64 hv_do_nested_hypercall(u64 control, void *input, void *output) +{ + return hv_do_hypercall(control | HV_HYPERCALL_NESTED, input, output); +} + /* Fast hypercall with 8 bytes of input and no output */ -static inline u64 hv_do_fast_hypercall8(u16 code, u64 input1) +static inline u64 _hv_do_fast_hypercall8(u64 control, u16 code, u64 input1) { - u64 hv_status, control = (u64)code | HV_HYPERCALL_FAST_BIT; + u64 hv_status; #ifdef CONFIG_X86_64 { @@ -105,10 +111,24 @@ static inline u64 hv_do_fast_hypercall8(u16 code, u64 input1) return hv_status; } +static inline u64 hv_do_fast_hypercall8(u16 code, u64 input1) +{ + u64 control = (u64)code | HV_HYPERCALL_FAST_BIT; + + return _hv_do_fast_hypercall8(control, code, input1); +} + +static inline u64 hv_do_fast_nested_hypercall8(u16 code, u64 input1) +{ + u64 control = (u64)code | HV_HYPERCALL_FAST_BIT | HV_HYPERCALL_NESTED; + + return _hv_do_fast_hypercall8(control, code, input1); +} + /* Fast hypercall with 16 bytes of input */ -static inline u64 hv_do_fast_hypercall16(u16 code, u64 input1, u64 input2) +static inline u64 _hv_do_fast_hypercall16(u64 control, u16 code, u64 input1, u64 input2) { - u64 hv_status, control = (u64)code | HV_HYPERCALL_FAST_BIT; + u64 hv_status; #ifdef CONFIG_X86_64 { @@ -139,6 +159,20 @@ static inline u64 hv_do_fast_hypercall16(u16 code, u64 input1, u64 input2) return hv_status; } +static inline u64 hv_do_fast_hypercall16(u16 code, u64 input1, u64 input2) +{ + u64 control = (u64)code | HV_HYPERCALL_FAST_BIT; + + return _hv_do_fast_hypercall16(control, code, input1, input2); +} + +static inline u64 hv_do_fast_nested_hypercall16(u16 code, u64 input1, u64 input2) +{ + u64 control = (u64)code | HV_HYPERCALL_FAST_BIT | HV_HYPERCALL_NESTED; + + return _hv_do_fast_hypercall16(control, code, input1, input2); +} + extern struct hv_vp_assist_page **hv_vp_assist_page; static inline struct hv_vp_assist_page *hv_get_vp_assist_page(unsigned int cpu) diff --git a/include/asm-generic/hyperv-tlfs.h b/include/asm-generic/hyperv-tlfs.h index b17c6eeb9afa..e61ee461c4fc 100644 --- a/include/asm-generic/hyperv-tlfs.h +++ b/include/asm-generic/hyperv-tlfs.h @@ -194,6 +194,7 @@ enum HV_GENERIC_SET_FORMAT { #define HV_HYPERCALL_VARHEAD_OFFSET 17 #define HV_HYPERCALL_VARHEAD_MASK GENMASK_ULL(26, 17) #define HV_HYPERCALL_RSVD0_MASK GENMASK_ULL(31, 27) +#define HV_HYPERCALL_NESTED BIT_ULL(31) #define HV_HYPERCALL_REP_COMP_OFFSET 32 #define HV_HYPERCALL_REP_COMP_1 BIT_ULL(32) #define HV_HYPERCALL_REP_COMP_MASK GENMASK_ULL(43, 32) From patchwork Fri Dec 9 05:32:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinank Jain X-Patchwork-Id: 31623 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp593702wrr; Thu, 8 Dec 2022 21:35:52 -0800 (PST) X-Google-Smtp-Source: AA0mqf7jUjvh2NugNRwox9pdmeWNozLiFnCPtfrAF9Qgx8axWUBxz/At6Da6lzFLcABB/hqAHm8/ X-Received: by 2002:a17:906:fb02:b0:78d:f454:386d with SMTP id lz2-20020a170906fb0200b0078df454386dmr3874909ejb.42.1670564152574; Thu, 08 Dec 2022 21:35:52 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670564152; cv=none; d=google.com; s=arc-20160816; b=LRmliZssCm+evQDYkmWxzjEh8IQzdNsDbJ0V6qVjwV86fFVX8rSGVuZx1caru2UpvV kYU/qDQaBS4xm4s7itLPGD4Kphz1FTTbYMgJfS1NT1Z2/k+OWDiPwwunqmqgCrVBgf1Z SZZG+JyLV5da9HIR1ZQdryoa1cdui3bgBmYT1GmujD8Ew1pzhGhGXi0+duPvYm3CD0Sz JhuCTnH1QuwUbAiXTYWK9lnkVjXrvvtAEPfAfJvDue2QZ8XgbodpNf5Xx4Lw7RxWq/1O 3c8cBzD5EugILQFOuIhzB1xRihGiFPrKmTDXacr8sDJbG7caahTFQjC2M1C0fIUyQAKS fARA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:dkim-filter; bh=WwgTdzq933yo46nX8RcCPBXut+LFdh0yPYBgTMkkcFM=; b=MPvkUUtC9hpWhFtIpvbh17mc1vq6eLK+pYFSiHN68EK8womGBbktPId1JZMl9wyXLB E7B7qAfeD2+ppBgvVq1Ey2KU1bUnP6+ic74pxYuicGmQuBb9l6BlB38zGgAxMWV4nbwV HO2/Hi2514P9skBap229iSu7pk4NFRP7ZQzaQRrMAT9KlXQrbFvuacq1MUdYc7kdJPpi DLL56IyEr2VQFLg28tJk5/aBKJREWtAnp+qf/KnXMg17yybZ25AK+z5NE2l2Z4shlWMY w0WJA4YBf1Eiq2cXpsVahRNRWJeSMOcESgyo7OHHY9PHqkzarXsbpTj1W9rehXr7WRT4 JPUg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b="MeGlx//V"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id a20-20020a17090680d400b007ae2368c8b7si422607ejx.138.2022.12.08.21.35.22; Thu, 08 Dec 2022 21:35:52 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b="MeGlx//V"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229632AbiLIFdj (ORCPT + 99 others); Fri, 9 Dec 2022 00:33:39 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52954 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229795AbiLIFdP (ORCPT ); Fri, 9 Dec 2022 00:33:15 -0500 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id C0C546B9A8; Thu, 8 Dec 2022 21:33:00 -0800 (PST) Received: from jinankjain-dranzer.zrrkmle5drku1h0apvxbr2u2ee.ix.internal.cloudapp.net (unknown [20.188.121.5]) by linux.microsoft.com (Postfix) with ESMTPSA id A4AC420B83E7; Thu, 8 Dec 2022 21:32:55 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com A4AC420B83E7 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1670563980; bh=WwgTdzq933yo46nX8RcCPBXut+LFdh0yPYBgTMkkcFM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=MeGlx//V0m4UF8HG4q5d2nTA+RsFVtz867EPSiVC6M2Xku3r7VzqPNZsGnkExDqfF b1X2m+a5uC2Ifi6Erz8InVQmS0R2zJ2iuj90y5I23rxbHK17FnoE8OeZetTPthiDl3 amHlCR9BE2injKxuqf6V0hH7tI79u+0DQya4cMIE= From: Jinank Jain To: jinankjain@microsoft.com Cc: kys@microsoft.com, haiyangz@microsoft.com, wei.liu@kernel.org, decui@microsoft.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, arnd@arndb.de, peterz@infradead.org, jpoimboe@kernel.org, jinankjain@linux.microsoft.com, seanjc@google.com, kirill.shutemov@linux.intel.com, ak@linux.intel.com, sathyanarayanan.kuppuswamy@linux.intel.com, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, anrayabh@linux.microsoft.com, mikelley@microsoft.com Subject: [PATCH v8 4/5] Drivers: hv: Enable vmbus driver for nested root partition Date: Fri, 9 Dec 2022 05:32:30 +0000 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-19.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_MED, SPF_HELO_PASS,SPF_PASS,USER_IN_DEF_DKIM_WL,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751713476829573137?= X-GMAIL-MSGID: =?utf-8?q?1751713476829573137?= Currently VMBus driver is not initialized for root partition but we need to enable the VMBus driver for nested root partition. This is required, so that L2 root can use the VMBus devices. Signed-off-by: Jinank Jain Reviewed-by: Nuno Das Neves --- drivers/hv/vmbus_drv.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c index 0f00d57b7c25..6324e01d5eec 100644 --- a/drivers/hv/vmbus_drv.c +++ b/drivers/hv/vmbus_drv.c @@ -2745,7 +2745,7 @@ static int __init hv_acpi_init(void) if (!hv_is_hyperv_initialized()) return -ENODEV; - if (hv_root_partition) + if (hv_root_partition && !hv_nested) return 0; /* From patchwork Fri Dec 9 05:32:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinank Jain X-Patchwork-Id: 31627 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp594197wrr; Thu, 8 Dec 2022 21:37:42 -0800 (PST) X-Google-Smtp-Source: AA0mqf7zW+FlYzSsHmiIk5g1oDB+WmdccF2rPi0vfNaqfWvKSZdTVkjivl3wAKEXiZGY1CTYZsz5 X-Received: by 2002:a17:902:bf06:b0:188:758f:f472 with SMTP id bi6-20020a170902bf0600b00188758ff472mr4637746plb.0.1670564262429; Thu, 08 Dec 2022 21:37:42 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670564262; cv=none; d=google.com; s=arc-20160816; b=BDoWZRWS8VosA7RxyjswY/RAskJeso4oZaihhP+M+vQvz54oSHCeJG4XJFRoaclYZr 4uAd4SIzm2p6Urrn2ZWcjddf8B/5n/xAR1N6r23hapOI0HkeeGA+nLRY3qoppdfg5eJP R0n7AsowfjWA+1CuVOe8txPiMmUkudVroYFjzJZvpWWj9wR4RiDl09NsUFke2T5jaD7P 8SV+XEL8OUXEhVvPcEE+MizaLSDWuDs9qBQRAKTeHFkJbnkLRw+1Oiz/xfWtOgIPcWSx YWA4b7dAaE5htog1aGkRW2efOGslAw/ykPXq1zTH/edmokZLmQSy8YYo+xmNBP8XuOu0 S5IA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:dkim-filter; bh=mvHPH26sfvX6dqItdLgyjEMsIkSbtmXqfahX8ng64PY=; b=uhXI90gDKl1fL94uxP6THlYk9lbK2EqYxlbH0b3LeIw4BMUx6OeHpFZZ2AC9h00L9I OsF1pjb3oJxwWUNen6Ce7GtSiJ4d8U+SZU2Kj13k3EIuqFLmZ4hWlNEQn8hgSWH0wiHa hEZeWOvODnYWf8BmoVQ/qECAG2vkwN00PGN+BL01UZNoThJQ5LUaYExMrBl9YmLjQsdw Dalj0Z947CsrCzj5/N4Fmcjc4WWrjpuUuqLL5uOkqSC+Kyh2A2qJobPmfU4Sh5mJ/WiG IczNawrIE/xdaU+zSo5ap1m5mU52eU4j3pegtgaep0mtDpRmNWZa3hC1hkVUBFmfHEzM qcMw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=p147wbFw; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id x24-20020a1709027c1800b0018965226454si657105pll.501.2022.12.08.21.37.29; Thu, 08 Dec 2022 21:37:42 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=p147wbFw; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229785AbiLIFd5 (ORCPT + 99 others); Fri, 9 Dec 2022 00:33:57 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53796 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229818AbiLIFda (ORCPT ); Fri, 9 Dec 2022 00:33:30 -0500 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id CC3E3944FB; Thu, 8 Dec 2022 21:33:06 -0800 (PST) Received: from jinankjain-dranzer.zrrkmle5drku1h0apvxbr2u2ee.ix.internal.cloudapp.net (unknown [20.188.121.5]) by linux.microsoft.com (Postfix) with ESMTPSA id 0E6E020B83C2; Thu, 8 Dec 2022 21:33:00 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 0E6E020B83C2 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1670563986; bh=mvHPH26sfvX6dqItdLgyjEMsIkSbtmXqfahX8ng64PY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=p147wbFw3DrZEzHTVIxYg3oYL6Z8IcWlnTbCq3q8QMp8YZzfsG8fY9EYRkLRF1s8a 848H5mnw+B00eXqn4m5V13bGnPGuEbiOE+Hd5xoHEi3i8ObSbY5VFQYEN87P0EkG7U c4Gz1byuW7/LQT5AhnwGLm/OMQf4kGP6I9sNlhfA= From: Jinank Jain To: jinankjain@microsoft.com Cc: kys@microsoft.com, haiyangz@microsoft.com, wei.liu@kernel.org, decui@microsoft.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, arnd@arndb.de, peterz@infradead.org, jpoimboe@kernel.org, jinankjain@linux.microsoft.com, seanjc@google.com, kirill.shutemov@linux.intel.com, ak@linux.intel.com, sathyanarayanan.kuppuswamy@linux.intel.com, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, anrayabh@linux.microsoft.com, mikelley@microsoft.com Subject: [PATCH v8 5/5] x86/hyperv: Change interrupt vector for nested root partition Date: Fri, 9 Dec 2022 05:32:31 +0000 Message-Id: <6e9988271f389360b1e14096e8529a8189fcb65d.1670561320.git.jinankjain@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-19.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_MED, SPF_HELO_PASS,SPF_PASS,USER_IN_DEF_DKIM_WL,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751713592022016244?= X-GMAIL-MSGID: =?utf-8?q?1751713592022016244?= Traditionally we have been using the HYPERVISOR_CALLBACK_VECTOR to relay the VMBus interrupt. But this does not work in case of nested hypervisor. Microsoft Hypervisor reserves 0x31 to 0x34 as the interrupt vector range for VMBus and thus we have to use one of the vectors from that range and setup the IDT accordingly. Signed-off-by: Jinank Jain --- arch/x86/include/asm/idtentry.h | 2 ++ arch/x86/include/asm/irq_vectors.h | 6 ++++++ arch/x86/kernel/cpu/mshyperv.c | 15 +++++++++++++++ arch/x86/kernel/idt.c | 10 ++++++++++ drivers/hv/vmbus_drv.c | 3 ++- 5 files changed, 35 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/idtentry.h b/arch/x86/include/asm/idtentry.h index 72184b0b2219..c0648e3e4d4a 100644 --- a/arch/x86/include/asm/idtentry.h +++ b/arch/x86/include/asm/idtentry.h @@ -686,6 +686,8 @@ DECLARE_IDTENTRY_SYSVEC(POSTED_INTR_NESTED_VECTOR, sysvec_kvm_posted_intr_nested DECLARE_IDTENTRY_SYSVEC(HYPERVISOR_CALLBACK_VECTOR, sysvec_hyperv_callback); DECLARE_IDTENTRY_SYSVEC(HYPERV_REENLIGHTENMENT_VECTOR, sysvec_hyperv_reenlightenment); DECLARE_IDTENTRY_SYSVEC(HYPERV_STIMER0_VECTOR, sysvec_hyperv_stimer0); +DECLARE_IDTENTRY_SYSVEC(HYPERV_INTR_NESTED_VMBUS_VECTOR, + sysvec_hyperv_nested_vmbus_intr); #endif #if IS_ENABLED(CONFIG_ACRN_GUEST) diff --git a/arch/x86/include/asm/irq_vectors.h b/arch/x86/include/asm/irq_vectors.h index 43dcb9284208..729d19eab7f5 100644 --- a/arch/x86/include/asm/irq_vectors.h +++ b/arch/x86/include/asm/irq_vectors.h @@ -102,6 +102,12 @@ #if IS_ENABLED(CONFIG_HYPERV) #define HYPERV_REENLIGHTENMENT_VECTOR 0xee #define HYPERV_STIMER0_VECTOR 0xed +/* + * FIXME: Change this, once Microsoft Hypervisor changes its assumption + * around VMBus interrupt vector allocation for nested root partition. + * Or provides a better interface to detect this instead of hardcoding. + */ +#define HYPERV_INTR_NESTED_VMBUS_VECTOR 0x31 #endif #define LOCAL_TIMER_VECTOR 0xec diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c index 47ffec5de9b8..dc2be733cadc 100644 --- a/arch/x86/kernel/cpu/mshyperv.c +++ b/arch/x86/kernel/cpu/mshyperv.c @@ -122,6 +122,21 @@ DEFINE_IDTENTRY_SYSVEC(sysvec_hyperv_callback) set_irq_regs(old_regs); } +DEFINE_IDTENTRY_SYSVEC(sysvec_hyperv_nested_vmbus_intr) +{ + struct pt_regs *old_regs = set_irq_regs(regs); + + inc_irq_stat(irq_hv_callback_count); + + if (vmbus_handler) + vmbus_handler(); + + if (ms_hyperv.hints & HV_DEPRECATING_AEOI_RECOMMENDED) + ack_APIC_irq(); + + set_irq_regs(old_regs); +} + void hv_setup_vmbus_handler(void (*handler)(void)) { vmbus_handler = handler; diff --git a/arch/x86/kernel/idt.c b/arch/x86/kernel/idt.c index a58c6bc1cd68..3536935cea39 100644 --- a/arch/x86/kernel/idt.c +++ b/arch/x86/kernel/idt.c @@ -160,6 +160,16 @@ static const __initconst struct idt_data apic_idts[] = { # endif INTG(SPURIOUS_APIC_VECTOR, asm_sysvec_spurious_apic_interrupt), INTG(ERROR_APIC_VECTOR, asm_sysvec_error_interrupt), +#ifdef CONFIG_HYPERV + /* + * This is a hack because we cannot install this interrupt handler + * via alloc_intr_gate as it does not allow interrupt vector less + * than FIRST_SYSTEM_VECTORS. And hyperv does not want anything other + * than 0x31-0x34 as the interrupt vector for vmbus interrupt in case + * of nested setup. + */ + INTG(HYPERV_INTR_NESTED_VMBUS_VECTOR, asm_sysvec_hyperv_nested_vmbus_intr), +#endif #endif }; diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c index 6324e01d5eec..740878367426 100644 --- a/drivers/hv/vmbus_drv.c +++ b/drivers/hv/vmbus_drv.c @@ -2768,7 +2768,8 @@ static int __init hv_acpi_init(void) * normal Linux IRQ mechanism is not used in this case. */ #ifdef HYPERVISOR_CALLBACK_VECTOR - vmbus_interrupt = HYPERVISOR_CALLBACK_VECTOR; + vmbus_interrupt = hv_nested ? HYPERV_INTR_NESTED_VMBUS_VECTOR : + HYPERVISOR_CALLBACK_VECTOR; vmbus_irq = -1; #endif