From patchwork Thu Nov 3 13:04:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinank Jain X-Patchwork-Id: 14863 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp519919wru; Thu, 3 Nov 2022 06:09:42 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6tUteQO7QxWNqEEubjctL4NUY9ky7K2Jj9Gm3jtkBvi/OMezSOWJOEqJBydmHq8rIh7YoX X-Received: by 2002:a63:f206:0:b0:446:eb31:47e0 with SMTP id v6-20020a63f206000000b00446eb3147e0mr26471937pgh.491.1667480982288; Thu, 03 Nov 2022 06:09:42 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667480982; cv=none; d=google.com; s=arc-20160816; b=CReN+xRWmccQsmkCwWncSJ4haicDBSSGPCnlpGqRNw52WhZJy1Wycv9jZm4d1vq24w MpVnzj432mCjpGI3H0CWXmxHAwQ4RBozDKdQrxGuuJ+sacOHoqKLcm/QimWPNglAnNi7 qkgHNymaeb09zAj2HfsVCiHfAr9sGKYF1L0HJutzWw2yOuGCmQkNo8Zh6MiUNrgoaI+/ bZLgbPQGYLgKkdSrYDU5B8rciXDcATu17Ls9sSGv8zZOGQglwwgwUaFOpwuptP5LHfTy PLviVvAjCHkY7lSWqOzaj6CaHPTOMbXFBJkxUyQZAqHXgEKSYYCDUnGdIivLW/DLx1X4 txtg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:dkim-filter; bh=HLgBEfqnfR/pSo3ZddLSeTCBJ4DSAnSoOy6+9I7tDbg=; b=IcjvYlOJRgJGze6h2Uqm4QeqLSpHfxCy2iplGsQSdTt7qb+k21Mw3DZxPkMWUGo7Ku avzZ5OZS1Ai4JSfnx/Q+XaS3UYYi1IWdOhQ/1oo60HN7KfvFTEjqNgcjK6PRsjhDn7fM dS/RYq98yrcj8IU2bNu/CAFHuNG+UNnV3u21JupY9hfOpavQI+Gf4W7b/VAgw6raXRUd rqYJ0AarD/7putf2PgFj04ZAb3CaVfkfPi/iMj+fx5rzfU7GW0SnoRCJ6lKJ+uzBtnH1 LRPgXwmp7/mAtrps/jc6imX4oKy049MhpZwtkOaPOk6qMa13s9WIrLA7Tx2xKsyqzKDZ 3Mjw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=tInIXzvZ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id lw3-20020a17090b180300b002131849ea4esi7378666pjb.140.2022.11.03.06.09.28; Thu, 03 Nov 2022 06:09:42 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=tInIXzvZ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231523AbiKCNFf (ORCPT + 99 others); Thu, 3 Nov 2022 09:05:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42808 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231226AbiKCNFL (ORCPT ); Thu, 3 Nov 2022 09:05:11 -0400 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id F358417E32; Thu, 3 Nov 2022 06:04:17 -0700 (PDT) Received: from jinankjain-dranzer.zrrkmle5drku1h0apvxbr2u2ee.ix.internal.cloudapp.net (unknown [20.188.121.5]) by linux.microsoft.com (Postfix) with ESMTPSA id A0E09205DA2F; Thu, 3 Nov 2022 06:04:13 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com A0E09205DA2F DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1667480657; bh=HLgBEfqnfR/pSo3ZddLSeTCBJ4DSAnSoOy6+9I7tDbg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=tInIXzvZ5DLoKnJIuYBKNbwTa4rSOgbayfk53bVeJyTmK26j7jjfnGAuzBxv2YiR3 3bR2rI8U/yG2dnJxzMBLwr9g16xnc4PWetTLiDM7QG3jRbkJ74Y8vyLLuUnPG8a2Li Q6dVE3ZkxlJuz+jLcNIFgwtwwh1m1xPSVqTTNfPs= From: Jinank Jain To: jinankjain@microsoft.com Cc: kys@microsoft.com, haiyangz@microsoft.com, wei.liu@kernel.org, decui@microsoft.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, arnd@arndb.de, peterz@infradead.org, jpoimboe@kernel.org, jinankjain@linux.microsoft.com, seanjc@google.com, kirill.shutemov@linux.intel.com, ak@linux.intel.com, sathyanarayanan.kuppuswamy@linux.intel.com, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, anrayabh@linux.microsoft.com, mikelley@microsoft.com Subject: [PATCH v3 1/5] x86/hyperv: Add support for detecting nested hypervisor Date: Thu, 3 Nov 2022 13:04:03 +0000 Message-Id: <285b15b90ac6f29ef8ab6b6ececeeef7d7c6f380.1667480257.git.jinankjain@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-19.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_MED, SPF_HELO_PASS,SPF_PASS,USER_IN_DEF_DKIM_WL,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748480538366490741?= X-GMAIL-MSGID: =?utf-8?q?1748480538366490741?= When Linux runs as a root partition for Microsoft Hypervisor. It is possible to detect if it is running as nested hypervisor using hints exposed by mshv. While at it expose a new variable called hv_nested which can be used later for making decisions specific to nested use case. Signed-off-by: Jinank Jain Reviewed-by: --- arch/x86/include/asm/hyperv-tlfs.h | 3 +++ arch/x86/include/asm/mshyperv.h | 2 ++ arch/x86/kernel/cpu/mshyperv.c | 7 +++++++ drivers/hv/hv_common.c | 7 +++++-- 4 files changed, 17 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/hyperv-tlfs.h b/arch/x86/include/asm/hyperv-tlfs.h index 3089ec352743..d9a611565859 100644 --- a/arch/x86/include/asm/hyperv-tlfs.h +++ b/arch/x86/include/asm/hyperv-tlfs.h @@ -114,6 +114,9 @@ /* Recommend using the newer ExProcessorMasks interface */ #define HV_X64_EX_PROCESSOR_MASKS_RECOMMENDED BIT(11) +/* Indicates that the hypervisor is nested within a Hyper-V partition. */ +#define HV_X64_HYPERV_NESTED BIT(12) + /* Recommend using enlightened VMCS */ #define HV_X64_ENLIGHTENED_VMCS_RECOMMENDED BIT(14) diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h index 61f0c206bff0..3c39923e5969 100644 --- a/arch/x86/include/asm/mshyperv.h +++ b/arch/x86/include/asm/mshyperv.h @@ -26,6 +26,8 @@ void hyperv_vector_handler(struct pt_regs *regs); #if IS_ENABLED(CONFIG_HYPERV) extern int hyperv_init_cpuhp; +extern bool hv_nested; + extern void *hv_hypercall_pg; extern u64 hv_current_partition_id; diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c index 831613959a92..9a4204139490 100644 --- a/arch/x86/kernel/cpu/mshyperv.c +++ b/arch/x86/kernel/cpu/mshyperv.c @@ -37,6 +37,8 @@ /* Is Linux running as the root partition? */ bool hv_root_partition; +/* Is Linux running on nested Microsoft Hypervisor */ +bool hv_nested; struct ms_hyperv_info ms_hyperv; #if IS_ENABLED(CONFIG_HYPERV) @@ -301,6 +303,11 @@ static void __init ms_hyperv_init_platform(void) pr_info("Hyper-V: running as root partition\n"); } + if (ms_hyperv.hints & HV_X64_HYPERV_NESTED) { + hv_nested = true; + pr_info("Hyper-V: running on a nested hypervisor\n"); + } + /* * Extract host information. */ diff --git a/drivers/hv/hv_common.c b/drivers/hv/hv_common.c index ae68298c0dca..dcb336ce374f 100644 --- a/drivers/hv/hv_common.c +++ b/drivers/hv/hv_common.c @@ -25,8 +25,8 @@ #include /* - * hv_root_partition and ms_hyperv are defined here with other Hyper-V - * specific globals so they are shared across all architectures and are + * hv_root_partition, ms_hyperv and hv_nested are defined here with other + * Hyper-V specific globals so they are shared across all architectures and are * built only when CONFIG_HYPERV is defined. But on x86, * ms_hyperv_init_platform() is built even when CONFIG_HYPERV is not * defined, and it uses these two variables. So mark them as __weak @@ -36,6 +36,9 @@ bool __weak hv_root_partition; EXPORT_SYMBOL_GPL(hv_root_partition); +bool __weak hv_nested; +EXPORT_SYMBOL_GPL(hv_nested); + struct ms_hyperv_info __weak ms_hyperv; EXPORT_SYMBOL_GPL(ms_hyperv); From patchwork Thu Nov 3 13:04:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinank Jain X-Patchwork-Id: 14864 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp520287wru; Thu, 3 Nov 2022 06:10:14 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4Am5r+vQzCtV1mSMmiI9VGacxE8yuTbILw94hc9SY5rJde6pCwSNNrAWhgNWJwmfAnCWFH X-Received: by 2002:a17:906:8469:b0:7ad:dd13:571c with SMTP id hx9-20020a170906846900b007addd13571cmr19007264ejc.141.1667481014584; Thu, 03 Nov 2022 06:10:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667481014; cv=none; d=google.com; s=arc-20160816; b=PG4tvZA0D2aX0fgborZr2UfwJtsqgScFmgWKK8xkC/zK81/5YbDtu0p2XfArlDGHnt UXiMCTL1iyd9arNhCRmUgs8ZQ82YalymjwuWqY91sxlLGzcvzSgrDOtq99cdSie6Ncdm zFQDTDqo5khZQl9IZzA8DT6swHIq4vUdbtKDLXnCJRYoo2ch++hlyU1zKZEX2oet9jWA 7T+62YlhLyLhGxAdEcOPy3KCWT3pbrDxDXIepBVd5+rUfLIRpKJOCdMBYP0wRDzt8LCx uYCyOKB97//d4X1PzbibqzvesMwgPkaFnPR1TEOwy5RQekTar95Ye0fh7pkdkudV1k2P RBFQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:dkim-filter; bh=1XFrn6tG+npSpwC5Fwga/EnPrm38noeKd85Wn2Esosk=; b=H9k7URReCz+3zlLTVscRYSIUaZIrgpj6qSd/3aTJ2PNFKyQ5AefFWL6XvMetBvD/v6 OU5BUVeFbPEtcZk1NTPEY37z+kyYTl+UircqQUYcO8+tTwZ5IhkNFdQUW6tiCwzmtC/y QJuC13voWtNqAl21/nplU6/ao2pKaPxpa/HAwMV7lDXHPOgAghhQT1E7clFlVh29Xy8C H/lE/H39iS0Ms5pB5+re0FgRTap+/G9SbcqTTKpeQavJQeEa/1Iy/rLUC63u0vq1ddsq Ihzm2At/OYpMO6qf5QbQ3qLkojPHA0Zkr1s+HHHQm71QgbPD2yta/ReocoZNDojeoNF6 5QGg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b="k97Gect/"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id qa31-20020a170907869f00b0078e093ae419si1537298ejc.8.2022.11.03.06.09.48; Thu, 03 Nov 2022 06:10:14 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b="k97Gect/"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231582AbiKCNF6 (ORCPT + 99 others); Thu, 3 Nov 2022 09:05:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41758 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231364AbiKCNFQ (ORCPT ); Thu, 3 Nov 2022 09:05:16 -0400 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 80F73186CE; Thu, 3 Nov 2022 06:04:22 -0700 (PDT) Received: from jinankjain-dranzer.zrrkmle5drku1h0apvxbr2u2ee.ix.internal.cloudapp.net (unknown [20.188.121.5]) by linux.microsoft.com (Postfix) with ESMTPSA id 3196B205DA31; Thu, 3 Nov 2022 06:04:18 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 3196B205DA31 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1667480662; bh=1XFrn6tG+npSpwC5Fwga/EnPrm38noeKd85Wn2Esosk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=k97Gect/eMWgpedcbZb7OQNMzenVH/EuFb7Zn2QfW64o+isSvJlhx3p/xN88jXNSE sPxD1RBnaKW0aalFOC+1n0RlefCHjWEy16M0BTuMRpFTEkvPOqJE/+IjY4KHtWi979 zP6np+CucZbtLJ5vpcwVuxV0eAMA/7tEGzu17pu4= From: Jinank Jain To: jinankjain@microsoft.com Cc: kys@microsoft.com, haiyangz@microsoft.com, wei.liu@kernel.org, decui@microsoft.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, arnd@arndb.de, peterz@infradead.org, jpoimboe@kernel.org, jinankjain@linux.microsoft.com, seanjc@google.com, kirill.shutemov@linux.intel.com, ak@linux.intel.com, sathyanarayanan.kuppuswamy@linux.intel.com, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, anrayabh@linux.microsoft.com, mikelley@microsoft.com Subject: [PATCH v3 2/5] Drivers: hv: Setup synic registers in case of nested root partition Date: Thu, 3 Nov 2022 13:04:04 +0000 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-19.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_MED, SPF_HELO_PASS,SPF_PASS,USER_IN_DEF_DKIM_WL,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748480572287383565?= X-GMAIL-MSGID: =?utf-8?q?1748480572287383565?= Child partitions are free to allocate SynIC message and event page but in case of root partition it must use the pages allocated by Microsoft Hypervisor (MSHV). Base address for these pages can be found using synthetic MSRs exposed by MSHV. There is a slight difference in those MSRs for nested vs non-nested root partition. Signed-off-by: Jinank Jain --- arch/x86/include/asm/hyperv-tlfs.h | 11 +++++++++++ arch/x86/include/asm/mshyperv.h | 24 ++++++++++++++++++++++++ drivers/hv/hv.c | 18 +++++++++++++----- 3 files changed, 48 insertions(+), 5 deletions(-) diff --git a/arch/x86/include/asm/hyperv-tlfs.h b/arch/x86/include/asm/hyperv-tlfs.h index d9a611565859..0319091e2019 100644 --- a/arch/x86/include/asm/hyperv-tlfs.h +++ b/arch/x86/include/asm/hyperv-tlfs.h @@ -225,6 +225,17 @@ enum hv_isolation_type { #define HV_REGISTER_SINT14 0x4000009E #define HV_REGISTER_SINT15 0x4000009F +/* + * Define synthetic interrupt controller model specific registers for + * nested hypervisor. + */ +#define HV_REGISTER_NESTED_SCONTROL 0x40001080 +#define HV_REGISTER_NESTED_SVERSION 0x40001081 +#define HV_REGISTER_NESTED_SIEFP 0x40001082 +#define HV_REGISTER_NESTED_SIMP 0x40001083 +#define HV_REGISTER_NESTED_EOM 0x40001084 +#define HV_REGISTER_NESTED_SINT0 0x40001090 + /* * Synthetic Timer MSRs. Four timers per vcpu. */ diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h index 3c39923e5969..b0f16d06a0c5 100644 --- a/arch/x86/include/asm/mshyperv.h +++ b/arch/x86/include/asm/mshyperv.h @@ -200,10 +200,31 @@ static inline bool hv_is_synic_reg(unsigned int reg) return false; } +static inline unsigned int hv_get_nested_reg(unsigned int reg) +{ + switch (reg) { + case HV_REGISTER_SIMP: + return HV_REGISTER_NESTED_SIMP; + case HV_REGISTER_NESTED_SIEFP: + return HV_REGISTER_SIEFP; + case HV_REGISTER_SCONTROL: + return HV_REGISTER_NESTED_SCONTROL; + case HV_REGISTER_SINT0: + return HV_REGISTER_NESTED_SINT0; + case HV_REGISTER_EOM: + return HV_REGISTER_NESTED_EOM; + default: + return reg; + } +} + static inline u64 hv_get_register(unsigned int reg) { u64 value; + if (hv_nested) + reg = hv_get_nested_reg(reg); + if (hv_is_synic_reg(reg) && hv_isolation_type_snp()) hv_ghcb_msr_read(reg, &value); else @@ -213,6 +234,9 @@ static inline u64 hv_get_register(unsigned int reg) static inline void hv_set_register(unsigned int reg, u64 value) { + if (hv_nested) + reg = hv_get_nested_reg(reg); + if (hv_is_synic_reg(reg) && hv_isolation_type_snp()) { hv_ghcb_msr_write(reg, value); diff --git a/drivers/hv/hv.c b/drivers/hv/hv.c index 4d6480d57546..9e1eb50cc76f 100644 --- a/drivers/hv/hv.c +++ b/drivers/hv/hv.c @@ -147,7 +147,7 @@ int hv_synic_alloc(void) * Synic message and event pages are allocated by paravisor. * Skip these pages allocation here. */ - if (!hv_isolation_type_snp()) { + if (!hv_isolation_type_snp() && !hv_root_partition) { hv_cpu->synic_message_page = (void *)get_zeroed_page(GFP_ATOMIC); if (hv_cpu->synic_message_page == NULL) { @@ -188,8 +188,16 @@ void hv_synic_free(void) struct hv_per_cpu_context *hv_cpu = per_cpu_ptr(hv_context.cpu_context, cpu); - free_page((unsigned long)hv_cpu->synic_event_page); - free_page((unsigned long)hv_cpu->synic_message_page); + if (hv_root_partition) { + if (hv_cpu->synic_event_page != NULL) + memunmap(hv_cpu->synic_event_page); + + if (hv_cpu->synic_message_page != NULL) + memunmap(hv_cpu->synic_message_page); + } else { + free_page((unsigned long)hv_cpu->synic_event_page); + free_page((unsigned long)hv_cpu->synic_message_page); + } free_page((unsigned long)hv_cpu->post_msg_page); } @@ -216,7 +224,7 @@ void hv_synic_enable_regs(unsigned int cpu) simp.as_uint64 = hv_get_register(HV_REGISTER_SIMP); simp.simp_enabled = 1; - if (hv_isolation_type_snp()) { + if (hv_isolation_type_snp() || hv_root_partition) { hv_cpu->synic_message_page = memremap(simp.base_simp_gpa << HV_HYP_PAGE_SHIFT, HV_HYP_PAGE_SIZE, MEMREMAP_WB); @@ -233,7 +241,7 @@ void hv_synic_enable_regs(unsigned int cpu) siefp.as_uint64 = hv_get_register(HV_REGISTER_SIEFP); siefp.siefp_enabled = 1; - if (hv_isolation_type_snp()) { + if (hv_isolation_type_snp() || hv_root_partition) { hv_cpu->synic_event_page = memremap(siefp.base_siefp_gpa << HV_HYP_PAGE_SHIFT, HV_HYP_PAGE_SIZE, MEMREMAP_WB); From patchwork Thu Nov 3 13:04:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinank Jain X-Patchwork-Id: 14861 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp518990wru; Thu, 3 Nov 2022 06:08:34 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4iyxKFr4e89waVN2n4muP2kdET8RsauO6aIc0jv47d40K5Z1FR/RGGNXC1SAbDDWOPHfjx X-Received: by 2002:a05:6a00:1687:b0:565:a932:f05a with SMTP id k7-20020a056a00168700b00565a932f05amr13720953pfc.21.1667480914101; Thu, 03 Nov 2022 06:08:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667480914; cv=none; d=google.com; s=arc-20160816; b=vz9SWUv23a8bL1EwBVjc//q6v4vgINW51VH3s52nXrQhoO8rASvHG6Nm84+9928/X0 9yAOoj/ZVNn+GaTCkJpKSUnB+WuUCQ7EGSVxUjaAMtMU74CI/AFk+Yrac6plRHuJxTWR GgJzdFbidmtURJ27CHU7OhvZ3YyCYuEFitG32I+qVotatBwdI5KuTvJmVMuSs8YnCLTe TPmIClQBIZPjvMDl8OhVpxTjyPK73N47D3DQEymScfrYLvuMNjTo4Hvw3gxrWYgJRuAV hmg5aokKAj3LO7AvpPnJS3F3h4W8C0pQzJfimQineogkr3i/jSjKAGMJkCZKtiGZbNt0 Da6Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:dkim-filter; bh=Jr/pihgkFRoNlRyuUdMyy9zaVqebY39GQd4GO2z7V9s=; b=Hh8YnFQq6JDcUeMQJwzdlgL6an8QN30zNIqwumbbi2OH6+kXKkTzK6pyioH6qvcD/Y ZMYf77qktU72wU1wChlwsrdPxOywyXg6vc4z2vU4ytTpVfVuHFifYUPw+JibYtHZdvo9 jREfPPtN+tdFIqQU81wUXkk+xrVq4d1lWKczrFHbQPNJ9OnEY++SFLXYj1529NHb7Vep gUntO2SjLj/3tILLn3PP5o5FlwB3Y+96jJxN/LhwqxCMaJptrTbO8Vl+YakQY3a2vlcO o/hR0j9bEcfvg+K7+bRfNu4Y5diOug8FN+wlnmhyYRt1A+DlUZHTHqNwll8qkLc5Dpp2 FJsQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=rItutFLv; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id c12-20020a655a8c000000b0046efd09c4f2si1173598pgt.440.2022.11.03.06.08.20; Thu, 03 Nov 2022 06:08:34 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=rItutFLv; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230233AbiKCNGC (ORCPT + 99 others); Thu, 3 Nov 2022 09:06:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42904 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231201AbiKCNFR (ORCPT ); Thu, 3 Nov 2022 09:05:17 -0400 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 10CC018B06; Thu, 3 Nov 2022 06:04:27 -0700 (PDT) Received: from jinankjain-dranzer.zrrkmle5drku1h0apvxbr2u2ee.ix.internal.cloudapp.net (unknown [20.188.121.5]) by linux.microsoft.com (Postfix) with ESMTPSA id B5FF9205DA32; Thu, 3 Nov 2022 06:04:22 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com B5FF9205DA32 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1667480666; bh=Jr/pihgkFRoNlRyuUdMyy9zaVqebY39GQd4GO2z7V9s=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rItutFLvdCWFx5U/Xhv3Yl3DpTiKKT6gdMC8oW/vPZURt1kD6Z7S9pIbs2DQoVpND gNAVHk0Kgw0YYLc2DFUyc/VkLM9COHThVCSV8SRlSo8ojyIByt5YdfiqUZgDRRA/jT fWUpmjkAcAfqXq3TV3nJdZowQbhzVLRQq2S9WK08= From: Jinank Jain To: jinankjain@microsoft.com Cc: kys@microsoft.com, haiyangz@microsoft.com, wei.liu@kernel.org, decui@microsoft.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, arnd@arndb.de, peterz@infradead.org, jpoimboe@kernel.org, jinankjain@linux.microsoft.com, seanjc@google.com, kirill.shutemov@linux.intel.com, ak@linux.intel.com, sathyanarayanan.kuppuswamy@linux.intel.com, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, anrayabh@linux.microsoft.com, mikelley@microsoft.com Subject: [PATCH v3 3/5] x86/hyperv: Add an interface to do nested hypercalls Date: Thu, 3 Nov 2022 13:04:05 +0000 Message-Id: <10fdfe01578e691e4815e3427e09e64b45c4af58.1667480257.git.jinankjain@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-19.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_MED, SPF_HELO_PASS,SPF_PASS,USER_IN_DEF_DKIM_WL,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748480467289289687?= X-GMAIL-MSGID: =?utf-8?q?1748480467289289687?= According to TLFS, in order to communicate to L0 hypervisor there needs to be an additional bit set in the control register. This communication is required to perform priviledged instructions which can only be performed by L0 hypervisor. An example of that could be setting up the VMBus infrastructure. Signed-off-by: Jinank Jain Reviewed-by: --- arch/x86/include/asm/hyperv-tlfs.h | 3 ++- arch/x86/include/asm/mshyperv.h | 42 +++++++++++++++++++++++++++--- include/asm-generic/hyperv-tlfs.h | 1 + 3 files changed, 41 insertions(+), 5 deletions(-) diff --git a/arch/x86/include/asm/hyperv-tlfs.h b/arch/x86/include/asm/hyperv-tlfs.h index 0319091e2019..fd066226f12b 100644 --- a/arch/x86/include/asm/hyperv-tlfs.h +++ b/arch/x86/include/asm/hyperv-tlfs.h @@ -380,7 +380,8 @@ struct hv_nested_enlightenments_control { __u32 reserved:31; } features; struct { - __u32 reserved; + __u32 inter_partition_comm:1; + __u32 reserved:31; } hypercallControls; } __packed; diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h index b0f16d06a0c5..32f6bed68e88 100644 --- a/arch/x86/include/asm/mshyperv.h +++ b/arch/x86/include/asm/mshyperv.h @@ -76,10 +76,16 @@ static inline u64 hv_do_hypercall(u64 control, void *input, void *output) return hv_status; } +/* Hypercall to the L0 hypervisor */ +static inline u64 hv_do_nested_hypercall(u64 control, void *input, void *output) +{ + return hv_do_hypercall(control | HV_HYPERCALL_NESTED, input, output); +} + /* Fast hypercall with 8 bytes of input and no output */ -static inline u64 hv_do_fast_hypercall8(u16 code, u64 input1) +static inline u64 _hv_do_fast_hypercall8(u64 control, u16 code, u64 input1) { - u64 hv_status, control = (u64)code | HV_HYPERCALL_FAST_BIT; + u64 hv_status; #ifdef CONFIG_X86_64 { @@ -107,10 +113,24 @@ static inline u64 hv_do_fast_hypercall8(u16 code, u64 input1) return hv_status; } +static inline u64 hv_do_fast_hypercall8(u16 code, u64 input1) +{ + u64 control = (u64)code | HV_HYPERCALL_FAST_BIT; + + return _hv_do_fast_hypercall8(control, code, input1); +} + +static inline u64 hv_do_fast_nested_hypercall8(u16 code, u64 input1) +{ + u64 control = (u64)code | HV_HYPERCALL_FAST_BIT | HV_HYPERCALL_NESTED; + + return _hv_do_fast_hypercall8(control, code, input1); +} + /* Fast hypercall with 16 bytes of input */ -static inline u64 hv_do_fast_hypercall16(u16 code, u64 input1, u64 input2) +static inline u64 _hv_do_fast_hypercall16(u64 control, u16 code, u64 input1, u64 input2) { - u64 hv_status, control = (u64)code | HV_HYPERCALL_FAST_BIT; + u64 hv_status; #ifdef CONFIG_X86_64 { @@ -141,6 +161,20 @@ static inline u64 hv_do_fast_hypercall16(u16 code, u64 input1, u64 input2) return hv_status; } +static inline u64 hv_do_fast_hypercall16(u16 code, u64 input1, u64 input2) +{ + u64 control = (u64)code | HV_HYPERCALL_FAST_BIT; + + return _hv_do_fast_hypercall16(control, code, input1, input2); +} + +static inline u64 hv_do_fast_nested_hypercall16(u16 code, u64 input1, u64 input2) +{ + u64 control = (u64)code | HV_HYPERCALL_FAST_BIT | HV_HYPERCALL_NESTED; + + return _hv_do_fast_hypercall16(control, code, input1, input2); +} + extern struct hv_vp_assist_page **hv_vp_assist_page; static inline struct hv_vp_assist_page *hv_get_vp_assist_page(unsigned int cpu) diff --git a/include/asm-generic/hyperv-tlfs.h b/include/asm-generic/hyperv-tlfs.h index fdce7a4cfc6f..3840958201cd 100644 --- a/include/asm-generic/hyperv-tlfs.h +++ b/include/asm-generic/hyperv-tlfs.h @@ -185,6 +185,7 @@ enum HV_GENERIC_SET_FORMAT { #define HV_HYPERCALL_VARHEAD_OFFSET 17 #define HV_HYPERCALL_VARHEAD_MASK GENMASK_ULL(26, 17) #define HV_HYPERCALL_RSVD0_MASK GENMASK_ULL(31, 27) +#define HV_HYPERCALL_NESTED BIT_ULL(31) #define HV_HYPERCALL_REP_COMP_OFFSET 32 #define HV_HYPERCALL_REP_COMP_1 BIT_ULL(32) #define HV_HYPERCALL_REP_COMP_MASK GENMASK_ULL(43, 32) From patchwork Thu Nov 3 13:04:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinank Jain X-Patchwork-Id: 14865 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp520323wru; Thu, 3 Nov 2022 06:10:18 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6jXNkBkbBlMVm36uTL+NTFU4SiuBAmAhN47Nf4Jtqr2QcmdMzlF+40l/QK9Z/LqT4br2BC X-Received: by 2002:a05:6402:1497:b0:462:c929:7e89 with SMTP id e23-20020a056402149700b00462c9297e89mr30754078edv.149.1667481018140; Thu, 03 Nov 2022 06:10:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667481018; cv=none; d=google.com; s=arc-20160816; b=Lip9ZFtY/PIEEJwi5ZmPLJx0T5l+I2QXLARNsmbbYWCA9o45p0AHclHSf3TCKTc6bc RemQCFnQ92TP1Hm2PDH2VYFB1F5Yg+Kp7JIu3pjb+ENv1EOMrzUg+RbCtd3I3LTWWXJQ 7KPpblltthlK9Ow1QNXOWE9hfDiJANuc2fn46imNhAsKpKO0aqwx7HuJwxJhhzYVjSR0 p3uDFX15D0yeYI9VtdbLTczrTEHFSDeTSzmkK6dnvi6V4fec81zrvsZcg0c6H//7LwES +JL1pMp03o6A/NsSP3BdTLVyYoqt/wN90MK+XpvU4pq7aF5SjXZrz/9gBU5ceIotFYXb OGPA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:dkim-filter; bh=wRg2RasbNcCTHIdfOjvFwZNFeGL2yHGK5ZAQjokBqBw=; b=wFcrqiJrED2+qELn/LhWRppiaJtUShw0kCehtzWZXtukmVlxC8YFCqgLbCV8Yu8FGL cfJ5dIRdnHfV+/EYsLf6iMz0rAx5DHYpMHH2pk4+/CasWaqFwSBvYMQFNXnKTRPGnpxo 63jHL98xGsw0dqYgkL3COMbQhgzGICn/mHOIsKwTbchlEuh5cVvzsSr5DlggO4r24lwN WfThmG/cduZdog+Dw0tKMOizcGEDVZ1JcMRAIY/7ISHZ4z0yZaYqg9va01mrr1W3VMQ+ lxPSLyGQSrZKPIeqY9w20Sk4nnrKTYDs2T0eIa3TvntfbTsPR8W48NZb7mrF8vSvt473 HTGQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=MqeEQJyA; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id i3-20020a05640242c300b0045c9904fdafsi1584270edc.74.2022.11.03.06.09.52; Thu, 03 Nov 2022 06:10:18 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=MqeEQJyA; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231459AbiKCNGI (ORCPT + 99 others); Thu, 3 Nov 2022 09:06:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43286 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229567AbiKCNFS (ORCPT ); Thu, 3 Nov 2022 09:05:18 -0400 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 96E1E167D0; Thu, 3 Nov 2022 06:04:31 -0700 (PDT) Received: from jinankjain-dranzer.zrrkmle5drku1h0apvxbr2u2ee.ix.internal.cloudapp.net (unknown [20.188.121.5]) by linux.microsoft.com (Postfix) with ESMTPSA id 47398205DA33; Thu, 3 Nov 2022 06:04:27 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 47398205DA33 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1667480671; bh=wRg2RasbNcCTHIdfOjvFwZNFeGL2yHGK5ZAQjokBqBw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=MqeEQJyAJF2j4RKPD3DLL0LoLiuwcCtGKPFBMGca/gLfUN24W3FY7E2Fn7FHwaXNF 9/H3lGZsztOaMxM/fjl2kboAqRCipoh90uzZr5fQWdo2dY26ZD5xg06e3ZF80lnjXc MjlicZ0qlQlPTBXEAehJmkBe1CCaxdIWo+lnEWI8= From: Jinank Jain To: jinankjain@microsoft.com Cc: kys@microsoft.com, haiyangz@microsoft.com, wei.liu@kernel.org, decui@microsoft.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, arnd@arndb.de, peterz@infradead.org, jpoimboe@kernel.org, jinankjain@linux.microsoft.com, seanjc@google.com, kirill.shutemov@linux.intel.com, ak@linux.intel.com, sathyanarayanan.kuppuswamy@linux.intel.com, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, anrayabh@linux.microsoft.com, mikelley@microsoft.com Subject: [PATCH v3 4/5] Drivers: hv: Enable vmbus driver for nested root partition Date: Thu, 3 Nov 2022 13:04:06 +0000 Message-Id: <9164389f797b6102dd3b3cd1c9d87656464b4bcf.1667480257.git.jinankjain@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-19.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_MED, SPF_HELO_PASS,SPF_PASS,USER_IN_DEF_DKIM_WL,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748480576187198053?= X-GMAIL-MSGID: =?utf-8?q?1748480576187198053?= Currently VMBus driver is not initialized for root partition but we need to enable the VMBus driver for nested root partition. This is required, so that L2 root can use the VMBus devices. Signed-off-by: Jinank Jain --- drivers/hv/vmbus_drv.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c index 8b2e413bf19c..2f0cf75e811b 100644 --- a/drivers/hv/vmbus_drv.c +++ b/drivers/hv/vmbus_drv.c @@ -2723,7 +2723,7 @@ static int __init hv_acpi_init(void) if (!hv_is_hyperv_initialized()) return -ENODEV; - if (hv_root_partition) + if (hv_root_partition && !hv_nested) return 0; /* From patchwork Thu Nov 3 13:04:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinank Jain X-Patchwork-Id: 14860 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp518944wru; Thu, 3 Nov 2022 06:08:30 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7ZLfhskwEsVEUKVqLI/9BFtFpHMnxhHbFcsYFuPNcFActt+nlua6BQQTsNux1LxVn82B2M X-Received: by 2002:a17:902:ebce:b0:186:9905:11bf with SMTP id p14-20020a170902ebce00b00186990511bfmr30803189plg.110.1667480909844; Thu, 03 Nov 2022 06:08:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667480909; cv=none; d=google.com; s=arc-20160816; b=qH/Rjgeo+Aw84rlxTwREeFOlKMw/6RFgWqfRGsjeNr9wZ+x0/Fq0zZa2LVvY4sdwQm 0LYJl3yzJvQW9LwHDCbYP2dfbmsk7gr6CIeEAnxh2YdbVbuz364H94yE5BTDvEW8lS/C 76J/eDFmQk7a9keueUD7QMoGc3IlZZJPrW9kdaygmuISCLR92F0LnwSFvFpFAAwYdJ/e nbTEqQoUVmCZIIOA8RQQIxwyndGUt0wm4WiJDsz308qrFXjWTivCAykEecpbduB2klpn 9AXeFdNrP09kkijRi7IrFs/ewILlp34U2VLQ+LpZGZvYafSVLcKeoztnpMj0htKy++2F PXGw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:dkim-filter; bh=Jgq8nB4XqD5KLtaoujuXvIgLdqXF2Xi2KbIzYn1bdsU=; b=woBPgNLgO2NWYIX9MmrbqmEWAhCcNUYvOBch2XrXg0Z7wrkqCruSFpf+4M+JgvH1/I 31hMIqiOdj2aNn+rL2L8c0020ABRAQ4l/WoWzwdYKdrzBr1DMpLBxll5A7MLn3GaXNBc +Jm8CkKU9wDCbyz1MzB7ZPeZJpocaSFTu6blQoQns3gkn/T4IGeC97yq9zbUOirkHoqi RVLQ6jot46uIxE/gEE7rzlfM4AtOkBASY/HQLImKjY0NuO3aSdNXgX/hr8042QyCMKK/ jy5GKtiWn3WY31nL8uErRSdRndpELLtxPac9aMsTmcn2kCja/BnY8xqVpF45uar08cJw AsoQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=H8W3vobo; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id nn6-20020a17090b38c600b00203336ddd4asi1322058pjb.56.2022.11.03.06.08.15; Thu, 03 Nov 2022 06:08:29 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=H8W3vobo; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231682AbiKCNGN (ORCPT + 99 others); Thu, 3 Nov 2022 09:06:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42250 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231174AbiKCNFU (ORCPT ); Thu, 3 Nov 2022 09:05:20 -0400 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 2443118B2E; Thu, 3 Nov 2022 06:04:36 -0700 (PDT) Received: from jinankjain-dranzer.zrrkmle5drku1h0apvxbr2u2ee.ix.internal.cloudapp.net (unknown [20.188.121.5]) by linux.microsoft.com (Postfix) with ESMTPSA id CC4CC20C28BD; Thu, 3 Nov 2022 06:04:31 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com CC4CC20C28BD DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1667480676; bh=Jgq8nB4XqD5KLtaoujuXvIgLdqXF2Xi2KbIzYn1bdsU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=H8W3voboWecpdhn4XVxcH8lzgnCSoI5XimYpbrwR5lla119BNM+QJsxNtN+r6xqJZ +vJSbnBKfagL6TPDfsbOenrdT1h809b0eXILw7z5nUhaniNk3HPQPDIUeSbZTeAo6+ QfFhFWz1xNB4XZ063QjrLJIZjq75YrdkrXzty1uI= From: Jinank Jain To: jinankjain@microsoft.com Cc: kys@microsoft.com, haiyangz@microsoft.com, wei.liu@kernel.org, decui@microsoft.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, arnd@arndb.de, peterz@infradead.org, jpoimboe@kernel.org, jinankjain@linux.microsoft.com, seanjc@google.com, kirill.shutemov@linux.intel.com, ak@linux.intel.com, sathyanarayanan.kuppuswamy@linux.intel.com, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, anrayabh@linux.microsoft.com, mikelley@microsoft.com Subject: [PATCH v3 5/5] x86/hyperv: Change interrupt vector for nested root partition Date: Thu, 3 Nov 2022 13:04:07 +0000 Message-Id: <049589c0d5dec0d286f3cab4051594fe8c50808d.1667480257.git.jinankjain@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-19.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_MED, SPF_HELO_PASS,SPF_PASS,USER_IN_DEF_DKIM_WL,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748480462854405757?= X-GMAIL-MSGID: =?utf-8?q?1748480462854405757?= Traditionally we have been using the HYPERVISOR_CALLBACK_VECTOR to relay the VMBus interrupt. But this does not work in case of nested hypervisor. Microsoft Hypervisor reserves 0x31 to 0x34 as the interrupt vector range for VMBus and thus we have to use one of the vectors from that range and setup the IDT accordingly. Signed-off-by: Jinank Jain --- arch/x86/include/asm/idtentry.h | 2 ++ arch/x86/include/asm/irq_vectors.h | 6 ++++++ arch/x86/kernel/cpu/mshyperv.c | 15 +++++++++++++++ arch/x86/kernel/idt.c | 9 +++++++++ drivers/hv/vmbus_drv.c | 3 ++- 5 files changed, 34 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/idtentry.h b/arch/x86/include/asm/idtentry.h index 72184b0b2219..c0648e3e4d4a 100644 --- a/arch/x86/include/asm/idtentry.h +++ b/arch/x86/include/asm/idtentry.h @@ -686,6 +686,8 @@ DECLARE_IDTENTRY_SYSVEC(POSTED_INTR_NESTED_VECTOR, sysvec_kvm_posted_intr_nested DECLARE_IDTENTRY_SYSVEC(HYPERVISOR_CALLBACK_VECTOR, sysvec_hyperv_callback); DECLARE_IDTENTRY_SYSVEC(HYPERV_REENLIGHTENMENT_VECTOR, sysvec_hyperv_reenlightenment); DECLARE_IDTENTRY_SYSVEC(HYPERV_STIMER0_VECTOR, sysvec_hyperv_stimer0); +DECLARE_IDTENTRY_SYSVEC(HYPERV_INTR_NESTED_VMBUS_VECTOR, + sysvec_hyperv_nested_vmbus_intr); #endif #if IS_ENABLED(CONFIG_ACRN_GUEST) diff --git a/arch/x86/include/asm/irq_vectors.h b/arch/x86/include/asm/irq_vectors.h index 43dcb9284208..729d19eab7f5 100644 --- a/arch/x86/include/asm/irq_vectors.h +++ b/arch/x86/include/asm/irq_vectors.h @@ -102,6 +102,12 @@ #if IS_ENABLED(CONFIG_HYPERV) #define HYPERV_REENLIGHTENMENT_VECTOR 0xee #define HYPERV_STIMER0_VECTOR 0xed +/* + * FIXME: Change this, once Microsoft Hypervisor changes its assumption + * around VMBus interrupt vector allocation for nested root partition. + * Or provides a better interface to detect this instead of hardcoding. + */ +#define HYPERV_INTR_NESTED_VMBUS_VECTOR 0x31 #endif #define LOCAL_TIMER_VECTOR 0xec diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c index 9a4204139490..703642bce423 100644 --- a/arch/x86/kernel/cpu/mshyperv.c +++ b/arch/x86/kernel/cpu/mshyperv.c @@ -61,6 +61,21 @@ DEFINE_IDTENTRY_SYSVEC(sysvec_hyperv_callback) set_irq_regs(old_regs); } +DEFINE_IDTENTRY_SYSVEC(sysvec_hyperv_nested_vmbus_intr) +{ + struct pt_regs *old_regs = set_irq_regs(regs); + + inc_irq_stat(irq_hv_callback_count); + + if (vmbus_handler) + vmbus_handler(); + + if (ms_hyperv.hints & HV_DEPRECATING_AEOI_RECOMMENDED) + ack_APIC_irq(); + + set_irq_regs(old_regs); +} + void hv_setup_vmbus_handler(void (*handler)(void)) { vmbus_handler = handler; diff --git a/arch/x86/kernel/idt.c b/arch/x86/kernel/idt.c index a58c6bc1cd68..ace648856a0b 100644 --- a/arch/x86/kernel/idt.c +++ b/arch/x86/kernel/idt.c @@ -160,6 +160,15 @@ static const __initconst struct idt_data apic_idts[] = { # endif INTG(SPURIOUS_APIC_VECTOR, asm_sysvec_spurious_apic_interrupt), INTG(ERROR_APIC_VECTOR, asm_sysvec_error_interrupt), +#ifdef CONFIG_HYPERV + /* + * This is a hack because we cannot install this interrupt handler via alloc_intr_gate + * as it does not allow interrupt vector less than FIRST_SYSTEM_VECTORS. And hyperv + * does not want anything other than 0x31-0x34 as the interrupt vector for vmbus + * interrupt in case of nested setup. + */ + INTG(HYPERV_INTR_NESTED_VMBUS_VECTOR, asm_sysvec_hyperv_nested_vmbus_intr), +#endif #endif }; diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c index 2f0cf75e811b..e6fb77fb44b9 100644 --- a/drivers/hv/vmbus_drv.c +++ b/drivers/hv/vmbus_drv.c @@ -2746,7 +2746,8 @@ static int __init hv_acpi_init(void) * normal Linux IRQ mechanism is not used in this case. */ #ifdef HYPERVISOR_CALLBACK_VECTOR - vmbus_interrupt = HYPERVISOR_CALLBACK_VECTOR; + vmbus_interrupt = hv_nested ? HYPERV_INTR_NESTED_VMBUS_VECTOR : + HYPERVISOR_CALLBACK_VECTOR; vmbus_irq = -1; #endif