Message ID | 20230613172054.3959700-1-quic_eberman@quicinc.com |
---|---|
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp723125vqr; Tue, 13 Jun 2023 10:45:23 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4WAIIh0LevfMs5QXAmq+Dvr3/NFJrMTQTkpKh5CYDSCsfDSaTiDirl0/36uVATAp6Bgzoi X-Received: by 2002:a05:6402:1219:b0:516:7964:97fb with SMTP id c25-20020a056402121900b00516796497fbmr7811674edw.24.1686678322961; Tue, 13 Jun 2023 10:45:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686678322; cv=none; d=google.com; s=arc-20160816; b=NXTDAWpNxcgfdQH4WGOXuRdH+MvD6Q0rcQ6DeAuW5XgfvoyvBOhbO1UNRowfjnXqYi ZM+ZkU1JKPMcAtibDvR2y/fPXEIVl4AgOfHhqdgxEzDhSsoTYi7wmh7HIYDS51VHUnWy Z3R+jPTdTsTNkcI11aXiMK6jGu84iCAR1Wn4n+xVmT6zVEoEBqVlD467eB7U8JVv5OT/ VkjUiM7vc6sCXjw7/iN56laeers5pPwrcEFxncllBjEoFGF+qq8jyxSTZ4i+MlNFP9du CqH0XDAoU/70rcZJlwWaYZ5iK+qZhpWsNvSL99Up5MLuYpR/MfE1cKIR1gfmrnqbpcpO jfJw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=ha2ld2ieQo11ftGRG//cbKSekk0/kXzhoOV7aaQ3Zl8=; b=NPIERepDVGSCdUxewpAGIwywYJ4RnRZInDu+L0CB+2dVg79EKXDGog96eZ1rBiIFuN IvmMEfeOKnz4pSOWeNrKKkDagMkrXgbnWeoAhanhFNHB/ywXrVnXo1JOHKbuhO65c+zg cQJl/E2WSSKoSKzSlylsHfGHX5junOdrZmL+fmYA58cM8i11HU+PVWa4jtjfKmT5Fbbz oxsNc1U6sTRwtRlXJQt2R+TE6n2xswpDwmwdIRvMNYdxzUeAaXf0nfqRAhJm1GFQEz0M yFFYhJOBnLFgt4hA7dWpibW1X0FfnI2yUFw2JRCQ48Ur+sDg0qccahppSe61mPKTg3mQ JNlg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=DKDolsyT; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id a17-20020a1709062b1100b00981df65116asi3158982ejg.1042.2023.06.13.10.44.58; Tue, 13 Jun 2023 10:45:22 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=DKDolsyT; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238669AbjFMRV4 (ORCPT <rfc822;lekhanya01809@gmail.com> + 99 others); Tue, 13 Jun 2023 13:21:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49850 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238368AbjFMRVv (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Tue, 13 Jun 2023 13:21:51 -0400 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ABBC3173C; Tue, 13 Jun 2023 10:21:45 -0700 (PDT) Received: from pps.filterd (m0279868.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35DDBCku004811; Tue, 13 Jun 2023 17:21:09 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : mime-version : content-transfer-encoding : content-type; s=qcppdkim1; bh=ha2ld2ieQo11ftGRG//cbKSekk0/kXzhoOV7aaQ3Zl8=; b=DKDolsyTIfpIUiNTipwujsDOeRLwpmEvUsywKRU2YY64oRYBghPCEb+8b/To9PHPhsm0 HFoJi8ldUlqwJf8ZdUr2oTuBICYIhKX3Y8BYeqXJHiBneHxzji4EJwU+E8Ja2vAV56OS UozLQIx9ppB2B0y6JsSJE+4fg4SsEDEyaIrPU9JbnawuNs/ExKB9amUnAEp3PRa1x0Al 5ZSnj/t6crV544m1rROiU1a9sKRDHDms2WScrwfTBOiCVWwrt0dPI60+7JmWxzrCMxuK ird/Qw9u5K1J6NvDgNOWYyf6Iufvs6QDiIpzPjPMmADY0rylzFG2MrW5XhX/qrfn1e2z 7w== Received: from nasanppmta02.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3r6s3wrna1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 13 Jun 2023 17:21:08 +0000 Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA02.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 35DHL5kK028735 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 13 Jun 2023 17:21:05 GMT Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.42; Tue, 13 Jun 2023 10:21:04 -0700 From: Elliot Berman <quic_eberman@quicinc.com> To: Alex Elder <elder@linaro.org>, Srinivas Kandagatla <srinivas.kandagatla@linaro.org> CC: Elliot Berman <quic_eberman@quicinc.com>, Murali Nalajala <quic_mnalajal@quicinc.com>, Trilok Soni <quic_tsoni@quicinc.com>, "Srivatsa Vaddagiri" <quic_svaddagi@quicinc.com>, Carl van Schaik <quic_cvanscha@quicinc.com>, Prakruthi Deepak Heragu <quic_pheragu@quicinc.com>, Dmitry Baryshkov <dmitry.baryshkov@linaro.org>, Bjorn Andersson <andersson@kernel.org>, Konrad Dybcio <konrad.dybcio@linaro.org>, Arnd Bergmann <arnd@arndb.de>, Greg Kroah-Hartman <gregkh@linuxfoundation.org>, Rob Herring <robh+dt@kernel.org>, "Krzysztof Kozlowski" <krzysztof.kozlowski+dt@linaro.org>, Conor Dooley <conor+dt@kernel.org>, Jonathan Corbet <corbet@lwn.net>, Bagas Sanjaya <bagasdotme@gmail.com>, Will Deacon <will@kernel.org>, Andy Gross <agross@kernel.org>, Catalin Marinas <catalin.marinas@arm.com>, Jassi Brar <jassisinghbrar@gmail.com>, <linux-arm-msm@vger.kernel.org>, <devicetree@vger.kernel.org>, <linux-kernel@vger.kernel.org>, <linux-doc@vger.kernel.org>, <linux-arm-kernel@lists.infradead.org> Subject: [PATCH v14 00/25] Drivers for Gunyah hypervisor Date: Tue, 13 Jun 2023 10:20:28 -0700 Message-ID: <20230613172054.3959700-1-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.40.0 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01b.na.qualcomm.com (10.47.209.197) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: CS_nMFYP-OjOZXf3iBdAcDHjObLtmMn0 X-Proofpoint-GUID: CS_nMFYP-OjOZXf3iBdAcDHjObLtmMn0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-06-13_19,2023-06-12_02,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 bulkscore=0 adultscore=0 mlxscore=0 mlxlogscore=999 clxscore=1011 impostorscore=0 lowpriorityscore=0 suspectscore=0 priorityscore=1501 phishscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306130153 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE, T_SCC_BODY_TEXT_LINE,T_SPF_TEMPERROR,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768610409032352580?= X-GMAIL-MSGID: =?utf-8?q?1768610409032352580?= |
Series |
Drivers for Gunyah hypervisor
|
|
Message
Elliot Berman
June 13, 2023, 5:20 p.m. UTC
Gunyah is a Type-1 hypervisor independent of any high-level OS kernel, and runs in a higher CPU privilege level. It does not depend on any lower-privileged OS kernel/code for its core functionality. This increases its security and can support a much smaller trusted computing base than a Type-2 hypervisor. Gunyah is an open source hypervisor. The source repo is available at https://github.com/quic/gunyah-hypervisor. The diagram below shows the architecture. :: VM A VM B +-----+ +-----+ | +-----+ +-----+ +-----+ | | | | | | | | | | | EL0 | APP | | APP | | | APP | | APP | | APP | | | | | | | | | | | | +-----+ +-----+ | +-----+ +-----+ +-----+ ---------------------|------------------------- +--------------+ | +----------------------+ | | | | | EL1 | Linux Kernel | | |Linux kernel/Other OS | ... | | | | | +--------------+ | +----------------------+ --------hvc/smc------|------hvc/smc------------ +----------------------------------------+ | | EL2 | Gunyah Hypervisor | | | +----------------------------------------+ Gunyah provides these following features. - Threads and Scheduling: The scheduler schedules virtual CPUs (VCPUs) on physical CPUs and enables time-sharing of the CPUs. - Memory Management: Gunyah tracks memory ownership and use of all memory under its control. Memory partitioning between VMs is a fundamental security feature. - Interrupt Virtualization: All interrupts are handled in the hypervisor and routed to the assigned VM. - Inter-VM Communication: There are several different mechanisms provided for communicating between VMs. - Device Virtualization: Para-virtualization of devices is supported using inter-VM communication. Low level system features and devices such as interrupt controllers are supported with emulation where required. This series adds the basic framework for detecting that Linux is running under Gunyah as a virtual machine, communication with the Gunyah Resource Manager, and a sample virtual machine manager capable of launching virtual machines. The series relies on two other patches posted separately: - https://lore.kernel.org/all/20230213181832.3489174-1-quic_eberman@quicinc.com/ - https://lore.kernel.org/all/20230213232537.2040976-2-quic_eberman@quicinc.com/ Changes in v14: - Coding/cosmetic tweaks suggested by Alex - Mark IRQs as wake-up capable Changes in v13: https://lore.kernel.org/all/20230509204801.2824351-1-quic_eberman@quicinc.com/ - Tweaks to message queue driver to address race condition between IRQ and mailbox registration - Allow removal of VM functions by function-specific comparison -- specifically to allow removing irqfd by label only and not requiring original FD to be provided. Changes in v12: https://lore.kernel.org/all/20230424231558.70911-1-quic_eberman@quicinc.com/ - Stylistic/cosmetic tweaks suggested by Alex - Remove patch "virt: gunyah: Identify hypervisor version" and squash the check that we're running under a reasonable Gunyah hypervisor into RM driver - Refactor platform hooks into a separate module per suggestion from Srini - GFP_KERNEL_ACCOUNT and account_locked_vm() for page pinning - enum-ify related constants Changes in v11: https://lore.kernel.org/all/20230304010632.2127470-1-quic_eberman@quicinc.com/ - Rename struct gh_vm_dtb_config:gpa -> guest_phys_addr & overflow checks for this - More docstrings throughout - Make resp_buf and resp_buf_size optional - Replace deprecated idr with xarray - Refconting on misc device instead of RM's platform device - Renaming variables, structs, etc. from gunyah_ -> gh_ - Drop removal of user mem regions - Drop mem_lend functionality; to converge with restricted_memfd later Changes in v10: https://lore.kernel.org/all/20230214211229.3239350-1-quic_eberman@quicinc.com/ - Fix bisectability (end result of series is same, --fixups applied to wrong commits) - Convert GH_ERROR_* and GH_RM_ERROR_* to enums - Correct race condition between allocating/freeing user memory - Replace offsetof with struct_size - Series-wide renaming of functions to be more consistent - VM shutdown & restart support added in vCPU and VM Manager patches - Convert VM function name (string) to type (number) - Convert VM function argument to value (which could be a pointer) to remove memory wastage for arguments - Remove defensive checks of hypervisor correctness - Clean ups to ioeventfd as suggested by Srivatsa Changes in v9: https://lore.kernel.org/all/20230120224627.4053418-1-quic_eberman@quicinc.com/ - Refactor Gunyah API flags to be exposed as feature flags at kernel level - Move mbox client cleanup into gunyah_msgq_remove() - Simplify gh_rm_call return value and response payload - Missing clean-up/error handling/little endian fixes as suggested by Srivatsa and Alex in v8 series Changes in v8: https://lore.kernel.org/all/20221219225850.2397345-1-quic_eberman@quicinc.com/ - Treat VM manager as a library of RM - Add patches 21-28 as RFC to support proxy-scheduled vCPUs and necessary bits to support virtio from Gunyah userspace Changes in v7: https://lore.kernel.org/all/20221121140009.2353512-1-quic_eberman@quicinc.com/ - Refactor to remove gunyah RM bus - Refactor allow multiple RM device instances - Bump UAPI to start at 0x0 - Refactor QCOM SCM's platform hooks to allow CONFIG_QCOM_SCM=Y/CONFIG_GUNYAH=M combinations Changes in v6: https://lore.kernel.org/all/20221026185846.3983888-1-quic_eberman@quicinc.com/ - *Replace gunyah-console with gunyah VM Manager* - Move include/asm-generic/gunyah.h into include/linux/gunyah.h - s/gunyah_msgq/gh_msgq/ - Minor tweaks and documentation tidying based on comments from Jiri, Greg, Arnd, Dmitry, and Bagas. Changes in v5: https://lore.kernel.org/all/20221011000840.289033-1-quic_eberman@quicinc.com/ - Dropped sysfs nodes - Switch from aux bus to Gunyah RM bus for the subdevices - Cleaning up RM console Changes in v4: https://lore.kernel.org/all/20220928195633.2348848-1-quic_eberman@quicinc.com/ - Tidied up documentation throughout based on questions/feedback received - Switched message queue implementation to use mailboxes - Renamed "gunyah_device" as "gunyah_resource" Changes in v3: https://lore.kernel.org/all/20220811214107.1074343-1-quic_eberman@quicinc.com/ - /Maintained/Supported/ in MAINTAINERS - Tidied up documentation throughout based on questions/feedback received - Moved hypercalls into arch/arm64/gunyah/; following hyper-v's implementation - Drop opaque typedefs - Move sysfs nodes under /sys/hypervisor/gunyah/ - Moved Gunyah console driver to drivers/tty/ - Reworked gh_device design to drop the Gunyah bus. Changes in v2: https://lore.kernel.org/all/20220801211240.597859-1-quic_eberman@quicinc.com/ - DT bindings clean up - Switch hypercalls to follow SMCCC v1: https://lore.kernel.org/all/20220223233729.1571114-1-quic_eberman@quicinc.com/ Elliot Berman (25): docs: gunyah: Introduce Gunyah Hypervisor dt-bindings: Add binding for gunyah hypervisor gunyah: Common types and error codes for Gunyah hypercalls virt: gunyah: Add hypercalls to identify Gunyah virt: gunyah: msgq: Add hypercalls to send and receive messages mailbox: Add Gunyah message queue mailbox gunyah: rsc_mgr: Add resource manager RPC core gunyah: rsc_mgr: Add VM lifecycle RPC gunyah: vm_mgr: Introduce basic VM Manager gunyah: rsc_mgr: Add RPC for sharing memory gunyah: vm_mgr: Add/remove user memory regions gunyah: vm_mgr: Add ioctls to support basic non-proxy VM boot samples: Add sample userspace Gunyah VM Manager gunyah: rsc_mgr: Add platform ops on mem_lend/mem_reclaim virt: gunyah: Add Qualcomm Gunyah platform ops docs: gunyah: Document Gunyah VM Manager virt: gunyah: Translate gh_rm_hyp_resource into gunyah_resource gunyah: vm_mgr: Add framework for VM Functions virt: gunyah: Add resource tickets virt: gunyah: Add IO handlers virt: gunyah: Add proxy-scheduled vCPUs virt: gunyah: Add hypercalls for sending doorbell virt: gunyah: Add irqfd interface virt: gunyah: Add ioeventfd MAINTAINERS: Add Gunyah hypervisor drivers section .../bindings/firmware/gunyah-hypervisor.yaml | 82 ++ .../userspace-api/ioctl/ioctl-number.rst | 1 + Documentation/virt/gunyah/index.rst | 114 +++ Documentation/virt/gunyah/message-queue.rst | 71 ++ Documentation/virt/gunyah/vm-manager.rst | 143 +++ Documentation/virt/index.rst | 1 + MAINTAINERS | 13 + arch/arm64/Kbuild | 1 + arch/arm64/gunyah/Makefile | 3 + arch/arm64/gunyah/gunyah_hypercall.c | 142 +++ arch/arm64/include/asm/gunyah.h | 24 + drivers/mailbox/Makefile | 2 + drivers/mailbox/gunyah-msgq.c | 219 +++++ drivers/virt/Kconfig | 2 + drivers/virt/Makefile | 1 + drivers/virt/gunyah/Kconfig | 59 ++ drivers/virt/gunyah/Makefile | 11 + drivers/virt/gunyah/gunyah_ioeventfd.c | 130 +++ drivers/virt/gunyah/gunyah_irqfd.c | 180 ++++ drivers/virt/gunyah/gunyah_platform_hooks.c | 80 ++ drivers/virt/gunyah/gunyah_qcom.c | 153 +++ drivers/virt/gunyah/gunyah_vcpu.c | 470 +++++++++ drivers/virt/gunyah/rsc_mgr.c | 909 ++++++++++++++++++ drivers/virt/gunyah/rsc_mgr.h | 19 + drivers/virt/gunyah/rsc_mgr_rpc.c | 500 ++++++++++ drivers/virt/gunyah/vm_mgr.c | 791 +++++++++++++++ drivers/virt/gunyah/vm_mgr.h | 70 ++ drivers/virt/gunyah/vm_mgr_mm.c | 252 +++++ include/linux/gunyah.h | 207 ++++ include/linux/gunyah_rsc_mgr.h | 162 ++++ include/linux/gunyah_vm_mgr.h | 153 +++ include/uapi/linux/gunyah.h | 293 ++++++ samples/Kconfig | 10 + samples/Makefile | 1 + samples/gunyah/.gitignore | 2 + samples/gunyah/Makefile | 6 + samples/gunyah/gunyah_vmm.c | 266 +++++ samples/gunyah/sample_vm.dts | 68 ++ 38 files changed, 5611 insertions(+) create mode 100644 Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml create mode 100644 Documentation/virt/gunyah/index.rst create mode 100644 Documentation/virt/gunyah/message-queue.rst create mode 100644 Documentation/virt/gunyah/vm-manager.rst create mode 100644 arch/arm64/gunyah/Makefile create mode 100644 arch/arm64/gunyah/gunyah_hypercall.c create mode 100644 arch/arm64/include/asm/gunyah.h create mode 100644 drivers/mailbox/gunyah-msgq.c create mode 100644 drivers/virt/gunyah/Kconfig create mode 100644 drivers/virt/gunyah/Makefile create mode 100644 drivers/virt/gunyah/gunyah_ioeventfd.c create mode 100644 drivers/virt/gunyah/gunyah_irqfd.c create mode 100644 drivers/virt/gunyah/gunyah_platform_hooks.c create mode 100644 drivers/virt/gunyah/gunyah_qcom.c create mode 100644 drivers/virt/gunyah/gunyah_vcpu.c create mode 100644 drivers/virt/gunyah/rsc_mgr.c create mode 100644 drivers/virt/gunyah/rsc_mgr.h create mode 100644 drivers/virt/gunyah/rsc_mgr_rpc.c create mode 100644 drivers/virt/gunyah/vm_mgr.c create mode 100644 drivers/virt/gunyah/vm_mgr.h create mode 100644 drivers/virt/gunyah/vm_mgr_mm.c create mode 100644 include/linux/gunyah.h create mode 100644 include/linux/gunyah_rsc_mgr.h create mode 100644 include/linux/gunyah_vm_mgr.h create mode 100644 include/uapi/linux/gunyah.h create mode 100644 samples/gunyah/.gitignore create mode 100644 samples/gunyah/Makefile create mode 100644 samples/gunyah/gunyah_vmm.c create mode 100644 samples/gunyah/sample_vm.dts base-commit: 858fd168a95c5b9669aac8db6c14a9aeab446375
Comments
On 6/13/23 12:20 PM, Elliot Berman wrote: > Gunyah is a Type-1 hypervisor independent of any > high-level OS kernel, and runs in a higher CPU privilege level. It does > not depend on any lower-privileged OS kernel/code for its core > functionality. This increases its security and can support a much smaller > trusted computing base than a Type-2 hypervisor. I believe I have marked "Reviewed-by" on this entire series now. If I'm mistaken about that, or if you make a change that you think warrants me taking another look, please call that to my attention. Thanks for being very responsive to my review comments. My greatest hope is that we've prevented some bugs from getting accepted, but I also hope you have found my feedback helpful. -Alex > Gunyah is an open source hypervisor. The source repo is available at > https://github.com/quic/gunyah-hypervisor. > > The diagram below shows the architecture. > > :: > > VM A VM B > +-----+ +-----+ | +-----+ +-----+ +-----+ > | | | | | | | | | | | > EL0 | APP | | APP | | | APP | | APP | | APP | > | | | | | | | | | | | > +-----+ +-----+ | +-----+ +-----+ +-----+ > ---------------------|------------------------- > +--------------+ | +----------------------+ > | | | | | > EL1 | Linux Kernel | | |Linux kernel/Other OS | ... > | | | | | > +--------------+ | +----------------------+ > --------hvc/smc------|------hvc/smc------------ > +----------------------------------------+ > | | > EL2 | Gunyah Hypervisor | > | | > +----------------------------------------+ > > Gunyah provides these following features. > > - Threads and Scheduling: The scheduler schedules virtual CPUs (VCPUs) on > physical CPUs and enables time-sharing of the CPUs. > - Memory Management: Gunyah tracks memory ownership and use of all memory > under its control. Memory partitioning between VMs is a fundamental > security feature. > - Interrupt Virtualization: All interrupts are handled in the hypervisor > and routed to the assigned VM. > - Inter-VM Communication: There are several different mechanisms provided > for communicating between VMs. > - Device Virtualization: Para-virtualization of devices is supported using > inter-VM communication. Low level system features and devices such as > interrupt controllers are supported with emulation where required. > > This series adds the basic framework for detecting that Linux is running > under Gunyah as a virtual machine, communication with the Gunyah Resource > Manager, and a sample virtual machine manager capable of launching virtual machines. > > The series relies on two other patches posted separately: > - https://lore.kernel.org/all/20230213181832.3489174-1-quic_eberman@quicinc.com/ > - https://lore.kernel.org/all/20230213232537.2040976-2-quic_eberman@quicinc.com/ > > > Changes in v14: > - Coding/cosmetic tweaks suggested by Alex > - Mark IRQs as wake-up capable > > Changes in v13: https://lore.kernel.org/all/20230509204801.2824351-1-quic_eberman@quicinc.com/ > - Tweaks to message queue driver to address race condition between IRQ and mailbox registration > - Allow removal of VM functions by function-specific comparison -- specifically to allow > removing irqfd by label only and not requiring original FD to be provided. > > Changes in v12: https://lore.kernel.org/all/20230424231558.70911-1-quic_eberman@quicinc.com/ > - Stylistic/cosmetic tweaks suggested by Alex > - Remove patch "virt: gunyah: Identify hypervisor version" and squash the > check that we're running under a reasonable Gunyah hypervisor into RM driver > - Refactor platform hooks into a separate module per suggestion from Srini > - GFP_KERNEL_ACCOUNT and account_locked_vm() for page pinning > - enum-ify related constants > > Changes in v11: https://lore.kernel.org/all/20230304010632.2127470-1-quic_eberman@quicinc.com/ > - Rename struct gh_vm_dtb_config:gpa -> guest_phys_addr & overflow checks for this > - More docstrings throughout > - Make resp_buf and resp_buf_size optional > - Replace deprecated idr with xarray > - Refconting on misc device instead of RM's platform device > - Renaming variables, structs, etc. from gunyah_ -> gh_ > - Drop removal of user mem regions > - Drop mem_lend functionality; to converge with restricted_memfd later > > Changes in v10: https://lore.kernel.org/all/20230214211229.3239350-1-quic_eberman@quicinc.com/ > - Fix bisectability (end result of series is same, --fixups applied to wrong commits) > - Convert GH_ERROR_* and GH_RM_ERROR_* to enums > - Correct race condition between allocating/freeing user memory > - Replace offsetof with struct_size > - Series-wide renaming of functions to be more consistent > - VM shutdown & restart support added in vCPU and VM Manager patches > - Convert VM function name (string) to type (number) > - Convert VM function argument to value (which could be a pointer) to remove memory wastage for arguments > - Remove defensive checks of hypervisor correctness > - Clean ups to ioeventfd as suggested by Srivatsa > > Changes in v9: https://lore.kernel.org/all/20230120224627.4053418-1-quic_eberman@quicinc.com/ > - Refactor Gunyah API flags to be exposed as feature flags at kernel level > - Move mbox client cleanup into gunyah_msgq_remove() > - Simplify gh_rm_call return value and response payload > - Missing clean-up/error handling/little endian fixes as suggested by Srivatsa and Alex in v8 series > > Changes in v8: https://lore.kernel.org/all/20221219225850.2397345-1-quic_eberman@quicinc.com/ > - Treat VM manager as a library of RM > - Add patches 21-28 as RFC to support proxy-scheduled vCPUs and necessary bits to support virtio > from Gunyah userspace > > Changes in v7: https://lore.kernel.org/all/20221121140009.2353512-1-quic_eberman@quicinc.com/ > - Refactor to remove gunyah RM bus > - Refactor allow multiple RM device instances > - Bump UAPI to start at 0x0 > - Refactor QCOM SCM's platform hooks to allow CONFIG_QCOM_SCM=Y/CONFIG_GUNYAH=M combinations > > Changes in v6: https://lore.kernel.org/all/20221026185846.3983888-1-quic_eberman@quicinc.com/ > - *Replace gunyah-console with gunyah VM Manager* > - Move include/asm-generic/gunyah.h into include/linux/gunyah.h > - s/gunyah_msgq/gh_msgq/ > - Minor tweaks and documentation tidying based on comments from Jiri, Greg, Arnd, Dmitry, and Bagas. > > Changes in v5: https://lore.kernel.org/all/20221011000840.289033-1-quic_eberman@quicinc.com/ > - Dropped sysfs nodes > - Switch from aux bus to Gunyah RM bus for the subdevices > - Cleaning up RM console > > Changes in v4: https://lore.kernel.org/all/20220928195633.2348848-1-quic_eberman@quicinc.com/ > - Tidied up documentation throughout based on questions/feedback received > - Switched message queue implementation to use mailboxes > - Renamed "gunyah_device" as "gunyah_resource" > > Changes in v3: https://lore.kernel.org/all/20220811214107.1074343-1-quic_eberman@quicinc.com/ > - /Maintained/Supported/ in MAINTAINERS > - Tidied up documentation throughout based on questions/feedback received > - Moved hypercalls into arch/arm64/gunyah/; following hyper-v's implementation > - Drop opaque typedefs > - Move sysfs nodes under /sys/hypervisor/gunyah/ > - Moved Gunyah console driver to drivers/tty/ > - Reworked gh_device design to drop the Gunyah bus. > > Changes in v2: https://lore.kernel.org/all/20220801211240.597859-1-quic_eberman@quicinc.com/ > - DT bindings clean up > - Switch hypercalls to follow SMCCC > > v1: https://lore.kernel.org/all/20220223233729.1571114-1-quic_eberman@quicinc.com/ > > Elliot Berman (25): > docs: gunyah: Introduce Gunyah Hypervisor > dt-bindings: Add binding for gunyah hypervisor > gunyah: Common types and error codes for Gunyah hypercalls > virt: gunyah: Add hypercalls to identify Gunyah > virt: gunyah: msgq: Add hypercalls to send and receive messages > mailbox: Add Gunyah message queue mailbox > gunyah: rsc_mgr: Add resource manager RPC core > gunyah: rsc_mgr: Add VM lifecycle RPC > gunyah: vm_mgr: Introduce basic VM Manager > gunyah: rsc_mgr: Add RPC for sharing memory > gunyah: vm_mgr: Add/remove user memory regions > gunyah: vm_mgr: Add ioctls to support basic non-proxy VM boot > samples: Add sample userspace Gunyah VM Manager > gunyah: rsc_mgr: Add platform ops on mem_lend/mem_reclaim > virt: gunyah: Add Qualcomm Gunyah platform ops > docs: gunyah: Document Gunyah VM Manager > virt: gunyah: Translate gh_rm_hyp_resource into gunyah_resource > gunyah: vm_mgr: Add framework for VM Functions > virt: gunyah: Add resource tickets > virt: gunyah: Add IO handlers > virt: gunyah: Add proxy-scheduled vCPUs > virt: gunyah: Add hypercalls for sending doorbell > virt: gunyah: Add irqfd interface > virt: gunyah: Add ioeventfd > MAINTAINERS: Add Gunyah hypervisor drivers section > > .../bindings/firmware/gunyah-hypervisor.yaml | 82 ++ > .../userspace-api/ioctl/ioctl-number.rst | 1 + > Documentation/virt/gunyah/index.rst | 114 +++ > Documentation/virt/gunyah/message-queue.rst | 71 ++ > Documentation/virt/gunyah/vm-manager.rst | 143 +++ > Documentation/virt/index.rst | 1 + > MAINTAINERS | 13 + > arch/arm64/Kbuild | 1 + > arch/arm64/gunyah/Makefile | 3 + > arch/arm64/gunyah/gunyah_hypercall.c | 142 +++ > arch/arm64/include/asm/gunyah.h | 24 + > drivers/mailbox/Makefile | 2 + > drivers/mailbox/gunyah-msgq.c | 219 +++++ > drivers/virt/Kconfig | 2 + > drivers/virt/Makefile | 1 + > drivers/virt/gunyah/Kconfig | 59 ++ > drivers/virt/gunyah/Makefile | 11 + > drivers/virt/gunyah/gunyah_ioeventfd.c | 130 +++ > drivers/virt/gunyah/gunyah_irqfd.c | 180 ++++ > drivers/virt/gunyah/gunyah_platform_hooks.c | 80 ++ > drivers/virt/gunyah/gunyah_qcom.c | 153 +++ > drivers/virt/gunyah/gunyah_vcpu.c | 470 +++++++++ > drivers/virt/gunyah/rsc_mgr.c | 909 ++++++++++++++++++ > drivers/virt/gunyah/rsc_mgr.h | 19 + > drivers/virt/gunyah/rsc_mgr_rpc.c | 500 ++++++++++ > drivers/virt/gunyah/vm_mgr.c | 791 +++++++++++++++ > drivers/virt/gunyah/vm_mgr.h | 70 ++ > drivers/virt/gunyah/vm_mgr_mm.c | 252 +++++ > include/linux/gunyah.h | 207 ++++ > include/linux/gunyah_rsc_mgr.h | 162 ++++ > include/linux/gunyah_vm_mgr.h | 153 +++ > include/uapi/linux/gunyah.h | 293 ++++++ > samples/Kconfig | 10 + > samples/Makefile | 1 + > samples/gunyah/.gitignore | 2 + > samples/gunyah/Makefile | 6 + > samples/gunyah/gunyah_vmm.c | 266 +++++ > samples/gunyah/sample_vm.dts | 68 ++ > 38 files changed, 5611 insertions(+) > create mode 100644 Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml > create mode 100644 Documentation/virt/gunyah/index.rst > create mode 100644 Documentation/virt/gunyah/message-queue.rst > create mode 100644 Documentation/virt/gunyah/vm-manager.rst > create mode 100644 arch/arm64/gunyah/Makefile > create mode 100644 arch/arm64/gunyah/gunyah_hypercall.c > create mode 100644 arch/arm64/include/asm/gunyah.h > create mode 100644 drivers/mailbox/gunyah-msgq.c > create mode 100644 drivers/virt/gunyah/Kconfig > create mode 100644 drivers/virt/gunyah/Makefile > create mode 100644 drivers/virt/gunyah/gunyah_ioeventfd.c > create mode 100644 drivers/virt/gunyah/gunyah_irqfd.c > create mode 100644 drivers/virt/gunyah/gunyah_platform_hooks.c > create mode 100644 drivers/virt/gunyah/gunyah_qcom.c > create mode 100644 drivers/virt/gunyah/gunyah_vcpu.c > create mode 100644 drivers/virt/gunyah/rsc_mgr.c > create mode 100644 drivers/virt/gunyah/rsc_mgr.h > create mode 100644 drivers/virt/gunyah/rsc_mgr_rpc.c > create mode 100644 drivers/virt/gunyah/vm_mgr.c > create mode 100644 drivers/virt/gunyah/vm_mgr.h > create mode 100644 drivers/virt/gunyah/vm_mgr_mm.c > create mode 100644 include/linux/gunyah.h > create mode 100644 include/linux/gunyah_rsc_mgr.h > create mode 100644 include/linux/gunyah_vm_mgr.h > create mode 100644 include/uapi/linux/gunyah.h > create mode 100644 samples/gunyah/.gitignore > create mode 100644 samples/gunyah/Makefile > create mode 100644 samples/gunyah/gunyah_vmm.c > create mode 100644 samples/gunyah/sample_vm.dts > > > base-commit: 858fd168a95c5b9669aac8db6c14a9aeab446375