From patchwork Mon Dec 19 22:58:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 3124 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:e747:0:0:0:0:0 with SMTP id c7csp2660381wrn; Mon, 19 Dec 2022 15:01:23 -0800 (PST) X-Google-Smtp-Source: AA0mqf4gkadsfftQH77RcSIxlDQe8UwslGGiHUGt4GbqCMtOUspiGnVe9tV0cmwHWPdODlKc4iGv X-Received: by 2002:a17:902:7d85:b0:18f:a5b6:54f9 with SMTP id a5-20020a1709027d8500b0018fa5b654f9mr31875279plm.11.1671490882852; Mon, 19 Dec 2022 15:01:22 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1671490882; cv=none; d=google.com; s=arc-20160816; b=bt4MAyVRO5uAPG24qpEaIb+tPLL1KBMjy4pub5uwIPnobybzzAHvKOdaxUeelJpoHT 5YY0qDr9fFaWa3y6M9DfhvhR3kTQsrRRjQiJq1NeYDDiwCaDZ/9nKHis4GpneoGUioyt seSUQmM+DtshwNI9VqFAIPONBK2eqtIWPoLmkPPfs0XmIt5JfQF0FE095Wd950SjcvTA FjavBEqcrbMzDt+Kyd+587Fs8dBavKeVTAlyGoNWrRjHlsu6eUwOlDuBuagsu1uW+Tb9 2G19rLUp2XXkAqfvkVqOnmFH58QkKN5mKWeKSCN8yID2ME5sAzavLShROMBCuZxnmPtp XWTA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=AhED40djqVFMvVEfys5HxwJPt3BvUKftjBxacBou9RU=; b=nKnEnpr1R3t8kirNhyRz1VI9eZu3tr79sBPYALMbI87d/wiPIkwhKrOIdgKp5sWpF8 dP6bRz6Pzd1Nc1apDT1Cf9uzNuxSyje332B3s5Bua9zjsjul3nZ9eglt0G7DF9VAz9MQ BUVEbHGChmkaxDidj4X0OSfW7udmWz0IQ1Qhhbi4PK9xTLg7ejCUVrcT+LcIuY668zPU oj70u2ZSRaSIBNYEKxFdky1QiG/GIRb34ZGxuEO6CvXU/3T4At2p8UG9OfYX3HJJZm/L as/9dRa0TRjPIm0lk4D1KoqgH5fpZfUOIVrWHORLalEVHQk+M4l+nFVV2TR3h/OcOxm3 QNcw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=o0K4O73U; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id n15-20020a170902d2cf00b0018930cc84c0si13362238plc.574.2022.12.19.15.01.09; Mon, 19 Dec 2022 15:01:22 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=o0K4O73U; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232476AbiLSW7q (ORCPT + 99 others); Mon, 19 Dec 2022 17:59:46 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52908 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232716AbiLSW7j (ORCPT ); Mon, 19 Dec 2022 17:59:39 -0500 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 334BD13CD3; Mon, 19 Dec 2022 14:59:38 -0800 (PST) Received: from pps.filterd (m0279870.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2BJLZO43017813; Mon, 19 Dec 2022 22:59:20 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : mime-version : content-transfer-encoding : content-type; s=qcppdkim1; bh=AhED40djqVFMvVEfys5HxwJPt3BvUKftjBxacBou9RU=; b=o0K4O73UitpOOl3y9Tf7GDVD4PPV4QVgX4/nqTMi6vgRWdFxN44OK8fuB+lrjUN9xHRE TtXOqElg9lVRPoX3s0IhuMDTnR+OPPnmUOaPGmrlUTosESBHcpnajwpAn2qrWk/b2uGk 7sxf9Sxm53D/4Vrxiig6bnGldM8VqLRXwJgqESZE9DXFZum1TPZz3AMJDSEcsXiu44sr EG9WG5Mos/muv/kdrAFqa48o1Q0W+Evw1jgkV5uEdx+QnHrvM5JKZe27SwbiiwJpR2rn gDWsKNgq+xcBSllje3QISf+/QkkD7sEFmEOfdotWf6X6RKBB2Lr0GeC457cZUWVB+14h wQ== Received: from nasanppmta01.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3mh5v25gxe-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 19 Dec 2022 22:59:20 +0000 Received: from nasanex01b.na.qualcomm.com (corens_vlan604_snip.qualcomm.com [10.53.140.1]) by NASANPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 2BJMxJqN005214 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 19 Dec 2022 22:59:19 GMT Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Mon, 19 Dec 2022 14:59:18 -0800 From: Elliot Berman To: Bjorn Andersson CC: Elliot Berman , Murali Nalajala , Trilok Soni , "Srivatsa Vaddagiri" , Carl van Schaik , Prakruthi Deepak Heragu , Arnd Bergmann , Greg Kroah-Hartman , Rob Herring , "Krzysztof Kozlowski" , Jonathan Corbet , Bagas Sanjaya , Catalin Marinas , Will Deacon , Jassi Brar , Sudeep Holla , Mark Rutland , Lorenzo Pieralisi , "Dmitry Baryshkov" , , , , , , Subject: [PATCH v8 00/28] Drivers for gunyah hypervisor Date: Mon, 19 Dec 2022 14:58:21 -0800 Message-ID: <20221219225850.2397345-1-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: WvwhfeK-kqNRtexhV9_eZ7zUAa7Tsg0t X-Proofpoint-GUID: WvwhfeK-kqNRtexhV9_eZ7zUAa7Tsg0t X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-19_01,2022-12-15_02,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 impostorscore=0 suspectscore=0 clxscore=1011 adultscore=0 malwarescore=0 bulkscore=0 mlxlogscore=999 mlxscore=0 spamscore=0 priorityscore=1501 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2212190201 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1752685224095864802?= X-GMAIL-MSGID: =?utf-8?q?1752685224095864802?= Gunyah is a Type-1 hypervisor independent of any high-level OS kernel, and runs in a higher CPU privilege level. It does not depend on any lower-privileged OS kernel/code for its core functionality. This increases its security and can support a much smaller trusted computing base than a Type-2 hypervisor. Gunyah is an open source hypervisor. The source repo is available at https://github.com/quic/gunyah-hypervisor. The diagram below shows the architecture. :: VM A VM B +-----+ +-----+ | +-----+ +-----+ +-----+ | | | | | | | | | | | EL0 | APP | | APP | | | APP | | APP | | APP | | | | | | | | | | | | +-----+ +-----+ | +-----+ +-----+ +-----+ ---------------------|------------------------- +--------------+ | +----------------------+ | | | | | EL1 | Linux Kernel | | |Linux kernel/Other OS | ... | | | | | +--------------+ | +----------------------+ --------hvc/smc------|------hvc/smc------------ +----------------------------------------+ | | EL2 | Gunyah Hypervisor | | | +----------------------------------------+ Gunyah provides these following features. - Threads and Scheduling: The scheduler schedules virtual CPUs (VCPUs) on physical CPUs and enables time-sharing of the CPUs. - Memory Management: Gunyah tracks memory ownership and use of all memory under its control. Memory partitioning between VMs is a fundamental security feature. - Interrupt Virtualization: All interrupts are handled in the hypervisor and routed to the assigned VM. - Inter-VM Communication: There are several different mechanisms provided for communicating between VMs. - Device Virtualization: Para-virtualization of devices is supported using inter-VM communication. Low level system features and devices such as interrupt controllers are supported with emulation where required. This series adds the basic framework for detecting that Linux is running under Gunyah as a virtual machine, communication with the Gunyah Resource Manager, and a virtual machine manager capable of launching virtual machines. Patches 21-28 are presently intended to be submitted separately and are included for initial RFC. These patches introudce "VM function" framework to expose further interfaces to interact with Gunyah Virtual Machines. With all 28 patches, it is possible to create a Gunyah VM supporting virtio. Changes in v8: - Treat VM manager as a library of RM - Add patches 21-28 as RFC to support proxy-scheduled vCPUs and necessary bits to support virtio from Gunyah userspace Changes in v7: https://lore.kernel.org/all/20221121140009.2353512-1-quic_eberman@quicinc.com/ - Refactor to remove gunyah RM bus - Refactor allow multiple RM device instances - Bump UAPI to start at 0x0 - Refactor QCOM SCM's platform hooks to allow CONFIG_QCOM_SCM=Y/CONFIG_GUNYAH=M combinations Changes in v6: https://lore.kernel.org/all/20221026185846.3983888-1-quic_eberman@quicinc.com/ - *Replace gunyah-console with gunyah VM Manager* - Move include/asm-generic/gunyah.h into include/linux/gunyah.h - s/gunyah_msgq/gh_msgq/ - Minor tweaks and documentation tidying based on comments from Jiri, Greg, Arnd, Dmitry, and Bagas. Changes in v5: https://lore.kernel.org/all/20221011000840.289033-1-quic_eberman@quicinc.com/ - Dropped sysfs nodes - Switch from aux bus to Gunyah RM bus for the subdevices - Cleaning up RM console Changes in v4: https://lore.kernel.org/all/20220928195633.2348848-1-quic_eberman@quicinc.com/ - Tidied up documentation throughout based on questions/feedback received - Switched message queue implementation to use mailboxes - Renamed "gunyah_device" as "gunyah_resource" Changes in v3: https://lore.kernel.org/all/20220811214107.1074343-1-quic_eberman@quicinc.com/ - /Maintained/Supported/ in MAINTAINERS - Tidied up documentation throughout based on questions/feedback received - Moved hypercalls into arch/arm64/gunyah/; following hyper-v's implementation - Drop opaque typedefs - Move sysfs nodes under /sys/hypervisor/gunyah/ - Moved Gunyah console driver to drivers/tty/ - Reworked gunyah_device design to drop the Gunyah bus. Changes in v2: https://lore.kernel.org/all/20220801211240.597859-1-quic_eberman@quicinc.com/ - DT bindings clean up - Switch hypercalls to follow SMCCC v1: https://lore.kernel.org/all/20220223233729.1571114-1-quic_eberman@quicinc.com/ Elliot Berman (28): docs: gunyah: Introduce Gunyah Hypervisor dt-bindings: Add binding for gunyah hypervisor gunyah: Common types and error codes for Gunyah hypercalls arm64: smccc: Include alternative-macros.h virt: gunyah: Add hypercalls to identify Gunyah virt: gunyah: Identify hypervisor version mailbox: Allow direct registration to a channel virt: gunyah: msgq: Add hypercalls to send and receive messages mailbox: Add Gunyah message queue mailbox gunyah: rsc_mgr: Add resource manager RPC core gunyah: rsc_mgr: Add VM lifecycle RPC gunyah: vm_mgr: Introduce basic VM Manager gunyah: rsc_mgr: Add RPC for sharing memory gunyah: vm_mgr: Add/remove user memory regions gunyah: vm_mgr: Add ioctls to support basic non-proxy VM boot samples: Add sample userspace Gunyah VM Manager gunyah: rsc_mgr: Add platform ops on mem_lend/mem_reclaim firmware: qcom_scm: Use fixed width src vm bitmap firmware: qcom_scm: Register Gunyah platform ops docs: gunyah: Document Gunyah VM Manager virt: gunyah: Translate gh_rm_hyp_resource into gunyah_resource gunyah: vm_mgr: Add framework to add VM Functions virt: gunyah: Add resource tickets virt: gunyah: Add IO handlers virt: gunyah: Add proxy-scheduled vCPUs virt: gunyah: Add hypercalls for sending doorbell virt: gunyah: Add irqfd interface virt: gunyah: Add ioeventfd .../bindings/firmware/gunyah-hypervisor.yaml | 82 ++ .../userspace-api/ioctl/ioctl-number.rst | 1 + Documentation/virt/gunyah/index.rst | 115 +++ Documentation/virt/gunyah/message-queue.rst | 64 ++ Documentation/virt/gunyah/vm-manager.rst | 187 ++++ Documentation/virt/index.rst | 1 + MAINTAINERS | 13 + arch/arm64/Kbuild | 1 + arch/arm64/gunyah/Makefile | 1 + arch/arm64/gunyah/gunyah_hypercall.c | 157 ++++ arch/arm64/include/asm/gunyah.h | 23 + drivers/firmware/Kconfig | 2 + drivers/firmware/qcom_scm.c | 107 ++- drivers/mailbox/Kconfig | 10 + drivers/mailbox/Makefile | 2 + drivers/mailbox/gunyah-msgq.c | 229 +++++ drivers/mailbox/mailbox.c | 96 ++- drivers/mailbox/omap-mailbox.c | 18 +- drivers/mailbox/pcc.c | 18 +- drivers/misc/fastrpc.c | 6 +- drivers/net/wireless/ath/ath10k/qmi.c | 4 +- drivers/remoteproc/qcom_q6v5_mss.c | 8 +- drivers/soc/qcom/rmtfs_mem.c | 2 +- drivers/virt/Kconfig | 1 + drivers/virt/Makefile | 1 + drivers/virt/gunyah/Kconfig | 73 ++ drivers/virt/gunyah/Makefile | 10 + drivers/virt/gunyah/gunyah.c | 46 + drivers/virt/gunyah/gunyah_ioeventfd.c | 109 +++ drivers/virt/gunyah/gunyah_irqfd.c | 180 ++++ drivers/virt/gunyah/gunyah_platform_hooks.c | 63 ++ drivers/virt/gunyah/gunyah_vcpu.c | 350 ++++++++ drivers/virt/gunyah/rsc_mgr.c | 795 ++++++++++++++++++ drivers/virt/gunyah/rsc_mgr.h | 124 +++ drivers/virt/gunyah/rsc_mgr_rpc.c | 428 ++++++++++ drivers/virt/gunyah/vm_mgr.c | 594 +++++++++++++ drivers/virt/gunyah/vm_mgr.h | 76 ++ drivers/virt/gunyah/vm_mgr_mm.c | 245 ++++++ include/linux/arm-smccc.h | 1 + include/linux/gunyah.h | 186 ++++ include/linux/gunyah_rsc_mgr.h | 136 +++ include/linux/gunyah_vm_mgr.h | 107 +++ include/linux/mailbox_client.h | 1 + include/linux/qcom_scm.h | 2 +- include/uapi/linux/gunyah.h | 114 +++ samples/Kconfig | 10 + samples/Makefile | 1 + samples/gunyah/.gitignore | 2 + samples/gunyah/Makefile | 6 + samples/gunyah/gunyah_vmm.c | 270 ++++++ samples/gunyah/sample_vm.dts | 69 ++ 51 files changed, 5075 insertions(+), 72 deletions(-) create mode 100644 Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml create mode 100644 Documentation/virt/gunyah/index.rst create mode 100644 Documentation/virt/gunyah/message-queue.rst create mode 100644 Documentation/virt/gunyah/vm-manager.rst create mode 100644 arch/arm64/gunyah/Makefile create mode 100644 arch/arm64/gunyah/gunyah_hypercall.c create mode 100644 arch/arm64/include/asm/gunyah.h create mode 100644 drivers/mailbox/gunyah-msgq.c create mode 100644 drivers/virt/gunyah/Kconfig create mode 100644 drivers/virt/gunyah/Makefile create mode 100644 drivers/virt/gunyah/gunyah.c create mode 100644 drivers/virt/gunyah/gunyah_ioeventfd.c create mode 100644 drivers/virt/gunyah/gunyah_irqfd.c create mode 100644 drivers/virt/gunyah/gunyah_platform_hooks.c create mode 100644 drivers/virt/gunyah/gunyah_vcpu.c create mode 100644 drivers/virt/gunyah/rsc_mgr.c create mode 100644 drivers/virt/gunyah/rsc_mgr.h create mode 100644 drivers/virt/gunyah/rsc_mgr_rpc.c create mode 100644 drivers/virt/gunyah/vm_mgr.c create mode 100644 drivers/virt/gunyah/vm_mgr.h create mode 100644 drivers/virt/gunyah/vm_mgr_mm.c create mode 100644 include/linux/gunyah.h create mode 100644 include/linux/gunyah_rsc_mgr.h create mode 100644 include/linux/gunyah_vm_mgr.h create mode 100644 include/uapi/linux/gunyah.h create mode 100644 samples/gunyah/.gitignore create mode 100644 samples/gunyah/Makefile create mode 100644 samples/gunyah/gunyah_vmm.c create mode 100644 samples/gunyah/sample_vm.dts base-commit: 830b3c68c1fb1e9176028d02ef86f3cf76aa2476