From patchwork Sat Dec 16 00:20:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 179684 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:3b04:b0:fb:cd0c:d3e with SMTP id c4csp9665755dys; Fri, 15 Dec 2023 16:23:43 -0800 (PST) X-Google-Smtp-Source: AGHT+IFm4VQ2+h1FGWOk52jbjQgHyt/o7DYgKudoQ4sc2cF9Wq2cYw14Tg/HrbIgBAHuKwlHKufX X-Received: by 2002:a7b:c385:0:b0:40c:24e1:b6bb with SMTP id s5-20020a7bc385000000b0040c24e1b6bbmr6164049wmj.128.1702686223140; Fri, 15 Dec 2023 16:23:43 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702686223; cv=none; d=google.com; s=arc-20160816; b=yp/KhglVpClkPIAtmNxJDqBkXzhnutYfjh6MvcE203Q3Xh86GvggmXY4Be+BZinByX UNB1Od9sv9jPM610fB8j48RU+cn/qXJasHEkLCOtbaMTWDcDJl2rp3QvA6cvOTD6DiDV uMtD0k0ss6S5smDUcb82F1jvQblhpMRssBOJv/Jvp0x/76ilqIKER9RsGgV/M+4VX7UI ALsmygPAVQjYQa5S5wWQ33DyHk1DUAxlkiDFz0alnZ+kq9Qi8p1bocex7QIVfjdCPDk0 SCE7cu5oqa4WFR0q06ID7wqDlkk3ARfvsu6D6V0opumg9b05mYllXfP9/7ovSa9AFbOm +mzA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:list-unsubscribe:list-subscribe:list-id:precedence :subject:date:from:dkim-signature; bh=jJkNpJRL7neQYiQYByTs6pHgdXtbYBZe+qazKy/T5cY=; fh=r3Y2jt68ZosIb7PuELonIxV/JFXOPKBxBTVJNB/Qarw=; b=GBP1t4G923xJp2fj0nTgfjU1z7ugclapWxYS1q4QiooXhvok5v9Co9g30sAKnf6DAJ r/XGGktRE93kYkXFJNA8wtI72Rz97iQdnkhQYf5LkDby/UVBc2DdSvmX+Qxo6A3BQRCx c6SGXrpG4s5rGlYJZSlH+kCjPJLEwoJhXUaSZYlr6H587xjRV2BRhBVMX0fdFyvEFBb3 yv5jqrkQAEUR6FWRYrzgNSqZHzYTmT/UZfi7IcpbjU8W3Lk8s30usW4CGlKjGpTOFN6E 1QOtolidGZKPr3qtkIqfHNe6vchsZsgtx1DLOtAifRxpRrMNejd0vrPUhxutpTbfZXzA cixA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=Qtu6pkZ7; spf=pass (google.com: domain of linux-kernel+bounces-1873-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1873-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id h18-20020a1709063c1200b00a2321c4b2e5si355232ejg.887.2023.12.15.16.23.42 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Dec 2023 16:23:43 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-1873-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=Qtu6pkZ7; spf=pass (google.com: domain of linux-kernel+bounces-1873-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1873-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 67AC81F25584 for ; Sat, 16 Dec 2023 00:23:42 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 534D21119E; Sat, 16 Dec 2023 00:21:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="Qtu6pkZ7" X-Original-To: linux-kernel@vger.kernel.org Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 853E11C38; Sat, 16 Dec 2023 00:21:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=quicinc.com Received: from pps.filterd (m0279872.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BFNTBdt030894; Sat, 16 Dec 2023 00:21:02 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:date:subject:mime-version:content-type :content-transfer-encoding:message-id:references:in-reply-to:to :cc; s=qcppdkim1; bh=jJkNpJRL7neQYiQYByTs6pHgdXtbYBZe+qazKy/T5cY =; b=Qtu6pkZ7K9L02b1XhF6cSfa7SGpz8W4JyLCTM06ODBU9+ifB90RJCKImt99 wTz4gEj55tr84sK1wav/ZnXeljQHI//IGhMsbpcaO0w53qh6EpHQCY3yJu9uxN1T Dfav75HJxS0DT5lMVjKoApbevGmtQLz1DkZ7EwtcZS0PxaRqy7oAsXlK+9J8de9C VU+T0B3hZ7JJhPF6lQnpbKVGErbf7U20Igmw+fDocWKHmGBeEISN5krbwJrgzmvS Cb1nYLm98PJAjpPtylKqEin7NbBy5I00YZVPgau1QGvszH0PPgnT+4ugEukB4c1t FJgp7ukjv4XsdT2gaIqllUN2TiQ== Received: from nasanppmta01.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v0hb02g6d-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:02 +0000 (GMT) Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 3BG0L1p0015403 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:01 GMT Received: from [169.254.0.1] (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.40; Fri, 15 Dec 2023 16:21:00 -0800 From: Elliot Berman Date: Fri, 15 Dec 2023 16:20:42 -0800 Subject: [PATCH RFC v15 01/30] docs: gunyah: Introduce Gunyah Hypervisor Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <20231215-gunyah-v15-1-192a5d872a30@quicinc.com> References: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> In-Reply-To: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> To: Alex Elder , Srinivas Kandagatla , Murali Nalajal , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Philip Derrin , Prakruthi Deepak Heragu , Jonathan Corbet , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Catalin Marinas , Will Deacon , Konrad Dybcio , Bjorn Andersson , Dmitry Baryshkov , "Fuad Tabba" , Sean Christopherson CC: , , , , , Elliot Berman , Bagas Sanjaya X-Mailer: b4 0.13-dev X-ClientProxiedBy: nalasex01a.na.qualcomm.com (10.47.209.196) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: S5I5G5ENmKdxv0-QTcb4uoK6CKOVLNQJ X-Proofpoint-GUID: S5I5G5ENmKdxv0-QTcb4uoK6CKOVLNQJ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 malwarescore=0 priorityscore=1501 impostorscore=0 spamscore=0 mlxlogscore=952 clxscore=1015 lowpriorityscore=0 bulkscore=0 suspectscore=0 adultscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312160000 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785395909080041894 X-GMAIL-MSGID: 1785395909080041894 Gunyah is an open-source Type-1 hypervisor developed by Qualcomm. It does not depend on any lower-privileged OS/kernel code for its core functionality. This increases its security and can support a smaller trusted computing based when compared to Type-2 hypervisors. Add documentation describing the Gunyah hypervisor and the main components of the Gunyah hypervisor which are of interest to Linux virtualization development. Reviewed-by: Bagas Sanjaya Signed-off-by: Elliot Berman --- Documentation/virt/gunyah/index.rst | 121 ++++++++++++++++++++++++++++ Documentation/virt/gunyah/message-queue.rst | 69 ++++++++++++++++ Documentation/virt/index.rst | 1 + 3 files changed, 191 insertions(+) diff --git a/Documentation/virt/gunyah/index.rst b/Documentation/virt/gunyah/index.rst new file mode 100644 index 000000000000..fe3f4192b836 --- /dev/null +++ b/Documentation/virt/gunyah/index.rst @@ -0,0 +1,121 @@ +.. SPDX-License-Identifier: GPL-2.0 + +================= +Gunyah Hypervisor +================= + +.. toctree:: + :maxdepth: 1 + + message-queue + +Gunyah is a Type-1 hypervisor which is independent of any OS kernel, and runs in +a higher CPU privilege level. It does not depend on any lower-privileged +operating system for its core functionality. This increases its security and can +support a much smaller trusted computing base than a Type-2 hypervisor. + +Gunyah is an open source hypervisor. The source repo is available at +https://github.com/quic/gunyah-hypervisor. + +Gunyah provides these following features. + +- Scheduling: + + A scheduler for virtual CPUs (vCPUs) on physical CPUs enables time-sharing + of the CPUs. Gunyah supports two models of scheduling: + + 1. "Behind the back" scheduling in which Gunyah hypervisor schedules vCPUS + on its own. + 2. "Proxy" scheduling in which a delegated VM can donate part of one of its + vCPU slice to another VM's vCPU via a hypercall. + +- Memory Management: + + APIs handling memory, abstracted as objects, limiting direct use of physical + addresses. Memory ownership and usage tracking of all memory under its control. + Memory partitioning between VMs is a fundamental security feature. + +- Interrupt Virtualization: + + Uses CPU hardware interrupt virtualization capabilities. Interrupts are + handled in the hypervisor and routed to the assigned VM. + +- Inter-VM Communication: + + There are several different mechanisms provided for communicating between VMs. + +- Virtual platform: + + Architectural devices such as interrupt controllers and CPU timers are + directly provided by the hypervisor as well as core virtual platform devices + and system APIs such as ARM PSCI. + +- Device Virtualization: + + Para-virtualization of devices is supported using inter-VM communication. + +Architectures supported +======================= +AArch64 with a GIC + +Resources and Capabilities +========================== + +Services/resources provide by the Gunyah hypervisor are described to a +virtual machine by capability IDs. For instance, inter-VM communication is +performed with doorbells and message queues. Gunyah allows access to interact +with that doorbell via the capability ID. These resources are described in Linux +as a struct gunyah_resource. + +High level management of these resources is performed by the resource manager +VM. RM informs a guest VM about resources it can access through either the +device tree or via guest-initiated RPC. + +Gunyah tracks all resources and selectively makes them available to some VMs via +VM-specific capability IDs. + +Resource Manager +================ + +The resource manager (RM) is a privileged application VM supporting the Gunyah +Hypervisor. It provides policy enforcement aspects of the virtualization system. +The resource manager can be treated as an extension of the Hypervisor but is +separated to its own partition to ensure that the hypervisor layer itself +remains small and secure and to maintain a separation of policy and mechanism in +the platform. The resource manager runs at arm64 NS-EL1, similar to other +virtual machines. + +Communication with the resource manager from other virtual machines happens with +message-queue.rst. Details about the specific messages can be found in +drivers/virt/gunyah/rsc_mgr.c + +:: + + +-------+ +--------+ +--------+ + | RM | | VM_A | | VM_B | + +-.-.-.-+ +---.----+ +---.----+ + | | | | + +-.-.-----------.------------.----+ + | | \==========/ | | + | \========================/ | + | Gunyah | + +---------------------------------+ + +The source for the resource manager is available at +https://github.com/quic/gunyah-resource-manager. + +The resource manager provides the following features: + +- VM lifecycle management: allocating a VM, starting VMs, destruction of VMs +- VM access control policy, including memory sharing and lending +- Interrupt routing configuration +- Forwarding of system-level events (e.g. VM shutdown) to owner VM +- Resource (capability) discovery + +Resource manager presently requires the guest virtual machines uses a device +tree to describe its configuration. The resource manager applies an overlay to +the/hypervisor node. This node lets guests know they are running as a Gunyah +guest VM, how to communicate with resource manager, and basic description and +capabilities of this VM. See +Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml for a +description of this node. diff --git a/Documentation/virt/gunyah/message-queue.rst b/Documentation/virt/gunyah/message-queue.rst new file mode 100644 index 000000000000..bd9947240c2a --- /dev/null +++ b/Documentation/virt/gunyah/message-queue.rst @@ -0,0 +1,69 @@ +.. SPDX-License-Identifier: GPL-2.0 + +Message Queues +============== +Message queue is a simple low-capacity IPC channel between two virtual machines. +It is intended for sending small control and configuration messages. Each +message queue is unidirectional, so a full-duplex IPC channel requires a pair of +queues. + +Messages can be up to 240 bytes in length. Longer messages require a further +protocol on top of the message queue messages themselves. For instance, +communication with the resource manager adds a header field for sending longer +messages via multiple message fragments. + +The diagram below shows how message queue works. A typical configuration +involves 2 message queues. Message queue 1 allows VM_A to send messages to VM_B. +Message queue 2 allows VM_B to send messages to VM_A. + +1. VM_A sends a message of up to 240 bytes in length. It raises a hypercall + with the message to inform the hypervisor to add the message to + message queue 1's queue. The hypervisor copies memory into the internal + message queue representation; the memory doesn't need to be shared between + VM_A and VM_B. + +2. Gunyah raises the corresponding interrupt for VM_B (Rx vIRQ) when any of + these happens: + + a. gunyah_msgq_send() has PUSH flag. This is a typical case. + b. Explicility with gunyah_msgq_push command from VM_A. + c. Message queue has reached a threshold depth. Typically, this threshold + depth is the size of the queue (in other words: when queue is full, Rx + vIRQ raised). + +3. VM_B calls gunyah_msgq_recv() and Gunyah copies message to requested buffer. + +4. Gunyah raises the corresponding interrupt for VM_A (Tx vIRQ) when the message + queue falls below a watermark depth. Typically, this is the size of the queue + (in other words: when the queue is no longer full, Tx vIRQ raised). Note the + watermark depth and the threshold depth for the Rx vIRQ are independent + values, although they are both typically the size of the queue. + Coincidentally, this signal is conceptually similar to Clear-to-Send. + +For VM_B to send a message to VM_A, the process is identical, except that +hypercalls reference message queue 2's capability ID. Each message queue has its +own independent vIRQ: two TX message queues will have two vIRQs (and two +capability IDs). + +:: + + +---------------+ +-----------------+ +---------------+ + | VM_A | |Gunyah hypervisor| | VM_B | + | | | | | | + | | | | | | + | | Tx | | | | + | |-------->| | Rx vIRQ | | + |gunyah_msgq_send() | Tx vIRQ |Message queue 1 |-------->|gunyah_msgq_recv() | + | |<------- | | | | + | | | | | | + | Message Queue | | | | Message Queue | + | driver | | | | driver | + | | | | | | + | | | | | | + | | | | Tx | | + | | Rx vIRQ | |<--------| | + |gunyah_msgq_recv() |<--------|Message queue 2 | Tx vIRQ |gunyah_msgq_send() | + | | | |-------->| | + | | | | | | + | | | | | | + +---------------+ +-----------------+ +---------------+ diff --git a/Documentation/virt/index.rst b/Documentation/virt/index.rst index 7fb55ae08598..15869ee059b3 100644 --- a/Documentation/virt/index.rst +++ b/Documentation/virt/index.rst @@ -16,6 +16,7 @@ Virtualization Support coco/sev-guest coco/tdx-guest hyperv/index + gunyah/index .. only:: html and subproject From patchwork Sat Dec 16 00:20:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 179679 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:3b04:b0:fb:cd0c:d3e with SMTP id c4csp9665085dys; Fri, 15 Dec 2023 16:22:01 -0800 (PST) X-Google-Smtp-Source: AGHT+IFgDc/zP6ZOFJSMa/WfKxUqZbYIR1c8eh+lSS4XGtP73/pwNyR3bvfOaR8RwC4IyyNI+VIr X-Received: by 2002:a05:6808:1245:b0:3b9:f69a:4fce with SMTP id o5-20020a056808124500b003b9f69a4fcemr13913011oiv.107.1702686121385; Fri, 15 Dec 2023 16:22:01 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702686121; cv=none; d=google.com; s=arc-20160816; b=Aej2AcUxhLYKrl0P0PwlciGjo5Ub6LkM2pxZWWdkGiAPlEVLVgJkFeMjcAjYz+J1OD RTthsNFVgf6En65SGRj6t/ekV8agl2jTU+fBjZqYTHM5SwInzk5bQhyZgmlImkFxHBI4 r4QctQKlastu7I0JQa05J+MSUsIH3AhXGmqqCPzvg+yz+HXkT5TJ9nhbnq76YWeg19f8 mh3gfVUiKOxkXOBdcqxADjwfqimigQiTY2TCMT0ujMmatcyK0vvFYlyTTkuJzkJkmg0A +l7+PxgWlaYxLgr2gD3oc9t/f/DosrWW5tbvfVsnVvLDbnmf1YUlS9bGIt1cC2x+3AXV BCFw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:list-unsubscribe:list-subscribe:list-id:precedence :subject:date:from:dkim-signature; bh=JkatVcy30T2EwYlk1D954DoSXJ0fv4ntvLv8PCbRndg=; fh=OomeRL1OtByYDlm2s6LAILczQ6Qum2c8fzxriHKmu0I=; b=mqPDdIAcACvUysYKaVknhtTIAqP67ocs8vo+PrG41JBaio0AjeQ9ClVL9bJvOqJNLL 5TZzc6FrJlyAbJb6zGcghy7m2jLTPNqnI9x7PftXVHB9DT76SHxa5JLvg1ZwH9kXUInd cpRY3BtudWzDasmIHIc76iJSsBR/vUVarzsqXvR1Z+b7DMrzNZ36kUrw5lDiWvF6ZKO6 eJGlrQ6wl2dbe7N+ABbNkhxHQM95G+FKw6OA4xFyy7pci9wPKLIB89URYBWjztHwV6Xb tbk9MXkkUopRS6/8DZA+izVydPWJhdR4/sQBCDGl5jdV9kXdbs5v8NjwR/yoZlZyzTm8 ZHRg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=G5esPZz2; spf=pass (google.com: domain of linux-kernel+bounces-1868-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1868-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [147.75.48.161]) by mx.google.com with ESMTPS id g187-20020a636bc4000000b005c66b8a7329si13772535pgc.15.2023.12.15.16.22.00 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Dec 2023 16:22:01 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-1868-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=G5esPZz2; spf=pass (google.com: domain of linux-kernel+bounces-1868-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1868-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 6D013B22BC5 for ; Sat, 16 Dec 2023 00:21:54 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 8364B1C32; Sat, 16 Dec 2023 00:21:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="G5esPZz2" X-Original-To: linux-kernel@vger.kernel.org Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 76703365; Sat, 16 Dec 2023 00:21:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=quicinc.com Received: from pps.filterd (m0279862.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BFMcLZv028677; Sat, 16 Dec 2023 00:21:03 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:date:subject:mime-version:content-type :content-transfer-encoding:message-id:references:in-reply-to:to :cc; s=qcppdkim1; bh=JkatVcy30T2EwYlk1D954DoSXJ0fv4ntvLv8PCbRndg =; b=G5esPZz2dwmuNSA9RSYIuSJJ0y9ml8NEMTrHwF0HbuB1qXMeUN9N0Z8jNON HajkGvTccgALR5x/32EUJOvAbAvL08qdvj2/SeOgxfeGryqGbkZ4xK5E1B+QkLcr 0pvvUbcv1AG0JyW6N1Pj2QFAXi+/scZxiH/Dbv3aWbLMoJmPJDyAgxnpozPm16Tg whtNxcePIOyXv7fRCoGia9FNJsOxSambfASfPHFztcCZQOBFm8qbk37zJW+IHTdL L/vV5/maeU5vJEZXY56zjNB6kwPv4m2Z1yWbUSdGyEjVN6Wikxb5YAMf+VwiV2SD npd4MVbzXrgZQc3c9bk5aY++F/w== Received: from nasanppmta01.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v0w198hp7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:02 +0000 (GMT) Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 3BG0L2fO015448 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:02 GMT Received: from [169.254.0.1] (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.40; Fri, 15 Dec 2023 16:21:01 -0800 From: Elliot Berman Date: Fri, 15 Dec 2023 16:20:43 -0800 Subject: [PATCH RFC v15 02/30] dt-bindings: Add binding for gunyah hypervisor Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <20231215-gunyah-v15-2-192a5d872a30@quicinc.com> References: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> In-Reply-To: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> To: Alex Elder , Srinivas Kandagatla , Murali Nalajal , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Philip Derrin , Prakruthi Deepak Heragu , Jonathan Corbet , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Catalin Marinas , Will Deacon , Konrad Dybcio , Bjorn Andersson , Dmitry Baryshkov , "Fuad Tabba" , Sean Christopherson CC: , , , , , Elliot Berman , Rob Herring X-Mailer: b4 0.13-dev X-ClientProxiedBy: nalasex01a.na.qualcomm.com (10.47.209.196) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: txtWmqSTCRvfZ8_OOLCkDkE_13G7yX7b X-Proofpoint-ORIG-GUID: txtWmqSTCRvfZ8_OOLCkDkE_13G7yX7b X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 mlxlogscore=999 suspectscore=0 mlxscore=0 impostorscore=0 lowpriorityscore=0 spamscore=0 adultscore=0 phishscore=0 clxscore=1015 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312160001 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785395802457491942 X-GMAIL-MSGID: 1785395802457491942 The Gunyah Resource Manager applies a devicetree overlay describing the virtual platform configuration of the guest VM, such as the message queue capability IDs for communicating with the Resource Manager. This information is not otherwise discoverable by a VM: the Gunyah hypervisor core does not provide a direct interface to discover capability IDs nor a way to communicate with RM without having already known the corresponding message queue capability ID. Add the DT bindings that Gunyah adheres for the hypervisor node and message queues. Reviewed-by: Rob Herring Signed-off-by: Elliot Berman --- .../bindings/firmware/gunyah-hypervisor.yaml | 82 ++++++++++++++++++++++ 1 file changed, 82 insertions(+) diff --git a/Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml b/Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml new file mode 100644 index 000000000000..cdeb4885a807 --- /dev/null +++ b/Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml @@ -0,0 +1,82 @@ +# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/firmware/gunyah-hypervisor.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Gunyah Hypervisor + +maintainers: + - Prakruthi Deepak Heragu + - Elliot Berman + +description: |+ + Gunyah virtual machines use this information to determine the capability IDs + of the message queues used to communicate with the Gunyah Resource Manager. + See also: https://github.com/quic/gunyah-resource-manager/blob/develop/src/vm_creation/dto_construct.c + +properties: + compatible: + const: gunyah-hypervisor + + "#address-cells": + description: Number of cells needed to represent 64-bit capability IDs. + const: 2 + + "#size-cells": + description: must be 0, because capability IDs are not memory address + ranges and do not have a size. + const: 0 + +patternProperties: + "^gunyah-resource-mgr(@.*)?": + type: object + description: + Resource Manager node which is required to communicate to Resource + Manager VM using Gunyah Message Queues. + + properties: + compatible: + const: gunyah-resource-manager + + reg: + items: + - description: Gunyah capability ID of the TX message queue + - description: Gunyah capability ID of the RX message queue + + interrupts: + items: + - description: Interrupt for the TX message queue + - description: Interrupt for the RX message queue + + additionalProperties: false + + required: + - compatible + - reg + - interrupts + +additionalProperties: false + +required: + - compatible + - "#address-cells" + - "#size-cells" + +examples: + - | + #include + + hypervisor { + #address-cells = <2>; + #size-cells = <0>; + compatible = "gunyah-hypervisor"; + + gunyah-resource-mgr@0 { + compatible = "gunyah-resource-manager"; + interrupts = , /* TX allowed IRQ */ + ; /* RX requested IRQ */ + reg = <0x00000000 0x00000000>, /* TX capability ID */ + <0x00000000 0x00000001>; /* RX capability ID */ + }; + }; From patchwork Sat Dec 16 00:20:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 179682 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:3b04:b0:fb:cd0c:d3e with SMTP id c4csp9665550dys; Fri, 15 Dec 2023 16:23:16 -0800 (PST) X-Google-Smtp-Source: AGHT+IFx3zOc8q9CvR5wfQJtI8zHLu/uQq8mF4nF565xxICUKQuRr0D1Bg0p1SlHPOwZcf+3fbFN X-Received: by 2002:a17:907:971d:b0:a16:f41c:3714 with SMTP id jg29-20020a170907971d00b00a16f41c3714mr8379554ejc.22.1702686195791; Fri, 15 Dec 2023 16:23:15 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702686195; cv=none; d=google.com; s=arc-20160816; b=ctSaMXth3EGtz+8uFTrho4o4qp4xBloTT/B6WyPp5g9qL9u0FKSP5wToOGb9GvtHKw LdEGrlfmszzYl+Ai+tQvNzE8yscVS9bnrQFyLIHqxooOq7wG7T+V/5v8xw9kFSJOYgoW UgV8MWva1HTSKEu9ZRQHV1mznc0dq4bt725NBBuyNMC2uCZj9lfFvaar8aOH4kHnfwtB 2ocVbmSo8VMW/WFFNmCkpa+SkBw7iEp81QDiYpTUcZDhDuCfb4fOTns1Si3om8OTd2jH /yihYzM9e0yVSiyVlTAAvVelL/prIiTISJEyKuLdllgrjEJcye7O1wSwGpr4meUHhHcI 4uZg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:list-unsubscribe:list-subscribe:list-id:precedence :subject:date:from:dkim-signature; bh=p214wKRdwlawgp2IxBOVfCkEdPuLMQeowShlXHUSbks=; fh=Pv/sMsQia5wmqAOYHIQUkkUFpDKhZjuODj71Ojb2ybw=; b=CYR/nYHT6mlcZJQgBnpVMNMMCDpXaSYxOIDu5mEziR/ZxPYKovfkqoZD3lJvoFKalN xqva5dj+91lEecQ8PItBCRECtkPoSGwckedlo1H9jJL1lFaEiLUMybIQSgkSCUNcMGQh NICvx67udDD/m6KXqU8NeL9MZSPIaYd1gEF2av4VZDWYo3cxn20LxoqRzdGhvCA0zaLl yUqQlh1QkhAYeH2RSWcUfdf1OpAIk3a8gstUJeKELflbaNATfB3P1Rd79pRYHi3BgLbn IWaWvS25IiCEyaWX6F0SpGbG86n4vVibSs2nGuzoeip+moRQji3VAx+gHLXp6+1eJNSN 513Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=hLH7Mx0a; spf=pass (google.com: domain of linux-kernel+bounces-1872-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1872-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id gt9-20020a170906f20900b00a23235f7dc4si263752ejb.220.2023.12.15.16.23.15 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Dec 2023 16:23:15 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-1872-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=hLH7Mx0a; spf=pass (google.com: domain of linux-kernel+bounces-1872-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1872-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 407171F2566F for ; Sat, 16 Dec 2023 00:23:15 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 44B42101D2; Sat, 16 Dec 2023 00:21:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="hLH7Mx0a" X-Original-To: linux-kernel@vger.kernel.org Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ED41615C9; Sat, 16 Dec 2023 00:21:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=quicinc.com Received: from pps.filterd (m0279869.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BFNpxf3006043; Sat, 16 Dec 2023 00:21:04 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:date:subject:mime-version:content-type :content-transfer-encoding:message-id:references:in-reply-to:to :cc; s=qcppdkim1; bh=p214wKRdwlawgp2IxBOVfCkEdPuLMQeowShlXHUSbks =; b=hLH7Mx0a24cTH1hVbY8yHYtBJ9I2uzVV2VsNgehXlY25KorGsvx1KYTwyKk v+Q4+i2bX20gZblTjT0Z1q5fCVO3CDXwDLD5m2iXY+Z7dHfUNhH9ZI34QsRbbmk3 VENAmzT/hFarp/BUGR6KFWfh3ETcrmaYKgIy8ZIF1VJdNNp1he5vx1cg8AbeWy/Q BVVoINWoXfQiINQ+KsTDUcETsrnIHsvpnfcmFZSb4dBWGZ2u/j6f+sWQmmbMRoUZ o/8GCp4GDSbkaYZIu4hcLcETmMYZCQ8L+9GX6zyz84EFKkI0gmNmwy0UDgPXf0BR pL0sulNgsS7Kw867Ga+wC5K1rTQ== Received: from nasanppmta04.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v0pqvshkv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:03 +0000 (GMT) Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA04.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 3BG0L2Hq016442 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:02 GMT Received: from [169.254.0.1] (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.40; Fri, 15 Dec 2023 16:21:01 -0800 From: Elliot Berman Date: Fri, 15 Dec 2023 16:20:44 -0800 Subject: [PATCH RFC v15 03/30] gunyah: Common types and error codes for Gunyah hypercalls Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <20231215-gunyah-v15-3-192a5d872a30@quicinc.com> References: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> In-Reply-To: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> To: Alex Elder , Srinivas Kandagatla , Murali Nalajal , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Philip Derrin , Prakruthi Deepak Heragu , Jonathan Corbet , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Catalin Marinas , Will Deacon , Konrad Dybcio , Bjorn Andersson , Dmitry Baryshkov , "Fuad Tabba" , Sean Christopherson CC: , , , , , Elliot Berman X-Mailer: b4 0.13-dev X-ClientProxiedBy: nalasex01a.na.qualcomm.com (10.47.209.196) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: qe_QxpLEkBMV61k9oiroscfWmud7XwMX X-Proofpoint-ORIG-GUID: qe_QxpLEkBMV61k9oiroscfWmud7XwMX X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 spamscore=0 malwarescore=0 priorityscore=1501 impostorscore=0 bulkscore=0 adultscore=0 suspectscore=0 clxscore=1015 mlxscore=0 mlxlogscore=453 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312160000 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785395880692202945 X-GMAIL-MSGID: 1785395880692202945 Add architecture-independent standard error codes, types, and macros for Gunyah hypercalls. Reviewed-by: Dmitry Baryshkov Reviewed-by: Srinivas Kandagatla Reviewed-by: Alex Elder Signed-off-by: Elliot Berman --- include/linux/gunyah.h | 106 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 106 insertions(+) diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h new file mode 100644 index 000000000000..e1a716d802ee --- /dev/null +++ b/include/linux/gunyah.h @@ -0,0 +1,106 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _LINUX_GUNYAH_H +#define _LINUX_GUNYAH_H + +#include +#include +#include + +/* Matches resource manager's resource types for VM_GET_HYP_RESOURCES RPC */ +enum gunyah_resource_type { + /* clang-format off */ + GUNYAH_RESOURCE_TYPE_BELL_TX = 0, + GUNYAH_RESOURCE_TYPE_BELL_RX = 1, + GUNYAH_RESOURCE_TYPE_MSGQ_TX = 2, + GUNYAH_RESOURCE_TYPE_MSGQ_RX = 3, + GUNYAH_RESOURCE_TYPE_VCPU = 4, + GUNYAH_RESOURCE_TYPE_MEM_EXTENT = 9, + GUNYAH_RESOURCE_TYPE_ADDR_SPACE = 10, + /* clang-format on */ +}; + +struct gunyah_resource { + enum gunyah_resource_type type; + u64 capid; + unsigned int irq; +}; + +/******************************************************************************/ +/* Common arch-independent definitions for Gunyah hypercalls */ +#define GUNYAH_CAPID_INVAL U64_MAX +#define GUNYAH_VMID_ROOT_VM 0xff + +enum gunyah_error { + /* clang-format off */ + GUNYAH_ERROR_OK = 0, + GUNYAH_ERROR_UNIMPLEMENTED = -1, + GUNYAH_ERROR_RETRY = -2, + + GUNYAH_ERROR_ARG_INVAL = 1, + GUNYAH_ERROR_ARG_SIZE = 2, + GUNYAH_ERROR_ARG_ALIGN = 3, + + GUNYAH_ERROR_NOMEM = 10, + + GUNYAH_ERROR_ADDR_OVFL = 20, + GUNYAH_ERROR_ADDR_UNFL = 21, + GUNYAH_ERROR_ADDR_INVAL = 22, + + GUNYAH_ERROR_DENIED = 30, + GUNYAH_ERROR_BUSY = 31, + GUNYAH_ERROR_IDLE = 32, + + GUNYAH_ERROR_IRQ_BOUND = 40, + GUNYAH_ERROR_IRQ_UNBOUND = 41, + + GUNYAH_ERROR_CSPACE_CAP_NULL = 50, + GUNYAH_ERROR_CSPACE_CAP_REVOKED = 51, + GUNYAH_ERROR_CSPACE_WRONG_OBJ_TYPE = 52, + GUNYAH_ERROR_CSPACE_INSUF_RIGHTS = 53, + GUNYAH_ERROR_CSPACE_FULL = 54, + + GUNYAH_ERROR_MSGQUEUE_EMPTY = 60, + GUNYAH_ERROR_MSGQUEUE_FULL = 61, + /* clang-format on */ +}; + +/** + * gunyah_error_remap() - Remap Gunyah hypervisor errors into a Linux error code + * @gunyah_error: Gunyah hypercall return value + */ +static inline int gunyah_error_remap(enum gunyah_error gunyah_error) +{ + switch (gunyah_error) { + case GUNYAH_ERROR_OK: + return 0; + case GUNYAH_ERROR_NOMEM: + return -ENOMEM; + case GUNYAH_ERROR_DENIED: + case GUNYAH_ERROR_CSPACE_CAP_NULL: + case GUNYAH_ERROR_CSPACE_CAP_REVOKED: + case GUNYAH_ERROR_CSPACE_WRONG_OBJ_TYPE: + case GUNYAH_ERROR_CSPACE_INSUF_RIGHTS: + return -EACCES; + case GUNYAH_ERROR_CSPACE_FULL: + case GUNYAH_ERROR_BUSY: + case GUNYAH_ERROR_IDLE: + return -EBUSY; + case GUNYAH_ERROR_IRQ_BOUND: + case GUNYAH_ERROR_IRQ_UNBOUND: + case GUNYAH_ERROR_MSGQUEUE_FULL: + case GUNYAH_ERROR_MSGQUEUE_EMPTY: + return -EIO; + case GUNYAH_ERROR_UNIMPLEMENTED: + return -EOPNOTSUPP; + case GUNYAH_ERROR_RETRY: + return -EAGAIN; + default: + return -EINVAL; + } +} + +#endif From patchwork Sat Dec 16 00:20:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 179680 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:3b04:b0:fb:cd0c:d3e with SMTP id c4csp9665106dys; Fri, 15 Dec 2023 16:22:05 -0800 (PST) X-Google-Smtp-Source: AGHT+IGqal0WxMcpxblHMctk/qIvI97yuwArRE67RLp44LOqd+ga9DHuXqRvljDHmzBNl5dDutnv X-Received: by 2002:a05:6a21:6da1:b0:18f:97c:6170 with SMTP id wl33-20020a056a216da100b0018f097c6170mr16184116pzb.109.1702686125562; Fri, 15 Dec 2023 16:22:05 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702686125; cv=none; d=google.com; s=arc-20160816; b=r3AsblGbyB6tf56BcnS0LEBd6caUlxSL//U22TxF9Wbnu3TFJrwPhJSLv2OA9WLpGt p3oBleIzjBs5aJe2S6TOJDY7ylopdDUaDkhTcS7A9Hzkq+mmm7G0U7Z3As0dQ/8Kw1mg y99SIiRez7uKRhI4e8pdG4REzMUv1OwQ8x6K0LJN9xQTlS8WKDiUqIkHSf5o2ZxVNXrf fUTW43uJ8T2LitreyitNNrWNshyCkWbjTAALP0QhTQCMqxW/UWdzNGCX796glu7mS0Ac dAIbu9lL0d+dwbYH2qoUgbSre8/A8d9hZj1sBr2GwSyMWSQyjjYu2P3gia/Jcd9R3oo7 w15Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:list-unsubscribe:list-subscribe:list-id:precedence :subject:date:from:dkim-signature; bh=u5SQ97skzOqdi3TIaqdelH9vOvqyauVPXkcOFGpPFCg=; fh=Pv/sMsQia5wmqAOYHIQUkkUFpDKhZjuODj71Ojb2ybw=; b=gKQbkDR1J0xNA8yrbPvRiqaGRI4JUVt2cvW0m5kfLE6UYuBOQl4FU3SAiY2Oc8ENYm wVDvo2WBWJlyNK33gNqoOwD4UII6E6HxL6ZmRIRU19Iqb85qPlfW5q9IfQyM5CacToJy VrDJErBVETvcMa9OLNo8jT3utfRksZcy+wu8zFNZbvwFnuAJzGjHIq8O0bp4QghM90hv kguJW13uvQfIIbeHNb4X3+w9D+tdf2aFokF585sc3YQQ8BJ8v1gz54fWuCFoiz1Dlms+ l5fXoOCfGy5J1MJpbQkN+TjfxHHU8JFbxyEeospNrgMEP3oFp+oBmjG1UVNe8jET4aQM bk9g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=gXixN4it; spf=pass (google.com: domain of linux-kernel+bounces-1869-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1869-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id b12-20020a63eb4c000000b005be1ee5be76si13477524pgk.373.2023.12.15.16.22.05 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Dec 2023 16:22:05 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-1869-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=gXixN4it; spf=pass (google.com: domain of linux-kernel+bounces-1869-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1869-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 21F9CB22BFE for ; Sat, 16 Dec 2023 00:21:58 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C5DD820F3; Sat, 16 Dec 2023 00:21:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="gXixN4it" X-Original-To: linux-kernel@vger.kernel.org Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 766B27E; Sat, 16 Dec 2023 00:21:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=quicinc.com Received: from pps.filterd (m0279863.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BFNswqv030328; Sat, 16 Dec 2023 00:21:04 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:date:subject:mime-version:content-type :content-transfer-encoding:message-id:references:in-reply-to:to :cc; s=qcppdkim1; bh=u5SQ97skzOqdi3TIaqdelH9vOvqyauVPXkcOFGpPFCg =; b=gXixN4it+um/2bvor7xj+ZD4KABkMuMLCHFxGaUDffUeB+wrTcoizxiWPcJ wTtiykkmvLtX6/jfJ4TJpoc75n1VtKGsJkfO2zbLCXkDwbRW7n8T1iJfbODgoi7M sG0UfJwurj7Wl7DTsvgZWB2cM3jPIApIzs+s4CoyR+rne0EEuuWvQfc1mMdlATKb /bm5VQYBNUIldvhPgGPEQnigd5Cv3hopSKWD+TzXcSHNf1XB+aIy9NT+fIos+lAn Lu6NnaVXBDhZgQ8c9MlAK6k/+Km9ccekdkgSeMhhTiyP52MzIUPXZ14TOtYYyG1H BgW2dppKeDwSAhsGtMSzFAh570g== Received: from nasanppmta04.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v0up20re4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:03 +0000 (GMT) Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA04.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 3BG0L3OM016469 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:03 GMT Received: from [169.254.0.1] (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.40; Fri, 15 Dec 2023 16:21:02 -0800 From: Elliot Berman Date: Fri, 15 Dec 2023 16:20:45 -0800 Subject: [PATCH RFC v15 04/30] virt: gunyah: Add hypercalls to identify Gunyah Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <20231215-gunyah-v15-4-192a5d872a30@quicinc.com> References: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> In-Reply-To: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> To: Alex Elder , Srinivas Kandagatla , Murali Nalajal , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Philip Derrin , Prakruthi Deepak Heragu , Jonathan Corbet , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Catalin Marinas , Will Deacon , Konrad Dybcio , Bjorn Andersson , Dmitry Baryshkov , "Fuad Tabba" , Sean Christopherson CC: , , , , , Elliot Berman X-Mailer: b4 0.13-dev X-ClientProxiedBy: nalasex01a.na.qualcomm.com (10.47.209.196) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: C1yvTZuhGi5LworAApbOZ9xZkEybxAlU X-Proofpoint-ORIG-GUID: C1yvTZuhGi5LworAApbOZ9xZkEybxAlU X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 bulkscore=0 mlxscore=0 phishscore=0 adultscore=0 suspectscore=0 mlxlogscore=999 impostorscore=0 priorityscore=1501 clxscore=1015 malwarescore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312160001 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785395806529786442 X-GMAIL-MSGID: 1785395806529786442 Add hypercalls to identify when Linux is running a virtual machine under Gunyah. There are two calls to help identify Gunyah: 1. gh_hypercall_get_uid() returns a UID when running under a Gunyah hypervisor. 2. gh_hypercall_hyp_identify() returns build information and a set of feature flags that are supported by Gunyah. Reviewed-by: Srinivas Kandagatla Reviewed-by: Alex Elder Signed-off-by: Elliot Berman --- arch/arm64/Kbuild | 1 + arch/arm64/gunyah/Makefile | 3 ++ arch/arm64/gunyah/gunyah_hypercall.c | 62 ++++++++++++++++++++++++++++++++++++ drivers/virt/Kconfig | 2 ++ drivers/virt/gunyah/Kconfig | 12 +++++++ include/linux/gunyah.h | 38 ++++++++++++++++++++++ 6 files changed, 118 insertions(+) diff --git a/arch/arm64/Kbuild b/arch/arm64/Kbuild index 5bfbf7d79c99..e4847ba0e3c9 100644 --- a/arch/arm64/Kbuild +++ b/arch/arm64/Kbuild @@ -3,6 +3,7 @@ obj-y += kernel/ mm/ net/ obj-$(CONFIG_KVM) += kvm/ obj-$(CONFIG_XEN) += xen/ obj-$(subst m,y,$(CONFIG_HYPERV)) += hyperv/ +obj-$(CONFIG_GUNYAH) += gunyah/ obj-$(CONFIG_CRYPTO) += crypto/ # for cleaning diff --git a/arch/arm64/gunyah/Makefile b/arch/arm64/gunyah/Makefile new file mode 100644 index 000000000000..84f1e38cafb1 --- /dev/null +++ b/arch/arm64/gunyah/Makefile @@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0 + +obj-$(CONFIG_GUNYAH) += gunyah_hypercall.o diff --git a/arch/arm64/gunyah/gunyah_hypercall.c b/arch/arm64/gunyah/gunyah_hypercall.c new file mode 100644 index 000000000000..739d7f9cef9f --- /dev/null +++ b/arch/arm64/gunyah/gunyah_hypercall.c @@ -0,0 +1,62 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include +#include +#include +#include + +/* {c1d58fcd-a453-5fdb-9265-ce36673d5f14} */ +static const uuid_t GUNYAH_UUID = UUID_INIT(0xc1d58fcd, 0xa453, 0x5fdb, 0x92, + 0x65, 0xce, 0x36, 0x67, 0x3d, 0x5f, + 0x14); + +bool arch_is_gunyah_guest(void) +{ + struct arm_smccc_res res; + uuid_t uuid; + u32 *up; + + arm_smccc_1_1_hvc(ARM_SMCCC_VENDOR_HYP_CALL_UID_FUNC_ID, &res); + + up = (u32 *)&uuid.b[0]; + up[0] = lower_32_bits(res.a0); + up[1] = lower_32_bits(res.a1); + up[2] = lower_32_bits(res.a2); + up[3] = lower_32_bits(res.a3); + + return uuid_equal(&uuid, &GUNYAH_UUID); +} +EXPORT_SYMBOL_GPL(arch_is_gunyah_guest); + +#define GUNYAH_HYPERCALL(fn) \ + ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, ARM_SMCCC_SMC_64, \ + ARM_SMCCC_OWNER_VENDOR_HYP, fn) + +/* clang-format off */ +#define GUNYAH_HYPERCALL_HYP_IDENTIFY GUNYAH_HYPERCALL(0x8000) +/* clang-format on */ + +/** + * gunyah_hypercall_hyp_identify() - Returns build information and feature flags + * supported by Gunyah. + * @hyp_identity: filled by the hypercall with the API info and feature flags. + */ +void gunyah_hypercall_hyp_identify( + struct gunyah_hypercall_hyp_identify_resp *hyp_identity) +{ + struct arm_smccc_res res; + + arm_smccc_1_1_hvc(GUNYAH_HYPERCALL_HYP_IDENTIFY, &res); + + hyp_identity->api_info = res.a0; + hyp_identity->flags[0] = res.a1; + hyp_identity->flags[1] = res.a2; + hyp_identity->flags[2] = res.a3; +} +EXPORT_SYMBOL_GPL(gunyah_hypercall_hyp_identify); + +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("Gunyah Hypervisor Hypercalls"); diff --git a/drivers/virt/Kconfig b/drivers/virt/Kconfig index 40129b6f0eca..172a6a12073c 100644 --- a/drivers/virt/Kconfig +++ b/drivers/virt/Kconfig @@ -50,4 +50,6 @@ source "drivers/virt/acrn/Kconfig" source "drivers/virt/coco/Kconfig" +source "drivers/virt/gunyah/Kconfig" + endif diff --git a/drivers/virt/gunyah/Kconfig b/drivers/virt/gunyah/Kconfig new file mode 100644 index 000000000000..6f4c85db80b5 --- /dev/null +++ b/drivers/virt/gunyah/Kconfig @@ -0,0 +1,12 @@ +# SPDX-License-Identifier: GPL-2.0-only + +config GUNYAH + tristate "Gunyah Virtualization drivers" + depends on ARM64 + help + The Gunyah drivers are the helper interfaces that run in a guest VM + such as basic inter-VM IPC and signaling mechanisms, and higher level + services such as memory/device sharing, IRQ sharing, and so on. + + Say Y/M here to enable the drivers needed to interact in a Gunyah + virtual environment. diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index e1a716d802ee..8cdeaba34aa9 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -6,9 +6,11 @@ #ifndef _LINUX_GUNYAH_H #define _LINUX_GUNYAH_H +#include #include #include #include +#include /* Matches resource manager's resource types for VM_GET_HYP_RESOURCES RPC */ enum gunyah_resource_type { @@ -103,4 +105,40 @@ static inline int gunyah_error_remap(enum gunyah_error gunyah_error) } } +enum gunyah_api_feature { + /* clang-format off */ + GUNYAH_FEATURE_DOORBELL = 1, + GUNYAH_FEATURE_MSGQUEUE = 2, + GUNYAH_FEATURE_VCPU = 5, + GUNYAH_FEATURE_MEMEXTENT = 6, + /* clang-format on */ +}; + +bool arch_is_gunyah_guest(void); + +#define GUNYAH_API_V1 1 + +/* Other bits reserved for future use and will be zero */ +/* clang-format off */ +#define GUNYAH_API_INFO_API_VERSION_MASK GENMASK_ULL(13, 0) +#define GUNYAH_API_INFO_BIG_ENDIAN BIT_ULL(14) +#define GUNYAH_API_INFO_IS_64BIT BIT_ULL(15) +#define GUNYAH_API_INFO_VARIANT_MASK GENMASK_ULL(63, 56) +/* clang-format on */ + +struct gunyah_hypercall_hyp_identify_resp { + u64 api_info; + u64 flags[3]; +}; + +static inline u16 +gunyah_api_version(const struct gunyah_hypercall_hyp_identify_resp *gunyah_api) +{ + return FIELD_GET(GUNYAH_API_INFO_API_VERSION_MASK, + gunyah_api->api_info); +} + +void gunyah_hypercall_hyp_identify( + struct gunyah_hypercall_hyp_identify_resp *hyp_identity); + #endif From patchwork Sat Dec 16 00:20:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 179683 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:3b04:b0:fb:cd0c:d3e with SMTP id c4csp9665688dys; Fri, 15 Dec 2023 16:23:33 -0800 (PST) X-Google-Smtp-Source: AGHT+IFwsgIb2PEXFwOBts5dxYkonyC/FyNxvtf7NqtEPrMyt4lTJENhgk40z0etjJHy8m0MbZsy X-Received: by 2002:a92:c243:0:b0:35f:8b5b:3d11 with SMTP id k3-20020a92c243000000b0035f8b5b3d11mr3098556ilo.72.1702686213184; Fri, 15 Dec 2023 16:23:33 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702686213; cv=none; d=google.com; s=arc-20160816; b=IwVR6vz9gsy056WQkjyDsJY1YpaNiZwYZcF+s5OimkoLbuU3P/thNqt2Cy7VtGReOk T/L06xrVF9vJH/OGZujOAY7o7IAGWAF7udJ6ILvQn4mtv5CrUs33s6mc3oeHzhqHAnpu sVhmKXunHiZ6OXCAFxLQkq2TOot5HG7qzpQj2GJtWUhJ2xTUB9lWWtUV+GdTWjjqN08O s3Geg8YlEZzU1hMGSgWWqp5tQksiVYOBATOVyZlMkh9u6rLqqo91Qheqe5ac/B5G+Cll WuuOTraGIwmP4B23Y8GO8P4PDJkoQgVPW7BOh35UkQW3Gm8BPPqy+D58YLVBB82KoHLo uqyQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:list-unsubscribe:list-subscribe:list-id:precedence :subject:date:from:dkim-signature; bh=dlIQYYiZkB6lUKK/+BFyt+atRuKMzeGXRa/ViJoTpro=; fh=Pv/sMsQia5wmqAOYHIQUkkUFpDKhZjuODj71Ojb2ybw=; b=I9oFUg/DSQNCTN8cjHEQbnsqJu8zPAsHaAv5ZW1f3xb9atxzg0NJcifLVJBO/BXHlA RkSWAdK9n+vWjUZn6KAmfuvNpKg3iwcFGHS4WvBuYZjy8enTs6rP1g2HAvUTTTluEZFJ hoORs8u201CuazBSn6hJI7YPHOmQZxt1WEooOHhRbiDYw5el77cqEKR2MywPjD+jtcY/ MQSmk7ZP58xs9ilWieCG/M+r6rx8JDRoVNiMBpmCr61mZgcLa5IcUPIiBUL7hn+ddQlQ ZLPQwR8+C5m6TUFk0BV3O2kXMYusFWH0wR+74K75LrNcrX9Tjy3vTb32haVH4wXjJKWj 49eA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=XgnrPOn8; spf=pass (google.com: domain of linux-kernel+bounces-1871-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1871-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id i24-20020a63e918000000b005c65e63ae63si13774465pgh.14.2023.12.15.16.23.32 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Dec 2023 16:23:33 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-1871-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=XgnrPOn8; spf=pass (google.com: domain of linux-kernel+bounces-1871-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1871-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id E9AB8B239B0 for ; Sat, 16 Dec 2023 00:23:10 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 1DE51EAFE; Sat, 16 Dec 2023 00:21:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="XgnrPOn8" X-Original-To: linux-kernel@vger.kernel.org Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ED3C515C8; Sat, 16 Dec 2023 00:21:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=quicinc.com Received: from pps.filterd (m0279872.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BFNKPMB017090; Sat, 16 Dec 2023 00:21:05 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:date:subject:mime-version:content-type :content-transfer-encoding:message-id:references:in-reply-to:to :cc; s=qcppdkim1; bh=dlIQYYiZkB6lUKK/+BFyt+atRuKMzeGXRa/ViJoTpro =; b=XgnrPOn8duX+dLX0rITrDHx5nyIJOqtBgdKK6flEtygjndVvl8mJYkH5DAK k5BTtT7O812gqs2GM51JL2Fnx9sg8c21LUJ02TyTs8CHQ276e8KsFQ04orqtna0o hhac9XsNzrH1OysLGH1C8gyvFI18EuDQ9Fd1JnUaC9Gym4yl6Uoc8mPPmnN2F/xP gkbYO0pQyZOOqeq+16IrdDAenEYM08yp9R9Y6vgQMJFigngfBw3tnEPcAD4mmkJg M2BM1fd+1A2aMPlwoJY5bbBgkPdtJ36Whs8D1C5jSJumn2OdChYbu/ZuzBUmgtRE leddYxO1sPHE98X8QtuNOfj1YRw== Received: from nasanppmta05.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v0hb02g6f-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:04 +0000 (GMT) Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA05.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 3BG0L317006238 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:03 GMT Received: from [169.254.0.1] (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.40; Fri, 15 Dec 2023 16:21:02 -0800 From: Elliot Berman Date: Fri, 15 Dec 2023 16:20:46 -0800 Subject: [PATCH RFC v15 05/30] virt: gunyah: Add hypervisor driver Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <20231215-gunyah-v15-5-192a5d872a30@quicinc.com> References: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> In-Reply-To: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> To: Alex Elder , Srinivas Kandagatla , Murali Nalajal , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Philip Derrin , Prakruthi Deepak Heragu , Jonathan Corbet , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Catalin Marinas , Will Deacon , Konrad Dybcio , Bjorn Andersson , Dmitry Baryshkov , "Fuad Tabba" , Sean Christopherson CC: , , , , , Elliot Berman X-Mailer: b4 0.13-dev X-ClientProxiedBy: nalasex01a.na.qualcomm.com (10.47.209.196) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: d6FbTRpad0tCTBg7ARO1ukrJs2SMoeD7 X-Proofpoint-GUID: d6FbTRpad0tCTBg7ARO1ukrJs2SMoeD7 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 malwarescore=0 priorityscore=1501 impostorscore=0 spamscore=0 mlxlogscore=999 clxscore=1015 lowpriorityscore=0 bulkscore=0 suspectscore=0 adultscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312160000 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785395899058971555 X-GMAIL-MSGID: 1785395899058971555 Add driver to detect when running under Gunyah. It performs basic identification hypercall and populates the platform bus for resource manager to probe. Signed-off-by: Elliot Berman --- drivers/virt/Makefile | 1 + drivers/virt/gunyah/Makefile | 3 +++ drivers/virt/gunyah/gunyah.c | 52 ++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 56 insertions(+) diff --git a/drivers/virt/Makefile b/drivers/virt/Makefile index f29901bd7820..ef6a3835d078 100644 --- a/drivers/virt/Makefile +++ b/drivers/virt/Makefile @@ -10,3 +10,4 @@ obj-y += vboxguest/ obj-$(CONFIG_NITRO_ENCLAVES) += nitro_enclaves/ obj-$(CONFIG_ACRN_HSM) += acrn/ obj-y += coco/ +obj-y += gunyah/ diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile new file mode 100644 index 000000000000..34f32110faf9 --- /dev/null +++ b/drivers/virt/gunyah/Makefile @@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0 + +obj-$(CONFIG_GUNYAH) += gunyah.o diff --git a/drivers/virt/gunyah/gunyah.c b/drivers/virt/gunyah/gunyah.c new file mode 100644 index 000000000000..20bd6029ffc4 --- /dev/null +++ b/drivers/virt/gunyah/gunyah.c @@ -0,0 +1,52 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include +#include +#include +#include + +static int gunyah_probe(struct platform_device *pdev) +{ + struct gunyah_hypercall_hyp_identify_resp gunyah_api; + + if (!arch_is_gunyah_guest()) + return -ENODEV; + + gunyah_hypercall_hyp_identify(&gunyah_api); + + pr_info("Running under Gunyah hypervisor %llx/v%u\n", + FIELD_GET(GUNYAH_API_INFO_VARIANT_MASK, gunyah_api.api_info), + gunyah_api_version(&gunyah_api)); + + /* Might move this out to individual drivers if there's ever an API version bump */ + if (gunyah_api_version(&gunyah_api) != GUNYAH_API_V1) { + pr_info("Unsupported Gunyah version: %u\n", + gunyah_api_version(&gunyah_api)); + return -ENODEV; + } + + return devm_of_platform_populate(&pdev->dev); +} + +static const struct of_device_id gunyah_of_match[] = { + { .compatible = "gunyah-hypervisor" }, + {} +}; +MODULE_DEVICE_TABLE(of, gunyah_of_match); + +/* clang-format off */ +static struct platform_driver gunyah_driver = { + .probe = gunyah_probe, + .driver = { + .name = "gunyah", + .of_match_table = gunyah_of_match, + } +}; +/* clang-format on */ +module_platform_driver(gunyah_driver); + +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("Gunyah Driver"); From patchwork Sat Dec 16 00:20:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 179681 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:3b04:b0:fb:cd0c:d3e with SMTP id c4csp9665291dys; Fri, 15 Dec 2023 16:22:36 -0800 (PST) X-Google-Smtp-Source: AGHT+IEdxAufxUOSCYtkPL0Yblyf3dwcu2zdoLSYz8/w1k15cbSS27Kp9SnBk1xyKkJp+f75jnB+ X-Received: by 2002:a92:cd87:0:b0:35d:4da0:1dc with SMTP id r7-20020a92cd87000000b0035d4da001dcmr17846961ilb.16.1702686156075; Fri, 15 Dec 2023 16:22:36 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702686156; cv=none; d=google.com; s=arc-20160816; b=vpCJ6lgnWTQeZVG4KSSYkyU6DyiEGQLD4FtQlYt+gRLGse8jXko9wgy2rwtOF1CZlQ QRAgPr+Px4hTl9P1GSPvD9dkLHVQanKcFfsyelK+8iAjiPeVO0VAn45GKH7+aFyWoi9y 9x9cXhzuZJeEd8JmMHXBwzvxzdo55Rl2mOepbjZGdPrPMkgPc0lZ5KSxD7HZ7x5HPRqC ULOgEMbDd1EgLUI95wDjBiRWF9Fd2rz8Mg2myxIc+Q+sfbPQgN9xir1aJ2Kwg8TBAfYE D69AKW4hHoqo41dUV1OtZ87HtP2pJoKu6wHCPU5xNNDTSxHHvPN/tLW8+EZ6pfLDb95d x5wQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:list-unsubscribe:list-subscribe:list-id:precedence :subject:date:from:dkim-signature; bh=OEEOctwZRiqnvSmFq54/UY9u6/0OSkhlKKf9mPwOG6w=; fh=Pv/sMsQia5wmqAOYHIQUkkUFpDKhZjuODj71Ojb2ybw=; b=RlcSc3F66egQq3VPGGXhyVgxFyRo1Ax9FafkCJ7qyNT1h2VehRghF5U9r/GtjJ/xee E74DAq9A8OJtm16OdJi34Wqml/G/Wg/YNIkqFedQ6oqEOs3sucNEuQDOE8xxPLJ0Kokk 8XY8G6UBu1+jSGo8VyolbOWnQZHB1k3siLNe3MqP2ai+yDnpFV+8HPAs7m3dI7zxO8dW n0cpQo5gHsFtE21bdGX7Wmj+WMGhy7WejWTWghYe9xXrq6MdRr8ZZRbgNonX1SO3phWg tPIUMCBlAnax6PJPXvmBIw1PlOovXIfUAvXjtN93auZ1CRX/hCSYK2TAuIivryps6LQJ 5vlQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=bBhf+SDl; spf=pass (google.com: domain of linux-kernel+bounces-1870-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1870-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [147.75.48.161]) by mx.google.com with ESMTPS id b5-20020a170902a9c500b001d0bc650573si13464933plr.225.2023.12.15.16.22.35 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Dec 2023 16:22:36 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-1870-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=bBhf+SDl; spf=pass (google.com: domain of linux-kernel+bounces-1870-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1870-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 478C5B226EA for ; Sat, 16 Dec 2023 00:22:23 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 67C634A3B; Sat, 16 Dec 2023 00:21:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="bBhf+SDl" X-Original-To: linux-kernel@vger.kernel.org Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5ED72A46; Sat, 16 Dec 2023 00:21:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=quicinc.com Received: from pps.filterd (m0279864.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BG0GDF0016567; Sat, 16 Dec 2023 00:21:05 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:date:subject:mime-version:content-type :content-transfer-encoding:message-id:references:in-reply-to:to :cc; s=qcppdkim1; bh=OEEOctwZRiqnvSmFq54/UY9u6/0OSkhlKKf9mPwOG6w =; b=bBhf+SDldV7WY7ylnnV6FoIgJtYHK9ui9F3aGt+Uhub0UjTy9B7PNuTUH2+ 42UBT4BfCbhjQim18xuqpHqWt/WMRuYkXb2FDGaYdPerw4phe9D1IMSumhxJvoWa ij8r770Oub3EQEErnFtYRmwaDTUnviiMfitGrlOX/5aKMPc8bnHLDpCLg2SUqI0y uaRjeUGjXTpGfApyaltWl/EVpCWOQ0AjeSlidrL9Bz1/plJTTNC1rC70/ehQnGlZ y1w9nuJ2KXC1hBQrhrCA7RWmRbL86l0kTHI8t0d0NNUhxmG24aQGhHtx6waMa2w4 y4B0BmTDJOK7EQDUgbWPfTSK+Tg== Received: from nasanppmta01.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v0hqcjf8e-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:05 +0000 (GMT) Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 3BG0L45w015519 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:04 GMT Received: from [169.254.0.1] (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.40; Fri, 15 Dec 2023 16:21:03 -0800 From: Elliot Berman Date: Fri, 15 Dec 2023 16:20:47 -0800 Subject: [PATCH RFC v15 06/30] virt: gunyah: msgq: Add hypercalls to send and receive messages Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <20231215-gunyah-v15-6-192a5d872a30@quicinc.com> References: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> In-Reply-To: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> To: Alex Elder , Srinivas Kandagatla , Murali Nalajal , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Philip Derrin , Prakruthi Deepak Heragu , Jonathan Corbet , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Catalin Marinas , Will Deacon , Konrad Dybcio , Bjorn Andersson , Dmitry Baryshkov , "Fuad Tabba" , Sean Christopherson CC: , , , , , Elliot Berman X-Mailer: b4 0.13-dev X-ClientProxiedBy: nalasex01a.na.qualcomm.com (10.47.209.196) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: QmBFIMYPI86yIje7NAN2g-T1lKudR__E X-Proofpoint-GUID: QmBFIMYPI86yIje7NAN2g-T1lKudR__E X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 mlxscore=0 phishscore=0 adultscore=0 priorityscore=1501 impostorscore=0 lowpriorityscore=0 mlxlogscore=999 clxscore=1015 suspectscore=0 spamscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312160001 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785395838657494509 X-GMAIL-MSGID: 1785395838657494509 Add hypercalls to send and receive messages on a Gunyah message queue. Reviewed-by: Alex Elder Reviewed-by: Srinivas Kandagatla Signed-off-by: Elliot Berman --- arch/arm64/gunyah/gunyah_hypercall.c | 34 ++++++++++++++++++++++++++++++++++ include/linux/gunyah.h | 7 +++++++ 2 files changed, 41 insertions(+) diff --git a/arch/arm64/gunyah/gunyah_hypercall.c b/arch/arm64/gunyah/gunyah_hypercall.c index 739d7f9cef9f..cadc512934c4 100644 --- a/arch/arm64/gunyah/gunyah_hypercall.c +++ b/arch/arm64/gunyah/gunyah_hypercall.c @@ -37,6 +37,8 @@ EXPORT_SYMBOL_GPL(arch_is_gunyah_guest); /* clang-format off */ #define GUNYAH_HYPERCALL_HYP_IDENTIFY GUNYAH_HYPERCALL(0x8000) +#define GUNYAH_HYPERCALL_MSGQ_SEND GUNYAH_HYPERCALL(0x801B) +#define GUNYAH_HYPERCALL_MSGQ_RECV GUNYAH_HYPERCALL(0x801C) /* clang-format on */ /** @@ -58,5 +60,37 @@ void gunyah_hypercall_hyp_identify( } EXPORT_SYMBOL_GPL(gunyah_hypercall_hyp_identify); +enum gunyah_error gunyah_hypercall_msgq_send(u64 capid, size_t size, void *buff, + u64 tx_flags, bool *ready) +{ + struct arm_smccc_res res; + + arm_smccc_1_1_hvc(GUNYAH_HYPERCALL_MSGQ_SEND, capid, size, + (uintptr_t)buff, tx_flags, 0, &res); + + if (res.a0 == GUNYAH_ERROR_OK) + *ready = !!res.a1; + + return res.a0; +} +EXPORT_SYMBOL_GPL(gunyah_hypercall_msgq_send); + +enum gunyah_error gunyah_hypercall_msgq_recv(u64 capid, void *buff, size_t size, + size_t *recv_size, bool *ready) +{ + struct arm_smccc_res res; + + arm_smccc_1_1_hvc(GUNYAH_HYPERCALL_MSGQ_RECV, capid, (uintptr_t)buff, + size, 0, &res); + + if (res.a0 == GUNYAH_ERROR_OK) { + *recv_size = res.a1; + *ready = !!res.a2; + } + + return res.a0; +} +EXPORT_SYMBOL_GPL(gunyah_hypercall_msgq_recv); + MODULE_LICENSE("GPL"); MODULE_DESCRIPTION("Gunyah Hypervisor Hypercalls"); diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index 8cdeaba34aa9..ea2c9eecce0e 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -141,4 +141,11 @@ gunyah_api_version(const struct gunyah_hypercall_hyp_identify_resp *gunyah_api) void gunyah_hypercall_hyp_identify( struct gunyah_hypercall_hyp_identify_resp *hyp_identity); +#define GUNYAH_HYPERCALL_MSGQ_TX_FLAGS_PUSH BIT(0) + +enum gunyah_error gunyah_hypercall_msgq_send(u64 capid, size_t size, void *buff, + u64 tx_flags, bool *ready); +enum gunyah_error gunyah_hypercall_msgq_recv(u64 capid, void *buff, size_t size, + size_t *recv_size, bool *ready); + #endif From patchwork Sat Dec 16 00:20:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 179687 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:3b04:b0:fb:cd0c:d3e with SMTP id c4csp9666214dys; Fri, 15 Dec 2023 16:24:59 -0800 (PST) X-Google-Smtp-Source: AGHT+IGyHHZ3OiiyVJ8WiJKGFfp1XbSLSjYKj7JRFacwvkOa1umPTLHBC1eAIwKjLQhKlTIcgUKU X-Received: by 2002:a05:6214:5191:b0:67f:e29:d4f1 with SMTP id kl17-20020a056214519100b0067f0e29d4f1mr3659193qvb.115.1702686299260; Fri, 15 Dec 2023 16:24:59 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702686299; cv=none; d=google.com; s=arc-20160816; b=HRMcUV1FuS2EJiM0B2IIN8DcKnevmgFrzPHHDPLTPui4YxZD7tD10dpab6hBVAGsCV 1byEpXg+H9fme7aYFT8JoYl50OvvChoRd/c/yzYrqPIdShjQgJYmAnU24m5bs52nf7xs 4Y5i5GEGSEtDCWm28aOFdDEu7oGSk+BeBVHoW6Obi9gavicLbxHFGOvLt/E8GdhZyU0S 593z343ndut6OAX192IlnZMmAIsjzgOqDfEobm91++MQHlG3J8MR8eXgCucs3GjQYyh6 jLnSmDSPeUpIdez9Dl+/dgWfZ/JkX0D+0pMmoQUfYwM6A4Wxi5ZQ2DwiyTwh7jb3fYco uA9w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:list-unsubscribe:list-subscribe:list-id:precedence :subject:date:from:dkim-signature; bh=K0TD9xOe034V4LjXXOI7EWo8ceuUNI/cL78dA4epa+w=; fh=Pv/sMsQia5wmqAOYHIQUkkUFpDKhZjuODj71Ojb2ybw=; b=eEO+ygBC3k8AbiBdTlmqi+TRcx6lD9OPBtZND5JCRIno2ZvkarxQx073X4cFEltuCd 0uYMn4YzR6QJiEQ+xDV9oyMYpCKG5h6Xh1DsypwkKJPjFT4NtZK2hnknmBw19O7cc+kg fJzIvMpmEUXvH1yeYZOiobVpvM01NcMqnQjQgxgJMA+sUzTMjfC4XwyxM1msQdcBECuA 1qSLbH4F0kjTzXDAk4i6KJ9d89c9kggRJI1M94tTUmzEfU3/Hne+L3+Y/a7ep+aKQ8Tl 08vgJsnObawMge1AZfOzu2difj0De4KZfat4RlXXTw6MhndgwvpGLxQw0BWdPNwNjl4j rx9w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b="gU7wxl/h"; spf=pass (google.com: domain of linux-kernel+bounces-1877-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1877-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id z17-20020a0cf011000000b0067a1d3d8b52si4320828qvk.568.2023.12.15.16.24.59 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Dec 2023 16:24:59 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-1877-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b="gU7wxl/h"; spf=pass (google.com: domain of linux-kernel+bounces-1877-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1877-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 030E41C2103F for ; Sat, 16 Dec 2023 00:24:59 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 0755918EC1; Sat, 16 Dec 2023 00:21:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="gU7wxl/h" X-Original-To: linux-kernel@vger.kernel.org Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 92F355229; Sat, 16 Dec 2023 00:21:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=quicinc.com Received: from pps.filterd (m0279865.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BFNcKv5027937; Sat, 16 Dec 2023 00:21:06 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:date:subject:mime-version:content-type :content-transfer-encoding:message-id:references:in-reply-to:to :cc; s=qcppdkim1; bh=K0TD9xOe034V4LjXXOI7EWo8ceuUNI/cL78dA4epa+w =; b=gU7wxl/hfNhNSGqzVxtw4BdEVYehxVD/ZuOpqwQh3oljEtdHQeKndyy5Qei Yb7oWYzVyoT2iBnH6RSFKzSHYPRSoSM7LNLIIUikX385klj8d0lDZonmDkFHmGpe VDupjlg//bUUESeFdCj+dnoSCzLNi6Fev+FhZCEsClT6rM03MWlhEoznUs2Piquz OkhZ89TldnMO+ThG3gVPsQVxWr/wYUiSy2F7AkIlzKG6eDwvT4pb5U+E1/1AXhzA LYnqdqqSgqzSPg2at7KLHxGQmAT55g01WgJ3f47Kr6Nd+okKARpmc3iRkyY/gItz k7G1zZ4XunmOQa93COmPJX2Q4Fw== Received: from nasanppmta02.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v0m46hwdw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:05 +0000 (GMT) Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA02.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 3BG0L5lt014620 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:05 GMT Received: from [169.254.0.1] (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.40; Fri, 15 Dec 2023 16:21:04 -0800 From: Elliot Berman Date: Fri, 15 Dec 2023 16:20:48 -0800 Subject: [PATCH RFC v15 07/30] gunyah: rsc_mgr: Add resource manager RPC core Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <20231215-gunyah-v15-7-192a5d872a30@quicinc.com> References: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> In-Reply-To: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> To: Alex Elder , Srinivas Kandagatla , Murali Nalajal , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Philip Derrin , Prakruthi Deepak Heragu , Jonathan Corbet , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Catalin Marinas , Will Deacon , Konrad Dybcio , Bjorn Andersson , Dmitry Baryshkov , "Fuad Tabba" , Sean Christopherson CC: , , , , , Elliot Berman X-Mailer: b4 0.13-dev X-ClientProxiedBy: nalasex01a.na.qualcomm.com (10.47.209.196) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: BuF0T7Rn0wEiF8MiJD4t8w_mCEKONyVX X-Proofpoint-ORIG-GUID: BuF0T7Rn0wEiF8MiJD4t8w_mCEKONyVX X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 suspectscore=0 adultscore=0 phishscore=0 mlxscore=0 clxscore=1015 mlxlogscore=999 impostorscore=0 malwarescore=0 bulkscore=0 spamscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312160001 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785395988907065238 X-GMAIL-MSGID: 1785395988907065238 The resource manager is a special virtual machine which is always running on a Gunyah system. It provides APIs for creating and destroying VMs, secure memory management, sharing/lending of memory between VMs, and setup of inter-VM communication. Calls to the resource manager are made via message queues. This patch implements the basic probing and RPC mechanism to make those API calls. Request/response calls can be made with gh_rm_call. Drivers can also register to notifications pushed by RM via gh_rm_register_notifier Specific API calls that resource manager supports will be implemented in subsequent patches. Reviewed-by: Alex Elder Reviewed-by: Srinivas Kandagatla Signed-off-by: Elliot Berman --- drivers/virt/gunyah/Makefile | 4 +- drivers/virt/gunyah/rsc_mgr.c | 725 +++++++++++++++++++++++++++++++++++++++++ drivers/virt/gunyah/rsc_mgr.h | 18 + include/linux/gunyah_rsc_mgr.h | 22 ++ 4 files changed, 768 insertions(+), 1 deletion(-) diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index 34f32110faf9..c2308389f551 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -1,3 +1,5 @@ # SPDX-License-Identifier: GPL-2.0 -obj-$(CONFIG_GUNYAH) += gunyah.o +gunyah_rsc_mgr-y += rsc_mgr.o + +obj-$(CONFIG_GUNYAH) += gunyah.o gunyah_rsc_mgr.o diff --git a/drivers/virt/gunyah/rsc_mgr.c b/drivers/virt/gunyah/rsc_mgr.c new file mode 100644 index 000000000000..2f9afe167aa6 --- /dev/null +++ b/drivers/virt/gunyah/rsc_mgr.c @@ -0,0 +1,725 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "rsc_mgr.h" + +/* clang-format off */ +#define RM_RPC_API_VERSION_MASK GENMASK(3, 0) +#define RM_RPC_HEADER_WORDS_MASK GENMASK(7, 4) +#define RM_RPC_API_VERSION FIELD_PREP(RM_RPC_API_VERSION_MASK, 1) +#define RM_RPC_HEADER_WORDS FIELD_PREP(RM_RPC_HEADER_WORDS_MASK, \ + (sizeof(struct gunyah_rm_rpc_hdr) / sizeof(u32))) +#define RM_RPC_API (RM_RPC_API_VERSION | RM_RPC_HEADER_WORDS) + +#define RM_RPC_TYPE_CONTINUATION 0x0 +#define RM_RPC_TYPE_REQUEST 0x1 +#define RM_RPC_TYPE_REPLY 0x2 +#define RM_RPC_TYPE_NOTIF 0x3 +#define RM_RPC_TYPE_MASK GENMASK(1, 0) + +#define GUNYAH_RM_MAX_NUM_FRAGMENTS 62 +#define RM_RPC_FRAGMENTS_MASK GENMASK(7, 2) +/* clang-format on */ + +struct gunyah_rm_rpc_hdr { + u8 api; + u8 type; + __le16 seq; + __le32 msg_id; +} __packed; + +struct gunyah_rm_rpc_reply_hdr { + struct gunyah_rm_rpc_hdr hdr; + __le32 err_code; /* GUNYAH_RM_ERROR_* */ +} __packed; + +#define GUNYAH_RM_MSGQ_MSG_SIZE 240 +#define GUNYAH_RM_PAYLOAD_SIZE \ + (GUNYAH_RM_MSGQ_MSG_SIZE - sizeof(struct gunyah_rm_rpc_hdr)) + +/* RM Error codes */ +enum gunyah_rm_error { + /* clang-format off */ + GUNYAH_RM_ERROR_OK = 0x0, + GUNYAH_RM_ERROR_UNIMPLEMENTED = 0xFFFFFFFF, + GUNYAH_RM_ERROR_NOMEM = 0x1, + GUNYAH_RM_ERROR_NORESOURCE = 0x2, + GUNYAH_RM_ERROR_DENIED = 0x3, + GUNYAH_RM_ERROR_INVALID = 0x4, + GUNYAH_RM_ERROR_BUSY = 0x5, + GUNYAH_RM_ERROR_ARGUMENT_INVALID = 0x6, + GUNYAH_RM_ERROR_HANDLE_INVALID = 0x7, + GUNYAH_RM_ERROR_VALIDATE_FAILED = 0x8, + GUNYAH_RM_ERROR_MAP_FAILED = 0x9, + GUNYAH_RM_ERROR_MEM_INVALID = 0xA, + GUNYAH_RM_ERROR_MEM_INUSE = 0xB, + GUNYAH_RM_ERROR_MEM_RELEASED = 0xC, + GUNYAH_RM_ERROR_VMID_INVALID = 0xD, + GUNYAH_RM_ERROR_LOOKUP_FAILED = 0xE, + GUNYAH_RM_ERROR_IRQ_INVALID = 0xF, + GUNYAH_RM_ERROR_IRQ_INUSE = 0x10, + GUNYAH_RM_ERROR_IRQ_RELEASED = 0x11, + /* clang-format on */ +}; + +/** + * struct gunyah_rm_message - Represents a complete message from resource manager + * @payload: Combined payload of all the fragments (msg headers stripped off). + * @size: Size of the payload received so far. + * @msg_id: Message ID from the header. + * @type: RM_RPC_TYPE_REPLY or RM_RPC_TYPE_NOTIF. + * @num_fragments: total number of fragments expected to be received. + * @fragments_received: fragments received so far. + * @reply: Fields used for request/reply sequences + */ +struct gunyah_rm_message { + void *payload; + size_t size; + u32 msg_id; + u8 type; + + u8 num_fragments; + u8 fragments_received; + + /** + * @ret: Linux return code, there was an error processing message + * @seq: Sequence ID for the main message. + * @rm_error: For request/reply sequences with standard replies + * @seq_done: Signals caller that the RM reply has been received + */ + struct { + int ret; + u16 seq; + enum gunyah_rm_error rm_error; + struct completion seq_done; + } reply; +}; + +/** + * struct gunyah_rm - private data for communicating w/Gunyah resource manager + * @dev: pointer to RM platform device + * @tx_ghrsc: message queue resource to TX to RM + * @rx_ghrsc: message queue resource to RX from RM + * @msgq: mailbox instance of TX/RX resources above + * @msgq_client: mailbox client of above msgq + * @active_rx_message: ongoing gunyah_rm_message for which we're receiving fragments + * @last_tx_ret: return value of last mailbox tx + * @call_xarray: xarray to allocate & lookup sequence IDs for Request/Response flows + * @next_seq: next ID to allocate (for xa_alloc_cyclic) + * @recv_msg: cached allocation for Rx messages + * @send_msg: cached allocation for Tx messages + * @send_lock: synchronization to allow only one request to be sent at a time + * @send_ready: completion to send messages again + * @nh: notifier chain for clients interested in RM notification messages + */ +struct gunyah_rm { + struct device *dev; + struct gunyah_resource tx_ghrsc; + struct gunyah_resource rx_ghrsc; + struct gunyah_rm_message *active_rx_message; + int last_tx_ret; + + struct xarray call_xarray; + u32 next_seq; + + unsigned char recv_msg[GUNYAH_RM_MSGQ_MSG_SIZE]; + unsigned char send_msg[GUNYAH_RM_MSGQ_MSG_SIZE]; + struct mutex send_lock; + struct completion send_ready; + struct blocking_notifier_head nh; +}; + +/** + * gunyah_rm_error_remap() - Remap Gunyah resource manager errors into a Linux error code + * @rm_error: "Standard" return value from Gunyah resource manager + */ +static inline int gunyah_rm_error_remap(enum gunyah_rm_error rm_error) +{ + switch (rm_error) { + case GUNYAH_RM_ERROR_OK: + return 0; + case GUNYAH_RM_ERROR_UNIMPLEMENTED: + return -EOPNOTSUPP; + case GUNYAH_RM_ERROR_NOMEM: + return -ENOMEM; + case GUNYAH_RM_ERROR_NORESOURCE: + return -ENODEV; + case GUNYAH_RM_ERROR_DENIED: + return -EPERM; + case GUNYAH_RM_ERROR_BUSY: + return -EBUSY; + case GUNYAH_RM_ERROR_INVALID: + case GUNYAH_RM_ERROR_ARGUMENT_INVALID: + case GUNYAH_RM_ERROR_HANDLE_INVALID: + case GUNYAH_RM_ERROR_VALIDATE_FAILED: + case GUNYAH_RM_ERROR_MAP_FAILED: + case GUNYAH_RM_ERROR_MEM_INVALID: + case GUNYAH_RM_ERROR_MEM_INUSE: + case GUNYAH_RM_ERROR_MEM_RELEASED: + case GUNYAH_RM_ERROR_VMID_INVALID: + case GUNYAH_RM_ERROR_LOOKUP_FAILED: + case GUNYAH_RM_ERROR_IRQ_INVALID: + case GUNYAH_RM_ERROR_IRQ_INUSE: + case GUNYAH_RM_ERROR_IRQ_RELEASED: + return -EINVAL; + default: + return -EBADMSG; + } +} + +static int gunyah_rm_init_message_payload(struct gunyah_rm_message *message, + const void *msg, size_t hdr_size, + size_t msg_size) +{ + const struct gunyah_rm_rpc_hdr *hdr = msg; + size_t max_buf_size, payload_size; + + if (msg_size < hdr_size) + return -EINVAL; + + payload_size = msg_size - hdr_size; + + message->num_fragments = FIELD_GET(RM_RPC_FRAGMENTS_MASK, hdr->type); + message->fragments_received = 0; + + /* There's not going to be any payload, no need to allocate buffer. */ + if (!payload_size && !message->num_fragments) + return 0; + + if (message->num_fragments > GUNYAH_RM_MAX_NUM_FRAGMENTS) + return -EINVAL; + + max_buf_size = payload_size + + (message->num_fragments * GUNYAH_RM_PAYLOAD_SIZE); + + message->payload = kzalloc(max_buf_size, GFP_KERNEL); + if (!message->payload) + return -ENOMEM; + + memcpy(message->payload, msg + hdr_size, payload_size); + message->size = payload_size; + return 0; +} + +static void gunyah_rm_abort_message(struct gunyah_rm *rm) +{ + switch (rm->active_rx_message->type) { + case RM_RPC_TYPE_REPLY: + rm->active_rx_message->reply.ret = -EIO; + complete(&rm->active_rx_message->reply.seq_done); + break; + case RM_RPC_TYPE_NOTIF: + fallthrough; + default: + kfree(rm->active_rx_message->payload); + kfree(rm->active_rx_message); + } + + rm->active_rx_message = NULL; +} + +static inline void gunyah_rm_try_complete_message(struct gunyah_rm *rm) +{ + struct gunyah_rm_message *message = rm->active_rx_message; + + if (!message || message->fragments_received != message->num_fragments) + return; + + switch (message->type) { + case RM_RPC_TYPE_REPLY: + complete(&message->reply.seq_done); + break; + case RM_RPC_TYPE_NOTIF: + blocking_notifier_call_chain(&rm->nh, message->msg_id, + message->payload); + + kfree(message->payload); + kfree(message); + break; + default: + dev_err_ratelimited(rm->dev, + "Invalid message type (%u) received\n", + message->type); + gunyah_rm_abort_message(rm); + break; + } + + rm->active_rx_message = NULL; +} + +static void gunyah_rm_process_notif(struct gunyah_rm *rm, const void *msg, + size_t msg_size) +{ + const struct gunyah_rm_rpc_hdr *hdr = msg; + struct gunyah_rm_message *message; + int ret; + + if (rm->active_rx_message) + gunyah_rm_abort_message(rm); + + message = kzalloc(sizeof(*message), GFP_KERNEL); + if (!message) + return; + + message->type = RM_RPC_TYPE_NOTIF; + message->msg_id = le32_to_cpu(hdr->msg_id); + + ret = gunyah_rm_init_message_payload(message, msg, sizeof(*hdr), + msg_size); + if (ret) { + dev_err(rm->dev, + "Failed to initialize message for notification: %d\n", + ret); + kfree(message); + return; + } + + rm->active_rx_message = message; + + gunyah_rm_try_complete_message(rm); +} + +static void gunyah_rm_process_reply(struct gunyah_rm *rm, const void *msg, + size_t msg_size) +{ + const struct gunyah_rm_rpc_reply_hdr *reply_hdr = msg; + struct gunyah_rm_message *message; + u16 seq_id; + + seq_id = le16_to_cpu(reply_hdr->hdr.seq); + message = xa_load(&rm->call_xarray, seq_id); + + if (!message || message->msg_id != le32_to_cpu(reply_hdr->hdr.msg_id)) + return; + + if (rm->active_rx_message) + gunyah_rm_abort_message(rm); + + if (gunyah_rm_init_message_payload(message, msg, sizeof(*reply_hdr), + msg_size)) { + dev_err(rm->dev, + "Failed to alloc message buffer for sequence %d\n", + seq_id); + /* Send message complete and error the client. */ + message->reply.ret = -ENOMEM; + complete(&message->reply.seq_done); + return; + } + + message->reply.rm_error = le32_to_cpu(reply_hdr->err_code); + rm->active_rx_message = message; + + gunyah_rm_try_complete_message(rm); +} + +static void gunyah_rm_process_cont(struct gunyah_rm *rm, + struct gunyah_rm_message *message, + const void *msg, size_t msg_size) +{ + const struct gunyah_rm_rpc_hdr *hdr = msg; + size_t payload_size = msg_size - sizeof(*hdr); + + if (!rm->active_rx_message) + return; + + /* + * hdr->fragments and hdr->msg_id preserves the value from first reply + * or notif message. To detect mishandling, check it's still intact. + */ + if (message->msg_id != le32_to_cpu(hdr->msg_id) || + message->num_fragments != + FIELD_GET(RM_RPC_FRAGMENTS_MASK, hdr->type)) { + gunyah_rm_abort_message(rm); + return; + } + + memcpy(message->payload + message->size, msg + sizeof(*hdr), + payload_size); + message->size += payload_size; + message->fragments_received++; + + gunyah_rm_try_complete_message(rm); +} + +static irqreturn_t gunyah_rm_rx(int irq, void *data) +{ + enum gunyah_error gunyah_error; + struct gunyah_rm_rpc_hdr *hdr; + struct gunyah_rm *rm = data; + void *msg = &rm->recv_msg[0]; + size_t len; + bool ready; + + do { + gunyah_error = gunyah_hypercall_msgq_recv(rm->rx_ghrsc.capid, + msg, + sizeof(rm->recv_msg), + &len, &ready); + if (gunyah_error != GUNYAH_ERROR_OK) { + if (gunyah_error != GUNYAH_ERROR_MSGQUEUE_EMPTY) + dev_warn(rm->dev, + "Failed to receive data: %d\n", + gunyah_error); + return IRQ_HANDLED; + } + + if (len < sizeof(*hdr)) { + dev_err_ratelimited( + rm->dev, + "Too small message received. size=%ld\n", len); + continue; + } + + hdr = msg; + if (hdr->api != RM_RPC_API) { + dev_err(rm->dev, "Unknown RM RPC API version: %x\n", + hdr->api); + return IRQ_HANDLED; + } + + switch (FIELD_GET(RM_RPC_TYPE_MASK, hdr->type)) { + case RM_RPC_TYPE_NOTIF: + gunyah_rm_process_notif(rm, msg, len); + break; + case RM_RPC_TYPE_REPLY: + gunyah_rm_process_reply(rm, msg, len); + break; + case RM_RPC_TYPE_CONTINUATION: + gunyah_rm_process_cont(rm, rm->active_rx_message, msg, + len); + break; + default: + dev_err(rm->dev, + "Invalid message type (%lu) received\n", + FIELD_GET(RM_RPC_TYPE_MASK, hdr->type)); + return IRQ_HANDLED; + } + } while (ready); + + return IRQ_HANDLED; +} + +static irqreturn_t gunyah_rm_tx(int irq, void *data) +{ + struct gunyah_rm *rm = data; + + complete_all(&rm->send_ready); + + return IRQ_HANDLED; +} + +static int gunyah_rm_msgq_send(struct gunyah_rm *rm, size_t size, bool push) + __must_hold(&rm->send_lock) +{ + const u64 tx_flags = push ? GUNYAH_HYPERCALL_MSGQ_TX_FLAGS_PUSH : 0; + enum gunyah_error gunyah_error; + void *data = &rm->send_msg[0]; + bool ready; + +again: + wait_for_completion(&rm->send_ready); + /* reinit completion before hypercall. As soon as hypercall returns, we could get the + * ready interrupt. This might be before we have time to reinit the completion + */ + reinit_completion(&rm->send_ready); + gunyah_error = gunyah_hypercall_msgq_send(rm->tx_ghrsc.capid, size, + data, tx_flags, &ready); + + /* Should never happen because Linux properly tracks the ready-state of the msgq */ + if (WARN_ON(gunyah_error == GUNYAH_ERROR_MSGQUEUE_FULL)) + goto again; + + if (ready) + complete_all(&rm->send_ready); + + return gunyah_error_remap(gunyah_error); +} + +static int gunyah_rm_send_request(struct gunyah_rm *rm, u32 message_id, + const void *req_buf, size_t req_buf_size, + struct gunyah_rm_message *message) +{ + size_t buf_size_remaining = req_buf_size; + const void *req_buf_curr = req_buf; + struct gunyah_rm_rpc_hdr *hdr = + (struct gunyah_rm_rpc_hdr *)&rm->send_msg[0]; + struct gunyah_rm_rpc_hdr hdr_template; + void *payload = hdr + 1; + u32 cont_fragments = 0; + size_t payload_size; + int ret; + + if (req_buf_size > + GUNYAH_RM_MAX_NUM_FRAGMENTS * GUNYAH_RM_PAYLOAD_SIZE) { + dev_warn( + rm->dev, + "Limit (%lu bytes) exceeded for the maximum message size: %lu\n", + GUNYAH_RM_MAX_NUM_FRAGMENTS * GUNYAH_RM_PAYLOAD_SIZE, + req_buf_size); + dump_stack(); + return -E2BIG; + } + + if (req_buf_size) + cont_fragments = (req_buf_size - 1) / GUNYAH_RM_PAYLOAD_SIZE; + + hdr_template.api = RM_RPC_API; + hdr_template.type = FIELD_PREP(RM_RPC_TYPE_MASK, RM_RPC_TYPE_REQUEST) | + FIELD_PREP(RM_RPC_FRAGMENTS_MASK, cont_fragments); + hdr_template.seq = cpu_to_le16(message->reply.seq); + hdr_template.msg_id = cpu_to_le32(message_id); + + ret = mutex_lock_interruptible(&rm->send_lock); + if (ret) + return ret; + + do { + *hdr = hdr_template; + + /* Copy payload */ + payload_size = min(buf_size_remaining, GUNYAH_RM_PAYLOAD_SIZE); + memcpy(payload, req_buf_curr, payload_size); + req_buf_curr += payload_size; + buf_size_remaining -= payload_size; + + /* Force the last fragment to immediately alert the receiver */ + ret = gunyah_rm_msgq_send(rm, sizeof(*hdr) + payload_size, + !buf_size_remaining); + if (ret) + break; + + hdr_template.type = + FIELD_PREP(RM_RPC_TYPE_MASK, RM_RPC_TYPE_CONTINUATION) | + FIELD_PREP(RM_RPC_FRAGMENTS_MASK, cont_fragments); + } while (buf_size_remaining); + + mutex_unlock(&rm->send_lock); + return ret; +} + +/** + * gunyah_rm_call: Achieve request-response type communication with RPC + * @rm: Pointer to Gunyah resource manager internal data + * @message_id: The RM RPC message-id + * @req_buf: Request buffer that contains the payload + * @req_buf_size: Total size of the payload + * @resp_buf: Pointer to a response buffer + * @resp_buf_size: Size of the response buffer + * + * Make a request to the Resource Manager and wait for reply back. For a successful + * response, the function returns the payload. The size of the payload is set in + * resp_buf_size. The resp_buf must be freed by the caller when 0 is returned + * and resp_buf_size != 0. + * + * req_buf should be not NULL for req_buf_size >0. If req_buf_size == 0, + * req_buf *can* be NULL and no additional payload is sent. + * + * Context: Process context. Will sleep waiting for reply. + * Return: 0 on success. <0 if error. + */ +int gunyah_rm_call(struct gunyah_rm *rm, u32 message_id, const void *req_buf, + size_t req_buf_size, void **resp_buf, size_t *resp_buf_size) +{ + struct gunyah_rm_message message = { 0 }; + u32 seq_id; + int ret; + + /* message_id 0 is reserved. req_buf_size implies req_buf is not NULL */ + if (!rm || !message_id || (!req_buf && req_buf_size)) + return -EINVAL; + + message.type = RM_RPC_TYPE_REPLY; + message.msg_id = message_id; + + message.reply.seq_done = + COMPLETION_INITIALIZER_ONSTACK(message.reply.seq_done); + + /* Allocate a new seq number for this message */ + ret = xa_alloc_cyclic(&rm->call_xarray, &seq_id, &message, xa_limit_16b, + &rm->next_seq, GFP_KERNEL); + if (ret < 0) + return ret; + message.reply.seq = lower_16_bits(seq_id); + + /* Send the request to the Resource Manager */ + ret = gunyah_rm_send_request(rm, message_id, req_buf, req_buf_size, + &message); + if (ret < 0) { + dev_warn(rm->dev, "Failed to send request. Error: %d\n", ret); + goto out; + } + + /* + * Wait for response. Uninterruptible because rollback based on what RM did to VM + * requires us to know how RM handled the call. + */ + wait_for_completion(&message.reply.seq_done); + + /* Check for internal (kernel) error waiting for the response */ + if (message.reply.ret) { + ret = message.reply.ret; + if (ret != -ENOMEM) + kfree(message.payload); + goto out; + } + + /* Got a response, did resource manager give us an error? */ + if (message.reply.rm_error != GUNYAH_RM_ERROR_OK) { + dev_warn(rm->dev, "RM rejected message %08x. Error: %d\n", + message_id, message.reply.rm_error); + ret = gunyah_rm_error_remap(message.reply.rm_error); + kfree(message.payload); + goto out; + } + + /* Everything looks good, return the payload */ + if (resp_buf_size) + *resp_buf_size = message.size; + + if (message.size && resp_buf) { + *resp_buf = message.payload; + } else { + /* kfree in case RM sent us multiple fragments but never any data in + * those fragments. We would've allocated memory for it, but message.size == 0 + */ + kfree(message.payload); + } + +out: + xa_erase(&rm->call_xarray, message.reply.seq); + return ret; +} + +int gunyah_rm_notifier_register(struct gunyah_rm *rm, struct notifier_block *nb) +{ + return blocking_notifier_chain_register(&rm->nh, nb); +} +EXPORT_SYMBOL_GPL(gunyah_rm_notifier_register); + +int gunyah_rm_notifier_unregister(struct gunyah_rm *rm, + struct notifier_block *nb) +{ + return blocking_notifier_chain_unregister(&rm->nh, nb); +} +EXPORT_SYMBOL_GPL(gunyah_rm_notifier_unregister); + +static int gunyah_platform_probe_capability(struct platform_device *pdev, + int idx, + struct gunyah_resource *ghrsc) +{ + int ret; + + ghrsc->irq = platform_get_irq(pdev, idx); + if (ghrsc->irq < 0) { + dev_err(&pdev->dev, "Failed to get %s irq: %d\n", + idx ? "rx" : "tx", ghrsc->irq); + return ghrsc->irq; + } + + ret = of_property_read_u64_index(pdev->dev.of_node, "reg", idx, + &ghrsc->capid); + if (ret) { + dev_err(&pdev->dev, "Failed to get %s capid: %d\n", + idx ? "rx" : "tx", ret); + return ret; + } + + return 0; +} + +static int gunyah_rm_probe_tx_msgq(struct gunyah_rm *rm, + struct platform_device *pdev) +{ + int ret; + + rm->tx_ghrsc.type = GUNYAH_RESOURCE_TYPE_MSGQ_TX; + ret = gunyah_platform_probe_capability(pdev, 0, &rm->tx_ghrsc); + if (ret) + return ret; + + enable_irq_wake(rm->tx_ghrsc.irq); + + return devm_request_threaded_irq(rm->dev, rm->tx_ghrsc.irq, NULL, + gunyah_rm_tx, IRQF_ONESHOT, + "gunyah_rm_tx", rm); +} + +static int gunyah_rm_probe_rx_msgq(struct gunyah_rm *rm, + struct platform_device *pdev) +{ + int ret; + + rm->rx_ghrsc.type = GUNYAH_RESOURCE_TYPE_MSGQ_RX; + ret = gunyah_platform_probe_capability(pdev, 1, &rm->rx_ghrsc); + if (ret) + return ret; + + enable_irq_wake(rm->rx_ghrsc.irq); + + return devm_request_threaded_irq(rm->dev, rm->rx_ghrsc.irq, NULL, + gunyah_rm_rx, IRQF_ONESHOT, + "gunyah_rm_rx", rm); +} + +static int gunyah_rm_probe(struct platform_device *pdev) +{ + struct gunyah_rm *rm; + int ret; + + rm = devm_kzalloc(&pdev->dev, sizeof(*rm), GFP_KERNEL); + if (!rm) + return -ENOMEM; + + platform_set_drvdata(pdev, rm); + rm->dev = &pdev->dev; + + mutex_init(&rm->send_lock); + init_completion(&rm->send_ready); + BLOCKING_INIT_NOTIFIER_HEAD(&rm->nh); + xa_init_flags(&rm->call_xarray, XA_FLAGS_ALLOC); + + ret = gunyah_rm_probe_tx_msgq(rm, pdev); + if (ret) + return ret; + /* assume RM is ready to receive messages from us */ + complete_all(&rm->send_ready); + + ret = gunyah_rm_probe_rx_msgq(rm, pdev); + if (ret) + return ret; + + return 0; +} + +static const struct of_device_id gunyah_rm_of_match[] = { + { .compatible = "gunyah-resource-manager" }, + {} +}; +MODULE_DEVICE_TABLE(of, gunyah_rm_of_match); + +static struct platform_driver gunyah_rm_driver = { + .probe = gunyah_rm_probe, + .driver = { + .name = "gunyah_rsc_mgr", + .of_match_table = gunyah_rm_of_match, + }, +}; +module_platform_driver(gunyah_rm_driver); + +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("Gunyah Resource Manager Driver"); diff --git a/drivers/virt/gunyah/rsc_mgr.h b/drivers/virt/gunyah/rsc_mgr.h new file mode 100644 index 000000000000..4ce6ec3668d4 --- /dev/null +++ b/drivers/virt/gunyah/rsc_mgr.h @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ +#ifndef __GUNYAH_RSC_MGR_PRIV_H +#define __GUNYAH_RSC_MGR_PRIV_H + +#include +#include +#include + +struct gunyah_rm; + +int gunyah_rm_call(struct gunyah_rm *rsc_mgr, u32 message_id, + const void *req_buf, size_t req_buf_size, void **resp_buf, + size_t *resp_buf_size); + +#endif diff --git a/include/linux/gunyah_rsc_mgr.h b/include/linux/gunyah_rsc_mgr.h new file mode 100644 index 000000000000..e151f66be96e --- /dev/null +++ b/include/linux/gunyah_rsc_mgr.h @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _GUNYAH_RSC_MGR_H +#define _GUNYAH_RSC_MGR_H + +#include +#include + +#define GUNYAH_VMID_INVAL U16_MAX + +struct gunyah_rm; +int gunyah_rm_notifier_register(struct gunyah_rm *rm, + struct notifier_block *nb); +int gunyah_rm_notifier_unregister(struct gunyah_rm *rm, + struct notifier_block *nb); +struct device *gunyah_rm_get(struct gunyah_rm *rm); +void gunyah_rm_put(struct gunyah_rm *rm); + +#endif From patchwork Sat Dec 16 00:20:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 179688 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:3b04:b0:fb:cd0c:d3e with SMTP id c4csp9666468dys; Fri, 15 Dec 2023 16:25:42 -0800 (PST) X-Google-Smtp-Source: AGHT+IEl2zcn8Wa4DkSVsURHlJVKhYiuqbvMT6QlpFJAuvDPtQEqw2lZqY0qOBRcCpsG6M5S0PY2 X-Received: by 2002:a05:6a20:e11c:b0:193:fcfb:1b39 with SMTP id kr28-20020a056a20e11c00b00193fcfb1b39mr1393676pzb.114.1702686342435; Fri, 15 Dec 2023 16:25:42 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702686342; cv=none; d=google.com; s=arc-20160816; b=oYLHnAk7ObCJUNNnRh9TEvm3ZtHl2AKc+xA1fzQTOfGf6w8xg80CL8e9nkVzxCOxzD 04R1rrysmTZyDYl+bMs2gNR3+FHOj2pFsRgxw5aISZ+4c+k2T1VaApfJP9EAwxNeSTh6 +0GDyQWXXAyORZnyEgHpvo7wwDpJtX+xoppYDqzI2bUT6+MYGAnCVTlmFv0LDor0Ud7J dQEiQeMf/yDo+dI1Tl6/cdFpogYI6l5GbUzWhME6JgTlkVwoqFtx13oz6T3Qe6k8tZoN Jg2UKzDCEY809gf0S3kxLRDzjelVUKtvju1ddc16AE5+69jwAv7tx1B3GnmhKK8ccWYw btbQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:list-unsubscribe:list-subscribe:list-id:precedence :subject:date:from:dkim-signature; bh=097Lg5UbfQcA28si9W7NPg/edYUMo3HojM0j6uxPB9A=; fh=Pv/sMsQia5wmqAOYHIQUkkUFpDKhZjuODj71Ojb2ybw=; b=zTpX0zH0RzY2IASC2p1Iq4Bo3aWra8RB2zT8e3ih0kETlKL+i7NXs6TSz6Cje1pHqb zrqJg1wciTz1wa909Npdv+EcDYK1PCsjDIszG9PtulVaX2hbTQMfQ5k+QdesJzopUw5c gmSzwv+yqRdsxotTAE0h9FiauKn/speMGzxJhwiV8kdN4eZ1fn3IrArpvn4/nEOuUyaD w7MZaM0ogdqLl04yAePw62MZPEowggu56B/68FDNyl32iy3XBTgXj+Jmfdq7S7JI4dV8 JBfiPQi0B4Jqd0rzzc9WwHcVMKLqlpqPZ9aTh2sXbWSTn0FG38x92Y+AhPj+bnVR1kWP tvVA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=kat3yxwR; spf=pass (google.com: domain of linux-kernel+bounces-1878-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1878-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id k190-20020a633dc7000000b005cd786d8e3bsi354583pga.249.2023.12.15.16.25.42 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Dec 2023 16:25:42 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-1878-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=kat3yxwR; spf=pass (google.com: domain of linux-kernel+bounces-1878-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1878-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 2FD492847F8 for ; Sat, 16 Dec 2023 00:25:42 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 56A9E1E484; Sat, 16 Dec 2023 00:21:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="kat3yxwR" X-Original-To: linux-kernel@vger.kernel.org Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 841D78478; Sat, 16 Dec 2023 00:21:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=quicinc.com Received: from pps.filterd (m0279871.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BFNwAFE004538; Sat, 16 Dec 2023 00:21:07 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:date:subject:mime-version:content-type :content-transfer-encoding:message-id:references:in-reply-to:to :cc; s=qcppdkim1; bh=097Lg5UbfQcA28si9W7NPg/edYUMo3HojM0j6uxPB9A =; b=kat3yxwRpOQkMdG39Jm4DuN9YrLwkX4DiquBeSy72/Wq4si6Y51jJiqdQx2 UKlboxYt5MGfPMiCiDXFL4qHCSAYpx8IoXojH4HVkjHzD+Wo8tWFLppF4mmY8YoZ FQ7f2auyfN6eZc+x62f6zy+Jo7TR+WXr+qBPcmpZdQ/1EUVL1Du7FBgzMHhD0qTL ciTwei/CNNkaTza1FvtsfuGMDulb/eylDp16CyzQde6MKikc/YdoeZXPDYtOPzD2 7YTIY/U719/E9HmG3VdW3zwE1PF9dK123Gpud/pfDchtWXSWXHuq3mj0C75fZrvZ xuG31OgvAXxPDI5wz6kXdMqXoqQ== Received: from nasanppmta01.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v0hb1te8h-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:06 +0000 (GMT) Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 3BG0L5vG015533 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:05 GMT Received: from [169.254.0.1] (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.40; Fri, 15 Dec 2023 16:21:04 -0800 From: Elliot Berman Date: Fri, 15 Dec 2023 16:20:49 -0800 Subject: [PATCH RFC v15 08/30] gunyah: rsc_mgr: Add VM lifecycle RPC Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <20231215-gunyah-v15-8-192a5d872a30@quicinc.com> References: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> In-Reply-To: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> To: Alex Elder , Srinivas Kandagatla , Murali Nalajal , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Philip Derrin , Prakruthi Deepak Heragu , Jonathan Corbet , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Catalin Marinas , Will Deacon , Konrad Dybcio , Bjorn Andersson , Dmitry Baryshkov , "Fuad Tabba" , Sean Christopherson CC: , , , , , Elliot Berman X-Mailer: b4 0.13-dev X-ClientProxiedBy: nalasex01a.na.qualcomm.com (10.47.209.196) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: bFBq5_zDxgzvm16wmXcgmF-2zqnatjci X-Proofpoint-ORIG-GUID: bFBq5_zDxgzvm16wmXcgmF-2zqnatjci X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 suspectscore=0 lowpriorityscore=0 phishscore=0 malwarescore=0 mlxlogscore=999 spamscore=0 adultscore=0 priorityscore=1501 bulkscore=0 impostorscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312160000 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785396034265504384 X-GMAIL-MSGID: 1785396034265504384 Add Gunyah Resource Manager RPC to launch an unauthenticated VM. Signed-off-by: Elliot Berman --- drivers/virt/gunyah/Makefile | 2 +- drivers/virt/gunyah/rsc_mgr_rpc.c | 570 ++++++++++++++++++++++++++++++++++++++ include/linux/gunyah_rsc_mgr.h | 144 ++++++++++ 3 files changed, 715 insertions(+), 1 deletion(-) diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index c2308389f551..b1bdf3e84155 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -1,5 +1,5 @@ # SPDX-License-Identifier: GPL-2.0 -gunyah_rsc_mgr-y += rsc_mgr.o +gunyah_rsc_mgr-y += rsc_mgr.o rsc_mgr_rpc.o obj-$(CONFIG_GUNYAH) += gunyah.o gunyah_rsc_mgr.o diff --git a/drivers/virt/gunyah/rsc_mgr_rpc.c b/drivers/virt/gunyah/rsc_mgr_rpc.c new file mode 100644 index 000000000000..9954512dbdde --- /dev/null +++ b/drivers/virt/gunyah/rsc_mgr_rpc.c @@ -0,0 +1,570 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include +#include "rsc_mgr.h" + +/* Message IDs: Memory Management */ +#define GUNYAH_RM_RPC_MEM_LEND 0x51000012 +#define GUNYAH_RM_RPC_MEM_SHARE 0x51000013 +#define GUNYAH_RM_RPC_MEM_RECLAIM 0x51000015 +#define GUNYAH_RM_RPC_MEM_APPEND 0x51000018 + +/* Message IDs: VM Management */ +/* clang-format off */ +#define GUNYAH_RM_RPC_VM_ALLOC_VMID 0x56000001 +#define GUNYAH_RM_RPC_VM_DEALLOC_VMID 0x56000002 +#define GUNYAH_RM_RPC_VM_START 0x56000004 +#define GUNYAH_RM_RPC_VM_STOP 0x56000005 +#define GUNYAH_RM_RPC_VM_RESET 0x56000006 +#define GUNYAH_RM_RPC_VM_CONFIG_IMAGE 0x56000009 +#define GUNYAH_RM_RPC_VM_INIT 0x5600000B +#define GUNYAH_RM_RPC_VM_SET_BOOT_CONTEXT 0x5600000C +#define GUNYAH_RM_RPC_VM_GET_HYP_RESOURCES 0x56000020 +#define GUNYAH_RM_RPC_VM_GET_VMID 0x56000024 +#define GUNYAH_RM_RPC_VM_SET_DEMAND_PAGING 0x56000033 +#define GUNYAH_RM_RPC_VM_SET_ADDRESS_LAYOUT 0x56000034 +/* clang-format on */ + +/* Message IDs: Miscellaneous */ +/* clang-format off */ +#define GUNYAH_RM_RPC_GET_LOG_ADDRESS 0x56000100 +/* clang-format on */ + +struct gunyah_rm_vm_common_vmid_req { + __le16 vmid; + __le16 _padding; +} __packed; + +/* Call: MEM_LEND, MEM_SHARE */ +#define GUNYAH_MEM_SHARE_REQ_FLAGS_APPEND BIT(1) + +struct gunyah_rm_mem_share_req_header { + u8 mem_type; + u8 _padding0; + u8 flags; + u8 _padding1; + __le32 label; +} __packed; + +struct gunyah_rm_mem_share_req_acl_section { + __le32 n_entries; + struct gunyah_rm_mem_acl_entry entries[]; +} __packed; + +struct gunyah_rm_mem_share_req_mem_section { + __le16 n_entries; + __le16 _padding; + struct gunyah_rm_mem_entry entries[]; +} __packed; + +/* Call: MEM_RELEASE */ +struct gunyah_rm_mem_release_req { + __le32 mem_handle; + u8 flags; /* currently not used */ + u8 _padding0; + __le16 _padding1; +} __packed; + +/* Call: MEM_APPEND */ +#define GUNYAH_MEM_APPEND_REQ_FLAGS_END BIT(0) + +struct gunyah_rm_mem_append_req_header { + __le32 mem_handle; + u8 flags; + u8 _padding0; + __le16 _padding1; +} __packed; + +/* Call: VM_ALLOC */ +struct gunyah_rm_vm_alloc_vmid_resp { + __le16 vmid; + __le16 _padding; +} __packed; + +/* Call: VM_STOP */ +#define GUNYAH_RM_VM_STOP_FLAG_FORCE_STOP BIT(0) + +#define GUNYAH_RM_VM_STOP_REASON_FORCE_STOP 3 + +struct gunyah_rm_vm_stop_req { + __le16 vmid; + u8 flags; + u8 _padding; + __le32 stop_reason; +} __packed; + +/* Call: VM_CONFIG_IMAGE */ +struct gunyah_rm_vm_config_image_req { + __le16 vmid; + __le16 auth_mech; + __le32 mem_handle; + __le64 image_offset; + __le64 image_size; + __le64 dtb_offset; + __le64 dtb_size; +} __packed; + +#define GUNYAH_RM_MAX_MEM_ENTRIES 512 + +/* Call: VM_SET_BOOT_CONTEXT */ +struct gunyah_rm_vm_set_boot_context_req { + __le16 vmid; + u8 reg_set; + u8 reg_index; + __le32 _padding; + __le64 value; +} __packed; + +/* Call: VM_SET_DEMAND_PAGING */ +struct gunyah_rm_vm_set_demand_paging_req { + __le16 vmid; + __le16 _padding; + __le32 range_count; + DECLARE_FLEX_ARRAY(struct gunyah_rm_mem_entry, ranges); +} __packed; + +/* Call: VM_SET_ADDRESS_LAYOUT */ +struct gunyah_rm_vm_set_address_layout_req { + __le16 vmid; + __le16 _padding; + __le32 range_id; + __le64 range_base; + __le64 range_size; +} __packed; + +/* + * Several RM calls take only a VMID as a parameter and give only standard + * response back. Deduplicate boilerplate code by using this common call. + */ +static int gunyah_rm_common_vmid_call(struct gunyah_rm *rm, u32 message_id, + u16 vmid) +{ + struct gunyah_rm_vm_common_vmid_req req_payload = { + .vmid = cpu_to_le16(vmid), + }; + + return gunyah_rm_call(rm, message_id, &req_payload, sizeof(req_payload), + NULL, NULL); +} + +static int gunyah_rm_mem_append(struct gunyah_rm *rm, u32 mem_handle, + struct gunyah_rm_mem_entry *entries, + size_t n_entries) +{ + struct gunyah_rm_mem_append_req_header *req __free(kfree) = NULL; + struct gunyah_rm_mem_share_req_mem_section *mem; + int ret = 0; + size_t n; + + req = kzalloc(sizeof(*req) + struct_size(mem, entries, GUNYAH_RM_MAX_MEM_ENTRIES), + GFP_KERNEL); + if (!req) + return -ENOMEM; + + req->mem_handle = cpu_to_le32(mem_handle); + mem = (void *)(req + 1); + + while (n_entries) { + req->flags = 0; + if (n_entries > GUNYAH_RM_MAX_MEM_ENTRIES) { + n = GUNYAH_RM_MAX_MEM_ENTRIES; + } else { + req->flags |= GUNYAH_MEM_APPEND_REQ_FLAGS_END; + n = n_entries; + } + + mem->n_entries = cpu_to_le16(n); + memcpy(mem->entries, entries, sizeof(*entries) * n); + + ret = gunyah_rm_call(rm, GUNYAH_RM_RPC_MEM_APPEND, req, + sizeof(*req) + struct_size(mem, entries, n), + NULL, NULL); + if (ret) + break; + + entries += n; + n_entries -= n; + } + + return ret; +} + +/** + * gunyah_rm_mem_share() - Share memory with other virtual machines. + * @rm: Handle to a Gunyah resource manager + * @p: Information about the memory to be shared. + * + * Sharing keeps Linux's access to the memory while the memory parcel is shared. + */ +int gunyah_rm_mem_share(struct gunyah_rm *rm, struct gunyah_rm_mem_parcel *p) +{ + u32 message_id = p->n_acl_entries == 1 ? GUNYAH_RM_RPC_MEM_LEND : + GUNYAH_RM_RPC_MEM_SHARE; + size_t msg_size, initial_mem_entries = p->n_mem_entries, resp_size; + struct gunyah_rm_mem_share_req_acl_section *acl; + struct gunyah_rm_mem_share_req_mem_section *mem; + struct gunyah_rm_mem_share_req_header *req_header; + size_t acl_size, mem_size; + u32 *attr_section; + bool need_append = false; + __le32 *resp; + void *msg; + int ret; + + if (!p->acl_entries || !p->n_acl_entries || !p->mem_entries || + !p->n_mem_entries || p->n_acl_entries > U8_MAX || + p->mem_handle != GUNYAH_MEM_HANDLE_INVAL) + return -EINVAL; + + if (initial_mem_entries > GUNYAH_RM_MAX_MEM_ENTRIES) { + initial_mem_entries = GUNYAH_RM_MAX_MEM_ENTRIES; + need_append = true; + } + + acl_size = struct_size(acl, entries, p->n_acl_entries); + mem_size = struct_size(mem, entries, initial_mem_entries); + + /* The format of the message goes: + * request header + * ACL entries (which VMs get what kind of access to this memory parcel) + * Memory entries (list of memory regions to share) + * Memory attributes (currently unused, we'll hard-code the size to 0) + */ + msg_size = sizeof(struct gunyah_rm_mem_share_req_header) + acl_size + + mem_size + + sizeof(u32); /* for memory attributes, currently unused */ + + msg = kzalloc(msg_size, GFP_KERNEL); + if (!msg) + return -ENOMEM; + + req_header = msg; + acl = (void *)req_header + sizeof(*req_header); + mem = (void *)acl + acl_size; + attr_section = (void *)mem + mem_size; + + req_header->mem_type = p->mem_type; + if (need_append) + req_header->flags |= GUNYAH_MEM_SHARE_REQ_FLAGS_APPEND; + req_header->label = cpu_to_le32(p->label); + + acl->n_entries = cpu_to_le32(p->n_acl_entries); + memcpy(acl->entries, p->acl_entries, + flex_array_size(acl, entries, p->n_acl_entries)); + + mem->n_entries = cpu_to_le16(initial_mem_entries); + memcpy(mem->entries, p->mem_entries, + flex_array_size(mem, entries, initial_mem_entries)); + + /* Set n_entries for memory attribute section to 0 */ + *attr_section = 0; + + ret = gunyah_rm_call(rm, message_id, msg, msg_size, (void **)&resp, + &resp_size); + kfree(msg); + + if (ret) + return ret; + + p->mem_handle = le32_to_cpu(*resp); + kfree(resp); + + if (need_append) { + ret = gunyah_rm_mem_append( + rm, p->mem_handle, &p->mem_entries[initial_mem_entries], + p->n_mem_entries - initial_mem_entries); + if (ret) { + gunyah_rm_mem_reclaim(rm, p); + p->mem_handle = GUNYAH_MEM_HANDLE_INVAL; + } + } + + return ret; +} + +/** + * gunyah_rm_mem_reclaim() - Reclaim a memory parcel + * @rm: Handle to a Gunyah resource manager + * @parcel: Information about the memory to be reclaimed. + * + * RM maps the associated memory back into the stage-2 page tables of the owner VM. + */ +int gunyah_rm_mem_reclaim(struct gunyah_rm *rm, + struct gunyah_rm_mem_parcel *parcel) +{ + struct gunyah_rm_mem_release_req req = { + .mem_handle = cpu_to_le32(parcel->mem_handle), + }; + + return gunyah_rm_call(rm, GUNYAH_RM_RPC_MEM_RECLAIM, &req, sizeof(req), + NULL, NULL); +} + +/** + * gunyah_rm_alloc_vmid() - Allocate a new VM in Gunyah. Returns the VM identifier. + * @rm: Handle to a Gunyah resource manager + * @vmid: Use 0 to dynamically allocate a VM. A reserved VMID can be supplied + * to request allocation of a platform-defined VM. + * + * Return: the allocated VMID or negative value on error + */ +int gunyah_rm_alloc_vmid(struct gunyah_rm *rm, u16 vmid) +{ + struct gunyah_rm_vm_common_vmid_req req_payload = { + .vmid = cpu_to_le16(vmid), + }; + struct gunyah_rm_vm_alloc_vmid_resp *resp_payload; + size_t resp_size; + void *resp; + int ret; + + ret = gunyah_rm_call(rm, GUNYAH_RM_RPC_VM_ALLOC_VMID, &req_payload, + sizeof(req_payload), &resp, &resp_size); + if (ret) + return ret; + + if (!vmid) { + resp_payload = resp; + ret = le16_to_cpu(resp_payload->vmid); + kfree(resp); + } + + return ret; +} + +/** + * gunyah_rm_dealloc_vmid() - Dispose of a VMID + * @rm: Handle to a Gunyah resource manager + * @vmid: VM identifier allocated with gunyah_rm_alloc_vmid + */ +int gunyah_rm_dealloc_vmid(struct gunyah_rm *rm, u16 vmid) +{ + return gunyah_rm_common_vmid_call(rm, GUNYAH_RM_RPC_VM_DEALLOC_VMID, + vmid); +} + +/** + * gunyah_rm_vm_reset() - Reset a VM's resources + * @rm: Handle to a Gunyah resource manager + * @vmid: VM identifier allocated with gunyah_rm_alloc_vmid + * + * As part of tearing down the VM, request RM to clean up all the VM resources + * associated with the VM. Only after this, Linux can clean up all the + * references it maintains to resources. + */ +int gunyah_rm_vm_reset(struct gunyah_rm *rm, u16 vmid) +{ + return gunyah_rm_common_vmid_call(rm, GUNYAH_RM_RPC_VM_RESET, vmid); +} + +/** + * gunyah_rm_vm_start() - Move a VM into "ready to run" state + * @rm: Handle to a Gunyah resource manager + * @vmid: VM identifier allocated with gunyah_rm_alloc_vmid + * + * On VMs which use proxy scheduling, vcpu_run is needed to actually run the VM. + * On VMs which use Gunyah's scheduling, the vCPUs start executing in accordance with Gunyah + * scheduling policies. + */ +int gunyah_rm_vm_start(struct gunyah_rm *rm, u16 vmid) +{ + return gunyah_rm_common_vmid_call(rm, GUNYAH_RM_RPC_VM_START, vmid); +} + +/** + * gunyah_rm_vm_stop() - Send a request to Resource Manager VM to forcibly stop a VM. + * @rm: Handle to a Gunyah resource manager + * @vmid: VM identifier allocated with gunyah_rm_alloc_vmid + */ +int gunyah_rm_vm_stop(struct gunyah_rm *rm, u16 vmid) +{ + struct gunyah_rm_vm_stop_req req_payload = { + .vmid = cpu_to_le16(vmid), + .flags = GUNYAH_RM_VM_STOP_FLAG_FORCE_STOP, + .stop_reason = cpu_to_le32(GUNYAH_RM_VM_STOP_REASON_FORCE_STOP), + }; + + return gunyah_rm_call(rm, GUNYAH_RM_RPC_VM_STOP, &req_payload, + sizeof(req_payload), NULL, NULL); +} + +/** + * gunyah_rm_vm_configure() - Prepare a VM to start and provide the common + * configuration needed by RM to configure a VM + * @rm: Handle to a Gunyah resource manager + * @vmid: VM identifier allocated with gunyah_rm_alloc_vmid + * @auth_mechanism: Authentication mechanism used by resource manager to verify + * the virtual machine + * @mem_handle: Handle to a previously shared memparcel that contains all parts + * of the VM image subject to authentication. + * @image_offset: Start address of VM image, relative to the start of memparcel + * @image_size: Size of the VM image + * @dtb_offset: Start address of the devicetree binary with VM configuration, + * relative to start of memparcel. + * @dtb_size: Maximum size of devicetree binary. + */ +int gunyah_rm_vm_configure(struct gunyah_rm *rm, u16 vmid, + enum gunyah_rm_vm_auth_mechanism auth_mechanism, + u32 mem_handle, u64 image_offset, u64 image_size, + u64 dtb_offset, u64 dtb_size) +{ + struct gunyah_rm_vm_config_image_req req_payload = { + .vmid = cpu_to_le16(vmid), + .auth_mech = cpu_to_le16(auth_mechanism), + .mem_handle = cpu_to_le32(mem_handle), + .image_offset = cpu_to_le64(image_offset), + .image_size = cpu_to_le64(image_size), + .dtb_offset = cpu_to_le64(dtb_offset), + .dtb_size = cpu_to_le64(dtb_size), + }; + + return gunyah_rm_call(rm, GUNYAH_RM_RPC_VM_CONFIG_IMAGE, &req_payload, + sizeof(req_payload), NULL, NULL); +} + +/** + * gunyah_rm_vm_init() - Move the VM to initialized state. + * @rm: Handle to a Gunyah resource manager + * @vmid: VM identifier + * + * RM will allocate needed resources for the VM. + */ +int gunyah_rm_vm_init(struct gunyah_rm *rm, u16 vmid) +{ + return gunyah_rm_common_vmid_call(rm, GUNYAH_RM_RPC_VM_INIT, vmid); +} + +int gunyah_rm_vm_set_boot_context(struct gunyah_rm *rm, u16 vmid, u8 reg_set, + u8 reg_index, u64 value) +{ + struct gunyah_rm_vm_set_boot_context_req req_payload = { + .vmid = cpu_to_le16(vmid), + .reg_set = reg_set, + .reg_index = reg_index, + .value = cpu_to_le64(value), + }; + + return gunyah_rm_call(rm, GUNYAH_RM_RPC_VM_SET_BOOT_CONTEXT, + &req_payload, sizeof(req_payload), NULL, NULL); +} + +/** + * gunyah_rm_get_hyp_resources() - Retrieve hypervisor resources (capabilities) associated with a VM + * @rm: Handle to a Gunyah resource manager + * @vmid: VMID of the other VM to get the resources of + * @resources: Set by gunyah_rm_get_hyp_resources and contains the returned hypervisor resources. + * Caller must free the resources pointer if successful. + */ +int gunyah_rm_get_hyp_resources(struct gunyah_rm *rm, u16 vmid, + struct gunyah_rm_hyp_resources **resources) +{ + struct gunyah_rm_vm_common_vmid_req req_payload = { + .vmid = cpu_to_le16(vmid), + }; + struct gunyah_rm_hyp_resources *resp; + size_t resp_size; + int ret; + + ret = gunyah_rm_call(rm, GUNYAH_RM_RPC_VM_GET_HYP_RESOURCES, + &req_payload, sizeof(req_payload), (void **)&resp, + &resp_size); + if (ret) + return ret; + + if (!resp_size) + return -EBADMSG; + + if (resp_size < struct_size(resp, entries, 0) || + resp_size != + struct_size(resp, entries, le32_to_cpu(resp->n_entries))) { + kfree(resp); + return -EBADMSG; + } + + *resources = resp; + return 0; +} + +/** + * gunyah_rm_get_vmid() - Retrieve VMID of this virtual machine + * @rm: Handle to a Gunyah resource manager + * @vmid: Filled with the VMID of this VM + */ +int gunyah_rm_get_vmid(struct gunyah_rm *rm, u16 *vmid) +{ + static u16 cached_vmid = GUNYAH_VMID_INVAL; + size_t resp_size; + __le32 *resp; + int ret; + + if (cached_vmid != GUNYAH_VMID_INVAL) { + *vmid = cached_vmid; + return 0; + } + + ret = gunyah_rm_call(rm, GUNYAH_RM_RPC_VM_GET_VMID, NULL, 0, + (void **)&resp, &resp_size); + if (ret) + return ret; + + *vmid = cached_vmid = lower_16_bits(le32_to_cpu(*resp)); + kfree(resp); + + return ret; +} +EXPORT_SYMBOL_GPL(gunyah_rm_get_vmid); + +/** + * gunyah_rm_vm_set_demand_paging() - Enable demand paging of memory regions + * @rm: Handle to a Gunyah resource manager + * @vmid: VMID of the other VM + * @count: Number of demand paged memory regions + * @entries: Array of the regions + */ +int gunyah_rm_vm_set_demand_paging(struct gunyah_rm *rm, u16 vmid, u32 count, + struct gunyah_rm_mem_entry *entries) +{ + struct gunyah_rm_vm_set_demand_paging_req *req __free(kfree) = NULL; + size_t req_size; + + req_size = struct_size(req, ranges, count); + if (req_size == SIZE_MAX) + return -EINVAL; + + req = kzalloc(req_size, GFP_KERNEL); + if (!req) + return -ENOMEM; + + req->vmid = cpu_to_le16(vmid); + req->range_count = cpu_to_le32(count); + memcpy(req->ranges, entries, sizeof(*entries) * count); + + return gunyah_rm_call(rm, GUNYAH_RM_RPC_VM_SET_DEMAND_PAGING, req, + req_size, NULL, NULL); +} + +/** + * gunyah_rm_vm_set_address_layout() - Set the start address of images + * @rm: Handle to a Gunyah resource manager + * @vmid: VMID of the other VM + * @range_id: Which image to set + * @base_address: Base address + * @size: Size + */ +int gunyah_rm_vm_set_address_layout(struct gunyah_rm *rm, u16 vmid, + enum gunyah_rm_range_id range_id, + u64 base_address, u64 size) +{ + struct gunyah_rm_vm_set_address_layout_req req = { + .vmid = cpu_to_le16(vmid), + .range_id = cpu_to_le32(range_id), + .range_base = cpu_to_le64(base_address), + .range_size = cpu_to_le64(size), + }; + + return gunyah_rm_call(rm, GUNYAH_RM_RPC_VM_SET_ADDRESS_LAYOUT, &req, + sizeof(req), NULL, NULL); +} diff --git a/include/linux/gunyah_rsc_mgr.h b/include/linux/gunyah_rsc_mgr.h index e151f66be96e..7bd2bee91f3e 100644 --- a/include/linux/gunyah_rsc_mgr.h +++ b/include/linux/gunyah_rsc_mgr.h @@ -10,6 +10,7 @@ #include #define GUNYAH_VMID_INVAL U16_MAX +#define GUNYAH_MEM_HANDLE_INVAL U32_MAX struct gunyah_rm; int gunyah_rm_notifier_register(struct gunyah_rm *rm, @@ -19,4 +20,147 @@ int gunyah_rm_notifier_unregister(struct gunyah_rm *rm, struct device *gunyah_rm_get(struct gunyah_rm *rm); void gunyah_rm_put(struct gunyah_rm *rm); +struct gunyah_rm_vm_exited_payload { + __le16 vmid; + __le16 exit_type; + __le32 exit_reason_size; + u8 exit_reason[]; +} __packed; + +enum gunyah_rm_notification_id { + /* clang-format off */ + GUNYAH_RM_NOTIFICATION_VM_EXITED = 0x56100001, + GUNYAH_RM_NOTIFICATION_VM_STATUS = 0x56100008, + /* clang-format on */ +}; + +enum gunyah_rm_vm_status { + /* clang-format off */ + GUNYAH_RM_VM_STATUS_NO_STATE = 0, + GUNYAH_RM_VM_STATUS_INIT = 1, + GUNYAH_RM_VM_STATUS_READY = 2, + GUNYAH_RM_VM_STATUS_RUNNING = 3, + GUNYAH_RM_VM_STATUS_PAUSED = 4, + GUNYAH_RM_VM_STATUS_LOAD = 5, + GUNYAH_RM_VM_STATUS_AUTH = 6, + GUNYAH_RM_VM_STATUS_INIT_FAILED = 8, + GUNYAH_RM_VM_STATUS_EXITED = 9, + GUNYAH_RM_VM_STATUS_RESETTING = 10, + GUNYAH_RM_VM_STATUS_RESET = 11, + /* clang-format on */ +}; + +struct gunyah_rm_vm_status_payload { + __le16 vmid; + u16 reserved; + u8 vm_status; + u8 os_status; + __le16 app_status; +} __packed; + +#define GUNYAH_RM_ACL_X BIT(0) +#define GUNYAH_RM_ACL_W BIT(1) +#define GUNYAH_RM_ACL_R BIT(2) + +struct gunyah_rm_mem_acl_entry { + __le16 vmid; + u8 perms; + u8 reserved; +} __packed; + +struct gunyah_rm_mem_entry { + __le64 phys_addr; + __le64 size; +} __packed; + +enum gunyah_rm_mem_type { + GUNYAH_RM_MEM_TYPE_NORMAL = 0, + GUNYAH_RM_MEM_TYPE_IO = 1, +}; + +/* + * struct gunyah_rm_mem_parcel - Info about memory to be lent/shared/donated/reclaimed + * @mem_type: The type of memory: normal (DDR) or IO + * @label: An client-specified identifier which can be used by the other VMs to identify the purpose + * of the memory parcel. + * @n_acl_entries: Count of the number of entries in the @acl_entries array. + * @acl_entries: An array of access control entries. Each entry specifies a VM and what access + * is allowed for the memory parcel. + * @n_mem_entries: Count of the number of entries in the @mem_entries array. + * @mem_entries: An array of regions to be associated with the memory parcel. Addresses should be + * (intermediate) physical addresses from Linux's perspective. + * @mem_handle: On success, filled with memory handle that RM allocates for this memory parcel + */ +struct gunyah_rm_mem_parcel { + enum gunyah_rm_mem_type mem_type; + u32 label; + size_t n_acl_entries; + struct gunyah_rm_mem_acl_entry *acl_entries; + size_t n_mem_entries; + struct gunyah_rm_mem_entry *mem_entries; + u32 mem_handle; +}; + +/* RPC Calls */ +int gunyah_rm_mem_share(struct gunyah_rm *rm, + struct gunyah_rm_mem_parcel *parcel); +int gunyah_rm_mem_reclaim(struct gunyah_rm *rm, + struct gunyah_rm_mem_parcel *parcel); + +int gunyah_rm_alloc_vmid(struct gunyah_rm *rm, u16 vmid); +int gunyah_rm_dealloc_vmid(struct gunyah_rm *rm, u16 vmid); +int gunyah_rm_vm_reset(struct gunyah_rm *rm, u16 vmid); +int gunyah_rm_vm_start(struct gunyah_rm *rm, u16 vmid); +int gunyah_rm_vm_stop(struct gunyah_rm *rm, u16 vmid); + +enum gunyah_rm_vm_auth_mechanism { + /* clang-format off */ + GUNYAH_RM_VM_AUTH_NONE = 0, + GUNYAH_RM_VM_AUTH_QCOM_PIL_ELF = 1, + GUNYAH_RM_VM_AUTH_QCOM_ANDROID_PVM = 2, + /* clang-format on */ +}; + +int gunyah_rm_vm_configure(struct gunyah_rm *rm, u16 vmid, + enum gunyah_rm_vm_auth_mechanism auth_mechanism, + u32 mem_handle, u64 image_offset, u64 image_size, + u64 dtb_offset, u64 dtb_size); +int gunyah_rm_vm_init(struct gunyah_rm *rm, u16 vmid); +int gunyah_rm_vm_set_boot_context(struct gunyah_rm *rm, u16 vmid, u8 reg_set, + u8 reg_index, u64 value); + +struct gunyah_rm_hyp_resource { + u8 type; + u8 reserved; + __le16 partner_vmid; + __le32 resource_handle; + __le32 resource_label; + __le64 cap_id; + __le32 virq_handle; + __le32 virq; + __le64 base; + __le64 size; +} __packed; + +struct gunyah_rm_hyp_resources { + __le32 n_entries; + struct gunyah_rm_hyp_resource entries[]; +} __packed; + +int gunyah_rm_get_hyp_resources(struct gunyah_rm *rm, u16 vmid, + struct gunyah_rm_hyp_resources **resources); +int gunyah_rm_get_vmid(struct gunyah_rm *rm, u16 *vmid); + +int gunyah_rm_vm_set_demand_paging(struct gunyah_rm *rm, u16 vmid, u32 count, + struct gunyah_rm_mem_entry *mem_entries); + +enum gunyah_rm_range_id { + GUNYAH_RM_RANGE_ID_IMAGE = 0, + GUNYAH_RM_RANGE_ID_FIRMWARE = 1, +}; + +int gunyah_rm_vm_set_address_layout(struct gunyah_rm *rm, u16 vmid, + enum gunyah_rm_range_id range_id, + u64 base_address, u64 size); + #endif From patchwork Sat Dec 16 00:20:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 179685 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:3b04:b0:fb:cd0c:d3e with SMTP id c4csp9665769dys; Fri, 15 Dec 2023 16:23:44 -0800 (PST) X-Google-Smtp-Source: AGHT+IHqYilF2UQHWPBS1Hkd+xGmx2Ra7TCEemXV84ZPtBw4k+s123m4OYRKq9Ry1H52C+9ELnbk X-Received: by 2002:a17:906:3289:b0:a1a:57f4:e5a4 with SMTP id 9-20020a170906328900b00a1a57f4e5a4mr6832647ejw.140.1702686224519; Fri, 15 Dec 2023 16:23:44 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702686224; cv=none; d=google.com; s=arc-20160816; b=Ift9B6PosJ36z6YsaqR63N9oGogpsoDtbjNc48mzzcbWPRujxPnT+eJ941Fsfb8VLH Gy2eTKv+mNCoYDdRahXXMXpdQ3YbJS4FTEedXCv2DHqX25mUqMH6dU6YIvF6MD8rDtrf hPZ9PHGo6vF60lpXKCTaMJOZPXbtpAaNqJGTk3fulgJ0MyDd7J0qOxCytEX8kqiE4CRg Qi0xCznXjxxBHmUsiKxMC3QJ5X7rPdnGK3qBYKfYW52hTWzZupmVerbN2jNpkd29Fp9N ArpFGZNaS2uy3cvfKKLtO68e2jHZ6ytLFzKNOx7O3VSau+2v1Fj0ZHJdvSWFw52l83CT YqCQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:list-unsubscribe:list-subscribe:list-id:precedence :subject:date:from:dkim-signature; bh=UMQK/jqI+zZBCX1hsbfmt3S4BKDtl7KD+PCoeHmQrA8=; fh=Pv/sMsQia5wmqAOYHIQUkkUFpDKhZjuODj71Ojb2ybw=; b=zmDlWeXAswF98pAye4d/yKQIOElRB9LyPR5dFpuXKJcEk3hWAYxYF4iOfDYkykkZOi iHeyD8UpbPHBX+KyWmEmSPUqAkUSMdfqvx6pilSPk+bn375opHe1lltpWptzCXsl9Uc5 TsD8wPJ4uwwpXPN6/VPh3r5z0bzU8OcWUN6WKo8yIQfHO5XCI+UhM3hyRSgR1YYYfCme IpqYqXGeQ7HD8UiyN1sx49GbmI3SckIoVingToDH29P9WIDmTu6ncnETqVOHxiXMat7E B76a0v67an9xTXNVxpn1AAqVn0lm5Q7lIKt4MTyIdbDBC2ampLNRpMAyt0RsJeSljpwN 1VRA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=Ho5R1vYp; spf=pass (google.com: domain of linux-kernel+bounces-1874-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1874-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id t26-20020a1709063e5a00b00a1d0e5a3042si7498052eji.260.2023.12.15.16.23.44 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Dec 2023 16:23:44 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-1874-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=Ho5R1vYp; spf=pass (google.com: domain of linux-kernel+bounces-1874-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1874-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id EE0511F256E1 for ; Sat, 16 Dec 2023 00:23:43 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 67CBC111AD; Sat, 16 Dec 2023 00:21:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="Ho5R1vYp" X-Original-To: linux-kernel@vger.kernel.org Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 03D9C2564; Sat, 16 Dec 2023 00:21:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=quicinc.com Received: from pps.filterd (m0279867.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BG0DSPq008061; Sat, 16 Dec 2023 00:21:07 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:date:subject:mime-version:content-type :content-transfer-encoding:message-id:references:in-reply-to:to :cc; s=qcppdkim1; bh=UMQK/jqI+zZBCX1hsbfmt3S4BKDtl7KD+PCoeHmQrA8 =; b=Ho5R1vYppdUkY6bpGHzifHG49ABnTX81t3d2r5jmAvFZQeL8nOtfH/BlNLV nOFtfBKmdcdZwBLwFF6HZYAl4o/qgg2TbAmOi4ysT84N/YLF3/i4UsfZ0m8AwYTY Ohdmga8sxOlUlpahSMCfk+GTjQibi+2hb/b33yYYjZZvCGm2X/SDRoWPgvg8VtY0 QY6LdzOhwGAiRrEv8lRzudo5Ay80hzUxMXqv9MaHKZbFVta7ZF104hruFAiaNDOl WQe4XISJb312apLLlp80sdD6w3SJ79Poq0gGyL8/ZkEn392otyNYGRzNVlAhdq6y YdIE9aQ+dzVJxRBSEafrIy5rQdg== Received: from nasanppmta04.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v0k90t5ea-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:07 +0000 (GMT) Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA04.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 3BG0L67r016515 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:06 GMT Received: from [169.254.0.1] (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.40; Fri, 15 Dec 2023 16:21:05 -0800 From: Elliot Berman Date: Fri, 15 Dec 2023 16:20:50 -0800 Subject: [PATCH RFC v15 09/30] gunyah: vm_mgr: Introduce basic VM Manager Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <20231215-gunyah-v15-9-192a5d872a30@quicinc.com> References: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> In-Reply-To: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> To: Alex Elder , Srinivas Kandagatla , Murali Nalajal , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Philip Derrin , Prakruthi Deepak Heragu , Jonathan Corbet , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Catalin Marinas , Will Deacon , Konrad Dybcio , Bjorn Andersson , Dmitry Baryshkov , "Fuad Tabba" , Sean Christopherson CC: , , , , , Elliot Berman X-Mailer: b4 0.13-dev X-ClientProxiedBy: nalasex01a.na.qualcomm.com (10.47.209.196) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: vh0_MAp-vZpZcMm-i162IpOByC_-xZqr X-Proofpoint-GUID: vh0_MAp-vZpZcMm-i162IpOByC_-xZqr X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 lowpriorityscore=0 impostorscore=0 priorityscore=1501 malwarescore=0 clxscore=1015 mlxlogscore=999 adultscore=0 suspectscore=0 bulkscore=0 spamscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312160001 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785395910717815032 X-GMAIL-MSGID: 1785395910717815032 Gunyah VM manager is a kernel moduel which exposes an interface to Gunyah userspace to load, run, and interact with other Gunyah virtual machines. The interface is a character device at /dev/gunyah. Add a basic VM manager driver. Upcoming patches will add more ioctls into this driver. Reviewed-by: Srinivas Kandagatla Reviewed-by: Alex Elder Co-developed-by: Prakruthi Deepak Heragu Signed-off-by: Prakruthi Deepak Heragu Signed-off-by: Elliot Berman --- Documentation/userspace-api/ioctl/ioctl-number.rst | 1 + drivers/virt/gunyah/Makefile | 2 +- drivers/virt/gunyah/rsc_mgr.c | 51 ++++++++++++ drivers/virt/gunyah/vm_mgr.c | 94 ++++++++++++++++++++++ drivers/virt/gunyah/vm_mgr.h | 22 +++++ include/uapi/linux/gunyah.h | 23 ++++++ 6 files changed, 192 insertions(+), 1 deletion(-) diff --git a/Documentation/userspace-api/ioctl/ioctl-number.rst b/Documentation/userspace-api/ioctl/ioctl-number.rst index d8b6cb1a3636..91e01925a41b 100644 --- a/Documentation/userspace-api/ioctl/ioctl-number.rst +++ b/Documentation/userspace-api/ioctl/ioctl-number.rst @@ -137,6 +137,7 @@ Code Seq# Include File Comments 'F' DD video/sstfb.h conflict! 'G' 00-3F drivers/misc/sgi-gru/grulib.h conflict! 'G' 00-0F xen/gntalloc.h, xen/gntdev.h conflict! +'G' 00-0F linux/gunyah.h conflict! 'H' 00-7F linux/hiddev.h conflict! 'H' 00-0F linux/hidraw.h conflict! 'H' 01 linux/mei.h conflict! diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index b1bdf3e84155..47f1fae5419b 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -1,5 +1,5 @@ # SPDX-License-Identifier: GPL-2.0 -gunyah_rsc_mgr-y += rsc_mgr.o rsc_mgr_rpc.o +gunyah_rsc_mgr-y += rsc_mgr.o rsc_mgr_rpc.o vm_mgr.o obj-$(CONFIG_GUNYAH) += gunyah.o gunyah_rsc_mgr.o diff --git a/drivers/virt/gunyah/rsc_mgr.c b/drivers/virt/gunyah/rsc_mgr.c index 2f9afe167aa6..052b3f072a85 100644 --- a/drivers/virt/gunyah/rsc_mgr.c +++ b/drivers/virt/gunyah/rsc_mgr.c @@ -15,8 +15,10 @@ #include #include #include +#include #include "rsc_mgr.h" +#include "vm_mgr.h" /* clang-format off */ #define RM_RPC_API_VERSION_MASK GENMASK(3, 0) @@ -126,6 +128,7 @@ struct gunyah_rm_message { * @send_lock: synchronization to allow only one request to be sent at a time * @send_ready: completion to send messages again * @nh: notifier chain for clients interested in RM notification messages + * @miscdev: /dev/gunyah */ struct gunyah_rm { struct device *dev; @@ -142,6 +145,8 @@ struct gunyah_rm { struct mutex send_lock; struct completion send_ready; struct blocking_notifier_head nh; + + struct miscdevice miscdev; }; /** @@ -618,6 +623,36 @@ int gunyah_rm_notifier_unregister(struct gunyah_rm *rm, } EXPORT_SYMBOL_GPL(gunyah_rm_notifier_unregister); +struct device *gunyah_rm_get(struct gunyah_rm *rm) +{ + return get_device(rm->miscdev.this_device); +} +EXPORT_SYMBOL_GPL(gunyah_rm_get); + +void gunyah_rm_put(struct gunyah_rm *rm) +{ + put_device(rm->miscdev.this_device); +} +EXPORT_SYMBOL_GPL(gunyah_rm_put); + +static long gunyah_dev_ioctl(struct file *filp, unsigned int cmd, + unsigned long arg) +{ + struct miscdevice *miscdev = filp->private_data; + struct gunyah_rm *rm = container_of(miscdev, struct gunyah_rm, miscdev); + + return gunyah_dev_vm_mgr_ioctl(rm, cmd, arg); +} + +static const struct file_operations gunyah_dev_fops = { + /* clang-format off */ + .owner = THIS_MODULE, + .unlocked_ioctl = gunyah_dev_ioctl, + .compat_ioctl = compat_ptr_ioctl, + .llseek = noop_llseek, + /* clang-format on */ +}; + static int gunyah_platform_probe_capability(struct platform_device *pdev, int idx, struct gunyah_resource *ghrsc) @@ -703,9 +738,24 @@ static int gunyah_rm_probe(struct platform_device *pdev) if (ret) return ret; + rm->miscdev.name = "gunyah"; + rm->miscdev.minor = MISC_DYNAMIC_MINOR; + rm->miscdev.fops = &gunyah_dev_fops; + + ret = misc_register(&rm->miscdev); + if (ret) + return ret; + return 0; } +static void gunyah_rm_remove(struct platform_device *pdev) +{ + struct gunyah_rm *rm = platform_get_drvdata(pdev); + + misc_deregister(&rm->miscdev); +} + static const struct of_device_id gunyah_rm_of_match[] = { { .compatible = "gunyah-resource-manager" }, {} @@ -714,6 +764,7 @@ MODULE_DEVICE_TABLE(of, gunyah_rm_of_match); static struct platform_driver gunyah_rm_driver = { .probe = gunyah_rm_probe, + .remove_new = gunyah_rm_remove, .driver = { .name = "gunyah_rsc_mgr", .of_match_table = gunyah_rm_of_match, diff --git a/drivers/virt/gunyah/vm_mgr.c b/drivers/virt/gunyah/vm_mgr.c new file mode 100644 index 000000000000..b4d44379e5ce --- /dev/null +++ b/drivers/virt/gunyah/vm_mgr.c @@ -0,0 +1,94 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#define pr_fmt(fmt) "gunyah_vm_mgr: " fmt + +#include +#include +#include +#include +#include + +#include + +#include "vm_mgr.h" + +static __must_check struct gunyah_vm *gunyah_vm_alloc(struct gunyah_rm *rm) +{ + struct gunyah_vm *ghvm; + + ghvm = kzalloc(sizeof(*ghvm), GFP_KERNEL); + if (!ghvm) + return ERR_PTR(-ENOMEM); + + ghvm->parent = gunyah_rm_get(rm); + ghvm->rm = rm; + + return ghvm; +} + +static int gunyah_vm_release(struct inode *inode, struct file *filp) +{ + struct gunyah_vm *ghvm = filp->private_data; + + gunyah_rm_put(ghvm->rm); + kfree(ghvm); + return 0; +} + +static const struct file_operations gunyah_vm_fops = { + .owner = THIS_MODULE, + .release = gunyah_vm_release, + .llseek = noop_llseek, +}; + +static long gunyah_dev_ioctl_create_vm(struct gunyah_rm *rm, unsigned long arg) +{ + struct gunyah_vm *ghvm; + struct file *file; + int fd, err; + + /* arg reserved for future use. */ + if (arg) + return -EINVAL; + + ghvm = gunyah_vm_alloc(rm); + if (IS_ERR(ghvm)) + return PTR_ERR(ghvm); + + fd = get_unused_fd_flags(O_CLOEXEC); + if (fd < 0) { + err = fd; + goto err_destroy_vm; + } + + file = anon_inode_getfile("gunyah-vm", &gunyah_vm_fops, ghvm, O_RDWR); + if (IS_ERR(file)) { + err = PTR_ERR(file); + goto err_put_fd; + } + + fd_install(fd, file); + + return fd; + +err_put_fd: + put_unused_fd(fd); +err_destroy_vm: + gunyah_rm_put(ghvm->rm); + kfree(ghvm); + return err; +} + +long gunyah_dev_vm_mgr_ioctl(struct gunyah_rm *rm, unsigned int cmd, + unsigned long arg) +{ + switch (cmd) { + case GUNYAH_CREATE_VM: + return gunyah_dev_ioctl_create_vm(rm, arg); + default: + return -ENOTTY; + } +} diff --git a/drivers/virt/gunyah/vm_mgr.h b/drivers/virt/gunyah/vm_mgr.h new file mode 100644 index 000000000000..434a510e360f --- /dev/null +++ b/drivers/virt/gunyah/vm_mgr.h @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _GUNYAH_VM_MGR_PRIV_H +#define _GUNYAH_VM_MGR_PRIV_H + +#include +#include + +#include + +long gunyah_dev_vm_mgr_ioctl(struct gunyah_rm *rm, unsigned int cmd, + unsigned long arg); + +struct gunyah_vm { + struct gunyah_rm *rm; + struct device *parent; +}; + +#endif diff --git a/include/uapi/linux/gunyah.h b/include/uapi/linux/gunyah.h new file mode 100644 index 000000000000..ac338ec4b85d --- /dev/null +++ b/include/uapi/linux/gunyah.h @@ -0,0 +1,23 @@ +/* SPDX-License-Identifier: GPL-2.0-only WITH Linux-syscall-note */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _UAPI_LINUX_GUNYAH_H +#define _UAPI_LINUX_GUNYAH_H + +/* + * Userspace interface for /dev/gunyah - gunyah based virtual machine + */ + +#include +#include + +#define GUNYAH_IOCTL_TYPE 'G' + +/* + * ioctls for /dev/gunyah fds: + */ +#define GUNYAH_CREATE_VM _IO(GUNYAH_IOCTL_TYPE, 0x0) /* Returns a Gunyah VM fd */ + +#endif From patchwork Sat Dec 16 00:20:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 179689 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:3b04:b0:fb:cd0c:d3e with SMTP id c4csp9666576dys; Fri, 15 Dec 2023 16:25:58 -0800 (PST) X-Google-Smtp-Source: AGHT+IHNpCDFjkzBa6oTiC5m7RHTOP3r4v6r/0R2KEq+qqBNZsBbSDg9R69ElE6YraRY4mZRU5I0 X-Received: by 2002:a2e:a914:0:b0:2cc:216f:54b2 with SMTP id j20-20020a2ea914000000b002cc216f54b2mr7611915ljq.101.1702686358607; Fri, 15 Dec 2023 16:25:58 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702686358; cv=none; d=google.com; s=arc-20160816; b=FYCAlzRlVj4spl1Ej6G9+icYe1Gj2F9TTttruxWS1JeYQUCL4jWf8/IU3vqB7Mrhob p4TZh/VzybzEwwuD1YNuupaBYeaM8pIvb8BAOi/ICSTv5JSFNnFu4SMArgbm/Pc6RoGs ATvv+IxVxyNuKugy1MHfq660Syv/qejfm6RG73rHKgQl86oNoQ/dQXr9+1q1Bbu0vx9g NQjamSaKmO0umf1hax6m+SKPZDXStbt/6NgEXlodywhgjm72IHGduYhPpQ2ByCUYWenD QUZgNr1UGrXHobWTrBtj6WYHkzPfIbR8N84pOl8bSwTnyB6YR/2dsd7Hrkwfqm9ng9Qp f8UA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:list-unsubscribe:list-subscribe:list-id:precedence :subject:date:from:dkim-signature; bh=JtC3tB/j5lbY9AVgjGdS9xv5kMbBIO7Z4MtW73yukyM=; fh=Pv/sMsQia5wmqAOYHIQUkkUFpDKhZjuODj71Ojb2ybw=; b=h8MTsj58OpUgYehOD6S1pqCEb6G0ZvL9LqDa8V3SgK7R+V2bUjubRwpBLSfMOHTjsr 5w6P/0I2305GZQ9yq66JWalJGkloSWPeGb7xz+XO7UrDGgmAczdngD4lj3IJHokpGOjI buj49Tttt9zsBkTJptzvPNsN1xMGTT7WOwUJ9MneH5ZCpE+fwBo3azy2pVbchjg1edLn bDhZfziMe1kUoKg6HkL4bGMK3uZaaH2B1KEnYJyc6UQ02YX6e8xOLHnnGAwR+ZKEYjXe HxTefpo72hg6Gt2HEzH9mqBiGtcK2kWzIRkg1Lz3T7ycCLlGyHTQepx2FCW9z/Rdvsvn t85w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b="mfPB/NIM"; spf=pass (google.com: domain of linux-kernel+bounces-1879-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1879-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id b16-20020a056402351000b0054cc95dad5csi8188431edd.359.2023.12.15.16.25.58 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Dec 2023 16:25:58 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-1879-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b="mfPB/NIM"; spf=pass (google.com: domain of linux-kernel+bounces-1879-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1879-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id E982D1F223BF for ; Sat, 16 Dec 2023 00:25:56 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id AC2661E4A4; Sat, 16 Dec 2023 00:21:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="mfPB/NIM" X-Original-To: linux-kernel@vger.kernel.org Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 34CB5F4F1; Sat, 16 Dec 2023 00:21:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=quicinc.com Received: from pps.filterd (m0279870.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BFNw2Zm008687; Sat, 16 Dec 2023 00:21:08 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:date:subject:mime-version:content-type :content-transfer-encoding:message-id:references:in-reply-to:to :cc; s=qcppdkim1; bh=JtC3tB/j5lbY9AVgjGdS9xv5kMbBIO7Z4MtW73yukyM =; b=mfPB/NIMdew7Is4iUrAyafOJvKCdB88RaY0aps/RprTx+fYlKjIIRQ+Bfx7 +ICAUCAduZyR4DgSb9NnVpjCA/lnGWdmdx1p6/O+LaNJ7f4zH8h4mm9QPW+PUSCS L+NMYkhW6A8b7Xyn6SQxYMrQqqkbVOQaD1xYoXiAEtdtmIOnb28PxjOejFCcxN0O I5CVFvZ+ZE68G3LMJSivXzx4WBNi7rLfq4XVdM9XlZKoWwYDSyg90qhOn8rEdpJY /cQlxWfSEtiS1+v0NYhd5nFgRXx74GYR9qlWf9mYaIkD8LhucKreEFrJ+kW4i3sT V5V9Xofvs3tqQ/CLWtnvWLXBdsg== Received: from nasanppmta04.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v0vv9ghj1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:07 +0000 (GMT) Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA04.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 3BG0L64V016525 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:06 GMT Received: from [169.254.0.1] (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.40; Fri, 15 Dec 2023 16:21:06 -0800 From: Elliot Berman Date: Fri, 15 Dec 2023 16:20:51 -0800 Subject: [PATCH RFC v15 10/30] gunyah: vm_mgr: Add ioctls to support basic non-proxy VM boot Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <20231215-gunyah-v15-10-192a5d872a30@quicinc.com> References: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> In-Reply-To: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> To: Alex Elder , Srinivas Kandagatla , Murali Nalajal , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Philip Derrin , Prakruthi Deepak Heragu , Jonathan Corbet , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Catalin Marinas , Will Deacon , Konrad Dybcio , Bjorn Andersson , Dmitry Baryshkov , "Fuad Tabba" , Sean Christopherson CC: , , , , , Elliot Berman X-Mailer: b4 0.13-dev X-ClientProxiedBy: nalasex01a.na.qualcomm.com (10.47.209.196) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: vsncKepodU3EQkR6g9-WtO_cbrJBQ1kh X-Proofpoint-ORIG-GUID: vsncKepodU3EQkR6g9-WtO_cbrJBQ1kh X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 impostorscore=0 phishscore=0 clxscore=1015 adultscore=0 malwarescore=0 priorityscore=1501 lowpriorityscore=0 suspectscore=0 mlxlogscore=999 mlxscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312160001 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785396051595870374 X-GMAIL-MSGID: 1785396051595870374 Add remaining ioctls to support non-proxy VM boot: - Gunyah Resource Manager uses the VM's devicetree to configure the virtual machine. The location of the devicetree in the guest's virtual memory can be declared via the SET_DTB_CONFIG ioctl. - Trigger start of the virtual machine with VM_START ioctl. Reviewed-by: Alex Elder Co-developed-by: Prakruthi Deepak Heragu Signed-off-by: Prakruthi Deepak Heragu Signed-off-by: Elliot Berman --- drivers/virt/gunyah/vm_mgr.c | 194 +++++++++++++++++++++++++++++++++++++++++++ drivers/virt/gunyah/vm_mgr.h | 11 +++ include/uapi/linux/gunyah.h | 5 ++ 3 files changed, 210 insertions(+) diff --git a/drivers/virt/gunyah/vm_mgr.c b/drivers/virt/gunyah/vm_mgr.c index b4d44379e5ce..1b1b26f568af 100644 --- a/drivers/virt/gunyah/vm_mgr.c +++ b/drivers/virt/gunyah/vm_mgr.c @@ -15,6 +15,68 @@ #include "vm_mgr.h" +static int gunyah_vm_rm_notification_status(struct gunyah_vm *ghvm, void *data) +{ + struct gunyah_rm_vm_status_payload *payload = data; + + if (le16_to_cpu(payload->vmid) != ghvm->vmid) + return NOTIFY_OK; + + /* All other state transitions are synchronous to a corresponding RM call */ + if (payload->vm_status == GUNYAH_RM_VM_STATUS_RESET) { + down_write(&ghvm->status_lock); + ghvm->vm_status = payload->vm_status; + up_write(&ghvm->status_lock); + wake_up(&ghvm->vm_status_wait); + } + + return NOTIFY_DONE; +} + +static int gunyah_vm_rm_notification_exited(struct gunyah_vm *ghvm, void *data) +{ + struct gunyah_rm_vm_exited_payload *payload = data; + + if (le16_to_cpu(payload->vmid) != ghvm->vmid) + return NOTIFY_OK; + + down_write(&ghvm->status_lock); + ghvm->vm_status = GUNYAH_RM_VM_STATUS_EXITED; + up_write(&ghvm->status_lock); + wake_up(&ghvm->vm_status_wait); + + return NOTIFY_DONE; +} + +static int gunyah_vm_rm_notification(struct notifier_block *nb, + unsigned long action, void *data) +{ + struct gunyah_vm *ghvm = container_of(nb, struct gunyah_vm, nb); + + switch (action) { + case GUNYAH_RM_NOTIFICATION_VM_STATUS: + return gunyah_vm_rm_notification_status(ghvm, data); + case GUNYAH_RM_NOTIFICATION_VM_EXITED: + return gunyah_vm_rm_notification_exited(ghvm, data); + default: + return NOTIFY_OK; + } +} + +static void gunyah_vm_stop(struct gunyah_vm *ghvm) +{ + int ret; + + if (ghvm->vm_status == GUNYAH_RM_VM_STATUS_RUNNING) { + ret = gunyah_rm_vm_stop(ghvm->rm, ghvm->vmid); + if (ret) + dev_warn(ghvm->parent, "Failed to stop VM: %d\n", ret); + } + + wait_event(ghvm->vm_status_wait, + ghvm->vm_status == GUNYAH_RM_VM_STATUS_EXITED); +} + static __must_check struct gunyah_vm *gunyah_vm_alloc(struct gunyah_rm *rm) { struct gunyah_vm *ghvm; @@ -24,14 +86,144 @@ static __must_check struct gunyah_vm *gunyah_vm_alloc(struct gunyah_rm *rm) return ERR_PTR(-ENOMEM); ghvm->parent = gunyah_rm_get(rm); + ghvm->vmid = GUNYAH_VMID_INVAL; ghvm->rm = rm; + init_rwsem(&ghvm->status_lock); + init_waitqueue_head(&ghvm->vm_status_wait); + ghvm->vm_status = GUNYAH_RM_VM_STATUS_NO_STATE; + return ghvm; } +static int gunyah_vm_start(struct gunyah_vm *ghvm) +{ + int ret; + + down_write(&ghvm->status_lock); + if (ghvm->vm_status != GUNYAH_RM_VM_STATUS_NO_STATE) { + up_write(&ghvm->status_lock); + return 0; + } + + ghvm->nb.notifier_call = gunyah_vm_rm_notification; + ret = gunyah_rm_notifier_register(ghvm->rm, &ghvm->nb); + if (ret) + goto err; + + ret = gunyah_rm_alloc_vmid(ghvm->rm, 0); + if (ret < 0) { + gunyah_rm_notifier_unregister(ghvm->rm, &ghvm->nb); + goto err; + } + ghvm->vmid = ret; + ghvm->vm_status = GUNYAH_RM_VM_STATUS_LOAD; + + ret = gunyah_rm_vm_configure(ghvm->rm, ghvm->vmid, ghvm->auth, 0, 0, 0, + 0, 0); + if (ret) { + dev_warn(ghvm->parent, "Failed to configure VM: %d\n", ret); + goto err; + } + + ret = gunyah_rm_vm_init(ghvm->rm, ghvm->vmid); + if (ret) { + ghvm->vm_status = GUNYAH_RM_VM_STATUS_INIT_FAILED; + dev_warn(ghvm->parent, "Failed to initialize VM: %d\n", ret); + goto err; + } + ghvm->vm_status = GUNYAH_RM_VM_STATUS_READY; + + ret = gunyah_rm_vm_start(ghvm->rm, ghvm->vmid); + if (ret) { + dev_warn(ghvm->parent, "Failed to start VM: %d\n", ret); + goto err; + } + + ghvm->vm_status = GUNYAH_RM_VM_STATUS_RUNNING; + up_write(&ghvm->status_lock); + return ret; +err: + /* gunyah_vm_free will handle releasing resources and reclaiming memory */ + up_write(&ghvm->status_lock); + return ret; +} + +static int gunyah_vm_ensure_started(struct gunyah_vm *ghvm) +{ + int ret; + + ret = down_read_interruptible(&ghvm->status_lock); + if (ret) + return ret; + + /* Unlikely because VM is typically started */ + if (unlikely(ghvm->vm_status == GUNYAH_RM_VM_STATUS_NO_STATE)) { + up_read(&ghvm->status_lock); + ret = gunyah_vm_start(ghvm); + if (ret) + return ret; + /** gunyah_vm_start() is guaranteed to bring status out of + * GUNYAH_RM_VM_STATUS_LOAD, thus infinitely recursive call is not + * possible + */ + return gunyah_vm_ensure_started(ghvm); + } + + /* Unlikely because VM is typically running */ + if (unlikely(ghvm->vm_status != GUNYAH_RM_VM_STATUS_RUNNING)) + ret = -ENODEV; + + up_read(&ghvm->status_lock); + return ret; +} + +static long gunyah_vm_ioctl(struct file *filp, unsigned int cmd, + unsigned long arg) +{ + struct gunyah_vm *ghvm = filp->private_data; + long r; + + switch (cmd) { + case GUNYAH_VM_START: { + r = gunyah_vm_ensure_started(ghvm); + break; + } + default: + r = -ENOTTY; + break; + } + + return r; +} + static int gunyah_vm_release(struct inode *inode, struct file *filp) { struct gunyah_vm *ghvm = filp->private_data; + int ret; + + if (ghvm->vm_status == GUNYAH_RM_VM_STATUS_RUNNING) + gunyah_vm_stop(ghvm); + + if (ghvm->vm_status != GUNYAH_RM_VM_STATUS_NO_STATE && + ghvm->vm_status != GUNYAH_RM_VM_STATUS_LOAD && + ghvm->vm_status != GUNYAH_RM_VM_STATUS_RESET) { + ret = gunyah_rm_vm_reset(ghvm->rm, ghvm->vmid); + if (ret) + dev_err(ghvm->parent, "Failed to reset the vm: %d\n", + ret); + wait_event(ghvm->vm_status_wait, + ghvm->vm_status == GUNYAH_RM_VM_STATUS_RESET); + } + + if (ghvm->vm_status > GUNYAH_RM_VM_STATUS_NO_STATE) { + gunyah_rm_notifier_unregister(ghvm->rm, &ghvm->nb); + + ret = gunyah_rm_dealloc_vmid(ghvm->rm, ghvm->vmid); + if (ret) + dev_warn(ghvm->parent, + "Failed to deallocate vmid: %d\n", ret); + } gunyah_rm_put(ghvm->rm); kfree(ghvm); @@ -40,6 +232,8 @@ static int gunyah_vm_release(struct inode *inode, struct file *filp) static const struct file_operations gunyah_vm_fops = { .owner = THIS_MODULE, + .unlocked_ioctl = gunyah_vm_ioctl, + .compat_ioctl = compat_ptr_ioctl, .release = gunyah_vm_release, .llseek = noop_llseek, }; diff --git a/drivers/virt/gunyah/vm_mgr.h b/drivers/virt/gunyah/vm_mgr.h index 434a510e360f..aa058722c5fe 100644 --- a/drivers/virt/gunyah/vm_mgr.h +++ b/drivers/virt/gunyah/vm_mgr.h @@ -8,6 +8,10 @@ #include #include +#include +#include +#include +#include #include @@ -15,8 +19,15 @@ long gunyah_dev_vm_mgr_ioctl(struct gunyah_rm *rm, unsigned int cmd, unsigned long arg); struct gunyah_vm { + u16 vmid; struct gunyah_rm *rm; struct device *parent; + enum gunyah_rm_vm_auth_mechanism auth; + + struct notifier_block nb; + enum gunyah_rm_vm_status vm_status; + wait_queue_head_t vm_status_wait; + struct rw_semaphore status_lock; }; #endif diff --git a/include/uapi/linux/gunyah.h b/include/uapi/linux/gunyah.h index ac338ec4b85d..31e7f79a6c39 100644 --- a/include/uapi/linux/gunyah.h +++ b/include/uapi/linux/gunyah.h @@ -20,4 +20,9 @@ */ #define GUNYAH_CREATE_VM _IO(GUNYAH_IOCTL_TYPE, 0x0) /* Returns a Gunyah VM fd */ +/* + * ioctls for gunyah-vm fds (returned by GUNYAH_CREATE_VM) + */ +#define GUNYAH_VM_START _IO(GUNYAH_IOCTL_TYPE, 0x3) + #endif From patchwork Sat Dec 16 00:20:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 179692 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:3b04:b0:fb:cd0c:d3e with SMTP id c4csp9666832dys; Fri, 15 Dec 2023 16:26:39 -0800 (PST) X-Google-Smtp-Source: AGHT+IE6hI2IuoNULEmKKlhFLZ2GBuLd6zSmUqWtCjMjanvUUIvX1umiQLzBdQCqrJDc/HOziiO0 X-Received: by 2002:a17:902:c081:b0:1cf:f868:5b8c with SMTP id j1-20020a170902c08100b001cff8685b8cmr13343348pld.8.1702686398879; Fri, 15 Dec 2023 16:26:38 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702686398; cv=none; d=google.com; s=arc-20160816; b=h18Fbkovh40+a1tF2NotdTaeGavnpSaWAtYOxRS2ZbYTp0K6pNm8rfPuv5d7naTr18 I12A1xA+H6AK0uM9iEikk8dIsKafmfcX58SWtZsy7MK9j0DjW0f0NNGhGEDTTTm6TNpw VUbI4LxcmVC1Fj7k300qeKE+Tum5rYnJ9GoicMy3q9LGMDPwz8/jJNxyhCJYKIjThyyu mZ0UL6F7ye0HqZXfYZmIKAviWjRJeOMW/9rqjM3Y3+087bvURMGtieyPNioAMPszXHSe k8pS8iDjeFBz2EmGOKViRMLtlJ2Bh4Vg+vU6/bmF2M7UEtZqSTS1GDhcoaTohW4c7zfg FLnA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:list-unsubscribe:list-subscribe:list-id:precedence :subject:date:from:dkim-signature; bh=iTaqAUcOpRnCyYGJ0l3m5/vSDTCcvFGH9tqjh544eL4=; fh=Pv/sMsQia5wmqAOYHIQUkkUFpDKhZjuODj71Ojb2ybw=; b=IT9LG2ilmFPdfv4piROLJNvtNuR3+utOec+dSqxlQ2mq+1E9yAZOVyQnKFb9GFj2AD qerjrPQzCUg3sHqbUbS9gPAnHR16tR4UwBeZ7Ur7yVxW5v74BwfOisVJG8ALaIzkBnuh WndFvAqtePfiiDXQfyvOBpnoa3U6pnbabpXMq8FNkAPmw1RmSSkkGqjSQD39UtYDqv9m YBm5GTPsRNyRXcC42I9i31ZwnOvM88t3HwcnLcWjSOzWeZgwKziJU4hsA/RnXyALNH6y knqWNl1wcM046G1AzMf84SI1ButCKyCjs2zgY5876fF9lt0KrPvQnwyPcZf5nYUN2LZ3 q1TA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=B5TWgck2; spf=pass (google.com: domain of linux-kernel+bounces-1881-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1881-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id e14-20020a17090301ce00b001d34125724asi6627497plh.591.2023.12.15.16.26.38 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Dec 2023 16:26:38 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-1881-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=B5TWgck2; spf=pass (google.com: domain of linux-kernel+bounces-1881-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1881-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 967E7285FBA for ; Sat, 16 Dec 2023 00:26:38 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 959A32E41D; Sat, 16 Dec 2023 00:21:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="B5TWgck2" X-Original-To: linux-kernel@vger.kernel.org Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B5383D286; Sat, 16 Dec 2023 00:21:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=quicinc.com Received: from pps.filterd (m0279870.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BFNVtTc032147; Sat, 16 Dec 2023 00:21:09 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:date:subject:mime-version:content-type :content-transfer-encoding:message-id:references:in-reply-to:to :cc; s=qcppdkim1; bh=iTaqAUcOpRnCyYGJ0l3m5/vSDTCcvFGH9tqjh544eL4 =; b=B5TWgck2K4jQCXDXIXtpg0n/j8Lmn6aBx9i6XSvu/Ndrr8htpVOatDgU36c SzO86s0LDTgX0Oknj9ZfrfkxDYjFmSMKCAVau4q1WE8xly/pVsb69pWgqSrex48E YMLrG37sHq1Pj+kYbBJeM3MjAqgdv2CTf+00xl9MT/jhHVW9fF3IWh72kEeKt7Ne JxdhvC5hdNj0bExWUQEAfXmH4vsRwoBqjo+iy1a19l/KamOCJtx6oXOUVgdgrkpW 04mgx1xFFj8UDHogTprUJOfyGme+1xoFKHe7/2qJVjVLNnNEOfj9GpYMAjCuFg9e vUW4PN6bolhIhCM72I73g15onEA== Received: from nasanppmta01.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v0vv9ghj5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:08 +0000 (GMT) Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 3BG0L7SM015556 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:07 GMT Received: from [169.254.0.1] (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.40; Fri, 15 Dec 2023 16:21:06 -0800 From: Elliot Berman Date: Fri, 15 Dec 2023 16:20:52 -0800 Subject: [PATCH RFC v15 11/30] gunyah: vm_mgr: Add framework for VM Functions Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <20231215-gunyah-v15-11-192a5d872a30@quicinc.com> References: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> In-Reply-To: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> To: Alex Elder , Srinivas Kandagatla , Murali Nalajal , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Philip Derrin , Prakruthi Deepak Heragu , Jonathan Corbet , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Catalin Marinas , Will Deacon , Konrad Dybcio , Bjorn Andersson , Dmitry Baryshkov , "Fuad Tabba" , Sean Christopherson CC: , , , , , Elliot Berman X-Mailer: b4 0.13-dev X-ClientProxiedBy: nalasex01a.na.qualcomm.com (10.47.209.196) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: 1nT_mBGLws1M_dRKl_RKL4-iwDUKhNXs X-Proofpoint-ORIG-GUID: 1nT_mBGLws1M_dRKl_RKL4-iwDUKhNXs X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 impostorscore=0 phishscore=0 clxscore=1015 adultscore=0 malwarescore=0 priorityscore=1501 lowpriorityscore=0 suspectscore=0 mlxlogscore=999 mlxscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312160001 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785396093212019643 X-GMAIL-MSGID: 1785396093212019643 Introduce a framework for Gunyah userspace to install VM functions. VM functions are optional interfaces to the virtual machine. vCPUs, ioeventfs, and irqfds are examples of such VM functions and are implemented in subsequent patches. A generic framework is implemented instead of individual ioctls to create vCPUs, irqfds, etc., in order to simplify the VM manager core implementation and allow dynamic loading of VM function modules. Signed-off-by: Elliot Berman --- drivers/virt/gunyah/vm_mgr.c | 217 +++++++++++++++++++++++++++++++++++++++++- drivers/virt/gunyah/vm_mgr.h | 6 ++ include/linux/gunyah_vm_mgr.h | 100 +++++++++++++++++++ include/uapi/linux/gunyah.h | 18 ++++ 4 files changed, 339 insertions(+), 2 deletions(-) diff --git a/drivers/virt/gunyah/vm_mgr.c b/drivers/virt/gunyah/vm_mgr.c index 1b1b26f568af..81bc73047e1e 100644 --- a/drivers/virt/gunyah/vm_mgr.c +++ b/drivers/virt/gunyah/vm_mgr.c @@ -6,15 +6,185 @@ #define pr_fmt(fmt) "gunyah_vm_mgr: " fmt #include +#include #include #include +#include #include #include +#include #include #include "vm_mgr.h" +static DEFINE_XARRAY(gunyah_vm_functions); + +static void gunyah_vm_put_function(struct gunyah_vm_function *fn) +{ + module_put(fn->mod); +} + +static struct gunyah_vm_function *gunyah_vm_get_function(u32 type) +{ + struct gunyah_vm_function *fn; + + fn = xa_load(&gunyah_vm_functions, type); + if (!fn) { + request_module("ghfunc:%d", type); + + fn = xa_load(&gunyah_vm_functions, type); + } + + if (!fn || !try_module_get(fn->mod)) + fn = ERR_PTR(-ENOENT); + + return fn; +} + +static void +gunyah_vm_remove_function_instance(struct gunyah_vm_function_instance *inst) + __must_hold(&inst->ghvm->fn_lock) +{ + inst->fn->unbind(inst); + list_del(&inst->vm_list); + gunyah_vm_put_function(inst->fn); + kfree(inst->argp); + kfree(inst); +} + +static void gunyah_vm_remove_functions(struct gunyah_vm *ghvm) +{ + struct gunyah_vm_function_instance *inst, *iiter; + + mutex_lock(&ghvm->fn_lock); + list_for_each_entry_safe(inst, iiter, &ghvm->functions, vm_list) { + gunyah_vm_remove_function_instance(inst); + } + mutex_unlock(&ghvm->fn_lock); +} + +static long gunyah_vm_add_function_instance(struct gunyah_vm *ghvm, + struct gunyah_fn_desc *f) +{ + struct gunyah_vm_function_instance *inst; + void __user *argp; + long r = 0; + + if (f->arg_size > GUNYAH_FN_MAX_ARG_SIZE) { + dev_err_ratelimited(ghvm->parent, "%s: arg_size > %d\n", + __func__, GUNYAH_FN_MAX_ARG_SIZE); + return -EINVAL; + } + + inst = kzalloc(sizeof(*inst), GFP_KERNEL); + if (!inst) + return -ENOMEM; + + inst->arg_size = f->arg_size; + if (inst->arg_size) { + inst->argp = kzalloc(inst->arg_size, GFP_KERNEL); + if (!inst->argp) { + r = -ENOMEM; + goto free; + } + + argp = u64_to_user_ptr(f->arg); + if (copy_from_user(inst->argp, argp, f->arg_size)) { + r = -EFAULT; + goto free_arg; + } + } + + inst->fn = gunyah_vm_get_function(f->type); + if (IS_ERR(inst->fn)) { + r = PTR_ERR(inst->fn); + goto free_arg; + } + + inst->ghvm = ghvm; + inst->rm = ghvm->rm; + + mutex_lock(&ghvm->fn_lock); + r = inst->fn->bind(inst); + if (r < 0) { + mutex_unlock(&ghvm->fn_lock); + gunyah_vm_put_function(inst->fn); + goto free_arg; + } + + list_add(&inst->vm_list, &ghvm->functions); + mutex_unlock(&ghvm->fn_lock); + + return r; +free_arg: + kfree(inst->argp); +free: + kfree(inst); + return r; +} + +static long gunyah_vm_rm_function_instance(struct gunyah_vm *ghvm, + struct gunyah_fn_desc *f) +{ + struct gunyah_vm_function_instance *inst, *iter; + void __user *user_argp; + void *argp; + long r = 0; + + r = mutex_lock_interruptible(&ghvm->fn_lock); + if (r) + return r; + + if (f->arg_size) { + argp = kzalloc(f->arg_size, GFP_KERNEL); + if (!argp) { + r = -ENOMEM; + goto out; + } + + user_argp = u64_to_user_ptr(f->arg); + if (copy_from_user(argp, user_argp, f->arg_size)) { + r = -EFAULT; + kfree(argp); + goto out; + } + + r = -ENOENT; + list_for_each_entry_safe(inst, iter, &ghvm->functions, + vm_list) { + if (inst->fn->type == f->type && + inst->fn->compare(inst, argp, f->arg_size)) { + gunyah_vm_remove_function_instance(inst); + r = 0; + } + } + + kfree(argp); + } + +out: + mutex_unlock(&ghvm->fn_lock); + return r; +} + +int gunyah_vm_function_register(struct gunyah_vm_function *fn) +{ + if (!fn->bind || !fn->unbind) + return -EINVAL; + + return xa_err(xa_store(&gunyah_vm_functions, fn->type, fn, GFP_KERNEL)); +} +EXPORT_SYMBOL_GPL(gunyah_vm_function_register); + +void gunyah_vm_function_unregister(struct gunyah_vm_function *fn) +{ + /* Expecting unregister to only come when unloading a module */ + WARN_ON(fn->mod && module_refcount(fn->mod)); + xa_erase(&gunyah_vm_functions, fn->type); +} +EXPORT_SYMBOL_GPL(gunyah_vm_function_unregister); + static int gunyah_vm_rm_notification_status(struct gunyah_vm *ghvm, void *data) { struct gunyah_rm_vm_status_payload *payload = data; @@ -91,7 +261,10 @@ static __must_check struct gunyah_vm *gunyah_vm_alloc(struct gunyah_rm *rm) init_rwsem(&ghvm->status_lock); init_waitqueue_head(&ghvm->vm_status_wait); + kref_init(&ghvm->kref); ghvm->vm_status = GUNYAH_RM_VM_STATUS_NO_STATE; + INIT_LIST_HEAD(&ghvm->functions); + mutex_init(&ghvm->fn_lock); return ghvm; } @@ -182,6 +355,7 @@ static long gunyah_vm_ioctl(struct file *filp, unsigned int cmd, unsigned long arg) { struct gunyah_vm *ghvm = filp->private_data; + void __user *argp = (void __user *)arg; long r; switch (cmd) { @@ -189,6 +363,24 @@ static long gunyah_vm_ioctl(struct file *filp, unsigned int cmd, r = gunyah_vm_ensure_started(ghvm); break; } + case GUNYAH_VM_ADD_FUNCTION: { + struct gunyah_fn_desc f; + + if (copy_from_user(&f, argp, sizeof(f))) + return -EFAULT; + + r = gunyah_vm_add_function_instance(ghvm, &f); + break; + } + case GUNYAH_VM_REMOVE_FUNCTION: { + struct gunyah_fn_desc f; + + if (copy_from_user(&f, argp, sizeof(f))) + return -EFAULT; + + r = gunyah_vm_rm_function_instance(ghvm, &f); + break; + } default: r = -ENOTTY; break; @@ -197,14 +389,22 @@ static long gunyah_vm_ioctl(struct file *filp, unsigned int cmd, return r; } -static int gunyah_vm_release(struct inode *inode, struct file *filp) +int __must_check gunyah_vm_get(struct gunyah_vm *ghvm) { - struct gunyah_vm *ghvm = filp->private_data; + return kref_get_unless_zero(&ghvm->kref); +} +EXPORT_SYMBOL_GPL(gunyah_vm_get); + +static void _gunyah_vm_put(struct kref *kref) +{ + struct gunyah_vm *ghvm = container_of(kref, struct gunyah_vm, kref); int ret; if (ghvm->vm_status == GUNYAH_RM_VM_STATUS_RUNNING) gunyah_vm_stop(ghvm); + gunyah_vm_remove_functions(ghvm); + if (ghvm->vm_status != GUNYAH_RM_VM_STATUS_NO_STATE && ghvm->vm_status != GUNYAH_RM_VM_STATUS_LOAD && ghvm->vm_status != GUNYAH_RM_VM_STATUS_RESET) { @@ -227,6 +427,19 @@ static int gunyah_vm_release(struct inode *inode, struct file *filp) gunyah_rm_put(ghvm->rm); kfree(ghvm); +} + +void gunyah_vm_put(struct gunyah_vm *ghvm) +{ + kref_put(&ghvm->kref, _gunyah_vm_put); +} +EXPORT_SYMBOL_GPL(gunyah_vm_put); + +static int gunyah_vm_release(struct inode *inode, struct file *filp) +{ + struct gunyah_vm *ghvm = filp->private_data; + + gunyah_vm_put(ghvm); return 0; } diff --git a/drivers/virt/gunyah/vm_mgr.h b/drivers/virt/gunyah/vm_mgr.h index aa058722c5fe..a9e0eaf212fd 100644 --- a/drivers/virt/gunyah/vm_mgr.h +++ b/drivers/virt/gunyah/vm_mgr.h @@ -9,6 +9,8 @@ #include #include #include +#include +#include #include #include #include @@ -28,6 +30,10 @@ struct gunyah_vm { enum gunyah_rm_vm_status vm_status; wait_queue_head_t vm_status_wait; struct rw_semaphore status_lock; + + struct kref kref; + struct mutex fn_lock; + struct list_head functions; }; #endif diff --git a/include/linux/gunyah_vm_mgr.h b/include/linux/gunyah_vm_mgr.h new file mode 100644 index 000000000000..f40e57f32fd8 --- /dev/null +++ b/include/linux/gunyah_vm_mgr.h @@ -0,0 +1,100 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _GUNYAH_VM_MGR_H +#define _GUNYAH_VM_MGR_H + +#include +#include +#include +#include +#include +#include + +#include + +struct gunyah_vm; + +int __must_check gunyah_vm_get(struct gunyah_vm *ghvm); +void gunyah_vm_put(struct gunyah_vm *ghvm); + +struct gunyah_vm_function_instance; +/** + * struct gunyah_vm_function - Represents a function type + * @type: value from &enum gunyah_fn_type + * @name: friendly name for debug purposes + * @mod: owner of the function type + * @bind: Called when a new function of this type has been allocated. + * @unbind: Called when the function instance is being destroyed. + * @compare: Compare function instance @f's argument to the provided arg. + * Return true if they are equivalent. Used on GUNYAH_VM_REMOVE_FUNCTION. + */ +struct gunyah_vm_function { + u32 type; + const char *name; + struct module *mod; + long (*bind)(struct gunyah_vm_function_instance *f); + void (*unbind)(struct gunyah_vm_function_instance *f); + bool (*compare)(const struct gunyah_vm_function_instance *f, + const void *arg, size_t size); +}; + +/** + * struct gunyah_vm_function_instance - Represents one function instance + * @arg_size: size of user argument + * @argp: pointer to user argument + * @ghvm: Pointer to VM instance + * @rm: Pointer to resource manager for the VM instance + * @fn: The ops for the function + * @data: Private data for function + * @vm_list: for gunyah_vm's functions list + * @fn_list: for gunyah_vm_function's instances list + */ +struct gunyah_vm_function_instance { + size_t arg_size; + void *argp; + struct gunyah_vm *ghvm; + struct gunyah_rm *rm; + struct gunyah_vm_function *fn; + void *data; + struct list_head vm_list; +}; + +int gunyah_vm_function_register(struct gunyah_vm_function *f); +void gunyah_vm_function_unregister(struct gunyah_vm_function *f); + +/* Since the function identifiers were setup in a uapi header as an + * enum and we do no want to change that, the user must supply the expanded + * constant as well and the compiler checks they are the same. + * See also MODULE_ALIAS_RDMA_NETLINK. + */ +#define MODULE_ALIAS_GUNYAH_VM_FUNCTION(_type, _idx) \ + static inline void __maybe_unused __chk##_idx(void) \ + { \ + BUILD_BUG_ON(_type != _idx); \ + } \ + MODULE_ALIAS("ghfunc:" __stringify(_idx)) + +#define DECLARE_GUNYAH_VM_FUNCTION(_name, _type, _bind, _unbind, _compare) \ + static struct gunyah_vm_function _name = { \ + .type = _type, \ + .name = __stringify(_name), \ + .mod = THIS_MODULE, \ + .bind = _bind, \ + .unbind = _unbind, \ + .compare = _compare, \ + } + +#define module_gunyah_vm_function(__gf) \ + module_driver(__gf, gunyah_vm_function_register, \ + gunyah_vm_function_unregister) + +#define DECLARE_GUNYAH_VM_FUNCTION_INIT(_name, _type, _idx, _bind, _unbind, \ + _compare) \ + DECLARE_GUNYAH_VM_FUNCTION(_name, _type, _bind, _unbind, _compare); \ + module_gunyah_vm_function(_name); \ + MODULE_ALIAS_GUNYAH_VM_FUNCTION(_type, _idx) + +#endif diff --git a/include/uapi/linux/gunyah.h b/include/uapi/linux/gunyah.h index 31e7f79a6c39..1b7cb5fde70a 100644 --- a/include/uapi/linux/gunyah.h +++ b/include/uapi/linux/gunyah.h @@ -25,4 +25,22 @@ */ #define GUNYAH_VM_START _IO(GUNYAH_IOCTL_TYPE, 0x3) +#define GUNYAH_FN_MAX_ARG_SIZE 256 + +/** + * struct gunyah_fn_desc - Arguments to create a VM function + * @type: Type of the function. See &enum gunyah_fn_type. + * @arg_size: Size of argument to pass to the function. arg_size <= GUNYAH_FN_MAX_ARG_SIZE + * @arg: Pointer to argument given to the function. See &enum gunyah_fn_type for expected + * arguments for a function type. + */ +struct gunyah_fn_desc { + __u32 type; + __u32 arg_size; + __u64 arg; +}; + +#define GUNYAH_VM_ADD_FUNCTION _IOW(GUNYAH_IOCTL_TYPE, 0x4, struct gunyah_fn_desc) +#define GUNYAH_VM_REMOVE_FUNCTION _IOW(GUNYAH_IOCTL_TYPE, 0x7, struct gunyah_fn_desc) + #endif From patchwork Sat Dec 16 00:20:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 179691 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:3b04:b0:fb:cd0c:d3e with SMTP id c4csp9666798dys; Fri, 15 Dec 2023 16:26:34 -0800 (PST) X-Google-Smtp-Source: AGHT+IF41FUFy1JHhlveUH6OTYIMaQ0gc6+aCLlMfNWtdnRY8W7tDA+pruzMXljJnMXPsDaTtlTX X-Received: by 2002:a05:6214:d6f:b0:67f:50b:4dc4 with SMTP id 15-20020a0562140d6f00b0067f050b4dc4mr6321967qvs.89.1702686394545; Fri, 15 Dec 2023 16:26:34 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702686394; cv=none; d=google.com; s=arc-20160816; b=cQBq9vhpvIsRBB0y9cDUK2HoPTBxff0dbKBcsKudpYnVlTX7uzUyBPymWAF1VRRN/1 zqdACxFo1VyBtsSHG9M059zELpNtt50n2LHCqcpZ0pKAbYepQxSLL34dZ+jmSfeWelK2 5MqyeocjLsT41uA2z79aUVNbQ4OZQ9tx2F1zyCV77wgJB3vBj/x8PQtcwKxOBKy2lP4H w34u+vlN9CGyMJXvMbKRZQxOv/02GEe2arc7TsFfcV36wLLjfH7LAuS9kXdOfpJgbCyO 2x25qYjj5arBiyiA7nnbsDtdNuY1hKbzxIgG+/yTCPvYIiUb4wRTw79aStSYpyXCIr6M jPRQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:list-unsubscribe:list-subscribe:list-id:precedence :subject:date:from:dkim-signature; bh=bqGXiy7rJWgunfcS0swUq9wvN7P/no+5hve3TGGiwYk=; fh=Pv/sMsQia5wmqAOYHIQUkkUFpDKhZjuODj71Ojb2ybw=; b=NWYMtrfBd3oJ0oS47ZcRok9He/wgUI3MjTxz0S2EKmAWhB/edqQ3B9SNN+WA2s8VMB kz99lu4Zce8bSzO89iH8TJ0ZH0W0Y9IGnQenyklrf2H9IV4pB35OCSeldAkfC1h+Qzou lyKQ8SdhJ7ZBu4gnOmLmRxX2Kqhh7DZJB+cvRPD10bKOWPc2btkZZcNxMYH6ppZqXpCr QTFB9puStWRT+F2mgcp6ui5Foi7C0yrwCgWY9hs8AaQVpgKLDtCfvoCzy6em/ZJgV8em gTiT/rujR++JJNzNC1N8e5ZylhOVui+xzVAUn4C/wFiZz7MLspbEBGVpiWDl49T/hNst nxAg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=mNiKvPnu; spf=pass (google.com: domain of linux-kernel+bounces-1882-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1882-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id dz6-20020a05620a2b8600b0077fbe2de1bbsi810346qkb.518.2023.12.15.16.26.34 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Dec 2023 16:26:34 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-1882-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=mNiKvPnu; spf=pass (google.com: domain of linux-kernel+bounces-1882-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1882-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 4A2DD1C247C0 for ; Sat, 16 Dec 2023 00:26:34 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 786C92E3E1; Sat, 16 Dec 2023 00:21:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="mNiKvPnu" X-Original-To: linux-kernel@vger.kernel.org Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 374F111181; Sat, 16 Dec 2023 00:21:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=quicinc.com Received: from pps.filterd (m0279867.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BFNuDjm011568; Sat, 16 Dec 2023 00:21:09 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:date:subject:mime-version:content-type :content-transfer-encoding:message-id:references:in-reply-to:to :cc; s=qcppdkim1; bh=bqGXiy7rJWgunfcS0swUq9wvN7P/no+5hve3TGGiwYk =; b=mNiKvPnuTFnWo0uoPUtkubYQuzZwN1hgYe/ZypSkUFonIWf0xazZ79/qMNR pVztBO3WZWk7CJ3QbSbpVn3n86lVhUNo5KGGwUjccFCJ5HAHrI96j2LQDuNcbIr1 XKwc1euqacMtNysA9AtVVcT2J1H26/sAs0lBtzrbiKqp2SSxMOe5h+8Lt5zzwRsB cJEUVjI0SDlmnLVT0OKd5KPak6CDfpQOVmMCBmP1vJOquiKKitgsb5zrOPAXjDHw DbL3OWeDgbv3zZ4kMEDGu9dYh0PlReHV6NRPcM3Vo6cOHKPbCKelKSQv3lN/EN9L ZdFJwgoTUIHyoRBh1UWorOZYpTA== Received: from nasanppmta02.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v0k90t5ed-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:09 +0000 (GMT) Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA02.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 3BG0L8DH014659 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:08 GMT Received: from [169.254.0.1] (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.40; Fri, 15 Dec 2023 16:21:07 -0800 From: Elliot Berman Date: Fri, 15 Dec 2023 16:20:53 -0800 Subject: [PATCH RFC v15 12/30] virt: gunyah: Translate gh_rm_hyp_resource into gunyah_resource Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <20231215-gunyah-v15-12-192a5d872a30@quicinc.com> References: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> In-Reply-To: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> To: Alex Elder , Srinivas Kandagatla , Murali Nalajal , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Philip Derrin , Prakruthi Deepak Heragu , Jonathan Corbet , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Catalin Marinas , Will Deacon , Konrad Dybcio , Bjorn Andersson , Dmitry Baryshkov , "Fuad Tabba" , Sean Christopherson CC: , , , , , Elliot Berman X-Mailer: b4 0.13-dev X-ClientProxiedBy: nalasex01a.na.qualcomm.com (10.47.209.196) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: T7ZeyOadqgP4Q6zbC8RuW7Sozwk6ZNoa X-Proofpoint-GUID: T7ZeyOadqgP4Q6zbC8RuW7Sozwk6ZNoa X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 lowpriorityscore=0 impostorscore=0 priorityscore=1501 malwarescore=0 clxscore=1015 mlxlogscore=948 adultscore=0 suspectscore=0 bulkscore=0 spamscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312160001 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785396089033024377 X-GMAIL-MSGID: 1785396089033024377 When booting a Gunyah virtual machine, the host VM may gain capabilities to interact with resources for the guest virtual machine. Examples of such resources are vCPUs or message queues. To use those resources, we need to translate the RM response into a gunyah_resource structure which are useful to Linux drivers. Presently, Linux drivers need only to know the type of resource, the capability ID, and an interrupt. On ARM64 systems, the interrupt reported by Gunyah is the GIC interrupt ID number and always a SPI. Reviewed-by: Alex Elder Signed-off-by: Elliot Berman --- arch/arm64/include/asm/gunyah.h | 36 +++++++++ drivers/virt/gunyah/rsc_mgr.c | 174 +++++++++++++++++++++++++++++++++++++++- include/linux/gunyah.h | 3 + include/linux/gunyah_rsc_mgr.h | 5 ++ 4 files changed, 217 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/gunyah.h b/arch/arm64/include/asm/gunyah.h new file mode 100644 index 000000000000..0cd3debe22b6 --- /dev/null +++ b/arch/arm64/include/asm/gunyah.h @@ -0,0 +1,36 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ +#ifndef _ASM_GUNYAH_H +#define _ASM_GUNYAH_H + +#include +#include + +static inline int arch_gunyah_fill_irq_fwspec_params(u32 virq, + struct irq_fwspec *fwspec) +{ + /* Assume that Gunyah gave us an SPI or ESPI; defensively check it */ + if (WARN(virq < 32, "Unexpected virq: %d\n", virq)) { + return -EINVAL; + } else if (virq <= 1019) { + fwspec->param_count = 3; + fwspec->param[0] = 0; /* GIC_SPI */ + fwspec->param[1] = virq - 32; /* virq 32 -> SPI 0 */ + fwspec->param[2] = IRQ_TYPE_EDGE_RISING; + } else if (WARN(virq < 4096, "Unexpected virq: %d\n", virq)) { + return -EINVAL; + } else if (virq < 5120) { + fwspec->param_count = 3; + fwspec->param[0] = 2; /* GIC_ESPI */ + fwspec->param[1] = virq - 4096; /* virq 4096 -> ESPI 0 */ + fwspec->param[2] = IRQ_TYPE_EDGE_RISING; + } else { + WARN(1, "Unexpected virq: %d\n", virq); + return -EINVAL; + } + return 0; +} + +#endif diff --git a/drivers/virt/gunyah/rsc_mgr.c b/drivers/virt/gunyah/rsc_mgr.c index 052b3f072a85..acd772deb950 100644 --- a/drivers/virt/gunyah/rsc_mgr.c +++ b/drivers/virt/gunyah/rsc_mgr.c @@ -17,6 +17,8 @@ #include #include +#include + #include "rsc_mgr.h" #include "vm_mgr.h" @@ -129,6 +131,7 @@ struct gunyah_rm_message { * @send_ready: completion to send messages again * @nh: notifier chain for clients interested in RM notification messages * @miscdev: /dev/gunyah + * @irq_domain: Domain to translate Gunyah hwirqs to Linux irqs */ struct gunyah_rm { struct device *dev; @@ -147,6 +150,7 @@ struct gunyah_rm { struct blocking_notifier_head nh; struct miscdevice miscdev; + struct irq_domain *irq_domain; }; /** @@ -187,6 +191,143 @@ static inline int gunyah_rm_error_remap(enum gunyah_rm_error rm_error) } } +struct gunyah_irq_chip_data { + u32 gunyah_virq; +}; + +static struct irq_chip gunyah_rm_irq_chip = { + /* clang-format off */ + .name = "Gunyah", + .irq_enable = irq_chip_enable_parent, + .irq_disable = irq_chip_disable_parent, + .irq_ack = irq_chip_ack_parent, + .irq_mask = irq_chip_mask_parent, + .irq_mask_ack = irq_chip_mask_ack_parent, + .irq_unmask = irq_chip_unmask_parent, + .irq_eoi = irq_chip_eoi_parent, + .irq_set_affinity = irq_chip_set_affinity_parent, + .irq_set_type = irq_chip_set_type_parent, + .irq_set_wake = irq_chip_set_wake_parent, + .irq_set_vcpu_affinity = irq_chip_set_vcpu_affinity_parent, + .irq_retrigger = irq_chip_retrigger_hierarchy, + .irq_get_irqchip_state = irq_chip_get_parent_state, + .irq_set_irqchip_state = irq_chip_set_parent_state, + .flags = IRQCHIP_SET_TYPE_MASKED | + IRQCHIP_SKIP_SET_WAKE | + IRQCHIP_MASK_ON_SUSPEND, + /* clang-format on */ +}; + +static int gunyah_rm_irq_domain_alloc(struct irq_domain *d, unsigned int virq, + unsigned int nr_irqs, void *arg) +{ + struct gunyah_irq_chip_data *chip_data, *spec = arg; + struct irq_fwspec parent_fwspec = {}; + struct gunyah_rm *rm = d->host_data; + u32 gunyah_virq = spec->gunyah_virq; + int ret; + + if (nr_irqs != 1) + return -EINVAL; + + chip_data = kzalloc(sizeof(*chip_data), GFP_KERNEL); + if (!chip_data) + return -ENOMEM; + + chip_data->gunyah_virq = gunyah_virq; + + ret = irq_domain_set_hwirq_and_chip(d, virq, chip_data->gunyah_virq, + &gunyah_rm_irq_chip, chip_data); + if (ret) + goto err_free_irq_data; + + parent_fwspec.fwnode = d->parent->fwnode; + ret = arch_gunyah_fill_irq_fwspec_params(chip_data->gunyah_virq, + &parent_fwspec); + if (ret) { + dev_err(rm->dev, "virq translation failed %u: %d\n", + chip_data->gunyah_virq, ret); + goto err_free_irq_data; + } + + ret = irq_domain_alloc_irqs_parent(d, virq, nr_irqs, &parent_fwspec); + if (ret) + goto err_free_irq_data; + + return ret; +err_free_irq_data: + kfree(chip_data); + return ret; +} + +static void gunyah_rm_irq_domain_free_single(struct irq_domain *d, + unsigned int virq) +{ + struct irq_data *irq_data; + + irq_data = irq_domain_get_irq_data(d, virq); + if (!irq_data) + return; + + kfree(irq_data->chip_data); + irq_data->chip_data = NULL; +} + +static void gunyah_rm_irq_domain_free(struct irq_domain *d, unsigned int virq, + unsigned int nr_irqs) +{ + unsigned int i; + + for (i = 0; i < nr_irqs; i++) + gunyah_rm_irq_domain_free_single(d, virq); +} + +static const struct irq_domain_ops gunyah_rm_irq_domain_ops = { + .alloc = gunyah_rm_irq_domain_alloc, + .free = gunyah_rm_irq_domain_free, +}; + +struct gunyah_resource * +gunyah_rm_alloc_resource(struct gunyah_rm *rm, + struct gunyah_rm_hyp_resource *hyp_resource) +{ + struct gunyah_resource *ghrsc; + int ret; + + ghrsc = kzalloc(sizeof(*ghrsc), GFP_KERNEL); + if (!ghrsc) + return NULL; + + ghrsc->type = hyp_resource->type; + ghrsc->capid = le64_to_cpu(hyp_resource->cap_id); + ghrsc->irq = IRQ_NOTCONNECTED; + ghrsc->rm_label = le32_to_cpu(hyp_resource->resource_label); + if (hyp_resource->virq) { + struct gunyah_irq_chip_data irq_data = { + .gunyah_virq = le32_to_cpu(hyp_resource->virq), + }; + + ret = irq_domain_alloc_irqs(rm->irq_domain, 1, NUMA_NO_NODE, + &irq_data); + if (ret < 0) { + dev_err(rm->dev, + "Failed to allocate interrupt for resource %d label: %d: %d\n", + ghrsc->type, ghrsc->rm_label, ret); + kfree(ghrsc); + return NULL; + } + ghrsc->irq = ret; + } + + return ghrsc; +} + +void gunyah_rm_free_resource(struct gunyah_resource *ghrsc) +{ + irq_dispose_mapping(ghrsc->irq); + kfree(ghrsc); +} + static int gunyah_rm_init_message_payload(struct gunyah_rm_message *message, const void *msg, size_t hdr_size, size_t msg_size) @@ -713,6 +854,8 @@ static int gunyah_rm_probe_rx_msgq(struct gunyah_rm *rm, static int gunyah_rm_probe(struct platform_device *pdev) { + struct irq_domain *parent_irq_domain; + struct device_node *parent_irq_node; struct gunyah_rm *rm; int ret; @@ -738,15 +881,43 @@ static int gunyah_rm_probe(struct platform_device *pdev) if (ret) return ret; + parent_irq_node = of_irq_find_parent(pdev->dev.of_node); + if (!parent_irq_node) { + dev_err(&pdev->dev, + "Failed to find interrupt parent of resource manager\n"); + return -ENODEV; + } + + parent_irq_domain = irq_find_host(parent_irq_node); + if (!parent_irq_domain) { + dev_err(&pdev->dev, + "Failed to find interrupt parent domain of resource manager\n"); + return -ENODEV; + } + + rm->irq_domain = irq_domain_add_hierarchy(parent_irq_domain, 0, 0, + pdev->dev.of_node, + &gunyah_rm_irq_domain_ops, + NULL); + if (!rm->irq_domain) { + dev_err(&pdev->dev, "Failed to add irq domain\n"); + return -ENODEV; + } + rm->irq_domain->host_data = rm; + + rm->miscdev.parent = &pdev->dev; rm->miscdev.name = "gunyah"; rm->miscdev.minor = MISC_DYNAMIC_MINOR; rm->miscdev.fops = &gunyah_dev_fops; ret = misc_register(&rm->miscdev); if (ret) - return ret; + goto err_irq_domain; return 0; +err_irq_domain: + irq_domain_remove(rm->irq_domain); + return ret; } static void gunyah_rm_remove(struct platform_device *pdev) @@ -754,6 +925,7 @@ static void gunyah_rm_remove(struct platform_device *pdev) struct gunyah_rm *rm = platform_get_drvdata(pdev); misc_deregister(&rm->miscdev); + irq_domain_remove(rm->irq_domain); } static const struct of_device_id gunyah_rm_of_match[] = { diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index ea2c9eecce0e..ecd62d0e36ad 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -29,6 +29,9 @@ struct gunyah_resource { enum gunyah_resource_type type; u64 capid; unsigned int irq; + + struct list_head list; + u32 rm_label; }; /******************************************************************************/ diff --git a/include/linux/gunyah_rsc_mgr.h b/include/linux/gunyah_rsc_mgr.h index 7bd2bee91f3e..0f61c2b10817 100644 --- a/include/linux/gunyah_rsc_mgr.h +++ b/include/linux/gunyah_rsc_mgr.h @@ -163,4 +163,9 @@ int gunyah_rm_vm_set_address_layout(struct gunyah_rm *rm, u16 vmid, enum gunyah_rm_range_id range_id, u64 base_address, u64 size); +struct gunyah_resource * +gunyah_rm_alloc_resource(struct gunyah_rm *rm, + struct gunyah_rm_hyp_resource *hyp_resource); +void gunyah_rm_free_resource(struct gunyah_resource *ghrsc); + #endif From patchwork Sat Dec 16 00:20:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 179686 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:3b04:b0:fb:cd0c:d3e with SMTP id c4csp9666052dys; Fri, 15 Dec 2023 16:24:33 -0800 (PST) X-Google-Smtp-Source: AGHT+IH5eYtrDw3uFz8ZJYSu4m2ESoLYTd4n1DUFktKWCgN0Xksk2DgRdxwa8Xa5VTG35xb9ihwf X-Received: by 2002:a05:6a00:4643:b0:6ce:2731:47b5 with SMTP id kp3-20020a056a00464300b006ce273147b5mr16682767pfb.21.1702686272806; Fri, 15 Dec 2023 16:24:32 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702686272; cv=none; d=google.com; s=arc-20160816; b=TSZygelB6SUoBCnxHsBx8DGHzZdM75TO8FIfgeSd916WOoDnTgsclA2Lcq4no15tBu l9O66K/dk6PK76Sp4Qxp/Rro4FpE1DArlScY7PqrKRbPA44hbwZwhjzDMoHelHlUSD+5 5Y0KKO3W2JaBVyYC5MJD8w7ecfNkMYGJ4eBI8UmaystCCcHuU6EZPyBCI1RyiG4C/f92 d0cBHzZHtkaQD4dL/Vfg9P7oytBXomv+Jb5b0FuKUPKF8H5HT2Nc4jG1RXWOsd+A9/nY YKv4rTSUyK9w+X6K9ZhLgt5Ivh6ksR4JKMOj7WbrvyGvG7XwNsHmY9ru37KsRvbXPu9M t82Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:list-unsubscribe:list-subscribe:list-id:precedence :subject:date:from:dkim-signature; bh=vRWXtC1lSA0hbTxHT+hthw8XxevZPZGu7CIYzJADcb8=; fh=Pv/sMsQia5wmqAOYHIQUkkUFpDKhZjuODj71Ojb2ybw=; b=DVMmdhu+AWqaBdOBenInJzfmCRhEjVFmv0r8yDIvVuvpif75V9g1CiA/VTcfrNvn+K TFDb1sJAitYiyX1e6VfsYH5j7KAJgVf53T36V7rizCiYyp8SCrzgc1EjKnqMr+FUBRC/ W+DGH/0S2k1sHoVae76BmRyx3tp2/9UdXJeyWnnkqPYhLS8fpZMNcw+MIW50KYEWDGIE KmRqT3dDZMXZovODoeH3uiEx7LXqkINq+FItO8id8xAy+V2YRndSCHUcXd7GXgYbJcNj s6Z0/vCzh+k7bWspaj/BBmOoVFBAeLlUeFjGEj8CAgUFq7++gYI/Po6EVkaRHu118B6S 6t0A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b="pozHD/JT"; spf=pass (google.com: domain of linux-kernel+bounces-1876-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1876-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id k190-20020a633dc7000000b005cd786d8e3bsi354583pga.249.2023.12.15.16.24.32 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Dec 2023 16:24:32 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-1876-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b="pozHD/JT"; spf=pass (google.com: domain of linux-kernel+bounces-1876-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1876-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 96DA22850DB for ; Sat, 16 Dec 2023 00:24:23 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id E52C814292; Sat, 16 Dec 2023 00:21:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="pozHD/JT" X-Original-To: linux-kernel@vger.kernel.org Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 384BF4A17; Sat, 16 Dec 2023 00:21:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=quicinc.com Received: from pps.filterd (m0279862.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BFNW4m3010066; Sat, 16 Dec 2023 00:21:09 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:date:subject:mime-version:content-type :content-transfer-encoding:message-id:references:in-reply-to:to :cc; s=qcppdkim1; bh=vRWXtC1lSA0hbTxHT+hthw8XxevZPZGu7CIYzJADcb8 =; b=pozHD/JTYeb0aKLS+MdrtHlmQrmhtB/atc0P0pfH4+8j3TCpmO9HDkhEQhT RULl+gJv26x6Ulx0rIHWsrytlwBDduc8xFAgCNwWVjcW/xZ2dhTf2TqJ4geBY4Va J/9+R3wT9kQVLz5qceqNbpMyowOkh8nbdL4+FydbJpTLd3/wqGfaqLjNfvigZ5KX hqrWKYrt30SC0svakaOqmwUFgzkiTGDYskYsFh1/W5beDY99mt9YSAJWCRba6kgj 5Zbm9Po5dxqLCk6WSz1RqkN4Mcc7Aj5asu8Ti0ZuIpXMc1DZusH58S+Vox94Dx80 dWWU8UEKpYTDavjZ1578v40+w3g== Received: from nasanppmta02.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v0w198hpe-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:09 +0000 (GMT) Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA02.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 3BG0L8S2014691 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:08 GMT Received: from [169.254.0.1] (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.40; Fri, 15 Dec 2023 16:21:07 -0800 From: Elliot Berman Date: Fri, 15 Dec 2023 16:20:54 -0800 Subject: [PATCH RFC v15 13/30] virt: gunyah: Add resource tickets Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <20231215-gunyah-v15-13-192a5d872a30@quicinc.com> References: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> In-Reply-To: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> To: Alex Elder , Srinivas Kandagatla , Murali Nalajal , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Philip Derrin , Prakruthi Deepak Heragu , Jonathan Corbet , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Catalin Marinas , Will Deacon , Konrad Dybcio , Bjorn Andersson , Dmitry Baryshkov , "Fuad Tabba" , Sean Christopherson CC: , , , , , Elliot Berman X-Mailer: b4 0.13-dev X-ClientProxiedBy: nalasex01a.na.qualcomm.com (10.47.209.196) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: _Aw1VxlShPrPXyS0SRREsh7uk7JE22FJ X-Proofpoint-ORIG-GUID: _Aw1VxlShPrPXyS0SRREsh7uk7JE22FJ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 mlxlogscore=999 suspectscore=0 mlxscore=0 impostorscore=0 lowpriorityscore=0 spamscore=0 adultscore=0 phishscore=0 clxscore=1015 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312160001 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785395961140656317 X-GMAIL-MSGID: 1785395961140656317 Some VM functions need to acquire Gunyah resources. For instance, Gunyah vCPUs are exposed to the host as a resource. The Gunyah vCPU function will register a resource ticket and be able to interact with the hypervisor once the resource ticket is filled. Resource tickets are the mechanism for functions to acquire ownership of Gunyah resources. Gunyah functions can be created before the VM's resources are created and made available to Linux. A resource ticket identifies a type of resource and a label of a resource which the ticket holder is interested in. Resources are created by Gunyah as configured in the VM's devicetree configuration. Gunyah doesn't process the label and that makes it possible for userspace to create multiple resources with the same label. Resource ticket owners need to be prepared for populate to be called multiple times if userspace created multiple resources with the same label. Reviewed-by: Alex Elder Signed-off-by: Elliot Berman --- drivers/virt/gunyah/vm_mgr.c | 127 +++++++++++++++++++++++++++++++++++++++++- drivers/virt/gunyah/vm_mgr.h | 3 + include/linux/gunyah_vm_mgr.h | 36 ++++++++++++ 3 files changed, 165 insertions(+), 1 deletion(-) diff --git a/drivers/virt/gunyah/vm_mgr.c b/drivers/virt/gunyah/vm_mgr.c index 81bc73047e1e..fb22459b21c8 100644 --- a/drivers/virt/gunyah/vm_mgr.c +++ b/drivers/virt/gunyah/vm_mgr.c @@ -185,6 +185,106 @@ void gunyah_vm_function_unregister(struct gunyah_vm_function *fn) } EXPORT_SYMBOL_GPL(gunyah_vm_function_unregister); +int gunyah_vm_add_resource_ticket(struct gunyah_vm *ghvm, + struct gunyah_vm_resource_ticket *ticket) +{ + struct gunyah_vm_resource_ticket *iter; + struct gunyah_resource *ghrsc, *rsc_iter; + int ret = 0; + + mutex_lock(&ghvm->resources_lock); + list_for_each_entry(iter, &ghvm->resource_tickets, vm_list) { + if (iter->resource_type == ticket->resource_type && + iter->label == ticket->label) { + ret = -EEXIST; + goto out; + } + } + + if (!try_module_get(ticket->owner)) { + ret = -ENODEV; + goto out; + } + + list_add(&ticket->vm_list, &ghvm->resource_tickets); + INIT_LIST_HEAD(&ticket->resources); + + list_for_each_entry_safe(ghrsc, rsc_iter, &ghvm->resources, list) { + if (ghrsc->type == ticket->resource_type && + ghrsc->rm_label == ticket->label) { + if (ticket->populate(ticket, ghrsc)) + list_move(&ghrsc->list, &ticket->resources); + } + } +out: + mutex_unlock(&ghvm->resources_lock); + return ret; +} +EXPORT_SYMBOL_GPL(gunyah_vm_add_resource_ticket); + +void gunyah_vm_remove_resource_ticket(struct gunyah_vm *ghvm, + struct gunyah_vm_resource_ticket *ticket) +{ + struct gunyah_resource *ghrsc, *iter; + + mutex_lock(&ghvm->resources_lock); + list_for_each_entry_safe(ghrsc, iter, &ticket->resources, list) { + ticket->unpopulate(ticket, ghrsc); + list_move(&ghrsc->list, &ghvm->resources); + } + + module_put(ticket->owner); + list_del(&ticket->vm_list); + mutex_unlock(&ghvm->resources_lock); +} +EXPORT_SYMBOL_GPL(gunyah_vm_remove_resource_ticket); + +static void gunyah_vm_add_resource(struct gunyah_vm *ghvm, + struct gunyah_resource *ghrsc) +{ + struct gunyah_vm_resource_ticket *ticket; + + mutex_lock(&ghvm->resources_lock); + list_for_each_entry(ticket, &ghvm->resource_tickets, vm_list) { + if (ghrsc->type == ticket->resource_type && + ghrsc->rm_label == ticket->label) { + if (ticket->populate(ticket, ghrsc)) + list_add(&ghrsc->list, &ticket->resources); + else + list_add(&ghrsc->list, &ghvm->resources); + /* unconditonal -- we prevent multiple identical + * resource tickets so there will not be some other + * ticket elsewhere in the list if populate() failed. + */ + goto found; + } + } + list_add(&ghrsc->list, &ghvm->resources); +found: + mutex_unlock(&ghvm->resources_lock); +} + +static void gunyah_vm_clean_resources(struct gunyah_vm *ghvm) +{ + struct gunyah_vm_resource_ticket *ticket, *titer; + struct gunyah_resource *ghrsc, *riter; + + mutex_lock(&ghvm->resources_lock); + if (!list_empty(&ghvm->resource_tickets)) { + dev_warn(ghvm->parent, "Dangling resource tickets:\n"); + list_for_each_entry_safe(ticket, titer, &ghvm->resource_tickets, + vm_list) { + dev_warn(ghvm->parent, " %pS\n", ticket->populate); + gunyah_vm_remove_resource_ticket(ghvm, ticket); + } + } + + list_for_each_entry_safe(ghrsc, riter, &ghvm->resources, list) { + gunyah_rm_free_resource(ghrsc); + } + mutex_unlock(&ghvm->resources_lock); +} + static int gunyah_vm_rm_notification_status(struct gunyah_vm *ghvm, void *data) { struct gunyah_rm_vm_status_payload *payload = data; @@ -263,6 +363,9 @@ static __must_check struct gunyah_vm *gunyah_vm_alloc(struct gunyah_rm *rm) init_waitqueue_head(&ghvm->vm_status_wait); kref_init(&ghvm->kref); ghvm->vm_status = GUNYAH_RM_VM_STATUS_NO_STATE; + mutex_init(&ghvm->resources_lock); + INIT_LIST_HEAD(&ghvm->resources); + INIT_LIST_HEAD(&ghvm->resource_tickets); INIT_LIST_HEAD(&ghvm->functions); mutex_init(&ghvm->fn_lock); @@ -271,7 +374,9 @@ static __must_check struct gunyah_vm *gunyah_vm_alloc(struct gunyah_rm *rm) static int gunyah_vm_start(struct gunyah_vm *ghvm) { - int ret; + struct gunyah_rm_hyp_resources *resources; + struct gunyah_resource *ghrsc; + int ret, i, n; down_write(&ghvm->status_lock); if (ghvm->vm_status != GUNYAH_RM_VM_STATUS_NO_STATE) { @@ -307,6 +412,25 @@ static int gunyah_vm_start(struct gunyah_vm *ghvm) } ghvm->vm_status = GUNYAH_RM_VM_STATUS_READY; + ret = gunyah_rm_get_hyp_resources(ghvm->rm, ghvm->vmid, &resources); + if (ret) { + dev_warn(ghvm->parent, + "Failed to get hypervisor resources for VM: %d\n", + ret); + goto err; + } + + for (i = 0, n = le32_to_cpu(resources->n_entries); i < n; i++) { + ghrsc = gunyah_rm_alloc_resource(ghvm->rm, + &resources->entries[i]); + if (!ghrsc) { + ret = -ENOMEM; + goto err; + } + + gunyah_vm_add_resource(ghvm, ghrsc); + } + ret = gunyah_rm_vm_start(ghvm->rm, ghvm->vmid); if (ret) { dev_warn(ghvm->parent, "Failed to start VM: %d\n", ret); @@ -404,6 +528,7 @@ static void _gunyah_vm_put(struct kref *kref) gunyah_vm_stop(ghvm); gunyah_vm_remove_functions(ghvm); + gunyah_vm_clean_resources(ghvm); if (ghvm->vm_status != GUNYAH_RM_VM_STATUS_NO_STATE && ghvm->vm_status != GUNYAH_RM_VM_STATUS_LOAD && diff --git a/drivers/virt/gunyah/vm_mgr.h b/drivers/virt/gunyah/vm_mgr.h index a9e0eaf212fd..b94f6d16b787 100644 --- a/drivers/virt/gunyah/vm_mgr.h +++ b/drivers/virt/gunyah/vm_mgr.h @@ -34,6 +34,9 @@ struct gunyah_vm { struct kref kref; struct mutex fn_lock; struct list_head functions; + struct mutex resources_lock; + struct list_head resources; + struct list_head resource_tickets; }; #endif diff --git a/include/linux/gunyah_vm_mgr.h b/include/linux/gunyah_vm_mgr.h index f40e57f32fd8..b02c6e89b207 100644 --- a/include/linux/gunyah_vm_mgr.h +++ b/include/linux/gunyah_vm_mgr.h @@ -97,4 +97,40 @@ void gunyah_vm_function_unregister(struct gunyah_vm_function *f); module_gunyah_vm_function(_name); \ MODULE_ALIAS_GUNYAH_VM_FUNCTION(_type, _idx) +/** + * struct gunyah_vm_resource_ticket - Represents a ticket to reserve access to VM resource(s) + * @vm_list: for @gunyah_vm->resource_tickets + * @resources: List of resource(s) associated with this ticket + * (members are from @gunyah_resource->list) + * @resource_type: Type of resource this ticket reserves + * @label: Label of the resource from resource manager this ticket reserves. + * @owner: owner of the ticket + * @populate: callback provided by the ticket owner and called when a resource is found that + * matches @resource_type and @label. Note that this callback could be called + * multiple times if userspace created mutliple resources with the same type/label. + * This callback may also have significant delay after gunyah_vm_add_resource_ticket() + * since gunyah_vm_add_resource_ticket() could be called before the VM starts. + * @unpopulate: callback provided by the ticket owner and called when the ticket owner should no + * longer use the resource provided in the argument. When unpopulate() returns, + * the ticket owner should not be able to use the resource any more as the resource + * might being freed. + */ +struct gunyah_vm_resource_ticket { + struct list_head vm_list; + struct list_head resources; + enum gunyah_resource_type resource_type; + u32 label; + + struct module *owner; + bool (*populate)(struct gunyah_vm_resource_ticket *ticket, + struct gunyah_resource *ghrsc); + void (*unpopulate)(struct gunyah_vm_resource_ticket *ticket, + struct gunyah_resource *ghrsc); +}; + +int gunyah_vm_add_resource_ticket(struct gunyah_vm *ghvm, + struct gunyah_vm_resource_ticket *ticket); +void gunyah_vm_remove_resource_ticket(struct gunyah_vm *ghvm, + struct gunyah_vm_resource_ticket *ticket); + #endif From patchwork Sat Dec 16 00:20:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 179690 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:3b04:b0:fb:cd0c:d3e with SMTP id c4csp9666635dys; Fri, 15 Dec 2023 16:26:08 -0800 (PST) X-Google-Smtp-Source: AGHT+IFoqDtfG1/ahTlfSCQwwrITRF7L68gPkWQXZrFnjnuNupJ9O5TnTpnoAxJhGLfsQqeDWC85 X-Received: by 2002:a05:620a:12d7:b0:77e:fba3:81d9 with SMTP id e23-20020a05620a12d700b0077efba381d9mr14800828qkl.111.1702686367996; Fri, 15 Dec 2023 16:26:07 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702686367; cv=none; d=google.com; s=arc-20160816; b=tLLD/tVGRcHFjzXB/wolW2ZioRK0q78euoEWcXlkYXxrwoWAwW3USmGzjPxm/2BkXB YpfvhDbz6r3i+beQ3Y9Sh4XFLCxr7ZuH/Mo+rcVof4WqZdOo8KiMu7NzjKmCHkoplult k8Opm4Dbgs7jLg80rUi2QStH7NcGIBib+9hHtS/S8zbXAJOkW9VokmVciB2FZomHHuvr mka/T5K43u7//6WyCcfMqKt8684T8bfUfVQbwJsSaB7NLoEHH4JGNONhh8ikTcQ60Gzy ao0vp+xor60Z9ehI/8+xpjA6UxxB1hBY1IczZa22sQeW9IqrBOiFONJH2yxCAMnvGeCq HlGg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:list-unsubscribe:list-subscribe:list-id:precedence :subject:date:from:dkim-signature; bh=/+Id6UngUNXl0d+rJA071fD+Qr412G4VIuAp81DQV5I=; fh=Pv/sMsQia5wmqAOYHIQUkkUFpDKhZjuODj71Ojb2ybw=; b=qe7PmjP0p6TVu+vGz07t1cU9iFNLe+2PKCGGrSAsPr33g6BHZy0qT0q/qgdQHkWhKN 09YIZABj1cigIl542yenpWx7uwZiwkzmpHYvQbeZPRKumX1ontz7xHgELXp+j4W9BF3g mtqKfMwnZWrXlwJtvDWUmYrE1odA7vk+1B9LyWV0qLDroZgEudKsJR0fTPhEEr4GiFMU qSsHSLeb6j0wmO/OmE4eogpc2jNvsV4VD9wn3UR7g477d5gd9uStglSu8/QSV1cnYx9N BS3AA+qDpizPamN5TfwKyTF7vxuX5Biq3VKrhrE1iZXnofcfSo7d3q1UPZRjRJQ49LMu wRBA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=LUKP349R; spf=pass (google.com: domain of linux-kernel+bounces-1880-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1880-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id dz6-20020a05620a2b8600b0077fbe2de1bbsi810346qkb.518.2023.12.15.16.26.07 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Dec 2023 16:26:07 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-1880-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=LUKP349R; spf=pass (google.com: domain of linux-kernel+bounces-1880-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1880-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id B36BB1C21827 for ; Sat, 16 Dec 2023 00:26:07 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 093F17F1; Sat, 16 Dec 2023 00:21:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="LUKP349R" X-Original-To: linux-kernel@vger.kernel.org Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9A927107A1; Sat, 16 Dec 2023 00:21:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=quicinc.com Received: from pps.filterd (m0279867.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BFMqUoH027173; Sat, 16 Dec 2023 00:21:10 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:date:subject:mime-version:content-type :content-transfer-encoding:message-id:references:in-reply-to:to :cc; s=qcppdkim1; bh=/+Id6UngUNXl0d+rJA071fD+Qr412G4VIuAp81DQV5I =; b=LUKP349RPynWGfGdn4h214nYsasY4jj+1a8uGryPSnj1jTPANSLvPR9QHry Jv50lyTAG3WqsgDyv9tR4w5BbQLlwZcQWRwKAGYHh0WIgl0VHxWlRaALEL4/QfP0 FeDpUf1aaFFr0Z3Hv/5T9MXO3LVg1jN+s4GRIeLV/ZeOZ8U/36Nk6idrFVFnIn9p fnMngljLK1Aq1BnuhuS4wtDsDMA+HDAUaDs3vReNV5CFgavJo7rgeG5yjvTrBGMu wQozXGPGvCgR5xr2gEVJQwUxHYUh8ANQKKNFdDuorSpW2MWqZm+Sr77WVVKy520e z1uL7kk1jhtQacGGukFPoGqF6Qg== Received: from nasanppmta05.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v0k90t5ee-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:10 +0000 (GMT) Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA05.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 3BG0L9j1006332 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:09 GMT Received: from [169.254.0.1] (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.40; Fri, 15 Dec 2023 16:21:08 -0800 From: Elliot Berman Date: Fri, 15 Dec 2023 16:20:55 -0800 Subject: [PATCH RFC v15 14/30] virt: gunyah: Add IO handlers Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <20231215-gunyah-v15-14-192a5d872a30@quicinc.com> References: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> In-Reply-To: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> To: Alex Elder , Srinivas Kandagatla , Murali Nalajal , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Philip Derrin , Prakruthi Deepak Heragu , Jonathan Corbet , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Catalin Marinas , Will Deacon , Konrad Dybcio , Bjorn Andersson , Dmitry Baryshkov , "Fuad Tabba" , Sean Christopherson CC: , , , , , Elliot Berman X-Mailer: b4 0.13-dev X-ClientProxiedBy: nalasex01a.na.qualcomm.com (10.47.209.196) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: viVGif0NYI_o8FMi5l-n1s0_bA6JvWVC X-Proofpoint-GUID: viVGif0NYI_o8FMi5l-n1s0_bA6JvWVC X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 lowpriorityscore=0 impostorscore=0 priorityscore=1501 malwarescore=0 clxscore=1015 mlxlogscore=936 adultscore=0 suspectscore=0 bulkscore=0 spamscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312160001 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785396060826083809 X-GMAIL-MSGID: 1785396060826083809 Add framework for VM functions to handle stage-2 write faults from Gunyah guest virtual machines. IO handlers have a range of addresses which they apply to. Optionally, they may apply to only when the value written matches the IO handler's value. Reviewed-by: Alex Elder Co-developed-by: Prakruthi Deepak Heragu Signed-off-by: Prakruthi Deepak Heragu Signed-off-by: Elliot Berman --- drivers/virt/gunyah/vm_mgr.c | 114 ++++++++++++++++++++++++++++++++++++++++++ drivers/virt/gunyah/vm_mgr.h | 5 ++ include/linux/gunyah_vm_mgr.h | 29 +++++++++++ 3 files changed, 148 insertions(+) diff --git a/drivers/virt/gunyah/vm_mgr.c b/drivers/virt/gunyah/vm_mgr.c index fb22459b21c8..8feb9302d7d2 100644 --- a/drivers/virt/gunyah/vm_mgr.c +++ b/drivers/virt/gunyah/vm_mgr.c @@ -285,6 +285,118 @@ static void gunyah_vm_clean_resources(struct gunyah_vm *ghvm) mutex_unlock(&ghvm->resources_lock); } +static int _gunyah_vm_io_handler_compare(const struct rb_node *node, + const struct rb_node *parent) +{ + struct gunyah_vm_io_handler *n = + container_of(node, struct gunyah_vm_io_handler, node); + struct gunyah_vm_io_handler *p = + container_of(parent, struct gunyah_vm_io_handler, node); + + if (n->addr < p->addr) + return -1; + if (n->addr > p->addr) + return 1; + if ((n->len && !p->len) || (!n->len && p->len)) + return 0; + if (n->len < p->len) + return -1; + if (n->len > p->len) + return 1; + /* one of the io handlers doesn't have datamatch and the other does. + * For purposes of comparison, that makes them identical since the + * one that doesn't have datamatch will cover the same handler that + * does. + */ + if (n->datamatch != p->datamatch) + return 0; + if (n->data < p->data) + return -1; + if (n->data > p->data) + return 1; + return 0; +} + +static int gunyah_vm_io_handler_compare(struct rb_node *node, + const struct rb_node *parent) +{ + return _gunyah_vm_io_handler_compare(node, parent); +} + +static int gunyah_vm_io_handler_find(const void *key, + const struct rb_node *node) +{ + const struct gunyah_vm_io_handler *k = key; + + return _gunyah_vm_io_handler_compare(&k->node, node); +} + +static struct gunyah_vm_io_handler * +gunyah_vm_mgr_find_io_hdlr(struct gunyah_vm *ghvm, u64 addr, u64 len, u64 data) +{ + struct gunyah_vm_io_handler key = { + .addr = addr, + .len = len, + .datamatch = true, + .data = data, + }; + struct rb_node *node; + + node = rb_find(&key, &ghvm->mmio_handler_root, + gunyah_vm_io_handler_find); + if (!node) + return NULL; + + return container_of(node, struct gunyah_vm_io_handler, node); +} + +int gunyah_vm_mmio_write(struct gunyah_vm *ghvm, u64 addr, u32 len, u64 data) +{ + struct gunyah_vm_io_handler *io_hdlr = NULL; + int ret; + + down_read(&ghvm->mmio_handler_lock); + io_hdlr = gunyah_vm_mgr_find_io_hdlr(ghvm, addr, len, data); + if (!io_hdlr || !io_hdlr->ops || !io_hdlr->ops->write) { + ret = -ENOENT; + goto out; + } + + ret = io_hdlr->ops->write(io_hdlr, addr, len, data); + +out: + up_read(&ghvm->mmio_handler_lock); + return ret; +} +EXPORT_SYMBOL_GPL(gunyah_vm_mmio_write); + +int gunyah_vm_add_io_handler(struct gunyah_vm *ghvm, + struct gunyah_vm_io_handler *io_hdlr) +{ + struct rb_node *found; + + if (io_hdlr->datamatch && + (!io_hdlr->len || io_hdlr->len > sizeof(io_hdlr->data))) + return -EINVAL; + + down_write(&ghvm->mmio_handler_lock); + found = rb_find_add(&io_hdlr->node, &ghvm->mmio_handler_root, + gunyah_vm_io_handler_compare); + up_write(&ghvm->mmio_handler_lock); + + return found ? -EEXIST : 0; +} +EXPORT_SYMBOL_GPL(gunyah_vm_add_io_handler); + +void gunyah_vm_remove_io_handler(struct gunyah_vm *ghvm, + struct gunyah_vm_io_handler *io_hdlr) +{ + down_write(&ghvm->mmio_handler_lock); + rb_erase(&io_hdlr->node, &ghvm->mmio_handler_root); + up_write(&ghvm->mmio_handler_lock); +} +EXPORT_SYMBOL_GPL(gunyah_vm_remove_io_handler); + static int gunyah_vm_rm_notification_status(struct gunyah_vm *ghvm, void *data) { struct gunyah_rm_vm_status_payload *payload = data; @@ -366,6 +478,8 @@ static __must_check struct gunyah_vm *gunyah_vm_alloc(struct gunyah_rm *rm) mutex_init(&ghvm->resources_lock); INIT_LIST_HEAD(&ghvm->resources); INIT_LIST_HEAD(&ghvm->resource_tickets); + init_rwsem(&ghvm->mmio_handler_lock); + ghvm->mmio_handler_root = RB_ROOT; INIT_LIST_HEAD(&ghvm->functions); mutex_init(&ghvm->fn_lock); diff --git a/drivers/virt/gunyah/vm_mgr.h b/drivers/virt/gunyah/vm_mgr.h index b94f6d16b787..a26bf81e94fc 100644 --- a/drivers/virt/gunyah/vm_mgr.h +++ b/drivers/virt/gunyah/vm_mgr.h @@ -12,6 +12,7 @@ #include #include #include +#include #include #include @@ -37,6 +38,10 @@ struct gunyah_vm { struct mutex resources_lock; struct list_head resources; struct list_head resource_tickets; + struct rb_root mmio_handler_root; + struct rw_semaphore mmio_handler_lock; }; +int gunyah_vm_mmio_write(struct gunyah_vm *ghvm, u64 addr, u32 len, u64 data); + #endif diff --git a/include/linux/gunyah_vm_mgr.h b/include/linux/gunyah_vm_mgr.h index b02c6e89b207..635adcbcc7d4 100644 --- a/include/linux/gunyah_vm_mgr.h +++ b/include/linux/gunyah_vm_mgr.h @@ -133,4 +133,33 @@ int gunyah_vm_add_resource_ticket(struct gunyah_vm *ghvm, void gunyah_vm_remove_resource_ticket(struct gunyah_vm *ghvm, struct gunyah_vm_resource_ticket *ticket); +/* + * gunyah_vm_io_handler contains the info about an io device and its associated + * addr and the ops associated with the io device. + */ +struct gunyah_vm_io_handler { + struct rb_node node; + u64 addr; + + bool datamatch; + u8 len; + u64 data; + struct gunyah_vm_io_handler_ops *ops; +}; + +/* + * gunyah_vm_io_handler_ops contains function pointers associated with an iodevice. + */ +struct gunyah_vm_io_handler_ops { + int (*read)(struct gunyah_vm_io_handler *io_dev, u64 addr, u32 len, + u64 data); + int (*write)(struct gunyah_vm_io_handler *io_dev, u64 addr, u32 len, + u64 data); +}; + +int gunyah_vm_add_io_handler(struct gunyah_vm *ghvm, + struct gunyah_vm_io_handler *io_dev); +void gunyah_vm_remove_io_handler(struct gunyah_vm *ghvm, + struct gunyah_vm_io_handler *io_dev); + #endif From patchwork Sat Dec 16 00:20:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 179693 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:3b04:b0:fb:cd0c:d3e with SMTP id c4csp9667158dys; Fri, 15 Dec 2023 16:27:24 -0800 (PST) X-Google-Smtp-Source: AGHT+IEEHqqjwe8nQK0qKHGZzwpgZ3WCg6o4nDw+LAzfjSM9dU4PpwReGcENOS1Ayzw3zMxs9gth X-Received: by 2002:a05:651c:19a4:b0:2cc:1db0:4a6f with SMTP id bx36-20020a05651c19a400b002cc1db04a6fmr5042432ljb.32.1702686444160; Fri, 15 Dec 2023 16:27:24 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702686444; cv=none; d=google.com; s=arc-20160816; b=ClYbLhWNvPor4ePhCbXNF8M9KebkRmFw5G1vopeEe8xHmDMkua6KH3bcrPFI+3o3zK Wkq8S6ZaJVUNNODaiWwvmRKw+VJa1u4L8vbrGWTRdBqFMDut5tfN+4fAGKdHSrPlgpqu RX3SiiBlbY6cja9VBRG1puM/p44YzgJ41uXvlHjzMwe8crIDNA+KciycNoeHEU2RUHMl IHYUssjtoxI0Me5g8HnjU1/WEokIFjHtJd0LJmnRaO+pvqTEm/svB2aqhUgqfNX2TiOS 7tHU/tZR4/O8j6/7C87z+sYmWERTyjyOO3e7TKxAZQHty2kU76LnsOAvJQW2TvvCzmoK 5zAA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:list-unsubscribe:list-subscribe:list-id:precedence :subject:date:from:dkim-signature; bh=IC3bcqcwKy8KffxllZXS23XN2gL4nURWbQSLOfvNnAU=; fh=Pv/sMsQia5wmqAOYHIQUkkUFpDKhZjuODj71Ojb2ybw=; b=YFujwE8Z5bDfOs0I4/b6ZuzUwxnGWTVE1zgK/ge3wxSVLQf/3f5ea9eZbnZM5ZYtrL hcjFctRifm31HaRfNgEQXY0ANK3UDU+k6IaBEqLfurTiOelSJ4+d7gx0LgrJDbA2N4Pj lu4bRLRnNrPNzdtviDxceNWQ295sSkPBCAn0VWTy+XZxq8WnY9sbeZTfQhlpIhvCYqTG OlmeuLQEedee1oW/7pYl347h+NPBkRnmGKcdhsIw4Z5QEm063m5pPye+p3vT6o4GVxqg eLHKxbDff2F/GEcxKzwB1ku6mG/+q99A74PrN88U9iyVkQ+W84TKmjzj/Fcw9I8sT1IF rHEg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=S+YBWtgP; spf=pass (google.com: domain of linux-kernel+bounces-1883-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1883-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id j17-20020a05640211d100b005525fbdec31si2264035edw.85.2023.12.15.16.27.23 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Dec 2023 16:27:24 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-1883-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=S+YBWtgP; spf=pass (google.com: domain of linux-kernel+bounces-1883-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1883-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 8F3EF1F25831 for ; Sat, 16 Dec 2023 00:27:23 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 91AF033089; Sat, 16 Dec 2023 00:21:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="S+YBWtgP" X-Original-To: linux-kernel@vger.kernel.org Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 20EB9134C0; Sat, 16 Dec 2023 00:21:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=quicinc.com Received: from pps.filterd (m0279873.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BG0Kf5x006568; Sat, 16 Dec 2023 00:21:11 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:date:subject:mime-version:content-type :content-transfer-encoding:message-id:references:in-reply-to:to :cc; s=qcppdkim1; bh=IC3bcqcwKy8KffxllZXS23XN2gL4nURWbQSLOfvNnAU =; b=S+YBWtgPCDuzmbm2bV2ViPHohni9kLP+h1SLlnTk6YvNRnFrDfQwexblwa+ 833rhGublJYHiBuCUOE8Bc2HoUC3k1O5y1s/IsIlvZtfKmhLI3DXq+5l8ZBCBtHz e4t04YJPNakHCold2EPxNghPgGtNdsBMcxYp7e0lPa+m7Z+wOn+IiVsPskkgtS7p cu8lm1BWxOx+vOCvd38QFBpf5zkrWxwo9QFgPkBNp8qVXNJijzPqyELCUf/JlDwm rRwEpubnGyHGNrTwm9MLGu1DE99NBeDdLgNTzkoBTOJwAUFDW5T7rGlrxVUWH9wJ 8/efdUmB/LbW6gLZ4BYnoHoYeaA== Received: from nasanppmta01.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v118s801t-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:11 +0000 (GMT) Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 3BG0LAbi015616 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:10 GMT Received: from [169.254.0.1] (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.40; Fri, 15 Dec 2023 16:21:09 -0800 From: Elliot Berman Date: Fri, 15 Dec 2023 16:20:56 -0800 Subject: [PATCH RFC v15 15/30] gunyah: Add hypercalls for demand paging Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <20231215-gunyah-v15-15-192a5d872a30@quicinc.com> References: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> In-Reply-To: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> To: Alex Elder , Srinivas Kandagatla , Murali Nalajal , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Philip Derrin , Prakruthi Deepak Heragu , Jonathan Corbet , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Catalin Marinas , Will Deacon , Konrad Dybcio , Bjorn Andersson , Dmitry Baryshkov , "Fuad Tabba" , Sean Christopherson CC: , , , , , Elliot Berman X-Mailer: b4 0.13-dev X-ClientProxiedBy: nalasex01a.na.qualcomm.com (10.47.209.196) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: cGamZTK9kCVm63Vm9fSgArhy6xIUUNve X-Proofpoint-GUID: cGamZTK9kCVm63Vm9fSgArhy6xIUUNve X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 mlxlogscore=556 malwarescore=0 phishscore=0 adultscore=0 priorityscore=1501 mlxscore=0 lowpriorityscore=0 impostorscore=0 bulkscore=0 spamscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312160001 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785396141241614253 X-GMAIL-MSGID: 1785396141241614253 Three hypercalls are needed to support demand paging. Signed-off-by: Elliot Berman --- arch/arm64/gunyah/gunyah_hypercall.c | 60 +++++++++++++++++++++++++++ arch/arm64/include/asm/gunyah.h | 21 ++++++++++ include/linux/gunyah.h | 79 ++++++++++++++++++++++++++++++++++++ 3 files changed, 160 insertions(+) diff --git a/arch/arm64/gunyah/gunyah_hypercall.c b/arch/arm64/gunyah/gunyah_hypercall.c index cadc512934c4..468a5bcb98b1 100644 --- a/arch/arm64/gunyah/gunyah_hypercall.c +++ b/arch/arm64/gunyah/gunyah_hypercall.c @@ -39,6 +39,9 @@ EXPORT_SYMBOL_GPL(arch_is_gunyah_guest); #define GUNYAH_HYPERCALL_HYP_IDENTIFY GUNYAH_HYPERCALL(0x8000) #define GUNYAH_HYPERCALL_MSGQ_SEND GUNYAH_HYPERCALL(0x801B) #define GUNYAH_HYPERCALL_MSGQ_RECV GUNYAH_HYPERCALL(0x801C) +#define GUNYAH_HYPERCALL_ADDRSPACE_MAP GUNYAH_HYPERCALL(0x802B) +#define GUNYAH_HYPERCALL_ADDRSPACE_UNMAP GUNYAH_HYPERCALL(0x802C) +#define GUNYAH_HYPERCALL_MEMEXTENT_DONATE GUNYAH_HYPERCALL(0x8061) /* clang-format on */ /** @@ -92,5 +95,62 @@ enum gunyah_error gunyah_hypercall_msgq_recv(u64 capid, void *buff, size_t size, } EXPORT_SYMBOL_GPL(gunyah_hypercall_msgq_recv); +enum gunyah_error gunyah_hypercall_addrspace_map(u64 capid, u64 extent_capid, u64 vbase, + u32 extent_attrs, u32 flags, u64 offset, u64 size) +{ + struct arm_smccc_1_2_regs args = { + .a0 = GUNYAH_HYPERCALL_ADDRSPACE_MAP, + .a1 = capid, + .a2 = extent_capid, + .a3 = vbase, + .a4 = extent_attrs, + .a5 = flags, + .a6 = offset, + .a7 = size, + /* C language says this will be implictly zero. Gunyah requires 0, so be explicit */ + .a8 = 0, + }; + struct arm_smccc_1_2_regs res; + + arm_smccc_1_2_hvc(&args, &res); + + return res.a0; +} +EXPORT_SYMBOL_GPL(gunyah_hypercall_addrspace_map); + +enum gunyah_error gunyah_hypercall_addrspace_unmap(u64 capid, u64 extent_capid, u64 vbase, + u32 flags, u64 offset, u64 size) +{ + struct arm_smccc_1_2_regs args = { + .a0 = GUNYAH_HYPERCALL_ADDRSPACE_UNMAP, + .a1 = capid, + .a2 = extent_capid, + .a3 = vbase, + .a4 = flags, + .a5 = offset, + .a6 = size, + /* C language says this will be implictly zero. Gunyah requires 0, so be explicit */ + .a7 = 0, + }; + struct arm_smccc_1_2_regs res; + + arm_smccc_1_2_hvc(&args, &res); + + return res.a0; +} +EXPORT_SYMBOL_GPL(gunyah_hypercall_addrspace_unmap); + +enum gunyah_error gunyah_hypercall_memextent_donate(u32 options, u64 from_capid, u64 to_capid, + u64 offset, u64 size) +{ + struct arm_smccc_res res; + + arm_smccc_1_1_hvc(GUNYAH_HYPERCALL_MEMEXTENT_DONATE, options, from_capid, to_capid, + offset, size, 0, &res); + + return res.a0; +} +EXPORT_SYMBOL_GPL(gunyah_hypercall_memextent_donate); + MODULE_LICENSE("GPL"); MODULE_DESCRIPTION("Gunyah Hypervisor Hypercalls"); diff --git a/arch/arm64/include/asm/gunyah.h b/arch/arm64/include/asm/gunyah.h index 0cd3debe22b6..4adf24977fd1 100644 --- a/arch/arm64/include/asm/gunyah.h +++ b/arch/arm64/include/asm/gunyah.h @@ -33,4 +33,25 @@ static inline int arch_gunyah_fill_irq_fwspec_params(u32 virq, return 0; } +enum arch_gunyah_memtype { + /* clang-format off */ + GUNYAH_MEMTYPE_DEVICE_nGnRnE = 0, + GUNYAH_DEVICE_nGnRE = 1, + GUNYAH_DEVICE_nGRE = 2, + GUNYAH_DEVICE_GRE = 3, + + GUNYAH_NORMAL_NC = 0b0101, + GUNYAH_NORMAL_ONC_IWT = 0b0110, + GUNYAH_NORMAL_ONC_IWB = 0b0111, + GUNYAH_NORMAL_OWT_INC = 0b1001, + GUNYAH_NORMAL_WT = 0b1010, + GUNYAH_NORMAL_OWT_IWB = 0b1011, + GUNYAH_NORMAL_OWB_INC = 0b1101, + GUNYAH_NORMAL_OWB_IWT = 0b1110, + GUNYAH_NORMAL_WB = 0b1111, + /* clang-format on */ +}; + +#define ARCH_GUNYAH_DEFAULT_MEMTYPE GUNYAH_NORMAL_WB + #endif diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index ecd62d0e36ad..86f48dc1d56d 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -151,4 +151,83 @@ enum gunyah_error gunyah_hypercall_msgq_send(u64 capid, size_t size, void *buff, enum gunyah_error gunyah_hypercall_msgq_recv(u64 capid, void *buff, size_t size, size_t *recv_size, bool *ready); +#define GUNYAH_ADDRSPACE_SELF_CAP 0 + +enum gunyah_pagetable_access { + /* clang-format off */ + GUNYAH_PAGETABLE_ACCESS_NONE = 0, + GUNYAH_PAGETABLE_ACCESS_X = 1, + GUNYAH_PAGETABLE_ACCESS_W = 2, + GUNYAH_PAGETABLE_ACCESS_R = 4, + GUNYAH_PAGETABLE_ACCESS_RX = 5, + GUNYAH_PAGETABLE_ACCESS_RW = 6, + GUNYAH_PAGETABLE_ACCESS_RWX = 7, + /* clang-format on */ +}; + +/* clang-format off */ +#define GUNYAH_MEMEXTENT_MAPPING_USER_ACCESS GENMASK_ULL(2, 0) +#define GUNYAH_MEMEXTENT_MAPPING_KERNEL_ACCESS GENMASK_ULL(6, 4) +#define GUNYAH_MEMEXTENT_MAPPING_TYPE GENMASK_ULL(23, 16) +/* clang-format on */ + +enum gunyah_memextent_donate_type { + /* clang-format off */ + GUNYAH_MEMEXTENT_DONATE_TO_CHILD = 0, + GUNYAH_MEMEXTENT_DONATE_TO_PARENT = 1, + GUNYAH_MEMEXTENT_DONATE_TO_SIBLING = 2, + GUNYAH_MEMEXTENT_DONATE_TO_PROTECTED = 3, + GUNYAH_MEMEXTENT_DONATE_FROM_PROTECTED = 4, + /* clang-format on */ +}; + +enum gunyah_addrspace_map_flag_bits { + /* clang-format off */ + GUNYAH_ADDRSPACE_MAP_FLAG_PARTIAL = 0, + GUNYAH_ADDRSPACE_MAP_FLAG_PRIVATE = 1, + GUNYAH_ADDRSPACE_MAP_FLAG_VMMIO = 2, + GUNYAH_ADDRSPACE_MAP_FLAG_NOSYNC = 31, + /* clang-format on */ +}; + +enum gunyah_error gunyah_hypercall_addrspace_map(u64 capid, u64 extent_capid, + u64 vbase, u32 extent_attrs, + u32 flags, u64 offset, + u64 size); +enum gunyah_error gunyah_hypercall_addrspace_unmap(u64 capid, u64 extent_capid, + u64 vbase, u32 flags, + u64 offset, u64 size); + +/* clang-format off */ +#define GUNYAH_MEMEXTENT_OPTION_TYPE_MASK GENMASK_ULL(7, 0) +#define GUNYAH_MEMEXTENT_OPTION_NOSYNC BIT(31) +/* clang-format on */ + +enum gunyah_error gunyah_hypercall_memextent_donate(u32 options, u64 from_capid, + u64 to_capid, u64 offset, + u64 size); + +struct gunyah_hypercall_vcpu_run_resp { + union { + enum { + /* clang-format off */ + /* VCPU is ready to run */ + GUNYAH_VCPU_STATE_READY = 0, + /* VCPU is sleeping until an interrupt arrives */ + GUNYAH_VCPU_STATE_EXPECTS_WAKEUP = 1, + /* VCPU is powered off */ + GUNYAH_VCPU_STATE_POWERED_OFF = 2, + /* VCPU is blocked in EL2 for unspecified reason */ + GUNYAH_VCPU_STATE_BLOCKED = 3, + /* VCPU has returned for MMIO READ */ + GUNYAH_VCPU_ADDRSPACE_VMMIO_READ = 4, + /* VCPU has returned for MMIO WRITE */ + GUNYAH_VCPU_ADDRSPACE_VMMIO_WRITE = 5, + /* clang-format on */ + } state; + u64 sized_state; + }; + u64 state_data[3]; +}; + #endif From patchwork Sat Dec 16 00:20:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 179696 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:3b04:b0:fb:cd0c:d3e with SMTP id c4csp9667456dys; Fri, 15 Dec 2023 16:28:18 -0800 (PST) X-Google-Smtp-Source: AGHT+IH0IhDfJtBMi2nzq8VzZN+QSz8CiwQ6BMF0Ipjsr8T+IS0mbqUBZ3kKHq+/HFuR0f1f1+/W X-Received: by 2002:a05:600c:808b:b0:40b:5e21:dd24 with SMTP id ew11-20020a05600c808b00b0040b5e21dd24mr6518310wmb.82.1702686498327; Fri, 15 Dec 2023 16:28:18 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702686498; cv=none; d=google.com; s=arc-20160816; b=0tZ5hx1evRT6zhnu9O+Q+t2aLI5M3Xr5tjSa55TkwA43+zv5D8FALdOZJdPU/ptCwG VoUr/+mkIuTinGyw0zV2YwtabTM/DP1fUteh4nsPYpP6oJEDscZZSVMBPtjz9B/yBQpq ead5we8tz2Q22rnvAQuDLBzE0XopUUcfpiTpQ0uwPNi0V4YjwSl0r4KoL07a/7qe3Fng D2zP2bAgO5Md9Ur+Et8y7w6Im0RfYw/k9HzdbTNaJ7fr+hVuuoJz6k5n5ROaW3yI0Apl La9e8VS/9ftdQq9x212oQ4LbI8ZGuVSanH93MAFs6CI84riQvEjaBh6GeRkUk+8Y4mwy yifg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:list-unsubscribe:list-subscribe:list-id:precedence :subject:date:from:dkim-signature; bh=EK6PaN/SzqxP6V9oHPvTpZJBwg0U4uYIaeICLpAcX5Q=; fh=Pv/sMsQia5wmqAOYHIQUkkUFpDKhZjuODj71Ojb2ybw=; b=ltC6uItdwBUv1nrDrOaxZ9t3PyZbNm9vzMzvdWQAniMXeBIahHKaVjkasSLGjzHGg0 OGMnzfyNo3BFs2bq5pQCr1+hNE4MMm4k7ctJVhpyTnfHDnb+fr3Oe6ltc1+OvZDXEYeS V86DQzbnABuMpZeAUAUZlGWS0PK7UcU6NqAsfSM2sUmngbpdFAuYy7g9rWKuB+110TIk Y/gzTfIhllcd0CSEbei+5QFjanUZevx0lcWitZDRZrviW6w5mDugawhk0WQr73Cuyy5N EIhKjHEucH91I1q9cd4EzFPdUBc1nOZsEKoR7PxCDfOMTi4qerP4tNrkkxdDxKjgslGG yqNQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=JIX6uOkw; spf=pass (google.com: domain of linux-kernel+bounces-1886-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1886-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id d6-20020a17090648c600b00a1e25d3070fsi7523680ejt.420.2023.12.15.16.28.18 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Dec 2023 16:28:18 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-1886-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=JIX6uOkw; spf=pass (google.com: domain of linux-kernel+bounces-1886-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1886-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id C1BB21F21027 for ; Sat, 16 Dec 2023 00:28:17 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D50A235284; Sat, 16 Dec 2023 00:21:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="JIX6uOkw" X-Original-To: linux-kernel@vger.kernel.org Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4C34215AEA; Sat, 16 Dec 2023 00:21:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=quicinc.com Received: from pps.filterd (m0279870.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BFNZFfT004744; Sat, 16 Dec 2023 00:21:12 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:date:subject:mime-version:content-type :content-transfer-encoding:message-id:references:in-reply-to:to :cc; s=qcppdkim1; bh=EK6PaN/SzqxP6V9oHPvTpZJBwg0U4uYIaeICLpAcX5Q =; b=JIX6uOkw0icXxtq+gDvzmAtLtWCvoqgBfR0pjmwyiyGDwnlr7SQRfJFPA3n DbkKw0wX5MYPB8jL86V/rwtOQzjWRMgckKSGoXhpPD58REOsz9mLKN9rv2wcMbzO W4rGrd3WoMULYGdDtMSf6HOUKr3xQUedVEyTJORmQ56nkB346PMtQT1aMu1WSwQ9 4zSBFXra4zGqTepUFW8G8ZIUeUAT9NM1bTRmG8r30jePdwhcjzW77Zz+MCzTX48e nHbQWU+AYvQ4WlAIuwlZJJDv9wRkDaviblFvApncVrsdu7YWCU68kLbcSvpDp16M zpYXiOGA4cFq2uktLEyqSP0ZiJQ== Received: from nasanppmta05.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v0vv9ghj9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:11 +0000 (GMT) Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA05.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 3BG0LAe2006343 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:10 GMT Received: from [169.254.0.1] (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.40; Fri, 15 Dec 2023 16:21:09 -0800 From: Elliot Berman Date: Fri, 15 Dec 2023 16:20:57 -0800 Subject: [PATCH RFC v15 16/30] virt: gunyah: Add interfaces to map memory into guest address space Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <20231215-gunyah-v15-16-192a5d872a30@quicinc.com> References: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> In-Reply-To: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> To: Alex Elder , Srinivas Kandagatla , Murali Nalajal , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Philip Derrin , Prakruthi Deepak Heragu , Jonathan Corbet , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Catalin Marinas , Will Deacon , Konrad Dybcio , Bjorn Andersson , Dmitry Baryshkov , "Fuad Tabba" , Sean Christopherson CC: , , , , , Elliot Berman X-Mailer: b4 0.13-dev X-ClientProxiedBy: nalasex01a.na.qualcomm.com (10.47.209.196) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: Y3nWYmRkIouILFFabqncMGF_nPqhrNzi X-Proofpoint-ORIG-GUID: Y3nWYmRkIouILFFabqncMGF_nPqhrNzi X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 impostorscore=0 phishscore=0 clxscore=1015 adultscore=0 malwarescore=0 priorityscore=1501 lowpriorityscore=0 suspectscore=0 mlxlogscore=999 mlxscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312160001 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785396197795933937 X-GMAIL-MSGID: 1785396197795933937 Gunyah virtual machines are created with all memory provided upon VM creation using Resource Manager's memory parcel construct or by enabling demand paging. Demand paging allows the host to map/unmap contiguous pages (folios) on its own, typically done when a guest vCPU faults on those pages. In order to map folios to a guest virtual machine, the host needs to add the folio to a memory extent that can be mapped into the guest virtual machine's address space and then map the folio into the guest address space. Memory extents are Gunyah's mechanism for access control and provide capability-based modifications to what memory virtual machines are allowed to access. When a virtual machine is configured for demand paging, 3 memory extent and 1 address space capabilities are provided to the host. The memory extents give the host ability to move memory between "host-only", "guest-only", and "host-and-guest-shared" extents. The guest address space can only map memory which is in the "guest-only" or "host-and-guest-shared" extents. Similarly, the host address space can only have mapped memory which is in the "host-only" or "host-and-guest-shared" extents. Implement two functions which Linux can use to move memory between the virtual machines: gunyah_provide_folio and gunyah_reclaim_folio. Memory that has been provided to the guest is tracked in a maple tree to be reclaimed later. Folios provided to the virtual machine are assumed to be owned Gunyah stack: the folio's ->private field is used for bookkeeping about whether page is mapped into virtual machine. Signed-off-by: Elliot Berman --- drivers/virt/gunyah/Makefile | 2 +- drivers/virt/gunyah/vm_mgr.c | 60 ++++++++ drivers/virt/gunyah/vm_mgr.h | 30 ++++ drivers/virt/gunyah/vm_mgr_mem.c | 314 +++++++++++++++++++++++++++++++++++++++ 4 files changed, 405 insertions(+), 1 deletion(-) diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index 47f1fae5419b..ccce90438434 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -1,5 +1,5 @@ # SPDX-License-Identifier: GPL-2.0 -gunyah_rsc_mgr-y += rsc_mgr.o rsc_mgr_rpc.o vm_mgr.o +gunyah_rsc_mgr-y += rsc_mgr.o rsc_mgr_rpc.o vm_mgr.o vm_mgr_mem.o obj-$(CONFIG_GUNYAH) += gunyah.o gunyah_rsc_mgr.o diff --git a/drivers/virt/gunyah/vm_mgr.c b/drivers/virt/gunyah/vm_mgr.c index 8feb9302d7d2..a5450dc8cc28 100644 --- a/drivers/virt/gunyah/vm_mgr.c +++ b/drivers/virt/gunyah/vm_mgr.c @@ -18,6 +18,15 @@ #include "vm_mgr.h" +#define GUNYAH_VM_ADDRSPACE_LABEL 0 +// For memory private to guest +#define GUNYAH_VM_MEM_EXTENT_GUEST_PRIVATE_LABEL 0 +#define GUNYAH_VM_MEM_EXTENT_HOST_SHARED_LABEL 1 +// For memory shared with the guest +#define GUNYAH_VM_MEM_EXTENT_GUEST_SHARED_LABEL 3 +// Our own mem extent +#define GUNYAH_VM_MEM_EXTENT_HOST_PRIVATE_LABEL 2 + static DEFINE_XARRAY(gunyah_vm_functions); static void gunyah_vm_put_function(struct gunyah_vm_function *fn) @@ -185,6 +194,16 @@ void gunyah_vm_function_unregister(struct gunyah_vm_function *fn) } EXPORT_SYMBOL_GPL(gunyah_vm_function_unregister); +static bool gunyah_vm_resource_ticket_populate_noop( + struct gunyah_vm_resource_ticket *ticket, struct gunyah_resource *ghrsc) +{ + return true; +} +static void gunyah_vm_resource_ticket_unpopulate_noop( + struct gunyah_vm_resource_ticket *ticket, struct gunyah_resource *ghrsc) +{ +} + int gunyah_vm_add_resource_ticket(struct gunyah_vm *ghvm, struct gunyah_vm_resource_ticket *ticket) { @@ -459,6 +478,17 @@ static void gunyah_vm_stop(struct gunyah_vm *ghvm) ghvm->vm_status == GUNYAH_RM_VM_STATUS_EXITED); } +static inline void setup_extent_ticket(struct gunyah_vm *ghvm, + struct gunyah_vm_resource_ticket *ticket, + u32 label) +{ + ticket->resource_type = GUNYAH_RESOURCE_TYPE_MEM_EXTENT; + ticket->label = label; + ticket->populate = gunyah_vm_resource_ticket_populate_noop; + ticket->unpopulate = gunyah_vm_resource_ticket_unpopulate_noop; + gunyah_vm_add_resource_ticket(ghvm, ticket); +} + static __must_check struct gunyah_vm *gunyah_vm_alloc(struct gunyah_rm *rm) { struct gunyah_vm *ghvm; @@ -483,6 +513,25 @@ static __must_check struct gunyah_vm *gunyah_vm_alloc(struct gunyah_rm *rm) INIT_LIST_HEAD(&ghvm->functions); mutex_init(&ghvm->fn_lock); + mt_init(&ghvm->gm); + + ghvm->addrspace_ticket.resource_type = GUNYAH_RESOURCE_TYPE_ADDR_SPACE; + ghvm->addrspace_ticket.label = GUNYAH_VM_ADDRSPACE_LABEL; + ghvm->addrspace_ticket.populate = + gunyah_vm_resource_ticket_populate_noop; + ghvm->addrspace_ticket.unpopulate = + gunyah_vm_resource_ticket_unpopulate_noop; + gunyah_vm_add_resource_ticket(ghvm, &ghvm->addrspace_ticket); + + setup_extent_ticket(ghvm, &ghvm->host_private_extent_ticket, + GUNYAH_VM_MEM_EXTENT_HOST_PRIVATE_LABEL); + setup_extent_ticket(ghvm, &ghvm->host_shared_extent_ticket, + GUNYAH_VM_MEM_EXTENT_HOST_SHARED_LABEL); + setup_extent_ticket(ghvm, &ghvm->guest_private_extent_ticket, + GUNYAH_VM_MEM_EXTENT_GUEST_PRIVATE_LABEL); + setup_extent_ticket(ghvm, &ghvm->guest_shared_extent_ticket, + GUNYAH_VM_MEM_EXTENT_GUEST_SHARED_LABEL); + return ghvm; } @@ -642,6 +691,15 @@ static void _gunyah_vm_put(struct kref *kref) gunyah_vm_stop(ghvm); gunyah_vm_remove_functions(ghvm); + + gunyah_vm_reclaim_memory(ghvm); + + gunyah_vm_remove_resource_ticket(ghvm, &ghvm->addrspace_ticket); + gunyah_vm_remove_resource_ticket(ghvm, &ghvm->host_shared_extent_ticket); + gunyah_vm_remove_resource_ticket(ghvm, &ghvm->host_private_extent_ticket); + gunyah_vm_remove_resource_ticket(ghvm, &ghvm->guest_shared_extent_ticket); + gunyah_vm_remove_resource_ticket(ghvm, &ghvm->guest_private_extent_ticket); + gunyah_vm_clean_resources(ghvm); if (ghvm->vm_status != GUNYAH_RM_VM_STATUS_NO_STATE && @@ -655,6 +713,8 @@ static void _gunyah_vm_put(struct kref *kref) ghvm->vm_status == GUNYAH_RM_VM_STATUS_RESET); } + mtree_destroy(&ghvm->gm); + if (ghvm->vm_status > GUNYAH_RM_VM_STATUS_NO_STATE) { gunyah_rm_notifier_unregister(ghvm->rm, &ghvm->nb); diff --git a/drivers/virt/gunyah/vm_mgr.h b/drivers/virt/gunyah/vm_mgr.h index a26bf81e94fc..d2f6ac212169 100644 --- a/drivers/virt/gunyah/vm_mgr.h +++ b/drivers/virt/gunyah/vm_mgr.h @@ -8,8 +8,10 @@ #include #include +#include #include #include +#include #include #include #include @@ -18,11 +20,26 @@ #include +static inline u64 gunyah_gpa_to_gfn(u64 gpa) +{ + return gpa >> PAGE_SHIFT; +} + +static inline u64 gunyah_gfn_to_gpa(u64 gfn) +{ + return gfn << PAGE_SHIFT; +} + long gunyah_dev_vm_mgr_ioctl(struct gunyah_rm *rm, unsigned int cmd, unsigned long arg); struct gunyah_vm { u16 vmid; + struct maple_tree gm; + struct gunyah_vm_resource_ticket addrspace_ticket, + host_private_extent_ticket, host_shared_extent_ticket, + guest_private_extent_ticket, guest_shared_extent_ticket; + struct gunyah_rm *rm; struct device *parent; enum gunyah_rm_vm_auth_mechanism auth; @@ -44,4 +61,17 @@ struct gunyah_vm { int gunyah_vm_mmio_write(struct gunyah_vm *ghvm, u64 addr, u32 len, u64 data); +int gunyah_vm_share_parcel(struct gunyah_vm *ghvm, + struct gunyah_rm_mem_parcel *parcel, u64 gfn, + u64 nr); +int gunyah_vm_parcel_to_paged(struct gunyah_vm *ghvm, + struct gunyah_rm_mem_parcel *parcel, u64 gfn, + u64 nr); +int gunyah_vm_reclaim_parcel(struct gunyah_vm *ghvm, + struct gunyah_rm_mem_parcel *parcel, u64 gfn); +int gunyah_vm_provide_folio(struct gunyah_vm *ghvm, struct folio *folio, + u64 gfn, bool share, bool write); +int gunyah_vm_reclaim_folio(struct gunyah_vm *ghvm, u64 gfn); +void gunyah_vm_reclaim_memory(struct gunyah_vm *ghvm); + #endif diff --git a/drivers/virt/gunyah/vm_mgr_mem.c b/drivers/virt/gunyah/vm_mgr_mem.c new file mode 100644 index 000000000000..d61fceeeac60 --- /dev/null +++ b/drivers/virt/gunyah/vm_mgr_mem.c @@ -0,0 +1,314 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#define pr_fmt(fmt) "gunyah_vm_mgr: " fmt + +#include +#include +#include + +#include "vm_mgr.h" + +#define WRITE_TAG (1 << 0) +#define SHARE_TAG (1 << 1) +#define MEM_PARCEL_TAG (1 << 2) + +static inline struct gunyah_resource * +__first_resource(struct gunyah_vm_resource_ticket *ticket) +{ + return list_first_entry_or_null(&ticket->resources, + struct gunyah_resource, list); +} + +int gunyah_vm_share_parcel(struct gunyah_vm *ghvm, + struct gunyah_rm_mem_parcel *parcel, u64 gfn, u64 nr) +{ + int ret; + + ret = mtree_insert_range(&ghvm->gm, gfn, gfn + nr - 1, + xa_tag_pointer(parcel, MEM_PARCEL_TAG), + GFP_KERNEL); + if (ret) + return ret; + + ret = gunyah_rm_mem_share(ghvm->rm, parcel); + if (ret) + mtree_erase(&ghvm->gm, gfn); + + return ret; +} + +int gunyah_vm_parcel_to_paged(struct gunyah_vm *ghvm, + struct gunyah_rm_mem_parcel *parcel, u64 gfn, + u64 nr) +{ + struct gunyah_rm_mem_entry *entry; + struct folio *folio; + unsigned long g, e, tag = 0; + pgoff_t off; + int ret; + + if (parcel->n_acl_entries > 1) + tag |= SHARE_TAG; + if (parcel->acl_entries[0].perms & GUNYAH_RM_ACL_W) + tag |= WRITE_TAG; + + for (e = 0, g = gfn; g < gfn + nr && e < parcel->n_mem_entries; e++) { + entry = &parcel->mem_entries[e]; + folio = pfn_folio(PHYS_PFN(entry->phys_addr)); + + for (off = 0; off < PHYS_PFN(entry->size); + off += folio_nr_pages(folio)) { + ret = mtree_store_range(&ghvm->gm, g + off, g + off - 1, + xa_tag_pointer(folio, tag), + GFP_KERNEL); + if (ret == -ENOMEM) + return ret; + BUG_ON(ret); + + folio = folio_next(folio); + } + } + + return 0; +} + +int gunyah_vm_reclaim_parcel(struct gunyah_vm *ghvm, + struct gunyah_rm_mem_parcel *parcel, u64 gfn) +{ + int ret; + + ret = gunyah_rm_mem_reclaim(ghvm->rm, parcel); + if (ret) + return ret; + + mtree_erase(&ghvm->gm, gfn); + + return ret; +} + +static inline u32 donate_flags(bool share) +{ + if (share) + return FIELD_PREP_CONST(GUNYAH_MEMEXTENT_OPTION_TYPE_MASK, + GUNYAH_MEMEXTENT_DONATE_TO_SIBLING); + else + return FIELD_PREP_CONST(GUNYAH_MEMEXTENT_OPTION_TYPE_MASK, + GUNYAH_MEMEXTENT_DONATE_TO_PROTECTED); +} + +static inline u32 reclaim_flags(bool share) +{ + if (share) + return FIELD_PREP_CONST(GUNYAH_MEMEXTENT_OPTION_TYPE_MASK, + GUNYAH_MEMEXTENT_DONATE_TO_SIBLING); + else + return FIELD_PREP_CONST(GUNYAH_MEMEXTENT_OPTION_TYPE_MASK, + GUNYAH_MEMEXTENT_DONATE_FROM_PROTECTED); +} + +int gunyah_vm_provide_folio(struct gunyah_vm *ghvm, struct folio *folio, + u64 gfn, bool share, bool write) +{ + struct gunyah_resource *guest_extent, *host_extent, *addrspace; + u32 map_flags = BIT(GUNYAH_ADDRSPACE_MAP_FLAG_PARTIAL); + u64 extent_attrs, gpa = gunyah_gfn_to_gpa(gfn); + phys_addr_t pa = PFN_PHYS(folio_pfn(folio)); + enum gunyah_pagetable_access access; + size_t size = folio_size(folio); + enum gunyah_error gunyah_error; + unsigned long tag = 0; + int ret; + + if (share) { + guest_extent = __first_resource(&ghvm->guest_shared_extent_ticket); + host_extent = __first_resource(&ghvm->host_shared_extent_ticket); + } else { + guest_extent = __first_resource(&ghvm->guest_private_extent_ticket); + host_extent = __first_resource(&ghvm->host_private_extent_ticket); + } + addrspace = __first_resource(&ghvm->addrspace_ticket); + + if (!addrspace || !guest_extent || !host_extent) + return -ENODEV; + + if (share) { + map_flags |= BIT(GUNYAH_ADDRSPACE_MAP_FLAG_VMMIO); + tag |= SHARE_TAG; + } else { + map_flags |= BIT(GUNYAH_ADDRSPACE_MAP_FLAG_PRIVATE); + } + + if (write) + tag |= WRITE_TAG; + + ret = mtree_insert_range(&ghvm->gm, gfn, + gfn + folio_nr_pages(folio) - 1, + xa_tag_pointer(folio, tag), GFP_KERNEL); + if (ret) + return ret; + + if (share && write) + access = GUNYAH_PAGETABLE_ACCESS_RW; + else if (share && !write) + access = GUNYAH_PAGETABLE_ACCESS_R; + else if (!share && write) + access = GUNYAH_PAGETABLE_ACCESS_RWX; + else /* !share && !write */ + access = GUNYAH_PAGETABLE_ACCESS_RX; + + gunyah_error = gunyah_hypercall_memextent_donate(donate_flags(share), + host_extent->capid, + guest_extent->capid, + pa, size); + if (gunyah_error != GUNYAH_ERROR_OK) { + pr_err("Failed to donate memory for guest address 0x%016llx: %d\n", + gpa, gunyah_error); + ret = gunyah_error_remap(gunyah_error); + goto remove; + } + + extent_attrs = + FIELD_PREP_CONST(GUNYAH_MEMEXTENT_MAPPING_TYPE, + ARCH_GUNYAH_DEFAULT_MEMTYPE) | + FIELD_PREP(GUNYAH_MEMEXTENT_MAPPING_USER_ACCESS, access) | + FIELD_PREP(GUNYAH_MEMEXTENT_MAPPING_KERNEL_ACCESS, access); + gunyah_error = gunyah_hypercall_addrspace_map(addrspace->capid, + guest_extent->capid, gpa, + extent_attrs, map_flags, + pa, size); + if (gunyah_error != GUNYAH_ERROR_OK) { + pr_err("Failed to map guest address 0x%016llx: %d\n", gpa, + gunyah_error); + ret = gunyah_error_remap(gunyah_error); + goto memextent_reclaim; + } + + folio_get(folio); + return 0; +memextent_reclaim: + gunyah_error = gunyah_hypercall_memextent_donate(reclaim_flags(share), + guest_extent->capid, + host_extent->capid, pa, + size); + if (gunyah_error != GUNYAH_ERROR_OK) + pr_err("Failed to reclaim memory donation for guest address 0x%016llx: %d\n", + gpa, gunyah_error); +remove: + mtree_erase(&ghvm->gm, gfn); + return ret; +} + +int gunyah_vm_reclaim_folio(struct gunyah_vm *ghvm, u64 gfn) +{ + const u32 map_flags = BIT(GUNYAH_ADDRSPACE_MAP_FLAG_PARTIAL); + struct gunyah_resource *guest_extent, *host_extent, *addrspace; + enum gunyah_pagetable_access access; + enum gunyah_error gunyah_error; + struct folio *folio; + bool write, share; + phys_addr_t pa; + size_t size; + void *entry; + int ret; + + addrspace = __first_resource(&ghvm->addrspace_ticket); + if (!addrspace) + return -ENODEV; + + entry = mtree_load(&ghvm->gm, gfn); + if (!entry) + return 0; + + share = !!(xa_pointer_tag(entry) & SHARE_TAG); + write = !!(xa_pointer_tag(entry) & WRITE_TAG); + folio = xa_untag_pointer(entry); + folio_lock(folio); + if (mtree_load(&ghvm->gm, gfn) != entry) { + ret = -EAGAIN; + goto err; + } + + if (share) { + guest_extent = __first_resource(&ghvm->guest_shared_extent_ticket); + host_extent = __first_resource(&ghvm->host_shared_extent_ticket); + } else { + guest_extent = __first_resource(&ghvm->guest_private_extent_ticket); + host_extent = __first_resource(&ghvm->host_private_extent_ticket); + } + + pa = PFN_PHYS(folio_pfn(folio)); + size = folio_size(folio); + + gunyah_error = gunyah_hypercall_addrspace_unmap(addrspace->capid, + guest_extent->capid, + gfn << PAGE_SHIFT, + map_flags, pa, size); + if (gunyah_error != GUNYAH_ERROR_OK) { + ret = gunyah_error_remap(gunyah_error); + goto err; + } + + gunyah_error = gunyah_hypercall_memextent_donate(reclaim_flags(share), + guest_extent->capid, + host_extent->capid, pa, + size); + if (gunyah_error != GUNYAH_ERROR_OK) { + pr_err_ratelimited( + "Failed to reclaim memory donation for guest address 0x%016llx: %d\n", + gunyah_gfn_to_gpa(gfn), gunyah_error); + ret = gunyah_error_remap(gunyah_error); + goto err; + } + + if (share && write) + access = GUNYAH_PAGETABLE_ACCESS_RW; + else if (share && !write) + access = GUNYAH_PAGETABLE_ACCESS_R; + else if (!share && write) + access = GUNYAH_PAGETABLE_ACCESS_RWX; + else /* !share && !write */ + access = GUNYAH_PAGETABLE_ACCESS_RX; + + gunyah_error = gunyah_hypercall_memextent_donate(donate_flags(share), + guest_extent->capid, + host_extent->capid, pa, + size); + if (gunyah_error != GUNYAH_ERROR_OK) { + pr_err("Failed to reclaim memory donation for guest address 0x%016llx: %d\n", + gfn << PAGE_SHIFT, gunyah_error); + ret = gunyah_error_remap(gunyah_error); + goto err; + } + + BUG_ON(mtree_erase(&ghvm->gm, gfn) != entry); + + folio_unlock(folio); + folio_put(folio); + return 0; +err: + folio_unlock(folio); + return ret; +} + +void gunyah_vm_reclaim_memory(struct gunyah_vm *ghvm) +{ + unsigned long gfn = 0; + void *entry; + int ret = 0; + + mt_for_each(&ghvm->gm, entry, gfn, ULONG_MAX) { + if (xa_pointer_tag(entry) == MEM_PARCEL_TAG) { + ret = gunyah_vm_reclaim_parcel( + ghvm, xa_untag_pointer(entry), gfn); + if (ret) + dev_err(ghvm->parent, + "Failed to reclaim guest parcel at %lx: %d\n", + gfn << PAGE_SHIFT, ret); + } else { + gunyah_vm_reclaim_folio(ghvm, gfn); + } + } +} From patchwork Sat Dec 16 00:20:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 179700 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:3b04:b0:fb:cd0c:d3e with SMTP id c4csp9667869dys; Fri, 15 Dec 2023 16:29:20 -0800 (PST) X-Google-Smtp-Source: AGHT+IF5c5toou/wUrQPxFPpcnBFpSkMUvQI3vaYtzntGNuzpJzMxGj4eq6l73ThxaRiFiMYd9PX X-Received: by 2002:a05:6871:d201:b0:1bf:b863:b6d with SMTP id pk1-20020a056871d20100b001bfb8630b6dmr16332496oac.1.1702686560312; Fri, 15 Dec 2023 16:29:20 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702686560; cv=none; d=google.com; s=arc-20160816; b=LU5jPTF3/o+Ovu7J694ovqjJ/64ZXJl6kY/rRu29q2Vkay+r92eEKv4cJoEF2KqTpr MGZBl6CrlTV00rg5f8BVsM4hd6CdiWJqu1IvpwrQM9ieup7ICu7KitDWlHp0Enm0R/0k dIommgqOFEPv0JQ4xLcODwHADDGvAMMgnzhIcV+MdNvpSLBwmdELmyBCRmU7nVCUoOSA +PEK6m8G1g+CsGc1seyICZ+PsjOD1HMUFqzAtaoaIhGdHmsP0ba0y54MMqvokevF3pTh iUdRUhSoDg8bw7MkbLbUF75cb5FHIduUxaTfH7rjPLggsjwfTp1v+eaSw1+G484BtUQj XXyA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:list-unsubscribe:list-subscribe:list-id:precedence :subject:date:from:dkim-signature; bh=ldkP+25ZzPVXxFYj2IhNVzk916Iu9dwPkeRF9ayDBAI=; fh=Pv/sMsQia5wmqAOYHIQUkkUFpDKhZjuODj71Ojb2ybw=; b=W6/3tn0PGuJi1VSw9fBuJm862V3xzFAJtNLHgR6CFZ0SOI8+zzTyJnvql4hP4DMaEX P3ZsXCxq6/8USx4J1aP71QMS5gWP6OqspEmXH5l9CnHwTmgLhAai5ps/E5qVbbYDqt5D R74L9xdF8MkTd1llEHxxBBiFvi7mHIMlUcDmg9qdK5IQcOzm/KcMLISF0R1Wu+Vt0tG4 soigZOQy/qlhQMrdgwMO/a8O9y9RA/JNFx3FA1NuCYySgzwT+VEOnE5DGNDAx9N7lvur GPsINTCs91q0F3VHYiyzOicBUyCf7O8BwFptL6uUlOcvDb+GKvSaqFqp0iWSZQpyegmg nOFw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=HpNtul72; spf=pass (google.com: domain of linux-kernel+bounces-1890-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1890-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id q6-20020a655246000000b005b92842d469si13914887pgp.62.2023.12.15.16.29.20 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Dec 2023 16:29:20 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-1890-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=HpNtul72; spf=pass (google.com: domain of linux-kernel+bounces-1890-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1890-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 707FA2862B6 for ; Sat, 16 Dec 2023 00:29:11 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C466537D2F; Sat, 16 Dec 2023 00:21:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="HpNtul72" X-Original-To: linux-kernel@vger.kernel.org Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6ECAA1640C; Sat, 16 Dec 2023 00:21:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=quicinc.com Received: from pps.filterd (m0279865.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BFN9ZZq010769; Sat, 16 Dec 2023 00:21:12 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:date:subject:mime-version:content-type :content-transfer-encoding:message-id:references:in-reply-to:to :cc; s=qcppdkim1; bh=ldkP+25ZzPVXxFYj2IhNVzk916Iu9dwPkeRF9ayDBAI =; b=HpNtul72bvWsP7vvVu0/5cuFPfvf3kzkflvsboqVMh9aymQL51ovyfqxiTH QGxEjXU7bCloB53CuAy3mB2W/zOS6LgIZDJolvxSu8Hq1Olkao/SIVr3DuJm2oZt I1gej66JEpZmzoIPo61uMx0YoYMrRNnrvnl9hVaWUj99FSeCcOpOzLcuWiAu264J lL0/FxqUnnq/xZk1tpAo969idmDvlLwGicnQ1kTsdweTvxLyALRfKU/dI867cEYa o8avt7UgpyKT517QidWTqoT36UJSr3Gcb5haWs9/AR9QZkQlv7iF9UI9QAicGtff WwO2veM56miIzada2vIs0y0lFmw== Received: from nasanppmta05.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v0m46hwe1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:11 +0000 (GMT) Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA05.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 3BG0LBIX006348 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:11 GMT Received: from [169.254.0.1] (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.40; Fri, 15 Dec 2023 16:21:10 -0800 From: Elliot Berman Date: Fri, 15 Dec 2023 16:20:58 -0800 Subject: [PATCH RFC v15 17/30] gunyah: rsc_mgr: Add platform ops on mem_lend/mem_reclaim Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <20231215-gunyah-v15-17-192a5d872a30@quicinc.com> References: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> In-Reply-To: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> To: Alex Elder , Srinivas Kandagatla , Murali Nalajal , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Philip Derrin , Prakruthi Deepak Heragu , Jonathan Corbet , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Catalin Marinas , Will Deacon , Konrad Dybcio , Bjorn Andersson , Dmitry Baryshkov , "Fuad Tabba" , Sean Christopherson CC: , , , , , Elliot Berman X-Mailer: b4 0.13-dev X-ClientProxiedBy: nalasex01a.na.qualcomm.com (10.47.209.196) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: Txm6gMjIyx9QJMHNU61otRjXuRRVDzZ7 X-Proofpoint-ORIG-GUID: Txm6gMjIyx9QJMHNU61otRjXuRRVDzZ7 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 suspectscore=0 adultscore=0 phishscore=0 mlxscore=0 clxscore=1015 mlxlogscore=999 impostorscore=0 malwarescore=0 bulkscore=0 spamscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312160001 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785396263062327323 X-GMAIL-MSGID: 1785396263062327323 On Qualcomm platforms, there is a firmware entity which controls access to physical pages. In order to share memory with another VM, this entity needs to be informed that the guest VM should have access to the memory. Co-developed-by: Prakruthi Deepak Heragu Signed-off-by: Prakruthi Deepak Heragu Signed-off-by: Elliot Berman --- drivers/virt/gunyah/Kconfig | 4 + drivers/virt/gunyah/Makefile | 1 + drivers/virt/gunyah/gunyah_platform_hooks.c | 115 ++++++++++++++++++++++++++++ drivers/virt/gunyah/rsc_mgr.h | 10 +++ drivers/virt/gunyah/rsc_mgr_rpc.c | 20 ++++- drivers/virt/gunyah/vm_mgr_mem.c | 32 +++++--- include/linux/gunyah_rsc_mgr.h | 37 +++++++++ 7 files changed, 206 insertions(+), 13 deletions(-) diff --git a/drivers/virt/gunyah/Kconfig b/drivers/virt/gunyah/Kconfig index 6f4c85db80b5..23ba523d25dc 100644 --- a/drivers/virt/gunyah/Kconfig +++ b/drivers/virt/gunyah/Kconfig @@ -3,6 +3,7 @@ config GUNYAH tristate "Gunyah Virtualization drivers" depends on ARM64 + select GUNYAH_PLATFORM_HOOKS help The Gunyah drivers are the helper interfaces that run in a guest VM such as basic inter-VM IPC and signaling mechanisms, and higher level @@ -10,3 +11,6 @@ config GUNYAH Say Y/M here to enable the drivers needed to interact in a Gunyah virtual environment. + +config GUNYAH_PLATFORM_HOOKS + tristate diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index ccce90438434..09e0fbca33fe 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -3,3 +3,4 @@ gunyah_rsc_mgr-y += rsc_mgr.o rsc_mgr_rpc.o vm_mgr.o vm_mgr_mem.o obj-$(CONFIG_GUNYAH) += gunyah.o gunyah_rsc_mgr.o +obj-$(CONFIG_GUNYAH_PLATFORM_HOOKS) += gunyah_platform_hooks.o diff --git a/drivers/virt/gunyah/gunyah_platform_hooks.c b/drivers/virt/gunyah/gunyah_platform_hooks.c new file mode 100644 index 000000000000..d2d6df21b784 --- /dev/null +++ b/drivers/virt/gunyah/gunyah_platform_hooks.c @@ -0,0 +1,115 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include +#include +#include +#include + +#include "rsc_mgr.h" + +static const struct gunyah_rm_platform_ops *rm_platform_ops; +static DECLARE_RWSEM(rm_platform_ops_lock); + +int gunyah_rm_platform_pre_mem_share(struct gunyah_rm *rm, + struct gunyah_rm_mem_parcel *mem_parcel) +{ + int ret = 0; + + down_read(&rm_platform_ops_lock); + if (rm_platform_ops && rm_platform_ops->pre_mem_share) + ret = rm_platform_ops->pre_mem_share(rm, mem_parcel); + up_read(&rm_platform_ops_lock); + return ret; +} +EXPORT_SYMBOL_GPL(gunyah_rm_platform_pre_mem_share); + +int gunyah_rm_platform_post_mem_reclaim(struct gunyah_rm *rm, + struct gunyah_rm_mem_parcel *mem_parcel) +{ + int ret = 0; + + down_read(&rm_platform_ops_lock); + if (rm_platform_ops && rm_platform_ops->post_mem_reclaim) + ret = rm_platform_ops->post_mem_reclaim(rm, mem_parcel); + up_read(&rm_platform_ops_lock); + return ret; +} +EXPORT_SYMBOL_GPL(gunyah_rm_platform_post_mem_reclaim); + +int gunyah_rm_platform_pre_demand_page(struct gunyah_rm *rm, u16 vmid, + u32 flags, struct folio *folio) +{ + int ret = 0; + + down_read(&rm_platform_ops_lock); + if (rm_platform_ops && rm_platform_ops->pre_demand_page) + ret = rm_platform_ops->pre_demand_page(rm, vmid, flags, folio); + up_read(&rm_platform_ops_lock); + return ret; +} +EXPORT_SYMBOL_GPL(gunyah_rm_platform_pre_demand_page); + +int gunyah_rm_platform_reclaim_demand_page(struct gunyah_rm *rm, u16 vmid, + u32 flags, struct folio *folio) +{ + int ret = 0; + + down_read(&rm_platform_ops_lock); + if (rm_platform_ops && rm_platform_ops->pre_demand_page) + ret = rm_platform_ops->release_demand_page(rm, vmid, flags, + folio); + up_read(&rm_platform_ops_lock); + return ret; +} +EXPORT_SYMBOL_GPL(gunyah_rm_platform_reclaim_demand_page); + +int gunyah_rm_register_platform_ops( + const struct gunyah_rm_platform_ops *platform_ops) +{ + int ret = 0; + + down_write(&rm_platform_ops_lock); + if (!rm_platform_ops) + rm_platform_ops = platform_ops; + else + ret = -EEXIST; + up_write(&rm_platform_ops_lock); + return ret; +} +EXPORT_SYMBOL_GPL(gunyah_rm_register_platform_ops); + +void gunyah_rm_unregister_platform_ops( + const struct gunyah_rm_platform_ops *platform_ops) +{ + down_write(&rm_platform_ops_lock); + if (rm_platform_ops == platform_ops) + rm_platform_ops = NULL; + up_write(&rm_platform_ops_lock); +} +EXPORT_SYMBOL_GPL(gunyah_rm_unregister_platform_ops); + +static void _devm_gunyah_rm_unregister_platform_ops(void *data) +{ + gunyah_rm_unregister_platform_ops( + (const struct gunyah_rm_platform_ops *)data); +} + +int devm_gunyah_rm_register_platform_ops( + struct device *dev, const struct gunyah_rm_platform_ops *ops) +{ + int ret; + + ret = gunyah_rm_register_platform_ops(ops); + if (ret) + return ret; + + return devm_add_action(dev, _devm_gunyah_rm_unregister_platform_ops, + (void *)ops); +} +EXPORT_SYMBOL_GPL(devm_gunyah_rm_register_platform_ops); + +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("Gunyah Platform Hooks"); diff --git a/drivers/virt/gunyah/rsc_mgr.h b/drivers/virt/gunyah/rsc_mgr.h index 4ce6ec3668d4..65ffcb63754a 100644 --- a/drivers/virt/gunyah/rsc_mgr.h +++ b/drivers/virt/gunyah/rsc_mgr.h @@ -15,4 +15,14 @@ int gunyah_rm_call(struct gunyah_rm *rsc_mgr, u32 message_id, const void *req_buf, size_t req_buf_size, void **resp_buf, size_t *resp_buf_size); +int gunyah_rm_platform_pre_mem_share(struct gunyah_rm *rm, + struct gunyah_rm_mem_parcel *mem_parcel); +int gunyah_rm_platform_post_mem_reclaim( + struct gunyah_rm *rm, struct gunyah_rm_mem_parcel *mem_parcel); + +int gunyah_rm_platform_pre_demand_page(struct gunyah_rm *rm, u16 vmid, + u32 flags, struct folio *folio); +int gunyah_rm_platform_reclaim_demand_page(struct gunyah_rm *rm, u16 vmid, + u32 flags, struct folio *folio); + #endif diff --git a/drivers/virt/gunyah/rsc_mgr_rpc.c b/drivers/virt/gunyah/rsc_mgr_rpc.c index 9954512dbdde..ce68933f9968 100644 --- a/drivers/virt/gunyah/rsc_mgr_rpc.c +++ b/drivers/virt/gunyah/rsc_mgr_rpc.c @@ -241,6 +241,12 @@ int gunyah_rm_mem_share(struct gunyah_rm *rm, struct gunyah_rm_mem_parcel *p) if (!msg) return -ENOMEM; + ret = gunyah_rm_platform_pre_mem_share(rm, p); + if (ret) { + kfree(msg); + return ret; + } + req_header = msg; acl = (void *)req_header + sizeof(*req_header); mem = (void *)acl + acl_size; @@ -266,8 +272,10 @@ int gunyah_rm_mem_share(struct gunyah_rm *rm, struct gunyah_rm_mem_parcel *p) &resp_size); kfree(msg); - if (ret) + if (ret) { + gunyah_rm_platform_post_mem_reclaim(rm, p); return ret; + } p->mem_handle = le32_to_cpu(*resp); kfree(resp); @@ -298,9 +306,15 @@ int gunyah_rm_mem_reclaim(struct gunyah_rm *rm, struct gunyah_rm_mem_release_req req = { .mem_handle = cpu_to_le32(parcel->mem_handle), }; + int ret; - return gunyah_rm_call(rm, GUNYAH_RM_RPC_MEM_RECLAIM, &req, sizeof(req), - NULL, NULL); + ret = gunyah_rm_call(rm, GUNYAH_RM_RPC_MEM_RECLAIM, &req, sizeof(req), + NULL, NULL); + /* Only call platform mem reclaim hooks if we reclaimed the memory */ + if (ret) + return ret; + + return gunyah_rm_platform_post_mem_reclaim(rm, parcel); } /** diff --git a/drivers/virt/gunyah/vm_mgr_mem.c b/drivers/virt/gunyah/vm_mgr_mem.c index d61fceeeac60..e1675633804f 100644 --- a/drivers/virt/gunyah/vm_mgr_mem.c +++ b/drivers/virt/gunyah/vm_mgr_mem.c @@ -9,6 +9,7 @@ #include #include +#include "rsc_mgr.h" #include "vm_mgr.h" #define WRITE_TAG (1 << 0) @@ -120,7 +121,7 @@ int gunyah_vm_provide_folio(struct gunyah_vm *ghvm, struct folio *folio, size_t size = folio_size(folio); enum gunyah_error gunyah_error; unsigned long tag = 0; - int ret; + int ret, tmp; if (share) { guest_extent = __first_resource(&ghvm->guest_shared_extent_ticket); @@ -159,6 +160,11 @@ int gunyah_vm_provide_folio(struct gunyah_vm *ghvm, struct folio *folio, else /* !share && !write */ access = GUNYAH_PAGETABLE_ACCESS_RX; + ret = gunyah_rm_platform_pre_demand_page(ghvm->rm, ghvm->vmid, access, + folio); + if (ret) + goto remove; + gunyah_error = gunyah_hypercall_memextent_donate(donate_flags(share), host_extent->capid, guest_extent->capid, @@ -167,7 +173,7 @@ int gunyah_vm_provide_folio(struct gunyah_vm *ghvm, struct folio *folio, pr_err("Failed to donate memory for guest address 0x%016llx: %d\n", gpa, gunyah_error); ret = gunyah_error_remap(gunyah_error); - goto remove; + goto platform_release; } extent_attrs = @@ -196,6 +202,14 @@ int gunyah_vm_provide_folio(struct gunyah_vm *ghvm, struct folio *folio, if (gunyah_error != GUNYAH_ERROR_OK) pr_err("Failed to reclaim memory donation for guest address 0x%016llx: %d\n", gpa, gunyah_error); +platform_release: + tmp = gunyah_rm_platform_reclaim_demand_page(ghvm->rm, ghvm->vmid, + access, folio); + if (tmp) { + pr_err("Platform failed to reclaim memory for guest address 0x%016llx: %d", + gpa, tmp); + return ret; + } remove: mtree_erase(&ghvm->gm, gfn); return ret; @@ -272,14 +286,12 @@ int gunyah_vm_reclaim_folio(struct gunyah_vm *ghvm, u64 gfn) else /* !share && !write */ access = GUNYAH_PAGETABLE_ACCESS_RX; - gunyah_error = gunyah_hypercall_memextent_donate(donate_flags(share), - guest_extent->capid, - host_extent->capid, pa, - size); - if (gunyah_error != GUNYAH_ERROR_OK) { - pr_err("Failed to reclaim memory donation for guest address 0x%016llx: %d\n", - gfn << PAGE_SHIFT, gunyah_error); - ret = gunyah_error_remap(gunyah_error); + ret = gunyah_rm_platform_reclaim_demand_page(ghvm->rm, ghvm->vmid, + access, folio); + if (ret) { + pr_err_ratelimited( + "Platform failed to reclaim memory for guest address 0x%016llx: %d", + gunyah_gfn_to_gpa(gfn), ret); goto err; } diff --git a/include/linux/gunyah_rsc_mgr.h b/include/linux/gunyah_rsc_mgr.h index 0f61c2b10817..f177f8dbcd4c 100644 --- a/include/linux/gunyah_rsc_mgr.h +++ b/include/linux/gunyah_rsc_mgr.h @@ -168,4 +168,41 @@ gunyah_rm_alloc_resource(struct gunyah_rm *rm, struct gunyah_rm_hyp_resource *hyp_resource); void gunyah_rm_free_resource(struct gunyah_resource *ghrsc); +struct gunyah_rm_platform_ops { + int (*pre_mem_share)(struct gunyah_rm *rm, + struct gunyah_rm_mem_parcel *mem_parcel); + int (*post_mem_reclaim)(struct gunyah_rm *rm, + struct gunyah_rm_mem_parcel *mem_parcel); + + int (*pre_demand_page)(struct gunyah_rm *rm, u16 vmid, u32 flags, + struct folio *folio); + int (*release_demand_page)(struct gunyah_rm *rm, u16 vmid, u32 flags, + struct folio *folio); +}; + +#if IS_ENABLED(CONFIG_GUNYAH_PLATFORM_HOOKS) +int gunyah_rm_register_platform_ops( + const struct gunyah_rm_platform_ops *platform_ops); +void gunyah_rm_unregister_platform_ops( + const struct gunyah_rm_platform_ops *platform_ops); +int devm_gunyah_rm_register_platform_ops( + struct device *dev, const struct gunyah_rm_platform_ops *ops); +#else +static inline int gunyah_rm_register_platform_ops( + const struct gunyah_rm_platform_ops *platform_ops) +{ + return 0; +} +static inline void gunyah_rm_unregister_platform_ops( + const struct gunyah_rm_platform_ops *platform_ops) +{ +} +static inline int +devm_gunyah_rm_register_platform_ops(struct device *dev, + const struct gunyah_rm_platform_ops *ops) +{ + return 0; +} +#endif + #endif From patchwork Sat Dec 16 00:20:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 179694 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:3b04:b0:fb:cd0c:d3e with SMTP id c4csp9667199dys; Fri, 15 Dec 2023 16:27:31 -0800 (PST) X-Google-Smtp-Source: AGHT+IFCO9JTVCWkrXODVaU6PArkM826OV9KMTnM+iYdawXM4n00R8YBze7UZ09eP606+UIPX+ok X-Received: by 2002:a17:906:20c3:b0:a1f:36e3:127d with SMTP id c3-20020a17090620c300b00a1f36e3127dmr7546123ejc.11.1702686450858; Fri, 15 Dec 2023 16:27:30 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702686450; cv=none; d=google.com; s=arc-20160816; b=BJNvlD+GuRu88a0gmPWdduszyIM95DuHV6d6i+KHnFoMLkpD5XH2qoryue+g7ttWSM /6wLH8T/D5T38nkbxInWFC+qp9o5IjsLFCkYrLDz3CwpJBke8YqFjHMBz41GaAzfHpZ9 t1vJcMwnZDcO+c9TXG5ELJY3OM+/GpNRzjqrx2Rzu7x0OB626mSqDLCut/k8znffYYWH vjIcdmtEwBZERvMsAjGhs6mtrEZy8tVUMXw2Mh1F88VEZVbhvvZasj+7ZmHLjJRHbspf NeRmimBMZ/NvFfFoEzX961hfJ8NfQopVYRNJfDCWda6kzeW1HXzH2fU/mkh7AyyAHLHE u9xQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:list-unsubscribe:list-subscribe:list-id:precedence :subject:date:from:dkim-signature; bh=w8KxVytBtry1FMZaLZUaG0sJPTbX8kFQw8U2uTkBjF4=; fh=Pv/sMsQia5wmqAOYHIQUkkUFpDKhZjuODj71Ojb2ybw=; b=S3kZOoLVZ1LQ4j0AsGSx/YydVFg9+oV0c5oM2PXxMLi6fJYb2zSELobBmw9NbPYkul ylAp2feujkyE7qMRrl7zcJGxgDC+0m8DfSl8gN0aVJNEKowFgE3PCDZqdrFmAfAB8OsW yqo6Szv94cTnwuy7ofDe0izBKKcxlI6phoNZ7unTfOa2cgkQAqpNlFYGc1mCRPcEDzbA 0utsQx3qKIThxM+rPFQoIwwAgMi+q4S+e0kmBWmfEnRmc+tNPg866bhra+3I99LNa5CT ysBgMuZWcaylwDnLA7+1PbtQIFPcRF+JwScr9xEZg6GI0pNRu3ettBs/kmL589q8VNrJ mcIg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=GbHzEV6H; spf=pass (google.com: domain of linux-kernel+bounces-1884-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1884-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id p26-20020a170906b21a00b00a23192537desi994067ejz.409.2023.12.15.16.27.30 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Dec 2023 16:27:30 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-1884-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=GbHzEV6H; spf=pass (google.com: domain of linux-kernel+bounces-1884-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1884-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 4386E1F25838 for ; Sat, 16 Dec 2023 00:27:30 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id BE3AD33CDC; Sat, 16 Dec 2023 00:21:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="GbHzEV6H" X-Original-To: linux-kernel@vger.kernel.org Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6C5B8101EA; Sat, 16 Dec 2023 00:21:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=quicinc.com Received: from pps.filterd (m0279864.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BG0I3Uq018440; Sat, 16 Dec 2023 00:21:12 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:date:subject:mime-version:content-type :content-transfer-encoding:message-id:references:in-reply-to:to :cc; s=qcppdkim1; bh=w8KxVytBtry1FMZaLZUaG0sJPTbX8kFQw8U2uTkBjF4 =; b=GbHzEV6HuN1auUq6L1vYts/+dF9x5gMjvL+2ZcKIhbGapFCgylEfOMCOfCE LZ3Y9IKAYSXgW+AwdTX+rmTOACwHANvCu+ovpoVz9XwYUdDAdPx+605FqrUh7z7r UP9RRw1Uj/eFBdZERaV/ZmOkRcn1pmRWXqgQdRy4rsHLMqqfFgQu+EaARkiYk7LY Tu0JuUksIaX5lKEi4Qbiy0L2FWNpZhQSj1dM3ciFNvP3VNCGHemvhIeojLLxYRf9 Urw3ifQ9u2O+Ohxly9yUcJ0maMNYo7OWLcnMcN/yABlt3YhTTKLHOzqwDfJh0lRB e5za2CPdmf7KBnwOQ1b220NaJKg== Received: from nasanppmta02.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v0hqcjf8p-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:12 +0000 (GMT) Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA02.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 3BG0LBHU014892 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:11 GMT Received: from [169.254.0.1] (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.40; Fri, 15 Dec 2023 16:21:10 -0800 From: Elliot Berman Date: Fri, 15 Dec 2023 16:20:59 -0800 Subject: [PATCH RFC v15 18/30] virt: gunyah: Add IO handlers Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <20231215-gunyah-v15-18-192a5d872a30@quicinc.com> References: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> In-Reply-To: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> To: Alex Elder , Srinivas Kandagatla , Murali Nalajal , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Philip Derrin , Prakruthi Deepak Heragu , Jonathan Corbet , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Catalin Marinas , Will Deacon , Konrad Dybcio , Bjorn Andersson , Dmitry Baryshkov , "Fuad Tabba" , Sean Christopherson CC: , , , , , Elliot Berman X-Mailer: b4 0.13-dev X-ClientProxiedBy: nalasex01a.na.qualcomm.com (10.47.209.196) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: wY-LT0uYNxOXV5TvNoc6Z7YAYqLDVJrl X-Proofpoint-GUID: wY-LT0uYNxOXV5TvNoc6Z7YAYqLDVJrl X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 mlxscore=0 phishscore=0 adultscore=0 priorityscore=1501 impostorscore=0 lowpriorityscore=0 mlxlogscore=999 clxscore=1015 suspectscore=0 spamscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312160001 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785396060826083809 X-GMAIL-MSGID: 1785396147823829570 Add framework for VM functions to handle stage-2 write faults from Gunyah guest virtual machines. IO handlers have a range of addresses which they apply to. Optionally, they may apply to only when the value written matches the IO handler's value. Reviewed-by: Alex Elder Co-developed-by: Prakruthi Deepak Heragu Signed-off-by: Prakruthi Deepak Heragu Signed-off-by: Elliot Berman --- drivers/virt/gunyah/vm_mgr.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/virt/gunyah/vm_mgr.h b/drivers/virt/gunyah/vm_mgr.h index d2f6ac212169..5d1ebce678ad 100644 --- a/drivers/virt/gunyah/vm_mgr.h +++ b/drivers/virt/gunyah/vm_mgr.h @@ -74,4 +74,6 @@ int gunyah_vm_provide_folio(struct gunyah_vm *ghvm, struct folio *folio, int gunyah_vm_reclaim_folio(struct gunyah_vm *ghvm, u64 gfn); void gunyah_vm_reclaim_memory(struct gunyah_vm *ghvm); +int gunyah_vm_mmio_write(struct gunyah_vm *ghvm, u64 addr, u32 len, u64 data); + #endif From patchwork Sat Dec 16 00:21:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 179702 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:3b04:b0:fb:cd0c:d3e with SMTP id c4csp9668202dys; Fri, 15 Dec 2023 16:30:17 -0800 (PST) X-Google-Smtp-Source: AGHT+IH2POQtueIFMyHP9XwBaWC5FSjDcLhBNm+yJ3L9So5S1I6FFYwkmR1WyaqfK56hZMaZ1/Ib X-Received: by 2002:a05:622a:16:b0:425:4043:8d6d with SMTP id x22-20020a05622a001600b0042540438d6dmr12693936qtw.136.1702686616855; Fri, 15 Dec 2023 16:30:16 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702686616; cv=none; d=google.com; s=arc-20160816; b=eQyjlsFWwrTCZdxFlux1eqhXRmAyWhupZ2Yu3FZTTdivMpPlfRP7TDUALgTbNui/gB kDAO2bhKV5osinyfmoc8/qzDschkLAX5DL0a/KGPwSpP7mf+PlbKMmEs+nsAHwhE4506 A7/4DMWSM8Al3DooiTEh4ceaSd4hh5lwODGwah7bxRcGH5UBFljRZO+6b2DFkV2apylf L79T92qLHSwPflEt6nPcJjeOCrwwEQ/QueHQ8PDyAZ33YHTsa+Z1hgzg8GDhhVFYeSbV S6W291iY5ltB6Mkl/SHx7ugcaCUjytMh/LcGC1T/+KuTbD61Utr3sSSaaUzlZeNehz0F /cdg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:list-unsubscribe:list-subscribe:list-id:precedence :subject:date:from:dkim-signature; bh=OOqhaOER/a0/uCwAVyWXXLkJyFyhTRipcrvwrNj4f1k=; fh=Pv/sMsQia5wmqAOYHIQUkkUFpDKhZjuODj71Ojb2ybw=; b=TWG5qfTKXcMA031eRVPs5Dz5WwQgXoVh4akC5335CbnvNtcJr8e4PahlmMU6ndrHNV YYw6MPDGMhee5nMFjI1hi7tWN1JnBkc2su3077VQZloGZEC6AIZV++SAhOUFhLTg0ELQ PAbdFL8MBYiVkMR6383wX5gBEeY75FrWoXAFP2+BaFPuE/fOLOlXJTeqNOtCZCyY+60F bcm3BUfiAFpvLaIUwJDx3TNDMvKsVDBSnYmNpSE8f/SJy9npgpGbpd7wLveGMKnyOtrh 0/djVliSCnwq8MiTx15Lbw71BeN+/Ne/oxFGYHPn8SWpQQIfgWxk+hqLRWz4uw/3FAQv 1+EA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=SqWUkSqo; spf=pass (google.com: domain of linux-kernel+bounces-1892-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1892-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id y24-20020ac85258000000b0042579ad34d1si17677375qtn.775.2023.12.15.16.30.16 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Dec 2023 16:30:16 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-1892-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=SqWUkSqo; spf=pass (google.com: domain of linux-kernel+bounces-1892-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1892-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 911BB1C2425C for ; Sat, 16 Dec 2023 00:30:16 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 687953D0C3; Sat, 16 Dec 2023 00:21:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="SqWUkSqo" X-Original-To: linux-kernel@vger.kernel.org Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3560032C62; Sat, 16 Dec 2023 00:21:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=quicinc.com Received: from pps.filterd (m0279863.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BFNswqx030328; Sat, 16 Dec 2023 00:21:13 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:date:subject:mime-version:content-type :content-transfer-encoding:message-id:references:in-reply-to:to :cc; s=qcppdkim1; bh=OOqhaOER/a0/uCwAVyWXXLkJyFyhTRipcrvwrNj4f1k =; b=SqWUkSqoXaNXtIw70ZH117LwPpbRcC+yuBbnoIGt3lyQQepY3cD0vGrY342 Ds0BonoqzDBwslaTQar7dS9C3c/yjsmUmUTJQ8y7BkHCKuhD9NjySanHiegWS8rC pdX7biYdG+w84JmOWPZOQltXLpIZrXoL37AWUhgB55ECeQ+IUEW+zDmJs8pdQgwJ l+2mmAWJGtN507U75c7rz3c1yWqgn303bRW7Zj0MvoWHIIs09Z63TVWevkBFs2FI zxKxrky5auJDvp8Sb0RFcIj/Zm0gMs05ItrJFDz54j2LOaSIL/xHwgHsMfrby7cJ U16Z0Dj8O5FNWnTt3/fRwy+AWgg== Received: from nasanppmta05.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v0up20rec-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:13 +0000 (GMT) Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA05.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 3BG0LCNZ006362 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:12 GMT Received: from [169.254.0.1] (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.40; Fri, 15 Dec 2023 16:21:11 -0800 From: Elliot Berman Date: Fri, 15 Dec 2023 16:21:00 -0800 Subject: [PATCH RFC v15 19/30] virt: gunyah: Add proxy-scheduled vCPUs Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <20231215-gunyah-v15-19-192a5d872a30@quicinc.com> References: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> In-Reply-To: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> To: Alex Elder , Srinivas Kandagatla , Murali Nalajal , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Philip Derrin , Prakruthi Deepak Heragu , Jonathan Corbet , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Catalin Marinas , Will Deacon , Konrad Dybcio , Bjorn Andersson , Dmitry Baryshkov , "Fuad Tabba" , Sean Christopherson CC: , , , , , Elliot Berman X-Mailer: b4 0.13-dev X-ClientProxiedBy: nalasex01a.na.qualcomm.com (10.47.209.196) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: B2gMzOL6OEg7uED_qPlGZYcaAmRwA1Eq X-Proofpoint-ORIG-GUID: B2gMzOL6OEg7uED_qPlGZYcaAmRwA1Eq X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 bulkscore=0 mlxscore=0 phishscore=0 adultscore=0 suspectscore=0 mlxlogscore=999 impostorscore=0 priorityscore=1501 clxscore=1015 malwarescore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312160001 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785396321806859791 X-GMAIL-MSGID: 1785396321806859791 Gunyah allows host virtual machines to schedule guest virtual machines and handle their MMIO accesses. vCPUs are presented to the host as a Gunyah resource and represented to userspace as a Gunyah VM function. Creating the vcpu VM function will create a file descriptor that: - can run an ioctl: GH_VCPU_RUN to schedule the guest vCPU until the next interrupt occurs on the host or when the guest vCPU can no longer be run. - can be mmap'd to share a gh_vcpu_run structure which can look up the reason why GH_VCPU_RUN returned and provide return values for MMIO access. Acked-by: Alex Elder Co-developed-by: Prakruthi Deepak Heragu Signed-off-by: Prakruthi Deepak Heragu Signed-off-by: Elliot Berman --- arch/arm64/gunyah/gunyah_hypercall.c | 27 ++ drivers/virt/gunyah/Makefile | 2 +- drivers/virt/gunyah/gunyah_vcpu.c | 542 +++++++++++++++++++++++++++++++++++ drivers/virt/gunyah/vm_mgr.c | 5 + drivers/virt/gunyah/vm_mgr.h | 1 + include/linux/gunyah.h | 10 + include/uapi/linux/gunyah.h | 163 +++++++++++ 7 files changed, 749 insertions(+), 1 deletion(-) diff --git a/arch/arm64/gunyah/gunyah_hypercall.c b/arch/arm64/gunyah/gunyah_hypercall.c index 468a5bcb98b1..47926df33876 100644 --- a/arch/arm64/gunyah/gunyah_hypercall.c +++ b/arch/arm64/gunyah/gunyah_hypercall.c @@ -42,6 +42,7 @@ EXPORT_SYMBOL_GPL(arch_is_gunyah_guest); #define GUNYAH_HYPERCALL_ADDRSPACE_MAP GUNYAH_HYPERCALL(0x802B) #define GUNYAH_HYPERCALL_ADDRSPACE_UNMAP GUNYAH_HYPERCALL(0x802C) #define GUNYAH_HYPERCALL_MEMEXTENT_DONATE GUNYAH_HYPERCALL(0x8061) +#define GUNYAH_HYPERCALL_VCPU_RUN GUNYAH_HYPERCALL(0x8065) /* clang-format on */ /** @@ -152,5 +153,31 @@ enum gunyah_error gunyah_hypercall_memextent_donate(u32 options, u64 from_capid, } EXPORT_SYMBOL_GPL(gunyah_hypercall_memextent_donate); +enum gunyah_error gunyah_hypercall_vcpu_run(u64 capid, u64 *resume_data, + struct gunyah_hypercall_vcpu_run_resp *resp) +{ + struct arm_smccc_1_2_regs args = { + .a0 = GUNYAH_HYPERCALL_VCPU_RUN, + .a1 = capid, + .a2 = resume_data[0], + .a3 = resume_data[1], + .a4 = resume_data[2], + /* C language says this will be implictly zero. Gunyah requires 0, so be explicit */ + .a5 = 0, + }; + struct arm_smccc_1_2_regs res; + + arm_smccc_1_2_hvc(&args, &res); + if (res.a0 == GUNYAH_ERROR_OK) { + resp->sized_state = res.a1; + resp->state_data[0] = res.a2; + resp->state_data[1] = res.a3; + resp->state_data[2] = res.a4; + } + + return res.a0; +} +EXPORT_SYMBOL_GPL(gunyah_hypercall_vcpu_run); + MODULE_LICENSE("GPL"); MODULE_DESCRIPTION("Gunyah Hypervisor Hypercalls"); diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index 09e0fbca33fe..ffcde0e0ccfa 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -2,5 +2,5 @@ gunyah_rsc_mgr-y += rsc_mgr.o rsc_mgr_rpc.o vm_mgr.o vm_mgr_mem.o -obj-$(CONFIG_GUNYAH) += gunyah.o gunyah_rsc_mgr.o +obj-$(CONFIG_GUNYAH) += gunyah.o gunyah_rsc_mgr.o gunyah_vcpu.o obj-$(CONFIG_GUNYAH_PLATFORM_HOOKS) += gunyah_platform_hooks.o diff --git a/drivers/virt/gunyah/gunyah_vcpu.c b/drivers/virt/gunyah/gunyah_vcpu.c new file mode 100644 index 000000000000..aa40f5977cbf --- /dev/null +++ b/drivers/virt/gunyah/gunyah_vcpu.c @@ -0,0 +1,542 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "vm_mgr.h" + +#include + +#define MAX_VCPU_NAME 20 /* gh-vcpu:strlen(U32::MAX)+NUL */ + +/** + * struct gunyah_vcpu - Track an instance of gunyah vCPU + * @f: Function instance (how we get associated with the main VM) + * @rsc: Pointer to the Gunyah vCPU resource, will be NULL until VM starts + * @run_lock: One userspace thread at a time should run the vCPU + * @ghvm: Pointer to the main VM struct; quicker look up than going through + * @f->ghvm + * @vcpu_run: Pointer to page shared with userspace to communicate vCPU state + * @state: Our copy of the state of the vCPU, since userspace could trick + * kernel to behave incorrectly if we relied on @vcpu_run + * @mmio_read_len: Our copy of @vcpu_run->mmio.len; see also @state + * @mmio_addr: Our copy of @vcpu_run->mmio.phys_addr; see also @state + * @ready: if vCPU goes to sleep, hypervisor reports to us that it's sleeping + * and will signal interrupt (from @rsc) when it's time to wake up. + * This completion signals that we can run vCPU again. + * @nb: When VM exits, the status of VM is reported via @vcpu_run->status. + * We need to track overall VM status, and the nb gives us the updates from + * Resource Manager. + * @ticket: resource ticket to claim vCPU# for the VM + * @kref: Reference counter + */ +struct gunyah_vcpu { + struct gunyah_vm_function_instance *f; + struct gunyah_resource *rsc; + struct mutex run_lock; + struct gunyah_vm *ghvm; + + struct gunyah_vcpu_run *vcpu_run; + + /** + * Track why vcpu_run left last time around. + * Mirrors the vcpu_run structure given to userspace, except + * kernel shouldn't trust userspace to not modify the state + * reported. + */ + enum { + GUNYAH_VCPU_UNKNOWN = 0, + GUNYAH_VCPU_READY, + GUNYAH_VCPU_MMIO_READ, + GUNYAH_VCPU_MMIO_WRITE, + GUNYAH_VCPU_SYSTEM_DOWN, + } state; + u8 mmio_read_len; + u64 mmio_addr; + + struct completion ready; + + struct notifier_block nb; + struct gunyah_vm_resource_ticket ticket; + struct kref kref; +}; + +static void vcpu_release(struct kref *kref) +{ + struct gunyah_vcpu *vcpu = container_of(kref, struct gunyah_vcpu, kref); + + free_page((unsigned long)vcpu->vcpu_run); + kfree(vcpu); +} + +/* + * When hypervisor allows us to schedule vCPU again, it gives us an interrupt + */ +static irqreturn_t gunyah_vcpu_irq_handler(int irq, void *data) +{ + struct gunyah_vcpu *vcpu = data; + + complete(&vcpu->ready); + return IRQ_HANDLED; +} + +static bool +gunyah_handle_mmio(struct gunyah_vcpu *vcpu, u64 state_data[3], + const struct gunyah_hypercall_vcpu_run_resp *vcpu_run_resp) +{ + int ret = 0; + u64 addr = vcpu_run_resp->state_data[0], + len = vcpu_run_resp->state_data[1], + data = vcpu_run_resp->state_data[2]; + + if (WARN_ON(len > sizeof(u64))) + len = sizeof(u64); + + if (vcpu_run_resp->state == GUNYAH_VCPU_ADDRSPACE_VMMIO_READ) { + vcpu->vcpu_run->mmio.is_write = 0; + /* Record that we need to give vCPU user's supplied value next gunyah_vcpu_run() */ + vcpu->state = GUNYAH_VCPU_MMIO_READ; + vcpu->mmio_read_len = len; + } else { /* GUNYAH_VCPU_ADDRSPACE_VMMIO_WRITE */ + /* Try internal handlers first */ + ret = gunyah_vm_mmio_write(vcpu->ghvm, addr, len, data); + if (!ret) + return true; + + /* Give userspace the info */ + vcpu->vcpu_run->mmio.is_write = 1; + memcpy(vcpu->vcpu_run->mmio.data, &data, len); + vcpu->state = GUNYAH_VCPU_MMIO_WRITE; + } + + vcpu->vcpu_run->mmio.phys_addr = addr; + vcpu->vcpu_run->mmio.len = len; + vcpu->vcpu_run->exit_reason = GUNYAH_VCPU_EXIT_MMIO; + + return false; +} + +static int gunyah_handle_mmio_resume(struct gunyah_vcpu *vcpu, u64 state_data[3]) +{ + switch (vcpu->vcpu_run->mmio.resume_action) { + case GUNYAH_VCPU_RESUME_HANDLED: + if (vcpu->state == GUNYAH_VCPU_MMIO_READ) { + if (unlikely(vcpu->mmio_read_len > + sizeof(state_data[0]))) + vcpu->mmio_read_len = sizeof(state_data[0]); + memcpy(&state_data[0], vcpu->vcpu_run->mmio.data, + vcpu->mmio_read_len); + } + state_data[1] = GUNYAH_ADDRSPACE_VMMIO_ACTION_EMULATE; + break; + case GUNYAH_VCPU_RESUME_FAULT: + state_data[1] = GUNYAH_ADDRSPACE_VMMIO_ACTION_FAULT; + break; + case GUNYAH_VCPU_RESUME_RETRY: + state_data[1] = GUNYAH_ADDRSPACE_VMMIO_ACTION_RETRY; + break; + default: + return -EINVAL; + } + + return 0; +} + +static int gunyah_vcpu_rm_notification(struct notifier_block *nb, + unsigned long action, void *data) +{ + struct gunyah_vcpu *vcpu = container_of(nb, struct gunyah_vcpu, nb); + struct gunyah_rm_vm_exited_payload *exit_payload = data; + + if (action == GUNYAH_RM_NOTIFICATION_VM_EXITED && + le16_to_cpu(exit_payload->vmid) == vcpu->ghvm->vmid) + complete(&vcpu->ready); + + return NOTIFY_OK; +} + +static inline enum gunyah_vm_status +remap_vm_status(enum gunyah_rm_vm_status rm_status) +{ + switch (rm_status) { + case GUNYAH_RM_VM_STATUS_INIT_FAILED: + return GUNYAH_VM_STATUS_LOAD_FAILED; + case GUNYAH_RM_VM_STATUS_EXITED: + return GUNYAH_VM_STATUS_EXITED; + default: + return GUNYAH_VM_STATUS_CRASHED; + } +} + +/** + * gunyah_vcpu_check_system() - Check whether VM as a whole is running + * @vcpu: Pointer to gunyah_vcpu + * + * Returns true if the VM is alive. + * Returns false if the vCPU is the VM is not alive (can only be that VM is shutting down). + */ +static bool gunyah_vcpu_check_system(struct gunyah_vcpu *vcpu) + __must_hold(&vcpu->run_lock) +{ + bool ret = true; + + down_read(&vcpu->ghvm->status_lock); + if (likely(vcpu->ghvm->vm_status == GUNYAH_RM_VM_STATUS_RUNNING)) + goto out; + + vcpu->vcpu_run->status.status = remap_vm_status(vcpu->ghvm->vm_status); + vcpu->vcpu_run->status.exit_info = vcpu->ghvm->exit_info; + vcpu->vcpu_run->exit_reason = GUNYAH_VCPU_EXIT_STATUS; + vcpu->state = GUNYAH_VCPU_SYSTEM_DOWN; + ret = false; +out: + up_read(&vcpu->ghvm->status_lock); + return ret; +} + +/** + * gunyah_vcpu_run() - Request Gunyah to begin scheduling this vCPU. + * @vcpu: The client descriptor that was obtained via gunyah_vcpu_alloc() + */ +static int gunyah_vcpu_run(struct gunyah_vcpu *vcpu) +{ + struct gunyah_hypercall_vcpu_run_resp vcpu_run_resp; + u64 resume_data[3] = { 0 }; + enum gunyah_error gunyah_error; + int ret = 0; + + if (!vcpu->f) + return -ENODEV; + + if (mutex_lock_interruptible(&vcpu->run_lock)) + return -ERESTARTSYS; + + if (!vcpu->rsc) { + ret = -ENODEV; + goto out; + } + + switch (vcpu->state) { + case GUNYAH_VCPU_UNKNOWN: + if (vcpu->ghvm->vm_status != GUNYAH_RM_VM_STATUS_RUNNING) { + /* Check if VM is up. If VM is starting, will block until VM is fully up + * since that thread does down_write. + */ + if (!gunyah_vcpu_check_system(vcpu)) + goto out; + } + vcpu->state = GUNYAH_VCPU_READY; + break; + case GUNYAH_VCPU_MMIO_READ: + case GUNYAH_VCPU_MMIO_WRITE: + ret = gunyah_handle_mmio_resume(vcpu, resume_data); + if (ret) + goto out; + vcpu->state = GUNYAH_VCPU_READY; + break; + case GUNYAH_VCPU_SYSTEM_DOWN: + goto out; + default: + break; + } + + while (!ret && !signal_pending(current)) { + if (vcpu->vcpu_run->immediate_exit) { + ret = -EINTR; + goto out; + } + + gunyah_error = gunyah_hypercall_vcpu_run( + vcpu->rsc->capid, resume_data, &vcpu_run_resp); + memset(resume_data, 0, sizeof(resume_data)); + if (gunyah_error == GUNYAH_ERROR_OK) { + switch (vcpu_run_resp.state) { + case GUNYAH_VCPU_STATE_READY: + if (need_resched()) + schedule(); + break; + case GUNYAH_VCPU_STATE_POWERED_OFF: + /* vcpu might be off because the VM is shut down. + * If so, it won't ever run again: exit back to user + */ + if (!gunyah_vcpu_check_system(vcpu)) + goto out; + /* Otherwise, another vcpu will turn it on (e.g. by PSCI) + * and hyp sends an interrupt to wake Linux up. + */ + fallthrough; + case GUNYAH_VCPU_STATE_EXPECTS_WAKEUP: + ret = wait_for_completion_interruptible( + &vcpu->ready); + /* reinitialize completion before next hypercall. If we reinitialize + * after the hypercall, interrupt may have already come before + * re-initializing the completion and then end up waiting for + * event that already happened. + */ + reinit_completion(&vcpu->ready); + /* Check system status again. Completion might've + * come from gunyah_vcpu_rm_notification + */ + if (!ret && !gunyah_vcpu_check_system(vcpu)) + goto out; + break; + case GUNYAH_VCPU_STATE_BLOCKED: + schedule(); + break; + case GUNYAH_VCPU_ADDRSPACE_VMMIO_READ: + case GUNYAH_VCPU_ADDRSPACE_VMMIO_WRITE: + if (!gunyah_handle_mmio(vcpu, resume_data, + &vcpu_run_resp)) + goto out; + break; + default: + pr_warn_ratelimited( + "Unknown vCPU state: %llx\n", + vcpu_run_resp.sized_state); + schedule(); + break; + } + } else if (gunyah_error == GUNYAH_ERROR_RETRY) { + schedule(); + } else { + ret = gunyah_error_remap(gunyah_error); + } + } + +out: + mutex_unlock(&vcpu->run_lock); + + if (signal_pending(current)) + return -ERESTARTSYS; + + return ret; +} + +static long gunyah_vcpu_ioctl(struct file *filp, unsigned int cmd, + unsigned long arg) +{ + struct gunyah_vcpu *vcpu = filp->private_data; + long ret = -ENOTTY; + + switch (cmd) { + case GUNYAH_VCPU_RUN: + ret = gunyah_vcpu_run(vcpu); + break; + case GUNYAH_VCPU_MMAP_SIZE: + ret = PAGE_SIZE; + break; + default: + break; + } + return ret; +} + +static int gunyah_vcpu_release(struct inode *inode, struct file *filp) +{ + struct gunyah_vcpu *vcpu = filp->private_data; + + gunyah_vm_put(vcpu->ghvm); + kref_put(&vcpu->kref, vcpu_release); + return 0; +} + +static vm_fault_t gunyah_vcpu_fault(struct vm_fault *vmf) +{ + struct gunyah_vcpu *vcpu = vmf->vma->vm_file->private_data; + struct page *page = NULL; + + if (vmf->pgoff == 0) + page = virt_to_page(vcpu->vcpu_run); + + get_page(page); + vmf->page = page; + return 0; +} + +static const struct vm_operations_struct gunyah_vcpu_ops = { + .fault = gunyah_vcpu_fault, +}; + +static int gunyah_vcpu_mmap(struct file *file, struct vm_area_struct *vma) +{ + vma->vm_ops = &gunyah_vcpu_ops; + return 0; +} + +static const struct file_operations gunyah_vcpu_fops = { + .owner = THIS_MODULE, + .unlocked_ioctl = gunyah_vcpu_ioctl, + .release = gunyah_vcpu_release, + .llseek = noop_llseek, + .mmap = gunyah_vcpu_mmap, +}; + +static bool gunyah_vcpu_populate(struct gunyah_vm_resource_ticket *ticket, + struct gunyah_resource *ghrsc) +{ + struct gunyah_vcpu *vcpu = + container_of(ticket, struct gunyah_vcpu, ticket); + int ret; + + mutex_lock(&vcpu->run_lock); + if (vcpu->rsc) { + pr_warn("vcpu%d already got a Gunyah resource. Check if multiple resources with same label were configured.\n", + vcpu->ticket.label); + ret = -EEXIST; + goto out; + } + + vcpu->rsc = ghrsc; + init_completion(&vcpu->ready); + + ret = request_irq(vcpu->rsc->irq, gunyah_vcpu_irq_handler, + IRQF_TRIGGER_RISING, "gunyah_vcpu", vcpu); + if (ret) + pr_warn("Failed to request vcpu irq %d: %d", vcpu->rsc->irq, + ret); + + enable_irq_wake(vcpu->rsc->irq); + +out: + mutex_unlock(&vcpu->run_lock); + return !ret; +} + +static void gunyah_vcpu_unpopulate(struct gunyah_vm_resource_ticket *ticket, + struct gunyah_resource *ghrsc) +{ + struct gunyah_vcpu *vcpu = + container_of(ticket, struct gunyah_vcpu, ticket); + + vcpu->vcpu_run->immediate_exit = true; + complete_all(&vcpu->ready); + mutex_lock(&vcpu->run_lock); + free_irq(vcpu->rsc->irq, vcpu); + vcpu->rsc = NULL; + mutex_unlock(&vcpu->run_lock); +} + +static long gunyah_vcpu_bind(struct gunyah_vm_function_instance *f) +{ + struct gunyah_fn_vcpu_arg *arg = f->argp; + struct gunyah_vcpu *vcpu; + char name[MAX_VCPU_NAME]; + struct file *file; + struct page *page; + int fd; + long r; + + if (f->arg_size != sizeof(*arg)) + return -EINVAL; + + vcpu = kzalloc(sizeof(*vcpu), GFP_KERNEL); + if (!vcpu) + return -ENOMEM; + + vcpu->f = f; + f->data = vcpu; + mutex_init(&vcpu->run_lock); + kref_init(&vcpu->kref); + + page = alloc_page(GFP_KERNEL | __GFP_ZERO); + if (!page) { + r = -ENOMEM; + goto err_destroy_vcpu; + } + vcpu->vcpu_run = page_address(page); + + vcpu->ticket.resource_type = GUNYAH_RESOURCE_TYPE_VCPU; + vcpu->ticket.label = arg->id; + vcpu->ticket.owner = THIS_MODULE; + vcpu->ticket.populate = gunyah_vcpu_populate; + vcpu->ticket.unpopulate = gunyah_vcpu_unpopulate; + + r = gunyah_vm_add_resource_ticket(f->ghvm, &vcpu->ticket); + if (r) + goto err_destroy_page; + + if (!gunyah_vm_get(f->ghvm)) { + r = -ENODEV; + goto err_remove_resource_ticket; + } + vcpu->ghvm = f->ghvm; + + vcpu->nb.notifier_call = gunyah_vcpu_rm_notification; + /* Ensure we run after the vm_mgr handles the notification and does + * any necessary state changes. We wake up to check the new state. + */ + vcpu->nb.priority = -1; + r = gunyah_rm_notifier_register(f->rm, &vcpu->nb); + if (r) + goto err_put_gunyah_vm; + + kref_get(&vcpu->kref); + + fd = get_unused_fd_flags(O_CLOEXEC); + if (fd < 0) { + r = fd; + goto err_notifier; + } + + snprintf(name, sizeof(name), "gh-vcpu:%u", vcpu->ticket.label); + file = anon_inode_getfile(name, &gunyah_vcpu_fops, vcpu, O_RDWR); + if (IS_ERR(file)) { + r = PTR_ERR(file); + goto err_put_fd; + } + + fd_install(fd, file); + + return fd; +err_put_fd: + put_unused_fd(fd); +err_notifier: + gunyah_rm_notifier_unregister(f->rm, &vcpu->nb); +err_put_gunyah_vm: + gunyah_vm_put(vcpu->ghvm); +err_remove_resource_ticket: + gunyah_vm_remove_resource_ticket(f->ghvm, &vcpu->ticket); +err_destroy_page: + free_page((unsigned long)vcpu->vcpu_run); +err_destroy_vcpu: + kfree(vcpu); + return r; +} + +static void gunyah_vcpu_unbind(struct gunyah_vm_function_instance *f) +{ + struct gunyah_vcpu *vcpu = f->data; + + gunyah_rm_notifier_unregister(f->rm, &vcpu->nb); + gunyah_vm_remove_resource_ticket(vcpu->ghvm, &vcpu->ticket); + vcpu->f = NULL; + + kref_put(&vcpu->kref, vcpu_release); +} + +static bool gunyah_vcpu_compare(const struct gunyah_vm_function_instance *f, + const void *arg, size_t size) +{ + const struct gunyah_fn_vcpu_arg *instance = f->argp, *other = arg; + + if (sizeof(*other) != size) + return false; + + return instance->id == other->id; +} + +DECLARE_GUNYAH_VM_FUNCTION_INIT(vcpu, GUNYAH_FN_VCPU, 1, gunyah_vcpu_bind, + gunyah_vcpu_unbind, gunyah_vcpu_compare); +MODULE_DESCRIPTION("Gunyah vCPU Function"); +MODULE_LICENSE("GPL"); diff --git a/drivers/virt/gunyah/vm_mgr.c b/drivers/virt/gunyah/vm_mgr.c index a5450dc8cc28..9e402e6ea7fe 100644 --- a/drivers/virt/gunyah/vm_mgr.c +++ b/drivers/virt/gunyah/vm_mgr.c @@ -443,6 +443,11 @@ static int gunyah_vm_rm_notification_exited(struct gunyah_vm *ghvm, void *data) down_write(&ghvm->status_lock); ghvm->vm_status = GUNYAH_RM_VM_STATUS_EXITED; + ghvm->exit_info.type = le16_to_cpu(payload->exit_type); + ghvm->exit_info.reason_size = le32_to_cpu(payload->exit_reason_size); + memcpy(&ghvm->exit_info.reason, payload->exit_reason, + min(GUNYAH_VM_MAX_EXIT_REASON_SIZE, + ghvm->exit_info.reason_size)); up_write(&ghvm->status_lock); wake_up(&ghvm->vm_status_wait); diff --git a/drivers/virt/gunyah/vm_mgr.h b/drivers/virt/gunyah/vm_mgr.h index 5d1ebce678ad..348dbeda57ec 100644 --- a/drivers/virt/gunyah/vm_mgr.h +++ b/drivers/virt/gunyah/vm_mgr.h @@ -48,6 +48,7 @@ struct gunyah_vm { enum gunyah_rm_vm_status vm_status; wait_queue_head_t vm_status_wait; struct rw_semaphore status_lock; + struct gunyah_vm_exit_info exit_info; struct kref kref; struct mutex fn_lock; diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index 86f48dc1d56d..ae9be5eef7a9 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -230,4 +230,14 @@ struct gunyah_hypercall_vcpu_run_resp { u64 state_data[3]; }; +enum { + GUNYAH_ADDRSPACE_VMMIO_ACTION_EMULATE = 0, + GUNYAH_ADDRSPACE_VMMIO_ACTION_RETRY = 1, + GUNYAH_ADDRSPACE_VMMIO_ACTION_FAULT = 2, +}; + +enum gunyah_error +gunyah_hypercall_vcpu_run(u64 capid, u64 *resume_data, + struct gunyah_hypercall_vcpu_run_resp *resp); + #endif diff --git a/include/uapi/linux/gunyah.h b/include/uapi/linux/gunyah.h index 1b7cb5fde70a..46f7d3aa61d0 100644 --- a/include/uapi/linux/gunyah.h +++ b/include/uapi/linux/gunyah.h @@ -25,8 +25,33 @@ */ #define GUNYAH_VM_START _IO(GUNYAH_IOCTL_TYPE, 0x3) +/** + * enum gunyah_fn_type - Valid types of Gunyah VM functions + * @GUNYAH_FN_VCPU: create a vCPU instance to control a vCPU + * &struct gunyah_fn_desc.arg is a pointer to &struct gunyah_fn_vcpu_arg + * Return: file descriptor to manipulate the vcpu. + */ +enum gunyah_fn_type { + GUNYAH_FN_VCPU = 1, +}; + #define GUNYAH_FN_MAX_ARG_SIZE 256 +/** + * struct gunyah_fn_vcpu_arg - Arguments to create a vCPU. + * @id: vcpu id + * + * Create this function with &GUNYAH_VM_ADD_FUNCTION using type &GUNYAH_FN_VCPU. + * + * The vcpu type will register with the VM Manager to expect to control + * vCPU number `vcpu_id`. It returns a file descriptor allowing interaction with + * the vCPU. See the Gunyah vCPU API description sections for interacting with + * the Gunyah vCPU file descriptors. + */ +struct gunyah_fn_vcpu_arg { + __u32 id; +}; + /** * struct gunyah_fn_desc - Arguments to create a VM function * @type: Type of the function. See &enum gunyah_fn_type. @@ -43,4 +68,142 @@ struct gunyah_fn_desc { #define GUNYAH_VM_ADD_FUNCTION _IOW(GUNYAH_IOCTL_TYPE, 0x4, struct gunyah_fn_desc) #define GUNYAH_VM_REMOVE_FUNCTION _IOW(GUNYAH_IOCTL_TYPE, 0x7, struct gunyah_fn_desc) +/* + * ioctls for vCPU fds + */ + +/** + * enum gunyah_vm_status - Stores status reason why VM is not runnable (exited). + * @GUNYAH_VM_STATUS_LOAD_FAILED: VM didn't start because it couldn't be loaded. + * @GUNYAH_VM_STATUS_EXITED: VM requested shutdown/reboot. + * Use &struct gunyah_vm_exit_info.reason for further details. + * @GUNYAH_VM_STATUS_CRASHED: VM state is unknown and has crashed. + */ +enum gunyah_vm_status { + GUNYAH_VM_STATUS_LOAD_FAILED = 1, + GUNYAH_VM_STATUS_EXITED = 2, + GUNYAH_VM_STATUS_CRASHED = 3, +}; + +/* + * Gunyah presently sends max 4 bytes of exit_reason. + * If that changes, this macro can be safely increased without breaking + * userspace so long as struct gunyah_vcpu_run < PAGE_SIZE. + */ +#define GUNYAH_VM_MAX_EXIT_REASON_SIZE 8u + +/** + * struct gunyah_vm_exit_info - Reason for VM exit as reported by Gunyah + * See Gunyah documentation for values. + * @type: Describes how VM exited + * @padding: padding bytes + * @reason_size: Number of bytes valid for `reason` + * @reason: See Gunyah documentation for interpretation. Note: these values are + * not interpreted by Linux and need to be converted from little-endian + * as applicable. + */ +struct gunyah_vm_exit_info { + __u16 type; + __u16 padding; + __u32 reason_size; + __u8 reason[GUNYAH_VM_MAX_EXIT_REASON_SIZE]; +}; + +/** + * enum gunyah_vcpu_exit - Stores reason why &GUNYAH_VCPU_RUN ioctl recently exited with status 0 + * @GUNYAH_VCPU_EXIT_UNKNOWN: Not used, status != 0 + * @GUNYAH_VCPU_EXIT_MMIO: vCPU performed a read or write that could not be handled + * by hypervisor or Linux. Use @struct gunyah_vcpu_run.mmio for + * details of the read/write. + * @GUNYAH_VCPU_EXIT_STATUS: vCPU not able to run because the VM has exited. + * Use @struct gunyah_vcpu_run.status for why VM has exited. + * @GUNYAH_VCPU_EXIT_PAGE_FAULT: vCPU tried to execute an instruction at an address + * for which memory hasn't been provided. Use + * @struct gunyah_vcpu_run.page_fault for details. + */ +enum gunyah_vcpu_exit { + GUNYAH_VCPU_EXIT_UNKNOWN, + GUNYAH_VCPU_EXIT_MMIO, + GUNYAH_VCPU_EXIT_STATUS, + GUNYAH_VCPU_EXIT_PAGE_FAULT, +}; + +/** + * enum gunyah_vcpu_resume_action - Provide resume action after an MMIO or page fault + * @GUNYAH_VCPU_RESUME_HANDLED: The mmio or page fault has been handled, continue + * normal operation of vCPU + * @GUNYAH_VCPU_RESUME_FAULT: The mmio or page fault could not be satisfied and + * inject the original fault back to the guest. + * @GUNYAH_VCPU_RESUME_RETRY: Retry the faulting instruction. Perhaps you added + * memory binding to satisfy the request. + */ +enum gunyah_vcpu_resume_action { + GUNYAH_VCPU_RESUME_HANDLED = 0, + GUNYAH_VCPU_RESUME_FAULT, + GUNYAH_VCPU_RESUME_RETRY, +}; + +/** + * struct gunyah_vcpu_run - Application code obtains a pointer to the gunyah_vcpu_run + * structure by mmap()ing a vcpu fd. + * @immediate_exit: polled when scheduling the vcpu. If set, immediately returns -EINTR. + * @padding: padding bytes + * @exit_reason: Set when GUNYAH_VCPU_RUN returns successfully and gives reason why + * GUNYAH_VCPU_RUN has stopped running the vCPU. See &enum gunyah_vcpu_exit. + * @mmio: Used when exit_reason == GUNYAH_VCPU_EXIT_MMIO + * The guest has faulted on an memory-mapped I/O that + * couldn't be satisfied by gunyah. + * @mmio.phys_addr: Address guest tried to access + * @mmio.data: the value that was written if `is_write == 1`. Filled by + * user for reads (`is_write == 0`). + * @mmio.len: Length of write. Only the first `len` bytes of `data` + * are considered by Gunyah. + * @mmio.is_write: 1 if VM tried to perform a write, 0 for a read + * @mmio.resume_action: See &enum gunyah_vcpu_resume_action + * @status: Used when exit_reason == GUNYAH_VCPU_EXIT_STATUS. + * The guest VM is no longer runnable. This struct informs why. + * @status.status: See &enum gunyah_vm_status for possible values + * @status.exit_info: Used when status == GUNYAH_VM_STATUS_EXITED + * @page_fault: Used when EXIT_REASON == GUNYAH_VCPU_EXIT_PAGE_FAULT + * The guest has faulted on a region that can only be provided + * by mapping memory at phys_addr. + * @page_fault.phys_addr: Address guest tried to access. + * @page_fault.attempt: Error code why Linux wasn't able to handle fault itself + * Typically, if no memory was mapped: -ENOENT, + * If permission bits weren't what the VM wanted: -EPERM + * @page_fault.resume_action: See &enum gunyah_vcpu_resume_action + */ +struct gunyah_vcpu_run { + /* in */ + __u8 immediate_exit; + __u8 padding[7]; + + /* out */ + __u32 exit_reason; + + union { + struct { + __u64 phys_addr; + __u8 data[8]; + __u32 len; + __u8 is_write; + __u8 resume_action; + } mmio; + + struct { + enum gunyah_vm_status status; + struct gunyah_vm_exit_info exit_info; + } status; + + struct { + __u64 phys_addr; + __s32 attempt; + __u8 resume_action; + } page_fault; + }; +}; + +#define GUNYAH_VCPU_RUN _IO(GUNYAH_IOCTL_TYPE, 0x5) +#define GUNYAH_VCPU_MMAP_SIZE _IO(GUNYAH_IOCTL_TYPE, 0x6) + #endif From patchwork Sat Dec 16 00:21:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 179697 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:3b04:b0:fb:cd0c:d3e with SMTP id c4csp9667480dys; Fri, 15 Dec 2023 16:28:22 -0800 (PST) X-Google-Smtp-Source: AGHT+IFMCa1c1VPcnGwkYY31vfMWWT+JKtL6BPH1k0miEGJvA7KQEJdejLwduXHM4E+v141XLswP X-Received: by 2002:a05:6830:1e38:b0:6d8:7ded:6cc5 with SMTP id t24-20020a0568301e3800b006d87ded6cc5mr13159144otr.41.1702686502460; Fri, 15 Dec 2023 16:28:22 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702686502; cv=none; d=google.com; s=arc-20160816; b=lTCN3yyYkuD1ru0RhF/xKjByu5ARqy0Qh09+d2aViJ9A7rt+9wx5PtT7VGUqelxKvT MfaynXFBa1FT45UgBAtxKrHns3wpHgfuVjT016TP3U5flKC+I4NBkTqXSHwTEll/2P0z Tr6/i2ufSo/arAXz3oxY2eXwXzSbjggUZa6ge5T3TzfMxvg9p5p6H94qtcfwExSHKlSb soMQfbpCddMOIwDR1akoQYJJ+xxRLqZKwVkBXWXosiWofaqg5q4K1wiq0JWSGlYhTx4q YYRsipKupnEt4FeFZ9yABsh4AeiNy1kGCTctrg7jYmYcBIfowTCEbEkXQGXaLFES5WUi vyDQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:list-unsubscribe:list-subscribe:list-id:precedence :subject:date:from:dkim-signature; bh=Pj31zjbqZFPkFusPu9Yv3U2hJdd1v8pz9tCQL82AZ/k=; fh=Pv/sMsQia5wmqAOYHIQUkkUFpDKhZjuODj71Ojb2ybw=; b=t3RsEqrECGpPIBPmVU2b4Lzvq2BtTLvvq7eS/olOiBv7Qp/iTG36uoFDIsTLYnKRbq lujPWvU7Nkg2F16oSfQdaPuq66ujn+WqOt53EHELdDOSTKtSHarMpjpzoeCFGa65O41j VEy9x33d9PCTAY4WJhwXy7cYqtn6Fdh4nyLuiMs+q0B+taeMf9k2jz6A8W96joR53Rd/ /ID0cdr2WGBbPouQVxitiSwWUVi2wHLXH6+p9mGGKG/UoUuNX7+ggIgeDV03Vm5Jh6lI DuRkSOlzkgm0mpAwRB9H0IryIm79+foRIA1R/ac1Z7yXCn2VVOUxR0fwo5KM+1ogw1zS yD9A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b="NL3t/F79"; spf=pass (google.com: domain of linux-kernel+bounces-1887-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1887-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id k13-20020a6568cd000000b005be007363a4si13500199pgt.669.2023.12.15.16.28.22 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Dec 2023 16:28:22 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-1887-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b="NL3t/F79"; spf=pass (google.com: domain of linux-kernel+bounces-1887-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1887-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 31ABC285445 for ; Sat, 16 Dec 2023 00:28:22 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 198F635293; Sat, 16 Dec 2023 00:21:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="NL3t/F79" X-Original-To: linux-kernel@vger.kernel.org Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 91DC51798D; Sat, 16 Dec 2023 00:21:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=quicinc.com Received: from pps.filterd (m0279862.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BFNe7sr027607; Sat, 16 Dec 2023 00:21:14 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:date:subject:mime-version:content-type :content-transfer-encoding:message-id:references:in-reply-to:to :cc; s=qcppdkim1; bh=Pj31zjbqZFPkFusPu9Yv3U2hJdd1v8pz9tCQL82AZ/k =; b=NL3t/F79t8LehsgKLsTNSLpjudW5vUXn++EttPLX9+4bXu3R1a7c9mjMUzm V8HrkxuE8vLwtL3YzTzdg7nMfh95F9i4RNHrP7aOv5b+fcMUPHgZV1Hk0W6MBBPF cIaPO9m8o0BifJmBpIOyUqpULedur6Rr4Yn6NschRptX2O0ny3DuC6WtWaakcCTr DtpdEgrZ/vCm83ulcauOLfhl/dxKLRb/NroCDHLPNmcwYo1KJ+c/ZODBhmV8RsVc 0964dLUoGfVbp+s3d8Xuk1SbDKVVIAb1xRCdk5cvT2dRj9NswS7dSuKH4nXggLQ1 WUYiOdlRdQ+kjAaY6fg68jskIvQ== Received: from nasanppmta05.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v0w198hpj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:13 +0000 (GMT) Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA05.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 3BG0LDrn006365 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:13 GMT Received: from [169.254.0.1] (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.40; Fri, 15 Dec 2023 16:21:12 -0800 From: Elliot Berman Date: Fri, 15 Dec 2023 16:21:01 -0800 Subject: [PATCH RFC v15 20/30] virt: gunyah: Implement guestmemfd Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <20231215-gunyah-v15-20-192a5d872a30@quicinc.com> References: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> In-Reply-To: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> To: Alex Elder , Srinivas Kandagatla , Murali Nalajal , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Philip Derrin , Prakruthi Deepak Heragu , Jonathan Corbet , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Catalin Marinas , Will Deacon , Konrad Dybcio , Bjorn Andersson , Dmitry Baryshkov , "Fuad Tabba" , Sean Christopherson CC: , , , , , Elliot Berman X-Mailer: b4 0.13-dev X-ClientProxiedBy: nalasex01a.na.qualcomm.com (10.47.209.196) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: e6UnKX19G6FgW5fsT85__qq9ITvodVUF X-Proofpoint-ORIG-GUID: e6UnKX19G6FgW5fsT85__qq9ITvodVUF X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 mlxlogscore=999 suspectscore=0 mlxscore=0 impostorscore=0 lowpriorityscore=0 spamscore=0 adultscore=0 phishscore=0 clxscore=1015 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312160001 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785396202216926708 X-GMAIL-MSGID: 1785396202216926708 Memory provided to Gunyah virtual machines are provided by a Gunyah guestmemfd. Because memory provided to virtual machines may be unmapped at stage-2 from the host (i.e. in the hypervisor's page tables for the host), special care needs to be taken to ensure that the kernel doesn't have a page mapped when it is lent to the guest. Without this tracking, a kernel panic could be induced by userspace tricking the kernel into accessing guest-private memory. Introduce the basic guestmemfd ops and ioctl. Userspace should be able to access the memory unless it is provided to the guest virtual machine: this is necessary to allow userspace to preload binaries such as the kernel Image prior to running the VM. Subsequent commits will wire up providing the memory to the guest. Signed-off-by: Elliot Berman --- drivers/virt/gunyah/Makefile | 2 +- drivers/virt/gunyah/guest_memfd.c | 269 ++++++++++++++++++++++++++++++++++++++ drivers/virt/gunyah/vm_mgr.c | 9 ++ drivers/virt/gunyah/vm_mgr.h | 2 + include/uapi/linux/gunyah.h | 19 +++ 5 files changed, 300 insertions(+), 1 deletion(-) diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index ffcde0e0ccfa..9d514b2baa3a 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -1,6 +1,6 @@ # SPDX-License-Identifier: GPL-2.0 -gunyah_rsc_mgr-y += rsc_mgr.o rsc_mgr_rpc.o vm_mgr.o vm_mgr_mem.o +gunyah_rsc_mgr-y += rsc_mgr.o rsc_mgr_rpc.o vm_mgr.o vm_mgr_mem.o guest_memfd.o obj-$(CONFIG_GUNYAH) += gunyah.o gunyah_rsc_mgr.o gunyah_vcpu.o obj-$(CONFIG_GUNYAH_PLATFORM_HOOKS) += gunyah_platform_hooks.o diff --git a/drivers/virt/gunyah/guest_memfd.c b/drivers/virt/gunyah/guest_memfd.c new file mode 100644 index 000000000000..709aae9a1f44 --- /dev/null +++ b/drivers/virt/gunyah/guest_memfd.c @@ -0,0 +1,269 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#define pr_fmt(fmt) "gunyah_guest_mem: " fmt + +#include +#include +#include +#include +#include +#include + +#include + +static struct folio *gunyah_gmem_get_huge_folio(struct inode *inode, + pgoff_t index) +{ +#ifdef CONFIG_TRANSPARENT_HUGEPAGE + unsigned long huge_index = round_down(index, HPAGE_PMD_NR); + unsigned long flags = (unsigned long)inode->i_private; + struct address_space *mapping = inode->i_mapping; + gfp_t gfp = mapping_gfp_mask(mapping); + struct folio *folio; + + if (!(flags & GHMF_ALLOW_HUGEPAGE)) + return NULL; + + if (filemap_range_has_page(mapping, huge_index << PAGE_SHIFT, + (huge_index + HPAGE_PMD_NR - 1) + << PAGE_SHIFT)) + return NULL; + + folio = filemap_alloc_folio(gfp, HPAGE_PMD_ORDER); + if (!folio) + return NULL; + + if (filemap_add_folio(mapping, folio, huge_index, gfp)) { + folio_put(folio); + return NULL; + } + + return folio; +#else + return NULL; +#endif +} + +static struct folio *gunyah_gmem_get_folio(struct inode *inode, pgoff_t index) +{ + struct folio *folio; + + folio = gunyah_gmem_get_huge_folio(inode, index); + if (!folio) { + folio = filemap_grab_folio(inode->i_mapping, index); + if (IS_ERR_OR_NULL(folio)) + return NULL; + } + + /* + * Use the up-to-date flag to track whether or not the memory has been + * zeroed before being handed off to the guest. There is no backing + * storage for the memory, so the folio will remain up-to-date until + * it's removed. + */ + if (!folio_test_uptodate(folio)) { + unsigned long nr_pages = folio_nr_pages(folio); + unsigned long i; + + for (i = 0; i < nr_pages; i++) + clear_highpage(folio_page(folio, i)); + + folio_mark_uptodate(folio); + } + + /* + * Ignore accessed, referenced, and dirty flags. The memory is + * unevictable and there is no storage to write back to. + */ + return folio; +} + +static vm_fault_t gunyah_gmem_host_fault(struct vm_fault *vmf) +{ + struct folio *folio; + + folio = gunyah_gmem_get_folio(file_inode(vmf->vma->vm_file), + vmf->pgoff); + if (!folio || folio_test_private(folio)) { + folio_unlock(folio); + folio_put(folio); + return VM_FAULT_SIGBUS; + } + + vmf->page = folio_file_page(folio, vmf->pgoff); + + return VM_FAULT_LOCKED; +} + +static const struct vm_operations_struct gunyah_gmem_vm_ops = { + .fault = gunyah_gmem_host_fault, +}; + +static int gunyah_gmem_mmap(struct file *file, struct vm_area_struct *vma) +{ + file_accessed(file); + vma->vm_ops = &gunyah_gmem_vm_ops; + return 0; +} + +static long gunyah_gmem_punch_hole(struct inode *inode, loff_t offset, + loff_t len) +{ + truncate_inode_pages_range(inode->i_mapping, offset, offset + len - 1); + + return 0; +} + +static long gunyah_gmem_allocate(struct inode *inode, loff_t offset, loff_t len) +{ + struct address_space *mapping = inode->i_mapping; + pgoff_t start, index, end; + int r; + + /* Dedicated guest is immutable by default. */ + if (offset + len > i_size_read(inode)) + return -EINVAL; + + filemap_invalidate_lock_shared(mapping); + + start = offset >> PAGE_SHIFT; + end = (offset + len) >> PAGE_SHIFT; + + r = 0; + for (index = start; index < end;) { + struct folio *folio; + + if (signal_pending(current)) { + r = -EINTR; + break; + } + + folio = gunyah_gmem_get_folio(inode, index); + if (!folio) { + r = -ENOMEM; + break; + } + + index = folio_next_index(folio); + + folio_unlock(folio); + folio_put(folio); + + /* 64-bit only, wrapping the index should be impossible. */ + if (WARN_ON_ONCE(!index)) + break; + + cond_resched(); + } + + filemap_invalidate_unlock_shared(mapping); + + return r; +} + +static long gunyah_gmem_fallocate(struct file *file, int mode, loff_t offset, + loff_t len) +{ + long ret; + + if (!(mode & FALLOC_FL_KEEP_SIZE)) + return -EOPNOTSUPP; + + if (mode & ~(FALLOC_FL_KEEP_SIZE | FALLOC_FL_PUNCH_HOLE | + FALLOC_FL_ZERO_RANGE)) + return -EOPNOTSUPP; + + if (!PAGE_ALIGNED(offset) || !PAGE_ALIGNED(len)) + return -EINVAL; + + if (mode & FALLOC_FL_PUNCH_HOLE) + ret = gunyah_gmem_punch_hole(file_inode(file), offset, len); + else + ret = gunyah_gmem_allocate(file_inode(file), offset, len); + + if (!ret) + file_modified(file); + return ret; +} + +static int gunyah_gmem_release(struct inode *inode, struct file *file) +{ + gunyah_gmem_punch_hole(inode, 0, U64_MAX); + + return 0; +} + +static const struct file_operations gunyah_gmem_fops = { + .owner = THIS_MODULE, + .llseek = generic_file_llseek, + .mmap = gunyah_gmem_mmap, + .open = generic_file_open, + .fallocate = gunyah_gmem_fallocate, + .release = gunyah_gmem_release, +}; + +static const struct address_space_operations gunyah_gmem_aops = { + .dirty_folio = noop_dirty_folio, + .migrate_folio = migrate_folio, + .error_remove_folio = generic_error_remove_folio, +}; + +int gunyah_guest_mem_create(struct gunyah_create_mem_args *args) +{ + const char *anon_name = "[gh-gmem]"; + unsigned long fd_flags = 0; + struct inode *inode; + struct file *file; + int fd, err; + + if (!PAGE_ALIGNED(args->size)) + return -EINVAL; + + if (args->flags & ~(GHMF_CLOEXEC | GHMF_ALLOW_HUGEPAGE)) + return -EINVAL; + + if (args->flags & GHMF_CLOEXEC) + fd_flags |= O_CLOEXEC; + + fd = get_unused_fd_flags(fd_flags); + if (fd < 0) + return fd; + + /* + * Use the so called "secure" variant, which creates a unique inode + * instead of reusing a single inode. Each guest_memfd instance needs + * its own inode to track the size, flags, etc. + */ + file = anon_inode_create_getfile(anon_name, &gunyah_gmem_fops, NULL, + O_RDWR, NULL); + if (IS_ERR(file)) { + err = PTR_ERR(file); + goto err_fd; + } + + file->f_flags |= O_LARGEFILE; + + inode = file->f_inode; + WARN_ON(file->f_mapping != inode->i_mapping); + + inode->i_private = (void *)(unsigned long)args->flags; + inode->i_mapping->a_ops = &gunyah_gmem_aops; + inode->i_mode |= S_IFREG; + inode->i_size = args->size; + mapping_set_gfp_mask(inode->i_mapping, GFP_HIGHUSER); + mapping_set_large_folios(inode->i_mapping); + mapping_set_unmovable(inode->i_mapping); + // mapping_set_release_always(inode->i_mapping); + /* Unmovable mappings are supposed to be marked unevictable as well. */ + WARN_ON_ONCE(!mapping_unevictable(inode->i_mapping)); + + fd_install(fd, file); + return fd; + +err_fd: + put_unused_fd(fd); + return err; +} diff --git a/drivers/virt/gunyah/vm_mgr.c b/drivers/virt/gunyah/vm_mgr.c index 9e402e6ea7fe..cd978d1ce93f 100644 --- a/drivers/virt/gunyah/vm_mgr.c +++ b/drivers/virt/gunyah/vm_mgr.c @@ -799,6 +799,15 @@ long gunyah_dev_vm_mgr_ioctl(struct gunyah_rm *rm, unsigned int cmd, switch (cmd) { case GUNYAH_CREATE_VM: return gunyah_dev_ioctl_create_vm(rm, arg); + case GUNYAH_CREATE_GUEST_MEM: { + struct gunyah_create_mem_args args; + + if (copy_from_user(&args, (const void __user *)arg, + sizeof(args))) + return -EFAULT; + + return gunyah_guest_mem_create(&args); + } default: return -ENOTTY; } diff --git a/drivers/virt/gunyah/vm_mgr.h b/drivers/virt/gunyah/vm_mgr.h index 348dbeda57ec..d26693d10d22 100644 --- a/drivers/virt/gunyah/vm_mgr.h +++ b/drivers/virt/gunyah/vm_mgr.h @@ -77,4 +77,6 @@ void gunyah_vm_reclaim_memory(struct gunyah_vm *ghvm); int gunyah_vm_mmio_write(struct gunyah_vm *ghvm, u64 addr, u32 len, u64 data); +int gunyah_guest_mem_create(struct gunyah_create_mem_args *args); + #endif diff --git a/include/uapi/linux/gunyah.h b/include/uapi/linux/gunyah.h index 46f7d3aa61d0..c5f506350364 100644 --- a/include/uapi/linux/gunyah.h +++ b/include/uapi/linux/gunyah.h @@ -20,6 +20,25 @@ */ #define GUNYAH_CREATE_VM _IO(GUNYAH_IOCTL_TYPE, 0x0) /* Returns a Gunyah VM fd */ +enum gunyah_mem_flags { + GHMF_CLOEXEC = (1UL << 0), + GHMF_ALLOW_HUGEPAGE = (1UL << 1), +}; + +/** + * struct gunyah_create_mem_args - Description of guest memory to create + * @flags: See GHMF_*. + */ +struct gunyah_create_mem_args { + __u64 flags; + __u64 size; + __u64 reserved[6]; +}; + +#define GUNYAH_CREATE_GUEST_MEM \ + _IOW(GUNYAH_IOCTL_TYPE, 0x8, \ + struct gunyah_create_mem_args) /* Returns a Gunyah memory fd */ + /* * ioctls for gunyah-vm fds (returned by GUNYAH_CREATE_VM) */ From patchwork Sat Dec 16 00:21:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 179698 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:3b04:b0:fb:cd0c:d3e with SMTP id c4csp9667554dys; Fri, 15 Dec 2023 16:28:33 -0800 (PST) X-Google-Smtp-Source: AGHT+IGJhzE8x5UlddQlo9Nvn89GzflelcR1lOWcshSIJ5jzqgBpYm7MDOSBAvkUpQjOgYfAzclx X-Received: by 2002:a50:9553:0:b0:552:8a59:a37f with SMTP id v19-20020a509553000000b005528a59a37fmr1629572eda.2.1702686513581; Fri, 15 Dec 2023 16:28:33 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702686513; cv=none; d=google.com; s=arc-20160816; b=WtkCbGvnjHZ4Q85QiRz2gDIhWT/W4ka6Gq84iDrRTxBKwZe8O+lQ5auaN8ILx9oEL1 2HHv8oZBB2TYvGdbx1sCfNuQ/BZgi7Q3hPKVgfBS7kFs240/AUOUm9FD1un3O5q3nXua RmetNhtmroBa4ub9QNG3Zt7TodlaK4ZqkQ9L2/AbT2Mj/4jBcGKRgBapMHONEuNyHD/c eB23yzajOEBz9izFEnMe6X1NnvQwGEIS2wts4szKh5JIzxX9vcls1TfL2axEvhnTzl27 Jzo4tso9cfJY2pqXLHwlgjpBmPaxZEentl5VWrHuiaEN55JHnqTTq9Qws13FoarnTjpf 1DIg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:list-unsubscribe:list-subscribe:list-id:precedence :subject:date:from:dkim-signature; bh=/cYOlK65HujdYzwJs0ckTKZs9llWD5IUZ2Dzlq7BYnI=; fh=Pv/sMsQia5wmqAOYHIQUkkUFpDKhZjuODj71Ojb2ybw=; b=FxToq8JCMb7rb6c+LTGNrCbagqIDl0kj9IoxSBUGHbM0O8bwn6OYwAcvym6yD13v4b zFngWskxk+r1jZyP1O8xjPquGETLe977UvKYFshFWDPk98te61M77TynvxEB0C0mXnDq PF+6P9tR2xTwp2ueswB4qV6E2yh9C2KCI2RnBBisu+fdf/wwon9xzXgubRIV1xPfxc6T nbrsbptRiZ7MbdHXmxGLHSU1jjg7OBI3VEP6MA5SNLNOzLi7fefkxrR9Nnad6kjhV6ZD yDp3m2x2+4ECqJqYfubtzrkWQb0z3UtHg4379WDSvmm7O4vk380OtOFo9UtSpi4pdcMQ ambg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=mx1hzJaM; spf=pass (google.com: domain of linux-kernel+bounces-1888-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1888-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id x6-20020a05640226c600b0054c6d83ba3asi7648265edd.60.2023.12.15.16.28.33 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Dec 2023 16:28:33 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-1888-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=mx1hzJaM; spf=pass (google.com: domain of linux-kernel+bounces-1888-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1888-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id F07981F25027 for ; Sat, 16 Dec 2023 00:28:32 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 5C447358AA; Sat, 16 Dec 2023 00:21:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="mx1hzJaM" X-Original-To: linux-kernel@vger.kernel.org Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BDB79182A4; Sat, 16 Dec 2023 00:21:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=quicinc.com Received: from pps.filterd (m0279867.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BFNSS0b027076; Sat, 16 Dec 2023 00:21:14 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:date:subject:mime-version:content-type :content-transfer-encoding:message-id:references:in-reply-to:to :cc; s=qcppdkim1; bh=/cYOlK65HujdYzwJs0ckTKZs9llWD5IUZ2Dzlq7BYnI =; b=mx1hzJaMIuuBYbnW+BfgQP5lN222evWI0RsOSIs+jWJVAvCg3YM3BlKfcq0 89ceV7FDqvSlDJTumEHa0oAm3MUxf9yzQ/YLzjpCf9nka37eM2jCKHdk/CaVXTr3 0t+BYcbBVMVi7R+dDt4MhQJA2ao3bPMXlsXY5B6kYpNTJX4igwbfk/W8T4FcANhx mZ6ZR3xnlQtpvdsgY+1u6RgNM4APA8QQOryapR1nSKHPlo0nFV57dadtNrQwpGQG a8ciUMnU2o7dKLN75E5UXasRd24coJLDI4jqFJa12VlcnA5+Wv7nRqHmdi6zxYDT gbfaRluUvmm/u1ivy67cHWDJ9xg== Received: from nasanppmta01.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v0k90t5ej-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:14 +0000 (GMT) Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 3BG0LD37015665 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:13 GMT Received: from [169.254.0.1] (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.40; Fri, 15 Dec 2023 16:21:12 -0800 From: Elliot Berman Date: Fri, 15 Dec 2023 16:21:02 -0800 Subject: [PATCH RFC v15 21/30] virt: gunyah: Add ioctl to bind guestmem to VMs Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <20231215-gunyah-v15-21-192a5d872a30@quicinc.com> References: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> In-Reply-To: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> To: Alex Elder , Srinivas Kandagatla , Murali Nalajal , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Philip Derrin , Prakruthi Deepak Heragu , Jonathan Corbet , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Catalin Marinas , Will Deacon , Konrad Dybcio , Bjorn Andersson , Dmitry Baryshkov , "Fuad Tabba" , Sean Christopherson CC: , , , , , Elliot Berman X-Mailer: b4 0.13-dev X-ClientProxiedBy: nalasex01a.na.qualcomm.com (10.47.209.196) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: t9N_XQ52W1tJiv5yM9H5eglzb0MovlC9 X-Proofpoint-GUID: t9N_XQ52W1tJiv5yM9H5eglzb0MovlC9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 lowpriorityscore=0 impostorscore=0 priorityscore=1501 malwarescore=0 clxscore=1015 mlxlogscore=999 adultscore=0 suspectscore=0 bulkscore=0 spamscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312160001 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785396213276044560 X-GMAIL-MSGID: 1785396213276044560 A maple tree is used to maintain a map from guest address ranges to a guestmemfd that provides the memory for that range of memory for the guest. The mapping of guest address range to guestmemfd is called a binding. Implement an ioctl to add/remove bindings to the virtual machine. The binding determines whether the memory is shared (host retains access) or lent (host loses access). Signed-off-by: Elliot Berman --- drivers/virt/gunyah/guest_memfd.c | 277 ++++++++++++++++++++++++++++++++++++++ drivers/virt/gunyah/vm_mgr.c | 15 +++ drivers/virt/gunyah/vm_mgr.h | 6 + include/uapi/linux/gunyah.h | 41 ++++++ 4 files changed, 339 insertions(+) diff --git a/drivers/virt/gunyah/guest_memfd.c b/drivers/virt/gunyah/guest_memfd.c index 709aae9a1f44..c38380c4dc50 100644 --- a/drivers/virt/gunyah/guest_memfd.c +++ b/drivers/virt/gunyah/guest_memfd.c @@ -9,11 +9,61 @@ #include #include #include +#include #include #include #include +#include "vm_mgr.h" + +/** + * struct gunyah_gmem_binding - Represents a binding of guestmem to a Gunyah VM + * @gfn: Guest address to place acquired folios + * @ghvm: Pointer to Gunyah VM in this binding + * @mt: Maple tree to track folios which have been provided to the VM + * @i_off: offset into the guestmem to grab folios from + * @inode: Pointer to guest mem inode + * @i_entry: list entry for inode->i_private_list + * @flags: Access flags for the binding + * @nr: Number of pages covered by this binding + */ +struct gunyah_gmem_binding { + u64 gfn; + struct gunyah_vm *ghvm; + struct maple_tree mt; + + pgoff_t i_off; + struct inode *inode; + struct list_head i_entry; + + u32 flags; + unsigned long nr; +}; + +static inline pgoff_t gunyah_gfn_to_off(struct gunyah_gmem_binding *b, u64 gfn) +{ + return gfn - b->gfn + b->i_off; +} + +static inline u64 gunyah_off_to_gfn(struct gunyah_gmem_binding *b, pgoff_t off) +{ + return off - b->i_off + b->gfn; +} + +static inline bool gunyah_guest_mem_is_lend(struct gunyah_vm *ghvm, u32 flags) +{ + u8 access = flags & GUNYAH_MEM_ACCESS_MASK; + + if (access == GUNYAH_MEM_FORCE_LEND) + return true; + else if (access == GUNYAH_MEM_FORCE_SHARE) + return false; + + /* RM requires all VMs to be protected (isolated) */ + return true; +} + static struct folio *gunyah_gmem_get_huge_folio(struct inode *inode, pgoff_t index) { @@ -191,8 +241,15 @@ static long gunyah_gmem_fallocate(struct file *file, int mode, loff_t offset, static int gunyah_gmem_release(struct inode *inode, struct file *file) { + struct gunyah_gmem_binding *b, *n; + gunyah_gmem_punch_hole(inode, 0, U64_MAX); + list_for_each_entry_safe(b, n, &inode->i_mapping->i_private_list, + i_entry) { + gunyah_gmem_remove_binding(b); + } + return 0; } @@ -267,3 +324,223 @@ int gunyah_guest_mem_create(struct gunyah_create_mem_args *args) put_unused_fd(fd); return err; } + +void gunyah_gmem_remove_binding(struct gunyah_gmem_binding *b) +{ + mtree_erase(&b->ghvm->mem_layout, b->gfn); + list_del(&b->i_entry); + kfree(b); +} + +static inline unsigned long gunyah_gmem_page_mask(struct inode *inode) +{ + unsigned long gmem_flags = (unsigned long)inode->i_private; + + if (gmem_flags & GHMF_ALLOW_HUGEPAGE) { +#if IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) + return HPAGE_PMD_MASK; +#else + return ULONG_MAX; +#endif + } + + return PAGE_MASK; +} + +static int gunyah_gmem_init_binding(struct gunyah_vm *ghvm, struct inode *inode, + struct gunyah_map_mem_args *args, + struct gunyah_gmem_binding *binding) +{ + const unsigned long page_mask = ~gunyah_gmem_page_mask(inode); + + if (args->flags & ~(GUNYAH_MEM_ALLOW_RWX | GUNYAH_MEM_ACCESS_MASK)) + return -EINVAL; + + if (args->guest_addr & page_mask) + return -EINVAL; + + if (args->offset & page_mask) + return -EINVAL; + + if (args->size & page_mask) + return -EINVAL; + + binding->gfn = gunyah_gpa_to_gfn(args->guest_addr); + binding->ghvm = ghvm; + binding->i_off = args->offset >> PAGE_SHIFT; + binding->inode = inode; + binding->flags = args->flags; + binding->nr = args->size >> PAGE_SHIFT; + + return 0; +} + +static int gunyah_gmem_remove_mapping(struct gunyah_vm *ghvm, + struct inode *inode, + struct gunyah_map_mem_args *args) +{ + struct gunyah_gmem_binding argb; + struct gunyah_gmem_binding *b = NULL; + unsigned long start_delta, end_delta; + int ret; + + ret = gunyah_gmem_init_binding(ghvm, inode, args, &argb); + if (ret) + return ret; + + filemap_invalidate_lock(inode->i_mapping); + list_for_each_entry(b, &inode->i_mapping->i_private_list, i_entry) { + if (b->ghvm != argb.ghvm || b->flags != argb.flags || + WARN_ON(b->inode != argb.inode)) + continue; + /* Check if argb guest addresses is within b */ + if (b->gfn > argb.gfn) + continue; + if (b->gfn + b->nr < argb.gfn + argb.nr) + continue; + start_delta = argb.gfn - b->gfn; + if (argb.i_off - b->i_off != start_delta) + continue; + end_delta = argb.gfn + argb.nr - b->gfn - b->nr; + if (!start_delta && !end_delta) { + /* wipe the mapping entirely */ + gunyah_gmem_remove_binding(b); + goto out; + } else if (start_delta && !end_delta) { + /* shrink the end */ + down_write(&ghvm->mem_lock); + mtree_erase(&b->ghvm->mem_layout, b->gfn); + b->nr = start_delta; + ret = mtree_insert_range(&ghvm->mem_layout, b->gfn, + b->gfn + b->nr - 1, b, + GFP_KERNEL); + up_write(&ghvm->mem_lock); + goto out; + } else if (!start_delta && end_delta) { + /* Shrink the beginning */ + down_write(&ghvm->mem_lock); + mtree_erase(&b->ghvm->mem_layout, b->gfn); + b->gfn += argb.nr; + b->i_off += argb.nr; + b->nr -= argb.nr; + ret = mtree_insert_range(&ghvm->mem_layout, b->gfn, + b->gfn + b->nr - 1, b, + GFP_KERNEL); + up_write(&ghvm->mem_lock); + goto out; + } else { + /* TODO: split the mapping into 2 */ + ret = -EINVAL; + goto out; + } + } + ret = -ENOENT; +out: + filemap_invalidate_unlock(inode->i_mapping); + return ret; +} + +static bool gunyah_gmem_binding_allowed_overlap(struct gunyah_gmem_binding *a, + struct gunyah_gmem_binding *b) +{ + /* Bindings can't overlap within a VM. Only one guest mem can + * provide for a given guest address + */ + if (a->ghvm == b->ghvm && a->gfn + a->nr <= b->gfn && + a->gfn >= b->gfn + b->nr) + return false; + + /* Gunyah only guarantees we can share a page with one VM and + * doesn't (currently) allow us to share same page with multiple VMs, + * regardless whether host can also access. + */ + if (a->inode == b->inode) { + if (a->ghvm == b->ghvm) { + if (gunyah_guest_mem_is_lend(a->ghvm, a->flags) || + gunyah_guest_mem_is_lend(b->ghvm, b->flags)) + return false; + } else { + if (a->i_off + a->nr < b->i_off) + return false; + if (a->i_off > b->i_off + b->nr) + return false; + } + } + + return true; +} + +static int gunyah_gmem_add_mapping(struct gunyah_vm *ghvm, struct inode *inode, + struct gunyah_map_mem_args *args) +{ + struct gunyah_gmem_binding *b, *tmp = NULL; + int ret; + + b = kzalloc(sizeof(*b), GFP_KERNEL); + if (!b) + return -ENOMEM; + + ret = gunyah_gmem_init_binding(ghvm, inode, args, b); + if (ret) + return ret; + + filemap_invalidate_lock(inode->i_mapping); + list_for_each_entry(tmp, &inode->i_mapping->i_private_list, i_entry) { + if (!gunyah_gmem_binding_allowed_overlap(b, tmp)) { + ret = -EEXIST; + goto unlock; + } + } + + ret = mtree_insert_range(&ghvm->mem_layout, b->gfn, b->gfn + b->nr - 1, + b, GFP_KERNEL); + if (ret) + goto unlock; + + list_add(&b->i_entry, &inode->i_mapping->i_private_list); + +unlock: + filemap_invalidate_unlock(inode->i_mapping); + return ret; +} + +int gunyah_gmem_modify_binding(struct gunyah_vm *ghvm, + struct gunyah_map_mem_args *args) +{ + u8 access = args->flags & GUNYAH_MEM_ACCESS_MASK; + struct file *file; + int ret = -EINVAL; + + file = fget(args->guest_mem_fd); + if (!file) + return -EINVAL; + + if (file->f_op != &gunyah_gmem_fops) + goto err_file; + + if (args->flags & ~(GUNYAH_MEM_ALLOW_RWX | GUNYAH_MEM_UNMAP | GUNYAH_MEM_ACCESS_MASK)) + goto err_file; + + /* VM needs to have some permissions to the memory */ + if (!(args->flags & GUNYAH_MEM_ALLOW_RWX)) + goto err_file; + + if (access != GUNYAH_MEM_DEFAULT_ACCESS && + access != GUNYAH_MEM_FORCE_LEND && access != GUNYAH_MEM_FORCE_SHARE) + goto err_file; + + if (!PAGE_ALIGNED(args->guest_addr) || !PAGE_ALIGNED(args->offset) || + !PAGE_ALIGNED(args->size)) + goto err_file; + + if (args->flags & GUNYAH_MEM_UNMAP) { + args->flags &= ~GUNYAH_MEM_UNMAP; + ret = gunyah_gmem_remove_mapping(ghvm, file_inode(file), args); + } else { + ret = gunyah_gmem_add_mapping(ghvm, file_inode(file), args); + } + +err_file: + fput(file); + return ret; +} diff --git a/drivers/virt/gunyah/vm_mgr.c b/drivers/virt/gunyah/vm_mgr.c index cd978d1ce93f..5666070453aa 100644 --- a/drivers/virt/gunyah/vm_mgr.c +++ b/drivers/virt/gunyah/vm_mgr.c @@ -519,6 +519,8 @@ static __must_check struct gunyah_vm *gunyah_vm_alloc(struct gunyah_rm *rm) mutex_init(&ghvm->fn_lock); mt_init(&ghvm->gm); + mt_init(&ghvm->mem_layout); + init_rwsem(&ghvm->mem_lock); ghvm->addrspace_ticket.resource_type = GUNYAH_RESOURCE_TYPE_ADDR_SPACE; ghvm->addrspace_ticket.label = GUNYAH_VM_ADDRSPACE_LABEL; @@ -673,6 +675,14 @@ static long gunyah_vm_ioctl(struct file *filp, unsigned int cmd, r = gunyah_vm_rm_function_instance(ghvm, &f); break; } + case GUNYAH_VM_MAP_MEM: { + struct gunyah_map_mem_args args; + + if (copy_from_user(&args, argp, sizeof(args))) + return -EFAULT; + + return gunyah_gmem_modify_binding(ghvm, &args); + } default: r = -ENOTTY; break; @@ -690,6 +700,8 @@ EXPORT_SYMBOL_GPL(gunyah_vm_get); static void _gunyah_vm_put(struct kref *kref) { struct gunyah_vm *ghvm = container_of(kref, struct gunyah_vm, kref); + struct gunyah_gmem_binding *b; + unsigned long idx = 0; int ret; if (ghvm->vm_status == GUNYAH_RM_VM_STATUS_RUNNING) @@ -697,6 +709,9 @@ static void _gunyah_vm_put(struct kref *kref) gunyah_vm_remove_functions(ghvm); + mt_for_each(&ghvm->mem_layout, b, idx, ULONG_MAX) + gunyah_gmem_remove_binding(b); + mtree_destroy(&ghvm->mem_layout); gunyah_vm_reclaim_memory(ghvm); gunyah_vm_remove_resource_ticket(ghvm, &ghvm->addrspace_ticket); diff --git a/drivers/virt/gunyah/vm_mgr.h b/drivers/virt/gunyah/vm_mgr.h index d26693d10d22..8f1c3ade08dd 100644 --- a/drivers/virt/gunyah/vm_mgr.h +++ b/drivers/virt/gunyah/vm_mgr.h @@ -36,6 +36,8 @@ long gunyah_dev_vm_mgr_ioctl(struct gunyah_rm *rm, unsigned int cmd, struct gunyah_vm { u16 vmid; struct maple_tree gm; + struct maple_tree mem_layout; + struct rw_semaphore mem_lock; struct gunyah_vm_resource_ticket addrspace_ticket, host_private_extent_ticket, host_shared_extent_ticket, guest_private_extent_ticket, guest_shared_extent_ticket; @@ -78,5 +80,9 @@ void gunyah_vm_reclaim_memory(struct gunyah_vm *ghvm); int gunyah_vm_mmio_write(struct gunyah_vm *ghvm, u64 addr, u32 len, u64 data); int gunyah_guest_mem_create(struct gunyah_create_mem_args *args); +int gunyah_gmem_modify_binding(struct gunyah_vm *ghvm, + struct gunyah_map_mem_args *args); +struct gunyah_gmem_binding; +void gunyah_gmem_remove_binding(struct gunyah_gmem_binding *binding); #endif diff --git a/include/uapi/linux/gunyah.h b/include/uapi/linux/gunyah.h index c5f506350364..1af4c5ae6bc3 100644 --- a/include/uapi/linux/gunyah.h +++ b/include/uapi/linux/gunyah.h @@ -87,6 +87,47 @@ struct gunyah_fn_desc { #define GUNYAH_VM_ADD_FUNCTION _IOW(GUNYAH_IOCTL_TYPE, 0x4, struct gunyah_fn_desc) #define GUNYAH_VM_REMOVE_FUNCTION _IOW(GUNYAH_IOCTL_TYPE, 0x7, struct gunyah_fn_desc) +/** + * enum gunyah_map_flags- Possible flags on &struct gunyah_map_mem_args + * @GUNYAH_MEM_DEFAULT_SHARE: Use default host access for the VM type + * @GUNYAH_MEM_FORCE_LEND: Force unmapping the memory once the guest starts to use + * @GUNYAH_MEM_FORCE_SHARE: Allow host to continue accessing memory when guest starts to use + * @GUNYAH_MEM_ALLOW_READ: Allow guest to read memory + * @GUNYAH_MEM_ALLOW_WRITE: Allow guest to write to the memory + * @GUNYAH_MEM_ALLOW_EXEC: Allow guest to execute instructions in the memory + */ +enum gunyah_map_flags { + GUNYAH_MEM_DEFAULT_ACCESS = 0, + GUNYAH_MEM_FORCE_LEND = 1, + GUNYAH_MEM_FORCE_SHARE = 2, +#define GUNYAH_MEM_ACCESS_MASK 0x7 + + GUNYAH_MEM_ALLOW_READ = 1UL << 4, + GUNYAH_MEM_ALLOW_WRITE = 1UL << 5, + GUNYAH_MEM_ALLOW_EXEC = 1UL << 6, + GUNYAH_MEM_ALLOW_RWX = + (GUNYAH_MEM_ALLOW_READ | GUNYAH_MEM_ALLOW_WRITE | GUNYAH_MEM_ALLOW_EXEC), + + GUNYAH_MEM_UNMAP = 1UL << 8, +}; + +/** + * struct gunyah_map_mem_args - Description to provide guest memory into a VM + * @guest_addr: Location in guest address space to place the memory + * @flags: See &enum gunyah_map_flags. + * @guest_mem_fd: File descriptor created by GUNYAH_CREATE_GUEST_MEM + * @offset: Offset into the guest memory file + */ +struct gunyah_map_mem_args { + __u64 guest_addr; + __u32 flags; + __u32 guest_mem_fd; + __u64 offset; + __u64 size; +}; + +#define GUNYAH_VM_MAP_MEM _IOW(GUNYAH_IOCTL_TYPE, 0x9, struct gunyah_map_mem_args) + /* * ioctls for vCPU fds */ From patchwork Sat Dec 16 00:21:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 179699 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:3b04:b0:fb:cd0c:d3e with SMTP id c4csp9667598dys; Fri, 15 Dec 2023 16:28:38 -0800 (PST) X-Google-Smtp-Source: AGHT+IGfU6MTnvLpJ9plSL0/FA1XM7Bf04QVnVPICJbA4dBbdAmh8yIJBEeytvg5GJd8Ba9CmWMR X-Received: by 2002:ac8:5c4b:0:b0:425:4043:18c2 with SMTP id j11-20020ac85c4b000000b00425404318c2mr18173740qtj.117.1702686517868; Fri, 15 Dec 2023 16:28:37 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702686517; cv=none; d=google.com; s=arc-20160816; b=GpXo2INkyX3boFOBeqIjmmDUtfpejS02h65W0U/Sdw/QREXhgZJQZVBPxIG8hGD9ne J6nGM3+LUl7Ju8Yh+yj7LosGZescUj1wUMbVO5S0bG8HP8qELkElaGDpP+JDIGufsA7v YgpQXvW0he+hbYAt6Ile7aOK/4kuPgC+YOZT/WjOq8DQO64zyVUs95YVMw4cosAtHbFm DBo51xDnL/+j5etPfd2ueDJLapBEqWaXAaDCwaS3obVRpys32kkW4y5IEwruT+/QG9o6 b0zWRQA8ECrB1pEDKKNxKPXXxyKiVFPakSBkfNQugruqyU/p6KTNEdsZxMXPxr4wsDRn 0X7Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:list-unsubscribe:list-subscribe:list-id:precedence :subject:date:from:dkim-signature; bh=xctNUcQvlpqTuEFQH3xW/yX6RJBdeipXq5ilyhd7G8k=; fh=Pv/sMsQia5wmqAOYHIQUkkUFpDKhZjuODj71Ojb2ybw=; b=Qo9S99LXx1yzyiuo9fPk4oNyf7PfpYdnx30BXgdNq77gcheaKLTMVombRr1M7hVCrv mWJ3cwYGC0alvTlONaj3zgSubw3g52ff1YfSHnt+Ax90NQmMwYaH7EWi7xnI2n34yjuT vjMj/Ba4DMoUpVW86n/4NdSkVFJX7VERAzT3Dn2ooxKAfrgJfMsm0DZg0GbBuCXoSTnr EeVimrq71i+4ooDCN9b1wuRNXVCEWG7Juleilvlrshqu1LkBDJL3CWKVyqXQ6JKX+Fhb 5Oj6eEls52/rpWCPjVaZNIVtREW27avXuF0zVXLQiuc25YSBcCmu4ytE+MkXWZ4HrrDM fhsw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=Ije1xgeW; spf=pass (google.com: domain of linux-kernel+bounces-1889-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1889-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id t11-20020a05622a148b00b00425f05c4063si7241972qtx.750.2023.12.15.16.28.37 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Dec 2023 16:28:37 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-1889-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=Ije1xgeW; spf=pass (google.com: domain of linux-kernel+bounces-1889-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1889-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 988FA1C2037A for ; Sat, 16 Dec 2023 00:28:37 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 9A03936AEB; Sat, 16 Dec 2023 00:21:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="Ije1xgeW" X-Original-To: linux-kernel@vger.kernel.org Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 24FCB19450; Sat, 16 Dec 2023 00:21:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=quicinc.com Received: from pps.filterd (m0279866.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BG0Gixe008289; Sat, 16 Dec 2023 00:21:15 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:date:subject:mime-version:content-type :content-transfer-encoding:message-id:references:in-reply-to:to :cc; s=qcppdkim1; bh=xctNUcQvlpqTuEFQH3xW/yX6RJBdeipXq5ilyhd7G8k =; b=Ije1xgeW4gs6FJWTbThF2eTxAMrbkFeBhI6wfY1o97dQBQhb2yKSx7Ip546 0+IxUjO+zoYJPpUewOyGQ4B+fTWfKYcCp3NiQxym2vP0/Gf2dcv2O8cpkVZVaR11 xSfN0HoJOadpc0aGnY392pl1IT+zRyYgah8ah3dXPtvpEc3eEk2qLTQXd39Xnibt r9Gucl7E0q6J3Avt6q17nTYtUQpreeMMolPO8+O3sEu4xJzOAxtIEtZvh8xP2Vn0 eXnjaDV3rC+3+YvT6c1fweCCaqPxmbr8cF7Zr9ZEjPRMkHDogbSNLwnAqTpsZBeK 7mM8pftqXnAfg4jU2ZmnoBMrNJQ== Received: from nasanppmta03.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v0hdfaffq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:15 +0000 (GMT) Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA03.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 3BG0LEMw004968 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:14 GMT Received: from [169.254.0.1] (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.40; Fri, 15 Dec 2023 16:21:13 -0800 From: Elliot Berman Date: Fri, 15 Dec 2023 16:21:03 -0800 Subject: [PATCH RFC v15 22/30] virt: gunyah: guestmem: Initialize RM mem parcels from guestmem Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <20231215-gunyah-v15-22-192a5d872a30@quicinc.com> References: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> In-Reply-To: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> To: Alex Elder , Srinivas Kandagatla , Murali Nalajal , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Philip Derrin , Prakruthi Deepak Heragu , Jonathan Corbet , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Catalin Marinas , Will Deacon , Konrad Dybcio , Bjorn Andersson , Dmitry Baryshkov , "Fuad Tabba" , Sean Christopherson CC: , , , , , Elliot Berman X-Mailer: b4 0.13-dev X-ClientProxiedBy: nalasex01a.na.qualcomm.com (10.47.209.196) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: xlCdkxLoUm7ByqI1D0XitK2ULxKUNfce X-Proofpoint-GUID: xlCdkxLoUm7ByqI1D0XitK2ULxKUNfce X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 mlxlogscore=983 impostorscore=0 spamscore=0 lowpriorityscore=0 adultscore=0 clxscore=1015 priorityscore=1501 mlxscore=0 phishscore=0 bulkscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312160000 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785396218374024553 X-GMAIL-MSGID: 1785396218374024553 Gunyah Resource Manager sets up a virtual machine based on a device tree which lives in guest memory. Resource manager requires this memory to be provided as a memory parcel for it to read and manipulate. Implement a function to construct a memory parcel from a guestmem binding. Signed-off-by: Elliot Berman --- drivers/virt/gunyah/guest_memfd.c | 188 ++++++++++++++++++++++++++++++++++++++ drivers/virt/gunyah/vm_mgr.h | 6 ++ 2 files changed, 194 insertions(+) diff --git a/drivers/virt/gunyah/guest_memfd.c b/drivers/virt/gunyah/guest_memfd.c index c38380c4dc50..419ed617bd0b 100644 --- a/drivers/virt/gunyah/guest_memfd.c +++ b/drivers/virt/gunyah/guest_memfd.c @@ -544,3 +544,191 @@ int gunyah_gmem_modify_binding(struct gunyah_vm *ghvm, fput(file); return ret; } + +int gunyah_gmem_share_parcel(struct gunyah_vm *ghvm, struct gunyah_rm_mem_parcel *parcel, + u64 *gfn, u64 *nr) +{ + struct folio *folio, *prev_folio; + unsigned long nr_entries, i, j, start, end; + struct gunyah_gmem_binding *b; + bool lend; + u64 size; + int ret; + + if (!*nr) + return -EINVAL; + + down_read(&ghvm->mem_lock); + b = mtree_load(&ghvm->mem_layout, *gfn); + if (!b) { + ret = -ENOENT; + goto unlock; + } + + start = *gfn - b->gfn; + end = min(*gfn + *nr, b->gfn + b->nr) - b->gfn; + + nr_entries = 0; + prev_folio = NULL; + for (i = start + b->i_off; i < end + b->i_off;) { + folio = gunyah_gmem_get_folio(b->inode, i); /* A */ + if (!folio) { + ret = -ENOMEM; + goto out; + } + + if (!prev_folio || folio_next(prev_folio) != folio) + nr_entries++; + i = folio_index(folio) + folio_nr_pages(folio); + prev_folio = folio; + } + + parcel->mem_entries = + kcalloc(nr_entries, sizeof(*parcel->mem_entries), GFP_KERNEL); + if (!parcel->mem_entries) { + ret = -ENOMEM; + goto out; + } + + j = 0; + prev_folio = NULL; + size = 0; + for (i = start + b->i_off; i < end + b->i_off;) { + folio = filemap_get_folio(b->inode->i_mapping, i); /* B */ + if (WARN_ON(IS_ERR(folio))) { + ret = PTR_ERR(folio); + i = end + b->i_off; + goto out; + } + + if (!prev_folio || folio_next(prev_folio) != folio) { + if (likely(prev_folio)) { + parcel->mem_entries[j].size = cpu_to_le64(size); + size = 0; + j++; + BUG_ON(j == nr_entries); + } + parcel->mem_entries[j].phys_addr = cpu_to_le64(PFN_PHYS(folio_pfn(folio))); + } + size += folio_size(folio); + if (likely(prev_folio)) + folio_put(prev_folio); /* B */ + i = folio_index(folio) + folio_nr_pages(folio); + prev_folio = folio; + } + folio_put(prev_folio); /* B */ + BUG_ON(j + 1 != nr_entries); + parcel->mem_entries[j].size = cpu_to_le64(size); + parcel->n_mem_entries = nr_entries; + + lend = parcel->n_acl_entries == 1 || gunyah_guest_mem_is_lend(ghvm, b->flags); + if (lend) + parcel->n_acl_entries = 1; + + parcel->acl_entries = kcalloc(parcel->n_acl_entries, + sizeof(*parcel->acl_entries), GFP_KERNEL); + if (!parcel->n_acl_entries) { + ret = -ENOMEM; + goto free_entries; + } + + parcel->acl_entries[0].vmid = cpu_to_le16(ghvm->vmid); + if (b->flags & GUNYAH_MEM_ALLOW_READ) + parcel->acl_entries[0].perms |= GUNYAH_RM_ACL_R; + if (b->flags & GUNYAH_MEM_ALLOW_WRITE) + parcel->acl_entries[0].perms |= GUNYAH_RM_ACL_W; + if (b->flags & GUNYAH_MEM_ALLOW_EXEC) + parcel->acl_entries[0].perms |= GUNYAH_RM_ACL_X; + + if (!lend) { + u16 host_vmid; + + ret = gunyah_rm_get_vmid(ghvm->rm, &host_vmid); + if (ret) + goto free_acl; + + parcel->acl_entries[1].vmid = cpu_to_le16(host_vmid); + parcel->acl_entries[1].perms = GUNYAH_RM_ACL_R | GUNYAH_RM_ACL_W | GUNYAH_RM_ACL_X; + } + + parcel->mem_handle = GUNYAH_MEM_HANDLE_INVAL; + *nr = end; + folio = filemap_get_folio(b->inode->i_mapping, start); /* C */ + *gfn = folio_index(folio) - b->i_off + b->gfn; + folio_put(folio); /* C */ + + ret = gunyah_rm_mem_share(ghvm->rm, parcel); + if (ret) + dev_warn(ghvm->parent, "Failed to share parcel for DTB: %d\n", + ret); + + goto out; +free_acl: + kfree(parcel->acl_entries); + parcel->acl_entries = NULL; +free_entries: + kfree(parcel->mem_entries); + parcel->mem_entries = NULL; +out: + for (j = start + b->i_off; j < i;) { + folio = filemap_get_folio(b->inode->i_mapping, j); /* D */ + if (IS_ERR(folio)) + continue; + j = folio_index(folio) + folio_nr_pages(folio); + if (!ret) { + folio_set_private(folio); + WARN_ON(mtree_insert_range(&b->mt, folio_index(folio), + j - 1, folio, GFP_KERNEL)); + } + folio_unlock(folio); /* A */ + folio_put(folio); /* D */ + /* matching folio_put for A is done at + * (1) gunyah_gmem_reclaim_parcel or + * (2) after gunyah_gmem_parcel_to_paged, gunyah_vm_reclaim_folio + */ + } +unlock: + up_read(&ghvm->mem_lock); + return ret; +} + +int gunyah_gmem_reclaim_parcel(struct gunyah_vm *ghvm, + struct gunyah_rm_mem_parcel *parcel, u64 gfn, + u64 nr) +{ + struct gunyah_gmem_binding *b; + struct folio *folio; + unsigned long i; + int ret; + + if (parcel->mem_handle != GUNYAH_MEM_HANDLE_INVAL) { + ret = gunyah_rm_mem_reclaim(ghvm->rm, parcel); + if (ret) { + dev_err(ghvm->parent, "Failed to reclaim parcel: %d\n", + ret); + /* We can't reclaim the pages -- hold onto the pages + * forever because we don't know what state the memory + * is in + */ + return ret; + } + parcel->mem_handle = 0; + kfree(parcel->mem_entries); + kfree(parcel->acl_entries); + } + + down_read(&ghvm->mem_lock); + b = mtree_load(&ghvm->mem_layout, gfn); + if (WARN_ON(!b)) { + up_read(&ghvm->mem_lock); + return 0; + } + + i = gfn - b->gfn + b->i_off; + mt_for_each(&b->mt, folio, i, nr + b->i_off) { + mtree_erase(&b->mt, i); + folio_put(folio); /* A */ + } + up_read(&ghvm->mem_lock); + return 0; +} diff --git a/drivers/virt/gunyah/vm_mgr.h b/drivers/virt/gunyah/vm_mgr.h index 8f1c3ade08dd..1cad729b017a 100644 --- a/drivers/virt/gunyah/vm_mgr.h +++ b/drivers/virt/gunyah/vm_mgr.h @@ -84,5 +84,11 @@ int gunyah_gmem_modify_binding(struct gunyah_vm *ghvm, struct gunyah_map_mem_args *args); struct gunyah_gmem_binding; void gunyah_gmem_remove_binding(struct gunyah_gmem_binding *binding); +int gunyah_gmem_share_parcel(struct gunyah_vm *ghvm, + struct gunyah_rm_mem_parcel *parcel, u64 *gfn, + u64 *nr); +int gunyah_gmem_reclaim_parcel(struct gunyah_vm *ghvm, + struct gunyah_rm_mem_parcel *parcel, u64 gfn, + u64 nr); #endif From patchwork Sat Dec 16 00:21:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 179695 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:3b04:b0:fb:cd0c:d3e with SMTP id c4csp9667314dys; Fri, 15 Dec 2023 16:27:52 -0800 (PST) X-Google-Smtp-Source: AGHT+IGGvPN9MswyZCpC+grOng938FSOzqL37SRUWv5eT5vj7G47+H9q7aU0tGGWD74oY9/UnfXF X-Received: by 2002:a17:907:3102:b0:a19:a19b:55f7 with SMTP id wl2-20020a170907310200b00a19a19b55f7mr5226528ejb.135.1702686472321; Fri, 15 Dec 2023 16:27:52 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702686472; cv=none; d=google.com; s=arc-20160816; b=eaiRl+rgOzuYPupf2rJJjGf9mwmwSqSl9n81OBd8KylwqUd76cbc8QFs90bgAKJnts KtsV9wO34A/4PGXOWX2jk/7h0HzyJuPCBpXX3x80/iB7gmn4l9uXwq1qIQ4kUM/ECI+F oy6Add1c7toyhPqxyZ4CBtH5qFw7Z8oB6l6KeMq3nIsRAsxbG5pKIZh9Pj5bLLXtw7Xx TD6D//TsFXvdogS5v9kP3+uezo2FDHQ7jQTatcjbjyknkQ+uAFyDB4RPxko1K/BznE/G +b7yNM83vlefccgTDi09vcPxVzerLHWc9N9vTjBP3T1AOgyoDqtRxv8yPN0hQIWyMgfV 919Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:list-unsubscribe:list-subscribe:list-id:precedence :subject:date:from:dkim-signature; bh=yd0ujJbXUCG0HEeQ97AI8hL5w0yeYQxF/m/DtIuMX4M=; fh=Pv/sMsQia5wmqAOYHIQUkkUFpDKhZjuODj71Ojb2ybw=; b=Q3uZW+eCuVmP2HoEQzrwitWDXGkXSBJ+HygSKomPMK4DExOh1Lj0bD0rVT+DL4TXhE RQ7/OEgLX09lg0U2ZQP7V6Q/Tkkjs0JqiIqmM6COGS0mZKN4V16V8s8KJfcPPyLL9wwG lMjuhqXSipuJHprqvLqLBdYh8qSTUIb3GNmKynI+4FyQm6me6+otTVJynoFeTKLnhpNo 1p/M9acVYk67ocy2G2IEJuZIeP2j4120nI4YoSQO54olL4uBTFmpBtRFVnblbPlw/rjs WJLVhgb48srhddJN3yRgF5c146T37/89HGtgdIlyniMt3mcHOzG54xjUnsWrd6iZQfLO 7VwQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=mtI81Ov4; spf=pass (google.com: domain of linux-kernel+bounces-1885-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1885-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id k19-20020a1709063e1300b00a1b83f4208asi7604473eji.1003.2023.12.15.16.27.52 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Dec 2023 16:27:52 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-1885-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=mtI81Ov4; spf=pass (google.com: domain of linux-kernel+bounces-1885-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1885-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id C0BD31F23405 for ; Sat, 16 Dec 2023 00:27:51 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 3ED123454D; Sat, 16 Dec 2023 00:21:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="mtI81Ov4" X-Original-To: linux-kernel@vger.kernel.org Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3035B15487; Sat, 16 Dec 2023 00:21:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=quicinc.com Received: from pps.filterd (m0279864.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BG06JiB031931; Sat, 16 Dec 2023 00:21:15 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:date:subject:mime-version:content-type :content-transfer-encoding:message-id:references:in-reply-to:to :cc; s=qcppdkim1; bh=yd0ujJbXUCG0HEeQ97AI8hL5w0yeYQxF/m/DtIuMX4M =; b=mtI81Ov4K3HHtLbLSInBPxOaJdd9TQ6mR2ynK7Ojm/xS2ykode/WULKKRS0 rXJxcieeI49epCtHUjL5i1uOjH4DyDJJu7HzMIbn2n1r12xqn3c6blOw6R2Xt7Sd 1H9GFI9V1fAKTQxp4j038S9sXDdwHBfDAAfihYdmTFhaTDSUwnNldUCqyW7aYTb7 0xOi63M/zWJmcj8t+i26ix6McGSPj8qnG3c80mpavCJesrsdji5TBCfHtWXuNxQk X0LDLWoKLG/LCSCwZItZEFCyc8iDz/kBS1FtN9Je4qgvDJbYirGtoKuScWA08mw0 sClIjHCF0oKt/VV6v3bLQ/DX6Iw== Received: from nasanppmta05.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v0hqcjf8r-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:15 +0000 (GMT) Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA05.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 3BG0LFEk006377 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:15 GMT Received: from [169.254.0.1] (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.40; Fri, 15 Dec 2023 16:21:14 -0800 From: Elliot Berman Date: Fri, 15 Dec 2023 16:21:04 -0800 Subject: [PATCH RFC v15 23/30] virt: gunyah: Allow userspace to initialize context of primary vCPU Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <20231215-gunyah-v15-23-192a5d872a30@quicinc.com> References: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> In-Reply-To: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> To: Alex Elder , Srinivas Kandagatla , Murali Nalajal , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Philip Derrin , Prakruthi Deepak Heragu , Jonathan Corbet , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Catalin Marinas , Will Deacon , Konrad Dybcio , Bjorn Andersson , Dmitry Baryshkov , "Fuad Tabba" , Sean Christopherson CC: , , , , , Elliot Berman X-Mailer: b4 0.13-dev X-ClientProxiedBy: nalasex01a.na.qualcomm.com (10.47.209.196) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: 13KfA0gwHXNKQRz-5J2CMUNFxhXrVfsf X-Proofpoint-GUID: 13KfA0gwHXNKQRz-5J2CMUNFxhXrVfsf X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 mlxscore=0 phishscore=0 adultscore=0 priorityscore=1501 impostorscore=0 lowpriorityscore=0 mlxlogscore=822 clxscore=1015 suspectscore=0 spamscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312160001 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785396170725685851 X-GMAIL-MSGID: 1785396170725685851 RM provides APIs to fill boot context the initial registers upon starting the vCPU. Signed-off-by: Elliot Berman --- drivers/virt/gunyah/vm_mgr.c | 75 ++++++++++++++++++++++++++++++++++++++++++++ drivers/virt/gunyah/vm_mgr.h | 1 + include/uapi/linux/gunyah.h | 23 ++++++++++++++ 3 files changed, 99 insertions(+) diff --git a/drivers/virt/gunyah/vm_mgr.c b/drivers/virt/gunyah/vm_mgr.c index 5666070453aa..a31a9f0a642c 100644 --- a/drivers/virt/gunyah/vm_mgr.c +++ b/drivers/virt/gunyah/vm_mgr.c @@ -542,6 +542,66 @@ static __must_check struct gunyah_vm *gunyah_vm_alloc(struct gunyah_rm *rm) return ghvm; } +static long gunyah_vm_set_boot_context(struct gunyah_vm *ghvm, + struct gunyah_vm_boot_context *boot_ctx) +{ + u8 reg_set, reg_index; /* to check values are reasonable */ + int ret; + + reg_set = (boot_ctx->reg >> GUNYAH_VM_BOOT_CONTEXT_REG_SHIFT) & 0xff; + reg_index = boot_ctx->reg & 0xff; + + switch (reg_set) { + case REG_SET_X: + if (reg_index > 31) + return -EINVAL; + break; + case REG_SET_PC: + if (reg_index) + return -EINVAL; + break; + case REG_SET_SP: + if (reg_index > 2) + return -EINVAL; + break; + default: + return -EINVAL; + } + + ret = down_read_interruptible(&ghvm->status_lock); + if (ret) + return ret; + + if (ghvm->vm_status != GUNYAH_RM_VM_STATUS_NO_STATE) { + ret = -EINVAL; + goto out; + } + + ret = xa_err(xa_store(&ghvm->boot_context, boot_ctx->reg, + (void *)boot_ctx->value, GFP_KERNEL)); +out: + up_read(&ghvm->status_lock); + return ret; +} + +static inline int gunyah_vm_fill_boot_context(struct gunyah_vm *ghvm) +{ + unsigned long reg_set, reg_index, id; + void *entry; + int ret; + + xa_for_each(&ghvm->boot_context, id, entry) { + reg_set = (id >> GUNYAH_VM_BOOT_CONTEXT_REG_SHIFT) & 0xff; + reg_index = id & 0xff; + ret = gunyah_rm_vm_set_boot_context( + ghvm->rm, ghvm->vmid, reg_set, reg_index, (u64)entry); + if (ret) + return ret; + } + + return 0; +} + static int gunyah_vm_start(struct gunyah_vm *ghvm) { struct gunyah_rm_hyp_resources *resources; @@ -582,6 +642,13 @@ static int gunyah_vm_start(struct gunyah_vm *ghvm) } ghvm->vm_status = GUNYAH_RM_VM_STATUS_READY; + ret = gunyah_vm_fill_boot_context(ghvm); + if (ret) { + dev_warn(ghvm->parent, "Failed to setup boot context: %d\n", + ret); + goto err; + } + ret = gunyah_rm_get_hyp_resources(ghvm->rm, ghvm->vmid, &resources); if (ret) { dev_warn(ghvm->parent, @@ -683,6 +750,14 @@ static long gunyah_vm_ioctl(struct file *filp, unsigned int cmd, return gunyah_gmem_modify_binding(ghvm, &args); } + case GUNYAH_VM_SET_BOOT_CONTEXT: { + struct gunyah_vm_boot_context boot_ctx; + + if (copy_from_user(&boot_ctx, argp, sizeof(boot_ctx))) + return -EFAULT; + + return gunyah_vm_set_boot_context(ghvm, &boot_ctx); + } default: r = -ENOTTY; break; diff --git a/drivers/virt/gunyah/vm_mgr.h b/drivers/virt/gunyah/vm_mgr.h index 1cad729b017a..a7fc7c376d1e 100644 --- a/drivers/virt/gunyah/vm_mgr.h +++ b/drivers/virt/gunyah/vm_mgr.h @@ -60,6 +60,7 @@ struct gunyah_vm { struct list_head resource_tickets; struct rb_root mmio_handler_root; struct rw_semaphore mmio_handler_lock; + struct xarray boot_context; }; int gunyah_vm_mmio_write(struct gunyah_vm *ghvm, u64 addr, u32 len, u64 data); diff --git a/include/uapi/linux/gunyah.h b/include/uapi/linux/gunyah.h index 1af4c5ae6bc3..a171185191ae 100644 --- a/include/uapi/linux/gunyah.h +++ b/include/uapi/linux/gunyah.h @@ -128,6 +128,29 @@ struct gunyah_map_mem_args { #define GUNYAH_VM_MAP_MEM _IOW(GUNYAH_IOCTL_TYPE, 0x9, struct gunyah_map_mem_args) +enum gunyah_vm_boot_context_reg { + REG_SET_X = 0, + REG_SET_PC = 1, + REG_SET_SP = 2, +}; + +#define GUNYAH_VM_BOOT_CONTEXT_REG_SHIFT 8 +#define GUNYAH_VM_BOOT_CONTEXT_REG(reg, idx) (((reg & 0xff) << GUNYAH_VM_BOOT_CONTEXT_REG_SHIFT) |\ + (idx & 0xff)) + +/** + * struct gunyah_vm_boot_context - Set an initial register for the VM + * @reg: Register to set. See GUNYAH_VM_BOOT_CONTEXT_REG_* macros + * @reserved: reserved for alignment + * @value: value to fill in the register + */ +struct gunyah_vm_boot_context { + __u32 reg; + __u32 reserved; + __u64 value; +}; +#define GUNYAH_VM_SET_BOOT_CONTEXT _IOW(GUNYAH_IOCTL_TYPE, 0xa, struct gunyah_vm_boot_context) + /* * ioctls for vCPU fds */ From patchwork Sat Dec 16 00:21:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 179701 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:3b04:b0:fb:cd0c:d3e with SMTP id c4csp9667996dys; Fri, 15 Dec 2023 16:29:44 -0800 (PST) X-Google-Smtp-Source: AGHT+IFZLYVG8og3J1kJ5ZFBp4ZtN6QSxMt4i7Aw18Pki2MUypUm9TlcwPo4/lPOt8myZngjWaes X-Received: by 2002:a17:907:7d9e:b0:a1c:9821:bed8 with SMTP id oz30-20020a1709077d9e00b00a1c9821bed8mr4091225ejc.110.1702686584607; Fri, 15 Dec 2023 16:29:44 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702686584; cv=none; d=google.com; s=arc-20160816; b=UyWI1rfs/J6V/hLZIFXvf8DEc1Qecy3oAsVFupczW1j4EwmLmrf8dPDhTMjE1Tu2al QeK80LHT0YUNAFVjAD9p/YnS8Mo3VDWwGta+0i8SdaQyxsbB0hiBGeQThG87cxxg2bFI a1MU8xKTWsqSaIcEfOF8MmtFEH6JCOcd9S4wAc86rou1XXA41Vu5Mqk4/KgtdIyK/yA9 Y+Nrn3+0im+Wezfd5EUzkPYgqJF3s1Jt0GFRHD8O106KlcgacEHhGuk9DUk/ZHHwkrKs CsrAHUGyMTRAMdgQVYcIbn6BZuBxuCTVN2Q51Ij4l4I6sALAXWsu+6J5onMy2Si3KcVQ KtWw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:list-unsubscribe:list-subscribe:list-id:precedence :subject:date:from:dkim-signature; bh=+fJqyZjDlK1gDGsRn0b0lLQzU8UiFh6dKZ7oqf9fYOE=; fh=Pv/sMsQia5wmqAOYHIQUkkUFpDKhZjuODj71Ojb2ybw=; b=llMUvrdCcW3u4v/f6ye16+q+RHj0gR8gd7M5GOK1XoDeOvbeF6h8Furif1tqt9ic+b nN1gyWb0rhXWrHRz3I8leYrobkyF5uIbpnWp2RnioB0jxpdu88vAmis3plk3U8vgLP2k QK+LTW9pZaRa1JB8yhKu+OpXae1Cm5JNbnJhO5g6znW1R06hYl046VJwioEGMi9QrGGL yYJLYnhaD/cBN1t5pYEfNj4E094xxFpcJBBLHoHJSVLDpnfkl5TFz+D0eV2LUO1hSF3T 7qsZ6debDFxA1WT1HoD5SEvSgWstPiPRrH0Bu1kONPzWr4mA2mKCu8N6ohZq7H8vTFDN S9jg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=i7dTOIUG; spf=pass (google.com: domain of linux-kernel+bounces-1891-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1891-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id s13-20020a170906bc4d00b00a1c4e411198si7414680ejv.560.2023.12.15.16.29.44 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Dec 2023 16:29:44 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-1891-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=i7dTOIUG; spf=pass (google.com: domain of linux-kernel+bounces-1891-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1891-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 03B5E1F259A5 for ; Sat, 16 Dec 2023 00:29:44 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 76A5939AEE; Sat, 16 Dec 2023 00:21:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="i7dTOIUG" X-Original-To: linux-kernel@vger.kernel.org Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EBE2731A82; Sat, 16 Dec 2023 00:21:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=quicinc.com Received: from pps.filterd (m0279868.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BFNqBDV031214; Sat, 16 Dec 2023 00:21:17 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:date:subject:mime-version:content-type :content-transfer-encoding:message-id:references:in-reply-to:to :cc; s=qcppdkim1; bh=+fJqyZjDlK1gDGsRn0b0lLQzU8UiFh6dKZ7oqf9fYOE =; b=i7dTOIUGSEwrzjFglnIkm0nT1HhBTHwg3Ihy7QlVsEavs9kiO1xvF3rIRPm PNMb/IO0/EegW4GAY+893FG7FqiWo5lAwbkw/bq/eA6D8ttioL8tz7kYM3EZTu7f 2P+UESHAkj8+SsAtV6UTWtf2fVBAFYNrjbhIRZdoAcZ2drlbB0y3yqBhtacH00sb OpCn4j2GHSKN3LETeEfMWUqzJMIUrReZMk1Ly4cNh0j2+Q02IeCigDIFZlLxcojF Xx8mxbOBTowT27nnTq2eyZnkOsRwyyTQMFvDjd6yrNF/xq6eRmv/sTEGCnNlCF1t YNIx0F2EK5gQQSbiEoo7HG1aXXA== Received: from nasanppmta03.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v0p101kj5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:16 +0000 (GMT) Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA03.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 3BG0LF0k004978 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:15 GMT Received: from [169.254.0.1] (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.40; Fri, 15 Dec 2023 16:21:14 -0800 From: Elliot Berman Date: Fri, 15 Dec 2023 16:21:05 -0800 Subject: [PATCH RFC v15 24/30] virt: gunyah: Share guest VM dtb configuration to Gunyah Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <20231215-gunyah-v15-24-192a5d872a30@quicinc.com> References: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> In-Reply-To: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> To: Alex Elder , Srinivas Kandagatla , Murali Nalajal , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Philip Derrin , Prakruthi Deepak Heragu , Jonathan Corbet , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Catalin Marinas , Will Deacon , Konrad Dybcio , Bjorn Andersson , Dmitry Baryshkov , "Fuad Tabba" , Sean Christopherson CC: , , , , , Elliot Berman X-Mailer: b4 0.13-dev X-ClientProxiedBy: nalasex01a.na.qualcomm.com (10.47.209.196) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: mWFg4ml3dCyYjzeM2T6lRlK1dpgLLWe4 X-Proofpoint-ORIG-GUID: mWFg4ml3dCyYjzeM2T6lRlK1dpgLLWe4 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 adultscore=0 spamscore=0 bulkscore=0 mlxlogscore=999 lowpriorityscore=0 priorityscore=1501 phishscore=0 malwarescore=0 impostorscore=0 clxscore=1015 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312160000 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785396288222778276 X-GMAIL-MSGID: 1785396288222778276 Gunyah Resource Manager sets up a virtual machine based on a device tree which lives in guest memory. Resource manager requires this memory to be provided as a memory parcel for it to read and manipulate. Construct a memory parcel, lend it to the virtual machine, and inform resource manager about the device tree location (the memory parcel ID and offset into the memory parcel). Signed-off-by: Elliot Berman --- drivers/virt/gunyah/vm_mgr.c | 46 ++++++++++++++++++++++++++++++++++++++++++-- drivers/virt/gunyah/vm_mgr.h | 6 ++++++ include/uapi/linux/gunyah.h | 14 ++++++++++++++ 3 files changed, 64 insertions(+), 2 deletions(-) diff --git a/drivers/virt/gunyah/vm_mgr.c b/drivers/virt/gunyah/vm_mgr.c index a31a9f0a642c..8f8d962c9f8c 100644 --- a/drivers/virt/gunyah/vm_mgr.c +++ b/drivers/virt/gunyah/vm_mgr.c @@ -627,8 +627,27 @@ static int gunyah_vm_start(struct gunyah_vm *ghvm) ghvm->vmid = ret; ghvm->vm_status = GUNYAH_RM_VM_STATUS_LOAD; - ret = gunyah_rm_vm_configure(ghvm->rm, ghvm->vmid, ghvm->auth, 0, 0, 0, - 0, 0); + ghvm->dtb.parcel_start = ghvm->dtb.config.guest_phys_addr >> PAGE_SHIFT; + ghvm->dtb.parcel_pages = ghvm->dtb.config.size >> PAGE_SHIFT; + /* RM requires the DTB parcel to be lent to guard against malicious + * modifications while starting VM. Force it so. + */ + ghvm->dtb.parcel.n_acl_entries = 1; + ret = gunyah_gmem_share_parcel(ghvm, &ghvm->dtb.parcel, + &ghvm->dtb.parcel_start, + &ghvm->dtb.parcel_pages); + if (ret) { + dev_warn(ghvm->parent, + "Failed to allocate parcel for DTB: %d\n", ret); + goto err; + } + + ret = gunyah_rm_vm_configure(ghvm->rm, ghvm->vmid, ghvm->auth, + ghvm->dtb.parcel.mem_handle, 0, 0, + ghvm->dtb.config.guest_phys_addr - + (ghvm->dtb.parcel_start + << PAGE_SHIFT), + ghvm->dtb.config.size); if (ret) { dev_warn(ghvm->parent, "Failed to configure VM: %d\n", ret); goto err; @@ -720,6 +739,21 @@ static long gunyah_vm_ioctl(struct file *filp, unsigned int cmd, long r; switch (cmd) { + case GUNYAH_VM_SET_DTB_CONFIG: { + struct gunyah_vm_dtb_config dtb_config; + + if (copy_from_user(&dtb_config, argp, sizeof(dtb_config))) + return -EFAULT; + + if (overflows_type(dtb_config.guest_phys_addr + dtb_config.size, + u64)) + return -EOVERFLOW; + + ghvm->dtb.config = dtb_config; + + r = 0; + break; + } case GUNYAH_VM_START: { r = gunyah_vm_ensure_started(ghvm); break; @@ -783,6 +817,14 @@ static void _gunyah_vm_put(struct kref *kref) gunyah_vm_stop(ghvm); gunyah_vm_remove_functions(ghvm); + if (ghvm->vm_status == GUNYAH_RM_VM_STATUS_LOAD) { + ret = gunyah_gmem_reclaim_parcel(ghvm, &ghvm->dtb.parcel, + ghvm->dtb.parcel_start, + ghvm->dtb.parcel_pages); + if (ret) + dev_err(ghvm->parent, + "Failed to reclaim DTB parcel: %d\n", ret); + } mt_for_each(&ghvm->mem_layout, b, idx, ULONG_MAX) gunyah_gmem_remove_binding(b); diff --git a/drivers/virt/gunyah/vm_mgr.h b/drivers/virt/gunyah/vm_mgr.h index a7fc7c376d1e..357414e67d72 100644 --- a/drivers/virt/gunyah/vm_mgr.h +++ b/drivers/virt/gunyah/vm_mgr.h @@ -60,6 +60,12 @@ struct gunyah_vm { struct list_head resource_tickets; struct rb_root mmio_handler_root; struct rw_semaphore mmio_handler_lock; + + struct { + struct gunyah_vm_dtb_config config; + u64 parcel_start, parcel_pages; + struct gunyah_rm_mem_parcel parcel; + } dtb; struct xarray boot_context; }; diff --git a/include/uapi/linux/gunyah.h b/include/uapi/linux/gunyah.h index a171185191ae..574116f54472 100644 --- a/include/uapi/linux/gunyah.h +++ b/include/uapi/linux/gunyah.h @@ -42,6 +42,20 @@ struct gunyah_create_mem_args { /* * ioctls for gunyah-vm fds (returned by GUNYAH_CREATE_VM) */ + +/** + * struct gunyah_vm_dtb_config - Set the location of the VM's devicetree blob + * @guest_phys_addr: Address of the VM's devicetree in guest memory. + * @size: Maximum size of the devicetree including space for overlays. + * Resource manager applies an overlay to the DTB and dtb_size should + * include room for the overlay. A page of memory is typicaly plenty. + */ +struct gunyah_vm_dtb_config { + __u64 guest_phys_addr; + __u64 size; +}; +#define GUNYAH_VM_SET_DTB_CONFIG _IOW(GUNYAH_IOCTL_TYPE, 0x2, struct gunyah_vm_dtb_config) + #define GUNYAH_VM_START _IO(GUNYAH_IOCTL_TYPE, 0x3) /** From patchwork Sat Dec 16 00:21:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 179704 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:3b04:b0:fb:cd0c:d3e with SMTP id c4csp9668479dys; Fri, 15 Dec 2023 16:30:49 -0800 (PST) X-Google-Smtp-Source: AGHT+IHCuW9EApPcdHeXJ20LvqpIpBhdedupKZaPCvjTt9xVMHMN7X4CjmZuY2y6hSucbhbwVOFq X-Received: by 2002:a05:622a:509:b0:425:4043:29e8 with SMTP id l9-20020a05622a050900b00425404329e8mr15683378qtx.99.1702686649449; Fri, 15 Dec 2023 16:30:49 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702686649; cv=none; d=google.com; s=arc-20160816; b=PUIpuOYEIgiU9YPgWpNA7Yd1vy0WXKUq0qaAiT88QK1CS9LSCL43c94cNoPEvprvfy 4SYMalTC2VVgYWW53+ieu92KX1WDGzXXChGDB4SNtAphfVA4pHfASlJZrZng9z+SkLWL ajfvZ1c5K5NyxARJrIqjj23P13XABL3yfp67bNAdNZLHejGIXCwR1XfMwrClOZHli7cl KqVqHwsEcaf/3ohfxaLCUALb4S9LbgGb05WTW4sqoL1ljzUwuNuH08LWmbyVCrRT2QGL l5CByfxqGxTfgoBoLQ5LlByoGahPRkAyr+ezpfovmezN9BXOQnd2Xm6th8n4Akj+vYj0 cfWw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:list-unsubscribe:list-subscribe:list-id:precedence :subject:date:from:dkim-signature; bh=8a59vkuZ6on+rnqsHsk3e+kiU8WVcrEgmmuTlTWOVYI=; fh=Pv/sMsQia5wmqAOYHIQUkkUFpDKhZjuODj71Ojb2ybw=; b=yvMyvSSTfD9vs4+V9KnGSJmalsJ12XJ6rsqdS0LeQKfOpJhHHCAC0HZ9VTMZIYOsHU /yFniHHQ3enMtwPLqjzCX40Hu4vIylt8dKrGG5xFRrBE1hiwfD5ysnC5+i4JyTDNCr1J UMGRw8dnyvWhM7oWYdPEiUPWhniQFxbUSmxbQ14O6PuPEImuQjs1r6A9oDRNTOYdoXCx PBErj+oDCEiNQR3v1kOWnc68V318FXHEyaYhlIgU5sP+QfogUYuUX8C7CYJELX49/o8f ObzvwZ/wHu9xUFATG88t89yLLH926YfQAp+MDs9MAuBD0JfhM+UYrMq6fi6kxyAZysIO Yp1A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=E0uFamLU; spf=pass (google.com: domain of linux-kernel+bounces-1894-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1894-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id fk21-20020a05622a559500b004237645a813si19710103qtb.417.2023.12.15.16.30.49 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Dec 2023 16:30:49 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-1894-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=E0uFamLU; spf=pass (google.com: domain of linux-kernel+bounces-1894-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1894-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 2E55D1C23D99 for ; Sat, 16 Dec 2023 00:30:49 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 87EE33FE3E; Sat, 16 Dec 2023 00:21:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="E0uFamLU" X-Original-To: linux-kernel@vger.kernel.org Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C43073527A; Sat, 16 Dec 2023 00:21:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=quicinc.com Received: from pps.filterd (m0279870.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BFNnCBF026098; Sat, 16 Dec 2023 00:21:17 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:date:subject:mime-version:content-type :content-transfer-encoding:message-id:references:in-reply-to:to :cc; s=qcppdkim1; bh=8a59vkuZ6on+rnqsHsk3e+kiU8WVcrEgmmuTlTWOVYI =; b=E0uFamLUPsT3I/kre60qY9ToAxwgwpsLkrdZZ1eXWaTIY5IAsytUo/d8ivG RP3EwL4vpR1KyNhN0KMmM9OfhsBD6rNxkibbihcJpRa/XUSRar3BX8+VrTdY7Lkh E20iCmx7zi7Xh97HwF0lcf8fVXNfXx4G9SA7BC1VdY6dpNyC8UYOetO1lweNegWi QHb3iIIV/qnlKyfffLhMN1gjoknGtTyV9e6Z6YXvfH/Ma6SqPt5dDpuZBwwPPT+V sbYPx5D6wj6EqLifngDz8lBd/ZIoBiHQKIYADplyUfpm6tNEPZkTsuQBSELtXROj MZW/QkCPxh3Qi4e+s0N0Ac3azbQ== Received: from nasanppmta01.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v0vv9ghjd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:17 +0000 (GMT) Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 3BG0LGVS015677 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:16 GMT Received: from [169.254.0.1] (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.40; Fri, 15 Dec 2023 16:21:15 -0800 From: Elliot Berman Date: Fri, 15 Dec 2023 16:21:06 -0800 Subject: [PATCH RFC v15 25/30] virt: gunyah: Enable demand paging Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <20231215-gunyah-v15-25-192a5d872a30@quicinc.com> References: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> In-Reply-To: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> To: Alex Elder , Srinivas Kandagatla , Murali Nalajal , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Philip Derrin , Prakruthi Deepak Heragu , Jonathan Corbet , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Catalin Marinas , Will Deacon , Konrad Dybcio , Bjorn Andersson , Dmitry Baryshkov , "Fuad Tabba" , Sean Christopherson CC: , , , , , Elliot Berman X-Mailer: b4 0.13-dev X-ClientProxiedBy: nalasex01a.na.qualcomm.com (10.47.209.196) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: i9H9U3HfAXjviR2sQmCMSiM37QlrkT9J X-Proofpoint-ORIG-GUID: i9H9U3HfAXjviR2sQmCMSiM37QlrkT9J X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 impostorscore=0 phishscore=0 clxscore=1015 adultscore=0 malwarescore=0 priorityscore=1501 lowpriorityscore=0 suspectscore=0 mlxlogscore=633 mlxscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312160001 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785396356200895506 X-GMAIL-MSGID: 1785396356200895506 Tell resource manager to enable demand paging and wire vCPU faults to provide the backing folio when a guestmemfd is bound to the faulting access. Signed-off-by: Elliot Berman --- drivers/virt/gunyah/guest_memfd.c | 92 +++++++++++++++++++++++++++++++++++++++ drivers/virt/gunyah/gunyah_vcpu.c | 39 ++++++++++++++++- drivers/virt/gunyah/vm_mgr.c | 17 ++++++++ drivers/virt/gunyah/vm_mgr.h | 3 ++ include/linux/gunyah.h | 2 + 5 files changed, 152 insertions(+), 1 deletion(-) diff --git a/drivers/virt/gunyah/guest_memfd.c b/drivers/virt/gunyah/guest_memfd.c index 419ed617bd0b..206ce51c9a7d 100644 --- a/drivers/virt/gunyah/guest_memfd.c +++ b/drivers/virt/gunyah/guest_memfd.c @@ -732,3 +732,95 @@ int gunyah_gmem_reclaim_parcel(struct gunyah_vm *ghvm, up_read(&ghvm->mem_lock); return 0; } + +int gunyah_gmem_setup_demand_paging(struct gunyah_vm *ghvm) +{ + struct gunyah_rm_mem_entry *entries; + struct gunyah_gmem_binding *b; + unsigned long index = 0; + u32 count = 0, i; + int ret = 0; + + down_read(&ghvm->mem_lock); + mt_for_each(&ghvm->mem_layout, b, index, ULONG_MAX) + if (gunyah_guest_mem_is_lend(ghvm, b->flags)) + count++; + + if (!count) + goto out; + + entries = kcalloc(count, sizeof(*entries), GFP_KERNEL); + if (!entries) { + ret = -ENOMEM; + goto out; + } + + index = i = 0; + mt_for_each(&ghvm->mem_layout, b, index, ULONG_MAX) { + if (!gunyah_guest_mem_is_lend(ghvm, b->flags)) + continue; + entries[i].phys_addr = cpu_to_le64(gunyah_gfn_to_gpa(b->gfn)); + entries[i].size = cpu_to_le64(b->nr << PAGE_SHIFT); + if (++i == count) + break; + } + + ret = gunyah_rm_vm_set_demand_paging(ghvm->rm, ghvm->vmid, i, entries); + kfree(entries); +out: + up_read(&ghvm->mem_lock); + return ret; +} + +int gunyah_gmem_demand_page(struct gunyah_vm *ghvm, u64 gpa, bool write) +{ + unsigned long gfn = gunyah_gpa_to_gfn(gpa); + struct gunyah_gmem_binding *b; + struct folio *folio; + int ret; + + down_read(&ghvm->mem_lock); + b = mtree_load(&ghvm->mem_layout, gfn); + if (!b) { + up_read(&ghvm->mem_lock); + return -ENOENT; + } + + if (write && !(b->flags & GUNYAH_MEM_ALLOW_WRITE)) + return -EPERM; + + folio = gunyah_gmem_get_folio(b->inode, gunyah_gfn_to_off(b, gfn)); + if (IS_ERR(folio)) { + up_read(&ghvm->mem_lock); + pr_err_ratelimited( + "Failed to obtain memory for guest addr %016llx: %ld\n", + gpa, PTR_ERR(folio)); + return PTR_ERR(folio); + } + + if (gunyah_guest_mem_is_lend(ghvm, b->flags) && folio_mapped(folio)) { + ret = -EPERM; + goto out; + } + + gfn = gunyah_off_to_gfn(b, folio_index(folio)); + + ret = gunyah_vm_provide_folio(ghvm, folio, gfn, + !gunyah_guest_mem_is_lend(ghvm, b->flags), + !!(b->flags & GUNYAH_MEM_ALLOW_WRITE)); + if (ret) { + pr_err_ratelimited( + "Failed to provide folio for guest addr: %016llx: %d\n", + gpa, ret); + goto out; + } + + if (gunyah_guest_mem_is_lend(ghvm, b->flags)) + folio_set_private(folio); +out: + folio_unlock(folio); + folio_put(folio); + up_read(&ghvm->mem_lock); + return ret; +} +EXPORT_SYMBOL_GPL(gunyah_gmem_demand_page); diff --git a/drivers/virt/gunyah/gunyah_vcpu.c b/drivers/virt/gunyah/gunyah_vcpu.c index aa40f5977cbf..1952a1797430 100644 --- a/drivers/virt/gunyah/gunyah_vcpu.c +++ b/drivers/virt/gunyah/gunyah_vcpu.c @@ -91,6 +91,29 @@ static irqreturn_t gunyah_vcpu_irq_handler(int irq, void *data) return IRQ_HANDLED; } +static bool gunyah_handle_page_fault( + struct gunyah_vcpu *vcpu, u64 state_data[3], + const struct gunyah_hypercall_vcpu_run_resp *vcpu_run_resp) +{ + int ret = 0; + u64 addr = vcpu_run_resp->state_data[0]; + bool write = !!vcpu_run_resp->state_data[1]; + + ret = gunyah_gmem_demand_page(vcpu->ghvm, addr, write); + if (ret) { + if (ret != -ENOENT) + pr_warn_ratelimited( + "Failed to provide guest address for: %08llx: %d\n", + addr, ret); + + vcpu->vcpu_run->page_fault.resume_action = GUNYAH_VCPU_RESUME_FAULT; + vcpu->vcpu_run->page_fault.attempt = ret; + vcpu->vcpu_run->page_fault.phys_addr = addr; + vcpu->vcpu_run->exit_reason = GUNYAH_VCPU_EXIT_PAGE_FAULT; + } + return !!ret; +} + static bool gunyah_handle_mmio(struct gunyah_vcpu *vcpu, u64 state_data[3], const struct gunyah_hypercall_vcpu_run_resp *vcpu_run_resp) @@ -103,6 +126,12 @@ gunyah_handle_mmio(struct gunyah_vcpu *vcpu, u64 state_data[3], if (WARN_ON(len > sizeof(u64))) len = sizeof(u64); + if (!gunyah_gmem_demand_page(vcpu->ghvm, addr, + vcpu->vcpu_run->mmio.is_write)) { + state_data[1] = GUNYAH_ADDRSPACE_VMMIO_ACTION_RETRY; + return true; + } + if (vcpu_run_resp->state == GUNYAH_VCPU_ADDRSPACE_VMMIO_READ) { vcpu->vcpu_run->mmio.is_write = 0; /* Record that we need to give vCPU user's supplied value next gunyah_vcpu_run() */ @@ -120,7 +149,8 @@ gunyah_handle_mmio(struct gunyah_vcpu *vcpu, u64 state_data[3], vcpu->state = GUNYAH_VCPU_MMIO_WRITE; } - vcpu->vcpu_run->mmio.phys_addr = addr; + vcpu->vcpu_run->mmio.resume_action = 0; + vcpu->mmio_addr = vcpu->vcpu_run->mmio.phys_addr = addr; vcpu->vcpu_run->mmio.len = len; vcpu->vcpu_run->exit_reason = GUNYAH_VCPU_EXIT_MMIO; @@ -144,6 +174,8 @@ static int gunyah_handle_mmio_resume(struct gunyah_vcpu *vcpu, u64 state_data[3] state_data[1] = GUNYAH_ADDRSPACE_VMMIO_ACTION_FAULT; break; case GUNYAH_VCPU_RESUME_RETRY: + gunyah_gmem_demand_page(vcpu->ghvm, vcpu->mmio_addr, + vcpu->state == GUNYAH_VCPU_MMIO_WRITE); state_data[1] = GUNYAH_ADDRSPACE_VMMIO_ACTION_RETRY; break; default: @@ -300,6 +332,11 @@ static int gunyah_vcpu_run(struct gunyah_vcpu *vcpu) &vcpu_run_resp)) goto out; break; + case GUNYAH_VCPU_ADDRSPACE_PAGE_FAULT: + if (gunyah_handle_page_fault(vcpu, resume_data, + &vcpu_run_resp)) + goto out; + break; default: pr_warn_ratelimited( "Unknown vCPU state: %llx\n", diff --git a/drivers/virt/gunyah/vm_mgr.c b/drivers/virt/gunyah/vm_mgr.c index 8f8d962c9f8c..0bb2b6a461a5 100644 --- a/drivers/virt/gunyah/vm_mgr.c +++ b/drivers/virt/gunyah/vm_mgr.c @@ -653,6 +653,23 @@ static int gunyah_vm_start(struct gunyah_vm *ghvm) goto err; } + ret = gunyah_gmem_setup_demand_paging(ghvm); + if (ret) { + dev_warn(ghvm->parent, + "Failed to set up gmem demand paging: %d\n", ret); + goto err; + } + + ret = gunyah_rm_vm_set_address_layout( + ghvm->rm, ghvm->vmid, GUNYAH_RM_RANGE_ID_IMAGE, + ghvm->dtb.parcel_start << PAGE_SHIFT, + ghvm->dtb.parcel_pages << PAGE_SHIFT); + if (ret) { + dev_warn(ghvm->parent, + "Failed to set location of DTB mem parcel: %d\n", ret); + goto err; + } + ret = gunyah_rm_vm_init(ghvm->rm, ghvm->vmid); if (ret) { ghvm->vm_status = GUNYAH_RM_VM_STATUS_INIT_FAILED; diff --git a/drivers/virt/gunyah/vm_mgr.h b/drivers/virt/gunyah/vm_mgr.h index 357414e67d72..088ed9c55376 100644 --- a/drivers/virt/gunyah/vm_mgr.h +++ b/drivers/virt/gunyah/vm_mgr.h @@ -98,4 +98,7 @@ int gunyah_gmem_reclaim_parcel(struct gunyah_vm *ghvm, struct gunyah_rm_mem_parcel *parcel, u64 gfn, u64 nr); +int gunyah_gmem_setup_demand_paging(struct gunyah_vm *ghvm); +int gunyah_gmem_demand_page(struct gunyah_vm *ghvm, u64 gpa, bool write); + #endif diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index ae9be5eef7a9..ed461acb9e1b 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -223,6 +223,8 @@ struct gunyah_hypercall_vcpu_run_resp { GUNYAH_VCPU_ADDRSPACE_VMMIO_READ = 4, /* VCPU has returned for MMIO WRITE */ GUNYAH_VCPU_ADDRSPACE_VMMIO_WRITE = 5, + /* Host needs to satisfy a page fault */ + GUNYAH_VCPU_ADDRSPACE_PAGE_FAULT = 7, /* clang-format on */ } state; u64 sized_state; From patchwork Sat Dec 16 00:21:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 179703 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:3b04:b0:fb:cd0c:d3e with SMTP id c4csp9668344dys; Fri, 15 Dec 2023 16:30:32 -0800 (PST) X-Google-Smtp-Source: AGHT+IGZo4pvH2zocHo3U+dtNk9NX4qvc+BY9PimUpvAtv6dE5tN2CNVDnj6Sn1u3TaLEiiIizNT X-Received: by 2002:a05:6808:2dcc:b0:3ba:f4a:4310 with SMTP id gn12-20020a0568082dcc00b003ba0f4a4310mr13158731oib.11.1702686631928; Fri, 15 Dec 2023 16:30:31 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702686631; cv=none; d=google.com; s=arc-20160816; b=cBV7sqJzch8Lk8EdBPDaLhXG3FpxibA8yYa3aB982sAj8+hp3bML6RWiuIWLbS5uQB bwKygewNMX3zt26/D6yIf5uYS0nCesZPizK2fOUR/5z0cto5hLgUugRqZhJoFY5KLq2j xUm4xLD/dEIv9+Nbmnz3l7fklEYNKkhVe7P/knpJh1NBkZnwQmKDun+25h1GZc35yoyv +xWqu1/gK07mt2QpqxBGWW4Hk7W8DTPcHYcremyKx3goQ0v9xkFT661/jJ3Y/9sr7fkk NyyZMUBdCfrgWa2vY2u/nG52XOkwVHTYvqLK7oLYAUrTcCsH+LmIyOJc8h9vfrQLGl+p psxQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:list-unsubscribe:list-subscribe:list-id:precedence :subject:date:from:dkim-signature; bh=AoJz6x3XHRsvMZUmjQluSOPQZI6QBrPLiwr7IUcs9qI=; fh=Pv/sMsQia5wmqAOYHIQUkkUFpDKhZjuODj71Ojb2ybw=; b=gs1xfJzEpDZgx1FWd4JDOo/onnBY293AwNsWLLScncbKQUfhsv27A0nQ/nsZ+PqDUt PdOnFYVNt29o5wDMwxEwkiznpwlcGIJ1c6w3lCGLavmO8TvI5yeUKes/UDEtr+zutU0/ Jq9dQVVE45G+Tia38TvKS57V0ZtgSgfoMFMvFQI3zChuhqO0oFlUsd977P4hg9omWd4i 1uXHZZEUHwG0muZTAhJAa07xpMM+fbjJWE5yseCLYTENlDdJEVmOgO/Nspu2o13ZpvYN Nl8okpJlN/0mB7QQ8C6OgkC1b5r5bpE1V3fwvEsv6iXLG/4co9QOwL3geCMYWC2r6H3K HVAA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=LBd4BiIk; spf=pass (google.com: domain of linux-kernel+bounces-1893-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1893-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id y21-20020a17090aa41500b0028a2f1852aasi13388525pjp.18.2023.12.15.16.30.31 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Dec 2023 16:30:31 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-1893-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=LBd4BiIk; spf=pass (google.com: domain of linux-kernel+bounces-1893-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1893-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 9D769281BA4 for ; Sat, 16 Dec 2023 00:30:31 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id DC8D13DB8D; Sat, 16 Dec 2023 00:21:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="LBd4BiIk" X-Original-To: linux-kernel@vger.kernel.org Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 69C1235263; Sat, 16 Dec 2023 00:21:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=quicinc.com Received: from pps.filterd (m0279864.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BG0A3mA005535; Sat, 16 Dec 2023 00:21:17 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:date:subject:mime-version:content-type :content-transfer-encoding:message-id:references:in-reply-to:to :cc; s=qcppdkim1; bh=AoJz6x3XHRsvMZUmjQluSOPQZI6QBrPLiwr7IUcs9qI =; b=LBd4BiIkq0UL8rGuEy1QUc1P6ieFNK6mqw+NsFobix2XJghYV2uMfsFtjdf QjfWxbVi7dJWoFxOxmmyPyVZDhEB75c4jXTZ+BsJjNZ6CPhBEU1lNUUz9QQ4+gBZ eEh7Z8yEGOxaPrYNtZ1DI5lcmTjT+gzj1JwLHCsHXqhOZ8Qlx/0OhZxKOsbRGWk4 Zo/8qdxSlyMMTL4pndzfEp6X1VvdZ5b1DmZ12xJs6bSr1lny8AjtCb5gKCZ+Fjdq UqIqoTWfWVhqx//q7H+XL4U7a7tsQYGhDzOYhrxtebOKoOKPbqA+OWTEH2Hj5nEk MMEFzONTklb0ZzaeOwoIY7gsz7A== Received: from nasanppmta02.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v0hqcjf8s-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:17 +0000 (GMT) Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA02.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 3BG0LGjD015189 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:16 GMT Received: from [169.254.0.1] (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.40; Fri, 15 Dec 2023 16:21:15 -0800 From: Elliot Berman Date: Fri, 15 Dec 2023 16:21:07 -0800 Subject: [PATCH RFC v15 26/30] virt: gunyah: Add Qualcomm Gunyah platform ops Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <20231215-gunyah-v15-26-192a5d872a30@quicinc.com> References: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> In-Reply-To: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> To: Alex Elder , Srinivas Kandagatla , Murali Nalajal , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Philip Derrin , Prakruthi Deepak Heragu , Jonathan Corbet , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Catalin Marinas , Will Deacon , Konrad Dybcio , Bjorn Andersson , Dmitry Baryshkov , "Fuad Tabba" , Sean Christopherson CC: , , , , , Elliot Berman X-Mailer: b4 0.13-dev X-ClientProxiedBy: nalasex01a.na.qualcomm.com (10.47.209.196) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: 5kHKOPXIMXrRSzcPItxdXfpH0sZPhm_H X-Proofpoint-GUID: 5kHKOPXIMXrRSzcPItxdXfpH0sZPhm_H X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 mlxscore=0 phishscore=0 adultscore=0 priorityscore=1501 impostorscore=0 lowpriorityscore=0 mlxlogscore=999 clxscore=1015 suspectscore=0 spamscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312160001 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785396337257303820 X-GMAIL-MSGID: 1785396337257303820 Qualcomm platforms have a firmware entity which performs access control to physical pages. Dynamically started Gunyah virtual machines use the QCOM_SCM_RM_MANAGED_VMID for access. Linux thus needs to assign access to the memory used by guest VMs. Gunyah doesn't do this operation for us since it is the current VM (typically VMID_HLOS) delegating the access and not Gunyah itself. Use the Gunyah platform ops to achieve this so that only Qualcomm platforms attempt to make the needed SCM calls. Reviewed-by: Alex Elder Co-developed-by: Prakruthi Deepak Heragu Signed-off-by: Prakruthi Deepak Heragu Signed-off-by: Elliot Berman --- drivers/virt/gunyah/Kconfig | 13 +++ drivers/virt/gunyah/Makefile | 1 + drivers/virt/gunyah/gunyah_qcom.c | 218 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 232 insertions(+) diff --git a/drivers/virt/gunyah/Kconfig b/drivers/virt/gunyah/Kconfig index 23ba523d25dc..fe2823dc48ba 100644 --- a/drivers/virt/gunyah/Kconfig +++ b/drivers/virt/gunyah/Kconfig @@ -4,6 +4,7 @@ config GUNYAH tristate "Gunyah Virtualization drivers" depends on ARM64 select GUNYAH_PLATFORM_HOOKS + imply GUNYAH_QCOM_PLATFORM if ARCH_QCOM help The Gunyah drivers are the helper interfaces that run in a guest VM such as basic inter-VM IPC and signaling mechanisms, and higher level @@ -14,3 +15,15 @@ config GUNYAH config GUNYAH_PLATFORM_HOOKS tristate + +config GUNYAH_QCOM_PLATFORM + tristate "Support for Gunyah on Qualcomm platforms" + depends on GUNYAH + select GUNYAH_PLATFORM_HOOKS + select QCOM_SCM + help + Enable support for interacting with Gunyah on Qualcomm + platforms. Interaction with Qualcomm firmware requires + extra platform-specific support. + + Say Y/M here to use Gunyah on Qualcomm platforms. diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index 9d514b2baa3a..c4505fce177d 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -4,3 +4,4 @@ gunyah_rsc_mgr-y += rsc_mgr.o rsc_mgr_rpc.o vm_mgr.o vm_mgr_mem.o guest_memfd.o obj-$(CONFIG_GUNYAH) += gunyah.o gunyah_rsc_mgr.o gunyah_vcpu.o obj-$(CONFIG_GUNYAH_PLATFORM_HOOKS) += gunyah_platform_hooks.o +obj-$(CONFIG_GUNYAH_QCOM_PLATFORM) += gunyah_qcom.o diff --git a/drivers/virt/gunyah/gunyah_qcom.c b/drivers/virt/gunyah/gunyah_qcom.c new file mode 100644 index 000000000000..58309e40da93 --- /dev/null +++ b/drivers/virt/gunyah/gunyah_qcom.c @@ -0,0 +1,218 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include +#include +#include +#include +#include +#include +#include + +#define QCOM_SCM_RM_MANAGED_VMID 0x3A +#define QCOM_SCM_MAX_MANAGED_VMID 0x3F + +static int +qcom_scm_gunyah_rm_pre_mem_share(struct gunyah_rm *rm, + struct gunyah_rm_mem_parcel *mem_parcel) +{ + struct qcom_scm_vmperm *new_perms __free(kfree) = NULL; + u64 src, src_cpy; + int ret = 0, i, n; + u16 vmid; + + new_perms = kcalloc(mem_parcel->n_acl_entries, sizeof(*new_perms), + GFP_KERNEL); + if (!new_perms) + return -ENOMEM; + + for (n = 0; n < mem_parcel->n_acl_entries; n++) { + vmid = le16_to_cpu(mem_parcel->acl_entries[n].vmid); + if (vmid <= QCOM_SCM_MAX_MANAGED_VMID) + new_perms[n].vmid = vmid; + else + new_perms[n].vmid = QCOM_SCM_RM_MANAGED_VMID; + if (mem_parcel->acl_entries[n].perms & GUNYAH_RM_ACL_X) + new_perms[n].perm |= QCOM_SCM_PERM_EXEC; + if (mem_parcel->acl_entries[n].perms & GUNYAH_RM_ACL_W) + new_perms[n].perm |= QCOM_SCM_PERM_WRITE; + if (mem_parcel->acl_entries[n].perms & GUNYAH_RM_ACL_R) + new_perms[n].perm |= QCOM_SCM_PERM_READ; + } + + src = BIT_ULL(QCOM_SCM_VMID_HLOS); + + for (i = 0; i < mem_parcel->n_mem_entries; i++) { + src_cpy = src; + ret = qcom_scm_assign_mem( + le64_to_cpu(mem_parcel->mem_entries[i].phys_addr), + le64_to_cpu(mem_parcel->mem_entries[i].size), &src_cpy, + new_perms, mem_parcel->n_acl_entries); + if (ret) + break; + } + + /* Did it work ok? */ + if (!ret) + return 0; + + src = 0; + for (n = 0; n < mem_parcel->n_acl_entries; n++) { + vmid = le16_to_cpu(mem_parcel->acl_entries[n].vmid); + if (vmid <= QCOM_SCM_MAX_MANAGED_VMID) + src |= BIT_ULL(vmid); + else + src |= BIT_ULL(QCOM_SCM_RM_MANAGED_VMID); + } + + new_perms[0].vmid = QCOM_SCM_VMID_HLOS; + + for (i--; i >= 0; i--) { + src_cpy = src; + WARN_ON_ONCE(qcom_scm_assign_mem( + le64_to_cpu(mem_parcel->mem_entries[i].phys_addr), + le64_to_cpu(mem_parcel->mem_entries[i].size), &src_cpy, + new_perms, 1)); + } + + return ret; +} + +static int +qcom_scm_gunyah_rm_post_mem_reclaim(struct gunyah_rm *rm, + struct gunyah_rm_mem_parcel *mem_parcel) +{ + struct qcom_scm_vmperm new_perms; + u64 src = 0, src_cpy; + int ret = 0, i, n; + u16 vmid; + + new_perms.vmid = QCOM_SCM_VMID_HLOS; + new_perms.perm = QCOM_SCM_PERM_EXEC | QCOM_SCM_PERM_WRITE | + QCOM_SCM_PERM_READ; + + for (n = 0; n < mem_parcel->n_acl_entries; n++) { + vmid = le16_to_cpu(mem_parcel->acl_entries[n].vmid); + if (vmid <= QCOM_SCM_MAX_MANAGED_VMID) + src |= (1ull << vmid); + else + src |= (1ull << QCOM_SCM_RM_MANAGED_VMID); + } + + for (i = 0; i < mem_parcel->n_mem_entries; i++) { + src_cpy = src; + ret = qcom_scm_assign_mem( + le64_to_cpu(mem_parcel->mem_entries[i].phys_addr), + le64_to_cpu(mem_parcel->mem_entries[i].size), &src_cpy, + &new_perms, 1); + WARN_ON_ONCE(ret); + } + + return ret; +} + +static int +qcom_scm_gunyah_rm_pre_demand_page(struct gunyah_rm *rm, u16 vmid, + enum gunyah_pagetable_access access, + struct folio *folio) +{ + struct qcom_scm_vmperm new_perms[2]; + unsigned int n = 1; + u64 src; + + new_perms[0].vmid = QCOM_SCM_RM_MANAGED_VMID; + new_perms[0].perm = QCOM_SCM_PERM_EXEC | QCOM_SCM_PERM_WRITE | + QCOM_SCM_PERM_READ; + if (access != GUNYAH_PAGETABLE_ACCESS_X && + access != GUNYAH_PAGETABLE_ACCESS_RX && + access != GUNYAH_PAGETABLE_ACCESS_RWX) { + new_perms[1].vmid = QCOM_SCM_VMID_HLOS; + new_perms[1].perm = QCOM_SCM_PERM_EXEC | QCOM_SCM_PERM_WRITE | + QCOM_SCM_PERM_READ; + n++; + } + + src = BIT_ULL(QCOM_SCM_VMID_HLOS); + + return qcom_scm_assign_mem(__pfn_to_phys(folio_pfn(folio)), + folio_size(folio), &src, new_perms, n); +} + +static int +qcom_scm_gunyah_rm_release_demand_page(struct gunyah_rm *rm, u16 vmid, + enum gunyah_pagetable_access access, + struct folio *folio) +{ + struct qcom_scm_vmperm new_perms; + u64 src; + + new_perms.vmid = QCOM_SCM_VMID_HLOS; + new_perms.perm = QCOM_SCM_PERM_EXEC | QCOM_SCM_PERM_WRITE | + QCOM_SCM_PERM_READ; + + src = BIT_ULL(QCOM_SCM_RM_MANAGED_VMID); + + if (access != GUNYAH_PAGETABLE_ACCESS_X && + access != GUNYAH_PAGETABLE_ACCESS_RX && + access != GUNYAH_PAGETABLE_ACCESS_RWX) + src |= BIT_ULL(QCOM_SCM_VMID_HLOS); + + return qcom_scm_assign_mem(__pfn_to_phys(folio_pfn(folio)), + folio_size(folio), &src, &new_perms, 1); +} + +static struct gunyah_rm_platform_ops qcom_scm_gunyah_rm_platform_ops = { + .pre_mem_share = qcom_scm_gunyah_rm_pre_mem_share, + .post_mem_reclaim = qcom_scm_gunyah_rm_post_mem_reclaim, + .pre_demand_page = qcom_scm_gunyah_rm_pre_demand_page, + .release_demand_page = qcom_scm_gunyah_rm_release_demand_page, +}; + +/* {19bd54bd-0b37-571b-946f-609b54539de6} */ +static const uuid_t QCOM_EXT_UUID = UUID_INIT(0x19bd54bd, 0x0b37, 0x571b, 0x94, + 0x6f, 0x60, 0x9b, 0x54, 0x53, + 0x9d, 0xe6); + +#define GUNYAH_QCOM_EXT_CALL_UUID_ID \ + ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, ARM_SMCCC_SMC_32, \ + ARM_SMCCC_OWNER_VENDOR_HYP, 0x3f01) + +static bool gunyah_has_qcom_extensions(void) +{ + struct arm_smccc_res res; + uuid_t uuid; + u32 *up; + + arm_smccc_1_1_smc(GUNYAH_QCOM_EXT_CALL_UUID_ID, &res); + + up = (u32 *)&uuid.b[0]; + up[0] = lower_32_bits(res.a0); + up[1] = lower_32_bits(res.a1); + up[2] = lower_32_bits(res.a2); + up[3] = lower_32_bits(res.a3); + + return uuid_equal(&uuid, &QCOM_EXT_UUID); +} + +static int __init qcom_gunyah_platform_hooks_register(void) +{ + if (!gunyah_has_qcom_extensions()) + return -ENODEV; + + pr_info("Enabling Gunyah hooks for Qualcomm platforms.\n"); + + return gunyah_rm_register_platform_ops( + &qcom_scm_gunyah_rm_platform_ops); +} + +static void __exit qcom_gunyah_platform_hooks_unregister(void) +{ + gunyah_rm_unregister_platform_ops(&qcom_scm_gunyah_rm_platform_ops); +} + +module_init(qcom_gunyah_platform_hooks_register); +module_exit(qcom_gunyah_platform_hooks_unregister); +MODULE_DESCRIPTION("Qualcomm Technologies, Inc. Platform Hooks for Gunyah"); +MODULE_LICENSE("GPL"); From patchwork Sat Dec 16 00:21:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 179705 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:3b04:b0:fb:cd0c:d3e with SMTP id c4csp9668496dys; Fri, 15 Dec 2023 16:30:52 -0800 (PST) X-Google-Smtp-Source: AGHT+IGMEyYGF2td39tmJ3t01Aws61LJJs8RqyYvHOIbOCqPZOBsciMcwPMf9yO3VaRT3WM9M8p4 X-Received: by 2002:a17:903:228b:b0:1d0:6ffe:a1a with SMTP id b11-20020a170903228b00b001d06ffe0a1amr13980990plh.120.1702686652181; Fri, 15 Dec 2023 16:30:52 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702686652; cv=none; d=google.com; s=arc-20160816; b=fhS6/+hieN+8MV8b5sf76/vTNoin4aRUpBe3jVQ5eGbdlyywS1h7nG4heuVXAvNWQP Puswr/i/5g5U6q7zM+4WHWdpakxggig5z/jboSorReL+9mfFfKo8qOz5bshxbufYzOtm oUxZbNpXCAYTVk9IobzhAi/QFBG2JxtAVTXpLe2s5S0sSoMP8Nmg+ILntqlYvLigDomV Bxlb+i8mw5+aj6f+S/Ka1HlCZ7FUd5/xP2oh64MocQ3IPKcSOIov/U65Q6F5c6qGdbSV paUIP84IBdO6yFy4db+MP/se0S7P2ddzVUze11wHpri01JQXsyxxJXESPRDa+HsUnSY+ Mpmw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:list-unsubscribe:list-subscribe:list-id:precedence :subject:date:from:dkim-signature; bh=HbIRZSqoWG7KxhoNG6rBUmi9dxk4637qd09P0tEjKhM=; fh=Pv/sMsQia5wmqAOYHIQUkkUFpDKhZjuODj71Ojb2ybw=; b=YuQuF0aPz0w1d648jYV3jw/+AldSivZwl8i/g5j5n72dtSiHmJcMGMcWCH6Yltqmhz QQPIpAmXXLwm2lrIOFTVb6Z402VTJfi+KD8UOpAlRMparaLR9lVWPHOUz+I6n/cur8lA qctNXYmPP5FR8AZZY1T9G2EyyItvWCXDWwvNc8WjRge7BmY5b9P3/r8EWbXdsO0ohlWd uZymyWghuAwTnbnrcsto8DWY+dduEotf9zVald3vZMQkEWyA1j/udTA+OLJJ1ZtyMlQb CLTQLVWKOWeqiQvLHvWJHMoI/zRe/8JJXSpeHTUMeLfity/BAa+q8mzZL6pKxGv4khT0 Oiiw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=fCYUWlLw; spf=pass (google.com: domain of linux-kernel+bounces-1895-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1895-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id b16-20020a170903229000b001d00ff4bf6csi14051111plh.466.2023.12.15.16.30.52 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Dec 2023 16:30:52 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-1895-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=fCYUWlLw; spf=pass (google.com: domain of linux-kernel+bounces-1895-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1895-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id EAAB32854DD for ; Sat, 16 Dec 2023 00:30:51 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id A3D3D405C9; Sat, 16 Dec 2023 00:21:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="fCYUWlLw" X-Original-To: linux-kernel@vger.kernel.org Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 20AA035298; Sat, 16 Dec 2023 00:21:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=quicinc.com Received: from pps.filterd (m0279869.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BFNo4i2000712; Sat, 16 Dec 2023 00:21:18 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:date:subject:mime-version:content-type :content-transfer-encoding:message-id:references:in-reply-to:to :cc; s=qcppdkim1; bh=HbIRZSqoWG7KxhoNG6rBUmi9dxk4637qd09P0tEjKhM =; b=fCYUWlLwaLcT+izAujed43ZDgvbQxtPynz2pWFIVEqm/q9Ik1cTzFpLy5wJ CoeAgiSqdshuGjYOtZeZAeEUYAlY0M56U4BZd6jXdWvhg+MONzZHiDkflpswdLNM jfZvTSk6Hh34gV9zaay6G7zhsQ7CZ8JFB8lUtZEbcM5dq+nGdpuLXJPa51BMUpfe YVhjqBvI8iTJSVWm0Y1j4K4DhKkj42WbqH953IVK9C6mQg0DU5/Dlg99SB5R+Eqb +QYFEJt/IQ78j63lqiXosBX7WcEisdcJZWAczRSCycjbqo67nbtJNM6AbgRfPgJb AJHA8O36WR/JQwjJRcL8FlcAb3Q== Received: from nasanppmta02.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v0pqvshm9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:18 +0000 (GMT) Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA02.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 3BG0LHjR015197 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:17 GMT Received: from [169.254.0.1] (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.40; Fri, 15 Dec 2023 16:21:16 -0800 From: Elliot Berman Date: Fri, 15 Dec 2023 16:21:08 -0800 Subject: [PATCH RFC v15 27/30] virt: gunyah: Add hypercalls for sending doorbell Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <20231215-gunyah-v15-27-192a5d872a30@quicinc.com> References: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> In-Reply-To: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> To: Alex Elder , Srinivas Kandagatla , Murali Nalajal , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Philip Derrin , Prakruthi Deepak Heragu , Jonathan Corbet , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Catalin Marinas , Will Deacon , Konrad Dybcio , Bjorn Andersson , Dmitry Baryshkov , "Fuad Tabba" , Sean Christopherson CC: , , , , , Elliot Berman X-Mailer: b4 0.13-dev X-ClientProxiedBy: nalasex01a.na.qualcomm.com (10.47.209.196) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: BMpa7lRJXJVJeVCGErXNrQtszqlauLaE X-Proofpoint-ORIG-GUID: BMpa7lRJXJVJeVCGErXNrQtszqlauLaE X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 spamscore=0 malwarescore=0 priorityscore=1501 impostorscore=0 bulkscore=0 adultscore=0 suspectscore=0 clxscore=1015 mlxscore=0 mlxlogscore=649 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312160000 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785396358735487301 X-GMAIL-MSGID: 1785396358735487301 Gunyah doorbells allow a virtual machine to signal another using interrupts. Add the hypercalls needed to assert the interrupt. Reviewed-by: Alex Elder Signed-off-by: Elliot Berman --- arch/arm64/gunyah/gunyah_hypercall.c | 26 ++++++++++++++++++++++++++ include/linux/gunyah.h | 5 +++++ 2 files changed, 31 insertions(+) diff --git a/arch/arm64/gunyah/gunyah_hypercall.c b/arch/arm64/gunyah/gunyah_hypercall.c index 47926df33876..ec3c92025b0b 100644 --- a/arch/arm64/gunyah/gunyah_hypercall.c +++ b/arch/arm64/gunyah/gunyah_hypercall.c @@ -37,6 +37,8 @@ EXPORT_SYMBOL_GPL(arch_is_gunyah_guest); /* clang-format off */ #define GUNYAH_HYPERCALL_HYP_IDENTIFY GUNYAH_HYPERCALL(0x8000) +#define GUNYAH_HYPERCALL_BELL_SEND GUNYAH_HYPERCALL(0x8012) +#define GUNYAH_HYPERCALL_BELL_SET_MASK GUNYAH_HYPERCALL(0x8015) #define GUNYAH_HYPERCALL_MSGQ_SEND GUNYAH_HYPERCALL(0x801B) #define GUNYAH_HYPERCALL_MSGQ_RECV GUNYAH_HYPERCALL(0x801C) #define GUNYAH_HYPERCALL_ADDRSPACE_MAP GUNYAH_HYPERCALL(0x802B) @@ -64,6 +66,30 @@ void gunyah_hypercall_hyp_identify( } EXPORT_SYMBOL_GPL(gunyah_hypercall_hyp_identify); +enum gunyah_error gunyah_hypercall_bell_send(u64 capid, u64 new_flags, u64 *old_flags) +{ + struct arm_smccc_res res; + + arm_smccc_1_1_hvc(GUNYAH_HYPERCALL_BELL_SEND, capid, new_flags, 0, &res); + + if (res.a0 == GUNYAH_ERROR_OK && old_flags) + *old_flags = res.a1; + + return res.a0; +} +EXPORT_SYMBOL_GPL(gunyah_hypercall_bell_send); + +enum gunyah_error gunyah_hypercall_bell_set_mask(u64 capid, u64 enable_mask, u64 ack_mask) +{ + struct arm_smccc_res res; + + arm_smccc_1_1_hvc(GUNYAH_HYPERCALL_BELL_SET_MASK, capid, enable_mask, ack_mask, 0, &res); + + return res.a0; +} +EXPORT_SYMBOL_GPL(gunyah_hypercall_bell_set_mask); + + enum gunyah_error gunyah_hypercall_msgq_send(u64 capid, size_t size, void *buff, u64 tx_flags, bool *ready) { diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index ed461acb9e1b..dc37c270290b 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -144,6 +144,11 @@ gunyah_api_version(const struct gunyah_hypercall_hyp_identify_resp *gunyah_api) void gunyah_hypercall_hyp_identify( struct gunyah_hypercall_hyp_identify_resp *hyp_identity); +enum gunyah_error gunyah_hypercall_bell_send(u64 capid, u64 new_flags, + u64 *old_flags); +enum gunyah_error gunyah_hypercall_bell_set_mask(u64 capid, u64 enable_mask, + u64 ack_mask); + #define GUNYAH_HYPERCALL_MSGQ_TX_FLAGS_PUSH BIT(0) enum gunyah_error gunyah_hypercall_msgq_send(u64 capid, size_t size, void *buff, From patchwork Sat Dec 16 00:21:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 179706 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:3b04:b0:fb:cd0c:d3e with SMTP id c4csp9668552dys; Fri, 15 Dec 2023 16:30:59 -0800 (PST) X-Google-Smtp-Source: AGHT+IHeJxK38EeIKrSjy9DfeLGnHhNYm+7NEnrWaypG9F8zl2wpN674jqztH20SABFTVfcUBVti X-Received: by 2002:a17:907:9854:b0:a19:a19b:c749 with SMTP id jj20-20020a170907985400b00a19a19bc749mr6228437ejc.153.1702686658988; Fri, 15 Dec 2023 16:30:58 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702686658; cv=none; d=google.com; s=arc-20160816; b=ekN485qR7wVwhMlJThjMDq6606IfxndyroKZ6++Pie4cxjEjARJ5t6ATfvzenZWrHi TJhoy4aqkPuM4IC50sQ8PzK4UaxhiAVAg5tYClsRrCJNqXQVeOldcnX1ot193+vtd/jV oCMTRzosqzFuu3ICcGcR3dIXcIuoPI2C2Sk/1sJddIDZSHv4u8K0HadLgEdrWCGlvBil gJps9OvNe4Nw+zxFeYMvbh3KdGsSK5tPzCfBWbp+H1MozUxf0NNmelJCRV8bliPvR9Xn VPYzrQGPP/c4PDO7k8QvZo49Y0e67sN1ssCNF3oM774TrYQcfepy5GPvwX1bK+HP2CgG mGXw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:list-unsubscribe:list-subscribe:list-id:precedence :subject:date:from:dkim-signature; bh=rY5adxQfJ6KvF/u/fHnmbUJMprL8AIQb6d7Wl4O19x0=; fh=Pv/sMsQia5wmqAOYHIQUkkUFpDKhZjuODj71Ojb2ybw=; b=luUWH8z2WZ45ChWgl021uwcqD5oEQY9E9KY/ZVcCZfBRANIm6u3Fy0DC8s9QcO4nKK +GkeoOBPxTBa0mBC+gbSg0mh2hSwIjYiTTU8tW8HSmmyF6HOmeTfhvRyP6xgcaLQKMfg o3tdmLLLkBu+tSyw/T2NuPVgV24P67DVKPdvTDx7f5ebYPPe1fyah//bM87fRXhCsVsG N4hfV3xKwiBaF6naRELwJN3aWdrrHZa+b8L5PPosJfLlDPqkBLTpTsz+nEuAZy4edSU7 70+617aRCGUlgS0S9rXIuUdbEuRzWg3aoCqhzsC4+JJ9HNI8alDn/I9T5hr3jaocfcpR Tb6A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=LP1UYNMS; spf=pass (google.com: domain of linux-kernel+bounces-1896-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1896-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id hb25-20020a170906b89900b00a1d5de6affdsi7965280ejb.870.2023.12.15.16.30.58 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Dec 2023 16:30:58 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-1896-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=LP1UYNMS; spf=pass (google.com: domain of linux-kernel+bounces-1896-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1896-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 70B211F238C5 for ; Sat, 16 Dec 2023 00:30:58 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id EB66D405DF; Sat, 16 Dec 2023 00:21:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="LP1UYNMS" X-Original-To: linux-kernel@vger.kernel.org Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 74D6435889; Sat, 16 Dec 2023 00:21:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=quicinc.com Received: from pps.filterd (m0279865.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BG0JIFD032069; Sat, 16 Dec 2023 00:21:19 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:date:subject:mime-version:content-type :content-transfer-encoding:message-id:references:in-reply-to:to :cc; s=qcppdkim1; bh=rY5adxQfJ6KvF/u/fHnmbUJMprL8AIQb6d7Wl4O19x0 =; b=LP1UYNMS8LlhnRHZaUcjXaHzoDy/oh9N5RrENiikcg/uPteN+pxZwMl9pnJ HcKt6YqFvSxOn5NJQAuHRZTfjn7jiHgQiLYacBGmxGlf92fwR1h9+nV2Q6uKFGjw CDLdVObTGketU4EYfrCryuT9EvxQHBTRk3jdjVx3Utm9e7OCDNhP4/RyLbOxoesT mkXiyR3xWosDPtcuFx/kWDsQjGyzY4ltb786k1OQxV4284Rcob+6ysqmczTqgqpm ZLa+zKwn7MGkEIM7TFD+Thhu4qVICBLXTHdgxcNQeZW6SGUxpiVmIVIR4H/ydchf bIDgYOdbC04SvCT8wxRA2ifYCFA== Received: from nasanppmta05.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v0m46hwe5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:18 +0000 (GMT) Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA05.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 3BG0LIY8006403 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:18 GMT Received: from [169.254.0.1] (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.40; Fri, 15 Dec 2023 16:21:17 -0800 From: Elliot Berman Date: Fri, 15 Dec 2023 16:21:09 -0800 Subject: [PATCH RFC v15 28/30] virt: gunyah: Add irqfd interface Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <20231215-gunyah-v15-28-192a5d872a30@quicinc.com> References: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> In-Reply-To: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> To: Alex Elder , Srinivas Kandagatla , Murali Nalajal , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Philip Derrin , Prakruthi Deepak Heragu , Jonathan Corbet , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Catalin Marinas , Will Deacon , Konrad Dybcio , Bjorn Andersson , Dmitry Baryshkov , "Fuad Tabba" , Sean Christopherson CC: , , , , , Elliot Berman X-Mailer: b4 0.13-dev X-ClientProxiedBy: nalasex01a.na.qualcomm.com (10.47.209.196) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: wjQpOSPX7aG6e3FizqX1--zqcHINpCnM X-Proofpoint-ORIG-GUID: wjQpOSPX7aG6e3FizqX1--zqcHINpCnM X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 suspectscore=0 adultscore=0 phishscore=0 mlxscore=0 clxscore=1015 mlxlogscore=999 impostorscore=0 malwarescore=0 bulkscore=0 spamscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312160001 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785396365924237119 X-GMAIL-MSGID: 1785396365924237119 Enable support for creating irqfds which can raise an interrupt on a Gunyah virtual machine. irqfds are exposed to userspace as a Gunyah VM function with the name "irqfd". If the VM devicetree is not configured to create a doorbell with the corresponding label, userspace will still be able to assert the eventfd but no interrupt will be raised on the guest. Acked-by: Alex Elder Co-developed-by: Prakruthi Deepak Heragu Signed-off-by: Prakruthi Deepak Heragu Signed-off-by: Elliot Berman --- drivers/virt/gunyah/Kconfig | 9 ++ drivers/virt/gunyah/Makefile | 1 + drivers/virt/gunyah/gunyah_irqfd.c | 191 +++++++++++++++++++++++++++++++++++++ include/uapi/linux/gunyah.h | 35 +++++++ 4 files changed, 236 insertions(+) diff --git a/drivers/virt/gunyah/Kconfig b/drivers/virt/gunyah/Kconfig index fe2823dc48ba..1685b75fb77a 100644 --- a/drivers/virt/gunyah/Kconfig +++ b/drivers/virt/gunyah/Kconfig @@ -27,3 +27,12 @@ config GUNYAH_QCOM_PLATFORM extra platform-specific support. Say Y/M here to use Gunyah on Qualcomm platforms. + +config GUNYAH_IRQFD + tristate "Gunyah irqfd interface" + depends on GUNYAH + help + Enable kernel support for creating irqfds which can raise an interrupt + on Gunyah virtual machine. + + Say Y/M here if unsure and you want to support Gunyah VMMs. diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index c4505fce177d..b41b02792921 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -5,3 +5,4 @@ gunyah_rsc_mgr-y += rsc_mgr.o rsc_mgr_rpc.o vm_mgr.o vm_mgr_mem.o guest_memfd.o obj-$(CONFIG_GUNYAH) += gunyah.o gunyah_rsc_mgr.o gunyah_vcpu.o obj-$(CONFIG_GUNYAH_PLATFORM_HOOKS) += gunyah_platform_hooks.o obj-$(CONFIG_GUNYAH_QCOM_PLATFORM) += gunyah_qcom.o +obj-$(CONFIG_GUNYAH_IRQFD) += gunyah_irqfd.o diff --git a/drivers/virt/gunyah/gunyah_irqfd.c b/drivers/virt/gunyah/gunyah_irqfd.c new file mode 100644 index 000000000000..0e377ead3213 --- /dev/null +++ b/drivers/virt/gunyah/gunyah_irqfd.c @@ -0,0 +1,191 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +struct gunyah_irqfd { + struct gunyah_resource *ghrsc; + struct gunyah_vm_resource_ticket ticket; + struct gunyah_vm_function_instance *f; + + bool level; + + struct eventfd_ctx *ctx; + wait_queue_entry_t wait; + poll_table pt; +}; + +static int irqfd_wakeup(wait_queue_entry_t *wait, unsigned int mode, int sync, + void *key) +{ + struct gunyah_irqfd *irqfd = + container_of(wait, struct gunyah_irqfd, wait); + __poll_t flags = key_to_poll(key); + int ret = 0; + + if (flags & EPOLLIN) { + if (irqfd->ghrsc) { + ret = gunyah_hypercall_bell_send(irqfd->ghrsc->capid, 1, + NULL); + if (ret) + pr_err_ratelimited( + "Failed to inject interrupt %d: %d\n", + irqfd->ticket.label, ret); + } else + pr_err_ratelimited( + "Premature injection of interrupt\n"); + } + + return 0; +} + +static void irqfd_ptable_queue_proc(struct file *file, wait_queue_head_t *wqh, + poll_table *pt) +{ + struct gunyah_irqfd *irq_ctx = + container_of(pt, struct gunyah_irqfd, pt); + + add_wait_queue(wqh, &irq_ctx->wait); +} + +static bool gunyah_irqfd_populate(struct gunyah_vm_resource_ticket *ticket, + struct gunyah_resource *ghrsc) +{ + struct gunyah_irqfd *irqfd = + container_of(ticket, struct gunyah_irqfd, ticket); + int ret; + + if (irqfd->ghrsc) { + pr_warn("irqfd%d already got a Gunyah resource. Check if multiple resources with same label were configured.\n", + irqfd->ticket.label); + return false; + } + + irqfd->ghrsc = ghrsc; + if (irqfd->level) { + /* Configure the bell to trigger when bit 0 is asserted (see + * irq_wakeup) and for bell to automatically clear bit 0 once + * received by the VM (ack_mask). need to make sure bit 0 is cleared right away, + * otherwise the line will never be deasserted. Emulating edge + * trigger interrupt does not need to set either mask + * because irq is listed only once per gunyah_hypercall_bell_send + */ + ret = gunyah_hypercall_bell_set_mask(irqfd->ghrsc->capid, 1, 1); + if (ret) + pr_warn("irq %d couldn't be set as level triggered. Might cause IRQ storm if asserted\n", + irqfd->ticket.label); + } + + return true; +} + +static void gunyah_irqfd_unpopulate(struct gunyah_vm_resource_ticket *ticket, + struct gunyah_resource *ghrsc) +{ + struct gunyah_irqfd *irqfd = + container_of(ticket, struct gunyah_irqfd, ticket); + u64 cnt; + + eventfd_ctx_remove_wait_queue(irqfd->ctx, &irqfd->wait, &cnt); +} + +static long gunyah_irqfd_bind(struct gunyah_vm_function_instance *f) +{ + struct gunyah_fn_irqfd_arg *args = f->argp; + struct gunyah_irqfd *irqfd; + __poll_t events; + struct fd fd; + long r; + + if (f->arg_size != sizeof(*args)) + return -EINVAL; + + /* All other flag bits are reserved for future use */ + if (args->flags & ~GUNYAH_IRQFD_FLAGS_LEVEL) + return -EINVAL; + + irqfd = kzalloc(sizeof(*irqfd), GFP_KERNEL); + if (!irqfd) + return -ENOMEM; + + irqfd->f = f; + f->data = irqfd; + + fd = fdget(args->fd); + if (!fd.file) { + kfree(irqfd); + return -EBADF; + } + + irqfd->ctx = eventfd_ctx_fileget(fd.file); + if (IS_ERR(irqfd->ctx)) { + r = PTR_ERR(irqfd->ctx); + goto err_fdput; + } + + if (args->flags & GUNYAH_IRQFD_FLAGS_LEVEL) + irqfd->level = true; + + init_waitqueue_func_entry(&irqfd->wait, irqfd_wakeup); + init_poll_funcptr(&irqfd->pt, irqfd_ptable_queue_proc); + + irqfd->ticket.resource_type = GUNYAH_RESOURCE_TYPE_BELL_TX; + irqfd->ticket.label = args->label; + irqfd->ticket.owner = THIS_MODULE; + irqfd->ticket.populate = gunyah_irqfd_populate; + irqfd->ticket.unpopulate = gunyah_irqfd_unpopulate; + + r = gunyah_vm_add_resource_ticket(f->ghvm, &irqfd->ticket); + if (r) + goto err_ctx; + + events = vfs_poll(fd.file, &irqfd->pt); + if (events & EPOLLIN) + pr_warn("Premature injection of interrupt\n"); + fdput(fd); + + return 0; +err_ctx: + eventfd_ctx_put(irqfd->ctx); +err_fdput: + fdput(fd); + kfree(irqfd); + return r; +} + +static void gunyah_irqfd_unbind(struct gunyah_vm_function_instance *f) +{ + struct gunyah_irqfd *irqfd = f->data; + + gunyah_vm_remove_resource_ticket(irqfd->f->ghvm, &irqfd->ticket); + eventfd_ctx_put(irqfd->ctx); + kfree(irqfd); +} + +static bool gunyah_irqfd_compare(const struct gunyah_vm_function_instance *f, + const void *arg, size_t size) +{ + const struct gunyah_fn_irqfd_arg *instance = f->argp, *other = arg; + + if (sizeof(*other) != size) + return false; + + return instance->label == other->label; +} + +DECLARE_GUNYAH_VM_FUNCTION_INIT(irqfd, GUNYAH_FN_IRQFD, 2, gunyah_irqfd_bind, + gunyah_irqfd_unbind, gunyah_irqfd_compare); +MODULE_DESCRIPTION("Gunyah irqfd VM Function"); +MODULE_LICENSE("GPL"); diff --git a/include/uapi/linux/gunyah.h b/include/uapi/linux/gunyah.h index 574116f54472..cb7b0bb9bef3 100644 --- a/include/uapi/linux/gunyah.h +++ b/include/uapi/linux/gunyah.h @@ -63,9 +63,12 @@ struct gunyah_vm_dtb_config { * @GUNYAH_FN_VCPU: create a vCPU instance to control a vCPU * &struct gunyah_fn_desc.arg is a pointer to &struct gunyah_fn_vcpu_arg * Return: file descriptor to manipulate the vcpu. + * @GUNYAH_FN_IRQFD: register eventfd to assert a Gunyah doorbell + * &struct gunyah_fn_desc.arg is a pointer to &struct gunyah_fn_irqfd_arg */ enum gunyah_fn_type { GUNYAH_FN_VCPU = 1, + GUNYAH_FN_IRQFD, }; #define GUNYAH_FN_MAX_ARG_SIZE 256 @@ -85,6 +88,38 @@ struct gunyah_fn_vcpu_arg { __u32 id; }; +/** + * enum gunyah_irqfd_flags - flags for use in gunyah_fn_irqfd_arg + * @GUNYAH_IRQFD_FLAGS_LEVEL: make the interrupt operate like a level triggered + * interrupt on guest side. Triggering IRQFD before + * guest handles the interrupt causes interrupt to + * stay asserted. + */ +enum gunyah_irqfd_flags { + GUNYAH_IRQFD_FLAGS_LEVEL = 1UL << 0, +}; + +/** + * struct gunyah_fn_irqfd_arg - Arguments to create an irqfd function. + * + * Create this function with &GUNYAH_VM_ADD_FUNCTION using type &GUNYAH_FN_IRQFD. + * + * Allows setting an eventfd to directly trigger a guest interrupt. + * irqfd.fd specifies the file descriptor to use as the eventfd. + * irqfd.label corresponds to the doorbell label used in the guest VM's devicetree. + * + * @fd: an eventfd which when written to will raise a doorbell + * @label: Label of the doorbell created on the guest VM + * @flags: see &enum gunyah_irqfd_flags + * @padding: padding bytes + */ +struct gunyah_fn_irqfd_arg { + __u32 fd; + __u32 label; + __u32 flags; + __u32 padding; +}; + /** * struct gunyah_fn_desc - Arguments to create a VM function * @type: Type of the function. See &enum gunyah_fn_type. From patchwork Sat Dec 16 00:21:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 179708 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:3b04:b0:fb:cd0c:d3e with SMTP id c4csp9668799dys; Fri, 15 Dec 2023 16:31:28 -0800 (PST) X-Google-Smtp-Source: AGHT+IHGiM6Yg5Xemi4yOYVopliVg9ZzL71hbz/B90iZb6UQV7X92aDk8vSpoFNqU12+CXmRezHx X-Received: by 2002:ac8:7d92:0:b0:425:4043:5f21 with SMTP id c18-20020ac87d92000000b0042540435f21mr17255069qtd.95.1702686688166; Fri, 15 Dec 2023 16:31:28 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702686688; cv=none; d=google.com; s=arc-20160816; b=pe9SJj1qU34tSJPpXDllmf9wIwSlB0OZJ9Lz02DwgCdEGtMzRnZN9uWkA7DvzmllJV /006uCBFKECCCGqkOWTngeKxUUt5lRf6tODxREor8dEgSImfOfuOSkTx5i4zjpiWhx5b LuOp+kTu8fR1FaKbuaJg4sDs8Icn2PLzc+Y2AuDcHchUpqQpmvPO7koe4EVRu/WvGl84 dX5Zkp/iKHTkgUXbjsxqs8nHAfDH4ZBl8B3EAF/hlH1DX9fMyc2j4+JB+wYp0oIpm9zI +YPoAvsn7yC3P9iXLtTvZCHq2LKksIwGqJacGpYcJ6ph3IC1mxZvMkrOcqb2ZfFoOYni m55w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:list-unsubscribe:list-subscribe:list-id:precedence :subject:date:from:dkim-signature; bh=F/OGCmjR0DSPJXRayuxDEVqL+YHknehjvQxRWqYAyas=; fh=Pv/sMsQia5wmqAOYHIQUkkUFpDKhZjuODj71Ojb2ybw=; b=tClDsuUzCA+d8ZyrqD04ax6ajuxCfpB3/0LwzNpjYrISF/CZgjezzsaYMtDfFL9U+7 Yt5VHXeZlWyQnmAVzViXW4ZG9OK2GINyHnb+0KZubPPSUKI4iT3wOhMce2q/YKVde41h SbMENN+QZhgpiyReEOq4BlLyi9E21Nbh2I85ICEVz/G3xHgqXxGq08KmcQ4PQ3MNVbfz KobBMwioEyS4WOii9OiFmrgwz2OvMc18Fa8gi3Om/Q63NDGOGPBz8U4kSR8hZWeb+XQ5 IgfjCoFZ3dXQlbRgUGbfAddcqs+tF3TYnmw/tfiIMvl2wu0NiglVHL5C4ZVDNzPgotYf bY1w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b="H/70+1YS"; spf=pass (google.com: domain of linux-kernel+bounces-1898-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1898-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id ay39-20020a05622a22a700b004237544461csi18833861qtb.355.2023.12.15.16.31.28 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Dec 2023 16:31:28 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-1898-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b="H/70+1YS"; spf=pass (google.com: domain of linux-kernel+bounces-1898-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1898-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id C398D1C24826 for ; Sat, 16 Dec 2023 00:31:27 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 27BA640BE1; Sat, 16 Dec 2023 00:21:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="H/70+1YS" X-Original-To: linux-kernel@vger.kernel.org Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E52E940BEA; Sat, 16 Dec 2023 00:21:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=quicinc.com Received: from pps.filterd (m0279862.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BFNkUHK004542; Sat, 16 Dec 2023 00:21:19 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:date:subject:mime-version:content-type :content-transfer-encoding:message-id:references:in-reply-to:to :cc; s=qcppdkim1; bh=F/OGCmjR0DSPJXRayuxDEVqL+YHknehjvQxRWqYAyas =; b=H/70+1YSPa6QdRIR1I1NRjSPMAQIKlKTLSmJyc7qfOvni2YfanACesCZ+XF k1UonfIK1KVler4MEXhBeSEBSZGqOHZ7MyFNKpdESKfNisUjYSVh6e+f/bG4xnj7 46/1QZgi25SRIcXhyL3LhSneiiTrldrqpS99+CJz76mcccbmijqXpqJiigJPAPSO ozDa4ysGoag7Nzcw0XaBt3ETwMM9F0fwKg2N4a5jgYPGlF7hoSdYC95zJCVmf5V8 S3lwhccVcHVytunhMK+15+liwlzPHkzFDvIZzljLSZt/hrYPf+Ci/qODGGavEdG/ fcN2vgcA1Mp5eFJwevemKi0MV9w== Received: from nasanppmta02.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v0w198hpr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:19 +0000 (GMT) Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA02.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 3BG0LIxK015207 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:18 GMT Received: from [169.254.0.1] (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.40; Fri, 15 Dec 2023 16:21:17 -0800 From: Elliot Berman Date: Fri, 15 Dec 2023 16:21:10 -0800 Subject: [PATCH RFC v15 29/30] virt: gunyah: Add ioeventfd Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <20231215-gunyah-v15-29-192a5d872a30@quicinc.com> References: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> In-Reply-To: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> To: Alex Elder , Srinivas Kandagatla , Murali Nalajal , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Philip Derrin , Prakruthi Deepak Heragu , Jonathan Corbet , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Catalin Marinas , Will Deacon , Konrad Dybcio , Bjorn Andersson , Dmitry Baryshkov , "Fuad Tabba" , Sean Christopherson CC: , , , , , Elliot Berman X-Mailer: b4 0.13-dev X-ClientProxiedBy: nalasex01a.na.qualcomm.com (10.47.209.196) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: dTUbYXM3z2MVkmsCA8U69h7Yqye2EGwK X-Proofpoint-ORIG-GUID: dTUbYXM3z2MVkmsCA8U69h7Yqye2EGwK X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 mlxlogscore=999 suspectscore=0 mlxscore=0 impostorscore=0 lowpriorityscore=0 spamscore=0 adultscore=0 phishscore=0 clxscore=1015 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312160001 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785396396625780565 X-GMAIL-MSGID: 1785396396625780565 Allow userspace to attach an ioeventfd to an mmio address within the guest. Userspace provides a description of the type of write to "subscribe" to and eventfd to trigger when that type of write is performed by the guest. This mechanism allows userspace to respond asynchronously to a guest manipulating a virtualized device and is similar to KVM's ioeventfd. Reviewed-by: Alex Elder Co-developed-by: Prakruthi Deepak Heragu Signed-off-by: Prakruthi Deepak Heragu Signed-off-by: Elliot Berman --- drivers/virt/gunyah/Kconfig | 9 +++ drivers/virt/gunyah/Makefile | 1 + drivers/virt/gunyah/gunyah_ioeventfd.c | 132 +++++++++++++++++++++++++++++++++ include/uapi/linux/gunyah.h | 37 +++++++++ 4 files changed, 179 insertions(+) diff --git a/drivers/virt/gunyah/Kconfig b/drivers/virt/gunyah/Kconfig index 1685b75fb77a..855d41a88b16 100644 --- a/drivers/virt/gunyah/Kconfig +++ b/drivers/virt/gunyah/Kconfig @@ -36,3 +36,12 @@ config GUNYAH_IRQFD on Gunyah virtual machine. Say Y/M here if unsure and you want to support Gunyah VMMs. + +config GUNYAH_IOEVENTFD + tristate "Gunyah ioeventfd interface" + depends on GUNYAH + help + Enable kernel support for creating ioeventfds which can alert userspace + when a Gunyah virtual machine accesses a memory address. + + Say Y/M here if unsure and you want to support Gunyah VMMs. diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index b41b02792921..2aec5989402b 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -6,3 +6,4 @@ obj-$(CONFIG_GUNYAH) += gunyah.o gunyah_rsc_mgr.o gunyah_vcpu.o obj-$(CONFIG_GUNYAH_PLATFORM_HOOKS) += gunyah_platform_hooks.o obj-$(CONFIG_GUNYAH_QCOM_PLATFORM) += gunyah_qcom.o obj-$(CONFIG_GUNYAH_IRQFD) += gunyah_irqfd.o +obj-$(CONFIG_GUNYAH_IOEVENTFD) += gunyah_ioeventfd.o diff --git a/drivers/virt/gunyah/gunyah_ioeventfd.c b/drivers/virt/gunyah/gunyah_ioeventfd.c new file mode 100644 index 000000000000..650a156c0102 --- /dev/null +++ b/drivers/virt/gunyah/gunyah_ioeventfd.c @@ -0,0 +1,132 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +struct gunyah_ioeventfd { + struct gunyah_vm_function_instance *f; + struct gunyah_vm_io_handler io_handler; + + struct eventfd_ctx *ctx; +}; + +static int gunyah_write_ioeventfd(struct gunyah_vm_io_handler *io_dev, u64 addr, + u32 len, u64 data) +{ + struct gunyah_ioeventfd *iofd = + container_of(io_dev, struct gunyah_ioeventfd, io_handler); + + eventfd_signal(iofd->ctx); + return 0; +} + +static struct gunyah_vm_io_handler_ops io_ops = { + .write = gunyah_write_ioeventfd, +}; + +static long gunyah_ioeventfd_bind(struct gunyah_vm_function_instance *f) +{ + const struct gunyah_fn_ioeventfd_arg *args = f->argp; + struct gunyah_ioeventfd *iofd; + struct eventfd_ctx *ctx; + int ret; + + if (f->arg_size != sizeof(*args)) + return -EINVAL; + + /* All other flag bits are reserved for future use */ + if (args->flags & ~GUNYAH_IOEVENTFD_FLAGS_DATAMATCH) + return -EINVAL; + + /* must be natural-word sized, or 0 to ignore length */ + switch (args->len) { + case 0: + case 1: + case 2: + case 4: + case 8: + break; + default: + return -EINVAL; + } + + /* check for range overflow */ + if (overflows_type(args->addr + args->len, u64)) + return -EINVAL; + + /* ioeventfd with no length can't be combined with DATAMATCH */ + if (!args->len && (args->flags & GUNYAH_IOEVENTFD_FLAGS_DATAMATCH)) + return -EINVAL; + + ctx = eventfd_ctx_fdget(args->fd); + if (IS_ERR(ctx)) + return PTR_ERR(ctx); + + iofd = kzalloc(sizeof(*iofd), GFP_KERNEL); + if (!iofd) { + ret = -ENOMEM; + goto err_eventfd; + } + + f->data = iofd; + iofd->f = f; + + iofd->ctx = ctx; + + if (args->flags & GUNYAH_IOEVENTFD_FLAGS_DATAMATCH) { + iofd->io_handler.datamatch = true; + iofd->io_handler.len = args->len; + iofd->io_handler.data = args->datamatch; + } + iofd->io_handler.addr = args->addr; + iofd->io_handler.ops = &io_ops; + + ret = gunyah_vm_add_io_handler(f->ghvm, &iofd->io_handler); + if (ret) + goto err_io_dev_add; + + return 0; + +err_io_dev_add: + kfree(iofd); +err_eventfd: + eventfd_ctx_put(ctx); + return ret; +} + +static void gunyah_ioevent_unbind(struct gunyah_vm_function_instance *f) +{ + struct gunyah_ioeventfd *iofd = f->data; + + eventfd_ctx_put(iofd->ctx); + gunyah_vm_remove_io_handler(iofd->f->ghvm, &iofd->io_handler); + kfree(iofd); +} + +static bool gunyah_ioevent_compare(const struct gunyah_vm_function_instance *f, + const void *arg, size_t size) +{ + const struct gunyah_fn_ioeventfd_arg *instance = f->argp, *other = arg; + + if (sizeof(*other) != size) + return false; + + return instance->addr == other->addr; +} + +DECLARE_GUNYAH_VM_FUNCTION_INIT(ioeventfd, GUNYAH_FN_IOEVENTFD, 3, + gunyah_ioeventfd_bind, gunyah_ioevent_unbind, + gunyah_ioevent_compare); +MODULE_DESCRIPTION("Gunyah ioeventfd VM Function"); +MODULE_LICENSE("GPL"); diff --git a/include/uapi/linux/gunyah.h b/include/uapi/linux/gunyah.h index cb7b0bb9bef3..fd461e2fe8b5 100644 --- a/include/uapi/linux/gunyah.h +++ b/include/uapi/linux/gunyah.h @@ -65,10 +65,13 @@ struct gunyah_vm_dtb_config { * Return: file descriptor to manipulate the vcpu. * @GUNYAH_FN_IRQFD: register eventfd to assert a Gunyah doorbell * &struct gunyah_fn_desc.arg is a pointer to &struct gunyah_fn_irqfd_arg + * @GUNYAH_FN_IOEVENTFD: register ioeventfd to trigger when VM faults on parameter + * &struct gunyah_fn_desc.arg is a pointer to &struct gunyah_fn_ioeventfd_arg */ enum gunyah_fn_type { GUNYAH_FN_VCPU = 1, GUNYAH_FN_IRQFD, + GUNYAH_FN_IOEVENTFD, }; #define GUNYAH_FN_MAX_ARG_SIZE 256 @@ -120,6 +123,40 @@ struct gunyah_fn_irqfd_arg { __u32 padding; }; +/** + * enum gunyah_ioeventfd_flags - flags for use in gunyah_fn_ioeventfd_arg + * @GUNYAH_IOEVENTFD_FLAGS_DATAMATCH: the event will be signaled only if the + * written value to the registered address is + * equal to &struct gunyah_fn_ioeventfd_arg.datamatch + */ +enum gunyah_ioeventfd_flags { + GUNYAH_IOEVENTFD_FLAGS_DATAMATCH = 1UL << 0, +}; + +/** + * struct gunyah_fn_ioeventfd_arg - Arguments to create an ioeventfd function + * @datamatch: data used when GUNYAH_IOEVENTFD_DATAMATCH is set + * @addr: Address in guest memory + * @len: Length of access + * @fd: When ioeventfd is matched, this eventfd is written + * @flags: See &enum gunyah_ioeventfd_flags + * @padding: padding bytes + * + * Create this function with &GUNYAH_VM_ADD_FUNCTION using type &GUNYAH_FN_IOEVENTFD. + * + * Attaches an ioeventfd to a legal mmio address within the guest. A guest write + * in the registered address will signal the provided event instead of triggering + * an exit on the GUNYAH_VCPU_RUN ioctl. + */ +struct gunyah_fn_ioeventfd_arg { + __u64 datamatch; + __u64 addr; /* legal mmio address */ + __u32 len; /* 1, 2, 4, or 8 bytes; or 0 to ignore length */ + __s32 fd; + __u32 flags; + __u32 padding; +}; + /** * struct gunyah_fn_desc - Arguments to create a VM function * @type: Type of the function. See &enum gunyah_fn_type. From patchwork Sat Dec 16 00:21:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 179707 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:3b04:b0:fb:cd0c:d3e with SMTP id c4csp9668620dys; Fri, 15 Dec 2023 16:31:07 -0800 (PST) X-Google-Smtp-Source: AGHT+IH9F8C5FxGU0XgplFI6Y7fOdzkCCyLbfJcQs9jw+ZfMRYqvdXje3PCtLTVtpf5wvP2/f72U X-Received: by 2002:a17:906:220c:b0:a1f:a302:1ae3 with SMTP id s12-20020a170906220c00b00a1fa3021ae3mr5619048ejs.12.1702686667742; Fri, 15 Dec 2023 16:31:07 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702686667; cv=none; d=google.com; s=arc-20160816; b=gAMSw6Vl8aO2cdcIzSxE+OcvQobamifB1YN3zQaYyoWSamU/8S13O7vhiYeaIMl9q3 ahsY7pPMr6WsPDtPAkfrW02CyMeNsCny7VlgGjn1r7wj4fjd2jmlPcI1d7VrFBFzj0Ez Fh+D6lb9iMODrX3BcrB4hcXExADOA+cBWxzfmzVChJ2VSp3e6p4nQZyaylZPcS7fMA5J bgpMuDJhMmH+y0oKJk5J6F/DfZ0Yaje8Zo/YDaXEp+/QaCsJ2ldnrGSVByxopO0Ijef2 hu4FBjxgKVKoo8r3e4TahkcBW7eAFwJtIn+wHMLuDEX+pa6/Trvb9gVVAUzfA0Wwhq4c jfcw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:list-unsubscribe:list-subscribe:list-id:precedence :subject:date:from:dkim-signature; bh=dyB/ua3LVYneyuF6Yg+6NdNcj7CDy3ND+aki9ubcU9M=; fh=Pv/sMsQia5wmqAOYHIQUkkUFpDKhZjuODj71Ojb2ybw=; b=f0pxsp2driCtEEJINwlnQpZTIVFqdkPZqymY8sq+asZtmXnc4eAA0qPci9eT5su4mf KaXDEtFZ6dFMwzwYivpbztfXzkXJwjwQbQuKu5pAEfvoIKYZFZ0gTxZFYrN/ah9XkOQW OyK9GbPgaPruf/6indYdefnMJyJw72Tb0jEOUyIghFjgEcdX8sN5g/1bXpRB/2njDSb9 fxj1dKhTsRAmKWtvMRthwnM/vI6oHQzCx8UcfsPNhGPsvlscP5d8R5rlIkuIenUdzTK/ xvBFMvIPShoBPYP2BcD5BzntDlhWp8O+1Z4xrGJF2dh/AdqiQTNB1unBAStUbmCc5R9I 7LuA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=JCfeh98Q; spf=pass (google.com: domain of linux-kernel+bounces-1897-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1897-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id r7-20020a170906a20700b00a1d9004fb1esi7559756ejy.203.2023.12.15.16.31.07 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Dec 2023 16:31:07 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-1897-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=JCfeh98Q; spf=pass (google.com: domain of linux-kernel+bounces-1897-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-1897-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 5CA881F2162B for ; Sat, 16 Dec 2023 00:31:07 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 936C640BED; Sat, 16 Dec 2023 00:21:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="JCfeh98Q" X-Original-To: linux-kernel@vger.kernel.org Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5E1FF37D20; Sat, 16 Dec 2023 00:21:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=quicinc.com Received: from pps.filterd (m0279867.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BFNYWWb005842; Sat, 16 Dec 2023 00:21:20 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:date:subject:mime-version:content-type :content-transfer-encoding:message-id:references:in-reply-to:to :cc; s=qcppdkim1; bh=dyB/ua3LVYneyuF6Yg+6NdNcj7CDy3ND+aki9ubcU9M =; b=JCfeh98QyKrD5I5NXZos1zEOUGTG0RwdUWMQmi9xjI2WeZzH2FedCu9SATv +qXxwn+D0pM5erkN3+9nQf/7TXgP8RKAIWJtA6k8mi3tRFdI0ddaaNYspzHmny9u alpzK9FCYqIKxpiyHEW97JdEXoJu4I0kZ+tUS+gFmQxWbdUJHcG6MlYrX5W/aPOQ H+yRspTzK+Qfn5nxXDBjisyuzCrdAlOFcOYgKw8R+0fdgptkTNc/y0JnrV33faUq k5+x1r/M8UdjacymdK3EWiXFn4CVVZGjPY9pyZWuFRXGSWmsOyhInSZ2a+4u6jye JPqwDYp9lD/P0dT3rAQdN2fCItg== Received: from nasanppmta03.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v0k90t5er-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:19 +0000 (GMT) Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA03.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 3BG0LJEp005053 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 Dec 2023 00:21:19 GMT Received: from [169.254.0.1] (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.40; Fri, 15 Dec 2023 16:21:18 -0800 From: Elliot Berman Date: Fri, 15 Dec 2023 16:21:11 -0800 Subject: [PATCH RFC v15 30/30] MAINTAINERS: Add Gunyah hypervisor drivers section Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <20231215-gunyah-v15-30-192a5d872a30@quicinc.com> References: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> In-Reply-To: <20231215-gunyah-v15-0-192a5d872a30@quicinc.com> To: Alex Elder , Srinivas Kandagatla , Murali Nalajal , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Philip Derrin , Prakruthi Deepak Heragu , Jonathan Corbet , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Catalin Marinas , Will Deacon , Konrad Dybcio , Bjorn Andersson , Dmitry Baryshkov , "Fuad Tabba" , Sean Christopherson CC: , , , , , Elliot Berman X-Mailer: b4 0.13-dev X-ClientProxiedBy: nalasex01a.na.qualcomm.com (10.47.209.196) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: DT_YwXeEHE8MiOYixEiRS891fRU6M1qa X-Proofpoint-GUID: DT_YwXeEHE8MiOYixEiRS891fRU6M1qa X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 lowpriorityscore=0 impostorscore=0 priorityscore=1501 malwarescore=0 clxscore=1015 mlxlogscore=854 adultscore=0 suspectscore=0 bulkscore=0 spamscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312160001 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785396375191007432 X-GMAIL-MSGID: 1785396375191007432 Add myself and Prakruthi as maintainers of Gunyah hypervisor drivers. Reviewed-by: Alex Elder Signed-off-by: Prakruthi Deepak Heragu Signed-off-by: Elliot Berman --- MAINTAINERS | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index b589218605b4..fa2fe28f922f 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -9251,6 +9251,19 @@ L: linux-efi@vger.kernel.org S: Maintained F: block/partitions/efi.* +GUNYAH HYPERVISOR DRIVER +M: Elliot Berman +M: Prakruthi Deepak Heragu +L: linux-arm-msm@vger.kernel.org +S: Supported +F: Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml +F: Documentation/virt/gunyah/ +F: arch/arm64/gunyah/ +F: drivers/mailbox/gunyah-msgq.c +F: drivers/virt/gunyah/ +F: include/linux/gunyah*.h +F: samples/gunyah/ + HABANALABS PCI DRIVER M: Oded Gabbay L: dri-devel@lists.freedesktop.org