From patchwork Sun Mar 12 12:01:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Huang Rui X-Patchwork-Id: 6481 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp691443wrd; Sun, 12 Mar 2023 05:17:10 -0700 (PDT) X-Google-Smtp-Source: AK7set+DaS6y6/te8sVVR1hcgFFFJ8b3r6nrZifYgaisWGVNZGIj76PnRI2CgU8dITrghlFazYVg X-Received: by 2002:a05:6a20:cd5c:b0:d3:c6f6:ac8c with SMTP id hn28-20020a056a20cd5c00b000d3c6f6ac8cmr2161341pzb.36.1678623429855; Sun, 12 Mar 2023 05:17:09 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1678623429; cv=pass; d=google.com; s=arc-20160816; b=iSv4IiWfhCDiScjMeXF5WgpfMKPL2cjCFKNfMsch4n79ei+fTKi1McVa/CjmL4KPE5 zgFToyZTnBoJiNWLnhpi+Lr8tuxal+iBSGdetNmDT2ZAIMVSSxqbB+NkQSkienSu/9gV gL1TsaSQI9OQzGjlGMLBr4ZkMH8D8cu6zHJ0vibGJPqUdr1MdRxDKhZpYCsFzont1z91 KrIKJycFUoBTEyTULuYBkjBkNjJzArXE+eQcO1kGLM8hGmRRucftd18leNkJZ8xVbt5r ybfR06o0wnE9o8nofNeeNsCFPB8tUzmGY4sT8+DxG+p1EgAP+pL0xzKFgVatiWIzTzVG WmtA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=VjmyypcRxDR0P5LaAKNpKqX/6LGTTRc03kykh31DjR8=; b=F2BkPrKKFBryG3CYsWab1Qj30ei4NDbEdg+pmlqaMUA5/6dmvkG/MYwQPxonU8tld6 TsGlR17LHBTtdXI4dpHyulHUq0E6t7ubP+YsyTL+uf4UBwpm1F4kHkzTBd17oddKIkvB cO5uysIBXyBDHILdkl1kxI9QFS9obaRug9Ucm9IZ1CdcmmjEr0L7aoNjZM6AWgAA++b7 aA3UhfM/7bCSlIIYCAzqGV6Nouc8/HBdFhpeU4mgFslGprqrdiWcqtnkWdkhTVb3f6jV SB8EtGNDL4wjjeEFtEe8q3daCOrWwvJNz6SEaQgKZ3Mt23tbvMCULUscb9v13H2sOL5y qjew== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b=o2RJEjz+; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id q10-20020a63f94a000000b005072bddb74csi4039608pgk.846.2023.03.12.05.16.54; Sun, 12 Mar 2023 05:17:09 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b=o2RJEjz+; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230048AbjCLMC2 (ORCPT + 99 others); Sun, 12 Mar 2023 08:02:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60132 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229514AbjCLMC0 (ORCPT ); Sun, 12 Mar 2023 08:02:26 -0400 Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on2070.outbound.protection.outlook.com [40.107.96.70]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ED49F47407 for ; Sun, 12 Mar 2023 05:02:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WqJo/vhAdUYUhyS0zEYT68gRnDM0UMfAF6rnSOtvl3QdN+l7Mj5NZvSV5h67eDYADq2wFpPf+ETurHX2tV/eamOuUiiOsAwWsmrY75AzhGUmw5gjRLfsgcsX8xKaStfoLAiwPMOsB+rOoOhlUGsd1kj4Qm8Sw2FX7P3oGxRIaUBV6QH28sg2n8KuyXO7lggWXVikbArG3yt+0nS0coOHK6FfxxuD9KZMZOUuPev5uCGvP5g+2BRqOY6YN62pQ/06ah0s0dPb2Sw8EZrXjYNn831yLhI6EiCX0XHk81M0nVJgr0FKRlSBt3em89xrRaxdhDsoWZz499N7NFRPCa8pKA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=VjmyypcRxDR0P5LaAKNpKqX/6LGTTRc03kykh31DjR8=; b=kQ69Lw+UfSV0ES3Sj5kurwWpYwh2R/GdequeXPEGnPY1CfRk4ld6yo3mMZIoRmkNu6jq2x9Rw5hJPm45oFYKOlmweiwjjmI2IuhmK6qy7NtVGsX9QCW8av3Nn2DtmkiIgocwIu33lVNEqPyiBkkQqz4gyqxaq5pDw0SdkyP+bT4O0T2rQFzO1T/6ZRAru0DVuGXB23FsskYlTQnXCh1QlbeuaQUcqnJKYfJFt1mNvfaCRqXqciwgIqGazcpJFPPsr1uymjpcQ6xDP3tbBxweSWWe/TAXzFf0nX/P8u14tDElSO6M0dby70wfJNP5w5AikaJWSAm4ptLce1l5Sq8few== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=suse.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=VjmyypcRxDR0P5LaAKNpKqX/6LGTTRc03kykh31DjR8=; b=o2RJEjz+f6FgQ2MrIdC+S0inOZspupXPwGkKg06RalTHE+i5Svhm75/EqFIMObSJu9U86ffyGHCgVQj9IKZ2gV/0MD4rU57kzlnOjxR8QuzaaQRHJ7tfFNcQ6OcBSugRPG65+r9+pKil+JP5AVS8BJ4f5nVO1BwnIzydV1/G1uo= Received: from MW4PR03CA0177.namprd03.prod.outlook.com (2603:10b6:303:8d::32) by MW4PR12MB7239.namprd12.prod.outlook.com (2603:10b6:303:228::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6178.24; Sun, 12 Mar 2023 12:02:20 +0000 Received: from CO1NAM11FT089.eop-nam11.prod.protection.outlook.com (2603:10b6:303:8d:cafe::95) by MW4PR03CA0177.outlook.office365.com (2603:10b6:303:8d::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6178.25 via Frontend Transport; Sun, 12 Mar 2023 12:02:20 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1NAM11FT089.mail.protection.outlook.com (10.13.175.179) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6178.24 via Frontend Transport; Sun, 12 Mar 2023 12:02:20 +0000 Received: from hr-amd.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Sun, 12 Mar 2023 07:02:15 -0500 From: Huang Rui To: Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Boris Ostrovsky , =?utf-8?q?Roger_Pau_Monn?= =?utf-8?q?=C3=A9?= , , , , CC: Alex Deucher , =?utf-8?q?Christian_K=C3=B6nig?= , "Stewart Hildebrand" , Xenia Ragiadakou , Honglei Huang , Julia Zhang , Chen Jiqian , Huang Rui Subject: [RFC PATCH 0/5] Add Xen PVH dom0 support for GPU Date: Sun, 12 Mar 2023 20:01:52 +0800 Message-ID: <20230312120157.452859-1-ray.huang@amd.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1NAM11FT089:EE_|MW4PR12MB7239:EE_ X-MS-Office365-Filtering-Correlation-Id: 23fac0cb-7791-4f2b-179d-08db22f1a221 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: HtyjX7Payq7uC5vApjUpNbNBOrhWO+u2KI0rWVaD5Q5VOm5uvnDKLq1gdMNaqwI2+T3zMQHwKuGSUAp5twS3i3URPHLdk5I8UL5iapUv9TA0wggEpVuPnckp0HJX5G2dNDOHDp2OTzzAhyWRZmSRg6kJIYP5eHL7agVwNuqv9/1kepFnsXyGFKHscQJPjpg3qs2sNyJzr/jBebNVgVtAcjdGamZPr2Dua19/f4Gt/2OozqOo6axhQDRi3je9mVQ9Vo5vwmOWcQ9EX1y5jG1fpAXPMTuqZNIS4O59R2F6VzLY/pXDBW8xLlE8EnGC2/0zvXQCXkxcJAfxaWkh6rZa2pHc9GUEEPL4j0VxXFupxhefxhTsCqvV2RCq6t9Z/TqCm3DvF1wXu6apAYxNVTH4eLkScaZ+KIwODX3dAdtPksYI6z1XWfTRyzRCm7r4YzB/w+vWJ3aZm+wFZd+A2N0Njd7rc4CbuVIBS6kYH8aWOql/niiF9zbZv6C51R8rR15WmqLaH86+BzSPA5yQ/4qTpo7e3U8Glb/n9Nkl7hhfNAKYuEc5l2C8cOva6zumtWwX7TkU9CmxX04tJOGRbtws9oSP/H0BhqcqGkKgT31nk6zBviyY0kIEcnEUWwfKoQapsU7SICOIcWvZ8zOS5/+WzGRCD9i+fkqqltC+d8YLjzP2vshNmRfr7SeOzImcltQFYyRZ01SL/zZdrAa4gjCQOCs4PLtUPOQGAIG4SNEjwctrNn9dt6FOhnz7L33NDFdIaZ9OYkSomRFEGSsek6X0uX/f53QwQDWZrYfJdrQCrk0= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230025)(4636009)(396003)(39860400002)(376002)(346002)(136003)(451199018)(46966006)(40470700004)(36840700001)(356005)(86362001)(82310400005)(36756003)(36860700001)(82740400003)(81166007)(40480700001)(40460700003)(4326008)(70586007)(70206006)(8676002)(8936002)(41300700001)(110136005)(54906003)(478600001)(316002)(5660300002)(7416002)(2906002)(336012)(66574015)(426003)(47076005)(2616005)(83380400001)(7696005)(966005)(186003)(16526019)(6666004)(1076003)(26005)(5930299012)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Mar 2023 12:02:20.3410 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 23fac0cb-7791-4f2b-179d-08db22f1a221 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT089.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB7239 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1760164241681416347?= X-GMAIL-MSGID: =?utf-8?q?1760164241681416347?= Hi all, Currently, we are working to add VirtIO GPU and Passthrough GPU support on Xen. We expected to use HVM on domU and PVH on dom0. The x86 PVH dom0 support needs a few modifications on our APU platform. These functions requires multiple software components support including kernel, xen, qemu, mesa, and virglrenderer. Please see the patch series on Xen and QEMU bleow. Xen: https://lists.xenproject.org/archives/html/xen-devel/2023-03/msg00714.html QEMU: https://lists.nongnu.org/archive/html/qemu-devel/2023-03/msg03972.html Kernel part mainly adds the PVH dom0 support: 1) Enable Xen PVH dom0 for AMDGPU Please check patch 1 to 3, that enable Xen PVH dom0 on amdgpu. Because we would like to use hardware IOMMU instead of swiotlb for buffer copy, PV dom0 only supported swiotlb. There still some workarounds in the kernel need to dig it out like below https://git.kernel.org/pub/scm/linux/kernel/git/rui/linux.git/commit/?h=upstream-fox-xen&id=9bee65dd3498dfc6aad283d22ff641198b5c91ed 2) Add PCIe Passthrough (GPU) on Xen PVH dom0 Please check patch 4 to 5, this implements acpi_register_gsi_xen_pvh API to register GSI for guest domU, amd make a new privcmd to handle the GSI from the IRQ. Below are the screenshot of these functions, please take a look. Passthrough GPU: https://drive.google.com/file/d/17onr5gvDK8KM_LniHTSQEI2hGJZlI09L/view?usp=share_link Venus: https://drive.google.com/file/d/1_lPq6DMwHu1JQv7LUUVRx31dBj0HJYcL/view?usp=share_link Zink: https://drive.google.com/file/d/1FxLmKu6X7uJOxx1ZzwOm1yA6IL5WMGzd/view?usp=share_link Repositories Kernel: https://git.kernel.org/pub/scm/linux/kernel/git/rui/linux.git/log/?h=upstream-fox-xen Xen: https://gitlab.com/huangrui123/xen/-/commits/upstream-for-xen QEMU: https://gitlab.com/huangrui123/qemu/-/commits/upstream-for-xen Mesa: https://gitlab.freedesktop.org/rui/mesa/-/commits/upstream-for-xen Virglrenderer: https://gitlab.freedesktop.org/rui/virglrenderer/-/commits/upstream-for-xen We are writting the documentation on xen wiki page, and will update it in feature version. Thanks, Ray Chen Jiqian (2): x86/xen: acpi registers gsi for xen pvh xen/privcmd: add IOCTL_PRIVCMD_GSI_FROM_IRQ Huang Rui (3): x86/xen: disable swiotlb for xen pvh xen/grants: update initialization order of xen grant table drm/amdgpu: set passthrough mode for xen pvh/hvm arch/x86/include/asm/apic.h | 7 ++++ arch/x86/include/asm/xen/pci.h | 5 +++ arch/x86/kernel/acpi/boot.c | 2 +- arch/x86/kernel/pci-dma.c | 8 ++++- arch/x86/pci/xen.c | 43 ++++++++++++++++++++++++ arch/x86/xen/grant-table.c | 2 +- drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c | 3 +- drivers/xen/events/events_base.c | 39 +++++++++++++++++++++ drivers/xen/grant-table.c | 2 +- drivers/xen/privcmd.c | 20 +++++++++++ include/uapi/xen/privcmd.h | 7 ++++ include/xen/events.h | 5 +++ 12 files changed, 138 insertions(+), 5 deletions(-)