From patchwork Mon May 15 09:15:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Schnelle X-Patchwork-Id: 9336 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp6781910vqo; Mon, 15 May 2023 02:25:27 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5CNYMCzRI5b+U1hclzQ/0u9K568RtrX1pWdJjh1khZyCVDpzQdmoLQRbLxFrF4Skr6sg2o X-Received: by 2002:a05:6a21:920b:b0:f0:3e78:715b with SMTP id tl11-20020a056a21920b00b000f03e78715bmr33174617pzb.40.1684142727439; Mon, 15 May 2023 02:25:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684142727; cv=none; d=google.com; s=arc-20160816; b=q/F/giNk6H4LnjgLteqh/nOq6iL9hamjpqh8Kj9PI0uv/37mRaHdD8TubueUoxP5/e eJUJd9Y8Ck/1e87pSjJez1BDVwpKLFYG+2NQFhxE5xt0gb4ZaJnwLPtszmrKeh7zNmSt E/zf8Jdy3jK9zK6G/1KYWsrjoY4Q6f2CGXdUboMXB0K8KARDVpZnACvgE6Ed7bIFqqXw SLMmQz4NmbQtuqhasBUN+RNBV3d18htGUr+XGzmD7Xr7HM9uUXqlwUrdCpuuxlXSzSEW ueqXgqff5nfTIVa+ZBbzOlrVLwiMdhwt6G09AZcI/nKfXvR9cSCCvYKgqSoSKfBajFs1 9R5g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:content-transfer-encoding:cc:to :message-id:date:subject:from:dkim-signature; bh=ka0qjy6unHncNjzpsf8Pyt9B+1j1Jy6uYBWZb2IJP/U=; b=yC+FRQg2urrx2XpWDLmSLRT2KhRaP+Cs176dPUGBToLbqkJ7fYGovUICPCEg+NUFDb H19gDD9E1YNvVPfgnz4Yi1oZyXkRSrZ7JdCvbtXJ/aofqwDdaAa0ENLE2+W9nzPS3UqF 5f6LITErzv/igUDX2aBiSisORJ+4+wCVw0awP2NWchqRSkQ7SGCzLsXtbriT6Hfp+kpO BJ73QaMkmk7HJBezoTvDscGpy1d5DHK7e+xHgvFZ0xN5e6WCEhgThtW2WlI8rMalyhca 9icD50ohFPOe3Y6TBvC+NORPMVVvXFbIYgPHFw7nbc1uEjrNzfI96WJw71iZ/q7rjh1D RBxA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@ibm.com header.s=pp1 header.b=rHO3MFrH; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=NONE dis=NONE) header.from=ibm.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id i4-20020a63e444000000b00513f070aaa1si15652377pgk.892.2023.05.15.02.25.14; Mon, 15 May 2023 02:25:27 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@ibm.com header.s=pp1 header.b=rHO3MFrH; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=NONE dis=NONE) header.from=ibm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240385AbjEOJWi (ORCPT + 99 others); Mon, 15 May 2023 05:22:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57540 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229603AbjEOJWA (ORCPT ); Mon, 15 May 2023 05:22:00 -0400 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0CEBC35B3; Mon, 15 May 2023 02:17:18 -0700 (PDT) Received: from pps.filterd (m0353727.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34F97bmd000806; Mon, 15 May 2023 09:16:24 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : subject : date : message-id : content-type : to : cc : content-transfer-encoding : mime-version; s=pp1; bh=ka0qjy6unHncNjzpsf8Pyt9B+1j1Jy6uYBWZb2IJP/U=; b=rHO3MFrHkG5ppeOiy4+KHZgKHwvvqmf3OwBAlh25wnbwnW0TBizDizdjYbc4uFd6x3kh jZo6q5bTBZAc9o+K0nEBxwRivBxnwxx7ufjwao4ixyIQBGfiQGF6pVWF+bjZwNPIjyAK yWcjxWDGA2qbaR+LZXmxKkStx/h2EXloAnzK9pw96wpgDOGf+BJ6LQoUqlDFprPfWgPZ UE8RG1u02XkGS1Dxbc3oqyFc0HoIdHgfnGKJcP1q8GqoSBN3wv6JhMDGG1o7Vn0yT3xM F2Bi7e4norUTRZA8FOZvDuEKk4oUOnf4a6HkK+aFKwl6yxmsHYDrPxnkaBBOzV1d2H+c lQ== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3qkd3mrpv3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 15 May 2023 09:16:23 +0000 Received: from m0353727.ppops.net (m0353727.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34F97tad002590; Mon, 15 May 2023 09:16:22 GMT Received: from ppma06ams.nl.ibm.com (66.31.33a9.ip4.static.sl-reverse.com [169.51.49.102]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3qkd3mrps4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 15 May 2023 09:16:22 +0000 Received: from pps.filterd (ppma06ams.nl.ibm.com [127.0.0.1]) by ppma06ams.nl.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 34F40vhA024749; Mon, 15 May 2023 09:16:17 GMT Received: from smtprelay02.fra02v.mail.ibm.com ([9.218.2.226]) by ppma06ams.nl.ibm.com (PPS) with ESMTPS id 3qj1tdryam-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 15 May 2023 09:16:17 +0000 Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com [10.20.54.103]) by smtprelay02.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 34F9GEeN27460148 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 15 May 2023 09:16:14 GMT Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 10EF820043; Mon, 15 May 2023 09:16:14 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 215B720040; Mon, 15 May 2023 09:16:13 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTP; Mon, 15 May 2023 09:16:13 +0000 (GMT) From: Niklas Schnelle Subject: [PATCH v9 0/6] iommu/dma: s390 DMA API conversion and optimized IOTLB flushing Date: Mon, 15 May 2023 11:15:50 +0200 Message-Id: <20230310-dma_iommu-v9-0-65bb8edd2beb@linux.ibm.com> X-B4-Tracking: v=1; b=H4sIAEb4YWQC/22Oy2rDMBBFfyVonTF6uVay6n+UUGR5XA3oESTbp AT/e5RsAqWzO8PMuffOKhbCys6HOyu4UaWcGpyOB+a8TT8INDVmkkvFleAwRftNOcYVeuTauF6 ZDz2wdj/aijAWm5xvH2kNoS2vBWe6vQK+Lo3nkiMsvqB9a6XkopdtOmGEUVqDgOp8whDwM1Bab x2NsXM5PmM81SWX31fjzTy9/5XbDHCQSg/TPOI02PmP6LLv+wNyj3CL/QAAAA== To: Joerg Roedel , Matthew Rosato , Will Deacon , Wenjia Zhang , Robin Murphy , Jason Gunthorpe Cc: Gerd Bayer , Julian Ruess , Pierre Morel , Alexandra Winter , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Suravee Suthikulpanit , Hector Martin , Sven Peter , Alyssa Rosenzweig , David Woodhouse , Lu Baolu , Andy Gross , Bjorn Andersson , Konrad Dybcio , Yong Wu , Matthias Brugger , AngeloGioacchino Del Regno , Gerald Schaefer , Orson Zhai , Baolin Wang , Chunyan Zhang , Chen-Yu Tsai , Jernej Skrabec , Samuel Holland , Thierry Reding , Krishna Reddy , Jonathan Hunter , Niklas Schnelle , Jonathan Corbet , linux-s390@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev, asahi@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-arm-msm@vger.kernel.org, linux-mediatek@lists.infradead.org, linux-sunxi@lists.linux.dev, linux-tegra@vger.kernel.org, linux-doc@vger.kernel.org X-Mailer: b4 0.12.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=5841; i=schnelle@linux.ibm.com; h=from:subject:message-id; bh=wR8vKH9qzWb5+M2sVnSQXIDDqBuHODWYlv/cdDQr+oY=; b=owGbwMvMwCH2Wz534YHOJ2GMp9WSGFISf0T8jV8/cWugurCKfBPHqoteMmysdem3fGdIpwbfD bnz7cGmjlIWBjEOBlkxRZZFXc5+6wqmmO4J6u+AmcPKBDKEgYtTACZipcvwmyV23c/nmVkLbjXs NzndkrHF9FTL1hvH/3j/1g9LbWa785bhf6Ss1SUJJvf27aL6D7+809DbfiSGNzbjxM0AVfXpohN f8AEA X-Developer-Key: i=schnelle@linux.ibm.com; a=openpgp; fpr=9DB000B2D2752030A5F72DDCAFE43F15E8C26090 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: R5kdhGb_6k2meXm9AjMLBJogm9_V8b81 X-Proofpoint-ORIG-GUID: J80AmfSVEo9tob-DBy4iTRwQJAGNiujj X-Proofpoint-UnRewURL: 0 URL was un-rewritten MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-05-15_06,2023-05-05_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 priorityscore=1501 lowpriorityscore=0 phishscore=0 suspectscore=0 adultscore=0 mlxlogscore=999 bulkscore=0 malwarescore=0 clxscore=1011 mlxscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305150078 X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1765951644801642858?= X-GMAIL-MSGID: =?utf-8?q?1765951644801642858?= Hi All, This patch series converts s390's PCI support from its platform specific DMA API implementation in arch/s390/pci/pci_dma.c to the common DMA IOMMU layer. The conversion itself is done in patches 3-4 with patch 2 providing the final necessary IOMMU driver improvement to handle s390's special IOTLB flush out-of-resource indication in virtualized environments. Patches 1-2 may be applied independently. The conversion itself only touches the s390 IOMMU driver and s390 arch code moving over remaining functions from the s390 DMA API implementation. No changes to common code are necessary. After patch 4 the basic conversion is done and on our partitioning machine hypervisor LPAR performance matches or exceeds the existing code. When running under z/VM or KVM however, performance plummets to about half of the existing code due to a much higher rate of IOTLB flushes for unmapped pages. Due to the hypervisors use of IOTLB flushes to synchronize their shadow tables these are very expensive and minimizing them is key for regaining the performance loss. To this end patches 5-6 add a new, single queue, IOTLB flushing scheme as an alternative to the existing per-CPU flush queues. Introducing an alternative scheme was also suggested by Robin Murphy[1]. In the previous RFC of this conversion Robin suggested reusing more of the existing queuing logic which I incorporated since v2. The single queue mode is introduced in patch 5 together with a new dma_iommu_options struct and tune_dma_iommu callback in IOMMU ops which allows IOMMU drivers to switch to a single flush queue. Then patch 6 enables variable queue sizes using power of 2 queue sizes and shift/mask to keep performance as close to the existing code as possible. The variable queue size and a variable timeout are added to the dma_iommu_options struct and utilized by s390 in the z/VM and KVM guest cases. As it is implemented in common code the single queue IOTLB flushing scheme can of course be used by other platforms with expensive IOTLB flushes. Particularly virtio-iommu may be a candidate. In a previous version I verified that the new scheme does work on my x86_64 Ryzen workstation by locally modifying iommu_subsys_init() to default to the single queue mode and verifying its use via "/sys/.../iommu_group/type". I did not find problems with an AMD GPU, Intel NIC (with SR-IOV and KVM pass-through), NVMes or any on board peripherals. This code is also available in the b4/dma_iommu topic branch of my git.kernel.org repository[3] with tags the version sent. NOTE: Due to the large drop in performance I think we should not merge the DMA API conversion (patch 4) until we have a more suited IOVA flushing scheme with similar improvements as the proposed changes. Best regards, Niklas [0] https://lore.kernel.org/linux-iommu/20221109142903.4080275-1-schnelle@linux.ibm.com/ [1] https://lore.kernel.org/linux-iommu/3e402947-61f9-b7e8-1414-fde006257b6f@arm.com/ [2] https://lore.kernel.org/linux-iommu/a8e778da-7b41-a6ba-83c3-c366a426c3da@arm.com/ [3] https://git.kernel.org/pub/scm/linux/kernel/git/niks/linux.git/ --- Changes in v9: - Rebased on v6.4-rc2 - Re-ordered iommu_group_store_type() to allow passing the device to iommu_dma_init_fq() - Link to v8: https://lore.kernel.org/r/20230310-dma_iommu-v8-0-2347dfbed7af@linux.ibm.com --- Niklas Schnelle (6): s390/ism: Set DMA coherent mask iommu: Allow .iotlb_sync_map to fail and handle s390's -ENOMEM return s390/pci: prepare is_passed_through() for dma-iommu s390/pci: Use dma-iommu layer iommu/dma: Allow a single FQ in addition to per-CPU FQs iommu/dma: Make flush queue sizes and timeout driver configurable Documentation/admin-guide/kernel-parameters.txt | 9 +- arch/s390/include/asm/pci.h | 7 - arch/s390/include/asm/pci_clp.h | 3 + arch/s390/include/asm/pci_dma.h | 119 +--- arch/s390/pci/Makefile | 2 +- arch/s390/pci/pci.c | 22 +- arch/s390/pci/pci_bus.c | 5 - arch/s390/pci/pci_debug.c | 12 +- arch/s390/pci/pci_dma.c | 735 ------------------------ arch/s390/pci/pci_event.c | 17 +- arch/s390/pci/pci_sysfs.c | 19 +- drivers/iommu/Kconfig | 4 +- drivers/iommu/amd/iommu.c | 5 +- drivers/iommu/apple-dart.c | 5 +- drivers/iommu/dma-iommu.c | 191 ++++-- drivers/iommu/dma-iommu.h | 4 +- drivers/iommu/intel/iommu.c | 5 +- drivers/iommu/iommu.c | 38 +- drivers/iommu/msm_iommu.c | 5 +- drivers/iommu/mtk_iommu.c | 5 +- drivers/iommu/s390-iommu.c | 435 +++++++++++++- drivers/iommu/sprd-iommu.c | 5 +- drivers/iommu/sun50i-iommu.c | 4 +- drivers/iommu/tegra-gart.c | 5 +- drivers/s390/net/ism_drv.c | 2 +- include/linux/iommu.h | 29 +- 26 files changed, 682 insertions(+), 1010 deletions(-) --- base-commit: f1fcbaa18b28dec10281551dfe6ed3a3ed80e3d6 change-id: 20230310-dma_iommu-5e048c538647 Best regards,