From patchwork Tue Oct 25 11:56:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Schnelle X-Patchwork-Id: 10726 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp963229wru; Tue, 25 Oct 2022 05:03:55 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5PYvxo5GbIljYPSSmotCpHM3dvkakAigNzeSLXe++T203GJVm4E9Ho/OMy93/CYCeBhj22 X-Received: by 2002:a17:907:168c:b0:7a5:74eb:d123 with SMTP id hc12-20020a170907168c00b007a574ebd123mr11496264ejc.268.1666699435267; Tue, 25 Oct 2022 05:03:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666699435; cv=none; d=google.com; s=arc-20160816; b=M/KA55J7buYKVE1NiBBVw6oAd2h8omJ571AkIWzJ3RrrHF3pKriH99i/I8h32kSTrO S45fz5m2rTcWNiOo+2jYX/rGVQ+u19Jta6W4VmjVotCnJCSOy9BKzsKi+Qyis052xaVK rIBL7yBBiayCW0Lfk3GfR/0AvrMjkl9I+gO3FqeI+03zKdxlTtR9UEz/cnlGmq7U2n7Z y9GGiQI6mdttMldvGLO4eglbaKUFxt0kW7U1pZxa3WdscNPiiHjdBDSoRgPjAxKhEIqK G0UX7f6nMF9ZWri3KFKl+mwf6Z12zeIKSPodz1UYFNh2/ByxyW7QgMzrL5Ee/Gqa/u0l zIVg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=6ap+xyTxqShdrg/o9l0PjSEFOFh+Z/FYE9NvPzet1+8=; b=yVRF/SP29ld7dbTONtFbvDmiiTbsnBqp51JnYukylLDba13pgGza38T6/ds+7BnCHM 2iBVjflJAGoWucQvLo7CwXnnqFupYYMbjSG8aC15ewvJNPOLOoJiB9hNvpNsaf5pvO0J 8q+1AjiBllfsKQ7ODUKeaiVtHLeqKueKdidU3mCBhEMEaw3nBmtDjgughYhAnZk0Cxj6 lYovwD/6nF8/TzV0keQ3lYQjf4o40iwXb/+38OVCMJt7yTACmpKfYM84YunPKi8/sM/i GfnFR5R7Drd5DdHZIFTFG8MpjH3j2eHHyHgfus1iLW0hvu3LkbGoyd6K8os1enmqMhNP Fstw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@ibm.com header.s=pp1 header.b=bqvxn1fP; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id v9-20020aa7d9c9000000b00461f5bb2b79si2244106eds.458.2022.10.25.05.03.29; Tue, 25 Oct 2022 05:03:55 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@ibm.com header.s=pp1 header.b=bqvxn1fP; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231544AbiJYL5n (ORCPT + 99 others); Tue, 25 Oct 2022 07:57:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38058 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231755AbiJYL51 (ORCPT ); Tue, 25 Oct 2022 07:57:27 -0400 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9DF394C01D; Tue, 25 Oct 2022 04:57:26 -0700 (PDT) Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 29PB7x7b022968; Tue, 25 Oct 2022 11:57:04 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=6ap+xyTxqShdrg/o9l0PjSEFOFh+Z/FYE9NvPzet1+8=; b=bqvxn1fPwOKaWqHY1RuYSqOI3wat9c6dg4MNETjPmty/60I+dnVnAW+i4o9YPtRmF1+T YretLXBFd5AYTitzMCeIIr/BVajKHBK8A/CwKATR+wk8AmNfGJJ3Hjw+W41V53YF1C7H fIEK/MCCNb7uOnRy6KbYAK+9vehmbq1vk1rWAf6jLwTFG+4K9XiYqPmTu83If1yBQ6Nr +RIIAvwq2hSPso8apWn2pj26gf/8IZOec7UPrkF1R352M+/tLuWhsVblIGD/f9IjQ9SI WvmdmFy/viyNgJQWl4ZFgf/V6RSl0Vgf7Uz5fU8KS/HVobd3h0uisbDT6d9/MiplwXZ0 7g== Received: from ppma04ams.nl.ibm.com (63.31.33a9.ip4.static.sl-reverse.com [169.51.49.99]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3kedmp3njb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 25 Oct 2022 11:57:04 +0000 Received: from pps.filterd (ppma04ams.nl.ibm.com [127.0.0.1]) by ppma04ams.nl.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 29PBppgo005336; Tue, 25 Oct 2022 11:57:01 GMT Received: from b06cxnps3074.portsmouth.uk.ibm.com (d06relay09.portsmouth.uk.ibm.com [9.149.109.194]) by ppma04ams.nl.ibm.com with ESMTP id 3kc859dj64-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 25 Oct 2022 11:57:01 +0000 Received: from d06av24.portsmouth.uk.ibm.com (d06av24.portsmouth.uk.ibm.com [9.149.105.60]) by b06cxnps3074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 29PBuwaU459512 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 25 Oct 2022 11:56:58 GMT Received: from d06av24.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 78FD04203F; Tue, 25 Oct 2022 11:56:58 +0000 (GMT) Received: from d06av24.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id EA95842041; Tue, 25 Oct 2022 11:56:57 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by d06av24.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 25 Oct 2022 11:56:57 +0000 (GMT) From: Niklas Schnelle To: Matthew Rosato , iommu@lists.linux.dev, Joerg Roedel , Will Deacon , Robin Murphy , Jason Gunthorpe Cc: Gerd Bayer , Pierre Morel , linux-s390@vger.kernel.org, borntraeger@linux.ibm.com, hca@linux.ibm.com, gor@linux.ibm.com, gerald.schaefer@linux.ibm.com, agordeev@linux.ibm.com, svens@linux.ibm.com, linux-kernel@vger.kernel.org, Wenjia Zhang , Julian Ruess Subject: [PATCH v8 1/6] iommu/s390: Fix duplicate domain attachments Date: Tue, 25 Oct 2022 13:56:52 +0200 Message-Id: <20221025115657.1666860-2-schnelle@linux.ibm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221025115657.1666860-1-schnelle@linux.ibm.com> References: <20221025115657.1666860-1-schnelle@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: weZxKZLm9eW69ZSesfCHehuoAzthYaYj X-Proofpoint-ORIG-GUID: weZxKZLm9eW69ZSesfCHehuoAzthYaYj X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-10-25_05,2022-10-25_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 clxscore=1015 impostorscore=0 mlxlogscore=999 suspectscore=0 phishscore=0 adultscore=0 priorityscore=1501 spamscore=0 mlxscore=0 lowpriorityscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2209130000 definitions=main-2210250067 X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747661027291267575?= X-GMAIL-MSGID: =?utf-8?q?1747661027291267575?= Since commit fa7e9ecc5e1c ("iommu/s390: Tolerate repeat attach_dev calls") we can end up with duplicates in the list of devices attached to a domain. This is inefficient and confusing since only one domain can actually be in control of the IOMMU translations for a device. Fix this by detaching the device from the previous domain, if any, on attach. Add a WARN_ON() in case we still have attached devices on freeing the domain. While here remove the re-attach on failure dance as it was determined to be unlikely to help and may confuse debug and recovery. Fixes: fa7e9ecc5e1c ("iommu/s390: Tolerate repeat attach_dev calls") Reviewed-by: Matthew Rosato Reviewed-by: Jason Gunthorpe Signed-off-by: Niklas Schnelle --- v7->v8: - Added Jason's R-b v6->v7: - Added Matt's R-b v5->v6: - Only set zdev->dma_table once zpci_register_ioat() succeeded (Matt) v4->v5: - Unregister IOAT and set zdev->dma_table on error (Matt) drivers/iommu/s390-iommu.c | 106 ++++++++++++++++--------------------- 1 file changed, 45 insertions(+), 61 deletions(-) diff --git a/drivers/iommu/s390-iommu.c b/drivers/iommu/s390-iommu.c index 3c071782f6f1..c2e5e81d609e 100644 --- a/drivers/iommu/s390-iommu.c +++ b/drivers/iommu/s390-iommu.c @@ -79,10 +79,36 @@ static void s390_domain_free(struct iommu_domain *domain) { struct s390_domain *s390_domain = to_s390_domain(domain); + WARN_ON(!list_empty(&s390_domain->devices)); dma_cleanup_tables(s390_domain->dma_table); kfree(s390_domain); } +static void __s390_iommu_detach_device(struct zpci_dev *zdev) +{ + struct s390_domain *s390_domain = zdev->s390_domain; + struct s390_domain_device *domain_device, *tmp; + unsigned long flags; + + if (!s390_domain) + return; + + spin_lock_irqsave(&s390_domain->list_lock, flags); + list_for_each_entry_safe(domain_device, tmp, &s390_domain->devices, + list) { + if (domain_device->zdev == zdev) { + list_del(&domain_device->list); + kfree(domain_device); + break; + } + } + spin_unlock_irqrestore(&s390_domain->list_lock, flags); + + zpci_unregister_ioat(zdev, 0); + zdev->s390_domain = NULL; + zdev->dma_table = NULL; +} + static int s390_iommu_attach_device(struct iommu_domain *domain, struct device *dev) { @@ -90,7 +116,7 @@ static int s390_iommu_attach_device(struct iommu_domain *domain, struct zpci_dev *zdev = to_zpci_dev(dev); struct s390_domain_device *domain_device; unsigned long flags; - int cc, rc; + int cc, rc = 0; if (!zdev) return -ENODEV; @@ -99,24 +125,18 @@ static int s390_iommu_attach_device(struct iommu_domain *domain, if (!domain_device) return -ENOMEM; - if (zdev->dma_table && !zdev->s390_domain) { - cc = zpci_dma_exit_device(zdev); - if (cc) { - rc = -EIO; - goto out_free; - } - } - if (zdev->s390_domain) - zpci_unregister_ioat(zdev, 0); + __s390_iommu_detach_device(zdev); + else if (zdev->dma_table) + zpci_dma_exit_device(zdev); - zdev->dma_table = s390_domain->dma_table; cc = zpci_register_ioat(zdev, 0, zdev->start_dma, zdev->end_dma, - virt_to_phys(zdev->dma_table)); + virt_to_phys(s390_domain->dma_table)); if (cc) { rc = -EIO; - goto out_restore; + goto out_free; } + zdev->dma_table = s390_domain->dma_table; spin_lock_irqsave(&s390_domain->list_lock, flags); /* First device defines the DMA range limits */ @@ -127,9 +147,9 @@ static int s390_iommu_attach_device(struct iommu_domain *domain, /* Allow only devices with identical DMA range limits */ } else if (domain->geometry.aperture_start != zdev->start_dma || domain->geometry.aperture_end != zdev->end_dma) { - rc = -EINVAL; spin_unlock_irqrestore(&s390_domain->list_lock, flags); - goto out_restore; + rc = -EINVAL; + goto out_unregister; } domain_device->zdev = zdev; zdev->s390_domain = s390_domain; @@ -138,14 +158,9 @@ static int s390_iommu_attach_device(struct iommu_domain *domain, return 0; -out_restore: - if (!zdev->s390_domain) { - zpci_dma_init_device(zdev); - } else { - zdev->dma_table = zdev->s390_domain->dma_table; - zpci_register_ioat(zdev, 0, zdev->start_dma, zdev->end_dma, - virt_to_phys(zdev->dma_table)); - } +out_unregister: + zpci_unregister_ioat(zdev, 0); + zdev->dma_table = NULL; out_free: kfree(domain_device); @@ -155,32 +170,12 @@ static int s390_iommu_attach_device(struct iommu_domain *domain, static void s390_iommu_detach_device(struct iommu_domain *domain, struct device *dev) { - struct s390_domain *s390_domain = to_s390_domain(domain); struct zpci_dev *zdev = to_zpci_dev(dev); - struct s390_domain_device *domain_device, *tmp; - unsigned long flags; - int found = 0; - if (!zdev) - return; + WARN_ON(zdev->s390_domain != to_s390_domain(domain)); - spin_lock_irqsave(&s390_domain->list_lock, flags); - list_for_each_entry_safe(domain_device, tmp, &s390_domain->devices, - list) { - if (domain_device->zdev == zdev) { - list_del(&domain_device->list); - kfree(domain_device); - found = 1; - break; - } - } - spin_unlock_irqrestore(&s390_domain->list_lock, flags); - - if (found && (zdev->s390_domain == s390_domain)) { - zdev->s390_domain = NULL; - zpci_unregister_ioat(zdev, 0); - zpci_dma_init_device(zdev); - } + __s390_iommu_detach_device(zdev); + zpci_dma_init_device(zdev); } static struct iommu_device *s390_iommu_probe_device(struct device *dev) @@ -198,24 +193,13 @@ static struct iommu_device *s390_iommu_probe_device(struct device *dev) static void s390_iommu_release_device(struct device *dev) { struct zpci_dev *zdev = to_zpci_dev(dev); - struct iommu_domain *domain; /* - * This is a workaround for a scenario where the IOMMU API common code - * "forgets" to call the detach_dev callback: After binding a device - * to vfio-pci and completing the VFIO_SET_IOMMU ioctl (which triggers - * the attach_dev), removing the device via - * "echo 1 > /sys/bus/pci/devices/.../remove" won't trigger detach_dev, - * only release_device will be called via the BUS_NOTIFY_REMOVED_DEVICE - * notifier. - * - * So let's call detach_dev from here if it hasn't been called before. + * release_device is expected to detach any domain currently attached + * to the device, but keep it attached to other devices in the group. */ - if (zdev && zdev->s390_domain) { - domain = iommu_get_domain_for_dev(dev); - if (domain) - s390_iommu_detach_device(domain, dev); - } + if (zdev) + __s390_iommu_detach_device(zdev); } static int s390_iommu_update_trans(struct s390_domain *s390_domain, From patchwork Tue Oct 25 11:56:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Schnelle X-Patchwork-Id: 10721 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp961271wru; Tue, 25 Oct 2022 05:00:53 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7Tj3H6hzU3yN2cm0m3tfnP/FfhQY9b/HhnAwJfPBHQ/OOIO+0PzkhztKRoloWHCQjXsRyf X-Received: by 2002:a05:6402:518d:b0:45d:d319:2a88 with SMTP id q13-20020a056402518d00b0045dd3192a88mr34464688edd.35.1666699253615; Tue, 25 Oct 2022 05:00:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666699253; cv=none; d=google.com; s=arc-20160816; b=ng/bUg6vNi20LsGX/BBLmx7/2OpAFePU8o2Ekb0H8a90CwHu/7ErV9O0GD25RMMN0X OBCD/Ae+xCYnAkjaBKRlwg6FcmUO8AGO18w0BuzhCzrKbAGWNuC3b5gopoa1JOYf34s7 JR/0CxA2atSNuuaYWORo6xXvMzxICNlUo7sP94F1NV8SsZU6/4mV9JzgVLvkLPdvlWue puP2sNuQC1jlgoyG15Uyaxy4a3zMqUOvUQ+42TqqFJf0nyf9I80DNctPs2AXJ4sUtFsr P8WVDcZ96Za9zTbo0ZJnztmgsbhI3P0kpFmkqZT1VFhMhQ6d9H1WQ/u4I+btASJwoHT/ 9HDw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=jEtGL4nz9h6k8vWHYH1QsuvVwxByWhonDU33AgEq7Us=; b=kjKj5+dYdly9v29FT5dFE+3u8Yx26W9vaJwX7dKHXmZx84wR3XExWZRGWyyYbZrMGP Y505L8vqFxCZX+RPG8H/KeROYtwIXkxKFcK3bzbzapcu+XZZsSD9FVfTv4vu7RxnxDho vEtAuR2VKJDGjYQBEvOSxMkvTB4tnpcnkT/vPIy1xVCMi2F0VEuYX1iviiew3LztMvzg vfzOEblA9Lhb6r3OzRuI/au6zrpNUqnngLaBys5AhttelniSmmVdt0lg0Zi7rmGPBtqd R6dObsygVe+nOSspjBkXs0w6Y5gBqSOsGduQVQelk/JE5QFfZrEiU+3vDuo6XlDTCLdi 1viw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@ibm.com header.s=pp1 header.b=UGRplly7; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id nb26-20020a1709071c9a00b007307fd1b9bbsi1967368ejc.589.2022.10.25.05.00.28; Tue, 25 Oct 2022 05:00:53 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@ibm.com header.s=pp1 header.b=UGRplly7; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230286AbiJYL5Z (ORCPT + 99 others); Tue, 25 Oct 2022 07:57:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37804 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230015AbiJYL5W (ORCPT ); Tue, 25 Oct 2022 07:57:22 -0400 Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3F23F75CED; Tue, 25 Oct 2022 04:57:20 -0700 (PDT) Received: from pps.filterd (m0098419.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 29PB8TXn028314; Tue, 25 Oct 2022 11:57:05 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=jEtGL4nz9h6k8vWHYH1QsuvVwxByWhonDU33AgEq7Us=; b=UGRplly7E9Y6U4jMgy23H+vH3T1PPAuqe3W4qTBhbZhXjqpcreTHgGmyefyGTV/blX85 LCgATJseaWYkljAacEIFfNpeEOI2P13riCItyhyVF/TwNprk2sejYBxZE+/1S4pIYfEk sN2/ozZAm4+kzdlIyze0gljbUT8qb2sB4iGm4Z+Fg8l6XI/L/hcjJeQT2fhRSGqAPOw1 0nB3c2Q0QC4P5GVHKlIeUT1ZdHEqBdZDDZo5p7H056g4C8o5sn99+Izf7IzvdWEwM2CF ZkCNGA8adjsMQZpxT7KwsXDtav7wZknyG17+y4cIDwBqjL0mJKGO+GAryBRMRZVxO5yF EA== Received: from ppma04fra.de.ibm.com (6a.4a.5195.ip4.static.sl-reverse.com [149.81.74.106]) by mx0b-001b2d01.pphosted.com (PPS) with ESMTPS id 3kebjt8p5x-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 25 Oct 2022 11:57:04 +0000 Received: from pps.filterd (ppma04fra.de.ibm.com [127.0.0.1]) by ppma04fra.de.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 29PBnkuP008081; Tue, 25 Oct 2022 11:57:02 GMT Received: from b06cxnps3075.portsmouth.uk.ibm.com (d06relay10.portsmouth.uk.ibm.com [9.149.109.195]) by ppma04fra.de.ibm.com with ESMTP id 3kc859me87-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 25 Oct 2022 11:57:02 +0000 Received: from d06av24.portsmouth.uk.ibm.com (mk.ibm.com [9.149.105.60]) by b06cxnps3075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 29PBuxql2491070 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 25 Oct 2022 11:56:59 GMT Received: from d06av24.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 132D54203F; Tue, 25 Oct 2022 11:56:59 +0000 (GMT) Received: from d06av24.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8531142042; Tue, 25 Oct 2022 11:56:58 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by d06av24.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 25 Oct 2022 11:56:58 +0000 (GMT) From: Niklas Schnelle To: Matthew Rosato , iommu@lists.linux.dev, Joerg Roedel , Will Deacon , Robin Murphy , Jason Gunthorpe Cc: Gerd Bayer , Pierre Morel , linux-s390@vger.kernel.org, borntraeger@linux.ibm.com, hca@linux.ibm.com, gor@linux.ibm.com, gerald.schaefer@linux.ibm.com, agordeev@linux.ibm.com, svens@linux.ibm.com, linux-kernel@vger.kernel.org, Wenjia Zhang , Julian Ruess Subject: [PATCH v8 2/6] iommu/s390: Get rid of s390_domain_device Date: Tue, 25 Oct 2022 13:56:53 +0200 Message-Id: <20221025115657.1666860-3-schnelle@linux.ibm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221025115657.1666860-1-schnelle@linux.ibm.com> References: <20221025115657.1666860-1-schnelle@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: tHyzi7oE8qPRvfbLQX7hLY2fuNhLyt8K X-Proofpoint-ORIG-GUID: tHyzi7oE8qPRvfbLQX7hLY2fuNhLyt8K X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-10-25_05,2022-10-25_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 malwarescore=0 suspectscore=0 spamscore=0 clxscore=1015 mlxscore=0 adultscore=0 lowpriorityscore=0 mlxlogscore=999 priorityscore=1501 impostorscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2209130000 definitions=main-2210250067 X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747660836634716872?= X-GMAIL-MSGID: =?utf-8?q?1747660836634716872?= The struct s390_domain_device serves the sole purpose as list entry for the devices list of a struct s390_domain. As it contains no additional information besides a list_head and a pointer to the struct zpci_dev we can simplify things and just thread the device list through struct zpci_dev directly. This removes the need to allocate during domain attach and gets rid of one level of indirection during mapping operations. Reviewed-by: Matthew Rosato Reviewed-by: Jason Gunthorpe Signed-off-by: Niklas Schnelle --- v7->v8: - Added Jason's R-b v5->v6: - On attach failure make sure the IOAT is not registered and zdev->dma_table == NULL - Dropped Jason's R-b arch/s390/include/asm/pci.h | 1 + drivers/iommu/s390-iommu.c | 37 +++++++------------------------------ 2 files changed, 8 insertions(+), 30 deletions(-) diff --git a/arch/s390/include/asm/pci.h b/arch/s390/include/asm/pci.h index 108e732d7b14..15f8714ca9b7 100644 --- a/arch/s390/include/asm/pci.h +++ b/arch/s390/include/asm/pci.h @@ -117,6 +117,7 @@ struct zpci_bus { struct zpci_dev { struct zpci_bus *zbus; struct list_head entry; /* list of all zpci_devices, needed for hotplug, etc. */ + struct list_head iommu_list; struct kref kref; struct hotplug_slot hotplug_slot; diff --git a/drivers/iommu/s390-iommu.c b/drivers/iommu/s390-iommu.c index c2e5e81d609e..af83ccde16a4 100644 --- a/drivers/iommu/s390-iommu.c +++ b/drivers/iommu/s390-iommu.c @@ -29,11 +29,6 @@ struct s390_domain { spinlock_t list_lock; }; -struct s390_domain_device { - struct list_head list; - struct zpci_dev *zdev; -}; - static struct s390_domain *to_s390_domain(struct iommu_domain *dom) { return container_of(dom, struct s390_domain, domain); @@ -87,21 +82,13 @@ static void s390_domain_free(struct iommu_domain *domain) static void __s390_iommu_detach_device(struct zpci_dev *zdev) { struct s390_domain *s390_domain = zdev->s390_domain; - struct s390_domain_device *domain_device, *tmp; unsigned long flags; if (!s390_domain) return; spin_lock_irqsave(&s390_domain->list_lock, flags); - list_for_each_entry_safe(domain_device, tmp, &s390_domain->devices, - list) { - if (domain_device->zdev == zdev) { - list_del(&domain_device->list); - kfree(domain_device); - break; - } - } + list_del_init(&zdev->iommu_list); spin_unlock_irqrestore(&s390_domain->list_lock, flags); zpci_unregister_ioat(zdev, 0); @@ -114,17 +101,12 @@ static int s390_iommu_attach_device(struct iommu_domain *domain, { struct s390_domain *s390_domain = to_s390_domain(domain); struct zpci_dev *zdev = to_zpci_dev(dev); - struct s390_domain_device *domain_device; unsigned long flags; int cc, rc = 0; if (!zdev) return -ENODEV; - domain_device = kzalloc(sizeof(*domain_device), GFP_KERNEL); - if (!domain_device) - return -ENOMEM; - if (zdev->s390_domain) __s390_iommu_detach_device(zdev); else if (zdev->dma_table) @@ -132,10 +114,8 @@ static int s390_iommu_attach_device(struct iommu_domain *domain, cc = zpci_register_ioat(zdev, 0, zdev->start_dma, zdev->end_dma, virt_to_phys(s390_domain->dma_table)); - if (cc) { - rc = -EIO; - goto out_free; - } + if (cc) + return -EIO; zdev->dma_table = s390_domain->dma_table; spin_lock_irqsave(&s390_domain->list_lock, flags); @@ -151,9 +131,8 @@ static int s390_iommu_attach_device(struct iommu_domain *domain, rc = -EINVAL; goto out_unregister; } - domain_device->zdev = zdev; zdev->s390_domain = s390_domain; - list_add(&domain_device->list, &s390_domain->devices); + list_add(&zdev->iommu_list, &s390_domain->devices); spin_unlock_irqrestore(&s390_domain->list_lock, flags); return 0; @@ -161,8 +140,6 @@ static int s390_iommu_attach_device(struct iommu_domain *domain, out_unregister: zpci_unregister_ioat(zdev, 0); zdev->dma_table = NULL; -out_free: - kfree(domain_device); return rc; } @@ -206,10 +183,10 @@ static int s390_iommu_update_trans(struct s390_domain *s390_domain, phys_addr_t pa, dma_addr_t dma_addr, size_t size, int flags) { - struct s390_domain_device *domain_device; phys_addr_t page_addr = pa & PAGE_MASK; dma_addr_t start_dma_addr = dma_addr; unsigned long irq_flags, nr_pages, i; + struct zpci_dev *zdev; unsigned long *entry; int rc = 0; @@ -234,8 +211,8 @@ static int s390_iommu_update_trans(struct s390_domain *s390_domain, } spin_lock(&s390_domain->list_lock); - list_for_each_entry(domain_device, &s390_domain->devices, list) { - rc = zpci_refresh_trans((u64) domain_device->zdev->fh << 32, + list_for_each_entry(zdev, &s390_domain->devices, iommu_list) { + rc = zpci_refresh_trans((u64)zdev->fh << 32, start_dma_addr, nr_pages * PAGE_SIZE); if (rc) break; From patchwork Tue Oct 25 11:56:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Schnelle X-Patchwork-Id: 10725 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp962987wru; Tue, 25 Oct 2022 05:03:28 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6uUez+8enfIvwWlCOUgKBC0BUPq508J8JlPcuIfeMBd+Sxn9RwD5Rey4CszHuUQovyvf4F X-Received: by 2002:a17:906:5dce:b0:78d:ec48:6a58 with SMTP id p14-20020a1709065dce00b0078dec486a58mr32101477ejv.209.1666699408135; Tue, 25 Oct 2022 05:03:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666699408; cv=none; d=google.com; s=arc-20160816; b=oXIJQ/F18NsLtR8Oe4oDEWvkvSZdpBvmVIABZdUCWij+fXchyOV1NMBm15HeJEJlVn uIB9F1rgvvNajGcCNySXmYIBmrVABd/HWgQcd8+MzKw4nEpcu5Lx2wukNR+CKsNfpGYF WPYGtIQZTTxPTrwaDblmCWTzFIEwLYfLmOmgpAnYNnEhPOIr4CagY5iwgPcFwt83ewgk qQ7aRgFh5oIVXAgZpfi7titErab0MDLJY1XPG42q8lHYIrSyY43+sf4h7TBhcqwlobpj ZioiZphjcFoc4Vm9qjHA2puzVt7ycpvzoM6rLsg+E/auDD50Jiuzb9YboxrnxAO5fehC 3bDw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=u+bp7DIW9VjPoZwaAFrgJMmr83Aha+pCvl3ztX85Qzk=; b=DwuUD3O/XpV4FUUEk1C8bN5+uitIW3pdY7mGW9BqXjzq7c1m6D25b2XfR9XIwM39Kd pxBgRhkoz0bvzergw+5nz2RNl03LeayXEVP5l8A1LhbA73EwnE774ufp+XDbozuVcmQt Aaxpt/kgpC2EbB60uVCDPzW51+qfwpOxlVNIiXWIF2iWxqqO5x78bUhNKw66VR7GlzZs NmA1BqLqoOtofZKWDYtKH5EKawsQdmANs+2Own0MEhvMAcWME8qJBywFO+EKIeCAy50U aqA3KMZvcxvs81Nu6Ym44/Aw9y5LObWNkHp6rgD864I2xbGLbxFLqBjY86BC67yCWktv zBCw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@ibm.com header.s=pp1 header.b=VCysFadJ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id dz10-20020a0564021d4a00b004602df03e36si3174843edb.486.2022.10.25.05.02.59; Tue, 25 Oct 2022 05:03:28 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@ibm.com header.s=pp1 header.b=VCysFadJ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231504AbiJYL5j (ORCPT + 99 others); Tue, 25 Oct 2022 07:57:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38082 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231758AbiJYL51 (ORCPT ); Tue, 25 Oct 2022 07:57:27 -0400 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B05D375CED; Tue, 25 Oct 2022 04:57:26 -0700 (PDT) Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 29PB80Ho023008; Tue, 25 Oct 2022 11:57:05 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=u+bp7DIW9VjPoZwaAFrgJMmr83Aha+pCvl3ztX85Qzk=; b=VCysFadJhZewhwUfmz3VoksrEQ690Ts2WboKacEpAJ5DrolOL8wFoky5YucyYkfmbe6/ BBgcfl522JcLtP6QLLoY/pfkgyHbkqPzwanjW6KMVT1Mrtflsw3wnWzf/l7hmnUJWwdb YHdHpZrXJtbP8Avkak+Jk5FozFLnqZs0QtWua7LNXC80VelcuwOhaiD5bGCfD3PG2V/v nTMhVnSgkGS8sZ0goc3ie2tWmB412zfO5HlHB7HnypcyxBtEJ8QNAHIKASmPjj1PFf90 M8Uqo4t/HC99ObQ5KgPwSxtM3a675Rpx66H5/egpSJZaRfsln6/8NqFkze0wZukfkAYe lA== Received: from ppma02fra.de.ibm.com (47.49.7a9f.ip4.static.sl-reverse.com [159.122.73.71]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3kedmp3njv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 25 Oct 2022 11:57:05 +0000 Received: from pps.filterd (ppma02fra.de.ibm.com [127.0.0.1]) by ppma02fra.de.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 29PBosVw013634; Tue, 25 Oct 2022 11:57:03 GMT Received: from b06cxnps3075.portsmouth.uk.ibm.com (d06relay10.portsmouth.uk.ibm.com [9.149.109.195]) by ppma02fra.de.ibm.com with ESMTP id 3kc859m0da-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 25 Oct 2022 11:57:02 +0000 Received: from d06av24.portsmouth.uk.ibm.com (d06av24.portsmouth.uk.ibm.com [9.149.105.60]) by b06cxnps3075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 29PBuxS42491072 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 25 Oct 2022 11:56:59 GMT Received: from d06av24.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9FB334203F; Tue, 25 Oct 2022 11:56:59 +0000 (GMT) Received: from d06av24.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1F75442041; Tue, 25 Oct 2022 11:56:59 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by d06av24.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 25 Oct 2022 11:56:59 +0000 (GMT) From: Niklas Schnelle To: Matthew Rosato , iommu@lists.linux.dev, Joerg Roedel , Will Deacon , Robin Murphy , Jason Gunthorpe Cc: Gerd Bayer , Pierre Morel , linux-s390@vger.kernel.org, borntraeger@linux.ibm.com, hca@linux.ibm.com, gor@linux.ibm.com, gerald.schaefer@linux.ibm.com, agordeev@linux.ibm.com, svens@linux.ibm.com, linux-kernel@vger.kernel.org, Wenjia Zhang , Julian Ruess Subject: [PATCH v8 3/6] iommu/s390: Fix potential s390_domain aperture shrinking Date: Tue, 25 Oct 2022 13:56:54 +0200 Message-Id: <20221025115657.1666860-4-schnelle@linux.ibm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221025115657.1666860-1-schnelle@linux.ibm.com> References: <20221025115657.1666860-1-schnelle@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: OkyfhYi4kw4dTh827ZrXMUZB7LNiJIq- X-Proofpoint-ORIG-GUID: OkyfhYi4kw4dTh827ZrXMUZB7LNiJIq- X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-10-25_05,2022-10-25_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 clxscore=1015 impostorscore=0 mlxlogscore=999 suspectscore=0 phishscore=0 adultscore=0 priorityscore=1501 spamscore=0 mlxscore=0 lowpriorityscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2209130000 definitions=main-2210250067 X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747660998586385936?= X-GMAIL-MSGID: =?utf-8?q?1747660998586385936?= The s390 IOMMU driver currently sets the IOMMU domain's aperture to match the device specific DMA address range of the device that is first attached. This is not ideal. For one if the domain has no device attached in the meantime the aperture could be shrunk allowing translations outside the aperture to exist in the translation tables. Also this is a bit of a misuse of the aperture which really should describe what addresses can be translated and not some device specific limitations. Instead of misusing the aperture like this we can instead create reserved ranges for the ranges inaccessible to the attached devices allowing devices with overlapping ranges to still share an IOMMU domain. This also significantly simplifies s390_iommu_attach_device() allowing us to move the aperture check to the beginning of the function and removing the need to hold the device list's lock to check the aperture. As we then use the same aperture for all domains and it only depends on the table properties we can already check zdev->start_dma/end_dma at probe time and turn the check on attach into a WARN_ON(). Suggested-by: Jason Gunthorpe Reviewed-by: Matthew Rosato Reviewed-by: Jason Gunthorpe Signed-off-by: Niklas Schnelle --- v7->v8: - Added Jason's R-b - Added GFP_KERNEL param to iommu_alloc_resv_region() v5->v6: - Return -EINVAL after WARN_ON() in attach v4->v5: - Make aperture check in attach a WARN_ON() and fail in probe if zdev->start_dma/end_dma doesn't git in aperture (Jason) drivers/iommu/s390-iommu.c | 63 ++++++++++++++++++++++++++------------ 1 file changed, 43 insertions(+), 20 deletions(-) diff --git a/drivers/iommu/s390-iommu.c b/drivers/iommu/s390-iommu.c index af83ccde16a4..9b3adc61005c 100644 --- a/drivers/iommu/s390-iommu.c +++ b/drivers/iommu/s390-iommu.c @@ -62,6 +62,9 @@ static struct iommu_domain *s390_domain_alloc(unsigned domain_type) kfree(s390_domain); return NULL; } + s390_domain->domain.geometry.force_aperture = true; + s390_domain->domain.geometry.aperture_start = 0; + s390_domain->domain.geometry.aperture_end = ZPCI_TABLE_SIZE_RT - 1; spin_lock_init(&s390_domain->dma_table_lock); spin_lock_init(&s390_domain->list_lock); @@ -102,11 +105,15 @@ static int s390_iommu_attach_device(struct iommu_domain *domain, struct s390_domain *s390_domain = to_s390_domain(domain); struct zpci_dev *zdev = to_zpci_dev(dev); unsigned long flags; - int cc, rc = 0; + int cc; if (!zdev) return -ENODEV; + if (WARN_ON(domain->geometry.aperture_start > zdev->end_dma || + domain->geometry.aperture_end < zdev->start_dma)) + return -EINVAL; + if (zdev->s390_domain) __s390_iommu_detach_device(zdev); else if (zdev->dma_table) @@ -118,30 +125,14 @@ static int s390_iommu_attach_device(struct iommu_domain *domain, return -EIO; zdev->dma_table = s390_domain->dma_table; - spin_lock_irqsave(&s390_domain->list_lock, flags); - /* First device defines the DMA range limits */ - if (list_empty(&s390_domain->devices)) { - domain->geometry.aperture_start = zdev->start_dma; - domain->geometry.aperture_end = zdev->end_dma; - domain->geometry.force_aperture = true; - /* Allow only devices with identical DMA range limits */ - } else if (domain->geometry.aperture_start != zdev->start_dma || - domain->geometry.aperture_end != zdev->end_dma) { - spin_unlock_irqrestore(&s390_domain->list_lock, flags); - rc = -EINVAL; - goto out_unregister; - } + zdev->dma_table = s390_domain->dma_table; zdev->s390_domain = s390_domain; + + spin_lock_irqsave(&s390_domain->list_lock, flags); list_add(&zdev->iommu_list, &s390_domain->devices); spin_unlock_irqrestore(&s390_domain->list_lock, flags); return 0; - -out_unregister: - zpci_unregister_ioat(zdev, 0); - zdev->dma_table = NULL; - - return rc; } static void s390_iommu_detach_device(struct iommu_domain *domain, @@ -155,6 +146,30 @@ static void s390_iommu_detach_device(struct iommu_domain *domain, zpci_dma_init_device(zdev); } +static void s390_iommu_get_resv_regions(struct device *dev, + struct list_head *list) +{ + struct zpci_dev *zdev = to_zpci_dev(dev); + struct iommu_resv_region *region; + + if (zdev->start_dma) { + region = iommu_alloc_resv_region(0, zdev->start_dma, 0, + IOMMU_RESV_RESERVED, GFP_KERNEL); + if (!region) + return; + list_add_tail(®ion->list, list); + } + + if (zdev->end_dma < ZPCI_TABLE_SIZE_RT - 1) { + region = iommu_alloc_resv_region(zdev->end_dma + 1, + ZPCI_TABLE_SIZE_RT - zdev->end_dma - 1, + 0, IOMMU_RESV_RESERVED, GFP_KERNEL); + if (!region) + return; + list_add_tail(®ion->list, list); + } +} + static struct iommu_device *s390_iommu_probe_device(struct device *dev) { struct zpci_dev *zdev; @@ -164,6 +179,13 @@ static struct iommu_device *s390_iommu_probe_device(struct device *dev) zdev = to_zpci_dev(dev); + if (zdev->start_dma > zdev->end_dma || + zdev->start_dma > ZPCI_TABLE_SIZE_RT - 1) + return ERR_PTR(-EINVAL); + + if (zdev->end_dma > ZPCI_TABLE_SIZE_RT - 1) + zdev->end_dma = ZPCI_TABLE_SIZE_RT - 1; + return &zdev->iommu_dev; } @@ -342,6 +364,7 @@ static const struct iommu_ops s390_iommu_ops = { .release_device = s390_iommu_release_device, .device_group = generic_device_group, .pgsize_bitmap = S390_IOMMU_PGSIZES, + .get_resv_regions = s390_iommu_get_resv_regions, .default_domain_ops = &(const struct iommu_domain_ops) { .attach_dev = s390_iommu_attach_device, .detach_dev = s390_iommu_detach_device, From patchwork Tue Oct 25 11:56:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Schnelle X-Patchwork-Id: 10723 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp961604wru; Tue, 25 Oct 2022 05:01:31 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7Guwnl0Zvsiw2dzrvz+0sA30b6MW793B6jEMeSRAX4nKOIu9ali4M6s3wA8YhtA8CiNcLO X-Received: by 2002:a17:906:730f:b0:791:9b75:2ca1 with SMTP id di15-20020a170906730f00b007919b752ca1mr30755543ejc.140.1666699291413; Tue, 25 Oct 2022 05:01:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666699291; cv=none; d=google.com; s=arc-20160816; b=jWrm63rMHFUMwqo7sEp1qazgyOEHGwPsWAxMivzSGs8s2NK3aAc4lH9tnCcAjildR1 CKdssA9lYRDlNok/oo3BQIZXZv+QfTaLK47AWkshFn1xxsTPrs3ScxMYK0iXPur7QpWL G3+y0BrBLCzu+tunyhVxsby/y2sHCjECyui+bLoN5EQi1RD2A00BAR6vbDLyuJDZPCMO qEwWf43RdxHiYDi9RpHQXyS+rR5DqdKzdWJeRc///67XYzB1OuASK5hQFigHdb+UebVD CQgj4hNEvgZ14d314XWam02Ek4quuDnrm/gCcqjfyMwIuUfQ0EhuxgGlS7kv/Nger4pt md6Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=unYoLKyySQNeEa7w7NV2iUuiZnAZ4UtNtoWgCnGGL4s=; b=ELXIseV1G5Ry3kcNzPIrVb2CF9l7lOHMmMqP8FWPw+yLNgMhLkWiZGMcgH0VD3XRIj eSkYuySawdjKH9Y0ZbppgheZNbwhnqpvQ17hfkYIV6hgUul4Cy/nLdml6cO3FeWK/X+G vnVIDoRC2QvMeEn4xbLstrnMmR6iX05OFnom/jNrpYmH5aPO3mXuqkOjnp6J+HofAyiN xqFA4pe6mmPrHHdFqkvSxhrxytOhCkq4i30P0QAFwPZx0zWejN1NAE0wLQF/RaFWuWOj IBmYJEX5GsAV9HSnbi7UavCYAy2btPODrc5DzCw8J9Nj4l1DS+RaWGVjqt2Kb1mwk9RE KwXw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@ibm.com header.s=pp1 header.b=Tdv8iDsU; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id ne7-20020a1709077b8700b007a4feae7adfsi2475512ejc.565.2022.10.25.05.01.06; Tue, 25 Oct 2022 05:01:31 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@ibm.com header.s=pp1 header.b=Tdv8iDsU; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231800AbiJYL52 (ORCPT + 99 others); Tue, 25 Oct 2022 07:57:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37864 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231368AbiJYL5Y (ORCPT ); Tue, 25 Oct 2022 07:57:24 -0400 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BF7A189931; Tue, 25 Oct 2022 04:57:23 -0700 (PDT) Received: from pps.filterd (m0098421.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 29PB8Vnr018212; Tue, 25 Oct 2022 11:57:06 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=unYoLKyySQNeEa7w7NV2iUuiZnAZ4UtNtoWgCnGGL4s=; b=Tdv8iDsUtOVxKZ5lgv5u/oTkEReQD8iUV/6UpmObUgTG4mPJXrksYJNt6pWsGP5sCNUI ORqOEpmh2jV41VG+8lDKgt7Yns7v0v/F5jdE6XvRKzvo9YFB+L+vLyRK13oBWOJrAdX7 lpj5KMqzq4PVa5o5Q4toxx5Jgb9HqttAhGbnPIz/DUnLM0RuSbZx5g8zVY8DJHZ+Ab3a d/yDwsegjpoisILQpXqTaGT0/C/5sc9x2+x+uN4ZKkbcQcY49VIkpRqI0eNP9h6ENUMH erp8hra3pFze1nfA0DlPuLV1lg9QWPaZ/qPve2pd+6n7wG9B26AwmCJy7peHW/IRdwCj rw== Received: from ppma03ams.nl.ibm.com (62.31.33a9.ip4.static.sl-reverse.com [169.51.49.98]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3keea7a69w-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 25 Oct 2022 11:57:05 +0000 Received: from pps.filterd (ppma03ams.nl.ibm.com [127.0.0.1]) by ppma03ams.nl.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 29PBoqLl000478; Tue, 25 Oct 2022 11:57:04 GMT Received: from b06cxnps4076.portsmouth.uk.ibm.com (d06relay13.portsmouth.uk.ibm.com [9.149.109.198]) by ppma03ams.nl.ibm.com with ESMTP id 3kdugat5r4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 25 Oct 2022 11:57:03 +0000 Received: from d06av24.portsmouth.uk.ibm.com (mk.ibm.com [9.149.105.60]) by b06cxnps4076.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 29PBv0oL3932904 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 25 Oct 2022 11:57:00 GMT Received: from d06av24.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 40C924203F; Tue, 25 Oct 2022 11:57:00 +0000 (GMT) Received: from d06av24.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id ABD4C42047; Tue, 25 Oct 2022 11:56:59 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by d06av24.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 25 Oct 2022 11:56:59 +0000 (GMT) From: Niklas Schnelle To: Matthew Rosato , iommu@lists.linux.dev, Joerg Roedel , Will Deacon , Robin Murphy , Jason Gunthorpe Cc: Gerd Bayer , Pierre Morel , linux-s390@vger.kernel.org, borntraeger@linux.ibm.com, hca@linux.ibm.com, gor@linux.ibm.com, gerald.schaefer@linux.ibm.com, agordeev@linux.ibm.com, svens@linux.ibm.com, linux-kernel@vger.kernel.org, Wenjia Zhang , Julian Ruess Subject: [PATCH v8 4/6] iommu/s390: Fix incorrect aperture check Date: Tue, 25 Oct 2022 13:56:55 +0200 Message-Id: <20221025115657.1666860-5-schnelle@linux.ibm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221025115657.1666860-1-schnelle@linux.ibm.com> References: <20221025115657.1666860-1-schnelle@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: 9RftsI3RZCAxYB0NfIuqzR71TKrTObBQ X-Proofpoint-GUID: 9RftsI3RZCAxYB0NfIuqzR71TKrTObBQ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-10-25_05,2022-10-25_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 lowpriorityscore=0 bulkscore=0 phishscore=0 mlxlogscore=939 mlxscore=0 spamscore=0 adultscore=0 impostorscore=0 clxscore=1015 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2209130000 definitions=main-2210250067 X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747660876236560250?= X-GMAIL-MSGID: =?utf-8?q?1747660876236560250?= The domain->geometry.aperture_end specifies the last valid address treat it as such when checking if a DMA address is valid. Reviewed-by: Pierre Morel Reviewed-by: Matthew Rosato Reviewed-by: Jason Gunthorpe Signed-off-by: Niklas Schnelle --- v7->v8: - Added Jason's R-b v3->v4: - Make check robust against overflow (Jason) drivers/iommu/s390-iommu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/iommu/s390-iommu.c b/drivers/iommu/s390-iommu.c index 9b3adc61005c..3e601ca6ee0f 100644 --- a/drivers/iommu/s390-iommu.c +++ b/drivers/iommu/s390-iommu.c @@ -213,7 +213,7 @@ static int s390_iommu_update_trans(struct s390_domain *s390_domain, int rc = 0; if (dma_addr < s390_domain->domain.geometry.aperture_start || - dma_addr + size > s390_domain->domain.geometry.aperture_end) + (dma_addr + size - 1) > s390_domain->domain.geometry.aperture_end) return -EINVAL; nr_pages = PAGE_ALIGN(size) >> PAGE_SHIFT; From patchwork Tue Oct 25 11:56:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Schnelle X-Patchwork-Id: 10722 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp961597wru; Tue, 25 Oct 2022 05:01:30 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7g8ha2l5jsRwBvipI+N9fXAJ2Ly0EnIo5Vsx+wEqYl34W9pQUR00WB14cDLtpB3/vdCIEp X-Received: by 2002:a17:906:9750:b0:798:9ccc:845d with SMTP id o16-20020a170906975000b007989ccc845dmr22958705ejy.760.1666699290618; Tue, 25 Oct 2022 05:01:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666699290; cv=none; d=google.com; s=arc-20160816; b=oo8j5PuIKOO7EPWWVyCobV8sTGguhIcfmZ1bqVmhbEGiz4PC6/DkDhQoyzVNduJMtB +WAtdFbCh9i1EUmgJBL48NQX2+NqIBDPUu4grcxJuPBFi/yK4dVZu4utRmpDNzaQso66 ANYoRJaJP85BdV8IBWHlrYAjBMqGpnvdwq1wroB/2SOQJKnaO3JZ7u2t1BpQw4aVCI+7 /nyxpZjrR2cMxEq5YuuRopcq2+OEd0iFmfALXMzRl/MgR96HTjcsKkQN8Jr4aVvNiEqW AF3XKYvtVOLaE+y5PGOES+4D4HVXB1sXk1OKaBtb0ctuntlSikY8kzTKtbPDOJhldGE3 MMEg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=OM0yc0r7iPSVDNJvuhXu6LTZBFoaNpGTvRFXUDrBdh0=; b=Ff6J8ATWCiLspJCPDiiUliAd8QamsfyLGNt7KyJozhslZYmToW3rvz8crl92S/XLRt qIQpiuHnux2AFVg40HlKo7TtoBDx6GqLL6kgZBoB0qsAStO0x87RrgjeYjhXhIFp3vWk 49JzzGmSTmZ1+TOkLq8fDPB8dAtaiGlBOEy9jKkFouJgogiHN5UIQCXzrjuQCcq3sPex KhqQ4/3LrdAAVLGCzXKxbIcFmXxFNO3hVcIemd5OXQM8iVGknui5+7oxTjqTK1WwqD5z RenMIOtqeQYQdbtRZ+inMIyx73pb6YQ8737jgHam58SCTcR73zkRsw/r+IMyncGAbbe/ Gh5A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@ibm.com header.s=pp1 header.b=DN5ONvwz; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id w2-20020a056402268200b00461f1a036f0si2871158edd.335.2022.10.25.05.01.04; Tue, 25 Oct 2022 05:01:30 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@ibm.com header.s=pp1 header.b=DN5ONvwz; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231492AbiJYL5f (ORCPT + 99 others); Tue, 25 Oct 2022 07:57:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37862 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231523AbiJYL5Y (ORCPT ); Tue, 25 Oct 2022 07:57:24 -0400 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 84F234C01D; Tue, 25 Oct 2022 04:57:22 -0700 (PDT) Received: from pps.filterd (m0098421.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 29PB8VVt018219; Tue, 25 Oct 2022 11:57:06 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=OM0yc0r7iPSVDNJvuhXu6LTZBFoaNpGTvRFXUDrBdh0=; b=DN5ONvwz2ifTDf2F7ctj2wpU1xYwJ8FxHfa4A1SmxAIcm3xkEX7tMkKZRQ9EpEBDDLxn 4dr7vQfHs6qVK11SdQx+ax0ObhwzJJbnGluZm+kapX86h4NGwtUTbPN7rE2S9fg2oEUo sttYO7Vo+HV13okm2aixQmAzmMhqQzGweu2ubEx+Er8jSAa2aM7/6YokUfKSl1uEQfoz WOaSqbcdZLrfJZ3MYFBVwfWIpRPxI+h2s8zBJCIOcTu0ii0Exqv3Skn/t8hCQdljaB84 lUQuU6/Th79PusMDVHxkwEN2ngbiqG2yiJJTY5xvmrQeA0FVNMBUn4Bw6UWFKKE/8Fke OA== Received: from ppma04ams.nl.ibm.com (63.31.33a9.ip4.static.sl-reverse.com [169.51.49.99]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3keea7a69x-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 25 Oct 2022 11:57:05 +0000 Received: from pps.filterd (ppma04ams.nl.ibm.com [127.0.0.1]) by ppma04ams.nl.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 29PBpcfb005230; Tue, 25 Oct 2022 11:57:04 GMT Received: from b06cxnps4076.portsmouth.uk.ibm.com (d06relay13.portsmouth.uk.ibm.com [9.149.109.198]) by ppma04ams.nl.ibm.com with ESMTP id 3kc859dj66-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 25 Oct 2022 11:57:03 +0000 Received: from d06av24.portsmouth.uk.ibm.com (mk.ibm.com [9.149.105.60]) by b06cxnps4076.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 29PBv0cG63570372 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 25 Oct 2022 11:57:01 GMT Received: from d06av24.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id CEA854203F; Tue, 25 Oct 2022 11:57:00 +0000 (GMT) Received: from d06av24.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4D18042042; Tue, 25 Oct 2022 11:57:00 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by d06av24.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 25 Oct 2022 11:57:00 +0000 (GMT) From: Niklas Schnelle To: Matthew Rosato , iommu@lists.linux.dev, Joerg Roedel , Will Deacon , Robin Murphy , Jason Gunthorpe Cc: Gerd Bayer , Pierre Morel , linux-s390@vger.kernel.org, borntraeger@linux.ibm.com, hca@linux.ibm.com, gor@linux.ibm.com, gerald.schaefer@linux.ibm.com, agordeev@linux.ibm.com, svens@linux.ibm.com, linux-kernel@vger.kernel.org, Wenjia Zhang , Julian Ruess Subject: [PATCH v8 5/6] iommu/s390: Fix incorrect pgsize_bitmap Date: Tue, 25 Oct 2022 13:56:56 +0200 Message-Id: <20221025115657.1666860-6-schnelle@linux.ibm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221025115657.1666860-1-schnelle@linux.ibm.com> References: <20221025115657.1666860-1-schnelle@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: CRXIUtk9lvnLR5zKbAoC-eYeayCIgrKF X-Proofpoint-GUID: CRXIUtk9lvnLR5zKbAoC-eYeayCIgrKF X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-10-25_05,2022-10-25_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 lowpriorityscore=0 bulkscore=0 phishscore=0 mlxlogscore=999 mlxscore=0 spamscore=0 adultscore=0 impostorscore=0 clxscore=1015 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2209130000 definitions=main-2210250067 X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747660875142078783?= X-GMAIL-MSGID: =?utf-8?q?1747660875142078783?= The .pgsize_bitmap property of struct iommu_ops is not a page mask but rather has a bit set for each size of pages the IOMMU supports. As the comment correctly pointed out at this moment the code only support 4K pages so simply use SZ_4K here. Reviewed-by: Matthew Rosato Reviewed-by: Jason Gunthorpe Signed-off-by: Niklas Schnelle --- drivers/iommu/s390-iommu.c | 9 +-------- 1 file changed, 1 insertion(+), 8 deletions(-) diff --git a/drivers/iommu/s390-iommu.c b/drivers/iommu/s390-iommu.c index 3e601ca6ee0f..104dfbec1037 100644 --- a/drivers/iommu/s390-iommu.c +++ b/drivers/iommu/s390-iommu.c @@ -12,13 +12,6 @@ #include #include -/* - * Physically contiguous memory regions can be mapped with 4 KiB alignment, - * we allow all page sizes that are an order of 4KiB (no special large page - * support so far). - */ -#define S390_IOMMU_PGSIZES (~0xFFFUL) - static const struct iommu_ops s390_iommu_ops; struct s390_domain { @@ -363,7 +356,7 @@ static const struct iommu_ops s390_iommu_ops = { .probe_device = s390_iommu_probe_device, .release_device = s390_iommu_release_device, .device_group = generic_device_group, - .pgsize_bitmap = S390_IOMMU_PGSIZES, + .pgsize_bitmap = SZ_4K, .get_resv_regions = s390_iommu_get_resv_regions, .default_domain_ops = &(const struct iommu_domain_ops) { .attach_dev = s390_iommu_attach_device, From patchwork Tue Oct 25 11:56:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Schnelle X-Patchwork-Id: 10724 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp962080wru; Tue, 25 Oct 2022 05:02:13 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6vzTFl4w8Uc8F0tqo1d1isxh8fpZaqb1NHuFNwAAFlNC8U5B3kl5wRUnEXIskvnA0jFggT X-Received: by 2002:a05:6402:28ca:b0:43b:5235:f325 with SMTP id ef10-20020a05640228ca00b0043b5235f325mr34855190edb.320.1666699333595; Tue, 25 Oct 2022 05:02:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666699333; cv=none; d=google.com; s=arc-20160816; b=Yzf2FP9MUfnAIPcZymGOn9MUkjZraJegYd0ZFyWokUNEncmSw4Y7QhnbkEjm555/AQ sP+gSGZWJAUdpOoTggKpyDt3AWtTppicUurJbVGsdsjGPrLScA8QzJ1TNmxc15g+ElE5 MEK806idMr0OhmMjqO1myj6ElGNo9g2DpwlKj5sG7CLUxglOGYTBWFSigDEkKk8f1bCH ArsE6ULySkbyCYkgghCuqoWQfeGGnrOnQ6nEZkPkRfha2JwrPnSkwzEr83EeccjXFG99 PqxC6UAvX1JbrTwEJ8jZywPedMoMgnZHW+xPoaNcLmOOjj2lH8Q6Jy77iAyKQsMCTT3F EUPA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=k5TWJEbnOa4d5XGxqWCqQIGLk27m8XCdjUeE9UlhQCQ=; b=XrOmNIJQmKVDYAD4bMLMkS5vgWF2hCLwK3PJ12HToTHMjhj1xYLO9s1+jGxzxzcwea 6z8vqwBTie7A7CLfTn1ED8Fx6ZS3oAUpsx+M8yxNvF+dQb7C3f/zpo5GRKX8BmJgORlv DE44suTS4TdkQ3D+qvQajyB780/P99DuzK8MZ5/R5S9KwQo7lL9hVCcP3/kT789pn/LC ADr8G5n6gjSMBBoAVYapry17vqL5LwZZGZnRXkPjkAdd3tVuml6ycaFiG+S+Ptbtrb7g VzhWfPgGMArnzTGbUn6syF3CxML5E6Jrarh/ar4u+NQ68Hb8K+hcg5mMd+1wHk9MGo3T pDHQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@ibm.com header.s=pp1 header.b=YqZt5WBy; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id he35-20020a1709073da300b007826e727aeasi3362117ejc.44.2022.10.25.05.01.42; Tue, 25 Oct 2022 05:02:13 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@ibm.com header.s=pp1 header.b=YqZt5WBy; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231288AbiJYL5s (ORCPT + 99 others); Tue, 25 Oct 2022 07:57:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38448 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231772AbiJYL5f (ORCPT ); Tue, 25 Oct 2022 07:57:35 -0400 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8A9C610EE65; Tue, 25 Oct 2022 04:57:30 -0700 (PDT) Received: from pps.filterd (m0098404.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 29PB7tYT030375; Tue, 25 Oct 2022 11:57:08 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=k5TWJEbnOa4d5XGxqWCqQIGLk27m8XCdjUeE9UlhQCQ=; b=YqZt5WByh/ZkjjbhsxfeRNMSLtddSCvQf6o4el7KbgJCDHAgzI24ua5CUcMeyBMI8wp1 2yPhnQB4bu+q5C2UC/S8/jaSmb60Ggvf1bxQ6tqQW1mUxLgnwh11lhAofIZy+ufXCMR/ xvMHo+a2LbjEWtYrRpjVB9iHcWMSL9LVXcXZnO9TeXLEIoMK7iQb4edP9FfLBACCjnN3 uW0PldRh/HiEaAIedNVGTyEnnbx9uyFsZbVFpWLTM5XefNUEorfdDqJIrxT/4llbZV8r A6sW1ftb5VMD9vgXhyqcX5DX07Q03+kr6vXJPXJCT33clanz/GJv1yx9fwnpw030ZBlT AQ== Received: from ppma06fra.de.ibm.com (48.49.7a9f.ip4.static.sl-reverse.com [159.122.73.72]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3kedu336ua-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 25 Oct 2022 11:57:07 +0000 Received: from pps.filterd (ppma06fra.de.ibm.com [127.0.0.1]) by ppma06fra.de.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 29PBoAn7027204; Tue, 25 Oct 2022 11:57:04 GMT Received: from b06avi18878370.portsmouth.uk.ibm.com (b06avi18878370.portsmouth.uk.ibm.com [9.149.26.194]) by ppma06fra.de.ibm.com with ESMTP id 3kc7sj40pv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 25 Oct 2022 11:57:04 +0000 Received: from d06av24.portsmouth.uk.ibm.com (d06av24.portsmouth.uk.ibm.com [9.149.105.60]) by b06avi18878370.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 29PBvZbd41550316 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 25 Oct 2022 11:57:35 GMT Received: from d06av24.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 66C9942047; Tue, 25 Oct 2022 11:57:01 +0000 (GMT) Received: from d06av24.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DAF5442041; Tue, 25 Oct 2022 11:57:00 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by d06av24.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 25 Oct 2022 11:57:00 +0000 (GMT) From: Niklas Schnelle To: Matthew Rosato , iommu@lists.linux.dev, Joerg Roedel , Will Deacon , Robin Murphy , Jason Gunthorpe Cc: Gerd Bayer , Pierre Morel , linux-s390@vger.kernel.org, borntraeger@linux.ibm.com, hca@linux.ibm.com, gor@linux.ibm.com, gerald.schaefer@linux.ibm.com, agordeev@linux.ibm.com, svens@linux.ibm.com, linux-kernel@vger.kernel.org, Wenjia Zhang , Julian Ruess Subject: [PATCH v8 6/6] iommu/s390: Implement map_pages()/unmap_pages() instead of map()/unmap() Date: Tue, 25 Oct 2022 13:56:57 +0200 Message-Id: <20221025115657.1666860-7-schnelle@linux.ibm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221025115657.1666860-1-schnelle@linux.ibm.com> References: <20221025115657.1666860-1-schnelle@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: SMs0Bc3OlKoJABiuRJjLU3w0MKYhvKLu X-Proofpoint-ORIG-GUID: SMs0Bc3OlKoJABiuRJjLU3w0MKYhvKLu X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-10-25_05,2022-10-25_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 clxscore=1015 priorityscore=1501 adultscore=0 spamscore=0 bulkscore=0 impostorscore=0 malwarescore=0 mlxscore=0 phishscore=0 mlxlogscore=999 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2209130000 definitions=main-2210250067 X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747660920153136188?= X-GMAIL-MSGID: =?utf-8?q?1747660920153136188?= While s390-iommu currently implements the map_page()/unmap_page() operations which only map/unmap a single page at a time the internal s390_iommu_update_trans() API already supports mapping/unmapping a range of pages at once. Take advantage of this by implementing the map_pages()/unmap_pages() operations instead thus allowing users of the IOMMU drivers to map multiple pages in a single call followed by a single I/O TLB flush if needed. Reviewed-by: Matthew Rosato Reviewed-by: Jason Gunthorpe Signed-off-by: Niklas Schnelle --- v7->v8: - Added Jason's R-b drivers/iommu/s390-iommu.c | 48 +++++++++++++++++++++++++------------- 1 file changed, 32 insertions(+), 16 deletions(-) diff --git a/drivers/iommu/s390-iommu.c b/drivers/iommu/s390-iommu.c index 104dfbec1037..7fb512bece9a 100644 --- a/drivers/iommu/s390-iommu.c +++ b/drivers/iommu/s390-iommu.c @@ -196,20 +196,15 @@ static void s390_iommu_release_device(struct device *dev) static int s390_iommu_update_trans(struct s390_domain *s390_domain, phys_addr_t pa, dma_addr_t dma_addr, - size_t size, int flags) + unsigned long nr_pages, int flags) { phys_addr_t page_addr = pa & PAGE_MASK; dma_addr_t start_dma_addr = dma_addr; - unsigned long irq_flags, nr_pages, i; + unsigned long irq_flags, i; struct zpci_dev *zdev; unsigned long *entry; int rc = 0; - if (dma_addr < s390_domain->domain.geometry.aperture_start || - (dma_addr + size - 1) > s390_domain->domain.geometry.aperture_end) - return -EINVAL; - - nr_pages = PAGE_ALIGN(size) >> PAGE_SHIFT; if (!nr_pages) return 0; @@ -252,11 +247,24 @@ static int s390_iommu_update_trans(struct s390_domain *s390_domain, return rc; } -static int s390_iommu_map(struct iommu_domain *domain, unsigned long iova, - phys_addr_t paddr, size_t size, int prot, gfp_t gfp) +static int s390_iommu_map_pages(struct iommu_domain *domain, + unsigned long iova, phys_addr_t paddr, + size_t pgsize, size_t pgcount, + int prot, gfp_t gfp, size_t *mapped) { struct s390_domain *s390_domain = to_s390_domain(domain); int flags = ZPCI_PTE_VALID, rc = 0; + size_t size = pgcount << __ffs(pgsize); + + if (pgsize != SZ_4K) + return -EINVAL; + + if (iova < s390_domain->domain.geometry.aperture_start || + (iova + size - 1) > s390_domain->domain.geometry.aperture_end) + return -EINVAL; + + if (!IS_ALIGNED(iova | paddr, pgsize)) + return -EINVAL; if (!(prot & IOMMU_READ)) return -EINVAL; @@ -265,7 +273,9 @@ static int s390_iommu_map(struct iommu_domain *domain, unsigned long iova, flags |= ZPCI_TABLE_PROTECTED; rc = s390_iommu_update_trans(s390_domain, paddr, iova, - size, flags); + pgcount, flags); + if (!rc) + *mapped = size; return rc; } @@ -301,21 +311,27 @@ static phys_addr_t s390_iommu_iova_to_phys(struct iommu_domain *domain, return phys; } -static size_t s390_iommu_unmap(struct iommu_domain *domain, - unsigned long iova, size_t size, - struct iommu_iotlb_gather *gather) +static size_t s390_iommu_unmap_pages(struct iommu_domain *domain, + unsigned long iova, + size_t pgsize, size_t pgcount, + struct iommu_iotlb_gather *gather) { struct s390_domain *s390_domain = to_s390_domain(domain); + size_t size = pgcount << __ffs(pgsize); int flags = ZPCI_PTE_INVALID; phys_addr_t paddr; int rc; + if (WARN_ON(iova < s390_domain->domain.geometry.aperture_start || + (iova + size - 1) > s390_domain->domain.geometry.aperture_end)) + return 0; + paddr = s390_iommu_iova_to_phys(domain, iova); if (!paddr) return 0; rc = s390_iommu_update_trans(s390_domain, paddr, iova, - size, flags); + pgcount, flags); if (rc) return 0; @@ -361,8 +377,8 @@ static const struct iommu_ops s390_iommu_ops = { .default_domain_ops = &(const struct iommu_domain_ops) { .attach_dev = s390_iommu_attach_device, .detach_dev = s390_iommu_detach_device, - .map = s390_iommu_map, - .unmap = s390_iommu_unmap, + .map_pages = s390_iommu_map_pages, + .unmap_pages = s390_iommu_unmap_pages, .iova_to_phys = s390_iommu_iova_to_phys, .free = s390_domain_free, }