From patchwork Mon Oct 17 12:45:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Schnelle X-Patchwork-Id: 3433 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4ac7:0:0:0:0:0 with SMTP id y7csp1430159wrs; Mon, 17 Oct 2022 05:56:06 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7/FVLEcTzzl1chD3FsWQHtKk5NFVcfq8NaWsyTB2iH7UgiPgSb4O4ALcuY0MvA0pKkZfZu X-Received: by 2002:a05:6402:5253:b0:45d:5914:245b with SMTP id t19-20020a056402525300b0045d5914245bmr9769941edd.227.1666011365976; Mon, 17 Oct 2022 05:56:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666011365; cv=none; d=google.com; s=arc-20160816; b=HSKzQQUd9qtY6SDaiEa3wDdx8uEDvIktcdbxFpwTVnFwR35dIBWmm9EpuocmtwPL1H UqWWZJd63x/IqXZ0PI2fHpGCb+R5AucCgwByOTWJUCc0CGEbvw5OcJSNeRK5ulOhwMgV KHSLxkLlqpl6kjpjnH6UbdJio8oNGyJiOWezVRlp1ZmtQYviYnzjPbsHHxSWKTeWhYu/ 5lCnh7r0Pt8jiKCQPVnwiZ2Zlhqovj2dJtTNDJlT8Y2S88HMklfX2THjgHKsjO6An7ck MG2PtXLFnh4VpHBW5wITnacJvwheHWOppLgRuDJmwhfqhXBIU3kba1/1etfKFTUvJEg8 ynPA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=fFOIzoJ3LaWmpMnXwnNfpLLraq/5CxuOkCXCe2n9UbU=; b=JQNCsLMSHy/tfy4llRf9TE7XGKU7grMdMd4uwyBVQHKclis+xD+BUviXIP3ufMZHh3 /r9P3YDvt6enJTQ4un7HN6NgVJMYPShLr1Oa5KXSmMEdFz7wzsfyQ2jQJfAQuV53K4Jd 1gNMHizd47X1s98bSEbRFquf+sqANgHGNnB3SB1u0KEbvRE6pdqbV4cIxGrNnPglkfy/ WUThUB264qOJmucGnj/djWq6Ir0IZOkvhyNhu/FVEDFoEHNGj/8FsQE1zyq0jdZerqt7 ckPkU/Qk5iX/AmulGvc2kfNB7NXC00ohHjlcLtUIGqyT9efI4hsEy/TFOwBdfrar5Acc R+3w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@ibm.com header.s=pp1 header.b=JTNIZMl0; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id jx23-20020a170906ca5700b0078d148daf4bsi7802626ejb.409.2022.10.17.05.55.34; Mon, 17 Oct 2022 05:56:05 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@ibm.com header.s=pp1 header.b=JTNIZMl0; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230465AbiJQMql (ORCPT + 99 others); Mon, 17 Oct 2022 08:46:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48478 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230338AbiJQMqX (ORCPT ); Mon, 17 Oct 2022 08:46:23 -0400 Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CCAE55508E; Mon, 17 Oct 2022 05:46:21 -0700 (PDT) Received: from pps.filterd (m0098419.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 29HB5qo3029181; Mon, 17 Oct 2022 12:46:05 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=fFOIzoJ3LaWmpMnXwnNfpLLraq/5CxuOkCXCe2n9UbU=; b=JTNIZMl079ByoHrayHdr/PoozGWgYW0mHuSFteNU2vKRGMAdm9dlHJUsPDNTNVIPRXB/ cG4loKYwPunTnutRqEpna51cFQxcA0NV1Ve0Li2/q78Wmr+v+E6kBjnwm9HhWpaDp98m 2s4cz3EDzlTeWpod9ILkerwwheJpOTavCXbX3RGJxwCc2pL9rGk333p1lrQ1YYrTeJWS xF8/fQSnUImFAg2fnZzn99FSvcsGWXdmTEYDomR2Rw996dhSvlxrVPw1q3v24YbM1UmA 61GIXvub83/j6hR2LcgNY3QPCF/dG/sn+ATxnBxHgR+aeO78o/0JLyqgoE8yydL9Oee6 CA== Received: from ppma03ams.nl.ibm.com (62.31.33a9.ip4.static.sl-reverse.com [169.51.49.98]) by mx0b-001b2d01.pphosted.com (PPS) with ESMTPS id 3k86g60x08-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 17 Oct 2022 12:46:05 +0000 Received: from pps.filterd (ppma03ams.nl.ibm.com [127.0.0.1]) by ppma03ams.nl.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 29HCZ6Z5024648; Mon, 17 Oct 2022 12:46:03 GMT Received: from b06cxnps4074.portsmouth.uk.ibm.com (d06relay11.portsmouth.uk.ibm.com [9.149.109.196]) by ppma03ams.nl.ibm.com with ESMTP id 3k7mg930m3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 17 Oct 2022 12:46:03 +0000 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps4074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 29HCk081262826 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 17 Oct 2022 12:46:00 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E573911C04A; Mon, 17 Oct 2022 12:45:59 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 745C311C058; Mon, 17 Oct 2022 12:45:59 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP; Mon, 17 Oct 2022 12:45:59 +0000 (GMT) From: Niklas Schnelle To: iommu@lists.linux.dev, Joerg Roedel , Will Deacon , Robin Murphy , Jason Gunthorpe Cc: Matthew Rosato , Gerd Bayer , Pierre Morel , linux-s390@vger.kernel.org, borntraeger@linux.ibm.com, hca@linux.ibm.com, gor@linux.ibm.com, gerald.schaefer@linux.ibm.com, agordeev@linux.ibm.com, svens@linux.ibm.com, linux-kernel@vger.kernel.org Subject: [PATCH v7 1/6] iommu/s390: Fix duplicate domain attachments Date: Mon, 17 Oct 2022 14:45:53 +0200 Message-Id: <20221017124558.1386337-2-schnelle@linux.ibm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221017124558.1386337-1-schnelle@linux.ibm.com> References: <20221017124558.1386337-1-schnelle@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: oZHmcjLCyp1UmvKL3uRkmVFckIUTDAuK X-Proofpoint-ORIG-GUID: oZHmcjLCyp1UmvKL3uRkmVFckIUTDAuK X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-10-17_09,2022-10-17_02,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 spamscore=0 lowpriorityscore=0 suspectscore=0 malwarescore=0 clxscore=1015 mlxlogscore=999 bulkscore=0 impostorscore=0 phishscore=0 mlxscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2209130000 definitions=main-2210170072 X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1746939534092630511?= X-GMAIL-MSGID: =?utf-8?q?1746939534092630511?= Since commit fa7e9ecc5e1c ("iommu/s390: Tolerate repeat attach_dev calls") we can end up with duplicates in the list of devices attached to a domain. This is inefficient and confusing since only one domain can actually be in control of the IOMMU translations for a device. Fix this by detaching the device from the previous domain, if any, on attach. Add a WARN_ON() in case we still have attached devices on freeing the domain. While here remove the re-attach on failure dance as it was determined to be unlikely to help and may confuse debug and recovery. Fixes: fa7e9ecc5e1c ("iommu/s390: Tolerate repeat attach_dev calls") Signed-off-by: Niklas Schnelle Reviewed-by: Matthew Rosato Reviewed-by: Jason Gunthorpe --- v6->v7: - Added Matt's R-b v5->v6: - Only set zdev->dma_table once zpci_register_ioat() succeeded (Matt) v4->v5: - Unregister IOAT and set zdev->dma_table on error (Matt) drivers/iommu/s390-iommu.c | 106 ++++++++++++++++--------------------- 1 file changed, 45 insertions(+), 61 deletions(-) diff --git a/drivers/iommu/s390-iommu.c b/drivers/iommu/s390-iommu.c index 3c071782f6f1..c2e5e81d609e 100644 --- a/drivers/iommu/s390-iommu.c +++ b/drivers/iommu/s390-iommu.c @@ -79,10 +79,36 @@ static void s390_domain_free(struct iommu_domain *domain) { struct s390_domain *s390_domain = to_s390_domain(domain); + WARN_ON(!list_empty(&s390_domain->devices)); dma_cleanup_tables(s390_domain->dma_table); kfree(s390_domain); } +static void __s390_iommu_detach_device(struct zpci_dev *zdev) +{ + struct s390_domain *s390_domain = zdev->s390_domain; + struct s390_domain_device *domain_device, *tmp; + unsigned long flags; + + if (!s390_domain) + return; + + spin_lock_irqsave(&s390_domain->list_lock, flags); + list_for_each_entry_safe(domain_device, tmp, &s390_domain->devices, + list) { + if (domain_device->zdev == zdev) { + list_del(&domain_device->list); + kfree(domain_device); + break; + } + } + spin_unlock_irqrestore(&s390_domain->list_lock, flags); + + zpci_unregister_ioat(zdev, 0); + zdev->s390_domain = NULL; + zdev->dma_table = NULL; +} + static int s390_iommu_attach_device(struct iommu_domain *domain, struct device *dev) { @@ -90,7 +116,7 @@ static int s390_iommu_attach_device(struct iommu_domain *domain, struct zpci_dev *zdev = to_zpci_dev(dev); struct s390_domain_device *domain_device; unsigned long flags; - int cc, rc; + int cc, rc = 0; if (!zdev) return -ENODEV; @@ -99,24 +125,18 @@ static int s390_iommu_attach_device(struct iommu_domain *domain, if (!domain_device) return -ENOMEM; - if (zdev->dma_table && !zdev->s390_domain) { - cc = zpci_dma_exit_device(zdev); - if (cc) { - rc = -EIO; - goto out_free; - } - } - if (zdev->s390_domain) - zpci_unregister_ioat(zdev, 0); + __s390_iommu_detach_device(zdev); + else if (zdev->dma_table) + zpci_dma_exit_device(zdev); - zdev->dma_table = s390_domain->dma_table; cc = zpci_register_ioat(zdev, 0, zdev->start_dma, zdev->end_dma, - virt_to_phys(zdev->dma_table)); + virt_to_phys(s390_domain->dma_table)); if (cc) { rc = -EIO; - goto out_restore; + goto out_free; } + zdev->dma_table = s390_domain->dma_table; spin_lock_irqsave(&s390_domain->list_lock, flags); /* First device defines the DMA range limits */ @@ -127,9 +147,9 @@ static int s390_iommu_attach_device(struct iommu_domain *domain, /* Allow only devices with identical DMA range limits */ } else if (domain->geometry.aperture_start != zdev->start_dma || domain->geometry.aperture_end != zdev->end_dma) { - rc = -EINVAL; spin_unlock_irqrestore(&s390_domain->list_lock, flags); - goto out_restore; + rc = -EINVAL; + goto out_unregister; } domain_device->zdev = zdev; zdev->s390_domain = s390_domain; @@ -138,14 +158,9 @@ static int s390_iommu_attach_device(struct iommu_domain *domain, return 0; -out_restore: - if (!zdev->s390_domain) { - zpci_dma_init_device(zdev); - } else { - zdev->dma_table = zdev->s390_domain->dma_table; - zpci_register_ioat(zdev, 0, zdev->start_dma, zdev->end_dma, - virt_to_phys(zdev->dma_table)); - } +out_unregister: + zpci_unregister_ioat(zdev, 0); + zdev->dma_table = NULL; out_free: kfree(domain_device); @@ -155,32 +170,12 @@ static int s390_iommu_attach_device(struct iommu_domain *domain, static void s390_iommu_detach_device(struct iommu_domain *domain, struct device *dev) { - struct s390_domain *s390_domain = to_s390_domain(domain); struct zpci_dev *zdev = to_zpci_dev(dev); - struct s390_domain_device *domain_device, *tmp; - unsigned long flags; - int found = 0; - if (!zdev) - return; + WARN_ON(zdev->s390_domain != to_s390_domain(domain)); - spin_lock_irqsave(&s390_domain->list_lock, flags); - list_for_each_entry_safe(domain_device, tmp, &s390_domain->devices, - list) { - if (domain_device->zdev == zdev) { - list_del(&domain_device->list); - kfree(domain_device); - found = 1; - break; - } - } - spin_unlock_irqrestore(&s390_domain->list_lock, flags); - - if (found && (zdev->s390_domain == s390_domain)) { - zdev->s390_domain = NULL; - zpci_unregister_ioat(zdev, 0); - zpci_dma_init_device(zdev); - } + __s390_iommu_detach_device(zdev); + zpci_dma_init_device(zdev); } static struct iommu_device *s390_iommu_probe_device(struct device *dev) @@ -198,24 +193,13 @@ static struct iommu_device *s390_iommu_probe_device(struct device *dev) static void s390_iommu_release_device(struct device *dev) { struct zpci_dev *zdev = to_zpci_dev(dev); - struct iommu_domain *domain; /* - * This is a workaround for a scenario where the IOMMU API common code - * "forgets" to call the detach_dev callback: After binding a device - * to vfio-pci and completing the VFIO_SET_IOMMU ioctl (which triggers - * the attach_dev), removing the device via - * "echo 1 > /sys/bus/pci/devices/.../remove" won't trigger detach_dev, - * only release_device will be called via the BUS_NOTIFY_REMOVED_DEVICE - * notifier. - * - * So let's call detach_dev from here if it hasn't been called before. + * release_device is expected to detach any domain currently attached + * to the device, but keep it attached to other devices in the group. */ - if (zdev && zdev->s390_domain) { - domain = iommu_get_domain_for_dev(dev); - if (domain) - s390_iommu_detach_device(domain, dev); - } + if (zdev) + __s390_iommu_detach_device(zdev); } static int s390_iommu_update_trans(struct s390_domain *s390_domain, From patchwork Mon Oct 17 12:45:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Schnelle X-Patchwork-Id: 3430 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4ac7:0:0:0:0:0 with SMTP id y7csp1426874wrs; Mon, 17 Oct 2022 05:47:30 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7hZz3LHPqDmuUjmO2tFgjusvVQoMT3K0tb30YUxYc0PMsowq4BsCPqcrKHtI0ky3gx0+GQ X-Received: by 2002:a05:6402:2146:b0:458:15d7:b99a with SMTP id bq6-20020a056402214600b0045815d7b99amr10217113edb.24.1666010850270; Mon, 17 Oct 2022 05:47:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666010850; cv=none; d=google.com; s=arc-20160816; b=DsKsocxyWSbACfT8JO+G48yeAZRGH/APvgnWorej5QWgaPH9rM4KkLV9CEiGegQ1ve /9y5mYypS7CawidmhxWU5ttDM2OVIvPu9RkuYQElFBvjq8JyYYxgI2471HQyTIhF4gG7 i/Ktvp4hbtHv8Pt9lC3VR8Fd3YxSrBbhA7Y9yKhd3+/YHvf8kPMIw3DmfRLvaHDsVJ9N WSnB4F1y0+BxMBCqJo8Bisp0COUWFgluXZ8FnTnwrfVz/nBJclU2DgIIaGQgeqXPqVY6 037t6M3a6b8IzaqgHuKKnO6Kotqedm+K5sDpU1j1p+Qvqk7wBzt1OF28DMbSaj7NsoUA EN1Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=NDfNIs/V5yOsnPnb7a+DInix+OEPhJc88G9erUaDEOU=; b=dLv2emEX1lfqDmcNgqPwjexuwzK8QP8ez8vf7HyvQrm02V/T6ghU3jVeNUVxSLegCM Z8CXfzWsdvxURn25JIwcDHpQcX3gYCRMsSlIuuYme7KPQLtK/eaIKZ5yZTzVXkHco87Y i+H3Ldn16FHYgcK+HI90Avwzbem34yClvU2MRwpUIiavUZGoY6c5eqOun6CqxpFSy6uw 9NILsoS5VGTtG6bP7yZ/gif/JGhQ4t3Gfgvhv4Lv00EkwuRwkvtISoWaB0EcfmP5bFCZ gP2NyZg6BWq/T7JvTY691hcFPxON9PsvsXCWR16LJguLKOpOyxqXYw2BQqV+D7BrxDfY fztQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@ibm.com header.s=pp1 header.b=UwPJ5Qa9; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id l15-20020a1709067d4f00b0078c73bd90e3si8763461ejp.371.2022.10.17.05.47.04; Mon, 17 Oct 2022 05:47:30 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@ibm.com header.s=pp1 header.b=UwPJ5Qa9; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230385AbiJQMq1 (ORCPT + 99 others); Mon, 17 Oct 2022 08:46:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48416 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230301AbiJQMqV (ORCPT ); Mon, 17 Oct 2022 08:46:21 -0400 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C5AEC4F181; Mon, 17 Oct 2022 05:46:19 -0700 (PDT) Received: from pps.filterd (m0098404.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 29HC57pw020059; Mon, 17 Oct 2022 12:46:07 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=NDfNIs/V5yOsnPnb7a+DInix+OEPhJc88G9erUaDEOU=; b=UwPJ5Qa9PFSwYmVbHGvvZ7y2Cm2rHAK/wj3/onW4uw+9ndlVwebdw2aB44FrKCyAfb2K XZzGVAWoT8fvkTY8V10dgzxpX1iLpifdu24BFRxNQU3wdeJgmjK4Eqto2kZ4fjOJjhQZ yUrDZmZfWDi01huNDDCs51El/35zjODh8haHTEU+IDZp4NYds+HLACTjtBVhYUN4Zcdq k4tFx5HENi8mBM+vIoHT4q+iGYNjgoZ2TTawfQpzmLHCtgXHicm51Ud7/+Q1nEOxD0LK 3PGXsZQd/pzFIeXm/AUOH9XzX0Jkt1bSny20BK88hhyHod4hIuK+1HCow19dD7Ka8CCd 2w== Received: from ppma04ams.nl.ibm.com (63.31.33a9.ip4.static.sl-reverse.com [169.51.49.99]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3k944bp15q-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 17 Oct 2022 12:46:06 +0000 Received: from pps.filterd (ppma04ams.nl.ibm.com [127.0.0.1]) by ppma04ams.nl.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 29HCZIVA023377; Mon, 17 Oct 2022 12:46:03 GMT Received: from b06cxnps4074.portsmouth.uk.ibm.com (d06relay11.portsmouth.uk.ibm.com [9.149.109.196]) by ppma04ams.nl.ibm.com with ESMTP id 3k7mg92yrv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 17 Oct 2022 12:46:03 +0000 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps4074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 29HCk0lV262832 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 17 Oct 2022 12:46:00 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6F18F11C05C; Mon, 17 Oct 2022 12:46:00 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id F197E11C050; Mon, 17 Oct 2022 12:45:59 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP; Mon, 17 Oct 2022 12:45:59 +0000 (GMT) From: Niklas Schnelle To: iommu@lists.linux.dev, Joerg Roedel , Will Deacon , Robin Murphy , Jason Gunthorpe Cc: Matthew Rosato , Gerd Bayer , Pierre Morel , linux-s390@vger.kernel.org, borntraeger@linux.ibm.com, hca@linux.ibm.com, gor@linux.ibm.com, gerald.schaefer@linux.ibm.com, agordeev@linux.ibm.com, svens@linux.ibm.com, linux-kernel@vger.kernel.org Subject: [PATCH v7 2/6] iommu/s390: Get rid of s390_domain_device Date: Mon, 17 Oct 2022 14:45:54 +0200 Message-Id: <20221017124558.1386337-3-schnelle@linux.ibm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221017124558.1386337-1-schnelle@linux.ibm.com> References: <20221017124558.1386337-1-schnelle@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: Z50lVSmMMxT5wGbXnOMrGlKvmqtpRcTI X-Proofpoint-ORIG-GUID: Z50lVSmMMxT5wGbXnOMrGlKvmqtpRcTI X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-10-17_09,2022-10-17_02,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 bulkscore=0 impostorscore=0 malwarescore=0 lowpriorityscore=0 mlxlogscore=999 spamscore=0 mlxscore=0 clxscore=1015 adultscore=0 phishscore=0 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2209130000 definitions=main-2210170072 X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1746938993043429172?= X-GMAIL-MSGID: =?utf-8?q?1746938993043429172?= The struct s390_domain_device serves the sole purpose as list entry for the devices list of a struct s390_domain. As it contains no additional information besides a list_head and a pointer to the struct zpci_dev we can simplify things and just thread the device list through struct zpci_dev directly. This removes the need to allocate during domain attach and gets rid of one level of indirection during mapping operations. Reviewed-by: Matthew Rosato Signed-off-by: Niklas Schnelle Reviewed-by: Jason Gunthorpe --- v5->v6: - On attach failure make sure the IOAT is not registered and zdev->dma_table == NULL - Dropped Jason's R-b arch/s390/include/asm/pci.h | 1 + drivers/iommu/s390-iommu.c | 37 +++++++------------------------------ 2 files changed, 8 insertions(+), 30 deletions(-) diff --git a/arch/s390/include/asm/pci.h b/arch/s390/include/asm/pci.h index 108e732d7b14..15f8714ca9b7 100644 --- a/arch/s390/include/asm/pci.h +++ b/arch/s390/include/asm/pci.h @@ -117,6 +117,7 @@ struct zpci_bus { struct zpci_dev { struct zpci_bus *zbus; struct list_head entry; /* list of all zpci_devices, needed for hotplug, etc. */ + struct list_head iommu_list; struct kref kref; struct hotplug_slot hotplug_slot; diff --git a/drivers/iommu/s390-iommu.c b/drivers/iommu/s390-iommu.c index c2e5e81d609e..af83ccde16a4 100644 --- a/drivers/iommu/s390-iommu.c +++ b/drivers/iommu/s390-iommu.c @@ -29,11 +29,6 @@ struct s390_domain { spinlock_t list_lock; }; -struct s390_domain_device { - struct list_head list; - struct zpci_dev *zdev; -}; - static struct s390_domain *to_s390_domain(struct iommu_domain *dom) { return container_of(dom, struct s390_domain, domain); @@ -87,21 +82,13 @@ static void s390_domain_free(struct iommu_domain *domain) static void __s390_iommu_detach_device(struct zpci_dev *zdev) { struct s390_domain *s390_domain = zdev->s390_domain; - struct s390_domain_device *domain_device, *tmp; unsigned long flags; if (!s390_domain) return; spin_lock_irqsave(&s390_domain->list_lock, flags); - list_for_each_entry_safe(domain_device, tmp, &s390_domain->devices, - list) { - if (domain_device->zdev == zdev) { - list_del(&domain_device->list); - kfree(domain_device); - break; - } - } + list_del_init(&zdev->iommu_list); spin_unlock_irqrestore(&s390_domain->list_lock, flags); zpci_unregister_ioat(zdev, 0); @@ -114,17 +101,12 @@ static int s390_iommu_attach_device(struct iommu_domain *domain, { struct s390_domain *s390_domain = to_s390_domain(domain); struct zpci_dev *zdev = to_zpci_dev(dev); - struct s390_domain_device *domain_device; unsigned long flags; int cc, rc = 0; if (!zdev) return -ENODEV; - domain_device = kzalloc(sizeof(*domain_device), GFP_KERNEL); - if (!domain_device) - return -ENOMEM; - if (zdev->s390_domain) __s390_iommu_detach_device(zdev); else if (zdev->dma_table) @@ -132,10 +114,8 @@ static int s390_iommu_attach_device(struct iommu_domain *domain, cc = zpci_register_ioat(zdev, 0, zdev->start_dma, zdev->end_dma, virt_to_phys(s390_domain->dma_table)); - if (cc) { - rc = -EIO; - goto out_free; - } + if (cc) + return -EIO; zdev->dma_table = s390_domain->dma_table; spin_lock_irqsave(&s390_domain->list_lock, flags); @@ -151,9 +131,8 @@ static int s390_iommu_attach_device(struct iommu_domain *domain, rc = -EINVAL; goto out_unregister; } - domain_device->zdev = zdev; zdev->s390_domain = s390_domain; - list_add(&domain_device->list, &s390_domain->devices); + list_add(&zdev->iommu_list, &s390_domain->devices); spin_unlock_irqrestore(&s390_domain->list_lock, flags); return 0; @@ -161,8 +140,6 @@ static int s390_iommu_attach_device(struct iommu_domain *domain, out_unregister: zpci_unregister_ioat(zdev, 0); zdev->dma_table = NULL; -out_free: - kfree(domain_device); return rc; } @@ -206,10 +183,10 @@ static int s390_iommu_update_trans(struct s390_domain *s390_domain, phys_addr_t pa, dma_addr_t dma_addr, size_t size, int flags) { - struct s390_domain_device *domain_device; phys_addr_t page_addr = pa & PAGE_MASK; dma_addr_t start_dma_addr = dma_addr; unsigned long irq_flags, nr_pages, i; + struct zpci_dev *zdev; unsigned long *entry; int rc = 0; @@ -234,8 +211,8 @@ static int s390_iommu_update_trans(struct s390_domain *s390_domain, } spin_lock(&s390_domain->list_lock); - list_for_each_entry(domain_device, &s390_domain->devices, list) { - rc = zpci_refresh_trans((u64) domain_device->zdev->fh << 32, + list_for_each_entry(zdev, &s390_domain->devices, iommu_list) { + rc = zpci_refresh_trans((u64)zdev->fh << 32, start_dma_addr, nr_pages * PAGE_SIZE); if (rc) break; From patchwork Mon Oct 17 12:45:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Schnelle X-Patchwork-Id: 3432 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4ac7:0:0:0:0:0 with SMTP id y7csp1427206wrs; Mon, 17 Oct 2022 05:48:20 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6UDShGR0Mx2X/uCIaHVp03zDo9yLCaE6WpNnA77RXUpH8nsuOiKR4kOB3sOTvgSAk0aPId X-Received: by 2002:a05:6402:159a:b0:458:d744:8975 with SMTP id c26-20020a056402159a00b00458d7448975mr10217743edv.200.1666010899621; Mon, 17 Oct 2022 05:48:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666010899; cv=none; d=google.com; s=arc-20160816; b=rRMqjA/jx1lLRaXLjIq1aamjtzZO8jkhLk+ZB1kC55u96ibr941EDPDrEQSTTfGybz aHmoQIt67ax2zhbQWZVAqi4itUHTiV9P+w+nQ3xQdja1jW06ZPMJeV+zeZR5R2s1j4zh G3Nglr7mnLfLvc5SuMS6G7IB7pMj75+GHjEJOcBkICAEAg0yU08N+xa5odkc9YoPCZ5f EcWVytjusrEFK5q19LKe66EqaQWK6mn5TQa7Rcf0FYY/EhVlhWlOzHIAPNmLjvheaY+S KruCGgtCPYJ+FfthIVkgn+FsmfiJv/83Ur1jsiI5drP3geLphGJS76OaP9ZolSOoOfrm GI5Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Wq1OLp/zBWDhX+pCc+Cbh57vb7Jl3/ifJLLtQSMTTHI=; b=huxPSlhsleKw3rySXfVbmeHyfUxUqp90cIvjATNTeuX32NO7/awfsvY8tA1kMVPDu2 OhZKqqJk3Y2Nk3QnHFsQG528SLws2fI4e4YuCGRd6/codSB0k+txws8CHSWprzOEAM7o 5bv/Ygf7Qg4NrywgsifHH/OdLzXt5FjQaZm1RRlOA8vNHAScFvEAVER7c/EWrSkW2JrF LN6dMZU/DbovulnuQlbzKVbX2XeSRj0RfiJ8uAovDm7wxtKR2oqSiCN6/tFNJdKlQwJ4 VYBroSz7ELmhEtpQ7YArLAEskkreq/nbnGC7fYiikUdoOzMDIFvszTKn3TbCh95zL2cj I5WA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@ibm.com header.s=pp1 header.b=oqa9aJnT; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id e3-20020a17090658c300b0073d5a794b43si9800985ejs.985.2022.10.17.05.47.53; Mon, 17 Oct 2022 05:48:19 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@ibm.com header.s=pp1 header.b=oqa9aJnT; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230399AbiJQMqb (ORCPT + 99 others); Mon, 17 Oct 2022 08:46:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48434 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230322AbiJQMqV (ORCPT ); Mon, 17 Oct 2022 08:46:21 -0400 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 51D985467F; Mon, 17 Oct 2022 05:46:20 -0700 (PDT) Received: from pps.filterd (m0098404.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 29HC57AT019931; Mon, 17 Oct 2022 12:46:07 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=Wq1OLp/zBWDhX+pCc+Cbh57vb7Jl3/ifJLLtQSMTTHI=; b=oqa9aJnTFiopeOov/C2H/wdDW+cWh5j6eMQHAOvjawNzrSqQJYCKIFaQB+g/RbcjDttI PBMx+HUn7rUwXdnc/SO5VLURqDW+LuzrL7HcLQDeMakqCXuvPvbMLwsZz7hgvooG49/j nK95EG4FwBlqjHnidS5u8jCVhBk0Qsv8G6d4CX0LXh/p9UoHpV9FNQF9rgxTUqUKf0JX 7Ys+/5qk8wmJgpQ48xgXNGz/XCtPnID4SovTA6v6ujhajDQ2rRUbHlK2E/uH1WdiEiwQ aJ/Y4MoBsaQs5OzH2xQvEpqNH8RUCswDVRBgGlmxW2tX5H/tlzIu9TCgTHC2UIDXBR4A dQ== Received: from ppma06ams.nl.ibm.com (66.31.33a9.ip4.static.sl-reverse.com [169.51.49.102]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3k944bp166-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 17 Oct 2022 12:46:06 +0000 Received: from pps.filterd (ppma06ams.nl.ibm.com [127.0.0.1]) by ppma06ams.nl.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 29HCb2Ox028155; Mon, 17 Oct 2022 12:46:04 GMT Received: from b06cxnps4075.portsmouth.uk.ibm.com (d06relay12.portsmouth.uk.ibm.com [9.149.109.197]) by ppma06ams.nl.ibm.com with ESMTP id 3k7m4jk0er-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 17 Oct 2022 12:46:04 +0000 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps4075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 29HCk1Pd21889556 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 17 Oct 2022 12:46:01 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id EC4F811C04A; Mon, 17 Oct 2022 12:46:00 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7B08711C064; Mon, 17 Oct 2022 12:46:00 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP; Mon, 17 Oct 2022 12:46:00 +0000 (GMT) From: Niklas Schnelle To: iommu@lists.linux.dev, Joerg Roedel , Will Deacon , Robin Murphy , Jason Gunthorpe Cc: Matthew Rosato , Gerd Bayer , Pierre Morel , linux-s390@vger.kernel.org, borntraeger@linux.ibm.com, hca@linux.ibm.com, gor@linux.ibm.com, gerald.schaefer@linux.ibm.com, agordeev@linux.ibm.com, svens@linux.ibm.com, linux-kernel@vger.kernel.org Subject: [PATCH v7 3/6] iommu/s390: Fix potential s390_domain aperture shrinking Date: Mon, 17 Oct 2022 14:45:55 +0200 Message-Id: <20221017124558.1386337-4-schnelle@linux.ibm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221017124558.1386337-1-schnelle@linux.ibm.com> References: <20221017124558.1386337-1-schnelle@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: 9tjkbeOPt9GoZcvP6Vf6Lz9wJG3Y__Sx X-Proofpoint-ORIG-GUID: 9tjkbeOPt9GoZcvP6Vf6Lz9wJG3Y__Sx X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-10-17_09,2022-10-17_02,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 bulkscore=0 impostorscore=0 malwarescore=0 lowpriorityscore=0 mlxlogscore=999 spamscore=0 mlxscore=0 clxscore=1015 adultscore=0 phishscore=0 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2209130000 definitions=main-2210170072 X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1746939044914012538?= X-GMAIL-MSGID: =?utf-8?q?1746939044914012538?= The s390 IOMMU driver currently sets the IOMMU domain's aperture to match the device specific DMA address range of the device that is first attached. This is not ideal. For one if the domain has no device attached in the meantime the aperture could be shrunk allowing translations outside the aperture to exist in the translation tables. Also this is a bit of a misuse of the aperture which really should describe what addresses can be translated and not some device specific limitations. Instead of misusing the aperture like this we can instead create reserved ranges for the ranges inaccessible to the attached devices allowing devices with overlapping ranges to still share an IOMMU domain. This also significantly simplifies s390_iommu_attach_device() allowing us to move the aperture check to the beginning of the function and removing the need to hold the device list's lock to check the aperture. As we then use the same aperture for all domains and it only depends on the table properties we can already check zdev->start_dma/end_dma at probe time and turn the check on attach into a WARN_ON(). Suggested-by: Jason Gunthorpe Reviewed-by: Matthew Rosato Signed-off-by: Niklas Schnelle Reviewed-by: Jason Gunthorpe --- v5->v6: - Return -EINVAL after WARN_ON() in attach v4->v5: - Make aperture check in attach a WARN_ON() and fail in probe if zdev->start_dma/end_dma doesn't git in aperture (Jason) drivers/iommu/s390-iommu.c | 63 ++++++++++++++++++++++++++------------ 1 file changed, 43 insertions(+), 20 deletions(-) diff --git a/drivers/iommu/s390-iommu.c b/drivers/iommu/s390-iommu.c index af83ccde16a4..c4203a37faa4 100644 --- a/drivers/iommu/s390-iommu.c +++ b/drivers/iommu/s390-iommu.c @@ -62,6 +62,9 @@ static struct iommu_domain *s390_domain_alloc(unsigned domain_type) kfree(s390_domain); return NULL; } + s390_domain->domain.geometry.force_aperture = true; + s390_domain->domain.geometry.aperture_start = 0; + s390_domain->domain.geometry.aperture_end = ZPCI_TABLE_SIZE_RT - 1; spin_lock_init(&s390_domain->dma_table_lock); spin_lock_init(&s390_domain->list_lock); @@ -102,11 +105,15 @@ static int s390_iommu_attach_device(struct iommu_domain *domain, struct s390_domain *s390_domain = to_s390_domain(domain); struct zpci_dev *zdev = to_zpci_dev(dev); unsigned long flags; - int cc, rc = 0; + int cc; if (!zdev) return -ENODEV; + if (WARN_ON(domain->geometry.aperture_start > zdev->end_dma || + domain->geometry.aperture_end < zdev->start_dma)) + return -EINVAL; + if (zdev->s390_domain) __s390_iommu_detach_device(zdev); else if (zdev->dma_table) @@ -118,30 +125,14 @@ static int s390_iommu_attach_device(struct iommu_domain *domain, return -EIO; zdev->dma_table = s390_domain->dma_table; - spin_lock_irqsave(&s390_domain->list_lock, flags); - /* First device defines the DMA range limits */ - if (list_empty(&s390_domain->devices)) { - domain->geometry.aperture_start = zdev->start_dma; - domain->geometry.aperture_end = zdev->end_dma; - domain->geometry.force_aperture = true; - /* Allow only devices with identical DMA range limits */ - } else if (domain->geometry.aperture_start != zdev->start_dma || - domain->geometry.aperture_end != zdev->end_dma) { - spin_unlock_irqrestore(&s390_domain->list_lock, flags); - rc = -EINVAL; - goto out_unregister; - } + zdev->dma_table = s390_domain->dma_table; zdev->s390_domain = s390_domain; + + spin_lock_irqsave(&s390_domain->list_lock, flags); list_add(&zdev->iommu_list, &s390_domain->devices); spin_unlock_irqrestore(&s390_domain->list_lock, flags); return 0; - -out_unregister: - zpci_unregister_ioat(zdev, 0); - zdev->dma_table = NULL; - - return rc; } static void s390_iommu_detach_device(struct iommu_domain *domain, @@ -155,6 +146,30 @@ static void s390_iommu_detach_device(struct iommu_domain *domain, zpci_dma_init_device(zdev); } +static void s390_iommu_get_resv_regions(struct device *dev, + struct list_head *list) +{ + struct zpci_dev *zdev = to_zpci_dev(dev); + struct iommu_resv_region *region; + + if (zdev->start_dma) { + region = iommu_alloc_resv_region(0, zdev->start_dma, 0, + IOMMU_RESV_RESERVED); + if (!region) + return; + list_add_tail(®ion->list, list); + } + + if (zdev->end_dma < ZPCI_TABLE_SIZE_RT - 1) { + region = iommu_alloc_resv_region(zdev->end_dma + 1, + ZPCI_TABLE_SIZE_RT - zdev->end_dma - 1, + 0, IOMMU_RESV_RESERVED); + if (!region) + return; + list_add_tail(®ion->list, list); + } +} + static struct iommu_device *s390_iommu_probe_device(struct device *dev) { struct zpci_dev *zdev; @@ -164,6 +179,13 @@ static struct iommu_device *s390_iommu_probe_device(struct device *dev) zdev = to_zpci_dev(dev); + if (zdev->start_dma > zdev->end_dma || + zdev->start_dma > ZPCI_TABLE_SIZE_RT - 1) + return ERR_PTR(-EINVAL); + + if (zdev->end_dma > ZPCI_TABLE_SIZE_RT - 1) + zdev->end_dma = ZPCI_TABLE_SIZE_RT - 1; + return &zdev->iommu_dev; } @@ -342,6 +364,7 @@ static const struct iommu_ops s390_iommu_ops = { .release_device = s390_iommu_release_device, .device_group = generic_device_group, .pgsize_bitmap = S390_IOMMU_PGSIZES, + .get_resv_regions = s390_iommu_get_resv_regions, .default_domain_ops = &(const struct iommu_domain_ops) { .attach_dev = s390_iommu_attach_device, .detach_dev = s390_iommu_detach_device, From patchwork Mon Oct 17 12:45:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Schnelle X-Patchwork-Id: 3429 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4ac7:0:0:0:0:0 with SMTP id y7csp1426758wrs; Mon, 17 Oct 2022 05:47:09 -0700 (PDT) X-Google-Smtp-Source: AMsMyM65sNmgBxAwpNrHK0qpU6vM83F3RoOK6Ih2TFzd+PNMjO81HItPFlWGh6RTSlcLuUK5pBw2 X-Received: by 2002:a17:906:c151:b0:78d:cdbc:9fb7 with SMTP id dp17-20020a170906c15100b0078dcdbc9fb7mr8082866ejc.688.1666010829485; Mon, 17 Oct 2022 05:47:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666010829; cv=none; d=google.com; s=arc-20160816; b=GqbwoZcGbBML+IkCQbhHSNi3Jon9KengbFTWKkGLP6xPB4fQm6MA96/CyAhvykelnC 9UlddnTTboYDJEtvZcDYNcsGdzt06zJ4zSlZa3wG4rXTlPar9+YL8SrS8qskz/v2Pgnl hXn8vtBU5vdskOdH9DVRAFcBmQkkgzqCXNp8xAQbkvp+ewfD9wKDXl0HvWGnebF1gpcP 9yC1BrDm3erHNVdTeFituNcCpiyaLW72hv0CJzDQy5CGZ6mgqfL50K7qad4ymIOtWWCK 1BkyOHFHPqNgTHaaZ7+MIWOCKMIwNsCJrNik7slsz+lZfqF3UBtRCYAznwVUed89Mhjt VYIA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=+Oxeb8UCfzjuNEnaMhEJFdLbzN16OMVcI7by7StU43g=; b=yDCoXJ0rkXy98HVmSmsLQRGiJUW/iaGEonaBI733uDe+gf3Ovk/UXDrrhU2sedeLZV kr7MpiZ1GFmpf8AlasZkvVykbL8j2FoOHtMUOl4XlkZr1KWEYFi/rMneggNai7IcDg2G n+VWg2WMgqCDtYlLffWJqP6kGH9MzNnaSs8Nx0uK7cMNApwCqh5lOzWtDbt2BNO2MTm1 +lg+6+yDSYxEQVjtfAp6zR6JnEujp+GMFYqTkvOMULn3xKov53skColK1RgXA1XVaJ4r OcUjMxZ9cDLUoyxxEWdIk5ZNsFwAhgWQG7Fh1KAezu4X3VzlVev1bOQhTSVDgqQjDnye G45Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@ibm.com header.s=pp1 header.b=jWLlaJXm; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id ho40-20020a1709070ea800b0078df1c345dasi9493534ejc.534.2022.10.17.05.46.42; Mon, 17 Oct 2022 05:47:09 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@ibm.com header.s=pp1 header.b=jWLlaJXm; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229765AbiJQMqT (ORCPT + 99 others); Mon, 17 Oct 2022 08:46:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48336 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229943AbiJQMqS (ORCPT ); Mon, 17 Oct 2022 08:46:18 -0400 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 35F6854C96; Mon, 17 Oct 2022 05:46:17 -0700 (PDT) Received: from pps.filterd (m0098399.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 29HCflLU026992; Mon, 17 Oct 2022 12:46:07 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=+Oxeb8UCfzjuNEnaMhEJFdLbzN16OMVcI7by7StU43g=; b=jWLlaJXmb5vLAlovyq8CIXKQ+trgrA4BhHFbQtVIdFts3K+6VXHEoQNcbUGyoo6u3o4b vZh8nUwdtBUBRVKPiRWZtxU75ajUAEZdR/o6srnXcB6I8ZI2Ql2s2q4gfQY/aX0mpiSe EW5mjJFhW1BMJrOm4XsmrVFeqgzhPCPsM524+pJmrrSOp+2YZl0R5qUmMTpYf61DbYcp XyZlfchdkyCdQPA1Z7tC9PChZr0F7nWuWWM+99iWw513dvcf5MQuA3h3vrYOtFwFIBLE yfvsSSfZgkosEMHGAAZrOrBIskuMkeunHvI3ASsEp09gvXiOQ4qsQ9+iGtbWMD0g5xMy +w== Received: from ppma02fra.de.ibm.com (47.49.7a9f.ip4.static.sl-reverse.com [159.122.73.71]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3k86sjs8e4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 17 Oct 2022 12:46:07 +0000 Received: from pps.filterd (ppma02fra.de.ibm.com [127.0.0.1]) by ppma02fra.de.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 29HCa9tO028076; Mon, 17 Oct 2022 12:46:04 GMT Received: from b06cxnps4075.portsmouth.uk.ibm.com (d06relay12.portsmouth.uk.ibm.com [9.149.109.197]) by ppma02fra.de.ibm.com with ESMTP id 3k7mg92954-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 17 Oct 2022 12:46:04 +0000 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps4075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 29HCk1174981284 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 17 Oct 2022 12:46:01 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7D49B11C050; Mon, 17 Oct 2022 12:46:01 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 03D3311C04C; Mon, 17 Oct 2022 12:46:01 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP; Mon, 17 Oct 2022 12:46:00 +0000 (GMT) From: Niklas Schnelle To: iommu@lists.linux.dev, Joerg Roedel , Will Deacon , Robin Murphy , Jason Gunthorpe Cc: Matthew Rosato , Gerd Bayer , Pierre Morel , linux-s390@vger.kernel.org, borntraeger@linux.ibm.com, hca@linux.ibm.com, gor@linux.ibm.com, gerald.schaefer@linux.ibm.com, agordeev@linux.ibm.com, svens@linux.ibm.com, linux-kernel@vger.kernel.org Subject: [PATCH v7 4/6] iommu/s390: Fix incorrect aperture check Date: Mon, 17 Oct 2022 14:45:56 +0200 Message-Id: <20221017124558.1386337-5-schnelle@linux.ibm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221017124558.1386337-1-schnelle@linux.ibm.com> References: <20221017124558.1386337-1-schnelle@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: 7wq5gA-XyXjMWuBa76emWmQ1L2cg_xe8 X-Proofpoint-GUID: 7wq5gA-XyXjMWuBa76emWmQ1L2cg_xe8 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-10-17_09,2022-10-17_02,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 adultscore=0 priorityscore=1501 phishscore=0 impostorscore=0 lowpriorityscore=0 malwarescore=0 mlxscore=0 spamscore=0 clxscore=1015 suspectscore=0 mlxlogscore=848 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2209130000 definitions=main-2210170072 X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1746938971664812453?= X-GMAIL-MSGID: =?utf-8?q?1746938971664812453?= The domain->geometry.aperture_end specifies the last valid address treat it as such when checking if a DMA address is valid. Reviewed-by: Pierre Morel Reviewed-by: Matthew Rosato Signed-off-by: Niklas Schnelle Reviewed-by: Jason Gunthorpe --- v3->v4: - Make check robust against overflow (Jason) drivers/iommu/s390-iommu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/iommu/s390-iommu.c b/drivers/iommu/s390-iommu.c index c4203a37faa4..fa8cb97bfc88 100644 --- a/drivers/iommu/s390-iommu.c +++ b/drivers/iommu/s390-iommu.c @@ -213,7 +213,7 @@ static int s390_iommu_update_trans(struct s390_domain *s390_domain, int rc = 0; if (dma_addr < s390_domain->domain.geometry.aperture_start || - dma_addr + size > s390_domain->domain.geometry.aperture_end) + (dma_addr + size - 1) > s390_domain->domain.geometry.aperture_end) return -EINVAL; nr_pages = PAGE_ALIGN(size) >> PAGE_SHIFT; From patchwork Mon Oct 17 12:45:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Schnelle X-Patchwork-Id: 3431 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4ac7:0:0:0:0:0 with SMTP id y7csp1426935wrs; Mon, 17 Oct 2022 05:47:39 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6KANS/XoaFLhZIM644QaQJJ9T4XzURucB7L5L3JQwPctX3Bn7EmW6r3p5lzdAzohtLzguS X-Received: by 2002:a05:6402:354d:b0:45c:b772:5ef4 with SMTP id f13-20020a056402354d00b0045cb7725ef4mr10306213edd.225.1666010859080; Mon, 17 Oct 2022 05:47:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666010859; cv=none; d=google.com; s=arc-20160816; b=eFstKm6fQ8o/Pk1vVhke5d50u32iLoi6XFC3Yldju0hb/W1vrsSMf89Nx1sEfjx6nt XCiRHSRNEHXpPDSjZqmhvl1a/l7PmWjE4RDVJKY1ZgCfQRCcYxBoWdEeAVbt2ykLJ6xZ 9rMq9R08NMl8Pw16FZUzDahDYRAQoro0SisYtMhvhpmKr+HdvdLjMcV/oEOVNIJ9tObz cc++fhGPMbPFSUW8OYhWvADMTcxBuIBV5TU0WIpyDngRi8FrB8ZlOsgqqHfxlZ9YfeDd HFF03nm5SO+UfKNGE0tip3bM8IFaXrWolPzaxQFj7+72/1ElzoRdTZM7ftcPcM0fgAtW gTQQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Sl3Pl+e312um5EfvhvLnKPcdCVoOuhPXamRqe7Sbe4c=; b=r62zRgd7EuQ8xnLahgievUR1mO2Dg49np2Py94mIo4V6fTJ0nL9P4HnImH01ip03Jt veYV5FellbpjD63dB/gBYOSKOhVYQtCBb3Uk3FCshh17tXHzIwjGtUESW/hAStWREfs+ yDlNZtTV8mEq6XWVE8eXuqB6p8Nwr/U6tWnfK8vurlVsgMH8Jmmo4E7igg2gqr+KdhPi Mn6B5vC6cY64oMsQiKODh1Cowt/5OAiqJvk1ZA5UM5RDtqUcYrM8xJplhPqqGvm/yTMA jhEkUB94r9PdxWAhZscjUHTO/NvOcMjcJqwxjR56Xk2gIiJQli+9wrCEAx4IZUeiAYan p6GQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@ibm.com header.s=pp1 header.b=eMroURnr; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id gt13-20020a1709072d8d00b0078d3b4510b5si9482180ejc.854.2022.10.17.05.47.12; Mon, 17 Oct 2022 05:47:39 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@ibm.com header.s=pp1 header.b=eMroURnr; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230325AbiJQMqV (ORCPT + 99 others); Mon, 17 Oct 2022 08:46:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48370 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230238AbiJQMqT (ORCPT ); Mon, 17 Oct 2022 08:46:19 -0400 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A175254C96; Mon, 17 Oct 2022 05:46:18 -0700 (PDT) Received: from pps.filterd (m0187473.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 29HBUJKp036486; Mon, 17 Oct 2022 12:46:08 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=Sl3Pl+e312um5EfvhvLnKPcdCVoOuhPXamRqe7Sbe4c=; b=eMroURnrnioBJBl+VVYFtVlvsF2RrHa+yOjtBFN6Ys5nht+KUr39jQ/CeHTPlNFUrMoU Hdigb9WCE9Y9rWhrc7FocNo4cQeqXt8aeF9tGe7nBmRRKM2tbVSCJK8mdfQ+DVCQxQeY sRjActuX9BDyruXoNSxYYBql0nNMEH81SVhFmoxuwcXyglcOExNL4vggCCJ+yU1oX5/x KdV3slb7BMG5zSEYCRi0OAhPRRvNmrQId/MZn4BI/9oCCjJCPx6M+jqFp1uvD6hBhwbO 4b3JGVPzNIcuKDv15lRAaCxYJuXFBR5Yy45+3B/LhCaZBiA5Q09AZDR4jKnkOo2vbl7d aQ== Received: from ppma02fra.de.ibm.com (47.49.7a9f.ip4.static.sl-reverse.com [159.122.73.71]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3k86ugh387-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 17 Oct 2022 12:46:07 +0000 Received: from pps.filterd (ppma02fra.de.ibm.com [127.0.0.1]) by ppma02fra.de.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 29HCZvNl028001; Mon, 17 Oct 2022 12:46:05 GMT Received: from b06cxnps4076.portsmouth.uk.ibm.com (d06relay13.portsmouth.uk.ibm.com [9.149.109.198]) by ppma02fra.de.ibm.com with ESMTP id 3k7mg92955-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 17 Oct 2022 12:46:05 +0000 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps4076.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 29HCk2Oo4260544 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 17 Oct 2022 12:46:02 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0477211C04A; Mon, 17 Oct 2022 12:46:02 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 88CDA11C054; Mon, 17 Oct 2022 12:46:01 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP; Mon, 17 Oct 2022 12:46:01 +0000 (GMT) From: Niklas Schnelle To: iommu@lists.linux.dev, Joerg Roedel , Will Deacon , Robin Murphy , Jason Gunthorpe Cc: Matthew Rosato , Gerd Bayer , Pierre Morel , linux-s390@vger.kernel.org, borntraeger@linux.ibm.com, hca@linux.ibm.com, gor@linux.ibm.com, gerald.schaefer@linux.ibm.com, agordeev@linux.ibm.com, svens@linux.ibm.com, linux-kernel@vger.kernel.org Subject: [PATCH v7 5/6] iommu/s390: Fix incorrect pgsize_bitmap Date: Mon, 17 Oct 2022 14:45:57 +0200 Message-Id: <20221017124558.1386337-6-schnelle@linux.ibm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221017124558.1386337-1-schnelle@linux.ibm.com> References: <20221017124558.1386337-1-schnelle@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: CCr_7XbiZUX6I-PFXIDZEMzR69iQm8C7 X-Proofpoint-ORIG-GUID: CCr_7XbiZUX6I-PFXIDZEMzR69iQm8C7 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-10-17_09,2022-10-17_02,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxlogscore=999 phishscore=0 bulkscore=0 lowpriorityscore=0 impostorscore=0 suspectscore=0 spamscore=0 malwarescore=0 priorityscore=1501 adultscore=0 mlxscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2209130000 definitions=main-2210170072 X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1746939002935935234?= X-GMAIL-MSGID: =?utf-8?q?1746939002935935234?= The .pgsize_bitmap property of struct iommu_ops is not a page mask but rather has a bit set for each size of pages the IOMMU supports. As the comment correctly pointed out at this moment the code only support 4K pages so simply use SZ_4K here. Reviewed-by: Matthew Rosato Reviewed-by: Jason Gunthorpe Signed-off-by: Niklas Schnelle --- drivers/iommu/s390-iommu.c | 9 +-------- 1 file changed, 1 insertion(+), 8 deletions(-) diff --git a/drivers/iommu/s390-iommu.c b/drivers/iommu/s390-iommu.c index fa8cb97bfc88..748e2e1b188a 100644 --- a/drivers/iommu/s390-iommu.c +++ b/drivers/iommu/s390-iommu.c @@ -12,13 +12,6 @@ #include #include -/* - * Physically contiguous memory regions can be mapped with 4 KiB alignment, - * we allow all page sizes that are an order of 4KiB (no special large page - * support so far). - */ -#define S390_IOMMU_PGSIZES (~0xFFFUL) - static const struct iommu_ops s390_iommu_ops; struct s390_domain { @@ -363,7 +356,7 @@ static const struct iommu_ops s390_iommu_ops = { .probe_device = s390_iommu_probe_device, .release_device = s390_iommu_release_device, .device_group = generic_device_group, - .pgsize_bitmap = S390_IOMMU_PGSIZES, + .pgsize_bitmap = SZ_4K, .get_resv_regions = s390_iommu_get_resv_regions, .default_domain_ops = &(const struct iommu_domain_ops) { .attach_dev = s390_iommu_attach_device, From patchwork Mon Oct 17 12:45:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Schnelle X-Patchwork-Id: 3435 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4ac7:0:0:0:0:0 with SMTP id y7csp1434799wrs; Mon, 17 Oct 2022 06:04:23 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6c12NlnaIJBaIv9y8jyFLNlIXXSgNDcf1mz1elVowP8GlIcAV3/pldjV3LihLklBtzxzzZ X-Received: by 2002:a63:455e:0:b0:439:9496:ddd8 with SMTP id u30-20020a63455e000000b004399496ddd8mr10586798pgk.261.1666011863447; Mon, 17 Oct 2022 06:04:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666011863; cv=none; d=google.com; s=arc-20160816; b=tMk3sdNYlwBkoRP83HxnQttr6afc3jyp03CGCHVK2lOIRPJnCNES1gXT6TRnxUx6Bz Ih2IqvPg9frjnEWbw4415zxHt0hxuq/GLdUkiyfL3cE1QTUQl/BdnPwscv8O6l4IBbsA m4uSa5V1ZZTGSdCr2BujA5teEm7YVcq8RP/GBE9zeRVE0PoYW77YkQa+aNvs+fBgOazi 4uVhpWc1fuRy9+0rNyLaKrcv31CTVi+tDIldSoXOMkydYgoFa/H72JmMn31P6cdjH2oZ SyNVc9AzD/yWJDEqgZok+yggezRaxhGACeuoYp1SBLspqbyZScm6rDoO+uMWO8GVx+2H 5bow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=1AodIQs5ECY1Bb78ho7XRkJc2DJzcyVFag1dTLCovlg=; b=wrDlGWf2Ex0ww+QoAkdRViXKtTr1r/4GQga6M/hoJHsVCH5dYX99Sd6hQUdBl+qQx1 Pg1Wk/zcN2iiLpBtRqCBZfSnrzYslHRTM6M07QWIHycZBR1a+KqhY9IOLBLPyiMlEZUh ggibkhpC0FQvHYSc/rvDM8uTO93sEYB7Evxo0h35IJmBb0X7N6J1kbilHjwSgCHurt/y fIusYqhfTw4wDZTd5cV7yrNKMpLRC9D2zHhBS3QKp9OiADnfwq9nnrJCm8EZgUgqGqPY 2MzF3Vvz9+aDzu5qP4hoX+M4HjNBrMCGQlKUJ1Y95DkbJtXTFXKvD1E4MqgEDBD6I3Mj Ej3g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@ibm.com header.s=pp1 header.b=lE+joaHx; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id u8-20020a170902bf4800b00174b83659fasi11022210pls.502.2022.10.17.06.03.55; Mon, 17 Oct 2022 06:04:23 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@ibm.com header.s=pp1 header.b=lE+joaHx; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230420AbiJQMqo (ORCPT + 99 others); Mon, 17 Oct 2022 08:46:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48516 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230313AbiJQMqY (ORCPT ); Mon, 17 Oct 2022 08:46:24 -0400 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C521754C96; Mon, 17 Oct 2022 05:46:22 -0700 (PDT) Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 29HBvuLp030938; Mon, 17 Oct 2022 12:46:09 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=1AodIQs5ECY1Bb78ho7XRkJc2DJzcyVFag1dTLCovlg=; b=lE+joaHxh8gS7IydGn6dewlyvS42ihBCjCF8L5No8Hyxdf1QXVSNf2+6/BTt0xmbnwOG GtM96uVlxj38e3ZZGKHdtw2kyKlx3RDX7n+3FcLyBILvDhzBM6hQ/XUU99JCm9Ub+XYO NCd0v8GNRKRMh5Z/Ib9BQ0IJv4CVZKSMq/Bxf5iuh/76Vkh1dEZaY2dFGOBUTb6TCR1U MWVkDqqogy3PGwx3C3nzikLQwwbouOf6gJ5RHnTIBzXaX7g4KRh3Fjfos5RK717DAon5 v1Sk0SV+1HzxGfMdW9IADlR6tObBZh7521kL1Wm4UO4cvOXUhaUxHp4viouyOz47Mj+3 Zw== Received: from ppma06fra.de.ibm.com (48.49.7a9f.ip4.static.sl-reverse.com [159.122.73.72]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3k865w1d8u-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 17 Oct 2022 12:46:08 +0000 Received: from pps.filterd (ppma06fra.de.ibm.com [127.0.0.1]) by ppma06fra.de.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 29HCZq8L026959; Mon, 17 Oct 2022 12:46:05 GMT Received: from b06cxnps4076.portsmouth.uk.ibm.com (d06relay13.portsmouth.uk.ibm.com [9.149.109.198]) by ppma06fra.de.ibm.com with ESMTP id 3k7m4jaad6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 17 Oct 2022 12:46:05 +0000 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps4076.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 29HCk2Ud4260548 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 17 Oct 2022 12:46:02 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 81EC711C04A; Mon, 17 Oct 2022 12:46:02 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1057111C04C; Mon, 17 Oct 2022 12:46:02 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP; Mon, 17 Oct 2022 12:46:02 +0000 (GMT) From: Niklas Schnelle To: iommu@lists.linux.dev, Joerg Roedel , Will Deacon , Robin Murphy , Jason Gunthorpe Cc: Matthew Rosato , Gerd Bayer , Pierre Morel , linux-s390@vger.kernel.org, borntraeger@linux.ibm.com, hca@linux.ibm.com, gor@linux.ibm.com, gerald.schaefer@linux.ibm.com, agordeev@linux.ibm.com, svens@linux.ibm.com, linux-kernel@vger.kernel.org Subject: [PATCH v7 6/6] iommu/s390: Implement map_pages()/unmap_pages() instead of map()/unmap() Date: Mon, 17 Oct 2022 14:45:58 +0200 Message-Id: <20221017124558.1386337-7-schnelle@linux.ibm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221017124558.1386337-1-schnelle@linux.ibm.com> References: <20221017124558.1386337-1-schnelle@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: Inrw7REhI73EcOI5lRzP6CGQQ6p7RgJ2 X-Proofpoint-ORIG-GUID: Inrw7REhI73EcOI5lRzP6CGQQ6p7RgJ2 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-10-17_09,2022-10-17_02,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxlogscore=913 malwarescore=0 spamscore=0 suspectscore=0 clxscore=1015 impostorscore=0 mlxscore=0 priorityscore=1501 bulkscore=0 adultscore=0 phishscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2209130000 definitions=main-2210170072 X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1746940055310403025?= X-GMAIL-MSGID: =?utf-8?q?1746940055310403025?= While s390-iommu currently implements the map_page()/unmap_page() operations which only map/unmap a single page at a time the internal s390_iommu_update_trans() API already supports mapping/unmapping a range of pages at once. Take advantage of this by implementing the map_pages()/unmap_pages() operations instead thus allowing users of the IOMMU drivers to map multiple pages in a single call followed by a single I/O TLB flush if needed. Reviewed-by: Matthew Rosato Signed-off-by: Niklas Schnelle Reviewed-by: Jason Gunthorpe --- drivers/iommu/s390-iommu.c | 48 +++++++++++++++++++++++++------------- 1 file changed, 32 insertions(+), 16 deletions(-) diff --git a/drivers/iommu/s390-iommu.c b/drivers/iommu/s390-iommu.c index 748e2e1b188a..6c407b61b25a 100644 --- a/drivers/iommu/s390-iommu.c +++ b/drivers/iommu/s390-iommu.c @@ -196,20 +196,15 @@ static void s390_iommu_release_device(struct device *dev) static int s390_iommu_update_trans(struct s390_domain *s390_domain, phys_addr_t pa, dma_addr_t dma_addr, - size_t size, int flags) + unsigned long nr_pages, int flags) { phys_addr_t page_addr = pa & PAGE_MASK; dma_addr_t start_dma_addr = dma_addr; - unsigned long irq_flags, nr_pages, i; + unsigned long irq_flags, i; struct zpci_dev *zdev; unsigned long *entry; int rc = 0; - if (dma_addr < s390_domain->domain.geometry.aperture_start || - (dma_addr + size - 1) > s390_domain->domain.geometry.aperture_end) - return -EINVAL; - - nr_pages = PAGE_ALIGN(size) >> PAGE_SHIFT; if (!nr_pages) return 0; @@ -252,11 +247,24 @@ static int s390_iommu_update_trans(struct s390_domain *s390_domain, return rc; } -static int s390_iommu_map(struct iommu_domain *domain, unsigned long iova, - phys_addr_t paddr, size_t size, int prot, gfp_t gfp) +static int s390_iommu_map_pages(struct iommu_domain *domain, + unsigned long iova, phys_addr_t paddr, + size_t pgsize, size_t pgcount, + int prot, gfp_t gfp, size_t *mapped) { struct s390_domain *s390_domain = to_s390_domain(domain); int flags = ZPCI_PTE_VALID, rc = 0; + size_t size = pgcount << __ffs(pgsize); + + if (pgsize != SZ_4K) + return -EINVAL; + + if (iova < s390_domain->domain.geometry.aperture_start || + (iova + size - 1) > s390_domain->domain.geometry.aperture_end) + return -EINVAL; + + if (!IS_ALIGNED(iova | paddr, pgsize)) + return -EINVAL; if (!(prot & IOMMU_READ)) return -EINVAL; @@ -265,7 +273,9 @@ static int s390_iommu_map(struct iommu_domain *domain, unsigned long iova, flags |= ZPCI_TABLE_PROTECTED; rc = s390_iommu_update_trans(s390_domain, paddr, iova, - size, flags); + pgcount, flags); + if (!rc) + *mapped = size; return rc; } @@ -301,21 +311,27 @@ static phys_addr_t s390_iommu_iova_to_phys(struct iommu_domain *domain, return phys; } -static size_t s390_iommu_unmap(struct iommu_domain *domain, - unsigned long iova, size_t size, - struct iommu_iotlb_gather *gather) +static size_t s390_iommu_unmap_pages(struct iommu_domain *domain, + unsigned long iova, + size_t pgsize, size_t pgcount, + struct iommu_iotlb_gather *gather) { struct s390_domain *s390_domain = to_s390_domain(domain); + size_t size = pgcount << __ffs(pgsize); int flags = ZPCI_PTE_INVALID; phys_addr_t paddr; int rc; + if (WARN_ON(iova < s390_domain->domain.geometry.aperture_start || + (iova + size - 1) > s390_domain->domain.geometry.aperture_end)) + return 0; + paddr = s390_iommu_iova_to_phys(domain, iova); if (!paddr) return 0; rc = s390_iommu_update_trans(s390_domain, paddr, iova, - size, flags); + pgcount, flags); if (rc) return 0; @@ -361,8 +377,8 @@ static const struct iommu_ops s390_iommu_ops = { .default_domain_ops = &(const struct iommu_domain_ops) { .attach_dev = s390_iommu_attach_device, .detach_dev = s390_iommu_detach_device, - .map = s390_iommu_map, - .unmap = s390_iommu_unmap, + .map_pages = s390_iommu_map_pages, + .unmap_pages = s390_iommu_unmap_pages, .iova_to_phys = s390_iommu_iova_to_phys, .free = s390_domain_free, }