From patchwork Thu Feb 2 12:40:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ganapatrao Kulkarni X-Patchwork-Id: 51900 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp214713wrn; Thu, 2 Feb 2023 04:44:40 -0800 (PST) X-Google-Smtp-Source: AK7set/AraO8/jFaIqA2Z659f/CuhZB2WfetwKdiCRM0bSGyJesX3nlG7KXAGxCFnLQCbBtljrem X-Received: by 2002:a17:902:d412:b0:196:3853:2926 with SMTP id b18-20020a170902d41200b0019638532926mr4853598ple.9.1675341880559; Thu, 02 Feb 2023 04:44:40 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1675341880; cv=pass; d=google.com; s=arc-20160816; b=oW0pqeIbrlAq2MDmRsfT7yyWn2IJ6FUzRMCnwtGr41xRqw5wm2XDUzPZnlIuvm6YWz 7oAKmarhf2CWucCKhQjBmTJS3H8YZZov1031i0vE6ZIjhZRRsXiFBN2NsxEVZ18XBnKE ZiiiVSq9vvOaZntQGBPURuk8Og5+7y4ZSWb1P0VwDOThOCzJs/q1gsrUXI7Q91S4P0fa fvFF8sL/bFZseIdEKd6rtA7lBtHTzuOh+JbaUWRol2UdDQZWt7w7l3FodiMWqlVmeZ6u 1gwNMAUCkjmUpba3VQjtvzrUV/9x5em0mrgv2sVeHdSOdgMjNs50QzLjdDB1wOielFaY 1Gmw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:content-transfer-encoding :message-id:date:subject:cc:to:from:dkim-signature; bh=g2Ux+NgFP24LT85zgffYOFdrl2w2hyjShQQDiQn3elE=; b=tZca5JgF8Bbxv5+PoA+a+HN0bE1okoRF66LaGRzeeuwYUlBJ6/Oh3NuuT+3VYE4aTw GPIAyrxxZHZ/hsS3TwM9cKn/DzhE0IR7qrC2hEVppZMGLcwLEL5cfv6y7n+R0g5Gn8p7 SlZ9DL1rRcrxYpQpf98O7G2CN0FW6eUm+89lz8rffCSDWhPFukIL3orM2EvAguOLxJGL VyDsoELTb66T3FQs52KIjLkw9MDDC/c+THC1GU/AU/8/NUdnLW2HbL4JgVqCMrTjjElD y024kUy+t1aEBxSUyl/gcDVS8HCUsMmLRO2vl2oCDFAw1lGMSp50WQimpC+TSByCKIyr LD/w== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@os.amperecomputing.com header.s=selector2 header.b=U5genDZY; arc=pass (i=1 spf=pass spfdomain=os.amperecomputing.com dkim=pass dkdomain=os.amperecomputing.com dmarc=pass fromdomain=os.amperecomputing.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amperecomputing.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id p21-20020a170902a41500b0018962933a3dsi339377plq.206.2023.02.02.04.44.28; Thu, 02 Feb 2023 04:44:40 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@os.amperecomputing.com header.s=selector2 header.b=U5genDZY; arc=pass (i=1 spf=pass spfdomain=os.amperecomputing.com dkim=pass dkdomain=os.amperecomputing.com dmarc=pass fromdomain=os.amperecomputing.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amperecomputing.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230047AbjBBMni (ORCPT + 99 others); Thu, 2 Feb 2023 07:43:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42298 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230496AbjBBMng (ORCPT ); Thu, 2 Feb 2023 07:43:36 -0500 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2124.outbound.protection.outlook.com [40.107.223.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6E6978D42A for ; Thu, 2 Feb 2023 04:42:53 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=N9x5AVPPk2pAC8TJfeYRaHpVL4GlyzRI1G/z2lBAF+ryznCzj/RKDzZ+9ZT1MWVwPkxqAXwXU7NNoM+6TkbXQuLg1PuyC0v91jQGGdW1GhQZvkCpbiKu7NU1K5Ws0JsTf4YTUIN6GGauoenYFI8GMaUehN+YcEABFVu9auoUp6l1bKsjYCrH7niKSnbn99ZnoPlxxb48eFWkEr7C+iCDb5d0QUReqWuNw2Ej32kSDlslBnWmmTNr0FHZhDDrg3ZDV12WkT+FI5uxc6JNzIU3CEHDncuDFtFRSWdX7+RWoX0BvpmtyWIzwuqaEe8ION9drIT/hAqSv1pR32E8750yfw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=g2Ux+NgFP24LT85zgffYOFdrl2w2hyjShQQDiQn3elE=; b=ZatOyWsUbuMRh4HP+OA3F8J86kYKJ1Ke5k3hEM8pzC1mVdtAoWJW63tQHQyXg8M0pFsHBYHwd4p/R+uPNKB9/sS++kBmXD4T6p3weTP98Mdci5ao0sbUk3qITx8U79c3HN2SC2urCbtXdU1K3Lee8sFJi6EbQNiKJi6IbH16huQprz2CanYwxE/+2aqZRaEobgPc/EM1k3SrQH+vVy+3i27GfAR5Wyl1HcxQPLvuS45W4N1tieZdIqYJTvBRe+kx9e6mXf8lHqaMdtPjUQQImNOQepnF9AWfx0ineeGB9qkHF4+QUGvDMeMTxXTHfhRa/lb3eS2NpEb5ZPAfEn/94Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=os.amperecomputing.com; dmarc=pass action=none header.from=os.amperecomputing.com; dkim=pass header.d=os.amperecomputing.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=os.amperecomputing.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=g2Ux+NgFP24LT85zgffYOFdrl2w2hyjShQQDiQn3elE=; b=U5genDZYLkOvFXXXTWfrrxAGZ2COrP2tlkJCVU0c/8/E/NWNqueatrpQZTGgrYKI19z8gnRJZQ6bleDNVmLttmIhqjjDvp7bXmYPGboSrS5Q5PpixXOtUyqyoR0IK6x2UzHglvkTSZrx02rDI+soLHPaSLpZQbNpflngj7SMuz8= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=os.amperecomputing.com; Received: from DM8PR01MB6824.prod.exchangelabs.com (2603:10b6:8:23::24) by BY3PR01MB6689.prod.exchangelabs.com (2603:10b6:a03:36d::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6064.24; Thu, 2 Feb 2023 12:41:14 +0000 Received: from DM8PR01MB6824.prod.exchangelabs.com ([fe80::6b5b:1242:818c:e70d]) by DM8PR01MB6824.prod.exchangelabs.com ([fe80::6b5b:1242:818c:e70d%3]) with mapi id 15.20.6064.027; Thu, 2 Feb 2023 12:41:14 +0000 From: Ganapatrao Kulkarni To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev, robin.murphy@arm.com, will@kernel.org, joro@8bytes.org Cc: jean-philippe@linaro.org, darren@os.amperecomputing.com, scott@os.amperecomputing.com, gankulkarni@os.amperecomputing.com Subject: [PATCH] iommu/arm-smmu-v3: Enable PCI ATS in passthrough mode as well Date: Thu, 2 Feb 2023 04:40:53 -0800 Message-Id: <20230202124053.848792-1-gankulkarni@os.amperecomputing.com> X-Mailer: git-send-email 2.38.1 X-ClientProxiedBy: CH0PR04CA0053.namprd04.prod.outlook.com (2603:10b6:610:77::28) To DM8PR01MB6824.prod.exchangelabs.com (2603:10b6:8:23::24) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM8PR01MB6824:EE_|BY3PR01MB6689:EE_ X-MS-Office365-Filtering-Correlation-Id: 25fbf559-9139-4a5f-0d91-08db051ac573 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 6vkBXr03tQvbDRLB3NH/fc7KqZVTgRTr07tY/I0JbrXgplS0xOjUX4yq+iwtQln+ZnVet2F3N5IjR4GMqhFSALhMJkXl3urbZQTMH2AjTLDh9V98lY+sMJ2yvNYxVicPuJZhZsFYBXbsTZPDjHPhHt09xmCidsMitzjn3FkPqI0zEFxl83BJiunWdEaO8nr+cjDPzEs6HbtN7vrDnbx9KNAvZ3QT27r7xZMi7uNej68QWpKd07vNX5DTvZOoWKbBioWVlSnMlMLMfEBsOeV3AHo8aB0GiVm2w9WumrGfQ+8b731DwU7LNH5oM2TMPdykC/8aCXIXPyp16YqyWosV3cenp+UhQDNKr8+sqx28C86dw+Lj/FrxgRDKkdb41fjvrM/3cAEe9JxSqeBJ+ummOtXU8hhW2Kl/WoLIYeISITJUBXRX+fEZKTRh0u49AvGpqRpAZVC8NGG7kFJUigF0F1g1AkXCg1XstLyQ0tJsiOX+FwL72gUgNYfvMeU4tyCV3DMGZKi+QAY37JZXU9afNc+PIcdVz/YzfvDNMIXI+gY9uwtXsciiFKqriJeJNFrYMCaB1vCZDjmiRO891LvvTlMrAw3nNYKPDVCjvcC0tqasIEwyqtYm1Uj6MKDOjrt87/2etVA+PLOHW0symbrSN3402S+0qoG917s2Mg4hp+S4eF6xf9f3/sJEqLKvBCef3olyVV43aFwKaL/kzn7QzQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM8PR01MB6824.prod.exchangelabs.com;PTR:;CAT:NONE;SFS:(13230025)(4636009)(346002)(376002)(396003)(366004)(136003)(39850400004)(451199018)(2906002)(6486002)(478600001)(8936002)(5660300002)(316002)(41300700001)(8676002)(66476007)(4326008)(66556008)(83380400001)(66946007)(86362001)(52116002)(1076003)(6506007)(38100700002)(2616005)(6512007)(186003)(38350700002)(26005)(107886003)(6666004);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: bSFKmPSomKCnMupTSDp2wAXqfilZG9xno+8lnvb4QaZhEeWvb1kYDwZrvXycPZxpL3FXyHreI5Gir7pK7CnH1J6RXptRly6DIYgRV7ZnKWX0Ny23hl3fgcnOlMNat55FMFhk8dbH1JpXrl7PXqoQ7M/aRUWf0Gpk7j/MRyYdv+Dm0Xz7bFheE86THXWagyptD1HYTr5qSebbTQSdPdAphz+mHFdkibtFxc7hL/zF/tH9zH4B+tJy64q6dCuIW9a0y6hU4N40V4OIJsCXbMdB6SnyTn0SKnv7t9xA+oqMU5iqGd9X6oC+P8aCxnwVT726VKpg4EdnhhLB77prehzgSt5iRj9PsIVsDpASnM9FcrWn/y1SGDBR28XWWGOEqpxHTFXJ7m9BBNJJH3MXR42K4kRZ25gcvHbZwoypwDk+rJIKilNWcb6gvtLDBuLKwHo9qySyf1Da9SHiWRvGz3S/HR84JsuVdnrNaCiBUBUrkxSnYXf4HH4wuBfxYHkmlAYOeZBccuvGAa7kNSd3HyzADlFXw7LjamtSBSxvJ6TA9QVCYGEr/a7lunHf+tbupApBJtZKcO6IV6kbJoe06mNgh1xHcGCTu5FT3JG8+OpnlPgJxq4k6WBIq0SNyPCjJ1lSRfpKKfkVtKqdcRdxmCQJ4+MONS6/7vx5AZhmHTf3YKlKxgdUTgCNHwk4Jzk6yMo2zGprD/zgvOnvRFWNlPeeqv0of76EVmguhGUA4PO20CWViRLiFQQ10ryLpgXy0orCQyRMeQWGuf9GBhoE3dMCaMb1X5Vd9+H8ruZSvGwN1TbvCUafVs3K6MW2H4/hnKWPNPoSJ12Qyi7INKbwqPiLRB8vEQcOJXJwnZug/63OKLIO32R8jGr819ernxU22v8cWxifNyc6LhE9BMan6M1DcuhbEkFyiC7pcb6hGA2xEHeOJJn8vclX8iTuNTTrOYTXZYp34kfQ4YO8ijFa8nrpC54IXPuvdimKJuh9ayfI//PxcDKKCPxsGAzDBXZlRyIrf/G6TEtK30TMpo4L0kioZ/3SvcCYi+kj9IKgrpiyO1rcH0I753Z4bVNQ4Pr6a1ZKoiZYUIdUyBk7BFjZcYRjL24nHBrzN+HUuhaUh8pyNuIbgfwO4LOtWbeFZTBCVK8uvbswZ81r2gB4HemKFzAi2yNMm5ZvHxULKvDJFyBf0HuNUm3IfdX6df7aEPtdpkDdinJfasyzNhT8sxf9DUVrBu4LWjra9Gpfw842Q4+QNrX5rs7wFVssX8xcIE3n+5IqUVb8j1dcqHJxOT8A/Q1yLm9N3f05IioMCqbdG0FU3mxXJXINv8REgUn9lb0s7bhSJfBuKq1gTjwmHuDyZyeXr+YvM0ZJ4hw7jKqfTbYVDYajRtZVVVt5sCBs4zdMrOKmK7r7T2dWh+DuM9DCaUl3QewLhQyoF+/JwQWZvP9+mtx5oAVKstfUjCAyb7kbq9o9KTlENVNVte+R2JIGJCuxeR1TpvKx4jNmkVaFJen140Ln0fVnezHx4iMIW5jqm8U91JhRJ4VarHqumzynWvGSUys53hGbrizIs55aNf1zT2iNyRlt6v3rPwaGuZ22Km/Oxem+/f/NU8zVKqXVhoOd1r/RPGQgwphajVWaBgKl2xA= X-OriginatorOrg: os.amperecomputing.com X-MS-Exchange-CrossTenant-Network-Message-Id: 25fbf559-9139-4a5f-0d91-08db051ac573 X-MS-Exchange-CrossTenant-AuthSource: DM8PR01MB6824.prod.exchangelabs.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Feb 2023 12:41:14.4870 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3bc2b170-fd94-476d-b0ce-4229bdc904a7 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 9obSNp/BVMPZW1zjQzkm+nJ4bb002nPbinRlOVkkYN6YBCah8O2h4fhbld23jL8yQznuD60orHDoGn0tjIAy30TiAU6SGidjv2TV+sGhiuhWG1pEKmLdDe/fF8dspAR2 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY3PR01MB6689 X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2, SPF_HELO_PASS,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756723287601902275?= X-GMAIL-MSGID: =?utf-8?q?1756723287601902275?= The current smmu-v3 driver does not enable PCI ATS for physical functions of ATS capable End Points when booted in smmu bypass mode (iommu.passthrough=1). This will not allow virtual functions to enable ATS(even though EP supports it) while they are attached to a VM using VFIO driver. This patch adds changes to enable ATS support for physical functions in passthrough/bypass mode as well. Also, adding check to avoid disabling of ATS if it is not enabled, to avoid unnecessary call-traces. Signed-off-by: Ganapatrao Kulkarni --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 6d5df91c5c46..5a605cb5ccef 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2313,11 +2313,16 @@ static void arm_smmu_enable_ats(struct arm_smmu_master *master) static void arm_smmu_disable_ats(struct arm_smmu_master *master) { struct arm_smmu_domain *smmu_domain = master->domain; + struct pci_dev *pdev; if (!master->ats_enabled) return; - pci_disable_ats(to_pci_dev(master->dev)); + pdev = to_pci_dev(master->dev); + + if (pdev->ats_enabled) + pci_disable_ats(pdev); + /* * Ensure ATS is disabled at the endpoint before we issue the * ATC invalidation via the SMMU. @@ -2453,8 +2458,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) master->domain = smmu_domain; - if (smmu_domain->stage != ARM_SMMU_DOMAIN_BYPASS) - master->ats_enabled = arm_smmu_ats_supported(master); + master->ats_enabled = arm_smmu_ats_supported(master); arm_smmu_install_ste_for_dev(master);