From patchwork Thu Aug 3 10:12:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 130505 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9f41:0:b0:3e4:2afc:c1 with SMTP id v1csp1087893vqx; Thu, 3 Aug 2023 04:45:16 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFhstu3NBTOCmXKgNdSd6CTNGn2SxXEqByp1Dg6ZH8SV54+b/Y9q3JCS1ssrh6gLgzZ0w9C X-Received: by 2002:a17:906:3095:b0:99c:7134:ab6d with SMTP id 21-20020a170906309500b0099c7134ab6dmr1161500ejv.41.1691063116577; Thu, 03 Aug 2023 04:45:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1691063116; cv=none; d=google.com; s=arc-20160816; b=ZHar3cyq00JxJTRoayOL0q39hY+PTGYcRbvejG6W8r5+kYlEiIzDZg6xASUBqlPoKC KdGn9LSaH3/l2+aRAqVuz+b06mn/IWEZjUb4cLFs22yqRxAHdSCuyChRRwmf9AqG/yLB azhnet0HxVnJAtUrYrEbJT9H6VBN6rCMeIemEDzw/i8XwjHrUw8IgQYQOJ7b2XVGewu9 AXWYz9V148xMxQUozgZdsvTCWFf4aZ2fHhnzeGn72rQIigZjpmGvc1pngBlGg/PNogRF AHWcLhNWYo0KGDron801gyysAfJjoMIjBFAoulwvjI2Bh4RJngwWIwHp9eq7hgNG+hYK thYQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=Y1wRhIRrggJAZYs6e7aV6RdIU6iSXzW0ONi/CWI4qUk=; fh=1E1ZQY+a9fAbTEzcn5ex4XiBDnDW+oWG5Bd6D1KRIpk=; b=Ns2lSAr5KV7lMrXKqHpv0ww4DRwUdkUizjQ6pO0h4MYeKSj3RRRCkoo0CPxXoRFFT1 RnyaNesCgExDViPsidTuJfIF3ev9ZGJaaGyc+aeV2fo+slXFAU/RXSLizTxtqHoHUPrH pmCY0PoP5OK1boUUhhdJ2Dap6plfwv0wvzb4zNiQMjHIAHgl60KDQWEVQ0SQ5DywgsRb Nfx5JXeAceo+ha2kSNJx01WD3a/VPRKGgHX26PSE05wqemYuekqotu0nV4spqqs3kTKa QWXoL/J+w7WCGUSzDxiXY7LBLefcZbuyw6qXBWHXi/gmcfmvSdHInwIZb9d0Ww+Zw1In WDpg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=v8M8Epvg; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id cf24-20020a170906b2d800b009877bb42154si6882089ejb.465.2023.08.03.04.44.51; Thu, 03 Aug 2023 04:45:16 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=v8M8Epvg; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235254AbjHCKON (ORCPT + 99 others); Thu, 3 Aug 2023 06:14:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51830 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235252AbjHCKOK (ORCPT ); Thu, 3 Aug 2023 06:14:10 -0400 Received: from mail-yw1-x1149.google.com (mail-yw1-x1149.google.com [IPv6:2607:f8b0:4864:20::1149]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 57C04272E for ; Thu, 3 Aug 2023 03:14:08 -0700 (PDT) Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-5840614b13cso13814577b3.0 for ; Thu, 03 Aug 2023 03:14:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1691057647; x=1691662447; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Y1wRhIRrggJAZYs6e7aV6RdIU6iSXzW0ONi/CWI4qUk=; b=v8M8EpvgykyvNj5+VFK4+5VS1GCx+4thN93X5qGdv2Q6mDg+Q9BprkKi/ZqVY1cMB5 hgughi4UN1aYu/EXNqY13NPNEQutZvvu2GYwDAvr8QOh2o+EQF3P9BYp4pftEx0ox7R7 H293Y2C2VTDWZWO6xMfqIKxlFoQitde1l6TRNNpWpcVfHJP20iQVEqHd2/LLHggfZnBa ZYsMt0hplLT0HqFF4jX+LVen+NA6yXSnT9/tIhVW1iyXGrZn3ULvFc83mH9XfV5ftRLp iOKnNntwlT0uTtypEVze4RGIkmvLieWvFRULXEHMTA02TJWP0zXal6LCAjTnDjAfnCr4 3weA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691057647; x=1691662447; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Y1wRhIRrggJAZYs6e7aV6RdIU6iSXzW0ONi/CWI4qUk=; b=IVTFdUOHs+O8L1KJaGQnOswhWScW4j+u8aHjX2ClOBPUPW+2Eu5+roOdKf1gQO53Bv eQDMmznvjvJzfy7cEW9Y1jmU2gYukZk1QPrc24uucrSAkBByZAchlhOCnEj2fl5QW/W/ XMmF10CnSbdO0DFsrxFjZq5bvyKJA0cHYQPSjduK+40PWxPmbgOrr+lucCDL9V0mtFYI D2mC8nrWhAB41415YMccg2+eDGJi/2Nd5up14nQS97j/bOewATLzI9o54MwQ2lTGpd8b 0Ff4B+gKLj0rZZJZmlsq8GmxsEtU864SViWL8K2nRtPv3Fx7Q1vcr/+uu3yE+u4a0vlR J3kA== X-Gm-Message-State: ABy/qLaJ0vr/5sYIYFwePexhynT3DsUJZF+Lj1TtkWK1L53wdFDYFW8q 5dRqWqLeLmq3RHo6WXDO7K4X+7HOhVsv X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:39d1:8774:b733:6210]) (user=mshavit job=sendgmr) by 2002:a81:b301:0:b0:573:54d8:e5be with SMTP id r1-20020a81b301000000b0057354d8e5bemr268745ywh.3.1691057647550; Thu, 03 Aug 2023 03:14:07 -0700 (PDT) Date: Thu, 3 Aug 2023 18:12:21 +0800 In-Reply-To: <20230803101351.1561031-1-mshavit@google.com> Mime-Version: 1.0 References: <20230803101351.1561031-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.585.gd2178a4bd4-goog Message-ID: <20230803181225.v5.1.I18d69f4908b8a9f801239117d2a6c3f5aa1e2e76@changeid> Subject: [PATCH v5 1/6] iommu/arm-smmu-v3: Simplify arm_smmu_enable_ats From: Michael Shavit To: iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: baolu.lu@linux.intel.com, will@kernel.org, jgg@nvidia.com, jean-philippe@linaro.org, robin.murphy@arm.com, nicolinc@nvidia.com, Michael Shavit X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE, USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1773208198347835425 X-GMAIL-MSGID: 1773208198347835425 arm_smmu_enable_ats's call to inv_domain would trigger an invalidation for all masters that a domain is attached to everytime it's attached to another ATS-enabled master. It doesn't seem like those invalidations are necessary, and it's easier to reason about arm_smmu_enable_ats if it only issues invalidation commands for the current master. Signed-off-by: Michael Shavit --- (no changes since v2) Changes in v2: - Fix commit message wrapping drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 61de66d17a5d5..4df335424b266 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2305,7 +2305,7 @@ static void arm_smmu_enable_ats(struct arm_smmu_master *master) pdev = to_pci_dev(master->dev); atomic_inc(&smmu_domain->nr_ats_masters); - arm_smmu_atc_inv_domain(smmu_domain, 0, 0, 0); + arm_smmu_atc_inv_master(master); if (pci_enable_ats(pdev, stu)) dev_err(master->dev, "Failed to enable ATS (STU %zu)\n", stu); } From patchwork Thu Aug 3 10:12:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 130470 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9f41:0:b0:3e4:2afc:c1 with SMTP id v1csp1066100vqx; Thu, 3 Aug 2023 04:05:37 -0700 (PDT) X-Google-Smtp-Source: APBJJlFZhijV6qDMdCvtK3ioO+MWJmzTpszLwvOrb2YdPZylW66Y7d1ihEYr+6n7TSKR/q1o3/iS X-Received: by 2002:a05:6a21:3b4a:b0:135:4527:efe4 with SMTP id zy10-20020a056a213b4a00b001354527efe4mr19647371pzb.10.1691060737464; Thu, 03 Aug 2023 04:05:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1691060737; cv=none; d=google.com; s=arc-20160816; b=ukCLcVcYcOS6FQd7jEZHAmwBS9iJgnxhkbt0aceKbibuQKKuGJrOrphCB4iVXznjTN BHR5U3TXez96WdM6WzRuBvXX6ya7sfcKMNcuBb4pyXbGreQ+bKPJSe30Ix4fZiAG0uL9 eTrJlTyYWX0mWuPoF/1Fpr+HdEcXxljAIWC2s7Ipk0RbuirHD1pkulN123BG5aSePokp 6X4IJ8K04aCCbRrEseRRgixuQrEt0tSGSyI3YDWlnNgR/07PrEctCwg5cdhB6rpywx+m WB3Z5LhslPEVTVh63w6vNPfvKtSWXbB8YTopcMzJeMBgj+XyK6gp3+MDZ9KJNEtZ7TVS L9jA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=IgE52kHcfRPOfED/lVwbKZpQuW1B4++XZftllscpWsM=; fh=1E1ZQY+a9fAbTEzcn5ex4XiBDnDW+oWG5Bd6D1KRIpk=; b=FHljXOZpSNSs2y5xhLkjaD/gfzYwOH7tCqQvZUpHeQw7CNdLXH8NZ+gPeODudMk3G+ zE5OXHYTyBstyCdimgwm67kasdFI6wJfLCkBotZIu+laM3urgpSNZQIyY5dUFpoXfUPr mZYRym9oEogmZgfNQO2KX9GVHdZ+1nk8Kw1wFIwUhyINLMnPXf4b6qthp9KWccKvxoih s8urTdQTXOoVP+gLMEB1Yi8xfTAGoPTwPYawaKrXqSBSmBjRNKRjVO2PfdrhZ32erndD Zem1a4pR+VFM65XboSFz7F0qWE5/6k+VLjZCN6iFRQpZbSc9Dxb6RgKU45n6a01cV6gK xqHQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=j2OVAQZZ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id cg24-20020a056a00291800b00674689bacacsi12294694pfb.77.2023.08.03.04.05.12; Thu, 03 Aug 2023 04:05:37 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=j2OVAQZZ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235283AbjHCKOX (ORCPT + 99 others); Thu, 3 Aug 2023 06:14:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51860 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235260AbjHCKOO (ORCPT ); Thu, 3 Aug 2023 06:14:14 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A4693211F for ; Thu, 3 Aug 2023 03:14:12 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-d087ffcc43cso828566276.3 for ; Thu, 03 Aug 2023 03:14:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1691057652; x=1691662452; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=IgE52kHcfRPOfED/lVwbKZpQuW1B4++XZftllscpWsM=; b=j2OVAQZZ7MDjPx3io3qsYNX350Y8kbR8uyyFo4vpsvU1CTrDPyy339XvDoTtkCfLKl hjuPhFUVdHds84l/I3DUxRLC11T1vIyAbXfh7nF0cLGUVaaLcssV1ZPLXJgRWTlUXiiS sAPOWKRpC4CWBpE5Tosmj7zGbyCzgLSkhayxDyzvH6arF2iY/dL3u2xXVqZfXf/BeTpc 3RHFbmb6uv9SkPCVhpZcis7e3ORDX//7PRXqKm2A1GBAsk3u9SrbHtdu+VCbDrNZeWpe JLQ2yz0xlOAwyFQGIXMYDDWqpsno2z24Vp47ND8adkdHa2UB5BBeMjjKzlqDfGOkFwSt Rsxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691057652; x=1691662452; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=IgE52kHcfRPOfED/lVwbKZpQuW1B4++XZftllscpWsM=; b=aFVRM9CuYOT0i0xH8Cjj5DocuB1T+eR/rMCnxRsvhpXXEVweEDPe5qxvg3NI8k7W/a 9C2TPr7tGUrxQ4KfSP8mg/mQOOzLvjprkBLg1VN4E/Bn42BxPa3Z2N+UZ8DDbVLVRGkp 8Jfui4mog6/KLEpVOQwiAV7ZGJ5XwaH3qalhFYZYsvXgy0NZiwre0rP3zDMAdgFyYuL6 LZva5icVFLfGaS7nl1WZDNM+PRXSw9FkP1VLjbfEX0buHZpfe6h89od+9TPHhTgQqC1Z /GLHC98ErJ3FIAc8xAOG8RiVHQ5mqBmRhxZI6Q3wjnxZOlmHhYH3GAnfk8dxzNqk3D16 uQSw== X-Gm-Message-State: ABy/qLY0M2OM18NHakaqs75qMgbQeBL6goyjM69+33EjQ8ZI65qfyNQu tGDloNVReIHtDMjfJQQx4D8CTUGJ78ur X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:39d1:8774:b733:6210]) (user=mshavit job=sendgmr) by 2002:a25:48a:0:b0:d37:353:b7eb with SMTP id 132-20020a25048a000000b00d370353b7ebmr74807ybe.11.1691057651922; Thu, 03 Aug 2023 03:14:11 -0700 (PDT) Date: Thu, 3 Aug 2023 18:12:22 +0800 In-Reply-To: <20230803101351.1561031-1-mshavit@google.com> Mime-Version: 1.0 References: <20230803101351.1561031-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.585.gd2178a4bd4-goog Message-ID: <20230803181225.v5.2.I8db07b9eaef3bd5ef9bfc5c8c6d44768a4d95293@changeid> Subject: [PATCH v5 2/6] iommu/arm-smmu-v3: Keep track of attached ssids From: Michael Shavit To: iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: baolu.lu@linux.intel.com, will@kernel.org, jgg@nvidia.com, jean-philippe@linaro.org, robin.murphy@arm.com, nicolinc@nvidia.com, Michael Shavit X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE, USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1773205703486199749 X-GMAIL-MSGID: 1773205703486199749 The arm-smmu-v3 driver keeps track of all masters that a domain is attached to so that it can re-write their STEs when the domain's ASID is upated by SVA. This tracking is also used to invalidate ATCs on all masters that a domain is attached to. This change introduces a new data structures to track all the CD entries that a domain is attached to. This change is a pre-requisite to allow domain attachment on non 0 SSIDs. Signed-off-by: Michael Shavit --- Changes in v5: - Renamed domain_head to list for consistency with other lists - Renamed attached_domains to attached_ssids to avoid confusion. This is a list of master/ssid pairs the domain is attached to, not a list of other domains. Changes in v4: - Remove reference to the master's domain accidentally re-introduced during a rebase. Make arm_smmu_atc_inv_domain static. Changes in v2: - Fix arm_smmu_atc_inv_cmd_set_ssid and other cosmetic changes .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 28 ++++-- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 99 +++++++++++-------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 22 ++++- 3 files changed, 98 insertions(+), 51 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index e3992a0c16377..d80c39e7e2fb5 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -37,21 +37,35 @@ struct arm_smmu_bond { static DEFINE_MUTEX(sva_lock); +/* + * When ssid is 0, update all the CD entries that this domain is attached to. + * When ssid is non-zero, write the CD into all the masters where this domain is + * the primary domain, with the provided SSID. This is used because SVA still + * piggybacks over the primary domain. + */ static int arm_smmu_write_ctx_desc_devices(struct arm_smmu_domain *smmu_domain, int ssid, struct arm_smmu_ctx_desc *cd) { + struct arm_smmu_attached_domain *attached_domain; struct arm_smmu_master *master; unsigned long flags; int ret; - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { - ret = arm_smmu_write_ctx_desc(master, ssid, cd); + spin_lock_irqsave(&smmu_domain->attached_ssids_lock, flags); + list_for_each_entry(attached_domain, &smmu_domain->attached_ssids, + list) { + master = attached_domain->master; + if (ssid && attached_domain->ssid == 0) { + ret = arm_smmu_write_ctx_desc(master, ssid, cd); + } else { + ret = arm_smmu_write_ctx_desc( + master, attached_domain->ssid, cd); + } if (ret) break; } - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_unlock_irqrestore(&smmu_domain->attached_ssids_lock, flags); return ret; } @@ -222,7 +236,7 @@ static void arm_smmu_mm_invalidate_range(struct mmu_notifier *mn, if (!(smmu_domain->smmu->features & ARM_SMMU_FEAT_BTM)) arm_smmu_tlb_inv_range_asid(start, size, smmu_mn->cd->asid, PAGE_SIZE, false, smmu_domain); - arm_smmu_atc_inv_domain(smmu_domain, mm->pasid, start, size); + arm_smmu_atc_inv_domain_ssid(smmu_domain, mm->pasid, start, size); } static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) @@ -243,7 +257,7 @@ static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) arm_smmu_write_ctx_desc_devices(smmu_domain, mm->pasid, &quiet_cd); arm_smmu_tlb_inv_asid(smmu_domain->smmu, smmu_mn->cd->asid); - arm_smmu_atc_inv_domain(smmu_domain, mm->pasid, 0, 0); + arm_smmu_atc_inv_domain_ssid(smmu_domain, mm->pasid, 0, 0); smmu_mn->cleared = true; mutex_unlock(&sva_lock); @@ -333,7 +347,7 @@ static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) */ if (!smmu_mn->cleared) { arm_smmu_tlb_inv_asid(smmu_domain->smmu, cd->asid); - arm_smmu_atc_inv_domain(smmu_domain, mm->pasid, 0, 0); + arm_smmu_atc_inv_domain_ssid(smmu_domain, mm->pasid, 0, 0); } /* Frees smmu_mn */ diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 4df335424b266..6e614ad12fb48 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1282,7 +1282,7 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, }; if (master) { - smmu_domain = master->domain; + smmu_domain = master->non_pasid_domain.domain; smmu = master->smmu; } @@ -1725,7 +1725,14 @@ static irqreturn_t arm_smmu_combined_irq_handler(int irq, void *dev) } static void -arm_smmu_atc_inv_to_cmd(int ssid, unsigned long iova, size_t size, +arm_smmu_atc_inv_cmd_set_ssid(int ssid, struct arm_smmu_cmdq_ent *cmd) +{ + cmd->substream_valid = !!ssid; + cmd->atc.ssid = ssid; +} + +static void +arm_smmu_atc_inv_to_cmd(unsigned long iova, size_t size, struct arm_smmu_cmdq_ent *cmd) { size_t log2_span; @@ -1750,8 +1757,8 @@ arm_smmu_atc_inv_to_cmd(int ssid, unsigned long iova, size_t size, */ *cmd = (struct arm_smmu_cmdq_ent) { .opcode = CMDQ_OP_ATC_INV, - .substream_valid = !!ssid, - .atc.ssid = ssid, + .substream_valid = false, + .atc.ssid = 0, }; if (!size) { @@ -1797,8 +1804,7 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) struct arm_smmu_cmdq_ent cmd; struct arm_smmu_cmdq_batch cmds; - arm_smmu_atc_inv_to_cmd(0, 0, 0, &cmd); - + arm_smmu_atc_inv_to_cmd(0, 0, &cmd); cmds.num = 0; for (i = 0; i < master->num_streams; i++) { cmd.atc.sid = master->streams[i].id; @@ -1808,13 +1814,19 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) return arm_smmu_cmdq_batch_submit(master->smmu, &cmds); } -int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, - unsigned long iova, size_t size) +/* + * If ssid is non-zero, issue atc invalidations with the given ssid instead of + * the one the domain is attached to. This is used by SVA since it's pasid + * attachments aren't recorded in smmu_domain yet. + */ +int arm_smmu_atc_inv_domain_ssid(struct arm_smmu_domain *smmu_domain, int ssid, + unsigned long iova, size_t size) { int i; unsigned long flags; struct arm_smmu_cmdq_ent cmd; struct arm_smmu_master *master; + struct arm_smmu_attached_domain *attached_domain; struct arm_smmu_cmdq_batch cmds; if (!(smmu_domain->smmu->features & ARM_SMMU_FEAT_ATS)) @@ -1837,25 +1849,37 @@ int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, if (!atomic_read(&smmu_domain->nr_ats_masters)) return 0; - arm_smmu_atc_inv_to_cmd(ssid, iova, size, &cmd); + arm_smmu_atc_inv_to_cmd(iova, size, &cmd); cmds.num = 0; - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { + spin_lock_irqsave(&smmu_domain->attached_ssids_lock, flags); + list_for_each_entry(attached_domain, &smmu_domain->attached_ssids, + list) { + master = attached_domain->master; if (!master->ats_enabled) continue; + if (ssid != 0) + arm_smmu_atc_inv_cmd_set_ssid(ssid, &cmd); + else + arm_smmu_atc_inv_cmd_set_ssid(attached_domain->ssid, &cmd); for (i = 0; i < master->num_streams; i++) { cmd.atc.sid = master->streams[i].id; arm_smmu_cmdq_batch_add(smmu_domain->smmu, &cmds, &cmd); } } - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_unlock_irqrestore(&smmu_domain->attached_ssids_lock, flags); return arm_smmu_cmdq_batch_submit(smmu_domain->smmu, &cmds); } +static int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, + unsigned long iova, size_t size) +{ + return arm_smmu_atc_inv_domain_ssid(smmu_domain, 0, iova, size); +} + /* IO_PGTABLE API */ static void arm_smmu_tlb_inv_context(void *cookie) { @@ -1877,7 +1901,7 @@ static void arm_smmu_tlb_inv_context(void *cookie) cmd.tlbi.vmid = smmu_domain->s2_cfg.vmid; arm_smmu_cmdq_issue_cmd_with_sync(smmu, &cmd); } - arm_smmu_atc_inv_domain(smmu_domain, 0, 0, 0); + arm_smmu_atc_inv_domain(smmu_domain, 0, 0); } static void __arm_smmu_tlb_inv_range(struct arm_smmu_cmdq_ent *cmd, @@ -1970,7 +1994,7 @@ static void arm_smmu_tlb_inv_range_domain(unsigned long iova, size_t size, * Unfortunately, this can't be leaf-only since we may have * zapped an entire table. */ - arm_smmu_atc_inv_domain(smmu_domain, 0, iova, size); + arm_smmu_atc_inv_domain(smmu_domain, iova, size); } void arm_smmu_tlb_inv_range_asid(unsigned long iova, size_t size, int asid, @@ -2050,8 +2074,8 @@ static struct iommu_domain *arm_smmu_domain_alloc(unsigned type) return NULL; mutex_init(&smmu_domain->init_mutex); - INIT_LIST_HEAD(&smmu_domain->devices); - spin_lock_init(&smmu_domain->devices_lock); + INIT_LIST_HEAD(&smmu_domain->attached_ssids); + spin_lock_init(&smmu_domain->attached_ssids_lock); INIT_LIST_HEAD(&smmu_domain->mmu_notifiers); return &smmu_domain->domain; @@ -2289,12 +2313,12 @@ static bool arm_smmu_ats_supported(struct arm_smmu_master *master) return dev_is_pci(dev) && pci_ats_supported(to_pci_dev(dev)); } -static void arm_smmu_enable_ats(struct arm_smmu_master *master) +static void arm_smmu_enable_ats(struct arm_smmu_master *master, + struct arm_smmu_domain *smmu_domain) { size_t stu; struct pci_dev *pdev; struct arm_smmu_device *smmu = master->smmu; - struct arm_smmu_domain *smmu_domain = master->domain; /* Don't enable ATS at the endpoint if it's not enabled in the STE */ if (!master->ats_enabled) @@ -2310,10 +2334,9 @@ static void arm_smmu_enable_ats(struct arm_smmu_master *master) dev_err(master->dev, "Failed to enable ATS (STU %zu)\n", stu); } -static void arm_smmu_disable_ats(struct arm_smmu_master *master) +static void arm_smmu_disable_ats(struct arm_smmu_master *master, + struct arm_smmu_domain *smmu_domain) { - struct arm_smmu_domain *smmu_domain = master->domain; - if (!master->ats_enabled) return; @@ -2377,19 +2400,19 @@ static void arm_smmu_disable_pasid(struct arm_smmu_master *master) static void arm_smmu_detach_dev(struct arm_smmu_master *master) { unsigned long flags; - struct arm_smmu_domain *smmu_domain = master->domain; + struct arm_smmu_domain *smmu_domain = master->non_pasid_domain.domain; if (!smmu_domain) return; - arm_smmu_disable_ats(master); + arm_smmu_disable_ats(master, smmu_domain); - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_del(&master->domain_head); - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_lock_irqsave(&smmu_domain->attached_ssids_lock, flags); + list_del(&master->non_pasid_domain.list); + spin_unlock_irqrestore(&smmu_domain->attached_ssids_lock, flags); - master->domain = NULL; master->ats_enabled = false; + master->non_pasid_domain.domain = NULL; arm_smmu_install_ste_for_dev(master); } @@ -2434,8 +2457,6 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) if (ret) return ret; - master->domain = smmu_domain; - /* * The SMMU does not support enabling ATS with bypass. When the STE is * in bypass (STE.Config[2:0] == 0b100), ATS Translation Requests and @@ -2449,26 +2470,26 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { if (!master->cd_table.cdtab) { ret = arm_smmu_alloc_cd_tables(master); - if (ret) { - master->domain = NULL; + if (ret) return ret; - } } ret = arm_smmu_write_ctx_desc(master, 0, &smmu_domain->cd); - if (ret) { - master->domain = NULL; + if (ret) return ret; - } } + master->non_pasid_domain.master = master; + master->non_pasid_domain.domain = smmu_domain; + master->non_pasid_domain.ssid = 0; arm_smmu_install_ste_for_dev(master); - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_add(&master->domain_head, &smmu_domain->devices); - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_lock_irqsave(&smmu_domain->attached_ssids_lock, flags); + list_add(&master->non_pasid_domain.list, + &smmu_domain->attached_ssids); + spin_unlock_irqrestore(&smmu_domain->attached_ssids_lock, flags); - arm_smmu_enable_ats(master); + arm_smmu_enable_ats(master, smmu_domain); return 0; } diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index e76452e735a04..66a492cafe2e8 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -689,11 +689,19 @@ struct arm_smmu_stream { struct rb_node node; }; +/* List of {masters, ssid} that a domain is attached to */ +struct arm_smmu_attached_domain { + struct list_head list; + struct arm_smmu_domain *domain; + struct arm_smmu_master *master; + int ssid; +}; + /* SMMU private data for each master */ struct arm_smmu_master { struct arm_smmu_device *smmu; struct device *dev; - struct arm_smmu_domain *domain; + struct arm_smmu_attached_domain non_pasid_domain; struct list_head domain_head; struct arm_smmu_stream *streams; /* Locked by the iommu core using the group mutex */ @@ -730,8 +738,12 @@ struct arm_smmu_domain { struct iommu_domain domain; - struct list_head devices; - spinlock_t devices_lock; + /* + * List of arm_smmu_attached_domain nodes used to track all the + * {master, ssid} pairs that this domain is attached to. + */ + struct list_head attached_ssids; + spinlock_t attached_ssids_lock; struct list_head mmu_notifiers; }; @@ -752,8 +764,8 @@ void arm_smmu_tlb_inv_range_asid(unsigned long iova, size_t size, int asid, size_t granule, bool leaf, struct arm_smmu_domain *smmu_domain); bool arm_smmu_free_asid(struct arm_smmu_ctx_desc *cd); -int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, - unsigned long iova, size_t size); +int arm_smmu_atc_inv_domain_ssid(struct arm_smmu_domain *smmu_domain, int ssid, + unsigned long iova, size_t size); #ifdef CONFIG_ARM_SMMU_V3_SVA bool arm_smmu_sva_supported(struct arm_smmu_device *smmu); From patchwork Thu Aug 3 10:12:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 130463 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9f41:0:b0:3e4:2afc:c1 with SMTP id v1csp1061273vqx; Thu, 3 Aug 2023 03:58:13 -0700 (PDT) X-Google-Smtp-Source: APBJJlGdXZC4C2TWx3Wv4GR1HcDK7C0fvoWCGcPzetQrvYedJ7aYs5bOMafQR79ASHq6k5gL8yCb X-Received: by 2002:aa7:dd15:0:b0:51d:95f2:ee76 with SMTP id i21-20020aa7dd15000000b0051d95f2ee76mr7111449edv.27.1691060293275; Thu, 03 Aug 2023 03:58:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1691060293; cv=none; d=google.com; s=arc-20160816; b=GO225sg7TRGuPxcGPyRm7ZLDZxVEHWlaCkkhgvNm80zb9e8zSWKF2zqP7oG91D4VCU 4G3h3Z9g2b/815AVHWWq0XuEp7dnVwYaBoz/l0hZR2QHXECbiyCSVQju+RkoRaSRrInj rQPgxKx8kOjkLuMm7ZxdDnIzjOeCwaujTOdgZbU39u6bT0gQd40dmGi2zPwynNuWjWnA p/t8GiCrHKSDJakUxQWKPpuAfXRKTQHg8qlvyNnJgVJV4OUWi9qD6r98WHClrIHne+Z4 zGM9iHb9K+do2foWG9uHKYnHh93G3h8rMOXi4Tbsnf4gC6GBvZ/DdJS6G52Avy8Rr5mS FTRw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=rOifn/Wuhm3Lbc3GJRKlq49fGVZtVjA5QOC6h6hHPVo=; fh=1E1ZQY+a9fAbTEzcn5ex4XiBDnDW+oWG5Bd6D1KRIpk=; b=YecUbH2qWTgJCC+R8QbNtij4mqG4LOliaWcJ6c0Vcqy8zj7OQzD+kbo72yqGpws6Ck QBHlx7vnTlXbjALwe9aFCMrtDo1aQTWIw1EogjQO6iaxtdhNATylUUhI71UKR7stkoZt IxSVvRd+TwKAyFTPBCbsnzJGvIOjVAALIPei09ZEEye/s82TrhWBQI2f1+asYf4ld3Ph WY9ALGiK99iFRtA1egkPuFHZi092b09rJLhpwEIoEluHweqOY1koIyJPAjfbae3XV1A/ 1xyQmFdIN1aipEo2yaNF89VBnShV7javImp7pEiUx4E7IX+GTfmJq6wujdKCWEBxn7Rn 32Yg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=zJO63gTN; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id w5-20020a056402128500b00522e601b5e1si3816564edv.212.2023.08.03.03.57.50; Thu, 03 Aug 2023 03:58:13 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=zJO63gTN; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235301AbjHCKOg (ORCPT + 99 others); Thu, 3 Aug 2023 06:14:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51998 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235280AbjHCKOU (ORCPT ); Thu, 3 Aug 2023 06:14:20 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8505335A4 for ; Thu, 3 Aug 2023 03:14:17 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-d061f324d64so888980276.1 for ; Thu, 03 Aug 2023 03:14:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1691057656; x=1691662456; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=rOifn/Wuhm3Lbc3GJRKlq49fGVZtVjA5QOC6h6hHPVo=; b=zJO63gTN+QV8Tfx+6gzEyj5dUonNQXVuyOR/4CqHTz6PD/3cyRUXHXVcphE0fpyuCR ZKz98+w5g8mFMMzzvIJ747TDQzawpo9yAa/1f6z+hOOvccv3tyru4rVG/SNKz9FZCl5s uIn2qnCLLDiJNac2CKJCfNSreSw5An3t/Kx+wsM8Pcw1M5SxyvLmfICMMkWalT4pT/fH IEFOJ549A2Nn1ee23hdlvSi7Y3WyIy4Tdve79X6p9c1NumDYR1CfcPF3sBfoSZXRe5Qr WrgHPCqRl2usHE3RPid/EudzlKgU3D4crbQvFz+8xt4Mn708pcq+X+TmyEB36fv7aHaX l0Og== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691057656; x=1691662456; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=rOifn/Wuhm3Lbc3GJRKlq49fGVZtVjA5QOC6h6hHPVo=; b=g+hHLUbla0g8bJWqaRSy1XLABRj9dIXznRoXlUCIMhHGoyq1HiEW6fQvbu1d32KmXK 1zCDIPpDGpH7psgoBE/9qK+GNrO8bkkC+K+gH3Yk+qI3zJvkJD5cZnPQuRphfUHtgQ3u qiv7UJnPE32RsLF7nV5deCeLw7LUYWWdZzteQ7NbvERHxdq7XubQoZQWNvCR63vPKCuK HSCWsGHFAWJgUwdMOTib4t47ieSTKXJu+GvPa0v0jXPGtrkammD0Ed60NenChlUXnhLh a9uiacnkb2Mx7IsjPQRMvrIA9/iOFt0NbManpAderGsq6fuLD5KiBEAA5cnCaSKrSOIu eyQg== X-Gm-Message-State: ABy/qLZf137JNbsIKYn36vtP8fHLaMYUOqE8gWPFDPaf05Fs61uY3653 v51i3w2D68n+3lR+Ha+3NIDlF4G0hij4 X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:39d1:8774:b733:6210]) (user=mshavit job=sendgmr) by 2002:a25:ab0b:0:b0:d0e:e780:81b3 with SMTP id u11-20020a25ab0b000000b00d0ee78081b3mr118271ybi.2.1691057656667; Thu, 03 Aug 2023 03:14:16 -0700 (PDT) Date: Thu, 3 Aug 2023 18:12:23 +0800 In-Reply-To: <20230803101351.1561031-1-mshavit@google.com> Mime-Version: 1.0 References: <20230803101351.1561031-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.585.gd2178a4bd4-goog Message-ID: <20230803181225.v5.3.I374b2826d8b01f3a0110fd42a01208d2e8adb0a2@changeid> Subject: [PATCH v5 3/6] iommu/arm-smmu-v3: Add helper for atc invalidation From: Michael Shavit To: iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: baolu.lu@linux.intel.com, will@kernel.org, jgg@nvidia.com, jean-philippe@linaro.org, robin.murphy@arm.com, nicolinc@nvidia.com, Michael Shavit X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE, USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1773205238440111160 X-GMAIL-MSGID: 1773205238440111160 This will be used to invalidate ATC entries made on an SSID for a master when detaching a domain with pasid. Signed-off-by: Michael Shavit --- (no changes since v2) Changes in v2: - Make use of arm_smmu_atc_inv_cmd_set_ssid drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 6e614ad12fb48..e0565c644ffdb 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1798,13 +1798,15 @@ arm_smmu_atc_inv_to_cmd(unsigned long iova, size_t size, cmd->atc.size = log2_span; } -static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) +static int arm_smmu_atc_inv_master_ssid(struct arm_smmu_master *master, + int ssid) { int i; struct arm_smmu_cmdq_ent cmd; struct arm_smmu_cmdq_batch cmds; arm_smmu_atc_inv_to_cmd(0, 0, &cmd); + arm_smmu_atc_inv_cmd_set_ssid(ssid, &cmd); cmds.num = 0; for (i = 0; i < master->num_streams; i++) { cmd.atc.sid = master->streams[i].id; @@ -1814,6 +1816,11 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) return arm_smmu_cmdq_batch_submit(master->smmu, &cmds); } +static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) +{ + return arm_smmu_atc_inv_master_ssid(master, 0); +} + /* * If ssid is non-zero, issue atc invalidations with the given ssid instead of * the one the domain is attached to. This is used by SVA since it's pasid From patchwork Thu Aug 3 10:12:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 130472 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9f41:0:b0:3e4:2afc:c1 with SMTP id v1csp1066240vqx; Thu, 3 Aug 2023 04:05:51 -0700 (PDT) X-Google-Smtp-Source: APBJJlEABhBwrVE9cS7lNtJBSBEYdzSJq0k6kDsFghLnpKkA8dkon1uRBQcHSg1O9H8E3x9eRQBp X-Received: by 2002:a05:6808:14c7:b0:3a7:1e3e:7f97 with SMTP id f7-20020a05680814c700b003a71e3e7f97mr17096753oiw.4.1691060751358; Thu, 03 Aug 2023 04:05:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1691060751; cv=none; d=google.com; s=arc-20160816; b=I0NRVukKNzVd54qZjxzfEf81ySuGQejBRwEvvQ1PKAxSSuVBUgwMzkYxU6Ze5slvzv nyxbpRA3f1RMBmkG9sMzYCOm+5Daixd2DGfaSeO2KVP/+22IxiZrqY4bPzNtZDnsI5W3 y8CY+3zLmxS4HSDzg88rhD8O370rNdnGyjNGcjH5zgtViMY7WjUwDF3DgP0FPUktn6eN wmZveDB5UzjS8ga44cYdvE1CPAXiSc2M28c5U+83NjJn+3vB7PC84OMwRhazq1VmrpUP Kz743g0zlM75JJh3HXMdyKeBujk4Hz/O7a8neU9Oh68Jd6EqtkYjdHPzTP6lpApkOIZp gPvA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=7z5OBENHOr2eyPDXE+tfWeU2HnsC9/l+M3HstrRsv94=; fh=1E1ZQY+a9fAbTEzcn5ex4XiBDnDW+oWG5Bd6D1KRIpk=; b=zQWI/8ArliSNEQziK1k3wu/U9tRWWzh2N5PEXoZavRaO9VyayLGMB55Kh2u7DcRGvl sOwA0VYfv6/NSXY3HQ/+gRZknGKLyDWIabzZx/sxECsZ/mj+30UImmJRA949pY/LiLeq 8n3KGZJvU5+fi1mTXBvLvRWekJIhy2SE5Uh0DsESoVHAGKFtNncVMIlfudEs3SzmLOT8 yrCC6ktLIV9C724/Mvhvl9tIbKRAH2s518GmSro7SLyw7XprQ8DlX6xcpv8PV6qIu5Hp svuWbrRKKrda1yoc9+itmTfyW0Qk3e8L7+wfeF9r4eSu9RxkORdBlYov7yxojMzG+DW8 7afA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=4QHFgQUz; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id s25-20020a639259000000b00563f317e54fsi678314pgn.284.2023.08.03.04.05.27; Thu, 03 Aug 2023 04:05:51 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=4QHFgQUz; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235342AbjHCKOt (ORCPT + 99 others); Thu, 3 Aug 2023 06:14:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52288 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235295AbjHCKOe (ORCPT ); Thu, 3 Aug 2023 06:14:34 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4278E3A8D for ; Thu, 3 Aug 2023 03:14:21 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-d052f49702dso845409276.3 for ; Thu, 03 Aug 2023 03:14:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1691057661; x=1691662461; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=7z5OBENHOr2eyPDXE+tfWeU2HnsC9/l+M3HstrRsv94=; b=4QHFgQUz86psObn8rqXE96jintVeFyZR35xzg9vOmflXpYO0PFbxuqsmlweMhUyr32 vvFq/X22VGeQUffOj2LowuNN14kNeh1exPxyIHK6iorLiAJaUvsTfmR54Ja3uWSYoj+K wHwk8QvAu93nEOl9iwvn2H9dvkc74xR7xbg3RtyEn4XCPooe2xQxuXh0RSIscAWlq0Sl IzhGppqwyVK3eZFtOSbFmx/hgQL7NAhApxNRK/g5uJhvrqX1D2Mskmt507nzwfbxGjuP AnsyYIfVraineTEMavWQs8qcmr1vAiZpzI1xLzKZeB0QitKz0Uqxb1nB+OxX2ujF9nFv NtgQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691057661; x=1691662461; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=7z5OBENHOr2eyPDXE+tfWeU2HnsC9/l+M3HstrRsv94=; b=WfIOoPlqYJqvYoPw2LaKaqrmlHdPk1RL2LUOkDiaVFwx6qwV2PD07m0h/ZEaaq7GXZ 8ds7MjFLHVvO5+Q1gb5biYiOuczvBZhiMl+Ib0ap0uflg8ZOfoDk6KppxhNYCznx/jUg eXz7Qe/qGUMGWCrMvuWXvsYyq1C6ml9UhI885/Ns1c+31RiNSBKXSV9P7uAyhsQV3qHR mkkMPNczxSOJBDCiweqmBhG4ZB8Jkr1ujgAg8Zw+ZdrLGa3oAifZxOOAQflMlkqhLdPt Aad+5Vk1d989Q2ypm3C1q1ilUfa60ETqLCAHIElX68sBcW2DGtRrCNYzhnuu8ih8ktoh 505w== X-Gm-Message-State: ABy/qLZLBjS0cviryovK+VtdvubWGPDyzoxFSQLtGlCFtoo6RB3SdGbP 4OWRnlAgMM+gR70H5yTqkG3mMN05km44 X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:39d1:8774:b733:6210]) (user=mshavit job=sendgmr) by 2002:a25:db82:0:b0:d0f:15a4:5a53 with SMTP id g124-20020a25db82000000b00d0f15a45a53mr121006ybf.2.1691057661160; Thu, 03 Aug 2023 03:14:21 -0700 (PDT) Date: Thu, 3 Aug 2023 18:12:24 +0800 In-Reply-To: <20230803101351.1561031-1-mshavit@google.com> Mime-Version: 1.0 References: <20230803101351.1561031-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.585.gd2178a4bd4-goog Message-ID: <20230803181225.v5.4.I4ba46c0f7d599f43094d6ba1113c0b4fe49bd908@changeid> Subject: [PATCH v5 4/6] iommu/arm-smmu-v3: Implement set_dev_pasid From: Michael Shavit To: iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: baolu.lu@linux.intel.com, will@kernel.org, jgg@nvidia.com, jean-philippe@linaro.org, robin.murphy@arm.com, nicolinc@nvidia.com, Michael Shavit X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE, USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1773205718079915793 X-GMAIL-MSGID: 1773205718079915793 This change enables the use of the iommu_attach_dev_pasid API for UNMANAGED domains. The primary use-case is to allow in-kernel users of the iommu API to manage domains with PASID. This change also allows for future support of pasid in the DMA api. Signed-off-by: Michael Shavit --- Changes in v5: - Fix missing error value return in set_dev_pasid - Fix issue where nr_attached_pasid_domains isn't updated when arm_smmu_write_ctx_desc fails - Fix missing free of the attached_domain node - Split off the CD table refactor to separate patch series: https://lore.kernel.org/all/20230802163328.2623773-1-mshavit@google.com/ - Link to v4: https://lore.kernel.org/all/20230621063825.268890-1-mshavit@google.com/ - Remove districting change where a NULL master is passed to arm_smmu_prepare_domain_for_smmu Changes in v4: - Fix build warning and error on patch 07. The error was introduced during a v1->v2 rebase and hidden by patch 09 which removed the offending line. - Link to v3: https://lore.kernel.org/all/20230614154304.2860121-1-mshavit@google.com/ Changes in v3: - Dropped the bulk of the SVA refactoring to re-work as a follow-up series. - Reworded cover letter to omit dropped changes. - Rebased on 6.4 tip - Link to v2: https://lore.kernel.org/all/20230606120854.4170244-1-mshavit@google.com/ Changes in v2: - Reworded cover letter and commits based on v1 feedback. - Split and reworked `iommu/arm-smmu-v3: Move cdtable to arm_smmu_master` - Added SVA clean-up and refactor. - A few other small bug fixes and cosmetics. - Link to v1: https://lore.kernel.org/all/20230510205054.2667898-1-mshavit@google.com/ - Add missing atc invalidation when detaching with pasid drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 156 ++++++++++++++++++-- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 1 + 2 files changed, 141 insertions(+), 16 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index e0565c644ffdb..7b296458dafec 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2388,6 +2388,11 @@ static int arm_smmu_enable_pasid(struct arm_smmu_master *master) return 0; } +static bool arm_smmu_master_has_pasid_domains(struct arm_smmu_master *master) +{ + return master->nr_attached_pasid_domains > 0; +} + static void arm_smmu_disable_pasid(struct arm_smmu_master *master) { struct pci_dev *pdev; @@ -2423,6 +2428,25 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) arm_smmu_install_ste_for_dev(master); } +static int arm_smmu_prepare_domain_for_smmu(struct arm_smmu_device *smmu, + struct arm_smmu_domain *smmu_domain, + struct arm_smmu_master *master) +{ + int ret = 0; + + mutex_lock(&smmu_domain->init_mutex); + if (!smmu_domain->smmu) { + smmu_domain->smmu = smmu; + ret = arm_smmu_domain_finalise(&smmu_domain->domain, master); + if (ret) + smmu_domain->smmu = NULL; + } else if (smmu_domain->smmu != smmu) + ret = -EINVAL; + + mutex_unlock(&smmu_domain->init_mutex); + return ret; +} + static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) { int ret = 0; @@ -2438,6 +2462,10 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) master = dev_iommu_priv_get(dev); smmu = master->smmu; + ret = arm_smmu_prepare_domain_for_smmu(smmu, smmu_domain, master); + if (ret) + return ret; + /* * Checking that SVA is disabled ensures that this device isn't bound to * any mm, and can be safely detached from its old domain. Bonds cannot @@ -2448,21 +2476,17 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) return -EBUSY; } - arm_smmu_detach_dev(master); - - mutex_lock(&smmu_domain->init_mutex); - - if (!smmu_domain->smmu) { - smmu_domain->smmu = smmu; - ret = arm_smmu_domain_finalise(domain, master); - if (ret) - smmu_domain->smmu = NULL; - } else if (smmu_domain->smmu != smmu) - ret = -EINVAL; + /* + * Attaching a bypass or stage 2 domain would break any domains attached + * with pasid. Attaching an S1 domain should be feasible but requires + * more complicated logic to handle. + */ + if (arm_smmu_master_has_pasid_domains(master)) { + dev_err(dev, "cannot attach - domain attached with pasid\n"); + return -EBUSY; + } - mutex_unlock(&smmu_domain->init_mutex); - if (ret) - return ret; + arm_smmu_detach_dev(master); /* * The SMMU does not support enabling ATS with bypass. When the STE is @@ -2500,6 +2524,72 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) return 0; } +static int arm_smmu_set_dev_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid) +{ + int ret = 0; + unsigned long flags; + struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); + struct arm_smmu_device *smmu; + struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); + struct arm_smmu_attached_domain *attached_domain; + struct arm_smmu_master *master; + + if (!fwspec) + return -ENOENT; + + master = dev_iommu_priv_get(dev); + smmu = master->smmu; + + ret = arm_smmu_prepare_domain_for_smmu(smmu, smmu_domain, master); + if (ret) + return ret; + + if (pasid == 0) { + dev_err(dev, "pasid 0 is reserved for the device's primary domain\n"); + return -ENODEV; + } + + if (smmu_domain->stage != ARM_SMMU_DOMAIN_S1) { + dev_err(dev, "set_dev_pasid only supports stage 1 domains\n"); + return -EINVAL; + } + + if (!master->cd_table.cdtab) + return -EBUSY; + + attached_domain = kzalloc(sizeof(*attached_domain), GFP_KERNEL); + if (!attached_domain) + return -ENOMEM; + + attached_domain->master = master; + attached_domain->domain = smmu_domain; + attached_domain->ssid = pasid; + + /* + * arm_smmu_share_asid may update the cd's asid value and write the + * ctx_desc for every attached_domains in the list. There's a potential + * race here regardless of whether we first write the ctx_desc or + * first insert into the domain's list. Grabbing the asic_lock prevents + * SVA from changing the cd's ASID while the cd is being attached. + */ + mutex_lock(&arm_smmu_asid_lock); + ret = arm_smmu_write_ctx_desc(master, pasid, &smmu_domain->cd); + if (ret) { + mutex_unlock(&arm_smmu_asid_lock); + kfree(attached_domain); + return ret; + } + + spin_lock_irqsave(&smmu_domain->attached_ssids_lock, flags); + list_add(&attached_domain->list, &smmu_domain->attached_ssids); + spin_unlock_irqrestore(&smmu_domain->attached_ssids_lock, flags); + mutex_unlock(&arm_smmu_asid_lock); + + master->nr_attached_pasid_domains += 1; + return 0; +} + static int arm_smmu_map_pages(struct iommu_domain *domain, unsigned long iova, phys_addr_t paddr, size_t pgsize, size_t pgcount, int prot, gfp_t gfp, size_t *mapped) @@ -2738,6 +2828,15 @@ static void arm_smmu_release_device(struct device *dev) if (WARN_ON(arm_smmu_master_sva_enabled(master))) iopf_queue_remove_device(master->smmu->evtq.iopf, dev); + if (WARN_ON(master->nr_attached_pasid_domains != 0)) { + /* + * TODO: Do we need to handle this case? + * This requires a mechanism to obtain all the pasid domains + * that this master is attached to so that we can clean up the + * domain's attached_domain list. + */ + } + arm_smmu_detach_dev(master); arm_smmu_disable_pasid(master); arm_smmu_remove_master(master); @@ -2874,12 +2973,36 @@ static int arm_smmu_def_domain_type(struct device *dev) static void arm_smmu_remove_dev_pasid(struct device *dev, ioasid_t pasid) { struct iommu_domain *domain; + struct arm_smmu_master *master = dev_iommu_priv_get(dev); + struct arm_smmu_domain *smmu_domain; + struct arm_smmu_attached_domain *attached_domain; + unsigned long flags; - domain = iommu_get_domain_for_dev_pasid(dev, pasid, IOMMU_DOMAIN_SVA); + if (!master || pasid == 0) + return; + + domain = iommu_get_domain_for_dev_pasid(dev, pasid, 0); if (WARN_ON(IS_ERR(domain)) || !domain) return; + if (domain->type == IOMMU_DOMAIN_SVA) + return arm_smmu_sva_remove_dev_pasid(domain, dev, pasid); - arm_smmu_sva_remove_dev_pasid(domain, dev, pasid); + smmu_domain = to_smmu_domain(domain); + mutex_lock(&arm_smmu_asid_lock); + spin_lock_irqsave(&smmu_domain->attached_ssids_lock, flags); + list_for_each_entry(attached_domain, &smmu_domain->attached_ssids, list) { + if (attached_domain->master != master || + attached_domain->ssid != pasid) + continue; + list_del(&attached_domain->list); + master->nr_attached_pasid_domains -= 1; + kfree(attached_domain); + break; + } + spin_unlock_irqrestore(&smmu_domain->attached_ssids_lock, flags); + arm_smmu_write_ctx_desc(master, pasid, NULL); + arm_smmu_atc_inv_master_ssid(master, pasid); + mutex_unlock(&arm_smmu_asid_lock); } static struct iommu_ops arm_smmu_ops = { @@ -2899,6 +3022,7 @@ static struct iommu_ops arm_smmu_ops = { .owner = THIS_MODULE, .default_domain_ops = &(const struct iommu_domain_ops) { .attach_dev = arm_smmu_attach_dev, + .set_dev_pasid = arm_smmu_set_dev_pasid, .map_pages = arm_smmu_map_pages, .unmap_pages = arm_smmu_unmap_pages, .flush_iotlb_all = arm_smmu_flush_iotlb_all, diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 66a492cafe2e8..433f58bd99dd2 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -713,6 +713,7 @@ struct arm_smmu_master { bool iopf_enabled; struct list_head bonds; unsigned int ssid_bits; + unsigned int nr_attached_pasid_domains; }; /* SMMU private data for an IOMMU domain */ From patchwork Thu Aug 3 10:12:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 130500 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9f41:0:b0:3e4:2afc:c1 with SMTP id v1csp1085783vqx; Thu, 3 Aug 2023 04:40:55 -0700 (PDT) X-Google-Smtp-Source: APBJJlFzCOnMLvWW68s2PhHlNk+jJkOKHNZCil6MokPgsv7dv0LqWbGO6IOyTqt5i86xm2xQ9oaJ X-Received: by 2002:a05:6a20:6a15:b0:134:d4d3:f09a with SMTP id p21-20020a056a206a1500b00134d4d3f09amr20751801pzk.58.1691062854717; Thu, 03 Aug 2023 04:40:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1691062854; cv=none; d=google.com; s=arc-20160816; b=fwN2MipuqVWkN05zb74rBNRSArmXUc/ERwf/iNSu1+3jiBy6ZtqeTgi6W6XwiwWK2g nur4DWqdBgn2BUGRlkDSxpedW6fmaZxcQINUplTeeLISnAkRh1wsll1vBNsAC2PdCQvB Ro7om5fqcI0cubd4WIZ1ZaMBNEW1p/5QYjEx+penlpZH7qO344peABJrzcVrZPah7fRO nWZpm0UTurr/FtqHGfkT9lXiocibK3bH93BYZw6uVEtMen9+HAsHpdwIyWKeSBbr47qL 5cOKagGCjyMTo9T0PMljjwevW+v6IkEOjcLpeAPVAUbGoYOkXeBdsvQ/zdV5wJ/MibKZ mtWA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=xOWJ0lqxCqyUuPaPvlunqjXsmwJAIBAp4YS6BHDu6Oc=; fh=1E1ZQY+a9fAbTEzcn5ex4XiBDnDW+oWG5Bd6D1KRIpk=; b=kxlQeSQlLYW0ya+RNUsDUg3Ft1MRQPzk3XIyYY+YHfRHh/SmUuiNMFJAbd7CGp3QPP L3EGRU/Mu8zP+apmEdRagKQ9Nm+JFF8iJF83oLs3Rz2mqoWo3JeQ2Mncvoj+ILQWIgWX DdJDOIKcDf7Hb3EEKhFp4WM4SkclEYDi3rBRBrSR01yCh7JRm16qFStRnHp3+vKdn54J 1kiorzC5aqncOjQL5cOMUKBS4WBzYZrPTkLy5PPTBhP30K7l2kP/PG27RtZvHjRu/L94 kUr+vNsDYTqhuvgh2AFYkG7ymGCfGBzh9V3+Eu9pMXS1xazaQywVSDJvx2Ql2qEqn+cz P8Uw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=q8ilSnaU; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id ea13-20020a056a004c0d00b0068730ff62dasi7645958pfb.60.2023.08.03.04.40.42; Thu, 03 Aug 2023 04:40:54 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=q8ilSnaU; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235314AbjHCKOy (ORCPT + 99 others); Thu, 3 Aug 2023 06:14:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51960 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235312AbjHCKOi (ORCPT ); Thu, 3 Aug 2023 06:14:38 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B167A3ABE for ; Thu, 3 Aug 2023 03:14:26 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-d1851c52f3dso819459276.1 for ; Thu, 03 Aug 2023 03:14:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1691057666; x=1691662466; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=xOWJ0lqxCqyUuPaPvlunqjXsmwJAIBAp4YS6BHDu6Oc=; b=q8ilSnaU6t9HzGSP25lQgwS3HkEavJ1kpDRxanKDeMxMFpXMt79mwz5/LyAooJlDvA ZFkcNzkxOdJkn+mQlAQajTVdFv7/wktHRMWl9+jtmycAuPJQ443dGFHW9mHfYob5d0dR 5gCKHa5Mr+kyEbJAgDyNiqFHQG8H2exc8qDgBJSYDg23RNV0TbdqhxbIdbAAfRYb1jMH hgM5mz2RHJacrXX1YxLeqEnyi0WI3oxPjjDRLSX5c3zy/nPep2zhTSl0jYzYmZaboMSF Yc8Hk93qM/6eAhy29HnpU9HJeSdlYmBnHfXx9w9kfu8bDRKPI6LjT6pzgjlkK/ioJoFh nk/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691057666; x=1691662466; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=xOWJ0lqxCqyUuPaPvlunqjXsmwJAIBAp4YS6BHDu6Oc=; b=aq1igE94KG/nXRQU42ZOBmQuGPUGJo1V4lzQIjiYoOJINDJh70cB5s/O8viiFUUPm6 aOu9nbqrnkiZW/JwsU1lgthzkypmQ+ZlJfd4orIoSZOfrP2qV1tOOayWt8FGE9EDui9r rCLHeSgeW+OjsKDaMNtIUVK/4INxdgoFI28T8oekIh7zh0vhr4L/INAoy4+D88zW7Zrp 34zyw6Cc27JYSll4rqm6oXfaSf/C9Y7MOYvWYl8X3SZ6NGOghOsKASoqimk7d+PWrLe4 ixd84Zwl1KwJT7hE+0WcHgjwFVPL4HGD31bgLIIBiA4Mk5nGf9+l7d6AZ7qRDLY1QY+I Cgbw== X-Gm-Message-State: ABy/qLavevps3RXxQMRn3Vv8FiuApf85g8bfLqYUVsrXFr1adNcV95rl 4NC3iz34gdkzgMY4IuOSOqLAVHlkzTH+ X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:39d1:8774:b733:6210]) (user=mshavit job=sendgmr) by 2002:a25:2042:0:b0:cb6:6c22:d0f8 with SMTP id g63-20020a252042000000b00cb66c22d0f8mr142117ybg.4.1691057665928; Thu, 03 Aug 2023 03:14:25 -0700 (PDT) Date: Thu, 3 Aug 2023 18:12:25 +0800 In-Reply-To: <20230803101351.1561031-1-mshavit@google.com> Mime-Version: 1.0 References: <20230803101351.1561031-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.585.gd2178a4bd4-goog Message-ID: <20230803181225.v5.5.I6f3fb0734ef5ef746ae7c9b27f632f506197eb30@changeid> Subject: [PATCH v5 5/6] iommu/arm-smmu-v3: Free pasid domains on iommu release From: Michael Shavit To: iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: baolu.lu@linux.intel.com, will@kernel.org, jgg@nvidia.com, jean-philippe@linaro.org, robin.murphy@arm.com, nicolinc@nvidia.com, Michael Shavit X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE, USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1773207924312871474 X-GMAIL-MSGID: 1773207924312871474 The iommu core doesn't guarantee that pasid domains will be detached before the device is released. Track the list of domains that a master is attached to with PASID, so that they can be freed when the iommu is released. Signed-off-by: Michael Shavit --- Changes in v5: - New commit: Free attached pasid domains on release_device() call drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 22 +++++++++++++++------ drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 10 +++++++++- 2 files changed, 25 insertions(+), 7 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 7b296458dafec..5fd6c4d4f0ae4 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2587,6 +2587,9 @@ static int arm_smmu_set_dev_pasid(struct iommu_domain *domain, mutex_unlock(&arm_smmu_asid_lock); master->nr_attached_pasid_domains += 1; + list_add(&attached_domain->list_in_master, + &master->attached_domains); + return 0; } @@ -2786,6 +2789,7 @@ static struct iommu_device *arm_smmu_probe_device(struct device *dev) master->dev = dev; master->smmu = smmu; INIT_LIST_HEAD(&master->bonds); + INIT_LIST_HEAD(&master->attached_domains); dev_iommu_priv_set(dev, master); ret = arm_smmu_insert_master(smmu, master); @@ -2825,16 +2829,21 @@ static struct iommu_device *arm_smmu_probe_device(struct device *dev) static void arm_smmu_release_device(struct device *dev) { struct arm_smmu_master *master = dev_iommu_priv_get(dev); + struct arm_smmu_attached_domain *attached_domain; + struct arm_smmu_domain *smmu_domain; + unsigned long flags; if (WARN_ON(arm_smmu_master_sva_enabled(master))) iopf_queue_remove_device(master->smmu->evtq.iopf, dev); if (WARN_ON(master->nr_attached_pasid_domains != 0)) { - /* - * TODO: Do we need to handle this case? - * This requires a mechanism to obtain all the pasid domains - * that this master is attached to so that we can clean up the - * domain's attached_domain list. - */ + list_for_each_entry(attached_domain, &master->attached_domains, list_in_master) { + smmu_domain = attached_domain->domain; + spin_lock_irqsave(&smmu_domain->attached_ssids_lock, flags); + list_del(&attached_domain->list); + list_del(&attached_domain->list_in_master); + kfree(&attached_domain->list_in_master); + spin_unlock_irqrestore(&smmu_domain->attached_ssids_lock, flags); + } } arm_smmu_detach_dev(master); @@ -2995,6 +3004,7 @@ static void arm_smmu_remove_dev_pasid(struct device *dev, ioasid_t pasid) attached_domain->ssid != pasid) continue; list_del(&attached_domain->list); + list_del(&attached_domain->list_in_master); master->nr_attached_pasid_domains -= 1; kfree(attached_domain); break; diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 433f58bd99dd2..efa428629f4d9 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -689,9 +689,15 @@ struct arm_smmu_stream { struct rb_node node; }; -/* List of {masters, ssid} that a domain is attached to */ +/* + * List of {masters, ssid} that a domain is attached to, and conversely of + * domains that a master is attached to. + */ struct arm_smmu_attached_domain { + /* List node arm_smmu_domain*/ struct list_head list; + /* List node in arm_smmu_master*/ + struct list_head list_in_master; struct arm_smmu_domain *domain; struct arm_smmu_master *master; int ssid; @@ -714,6 +720,8 @@ struct arm_smmu_master { struct list_head bonds; unsigned int ssid_bits; unsigned int nr_attached_pasid_domains; + /* Locked by the iommu core using the group mutex */ + struct list_head attached_domains; }; /* SMMU private data for an IOMMU domain */ From patchwork Thu Aug 3 10:12:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 130490 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9f41:0:b0:3e4:2afc:c1 with SMTP id v1csp1078803vqx; Thu, 3 Aug 2023 04:28:34 -0700 (PDT) X-Google-Smtp-Source: APBJJlGbNYQLUpdY/jxKziCAAhdUwc7XrlQAkTxWEEsNkmgqxCDVDPzS/jR9N2O0DVxSXRoKFqpO X-Received: by 2002:a17:907:2c75:b0:997:e7d9:50f7 with SMTP id ib21-20020a1709072c7500b00997e7d950f7mr6724341ejc.66.1691062114276; Thu, 03 Aug 2023 04:28:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1691062114; cv=none; d=google.com; s=arc-20160816; b=1F/8AuMQ6gpL6l9Q0m4L3/+PIvgMghZVsrJ7C+PicEAomLskGsTdtGWMFteM1TOpiz V+t78ZdGdQRn7+bdp3Ky6KucZ+XHB7eFCyCn3y3BtwnLFVmv7yuDdfUQiUpQu/y2ouQm VFdjXF6NMNwtGF3MguUMAtZHX5p1sdIgMDuntYjANS2O+rsDtfdc8OexaX3SV5VRBgRt JIr0DuhuLZH16NV42aP8gtLXW+1QWywWtUrL+jrdKBglGXyeXBPq97SAhpMV3C2VrdXV zaisCng7fRMuc7NpeKBmWKjtqB1TommANsrEIKK6z8SWa/Q6cTLizqV/BIEuRu3pUcKD w+Rg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=DLa2HUSCTWFTI0X2o3f102hvfCfHBN0tK5dUlzxCyIc=; fh=1E1ZQY+a9fAbTEzcn5ex4XiBDnDW+oWG5Bd6D1KRIpk=; b=wc/sUFmM56DwEANYg+IBvQTkw5sXlLv3hgGjf/KL+e0+dPe5yeRmiz/O8JDq/vHmJN Lnsgn0TBkOvCf4k6MfuVcHvW3HwTlhs1QP9lFIvCNIfcsP7Xb2lqlTshrTVDj2dwffJ4 ki91RlUq4QB2rvXehCESi9ohYeOIlP+K7RwPdlCOn+x1RQw6W4wD+PB5smXD4xLEC8dj 20kFIihy/giKH4z4lXQSziSnENAAhTGCwV4BTPO/2P0S4hQp1fHH5XzvWKLH+/N/QIUR imtRKP/mg9Uwkpipb5kqBk2gdp7PlsgpaxNinQTOWjn27sdaIC+7MRGtq4wr2oQw4qgL EAkw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=a6jeCmc4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id dv22-20020a170906b81600b00992bd86ece6si11662991ejb.725.2023.08.03.04.28.10; Thu, 03 Aug 2023 04:28:34 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=a6jeCmc4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235273AbjHCKPM (ORCPT + 99 others); Thu, 3 Aug 2023 06:15:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52568 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235329AbjHCKOp (ORCPT ); Thu, 3 Aug 2023 06:14:45 -0400 Received: from mail-yw1-x114a.google.com (mail-yw1-x114a.google.com [IPv6:2607:f8b0:4864:20::114a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6F1A33C38 for ; Thu, 3 Aug 2023 03:14:31 -0700 (PDT) Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-5844a99c268so8276687b3.2 for ; Thu, 03 Aug 2023 03:14:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1691057670; x=1691662470; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=DLa2HUSCTWFTI0X2o3f102hvfCfHBN0tK5dUlzxCyIc=; b=a6jeCmc4bN/A2ChgfTwyYgnoSOjzynwD0hKdab97fxFiQfSeGipLdBD8vrco7vSA/g QOt22+Zq6NX8xu96ciZpkKp4t5eOm+ZaDlxM6cwB96C4VYR8qRCGUhy2F8aLjmrZX7BK lgPFNpSbNKKYhMqQtdmKUOe2mtgb5tKgGOgDmq9z4UUNQSfYVDQitc/nmlYflTh8r71x WHXjiURsl1a7dnvRWS7u0ZIp9NkZQ7glmppXYAaIMaXGtS+Nd08ghmzEF5M9LzrJXJmc KqvH0cWbBfcQls+aRNNfkuVmAcdzik2ZO7MoKwsaPcJ6aKQIAFn69fAb4koX4ECdcQQ0 esCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691057670; x=1691662470; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=DLa2HUSCTWFTI0X2o3f102hvfCfHBN0tK5dUlzxCyIc=; b=Uor52Kf4esgcfSWHSLZuV0yL7N0wU5fhsALmCqOd/PItm/wh/eaXyigyW5qXMIzqbZ UU/lSckMBI3FW6fiiQx3oq3o0ZZaX3lZs8Oh/yF9js24QBjdor5pAST7d1+YKpUApDos HnhdoJK2HCJgxb36DtY6ReC3rc2j3o9eRWsJl7lq8fMC7FjAye6lyUSNpgE1+m34bjy+ aP1DbRi/UvUv/r30sxKG2nJykDmTujoS5Ye+h3/NZm2wI1pA3COCxyoO90AcbpReC8q2 o9c6ZdZmU6amhDsJniGyEzSvr7U+DjKStfkTDMsfJ9xuugP9AuE9lEQCh/oWI0M3H6GN cWMQ== X-Gm-Message-State: ABy/qLZDcPHXNU4IhfcZzfZQLOS0Dnu1Y95dQiFF6WnzdUpDWxqMO7Kg 28o9oYFoSMvsIc2jVLWtojlLF9EqGXt2 X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:39d1:8774:b733:6210]) (user=mshavit job=sendgmr) by 2002:a81:b625:0:b0:586:50cf:e13f with SMTP id u37-20020a81b625000000b0058650cfe13fmr100164ywh.1.1691057670282; Thu, 03 Aug 2023 03:14:30 -0700 (PDT) Date: Thu, 3 Aug 2023 18:12:26 +0800 In-Reply-To: <20230803101351.1561031-1-mshavit@google.com> Mime-Version: 1.0 References: <20230803101351.1561031-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.585.gd2178a4bd4-goog Message-ID: <20230803181225.v5.6.Iff18df41564b9df82bf40b3ec7af26b87f08ef6e@changeid> Subject: [PATCH v5 6/6] iommu/arm-smmu-v3: Cleanup arm_smmu_domain_finalise From: Michael Shavit To: iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: baolu.lu@linux.intel.com, will@kernel.org, jgg@nvidia.com, jean-philippe@linaro.org, robin.murphy@arm.com, nicolinc@nvidia.com, Michael Shavit X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE, USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1773207147277258425 X-GMAIL-MSGID: 1773207147277258425 Remove unused master parameter now that the CD table is allocated elsewhere. Signed-off-by: Michael Shavit Reviewed-by: Jason Gunthorpe --- (no changes since v1) --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 17 ++++++----------- 1 file changed, 6 insertions(+), 11 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 5fd6c4d4f0ae4..db8fd4b3591b5 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2129,7 +2129,6 @@ static void arm_smmu_domain_free(struct iommu_domain *domain) } static int arm_smmu_domain_finalise_cd(struct arm_smmu_domain *smmu_domain, - struct arm_smmu_master *master, struct io_pgtable_cfg *pgtbl_cfg) { int ret; @@ -2167,7 +2166,6 @@ static int arm_smmu_domain_finalise_cd(struct arm_smmu_domain *smmu_domain, } static int arm_smmu_domain_finalise_s2(struct arm_smmu_domain *smmu_domain, - struct arm_smmu_master *master, struct io_pgtable_cfg *pgtbl_cfg) { int vmid; @@ -2192,8 +2190,7 @@ static int arm_smmu_domain_finalise_s2(struct arm_smmu_domain *smmu_domain, return 0; } -static int arm_smmu_domain_finalise(struct iommu_domain *domain, - struct arm_smmu_master *master) +static int arm_smmu_domain_finalise(struct iommu_domain *domain) { int ret; unsigned long ias, oas; @@ -2201,7 +2198,6 @@ static int arm_smmu_domain_finalise(struct iommu_domain *domain, struct io_pgtable_cfg pgtbl_cfg; struct io_pgtable_ops *pgtbl_ops; int (*finalise_stage_fn)(struct arm_smmu_domain *, - struct arm_smmu_master *, struct io_pgtable_cfg *); struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); struct arm_smmu_device *smmu = smmu_domain->smmu; @@ -2253,7 +2249,7 @@ static int arm_smmu_domain_finalise(struct iommu_domain *domain, domain->geometry.aperture_end = (1UL << pgtbl_cfg.ias) - 1; domain->geometry.force_aperture = true; - ret = finalise_stage_fn(smmu_domain, master, &pgtbl_cfg); + ret = finalise_stage_fn(smmu_domain, &pgtbl_cfg); if (ret < 0) { free_io_pgtable_ops(pgtbl_ops); return ret; @@ -2429,15 +2425,14 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) } static int arm_smmu_prepare_domain_for_smmu(struct arm_smmu_device *smmu, - struct arm_smmu_domain *smmu_domain, - struct arm_smmu_master *master) + struct arm_smmu_domain *smmu_domain) { int ret = 0; mutex_lock(&smmu_domain->init_mutex); if (!smmu_domain->smmu) { smmu_domain->smmu = smmu; - ret = arm_smmu_domain_finalise(&smmu_domain->domain, master); + ret = arm_smmu_domain_finalise(&smmu_domain->domain); if (ret) smmu_domain->smmu = NULL; } else if (smmu_domain->smmu != smmu) @@ -2462,7 +2457,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) master = dev_iommu_priv_get(dev); smmu = master->smmu; - ret = arm_smmu_prepare_domain_for_smmu(smmu, smmu_domain, master); + ret = arm_smmu_prepare_domain_for_smmu(smmu, smmu_domain); if (ret) return ret; @@ -2541,7 +2536,7 @@ static int arm_smmu_set_dev_pasid(struct iommu_domain *domain, master = dev_iommu_priv_get(dev); smmu = master->smmu; - ret = arm_smmu_prepare_domain_for_smmu(smmu, smmu_domain, master); + ret = arm_smmu_prepare_domain_for_smmu(smmu, smmu_domain); if (ret) return ret;