From patchwork Tue Jun 6 12:07:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 103776 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp3354668vqr; Tue, 6 Jun 2023 05:26:54 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7vg+X6XFzxL88+yNfqTRvppRNeQF6g5UnSeF7zT/3UkmU++jaLSfFyYBAaFGWhDODICjLk X-Received: by 2002:a17:90a:1c5:b0:259:b2c3:120f with SMTP id 5-20020a17090a01c500b00259b2c3120fmr30556pjd.34.1686054413923; Tue, 06 Jun 2023 05:26:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686054413; cv=none; d=google.com; s=arc-20160816; b=dzN0oiGwAyV+q8GC1WOBPnt1QHd7K71Bp/BdCHn10dnLeSQo1lOKPcJkqWErQatYfF OnA70vlTUv2WLosGNqQwRLwj947hgBbzd9jJbRyuRncfJaIaP8joBvfhku3BMGGogbWd 7tU7YchhvjH4M90isoNGQhQnh0HJiyCMOpjw2EKsWokPmLk3tDnbQth73ZTDJl12+AZy hc2rnRsNOg/SYXiOEDXwvwN7P63Y71crG0ADhFQjsyq+BatL8owLQoiZkZv2HUFdAzxP heAnvwJ0DZercmXr3/sRwv+vL851GQfzX7uyVVKgpt3nE6kHHoPOaaTYYHTk262ETFBO CVCQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=lJ3cILiWBl6sHwkWb+xA2FBDWFhU70WEk+uPmzetVys=; b=SYlaNW36gIBxLpO7ElcbIsHNFO3TdxmwEcnWyJvcn3bOTKmcfwdf8XUeBqWgFBh8xX KD86ORhVJogts06Zy4Q6NEccA2TGiTV4FWG3uzVKY6CSRJPVk+ZTDAfXsknKB7ADG+c3 0fOFzHA5ltpC97q0JxgNru1xthNxzk0N1ltjTUX0f6GtNf0wYwVgzCvMtf7MnJPiLhwM Oac5J1hTcDL/dT+lkXkrSbJKI4H643i7VIqbLZ25w12PBfDgHeUxFyLuNIp1Ck8kZ4Gx uYuuTZpeox4V+Bb/Avsf/afXLOlrTkx7j0at2fyTCsl9ar7xenVIe183VvyKuiqdu5fe u5BA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=waAz3jZd; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id pm11-20020a17090b3c4b00b00256078a4c6bsi9179983pjb.78.2023.06.06.05.26.41; Tue, 06 Jun 2023 05:26:53 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=waAz3jZd; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236438AbjFFMJ3 (ORCPT + 99 others); Tue, 6 Jun 2023 08:09:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46062 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236360AbjFFMJ1 (ORCPT ); Tue, 6 Jun 2023 08:09:27 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 44FB410C3 for ; Tue, 6 Jun 2023 05:09:26 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-bb0d11a56abso8021001276.2 for ; Tue, 06 Jun 2023 05:09:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686053365; x=1688645365; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=lJ3cILiWBl6sHwkWb+xA2FBDWFhU70WEk+uPmzetVys=; b=waAz3jZdXTtDO8YssmYN5G/omZlhLk3AOZz+YlKxfJTmIQFgsouKNpICkvLnj5PC/H bZqWVLF3XcBri+fZh1WmYCnXCapku6Wrt/uBr9E/faVNwT/kYzGDxtQq94OVjLaHSWXT lXowvoVL5l6wuhwdpgDhR3x7IXZKL7LtKM68ekc5whSTKAPi9JRVV7858iLvKqE+rAgQ V2z91OUSOPBgHt4rZ0RPDqTgQR0hkZpkD9w6Hgeww7FbN7MUsMroas3NWhEUMDgh1Un4 LEaUgWdNTiF8qSbnloEwbSGHXijMkKZgdWQFvALN51/9Hcu+UqMrEXjYbAHZLscdB6Ev xhmg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686053365; x=1688645365; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=lJ3cILiWBl6sHwkWb+xA2FBDWFhU70WEk+uPmzetVys=; b=RZyjqMpxoNW0e88BYYtDx7pw4/bhg5h3nTtQmDNMLZi7sG/JutggQrdBc9x5Vyy9mg OFDMBFrxEpr2p2JLFZ5bkuc+l2TFuBFu6VtWH7vr/yGGqO7saMPHfhEWQyU3WSM9wa10 pAEhqcXWHUOHQkPZFBnUoiJQ2nOhcV6OU6xKFEdh0mHR2bdV80ZqDDP2aw5kArggWgVB bBBdV00ViuGQfFiEE6m7hbcC5ZYspijLDMFe85CsAQtj4C88Q6c+UcOmPzPNIC8uzhXK 75f+U4gROvT+uWS2IOdTqp4bxfvk3fZ8XzYbNpHUTCafnmGMuhMybsOqoJyFA2cFOZYx Uwfg== X-Gm-Message-State: AC+VfDzWTYIBTRdifHD7PDvBrCV4nX+RKeBSxCnzs/2QfHNw0AhLDGhg Fur5cE3co6fcwqupGo3sKGzRZDJ754FI X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:a615:63d5:b54e:6919]) (user=mshavit job=sendgmr) by 2002:a25:e794:0:b0:ba8:4ff5:4671 with SMTP id e142-20020a25e794000000b00ba84ff54671mr1024741ybh.9.1686053365471; Tue, 06 Jun 2023 05:09:25 -0700 (PDT) Date: Tue, 6 Jun 2023 20:07:37 +0800 In-Reply-To: <20230606120854.4170244-1-mshavit@google.com> Mime-Version: 1.0 References: <20230606120854.4170244-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.rc0.172.g3f132b7071-goog Message-ID: <20230606120854.4170244-2-mshavit@google.com> Subject: [PATCH v2 01/18] iommu/arm-smmu-v3: Move ctx_desc out of s1_cfg From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767956193165995177?= X-GMAIL-MSGID: =?utf-8?q?1767956193165995177?= s1_cfg describes the CD table that is inserted into an SMMU's STEs. It's weird for s1_cfg to also own ctx_desc which describes a CD that is inserted into that table. It is more appropriate for arm_smmu_domain to own ctx_desc. Signed-off-by: Michael Shavit --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 2 +- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 23 +++++++-------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 28 ++++++++++--------- 3 files changed, 28 insertions(+), 25 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index a5a63b1c947eb..968559d625c40 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -62,7 +62,7 @@ arm_smmu_share_asid(struct mm_struct *mm, u16 asid) return cd; } - smmu_domain = container_of(cd, struct arm_smmu_domain, s1_cfg.cd); + smmu_domain = container_of(cd, struct arm_smmu_domain, cd); smmu = smmu_domain->smmu; ret = xa_alloc(&arm_smmu_asid_xa, &new_asid, cd, diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 3fd83fb757227..beff04b897718 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1863,7 +1863,7 @@ static void arm_smmu_tlb_inv_context(void *cookie) * careful, 007. */ if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { - arm_smmu_tlb_inv_asid(smmu, smmu_domain->s1_cfg.cd.asid); + arm_smmu_tlb_inv_asid(smmu, smmu_domain->cd.asid); } else { cmd.opcode = CMDQ_OP_TLBI_S12_VMALL; cmd.tlbi.vmid = smmu_domain->s2_cfg.vmid; @@ -1946,7 +1946,7 @@ static void arm_smmu_tlb_inv_range_domain(unsigned long iova, size_t size, if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { cmd.opcode = smmu_domain->smmu->features & ARM_SMMU_FEAT_E2H ? CMDQ_OP_TLBI_EL2_VA : CMDQ_OP_TLBI_NH_VA; - cmd.tlbi.asid = smmu_domain->s1_cfg.cd.asid; + cmd.tlbi.asid = smmu_domain->cd.asid; } else { cmd.opcode = CMDQ_OP_TLBI_S2_IPA; cmd.tlbi.vmid = smmu_domain->s2_cfg.vmid; @@ -2077,7 +2077,7 @@ static void arm_smmu_domain_free(struct iommu_domain *domain) mutex_lock(&arm_smmu_asid_lock); if (cfg->cdcfg.cdtab) arm_smmu_free_cd_tables(smmu_domain); - arm_smmu_free_asid(&cfg->cd); + arm_smmu_free_asid(&smmu_domain->cd); mutex_unlock(&arm_smmu_asid_lock); } else { struct arm_smmu_s2_cfg *cfg = &smmu_domain->s2_cfg; @@ -2096,13 +2096,14 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, u32 asid; struct arm_smmu_device *smmu = smmu_domain->smmu; struct arm_smmu_s1_cfg *cfg = &smmu_domain->s1_cfg; + struct arm_smmu_ctx_desc *cd = &smmu_domain->cd; typeof(&pgtbl_cfg->arm_lpae_s1_cfg.tcr) tcr = &pgtbl_cfg->arm_lpae_s1_cfg.tcr; - refcount_set(&cfg->cd.refs, 1); + refcount_set(&cd->refs, 1); /* Prevent SVA from modifying the ASID until it is written to the CD */ mutex_lock(&arm_smmu_asid_lock); - ret = xa_alloc(&arm_smmu_asid_xa, &asid, &cfg->cd, + ret = xa_alloc(&arm_smmu_asid_xa, &asid, cd, XA_LIMIT(1, (1 << smmu->asid_bits) - 1), GFP_KERNEL); if (ret) goto out_unlock; @@ -2115,23 +2116,23 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, if (ret) goto out_free_asid; - cfg->cd.asid = (u16)asid; - cfg->cd.ttbr = pgtbl_cfg->arm_lpae_s1_cfg.ttbr; - cfg->cd.tcr = FIELD_PREP(CTXDESC_CD_0_TCR_T0SZ, tcr->tsz) | + cd->asid = (u16)asid; + cd->ttbr = pgtbl_cfg->arm_lpae_s1_cfg.ttbr; + cd->tcr = FIELD_PREP(CTXDESC_CD_0_TCR_T0SZ, tcr->tsz) | FIELD_PREP(CTXDESC_CD_0_TCR_TG0, tcr->tg) | FIELD_PREP(CTXDESC_CD_0_TCR_IRGN0, tcr->irgn) | FIELD_PREP(CTXDESC_CD_0_TCR_ORGN0, tcr->orgn) | FIELD_PREP(CTXDESC_CD_0_TCR_SH0, tcr->sh) | FIELD_PREP(CTXDESC_CD_0_TCR_IPS, tcr->ips) | CTXDESC_CD_0_TCR_EPD1 | CTXDESC_CD_0_AA64; - cfg->cd.mair = pgtbl_cfg->arm_lpae_s1_cfg.mair; + cd->mair = pgtbl_cfg->arm_lpae_s1_cfg.mair; /* * Note that this will end up calling arm_smmu_sync_cd() before * the master has been added to the devices list for this domain. * This isn't an issue because the STE hasn't been installed yet. */ - ret = arm_smmu_write_ctx_desc(smmu_domain, 0, &cfg->cd); + ret = arm_smmu_write_ctx_desc(smmu_domain, 0, cd); if (ret) goto out_free_cd_tables; @@ -2141,7 +2142,7 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, out_free_cd_tables: arm_smmu_free_cd_tables(smmu_domain); out_free_asid: - arm_smmu_free_asid(&cfg->cd); + arm_smmu_free_asid(cd); out_unlock: mutex_unlock(&arm_smmu_asid_lock); return ret; diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index b574c58a34876..68d519f21dbd8 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -593,7 +593,6 @@ struct arm_smmu_ctx_desc_cfg { struct arm_smmu_s1_cfg { struct arm_smmu_ctx_desc_cfg cdcfg; - struct arm_smmu_ctx_desc cd; u8 s1fmt; u8 s1cdmax; }; @@ -707,25 +706,28 @@ enum arm_smmu_domain_stage { }; struct arm_smmu_domain { - struct arm_smmu_device *smmu; - struct mutex init_mutex; /* Protects smmu pointer */ + struct arm_smmu_device *smmu; + struct mutex init_mutex; /* Protects smmu pointer */ - struct io_pgtable_ops *pgtbl_ops; - bool stall_enabled; - atomic_t nr_ats_masters; + struct io_pgtable_ops *pgtbl_ops; + bool stall_enabled; + atomic_t nr_ats_masters; - enum arm_smmu_domain_stage stage; + enum arm_smmu_domain_stage stage; union { - struct arm_smmu_s1_cfg s1_cfg; - struct arm_smmu_s2_cfg s2_cfg; + struct { + struct arm_smmu_ctx_desc cd; + struct arm_smmu_s1_cfg s1_cfg; + }; + struct arm_smmu_s2_cfg s2_cfg; }; - struct iommu_domain domain; + struct iommu_domain domain; - struct list_head devices; - spinlock_t devices_lock; + struct list_head devices; + spinlock_t devices_lock; - struct list_head mmu_notifiers; + struct list_head mmu_notifiers; }; static inline struct arm_smmu_domain *to_smmu_domain(struct iommu_domain *dom) From patchwork Tue Jun 6 12:07:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 103763 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp3349694vqr; Tue, 6 Jun 2023 05:18:51 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7rCHSgg4kkV+JkQPxoPn/xcJLe12HbfZe8xOFufmXAGck5kxmNka9o33ereBpUnx/QHdJ2 X-Received: by 2002:a05:622a:110e:b0:3f4:f1be:557 with SMTP id e14-20020a05622a110e00b003f4f1be0557mr2124696qty.1.1686053931133; Tue, 06 Jun 2023 05:18:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686053931; cv=none; d=google.com; s=arc-20160816; b=nQwxqwGJyDecWc5in4RKe3G0p0nBvRY8xRBRDlIreSdboadbOIgoxd/VvLKElE9IUz HeUwqqFHJl7dRe8jLMHtRV8wQya98XTjpK/wbBFjK16PwiA4+uIHP9vAvbGRG3HilsuW cAx9B7S0soRisdeikxTI+Y28q0kit8GcA3qO6VVAzCNAoH6ZPYclf5h/X+ilElLlNJt1 Xkwqag4os9GCPFjJGjbp7K3NJ4V4Ua2Pt3T91sIoqGgIDBfDiswNtj2RZIDP9I2ACaXN 2818aBvXw91/oFJAOwGji1l4vOIs1KIVj9/r9ZwV52tgroIOHGsJlZW+c2TexprX2FYc SHYg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=0tYLIcPaSiiyZHuB/nJJEFPBa3OnAACeS9baH6iUupo=; b=D3RrW+oli06h6apeGGxCtXBXOhBT3Yi5UsHXqOn2SxOmu29ydW136Os0JkIjzfTsx8 dIPVIFlM4CUq0nbtMlm52Q9290VcE9pX0Px6piNZnFXcwBhHg1FOaSQO1uRX2uVLjtn9 eyTqEWC5hVsqMaZfEDLknQ9puzur/7Y2zVwvlrT4I5RlgPnvoFwkTANa+g5VWTIdGKtj xDJJvTAyghp7IDdYgPmjlN5ZnH8edZLY2oQ/HzPW/uDbIG3VULvwFhmQA14TSjgEW+i5 HOZwQKya2DDVv9er5PHQJyV16BKr8PyYDtrplm9ZLaXJxF7VG7CB0MyuyLB0GsiUlro4 9sHg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=22Yh0Qre; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id j22-20020ac85f96000000b003f7f9d31712si6236793qta.308.2023.06.06.05.18.36; Tue, 06 Jun 2023 05:18:51 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=22Yh0Qre; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236751AbjFFMJj (ORCPT + 99 others); Tue, 6 Jun 2023 08:09:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46186 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236838AbjFFMJh (ORCPT ); Tue, 6 Jun 2023 08:09:37 -0400 Received: from mail-yw1-x114a.google.com (mail-yw1-x114a.google.com [IPv6:2607:f8b0:4864:20::114a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 87D8610D9 for ; Tue, 6 Jun 2023 05:09:33 -0700 (PDT) Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-5689bcc5f56so89535757b3.2 for ; Tue, 06 Jun 2023 05:09:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686053372; x=1688645372; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=0tYLIcPaSiiyZHuB/nJJEFPBa3OnAACeS9baH6iUupo=; b=22Yh0QrehWwX8i9ZY6o2wrXkvcH70GK73SeCmuwBZwKKIBgf4Y/t35VFAUZIJ/leF9 ConLKIZYFWTFR9V/OVafYZ02wvG46i5Vcp14Qf81Xm5C2Hs2dD7L/2ySTOOyN1a8I9os 4KkwnWaTT0rZuSQh804kQxJBizMU3MoLg1EUBgrAqB0y5FsizZiprrl0hrnBZdZijww+ D5N9/JQY4pwY47dUeXjE6rWyX9nbabwmJ92YYT9x/UhrcwjJqa7OCH1M1+uppz53YBcW j19D9te7IclkE5OlyAyd0o1t6I3Pab7vT1wTIv76Y26ZCgau5H3C7rRmBE3W4ahxmoaC L1DA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686053372; x=1688645372; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=0tYLIcPaSiiyZHuB/nJJEFPBa3OnAACeS9baH6iUupo=; b=gbpWfTucgySQ8fBZrRGRVcSnxpjPR5YRH08aewKp5bonvq1ISUlTmqkX+9Pxh+qLm3 Fje+BFB4lh5hxfSszPGVjn9HeIxrKlos+Y/Et5e1i/caNt6H3tZfVMxANnwe149/lauq aYx2QaYuRSRDWz2sw8Mg/9dcjeNkv0l6DgNaz39Db7LQGZcLqW6r6X/NYNmWBW/8um3A fz6DPRPNwmhtuVR9VLqE2XkJ3SxCCxDRcej3FgGdPI5nf7LzmoBr/cHbd/HseCvq1eRw J0astplu/72Na2xsNR2CWA4QoTUA893YVCzr2q9nXGBuPTPFOQzCJsNM3UVwGFU1V5hT xxxA== X-Gm-Message-State: AC+VfDwiInYTg3lFHo++oxIDs+rXEMJbiJNjAXd3ZqfJaowh3DcMIRMl Y/glvyBphl8ji4UGtQIoefxajdcrlOLL X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:a615:63d5:b54e:6919]) (user=mshavit job=sendgmr) by 2002:a81:8d4d:0:b0:561:81b:7369 with SMTP id w13-20020a818d4d000000b00561081b7369mr949415ywj.8.1686053372788; Tue, 06 Jun 2023 05:09:32 -0700 (PDT) Date: Tue, 6 Jun 2023 20:07:38 +0800 In-Reply-To: <20230606120854.4170244-1-mshavit@google.com> Mime-Version: 1.0 References: <20230606120854.4170244-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.rc0.172.g3f132b7071-goog Message-ID: <20230606120854.4170244-3-mshavit@google.com> Subject: [PATCH v2 02/18] iommu/arm-smmu-v3: Add smmu_s1_cfg to smmu_master From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767955687063204242?= X-GMAIL-MSGID: =?utf-8?q?1767955687063204242?= Except for Nested domains, arm_smmu_master will own the STEs that are inserted into the arm_smmu_device's STE table. Signed-off-by: Michael Shavit --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 28 +++++++++++++-------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 1 + 2 files changed, 18 insertions(+), 11 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index beff04b897718..023769f5ca79a 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1126,15 +1126,16 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, return 0; } -static int arm_smmu_alloc_cd_tables(struct arm_smmu_domain *smmu_domain) +static int arm_smmu_init_s1_cfg(struct arm_smmu_master *master, + struct arm_smmu_s1_cfg *cfg) { int ret; size_t l1size; size_t max_contexts; - struct arm_smmu_device *smmu = smmu_domain->smmu; - struct arm_smmu_s1_cfg *cfg = &smmu_domain->s1_cfg; + struct arm_smmu_device *smmu = master->smmu; struct arm_smmu_ctx_desc_cfg *cdcfg = &cfg->cdcfg; + cfg->s1cdmax = master->ssid_bits; max_contexts = 1 << cfg->s1cdmax; if (!(smmu->features & ARM_SMMU_FEAT_2_LVL_CDTAB) || @@ -1175,12 +1176,11 @@ static int arm_smmu_alloc_cd_tables(struct arm_smmu_domain *smmu_domain) return ret; } -static void arm_smmu_free_cd_tables(struct arm_smmu_domain *smmu_domain) +static void arm_smmu_free_cd_tables(struct arm_smmu_device *smmu, + struct arm_smmu_ctx_desc_cfg *cdcfg) { int i; size_t size, l1size; - struct arm_smmu_device *smmu = smmu_domain->smmu; - struct arm_smmu_ctx_desc_cfg *cdcfg = &smmu_domain->s1_cfg.cdcfg; if (cdcfg->l1_desc) { size = CTXDESC_L2_ENTRIES * (CTXDESC_CD_DWORDS << 3); @@ -2076,7 +2076,7 @@ static void arm_smmu_domain_free(struct iommu_domain *domain) /* Prevent SVA from touching the CD while we're freeing it */ mutex_lock(&arm_smmu_asid_lock); if (cfg->cdcfg.cdtab) - arm_smmu_free_cd_tables(smmu_domain); + arm_smmu_free_cd_tables(smmu, &cfg->cdcfg); arm_smmu_free_asid(&smmu_domain->cd); mutex_unlock(&arm_smmu_asid_lock); } else { @@ -2108,11 +2108,9 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, if (ret) goto out_unlock; - cfg->s1cdmax = master->ssid_bits; - smmu_domain->stall_enabled = master->stall_enabled; - ret = arm_smmu_alloc_cd_tables(smmu_domain); + ret = arm_smmu_init_s1_cfg(master, cfg); if (ret) goto out_free_asid; @@ -2140,7 +2138,7 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, return 0; out_free_cd_tables: - arm_smmu_free_cd_tables(smmu_domain); + arm_smmu_free_cd_tables(smmu, &cfg->cdcfg); out_free_asid: arm_smmu_free_asid(cd); out_unlock: @@ -2704,6 +2702,13 @@ static struct iommu_device *arm_smmu_probe_device(struct device *dev) smmu->features & ARM_SMMU_FEAT_STALL_FORCE) master->stall_enabled = true; + ret = arm_smmu_init_s1_cfg(master, &master->owned_s1_cfg); + if (ret) { + arm_smmu_disable_pasid(master); + arm_smmu_remove_master(master); + goto err_free_master; + } + return &smmu->iommu; err_free_master: @@ -2719,6 +2724,7 @@ static void arm_smmu_release_device(struct device *dev) if (WARN_ON(arm_smmu_master_sva_enabled(master))) iopf_queue_remove_device(master->smmu->evtq.iopf, dev); arm_smmu_detach_dev(master); + arm_smmu_free_cd_tables(master->smmu, &master->owned_s1_cfg.cdcfg); arm_smmu_disable_pasid(master); arm_smmu_remove_master(master); kfree(master); diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 68d519f21dbd8..053cc14c23969 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -688,6 +688,7 @@ struct arm_smmu_master { struct arm_smmu_domain *domain; struct list_head domain_head; struct arm_smmu_stream *streams; + struct arm_smmu_s1_cfg owned_s1_cfg; unsigned int num_streams; bool ats_enabled; bool stall_enabled; From patchwork Tue Jun 6 12:07:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 103769 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp3351371vqr; Tue, 6 Jun 2023 05:21:20 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4h4klpWSaU3oad5uc/P8GM7ykpffjJo4yrpELoc5UkAR51qQzk7R+yXKoYIRTDU8WLopoL X-Received: by 2002:ac8:5c82:0:b0:3f6:b713:f3cd with SMTP id r2-20020ac85c82000000b003f6b713f3cdmr2022041qta.0.1686054080612; Tue, 06 Jun 2023 05:21:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686054080; cv=none; d=google.com; s=arc-20160816; b=Tfi0Nia7p59YTdgyI8P+MBFglerPtib1jvXOO+fp6XHjWCyaHuJeadlvUUJnypjE4N l9x9+nCLvNdP0EXhOUnvm0BDpt6VI/b8BFypPOE3H/TKAt0+LgxNGjnNUfDHmKIhEcvi QnVweZdWzpVSxcbN3RWdimwejnYQFJpKknGuAvplExulY5Jk7ie4TZdl1w+VtetHqvma OsqWuLNEuuR614eKw5zlMEQ/C4hacLYyJI/2JawhPu8ZhFRhodtpAzXHa1F2/X7XGZW3 UWv8m99mbmkVLCe9vsoxmX+hSQ5wRQp13DXwRXpdmhh7DqEx4s0VXcb5K/Q2dHnb7IVG a2Gg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=hRUuglN2mFNqzu8aGhjOdpRI6fbxHMR7AUFnYMKbqRg=; b=lPsMHqbjqI6KzVTWJV/4Smx4bYAd+h2vCnHcBYoHl6J2KoolNl2F1VoVAlxE6GWWl1 ME57MN0scEzcyzhg5K7SwIqAg1YYUjgFPaSzZYehhKAp0mL62FZ5g+sKWE/PaINqg4ve fwCPOdHyHbRzrRLw/ggh+af+7ZSPEfnJ81UUhULKIj0tCssBxjaPMoARmyHK4AtrHRo5 N9ohIGCr/hmFgbLBHvqqK+T+cJhXEfxF9Zn3UGlllDZE15Hmy9Jxu4V3czk5DrokjsyB oN4WrjGsasThVxe8jiZaLjHgn4wlXQSAcosp51bgLSTL/LDrEtjG7rH9VZavcSkctHNF F7Ng== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=kFD8qk5n; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id w18-20020a05622a135200b003e457751d2esi6238674qtk.494.2023.06.06.05.21.04; Tue, 06 Jun 2023 05:21:20 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=kFD8qk5n; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233808AbjFFMJ5 (ORCPT + 99 others); Tue, 6 Jun 2023 08:09:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46376 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237396AbjFFMJu (ORCPT ); Tue, 6 Jun 2023 08:09:50 -0400 Received: from mail-yw1-x1149.google.com (mail-yw1-x1149.google.com [IPv6:2607:f8b0:4864:20::1149]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F28CB10E5 for ; Tue, 6 Jun 2023 05:09:40 -0700 (PDT) Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-565d1b86a63so100114427b3.0 for ; Tue, 06 Jun 2023 05:09:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686053380; x=1688645380; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=hRUuglN2mFNqzu8aGhjOdpRI6fbxHMR7AUFnYMKbqRg=; b=kFD8qk5nz8nEkhEGuavEr1Filv5HOrp6syWncQpUGLaJi3DYvQwXC5BFVUTs6ShZMw h2Y95axOqxnM9XTmFJ6HM/LFDny8rzY6qFSMC/+3xIlGp/7hPbzW7cKJtEwfD1qJdZ7/ zyK5eZGzR/IklUms3hb6ynufw6ajbdu3QxaJ7Jfuq+CZShyStZGF4+K+2RlTw70PAT9j KenSIraTK1+dvYOx1AWhSiCb+EOCSpPRtSBeuDYUWj3jT31ikuHdVpsQd4VxIg0AQeVX Rc82mEmNmqsBHLrx5rXhOEoFNFvIeWbCvbY6ADIAislq33tv4hCanDFstI6fN4sCfrsT hQAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686053380; x=1688645380; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=hRUuglN2mFNqzu8aGhjOdpRI6fbxHMR7AUFnYMKbqRg=; b=cPrn5wYv4P5hwyJFs5ExsLYtfysZ0xglkXgX10gWeHDY1/UTFDtPKNOWIzHML3NKrr RyJJJL0ji6Wh6MLS9A5vBLFPUSp6MOJSVh1rxpKcY7QsyRi93WCK/HwNVEzUZ/104gtX zeK9xzem3DQ2Upo66BBL+NrvaJXN8sliejjPPbZZkxF2ymKctuBNM+FAc9WRL75L2kmo Y8egpiPD48G4k98IrCgktE8FdYxTEY8nEMKTVxbDPQdtmj73AWi7d0ZBcoMsVGhxcG8X zqrrx01aqapqItNd4++D/0MXX50z8KIn5p9Ya4YtUyjpb01JnwpjHq2Zs/xCxwY9TOaK cqfQ== X-Gm-Message-State: AC+VfDwURdB/sUl+mrLPxBzklGoYf5r+eBjmrClEHMnB1R0KTUHOh6Ql xDF4FYlWWC9I6zh0tWFUum+UwXDcTtfz X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:a615:63d5:b54e:6919]) (user=mshavit job=sendgmr) by 2002:a81:a7c3:0:b0:568:ee83:d87d with SMTP id e186-20020a81a7c3000000b00568ee83d87dmr903702ywh.5.1686053380117; Tue, 06 Jun 2023 05:09:40 -0700 (PDT) Date: Tue, 6 Jun 2023 20:07:39 +0800 In-Reply-To: <20230606120854.4170244-1-mshavit@google.com> Mime-Version: 1.0 References: <20230606120854.4170244-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.rc0.172.g3f132b7071-goog Message-ID: <20230606120854.4170244-4-mshavit@google.com> Subject: [PATCH v2 03/18] iommu/arm-smmu-v3: Refactor write_strtab_ent From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767955843653123206?= X-GMAIL-MSGID: =?utf-8?q?1767955843653123206?= Explicity keep track of the s1_cfg and s2_cfg that are attached to a master in arm_smmu_master, regardless of whether they are owned by arm_smmu_master, arm_smmu_domain or userspace. Signed-off-by: Michael Shavit --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 37 +++++++++------------ drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 2 ++ 2 files changed, 17 insertions(+), 22 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 023769f5ca79a..d79c6ef5d6ed4 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1269,10 +1269,9 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, */ u64 val = le64_to_cpu(dst[0]); bool ste_live = false; - struct arm_smmu_device *smmu = NULL; + struct arm_smmu_device *smmu = master->smmu; struct arm_smmu_s1_cfg *s1_cfg = NULL; struct arm_smmu_s2_cfg *s2_cfg = NULL; - struct arm_smmu_domain *smmu_domain = NULL; struct arm_smmu_cmdq_ent prefetch_cmd = { .opcode = CMDQ_OP_PREFETCH_CFG, .prefetch = { @@ -1280,24 +1279,10 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, }, }; - if (master) { - smmu_domain = master->domain; - smmu = master->smmu; - } - - if (smmu_domain) { - switch (smmu_domain->stage) { - case ARM_SMMU_DOMAIN_S1: - s1_cfg = &smmu_domain->s1_cfg; - break; - case ARM_SMMU_DOMAIN_S2: - case ARM_SMMU_DOMAIN_NESTED: - s2_cfg = &smmu_domain->s2_cfg; - break; - default: - break; - } - } + if (master->s1_cfg) + s1_cfg = master->s1_cfg; + else if (master->s2_cfg) + s2_cfg = master->s2_cfg; if (val & STRTAB_STE_0_V) { switch (FIELD_GET(STRTAB_STE_0_CFG, val)) { @@ -1319,8 +1304,8 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, val = STRTAB_STE_0_V; /* Bypass/fault */ - if (!smmu_domain || !(s1_cfg || s2_cfg)) { - if (!smmu_domain && disable_bypass) + if (!(s1_cfg || s2_cfg)) { + if (disable_bypass) val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_ABORT); else val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_BYPASS); @@ -2401,6 +2386,8 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) master->domain = NULL; master->ats_enabled = false; + master->s1_cfg = NULL; + master->s2_cfg = NULL; arm_smmu_install_ste_for_dev(master); } @@ -2454,6 +2441,12 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) } master->domain = smmu_domain; + if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { + master->s1_cfg = &smmu_domain->s1_cfg; + } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S2 || + smmu_domain->stage == ARM_SMMU_DOMAIN_NESTED) { + master->s2_cfg = &smmu_domain->s2_cfg; + } /* * The SMMU does not support enabling ATS with bypass. When the STE is diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 053cc14c23969..3c614fbe2b8b9 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -689,6 +689,8 @@ struct arm_smmu_master { struct list_head domain_head; struct arm_smmu_stream *streams; struct arm_smmu_s1_cfg owned_s1_cfg; + struct arm_smmu_s1_cfg *s1_cfg; + struct arm_smmu_s2_cfg *s2_cfg; unsigned int num_streams; bool ats_enabled; bool stall_enabled; From patchwork Tue Jun 6 12:07:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 103770 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp3352921vqr; Tue, 6 Jun 2023 05:23:42 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5LUurm6CanIPA0rL0WHTdG2cYRCSVJI2kgA+ZJDg/gjZGK9PJS+6QUi23y4oFWnGWU+xLj X-Received: by 2002:a05:6358:4e16:b0:123:57ad:62b6 with SMTP id cf22-20020a0563584e1600b0012357ad62b6mr2822741rwb.0.1686054222261; Tue, 06 Jun 2023 05:23:42 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686054222; cv=none; d=google.com; s=arc-20160816; b=nRMzvcWbkCOUr7GZBsDag3X1XOmSxgxVJCYO0LdTOsknVONgoOk4kTO3kpSp1rjzL5 WPCK5Xm3AxHSpTSULIEkMMQ2SUpRI23MMd7+KxaDR4GRJv/JlCW3bwpnZf+9p0+OD12L gXtOraGL4Yndo+dry7FcWZzzH7iJeMvtVSGYsACSufTQOvTYn/hp/7yD3s92XQyj1uur 9bV6y9lTmArtltcrxmLXylp3QAiWKEfamqTbYX1dSnAsX+khVyljEm/PBNI4wPw2Nkit NHfbKgtRbvymbgYtOv7QvKaWoLAOC7MeMAmH6qFrD7/VqmwpTvvdgH8RAr+wA+RsIqrh YYgw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=2jn7i2fkADg5o5hw9/ST6xj0QVfVf89sxP9WIoaewAo=; b=XmccJ5v4WrPUnFq+JclRqXcOFbtBcp7YdK9bIVEaBnLNJQocMXp6Rm9omEbQx7jxi1 8MbkrwabAbOjcMmJIzV0TN4oSWvU5usfCJomIRBTmeq4IavkuYHZGsQ4golTdMPP8TRp BkBOCDeJNLQsA6MGmiuF+aiyY733V3Zh/jXtycRu0fuifADz/fBQCm4OuL+IWQPOU1AP t3HuqmgYCr4VyF/qzethVbKafS+Ls9yhkpnB9rCEFjZM6Bxid2fCWr/RUmqDETkfORSq R8+zhc4Dwy466RQ8FhCs26IaMbbLuTdPUTJZYd+Cls5+HzzVtqGyazh+nZWPVDsjZWzt eUAQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b="3aV/qxiz"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id mj4-20020a17090b368400b00256414fdd5bsi8917234pjb.143.2023.06.06.05.23.29; Tue, 06 Jun 2023 05:23:42 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b="3aV/qxiz"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237297AbjFFMKE (ORCPT + 99 others); Tue, 6 Jun 2023 08:10:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46646 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237031AbjFFMKB (ORCPT ); Tue, 6 Jun 2023 08:10:01 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E254E1708 for ; Tue, 6 Jun 2023 05:09:47 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-bb3a35ba742so854499276.0 for ; Tue, 06 Jun 2023 05:09:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686053387; x=1688645387; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=2jn7i2fkADg5o5hw9/ST6xj0QVfVf89sxP9WIoaewAo=; b=3aV/qxiznuH8GTuMuHr9HvpNdOF3Mj6M/f2Q90XcVu/Pc4u1FsruIXD/E8piJxf1Ld 9qy4d0v7FNE6cU6qEPuO/5fPtlmSHiyOIyexir61uAQrLoGwxi8p5OWNfUmek3+ZSMJ+ 8VPc1vnq7RsBifEkgFpZO+d7+QTxQRcbpYWGXSlwrajLLp5U9n8Gnd93vgQ9OOu2Lr9W bBFvwx0mgkOk8iy4p/loN0TDrX7pUoseXwjcH7b3tWzgwPb7EDDT3PCipUxmbvw64A5a Sp0Cx0m47oHm0wYXpWuCeC1Z9Q6nACUySk/VAqRlc1n5WzufL2dQI3uqSemcShORn4D9 lQKQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686053387; x=1688645387; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=2jn7i2fkADg5o5hw9/ST6xj0QVfVf89sxP9WIoaewAo=; b=IauHP6/6Tqkh7ZgZRWnP7+Y6cEqN9pMYcrRax1zMsLWlnukCgs8U50cHimq8S6N79W a/wIqwJPumiNt3h3VnNIMOnjJ9resF3wW19JAdi9FwdzHOZnwZwD6N09b04gFlIy2lEx jpciz7qa1d54/i2O+JQNUK/Jtatlvc08HOHyQuadaNIv71ivwlUP8jNXc3apWhqXctka KgjZ5eVLMHdiviQjC85/Hcmi5KPxj4W4hpNlY/RylMyCylzZ5UuPMu85PjNiJ5iFhi9n +FU7kZbJBbAYKm9511NuV7hUiM1eaK+eyH+re6SVmeOZPlLHU3h+2TDaLYZRN3csC+4i I6iA== X-Gm-Message-State: AC+VfDxfKjy6+WVQ7obl+ZFx0gNWjBhP/slRi9MZlKLZywpSD3yfSJc3 vkU2Fhol5jdXJ5Arr6fLYmr+gOOUkmzl X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:a615:63d5:b54e:6919]) (user=mshavit job=sendgmr) by 2002:a05:6902:1204:b0:bac:f582:ef18 with SMTP id s4-20020a056902120400b00bacf582ef18mr1089911ybu.5.1686053387100; Tue, 06 Jun 2023 05:09:47 -0700 (PDT) Date: Tue, 6 Jun 2023 20:07:40 +0800 In-Reply-To: <20230606120854.4170244-1-mshavit@google.com> Mime-Version: 1.0 References: <20230606120854.4170244-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.rc0.172.g3f132b7071-goog Message-ID: <20230606120854.4170244-5-mshavit@google.com> Subject: [PATCH v2 04/18] iommu/arm-smmu-v3: Refactor write_ctx_desc From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED, USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767955991782417947?= X-GMAIL-MSGID: =?utf-8?q?1767955991782417947?= Update arm_smmu_write_ctx_desc and downstream functions to be agnostic of the CD table's ownership. Note that in practice, it will only be called in cases where the table is owned by arm_smmu_master. Whether arm_smmu_write_ctx_desc will trigger a sync is also made part of the API. Note that this change isn't a nop refactor since SVA will call arm_smmu_write_ctx_desc in a loop for every master the domain is attached to despite the fact that they all share the same CD table. Note that the next commit ceases this sharing of the s1_cfg which makes that loop necessary. Signed-off-by: Michael Shavit --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 36 ++++++++- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 80 +++++++++++-------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 6 +- 3 files changed, 81 insertions(+), 41 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 968559d625c40..48fa8eb271a45 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -45,10 +45,12 @@ static struct arm_smmu_ctx_desc * arm_smmu_share_asid(struct mm_struct *mm, u16 asid) { int ret; + unsigned long flags; u32 new_asid; struct arm_smmu_ctx_desc *cd; struct arm_smmu_device *smmu; struct arm_smmu_domain *smmu_domain; + struct arm_smmu_master *master; cd = xa_load(&arm_smmu_asid_xa, asid); if (!cd) @@ -80,7 +82,11 @@ arm_smmu_share_asid(struct mm_struct *mm, u16 asid) * be some overlap between use of both ASIDs, until we invalidate the * TLB. */ - arm_smmu_write_ctx_desc(smmu_domain, 0, cd); + spin_lock_irqsave(&smmu_domain->devices_lock, flags); + list_for_each_entry(master, &smmu_domain->devices, domain_head) { + arm_smmu_write_ctx_desc(smmu, master->s1_cfg, master, 0, cd); + } + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); /* Invalidate TLB entries previously associated with that context */ arm_smmu_tlb_inv_asid(smmu, asid); @@ -211,6 +217,8 @@ static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) { struct arm_smmu_mmu_notifier *smmu_mn = mn_to_smmu(mn); struct arm_smmu_domain *smmu_domain = smmu_mn->domain; + struct arm_smmu_master *master; + unsigned long flags; mutex_lock(&sva_lock); if (smmu_mn->cleared) { @@ -222,7 +230,12 @@ static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) * DMA may still be running. Keep the cd valid to avoid C_BAD_CD events, * but disable translation. */ - arm_smmu_write_ctx_desc(smmu_domain, mm->pasid, &quiet_cd); + spin_lock_irqsave(&smmu_domain->devices_lock, flags); + list_for_each_entry(master, &smmu_domain->devices, domain_head) { + arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, + mm->pasid, &quiet_cd); + } + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); arm_smmu_tlb_inv_asid(smmu_domain->smmu, smmu_mn->cd->asid); arm_smmu_atc_inv_domain(smmu_domain, mm->pasid, 0, 0); @@ -248,8 +261,10 @@ arm_smmu_mmu_notifier_get(struct arm_smmu_domain *smmu_domain, struct mm_struct *mm) { int ret; + unsigned long flags; struct arm_smmu_ctx_desc *cd; struct arm_smmu_mmu_notifier *smmu_mn; + struct arm_smmu_master *master; list_for_each_entry(smmu_mn, &smmu_domain->mmu_notifiers, list) { if (smmu_mn->mn.mm == mm) { @@ -279,7 +294,12 @@ arm_smmu_mmu_notifier_get(struct arm_smmu_domain *smmu_domain, goto err_free_cd; } - ret = arm_smmu_write_ctx_desc(smmu_domain, mm->pasid, cd); + spin_lock_irqsave(&smmu_domain->devices_lock, flags); + list_for_each_entry(master, &smmu_domain->devices, domain_head) { + ret = arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, + master, mm->pasid, cd); + } + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); if (ret) goto err_put_notifier; @@ -296,15 +316,23 @@ arm_smmu_mmu_notifier_get(struct arm_smmu_domain *smmu_domain, static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) { + unsigned long flags; struct mm_struct *mm = smmu_mn->mn.mm; struct arm_smmu_ctx_desc *cd = smmu_mn->cd; + struct arm_smmu_master *master; struct arm_smmu_domain *smmu_domain = smmu_mn->domain; if (!refcount_dec_and_test(&smmu_mn->refs)) return; list_del(&smmu_mn->list); - arm_smmu_write_ctx_desc(smmu_domain, mm->pasid, NULL); + + spin_lock_irqsave(&smmu_domain->devices_lock, flags); + list_for_each_entry(master, &smmu_domain->devices, domain_head) { + arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, + mm->pasid, NULL); + } + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); /* * If we went through clear(), we've already invalidated, and no diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index d79c6ef5d6ed4..b6f7cf60f8f3d 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -965,14 +965,13 @@ void arm_smmu_tlb_inv_asid(struct arm_smmu_device *smmu, u16 asid) arm_smmu_cmdq_issue_cmd_with_sync(smmu, &cmd); } -static void arm_smmu_sync_cd(struct arm_smmu_domain *smmu_domain, +/* master may be null */ +static void arm_smmu_sync_cd(struct arm_smmu_master *master, int ssid, bool leaf) { size_t i; - unsigned long flags; - struct arm_smmu_master *master; struct arm_smmu_cmdq_batch cmds; - struct arm_smmu_device *smmu = smmu_domain->smmu; + struct arm_smmu_device *smmu; struct arm_smmu_cmdq_ent cmd = { .opcode = CMDQ_OP_CFGI_CD, .cfgi = { @@ -981,16 +980,15 @@ static void arm_smmu_sync_cd(struct arm_smmu_domain *smmu_domain, }, }; - cmds.num = 0; + if (!master) + return; - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { - for (i = 0; i < master->num_streams; i++) { - cmd.cfgi.sid = master->streams[i].id; - arm_smmu_cmdq_batch_add(smmu, &cmds, &cmd); - } + smmu = master->smmu; + cmds.num = 0; + for (i = 0; i < master->num_streams; i++) { + cmd.cfgi.sid = master->streams[i].id; + arm_smmu_cmdq_batch_add(smmu, &cmds, &cmd); } - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); arm_smmu_cmdq_batch_submit(smmu, &cmds); } @@ -1020,16 +1018,18 @@ static void arm_smmu_write_cd_l1_desc(__le64 *dst, WRITE_ONCE(*dst, cpu_to_le64(val)); } -static __le64 *arm_smmu_get_cd_ptr(struct arm_smmu_domain *smmu_domain, +/* master may be null */ +static __le64 *arm_smmu_get_cd_ptr(struct arm_smmu_device *smmu, + struct arm_smmu_s1_cfg *s1_cfg, + struct arm_smmu_master *master, u32 ssid) { __le64 *l1ptr; unsigned int idx; struct arm_smmu_l1_ctx_desc *l1_desc; - struct arm_smmu_device *smmu = smmu_domain->smmu; - struct arm_smmu_ctx_desc_cfg *cdcfg = &smmu_domain->s1_cfg.cdcfg; + struct arm_smmu_ctx_desc_cfg *cdcfg = &s1_cfg->cdcfg; - if (smmu_domain->s1_cfg.s1fmt == STRTAB_STE_0_S1FMT_LINEAR) + if (s1_cfg->s1fmt == STRTAB_STE_0_S1FMT_LINEAR) return cdcfg->cdtab + ssid * CTXDESC_CD_DWORDS; idx = ssid >> CTXDESC_SPLIT; @@ -1041,13 +1041,21 @@ static __le64 *arm_smmu_get_cd_ptr(struct arm_smmu_domain *smmu_domain, l1ptr = cdcfg->cdtab + idx * CTXDESC_L1_DESC_DWORDS; arm_smmu_write_cd_l1_desc(l1ptr, l1_desc); /* An invalid L1CD can be cached */ - arm_smmu_sync_cd(smmu_domain, ssid, false); + arm_smmu_sync_cd(master, ssid, false); } idx = ssid & (CTXDESC_L2_ENTRIES - 1); return l1_desc->l2ptr + idx * CTXDESC_CD_DWORDS; } -int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, +/* + * master must be provided if a CD sync is required but may be null otherwise + * (such as when the CD table isn't inserted into the STE yet, or is about to + * be detached. + */ +int arm_smmu_write_ctx_desc(struct arm_smmu_device *smmu, + struct arm_smmu_s1_cfg *s1_cfg, + struct arm_smmu_master *master, + int ssid, struct arm_smmu_ctx_desc *cd) { /* @@ -1065,10 +1073,10 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, bool cd_live; __le64 *cdptr; - if (WARN_ON(ssid >= (1 << smmu_domain->s1_cfg.s1cdmax))) + if (WARN_ON(ssid >= (1 << s1_cfg->s1cdmax))) return -E2BIG; - cdptr = arm_smmu_get_cd_ptr(smmu_domain, ssid); + cdptr = arm_smmu_get_cd_ptr(smmu, s1_cfg, master, ssid); if (!cdptr) return -ENOMEM; @@ -1092,11 +1100,11 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, cdptr[3] = cpu_to_le64(cd->mair); /* - * STE is live, and the SMMU might read dwords of this CD in any - * order. Ensure that it observes valid values before reading - * V=1. + * STE may be live, and the SMMU might read dwords of this CD + * in any order. Ensure that it observes valid values before + * reading V=1. */ - arm_smmu_sync_cd(smmu_domain, ssid, true); + arm_smmu_sync_cd(master, ssid, true); val = cd->tcr | #ifdef __BIG_ENDIAN @@ -1108,7 +1116,7 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, FIELD_PREP(CTXDESC_CD_0_ASID, cd->asid) | CTXDESC_CD_0_V; - if (smmu_domain->stall_enabled) + if (s1_cfg->stall_enabled) val |= CTXDESC_CD_0_S; } @@ -1122,7 +1130,7 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, * without first making the structure invalid. */ WRITE_ONCE(cdptr[0], cpu_to_le64(val)); - arm_smmu_sync_cd(smmu_domain, ssid, true); + arm_smmu_sync_cd(master, ssid, true); return 0; } @@ -1136,6 +1144,7 @@ static int arm_smmu_init_s1_cfg(struct arm_smmu_master *master, struct arm_smmu_ctx_desc_cfg *cdcfg = &cfg->cdcfg; cfg->s1cdmax = master->ssid_bits; + cfg->stall_enabled = master->stall_enabled; max_contexts = 1 << cfg->s1cdmax; if (!(smmu->features & ARM_SMMU_FEAT_2_LVL_CDTAB) || @@ -2093,8 +2102,6 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, if (ret) goto out_unlock; - smmu_domain->stall_enabled = master->stall_enabled; - ret = arm_smmu_init_s1_cfg(master, cfg); if (ret) goto out_free_asid; @@ -2110,12 +2117,9 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, CTXDESC_CD_0_TCR_EPD1 | CTXDESC_CD_0_AA64; cd->mair = pgtbl_cfg->arm_lpae_s1_cfg.mair; - /* - * Note that this will end up calling arm_smmu_sync_cd() before - * the master has been added to the devices list for this domain. - * This isn't an issue because the STE hasn't been installed yet. - */ - ret = arm_smmu_write_ctx_desc(smmu_domain, 0, cd); + ret = arm_smmu_write_ctx_desc(smmu, cfg, + NULL /*Not attached to a master yet */, + 0, cd); if (ret) goto out_free_cd_tables; @@ -2386,6 +2390,11 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) master->domain = NULL; master->ats_enabled = false; + if (master->s1_cfg) + arm_smmu_write_ctx_desc( + master->smmu, master->s1_cfg, + NULL /* Skip sync since we detach the CD table next*/, + 0, NULL); master->s1_cfg = NULL; master->s2_cfg = NULL; arm_smmu_install_ste_for_dev(master); @@ -2435,7 +2444,8 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) ret = -EINVAL; goto out_unlock; } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1 && - smmu_domain->stall_enabled != master->stall_enabled) { + smmu_domain->s1_cfg.stall_enabled != + master->stall_enabled) { ret = -EINVAL; goto out_unlock; } diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 3c614fbe2b8b9..00a493442d6f9 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -595,6 +595,7 @@ struct arm_smmu_s1_cfg { struct arm_smmu_ctx_desc_cfg cdcfg; u8 s1fmt; u8 s1cdmax; + bool stall_enabled; }; struct arm_smmu_s2_cfg { @@ -713,7 +714,6 @@ struct arm_smmu_domain { struct mutex init_mutex; /* Protects smmu pointer */ struct io_pgtable_ops *pgtbl_ops; - bool stall_enabled; atomic_t nr_ats_masters; enum arm_smmu_domain_stage stage; @@ -742,7 +742,9 @@ extern struct xarray arm_smmu_asid_xa; extern struct mutex arm_smmu_asid_lock; extern struct arm_smmu_ctx_desc quiet_cd; -int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, +int arm_smmu_write_ctx_desc(struct arm_smmu_device *smmu, + struct arm_smmu_s1_cfg *s1_cfg, + struct arm_smmu_master *smmu_master, int ssid, struct arm_smmu_ctx_desc *cd); void arm_smmu_tlb_inv_asid(struct arm_smmu_device *smmu, u16 asid); void arm_smmu_tlb_inv_range_asid(unsigned long iova, size_t size, int asid, From patchwork Tue Jun 6 12:07:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 103766 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp3349896vqr; Tue, 6 Jun 2023 05:19:08 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4HD42PRyMseK4q2S3UjuYOARhtoLq3ixSIRRWnW9/lPwLs3TL0zVJIvHYQr4ySNarS/g9z X-Received: by 2002:a05:6214:c8c:b0:625:aa49:9ab5 with SMTP id r12-20020a0562140c8c00b00625aa499ab5mr1530896qvr.57.1686053948006; Tue, 06 Jun 2023 05:19:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686053947; cv=none; d=google.com; s=arc-20160816; b=Q3wfxlfAFrXrKogWz9BG8uQIwBk+8d4z/4//6Tab3UBcGqu/7m85RratWX82iVHSDZ n4MSTF1135Mw/BiVea/zI2PszoA1f61N2anSm83Mz3oRAPAiyuFAp8vyAnNO9DznBwtX p6JHLDD7MTkXPmjqDqW0Do/RJJVg+QSCZ5uG6CqpSQxKQDJFPCn6IiHZQ/NoMpa5UlgV zZOSQPvPNkCO2WOdB/bClbtZV5kn++jXLiDbokNA5g/M5OBcgy6aXSCoz6zlvHTq54jd t80YlR4YMuipwV6/PJRvYrqV1bMkp1sIELD50HBz+yip7sSdDnxp6e/UHtN5W4NtEaQ9 kzAw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=Wf1RkoS1TdpddcvtRBe1GBIueZ5xLUPAAm9eoAp49Lo=; b=Ro+IiSRp4PP59TMQVqrc9Dm8MMlDU6l5w87Cpmyp4MlrzBFi9wJKh74rSX05TDfEYF H1omeTvG93yCyhtNy4RpU/uzlZrZ4NHVrPhES5xpHRVXLM3E1QKWCjt5o4oTqZv58ImP jKNw2PRcIfQrEBX24y8pTDR86UUAcEkO0yKAv3HmuNfBpoygad4ZgridpuLAp/3KUSdR VvJ+z0sTtygxtEHq2qasQCNYfC9/VOQApRnJ7WSIZjC4kcse2TXFn4esyebipS5hZGZ9 uCaAEbFotdFfibXwaVkMED1pCnBR6Z1VLDk3WYG7CXahV5upXgTuFKRaw0IQkqp7CPaJ fKfg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=TtzN+i4N; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id u15-20020ad45aaf000000b0061b69d0a7d5si6221517qvg.243.2023.06.06.05.18.53; Tue, 06 Jun 2023 05:19:07 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=TtzN+i4N; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236360AbjFFMKO (ORCPT + 99 others); Tue, 6 Jun 2023 08:10:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46830 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237308AbjFFMKL (ORCPT ); Tue, 6 Jun 2023 08:10:11 -0400 Received: from mail-pl1-x64a.google.com (mail-pl1-x64a.google.com [IPv6:2607:f8b0:4864:20::64a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AF5B410D0 for ; Tue, 6 Jun 2023 05:09:54 -0700 (PDT) Received: by mail-pl1-x64a.google.com with SMTP id d9443c01a7336-1b03057588cso55848405ad.1 for ; Tue, 06 Jun 2023 05:09:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686053394; x=1688645394; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Wf1RkoS1TdpddcvtRBe1GBIueZ5xLUPAAm9eoAp49Lo=; b=TtzN+i4Nv9gGr4efu4ItozcQGiq9M+zfsaBG6ZPDEmf81aQoXMt7jAp6xIL5vWASUy 3JdC1llGmcgJD2YE7TPR/xcb6ID6WzMEjTW4IHN5nSUGXpQVxCJg0LmIKvdtJIn1rMz8 nNNY5NNwnoHArYIpzkOPscZ6R0OkHXOXMVxldeeYAaksx1lr3e53zrPlL256ai4CIH1f zGOuYxToD9xJMrDHMB1iLVaxVc5NERQPjpPr+2JH3aqH2MJQE97/yB9SZbpHuiJhE3a9 QDSw6EFpOJ3nqmmYIKiSlidtEnlVEixjyTGMqYLMAZA9Rv7DXt+j6E8FhEmAHXV3WIXy ZIGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686053394; x=1688645394; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Wf1RkoS1TdpddcvtRBe1GBIueZ5xLUPAAm9eoAp49Lo=; b=Zd5iD4410aa2uEZPLdEGaAAa+25HL1McUcWLO1iXKg8zUsDOKB+7IEv/5HEMZcE88c r8OcVxtFLctdohxNKAWLWAcwP+69JiRHZ2P01EBfpwkZKiSmLniVBI7xuYlf/mTUMNm7 D/m6L7UeFG5A8jr1YVSzqlREG/E780ewaPHibffI0ERBQECocRXqHWdj8EIcrb6SoCD/ ISrrwZbWekw3SQhNGsdIWZXboNut9sXB1thInNi8DS7JazPJPBKkbJQBVmb3DKuVJ01t KDG9YovSnNGAuuiAfHUbK7E6nGnQcTz3O2RDT/qPIc0u3G3fs3cZQg2Ehdp8k3sFwA/s AbAQ== X-Gm-Message-State: AC+VfDxxNzPUSb8s2FzYBJ7efhw5LnCXubk/B+NUtO9RkHGCP3P62dmM 9kD0rBqFbHgEf/gukLWEV7Op1Nk04CBj X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:a615:63d5:b54e:6919]) (user=mshavit job=sendgmr) by 2002:a17:903:32cf:b0:1b0:6a10:1ba1 with SMTP id i15-20020a17090332cf00b001b06a101ba1mr582137plr.13.1686053394148; Tue, 06 Jun 2023 05:09:54 -0700 (PDT) Date: Tue, 6 Jun 2023 20:07:41 +0800 In-Reply-To: <20230606120854.4170244-1-mshavit@google.com> Mime-Version: 1.0 References: <20230606120854.4170244-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.rc0.172.g3f132b7071-goog Message-ID: <20230606120854.4170244-6-mshavit@google.com> Subject: [PATCH v2 05/18] iommu/arm-smmu-v3: Use the master-owned s1_cfg From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767955704707821063?= X-GMAIL-MSGID: =?utf-8?q?1767955704707821063?= Insert CDs for STAGE_1 domains into a CD table owned by the arm_smmu_master. Remove the CD table that was owned by arm_smmu_domain. Signed-off-by: Michael Shavit --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 43 ++++++--------------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 3 -- 2 files changed, 12 insertions(+), 34 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index b6f7cf60f8f3d..08f440fe1da6d 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2065,12 +2065,8 @@ static void arm_smmu_domain_free(struct iommu_domain *domain) /* Free the CD and ASID, if we allocated them */ if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { - struct arm_smmu_s1_cfg *cfg = &smmu_domain->s1_cfg; - /* Prevent SVA from touching the CD while we're freeing it */ mutex_lock(&arm_smmu_asid_lock); - if (cfg->cdcfg.cdtab) - arm_smmu_free_cd_tables(smmu, &cfg->cdcfg); arm_smmu_free_asid(&smmu_domain->cd); mutex_unlock(&arm_smmu_asid_lock); } else { @@ -2082,14 +2078,13 @@ static void arm_smmu_domain_free(struct iommu_domain *domain) kfree(smmu_domain); } -static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, +static int arm_smmu_domain_finalise_cd(struct arm_smmu_domain *smmu_domain, struct arm_smmu_master *master, struct io_pgtable_cfg *pgtbl_cfg) { int ret; u32 asid; struct arm_smmu_device *smmu = smmu_domain->smmu; - struct arm_smmu_s1_cfg *cfg = &smmu_domain->s1_cfg; struct arm_smmu_ctx_desc *cd = &smmu_domain->cd; typeof(&pgtbl_cfg->arm_lpae_s1_cfg.tcr) tcr = &pgtbl_cfg->arm_lpae_s1_cfg.tcr; @@ -2102,10 +2097,6 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, if (ret) goto out_unlock; - ret = arm_smmu_init_s1_cfg(master, cfg); - if (ret) - goto out_free_asid; - cd->asid = (u16)asid; cd->ttbr = pgtbl_cfg->arm_lpae_s1_cfg.ttbr; cd->tcr = FIELD_PREP(CTXDESC_CD_0_TCR_T0SZ, tcr->tsz) | @@ -2117,19 +2108,9 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, CTXDESC_CD_0_TCR_EPD1 | CTXDESC_CD_0_AA64; cd->mair = pgtbl_cfg->arm_lpae_s1_cfg.mair; - ret = arm_smmu_write_ctx_desc(smmu, cfg, - NULL /*Not attached to a master yet */, - 0, cd); - if (ret) - goto out_free_cd_tables; - mutex_unlock(&arm_smmu_asid_lock); return 0; -out_free_cd_tables: - arm_smmu_free_cd_tables(smmu, &cfg->cdcfg); -out_free_asid: - arm_smmu_free_asid(cd); out_unlock: mutex_unlock(&arm_smmu_asid_lock); return ret; @@ -2192,7 +2173,7 @@ static int arm_smmu_domain_finalise(struct iommu_domain *domain, ias = min_t(unsigned long, ias, VA_BITS); oas = smmu->ias; fmt = ARM_64_LPAE_S1; - finalise_stage_fn = arm_smmu_domain_finalise_s1; + finalise_stage_fn = arm_smmu_domain_finalise_cd; break; case ARM_SMMU_DOMAIN_NESTED: case ARM_SMMU_DOMAIN_S2: @@ -2439,20 +2420,20 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) } else if (smmu_domain->smmu != smmu) { ret = -EINVAL; goto out_unlock; - } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1 && - master->ssid_bits != smmu_domain->s1_cfg.s1cdmax) { - ret = -EINVAL; - goto out_unlock; - } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1 && - smmu_domain->s1_cfg.stall_enabled != - master->stall_enabled) { - ret = -EINVAL; - goto out_unlock; } master->domain = smmu_domain; if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { - master->s1_cfg = &smmu_domain->s1_cfg; + master->s1_cfg = &master->owned_s1_cfg; + ret = arm_smmu_write_ctx_desc( + smmu, + master->s1_cfg, NULL /*Not attached to a master yet */, + 0, &smmu_domain->cd); + if (ret) { + master->s1_cfg = NULL; + master->domain = NULL; + goto out_unlock; + } } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S2 || smmu_domain->stage == ARM_SMMU_DOMAIN_NESTED) { master->s2_cfg = &smmu_domain->s2_cfg; diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 00a493442d6f9..dff0fa8345462 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -718,10 +718,7 @@ struct arm_smmu_domain { enum arm_smmu_domain_stage stage; union { - struct { struct arm_smmu_ctx_desc cd; - struct arm_smmu_s1_cfg s1_cfg; - }; struct arm_smmu_s2_cfg s2_cfg; }; From patchwork Tue Jun 6 12:07:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 103768 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp3350309vqr; Tue, 6 Jun 2023 05:19:48 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4Tv+yV5iu9i4r5VuFZrWJDzXkTZ7W5WOy+xmyK1+DL6Z3o8edmhmCKaIlAxSYlQPcupUQC X-Received: by 2002:a05:6214:29e5:b0:626:2b44:40c with SMTP id jv5-20020a05621429e500b006262b44040cmr1582768qvb.59.1686053988586; Tue, 06 Jun 2023 05:19:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686053988; cv=none; d=google.com; s=arc-20160816; b=tDKpKV5bRshiTJhmVVHw90NamFXfHIVxAFaH1IKQQ1CJ7+f75ENWP4zc3mTT0Kq1J2 K4kShh5sodcVOICo75/GmFNRsuUdYrm6D0MzWNgKrcaMs5JCo/gdQMJKy1mcrM5VIR7a YGMY+51a9T336cpfdZP+T0rYcrMFhLOJc+XiVFNoM8TuAvlACBhqY5hKfkmCxdETw3z2 41r3FCPY8znYBezF0M8iwXWRb95iT8UrnLJlYFuiNZkpeAVchW95D2iWAScXtAesdOqQ ANhUUBReGD4hm4+wFg2KstrNZd4XikMvFnv7KOwys4j6IaWTp/ASvx6h/9THmU/T1G9X dsbQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=q3oiLY+ykJwBM19GqpOciDwpUp4BC6bUpIyUkrOw/lU=; b=rT/L78qe9NbMIP6BlJPzXk3WLvwbDOrXxA1T01RG601GMQSd0eZB2vfLqrFLO5b5dc Cd83SMwxnd5s16kTKHkf9hlt+dGtDIYx94SlC2uWmzosTkz6CkrbjUcMb+qMxC5lUJby M9MR7fRvCn4Z2xoojGDdax9uXfo2CTCEdmYPUf2jYA+xNEcecBTPUyDyXYa+aVOyoo/k FS6NyNnr+qsQvuJOgmp2PCxt8uVJdxmYZEV9h1aFTESvPzV3o0ayNnWjcgvk/x9V+M7C jrcyOhGYtP6THDP0UZVo+gl0eK8fAPsD0po/t91kEwKS/GAE/STQm1uMymHcbV1wjLbg xaVA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=3hPQpvgL; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id fn1-20020ad45d61000000b006262021ac11si6242832qvb.258.2023.06.06.05.19.33; Tue, 06 Jun 2023 05:19:48 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=3hPQpvgL; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237383AbjFFMKT (ORCPT + 99 others); Tue, 6 Jun 2023 08:10:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46922 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237031AbjFFMKP (ORCPT ); Tue, 6 Jun 2023 08:10:15 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 94DC610DE for ; Tue, 6 Jun 2023 05:10:01 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-bacfa4ef059so9693846276.2 for ; Tue, 06 Jun 2023 05:10:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686053400; x=1688645400; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=q3oiLY+ykJwBM19GqpOciDwpUp4BC6bUpIyUkrOw/lU=; b=3hPQpvgL+wDxU0OjZQfIfZ4PvVC+sRTH9qlAAPJqZw1xiM2Q3iNaeKDiXdelb0sVio D6xhz8ENSzVt3nnIIlVKdYJfctHXIAa0AVbONky2oo5QejO9Xh6VEvVppH5VVD/R0DTg Rz+4hNgCNRDD7UxQBbLKaWHBeXwxqhF9abdIbDSldq7wOSbdk8rtegg1ed7wG1S+DX4F 9gDbr52SBPiRvOlnYTCDE5jSJQThXBeNqvfSEcx2+4zLLOYwZ5zlaQVcRWkWS8JjZRaJ XFM2GVTbPxXllz89+WNgxWz51t+GrQPq+ZkabafS5wNcTuWm2e98+FU9XPrgilTH4onp qvPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686053400; x=1688645400; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=q3oiLY+ykJwBM19GqpOciDwpUp4BC6bUpIyUkrOw/lU=; b=cPbsJNERGABPpdd+o4HcVPl6bjF5MUhtEj0PGjRKA7TUMDDptgW29F/g4tUzH7pMmu 6KRRH5q0oXVsN/adPkfUbChTH7av5zwX79cvEjfXMEZ3pmfLm1lDgn4Brr+naa3BnGjh WrAI6tn7OIZE0LNR/naPAWpGQu3vGdhYYGVyW9SEoWNoOL+7o0NLgNgFkpKWBSmixhxA jPI6dxcVJf9+HyUaW/HQURTqayShaCM2d7X3vUImAHwzx3+SlSkU7XtD6l9xmbiZ4uEs ZWoZ60RXo5Qpl9yedTM8+xGviDkik0G1iogKPEB1pkEDwYFv+V8utSwPdY0rI3TEHqqh m1QA== X-Gm-Message-State: AC+VfDyTH3vZd3qHGsniocOi32K+emVa18NHI41IX0POYOD74D+z0W+Q 2Qcw77HgjMVZhaxOPtNg8WjDfoDFw6yt X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:a615:63d5:b54e:6919]) (user=mshavit job=sendgmr) by 2002:a05:6902:10c8:b0:ba8:797c:9bc7 with SMTP id w8-20020a05690210c800b00ba8797c9bc7mr1064725ybu.11.1686053400830; Tue, 06 Jun 2023 05:10:00 -0700 (PDT) Date: Tue, 6 Jun 2023 20:07:42 +0800 In-Reply-To: <20230606120854.4170244-1-mshavit@google.com> Mime-Version: 1.0 References: <20230606120854.4170244-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.rc0.172.g3f132b7071-goog Message-ID: <20230606120854.4170244-7-mshavit@google.com> Subject: [PATCH v2 06/18] iommu/arm-smmu-v3: Simplify arm_smmu_enable_ats From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767955747036605180?= X-GMAIL-MSGID: =?utf-8?q?1767955747036605180?= arm_smmu_enable_ats's call to inv_domain would trigger an invalidation for all masters that a domain is attached to everytime it's attached to another ATS-enabled master. It doesn't seem like those invalidations are necessary, and it's easier to reason about arm_smmu_enable_ats if it only issues invalidation commands for the current master. Signed-off-by: Michael Shavit --- v1->v2: Fix commit message wrapping --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 08f440fe1da6d..dc7a59e87a2b4 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2286,7 +2286,7 @@ static void arm_smmu_enable_ats(struct arm_smmu_master *master) pdev = to_pci_dev(master->dev); atomic_inc(&smmu_domain->nr_ats_masters); - arm_smmu_atc_inv_domain(smmu_domain, 0, 0, 0); + arm_smmu_atc_inv_master(master); if (pci_enable_ats(pdev, stu)) dev_err(master->dev, "Failed to enable ATS (STU %zu)\n", stu); } From patchwork Tue Jun 6 12:07:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 103764 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp3349775vqr; Tue, 6 Jun 2023 05:18:58 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ550TnHHxNPC8lFxNCWLVnWxerzlNsTQ+CjjFwGEdn7gXh5+gqzGvW7il0w4Ejcgun1H4YJ X-Received: by 2002:a05:6214:d4d:b0:61b:79ab:7129 with SMTP id 13-20020a0562140d4d00b0061b79ab7129mr2306036qvr.37.1686053937790; Tue, 06 Jun 2023 05:18:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686053937; cv=none; d=google.com; s=arc-20160816; b=nMh+ZQSjO/GhDcaIdwnI0u/Sgo0mrWUH796kAN3jWZwEoJrSFupy/3w25xGFsoz7YK +7Kv+jmusmKpSaghnJAId4L9CC11oyH0QF3iAm7wW1LU+vH0n8+Kk9BiH7XHMemK+UtS 32AMdRvuBoC4JrWJSLHZT8Kh3nk0h7zSjzoCPaIE0l6QGbtX72TL80GO64ba9LfhFgM4 ++2FjfUJtpdyad9riyzKC0hn7xEqLU8xm6HFMY7Is6VCN5xlQAAu5Dtmqf96I4ACdN/j TBVT5Wgo8BN/16XW4wMpWUSd3PvErGnpmrKmIJjz2N0UfsQllhk1siZ+O6e0I1r/ay0R q6sg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=xd2apE9I9Ql686K5E2cINv1GKkymHr8pAPZ24057YLo=; b=E7fBAJspO3FZ8EYIblLE0OyE2iJ4krQB/6WzmHho2ltlo1UAjww2tfC7lCFgB20kJC /qmwI+F+KtiNgBw+mLQR1CJbuUot8FVMZEz4tkyN2umcKUpCHSl3S9z9zw/aCqIbnwSK KShsZTM9DxYafUwSS3h0Poy1qFA+RzYjJgms/8OTgarniC6N7jZT2w1VjArUbGu6f45Z fHg4DMRmjVuaV55Dn6zGaa/ikluVnA4ggc3Sdo2jLd81WixZ5RKmF4I9qP6cAY1WpcqG Pf9HBmTe4Z1PHMvikMi5FrhNlNkurVJH0jWBuqJzIsMMkYigWla6nX8orm5y8IOU2A7H toaA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=dUvBUGJ2; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id gh12-20020a05621429cc00b005f26bab983asi5997598qvb.159.2023.06.06.05.18.43; Tue, 06 Jun 2023 05:18:57 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=dUvBUGJ2; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237420AbjFFMKa (ORCPT + 99 others); Tue, 6 Jun 2023 08:10:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47046 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237299AbjFFMK0 (ORCPT ); Tue, 6 Jun 2023 08:10:26 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 80F0D1703 for ; Tue, 6 Jun 2023 05:10:08 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-bacfa4ef059so9694091276.2 for ; Tue, 06 Jun 2023 05:10:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686053407; x=1688645407; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=xd2apE9I9Ql686K5E2cINv1GKkymHr8pAPZ24057YLo=; b=dUvBUGJ2uTFCpl8ZzYeV2M0aI5C148+z57kDrwYIZVrDrrDOTlwtK/k6da1JB47kly YHAETWdmnBmikZon7aFfK/ZKCgh4Enp6nkCmTtT5GvNqpLkDH2PgvgWfMhbmn6mgG7Kw An5suNQm3aoYqek50TCwOmQWe00ojcep/USlSNuwDmRCo9E8r0d6RG4bQu4fcFT6/OzN 8RUmKOHjiuRYxpF1976rRclQJRTy6zUNQJjRUEDHb/5iXfGsp33uXLP9ViCApsZtuT26 YCEq7EKWHDuJioM1w3nhhLDclVMfd+l5MQuVPq2qMPNzfworQYfG1hk0tMuJ40k2YsfJ NgiA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686053407; x=1688645407; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=xd2apE9I9Ql686K5E2cINv1GKkymHr8pAPZ24057YLo=; b=g1sg38AMd5WXVhHkf/NG4D8iCOpQR4k+ds0nU0zpKc/VIFXh87e8igzoSJ9B6je1vN JkkfNxxKZ5vuGfW7QgJwlkTB3ZYFATNoP6cCzoJgFtMuW67a5hFlwERDqIFqAf1C96q4 /6FFmsnxFYGzNG3zIsxhcm9PN/t60DVCjdah8ctXcay2PdjeBfTh7tZVqS+KcIT8udYB jSX+klS9Y/Hc8AE14s1NO9UntuCxy2fvYzVBFEuQJdYgjrlFP3H6edeB8C5Q4fW9PmyW iAMxFdGoUyes5U15qb4BUz7WwkfYPmrqqdDFDT/FugrEfA8eqjtUxb8knXtK625fTodV RkDw== X-Gm-Message-State: AC+VfDzGiFe6X5PskwJpbdNobIIcALa6womVjIQInIZoOiatclYM42cN xKYNFmUS4rnf3UwyAOMD8YAvdMqJ0UXp X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:a615:63d5:b54e:6919]) (user=mshavit job=sendgmr) by 2002:a05:6902:10c8:b0:ba8:797c:9bc7 with SMTP id w8-20020a05690210c800b00ba8797c9bc7mr1064923ybu.11.1686053407816; Tue, 06 Jun 2023 05:10:07 -0700 (PDT) Date: Tue, 6 Jun 2023 20:07:43 +0800 In-Reply-To: <20230606120854.4170244-1-mshavit@google.com> Mime-Version: 1.0 References: <20230606120854.4170244-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.rc0.172.g3f132b7071-goog Message-ID: <20230606120854.4170244-8-mshavit@google.com> Subject: [PATCH v2 07/18] iommu/arm-smmu-v3: Keep track of attached ssids From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED, USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767955694258887747?= X-GMAIL-MSGID: =?utf-8?q?1767955694258887747?= The arm-smmu-v3 driver keeps track of all masters that a domain is attached to so that it can re-write their STEs when the domain's ASID is upated by SVA. This tracking is also used to invalidate ATCs on all masters that a domain is attached to. This change introduces a new data structures to track all the CD entries that a domain is attached to. This change is a pre-requisite to allow domain attachment on non 0 SSIDs. Signed-off-by: Michael Shavit --- The arm_smmu_atc_inv_domain_ssid function is only temporarily introduced to make these changes atomic, but is eventually removed in latter SVA refactoring patches. v1->v2: Fix arm_smmu_atc_inv_cmd_set_ssid and other cosmetic changes --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 53 +++++++---- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 88 ++++++++++++------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 18 ++-- 3 files changed, 105 insertions(+), 54 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 48fa8eb271a45..d07c08b53c5cf 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -51,6 +51,7 @@ arm_smmu_share_asid(struct mm_struct *mm, u16 asid) struct arm_smmu_device *smmu; struct arm_smmu_domain *smmu_domain; struct arm_smmu_master *master; + struct arm_smmu_attached_domain *attached_domain; cd = xa_load(&arm_smmu_asid_xa, asid); if (!cd) @@ -82,11 +83,14 @@ arm_smmu_share_asid(struct mm_struct *mm, u16 asid) * be some overlap between use of both ASIDs, until we invalidate the * TLB. */ - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { - arm_smmu_write_ctx_desc(smmu, master->s1_cfg, master, 0, cd); + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_for_each_entry(attached_domain, &smmu_domain->attached_domains, + domain_head) { + master = attached_domain->master; + arm_smmu_write_ctx_desc(smmu, master->s1_cfg, master, + attached_domain->ssid, cd); } - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); /* Invalidate TLB entries previously associated with that context */ arm_smmu_tlb_inv_asid(smmu, asid); @@ -210,7 +214,7 @@ static void arm_smmu_mm_invalidate_range(struct mmu_notifier *mn, if (!(smmu_domain->smmu->features & ARM_SMMU_FEAT_BTM)) arm_smmu_tlb_inv_range_asid(start, size, smmu_mn->cd->asid, PAGE_SIZE, false, smmu_domain); - arm_smmu_atc_inv_domain(smmu_domain, mm->pasid, start, size); + arm_smmu_atc_inv_domain_ssid(smmu_domain, mm->pasid, start, size); } static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) @@ -218,6 +222,7 @@ static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) struct arm_smmu_mmu_notifier *smmu_mn = mn_to_smmu(mn); struct arm_smmu_domain *smmu_domain = smmu_mn->domain; struct arm_smmu_master *master; + struct arm_smmu_attached_domain *attached_domain; unsigned long flags; mutex_lock(&sva_lock); @@ -230,15 +235,21 @@ static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) * DMA may still be running. Keep the cd valid to avoid C_BAD_CD events, * but disable translation. */ - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { - arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, - mm->pasid, &quiet_cd); + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_for_each_entry(attached_domain, &smmu_domain->attached_domains, + domain_head) { + master = attached_domain->master; + /* + * SVA domains piggyback on the attached_domain with SSID 0. + */ + if (attached_domain->ssid == 0) + arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, + master, mm->pasid, &quiet_cd); } - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); arm_smmu_tlb_inv_asid(smmu_domain->smmu, smmu_mn->cd->asid); - arm_smmu_atc_inv_domain(smmu_domain, mm->pasid, 0, 0); + arm_smmu_atc_inv_domain_ssid(smmu_domain, mm->pasid, 0, 0); smmu_mn->cleared = true; mutex_unlock(&sva_lock); @@ -265,6 +276,7 @@ arm_smmu_mmu_notifier_get(struct arm_smmu_domain *smmu_domain, struct arm_smmu_ctx_desc *cd; struct arm_smmu_mmu_notifier *smmu_mn; struct arm_smmu_master *master; + struct arm_smmu_attached_domain *attached_domain; list_for_each_entry(smmu_mn, &smmu_domain->mmu_notifiers, list) { if (smmu_mn->mn.mm == mm) { @@ -294,12 +306,14 @@ arm_smmu_mmu_notifier_get(struct arm_smmu_domain *smmu_domain, goto err_free_cd; } - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_for_each_entry(attached_domain, &smmu_domain->attached_domains, + domain_head) { + master = attached_domain->master; ret = arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, mm->pasid, cd); } - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); if (ret) goto err_put_notifier; @@ -319,6 +333,7 @@ static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) unsigned long flags; struct mm_struct *mm = smmu_mn->mn.mm; struct arm_smmu_ctx_desc *cd = smmu_mn->cd; + struct arm_smmu_attached_domain *attached_domain; struct arm_smmu_master *master; struct arm_smmu_domain *smmu_domain = smmu_mn->domain; @@ -327,12 +342,14 @@ static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) list_del(&smmu_mn->list); - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_for_each_entry(attached_domain, &smmu_domain->attached_domains, + domain_head) { + master = attached_domain->master; arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, mm->pasid, NULL); } - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); /* * If we went through clear(), we've already invalidated, and no @@ -340,7 +357,7 @@ static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) */ if (!smmu_mn->cleared) { arm_smmu_tlb_inv_asid(smmu_domain->smmu, cd->asid); - arm_smmu_atc_inv_domain(smmu_domain, mm->pasid, 0, 0); + arm_smmu_atc_inv_domain_ssid(smmu_domain, mm->pasid, 0, 0); } /* Frees smmu_mn */ diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index dc7a59e87a2b4..70580ba7065dc 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1711,7 +1711,14 @@ static irqreturn_t arm_smmu_combined_irq_handler(int irq, void *dev) } static void -arm_smmu_atc_inv_to_cmd(int ssid, unsigned long iova, size_t size, +arm_smmu_atc_inv_cmd_set_ssid(int ssid, struct arm_smmu_cmdq_ent *cmd) +{ + cmd->substream_valid = !!ssid; + cmd->atc.ssid = ssid; +} + +static void +arm_smmu_atc_inv_to_cmd(unsigned long iova, size_t size, struct arm_smmu_cmdq_ent *cmd) { size_t log2_span; @@ -1736,8 +1743,8 @@ arm_smmu_atc_inv_to_cmd(int ssid, unsigned long iova, size_t size, */ *cmd = (struct arm_smmu_cmdq_ent) { .opcode = CMDQ_OP_ATC_INV, - .substream_valid = !!ssid, - .atc.ssid = ssid, + .substream_valid = false, + .atc.ssid = 0, }; if (!size) { @@ -1783,8 +1790,7 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) struct arm_smmu_cmdq_ent cmd; struct arm_smmu_cmdq_batch cmds; - arm_smmu_atc_inv_to_cmd(0, 0, 0, &cmd); - + arm_smmu_atc_inv_to_cmd(0, 0, &cmd); cmds.num = 0; for (i = 0; i < master->num_streams; i++) { cmd.atc.sid = master->streams[i].id; @@ -1794,13 +1800,19 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) return arm_smmu_cmdq_batch_submit(master->smmu, &cmds); } -int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, - unsigned long iova, size_t size) +/* + * If ssid is non-zero, issue atc invalidations with the given ssid instead of + * the one the domain is attached to. This is used by SVA since it's pasid + * attachments aren't recorded in smmu_domain yet. + */ +int arm_smmu_atc_inv_domain_ssid(struct arm_smmu_domain *smmu_domain, int ssid, + unsigned long iova, size_t size) { int i; unsigned long flags; struct arm_smmu_cmdq_ent cmd; struct arm_smmu_master *master; + struct arm_smmu_attached_domain *attached_domain; struct arm_smmu_cmdq_batch cmds; if (!(smmu_domain->smmu->features & ARM_SMMU_FEAT_ATS)) @@ -1823,25 +1835,37 @@ int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, if (!atomic_read(&smmu_domain->nr_ats_masters)) return 0; - arm_smmu_atc_inv_to_cmd(ssid, iova, size, &cmd); + arm_smmu_atc_inv_to_cmd(iova, size, &cmd); cmds.num = 0; - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_for_each_entry(attached_domain, &smmu_domain->attached_domains, + domain_head) { + master = attached_domain->master; if (!master->ats_enabled) continue; + if (ssid != 0) + arm_smmu_atc_inv_cmd_set_ssid(ssid, &cmd); + else + arm_smmu_atc_inv_cmd_set_ssid(attached_domain->ssid, &cmd); for (i = 0; i < master->num_streams; i++) { cmd.atc.sid = master->streams[i].id; arm_smmu_cmdq_batch_add(smmu_domain->smmu, &cmds, &cmd); } } - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); return arm_smmu_cmdq_batch_submit(smmu_domain->smmu, &cmds); } +int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, + unsigned long iova, size_t size) +{ + return arm_smmu_atc_inv_domain_ssid(smmu_domain, 0, iova, size); +} + /* IO_PGTABLE API */ static void arm_smmu_tlb_inv_context(void *cookie) { @@ -1863,7 +1887,7 @@ static void arm_smmu_tlb_inv_context(void *cookie) cmd.tlbi.vmid = smmu_domain->s2_cfg.vmid; arm_smmu_cmdq_issue_cmd_with_sync(smmu, &cmd); } - arm_smmu_atc_inv_domain(smmu_domain, 0, 0, 0); + arm_smmu_atc_inv_domain(smmu_domain, 0, 0); } static void __arm_smmu_tlb_inv_range(struct arm_smmu_cmdq_ent *cmd, @@ -1951,7 +1975,7 @@ static void arm_smmu_tlb_inv_range_domain(unsigned long iova, size_t size, * Unfortunately, this can't be leaf-only since we may have * zapped an entire table. */ - arm_smmu_atc_inv_domain(smmu_domain, 0, iova, size); + arm_smmu_atc_inv_domain(smmu_domain, iova, size); } void arm_smmu_tlb_inv_range_asid(unsigned long iova, size_t size, int asid, @@ -2031,8 +2055,8 @@ static struct iommu_domain *arm_smmu_domain_alloc(unsigned type) return NULL; mutex_init(&smmu_domain->init_mutex); - INIT_LIST_HEAD(&smmu_domain->devices); - spin_lock_init(&smmu_domain->devices_lock); + INIT_LIST_HEAD(&smmu_domain->attached_domains); + spin_lock_init(&smmu_domain->attached_domains_lock); INIT_LIST_HEAD(&smmu_domain->mmu_notifiers); return &smmu_domain->domain; @@ -2270,12 +2294,12 @@ static bool arm_smmu_ats_supported(struct arm_smmu_master *master) return dev_is_pci(dev) && pci_ats_supported(to_pci_dev(dev)); } -static void arm_smmu_enable_ats(struct arm_smmu_master *master) +static void arm_smmu_enable_ats(struct arm_smmu_master *master, + struct arm_smmu_domain *smmu_domain) { size_t stu; struct pci_dev *pdev; struct arm_smmu_device *smmu = master->smmu; - struct arm_smmu_domain *smmu_domain = master->domain; /* Don't enable ATS at the endpoint if it's not enabled in the STE */ if (!master->ats_enabled) @@ -2291,10 +2315,9 @@ static void arm_smmu_enable_ats(struct arm_smmu_master *master) dev_err(master->dev, "Failed to enable ATS (STU %zu)\n", stu); } -static void arm_smmu_disable_ats(struct arm_smmu_master *master) +static void arm_smmu_disable_ats(struct arm_smmu_master *master, + struct arm_smmu_domain *smmu_domain) { - struct arm_smmu_domain *smmu_domain = master->domain; - if (!master->ats_enabled) return; @@ -2358,18 +2381,17 @@ static void arm_smmu_disable_pasid(struct arm_smmu_master *master) static void arm_smmu_detach_dev(struct arm_smmu_master *master) { unsigned long flags; - struct arm_smmu_domain *smmu_domain = master->domain; + struct arm_smmu_domain *smmu_domain = master->non_pasid_domain.domain; if (!smmu_domain) return; - arm_smmu_disable_ats(master); + arm_smmu_disable_ats(master, smmu_domain); - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_del(&master->domain_head); - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_del(&master->non_pasid_domain.domain_head); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); - master->domain = NULL; master->ats_enabled = false; if (master->s1_cfg) arm_smmu_write_ctx_desc( @@ -2378,6 +2400,7 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) 0, NULL); master->s1_cfg = NULL; master->s2_cfg = NULL; + master->non_pasid_domain.domain = NULL; arm_smmu_install_ste_for_dev(master); } @@ -2422,7 +2445,6 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) goto out_unlock; } - master->domain = smmu_domain; if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { master->s1_cfg = &master->owned_s1_cfg; ret = arm_smmu_write_ctx_desc( @@ -2449,13 +2471,17 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) if (smmu_domain->stage != ARM_SMMU_DOMAIN_BYPASS) master->ats_enabled = arm_smmu_ats_supported(master); + master->non_pasid_domain.master = master; + master->non_pasid_domain.domain = smmu_domain; + master->non_pasid_domain.ssid = 0; arm_smmu_install_ste_for_dev(master); - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_add(&master->domain_head, &smmu_domain->devices); - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_add(&master->non_pasid_domain.domain_head, + &smmu_domain->attached_domains); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); - arm_smmu_enable_ats(master); + arm_smmu_enable_ats(master, smmu_domain); out_unlock: mutex_unlock(&smmu_domain->init_mutex); diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index dff0fa8345462..6929590530367 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -682,11 +682,19 @@ struct arm_smmu_stream { struct rb_node node; }; +/* List of {masters, ssid} that a domain is attached to */ +struct arm_smmu_attached_domain { + struct list_head domain_head; + struct arm_smmu_domain *domain; + struct arm_smmu_master *master; + int ssid; +}; + /* SMMU private data for each master */ struct arm_smmu_master { struct arm_smmu_device *smmu; struct device *dev; - struct arm_smmu_domain *domain; + struct arm_smmu_attached_domain non_pasid_domain; struct list_head domain_head; struct arm_smmu_stream *streams; struct arm_smmu_s1_cfg owned_s1_cfg; @@ -724,8 +732,8 @@ struct arm_smmu_domain { struct iommu_domain domain; - struct list_head devices; - spinlock_t devices_lock; + struct list_head attached_domains; + spinlock_t attached_domains_lock; struct list_head mmu_notifiers; }; @@ -748,8 +756,8 @@ void arm_smmu_tlb_inv_range_asid(unsigned long iova, size_t size, int asid, size_t granule, bool leaf, struct arm_smmu_domain *smmu_domain); bool arm_smmu_free_asid(struct arm_smmu_ctx_desc *cd); -int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, - unsigned long iova, size_t size); +int arm_smmu_atc_inv_domain_ssid(struct arm_smmu_domain *smmu_domain, int ssid, + unsigned long iova, size_t size); #ifdef CONFIG_ARM_SMMU_V3_SVA bool arm_smmu_sva_supported(struct arm_smmu_device *smmu); From patchwork Tue Jun 6 12:07:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 103774 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp3354597vqr; Tue, 6 Jun 2023 05:26:45 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ48k+uG/el758qGY8MSdeboRPYyCqrLeQScZzW006dbrSlc+YPhm9YNlUdaLMIWMpet5IPb X-Received: by 2002:aca:100e:0:b0:398:58c5:b853 with SMTP id 14-20020aca100e000000b0039858c5b853mr2191505oiq.47.1686054405416; Tue, 06 Jun 2023 05:26:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686054405; cv=none; d=google.com; s=arc-20160816; b=QhzOW4ZIu5bid62KQaLCy2IQEtIV8/P5LfwXsbGLNvfYFzZm1CRAnbaNpBEbU3Za9Z rM58Ojd8IFfbZh5RtnGytSBIGJ2krXr7qBvJKAmZzQMNNaVQkjFtPOf/VItuVKGuxWFi gaGYY72kXqSOv9sdfquI23FlrLQEgshcIScaOHZ8rxXBZT3hDBKjnhevqKnWVs4gYWzd tDgFMycltVj8hqQr3OkR16D76Ax9b+ES1WsMBoMBPmqiQSRntbrBlyBWlolM9jywjWi3 XEy6lJq+7Srk0Zek8bu2zjo+Etn8aYPOhDbbPhT1zS8/3HIgWWb2+tztrbqZOOLbajzW Y5fA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=7Q0Iu14cVZzOpWwfTMTOhF5XX3kNov0fzYqifn+D0GY=; b=gQ7VvzyfLlLrydMUuH0b+EEjZPdfpn8URG1DjG+Hok1DLb/qZO+EcMwSC4pytgfy99 Z5MApNYbw7qC/hOXfWqL64LReOZUAs/kAi5f9kRvSBSdUwyFNcvYu368Rr+HbKNqZPI1 stw9BQ6ZSQrXXJStql9zCxU1j55iVwwxLmAy3j8YNAsyebzzmHQR+XpuzRX6KKCMvidi OgzuiiC1uyVrVAAxg62/3mevo3fxQQl/uebB8QWIDD1AgVn/KlbYm3wBFDXR5rr2ei4q cpwLKbkihtiLDtU1xIsiEjMaO5zS65VL1lf/X20ItGixgGZRWdvWvH5T4pYXy5PPj7L9 BvMg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=Qj9FF129; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id i15-20020aa796ef000000b0064d43f63156si6827390pfq.354.2023.06.06.05.26.33; Tue, 06 Jun 2023 05:26:45 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=Qj9FF129; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237368AbjFFMKl (ORCPT + 99 others); Tue, 6 Jun 2023 08:10:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47300 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237398AbjFFMKf (ORCPT ); Tue, 6 Jun 2023 08:10:35 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AB3371725 for ; Tue, 6 Jun 2023 05:10:15 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-bb05f0e6ef9so8995233276.1 for ; Tue, 06 Jun 2023 05:10:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686053415; x=1688645415; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=7Q0Iu14cVZzOpWwfTMTOhF5XX3kNov0fzYqifn+D0GY=; b=Qj9FF129nefazp7rsEoZgyUDmzsW8Dl4oEfnoMt+BCvxl+NTWVy4OR8Ok5LqOWKaUx OH7Ra2Dk8NTJ1kmkTIH+GHrVJ6MooFUGr5V25M/4AriM4Cz8/19OmBMafzikZWYs40Qm Fm13HyQurOg1VgQWnI/LXtDi8eOJhMzFYZLUgVv4ltWLJ/xFS52hTWzsa+AUMGoRw701 UWMML1rZU2decc6dYLad29aIonKRI1Ygob0M9Vxe7Oe5pnNn9zl3Yxvx8/CaB3mrwLhR DUfVGLtQJ2r9UdgtgOlHuFp9zLZ1wVr8e0hzTvXgagApF3nwANZv4uRM+s4bJO8+8g/o gfAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686053415; x=1688645415; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=7Q0Iu14cVZzOpWwfTMTOhF5XX3kNov0fzYqifn+D0GY=; b=VYHCbEWVJpN5O6g+AMBckNDi9kEM0xIeZiUqumgl3m9mfeG3T/YfLukC0msH8RzCI9 1He20qOrOF6x9C6P7zglF84buVXTLLYY670ns1dYGPrVRhiCiD6qVz2L2rAqfQTik5sM YkpptGtEAQ+QOL+JBSG0uM6ATeVr3c8dja0hV30rzHwmHXGa9IJoHD55yyYJUz/yfNK6 NjBjbB/9/9J6BzaH9mo6TVR2eJGox1+2QuEM4QzSilnGIzAUxYa8mKdSm63AyCTCCjk+ wSxUNoV6o9n6Pbg1eqpLJ45X5LR2XS9Z0Hr9b/fWly/PVdl0TpiQq5+yAtJeEr8jiuvt 9syg== X-Gm-Message-State: AC+VfDyuBa0TjFhPmKQSYFqvsJ6Q8al7l9nWaE3celAS1K/pKQuW5SSB TkQPWsd8V43XjFcvAzfKWFMaRuKKBP7c X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:a615:63d5:b54e:6919]) (user=mshavit job=sendgmr) by 2002:a25:7d02:0:b0:ba8:1e72:2c88 with SMTP id y2-20020a257d02000000b00ba81e722c88mr941602ybc.13.1686053414834; Tue, 06 Jun 2023 05:10:14 -0700 (PDT) Date: Tue, 6 Jun 2023 20:07:44 +0800 In-Reply-To: <20230606120854.4170244-1-mshavit@google.com> Mime-Version: 1.0 References: <20230606120854.4170244-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.rc0.172.g3f132b7071-goog Message-ID: <20230606120854.4170244-9-mshavit@google.com> Subject: [PATCH v2 08/18] iommu/arm-smmu-v3: Add helper for atc invalidation From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767956184274978347?= X-GMAIL-MSGID: =?utf-8?q?1767956184274978347?= This will be used to invalidate ATC entries made on an SSID for a master when detaching a domain with pasid. Signed-off-by: Michael Shavit --- v1->v2: Make use of arm_smmu_atc_inv_cmd_set_ssid --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 70580ba7065dc..176013bb974b8 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1784,13 +1784,15 @@ arm_smmu_atc_inv_to_cmd(unsigned long iova, size_t size, cmd->atc.size = log2_span; } -static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) +static int arm_smmu_atc_inv_master_ssid(struct arm_smmu_master *master, + int ssid) { int i; struct arm_smmu_cmdq_ent cmd; struct arm_smmu_cmdq_batch cmds; arm_smmu_atc_inv_to_cmd(0, 0, &cmd); + arm_smmu_atc_inv_cmd_set_ssid(ssid, &cmd); cmds.num = 0; for (i = 0; i < master->num_streams; i++) { cmd.atc.sid = master->streams[i].id; @@ -1800,6 +1802,11 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) return arm_smmu_cmdq_batch_submit(master->smmu, &cmds); } +static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) +{ + return arm_smmu_atc_inv_master_ssid(master, 0); +} + /* * If ssid is non-zero, issue atc invalidations with the given ssid instead of * the one the domain is attached to. This is used by SVA since it's pasid From patchwork Tue Jun 6 12:07:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 103762 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp3349617vqr; Tue, 6 Jun 2023 05:18:43 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ61eKGQZyExtRTf1T1cZCGbqbDaWoX4O7XKY/sYRxtOrEGBzNolFXblpR0QNMgScwC9H5q/ X-Received: by 2002:a17:903:284:b0:1af:b682:7a78 with SMTP id j4-20020a170903028400b001afb6827a78mr1472190plr.52.1686053922932; Tue, 06 Jun 2023 05:18:42 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686053922; cv=none; d=google.com; s=arc-20160816; b=z3G3u7A09cXQEs6Asb46SbFQaof0bX/OMA7Gu4d8W4ZTnvHfCVEV//6SkeNr3fzBQf jzYqbxZmMvaGotvgNvUbJKbogI5M34oVrlf4hsv5PxhawCz6uSeGwN1eMMZDvSCFlg8g mjFbf7Xjt4EBfTW6WckexG8Of4y7smnHZg+GUgBhK88C6WoMup+N0FyuWJRjlMmeaTLy pgnoY89OHOQNHKN1UpoeWrkjOxgTMmn+lkp3LdrtOoTaCF+zCt7/jg++jiBBOzS5EPUU QY02CFyQIzbZpodcl21J6jHJLxlWGv44knviKOmIt8NCrtNIu8R5nOLRbOpGUPVbla8F 49PA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=3LYjw95yD0sZH0rb5+NoUTso90gxnubtDPkmj/Mi1ew=; b=LwX6TPSON84SCfNCgg+vRRPHHRBk87U1PDK5d6sP1XChig2waA3nlwTEDsDoN1R+XW IEEwLb6+w1viPBqdyXchAis/Ncm+k+yPYG3zYjCXnMsdUhQ1AuIrkzL16824Kd+xERFR TCk8KwKqCvPIwnySakksIEb/MODaJx+yuVsBZJgBu9/Yf+THpGSFkYgEqpH/wK5iDhKh 4gDH+u4NOZtp4m4IeUoWxO2O14nmaNdxhgxyFq+kApAxalcY0mNAvNdZx/eAeymXldTx R6clEYvIEGmn/arMYDwgNgY1EsWA6V7OvSl2vaaOI/lan6r56ScUQVu9/kwUwiycCTQs IYeA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b="bZ86/wFT"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id e3-20020a17090301c300b001b04c797f7csi7420256plh.210.2023.06.06.05.18.30; Tue, 06 Jun 2023 05:18:42 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b="bZ86/wFT"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237327AbjFFMKv (ORCPT + 99 others); Tue, 6 Jun 2023 08:10:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47424 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237401AbjFFMKq (ORCPT ); Tue, 6 Jun 2023 08:10:46 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3944B1988 for ; Tue, 6 Jun 2023 05:10:22 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-bacfa4ef059so9694513276.2 for ; Tue, 06 Jun 2023 05:10:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686053422; x=1688645422; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=3LYjw95yD0sZH0rb5+NoUTso90gxnubtDPkmj/Mi1ew=; b=bZ86/wFTbj2wtUpoCgOw4xZy8dz24Kk0iZH7f3z3LZSNyHenlrwf03yuk8k4lY8GhZ y1ArvYVhRcMz1IOi04omxYB0mcrRZI5V09XLi+Sa8QGGBr6VyEjj2P52WgwNdQQpIDLp nDqySVm3Rebb+dFrdx1Xw7MsWcJz0Q8EW3QqgA9BQUlw/qyzisMLaUBYsB+Rt9CUiT+y 4cEYoghynadDQo5J3wLuTsDPNNGXwnpsUhMWqgm+3i6dZ6/Y0Ok5gDdXY8KyIijuVa8o ZL98lanUHXZy3CyBf3yE134mXtHwlcsYxsA9IjwBUkMQpmTv+Plz3nJMC+BTxI/sUXN5 g3dg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686053422; x=1688645422; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=3LYjw95yD0sZH0rb5+NoUTso90gxnubtDPkmj/Mi1ew=; b=USci2fZ/QUKUKPV/RBNI/h2ZHYtbg5BjuxL7tqy0bLgLXO+vBRVeh7O5zO+jtqqiPQ eAGv7mucQhw7smuNPQSB3NbqRjU0sHxGyO9cdtYrV967jO6C3k69QVNQEargiEhhWeg7 DsIA3qCTDd8t+cPtxagFwzTmnTLo2l0JxN/CEc64PDUHYP9j3MYGzU87gkjazhXXCpG8 XaQw4TPQyO73ESbINU3Ba1l7GQbOQwvWmvcHEpKiML3uqFuDHOhpg5GJMAZaS6AeSLrJ aANcyRFOyC1fS9/jyex8ro853KSZXcoBHK7k1pmLT0074xyJ1/nigQ9mxYh3z96v2jf7 E5Aw== X-Gm-Message-State: AC+VfDzUwRahvRn1up20IzczVs0i5XUJZUQ8lBrdSiVbNKnewRwM/8As GNyQ345pwfAS53EHnERlT1BaDRGIR2Oh X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:a615:63d5:b54e:6919]) (user=mshavit job=sendgmr) by 2002:a05:6902:161e:b0:ba8:456a:fac7 with SMTP id bw30-20020a056902161e00b00ba8456afac7mr1108645ybb.0.1686053422637; Tue, 06 Jun 2023 05:10:22 -0700 (PDT) Date: Tue, 6 Jun 2023 20:07:45 +0800 In-Reply-To: <20230606120854.4170244-1-mshavit@google.com> Mime-Version: 1.0 References: <20230606120854.4170244-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.rc0.172.g3f132b7071-goog Message-ID: <20230606120854.4170244-10-mshavit@google.com> Subject: [PATCH v2 09/18] iommu/arm-smmu-v3: Implement set_dev_pasid From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767955678475240108?= X-GMAIL-MSGID: =?utf-8?q?1767955678475240108?= This change enables the use of the iommu_attach_dev_pasid API for UNMANAGED domains. The primary use-case is to allow in-kernel users of the iommu API to manage domains with PASID. This change also allows for future support of pasid in the DMA api. Signed-off-by: Michael Shavit --- v1->v2: Add missing atc invalidation when detaching with pasid --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 168 +++++++++++++++++--- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 1 + 2 files changed, 149 insertions(+), 20 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 176013bb974b8..a6fa56585c219 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2173,6 +2173,10 @@ static int arm_smmu_domain_finalise_s2(struct arm_smmu_domain *smmu_domain, return 0; } +/* + * master may be null for domain types that are finalized before being attached + * to a master. + */ static int arm_smmu_domain_finalise(struct iommu_domain *domain, struct arm_smmu_master *master) { @@ -2369,6 +2373,11 @@ static int arm_smmu_enable_pasid(struct arm_smmu_master *master) return 0; } +static bool arm_smmu_master_has_pasid_domains(struct arm_smmu_master *master) +{ + return master->nr_attached_pasid_domains > 0; +} + static void arm_smmu_disable_pasid(struct arm_smmu_master *master) { struct pci_dev *pdev; @@ -2411,6 +2420,28 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) arm_smmu_install_ste_for_dev(master); } +/* + * Once attached for the first time, a domain can no longer be attached to any + * master with a distinct upstream SMMU. + */ +static int arm_smmu_prepare_domain_for_smmu(struct arm_smmu_device *smmu, + struct arm_smmu_domain *smmu_domain) +{ + int ret = 0; + + mutex_lock(&smmu_domain->init_mutex); + if (!smmu_domain->smmu) { + smmu_domain->smmu = smmu; + ret = arm_smmu_domain_finalise(&smmu_domain->domain, NULL); + if (ret) + smmu_domain->smmu = NULL; + } else if (smmu_domain->smmu != smmu) { + ret = -EINVAL; + } + mutex_unlock(&smmu_domain->init_mutex); + return ret; +} + static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) { int ret = 0; @@ -2426,6 +2457,10 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) master = dev_iommu_priv_get(dev); smmu = master->smmu; + ret = arm_smmu_prepare_domain_for_smmu(smmu, smmu_domain); + if (ret) + return ret; + /* * Checking that SVA is disabled ensures that this device isn't bound to * any mm, and can be safely detached from its old domain. Bonds cannot @@ -2436,22 +2471,18 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) return -EBUSY; } - arm_smmu_detach_dev(master); - - mutex_lock(&smmu_domain->init_mutex); - - if (!smmu_domain->smmu) { - smmu_domain->smmu = smmu; - ret = arm_smmu_domain_finalise(domain, master); - if (ret) { - smmu_domain->smmu = NULL; - goto out_unlock; - } - } else if (smmu_domain->smmu != smmu) { - ret = -EINVAL; - goto out_unlock; + /* + * Attaching a bypass or stage 2 domain would break any domains attached + * with pasid. Attaching an S1 domain should be feasible but requires + * more complicated logic to handle. + */ + if (arm_smmu_master_has_pasid_domains(master)) { + dev_err(dev, "cannot attach - domain attached with pasid\n"); + return -EBUSY; } + arm_smmu_detach_dev(master); + if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { master->s1_cfg = &master->owned_s1_cfg; ret = arm_smmu_write_ctx_desc( @@ -2460,8 +2491,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) 0, &smmu_domain->cd); if (ret) { master->s1_cfg = NULL; - master->domain = NULL; - goto out_unlock; + return ret; } } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S2 || smmu_domain->stage == ARM_SMMU_DOMAIN_NESTED) { @@ -2490,11 +2520,75 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) arm_smmu_enable_ats(master, smmu_domain); -out_unlock: - mutex_unlock(&smmu_domain->init_mutex); return ret; } +static int arm_smmu_set_dev_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid) +{ + int ret = 0; + unsigned long flags; + struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); + struct arm_smmu_device *smmu; + struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); + struct arm_smmu_attached_domain *attached_domain; + struct arm_smmu_master *master; + + if (!fwspec) + return -ENOENT; + + master = dev_iommu_priv_get(dev); + smmu = master->smmu; + + ret = arm_smmu_prepare_domain_for_smmu(smmu, smmu_domain); + if (ret) + return ret; + + if (pasid == 0) { + dev_err(dev, "pasid 0 is reserved for the device's primary domain\n"); + return -ENODEV; + } + + if (smmu_domain->stage != ARM_SMMU_DOMAIN_S1) { + dev_err(dev, "set_dev_pasid only supports stage 1 domains\n"); + return -EINVAL; + } + + if (!master->s1_cfg || master->s2_cfg) + return -EBUSY; + + attached_domain = kzalloc(sizeof(*attached_domain), GFP_KERNEL); + if (!attached_domain) + return -ENOMEM; + + attached_domain->master = master; + attached_domain->domain = smmu_domain; + attached_domain->ssid = pasid; + + master->nr_attached_pasid_domains += 1; + /* + * arm_smmu_share_asid may update the cd's asid value and write the + * ctx_desc for every attached_domains in the list. There's a potential + * race here regardless of whether we first write the ctx_desc or + * first insert into the domain's list. Grabbing the asic_lock prevents + * SVA from changing the cd's ASID while the cd is being attached. + */ + mutex_lock(&arm_smmu_asid_lock); + ret = arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, + pasid, &smmu_domain->cd); + if (ret) { + mutex_unlock(&arm_smmu_asid_lock); + kfree(attached_domain); + } + + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_add(&attached_domain->domain_head, &smmu_domain->attached_domains); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); + mutex_unlock(&arm_smmu_asid_lock); + + return 0; +} + static int arm_smmu_map_pages(struct iommu_domain *domain, unsigned long iova, phys_addr_t paddr, size_t pgsize, size_t pgcount, int prot, gfp_t gfp, size_t *mapped) @@ -2740,6 +2834,15 @@ static void arm_smmu_release_device(struct device *dev) if (WARN_ON(arm_smmu_master_sva_enabled(master))) iopf_queue_remove_device(master->smmu->evtq.iopf, dev); + if (WARN_ON(master->nr_attached_pasid_domains != 0)) { + /* + * TODO: Do we need to handle this case? + * This requires a mechanism to obtain all the pasid domains + * that this master is attached to so that we can clean up the + * domain's attached_domain list. + */ + } + arm_smmu_detach_dev(master); arm_smmu_free_cd_tables(master->smmu, &master->owned_s1_cfg.cdcfg); arm_smmu_disable_pasid(master); @@ -2875,12 +2978,36 @@ static int arm_smmu_def_domain_type(struct device *dev) static void arm_smmu_remove_dev_pasid(struct device *dev, ioasid_t pasid) { struct iommu_domain *domain; + struct arm_smmu_master *master = dev_iommu_priv_get(dev); + struct arm_smmu_domain *smmu_domain; + struct arm_smmu_attached_domain *attached_domain; + unsigned long flags; - domain = iommu_get_domain_for_dev_pasid(dev, pasid, IOMMU_DOMAIN_SVA); + if (!master || pasid == 0) + return; + + domain = iommu_get_domain_for_dev_pasid(dev, pasid, 0); if (WARN_ON(IS_ERR(domain)) || !domain) return; + if (domain->type == IOMMU_DOMAIN_SVA) + return arm_smmu_sva_remove_dev_pasid(domain, dev, pasid); - arm_smmu_sva_remove_dev_pasid(domain, dev, pasid); + smmu_domain = to_smmu_domain(domain); + mutex_lock(&arm_smmu_asid_lock); + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_for_each_entry(attached_domain, &smmu_domain->attached_domains, domain_head) { + if (attached_domain->master != master || + attached_domain->ssid != pasid) + continue; + list_del(&attached_domain->domain_head); + break; + } + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); + arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, pasid, + NULL); + arm_smmu_atc_inv_master_ssid(master, pasid); + master->nr_attached_pasid_domains -= 1; + mutex_unlock(&arm_smmu_asid_lock); } static struct iommu_ops arm_smmu_ops = { @@ -2900,6 +3027,7 @@ static struct iommu_ops arm_smmu_ops = { .owner = THIS_MODULE, .default_domain_ops = &(const struct iommu_domain_ops) { .attach_dev = arm_smmu_attach_dev, + .set_dev_pasid = arm_smmu_set_dev_pasid, .map_pages = arm_smmu_map_pages, .unmap_pages = arm_smmu_unmap_pages, .flush_iotlb_all = arm_smmu_flush_iotlb_all, diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 6929590530367..48795a7287b69 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -707,6 +707,7 @@ struct arm_smmu_master { bool iopf_enabled; struct list_head bonds; unsigned int ssid_bits; + unsigned int nr_attached_pasid_domains; }; /* SMMU private data for an IOMMU domain */ From patchwork Tue Jun 6 12:07:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 103761 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp3349560vqr; Tue, 6 Jun 2023 05:18:35 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5eZRM6xq7oQr1i7A59wV/j0xxnaw1AXczQbKcsGwDzHlZQdBhxa4KQQE20viYDh02QA4JH X-Received: by 2002:a05:622a:11d0:b0:3f7:f60c:7806 with SMTP id n16-20020a05622a11d000b003f7f60c7806mr2089351qtk.12.1686053915708; Tue, 06 Jun 2023 05:18:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686053915; cv=none; d=google.com; s=arc-20160816; b=fXT4sEO+2ZE3G1Whm4Dwm9KgcKx+CMv+ViLx80sB4O/1lzecJFPktwHIWNhTRYohe5 wuibTUwum2sXXLHyXNYGENp58I2jEc4GxUTSeS+a5IiaAgM2VGdUSugMnCxSDcOaD3dt mWOuWr0ZLgvOqDlgP+XbulekrTewUEhDXs6uB+riDPqx5+UqdWnT69/kuAneTVoMQDOq T9cGHziuwrI5FH5arzLcFe7Yim9u2iJJiW4qlJfQNMGvHlRxAFn1Im5+wjdaKLgXBZR8 2TTMp64B0t8pBQTcwa2uZWn+4iLKdopluuImvxN6w7QLnlQOQl5aRHCiOHjHkDRBD/Lh ChNQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=83dlZFDhtSz2rg2AR6ggZ2Mh/ASzqdpRvSd02rbzaCk=; b=Z+foVoKS7yh7rhbte5icVZqZcGhFl1WXRY1gG4Qyfk0NqT7SukcnEUJ21rss8mD9oP UWtpYwwmQQdFEI9DLoOooP8kVhEPQmqJ8xgf33GMhc3IFdV2LKjIgdgfnsMuPvz45Wi4 TH2cUmC/Lc8hhmVCxWCTYaZj2+6WwRzy0qaGHgxOk39A39/pw3/VpPh1q86zxwtNnR3x PcYsFYaHRzy8ndP1/jlJLY4oEMDXrwD5ZXD/ZJfpkgzJyMc23fpVh2epvm/VjHVIlHRT V2PJzTu3n7jvh5hitir5+oTUGmyXBFZ4pA+6s6Nye7DG5Fk5DlMypze7Dp/81/pAqVta Sntg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=IoFA55ul; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id iy10-20020a0562140f6a00b0056f0669018bsi6172116qvb.35.2023.06.06.05.18.20; Tue, 06 Jun 2023 05:18:35 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=IoFA55ul; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237422AbjFFMLC (ORCPT + 99 others); Tue, 6 Jun 2023 08:11:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47682 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237401AbjFFMK5 (ORCPT ); Tue, 6 Jun 2023 08:10:57 -0400 Received: from mail-yw1-x114a.google.com (mail-yw1-x114a.google.com [IPv6:2607:f8b0:4864:20::114a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D007E10C0 for ; Tue, 6 Jun 2023 05:10:31 -0700 (PDT) Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-565a336caa0so97995387b3.1 for ; Tue, 06 Jun 2023 05:10:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686053430; x=1688645430; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=83dlZFDhtSz2rg2AR6ggZ2Mh/ASzqdpRvSd02rbzaCk=; b=IoFA55ulrjtnJDTW5pbuUTS9XeM0qmUYpaAJ3OnF3fjh404oOtNySp8WzgeTzaaiiO mVRlBZ2obwVwh4OUFmiQJt7UUnDxoDMkllhaIBQAmBhFfw48M+MKZQjL4e//71A1xQ+M ZQe0Lx0j/uNjanZ997DNS7XHK7FjkoWOzElSxea08x+rZug6VZyihB4gEnokoKygwiFy oohF6tU936F+ATOKuIyJj4FuSck9QyYVTKPs6mJWVCMGD4SJTG4PI7zHHboXxVFaZsG2 nDS7Gx45dkGyCy6J2pqiwfXPIBXnbTfVR9nkfcP+mnZCb8QJyIFTWcKIELfCjtVWDZgK oB0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686053430; x=1688645430; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=83dlZFDhtSz2rg2AR6ggZ2Mh/ASzqdpRvSd02rbzaCk=; b=Ic9OWiyZlycETbO/O3xrXiErgSR6jf5HCBtoiKJb46n5VG58EDG88EnIpykPstMFV6 6+eWb0JS7poGaV/fcBd7gz82dfeIf2BURJBXjb5ibvn8gp+llTyciAzZI7wT78kVV/TO lMlM5dtYBAsqfv+jJ2Pq4t8yn6lQErE0qresx12JSjSUYp+3QUuYO8wxaIMArNEwuo1t PjJywnebVsNaaRUnK7tHYlOrnmgYQ/3aV8kTpRSH3Bi6e6UQX2YTR+x5dfK/J9McwH1n 9bq0djzs3CQH7yPjRrlgMsBU7dDLOxJ0vSTmcGTu4uoqR1EWm/SSHdlCVyICnJpXWc7A bjHA== X-Gm-Message-State: AC+VfDxq28RCZ7rgEYXjtT/eYShbsu5G1gRlYgL+ces6bzdGF9k61zN2 LYiqTbl4xD+8lXWP/1TeyBS5eVDNjWYJ X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:a615:63d5:b54e:6919]) (user=mshavit job=sendgmr) by 2002:a81:b289:0:b0:562:837:122f with SMTP id q131-20020a81b289000000b005620837122fmr942001ywh.9.1686053430214; Tue, 06 Jun 2023 05:10:30 -0700 (PDT) Date: Tue, 6 Jun 2023 20:07:46 +0800 In-Reply-To: <20230606120854.4170244-1-mshavit@google.com> Mime-Version: 1.0 References: <20230606120854.4170244-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.rc0.172.g3f132b7071-goog Message-ID: <20230606120854.4170244-11-mshavit@google.com> Subject: [PATCH v2 10/18] iommu/arm-smmu-v3-sva: Remove bond refcount From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED, USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767955670891176813?= X-GMAIL-MSGID: =?utf-8?q?1767955670891176813?= The iommu-sva framework checks if a bond between a device and mm already exists and handles refcounting at the iommu_domain level. __arm_smmu_sva_bind is therefore only called once for a device/mm pair. Signed-off-by: Michael Shavit --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 12 +----------- 1 file changed, 1 insertion(+), 11 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index d07c08b53c5cf..20301d0a2c0b0 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -29,7 +29,6 @@ struct arm_smmu_bond { struct mm_struct *mm; struct arm_smmu_mmu_notifier *smmu_mn; struct list_head list; - refcount_t refs; }; #define sva_to_bond(handle) \ @@ -377,21 +376,12 @@ __arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) if (!master || !master->sva_enabled) return ERR_PTR(-ENODEV); - /* If bind() was already called for this {dev, mm} pair, reuse it. */ - list_for_each_entry(bond, &master->bonds, list) { - if (bond->mm == mm) { - refcount_inc(&bond->refs); - return &bond->sva; - } - } - bond = kzalloc(sizeof(*bond), GFP_KERNEL); if (!bond) return ERR_PTR(-ENOMEM); bond->mm = mm; bond->sva.dev = dev; - refcount_set(&bond->refs, 1); bond->smmu_mn = arm_smmu_mmu_notifier_get(smmu_domain, mm); if (IS_ERR(bond->smmu_mn)) { @@ -570,7 +560,7 @@ void arm_smmu_sva_remove_dev_pasid(struct iommu_domain *domain, } } - if (!WARN_ON(!bond) && refcount_dec_and_test(&bond->refs)) { + if (!WARN_ON(!bond)) { list_del(&bond->list); arm_smmu_mmu_notifier_put(bond->smmu_mn); kfree(bond); From patchwork Tue Jun 6 12:07:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 103767 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp3350058vqr; Tue, 6 Jun 2023 05:19:24 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4s4WN3QcPejzyHV0sldu1wwySfaKCXjoPG/Pzr4IimjtG5uMo8FX/uFMV36539Y8LhYlO1 X-Received: by 2002:a05:6214:76f:b0:624:3af6:21d2 with SMTP id f15-20020a056214076f00b006243af621d2mr1722707qvz.13.1686053964508; Tue, 06 Jun 2023 05:19:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686053964; cv=none; d=google.com; s=arc-20160816; b=dWAYjleCqB3Hjdo0Uf1bWoSN7A/eAerdLGIAdG8ZgqNdOqFxWLUaHtQLMF+XBgYxqm X2m5xmte3bOHpfIRK0xUgsaP8VT8624dsdXjOI6QoY/sZ+UfDW2/pVcz78RcoyJ6enXX T7G5CZMuMtokbD7skQfGYOzcFqOhalEKQZaJNv2odL0S6TF/wn2o8CnGEcF53Jb4FGFc /BZveGaxGWQIKrWvJmsD4z4BKEzAOPlWJn3NwviMQkdkEqIfYJfrkjyZkjt2fY7lO+Dh qsuOJbCDS6APjIhafJ9KWQt59qIPTZb2LlxyAa/FO34rOb/NP/dDYgPfeYkchYi1O8kc Ye2A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=0QGlHe+zKTAk3+6sKXLWwU9ZaaAOqSw/Vt2fUhgy2WA=; b=QxDXZgjcLEY/eC7kWSierJrWglF9ZHyWCuXrBOEXLqdZ5chG7p+xezKmU073aYVDL8 0GZ/pkZCuYJ84Z4goGUzrBVPgK5Kulxm2atlqSOXwOSFDruXM/5YwCHWjd0+MhYTDfaj 8PhbvMwzNIMBk5lKVv+fOaOmyWzuQb1ruj4aMUFGXfAmIRB8qmRiU71zHDzGYw/dK43U WKPhe8kLlUMdlfEH9qAdnj2IUDHt9l+bivQQGYeCL/3n46TGSzHmqnfZz/PfsBA4t1l2 4ZvYgQP3BB0TXrW/igNyn2QOk8NNLebyX0RmQKxGmy1+aX4iW6bMjvFhVOAmvNbYWHkx G9JQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=bS4nn54X; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id g4-20020a0562140ac400b005a1900ab5e4si6367294qvi.223.2023.06.06.05.19.10; Tue, 06 Jun 2023 05:19:24 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=bS4nn54X; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237398AbjFFMLF (ORCPT + 99 others); Tue, 6 Jun 2023 08:11:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47784 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237435AbjFFMLD (ORCPT ); Tue, 6 Jun 2023 08:11:03 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E030510D7 for ; Tue, 6 Jun 2023 05:10:38 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-bb0d11a56abso8022769276.2 for ; Tue, 06 Jun 2023 05:10:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686053437; x=1688645437; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=0QGlHe+zKTAk3+6sKXLWwU9ZaaAOqSw/Vt2fUhgy2WA=; b=bS4nn54XlFs3QFnaLf26ObLJelQ97e9PnCVNdve77ggjCR3JfNlgF6+o8d1vUqjrUM wvbXSNOisrmhnBwK5+9Sazm5rU2Zgirf/L3AYbVVxxno+05JhYVLwDDz3CWGUB4CgacW 7FI2dvkLiNurNExQW8FFZfErrWB4Og5cGkpTlMRWPQdtpgidl8RlYM87aELCqrHicqXT zpsqKEpKie2x+xoq6gnaxrUin3F+7UedVAY8bBpVCRRrbKzzvtt973F6bc+QBgFcfs/w A85oh7tf3LR1d+Jaic4iI6vwBIZt8daoHhctT54OQIG6Li3F73eKCdyQ6LRcKn5ogmAj Q1uA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686053437; x=1688645437; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=0QGlHe+zKTAk3+6sKXLWwU9ZaaAOqSw/Vt2fUhgy2WA=; b=MplL335TpQ0qF/qDC/l0N8slRnPEr0obs4N+1t90ZxyA9ZIU9E30t0DOahkPPCikJc /1zVX82QxtAElXXOoqZm+2vAuDMGCXGxxBRPfn68XIbCSl8BvPsrUqW11tcQ7zRnls/g 16VALtjttr+kU1CfVUEpmWeU+MLHtwEn4o4FbbZGFuglRQIbAPDwr8LZNkD3u3RJwl+1 N6ReDlJUNPrRV2QtjrjUy0P8JcpjlqOcWziPfFsDG/9mWyD9ON5ucOq4LVmBD7LV2H+e gg2OvO6/u7FZQ2RSKr+lJtTzNPf/87R0j92G/eGQazsNebNSNcGd7hCc8DEkufsmWoOy JrKQ== X-Gm-Message-State: AC+VfDwCV2waSH1Xk8e4JunktFZvsMDYmND5if60Lg2dRjtJrxPcGkqa bMe8OV5BW/e6uhmCg120T24ycwDaaxNK X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:a615:63d5:b54e:6919]) (user=mshavit job=sendgmr) by 2002:a05:6902:703:b0:bac:f608:7113 with SMTP id k3-20020a056902070300b00bacf6087113mr1100722ybt.4.1686053437004; Tue, 06 Jun 2023 05:10:37 -0700 (PDT) Date: Tue, 6 Jun 2023 20:07:47 +0800 In-Reply-To: <20230606120854.4170244-1-mshavit@google.com> Mime-Version: 1.0 References: <20230606120854.4170244-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.rc0.172.g3f132b7071-goog Message-ID: <20230606120854.4170244-12-mshavit@google.com> Subject: [PATCH v2 11/18] iommu/arm-smmu-v3-sva: Clean unused iommu_sva From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED, USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767955722047330291?= X-GMAIL-MSGID: =?utf-8?q?1767955722047330291?= The __arm_smmu_sva_bind function returned an unused iommu_sva handle that can be removed. Signed-off-by: Michael Shavit --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 18 ++++++------------ 1 file changed, 6 insertions(+), 12 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 20301d0a2c0b0..650c9c9ad52f1 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -25,7 +25,6 @@ struct arm_smmu_mmu_notifier { #define mn_to_smmu(mn) container_of(mn, struct arm_smmu_mmu_notifier, mn) struct arm_smmu_bond { - struct iommu_sva sva; struct mm_struct *mm; struct arm_smmu_mmu_notifier *smmu_mn; struct list_head list; @@ -364,8 +363,7 @@ static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) arm_smmu_free_shared_cd(cd); } -static struct iommu_sva * -__arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) +static int __arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) { int ret; struct arm_smmu_bond *bond; @@ -374,14 +372,13 @@ __arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); if (!master || !master->sva_enabled) - return ERR_PTR(-ENODEV); + return -ENODEV; bond = kzalloc(sizeof(*bond), GFP_KERNEL); if (!bond) - return ERR_PTR(-ENOMEM); + return -ENOMEM; bond->mm = mm; - bond->sva.dev = dev; bond->smmu_mn = arm_smmu_mmu_notifier_get(smmu_domain, mm); if (IS_ERR(bond->smmu_mn)) { @@ -390,11 +387,11 @@ __arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) } list_add(&bond->list, &master->bonds); - return &bond->sva; + return 0; err_free_bond: kfree(bond); - return ERR_PTR(ret); + return ret; } bool arm_smmu_sva_supported(struct arm_smmu_device *smmu) @@ -572,13 +569,10 @@ static int arm_smmu_sva_set_dev_pasid(struct iommu_domain *domain, struct device *dev, ioasid_t id) { int ret = 0; - struct iommu_sva *handle; struct mm_struct *mm = domain->mm; mutex_lock(&sva_lock); - handle = __arm_smmu_sva_bind(dev, mm); - if (IS_ERR(handle)) - ret = PTR_ERR(handle); + ret = __arm_smmu_sva_bind(dev, mm); mutex_unlock(&sva_lock); return ret; From patchwork Tue Jun 6 12:07:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 103759 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp3346538vqr; Tue, 6 Jun 2023 05:13:49 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6W90zGtgC1q7MYvedGyat9ZB+YWgOnc3sYkmUsM0c1ARC7oSa0u7NsGrMW5MQ1BMPAFDlJ X-Received: by 2002:a05:6a20:54a3:b0:100:efa1:eab0 with SMTP id i35-20020a056a2054a300b00100efa1eab0mr1717190pzk.2.1686053629238; Tue, 06 Jun 2023 05:13:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686053629; cv=none; d=google.com; s=arc-20160816; b=FZz5IFWST6swqneC9oCxIMoPO/D3pz7c9BLjpJEdX3NOQTG7QXjv6NClaJrjot/bUR f0ZyRRWIkRIl/TspwRIxqUQCVV0XIEYDdAiAv4fsvwq5GE2n1bz571zWlDmVHyay5JCR EeACv+C5xqFLQ4yE/2Br2XN18n8eGaVoFAeywOS4NT8mlDpXg3soZBO2+ueNGofDFOS8 S2oURNePP7qxZJA0xl+9dVGuvSE6dYv9Lm7E/iSWnphSyGM0QitEYlurDQdYGyMuWnrr 2aCRfDveB+IJ6KLE2sTmuVBsAbCV7UkC3aP5IQRAo7iOJb9gnGB5dond9RAs+K901yH4 QmAA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=FnWnvbHmIF1ltrhpko3AmatpWzsBYOi2I16/Atgd3Oo=; b=K+jvdTZt56bHQ/ds932FcHxzm2OvQImZN5zTUphzSsgOxIFArkIH+c8tJfmLhcQ8h8 dVPhvBGWkX3fpf8fwqapQnJ9kzWTIKfQeeh3SIBHKxhNGTMbL52xfFNAfI+3a3Y5MJmD YMMwRu82o2j1xNgGq+ioXrDKrsniOv77h7lXIG8gZ7P8rgLo3lPNFeaRhSAiZeiPM/il VChmcsq9namFn92KWkIJQMoRiXzyouEMLoSoogU8FxBITZQSAUUdbuVovQojO+caTfKM 0XNaMXmkHgBxJlUk/BhAsB1DamCCnYqV00ojspO8I/zDgQ3QP+xdRmVABN7ONSg5q7Lc Bg3g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=OT+iGAb6; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id x6-20020a170902a38600b001ab20d029e0si6891804pla.582.2023.06.06.05.13.35; Tue, 06 Jun 2023 05:13:49 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=OT+iGAb6; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237463AbjFFMLQ (ORCPT + 99 others); Tue, 6 Jun 2023 08:11:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47956 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237413AbjFFMLP (ORCPT ); Tue, 6 Jun 2023 08:11:15 -0400 Received: from mail-pl1-x649.google.com (mail-pl1-x649.google.com [IPv6:2607:f8b0:4864:20::649]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 40443172C for ; Tue, 6 Jun 2023 05:10:48 -0700 (PDT) Received: by mail-pl1-x649.google.com with SMTP id d9443c01a7336-1b04c328aa1so23992515ad.2 for ; Tue, 06 Jun 2023 05:10:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686053444; x=1688645444; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=FnWnvbHmIF1ltrhpko3AmatpWzsBYOi2I16/Atgd3Oo=; b=OT+iGAb66chgU7NBKy4tFHkZVVSeUEqENw2JBDh6NO0OJc11YO1CWkwgNkZjHtYtDP lZ7NxCAGg5dgVzWFUtOEhk2CWrx1Pfiqw0NYIcjLZSSHU16TLJ0GKWraeJX9Q0twcSP6 3YgWeiLUY98PryAP+mhZtWKw3M6BYA3n/XIGSGx3+UrSbGweWJffi8+xWt2FfV3NThzo XBXaKTbXD4ydtH9xv2nOk9efQS5Rjmd49C5Tsn1GOsK8wtmeQmZU8CsM5vSV2dmEDWv+ GKDEFYPFJzpH0Rq9Q4iEdy1A3yyzmEYO02gaGn+H03RDX5nInQ1OlVuD38G4glG62cFv r7RA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686053444; x=1688645444; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=FnWnvbHmIF1ltrhpko3AmatpWzsBYOi2I16/Atgd3Oo=; b=LXvalhbYiVGVSARRffSh/yASgUXkHLgpjSZKANX7DsZR+PZd/yw9SQpqymzztKU+l/ s+Kn1BZ08lpsCWh8PE/eu516AyhBqKiso7bTGnep/6vtaojwLmLk9zk7KWM547tz+Gzc 79p4jH+NmqUiu3JBXRTdHEYl33nF+ejQ58xku/TrmEkYMmS8SvsrOvDeUHHrsWNo/sPo uHYOxzMpTHWHltSE+qNNUWrZRPhXGkoAwO9mDUhR71Z0BK/530Ts4W+OuN/8Xu2uThS+ TG83YGuSI8qwV3hEEat2+8CubepfzuynnZjMMFdZTzofno4EQlq62uW9BL0etLKCOvNo p/UA== X-Gm-Message-State: AC+VfDyNSnI9Q8JgRoSglZ4akX98Ui78AuwwZ/Pv6thtcVaK3//aol7c HGXr9O7LgK72penmBecA+jMXstbRHGVj X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:a615:63d5:b54e:6919]) (user=mshavit job=sendgmr) by 2002:a17:902:8693:b0:1af:ff00:dfc2 with SMTP id g19-20020a170902869300b001afff00dfc2mr602005plo.5.1686053443904; Tue, 06 Jun 2023 05:10:43 -0700 (PDT) Date: Tue, 6 Jun 2023 20:07:48 +0800 In-Reply-To: <20230606120854.4170244-1-mshavit@google.com> Mime-Version: 1.0 References: <20230606120854.4170244-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.rc0.172.g3f132b7071-goog Message-ID: <20230606120854.4170244-13-mshavit@google.com> Subject: [PATCH v2 12/18] iommu/arm-smmu-v3-sva: Remove arm_smmu_bond From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767955370493701828?= X-GMAIL-MSGID: =?utf-8?q?1767955370493701828?= There's a 1:1 relationship between arm_smmu_bond and the iommu_domain used in set_dev_pasid/remove_dev_pasid. arm_smmu_bond has become an unnecessary complication. It's more natural to store any needed information at the iommu_domain container level. Signed-off-by: Michael Shavit --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 69 +++++++------------ drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 1 - drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 2 +- 3 files changed, 24 insertions(+), 48 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 650c9c9ad52f1..b615a85e6a54e 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -24,14 +24,13 @@ struct arm_smmu_mmu_notifier { #define mn_to_smmu(mn) container_of(mn, struct arm_smmu_mmu_notifier, mn) -struct arm_smmu_bond { - struct mm_struct *mm; +struct arm_smmu_sva_domain { + struct iommu_domain iommu_domain; struct arm_smmu_mmu_notifier *smmu_mn; - struct list_head list; }; -#define sva_to_bond(handle) \ - container_of(handle, struct arm_smmu_bond, sva) +#define to_sva_domain(domain) \ + container_of(domain, struct arm_smmu_sva_domain, iommu_domain) static DEFINE_MUTEX(sva_lock); @@ -363,10 +362,10 @@ static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) arm_smmu_free_shared_cd(cd); } -static int __arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) +static int __arm_smmu_sva_bind(struct device *dev, + struct arm_smmu_sva_domain *sva_domain, + struct mm_struct *mm) { - int ret; - struct arm_smmu_bond *bond; struct arm_smmu_master *master = dev_iommu_priv_get(dev); struct iommu_domain *domain = iommu_get_domain_for_dev(dev); struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); @@ -374,24 +373,14 @@ static int __arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) if (!master || !master->sva_enabled) return -ENODEV; - bond = kzalloc(sizeof(*bond), GFP_KERNEL); - if (!bond) - return -ENOMEM; - - bond->mm = mm; - - bond->smmu_mn = arm_smmu_mmu_notifier_get(smmu_domain, mm); - if (IS_ERR(bond->smmu_mn)) { - ret = PTR_ERR(bond->smmu_mn); - goto err_free_bond; + sva_domain->smmu_mn = arm_smmu_mmu_notifier_get(smmu_domain, + mm); + if (IS_ERR(sva_domain->smmu_mn)) { + sva_domain->smmu_mn = NULL; + return PTR_ERR(sva_domain->smmu_mn); } - - list_add(&bond->list, &master->bonds); + master->nr_attached_sva_domains += 1; return 0; - -err_free_bond: - kfree(bond); - return ret; } bool arm_smmu_sva_supported(struct arm_smmu_device *smmu) @@ -521,7 +510,7 @@ int arm_smmu_master_enable_sva(struct arm_smmu_master *master) int arm_smmu_master_disable_sva(struct arm_smmu_master *master) { mutex_lock(&sva_lock); - if (!list_empty(&master->bonds)) { + if (master->nr_attached_sva_domains != 0) { dev_err(master->dev, "cannot disable SVA, device is bound\n"); mutex_unlock(&sva_lock); return -EBUSY; @@ -545,23 +534,12 @@ void arm_smmu_sva_notifier_synchronize(void) void arm_smmu_sva_remove_dev_pasid(struct iommu_domain *domain, struct device *dev, ioasid_t id) { - struct mm_struct *mm = domain->mm; - struct arm_smmu_bond *bond = NULL, *t; + struct arm_smmu_sva_domain *sva_domain = to_sva_domain(domain); struct arm_smmu_master *master = dev_iommu_priv_get(dev); mutex_lock(&sva_lock); - list_for_each_entry(t, &master->bonds, list) { - if (t->mm == mm) { - bond = t; - break; - } - } - - if (!WARN_ON(!bond)) { - list_del(&bond->list); - arm_smmu_mmu_notifier_put(bond->smmu_mn); - kfree(bond); - } + master->nr_attached_sva_domains -= 1; + arm_smmu_mmu_notifier_put(sva_domain->smmu_mn); mutex_unlock(&sva_lock); } @@ -572,7 +550,7 @@ static int arm_smmu_sva_set_dev_pasid(struct iommu_domain *domain, struct mm_struct *mm = domain->mm; mutex_lock(&sva_lock); - ret = __arm_smmu_sva_bind(dev, mm); + ret = __arm_smmu_sva_bind(dev, to_sva_domain(domain), mm); mutex_unlock(&sva_lock); return ret; @@ -590,12 +568,11 @@ static const struct iommu_domain_ops arm_smmu_sva_domain_ops = { struct iommu_domain *arm_smmu_sva_domain_alloc(void) { - struct iommu_domain *domain; + struct arm_smmu_sva_domain *sva_domain; - domain = kzalloc(sizeof(*domain), GFP_KERNEL); - if (!domain) + sva_domain = kzalloc(sizeof(*sva_domain), GFP_KERNEL); + if (!sva_domain) return NULL; - domain->ops = &arm_smmu_sva_domain_ops; - - return domain; + sva_domain->iommu_domain.ops = &arm_smmu_sva_domain_ops; + return &sva_domain->iommu_domain; } diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index a6fa56585c219..b7f834dde85d1 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2784,7 +2784,6 @@ static struct iommu_device *arm_smmu_probe_device(struct device *dev) master->dev = dev; master->smmu = smmu; - INIT_LIST_HEAD(&master->bonds); dev_iommu_priv_set(dev, master); ret = arm_smmu_insert_master(smmu, master); diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 48795a7287b69..3525d60668c23 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -705,7 +705,7 @@ struct arm_smmu_master { bool stall_enabled; bool sva_enabled; bool iopf_enabled; - struct list_head bonds; + unsigned int nr_attached_sva_domains; unsigned int ssid_bits; unsigned int nr_attached_pasid_domains; }; From patchwork Tue Jun 6 12:07:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 103760 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp3347917vqr; Tue, 6 Jun 2023 05:15:54 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ768zBbaBtLXrbm/OVyN1WkG5FVnchF0COf+2+qxkpCOROiwUNJV9ltK7/2F73wzloAMJfK X-Received: by 2002:a17:90a:bd8d:b0:256:dd38:6563 with SMTP id z13-20020a17090abd8d00b00256dd386563mr496754pjr.45.1686053753970; Tue, 06 Jun 2023 05:15:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686053753; cv=none; d=google.com; s=arc-20160816; b=CYb78uR5u3hgvz1xGVu21bv+gTjFJf/tbZOWbAxBNXXbYGoA37RH07PYcHqdLMUE83 Q69k5meskc/CI3C8VB8PhCdZF2uOKpMzrkIgTCCXg2lnTL7MngZBmcL0A8lVOxFvtKEm TDYCscwUyMORPYvBExXTUfL6+yDzcA+McXVZjBP549tW0ugWcZSGh74Eh84JgtA/7CuZ LLGd0NId2+M+dSyZWsAJGiAec3X/hEWV4SL8y5oI3mvgN/MEx/IXh37wHTnS4Jhxb1FO WqnZzzOtNYmYxdZszh1DsC13f2Kmidf8G/4bc4I/3TwmdUQ0m0V5kydRy5GZll/xtokK JGnw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=C0PSyQeXC7Iz/P3YpxIMlG3+vOjTAssvVZqtnA/BFhw=; b=FuRVBHmaZjDNK79et3knP+7dXxjt1hKXQSDUyA4C/XtHpyheLygGZHfBrzlHEoZLOE 7CazeLOAnzLrA4x5qhByxIgfSo92Th451hlvHCMm8+wo6KOAeg0g3jzppZM4Zv9tODk5 VDbMqlVc4lF5sNaFxzLlGMHzVR2bKnNlUxKo1Mefby4PDMB2ys5V8vYaj9R4Ijgs6m9i DNEPtU2eoj3ERo/zCkrtozqAIny7rs7MHpVA8Ao0R/+e75GBOapYiUmiJ4l3/3MM92KD u2zJXXzn5db2tEWFabcDSroDznM590yTm3s9LnaiQqXeNuOx9s9ghCGvByu09q2KKsAh O55Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=2YvtQkht; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id gb13-20020a17090b060d00b0025677f1197asi8944238pjb.12.2023.06.06.05.15.41; Tue, 06 Jun 2023 05:15:53 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=2YvtQkht; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237487AbjFFMLW (ORCPT + 99 others); Tue, 6 Jun 2023 08:11:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48030 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237479AbjFFMLT (ORCPT ); Tue, 6 Jun 2023 08:11:19 -0400 Received: from mail-pl1-x649.google.com (mail-pl1-x649.google.com [IPv6:2607:f8b0:4864:20::649]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A94D010E2 for ; Tue, 6 Jun 2023 05:10:51 -0700 (PDT) Received: by mail-pl1-x649.google.com with SMTP id d9443c01a7336-1b0371a51cfso21043365ad.0 for ; Tue, 06 Jun 2023 05:10:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686053451; x=1688645451; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=C0PSyQeXC7Iz/P3YpxIMlG3+vOjTAssvVZqtnA/BFhw=; b=2YvtQkhtruPyaU6tyIiOFVtpwOKVajkps/SV+rmgxl5WcQGFxK06oATnOVAhefKsb/ /+tX6arfJ/Za9QYTa80Ei6E11onSKYS7FIz7iInWoRFPOWEY5aPsmfcRL0zuObD3kAq0 teU/b922dsyZ//kQGYgXVvkHMSBnz3ieADNsR8xmAE0B3/N50Bl+vA1z7orEKcxD09Na bdbw4y7eF6C+HCfUimAIszfp3GX30HXWkqD/lwSejlJbb9AM/4g8brT1EpgR9qNhnEbM IypWkQ/P7yNKwLZhGySBmvzzhXMm7ZZFnsb2tJniozWmxdNQhN6ba8XMv52XGQjbrdS9 De3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686053451; x=1688645451; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=C0PSyQeXC7Iz/P3YpxIMlG3+vOjTAssvVZqtnA/BFhw=; b=eYIaFcQQL0RmDGhZxBlrnqAsiL+eUVfRTFMt4MhC8qSxaRpJYsAgApM9q5s9TQhMRp WrzcVmsQfNj04Vwr6ZYmtKZKwWYHukf/NJbSOMOQkja0GcPvjMDyrKUnTXMyz75f5d/u vTIBP6nkbITta04hjtxt7eTlZxfLTHvnkefK1XqUwmK+06l9nhWs6ZuB7TFEupojsi9G uJE73hx0GHgTQI0LebVUf1S2sqU8otBDLzvOF0NrT/T1aS0+G4dOeNaXDT5C4MgMcwus K16ocnOUy8DZG4iT1FOcATgmLVo8DL+iORl5BgTZ/0b8WE9o9WkxfvbnoBbwoRqQ7XWy frCw== X-Gm-Message-State: AC+VfDw+1yyjmV0UIMu+Iw/sP3mZGrR6t2CE3bKOIkO31KVsgR0PstuV gBMdhfIMnc/Cg5i7rKmdRRs3dKar0IzB X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:a615:63d5:b54e:6919]) (user=mshavit job=sendgmr) by 2002:a17:902:d483:b0:1ae:485e:d192 with SMTP id c3-20020a170902d48300b001ae485ed192mr629661plg.8.1686053451002; Tue, 06 Jun 2023 05:10:51 -0700 (PDT) Date: Tue, 6 Jun 2023 20:07:49 +0800 In-Reply-To: <20230606120854.4170244-1-mshavit@google.com> Mime-Version: 1.0 References: <20230606120854.4170244-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.rc0.172.g3f132b7071-goog Message-ID: <20230606120854.4170244-14-mshavit@google.com> Subject: [PATCH v2 13/18] iommu/arm-smmu-v3-sva: Add check when enabling sva From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767955501287040671?= X-GMAIL-MSGID: =?utf-8?q?1767955501287040671?= SVA domains can only be attached when the master's STEs have a stage 1 domain. Signed-off-by: Michael Shavit --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index b615a85e6a54e..e2a91f20f0906 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -499,9 +499,15 @@ int arm_smmu_master_enable_sva(struct arm_smmu_master *master) int ret; mutex_lock(&sva_lock); + + if (!master->s1_cfg) { + ret = -EBUSY; + goto unlock; + } ret = arm_smmu_master_sva_enable_iopf(master); if (!ret) master->sva_enabled = true; +unlock: mutex_unlock(&sva_lock); return ret; From patchwork Tue Jun 6 12:07:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 103772 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp3353480vqr; Tue, 6 Jun 2023 05:24:41 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7yKaDiXikVNW3/DC+DOVqpIsPDB/i4hgGeRGv0zjQHg5Y4jmiGTQ1xMxyFT6/dRAQJm7ev X-Received: by 2002:a05:6a20:430d:b0:10f:472f:ffbb with SMTP id h13-20020a056a20430d00b0010f472fffbbmr10944321pzk.7.1686054281449; Tue, 06 Jun 2023 05:24:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686054281; cv=none; d=google.com; s=arc-20160816; b=1CGSUw6MLklPMeEC/oapdjL+2OsSJhCu9NTv5y6LCcU/avhPRH3Ymjcs28ameIY1Iw epsoCT3msK1in3vLERcht2TcQOPGPzd8N+fmBHwFJX6iij+7AftlGhPpYfPs0d71AdB3 iBbFlE1VWpu8FGEUHRSZOwCr2wxBMuanHAHEeci8MnvwrltuKIzUbKc/jlZmRvdRQroM 6aHQjUJ3wregq0isM+1YHIM1D2PboVgM90jiGPeE0WAEOgXSuRaFvTr7ECRQDPT75j6B PE/wDdC3XKa8DDBJ5scdhYPOHLMAX61R3kEdJqytbaFpxcNObc76I3xFbAzYRZELfwcW BwEQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=e/IPHrVLjUq2s1o0Kk8jlOX8iem0l8MY258dkWJhSSA=; b=j8Z5ksvE1DIqC4wLLdqZy/9bbU4J/q6pCjesMKlT3SgbkAea+wwDaesFswby7lzkRI MrfVUAvYNN79kml2krvp9zEYh6BRRQngKUf+JERE4UV4eco/NtTd5f5iZr3JcdyJlQgw VzfKyH+OgWOXihqio0XsShYKmYt4l+QhITbuQpMCf9XO46Fzxn8BwYzZxcA7eBbKGEuA 0wmKdytfVvqbBYuHbOPVXD3ElPFWGPmLTWbSzJqqD1Mj6EDOj6wU+fg6eE9AlNweovmC V4ZBsgAYC0bMPLmGBVTh6pzD9Z61b2mfqgiWBb0wibgbwBXC+hnuHVs+L8JitlBau9c3 TEvw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=VZ4aVUeg; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 123-20020a630181000000b005303c3dedc2si7218975pgb.419.2023.06.06.05.24.28; Tue, 06 Jun 2023 05:24:41 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=VZ4aVUeg; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237498AbjFFML3 (ORCPT + 99 others); Tue, 6 Jun 2023 08:11:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48074 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237486AbjFFMLZ (ORCPT ); Tue, 6 Jun 2023 08:11:25 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 441941703 for ; Tue, 6 Jun 2023 05:11:00 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-bb2202e0108so7551099276.1 for ; Tue, 06 Jun 2023 05:11:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686053458; x=1688645458; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=e/IPHrVLjUq2s1o0Kk8jlOX8iem0l8MY258dkWJhSSA=; b=VZ4aVUegMrv+4Q6yvYwHunY/3cwuUlcr0h58VcDyy+whkFdBNoq1bmH2cIlw3NPTGf 2xwwiPuSrWUGDBls3d7jNYpe94klnR35TS5OkpOjjZPOxAO8MIWR85bzh29QZGu9r6vq Uhrp4oqPT2o6ewJtPDlh64Ybaq2kCogm4tvMgZDNn5hii7e3Sp7nkxS2CPQ+QqPKUqO2 jr7HF72nwtGIZwbU2rVIDb/0fXNIWezbdzedeFrmGi0/fABER6IwQMRfR22nwYnhBjrI RYCBaFffmg4sgiF4vWOkI7rmUWylAkhG7nKO4jBF3BV258AG4UrAU6eDZogjLom8Cb8J fe7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686053458; x=1688645458; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=e/IPHrVLjUq2s1o0Kk8jlOX8iem0l8MY258dkWJhSSA=; b=CQtCnGzW/XmEn4N2MOYYsz+DsCXR++e5FzynOaNKRRqzm5rLTszRGqu8IgWTNCQU/E 4ipoo4w0OOCaigjEV0pqFReHKhibN7fbAuon/u01Aj4XwxdGgH0PZHPAHPP66QIaOcfo JHtDyqZQj2u89wiSSBFNmAZL7V7W0M98/zo+8nIEVtjvphQ3VMq8NSsEg8icPz6V/cg9 1fox2eZ65ca5T4IEN07AQJ7i9wfP7GbugBe3Fg/2WUUJV/LV0nKpO2DIprXe4JkkDdjh ALDfHStnFDiRow5/vYN/AoYluagKcjaIvS0Sq6EaI1ZGLPYEH9+nCyhrulGjDJyMueyS 6isg== X-Gm-Message-State: AC+VfDxakWzChd8CRhxv8Ovc6jxe9WWOj3Ye68yfFsGEUZxjZ64De3h4 YYdZ3LcPZm40PLadqSu39UiPXaZiidwu X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:a615:63d5:b54e:6919]) (user=mshavit job=sendgmr) by 2002:a05:6902:1183:b0:ba7:7478:195f with SMTP id m3-20020a056902118300b00ba77478195fmr620081ybu.13.1686053458202; Tue, 06 Jun 2023 05:10:58 -0700 (PDT) Date: Tue, 6 Jun 2023 20:07:50 +0800 In-Reply-To: <20230606120854.4170244-1-mshavit@google.com> Mime-Version: 1.0 References: <20230606120854.4170244-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.rc0.172.g3f132b7071-goog Message-ID: <20230606120854.4170244-15-mshavit@google.com> Subject: [PATCH v2 14/18] iommu/arm-smmu-v3: Support domains with shared CDs From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767956054501007597?= X-GMAIL-MSGID: =?utf-8?q?1767956054501007597?= SVA may attach a CD to masters that have different upstream SMMU devices. The arm_smmu_domain structure can only be attached to a single upstream SMMU device however. To work around this limitation, we propose an ARM_SMMU_DOMAIN_S1_SHARED domain type for domains that attach a CD shared across with arm_smmu_domains (each attached to a different upstream SMMU device). Signed-off-by: Michael Shavit --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 26 ++++++++++++++++----- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 2 ++ 2 files changed, 22 insertions(+), 6 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index b7f834dde85d1..69b1d09fd0284 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -965,6 +965,20 @@ void arm_smmu_tlb_inv_asid(struct arm_smmu_device *smmu, u16 asid) arm_smmu_cmdq_issue_cmd_with_sync(smmu, &cmd); } +static struct arm_smmu_ctx_desc *arm_smmu_get_cd(struct arm_smmu_domain *domain) +{ + if (domain->stage == ARM_SMMU_DOMAIN_S1_SHARED_CD) + return domain->shared_cd; + else + return &domain->cd; +} + +static bool arm_smmu_is_s1_domain(struct arm_smmu_domain *domain) +{ + return domain->stage == ARM_SMMU_DOMAIN_S1_SHARED_CD || + domain->stage == ARM_SMMU_DOMAIN_S1; +} + /* master may be null */ static void arm_smmu_sync_cd(struct arm_smmu_master *master, int ssid, bool leaf) @@ -1887,8 +1901,8 @@ static void arm_smmu_tlb_inv_context(void *cookie) * insertion to guarantee those are observed before the TLBI. Do be * careful, 007. */ - if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { - arm_smmu_tlb_inv_asid(smmu, smmu_domain->cd.asid); + if (arm_smmu_is_s1_domain(smmu_domain)) { + arm_smmu_tlb_inv_asid(smmu, arm_smmu_get_cd(smmu_domain)->asid); } else { cmd.opcode = CMDQ_OP_TLBI_S12_VMALL; cmd.tlbi.vmid = smmu_domain->s2_cfg.vmid; @@ -1968,10 +1982,10 @@ static void arm_smmu_tlb_inv_range_domain(unsigned long iova, size_t size, }, }; - if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { + if (arm_smmu_is_s1_domain(smmu_domain)) { cmd.opcode = smmu_domain->smmu->features & ARM_SMMU_FEAT_E2H ? CMDQ_OP_TLBI_EL2_VA : CMDQ_OP_TLBI_NH_VA; - cmd.tlbi.asid = smmu_domain->cd.asid; + cmd.tlbi.asid = arm_smmu_get_cd(smmu_domain)->asid; } else { cmd.opcode = CMDQ_OP_TLBI_S2_IPA; cmd.tlbi.vmid = smmu_domain->s2_cfg.vmid; @@ -2549,7 +2563,7 @@ static int arm_smmu_set_dev_pasid(struct iommu_domain *domain, return -ENODEV; } - if (smmu_domain->stage != ARM_SMMU_DOMAIN_S1) { + if (!arm_smmu_is_s1_domain(smmu_domain)) { dev_err(dev, "set_dev_pasid only supports stage 1 domains\n"); return -EINVAL; } @@ -2575,7 +2589,7 @@ static int arm_smmu_set_dev_pasid(struct iommu_domain *domain, */ mutex_lock(&arm_smmu_asid_lock); ret = arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, - pasid, &smmu_domain->cd); + pasid, arm_smmu_get_cd(smmu_domain)); if (ret) { mutex_unlock(&arm_smmu_asid_lock); kfree(attached_domain); diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 3525d60668c23..4ac69427abf1c 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -713,6 +713,7 @@ struct arm_smmu_master { /* SMMU private data for an IOMMU domain */ enum arm_smmu_domain_stage { ARM_SMMU_DOMAIN_S1 = 0, + ARM_SMMU_DOMAIN_S1_SHARED_CD, ARM_SMMU_DOMAIN_S2, ARM_SMMU_DOMAIN_NESTED, ARM_SMMU_DOMAIN_BYPASS, @@ -728,6 +729,7 @@ struct arm_smmu_domain { enum arm_smmu_domain_stage stage; union { struct arm_smmu_ctx_desc cd; + struct arm_smmu_ctx_desc *shared_cd; struct arm_smmu_s2_cfg s2_cfg; }; From patchwork Tue Jun 6 12:07:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 103771 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp3353061vqr; Tue, 6 Jun 2023 05:23:58 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4eZ8V590uXIcuVo+DXhGOlgtMxFmIs/C3ds+d3jkq7mVz09nUE9lGyHh7TxB53L+zBu8V3 X-Received: by 2002:a05:6214:2aa8:b0:625:aa49:19f3 with SMTP id js8-20020a0562142aa800b00625aa4919f3mr1392112qvb.64.1686054238407; Tue, 06 Jun 2023 05:23:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686054238; cv=none; d=google.com; s=arc-20160816; b=YLgn0+2OwFjWWFUELVyrNiOliDPS8pCZb6/7dKqEslSRrXn/SaZx/RuZCRHnIWKTTW O6r7XfUUedfFraaNZFhgnjpBUTeSPGdZm2It7XZKbwPYlkQNLCzc9PzdKiBXdk2HDhWF H8v4eaF5+BjcrxaLEgusdwOCh7hcLoQbz3hW8yNT/ICYqsdJlPFql/uwKs+2WXKi3vFF mtYWfosQ6Y/ackRDdTLENyEkUSsJe5L2r9hiYPmX0xTaYIr1Kr1fAKQqhz27JAatW41I UZoGovO+H9C69RzoXgIphvh2owCtlIT/HanCFAFcIuanToPTooYa9+yf5qS91Nsju9Cw BL0g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=KjUOU0DPHGHGmTP8anm43RkbEqkADC+hmHbnOauQw70=; b=1C/mM+p+P4Oc25kOZ0TcTgaRsFDjcC2MyiqoXRijHfJL/Qf6czA8KffJJ9VUlACArI IyDEBPJIDfkgQ1upaxcGbdkpL3YhsR8270qaHeFxbR86VO+2f/tYwCH9sY/dgmDv1k9s KqRpy4OVeuTUCd8AYiylZYg6bj04tvIKIHG6co8FTwFPzS8KGNj3NmEapaktl4JsLPkq iwN8P0jsaZP5IdGKmlH6Y1sCOueVOoMvQtjPRZhcS8IrLJ2h45WUHPxYVG1fv48okK40 HZxhIttTiYTeXEUu2QM1mCduJg8680NE+HL2g6FzCU6oHwS8F9btOM8H0dtt0C1E+9e8 l64Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=udNKb9hw; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id u15-20020ad45aaf000000b0061b69d0a7d5si6221517qvg.243.2023.06.06.05.23.44; Tue, 06 Jun 2023 05:23:58 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=udNKb9hw; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237533AbjFFMLj (ORCPT + 99 others); Tue, 6 Jun 2023 08:11:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48082 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237449AbjFFMLc (ORCPT ); Tue, 6 Jun 2023 08:11:32 -0400 Received: from mail-yw1-x1149.google.com (mail-yw1-x1149.google.com [IPv6:2607:f8b0:4864:20::1149]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 93CEA170C for ; Tue, 6 Jun 2023 05:11:09 -0700 (PDT) Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-564fb1018bcso98279107b3.0 for ; Tue, 06 Jun 2023 05:11:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686053465; x=1688645465; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=KjUOU0DPHGHGmTP8anm43RkbEqkADC+hmHbnOauQw70=; b=udNKb9hwr0Kk9+BMB5itHj3lPHTSQqr87Qcscot19+Y4rmBCzb19uWrHkx/Yg/muYr MGnuG3BikPGCJJpxU6JP94pix6vlJnCu6PPp2wr+vKwfs8xuLQYo48h5eTdLrLdFCsDq lDUqnvfRISjHbk7HBIJh4zxYeGfUKG+Uh6XEG8xJGMiCYyuh31GHhpeObSF6rfLCWKMQ hSF6T73IH9AXGpQgRV9XBJv7cE02pjOCpymzblrPCUrYRuRkPT2C8bQDiH+m09tXB/7G mK8/B294f8Jybhs9wu6cVKkJO0K5faZneZGSNqaCMhqKBxcMW5E+WPg+FsoEqeUXUccd OJpA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686053465; x=1688645465; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=KjUOU0DPHGHGmTP8anm43RkbEqkADC+hmHbnOauQw70=; b=Zj3T8O1LJImgiLeXhu7zAnJb1CLv9iaD1YYG1qnjeRndYAOVL9cPlDCKXW+A+lws1u kne35cxt7YVprujrXSaxyVNShKguEa7YnHXkxurWEHqmHWCu641wRp29ZWc5zIdUGjb5 nyQ0bbQ0EjsXqd5pssT7O795Zm4dVFDtqQeP5vnjVo7KAIGYcmNBUHQqEkbt8iebFEKv +nAl9/xS7GiPJDb+7WQJmDVV3NqFMtfDvNQmQyofGzZhWndRqbXWI8/FZlVreWnSfgVF 4JCsGtHNhixpIfx/yJXpxXy/6E4yPj6T7ZfAIAb7c59GBQe90no26XaBXYOlZGWut+fh m4Tg== X-Gm-Message-State: AC+VfDyZf+OSLjfWcFkXdrVrKGPEivyRPKcKTrSft5meWg/B0zG7frU9 gJYVsj+ny7rqpDtntSw/faXdmOGNPLEg X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:a615:63d5:b54e:6919]) (user=mshavit job=sendgmr) by 2002:a25:ac4e:0:b0:ba8:5bdf:2e84 with SMTP id r14-20020a25ac4e000000b00ba85bdf2e84mr953711ybd.10.1686053465052; Tue, 06 Jun 2023 05:11:05 -0700 (PDT) Date: Tue, 6 Jun 2023 20:07:51 +0800 In-Reply-To: <20230606120854.4170244-1-mshavit@google.com> Mime-Version: 1.0 References: <20230606120854.4170244-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.rc0.172.g3f132b7071-goog Message-ID: <20230606120854.4170244-16-mshavit@google.com> Subject: [PATCH v2 15/18] iommu/arm-smmu-v3: Allow more re-use for SVA From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED, USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767956008962770717?= X-GMAIL-MSGID: =?utf-8?q?1767956008962770717?= Now that arm-smmu-v3.c supports attaching domains with pasid, SVA can also re-use much of the same logic. This change allows SVA to allocate arm_smmu_domains with a shared CD and attach them using the arm-smmu-v3 set_dev_pasid implementation. Because these domains aren't backed by an iommu_domain we must make sure that an arm_smmu_domain's backing iommu_domain isn't accessed on functions used by SVA. A good rule here would be to allow domain_finalize to access it. Signed-off-by: Michael Shavit --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 63 ++++++++++++++++----- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 11 ++++ 2 files changed, 60 insertions(+), 14 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 69b1d09fd0284..3c5ff4f58934a 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1926,7 +1926,7 @@ static void __arm_smmu_tlb_inv_range(struct arm_smmu_cmdq_ent *cmd, if (smmu->features & ARM_SMMU_FEAT_RANGE_INV) { /* Get the leaf page size */ - tg = __ffs(smmu_domain->domain.pgsize_bitmap); + tg = __ffs(smmu_domain->smmu->pgsize_bitmap); /* Convert page size of 12,14,16 (log2) to 1,2,3 */ cmd->tlbi.tg = (tg - 10) / 2; @@ -2053,6 +2053,14 @@ static bool arm_smmu_capable(struct device *dev, enum iommu_cap cap) } } +static void arm_smmu_init_smmu_domain(struct arm_smmu_domain *smmu_domain) +{ + mutex_init(&smmu_domain->init_mutex); + INIT_LIST_HEAD(&smmu_domain->attached_domains); + spin_lock_init(&smmu_domain->attached_domains_lock); + INIT_LIST_HEAD(&smmu_domain->mmu_notifiers); +} + static struct iommu_domain *arm_smmu_domain_alloc(unsigned type) { struct arm_smmu_domain *smmu_domain; @@ -2075,14 +2083,22 @@ static struct iommu_domain *arm_smmu_domain_alloc(unsigned type) if (!smmu_domain) return NULL; - mutex_init(&smmu_domain->init_mutex); - INIT_LIST_HEAD(&smmu_domain->attached_domains); - spin_lock_init(&smmu_domain->attached_domains_lock); - INIT_LIST_HEAD(&smmu_domain->mmu_notifiers); - + arm_smmu_init_smmu_domain(smmu_domain); return &smmu_domain->domain; } +struct arm_smmu_domain * +arm_smmu_init_shared_cd_domain(struct arm_smmu_device *smmu, + struct arm_smmu_domain *smmu_domain, + struct arm_smmu_ctx_desc *cd) +{ + arm_smmu_init_smmu_domain(smmu_domain); + smmu_domain->smmu = smmu; + smmu_domain->stage = ARM_SMMU_DOMAIN_S1_SHARED_CD; + smmu_domain->shared_cd = cd; + return smmu_domain; +} + static int arm_smmu_bitmap_alloc(unsigned long *map, int span) { int idx, size = 1 << span; @@ -2541,11 +2557,9 @@ static int arm_smmu_set_dev_pasid(struct iommu_domain *domain, struct device *dev, ioasid_t pasid) { int ret = 0; - unsigned long flags; struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); struct arm_smmu_device *smmu; struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); - struct arm_smmu_attached_domain *attached_domain; struct arm_smmu_master *master; if (!fwspec) @@ -2558,6 +2572,18 @@ static int arm_smmu_set_dev_pasid(struct iommu_domain *domain, if (ret) return ret; + return arm_smmu_domain_set_dev_pasid(dev, master, smmu_domain, pasid); +} + +int arm_smmu_domain_set_dev_pasid(struct device *dev, + struct arm_smmu_master *master, + struct arm_smmu_domain *smmu_domain, + ioasid_t pasid) +{ + unsigned long flags; + struct arm_smmu_attached_domain *attached_domain; + int ret; + if (pasid == 0) { dev_err(dev, "pasid 0 is reserved for the device's primary domain\n"); return -ENODEV; @@ -2991,12 +3017,8 @@ static int arm_smmu_def_domain_type(struct device *dev) static void arm_smmu_remove_dev_pasid(struct device *dev, ioasid_t pasid) { struct iommu_domain *domain; - struct arm_smmu_master *master = dev_iommu_priv_get(dev); - struct arm_smmu_domain *smmu_domain; - struct arm_smmu_attached_domain *attached_domain; - unsigned long flags; - if (!master || pasid == 0) + if (pasid == 0) return; domain = iommu_get_domain_for_dev_pasid(dev, pasid, 0); @@ -3005,7 +3027,20 @@ static void arm_smmu_remove_dev_pasid(struct device *dev, ioasid_t pasid) if (domain->type == IOMMU_DOMAIN_SVA) return arm_smmu_sva_remove_dev_pasid(domain, dev, pasid); - smmu_domain = to_smmu_domain(domain); + arm_smmu_domain_remove_dev_pasid(dev, to_smmu_domain(domain), pasid); +} + +void arm_smmu_domain_remove_dev_pasid(struct device *dev, + struct arm_smmu_domain *smmu_domain, + ioasid_t pasid) +{ + struct arm_smmu_attached_domain *attached_domain; + struct arm_smmu_master *master = dev_iommu_priv_get(dev); + unsigned long flags; + + if (!master) + return; + mutex_lock(&arm_smmu_asid_lock); spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); list_for_each_entry(attached_domain, &smmu_domain->attached_domains, domain_head) { diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 4ac69427abf1c..2c33c0461036d 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -761,6 +761,17 @@ void arm_smmu_tlb_inv_range_asid(unsigned long iova, size_t size, int asid, bool arm_smmu_free_asid(struct arm_smmu_ctx_desc *cd); int arm_smmu_atc_inv_domain_ssid(struct arm_smmu_domain *smmu_domain, int ssid, unsigned long iova, size_t size); +int arm_smmu_domain_set_dev_pasid(struct device *dev, + struct arm_smmu_master *master, + struct arm_smmu_domain *smmu_domain, + ioasid_t pasid); +void arm_smmu_domain_remove_dev_pasid(struct device *dev, + struct arm_smmu_domain *smmu_domain, + ioasid_t pasid); +struct arm_smmu_domain * +arm_smmu_init_shared_cd_domain(struct arm_smmu_device *smmu, + struct arm_smmu_domain *smmu_domain, + struct arm_smmu_ctx_desc *cd); #ifdef CONFIG_ARM_SMMU_V3_SVA bool arm_smmu_sva_supported(struct arm_smmu_device *smmu); From patchwork Tue Jun 6 12:07:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 103791 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp3365036vqr; Tue, 6 Jun 2023 05:44:52 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ76DJwWiyKdQuNzPnwo8UJaA0DkNJKkG/EBV2SR80OJ7uvADULgPx2yBtpUVXFT96TND+bI X-Received: by 2002:a05:6a21:32a2:b0:10f:8b61:195f with SMTP id yt34-20020a056a2132a200b0010f8b61195fmr2516086pzb.46.1686055491818; Tue, 06 Jun 2023 05:44:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686055491; cv=none; d=google.com; s=arc-20160816; b=DMO7oCU/2pFZbDyh5bmF/MQOAJPfHieMtG7v0XAFEm2w8F1/G7I4/VstfKjg0jLrCi 8Z+A27UMYIns7a301K88+p0+SOLXEQefDlxVEbL9l/0afIzgnJW5K4qoIfbRe5cdmdxl b43dCgE4WxZHCuYnObQqIYU3a3Z0WPn+2diF+fpeDyXCPTEHwLGLCu0MdO8K3GTtfJZe HYkdHNciiGbK6UesdDQqWVn6V2swriJfitt6j/v//fpRw2qU8VF4Syg6MrQujcwcd4sR HvTNBMvY8Mui0gNx00OpwBHzGgDQCKsx/WU9YgcMZPzpNtKIQjiMtBER+u4jJQe2TKfT X1Zw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=k8coPEvUwDlKI+G80MftH9xxXZgTa0epa7omkndSMHA=; b=0hNWqo/8IK/7HVG7U5iLNTvUsz2hj6nKPayMjy0VAosd6pUj4J3YnT+T1tTbG5jfcj LsmKi7TUpllnUFswcQxTx5jAcXPiSKyYHkAN2bHAy3kdMvHb01pEmnBTmc/yiWh83TCG nkHE3GkUqsQ4uWvaOt1miXDaN7kbm6qdElnajYw6yEf1TGNes4k/Nu60MNxVbJyxNQ8l JNWPDDaWaQOpeUNVZQJIh2BYLtaD1BcIO1I83EoLUXd1S/zM5gegiBjTVuf28x71RWci 4PdwB6Rl03wbQlr6oF1de7nFaDtUs23qpo0LNw+QoBECh2cdZErAKefxNZr7f4sQ3GiE UadQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=vWzHFe+f; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id d12-20020a170902cecc00b001b039693da8si7326319plg.635.2023.06.06.05.44.39; Tue, 06 Jun 2023 05:44:51 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=vWzHFe+f; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237522AbjFFMLl (ORCPT + 99 others); Tue, 6 Jun 2023 08:11:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48276 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237515AbjFFMLh (ORCPT ); Tue, 6 Jun 2023 08:11:37 -0400 Received: from mail-yw1-x1149.google.com (mail-yw1-x1149.google.com [IPv6:2607:f8b0:4864:20::1149]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9B03910E0 for ; Tue, 6 Jun 2023 05:11:14 -0700 (PDT) Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-568a8704f6dso89530197b3.1 for ; Tue, 06 Jun 2023 05:11:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686053472; x=1688645472; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=k8coPEvUwDlKI+G80MftH9xxXZgTa0epa7omkndSMHA=; b=vWzHFe+f6b0SO9ZjCJqoUru52xdUuNnZIq6LnMnCtriPlDLRJZ18LiSlZY3NpYKKCf nMe/iV6ctP3Oz5SwfbarLiRIxyCChHxJbaf7V04aWyS1M/t6tk7jjopXLvNz/UIoBA5O CDebWGr2/1v3E1fW0PClyPNl068DVDA6EOjD4ZobvbmF2BTjWvCkGSao48mifbZOWmNb /QyrEUnZXV/JMNSnzX++RYhSbPNvVocemSoVMQNs4El0N9p5AXTml140OsqVQ7c5W4kG b1q7xBiEzKY39mS8kM2Bpbgxd/iihYyrRoZV2ieeGptRuyuSzNFCxplOVUU1visxhY6O kFPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686053472; x=1688645472; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=k8coPEvUwDlKI+G80MftH9xxXZgTa0epa7omkndSMHA=; b=QOHVQnB8FCcO888UljWUwkjV6GHCDmC11YLLGXp2SPMpFjBdktErzJKGyZR5kDd5Gb KV54nGqXtd/PtOv7G1cLR386CsrJK+oa3nCuNGmq0eHXXWAMW12HmlMg02krtTVZSxEA sRk9CutDc7qjkJXyhgzlAZp8Or+vwJpgLj7DHQnAujD+HGbxmU7FG7x48GoOEiLWgmuB fMGEtYLAzwgxbtscbx/K2SREx1EaZhPhk9Vpztxd2aw6AU2y7cg+iyZERowZT56u/iXe VCwrx0HaJKc031JvtHB2qmeAAJnJgu5AuJjGp4MRZEY3J52ZteAkgvbSrGNusv++6MPx eqKw== X-Gm-Message-State: AC+VfDz2lT7v07PHGE+ImxOqHj7se8kdNychY2SJNiRZL5QRfTFaroz1 2wHgLgMQP4/JFhylY3z+4pWNV96KhvGS X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:a615:63d5:b54e:6919]) (user=mshavit job=sendgmr) by 2002:a81:ad1c:0:b0:55d:6af3:1e2c with SMTP id l28-20020a81ad1c000000b0055d6af31e2cmr959734ywh.3.1686053472400; Tue, 06 Jun 2023 05:11:12 -0700 (PDT) Date: Tue, 6 Jun 2023 20:07:52 +0800 In-Reply-To: <20230606120854.4170244-1-mshavit@google.com> Mime-Version: 1.0 References: <20230606120854.4170244-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.rc0.172.g3f132b7071-goog Message-ID: <20230606120854.4170244-17-mshavit@google.com> Subject: [PATCH v2 16/18] iommu/arm-smmu-v3-sva: Attach S1_SHARED_CD domain From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED, USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767957323407393136?= X-GMAIL-MSGID: =?utf-8?q?1767957323407393136?= Prepare an smmu domain of type S1_SHARED_CD per smmu_mmu_notifier. Attach that domain using the common arm_smmu_domain_set_dev_pasid implementation when attaching an SVA domain. Signed-off-by: Michael Shavit --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 67 ++++++------------- 1 file changed, 22 insertions(+), 45 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index e2a91f20f0906..9a2da579c3563 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -19,7 +19,7 @@ struct arm_smmu_mmu_notifier { bool cleared; refcount_t refs; struct list_head list; - struct arm_smmu_domain *domain; + struct arm_smmu_domain domain; }; #define mn_to_smmu(mn) container_of(mn, struct arm_smmu_mmu_notifier, mn) @@ -198,7 +198,7 @@ static void arm_smmu_mm_invalidate_range(struct mmu_notifier *mn, unsigned long start, unsigned long end) { struct arm_smmu_mmu_notifier *smmu_mn = mn_to_smmu(mn); - struct arm_smmu_domain *smmu_domain = smmu_mn->domain; + struct arm_smmu_domain *smmu_domain = &smmu_mn->domain; size_t size; /* @@ -217,7 +217,7 @@ static void arm_smmu_mm_invalidate_range(struct mmu_notifier *mn, static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) { struct arm_smmu_mmu_notifier *smmu_mn = mn_to_smmu(mn); - struct arm_smmu_domain *smmu_domain = smmu_mn->domain; + struct arm_smmu_domain *smmu_domain = &smmu_mn->domain; struct arm_smmu_master *master; struct arm_smmu_attached_domain *attached_domain; unsigned long flags; @@ -233,15 +233,10 @@ static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) * but disable translation. */ spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); - list_for_each_entry(attached_domain, &smmu_domain->attached_domains, - domain_head) { + list_for_each_entry(attached_domain, &smmu_domain->attached_domains, domain_head) { master = attached_domain->master; - /* - * SVA domains piggyback on the attached_domain with SSID 0. - */ - if (attached_domain->ssid == 0) - arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, - master, mm->pasid, &quiet_cd); + arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, + attached_domain->ssid, &quiet_cd); } spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); @@ -265,15 +260,13 @@ static const struct mmu_notifier_ops arm_smmu_mmu_notifier_ops = { /* Allocate or get existing MMU notifier for this {domain, mm} pair */ static struct arm_smmu_mmu_notifier * -arm_smmu_mmu_notifier_get(struct arm_smmu_domain *smmu_domain, +arm_smmu_mmu_notifier_get(struct arm_smmu_device *smmu, + struct arm_smmu_domain *smmu_domain, struct mm_struct *mm) { int ret; - unsigned long flags; struct arm_smmu_ctx_desc *cd; struct arm_smmu_mmu_notifier *smmu_mn; - struct arm_smmu_master *master; - struct arm_smmu_attached_domain *attached_domain; list_for_each_entry(smmu_mn, &smmu_domain->mmu_notifiers, list) { if (smmu_mn->mn.mm == mm) { @@ -294,7 +287,6 @@ arm_smmu_mmu_notifier_get(struct arm_smmu_domain *smmu_domain, refcount_set(&smmu_mn->refs, 1); smmu_mn->cd = cd; - smmu_mn->domain = smmu_domain; smmu_mn->mn.ops = &arm_smmu_mmu_notifier_ops; ret = mmu_notifier_register(&smmu_mn->mn, mm); @@ -302,24 +294,11 @@ arm_smmu_mmu_notifier_get(struct arm_smmu_domain *smmu_domain, kfree(smmu_mn); goto err_free_cd; } - - spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); - list_for_each_entry(attached_domain, &smmu_domain->attached_domains, - domain_head) { - master = attached_domain->master; - ret = arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, - master, mm->pasid, cd); - } - spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); - if (ret) - goto err_put_notifier; + arm_smmu_init_shared_cd_domain(smmu, &smmu_mn->domain, cd); list_add(&smmu_mn->list, &smmu_domain->mmu_notifiers); return smmu_mn; -err_put_notifier: - /* Frees smmu_mn */ - mmu_notifier_put(&smmu_mn->mn); err_free_cd: arm_smmu_free_shared_cd(cd); return ERR_PTR(ret); @@ -327,27 +306,15 @@ arm_smmu_mmu_notifier_get(struct arm_smmu_domain *smmu_domain, static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) { - unsigned long flags; struct mm_struct *mm = smmu_mn->mn.mm; struct arm_smmu_ctx_desc *cd = smmu_mn->cd; - struct arm_smmu_attached_domain *attached_domain; - struct arm_smmu_master *master; - struct arm_smmu_domain *smmu_domain = smmu_mn->domain; + struct arm_smmu_domain *smmu_domain = &smmu_mn->domain; if (!refcount_dec_and_test(&smmu_mn->refs)) return; list_del(&smmu_mn->list); - spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); - list_for_each_entry(attached_domain, &smmu_domain->attached_domains, - domain_head) { - master = attached_domain->master; - arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, - mm->pasid, NULL); - } - spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); - /* * If we went through clear(), we've already invalidated, and no * new TLB entry can have been formed. @@ -369,17 +336,26 @@ static int __arm_smmu_sva_bind(struct device *dev, struct arm_smmu_master *master = dev_iommu_priv_get(dev); struct iommu_domain *domain = iommu_get_domain_for_dev(dev); struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); + int ret; if (!master || !master->sva_enabled) return -ENODEV; - sva_domain->smmu_mn = arm_smmu_mmu_notifier_get(smmu_domain, + sva_domain->smmu_mn = arm_smmu_mmu_notifier_get(master->smmu, + smmu_domain, mm); if (IS_ERR(sva_domain->smmu_mn)) { sva_domain->smmu_mn = NULL; return PTR_ERR(sva_domain->smmu_mn); } + master->nr_attached_sva_domains += 1; + smmu_domain = &sva_domain->smmu_mn->domain; + ret = arm_smmu_domain_set_dev_pasid(dev, master, smmu_domain, mm->pasid); + if (ret) { + arm_smmu_mmu_notifier_put(sva_domain->smmu_mn); + return ret; + } return 0; } @@ -544,8 +520,9 @@ void arm_smmu_sva_remove_dev_pasid(struct iommu_domain *domain, struct arm_smmu_master *master = dev_iommu_priv_get(dev); mutex_lock(&sva_lock); - master->nr_attached_sva_domains -= 1; + arm_smmu_domain_remove_dev_pasid(dev, &sva_domain->smmu_mn->domain, id); arm_smmu_mmu_notifier_put(sva_domain->smmu_mn); + master->nr_attached_sva_domains -= 1; mutex_unlock(&sva_lock); } From patchwork Tue Jun 6 12:07:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 103785 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp3362699vqr; Tue, 6 Jun 2023 05:40:13 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6jhsc2VbnLTZInK7tmCMcGD3pkXOmOLXTaa8gJ78YHeJHbIRmhoZc4XymWNyAuXvMZng/U X-Received: by 2002:a05:6808:18d:b0:398:2f85:ff7f with SMTP id w13-20020a056808018d00b003982f85ff7fmr2118184oic.50.1686055213090; Tue, 06 Jun 2023 05:40:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686055213; cv=none; d=google.com; s=arc-20160816; b=hIlLkDTtAT8XaWe9+aj40eSvqkb4BphWpMBNap3XnLb+Kk2RLZwJDePduQKhB8R0UT 2KKXfEmTOcZ00JL9JAmIkMOqCYZBM7IsYZKhbUlQSze6V9eIEsEt2kFwD1Qznwbl3lqO vqiyNxjlcqCWehq2GqwEZcmTnW30+oLQrQdKYzTdwS+e0dY2ePMi+zgLoo/k/S1uOlpx ekDWHkc4MSn6KPRfTREkA+0CI2TFuzfeyEY5aXAbaf/AT0oOgCQQqswhiRR4VZgoE3FI Kn9uOzv+FZZEEmBqj+B/Lk3SGu95EMmzuBn/LoD1RgPiECrHV2zrtlVnm9DcpKGgvA1L 9LTw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=t4Hj/7++tZGwCPYZCwYpw7ebOJKZDYQm/8wAZtGxvBQ=; b=X7o3JY5jTgTR7ceD30K5CCUv6dxEC83nfCzZrEyCarP4463rE8aZwgnxU9UbIcE2tO aSVyqgkG2ErCA8CTu3pois3zC4C74Psp2eNhc0iQZSMR8d/0jqujAl/FOha8VYmG7Vmr uqGVkTDS1gDs4PseckJVktDKBF1VCFh2b96z+h7SJZQC26gFcpvzXDyDOCQA6oaZSDcY dypvqUgYzotYOMp4eRqtjWNr+f+PZpGmA9GZ/02LI5G9C/Rmh9ZQZW4xpUjaGKd4mWDm TFF2HgFyt/w00ZslKwud3opD9KlhgOT63zyT2kVmD4tmDCJvy3DrDLjwis3+a10Z0rzH KuCA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=tvcKBuXm; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id v24-20020a637a18000000b00524ecfa05d8si7505731pgc.15.2023.06.06.05.40.00; Tue, 06 Jun 2023 05:40:13 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=tvcKBuXm; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237561AbjFFMLv (ORCPT + 99 others); Tue, 6 Jun 2023 08:11:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48402 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237449AbjFFMLj (ORCPT ); Tue, 6 Jun 2023 08:11:39 -0400 Received: from mail-yw1-x1149.google.com (mail-yw1-x1149.google.com [IPv6:2607:f8b0:4864:20::1149]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 93CAD10E6 for ; Tue, 6 Jun 2023 05:11:21 -0700 (PDT) Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-565a336caa0so98009867b3.1 for ; Tue, 06 Jun 2023 05:11:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686053479; x=1688645479; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=t4Hj/7++tZGwCPYZCwYpw7ebOJKZDYQm/8wAZtGxvBQ=; b=tvcKBuXmjcLSD6iUqEdjKkd3pjcW4Bgh2u7k7FhSOJ9MGZTQnIaSWKko/P99xTYVVN iQnaTLb8zj7e/VoY15bc6Mw1W9iPTt0hZMjbPFyPEYnptWJCScPiJPt4tWdXsyXHsO+P gp36FkV0kxNfBjzogTXR9DDW9oucqrdXHcWBZLfDLsvddtSL990dMhINvFRRLE7S0+MX ZynWgj6jxtuOzKbhLTqSqz3k6XpUEVduVIFfHenqn5whjfD/DaqykP3VEdW3yujAx8OO aoDD2uEAol7D8se7oYPQAf+WTlEmk3xSuhFYNAY0mS+9pV4pFZDuvyzqmGrDYSDktahz d0/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686053479; x=1688645479; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=t4Hj/7++tZGwCPYZCwYpw7ebOJKZDYQm/8wAZtGxvBQ=; b=Lf02O/puPPCG5SrTNploEqcxerPDNYeDz1+eSdFK1DER2EcU1gqxNBdEt3gQMVEKE0 645GD7fbJG379gvLsMSeDWG2eLWI+7QOFXDnUDZhWqsROhltcI6UCTmDk+zKQwYC3y/G FsJFymKwumnkWbNADaHcNhq+RSSpmc+MhoLtZpg+Oq3UYJrtaTPKdbxSsj8q6tYeVDv6 UcNwt3QjL3tm1OaeEKsd8RJZ5swu1GC2pzB8USKRl12JHtz1+7Bpvg9YkxgUCJnpIaNq r5eJLG3T11urdyKezJHNEdyuOL7xet3XtLPnceyztaRXTY2G7G1zemYX0Qwi2ty4rWva qyog== X-Gm-Message-State: AC+VfDwN8wjmk08OtPkhVqD03j4oEO5cWS8lD6EL88XIHIVOEXGnBG8w JPZp130Ai9oWJkNU06Ew4q93Aiz/+Zuf X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:a615:63d5:b54e:6919]) (user=mshavit job=sendgmr) by 2002:a81:aa12:0:b0:55a:3133:86fa with SMTP id i18-20020a81aa12000000b0055a313386famr966118ywh.3.1686053479602; Tue, 06 Jun 2023 05:11:19 -0700 (PDT) Date: Tue, 6 Jun 2023 20:07:53 +0800 In-Reply-To: <20230606120854.4170244-1-mshavit@google.com> Mime-Version: 1.0 References: <20230606120854.4170244-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.rc0.172.g3f132b7071-goog Message-ID: <20230606120854.4170244-18-mshavit@google.com> Subject: [PATCH v2 17/18] iommu/arm-smmu-v3-sva: Alloc notifier for {smmu,mn} From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED, USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767957030915835391?= X-GMAIL-MSGID: =?utf-8?q?1767957030915835391?= The arm_smmu_nofitier for an mn can be shared across all devices with the same upstream smmu. This breaks the last remaining explicit dependency on the device's primary domain in arm-smmu-v3-sva. Signed-off-by: Michael Shavit --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 18 +++++++----------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 2 +- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 4 ++-- 3 files changed, 10 insertions(+), 14 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 9a2da579c3563..3e49838e4f55c 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -258,17 +258,16 @@ static const struct mmu_notifier_ops arm_smmu_mmu_notifier_ops = { .free_notifier = arm_smmu_mmu_notifier_free, }; -/* Allocate or get existing MMU notifier for this {domain, mm} pair */ +/* Allocate or get existing MMU notifier for this {smmu, mm} pair */ static struct arm_smmu_mmu_notifier * arm_smmu_mmu_notifier_get(struct arm_smmu_device *smmu, - struct arm_smmu_domain *smmu_domain, struct mm_struct *mm) { int ret; struct arm_smmu_ctx_desc *cd; struct arm_smmu_mmu_notifier *smmu_mn; - list_for_each_entry(smmu_mn, &smmu_domain->mmu_notifiers, list) { + list_for_each_entry(smmu_mn, &smmu->mmu_notifiers, list) { if (smmu_mn->mn.mm == mm) { refcount_inc(&smmu_mn->refs); return smmu_mn; @@ -296,9 +295,8 @@ arm_smmu_mmu_notifier_get(struct arm_smmu_device *smmu, } arm_smmu_init_shared_cd_domain(smmu, &smmu_mn->domain, cd); - list_add(&smmu_mn->list, &smmu_domain->mmu_notifiers); + list_add(&smmu_mn->list, &smmu->mmu_notifiers); return smmu_mn; - err_free_cd: arm_smmu_free_shared_cd(cd); return ERR_PTR(ret); @@ -314,7 +312,6 @@ static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) return; list_del(&smmu_mn->list); - /* * If we went through clear(), we've already invalidated, and no * new TLB entry can have been formed. @@ -331,18 +328,17 @@ static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) static int __arm_smmu_sva_bind(struct device *dev, struct arm_smmu_sva_domain *sva_domain, - struct mm_struct *mm) + struct mm_struct *mm, + ioasid_t id) { struct arm_smmu_master *master = dev_iommu_priv_get(dev); - struct iommu_domain *domain = iommu_get_domain_for_dev(dev); - struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); + struct arm_smmu_domain *smmu_domain; int ret; if (!master || !master->sva_enabled) return -ENODEV; sva_domain->smmu_mn = arm_smmu_mmu_notifier_get(master->smmu, - smmu_domain, mm); if (IS_ERR(sva_domain->smmu_mn)) { sva_domain->smmu_mn = NULL; @@ -533,7 +529,7 @@ static int arm_smmu_sva_set_dev_pasid(struct iommu_domain *domain, struct mm_struct *mm = domain->mm; mutex_lock(&sva_lock); - ret = __arm_smmu_sva_bind(dev, to_sva_domain(domain), mm); + ret = __arm_smmu_sva_bind(dev, to_sva_domain(domain), mm, id); mutex_unlock(&sva_lock); return ret; diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 3c5ff4f58934a..e68c5264c6171 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2058,7 +2058,6 @@ static void arm_smmu_init_smmu_domain(struct arm_smmu_domain *smmu_domain) mutex_init(&smmu_domain->init_mutex); INIT_LIST_HEAD(&smmu_domain->attached_domains); spin_lock_init(&smmu_domain->attached_domains_lock); - INIT_LIST_HEAD(&smmu_domain->mmu_notifiers); } static struct iommu_domain *arm_smmu_domain_alloc(unsigned type) @@ -2859,6 +2858,7 @@ static struct iommu_device *arm_smmu_probe_device(struct device *dev) goto err_free_master; } + INIT_LIST_HEAD(&smmu->mmu_notifiers); return &smmu->iommu; err_free_master: diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 2c33c0461036d..041b0e532ac3d 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -674,6 +674,8 @@ struct arm_smmu_device { struct rb_root streams; struct mutex streams_mutex; + + struct list_head mmu_notifiers; }; struct arm_smmu_stream { @@ -737,8 +739,6 @@ struct arm_smmu_domain { struct list_head attached_domains; spinlock_t attached_domains_lock; - - struct list_head mmu_notifiers; }; static inline struct arm_smmu_domain *to_smmu_domain(struct iommu_domain *dom) From patchwork Tue Jun 6 12:07:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 103787 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp3363057vqr; Tue, 6 Jun 2023 05:40:55 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5oDAaM3VAR8uJu4q6k3X5vP4ZCIMpYKPmy1lpzsMg46lh1q3xSl8eHNiINideVDMvgChXz X-Received: by 2002:a05:6a20:4295:b0:110:9210:f6b8 with SMTP id o21-20020a056a20429500b001109210f6b8mr3090447pzj.35.1686055254803; Tue, 06 Jun 2023 05:40:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686055254; cv=none; d=google.com; s=arc-20160816; b=isW+mm7BffR6rHOFSJ+dwhjrt8YYGBfFx0aXg72R2HrXbq86+DMfr1fulePiXM3gJF Yyfp10pc+V3Wdm8Vmvb1CtUpgXzxq10DjoHXW2vZ8dxFLnuFiLyHkAzTvnbojnJZ0Vl/ kAIMhYJrf09S65CNA8rmYztCWhXpucKKXSyaVmiG0I+GLWRfYSaKxFLtGrAtzE7eUHOt By+VWYWFSaQNVX8jUGziL/E192y5OTZoPaP4PFEtZ54/LzlM3whxdanQmuALB1QFA5/T aeDjd/xqH8/K2AeF9OIezPKqrNXPBX+lfc3JMtL4F8Rfplq13Tnjtqcz2IKXhulh15Eg l9JA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=690IhMaVKQOheawjxO/ZDJYW5nna84UvkxSb29OMpJY=; b=SkefzIjLYiibnhgingCFyA/9g1k+c21q08f6yqxTbgP9geQ/2cE3Bntm1dGMrGWD/y njxOpDXUhhFemcP2FZkErjSYlAbjjpD6dvfpt4OJoFC+hB0i4XBqN9jGxRR299fDBPN2 6CFewvJcgzgPHOjbUkmWu6yuNhiV+GMPuAwNrKDKekHlkPXZm8yeWTlT4Y+RAsoYqKaD OeAivToEFYxGw6enVOg8JR+lAhKQwvgATJAEFpFMOOvL2sMJXhioz08hJ2R6GM/GINZF nJ5vB84TmZtJ4+FNU7qvlAK3T1kGJn4ks7JInD6JH/1mLQ8a7U3T7mJMKPJhHf/4f3fu RrWQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=eW2xKOUz; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id y72-20020a638a4b000000b00517f7c24652si7298956pgd.890.2023.06.06.05.40.40; Tue, 06 Jun 2023 05:40:54 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=eW2xKOUz; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237571AbjFFML7 (ORCPT + 99 others); Tue, 6 Jun 2023 08:11:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48210 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237558AbjFFMLt (ORCPT ); Tue, 6 Jun 2023 08:11:49 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 71A04E7E for ; Tue, 6 Jun 2023 05:11:28 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-bacd408046cso8996937276.3 for ; Tue, 06 Jun 2023 05:11:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686053486; x=1688645486; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=690IhMaVKQOheawjxO/ZDJYW5nna84UvkxSb29OMpJY=; b=eW2xKOUzDLrIwpVZxF6w1AffFH+eybFfzAor2EtdG3o/dhStpX5h+hko1PkiB6GL7y mmS6o7zPFxcUQRSRX5DG/d0PY60IviKLBvNBEtNsYPu/n5rjxNTwETco8yESLc1KVOOd wk+YW7cQw9e9JkvXKhVdJLpbLQn3z9yJ/KLwX3YpX8iS5O24ALewvrvR0jWyO9RzWlM8 oBsNOOWQlcWJkiplzzMhLA+1nJr7ffwmXaxT30h7zIprl7PUkqEBHdk4bIQdyCKLuL2P Z/jOGUaNT82GRH9Ra+nCHqqwczwcKPcwLQGL1DiptDxhlo6Fn6zbRcEFclgC4GlKyr69 EAqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686053486; x=1688645486; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=690IhMaVKQOheawjxO/ZDJYW5nna84UvkxSb29OMpJY=; b=c38LxmX0rNOL/BHIerVNR4eiW0/pZYwKtQoMWXee+YMCc9CeyYb2CWcuPEvsNaBBzw Zg21+UhoidQ0C7hGipIosa6zC02xkyupNEmdhi/bGshbblmga3LhqIfBoG0uimKutMRx BCvjvexITCXLDdTvmrBGO/xXvMJdcqyJpp+gHKOobllOZP1MgWUhAKrZiHWevYK3uxwh AwMc4cThRTYDHDI6KwEsQmRGWypzu3tpI0W6Bbo1koYcKiD68BFWpEaygVzg2LXJR0Fu 1nZfMt57zGH9x/07Zrw424TvlwHplZSJgLsxQ9cDJwZ7nEY0poNdbe4rptoDQI05kBeW CwHQ== X-Gm-Message-State: AC+VfDxnnQ/3mLpPj7K8Xlb6XG+Ow1bK9xnvNsWs2Lmy0j0rciagbZmX tBGdnz9lmTvDGuyhaF61I3qpzBatirqE X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:a615:63d5:b54e:6919]) (user=mshavit job=sendgmr) by 2002:a05:6902:100b:b0:bad:600:1833 with SMTP id w11-20020a056902100b00b00bad06001833mr1035863ybt.0.1686053486705; Tue, 06 Jun 2023 05:11:26 -0700 (PDT) Date: Tue, 6 Jun 2023 20:07:54 +0800 In-Reply-To: <20230606120854.4170244-1-mshavit@google.com> Mime-Version: 1.0 References: <20230606120854.4170244-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.rc0.172.g3f132b7071-goog Message-ID: <20230606120854.4170244-19-mshavit@google.com> Subject: [PATCH v2 18/18] iommu/arm-smmu-v3-sva: Remove atc_inv_domain_ssid From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767957074921184163?= X-GMAIL-MSGID: =?utf-8?q?1767957074921184163?= arm_smmu_atc_inv_domain is sufficient in all cases now that arm_smmu_domain always tracks all the master/ssids that it is attached to. Also remove the last usage of mm->pasid in arm-smmu-v3-sva. Signed-off-by: Michael Shavit --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 9 ++++---- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 21 +++---------------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 4 ++-- 3 files changed, 9 insertions(+), 25 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 3e49838e4f55c..5a124281bbef6 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -211,7 +211,7 @@ static void arm_smmu_mm_invalidate_range(struct mmu_notifier *mn, if (!(smmu_domain->smmu->features & ARM_SMMU_FEAT_BTM)) arm_smmu_tlb_inv_range_asid(start, size, smmu_mn->cd->asid, PAGE_SIZE, false, smmu_domain); - arm_smmu_atc_inv_domain_ssid(smmu_domain, mm->pasid, start, size); + arm_smmu_atc_inv_domain(smmu_domain, start, size); } static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) @@ -241,7 +241,7 @@ static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); arm_smmu_tlb_inv_asid(smmu_domain->smmu, smmu_mn->cd->asid); - arm_smmu_atc_inv_domain_ssid(smmu_domain, mm->pasid, 0, 0); + arm_smmu_atc_inv_domain(smmu_domain, 0, 0); smmu_mn->cleared = true; mutex_unlock(&sva_lock); @@ -304,7 +304,6 @@ arm_smmu_mmu_notifier_get(struct arm_smmu_device *smmu, static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) { - struct mm_struct *mm = smmu_mn->mn.mm; struct arm_smmu_ctx_desc *cd = smmu_mn->cd; struct arm_smmu_domain *smmu_domain = &smmu_mn->domain; @@ -318,7 +317,7 @@ static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) */ if (!smmu_mn->cleared) { arm_smmu_tlb_inv_asid(smmu_domain->smmu, cd->asid); - arm_smmu_atc_inv_domain_ssid(smmu_domain, mm->pasid, 0, 0); + arm_smmu_atc_inv_domain(smmu_domain, 0, 0); } /* Frees smmu_mn */ @@ -347,7 +346,7 @@ static int __arm_smmu_sva_bind(struct device *dev, master->nr_attached_sva_domains += 1; smmu_domain = &sva_domain->smmu_mn->domain; - ret = arm_smmu_domain_set_dev_pasid(dev, master, smmu_domain, mm->pasid); + ret = arm_smmu_domain_set_dev_pasid(dev, master, smmu_domain, id); if (ret) { arm_smmu_mmu_notifier_put(sva_domain->smmu_mn); return ret; diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index e68c5264c6171..1e02e73e586f7 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1821,13 +1821,8 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) return arm_smmu_atc_inv_master_ssid(master, 0); } -/* - * If ssid is non-zero, issue atc invalidations with the given ssid instead of - * the one the domain is attached to. This is used by SVA since it's pasid - * attachments aren't recorded in smmu_domain yet. - */ -int arm_smmu_atc_inv_domain_ssid(struct arm_smmu_domain *smmu_domain, int ssid, - unsigned long iova, size_t size) +int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, + unsigned long iova, size_t size) { int i; unsigned long flags; @@ -1866,11 +1861,7 @@ int arm_smmu_atc_inv_domain_ssid(struct arm_smmu_domain *smmu_domain, int ssid, master = attached_domain->master; if (!master->ats_enabled) continue; - if (ssid != 0) - arm_smmu_atc_inv_cmd_set_ssid(ssid, &cmd); - else - arm_smmu_atc_inv_cmd_set_ssid(attached_domain->ssid, &cmd); - + arm_smmu_atc_inv_cmd_set_ssid(attached_domain->ssid, &cmd); for (i = 0; i < master->num_streams; i++) { cmd.atc.sid = master->streams[i].id; arm_smmu_cmdq_batch_add(smmu_domain->smmu, &cmds, &cmd); @@ -1881,12 +1872,6 @@ int arm_smmu_atc_inv_domain_ssid(struct arm_smmu_domain *smmu_domain, int ssid, return arm_smmu_cmdq_batch_submit(smmu_domain->smmu, &cmds); } -int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, - unsigned long iova, size_t size) -{ - return arm_smmu_atc_inv_domain_ssid(smmu_domain, 0, iova, size); -} - /* IO_PGTABLE API */ static void arm_smmu_tlb_inv_context(void *cookie) { diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 041b0e532ac3d..9c382bc8c0549 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -759,8 +759,8 @@ void arm_smmu_tlb_inv_range_asid(unsigned long iova, size_t size, int asid, size_t granule, bool leaf, struct arm_smmu_domain *smmu_domain); bool arm_smmu_free_asid(struct arm_smmu_ctx_desc *cd); -int arm_smmu_atc_inv_domain_ssid(struct arm_smmu_domain *smmu_domain, int ssid, - unsigned long iova, size_t size); +int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, + unsigned long iova, size_t size); int arm_smmu_domain_set_dev_pasid(struct device *dev, struct arm_smmu_master *master, struct arm_smmu_domain *smmu_domain,