From patchwork Wed Jun 14 15:41:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 108039 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp78043vqr; Wed, 14 Jun 2023 09:34:31 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5yZqzI4GGxf/zooEWiFxUosai6L2RtOGct2VSQ0PaF6ZC6SxYChTbpsBD8vbmgEEp71yUq X-Received: by 2002:a2e:910e:0:b0:2af:160d:888a with SMTP id m14-20020a2e910e000000b002af160d888amr8095108ljg.1.1686760471562; Wed, 14 Jun 2023 09:34:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686760471; cv=none; d=google.com; s=arc-20160816; b=fJ8JrukpupSFyqnbwZzeBsLnGImqohC1lR+jm3JNKMpLren08HXuU7Dgq2sZ2oshmy feTgZ+3x1OaDkYntv3eIEs7kEyaGxtKtrSLzFJm+D2cLz0AvzkEkczCmSJroqsVOyxPe akeeDgidD8HLtekmG8QemylMHTSF1AboGVDenE8y59s/TEbHaYL34hQJS9z+XHCw0QZi 6g4eYyEuL8m8DGcM2w6xn8B/mA8P6ye4F2l5y/21zJ65+D7LgYW5Vd5QbuKHgeOAnm3e b09XJHL3yA/TmxXT379nSDcytgPOfWiEPblx7xEHYz56Iq6I0OsQgss1nuVbUcWDA8Ux 8Wtw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=xwDD9pVjLa+TJXAJEbbYyI+224KEPbc+a+2w73kHUiI=; b=SnEY9LQ4fhOtsJ/s5F2TsHtBZuUdqOcCYuf3xexvqBTVE1d/xFYlmhX/yIntjoQbaw wv70FUFJaZHfH9Ou+KUjU2ufEVI5VxqRetkBM5xEa5c7W6tcGtW+oT1CJ0hx/F8PazHf 4+zL/t1B1xeHRn8q58C/Z1eLc5HBU3lYgI8X7N8/Uz1bz2lFZqGZq7x4KbvKpXcClzAx 1PiqACU0pIh7XBdMOhe7gs4DIpVc0l3B2ijHXuOFqIQFkolPuM5ZC/fm1kEXu5ogxzKK RZTxaE540nfYRAY4r78feVTEsJKkD/exKBTwYFvJdFs6e9KWzM3/fZKUXc0uxuxSldF6 eU5w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=XcbpIeR4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id w19-20020a50fa93000000b00518584f0859si3261567edr.120.2023.06.14.09.34.00; Wed, 14 Jun 2023 09:34:31 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=XcbpIeR4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244500AbjFNPnl (ORCPT + 99 others); Wed, 14 Jun 2023 11:43:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42360 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244362AbjFNPnh (ORCPT ); Wed, 14 Jun 2023 11:43:37 -0400 Received: from mail-yw1-x114a.google.com (mail-yw1-x114a.google.com [IPv6:2607:f8b0:4864:20::114a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AD5D42101 for ; Wed, 14 Jun 2023 08:43:35 -0700 (PDT) Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-561ceb5b584so11999697b3.3 for ; Wed, 14 Jun 2023 08:43:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686757415; x=1689349415; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=xwDD9pVjLa+TJXAJEbbYyI+224KEPbc+a+2w73kHUiI=; b=XcbpIeR43JDZfn4lLCsVf0JdsLi6nRjRM50ZNMNNgoiXVaiw+/ZGHnE5KTJjMCs9aO vnK/+7uf6oxkg7kl2xEDoB5F4B2QtCLAzLVEn4t/E3os9JAaCcYrN3AO74r8GCTCI1ai CNeyG9s42Dtj6TN/jMZhQ7UQlbA6doeYtm3JtoHKSi/95yADdM887JCovkOxifmKCPA2 z9m+8SiMXacLLqG5IobUr532pG3VZDDME93RDKR+p2TzHi9YlbOhVRmhqLQz3NpdaYUk zE23NqAZ3c+vCO3XhN19QdpZYVPa2PS0tkE5LTgBL+aO1vkz+gKWJS2k5UoUETqYjKW/ Dl/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686757415; x=1689349415; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=xwDD9pVjLa+TJXAJEbbYyI+224KEPbc+a+2w73kHUiI=; b=Hh6tVOmnI3vqq6paByUgg6VUJ0DwT1wK7dlIk6rpSeuqzOgkotpqQmoKO5ztiVpMjG lEyOgr9HmdY/3mzF6ssi8PtMoFbwtfCAEYV0bjaejRfFCj7kmehrqgcIFpg+asAJJ2GG Xal5dLXhd+ILQgzx3Dtl42Msml3sh+jY4Yc5JwvCaDe+R23Xtmco6fZCvEsoTUB6MEWP eNQFOVQ3jW35QMtwLyvva5i71CCvkdR+YH0aIBnaFgmb9sh3TdosIAbR/QLT84oR3ru3 xsYblU4fRYiKI1KDTi8JHlTHJX5uenQuISJsKvgRYqr7wSWNPZsSmuqrpdFyKQRCwoC4 Rhww== X-Gm-Message-State: AC+VfDzMO2pMrxtfh5xBjvPYf79JKVb40hdF98bSSyBASugZly0evlAC hDedJTz/GzIxsp+FX/VhSvpObS0ZiXZr X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:c6e6:49bf:5c44:5965]) (user=mshavit job=sendgmr) by 2002:a81:ac49:0:b0:561:c9c0:98d9 with SMTP id z9-20020a81ac49000000b00561c9c098d9mr1044537ywj.4.1686757414962; Wed, 14 Jun 2023 08:43:34 -0700 (PDT) Date: Wed, 14 Jun 2023 23:41:53 +0800 In-Reply-To: <20230614154304.2860121-1-mshavit@google.com> Mime-Version: 1.0 References: <20230614154304.2860121-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230614154304.2860121-2-mshavit@google.com> Subject: [PATCH v3 01/13] iommu/arm-smmu-v3: Move ctx_desc out of s1_cfg From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768696548004631529?= X-GMAIL-MSGID: =?utf-8?q?1768696548004631529?= s1_cfg describes the CD table that is inserted into an SMMU's STEs. It's weird for s1_cfg to also own ctx_desc which describes a CD that is inserted into that table. It is more appropriate for arm_smmu_domain to own ctx_desc. Signed-off-by: Michael Shavit --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 2 +- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 23 +++++++-------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 28 ++++++++++--------- 3 files changed, 28 insertions(+), 25 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index a5a63b1c947eb..968559d625c40 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -62,7 +62,7 @@ arm_smmu_share_asid(struct mm_struct *mm, u16 asid) return cd; } - smmu_domain = container_of(cd, struct arm_smmu_domain, s1_cfg.cd); + smmu_domain = container_of(cd, struct arm_smmu_domain, cd); smmu = smmu_domain->smmu; ret = xa_alloc(&arm_smmu_asid_xa, &new_asid, cd, diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 3fd83fb757227..beff04b897718 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1863,7 +1863,7 @@ static void arm_smmu_tlb_inv_context(void *cookie) * careful, 007. */ if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { - arm_smmu_tlb_inv_asid(smmu, smmu_domain->s1_cfg.cd.asid); + arm_smmu_tlb_inv_asid(smmu, smmu_domain->cd.asid); } else { cmd.opcode = CMDQ_OP_TLBI_S12_VMALL; cmd.tlbi.vmid = smmu_domain->s2_cfg.vmid; @@ -1946,7 +1946,7 @@ static void arm_smmu_tlb_inv_range_domain(unsigned long iova, size_t size, if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { cmd.opcode = smmu_domain->smmu->features & ARM_SMMU_FEAT_E2H ? CMDQ_OP_TLBI_EL2_VA : CMDQ_OP_TLBI_NH_VA; - cmd.tlbi.asid = smmu_domain->s1_cfg.cd.asid; + cmd.tlbi.asid = smmu_domain->cd.asid; } else { cmd.opcode = CMDQ_OP_TLBI_S2_IPA; cmd.tlbi.vmid = smmu_domain->s2_cfg.vmid; @@ -2077,7 +2077,7 @@ static void arm_smmu_domain_free(struct iommu_domain *domain) mutex_lock(&arm_smmu_asid_lock); if (cfg->cdcfg.cdtab) arm_smmu_free_cd_tables(smmu_domain); - arm_smmu_free_asid(&cfg->cd); + arm_smmu_free_asid(&smmu_domain->cd); mutex_unlock(&arm_smmu_asid_lock); } else { struct arm_smmu_s2_cfg *cfg = &smmu_domain->s2_cfg; @@ -2096,13 +2096,14 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, u32 asid; struct arm_smmu_device *smmu = smmu_domain->smmu; struct arm_smmu_s1_cfg *cfg = &smmu_domain->s1_cfg; + struct arm_smmu_ctx_desc *cd = &smmu_domain->cd; typeof(&pgtbl_cfg->arm_lpae_s1_cfg.tcr) tcr = &pgtbl_cfg->arm_lpae_s1_cfg.tcr; - refcount_set(&cfg->cd.refs, 1); + refcount_set(&cd->refs, 1); /* Prevent SVA from modifying the ASID until it is written to the CD */ mutex_lock(&arm_smmu_asid_lock); - ret = xa_alloc(&arm_smmu_asid_xa, &asid, &cfg->cd, + ret = xa_alloc(&arm_smmu_asid_xa, &asid, cd, XA_LIMIT(1, (1 << smmu->asid_bits) - 1), GFP_KERNEL); if (ret) goto out_unlock; @@ -2115,23 +2116,23 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, if (ret) goto out_free_asid; - cfg->cd.asid = (u16)asid; - cfg->cd.ttbr = pgtbl_cfg->arm_lpae_s1_cfg.ttbr; - cfg->cd.tcr = FIELD_PREP(CTXDESC_CD_0_TCR_T0SZ, tcr->tsz) | + cd->asid = (u16)asid; + cd->ttbr = pgtbl_cfg->arm_lpae_s1_cfg.ttbr; + cd->tcr = FIELD_PREP(CTXDESC_CD_0_TCR_T0SZ, tcr->tsz) | FIELD_PREP(CTXDESC_CD_0_TCR_TG0, tcr->tg) | FIELD_PREP(CTXDESC_CD_0_TCR_IRGN0, tcr->irgn) | FIELD_PREP(CTXDESC_CD_0_TCR_ORGN0, tcr->orgn) | FIELD_PREP(CTXDESC_CD_0_TCR_SH0, tcr->sh) | FIELD_PREP(CTXDESC_CD_0_TCR_IPS, tcr->ips) | CTXDESC_CD_0_TCR_EPD1 | CTXDESC_CD_0_AA64; - cfg->cd.mair = pgtbl_cfg->arm_lpae_s1_cfg.mair; + cd->mair = pgtbl_cfg->arm_lpae_s1_cfg.mair; /* * Note that this will end up calling arm_smmu_sync_cd() before * the master has been added to the devices list for this domain. * This isn't an issue because the STE hasn't been installed yet. */ - ret = arm_smmu_write_ctx_desc(smmu_domain, 0, &cfg->cd); + ret = arm_smmu_write_ctx_desc(smmu_domain, 0, cd); if (ret) goto out_free_cd_tables; @@ -2141,7 +2142,7 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, out_free_cd_tables: arm_smmu_free_cd_tables(smmu_domain); out_free_asid: - arm_smmu_free_asid(&cfg->cd); + arm_smmu_free_asid(cd); out_unlock: mutex_unlock(&arm_smmu_asid_lock); return ret; diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index b574c58a34876..68d519f21dbd8 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -593,7 +593,6 @@ struct arm_smmu_ctx_desc_cfg { struct arm_smmu_s1_cfg { struct arm_smmu_ctx_desc_cfg cdcfg; - struct arm_smmu_ctx_desc cd; u8 s1fmt; u8 s1cdmax; }; @@ -707,25 +706,28 @@ enum arm_smmu_domain_stage { }; struct arm_smmu_domain { - struct arm_smmu_device *smmu; - struct mutex init_mutex; /* Protects smmu pointer */ + struct arm_smmu_device *smmu; + struct mutex init_mutex; /* Protects smmu pointer */ - struct io_pgtable_ops *pgtbl_ops; - bool stall_enabled; - atomic_t nr_ats_masters; + struct io_pgtable_ops *pgtbl_ops; + bool stall_enabled; + atomic_t nr_ats_masters; - enum arm_smmu_domain_stage stage; + enum arm_smmu_domain_stage stage; union { - struct arm_smmu_s1_cfg s1_cfg; - struct arm_smmu_s2_cfg s2_cfg; + struct { + struct arm_smmu_ctx_desc cd; + struct arm_smmu_s1_cfg s1_cfg; + }; + struct arm_smmu_s2_cfg s2_cfg; }; - struct iommu_domain domain; + struct iommu_domain domain; - struct list_head devices; - spinlock_t devices_lock; + struct list_head devices; + spinlock_t devices_lock; - struct list_head mmu_notifiers; + struct list_head mmu_notifiers; }; static inline struct arm_smmu_domain *to_smmu_domain(struct iommu_domain *dom) From patchwork Wed Jun 14 15:41:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 108037 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp76380vqr; Wed, 14 Jun 2023 09:32:11 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7qAh83oytrGRXuCrh7J2T1JFhypnEf7rSqJP0YYRKt2MJaoaOXJQ8jUVPFpIne8KdvJIvY X-Received: by 2002:a17:907:1629:b0:96a:fd8a:f840 with SMTP id hb41-20020a170907162900b0096afd8af840mr17742081ejc.49.1686760331544; Wed, 14 Jun 2023 09:32:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686760331; cv=none; d=google.com; s=arc-20160816; b=rJTwhiAH9b4jW3BJ/GldYgJZwkxXC4/LNEmE0doHE2uiEhEAsFk7G9etY9Dbf5UVL0 Tz2PdCCIo/Qxqoxj/fJxzXWQFk3Yth6Ur6nROHynVEcO7AmjtjRoAUvmIRWjWl5/kRkp f/Rb4/XMVJFgcbG3k9TS+734xbcXPGisiKErp5d78U4dT4//i/AcanpmHeYcRpl9MUXD oJeVmAtTGIOm1kAGcIHSBggF6VCGgSKlZpqRg1kXEj5FQd+EUft1/a/IcvbedIStuKv3 HwsXzkqHaPiLt9sB1+DK7pKu7sT3lxUT6vJbNM09RUsEta2rrWHW4/QYMkR9AFuzw/dr Le1w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=9vic1ayWBGr79y9f6y1FKqdm9/OskuVUCQ3Tr10KopQ=; b=PLU+nMs+VpVFDudjD7qqeFyM3SBXNM1KIcg50kPxpRKxVQPWygsDIfm154nGREiX9p 56vC5D/foTc1fuGBg1juh4ZEsWV+9llVVQLPVCK5hxJDOX1HP8pqj9NG8yNKgvGA8JrN AChNmLSINANF5HqXCKk5O3pXP6zAX5sNa1IkwQLaVTJgDUtVCkxtGfYBXfYgJRZJmeOi HLdbcdIjW/2Cv6syaFWIEbLNBfWwydIj6cUC+y3ZmGq0xaMt6g3izb1v+D3jkwWHIgLV Z+f/iybvKOnUDV6y8NgNIkqKZLgbjNuY/MThKWNsK1FYBwtRGrKyMSMSpbs4Kt9olwtd wJDg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=qhhtV69W; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id uz25-20020a170907119900b0096a3525cb1fsi8783721ejb.166.2023.06.14.09.31.32; Wed, 14 Jun 2023 09:32:11 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=qhhtV69W; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236184AbjFNPnu (ORCPT + 99 others); Wed, 14 Jun 2023 11:43:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42436 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244579AbjFNPno (ORCPT ); Wed, 14 Jun 2023 11:43:44 -0400 Received: from mail-yw1-x114a.google.com (mail-yw1-x114a.google.com [IPv6:2607:f8b0:4864:20::114a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9D4B32114 for ; Wed, 14 Jun 2023 08:43:43 -0700 (PDT) Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-57003dac4a8so16250487b3.1 for ; Wed, 14 Jun 2023 08:43:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686757423; x=1689349423; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=9vic1ayWBGr79y9f6y1FKqdm9/OskuVUCQ3Tr10KopQ=; b=qhhtV69WKH2JluNtYemjQLj4d4hfClGTE//m6T+O7FE46Wo3W/vrmENSH/WtXfrT48 8a5S07rhQ59AKc8+BgfzUy5Bj2swQlHUnRk3ucu69XFSSsZ+AomWUEn1sB396DApLgAe 0adprdBEbSVUxQQfd4dT6RDzrQc6KTQN6bsgOYjw9CUc9dhFHYC2qWnzn+HRn1UPA9aH kjf90LWp0yxILw0IiPJOuiriOeSoOMmi9XkZKcrRXV97WwWsR1pNdEew2aR6B2qrQLUm 3F+WX1+FcFiI+9BtyS27Rf8odZ0hD84gSWCl+MLkesx9GDJcjrjG+T0HGly7ZGRIuVWx 6KSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686757423; x=1689349423; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=9vic1ayWBGr79y9f6y1FKqdm9/OskuVUCQ3Tr10KopQ=; b=Z+rOKqD7Wb0uT8nqbz8I5s/iIaFrmZoHBeVPUfq/yLHKeTg1kgIhKYGuOR2Tuq+jTW d6RAWPamXGQsqESxDclPcMCACSF+rGaNqM2Wr6PbiG/BeWOZqm5oYiQnnxaY4X1vj1NY kWZ/o7SVaERvx66wS+cDe1XY0NlFMwxfZPW7EB4UXNooJG5BejlxDIkz+wI8aWJ8Bdv+ 44FhPxD4aYNxkRuDHQpGVdmKNx0VjpEF0sGAV8S6yKC7eqfWw8sDYbudD5/ggCoFLS6G j/gpEhpfZbiC3yU1i4FEe2s+w/IcMqIdDhx4s7Uba3fwkKQcTip3d68cPb/diQ+AXXHA UdIg== X-Gm-Message-State: AC+VfDzqYICCok6JpU01W90FxcXwmnbRMT43O0QtUNQExFStX1i+UkvQ MQDzHNozZRe9ELpu2tIQ/H5nzlf9z8nt X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:c6e6:49bf:5c44:5965]) (user=mshavit job=sendgmr) by 2002:a25:e6c4:0:b0:bc7:f6af:8cfe with SMTP id d187-20020a25e6c4000000b00bc7f6af8cfemr880044ybh.4.1686757422932; Wed, 14 Jun 2023 08:43:42 -0700 (PDT) Date: Wed, 14 Jun 2023 23:41:54 +0800 In-Reply-To: <20230614154304.2860121-1-mshavit@google.com> Mime-Version: 1.0 References: <20230614154304.2860121-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230614154304.2860121-3-mshavit@google.com> Subject: [PATCH v3 02/13] iommu/arm-smmu-v3: Add smmu_s1_cfg to smmu_master From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768696401528505414?= X-GMAIL-MSGID: =?utf-8?q?1768696401528505414?= Except for Nested domains, arm_smmu_master will own the STEs that are inserted into the arm_smmu_device's STE table. Signed-off-by: Michael Shavit --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 28 +++++++++++++-------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 1 + 2 files changed, 18 insertions(+), 11 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index beff04b897718..023769f5ca79a 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1126,15 +1126,16 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, return 0; } -static int arm_smmu_alloc_cd_tables(struct arm_smmu_domain *smmu_domain) +static int arm_smmu_init_s1_cfg(struct arm_smmu_master *master, + struct arm_smmu_s1_cfg *cfg) { int ret; size_t l1size; size_t max_contexts; - struct arm_smmu_device *smmu = smmu_domain->smmu; - struct arm_smmu_s1_cfg *cfg = &smmu_domain->s1_cfg; + struct arm_smmu_device *smmu = master->smmu; struct arm_smmu_ctx_desc_cfg *cdcfg = &cfg->cdcfg; + cfg->s1cdmax = master->ssid_bits; max_contexts = 1 << cfg->s1cdmax; if (!(smmu->features & ARM_SMMU_FEAT_2_LVL_CDTAB) || @@ -1175,12 +1176,11 @@ static int arm_smmu_alloc_cd_tables(struct arm_smmu_domain *smmu_domain) return ret; } -static void arm_smmu_free_cd_tables(struct arm_smmu_domain *smmu_domain) +static void arm_smmu_free_cd_tables(struct arm_smmu_device *smmu, + struct arm_smmu_ctx_desc_cfg *cdcfg) { int i; size_t size, l1size; - struct arm_smmu_device *smmu = smmu_domain->smmu; - struct arm_smmu_ctx_desc_cfg *cdcfg = &smmu_domain->s1_cfg.cdcfg; if (cdcfg->l1_desc) { size = CTXDESC_L2_ENTRIES * (CTXDESC_CD_DWORDS << 3); @@ -2076,7 +2076,7 @@ static void arm_smmu_domain_free(struct iommu_domain *domain) /* Prevent SVA from touching the CD while we're freeing it */ mutex_lock(&arm_smmu_asid_lock); if (cfg->cdcfg.cdtab) - arm_smmu_free_cd_tables(smmu_domain); + arm_smmu_free_cd_tables(smmu, &cfg->cdcfg); arm_smmu_free_asid(&smmu_domain->cd); mutex_unlock(&arm_smmu_asid_lock); } else { @@ -2108,11 +2108,9 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, if (ret) goto out_unlock; - cfg->s1cdmax = master->ssid_bits; - smmu_domain->stall_enabled = master->stall_enabled; - ret = arm_smmu_alloc_cd_tables(smmu_domain); + ret = arm_smmu_init_s1_cfg(master, cfg); if (ret) goto out_free_asid; @@ -2140,7 +2138,7 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, return 0; out_free_cd_tables: - arm_smmu_free_cd_tables(smmu_domain); + arm_smmu_free_cd_tables(smmu, &cfg->cdcfg); out_free_asid: arm_smmu_free_asid(cd); out_unlock: @@ -2704,6 +2702,13 @@ static struct iommu_device *arm_smmu_probe_device(struct device *dev) smmu->features & ARM_SMMU_FEAT_STALL_FORCE) master->stall_enabled = true; + ret = arm_smmu_init_s1_cfg(master, &master->owned_s1_cfg); + if (ret) { + arm_smmu_disable_pasid(master); + arm_smmu_remove_master(master); + goto err_free_master; + } + return &smmu->iommu; err_free_master: @@ -2719,6 +2724,7 @@ static void arm_smmu_release_device(struct device *dev) if (WARN_ON(arm_smmu_master_sva_enabled(master))) iopf_queue_remove_device(master->smmu->evtq.iopf, dev); arm_smmu_detach_dev(master); + arm_smmu_free_cd_tables(master->smmu, &master->owned_s1_cfg.cdcfg); arm_smmu_disable_pasid(master); arm_smmu_remove_master(master); kfree(master); diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 68d519f21dbd8..053cc14c23969 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -688,6 +688,7 @@ struct arm_smmu_master { struct arm_smmu_domain *domain; struct list_head domain_head; struct arm_smmu_stream *streams; + struct arm_smmu_s1_cfg owned_s1_cfg; unsigned int num_streams; bool ats_enabled; bool stall_enabled; From patchwork Wed Jun 14 15:41:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 108028 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp66607vqr; Wed, 14 Jun 2023 09:18:25 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6viyBcZS91UOqPZZQeE/sDXPLIEYf0sARCXep5IURyncZ3qifJ7JMzSoAeh3zkpIcftwr2 X-Received: by 2002:a17:90a:4b85:b0:25c:2558:e208 with SMTP id i5-20020a17090a4b8500b0025c2558e208mr1701542pjh.38.1686759505402; Wed, 14 Jun 2023 09:18:25 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686759505; cv=none; d=google.com; s=arc-20160816; b=vZ4JbHLp40gK4MTlOKrydEFB6K3IkRgM8/8HhhQQeO9Om2KcymUplO08lxuBwHt5Vu Dqd/78Hjebjd8b5ogaMD/WB9qy5Hpeije3QBC3tR9NMFf+S6M4vg99Vas9UtHxEBQba2 2yEL8GquGjY88MnhsfiKEuKNGCAiAhWcTNtl7UgJtzmhBdgub9yrcUIv9+dIcn3XnXb2 Sa5Y2JtdafLO6LeqNMbhAABT+tbWAMAqPj0qkV2GEouzyP5tQ1OO9P5fHJeE3tT7A4z7 F76ygK7SnxmkqRVb5tV3AZFVU56q5kuF8H+FZKlfobXbameYMuvpdAt1lKH/Qp8/kYdO D6rQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=jzXQy+3gWCRrOq/bPJGSod35GlRHXYXuXCqPkpYte2c=; b=H80WtPVjyE0pz+BSahW1Em0vOa0aR0LkoCWHTMuqAattG2f1IeC9IcH9+RtKMU3Cmj hUnPJ3LdJue00Xb3bZedtDziO2x2OQALyA8jrKmF5q177xZ2OoFxjLE4oAqoxo4dfli5 kx1YAL8AKTCPUup6BMYVfdl21T+A1fhBTBZq4xNmiDp4rsJhflg4GzrcjV9uH1GD1mch NBYTVngtdUpczJ4SHkM40bzIIg5ffhtRqMKA5R+uSUQ12XZQaVXBnN4a/u9br1Mbn+W2 HtWvwMSxS0OvckRv3eTUmbye1Ne8oohwvriwvdtaJ+lUxXebs+UN1qV6HNoD7C/CpsPY 8NMg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=JXTplwSs; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id k189-20020a636fc6000000b00543e355adb3si11011564pgc.275.2023.06.14.09.18.10; Wed, 14 Jun 2023 09:18:25 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=JXTplwSs; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244582AbjFNPn7 (ORCPT + 99 others); Wed, 14 Jun 2023 11:43:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42548 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244489AbjFNPnx (ORCPT ); Wed, 14 Jun 2023 11:43:53 -0400 Received: from mail-yw1-x114a.google.com (mail-yw1-x114a.google.com [IPv6:2607:f8b0:4864:20::114a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DBC771FFF for ; Wed, 14 Jun 2023 08:43:51 -0700 (PDT) Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-561ceb5b584so12002457b3.3 for ; Wed, 14 Jun 2023 08:43:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686757431; x=1689349431; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=jzXQy+3gWCRrOq/bPJGSod35GlRHXYXuXCqPkpYte2c=; b=JXTplwSsE5VOimvzwSxoOBAXrvM0DkSnTFs5atKOgzNUFE/C6yugZQCGi8Ug+Q8za4 5m/tPpgqerRmaFNQv3MeyUdn3zLMeQPVwdgidcdiQSni224I4J4BLjawCnAL5dEuCm4B GttF1B/Zse1OdabCBCXmYiTJKA+xHsGxGmBk9ODJn0kHpM9qNz7aVzxYApxkUb654QRM ZbG1mWIJsW+aW9HEn9KYHX9jCB1Y3C20S8I1DWRQBR0nMQHa2kwM/UwkgmmL2shc3mGR 6bxqrYSA4iicQ8gtGX2d5ts0FDp2lDELfAQh+Q2Venkpo5DaS6bweeY9O1W8mf1klzN7 iDyQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686757431; x=1689349431; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=jzXQy+3gWCRrOq/bPJGSod35GlRHXYXuXCqPkpYte2c=; b=aPIuehZzn5S/ltQgvBuA1pDwHz1MvAEzPP7mW1hETqoB3Z/qlJcyi+gPJRDECnwtq3 aDmjB0xZ4j4fGRMREVKLVL+1XRVlHUNgOkos+wFlH8QTCVqTr4UPsfmGyBEM5Cx9SXoD PczwLLrO4ZRc+Oi6S5FxFdbh/F7lTAwKS7F0aRDJa6EPZa9KJyN584017PJB9kZe9zsj viJRmnqZlBjH+fiahiUeFCCt3Lv8k/ZTz07BxqZaSCWUmWvFuuRCMh7mISSl2ACl4j05 Dju6oWxtKq1iTn0QPxMycTsFMJeq+qARF/ikpqJyFMUk0uNhZf8fPVHrSU/lzaeqLnHv pHiQ== X-Gm-Message-State: AC+VfDxwsbXQyBcdxMwnwc+x2qua7PhcZs/Lp7VkMmY7J2zqZQVJnNNM 4kEF9KcwaHhi7ud2wpveowvjGnWuFmrq X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:c6e6:49bf:5c44:5965]) (user=mshavit job=sendgmr) by 2002:a25:abeb:0:b0:bc8:c749:eede with SMTP id v98-20020a25abeb000000b00bc8c749eedemr1129224ybi.7.1686757431181; Wed, 14 Jun 2023 08:43:51 -0700 (PDT) Date: Wed, 14 Jun 2023 23:41:55 +0800 In-Reply-To: <20230614154304.2860121-1-mshavit@google.com> Mime-Version: 1.0 References: <20230614154304.2860121-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230614154304.2860121-4-mshavit@google.com> Subject: [PATCH v3 03/13] iommu/arm-smmu-v3: Refactor write_strtab_ent From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768695534873612330?= X-GMAIL-MSGID: =?utf-8?q?1768695534873612330?= Explicity keep track of the s1_cfg and s2_cfg that are attached to a master in arm_smmu_master, regardless of whether they are owned by arm_smmu_master, arm_smmu_domain or userspace. Signed-off-by: Michael Shavit --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 37 +++++++++------------ drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 2 ++ 2 files changed, 17 insertions(+), 22 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 023769f5ca79a..d79c6ef5d6ed4 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1269,10 +1269,9 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, */ u64 val = le64_to_cpu(dst[0]); bool ste_live = false; - struct arm_smmu_device *smmu = NULL; + struct arm_smmu_device *smmu = master->smmu; struct arm_smmu_s1_cfg *s1_cfg = NULL; struct arm_smmu_s2_cfg *s2_cfg = NULL; - struct arm_smmu_domain *smmu_domain = NULL; struct arm_smmu_cmdq_ent prefetch_cmd = { .opcode = CMDQ_OP_PREFETCH_CFG, .prefetch = { @@ -1280,24 +1279,10 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, }, }; - if (master) { - smmu_domain = master->domain; - smmu = master->smmu; - } - - if (smmu_domain) { - switch (smmu_domain->stage) { - case ARM_SMMU_DOMAIN_S1: - s1_cfg = &smmu_domain->s1_cfg; - break; - case ARM_SMMU_DOMAIN_S2: - case ARM_SMMU_DOMAIN_NESTED: - s2_cfg = &smmu_domain->s2_cfg; - break; - default: - break; - } - } + if (master->s1_cfg) + s1_cfg = master->s1_cfg; + else if (master->s2_cfg) + s2_cfg = master->s2_cfg; if (val & STRTAB_STE_0_V) { switch (FIELD_GET(STRTAB_STE_0_CFG, val)) { @@ -1319,8 +1304,8 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, val = STRTAB_STE_0_V; /* Bypass/fault */ - if (!smmu_domain || !(s1_cfg || s2_cfg)) { - if (!smmu_domain && disable_bypass) + if (!(s1_cfg || s2_cfg)) { + if (disable_bypass) val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_ABORT); else val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_BYPASS); @@ -2401,6 +2386,8 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) master->domain = NULL; master->ats_enabled = false; + master->s1_cfg = NULL; + master->s2_cfg = NULL; arm_smmu_install_ste_for_dev(master); } @@ -2454,6 +2441,12 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) } master->domain = smmu_domain; + if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { + master->s1_cfg = &smmu_domain->s1_cfg; + } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S2 || + smmu_domain->stage == ARM_SMMU_DOMAIN_NESTED) { + master->s2_cfg = &smmu_domain->s2_cfg; + } /* * The SMMU does not support enabling ATS with bypass. When the STE is diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 053cc14c23969..3c614fbe2b8b9 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -689,6 +689,8 @@ struct arm_smmu_master { struct list_head domain_head; struct arm_smmu_stream *streams; struct arm_smmu_s1_cfg owned_s1_cfg; + struct arm_smmu_s1_cfg *s1_cfg; + struct arm_smmu_s2_cfg *s2_cfg; unsigned int num_streams; bool ats_enabled; bool stall_enabled; From patchwork Wed Jun 14 15:41:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 108038 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp77447vqr; Wed, 14 Jun 2023 09:33:41 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7tviKV7hLrY+XC9qoPaIO1jzSVZ5TonfnUEJSS1ytmfEcO6id4OCZ6KE9C0pDWtH+moUdw X-Received: by 2002:a17:907:1b1a:b0:978:992e:efd1 with SMTP id mp26-20020a1709071b1a00b00978992eefd1mr14237047ejc.35.1686760421068; Wed, 14 Jun 2023 09:33:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686760421; cv=none; d=google.com; s=arc-20160816; b=OjFTjKO4z2Ok1DvXDiYHHRSFOtQ8chivD5i9tSsR5Tm2u9NQv2p1F5r34IV55fevoV uYWj4p5AaLBGIjFmZeapfg9epUNFIZsW4cN1n9kTyHm0ImzcYocttNuK6fMil0Vgv62x 4Zyy0ytaSaYw3aeMts+nIDnKV7+C1AFHT2DhJgrSkaBgocrmMQqY9mIX5EsWWPfpy9H5 kwSzqcSlwWr2qEtUoAKjytDKLGIVlpFN5OmnrxzrCYxCJ2we302lLQb1z5uZfVp04TgQ Qe5Q2KCMI4p+nmPWVIx4Bpwr6P75JD7Va/s94lp3JGfkzu28imPp6e7iD9IYWIQMvv2L HDtw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=T3tJEpWHCJm3S04xQ0T/8NFLZuI/7sadQhRIHV1s9Zo=; b=cJHNGBQirpl7EAhOXBZ9k6v3ri27F3YM0+dYIzqLpUOLapY0ePxSlMaeQDRAx8SfIH 02YY5AhgTMzTNmTRRgghZQKyW0B9LcKrdcgcOGNSGhPslDetgtoVRu180L4H0A/crngS lZR8F5I+19DGcKjx+jLx6h6q1O173tAZ3gTNJqWKkBLBitY0qAT4PPEHQRPHJeAvclyt X7ZCjxpFDo48xlDxbUDQlnK9wbzrCpgZAXKUtinNgVsB5tR3xQzBSWNMobmf5ywohjvH JfWuUhQrDl958ZTjjpu4V0nbJVTizkyr/I74W5fi/Jrd4BBEQVAVoHSZNqAkGlv7YxBq dH3g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=yTgaJKyh; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id x14-20020a1709060a4e00b00982278e2ce0si3606008ejf.210.2023.06.14.09.33.11; Wed, 14 Jun 2023 09:33:41 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=yTgaJKyh; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244647AbjFNPoH (ORCPT + 99 others); Wed, 14 Jun 2023 11:44:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42656 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244568AbjFNPoB (ORCPT ); Wed, 14 Jun 2023 11:44:01 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AAB552107 for ; Wed, 14 Jun 2023 08:43:59 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-bd5059e5347so1729629276.0 for ; Wed, 14 Jun 2023 08:43:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686757439; x=1689349439; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=T3tJEpWHCJm3S04xQ0T/8NFLZuI/7sadQhRIHV1s9Zo=; b=yTgaJKyhLJN0Sp5lo50YLdxEaUm3GgA7aXol24wNrGXS9BmaZCSXsBzcwf4Qnp+cxw MlQpJOuDL9kZUj/a5rrSo8t1KtW6798cWJvnCfnrfPiJbxg2BRihO+S5JisosY1Tqp58 yIGYg9aw8YKoSYorcITNOl8mk05e/qbpqsKk+hFZOkQ5ufs0Q6KiWNeexHmceSWU2WQK Dqf5E9lb4jGnDahmR9g4vcKZuxX76Jk2uA+xNW4FmWQJnFHkM1e1iREGtDSxGiH1DxEn 6c+PtrzgFhbj6sFtPYCjfglMkJQSVX6bEJdWxnPbAvYSq+LJSgSsOfzjwqdAqEdYQwTX uONQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686757439; x=1689349439; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=T3tJEpWHCJm3S04xQ0T/8NFLZuI/7sadQhRIHV1s9Zo=; b=TJuaty6HiFfgC8ewmnRgs9d4yaxX6d+s+4YaZ/6VR4AuWYB3L8wqV7mvCqV0mb/KEK erOI11O+xyYxL820IBWXNxW/ktwVLA7f3YQXvoxBOpO4QKKFokSwYUgrBR/iMF6BrsUn KcQyfB4fX8AeDVl0vU+o3gFlXtHjGzKEkr1xGeQJBokasG1DeKJrp5eqCelQrJYfpZIY c5Xe8lTwoj5Y/PaTeBWmwS7iQ4TCSw+u34LiYI3hE2crZA51WoQqyGOtrEON/1/6RG5X S95fWHulZ1a4auEYnwF9+RVbLnFx0/eR/wrNjYtZKx8Xm0QtNY3iDR9DavDKDqLgVq76 a16g== X-Gm-Message-State: AC+VfDxZoQIfVuBDVoNwJuS4CyHSnL2wav0VBGeC/D2RiIzF43RWha30 QK8wQEtccL8vlymmEBZQ4K7xjUj3VI0n X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:c6e6:49bf:5c44:5965]) (user=mshavit job=sendgmr) by 2002:a05:6902:1185:b0:ba8:381b:f764 with SMTP id m5-20020a056902118500b00ba8381bf764mr887496ybu.3.1686757438966; Wed, 14 Jun 2023 08:43:58 -0700 (PDT) Date: Wed, 14 Jun 2023 23:41:56 +0800 In-Reply-To: <20230614154304.2860121-1-mshavit@google.com> Mime-Version: 1.0 References: <20230614154304.2860121-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230614154304.2860121-5-mshavit@google.com> Subject: [PATCH v3 04/13] iommu/arm-smmu-v3: Refactor write_ctx_desc From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED, USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768696495429227912?= X-GMAIL-MSGID: =?utf-8?q?1768696495429227912?= Update arm_smmu_write_ctx_desc and downstream functions to be agnostic of the CD table's ownership. Note that in practice, it will only be called in cases where the table is owned by arm_smmu_master. Whether arm_smmu_write_ctx_desc will trigger a sync is also made part of the API. Note that this change isn't a nop refactor since SVA will call arm_smmu_write_ctx_desc in a loop for every master the domain is attached to despite the fact that they all share the same CD table. Note that the next commit ceases this sharing of the s1_cfg which makes that loop necessary. Signed-off-by: Michael Shavit --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 36 ++++++++- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 80 +++++++++++-------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 6 +- 3 files changed, 81 insertions(+), 41 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 968559d625c40..48fa8eb271a45 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -45,10 +45,12 @@ static struct arm_smmu_ctx_desc * arm_smmu_share_asid(struct mm_struct *mm, u16 asid) { int ret; + unsigned long flags; u32 new_asid; struct arm_smmu_ctx_desc *cd; struct arm_smmu_device *smmu; struct arm_smmu_domain *smmu_domain; + struct arm_smmu_master *master; cd = xa_load(&arm_smmu_asid_xa, asid); if (!cd) @@ -80,7 +82,11 @@ arm_smmu_share_asid(struct mm_struct *mm, u16 asid) * be some overlap between use of both ASIDs, until we invalidate the * TLB. */ - arm_smmu_write_ctx_desc(smmu_domain, 0, cd); + spin_lock_irqsave(&smmu_domain->devices_lock, flags); + list_for_each_entry(master, &smmu_domain->devices, domain_head) { + arm_smmu_write_ctx_desc(smmu, master->s1_cfg, master, 0, cd); + } + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); /* Invalidate TLB entries previously associated with that context */ arm_smmu_tlb_inv_asid(smmu, asid); @@ -211,6 +217,8 @@ static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) { struct arm_smmu_mmu_notifier *smmu_mn = mn_to_smmu(mn); struct arm_smmu_domain *smmu_domain = smmu_mn->domain; + struct arm_smmu_master *master; + unsigned long flags; mutex_lock(&sva_lock); if (smmu_mn->cleared) { @@ -222,7 +230,12 @@ static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) * DMA may still be running. Keep the cd valid to avoid C_BAD_CD events, * but disable translation. */ - arm_smmu_write_ctx_desc(smmu_domain, mm->pasid, &quiet_cd); + spin_lock_irqsave(&smmu_domain->devices_lock, flags); + list_for_each_entry(master, &smmu_domain->devices, domain_head) { + arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, + mm->pasid, &quiet_cd); + } + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); arm_smmu_tlb_inv_asid(smmu_domain->smmu, smmu_mn->cd->asid); arm_smmu_atc_inv_domain(smmu_domain, mm->pasid, 0, 0); @@ -248,8 +261,10 @@ arm_smmu_mmu_notifier_get(struct arm_smmu_domain *smmu_domain, struct mm_struct *mm) { int ret; + unsigned long flags; struct arm_smmu_ctx_desc *cd; struct arm_smmu_mmu_notifier *smmu_mn; + struct arm_smmu_master *master; list_for_each_entry(smmu_mn, &smmu_domain->mmu_notifiers, list) { if (smmu_mn->mn.mm == mm) { @@ -279,7 +294,12 @@ arm_smmu_mmu_notifier_get(struct arm_smmu_domain *smmu_domain, goto err_free_cd; } - ret = arm_smmu_write_ctx_desc(smmu_domain, mm->pasid, cd); + spin_lock_irqsave(&smmu_domain->devices_lock, flags); + list_for_each_entry(master, &smmu_domain->devices, domain_head) { + ret = arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, + master, mm->pasid, cd); + } + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); if (ret) goto err_put_notifier; @@ -296,15 +316,23 @@ arm_smmu_mmu_notifier_get(struct arm_smmu_domain *smmu_domain, static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) { + unsigned long flags; struct mm_struct *mm = smmu_mn->mn.mm; struct arm_smmu_ctx_desc *cd = smmu_mn->cd; + struct arm_smmu_master *master; struct arm_smmu_domain *smmu_domain = smmu_mn->domain; if (!refcount_dec_and_test(&smmu_mn->refs)) return; list_del(&smmu_mn->list); - arm_smmu_write_ctx_desc(smmu_domain, mm->pasid, NULL); + + spin_lock_irqsave(&smmu_domain->devices_lock, flags); + list_for_each_entry(master, &smmu_domain->devices, domain_head) { + arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, + mm->pasid, NULL); + } + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); /* * If we went through clear(), we've already invalidated, and no diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index d79c6ef5d6ed4..b6f7cf60f8f3d 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -965,14 +965,13 @@ void arm_smmu_tlb_inv_asid(struct arm_smmu_device *smmu, u16 asid) arm_smmu_cmdq_issue_cmd_with_sync(smmu, &cmd); } -static void arm_smmu_sync_cd(struct arm_smmu_domain *smmu_domain, +/* master may be null */ +static void arm_smmu_sync_cd(struct arm_smmu_master *master, int ssid, bool leaf) { size_t i; - unsigned long flags; - struct arm_smmu_master *master; struct arm_smmu_cmdq_batch cmds; - struct arm_smmu_device *smmu = smmu_domain->smmu; + struct arm_smmu_device *smmu; struct arm_smmu_cmdq_ent cmd = { .opcode = CMDQ_OP_CFGI_CD, .cfgi = { @@ -981,16 +980,15 @@ static void arm_smmu_sync_cd(struct arm_smmu_domain *smmu_domain, }, }; - cmds.num = 0; + if (!master) + return; - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { - for (i = 0; i < master->num_streams; i++) { - cmd.cfgi.sid = master->streams[i].id; - arm_smmu_cmdq_batch_add(smmu, &cmds, &cmd); - } + smmu = master->smmu; + cmds.num = 0; + for (i = 0; i < master->num_streams; i++) { + cmd.cfgi.sid = master->streams[i].id; + arm_smmu_cmdq_batch_add(smmu, &cmds, &cmd); } - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); arm_smmu_cmdq_batch_submit(smmu, &cmds); } @@ -1020,16 +1018,18 @@ static void arm_smmu_write_cd_l1_desc(__le64 *dst, WRITE_ONCE(*dst, cpu_to_le64(val)); } -static __le64 *arm_smmu_get_cd_ptr(struct arm_smmu_domain *smmu_domain, +/* master may be null */ +static __le64 *arm_smmu_get_cd_ptr(struct arm_smmu_device *smmu, + struct arm_smmu_s1_cfg *s1_cfg, + struct arm_smmu_master *master, u32 ssid) { __le64 *l1ptr; unsigned int idx; struct arm_smmu_l1_ctx_desc *l1_desc; - struct arm_smmu_device *smmu = smmu_domain->smmu; - struct arm_smmu_ctx_desc_cfg *cdcfg = &smmu_domain->s1_cfg.cdcfg; + struct arm_smmu_ctx_desc_cfg *cdcfg = &s1_cfg->cdcfg; - if (smmu_domain->s1_cfg.s1fmt == STRTAB_STE_0_S1FMT_LINEAR) + if (s1_cfg->s1fmt == STRTAB_STE_0_S1FMT_LINEAR) return cdcfg->cdtab + ssid * CTXDESC_CD_DWORDS; idx = ssid >> CTXDESC_SPLIT; @@ -1041,13 +1041,21 @@ static __le64 *arm_smmu_get_cd_ptr(struct arm_smmu_domain *smmu_domain, l1ptr = cdcfg->cdtab + idx * CTXDESC_L1_DESC_DWORDS; arm_smmu_write_cd_l1_desc(l1ptr, l1_desc); /* An invalid L1CD can be cached */ - arm_smmu_sync_cd(smmu_domain, ssid, false); + arm_smmu_sync_cd(master, ssid, false); } idx = ssid & (CTXDESC_L2_ENTRIES - 1); return l1_desc->l2ptr + idx * CTXDESC_CD_DWORDS; } -int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, +/* + * master must be provided if a CD sync is required but may be null otherwise + * (such as when the CD table isn't inserted into the STE yet, or is about to + * be detached. + */ +int arm_smmu_write_ctx_desc(struct arm_smmu_device *smmu, + struct arm_smmu_s1_cfg *s1_cfg, + struct arm_smmu_master *master, + int ssid, struct arm_smmu_ctx_desc *cd) { /* @@ -1065,10 +1073,10 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, bool cd_live; __le64 *cdptr; - if (WARN_ON(ssid >= (1 << smmu_domain->s1_cfg.s1cdmax))) + if (WARN_ON(ssid >= (1 << s1_cfg->s1cdmax))) return -E2BIG; - cdptr = arm_smmu_get_cd_ptr(smmu_domain, ssid); + cdptr = arm_smmu_get_cd_ptr(smmu, s1_cfg, master, ssid); if (!cdptr) return -ENOMEM; @@ -1092,11 +1100,11 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, cdptr[3] = cpu_to_le64(cd->mair); /* - * STE is live, and the SMMU might read dwords of this CD in any - * order. Ensure that it observes valid values before reading - * V=1. + * STE may be live, and the SMMU might read dwords of this CD + * in any order. Ensure that it observes valid values before + * reading V=1. */ - arm_smmu_sync_cd(smmu_domain, ssid, true); + arm_smmu_sync_cd(master, ssid, true); val = cd->tcr | #ifdef __BIG_ENDIAN @@ -1108,7 +1116,7 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, FIELD_PREP(CTXDESC_CD_0_ASID, cd->asid) | CTXDESC_CD_0_V; - if (smmu_domain->stall_enabled) + if (s1_cfg->stall_enabled) val |= CTXDESC_CD_0_S; } @@ -1122,7 +1130,7 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, * without first making the structure invalid. */ WRITE_ONCE(cdptr[0], cpu_to_le64(val)); - arm_smmu_sync_cd(smmu_domain, ssid, true); + arm_smmu_sync_cd(master, ssid, true); return 0; } @@ -1136,6 +1144,7 @@ static int arm_smmu_init_s1_cfg(struct arm_smmu_master *master, struct arm_smmu_ctx_desc_cfg *cdcfg = &cfg->cdcfg; cfg->s1cdmax = master->ssid_bits; + cfg->stall_enabled = master->stall_enabled; max_contexts = 1 << cfg->s1cdmax; if (!(smmu->features & ARM_SMMU_FEAT_2_LVL_CDTAB) || @@ -2093,8 +2102,6 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, if (ret) goto out_unlock; - smmu_domain->stall_enabled = master->stall_enabled; - ret = arm_smmu_init_s1_cfg(master, cfg); if (ret) goto out_free_asid; @@ -2110,12 +2117,9 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, CTXDESC_CD_0_TCR_EPD1 | CTXDESC_CD_0_AA64; cd->mair = pgtbl_cfg->arm_lpae_s1_cfg.mair; - /* - * Note that this will end up calling arm_smmu_sync_cd() before - * the master has been added to the devices list for this domain. - * This isn't an issue because the STE hasn't been installed yet. - */ - ret = arm_smmu_write_ctx_desc(smmu_domain, 0, cd); + ret = arm_smmu_write_ctx_desc(smmu, cfg, + NULL /*Not attached to a master yet */, + 0, cd); if (ret) goto out_free_cd_tables; @@ -2386,6 +2390,11 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) master->domain = NULL; master->ats_enabled = false; + if (master->s1_cfg) + arm_smmu_write_ctx_desc( + master->smmu, master->s1_cfg, + NULL /* Skip sync since we detach the CD table next*/, + 0, NULL); master->s1_cfg = NULL; master->s2_cfg = NULL; arm_smmu_install_ste_for_dev(master); @@ -2435,7 +2444,8 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) ret = -EINVAL; goto out_unlock; } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1 && - smmu_domain->stall_enabled != master->stall_enabled) { + smmu_domain->s1_cfg.stall_enabled != + master->stall_enabled) { ret = -EINVAL; goto out_unlock; } diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 3c614fbe2b8b9..00a493442d6f9 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -595,6 +595,7 @@ struct arm_smmu_s1_cfg { struct arm_smmu_ctx_desc_cfg cdcfg; u8 s1fmt; u8 s1cdmax; + bool stall_enabled; }; struct arm_smmu_s2_cfg { @@ -713,7 +714,6 @@ struct arm_smmu_domain { struct mutex init_mutex; /* Protects smmu pointer */ struct io_pgtable_ops *pgtbl_ops; - bool stall_enabled; atomic_t nr_ats_masters; enum arm_smmu_domain_stage stage; @@ -742,7 +742,9 @@ extern struct xarray arm_smmu_asid_xa; extern struct mutex arm_smmu_asid_lock; extern struct arm_smmu_ctx_desc quiet_cd; -int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, +int arm_smmu_write_ctx_desc(struct arm_smmu_device *smmu, + struct arm_smmu_s1_cfg *s1_cfg, + struct arm_smmu_master *smmu_master, int ssid, struct arm_smmu_ctx_desc *cd); void arm_smmu_tlb_inv_asid(struct arm_smmu_device *smmu, u16 asid); void arm_smmu_tlb_inv_range_asid(unsigned long iova, size_t size, int asid, From patchwork Wed Jun 14 15:41:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 108025 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp65734vqr; Wed, 14 Jun 2023 09:17:19 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4P4r+0Rfk4Yboc1jt+ZloZRie5BVBVz1Hg+gPAO0/PVWlItYWHeSIKJXpueyRmGwyuVJeb X-Received: by 2002:a92:280f:0:b0:33b:94d:b61b with SMTP id l15-20020a92280f000000b0033b094db61bmr11984982ilf.22.1686759438989; Wed, 14 Jun 2023 09:17:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686759438; cv=none; d=google.com; s=arc-20160816; b=DYCoMiyvnGK+oGtasFk+DoHcf99KSkQ9B8EfXiPoTvAOQ5gJzArNkuVeZAJyYhPfJJ zACsOx819LpEdOpMPsR4vDczmkfwLsY8/l7cYWSJlWLZKK4FoBPo0IOq9m/D+RZNw6Mr VhUH5WjP3K86exKfUfxOCuld0xv6SiWyqABpCnTF5S4weQ4JNjEG2D79EDVYZneCrryG JUlny9mddGhk1U8OVrrs86+wm72dyQ395JaOBHnFrU7fVvLQ4uo94AqGfMpeyGxXILBQ 9QxQpaEzxmvFoEDBsogE0l25K4P+MYgTgclIcn1I8YbmhFllOsGlcFe0e/r0uIyCKh91 dB+A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=6vzESexuWCV4LLT4ZlWlOvb8u8164/HANB5QcSOGct0=; b=rdyPrAsgrFSu6p4Uu2rYLwVk91VR4mwLes3WtmMMX/SaKY3AwDw8gh+VCwM221WJ5Q p3B5G2T8eBtRLjRUs+Zmmfj0gl/YS5UYZ7e9h/mTxDoQWHWqzmQqOIlEiIuanP8Ljl3c D7Vso+Xf1Ho/EuBqOaQubgi79xgm++XAh9R/QpUO4jmwuOfVS+FZVQTh7BF8aW4oqRQn 6nOCZvk6OsUnpomp8TiEB+yNidBmOEVgUP4gRnrqCAjc5rtbehtC4u+lAjmqjhUYSWoZ J6V48P7ekMrlyDc6gmos9pCwTamoNrWGHxI1HIX9ZiBpXG/qW/DMWtYW7doO6tMM0J+j GfTA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=tKbqq+Sa; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id d130-20020a633688000000b0051358b5c88bsi10765032pga.452.2023.06.14.09.17.06; Wed, 14 Jun 2023 09:17:18 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=tKbqq+Sa; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244869AbjFNPoc (ORCPT + 99 others); Wed, 14 Jun 2023 11:44:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42708 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244669AbjFNPoJ (ORCPT ); Wed, 14 Jun 2023 11:44:09 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C15552116 for ; Wed, 14 Jun 2023 08:44:07 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-bc45096dfd0so860926276.1 for ; Wed, 14 Jun 2023 08:44:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686757447; x=1689349447; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=6vzESexuWCV4LLT4ZlWlOvb8u8164/HANB5QcSOGct0=; b=tKbqq+SaTI8WzYXCxXw85rHSkOoIOr6eh4ii5g7z7oVm96FiPbCikxVlsDi1F+HwC2 dn1mWxIi51qsEvtY5sEZwXIhwhpWDJFjaT9RH2MzSRlK3QV+ElSp1EHRBAYKVhsynkLx iPVmHZjb2ZJzh6cYEahzlpwRheLpFuNbypKJcMzn2fsSBC53RC9koPOVxYRRQt57PIiU EK5hKMvKJc774Zr79U33ubKpVacDJqsAdVE/UJ70e3MNg8mtmbtuj4qIoQI/QoMhH6v3 JJV6hSq7MUNSZ7giHoLoLKjz57HKq8Z8nTIDRkVAF3+KsJV9QKIqSJkruMChmA6orr95 GT+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686757447; x=1689349447; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=6vzESexuWCV4LLT4ZlWlOvb8u8164/HANB5QcSOGct0=; b=hSPNsbHSm7nl2AwaTXKiiPQe3zuv2CYy1Rfo1nv4lx7TnK6SlkZIDNKn3cxk2IrVge uxlXvzAgGAZyhFEVkeHtUMozXqGK4p/71VVXP/buJNewcj0hS69LrX6Of/ShCS5OLx+G ArIOlmiYZI8diudWM7P19jE20UCOvXchIQIkQDUNL0Nh2H3QVOJivDowwAKJKCoaXfRP 0cRvMtXeyZ85U5tZdVYJj7OKQ56S1kkGUHGbVRmjxbL9x0h6CuuQ+nr85NlnXWKv7ka/ HJvw4CNDCPjCUzMzsOuVQWotefVhkESTHOIF/r57ClsOoQSodHD95K388QsE2S4mEMHO jtPA== X-Gm-Message-State: AC+VfDzUVRAgzFKRiTcAX3kepYRkSLRX88/aw4y3AffG+yZE6ZFaTVsa yq0kUkY254HZ4B9kSLdDqqj+BsFzRf1U X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:c6e6:49bf:5c44:5965]) (user=mshavit job=sendgmr) by 2002:a25:d001:0:b0:bcf:7f0b:e1d0 with SMTP id h1-20020a25d001000000b00bcf7f0be1d0mr328965ybg.0.1686757447045; Wed, 14 Jun 2023 08:44:07 -0700 (PDT) Date: Wed, 14 Jun 2023 23:41:57 +0800 In-Reply-To: <20230614154304.2860121-1-mshavit@google.com> Mime-Version: 1.0 References: <20230614154304.2860121-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230614154304.2860121-6-mshavit@google.com> Subject: [PATCH v3 05/13] iommu/arm-smmu-v3: Use the master-owned s1_cfg From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768695465443675025?= X-GMAIL-MSGID: =?utf-8?q?1768695465443675025?= Insert CDs for STAGE_1 domains into a CD table owned by the arm_smmu_master. Remove the CD table that was owned by arm_smmu_domain. Signed-off-by: Michael Shavit --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 43 ++++++--------------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 3 -- 2 files changed, 12 insertions(+), 34 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index b6f7cf60f8f3d..08f440fe1da6d 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2065,12 +2065,8 @@ static void arm_smmu_domain_free(struct iommu_domain *domain) /* Free the CD and ASID, if we allocated them */ if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { - struct arm_smmu_s1_cfg *cfg = &smmu_domain->s1_cfg; - /* Prevent SVA from touching the CD while we're freeing it */ mutex_lock(&arm_smmu_asid_lock); - if (cfg->cdcfg.cdtab) - arm_smmu_free_cd_tables(smmu, &cfg->cdcfg); arm_smmu_free_asid(&smmu_domain->cd); mutex_unlock(&arm_smmu_asid_lock); } else { @@ -2082,14 +2078,13 @@ static void arm_smmu_domain_free(struct iommu_domain *domain) kfree(smmu_domain); } -static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, +static int arm_smmu_domain_finalise_cd(struct arm_smmu_domain *smmu_domain, struct arm_smmu_master *master, struct io_pgtable_cfg *pgtbl_cfg) { int ret; u32 asid; struct arm_smmu_device *smmu = smmu_domain->smmu; - struct arm_smmu_s1_cfg *cfg = &smmu_domain->s1_cfg; struct arm_smmu_ctx_desc *cd = &smmu_domain->cd; typeof(&pgtbl_cfg->arm_lpae_s1_cfg.tcr) tcr = &pgtbl_cfg->arm_lpae_s1_cfg.tcr; @@ -2102,10 +2097,6 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, if (ret) goto out_unlock; - ret = arm_smmu_init_s1_cfg(master, cfg); - if (ret) - goto out_free_asid; - cd->asid = (u16)asid; cd->ttbr = pgtbl_cfg->arm_lpae_s1_cfg.ttbr; cd->tcr = FIELD_PREP(CTXDESC_CD_0_TCR_T0SZ, tcr->tsz) | @@ -2117,19 +2108,9 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, CTXDESC_CD_0_TCR_EPD1 | CTXDESC_CD_0_AA64; cd->mair = pgtbl_cfg->arm_lpae_s1_cfg.mair; - ret = arm_smmu_write_ctx_desc(smmu, cfg, - NULL /*Not attached to a master yet */, - 0, cd); - if (ret) - goto out_free_cd_tables; - mutex_unlock(&arm_smmu_asid_lock); return 0; -out_free_cd_tables: - arm_smmu_free_cd_tables(smmu, &cfg->cdcfg); -out_free_asid: - arm_smmu_free_asid(cd); out_unlock: mutex_unlock(&arm_smmu_asid_lock); return ret; @@ -2192,7 +2173,7 @@ static int arm_smmu_domain_finalise(struct iommu_domain *domain, ias = min_t(unsigned long, ias, VA_BITS); oas = smmu->ias; fmt = ARM_64_LPAE_S1; - finalise_stage_fn = arm_smmu_domain_finalise_s1; + finalise_stage_fn = arm_smmu_domain_finalise_cd; break; case ARM_SMMU_DOMAIN_NESTED: case ARM_SMMU_DOMAIN_S2: @@ -2439,20 +2420,20 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) } else if (smmu_domain->smmu != smmu) { ret = -EINVAL; goto out_unlock; - } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1 && - master->ssid_bits != smmu_domain->s1_cfg.s1cdmax) { - ret = -EINVAL; - goto out_unlock; - } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1 && - smmu_domain->s1_cfg.stall_enabled != - master->stall_enabled) { - ret = -EINVAL; - goto out_unlock; } master->domain = smmu_domain; if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { - master->s1_cfg = &smmu_domain->s1_cfg; + master->s1_cfg = &master->owned_s1_cfg; + ret = arm_smmu_write_ctx_desc( + smmu, + master->s1_cfg, NULL /*Not attached to a master yet */, + 0, &smmu_domain->cd); + if (ret) { + master->s1_cfg = NULL; + master->domain = NULL; + goto out_unlock; + } } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S2 || smmu_domain->stage == ARM_SMMU_DOMAIN_NESTED) { master->s2_cfg = &smmu_domain->s2_cfg; diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 00a493442d6f9..dff0fa8345462 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -718,10 +718,7 @@ struct arm_smmu_domain { enum arm_smmu_domain_stage stage; union { - struct { struct arm_smmu_ctx_desc cd; - struct arm_smmu_s1_cfg s1_cfg; - }; struct arm_smmu_s2_cfg s2_cfg; }; From patchwork Wed Jun 14 15:41:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 108027 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp66604vqr; Wed, 14 Jun 2023 09:18:25 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7cDQTZvfhhkpIL+ghHxaQTqQj1tAPsDLMk34MhxAa5x6FK/EbVdT0t3URueKExjWGDR0UA X-Received: by 2002:a05:6a20:938a:b0:101:4e04:cef1 with SMTP id x10-20020a056a20938a00b001014e04cef1mr2973797pzh.27.1686759505203; Wed, 14 Jun 2023 09:18:25 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686759505; cv=none; d=google.com; s=arc-20160816; b=DFa47rqBwTMKtgcqvbJ51QtA5AKV1Aux17JFHhpoRqchJ7n5WWexAVonXsCK5wia4s InFoHwUOc4K1soTSpUPwMOAgO81BeblD2E5j6JEizSOsKxCz1MSGwh+eyyGJ8Fa5KX1u NUFL2Jft4T0yqWTqZlUf/Iq05naz25AlYJmqKk7fUMALjfhyXX+3cFk0PYizPJltgbfB FPBo0sWnNoFcaMMXviruSM/VG+nJYm6I7V7CcXhnNWRF35QtgHgORFOC+AZYP1EFAxbq Cw5NcKR3MWl/aqoUCyyEOVcforkX6C2MHfzHIjsJCSK0Z1E4ykgLHfeDeDAnTOaCZ+EI Yteg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=kKyYB6nGtYqnelo6XDlWvQzY0JjgxL/I8n3xKSQjyKM=; b=qtQbwPeIIrVfaPu9yJyI7ihYrl1hrWP8poZawIJiSGiAqjRsLpgDRaawy7jMPs2H6r aGThUkG1cHUbht3yHx1RfE69h5Cng6i4L1+ykLMlhVZHkE8FdPeupjNKikZ+7RNLz0gT yjPM0K9tL2dwWDNbEKU+W8OoyGsUp+088gkyT2QGqt5fwEOc4ttt/hBp3Mi4CD/xB84R xJyq102RqqNrFZUyX3aod4+YfCpxGpre57U/cNaKTA4/Ohb0cwOTp9rtgFR/t0SRfp7I L6+Uvk0xJeeiB3c2rfcq+//6BchU2sp4wg3AzDbdK4qEeQPf/wU3Sn0A4b5GBgkhsoZ+ fVtw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=ZreyidDp; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id y184-20020a638ac1000000b0053467c7b635si6385838pgd.85.2023.06.14.09.18.11; Wed, 14 Jun 2023 09:18:25 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=ZreyidDp; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244634AbjFNPoj (ORCPT + 99 others); Wed, 14 Jun 2023 11:44:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42706 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244655AbjFNPoT (ORCPT ); Wed, 14 Jun 2023 11:44:19 -0400 Received: from mail-pf1-x449.google.com (mail-pf1-x449.google.com [IPv6:2607:f8b0:4864:20::449]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 75CC2210B for ; Wed, 14 Jun 2023 08:44:15 -0700 (PDT) Received: by mail-pf1-x449.google.com with SMTP id d2e1a72fcca58-650cb5a6b0cso6469492b3a.1 for ; Wed, 14 Jun 2023 08:44:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686757455; x=1689349455; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=kKyYB6nGtYqnelo6XDlWvQzY0JjgxL/I8n3xKSQjyKM=; b=ZreyidDpJbP1PBXH0/670/kPY2SsUh00/efeS6mDpY0HajYSURNuwl2jgv15mHAY+8 BQnjoS82KZhUCP+ZW8JKdnnpOoil/pxglVP0s2csLwMNXQxHU51EaIYCRwcdKqquCh0q 3bWW4wrsmrxasyohwzBecrXf6yKsHWQhn0bySkpCPo4xUAgaHO41cs19iiK6VFDnmGcL 1z3TlJElBou8TAuRPKhYnjDJwUUqq75L0bf3w6VNMdMUoFGls32SxQ9iHK9xTnV32fN7 YIabDW9594lEZ4404D02Bxq7KeKnTkNz75qnKx2pKgIU+oWiPHE0kF1iXqTgciI2k5V0 H35Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686757455; x=1689349455; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=kKyYB6nGtYqnelo6XDlWvQzY0JjgxL/I8n3xKSQjyKM=; b=f0GOWWPGSroVzdT7/EUsbtR8+oIkP5RBmSJPylpmBk2adSgWEM34Dd2kutvk3AiuRA i7+cPUwScgVDGtadQHY+ZpqKwlS5oTah+W7zeD24/GIqT7NXYtjG/hkJAm8COC/+LG38 1de1VyqzFaLYpO40ePEZtoTVSuIN8waSts4p9KWD9wJSe7b/9bIkkBuClE7KsKfLzfNG 80+QnfVUWqlWZsmLFSFnykQN0aaOgId4B3f8+x8BcDesEPGWC8c+xAA9F2MVVDkvhgAF CCcC7DzCXQBLABF5fFp5Vimmmr0QayMIxC8p2jYrYblMsL2Ov5QI56VzUROfX06cdtVr GpPQ== X-Gm-Message-State: AC+VfDzN8vto3tBMoeLgWFyVt7VLI7xMYEBhrVIJ5rfZHNdIFY7hA8Wj Fm9IYL6OAkASQ2ZVz0I8dJ67z8PsG/CV X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:c6e6:49bf:5c44:5965]) (user=mshavit job=sendgmr) by 2002:a05:6a00:2e99:b0:654:424b:17d2 with SMTP id fd25-20020a056a002e9900b00654424b17d2mr664816pfb.2.1686757454862; Wed, 14 Jun 2023 08:44:14 -0700 (PDT) Date: Wed, 14 Jun 2023 23:41:58 +0800 In-Reply-To: <20230614154304.2860121-1-mshavit@google.com> Mime-Version: 1.0 References: <20230614154304.2860121-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230614154304.2860121-7-mshavit@google.com> Subject: [PATCH v3 06/13] iommu/arm-smmu-v3: Simplify arm_smmu_enable_ats From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768695534772409608?= X-GMAIL-MSGID: =?utf-8?q?1768695534772409608?= arm_smmu_enable_ats's call to inv_domain would trigger an invalidation for all masters that a domain is attached to everytime it's attached to another ATS-enabled master. It doesn't seem like those invalidations are necessary, and it's easier to reason about arm_smmu_enable_ats if it only issues invalidation commands for the current master. Signed-off-by: Michael Shavit --- v1->v2: Fix commit message wrapping --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 08f440fe1da6d..dc7a59e87a2b4 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2286,7 +2286,7 @@ static void arm_smmu_enable_ats(struct arm_smmu_master *master) pdev = to_pci_dev(master->dev); atomic_inc(&smmu_domain->nr_ats_masters); - arm_smmu_atc_inv_domain(smmu_domain, 0, 0, 0); + arm_smmu_atc_inv_master(master); if (pci_enable_ats(pdev, stu)) dev_err(master->dev, "Failed to enable ATS (STU %zu)\n", stu); } From patchwork Wed Jun 14 15:41:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 108032 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp69497vqr; Wed, 14 Jun 2023 09:22:11 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ61cboO3s52i8ch4IKQdSwhzu0bHW11COkGIkEdhO6aIj0bTXhZ1nsfbH/bYQEuCCOZJqvF X-Received: by 2002:a05:6808:306:b0:398:4219:3817 with SMTP id i6-20020a056808030600b0039842193817mr9459594oie.40.1686759730714; Wed, 14 Jun 2023 09:22:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686759730; cv=none; d=google.com; s=arc-20160816; b=pOukMx+NX6DkeL5zDDZDepxq/sr/B5WVfajeHIxNnz2jqvAStR5L4ZGPn8X2w3b39i h0hsnIsKoJORC9AwEVmHixtCQBfEe9Ck/JnMZdfTXgJeeIpOiYwBb44bK1LCKPGPliYc U3CkJSu6G+vklg8pebq1E7JUEmdz/4DQ++0gt7vukYdSYA6fj3zPgCcX/9aSC5q3sMjN W+pVPqtE5nBB5X9kKLfhK7c+VIQf2Pt18swR4d2U+33LXs4XsqB3FCi/Xp35zBhOqIoQ HNwtrTcrxRs2djs51Sd5QCOiLpzV+7Edl9ywGWCBcuTXhUThrJ5JqJJkzmGcFfQRIHiI 6D0w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=nXG6hIyHVJ6VKFy81TWuK5UaUuQM8CF9N4uqi7zpqBY=; b=kwVTNLkQ/nEIHJYAEbCR22b7Xumzw77UmOOiZv5iBp9m6X0GU+z1ZferW3DdkgsI1V 9TzC/owTwMjLhXsw+SA0jZ6AQ944jGNgXW9jr1lty5e/zIxzQK1y6F4iKxoJRiaWCRzI PoxTjWLReMcvpd6ZKJuMT5YrRTkWsqRg5mmi78EGYGXYUvgyutsZSLVB1AeArHsHMCuV +yjj5/Lm7D7jCId3IH4Ni/TPXY01RIJlNgbj7oFkZ4BWfWfC+ZCcirDc99u6Ib6lLfNB HyeWnNWeIMjdJ8F61beBm5jQnSLRVktBBtqFkb9g5DZ2G4a6ZHJ4DfLSKbX/NPlMkR4t c3OA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=R7FYyPY8; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id k62-20020a636f41000000b0054201a157fesi11037551pgc.427.2023.06.14.09.21.56; Wed, 14 Jun 2023 09:22:10 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=R7FYyPY8; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244502AbjFNPpS (ORCPT + 99 others); Wed, 14 Jun 2023 11:45:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43414 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244700AbjFNPog (ORCPT ); Wed, 14 Jun 2023 11:44:36 -0400 Received: from mail-yw1-x1149.google.com (mail-yw1-x1149.google.com [IPv6:2607:f8b0:4864:20::1149]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A466F2721 for ; Wed, 14 Jun 2023 08:44:23 -0700 (PDT) Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-570022400b9so10154197b3.0 for ; Wed, 14 Jun 2023 08:44:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686757462; x=1689349462; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=nXG6hIyHVJ6VKFy81TWuK5UaUuQM8CF9N4uqi7zpqBY=; b=R7FYyPY8TwvB20rCtxYPY/uQ5V5mDRkEHmhZfOb9KmTXHRM7eCTlVaQLZTxHDxNfqQ NmcTodCbjXklTQtaW9342WzLO/qDXql2oLQDMgBHUYeCESt+DxwMmAfoRk5gB5eHnSN6 VNGNlCaRnEjOw7yP6Q7EfZPKHD8LfWOhNKARyLC2efpqVESxw7rxqoSYKKlEystXOSoG 9zY7HcKK7aUsKlwsobKQjbGSkVDpSqiBA1ykeBjIrXXiTSGJZLGbiGlL2z9pJ5yWw9tQ PUyzvANopNaCfONGGpI3ju5/ofPKh0JCJ82lCwIX+Foujvvhvrmb1qY7e8P6edZ6x9Mc t1lw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686757462; x=1689349462; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=nXG6hIyHVJ6VKFy81TWuK5UaUuQM8CF9N4uqi7zpqBY=; b=NyqVXi8JBzCe+e07skvXX/TD8eaFkU81Uq/YBt2t24AZNKo0AteWLCqS2d8uFoHX/l GYHIiI6p9mlN+lk5HipR6Gq7Z+grP440YByGQvFqJx7ec9yFv5L3mCostwo4OXxnRgc7 hLp4Bd2r2u1ELLuxhM4jAPlpm3BGeWjSrxWSPwmUSAc/cUcG+YbpPnkM/K/LIb/bRnTJ Wphr60NcJycGetkMCpJdYOC2Mgl34/Zhiflg7Yje0/kXgm2w/Bo20gwPfk7K1uomu52z MWmhH5xQcADegiVmbz7GTaIdrgB614xnM+wJb+PhpjGULhoPXwRPV0BL9sS4MoUyQDAa LXpA== X-Gm-Message-State: AC+VfDw2lLMgGz/ZDl/VMnmWT4xH+K8uQP4yOVAzFtoMmuAoangkULtb 6MGvkN/+VZjJVtVxTquIDh2Dajylk/30 X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:c6e6:49bf:5c44:5965]) (user=mshavit job=sendgmr) by 2002:a05:690c:711:b0:56d:1f95:fde8 with SMTP id bs17-20020a05690c071100b0056d1f95fde8mr996359ywb.4.1686757462821; Wed, 14 Jun 2023 08:44:22 -0700 (PDT) Date: Wed, 14 Jun 2023 23:41:59 +0800 In-Reply-To: <20230614154304.2860121-1-mshavit@google.com> Mime-Version: 1.0 References: <20230614154304.2860121-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230614154304.2860121-8-mshavit@google.com> Subject: [PATCH v3 07/13] iommu/arm-smmu-v3: Keep track of attached ssids From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED, USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768695771330633421?= X-GMAIL-MSGID: =?utf-8?q?1768695771330633421?= The arm-smmu-v3 driver keeps track of all masters that a domain is attached to so that it can re-write their STEs when the domain's ASID is upated by SVA. This tracking is also used to invalidate ATCs on all masters that a domain is attached to. This change introduces a new data structures to track all the CD entries that a domain is attached to. This change is a pre-requisite to allow domain attachment on non 0 SSIDs. Signed-off-by: Michael Shavit --- v1->v2: Fix arm_smmu_atc_inv_cmd_set_ssid and other cosmetic changes --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 53 +++++++---- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 88 ++++++++++++------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 18 ++-- 3 files changed, 105 insertions(+), 54 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 48fa8eb271a45..d07c08b53c5cf 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -51,6 +51,7 @@ arm_smmu_share_asid(struct mm_struct *mm, u16 asid) struct arm_smmu_device *smmu; struct arm_smmu_domain *smmu_domain; struct arm_smmu_master *master; + struct arm_smmu_attached_domain *attached_domain; cd = xa_load(&arm_smmu_asid_xa, asid); if (!cd) @@ -82,11 +83,14 @@ arm_smmu_share_asid(struct mm_struct *mm, u16 asid) * be some overlap between use of both ASIDs, until we invalidate the * TLB. */ - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { - arm_smmu_write_ctx_desc(smmu, master->s1_cfg, master, 0, cd); + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_for_each_entry(attached_domain, &smmu_domain->attached_domains, + domain_head) { + master = attached_domain->master; + arm_smmu_write_ctx_desc(smmu, master->s1_cfg, master, + attached_domain->ssid, cd); } - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); /* Invalidate TLB entries previously associated with that context */ arm_smmu_tlb_inv_asid(smmu, asid); @@ -210,7 +214,7 @@ static void arm_smmu_mm_invalidate_range(struct mmu_notifier *mn, if (!(smmu_domain->smmu->features & ARM_SMMU_FEAT_BTM)) arm_smmu_tlb_inv_range_asid(start, size, smmu_mn->cd->asid, PAGE_SIZE, false, smmu_domain); - arm_smmu_atc_inv_domain(smmu_domain, mm->pasid, start, size); + arm_smmu_atc_inv_domain_ssid(smmu_domain, mm->pasid, start, size); } static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) @@ -218,6 +222,7 @@ static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) struct arm_smmu_mmu_notifier *smmu_mn = mn_to_smmu(mn); struct arm_smmu_domain *smmu_domain = smmu_mn->domain; struct arm_smmu_master *master; + struct arm_smmu_attached_domain *attached_domain; unsigned long flags; mutex_lock(&sva_lock); @@ -230,15 +235,21 @@ static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) * DMA may still be running. Keep the cd valid to avoid C_BAD_CD events, * but disable translation. */ - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { - arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, - mm->pasid, &quiet_cd); + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_for_each_entry(attached_domain, &smmu_domain->attached_domains, + domain_head) { + master = attached_domain->master; + /* + * SVA domains piggyback on the attached_domain with SSID 0. + */ + if (attached_domain->ssid == 0) + arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, + master, mm->pasid, &quiet_cd); } - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); arm_smmu_tlb_inv_asid(smmu_domain->smmu, smmu_mn->cd->asid); - arm_smmu_atc_inv_domain(smmu_domain, mm->pasid, 0, 0); + arm_smmu_atc_inv_domain_ssid(smmu_domain, mm->pasid, 0, 0); smmu_mn->cleared = true; mutex_unlock(&sva_lock); @@ -265,6 +276,7 @@ arm_smmu_mmu_notifier_get(struct arm_smmu_domain *smmu_domain, struct arm_smmu_ctx_desc *cd; struct arm_smmu_mmu_notifier *smmu_mn; struct arm_smmu_master *master; + struct arm_smmu_attached_domain *attached_domain; list_for_each_entry(smmu_mn, &smmu_domain->mmu_notifiers, list) { if (smmu_mn->mn.mm == mm) { @@ -294,12 +306,14 @@ arm_smmu_mmu_notifier_get(struct arm_smmu_domain *smmu_domain, goto err_free_cd; } - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_for_each_entry(attached_domain, &smmu_domain->attached_domains, + domain_head) { + master = attached_domain->master; ret = arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, mm->pasid, cd); } - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); if (ret) goto err_put_notifier; @@ -319,6 +333,7 @@ static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) unsigned long flags; struct mm_struct *mm = smmu_mn->mn.mm; struct arm_smmu_ctx_desc *cd = smmu_mn->cd; + struct arm_smmu_attached_domain *attached_domain; struct arm_smmu_master *master; struct arm_smmu_domain *smmu_domain = smmu_mn->domain; @@ -327,12 +342,14 @@ static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) list_del(&smmu_mn->list); - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_for_each_entry(attached_domain, &smmu_domain->attached_domains, + domain_head) { + master = attached_domain->master; arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, mm->pasid, NULL); } - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); /* * If we went through clear(), we've already invalidated, and no @@ -340,7 +357,7 @@ static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) */ if (!smmu_mn->cleared) { arm_smmu_tlb_inv_asid(smmu_domain->smmu, cd->asid); - arm_smmu_atc_inv_domain(smmu_domain, mm->pasid, 0, 0); + arm_smmu_atc_inv_domain_ssid(smmu_domain, mm->pasid, 0, 0); } /* Frees smmu_mn */ diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index dc7a59e87a2b4..70580ba7065dc 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1711,7 +1711,14 @@ static irqreturn_t arm_smmu_combined_irq_handler(int irq, void *dev) } static void -arm_smmu_atc_inv_to_cmd(int ssid, unsigned long iova, size_t size, +arm_smmu_atc_inv_cmd_set_ssid(int ssid, struct arm_smmu_cmdq_ent *cmd) +{ + cmd->substream_valid = !!ssid; + cmd->atc.ssid = ssid; +} + +static void +arm_smmu_atc_inv_to_cmd(unsigned long iova, size_t size, struct arm_smmu_cmdq_ent *cmd) { size_t log2_span; @@ -1736,8 +1743,8 @@ arm_smmu_atc_inv_to_cmd(int ssid, unsigned long iova, size_t size, */ *cmd = (struct arm_smmu_cmdq_ent) { .opcode = CMDQ_OP_ATC_INV, - .substream_valid = !!ssid, - .atc.ssid = ssid, + .substream_valid = false, + .atc.ssid = 0, }; if (!size) { @@ -1783,8 +1790,7 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) struct arm_smmu_cmdq_ent cmd; struct arm_smmu_cmdq_batch cmds; - arm_smmu_atc_inv_to_cmd(0, 0, 0, &cmd); - + arm_smmu_atc_inv_to_cmd(0, 0, &cmd); cmds.num = 0; for (i = 0; i < master->num_streams; i++) { cmd.atc.sid = master->streams[i].id; @@ -1794,13 +1800,19 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) return arm_smmu_cmdq_batch_submit(master->smmu, &cmds); } -int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, - unsigned long iova, size_t size) +/* + * If ssid is non-zero, issue atc invalidations with the given ssid instead of + * the one the domain is attached to. This is used by SVA since it's pasid + * attachments aren't recorded in smmu_domain yet. + */ +int arm_smmu_atc_inv_domain_ssid(struct arm_smmu_domain *smmu_domain, int ssid, + unsigned long iova, size_t size) { int i; unsigned long flags; struct arm_smmu_cmdq_ent cmd; struct arm_smmu_master *master; + struct arm_smmu_attached_domain *attached_domain; struct arm_smmu_cmdq_batch cmds; if (!(smmu_domain->smmu->features & ARM_SMMU_FEAT_ATS)) @@ -1823,25 +1835,37 @@ int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, if (!atomic_read(&smmu_domain->nr_ats_masters)) return 0; - arm_smmu_atc_inv_to_cmd(ssid, iova, size, &cmd); + arm_smmu_atc_inv_to_cmd(iova, size, &cmd); cmds.num = 0; - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_for_each_entry(attached_domain, &smmu_domain->attached_domains, + domain_head) { + master = attached_domain->master; if (!master->ats_enabled) continue; + if (ssid != 0) + arm_smmu_atc_inv_cmd_set_ssid(ssid, &cmd); + else + arm_smmu_atc_inv_cmd_set_ssid(attached_domain->ssid, &cmd); for (i = 0; i < master->num_streams; i++) { cmd.atc.sid = master->streams[i].id; arm_smmu_cmdq_batch_add(smmu_domain->smmu, &cmds, &cmd); } } - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); return arm_smmu_cmdq_batch_submit(smmu_domain->smmu, &cmds); } +int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, + unsigned long iova, size_t size) +{ + return arm_smmu_atc_inv_domain_ssid(smmu_domain, 0, iova, size); +} + /* IO_PGTABLE API */ static void arm_smmu_tlb_inv_context(void *cookie) { @@ -1863,7 +1887,7 @@ static void arm_smmu_tlb_inv_context(void *cookie) cmd.tlbi.vmid = smmu_domain->s2_cfg.vmid; arm_smmu_cmdq_issue_cmd_with_sync(smmu, &cmd); } - arm_smmu_atc_inv_domain(smmu_domain, 0, 0, 0); + arm_smmu_atc_inv_domain(smmu_domain, 0, 0); } static void __arm_smmu_tlb_inv_range(struct arm_smmu_cmdq_ent *cmd, @@ -1951,7 +1975,7 @@ static void arm_smmu_tlb_inv_range_domain(unsigned long iova, size_t size, * Unfortunately, this can't be leaf-only since we may have * zapped an entire table. */ - arm_smmu_atc_inv_domain(smmu_domain, 0, iova, size); + arm_smmu_atc_inv_domain(smmu_domain, iova, size); } void arm_smmu_tlb_inv_range_asid(unsigned long iova, size_t size, int asid, @@ -2031,8 +2055,8 @@ static struct iommu_domain *arm_smmu_domain_alloc(unsigned type) return NULL; mutex_init(&smmu_domain->init_mutex); - INIT_LIST_HEAD(&smmu_domain->devices); - spin_lock_init(&smmu_domain->devices_lock); + INIT_LIST_HEAD(&smmu_domain->attached_domains); + spin_lock_init(&smmu_domain->attached_domains_lock); INIT_LIST_HEAD(&smmu_domain->mmu_notifiers); return &smmu_domain->domain; @@ -2270,12 +2294,12 @@ static bool arm_smmu_ats_supported(struct arm_smmu_master *master) return dev_is_pci(dev) && pci_ats_supported(to_pci_dev(dev)); } -static void arm_smmu_enable_ats(struct arm_smmu_master *master) +static void arm_smmu_enable_ats(struct arm_smmu_master *master, + struct arm_smmu_domain *smmu_domain) { size_t stu; struct pci_dev *pdev; struct arm_smmu_device *smmu = master->smmu; - struct arm_smmu_domain *smmu_domain = master->domain; /* Don't enable ATS at the endpoint if it's not enabled in the STE */ if (!master->ats_enabled) @@ -2291,10 +2315,9 @@ static void arm_smmu_enable_ats(struct arm_smmu_master *master) dev_err(master->dev, "Failed to enable ATS (STU %zu)\n", stu); } -static void arm_smmu_disable_ats(struct arm_smmu_master *master) +static void arm_smmu_disable_ats(struct arm_smmu_master *master, + struct arm_smmu_domain *smmu_domain) { - struct arm_smmu_domain *smmu_domain = master->domain; - if (!master->ats_enabled) return; @@ -2358,18 +2381,17 @@ static void arm_smmu_disable_pasid(struct arm_smmu_master *master) static void arm_smmu_detach_dev(struct arm_smmu_master *master) { unsigned long flags; - struct arm_smmu_domain *smmu_domain = master->domain; + struct arm_smmu_domain *smmu_domain = master->non_pasid_domain.domain; if (!smmu_domain) return; - arm_smmu_disable_ats(master); + arm_smmu_disable_ats(master, smmu_domain); - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_del(&master->domain_head); - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_del(&master->non_pasid_domain.domain_head); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); - master->domain = NULL; master->ats_enabled = false; if (master->s1_cfg) arm_smmu_write_ctx_desc( @@ -2378,6 +2400,7 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) 0, NULL); master->s1_cfg = NULL; master->s2_cfg = NULL; + master->non_pasid_domain.domain = NULL; arm_smmu_install_ste_for_dev(master); } @@ -2422,7 +2445,6 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) goto out_unlock; } - master->domain = smmu_domain; if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { master->s1_cfg = &master->owned_s1_cfg; ret = arm_smmu_write_ctx_desc( @@ -2449,13 +2471,17 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) if (smmu_domain->stage != ARM_SMMU_DOMAIN_BYPASS) master->ats_enabled = arm_smmu_ats_supported(master); + master->non_pasid_domain.master = master; + master->non_pasid_domain.domain = smmu_domain; + master->non_pasid_domain.ssid = 0; arm_smmu_install_ste_for_dev(master); - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_add(&master->domain_head, &smmu_domain->devices); - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_add(&master->non_pasid_domain.domain_head, + &smmu_domain->attached_domains); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); - arm_smmu_enable_ats(master); + arm_smmu_enable_ats(master, smmu_domain); out_unlock: mutex_unlock(&smmu_domain->init_mutex); diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index dff0fa8345462..6929590530367 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -682,11 +682,19 @@ struct arm_smmu_stream { struct rb_node node; }; +/* List of {masters, ssid} that a domain is attached to */ +struct arm_smmu_attached_domain { + struct list_head domain_head; + struct arm_smmu_domain *domain; + struct arm_smmu_master *master; + int ssid; +}; + /* SMMU private data for each master */ struct arm_smmu_master { struct arm_smmu_device *smmu; struct device *dev; - struct arm_smmu_domain *domain; + struct arm_smmu_attached_domain non_pasid_domain; struct list_head domain_head; struct arm_smmu_stream *streams; struct arm_smmu_s1_cfg owned_s1_cfg; @@ -724,8 +732,8 @@ struct arm_smmu_domain { struct iommu_domain domain; - struct list_head devices; - spinlock_t devices_lock; + struct list_head attached_domains; + spinlock_t attached_domains_lock; struct list_head mmu_notifiers; }; @@ -748,8 +756,8 @@ void arm_smmu_tlb_inv_range_asid(unsigned long iova, size_t size, int asid, size_t granule, bool leaf, struct arm_smmu_domain *smmu_domain); bool arm_smmu_free_asid(struct arm_smmu_ctx_desc *cd); -int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, - unsigned long iova, size_t size); +int arm_smmu_atc_inv_domain_ssid(struct arm_smmu_domain *smmu_domain, int ssid, + unsigned long iova, size_t size); #ifdef CONFIG_ARM_SMMU_V3_SVA bool arm_smmu_sva_supported(struct arm_smmu_device *smmu); From patchwork Wed Jun 14 15:42:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 108020 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp47827vqr; Wed, 14 Jun 2023 08:54:29 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6gE9xZaAoKAa6F7NoR1f8b6wLpGUARsQEJx6LR96LQ+apaf5uQn6zAx3GnnQbsX1elTC48 X-Received: by 2002:a05:6402:12d0:b0:518:7a48:8cef with SMTP id k16-20020a05640212d000b005187a488cefmr3041983edx.33.1686758069557; Wed, 14 Jun 2023 08:54:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686758069; cv=none; d=google.com; s=arc-20160816; b=yn9pnFyrKa6JanNWDsLbVNLgbxvz+DVTrJfBMqqq3DERz71Y/eoXv0gB8By5DXyR3X a+fW6TqmO1CuazsNHhPzYDXkcDgX9krYsqJHGfFQZ0Lcxb5QJI2Zt8iQ+nvG5UBj+v5O i/1EyLlnosySe+/jMY9rEZkTesgE+triog7i4D9A2vSsKri84M1Nv7911H5vfwtOTzXv vcYzblNrHKJq9GELGWXjwU3E1vzkQH4RClrLDa9fYtIa7Xu8rJGrcKLy4037yVLSCRnT QzhubjTuvW0ZFR/ObieeFryzHuzQ4wqrlSKb+6qm2PfVX07c1bzKJu3DV3Lo8ENfdKbo XBBQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=omVhI9NBPuwnuDBL2dAuwy0d7VVlvqTslf6ohlhvBoo=; b=mbIBFxd6acWpE7FwWppb94doUP/73MwRTbfjEI3WyL4m57S4qU0tOxV9+/aec0uYLV dahJsZIwttukIvl/NtTEc0gr8UALokB8qSry0vCSVM0HwmOh4duniG9ktaIXfEpS+pyn UqdqqBrTLyTmS0WzzNv29c3NgnEISt0ZvmHYiLWPABaY2xEq0zgziDKsl43IsUeHngeq s744b1qkBKem5qi4oVQ1x0m9PwpXxV1jWtAsq425zz6J+uHdQmdtGJeqJNTFvApPcFLf vz+eKUQTzCu1ksRuQi5aBxiRsMZk+dcLUwoLscmPY1WcCOKSag6MFY1sOeX+TrqyGiFt RTvA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b="jMlnk/MK"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id ay24-20020a056402203800b00514b9801aa2si8935318edb.364.2023.06.14.08.53.11; Wed, 14 Jun 2023 08:54:29 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b="jMlnk/MK"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244662AbjFNPpV (ORCPT + 99 others); Wed, 14 Jun 2023 11:45:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43562 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244844AbjFNPoz (ORCPT ); Wed, 14 Jun 2023 11:44:55 -0400 Received: from mail-yw1-x1149.google.com (mail-yw1-x1149.google.com [IPv6:2607:f8b0:4864:20::1149]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B4DEC2964 for ; Wed, 14 Jun 2023 08:44:31 -0700 (PDT) Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-56fffdea2d0so11289977b3.1 for ; Wed, 14 Jun 2023 08:44:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686757471; x=1689349471; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=omVhI9NBPuwnuDBL2dAuwy0d7VVlvqTslf6ohlhvBoo=; b=jMlnk/MKenoGoe+vncInH3bNLbjsOY0Xrv5py/yEVhJC0Voa9L6CSXUuAVOi6fmYCJ Tgf0uByedAMjl4kDugva5Z7ex59EM1z4IazeM7oUL8ziiQDhW3iZaQ0JG9Qm40ngiaZL +M5Dg5le9q5NHjjTl7k9QZk59/KC3EuCDrBCsL0bF4lycPrvzpsN0P/LhUKYz/tUyV0x isJj8aPRjzh8meDpJOrtJNeFlx8yvD55tCSLvcLBEnDtcXfsEbJZtuB1BwtuBcopqymr /Rvz8IKzzGTFog5Cr0gOu/QV8fyEzYweTi1tUgmgmlB4M3eES0bWGS4bsAPec9Qi8ALG 4M8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686757471; x=1689349471; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=omVhI9NBPuwnuDBL2dAuwy0d7VVlvqTslf6ohlhvBoo=; b=Bh3Gb5S7qzj0H97bclROhZb9eLtQupad0t3DkCAoXb/OQacU/locz5zauFM5fFzyXb COwG0Mj7RZYcAjaM3jArmjDJWiYWmds4jWsqlTNoROJ7rtQ84QfhDk6ZgLy6J3sYrDIU Hg23lZQgoRIgGrx6UVCbHWfehRzHWfgAB4LTO41QIaZLQmVgDDht+oePqqHl9SikqCuX 5bFUeZM97LjR6E0BObvBumpIN3h9EYtbahvfFQt+Kn66/585FVMkt0WkNEuUvygMlHx8 AOIe34lB9BHFrK8HqC5KzdlvQoACmKmIdYE8wYfFGlIwVMWpeGvZO+NiwrWERAt+JCFF 9w9g== X-Gm-Message-State: AC+VfDy8oqcEk02luulnnK9hnclvERiSsfLcJlhjN69rAdftv565u7ZV FYbBn47Phzh+WMubDuRfYE6GrehjPW7W X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:c6e6:49bf:5c44:5965]) (user=mshavit job=sendgmr) by 2002:a81:4426:0:b0:568:c4ea:ce66 with SMTP id r38-20020a814426000000b00568c4eace66mr971778ywa.5.1686757470875; Wed, 14 Jun 2023 08:44:30 -0700 (PDT) Date: Wed, 14 Jun 2023 23:42:00 +0800 In-Reply-To: <20230614154304.2860121-1-mshavit@google.com> Mime-Version: 1.0 References: <20230614154304.2860121-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230614154304.2860121-9-mshavit@google.com> Subject: [PATCH v3 08/13] iommu/arm-smmu-v3: Add helper for atc invalidation From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768694029691929911?= X-GMAIL-MSGID: =?utf-8?q?1768694029691929911?= This will be used to invalidate ATC entries made on an SSID for a master when detaching a domain with pasid. Signed-off-by: Michael Shavit --- v1->v2: Make use of arm_smmu_atc_inv_cmd_set_ssid --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 70580ba7065dc..176013bb974b8 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1784,13 +1784,15 @@ arm_smmu_atc_inv_to_cmd(unsigned long iova, size_t size, cmd->atc.size = log2_span; } -static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) +static int arm_smmu_atc_inv_master_ssid(struct arm_smmu_master *master, + int ssid) { int i; struct arm_smmu_cmdq_ent cmd; struct arm_smmu_cmdq_batch cmds; arm_smmu_atc_inv_to_cmd(0, 0, &cmd); + arm_smmu_atc_inv_cmd_set_ssid(ssid, &cmd); cmds.num = 0; for (i = 0; i < master->num_streams; i++) { cmd.atc.sid = master->streams[i].id; @@ -1800,6 +1802,11 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) return arm_smmu_cmdq_batch_submit(master->smmu, &cmds); } +static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) +{ + return arm_smmu_atc_inv_master_ssid(master, 0); +} + /* * If ssid is non-zero, issue atc invalidations with the given ssid instead of * the one the domain is attached to. This is used by SVA since it's pasid From patchwork Wed Jun 14 15:42:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 108031 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp68423vqr; Wed, 14 Jun 2023 09:20:52 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5od5DRhVHVNLS62BfhJz/78cUzd5G3+5oE769dxey0NDJOl6hYh6HTzagxxlvJD97d8//y X-Received: by 2002:a54:400a:0:b0:39a:b525:2a2e with SMTP id x10-20020a54400a000000b0039ab5252a2emr9751346oie.41.1686759652702; Wed, 14 Jun 2023 09:20:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686759652; cv=none; d=google.com; s=arc-20160816; b=RNkD4If+LAY/xyY2sBCW5IIEwAvYe1jId80tyUglSu1E4LNpprjk2jniHdZpDEmAtV RMDu+YYxTqy+Gjj5/s0ELFi6xTub2ds74enzEvwDTcyS5UW8MtUlcaCUU7a7wSKIL4KC 53Qga8s0ryWwpCP8jQWeQ8BwlDmb2H09HQZmz+GKJt1ZctmPj/TdKOopgy+qBC1G0dXV QXBvhPjrzarxspVZmPtlnE/Wz+xB3hJN2NDdbRhIv8PSnmruoNxxjKpoxthCNeqE2A9M a2lM615PbPANAurOtpBqqzcVh4TlT24vbAiB5GAOTbPZY3Q+0ePRpu49o8ls8Uo1yN05 Cg1A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=zp8WWvGOizfWW9ZsurLF24HEAfBobFq4bOmSr8VWFOI=; b=G04OfK9JmXiOjLXP0hAjJUijdiRlfrUiQawqv65d1GuauqL6GgNDv8utjqyhJPdnLJ w+WRqV/CgOsBNHKq+UYI0R6kX3zwWor0O5IgPSzUIilE+CUkjaGEHjLd333xbTqXkwU1 nO0PHShB+I6Xp+Ryi68V2PRMmShswKMOcq9DJVhCVgy619YtrHu5/7v3izPQ0SRKXkUG 9cbLmQQ6El9FnI1F7Z4U3Ny9Lg2yNAbWBWL2Ooj4KKaYoT0CnoFoHf6yHU3nSP7V9xW4 PZIttM/MGyGqQlvKuiTLphF3l9CWvUtnNctWf/d4tX3PQKKzDTA/rjeDUzPbhDbRSDh3 nL9A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=P33CVxX4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id u4-20020a17090a4bc400b0025636b43cbasi10857422pjl.129.2023.06.14.09.20.39; Wed, 14 Jun 2023 09:20:52 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=P33CVxX4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244804AbjFNPp0 (ORCPT + 99 others); Wed, 14 Jun 2023 11:45:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42750 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245105AbjFNPpG (ORCPT ); Wed, 14 Jun 2023 11:45:06 -0400 Received: from mail-pf1-x44a.google.com (mail-pf1-x44a.google.com [IPv6:2607:f8b0:4864:20::44a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5B843296C for ; Wed, 14 Jun 2023 08:44:39 -0700 (PDT) Received: by mail-pf1-x44a.google.com with SMTP id d2e1a72fcca58-6662fcaac6dso859945b3a.3 for ; Wed, 14 Jun 2023 08:44:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686757478; x=1689349478; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=zp8WWvGOizfWW9ZsurLF24HEAfBobFq4bOmSr8VWFOI=; b=P33CVxX4esw3dBSK5S17bbxhmwwXbDXFc5BNId/SskkIfaAcUMv/viU+8T/DGWsJKX W9bktuoWVfSWVcE8R8uDRfAvQ4J8cBGtPOqgyU2/iC9WsioU9cykfvXPVsg70c2BTtRO Gn53yG3SbQkxPaXuBcq5feCokxIsGkfeR1AnKvxlwKGyvIJf18/P/bs0RKpu0Zo9BCVG RFqiWZSuLLIGNb+GzEW5JjCnRYL4iGTuffbe32LPPmJP0d7z/jH5V0+foHmxOBBmhJ2e B2976zpIyp5dSuDIvhNrhiaSnDq4OVl9P5ifH6jbPCuE4HSZXUETXYqnZp7CStHkqLdk cEKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686757478; x=1689349478; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=zp8WWvGOizfWW9ZsurLF24HEAfBobFq4bOmSr8VWFOI=; b=ewVySoioK0NDby4k6HRZCHJoXXwY2qZsYlPtygOMI+UEZ3XKjmwHwxwHDqP8ypL2s2 K1iVf5fmkQx02nAGzlYmn9QiKnteJtxsNdBp32NS0asL1OTUjoh0980lKnip8mNhI7x0 dd/YsfS6nHmB5oWq8yTJzKA+2tN8mA/2nADmlUGplbXgKJfIRfDPttUu8/DfoVW1ysn5 vxjF5FtQ0O1J/JImpiQwPargC5YQPf4GAkdizwXHNGkCng8m2mDMy+TiEKZv+wAYXS1K xu6nhENKAgw2LqX07lJ6MuSj+nKr+tuRTQvhwUgDXOF3fbMdwSao670dGvDloTyBf97i 7new== X-Gm-Message-State: AC+VfDzEYM0WrU3H3ZEbImb2xF8HEae5Z5jTVCNlePYqPpZOm84ihLFe /8Xq2uXz5FhG8VF375l7pgtYtvaeEcjj X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:c6e6:49bf:5c44:5965]) (user=mshavit job=sendgmr) by 2002:a05:6a00:1513:b0:64f:c0b1:6967 with SMTP id q19-20020a056a00151300b0064fc0b16967mr682372pfu.1.1686757478685; Wed, 14 Jun 2023 08:44:38 -0700 (PDT) Date: Wed, 14 Jun 2023 23:42:01 +0800 In-Reply-To: <20230614154304.2860121-1-mshavit@google.com> Mime-Version: 1.0 References: <20230614154304.2860121-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230614154304.2860121-10-mshavit@google.com> Subject: [PATCH v3 09/13] iommu/arm-smmu-v3: Implement set_dev_pasid From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768695689588585295?= X-GMAIL-MSGID: =?utf-8?q?1768695689588585295?= This change enables the use of the iommu_attach_dev_pasid API for UNMANAGED domains. The primary use-case is to allow in-kernel users of the iommu API to manage domains with PASID. This change also allows for future support of pasid in the DMA api. Signed-off-by: Michael Shavit --- v1->v2: Add missing atc invalidation when detaching with pasid --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 168 +++++++++++++++++--- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 1 + 2 files changed, 149 insertions(+), 20 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 176013bb974b8..a6fa56585c219 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2173,6 +2173,10 @@ static int arm_smmu_domain_finalise_s2(struct arm_smmu_domain *smmu_domain, return 0; } +/* + * master may be null for domain types that are finalized before being attached + * to a master. + */ static int arm_smmu_domain_finalise(struct iommu_domain *domain, struct arm_smmu_master *master) { @@ -2369,6 +2373,11 @@ static int arm_smmu_enable_pasid(struct arm_smmu_master *master) return 0; } +static bool arm_smmu_master_has_pasid_domains(struct arm_smmu_master *master) +{ + return master->nr_attached_pasid_domains > 0; +} + static void arm_smmu_disable_pasid(struct arm_smmu_master *master) { struct pci_dev *pdev; @@ -2411,6 +2420,28 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) arm_smmu_install_ste_for_dev(master); } +/* + * Once attached for the first time, a domain can no longer be attached to any + * master with a distinct upstream SMMU. + */ +static int arm_smmu_prepare_domain_for_smmu(struct arm_smmu_device *smmu, + struct arm_smmu_domain *smmu_domain) +{ + int ret = 0; + + mutex_lock(&smmu_domain->init_mutex); + if (!smmu_domain->smmu) { + smmu_domain->smmu = smmu; + ret = arm_smmu_domain_finalise(&smmu_domain->domain, NULL); + if (ret) + smmu_domain->smmu = NULL; + } else if (smmu_domain->smmu != smmu) { + ret = -EINVAL; + } + mutex_unlock(&smmu_domain->init_mutex); + return ret; +} + static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) { int ret = 0; @@ -2426,6 +2457,10 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) master = dev_iommu_priv_get(dev); smmu = master->smmu; + ret = arm_smmu_prepare_domain_for_smmu(smmu, smmu_domain); + if (ret) + return ret; + /* * Checking that SVA is disabled ensures that this device isn't bound to * any mm, and can be safely detached from its old domain. Bonds cannot @@ -2436,22 +2471,18 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) return -EBUSY; } - arm_smmu_detach_dev(master); - - mutex_lock(&smmu_domain->init_mutex); - - if (!smmu_domain->smmu) { - smmu_domain->smmu = smmu; - ret = arm_smmu_domain_finalise(domain, master); - if (ret) { - smmu_domain->smmu = NULL; - goto out_unlock; - } - } else if (smmu_domain->smmu != smmu) { - ret = -EINVAL; - goto out_unlock; + /* + * Attaching a bypass or stage 2 domain would break any domains attached + * with pasid. Attaching an S1 domain should be feasible but requires + * more complicated logic to handle. + */ + if (arm_smmu_master_has_pasid_domains(master)) { + dev_err(dev, "cannot attach - domain attached with pasid\n"); + return -EBUSY; } + arm_smmu_detach_dev(master); + if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { master->s1_cfg = &master->owned_s1_cfg; ret = arm_smmu_write_ctx_desc( @@ -2460,8 +2491,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) 0, &smmu_domain->cd); if (ret) { master->s1_cfg = NULL; - master->domain = NULL; - goto out_unlock; + return ret; } } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S2 || smmu_domain->stage == ARM_SMMU_DOMAIN_NESTED) { @@ -2490,11 +2520,75 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) arm_smmu_enable_ats(master, smmu_domain); -out_unlock: - mutex_unlock(&smmu_domain->init_mutex); return ret; } +static int arm_smmu_set_dev_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid) +{ + int ret = 0; + unsigned long flags; + struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); + struct arm_smmu_device *smmu; + struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); + struct arm_smmu_attached_domain *attached_domain; + struct arm_smmu_master *master; + + if (!fwspec) + return -ENOENT; + + master = dev_iommu_priv_get(dev); + smmu = master->smmu; + + ret = arm_smmu_prepare_domain_for_smmu(smmu, smmu_domain); + if (ret) + return ret; + + if (pasid == 0) { + dev_err(dev, "pasid 0 is reserved for the device's primary domain\n"); + return -ENODEV; + } + + if (smmu_domain->stage != ARM_SMMU_DOMAIN_S1) { + dev_err(dev, "set_dev_pasid only supports stage 1 domains\n"); + return -EINVAL; + } + + if (!master->s1_cfg || master->s2_cfg) + return -EBUSY; + + attached_domain = kzalloc(sizeof(*attached_domain), GFP_KERNEL); + if (!attached_domain) + return -ENOMEM; + + attached_domain->master = master; + attached_domain->domain = smmu_domain; + attached_domain->ssid = pasid; + + master->nr_attached_pasid_domains += 1; + /* + * arm_smmu_share_asid may update the cd's asid value and write the + * ctx_desc for every attached_domains in the list. There's a potential + * race here regardless of whether we first write the ctx_desc or + * first insert into the domain's list. Grabbing the asic_lock prevents + * SVA from changing the cd's ASID while the cd is being attached. + */ + mutex_lock(&arm_smmu_asid_lock); + ret = arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, + pasid, &smmu_domain->cd); + if (ret) { + mutex_unlock(&arm_smmu_asid_lock); + kfree(attached_domain); + } + + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_add(&attached_domain->domain_head, &smmu_domain->attached_domains); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); + mutex_unlock(&arm_smmu_asid_lock); + + return 0; +} + static int arm_smmu_map_pages(struct iommu_domain *domain, unsigned long iova, phys_addr_t paddr, size_t pgsize, size_t pgcount, int prot, gfp_t gfp, size_t *mapped) @@ -2740,6 +2834,15 @@ static void arm_smmu_release_device(struct device *dev) if (WARN_ON(arm_smmu_master_sva_enabled(master))) iopf_queue_remove_device(master->smmu->evtq.iopf, dev); + if (WARN_ON(master->nr_attached_pasid_domains != 0)) { + /* + * TODO: Do we need to handle this case? + * This requires a mechanism to obtain all the pasid domains + * that this master is attached to so that we can clean up the + * domain's attached_domain list. + */ + } + arm_smmu_detach_dev(master); arm_smmu_free_cd_tables(master->smmu, &master->owned_s1_cfg.cdcfg); arm_smmu_disable_pasid(master); @@ -2875,12 +2978,36 @@ static int arm_smmu_def_domain_type(struct device *dev) static void arm_smmu_remove_dev_pasid(struct device *dev, ioasid_t pasid) { struct iommu_domain *domain; + struct arm_smmu_master *master = dev_iommu_priv_get(dev); + struct arm_smmu_domain *smmu_domain; + struct arm_smmu_attached_domain *attached_domain; + unsigned long flags; - domain = iommu_get_domain_for_dev_pasid(dev, pasid, IOMMU_DOMAIN_SVA); + if (!master || pasid == 0) + return; + + domain = iommu_get_domain_for_dev_pasid(dev, pasid, 0); if (WARN_ON(IS_ERR(domain)) || !domain) return; + if (domain->type == IOMMU_DOMAIN_SVA) + return arm_smmu_sva_remove_dev_pasid(domain, dev, pasid); - arm_smmu_sva_remove_dev_pasid(domain, dev, pasid); + smmu_domain = to_smmu_domain(domain); + mutex_lock(&arm_smmu_asid_lock); + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_for_each_entry(attached_domain, &smmu_domain->attached_domains, domain_head) { + if (attached_domain->master != master || + attached_domain->ssid != pasid) + continue; + list_del(&attached_domain->domain_head); + break; + } + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); + arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, pasid, + NULL); + arm_smmu_atc_inv_master_ssid(master, pasid); + master->nr_attached_pasid_domains -= 1; + mutex_unlock(&arm_smmu_asid_lock); } static struct iommu_ops arm_smmu_ops = { @@ -2900,6 +3027,7 @@ static struct iommu_ops arm_smmu_ops = { .owner = THIS_MODULE, .default_domain_ops = &(const struct iommu_domain_ops) { .attach_dev = arm_smmu_attach_dev, + .set_dev_pasid = arm_smmu_set_dev_pasid, .map_pages = arm_smmu_map_pages, .unmap_pages = arm_smmu_unmap_pages, .flush_iotlb_all = arm_smmu_flush_iotlb_all, diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 6929590530367..48795a7287b69 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -707,6 +707,7 @@ struct arm_smmu_master { bool iopf_enabled; struct list_head bonds; unsigned int ssid_bits; + unsigned int nr_attached_pasid_domains; }; /* SMMU private data for an IOMMU domain */ From patchwork Wed Jun 14 15:42:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 108036 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp76128vqr; Wed, 14 Jun 2023 09:31:50 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ42QmC6aBpOHVCZheh26I3VJNA5iTsgb8rWsaG81yf1/IkX8WFvRH9ItBooyWoIx75GkujA X-Received: by 2002:ac2:5bd1:0:b0:4f6:2852:5f56 with SMTP id u17-20020ac25bd1000000b004f628525f56mr8414923lfn.18.1686760310574; Wed, 14 Jun 2023 09:31:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686760310; cv=none; d=google.com; s=arc-20160816; b=CWV7ShafH9yB2dVhEPOdl5q7qCeV04bst1tSKwtv5WqqZoQfym+BOx6eT14bLA77LW 0utuBBlopxb3ZEDoK9EmJMTKOF4HBEuZ3ftpuiOIU0zhpqypAu9E3YoNVKKQn8Nvl+PW 9j5jR1BjLSoQRbfnjLj0UkB9A7nDcUB79ryHiHMviu8k/MrmwHRaBuy/14WFU940vvDo pPNifepLMfzIyl8m2WJlUp/K1Ao91vfGNQFxbKq+vN9YCsvmCS3DLLCkY3OjtHWohICk XY8fR5QAzr6XrUHpa7pn2CwNFjAXR9Jwg3TFrZWXDSz1mescZvOO6fR4gNh2qCzHQMdk /uPg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=tvomj7iW2UACU6/yECNbFWEXmb4EH1aV19zctsWiNw0=; b=o+TnUJDRq99tsVYRum2tF3BUI77x5TWNxMFD08ZNuUUAFJjTgV1dXHCCarFvGF2NY3 w5bZE7j7O9PWxOq36xP0Adf1TE8iS47F0PLPjFVlgCUV02SUHGSENL9IP+ceo/atrWFP k4abmPunut1TJ8pyVsEYGonLJnHzq9ZY+5Ru84NXPkTvAK564IbIPjVvASS4QDNlYR1U uSfdpDY53iSesH5tVqc43OGSJf4BkWn9q2cKKDY30KJ/icDFMk4ORf7MglNbS75rvwa4 tsjXSE/OGK6hH0iGGHTIPgOpFshE01LVVQZAuCQDqxhACnTUn1NcGytf2OxGqYuv+xYH pOvw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=vuL1XrQp; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id kq19-20020a170906abd300b00965cf8f1be1si6026793ejb.617.2023.06.14.09.31.10; Wed, 14 Jun 2023 09:31:50 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=vuL1XrQp; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244881AbjFNPpw (ORCPT + 99 others); Wed, 14 Jun 2023 11:45:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44264 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244622AbjFNPp2 (ORCPT ); Wed, 14 Jun 2023 11:45:28 -0400 Received: from mail-yw1-x1149.google.com (mail-yw1-x1149.google.com [IPv6:2607:f8b0:4864:20::1149]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CFE3F2101 for ; Wed, 14 Jun 2023 08:45:05 -0700 (PDT) Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-56ff81be091so12265027b3.0 for ; Wed, 14 Jun 2023 08:45:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686757486; x=1689349486; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=tvomj7iW2UACU6/yECNbFWEXmb4EH1aV19zctsWiNw0=; b=vuL1XrQpU5Uq0e2ygjk0DfQm99VPq2z9YNT+avIv27XZ5urDEgwpmKbViVzjQ8Ors2 T0rraZDZVM0ODSgkQTw6g6hdTWSyZHC/ZwHObRZETsh7v9KsqI7mfjRVjTddqwb75Yd6 PkKi9NkGSgecoW5IThOwn0gGyOxQIy0qaPS8Ued69ldHUGXC3bEZuqzTL/jNRXCrcJi3 uAxDiKxWDeKXd8Icj/EZsPQK5O6GRp7wJwAZct0hM+oyxo3a3VP2+SBs18GWULEH6AcA +Q3vGN3bOvN8pQVCjBFeK+N8axcpPszN+TMygeq+12fShtLMkNw5EOWGCZf8F5q4Py43 k0CQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686757486; x=1689349486; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=tvomj7iW2UACU6/yECNbFWEXmb4EH1aV19zctsWiNw0=; b=M+v16h2MnOGFrv1NuWcGXbqky6y5WLQh7/ySc+7rWrjwXhosOJm1aYD3IK2Ne+LLw7 Ed+hJBobZtBAOSwrb1T842IEgBRyLW72EBJqB9H7AmcZlB2ITS9tVthjCTuStP4tf15P DjjKZoBDtIvm1YYHBr9W7sXxdyqEDCvtj/iB10pUkrlGwivGV3aA5k5ReMvF6zT1J054 ENW4n8ZcX6VFfOpFE2A8DYP2ysGL6OG77KUxxhzCVsXMtMUGzbjV2pHDEbmnFOAH3viW W7rHyJv21pX8s5GwIFlrYeETouzFe+N1aBXjT+bQgTwajPotvRx3X4nQUl6ZwKMOcHFG 9Gkw== X-Gm-Message-State: AC+VfDxYkZniZ3XdeSRnAKk+LKcD97h+Sh3fyOmxzbzjePSdyJv1LvzH vJHLD0NycZqjkEefuE/mp4rzMShVCbC5 X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:c6e6:49bf:5c44:5965]) (user=mshavit job=sendgmr) by 2002:a81:440c:0:b0:56d:791:d1a4 with SMTP id r12-20020a81440c000000b0056d0791d1a4mr1051771ywa.7.1686757486567; Wed, 14 Jun 2023 08:44:46 -0700 (PDT) Date: Wed, 14 Jun 2023 23:42:02 +0800 In-Reply-To: <20230614154304.2860121-1-mshavit@google.com> Mime-Version: 1.0 References: <20230614154304.2860121-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230614154304.2860121-11-mshavit@google.com> Subject: [PATCH v3 10/13] iommu/arm-smmu-v3-sva: Remove bond refcount From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED, USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768696379633094739?= X-GMAIL-MSGID: =?utf-8?q?1768696379633094739?= The iommu-sva framework checks if a bond between a device and mm already exists and handles refcounting at the iommu_domain level. __arm_smmu_sva_bind is therefore only called once for a device/mm pair. Signed-off-by: Michael Shavit --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 12 +----------- 1 file changed, 1 insertion(+), 11 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index d07c08b53c5cf..20301d0a2c0b0 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -29,7 +29,6 @@ struct arm_smmu_bond { struct mm_struct *mm; struct arm_smmu_mmu_notifier *smmu_mn; struct list_head list; - refcount_t refs; }; #define sva_to_bond(handle) \ @@ -377,21 +376,12 @@ __arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) if (!master || !master->sva_enabled) return ERR_PTR(-ENODEV); - /* If bind() was already called for this {dev, mm} pair, reuse it. */ - list_for_each_entry(bond, &master->bonds, list) { - if (bond->mm == mm) { - refcount_inc(&bond->refs); - return &bond->sva; - } - } - bond = kzalloc(sizeof(*bond), GFP_KERNEL); if (!bond) return ERR_PTR(-ENOMEM); bond->mm = mm; bond->sva.dev = dev; - refcount_set(&bond->refs, 1); bond->smmu_mn = arm_smmu_mmu_notifier_get(smmu_domain, mm); if (IS_ERR(bond->smmu_mn)) { @@ -570,7 +560,7 @@ void arm_smmu_sva_remove_dev_pasid(struct iommu_domain *domain, } } - if (!WARN_ON(!bond) && refcount_dec_and_test(&bond->refs)) { + if (!WARN_ON(!bond)) { list_del(&bond->list); arm_smmu_mmu_notifier_put(bond->smmu_mn); kfree(bond); From patchwork Wed Jun 14 15:42:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 108023 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp61514vqr; Wed, 14 Jun 2023 09:11:54 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7U9gyIC8tvRG3BH1bqVTxlNVI5tCOJHbMI0WAm+lBzKFqDHX4TIJNT/8HD1YUrZb5j8Hu6 X-Received: by 2002:a05:6402:2683:b0:4ea:a9b0:a518 with SMTP id w3-20020a056402268300b004eaa9b0a518mr2218666edd.17.1686759114025; Wed, 14 Jun 2023 09:11:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686759114; cv=none; d=google.com; s=arc-20160816; b=Hdrd4WkOYswqEkL28ngSQ5xRfK8fK9p4gGMc6+rDPIjIdjnD62rl655C6pNVxp+dbw iXhosBKNOYH5mb67OD92ejuG6n612cKNYdDQ8vkwHCFxZ+eaW+OrnFv/2UF+cWqajBQ5 6gPqbMzycS9ZjMSmR8fgmdY6iwMPNXGg2Zipz1t5mokx+95Ob4nDwTnCZuBcfuf7uKEg lTfo38HlX/BVaSAu9m4V1TH11Pq7PDXzCWsU77EuBv3ygXgyIW4kV6yiA8CB2kE5OstB OTpml3u5Wop4702Hq4OyrEZq31oNj5HtoU9l3fzcyswFgyd7Z+9DshsfUHpvRhLV30EH HJkg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=1HNFofEgftQ/0viD8hP2C9StxNG8tEEj8VIgof4alao=; b=oXbiIhxzeWXaMeLYkSaP4OXxymp6wHsFyd5IuL19R08md1AdYiPvlAj6qQxXLXlGpx a7WmAKPnWtGM5R8J5mYeUw387dvk+NrKy20XP18KJcvk1ejw88OAtUg1eDnSjy1+UqzM EnkkVtnDcbk1OaklsoC2/OTxq2rRUevcMhN/5psN+u9L7et/cG0yQEGiyc/ThlZybBbn PbxtL47l1DVIco2YXik7kS6VRgC/W9AAjYIxLAOH29oarswXausrtcFdV7miW9dCxpzJ /2Lqol8oo1A5F8y2ra/gmQaj/OnR9sl4Qg5uAYhKokRt75KUG0cUHHyODj8tmOkEqciX aPew== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=miKKuVOq; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id g10-20020a056402114a00b0050bc5550f49si8796673edw.496.2023.06.14.09.11.05; Wed, 14 Jun 2023 09:11:54 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=miKKuVOq; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244410AbjFNPp7 (ORCPT + 99 others); Wed, 14 Jun 2023 11:45:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44282 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244874AbjFNPp2 (ORCPT ); Wed, 14 Jun 2023 11:45:28 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 879722116 for ; Wed, 14 Jun 2023 08:45:06 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-bacfa4ef059so981814276.2 for ; Wed, 14 Jun 2023 08:45:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686757494; x=1689349494; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=1HNFofEgftQ/0viD8hP2C9StxNG8tEEj8VIgof4alao=; b=miKKuVOqy3O5VFZtsBJw0DWAr4nJ6ea8nlIylue8vHaIsdYyKs5VFM0KTJacu51Qzm +DH3Zl9jsvtSJPHee4tayDVENGPTKWf9gWXrcL4hC1Sc2GgiVQBtqQ7FnZG8+YwqkBPh yA8Et/DjbQCMxnJvenLQiNf7A7GFBipkyxcRcbX25mPlSQAORH+f6lepCF7STCNnbSjz QEVoAOVVQy/KvCuZreeHmBUtlj6a0/QT8lJ7Ls6W5a6m4+Xc6bUdeRYUvj91Jh2RQGIL 1UcOZtIRMZQKKjjJou6pQavpNlTolKJ9ovsaIENZITg6nYLeQ/1+rErxMkqfwI2pqIqu 2vrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686757494; x=1689349494; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=1HNFofEgftQ/0viD8hP2C9StxNG8tEEj8VIgof4alao=; b=X/baVtJN6J3qkZ1B9jJOzs1e1ngiG9GC2EfXoty22uR+3PABZQSsc1Dwi5sVXc75Pt ikonkDPI6mxQtOGviCxEnvcJRKfLhWIvob4v6LfiTOUNDvG2F9s3XHoqogEpsEdWwetU XtzQxBjxQAxRCHX5Yt2D8mCXzxYt14xbDe9TGStuXTVbfNHDNz3PASKWioWxwv0hWS99 L1BH83dpjTBAg2BOaIV8XiE7v0JzQus0+gTIdF8TtLxkQde9zGf6peDxyIptBvTYic8a Nz4QKoV+DPrKc3TCYKIfDmCh5+aDVhfe5R4ftG4SmkR0FpD6PkUfImDVeIiBv/iWNdb7 0VQw== X-Gm-Message-State: AC+VfDzb3Zw0ECOcTiQsQfTTqhQOBI/P09JFfbN7tT2ZrMIPCBvzEATs jYVI8DuQTxMCgdYSrL3WdBjzj5d5fnJZ X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:c6e6:49bf:5c44:5965]) (user=mshavit job=sendgmr) by 2002:a25:7356:0:b0:bc7:f6af:8cff with SMTP id o83-20020a257356000000b00bc7f6af8cffmr1176382ybc.2.1686757494488; Wed, 14 Jun 2023 08:44:54 -0700 (PDT) Date: Wed, 14 Jun 2023 23:42:03 +0800 In-Reply-To: <20230614154304.2860121-1-mshavit@google.com> Mime-Version: 1.0 References: <20230614154304.2860121-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230614154304.2860121-12-mshavit@google.com> Subject: [PATCH v3 11/13] iommu/arm-smmu-v3-sva: Clean unused iommu_sva From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED, USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768695124493469167?= X-GMAIL-MSGID: =?utf-8?q?1768695124493469167?= The __arm_smmu_sva_bind function returned an unused iommu_sva handle that can be removed. Signed-off-by: Michael Shavit --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 18 ++++++------------ 1 file changed, 6 insertions(+), 12 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 20301d0a2c0b0..650c9c9ad52f1 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -25,7 +25,6 @@ struct arm_smmu_mmu_notifier { #define mn_to_smmu(mn) container_of(mn, struct arm_smmu_mmu_notifier, mn) struct arm_smmu_bond { - struct iommu_sva sva; struct mm_struct *mm; struct arm_smmu_mmu_notifier *smmu_mn; struct list_head list; @@ -364,8 +363,7 @@ static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) arm_smmu_free_shared_cd(cd); } -static struct iommu_sva * -__arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) +static int __arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) { int ret; struct arm_smmu_bond *bond; @@ -374,14 +372,13 @@ __arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); if (!master || !master->sva_enabled) - return ERR_PTR(-ENODEV); + return -ENODEV; bond = kzalloc(sizeof(*bond), GFP_KERNEL); if (!bond) - return ERR_PTR(-ENOMEM); + return -ENOMEM; bond->mm = mm; - bond->sva.dev = dev; bond->smmu_mn = arm_smmu_mmu_notifier_get(smmu_domain, mm); if (IS_ERR(bond->smmu_mn)) { @@ -390,11 +387,11 @@ __arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) } list_add(&bond->list, &master->bonds); - return &bond->sva; + return 0; err_free_bond: kfree(bond); - return ERR_PTR(ret); + return ret; } bool arm_smmu_sva_supported(struct arm_smmu_device *smmu) @@ -572,13 +569,10 @@ static int arm_smmu_sva_set_dev_pasid(struct iommu_domain *domain, struct device *dev, ioasid_t id) { int ret = 0; - struct iommu_sva *handle; struct mm_struct *mm = domain->mm; mutex_lock(&sva_lock); - handle = __arm_smmu_sva_bind(dev, mm); - if (IS_ERR(handle)) - ret = PTR_ERR(handle); + ret = __arm_smmu_sva_bind(dev, mm); mutex_unlock(&sva_lock); return ret; From patchwork Wed Jun 14 15:42:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 108024 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp64229vqr; Wed, 14 Jun 2023 09:15:35 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5mdxJ009fxLLXHPLqxF6uNh7usgcQ8355ez9W/2ITKJlLpGFZ1dCdNDCDnTPX3AGl75hB9 X-Received: by 2002:a17:906:4788:b0:977:4b19:97a with SMTP id cw8-20020a170906478800b009774b19097amr16763878ejc.73.1686759334921; Wed, 14 Jun 2023 09:15:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686759334; cv=none; d=google.com; s=arc-20160816; b=S/W8E0opEo/aStUIs3pggcq47hEUQkuwLT7jA82lsN6KqHkUCX2BijZ7mZYfRGuZcY mDvgEZSQfPWR6D0kfmj8BDT7RvDAsqA4ZQTtgy4fsTjVMS+CyO+z8dD7lvb22KI37Lmx NWNOlSuWMZf0Cqahahsh283C5dt/pTiO+kOIfniSUW7sr3LxjuBNp1NiaQaitBBA8gXj 75PIxMU2AYronf8yNNCjLI+K2/HH14uLvhhQRhkPEn6sIBrDQaEKxABDCv54ePoMMAgm RTtgH/oy9IYq/eLd6jY9jfXQqtoGqkaN8ffwYgvs0aLDcIo5XoH8SKtEn6OLLAGIxwl3 TP6g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=rIYz1+kUM82lpQwp18pWI6n2YGQKaG9udPtQlpxJvZA=; b=PaJPZaeQFQKENa15LLYiK2SXs2GNZNiAsO7C2DiAocc+DvvX/butr625EF0BqMoMZe oypVHXS554MOXuuSn9CaQ9t2o8J0CH0kPMIEHS23TnEIr2ezkaoNHS1FE3MNd1lh/yk+ z44tHbCEgMH2LeYMGAoMO+LXPyGdWcwFuYNMtTpL8sJeQmTrRi3mN/Sipl0WgyWuaBtY D+Fqtnar0QJQWlnsywNSZJL/yKrup/PBLlu+4AOx0nHIg7VjqO9doMy9jNm0gtdGx27W SFvBIfdLkDYPeP+Gbza4htlstoShcV4HH9MhhJVeCmnUqa4wWqpPk+46q7Q/qLGAoCpP kH7g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=ITLTLTaL; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id sd12-20020a170906ce2c00b00977eb8d57afsi8122393ejb.821.2023.06.14.09.15.09; Wed, 14 Jun 2023 09:15:34 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=ITLTLTaL; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244955AbjFNPpz (ORCPT + 99 others); Wed, 14 Jun 2023 11:45:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44290 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244879AbjFNPp2 (ORCPT ); Wed, 14 Jun 2023 11:45:28 -0400 Received: from mail-yw1-x114a.google.com (mail-yw1-x114a.google.com [IPv6:2607:f8b0:4864:20::114a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CE546273A for ; Wed, 14 Jun 2023 08:45:06 -0700 (PDT) Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-568a85f180dso9968977b3.2 for ; Wed, 14 Jun 2023 08:45:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686757502; x=1689349502; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=rIYz1+kUM82lpQwp18pWI6n2YGQKaG9udPtQlpxJvZA=; b=ITLTLTaLCsc9HGbm9o39sZh1FY+PCkOK3DqheGkhRPBC/5m4sMot8zJeqp6fvfWE1r t3T//BbToTEl9NchoFS4gdxcOle/PsfltFonLoaAgA2DAOWZERwJ7kzdxMSY2fW6ZZ8y pY+tDn0S1BbfL62uKyPVDPJuYg7OXqEm7XeJ9+DUWbpMulT67bfgJEo9dgHiDVDe4FGN rscahF06lVNVc5DgWu9mjCNiD2g7elxaGjWG9gVInQls6qTkbGhdTgpJbQc97OTMvGT9 oNdhtFouxQm4yOzmlFFhwPGSPEOZr2p+ZrgnJzHBF4mAXsbdtpPsvzu8r/CxQgvc5R+4 6r/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686757502; x=1689349502; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=rIYz1+kUM82lpQwp18pWI6n2YGQKaG9udPtQlpxJvZA=; b=L/v9j2JPBpHPixDE4vZwY47A2QK1HJ2TN9b9cJ/ZN7jqtq87aIl8MOSykUuFxzK9D0 ABIy5VqPp9ZUapYWin9deaxW49xn3s6k586ZmD/Es8FFGTzZlRTO/674UdoPglpwZfwo DQrPUXWLddEmtPEGJP5jJbDLaTwTG/RqAs3ZL4a7mOjbXhZNLBLJCat7Cl4/W9El2QO3 rsP2GlngxVfXcYhG41Qn8Pl6IB4jSq+2cYZVN/xFYU5FL0F9XSi1aZdVowHcCS2VDr6g PsWpbcQtNlbxhKSBr+ptySWDQMUi+nFvpTlmIJaZFRUNzjkwATIbC6DxnsrLL1ashh/O DcmQ== X-Gm-Message-State: AC+VfDyEV667TvHm65rQ4Nv9HmyDA9R6kTH9M+z+jdg8yzBTeJuTTMf7 Q5QV1PDsm988z4kbubYM0fpQzvJwNCB9 X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:c6e6:49bf:5c44:5965]) (user=mshavit job=sendgmr) by 2002:a25:ab85:0:b0:ba8:4ba3:5b54 with SMTP id v5-20020a25ab85000000b00ba84ba35b54mr311468ybi.11.1686757502482; Wed, 14 Jun 2023 08:45:02 -0700 (PDT) Date: Wed, 14 Jun 2023 23:42:04 +0800 In-Reply-To: <20230614154304.2860121-1-mshavit@google.com> Mime-Version: 1.0 References: <20230614154304.2860121-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230614154304.2860121-13-mshavit@google.com> Subject: [PATCH v3 12/13] iommu/arm-smmu-v3-sva: Remove arm_smmu_bond From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768695355990362535?= X-GMAIL-MSGID: =?utf-8?q?1768695355990362535?= There's a 1:1 relationship between arm_smmu_bond and the iommu_domain used in set_dev_pasid/remove_dev_pasid. arm_smmu_bond has become an unnecessary complication. It's more natural to store any needed information at the iommu_domain container level. Signed-off-by: Michael Shavit --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 69 +++++++------------ drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 1 - drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 2 +- 3 files changed, 24 insertions(+), 48 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 650c9c9ad52f1..b615a85e6a54e 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -24,14 +24,13 @@ struct arm_smmu_mmu_notifier { #define mn_to_smmu(mn) container_of(mn, struct arm_smmu_mmu_notifier, mn) -struct arm_smmu_bond { - struct mm_struct *mm; +struct arm_smmu_sva_domain { + struct iommu_domain iommu_domain; struct arm_smmu_mmu_notifier *smmu_mn; - struct list_head list; }; -#define sva_to_bond(handle) \ - container_of(handle, struct arm_smmu_bond, sva) +#define to_sva_domain(domain) \ + container_of(domain, struct arm_smmu_sva_domain, iommu_domain) static DEFINE_MUTEX(sva_lock); @@ -363,10 +362,10 @@ static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) arm_smmu_free_shared_cd(cd); } -static int __arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) +static int __arm_smmu_sva_bind(struct device *dev, + struct arm_smmu_sva_domain *sva_domain, + struct mm_struct *mm) { - int ret; - struct arm_smmu_bond *bond; struct arm_smmu_master *master = dev_iommu_priv_get(dev); struct iommu_domain *domain = iommu_get_domain_for_dev(dev); struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); @@ -374,24 +373,14 @@ static int __arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) if (!master || !master->sva_enabled) return -ENODEV; - bond = kzalloc(sizeof(*bond), GFP_KERNEL); - if (!bond) - return -ENOMEM; - - bond->mm = mm; - - bond->smmu_mn = arm_smmu_mmu_notifier_get(smmu_domain, mm); - if (IS_ERR(bond->smmu_mn)) { - ret = PTR_ERR(bond->smmu_mn); - goto err_free_bond; + sva_domain->smmu_mn = arm_smmu_mmu_notifier_get(smmu_domain, + mm); + if (IS_ERR(sva_domain->smmu_mn)) { + sva_domain->smmu_mn = NULL; + return PTR_ERR(sva_domain->smmu_mn); } - - list_add(&bond->list, &master->bonds); + master->nr_attached_sva_domains += 1; return 0; - -err_free_bond: - kfree(bond); - return ret; } bool arm_smmu_sva_supported(struct arm_smmu_device *smmu) @@ -521,7 +510,7 @@ int arm_smmu_master_enable_sva(struct arm_smmu_master *master) int arm_smmu_master_disable_sva(struct arm_smmu_master *master) { mutex_lock(&sva_lock); - if (!list_empty(&master->bonds)) { + if (master->nr_attached_sva_domains != 0) { dev_err(master->dev, "cannot disable SVA, device is bound\n"); mutex_unlock(&sva_lock); return -EBUSY; @@ -545,23 +534,12 @@ void arm_smmu_sva_notifier_synchronize(void) void arm_smmu_sva_remove_dev_pasid(struct iommu_domain *domain, struct device *dev, ioasid_t id) { - struct mm_struct *mm = domain->mm; - struct arm_smmu_bond *bond = NULL, *t; + struct arm_smmu_sva_domain *sva_domain = to_sva_domain(domain); struct arm_smmu_master *master = dev_iommu_priv_get(dev); mutex_lock(&sva_lock); - list_for_each_entry(t, &master->bonds, list) { - if (t->mm == mm) { - bond = t; - break; - } - } - - if (!WARN_ON(!bond)) { - list_del(&bond->list); - arm_smmu_mmu_notifier_put(bond->smmu_mn); - kfree(bond); - } + master->nr_attached_sva_domains -= 1; + arm_smmu_mmu_notifier_put(sva_domain->smmu_mn); mutex_unlock(&sva_lock); } @@ -572,7 +550,7 @@ static int arm_smmu_sva_set_dev_pasid(struct iommu_domain *domain, struct mm_struct *mm = domain->mm; mutex_lock(&sva_lock); - ret = __arm_smmu_sva_bind(dev, mm); + ret = __arm_smmu_sva_bind(dev, to_sva_domain(domain), mm); mutex_unlock(&sva_lock); return ret; @@ -590,12 +568,11 @@ static const struct iommu_domain_ops arm_smmu_sva_domain_ops = { struct iommu_domain *arm_smmu_sva_domain_alloc(void) { - struct iommu_domain *domain; + struct arm_smmu_sva_domain *sva_domain; - domain = kzalloc(sizeof(*domain), GFP_KERNEL); - if (!domain) + sva_domain = kzalloc(sizeof(*sva_domain), GFP_KERNEL); + if (!sva_domain) return NULL; - domain->ops = &arm_smmu_sva_domain_ops; - - return domain; + sva_domain->iommu_domain.ops = &arm_smmu_sva_domain_ops; + return &sva_domain->iommu_domain; } diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index a6fa56585c219..b7f834dde85d1 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2784,7 +2784,6 @@ static struct iommu_device *arm_smmu_probe_device(struct device *dev) master->dev = dev; master->smmu = smmu; - INIT_LIST_HEAD(&master->bonds); dev_iommu_priv_set(dev, master); ret = arm_smmu_insert_master(smmu, master); diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 48795a7287b69..3525d60668c23 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -705,7 +705,7 @@ struct arm_smmu_master { bool stall_enabled; bool sva_enabled; bool iopf_enabled; - struct list_head bonds; + unsigned int nr_attached_sva_domains; unsigned int ssid_bits; unsigned int nr_attached_pasid_domains; }; From patchwork Wed Jun 14 15:42:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 108034 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp75327vqr; Wed, 14 Jun 2023 09:30:53 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7HoJjlVahz36coZ5yEQL95t29MYDA6BYUJoT73E8JIBeIZ8ongOzkRsHFz/wWmTg2MV6NP X-Received: by 2002:a05:6a20:12cf:b0:10e:bd13:f929 with SMTP id v15-20020a056a2012cf00b0010ebd13f929mr2279040pzg.1.1686760253303; Wed, 14 Jun 2023 09:30:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686760253; cv=none; d=google.com; s=arc-20160816; b=ecttxDrs3utcutw6r8F4zL3LpdBJSBEso1x1GfsT+RNuicmGydqvke5BKP/FGpb3Jc SjxEVMEeF+scyzN+/CBtk5ft4DS6h56kriaFwaLVZKfglugdFKIUFIFoc/q4i8TSfvnG 3vmZbdc09rVHMtGNZ2RWTWWWyp8qYIs+xpVZ4mGazRA4e3DpbdFZUVqq/w57cIjgkrAb WGzTJeGsmBlm3zki8fNT5HkJnze+sO2uC+52cvRsDkrDYNw7nvVevM6LukQhNAoTlTC5 dScPs/nvyKApn/BZlE+l8f6+z642SdMr9/fyLeXpIwiM7bpAbZuxT2uon8GUQhW5DvSw ZTvg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=GFfBSOUvUPqFSx6n4vEtFQq3Hc8Ha+kDvCzi3+3mgos=; b=GJfKlJzultPcvsMInE2SuaQtSYz7ZZSC4O4l4TTQ9vdYfPmPQxNBFnpK3SPcu5Fkox SencHUqmHp/Dnds6pxhhWgjznzWBV0hhIku0lKLXBGLmPbaiMe5oxihMvp+A7Vvfyz9L wsD3Nbj/6KeR2+dIq6l/4cAtGNhd7yxlBrNQcqpuD+MfM6Js1QNnKFkw+PwDg6OcDj6p nyczvPl+zXCrKUfAu1NrYFFqZ9h/Y6IYttbdcQSrg8qTPkjiKAsWuEq8eb+wrCR1OB5J dDGoLzEcp7lD/KXs71+FaphinpdVxwUR0zLTVIWtDoldrRQ3zgNMxFB2d1ZMNnFc7HSX jeEg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=vp6Nlsbz; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id h12-20020aa79f4c000000b006439d34a090si11139333pfr.238.2023.06.14.09.30.37; Wed, 14 Jun 2023 09:30:53 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=vp6Nlsbz; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244971AbjFNPqC (ORCPT + 99 others); Wed, 14 Jun 2023 11:46:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44356 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244938AbjFNPpb (ORCPT ); Wed, 14 Jun 2023 11:45:31 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 168BB2683 for ; Wed, 14 Jun 2023 08:45:11 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-bc7267224adso1730537276.0 for ; Wed, 14 Jun 2023 08:45:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686757510; x=1689349510; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=GFfBSOUvUPqFSx6n4vEtFQq3Hc8Ha+kDvCzi3+3mgos=; b=vp6Nlsbz0oC9g4ftIwLdhomK6sQnklY1ZxeEkHnIXfEXrOB32EsbD/Sxec0iRhhQHg FG7D8M3PpWeaM8iqTIew0USWOhnUT0FjBe9eXnTTSV/LhmO3ZmKELA9cij4LrrquyyEX xN4kZsa+CrEDpQ+9SCj337Iz/uwRx2q05yM0XfIncqMYOokbXzbVERwKppl5BO38z5eP hHmqL5KhrByMtgmNr4kVUihXIw6qeRCfAOy3kJEM2Xe0aOvPnzQXKdDIUXJE6sDAuRHp s5nPeYy80N984gbWZ3p5ANSaT8JevwNNBkebJ7ua9ynBAmNYrnU+pSstx30pek9GSY+/ b8wg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686757510; x=1689349510; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=GFfBSOUvUPqFSx6n4vEtFQq3Hc8Ha+kDvCzi3+3mgos=; b=JyN8genmUSaSv4q9wszvI207vt3KSvlZi3CYd8+pKAJfJHcT+NM5ED+MiOt0XV/jZA +jlrOPSz72ddh6Nk6J4biVsW3Wk2Cocu3AmeEwvGKWXsT4Kwyfua6phHh1g8gRXjo793 UWFxVoWgJEttps+G0B7Wz6lpBP3mZpQQSGw0I1zLmW6igGoeUBBFVRyNAoW12KsSD3TA BI62yqmJ2hu/NdAwcPmKkjFdeMpCIWJHvCBRIAEj5DFoftef+LMNXdn7HTcjkyeJltp0 CWDk1vf8Gn/Q58Qj2Wf6Z6vNdjylyb9yxZG+E8tLkQ4Umjx4pD0QrFfcNKpFCKwZ0JWq wHGg== X-Gm-Message-State: AC+VfDyxZBqag7BhPSVm7r7zZ1mh8r6V9c0VJS9WT1iQhXWIs/n6nUGs wCsKFn4Kjg1ge95Z9yuNCn8hh0at7lWw X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:c6e6:49bf:5c44:5965]) (user=mshavit job=sendgmr) by 2002:a05:6902:1185:b0:ba8:381b:f764 with SMTP id m5-20020a056902118500b00ba8381bf764mr888385ybu.3.1686757510234; Wed, 14 Jun 2023 08:45:10 -0700 (PDT) Date: Wed, 14 Jun 2023 23:42:05 +0800 In-Reply-To: <20230614154304.2860121-1-mshavit@google.com> Mime-Version: 1.0 References: <20230614154304.2860121-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230614154304.2860121-14-mshavit@google.com> Subject: [PATCH v3 13/13] iommu/arm-smmu-v3-sva: Add check when enabling sva From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768696319476826258?= X-GMAIL-MSGID: =?utf-8?q?1768696319476826258?= SVA domains can only be attached when the master's STEs have a stage 1 domain. Signed-off-by: Michael Shavit --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index b615a85e6a54e..e2a91f20f0906 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -499,9 +499,15 @@ int arm_smmu_master_enable_sva(struct arm_smmu_master *master) int ret; mutex_lock(&sva_lock); + + if (!master->s1_cfg) { + ret = -EBUSY; + goto unlock; + } ret = arm_smmu_master_sva_enable_iopf(master); if (!ret) master->sva_enabled = true; +unlock: mutex_unlock(&sva_lock); return ret;