From patchwork Thu Jul 27 18:26:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 127110 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:a985:0:b0:3e4:2afc:c1 with SMTP id t5csp1296765vqo; Thu, 27 Jul 2023 11:48:14 -0700 (PDT) X-Google-Smtp-Source: APBJJlEGoHrTsM162c1y8J4KXS+SY8JSzYbMR5f1PxgRBJAK/0gMMIAavVNi/TVyRCuBlGf+wSIY X-Received: by 2002:a17:907:7e94:b0:96a:2dd7:2ee0 with SMTP id qb20-20020a1709077e9400b0096a2dd72ee0mr3600248ejc.5.1690483693866; Thu, 27 Jul 2023 11:48:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690483693; cv=none; d=google.com; s=arc-20160816; b=GYqPKS5MKx1d4rYKq+iCr8SLs3hnFEtRaVlMeWoG8HUUO+8JkLh7bHkceaNhZzNdp7 XfYK3xWbpPJ72qbeT8tpldOrOPhCFGoicaMy13L2RgGDBB44wHo5bF9Z5X6KlGX0L3BW ca4+Qf4mU61koBfOJ58keHduo73FDwCOdSpCwwS3HyfR6YKfyGB1YqGEFH1bU7bqNTTh hhUjaWO43Epg4rY4dpCLaNJiRGS5EVv9vCHsME1zL6JecqkVUJjsdpFCdsDDJT/A4WPM cIYZ8vBVTxH04Xw3L4mCF7xtFnNnPLnDH6HmlX+977xU+UzyWij8ipoKz3r4ZlkSdNaa iaeA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=fYD6rkTLiSYQl+qzaa/e799G5U/EZRM1YfnI2GTzV2A=; fh=HnAFHWFlJx6556zbVk+xHReF/Bq7K9G33irMhQzl/Ec=; b=tpwN4SXkhPkOrM/l/EoZp0Hw3dNhAKLCr6gC1BaSKp2jxnLK6WnNimyt3jGG8vfKKO UfNdu1KTGgwv2Wb0ldgqgBnSDberovgGvRiw44cUicDrkgkQtdCwFzCCkfiHbokFLUr/ 7IaGVKM3XEscsNb4Wfk6h1PaTGw6YapLyhuLKABeAUbDbcQrY+YvjX9+VmunMN6o5D6A XpO15ZGuhq0xvAuM3FCa2NKl52HtzMixrAOni1b4Hs4wjph2G/05buan3PXFsVi/J3uq 3Wx6yPWPtGN3K5ey40uv6aQRyNsIh0ASSwGfgmRgRv/3SLgNMbK+ceg60ccArpbiI8fv dnSg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=sLjnB0sP; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id j13-20020a17090686cd00b0099b930da46fsi1364618ejy.99.2023.07.27.11.47.47; Thu, 27 Jul 2023 11:48:13 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=sLjnB0sP; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231569AbjG0S3V (ORCPT + 99 others); Thu, 27 Jul 2023 14:29:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56086 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231535AbjG0S3N (ORCPT ); Thu, 27 Jul 2023 14:29:13 -0400 Received: from mail-yw1-x114a.google.com (mail-yw1-x114a.google.com [IPv6:2607:f8b0:4864:20::114a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C52713A97 for ; Thu, 27 Jul 2023 11:28:51 -0700 (PDT) Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-56cf9a86277so12120577b3.3 for ; Thu, 27 Jul 2023 11:28:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1690482530; x=1691087330; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=fYD6rkTLiSYQl+qzaa/e799G5U/EZRM1YfnI2GTzV2A=; b=sLjnB0sPTngxPiNbukoCIlg09L30sRsZuMxmeBGW+yyF3CXStNtXFcXwJB+xTqjRSv RAFR0PsBB9oxZZ1fXPP5gF5yfM21wlPFwklW6KezALjohicrNsTNsVx9Gf7/8soKq5JA OkkuGVmv19Q3wy1mdJEgtAoBAlSN1mHkfnOIqHpFQAlTXstcMK+Zv4LmS6K4yC2kDhr+ B3v56N560Ut8/P+1nX9Do+57ytbqaOgz5cbk4wkiYlEyoOUZ7H1uuXwQKOoKilB/D+PQ 5i9yb0kCMmyb6Xr7DyTUC1IKTO5c19lhzEww4dU3WHeTeNorlN++k6oHnwQVI/CfLoI/ Cxlw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690482530; x=1691087330; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=fYD6rkTLiSYQl+qzaa/e799G5U/EZRM1YfnI2GTzV2A=; b=ZptDsRB2zHOPT1N/KC+2U7+QTUennoM4jcr4AnGeDzAsgb1vSBFE5yjgw8OTe4HmeA 06bJSkBgDrarqCTpjAIXxnpQfSz06KGywiTxuhcMU0D70Bo6Q/36GYQcW8TR/VoNMcuS jUi5msvzRS5k1V/ONs4aFG6RwIYNKa2Dz7m5kWX8MeHIRH0N8OQfRrcJKacLvlipoSb4 e3GDypvEFVX80fp5yTW022p7caya++LgrwMwaj78S9LOXWGyOZG4BcnHTTxNNku8UNAE 8H1OMin5QD1bE2CeJYwnBUxHbqtBuDyEBS4cUnpdSc1xL1GqcecOZ6aEFxBApm09fwr/ w37w== X-Gm-Message-State: ABy/qLa3BV3/cjPk1kelZPyw2EfUyAIZ1etw34FI5v5mLSBhTtVmzYHE 9UVuWNlt3yVknDJ6K6vHswleFxfA2cxK X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:780f:26da:d952:3463]) (user=mshavit job=sendgmr) by 2002:a5b:743:0:b0:c4d:9831:9712 with SMTP id s3-20020a5b0743000000b00c4d98319712mr1200ybq.0.1690482530704; Thu, 27 Jul 2023 11:28:50 -0700 (PDT) Date: Fri, 28 Jul 2023 02:26:23 +0800 In-Reply-To: <20230727182647.4106140-1-mshavit@google.com> Mime-Version: 1.0 References: <20230727182647.4106140-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.585.gd2178a4bd4-goog Message-ID: <20230727182647.4106140-8-mshavit@google.com> Subject: [PATCH v1 7/7] iommu/arm-smmu-v3: Move CD table to arm_smmu_master From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1772600629983890106 X-GMAIL-MSGID: 1772600629983890106 Each master is now allocated a CD table at probe time, and attaching a stage 1 domain now installs CD entries into the master's CD table. SVA writes its CD entries into each master's CD table if the domain is shared across masters. Signed-off-by: Michael Shavit --- v4->v5: The master's CD table allocation was previously split to a different commit. This change now atomically allocates the new CD table, uses it, and removes the old one. --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 65 +++++++++------------ drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 4 +- 2 files changed, 28 insertions(+), 41 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index b211424a85fb2..a58a0f811d531 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -985,7 +985,7 @@ static void arm_smmu_sync_cd(struct arm_smmu_master *master, }, }; - if (!master->domain->cd_table.installed) + if (!master->cd_table.installed) return; smmu = master->smmu; @@ -1028,7 +1028,7 @@ static __le64 *arm_smmu_get_cd_ptr(struct arm_smmu_master *master, u32 ssid) __le64 *l1ptr; unsigned int idx; struct arm_smmu_l1_ctx_desc *l1_desc; - struct arm_smmu_ctx_desc_cfg *cdcfg = &master->domain->cd_table; + struct arm_smmu_ctx_desc_cfg *cdcfg = &master->cd_table; if (!cdcfg->l1_desc) return cdcfg->cdtab + ssid * CTXDESC_CD_DWORDS; @@ -1065,7 +1065,7 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_master *master, int ssid, u64 val; bool cd_live; __le64 *cdptr; - struct arm_smmu_ctx_desc_cfg *cd_table = &master->domain->cd_table; + struct arm_smmu_ctx_desc_cfg *cd_table = &master->cd_table; if (WARN_ON(ssid >= (1 << cd_table->max_cds_bits))) return -E2BIG; @@ -1128,14 +1128,13 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_master *master, int ssid, return 0; } -static int arm_smmu_alloc_cd_tables(struct arm_smmu_domain *smmu_domain, - struct arm_smmu_master *master) +static int arm_smmu_alloc_cd_tables(struct arm_smmu_master *master) { int ret; size_t l1size; size_t max_contexts; struct arm_smmu_device *smmu = master->smmu; - struct arm_smmu_ctx_desc_cfg *cdcfg = &smmu_domain->cd_table; + struct arm_smmu_ctx_desc_cfg *cdcfg = &master->cd_table; cdcfg->stall_enabled = master->stall_enabled; cdcfg->max_cds_bits = master->ssid_bits; @@ -1177,12 +1176,12 @@ static int arm_smmu_alloc_cd_tables(struct arm_smmu_domain *smmu_domain, return ret; } -static void arm_smmu_free_cd_tables(struct arm_smmu_domain *smmu_domain) +static void arm_smmu_free_cd_tables(struct arm_smmu_master *master) { int i; size_t size, l1size; - struct arm_smmu_device *smmu = smmu_domain->smmu; - struct arm_smmu_ctx_desc_cfg *cdcfg = &smmu_domain->cd_table; + struct arm_smmu_device *smmu = master->smmu; + struct arm_smmu_ctx_desc_cfg *cdcfg = &master->cd_table; if (cdcfg->l1_desc) { size = CTXDESC_L2_ENTRIES * (CTXDESC_CD_DWORDS << 3); @@ -1290,7 +1289,7 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, if (smmu_domain) { switch (smmu_domain->stage) { case ARM_SMMU_DOMAIN_S1: - cd_table = &smmu_domain->cd_table; + cd_table = &master->cd_table; break; case ARM_SMMU_DOMAIN_S2: case ARM_SMMU_DOMAIN_NESTED: @@ -2081,14 +2080,10 @@ static void arm_smmu_domain_free(struct iommu_domain *domain) free_io_pgtable_ops(smmu_domain->pgtbl_ops); - /* Free the CD and ASID, if we allocated them */ + /* Free the ASID or VMID */ if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { - struct arm_smmu_ctx_desc_cfg *cdcfg = &smmu_domain->cd_table; - /* Prevent SVA from touching the CD while we're freeing it */ mutex_lock(&arm_smmu_asid_lock); - if (cdcfg->cdtab) - arm_smmu_free_cd_tables(smmu_domain); arm_smmu_free_asid(&smmu_domain->cd); mutex_unlock(&arm_smmu_asid_lock); } else { @@ -2100,7 +2095,7 @@ static void arm_smmu_domain_free(struct iommu_domain *domain) kfree(smmu_domain); } -static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, +static int arm_smmu_domain_finalise_cd(struct arm_smmu_domain *smmu_domain, struct arm_smmu_master *master, struct io_pgtable_cfg *pgtbl_cfg) { @@ -2119,10 +2114,6 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, if (ret) goto out_unlock; - ret = arm_smmu_alloc_cd_tables(smmu_domain, master); - if (ret) - goto out_free_asid; - cd->asid = (u16)asid; cd->ttbr = pgtbl_cfg->arm_lpae_s1_cfg.ttbr; cd->tcr = FIELD_PREP(CTXDESC_CD_0_TCR_T0SZ, tcr->tsz) | @@ -2134,17 +2125,9 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, CTXDESC_CD_0_TCR_EPD1 | CTXDESC_CD_0_AA64; cd->mair = pgtbl_cfg->arm_lpae_s1_cfg.mair; - ret = arm_smmu_write_ctx_desc(master, 0, cd); - if (ret) - goto out_free_cd_tables; - mutex_unlock(&arm_smmu_asid_lock); return 0; -out_free_cd_tables: - arm_smmu_free_cd_tables(smmu_domain); -out_free_asid: - arm_smmu_free_asid(cd); out_unlock: mutex_unlock(&arm_smmu_asid_lock); return ret; @@ -2207,7 +2190,7 @@ static int arm_smmu_domain_finalise(struct iommu_domain *domain, ias = min_t(unsigned long, ias, VA_BITS); oas = smmu->ias; fmt = ARM_64_LPAE_S1; - finalise_stage_fn = arm_smmu_domain_finalise_s1; + finalise_stage_fn = arm_smmu_domain_finalise_cd; break; case ARM_SMMU_DOMAIN_NESTED: case ARM_SMMU_DOMAIN_S2: @@ -2447,16 +2430,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) } else if (smmu_domain->smmu != smmu) { ret = -EINVAL; goto out_unlock; - } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1 && - master->ssid_bits != smmu_domain->cd_table.max_cds_bits) { - ret = -EINVAL; - goto out_unlock; - } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1 && - smmu_domain->cd_table.stall_enabled != master->stall_enabled) { - ret = -EINVAL; - goto out_unlock; } - master->domain = smmu_domain; /* @@ -2469,6 +2443,14 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) if (smmu_domain->stage != ARM_SMMU_DOMAIN_BYPASS) master->ats_enabled = arm_smmu_ats_supported(master); + if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { + ret = arm_smmu_write_ctx_desc(master, 0, &smmu_domain->cd); + if (ret) { + master->domain = NULL; + goto out_unlock; + } + } + arm_smmu_install_ste_for_dev(master); spin_lock_irqsave(&smmu_domain->devices_lock, flags); @@ -2706,6 +2688,12 @@ static struct iommu_device *arm_smmu_probe_device(struct device *dev) smmu->features & ARM_SMMU_FEAT_STALL_FORCE) master->stall_enabled = true; + ret = arm_smmu_alloc_cd_tables(master); + if (ret) { + arm_smmu_disable_pasid(master); + arm_smmu_remove_master(master); + goto err_free_master; + } return &smmu->iommu; err_free_master: @@ -2723,6 +2711,7 @@ static void arm_smmu_release_device(struct device *dev) arm_smmu_detach_dev(master); arm_smmu_disable_pasid(master); arm_smmu_remove_master(master); + arm_smmu_free_cd_tables(master); kfree(master); } diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index a8cc2de0cc254..ebd082b552699 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -696,6 +696,7 @@ struct arm_smmu_master { struct arm_smmu_domain *domain; struct list_head domain_head; struct arm_smmu_stream *streams; + struct arm_smmu_ctx_desc_cfg cd_table; unsigned int num_streams; bool ats_enabled; bool stall_enabled; @@ -722,10 +723,7 @@ struct arm_smmu_domain { enum arm_smmu_domain_stage stage; union { - struct { struct arm_smmu_ctx_desc cd; - struct arm_smmu_ctx_desc_cfg cd_table; - }; struct arm_smmu_s2_cfg s2_cfg; };