From patchwork Wed Jun 21 06:37:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 110823 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp4173738vqr; Wed, 21 Jun 2023 00:09:50 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7A1AEucUwHFgcjgUN3Kj5Q3h9hgrxuIXSFRosfWVi0hLERGHfkYD0OQINIOb0Vcb/5JKZs X-Received: by 2002:a17:902:eacb:b0:1b3:fb76:215b with SMTP id p11-20020a170902eacb00b001b3fb76215bmr10588854pld.48.1687331389513; Wed, 21 Jun 2023 00:09:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1687331389; cv=none; d=google.com; s=arc-20160816; b=hQW+TvXo0N/EbZM5IX0X6DX40VtgNi5u8Qv5o8n6BJr2Q2t1h2C+kTHvmR4DSDUVZe 7qml7jW7SwgioiQSSs2ufD5PLC7Z6PVarTNh7w4ItBDdZVj8yK2wJHNDh0NFAnLGhNV4 bcJ0s/umayDggtywCE2Yn4sB7/Jyr/oBLsL5TgJtuee+espi003F2A6G3MPzzcWfzuih xs3Hoc6hq4Mm9pBwDgmelaviXDxctJ2shpU337RVwn5pDNOKQpIHFmBDhTO7Gt48tE7Z ujjLh7YZJmi25Hhokm0gZqOWSgGoLsqShAwCeZyw1GLaZB0ORZJGpU8/nog20SoFE9m/ uEOw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=xwDD9pVjLa+TJXAJEbbYyI+224KEPbc+a+2w73kHUiI=; b=v7CVbn7yphjgW1ZqFCTLxGMiuS7f6/NUj8uCxsRyiSu45K0bg5MBWZfSdUnKqNk8Bw Xry4Htw4HDAotJoH5LxH7Kuy3/CQ8gxjnrV7yb6umzryDBLW500rN+5plXbRMFh0R+bI 2ZurI/Wn7I2fjBs7uvfuvggxokBTLgjJnS9U5addVOt0js461HmeB07me5oDWYCAkOLv MAyPZ9occ51Ezmp3vunlo1eCw5JcNdT1fq8Qszy3glbLwP9OXkIglnq/zzulzyZrrjD7 jKvin1cX09JR8icTE8TMrFYqZmXql3jhSWBVSZQIzMc4bK4NFqOV9xxHMxdQWBNmxFOo fBwA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=GklP6KDi; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id l9-20020a170903120900b001aaf1734d1dsi3910819plh.241.2023.06.21.00.09.36; Wed, 21 Jun 2023 00:09:49 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=GklP6KDi; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230505AbjFUGno (ORCPT + 99 others); Wed, 21 Jun 2023 02:43:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39670 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230496AbjFUGnd (ORCPT ); Wed, 21 Jun 2023 02:43:33 -0400 Received: from mail-yw1-x1149.google.com (mail-yw1-x1149.google.com [IPv6:2607:f8b0:4864:20::1149]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 691A019AD for ; Tue, 20 Jun 2023 23:43:24 -0700 (PDT) Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-561eb6c66f6so63732567b3.0 for ; Tue, 20 Jun 2023 23:43:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1687329803; x=1689921803; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=xwDD9pVjLa+TJXAJEbbYyI+224KEPbc+a+2w73kHUiI=; b=GklP6KDiFXf3o6gA21y5VFJbUznyTc1R75l4mBRj5nFR4hoMeSZ10ocJBZmnS2WrDG y1e2Cn0bqo2ZayzQKL1V6VETTYM7805/cLncPYBXTbFCKA7UmtknIdrgLlnp/Lh7+41l ODS46uvmds+v5mJnejS+409PsLB0XhiizFZfkGk4lnkMeYF/ExizizPw0JPNEUIXu9de Y8fBSonTcoEaIv/WAbBuvtRvAr/ZPBwHCDKc5Gk6b9iQdwlvfCMDP/SgbMP9mJDNNXlW QRd5ZR/ihnnjNFE1GKLMAXC1B8cUlpYrZ3N7lw9Ph/+gpwyT7oMbiGOM/J7XzkNTlTEe g4jA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687329803; x=1689921803; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=xwDD9pVjLa+TJXAJEbbYyI+224KEPbc+a+2w73kHUiI=; b=MV82tB4vDR2UpLXXqRyBLe8+DOKahT5jnuebf8denDXMki39eYMLbVxdgYWv69J+2K swOvluWdKDi5ZuFBVe2xVG0Lejlu3en89AsIizGfE5YtSptHxn/ejMX7wyf0XtcC0R11 bm9DKyrSb75W4Wkd1o+lWAs5/9Z+/3DvKGhqirZVFuzo6mNeJtHhEnW3D2Aq99RubNl2 54biCDWO8lBMdofrosuZr6mi8EEJLX26IALON0NIPc+znELhRbDv014chvGgdaVgjk3x 6TD7zYBJWn/IaCjSAP3yr1mNxlHwNtX+MLlCMXZAMQLsJg61Q+LINNIzVQzjOgm5ZIef IWow== X-Gm-Message-State: AC+VfDy28bGI9B+Asx7CbOuSp225iLKXHERLYU5t6glJT4CgC93rF4Vp npf6U7YjjjS16Zbg+jw0XZ4YU/36n1I7 X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:384f:f7da:c61d:5a3e]) (user=mshavit job=sendgmr) by 2002:a81:b647:0:b0:56d:3c2b:2471 with SMTP id h7-20020a81b647000000b0056d3c2b2471mr6326664ywk.3.1687329803724; Tue, 20 Jun 2023 23:43:23 -0700 (PDT) Date: Wed, 21 Jun 2023 14:37:13 +0800 In-Reply-To: <20230621063825.268890-1-mshavit@google.com> Mime-Version: 1.0 References: <20230621063825.268890-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230621063825.268890-2-mshavit@google.com> Subject: [PATCH v4 01/13] iommu/arm-smmu-v3: Move ctx_desc out of s1_cfg From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1769295199185373482?= X-GMAIL-MSGID: =?utf-8?q?1769295199185373482?= s1_cfg describes the CD table that is inserted into an SMMU's STEs. It's weird for s1_cfg to also own ctx_desc which describes a CD that is inserted into that table. It is more appropriate for arm_smmu_domain to own ctx_desc. Signed-off-by: Michael Shavit --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 2 +- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 23 +++++++-------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 28 ++++++++++--------- 3 files changed, 28 insertions(+), 25 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index a5a63b1c947eb..968559d625c40 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -62,7 +62,7 @@ arm_smmu_share_asid(struct mm_struct *mm, u16 asid) return cd; } - smmu_domain = container_of(cd, struct arm_smmu_domain, s1_cfg.cd); + smmu_domain = container_of(cd, struct arm_smmu_domain, cd); smmu = smmu_domain->smmu; ret = xa_alloc(&arm_smmu_asid_xa, &new_asid, cd, diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 3fd83fb757227..beff04b897718 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1863,7 +1863,7 @@ static void arm_smmu_tlb_inv_context(void *cookie) * careful, 007. */ if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { - arm_smmu_tlb_inv_asid(smmu, smmu_domain->s1_cfg.cd.asid); + arm_smmu_tlb_inv_asid(smmu, smmu_domain->cd.asid); } else { cmd.opcode = CMDQ_OP_TLBI_S12_VMALL; cmd.tlbi.vmid = smmu_domain->s2_cfg.vmid; @@ -1946,7 +1946,7 @@ static void arm_smmu_tlb_inv_range_domain(unsigned long iova, size_t size, if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { cmd.opcode = smmu_domain->smmu->features & ARM_SMMU_FEAT_E2H ? CMDQ_OP_TLBI_EL2_VA : CMDQ_OP_TLBI_NH_VA; - cmd.tlbi.asid = smmu_domain->s1_cfg.cd.asid; + cmd.tlbi.asid = smmu_domain->cd.asid; } else { cmd.opcode = CMDQ_OP_TLBI_S2_IPA; cmd.tlbi.vmid = smmu_domain->s2_cfg.vmid; @@ -2077,7 +2077,7 @@ static void arm_smmu_domain_free(struct iommu_domain *domain) mutex_lock(&arm_smmu_asid_lock); if (cfg->cdcfg.cdtab) arm_smmu_free_cd_tables(smmu_domain); - arm_smmu_free_asid(&cfg->cd); + arm_smmu_free_asid(&smmu_domain->cd); mutex_unlock(&arm_smmu_asid_lock); } else { struct arm_smmu_s2_cfg *cfg = &smmu_domain->s2_cfg; @@ -2096,13 +2096,14 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, u32 asid; struct arm_smmu_device *smmu = smmu_domain->smmu; struct arm_smmu_s1_cfg *cfg = &smmu_domain->s1_cfg; + struct arm_smmu_ctx_desc *cd = &smmu_domain->cd; typeof(&pgtbl_cfg->arm_lpae_s1_cfg.tcr) tcr = &pgtbl_cfg->arm_lpae_s1_cfg.tcr; - refcount_set(&cfg->cd.refs, 1); + refcount_set(&cd->refs, 1); /* Prevent SVA from modifying the ASID until it is written to the CD */ mutex_lock(&arm_smmu_asid_lock); - ret = xa_alloc(&arm_smmu_asid_xa, &asid, &cfg->cd, + ret = xa_alloc(&arm_smmu_asid_xa, &asid, cd, XA_LIMIT(1, (1 << smmu->asid_bits) - 1), GFP_KERNEL); if (ret) goto out_unlock; @@ -2115,23 +2116,23 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, if (ret) goto out_free_asid; - cfg->cd.asid = (u16)asid; - cfg->cd.ttbr = pgtbl_cfg->arm_lpae_s1_cfg.ttbr; - cfg->cd.tcr = FIELD_PREP(CTXDESC_CD_0_TCR_T0SZ, tcr->tsz) | + cd->asid = (u16)asid; + cd->ttbr = pgtbl_cfg->arm_lpae_s1_cfg.ttbr; + cd->tcr = FIELD_PREP(CTXDESC_CD_0_TCR_T0SZ, tcr->tsz) | FIELD_PREP(CTXDESC_CD_0_TCR_TG0, tcr->tg) | FIELD_PREP(CTXDESC_CD_0_TCR_IRGN0, tcr->irgn) | FIELD_PREP(CTXDESC_CD_0_TCR_ORGN0, tcr->orgn) | FIELD_PREP(CTXDESC_CD_0_TCR_SH0, tcr->sh) | FIELD_PREP(CTXDESC_CD_0_TCR_IPS, tcr->ips) | CTXDESC_CD_0_TCR_EPD1 | CTXDESC_CD_0_AA64; - cfg->cd.mair = pgtbl_cfg->arm_lpae_s1_cfg.mair; + cd->mair = pgtbl_cfg->arm_lpae_s1_cfg.mair; /* * Note that this will end up calling arm_smmu_sync_cd() before * the master has been added to the devices list for this domain. * This isn't an issue because the STE hasn't been installed yet. */ - ret = arm_smmu_write_ctx_desc(smmu_domain, 0, &cfg->cd); + ret = arm_smmu_write_ctx_desc(smmu_domain, 0, cd); if (ret) goto out_free_cd_tables; @@ -2141,7 +2142,7 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, out_free_cd_tables: arm_smmu_free_cd_tables(smmu_domain); out_free_asid: - arm_smmu_free_asid(&cfg->cd); + arm_smmu_free_asid(cd); out_unlock: mutex_unlock(&arm_smmu_asid_lock); return ret; diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index b574c58a34876..68d519f21dbd8 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -593,7 +593,6 @@ struct arm_smmu_ctx_desc_cfg { struct arm_smmu_s1_cfg { struct arm_smmu_ctx_desc_cfg cdcfg; - struct arm_smmu_ctx_desc cd; u8 s1fmt; u8 s1cdmax; }; @@ -707,25 +706,28 @@ enum arm_smmu_domain_stage { }; struct arm_smmu_domain { - struct arm_smmu_device *smmu; - struct mutex init_mutex; /* Protects smmu pointer */ + struct arm_smmu_device *smmu; + struct mutex init_mutex; /* Protects smmu pointer */ - struct io_pgtable_ops *pgtbl_ops; - bool stall_enabled; - atomic_t nr_ats_masters; + struct io_pgtable_ops *pgtbl_ops; + bool stall_enabled; + atomic_t nr_ats_masters; - enum arm_smmu_domain_stage stage; + enum arm_smmu_domain_stage stage; union { - struct arm_smmu_s1_cfg s1_cfg; - struct arm_smmu_s2_cfg s2_cfg; + struct { + struct arm_smmu_ctx_desc cd; + struct arm_smmu_s1_cfg s1_cfg; + }; + struct arm_smmu_s2_cfg s2_cfg; }; - struct iommu_domain domain; + struct iommu_domain domain; - struct list_head devices; - spinlock_t devices_lock; + struct list_head devices; + spinlock_t devices_lock; - struct list_head mmu_notifiers; + struct list_head mmu_notifiers; }; static inline struct arm_smmu_domain *to_smmu_domain(struct iommu_domain *dom) From patchwork Wed Jun 21 06:37:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 110838 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp4184307vqr; Wed, 21 Jun 2023 00:33:46 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4hGdSRhfdZW7scbxU8wM+IZ2o931HhRNBgsQS+8GmwXb8kqmdIAwkXzd5P+peKHAcP1XIY X-Received: by 2002:a05:6a20:3d27:b0:121:5c4b:45bc with SMTP id y39-20020a056a203d2700b001215c4b45bcmr11793197pzi.58.1687332826030; Wed, 21 Jun 2023 00:33:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1687332826; cv=none; d=google.com; s=arc-20160816; b=zSa/nEhWSIwSZIhukW3bXntOUTsu0uwEwIdoHy5996R2OKsvl75u7r1G8xU57S8OMe F2V1Kw6ksvI2+/YFtNzbDBc/mT9YaxxkqM3N9ymiqVe/2WGNDpwgJwQuAUhyCH6nnog3 sPT41Tptm+9zXcoVlenLjNK77Yz6VCVyVzBHaq3xikmzQcyxYCVRlMhWvsyeOfAU/uWz aYE0DEP9PgmMCjiOMEvn3A2AaxcJArPHWCRyrEfuZ0USWInRpz0AB3yZxWCYXbeLMgPQ NOXF8wqy86WliFC0TDhBEWiWIawuec0mUw1HQq0q5rES0yV1hVaVVYxPlvkxiVYs95Aq ys0g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=9vic1ayWBGr79y9f6y1FKqdm9/OskuVUCQ3Tr10KopQ=; b=ERb70gfRZbF2/ybL0HtkRWsrPDk43+NoLZX0rzWLMTaYhhVvWUYpIggpYvL2bNQ0QE HHbiU+2jwZBMPTBoKQ4JPX5RdT+sgv1wcTFgTAQOXbivMDdpk+jij/Z/b2+de2B3VguF l/MmvANBaKTqvodNd39yTfKb7pYjYqSWe8B2aAHlAZBkCdkW46x5R0VJauYCihIZiCTt u8tm83f+NUO3q4ZF0eKgsV+oXMjNs4EDN/oXWLeoZ2Fd5UitiOys+Id9+865BD04NNwM 1CH/1kvvUY+mxwRnDLBOJ0wdXmg0TtQQLfNNP4bTb4yW3GIjNnV35QNt6meI7ClThQHr mIEQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=RHaZJpoc; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id x15-20020aa7940f000000b006630be2db05si3406475pfo.182.2023.06.21.00.33.33; Wed, 21 Jun 2023 00:33:46 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=RHaZJpoc; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231133AbjFUGoF (ORCPT + 99 others); Wed, 21 Jun 2023 02:44:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39912 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230013AbjFUGni (ORCPT ); Wed, 21 Jun 2023 02:43:38 -0400 Received: from mail-yw1-x114a.google.com (mail-yw1-x114a.google.com [IPv6:2607:f8b0:4864:20::114a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DA09B1709 for ; Tue, 20 Jun 2023 23:43:32 -0700 (PDT) Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-56cf9a86277so64701257b3.3 for ; Tue, 20 Jun 2023 23:43:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1687329812; x=1689921812; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=9vic1ayWBGr79y9f6y1FKqdm9/OskuVUCQ3Tr10KopQ=; b=RHaZJpocBAQex9N5sK+TVgzUKOuPn86MHuZT6EA4DsHyu3bgrSiJ/6leoQ9tr/MTNx fVcPxmgqsOc7bhA2bsToHFQk+6KAT5J8fjFvvBWV+GJ4hDTWKk5sDKZrS6MdI7si7KLi U0w462zitIRC0tsUnPZ2VYTHS65xaYWpUBs3saGloaPiHLGGDwtc031gios/uY13QTlj SLI28LI7OgWZJ/KaLkGSiBakAq8YtZIW1WVFV/BHehBRDjzQk0BaP4UntwQQWNb9F+I3 WNjFWpxs2VsOOSkh253vfrsykGVgRTXkImRD20J3FM6r7C4ZtfnPpiaPfw4QU//NgebR VONQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687329812; x=1689921812; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=9vic1ayWBGr79y9f6y1FKqdm9/OskuVUCQ3Tr10KopQ=; b=iry8OVdYcROmVOCX1TzbhtUzbOFGG2sqXhMcvwSC00foUcgZsEqy8Vk8tsofkgIrY+ zP8WBX+yMLywhMASZKFI6eQYzqzTVVyLMISMYmqC38NWFdYj4akEsVJdBON4Pet811Xi wovbksTsJ36z7Df+GWbdWApsh4cCveL0lJrIEyXE7YsqHoPBks9rw+vk7FnKTibmjzBT VTD2XhthnH263VivTiEqS59D27QXdJZHR16EaUoahnBDV//f81/Qq+ePMqYZrGZhW7wD Bc0dBRPcFRSD4USZpv0/9gtxUjGdDcuipjftDwonU3r0JHIHPoVNVngpPv0kauh7HLnJ joqQ== X-Gm-Message-State: AC+VfDwYf0+wfwVPAoJjpoyU74cPkOm7D/L2hmJ6FC7tWLQ1rZnaXUVj okpzMcVbcMaVexx+UA48m6OIl1xIVaoE X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:384f:f7da:c61d:5a3e]) (user=mshavit job=sendgmr) by 2002:a25:c514:0:b0:ba8:b828:c8ff with SMTP id v20-20020a25c514000000b00ba8b828c8ffmr1585519ybe.10.1687329812105; Tue, 20 Jun 2023 23:43:32 -0700 (PDT) Date: Wed, 21 Jun 2023 14:37:14 +0800 In-Reply-To: <20230621063825.268890-1-mshavit@google.com> Mime-Version: 1.0 References: <20230621063825.268890-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230621063825.268890-3-mshavit@google.com> Subject: [PATCH v4 02/13] iommu/arm-smmu-v3: Add smmu_s1_cfg to smmu_master From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1769296705068661643?= X-GMAIL-MSGID: =?utf-8?q?1769296705068661643?= Except for Nested domains, arm_smmu_master will own the STEs that are inserted into the arm_smmu_device's STE table. Signed-off-by: Michael Shavit --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 28 +++++++++++++-------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 1 + 2 files changed, 18 insertions(+), 11 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index beff04b897718..023769f5ca79a 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1126,15 +1126,16 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, return 0; } -static int arm_smmu_alloc_cd_tables(struct arm_smmu_domain *smmu_domain) +static int arm_smmu_init_s1_cfg(struct arm_smmu_master *master, + struct arm_smmu_s1_cfg *cfg) { int ret; size_t l1size; size_t max_contexts; - struct arm_smmu_device *smmu = smmu_domain->smmu; - struct arm_smmu_s1_cfg *cfg = &smmu_domain->s1_cfg; + struct arm_smmu_device *smmu = master->smmu; struct arm_smmu_ctx_desc_cfg *cdcfg = &cfg->cdcfg; + cfg->s1cdmax = master->ssid_bits; max_contexts = 1 << cfg->s1cdmax; if (!(smmu->features & ARM_SMMU_FEAT_2_LVL_CDTAB) || @@ -1175,12 +1176,11 @@ static int arm_smmu_alloc_cd_tables(struct arm_smmu_domain *smmu_domain) return ret; } -static void arm_smmu_free_cd_tables(struct arm_smmu_domain *smmu_domain) +static void arm_smmu_free_cd_tables(struct arm_smmu_device *smmu, + struct arm_smmu_ctx_desc_cfg *cdcfg) { int i; size_t size, l1size; - struct arm_smmu_device *smmu = smmu_domain->smmu; - struct arm_smmu_ctx_desc_cfg *cdcfg = &smmu_domain->s1_cfg.cdcfg; if (cdcfg->l1_desc) { size = CTXDESC_L2_ENTRIES * (CTXDESC_CD_DWORDS << 3); @@ -2076,7 +2076,7 @@ static void arm_smmu_domain_free(struct iommu_domain *domain) /* Prevent SVA from touching the CD while we're freeing it */ mutex_lock(&arm_smmu_asid_lock); if (cfg->cdcfg.cdtab) - arm_smmu_free_cd_tables(smmu_domain); + arm_smmu_free_cd_tables(smmu, &cfg->cdcfg); arm_smmu_free_asid(&smmu_domain->cd); mutex_unlock(&arm_smmu_asid_lock); } else { @@ -2108,11 +2108,9 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, if (ret) goto out_unlock; - cfg->s1cdmax = master->ssid_bits; - smmu_domain->stall_enabled = master->stall_enabled; - ret = arm_smmu_alloc_cd_tables(smmu_domain); + ret = arm_smmu_init_s1_cfg(master, cfg); if (ret) goto out_free_asid; @@ -2140,7 +2138,7 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, return 0; out_free_cd_tables: - arm_smmu_free_cd_tables(smmu_domain); + arm_smmu_free_cd_tables(smmu, &cfg->cdcfg); out_free_asid: arm_smmu_free_asid(cd); out_unlock: @@ -2704,6 +2702,13 @@ static struct iommu_device *arm_smmu_probe_device(struct device *dev) smmu->features & ARM_SMMU_FEAT_STALL_FORCE) master->stall_enabled = true; + ret = arm_smmu_init_s1_cfg(master, &master->owned_s1_cfg); + if (ret) { + arm_smmu_disable_pasid(master); + arm_smmu_remove_master(master); + goto err_free_master; + } + return &smmu->iommu; err_free_master: @@ -2719,6 +2724,7 @@ static void arm_smmu_release_device(struct device *dev) if (WARN_ON(arm_smmu_master_sva_enabled(master))) iopf_queue_remove_device(master->smmu->evtq.iopf, dev); arm_smmu_detach_dev(master); + arm_smmu_free_cd_tables(master->smmu, &master->owned_s1_cfg.cdcfg); arm_smmu_disable_pasid(master); arm_smmu_remove_master(master); kfree(master); diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 68d519f21dbd8..053cc14c23969 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -688,6 +688,7 @@ struct arm_smmu_master { struct arm_smmu_domain *domain; struct list_head domain_head; struct arm_smmu_stream *streams; + struct arm_smmu_s1_cfg owned_s1_cfg; unsigned int num_streams; bool ats_enabled; bool stall_enabled; From patchwork Wed Jun 21 06:37:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 110818 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp4170021vqr; Wed, 21 Jun 2023 00:02:21 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4pyZj6ZqG5Q7hEDETk2bhFBrc1Yiu8ksj2BT0eknpLxgHkbQC5wOXveJ9/aJnHklWn+/VD X-Received: by 2002:a17:90a:d310:b0:25e:b5ba:8bb9 with SMTP id p16-20020a17090ad31000b0025eb5ba8bb9mr10607114pju.5.1687330941287; Wed, 21 Jun 2023 00:02:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1687330941; cv=none; d=google.com; s=arc-20160816; b=yhy4YnZxLp8vzjKppMXV3xCrWm4DM067STJvc+MdBVmQtyYruhfFSFeMlq+4fYOlrX k0BGFvwjoPYDuobMmewHT8NQrtKKBlsgcgxYur5jR5PKy+XBXIC0TjLywxi4cW/vyGQs l0s6ZxJis36dZlEHwWp5GIS+HeWwsI3gRLQ4iDrszh6lFlNhuB5OmKTv/uMN6a17tfWk UCkNr/VG2RY/a+GkSEnF0BxHfrxY+r8r6BFXy1hmTqUy/LWU07VrBymK4pjc/UdJ+1G0 nLIJUVBfbkZ0ltHPLzCf18bWnD4whwR0k0rUuzxRCEtrH8gk/0labrLnNWwTJCreB5m1 0NVg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=jzXQy+3gWCRrOq/bPJGSod35GlRHXYXuXCqPkpYte2c=; b=ZNi2VVxshZK1dgf35RIboNlgCoygFFasJAMeC4swfSA8rH/f+b+zYnB3iSFxsSSNyH 8fZEAgdE2488UJt/cTr1rKgFeCKAci6eF634HQTK0cjStviezGx4jpeiX2lKzc8wRyL7 6Hr0TgN8PGgBV+yoXG8C5LeGLa13jpP2HLMuPzwigNSSOQ/J98F0k6VpP84AUH4JEbqq 4T925kNY+jOyQxbcIZ0uhydThSeKx29UGaq8fewQAxz3sea/3o3FPF3brWmzTSbA4cHp h+QXhHMGSpmXtLLp5AB01csop5MeKpEW9UWSE00kNdEHWh4LXsJWptJ9qr9A0KIiQZZS 9vMw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=rvLyDf34; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id j7-20020a17090a31c700b002508889985fsi10826344pjf.95.2023.06.21.00.02.04; Wed, 21 Jun 2023 00:02:21 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=rvLyDf34; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230499AbjFUGoL (ORCPT + 99 others); Wed, 21 Jun 2023 02:44:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39944 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230489AbjFUGnn (ORCPT ); Wed, 21 Jun 2023 02:43:43 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CA328198E for ; Tue, 20 Jun 2023 23:43:40 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-bd69bb4507eso6010283276.2 for ; Tue, 20 Jun 2023 23:43:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1687329820; x=1689921820; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=jzXQy+3gWCRrOq/bPJGSod35GlRHXYXuXCqPkpYte2c=; b=rvLyDf3473Ea5KgbE9MzK4m81ED5MczQdqagmULEilVLDvyqXUuA956dDqxQL8afkd I/KMEhrIM3w+cmdo4c6BxJzNUY5csQMLavWP2+i17nXGNqCj+ZP70yPu0bi+pTgs225a rAZ2HNjpybLfLtg0BQ98c3tHjAB9BijbI8q7UEO3+8148457WZur9CdIO5YNp4lm+Vdl Rq0CSZOcZtZN/6WN2azDEvWHFdlHaHUH9P77mTDTDjJVMQbgx5Uae087uO5s5uldv2nM l7OWmzWlWv7Q89Y0t3V/kW5ayh5OzqWwRmyT8IrUe+tNL09iKKoA09VyeydJpGmNpmxs Pbww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687329820; x=1689921820; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=jzXQy+3gWCRrOq/bPJGSod35GlRHXYXuXCqPkpYte2c=; b=llm04Lonz2x7I5/uzi1EAewIWMJNz0FGpSBjP6HrcDTy0n/mP2Fa1kMdIaZe5726Xi jvMslTBBesh3SzGImpvmnd3jf19ya6wQJypPL9x0XGlf9VpxIFDhvbBroxQnxw25KrTU kgoM8ko1mxtAjioj2DttGhgo2oPKVkmQc2KAeMIrqxKRzvnTK5l6f8jS1bLBLTUB2moG J006xuaGmK31VuQ1tbNm5YPKbMHVcJDDuNpwsaJGl9C1QJweEex0N6FWxomPhmLmZMAi KMGX02Jp98nYg6ZXmmqK3rszrKJYQAMKGQZHU8LIX8dmrubbSoDCP0M36tK8dPzDFDnP jlsQ== X-Gm-Message-State: AC+VfDxjKbRGTpESdtJitsCsvxYwBmeGjYNMuI8/k1+K6LXGlt/BLLX8 Idgg463w5xnZEoyQSbFN28ED+9pKkE37 X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:384f:f7da:c61d:5a3e]) (user=mshavit job=sendgmr) by 2002:a05:6902:15c4:b0:be4:d646:48d8 with SMTP id l4-20020a05690215c400b00be4d64648d8mr1659256ybu.8.1687329820101; Tue, 20 Jun 2023 23:43:40 -0700 (PDT) Date: Wed, 21 Jun 2023 14:37:15 +0800 In-Reply-To: <20230621063825.268890-1-mshavit@google.com> Mime-Version: 1.0 References: <20230621063825.268890-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230621063825.268890-4-mshavit@google.com> Subject: [PATCH v4 03/13] iommu/arm-smmu-v3: Refactor write_strtab_ent From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1769294728799555703?= X-GMAIL-MSGID: =?utf-8?q?1769294728799555703?= Explicity keep track of the s1_cfg and s2_cfg that are attached to a master in arm_smmu_master, regardless of whether they are owned by arm_smmu_master, arm_smmu_domain or userspace. Signed-off-by: Michael Shavit --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 37 +++++++++------------ drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 2 ++ 2 files changed, 17 insertions(+), 22 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 023769f5ca79a..d79c6ef5d6ed4 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1269,10 +1269,9 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, */ u64 val = le64_to_cpu(dst[0]); bool ste_live = false; - struct arm_smmu_device *smmu = NULL; + struct arm_smmu_device *smmu = master->smmu; struct arm_smmu_s1_cfg *s1_cfg = NULL; struct arm_smmu_s2_cfg *s2_cfg = NULL; - struct arm_smmu_domain *smmu_domain = NULL; struct arm_smmu_cmdq_ent prefetch_cmd = { .opcode = CMDQ_OP_PREFETCH_CFG, .prefetch = { @@ -1280,24 +1279,10 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, }, }; - if (master) { - smmu_domain = master->domain; - smmu = master->smmu; - } - - if (smmu_domain) { - switch (smmu_domain->stage) { - case ARM_SMMU_DOMAIN_S1: - s1_cfg = &smmu_domain->s1_cfg; - break; - case ARM_SMMU_DOMAIN_S2: - case ARM_SMMU_DOMAIN_NESTED: - s2_cfg = &smmu_domain->s2_cfg; - break; - default: - break; - } - } + if (master->s1_cfg) + s1_cfg = master->s1_cfg; + else if (master->s2_cfg) + s2_cfg = master->s2_cfg; if (val & STRTAB_STE_0_V) { switch (FIELD_GET(STRTAB_STE_0_CFG, val)) { @@ -1319,8 +1304,8 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, val = STRTAB_STE_0_V; /* Bypass/fault */ - if (!smmu_domain || !(s1_cfg || s2_cfg)) { - if (!smmu_domain && disable_bypass) + if (!(s1_cfg || s2_cfg)) { + if (disable_bypass) val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_ABORT); else val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_BYPASS); @@ -2401,6 +2386,8 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) master->domain = NULL; master->ats_enabled = false; + master->s1_cfg = NULL; + master->s2_cfg = NULL; arm_smmu_install_ste_for_dev(master); } @@ -2454,6 +2441,12 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) } master->domain = smmu_domain; + if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { + master->s1_cfg = &smmu_domain->s1_cfg; + } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S2 || + smmu_domain->stage == ARM_SMMU_DOMAIN_NESTED) { + master->s2_cfg = &smmu_domain->s2_cfg; + } /* * The SMMU does not support enabling ATS with bypass. When the STE is diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 053cc14c23969..3c614fbe2b8b9 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -689,6 +689,8 @@ struct arm_smmu_master { struct list_head domain_head; struct arm_smmu_stream *streams; struct arm_smmu_s1_cfg owned_s1_cfg; + struct arm_smmu_s1_cfg *s1_cfg; + struct arm_smmu_s2_cfg *s2_cfg; unsigned int num_streams; bool ats_enabled; bool stall_enabled; From patchwork Wed Jun 21 06:37:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 110855 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp4201032vqr; Wed, 21 Jun 2023 01:13:02 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5ZFVWXgJekHYnz6qxDb+uAX9SJz5slyIojIbdyUwUc/ESUfLUfx09HRiOsLZM5whSjKp1/ X-Received: by 2002:a17:902:ce8e:b0:1ad:cba5:5505 with SMTP id f14-20020a170902ce8e00b001adcba55505mr17730200plg.14.1687335181756; Wed, 21 Jun 2023 01:13:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1687335181; cv=none; d=google.com; s=arc-20160816; b=ekVEEOppjymnF4qpk/g+C8wOYA1V0B6rIBjGcft4JciuPP4nlAslOQJ5hHUAIx9eQx 1IUMuGkCYEWqHG1zxvxCnVhroUS57JMOlTxJtPJ6ULFYKcBbxh+FWdRk63qqdShUdM1y f4wLDF2FSuji6DVsTdMcrp5zCAJEk9KfbjdcEpjiO5/f3pxhSSiHpL416P6oWc02Qmdm GG08keRq6h98ipNykvJeOEwEXUvGH15TCZDIjjD+6aqmfei6I/XEmca18mTQhitzCLtd nn4j9QfwNhcqzM5JQbd3W7KZYO1sndP1KgHDtgS+TFZXk1SHVqHID/bRcSNbCODv76bK rAOA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=T3tJEpWHCJm3S04xQ0T/8NFLZuI/7sadQhRIHV1s9Zo=; b=rgIjHoejiNvbLCEU5Hdk81/pr6uWNhBpUuUuNjFbUfkEs5ho330QUM9gmoPa19MAbO mUCeiaI5BJaBr2lLb7kU1IBY/bk0ioKCTdIAfgbTytr/MCZP+khN3pAGgCJsOO4UNv8w J0ZWFVjmrJ7Qh6SoEqwYTkJYoVmJyRBVjSxPxqJ2nJJioVB3atTzHgWQlI/U5Z8EEkx3 cJDSs8auiuJQnQS0yIKpA3IYI51+riyL9Ax0po+XOgPYQjJcJXPi0p8A3WfRo/PJBpHT j5iovJ1XHf6hpxctmX0lez12jIPZkHALfc8Pqccz23CTUIZa6LhNkDjFtw5qxPOzO037 kfHA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=tDLP783t; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id jn22-20020a170903051600b001b5497b0030si3575631plb.97.2023.06.21.01.12.45; Wed, 21 Jun 2023 01:13:01 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=tDLP783t; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230517AbjFUGoW (ORCPT + 99 others); Wed, 21 Jun 2023 02:44:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40136 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230515AbjFUGnv (ORCPT ); Wed, 21 Jun 2023 02:43:51 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3E8DC19A5 for ; Tue, 20 Jun 2023 23:43:49 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-bd5b8a9d82cso6185852276.0 for ; Tue, 20 Jun 2023 23:43:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1687329828; x=1689921828; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=T3tJEpWHCJm3S04xQ0T/8NFLZuI/7sadQhRIHV1s9Zo=; b=tDLP783tJ0VQXKySnXfcycNFVcu1CwUxp8tUrTH0xu2NVvWELKSyiZsFc93tdsXdvB xETbRFAM6aKkKwECQOiXpyJILPwdCxEBih5pnCop8eUyqe0rhCwyzuViEs0sbi5rGrej 9rL1XXQz69682N/N0/xt2Ohaot08bo5JQ9YQ/7IvhK9fLsoPnGVAokhZW6l/U5V/54UZ dA4BV3jTZuZEfgKZnQXiYtuBp4sLNnewc3vxY6RJ0YJwjoeeizsZilVlVAnwctKlJt3T u7RzkL909yZir1kXSjPGUWrgU6j5aLvYF7i8+sm5hJkwQTUqGkvq3HSxVSeKXV8QvaHd RFmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687329828; x=1689921828; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=T3tJEpWHCJm3S04xQ0T/8NFLZuI/7sadQhRIHV1s9Zo=; b=AjRRRDy/PTTcGQ6aojT5C99P9fyW2hOdgWVwvXygeESeFjE6Kn5Wt7ehTUI9lga9D4 WnUiTXBAX3no2PK2mGYxxM6CFqQyvTXhxJRxTLBV6su3dkdiPqBuNwD9koRbAoq/Cc+D bIoDdC+op7BeSyY25pCrG7rFXeaB/MGW3Gexl57M6TWsYpYDn8H1U/sTqK1PZmm+Ag3G udOo5Cm5nc1p6XZkl5Fk+XIRojCIeSj1NNo2b/ViPe8Lps17joDXQV78lGgbBC8839it dynA0bTVrNGkC1nJ2Q6BFnO85UZZod9JU9w+ikxs37RybAJlZIgSspqDl/mDEryUl537 jPeQ== X-Gm-Message-State: AC+VfDx3n+EBGWN13GZFMcD4TtInD8dOYx20A4UMg/imHbJPkQo4Pnx9 /yAw6AK+YDkgwXYXg3r8UdX9jW8Wlv+8 X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:384f:f7da:c61d:5a3e]) (user=mshavit job=sendgmr) by 2002:a25:748e:0:b0:bac:fd63:b567 with SMTP id p136-20020a25748e000000b00bacfd63b567mr5997224ybc.4.1687329828409; Tue, 20 Jun 2023 23:43:48 -0700 (PDT) Date: Wed, 21 Jun 2023 14:37:16 +0800 In-Reply-To: <20230621063825.268890-1-mshavit@google.com> Mime-Version: 1.0 References: <20230621063825.268890-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230621063825.268890-5-mshavit@google.com> Subject: [PATCH v4 04/13] iommu/arm-smmu-v3: Refactor write_ctx_desc From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED, USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1769299175132160910?= X-GMAIL-MSGID: =?utf-8?q?1769299175132160910?= Update arm_smmu_write_ctx_desc and downstream functions to be agnostic of the CD table's ownership. Note that in practice, it will only be called in cases where the table is owned by arm_smmu_master. Whether arm_smmu_write_ctx_desc will trigger a sync is also made part of the API. Note that this change isn't a nop refactor since SVA will call arm_smmu_write_ctx_desc in a loop for every master the domain is attached to despite the fact that they all share the same CD table. Note that the next commit ceases this sharing of the s1_cfg which makes that loop necessary. Signed-off-by: Michael Shavit --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 36 ++++++++- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 80 +++++++++++-------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 6 +- 3 files changed, 81 insertions(+), 41 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 968559d625c40..48fa8eb271a45 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -45,10 +45,12 @@ static struct arm_smmu_ctx_desc * arm_smmu_share_asid(struct mm_struct *mm, u16 asid) { int ret; + unsigned long flags; u32 new_asid; struct arm_smmu_ctx_desc *cd; struct arm_smmu_device *smmu; struct arm_smmu_domain *smmu_domain; + struct arm_smmu_master *master; cd = xa_load(&arm_smmu_asid_xa, asid); if (!cd) @@ -80,7 +82,11 @@ arm_smmu_share_asid(struct mm_struct *mm, u16 asid) * be some overlap between use of both ASIDs, until we invalidate the * TLB. */ - arm_smmu_write_ctx_desc(smmu_domain, 0, cd); + spin_lock_irqsave(&smmu_domain->devices_lock, flags); + list_for_each_entry(master, &smmu_domain->devices, domain_head) { + arm_smmu_write_ctx_desc(smmu, master->s1_cfg, master, 0, cd); + } + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); /* Invalidate TLB entries previously associated with that context */ arm_smmu_tlb_inv_asid(smmu, asid); @@ -211,6 +217,8 @@ static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) { struct arm_smmu_mmu_notifier *smmu_mn = mn_to_smmu(mn); struct arm_smmu_domain *smmu_domain = smmu_mn->domain; + struct arm_smmu_master *master; + unsigned long flags; mutex_lock(&sva_lock); if (smmu_mn->cleared) { @@ -222,7 +230,12 @@ static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) * DMA may still be running. Keep the cd valid to avoid C_BAD_CD events, * but disable translation. */ - arm_smmu_write_ctx_desc(smmu_domain, mm->pasid, &quiet_cd); + spin_lock_irqsave(&smmu_domain->devices_lock, flags); + list_for_each_entry(master, &smmu_domain->devices, domain_head) { + arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, + mm->pasid, &quiet_cd); + } + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); arm_smmu_tlb_inv_asid(smmu_domain->smmu, smmu_mn->cd->asid); arm_smmu_atc_inv_domain(smmu_domain, mm->pasid, 0, 0); @@ -248,8 +261,10 @@ arm_smmu_mmu_notifier_get(struct arm_smmu_domain *smmu_domain, struct mm_struct *mm) { int ret; + unsigned long flags; struct arm_smmu_ctx_desc *cd; struct arm_smmu_mmu_notifier *smmu_mn; + struct arm_smmu_master *master; list_for_each_entry(smmu_mn, &smmu_domain->mmu_notifiers, list) { if (smmu_mn->mn.mm == mm) { @@ -279,7 +294,12 @@ arm_smmu_mmu_notifier_get(struct arm_smmu_domain *smmu_domain, goto err_free_cd; } - ret = arm_smmu_write_ctx_desc(smmu_domain, mm->pasid, cd); + spin_lock_irqsave(&smmu_domain->devices_lock, flags); + list_for_each_entry(master, &smmu_domain->devices, domain_head) { + ret = arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, + master, mm->pasid, cd); + } + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); if (ret) goto err_put_notifier; @@ -296,15 +316,23 @@ arm_smmu_mmu_notifier_get(struct arm_smmu_domain *smmu_domain, static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) { + unsigned long flags; struct mm_struct *mm = smmu_mn->mn.mm; struct arm_smmu_ctx_desc *cd = smmu_mn->cd; + struct arm_smmu_master *master; struct arm_smmu_domain *smmu_domain = smmu_mn->domain; if (!refcount_dec_and_test(&smmu_mn->refs)) return; list_del(&smmu_mn->list); - arm_smmu_write_ctx_desc(smmu_domain, mm->pasid, NULL); + + spin_lock_irqsave(&smmu_domain->devices_lock, flags); + list_for_each_entry(master, &smmu_domain->devices, domain_head) { + arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, + mm->pasid, NULL); + } + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); /* * If we went through clear(), we've already invalidated, and no diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index d79c6ef5d6ed4..b6f7cf60f8f3d 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -965,14 +965,13 @@ void arm_smmu_tlb_inv_asid(struct arm_smmu_device *smmu, u16 asid) arm_smmu_cmdq_issue_cmd_with_sync(smmu, &cmd); } -static void arm_smmu_sync_cd(struct arm_smmu_domain *smmu_domain, +/* master may be null */ +static void arm_smmu_sync_cd(struct arm_smmu_master *master, int ssid, bool leaf) { size_t i; - unsigned long flags; - struct arm_smmu_master *master; struct arm_smmu_cmdq_batch cmds; - struct arm_smmu_device *smmu = smmu_domain->smmu; + struct arm_smmu_device *smmu; struct arm_smmu_cmdq_ent cmd = { .opcode = CMDQ_OP_CFGI_CD, .cfgi = { @@ -981,16 +980,15 @@ static void arm_smmu_sync_cd(struct arm_smmu_domain *smmu_domain, }, }; - cmds.num = 0; + if (!master) + return; - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { - for (i = 0; i < master->num_streams; i++) { - cmd.cfgi.sid = master->streams[i].id; - arm_smmu_cmdq_batch_add(smmu, &cmds, &cmd); - } + smmu = master->smmu; + cmds.num = 0; + for (i = 0; i < master->num_streams; i++) { + cmd.cfgi.sid = master->streams[i].id; + arm_smmu_cmdq_batch_add(smmu, &cmds, &cmd); } - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); arm_smmu_cmdq_batch_submit(smmu, &cmds); } @@ -1020,16 +1018,18 @@ static void arm_smmu_write_cd_l1_desc(__le64 *dst, WRITE_ONCE(*dst, cpu_to_le64(val)); } -static __le64 *arm_smmu_get_cd_ptr(struct arm_smmu_domain *smmu_domain, +/* master may be null */ +static __le64 *arm_smmu_get_cd_ptr(struct arm_smmu_device *smmu, + struct arm_smmu_s1_cfg *s1_cfg, + struct arm_smmu_master *master, u32 ssid) { __le64 *l1ptr; unsigned int idx; struct arm_smmu_l1_ctx_desc *l1_desc; - struct arm_smmu_device *smmu = smmu_domain->smmu; - struct arm_smmu_ctx_desc_cfg *cdcfg = &smmu_domain->s1_cfg.cdcfg; + struct arm_smmu_ctx_desc_cfg *cdcfg = &s1_cfg->cdcfg; - if (smmu_domain->s1_cfg.s1fmt == STRTAB_STE_0_S1FMT_LINEAR) + if (s1_cfg->s1fmt == STRTAB_STE_0_S1FMT_LINEAR) return cdcfg->cdtab + ssid * CTXDESC_CD_DWORDS; idx = ssid >> CTXDESC_SPLIT; @@ -1041,13 +1041,21 @@ static __le64 *arm_smmu_get_cd_ptr(struct arm_smmu_domain *smmu_domain, l1ptr = cdcfg->cdtab + idx * CTXDESC_L1_DESC_DWORDS; arm_smmu_write_cd_l1_desc(l1ptr, l1_desc); /* An invalid L1CD can be cached */ - arm_smmu_sync_cd(smmu_domain, ssid, false); + arm_smmu_sync_cd(master, ssid, false); } idx = ssid & (CTXDESC_L2_ENTRIES - 1); return l1_desc->l2ptr + idx * CTXDESC_CD_DWORDS; } -int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, +/* + * master must be provided if a CD sync is required but may be null otherwise + * (such as when the CD table isn't inserted into the STE yet, or is about to + * be detached. + */ +int arm_smmu_write_ctx_desc(struct arm_smmu_device *smmu, + struct arm_smmu_s1_cfg *s1_cfg, + struct arm_smmu_master *master, + int ssid, struct arm_smmu_ctx_desc *cd) { /* @@ -1065,10 +1073,10 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, bool cd_live; __le64 *cdptr; - if (WARN_ON(ssid >= (1 << smmu_domain->s1_cfg.s1cdmax))) + if (WARN_ON(ssid >= (1 << s1_cfg->s1cdmax))) return -E2BIG; - cdptr = arm_smmu_get_cd_ptr(smmu_domain, ssid); + cdptr = arm_smmu_get_cd_ptr(smmu, s1_cfg, master, ssid); if (!cdptr) return -ENOMEM; @@ -1092,11 +1100,11 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, cdptr[3] = cpu_to_le64(cd->mair); /* - * STE is live, and the SMMU might read dwords of this CD in any - * order. Ensure that it observes valid values before reading - * V=1. + * STE may be live, and the SMMU might read dwords of this CD + * in any order. Ensure that it observes valid values before + * reading V=1. */ - arm_smmu_sync_cd(smmu_domain, ssid, true); + arm_smmu_sync_cd(master, ssid, true); val = cd->tcr | #ifdef __BIG_ENDIAN @@ -1108,7 +1116,7 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, FIELD_PREP(CTXDESC_CD_0_ASID, cd->asid) | CTXDESC_CD_0_V; - if (smmu_domain->stall_enabled) + if (s1_cfg->stall_enabled) val |= CTXDESC_CD_0_S; } @@ -1122,7 +1130,7 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, * without first making the structure invalid. */ WRITE_ONCE(cdptr[0], cpu_to_le64(val)); - arm_smmu_sync_cd(smmu_domain, ssid, true); + arm_smmu_sync_cd(master, ssid, true); return 0; } @@ -1136,6 +1144,7 @@ static int arm_smmu_init_s1_cfg(struct arm_smmu_master *master, struct arm_smmu_ctx_desc_cfg *cdcfg = &cfg->cdcfg; cfg->s1cdmax = master->ssid_bits; + cfg->stall_enabled = master->stall_enabled; max_contexts = 1 << cfg->s1cdmax; if (!(smmu->features & ARM_SMMU_FEAT_2_LVL_CDTAB) || @@ -2093,8 +2102,6 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, if (ret) goto out_unlock; - smmu_domain->stall_enabled = master->stall_enabled; - ret = arm_smmu_init_s1_cfg(master, cfg); if (ret) goto out_free_asid; @@ -2110,12 +2117,9 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, CTXDESC_CD_0_TCR_EPD1 | CTXDESC_CD_0_AA64; cd->mair = pgtbl_cfg->arm_lpae_s1_cfg.mair; - /* - * Note that this will end up calling arm_smmu_sync_cd() before - * the master has been added to the devices list for this domain. - * This isn't an issue because the STE hasn't been installed yet. - */ - ret = arm_smmu_write_ctx_desc(smmu_domain, 0, cd); + ret = arm_smmu_write_ctx_desc(smmu, cfg, + NULL /*Not attached to a master yet */, + 0, cd); if (ret) goto out_free_cd_tables; @@ -2386,6 +2390,11 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) master->domain = NULL; master->ats_enabled = false; + if (master->s1_cfg) + arm_smmu_write_ctx_desc( + master->smmu, master->s1_cfg, + NULL /* Skip sync since we detach the CD table next*/, + 0, NULL); master->s1_cfg = NULL; master->s2_cfg = NULL; arm_smmu_install_ste_for_dev(master); @@ -2435,7 +2444,8 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) ret = -EINVAL; goto out_unlock; } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1 && - smmu_domain->stall_enabled != master->stall_enabled) { + smmu_domain->s1_cfg.stall_enabled != + master->stall_enabled) { ret = -EINVAL; goto out_unlock; } diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 3c614fbe2b8b9..00a493442d6f9 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -595,6 +595,7 @@ struct arm_smmu_s1_cfg { struct arm_smmu_ctx_desc_cfg cdcfg; u8 s1fmt; u8 s1cdmax; + bool stall_enabled; }; struct arm_smmu_s2_cfg { @@ -713,7 +714,6 @@ struct arm_smmu_domain { struct mutex init_mutex; /* Protects smmu pointer */ struct io_pgtable_ops *pgtbl_ops; - bool stall_enabled; atomic_t nr_ats_masters; enum arm_smmu_domain_stage stage; @@ -742,7 +742,9 @@ extern struct xarray arm_smmu_asid_xa; extern struct mutex arm_smmu_asid_lock; extern struct arm_smmu_ctx_desc quiet_cd; -int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, +int arm_smmu_write_ctx_desc(struct arm_smmu_device *smmu, + struct arm_smmu_s1_cfg *s1_cfg, + struct arm_smmu_master *smmu_master, int ssid, struct arm_smmu_ctx_desc *cd); void arm_smmu_tlb_inv_asid(struct arm_smmu_device *smmu, u16 asid); void arm_smmu_tlb_inv_range_asid(unsigned long iova, size_t size, int asid, From patchwork Wed Jun 21 06:37:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 110834 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp4181612vqr; Wed, 21 Jun 2023 00:27:41 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5CrqBlpm9DW6d0dbzqX7aLqAuB/DJcrR/UrfWnNbjyX+Oi2vRcDD2o6hFKe+BkehVO03Lt X-Received: by 2002:a17:903:1250:b0:1ac:aba5:7885 with SMTP id u16-20020a170903125000b001acaba57885mr17173028plh.47.1687332460729; Wed, 21 Jun 2023 00:27:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1687332460; cv=none; d=google.com; s=arc-20160816; b=d4scRUc4/FEktgDpCB5ocTMsm5YpkROgZlpvZc4fBftmOk+fGKunn7Inkd406Cn4xs atVp9YZT2RlVV7hE2v0S7iUHz0HztUYo2SO6jWyQWiTc4W3hcQ7iFkur6lsEIncaKU0D ggubfAVP9HQK9Zn2gmF31etqwNnlgyy8ymmQkKaeTXn7z51ktaET1MpJvYd64wV9zdcW 0++mq3Qv+T5jXV/0HB8sVGg+b4KaghIkPCQTmUE93CJTuKyE36vsXW2yzPiDlvGgTLBS W5wY7EZxnI7lsmFyhWZSpYPClYlJYXSq4iKBz/rT1d8bRe6DQGgADFL6KO9K68MG26nG f8Yw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=6vzESexuWCV4LLT4ZlWlOvb8u8164/HANB5QcSOGct0=; b=VIOD6aaDCEAOofw3U6LDW3q60GfJ3KHwQqTVnySs5KPiC2YckLOrFIjTJsIFNOkkE8 2vN1KcQXjT10PRAHNGFTpPgrhIN1DodNLXEVTIksHCKIFPy6TKTW9YrnCHM6eaAU2+2Z ndVpFeKyD331pD6YT0b0bSl9Mta0QRsIyRPjNvLa6y1qL20FZAEAzy6OI9qJlKb24/Rf 3Qi84Zb0KvPhiAXjaBODHB1TTf/Y3ptMBogl/w7NqCsVbIxrCRccAz+CFazykxB0d8pq I+aYzxiPs+pNfCSdt3xKn9M76JDJCmtjdWHgn6Cnnn1V7eXqVYVsG3DK2uw8ZauIzZKh /Fkw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=svmsiobx; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id o11-20020a170903210b00b001aafeb7d2f2si3318880ple.287.2023.06.21.00.27.27; Wed, 21 Jun 2023 00:27:40 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=svmsiobx; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231129AbjFUGoe (ORCPT + 99 others); Wed, 21 Jun 2023 02:44:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40292 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230527AbjFUGoD (ORCPT ); Wed, 21 Jun 2023 02:44:03 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 370231BCF for ; Tue, 20 Jun 2023 23:43:57 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-bcef239a9fbso13159717276.1 for ; Tue, 20 Jun 2023 23:43:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1687329836; x=1689921836; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=6vzESexuWCV4LLT4ZlWlOvb8u8164/HANB5QcSOGct0=; b=svmsiobx2Wo1tGVdmspzGY8XKIT2RMMfZsVIGP8+D0VskhD3W00QvnB74yg6w3Znfg FF0PHmvC5XVe190m9JDChplAtynZBkJ4zmC/GZXSj4PHFQu9Po+4/dccCy/wbGuWhmxh se005ZBXaisFJn7BGyDl48Svnfz+/u/MRUw7ZsoPofZBjic/m2iZTrsM/fqSlADWGFY/ De6MKp6cEBEd3qxpL6wwFrJQp9hAuRrVmgyq3yIR5w/vn99QHwt8yL/0Jng0DUo0EAV9 NZlsTatB05xTsImbbonTm96Wvoi3kQc1kObhyM6VDa93ATuNWP6gZ4K7nJlIb1va4fXm sPbQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687329836; x=1689921836; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=6vzESexuWCV4LLT4ZlWlOvb8u8164/HANB5QcSOGct0=; b=g8WEareeH0eyVpZANwkSYE5Xl8yxjHq6jXe0n0dny8ECzA0cJbuvxJZkqWZ4HAG6iz As4sF2PswoGzBvBazG2ao+qKT4RRp6KtKsr2SFGH6ksiC3zTJcwCCZGgqrs+0WNB87UP iqG6T3Lw/54wSkhLr6W6E8tJdaLycP0KqF+Gr5T9cLOs8YJh/xLPE5NFBCdcWlb2TnQ+ j1SjfX3B0RkcWdM89JG6Hjzqd+im+iD8nwaJDPdK7aR1fIT84SQFg1yJVdFHxBB5EaHR gNgjMdNxFJH9/0K+HwaSkb5nAVRaqk1dkfOnaoNm6eDzWFpmMPnDMxtFlFHyJnLlTFHn YgLQ== X-Gm-Message-State: AC+VfDyrOC9V3AQae0dMJLYuCh3QlMv4tFkEFdLIeHTSAX8axPcwOzTm A4BY24giDqzPY17OGm0PyDYf4aJs1Ugt X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:384f:f7da:c61d:5a3e]) (user=mshavit job=sendgmr) by 2002:a25:abee:0:b0:ba8:4ff5:3214 with SMTP id v101-20020a25abee000000b00ba84ff53214mr3393703ybi.2.1687329836534; Tue, 20 Jun 2023 23:43:56 -0700 (PDT) Date: Wed, 21 Jun 2023 14:37:17 +0800 In-Reply-To: <20230621063825.268890-1-mshavit@google.com> Mime-Version: 1.0 References: <20230621063825.268890-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230621063825.268890-6-mshavit@google.com> Subject: [PATCH v4 05/13] iommu/arm-smmu-v3: Use the master-owned s1_cfg From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1769296322279771664?= X-GMAIL-MSGID: =?utf-8?q?1769296322279771664?= Insert CDs for STAGE_1 domains into a CD table owned by the arm_smmu_master. Remove the CD table that was owned by arm_smmu_domain. Signed-off-by: Michael Shavit --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 43 ++++++--------------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 3 -- 2 files changed, 12 insertions(+), 34 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index b6f7cf60f8f3d..08f440fe1da6d 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2065,12 +2065,8 @@ static void arm_smmu_domain_free(struct iommu_domain *domain) /* Free the CD and ASID, if we allocated them */ if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { - struct arm_smmu_s1_cfg *cfg = &smmu_domain->s1_cfg; - /* Prevent SVA from touching the CD while we're freeing it */ mutex_lock(&arm_smmu_asid_lock); - if (cfg->cdcfg.cdtab) - arm_smmu_free_cd_tables(smmu, &cfg->cdcfg); arm_smmu_free_asid(&smmu_domain->cd); mutex_unlock(&arm_smmu_asid_lock); } else { @@ -2082,14 +2078,13 @@ static void arm_smmu_domain_free(struct iommu_domain *domain) kfree(smmu_domain); } -static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, +static int arm_smmu_domain_finalise_cd(struct arm_smmu_domain *smmu_domain, struct arm_smmu_master *master, struct io_pgtable_cfg *pgtbl_cfg) { int ret; u32 asid; struct arm_smmu_device *smmu = smmu_domain->smmu; - struct arm_smmu_s1_cfg *cfg = &smmu_domain->s1_cfg; struct arm_smmu_ctx_desc *cd = &smmu_domain->cd; typeof(&pgtbl_cfg->arm_lpae_s1_cfg.tcr) tcr = &pgtbl_cfg->arm_lpae_s1_cfg.tcr; @@ -2102,10 +2097,6 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, if (ret) goto out_unlock; - ret = arm_smmu_init_s1_cfg(master, cfg); - if (ret) - goto out_free_asid; - cd->asid = (u16)asid; cd->ttbr = pgtbl_cfg->arm_lpae_s1_cfg.ttbr; cd->tcr = FIELD_PREP(CTXDESC_CD_0_TCR_T0SZ, tcr->tsz) | @@ -2117,19 +2108,9 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, CTXDESC_CD_0_TCR_EPD1 | CTXDESC_CD_0_AA64; cd->mair = pgtbl_cfg->arm_lpae_s1_cfg.mair; - ret = arm_smmu_write_ctx_desc(smmu, cfg, - NULL /*Not attached to a master yet */, - 0, cd); - if (ret) - goto out_free_cd_tables; - mutex_unlock(&arm_smmu_asid_lock); return 0; -out_free_cd_tables: - arm_smmu_free_cd_tables(smmu, &cfg->cdcfg); -out_free_asid: - arm_smmu_free_asid(cd); out_unlock: mutex_unlock(&arm_smmu_asid_lock); return ret; @@ -2192,7 +2173,7 @@ static int arm_smmu_domain_finalise(struct iommu_domain *domain, ias = min_t(unsigned long, ias, VA_BITS); oas = smmu->ias; fmt = ARM_64_LPAE_S1; - finalise_stage_fn = arm_smmu_domain_finalise_s1; + finalise_stage_fn = arm_smmu_domain_finalise_cd; break; case ARM_SMMU_DOMAIN_NESTED: case ARM_SMMU_DOMAIN_S2: @@ -2439,20 +2420,20 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) } else if (smmu_domain->smmu != smmu) { ret = -EINVAL; goto out_unlock; - } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1 && - master->ssid_bits != smmu_domain->s1_cfg.s1cdmax) { - ret = -EINVAL; - goto out_unlock; - } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1 && - smmu_domain->s1_cfg.stall_enabled != - master->stall_enabled) { - ret = -EINVAL; - goto out_unlock; } master->domain = smmu_domain; if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { - master->s1_cfg = &smmu_domain->s1_cfg; + master->s1_cfg = &master->owned_s1_cfg; + ret = arm_smmu_write_ctx_desc( + smmu, + master->s1_cfg, NULL /*Not attached to a master yet */, + 0, &smmu_domain->cd); + if (ret) { + master->s1_cfg = NULL; + master->domain = NULL; + goto out_unlock; + } } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S2 || smmu_domain->stage == ARM_SMMU_DOMAIN_NESTED) { master->s2_cfg = &smmu_domain->s2_cfg; diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 00a493442d6f9..dff0fa8345462 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -718,10 +718,7 @@ struct arm_smmu_domain { enum arm_smmu_domain_stage stage; union { - struct { struct arm_smmu_ctx_desc cd; - struct arm_smmu_s1_cfg s1_cfg; - }; struct arm_smmu_s2_cfg s2_cfg; }; From patchwork Wed Jun 21 06:37:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 110822 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp4172980vqr; Wed, 21 Jun 2023 00:08:23 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5W1IMWZlF8w2BlYp4L7+q3Zh0I4grFJFPnJwEgS38J4sIU/NWPN47kAe30Civuso+mevaJ X-Received: by 2002:a05:6358:9f88:b0:130:e3dd:a8f3 with SMTP id fy8-20020a0563589f8800b00130e3dda8f3mr5451264rwb.24.1687331302753; Wed, 21 Jun 2023 00:08:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1687331302; cv=none; d=google.com; s=arc-20160816; b=pm39a810plOiIA6Y2of2ODzhTRdG75jcTCYHm+OmzgDeR5nJGe9L9QqKt3DNM12h+A 3xtqHkdcOZX1BwXLJxcV02S+cvhUAFwbioKFkyiByK22TYb0ryFYr1vAFk9tQrTsXHRQ e36rp7ySRXOPu5gsPcpsCXnwx/Ay5EpeRkcke33Zjfr/Arymi+mP9NfO31W61xYRV0JU hX/sRNR03xtxpVGVKM380mXf4OuIpckZQA3sGAXloXePmR1WoYQQGl/Fk7CvOcpUf8Zl un/hTwTCqk7eixlJ0HpYOA3MVoSytArA2wblUpa8dm87rBbvtr/fuFmrtRP2KsGzXYKw PVzQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=kKyYB6nGtYqnelo6XDlWvQzY0JjgxL/I8n3xKSQjyKM=; b=VjFBXjiSnHBhoButwmHgRBY4UNsRR2McMUQbvb/OcX90MBKt/JGLj3gRAzGGcwAqNW JIKFoqGcUpSOAk0SFFdm9JufAI4Qp/tcOG4D4NJ0KCGDuazjT8gZhBPohWqDRetI/jCj fa6tkxeak7lHm/jb1kB/9kFJdo2IemCYHw0MsZ+48li2CbFSetM3lgk07RxRdvMeA4Lo KOEOXlWkRgPUF/PtDn0eWNOzOeneFeOMHx1+zaNKFDU1RhylnfFYUxpsNJR8GL71RTg+ K1bebkg+MrrWiNuwjaGQXYrHrSNNeIOc7cNZm1BvlkS8KnAVFXCdJtGfaxarT6yYs/ur CDlQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=1gaS0SkW; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id s15-20020a63af4f000000b005536fc5187asi3395578pgo.47.2023.06.21.00.08.09; Wed, 21 Jun 2023 00:08:22 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=1gaS0SkW; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229951AbjFUGpO (ORCPT + 99 others); Wed, 21 Jun 2023 02:45:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40460 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231156AbjFUGoI (ORCPT ); Wed, 21 Jun 2023 02:44:08 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3E2481BF4 for ; Tue, 20 Jun 2023 23:44:05 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-bc77a2d3841so5687269276.3 for ; Tue, 20 Jun 2023 23:44:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1687329844; x=1689921844; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=kKyYB6nGtYqnelo6XDlWvQzY0JjgxL/I8n3xKSQjyKM=; b=1gaS0SkW/Sq1tXITKeGmUGTrKdPbcpqB9xL6/exaE20A1NUvR9wFlTNBFxrCnmmx1W noltPOrcY64SY3zBIRAqGGfbitthNieh+upEIdv0gNliNAenwXtXVAvyOqY984mtAd6s nZgSwJLTL85y1xBHpiEsiHarlo3Bmr5RbJEloL0e473xQwEsJxKBB34H4CbuybmXI0iF wQwBCGVX4lZWA1O+4ztFmOn8qil9C1LlaOpKaRxZ4fVPxqw4z+x6RhcaqWte8IpC0cAV lWLdyIWdY+62QfGdY9MFUSCJ+NVCLxGu5r6cyaSOI9Fao6Wur1O0Gop63zLuED1qwKea cevQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687329844; x=1689921844; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=kKyYB6nGtYqnelo6XDlWvQzY0JjgxL/I8n3xKSQjyKM=; b=dkNSv1PxXteX/8YVt7fWsO3ybQLWTTr6TCdeYayxDpIjpPtlFa0HQikbpfy6Pccd21 cCguyLkMmV6iLxp+DMTKVOnNZq83MX3kTCtGsexjH0ArnXIOOH+HnBDIV2kGPQdGqQxg CZW0xpAQncS4Dvdcu2q7ehbFTSCu5AIudDBQfp1L+tmH2iVIi/u7xVGz72nXFXy/D69n koz+4sSNIpBwWZafec5QOpJExSUXnNLyiUzo86xFIIfv6Tve9bCLQkIwJOAm/ZWkhO+j XjR4IB9oNYDjT93g3M3DENUCA5IPatpvfeCPTskulfQB0W1wVjxv7M9otKcLeGiBpcXu xS/A== X-Gm-Message-State: AC+VfDwZoy8EgHgVh7KHKvPoBSmcs0dZEhlP7/bRBil7o5e8wCqsq4fh CbLdOjlTG9hN1TdU9rfRUMopVwsIYZWo X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:384f:f7da:c61d:5a3e]) (user=mshavit job=sendgmr) by 2002:a05:6902:1363:b0:bea:918f:2ef6 with SMTP id bt3-20020a056902136300b00bea918f2ef6mr1543181ybb.0.1687329844564; Tue, 20 Jun 2023 23:44:04 -0700 (PDT) Date: Wed, 21 Jun 2023 14:37:18 +0800 In-Reply-To: <20230621063825.268890-1-mshavit@google.com> Mime-Version: 1.0 References: <20230621063825.268890-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230621063825.268890-7-mshavit@google.com> Subject: [PATCH v4 06/13] iommu/arm-smmu-v3: Simplify arm_smmu_enable_ats From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1769295107948875716?= X-GMAIL-MSGID: =?utf-8?q?1769295107948875716?= arm_smmu_enable_ats's call to inv_domain would trigger an invalidation for all masters that a domain is attached to everytime it's attached to another ATS-enabled master. It doesn't seem like those invalidations are necessary, and it's easier to reason about arm_smmu_enable_ats if it only issues invalidation commands for the current master. Signed-off-by: Michael Shavit --- v1->v2: Fix commit message wrapping --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 08f440fe1da6d..dc7a59e87a2b4 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2286,7 +2286,7 @@ static void arm_smmu_enable_ats(struct arm_smmu_master *master) pdev = to_pci_dev(master->dev); atomic_inc(&smmu_domain->nr_ats_masters); - arm_smmu_atc_inv_domain(smmu_domain, 0, 0, 0); + arm_smmu_atc_inv_master(master); if (pci_enable_ats(pdev, stu)) dev_err(master->dev, "Failed to enable ATS (STU %zu)\n", stu); } From patchwork Wed Jun 21 06:37:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 110821 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp4172898vqr; Wed, 21 Jun 2023 00:08:13 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6vu6XYQAj4pUat7BhC8+/5PQbyAu06A/PFVww60UlCs+VsSVgXKqR75So7llqheYPsUfIz X-Received: by 2002:a05:622a:145:b0:3ff:258a:db2a with SMTP id v5-20020a05622a014500b003ff258adb2amr8195527qtw.1.1687331293231; Wed, 21 Jun 2023 00:08:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1687331293; cv=none; d=google.com; s=arc-20160816; b=RIvEpFzt65RvzRZCSOUdBk90ssVqnpMFGfkhfpg+M4xYtLGDwwczoETTZMlDXG8IfH 7ppElZ+HNCela8Nuh4/bcSc7cOhLjksMbFayWdDaFO8+bdxWMM7oT5nWQpgU0IzNps3O D3fFQ0UOG8A6BNSvRm034kO+zEQRtV4BijqI1XF1Pf5/3JWcHKvWY+mKQ8OPrbbky5Ud sjSQAAz03KJAD8SkMXH6D516wplz5HKCsZx8OH71UFdYbi70F4H+xLdNGbY/Y73XMUKz b06w/GKVrHULVlWb3aAbGQv8HljamF77o73e2VH47jgaeaSQSO/jj9Yz0/tpFZKuAQYH eYuA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=3AqNtmxD8/oO/z6ww5oLsHi1wrWCEx0tKqFMiqVzP6A=; b=Vcvu3CF9gntjzJeKNl/aTlAf43/9KFxllCb5SaWujHqA5NY4/kG7XgxbAtydZOt5Yd 4Mh96mWnTUE4TJI1qDDTascaUwBFoAaC44VIMJ2YlPPlXFXqC3IRdyt2xtLwiMor5bnI 4HivpFB4JaOvuyJTSjEiuBkeu0Y6sYMv8xNnTL/bBD7Zk5akBoPRq1DwB4hLXpPkOCL6 M/+i4QpYwSB6stIRRpphsT8FhC2v5el74gRoUyOpdNKDnaEsKY06mpQHxuOFAJKjcjhv mAHYYetHQtMVGSDxLFD13pL8YA8d1ERXAMcudsyi3r02byNyQYhpGQSDoXpgw7yNs/E0 wRCQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=n2U7hoM7; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id y26-20020a63b51a000000b0055379a7131csi3341650pge.721.2023.06.21.00.07.59; Wed, 21 Jun 2023 00:08:13 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=n2U7hoM7; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230448AbjFUGpY (ORCPT + 99 others); Wed, 21 Jun 2023 02:45:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39856 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230013AbjFUGoo (ORCPT ); Wed, 21 Jun 2023 02:44:44 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 09A001FE0 for ; Tue, 20 Jun 2023 23:44:13 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-bc8ea14f4eeso6912624276.0 for ; Tue, 20 Jun 2023 23:44:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1687329853; x=1689921853; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=3AqNtmxD8/oO/z6ww5oLsHi1wrWCEx0tKqFMiqVzP6A=; b=n2U7hoM7km5VMyO9tQ7aCYTdIMhARsT7MatmhovkwIXV2Qz88RLJgC5HnSksVvHsKm rPQaUditDHkJ6h0Ku6MbsaD2YvShF3Ix5mqqs/zYd+3vr4EOwnOWKy4lcLXuJ4/a7Tqw tvN+QaQUsh1WKWAakGfRT3yJSF+sxzPW0pwBBxxzj7GphA1Meqnip54EVYH5zl+5EATo +HWEH0NNDeX7WFiw2GsAMSdW5VFcK25Gyl6jzy7ccQmIXn9bqolKll5MbYerkBz2J37L /HOTx1oGr5IaHFiGYr+inO1jyJn+Fu1bfbbNlO6ZwGhe8h69qo1gwui0ApXeUHja4F/8 NDYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687329853; x=1689921853; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=3AqNtmxD8/oO/z6ww5oLsHi1wrWCEx0tKqFMiqVzP6A=; b=ONgxoGRQaCf7/U88Cv4lg/KV5/UxN5F+T6Ouv1oHxyAXwBBqsIZOAdcuRXJT6jcaqU UN8xaJ1CikvXpEVDAfKbcxJ+SCkQD+e4NR08qz2136877sd5WSwcY1AE4JPI47CK2wM2 RQ8UHY02Pe070n6DZdy1+6J9k1ZWE3sekdMykq+O2w18+zit9djtSdkNE8rSbqDAyJIi sGReCyp6IQyCv4PTtHuYBEx3pBlJ0/hZo3z6woid56AsqlW99o1Fj8PW74+E4rRSd8fX MwRfi/1GzCsgKc4LmYEnBMLrmg5ObzD1BpAn33REfBcYK3r0/AF1kIT0eoSOqij/KJtD rV9g== X-Gm-Message-State: AC+VfDws+qwjuSRKgd9vfS22THV2pnMeTol8EaOmUdid359s/HaqWaGE lE08MD1xUZheIWgHyv6AolHMRjGjtOtq X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:384f:f7da:c61d:5a3e]) (user=mshavit job=sendgmr) by 2002:a25:7408:0:b0:ba8:929a:2073 with SMTP id p8-20020a257408000000b00ba8929a2073mr5832292ybc.1.1687329852794; Tue, 20 Jun 2023 23:44:12 -0700 (PDT) Date: Wed, 21 Jun 2023 14:37:19 +0800 In-Reply-To: <20230621063825.268890-1-mshavit@google.com> Mime-Version: 1.0 References: <20230621063825.268890-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230621063825.268890-8-mshavit@google.com> Subject: [PATCH v4 07/13] iommu/arm-smmu-v3: Keep track of attached ssids From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED, USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1769295097851589917?= X-GMAIL-MSGID: =?utf-8?q?1769295097851589917?= The arm-smmu-v3 driver keeps track of all masters that a domain is attached to so that it can re-write their STEs when the domain's ASID is upated by SVA. This tracking is also used to invalidate ATCs on all masters that a domain is attached to. This change introduces a new data structures to track all the CD entries that a domain is attached to. This change is a pre-requisite to allow domain attachment on non 0 SSIDs. Signed-off-by: Michael Shavit --- v3->v4: Remove reference to the master's domain accidentally re-introduced during a rebase. Make arm_smmu_atc_inv_domain static. v1->v2: Fix arm_smmu_atc_inv_cmd_set_ssid and other cosmetic changes --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 53 +++++++---- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 89 ++++++++++++------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 18 ++-- 3 files changed, 105 insertions(+), 55 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 48fa8eb271a45..d07c08b53c5cf 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -51,6 +51,7 @@ arm_smmu_share_asid(struct mm_struct *mm, u16 asid) struct arm_smmu_device *smmu; struct arm_smmu_domain *smmu_domain; struct arm_smmu_master *master; + struct arm_smmu_attached_domain *attached_domain; cd = xa_load(&arm_smmu_asid_xa, asid); if (!cd) @@ -82,11 +83,14 @@ arm_smmu_share_asid(struct mm_struct *mm, u16 asid) * be some overlap between use of both ASIDs, until we invalidate the * TLB. */ - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { - arm_smmu_write_ctx_desc(smmu, master->s1_cfg, master, 0, cd); + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_for_each_entry(attached_domain, &smmu_domain->attached_domains, + domain_head) { + master = attached_domain->master; + arm_smmu_write_ctx_desc(smmu, master->s1_cfg, master, + attached_domain->ssid, cd); } - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); /* Invalidate TLB entries previously associated with that context */ arm_smmu_tlb_inv_asid(smmu, asid); @@ -210,7 +214,7 @@ static void arm_smmu_mm_invalidate_range(struct mmu_notifier *mn, if (!(smmu_domain->smmu->features & ARM_SMMU_FEAT_BTM)) arm_smmu_tlb_inv_range_asid(start, size, smmu_mn->cd->asid, PAGE_SIZE, false, smmu_domain); - arm_smmu_atc_inv_domain(smmu_domain, mm->pasid, start, size); + arm_smmu_atc_inv_domain_ssid(smmu_domain, mm->pasid, start, size); } static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) @@ -218,6 +222,7 @@ static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) struct arm_smmu_mmu_notifier *smmu_mn = mn_to_smmu(mn); struct arm_smmu_domain *smmu_domain = smmu_mn->domain; struct arm_smmu_master *master; + struct arm_smmu_attached_domain *attached_domain; unsigned long flags; mutex_lock(&sva_lock); @@ -230,15 +235,21 @@ static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) * DMA may still be running. Keep the cd valid to avoid C_BAD_CD events, * but disable translation. */ - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { - arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, - mm->pasid, &quiet_cd); + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_for_each_entry(attached_domain, &smmu_domain->attached_domains, + domain_head) { + master = attached_domain->master; + /* + * SVA domains piggyback on the attached_domain with SSID 0. + */ + if (attached_domain->ssid == 0) + arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, + master, mm->pasid, &quiet_cd); } - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); arm_smmu_tlb_inv_asid(smmu_domain->smmu, smmu_mn->cd->asid); - arm_smmu_atc_inv_domain(smmu_domain, mm->pasid, 0, 0); + arm_smmu_atc_inv_domain_ssid(smmu_domain, mm->pasid, 0, 0); smmu_mn->cleared = true; mutex_unlock(&sva_lock); @@ -265,6 +276,7 @@ arm_smmu_mmu_notifier_get(struct arm_smmu_domain *smmu_domain, struct arm_smmu_ctx_desc *cd; struct arm_smmu_mmu_notifier *smmu_mn; struct arm_smmu_master *master; + struct arm_smmu_attached_domain *attached_domain; list_for_each_entry(smmu_mn, &smmu_domain->mmu_notifiers, list) { if (smmu_mn->mn.mm == mm) { @@ -294,12 +306,14 @@ arm_smmu_mmu_notifier_get(struct arm_smmu_domain *smmu_domain, goto err_free_cd; } - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_for_each_entry(attached_domain, &smmu_domain->attached_domains, + domain_head) { + master = attached_domain->master; ret = arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, mm->pasid, cd); } - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); if (ret) goto err_put_notifier; @@ -319,6 +333,7 @@ static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) unsigned long flags; struct mm_struct *mm = smmu_mn->mn.mm; struct arm_smmu_ctx_desc *cd = smmu_mn->cd; + struct arm_smmu_attached_domain *attached_domain; struct arm_smmu_master *master; struct arm_smmu_domain *smmu_domain = smmu_mn->domain; @@ -327,12 +342,14 @@ static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) list_del(&smmu_mn->list); - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_for_each_entry(attached_domain, &smmu_domain->attached_domains, + domain_head) { + master = attached_domain->master; arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, mm->pasid, NULL); } - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); /* * If we went through clear(), we've already invalidated, and no @@ -340,7 +357,7 @@ static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) */ if (!smmu_mn->cleared) { arm_smmu_tlb_inv_asid(smmu_domain->smmu, cd->asid); - arm_smmu_atc_inv_domain(smmu_domain, mm->pasid, 0, 0); + arm_smmu_atc_inv_domain_ssid(smmu_domain, mm->pasid, 0, 0); } /* Frees smmu_mn */ diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index dc7a59e87a2b4..65e2dfd28b7d8 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1711,7 +1711,14 @@ static irqreturn_t arm_smmu_combined_irq_handler(int irq, void *dev) } static void -arm_smmu_atc_inv_to_cmd(int ssid, unsigned long iova, size_t size, +arm_smmu_atc_inv_cmd_set_ssid(int ssid, struct arm_smmu_cmdq_ent *cmd) +{ + cmd->substream_valid = !!ssid; + cmd->atc.ssid = ssid; +} + +static void +arm_smmu_atc_inv_to_cmd(unsigned long iova, size_t size, struct arm_smmu_cmdq_ent *cmd) { size_t log2_span; @@ -1736,8 +1743,8 @@ arm_smmu_atc_inv_to_cmd(int ssid, unsigned long iova, size_t size, */ *cmd = (struct arm_smmu_cmdq_ent) { .opcode = CMDQ_OP_ATC_INV, - .substream_valid = !!ssid, - .atc.ssid = ssid, + .substream_valid = false, + .atc.ssid = 0, }; if (!size) { @@ -1783,8 +1790,7 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) struct arm_smmu_cmdq_ent cmd; struct arm_smmu_cmdq_batch cmds; - arm_smmu_atc_inv_to_cmd(0, 0, 0, &cmd); - + arm_smmu_atc_inv_to_cmd(0, 0, &cmd); cmds.num = 0; for (i = 0; i < master->num_streams; i++) { cmd.atc.sid = master->streams[i].id; @@ -1794,13 +1800,19 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) return arm_smmu_cmdq_batch_submit(master->smmu, &cmds); } -int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, - unsigned long iova, size_t size) +/* + * If ssid is non-zero, issue atc invalidations with the given ssid instead of + * the one the domain is attached to. This is used by SVA since it's pasid + * attachments aren't recorded in smmu_domain yet. + */ +int arm_smmu_atc_inv_domain_ssid(struct arm_smmu_domain *smmu_domain, int ssid, + unsigned long iova, size_t size) { int i; unsigned long flags; struct arm_smmu_cmdq_ent cmd; struct arm_smmu_master *master; + struct arm_smmu_attached_domain *attached_domain; struct arm_smmu_cmdq_batch cmds; if (!(smmu_domain->smmu->features & ARM_SMMU_FEAT_ATS)) @@ -1823,25 +1835,37 @@ int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, if (!atomic_read(&smmu_domain->nr_ats_masters)) return 0; - arm_smmu_atc_inv_to_cmd(ssid, iova, size, &cmd); + arm_smmu_atc_inv_to_cmd(iova, size, &cmd); cmds.num = 0; - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_for_each_entry(attached_domain, &smmu_domain->attached_domains, + domain_head) { + master = attached_domain->master; if (!master->ats_enabled) continue; + if (ssid != 0) + arm_smmu_atc_inv_cmd_set_ssid(ssid, &cmd); + else + arm_smmu_atc_inv_cmd_set_ssid(attached_domain->ssid, &cmd); for (i = 0; i < master->num_streams; i++) { cmd.atc.sid = master->streams[i].id; arm_smmu_cmdq_batch_add(smmu_domain->smmu, &cmds, &cmd); } } - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); return arm_smmu_cmdq_batch_submit(smmu_domain->smmu, &cmds); } +static int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, + unsigned long iova, size_t size) +{ + return arm_smmu_atc_inv_domain_ssid(smmu_domain, 0, iova, size); +} + /* IO_PGTABLE API */ static void arm_smmu_tlb_inv_context(void *cookie) { @@ -1863,7 +1887,7 @@ static void arm_smmu_tlb_inv_context(void *cookie) cmd.tlbi.vmid = smmu_domain->s2_cfg.vmid; arm_smmu_cmdq_issue_cmd_with_sync(smmu, &cmd); } - arm_smmu_atc_inv_domain(smmu_domain, 0, 0, 0); + arm_smmu_atc_inv_domain(smmu_domain, 0, 0); } static void __arm_smmu_tlb_inv_range(struct arm_smmu_cmdq_ent *cmd, @@ -1951,7 +1975,7 @@ static void arm_smmu_tlb_inv_range_domain(unsigned long iova, size_t size, * Unfortunately, this can't be leaf-only since we may have * zapped an entire table. */ - arm_smmu_atc_inv_domain(smmu_domain, 0, iova, size); + arm_smmu_atc_inv_domain(smmu_domain, iova, size); } void arm_smmu_tlb_inv_range_asid(unsigned long iova, size_t size, int asid, @@ -2031,8 +2055,8 @@ static struct iommu_domain *arm_smmu_domain_alloc(unsigned type) return NULL; mutex_init(&smmu_domain->init_mutex); - INIT_LIST_HEAD(&smmu_domain->devices); - spin_lock_init(&smmu_domain->devices_lock); + INIT_LIST_HEAD(&smmu_domain->attached_domains); + spin_lock_init(&smmu_domain->attached_domains_lock); INIT_LIST_HEAD(&smmu_domain->mmu_notifiers); return &smmu_domain->domain; @@ -2270,12 +2294,12 @@ static bool arm_smmu_ats_supported(struct arm_smmu_master *master) return dev_is_pci(dev) && pci_ats_supported(to_pci_dev(dev)); } -static void arm_smmu_enable_ats(struct arm_smmu_master *master) +static void arm_smmu_enable_ats(struct arm_smmu_master *master, + struct arm_smmu_domain *smmu_domain) { size_t stu; struct pci_dev *pdev; struct arm_smmu_device *smmu = master->smmu; - struct arm_smmu_domain *smmu_domain = master->domain; /* Don't enable ATS at the endpoint if it's not enabled in the STE */ if (!master->ats_enabled) @@ -2291,10 +2315,9 @@ static void arm_smmu_enable_ats(struct arm_smmu_master *master) dev_err(master->dev, "Failed to enable ATS (STU %zu)\n", stu); } -static void arm_smmu_disable_ats(struct arm_smmu_master *master) +static void arm_smmu_disable_ats(struct arm_smmu_master *master, + struct arm_smmu_domain *smmu_domain) { - struct arm_smmu_domain *smmu_domain = master->domain; - if (!master->ats_enabled) return; @@ -2358,18 +2381,17 @@ static void arm_smmu_disable_pasid(struct arm_smmu_master *master) static void arm_smmu_detach_dev(struct arm_smmu_master *master) { unsigned long flags; - struct arm_smmu_domain *smmu_domain = master->domain; + struct arm_smmu_domain *smmu_domain = master->non_pasid_domain.domain; if (!smmu_domain) return; - arm_smmu_disable_ats(master); + arm_smmu_disable_ats(master, smmu_domain); - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_del(&master->domain_head); - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_del(&master->non_pasid_domain.domain_head); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); - master->domain = NULL; master->ats_enabled = false; if (master->s1_cfg) arm_smmu_write_ctx_desc( @@ -2378,6 +2400,7 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) 0, NULL); master->s1_cfg = NULL; master->s2_cfg = NULL; + master->non_pasid_domain.domain = NULL; arm_smmu_install_ste_for_dev(master); } @@ -2422,7 +2445,6 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) goto out_unlock; } - master->domain = smmu_domain; if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { master->s1_cfg = &master->owned_s1_cfg; ret = arm_smmu_write_ctx_desc( @@ -2431,7 +2453,6 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) 0, &smmu_domain->cd); if (ret) { master->s1_cfg = NULL; - master->domain = NULL; goto out_unlock; } } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S2 || @@ -2449,13 +2470,17 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) if (smmu_domain->stage != ARM_SMMU_DOMAIN_BYPASS) master->ats_enabled = arm_smmu_ats_supported(master); + master->non_pasid_domain.master = master; + master->non_pasid_domain.domain = smmu_domain; + master->non_pasid_domain.ssid = 0; arm_smmu_install_ste_for_dev(master); - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_add(&master->domain_head, &smmu_domain->devices); - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_add(&master->non_pasid_domain.domain_head, + &smmu_domain->attached_domains); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); - arm_smmu_enable_ats(master); + arm_smmu_enable_ats(master, smmu_domain); out_unlock: mutex_unlock(&smmu_domain->init_mutex); diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index dff0fa8345462..6929590530367 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -682,11 +682,19 @@ struct arm_smmu_stream { struct rb_node node; }; +/* List of {masters, ssid} that a domain is attached to */ +struct arm_smmu_attached_domain { + struct list_head domain_head; + struct arm_smmu_domain *domain; + struct arm_smmu_master *master; + int ssid; +}; + /* SMMU private data for each master */ struct arm_smmu_master { struct arm_smmu_device *smmu; struct device *dev; - struct arm_smmu_domain *domain; + struct arm_smmu_attached_domain non_pasid_domain; struct list_head domain_head; struct arm_smmu_stream *streams; struct arm_smmu_s1_cfg owned_s1_cfg; @@ -724,8 +732,8 @@ struct arm_smmu_domain { struct iommu_domain domain; - struct list_head devices; - spinlock_t devices_lock; + struct list_head attached_domains; + spinlock_t attached_domains_lock; struct list_head mmu_notifiers; }; @@ -748,8 +756,8 @@ void arm_smmu_tlb_inv_range_asid(unsigned long iova, size_t size, int asid, size_t granule, bool leaf, struct arm_smmu_domain *smmu_domain); bool arm_smmu_free_asid(struct arm_smmu_ctx_desc *cd); -int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, - unsigned long iova, size_t size); +int arm_smmu_atc_inv_domain_ssid(struct arm_smmu_domain *smmu_domain, int ssid, + unsigned long iova, size_t size); #ifdef CONFIG_ARM_SMMU_V3_SVA bool arm_smmu_sva_supported(struct arm_smmu_device *smmu); From patchwork Wed Jun 21 06:37:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 110836 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp4184063vqr; Wed, 21 Jun 2023 00:33:12 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5wDIptUi3iuNybndzya66MM3gOiyDEA+ooH3EPeYI1fkoS8J4rXDyrJq+JQdm3NX6gUIpH X-Received: by 2002:a05:6a21:6da4:b0:121:b5af:acb1 with SMTP id wl36-20020a056a216da400b00121b5afacb1mr10396921pzb.29.1687332791812; Wed, 21 Jun 2023 00:33:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1687332791; cv=none; d=google.com; s=arc-20160816; b=H5nem2woncsgjGKjsXS8nEKvhBCBF4n/nU2TamnxPDxPAEH7T/v1+Gim5gsFIscfAr fmjqHK8r2Z4ekylDBJifqEokJF946whrNFqI2nRj/gmZnv16a14BlmMZ51rdGUa3IsJa Qo+z/YKLcQWXACYz07pN+UqOT3donqNJxZW4fkin14PJChsETab7CDww1Ji9qpo5l73Z EcdzaZzLX0F00qgOC+SBduG4RHGWvIznoWgoSSgqLT1p4IOcyAYGnl9BZYEfubyf+26j xI8IBDJyyk+X3wdeKVn2sjT3W+3kgzRY5M2eYGrqjEmR40hSTjdmMfzC2uQBSTrk5Jso 4Owg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=C8az5zJDH0ESFjwiqnExbJyBaF+tF0BTQf7Bxk21/Xo=; b=Z+wQZvskwyBI6ijOLMWCFC1z1r3gWRe8oyXM4mfQNRnKF0lG/d2tvf6E+e3irekJmW QjG5L/rH5hGaJvuFluOiR6veECgoYGHele6rXrDebe/Wlnn90cP0to/hXdefJPQQpnpI UEhUqKcX9ZqfF0udd4cUim0B2gZAu9FWs9VKqt2GjpjN8KUV1op2a7YFNK0otb0X863u LRBcy904MnOUBdWB/y15h0svQVH606sLuTs6jEtPbBdN+0mFRi8bgijed/urKLV77muU oPzlgHR3Mdk9DfZewk1LeILSizD1+dGA1+qBz/2f+CWF45KNT6YSg/WUQGGmW6VjplPT nu9Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=MmssWY+3; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id x15-20020aa7940f000000b006630be2db05si3406475pfo.182.2023.06.21.00.32.58; Wed, 21 Jun 2023 00:33:11 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=MmssWY+3; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231149AbjFUGpp (ORCPT + 99 others); Wed, 21 Jun 2023 02:45:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39452 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231267AbjFUGo7 (ORCPT ); Wed, 21 Jun 2023 02:44:59 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1260B2112 for ; Tue, 20 Jun 2023 23:44:22 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-bc7267224adso10882628276.0 for ; Tue, 20 Jun 2023 23:44:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1687329861; x=1689921861; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=C8az5zJDH0ESFjwiqnExbJyBaF+tF0BTQf7Bxk21/Xo=; b=MmssWY+3YFktxIF/WA4KOY5DVa4UBDzQVE2mTxNFaVyKAMTQcPmzrUyKnRow1u5/1i 0meUrWB+7qdjGxGk/BZXzmzb1pQ53nSbLZW7KEXzmJ1wTWPeYpDaHRWw5MR6gg5fRS/t ku0AYHWl3On0+y1JE0YaBf7Aeap5dO2iKcztXzsyi5LQbiU3nBTbOOBGFhj+kVO1nPS/ L1ltlHWhVDA8Qr/1MoVmRNWP3cN11BDGjlQjTXU+uttVYyoXzTzGdapoPcBYaZqSXJL0 P6GwQQiyXHV7CzI5aqwk/8XCza5dBcBmpLoo5ytAswHLv7lm0GTilChwtaq7QlU0zcFb bo5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687329861; x=1689921861; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=C8az5zJDH0ESFjwiqnExbJyBaF+tF0BTQf7Bxk21/Xo=; b=daZGyYb/OLJiyxJkNjW3gKexX752r6/FAkASZygWU3wLmUSk4q0p6kwsxz32bZpIyH uvMvL+uP02nGF3ikYyXkTr3ghy0H+eEO/XMUh038ZY0ANC979352nGkAwhF9i3qqcO0l 6oDOaqmXkdo6/gDqTxsQh49W38c4Yco0tHaVsuJLW/5+ybl+hYipUtB+vrbZTcVjrFY3 mpIqYJYhR90heXt0k29vpsvcZPADHwIhdU/tK6Vt7nR3uKIcGdVS5HS7rCnE9vm1ytrc zCRhgHEUdpiYzF+/fG9dOb2ujAFV50xSw0tFgJjPLRoXnjnKYrRUXYQWdQeuobdDhysR 4Rnw== X-Gm-Message-State: AC+VfDzShRkFrVXtmQwJ4+FCS3Rd63bGicmUJ9PleoM+GsXXazN7hUww q83ZwgQv95DjGVBhqk7NcZdmYE1JlqT/ X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:384f:f7da:c61d:5a3e]) (user=mshavit job=sendgmr) by 2002:a05:6902:92:b0:bc4:a660:528f with SMTP id h18-20020a056902009200b00bc4a660528fmr3348263ybs.5.1687329861096; Tue, 20 Jun 2023 23:44:21 -0700 (PDT) Date: Wed, 21 Jun 2023 14:37:20 +0800 In-Reply-To: <20230621063825.268890-1-mshavit@google.com> Mime-Version: 1.0 References: <20230621063825.268890-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230621063825.268890-9-mshavit@google.com> Subject: [PATCH v4 08/13] iommu/arm-smmu-v3: Add helper for atc invalidation From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1769296669087545084?= X-GMAIL-MSGID: =?utf-8?q?1769296669087545084?= This will be used to invalidate ATC entries made on an SSID for a master when detaching a domain with pasid. Signed-off-by: Michael Shavit --- v1->v2: Make use of arm_smmu_atc_inv_cmd_set_ssid --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 65e2dfd28b7d8..0a5e875abda86 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1784,13 +1784,15 @@ arm_smmu_atc_inv_to_cmd(unsigned long iova, size_t size, cmd->atc.size = log2_span; } -static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) +static int arm_smmu_atc_inv_master_ssid(struct arm_smmu_master *master, + int ssid) { int i; struct arm_smmu_cmdq_ent cmd; struct arm_smmu_cmdq_batch cmds; arm_smmu_atc_inv_to_cmd(0, 0, &cmd); + arm_smmu_atc_inv_cmd_set_ssid(ssid, &cmd); cmds.num = 0; for (i = 0; i < master->num_streams; i++) { cmd.atc.sid = master->streams[i].id; @@ -1800,6 +1802,11 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) return arm_smmu_cmdq_batch_submit(master->smmu, &cmds); } +static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) +{ + return arm_smmu_atc_inv_master_ssid(master, 0); +} + /* * If ssid is non-zero, issue atc invalidations with the given ssid instead of * the one the domain is attached to. This is used by SVA since it's pasid From patchwork Wed Jun 21 06:37:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 110824 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp4180156vqr; Wed, 21 Jun 2023 00:24:05 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7iLktB4goyrjP1prsE/CtEeoTP3QWyiy2MyGaRoIkkvn0+QfsuopILUjsMlgTvWDeMKbxB X-Received: by 2002:a05:6808:1a08:b0:39a:bd3b:9cb3 with SMTP id bk8-20020a0568081a0800b0039abd3b9cb3mr19737119oib.23.1687332245554; Wed, 21 Jun 2023 00:24:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1687332245; cv=none; d=google.com; s=arc-20160816; b=TMnuAJFtvIqrlBs4UHUwRiQn64ut4sEeEqEGtI7z4OgS7TrOdqfUujOGktLAjjGPQ1 FYx70E/HspWjc1RHl7PW08Iv3xg5ZgG/XlLmu8JQNjCAaxRIGXSOLdP3Sf3sAnCSXBBR rherR5T31TPIAJZCarRrTHANxycBP0xTeZp/1xI5cc0DGSt3uo11A1Fp4azlV+HvkY9j M6Ot6Nvg8XvHAK+o/2NDt2DE8ttPgkYqiaj/RdZzKlawgzA4D9wySgKpcyXFetkq4LYh pqTwnuOmT19Uc5h7hqGdDaqQC3X4gLGfwBgkL/mjtFIBvSUgfYXIQt3+DRT5OwAlvx2K KPvg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=yj2G77LxXK3CUewBPucFF3sEDs9aMy8klQel7QUDOOo=; b=Y46eq17Ws02NuhLYAkZjZAVevFUxhnP+f8qazcD3h/mge/H9bP6io4n39ff5iZYuGx HOpCX4eFYNBfOGjtEAEQRXR+kgFhhQHzuqUMe1U1NVQIbEjxp0Urma3FtVkglpPAI0vf fYaSxxq7OfpkraHG4HoLMrKU3SFWdJjM6Uvg/uTU8i0hNQblzpiyLqX3paAJpo+z2pUT SdRJVsjLdMUgBPNWbIUTgnCCX2cEcqFhyg87ivGVLj8AJncFk9QEVE6MMJ4ddXbDATOV /NdCrkg1qpx9XWaIllj1KsgpUvhg4oHadI1EEEGyIvKFoDs322t7PV8QRTwprCk3Ahcj 1fcw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=AVFwDUMd; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id u71-20020a63854a000000b0053f32b910c1si3548677pgd.700.2023.06.21.00.23.45; Wed, 21 Jun 2023 00:24:05 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=AVFwDUMd; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230511AbjFUGqC (ORCPT + 99 others); Wed, 21 Jun 2023 02:46:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40626 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230489AbjFUGpe (ORCPT ); Wed, 21 Jun 2023 02:45:34 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CECFA26A2 for ; Tue, 20 Jun 2023 23:44:30 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-bb2a7308f21so7094005276.2 for ; Tue, 20 Jun 2023 23:44:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1687329869; x=1689921869; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=yj2G77LxXK3CUewBPucFF3sEDs9aMy8klQel7QUDOOo=; b=AVFwDUMdX9bm/1NAs/fP2460ErtAUK5gKa5D2fW3RzoiQ2i5K3I13ge21qxKjeNd3a wB3JqjD4MBapxFZNlWKOZkn9Z9Ps2U2DFeRXbm2iHFMcA5NOtca3EuKojlb8xFQiTMnh t5RdMnxU8ESVO6NCh1NGlXkPJJz7WM411mxVBP6tBx+vRZqeoCiln6p7c0lfLfpsWl0e jsz1vaaObWTpYfa5mpP0xLM3YRK6mcqHMxKnRxN1QOk+1DFiRqYKb10H3sXpY73yTVRF XbVFXrzPdekEKW6Dm35Vpy78ijKrkWqacRdb80QMb/w30C1UyJObegvDjmL/TzhUiTpb Hj7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687329869; x=1689921869; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=yj2G77LxXK3CUewBPucFF3sEDs9aMy8klQel7QUDOOo=; b=mB/4F1XBHPJ8cYgXZqM7L8EVUtxG3VsWLvTR6xQxYokZl4SihEJHn/kJ1kLtK5nnK7 w4BZt7jdm6b2MSXXnQq6vCZzASBXXFY0rhBlY8CF45A4C8Drt6hlI3PJkGHkoiJVWsgZ SxENxNSbBCeHM5v8RZq+8Um/BIrUWvE9/XeXlOZwhPh5BsOdtfmtdRSChnFd9z3Vo+6A wSD54sKOnuOyh5KkO+uNWRDFCTT4yyQjUiqR5NjCUaVPqCsjE+WoH6Z1ERltPjixPa17 TPJ17JlJjUTCx5bS7a0G+8tonT0K/j2tnKN3xEe50incADUdVAG5Lg8mUHYszkd7LLyL KfUw== X-Gm-Message-State: AC+VfDzOL8qA4vSPWo8/USJ2z7Ln7s8wOOk/LlfCLgnmIzWArqMohk5X WEe6qW/Zmn0odaKXzqlFSN1PBAdTlwKf X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:384f:f7da:c61d:5a3e]) (user=mshavit job=sendgmr) by 2002:a25:ce07:0:b0:bad:99d:f088 with SMTP id x7-20020a25ce07000000b00bad099df088mr5471090ybe.11.1687329869352; Tue, 20 Jun 2023 23:44:29 -0700 (PDT) Date: Wed, 21 Jun 2023 14:37:21 +0800 In-Reply-To: <20230621063825.268890-1-mshavit@google.com> Mime-Version: 1.0 References: <20230621063825.268890-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230621063825.268890-10-mshavit@google.com> Subject: [PATCH v4 09/13] iommu/arm-smmu-v3: Implement set_dev_pasid From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1769296096944383567?= X-GMAIL-MSGID: =?utf-8?q?1769296096944383567?= This change enables the use of the iommu_attach_dev_pasid API for UNMANAGED domains. The primary use-case is to allow in-kernel users of the iommu API to manage domains with PASID. This change also allows for future support of pasid in the DMA api. Signed-off-by: Michael Shavit --- v1->v2: Add missing atc invalidation when detaching with pasid --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 167 +++++++++++++++++--- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 1 + 2 files changed, 149 insertions(+), 19 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 0a5e875abda86..b928997d35ed3 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2173,6 +2173,10 @@ static int arm_smmu_domain_finalise_s2(struct arm_smmu_domain *smmu_domain, return 0; } +/* + * master may be null for domain types that are finalized before being attached + * to a master. + */ static int arm_smmu_domain_finalise(struct iommu_domain *domain, struct arm_smmu_master *master) { @@ -2369,6 +2373,11 @@ static int arm_smmu_enable_pasid(struct arm_smmu_master *master) return 0; } +static bool arm_smmu_master_has_pasid_domains(struct arm_smmu_master *master) +{ + return master->nr_attached_pasid_domains > 0; +} + static void arm_smmu_disable_pasid(struct arm_smmu_master *master) { struct pci_dev *pdev; @@ -2411,6 +2420,28 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) arm_smmu_install_ste_for_dev(master); } +/* + * Once attached for the first time, a domain can no longer be attached to any + * master with a distinct upstream SMMU. + */ +static int arm_smmu_prepare_domain_for_smmu(struct arm_smmu_device *smmu, + struct arm_smmu_domain *smmu_domain) +{ + int ret = 0; + + mutex_lock(&smmu_domain->init_mutex); + if (!smmu_domain->smmu) { + smmu_domain->smmu = smmu; + ret = arm_smmu_domain_finalise(&smmu_domain->domain, NULL); + if (ret) + smmu_domain->smmu = NULL; + } else if (smmu_domain->smmu != smmu) { + ret = -EINVAL; + } + mutex_unlock(&smmu_domain->init_mutex); + return ret; +} + static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) { int ret = 0; @@ -2426,6 +2457,10 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) master = dev_iommu_priv_get(dev); smmu = master->smmu; + ret = arm_smmu_prepare_domain_for_smmu(smmu, smmu_domain); + if (ret) + return ret; + /* * Checking that SVA is disabled ensures that this device isn't bound to * any mm, and can be safely detached from its old domain. Bonds cannot @@ -2436,22 +2471,18 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) return -EBUSY; } - arm_smmu_detach_dev(master); - - mutex_lock(&smmu_domain->init_mutex); - - if (!smmu_domain->smmu) { - smmu_domain->smmu = smmu; - ret = arm_smmu_domain_finalise(domain, master); - if (ret) { - smmu_domain->smmu = NULL; - goto out_unlock; - } - } else if (smmu_domain->smmu != smmu) { - ret = -EINVAL; - goto out_unlock; + /* + * Attaching a bypass or stage 2 domain would break any domains attached + * with pasid. Attaching an S1 domain should be feasible but requires + * more complicated logic to handle. + */ + if (arm_smmu_master_has_pasid_domains(master)) { + dev_err(dev, "cannot attach - domain attached with pasid\n"); + return -EBUSY; } + arm_smmu_detach_dev(master); + if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { master->s1_cfg = &master->owned_s1_cfg; ret = arm_smmu_write_ctx_desc( @@ -2460,7 +2491,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) 0, &smmu_domain->cd); if (ret) { master->s1_cfg = NULL; - goto out_unlock; + return ret; } } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S2 || smmu_domain->stage == ARM_SMMU_DOMAIN_NESTED) { @@ -2489,11 +2520,75 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) arm_smmu_enable_ats(master, smmu_domain); -out_unlock: - mutex_unlock(&smmu_domain->init_mutex); return ret; } +static int arm_smmu_set_dev_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid) +{ + int ret = 0; + unsigned long flags; + struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); + struct arm_smmu_device *smmu; + struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); + struct arm_smmu_attached_domain *attached_domain; + struct arm_smmu_master *master; + + if (!fwspec) + return -ENOENT; + + master = dev_iommu_priv_get(dev); + smmu = master->smmu; + + ret = arm_smmu_prepare_domain_for_smmu(smmu, smmu_domain); + if (ret) + return ret; + + if (pasid == 0) { + dev_err(dev, "pasid 0 is reserved for the device's primary domain\n"); + return -ENODEV; + } + + if (smmu_domain->stage != ARM_SMMU_DOMAIN_S1) { + dev_err(dev, "set_dev_pasid only supports stage 1 domains\n"); + return -EINVAL; + } + + if (!master->s1_cfg || master->s2_cfg) + return -EBUSY; + + attached_domain = kzalloc(sizeof(*attached_domain), GFP_KERNEL); + if (!attached_domain) + return -ENOMEM; + + attached_domain->master = master; + attached_domain->domain = smmu_domain; + attached_domain->ssid = pasid; + + master->nr_attached_pasid_domains += 1; + /* + * arm_smmu_share_asid may update the cd's asid value and write the + * ctx_desc for every attached_domains in the list. There's a potential + * race here regardless of whether we first write the ctx_desc or + * first insert into the domain's list. Grabbing the asic_lock prevents + * SVA from changing the cd's ASID while the cd is being attached. + */ + mutex_lock(&arm_smmu_asid_lock); + ret = arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, + pasid, &smmu_domain->cd); + if (ret) { + mutex_unlock(&arm_smmu_asid_lock); + kfree(attached_domain); + } + + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_add(&attached_domain->domain_head, &smmu_domain->attached_domains); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); + mutex_unlock(&arm_smmu_asid_lock); + + return 0; +} + static int arm_smmu_map_pages(struct iommu_domain *domain, unsigned long iova, phys_addr_t paddr, size_t pgsize, size_t pgcount, int prot, gfp_t gfp, size_t *mapped) @@ -2739,6 +2834,15 @@ static void arm_smmu_release_device(struct device *dev) if (WARN_ON(arm_smmu_master_sva_enabled(master))) iopf_queue_remove_device(master->smmu->evtq.iopf, dev); + if (WARN_ON(master->nr_attached_pasid_domains != 0)) { + /* + * TODO: Do we need to handle this case? + * This requires a mechanism to obtain all the pasid domains + * that this master is attached to so that we can clean up the + * domain's attached_domain list. + */ + } + arm_smmu_detach_dev(master); arm_smmu_free_cd_tables(master->smmu, &master->owned_s1_cfg.cdcfg); arm_smmu_disable_pasid(master); @@ -2874,12 +2978,36 @@ static int arm_smmu_def_domain_type(struct device *dev) static void arm_smmu_remove_dev_pasid(struct device *dev, ioasid_t pasid) { struct iommu_domain *domain; + struct arm_smmu_master *master = dev_iommu_priv_get(dev); + struct arm_smmu_domain *smmu_domain; + struct arm_smmu_attached_domain *attached_domain; + unsigned long flags; - domain = iommu_get_domain_for_dev_pasid(dev, pasid, IOMMU_DOMAIN_SVA); + if (!master || pasid == 0) + return; + + domain = iommu_get_domain_for_dev_pasid(dev, pasid, 0); if (WARN_ON(IS_ERR(domain)) || !domain) return; + if (domain->type == IOMMU_DOMAIN_SVA) + return arm_smmu_sva_remove_dev_pasid(domain, dev, pasid); - arm_smmu_sva_remove_dev_pasid(domain, dev, pasid); + smmu_domain = to_smmu_domain(domain); + mutex_lock(&arm_smmu_asid_lock); + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_for_each_entry(attached_domain, &smmu_domain->attached_domains, domain_head) { + if (attached_domain->master != master || + attached_domain->ssid != pasid) + continue; + list_del(&attached_domain->domain_head); + break; + } + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); + arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, pasid, + NULL); + arm_smmu_atc_inv_master_ssid(master, pasid); + master->nr_attached_pasid_domains -= 1; + mutex_unlock(&arm_smmu_asid_lock); } static struct iommu_ops arm_smmu_ops = { @@ -2899,6 +3027,7 @@ static struct iommu_ops arm_smmu_ops = { .owner = THIS_MODULE, .default_domain_ops = &(const struct iommu_domain_ops) { .attach_dev = arm_smmu_attach_dev, + .set_dev_pasid = arm_smmu_set_dev_pasid, .map_pages = arm_smmu_map_pages, .unmap_pages = arm_smmu_unmap_pages, .flush_iotlb_all = arm_smmu_flush_iotlb_all, diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 6929590530367..48795a7287b69 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -707,6 +707,7 @@ struct arm_smmu_master { bool iopf_enabled; struct list_head bonds; unsigned int ssid_bits; + unsigned int nr_attached_pasid_domains; }; /* SMMU private data for an IOMMU domain */ From patchwork Wed Jun 21 06:37:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 110833 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp4181610vqr; Wed, 21 Jun 2023 00:27:41 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7SHWBz2+BeIoZM09yX8Qx7L270JX6MUknSDP6jbs4Nt9f4FRAo92s2oS/iM72X3it5mO3I X-Received: by 2002:a05:6a20:2590:b0:119:87f9:3b04 with SMTP id k16-20020a056a20259000b0011987f93b04mr11920627pzd.6.1687332460657; Wed, 21 Jun 2023 00:27:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1687332460; cv=none; d=google.com; s=arc-20160816; b=qKhmIsevlWlJHT2Z5FBacsIQWZBSaA9V9NAez5bU9TjV4FuDtYG0iHwjed0irsuHgn H2AbPTXuOfV+HO2OhMuS4D09PdlJHCw/AjokWZojhJAppmM/QD1sdWQHV+NgPbxYe2s1 WR/+h8oO9ADl57r2DrZAhd5fYFl8SicGlXFUv+jyvNUdPQaKEoYZ4AkbxKn5ki3ITymL L5C4gI/sxS1VZZd49hFxSTMfSFIYQAGKO3NxWMKyRYpxWMdQDW146zoYwjUIONUYt9B8 SxJkJRP0m24o8ivN01A2NVRXg9O7hQgMNQsgp9LybtcXBnZMok38n7lbq8kk/wz5mOtM baKQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=tvomj7iW2UACU6/yECNbFWEXmb4EH1aV19zctsWiNw0=; b=QARXS1jh+bFlJE7Sq9+x3Pn+lcR6QYmYXSzpu5CaC3v9Fte45PQCrZWc+vgZMOtTDS Q615V8iY9sBL1QewcGOdrNEAz1wnlrot2RUfdlA78Inr3GtU7/y5Sr1s92X73ixhnHUw 5tfwsbHYy68E7tnXRQZq2rjFWDwS2OoHsOvGEBLGpXnrOLfsnE6rRUsjskO3Rzug7QPU RVBo9yrWZP8APxBp8ccK0wIkj4xw/4GG7PFapWMJS+OfHbtyQn1/ZPu2QsJZIBrg/kSh Ty6tW7Sogcp1gw/nyD1/galaBJvZjX+0IwjWvEma/uEZm1BikTOyWaeX75HmQ1x84trr q9BA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=bFIfZORz; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id d19-20020aa797b3000000b00666887b7eecsi1611284pfq.312.2023.06.21.00.27.27; Wed, 21 Jun 2023 00:27:40 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=bFIfZORz; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231166AbjFUGqX (ORCPT + 99 others); Wed, 21 Jun 2023 02:46:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40362 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231164AbjFUGpu (ORCPT ); Wed, 21 Jun 2023 02:45:50 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EF1681BD0 for ; Tue, 20 Jun 2023 23:44:39 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-bfec07e5eb0so736401276.2 for ; Tue, 20 Jun 2023 23:44:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1687329877; x=1689921877; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=tvomj7iW2UACU6/yECNbFWEXmb4EH1aV19zctsWiNw0=; b=bFIfZORz4qlyvHdesSBGaoACwrXyukUmDFr+sNMMwiGcbhJJOqX5vCINTCGVK/kqb1 M2pwDMPHBCg5EsNV8co3V/w8l6vhX4vTin9L9Y5GtkOrDe1gld2dyDd8Gs1HqLfNwdN1 G1FtBdPsy2NHnG2IC3hIdVsnPTWng2eGQ+qtojfzCdm1ORIt9R40o+eakI5Ec4ojnwqT F2VattLO/4nUSbZD2O/jFgdRuiN0pvkwRjTfp2vimZsBjB7peBN0fhLflu4T05WVZiO1 hJkx9W1oX0Pcb7kL4Ccd0fi2GEjiJaL7AxA3aaF5kFJsUz+3Ce+yO5GZBwoTHZZ/M+Ot GroQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687329877; x=1689921877; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=tvomj7iW2UACU6/yECNbFWEXmb4EH1aV19zctsWiNw0=; b=fwba4Jk3WNHBFx8nCdnOkJ1roBJuHKqxT/UBVVwH7ANOVfaAiPImnTVorp/4rNN+Cy u3wMvNWOp9qJoCMTLZ8mfAh/xwLHLhbDeNZ4Tdx4RvwvWTvrDW8qyXLqw6tNnCTTVkD9 yY0CsB6BngJbfsYTGlGW8VQBcbdbVnEAFZL/73cM+4CK4Gz598gO0rupEp3cIPNjv1JJ 89jE8sBrljRUIcnpfk2ceab4E/elJiDTiWmef89JBhY8yVt8yGOLeYYGExxSqTY1m+9q Gb2wXcmfl9H+t02fexZtEdr5fCHbE6HnpMyCs6B8OIA1DIcHX2LZYLu3o7XwkSvH46PK q/Yw== X-Gm-Message-State: AC+VfDzLV7Sx2H3enPRzQISEDtf0OJVO/RagjNrp7X3mRccC6MIRW2lt a1rulwyBFRqO6NReiDsFy0nlrRwAFePA X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:384f:f7da:c61d:5a3e]) (user=mshavit job=sendgmr) by 2002:a25:d658:0:b0:bbb:8c13:ce26 with SMTP id n85-20020a25d658000000b00bbb8c13ce26mr5885974ybg.11.1687329877614; Tue, 20 Jun 2023 23:44:37 -0700 (PDT) Date: Wed, 21 Jun 2023 14:37:22 +0800 In-Reply-To: <20230621063825.268890-1-mshavit@google.com> Mime-Version: 1.0 References: <20230621063825.268890-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230621063825.268890-11-mshavit@google.com> Subject: [PATCH v4 10/13] iommu/arm-smmu-v3-sva: Remove bond refcount From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED, USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1769296322461034414?= X-GMAIL-MSGID: =?utf-8?q?1769296322461034414?= The iommu-sva framework checks if a bond between a device and mm already exists and handles refcounting at the iommu_domain level. __arm_smmu_sva_bind is therefore only called once for a device/mm pair. Signed-off-by: Michael Shavit --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 12 +----------- 1 file changed, 1 insertion(+), 11 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index d07c08b53c5cf..20301d0a2c0b0 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -29,7 +29,6 @@ struct arm_smmu_bond { struct mm_struct *mm; struct arm_smmu_mmu_notifier *smmu_mn; struct list_head list; - refcount_t refs; }; #define sva_to_bond(handle) \ @@ -377,21 +376,12 @@ __arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) if (!master || !master->sva_enabled) return ERR_PTR(-ENODEV); - /* If bind() was already called for this {dev, mm} pair, reuse it. */ - list_for_each_entry(bond, &master->bonds, list) { - if (bond->mm == mm) { - refcount_inc(&bond->refs); - return &bond->sva; - } - } - bond = kzalloc(sizeof(*bond), GFP_KERNEL); if (!bond) return ERR_PTR(-ENOMEM); bond->mm = mm; bond->sva.dev = dev; - refcount_set(&bond->refs, 1); bond->smmu_mn = arm_smmu_mmu_notifier_get(smmu_domain, mm); if (IS_ERR(bond->smmu_mn)) { @@ -570,7 +560,7 @@ void arm_smmu_sva_remove_dev_pasid(struct iommu_domain *domain, } } - if (!WARN_ON(!bond) && refcount_dec_and_test(&bond->refs)) { + if (!WARN_ON(!bond)) { list_del(&bond->list); arm_smmu_mmu_notifier_put(bond->smmu_mn); kfree(bond); From patchwork Wed Jun 21 06:37:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 110828 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp4180567vqr; Wed, 21 Jun 2023 00:25:11 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7qGClndAAIQOwLFDXagXTNl5XznH30oi5UW1IkwPqcz7Pb+y/LicqnraVFz+h5gp2I7k1I X-Received: by 2002:a05:6808:1782:b0:398:54a2:a9bd with SMTP id bg2-20020a056808178200b0039854a2a9bdmr12567642oib.46.1687332311307; Wed, 21 Jun 2023 00:25:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1687332311; cv=none; d=google.com; s=arc-20160816; b=htSXR6eLnzjwOEZMOwvQyaLxez+jxQ97U1hJODhktRFpSXyuCyVmVIJH2kTZdeRQ5B hV1cG7nkCQQ/lHOT4z1tbG03jGqEFKwG1anSo681rnB6G27THIkEJM5uVh8OKFkYp6m/ 54OsDA7jEQ0Px92lIproZonnU0mrnmwet44rvF56uTVgH/wIbhwj0lqWhzx1GBvVaKes y0Xixp4S1nMMP3xVbviZqCpAQIDUxarADVISLzj/7goYSBLFx700AFy1soaBSluXoXbt U2vnK1x00ax67IYG95Jc3U+4VhMyb5L7YHDQcMGU2Wqi4qOaHdrkND4wXuRKeuU2DLB8 UejQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=1HNFofEgftQ/0viD8hP2C9StxNG8tEEj8VIgof4alao=; b=Jrf8bz94gUe//ZzXmuEfNl3tnztKqAGuDmaVfq52ZNMzbPymJ6G/EcPIf13QgIK+LW 8x9n8UoseElNHMPRdID6JSkicnWa4cyLijvEd12p9tnk9gtv5nKl1HGJoZa+lRsvGmJx mWTNd2/YtKqQqadUmADgxUpmJWpCVwjntIXEFQlDKlwRpgOFa65eKsCn5dM6vFdsYuEX b7kfQ85VJOWWsqwLG/ArivXZEm6GO1uZ9u2vFo1jwn4niQwIYiQ0e5ZZPBuh64yh0GtB 7N/THYHfJKajmhvoq9OlGtF9mb2Hx+d1komVvQHqUCDIIr18W8Csj8DIdpjMeglwBFya TDGQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=NwyrXz6u; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id p22-20020a637f56000000b00543d32ee960si3585976pgn.594.2023.06.21.00.24.58; Wed, 21 Jun 2023 00:25:11 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=NwyrXz6u; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230013AbjFUGqo (ORCPT + 99 others); Wed, 21 Jun 2023 02:46:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39676 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229709AbjFUGp6 (ORCPT ); Wed, 21 Jun 2023 02:45:58 -0400 Received: from mail-yw1-x114a.google.com (mail-yw1-x114a.google.com [IPv6:2607:f8b0:4864:20::114a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A11821BEF for ; Tue, 20 Jun 2023 23:44:49 -0700 (PDT) Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-57000a7480bso77142067b3.0 for ; Tue, 20 Jun 2023 23:44:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1687329886; x=1689921886; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=1HNFofEgftQ/0viD8hP2C9StxNG8tEEj8VIgof4alao=; b=NwyrXz6ubgbYeu5se4OnmTGRl/ECZnN/7JXUGfb3yfJHIleuEhl/vBOWZdfzMER6Gx BiPVvWA9Zp+4QaWrnn9OAzgPxD8EtCxqeits8Wu66pkUEHOGaiwDQhRW/J+yNdDMsAlF y8tREQZuazgzoLW1lJOdMAgC3jROf3yU6f6/ooGUM/ed0cdMK69TTIuHL8pi8+hdgxg4 sMmuWLwUHHqnGxYMofKOeij/4hhaVO7yrHVtyCzzPIP3fz7HtdUP47sUq7HkSrrLt0u4 AKIy0MweTpJJB47zOUsszP8BDsLy6b518MCCeqGT4lRkc+iJWu0D/oSX+S9wtNgHByFL IhfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687329886; x=1689921886; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=1HNFofEgftQ/0viD8hP2C9StxNG8tEEj8VIgof4alao=; b=SKzXZyh35Cr1YMOXR+EXdWO+xgDMmPS4D+8LsxokkAy6uZucnRKFvds54lw/JQJO6U ojMWq/I3jaBbY+VjcboINOYJIHGyGXr9tJ46fPamF7MB30q2MjM0PmSG8ooBkVUI3ZQM YmWIsU20MqR13EUuszf8hj2aPxaqwl3pq5JNcezMuYfxRzrmDmicTx2WVOf0+CsbfDdj iBLlT26z0ChNQuTOoCZr/VaKzkMhQb5OT0KtmhIHXAAK9hMG2q8LFIgmRXsGWWpVJJLO T5O0kc4XzdY3oYKVnaJHDEnREI5tcusSIvwWfJQgN/dwJfTVJZFngMEDM4qT13zPgn0u iwBg== X-Gm-Message-State: AC+VfDwsf8GxCU26TY7vyDuOscecljKee145KOosNz4qjy8AdZKYRfun eYNJdxHh51kY9GyiNpaVBh9E4/e5ZnL/ X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:384f:f7da:c61d:5a3e]) (user=mshavit job=sendgmr) by 2002:a81:bc4d:0:b0:56f:2841:aa24 with SMTP id b13-20020a81bc4d000000b0056f2841aa24mr6215155ywl.10.1687329885894; Tue, 20 Jun 2023 23:44:45 -0700 (PDT) Date: Wed, 21 Jun 2023 14:37:23 +0800 In-Reply-To: <20230621063825.268890-1-mshavit@google.com> Mime-Version: 1.0 References: <20230621063825.268890-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230621063825.268890-12-mshavit@google.com> Subject: [PATCH v4 11/13] iommu/arm-smmu-v3-sva: Clean unused iommu_sva From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED, USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1769296165499560607?= X-GMAIL-MSGID: =?utf-8?q?1769296165499560607?= The __arm_smmu_sva_bind function returned an unused iommu_sva handle that can be removed. Signed-off-by: Michael Shavit --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 18 ++++++------------ 1 file changed, 6 insertions(+), 12 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 20301d0a2c0b0..650c9c9ad52f1 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -25,7 +25,6 @@ struct arm_smmu_mmu_notifier { #define mn_to_smmu(mn) container_of(mn, struct arm_smmu_mmu_notifier, mn) struct arm_smmu_bond { - struct iommu_sva sva; struct mm_struct *mm; struct arm_smmu_mmu_notifier *smmu_mn; struct list_head list; @@ -364,8 +363,7 @@ static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) arm_smmu_free_shared_cd(cd); } -static struct iommu_sva * -__arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) +static int __arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) { int ret; struct arm_smmu_bond *bond; @@ -374,14 +372,13 @@ __arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); if (!master || !master->sva_enabled) - return ERR_PTR(-ENODEV); + return -ENODEV; bond = kzalloc(sizeof(*bond), GFP_KERNEL); if (!bond) - return ERR_PTR(-ENOMEM); + return -ENOMEM; bond->mm = mm; - bond->sva.dev = dev; bond->smmu_mn = arm_smmu_mmu_notifier_get(smmu_domain, mm); if (IS_ERR(bond->smmu_mn)) { @@ -390,11 +387,11 @@ __arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) } list_add(&bond->list, &master->bonds); - return &bond->sva; + return 0; err_free_bond: kfree(bond); - return ERR_PTR(ret); + return ret; } bool arm_smmu_sva_supported(struct arm_smmu_device *smmu) @@ -572,13 +569,10 @@ static int arm_smmu_sva_set_dev_pasid(struct iommu_domain *domain, struct device *dev, ioasid_t id) { int ret = 0; - struct iommu_sva *handle; struct mm_struct *mm = domain->mm; mutex_lock(&sva_lock); - handle = __arm_smmu_sva_bind(dev, mm); - if (IS_ERR(handle)) - ret = PTR_ERR(handle); + ret = __arm_smmu_sva_bind(dev, mm); mutex_unlock(&sva_lock); return ret; From patchwork Wed Jun 21 06:37:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 110847 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp4196377vqr; Wed, 21 Jun 2023 01:03:15 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4DUCmMs3Yg+qbC4F5VJANeJQ8ZPfYTRQxlGm2G7Rj0/T0wXFrQqusB9Sr4XrA6jxF/BdWY X-Received: by 2002:a05:6a20:442a:b0:110:6146:1056 with SMTP id ce42-20020a056a20442a00b0011061461056mr22270967pzb.2.1687334595143; Wed, 21 Jun 2023 01:03:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1687334595; cv=none; d=google.com; s=arc-20160816; b=SayPD05d18gml+GrvNkKQL4S3kML5e175fBpr8kJV5UmNBCaHlRNeR+f+GMb0Fywdv o1+msKBN7dqbZ4QJZt8ZkHqdYBl3rNkBhww+EV0JjoS5PRSwbc4InUbAwiFoh5XpA2p0 lAXQLzUPIXEicfIxPYr1TRR6opFNt1wl/jpNUzUDruhvoZsulsypeHhLcuPW65ifnwTs SvzX61Ln3BahKNO0G21qUwxJ/b8rRMs0R5HR27DNW3IGSJiPmTiDHKqcEbZfGNK+CsgR Zr/BlfA9vRb+UqwIygHD5mwqA9NO/WlNg3Tmq3rrSmlXQqF8qV3aKyJfzVJdVCpkk/gC hAQA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=Iaiq8BTRsoloLG7rldnwk3G52sAWylq1kN+2LzQv5tM=; b=RPepeHK12uUi+CgBV3delgoMLMlrendMmzbdwFLifQhSPz0QrzjTks7mImpq3BN4cv Cs9/GY42kfUToUmu7P7eMOcvsX2HjxvnhcLdcTxoPDcLKP7v50EpozFbSL9S2HX/VO+P VbGiZ4OuwgIFqg0dvamz4uYJcK8iEws0K1RzDBQV20+tQDk08Bk3bt6nz0yXa1RaPWTt eac2SVP4YdXNdSNOBjRnLRjxPmQFXhBWBpo2MWt+gul1Rg0exAHSLiWQYgS20uDeCr7h fqOMfVUFr74OQUbWmkM1yxpr5ag41y/3Kw+5Kt92+ywlBgQS4yldljhqs/LosFWkxuZ4 IwFw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=f2p4RRe6; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id h3-20020aa796c3000000b0064d41566266si533918pfq.247.2023.06.21.01.02.59; Wed, 21 Jun 2023 01:03:15 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=f2p4RRe6; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231146AbjFUGq4 (ORCPT + 99 others); Wed, 21 Jun 2023 02:46:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40014 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231145AbjFUGqS (ORCPT ); Wed, 21 Jun 2023 02:46:18 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 728401992 for ; Tue, 20 Jun 2023 23:45:00 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-bce829aca1bso6009205276.3 for ; Tue, 20 Jun 2023 23:45:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1687329893; x=1689921893; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Iaiq8BTRsoloLG7rldnwk3G52sAWylq1kN+2LzQv5tM=; b=f2p4RRe6xTN7igEKKPHZGsDF92MIFONOcBrriUfaGfiU3ABD3qhOoJJOHan/Szh8sJ Kph4EWS3o0wtAioESPtVkWwEA9xFadSpa4206f5f3djBGbzfWOB1yGO3McRAFLPpCABM fvhSEzmduefkQzfFvvyoq4Ij4xlBU9yXRK8Lkbeizarjd2Ow4YKI2mXssopkMc4dGBkD uAd3/uApx4hTbP0KqhvPL1hTILkWXfRdIqXtfMtN7PIPEbOyvMIFFJeq1AN2pA1kVws8 6R0dGFwCrLR2eZbdJbjNe6bcjYYwXVRTcsXyD48CL4OOgvxLnLwqOdLIqmLyHh29irfJ cesA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687329894; x=1689921894; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Iaiq8BTRsoloLG7rldnwk3G52sAWylq1kN+2LzQv5tM=; b=LjkyYlGoa17PLbGE6pIDhuH0GgDfJzIzBP/S25H6MG/AsPb7BvpWIAZ6TYAkZ2RejR ico26b8FfVIrhtnEpo3ckvlvyUtjLa7hhaFXsJoxDKO7v8TBq8iTR10ih5VeQCiJOXyz NdZFxO3jpwSMq1tNwWNJavmH5UGdYOyISCWcmRVcpGQ6T+kZNBjGzmw9nKhqEPPo6j1j JC7T3UCj8MFtG0J4XUQhi5Bz5Ci12zU2ntnJjLkF7xgYhiB4hjq2Fe6JD6IH5P1cYyE7 j31nTvFZD7xoobdi2Uw9hNLIdGaBqiARAXU1+PyEb8SpUagNNk8yISHaQ8rTWMBT+ubw Ijzg== X-Gm-Message-State: AC+VfDyMkCeqpcZ1eWNjfBJZ2t78tKAoPzbSszAH6wl18VOEc01lDDrD 0ibfrkT7oybOLFmyWbzDwTj+ITLdJjhj X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:384f:f7da:c61d:5a3e]) (user=mshavit job=sendgmr) by 2002:a25:882:0:b0:bfe:ea69:91b4 with SMTP id 124-20020a250882000000b00bfeea6991b4mr167148ybi.4.1687329893822; Tue, 20 Jun 2023 23:44:53 -0700 (PDT) Date: Wed, 21 Jun 2023 14:37:24 +0800 In-Reply-To: <20230621063825.268890-1-mshavit@google.com> Mime-Version: 1.0 References: <20230621063825.268890-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230621063825.268890-13-mshavit@google.com> Subject: [PATCH v4 12/13] iommu/arm-smmu-v3-sva: Remove arm_smmu_bond From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1769298560495004102?= X-GMAIL-MSGID: =?utf-8?q?1769298560495004102?= There's a 1:1 relationship between arm_smmu_bond and the iommu_domain used in set_dev_pasid/remove_dev_pasid. arm_smmu_bond has become an unnecessary complication. It's more natural to store any needed information at the iommu_domain container level. Signed-off-by: Michael Shavit --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 69 +++++++------------ drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 1 - drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 2 +- 3 files changed, 24 insertions(+), 48 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 650c9c9ad52f1..b615a85e6a54e 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -24,14 +24,13 @@ struct arm_smmu_mmu_notifier { #define mn_to_smmu(mn) container_of(mn, struct arm_smmu_mmu_notifier, mn) -struct arm_smmu_bond { - struct mm_struct *mm; +struct arm_smmu_sva_domain { + struct iommu_domain iommu_domain; struct arm_smmu_mmu_notifier *smmu_mn; - struct list_head list; }; -#define sva_to_bond(handle) \ - container_of(handle, struct arm_smmu_bond, sva) +#define to_sva_domain(domain) \ + container_of(domain, struct arm_smmu_sva_domain, iommu_domain) static DEFINE_MUTEX(sva_lock); @@ -363,10 +362,10 @@ static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) arm_smmu_free_shared_cd(cd); } -static int __arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) +static int __arm_smmu_sva_bind(struct device *dev, + struct arm_smmu_sva_domain *sva_domain, + struct mm_struct *mm) { - int ret; - struct arm_smmu_bond *bond; struct arm_smmu_master *master = dev_iommu_priv_get(dev); struct iommu_domain *domain = iommu_get_domain_for_dev(dev); struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); @@ -374,24 +373,14 @@ static int __arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) if (!master || !master->sva_enabled) return -ENODEV; - bond = kzalloc(sizeof(*bond), GFP_KERNEL); - if (!bond) - return -ENOMEM; - - bond->mm = mm; - - bond->smmu_mn = arm_smmu_mmu_notifier_get(smmu_domain, mm); - if (IS_ERR(bond->smmu_mn)) { - ret = PTR_ERR(bond->smmu_mn); - goto err_free_bond; + sva_domain->smmu_mn = arm_smmu_mmu_notifier_get(smmu_domain, + mm); + if (IS_ERR(sva_domain->smmu_mn)) { + sva_domain->smmu_mn = NULL; + return PTR_ERR(sva_domain->smmu_mn); } - - list_add(&bond->list, &master->bonds); + master->nr_attached_sva_domains += 1; return 0; - -err_free_bond: - kfree(bond); - return ret; } bool arm_smmu_sva_supported(struct arm_smmu_device *smmu) @@ -521,7 +510,7 @@ int arm_smmu_master_enable_sva(struct arm_smmu_master *master) int arm_smmu_master_disable_sva(struct arm_smmu_master *master) { mutex_lock(&sva_lock); - if (!list_empty(&master->bonds)) { + if (master->nr_attached_sva_domains != 0) { dev_err(master->dev, "cannot disable SVA, device is bound\n"); mutex_unlock(&sva_lock); return -EBUSY; @@ -545,23 +534,12 @@ void arm_smmu_sva_notifier_synchronize(void) void arm_smmu_sva_remove_dev_pasid(struct iommu_domain *domain, struct device *dev, ioasid_t id) { - struct mm_struct *mm = domain->mm; - struct arm_smmu_bond *bond = NULL, *t; + struct arm_smmu_sva_domain *sva_domain = to_sva_domain(domain); struct arm_smmu_master *master = dev_iommu_priv_get(dev); mutex_lock(&sva_lock); - list_for_each_entry(t, &master->bonds, list) { - if (t->mm == mm) { - bond = t; - break; - } - } - - if (!WARN_ON(!bond)) { - list_del(&bond->list); - arm_smmu_mmu_notifier_put(bond->smmu_mn); - kfree(bond); - } + master->nr_attached_sva_domains -= 1; + arm_smmu_mmu_notifier_put(sva_domain->smmu_mn); mutex_unlock(&sva_lock); } @@ -572,7 +550,7 @@ static int arm_smmu_sva_set_dev_pasid(struct iommu_domain *domain, struct mm_struct *mm = domain->mm; mutex_lock(&sva_lock); - ret = __arm_smmu_sva_bind(dev, mm); + ret = __arm_smmu_sva_bind(dev, to_sva_domain(domain), mm); mutex_unlock(&sva_lock); return ret; @@ -590,12 +568,11 @@ static const struct iommu_domain_ops arm_smmu_sva_domain_ops = { struct iommu_domain *arm_smmu_sva_domain_alloc(void) { - struct iommu_domain *domain; + struct arm_smmu_sva_domain *sva_domain; - domain = kzalloc(sizeof(*domain), GFP_KERNEL); - if (!domain) + sva_domain = kzalloc(sizeof(*sva_domain), GFP_KERNEL); + if (!sva_domain) return NULL; - domain->ops = &arm_smmu_sva_domain_ops; - - return domain; + sva_domain->iommu_domain.ops = &arm_smmu_sva_domain_ops; + return &sva_domain->iommu_domain; } diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index b928997d35ed3..2ed6c9e0df241 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2784,7 +2784,6 @@ static struct iommu_device *arm_smmu_probe_device(struct device *dev) master->dev = dev; master->smmu = smmu; - INIT_LIST_HEAD(&master->bonds); dev_iommu_priv_set(dev, master); ret = arm_smmu_insert_master(smmu, master); diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 48795a7287b69..3525d60668c23 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -705,7 +705,7 @@ struct arm_smmu_master { bool stall_enabled; bool sva_enabled; bool iopf_enabled; - struct list_head bonds; + unsigned int nr_attached_sva_domains; unsigned int ssid_bits; unsigned int nr_attached_pasid_domains; }; From patchwork Wed Jun 21 06:37:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 110850 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp4198066vqr; Wed, 21 Jun 2023 01:06:34 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6EqJCQTWLZMpkI8kXJHAU0+LkClgs3AoL+nXDz0ZnlyrcEcVR6/R5AaJjorU9d1P4fZ4cr X-Received: by 2002:a05:6a00:2303:b0:668:8c3c:313e with SMTP id h3-20020a056a00230300b006688c3c313emr6119753pfh.22.1687334793799; Wed, 21 Jun 2023 01:06:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1687334793; cv=none; d=google.com; s=arc-20160816; b=Y/kA300dRD0h0L+mtDaDt+kGyksvwWsZLAGaYRpNNtsWed0r6u/N7A9FOwfr+dynkc psXOzBhxh0de1P8Q4enyHY/40GmobMc3C9lvVwPIOjyUP2OdDwSCA5+rEEY6tH02BmiX zabuz1yAYkhukMrOHPUOPWak+OkOAu7HBhhKul8bCh+rh9g278Ui/TyqCgeoB4OYI6OL HksrzozNxyAGph7ThM5vO7uYRR6WN3xmQ0717s6E5/w2F7PncbUpCeefsawsECY9XrwS jKdycgs0+/CA2LB7dic/LS+DQ8xCY5zQvBGq2AYmvkIFzebvLDA7s/MTu/WiHVEffMsi WiAQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=GFfBSOUvUPqFSx6n4vEtFQq3Hc8Ha+kDvCzi3+3mgos=; b=osNwrW/Efp0XBLfuw+KwB2ck9Di1AO7wVh3l/tiJO0pBj9uQBq7yTeIye8vFIKci5V nSS31wzKFApVPiTs09qsLPY1puUk1Ia1AZHSEhOoSN6iQHlajcawXLBQNCA6Uf3ZvHG5 AiuygmepbPt9lRYdvo9wAT5Cs0U0f6v0gL1RmilYz+j71jzmXcwlFIqc1K4L9WtGCRMH LkjB5LU9IDvDlxBnr0YmIcsbxkBQIFQ9MS8blcp04bw56rB0Q+g/sd5KeLbw3NUeQlqU m4GwQyZJfpsOk2G1VXmi78TjC+62DAp4HyufbtD4thri2putfayb2ARLkJMAukIHYfL/ q6Aw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=t4HhyrUZ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id e28-20020a056a0000dc00b00659af029c39si3454761pfj.90.2023.06.21.01.06.20; Wed, 21 Jun 2023 01:06:33 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=t4HhyrUZ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231165AbjFUGrB (ORCPT + 99 others); Wed, 21 Jun 2023 02:47:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40338 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230170AbjFUGqT (ORCPT ); Wed, 21 Jun 2023 02:46:19 -0400 Received: from mail-pj1-x1049.google.com (mail-pj1-x1049.google.com [IPv6:2607:f8b0:4864:20::1049]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5DE631FC4 for ; Tue, 20 Jun 2023 23:45:02 -0700 (PDT) Received: by mail-pj1-x1049.google.com with SMTP id 98e67ed59e1d1-25e89d51bf2so2581661a91.0 for ; Tue, 20 Jun 2023 23:45:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1687329902; x=1689921902; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=GFfBSOUvUPqFSx6n4vEtFQq3Hc8Ha+kDvCzi3+3mgos=; b=t4HhyrUZx343POQssx8xWB3bNqEwvVz5axTthBdUwUeHP98YwM0/jOGBkI8RNJ6CfV q813TeRnvgUNnvQ5yy+drxSkGdqSae/ZdfJL2CLBJbfVgemKrHUzyxNImNn9WE4R5pN1 QgALlayI2ddPtiW6CPlWgIpz9CmCsE/e1EJarXWy6qGQnc+hIy5agSCGPnJEUO2/Lcx0 w31wnEnCEJrXmNRkg29aSfwTjpHDwxP6w/ngMobxEgWSJDGOmFnCq3Hf+OQierGh/m/V vjKc3DIyPkMR9sdzlVpuqyKBLwgRtHl253JAEGraYWPxylZDrHFJArJHYf6fcfRkfMfq s4QQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687329902; x=1689921902; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=GFfBSOUvUPqFSx6n4vEtFQq3Hc8Ha+kDvCzi3+3mgos=; b=GCKtJzarPQ3nonHalz0+J0b9efmwtlITGzAt09JU1Ja4BmKwmM/Lcp4uaR7vPIhr2N e7k9YOrYY2Tdpu8k7ICM4X35APA+BUrCP6CKgxHxTJAHG4qWcSHQrIdy8WQKnqKqQmg4 TUoCLvGNNJTlniK/9KqXNTAgFozaR1L+zazEwswGq9f58g4GGr6U35Z5XxshsGTdzyMH akh2NmdnnyJaNmM25M6E9eLiI4QjUgP7OE6ypAxmNkodg8aTk99Kbt3V/2EVUbfKD9Rd b5nCsS02k56VZ3K79ynJhbiWnnXjV8FcW5sLIpD4PRYY2A30853rYUn2cIBUnkDGReS5 pwfQ== X-Gm-Message-State: AC+VfDwAr2r3LDkDU/dvcl+GmPHNtFbzcSpO6xebca/HobIPsO8uf8Gq R/xQ8juK0SKheN9LTuOMkNtR84lHZSMq X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:384f:f7da:c61d:5a3e]) (user=mshavit job=sendgmr) by 2002:a17:90a:fa06:b0:25b:d689:72a4 with SMTP id cm6-20020a17090afa0600b0025bd68972a4mr1992026pjb.4.1687329901714; Tue, 20 Jun 2023 23:45:01 -0700 (PDT) Date: Wed, 21 Jun 2023 14:37:25 +0800 In-Reply-To: <20230621063825.268890-1-mshavit@google.com> Mime-Version: 1.0 References: <20230621063825.268890-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230621063825.268890-14-mshavit@google.com> Subject: [PATCH v4 13/13] iommu/arm-smmu-v3-sva: Add check when enabling sva From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1769298768901258983?= X-GMAIL-MSGID: =?utf-8?q?1769298768901258983?= SVA domains can only be attached when the master's STEs have a stage 1 domain. Signed-off-by: Michael Shavit --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index b615a85e6a54e..e2a91f20f0906 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -499,9 +499,15 @@ int arm_smmu_master_enable_sva(struct arm_smmu_master *master) int ret; mutex_lock(&sva_lock); + + if (!master->s1_cfg) { + ret = -EBUSY; + goto unlock; + } ret = arm_smmu_master_sva_enable_iopf(master); if (!ret) master->sva_enabled = true; +unlock: mutex_unlock(&sva_lock); return ret;