From patchwork Tue Oct 17 03:20:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Zhang, Tina" X-Patchwork-Id: 15462 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp3872284vqb; Mon, 16 Oct 2023 20:21:43 -0700 (PDT) X-Google-Smtp-Source: AGHT+IESK91AAvgG4zuq3PE9yC6qoV2wsl8ZQ9sZY6u5Zp/h6BhFteDWV8g2Jz+gbr7sz6ILfNsy X-Received: by 2002:a17:902:7893:b0:1c9:e121:ccc1 with SMTP id q19-20020a170902789300b001c9e121ccc1mr1156299pll.5.1697512903736; Mon, 16 Oct 2023 20:21:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697512903; cv=none; d=google.com; s=arc-20160816; b=DvEEUpeXpOzvTIwl3EFHrK0vc5YR+J6xJftTTRRSxrCQBUQ1kjDHW/NvjRyk39Z8O2 G40neSDMYI07p5JYTgE/p6qfQROsOULfsswgDRLKNES1QgO5bfrwFnuPGU7Sth7YZRUv VbkxjiqFOH0p+O3rxjoSDN2mlvlnVoiyTrO1tUjUqq7XJMEWsVqoqR27bgjJtQX3b4FY yG86mamZ4tj/D23DI/SrhnofDHwBe9gy+x1KKLgSVwacV0OZR7zHZbz6uC8zKGUu6fi3 xtCVrS76ZeXw4j31bAEgjipBf4iOyu6npLqi0klkOWJADkUFc+l6q+l2HYXTu4iIc/Om GeUA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=roAszNAJKqGSuEnLaKwJ7DBIMHSpofvOj/58DbRClkw=; fh=R49q8kPD/ziIvsyYshS2LccDCxkbN9TSdzSyy3EleF0=; b=sUmhQRhcv8MpjlL1gdacyjMjJbmlY7eZXCi1zwOugkVw2agkj2f6GgXMsNF/49P3aN lLs5DuPvS0VQRmexGaZ+MGzhRA+SB1k2Hg8lkLD8uIVj6bv3NaF8x8LMcXefae96auKI QkblKvMDx1ObryfsXgCL6hvDmIfXxPaa+5r3P086ECs/+QIVPC0vNILS+KnspkydPw8V WRatp2S8TobFQyMGnTUymTzoOT1FJS7m7eWIAE/Ki/LetUX6+TdXnOtTfETdifKwkeOM Oc6FHKj+Rw7RhXByDx2I3BQtwWEiFm3ijejn9G66mwCEa58OmjCmEhWiOK+jGe5D06Wi d2Tw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=LsokCLPH; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from fry.vger.email (fry.vger.email. [2620:137:e000::3:8]) by mx.google.com with ESMTPS id y7-20020a17090322c700b001bbd83490efsi910362plg.138.2023.10.16.20.21.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 16 Oct 2023 20:21:43 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) client-ip=2620:137:e000::3:8; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=LsokCLPH; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by fry.vger.email (Postfix) with ESMTP id DE2AA80A0DEF; Mon, 16 Oct 2023 20:21:35 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at fry.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234097AbjJQDVP (ORCPT + 19 others); Mon, 16 Oct 2023 23:21:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57400 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232701AbjJQDVL (ORCPT ); Mon, 16 Oct 2023 23:21:11 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 588A49F; Mon, 16 Oct 2023 20:21:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1697512869; x=1729048869; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=laaAssHt8V3R6F9hNgPsAVESR40dtu6/j3R2Nhzmd3s=; b=LsokCLPHq56br8Inw/AMe2M2Iai6F4VcIPTn4HF4LmamTAIqQlMvgqGV s3P9uSv+CiPiLAdaWU2EQVlx99zd+ieiDrNuXPHmjLFvB2tVIsCQ+L2r1 C3WbEZBNNaS6mnZ/qnbo9d+DURinfYx0vb/sVlJoxLfP526nx0+gQFEX1 h7STgWUuJb7rYRXLTznyolpf95ZwqIq388mlWGtE9F3jCI/ojAucE1ekm 599lbKCiMiHrl6lLJ2Pif/EuiM45GF4lKlSWwFNFiTR2L+V9+K1gpkOl8 CcTpMf7aDpv7WCWPnCR/1Ip9TIPrSedH8OD2LIyBK4fkZ1c+wdRproCYp A==; X-IronPort-AV: E=McAfee;i="6600,9927,10865"; a="389560769" X-IronPort-AV: E=Sophos;i="6.03,231,1694761200"; d="scan'208";a="389560769" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Oct 2023 20:21:08 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10865"; a="826269892" X-IronPort-AV: E=Sophos;i="6.03,231,1694761200"; d="scan'208";a="826269892" Received: from sqa-gate.sh.intel.com (HELO spr-2s5.tsp.org) ([10.239.48.212]) by fmsmga004.fm.intel.com with ESMTP; 16 Oct 2023 20:21:05 -0700 From: Tina Zhang To: iommu@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: David Woodhouse , Lu Baolu , Joerg Roedel , Jason Gunthorpe , Kevin Tian , Tina Zhang Subject: [RFC PATCH 00/12] iommu/vt-d: Remove superfluous IOMMU IOTLB invalidations Date: Tue, 17 Oct 2023 11:20:32 +0800 Message-Id: <20231017032045.114868-1-tina.zhang@intel.com> X-Mailer: git-send-email 2.39.3 MIME-Version: 1.0 X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on fry.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (fry.vger.email [0.0.0.0]); Mon, 16 Oct 2023 20:21:35 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779971290950664516 X-GMAIL-MSGID: 1779971290950664516 This series based on "Share sva" patch-set[1], aims to remove superfluous IOMMU IOTLB invalidations in VT-d driver. In the current VT-d driver, IOMMU IOTLB invalidation commands and device-TLB commands are performed per device, which leads to superfluous IOTLB invalidations. For example, if there are four devices behind a IOMMU are attached to one sva domain (which could be a common case in virtualization scenarios where one virtual IOMMU working for all the virtual devices), four IOTLB invalidation commands and four device-IOTLB invalidation commands will be issued. However, only one IOTLB invalidation command and four device-IOTLB invalidation commands are necessary. Superfluous IOMMU IOTLB invalidations impact run-time performance. Although the goal could be built straight-forwardly in the current VT-d driver, some refactoring works are considered necessary before landing the one solving the problem of redundant IOTLB invalidations: 1) VT-d driver uses different structures to keep attached device info. For default domain, it uses struct dev_pasids_info and some related fields of struct dmar_domain. For sva domain, it uses struct intel_svm and struct intel_svm_dev. The reason of it is because previously the sva domain is not shared among devices behind different IOMMUs and therefore dmar_domain and its fields cannot be used globally to keep all attached device info. After the patch-set[1] gets acceptance, sva domain is global to the attached devices. Thus, retiring struct intel_svm/intel_svm_dev is the main refactoring work of this patch-set. 2) Most logic of intel_svm_set_dev_pasid() can be covered by intel_iommu_set_dev_pasid(). Refactoring both intel_svm_set_dev_pasid() and intel_iommu_set_dev_pasid() and let the former call the latter for set_dev_pasid operation to avoid duplicating code. 3) Last but not least, struct mmu_notifier is proposed to iommu_domain. This is a change to IOMMU core, which helps to centralize info required by sva to sva domain and therefore can help cleanup the code in IOMMU drivers. This patchset is on github per-iommu_IOTLB_invalidation branch[2]. [1]: https://lore.kernel.org/linux-iommu/20231017004802.109618-1-tina.zhang@intel.com/ [2]: https://github.com/TinaZhangZW/linux/tree/per-iommu_IOTLB_invalidation Tina Zhang (12): iommu/vt-d: Retire the treatment for revoking PASIDs with pending pgfaults iommu/vt-d: Remove initialization for dynamically heap-allocated rcu_head iommu/vt-d: Retire intel_svm_bind_mm() iommu/vt-d: Make dev_to_intel_iommu() helper global iommu/vt-d: Retire struct intel_svm_dev iommu: Add mmu_notifier to sva domain iommu/vt-d: Retire struct intel_svm iommu/vt-d: Use RCU for dev_pasids list updates in set/remove_dev_pasid() iommu/vt-d: Refactor intel_iommu_set_dev_pasid() iommu/vt-d: Refactor intel_iommu_remove_dev_pasid() iommu/vt-d: Use intel_iommu_set_dev_pasid() for sva domain iommu/vt-d: Remove superfluous IOMMU IOTLB invalidations drivers/iommu/intel/iommu.c | 91 +++++++++---- drivers/iommu/intel/iommu.h | 30 ++--- drivers/iommu/intel/svm.c | 261 ++++++++---------------------------- include/linux/iommu.h | 2 + 4 files changed, 128 insertions(+), 256 deletions(-)