From patchwork Wed Jan 11 20:24:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Liang, Kan" X-Patchwork-Id: 42182 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp3523671wrt; Wed, 11 Jan 2023 12:32:36 -0800 (PST) X-Google-Smtp-Source: AMrXdXvvTEhzHTaOieNLR/Q4iBRI8V7+eaUaowyRCLSkZzq2WjX1UUgN6XICEnVPi8TztoFf/DID X-Received: by 2002:a62:ee19:0:b0:583:312a:b344 with SMTP id e25-20020a62ee19000000b00583312ab344mr20432253pfi.31.1673469155951; Wed, 11 Jan 2023 12:32:35 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1673469155; cv=none; d=google.com; s=arc-20160816; b=vihAVHE7gO+1YL5TFeCoDms432y7qo6N0TwMeKR5NlQtVe6djxuYcVrlcuY+LCrcwW QOjDgOEgrrhIQzN7WMYeJv5XC/aUdiMxr4FH0zYb7d0y2h6xA0bWV70egK8AtZ/zG08m JvSCxK5qlsFqA7S+pqvB8Z6i3BTzORVAoewet11KG6NfdGouqGCYccPDWE4/bnmd3uRs PUPJQ0S8uCS3f+2zZkuWA14BRWUHumL0BldBCmXL1mXR1lmWqtCbmz86yq32javiTaVD YzsCsiuZd+Ct80r0dcT3eYyg/vdfDio+VXtV19tpQRug7tR/BLzdJlCjWue9Kj8TVD4e Xh7w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=w7SeQwRWkZz1WXM5zI6a32lN4gHwM70Bkv7RXnTHKo8=; b=sDrGB5wEtgEDX8BY0YDdwve5NZ9clQqvHjzq51h+X0+09zBXEHEAN+Ze4JO7V0pJ95 EtizCyP4hrNRSB6eFk/h4t13elNiMHxC+SqF0h+q6Bg7IMshWCoQ16a9ryWjUy+zqUHH 7/DvxcLYXQYlt+tpSvGdRQstmFw11u4mjvleXbbiNaHhd+WVdEZeAVWIF/ray255YUqb fIXDKEpvEMn9mizp1DDmDr1eTza9l5kjFcvdJPo32jbnqncd2bbH0KjOtru5PHjsmPZw Kxs3KVZFL/UhTRmGE3XhW7GJmDPq1/CPS2T9npl1OEI64Rh2ifv/3fNmu6FRW0wrkEQi qZAQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=hr3N5O6n; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id a9-20020a056a000c8900b00572dc2ed1f7si16375637pfv.191.2023.01.11.12.32.23; Wed, 11 Jan 2023 12:32:35 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=hr3N5O6n; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236142AbjAKUZj (ORCPT + 99 others); Wed, 11 Jan 2023 15:25:39 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44438 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234950AbjAKUZb (ORCPT ); Wed, 11 Jan 2023 15:25:31 -0500 Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 167A01260C for ; Wed, 11 Jan 2023 12:25:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1673468730; x=1705004730; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=rjUMC3WP9nxqI0yDEY/Vy+fMFkhGKuCo4eL3dfoCdmk=; b=hr3N5O6nyAcIIE/SQ2Oki2NWfs3B81md7bOdSM3JzmX496mGHMFJQQOq mFKEjdwvVs8rdfDdQc2sgaVmfcuTaje5X2Dm9QzBPaOpzZgvP3YZx6bRy SUxwIzXQ1Pe35MB7Pbu6RGQ5NO41WFAAWewmHeYWkQbqxYrNp6+8R8h+G hVrdyeUUlYsIlceybmbaXKoWIc3vD1nY12cG0S39eipDkhnd9f+WxWsJ+ itRdrEE+zU4dKhiys7C+X6CZqh4/V2M4EEohYCkr/jShtUudZJllgnbQE NxJjgQyYp78kI1MbqKvcUsldIlpLd+mMSIJ5Xhu6iNoQfLBTRk5zH9EVc A==; X-IronPort-AV: E=McAfee;i="6500,9779,10586"; a="350754976" X-IronPort-AV: E=Sophos;i="5.96,317,1665471600"; d="scan'208";a="350754976" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Jan 2023 12:25:28 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10586"; a="781518062" X-IronPort-AV: E=Sophos;i="5.96,317,1665471600"; d="scan'208";a="781518062" Received: from kanliang-dev.jf.intel.com ([10.165.154.102]) by orsmga004.jf.intel.com with ESMTP; 11 Jan 2023 12:25:28 -0800 From: kan.liang@linux.intel.com To: joro@8bytes.org, will@kernel.org, baolu.lu@linux.intel.com, dwmw2@infradead.org, robin.murphy@arm.com, robert.moore@intel.com, rafael.j.wysocki@intel.com, lenb@kernel.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org Cc: Kan Liang Subject: [PATCH 1/7] iommu/vt-d: Support size of the register set in DRHD Date: Wed, 11 Jan 2023 12:24:58 -0800 Message-Id: <20230111202504.378258-2-kan.liang@linux.intel.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20230111202504.378258-1-kan.liang@linux.intel.com> References: <20230111202504.378258-1-kan.liang@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1754759594006291501?= X-GMAIL-MSGID: =?utf-8?q?1754759594006291501?= From: Kan Liang A new field, which indicates the size of the remapping hardware register set for this remapping unit, is introduced in the DMA-remapping hardware unit definition (DRHD) structure with the VT-d Spec 4.0. With the information, SW doesn't need to 'guess' the size of the register set anymore. Update the struct acpi_dmar_hardware_unit to reflect the field. Store the size of the register set in struct dmar_drhd_unit for each dmar device. The 'size' information is ResvZ for the old BIOS and platforms. Fall back to the old guessing method. There is nothing changed. Signed-off-by: Kan Liang --- drivers/iommu/intel/dmar.c | 25 ++++++++++++++++++------- include/acpi/actbl1.h | 2 +- include/linux/dmar.h | 1 + 3 files changed, 20 insertions(+), 8 deletions(-) diff --git a/drivers/iommu/intel/dmar.c b/drivers/iommu/intel/dmar.c index b00a0ceb2d13..6a411d964474 100644 --- a/drivers/iommu/intel/dmar.c +++ b/drivers/iommu/intel/dmar.c @@ -399,6 +399,13 @@ dmar_find_dmaru(struct acpi_dmar_hardware_unit *drhd) return NULL; } +/* The size of the register set is 2 ^ N 4 KB pages. */ +static unsigned long +dmar_get_drhd_reg_size(u8 npages) +{ + return 1UL << (npages + 12); +} + /* * dmar_parse_one_drhd - parses exactly one DMA remapping hardware definition * structure which uniquely represent one DMA remapping hardware unit @@ -427,6 +434,7 @@ static int dmar_parse_one_drhd(struct acpi_dmar_header *header, void *arg) memcpy(dmaru->hdr, header, header->length); dmaru->reg_base_addr = drhd->address; dmaru->segment = drhd->segment; + dmaru->reg_size = dmar_get_drhd_reg_size(drhd->size); dmaru->include_all = drhd->flags & 0x1; /* BIT0: INCLUDE_ALL */ dmaru->devices = dmar_alloc_dev_scope((void *)(drhd + 1), ((void *)drhd) + drhd->header.length, @@ -880,6 +888,7 @@ dmar_validate_one_drhd(struct acpi_dmar_header *entry, void *arg) struct acpi_dmar_hardware_unit *drhd; void __iomem *addr; u64 cap, ecap; + unsigned long size; drhd = (void *)entry; if (!drhd->address) { @@ -887,10 +896,11 @@ dmar_validate_one_drhd(struct acpi_dmar_header *entry, void *arg) return -EINVAL; } + size = dmar_get_drhd_reg_size(drhd->size); if (arg) - addr = ioremap(drhd->address, VTD_PAGE_SIZE); + addr = ioremap(drhd->address, size); else - addr = early_ioremap(drhd->address, VTD_PAGE_SIZE); + addr = early_ioremap(drhd->address, size); if (!addr) { pr_warn("Can't validate DRHD address: %llx\n", drhd->address); return -EINVAL; @@ -902,7 +912,7 @@ dmar_validate_one_drhd(struct acpi_dmar_header *entry, void *arg) if (arg) iounmap(addr); else - early_iounmap(addr, VTD_PAGE_SIZE); + early_iounmap(addr, size); if (cap == (uint64_t)-1 && ecap == (uint64_t)-1) { warn_invalid_dmar(drhd->address, " returns all ones"); @@ -956,17 +966,18 @@ static void unmap_iommu(struct intel_iommu *iommu) /** * map_iommu: map the iommu's registers * @iommu: the iommu to map - * @phys_addr: the physical address of the base resgister + * @drhd: DMA remapping hardware definition structure * * Memory map the iommu's registers. Start w/ a single page, and * possibly expand if that turns out to be insufficent. */ -static int map_iommu(struct intel_iommu *iommu, u64 phys_addr) +static int map_iommu(struct intel_iommu *iommu, struct dmar_drhd_unit *drhd) { + u64 phys_addr = drhd->reg_base_addr; int map_size, err=0; iommu->reg_phys = phys_addr; - iommu->reg_size = VTD_PAGE_SIZE; + iommu->reg_size = drhd->reg_size; if (!request_mem_region(iommu->reg_phys, iommu->reg_size, iommu->name)) { pr_err("Can't reserve memory\n"); @@ -1050,7 +1061,7 @@ static int alloc_iommu(struct dmar_drhd_unit *drhd) } sprintf(iommu->name, "dmar%d", iommu->seq_id); - err = map_iommu(iommu, drhd->reg_base_addr); + err = map_iommu(iommu, drhd); if (err) { pr_err("Failed to map %s\n", iommu->name); goto error_free_seq_id; diff --git a/include/acpi/actbl1.h b/include/acpi/actbl1.h index 4175dce3967c..bdded0ac46eb 100644 --- a/include/acpi/actbl1.h +++ b/include/acpi/actbl1.h @@ -802,7 +802,7 @@ struct acpi_dmar_pci_path { struct acpi_dmar_hardware_unit { struct acpi_dmar_header header; u8 flags; - u8 reserved; + u8 size; /* Size of the register set */ u16 segment; u64 address; /* Register Base Address */ }; diff --git a/include/linux/dmar.h b/include/linux/dmar.h index d81a51978d01..51d498f0a02b 100644 --- a/include/linux/dmar.h +++ b/include/linux/dmar.h @@ -46,6 +46,7 @@ struct dmar_drhd_unit { u8 include_all:1; u8 gfx_dedicated:1; /* graphic dedicated */ struct intel_iommu *iommu; + unsigned long reg_size; /* size of register set */ }; struct dmar_pci_path { From patchwork Wed Jan 11 20:24:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Liang, Kan" X-Patchwork-Id: 42184 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp3523789wrt; Wed, 11 Jan 2023 12:32:52 -0800 (PST) X-Google-Smtp-Source: AMrXdXvK6RcBTqibGYjw5Z0Lh3fsVRtt5aAqQjUi8GAAwxJgFAlSCKrywVaayA2YQ0mq2ckWfnm7 X-Received: by 2002:a17:902:a981:b0:187:403c:7a3b with SMTP id bh1-20020a170902a98100b00187403c7a3bmr66444595plb.69.1673469171688; Wed, 11 Jan 2023 12:32:51 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1673469171; cv=none; d=google.com; s=arc-20160816; b=MBgaZivXFe5Bu0Q0Tucn0M5w0TUGMbZuYRZVP3xEnKkiMXw9HnrDgKiwgozCuLDOAz uQBSVwBySI0zorWdsGyADk/G3iXEc13WviNq7Oz5PUBptuRSYJpfMn7vlvXYHmwoN/a/ VAATfHe5A4XuZiR5MVsO4g8YK90s6MUYwDY5lPJZSBicjrTLp6bMDkerOXpgzjkftDSn 73Mjtt2TxvbdQsJYqNqmJ+U4XZWujC01YvDnt/yAWFGv6S9xBV8YKw5dVMOHQ4Ok2IXN 4xG82FUYKGeOa5GYKCXPHA72TcYxBf4/r3IrYpVXTmLU0ilLSBuqInDz2run+54dVkLo PeLw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=+wtneJfIaImxCNp0cmyHZiMNdX9JHpDbGvHiyRPn+90=; b=Sb5ZYd4gLHnnbqRce//rj0lhsk24kF2dJjMTCwFCwNfZ9WmKDhWMMEGvjpTlNcqOjY UInwuJ3uXTXaoOwWpCgT4Gb1MVr+v1sNh5AqaWfXNV+MPZiqMR00PkWMYDB9Ag+/sxeR VMendOeBh+oy00q2/5YX4MiPxhj9Zw9ydYs25315QRbGyKuCuARElpCF97ra5hYIoN/s 7ZFBPCAxlZTMSdRpLSoswGrWvmRcL0J6VsrNz1OBv9CcCbFhn7khpP8d7yJP/gcwarVX a97nP5Px/uMvDTnM6bLzeu8uo8DKK1qiUoIr8PPRyMrJPqZwP3HuWKsVgW7wKYWzHn03 vnwQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=advp4z0m; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id i64-20020a638743000000b004b5b1c44f6csi2801757pge.183.2023.01.11.12.32.39; Wed, 11 Jan 2023 12:32:51 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=advp4z0m; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229833AbjAKUZu (ORCPT + 99 others); Wed, 11 Jan 2023 15:25:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44452 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235089AbjAKUZc (ORCPT ); Wed, 11 Jan 2023 15:25:32 -0500 Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1355212AEB for ; Wed, 11 Jan 2023 12:25:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1673468731; x=1705004731; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Xl1jBBH60sMm4WbuZU3wql+1yc2hka4yYYnntjPLw3I=; b=advp4z0mbaPi4jE4zhy853cPJvc3i3EQb1a0twkgVWHA5JUM6SDF1GF3 BLXgPvUgm23JWaE8Hz3Td2MAfZPodacXFqUtfiuVUOxB0lwerARuzG8tP Vw8dsb9EDXY+FfORPk39S26mKHCGl8U1WXH6ttivSrseBoNGyZQv2iTty yK9AN4odb+VTIRIPORzedWZw3325zolV71Psp44ZXm6pMKr7P8SrFGijJ FqX2wTBLGbR5u3Fcf3C9PstNhNzLRoiOsK9mbUzVLbeL+75/yTPNuvGc7 5rT6M56TgbGDXTgHE4j/yKgcdmSANRX3P32R4TBbUUlR02y/uHMjusaWQ Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10586"; a="350754983" X-IronPort-AV: E=Sophos;i="5.96,317,1665471600"; d="scan'208";a="350754983" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Jan 2023 12:25:28 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10586"; a="781518065" X-IronPort-AV: E=Sophos;i="5.96,317,1665471600"; d="scan'208";a="781518065" Received: from kanliang-dev.jf.intel.com ([10.165.154.102]) by orsmga004.jf.intel.com with ESMTP; 11 Jan 2023 12:25:28 -0800 From: kan.liang@linux.intel.com To: joro@8bytes.org, will@kernel.org, baolu.lu@linux.intel.com, dwmw2@infradead.org, robin.murphy@arm.com, robert.moore@intel.com, rafael.j.wysocki@intel.com, lenb@kernel.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org Cc: Kan Liang Subject: [PATCH 2/7] iommu/vt-d: Retrieve IOMMU perfmon capability information Date: Wed, 11 Jan 2023 12:24:59 -0800 Message-Id: <20230111202504.378258-3-kan.liang@linux.intel.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20230111202504.378258-1-kan.liang@linux.intel.com> References: <20230111202504.378258-1-kan.liang@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1754759609988601076?= X-GMAIL-MSGID: =?utf-8?q?1754759609988601076?= From: Kan Liang The performance monitoring infrastructure, perfmon, is to support collection of information about key events occurring during operation of the remapping hardware, to aid performance tuning and debug. Each remapping hardware unit has capability registers that indicate support for performance monitoring features and enumerate the capabilities. Add alloc_iommu_pmu() to retrieve IOMMU perfmon capability information for each iommu unit. The information is stored in the iommu->pmu data structure. Capability registers are read-only, so it's safe to prefetch and store them in the pmu structure. This could avoid unnecessary VMEXIT when this code is running in the virtualization environment. Add free_iommu_pmu() to free the saved capability information when freeing the iommu unit. Add a kernel config option for the IOMMU perfmon feature. Unless a user explicitly uses the perf tool to monitor the IOMMU perfmon event, there isn't any impact for the existing IOMMU. Enable it by default. Signed-off-by: Kan Liang --- drivers/iommu/intel/Kconfig | 9 ++ drivers/iommu/intel/Makefile | 1 + drivers/iommu/intel/dmar.c | 7 ++ drivers/iommu/intel/iommu.h | 41 +++++++++ drivers/iommu/intel/perfmon.c | 159 ++++++++++++++++++++++++++++++++++ drivers/iommu/intel/perfmon.h | 41 +++++++++ 6 files changed, 258 insertions(+) create mode 100644 drivers/iommu/intel/perfmon.c create mode 100644 drivers/iommu/intel/perfmon.h diff --git a/drivers/iommu/intel/Kconfig b/drivers/iommu/intel/Kconfig index b7dff5092fd2..1a4aebddc9a6 100644 --- a/drivers/iommu/intel/Kconfig +++ b/drivers/iommu/intel/Kconfig @@ -96,4 +96,13 @@ config INTEL_IOMMU_SCALABLE_MODE_DEFAULT_ON passing intel_iommu=sm_on to the kernel. If not sure, please use the default value. +config INTEL_IOMMU_PERF_EVENTS + def_bool y + bool "Intel IOMMU performance events" + depends on INTEL_IOMMU && PERF_EVENTS + help + Include support for Intel IOMMU performance events. These are + available on modern processors which support Intel VT-d 4.0 and + later. + endif # INTEL_IOMMU diff --git a/drivers/iommu/intel/Makefile b/drivers/iommu/intel/Makefile index fa0dae16441c..7af3b8a4f2a0 100644 --- a/drivers/iommu/intel/Makefile +++ b/drivers/iommu/intel/Makefile @@ -6,3 +6,4 @@ obj-$(CONFIG_DMAR_PERF) += perf.o obj-$(CONFIG_INTEL_IOMMU_DEBUGFS) += debugfs.o obj-$(CONFIG_INTEL_IOMMU_SVM) += svm.o obj-$(CONFIG_IRQ_REMAP) += irq_remapping.o +obj-$(CONFIG_INTEL_IOMMU_PERF_EVENTS) += perfmon.o diff --git a/drivers/iommu/intel/dmar.c b/drivers/iommu/intel/dmar.c index 6a411d964474..91bb48267df2 100644 --- a/drivers/iommu/intel/dmar.c +++ b/drivers/iommu/intel/dmar.c @@ -34,6 +34,7 @@ #include "../irq_remapping.h" #include "perf.h" #include "trace.h" +#include "perfmon.h" typedef int (*dmar_res_handler_t)(struct acpi_dmar_header *, void *); struct dmar_res_callback { @@ -1114,6 +1115,9 @@ static int alloc_iommu(struct dmar_drhd_unit *drhd) if (sts & DMA_GSTS_QIES) iommu->gcmd |= DMA_GCMD_QIE; + if (alloc_iommu_pmu(iommu)) + pr_debug("Cannot alloc PMU for iommu (seq_id = %d)\n", iommu->seq_id); + raw_spin_lock_init(&iommu->register_lock); /* @@ -1148,6 +1152,7 @@ static int alloc_iommu(struct dmar_drhd_unit *drhd) err_sysfs: iommu_device_sysfs_remove(&iommu->iommu); err_unmap: + free_iommu_pmu(iommu); unmap_iommu(iommu); error_free_seq_id: ida_free(&dmar_seq_ids, iommu->seq_id); @@ -1163,6 +1168,8 @@ static void free_iommu(struct intel_iommu *iommu) iommu_device_sysfs_remove(&iommu->iommu); } + free_iommu_pmu(iommu); + if (iommu->irq) { if (iommu->pr_irq) { free_irq(iommu->pr_irq, iommu); diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h index 06e61e474856..5bcefbea55c9 100644 --- a/drivers/iommu/intel/iommu.h +++ b/drivers/iommu/intel/iommu.h @@ -125,6 +125,11 @@ #define DMAR_MTRR_PHYSMASK8_REG 0x208 #define DMAR_MTRR_PHYSBASE9_REG 0x210 #define DMAR_MTRR_PHYSMASK9_REG 0x218 +#define DMAR_PERFCAP_REG 0x300 +#define DMAR_PERFCFGOFF_REG 0x310 +#define DMAR_PERFOVFOFF_REG 0x318 +#define DMAR_PERFCNTROFF_REG 0x31c +#define DMAR_PERFEVNTCAP_REG 0x380 #define DMAR_VCCAP_REG 0xe30 /* Virtual command capability register */ #define DMAR_VCMD_REG 0xe00 /* Virtual command register */ #define DMAR_VCRSP_REG 0xe10 /* Virtual command response register */ @@ -148,6 +153,7 @@ */ #define cap_esrtps(c) (((c) >> 63) & 1) #define cap_esirtps(c) (((c) >> 62) & 1) +#define cap_ecmds(c) (((c) >> 61) & 1) #define cap_fl5lp_support(c) (((c) >> 60) & 1) #define cap_pi_support(c) (((c) >> 59) & 1) #define cap_fl1gp_support(c) (((c) >> 56) & 1) @@ -179,6 +185,7 @@ * Extended Capability Register */ +#define ecap_pms(e) (((e) >> 51) & 0x1) #define ecap_rps(e) (((e) >> 49) & 0x1) #define ecap_smpwc(e) (((e) >> 48) & 0x1) #define ecap_flts(e) (((e) >> 47) & 0x1) @@ -210,6 +217,22 @@ #define ecap_max_handle_mask(e) (((e) >> 20) & 0xf) #define ecap_sc_support(e) (((e) >> 7) & 0x1) /* Snooping Control */ +/* + * Decoding Perf Capability Register + */ +#define pcap_num_cntr(p) ((p) & 0xffff) +#define pcap_cntr_width(p) (((p) >> 16) & 0x7f) +#define pcap_num_event_group(p) (((p) >> 24) & 0x1f) +#define pcap_filters_mask(p) (((p) >> 32) & 0x1f) +#define pcap_interrupt(p) (((p) >> 50) & 0x1) +/* The counter stride is calculated as 2 ^ (x+10) bytes */ +#define pcap_cntr_stride(p) (1ULL << ((((p) >> 52) & 0x7) + 10)) + +/* + * Decoding Perf Event Capability Register + */ +#define pecap_es(p) ((p) & 0xfffffff) + /* Virtual command interface capability */ #define vccap_pasid(v) (((v) & DMA_VCS_PAS)) /* PASID allocation */ @@ -554,6 +577,22 @@ struct dmar_domain { iommu core */ }; +struct iommu_pmu { + struct intel_iommu *iommu; + u32 num_cntr; /* Number of counters */ + u32 num_eg; /* Number of event group */ + u32 cntr_width; /* Counter width */ + u32 cntr_stride; /* Counter Stride */ + u32 filter; /* Bitmask of filter support */ + void __iomem *base; /* the PerfMon base address */ + void __iomem *cfg_reg; /* counter configuration base address */ + void __iomem *cntr_reg; /* counter 0 address*/ + void __iomem *overflow; /* overflow status register */ + + u64 *evcap; /* Indicates all supported events */ + u32 **cntr_evcap; /* Supported events of each counter. */ +}; + struct intel_iommu { void __iomem *reg; /* Pointer to hardware regs, virtual addr */ u64 reg_phys; /* physical address of hw register set */ @@ -600,6 +639,8 @@ struct intel_iommu { struct dmar_drhd_unit *drhd; void *perf_statistic; + + struct iommu_pmu *pmu; }; /* PCI domain-device relationship */ diff --git a/drivers/iommu/intel/perfmon.c b/drivers/iommu/intel/perfmon.c new file mode 100644 index 000000000000..bc090f329c32 --- /dev/null +++ b/drivers/iommu/intel/perfmon.c @@ -0,0 +1,159 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Support Intel IOMMU PerfMon + * Copyright(c) 2022 Intel Corporation. + */ + +#include +#include "iommu.h" +#include "perfmon.h" + +static inline void __iomem * +get_perf_reg_address(struct intel_iommu *iommu, u32 offset) +{ + u32 off = dmar_readl(iommu->reg + offset); + + return iommu->reg + off; +} + +int alloc_iommu_pmu(struct intel_iommu *iommu) +{ + struct iommu_pmu *iommu_pmu; + int i, j, ret; + u64 perfcap; + u32 cap; + + /* The IOMMU PMU requires the ECMD support as well */ + if (!ecap_pms(iommu->ecap) || !cap_ecmds(iommu->cap)) + return -ENODEV; + + perfcap = dmar_readq(iommu->reg + DMAR_PERFCAP_REG); + /* The performance monitoring is not supported. */ + if (!perfcap) + return -ENODEV; + + /* Sanity check for the number of the counters and event groups */ + if (!pcap_num_cntr(perfcap) || !pcap_num_event_group(perfcap)) + return -ENODEV; + + /* The interrupt on overflow is required */ + if (!pcap_interrupt(perfcap)) + return -ENODEV; + + iommu_pmu = kzalloc(sizeof(*iommu_pmu), GFP_KERNEL); + if (!iommu_pmu) + return -ENOMEM; + + iommu_pmu->num_cntr = pcap_num_cntr(perfcap); + iommu_pmu->cntr_width = pcap_cntr_width(perfcap); + iommu_pmu->filter = pcap_filters_mask(perfcap); + iommu_pmu->cntr_stride = pcap_cntr_stride(perfcap); + iommu_pmu->num_eg = pcap_num_event_group(perfcap); + + iommu_pmu->evcap = kcalloc(iommu_pmu->num_eg, sizeof(u64), GFP_KERNEL); + if (!iommu_pmu->evcap) { + ret = -ENOMEM; + goto free_pmu; + } + + /* Parse event group capabilities */ + for (i = 0; i < iommu_pmu->num_eg; i++) { + u64 pcap; + + pcap = dmar_readq(iommu->reg + DMAR_PERFEVNTCAP_REG + + i * IOMMU_PMU_CAP_REGS_STEP); + iommu_pmu->evcap[i] = pecap_es(pcap); + } + + iommu_pmu->cntr_evcap = kcalloc(iommu_pmu->num_cntr, sizeof(u32 *), GFP_KERNEL); + if (!iommu_pmu->cntr_evcap) { + ret = -ENOMEM; + goto free_pmu_evcap; + } + for (i = 0; i < iommu_pmu->num_cntr; i++) { + iommu_pmu->cntr_evcap[i] = kcalloc(iommu_pmu->num_eg, sizeof(u32), GFP_KERNEL); + if (!iommu_pmu->cntr_evcap[i]) { + ret = -ENOMEM; + iommu_pmu->num_cntr = i; + goto free_pmu_cntr_evcap; + } + /* + * Set to the global capabilities, will adjust according + * to per-counter capabilities later. + */ + for (j = 0; j < iommu_pmu->num_eg; j++) + iommu_pmu->cntr_evcap[i][j] = (u32)iommu_pmu->evcap[j]; + } + + iommu_pmu->cfg_reg = get_perf_reg_address(iommu, DMAR_PERFCFGOFF_REG); + iommu_pmu->cntr_reg = get_perf_reg_address(iommu, DMAR_PERFCNTROFF_REG); + iommu_pmu->overflow = get_perf_reg_address(iommu, DMAR_PERFOVFOFF_REG); + + /* + * Check per-counter capabilities + * All counters should have the same capabilities on + * Interrupt on Overflow Support and Counter Width + */ + for (i = 0; i < iommu_pmu->num_cntr; i++) { + cap = dmar_readl(iommu_pmu->cfg_reg + + i * IOMMU_PMU_CFG_OFFSET + + IOMMU_PMU_CFG_CNTRCAP_OFFSET); + if (!iommu_cntrcap_pcc(cap)) + continue; + if ((iommu_cntrcap_cw(cap) != iommu_pmu->cntr_width) || + !iommu_cntrcap_ios(cap)) + iommu_pmu->num_cntr = i; + + /* Clear the pre-defined events group */ + for (j = 0; j < iommu_pmu->num_eg; j++) + iommu_pmu->cntr_evcap[i][j] = 0; + + /* Override with per-counter event capabilities */ + for (j = 0; j < iommu_cntrcap_egcnt(cap); j++) { + cap = dmar_readl(iommu_pmu->cfg_reg + i * IOMMU_PMU_CFG_OFFSET + + IOMMU_PMU_CFG_CNTREVCAP_OFFSET + + (j * IOMMU_PMU_OFF_REGS_STEP)); + iommu_pmu->cntr_evcap[i][iommu_event_group(cap)] = iommu_event_select(cap); + /* + * Some events may only be supported by a specific counter. + * Track them in the evcap as well. + */ + iommu_pmu->evcap[iommu_event_group(cap)] |= iommu_event_select(cap); + } + } + + iommu_pmu->iommu = iommu; + iommu->pmu = iommu_pmu; + + return 0; + +free_pmu_cntr_evcap: + for (i = 0; i < iommu_pmu->num_cntr; i++) + kfree(iommu_pmu->cntr_evcap[i]); + kfree(iommu_pmu->cntr_evcap); +free_pmu_evcap: + kfree(iommu_pmu->evcap); +free_pmu: + kfree(iommu_pmu); + + return ret; +} + +void free_iommu_pmu(struct intel_iommu *iommu) +{ + struct iommu_pmu *iommu_pmu = iommu->pmu; + + if (!iommu_pmu) + return; + + if (iommu_pmu->evcap) { + int i; + + for (i = 0; i < iommu_pmu->num_cntr; i++) + kfree(iommu_pmu->cntr_evcap[i]); + kfree(iommu_pmu->cntr_evcap); + } + kfree(iommu_pmu->evcap); + kfree(iommu_pmu); + iommu->pmu = NULL; +} diff --git a/drivers/iommu/intel/perfmon.h b/drivers/iommu/intel/perfmon.h new file mode 100644 index 000000000000..8587c80501cd --- /dev/null +++ b/drivers/iommu/intel/perfmon.h @@ -0,0 +1,41 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +/* + * PERFCFGOFF_REG, PERFFRZOFF_REG + * PERFOVFOFF_REG, PERFCNTROFF_REG + */ +#define IOMMU_PMU_NUM_OFF_REGS 4 +#define IOMMU_PMU_OFF_REGS_STEP 4 + +#define IOMMU_PMU_CFG_OFFSET 0x100 +#define IOMMU_PMU_CFG_CNTRCAP_OFFSET 0x80 +#define IOMMU_PMU_CFG_CNTREVCAP_OFFSET 0x84 +#define IOMMU_PMU_CFG_SIZE 0x8 +#define IOMMU_PMU_CFG_FILTERS_OFFSET 0x4 + + +#define IOMMU_PMU_CAP_REGS_STEP 8 + +#define iommu_cntrcap_pcc(p) ((p) & 0x1) +#define iommu_cntrcap_cw(p) ((p >> 8) & 0xff) +#define iommu_cntrcap_ios(p) ((p >> 16) & 0x1) +#define iommu_cntrcap_egcnt(p) ((p >> 28) & 0xf) + +#define iommu_event_select(p) ((p) & 0xfffffff) +#define iommu_event_group(p) ((p >> 28) & 0xf) + +#ifdef CONFIG_INTEL_IOMMU_PERF_EVENTS +int alloc_iommu_pmu(struct intel_iommu *iommu); +void free_iommu_pmu(struct intel_iommu *iommu); +#else +static inline int +alloc_iommu_pmu(struct intel_iommu *iommu) +{ + return 0; +} + +static inline void +free_iommu_pmu(struct intel_iommu *iommu) +{ +} +#endif /* CONFIG_INTEL_IOMMU_PERF_EVENTS */ From patchwork Wed Jan 11 20:25:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Liang, Kan" X-Patchwork-Id: 42183 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp3523793wrt; Wed, 11 Jan 2023 12:32:52 -0800 (PST) X-Google-Smtp-Source: AMrXdXtwGhpeU4tXfoTCmeLufqbJ325fCq4tkxv6hDQKLf2lgMbi8fFRSVdSNCj2ZlnGymiLTqDd X-Received: by 2002:a17:903:2447:b0:191:1a7c:ef9f with SMTP id l7-20020a170903244700b001911a7cef9fmr105679687pls.1.1673469172000; Wed, 11 Jan 2023 12:32:52 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1673469171; cv=none; d=google.com; s=arc-20160816; b=LFpFN2HhVoV5xVcl5mVoFYpLXyHH6vHHdoLPiaNRo/3bVS9sCXsPqQTlx1tQTSOxCE gTCnEydzsta5JelwDbFxgSftB88jascKqgoB1PkFNRSqPZeZOeEJzLbNsSkGPvZ5gf7o RCHsXldy6ecX60S5Cp9fLz+G7ncX7BWk1WgvgjYiQ1zs0eFvqbvdtU9WBFNR8We5akbK YYAPRg5jnhQLMihfv7Xa3/djrFq1cCuCJnUy86yrVQ/BM1W7mBX8s6aQJdoBa9uMtpsh EyFkNGqqUgWqfOj/JPZ8MA4LM5EKtlIctFtOiKHjlepFtNMvFMZImpqU/3GAUitLxNW8 zf0w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=1BoKLz+rKPtwB1JFtWISiRudE/LZinfxRW/8f+zW8pc=; b=TFVqwl07jPNemRh2sMiN9zdIyAJL7e3t2cQDUqYB7B8hbRzOQ23RZrtrNJF0BGXcjc iG+hzANClxmNWuSZ/uO8RxO8xAlsbJWsArnbnfw/iNjhV/GuaZ5ZLI/wal3emT8iA5Fk Ci7WwFkMaHjSIDEjp+0xgJeRRbOcUy8Rg2cnCpSBtt8fe9pr6lYQJ2c8i1WMGrTtkKLV ZoDoNrqYfQZn2gKEo+cK+Eeetw7D92DDouSBhmHjLFRS/2COoeH2dD/bUBPp+l2ondSX gofH3cKBO8fRay1yJGSmyfKqnkL82vLogLBmCitzGxbbj1yjl9B9W2D6Qxh+2EW4z5gT m6MA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Qk2dd4Kx; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id m11-20020a170902768b00b0017f908814c6si4445847pll.532.2023.01.11.12.32.39; Wed, 11 Jan 2023 12:32:51 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Qk2dd4Kx; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236500AbjAKUZo (ORCPT + 99 others); Wed, 11 Jan 2023 15:25:44 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44454 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235325AbjAKUZc (ORCPT ); Wed, 11 Jan 2023 15:25:32 -0500 Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 16DA713DE7 for ; Wed, 11 Jan 2023 12:25:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1673468731; x=1705004731; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=oYG9pyvEnoir5ijKaJji/tARGhQScPMXvWnM9ZJ6oYU=; b=Qk2dd4KxtLtwVIFKgYpogQB+2uUy7qff8qkZw5oFSlWNQjLPkoWxuCID KWTb4AdDQvszT12qC7vwuOljHWoTngvc5ux7bktqfUTSdLJlyLb7YSjhJ xoV7Ad+7ijC5MsJTTBTJuaKcDej2XLeGEC8tmD+9ZRzw7V4DB7P/Kvt7C SLJwM6lMW1P11kW7YyAjNAxFfZG3jCTA4Kv6feAcR1iMBH/a8j4QFBdlG 5yBgcG5fWtfP4SLyyjGeva4l9B3s7cV2eItt5rcQ6Iu1doNclaHgeoZ8d pB0SDzjmmvOayMC1tvd1PeI+a6H+JRXFF52Z0a6EK3aCSQETPdn9fgOVg g==; X-IronPort-AV: E=McAfee;i="6500,9779,10586"; a="350754990" X-IronPort-AV: E=Sophos;i="5.96,317,1665471600"; d="scan'208";a="350754990" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Jan 2023 12:25:29 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10586"; a="781518068" X-IronPort-AV: E=Sophos;i="5.96,317,1665471600"; d="scan'208";a="781518068" Received: from kanliang-dev.jf.intel.com ([10.165.154.102]) by orsmga004.jf.intel.com with ESMTP; 11 Jan 2023 12:25:28 -0800 From: kan.liang@linux.intel.com To: joro@8bytes.org, will@kernel.org, baolu.lu@linux.intel.com, dwmw2@infradead.org, robin.murphy@arm.com, robert.moore@intel.com, rafael.j.wysocki@intel.com, lenb@kernel.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org Cc: Kan Liang Subject: [PATCH 3/7] iommu/vt-d: Support Enhanced Command Interface Date: Wed, 11 Jan 2023 12:25:00 -0800 Message-Id: <20230111202504.378258-4-kan.liang@linux.intel.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20230111202504.378258-1-kan.liang@linux.intel.com> References: <20230111202504.378258-1-kan.liang@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1754759610574117177?= X-GMAIL-MSGID: =?utf-8?q?1754759610574117177?= From: Kan Liang The Enhanced Command Register is to submit command and operand of enhanced commands to DMA Remapping hardware. It can supports upto 256 enhanced commands. There is a HW register to indicate the availability of all 256 enhanced commands. Each bit stands for each command. But there isn't an existing interface to read/write all 256 bits. Introduce the u64 ecmdcap[4] to store the existence of each enhanced command. Read 4 times to get all of them in map_iommu(). Add a helper to facilitate an enhanced command launch. Make sure hardware complete the command. Add a helper to facilitate the check of PMU essentials. The helpers will be used later. Signed-off-by: Kan Liang --- drivers/iommu/intel/dmar.c | 63 +++++++++++++++++++++++++++++++++++++ drivers/iommu/intel/iommu.h | 46 +++++++++++++++++++++++++++ 2 files changed, 109 insertions(+) diff --git a/drivers/iommu/intel/dmar.c b/drivers/iommu/intel/dmar.c index 91bb48267df2..dcafa32875c1 100644 --- a/drivers/iommu/intel/dmar.c +++ b/drivers/iommu/intel/dmar.c @@ -1025,6 +1025,16 @@ static int map_iommu(struct intel_iommu *iommu, struct dmar_drhd_unit *drhd) goto release; } } + + if (cap_ecmds(iommu->cap)) { + int i; + + for (i = 0; i < DMA_MAX_NUM_ECMDCAP; i++) { + iommu->ecmdcap[i] = dmar_readq(iommu->reg + DMAR_ECCAP_REG + + i * DMA_ECMD_REG_STEP); + } + } + err = 0; goto out; @@ -2434,3 +2444,56 @@ bool dmar_platform_optin(void) return ret; } EXPORT_SYMBOL_GPL(dmar_platform_optin); + +#ifdef CONFIG_INTEL_IOMMU +#define ecmd_get_status_code(res) ((res & 0xff) >> 1) + +/* + * Function to submit a command to the enhanced command interface. The + * valid enhanced command descriptions are defined in Table 47 of the + * VT-d spec. The VT-d hardware implementation may support some but not + * all commands, which can be determined by checking the Enhanced + * Command Capability Register. + * + * Return values: + * - 0: Command successful without any error; + * - Negative: software error value; + * - Nonzero positive: failure status code defined in Table 48. + */ +int ecmd_submit_sync(struct intel_iommu *iommu, u8 ecmd, + u64 oa, bool has_ob, u64 ob) +{ + unsigned long flags; + u64 res; + int ret; + + if (!cap_ecmds(iommu->cap)) + return -ENODEV; + + raw_spin_lock_irqsave(&iommu->register_lock, flags); + + res = dmar_readq(iommu->reg + DMAR_ECRSP_REG); + if (res & DMA_ECMD_ECRSP_IP) { + ret = -EBUSY; + goto err; + } + + if (has_ob) + dmar_writeq(iommu->reg + DMAR_ECEO_REG, ob); + dmar_writeq(iommu->reg + DMAR_ECMD_REG, ecmd | (oa << DMA_ECMD_OA_SHIFT)); + + IOMMU_WAIT_OP(iommu, DMAR_ECRSP_REG, dmar_readq, + !(res & DMA_ECMD_ECRSP_IP), res); + + if (res & DMA_ECMD_ECRSP_IP) { + ret = -ETIMEDOUT; + goto err; + } + + ret = ecmd_get_status_code(res); +err: + raw_spin_unlock_irqrestore(&iommu->register_lock, flags); + + return ret; +} +#endif /* CONFIG_INTEL_IOMMU */ diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h index 5bcefbea55c9..f227107434ac 100644 --- a/drivers/iommu/intel/iommu.h +++ b/drivers/iommu/intel/iommu.h @@ -130,6 +130,10 @@ #define DMAR_PERFOVFOFF_REG 0x318 #define DMAR_PERFCNTROFF_REG 0x31c #define DMAR_PERFEVNTCAP_REG 0x380 +#define DMAR_ECMD_REG 0x400 +#define DMAR_ECEO_REG 0x408 +#define DMAR_ECRSP_REG 0x410 +#define DMAR_ECCAP_REG 0x430 #define DMAR_VCCAP_REG 0xe30 /* Virtual command capability register */ #define DMAR_VCMD_REG 0xe00 /* Virtual command register */ #define DMAR_VCRSP_REG 0xe10 /* Virtual command response register */ @@ -304,6 +308,26 @@ #define DMA_CCMD_SID(s) (((u64)((s) & 0xffff)) << 16) #define DMA_CCMD_DID(d) ((u64)((d) & 0xffff)) +/* ECMD_REG */ +#define DMA_MAX_NUM_ECMD 256 +#define DMA_MAX_NUM_ECMDCAP (DMA_MAX_NUM_ECMD / 64) +#define DMA_ECMD_REG_STEP 8 +#define DMA_ECMD_ENABLE 0xf0 +#define DMA_ECMD_DISABLE 0xf1 +#define DMA_ECMD_FREEZE 0xf4 +#define DMA_ECMD_UNFREEZE 0xf5 +#define DMA_ECMD_OA_SHIFT 16 +#define DMA_ECMD_ECRSP_IP 0x1 +#define DMA_ECMD_ECCAP3 3 +#define DMA_ECMD_ECCAP3_ECNTS (1ULL << 48) +#define DMA_ECMD_ECCAP3_DCNTS (1ULL << 49) +#define DMA_ECMD_ECCAP3_FCNTS (1ULL << 52) +#define DMA_ECMD_ECCAP3_UFCNTS (1ULL << 53) +#define DMA_ECMD_ECCAP3_ESSENTIAL (DMA_ECMD_ECCAP3_ECNTS | \ + DMA_ECMD_ECCAP3_DCNTS | \ + DMA_ECMD_ECCAP3_FCNTS | \ + DMA_ECMD_ECCAP3_UFCNTS) + /* FECTL_REG */ #define DMA_FECTL_IM (((u32)1) << 31) @@ -600,6 +624,7 @@ struct intel_iommu { u64 cap; u64 ecap; u64 vccap; + u64 ecmdcap[DMA_MAX_NUM_ECMDCAP]; u32 gcmd; /* Holds TE, EAFL. Don't need SRTP, SFL, WBF */ raw_spinlock_t register_lock; /* protect register handling */ int seq_id; /* sequence id of the iommu */ @@ -841,6 +866,15 @@ extern const struct iommu_ops intel_iommu_ops; extern int intel_iommu_sm; extern int iommu_calculate_agaw(struct intel_iommu *iommu); extern int iommu_calculate_max_sagaw(struct intel_iommu *iommu); +extern int ecmd_submit_sync(struct intel_iommu *iommu, u8 ecmd, + u64 oa, bool has_ob, u64 ob); + +static inline bool ecmd_has_pmu_essential(struct intel_iommu *iommu) +{ + return (iommu->ecmdcap[DMA_ECMD_ECCAP3] & DMA_ECMD_ECCAP3_ESSENTIAL) == + DMA_ECMD_ECCAP3_ESSENTIAL; +} + extern int dmar_disabled; extern int intel_iommu_enabled; #else @@ -852,6 +886,18 @@ static inline int iommu_calculate_max_sagaw(struct intel_iommu *iommu) { return 0; } + +static inline int ecmd_submit_sync(struct intel_iommu *iommu, u8 ecmd, + u64 oa, bool has_ob, u64 ob) +{ + return -EOPNOTSUPP; +} + +static inline bool ecmd_has_pmu_essential(struct intel_iommu *iommu) +{ + return false; +} + #define dmar_disabled (1) #define intel_iommu_enabled (0) #define intel_iommu_sm (0) From patchwork Wed Jan 11 20:25:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Liang, Kan" X-Patchwork-Id: 42185 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp3523939wrt; Wed, 11 Jan 2023 12:33:09 -0800 (PST) X-Google-Smtp-Source: AMrXdXv3CbBa5/HHaqqxstE52KS42CkspYF119/pQSDZMRf/KninulHSu9sIosNrYDk1pTfHTAxG X-Received: by 2002:a05:6a20:1455:b0:a4:829f:a43f with SMTP id a21-20020a056a20145500b000a4829fa43fmr107348155pzi.2.1673469189454; Wed, 11 Jan 2023 12:33:09 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1673469189; cv=none; d=google.com; s=arc-20160816; b=ls3inSDpHrU7FQI5MTLvkOI/kyIF2VY1srQxKgsdITb+H23w/mq7vkONbJB7t3NJEK V7T11zCe9yK+8jHxuJ7vrK7vLmjZig1FJCvU5851tp6KGxHW4vglujOuaJ6qgLPZ86sh 6qN4ZRrklq4ObSD2W0OJqY8oy3cX5C5OzJOET1jOXK3RXRiPQh8UiHbNBj89NVTP5Bs8 70tj+qwlz4CJ3QnvG6jJRmOpT/BjlVrBzWrb4GPDm+HdA6qGV8tla0+SNGXjhZxR2Z6A KkZJ3fMIfsP5CfhUS4gYaAFj9+Xt2Af6Gf8+OCgmXLi0mmx62JPeyBheNr/4ZmXLcmm5 GNBw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=CkyHQZ+0DYyupgjV9atXBcJ1XLtR8sP8lWGOAnLkQwA=; b=qNXF+DWf8kzVTCGxlCi0gzOKRsWMWXdAxA/VZI/x2lSgyQuBsNyN5RQa/VSrkqCFhk kAbO3NXDsatbdv3a9ulFteayP9DAYavlHx4jGnKolKtx5tUw/IxbXizJELcFdmbydGTz 0Vzo7C3gu4Vs44ptHoa40ubzrutiY1eBvyh4j5XFOUqlmLmdHwzFPB5XYqMp/6awAAC7 Z1EZvvUJYw6MxgvYLpUHYzdxcCwQXmX6lYaQu3ppED+HlJJ2exUNKiEqCQyXxga6K9sw NW8SoRW02pa5G2lEvHnTsiCNfOld5UpC/vGwDgIk+Y1ry9FzaTj2BN6a0T9Joipu/Oqa NzcA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=gYFZ0UZj; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id j6-20020a632306000000b004b484eb6657si7388445pgj.576.2023.01.11.12.32.56; Wed, 11 Jan 2023 12:33:09 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=gYFZ0UZj; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236652AbjAKU0C (ORCPT + 99 others); Wed, 11 Jan 2023 15:26:02 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44464 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235503AbjAKUZe (ORCPT ); Wed, 11 Jan 2023 15:25:34 -0500 Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 303FB13E2C for ; Wed, 11 Jan 2023 12:25:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1673468731; x=1705004731; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ac+A4EC828ipP6xquzOaR4sgT5/PuouJPXcDkBfbNng=; b=gYFZ0UZj1PuLAPG7VT+QnAQrbIpFbmcxQwy8D/PGa2GiH6qkbZ4NEVRT 0wDYwXGwqqURZa/YFSUxeV604QXyPZob6y/xJGZd9ULKvSafEVTGbvtGw NCkFEkpGYzFzooUoFHTSi4U938uXiCMj4NKsHwe1wHZgJKUoboIe7SQBm Q0KQy1VDB8gwrkJ00UgkLiO1h/qcKsb2gQzFcgraUGqwoKreDr/NnZLxF 8vJlGsFGyRAi7TjTnIsYhmGTzpGzuj2n1sisxjN28ulYqyDFYkwASV4VS WcrKn+p1WMJcYrJH9XK+GFLP/eF0E7zcdFWQGkqkSAJ0tiyMXaqnCDsfC w==; X-IronPort-AV: E=McAfee;i="6500,9779,10586"; a="350754995" X-IronPort-AV: E=Sophos;i="5.96,317,1665471600"; d="scan'208";a="350754995" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Jan 2023 12:25:29 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10586"; a="781518071" X-IronPort-AV: E=Sophos;i="5.96,317,1665471600"; d="scan'208";a="781518071" Received: from kanliang-dev.jf.intel.com ([10.165.154.102]) by orsmga004.jf.intel.com with ESMTP; 11 Jan 2023 12:25:28 -0800 From: kan.liang@linux.intel.com To: joro@8bytes.org, will@kernel.org, baolu.lu@linux.intel.com, dwmw2@infradead.org, robin.murphy@arm.com, robert.moore@intel.com, rafael.j.wysocki@intel.com, lenb@kernel.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org Cc: Kan Liang Subject: [PATCH 4/7] iommu/vt-d: Add IOMMU perfmon support Date: Wed, 11 Jan 2023 12:25:01 -0800 Message-Id: <20230111202504.378258-5-kan.liang@linux.intel.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20230111202504.378258-1-kan.liang@linux.intel.com> References: <20230111202504.378258-1-kan.liang@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1754759628833606318?= X-GMAIL-MSGID: =?utf-8?q?1754759628833606318?= From: Kan Liang Implement the IOMMU performance monitor capability, which supports the collection of information about key events occurring during operation of the remapping hardware, to aid performance tuning and debug. The IOMMU perfmon support is implemented as part of the IOMMU driver and interfaces with the Linux perf subsystem. The IOMMU PMU has the following unique features compared with the other PMUs. - Support counting. Not support sampling. - Does not support per-thread counting. The scope is system-wide. - Support per-counter capability register. The event constraints can be enumerated. - The available event and event group can also be enumerated. - Extra Enhanced Commands are introduced to control the counters. Add a new variable, struct iommu_pmu *pmu, to in the struct intel_iommu to track the PMU related information. Add iommu_pmu_register() and iommu_pmu_unregister() to register and unregister a IOMMU PMU. The register function setup the IOMMU PMU ops and invoke the standard perf_pmu_register() interface to register a PMU in the perf subsystem. This patch only exposes the functions. The following patch will enable them in the IOMMU driver. The IOMMU PMUs can be found under /sys/bus/event_source/devices/dmar* The available filters and event format can be found at the format folder $ ls /sys/bus/event_source/devices/dmar0/format/ event event_group filter_ats filter_page_table The supported events can be found at the events folder $ ls /sys/bus/event_source/devices/dmar0/events/ ats_blocked int_cache_hit_nonposted iommu_mrds pasid_cache_lookup ctxt_cache_hit int_cache_hit_posted iommu_requests pg_req_posted ctxt_cache_lookup int_cache_lookup iotlb_hit pw_occupancy fs_nonleaf_hit iommu_clocks iotlb_lookup ss_nonleaf_hit fs_nonleaf_lookup iommu_mem_blocked pasid_cache_hit ss_nonleaf_lookup The command below illustrates filter usage with a simple example. $ perf stat -e dmar0/iommu_requests,filter_ats=0/ -a sleep 1 Performance counter stats for 'system wide': 2135 dmar0/iommu_requests,filter_ats=0/ 1.001087695 seconds time elapsed Signed-off-by: Kan Liang --- .../sysfs-bus-event_source-devices-iommu | 24 + drivers/iommu/intel/iommu.h | 15 + drivers/iommu/intel/perfmon.c | 496 ++++++++++++++++++ drivers/iommu/intel/perfmon.h | 24 + 4 files changed, 559 insertions(+) create mode 100644 Documentation/ABI/testing/sysfs-bus-event_source-devices-iommu diff --git a/Documentation/ABI/testing/sysfs-bus-event_source-devices-iommu b/Documentation/ABI/testing/sysfs-bus-event_source-devices-iommu new file mode 100644 index 000000000000..04e08851d8e6 --- /dev/null +++ b/Documentation/ABI/testing/sysfs-bus-event_source-devices-iommu @@ -0,0 +1,24 @@ +What: /sys/bus/event_source/devices/dmar*/format +Date: Jan 2023 +KernelVersion: 6.3 +Contact: Kan Liang +Description: Read-only. Attribute group to describe the magic bits + that go into perf_event_attr.config, + perf_event_attr.config1 or perf_event_attr.config2 for + the IOMMU pmu. (See also + ABI/testing/sysfs-bus-event_source-devices-format). + + Each attribute in this group defines a bit range in + perf_event_attr.config, perf_event_attr.config1, + or perf_event_attr.config2. All supported attributes + are listed below (See the VT-d Spec 4.0 for possible + attribute values):: + + event = "config:0-27" - event ID + event_group = "config:28-31" - event group ID + + filter_requester_id = "config1:0-15" - Requester ID filter + filter_domain = "config1:16-31" - Domain ID filter + filter_pasid = "config1:32-53" - PASID filter + filter_ats = "config2:0-4" - Address Type filter + filter_page_table = "config2:8-12" - Page Table Level filter diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h index f227107434ac..bbc5dda903e9 100644 --- a/drivers/iommu/intel/iommu.h +++ b/drivers/iommu/intel/iommu.h @@ -22,6 +22,7 @@ #include #include #include +#include #include #include @@ -601,6 +602,16 @@ struct dmar_domain { iommu core */ }; +/* + * In theory, the VT-d 4.0 spec can support up to 2 ^ 16 counters. + * But in practice, there are only 14 counters for the existing + * platform. Setting the max number of counters to 64 should be good + * enough for a long time. Also, supporting more than 64 counters + * requires more extras, e.g., extra freeze and overflow registers, + * which is not necessary for now. + */ +#define IOMMU_PMU_IDX_MAX 64 + struct iommu_pmu { struct intel_iommu *iommu; u32 num_cntr; /* Number of counters */ @@ -615,6 +626,10 @@ struct iommu_pmu { u64 *evcap; /* Indicates all supported events */ u32 **cntr_evcap; /* Supported events of each counter. */ + + struct pmu pmu; + DECLARE_BITMAP(used_mask, IOMMU_PMU_IDX_MAX); + struct perf_event *event_list[IOMMU_PMU_IDX_MAX]; }; struct intel_iommu { diff --git a/drivers/iommu/intel/perfmon.c b/drivers/iommu/intel/perfmon.c index bc090f329c32..43a5075eaecd 100644 --- a/drivers/iommu/intel/perfmon.c +++ b/drivers/iommu/intel/perfmon.c @@ -8,6 +8,475 @@ #include "iommu.h" #include "perfmon.h" +PMU_FORMAT_ATTR(event, "config:0-27"); /* ES: Events Select */ +PMU_FORMAT_ATTR(event_group, "config:28-31"); /* EGI: Event Group Index */ + +static struct attribute *iommu_pmu_format_attrs[] = { + &format_attr_event_group.attr, + &format_attr_event.attr, + NULL +}; + +static struct attribute_group iommu_pmu_format_attr_group = { + .name = "format", + .attrs = iommu_pmu_format_attrs, +}; + +/* The available events are added in attr_update later */ +static struct attribute *attrs_empty[] = { + NULL +}; + +static struct attribute_group iommu_pmu_events_attr_group = { + .name = "events", + .attrs = attrs_empty, +}; + +static const struct attribute_group *iommu_pmu_attr_groups[] = { + &iommu_pmu_format_attr_group, + &iommu_pmu_events_attr_group, + NULL +}; + +static inline struct iommu_pmu *dev_to_iommu_pmu(struct device *dev) +{ + /* + * The perf_event creates its own dev for each PMU. + * See pmu_dev_alloc() + */ + return container_of(dev_get_drvdata(dev), struct iommu_pmu, pmu); +} + +#define IOMMU_PMU_ATTR(_name, _format, _filter) \ + PMU_FORMAT_ATTR(_name, _format); \ + \ +static struct attribute *_name##_attr[] = { \ + &format_attr_##_name.attr, \ + NULL \ +}; \ + \ +static umode_t \ +_name##_is_visible(struct kobject *kobj, struct attribute *attr, int i) \ +{ \ + struct device *dev = kobj_to_dev(kobj); \ + struct iommu_pmu *iommu_pmu = dev_to_iommu_pmu(dev); \ + \ + if (!iommu_pmu) \ + return 0; \ + return (iommu_pmu->filter & _filter) ? attr->mode : 0; \ +} \ + \ +static struct attribute_group _name = { \ + .name = "format", \ + .attrs = _name##_attr, \ + .is_visible = _name##_is_visible, \ +}; + +IOMMU_PMU_ATTR(filter_requester_id, "config1:0-15", IOMMU_PMU_FILTER_REQUESTER_ID); +IOMMU_PMU_ATTR(filter_domain, "config1:16-31", IOMMU_PMU_FILTER_DOMAIN); +IOMMU_PMU_ATTR(filter_pasid, "config1:32-53", IOMMU_PMU_FILTER_PASID); +IOMMU_PMU_ATTR(filter_ats, "config2:0-4", IOMMU_PMU_FILTER_ATS); +IOMMU_PMU_ATTR(filter_page_table, "config2:8-12", IOMMU_PMU_FILTER_PAGE_TABLE); + +#define iommu_pmu_get_requester_id(filter) ((filter) & 0xffff) +#define iommu_pmu_get_domain(filter) (((filter) >> 16) & 0xffff) +#define iommu_pmu_get_pasid(filter) (((filter) >> 32) & 0x3fffff) +#define iommu_pmu_get_ats(filter) ((filter) & 0x1f) +#define iommu_pmu_get_page_table(filter) (((filter) >> 8) & 0x1f) + +#define iommu_pmu_set_filter(_name, _config, _filter, _idx) \ +{ \ + if ((iommu_pmu->filter & _filter) && iommu_pmu_get_##_name(_config)) { \ + dmar_writel(iommu_pmu->cfg_reg + _idx * IOMMU_PMU_CFG_OFFSET + \ + IOMMU_PMU_CFG_SIZE + \ + (ffs(_filter) - 1) * IOMMU_PMU_CFG_FILTERS_OFFSET, \ + iommu_pmu_get_##_name(_config) | IOMMU_PMU_FILTER_EN);\ + } \ +} + +#define iommu_pmu_clear_filter(_filter, _idx) \ +{ \ + if (iommu_pmu->filter & _filter) { \ + dmar_writel(iommu_pmu->cfg_reg + _idx * IOMMU_PMU_CFG_OFFSET + \ + IOMMU_PMU_CFG_SIZE + \ + (ffs(_filter) - 1) * IOMMU_PMU_CFG_FILTERS_OFFSET, \ + 0); \ + } \ +} + +/* + * Define the event attr related functions + * Input: _name: event attr name + * _string: string of the event in sysfs + * _g_idx: event group encoding + * _event: event encoding + */ +#define IOMMU_PMU_EVENT_ATTR(_name, _string, _g_idx, _event) \ + PMU_EVENT_ATTR_STRING(_name, event_attr_##_name, _string) \ + \ +static struct attribute *_name##_attr[] = { \ + &event_attr_##_name.attr.attr, \ + NULL \ +}; \ + \ +static umode_t \ +_name##_is_visible(struct kobject *kobj, struct attribute *attr, int i) \ +{ \ + struct device *dev = kobj_to_dev(kobj); \ + struct iommu_pmu *iommu_pmu = dev_to_iommu_pmu(dev); \ + \ + if (!iommu_pmu) \ + return 0; \ + return (iommu_pmu->evcap[_g_idx] & _event) ? attr->mode : 0; \ +} \ + \ +static struct attribute_group _name = { \ + .name = "events", \ + .attrs = _name##_attr, \ + .is_visible = _name##_is_visible, \ +}; + +IOMMU_PMU_EVENT_ATTR(iommu_clocks, "event_group=0x0,event=0x001", 0x0, 0x001) +IOMMU_PMU_EVENT_ATTR(iommu_requests, "event_group=0x0,event=0x002", 0x0, 0x002) +IOMMU_PMU_EVENT_ATTR(pw_occupancy, "event_group=0x0,event=0x004", 0x0, 0x004) +IOMMU_PMU_EVENT_ATTR(ats_blocked, "event_group=0x0,event=0x008", 0x0, 0x008) +IOMMU_PMU_EVENT_ATTR(iommu_mrds, "event_group=0x1,event=0x001", 0x1, 0x001) +IOMMU_PMU_EVENT_ATTR(iommu_mem_blocked, "event_group=0x1,event=0x020", 0x1, 0x020) +IOMMU_PMU_EVENT_ATTR(pg_req_posted, "event_group=0x1,event=0x040", 0x1, 0x040) +IOMMU_PMU_EVENT_ATTR(ctxt_cache_lookup, "event_group=0x2,event=0x001", 0x2, 0x001) +IOMMU_PMU_EVENT_ATTR(ctxt_cache_hit, "event_group=0x2,event=0x002", 0x2, 0x002) +IOMMU_PMU_EVENT_ATTR(pasid_cache_lookup, "event_group=0x2,event=0x004", 0x2, 0x004) +IOMMU_PMU_EVENT_ATTR(pasid_cache_hit, "event_group=0x2,event=0x008", 0x2, 0x008) +IOMMU_PMU_EVENT_ATTR(ss_nonleaf_lookup, "event_group=0x2,event=0x010", 0x2, 0x010) +IOMMU_PMU_EVENT_ATTR(ss_nonleaf_hit, "event_group=0x2,event=0x020", 0x2, 0x020) +IOMMU_PMU_EVENT_ATTR(fs_nonleaf_lookup, "event_group=0x2,event=0x040", 0x2, 0x040) +IOMMU_PMU_EVENT_ATTR(fs_nonleaf_hit, "event_group=0x2,event=0x080", 0x2, 0x080) +IOMMU_PMU_EVENT_ATTR(hpt_nonleaf_lookup, "event_group=0x2,event=0x100", 0x2, 0x100) +IOMMU_PMU_EVENT_ATTR(hpt_nonleaf_hit, "event_group=0x2,event=0x200", 0x2, 0x200) +IOMMU_PMU_EVENT_ATTR(iotlb_lookup, "event_group=0x3,event=0x001", 0x3, 0x001) +IOMMU_PMU_EVENT_ATTR(iotlb_hit, "event_group=0x3,event=0x002", 0x3, 0x002) +IOMMU_PMU_EVENT_ATTR(hpt_leaf_lookup, "event_group=0x3,event=0x004", 0x3, 0x004) +IOMMU_PMU_EVENT_ATTR(hpt_leaf_hit, "event_group=0x3,event=0x008", 0x3, 0x008) +IOMMU_PMU_EVENT_ATTR(int_cache_lookup, "event_group=0x4,event=0x001", 0x4, 0x001) +IOMMU_PMU_EVENT_ATTR(int_cache_hit_nonposted, "event_group=0x4,event=0x002", 0x4, 0x002) +IOMMU_PMU_EVENT_ATTR(int_cache_hit_posted, "event_group=0x4,event=0x004", 0x4, 0x004) + + +static const struct attribute_group *iommu_pmu_attr_update[] = { + &filter_requester_id, + &filter_domain, + &filter_pasid, + &filter_ats, + &filter_page_table, + &iommu_clocks, + &iommu_requests, + &pw_occupancy, + &ats_blocked, + &iommu_mrds, + &iommu_mem_blocked, + &pg_req_posted, + &ctxt_cache_lookup, + &ctxt_cache_hit, + &pasid_cache_lookup, + &pasid_cache_hit, + &ss_nonleaf_lookup, + &ss_nonleaf_hit, + &fs_nonleaf_lookup, + &fs_nonleaf_hit, + &hpt_nonleaf_lookup, + &hpt_nonleaf_hit, + &iotlb_lookup, + &iotlb_hit, + &hpt_leaf_lookup, + &hpt_leaf_hit, + &int_cache_lookup, + &int_cache_hit_nonposted, + &int_cache_hit_posted, + NULL +}; + +static inline void __iomem * +iommu_event_base(struct iommu_pmu *iommu_pmu, int idx) +{ + return iommu_pmu->cntr_reg + idx * iommu_pmu->cntr_stride; +} + +static inline void __iomem * +iommu_config_base(struct iommu_pmu *iommu_pmu, int idx) +{ + return iommu_pmu->cfg_reg + idx * IOMMU_PMU_CFG_OFFSET; +} + +static inline struct iommu_pmu *iommu_event_to_pmu(struct perf_event *event) +{ + return container_of(event->pmu, struct iommu_pmu, pmu); +} + +static inline u64 iommu_event_config(struct perf_event *event) +{ + u64 config = event->attr.config; + + return (iommu_event_select(config) << IOMMU_EVENT_CFG_ES_SHIFT) | + (iommu_event_group(config) << IOMMU_EVENT_CFG_EGI_SHIFT) | + IOMMU_EVENT_CFG_INT; +} + +static inline bool is_iommu_pmu_event(struct iommu_pmu *iommu_pmu, + struct perf_event *event) +{ + return event->pmu == &iommu_pmu->pmu; +} + +static int iommu_pmu_validate_event(struct perf_event *event) +{ + struct iommu_pmu *iommu_pmu = iommu_event_to_pmu(event); + u32 event_group = iommu_event_group(event->attr.config); + + if (event_group >= iommu_pmu->num_eg) + return -EINVAL; + + return 0; +} + +static int iommu_pmu_validate_group(struct perf_event *event) +{ + struct iommu_pmu *iommu_pmu = iommu_event_to_pmu(event); + struct perf_event *sibling; + int nr = 0; + + /* + * All events in a group must be scheduled simultaneously. + * Check whether there is enough counters for all the events. + */ + for_each_sibling_event(sibling, event->group_leader) { + if (!is_iommu_pmu_event(iommu_pmu, sibling) || + sibling->state <= PERF_EVENT_STATE_OFF) + continue; + + if (++nr > iommu_pmu->num_cntr) + return -EINVAL; + } + + return 0; +} + +static int iommu_pmu_event_init(struct perf_event *event) +{ + struct hw_perf_event *hwc = &event->hw; + + if (event->attr.type != event->pmu->type) + return -ENOENT; + + /* sampling not supported */ + if (event->attr.sample_period) + return -EINVAL; + + if (event->cpu < 0) + return -EINVAL; + + if (iommu_pmu_validate_event(event)) + return -EINVAL; + + hwc->config = iommu_event_config(event); + + return iommu_pmu_validate_group(event); +} + +static void iommu_pmu_event_update(struct perf_event *event) +{ + struct iommu_pmu *iommu_pmu = iommu_event_to_pmu(event); + struct hw_perf_event *hwc = &event->hw; + u64 prev_count, new_count, delta; + int shift = 64 - iommu_pmu->cntr_width; + +again: + prev_count = local64_read(&hwc->prev_count); + new_count = dmar_readq(iommu_event_base(iommu_pmu, hwc->idx)); + if (local64_xchg(&hwc->prev_count, new_count) != prev_count) + goto again; + + /* + * The counter width is enumerated. + * Always shift the counter before using it. + */ + delta = (new_count << shift) - (prev_count << shift); + delta >>= shift; + + local64_add(delta, &event->count); +} + +static void iommu_pmu_start(struct perf_event *event, int flags) +{ + struct iommu_pmu *iommu_pmu = iommu_event_to_pmu(event); + struct intel_iommu *iommu = iommu_pmu->iommu; + struct hw_perf_event *hwc = &event->hw; + u64 count; + + if (WARN_ON_ONCE(!(hwc->state & PERF_HES_STOPPED))) + return; + + if (WARN_ON_ONCE(hwc->idx < 0 || hwc->idx >= IOMMU_PMU_IDX_MAX)) + return; + + if (flags & PERF_EF_RELOAD) + WARN_ON_ONCE(!(event->hw.state & PERF_HES_UPTODATE)); + + hwc->state = 0; + + /* Always reprogram the period */ + count = dmar_readq(iommu_event_base(iommu_pmu, hwc->idx)); + local64_set((&hwc->prev_count), count); + + ecmd_submit_sync(iommu, DMA_ECMD_ENABLE, hwc->idx, false, 0); + + perf_event_update_userpage(event); +} + +static void iommu_pmu_stop(struct perf_event *event, int flags) +{ + struct iommu_pmu *iommu_pmu = iommu_event_to_pmu(event); + struct intel_iommu *iommu = iommu_pmu->iommu; + struct hw_perf_event *hwc = &event->hw; + + if (!(hwc->state & PERF_HES_STOPPED)) { + ecmd_submit_sync(iommu, DMA_ECMD_DISABLE, hwc->idx, false, 0); + + iommu_pmu_event_update(event); + + hwc->state |= PERF_HES_STOPPED | PERF_HES_UPTODATE; + } +} + +static inline int +iommu_pmu_validate_per_cntr_event(struct iommu_pmu *iommu_pmu, + int idx, struct perf_event *event) +{ + u32 event_group = iommu_event_group(event->attr.config); + u32 select = iommu_event_select(event->attr.config); + + if (!(iommu_pmu->cntr_evcap[idx][event_group] & select)) + return -EINVAL; + + return 0; +} + +static int iommu_pmu_assign_event(struct iommu_pmu *iommu_pmu, + struct perf_event *event) +{ + struct hw_perf_event *hwc = &event->hw; + int idx; + + /* + * The counters which support limited events are usually at the end. + * Schedule them first to accommodate more events. + */ + for (idx = iommu_pmu->num_cntr - 1; idx >= 0; idx--) { + if (test_and_set_bit(idx, iommu_pmu->used_mask)) + continue; + /* Check per-counter event capabilities */ + if (!iommu_pmu_validate_per_cntr_event(iommu_pmu, idx, event)) + break; + clear_bit(idx, iommu_pmu->used_mask); + } + if (idx < 0) + return -EINVAL; + + iommu_pmu->event_list[idx] = event; + hwc->idx = idx; + + /* config events */ + dmar_writeq(iommu_config_base(iommu_pmu, idx), hwc->config); + + iommu_pmu_set_filter(requester_id, event->attr.config1, + IOMMU_PMU_FILTER_REQUESTER_ID, idx); + iommu_pmu_set_filter(domain, event->attr.config1, + IOMMU_PMU_FILTER_DOMAIN, idx); + iommu_pmu_set_filter(pasid, event->attr.config1, + IOMMU_PMU_FILTER_PASID, idx); + iommu_pmu_set_filter(ats, event->attr.config2, + IOMMU_PMU_FILTER_ATS, idx); + iommu_pmu_set_filter(page_table, event->attr.config2, + IOMMU_PMU_FILTER_PAGE_TABLE, idx); + + return 0; +} + +static int iommu_pmu_add(struct perf_event *event, int flags) +{ + struct iommu_pmu *iommu_pmu = iommu_event_to_pmu(event); + struct hw_perf_event *hwc = &event->hw; + int ret; + + ret = iommu_pmu_assign_event(iommu_pmu, event); + if (ret < 0) + return ret; + + hwc->state = PERF_HES_UPTODATE | PERF_HES_STOPPED; + + if (flags & PERF_EF_START) + iommu_pmu_start(event, 0); + + return 0; +} + +static void iommu_pmu_del(struct perf_event *event, int flags) +{ + struct iommu_pmu *iommu_pmu = iommu_event_to_pmu(event); + int idx = event->hw.idx; + + iommu_pmu_stop(event, PERF_EF_UPDATE); + + iommu_pmu_clear_filter(IOMMU_PMU_FILTER_REQUESTER_ID, idx); + iommu_pmu_clear_filter(IOMMU_PMU_FILTER_DOMAIN, idx); + iommu_pmu_clear_filter(IOMMU_PMU_FILTER_PASID, idx); + iommu_pmu_clear_filter(IOMMU_PMU_FILTER_ATS, idx); + iommu_pmu_clear_filter(IOMMU_PMU_FILTER_PAGE_TABLE, idx); + + iommu_pmu->event_list[idx] = NULL; + event->hw.idx = -1; + clear_bit(idx, iommu_pmu->used_mask); + + perf_event_update_userpage(event); +} + +static void iommu_pmu_enable(struct pmu *pmu) +{ + struct iommu_pmu *iommu_pmu = container_of(pmu, struct iommu_pmu, pmu); + struct intel_iommu *iommu = iommu_pmu->iommu; + + ecmd_submit_sync(iommu, DMA_ECMD_UNFREEZE, 0, false, 0); +} + +static void iommu_pmu_disable(struct pmu *pmu) +{ + struct iommu_pmu *iommu_pmu = container_of(pmu, struct iommu_pmu, pmu); + struct intel_iommu *iommu = iommu_pmu->iommu; + + ecmd_submit_sync(iommu, DMA_ECMD_FREEZE, 0, false, 0); +} + +static int __iommu_pmu_register(struct intel_iommu *iommu) +{ + struct iommu_pmu *iommu_pmu = iommu->pmu; + + iommu_pmu->pmu.name = iommu->name; + iommu_pmu->pmu.task_ctx_nr = perf_invalid_context; + iommu_pmu->pmu.event_init = iommu_pmu_event_init; + iommu_pmu->pmu.pmu_enable = iommu_pmu_enable; + iommu_pmu->pmu.pmu_disable = iommu_pmu_disable; + iommu_pmu->pmu.add = iommu_pmu_add; + iommu_pmu->pmu.del = iommu_pmu_del; + iommu_pmu->pmu.start = iommu_pmu_start; + iommu_pmu->pmu.stop = iommu_pmu_stop; + iommu_pmu->pmu.read = iommu_pmu_event_update; + iommu_pmu->pmu.attr_groups = iommu_pmu_attr_groups; + iommu_pmu->pmu.attr_update = iommu_pmu_attr_update; + iommu_pmu->pmu.capabilities = PERF_PMU_CAP_NO_EXCLUDE; + iommu_pmu->pmu.module = THIS_MODULE; + + return perf_pmu_register(&iommu_pmu->pmu, iommu_pmu->pmu.name, -1); +} + static inline void __iomem * get_perf_reg_address(struct intel_iommu *iommu, u32 offset) { @@ -40,11 +509,21 @@ int alloc_iommu_pmu(struct intel_iommu *iommu) if (!pcap_interrupt(perfcap)) return -ENODEV; + /* Check required Enhanced Command Capability */ + if (!ecmd_has_pmu_essential(iommu)) + return -ENODEV; + iommu_pmu = kzalloc(sizeof(*iommu_pmu), GFP_KERNEL); if (!iommu_pmu) return -ENOMEM; iommu_pmu->num_cntr = pcap_num_cntr(perfcap); + if (iommu_pmu->num_cntr > IOMMU_PMU_IDX_MAX) { + WARN_ONCE(1, "The number of IOMMU counters %d > max(%d), clipping!", + iommu_pmu->num_cntr, IOMMU_PMU_IDX_MAX); + iommu_pmu->num_cntr = IOMMU_PMU_IDX_MAX; + } + iommu_pmu->cntr_width = pcap_cntr_width(perfcap); iommu_pmu->filter = pcap_filters_mask(perfcap); iommu_pmu->cntr_stride = pcap_cntr_stride(perfcap); @@ -157,3 +636,20 @@ void free_iommu_pmu(struct intel_iommu *iommu) kfree(iommu_pmu); iommu->pmu = NULL; } + +void iommu_pmu_register(struct intel_iommu *iommu) +{ + if (!iommu->pmu) + return; + + if (__iommu_pmu_register(iommu)) { + pr_err("Failed to register PMU for iommu (seq_id = %d)\n", + iommu->seq_id); + } +} + +void iommu_pmu_unregister(struct intel_iommu *iommu) +{ + if (iommu->pmu) + perf_pmu_unregister(&iommu->pmu->pmu); +} diff --git a/drivers/iommu/intel/perfmon.h b/drivers/iommu/intel/perfmon.h index 8587c80501cd..b60f0cad5bfd 100644 --- a/drivers/iommu/intel/perfmon.h +++ b/drivers/iommu/intel/perfmon.h @@ -7,6 +7,14 @@ #define IOMMU_PMU_NUM_OFF_REGS 4 #define IOMMU_PMU_OFF_REGS_STEP 4 +#define IOMMU_PMU_FILTER_REQUESTER_ID 0x01 +#define IOMMU_PMU_FILTER_DOMAIN 0x02 +#define IOMMU_PMU_FILTER_PASID 0x04 +#define IOMMU_PMU_FILTER_ATS 0x08 +#define IOMMU_PMU_FILTER_PAGE_TABLE 0x10 + +#define IOMMU_PMU_FILTER_EN (1 << 31) + #define IOMMU_PMU_CFG_OFFSET 0x100 #define IOMMU_PMU_CFG_CNTRCAP_OFFSET 0x80 #define IOMMU_PMU_CFG_CNTREVCAP_OFFSET 0x84 @@ -21,12 +29,18 @@ #define iommu_cntrcap_ios(p) ((p >> 16) & 0x1) #define iommu_cntrcap_egcnt(p) ((p >> 28) & 0xf) +#define IOMMU_EVENT_CFG_EGI_SHIFT 8 +#define IOMMU_EVENT_CFG_ES_SHIFT 32 +#define IOMMU_EVENT_CFG_INT (1ULL << 1) + #define iommu_event_select(p) ((p) & 0xfffffff) #define iommu_event_group(p) ((p >> 28) & 0xf) #ifdef CONFIG_INTEL_IOMMU_PERF_EVENTS int alloc_iommu_pmu(struct intel_iommu *iommu); void free_iommu_pmu(struct intel_iommu *iommu); +void iommu_pmu_register(struct intel_iommu *iommu); +void iommu_pmu_unregister(struct intel_iommu *iommu); #else static inline int alloc_iommu_pmu(struct intel_iommu *iommu) @@ -38,4 +52,14 @@ static inline void free_iommu_pmu(struct intel_iommu *iommu) { } + +static inline void +iommu_pmu_register(struct intel_iommu *iommu) +{ +} + +static inline void +iommu_pmu_unregister(struct intel_iommu *iommu) +{ +} #endif /* CONFIG_INTEL_IOMMU_PERF_EVENTS */ From patchwork Wed Jan 11 20:25:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Liang, Kan" X-Patchwork-Id: 42187 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp3524120wrt; Wed, 11 Jan 2023 12:33:40 -0800 (PST) X-Google-Smtp-Source: AMrXdXt1cfembPkan1e/Q2MFuGnsJ7JxTQVmhzDqTtHo81ZFTUyTc0uJG6WcWObtRS8FZX9y0PRo X-Received: by 2002:a05:6a20:e606:b0:9d:efd3:66e5 with SMTP id my6-20020a056a20e60600b0009defd366e5mr82945532pzb.44.1673469220029; Wed, 11 Jan 2023 12:33:40 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1673469220; cv=none; d=google.com; s=arc-20160816; b=Vqo2v4ve8IN5ijL0Ar0b07utsBCowc/vHNy1FU7Fwnjdg+8MwAkX+aik7aaefpGhlv LHc5JbKLUYPER3Jj9HhCtzJBk/MOrBL6J4wWdMMIc7/FynS1M0g0fqS2LOgfKRC1LDkm DoZMZQDsgaM4PJRFP3Uwusg8X4cp3D9YjWU5+gSfB8hHnU2oxyTM4Mo3wQdmnczm5Kdb XFKwkd1oVK2L0Cs9ow8aAAxvEKZBCMN/Ci7cSyzLKk7XcRDrayI2NdZ4C9U439Dm88rY 2IhRIZLNb07tLFUrXhnGuVkJ+5W91XVZlv4AkfxcbZeA+imQOJ5EVqy3UZxKB7iQDUzO TZ2w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=nAjgCqZJ955YjlKaa02lhctLi9k//2YFA0pbet6VWK0=; b=aZ4eJdU50RndYXwH562f19ZNKy5hDr0t6OTWk7k2MDQ3TxCm+4MIfLTUHiZT5BgmQj 6udEREL3ubbpNbTjyQZDFU1nehKjAP6nccEtpeVg/PMp/MiLFSHp2tAyGaMRh4SREKnD 4Oy/otiWyaWaLb+HBgfCRwtyc9MmZ0eJP7BYL30pJH05HjLOnyXjv2Tjki0agRqE2lV8 B5bkV+1tcqsD6F2+tNISBn5jRdQksvtc20/S2iCsBK7CpeV85hkS+sJDnRtF2Rc0w/sC +VsOG1gZJZBV1dfKMqjig3Hn+i9ItBmPV0JzhQNseb5MirIOPACY2MBGw2RGKDmED/+b xF2A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=N245u+Kk; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id e188-20020a6369c5000000b00476c369eaa9si14997979pgc.146.2023.01.11.12.33.25; Wed, 11 Jan 2023 12:33:39 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=N245u+Kk; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235916AbjAKU0U (ORCPT + 99 others); Wed, 11 Jan 2023 15:26:20 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44466 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233416AbjAKUZe (ORCPT ); Wed, 11 Jan 2023 15:25:34 -0500 Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D94E513E36 for ; Wed, 11 Jan 2023 12:25:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1673468732; x=1705004732; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=GMeoi/8w6qsbk40t9XpztUuwD163lNMlvbB+trQLXbE=; b=N245u+KknMeW1p/yusOzv5KVwal+aNVI4DJegWg+LLlxPKBeu3AmImgo IrFfuotI6QyE+/GrMIB1epA+PwcfWBxBzLrgBr7HcjPkx5t7wIaWTNLEK CcmJN2m3wTdJO4RM5OXJ/eslI2OUueccJz0CVPrC3TfzmmiHwU7rOLVax +fMrCr7OpPsj90rDKDJh0jI4zXN7NKdJstTY4fDtVR6kf2SRZKPQd5quv KSYypWpRa8eeVieW3EgCFdrrFkwCLRB3UYPJD+ULZ10B8mIo352a4yJ6x E816JadM6ZsxYtSi9ChUluaHGynbnbx/pqqFFuN2ugJwc7IvQO5xbAE+A A==; X-IronPort-AV: E=McAfee;i="6500,9779,10586"; a="350755002" X-IronPort-AV: E=Sophos;i="5.96,317,1665471600"; d="scan'208";a="350755002" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Jan 2023 12:25:29 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10586"; a="781518074" X-IronPort-AV: E=Sophos;i="5.96,317,1665471600"; d="scan'208";a="781518074" Received: from kanliang-dev.jf.intel.com ([10.165.154.102]) by orsmga004.jf.intel.com with ESMTP; 11 Jan 2023 12:25:28 -0800 From: kan.liang@linux.intel.com To: joro@8bytes.org, will@kernel.org, baolu.lu@linux.intel.com, dwmw2@infradead.org, robin.murphy@arm.com, robert.moore@intel.com, rafael.j.wysocki@intel.com, lenb@kernel.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org Cc: Kan Liang Subject: [PATCH 5/7] iommu/vt-d: Support cpumask for IOMMU perfmon Date: Wed, 11 Jan 2023 12:25:02 -0800 Message-Id: <20230111202504.378258-6-kan.liang@linux.intel.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20230111202504.378258-1-kan.liang@linux.intel.com> References: <20230111202504.378258-1-kan.liang@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1754759660869532287?= X-GMAIL-MSGID: =?utf-8?q?1754759660869532287?= From: Kan Liang The perf subsystem assumes that all counters are by default per-CPU. So the user space tool reads a counter from each CPU. However, the IOMMU counters are system-wide and can be read from any CPU. Here we use a CPU mask to restrict counting to one CPU to handle the issue. (with CPU hotplug notifier to choose a different CPU if the chosen one is taken off-line). The CPU is exposed to /sys/bus/event_source/devices/dmar*/cpumask for the user space perf tool. Signed-off-by: Kan Liang --- .../sysfs-bus-event_source-devices-iommu | 8 ++ drivers/iommu/intel/perfmon.c | 113 ++++++++++++++++-- include/linux/cpuhotplug.h | 1 + 3 files changed, 115 insertions(+), 7 deletions(-) diff --git a/Documentation/ABI/testing/sysfs-bus-event_source-devices-iommu b/Documentation/ABI/testing/sysfs-bus-event_source-devices-iommu index 04e08851d8e6..3519954fe713 100644 --- a/Documentation/ABI/testing/sysfs-bus-event_source-devices-iommu +++ b/Documentation/ABI/testing/sysfs-bus-event_source-devices-iommu @@ -22,3 +22,11 @@ Description: Read-only. Attribute group to describe the magic bits filter_pasid = "config1:32-53" - PASID filter filter_ats = "config2:0-4" - Address Type filter filter_page_table = "config2:8-12" - Page Table Level filter + +What: /sys/bus/event_source/devices/dmar*/cpumask +Date: Jan 2023 +KernelVersion: 6.3 +Contact: Kan Liang +Description: Read-only. This file always returns the CPU to which the + IOMMU pmu is bound for access to all IOMMU pmu + performance monitoring events. diff --git a/drivers/iommu/intel/perfmon.c b/drivers/iommu/intel/perfmon.c index 43a5075eaecd..f332232bb345 100644 --- a/drivers/iommu/intel/perfmon.c +++ b/drivers/iommu/intel/perfmon.c @@ -32,9 +32,30 @@ static struct attribute_group iommu_pmu_events_attr_group = { .attrs = attrs_empty, }; +static cpumask_t iommu_pmu_cpu_mask; + +static ssize_t iommu_pmu_cpumask_show(struct device *dev, + struct device_attribute *attr, + char *buf) +{ + return cpumap_print_to_pagebuf(true, buf, &iommu_pmu_cpu_mask); +} + +static DEVICE_ATTR(cpumask, S_IRUGO, iommu_pmu_cpumask_show, NULL); + +static struct attribute *iommu_pmu_cpumask_attrs[] = { + &dev_attr_cpumask.attr, + NULL +}; + +static struct attribute_group iommu_pmu_cpumask_attr_group = { + .attrs = iommu_pmu_cpumask_attrs, +}; + static const struct attribute_group *iommu_pmu_attr_groups[] = { &iommu_pmu_format_attr_group, &iommu_pmu_events_attr_group, + &iommu_pmu_cpumask_attr_group, NULL }; @@ -637,19 +658,97 @@ void free_iommu_pmu(struct intel_iommu *iommu) iommu->pmu = NULL; } +static int iommu_pmu_cpu_online(unsigned int cpu) +{ + if (cpumask_empty(&iommu_pmu_cpu_mask)) + cpumask_set_cpu(cpu, &iommu_pmu_cpu_mask); + + return 0; +} + +static int iommu_pmu_cpu_offline(unsigned int cpu) +{ + struct dmar_drhd_unit *drhd; + struct intel_iommu *iommu; + int target; + + if (!cpumask_test_and_clear_cpu(cpu, &iommu_pmu_cpu_mask)) + return 0; + + target = cpumask_any_but(cpu_online_mask, cpu); + + if (target < nr_cpu_ids) + cpumask_set_cpu(target, &iommu_pmu_cpu_mask); + else + target = -1; + + rcu_read_lock(); + + for_each_iommu(iommu, drhd) { + if (!iommu->pmu) + continue; + perf_pmu_migrate_context(&iommu->pmu->pmu, cpu, target); + } + rcu_read_unlock(); + + return 0; +} + +static int nr_iommu_pmu; + +static int iommu_pmu_cpuhp_setup(struct iommu_pmu *iommu_pmu) +{ + int ret; + + if (nr_iommu_pmu++) + return 0; + + ret = cpuhp_setup_state(CPUHP_AP_PERF_X86_IOMMU_PERF_ONLINE, + "driver/iommu/intel/perfmon:online", + iommu_pmu_cpu_online, + iommu_pmu_cpu_offline); + if (ret) + nr_iommu_pmu = 0; + + return ret; +} + +static void iommu_pmu_cpuhp_free(struct iommu_pmu *iommu_pmu) +{ + if (--nr_iommu_pmu) + return; + + cpuhp_remove_state(CPUHP_AP_PERF_X86_IOMMU_PERF_ONLINE); +} + void iommu_pmu_register(struct intel_iommu *iommu) { - if (!iommu->pmu) + struct iommu_pmu *iommu_pmu = iommu->pmu; + + if (!iommu_pmu) return; - if (__iommu_pmu_register(iommu)) { - pr_err("Failed to register PMU for iommu (seq_id = %d)\n", - iommu->seq_id); - } + if (__iommu_pmu_register(iommu)) + goto err; + + if (iommu_pmu_cpuhp_setup(iommu_pmu)) + goto unregister; + + return; + +unregister: + perf_pmu_unregister(&iommu_pmu->pmu); +err: + pr_err("Failed to register PMU for iommu (seq_id = %d)\n", iommu->seq_id); } void iommu_pmu_unregister(struct intel_iommu *iommu) { - if (iommu->pmu) - perf_pmu_unregister(&iommu->pmu->pmu); + struct iommu_pmu *iommu_pmu = iommu->pmu; + + if (!iommu_pmu) + return; + + iommu_pmu_cpuhp_free(iommu_pmu); + perf_pmu_unregister(&iommu_pmu->pmu); } diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h index 6c6859bfc454..f2ea348ce3b0 100644 --- a/include/linux/cpuhotplug.h +++ b/include/linux/cpuhotplug.h @@ -221,6 +221,7 @@ enum cpuhp_state { CPUHP_AP_PERF_X86_CQM_ONLINE, CPUHP_AP_PERF_X86_CSTATE_ONLINE, CPUHP_AP_PERF_X86_IDXD_ONLINE, + CPUHP_AP_PERF_X86_IOMMU_PERF_ONLINE, CPUHP_AP_PERF_S390_CF_ONLINE, CPUHP_AP_PERF_S390_SF_ONLINE, CPUHP_AP_PERF_ARM_CCI_ONLINE, From patchwork Wed Jan 11 20:25:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Liang, Kan" X-Patchwork-Id: 42186 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp3524031wrt; Wed, 11 Jan 2023 12:33:23 -0800 (PST) X-Google-Smtp-Source: AMrXdXuv8PWe5rWqF+O8CVZEL1PDbbrK5sbpqlV6U7XCKQClnTK241af2AhVRw/+ewj+AR3/scUG X-Received: by 2002:a05:6a00:4149:b0:576:5a71:454a with SMTP id bv9-20020a056a00414900b005765a71454amr3908941pfb.2.1673469203377; Wed, 11 Jan 2023 12:33:23 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1673469203; cv=none; d=google.com; s=arc-20160816; b=cd8yuRF4cOsHbQ4E7eDBggy8EPOhKKLnq41jYmCzwu2ulQqxzlt+WrO3lm6yiWvaGS Tr68St+8QudFwLjrlvAIDHgdx7084J2e3A1fXFLjf3g+ROQxFIEw5nkpOVmxISMDyePO wPgegi1HLpx6OjFp9JghXv4cNhJBx6Yhkj9FO6CKOFuVGiP63NfN7jv9cr6cjBZEoolB IaZVz2KzVf3lyuGwiheWSa3Ic8tVlQaHaeWv9oVdauzOzOtmXb9Jq37bWnnB74y1n7P1 mvhvjYiHbLx8+N0bmt4knIpfr86V1tea6aLesLTp/EqvX3/w1fxeFr1H6jSgYHcoSBgD Liig== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=lhwvIjEtLUnKarTYmt6UmKVD+3o9ULso98IebHBc3bE=; b=oCLQj5CgxWKbIrwCcKc3fKC+7CJktGS8vFW1Pxr69LzgGIoSZMcscWlgDc1ljKJAYn OcP6y/3SFG+rlm5ELSTg/RKzoffNqgVV5dHRdWLUufni9Wf+PeK02WiJ6nlS5XC2xheZ LX97dTTLRVN1YzXLFcz7XdV1qNbSrL/bX3R9guXw4Bae95m/yLz+b6Fdiua0yFjpn9f0 ULcTSyAwWX9NDx9ln0bS1ze6p0p6kCvVX/z+meRYjMNce/CgS2+bum9i29Kxqk3PF5Ab Yc76+LX144QG321Qr2i6z8+T1Q0TvAqSoGcIrxDGpZLQPeZONYahsFjo93xsZb0gbDhH uRMw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=BbmVLzjg; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id n24-20020a62e518000000b0058ba0380419si313964pff.235.2023.01.11.12.33.10; Wed, 11 Jan 2023 12:33:23 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=BbmVLzjg; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236856AbjAKU0N (ORCPT + 99 others); Wed, 11 Jan 2023 15:26:13 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44468 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235916AbjAKUZe (ORCPT ); Wed, 11 Jan 2023 15:25:34 -0500 Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 179B3E0D1 for ; Wed, 11 Jan 2023 12:25:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1673468733; x=1705004733; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Znenp5W3nNsM5k6PZyd4pNB4aEJiEyv6Na4fQa12reU=; b=BbmVLzjgiXENNNBqLZVXdWVsVh5AcEGFFLST9BvDtIuMdV6+7hja3jUz zVH2SrL6yrH8Pg6K0EdxtfFnIKG8XNAElLvHNIBEpN18X+Wc/w1iGPj4c mR7DRI6T5A6nQwwQrGIdog8aFWcCDTqM4GAe1JW/qLSdUaodDlKix8SDZ ep7cS01Cy+LW2maosHEOPMFIN0kPzpkg5zjVlMtSymQBbeziI168hDUHq g9QotwUNnIHzLZPbdKQCXlgNiOmcSmR98VgDYATQpZNAPEChmdmNszAPD vjqtUHuNXsd3uHcOiW5REEAs4iqGYI/Uy7G9aM5/d1rYQ0NNP2hWyHmaS A==; X-IronPort-AV: E=McAfee;i="6500,9779,10586"; a="350755008" X-IronPort-AV: E=Sophos;i="5.96,317,1665471600"; d="scan'208";a="350755008" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Jan 2023 12:25:29 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10586"; a="781518077" X-IronPort-AV: E=Sophos;i="5.96,317,1665471600"; d="scan'208";a="781518077" Received: from kanliang-dev.jf.intel.com ([10.165.154.102]) by orsmga004.jf.intel.com with ESMTP; 11 Jan 2023 12:25:28 -0800 From: kan.liang@linux.intel.com To: joro@8bytes.org, will@kernel.org, baolu.lu@linux.intel.com, dwmw2@infradead.org, robin.murphy@arm.com, robert.moore@intel.com, rafael.j.wysocki@intel.com, lenb@kernel.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org Cc: Kan Liang Subject: [PATCH 6/7] iommu/vt-d: Add IOMMU perfmon overflow handler support Date: Wed, 11 Jan 2023 12:25:03 -0800 Message-Id: <20230111202504.378258-7-kan.liang@linux.intel.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20230111202504.378258-1-kan.liang@linux.intel.com> References: <20230111202504.378258-1-kan.liang@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1754759643353716411?= X-GMAIL-MSGID: =?utf-8?q?1754759643353716411?= From: Kan Liang While enabled to count events and an event occurrence causes the counter value to increment and roll over to or past zero, this is termed a counter overflow. The overflow can trigger an interrupt. The IOMMU perfmon needs to handle the case properly. New HW IRQs are allocated for each IOMMU device for perfmon. The IRQ IDs are after the SVM range. In the overflow handler, the counter is not frozen. It's very unlikely that the same counter overflows again during the period. But it's possible that other counters overflow at the same time. Read the overflow register at the end of the handler and check whether there are more. Signed-off-by: Kan Liang --- drivers/iommu/intel/dmar.c | 2 + drivers/iommu/intel/iommu.h | 11 ++++- drivers/iommu/intel/perfmon.c | 82 +++++++++++++++++++++++++++++++++++ drivers/iommu/intel/svm.c | 2 +- 4 files changed, 95 insertions(+), 2 deletions(-) diff --git a/drivers/iommu/intel/dmar.c b/drivers/iommu/intel/dmar.c index dcafa32875c1..94e314320cd9 100644 --- a/drivers/iommu/intel/dmar.c +++ b/drivers/iommu/intel/dmar.c @@ -1887,6 +1887,8 @@ static inline int dmar_msi_reg(struct intel_iommu *iommu, int irq) return DMAR_FECTL_REG; else if (iommu->pr_irq == irq) return DMAR_PECTL_REG; + else if (iommu->perf_irq == irq) + return DMAR_PERFINTRCTL_REG; else BUG(); } diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h index bbc5dda903e9..f616e4f3d765 100644 --- a/drivers/iommu/intel/iommu.h +++ b/drivers/iommu/intel/iommu.h @@ -130,6 +130,8 @@ #define DMAR_PERFCFGOFF_REG 0x310 #define DMAR_PERFOVFOFF_REG 0x318 #define DMAR_PERFCNTROFF_REG 0x31c +#define DMAR_PERFINTRSTS_REG 0x324 +#define DMAR_PERFINTRCTL_REG 0x328 #define DMAR_PERFEVNTCAP_REG 0x380 #define DMAR_ECMD_REG 0x400 #define DMAR_ECEO_REG 0x408 @@ -357,6 +359,9 @@ #define DMA_VCS_PAS ((u64)1) +/* PERFINTRSTS_REG */ +#define DMA_PERFINTRSTS_PIS ((u32)1) + #define IOMMU_WAIT_OP(iommu, offset, op, cond, sts) \ do { \ cycles_t start_time = get_cycles(); \ @@ -630,8 +635,12 @@ struct iommu_pmu { struct pmu pmu; DECLARE_BITMAP(used_mask, IOMMU_PMU_IDX_MAX); struct perf_event *event_list[IOMMU_PMU_IDX_MAX]; + unsigned char irq_name[16]; }; +#define IOMMU_IRQ_ID_OFFSET_PRQ (DMAR_UNITS_SUPPORTED) +#define IOMMU_IRQ_ID_OFFSET_PERF (2 * DMAR_UNITS_SUPPORTED) + struct intel_iommu { void __iomem *reg; /* Pointer to hardware regs, virtual addr */ u64 reg_phys; /* physical address of hw register set */ @@ -645,7 +654,7 @@ struct intel_iommu { int seq_id; /* sequence id of the iommu */ int agaw; /* agaw of this iommu */ int msagaw; /* max sagaw of this iommu */ - unsigned int irq, pr_irq; + unsigned int irq, pr_irq, perf_irq; u16 segment; /* PCI segment# */ unsigned char name[13]; /* Device Name */ diff --git a/drivers/iommu/intel/perfmon.c b/drivers/iommu/intel/perfmon.c index f332232bb345..d0fbf784c827 100644 --- a/drivers/iommu/intel/perfmon.c +++ b/drivers/iommu/intel/perfmon.c @@ -476,6 +476,49 @@ static void iommu_pmu_disable(struct pmu *pmu) ecmd_submit_sync(iommu, DMA_ECMD_FREEZE, 0, false, 0); } +static void iommu_pmu_counter_overflow(struct iommu_pmu *iommu_pmu) +{ + struct perf_event *event; + int i, handled = 0; + u64 status; + + /* + * Two counters may be overflowed very close. + * Always check whether there are more to handle. + */ + while ((status = dmar_readq(iommu_pmu->overflow))) { + for_each_set_bit(i, (unsigned long *)&status, iommu_pmu->num_cntr) { + handled++; + + /* + * Find the assigned event of the counter. + * Accumulate the value into the event->count. + */ + event = iommu_pmu->event_list[i]; + if (WARN_ON_ONCE(!event)) + continue; + iommu_pmu_event_update(event); + } + + dmar_writeq(iommu_pmu->overflow, status); + } +} + +static irqreturn_t iommu_pmu_irq_handler(int irq, void *dev_id) +{ + struct intel_iommu *iommu = dev_id; + + if (!dmar_readl(iommu->reg + DMAR_PERFINTRSTS_REG)) + return IRQ_NONE; + + iommu_pmu_counter_overflow(iommu->pmu); + + /* Clear the status bit */ + dmar_writel(iommu->reg + DMAR_PERFINTRSTS_REG, DMA_PERFINTRSTS_PIS); + + return IRQ_HANDLED; +} + static int __iommu_pmu_register(struct intel_iommu *iommu) { struct iommu_pmu *iommu_pmu = iommu->pmu; @@ -658,6 +701,38 @@ void free_iommu_pmu(struct intel_iommu *iommu) iommu->pmu = NULL; } +static int iommu_pmu_set_interrupt(struct intel_iommu *iommu) +{ + struct iommu_pmu *iommu_pmu = iommu->pmu; + int irq, ret; + + irq = dmar_alloc_hwirq(IOMMU_IRQ_ID_OFFSET_PERF + iommu->seq_id, iommu->node, iommu); + if (irq <= 0) + return -EINVAL; + + snprintf(iommu_pmu->irq_name, sizeof(iommu_pmu->irq_name), "dmar%d-perf", iommu->seq_id); + + iommu->perf_irq = irq; + ret = request_threaded_irq(irq, NULL, iommu_pmu_irq_handler, + IRQF_ONESHOT, iommu_pmu->irq_name, iommu); + if (ret) { + dmar_free_hwirq(irq); + iommu->perf_irq = 0; + return ret; + } + return 0; +} + +static void iommu_pmu_unset_interrupt(struct intel_iommu *iommu) +{ + if (!iommu->perf_irq) + return; + + free_irq(iommu->perf_irq, iommu); + dmar_free_hwirq(iommu->perf_irq); + iommu->perf_irq = 0; +} + static int iommu_pmu_cpu_online(unsigned int cpu) { if (cpumask_empty(&iommu_pmu_cpu_mask)) @@ -734,8 +809,14 @@ void iommu_pmu_register(struct intel_iommu *iommu) if (iommu_pmu_cpuhp_setup(iommu_pmu)) goto unregister; + /* Set interrupt for overflow */ + if (iommu_pmu_set_interrupt(iommu)) + goto cpuhp_free; + return; +cpuhp_free: + iommu_pmu_cpuhp_free(iommu_pmu); unregister: perf_pmu_unregister(&iommu_pmu->pmu); err: @@ -749,6 +830,7 @@ void iommu_pmu_unregister(struct intel_iommu *iommu) if (!iommu_pmu) return; + iommu_pmu_unset_interrupt(iommu); iommu_pmu_cpuhp_free(iommu_pmu); perf_pmu_unregister(&iommu_pmu->pmu); } diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c index c76b66263467..b6c5edd80d5d 100644 --- a/drivers/iommu/intel/svm.c +++ b/drivers/iommu/intel/svm.c @@ -79,7 +79,7 @@ int intel_svm_enable_prq(struct intel_iommu *iommu) } iommu->prq = page_address(pages); - irq = dmar_alloc_hwirq(DMAR_UNITS_SUPPORTED + iommu->seq_id, iommu->node, iommu); + irq = dmar_alloc_hwirq(IOMMU_IRQ_ID_OFFSET_PRQ + iommu->seq_id, iommu->node, iommu); if (irq <= 0) { pr_err("IOMMU: %s: Failed to create IRQ vector for page request queue\n", iommu->name); From patchwork Wed Jan 11 20:25:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Liang, Kan" X-Patchwork-Id: 42188 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp3524308wrt; Wed, 11 Jan 2023 12:34:10 -0800 (PST) X-Google-Smtp-Source: AMrXdXtQz5VyMmmzJSw3PttMd4DfJuT2BgGTEzoQ4/y0JcrTqfud9Ys/8CCVafuwKdAB14FJZG3z X-Received: by 2002:a05:6a20:2ca:b0:b6:4ca8:eb7d with SMTP id 10-20020a056a2002ca00b000b64ca8eb7dmr2935708pzb.50.1673469250234; Wed, 11 Jan 2023 12:34:10 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1673469250; cv=none; d=google.com; s=arc-20160816; b=wxVbGMJwA/78mFeQbonVhPzMBpLja443TqJnmtC68/duUPelGrIjwrqOTDpkEuhfJ3 I6Xo92b6OlbCgvV48ku5OCTe4LO3LAnSs+BxKifiWRSaBuyEAcEq9kSe/8ChQjSgqpod kPPwkpaYKc7/RlN/twrTtGiCX79xJ/ZsAKgPVPKo5YmQdkxFxFu4pgxJ+3rx8fBSxa+9 FlH7T5c4OZc/0fiXu0413GEi9nKJwSCu3bXe3NAf3bVCxBHY/LjonwiJSwdkdHNP4FkJ 2mUbW4swhG5Ua48pAXaFtaB7uwSQ33DBB5HQKJfcV30Kokje2wFJ4YnjEaEKI8uwJkX3 akkA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=fLF54QdZMiYKwIC5D1w0DyQ0LgPRur4tnYaIbPpMKM0=; b=J7Ui/oPXkxY5bgUDgum6y4g6TfJVXOOkGNmZYU95V375puXm24V+lTL8xswBOR2JX4 qqPUSBnTgjsP1T2ocUjDhS29dMCp2L3v9qUlKqri+S1wtQLcIVb19j6HZE77SdhweCHL R7Z5yqD2P49K2AI+2AitoEL3Eq3WhrBpAz30E7NQ85LJdAFQDWrvnQxbxEf1/JfpmAuj /S3FWPo4ZmB3WK7tmIm9/OMGphzj1WGPwnn1eYsiS0gb2GeGUQ+HJVdS54800OGVrV3E cpgwo8eQUiG4mX1P7I9fgdCaxM1z4eceVaR7eUOw4Q65gzjZ6gvWh3MEFFV3KZeXWNfQ Hb2A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=cJGBnAOL; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id i64-20020a638743000000b004b5b1c44f6csi2801757pge.183.2023.01.11.12.33.56; Wed, 11 Jan 2023 12:34:10 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=cJGBnAOL; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236169AbjAKU01 (ORCPT + 99 others); Wed, 11 Jan 2023 15:26:27 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44474 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235935AbjAKUZf (ORCPT ); Wed, 11 Jan 2023 15:25:35 -0500 Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A4A001260C for ; Wed, 11 Jan 2023 12:25:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1673468734; x=1705004734; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=PbKbzo0ebhNuat4Mu85n5Nyk3Ge5hASt79m3RxYgmac=; b=cJGBnAOLXcfKCnhZVlBMpX1MyvB71oCzPIBYYEtMgcGsv3LRaUIphrmr 4Haxm7WSZPqa7fkuLyzmkGZ6agVNb0p2M04NJspw19ISjxuXncSZL6iS9 Um/5HSrswZW5UUca0oMdp/QkX0CB+XIiWhS+UVqhoSdpA3KNimdOfD3sp TEkfUJ9fhE7iKjRPUz2h7Tc5F9zSerTczhgsgZpur4lM+O8x2QfuyYlHS x3cJPq8nRyo4D6jmsJsdarI1zXGoqx+b8PzXBnN8BQYccivywptou5OHw k/4CUNXgplCdwt2N/oIfbhrpz8tMYOs5dF52GhKRkiapseU/wAAdNucO7 g==; X-IronPort-AV: E=McAfee;i="6500,9779,10586"; a="350755011" X-IronPort-AV: E=Sophos;i="5.96,317,1665471600"; d="scan'208";a="350755011" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Jan 2023 12:25:29 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10586"; a="781518080" X-IronPort-AV: E=Sophos;i="5.96,317,1665471600"; d="scan'208";a="781518080" Received: from kanliang-dev.jf.intel.com ([10.165.154.102]) by orsmga004.jf.intel.com with ESMTP; 11 Jan 2023 12:25:28 -0800 From: kan.liang@linux.intel.com To: joro@8bytes.org, will@kernel.org, baolu.lu@linux.intel.com, dwmw2@infradead.org, robin.murphy@arm.com, robert.moore@intel.com, rafael.j.wysocki@intel.com, lenb@kernel.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org Cc: Kan Liang Subject: [PATCH 7/7] iommu/vt-d: Enable IOMMU perfmon support Date: Wed, 11 Jan 2023 12:25:04 -0800 Message-Id: <20230111202504.378258-8-kan.liang@linux.intel.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20230111202504.378258-1-kan.liang@linux.intel.com> References: <20230111202504.378258-1-kan.liang@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1754759692568248409?= X-GMAIL-MSGID: =?utf-8?q?1754759692568248409?= From: Kan Liang Register and enable an IOMMU perfmon for each active IOMMU device. The failure of IOMMU perfmon registration doesn't impact other functionalities of an IOMMU device. Signed-off-by: Kan Liang --- drivers/iommu/intel/dmar.c | 3 +++ drivers/iommu/intel/iommu.c | 3 +++ 2 files changed, 6 insertions(+) diff --git a/drivers/iommu/intel/dmar.c b/drivers/iommu/intel/dmar.c index 94e314320cd9..5a33df9e19ce 100644 --- a/drivers/iommu/intel/dmar.c +++ b/drivers/iommu/intel/dmar.c @@ -1152,6 +1152,8 @@ static int alloc_iommu(struct dmar_drhd_unit *drhd) err = iommu_device_register(&iommu->iommu, &intel_iommu_ops, NULL); if (err) goto err_sysfs; + + iommu_pmu_register(iommu); } drhd->iommu = iommu; @@ -1174,6 +1176,7 @@ static int alloc_iommu(struct dmar_drhd_unit *drhd) static void free_iommu(struct intel_iommu *iommu) { if (intel_iommu_enabled && !iommu->drhd->ignored) { + iommu_pmu_unregister(iommu); iommu_device_unregister(&iommu->iommu); iommu_device_sysfs_remove(&iommu->iommu); } diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 59df7e42fd53..c57e60c5f353 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -30,6 +30,7 @@ #include "../iommu-sva.h" #include "pasid.h" #include "cap_audit.h" +#include "perfmon.h" #define ROOT_SIZE VTD_PAGE_SIZE #define CONTEXT_SIZE VTD_PAGE_SIZE @@ -4013,6 +4014,8 @@ int __init intel_iommu_init(void) intel_iommu_groups, "%s", iommu->name); iommu_device_register(&iommu->iommu, &intel_iommu_ops, NULL); + + iommu_pmu_register(iommu); } up_read(&dmar_global_lock);