From patchwork Mon Mar 13 17:02:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fenghua Yu X-Patchwork-Id: 68955 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp1305580wrd; Mon, 13 Mar 2023 10:14:18 -0700 (PDT) X-Google-Smtp-Source: AK7set8CfZ2QYU8m75MKTlPv0jJzXCR+QBxGYY6IOqjcCSdhQTmSektdcEUF1IT7PX0JUZWoZyGv X-Received: by 2002:a05:6a00:1819:b0:5a8:ac15:534d with SMTP id y25-20020a056a00181900b005a8ac15534dmr13302237pfa.13.1678727658024; Mon, 13 Mar 2023 10:14:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1678727658; cv=none; d=google.com; s=arc-20160816; b=BZetQfCWxV0uK3mgitQsfqcBIlJcyuuT9091BTwOiCnPvd+UCaGdx+HpiM5F8UBjKz 8B5LVcaaeYUDzxxw+CNtm75ozlfwGvNbrFMKj00O/IBmlrYivMdd3Acs8mZdQc+KsGdE ZW241jl7U5kLhXB+UWSrquxbCDzIp7nNiCZLxcFE02yiHqICP5DGYK/6Af1+eZiUlits tlvmr+iJVHaQFopTh0hoPVLiQa65GZsoDZzvvvWSjvO5sO+2y24YL5snLmjTjAe6weTO j+Y6X09y75D88tlOhlNvzgC0PeuUqRu/AqdqZZbgBEKKtgo0IsHtZzHgonVwkOiPpwhC X6gA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=uANeK8TMsD3aD8EjOWoTcIVCT30YBQcbC8b7XWfMhME=; b=BVhmyBZRBS18oNsY2YnftFZfudI7NVLikROORbuPwqKoA7jXyJ3sJRRtWx+B3QA0bi 8isESBYYgb7U/507v7JWu6dcJhqZyl9a83rhkBen582wi5Joel7N1QB8tOeqkyDdGTZg 3nT2qUdPeFb7D70lbfblsHSS55ngwVWMPe9j7VSqhYMUocc2DYsQubcPWIDEuV5TB92E 4vAi1Ybd17f5LTY3aRxSw1uZj8kCbvIe57PWfKzHyKGdM5qLdJEbJOpJ2AYN7fWKvKiS fhLFNKYGEWj6U+2XehYNF87ET7E99gnI1PNGz6ecUV74dfoh5VRhva/rACb4TPFE/EW2 9nCQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=VKvuY0pb; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id p23-20020a63e657000000b00502f457059dsi6966048pgj.330.2023.03.13.10.14.05; Mon, 13 Mar 2023 10:14:18 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=VKvuY0pb; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230230AbjCMRF1 (ORCPT + 99 others); Mon, 13 Mar 2023 13:05:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41142 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230327AbjCMRE4 (ORCPT ); Mon, 13 Mar 2023 13:04:56 -0400 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3F57742BE7; Mon, 13 Mar 2023 10:04:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1678727062; x=1710263062; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=MwbXdVdXHEET8kRfN+0wwJVjHIBrLAgznNDkutHFUxs=; b=VKvuY0pbFVER+9UfE9QEz9oh6kH2ZMzS5JwQV9bKyZGBogg9QlrmK9EB fIhM/2GTr+L3DXIoyXh/YT1rMM5DRseTPX+MWVM80FTEV4N+vsQtIf0PA zPN1vqPYnUsdM3UHafdHPToHPTeoWDnrQkU6NJ8GVJgVxh4X3IUnLLpBy hsS5ajYdhfSIq+fWKvfxutTNUAbnM+YznYTGAHPtYxzQf/675dHZLg6t+ yh1LDBZC52sGKSL9ds8KkI/p5S+avJBKT5xPrUYiWx21U2wfkT+zk0yP/ fB/EPjIO2c5FcFOqyUbDanx7CiJaclNCfuhneuO+cXmq548TzTLDGbQa1 Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10648"; a="334679615" X-IronPort-AV: E=Sophos;i="5.98,257,1673942400"; d="scan'208";a="334679615" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Mar 2023 10:02:40 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10648"; a="708950888" X-IronPort-AV: E=Sophos;i="5.98,257,1673942400"; d="scan'208";a="708950888" Received: from fyu1.sc.intel.com ([172.25.103.126]) by orsmga008.jf.intel.com with ESMTP; 13 Mar 2023 10:02:39 -0700 From: Fenghua Yu To: "Vinod Koul" , "Dave Jiang" Cc: dmaengine@vger.kernel.org, "linux-kernel" , Sanjay Kumar , Tony Zhu , Fenghua Yu Subject: [PATCH v3 01/16] dmaengine: idxd: make misc interrupt one shot Date: Mon, 13 Mar 2023 10:02:04 -0700 Message-Id: <20230313170219.1956012-2-fenghua.yu@intel.com> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20230313170219.1956012-1-fenghua.yu@intel.com> References: <20230313170219.1956012-1-fenghua.yu@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1760273532805001597?= X-GMAIL-MSGID: =?utf-8?q?1760273532805001597?= From: Dave Jiang Current code continuously processes the interrupt as long as the hardware is setting the status bit. There's no reason to do that since the threaded handler will get called again if another interrupt is asserted. Also through testing, it has shown that if a misprogrammed (or malicious) agent can continuously submit descriptors with bad completion record and causes errors to be reported via the misc interrupt. Continuous processing by the thread can cause software hang watchdog to kick off since the thread isn't giving up the CPU. Reported-by: Sanjay Kumar Tested-by: Tony Zhu Signed-off-by: Dave Jiang Co-developed-by: Fenghua Yu Signed-off-by: Fenghua Yu --- drivers/dma/idxd/irq.c | 38 ++++++++++++-------------------------- 1 file changed, 12 insertions(+), 26 deletions(-) diff --git a/drivers/dma/idxd/irq.c b/drivers/dma/idxd/irq.c index aa314ebec587..0d639303b515 100644 --- a/drivers/dma/idxd/irq.c +++ b/drivers/dma/idxd/irq.c @@ -217,13 +217,22 @@ static void idxd_int_handle_revoke(struct work_struct *work) kfree(revoke); } -static int process_misc_interrupts(struct idxd_device *idxd, u32 cause) +irqreturn_t idxd_misc_thread(int vec, void *data) { + struct idxd_irq_entry *irq_entry = data; + struct idxd_device *idxd = ie_to_idxd(irq_entry); struct device *dev = &idxd->pdev->dev; union gensts_reg gensts; u32 val = 0; int i; bool err = false; + u32 cause; + + cause = ioread32(idxd->reg_base + IDXD_INTCAUSE_OFFSET); + if (!cause) + return IRQ_NONE; + + iowrite32(cause, idxd->reg_base + IDXD_INTCAUSE_OFFSET); if (cause & IDXD_INTC_HALT_STATE) goto halt; @@ -301,7 +310,7 @@ static int process_misc_interrupts(struct idxd_device *idxd, u32 cause) val); if (!err) - return 0; + goto out; halt: gensts.bits = ioread32(idxd->reg_base + IDXD_GENSTATS_OFFSET); @@ -324,33 +333,10 @@ static int process_misc_interrupts(struct idxd_device *idxd, u32 cause) "idxd halted, need %s.\n", gensts.reset_type == IDXD_DEVICE_RESET_FLR ? "FLR" : "system reset"); - return -ENXIO; } } - return 0; -} - -irqreturn_t idxd_misc_thread(int vec, void *data) -{ - struct idxd_irq_entry *irq_entry = data; - struct idxd_device *idxd = ie_to_idxd(irq_entry); - int rc; - u32 cause; - - cause = ioread32(idxd->reg_base + IDXD_INTCAUSE_OFFSET); - if (cause) - iowrite32(cause, idxd->reg_base + IDXD_INTCAUSE_OFFSET); - - while (cause) { - rc = process_misc_interrupts(idxd, cause); - if (rc < 0) - break; - cause = ioread32(idxd->reg_base + IDXD_INTCAUSE_OFFSET); - if (cause) - iowrite32(cause, idxd->reg_base + IDXD_INTCAUSE_OFFSET); - } - +out: return IRQ_HANDLED; } From patchwork Mon Mar 13 17:02:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fenghua Yu X-Patchwork-Id: 68980 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp1310523wrd; Mon, 13 Mar 2023 10:24:50 -0700 (PDT) X-Google-Smtp-Source: AK7set96FBTG1GJs3GBTsjQHWK89IN932lOKCgVzfTdYSh+XRSJcXvF8mnfREkO8Y6NplbSWtl9C X-Received: by 2002:a17:90a:2c2:b0:233:e305:f617 with SMTP id d2-20020a17090a02c200b00233e305f617mr36452146pjd.32.1678728289848; Mon, 13 Mar 2023 10:24:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1678728289; cv=none; d=google.com; s=arc-20160816; b=FtUe6uuArRHHcJ11rfz8gel/M0rV7NfRNapMiN/5K/gBix15qHT9drNQd0YqcXxmDo pqSXiZJCQWZ/EivXMQ17ZLs+vrZqNUZ44WPBsgnpbzheVmp+93oB7IUPBdjPJIZmS+Yt oInVhex0w/AGamkGlffcw+Z9JApVOeBICH6oHpYrFKS8+gMcqHJePNv2LnGUuNwyJTWr EhOx5Nx5XreH8jHsfu9s16PjGu1feTfdI6u6VVgD8VMSdq/YrCJgBiuEEGL8DrBRo68k oWPTywlihXj87oqFC0py1sZWpvgQ21NDycwiN5sRqDwN9As/q/kebaK4RDAx1Gh0I1J+ Hp3w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=1179QC9SBVbCFTTBX2rTUWq2Qylz0xnBM637w5DLG1U=; b=iQM3ErZ36i058olT590kiPkni1PUX5qVK5bvQmhzflxCCa2NM5TjeSeRHiDnVUPJdN LrTjCKo0n8JNR/6XcEuKxOJVYp7jeJiKStxbosYsVgs44SE5bPIKI8gJAknkio9b/wY6 4ovEv/+Do/1bR3Er/Lp73/PLabS1JN36Li5qa2XcxKzlsN8F/MHSjPu+SbPwj1Oe+mSK Do3z3ZaBAttY14t7HBQSWfH+RhysUqmGC4jjIX0EBzWCU3Vo9G8Z+DKHbSelGVxG8pVq Q05eT37VmXBpO4EawqT9Zmvh/fAxEYYpJGDTKpfWH6VOAKBrdk+qB3lKWwqyoXNXaPgc ueLQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=W0BrlNcl; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id ot17-20020a17090b3b5100b002299b06dca9si250053pjb.83.2023.03.13.10.24.34; Mon, 13 Mar 2023 10:24:49 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=W0BrlNcl; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230272AbjCMRFb (ORCPT + 99 others); Mon, 13 Mar 2023 13:05:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41262 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230395AbjCMRE5 (ORCPT ); Mon, 13 Mar 2023 13:04:57 -0400 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 52AA0211E4; Mon, 13 Mar 2023 10:04:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1678727065; x=1710263065; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=TQA41H+RPAVqxuNbQ0/QsFDmoozKBiZjnSHPBC3zUcc=; b=W0BrlNclq1h/WGSF9qCXRdB5CaVUD3QbpseFd2WcsSShOheapm0QDdR9 M5sq2bdu08nvR9zNLrZ8vBmLWmxEgj7tRzANwZjcqCHgntp5sy7xjMtpi 4g53FEjAWyUleVFddjyekY1KcKveKOVO8BXgV6Z5Svd8bkvP8iWOhfFmt rdc7DzNBcaHlBvxV0IlrdZz7miJ80LHLHcSDhaMhZJmYX/ufH8+JRUu/+ HyiMM7D91IwUa0YLX/z1zmvcnplWeaDXSkUHbxKU0KVPyDasdoLMD+o1F oXzCb3wWOO2/hA/g6MQ0+Tfc2VWgLv+pEVz/voFoTgsKrfp4HkZEOzmZc w==; X-IronPort-AV: E=McAfee;i="6500,9779,10648"; a="334679621" X-IronPort-AV: E=Sophos;i="5.98,257,1673942400"; d="scan'208";a="334679621" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Mar 2023 10:02:40 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10648"; a="708950892" X-IronPort-AV: E=Sophos;i="5.98,257,1673942400"; d="scan'208";a="708950892" Received: from fyu1.sc.intel.com ([172.25.103.126]) by orsmga008.jf.intel.com with ESMTP; 13 Mar 2023 10:02:40 -0700 From: Fenghua Yu To: "Vinod Koul" , "Dave Jiang" Cc: dmaengine@vger.kernel.org, "linux-kernel" , Tony Zhu , Fenghua Yu Subject: [PATCH v3 02/16] dmaengine: idxd: add event log size sysfs attribute Date: Mon, 13 Mar 2023 10:02:05 -0700 Message-Id: <20230313170219.1956012-3-fenghua.yu@intel.com> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20230313170219.1956012-1-fenghua.yu@intel.com> References: <20230313170219.1956012-1-fenghua.yu@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1760274195295724911?= X-GMAIL-MSGID: =?utf-8?q?1760274195295724911?= From: Dave Jiang Add support for changing of the event log size. Event log is a feature added to DSA 2.0 hardware to improve error reporting. It supersedes the SWERROR register on DSA 1.0 hardware and hope to prevent loss of reported errors. The error log size determines how many error entries supported for the device. It can be configured by the user via sysfs attribute. Tested-by: Tony Zhu Signed-off-by: Dave Jiang Co-developed-by: Fenghua Yu Signed-off-by: Fenghua Yu --- .../ABI/stable/sysfs-driver-dma-idxd | 8 +++ drivers/dma/idxd/idxd.h | 5 ++ drivers/dma/idxd/init.c | 23 ++++++++ drivers/dma/idxd/registers.h | 7 ++- drivers/dma/idxd/sysfs.c | 52 +++++++++++++++++++ 5 files changed, 94 insertions(+), 1 deletion(-) diff --git a/Documentation/ABI/stable/sysfs-driver-dma-idxd b/Documentation/ABI/stable/sysfs-driver-dma-idxd index d5e3dd3d8434..e01916611452 100644 --- a/Documentation/ABI/stable/sysfs-driver-dma-idxd +++ b/Documentation/ABI/stable/sysfs-driver-dma-idxd @@ -144,6 +144,14 @@ Description: IAA (IAX) capability mask. Exported to user space for application consumption. This attribute should only be visible on IAA devices that are version 2 or later. +What: /sys/bus/dsa/devices/dsa/event_log_size +Date: Sept 14, 2022 +KernelVersion: 6.4.0 +Contact: dmaengine@vger.kernel.org +Description: The event log size to be configured. Default is 64 entries and + occupies 4k size if the evl entry is 64 bytes. It's visible + only on platforms that support the capability. + What: /sys/bus/dsa/devices/wq./block_on_fault Date: Oct 27, 2020 KernelVersion: 5.11.0 diff --git a/drivers/dma/idxd/idxd.h b/drivers/dma/idxd/idxd.h index eca2c9d76db6..2a71273f1822 100644 --- a/drivers/dma/idxd/idxd.h +++ b/drivers/dma/idxd/idxd.h @@ -261,6 +261,10 @@ struct idxd_driver_data { int align; }; +struct idxd_evl { + u16 size; +}; + struct idxd_device { struct idxd_dev idxd_dev; struct idxd_driver_data *data; @@ -317,6 +321,7 @@ struct idxd_device { struct idxd_pmu *idxd_pmu; unsigned long *opcap_bmap; + struct idxd_evl *evl; }; /* IDXD software descriptor */ diff --git a/drivers/dma/idxd/init.c b/drivers/dma/idxd/init.c index a408fc91144d..d1fb01c115d8 100644 --- a/drivers/dma/idxd/init.c +++ b/drivers/dma/idxd/init.c @@ -332,6 +332,23 @@ static void idxd_cleanup_internals(struct idxd_device *idxd) destroy_workqueue(idxd->wq); } +static int idxd_init_evl(struct idxd_device *idxd) +{ + struct device *dev = &idxd->pdev->dev; + struct idxd_evl *evl; + + if (idxd->hw.gen_cap.evl_support == 0) + return 0; + + evl = kzalloc_node(sizeof(*evl), GFP_KERNEL, dev_to_node(dev)); + if (!evl) + return -ENOMEM; + + evl->size = IDXD_EVL_SIZE_MIN; + idxd->evl = evl; + return 0; +} + static int idxd_setup_internals(struct idxd_device *idxd) { struct device *dev = &idxd->pdev->dev; @@ -357,8 +374,14 @@ static int idxd_setup_internals(struct idxd_device *idxd) goto err_wkq_create; } + rc = idxd_init_evl(idxd); + if (rc < 0) + goto err_evl; + return 0; + err_evl: + destroy_workqueue(idxd->wq); err_wkq_create: for (i = 0; i < idxd->max_groups; i++) put_device(group_confdev(idxd->groups[i])); diff --git a/drivers/dma/idxd/registers.h b/drivers/dma/idxd/registers.h index 338289a66f00..ea3a499a3c3c 100644 --- a/drivers/dma/idxd/registers.h +++ b/drivers/dma/idxd/registers.h @@ -31,7 +31,9 @@ union gen_cap_reg { u64 rsvd:3; u64 dest_readback:1; u64 drain_readback:1; - u64 rsvd2:6; + u64 rsvd2:3; + u64 evl_support:2; + u64 rsvd4:1; u64 max_xfer_shift:5; u64 max_batch_shift:4; u64 max_ims_mult:6; @@ -297,6 +299,9 @@ union iaa_cap_reg { #define IDXD_IAACAP_OFFSET 0x180 +#define IDXD_EVL_SIZE_MIN 0x0040 +#define IDXD_EVL_SIZE_MAX 0xffff + union msix_perm { struct { u32 rsvd:2; diff --git a/drivers/dma/idxd/sysfs.c b/drivers/dma/idxd/sysfs.c index 2eba8cab25a1..85644e5bde83 100644 --- a/drivers/dma/idxd/sysfs.c +++ b/drivers/dma/idxd/sysfs.c @@ -1573,6 +1573,46 @@ static ssize_t iaa_cap_show(struct device *dev, } static DEVICE_ATTR_RO(iaa_cap); +static ssize_t event_log_size_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + struct idxd_device *idxd = confdev_to_idxd(dev); + + if (!idxd->evl) + return -EOPNOTSUPP; + + return sysfs_emit(buf, "%u\n", idxd->evl->size); +} + +static ssize_t event_log_size_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t count) +{ + struct idxd_device *idxd = confdev_to_idxd(dev); + unsigned long val; + int rc; + + if (!idxd->evl) + return -EOPNOTSUPP; + + rc = kstrtoul(buf, 10, &val); + if (rc < 0) + return -EINVAL; + + if (idxd->state == IDXD_DEV_ENABLED) + return -EPERM; + + if (!test_bit(IDXD_FLAG_CONFIGURABLE, &idxd->flags)) + return -EPERM; + + if (val < IDXD_EVL_SIZE_MIN || val > IDXD_EVL_SIZE_MAX) + return -EINVAL; + + idxd->evl->size = val; + return count; +} +static DEVICE_ATTR_RW(event_log_size); + static bool idxd_device_attr_max_batch_size_invisible(struct attribute *attr, struct idxd_device *idxd) { @@ -1603,6 +1643,13 @@ static bool idxd_device_attr_iaa_cap_invisible(struct attribute *attr, idxd->hw.version < DEVICE_VERSION_2); } +static bool idxd_device_attr_event_log_size_invisible(struct attribute *attr, + struct idxd_device *idxd) +{ + return (attr == &dev_attr_event_log_size.attr && + !idxd->hw.gen_cap.evl_support); +} + static umode_t idxd_device_attr_visible(struct kobject *kobj, struct attribute *attr, int n) { @@ -1618,6 +1665,9 @@ static umode_t idxd_device_attr_visible(struct kobject *kobj, if (idxd_device_attr_iaa_cap_invisible(attr, idxd)) return 0; + if (idxd_device_attr_event_log_size_invisible(attr, idxd)) + return 0; + return attr->mode; } @@ -1644,6 +1694,7 @@ static struct attribute *idxd_device_attributes[] = { &dev_attr_cdev_major.attr, &dev_attr_cmd_status.attr, &dev_attr_iaa_cap.attr, + &dev_attr_event_log_size.attr, NULL, }; @@ -1665,6 +1716,7 @@ static void idxd_conf_device_release(struct device *dev) bitmap_free(idxd->wq_enable_map); kfree(idxd->wqs); kfree(idxd->engines); + kfree(idxd->evl); ida_free(&idxd_ida, idxd->id); bitmap_free(idxd->opcap_bmap); kfree(idxd); From patchwork Mon Mar 13 17:02:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fenghua Yu X-Patchwork-Id: 68969 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp1308039wrd; Mon, 13 Mar 2023 10:19:08 -0700 (PDT) X-Google-Smtp-Source: AK7set8uyBMuj168jwxrLxyHUFAyB85XFkLaLFt6l8LZJxw8kP9AznoOTJssXT4tMvz2OyjDpB/Y X-Received: by 2002:a05:6a20:12cd:b0:cd:40b4:bc75 with SMTP id v13-20020a056a2012cd00b000cd40b4bc75mr38656335pzg.48.1678727948300; Mon, 13 Mar 2023 10:19:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1678727948; cv=none; d=google.com; s=arc-20160816; b=QC1JpjuHtDWAhrERLEsKOK5txjQA9n9JFxZ3YCL7nXCrFrkWouX9BQJxB/N2AcFcE1 g49h0qd50uYudjvIth9Au/6dNp/8JyrhynuV7+V2YyZS8/3bvquwv4JOTNvHYmtSl8Ru ZLe2XhrKG6iNJyU3+WK2aiwiYDg/traEc0Drp6HD2mVUa3aJ7+R8VSKvHh7VMjorbT2X 1MUjgrFjCmNT91mwrbfyk4177fsigkGbP4ZPdBiQurbVgVrRMLKHRZfDSXz2Q1EupZxn Sz0TWlZAdrcqE6im53+dCE7ibQyDLLNVo6pSL+j2gTkXFGTjWUTOxRaVRKqOJMKnQLu2 tFIw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=QPbswPM1rkZFG3rXV67QYWBdJnaYtovzqhBz/PFaks8=; b=YP4muE965ZCYSGVkEdN+++n7YpJh0VBLLm5TfCcX5ff2oCtgLi9JmiIbAJfXEQUIgW lZGZHjjR9/ZCuagZafeG4MAB0TeMBc0tC1rUHh/EdN1nKFMLC8Gu2W/rw75eq3bvDnCJ /AjS0NEK8JH1DKDvE/0kPtutEJKWon6wRjvI5PpJLJBFv5X73/nogWI0LfHoNYSuBvgH P3x74GbeEb1LyTMDvrtYW9Fj13fxvDpJuzl31QryvooM4IvuI0ll+PR5A36bhhxFscAW POR8+zJmwFXB2iXLEi5z2OWXiYXC/cfMcyD8/lxKIUy5EwnttQdoCoJz6iuopiBqlKkX L1Tg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=arlMCllv; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id b7-20020a656687000000b00508ef6ce863si5758261pgw.581.2023.03.13.10.18.55; Mon, 13 Mar 2023 10:19:08 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=arlMCllv; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230141AbjCMRFg (ORCPT + 99 others); Mon, 13 Mar 2023 13:05:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41526 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230466AbjCMRFA (ORCPT ); Mon, 13 Mar 2023 13:05:00 -0400 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 21A593755B; Mon, 13 Mar 2023 10:04:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1678727068; x=1710263068; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=vyRGDapMyyLqsA7crlN6WhQHps1C7Gco3rKRxAu1AFQ=; b=arlMCllv2O/qVCJZkOgbb7P/vocefZJYS73iFVSWaPmaAwTHUbYDRlo2 nJJarmEz8Gp9U4hHtDTurq6FbNgsfA+ojtb668dLCYvWvTW8Sv26OF4Lg DEaeLq6dyT7yxEP0ypHqtRvx+/Nyu7C1oqxDho+HUnEbTEUdDyNZ1OSk6 EygZNRflXCsIHXsQSfEYhaIaVi6GNdrAohpaxhtB0/j3tCh0b9GflZZhI Ts3QCd7H2i3TV48dwxaKBFDhBtCi/HlahuaRgJM/BKchol2qReNNPdYgj r6mT8ZonNwLqpW2Y3qYALjDPG63veDyuSL9V7yx1lvsvsupS50diamg17 g==; X-IronPort-AV: E=McAfee;i="6500,9779,10648"; a="334679628" X-IronPort-AV: E=Sophos;i="5.98,257,1673942400"; d="scan'208";a="334679628" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Mar 2023 10:02:40 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10648"; a="708950898" X-IronPort-AV: E=Sophos;i="5.98,257,1673942400"; d="scan'208";a="708950898" Received: from fyu1.sc.intel.com ([172.25.103.126]) by orsmga008.jf.intel.com with ESMTP; 13 Mar 2023 10:02:40 -0700 From: Fenghua Yu To: "Vinod Koul" , "Dave Jiang" Cc: dmaengine@vger.kernel.org, "linux-kernel" , Tony Zhu , Fenghua Yu Subject: [PATCH v3 03/16] dmaengine: idxd: setup event log configuration Date: Mon, 13 Mar 2023 10:02:06 -0700 Message-Id: <20230313170219.1956012-4-fenghua.yu@intel.com> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20230313170219.1956012-1-fenghua.yu@intel.com> References: <20230313170219.1956012-1-fenghua.yu@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1760273836818590265?= X-GMAIL-MSGID: =?utf-8?q?1760273836818590265?= From: Dave Jiang Add setup of event log feature for supported device. Event log addresses error reporting that was lacking in gen 1 DSA devices where a second error event does not get reported when a first event is pending software handling. The event log allows a circular buffer that the device can push error events to. It is up to the user to create a large enough event log ring in order to capture the expected events. The evl size can be set in the device sysfs attribute. By default 64 entries are supported as minimal when event log is enabled. Tested-by: Tony Zhu Signed-off-by: Dave Jiang Co-developed-by: Fenghua Yu Signed-off-by: Fenghua Yu --- drivers/dma/idxd/device.c | 89 +++++++++++++++++++++++++++++++++++- drivers/dma/idxd/idxd.h | 19 ++++++++ drivers/dma/idxd/init.c | 1 + drivers/dma/idxd/registers.h | 72 ++++++++++++++++++++++++++++- drivers/dma/idxd/sysfs.c | 3 +- include/uapi/linux/idxd.h | 1 + 6 files changed, 181 insertions(+), 4 deletions(-) diff --git a/drivers/dma/idxd/device.c b/drivers/dma/idxd/device.c index 5f321f3b4242..230fe9bb56ae 100644 --- a/drivers/dma/idxd/device.c +++ b/drivers/dma/idxd/device.c @@ -752,6 +752,83 @@ void idxd_device_clear_state(struct idxd_device *idxd) spin_unlock(&idxd->dev_lock); } +static int idxd_device_evl_setup(struct idxd_device *idxd) +{ + union gencfg_reg gencfg; + union evlcfg_reg evlcfg; + union genctrl_reg genctrl; + struct device *dev = &idxd->pdev->dev; + void *addr; + dma_addr_t dma_addr; + int size; + struct idxd_evl *evl = idxd->evl; + + if (!evl) + return 0; + + size = evl_size(idxd); + /* + * Address needs to be page aligned. However, dma_alloc_coherent() provides + * at minimal page size aligned address. No manual alignment required. + */ + addr = dma_alloc_coherent(dev, size, &dma_addr, GFP_KERNEL); + if (!addr) + return -ENOMEM; + + memset(addr, 0, size); + + spin_lock(&evl->lock); + evl->log = addr; + evl->dma = dma_addr; + evl->log_size = size; + + memset(&evlcfg, 0, sizeof(evlcfg)); + evlcfg.bits[0] = dma_addr & GENMASK(63, 12); + evlcfg.size = evl->size; + + iowrite64(evlcfg.bits[0], idxd->reg_base + IDXD_EVLCFG_OFFSET); + iowrite64(evlcfg.bits[1], idxd->reg_base + IDXD_EVLCFG_OFFSET + 8); + + genctrl.bits = ioread32(idxd->reg_base + IDXD_GENCTRL_OFFSET); + genctrl.evl_int_en = 1; + iowrite32(genctrl.bits, idxd->reg_base + IDXD_GENCTRL_OFFSET); + + gencfg.bits = ioread32(idxd->reg_base + IDXD_GENCFG_OFFSET); + gencfg.evl_en = 1; + iowrite32(gencfg.bits, idxd->reg_base + IDXD_GENCFG_OFFSET); + + spin_unlock(&evl->lock); + return 0; +} + +static void idxd_device_evl_free(struct idxd_device *idxd) +{ + union gencfg_reg gencfg; + union genctrl_reg genctrl; + struct device *dev = &idxd->pdev->dev; + struct idxd_evl *evl = idxd->evl; + + gencfg.bits = ioread32(idxd->reg_base + IDXD_GENCFG_OFFSET); + if (!gencfg.evl_en) + return; + + spin_lock(&evl->lock); + gencfg.evl_en = 0; + iowrite32(gencfg.bits, idxd->reg_base + IDXD_GENCFG_OFFSET); + + genctrl.bits = ioread32(idxd->reg_base + IDXD_GENCTRL_OFFSET); + genctrl.evl_int_en = 0; + iowrite32(genctrl.bits, idxd->reg_base + IDXD_GENCTRL_OFFSET); + + iowrite64(0, idxd->reg_base + IDXD_EVLCFG_OFFSET); + iowrite64(0, idxd->reg_base + IDXD_EVLCFG_OFFSET + 8); + + dma_free_coherent(dev, evl->log_size, evl->log, evl->dma); + evl->log = NULL; + evl->size = IDXD_EVL_SIZE_MIN; + spin_unlock(&evl->lock); +} + static void idxd_group_config_write(struct idxd_group *group) { struct idxd_device *idxd = group->idxd; @@ -1451,15 +1528,24 @@ int idxd_device_drv_probe(struct idxd_dev *idxd_dev) if (rc < 0) return -ENXIO; + rc = idxd_device_evl_setup(idxd); + if (rc < 0) { + idxd->cmd_status = IDXD_SCMD_DEV_EVL_ERR; + return rc; + } + /* Start device */ rc = idxd_device_enable(idxd); - if (rc < 0) + if (rc < 0) { + idxd_device_evl_free(idxd); return rc; + } /* Setup DMA device without channels */ rc = idxd_register_dma_device(idxd); if (rc < 0) { idxd_device_disable(idxd); + idxd_device_evl_free(idxd); idxd->cmd_status = IDXD_SCMD_DEV_DMA_ERR; return rc; } @@ -1488,6 +1574,7 @@ void idxd_device_drv_remove(struct idxd_dev *idxd_dev) idxd_device_disable(idxd); if (test_bit(IDXD_FLAG_CONFIGURABLE, &idxd->flags)) idxd_device_reset(idxd); + idxd_device_evl_free(idxd); } static enum idxd_dev_type dev_types[] = { diff --git a/drivers/dma/idxd/idxd.h b/drivers/dma/idxd/idxd.h index 2a71273f1822..c74681f02b18 100644 --- a/drivers/dma/idxd/idxd.h +++ b/drivers/dma/idxd/idxd.h @@ -262,7 +262,15 @@ struct idxd_driver_data { }; struct idxd_evl { + /* Lock to protect event log access. */ + spinlock_t lock; + void *log; + dma_addr_t dma; + /* Total size of event log = number of entries * entry size. */ + unsigned int log_size; + /* The number of entries in the event log. */ u16 size; + u16 head; }; struct idxd_device { @@ -324,6 +332,17 @@ struct idxd_device { struct idxd_evl *evl; }; +static inline unsigned int evl_ent_size(struct idxd_device *idxd) +{ + return idxd->hw.gen_cap.evl_support ? + (32 * (1 << idxd->hw.gen_cap.evl_support)) : 0; +} + +static inline unsigned int evl_size(struct idxd_device *idxd) +{ + return idxd->evl->size * evl_ent_size(idxd); +} + /* IDXD software descriptor */ struct idxd_desc { union { diff --git a/drivers/dma/idxd/init.c b/drivers/dma/idxd/init.c index d1fb01c115d8..2ffeb2f3a2c8 100644 --- a/drivers/dma/idxd/init.c +++ b/drivers/dma/idxd/init.c @@ -344,6 +344,7 @@ static int idxd_init_evl(struct idxd_device *idxd) if (!evl) return -ENOMEM; + spin_lock_init(&evl->lock); evl->size = IDXD_EVL_SIZE_MIN; idxd->evl = evl; return 0; diff --git a/drivers/dma/idxd/registers.h b/drivers/dma/idxd/registers.h index ea3a499a3c3c..11bb97cf7481 100644 --- a/drivers/dma/idxd/registers.h +++ b/drivers/dma/idxd/registers.h @@ -3,6 +3,8 @@ #ifndef _IDXD_REGISTERS_H_ #define _IDXD_REGISTERS_H_ +#include + /* PCI Config */ #define PCI_DEVICE_ID_INTEL_DSA_SPR0 0x0b25 #define PCI_DEVICE_ID_INTEL_IAX_SPR0 0x0cfe @@ -119,7 +121,8 @@ union gencfg_reg { u32 rdbuf_limit:8; u32 rsvd:4; u32 user_int_en:1; - u32 rsvd2:19; + u32 evl_en:1; + u32 rsvd2:18; }; u32 bits; } __packed; @@ -129,7 +132,8 @@ union genctrl_reg { struct { u32 softerr_int_en:1; u32 halt_int_en:1; - u32 rsvd:30; + u32 evl_int_en:1; + u32 rsvd:29; }; u32 bits; } __packed; @@ -299,6 +303,21 @@ union iaa_cap_reg { #define IDXD_IAACAP_OFFSET 0x180 +#define IDXD_EVLCFG_OFFSET 0xe0 +union evlcfg_reg { + struct { + u64 pasid_en:1; + u64 priv:1; + u64 rsvd:10; + u64 base_addr:52; + + u64 size:16; + u64 pasid:20; + u64 rsvd2:28; + }; + u64 bits[2]; +} __packed; + #define IDXD_EVL_SIZE_MIN 0x0040 #define IDXD_EVL_SIZE_MAX 0xffff @@ -539,4 +558,53 @@ union filter_cfg { u64 val; } __packed; +struct __evl_entry { + u64 rsvd:2; + u64 desc_valid:1; + u64 wq_idx_valid:1; + u64 batch:1; + u64 fault_rw:1; + u64 priv:1; + u64 err_info_valid:1; + u64 error:8; + u64 wq_idx:8; + u64 batch_id:8; + u64 operation:8; + u64 pasid:20; + u64 rsvd2:4; + + u16 batch_idx; + u16 rsvd3; + union { + /* Invalid Flags 0x11 */ + u32 invalid_flags; + /* Invalid Int Handle 0x19 */ + /* Page fault 0x1a */ + /* Page fault 0x06, 0x1f, only operand_id */ + /* Page fault before drain or in batch, 0x26, 0x27 */ + struct { + u16 int_handle; + u16 rci:1; + u16 ims:1; + u16 rcr:1; + u16 first_err_in_batch:1; + u16 rsvd4_2:9; + u16 operand_id:3; + }; + }; + u64 fault_addr; + u64 rsvd5; +} __packed; + +struct dsa_evl_entry { + struct __evl_entry e; + struct dsa_completion_record cr; +} __packed; + +struct iax_evl_entry { + struct __evl_entry e; + u64 rsvd[4]; + struct iax_completion_record cr; +} __packed; + #endif diff --git a/drivers/dma/idxd/sysfs.c b/drivers/dma/idxd/sysfs.c index 85644e5bde83..163fdfaa5022 100644 --- a/drivers/dma/idxd/sysfs.c +++ b/drivers/dma/idxd/sysfs.c @@ -1605,7 +1605,8 @@ static ssize_t event_log_size_store(struct device *dev, if (!test_bit(IDXD_FLAG_CONFIGURABLE, &idxd->flags)) return -EPERM; - if (val < IDXD_EVL_SIZE_MIN || val > IDXD_EVL_SIZE_MAX) + if (val < IDXD_EVL_SIZE_MIN || val > IDXD_EVL_SIZE_MAX || + (val * evl_ent_size(idxd) > ULONG_MAX - idxd->evl->dma)) return -EINVAL; idxd->evl->size = val; diff --git a/include/uapi/linux/idxd.h b/include/uapi/linux/idxd.h index fc47635b57dc..5d05bf12f2bd 100644 --- a/include/uapi/linux/idxd.h +++ b/include/uapi/linux/idxd.h @@ -30,6 +30,7 @@ enum idxd_scmd_stat { IDXD_SCMD_WQ_NO_PRIV = 0x800f0000, IDXD_SCMD_WQ_IRQ_ERR = 0x80100000, IDXD_SCMD_WQ_USER_NO_IOMMU = 0x80110000, + IDXD_SCMD_DEV_EVL_ERR = 0x80120000, }; #define IDXD_SCMD_SOFTERR_MASK 0x80000000 From patchwork Mon Mar 13 17:02:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fenghua Yu X-Patchwork-Id: 68978 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp1310516wrd; Mon, 13 Mar 2023 10:24:49 -0700 (PDT) X-Google-Smtp-Source: AK7set8BNQ0IEzFAOpvy4uMLQ8jKlLR1H1a9Fw+vSUywC3vn0krvbdkuD9nAJnblMPUyTLkb+GOc X-Received: by 2002:a05:6a20:a624:b0:cc:92ee:b119 with SMTP id bb36-20020a056a20a62400b000cc92eeb119mr27013879pzb.45.1678728289442; Mon, 13 Mar 2023 10:24:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1678728289; cv=none; d=google.com; s=arc-20160816; b=TqD+GUR1jCKmfMOZlvKc0x2148gwksfyqGIlNko+q3sm+3pqBS67dOEfgWcM+iBps3 FXFYtdKuoO213XONcue68tRjEqHG/haWCDpnN1R7p+IjYuXCC1S5mWXaAL50ZCYRznm7 7+xH2n8s0iTCoa6IbL9fABQEo1JkpVNPwL4lvtWN/UulyNbo+AqQ+FEhMuN+HmhWSs5z lSs9GUSf76ecbeQ2QmGUZUd540KtwkxFvefw6XXB3r0Vzel1JeEkr76VnhP6p938Eya9 npWgBzSwWq5K93u22iSREItYdEpIpnaKCMTbEP3z5TgM/Yy+hdHGJNKR95Xer5iLyTSP alKg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=KUlNR0Ek4zBMi9pf0fO8i7DlkB6NM9I1n9ESMKiCCak=; b=cE4iqtQDsYn6Uzj6Q4ykGQgAHGvZUUA4+PVZ2ByLIWQ8cVoHfWGr+3TSAUjVj65iGn puWh1yzq8SBeIukKNDwJ6vyIkB/NHpi1RqOdzsr8xJueU5/ZCbGxuFFbUXc+Cct/cOz8 aKQedWK7C8Iy3Cb8NyvSytHp1kVkb3PYisWy6HvWRbbWGkyGRIebbTwPd5b0p4NY1e+b EmUwMVZkSJq8TB+GE8VZrugKrtSoq7G1xLog2VuNeRT6Mnfwou3/d1Fn7H/CvsgYyzCk gRB1CwZ4LPCGlBn06SIi84RGZvmZlixL21VBM8HRfz2oFya/6ssor/CXmxSNP0HfTbaO UEUA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=JeQbXfx8; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id i67-20020a626d46000000b0062403d19c83si3531073pfc.341.2023.03.13.10.24.34; Mon, 13 Mar 2023 10:24:49 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=JeQbXfx8; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230515AbjCMRFv (ORCPT + 99 others); Mon, 13 Mar 2023 13:05:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42436 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230327AbjCMRFe (ORCPT ); Mon, 13 Mar 2023 13:05:34 -0400 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B4B1C6A7C; Mon, 13 Mar 2023 10:04:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1678727095; x=1710263095; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=3IHWCX0EXLUJ13RD60A81ekSWW5wHvs86VNvAujHKq8=; b=JeQbXfx8mPJGo5QKjprzflTRtkqVSgNYIXbit79ihOYAkY3iPfRJD1W3 IgyfGbfbN9kCAlrz9hs5CrMeGzOZzhIxhhxH0dT3j/Ng7bUOrWqFtiBJs xHD9Adjog8LJRiYbL4JYb10jojGDZ6ma8lBqzSpArwlnFtASo7bXeI3T4 LbPSljcNy0qsK4kYKtZhAzNd7QnxNQ/Qm8qL/uQSVD+GzMW0WiLKpo5nV Cz0/ZLquWAOhYciV9mTx7XWbqRiGjzglOJtJbpLz0xT7h6RzGqZAS7lol nSW7uFgDHGRXXMmBLXjvreDIdWWmTQ+XN/SnHIQi1U2lZ7IWgU0my61SB Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10648"; a="334679631" X-IronPort-AV: E=Sophos;i="5.98,257,1673942400"; d="scan'208";a="334679631" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Mar 2023 10:02:40 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10648"; a="708950902" X-IronPort-AV: E=Sophos;i="5.98,257,1673942400"; d="scan'208";a="708950902" Received: from fyu1.sc.intel.com ([172.25.103.126]) by orsmga008.jf.intel.com with ESMTP; 13 Mar 2023 10:02:40 -0700 From: Fenghua Yu To: "Vinod Koul" , "Dave Jiang" Cc: dmaengine@vger.kernel.org, "linux-kernel" , Tony Zhu , Fenghua Yu Subject: [PATCH v3 04/16] dmaengine: idxd: add interrupt handling for event log Date: Mon, 13 Mar 2023 10:02:07 -0700 Message-Id: <20230313170219.1956012-5-fenghua.yu@intel.com> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20230313170219.1956012-1-fenghua.yu@intel.com> References: <20230313170219.1956012-1-fenghua.yu@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1760274195009144980?= X-GMAIL-MSGID: =?utf-8?q?1760274195009144980?= From: Dave Jiang An event log interrupt is raised in the misc interrupt INTCAUSE register when an event is written by the hardware. Add basic event log processing support to the interrupt handler. The event log is a ring where the hardware owns the tail and the software owns the head. The hardware will advance the tail index when an additional event has been pushed to memory. The software will process the log entry and then advances the head. The log is full when (tail + 1) % log_size = head. The hardware will stop writing when the log is full. The user is expected to create a log size large enough to handle all the expected events. Tested-by: Tony Zhu Signed-off-by: Dave Jiang Co-developed-by: Fenghua Yu Signed-off-by: Fenghua Yu --- drivers/dma/idxd/irq.c | 48 ++++++++++++++++++++++++++++++++++++ drivers/dma/idxd/registers.h | 19 ++++++++++++++ include/uapi/linux/idxd.h | 1 + 3 files changed, 68 insertions(+) diff --git a/drivers/dma/idxd/irq.c b/drivers/dma/idxd/irq.c index 0d639303b515..52b8b7d9db22 100644 --- a/drivers/dma/idxd/irq.c +++ b/drivers/dma/idxd/irq.c @@ -217,6 +217,49 @@ static void idxd_int_handle_revoke(struct work_struct *work) kfree(revoke); } +static void process_evl_entry(struct idxd_device *idxd, struct __evl_entry *entry_head) +{ + struct device *dev = &idxd->pdev->dev; + u8 status; + + status = DSA_COMP_STATUS(entry_head->error); + dev_warn_ratelimited(dev, "Device error %#x operation: %#x fault addr: %#llx\n", + status, entry_head->operation, entry_head->fault_addr); +} + +static void process_evl_entries(struct idxd_device *idxd) +{ + union evl_status_reg evl_status; + unsigned int h, t; + struct idxd_evl *evl = idxd->evl; + struct __evl_entry *entry_head; + unsigned int ent_size = evl_ent_size(idxd); + u32 size; + + evl_status.bits = 0; + evl_status.int_pending = 1; + + spin_lock(&evl->lock); + /* Clear interrupt pending bit */ + iowrite32(evl_status.bits_upper32, + idxd->reg_base + IDXD_EVLSTATUS_OFFSET + sizeof(u32)); + h = evl->head; + evl_status.bits = ioread64(idxd->reg_base + IDXD_EVLSTATUS_OFFSET); + t = evl_status.tail; + size = idxd->evl->size; + + while (h != t) { + entry_head = (struct __evl_entry *)(evl->log + (h * ent_size)); + process_evl_entry(idxd, entry_head); + h = (h + 1) % size; + } + + evl->head = h; + evl_status.head = h; + iowrite32(evl_status.bits_lower32, idxd->reg_base + IDXD_EVLSTATUS_OFFSET); + spin_unlock(&evl->lock); +} + irqreturn_t idxd_misc_thread(int vec, void *data) { struct idxd_irq_entry *irq_entry = data; @@ -304,6 +347,11 @@ irqreturn_t idxd_misc_thread(int vec, void *data) perfmon_counter_overflow(idxd); } + if (cause & IDXD_INTC_EVL) { + val |= IDXD_INTC_EVL; + process_evl_entries(idxd); + } + val ^= cause; if (val) dev_warn_once(dev, "Unexpected interrupt cause bits set: %#x\n", diff --git a/drivers/dma/idxd/registers.h b/drivers/dma/idxd/registers.h index 11bb97cf7481..148db94f9373 100644 --- a/drivers/dma/idxd/registers.h +++ b/drivers/dma/idxd/registers.h @@ -168,6 +168,7 @@ enum idxd_device_reset_type { #define IDXD_INTC_OCCUPY 0x04 #define IDXD_INTC_PERFMON_OVFL 0x08 #define IDXD_INTC_HALT_STATE 0x10 +#define IDXD_INTC_EVL 0x20 #define IDXD_INTC_INT_HANDLE_REVOKED 0x80000000 #define IDXD_CMD_OFFSET 0xa0 @@ -558,6 +559,24 @@ union filter_cfg { u64 val; } __packed; +#define IDXD_EVLSTATUS_OFFSET 0xf0 + +union evl_status_reg { + struct { + u32 head:16; + u32 rsvd:16; + u32 tail:16; + u32 rsvd2:14; + u32 int_pending:1; + u32 rsvd3:1; + }; + struct { + u32 bits_lower32; + u32 bits_upper32; + }; + u64 bits; +} __packed; + struct __evl_entry { u64 rsvd:2; u64 desc_valid:1; diff --git a/include/uapi/linux/idxd.h b/include/uapi/linux/idxd.h index 5d05bf12f2bd..0bc8eea18586 100644 --- a/include/uapi/linux/idxd.h +++ b/include/uapi/linux/idxd.h @@ -170,6 +170,7 @@ enum iax_completion_status { #define DSA_COMP_STATUS_MASK 0x7f #define DSA_COMP_STATUS_WRITE 0x80 +#define DSA_COMP_STATUS(status) ((status) & DSA_COMP_STATUS_MASK) struct dsa_hw_desc { uint32_t pasid:20; From patchwork Mon Mar 13 17:02:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fenghua Yu X-Patchwork-Id: 68965 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp1307489wrd; Mon, 13 Mar 2023 10:18:03 -0700 (PDT) X-Google-Smtp-Source: AK7set9BTSqyMuqblbmvFWbw1/VEAstT3yvsFr/o/bKE8SdVWbeQpunFwKS0f7uhzPXFCMiAgyO6 X-Received: by 2002:a17:902:cecc:b0:19a:a815:2858 with SMTP id d12-20020a170902cecc00b0019aa8152858mr40956387plg.51.1678727883568; Mon, 13 Mar 2023 10:18:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1678727883; cv=none; d=google.com; s=arc-20160816; b=LfRylSyLm+6vuHqC9vvpaR3+VJIWty9K8pMKEQGTK3xmTWAcGRSUbzhLZNkHtgeSr8 0vWObqUPxs3MLYD+ne8NM12XczaEfvlH/0f2QyphP7L/AI4npehSe90idQfeUH3tduS+ kHmdMYyJWVYgHdAFXbk7HaZTW8VOYayIggUWaGMYud4q1VBZyZWKKtyun97ZM0DtgdBL trttZcaabmr51ekmWmqhymSXfB4MLv4kAtXOOJndjfzJSjIkSS3F71/kk9bHQiN7Hrsy c4KM2n7h9ZCLYlaxA3nK5qSzeCmh9pOqWQE6ESjBn5OD0mJBmieZXAD7w6aREQhKUYgB mfHg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=D9yMz6BAnlJHxfIadTgqKjB7kYgBQ5HZN6A8xQsveLo=; b=u8pNxJU8qg++oWnVNeSIb85F8oaqhuEXCgulgLE/KCv/OFU7yxecb28K5o/BYobqzB jRjjMLdlZ6HHP5rdIOjad7wH4g9vhacI15mJcc7SMlvoyiB/D5LptmCklAzqgD3kgjWi lsfu7LgzlWqxw82n4NZCZqChbOmOqZ9358cxoj0AUId7YCGI4nGQ5WGPtRc/tx/8v8E7 plN1mgIHaQtecfft+KotXsvPmIPM39seqJgL36NChEAusi0I0lBlKsN57F9uVBbJoyKD u1/ZjDNRKuPGEXL4xTirFFf77tN876HrjAmlo1G27aPgJebaN45z43ikOL++p0I6aVTT zVCA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=ZId+1skH; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id kt5-20020a170903088500b0019a7278f15dsi226213plb.96.2023.03.13.10.17.51; Mon, 13 Mar 2023 10:18:03 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=ZId+1skH; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230338AbjCMRFs (ORCPT + 99 others); Mon, 13 Mar 2023 13:05:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42254 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229975AbjCMRFZ (ORCPT ); Mon, 13 Mar 2023 13:05:25 -0400 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8902148E2D; Mon, 13 Mar 2023 10:04:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1678727091; x=1710263091; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=KjifzwqiI5VZvv5uFKQMheCb52dPcD6KD7anBqOysQE=; b=ZId+1skH0G5RRw+KkGhhZqJyI2dRvSXUKRPFWIQqTZTnufxAoaW9Q9f+ 6RkqMk1vsmUZ29tdQL8l2YEl1q+/yni11SQdAr4shBR4QepF6NDTPb/8w 1WtAP8CiQ812ApwDDECHBLErsr5DXcb0Sm+SFQVbBWpt5N47gXS7Vm3U5 DLk3fRpHlLYDJNEAw03Rn3L0CAGdJAxy7SJm93OdCUax3vzP90JYN/RN5 5UrMtKBD1tGh9fD6U1zK5D+aDzXgasdfIQVgyOWgq+Ob/LUl/JR9kFeHp 160FCMKBcgYU2k38svfyyLGq1KvaEMguzDArBkPqQdNyxZWhiiwsKW9GA A==; X-IronPort-AV: E=McAfee;i="6500,9779,10648"; a="334679638" X-IronPort-AV: E=Sophos;i="5.98,257,1673942400"; d="scan'208";a="334679638" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Mar 2023 10:02:41 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10648"; a="708950905" X-IronPort-AV: E=Sophos;i="5.98,257,1673942400"; d="scan'208";a="708950905" Received: from fyu1.sc.intel.com ([172.25.103.126]) by orsmga008.jf.intel.com with ESMTP; 13 Mar 2023 10:02:40 -0700 From: Fenghua Yu To: "Vinod Koul" , "Dave Jiang" Cc: dmaengine@vger.kernel.org, "linux-kernel" , Tony Zhu , Fenghua Yu Subject: [PATCH v3 05/16] dmanegine: idxd: add debugfs for event log dump Date: Mon, 13 Mar 2023 10:02:08 -0700 Message-Id: <20230313170219.1956012-6-fenghua.yu@intel.com> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20230313170219.1956012-1-fenghua.yu@intel.com> References: <20230313170219.1956012-1-fenghua.yu@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1760273769083258494?= X-GMAIL-MSGID: =?utf-8?q?1760273769083258494?= From: Dave Jiang Add debugfs entry to dump the content of the event log for debugging. The function will dump all non-zero entries in the event log. It will note which entries are processed and which entries are still pending processing at the time of the dump. The entries may not always be in chronological order due to the log is a circular buffer. Tested-by: Tony Zhu Signed-off-by: Dave Jiang Co-developed-by: Fenghua Yu Signed-off-by: Fenghua Yu --- drivers/dma/idxd/Makefile | 2 +- drivers/dma/idxd/debugfs.c | 138 +++++++++++++++++++++++++++++++++++++ drivers/dma/idxd/idxd.h | 9 +++ drivers/dma/idxd/init.c | 12 ++++ include/uapi/linux/idxd.h | 6 +- 5 files changed, 164 insertions(+), 3 deletions(-) create mode 100644 drivers/dma/idxd/debugfs.c diff --git a/drivers/dma/idxd/Makefile b/drivers/dma/idxd/Makefile index a1e9f2b3a37c..dc096839ac63 100644 --- a/drivers/dma/idxd/Makefile +++ b/drivers/dma/idxd/Makefile @@ -1,7 +1,7 @@ ccflags-y += -DDEFAULT_SYMBOL_NAMESPACE=IDXD obj-$(CONFIG_INTEL_IDXD) += idxd.o -idxd-y := init.o irq.o device.o sysfs.o submit.o dma.o cdev.o +idxd-y := init.o irq.o device.o sysfs.o submit.o dma.o cdev.o debugfs.o idxd-$(CONFIG_INTEL_IDXD_PERFMON) += perfmon.o diff --git a/drivers/dma/idxd/debugfs.c b/drivers/dma/idxd/debugfs.c new file mode 100644 index 000000000000..9cfbd9b14c4c --- /dev/null +++ b/drivers/dma/idxd/debugfs.c @@ -0,0 +1,138 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright(c) 2021 Intel Corporation. All rights rsvd. */ +#include +#include +#include +#include +#include +#include +#include +#include "idxd.h" +#include "registers.h" + +static struct dentry *idxd_debugfs_dir; + +static void dump_event_entry(struct idxd_device *idxd, struct seq_file *s, + u16 index, int *count, bool processed) +{ + struct idxd_evl *evl = idxd->evl; + struct dsa_evl_entry *entry; + struct dsa_completion_record *cr; + u64 *raw; + int i; + int evl_strides = evl_ent_size(idxd) / sizeof(u64); + + entry = (struct dsa_evl_entry *)evl->log + index; + + if (!entry->e.desc_valid) + return; + + seq_printf(s, "Event Log entry %d (real index %u) processed: %u\n", + *count, index, processed); + + seq_printf(s, "desc valid %u wq idx valid %u\n" + "batch %u fault rw %u priv %u error 0x%x\n" + "wq idx %u op %#x pasid %u batch idx %u\n" + "fault addr %#llx\n", + entry->e.desc_valid, entry->e.wq_idx_valid, + entry->e.batch, entry->e.fault_rw, entry->e.priv, + entry->e.error, entry->e.wq_idx, entry->e.operation, + entry->e.pasid, entry->e.batch_idx, entry->e.fault_addr); + + cr = &entry->cr; + seq_printf(s, "status %#x result %#x fault_info %#x bytes_completed %u\n" + "fault addr %#llx inv flags %#x\n\n", + cr->status, cr->result, cr->fault_info, cr->bytes_completed, + cr->fault_addr, cr->invalid_flags); + + raw = (u64 *)entry; + + for (i = 0; i < evl_strides; i++) + seq_printf(s, "entry[%d] = %#llx\n", i, raw[i]); + + seq_puts(s, "\n"); + *count += 1; +} + +static int debugfs_evl_show(struct seq_file *s, void *d) +{ + struct idxd_device *idxd = s->private; + struct idxd_evl *evl = idxd->evl; + union evl_status_reg evl_status; + u16 h, t, evl_size, i; + int count = 0; + bool processed = true; + + if (!evl || !evl->log) + return 0; + + spin_lock(&evl->lock); + + h = evl->head; + evl_status.bits = ioread64(idxd->reg_base + IDXD_EVLSTATUS_OFFSET); + t = evl_status.tail; + evl_size = evl->size; + + seq_printf(s, "Event Log head %u tail %u interrupt pending %u\n\n", + evl_status.head, evl_status.tail, evl_status.int_pending); + + i = t; + while (1) { + i = (i + 1) % evl_size; + if (i == t) + break; + + if (processed && i == h) + processed = false; + dump_event_entry(idxd, s, i, &count, processed); + } + + spin_unlock(&evl->lock); + return 0; +} + +DEFINE_SHOW_ATTRIBUTE(debugfs_evl); + +int idxd_device_init_debugfs(struct idxd_device *idxd) +{ + if (IS_ERR_OR_NULL(idxd_debugfs_dir)) + return 0; + + idxd->dbgfs_dir = debugfs_create_dir(dev_name(idxd_confdev(idxd)), idxd_debugfs_dir); + if (IS_ERR(idxd->dbgfs_dir)) + return PTR_ERR(idxd->dbgfs_dir); + + if (idxd->evl) { + idxd->dbgfs_evl_file = debugfs_create_file("event_log", 0400, + idxd->dbgfs_dir, idxd, + &debugfs_evl_fops); + if (IS_ERR(idxd->dbgfs_evl_file)) { + debugfs_remove_recursive(idxd->dbgfs_dir); + idxd->dbgfs_dir = NULL; + return PTR_ERR(idxd->dbgfs_evl_file); + } + } + + return 0; +} + +void idxd_device_remove_debugfs(struct idxd_device *idxd) +{ + debugfs_remove_recursive(idxd->dbgfs_dir); +} + +int idxd_init_debugfs(void) +{ + if (!debugfs_initialized()) + return 0; + + idxd_debugfs_dir = debugfs_create_dir(KBUILD_MODNAME, NULL); + if (IS_ERR(idxd_debugfs_dir)) + return PTR_ERR(idxd_debugfs_dir); + return 0; +} + +void idxd_remove_debugfs(void) +{ + debugfs_remove_recursive(idxd_debugfs_dir); +} diff --git a/drivers/dma/idxd/idxd.h b/drivers/dma/idxd/idxd.h index c74681f02b18..b923b90b7299 100644 --- a/drivers/dma/idxd/idxd.h +++ b/drivers/dma/idxd/idxd.h @@ -330,6 +330,9 @@ struct idxd_device { unsigned long *opcap_bmap; struct idxd_evl *evl; + + struct dentry *dbgfs_dir; + struct dentry *dbgfs_evl_file; }; static inline unsigned int evl_ent_size(struct idxd_device *idxd) @@ -704,4 +707,10 @@ static inline void perfmon_init(void) {} static inline void perfmon_exit(void) {} #endif +/* debugfs */ +int idxd_device_init_debugfs(struct idxd_device *idxd); +void idxd_device_remove_debugfs(struct idxd_device *idxd); +int idxd_init_debugfs(void); +void idxd_remove_debugfs(void); + #endif diff --git a/drivers/dma/idxd/init.c b/drivers/dma/idxd/init.c index 2ffeb2f3a2c8..d19bc6389221 100644 --- a/drivers/dma/idxd/init.c +++ b/drivers/dma/idxd/init.c @@ -670,6 +670,10 @@ static int idxd_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) goto err_dev_register; } + rc = idxd_device_init_debugfs(idxd); + if (rc) + dev_warn(dev, "IDXD debugfs failed to setup\n"); + dev_info(&pdev->dev, "Intel(R) Accelerator Device (v%x)\n", idxd->hw.version); @@ -732,6 +736,7 @@ static void idxd_remove(struct pci_dev *pdev) idxd_shutdown(pdev); if (device_pasid_enabled(idxd)) idxd_disable_system_pasid(idxd); + idxd_device_remove_debugfs(idxd); irq_entry = idxd_get_ie(idxd, 0); free_irq(irq_entry->vector, irq_entry); @@ -789,6 +794,10 @@ static int __init idxd_init_module(void) if (err) goto err_cdev_register; + err = idxd_init_debugfs(); + if (err) + goto err_debugfs; + err = pci_register_driver(&idxd_pci_driver); if (err) goto err_pci_register; @@ -796,6 +805,8 @@ static int __init idxd_init_module(void) return 0; err_pci_register: + idxd_remove_debugfs(); +err_debugfs: idxd_cdev_remove(); err_cdev_register: idxd_driver_unregister(&idxd_user_drv); @@ -816,5 +827,6 @@ static void __exit idxd_exit_module(void) pci_unregister_driver(&idxd_pci_driver); idxd_cdev_remove(); perfmon_exit(); + idxd_remove_debugfs(); } module_exit(idxd_exit_module); diff --git a/include/uapi/linux/idxd.h b/include/uapi/linux/idxd.h index 0bc8eea18586..e86199d09a91 100644 --- a/include/uapi/linux/idxd.h +++ b/include/uapi/linux/idxd.h @@ -311,7 +311,8 @@ struct dsa_completion_record { uint8_t result; uint8_t dif_status; }; - uint16_t rsvd; + uint8_t fault_info; + uint8_t rsvd; uint32_t bytes_completed; uint64_t fault_addr; union { @@ -368,7 +369,8 @@ struct dsa_raw_completion_record { struct iax_completion_record { volatile uint8_t status; uint8_t error_code; - uint16_t rsvd; + uint8_t fault_info; + uint8_t rsvd; uint32_t bytes_completed; uint64_t fault_addr; uint32_t invalid_flags; From patchwork Mon Mar 13 17:02:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fenghua Yu X-Patchwork-Id: 68961 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp1307390wrd; Mon, 13 Mar 2023 10:17:50 -0700 (PDT) X-Google-Smtp-Source: AK7set9bLQmyWUYIDhrMPZNqE8pL5HIgCYJH4Xhkly8KsIfg8dIR3zH7orhgoHh4zijazV/c5m/o X-Received: by 2002:a17:902:ec8a:b0:19c:f2d5:5830 with SMTP id x10-20020a170902ec8a00b0019cf2d55830mr42825156plg.44.1678727869882; Mon, 13 Mar 2023 10:17:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1678727869; cv=none; d=google.com; s=arc-20160816; b=r+33bynxf+yz9pot2JUOljLAwSAjFkZB5l4zwtQ8bOFBb4scjeXkD//ZQ1eDLxZGbp GZV1E7H4fwIadSvKrhGcmr2pcaZMU6zaYLm5jYGoeVOffgEMna/kPaRD0aaqq5rXya87 qVmZwYICASgmGiAj0viCzqmkkDZ2IrGy+uvJiR5W9sKF6JEBJjjR7LfnzVK9AULLViKW fq1KMCDBMlR7gJM4TK5sEZREa6fqIj7K4++zeLpEz7DhY5kadeuthAEYTulLo4QEgI7k 2sKQUGe27OQdl9vbP+rJkVgbj9goRV5rTnJnZ7udX8Bwg6Lsgm9/lwh4+kQ0RjiKpqES /8Ig== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=563lz7b++xCnS1fkUFrfVn1DUbjZXMbmcRqaPB/7Hc4=; b=Ym4yLJu1NujTD+q8/5YnrwKyLINy4zrunfLZsxF4+f3TNgebEMb4H51uaVQxyThCJh SabA8IR3QNqWctGk7RC+GcXMSa4dgKSIQoVe7eJ6joZho7ZVEhLgb9V2SNA+0Om1bZwq 0HyMuKd6l5uJepJHV8VbhPxuoOSULvKHagHiTlEz+SwehfKMh4i+VynTPipQPU76D0RJ 8pzkyTrfAW0+cx6NCcx+J9Sr2uwJDhnWwwsCGIyKBfzcS2+nnCi8y3PPYAea+lqFnDAZ 6IVOy00NJuDWufPPwrnAiCQVkNsRwaQ61t2PbfNUjfvnSDJgPLH5i46XEw6jfh0fu2IW /SBg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="XFLYA/9V"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id l4-20020a17090a4d4400b0023d1bd70913si224234pjh.42.2023.03.13.10.17.37; Mon, 13 Mar 2023 10:17:49 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="XFLYA/9V"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229552AbjCMRGA (ORCPT + 99 others); Mon, 13 Mar 2023 13:06:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42290 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230355AbjCMRFe (ORCPT ); Mon, 13 Mar 2023 13:05:34 -0400 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 518812410F; Mon, 13 Mar 2023 10:05:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1678727100; x=1710263100; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=f44hJzsv5G4du9Q7bT5CpVrZ9HY7StygNVQkHIRMAYU=; b=XFLYA/9VdXnbXt0BIHW9jhKH9Q68KrU51soTpgY8CUXmxqGbO4q0KgK/ eZ26iyy7s4EYe2sMOIC7FM/MK9erJ2poHEBciD3FIl8+rCc2Gte11UgsQ zpVyTMvXPTjKJ+pHwTfaulkLuvfN3P7l7B2BezwjpvJKKdS/jp8Yf6+cE AogZjHdor0DSpqQ7R+pSGV07r2/7Gh+Q5yEArjJ1Uj135WMNhJC2uMrfA dV3kFx2wCl1k4FbYEKDjF0Q5K/hUhrWzdFuWsTvuLHe4lJkxZlCAYXeOK qr6WOU5bZvS1LeVpa+05WX4Kvt9Cc4fH7abe7pMyDBckyScV4wEbg5+rj g==; X-IronPort-AV: E=McAfee;i="6500,9779,10648"; a="334679645" X-IronPort-AV: E=Sophos;i="5.98,257,1673942400"; d="scan'208";a="334679645" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Mar 2023 10:02:41 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10648"; a="708950909" X-IronPort-AV: E=Sophos;i="5.98,257,1673942400"; d="scan'208";a="708950909" Received: from fyu1.sc.intel.com ([172.25.103.126]) by orsmga008.jf.intel.com with ESMTP; 13 Mar 2023 10:02:40 -0700 From: Fenghua Yu To: "Vinod Koul" , "Dave Jiang" Cc: dmaengine@vger.kernel.org, "linux-kernel" , Tony Zhu , Fenghua Yu Subject: [PATCH v3 06/16] dmaengine: idxd: add per DSA wq workqueue for processing cr faults Date: Mon, 13 Mar 2023 10:02:09 -0700 Message-Id: <20230313170219.1956012-7-fenghua.yu@intel.com> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20230313170219.1956012-1-fenghua.yu@intel.com> References: <20230313170219.1956012-1-fenghua.yu@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1760273754845429070?= X-GMAIL-MSGID: =?utf-8?q?1760273754845429070?= From: Dave Jiang Add a workqueue for user submitted completion record fault processing. The workqueue creation and destruction lifetime will be tied to the user sub-driver since it will only be used when the wq is a user type. Tested-by: Tony Zhu Signed-off-by: Dave Jiang Co-developed-by: Fenghua Yu Signed-off-by: Fenghua Yu --- drivers/dma/idxd/cdev.c | 11 +++++++++++ drivers/dma/idxd/idxd.h | 1 + 2 files changed, 12 insertions(+) diff --git a/drivers/dma/idxd/cdev.c b/drivers/dma/idxd/cdev.c index 674bfefca088..cbe29e1a6a44 100644 --- a/drivers/dma/idxd/cdev.c +++ b/drivers/dma/idxd/cdev.c @@ -330,6 +330,13 @@ static int idxd_user_drv_probe(struct idxd_dev *idxd_dev) } mutex_lock(&wq->wq_lock); + + wq->wq = create_workqueue(dev_name(wq_confdev(wq))); + if (!wq->wq) { + rc = -ENOMEM; + goto wq_err; + } + wq->type = IDXD_WQT_USER; rc = drv_enable_wq(wq); if (rc < 0) @@ -348,7 +355,9 @@ static int idxd_user_drv_probe(struct idxd_dev *idxd_dev) err_cdev: drv_disable_wq(wq); err: + destroy_workqueue(wq->wq); wq->type = IDXD_WQT_NONE; +wq_err: mutex_unlock(&wq->wq_lock); return rc; } @@ -361,6 +370,8 @@ static void idxd_user_drv_remove(struct idxd_dev *idxd_dev) idxd_wq_del_cdev(wq); drv_disable_wq(wq); wq->type = IDXD_WQT_NONE; + destroy_workqueue(wq->wq); + wq->wq = NULL; mutex_unlock(&wq->wq_lock); } diff --git a/drivers/dma/idxd/idxd.h b/drivers/dma/idxd/idxd.h index b923b90b7299..6e56361ae658 100644 --- a/drivers/dma/idxd/idxd.h +++ b/drivers/dma/idxd/idxd.h @@ -185,6 +185,7 @@ struct idxd_wq { struct idxd_dev idxd_dev; struct idxd_cdev *idxd_cdev; struct wait_queue_head err_queue; + struct workqueue_struct *wq; struct idxd_device *idxd; int id; struct idxd_irq_entry ie; From patchwork Mon Mar 13 17:02:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fenghua Yu X-Patchwork-Id: 68974 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp1310124wrd; Mon, 13 Mar 2023 10:23:58 -0700 (PDT) X-Google-Smtp-Source: AK7set/gp85zYHnVC0MtZ/RhZ9lXxbq9O/6iFSDMpZ9R1ShjTe64Ut7RtHwT4zDnItr+Gjd32B7J X-Received: by 2002:a05:6a20:1582:b0:ce:5bf0:3be7 with SMTP id h2-20020a056a20158200b000ce5bf03be7mr11616937pzj.27.1678728238653; Mon, 13 Mar 2023 10:23:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1678728238; cv=none; d=google.com; s=arc-20160816; b=kOdLBsPFkfFJe1G2vUzqprzeH2VbNu/vBBaeDu9pu8orYSddwSZdJX+yw8w+LgK00A HqGOeh0D982Yovc7/ecOuZ6leDneRjjv20T8F5axHBm5YrvrJPCLsMkqV8LqFlUCk21N 3WD0X15s/uq8XNz5fq4qC+O3bWqCHDgZGAnRZXJ/D79CK0H9JSqtIMN68wCBuwbwG3QH LFKk+2q9gX7VolZ9EJTe4T96ZH7YM0a1us55E828W1s0o5mubsQvfifSXODGjUiNDj/z kQqMvxtSVsR/KMirKvU7r506U+Zv/Rh5m6P6khYn+we8V5XsACrd2mAuzJ3DSfHoQJXh vTGw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=QcthPXCXeFXjZ9ptWd0SLKUd/bUt6IGaaFMR1shFnCc=; b=cIZP8hS+gV+OJIktQcK3w/U6ZtGYBI64n0vrj3xzdx6qPiTtfvWmXIhA5eejP1qOmi May7i8tXKnb3i3s00q/WTlIorxBmQxT3S6qh8HGq22dGHPCjM08Tp4lahXPSuaTric/n 9WOsTu3HsT0Gv8bBOXHp6qw0/NnsnVGA6cdSOKOG4Wvmi/K9lKPuI3aXJ0VjsJvYrPyg eSAFSwdHPkWSBzwW7yNGkQqtAYicMzHeziod1Qb95nSkDlQlMsxl442SFi/WRHKNhtCQ FqyNFLSx5b2ffzliiPA7nKN2bS91lTeJaTW/3ZJ2QGfKShe05QXdSBx0cPGh2PcAHwqe A6Cw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=eVQgSYpr; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id n2-20020a654502000000b00503012d18f8si7169951pgq.858.2023.03.13.10.23.44; Mon, 13 Mar 2023 10:23:58 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=eVQgSYpr; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231139AbjCMRF4 (ORCPT + 99 others); Mon, 13 Mar 2023 13:05:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42438 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230354AbjCMRFe (ORCPT ); Mon, 13 Mar 2023 13:05:34 -0400 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D8E321716D; Mon, 13 Mar 2023 10:04:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1678727097; x=1710263097; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=mn1nFVof/uaU7GNK/6NGBnoeC6aCmePHCwCCzsaRLNQ=; b=eVQgSYprQ57B++z+/2qJdOICYZFfsDDhfrFdcvUWQfFlDN7GtHsikR+R P8mvNf5NAzIvh28mghGgecX7fSTrO1TMym9nHKXXUZHFSixrasxn9nYCq bL5BUM76657yz8P3PxH36k/TczkaFQZDnp21iaCD2IsfXPOkb3tyYPjOP yHsJhIcWfSFPPIhaNSaHRrukJPCrycr9yaB+OZpsmZMsYqZxpgel7h3ub ui3S+RRN1omw2UqwUU5AvHtNrEIQnDyrxIlH67FdMDvVxG5sB3AC2gy4n 6ceOXR6V0sKj7VhmH9BvRRb+1hnYVM84cEgTb9VQY0P1XPZc9eqxgggVM w==; X-IronPort-AV: E=McAfee;i="6500,9779,10648"; a="334679653" X-IronPort-AV: E=Sophos;i="5.98,257,1673942400"; d="scan'208";a="334679653" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Mar 2023 10:02:41 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10648"; a="708950914" X-IronPort-AV: E=Sophos;i="5.98,257,1673942400"; d="scan'208";a="708950914" Received: from fyu1.sc.intel.com ([172.25.103.126]) by orsmga008.jf.intel.com with ESMTP; 13 Mar 2023 10:02:41 -0700 From: Fenghua Yu To: "Vinod Koul" , "Dave Jiang" Cc: dmaengine@vger.kernel.org, "linux-kernel" , Tony Zhu , Fenghua Yu Subject: [PATCH v3 07/16] dmaengine: idxd: create kmem cache for event log fault items Date: Mon, 13 Mar 2023 10:02:10 -0700 Message-Id: <20230313170219.1956012-8-fenghua.yu@intel.com> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20230313170219.1956012-1-fenghua.yu@intel.com> References: <20230313170219.1956012-1-fenghua.yu@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1760274141724586015?= X-GMAIL-MSGID: =?utf-8?q?1760274141724586015?= From: Dave Jiang Add a kmem cache per device for allocating event log fault context. The context allows an event log entry to be copied and passed to a software workqueue to be processed. Due to each device can have different sized event log entry depending on device type, it's not possible to have a global kmem cache. Tested-by: Tony Zhu Signed-off-by: Dave Jiang Co-developed-by: Fenghua Yu Signed-off-by: Fenghua Yu --- drivers/dma/idxd/idxd.h | 10 ++++++++++ drivers/dma/idxd/init.c | 9 +++++++++ drivers/dma/idxd/sysfs.c | 1 + 3 files changed, 20 insertions(+) diff --git a/drivers/dma/idxd/idxd.h b/drivers/dma/idxd/idxd.h index 6e56361ae658..c5d99c179902 100644 --- a/drivers/dma/idxd/idxd.h +++ b/drivers/dma/idxd/idxd.h @@ -274,6 +274,15 @@ struct idxd_evl { u16 head; }; +struct idxd_evl_fault { + struct work_struct work; + struct idxd_wq *wq; + u8 status; + + /* make this last member always */ + struct __evl_entry entry[]; +}; + struct idxd_device { struct idxd_dev idxd_dev; struct idxd_driver_data *data; @@ -331,6 +340,7 @@ struct idxd_device { unsigned long *opcap_bmap; struct idxd_evl *evl; + struct kmem_cache *evl_cache; struct dentry *dbgfs_dir; struct dentry *dbgfs_evl_file; diff --git a/drivers/dma/idxd/init.c b/drivers/dma/idxd/init.c index d19bc6389221..a7c98fac7a85 100644 --- a/drivers/dma/idxd/init.c +++ b/drivers/dma/idxd/init.c @@ -346,6 +346,15 @@ static int idxd_init_evl(struct idxd_device *idxd) spin_lock_init(&evl->lock); evl->size = IDXD_EVL_SIZE_MIN; + + idxd->evl_cache = kmem_cache_create(dev_name(idxd_confdev(idxd)), + sizeof(struct idxd_evl_fault) + evl_ent_size(idxd), + 0, 0, NULL); + if (!idxd->evl_cache) { + kfree(evl); + return -ENOMEM; + } + idxd->evl = evl; return 0; } diff --git a/drivers/dma/idxd/sysfs.c b/drivers/dma/idxd/sysfs.c index 163fdfaa5022..8b9dfa0d2b99 100644 --- a/drivers/dma/idxd/sysfs.c +++ b/drivers/dma/idxd/sysfs.c @@ -1718,6 +1718,7 @@ static void idxd_conf_device_release(struct device *dev) kfree(idxd->wqs); kfree(idxd->engines); kfree(idxd->evl); + kmem_cache_destroy(idxd->evl_cache); ida_free(&idxd_ida, idxd->id); bitmap_free(idxd->opcap_bmap); kfree(idxd); From patchwork Mon Mar 13 17:02:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fenghua Yu X-Patchwork-Id: 68966 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp1307574wrd; Mon, 13 Mar 2023 10:18:17 -0700 (PDT) X-Google-Smtp-Source: AK7set+cYXCzSlSCaeH2CRmI/In7y6vAN9mH3sJ9/TxDIwG1HeOt/iiOhtd62VrpoJB6sHUDCSQh X-Received: by 2002:a17:90b:1b4a:b0:237:b5d4:c0df with SMTP id nv10-20020a17090b1b4a00b00237b5d4c0dfmr34740379pjb.6.1678727897290; Mon, 13 Mar 2023 10:18:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1678727897; cv=none; d=google.com; s=arc-20160816; b=o4FsrYjWYK5A91uAKNSl8fO6I3INrtlbjwZ+qKA6NIT1PHh3S+TbAdkXeqSARu2PFv +mMNBHLPzv1SegBZdgUrepi860KbGZ8DvVxx7BgEmHLN1IsyU32JWVZIqDbS25JLEhcg GSlEo6r6WGHBkwfrOnmPu8+SZBW7+OJ41jacSE/VNdQrcFrYF57kRmiP7TxHNrQ7HnjQ q9PqBZY2Kqfn9PtcBP0zgy/TTAjiFmUPJV4F0IdrEVf5mNCxBENptsYV3XhU0n5IP6H8 HSOXqDuTOH2IOLsYuFcXeEWh1AmeLPb1UkvG2T1wD0UCIX+i2OML4dAwREccH2bcF7ml 808A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=O8KvXoRtO7dLJXCpe5dNxZ3Coomcdj0F7pRLpNPnkrw=; b=gTpSwZ9Cp3YjbCGy+BVOX051NiTEzUADkpMYbsgBwPor/tencpc+GaI2/lpEqoqiYb eCJiHZGPxV/yplOJCIGfkjt2xh9gS8UFbPK+nR5l96lkvas9WPs276ka17SYlYMD0UXp CDxFbysI9OKSxKBJ1dkYhy2zA865nya4rymgmZG4CIOmP72QUZDiNtNVI5H8HuhddlDz PgEclLLqq1O9ps11E1Jzy7FrwZdsH7U0m+xRC9j4PnCBNmZ9yOsATEYLQVP9I7b0PnGV N6vp5m5VF/QxpTiFZ65TtEInIBcQYg4SYQViXrJ6wyEjtrEpuY7fhLXm/DTMPVWLpZvO C11A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=fLauRILh; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id k1-20020a17090ad08100b0023d2847e355si254070pju.0.2023.03.13.10.18.04; Mon, 13 Mar 2023 10:18:17 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=fLauRILh; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231398AbjCMRGb (ORCPT + 99 others); Mon, 13 Mar 2023 13:06:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42838 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230491AbjCMRGS (ORCPT ); Mon, 13 Mar 2023 13:06:18 -0400 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0E1BC78CAE; Mon, 13 Mar 2023 10:05:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1678727126; x=1710263126; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=MnpLq/KXUUwPvadOwJMGjEgqddJKg/VVIpHgp4pkIOQ=; b=fLauRILh8v7pQ32opo/DiKBMYkN0mgVkDC0VINHdbo2E+a7F+zCktxGV ulbi5wCTlUG0Jh2kVhMNompeQeKk+aimqo2AmvwZSc36O9MshLn1qaYUy K6eGDjG2yDbQn/Cga2fAdR5MnHh7yGFgj2xFqquoio/OSA/9j3v2urm+I g9nwP2/fiJ9tpSL3peXGrkl+QLilXPwTQpGnLajORovTcKLd49Snpx9LC l/bWSEd7CoePdjaClSt4Zl6kjQSOhKGf8nuXLeSugVkcF1CuS9wDdWLNG z9JA4H94u/IyNiw/qJol0plF9BC9cn4hIncrBQzuaG2WVb5OXuV5sKmW8 A==; X-IronPort-AV: E=McAfee;i="6500,9779,10648"; a="334679660" X-IronPort-AV: E=Sophos;i="5.98,257,1673942400"; d="scan'208";a="334679660" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Mar 2023 10:02:42 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10648"; a="708950920" X-IronPort-AV: E=Sophos;i="5.98,257,1673942400"; d="scan'208";a="708950920" Received: from fyu1.sc.intel.com ([172.25.103.126]) by orsmga008.jf.intel.com with ESMTP; 13 Mar 2023 10:02:41 -0700 From: Fenghua Yu To: "Vinod Koul" , "Dave Jiang" Cc: dmaengine@vger.kernel.org, "linux-kernel" , Fenghua Yu Subject: [PATCH v3 08/16] dmaengine: idxd: define idxd_copy_cr() Date: Mon, 13 Mar 2023 10:02:11 -0700 Message-Id: <20230313170219.1956012-9-fenghua.yu@intel.com> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20230313170219.1956012-1-fenghua.yu@intel.com> References: <20230313170219.1956012-1-fenghua.yu@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1760273783552160046?= X-GMAIL-MSGID: =?utf-8?q?1760273783552160046?= Define idxd_copy_cr() to copy completion record to fault address in user address that is found by wq and PASID. It will be used to write the user's completion record that the hardware device is not able to write to due to user page fault. An xarray is added to associate the PASID and mm with the struct idxd_user_context so mm can be found by PASID and wq. Although access_remote_vm() can access remote mm, it's not exported and should not be exported because driver callers may easily make mistakes by missing mm reference. Since access_remote_vm() cannot be called directly, _idxd_copy_cr() implements a simplified version of access_remote_vm() to copy completion record to a remote mm. Thus, there is duplicate code between the two functions. Signed-off-by: Fenghua Yu Reviewed-by: Dave Jiang --- v3: - Since iommu_sva_find() will be removed in IOMMU and access_remote_vm() cannot be exported, the completion record copy function idxd_copy_cr() is rewritten by maintaining and finding mm in xarray and copy completion record to the mm. Please check discussion on iommu_sva_find() will be removed and access_remote_vm() cannot be exported: 1. https://lore.kernel.org/lkml/ZAjSsm4%2FPDRqViwa@nvidia.com/ 2. https://lore.kernel.org/lkml/20230306163138.587484-1-fenghua.yu@intel.com/T/#m1fc97725a0e56ea269c8bdabacee447070d51846 v2: - Define and export iommu_access_remote_vm() for IDXD driver to write completion record to user address space. This change removes patch 8 and 9 in v1 (Alistair Popple) drivers/dma/idxd/cdev.c | 127 +++++++++++++++++++++++++++++++++++++-- drivers/dma/idxd/idxd.h | 6 ++ drivers/dma/idxd/init.c | 2 + drivers/dma/idxd/sysfs.c | 1 + 4 files changed, 131 insertions(+), 5 deletions(-) diff --git a/drivers/dma/idxd/cdev.c b/drivers/dma/idxd/cdev.c index cbe29e1a6a44..17d83ecf54e9 100644 --- a/drivers/dma/idxd/cdev.c +++ b/drivers/dma/idxd/cdev.c @@ -11,7 +11,9 @@ #include #include #include +#include #include +#include #include "registers.h" #include "idxd.h" @@ -34,6 +36,7 @@ struct idxd_user_context { struct idxd_wq *wq; struct task_struct *task; unsigned int pasid; + struct mm_struct *mm; unsigned int flags; struct iommu_sva *sva; }; @@ -68,6 +71,19 @@ static inline struct idxd_wq *inode_wq(struct inode *inode) return idxd_cdev->wq; } +static void idxd_xa_pasid_remove(struct idxd_user_context *ctx) +{ + struct idxd_wq *wq = ctx->wq; + void *ptr; + + mutex_lock(&wq->uc_lock); + ptr = xa_cmpxchg(&wq->upasid_xa, ctx->pasid, ctx, NULL, GFP_KERNEL); + if (ptr != (void *)ctx) + dev_warn(&wq->idxd->pdev->dev, "xarray cmpxchg failed for pasid %u\n", + ctx->pasid); + mutex_unlock(&wq->uc_lock); +} + static int idxd_cdev_open(struct inode *inode, struct file *filp) { struct idxd_user_context *ctx; @@ -108,20 +124,26 @@ static int idxd_cdev_open(struct inode *inode, struct file *filp) pasid = iommu_sva_get_pasid(sva); if (pasid == IOMMU_PASID_INVALID) { - iommu_sva_unbind_device(sva); rc = -EINVAL; - goto failed; + goto failed_get_pasid; } ctx->sva = sva; ctx->pasid = pasid; + ctx->mm = current->mm; + + mutex_lock(&wq->uc_lock); + rc = xa_insert(&wq->upasid_xa, pasid, ctx, GFP_KERNEL); + mutex_unlock(&wq->uc_lock); + if (rc < 0) + dev_warn(dev, "PASID entry already exist in xarray.\n"); if (wq_dedicated(wq)) { rc = idxd_wq_set_pasid(wq, pasid); if (rc < 0) { iommu_sva_unbind_device(sva); dev_err(dev, "wq set pasid failed: %d\n", rc); - goto failed; + goto failed_set_pasid; } } } @@ -130,7 +152,13 @@ static int idxd_cdev_open(struct inode *inode, struct file *filp) mutex_unlock(&wq->wq_lock); return 0; - failed: +failed_set_pasid: + if (device_user_pasid_enabled(idxd)) + idxd_xa_pasid_remove(ctx); +failed_get_pasid: + if (device_user_pasid_enabled(idxd)) + iommu_sva_unbind_device(sva); +failed: mutex_unlock(&wq->wq_lock); kfree(ctx); return rc; @@ -161,8 +189,10 @@ static int idxd_cdev_release(struct inode *node, struct file *filep) } } - if (ctx->sva) + if (ctx->sva) { iommu_sva_unbind_device(ctx->sva); + idxd_xa_pasid_remove(ctx); + } kfree(ctx); mutex_lock(&wq->wq_lock); idxd_wq_put(wq); @@ -418,3 +448,90 @@ void idxd_cdev_remove(void) ida_destroy(&ictx[i].minor_ida); } } + +static int _idxd_copy_cr(struct mm_struct *mm, unsigned long addr, void *cr, + int len) +{ + void *old_cr = cr; + + if (mmap_read_lock_killable(mm)) + return 0; + + /* + * Page backing on user address is not available. Need to get the + * pages and write the completion record to the pages. + */ + while (len) { + struct vm_area_struct *vma; + struct page *page = NULL; + int bytes, ret, offset; + void *maddr; + + ret = get_user_pages_remote(mm, addr, 1, FOLL_WRITE, + &page, &vma, NULL); + if (ret <= 0) + break; + + bytes = len; + offset = addr & (PAGE_SIZE - 1); + if (bytes > PAGE_SIZE - offset) + bytes = PAGE_SIZE - offset; + + maddr = kmap_local_page(page); + copy_to_user_page(vma, page, addr, + maddr + offset, cr, bytes); + set_page_dirty_lock(page); + kunmap_local(maddr); + put_page(page); + + len -= bytes; + cr += bytes; + addr += bytes; + } + + mmap_read_unlock(mm); + + return cr - old_cr; +} + +/** + * idxd_copy_cr - copy completion record to user address space found by wq and + * PASID + * @wq: work queue + * @pasid: PASID + * @addr: user fault address to write + * @cr: completion record + * @len: number of bytes to copy + * + * Return: number of bytes copied. + */ +int idxd_copy_cr(struct idxd_wq *wq, ioasid_t pasid, unsigned long addr, + void *cr, int len) +{ + struct device *dev = &wq->idxd->pdev->dev; + struct idxd_user_context *ctx; + struct mm_struct *mm; + int copied = 0; + + mutex_lock(&wq->uc_lock); + + ctx = xa_load(&wq->upasid_xa, pasid); + if (!ctx) { + dev_warn(dev, "No user context\n"); + goto out; + } + + mm = ctx->mm; + if (!mmget_not_zero(mm)) { + dev_warn(dev, "Cannot get mm\n"); + goto out; + } + + copied = _idxd_copy_cr(mm, addr, cr, len); + mmput(mm); + +out: + mutex_unlock(&wq->uc_lock); + + return copied; +} diff --git a/drivers/dma/idxd/idxd.h b/drivers/dma/idxd/idxd.h index c5d99c179902..b3f9a12adce2 100644 --- a/drivers/dma/idxd/idxd.h +++ b/drivers/dma/idxd/idxd.h @@ -215,6 +215,10 @@ struct idxd_wq { char name[WQ_NAME_SIZE + 1]; u64 max_xfer_bytes; u32 max_batch_size; + + /* Lock to protect upasid_xa access. */ + struct mutex uc_lock; + struct xarray upasid_xa; }; struct idxd_engine { @@ -702,6 +706,8 @@ void idxd_cdev_remove(void); int idxd_cdev_get_major(struct idxd_device *idxd); int idxd_wq_add_cdev(struct idxd_wq *wq); void idxd_wq_del_cdev(struct idxd_wq *wq); +int idxd_copy_cr(struct idxd_wq *wq, ioasid_t pasid, unsigned long addr, + void *buf, int len); /* perfmon */ #if IS_ENABLED(CONFIG_INTEL_IDXD_PERFMON) diff --git a/drivers/dma/idxd/init.c b/drivers/dma/idxd/init.c index a7c98fac7a85..912753a99747 100644 --- a/drivers/dma/idxd/init.c +++ b/drivers/dma/idxd/init.c @@ -200,6 +200,8 @@ static int idxd_setup_wqs(struct idxd_device *idxd) } bitmap_copy(wq->opcap_bmap, idxd->opcap_bmap, IDXD_MAX_OPCAP_BITS); } + mutex_init(&wq->uc_lock); + xa_init(&wq->upasid_xa); idxd->wqs[i] = wq; } diff --git a/drivers/dma/idxd/sysfs.c b/drivers/dma/idxd/sysfs.c index 8b9dfa0d2b99..465d2e7627e4 100644 --- a/drivers/dma/idxd/sysfs.c +++ b/drivers/dma/idxd/sysfs.c @@ -1292,6 +1292,7 @@ static void idxd_conf_wq_release(struct device *dev) bitmap_free(wq->opcap_bmap); kfree(wq->wqcfg); + xa_destroy(&wq->upasid_xa); kfree(wq); } From patchwork Mon Mar 13 17:02:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fenghua Yu X-Patchwork-Id: 68960 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp1307398wrd; Mon, 13 Mar 2023 10:17:50 -0700 (PDT) X-Google-Smtp-Source: AK7set/+l/MuiD+3/qPAUbM6Pciw/kqK2nlqrMj10g2S5xs9ot3hjJCDbF/kyUOQbAsAxZ1Q3c1p X-Received: by 2002:aa7:9e5e:0:b0:622:844e:80b2 with SMTP id z30-20020aa79e5e000000b00622844e80b2mr5758383pfq.24.1678727870475; Mon, 13 Mar 2023 10:17:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1678727870; cv=none; d=google.com; s=arc-20160816; b=zagdOfEdqRvkXWF7qtQjYQ080sqQFpHSVVp45NXOE65DEH9T78ppEV/ZRN2dLvrTcs XzNZPy5Q6kofDAr8ktGvKjBY5MUI01iGUlvnC3Dfc6iRfvtCZR1Rg4Y8LDHPdR0zYjlC D+r7L+N1REtCCV4KahDGng0WSWHMRE6E+73qmlWNpFLsX9GtHV0hNjeg3urgvM5we4Rr NVJqyslVrvLNrsUetQfpnZI6lf361uhAkUnTfaYRfg7s9nYcbWnD11NtkNI3oG0YBbNb d0spMva8HC+iyIrJGUR4T48FidZIXkNJg3gxKXyrrYJH5FvffAlIhe3n0sHqDGGoMhDf c4Rg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=QT0ZNNwuGUr/nxvLBm+eBWLGgmfhI2Ri0Sw7QOTVgMA=; b=THuTgrRmdeL1vitY303wRW5H3Gjmy3dZUhxnIdQHLyhPVNerJyOD+2Tw7XqMuovUGH 8YNS2piTYND6Z7KVUjBbFbLrllm/IYoafGHQf+fgPlSL4kskDw1BbdvMRuuPLe/gAE8w rZIYeblpBtK1Kyt5wEvN6K53ja6svvtzqGUsKMdR1c3+RsWmBPeAoTWc0cZLZLkBqY7+ bEySCqvikI/MCu2hzRD3dPR1RxULJwoRV5Nh1cY0QDVz5nYSzbxDB4vIEL3iPj3pHnQw cdmRYGO44gdztVwkXYBR4JVLOL+LwzcjPUfaweD8F40v03QrSjai2JhM08BCcFBaQSe7 EoCw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=mSzTneJG; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id i29-20020a056a00005d00b005a911b0a4e0si6698404pfk.240.2023.03.13.10.17.38; Mon, 13 Mar 2023 10:17:50 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=mSzTneJG; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231346AbjCMRGf (ORCPT + 99 others); Mon, 13 Mar 2023 13:06:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43850 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230494AbjCMRGT (ORCPT ); Mon, 13 Mar 2023 13:06:19 -0400 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 48C447C956; Mon, 13 Mar 2023 10:05:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1678727129; x=1710263129; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ureOUccbBjiZp4q5Ed94teDiRAssgIZrA3C3TcZXraQ=; b=mSzTneJGdqsS57NZVP98VQZQ015Nwh1I/AS1S4WSj6KsOOLwCOEbrpap 8CQPYAUQBbNYIuK1lTYhfOrl41HDp2CO/ootLvPNTRC20VEG0LalRF0jo y7Pb1IwwRxbQtGeVQQuwhpUMh8btUFxWolUGvVLd6Rw0b+2qme3UDgCJt oSl9O6BVy8cmuuIvQPcKQyn2t7zfRGYL7+rnSektgKOZkO+bT/rTAypd5 0dz0MMe35ccA9CwSpqBMSIsrVR0QPjabA0wyAjcjUOA8LkHsT9YodDkxP cyIysshlOrzYreaAffS3OOoSSpe3NB7jAql/DqV9l9knse58/0L8mGFGM A==; X-IronPort-AV: E=McAfee;i="6500,9779,10648"; a="334679667" X-IronPort-AV: E=Sophos;i="5.98,257,1673942400"; d="scan'208";a="334679667" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Mar 2023 10:02:42 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10648"; a="708950925" X-IronPort-AV: E=Sophos;i="5.98,257,1673942400"; d="scan'208";a="708950925" Received: from fyu1.sc.intel.com ([172.25.103.126]) by orsmga008.jf.intel.com with ESMTP; 13 Mar 2023 10:02:41 -0700 From: Fenghua Yu To: "Vinod Koul" , "Dave Jiang" Cc: dmaengine@vger.kernel.org, "linux-kernel" , Tony Zhu , Fenghua Yu Subject: [PATCH v3 09/16] dmaengine: idxd: process user page faults for completion record Date: Mon, 13 Mar 2023 10:02:12 -0700 Message-Id: <20230313170219.1956012-10-fenghua.yu@intel.com> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20230313170219.1956012-1-fenghua.yu@intel.com> References: <20230313170219.1956012-1-fenghua.yu@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1760273755066145322?= X-GMAIL-MSGID: =?utf-8?q?1760273755066145322?= From: Dave Jiang DSA supports page fault handling through PRS. However, the DMA engine that's processing the descriptor is blocked until the PRS response is received. Other workqueues sharing the engine are also blocked. Page fault handing by the driver with PRS disabled can be used to mitigate the stalling. With PRS disabled while ATS remain enabled, DSA handles page faults on a completion record by reporting an event in the event log. In this instance, the descriptor is completed and the event log contains the completion record address and the contents of the completion record. Add support to the event log handling code to fault in the completion record and copy the content of the completion record to user memory. A bitmap is introduced to keep track of discarded event log entries. When the user process initiates ->release() of the char device, it no longer is interested in any remaining event log entries tied to the relevant wq and PASID. The driver will mark the event log entry index in the bitmap. Upon encountering the entries during processing, the event log handler will just clear the bitmap bit and skip the entry rather than attempt to process the event log entry. Tested-by: Tony Zhu Signed-off-by: Dave Jiang Co-developed-by: Fenghua Yu Signed-off-by: Fenghua Yu --- v3: - Call new function idxd_copy_cr(). v2: - Call iommu_access_remote_vm() to copy completion record to user. drivers/dma/Kconfig | 1 + drivers/dma/idxd/cdev.c | 30 +++++++++++++++ drivers/dma/idxd/device.c | 22 ++++++++++- drivers/dma/idxd/idxd.h | 2 + drivers/dma/idxd/init.c | 2 + drivers/dma/idxd/irq.c | 78 ++++++++++++++++++++++++++++++++++++--- include/uapi/linux/idxd.h | 1 + 7 files changed, 129 insertions(+), 7 deletions(-) diff --git a/drivers/dma/Kconfig b/drivers/dma/Kconfig index fb7073fc034f..c8a2d255930e 100644 --- a/drivers/dma/Kconfig +++ b/drivers/dma/Kconfig @@ -297,6 +297,7 @@ config INTEL_IDXD depends on PCI_PASID depends on SBITMAP select DMA_ENGINE + select IOMMU_SVA help Enable support for the Intel(R) data accelerators present in Intel Xeon CPU. diff --git a/drivers/dma/idxd/cdev.c b/drivers/dma/idxd/cdev.c index 17d83ecf54e9..fa71059825e2 100644 --- a/drivers/dma/idxd/cdev.c +++ b/drivers/dma/idxd/cdev.c @@ -164,6 +164,35 @@ static int idxd_cdev_open(struct inode *inode, struct file *filp) return rc; } +static void idxd_cdev_evl_drain_pasid(struct idxd_wq *wq, u32 pasid) +{ + struct idxd_device *idxd = wq->idxd; + struct idxd_evl *evl = idxd->evl; + union evl_status_reg status; + u16 h, t, size; + int ent_size = evl_ent_size(idxd); + struct __evl_entry *entry_head; + + if (!evl) + return; + + spin_lock(&evl->lock); + status.bits = ioread64(idxd->reg_base + IDXD_EVLSTATUS_OFFSET); + t = status.tail; + h = evl->head; + size = evl->size; + + while (h != t) { + entry_head = (struct __evl_entry *)(evl->log + (h * ent_size)); + if (entry_head->pasid == pasid && entry_head->wq_idx == wq->id) + set_bit(h, evl->bmap); + h = (h + 1) % size; + } + spin_unlock(&evl->lock); + + drain_workqueue(wq->wq); +} + static int idxd_cdev_release(struct inode *node, struct file *filep) { struct idxd_user_context *ctx = filep->private_data; @@ -190,6 +219,7 @@ static int idxd_cdev_release(struct inode *node, struct file *filep) } if (ctx->sva) { + idxd_cdev_evl_drain_pasid(wq, ctx->pasid); iommu_sva_unbind_device(ctx->sva); idxd_xa_pasid_remove(ctx); } diff --git a/drivers/dma/idxd/device.c b/drivers/dma/idxd/device.c index 230fe9bb56ae..fd97b2b58734 100644 --- a/drivers/dma/idxd/device.c +++ b/drivers/dma/idxd/device.c @@ -762,18 +762,29 @@ static int idxd_device_evl_setup(struct idxd_device *idxd) dma_addr_t dma_addr; int size; struct idxd_evl *evl = idxd->evl; + unsigned long *bmap; + int rc; if (!evl) return 0; size = evl_size(idxd); + + bmap = bitmap_zalloc(size, GFP_KERNEL); + if (!bmap) { + rc = -ENOMEM; + goto err_bmap; + } + /* * Address needs to be page aligned. However, dma_alloc_coherent() provides * at minimal page size aligned address. No manual alignment required. */ addr = dma_alloc_coherent(dev, size, &dma_addr, GFP_KERNEL); - if (!addr) - return -ENOMEM; + if (!addr) { + rc = -ENOMEM; + goto err_alloc; + } memset(addr, 0, size); @@ -781,6 +792,7 @@ static int idxd_device_evl_setup(struct idxd_device *idxd) evl->log = addr; evl->dma = dma_addr; evl->log_size = size; + evl->bmap = bmap; memset(&evlcfg, 0, sizeof(evlcfg)); evlcfg.bits[0] = dma_addr & GENMASK(63, 12); @@ -799,6 +811,11 @@ static int idxd_device_evl_setup(struct idxd_device *idxd) spin_unlock(&evl->lock); return 0; + +err_alloc: + bitmap_free(bmap); +err_bmap: + return rc; } static void idxd_device_evl_free(struct idxd_device *idxd) @@ -824,6 +841,7 @@ static void idxd_device_evl_free(struct idxd_device *idxd) iowrite64(0, idxd->reg_base + IDXD_EVLCFG_OFFSET + 8); dma_free_coherent(dev, evl->log_size, evl->log, evl->dma); + bitmap_free(evl->bmap); evl->log = NULL; evl->size = IDXD_EVL_SIZE_MIN; spin_unlock(&evl->lock); diff --git a/drivers/dma/idxd/idxd.h b/drivers/dma/idxd/idxd.h index b3f9a12adce2..3963c83165a6 100644 --- a/drivers/dma/idxd/idxd.h +++ b/drivers/dma/idxd/idxd.h @@ -264,6 +264,7 @@ struct idxd_driver_data { struct device_type *dev_type; int compl_size; int align; + int evl_cr_off; }; struct idxd_evl { @@ -276,6 +277,7 @@ struct idxd_evl { /* The number of entries in the event log. */ u16 size; u16 head; + unsigned long *bmap; }; struct idxd_evl_fault { diff --git a/drivers/dma/idxd/init.c b/drivers/dma/idxd/init.c index 912753a99747..be4f3676e1a6 100644 --- a/drivers/dma/idxd/init.c +++ b/drivers/dma/idxd/init.c @@ -47,6 +47,7 @@ static struct idxd_driver_data idxd_driver_data[] = { .compl_size = sizeof(struct dsa_completion_record), .align = 32, .dev_type = &dsa_device_type, + .evl_cr_off = offsetof(struct dsa_evl_entry, cr), }, [IDXD_TYPE_IAX] = { .name_prefix = "iax", @@ -54,6 +55,7 @@ static struct idxd_driver_data idxd_driver_data[] = { .compl_size = sizeof(struct iax_completion_record), .align = 64, .dev_type = &iax_device_type, + .evl_cr_off = offsetof(struct iax_evl_entry, cr), }, }; diff --git a/drivers/dma/idxd/irq.c b/drivers/dma/idxd/irq.c index 52b8b7d9db22..a428d89de077 100644 --- a/drivers/dma/idxd/irq.c +++ b/drivers/dma/idxd/irq.c @@ -7,6 +7,8 @@ #include #include #include +#include +#include #include #include "../dmaengine.h" #include "idxd.h" @@ -217,14 +219,80 @@ static void idxd_int_handle_revoke(struct work_struct *work) kfree(revoke); } -static void process_evl_entry(struct idxd_device *idxd, struct __evl_entry *entry_head) +static void idxd_evl_fault_work(struct work_struct *work) +{ + struct idxd_evl_fault *fault = container_of(work, struct idxd_evl_fault, work); + struct idxd_wq *wq = fault->wq; + struct idxd_device *idxd = wq->idxd; + struct device *dev = &idxd->pdev->dev; + struct __evl_entry *entry_head = fault->entry; + void *cr = (void *)entry_head + idxd->data->evl_cr_off; + int cr_size = idxd->data->compl_size, copied; + + switch (fault->status) { + case DSA_COMP_CRA_XLAT: + case DSA_COMP_DRAIN_EVL: + /* + * Copy completion record to fault_addr in user address space + * that is found by wq and PASID. + */ + copied = idxd_copy_cr(wq, entry_head->pasid, + entry_head->fault_addr, + cr, cr_size); + if (copied != cr_size) { + dev_err(dev, "Failed to write to completion record. (%d:%d)\n", + cr_size, copied); + } + break; + default: + dev_err(dev, "Unrecognized error code: %#x\n", + DSA_COMP_STATUS(entry_head->error)); + break; + } + + kmem_cache_free(idxd->evl_cache, fault); +} + +static void process_evl_entry(struct idxd_device *idxd, + struct __evl_entry *entry_head, unsigned int index) { struct device *dev = &idxd->pdev->dev; + struct idxd_evl *evl = idxd->evl; u8 status; - status = DSA_COMP_STATUS(entry_head->error); - dev_warn_ratelimited(dev, "Device error %#x operation: %#x fault addr: %#llx\n", - status, entry_head->operation, entry_head->fault_addr); + if (test_bit(index, evl->bmap)) { + clear_bit(index, evl->bmap); + } else { + status = DSA_COMP_STATUS(entry_head->error); + + if (status == DSA_COMP_CRA_XLAT || status == DSA_COMP_DRAIN_EVL) { + struct idxd_evl_fault *fault; + int ent_size = evl_ent_size(idxd); + + if (entry_head->rci) + dev_dbg(dev, "Completion Int Req set, ignoring!\n"); + + if (!entry_head->rcr && status == DSA_COMP_DRAIN_EVL) + return; + + fault = kmem_cache_alloc(idxd->evl_cache, GFP_ATOMIC); + if (fault) { + struct idxd_wq *wq = idxd->wqs[entry_head->wq_idx]; + + fault->wq = wq; + fault->status = status; + memcpy(&fault->entry, entry_head, ent_size); + INIT_WORK(&fault->work, idxd_evl_fault_work); + queue_work(wq->wq, &fault->work); + } else { + dev_warn(dev, "Failed to service fault work.\n"); + } + } else { + dev_warn_ratelimited(dev, "Device error %#x operation: %#x fault addr: %#llx\n", + status, entry_head->operation, + entry_head->fault_addr); + } + } } static void process_evl_entries(struct idxd_device *idxd) @@ -250,7 +318,7 @@ static void process_evl_entries(struct idxd_device *idxd) while (h != t) { entry_head = (struct __evl_entry *)(evl->log + (h * ent_size)); - process_evl_entry(idxd, entry_head); + process_evl_entry(idxd, entry_head, h); h = (h + 1) % size; } diff --git a/include/uapi/linux/idxd.h b/include/uapi/linux/idxd.h index e86199d09a91..4b584d5afd87 100644 --- a/include/uapi/linux/idxd.h +++ b/include/uapi/linux/idxd.h @@ -135,6 +135,7 @@ enum dsa_completion_status { DSA_COMP_HW_ERR1, DSA_COMP_HW_ERR_DRB, DSA_COMP_TRANSLATION_FAIL, + DSA_COMP_DRAIN_EVL = 0x26, }; enum iax_completion_status { From patchwork Mon Mar 13 17:02:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fenghua Yu X-Patchwork-Id: 68953 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp1303034wrd; Mon, 13 Mar 2023 10:09:24 -0700 (PDT) X-Google-Smtp-Source: AK7set8JHfvJjgT7dSZFRYsFKYZjcdoBZONJ/UeIJJjJ6xnEW53VUNlQhmSLXPWb44sV88Nn3iIT X-Received: by 2002:a62:79c6:0:b0:625:4b46:e019 with SMTP id u189-20020a6279c6000000b006254b46e019mr1792714pfc.9.1678727364032; Mon, 13 Mar 2023 10:09:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1678727364; cv=none; d=google.com; s=arc-20160816; b=zCKUwNPQd3sIeTPGRwB2FnjMCsQbFrhQB0xpTHmC/XPpnA/Nmpe9fp6dXHWPWc5ZZB fdNlgMnNeXs0I16aFAWd3knTZRHWUl5P1PYK0AaRBD82LXSR+WsFMnJ5d+xa786USAMd mHZ/BrfXGcGtG8EWax9zOFdDLOAcc/d59qIQSuYSwIHHGggbMSB3ON1YxQ2VFbejrEaF 5zaMSkNyPDNA3dOPQ5VSCgCz5JWSbVBqArTt8tSiU/2qUGduzzV//ImMhGXCzhHtW06z 6GVH+8jV4LbCUs4hmuGZXUJN/t79RHJtUdYHhLLivhbrvawC9bj+9yFafi4Uy22vqqXt B72Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=RB3HnA4bSYy0DDRfEDhzCMHzflg+Qx3/raJ50AvKVe8=; b=U/gDxBBnonOaR/QrYIuaXipu09CsHeq0r7W42DWlvdr0TF7Bu5HJa+0S8fWlVXlCvG Os7T+LEACuU996U1qBao4jOYRnk4wAxpU6wXhN1QSsxTAc7W3MDyzChtnpSjxdEYg1Nx N4VpstoHwArZkxlZIXnMgX6gek18545qyM0VovqoQMrn7CZYWPgUv9eYDccF0S56y+Oc m9C/02wt1pIZ0rWX3rOiVIQXyXbjBPSdohCKwBE6w8sICDeogvGJA8hs0qXeJYzlj/7x YETL7OT+QpjBU+LfvWkd6i+ioci+H7TbjL6XPlomspzctOv9HtbsT0KtYxvCTKz1vgCR NRmQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=LfZTG8vA; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 18-20020a630712000000b0050336b0b08asi6904754pgh.449.2023.03.13.10.09.07; Mon, 13 Mar 2023 10:09:24 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=LfZTG8vA; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229845AbjCMRGm (ORCPT + 99 others); Mon, 13 Mar 2023 13:06:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42252 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231339AbjCMRGU (ORCPT ); Mon, 13 Mar 2023 13:06:20 -0400 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1CF207D08C; Mon, 13 Mar 2023 10:05:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1678727131; x=1710263131; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=g/s7XMoaD0305pZc+v+dkPi6sAXzjOyWDTcLsZDgBP0=; b=LfZTG8vAr/gN4WOFkSr5S97iC2H5AYJfeFicf4I5QQybmpyRba7W+FPD Y21JJxCreE0QbjqrZGhTweG0Y0vbeLgGnYOzyuiiEOjMUzMrt0SDjtukM VNLExhHwG+5PpwXB/gEmjgibrkMnkQFm8ojgnXSsCItUhLH3Lo/iaQAFg ACySFoPsLC3NDIBIJMlVrQq/GBkH2c71luWUQhKyoqb5c9xUdcQ14SoD/ wiThRzIPAHZpMd/4B+hpvIkFqUfN3kMlo6xPJrnf5L2OoLIV3AkrPs7TJ JsXrEE6YaDU0wtWY2UEyT5zcaHa5klGz27pnPTbLId5KGP0nuEpx1f0f2 g==; X-IronPort-AV: E=McAfee;i="6500,9779,10648"; a="334679668" X-IronPort-AV: E=Sophos;i="5.98,257,1673942400"; d="scan'208";a="334679668" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Mar 2023 10:02:42 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10648"; a="708950929" X-IronPort-AV: E=Sophos;i="5.98,257,1673942400"; d="scan'208";a="708950929" Received: from fyu1.sc.intel.com ([172.25.103.126]) by orsmga008.jf.intel.com with ESMTP; 13 Mar 2023 10:02:41 -0700 From: Fenghua Yu To: "Vinod Koul" , "Dave Jiang" Cc: dmaengine@vger.kernel.org, "linux-kernel" , Tony Zhu , Fenghua Yu Subject: [PATCH v3 10/16] dmaengine: idxd: add descs_completed field for completion record Date: Mon, 13 Mar 2023 10:02:13 -0700 Message-Id: <20230313170219.1956012-11-fenghua.yu@intel.com> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20230313170219.1956012-1-fenghua.yu@intel.com> References: <20230313170219.1956012-1-fenghua.yu@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1760273224370494629?= X-GMAIL-MSGID: =?utf-8?q?1760273224370494629?= From: Dave Jiang The descs_completed field for a completion record is part of a batch descriptor completion record. It takes the same location as bytes_completed in a normal descriptor field. Add to expose to user. Tested-by: Tony Zhu Signed-off-by: Dave Jiang Co-developed-by: Fenghua Yu Signed-off-by: Fenghua Yu --- include/uapi/linux/idxd.h | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/include/uapi/linux/idxd.h b/include/uapi/linux/idxd.h index 4b584d5afd87..76ad71bf751e 100644 --- a/include/uapi/linux/idxd.h +++ b/include/uapi/linux/idxd.h @@ -314,7 +314,10 @@ struct dsa_completion_record { }; uint8_t fault_info; uint8_t rsvd; - uint32_t bytes_completed; + union { + uint32_t bytes_completed; + uint32_t descs_completed; + }; uint64_t fault_addr; union { /* common record */ From patchwork Mon Mar 13 17:02:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fenghua Yu X-Patchwork-Id: 68973 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp1309296wrd; Mon, 13 Mar 2023 10:22:09 -0700 (PDT) X-Google-Smtp-Source: AK7set+71wbgvp1Mz2cFctzBjy9FAl/ulN/qwZ+/pSsWijNfn7h6LK4h8nFjQl984e7jpu79uAPq X-Received: by 2002:a05:6a20:5481:b0:cc:c3f7:9178 with SMTP id i1-20020a056a20548100b000ccc3f79178mr42701695pzk.31.1678728129216; Mon, 13 Mar 2023 10:22:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1678728129; cv=none; d=google.com; s=arc-20160816; b=HE55tIiGbStsCKGJ0ut7DQ86TNnkir+44E/SEDDh83EwEIGbhsHR+iVZQyBinWG0Jx Be6oaQolkhbD/J9qxdfVfp9C9Orsmgmj2GNVUha/kfGNEYWonKwSYuz2LChNgEpSH8Bu jwE+A0f1WHSWTkHuFw+WCrenzXRn+vH2j66h/38ibA1++wRJoeiUR++hOVPTPenxUD6L W777noCDZfW+M2BC2rmcKUggjiR4bbe4O3f/w0g6dYuOV4YTUIeJlU9Pb57bvVKty2Lq e/R85Hbhv91gfctNVjPoWBZo080lCH2NaKSrGXwETOmnUTQrnlg1JliMkb0428f1xl1A z9Ug== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=W1zzBl5xFXi+rSXldyiUQI3G920LcVTiIzHWkNy9kuc=; b=AIgpdZQ2gvLtB9DCpMjWk5O19LFLB4VbXwIaCprlosyj8bOT3QHjHBCiY6Zrh/BeLG cFZoE/YXJIyiv1cxDahlrXvvQtW3JS3jqtY57jNrPB9zuE/rix/S1I2G/ziTcS+vd1Bw iXtmDStn9i9Cv5lmInfv/hy8ds4aCfQtRVmx0vxi4pW/QNcB3FupMpqOibYnZUhYrkEM gkPUhBrcLCmQHfWIQe9jpIbtM0KBZ21NOvO9/S4mJP+HdClk2BQNLfS4SnzYRYas7HIj QBONTdHWR5E8C/U79Nld8ji+pCBfW3F5ENZX+sAg28c7T5NFdgntGxl50cUe8/DuRJs/ PuMA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=YfG7+H8M; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id a14-20020a634d0e000000b004fc2413ec5esi14333pgb.189.2023.03.13.10.21.53; Mon, 13 Mar 2023 10:22:09 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=YfG7+H8M; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231254AbjCMRHH (ORCPT + 99 others); Mon, 13 Mar 2023 13:07:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43906 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231143AbjCMRGc (ORCPT ); Mon, 13 Mar 2023 13:06:32 -0400 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A0F0E7EA30; Mon, 13 Mar 2023 10:05:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1678727157; x=1710263157; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=6O21OYN8R4QD9eH6uwFxO2gXmvwc5fLcx9/yhaQdohw=; b=YfG7+H8MchtpTYfna+umjLumC6XG4aYFLgDAohDV/3KXgBeySc4xOjx3 Qm++S2MVlEgp5yH/qXNYykY4CGIhPpygk6qjv9JFIkTv/YUgVewUDbKXo nxv843/APhtxcLh/g39gzm3H9X+6qpsLUUVDp0jKEtxHSLFSJ4Z23FIcE xpq3J5Qxt0l8DqMAd6MyU4SeqZJwrOpKQIgLkgKD5MX3x9kGuwO84g47h LvWIiWFiLLqFJuDiPu47gq8kSnfKFWbIwQXXMRwRChAPvgPifxyaMVbDG FzGiigfjQ/d99eJVvFgL4kAHNq/IuZ7Mo71U53DakMzFioUaaLCoWCGTQ w==; X-IronPort-AV: E=McAfee;i="6500,9779,10648"; a="334679676" X-IronPort-AV: E=Sophos;i="5.98,257,1673942400"; d="scan'208";a="334679676" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Mar 2023 10:02:42 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10648"; a="708950936" X-IronPort-AV: E=Sophos;i="5.98,257,1673942400"; d="scan'208";a="708950936" Received: from fyu1.sc.intel.com ([172.25.103.126]) by orsmga008.jf.intel.com with ESMTP; 13 Mar 2023 10:02:41 -0700 From: Fenghua Yu To: "Vinod Koul" , "Dave Jiang" Cc: dmaengine@vger.kernel.org, "linux-kernel" , Tony Zhu , Fenghua Yu Subject: [PATCH v3 11/16] dmaengine: idxd: process batch descriptor completion record faults Date: Mon, 13 Mar 2023 10:02:14 -0700 Message-Id: <20230313170219.1956012-12-fenghua.yu@intel.com> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20230313170219.1956012-1-fenghua.yu@intel.com> References: <20230313170219.1956012-1-fenghua.yu@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1760274027026670508?= X-GMAIL-MSGID: =?utf-8?q?1760274027026670508?= From: Dave Jiang Add event log processing for faulting of user batch descriptor completion record. When encountering an event log entry for a page fault on a completion record, the driver is expected to do the following: 1. If the "first error in batch" bit in event log entry error info is set, discard any previously recorded errors associated with the "batch identifier". 2. Fix the page fault according to the fault address in the event log. If successful, write the completion record to the fault address in user space. 3. If an error is encountered while writing the completion record and it is associated to a descriptor in the batch, the driver associates the error with the batch identifier of the event log entry and tracks it until the event log entry for the corresponding batch desc is encountered. While processing an event log entry for a batch descriptor with error indicating that one or more descs in the batch had event log entries, the driver will do the following before writing the batch completion record: 1. If the status field of the completion record is 0x1, the driver will change it to error code 0x5 (one or more operations in batch completed with status not successful) and changes the result field to 1. 2. If the status is error code 0x6 (page fault on batch descriptor list address), change the result field to 1. 3. If status is any other value, the completion record is not changed. 4. Clear the recorded error in preparation for next batch with same batch identifier. The result field is for user software to determine whether to set the "Batch Error" flag bit in the descriptor for continuation of partial batch descriptor completion. See DSA spec 2.0 for additional information. If no error has been recorded for the batch, the batch completion record is written to user space as is. Tested-by: Tony Zhu Signed-off-by: Dave Jiang Co-developed-by: Fenghua Yu Signed-off-by: Fenghua Yu --- v3: - Call new function idxd_copy_cr(). v2: - Call iommu_access_remote_vm() to copy completion record to user. drivers/dma/idxd/idxd.h | 3 ++ drivers/dma/idxd/init.c | 4 ++ drivers/dma/idxd/irq.c | 74 ++++++++++++++++++++++++++++-------- drivers/dma/idxd/registers.h | 4 +- include/uapi/linux/idxd.h | 1 + 5 files changed, 70 insertions(+), 16 deletions(-) diff --git a/drivers/dma/idxd/idxd.h b/drivers/dma/idxd/idxd.h index 3963c83165a6..4c4baa80c731 100644 --- a/drivers/dma/idxd/idxd.h +++ b/drivers/dma/idxd/idxd.h @@ -265,6 +265,8 @@ struct idxd_driver_data { int compl_size; int align; int evl_cr_off; + int cr_status_off; + int cr_result_off; }; struct idxd_evl { @@ -278,6 +280,7 @@ struct idxd_evl { u16 size; u16 head; unsigned long *bmap; + bool batch_fail[IDXD_MAX_BATCH_IDENT]; }; struct idxd_evl_fault { diff --git a/drivers/dma/idxd/init.c b/drivers/dma/idxd/init.c index be4f3676e1a6..9b3e7f0770d1 100644 --- a/drivers/dma/idxd/init.c +++ b/drivers/dma/idxd/init.c @@ -48,6 +48,8 @@ static struct idxd_driver_data idxd_driver_data[] = { .align = 32, .dev_type = &dsa_device_type, .evl_cr_off = offsetof(struct dsa_evl_entry, cr), + .cr_status_off = offsetof(struct dsa_completion_record, status), + .cr_result_off = offsetof(struct dsa_completion_record, result), }, [IDXD_TYPE_IAX] = { .name_prefix = "iax", @@ -56,6 +58,8 @@ static struct idxd_driver_data idxd_driver_data[] = { .align = 64, .dev_type = &iax_device_type, .evl_cr_off = offsetof(struct iax_evl_entry, cr), + .cr_status_off = offsetof(struct iax_completion_record, status), + .cr_result_off = offsetof(struct iax_completion_record, error_code), }, }; diff --git a/drivers/dma/idxd/irq.c b/drivers/dma/idxd/irq.c index a428d89de077..155c1970497f 100644 --- a/drivers/dma/idxd/irq.c +++ b/drivers/dma/idxd/irq.c @@ -225,28 +225,71 @@ static void idxd_evl_fault_work(struct work_struct *work) struct idxd_wq *wq = fault->wq; struct idxd_device *idxd = wq->idxd; struct device *dev = &idxd->pdev->dev; + struct idxd_evl *evl = idxd->evl; struct __evl_entry *entry_head = fault->entry; void *cr = (void *)entry_head + idxd->data->evl_cr_off; - int cr_size = idxd->data->compl_size, copied; + int cr_size = idxd->data->compl_size; + u8 *status = (u8 *)cr + idxd->data->cr_status_off; + u8 *result = (u8 *)cr + idxd->data->cr_result_off; + int copied, copy_size; + bool *bf; switch (fault->status) { case DSA_COMP_CRA_XLAT: - case DSA_COMP_DRAIN_EVL: - /* - * Copy completion record to fault_addr in user address space - * that is found by wq and PASID. - */ - copied = idxd_copy_cr(wq, entry_head->pasid, - entry_head->fault_addr, - cr, cr_size); - if (copied != cr_size) { - dev_err(dev, "Failed to write to completion record. (%d:%d)\n", - cr_size, copied); + if (entry_head->batch && entry_head->first_err_in_batch) + evl->batch_fail[entry_head->batch_id] = false; + + copy_size = cr_size; + break; + case DSA_COMP_BATCH_EVL_ERR: + bf = &evl->batch_fail[entry_head->batch_id]; + + copy_size = entry_head->rcr || *bf ? cr_size : 0; + if (*bf) { + if (*status == DSA_COMP_SUCCESS) + *status = DSA_COMP_BATCH_FAIL; + *result = 1; + *bf = false; } break; + case DSA_COMP_DRAIN_EVL: + copy_size = cr_size; + break; default: - dev_err(dev, "Unrecognized error code: %#x\n", - DSA_COMP_STATUS(entry_head->error)); + copy_size = 0; + dev_err(dev, "Unrecognized error code: %#x\n", fault->status); + break; + } + + if (copy_size == 0) + return; + + /* + * Copy completion record to fault_addr in user address space + * that is found by wq and PASID. + */ + copied = idxd_copy_cr(wq, entry_head->pasid, entry_head->fault_addr, + cr, copy_size); + + switch (fault->status) { + case DSA_COMP_CRA_XLAT: + if (copied != copy_size) { + dev_err(dev, "Failed to write to completion record: (%d:%d)\n", + copy_size, copied); + if (entry_head->batch) + evl->batch_fail[entry_head->batch_id] = true; + } + break; + case DSA_COMP_BATCH_EVL_ERR: + if (copied != copy_size) { + dev_err(dev, "Failed to write to batch completion record: (%d:%d)\n", + copy_size, copied); + } + break; + case DSA_COMP_DRAIN_EVL: + if (copied != copy_size) + dev_err(dev, "Failed to write to drain completion record: (%d:%d)\n", + copy_size, copied); break; } @@ -265,7 +308,8 @@ static void process_evl_entry(struct idxd_device *idxd, } else { status = DSA_COMP_STATUS(entry_head->error); - if (status == DSA_COMP_CRA_XLAT || status == DSA_COMP_DRAIN_EVL) { + if (status == DSA_COMP_CRA_XLAT || status == DSA_COMP_DRAIN_EVL || + status == DSA_COMP_BATCH_EVL_ERR) { struct idxd_evl_fault *fault; int ent_size = evl_ent_size(idxd); diff --git a/drivers/dma/idxd/registers.h b/drivers/dma/idxd/registers.h index 148db94f9373..9f3959d001b6 100644 --- a/drivers/dma/idxd/registers.h +++ b/drivers/dma/idxd/registers.h @@ -35,7 +35,7 @@ union gen_cap_reg { u64 drain_readback:1; u64 rsvd2:3; u64 evl_support:2; - u64 rsvd4:1; + u64 batch_continuation:1; u64 max_xfer_shift:5; u64 max_batch_shift:4; u64 max_ims_mult:6; @@ -577,6 +577,8 @@ union evl_status_reg { u64 bits; } __packed; +#define IDXD_MAX_BATCH_IDENT 256 + struct __evl_entry { u64 rsvd:2; u64 desc_valid:1; diff --git a/include/uapi/linux/idxd.h b/include/uapi/linux/idxd.h index 76ad71bf751e..606b52e88ce3 100644 --- a/include/uapi/linux/idxd.h +++ b/include/uapi/linux/idxd.h @@ -136,6 +136,7 @@ enum dsa_completion_status { DSA_COMP_HW_ERR_DRB, DSA_COMP_TRANSLATION_FAIL, DSA_COMP_DRAIN_EVL = 0x26, + DSA_COMP_BATCH_EVL_ERR, }; enum iax_completion_status { From patchwork Mon Mar 13 17:02:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fenghua Yu X-Patchwork-Id: 68979 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp1310520wrd; Mon, 13 Mar 2023 10:24:49 -0700 (PDT) X-Google-Smtp-Source: AK7set+tBnGQ5iKXukBIOumryIEjGq0IJZgqABakRm6ehTgtDyXDNJWRNRgtlFqd+GimQWUmp2yi X-Received: by 2002:a62:1b14:0:b0:622:ece1:35d3 with SMTP id b20-20020a621b14000000b00622ece135d3mr6370107pfb.5.1678728289621; Mon, 13 Mar 2023 10:24:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1678728289; cv=none; d=google.com; s=arc-20160816; b=eg8KD3JeQkdniw8p2gmhUonDwxky0jHfQsduI4qJj2p1ipdr8DwyVQ1QrObSJsPkEz TgVPxAbiCuMc0V60o8x2A5EFGVu6IKKQhyKTo80fC2RxikVZqcDI3U1zb5GDW7n/mQTZ M/5hLc6iF4jpQGaAWuuyAwbARRHZ6dPrwCJTUDbnLzb5fthdfdcjpbHiedl0enamd2Db sv0xxO0tOqc6/aPesZoEft9eBPlLKGF5+QyUzsYCpuf3t5mVBN4LKUwvmupCEJrWg7Ki GBCRKlNbnxYXrazbpsqR2osTI0YHy4B55riDtMsqKL5zOLyY8aVky5sXTmQ3JdETozw2 Btbw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=1qV6lq1P9nJzNJsLVkzb+/ibL7CT//TY2t3k6pe037I=; b=QZSEGY3W2mBbpnnbKKsvKDY+T9WSc6whOa/uStHhJI0SbgoaiidF33rvMTwKW8gSQQ /VZr1YV8Neb64FSonG1cb/BCNtWRpZjxAehGmd6dIjtghWTZTfDL1hOqhhhbGjgQoSJc PRUghBDQmDexw8HGeTYtSBlqeDm5NLSH3zgoqteSNqgK+ashcyAcer5MK8p/CN9IBkcc TNPhjvQVzZa0XSzMkP42PoQxtyAVyK3yM1lBAq1gyvGvFh60xCWMFUrKZ22pnw+FspO/ 5eerQdUf3PyDeM/P2WYvCYhuHAP4n8g4E2yXIKRhY10xmYbPPhSvPJdFvX2Qz+KKbxBm JeNg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=HHFKXR6J; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id a19-20020aa794b3000000b005939fff6d47si6817766pfl.326.2023.03.13.10.24.34; Mon, 13 Mar 2023 10:24:49 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=HHFKXR6J; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230261AbjCMRGr (ORCPT + 99 others); Mon, 13 Mar 2023 13:06:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42400 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231342AbjCMRGU (ORCPT ); Mon, 13 Mar 2023 13:06:20 -0400 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BBAAD76F52; Mon, 13 Mar 2023 10:05:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1678727131; x=1710263131; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=YcNFbGwy45YWCrz1Qq4i605c5vCVJqCsUKnpbaNY/s4=; b=HHFKXR6J2st4i54R1b05GszSc0kkVNOqjyqDtMeSqyO+88YGkRGjGrkT ymFJeuRLzLsO4IhfJ5jeXdXwHgUP1uNJM5FBy8URXAzd9gNtH7qn2rKTl sVPPeJaqoLB+XZ1gRKh83iVO7xP8+czzgOYSOfIxwWauKRbTqlCUHh3Mb IV03y0Ozr7xss+iuzTcg1fwksS+x9aaHRbn6Wru8TD1PfGMqudyviSfV3 ayg7c4nlSpBLCFSJqaZjZpjS/qbW+J6u3Cu6+EhXNvDgtxH5XgdnGnjFM DVIAN6ClRKu9c6to2sI6KH9YGl7lwdFRZM9ihFXFROLK6M9V6i2tdDaEZ A==; X-IronPort-AV: E=McAfee;i="6500,9779,10648"; a="334679677" X-IronPort-AV: E=Sophos;i="5.98,257,1673942400"; d="scan'208";a="334679677" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Mar 2023 10:02:42 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10648"; a="708950942" X-IronPort-AV: E=Sophos;i="5.98,257,1673942400"; d="scan'208";a="708950942" Received: from fyu1.sc.intel.com ([172.25.103.126]) by orsmga008.jf.intel.com with ESMTP; 13 Mar 2023 10:02:42 -0700 From: Fenghua Yu To: "Vinod Koul" , "Dave Jiang" Cc: dmaengine@vger.kernel.org, "linux-kernel" , Tony Zhu , Fenghua Yu Subject: [PATCH v3 12/16] dmaengine: idxd: add per file user counters for completion record faults Date: Mon, 13 Mar 2023 10:02:15 -0700 Message-Id: <20230313170219.1956012-13-fenghua.yu@intel.com> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20230313170219.1956012-1-fenghua.yu@intel.com> References: <20230313170219.1956012-1-fenghua.yu@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1760274195108181951?= X-GMAIL-MSGID: =?utf-8?q?1760274195108181951?= From: Dave Jiang Add counters per opened file for the char device in order to keep track how many completion record faults occurred and how many of those faults failed the writeback by the driver after attempt to fault in the page. The counters are managed by xarray that associates the PASID with struct idxd_user_context. Tested-by: Tony Zhu Signed-off-by: Dave Jiang Co-developed-by: Fenghua Yu Signed-off-by: Fenghua Yu --- v3: - Move majority of the xarry code to patch 8 which implements new function idxd_copy_cr() since the function needs the xarry to maintain and find mm by wq and PASID. Only keep the user counters related xarry code here. drivers/dma/idxd/cdev.c | 18 ++++++++++++++++++ drivers/dma/idxd/idxd.h | 7 +++++++ drivers/dma/idxd/irq.c | 4 ++++ 3 files changed, 29 insertions(+) diff --git a/drivers/dma/idxd/cdev.c b/drivers/dma/idxd/cdev.c index fa71059825e2..6936c4331abd 100644 --- a/drivers/dma/idxd/cdev.c +++ b/drivers/dma/idxd/cdev.c @@ -39,6 +39,7 @@ struct idxd_user_context { struct mm_struct *mm; unsigned int flags; struct iommu_sva *sva; + u64 counters[COUNTER_MAX]; }; static void idxd_cdev_dev_release(struct device *dev) @@ -84,6 +85,23 @@ static void idxd_xa_pasid_remove(struct idxd_user_context *ctx) mutex_unlock(&wq->uc_lock); } +void idxd_user_counter_increment(struct idxd_wq *wq, u32 pasid, int index) +{ + struct idxd_user_context *ctx; + + if (index >= COUNTER_MAX) + return; + + mutex_lock(&wq->uc_lock); + ctx = xa_load(&wq->upasid_xa, pasid); + if (!ctx) { + mutex_unlock(&wq->uc_lock); + return; + } + ctx->counters[index]++; + mutex_unlock(&wq->uc_lock); +} + static int idxd_cdev_open(struct inode *inode, struct file *filp) { struct idxd_user_context *ctx; diff --git a/drivers/dma/idxd/idxd.h b/drivers/dma/idxd/idxd.h index 4c4baa80c731..9fb26d017285 100644 --- a/drivers/dma/idxd/idxd.h +++ b/drivers/dma/idxd/idxd.h @@ -127,6 +127,12 @@ struct idxd_pmu { #define IDXD_MAX_PRIORITY 0xf +enum { + COUNTER_FAULTS = 0, + COUNTER_FAULT_FAILS, + COUNTER_MAX +}; + enum idxd_wq_state { IDXD_WQ_DISABLED = 0, IDXD_WQ_ENABLED, @@ -713,6 +719,7 @@ int idxd_wq_add_cdev(struct idxd_wq *wq); void idxd_wq_del_cdev(struct idxd_wq *wq); int idxd_copy_cr(struct idxd_wq *wq, ioasid_t pasid, unsigned long addr, void *buf, int len); +void idxd_user_counter_increment(struct idxd_wq *wq, u32 pasid, int index); /* perfmon */ #if IS_ENABLED(CONFIG_INTEL_IDXD_PERFMON) diff --git a/drivers/dma/idxd/irq.c b/drivers/dma/idxd/irq.c index 155c1970497f..732234f51972 100644 --- a/drivers/dma/idxd/irq.c +++ b/drivers/dma/idxd/irq.c @@ -240,6 +240,7 @@ static void idxd_evl_fault_work(struct work_struct *work) evl->batch_fail[entry_head->batch_id] = false; copy_size = cr_size; + idxd_user_counter_increment(wq, entry_head->pasid, COUNTER_FAULTS); break; case DSA_COMP_BATCH_EVL_ERR: bf = &evl->batch_fail[entry_head->batch_id]; @@ -251,6 +252,7 @@ static void idxd_evl_fault_work(struct work_struct *work) *result = 1; *bf = false; } + idxd_user_counter_increment(wq, entry_head->pasid, COUNTER_FAULTS); break; case DSA_COMP_DRAIN_EVL: copy_size = cr_size; @@ -274,6 +276,7 @@ static void idxd_evl_fault_work(struct work_struct *work) switch (fault->status) { case DSA_COMP_CRA_XLAT: if (copied != copy_size) { + idxd_user_counter_increment(wq, entry_head->pasid, COUNTER_FAULT_FAILS); dev_err(dev, "Failed to write to completion record: (%d:%d)\n", copy_size, copied); if (entry_head->batch) @@ -282,6 +285,7 @@ static void idxd_evl_fault_work(struct work_struct *work) break; case DSA_COMP_BATCH_EVL_ERR: if (copied != copy_size) { + idxd_user_counter_increment(wq, entry_head->pasid, COUNTER_FAULT_FAILS); dev_err(dev, "Failed to write to batch completion record: (%d:%d)\n", copy_size, copied); } From patchwork Mon Mar 13 17:02:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fenghua Yu X-Patchwork-Id: 68956 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp1306311wrd; Mon, 13 Mar 2023 10:15:47 -0700 (PDT) X-Google-Smtp-Source: AK7set8IOAOGFpk4BeTLldNpWS7UYqMX4njIuDgSQPOvU8iqhQ+hxAnWvyoj/J0hA+Jhh5lxRE2I X-Received: by 2002:a17:902:c948:b0:19a:a4f3:6d4c with SMTP id i8-20020a170902c94800b0019aa4f36d4cmr41456641pla.67.1678727747346; Mon, 13 Mar 2023 10:15:47 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1678727747; cv=none; d=google.com; s=arc-20160816; b=SDU9k9IS4TNYAO8kNTRGUEaTh7h9vaU6H7axLoxG7jUvmZA+29Pp9i6XTMNo1rVA6F piYnsuA9kM/m7XFIIk1m1lbJmRH01Qww9lllNVJOb577NAcxazkZPedU5w6d8AzcfJJr Ik3b3uuoLM92XO4s9BOLtRkTxC/8K3V6iNXz7YsUqS9RwODqj7vjfZ579kyrOQYxAXo+ ypsLAymBhjnHKlyRFWfUzNcz1obnA9EoeLhVdHp/q3Ll45M05fi6tTYleRxGXpNk0gNP IwCP7P/PdgQinmsQmUuvabCTSm4Eo0wYiVGJHeDks8C1qMPZLsosBw1qPqceQMx+68HD dQpg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=nQkI6CEUdv44T48TQ/8OxCF4SAhjy/B+kerO88oyHSA=; b=vA1PQTtIkFRsPzrIRfk2BWpajvH+3htp1wkk/yAdthJ4dFZH4kR0ElOxmcNI4jqdQ+ N+8FxXRBwKlNLXxpB4x01TZoCC1zgwr2OjP4mKQo8Zlh1Ahzfcc3uSdu5DQVKxt0VD3i 2SiGWUJmKHZWJ8GLfF27hDG6jfZPW7r/C+dSbMTjd+oIQV/xWFHqT7Ak2TxDAdVZWSd/ b1AwTK4VPKka1e7PRAUHWX1J4K+KQvOe2XZx5Nt4a2bqIxrN5iR8R1P3uk48Fo1VyAjy /nxM6/Lm0n02VkwGoXm/ak1BhjWSFDLtaWHdhIZlVEMPMpewCGeWFKxvEF5ouzXn/1hc QvDQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=SxgrhVnz; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id le4-20020a170902fb0400b001a055028d05si231055plb.85.2023.03.13.10.15.34; Mon, 13 Mar 2023 10:15:47 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=SxgrhVnz; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231363AbjCMRHK (ORCPT + 99 others); Mon, 13 Mar 2023 13:07:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43972 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231438AbjCMRGe (ORCPT ); Mon, 13 Mar 2023 13:06:34 -0400 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 942E57F014; Mon, 13 Mar 2023 10:06:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1678727162; x=1710263162; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=kAfNykF7X4GytuzebLQxlRbdCHnFZMTfWhJOhuRl4LQ=; b=SxgrhVnziHHxVXBeuMMmdmq6g5ymZ/zoZbwCy+suUpKFAgmtVXNB7tPB O9Mi6XQGWChQdlK8QpsWmp+znWEtiz9m5hdvoayM8ODYxFxVHOjPbLtMb nzeBkwr/83jk2Zi3Vwq3Q5Xp5V6GSuRTXGeZG10GLkm8rMEbNwUff0lyW nXwtcFCN6AnOJTg0CBw8MTBUufTv5+h/I7TF4JoqEqdfjWCUBUd5/tKTL czEjio+II1de/kDlCoiEtgMj5Yk5woSeGfucT58WK8qH8QqTprN3ZMWEX qf2uV6q18miR63xhbHqYceVzyY6xlKWx8V3aYo5F82nl+MeTumOXxm4d1 A==; X-IronPort-AV: E=McAfee;i="6500,9779,10648"; a="334679680" X-IronPort-AV: E=Sophos;i="5.98,257,1673942400"; d="scan'208";a="334679680" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Mar 2023 10:02:42 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10648"; a="708950947" X-IronPort-AV: E=Sophos;i="5.98,257,1673942400"; d="scan'208";a="708950947" Received: from fyu1.sc.intel.com ([172.25.103.126]) by orsmga008.jf.intel.com with ESMTP; 13 Mar 2023 10:02:42 -0700 From: Fenghua Yu To: "Vinod Koul" , "Dave Jiang" Cc: dmaengine@vger.kernel.org, "linux-kernel" , Tony Zhu , Fenghua Yu Subject: [PATCH v3 13/16] dmaengine: idxd: add a device to represent the file opened Date: Mon, 13 Mar 2023 10:02:16 -0700 Message-Id: <20230313170219.1956012-14-fenghua.yu@intel.com> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20230313170219.1956012-1-fenghua.yu@intel.com> References: <20230313170219.1956012-1-fenghua.yu@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1760273626180101054?= X-GMAIL-MSGID: =?utf-8?q?1760273626180101054?= From: Dave Jiang Embed a struct device for the user file context in order to export sysfs attributes related with the opened file. Tie the lifetime of the file context to the device. The sysfs entry will be added under the char device. Tested-by: Tony Zhu Signed-off-by: Dave Jiang Co-developed-by: Fenghua Yu Signed-off-by: Fenghua Yu --- drivers/dma/idxd/cdev.c | 119 ++++++++++++++++++++++++++++++++-------- drivers/dma/idxd/idxd.h | 2 + 2 files changed, 97 insertions(+), 24 deletions(-) diff --git a/drivers/dma/idxd/cdev.c b/drivers/dma/idxd/cdev.c index 6936c4331abd..7b35e13f4ad6 100644 --- a/drivers/dma/idxd/cdev.c +++ b/drivers/dma/idxd/cdev.c @@ -23,6 +23,13 @@ struct idxd_cdev_context { struct ida minor_ida; }; +/* + * Since user file names are global in DSA devices, define their ida's as + * global to avoid conflict file names. + */ +static DEFINE_IDA(file_ida); +static DEFINE_MUTEX(ida_lock); + /* * ictx is an array based off of accelerator types. enum idxd_type * is used as index @@ -39,7 +46,60 @@ struct idxd_user_context { struct mm_struct *mm; unsigned int flags; struct iommu_sva *sva; + struct idxd_dev idxd_dev; u64 counters[COUNTER_MAX]; + int id; +}; + +static void idxd_cdev_evl_drain_pasid(struct idxd_wq *wq, u32 pasid); +static void idxd_xa_pasid_remove(struct idxd_user_context *ctx); + +static inline struct idxd_user_context *dev_to_uctx(struct device *dev) +{ + struct idxd_dev *idxd_dev = confdev_to_idxd_dev(dev); + + return container_of(idxd_dev, struct idxd_user_context, idxd_dev); +} + +static void idxd_file_dev_release(struct device *dev) +{ + struct idxd_user_context *ctx = dev_to_uctx(dev); + struct idxd_wq *wq = ctx->wq; + struct idxd_device *idxd = wq->idxd; + int rc; + + mutex_lock(&ida_lock); + ida_free(&file_ida, ctx->id); + mutex_unlock(&ida_lock); + + /* Wait for in-flight operations to complete. */ + if (wq_shared(wq)) { + idxd_device_drain_pasid(idxd, ctx->pasid); + } else { + if (device_user_pasid_enabled(idxd)) { + /* The wq disable in the disable pasid function will drain the wq */ + rc = idxd_wq_disable_pasid(wq); + if (rc < 0) + dev_err(dev, "wq disable pasid failed.\n"); + } else { + idxd_wq_drain(wq); + } + } + + if (ctx->sva) { + idxd_cdev_evl_drain_pasid(wq, ctx->pasid); + iommu_sva_unbind_device(ctx->sva); + idxd_xa_pasid_remove(ctx); + } + kfree(ctx); + mutex_lock(&wq->wq_lock); + idxd_wq_put(wq); + mutex_unlock(&wq->wq_lock); +} + +static struct device_type idxd_cdev_file_type = { + .name = "idxd_file", + .release = idxd_file_dev_release, }; static void idxd_cdev_dev_release(struct device *dev) @@ -107,10 +167,11 @@ static int idxd_cdev_open(struct inode *inode, struct file *filp) struct idxd_user_context *ctx; struct idxd_device *idxd; struct idxd_wq *wq; - struct device *dev; + struct device *dev, *fdev; int rc = 0; struct iommu_sva *sva; unsigned int pasid; + struct idxd_cdev *idxd_cdev; wq = inode_wq(inode); idxd = wq->idxd; @@ -166,10 +227,41 @@ static int idxd_cdev_open(struct inode *inode, struct file *filp) } } + idxd_cdev = wq->idxd_cdev; + mutex_lock(&ida_lock); + ctx->id = ida_alloc(&file_ida, GFP_KERNEL); + mutex_unlock(&ida_lock); + if (ctx->id < 0) { + dev_warn(dev, "ida alloc failure\n"); + goto failed_ida; + } + ctx->idxd_dev.type = IDXD_DEV_CDEV_FILE; + fdev = user_ctx_dev(ctx); + device_initialize(fdev); + fdev->parent = cdev_dev(idxd_cdev); + fdev->bus = &dsa_bus_type; + fdev->type = &idxd_cdev_file_type; + + rc = dev_set_name(fdev, "file%d", ctx->id); + if (rc < 0) { + dev_warn(dev, "set name failure\n"); + goto failed_dev_name; + } + + rc = device_add(fdev); + if (rc < 0) { + dev_warn(dev, "file device add failure\n"); + goto failed_dev_add; + } + idxd_wq_get(wq); mutex_unlock(&wq->wq_lock); return 0; +failed_dev_add: +failed_dev_name: + put_device(fdev); +failed_ida: failed_set_pasid: if (device_user_pasid_enabled(idxd)) idxd_xa_pasid_remove(ctx); @@ -217,34 +309,12 @@ static int idxd_cdev_release(struct inode *node, struct file *filep) struct idxd_wq *wq = ctx->wq; struct idxd_device *idxd = wq->idxd; struct device *dev = &idxd->pdev->dev; - int rc; dev_dbg(dev, "%s called\n", __func__); filep->private_data = NULL; - /* Wait for in-flight operations to complete. */ - if (wq_shared(wq)) { - idxd_device_drain_pasid(idxd, ctx->pasid); - } else { - if (device_user_pasid_enabled(idxd)) { - /* The wq disable in the disable pasid function will drain the wq */ - rc = idxd_wq_disable_pasid(wq); - if (rc < 0) - dev_err(dev, "wq disable pasid failed.\n"); - } else { - idxd_wq_drain(wq); - } - } + device_unregister(user_ctx_dev(ctx)); - if (ctx->sva) { - idxd_cdev_evl_drain_pasid(wq, ctx->pasid); - iommu_sva_unbind_device(ctx->sva); - idxd_xa_pasid_remove(ctx); - } - kfree(ctx); - mutex_lock(&wq->wq_lock); - idxd_wq_put(wq); - mutex_unlock(&wq->wq_lock); return 0; } @@ -375,6 +445,7 @@ void idxd_wq_del_cdev(struct idxd_wq *wq) struct idxd_cdev *idxd_cdev; idxd_cdev = wq->idxd_cdev; + ida_destroy(&file_ida); wq->idxd_cdev = NULL; cdev_device_del(&idxd_cdev->cdev, cdev_dev(idxd_cdev)); put_device(cdev_dev(idxd_cdev)); diff --git a/drivers/dma/idxd/idxd.h b/drivers/dma/idxd/idxd.h index 9fb26d017285..bd544eb2ddcb 100644 --- a/drivers/dma/idxd/idxd.h +++ b/drivers/dma/idxd/idxd.h @@ -32,6 +32,7 @@ enum idxd_dev_type { IDXD_DEV_GROUP, IDXD_DEV_ENGINE, IDXD_DEV_CDEV, + IDXD_DEV_CDEV_FILE, IDXD_DEV_MAX_TYPE, }; @@ -405,6 +406,7 @@ enum idxd_completion_status { #define engine_confdev(engine) &engine->idxd_dev.conf_dev #define group_confdev(group) &group->idxd_dev.conf_dev #define cdev_dev(cdev) &cdev->idxd_dev.conf_dev +#define user_ctx_dev(ctx) (&(ctx)->idxd_dev.conf_dev) #define confdev_to_idxd_dev(dev) container_of(dev, struct idxd_dev, conf_dev) #define idxd_dev_to_idxd(idxd_dev) container_of(idxd_dev, struct idxd_device, idxd_dev) From patchwork Mon Mar 13 17:02:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fenghua Yu X-Patchwork-Id: 68962 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp1307405wrd; Mon, 13 Mar 2023 10:17:51 -0700 (PDT) X-Google-Smtp-Source: AK7set8BqpkcCpOQV1WKIwdn3aXGuP2MfsXjiHzoqZO4GAbjUQVh8CX39MUOE55cS9FRHzIDf3/8 X-Received: by 2002:a05:6a20:12c5:b0:cd:9654:d97d with SMTP id v5-20020a056a2012c500b000cd9654d97dmr36555187pzg.57.1678727871463; Mon, 13 Mar 2023 10:17:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1678727871; cv=none; d=google.com; s=arc-20160816; b=J9CdC/PzE9hmawlZ0GJYcWtzHlp7yVK+hpdcykjGjfd+P/1bHnHlMcPEz1SNeOvhhJ 6/FlEfSJ6WEEGlh6zMF+bE5fMmF1trEt+d6XfLvXHXSraSo7p3dQtEJknd9Y2SLcBgIn TNj+xnKu7U5/bGhPtt9GBr6ZpxNM2+Y5fHy4myptQFr2UsVJnW4n5Dw772D6fMewDqB5 IU5VNk20Aoeq8g8Y7VgKJQ3XqxlHf09Ne9x1HJdmmNf4v3QVhAFIxxSEHu4kx1OHxyWX MqiZ2b+W+6QA47mnnwXJuwNUd/db6m9m14jQZuV4tYesQMCNbmErBcgRsPi5glB2SW1q EJDA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=a4bJc38BxUvwmIRS7xdgjHEWFFfVXUtNx0gdh8U01rg=; b=pDHMrKXuey1ISkdp8Ar/FgzdklF+bSb9M0jenFgxZhdbOklnXy8PbiUS8chEfWP9Me oXp1PdlhNRIojuylN7cewvodlYq6EfGiggW1op6n9YF/Wf4+EXDqcJUY8SjRLFgP6c+s vMQihX8Frfj1QFvLl7apETz4CWooAXjnxW5NQhixYWLEJUK0Crrl8I25dh74tMhJLtmC tN+7o4cGJU2HGEHNANU8XSd5lf45EkMo9UPdhaORm+1edmYC162U42lXRk3l2OmFU8fz c2PPJa2aI5H5BSDLBhhk2RVh0cfyPEWCVITUv7OBoQLEi4y76CDcMHAaft8SbziNEHK8 cUBA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=IL10LqcN; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id t28-20020a63535c000000b0050017092042si7007348pgl.383.2023.03.13.10.17.38; Mon, 13 Mar 2023 10:17:51 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=IL10LqcN; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231434AbjCMRHS (ORCPT + 99 others); Mon, 13 Mar 2023 13:07:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44008 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231470AbjCMRGh (ORCPT ); Mon, 13 Mar 2023 13:06:37 -0400 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 790487E896; Mon, 13 Mar 2023 10:06:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1678727164; x=1710263164; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=GpOFglttVmd6ywS/di6du3VEP74eX4CmMmPlI9HOlM0=; b=IL10LqcNH5aSU03/bbdjz89DPUQdhfhxkZ55mLq1rLpvO+w2CaKr5Lgj wcCvKVsPAX7U8UxqYn2wQY9eQxyiHEZYROIi5cX4gwquSK6htNxuaQIN/ 0/W0GlWgJh/SnraT5MYdddlczePPKSpOUlsKci+yhGP6GjA0Avc1S0yB2 LzkbKTtihV5iaiztSdFFZJan2Kgn+zvz1EdzAar6fu6wPAcTZ82tM1w2C sRrsGfqe2Rcbe844FGYj/iXZfglvsu+TRL85lEXXs9wtXitvDI9Uw4J9g bgz1a22xjCEaUxMBlsVFRKmOrteXiB0rDayFIrxb/agJKQnZxeXJ3PXYb g==; X-IronPort-AV: E=McAfee;i="6500,9779,10648"; a="334679690" X-IronPort-AV: E=Sophos;i="5.98,257,1673942400"; d="scan'208";a="334679690" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Mar 2023 10:02:43 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10648"; a="708950953" X-IronPort-AV: E=Sophos;i="5.98,257,1673942400"; d="scan'208";a="708950953" Received: from fyu1.sc.intel.com ([172.25.103.126]) by orsmga008.jf.intel.com with ESMTP; 13 Mar 2023 10:02:42 -0700 From: Fenghua Yu To: "Vinod Koul" , "Dave Jiang" Cc: dmaengine@vger.kernel.org, "linux-kernel" , Tony Zhu , Fenghua Yu Subject: [PATCH v3 14/16] dmaengine: idxd: expose fault counters to sysfs Date: Mon, 13 Mar 2023 10:02:17 -0700 Message-Id: <20230313170219.1956012-15-fenghua.yu@intel.com> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20230313170219.1956012-1-fenghua.yu@intel.com> References: <20230313170219.1956012-1-fenghua.yu@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1760273756063944043?= X-GMAIL-MSGID: =?utf-8?q?1760273756063944043?= From: Dave Jiang Expose cr_faults and cr_fault_failures counters to the user space. This allows a user app to keep track of how many fault the application is causing with the completion record (CR) and also the number of failures of the CR writeback. Having a high number of cr_fault_failures is bad as the app is submitting descriptors with the CR addresses that are bad. User monitoring daemon may want to consider killing the application as it may be malicious and attempting to flood the device event log. Tested-by: Tony Zhu Signed-off-by: Dave Jiang Co-developed-by: Fenghua Yu Signed-off-by: Fenghua Yu --- .../ABI/stable/sysfs-driver-dma-idxd | 17 +++++++ drivers/dma/idxd/cdev.c | 46 +++++++++++++++++++ 2 files changed, 63 insertions(+) diff --git a/Documentation/ABI/stable/sysfs-driver-dma-idxd b/Documentation/ABI/stable/sysfs-driver-dma-idxd index e01916611452..73ab86196a41 100644 --- a/Documentation/ABI/stable/sysfs-driver-dma-idxd +++ b/Documentation/ABI/stable/sysfs-driver-dma-idxd @@ -318,3 +318,20 @@ Description: Allows control of the number of batch descriptors that can be 1 (1/2 of max value), 2 (1/4 of the max value), and 3 (1/8 of the max value). It's visible only on platforms that support the capability. + +What: /sys/bus/dsa/devices/wq./dsa\!wq./file/cr_faults +Date: Sept 14, 2022 +KernelVersion: 6.4.0 +Contact: dmaengine@vger.kernel.org +Description: Show the number of Completion Record (CR) faults this application + has caused. + +What: /sys/bus/dsa/devices/wq./dsa\!wq./file/cr_fault_failures +Date: Sept 14, 2022 +KernelVersion: 6.4.0 +Contact: dmaengine@vger.kernel.org +Description: Show the number of Completion Record (CR) faults failures that this + application has caused. The failure counter is incremented when the + driver cannot fault in the address for the CR. Typically this is caused + by a bad address programmed in the submitted descriptor or a malicious + submitter is using bad CR address on purpose. diff --git a/drivers/dma/idxd/cdev.c b/drivers/dma/idxd/cdev.c index 7b35e13f4ad6..d93d26834dc7 100644 --- a/drivers/dma/idxd/cdev.c +++ b/drivers/dma/idxd/cdev.c @@ -61,6 +61,51 @@ static inline struct idxd_user_context *dev_to_uctx(struct device *dev) return container_of(idxd_dev, struct idxd_user_context, idxd_dev); } +static ssize_t cr_faults_show(struct device *dev, struct device_attribute *attr, char *buf) +{ + struct idxd_user_context *ctx = dev_to_uctx(dev); + + return sysfs_emit(buf, "%llu\n", ctx->counters[COUNTER_FAULTS]); +} +static DEVICE_ATTR_RO(cr_faults); + +static ssize_t cr_fault_failures_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + struct idxd_user_context *ctx = dev_to_uctx(dev); + + return sysfs_emit(buf, "%llu\n", ctx->counters[COUNTER_FAULT_FAILS]); +} +static DEVICE_ATTR_RO(cr_fault_failures); + +static struct attribute *cdev_file_attributes[] = { + &dev_attr_cr_faults.attr, + &dev_attr_cr_fault_failures.attr, + NULL +}; + +static umode_t cdev_file_attr_visible(struct kobject *kobj, struct attribute *a, int n) +{ + struct device *dev = container_of(kobj, typeof(*dev), kobj); + struct idxd_user_context *ctx = dev_to_uctx(dev); + struct idxd_wq *wq = ctx->wq; + + if (!wq_pasid_enabled(wq)) + return 0; + + return a->mode; +} + +static const struct attribute_group cdev_file_attribute_group = { + .attrs = cdev_file_attributes, + .is_visible = cdev_file_attr_visible, +}; + +static const struct attribute_group *cdev_file_attribute_groups[] = { + &cdev_file_attribute_group, + NULL +}; + static void idxd_file_dev_release(struct device *dev) { struct idxd_user_context *ctx = dev_to_uctx(dev); @@ -100,6 +145,7 @@ static void idxd_file_dev_release(struct device *dev) static struct device_type idxd_cdev_file_type = { .name = "idxd_file", .release = idxd_file_dev_release, + .groups = cdev_file_attribute_groups, }; static void idxd_cdev_dev_release(struct device *dev) From patchwork Mon Mar 13 17:02:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fenghua Yu X-Patchwork-Id: 68967 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp1307588wrd; Mon, 13 Mar 2023 10:18:19 -0700 (PDT) X-Google-Smtp-Source: AK7set+bSg7ju5nsT6qlsnGTdZjoNx4+CwENuqeZ+70dEAfhZ4jqFzik+1LvPAWjf1mmfe+3jTl4 X-Received: by 2002:a17:902:e886:b0:199:4a00:9788 with SMTP id w6-20020a170902e88600b001994a009788mr44462357plg.19.1678727898988; Mon, 13 Mar 2023 10:18:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1678727898; cv=none; d=google.com; s=arc-20160816; b=C7B7hUft8aZ0owpP2Eduppm1LxIE33r892g1GrXK8G0HQn1pq96dHFHDX8f9hnSP5G p6UOhgTlso82N0SiDlspe5rBOIpEdAQgIiiCMipzXElW70Bclrb5oW8X4xwKGlXsCcKW sOsya6qke8zyLYy0evviaiJ4Y63ReFyC09smuujz0c88MSTBzoKi1DVbQjcTRNOwkliY DJVpy20JzWcYy0erPpUM42th4sf68+YZe+Zq+yJ32ZvAxs/7ZrLbIIPi6QXVqZXd6Y2C yHj03olCosjEMQun3UpFzIOEh/z6Bl7cBkRDozSUbIEWKeir4QzaDnHU/oJmxEw/or1c vPIA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=37k44xInSHVJOY0Sk5m/HpT0mQ5RgGF3cFAMr//qF/s=; b=Yj2hQ9ebJIgV17rSo/XXOIt7UbHi2KzmUgeAsr+n/OfrSSB1xKiBkkqBGdvsgCgqVK 3LMAeJg9Ln6fvCIUp7v9GaUaQGHLN9kxGjH1V+Zi73WKGjc4KSR7TKsx67z6NRqTJjf2 1sOxtPIs9r/vZ1uvD8VZH6qbJzAkGC68xkeHKf3b8Yvt6lDDU3vG16xT0vZDCPEd4DJG Jmea6mN7a64kUXRibAeZd1xxr5AUgwso1EfKGzKHnEFMBrnH4hNCkKi41hH+IkYYZjOC Umemp71pvlE6omSF49rs7cEMtGkbsyPL32i8y+zl2XSllAFmcmTcDgoIDWS2a9BETfuT w/gw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=FHsPgXcL; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id jc7-20020a17090325c700b0019ac69e7385si227905plb.123.2023.03.13.10.18.06; Mon, 13 Mar 2023 10:18:18 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=FHsPgXcL; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231402AbjCMRHQ (ORCPT + 99 others); Mon, 13 Mar 2023 13:07:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44010 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231483AbjCMRGl (ORCPT ); Mon, 13 Mar 2023 13:06:41 -0400 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BF6A87F01F; Mon, 13 Mar 2023 10:06:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1678727164; x=1710263164; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=U1FC6TvDrEJVxS1ZALZrUjW7uENM7EPrWiu39Wh5u84=; b=FHsPgXcLdH+8zZemriZhfqkjsxErnr3JK/3MVAAtDFe6Y9ePsEEGVN4U +0Jup3ANSRjVa5ROSO3wgGgn7zLXWRACMiquflbfP1sZwTqJ4OR7zAwXq W6jaE3jWg7H6iJ26IRDUITEJZvYGCl55Xs+eYAeCWsQkqbnYGqO8oBdGk q8r+SIf8QueRdzdwA3XoBa9MUysDmpCNuXJcw17WA8o2OKlQ6/EszDoIc bOZsfsa6ViT9J0zbE3vk/N2EAwjRW4uadFugTooMg9KYD2/NSayJng9va 4g7nYa4XEBAtrs5u+oDE7scJcMl7IMHTsjY1KPY+Rabd0HI4XHdZ57cx0 w==; X-IronPort-AV: E=McAfee;i="6500,9779,10648"; a="334679696" X-IronPort-AV: E=Sophos;i="5.98,257,1673942400"; d="scan'208";a="334679696" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Mar 2023 10:02:43 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10648"; a="708950959" X-IronPort-AV: E=Sophos;i="5.98,257,1673942400"; d="scan'208";a="708950959" Received: from fyu1.sc.intel.com ([172.25.103.126]) by orsmga008.jf.intel.com with ESMTP; 13 Mar 2023 10:02:43 -0700 From: Fenghua Yu To: "Vinod Koul" , "Dave Jiang" Cc: dmaengine@vger.kernel.org, "linux-kernel" , Tony Zhu , Fenghua Yu Subject: [PATCH v3 15/16] dmaengine: idxd: add pid to exported sysfs attribute for opened file Date: Mon, 13 Mar 2023 10:02:18 -0700 Message-Id: <20230313170219.1956012-16-fenghua.yu@intel.com> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20230313170219.1956012-1-fenghua.yu@intel.com> References: <20230313170219.1956012-1-fenghua.yu@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1760273785284727661?= X-GMAIL-MSGID: =?utf-8?q?1760273785284727661?= From: Dave Jiang Provide the pid of the application for the opened file. This allows the monitor daemon to easily correlate which app opened the file and easily kill the app by pid if that is desired action. Tested-by: Tony Zhu Signed-off-by: Dave Jiang Co-developed-by: Fenghua Yu Signed-off-by: Fenghua Yu --- Documentation/ABI/stable/sysfs-driver-dma-idxd | 8 ++++++++ drivers/dma/idxd/cdev.c | 11 +++++++++++ 2 files changed, 19 insertions(+) diff --git a/Documentation/ABI/stable/sysfs-driver-dma-idxd b/Documentation/ABI/stable/sysfs-driver-dma-idxd index 73ab86196a41..5d0df57f5298 100644 --- a/Documentation/ABI/stable/sysfs-driver-dma-idxd +++ b/Documentation/ABI/stable/sysfs-driver-dma-idxd @@ -335,3 +335,11 @@ Description: Show the number of Completion Record (CR) faults failures that this driver cannot fault in the address for the CR. Typically this is caused by a bad address programmed in the submitted descriptor or a malicious submitter is using bad CR address on purpose. + +What: /sys/bus/dsa/devices/wq./dsa\!wq./file/pid +Date: Sept 14, 2022 +KernelVersion: 6.4.0 +Contact: dmaengine@vger.kernel.org +Description: Show the process id of the application that opened the file. This is + helpful information for a monitor daemon that wants to kill the + application that opened the file. diff --git a/drivers/dma/idxd/cdev.c b/drivers/dma/idxd/cdev.c index d93d26834dc7..3d905087cbff 100644 --- a/drivers/dma/idxd/cdev.c +++ b/drivers/dma/idxd/cdev.c @@ -49,6 +49,7 @@ struct idxd_user_context { struct idxd_dev idxd_dev; u64 counters[COUNTER_MAX]; int id; + pid_t pid; }; static void idxd_cdev_evl_drain_pasid(struct idxd_wq *wq, u32 pasid); @@ -78,9 +79,18 @@ static ssize_t cr_fault_failures_show(struct device *dev, } static DEVICE_ATTR_RO(cr_fault_failures); +static ssize_t pid_show(struct device *dev, struct device_attribute *attr, char *buf) +{ + struct idxd_user_context *ctx = dev_to_uctx(dev); + + return sysfs_emit(buf, "%u\n", ctx->pid); +} +static DEVICE_ATTR_RO(pid); + static struct attribute *cdev_file_attributes[] = { &dev_attr_cr_faults.attr, &dev_attr_cr_fault_failures.attr, + &dev_attr_pid.attr, NULL }; @@ -238,6 +248,7 @@ static int idxd_cdev_open(struct inode *inode, struct file *filp) ctx->wq = wq; filp->private_data = ctx; + ctx->pid = current->pid; if (device_user_pasid_enabled(idxd)) { sva = iommu_sva_bind_device(dev, current->mm); From patchwork Mon Mar 13 17:02:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fenghua Yu X-Patchwork-Id: 68957 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp1306333wrd; Mon, 13 Mar 2023 10:15:50 -0700 (PDT) X-Google-Smtp-Source: AK7set/fIYPQtRJ3mWTtSZ6XXRwEpl2z0T+WjO9C6vreNPZaFD415tdQDlzkO7r+SbzkiN5aruW0 X-Received: by 2002:a17:902:eccb:b0:19f:2503:a201 with SMTP id a11-20020a170902eccb00b0019f2503a201mr8068453plh.29.1678727749984; Mon, 13 Mar 2023 10:15:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1678727749; cv=none; d=google.com; s=arc-20160816; b=aOtmKC4zZlOPG9cEWAFJH7Ln1b4A6uzedKD4PvlOVbZyRBj1CDWrj8MSnhw1uOk2vJ 1ZvkI4kd0hcW2PVCKfzcMQATVQ7s6LJ6RVqWVdwOPyPyMoy7PhyBcWtKi8u+n9rC6ORs 48qBu3nnbHkXwT51L+D9CEWjbCo+YHUi3Fi+XWqjWkBcaU3LCpQkQ6bTfc/prqlpCuvQ FmySY8mAp059oEXCZNuVnXuyZ1Wv4KvUwngmgssH1YeRyORTDBUTBlUBJvexPM3ZM3Hy odgwIUIoo9J/TuBjyIqF0/TSlgtpuH9DhrqWdEYGAhFAzqoIvYPBJDLRID/5fIGceXAT XdYw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=e6qYQT7lHw2+ZOR5sBAWJ3iuwiOjzVb8LhAMGQkcwV4=; b=CllsJfTccogy69GEY42CthpXICdg4xHqLawh4RsZeiAUWfCRqc9yyQSRWCtT7cgsKG SO0/8L/dXecTW8TpFSRLsW8Umjpdy8ilRF6BPXfjuvGkYLBk4OMtYvuW42CIEFJBRgJL KrCTBkHwdufXuwlB6C/1Jb0f2wJvcQBmzPY4MnS8aWKcvUJerKwzhBO56i7/rcVgmoCP 7ApwzQSQ4xPOHg72K5T6IDF+SX5hpleO9Y6ArDEbLgwzB+lNzXSJCgb54aqYzYlLt208 C3zIbxAlfvERpPYXyAZqHB0sFdpCz6S2tdhsB0HuvCfSYQyu+9i2WkDC+fD2T3hiX70x cVVA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=TxE2ZDZv; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id kl6-20020a170903074600b0019b0b007994si211781plb.163.2023.03.13.10.15.37; Mon, 13 Mar 2023 10:15:49 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=TxE2ZDZv; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231143AbjCMRHj (ORCPT + 99 others); Mon, 13 Mar 2023 13:07:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43972 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231273AbjCMRHI (ORCPT ); Mon, 13 Mar 2023 13:07:08 -0400 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9400C23305; Mon, 13 Mar 2023 10:06:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1678727186; x=1710263186; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=hPT8u68bZ2bhw3ksoI+ABo14zNuLKH375W6tuyoGx1s=; b=TxE2ZDZvGOyN+Z0f+dIElC5p/f0uTGmxZeDgAPx663+1R31H320O6IOi nBHaiISp7GxMsEswXLST6R4lYlznnIbLh4IfVlAuRidgjhKiLvtn1mR7R iVw6IPD3cB3WLFiEUlPCy6KWMOnAHQx+NfY+pTeTsbfUTE8fWHhbxXEqZ zV9sHL+HxZHlijvXNFRCAtsUHT5JZ5F29F7Bw/eHl/Jf3BGdEH+QHtXH1 wOMaeGnE9nok82E3bTLSUAc6j/HDfoQLfsGXebO2hX1ui8EUCSqcce+l4 DoC1+MjQaOd17YkvBtbu7yLibGEMiSJOuuklLoviK3sg2rXxWPvxqQJXi g==; X-IronPort-AV: E=McAfee;i="6500,9779,10648"; a="334679699" X-IronPort-AV: E=Sophos;i="5.98,257,1673942400"; d="scan'208";a="334679699" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Mar 2023 10:02:43 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10648"; a="708950962" X-IronPort-AV: E=Sophos;i="5.98,257,1673942400"; d="scan'208";a="708950962" Received: from fyu1.sc.intel.com ([172.25.103.126]) by orsmga008.jf.intel.com with ESMTP; 13 Mar 2023 10:02:43 -0700 From: Fenghua Yu To: "Vinod Koul" , "Dave Jiang" Cc: dmaengine@vger.kernel.org, "linux-kernel" , Tony Zhu , Fenghua Yu Subject: [PATCH v3 16/16] dmaengine: idxd: add per wq PRS disable Date: Mon, 13 Mar 2023 10:02:19 -0700 Message-Id: <20230313170219.1956012-17-fenghua.yu@intel.com> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20230313170219.1956012-1-fenghua.yu@intel.com> References: <20230313170219.1956012-1-fenghua.yu@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1760273628829584849?= X-GMAIL-MSGID: =?utf-8?q?1760273628829584849?= From: Dave Jiang Add sysfs knob for per wq Page Request Service disable. This knob disables PRS support for the specific wq. When this bit is set, it also overrides the wq's block on fault enabling. Tested-by: Tony Zhu Signed-off-by: Dave Jiang Co-developed-by: Fenghua Yu Signed-off-by: Fenghua Yu --- .../ABI/stable/sysfs-driver-dma-idxd | 10 ++++ drivers/dma/idxd/device.c | 6 +- drivers/dma/idxd/idxd.h | 1 + drivers/dma/idxd/registers.h | 5 +- drivers/dma/idxd/sysfs.c | 57 ++++++++++++++++++- 5 files changed, 74 insertions(+), 5 deletions(-) diff --git a/Documentation/ABI/stable/sysfs-driver-dma-idxd b/Documentation/ABI/stable/sysfs-driver-dma-idxd index 5d0df57f5298..534b7a3d59fc 100644 --- a/Documentation/ABI/stable/sysfs-driver-dma-idxd +++ b/Documentation/ABI/stable/sysfs-driver-dma-idxd @@ -235,6 +235,16 @@ Contact: dmaengine@vger.kernel.org Description: Indicate whether ATS disable is turned on for the workqueue. 0 indicates ATS is on, and 1 indicates ATS is off for the workqueue. +What: /sys/bus/dsa/devices/wq./prs_disable +Date: Sept 14, 2022 +KernelVersion: 6.4.0 +Contact: dmaengine@vger.kernel.org +Description: Controls whether PRS disable is turned on for the workqueue. + 0 indicates PRS is on, and 1 indicates PRS is off for the + workqueue. This option overrides block_on_fault attribute + if set. It's visible only on platforms that support the + capability. + What: /sys/bus/dsa/devices/wq./occupancy Date May 25, 2021 KernelVersion: 5.14.0 diff --git a/drivers/dma/idxd/device.c b/drivers/dma/idxd/device.c index fd97b2b58734..3c80b9681c72 100644 --- a/drivers/dma/idxd/device.c +++ b/drivers/dma/idxd/device.c @@ -967,12 +967,16 @@ static int idxd_wq_config_write(struct idxd_wq *wq) wq->wqcfg->priority = wq->priority; if (idxd->hw.gen_cap.block_on_fault && - test_bit(WQ_FLAG_BLOCK_ON_FAULT, &wq->flags)) + test_bit(WQ_FLAG_BLOCK_ON_FAULT, &wq->flags) && + !test_bit(WQ_FLAG_PRS_DISABLE, &wq->flags)) wq->wqcfg->bof = 1; if (idxd->hw.wq_cap.wq_ats_support) wq->wqcfg->wq_ats_disable = test_bit(WQ_FLAG_ATS_DISABLE, &wq->flags); + if (idxd->hw.wq_cap.wq_prs_support) + wq->wqcfg->wq_prs_disable = test_bit(WQ_FLAG_PRS_DISABLE, &wq->flags); + /* bytes 12-15 */ wq->wqcfg->max_xfer_shift = ilog2(wq->max_xfer_bytes); idxd_wqcfg_set_max_batch_shift(idxd->data->type, wq->wqcfg, ilog2(wq->max_batch_size)); diff --git a/drivers/dma/idxd/idxd.h b/drivers/dma/idxd/idxd.h index bd544eb2ddcb..e44b1d45ccd5 100644 --- a/drivers/dma/idxd/idxd.h +++ b/drivers/dma/idxd/idxd.h @@ -143,6 +143,7 @@ enum idxd_wq_flag { WQ_FLAG_DEDICATED = 0, WQ_FLAG_BLOCK_ON_FAULT, WQ_FLAG_ATS_DISABLE, + WQ_FLAG_PRS_DISABLE, }; enum idxd_wq_type { diff --git a/drivers/dma/idxd/registers.h b/drivers/dma/idxd/registers.h index 9f3959d001b6..7b54a3939ea1 100644 --- a/drivers/dma/idxd/registers.h +++ b/drivers/dma/idxd/registers.h @@ -59,7 +59,8 @@ union wq_cap_reg { u64 occupancy:1; u64 occupancy_int:1; u64 op_config:1; - u64 rsvd3:9; + u64 wq_prs_support:1; + u64 rsvd4:8; }; u64 bits; } __packed; @@ -371,7 +372,7 @@ union wqcfg { u32 mode:1; /* shared or dedicated */ u32 bof:1; /* block on fault */ u32 wq_ats_disable:1; - u32 rsvd2:1; + u32 wq_prs_disable:1; u32 priority:4; u32 pasid:20; u32 pasid_en:1; diff --git a/drivers/dma/idxd/sysfs.c b/drivers/dma/idxd/sysfs.c index 465d2e7627e4..293739ac5596 100644 --- a/drivers/dma/idxd/sysfs.c +++ b/drivers/dma/idxd/sysfs.c @@ -822,10 +822,14 @@ static ssize_t wq_block_on_fault_store(struct device *dev, if (rc < 0) return rc; - if (bof) + if (bof) { + if (test_bit(WQ_FLAG_PRS_DISABLE, &wq->flags)) + return -EOPNOTSUPP; + set_bit(WQ_FLAG_BLOCK_ON_FAULT, &wq->flags); - else + } else { clear_bit(WQ_FLAG_BLOCK_ON_FAULT, &wq->flags); + } return count; } @@ -1109,6 +1113,44 @@ static ssize_t wq_ats_disable_store(struct device *dev, struct device_attribute static struct device_attribute dev_attr_wq_ats_disable = __ATTR(ats_disable, 0644, wq_ats_disable_show, wq_ats_disable_store); +static ssize_t wq_prs_disable_show(struct device *dev, struct device_attribute *attr, char *buf) +{ + struct idxd_wq *wq = confdev_to_wq(dev); + + return sysfs_emit(buf, "%u\n", test_bit(WQ_FLAG_PRS_DISABLE, &wq->flags)); +} + +static ssize_t wq_prs_disable_store(struct device *dev, struct device_attribute *attr, + const char *buf, size_t count) +{ + struct idxd_wq *wq = confdev_to_wq(dev); + struct idxd_device *idxd = wq->idxd; + bool prs_dis; + int rc; + + if (wq->state != IDXD_WQ_DISABLED) + return -EPERM; + + if (!idxd->hw.wq_cap.wq_prs_support) + return -EOPNOTSUPP; + + rc = kstrtobool(buf, &prs_dis); + if (rc < 0) + return rc; + + if (prs_dis) { + set_bit(WQ_FLAG_PRS_DISABLE, &wq->flags); + /* when PRS is disabled, BOF needs to be off as well */ + clear_bit(WQ_FLAG_BLOCK_ON_FAULT, &wq->flags); + } else { + clear_bit(WQ_FLAG_PRS_DISABLE, &wq->flags); + } + return count; +} + +static struct device_attribute dev_attr_wq_prs_disable = + __ATTR(prs_disable, 0644, wq_prs_disable_show, wq_prs_disable_store); + static ssize_t wq_occupancy_show(struct device *dev, struct device_attribute *attr, char *buf) { struct idxd_wq *wq = confdev_to_wq(dev); @@ -1239,6 +1281,7 @@ static struct attribute *idxd_wq_attributes[] = { &dev_attr_wq_max_transfer_size.attr, &dev_attr_wq_max_batch_size.attr, &dev_attr_wq_ats_disable.attr, + &dev_attr_wq_prs_disable.attr, &dev_attr_wq_occupancy.attr, &dev_attr_wq_enqcmds_retries.attr, &dev_attr_wq_op_config.attr, @@ -1260,6 +1303,13 @@ static bool idxd_wq_attr_max_batch_size_invisible(struct attribute *attr, idxd->data->type == IDXD_TYPE_IAX; } +static bool idxd_wq_attr_wq_prs_disable_invisible(struct attribute *attr, + struct idxd_device *idxd) +{ + return attr == &dev_attr_wq_prs_disable.attr && + !idxd->hw.wq_cap.wq_prs_support; +} + static umode_t idxd_wq_attr_visible(struct kobject *kobj, struct attribute *attr, int n) { @@ -1273,6 +1323,9 @@ static umode_t idxd_wq_attr_visible(struct kobject *kobj, if (idxd_wq_attr_max_batch_size_invisible(attr, idxd)) return 0; + if (idxd_wq_attr_wq_prs_disable_invisible(attr, idxd)) + return 0; + return attr->mode; }