From patchwork Thu Feb 9 23:32:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alison Schofield X-Patchwork-Id: 55136 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp637368wrn; Thu, 9 Feb 2023 15:36:55 -0800 (PST) X-Google-Smtp-Source: AK7set/SQlNDkKTkv3UsMFZ1AFMxoqXb4AmMEQ2T4icrKc72Bg4wml+ioKIjs9koqdN4Z7ZrG/iy X-Received: by 2002:a17:903:22c1:b0:196:3feb:1f1e with SMTP id y1-20020a17090322c100b001963feb1f1emr15687372plg.47.1675985814763; Thu, 09 Feb 2023 15:36:54 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675985814; cv=none; d=google.com; s=arc-20160816; b=lr61Cl7Ami2zQDUixwS2UOXhrLbbSYH20NMcLU1xQWhKR2Bv6mCNh3j+lHcvYP74aN Zf3aVkOssoJS0btTtG/SSN8BoMw3hAsIPG7mzPx7OrcbH2FcwFxyoNemQBn2JGGkvoXi 6g6e/4sUNOpo5+m5yfOIArk/83ye5DxnGJq5vl61QCx8vA3eTkBFG7aFfg0Dj49mX24s 9Fh+F4GAuaejwKpNcKvbUU6A080f6Tyy8Y0SmL+sy6lUk8LUqOcISAoerSEXe8iwGpMv AGbb6eDvKhl0fMCiyYeP/Mz5wYjHwBmnFDv8xu5hNr8UDBMCPvSZVqvaNkLF4AF5mdMh SNPA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=53uN+xd1AvwJgTPrQaBkefLArukkD5j6KmVLv718Xts=; b=k6fXIg7PLiYavZGm9drIb1pqrPM3K9PxQuwfQC9j2I86QecrVupze2TRgHALEHEJAJ GCN/XDn0ezFitecB6HoPrxyXeCDsUSLC0icNHi+5lh5EC2Nal9cM2+BE+wQPdyoE79A0 P1xH+RqlVwraHmGXUpYCqGQPkZF2WT6/aUC27WR8srkuK7mW7LoRbwdJpQwva2KJ08J6 p5HFbMZpJYZi2b3BmFpyo+6YBMYX+uAWZZBMxZaoGlpOVDjDNhTkQmQD5xgh4hMbriq9 +d9l0fxk0vohuIuuigHsz1afuiK3o73G5cb9vz4xxsQHDZ0/Rh7s0LKuFLomh490fEpi HNpA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=UxcwgfrZ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id e9-20020a17090301c900b00194c4985c98si3258959plh.248.2023.02.09.15.36.41; Thu, 09 Feb 2023 15:36:54 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=UxcwgfrZ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231216AbjBIXfg (ORCPT + 99 others); Thu, 9 Feb 2023 18:35:36 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41606 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231210AbjBIXfW (ORCPT ); Thu, 9 Feb 2023 18:35:22 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 59F8E6D8DE; Thu, 9 Feb 2023 15:34:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675985695; x=1707521695; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Cv6WybSbF9U1JSwkfFCNpheiIUUtR7LMTKLupRFUKQ8=; b=UxcwgfrZhniLHUgbJOfe6cMWhVd1y0yZnNl1yswMFdIg3YgMe/h30PLw 1rpIJcBd8ai2+nvwCCVkLQGj4LONfDYGus+QES/R46WKw6lMbPi9Mumst TGcO2doeqOeyqKNQjFVFLM7Kt1zCOPuzHWUUNrfGb8nDDDFHT0Ei8Edjx WaCwVRG+6Wou1v7drRSGKp8jgtCds2nLKjV9HLSOcjPHpOfVYrOoZvtuJ pwjL8OfeKc0Mq9zUI4YHVYgpLRMcZHzuzxsFP/i/BBjrMar5bv5XVPRNA jb7YYUcy/Eza31WTP4+zNH4qiKU35GdJBckhhpyxe3jUBP5LdSxG2bNGJ A==; X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="416508730" X-IronPort-AV: E=Sophos;i="5.97,285,1669104000"; d="scan'208";a="416508730" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Feb 2023 15:33:09 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="756592045" X-IronPort-AV: E=Sophos;i="5.97,285,1669104000"; d="scan'208";a="756592045" Received: from aschofie-mobl2.amr.corp.intel.com (HELO localhost) ([10.209.13.232]) by fmsmga003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Feb 2023 15:33:08 -0800 From: alison.schofield@intel.com To: Dan Williams , Ira Weiny , Vishal Verma , Dave Jiang , Ben Widawsky , Steven Rostedt Cc: Alison Schofield , linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v6 4/6] cxl/region: Provide region info to the cxl_poison trace event Date: Thu, 9 Feb 2023 15:32:57 -0800 Message-Id: <7862ef793a3b4fce49b0c8074b014b53b389ce12.1675983077.git.alison.schofield@intel.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757398501888524167?= X-GMAIL-MSGID: =?utf-8?q?1757398501888524167?= From: Alison Schofield User space may need to know which region, if any, maps the poison address(es) logged in a cxl_poison trace event. Since the mapping of DPAs (device physical addresses) to a region can change, the kernel must provide this information at the time the poison list is read. The event informs user space that at event this mapped to this , which is poisoned. The cxl_poison trace event is already wired up to log the region name and uuid if it receives param 'struct cxl_region'. In order to provide that cxl_region, add another method for gathering poison - by committed endpoint decoder mappings. This method is only available with CONFIG_CXL_REGION and is only used if a region actually maps the memdev where poison is being read. The default method remains: read the poison by memdev resource. Signed-off-by: Alison Schofield --- drivers/cxl/core/core.h | 5 +++ drivers/cxl/core/memdev.c | 14 ++++++- drivers/cxl/core/region.c | 82 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 100 insertions(+), 1 deletion(-) diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h index 8c04672dca56..2f9bd8651eb1 100644 --- a/drivers/cxl/core/core.h +++ b/drivers/cxl/core/core.h @@ -22,7 +22,12 @@ void cxl_decoder_kill_region(struct cxl_endpoint_decoder *cxled); #define CXL_PMEM_REGION_TYPE(x) (&cxl_pmem_region_type) int cxl_region_init(void); void cxl_region_exit(void); +int cxl_get_poison_by_endpoint(struct device *dev, void *data); #else +static inline int cxl_get_poison_by_endpoint(struct device *dev, void *data) +{ + return 0; +} static inline void cxl_decoder_kill_region(struct cxl_endpoint_decoder *cxled) { } diff --git a/drivers/cxl/core/memdev.c b/drivers/cxl/core/memdev.c index 19b833c9cf35..8696d7b508b6 100644 --- a/drivers/cxl/core/memdev.c +++ b/drivers/cxl/core/memdev.c @@ -139,14 +139,26 @@ static ssize_t trigger_poison_list_store(struct device *dev, const char *buf, size_t len) { struct cxl_memdev *cxlmd = to_cxl_memdev(dev); + struct cxl_port *port; bool trigger; int rc; if (kstrtobool(buf, &trigger) || !trigger) return -EINVAL; + port = dev_get_drvdata(&cxlmd->dev); + if (!port || !is_cxl_endpoint(port)) + return -EINVAL; + down_read(&cxl_dpa_rwsem); - rc = cxl_get_poison_by_memdev(cxlmd); + if (port->commit_end == -1) + /* No regions mapped to this memdev */ + rc = cxl_get_poison_by_memdev(cxlmd); + else + /* Regions mapped, collect poison by endpoint */ + rc = device_for_each_child(&port->dev, port, + cxl_get_poison_by_endpoint); + up_read(&cxl_dpa_rwsem); return rc ? rc : len; diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 67e83d961670..0ac08e9106af 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -1826,6 +1826,88 @@ struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev) } EXPORT_SYMBOL_NS_GPL(to_cxl_pmem_region, CXL); +int cxl_get_poison_by_endpoint(struct device *dev, void *data) +{ + struct cxl_endpoint_decoder *cxled; + struct cxl_port *port = data; + struct cxl_dev_state *cxlds; + struct cxl_memdev *cxlmd; + u64 offset, length; + int rc = 0; + + down_read(&cxl_dpa_rwsem); + + if (!is_endpoint_decoder(dev)) + goto out; + + cxled = to_cxl_endpoint_decoder(dev); + if (!cxled->dpa_res || !resource_size(cxled->dpa_res)) + goto out; + + /* + * Get the poison by decoder for mapped resources. This + * separates pmem and ram poison list reads, as the spec + * requires, and provides the region for the trace event. + */ + cxlmd = cxled_to_memdev(cxled); + length = cxled->dpa_res->end - cxled->dpa_res->start + 1; + rc = cxl_mem_get_poison(cxlmd, cxled->dpa_res->start, length, + cxled->cxld.region); + if (rc == -EFAULT && cxled->mode == CXL_DECODER_RAM) + rc = 0; + if (rc) + goto out; + + /* Get poison in a skip range */ + if (cxled->skip) { + rc = cxl_mem_get_poison(cxlmd, 0, cxled->skip, NULL); + if (rc == -EFAULT && cxled->mode == CXL_DECODER_RAM) + rc = 0; + if (rc) + goto out; + } + + /* Iterate until commit_end is reached */ + if (cxled->cxld.id < port->commit_end) + goto out; + + /* + * Reach here with the last committed decoder only. + * Knowing that PMEM must always follow RAM, get poison + * for unmapped ranges based on the last decoder's mode: + * ram: scan remains of ram range, then scan for pmem + * pmem: scan remains of pmem range + */ + cxlds = cxlmd->cxlds; + + if (cxled->mode == CXL_DECODER_RAM) { + offset = cxled->dpa_res->end + 1; + length = resource_size(&cxlds->ram_res) - offset; + rc = cxl_mem_get_poison(cxlmd, offset, length, NULL); + if (rc == -EFAULT) + rc = 0; + if (rc) + goto out; + } + if (cxled->mode == CXL_DECODER_PMEM) { + offset = cxled->dpa_res->end + 1; + length = resource_size(&cxlds->pmem_res) - offset; + } else if (resource_size(&cxlds->pmem_res)) { + offset = cxlds->pmem_res.start; + length = resource_size(&cxlds->pmem_res); + } else { + rc = 1; + goto out; + } + /* Final get poison call. Return rc or 1 to stop iteration. */ + rc = cxl_mem_get_poison(cxlmd, offset, length, NULL); + if (!rc) + rc = 1; +out: + up_read(&cxl_dpa_rwsem); + return rc; +} + static struct lock_class_key cxl_pmem_region_key; static struct cxl_pmem_region *cxl_pmem_region_alloc(struct cxl_region *cxlr)