From patchwork Tue Apr 18 21:05:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Smita Koralahalli X-Patchwork-Id: 85072 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp3134445vqo; Tue, 18 Apr 2023 14:21:40 -0700 (PDT) X-Google-Smtp-Source: AKy350YfAccDMpki5qJYDbXM+jnTzDAItITJHDDJbOrSbFiQ5Iija926cGUwcwDIp/edxyxJydjN X-Received: by 2002:a17:902:7049:b0:1a3:d392:2f29 with SMTP id h9-20020a170902704900b001a3d3922f29mr3129144plt.20.1681852899822; Tue, 18 Apr 2023 14:21:39 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1681852899; cv=pass; d=google.com; s=arc-20160816; b=r6vOrAKN8aK4hPUsfjMVrGvUIGqS2Qy6gRc9tnJDO+pm1N52ampc/oc9MidQV2Qfmh y03Th6gOloERu9BS/OXs/FUw/Pu4ciDv8DjQAbwER6M6zktB9zbjVJpXXx3S2MrKu1UV Mrv9cRN+/D1nuX/krNvOIi1c88qlNuN6Yn44XxxZ0r3btoiGx0no8fiICgi0HCHZ+0xZ VEDfnWQSNR4NTifX2LhIZ/VW2QGNEKZuV53CsdZGzR7f3SwNrttW/WlMC6qJkeVbQt5v T1IY4bj9NW/hvD9+krwi9H05vDL96xAH7hf6BQLvSfpnAMw2EdeHEPIf32oLI6uF2NlV g/wA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=BUPcbBKETtUo3RZHR2pekMCFyls3hE1EC7o6j1moNmg=; b=YMv6WvGG7Vog5D898RE//siohoigZEWIuWNQU0ZXgwx3kSKVlyhKtkBXBG/HXNLiPd ylZob4QwWTAamvmhgCFXU42IbR0JVEncD1VkEce4IqNAbzB7YZE54u8RGl86qSr5AD7+ iFWiFnD12gcy/Vsmfw4cy/JPE9DQ2WYw68dfTFPDa8KCUkVqyvlkDxmt1ZWNpouRR6aS dk5ox/Vc1nmcTsFxC37kA8P/7g9dsTL28Y9cp35Qv5DqWCXjETkn9rff6XbTzhYQeQrA q859IvHq0vRC031HdWJFReEn3NiYpg6m0v6DvjcCgIhWYZJ6yjcL3Dml7R+udCFtsIxP 4Uvw== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b=ZPyBErov; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id x2-20020a170902820200b001a6ee58ebdfsi3634345pln.524.2023.04.18.14.21.12; Tue, 18 Apr 2023 14:21:39 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b=ZPyBErov; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232848AbjDRVFw (ORCPT + 99 others); Tue, 18 Apr 2023 17:05:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58040 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232824AbjDRVFr (ORCPT ); Tue, 18 Apr 2023 17:05:47 -0400 Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on2086.outbound.protection.outlook.com [40.107.96.86]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DBFB49031; Tue, 18 Apr 2023 14:05:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=h0iEvH/U0HPV1aVZSvCW25cBHeNRJ0z7ak2Gk+HwGZli9i7g1jGUTTdubIGImkhmqdXdZKXz+Eobl8OLPeBi0DFExtElq50/SOd/UVsB/5pdWvCLFTzzz3HmeQyaP9Wrf04pIVXKnwx7s7FoWZBAdL4SSM+oW/vdzVezEWaG6iPE1vMS0UsDpREuXUxmneUD2RVvK4gG1g2C+yPy+qew4zSiQy3SsqU3MSzpJx3tVCWVLuutfRnTradc1IAkJueqxjsk9B+4Qr81AF/RVz4mVMvPlGfMD7gMJ9Tsy1g6IcCzT/n/9yJ1Xt1Jr64qVAiNX543ytvn6LCbBdFDOG/z0g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=BUPcbBKETtUo3RZHR2pekMCFyls3hE1EC7o6j1moNmg=; b=FZ6HUtymrCUVgFHYHH9cVFvbtDNgbSo7H9l1ROxxCfPuOKuxVRpXz+QAobSuJ7hC9ajYNXDvSwSb8uJJQjJltpurv2qAO2113+2xyHr4XLE5BQpHZfsemwZVqAG1xlYaXm33snCep4ds2SBdRcVjWir91T+0BSOW+v+OSEG5zIG7NN1MGadEhT+uWGz5BSutFTnRkXcN3mFrWQUtOU7gsyMqTkGhmg3ZTO3AbwVUlDqnEE+ot24bCcOzFka9lX2XV4DP1qDNSeuVS/3Oiq6rSw7MVhEACIPB7PowWPuBl8MXsItRyWCeP2UI+Wcy/VOus8Il7WthzomA9l7DAlYpNA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BUPcbBKETtUo3RZHR2pekMCFyls3hE1EC7o6j1moNmg=; b=ZPyBErovOazOPxEf+o9zGqTauLr4Xl0SRVEMzCRA04PxLTOmN7fIDHPVAgOZzLm2vUfcRymAICAJg2LDkKns6Km2uBqMdONaVnv32S2cCbn4/jRqTNJnjOenELb1tvf/EE9BVKStlFH8e3HMob8beDqfLL7r08W5SxH0Xv4c/1k= Received: from MW4PR03CA0223.namprd03.prod.outlook.com (2603:10b6:303:b9::18) by LV2PR12MB5872.namprd12.prod.outlook.com (2603:10b6:408:173::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6298.45; Tue, 18 Apr 2023 21:05:43 +0000 Received: from CO1NAM11FT003.eop-nam11.prod.protection.outlook.com (2603:10b6:303:b9:cafe::52) by MW4PR03CA0223.outlook.office365.com (2603:10b6:303:b9::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6298.46 via Frontend Transport; Tue, 18 Apr 2023 21:05:42 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1NAM11FT003.mail.protection.outlook.com (10.13.175.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6319.20 via Frontend Transport; Tue, 18 Apr 2023 21:05:42 +0000 Received: from ethanolx50f7host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Tue, 18 Apr 2023 16:05:41 -0500 From: Smita Koralahalli To: , CC: Bjorn Helgaas , , "Mahesh J Salgaonkar" , Lukas Wunner , "Kuppuswamy Sathyanarayanan" , Yazen Ghannam , Fontenot Nathan , "Smita Koralahalli" Subject: [PATCH v2 1/2] PCI: pciehp: Add support for async hotplug with native AER and DPC/EDR Date: Tue, 18 Apr 2023 21:05:25 +0000 Message-ID: <20230418210526.36514-2-Smita.KoralahalliChannabasappa@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230418210526.36514-1-Smita.KoralahalliChannabasappa@amd.com> References: <20230418210526.36514-1-Smita.KoralahalliChannabasappa@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1NAM11FT003:EE_|LV2PR12MB5872:EE_ X-MS-Office365-Filtering-Correlation-Id: 95197c22-df2c-42ad-1bfb-08db4050abcd X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: NwBWlp5l6tk03CBMksHAXqcKNkA/ICWE/XFDSBM+EpI9eC0HHf4BRG5Mqxl1j68UngHe3KvE0ujC/XCo6UjJvE5cFV1pCZaBFnCxnc9V70Ns76lj2+1SER+Mi8dI7+Qb4XuUYmK+xFfHikdnax2Suq4u8FP/bkSeDy7P7tjXzeAsg4nUQGNloGEg6XpFlT7CHWE4uaAy3cSItWrxDY1+K7TxbaHrQsC+6EwuWuOAFBTP2Fx4FpIv1GtimzxHhpS5bemKk5VyQO0cmPThmYhlcspaQJUAdJmUE04v9rMG2A0RYNq7Z5B9t0iCNxW2WQB+BKZzBjR0NApdH8ZS50ygMZfbCWHkOyEjF7k5ImwNyy0jdkBvlqnoOpN2qnm/YR1z50lWp7ylwOa8c+l1VxH4/bsApd/e5aPZoLCvOVj1Zy/AgBGMQbMtaWqaMa7GNYD03pEskWYz48t3qver3tO4pWbisT7+Okzhe3qaA8Qke+mBRi26WjjlRzjG0FAOKYdXOm6AkwGeHSEdMACds2PMz9xWRB9uujWUMiEFgo+61e4tqAOCz1oqzOdponEWDY+9seMj4z2FpTtPyISIM+dtHIMMLhmA6vsOGrzCsiOWv10qELZsAKb3xtbmG/oJyyEHx6AneDHLy+M/xZdYYyRNJfSUEoPyf6FvneVCHOjCPbwoo2mmwUND7ZqgTsT8Zd5TVyrMswANRnk60rCewJctjY2J/a3K1hTa4xtuiDT1VCH3GZ+6vfJyIsRorW5kYBasr0SC13HsOQc2SCvqCBCj/g== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(396003)(39860400002)(376002)(136003)(346002)(451199021)(46966006)(40470700004)(36840700001)(356005)(82740400003)(26005)(1076003)(40460700003)(81166007)(336012)(40480700001)(47076005)(426003)(16526019)(83380400001)(36860700001)(186003)(2616005)(5660300002)(8936002)(8676002)(2906002)(36756003)(41300700001)(316002)(86362001)(6666004)(478600001)(966005)(7696005)(70206006)(4326008)(70586007)(54906003)(110136005)(82310400005)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Apr 2023 21:05:42.4308 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 95197c22-df2c-42ad-1bfb-08db4050abcd X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT003.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV2PR12MB5872 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE, T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1763550586222395329?= X-GMAIL-MSGID: =?utf-8?q?1763550586222395329?= According to Section 6.7.6 of PCIe Base Specification [1], async removal with DPC and EDR may be unexpected and may result in a Surprise Down error. This error is just a side effect of hot remove. Most of the time, these errors will be abstract to the OS as current systems rely on Firmware-First model for AER and DPC, where the error handling (side effects of async remove) and other necessary HW sequencing actions is taken care by the FW and is abstract to the OS. However, FW-First model poses issues while rolling out updates or fixing bugs as the servers need to be brought down for firmware updates. Add support for async hot-plug with native AER and DPC/EDR. Here, OS is responsible for handling async add and remove along with handling of AER and DPC events which are generated as a side-effect of async remove. The implementation is as follows: On an async remove a DPC is triggered along with a Presence Detect State change. Determine it's an async remove by checking for DPC Trigger Status in DPC Status Register and Surprise Down Error Status in AER Uncorrected Error Status to be non-zero. If true, treat the DPC event as a side-effect of async remove, clear the error status registers and continue with hot-plug tear down routines. If not, follow the existing routine to handle AER and DPC errors. Dmesg before: pcieport 0000:00:01.4: DPC: containment event, status:0x1f01 source:0x0000 pcieport 0000:00:01.4: DPC: unmasked uncorrectable error detected pcieport 0000:00:01.4: PCIe Bus Error: severity=Uncorrected (Fatal), type=Transaction Layer, (Receiver ID) pcieport 0000:00:01.4: device [1022:14ab] error status/mask=00000020/04004000 pcieport 0000:00:01.4: [ 5] SDES (First) nvme nvme2: frozen state error detected, reset controller pcieport 0000:00:01.4: DPC: Data Link Layer Link Active not set in 1000 msec pcieport 0000:00:01.4: AER: subordinate device reset failed pcieport 0000:00:01.4: AER: device recovery failed pcieport 0000:00:01.4: pciehp: Slot(16): Link Down nvme2n1: detected capacity change from 1953525168 to 0 pci 0000:04:00.0: Removing from iommu group 49 Dmesg after: pcieport 0000:00:01.4: pciehp: Slot(16): Link Down nvme1n1: detected capacity change from 1953525168 to 0 pci 0000:04:00.0: Removing from iommu group 37 [1] PCI Express Base Specification Revision 6.0, Dec 16 2021. https://members.pcisig.com/wg/PCI-SIG/document/16609 Signed-off-by: Smita Koralahalli --- v2: Indentation is taken care. (Bjorn) Unrelevant dmesg logs are removed. (Bjorn) Rephrased commit message, to be clear on native vs FW-First handling. (Bjorn and Sathyanarayanan) Prefix changed from pciehp_ to dpc_. (Lukas) Clearing ARI and AtomicOp Requester are performed as a part of (de-)enumeration in pciehp_unconfigure_device(). (Lukas) Changed to clearing all optional capabilities in DEVCTL2. OS-First -> native. (Sathyanarayanan) Please note that, I have provided explanation why I'm not setting the Surprise Down bit in uncorrectable error mask register in AER. https://lore.kernel.org/all/fba22d6b-c225-4b44-674b-2c62306135ed@amd.com/ Also, while testing I noticed PCI_STATUS and PCI_EXP_DEVSTA will be set on an async remove and will not be cleared while the device is brought down. I have included clearing them here in order to mask any kind of appearance that there was an error and as well duplicating our BIOS functionality. I can remove if its not necessary. On AMD systems we observe Presence Detect State change along with DPC event on an async remove. Hence, the errors observed are benign on AMD systems and the device will be brought down normally with PDSC. But the errors logged might confuse users. --- drivers/pci/pcie/dpc.c | 50 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 50 insertions(+) diff --git a/drivers/pci/pcie/dpc.c b/drivers/pci/pcie/dpc.c index a5d7c69b764e..78559188b9ac 100644 --- a/drivers/pci/pcie/dpc.c +++ b/drivers/pci/pcie/dpc.c @@ -293,10 +293,60 @@ void dpc_process_error(struct pci_dev *pdev) } } +static void pci_clear_surpdn_errors(struct pci_dev *pdev) +{ + u16 reg16; + u32 reg32; + + pci_read_config_dword(pdev, pdev->dpc_cap + PCI_EXP_DPC_RP_PIO_STATUS, ®32); + pci_write_config_dword(pdev, pdev->dpc_cap + PCI_EXP_DPC_RP_PIO_STATUS, reg32); + + pci_read_config_word(pdev, PCI_STATUS, ®16); + pci_write_config_word(pdev, PCI_STATUS, reg16); + + pcie_capability_read_word(pdev, PCI_EXP_DEVSTA, ®16); + pcie_capability_write_word(pdev, PCI_EXP_DEVSTA, reg16); +} + +static void dpc_handle_surprise_removal(struct pci_dev *pdev) +{ + if (pdev->dpc_rp_extensions && dpc_wait_rp_inactive(pdev)) + return; + + /* + * According to Section 6.7.6 of the PCIe Base Spec 6.0, since async + * removal might be unexpected, errors might be reported as a side + * effect of the event and software should handle them as an expected + * part of this event. + */ + pci_aer_raw_clear_status(pdev); + pci_clear_surpdn_errors(pdev); + + pci_write_config_word(pdev, pdev->dpc_cap + PCI_EXP_DPC_STATUS, + PCI_EXP_DPC_STATUS_TRIGGER); +} + +static bool dpc_is_surprise_removal(struct pci_dev *pdev) +{ + u16 status; + + pci_read_config_word(pdev, pdev->aer_cap + PCI_ERR_UNCOR_STATUS, &status); + + if (!(status & PCI_ERR_UNC_SURPDN)) + return false; + + dpc_handle_surprise_removal(pdev); + + return true; +} + static irqreturn_t dpc_handler(int irq, void *context) { struct pci_dev *pdev = context; + if (dpc_is_surprise_removal(pdev)) + return IRQ_HANDLED; + dpc_process_error(pdev); /* We configure DPC so it only triggers on ERR_FATAL */ From patchwork Tue Apr 18 21:05:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Smita Koralahalli X-Patchwork-Id: 85074 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp3135298vqo; Tue, 18 Apr 2023 14:23:39 -0700 (PDT) X-Google-Smtp-Source: AKy350ZCwjk0su4Sy+tAsc6m9HRnzcAKCIbBNTc4oQ3bBjcD4aFKAdUkTrEYcEHpjqlN1Vfv7JHg X-Received: by 2002:a05:6a00:1255:b0:637:aea0:b23d with SMTP id u21-20020a056a00125500b00637aea0b23dmr1340829pfi.10.1681853019129; Tue, 18 Apr 2023 14:23:39 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1681853019; cv=pass; d=google.com; s=arc-20160816; b=WC2P3EtXjZwkssZGskbyXJyyNtHiNf60AprzFIPFqFgGQr8y+vxSmXyQrgzQ7HljP3 wG4dlTvgDHUnR/6aGWiB79eWAv1dL3ZrcG9a0sbgS3AqmDQqi9mWyZe4llFVzNOkMbaB /PUsbiNiBBYUwXI2rohJwEc0D8Wn7oljCZ0v1njXTEjnj/IYDIIJDJhrWwb0tr3iXlIK KEWZzxgZ22wwfk4uUqDPsEbu0RHPKGmVMSJdWtBYMF+He9u0yzUCW0KYviRStHEvrhp6 Ki18mJKZ+b39nG4qqo9mMRNi6nJWQezR5JhlElHLwQA/pXulwxu96ulnbwYeFY2wUqKq gm0Q== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=QtePaqksZc8XoaO/o8KDI+VRfVhOhKrJ0skPy4eEXPQ=; b=hf52bTNlHquIg4XA6ZeQKLXWyowqXZjcqLJI6pCerVTRSoZfrAxG2WHlS7OYbDykeR CNFKlrNXzlAYC8eUSbrgfk2VsLLAqiViyVQAmVgFIdDDrGpswbUYRG6j3J9h74Z4aElg 5vIRVBbzThi5GQH4AWxjRKiKuxtEIp6k1ugaFwGFPguhgBcXZQtjuvD7j3yOjsM2spvl u7rVJ8FG+4M2jkc1uC1maYrGKof5G7LUzeVxslMeZe32sTlqBkcnRkNCv6coMRNuMxRL cmePv5fbmLKDadsgBOBkrufLkDz2VoDUgvTIL6ScLk1UAykI8R0urEWwjeEbtz5Nj752 IGHA== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b=tcdbBRjj; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id w129-20020a623087000000b005a0c57eac92si10512770pfw.232.2023.04.18.14.23.20; Tue, 18 Apr 2023 14:23:39 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b=tcdbBRjj; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232854AbjDRVFx (ORCPT + 99 others); Tue, 18 Apr 2023 17:05:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58038 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232834AbjDRVFs (ORCPT ); Tue, 18 Apr 2023 17:05:48 -0400 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2041.outbound.protection.outlook.com [40.107.236.41]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 409179EC6; Tue, 18 Apr 2023 14:05:47 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MOfmHt8dJFdnfmnRJHbjIqdGr15IrOZOZG30neTl3rAwd8jqvpi6Ur5AYLakNOpOPSDqmLZ8vKCr+NqW2BrmV+6X/H6FPkBUFOJZ1cWv6WItVAb6KHEm6q7Eo8udGeIi9iA5O+I34AenCIrMoEV8zLP7720v5Bhwq99l5i0sW5goIpoBzR+ddM80TaBWMB0CuVIPWpLqvv0392dJNCw/saO8cuhiB8NPUtF4iTe9vYzXxqveRL0vfFHudE3bbCZOxlSaYVTI6ZYTbgIkJf8a0Pe6nT4ip5xlDjg21AB8uBWMCuopMLbxxzAUuUROcUZBhtaICuuitTNffQ3nw9Q+zg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=QtePaqksZc8XoaO/o8KDI+VRfVhOhKrJ0skPy4eEXPQ=; b=NfQR5IrZ9yZMFQvFHccDYJy0HocVZjPNBcexTkoNwsf6w6yCIRSVETsow7ML2Ua8B899x6Jq/3yZGOuONXR84Y8OMCg+YMYdWhfxjs+ND294GvyP6Adm+kUdNLjfreOLBCOrVUC+DrbZLNXPFU3gDG9TXqbCDRCdE1dw9cW7/sZ50hpWnalIGjF5rxkgLkpzCpZDG1RB4t0rVTIMWbvW8Ty9ULre7c9R1gzqDIwh/PCZU11U/1D9IlIM5VC6Nm1GeFgkJtdQ4X3XuHSCdNjmvJ8xEe8aHkxDIQo+67ifinGq4Yg5UqetgtsRuyrXKKX/+jHBpboUD4CBwzIC1jBByQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=QtePaqksZc8XoaO/o8KDI+VRfVhOhKrJ0skPy4eEXPQ=; b=tcdbBRjjxcJwNIG8kmLywb1SizIzdV2IRWAfWxS3lG07rySPL4AU5zYzYb/v/Ap2mpDRDJg+aag/eVGaLHo88cXVDkEeYw6UZHFgo9yfqYfa27VE58S6+KhH1rRw4iMfr2lfj65KE20v8DQEMR+yvq1rSJN7N/yMyzu55hDS7Q0= Received: from MW4PR03CA0238.namprd03.prod.outlook.com (2603:10b6:303:b9::33) by BN9PR12MB5145.namprd12.prod.outlook.com (2603:10b6:408:136::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6298.45; Tue, 18 Apr 2023 21:05:44 +0000 Received: from CO1NAM11FT003.eop-nam11.prod.protection.outlook.com (2603:10b6:303:b9:cafe::1f) by MW4PR03CA0238.outlook.office365.com (2603:10b6:303:b9::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6298.46 via Frontend Transport; Tue, 18 Apr 2023 21:05:44 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1NAM11FT003.mail.protection.outlook.com (10.13.175.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6319.20 via Frontend Transport; Tue, 18 Apr 2023 21:05:43 +0000 Received: from ethanolx50f7host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Tue, 18 Apr 2023 16:05:42 -0500 From: Smita Koralahalli To: , CC: Bjorn Helgaas , , "Mahesh J Salgaonkar" , Lukas Wunner , "Kuppuswamy Sathyanarayanan" , Yazen Ghannam , Fontenot Nathan , "Smita Koralahalli" Subject: [PATCH v2 2/2] PCI: pciehp: Clear the optional capabilities in DEVCTL2 on a hot-plug Date: Tue, 18 Apr 2023 21:05:26 +0000 Message-ID: <20230418210526.36514-3-Smita.KoralahalliChannabasappa@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230418210526.36514-1-Smita.KoralahalliChannabasappa@amd.com> References: <20230418210526.36514-1-Smita.KoralahalliChannabasappa@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1NAM11FT003:EE_|BN9PR12MB5145:EE_ X-MS-Office365-Filtering-Correlation-Id: c7e506fb-1fd5-49f0-84c8-08db4050ac9c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: p+VVOXLQ6VEx9+q+2rnEAWn9PvSL+mB4SBk23H1O9lkf/lxXf+cw4Zgn1TDySlDFaVHz34GSVxdvYTCwP8Hhx8aFPo4xZQJY3PN2DvLW9UtCMKKXnlwc2+z/hgxvYzoVoNJfvX8cJW8rmiIG4vNj2T915umx//HXbaVl7xy58BLRCV21E6BBgIHjfNUu9t8H6kIrUFP6IXAlZQNNHXDhKuOcGo1NQ3WkzIOAmDZJKyXOoAvOP/mDgVEtGIBYd+93ex6Yhg3IsVo2vX/O4Qtjzjo1boaxIJ5sI2R5FKxyC386v8wFB64DJ+2Tjb9Hiy4mkTDr5BFEcsWWf1zGj6AygSk6eiH9ZB4Ax0tAvJbfyXBbR+0Ea7woHiATXEk3aB9fuFOsSzaq9jAdDKRnAbcDhF57tpCxBZx0nJ2znwcqrQR5XlmDGczCOHFTjw6xz36vVWau4Pfc/YLVeogR6SpSRuPzgme8CemJUujaXhoLHfZ4WS7+/yqdt7L56zvjuIZH4l9n8BPiuFF77Pwbs2V8c/aBsChJ6Qywfz7Og6aaasHuinDHDCEvYtK7pvb6gt5jfJs8fX4kH4EuY2icvGu/ZBzTyQxO+At/i5kqTbcAbo61NHPK6jFZUE5yQOjS6kOh42eX8Z8oUYwvcNiUCQ6qORtJ/dtFj4n4Al1x+2GXPhj4GFPtlUHSE3y1cBqqOotCRB5bkWsD3uGDCcuYIK6ZgoxaZt//iV2ND6wBioiGqys= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(376002)(396003)(346002)(39860400002)(136003)(451199021)(40470700004)(36840700001)(46966006)(36860700001)(2616005)(54906003)(110136005)(26005)(1076003)(478600001)(82740400003)(70586007)(70206006)(47076005)(316002)(83380400001)(7696005)(6666004)(186003)(966005)(16526019)(336012)(4326008)(426003)(8676002)(8936002)(2906002)(41300700001)(5660300002)(356005)(82310400005)(81166007)(36756003)(40460700003)(40480700001)(86362001)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Apr 2023 21:05:43.7744 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c7e506fb-1fd5-49f0-84c8-08db4050ac9c X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT003.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN9PR12MB5145 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE, T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1763550711091327453?= X-GMAIL-MSGID: =?utf-8?q?1763550711091327453?= Clear all capabilities in Device Control 2 register as they are optional and it is not determined whether the next device inserted will support these capabilities. Moreover, Section 6.13 of the PCIe Base Specification [1], recommends clearing the ARI Forwarding Enable bit on a hot-plug event as its not guaranteed that the newly added component is in fact an ARI device. [1] PCI Express Base Specification Revision 6.0, Dec 16 2021. https://members.pcisig.com/wg/PCI-SIG/document/16609 Signed-off-by: Smita Koralahalli --- v2: Clear all optional capabilities in Device Control 2 register instead of individually clearing ARI Forwarding Enable, AtomicOp Requestor Enable and 10-bit Tag Requestor Enable. --- drivers/pci/hotplug/pciehp_pci.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/pci/hotplug/pciehp_pci.c b/drivers/pci/hotplug/pciehp_pci.c index d17f3bf36f70..aabf7884ff30 100644 --- a/drivers/pci/hotplug/pciehp_pci.c +++ b/drivers/pci/hotplug/pciehp_pci.c @@ -104,6 +104,7 @@ void pciehp_unconfigure_device(struct controller *ctrl, bool presence) list_for_each_entry_safe_reverse(dev, temp, &parent->devices, bus_list) { pci_dev_get(dev); + pcie_capability_clear_word(dev, PCI_EXP_DEVCTL2, 0xffff); pci_stop_and_remove_bus_device(dev); /* * Ensure that no new Requests will be generated from