From patchwork Tue Nov 22 12:16:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Conor Dooley X-Patchwork-Id: 24336 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp2168519wrr; Tue, 22 Nov 2022 04:25:13 -0800 (PST) X-Google-Smtp-Source: AA0mqf7dlEknKb99NPDA1Pb1HczC81Rx8UEeWIYvk9f8ZgSYt3JvBFYFiJWlW3ZcOubnNB6rEmOg X-Received: by 2002:a17:90b:378d:b0:218:9ea5:b081 with SMTP id mz13-20020a17090b378d00b002189ea5b081mr14528749pjb.105.1669119912692; Tue, 22 Nov 2022 04:25:12 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669119912; cv=none; d=google.com; s=arc-20160816; b=Evh6N2IpVnfhlBzopAS/NFunPlggH5Ru7W0HNbOYGI51dSKsUGyX6oGIxKZAbwlawV r26qeWt6TbsV5i1Mk2txqRqNrNniwEhhcdV3nPFfqUC91BVQbqgLTxT6YMgMNVy3Snkl DFzNPDaqf8CrLlzgz75Fe+a6AdZHzQTyL7qD6XSpFt1sIRRec8oX3nV7GbIjiKYYpyU2 5HvlTXssaO2bh8CjcgRXrHeKxuv0oEeBd19ECDCC5h7pT1hOUzRmldMT7pv1MseBf1ho CW5H0AHq6QBv5fcX83iQ1ys7QgxfIJBtfcTOvrQZIIJBqLYGEeY997x0/p8VhkcXOpAD +iEA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=VYUqeCXQxkeJEPXlGhcZ5NQH/SDg4j42w2WLflpaJSc=; b=S7apxDm21UAPl62o5Qihq8JHew7dgs3C5tQdudsoRRlBKcIPVxpQYPOnTlytyvuvH1 Moi1qKKkSXN0zTuoSIoXLmMmRLTFoznhoj4nu82xPzG6HFkNgg1NE5pMWWpVMPPC9u0Q f4cRigvdEf7iFwQBzjuOHlMAUMw3BhuxV8F3oQEi9FvYJi0lw3zrQlha8FoXC0eeaBi6 7CDv0S8EFFoGGB0Qa+iXAH5sfs9ZfaUrWdjAVIQp2yr3X1GTlb9GRSytMdzjN38zSROP rIY1cbxkowQ1YQE0sqqWUWkhNtQIFNkgGRgejNQeh1FJfnyj84d6Yo3nPGMi9Btueq64 iIgg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@microchip.com header.s=mchp header.b=1oM+TlZZ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=microchip.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id a11-20020a056a000c8b00b0056301324a24si14878716pfv.133.2022.11.22.04.24.58; Tue, 22 Nov 2022 04:25:12 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@microchip.com header.s=mchp header.b=1oM+TlZZ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=microchip.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233706AbiKVMRT (ORCPT + 99 others); Tue, 22 Nov 2022 07:17:19 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60476 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233680AbiKVMRR (ORCPT ); Tue, 22 Nov 2022 07:17:17 -0500 Received: from esa.microchip.iphmx.com (esa.microchip.iphmx.com [68.232.153.233]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7AB1412A8A for ; Tue, 22 Nov 2022 04:17:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=microchip.com; i=@microchip.com; q=dns/txt; s=mchp; t=1669119431; x=1700655431; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=RzUAmT+mOx/mY30QUVZdyICgGAvt8beLzIIw/PvKyEU=; b=1oM+TlZZlIx6r+1wg9D3Mm5MUDRWKVGKY9UNgjBdlEjfImKa6XAJ1kDm jMNqGHvSco0UIiJdUAKYFR19CL2QDA8mKkgsZyc6BdD7HiwJlnrL4Zf13 lrCxDOk5dV/KlQ+7uTD/dJ8re2LpGAPQOoEZap8wnqNzNjUCOQPmjkvIl QCFYPljdPhOG6huM7LZqEa1T5YilOvPVNsDBL3a84+ZAbd++X/rvvdp/S DDAqR4xpY4dyiVD4VHHWAmYWYPN3OlfBTja+812zU1TwPqCC4U9eKueyd uWHKQ0Ztg3JwaTTvQCOZYkYTV39YXlCfvjrSuo0hdNby9eRre99C9m9rS A==; X-IronPort-AV: E=Sophos;i="5.96,183,1665471600"; d="scan'208";a="200899064" Received: from unknown (HELO email.microchip.com) ([170.129.1.10]) by esa1.microchip.iphmx.com with ESMTP/TLS/AES256-SHA256; 22 Nov 2022 05:17:10 -0700 Received: from chn-vm-ex03.mchp-main.com (10.10.85.151) by chn-vm-ex03.mchp-main.com (10.10.85.151) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.12; Tue, 22 Nov 2022 05:17:05 -0700 Received: from wendy.microchip.com (10.10.115.15) by chn-vm-ex03.mchp-main.com (10.10.85.151) with Microsoft SMTP Server id 15.1.2507.12 via Frontend Transport; Tue, 22 Nov 2022 05:17:03 -0700 From: Conor Dooley To: , CC: Conor Dooley , Samuel Holland , Anup Patel , Palmer Dabbelt , Palmer Dabbelt , , , , , , Subject: [PATCH v2] Revert "clocksource/drivers/riscv: Events are stopped during CPU suspend" Date: Tue, 22 Nov 2022 12:16:21 +0000 Message-ID: <20221122121620.3522431-1-conor.dooley@microchip.com> X-Mailer: git-send-email 2.38.1 MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750199081559725636?= X-GMAIL-MSGID: =?utf-8?q?1750199081559725636?= This reverts commit 232ccac1bd9b5bfe73895f527c08623e7fa0752d. On the subject of suspend, the RISC-V SBI spec states: > Request the SBI implementation to put the calling hart in a platform > specific suspend (or low power) state specified by the suspend_type > parameter. The hart will automatically come out of suspended state and > resume normal execution when it receives an interrupt or platform > specific hardware event. This does not cover whether any given events actually reach the hart or not, just what the hart will do if it receives an event. On PolarFire SoC, and potentially other SiFive based implementations, events from the RISC-V timer do reach a hart during suspend. This is not the case for the implementation on the Allwinner D1 - there timer events are not received during suspend. To fix this, the x86 C3STOP feature was enabled for the timer driver - but this has broken both RCU stall detection and timers generally on PolarFire SoC (and potentially other SiFive based implementations). If an AXI read to the PCIe controller on PolarFire SoC times out, the system will stall, however, with this patch applied, the system just locks up without RCU stalling: io scheduler mq-deadline registered io scheduler kyber registered microchip-pcie 2000000000.pcie: host bridge /soc/pcie@2000000000 ranges: microchip-pcie 2000000000.pcie: MEM 0x2008000000..0x2087ffffff -> 0x0008000000 microchip-pcie 2000000000.pcie: sec error in pcie2axi buffer microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer microchip-pcie 2000000000.pcie: axi read request error microchip-pcie 2000000000.pcie: axi read timeout microchip-pcie 2000000000.pcie: sec error in pcie2axi buffer microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer microchip-pcie 2000000000.pcie: sec error in pcie2axi buffer microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer microchip-pcie 2000000000.pcie: sec error in pcie2axi buffer microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer Freeing initrd memory: 7332K Similarly issues were reported with clock_nanosleep() - with a test app that sleeps each cpu for 6, 5, 4, 3 ms respectively, HZ=250 & the blamed commit in place, the sleep times are rounded up to the next jiffy: == CPU: 1 == == CPU: 2 == == CPU: 3 == == CPU: 4 == Mean: 7.974992 Mean: 7.976534 Mean: 7.962591 Mean: 3.952179 Std Dev: 0.154374 Std Dev: 0.156082 Std Dev: 0.171018 Std Dev: 0.076193 Hi: 9.472000 Hi: 10.495000 Hi: 8.864000 Hi: 4.736000 Lo: 6.087000 Lo: 6.380000 Lo: 4.872000 Lo: 3.403000 Samples: 521 Samples: 521 Samples: 521 Samples: 521 Fortunately, the D1 has a second timer, which is "currently used in preference to the RISC-V/SBI timer driver" so a revert here does not hurt operation of D1 in it's current form. Ultimately, a DeviceTree property (or node) will be added to encode the behaviour of the timers, but until then revert the addition of CLOCK_EVT_FEAT_C3STOP. Link: https://lore.kernel.org/linux-riscv/YzYTNQRxLr7Q9JR0@spud/ Link: https://github.com/riscv-non-isa/riscv-sbi-doc/issues/98/ Link: https://lore.kernel.org/linux-riscv/bf6d3b1f-f703-4a25-833e-972a44a04114@sholland.org/ Fixes: 232ccac1bd9b ("clocksource/drivers/riscv: Events are stopped during CPU suspend") CC: Samuel Holland CC: Anup Patel CC: Palmer Dabbelt Reviewed-by: Palmer Dabbelt Acked-by: Palmer Dabbelt Signed-off-by: Conor Dooley Acked-by: Samuel Holland --- For v2, I have re-worked the commit message to (hopefully) add improved context. CC: aou@eecs.berkeley.edu CC: atishp@atishpatra.org CC: daniel.lezcano@linaro.org CC: dmitriy@oss-tech.org CC: paul.walmsley@sifive.com CC: tglx@linutronix.de CC: linux-kernel@vger.kernel.org CC: linux-riscv@lists.infradead.org --- drivers/clocksource/timer-riscv.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/clocksource/timer-riscv.c b/drivers/clocksource/timer-riscv.c index 969a552da8d2..a0d66fabf073 100644 --- a/drivers/clocksource/timer-riscv.c +++ b/drivers/clocksource/timer-riscv.c @@ -51,7 +51,7 @@ static int riscv_clock_next_event(unsigned long delta, static unsigned int riscv_clock_event_irq; static DEFINE_PER_CPU(struct clock_event_device, riscv_clock_event) = { .name = "riscv_timer_clockevent", - .features = CLOCK_EVT_FEAT_ONESHOT | CLOCK_EVT_FEAT_C3STOP, + .features = CLOCK_EVT_FEAT_ONESHOT, .rating = 100, .set_next_event = riscv_clock_next_event, };