From patchwork Fri Jan 6 06:05:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: JeeHeng Sia X-Patchwork-Id: 39951 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp668738wrt; Thu, 5 Jan 2023 22:16:26 -0800 (PST) X-Google-Smtp-Source: AMrXdXsJ1jCdb0/Li50wOgpX9VRdpigvKHOyxq76stX3Rux0nbK6fzKx5QS45Xt3XffobkRndsKe X-Received: by 2002:a17:907:8c86:b0:7c1:1adc:46fd with SMTP id td6-20020a1709078c8600b007c11adc46fdmr50259615ejc.34.1672985786192; Thu, 05 Jan 2023 22:16:26 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1672985786; cv=none; d=google.com; s=arc-20160816; b=S0rrfpuYk1O5D4Qp4rJYxv/4Pz0BrIqgc8VTIvg9K7aE1CwwihNuHjHVwTdTN8Qep6 ApSGTTdP6pIAvMCgn2S5HMiLBQRMwlYVZ1vepxx3fdxF+6fl735Ym9xKb1GhBnsFx3/C 8y6wwQ0crb6fKTYEZ/H9xcoyS4m3CX0jwpmhvjRh4RaBl18qeR56OQzSI06y+wj8Mf0g VzrTyXCdFt3BTq0bIhogfpSvTDS/vxI9DFel7J7MWM030x0njatt646OurUVe6y6uLMP +m94bYjY7BGvENSkfuq7ScD6JL7iBmO+lRbXGoFLqtNkSwFgwxGH87RTC59TZ/6KUFCB 3IbA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=8tD6XVBYIrdB1rXsVbv3PFTqO12NGsVNAj1/vrSNP78=; b=eKY1xYMEjzrcKSP5ozBEgA7+2UxcBDU3CHmz5/6EZrJHvEGsTma718MgJroy00Ykf9 PjsHOmP4jH8cCvTzg9XHVVx2/W1a8XaZDzPmdC8M6OD+73fXKRe9PapMuH+BSFl2s1Wu tomhBXZfbuq7twQ+8kfVn1b0xzhutdcgLQbHAXyXJYQdpzjY/BU8JVg6TN/B6sQwIrLd om+8eLjY+B3RJNBMme4JWQpbMiVMi4qIOkWzrvoPElnHudQLAkzrfo2IzPE8B3McESRt DmznZZ69h70CrCr3NUk9HRablHM4Cl6B75DzcJySA/WB5R+UT3dZITliE/8bS9F+9D1G e7gw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id ga38-20020a1709070c2600b0073d9f16e5b3si456330ejc.258.2023.01.05.22.16.01; Thu, 05 Jan 2023 22:16:26 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231315AbjAFGGA convert rfc822-to-8bit (ORCPT + 99 others); Fri, 6 Jan 2023 01:06:00 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50454 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229516AbjAFGF4 (ORCPT ); Fri, 6 Jan 2023 01:05:56 -0500 Received: from fd01.gateway.ufhost.com (fd01.gateway.ufhost.com [61.152.239.71]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 83C6B6147A for ; Thu, 5 Jan 2023 22:05:52 -0800 (PST) Received: from EXMBX165.cuchost.com (unknown [175.102.18.54]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "EXMBX165", Issuer "EXMBX165" (not verified)) by fd01.gateway.ufhost.com (Postfix) with ESMTP id 301AB24E02F; Fri, 6 Jan 2023 14:05:50 +0800 (CST) Received: from EXMBX066.cuchost.com (172.16.7.66) by EXMBX165.cuchost.com (172.16.6.75) with Microsoft SMTP Server (TLS) id 15.0.1497.42; Fri, 6 Jan 2023 14:05:50 +0800 Received: from jsia-virtual-machine.localdomain (202.188.176.82) by EXMBX066.cuchost.com (172.16.6.66) with Microsoft SMTP Server (TLS) id 15.0.1497.42; Fri, 6 Jan 2023 14:05:46 +0800 From: Sia Jee Heng To: , , CC: , , , , Subject: [PATCH 1/3] RISC-V: Change suspend_save_csrs and suspend_restore_csrs to public function Date: Fri, 6 Jan 2023 14:05:33 +0800 Message-ID: <20230106060535.104321-2-jeeheng.sia@starfivetech.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230106060535.104321-1-jeeheng.sia@starfivetech.com> References: <20230106060535.104321-1-jeeheng.sia@starfivetech.com> MIME-Version: 1.0 X-Originating-IP: [202.188.176.82] X-ClientProxiedBy: EXCAS061.cuchost.com (172.16.6.21) To EXMBX066.cuchost.com (172.16.6.66) X-YovoleRuleAgent: yovoleflag X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RCVD_IN_MSPIKE_H2, SPF_HELO_PASS,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1754252743849220635?= X-GMAIL-MSGID: =?utf-8?q?1754252743849220635?= Currently suspend_save_csrs() and suspend_restore_csrs() functions are statically defined in the suspend.c. Change the function's attribute to public so that the functions can be used by hibernation as well. Signed-off-by: Sia Jee Heng Reviewed-by: Ley Foon Tan Reviewed-by: Mason Huo --- arch/riscv/include/asm/suspend.h | 3 +++ arch/riscv/kernel/suspend.c | 4 ++-- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/arch/riscv/include/asm/suspend.h b/arch/riscv/include/asm/suspend.h index 8be391c2aecb..75419c5ca272 100644 --- a/arch/riscv/include/asm/suspend.h +++ b/arch/riscv/include/asm/suspend.h @@ -33,4 +33,7 @@ int cpu_suspend(unsigned long arg, /* Low-level CPU resume entry function */ int __cpu_resume_enter(unsigned long hartid, unsigned long context); +/* Used to save and restore the csr */ +void suspend_save_csrs(struct suspend_context *context); +void suspend_restore_csrs(struct suspend_context *context); #endif diff --git a/arch/riscv/kernel/suspend.c b/arch/riscv/kernel/suspend.c index 9ba24fb8cc93..3c89b8ec69c4 100644 --- a/arch/riscv/kernel/suspend.c +++ b/arch/riscv/kernel/suspend.c @@ -8,7 +8,7 @@ #include #include -static void suspend_save_csrs(struct suspend_context *context) +void suspend_save_csrs(struct suspend_context *context) { context->scratch = csr_read(CSR_SCRATCH); context->tvec = csr_read(CSR_TVEC); @@ -29,7 +29,7 @@ static void suspend_save_csrs(struct suspend_context *context) #endif } -static void suspend_restore_csrs(struct suspend_context *context) +void suspend_restore_csrs(struct suspend_context *context) { csr_write(CSR_SCRATCH, context->scratch); csr_write(CSR_TVEC, context->tvec); From patchwork Fri Jan 6 06:05:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: JeeHeng Sia X-Patchwork-Id: 39950 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp668372wrt; Thu, 5 Jan 2023 22:15:16 -0800 (PST) X-Google-Smtp-Source: AMrXdXsPJ7Fg6HgcFWS9UbnlvWEgQYYQzvf6wsypZV2w/x4kWfbixD86YKWmqrJZhYmwoAX52q/5 X-Received: by 2002:a05:6402:3787:b0:48f:a9a2:29f4 with SMTP id et7-20020a056402378700b0048fa9a229f4mr7010694edb.1.1672985716836; Thu, 05 Jan 2023 22:15:16 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1672985716; cv=none; d=google.com; s=arc-20160816; b=x4Krz5oBvRvPXfTFY+6FFn//MZuYEGcp7FuTdzsarikf7F0BlJ4owlFdAMrGaQ2DG4 JRB+DxRMOqjeHxr8VUlSFDVccgds3RMiIImTScWr/K0u3VSBzPWBp02UKcbhBG9RVO+W cUFDEgv8H2FGxgFITygDENAmmOhTusfg5NUFKLK8KdlY8aHlg0XNl9gBaHslTHkB3IIp sahqmTy8aJ0Msb5aOaI3wpGkPjHPjWL5MYvRHEYt+qa9zpVkoK6ge3zuiApntRq1YXXj 4CY/DoQJYs0d66FfNqaV6lw1GSbkUB0YoNcCAkA7aSfnzLQpR523rmLOm5BMg1tc9jjw RoZg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=mFlxVqZIOdv3euFwLyuuaJdG6cZPX5ZgtTg+Jd3OndQ=; b=WOiA30EPE9tfOT72qycRdZNrrIiU+U9ZwbKDwqYWu333kzEuEdSBaOk0KIu5AdLx5d FbFRD+Z0w/DuHigaFqeZ//taAp6IqOK3VCYUojbwILgKJpem2ytI23KbskgqEWWAuccH 9uMg9cDyEv2EvOubJAQDt5ojPkhvUfYjHjHhvW3QwCrbms6UJaqUmyKub49uHW3Ya6LI lvHevBMZA7Ije+Sf/UOxz0HptMVHN3pjLj6QwNPyzc/nZfS3Fr6EDHJ9hJE1rdAYggYT vki6jRzCXFLeAmHbbRHBlNBfl4SvEv/TqOwH9XiEt3Yqt7zSzb0+KFOY/HL3KFZFfvnh en+w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id y10-20020a056402134a00b00458e689f41fsi571699edw.415.2023.01.05.22.14.50; Thu, 05 Jan 2023 22:15:16 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231496AbjAFGGD convert rfc822-to-8bit (ORCPT + 99 others); Fri, 6 Jan 2023 01:06:03 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50460 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230344AbjAFGF6 (ORCPT ); Fri, 6 Jan 2023 01:05:58 -0500 Received: from fd01.gateway.ufhost.com (fd01.gateway.ufhost.com [61.152.239.71]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1B8C91C41E for ; Thu, 5 Jan 2023 22:05:57 -0800 (PST) Received: from EXMBX166.cuchost.com (unknown [175.102.18.54]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "EXMBX166", Issuer "EXMBX166" (not verified)) by fd01.gateway.ufhost.com (Postfix) with ESMTP id 8FBB424E1B3; Fri, 6 Jan 2023 14:05:55 +0800 (CST) Received: from EXMBX066.cuchost.com (172.16.7.66) by EXMBX166.cuchost.com (172.16.6.76) with Microsoft SMTP Server (TLS) id 15.0.1497.42; Fri, 6 Jan 2023 14:05:55 +0800 Received: from jsia-virtual-machine.localdomain (202.188.176.82) by EXMBX066.cuchost.com (172.16.6.66) with Microsoft SMTP Server (TLS) id 15.0.1497.42; Fri, 6 Jan 2023 14:05:51 +0800 From: Sia Jee Heng To: , , CC: , , , , Subject: [PATCH 2/3] RISC-V: mm: Enable huge page support to kernel_page_present() function Date: Fri, 6 Jan 2023 14:05:34 +0800 Message-ID: <20230106060535.104321-3-jeeheng.sia@starfivetech.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230106060535.104321-1-jeeheng.sia@starfivetech.com> References: <20230106060535.104321-1-jeeheng.sia@starfivetech.com> MIME-Version: 1.0 X-Originating-IP: [202.188.176.82] X-ClientProxiedBy: EXCAS061.cuchost.com (172.16.6.21) To EXMBX066.cuchost.com (172.16.6.66) X-YovoleRuleAgent: yovoleflag X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RCVD_IN_MSPIKE_H2, SPF_HELO_PASS,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1754252671288429542?= X-GMAIL-MSGID: =?utf-8?q?1754252671288429542?= Currently kernel_page_present() function doesn't support huge page detection causes the function to mistakenly return false to the hibernation core. Add huge page detection to the function to solve the problem. Signed-off-by: Sia Jee Heng Reviewed-by: Ley Foon Tan Reviewed-by: Mason Huo --- arch/riscv/mm/pageattr.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/arch/riscv/mm/pageattr.c b/arch/riscv/mm/pageattr.c index 86c56616e5de..fb59d494d708 100644 --- a/arch/riscv/mm/pageattr.c +++ b/arch/riscv/mm/pageattr.c @@ -221,14 +221,20 @@ bool kernel_page_present(struct page *page) p4d = p4d_offset(pgd, addr); if (!p4d_present(*p4d)) return false; + if (pgtable_l5_enabled && p4d_leaf(*pud)) + return true; pud = pud_offset(p4d, addr); if (!pud_present(*pud)) return false; + if (pgtable_l4_enabled && pud_leaf(*pud)) + return true; pmd = pmd_offset(pud, addr); if (!pmd_present(*pmd)) return false; + if (pmd_leaf(*pmd)) + return true; pte = pte_offset_kernel(pmd, addr); return pte_present(*pte); From patchwork Fri Jan 6 06:05:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: JeeHeng Sia X-Patchwork-Id: 39949 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp666796wrt; Thu, 5 Jan 2023 22:10:35 -0800 (PST) X-Google-Smtp-Source: AMrXdXvORo9FFkKpKpUaQpO80IkyUDMWB4Hw89iBFqo5+i/g+yR7LKc9P1of0Uz3Chf+45/fHR+p X-Received: by 2002:a17:902:8f97:b0:192:ad96:10d4 with SMTP id z23-20020a1709028f9700b00192ad9610d4mr23829265plo.7.1672985435140; Thu, 05 Jan 2023 22:10:35 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1672985435; cv=none; d=google.com; s=arc-20160816; b=DM8Di30B6odzHpm0JLMDotbJRknR5knkWqUlWh6mg+o7+O568xmwoEAYkm36Ag8Ar1 vzssZLFROpRADi98sJ81SdDyGx7p/2Av8p7Fytxj5pDTrT3jiusiAMhB63STkgu7ZuTI H88bLWlsbSmnwTYppf4zaFzdooE4ZvIgV3xkXyjXOMqot7CNdSw9xgfMphs2vCGbZeEc DROxDUcpVrjWwHyFFwmdzwQMu4Hggr1QJ52UV7YD9eGdqVWwJvsMSc2siJdAfXOPrECf gpMBTqNFT5Uqs8qAMkllDnVQggEveOhw3PnvdvTYldSnp41gEM/ufYli2jhXll1q5vmA 8Aww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=lJeNytPqPdJpcBfXtphzz+xigxKCHSPgZajL5h8JYPs=; b=xZ/SgYCZFkQ3dtLkPOjrGtWcz9DdWp27P+q9XTm/vfcX1w4sfzMa0i57KyIbuR1/N2 6pzbroSvwCxb+9mL/qFHbpNWcJ6h5mwv9CRZXT13Q4RQGCADytVLobrNW8bEXlWzuCat 4oLEApJoffro80+6/34L6lN1ShAksy7yKoewfKln7wjc/hPXSmpewFPkDngCzqyUcZrI 2CHkYJgkQ2YPecaoMp0TPS2m14631VoN9ZAO2WfYbJQx8xyv/bzPwAc5rJMZocO7Ayw0 IhfVLxJBhIZq5HuKsyLdE/KQrqkdMHsgbO7RJYR9SbEkXEUETQd+9xtJaUe0qaQxv+BT pacw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id p8-20020a170902e74800b001871c762263si114121plf.185.2023.01.05.22.10.22; Thu, 05 Jan 2023 22:10:35 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231643AbjAFGGL convert rfc822-to-8bit (ORCPT + 99 others); Fri, 6 Jan 2023 01:06:11 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50558 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231344AbjAFGGH (ORCPT ); Fri, 6 Jan 2023 01:06:07 -0500 Received: from ex01.ufhost.com (ex01.ufhost.com [61.152.239.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0A7A56B588 for ; Thu, 5 Jan 2023 22:06:02 -0800 (PST) Received: from EXMBX165.cuchost.com (unknown [175.102.18.54]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "EXMBX165", Issuer "EXMBX165" (not verified)) by ex01.ufhost.com (Postfix) with ESMTP id E11AB24E1F0; Fri, 6 Jan 2023 14:06:00 +0800 (CST) Received: from EXMBX066.cuchost.com (172.16.7.66) by EXMBX165.cuchost.com (172.16.6.75) with Microsoft SMTP Server (TLS) id 15.0.1497.42; Fri, 6 Jan 2023 14:06:00 +0800 Received: from jsia-virtual-machine.localdomain (202.188.176.82) by EXMBX066.cuchost.com (172.16.6.66) with Microsoft SMTP Server (TLS) id 15.0.1497.42; Fri, 6 Jan 2023 14:05:56 +0800 From: Sia Jee Heng To: , , CC: , , , , Subject: [PATCH 3/3] RISC-V: Add arch functions to support hibernation/suspend-to-disk Date: Fri, 6 Jan 2023 14:05:35 +0800 Message-ID: <20230106060535.104321-4-jeeheng.sia@starfivetech.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230106060535.104321-1-jeeheng.sia@starfivetech.com> References: <20230106060535.104321-1-jeeheng.sia@starfivetech.com> MIME-Version: 1.0 X-Originating-IP: [202.188.176.82] X-ClientProxiedBy: EXCAS061.cuchost.com (172.16.6.21) To EXMBX066.cuchost.com (172.16.6.66) X-YovoleRuleAgent: yovoleflag X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1754252375351254524?= X-GMAIL-MSGID: =?utf-8?q?1754252375351254524?= Low level Arch functions were created to support hibernation. swsusp_arch_suspend() relies code from __cpu_suspend_enter() to write cpu state onto the stack, then calling swsusp_save() to save the memory image. arch_hibernation_header_restore() and arch_hibernation_header_save() functions are implemented to prevent kernel crash when resume, the kernel built version is saved into the hibernation image header to making sure only the same kernel is restore when resume. swsusp_arch_resume() creates a temporary page table that covering only the linear map, copies the restore code to a 'safe' page, then start to restore the memory image. Once completed, it restores the original kernel's page table. It then calls into __hibernate_cpu_resume() to restore the CPU context. Finally, it follows the normal hibernation path back to the hibernation core. To enable hibernation/suspend to disk into RISCV, the below config need to be enabled: - CONFIG_ARCH_HIBERNATION_HEADER - CONFIG_ARCH_HIBERNATION_POSSIBLE Signed-off-by: Sia Jee Heng Reviewed-by: Ley Foon Tan Reviewed-by: Mason Huo --- arch/riscv/Kconfig | 7 + arch/riscv/include/asm/suspend.h | 20 ++ arch/riscv/kernel/Makefile | 2 +- arch/riscv/kernel/asm-offsets.c | 5 + arch/riscv/kernel/hibernate-asm.S | 123 +++++++++++ arch/riscv/kernel/hibernate.c | 353 ++++++++++++++++++++++++++++++ 6 files changed, 509 insertions(+), 1 deletion(-) create mode 100644 arch/riscv/kernel/hibernate-asm.S create mode 100644 arch/riscv/kernel/hibernate.c diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index e2b656043abf..50474d3aa62e 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -690,6 +690,13 @@ menu "Power management options" source "kernel/power/Kconfig" +config ARCH_HIBERNATION_POSSIBLE + def_bool y + +config ARCH_HIBERNATION_HEADER + def_bool y + depends on HIBERNATION + endmenu # "Power management options" menu "CPU Power Management" diff --git a/arch/riscv/include/asm/suspend.h b/arch/riscv/include/asm/suspend.h index 75419c5ca272..ebaf103aec40 100644 --- a/arch/riscv/include/asm/suspend.h +++ b/arch/riscv/include/asm/suspend.h @@ -21,6 +21,11 @@ struct suspend_context { #endif }; +/* This parameter will be assigned to 0 during resume and will be used by + * hibernation core for the subsequent resume sequence + */ +extern int in_suspend; + /* Low-level CPU suspend entry function */ int __cpu_suspend_enter(struct suspend_context *context); @@ -36,4 +41,19 @@ int __cpu_resume_enter(unsigned long hartid, unsigned long context); /* Used to save and restore the csr */ void suspend_save_csrs(struct suspend_context *context); void suspend_restore_csrs(struct suspend_context *context); + +/* Low-level API to support hibernation */ +int swsusp_arch_suspend(void); +int swsusp_arch_resume(void); +int arch_hibernation_header_save(void *addr, unsigned int max_size); +int arch_hibernation_header_restore(void *addr); +int __hibernate_cpu_resume(void); + +/* Used to resume on the CPU we hibernated on */ +int hibernate_resume_nonboot_cpu_disable(void); + +/* Used to restore the hibernated image */ +asmlinkage void restore_image(unsigned long resume_satp, unsigned long satp_temp, + unsigned long cpu_resume); +asmlinkage int core_restore_code(void); #endif diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile index 4cf303a779ab..df83b8cea631 100644 --- a/arch/riscv/kernel/Makefile +++ b/arch/riscv/kernel/Makefile @@ -64,7 +64,7 @@ obj-$(CONFIG_MODULES) += module.o obj-$(CONFIG_MODULE_SECTIONS) += module-sections.o obj-$(CONFIG_CPU_PM) += suspend_entry.o suspend.o - +obj-$(CONFIG_HIBERNATION) += hibernate.o hibernate-asm.o obj-$(CONFIG_FUNCTION_TRACER) += mcount.o ftrace.o obj-$(CONFIG_DYNAMIC_FTRACE) += mcount-dyn.o diff --git a/arch/riscv/kernel/asm-offsets.c b/arch/riscv/kernel/asm-offsets.c index df9444397908..d6a75aac1d27 100644 --- a/arch/riscv/kernel/asm-offsets.c +++ b/arch/riscv/kernel/asm-offsets.c @@ -9,6 +9,7 @@ #include #include #include +#include #include #include #include @@ -116,6 +117,10 @@ void asm_offsets(void) OFFSET(SUSPEND_CONTEXT_REGS, suspend_context, regs); + OFFSET(HIBERN_PBE_ADDR, pbe, address); + OFFSET(HIBERN_PBE_ORIG, pbe, orig_address); + OFFSET(HIBERN_PBE_NEXT, pbe, next); + OFFSET(KVM_ARCH_GUEST_ZERO, kvm_vcpu_arch, guest_context.zero); OFFSET(KVM_ARCH_GUEST_RA, kvm_vcpu_arch, guest_context.ra); OFFSET(KVM_ARCH_GUEST_SP, kvm_vcpu_arch, guest_context.sp); diff --git a/arch/riscv/kernel/hibernate-asm.S b/arch/riscv/kernel/hibernate-asm.S new file mode 100644 index 000000000000..81d9dc98d0ad --- /dev/null +++ b/arch/riscv/kernel/hibernate-asm.S @@ -0,0 +1,123 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Hibernation support specific for RISCV + * + * Copyright (C) 2023 StarFive Technology Co., Ltd. + * + * Author: Jee Heng Sia + */ + +#include +#include +#include +#include + +/* + * These code are to be executed when resume from the hibernation. + * + * It begins with loads the temporary page table then restores the memory image. + * Finally branches to __hibernate_cpu_resume() to restore the state saved by + * swsusp_arch_suspend(). + */ + +/* + * int __hibernate_cpu_resume(void) + * Switch back to the hibernated image's page table prior to restore the CPU + * context. + * + * Always returns 0 to the C code. + */ +ENTRY(__hibernate_cpu_resume) + /* switch to hibernated image's page table */ + csrw CSR_SATP, s0 + sfence.vma + + ld a0, hibernate_cpu_context + + /* Restore CSRs */ + REG_L t0, (SUSPEND_CONTEXT_REGS + PT_EPC)(a0) + csrw CSR_EPC, t0 + REG_L t0, (SUSPEND_CONTEXT_REGS + PT_STATUS)(a0) + csrw CSR_STATUS, t0 + REG_L t0, (SUSPEND_CONTEXT_REGS + PT_BADADDR)(a0) + csrw CSR_TVAL, t0 + REG_L t0, (SUSPEND_CONTEXT_REGS + PT_CAUSE)(a0) + csrw CSR_CAUSE, t0 + + /* Restore registers (except A0 and T0-T6) */ + REG_L ra, (SUSPEND_CONTEXT_REGS + PT_RA)(a0) + REG_L sp, (SUSPEND_CONTEXT_REGS + PT_SP)(a0) + REG_L gp, (SUSPEND_CONTEXT_REGS + PT_GP)(a0) + REG_L tp, (SUSPEND_CONTEXT_REGS + PT_TP)(a0) + + REG_L s0, (SUSPEND_CONTEXT_REGS + PT_S0)(a0) + REG_L s1, (SUSPEND_CONTEXT_REGS + PT_S1)(a0) + REG_L a1, (SUSPEND_CONTEXT_REGS + PT_A1)(a0) + REG_L a2, (SUSPEND_CONTEXT_REGS + PT_A2)(a0) + REG_L a3, (SUSPEND_CONTEXT_REGS + PT_A3)(a0) + REG_L a4, (SUSPEND_CONTEXT_REGS + PT_A4)(a0) + REG_L a5, (SUSPEND_CONTEXT_REGS + PT_A5)(a0) + REG_L a6, (SUSPEND_CONTEXT_REGS + PT_A6)(a0) + REG_L a7, (SUSPEND_CONTEXT_REGS + PT_A7)(a0) + REG_L s2, (SUSPEND_CONTEXT_REGS + PT_S2)(a0) + REG_L s3, (SUSPEND_CONTEXT_REGS + PT_S3)(a0) + REG_L s4, (SUSPEND_CONTEXT_REGS + PT_S4)(a0) + REG_L s5, (SUSPEND_CONTEXT_REGS + PT_S5)(a0) + REG_L s6, (SUSPEND_CONTEXT_REGS + PT_S6)(a0) + REG_L s7, (SUSPEND_CONTEXT_REGS + PT_S7)(a0) + REG_L s8, (SUSPEND_CONTEXT_REGS + PT_S8)(a0) + REG_L s9, (SUSPEND_CONTEXT_REGS + PT_S9)(a0) + REG_L s10, (SUSPEND_CONTEXT_REGS + PT_S10)(a0) + REG_L s11, (SUSPEND_CONTEXT_REGS + PT_S11)(a0) + + /* Return zero value */ + add a0, zero, zero + + ret +END(__hibernate_cpu_resume) + +/* + * Prepare to restore the image. + * a0: satp of saved page tables + * a1: satp of temporary page tables + * a2: cpu_resume + */ +ENTRY(restore_image) + mv s0, a0 + mv s1, a1 + mv s2, a2 + ld s4, restore_pblist + ld a1, relocated_restore_code + + jalr a1 +END(restore_image) + +/* + * The below code will be executed from a 'safe' page. + * It first switches to the temporary page table, then start to copy the pages + * back to the original memory location. Finally, it jumps to the __hibernate_cpu_resume() + * to restore the CPU context. + */ +ENTRY(core_restore_code) + /* switch to temp page table */ + csrw satp, s1 + sfence.vma + beqz s4, done +loop: + /* The below code will restore the hibernated image. */ + ld a1, HIBERN_PBE_ADDR(s4) + ld a0, HIBERN_PBE_ORIG(s4) + + lui a4, 0x1 + add a4, a4, a0 +copy: ld a5, 0(a1) + addi a0, a0, 8 + addi a1, a1, 8 + sd a5, -8(a0) + bne a4, a0, copy + + ld s4, HIBERN_PBE_NEXT(s4) + bnez s4, loop +done: + jalr s2 +END(core_restore_code) diff --git a/arch/riscv/kernel/hibernate.c b/arch/riscv/kernel/hibernate.c new file mode 100644 index 000000000000..78ffad3bdbd8 --- /dev/null +++ b/arch/riscv/kernel/hibernate.c @@ -0,0 +1,353 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Hibernation support specific for RISCV + * + * Copyright (C) 2023 StarFive Technology Co., Ltd. + * + * Author: Jee Heng Sia + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include + +/* + * The logical cpu number we should resume on, initialised to a non-cpu + * number. + */ +static int sleep_cpu = -EINVAL; + +/* CPU context to be saved */ +struct suspend_context *hibernate_cpu_context; + +unsigned long relocated_restore_code; + +/* Pointer to the temporary resume page tables */ +pgd_t *resume_pg_dir; + +/* + * Save the build number and date so that the we are not resume with a different kernel + */ +struct arch_hibernate_hdr_invariants { + char uts_version[__NEW_UTS_LEN + 1]; +}; + +/* Helper parameters that help us to restore the image. + * @hartid: To make sure same boot_cpu executing the hibernate/restore code. + * @saved_satp: Original page table used by the hibernated image. + * @restore_cpu_addr: The kernel's image address to restore the CPU context. + */ +static struct arch_hibernate_hdr { + struct arch_hibernate_hdr_invariants invariants; + unsigned long hartid; + unsigned long saved_satp; + unsigned long restore_cpu_addr; +} resume_hdr; + +static inline void arch_hdr_invariants(struct arch_hibernate_hdr_invariants *i) +{ + memset(i, 0, sizeof(*i)); + memcpy(i->uts_version, init_utsname()->version, sizeof(i->uts_version)); +} + +/* + * Check if the given pfn is in the 'nosave' section. + */ +int pfn_is_nosave(unsigned long pfn) +{ + unsigned long nosave_begin_pfn = sym_to_pfn(&__nosave_begin); + unsigned long nosave_end_pfn = sym_to_pfn(&__nosave_end - 1); + + return ((pfn >= nosave_begin_pfn) && (pfn <= nosave_end_pfn)); +} + +void notrace save_processor_state(void) +{ + WARN_ON(num_online_cpus() != 1); +} + +void notrace restore_processor_state(void) +{ +} + +/* + * Helper parameters need to be saved to the hibernation image header. + */ +int arch_hibernation_header_save(void *addr, unsigned int max_size) +{ + struct arch_hibernate_hdr *hdr = addr; + + if (max_size < sizeof(*hdr)) + return -EOVERFLOW; + + arch_hdr_invariants(&hdr->invariants); + + hdr->hartid = cpuid_to_hartid_map(sleep_cpu); + hdr->saved_satp = csr_read(CSR_SATP); + hdr->restore_cpu_addr = (unsigned long) __hibernate_cpu_resume; + + return 0; +} +EXPORT_SYMBOL(arch_hibernation_header_save); + +/* + * Retrieve the helper parameters from the hibernation image header + */ +int arch_hibernation_header_restore(void *addr) +{ + struct arch_hibernate_hdr_invariants invariants; + struct arch_hibernate_hdr *hdr = addr; + int ret = 0; + + arch_hdr_invariants(&invariants); + + if (memcmp(&hdr->invariants, &invariants, sizeof(invariants))) { + pr_crit("Hibernate image not generated by this kernel!\n"); + return -EINVAL; + } + + sleep_cpu = riscv_hartid_to_cpuid(hdr->hartid); + if (sleep_cpu < 0) { + pr_crit("Hibernated on a CPU not known to this kernel!\n"); + sleep_cpu = -EINVAL; + return -EINVAL; + } + +#ifdef CONFIG_SMP + ret = bringup_hibernate_cpu(sleep_cpu); + if (ret) { + sleep_cpu = -EINVAL; + return ret; + } +#endif + resume_hdr = *hdr; + + return ret; +} +EXPORT_SYMBOL(arch_hibernation_header_restore); + +int swsusp_arch_suspend(void) +{ + int ret = 0; + + if (__cpu_suspend_enter(hibernate_cpu_context)) { + sleep_cpu = smp_processor_id(); + suspend_save_csrs(hibernate_cpu_context); + ret = swsusp_save(); + } else { + suspend_restore_csrs(hibernate_cpu_context); + flush_tlb_all(); + + /* Invalidated Icache */ + flush_icache_all(); + + /* + * Tell the hibernation core that we've just restored + * the memory + */ + in_suspend = 0; + sleep_cpu = -EINVAL; + } + + return ret; +} + +#define temp_pgtable_map_pgd_next(pgdp, vaddr, prot) \ + (pgtable_l5_enabled ? \ + temp_pgtable_map_p4d(pgdp, vaddr, prot) : \ + (pgtable_l4_enabled ? \ + temp_pgtable_map_pud((pud_t *)pgdp, vaddr, prot) : \ + temp_pgtable_map_pmd((pmd_t *)pgdp, vaddr, prot))) + +unsigned long temp_pgtable_map_pte(pte_t *ptep, unsigned long vaddr, pgprot_t prot) +{ + uintptr_t pte_idx = pte_index(vaddr); + + ptep[pte_idx] = pfn_pte(PFN_DOWN(__pa(vaddr)), prot); + + return 0; +} + +unsigned long temp_pgtable_map_pmd(pmd_t *pmdp, unsigned long vaddr, pgprot_t prot) +{ + uintptr_t pmd_idx = pmd_index(vaddr); + pte_t *ptep; + + if (pmd_none(pmdp[pmd_idx])) { + ptep = (pte_t *) get_safe_page(GFP_ATOMIC); + if (!ptep) + return -ENOMEM; + + memset(ptep, 0, PAGE_SIZE); + pmdp[pmd_idx] = pfn_pmd(PFN_DOWN(__pa(ptep)), PAGE_TABLE); + } else { + ptep = (pte_t *) __va(PFN_PHYS(_pmd_pfn(pmdp[pmd_idx]))); + } + + return temp_pgtable_map_pte(ptep, vaddr, prot); +} + +unsigned long temp_pgtable_map_pud(pud_t *pudp, unsigned long vaddr, pgprot_t prot) +{ + + uintptr_t pud_index = pud_index(vaddr); + pmd_t *pmdp; + + if (pud_val(pudp[pud_index]) == 0) { + pmdp = (pmd_t *) get_safe_page(GFP_ATOMIC); + if (!pmdp) + return -ENOMEM; + + memset(pmdp, 0, PAGE_SIZE); + pudp[pud_index] = pfn_pud(PFN_DOWN(__pa(pmdp)), PAGE_TABLE); + } else { + pmdp = (pmd_t *) __va(PFN_PHYS(_pud_pfn(pudp[pud_index]))); + } + + return temp_pgtable_map_pmd(pmdp, vaddr, prot); +} + +unsigned long temp_pgtable_map_p4d(p4d_t *p4dp, unsigned long vaddr, pgprot_t prot) +{ + uintptr_t p4d_index = p4d_index(vaddr); + pud_t *pudp; + + if (p4d_val(p4dp[p4d_index]) == 0) { + pudp = (pud_t *) get_safe_page(GFP_ATOMIC); + if (!pudp) + return -ENOMEM; + + memset(pudp, 0, PAGE_SIZE); + p4dp[p4d_index] = pfn_p4d(PFN_DOWN(__pa(pudp)), PAGE_TABLE); + } else { + pudp = (pud_t *) __va(PFN_PHYS(_p4d_pfn(p4dp[p4d_index]))); + } + + return temp_pgtable_map_pud(pudp, vaddr, prot); +} + +unsigned long temp_pgtable_map_pgd(pgd_t *pgdp, unsigned long vaddr, pgprot_t prot) +{ + uintptr_t pgd_idx = pgd_index(vaddr); + void *nextp; + + if (pgd_val(pgdp[pgd_idx]) == 0) { + nextp = (void *) get_safe_page(GFP_ATOMIC); + if (!nextp) + return -ENOMEM; + + memset(nextp, 0, PAGE_SIZE); + pgdp[pgd_idx] = pfn_pgd(PFN_DOWN(__pa(nextp)), PAGE_TABLE); + } else { + nextp = (void *) __va(PFN_PHYS(_pgd_pfn(pgdp[pgd_idx]))); + } + + return temp_pgtable_map_pgd_next(nextp, vaddr, prot); +} + +unsigned long temp_pgtable_mapping(pgd_t *pgdp, unsigned long vaddr, pgprot_t prot) +{ + return temp_pgtable_map_pgd(pgdp, vaddr, prot); +} + +unsigned long relocate_restore_code(void) +{ + unsigned long ret; + void *page = (void *)get_safe_page(GFP_ATOMIC); + + if (!page) + return -ENOMEM; + + copy_page(page, core_restore_code); + + /* Make the page containing the relocated code executable */ + set_memory_x((unsigned long)page, 1); + + ret = temp_pgtable_mapping(resume_pg_dir, (unsigned long)page, PAGE_KERNEL_READ_EXEC); + if (ret) + return ret; + + return (unsigned long)page; +} + +int swsusp_arch_resume(void) +{ + unsigned long addr = PAGE_OFFSET; + unsigned long ret; + + /* + * Memory allocated by get_safe_page() will be dealt with by the hibernation core, + * we don't need to free it here. + */ + resume_pg_dir = (pgd_t *)get_safe_page(GFP_ATOMIC); + if (!resume_pg_dir) + return -ENOMEM; + + /* + * The pages need to be wrote-able when restoring the image. + * Create a second copy of page table just for the linear map, and use this when + * restoring. + */ + for (; addr <= (unsigned long)pfn_to_virt(max_low_pfn); addr += PAGE_SIZE) { + ret = temp_pgtable_mapping(resume_pg_dir, addr, PAGE_KERNEL); + if (ret) + return (int) ret; + } + + /* Move the restore code to a new page so that it doesn't get overwritten by itself */ + relocated_restore_code = relocate_restore_code(); + if (relocated_restore_code == -ENOMEM) + return -ENOMEM; + + /* Map the __hibernate_cpu_resume() address to the temporary page table so that the + * restore code can jump to it after finished restore the image. The next execution + * code doesn't find itself in a different address space after switching over to the + * original page table used by the hibernated image. + */ + ret = temp_pgtable_mapping(resume_pg_dir, (unsigned long)resume_hdr.restore_cpu_addr, + PAGE_KERNEL_READ_EXEC); + if (ret) + return ret; + + restore_image(resume_hdr.saved_satp, (PFN_DOWN(__pa(resume_pg_dir)) | satp_mode), + resume_hdr.restore_cpu_addr); + + return 0; +} + +#ifdef CONFIG_SMP +int hibernate_resume_nonboot_cpu_disable(void) +{ + if (sleep_cpu < 0) { + pr_err("Failing to resume from hibernate on an unknown CPU.\n"); + return -ENODEV; + } + + return freeze_secondary_cpus(sleep_cpu); +} +#endif + +static int __init riscv_hibernate__init(void) +{ + hibernate_cpu_context = kcalloc(1, sizeof(struct suspend_context), GFP_KERNEL); + + if (WARN_ON(!hibernate_cpu_context)) + return -ENOMEM; + + return 0; +} + +early_initcall(riscv_hibernate__init);