From patchwork Fri Jan 27 09:10:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: JeeHeng Sia X-Patchwork-Id: 49119 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp734305wrn; Fri, 27 Jan 2023 01:18:15 -0800 (PST) X-Google-Smtp-Source: AMrXdXsUgVPL4vLOQcHG9zCoNCSoYv2GqLJd39+2Rw1+ZJMxwOQmnDqgVcJdpgUlwxlPQEaqYv8y X-Received: by 2002:aa7:c845:0:b0:497:b6bc:b811 with SMTP id g5-20020aa7c845000000b00497b6bcb811mr38968854edt.33.1674811095079; Fri, 27 Jan 2023 01:18:15 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674811095; cv=none; d=google.com; s=arc-20160816; b=nkVvuqvJY0oxJhw+6HgV0SahPw2q/LV4y0rV3zE6fSwYGtPD/vKSnzTCa3KQ7EobeI BIwRa0udMVIhCWgsWtj0yODwcclxU7t5PZ+/UzUfij0QzusrrwGuJUFs34D457Wllnr2 Gi1Y4Rf01pEfnj9eaLjmmvsE66kVdw7G4G6ajDi8gejQ6vZz72monvocxogFPQppT4rB MA69iY9eOdceApEm84zhsHT2v2+KFXHhFNdGiIp+xUNEkqa761I9t2y1KMqDiXWPUItk 5xJl7ew5d4jhlJ1SxH1lOiFCAZEc3q40I9Bv+2SpA+Rwqs37LO4ooaRTSNp4i5F4uwcQ G1BA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=8tD6XVBYIrdB1rXsVbv3PFTqO12NGsVNAj1/vrSNP78=; b=c/OAHucgFFZManiDk4Ak2Yz1bco/BAtUGFWRru8KNx06hLKo+vwNuaT4AzNEUDTutw pbjBzqyDFoG9rvGYa/uwXh+nJ50EItV8ETQwIUiD4VAsTrSBMohHUT1bWq45viNK9rWc 1RCbTGHXoRRBxVaWsFcKg5Aka+XFmZmxvWV0eO4gg9IZHFwY7qNjLKfthOHL9rugcKKH 6LAlhC3L1ToyJsGN8Mbnb0kMdQgF8SlDOme/NARx27bxBCxP+0FhiN3LxeU/82TQChvw I2GYQGaS1VFkB4s+n7KEylRB7usqqStQVbAAieLLcmgwF7z9ButTZq2D63Zh3Jal5Aul Mbhg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id s12-20020aa7c54c000000b0049e28b8b54fsi4255091edr.213.2023.01.27.01.17.50; Fri, 27 Jan 2023 01:18:15 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232728AbjA0JL1 convert rfc822-to-8bit (ORCPT + 99 others); Fri, 27 Jan 2023 04:11:27 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57396 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232823AbjA0JLZ (ORCPT ); Fri, 27 Jan 2023 04:11:25 -0500 Received: from fd01.gateway.ufhost.com (fd01.gateway.ufhost.com [61.152.239.71]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8726B761DB for ; Fri, 27 Jan 2023 01:11:18 -0800 (PST) Received: from EXMBX166.cuchost.com (unknown [175.102.18.54]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "EXMBX166", Issuer "EXMBX166" (not verified)) by fd01.gateway.ufhost.com (Postfix) with ESMTP id 0AD4624DBD3; Fri, 27 Jan 2023 17:11:16 +0800 (CST) Received: from EXMBX066.cuchost.com (172.16.7.66) by EXMBX166.cuchost.com (172.16.6.76) with Microsoft SMTP Server (TLS) id 15.0.1497.42; Fri, 27 Jan 2023 17:11:16 +0800 Received: from jsia-virtual-machine.localdomain (60.50.196.81) by EXMBX066.cuchost.com (172.16.6.66) with Microsoft SMTP Server (TLS) id 15.0.1497.42; Fri, 27 Jan 2023 17:11:10 +0800 From: Sia Jee Heng To: , , CC: , , , , Subject: [PATCH v3 1/4] RISC-V: Change suspend_save_csrs and suspend_restore_csrs to public function Date: Fri, 27 Jan 2023 17:10:48 +0800 Message-ID: <20230127091051.1465278-2-jeeheng.sia@starfivetech.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230127091051.1465278-1-jeeheng.sia@starfivetech.com> References: <20230127091051.1465278-1-jeeheng.sia@starfivetech.com> MIME-Version: 1.0 X-Originating-IP: [60.50.196.81] X-ClientProxiedBy: EXCAS066.cuchost.com (172.16.6.26) To EXMBX066.cuchost.com (172.16.6.66) X-YovoleRuleAgent: yovoleflag X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RCVD_IN_MSPIKE_H2, SPF_HELO_PASS,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756166719290885257?= X-GMAIL-MSGID: =?utf-8?q?1756166719290885257?= Currently suspend_save_csrs() and suspend_restore_csrs() functions are statically defined in the suspend.c. Change the function's attribute to public so that the functions can be used by hibernation as well. Signed-off-by: Sia Jee Heng Reviewed-by: Ley Foon Tan Reviewed-by: Mason Huo Reviewed-by: Conor Dooley --- arch/riscv/include/asm/suspend.h | 3 +++ arch/riscv/kernel/suspend.c | 4 ++-- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/arch/riscv/include/asm/suspend.h b/arch/riscv/include/asm/suspend.h index 8be391c2aecb..75419c5ca272 100644 --- a/arch/riscv/include/asm/suspend.h +++ b/arch/riscv/include/asm/suspend.h @@ -33,4 +33,7 @@ int cpu_suspend(unsigned long arg, /* Low-level CPU resume entry function */ int __cpu_resume_enter(unsigned long hartid, unsigned long context); +/* Used to save and restore the csr */ +void suspend_save_csrs(struct suspend_context *context); +void suspend_restore_csrs(struct suspend_context *context); #endif diff --git a/arch/riscv/kernel/suspend.c b/arch/riscv/kernel/suspend.c index 9ba24fb8cc93..3c89b8ec69c4 100644 --- a/arch/riscv/kernel/suspend.c +++ b/arch/riscv/kernel/suspend.c @@ -8,7 +8,7 @@ #include #include -static void suspend_save_csrs(struct suspend_context *context) +void suspend_save_csrs(struct suspend_context *context) { context->scratch = csr_read(CSR_SCRATCH); context->tvec = csr_read(CSR_TVEC); @@ -29,7 +29,7 @@ static void suspend_save_csrs(struct suspend_context *context) #endif } -static void suspend_restore_csrs(struct suspend_context *context) +void suspend_restore_csrs(struct suspend_context *context) { csr_write(CSR_SCRATCH, context->scratch); csr_write(CSR_TVEC, context->tvec); From patchwork Fri Jan 27 09:10:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: JeeHeng Sia X-Patchwork-Id: 49121 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp734306wrn; Fri, 27 Jan 2023 01:18:15 -0800 (PST) X-Google-Smtp-Source: AK7set9sSNUihcCx31kSftMI/icoZnjvTmzbAXbUdWBImDzrkqVyqa3PXfO4FIdGKtFnF1XBPBXW X-Received: by 2002:a17:906:c28e:b0:87b:db53:3829 with SMTP id r14-20020a170906c28e00b0087bdb533829mr541995ejz.46.1674811095314; Fri, 27 Jan 2023 01:18:15 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674811095; cv=none; d=google.com; s=arc-20160816; b=obNd6LVU6q1GqO7AT9ELBmfd4fk+I3OpHgaWp/AQECiagHMzxxCOQgf8q7MzotNsWF n9+jYzZDRVqE0qWGQz8ENNhBJp0vZJkbpxo2kttiATck32JyrCoUesPGyOauQV1EIEbB eAOYJ2xtjeTw7nmEFkUvGk71z22p4NQ3vC8quuehRIkDlqpCea4J9z1/bjRDJ8LSSh6O Erek4XKW8V6yh3Mw2Pl9TTcxlXuchLeEN9yu/lX3EzPThbzuQbgLs3L4w0QPWOxfajiB jYxG9s7lwjB4JA36utJodWuIjGLSz2qSgKA9ZXoFMpphC5JhX2kQoMl5OEXTBNudaPjq s1Xg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=A5Skxz3m763Oue2a6B7HnHFF34PrLZGJGYPdphBezUE=; b=Bs1Wp7x3h+tOHsSg+jwTX6DrV1OfjhrND1RSagtqi/eSwe4bZ5z5SWQzaHVudswaoj GWbasbZUTPHi2nDDvNnUn56JcUuRlTYhhj8iBWe4E1+L6cT3Z/U6ISttGyBdRfy7KMiy aVk+sLbR7HCDIVy4HX68TCdbl99DlX1M0BujlqrYeDJyiqph77o+1E1tmKChEpOxxEUQ eKIRmA2CqWCv7Ap+W8bTf1I/Est2hrUT2F1dfJZEACX7XOuokQ1blGvT60WTxSKhEM9b 7EP7kXGBm1FDPLE8cccznJLRQcK8LnnWb5oOC5oGLpN4vaT9YpPBc38luAQGY8U9hIF+ pEuQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id jr19-20020a170906515300b00870d2ca6798si4039489ejc.800.2023.01.27.01.17.51; Fri, 27 Jan 2023 01:18:15 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232837AbjA0JLc convert rfc822-to-8bit (ORCPT + 99 others); Fri, 27 Jan 2023 04:11:32 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57480 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232823AbjA0JL2 (ORCPT ); Fri, 27 Jan 2023 04:11:28 -0500 Received: from fd01.gateway.ufhost.com (fd01.gateway.ufhost.com [61.152.239.71]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 34B5077DD7 for ; Fri, 27 Jan 2023 01:11:23 -0800 (PST) Received: from EXMBX165.cuchost.com (unknown [175.102.18.54]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "EXMBX165", Issuer "EXMBX165" (not verified)) by fd01.gateway.ufhost.com (Postfix) with ESMTP id 12BC424E004; Fri, 27 Jan 2023 17:11:22 +0800 (CST) Received: from EXMBX066.cuchost.com (172.16.7.66) by EXMBX165.cuchost.com (172.16.6.75) with Microsoft SMTP Server (TLS) id 15.0.1497.42; Fri, 27 Jan 2023 17:11:22 +0800 Received: from jsia-virtual-machine.localdomain (60.50.196.81) by EXMBX066.cuchost.com (172.16.6.66) with Microsoft SMTP Server (TLS) id 15.0.1497.42; Fri, 27 Jan 2023 17:11:17 +0800 From: Sia Jee Heng To: , , CC: , , , , Subject: [PATCH v3 2/4] RISC-V: Factor out common code of __cpu_resume_enter() Date: Fri, 27 Jan 2023 17:10:49 +0800 Message-ID: <20230127091051.1465278-3-jeeheng.sia@starfivetech.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230127091051.1465278-1-jeeheng.sia@starfivetech.com> References: <20230127091051.1465278-1-jeeheng.sia@starfivetech.com> MIME-Version: 1.0 X-Originating-IP: [60.50.196.81] X-ClientProxiedBy: EXCAS066.cuchost.com (172.16.6.26) To EXMBX066.cuchost.com (172.16.6.66) X-YovoleRuleAgent: yovoleflag X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RCVD_IN_MSPIKE_H2, SPF_HELO_PASS,SPF_PASS,UPPERCASE_50_75 autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756166718999503122?= X-GMAIL-MSGID: =?utf-8?q?1756166718999503122?= The cpu_resume() function is very similar for the suspend to disk and suspend to ram cases. Factor out the common code into restore_csr macro and restore_reg macro. Signed-off-by: Sia Jee Heng --- arch/riscv/include/asm/assembler.h | 62 ++++++++++++++++++++++++++++++ arch/riscv/kernel/suspend_entry.S | 34 ++-------------- 2 files changed, 65 insertions(+), 31 deletions(-) create mode 100644 arch/riscv/include/asm/assembler.h diff --git a/arch/riscv/include/asm/assembler.h b/arch/riscv/include/asm/assembler.h new file mode 100644 index 000000000000..ef1283d04b70 --- /dev/null +++ b/arch/riscv/include/asm/assembler.h @@ -0,0 +1,62 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2023 StarFive Technology Co., Ltd. + * + * Author: Jee Heng Sia + */ + +#ifndef __ASSEMBLY__ +#error "Only include this from assembly code" +#endif + +#ifndef __ASM_ASSEMBLER_H +#define __ASM_ASSEMBLER_H + +#include +#include +#include + +/** + * restore_csr - restore hart's CSR value + */ + .macro restore_csr + REG_L t0, (SUSPEND_CONTEXT_REGS + PT_EPC)(a0) + csrw CSR_EPC, t0 + REG_L t0, (SUSPEND_CONTEXT_REGS + PT_STATUS)(a0) + csrw CSR_STATUS, t0 + REG_L t0, (SUSPEND_CONTEXT_REGS + PT_BADADDR)(a0) + csrw CSR_TVAL, t0 + REG_L t0, (SUSPEND_CONTEXT_REGS + PT_CAUSE)(a0) + csrw CSR_CAUSE, t0 + .endm + +/** + * restore_reg - Restore registers (except A0 and T0-T6) + */ + .macro restore_reg + REG_L ra, (SUSPEND_CONTEXT_REGS + PT_RA)(a0) + REG_L sp, (SUSPEND_CONTEXT_REGS + PT_SP)(a0) + REG_L gp, (SUSPEND_CONTEXT_REGS + PT_GP)(a0) + REG_L tp, (SUSPEND_CONTEXT_REGS + PT_TP)(a0) + REG_L s0, (SUSPEND_CONTEXT_REGS + PT_S0)(a0) + REG_L s1, (SUSPEND_CONTEXT_REGS + PT_S1)(a0) + REG_L a1, (SUSPEND_CONTEXT_REGS + PT_A1)(a0) + REG_L a2, (SUSPEND_CONTEXT_REGS + PT_A2)(a0) + REG_L a3, (SUSPEND_CONTEXT_REGS + PT_A3)(a0) + REG_L a4, (SUSPEND_CONTEXT_REGS + PT_A4)(a0) + REG_L a5, (SUSPEND_CONTEXT_REGS + PT_A5)(a0) + REG_L a6, (SUSPEND_CONTEXT_REGS + PT_A6)(a0) + REG_L a7, (SUSPEND_CONTEXT_REGS + PT_A7)(a0) + REG_L s2, (SUSPEND_CONTEXT_REGS + PT_S2)(a0) + REG_L s3, (SUSPEND_CONTEXT_REGS + PT_S3)(a0) + REG_L s4, (SUSPEND_CONTEXT_REGS + PT_S4)(a0) + REG_L s5, (SUSPEND_CONTEXT_REGS + PT_S5)(a0) + REG_L s6, (SUSPEND_CONTEXT_REGS + PT_S6)(a0) + REG_L s7, (SUSPEND_CONTEXT_REGS + PT_S7)(a0) + REG_L s8, (SUSPEND_CONTEXT_REGS + PT_S8)(a0) + REG_L s9, (SUSPEND_CONTEXT_REGS + PT_S9)(a0) + REG_L s10, (SUSPEND_CONTEXT_REGS + PT_S10)(a0) + REG_L s11, (SUSPEND_CONTEXT_REGS + PT_S11)(a0) + .endm + +#endif /* __ASM_ASSEMBLER_H */ diff --git a/arch/riscv/kernel/suspend_entry.S b/arch/riscv/kernel/suspend_entry.S index aafcca58c19d..74a8fab8e0f6 100644 --- a/arch/riscv/kernel/suspend_entry.S +++ b/arch/riscv/kernel/suspend_entry.S @@ -7,6 +7,7 @@ #include #include #include +#include #include #include @@ -83,39 +84,10 @@ ENTRY(__cpu_resume_enter) add a0, a1, zero /* Restore CSRs */ - REG_L t0, (SUSPEND_CONTEXT_REGS + PT_EPC)(a0) - csrw CSR_EPC, t0 - REG_L t0, (SUSPEND_CONTEXT_REGS + PT_STATUS)(a0) - csrw CSR_STATUS, t0 - REG_L t0, (SUSPEND_CONTEXT_REGS + PT_BADADDR)(a0) - csrw CSR_TVAL, t0 - REG_L t0, (SUSPEND_CONTEXT_REGS + PT_CAUSE)(a0) - csrw CSR_CAUSE, t0 + restore_csr /* Restore registers (except A0 and T0-T6) */ - REG_L ra, (SUSPEND_CONTEXT_REGS + PT_RA)(a0) - REG_L sp, (SUSPEND_CONTEXT_REGS + PT_SP)(a0) - REG_L gp, (SUSPEND_CONTEXT_REGS + PT_GP)(a0) - REG_L tp, (SUSPEND_CONTEXT_REGS + PT_TP)(a0) - REG_L s0, (SUSPEND_CONTEXT_REGS + PT_S0)(a0) - REG_L s1, (SUSPEND_CONTEXT_REGS + PT_S1)(a0) - REG_L a1, (SUSPEND_CONTEXT_REGS + PT_A1)(a0) - REG_L a2, (SUSPEND_CONTEXT_REGS + PT_A2)(a0) - REG_L a3, (SUSPEND_CONTEXT_REGS + PT_A3)(a0) - REG_L a4, (SUSPEND_CONTEXT_REGS + PT_A4)(a0) - REG_L a5, (SUSPEND_CONTEXT_REGS + PT_A5)(a0) - REG_L a6, (SUSPEND_CONTEXT_REGS + PT_A6)(a0) - REG_L a7, (SUSPEND_CONTEXT_REGS + PT_A7)(a0) - REG_L s2, (SUSPEND_CONTEXT_REGS + PT_S2)(a0) - REG_L s3, (SUSPEND_CONTEXT_REGS + PT_S3)(a0) - REG_L s4, (SUSPEND_CONTEXT_REGS + PT_S4)(a0) - REG_L s5, (SUSPEND_CONTEXT_REGS + PT_S5)(a0) - REG_L s6, (SUSPEND_CONTEXT_REGS + PT_S6)(a0) - REG_L s7, (SUSPEND_CONTEXT_REGS + PT_S7)(a0) - REG_L s8, (SUSPEND_CONTEXT_REGS + PT_S8)(a0) - REG_L s9, (SUSPEND_CONTEXT_REGS + PT_S9)(a0) - REG_L s10, (SUSPEND_CONTEXT_REGS + PT_S10)(a0) - REG_L s11, (SUSPEND_CONTEXT_REGS + PT_S11)(a0) + restore_reg /* Return zero value */ add a0, zero, zero From patchwork Fri Jan 27 09:10:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: JeeHeng Sia X-Patchwork-Id: 49120 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp734307wrn; Fri, 27 Jan 2023 01:18:15 -0800 (PST) X-Google-Smtp-Source: AMrXdXtXBkV58557EvACHFA8yct9lB/qe8kMrc/VavgPIII6uOBDCKS/YirhFt0GqszenuHHvF4b X-Received: by 2002:a17:907:d089:b0:7ad:aed7:a5da with SMTP id vc9-20020a170907d08900b007adaed7a5damr45084060ejc.28.1674811095361; Fri, 27 Jan 2023 01:18:15 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674811095; cv=none; d=google.com; s=arc-20160816; b=SfShYlWSDMUNAeauGryKQ56dIKGdbvBMaxwyHgVYZmqZeoTSY5ii6oC4fYx7kzUdcz abSkRb5GgDHiTOSyLt6HfhfSPLixIRRlGDoFjvuWdWtFrxo5kecRda3Kr9pQJ9LxVkfu kf1U+fT5Gv1nyr4UT18oUq9XDc/4JufSICGFHKmzQFiEXrIkpB0rFseS7za9d9WC9iKd UJfKyURxFY+njobPrbhWM+HrSoQ3eKo0J5cFraRnk2ku724+d2CP2/xMldFhadPfhzb/ iOpAMUEcBWV1u4ecWTKbG4wKC11X6Y/tqq7O7kQAJIiD2LkEdxon1q5/nWe+p6spAFvW PPAA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=nyvTOyejeZM6shN79UmQi7uh31f5As/2h2AZKQdb2mI=; b=J6PrMLZFrrU6B6NeMCqCvJCRb452J2uYcAzdIkDpfSV0KPsEmTMlUmJ/JUcycU9BY9 J/2JeQa1XWjoz4tVSDZETGHmOKKhyTKevepaPWWv2y84bt5JD7zpRXKEjrCpVojkS6zv bDpqFseLzDpa/Nv1Y+oHpEnTrjxd4tIox6r4Vr/4HIDxS8AAcoLbahwC0J/Hqq53EiAB GamkzqGAbplsD6N5a48CTrum/HGULSReb8q8eZtdNWU74yGhIRzPgC+2yVL67+JHgs6H 4dW8uimjagCfkgdCn6eeIqZiHWgYPUdXjB7lvRbcE5di8QOvx2A21QdFAbOt5xHIHBWB Uz2w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id ss2-20020a170907c00200b0086f53c24da4si5027505ejc.635.2023.01.27.01.17.52; Fri, 27 Jan 2023 01:18:15 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232859AbjA0JLm convert rfc822-to-8bit (ORCPT + 99 others); Fri, 27 Jan 2023 04:11:42 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57902 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232861AbjA0JLk (ORCPT ); Fri, 27 Jan 2023 04:11:40 -0500 Received: from fd01.gateway.ufhost.com (fd01.gateway.ufhost.com [61.152.239.71]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0C78D783CC for ; Fri, 27 Jan 2023 01:11:28 -0800 (PST) Received: from EXMBX166.cuchost.com (unknown [175.102.18.54]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "EXMBX166", Issuer "EXMBX166" (not verified)) by fd01.gateway.ufhost.com (Postfix) with ESMTP id AE99224E02F; Fri, 27 Jan 2023 17:11:27 +0800 (CST) Received: from EXMBX066.cuchost.com (172.16.7.66) by EXMBX166.cuchost.com (172.16.6.76) with Microsoft SMTP Server (TLS) id 15.0.1497.42; Fri, 27 Jan 2023 17:11:27 +0800 Received: from jsia-virtual-machine.localdomain (60.50.196.81) by EXMBX066.cuchost.com (172.16.6.66) with Microsoft SMTP Server (TLS) id 15.0.1497.42; Fri, 27 Jan 2023 17:11:23 +0800 From: Sia Jee Heng To: , , CC: , , , , Subject: [PATCH v3 3/4] RISC-V: mm: Enable huge page support to kernel_page_present() function Date: Fri, 27 Jan 2023 17:10:50 +0800 Message-ID: <20230127091051.1465278-4-jeeheng.sia@starfivetech.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230127091051.1465278-1-jeeheng.sia@starfivetech.com> References: <20230127091051.1465278-1-jeeheng.sia@starfivetech.com> MIME-Version: 1.0 X-Originating-IP: [60.50.196.81] X-ClientProxiedBy: EXCAS066.cuchost.com (172.16.6.26) To EXMBX066.cuchost.com (172.16.6.66) X-YovoleRuleAgent: yovoleflag X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RCVD_IN_MSPIKE_H2, SPF_HELO_PASS,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756166719251262858?= X-GMAIL-MSGID: =?utf-8?q?1756166719251262858?= Currently kernel_page_present() function doesn't support huge page detection causes the function to mistakenly return false to the hibernation core. Add huge page detection to the function to solve the problem. Signed-off-by: Sia Jee Heng Reviewed-by: Ley Foon Tan Reviewed-by: Mason Huo --- arch/riscv/mm/pageattr.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/arch/riscv/mm/pageattr.c b/arch/riscv/mm/pageattr.c index 86c56616e5de..792b8d10cdfc 100644 --- a/arch/riscv/mm/pageattr.c +++ b/arch/riscv/mm/pageattr.c @@ -221,14 +221,20 @@ bool kernel_page_present(struct page *page) p4d = p4d_offset(pgd, addr); if (!p4d_present(*p4d)) return false; + if (p4d_leaf(*p4d)) + return true; pud = pud_offset(p4d, addr); if (!pud_present(*pud)) return false; + if (pud_leaf(*pud)) + return true; pmd = pmd_offset(pud, addr); if (!pmd_present(*pmd)) return false; + if (pmd_leaf(*pmd)) + return true; pte = pte_offset_kernel(pmd, addr); return pte_present(*pte); From patchwork Fri Jan 27 09:10:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: JeeHeng Sia X-Patchwork-Id: 49123 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp735592wrn; Fri, 27 Jan 2023 01:22:03 -0800 (PST) X-Google-Smtp-Source: AMrXdXsgIl+gkIyE9tibY2qw6CpETIVIjKDtWHkQlX5N+zLJF/CqRNxdenS/15HZQ41sBXIL2HR6 X-Received: by 2002:a17:906:3a97:b0:86e:bbdf:95c5 with SMTP id y23-20020a1709063a9700b0086ebbdf95c5mr42884373ejd.26.1674811323119; Fri, 27 Jan 2023 01:22:03 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674811323; cv=none; d=google.com; s=arc-20160816; b=AWDud93WYqzi5fyx/pJjotMwnqb0LABvAQSL/yRo3+NI6XDqvkIfJvdisc5CfkZ37u hRTdyBpzbRtoBREVnEgW4fAlp8LRhBa8uPu/SHVSECoCiGc9OCHYWvIqmh6gG/dY6S3E feNbpTVUUbU24C1djFc+Q86403AhlCfWDSD5yKVtb/opP8DdizXdp65rvbayczvLQCnM XlXZoXgoYknxAjb4vmV6T3vRPuY7RK0z8C0tt2yrDJ/aC/m41/snwOL/WgeXUfTmP7Gu fwXsb1tkXp2R5ikc650X6DlTOFhM9PAxdyxvjOSzsFiP9fPOG86DuelS22axjk2DUK/3 Tm5w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=v/l954fJ+3SdULf7R6myezIAts59/PrARP5a6AJe51Y=; b=x84pkqJa/tRWtPZxk6eJAgyQvGxhUbzoWPzrSyQZXN0S4jxkU23aECYFLi1PQbrfNo 5Oc9J+9O6SueAhhOpfctbdyeMDOaH5fNn8wH42XOZznOelCkJxuaLXTGJ3N0bcVpMsvp M8MiwoZ/6Oz6camdrDZZv3szMurz0+DVJJBz9qrF8iF0hpCIVBA+dmz5Jec0suS5Vlt/ 48zc59dLJlj5eqYl2QE8fcJG0hO2Nbbq17b45Xp5ba+Mu/FPxbXa8g0G/6SHXZz4Fy3p o/1sDWbQxxxPbG6v8OzAxY3Zwt+69wBAG5AcpqtbKj4VOynihcIBZ54zsONHpL9U+tGL rhYQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id f5-20020a1709063f4500b00871bf3d106csi4184679ejj.607.2023.01.27.01.21.37; Fri, 27 Jan 2023 01:22:02 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232129AbjA0JLv convert rfc822-to-8bit (ORCPT + 99 others); Fri, 27 Jan 2023 04:11:51 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58196 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232663AbjA0JLu (ORCPT ); Fri, 27 Jan 2023 04:11:50 -0500 Received: from fd01.gateway.ufhost.com (fd01.gateway.ufhost.com [61.152.239.71]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6CCEA783EA for ; Fri, 27 Jan 2023 01:11:34 -0800 (PST) Received: from EXMBX165.cuchost.com (unknown [175.102.18.54]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "EXMBX165", Issuer "EXMBX165" (not verified)) by fd01.gateway.ufhost.com (Postfix) with ESMTP id 308B124E22F; Fri, 27 Jan 2023 17:11:33 +0800 (CST) Received: from EXMBX066.cuchost.com (172.16.7.66) by EXMBX165.cuchost.com (172.16.6.75) with Microsoft SMTP Server (TLS) id 15.0.1497.42; Fri, 27 Jan 2023 17:11:33 +0800 Received: from jsia-virtual-machine.localdomain (60.50.196.81) by EXMBX066.cuchost.com (172.16.6.66) with Microsoft SMTP Server (TLS) id 15.0.1497.42; Fri, 27 Jan 2023 17:11:28 +0800 From: Sia Jee Heng To: , , CC: , , , , Subject: [PATCH v3 4/4] RISC-V: Add arch functions to support hibernation/suspend-to-disk Date: Fri, 27 Jan 2023 17:10:51 +0800 Message-ID: <20230127091051.1465278-5-jeeheng.sia@starfivetech.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230127091051.1465278-1-jeeheng.sia@starfivetech.com> References: <20230127091051.1465278-1-jeeheng.sia@starfivetech.com> MIME-Version: 1.0 X-Originating-IP: [60.50.196.81] X-ClientProxiedBy: EXCAS066.cuchost.com (172.16.6.26) To EXMBX066.cuchost.com (172.16.6.66) X-YovoleRuleAgent: yovoleflag X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RCVD_IN_MSPIKE_H2, SPF_HELO_PASS,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756166958048766202?= X-GMAIL-MSGID: =?utf-8?q?1756166958048766202?= Low level Arch functions were created to support hibernation. swsusp_arch_suspend() relies code from __cpu_suspend_enter() to write cpu state onto the stack, then calling swsusp_save() to save the memory image. Arch specific hibernation header is implemented and is utilized by the arch_hibernation_header_restore() and arch_hibernation_header_save() functions. The arch specific hibernation header consists of satp, hartid, and the cpu_resume address. The kernel built version is also need to be saved into the hibernation image header to making sure only the same kernel is restore when resume. swsusp_arch_resume() creates a temporary page table that covering only the linear map. It copies the restore code to a 'safe' page, then start to restore the memory image. Once completed, it restores the original kernel's page table. It then calls into __hibernate_cpu_resume() to restore the CPU context. Finally, it follows the normal hibernation path back to the hibernation core. To enable hibernation/suspend to disk into RISCV, the below config need to be enabled: - CONFIG_ARCH_HIBERNATION_HEADER - CONFIG_ARCH_HIBERNATION_POSSIBLE Signed-off-by: Sia Jee Heng Reviewed-by: Ley Foon Tan Reviewed-by: Mason Huo --- arch/riscv/Kconfig | 7 + arch/riscv/include/asm/assembler.h | 20 ++ arch/riscv/include/asm/suspend.h | 21 ++ arch/riscv/kernel/Makefile | 1 + arch/riscv/kernel/asm-offsets.c | 5 + arch/riscv/kernel/hibernate-asm.S | 89 +++++++ arch/riscv/kernel/hibernate.c | 360 +++++++++++++++++++++++++++++ 7 files changed, 503 insertions(+) create mode 100644 arch/riscv/kernel/hibernate-asm.S create mode 100644 arch/riscv/kernel/hibernate.c diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index e2b656043abf..4555848a817f 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -690,6 +690,13 @@ menu "Power management options" source "kernel/power/Kconfig" +config ARCH_HIBERNATION_POSSIBLE + def_bool y + +config ARCH_HIBERNATION_HEADER + def_bool y + depends on HIBERNATION + endmenu # "Power management options" menu "CPU Power Management" diff --git a/arch/riscv/include/asm/assembler.h b/arch/riscv/include/asm/assembler.h index ef1283d04b70..3de70d3e6ceb 100644 --- a/arch/riscv/include/asm/assembler.h +++ b/arch/riscv/include/asm/assembler.h @@ -59,4 +59,24 @@ REG_L s11, (SUSPEND_CONTEXT_REGS + PT_S11)(a0) .endm +/** + * copy_page - copy 1 page (4KB) of data from source to destination + * @a0 - destination + * @a1 - source + */ + .macro copy_page a0, a1 + lui a2, 0x1 + add a2, a2, a0 +.1 : + REG_L t0, 0(a1) + REG_L t1, SZREG(a1) + + REG_S t0, 0(a0) + REG_S t1, SZREG(a0) + + addi a0, a0, 2 * SZREG + addi a1, a1, 2 * SZREG + bne a2, a0, .1 + .endm + #endif /* __ASM_ASSEMBLER_H */ diff --git a/arch/riscv/include/asm/suspend.h b/arch/riscv/include/asm/suspend.h index 75419c5ca272..db40ae433aa9 100644 --- a/arch/riscv/include/asm/suspend.h +++ b/arch/riscv/include/asm/suspend.h @@ -21,6 +21,12 @@ struct suspend_context { #endif }; +/* + * This parameter will be assigned to 0 during resume and will be used by + * hibernation core for the subsequent resume sequence + */ +extern int in_suspend; + /* Low-level CPU suspend entry function */ int __cpu_suspend_enter(struct suspend_context *context); @@ -36,4 +42,19 @@ int __cpu_resume_enter(unsigned long hartid, unsigned long context); /* Used to save and restore the csr */ void suspend_save_csrs(struct suspend_context *context); void suspend_restore_csrs(struct suspend_context *context); + +/* Low-level API to support hibernation */ +int swsusp_arch_suspend(void); +int swsusp_arch_resume(void); +int arch_hibernation_header_save(void *addr, unsigned int max_size); +int arch_hibernation_header_restore(void *addr); +int __hibernate_cpu_resume(void); + +/* Used to resume on the CPU we hibernated on */ +int hibernate_resume_nonboot_cpu_disable(void); + +/* Used to restore the hibernated image */ +asmlinkage void restore_image(unsigned long resume_satp, unsigned long satp_temp, + unsigned long cpu_resume); +asmlinkage int core_restore_code(void); #endif diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile index 4cf303a779ab..daab341d55e4 100644 --- a/arch/riscv/kernel/Makefile +++ b/arch/riscv/kernel/Makefile @@ -64,6 +64,7 @@ obj-$(CONFIG_MODULES) += module.o obj-$(CONFIG_MODULE_SECTIONS) += module-sections.o obj-$(CONFIG_CPU_PM) += suspend_entry.o suspend.o +obj-$(CONFIG_HIBERNATION) += hibernate.o hibernate-asm.o obj-$(CONFIG_FUNCTION_TRACER) += mcount.o ftrace.o obj-$(CONFIG_DYNAMIC_FTRACE) += mcount-dyn.o diff --git a/arch/riscv/kernel/asm-offsets.c b/arch/riscv/kernel/asm-offsets.c index df9444397908..d6a75aac1d27 100644 --- a/arch/riscv/kernel/asm-offsets.c +++ b/arch/riscv/kernel/asm-offsets.c @@ -9,6 +9,7 @@ #include #include #include +#include #include #include #include @@ -116,6 +117,10 @@ void asm_offsets(void) OFFSET(SUSPEND_CONTEXT_REGS, suspend_context, regs); + OFFSET(HIBERN_PBE_ADDR, pbe, address); + OFFSET(HIBERN_PBE_ORIG, pbe, orig_address); + OFFSET(HIBERN_PBE_NEXT, pbe, next); + OFFSET(KVM_ARCH_GUEST_ZERO, kvm_vcpu_arch, guest_context.zero); OFFSET(KVM_ARCH_GUEST_RA, kvm_vcpu_arch, guest_context.ra); OFFSET(KVM_ARCH_GUEST_SP, kvm_vcpu_arch, guest_context.sp); diff --git a/arch/riscv/kernel/hibernate-asm.S b/arch/riscv/kernel/hibernate-asm.S new file mode 100644 index 000000000000..a83d534b89bd --- /dev/null +++ b/arch/riscv/kernel/hibernate-asm.S @@ -0,0 +1,89 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Hibernation support specific for RISCV + * + * Copyright (C) 2023 StarFive Technology Co., Ltd. + * + * Author: Jee Heng Sia + */ + +#include +#include +#include +#include + +#include + +/* + * This code is executed when resume from the hibernation. + * + * It begins with loading the temporary page table then restores the memory image. + * Finally branches to __hibernate_cpu_resume() to restore the state saved by + * swsusp_arch_suspend(). + */ + +/* + * int __hibernate_cpu_resume(void) + * Switch back to the hibernated image's page table prior to restore the CPU + * context. + * + * Always returns 0 to the C code. + */ +ENTRY(__hibernate_cpu_resume) + /* switch to hibernated image's page table */ + csrw CSR_SATP, s0 + sfence.vma + + REG_L a0, hibernate_cpu_context + + /* Restore CSRs */ + restore_csr + + /* Restore registers (except A0 and T0-T6) */ + restore_reg + + /* Return zero value */ + add a0, zero, zero + + /* Return to C code */ + ret +END(__hibernate_cpu_resume) + +/* + * Prepare to restore the image. + * a0: satp of saved page tables + * a1: satp of temporary page tables + * a2: cpu_resume + */ +ENTRY(restore_image) + mv s0, a0 + mv s1, a1 + mv s2, a2 + REG_L s4, restore_pblist + REG_L a1, relocated_restore_code + + jalr a1 +END(restore_image) + +/* + * The below code will be executed from a 'safe' page. + * It first switches to the temporary page table, then start to copy the pages + * back to the original memory location. Finally, it jumps to the __hibernate_cpu_resume() + * to restore the CPU context. + */ +ENTRY(core_restore_code) + /* switch to temp page table */ + csrw satp, s1 + sfence.vma +.Lcopy: + /* The below code will restore the hibernated image. */ + REG_L a1, HIBERN_PBE_ADDR(s4) + REG_L a0, HIBERN_PBE_ORIG(s4) + + copy_page a0, a1 + + REG_L s4, HIBERN_PBE_NEXT(s4) + bnez s4, .Lcopy + + jalr s2 +END(core_restore_code) diff --git a/arch/riscv/kernel/hibernate.c b/arch/riscv/kernel/hibernate.c new file mode 100644 index 000000000000..bf7f3c781820 --- /dev/null +++ b/arch/riscv/kernel/hibernate.c @@ -0,0 +1,360 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Hibernation support specific for RISCV + * + * Copyright (C) 2023 StarFive Technology Co., Ltd. + * + * Author: Jee Heng Sia + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include + +/* The logical cpu number we should resume on, initialised to a non-cpu number */ +static int sleep_cpu = -EINVAL; + +/* CPU context to be saved */ +struct suspend_context *hibernate_cpu_context; + +unsigned long relocated_restore_code; + +/* Pointer to the temporary resume page table */ +pgd_t *resume_pg_dir; + +/** + * struct arch_hibernate_hdr_invariants - container to store kernel build version + * @uts_version: to save the build number and date so that the we are not resume with + * a different kernel + */ +struct arch_hibernate_hdr_invariants { + char uts_version[__NEW_UTS_LEN + 1]; +}; + +/** + * struct arch_hibernate_hdr - helper parameters that help us to restore the image + * @invariants: container to store kernel build version + * @hartid: to make sure same boot_cpu executing the hibernate/restore code. + * @saved_satp: original page table used by the hibernated image. + * @restore_cpu_addr: the kernel's image address to restore the CPU context. + */ +static struct arch_hibernate_hdr { + struct arch_hibernate_hdr_invariants invariants; + unsigned long hartid; + unsigned long saved_satp; + unsigned long restore_cpu_addr; +} resume_hdr; + +static inline void arch_hdr_invariants(struct arch_hibernate_hdr_invariants *i) +{ + memset(i, 0, sizeof(*i)); + memcpy(i->uts_version, init_utsname()->version, sizeof(i->uts_version)); +} + +/* + * Check if the given pfn is in the 'nosave' section. + */ +int pfn_is_nosave(unsigned long pfn) +{ + unsigned long nosave_begin_pfn = sym_to_pfn(&__nosave_begin); + unsigned long nosave_end_pfn = sym_to_pfn(&__nosave_end - 1); + + return ((pfn >= nosave_begin_pfn) && (pfn <= nosave_end_pfn)); +} + +void notrace save_processor_state(void) +{ + WARN_ON(num_online_cpus() != 1); +} + +void notrace restore_processor_state(void) +{ +} + +/* + * Helper parameters need to be saved to the hibernation image header. + */ +int arch_hibernation_header_save(void *addr, unsigned int max_size) +{ + struct arch_hibernate_hdr *hdr = addr; + + if (max_size < sizeof(*hdr)) + return -EOVERFLOW; + + arch_hdr_invariants(&hdr->invariants); + + hdr->hartid = cpuid_to_hartid_map(sleep_cpu); + hdr->saved_satp = csr_read(CSR_SATP); + hdr->restore_cpu_addr = (unsigned long)__hibernate_cpu_resume; + + return 0; +} +EXPORT_SYMBOL(arch_hibernation_header_save); + +/* + * Retrieve the helper parameters from the hibernation image header + */ +int arch_hibernation_header_restore(void *addr) +{ + struct arch_hibernate_hdr_invariants invariants; + struct arch_hibernate_hdr *hdr = addr; + int ret = 0; + + arch_hdr_invariants(&invariants); + + if (memcmp(&hdr->invariants, &invariants, sizeof(invariants))) { + pr_crit("Hibernate image not generated by this kernel!\n"); + return -EINVAL; + } + + sleep_cpu = riscv_hartid_to_cpuid(hdr->hartid); + if (sleep_cpu < 0) { + pr_crit("Hibernated on a CPU not known to this kernel!\n"); + sleep_cpu = -EINVAL; + return -EINVAL; + } + +#ifdef CONFIG_SMP + ret = bringup_hibernate_cpu(sleep_cpu); + if (ret) { + sleep_cpu = -EINVAL; + return ret; + } +#endif + resume_hdr = *hdr; + + return ret; +} +EXPORT_SYMBOL(arch_hibernation_header_restore); + +int swsusp_arch_suspend(void) +{ + int ret = 0; + + if (__cpu_suspend_enter(hibernate_cpu_context)) { + sleep_cpu = smp_processor_id(); + suspend_save_csrs(hibernate_cpu_context); + ret = swsusp_save(); + } else { + suspend_restore_csrs(hibernate_cpu_context); + flush_tlb_all(); + + /* Invalidated Icache */ + flush_icache_all(); + + /* + * Tell the hibernation core that we've just restored + * the memory + */ + in_suspend = 0; + sleep_cpu = -EINVAL; + } + + return ret; +} + +static unsigned long temp_pgtable_map_pte(pte_t *ptep, unsigned long vaddr, pgprot_t prot) +{ + uintptr_t pte_idx = pte_index(vaddr); + + ptep[pte_idx] = pfn_pte(PFN_DOWN(__pa(vaddr)), prot); + + return 0; +} + +#ifndef __PAGETABLE_PMD_FOLDED +#define temp_pgtable_map_pgd_next(pgdp, vaddr, prot) \ + (pgtable_l5_enabled ? \ + temp_pgtable_map_p4d(pgdp, vaddr, prot) : \ + (pgtable_l4_enabled ? \ + temp_pgtable_map_pud((pud_t *)pgdp, vaddr, prot) : \ + temp_pgtable_map_pmd((pmd_t *)pgdp, vaddr, prot))) + +static unsigned long temp_pgtable_map_pmd(pmd_t *pmdp, unsigned long vaddr, pgprot_t prot) +{ + uintptr_t pmd_idx = pmd_index(vaddr); + pte_t *ptep; + + if (pmd_none(pmdp[pmd_idx])) { + ptep = (pte_t *)get_safe_page(GFP_ATOMIC); + if (!ptep) + return -ENOMEM; + + memset(ptep, 0, PAGE_SIZE); + pmdp[pmd_idx] = pfn_pmd(PFN_DOWN(__pa(ptep)), PAGE_TABLE); + } else { + ptep = (pte_t *)__va(PFN_PHYS(_pmd_pfn(pmdp[pmd_idx]))); + } + + return temp_pgtable_map_pte(ptep, vaddr, prot); +} + +static unsigned long temp_pgtable_map_pud(pud_t *pudp, unsigned long vaddr, pgprot_t prot) +{ + uintptr_t pud_index = pud_index(vaddr); + pmd_t *pmdp; + + if (pud_val(pudp[pud_index]) == 0) { + pmdp = (pmd_t *)get_safe_page(GFP_ATOMIC); + if (!pmdp) + return -ENOMEM; + + memset(pmdp, 0, PAGE_SIZE); + pudp[pud_index] = pfn_pud(PFN_DOWN(__pa(pmdp)), PAGE_TABLE); + } else { + pmdp = (pmd_t *)__va(PFN_PHYS(_pud_pfn(pudp[pud_index]))); + } + + return temp_pgtable_map_pmd(pmdp, vaddr, prot); +} + +static unsigned long temp_pgtable_map_p4d(p4d_t *p4dp, unsigned long vaddr, pgprot_t prot) +{ + uintptr_t p4d_index = p4d_index(vaddr); + pud_t *pudp; + + if (p4d_val(p4dp[p4d_index]) == 0) { + pudp = (pud_t *)get_safe_page(GFP_ATOMIC); + if (!pudp) + return -ENOMEM; + + memset(pudp, 0, PAGE_SIZE); + p4dp[p4d_index] = pfn_p4d(PFN_DOWN(__pa(pudp)), PAGE_TABLE); + } else { + pudp = (pud_t *)__va(PFN_PHYS(_p4d_pfn(p4dp[p4d_index]))); + } + + return temp_pgtable_map_pud(pudp, vaddr, prot); +} + +#else +#define temp_pgtable_map_pgd_next(nextp, vaddr, prot) \ + temp_pgtable_map_pte((pte_t *)nextp, vaddr, prot) +#endif /* __PAGETABLE_PMD_FOLDED */ + +static unsigned long temp_pgtable_map_pgd(pgd_t *pgdp, unsigned long vaddr, pgprot_t prot) +{ + uintptr_t pgd_idx = pgd_index(vaddr); + void *nextp; + + if (pgd_val(pgdp[pgd_idx]) == 0) { + nextp = (void *)get_safe_page(GFP_ATOMIC); + if (!nextp) + return -ENOMEM; + + memset(nextp, 0, PAGE_SIZE); + pgdp[pgd_idx] = pfn_pgd(PFN_DOWN(__pa(nextp)), PAGE_TABLE); + } else { + nextp = (void *)__va(PFN_PHYS(_pgd_pfn(pgdp[pgd_idx]))); + } + + return temp_pgtable_map_pgd_next(nextp, vaddr, prot); +} + +static unsigned long temp_pgtable_mapping(pgd_t *pgdp, unsigned long vaddr, pgprot_t prot) +{ + return temp_pgtable_map_pgd(pgdp, vaddr, prot); +} + +static unsigned long relocate_restore_code(void) +{ + unsigned long ret; + void *page = (void *)get_safe_page(GFP_ATOMIC); + + if (!page) + return -ENOMEM; + + copy_page(page, core_restore_code); + + /* Make the page containing the relocated code executable */ + set_memory_x((unsigned long)page, 1); + + ret = temp_pgtable_mapping(resume_pg_dir, (unsigned long)page, PAGE_KERNEL_READ_EXEC); + if (ret) + return ret; + + return (unsigned long)page; +} + +int swsusp_arch_resume(void) +{ + unsigned long addr = PAGE_OFFSET; + unsigned long ret; + + /* + * Memory allocated by get_safe_page() will be dealt with by the hibernation core, + * we don't need to free it here. + */ + resume_pg_dir = (pgd_t *)get_safe_page(GFP_ATOMIC); + if (!resume_pg_dir) + return -ENOMEM; + + /* + * The pages need to be writable when restoring the image. + * Create a second copy of page table just for the linear map, and use this when + * restoring. + */ + for (; addr <= (unsigned long)pfn_to_virt(max_low_pfn); addr += PAGE_SIZE) { + ret = temp_pgtable_mapping(resume_pg_dir, addr, PAGE_KERNEL); + if (ret) + return (int)ret; + } + + /* Move the restore code to a new page so that it doesn't get overwritten by itself */ + relocated_restore_code = relocate_restore_code(); + if (relocated_restore_code == -ENOMEM) + return -ENOMEM; + + /* + * Map the __hibernate_cpu_resume() address to the temporary page table so that the + * restore code can jump to it after finished restore the image. The next execution + * code doesn't find itself in a different address space after switching over to the + * original page table used by the hibernated image. + */ + ret = temp_pgtable_mapping(resume_pg_dir, (unsigned long)resume_hdr.restore_cpu_addr, + PAGE_KERNEL_READ_EXEC); + if (ret) + return ret; + + restore_image(resume_hdr.saved_satp, (PFN_DOWN(__pa(resume_pg_dir)) | satp_mode), + resume_hdr.restore_cpu_addr); + + return 0; +} + +#ifdef CONFIG_PM_SLEEP_SMP +int hibernate_resume_nonboot_cpu_disable(void) +{ + if (sleep_cpu < 0) { + pr_err("Failing to resume from hibernate on an unknown CPU.\n"); + return -ENODEV; + } + + return freeze_secondary_cpus(sleep_cpu); +} +#endif + +static int __init riscv_hibernate_init(void) +{ + hibernate_cpu_context = kcalloc(1, sizeof(struct suspend_context), GFP_KERNEL); + + if (WARN_ON(!hibernate_cpu_context)) + return -ENOMEM; + + return 0; +} + +early_initcall(riscv_hibernate_init);