From patchwork Sun Jul 16 21:51:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 120937 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:c923:0:b0:3e4:2afc:c1 with SMTP id j3csp788825vqt; Sun, 16 Jul 2023 14:56:36 -0700 (PDT) X-Google-Smtp-Source: APBJJlHn3gPmzBW+3y98qnrZRTSSud7OQc6fdQcZekgwvcxJZ6N8+mwnQu3jHkUYN5KoKEZg6f/g X-Received: by 2002:a05:6a20:2c98:b0:134:a4e2:4ac8 with SMTP id g24-20020a056a202c9800b00134a4e24ac8mr1926259pzj.39.1689544596710; Sun, 16 Jul 2023 14:56:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1689544596; cv=none; d=google.com; s=arc-20160816; b=NVVjzgZN1DRLQr2Qgf090G/slVxaDp+xAWY119qBv4xhvYuFkLzGe200/9Y1yuAiYk AYKIMwp0C2GhxKcTNSlbqlalNreZlUN+jKtwxlPsocU1Zsvdv+MbZeauxjUL7cBobGtR 2YG7CYQJyRIf2tre+PfRYDbQIKoLkh36DJHN0cTpWvp0we0zwz0yJhAh/lY31sDjoFRk QYcvQGhliMO14CP2TnQwabtYczHbsqljmPja/q5ag2ovH1ijvpNJlEYM8gitRxmjbIrV Q2Dh8mwI7djzt/1j8qp8hVDVpI+D/kLier8a6JXgjHOd+zHTyuI82W0GfwC/66vn2BuZ L53w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:in-reply-to:references:message-id :content-transfer-encoding:mime-version:subject:date:from :dkim-signature; bh=B999s6g/ipLtJ/XGBQqngHGg0XyqvuBFMFqFXjAGRfk=; fh=EdAFSmIgUzZTC6WCy4Jg1wBZ0/m6m2d/OjiRUJ3BsTI=; b=qPUHLnzKF+qqApjvx9fXLIXNZFZSHrHmQ0zCC1xzfvNHPRDtgerHqajZs14Wv+47Qm 9egcT4dkJPsD4sx7HfUSlWt2pg5DGOfR3oJN7V0i3O8weOoqS8VxpWa3Dqvn7FRrwwrP cYE02uDyH/VWo5SBBr7hVwldCQEHxamYcmODq2KXvI6VkpkbqFN6ed2SJ72B5ModyZlz Yn/l8Vg7Cqh3KgzZ6agsE+f/5N8b8LGdLizCMw7Z58+eDhcKyZhLxNfmrLJmEUndVo1f VxqTAOBX4SZe4ldRvtlD+zH9qdqhHibA7l0c7XEqtQXWxE6K3tb/+YCNvdKd4U0ZjPz4 /nOg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=St80ihEN; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id t33-20020a634461000000b005582ab00bedsi11013818pgk.387.2023.07.16.14.56.23; Sun, 16 Jul 2023 14:56:36 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=St80ihEN; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229491AbjGPVzk (ORCPT + 99 others); Sun, 16 Jul 2023 17:55:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53272 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231206AbjGPVy5 (ORCPT ); Sun, 16 Jul 2023 17:54:57 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5579F26B1; Sun, 16 Jul 2023 14:54:12 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 36D5660E71; Sun, 16 Jul 2023 21:54:12 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 90C06C433CD; Sun, 16 Jul 2023 21:54:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1689544451; bh=5smHZ4W7RhTfanvJnpNO7WAAfTjfLDVmUyZjClF1Lpo=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=St80ihENKA8tqKHUDgDDetHM4Me6OmsIO/5Kn1rfjFdHzdqn+d1BJ1qC9HN0DJUsk +BxcVbDyNgsj7+8D8XWaG/Lbxbw29b97isJaTqrueBr8Up2qDSSZgjuFcpS4g3wygO 9pZeO1qzn4To0m1js+m4UY6ewJLnjQW5KpAaa2MCbo7BZqGP82p0Eev6FX13P0cIVH esE9YJ73uwhE4BBkgUZOBurhg0y4LPEZHk8YCnYCH1/r/XgYkVLmt12aKTeyZpFr0w B0A632JEz+QvGZVAhqZS9usCc3iWddyHWNt3b4nZMJIwUP1rRcHRtdMmSTqfHmPmgD 1rNQxl4K/7sZQ== From: Mark Brown Date: Sun, 16 Jul 2023 22:51:09 +0100 Subject: [PATCH 13/35] KVM: arm64: Manage GCS registers for guests MIME-Version: 1.0 Message-Id: <20230716-arm64-gcs-v1-13-bf567f93bba6@kernel.org> References: <20230716-arm64-gcs-v1-0-bf567f93bba6@kernel.org> In-Reply-To: <20230716-arm64-gcs-v1-0-bf567f93bba6@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-099c9 X-Developer-Signature: v=1; a=openpgp-sha256; l=5347; i=broonie@kernel.org; h=from:subject:message-id; bh=5smHZ4W7RhTfanvJnpNO7WAAfTjfLDVmUyZjClF1Lpo=; b=owEBbAGT/pANAwAKASTWi3JdVIfQAcsmYgBktGaZrbq4+idLyXtYDjhxjU47zM2RQq1sPBxgQzNl ETBuuliJATIEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZLRmmQAKCRAk1otyXVSH0PE+B/ dxrMXJjspzTj1qyUJx7pswmVIH+n2f8NOk00cAJH8JgQ+1q4H2wzSSyMLJOOSvHeWyXW1ufVEJFuvX R1wHxj1FovQtGg9xTrab9n9Rv6v+6lpuZtjsCmNBZ782xRidxw0ejwuDuwDBfrXrfg1h7E4kUmXDNE fQ1EwrKFs9N+FuhI8dV0yJQ8lZtf+9zOWRAELNey/2NLhpNXDJPN5MC+CbkrHku46dIEPMbpfxObi8 0WGMquDYQTOkyI8CC0I/Mh6ICXrB2RYpjAOo8O2aWV2SYtQ1JrS3xCuxxrZwSG3HM2bHeoedXu5QY1 WnK5CzNyIcIcUGj8Ru05WdNOkJnhM= X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1771615915244697201 X-GMAIL-MSGID: 1771615915244697201 GCS introduces a number of system registers for EL1 and EL0, on systems with GCS we need to context switch them and expose them to VMMs to allow guests to use GCS. Traps are already disabled. Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_host.h | 12 ++++++++++++ arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 17 +++++++++++++++++ arch/arm64/kvm/sys_regs.c | 22 ++++++++++++++++++++++ 3 files changed, 51 insertions(+) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 8b6096753740..67f3894a86f1 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -364,6 +364,12 @@ enum vcpu_sysreg { PIR_EL1, /* Permission Indirection Register 1 (EL1) */ PIRE0_EL1, /* Permission Indirection Register 0 (EL1) */ + /* Guarded Control Stack registers */ + GCSCRE0_EL1, /* Guarded Control Stack Control (EL0) */ + GCSCR_EL1, /* Guarded Control Stack Control (EL1) */ + GCSPR_EL0, /* Guarded Control Stack Pointer (EL0) */ + GCSPR_EL1, /* Guarded Control Stack Pointer (EL1) */ + /* 32bit specific registers. */ DACR32_EL2, /* Domain Access Control Register */ IFSR32_EL2, /* Instruction Fault Status Register */ @@ -1134,6 +1140,12 @@ bool kvm_arm_vcpu_is_finalized(struct kvm_vcpu *vcpu); #define kvm_vm_has_ran_once(kvm) \ (test_bit(KVM_ARCH_FLAG_HAS_RAN_ONCE, &(kvm)->arch.flags)) +static inline bool has_gcs(void) +{ + return IS_ENABLED(CONFIG_ARM64_GCS) && + cpus_have_final_cap(ARM64_HAS_GCS); +} + int kvm_trng_call(struct kvm_vcpu *vcpu); #ifdef CONFIG_KVM extern phys_addr_t hyp_mem_base; diff --git a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h index bb6b571ec627..ec34d4a90717 100644 --- a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h +++ b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h @@ -25,6 +25,8 @@ static inline void __sysreg_save_user_state(struct kvm_cpu_context *ctxt) { ctxt_sys_reg(ctxt, TPIDR_EL0) = read_sysreg(tpidr_el0); ctxt_sys_reg(ctxt, TPIDRRO_EL0) = read_sysreg(tpidrro_el0); + if (has_gcs()) + ctxt_sys_reg(ctxt, GCSPR_EL0) = read_sysreg_s(SYS_GCSPR_EL0); } static inline bool ctxt_has_mte(struct kvm_cpu_context *ctxt) @@ -62,6 +64,12 @@ static inline void __sysreg_save_el1_state(struct kvm_cpu_context *ctxt) ctxt_sys_reg(ctxt, PAR_EL1) = read_sysreg_par(); ctxt_sys_reg(ctxt, TPIDR_EL1) = read_sysreg(tpidr_el1); + if (has_gcs()) { + ctxt_sys_reg(ctxt, GCSPR_EL1) = read_sysreg_el1(SYS_GCSPR); + ctxt_sys_reg(ctxt, GCSCR_EL1) = read_sysreg_el1(SYS_GCSCR); + ctxt_sys_reg(ctxt, GCSCRE0_EL1) = read_sysreg_s(SYS_GCSCRE0_EL1); + } + if (ctxt_has_mte(ctxt)) { ctxt_sys_reg(ctxt, TFSR_EL1) = read_sysreg_el1(SYS_TFSR); ctxt_sys_reg(ctxt, TFSRE0_EL1) = read_sysreg_s(SYS_TFSRE0_EL1); @@ -95,6 +103,8 @@ static inline void __sysreg_restore_user_state(struct kvm_cpu_context *ctxt) { write_sysreg(ctxt_sys_reg(ctxt, TPIDR_EL0), tpidr_el0); write_sysreg(ctxt_sys_reg(ctxt, TPIDRRO_EL0), tpidrro_el0); + if (has_gcs()) + write_sysreg_s(ctxt_sys_reg(ctxt, GCSPR_EL0), SYS_GCSPR_EL0); } static inline void __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt) @@ -138,6 +148,13 @@ static inline void __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt) write_sysreg(ctxt_sys_reg(ctxt, PAR_EL1), par_el1); write_sysreg(ctxt_sys_reg(ctxt, TPIDR_EL1), tpidr_el1); + if (has_gcs()) { + write_sysreg_el1(ctxt_sys_reg(ctxt, GCSPR_EL1), SYS_GCSPR); + write_sysreg_el1(ctxt_sys_reg(ctxt, GCSCR_EL1), SYS_GCSCR); + write_sysreg_s(ctxt_sys_reg(ctxt, GCSCRE0_EL1), + SYS_GCSCRE0_EL1); + } + if (ctxt_has_mte(ctxt)) { write_sysreg_el1(ctxt_sys_reg(ctxt, TFSR_EL1), SYS_TFSR); write_sysreg_s(ctxt_sys_reg(ctxt, TFSRE0_EL1), SYS_TFSRE0_EL1); diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c index bd3431823ec5..e829400aa911 100644 --- a/arch/arm64/kvm/sys_regs.c +++ b/arch/arm64/kvm/sys_regs.c @@ -1768,6 +1768,23 @@ static unsigned int mte_visibility(const struct kvm_vcpu *vcpu, .visibility = mte_visibility, \ } +static unsigned int gcs_visibility(const struct kvm_vcpu *vcpu, + const struct sys_reg_desc *rd) +{ + if (has_gcs()) + return 0; + + return REG_HIDDEN; +} + +#define GCS_REG(name) { \ + SYS_DESC(SYS_##name), \ + .access = undef_access, \ + .reset = reset_unknown, \ + .reg = name, \ + .visibility = gcs_visibility, \ +} + static unsigned int el2_visibility(const struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd) { @@ -2080,6 +2097,10 @@ static const struct sys_reg_desc sys_reg_descs[] = { PTRAUTH_KEY(APDB), PTRAUTH_KEY(APGA), + GCS_REG(GCSCR_EL1), + GCS_REG(GCSPR_EL1), + GCS_REG(GCSCRE0_EL1), + { SYS_DESC(SYS_SPSR_EL1), access_spsr}, { SYS_DESC(SYS_ELR_EL1), access_elr}, @@ -2162,6 +2183,7 @@ static const struct sys_reg_desc sys_reg_descs[] = { { SYS_DESC(SYS_SMIDR_EL1), undef_access }, { SYS_DESC(SYS_CSSELR_EL1), access_csselr, reset_unknown, CSSELR_EL1 }, { SYS_DESC(SYS_CTR_EL0), access_ctr }, + GCS_REG(GCSPR_EL0), { SYS_DESC(SYS_SVCR), undef_access }, { PMU_SYS_REG(SYS_PMCR_EL0), .access = access_pmcr,