From patchwork Fri Aug 11 15:52:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 134583 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b824:0:b0:3f2:4152:657d with SMTP id z4csp1213453vqi; Fri, 11 Aug 2023 09:30:28 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHzA5rS3dL4G8FXi1eVK+CoRdTZw67PE3AQr4duaBALQBki+fevJJVbhxC5Le1Ckq2IcyO6 X-Received: by 2002:a17:90b:1884:b0:263:311f:9bcc with SMTP id mn4-20020a17090b188400b00263311f9bccmr1761237pjb.35.1691771427824; Fri, 11 Aug 2023 09:30:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1691771427; cv=none; d=google.com; s=arc-20160816; b=f8rSIPPXFFUp25ZngXc/iMvhbqwgtcYFC6IH75xhojVVWosMQeWA3FvquixTVb/KNs EVZ3NflcUJJytCiy2C03VRPPjvLaYgz2sq2mpUqNVUv20or3vSBGc8UQ010hLVrlYjPb tKk8mKegKaAsINKQsrWsB0QuaF6sA7uILzku9lNH4rt1R6yQQjUPVeWdE//ad7U0pHLu 76Vhe7GZHOBogxWPoKKzG13uJhHgtKMNRmH4zxl2VAGvh0wU6OM6uR1V3SpFisMGe8Fm 8EMbTXasBvPAPftg+Bu2YmuRMb67ORJ6CJltvVOCRuqLmUHWpoWtrvbhHPUr9SdrIQvx 4E5g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:mime-version:date :reply-to:dkim-signature; bh=CYcyNvoCbMZnvtJsGMVLujGIB26i5/C23iYXX1YXZjc=; fh=q/aWTGhBiFin5x0VZ4h8+TsdTNa2skQcUyalU8mPIQc=; b=ZEfvImYFuI9CPJdCOQ0Sanu7fA5s9Gnvuexvoa0FJAOU8I5EKJZWkdW0Hv09YVFGSg WWKT23eNsB9HvTaew5/VPxukEDMbbIAwu2dx/didQ4HF0KcgnZsBMHp7H1yZq3pqkSy0 kOyDQShFp9oZWetSlBDew976tU9WUa4Jkrz9Ejn3BpKJu7pxXf9SShoycVXGLbQjtMs7 YlZRX7EKMc1/3SGWShp8MglRqUEGdGxwSZyiaMxek3/SL2VbtSTA/XHnbkDBTVIldICp HZgAwJ/p1kdin0bAYJFrBARMfcgtBoVVEDW1+ypnIa+3YQdxeKu6++uzDjcwD4XnWxgu lOWg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=0cFCc1oF; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id qe8-20020a17090b4f8800b002682d73d813si6127286pjb.160.2023.08.11.09.30.03; Fri, 11 Aug 2023 09:30:27 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=0cFCc1oF; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236626AbjHKPxA (ORCPT + 99 others); Fri, 11 Aug 2023 11:53:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40904 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232094AbjHKPw7 (ORCPT ); Fri, 11 Aug 2023 11:52:59 -0400 Received: from mail-pj1-x104a.google.com (mail-pj1-x104a.google.com [IPv6:2607:f8b0:4864:20::104a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3B1B12123 for ; Fri, 11 Aug 2023 08:52:59 -0700 (PDT) Received: by mail-pj1-x104a.google.com with SMTP id 98e67ed59e1d1-26b06bb1972so2286408a91.0 for ; Fri, 11 Aug 2023 08:52:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1691769179; x=1692373979; h=cc:to:from:subject:message-id:mime-version:date:reply-to:from:to:cc :subject:date:message-id:reply-to; bh=CYcyNvoCbMZnvtJsGMVLujGIB26i5/C23iYXX1YXZjc=; b=0cFCc1oFNGhv/Pq1KPcZ+Zf5Het5DlyRjdNr3jPKG9GCsUh840eh/lvJM+vrfDpVJh ptGXHenj0jhD4mnL8To8rBXK0xbfF9/Lfl7npBV2gHv14ujjRnW7Y2jghejDmtJ7gVR5 zQgD/+1SgZDc2ND5f267TS5KVbfp1vWKizAHixGJWTIApmBSvCILO0NRDT96Gsqq7lcH yDchLta2PptFS4vvUMDs5B65yrBX/OnpuMNLbKvuCrFIE8fWdbfHscjqYHkbo4vXOsih fGYvq8//xPHPgqdHhh2ZUdaxCu1oyI8RBP43GbaZp27pb9DkR6hlVtCdKPoJZ16+HFnm VF8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691769179; x=1692373979; h=cc:to:from:subject:message-id:mime-version:date:reply-to :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=CYcyNvoCbMZnvtJsGMVLujGIB26i5/C23iYXX1YXZjc=; b=jzchjSBfHSOsDJG4q5ELsixLZL4dCNiJF2JBA9EuA+A6dXhLLqqthrDPYBssvNHgQ4 h049VN01PWHQJ7CvLaL3OaaHfkdhBO2Nmjzs+i5Wg1AfDs4OLgR7kICR5Ryt0Jk6t7a4 0ql1zioifHnjlp97qQWzyebBPFvkLXOy9gPu0V6tyGDJ3hNmEJzyIlZ0o2CDTlZZZln1 afi6vtkQgpGQT0MCEUE/hvnBDc5p9mRLIXJoc83lTWom3e3xWjDZWmoAw9aaEv05vYYA MCZVqlzn+tnBgtdyv+iT0/d+KOuQR1qbHbEvWEWwiDALjQo+VmD3d3HIx7zRMS7GMPE5 AW8w== X-Gm-Message-State: AOJu0YzoV50RW9zG98jbkMmvS7n/nRcOcjGLQl3lHbbumsCAuTl5+xnQ FPbp8fUggWIxQsgjiB4BJnalPdLuh8Q= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:90b:fc7:b0:263:8c8b:e7b2 with SMTP id gd7-20020a17090b0fc700b002638c8be7b2mr483565pjb.3.1691769178755; Fri, 11 Aug 2023 08:52:58 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 11 Aug 2023 08:52:55 -0700 Mime-Version: 1.0 X-Mailer: git-send-email 2.41.0.694.ge786442a9b-goog Message-ID: <20230811155255.250835-1-seanjc@google.com> Subject: [PATCH] x86/retpoline: Don't clobber RFLAGS during srso_safe_ret() From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org Cc: linux-kernel@vger.kernel.org, Srikanth Aithal , kvm@vger.kernel.org, Paolo Bonzini , Sean Christopherson X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1773950916699216682 X-GMAIL-MSGID: 1773950916699216682 Use 'lea' instead of 'add' when adjusting %rsp in srso_safe_ret() so as to avoid clobbering flags. Drop one of the INT3 instructions to account for the LEA consuming one more byte than the ADD. KVM's emulator makes indirect calls into a jump table of sorts, where the destination of each call is a small blob of code that performs fast emulation by executing the target instruction with fixed operands. E.g. to emulate ADC, fastop() invokes adcb_al_dl(): adcb_al_dl: 0xffffffff8105f5f0 <+0>: adc %dl,%al 0xffffffff8105f5f2 <+2>: jmp 0xffffffff81a39270 <__x86_return_thunk> A major motivation for doing fast emulation is to leverage the CPU to handle consumption and manipulation of arithmetic flags, i.e. RFLAGS is both an input and output to the target of the call. fastop() collects the RFLAGS result by pushing RFLAGS onto the stack and popping them back into a variable (held in RDI in this case) asm("push %[flags]; popf; " CALL_NOSPEC " ; pushf; pop %[flags]\n" 0xffffffff81062be7 <+71>: mov 0xc0(%r8),%rdx 0xffffffff81062bee <+78>: mov 0x100(%r8),%rcx 0xffffffff81062bf5 <+85>: push %rdi 0xffffffff81062bf6 <+86>: popf 0xffffffff81062bf7 <+87>: call *%rsi 0xffffffff81062bf9 <+89>: nop 0xffffffff81062bfa <+90>: nop 0xffffffff81062bfb <+91>: nop 0xffffffff81062bfc <+92>: pushf 0xffffffff81062bfd <+93>: pop %rdi and then propagating the arithmetic flags into the vCPU's emulator state: ctxt->eflags = (ctxt->eflags & ~EFLAGS_MASK) | (flags & EFLAGS_MASK); 0xffffffff81062be0 <+64>: and $0xfffffffffffff72a,%r9 0xffffffff81062bfe <+94>: and $0x8d5,%edi 0xffffffff81062c0d <+109>: or %rdi,%r9 0xffffffff81062c1a <+122>: mov %r9,0x10(%r8) The failures can be most easily reproduced by running the "emulator" test in KVM-Unit-Tests. If you're feeling a bit of deja vu, see commit b63f20a778c8 ("x86/retpoline: Don't clobber RFLAGS during CALL_NOSPEC on i386"). Fixes: fb3bd914b3ec ("x86/srso: Add a Speculative RAS Overflow mitigation") Reported-by: Srikanth Aithal Closes: https://lore.kernel.org/all/de474347-122d-54cd-eabf-9dcc95ab9eae@amd.com Cc: stable@vger.kernel.org Cc: kvm@vger.kernel.org Cc: Paolo Bonzini Signed-off-by: Sean Christopherson Tested-by: Nathan Chancellor --- Those that fail to learn from history are doomed to repeat it. :-D arch/x86/lib/retpoline.S | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) base-commit: 25aa0bebba72b318e71fe205bfd1236550cc9534 diff --git a/arch/x86/lib/retpoline.S b/arch/x86/lib/retpoline.S index 2cff585f22f2..132cedbf9e57 100644 --- a/arch/x86/lib/retpoline.S +++ b/arch/x86/lib/retpoline.S @@ -164,7 +164,7 @@ __EXPORT_THUNK(srso_untrain_ret_alias) /* Needs a definition for the __x86_return_thunk alternative below. */ SYM_START(srso_safe_ret_alias, SYM_L_GLOBAL, SYM_A_NONE) #ifdef CONFIG_CPU_SRSO - add $8, %_ASM_SP + lea 8(%_ASM_SP), %_ASM_SP UNWIND_HINT_FUNC #endif ANNOTATE_UNRET_SAFE @@ -239,7 +239,7 @@ __EXPORT_THUNK(zen_untrain_ret) * SRSO untraining sequence for Zen1/2, similar to zen_untrain_ret() * above. On kernel entry, srso_untrain_ret() is executed which is a * - * movabs $0xccccccc308c48348,%rax + * movabs $0xccccc30824648d48,%rax * * and when the return thunk executes the inner label srso_safe_ret() * later, it is a stack manipulation and a RET which is mispredicted and @@ -252,11 +252,10 @@ SYM_START(srso_untrain_ret, SYM_L_GLOBAL, SYM_A_NONE) .byte 0x48, 0xb8 SYM_INNER_LABEL(srso_safe_ret, SYM_L_GLOBAL) - add $8, %_ASM_SP + lea 8(%_ASM_SP), %_ASM_SP ret int3 int3 - int3 lfence call srso_safe_ret int3