From patchwork Thu Oct 5 09:16:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: David Woodhouse X-Patchwork-Id: 148906 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2016:b0:403:3b70:6f57 with SMTP id fe22csp425757vqb; Thu, 5 Oct 2023 09:41:47 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHVHX/3db7Ps2jNkFw9+iMNCc/AjaIF/02RAkCTbXiO8b16399sjjDwohKlPOk/A0A9GmEo X-Received: by 2002:a05:6a20:72a2:b0:11f:4707:7365 with SMTP id o34-20020a056a2072a200b0011f47077365mr6969709pzk.38.1696524107244; Thu, 05 Oct 2023 09:41:47 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1696524107; cv=none; d=google.com; s=arc-20160816; b=Wc6gqUrwTRCNl20qefy8RE4tJ95AQciDsi0gh8IUmkE8kVNyDnXcFT3k56rcXQsX7i DhLrliRHRpa/cykjI6L62SUSLvL/VF0pNutrBc0XPlf25lwpxwv7SivMe7vhFA+CXiOs VqpPlfNuRPCGPw4lsLpyMftbDUe7Po/EfJtVtO18z33lbKcNfRvSl7ETPiriZ7HV+EXQ 8/vOMv+bYcfVV00TsIhOkinJ8xQWJZgqNUpE9wni+/PMxPecAcshaeTzq6vlARV3NFry sTw7dP0NF7Yk8vWk+q91RAzJbE3+8pDOHepQLNZOacy1sYZxmcfFkmF4XiJYxKQHjQl7 9QAg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:user-agent:date:cc:to:from:subject :message-id:dkim-signature; bh=QfAI7m6yJa+TxGdvW87/6sjYsbBMYOvFikYd7o63kLs=; fh=OPgxEoA93NNcvZ2Rve9gNsh8nKLRjG0ttpknGKBFGgo=; b=u3SqgL/8gCrDKmODAw3+iHHcw3SLYNCtfDJXckVxGXu36w0ukI60JEEut6rUR2fbhx 4eADWBK8VO+roWoOrlxnhUGGb1Lm2Ipj7hVEioElj7uYc90dvvN+6cSvlhwJQiavtegg E8nB8DekX+VG16xvFxsWu9et14mfgYx/B/haYn+8TcNL+G/HUwljUl+6DggOAWbi1UBV ZiYQajZCRU/0Ig0kbhv7f6XVKns9/5kvaVEDQILbwzKV+YHvIxiPgqc/SOq0uGPziw7d jSfaYMincGoWljfv0t8Dw9vsTuHf5JxnbmaCMN6at6t4mfW0V9QVSvTlXrdixFDKv5NU A75Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@infradead.org header.s=casper.20170209 header.b=H8S27kV4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from pete.vger.email (pete.vger.email. [23.128.96.36]) by mx.google.com with ESMTPS id t13-20020a056a00138d00b006935df3019esi1793566pfg.235.2023.10.05.09.41.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 05 Oct 2023 09:41:47 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) client-ip=23.128.96.36; Authentication-Results: mx.google.com; dkim=pass header.i=@infradead.org header.s=casper.20170209 header.b=H8S27kV4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by pete.vger.email (Postfix) with ESMTP id 6BF38809B094; Thu, 5 Oct 2023 09:41:14 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at pete.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231749AbjJEQjk (ORCPT + 19 others); Thu, 5 Oct 2023 12:39:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33360 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231338AbjJEQiy (ORCPT ); Thu, 5 Oct 2023 12:38:54 -0400 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 445EEA24A; Thu, 5 Oct 2023 02:16:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=MIME-Version:Content-Type:Date:Cc:To: From:Subject:Message-ID:Sender:Reply-To:Content-Transfer-Encoding:Content-ID: Content-Description:In-Reply-To:References; bh=QfAI7m6yJa+TxGdvW87/6sjYsbBMYOvFikYd7o63kLs=; b=H8S27kV4xK2GZATwUFX//mwM3U bQaqOI3oPpERVhfWhJNAne0pnSoJ+/JOzpP1Q8ve2YJiKg0a4xhtFDJjm39mGafuSLyw/PATGxZwf GVzjMN3Mv4sKtKhZ1PuD7u4dyg12S9x0p4By1JgREWsqiiFTeUvPL3aArz9QI13MHhI69mcuCEZmj G2GNtBC8UHrOrutJizokg/f4wMFBn2b212beZkhGwZYZVL9tgLzR4wc1hczedeUSPRGPBA0wK0UQA 0/CzsB9MkVQoM9hiw4JOrTT9L5RqhD7SwmQNwvcLhey4I3zg/toT9VdwxKYQQV2aYi4Oad07cz5vo cm50EGNw==; Received: from [2001:8b0:10b:5:ac17:af40:f98e:776c] (helo=u3832b3a9db3152.ant.amazon.com) by casper.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1qoKSp-008YhI-66; Thu, 05 Oct 2023 09:16:11 +0000 Message-ID: Subject: [PATCH v2] KVM: x86: Refine calculation of guest wall clock to use a single TSC read From: David Woodhouse To: kvm@vger.kernel.org Cc: Sean Christopherson , Paolo Bonzini , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Paul Durrant , linux-kernel Date: Thu, 05 Oct 2023 10:16:10 +0100 User-Agent: Evolution 3.44.4-0ubuntu2 MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Thu, 05 Oct 2023 09:41:14 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1778934461834281463 X-GMAIL-MSGID: 1778934461834281463 From: David Woodhouse When populating the guest's PV wall clock information, KVM currently does a simple 'kvm_get_real_ns() - get_kvmclock_ns(kvm)'. This is an antipattern which should be avoided; when working with the relationship between two clocks, it's never correct to obtain one of them "now" and then the other at a slightly different "now" after an unspecified period of preemption (which might not even be under the control of the kernel, if this is an L1 hosting an L2 guest under nested virtualization). Add a kvm_get_wall_clock_epoch() function to return the guest wall clock epoch in nanoseconds using the same method as __get_kvmclock() — by using kvm_get_walltime_and_clockread() to calculate both the wall clock and KVM clock time from a *single* TSC reading. The condition using get_cpu_tsc_khz() is equivalent to the version in __get_kvmclock() which separately checks for the CONSTANT_TSC feature or the per-CPU cpu_tsc_khz. Which is what get_cpu_tsc_khz() does anyway. Signed-off-by: David Woodhouse --- • v2: Improve comments, zero local_tsc_khz each time round the loop, move put_cpu() a little earlier. arch/x86/kvm/x86.c | 85 ++++++++++++++++++++++++++++++++++++++++++---- arch/x86/kvm/x86.h | 2 ++ arch/x86/kvm/xen.c | 4 +-- 3 files changed, 82 insertions(+), 9 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index a3a02d62aa6a..14267bacd5db 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -2331,14 +2331,9 @@ static void kvm_write_wall_clock(struct kvm *kvm, gpa_t wall_clock, int sec_hi_o if (kvm_write_guest(kvm, wall_clock, &version, sizeof(version))) return; - /* - * The guest calculates current wall clock time by adding - * system time (updated by kvm_guest_time_update below) to the - * wall clock specified here. We do the reverse here. - */ - wall_nsec = ktime_get_real_ns() - get_kvmclock_ns(kvm); + wall_nsec = kvm_get_wall_clock_epoch(kvm); - wc.nsec = do_div(wall_nsec, 1000000000); + wc.nsec = do_div(wall_nsec, NSEC_PER_SEC); wc.sec = (u32)wall_nsec; /* overflow in 2106 guest time */ wc.version = version; @@ -3243,6 +3238,82 @@ static int kvm_guest_time_update(struct kvm_vcpu *v) return 0; } +/* + * The pvclock_wall_clock ABI tells the guest the wall clock time at + * which it started (i.e. its epoch, when its kvmclock was zero). + * + * In fact those clocks are subtly different; wall clock frequency is + * adjusted by NTP and has leap seconds, while the kvmclock is a + * simple function of the TSC without any such adjustment. + * + * Perhaps the ABI should have exposed CLOCK_TAI and a ratio between + * that and kvmclock, but even that would be subject to change over + * time. + * + * Attempt to calculate the epoch at a given moment using the *same* + * TSC reading via kvm_get_walltime_and_clockread() to obtain both + * wallclock and kvmclock times, and subtracting one from the other. + * + * Fall back to using their values at slightly different moments by + * calling ktime_get_real_ns() and get_kvmclock_ns() separately. + */ +uint64_t kvm_get_wall_clock_epoch(struct kvm *kvm) +{ +#ifdef CONFIG_X86_64 + struct pvclock_vcpu_time_info hv_clock; + struct kvm_arch *ka = &kvm->arch; + unsigned long seq, local_tsc_khz; + struct timespec64 ts; + uint64_t host_tsc; + + do { + seq = read_seqcount_begin(&ka->pvclock_sc); + + local_tsc_khz = 0; + if (!ka->use_master_clock) + break; + + /* + * The TSC read and the call to get_cpu_tsc_khz() must happen + * on the same CPU. + */ + get_cpu(); + + local_tsc_khz = get_cpu_tsc_khz(); + + if (local_tsc_khz && + !kvm_get_walltime_and_clockread(&ts, &host_tsc)) + local_tsc_khz = 0; /* Fall back to old method */ + + put_cpu(); + + /* + * These values must be snapshotted within the seqcount loop. + * After that, it's just mathematics which can happen on any + * CPU at any time. + */ + hv_clock.tsc_timestamp = ka->master_cycle_now; + hv_clock.system_time = ka->master_kernel_ns + ka->kvmclock_offset; + + } while (read_seqcount_retry(&ka->pvclock_sc, seq)); + + /* + * If the conditions were right, and obtaining the wallclock+TSC was + * successful, calculate the KVM clock at the corresponding time and + * subtract one from the other to get the guest's epoch in nanoseconds + * since 1970-01-01. + */ + if (local_tsc_khz) { + kvm_get_time_scale(NSEC_PER_SEC, local_tsc_khz * NSEC_PER_USEC, + &hv_clock.tsc_shift, + &hv_clock.tsc_to_system_mul); + return ts.tv_nsec + NSEC_PER_SEC * ts.tv_sec - + __pvclock_read_cycles(&hv_clock, host_tsc); + } +#endif + return ktime_get_real_ns() - get_kvmclock_ns(kvm); +} + /* * kvmclock updates which are isolated to a given vcpu, such as * vcpu->cpu migration, should not allow system_timestamp from diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index 1e7be1f6ab29..ed1a69942347 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -290,6 +290,8 @@ static inline bool kvm_check_has_quirk(struct kvm *kvm, u64 quirk) return !(kvm->arch.disabled_quirks & quirk); } +uint64_t kvm_get_wall_clock_epoch(struct kvm *kvm); + void kvm_inject_realmode_interrupt(struct kvm_vcpu *vcpu, int irq, int inc_eip); u64 get_kvmclock_ns(struct kvm *kvm); diff --git a/arch/x86/kvm/xen.c b/arch/x86/kvm/xen.c index c539f18e0b60..e53fad915a62 100644 --- a/arch/x86/kvm/xen.c +++ b/arch/x86/kvm/xen.c @@ -59,7 +59,7 @@ static int kvm_xen_shared_info_init(struct kvm *kvm, gfn_t gfn) * This code mirrors kvm_write_wall_clock() except that it writes * directly through the pfn cache and doesn't mark the page dirty. */ - wall_nsec = ktime_get_real_ns() - get_kvmclock_ns(kvm); + wall_nsec = kvm_get_wall_clock_epoch(kvm); /* It could be invalid again already, so we need to check */ read_lock_irq(&gpc->lock); @@ -98,7 +98,7 @@ static int kvm_xen_shared_info_init(struct kvm *kvm, gfn_t gfn) wc_version = wc->version = (wc->version + 1) | 1; smp_wmb(); - wc->nsec = do_div(wall_nsec, 1000000000); + wc->nsec = do_div(wall_nsec, NSEC_PER_SEC); wc->sec = (u32)wall_nsec; *wc_sec_hi = wall_nsec >> 32; smp_wmb();