From patchwork Wed Jan 3 03:14:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Mi, Dapeng" X-Patchwork-Id: 184595 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:6f82:b0:100:9c79:88ff with SMTP id tb2csp4806993dyb; Tue, 2 Jan 2024 19:14:15 -0800 (PST) X-Google-Smtp-Source: AGHT+IEfCq0iGosF6PV4oB21TS5nYYjL62J3n1IGsIxGeu/atkIR+QDu0UKBDQOvyjMxPcUxDiJW X-Received: by 2002:a17:903:22ca:b0:1d4:ceaf:e759 with SMTP id y10-20020a17090322ca00b001d4ceafe759mr732281plg.32.1704251655428; Tue, 02 Jan 2024 19:14:15 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1704251655; cv=none; d=google.com; s=arc-20160816; b=ZKhdsv3ukh+21ahW/yNa0NcZzYQoxjH5L56mocvhjOSf35sF6glcXQ4DQlT/Kq93EM /CFfOGhsBmCVeFzdYWEosew1424X+53T4LFJJWD74jdG8UAaV3CMup/Wo4BpUrM5ZJal sRoukJTAdo3MxPzuzdnEnwfg3/HXxKqVQXarcOHl6TNlj1/aLItIcb/ISbpfVF/Ws6cp PhYaEc6dFOaHwUe+PePOBFup0zusYN1vYWOe31SS/6OLXJdbHwFW1MtUmMXXxPhAjfuh GXmr9H7sRi2vyDN1FwL3sFtBVX0oUtoR4kbuAP5RAmgWITSo8tqrp7SvDkoYJIWrxwnA pRFg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=msJ6c3J2HW8/jr71ECpmogxTxPX9xcaVKgVRmGAQujA=; fh=YsfTl7qN5yLiZ/GhYjuBdDgWBv8FNcMbD5266uBeHkY=; b=GZ0mH8Wtx5Lx93ajUxEXrAOLuHbKfmWYFrUGEBRtlRB1BYbiyvmdJUKkuZu+eBKe8h atXwBdrtgWz/Kqgel/RND6VlBeXJY+3TFo7iUdfoL3sJLgg0bUMUzeeezh0kqgt4P1mr 9sFRNw43mS8JRDJE4qoEOSnv5sOF/6nNUrmENQtFvhplL/yTnGW6HLOxTVRANsUqwn+n avZLr6NdWN7GLtyDEuj3rC0Qa3cMLD4KBSMh/L2jOMJL5yqBDQUzviewTBxic3WDmc2+ ATT0/KKCo2pCHQC/w1jzjeNjOMxqgPsT1KFKIuiWKGYZtwpaLWZ+fTMdXzS4NlG+sl/B AE0A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=ixumY8aB; spf=pass (google.com: domain of linux-kernel+bounces-15125-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-15125-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [147.75.48.161]) by mx.google.com with ESMTPS id t12-20020a17090a024c00b0028c17b55221si479793pje.16.2024.01.02.19.14.15 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jan 2024 19:14:15 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-15125-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=ixumY8aB; spf=pass (google.com: domain of linux-kernel+bounces-15125-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-15125-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id F2113B21CF2 for ; Wed, 3 Jan 2024 03:13:57 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id DFB4119BCB; Wed, 3 Jan 2024 03:10:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="ixumY8aB" X-Original-To: linux-kernel@vger.kernel.org Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0F0B5199BA; Wed, 3 Jan 2024 03:10:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1704251407; x=1735787407; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=z72wvalERNtglXdFe8pPE83KEOIKaIWcyGSzLUbOvDI=; b=ixumY8aBQOIA1m9LZaMEx+qy7AZHiVz52HmKzcu6l//fAJ2NBF/2zVBr mpERtf11o9D85zSiw6up3ufVuXrdPKgyvr+YZbr77JJiXQtpCdaWTthqZ qVFvaXE6xEd9GDKbedJETIzdxNkni54XtGHLKdB9P06E95B1v8We4TU2W 3fc2vB0nZrMucpDayYizIcgZYssl+lZAQNaFpOGDjWPiswYLhPrbmJmIq fiDx6aRfC8sNmqdmOj7eCEezwdHjMXWe83Wfh8dPaq1lhGfRAhq8MHh9J t1RUbn/0M7S+uWnlzYw+Fj8RwWRxkTg6ivKpDHIRWskGVZfftrG75ACwQ Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10941"; a="10343176" X-IronPort-AV: E=Sophos;i="6.04,326,1695711600"; d="scan'208";a="10343176" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Jan 2024 19:10:06 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10941"; a="729666003" X-IronPort-AV: E=Sophos;i="6.04,326,1695711600"; d="scan'208";a="729666003" Received: from dmi-pnp-i7.sh.intel.com ([10.239.159.155]) by orsmga003.jf.intel.com with ESMTP; 02 Jan 2024 19:10:02 -0800 From: Dapeng Mi To: Sean Christopherson , Paolo Bonzini , Jim Mattson Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Zhenyu Wang , Zhang Xiong , Mingwei Zhang , Like Xu , Jinrong Liang , Dapeng Mi , Dapeng Mi Subject: [kvm-unit-tests Patch v3 10/11] x86: pmu: Add IBPB indirect jump asm blob Date: Wed, 3 Jan 2024 11:14:08 +0800 Message-Id: <20240103031409.2504051-11-dapeng1.mi@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240103031409.2504051-1-dapeng1.mi@linux.intel.com> References: <20240103031409.2504051-1-dapeng1.mi@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1787037383855407190 X-GMAIL-MSGID: 1787037383855407190 Currently the lower boundary of branch misses event is set to 0. Strictly speaking 0 shouldn't be a valid count since it can't tell us if branch misses event counter works correctly or even disabled. Whereas it's also possible and reasonable that branch misses event count is 0 especailly for such simple loop() program with advanced branch predictor. To eliminate such ambiguity and make branch misses event verification more acccurately, an extra IBPB indirect jump asm blob is appended and IBPB command is leveraged to clear the branch target buffer and force to cause a branch miss for the indirect jump. Suggested-by: Jim Mattson Signed-off-by: Dapeng Mi --- x86/pmu.c | 56 +++++++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 42 insertions(+), 14 deletions(-) diff --git a/x86/pmu.c b/x86/pmu.c index 8fd3db0fbf81..c8d4a0dcd362 100644 --- a/x86/pmu.c +++ b/x86/pmu.c @@ -27,14 +27,26 @@ "nop; nop; nop; nop; nop; nop; nop;\n\t" \ "loop 1b;\n\t" -/*Enable GLOBAL_CTRL + disable GLOBAL_CTRL + clflush/mfence instructions */ -#define PRECISE_EXTRA_INSTRNS (2 + 4 + 2) +#define IBPB_JMP_INSTRNS 7 +#define IBPB_JMP_BRANCHES 1 +#define IBPB_JMP_ASM(_wrmsr) \ + "mov $1, %%eax; xor %%edx, %%edx;\n\t" \ + "mov $73, %%ecx;\n\t" \ + _wrmsr "\n\t" \ + "lea 2f, %%rax;\n\t" \ + "jmp *%%rax;\n\t" \ + "nop;\n\t" \ + "2: nop;\n\t" + +/* GLOBAL_CTRL enable + disable + clflush/mfence + IBPB_JMP */ +#define PRECISE_EXTRA_INSTRNS (2 + 4 + 2 + IBPB_JMP_INSTRNS) #define PRECISE_LOOP_INSTRNS (N * LOOP_INSTRNS + PRECISE_EXTRA_INSTRNS) -#define PRECISE_LOOP_BRANCHES (N) -#define PRECISE_LOOP_ASM(_clflush) \ +#define PRECISE_LOOP_BRANCHES (N + IBPB_JMP_BRANCHES) +#define PRECISE_LOOP_ASM(_clflush, _wrmsr) \ "wrmsr;\n\t" \ "mov %%ecx, %%edi; mov %%ebx, %%ecx;\n\t" \ LOOP_ASM(_clflush) \ + IBPB_JMP_ASM(_wrmsr) \ "mov %%edi, %%ecx; xor %%eax, %%eax; xor %%edx, %%edx;\n\t" \ "wrmsr;\n\t" @@ -74,30 +86,42 @@ char *buf; static struct pmu_event *gp_events; static unsigned int gp_events_size; -#define _loop_asm(_clflush) \ +#define _loop_asm(_clflush, _wrmsr) \ do { \ asm volatile(LOOP_ASM(_clflush) \ + IBPB_JMP_ASM(_wrmsr) \ : "=c"(tmp), "=r"(tmp2), "=r"(tmp3) \ - : "0"(N), "1"(buf)); \ + : "0"(N), "1"(buf) \ + : "eax", "edx"); \ } while (0) -#define _precise_loop_asm(_clflush) \ +#define _precise_loop_asm(_clflush, _wrmsr) \ do { \ - asm volatile(PRECISE_LOOP_ASM(_clflush) \ + asm volatile(PRECISE_LOOP_ASM(_clflush, _wrmsr) \ : "=b"(tmp), "=r"(tmp2), "=r"(tmp3) \ : "a"(eax), "d"(edx), "c"(global_ctl), \ "0"(N), "1"(buf) \ : "edi"); \ } while (0) +static int has_ibpb(void) +{ + return this_cpu_has(X86_FEATURE_SPEC_CTRL) || + this_cpu_has(X86_FEATURE_AMD_IBPB); +} + static inline void __loop(void) { unsigned long tmp, tmp2, tmp3; - if (this_cpu_has(X86_FEATURE_CLFLUSH)) - _loop_asm("clflush (%1)"); + if (this_cpu_has(X86_FEATURE_CLFLUSH) && has_ibpb()) + _loop_asm("clflush (%1)", "wrmsr"); + else if (this_cpu_has(X86_FEATURE_CLFLUSH)) + _loop_asm("clflush (%1)", "nop"); + else if (has_ibpb()) + _loop_asm("nop", "wrmsr"); else - _loop_asm("nop"); + _loop_asm("nop", "nop"); } /* @@ -114,10 +138,14 @@ static inline void __precise_count_loop(u64 cntrs) u32 eax = cntrs & (BIT_ULL(32) - 1); u32 edx = cntrs >> 32; - if (this_cpu_has(X86_FEATURE_CLFLUSH)) - _precise_loop_asm("clflush (%1)"); + if (this_cpu_has(X86_FEATURE_CLFLUSH) && has_ibpb()) + _precise_loop_asm("clflush (%1)", "wrmsr"); + else if (this_cpu_has(X86_FEATURE_CLFLUSH)) + _precise_loop_asm("clflush (%1)", "nop"); + else if (has_ibpb()) + _precise_loop_asm("nop", "wrmsr"); else - _precise_loop_asm("nop"); + _precise_loop_asm("nop", "nop"); } static inline void loop(u64 cntrs)