From patchwork Mon May 22 11:30:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Liang, Kan" X-Patchwork-Id: 97344 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1389035vqo; Mon, 22 May 2023 04:57:55 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4N6dV6Ge50J1PK3W/vIMtnW0PicPmG+dX83v5jCo8D5qjAb87IGgbims1gfoLNqdZFVu2U X-Received: by 2002:a17:902:f547:b0:1ae:b2a:8ed7 with SMTP id h7-20020a170902f54700b001ae0b2a8ed7mr11419753plf.29.1684756675011; Mon, 22 May 2023 04:57:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684756674; cv=none; d=google.com; s=arc-20160816; b=Lk+i80m8Cn2YdracB3N/doMFCw7IUOKuqKxHwgMUw7TqVtP9DJREqlNFZ/AVbTkTPb PZ9Ho9kVZyrqzaDSM84e3dp2J2xh9fPvYWGqwbXZSQh+6XfBXzuipEF+Q4F7vmxQkJtI FQcB2JA34J7xSyg6PLZwJMU7wcXdfLtTSIkcc1k/YW5zzH7RzhF12JlOnSG9s26QLXsO QiE1CVjEFkpL7Wk0pJGPqNPZsooG/MP8gSOBRbXqTEzM+GomyRZM20Vv/V0Pni1xxl6K PaCCC/gvtHWEERMMTdFekEcO/Iu6dQ1o8tS6U1jh4N7sqOsuwPxiSn3lihxpe90CnXji sIcQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Abodo+vgctSHQjrLTQk0AFENDU1x6gyE3OoGllgoxTc=; b=xMiCn15VrE0mePxmWC10mdy/3vamLI4PF37aDomC20Y01snVmpGlifHdjHQ+jzN1Ep AFEMDWPdV5R6ze+K3XAhx5d+c3Uv7liDvtRI1S4k1YYVRRp9hPw86dyNOzAB4UcKIORM qhlsRWXMq5WlbUWiNk41WxI3jwK0+m4ZAAUG+p3xq/5tVCVHZdVShtolm06cpWctgwqZ kGYGD47nw609ioXdmYZwiNSEWg1lJHwRO0JH/v1LAp7VH1/7MsjVUYIttxFGVpiTyyZK SLC+jlkipavnIpcjO0yW2YGzeRtCbCf4iEFagTIAZ0Vkt8ydYp97INPgxNaUPegm4ePs 37AA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=nC4RtKEL; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id o5-20020a170903210500b001a6d4eab490si4376878ple.63.2023.05.22.04.57.42; Mon, 22 May 2023 04:57:54 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=nC4RtKEL; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232879AbjEVLbQ (ORCPT + 99 others); Mon, 22 May 2023 07:31:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43986 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230523AbjEVLbM (ORCPT ); Mon, 22 May 2023 07:31:12 -0400 Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 84F29102 for ; Mon, 22 May 2023 04:30:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1684755059; x=1716291059; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=pDJxX0xEINN52I76eruquUlqRmcdL87f6osPOtH4oY0=; b=nC4RtKELTVRjBE0mV7ekWchr1Nl8wAHQqVRWTkLt9Vyb5F9evbE2IXaT ooe3BCxNVgJZd0DFFx/gh7snnMHJkMNBkxTGnPJvB9iufpw0iqpGdCVhT rApZOi9yQux4eWFG/Zspd7O2SUex6Rmm73Bk1pkxkWWGPhwe2YifYRg5o GvN8hQXM40+ACMEK2eKq+dzllUk/x2SaLH/2EAtB1Wkk1OkzJozAg+VSP Lrg5jJR7v9Mv30q7ZzmE5eJtw/Ba8t4Dzk/8/kMM8LVsnhVCMM8FKhXys bYV30fIGMR9nUHCEb6q5p0QP1QxIYGypD081ycKZk4kJPVNWeY8x4KJof w==; X-IronPort-AV: E=McAfee;i="6600,9927,10717"; a="416356739" X-IronPort-AV: E=Sophos;i="6.00,184,1681196400"; d="scan'208";a="416356739" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 May 2023 04:30:58 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10717"; a="703468252" X-IronPort-AV: E=Sophos;i="6.00,184,1681196400"; d="scan'208";a="703468252" Received: from kanliang-dev.jf.intel.com ([10.165.154.102]) by orsmga002.jf.intel.com with ESMTP; 22 May 2023 04:30:58 -0700 From: kan.liang@linux.intel.com To: peterz@infradead.org, mingo@redhat.com, acme@kernel.org, linux-kernel@vger.kernel.org Cc: mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, namhyung@kernel.org, irogers@google.com, adrian.hunter@intel.com, ak@linux.intel.com, eranian@google.com, alexey.v.bayduraev@linux.intel.com, tinghao.zhang@intel.com, Kan Liang , Sandipan Das , Ravi Bangoria , Athira Rajeev Subject: [PATCH V2 2/6] perf: Add branch stack extension Date: Mon, 22 May 2023 04:30:36 -0700 Message-Id: <20230522113040.2329924-2-kan.liang@linux.intel.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20230522113040.2329924-1-kan.liang@linux.intel.com> References: <20230522113040.2329924-1-kan.liang@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1766595414929639699?= X-GMAIL-MSGID: =?utf-8?q?1766595414929639699?= From: Kan Liang Currently, the extra information of a branch entry is stored in a u64 space. With more and more information added, the space is running out. For example, the information of occurrences of events will be added for each branch. Add an extension space to record the new information for each branch entry. The space is appended after the struct perf_branch_stack. Add a bit in struct perf_branch_entry to indicate whether the extra information is included. Reviewed-by: Andi Kleen Signed-off-by: Kan Liang Cc: Sandipan Das Cc: Ravi Bangoria Cc: Athira Rajeev Reviewed-by: Sandipan Das --- New patch - Introduce a generic extension space which can be used to store the LBR event information for Intel. It can also be used by other ARCHs for the other purpose. - Add a new bit in struct perf_branch_entry to indicate whether the extra information is included. arch/powerpc/perf/core-book3s.c | 2 +- arch/x86/events/amd/core.c | 2 +- arch/x86/events/intel/core.c | 2 +- arch/x86/events/intel/ds.c | 4 ++-- include/linux/perf_event.h | 18 +++++++++++++++++- include/uapi/linux/perf_event.h | 4 +++- kernel/events/core.c | 5 +++++ 7 files changed, 30 insertions(+), 7 deletions(-) diff --git a/arch/powerpc/perf/core-book3s.c b/arch/powerpc/perf/core-book3s.c index 8c1f7def596e..3c14596bbfaf 100644 --- a/arch/powerpc/perf/core-book3s.c +++ b/arch/powerpc/perf/core-book3s.c @@ -2313,7 +2313,7 @@ static void record_and_restart(struct perf_event *event, unsigned long val, struct cpu_hw_events *cpuhw; cpuhw = this_cpu_ptr(&cpu_hw_events); power_pmu_bhrb_read(event, cpuhw); - perf_sample_save_brstack(&data, event, &cpuhw->bhrb_stack); + perf_sample_save_brstack(&data, event, &cpuhw->bhrb_stack, NULL); } if (event->attr.sample_type & PERF_SAMPLE_DATA_SRC && diff --git a/arch/x86/events/amd/core.c b/arch/x86/events/amd/core.c index bccea57dee81..facee84aeecb 100644 --- a/arch/x86/events/amd/core.c +++ b/arch/x86/events/amd/core.c @@ -930,7 +930,7 @@ static int amd_pmu_v2_handle_irq(struct pt_regs *regs) continue; if (has_branch_stack(event)) - perf_sample_save_brstack(&data, event, &cpuc->lbr_stack); + perf_sample_save_brstack(&data, event, &cpuc->lbr_stack, NULL); if (perf_event_overflow(event, &data, regs)) x86_pmu_stop(event, 0); diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c index ba2a971e6b8a..21566f66bfd8 100644 --- a/arch/x86/events/intel/core.c +++ b/arch/x86/events/intel/core.c @@ -3048,7 +3048,7 @@ static int handle_pmi_common(struct pt_regs *regs, u64 status) perf_sample_data_init(&data, 0, event->hw.last_period); if (has_branch_stack(event)) - perf_sample_save_brstack(&data, event, &cpuc->lbr_stack); + perf_sample_save_brstack(&data, event, &cpuc->lbr_stack, NULL); if (perf_event_overflow(event, &data, regs)) x86_pmu_stop(event, 0); diff --git a/arch/x86/events/intel/ds.c b/arch/x86/events/intel/ds.c index 608e220e46aa..3f16e95e99dd 100644 --- a/arch/x86/events/intel/ds.c +++ b/arch/x86/events/intel/ds.c @@ -1747,7 +1747,7 @@ static void setup_pebs_fixed_sample_data(struct perf_event *event, setup_pebs_time(event, data, pebs->tsc); if (has_branch_stack(event)) - perf_sample_save_brstack(data, event, &cpuc->lbr_stack); + perf_sample_save_brstack(data, event, &cpuc->lbr_stack, NULL); } static void adaptive_pebs_save_regs(struct pt_regs *regs, @@ -1904,7 +1904,7 @@ static void setup_pebs_adaptive_sample_data(struct perf_event *event, if (has_branch_stack(event)) { intel_pmu_store_pebs_lbrs(lbr); - perf_sample_save_brstack(data, event, &cpuc->lbr_stack); + perf_sample_save_brstack(data, event, &cpuc->lbr_stack, NULL); } } diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h index d5628a7b5eaa..e2e04dc39199 100644 --- a/include/linux/perf_event.h +++ b/include/linux/perf_event.h @@ -126,6 +126,16 @@ struct perf_branch_stack { struct perf_branch_entry entries[]; }; +/* + * The extension space is appended after the struct perf_branch_stack. + * It is used to store the extra data of each branch, e.g., + * the occurrences of events since the last branch entry for Intel LBR. + */ +struct perf_branch_stack_ext { + __u64 nr; + __u64 data[]; +}; + struct task_struct; /* @@ -1161,6 +1171,7 @@ struct perf_sample_data { struct perf_callchain_entry *callchain; struct perf_raw_record *raw; struct perf_branch_stack *br_stack; + struct perf_branch_stack_ext *br_stack_ext; union perf_sample_weight weight; union perf_mem_data_src data_src; u64 txn; @@ -1237,7 +1248,8 @@ static inline void perf_sample_save_raw_data(struct perf_sample_data *data, static inline void perf_sample_save_brstack(struct perf_sample_data *data, struct perf_event *event, - struct perf_branch_stack *brs) + struct perf_branch_stack *brs, + struct perf_branch_stack_ext *brs_ext) { int size = sizeof(u64); /* nr */ @@ -1245,7 +1257,11 @@ static inline void perf_sample_save_brstack(struct perf_sample_data *data, size += sizeof(u64); size += brs->nr * sizeof(struct perf_branch_entry); + if (brs_ext) + size += (1 + brs_ext->nr) * sizeof(u64); + data->br_stack = brs; + data->br_stack_ext = brs_ext; data->dyn_size += size; data->sample_flags |= PERF_SAMPLE_BRANCH_STACK; } diff --git a/include/uapi/linux/perf_event.h b/include/uapi/linux/perf_event.h index 37675437b768..1b3b90965b6b 100644 --- a/include/uapi/linux/perf_event.h +++ b/include/uapi/linux/perf_event.h @@ -1410,6 +1410,7 @@ union perf_mem_data_src { * cycles: cycles from last branch (or 0 if not supported) * type: branch type * spec: branch speculation info (or 0 if not supported) + * ext: has extension space for extra info (or 0 if not supported) */ struct perf_branch_entry { __u64 from; @@ -1423,7 +1424,8 @@ struct perf_branch_entry { spec:2, /* branch speculation info */ new_type:4, /* additional branch type */ priv:3, /* privilege level */ - reserved:31; + ext:1, /* has extension */ + reserved:30; }; union perf_sample_weight { diff --git a/kernel/events/core.c b/kernel/events/core.c index 435815d3be3f..dfd6703139a1 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -7324,6 +7324,11 @@ void perf_output_sample(struct perf_output_handle *handle, if (branch_sample_hw_index(event)) perf_output_put(handle, data->br_stack->hw_idx); perf_output_copy(handle, data->br_stack->entries, size); + if (data->br_stack_ext) { + size = data->br_stack_ext->nr * sizeof(u64); + perf_output_put(handle, data->br_stack_ext->nr); + perf_output_copy(handle, data->br_stack_ext->data, size); + } } else { /* * we always store at least the value of nr