From patchwork Wed Jan 18 06:05:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Namhyung Kim X-Patchwork-Id: 44986 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2171153wrn; Tue, 17 Jan 2023 22:22:27 -0800 (PST) X-Google-Smtp-Source: AMrXdXuMjUCkldfbI1ouckSKpCJ+P5+XwI5ZXUJUT0WkEPQ7+QqMcMRHge/maD4EiqzupNajv57S X-Received: by 2002:a05:6a00:26e3:b0:58b:c4b4:86e3 with SMTP id p35-20020a056a0026e300b0058bc4b486e3mr15832120pfw.24.1674022946709; Tue, 17 Jan 2023 22:22:26 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674022946; cv=none; d=google.com; s=arc-20160816; b=AIW++gA08l7tfH6V3hyMxob/ZoMN9hr9ujolOml4cSz2Wi2pgs6PQ6TTvl4HE4x8tj cA4MjN576Ex5iBHVBl9NZKT0PlurIn3Dlo08rbMI3w8TRjdzZS/vYJVCiou9H05TTvEV Vlk+AO/faumjLBefDRzIF0H56DsNcNh4tW5X1w6ahGTy08Z5sBDKsulC+fU4F767I3gk HBqdLpi8+m0kuOCnLVULIc1bPT9dXJ2NEzkS10XNCMArCY9mUpVvxFI/r396TrczqDe3 LO7JGnlbsv8fXaLzqaNZBC+Kd/rISENlE0YtRw6hx3w2NmD7f5MRW0Oct6g9z+Fuj9Jy SqqA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=Q4FWYNGqeIRLLvbuHSi4QxIqv+Z+4df4xqBN1sLuHUI=; b=INHRVdCan3z5JRQzVI1nR10icfq2Tg39pUdV2obU4/J5S3qagY7Ah1NL9JpH4J5d1Z DFRsTfKS2YtMPOang8kPrfxh+QmhLCP3VC1KoRfSvLCfOm46uKhcr+dKx5YVjCNs5HEg VUvHJFpU5b4VVEaQC8bkgfDLOOkiSg+ecBuFeJbCnULUIHJlORe5EzUgYbtpQ6sKxPyV Y0bTKPGsW65hCYINjutxDap3uT8SXAiOralLaI6OOnqTUPwYMG0ZhDXQopPIvvfv8aSg bfzJSrTcabRNLS3b7tgibo3o3AMBrwpzPHAZepwHfs+QXic9SjzRRxa9IVnXKcW6XLBC cbcw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=LWvdJAHv; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id q21-20020a056a00089500b005827904e343si36598875pfj.220.2023.01.17.22.22.14; Tue, 17 Jan 2023 22:22:26 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=LWvdJAHv; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229475AbjARGUh (ORCPT + 99 others); Wed, 18 Jan 2023 01:20:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53472 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229521AbjARGSR (ORCPT ); Wed, 18 Jan 2023 01:18:17 -0500 Received: from mail-pg1-x533.google.com (mail-pg1-x533.google.com [IPv6:2607:f8b0:4864:20::533]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 03C7D442D4; Tue, 17 Jan 2023 22:06:03 -0800 (PST) Received: by mail-pg1-x533.google.com with SMTP id q9so23748273pgq.5; Tue, 17 Jan 2023 22:06:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=Q4FWYNGqeIRLLvbuHSi4QxIqv+Z+4df4xqBN1sLuHUI=; b=LWvdJAHv1jZf2z/7zujPgBAeAQ9KRsb7GuuOIaayNaFGOHtclEFJNra3G9kr6/uxei zR2Juw7BZwh2ifMPPiDuNeZeXPCA48C1PahDHi7cWjsCw8fBkvm2qfeIw5L5lltkzDFF WYb58icJKGcvZWXpubVZToPAOkDHgyJEr2xn9IP/zyx/kxIb0TLonxdTcmBotQFeeMGR JQhjTOfvONTkw+R2EQ5+xeCiOtjEpdsE3/xy4H4W8R8E5UwLyNrR3JYh4rldqVb+VWEe b1jWcd8/kyha9QadyEQuIu7zIy4uzz/xsHbt8GObU0YbAW2yMi9EnRC+mFAVX7DuiN6f oEDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=Q4FWYNGqeIRLLvbuHSi4QxIqv+Z+4df4xqBN1sLuHUI=; b=tSgeZaUOzcqGTPO0151PFgXi2crUWlIySGcLGi50TiLLH9C2ZfQ5AmQ3XTFTPPOUFj 7ToHVAqXw3/3wJKGIXudIGsx/GRNb5TFBBoBT4ag4kb55x2Xhu8UH5F0SFrA39pMcVwz M1INgCjrjnwZxK37vPygzj8GyZ7mXMJFVM2uZwY7JQOBbVLXb6VZ7ZeMmustEVb5iXUL X0GQ9QrajGl1pyEj2QPJPk/Ox8PYH3xg221rKbiwkbohYZKfdAng6QOYS6AAvyu9Nt7m KPtQhwQ22+a+NOGpvvKmqCXxdqGrg+WSO6x0q2ullJEWZEbJsYUh3APEMWc6q3LooGBo +wQg== X-Gm-Message-State: AFqh2koWr93iWJFmfwks3xGl3mHurvrko9IhDcILICZgCp3Y2M+NzeV7 JtpFGKs2+Uo7BHO2mAtatnSlCvXYylA= X-Received: by 2002:a62:640f:0:b0:581:f301:23fc with SMTP id y15-20020a62640f000000b00581f30123fcmr24054116pfb.12.1674021962020; Tue, 17 Jan 2023 22:06:02 -0800 (PST) Received: from balhae.hsd1.ca.comcast.net ([2601:647:6780:ff0:3749:9eb3:dfb5:f449]) by smtp.gmail.com with ESMTPSA id b126-20020a62cf84000000b00574e84ed847sm10990180pfg.24.2023.01.17.22.06.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Jan 2023 22:06:01 -0800 (PST) Sender: Namhyung Kim From: Namhyung Kim To: Peter Zijlstra , Ingo Molnar Cc: Mark Rutland , Alexander Shishkin , Arnaldo Carvalho de Melo , Jiri Olsa , LKML , Song Liu , bpf@vger.kernel.org Subject: [PATCH 1/8] perf/core: Save the dynamic parts of sample data size Date: Tue, 17 Jan 2023 22:05:52 -0800 Message-Id: <20230118060559.615653-2-namhyung@kernel.org> X-Mailer: git-send-email 2.39.0.314.g84b9a713c41-goog In-Reply-To: <20230118060559.615653-1-namhyung@kernel.org> References: <20230118060559.615653-1-namhyung@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1755340285139165379?= X-GMAIL-MSGID: =?utf-8?q?1755340285139165379?= The perf sample data can be divided into parts. The event->header_size and event->id_header_size keep the static part of the sample data which is determined by the sample_type flags. But other parts like CALLCHAIN and BRANCH_STACK are changing dynamically so it needs to see the actual data. In preparation of handling repeated calls for perf_prepare_sample(), it can save the dynamic size to the perf sample data to avoid the duplicate work. Acked-by: Jiri Olsa Acked-by: Song Liu Tested-by: Jiri Olsa Signed-off-by: Namhyung Kim --- include/linux/perf_event.h | 2 ++ kernel/events/core.c | 17 ++++++++++------- 2 files changed, 12 insertions(+), 7 deletions(-) diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h index 03949d017ac9..16b980014449 100644 --- a/include/linux/perf_event.h +++ b/include/linux/perf_event.h @@ -1103,6 +1103,7 @@ struct perf_sample_data { */ u64 sample_flags; u64 period; + u64 dyn_size; /* * Fields commonly set by __perf_event_header__init_id(), @@ -1158,6 +1159,7 @@ static inline void perf_sample_data_init(struct perf_sample_data *data, /* remaining struct members initialized in perf_prepare_sample() */ data->sample_flags = PERF_SAMPLE_PERIOD; data->period = period; + data->dyn_size = 0; if (addr) { data->addr = addr; diff --git a/kernel/events/core.c b/kernel/events/core.c index eacc3702654d..8c8de26f04ab 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -7593,7 +7593,7 @@ void perf_prepare_sample(struct perf_event_header *header, size += data->callchain->nr; - header->size += size * sizeof(u64); + data->dyn_size += size * sizeof(u64); } if (sample_type & PERF_SAMPLE_RAW) { @@ -7619,7 +7619,7 @@ void perf_prepare_sample(struct perf_event_header *header, data->raw = NULL; } - header->size += size; + data->dyn_size += size; } if (sample_type & PERF_SAMPLE_BRANCH_STACK) { @@ -7631,7 +7631,7 @@ void perf_prepare_sample(struct perf_event_header *header, size += data->br_stack->nr * sizeof(struct perf_branch_entry); } - header->size += size; + data->dyn_size += size; } if (sample_type & (PERF_SAMPLE_REGS_USER | PERF_SAMPLE_STACK_USER)) @@ -7646,7 +7646,7 @@ void perf_prepare_sample(struct perf_event_header *header, size += hweight64(mask) * sizeof(u64); } - header->size += size; + data->dyn_size += size; } if (sample_type & PERF_SAMPLE_STACK_USER) { @@ -7671,7 +7671,7 @@ void perf_prepare_sample(struct perf_event_header *header, size += sizeof(u64) + stack_size; data->stack_user_size = stack_size; - header->size += size; + data->dyn_size += size; } if (filtered_sample_type & PERF_SAMPLE_WEIGHT_TYPE) @@ -7700,7 +7700,7 @@ void perf_prepare_sample(struct perf_event_header *header, size += hweight64(mask) * sizeof(u64); } - header->size += size; + data->dyn_size += size; } if (sample_type & PERF_SAMPLE_PHYS_ADDR && @@ -7745,8 +7745,11 @@ void perf_prepare_sample(struct perf_event_header *header, size = perf_prepare_sample_aux(event, data, size); WARN_ON_ONCE(size + header->size > U16_MAX); - header->size += size; + data->dyn_size += size + sizeof(u64); /* size above */ } + + header->size += data->dyn_size; + /* * If you're adding more sample types here, you likely need to do * something about the overflowing header::size, like repurpose the From patchwork Wed Jan 18 06:05:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Namhyung Kim X-Patchwork-Id: 45037 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2178710wrn; Tue, 17 Jan 2023 22:46:32 -0800 (PST) X-Google-Smtp-Source: AMrXdXumvMqQGMds+o5DvscWoQY46iaAEmRIrHXJEIH93ZYXieIZGAtnfyP8WsvznS7QF/y5SwEa X-Received: by 2002:a17:90a:b38c:b0:223:4308:1ab4 with SMTP id e12-20020a17090ab38c00b0022343081ab4mr6010622pjr.46.1674024392593; Tue, 17 Jan 2023 22:46:32 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674024392; cv=none; d=google.com; s=arc-20160816; b=zrYUln+j/LjHDSb+YLe/KBOu60WYi5dQZOesL3qCVKvkOPeaFaXXodBb5Apos/eSQp PDGjb9KJED1wynBUmIakdeD4OmZr7ELGhMDARLDOaTpIzdKOCJTeSTdy4zsfUDZzkmv6 Exz51rzvDdDTJ52EI7uU1mIq4UF4M8fIwQjlOgWNsLw7F+s7xZGLIZfup2oe1qHAP3TV bUUcj75BJ+4J3e4WOTl+sjyZcARp9ByQ7Tt1MvQie9aKUr5Sp3UwKrZJ2N+RH0khWjda KgEGHd3w2FTs2aK0H0MFWzNRgSo/4fCXQ+/+05eb8yv+0AOSs88Rv45M4KdnbllVLMjF UroA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=Xzb3dDf80FIRCQmibSPI5U5nXORwuSiNGSgRt/kZLJ0=; b=MCVaV1sbRYH8+LOfgf+HAHrDlD/PEC04UvvEsTYIEdn1x7x3Lp/ZiWrHcFr7L5JKkM cqub9MmkNHbiEQNV0B5gSk9G8CHl7s4terD7gG3SDx9XFHxHNxsvYVO5nrdmcITpegwy JLc91E1laMRU6HczSt0iDsEVNvuKtGxLmjqGSWjYEcDl9uGfssS9YI9eQ0VaWJ2Z7sFP UCUvOEcAuNj/Z4n9PuZFc2aRQWYwp6UaZP0LUtGwo32DJ1xZWlt3Qth2dF+K1PkAJyVB 88oJc+nQr4Jzm5SojEELnt7/ylNpz7afMje9NXucEO7sugmAczzkCEbMSbzgjnfuczDK QKyg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=ktpzLfRd; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id nr1-20020a17090b240100b002199a9891desi1142335pjb.141.2023.01.17.22.46.20; Tue, 17 Jan 2023 22:46:32 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=ktpzLfRd; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229666AbjARGoX (ORCPT + 99 others); Wed, 18 Jan 2023 01:44:23 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57038 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229802AbjARGZy (ORCPT ); Wed, 18 Jan 2023 01:25:54 -0500 Received: from mail-pj1-x102e.google.com (mail-pj1-x102e.google.com [IPv6:2607:f8b0:4864:20::102e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7440D552B3; Tue, 17 Jan 2023 22:14:26 -0800 (PST) Received: by mail-pj1-x102e.google.com with SMTP id u1-20020a17090a450100b0022936a63a21so1267093pjg.4; Tue, 17 Jan 2023 22:14:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=Xzb3dDf80FIRCQmibSPI5U5nXORwuSiNGSgRt/kZLJ0=; b=ktpzLfRdqURJuPDPKtq3ilBF/shlSgM+zuRpAf6OrVPdaUahGR82u/YgTl0v+6Y7S2 UXi7fChjOc2fX/K2UsltwuKWnVpQW3M7e1MZWB361SDU4d4sy7nKEnD+YkbEL+Kxi09S fgAINUrK9U7b6YCXtKY8y2CaP6CWmefcU/nnQsXW7WdxHjHef/h8icQTgnho0VpcW/hn 7oVF7Brnx5fsgldJZcPCutOsUUkT5e7SuQDvdmXLLhcIH1JPL/tiowMnNdIWbB4+p6eR uwbxYax8x4khmav4eyaMznL2YQ8uybHKOUd43982irD7ZXorKN5n7KdQgt+4dhJEbUnP pwjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=Xzb3dDf80FIRCQmibSPI5U5nXORwuSiNGSgRt/kZLJ0=; b=c2YX7KQ5XqCMkN7BS7HneeosTaUgojjHi24t9Qh1uZz/RD3Fi3BnFBlIztiB+mSXlf msPNJpVqImRvWixZse5vg9o6mQjBfkVVps5ePhPsubju1+CyU8qpAOQmvmsswPYX7PXm 8jEHoTMeWVifJP694d7gpEhI5WuM6vJvww9oM+wgE45ly+1mtnAmW1KZyZGI5Q8gm4cd HpJf+BBUcKLBxoVYB764N2qWI+M4aSAOwrrOf0z1id+xAajCYOKhKOs+ko0y0O96DhyN P/JVequchCB7nuN8xc0Ebz7MSeGhi8zLd1m8sv8nFMGS+k2YdRJyP3h5RjJ2Tk8hNUvO WYXA== X-Gm-Message-State: AFqh2kpsIrhKp0OrTtM5Z2VbCPs4sYHuDoLi1GaUpcX3Qn9MBgH6KF5f GZ8hKijBB1ORw9xxBlM1a6tS+fY6KOA= X-Received: by 2002:a05:6a00:4ac8:b0:581:bae0:d5d5 with SMTP id ds8-20020a056a004ac800b00581bae0d5d5mr5814871pfb.9.1674021963543; Tue, 17 Jan 2023 22:06:03 -0800 (PST) Received: from balhae.hsd1.ca.comcast.net ([2601:647:6780:ff0:3749:9eb3:dfb5:f449]) by smtp.gmail.com with ESMTPSA id b126-20020a62cf84000000b00574e84ed847sm10990180pfg.24.2023.01.17.22.06.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Jan 2023 22:06:03 -0800 (PST) Sender: Namhyung Kim From: Namhyung Kim To: Peter Zijlstra , Ingo Molnar Cc: Mark Rutland , Alexander Shishkin , Arnaldo Carvalho de Melo , Jiri Olsa , LKML , Song Liu , bpf@vger.kernel.org, x86@kernel.org Subject: [PATCH 2/8] perf/core: Add perf_sample_save_callchain() helper Date: Tue, 17 Jan 2023 22:05:53 -0800 Message-Id: <20230118060559.615653-3-namhyung@kernel.org> X-Mailer: git-send-email 2.39.0.314.g84b9a713c41-goog In-Reply-To: <20230118060559.615653-1-namhyung@kernel.org> References: <20230118060559.615653-1-namhyung@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1755341801112084352?= X-GMAIL-MSGID: =?utf-8?q?1755341801112084352?= When it saves the callchain to the perf sample data, it needs to update the sample flags and the dynamic size. To make sure this, add the perf_sample_save_callchain() helper and convert all call sites. Cc: x86@kernel.org Suggested-by: Peter Zijlstra Acked-by: Jiri Olsa Acked-by: Song Liu Tested-by: Jiri Olsa Signed-off-by: Namhyung Kim --- arch/x86/events/amd/ibs.c | 6 ++---- arch/x86/events/intel/ds.c | 12 ++++-------- include/linux/perf_event.h | 16 +++++++++++++++- kernel/events/core.c | 12 ++---------- 4 files changed, 23 insertions(+), 23 deletions(-) diff --git a/arch/x86/events/amd/ibs.c b/arch/x86/events/amd/ibs.c index da3f5ebac4e1..417c80bd3274 100644 --- a/arch/x86/events/amd/ibs.c +++ b/arch/x86/events/amd/ibs.c @@ -1122,10 +1122,8 @@ static int perf_ibs_handle_irq(struct perf_ibs *perf_ibs, struct pt_regs *iregs) * recorded as part of interrupt regs. Thus we need to use rip from * interrupt regs while unwinding call stack. */ - if (event->attr.sample_type & PERF_SAMPLE_CALLCHAIN) { - data.callchain = perf_callchain(event, iregs); - data.sample_flags |= PERF_SAMPLE_CALLCHAIN; - } + if (event->attr.sample_type & PERF_SAMPLE_CALLCHAIN) + perf_sample_save_callchain(&data, event, iregs); throttle = perf_event_overflow(event, &data, ®s); out: diff --git a/arch/x86/events/intel/ds.c b/arch/x86/events/intel/ds.c index 6ec326b47e2e..158cf845fc80 100644 --- a/arch/x86/events/intel/ds.c +++ b/arch/x86/events/intel/ds.c @@ -1617,10 +1617,8 @@ static void setup_pebs_fixed_sample_data(struct perf_event *event, * previous PMI context or an (I)RET happened between the record and * PMI. */ - if (sample_type & PERF_SAMPLE_CALLCHAIN) { - data->callchain = perf_callchain(event, iregs); - data->sample_flags |= PERF_SAMPLE_CALLCHAIN; - } + if (sample_type & PERF_SAMPLE_CALLCHAIN) + perf_sample_save_callchain(data, event, iregs); /* * We use the interrupt regs as a base because the PEBS record does not @@ -1795,10 +1793,8 @@ static void setup_pebs_adaptive_sample_data(struct perf_event *event, * previous PMI context or an (I)RET happened between the record and * PMI. */ - if (sample_type & PERF_SAMPLE_CALLCHAIN) { - data->callchain = perf_callchain(event, iregs); - data->sample_flags |= PERF_SAMPLE_CALLCHAIN; - } + if (sample_type & PERF_SAMPLE_CALLCHAIN) + perf_sample_save_callchain(data, event, iregs); *regs = *iregs; /* The ip in basic is EventingIP */ diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h index 16b980014449..a9419608402b 100644 --- a/include/linux/perf_event.h +++ b/include/linux/perf_event.h @@ -1095,6 +1095,8 @@ int perf_event_read_local(struct perf_event *event, u64 *value, extern u64 perf_event_read_value(struct perf_event *event, u64 *enabled, u64 *running); +extern struct perf_callchain_entry *perf_callchain(struct perf_event *event, struct pt_regs *regs); + struct perf_sample_data { /* @@ -1167,6 +1169,19 @@ static inline void perf_sample_data_init(struct perf_sample_data *data, } } +static inline void perf_sample_save_callchain(struct perf_sample_data *data, + struct perf_event *event, + struct pt_regs *regs) +{ + int size = 1; + + data->callchain = perf_callchain(event, regs); + size += data->callchain->nr; + + data->dyn_size += size * sizeof(u64); + data->sample_flags |= PERF_SAMPLE_CALLCHAIN; +} + /* * Clear all bitfields in the perf_branch_entry. * The to and from fields are not cleared because they are @@ -1408,7 +1423,6 @@ extern void perf_callchain_kernel(struct perf_callchain_entry_ctx *entry, struct extern struct perf_callchain_entry * get_perf_callchain(struct pt_regs *regs, u32 init_nr, bool kernel, bool user, u32 max_stack, bool crosstask, bool add_mark); -extern struct perf_callchain_entry *perf_callchain(struct perf_event *event, struct pt_regs *regs); extern int get_callchain_buffers(int max_stack); extern void put_callchain_buffers(void); extern struct perf_callchain_entry *get_callchain_entry(int *rctx); diff --git a/kernel/events/core.c b/kernel/events/core.c index 8c8de26f04ab..0fba98b9cd65 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -7585,16 +7585,8 @@ void perf_prepare_sample(struct perf_event_header *header, if (sample_type & (PERF_SAMPLE_IP | PERF_SAMPLE_CODE_PAGE_SIZE)) data->ip = perf_instruction_pointer(regs); - if (sample_type & PERF_SAMPLE_CALLCHAIN) { - int size = 1; - - if (filtered_sample_type & PERF_SAMPLE_CALLCHAIN) - data->callchain = perf_callchain(event, regs); - - size += data->callchain->nr; - - data->dyn_size += size * sizeof(u64); - } + if (filtered_sample_type & PERF_SAMPLE_CALLCHAIN) + perf_sample_save_callchain(data, event, regs); if (sample_type & PERF_SAMPLE_RAW) { struct perf_raw_record *raw = data->raw; From patchwork Wed Jan 18 06:05:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Namhyung Kim X-Patchwork-Id: 44990 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2171656wrn; Tue, 17 Jan 2023 22:23:56 -0800 (PST) X-Google-Smtp-Source: AMrXdXt9Hy01YQ+tuq24oZodrRLIROhItsovQzD4O7Pt9umMopjRXfvVDPvjbIeljkWgxifils+I X-Received: by 2002:a62:1b8a:0:b0:56c:318a:f83b with SMTP id b132-20020a621b8a000000b0056c318af83bmr4599027pfb.13.1674023036154; Tue, 17 Jan 2023 22:23:56 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674023036; cv=none; d=google.com; s=arc-20160816; b=ZyGyBHPmClCXDsurtQp6bYSU/Z0Lnn/vUabvbfePWx7SW5so4ni6BdV51JGddRczqO 4vcNr+A5gyoPT/f+AfzTDuNY9voQ4JqfaD5qpurQG17mglZskUPCd9d3QpQGKATClWcY ypgi4LEDZhPECyds5WO2AqjeKHfYOiwKeyZ49L2g7xQVl4duS1r9W3oTMUZ5ChRjfb9l zUSSZTcS8swZQN1KTfG8Qmu7ixcoCxCVhVkPJPRj0LSjwb2JvwkCSsSUkQX5np42fizc WryY4oSkW27283kjTUwqlSG63OXJixZcz89j9WYnGCVkaWmBpoPcyWQC9B40JP9Occy2 RsZQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=1UMOA0c+N7OAsEuvPNaOaIppjJrZZvJ4XmiXNPGO7jM=; b=t54d/g3DuNPX75Bmi/NDR6RG7mKir06z1VjqsCgQRDLDduYGcXcwxF6wZ1+OUKiohc 5OJQOv+Z5DkVy50Kpy7tozQzMAli8jM8l3jmwwDRtpEeyRtzOmE/svryJX/XLolLx5eQ Gax1D5Ds5rARp8X/1ipJ0LMupSGP56OBPo6UcifGawUL4aLPsP/W0nQskqCq6+WM4Ttp q2McIRhuB8RooNYpGgmG8lbqKqZ+60lbkUnAW3hzdBsMfc8Uqysd6JuOVmG+ljCgCICq bgaMyMckEvWT2ssgfgCoj4FSWeOmd4kYNNypqp4uXtOTQF3/2Tvb3LY+F+B+ZvOmXePV OX8g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=nRTb7lNQ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id n184-20020a6227c1000000b005634d0b379dsi32365969pfn.115.2023.01.17.22.23.43; Tue, 17 Jan 2023 22:23:56 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=nRTb7lNQ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229473AbjARGWd (ORCPT + 99 others); Wed, 18 Jan 2023 01:22:33 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54026 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229718AbjARGTs (ORCPT ); Wed, 18 Jan 2023 01:19:48 -0500 Received: from mail-pg1-x52c.google.com (mail-pg1-x52c.google.com [IPv6:2607:f8b0:4864:20::52c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7C94C1E284; Tue, 17 Jan 2023 22:06:12 -0800 (PST) Received: by mail-pg1-x52c.google.com with SMTP id 7so23755898pga.1; Tue, 17 Jan 2023 22:06:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=1UMOA0c+N7OAsEuvPNaOaIppjJrZZvJ4XmiXNPGO7jM=; b=nRTb7lNQXs0UljzO9lnEQzwEFyUDs2cUuu24BBrSK7xU5xhX7gkvGADL+JeOfVtOC7 bIh9UfRtdt5x0p7jQ5buGbLMxxD4GH68XWLFI0pEfTkFFrBnIQmXCiffcKS8udc0Pd1t eoRJ3+NjwhuRuGPZo1KBHobDhbItVN1Z950lTWYGOZ3R8YEJCksgCmHT31TALo18DEmp 2+xlckgBjhhKib0P9MNtdZhYONTXiuL1OwwZZHMjda84Ftk8HHJpHXic5ob6JMuLrSRY yHXI+9wB/K1UC6n/zbwENCNGT5l6reo1OGB5/TT/TxEFYoz9bZJfTx/FHjiM6Er9Akyb V0BQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=1UMOA0c+N7OAsEuvPNaOaIppjJrZZvJ4XmiXNPGO7jM=; b=k6HkI6AnKJIUeALw0c1tT+EQmAknvPIArmqQSHn2EJQtKtyzf0ueD8cmNQZ/pwPDCf /n97P5nXuwtUH7hoe/8FE+2xA37lxgZLu2OF8AUmKgoV+vMhgRYGoimvN/End/NUJr2y tbWQPYeAe8F/zA/ycmPd25zarlf5j8sFHxDnWyh63FDHhpRlpwdIKoZ4qvQifTmqzORD M0TxHUtJidQQRMLGki0hgXRhJJUIn8ydmOW7XEn+5X8Avua48WDXueVI/ZH1HHeWocuZ YNK0iqZsQoXHD2AIt+qYhZY48Kb1Trha9yIPDB3w9eWF6M8HL+cgalbwjW3/q1CR3svc lYcQ== X-Gm-Message-State: AFqh2krK8Sb7viN110NW0vGxvU3DbpMEiFbiuzcWjaZO4e9Qo/TgcZIi plKj5tVHLgqcVhWk/+2Qew0= X-Received: by 2002:aa7:9607:0:b0:580:9e9e:c442 with SMTP id q7-20020aa79607000000b005809e9ec442mr5956540pfg.24.1674021964985; Tue, 17 Jan 2023 22:06:04 -0800 (PST) Received: from balhae.hsd1.ca.comcast.net ([2601:647:6780:ff0:3749:9eb3:dfb5:f449]) by smtp.gmail.com with ESMTPSA id b126-20020a62cf84000000b00574e84ed847sm10990180pfg.24.2023.01.17.22.06.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Jan 2023 22:06:04 -0800 (PST) Sender: Namhyung Kim From: Namhyung Kim To: Peter Zijlstra , Ingo Molnar Cc: Mark Rutland , Alexander Shishkin , Arnaldo Carvalho de Melo , Jiri Olsa , LKML , Song Liu , bpf@vger.kernel.org, linux-s390@vger.kernel.org, x86@kernel.org Subject: [PATCH 3/8] perf/core: Add perf_sample_save_raw_data() helper Date: Tue, 17 Jan 2023 22:05:54 -0800 Message-Id: <20230118060559.615653-4-namhyung@kernel.org> X-Mailer: git-send-email 2.39.0.314.g84b9a713c41-goog In-Reply-To: <20230118060559.615653-1-namhyung@kernel.org> References: <20230118060559.615653-1-namhyung@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1755340379267894844?= X-GMAIL-MSGID: =?utf-8?q?1755340379267894844?= When it saves the raw_data to the perf sample data, it needs to update the sample flags and the dynamic size. To make sure this, add the perf_sample_save_raw_data() helper and convert all call sites. Cc: linux-s390@vger.kernel.org Cc: x86@kernel.org Cc: bpf@vger.kernel.org Acked-by: Jiri Olsa Tested-by: Jiri Olsa Suggested-by: Peter Zijlstra Signed-off-by: Namhyung Kim --- arch/s390/kernel/perf_cpum_cf.c | 4 +--- arch/s390/kernel/perf_pai_crypto.c | 4 +--- arch/s390/kernel/perf_pai_ext.c | 4 +--- arch/x86/events/amd/ibs.c | 3 +-- include/linux/perf_event.h | 33 +++++++++++++++++++++++++----- kernel/events/core.c | 31 +++++----------------------- kernel/trace/bpf_trace.c | 6 ++---- 7 files changed, 39 insertions(+), 46 deletions(-) diff --git a/arch/s390/kernel/perf_cpum_cf.c b/arch/s390/kernel/perf_cpum_cf.c index f043a7ff220b..aa38649c7c27 100644 --- a/arch/s390/kernel/perf_cpum_cf.c +++ b/arch/s390/kernel/perf_cpum_cf.c @@ -662,9 +662,7 @@ static int cfdiag_push_sample(struct perf_event *event, if (event->attr.sample_type & PERF_SAMPLE_RAW) { raw.frag.size = cpuhw->usedss; raw.frag.data = cpuhw->stop; - raw.size = raw.frag.size; - data.raw = &raw; - data.sample_flags |= PERF_SAMPLE_RAW; + perf_sample_save_raw_data(&data, &raw); } overflow = perf_event_overflow(event, &data, ®s); diff --git a/arch/s390/kernel/perf_pai_crypto.c b/arch/s390/kernel/perf_pai_crypto.c index 985e243a2ed8..a7b339c4fd7c 100644 --- a/arch/s390/kernel/perf_pai_crypto.c +++ b/arch/s390/kernel/perf_pai_crypto.c @@ -362,9 +362,7 @@ static int paicrypt_push_sample(void) if (event->attr.sample_type & PERF_SAMPLE_RAW) { raw.frag.size = rawsize; raw.frag.data = cpump->save; - raw.size = raw.frag.size; - data.raw = &raw; - data.sample_flags |= PERF_SAMPLE_RAW; + perf_sample_save_raw_data(&data, &raw); } overflow = perf_event_overflow(event, &data, ®s); diff --git a/arch/s390/kernel/perf_pai_ext.c b/arch/s390/kernel/perf_pai_ext.c index 1138f57baae3..555597222bad 100644 --- a/arch/s390/kernel/perf_pai_ext.c +++ b/arch/s390/kernel/perf_pai_ext.c @@ -451,9 +451,7 @@ static int paiext_push_sample(void) if (event->attr.sample_type & PERF_SAMPLE_RAW) { raw.frag.size = rawsize; raw.frag.data = cpump->save; - raw.size = raw.frag.size; - data.raw = &raw; - data.sample_flags |= PERF_SAMPLE_RAW; + perf_sample_save_raw_data(&data, &raw); } overflow = perf_event_overflow(event, &data, ®s); diff --git a/arch/x86/events/amd/ibs.c b/arch/x86/events/amd/ibs.c index 417c80bd3274..64582954b5f6 100644 --- a/arch/x86/events/amd/ibs.c +++ b/arch/x86/events/amd/ibs.c @@ -1110,8 +1110,7 @@ static int perf_ibs_handle_irq(struct perf_ibs *perf_ibs, struct pt_regs *iregs) .data = ibs_data.data, }, }; - data.raw = &raw; - data.sample_flags |= PERF_SAMPLE_RAW; + perf_sample_save_raw_data(&data, &raw); } if (perf_ibs == &perf_ibs_op) diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h index a9419608402b..569dfac5887f 100644 --- a/include/linux/perf_event.h +++ b/include/linux/perf_event.h @@ -95,6 +95,11 @@ struct perf_raw_record { u32 size; }; +static __always_inline bool perf_raw_frag_last(const struct perf_raw_frag *frag) +{ + return frag->pad < sizeof(u64); +} + /* * branch stack layout: * nr: number of taken branches stored in entries[] @@ -1182,6 +1187,29 @@ static inline void perf_sample_save_callchain(struct perf_sample_data *data, data->sample_flags |= PERF_SAMPLE_CALLCHAIN; } +static inline void perf_sample_save_raw_data(struct perf_sample_data *data, + struct perf_raw_record *raw) +{ + struct perf_raw_frag *frag = &raw->frag; + u32 sum = 0; + int size; + + do { + sum += frag->size; + if (perf_raw_frag_last(frag)) + break; + frag = frag->next; + } while (1); + + size = round_up(sum + sizeof(u32), sizeof(u64)); + raw->size = size - sizeof(u32); + frag->pad = raw->size - sum; + + data->raw = raw; + data->dyn_size += size; + data->sample_flags |= PERF_SAMPLE_RAW; +} + /* * Clear all bitfields in the perf_branch_entry. * The to and from fields are not cleared because they are @@ -1690,11 +1718,6 @@ extern void perf_restore_debug_store(void); static inline void perf_restore_debug_store(void) { } #endif -static __always_inline bool perf_raw_frag_last(const struct perf_raw_frag *frag) -{ - return frag->pad < sizeof(u64); -} - #define perf_output_put(handle, x) perf_output_copy((handle), &(x), sizeof(x)) struct perf_pmu_events_attr { diff --git a/kernel/events/core.c b/kernel/events/core.c index 0fba98b9cd65..133894ae5e30 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -7588,30 +7588,10 @@ void perf_prepare_sample(struct perf_event_header *header, if (filtered_sample_type & PERF_SAMPLE_CALLCHAIN) perf_sample_save_callchain(data, event, regs); - if (sample_type & PERF_SAMPLE_RAW) { - struct perf_raw_record *raw = data->raw; - int size; - - if (raw && (data->sample_flags & PERF_SAMPLE_RAW)) { - struct perf_raw_frag *frag = &raw->frag; - u32 sum = 0; - - do { - sum += frag->size; - if (perf_raw_frag_last(frag)) - break; - frag = frag->next; - } while (1); - - size = round_up(sum + sizeof(u32), sizeof(u64)); - raw->size = size - sizeof(u32); - frag->pad = raw->size - sum; - } else { - size = sizeof(u64); - data->raw = NULL; - } - - data->dyn_size += size; + if (filtered_sample_type & PERF_SAMPLE_RAW) { + data->raw = NULL; + data->dyn_size += sizeof(u64); + data->sample_flags |= PERF_SAMPLE_RAW; } if (sample_type & PERF_SAMPLE_BRANCH_STACK) { @@ -10127,8 +10107,7 @@ void perf_tp_event(u16 event_type, u64 count, void *record, int entry_size, }; perf_sample_data_init(&data, 0, 0); - data.raw = &raw; - data.sample_flags |= PERF_SAMPLE_RAW; + perf_sample_save_raw_data(&data, &raw); perf_trace_buf_update(record, event_type); diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index 3bbd3f0c810c..ad37608afc35 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -687,8 +687,7 @@ BPF_CALL_5(bpf_perf_event_output, struct pt_regs *, regs, struct bpf_map *, map, } perf_sample_data_init(sd, 0, 0); - sd->raw = &raw; - sd->sample_flags |= PERF_SAMPLE_RAW; + perf_sample_save_raw_data(sd, &raw); err = __bpf_perf_event_output(regs, map, flags, sd); @@ -746,8 +745,7 @@ u64 bpf_event_output(struct bpf_map *map, u64 flags, void *meta, u64 meta_size, perf_fetch_caller_regs(regs); perf_sample_data_init(sd, 0, 0); - sd->raw = &raw; - sd->sample_flags |= PERF_SAMPLE_RAW; + perf_sample_save_raw_data(sd, &raw); ret = __bpf_perf_event_output(regs, map, flags, sd); out: From patchwork Wed Jan 18 06:05:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Namhyung Kim X-Patchwork-Id: 44988 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2171549wrn; Tue, 17 Jan 2023 22:23:38 -0800 (PST) X-Google-Smtp-Source: AMrXdXvLMffmZpmnQY7mzC9Yg6F5cFpPqCbaRFb9WJ5UdRlv0kQnpvmVgb0RIMBwZbkXhcK6Hf8R X-Received: by 2002:a17:90a:748c:b0:227:1110:8aa6 with SMTP id p12-20020a17090a748c00b0022711108aa6mr6320125pjk.4.1674023018653; Tue, 17 Jan 2023 22:23:38 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674023018; cv=none; d=google.com; s=arc-20160816; b=uwNKAKnQvljivd4AkW2+9oAeQeBvnApn+kaUfYb4GO8gshpQwHj0rnuzfprxltmNT1 siMsw4NuYyqLxZc/UwzG1XicQSp2l1Im03CxqbHN/8Asj/D/zxaMRUO2SfyE9kw4zUM4 wFJREA14rsuTd0vkok9nlXD4osYGQFQKPMj8sAapSvaXfxeazkLz5j3JDi9eTeALBE8o PU/zn6b+JHIx0uRsl6bECNMuZKbJW0LtNsJP/vfbgaab+FdnQncAYSDf3vXOWEQSUPrN CwQph61EHd4wfpAQcSPCT4Rh7dMc9/AQGcEitNahygQFR4mvSIobhA8Erds9uy6FYham gA5A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=kAqivUnaLGSiiUVP/9yUUwDL5c/NOTjH6M1nhFWggWg=; b=fVMAb/GAJjYsYjrs+gRyi2mU1hdbgOUfTBtvu9yBBCXzC3xq5yQQf4FS50wngaOCwP s9o5PcKUoSpTuJjDFc5ht1SELXn3ORS3YTsDErBmeBjLt9OJ8q4EOgZxmZjmbWLfKy85 tumxKfTh8Zn52kZjeEqJ6IJfTBSqj5btvMUDD5eSp8inRiAKBGrHLFCp40GOwjfqY2S2 9IL355aOx8cUOU3r1aPOa3+RNY6VHwOtqVrfOgD+VQOfNXEInNsIrhuAxwKvjFnP+jlo OnAYllcNO44dLr29NJHmtdYgD01qI7JOZiQv/caCgNtNrVBv5PER9ixnqRlnVeIBfjnJ 12uA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=ATO5EJCg; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id nk3-20020a17090b194300b0022630c5cbc8si1192748pjb.24.2023.01.17.22.23.23; Tue, 17 Jan 2023 22:23:38 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=ATO5EJCg; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229436AbjARGWP (ORCPT + 99 others); Wed, 18 Jan 2023 01:22:15 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53482 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229728AbjARGTT (ORCPT ); Wed, 18 Jan 2023 01:19:19 -0500 Received: from mail-pl1-x636.google.com (mail-pl1-x636.google.com [IPv6:2607:f8b0:4864:20::636]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7CF0E1E5D6; Tue, 17 Jan 2023 22:06:12 -0800 (PST) Received: by mail-pl1-x636.google.com with SMTP id d3so35849757plr.10; Tue, 17 Jan 2023 22:06:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=kAqivUnaLGSiiUVP/9yUUwDL5c/NOTjH6M1nhFWggWg=; b=ATO5EJCg/oM9Opfk75b9Fk6ftnjKeXw3k6lZVQhZ1oTDVpjgjxnHDrOni75BCwiHAi PkskBMxjj5cDFpulLJwmhq0gSt8UdeM/zaEyZ1VQZ2kDdZkBbTru6NCU5Y9JOY3EPXis YaupmNpHs+OQD7gEzvRQozc0VQTIRPEM6d8HdSSEuiSV6RWEITTvCvm5XZcWw3uSzTpL t9uhO5iBH5f4qHXWYAGhR2NRnEcEpp38sRhdD9M/HMUVSvczlJqD0SQ00J2wY6vce4PM qgksN48Yv/gCXuLPPOmt+vv5OtSablQeA/dm5orZr9KoQZDhXqh4zi8+YoaZNj2xne8s Rwzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=kAqivUnaLGSiiUVP/9yUUwDL5c/NOTjH6M1nhFWggWg=; b=Rjhmd7PlvaSg3gnKPjg32rdiU68IKBwTuAKlIrgz4Fa0eHpUdx0YxxXh6N9mLBESYB hnESnMoJxdgs4B/be0+vAFSYm39scd/8Zlz/4Vaiu6x8xMAUKs3oaX5yDWt7Vmt1oCrZ +AjKGG63OCZ7ZUWJChFBlbeP7xe0F6llbIRqzonTGRPZicBSnwB7V9O1awf6A/9fherF yul/EK84OxkuFKjCf4cl0ZmdsFgs+xwjwQ+eXyMCoET0j1mweHPccWgXA56WmZf93EAa ut72jHfYG8iuVRgv8O6GE4TZa5hHklKpXJACfNHVrtXy+mHqx52szV6kGIX62MQgdGBX 7TWA== X-Gm-Message-State: AFqh2kq1R71tVRAYROT5K6im51ui9zXHaL1IsjWlItJL1qTWlxrKQYqT N0iAwGzlbOXkDyF7JUh2cTwRccFqGS4= X-Received: by 2002:a05:6a20:d2cd:b0:b5:c751:78bb with SMTP id ir13-20020a056a20d2cd00b000b5c75178bbmr6346664pzb.6.1674021966593; Tue, 17 Jan 2023 22:06:06 -0800 (PST) Received: from balhae.hsd1.ca.comcast.net ([2601:647:6780:ff0:3749:9eb3:dfb5:f449]) by smtp.gmail.com with ESMTPSA id b126-20020a62cf84000000b00574e84ed847sm10990180pfg.24.2023.01.17.22.06.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Jan 2023 22:06:06 -0800 (PST) Sender: Namhyung Kim From: Namhyung Kim To: Peter Zijlstra , Ingo Molnar Cc: Mark Rutland , Alexander Shishkin , Arnaldo Carvalho de Melo , Jiri Olsa , LKML , Song Liu , bpf@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, x86@kernel.org Subject: [PATCH 4/8] perf/core: Add perf_sample_save_brstack() helper Date: Tue, 17 Jan 2023 22:05:55 -0800 Message-Id: <20230118060559.615653-5-namhyung@kernel.org> X-Mailer: git-send-email 2.39.0.314.g84b9a713c41-goog In-Reply-To: <20230118060559.615653-1-namhyung@kernel.org> References: <20230118060559.615653-1-namhyung@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1755340360743805988?= X-GMAIL-MSGID: =?utf-8?q?1755340360743805988?= When it saves the branch stack to the perf sample data, it needs to update the sample flags and the dynamic size. To make sure this, add the perf_sample_save_brstack() helper and convert all call sites. Cc: linuxppc-dev@lists.ozlabs.org Cc: x86@kernel.org Suggested-by: Peter Zijlstra Acked-by: Jiri Olsa Tested-by: Jiri Olsa Signed-off-by: Namhyung Kim Acked-by: Athira Rajeev --- arch/powerpc/perf/core-book3s.c | 3 +- arch/x86/events/amd/core.c | 6 +-- arch/x86/events/intel/core.c | 6 +-- arch/x86/events/intel/ds.c | 9 ++--- include/linux/perf_event.h | 66 ++++++++++++++++++++------------- kernel/events/core.c | 16 +++----- 6 files changed, 53 insertions(+), 53 deletions(-) diff --git a/arch/powerpc/perf/core-book3s.c b/arch/powerpc/perf/core-book3s.c index bf318dd9b709..8c1f7def596e 100644 --- a/arch/powerpc/perf/core-book3s.c +++ b/arch/powerpc/perf/core-book3s.c @@ -2313,8 +2313,7 @@ static void record_and_restart(struct perf_event *event, unsigned long val, struct cpu_hw_events *cpuhw; cpuhw = this_cpu_ptr(&cpu_hw_events); power_pmu_bhrb_read(event, cpuhw); - data.br_stack = &cpuhw->bhrb_stack; - data.sample_flags |= PERF_SAMPLE_BRANCH_STACK; + perf_sample_save_brstack(&data, event, &cpuhw->bhrb_stack); } if (event->attr.sample_type & PERF_SAMPLE_DATA_SRC && diff --git a/arch/x86/events/amd/core.c b/arch/x86/events/amd/core.c index d6f3703e4119..463f3eb8bbd7 100644 --- a/arch/x86/events/amd/core.c +++ b/arch/x86/events/amd/core.c @@ -928,10 +928,8 @@ static int amd_pmu_v2_handle_irq(struct pt_regs *regs) if (!x86_perf_event_set_period(event)) continue; - if (has_branch_stack(event)) { - data.br_stack = &cpuc->lbr_stack; - data.sample_flags |= PERF_SAMPLE_BRANCH_STACK; - } + if (has_branch_stack(event)) + perf_sample_save_brstack(&data, event, &cpuc->lbr_stack); if (perf_event_overflow(event, &data, regs)) x86_pmu_stop(event, 0); diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c index 29d2d0411caf..14f0a746257d 100644 --- a/arch/x86/events/intel/core.c +++ b/arch/x86/events/intel/core.c @@ -3036,10 +3036,8 @@ static int handle_pmi_common(struct pt_regs *regs, u64 status) perf_sample_data_init(&data, 0, event->hw.last_period); - if (has_branch_stack(event)) { - data.br_stack = &cpuc->lbr_stack; - data.sample_flags |= PERF_SAMPLE_BRANCH_STACK; - } + if (has_branch_stack(event)) + perf_sample_save_brstack(&data, event, &cpuc->lbr_stack); if (perf_event_overflow(event, &data, regs)) x86_pmu_stop(event, 0); diff --git a/arch/x86/events/intel/ds.c b/arch/x86/events/intel/ds.c index 158cf845fc80..07c8a2cdc3ee 100644 --- a/arch/x86/events/intel/ds.c +++ b/arch/x86/events/intel/ds.c @@ -1720,10 +1720,8 @@ static void setup_pebs_fixed_sample_data(struct perf_event *event, data->sample_flags |= PERF_SAMPLE_TIME; } - if (has_branch_stack(event)) { - data->br_stack = &cpuc->lbr_stack; - data->sample_flags |= PERF_SAMPLE_BRANCH_STACK; - } + if (has_branch_stack(event)) + perf_sample_save_brstack(data, event, &cpuc->lbr_stack); } static void adaptive_pebs_save_regs(struct pt_regs *regs, @@ -1883,8 +1881,7 @@ static void setup_pebs_adaptive_sample_data(struct perf_event *event, if (has_branch_stack(event)) { intel_pmu_store_pebs_lbrs(lbr); - data->br_stack = &cpuc->lbr_stack; - data->sample_flags |= PERF_SAMPLE_BRANCH_STACK; + perf_sample_save_brstack(data, event, &cpuc->lbr_stack); } } diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h index 569dfac5887f..7db0e9cc2682 100644 --- a/include/linux/perf_event.h +++ b/include/linux/perf_event.h @@ -1102,6 +1102,31 @@ extern u64 perf_event_read_value(struct perf_event *event, extern struct perf_callchain_entry *perf_callchain(struct perf_event *event, struct pt_regs *regs); +static inline bool branch_sample_no_flags(const struct perf_event *event) +{ + return event->attr.branch_sample_type & PERF_SAMPLE_BRANCH_NO_FLAGS; +} + +static inline bool branch_sample_no_cycles(const struct perf_event *event) +{ + return event->attr.branch_sample_type & PERF_SAMPLE_BRANCH_NO_CYCLES; +} + +static inline bool branch_sample_type(const struct perf_event *event) +{ + return event->attr.branch_sample_type & PERF_SAMPLE_BRANCH_TYPE_SAVE; +} + +static inline bool branch_sample_hw_index(const struct perf_event *event) +{ + return event->attr.branch_sample_type & PERF_SAMPLE_BRANCH_HW_INDEX; +} + +static inline bool branch_sample_priv(const struct perf_event *event) +{ + return event->attr.branch_sample_type & PERF_SAMPLE_BRANCH_PRIV_SAVE; +} + struct perf_sample_data { /* @@ -1210,6 +1235,21 @@ static inline void perf_sample_save_raw_data(struct perf_sample_data *data, data->sample_flags |= PERF_SAMPLE_RAW; } +static inline void perf_sample_save_brstack(struct perf_sample_data *data, + struct perf_event *event, + struct perf_branch_stack *brs) +{ + int size = sizeof(u64); /* nr */ + + if (branch_sample_hw_index(event)) + size += sizeof(u64); + size += brs->nr * sizeof(struct perf_branch_entry); + + data->br_stack = brs; + data->dyn_size += size; + data->sample_flags |= PERF_SAMPLE_BRANCH_STACK; +} + /* * Clear all bitfields in the perf_branch_entry. * The to and from fields are not cleared because they are @@ -1827,30 +1867,4 @@ static inline void perf_lopwr_cb(bool mode) } #endif -#ifdef CONFIG_PERF_EVENTS -static inline bool branch_sample_no_flags(const struct perf_event *event) -{ - return event->attr.branch_sample_type & PERF_SAMPLE_BRANCH_NO_FLAGS; -} - -static inline bool branch_sample_no_cycles(const struct perf_event *event) -{ - return event->attr.branch_sample_type & PERF_SAMPLE_BRANCH_NO_CYCLES; -} - -static inline bool branch_sample_type(const struct perf_event *event) -{ - return event->attr.branch_sample_type & PERF_SAMPLE_BRANCH_TYPE_SAVE; -} - -static inline bool branch_sample_hw_index(const struct perf_event *event) -{ - return event->attr.branch_sample_type & PERF_SAMPLE_BRANCH_HW_INDEX; -} - -static inline bool branch_sample_priv(const struct perf_event *event) -{ - return event->attr.branch_sample_type & PERF_SAMPLE_BRANCH_PRIV_SAVE; -} -#endif /* CONFIG_PERF_EVENTS */ #endif /* _LINUX_PERF_EVENT_H */ diff --git a/kernel/events/core.c b/kernel/events/core.c index 133894ae5e30..0218b6ffaf36 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -7317,7 +7317,7 @@ void perf_output_sample(struct perf_output_handle *handle, } if (sample_type & PERF_SAMPLE_BRANCH_STACK) { - if (data->sample_flags & PERF_SAMPLE_BRANCH_STACK) { + if (data->br_stack) { size_t size; size = data->br_stack->nr @@ -7594,16 +7594,10 @@ void perf_prepare_sample(struct perf_event_header *header, data->sample_flags |= PERF_SAMPLE_RAW; } - if (sample_type & PERF_SAMPLE_BRANCH_STACK) { - int size = sizeof(u64); /* nr */ - if (data->sample_flags & PERF_SAMPLE_BRANCH_STACK) { - if (branch_sample_hw_index(event)) - size += sizeof(u64); - - size += data->br_stack->nr - * sizeof(struct perf_branch_entry); - } - data->dyn_size += size; + if (filtered_sample_type & PERF_SAMPLE_BRANCH_STACK) { + data->br_stack = NULL; + data->dyn_size += sizeof(u64); + data->sample_flags |= PERF_SAMPLE_BRANCH_STACK; } if (sample_type & (PERF_SAMPLE_REGS_USER | PERF_SAMPLE_STACK_USER)) From patchwork Wed Jan 18 06:05:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Namhyung Kim X-Patchwork-Id: 44991 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2171921wrn; Tue, 17 Jan 2023 22:24:46 -0800 (PST) X-Google-Smtp-Source: AMrXdXvp2MBse5s4LSd4zJe//Ieig8Pr/xMomYfMkntkkRRxVhRnzPXmUrsfCw1kc1keM9SEFVg4 X-Received: by 2002:a17:902:eb86:b0:194:7696:c024 with SMTP id q6-20020a170902eb8600b001947696c024mr6562768plg.5.1674023085936; Tue, 17 Jan 2023 22:24:45 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674023085; cv=none; d=google.com; s=arc-20160816; b=cZ3rwXm4WVG/ClSinJpnLivQeZJ/B8IdmBlambgqwTfKvzEc6RrKnZy+T+Itt4AIKQ 4I9c6KnecpyUJC/w54d3jmCuOUZPtyjWYYJs2UkPflf/CZL9aleQDJPCRwW0FGcjZcLg WwdZbtegeEBtyDqA11wwk433chgi4yXrCgjC6VYB2+SOs03mPECTRnbZyQtLp4I0dOea nDU/bEMEYsyszWkSCE81bMkd8fNV3gy4KyXsLDLzjBaU6Qr55eyeCjWquWEeShBafius WVl4gJQFyBl5VlOfDyPiGr3CgdfjM/WhpR91w5Cp655nbFQjJ0+vAd3hJdEoFaJEaB0J Lp+Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=u+qpR2MM6KIrediMBwRe+lRsg42bPXrIFANMIti0WTY=; b=WQZTJ5lLRb8X9DlTBmtouXSaSvxYHBubBvEGSSPaGavK03kE8kEctprZijozfgKCOA b2cuWDIcl71rE4CHELYwz87Qd0wfg9m7qL2Cc09jF3Wy4jmDV1WAnSyfZCCRBsB4Z0IB mGssHZNGn4q3KlZ2daTtm3J66FdlfEbvXxvhQ0jeyoof3f0n1OuYdV3nQgndnQz8vtC5 G184KxibLMnSlZ4lawbvN7B4Jz1tE28JH1g0m6XtWHa2Ktr65SIn6OuJuVJ39M8EyKby R0BYCrBkOviRgccbHmPjEmeXoHmw1KiSEbOC+Y8pHIeeNSkmz9TUTJyXkpTwdBp7H/hb zfTA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=AHn0AGCh; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id g10-20020a170902d1ca00b00186bb39ff55si31040389plb.427.2023.01.17.22.24.26; Tue, 17 Jan 2023 22:24:45 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=AHn0AGCh; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229558AbjARGXS (ORCPT + 99 others); Wed, 18 Jan 2023 01:23:18 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53490 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229729AbjARGTT (ORCPT ); Wed, 18 Jan 2023 01:19:19 -0500 Received: from mail-pf1-x435.google.com (mail-pf1-x435.google.com [IPv6:2607:f8b0:4864:20::435]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7CAEB1E2A7; Tue, 17 Jan 2023 22:06:12 -0800 (PST) Received: by mail-pf1-x435.google.com with SMTP id 20so19939506pfu.13; Tue, 17 Jan 2023 22:06:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=u+qpR2MM6KIrediMBwRe+lRsg42bPXrIFANMIti0WTY=; b=AHn0AGChsQpTHgkbeZqtS/cfWyiMw8UpNXfgmWSmeJuU6tUlAz5UuaQC0cE49NcCo1 ZYBSkvYlKwaHXdqqPYIUZsZPG2IlJv3VswZfphc1VgaCFXsV1LPfEnqmWCTm2ziAdA3Y 4sfmgpSyPHYauwWQ7gvL8rpb58O8+SHmtaHa3FpViqNtZUtHcGw63UdKqlKPbSfLVyN5 rKBeMf5KszGcAlfCZ50W++c2536cRsnZEtNxmmw50j/SBjC5yl7XOKcumoaJLnOYd1Eo TDhpv3/GTjIeoVizWwTTqo7iUiU92Lmid101ChaTqQMqdlla3u/B/ZGqmPnC9wKZklVb x7CQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=u+qpR2MM6KIrediMBwRe+lRsg42bPXrIFANMIti0WTY=; b=FElfg+x+YPg4WJVlI+j1i0HFsJJ6p46bKBewGeh6YpIxYeO+dT9d1B4rWcnlp2Z6iU y2Z/g9xBjrrPxZAXsyCoaZ1WPa3j2RTBDcLnyavnHjRU8DqeQoS/kaUo/yc8xk/lU/gA XiAg14pdYqL+XYP729McTAyh2BicbRj6FIYT0u8kb+6HAInbKzVRvlWdBve5ej0CRwOu r3HUWmTOrmkX3UABKoaIbZn6x/s/FmreD10q53j3/jbCOHho+HSDcNaQ2Z7EEmkwWUPl ZojtmpDWKoTXEivuDVzJOgx9BZFuuK/l792hVCUlu5G9vbYps9MPdfpxuEG3MjJPosKM QmSA== X-Gm-Message-State: AFqh2krXi6ggRNg61zHZbgPBWT18jlDmXB8CsFfRABKVp3szwJew280S jQHHORjaB4Jlieb81fCi5ec= X-Received: by 2002:a05:6a00:4c0b:b0:58d:8d8f:cc3d with SMTP id ea11-20020a056a004c0b00b0058d8d8fcc3dmr6930488pfb.23.1674021967889; Tue, 17 Jan 2023 22:06:07 -0800 (PST) Received: from balhae.hsd1.ca.comcast.net ([2601:647:6780:ff0:3749:9eb3:dfb5:f449]) by smtp.gmail.com with ESMTPSA id b126-20020a62cf84000000b00574e84ed847sm10990180pfg.24.2023.01.17.22.06.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Jan 2023 22:06:07 -0800 (PST) Sender: Namhyung Kim From: Namhyung Kim To: Peter Zijlstra , Ingo Molnar Cc: Mark Rutland , Alexander Shishkin , Arnaldo Carvalho de Melo , Jiri Olsa , LKML , Song Liu , bpf@vger.kernel.org Subject: [PATCH 5/8] perf/core: Set data->sample_flags in perf_prepare_sample() Date: Tue, 17 Jan 2023 22:05:56 -0800 Message-Id: <20230118060559.615653-6-namhyung@kernel.org> X-Mailer: git-send-email 2.39.0.314.g84b9a713c41-goog In-Reply-To: <20230118060559.615653-1-namhyung@kernel.org> References: <20230118060559.615653-1-namhyung@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1755340431153700099?= X-GMAIL-MSGID: =?utf-8?q?1755340431153700099?= The perf_prepare_sample() sets the perf_sample_data according to the attr->sample_type before copying it to the ring buffer. But BPF also wants to access the sample data so it needs to prepare the sample even before the regular path. That means the perf_prepare_sample() can be called more than once. Set the data->sample_flags consistently so that it can indicate which fields are set already and skip them if sets. Also update the filtered_sample_type to have the dependent flags to reduce the number of branches. Suggested-by: Peter Zijlstra Acked-by: Jiri Olsa Tested-by: Jiri Olsa Signed-off-by: Namhyung Kim --- kernel/events/core.c | 85 +++++++++++++++++++++++++++++++++----------- 1 file changed, 65 insertions(+), 20 deletions(-) diff --git a/kernel/events/core.c b/kernel/events/core.c index 0218b6ffaf36..1384137a90f7 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -7053,12 +7053,21 @@ static void perf_aux_sample_output(struct perf_event *event, ring_buffer_put(rb); } +/* + * A set of common sample data types saved even for non-sample records + * when event->attr.sample_id_all is set. + */ +#define PERF_SAMPLE_ID_ALL (PERF_SAMPLE_TID | PERF_SAMPLE_TIME | \ + PERF_SAMPLE_ID | PERF_SAMPLE_STREAM_ID | \ + PERF_SAMPLE_CPU | PERF_SAMPLE_IDENTIFIER) + static void __perf_event_header__init_id(struct perf_event_header *header, struct perf_sample_data *data, struct perf_event *event, u64 sample_type) { data->type = event->attr.sample_type; + data->sample_flags |= data->type & PERF_SAMPLE_ID_ALL; header->size += event->id_header_size; if (sample_type & PERF_SAMPLE_TID) { @@ -7561,6 +7570,11 @@ perf_callchain(struct perf_event *event, struct pt_regs *regs) return callchain ?: &__empty_callchain; } +static __always_inline u64 __cond_set(u64 flags, u64 s, u64 d) +{ + return d * !!(flags & s); +} + void perf_prepare_sample(struct perf_event_header *header, struct perf_sample_data *data, struct perf_event *event, @@ -7576,14 +7590,24 @@ void perf_prepare_sample(struct perf_event_header *header, header->misc |= perf_misc_flags(regs); /* - * Clear the sample flags that have already been done by the - * PMU driver. + * Add the sample flags that are dependent to others. And clear the + * sample flags that have already been done by the PMU driver. */ - filtered_sample_type = sample_type & ~data->sample_flags; + filtered_sample_type = sample_type; + filtered_sample_type |= __cond_set(sample_type, PERF_SAMPLE_CODE_PAGE_SIZE, + PERF_SAMPLE_IP); + filtered_sample_type |= __cond_set(sample_type, PERF_SAMPLE_DATA_PAGE_SIZE | + PERF_SAMPLE_PHYS_ADDR, PERF_SAMPLE_ADDR); + filtered_sample_type |= __cond_set(sample_type, PERF_SAMPLE_STACK_USER, + PERF_SAMPLE_REGS_USER); + filtered_sample_type &= ~data->sample_flags; + __perf_event_header__init_id(header, data, event, filtered_sample_type); - if (sample_type & (PERF_SAMPLE_IP | PERF_SAMPLE_CODE_PAGE_SIZE)) + if (filtered_sample_type & PERF_SAMPLE_IP) { data->ip = perf_instruction_pointer(regs); + data->sample_flags |= PERF_SAMPLE_IP; + } if (filtered_sample_type & PERF_SAMPLE_CALLCHAIN) perf_sample_save_callchain(data, event, regs); @@ -7600,10 +7624,15 @@ void perf_prepare_sample(struct perf_event_header *header, data->sample_flags |= PERF_SAMPLE_BRANCH_STACK; } - if (sample_type & (PERF_SAMPLE_REGS_USER | PERF_SAMPLE_STACK_USER)) + if (filtered_sample_type & PERF_SAMPLE_REGS_USER) perf_sample_regs_user(&data->regs_user, regs); - if (sample_type & PERF_SAMPLE_REGS_USER) { + /* + * It cannot use the filtered_sample_type here as REGS_USER can be set + * by STACK_USER (using __cond_set() above) and we don't want to update + * the dyn_size if it's not requested by users. + */ + if ((sample_type & ~data->sample_flags) & PERF_SAMPLE_REGS_USER) { /* regs dump ABI info */ int size = sizeof(u64); @@ -7613,9 +7642,10 @@ void perf_prepare_sample(struct perf_event_header *header, } data->dyn_size += size; + data->sample_flags |= PERF_SAMPLE_REGS_USER; } - if (sample_type & PERF_SAMPLE_STACK_USER) { + if (filtered_sample_type & PERF_SAMPLE_STACK_USER) { /* * Either we need PERF_SAMPLE_STACK_USER bit to be always * processed as the last one or have additional check added @@ -7638,23 +7668,30 @@ void perf_prepare_sample(struct perf_event_header *header, data->stack_user_size = stack_size; data->dyn_size += size; + data->sample_flags |= PERF_SAMPLE_STACK_USER; } - if (filtered_sample_type & PERF_SAMPLE_WEIGHT_TYPE) + if (filtered_sample_type & PERF_SAMPLE_WEIGHT_TYPE) { data->weight.full = 0; + data->sample_flags |= PERF_SAMPLE_WEIGHT_TYPE; + } - if (filtered_sample_type & PERF_SAMPLE_DATA_SRC) + if (filtered_sample_type & PERF_SAMPLE_DATA_SRC) { data->data_src.val = PERF_MEM_NA; + data->sample_flags |= PERF_SAMPLE_DATA_SRC; + } - if (filtered_sample_type & PERF_SAMPLE_TRANSACTION) + if (filtered_sample_type & PERF_SAMPLE_TRANSACTION) { data->txn = 0; + data->sample_flags |= PERF_SAMPLE_TRANSACTION; + } - if (sample_type & (PERF_SAMPLE_ADDR | PERF_SAMPLE_PHYS_ADDR | PERF_SAMPLE_DATA_PAGE_SIZE)) { - if (filtered_sample_type & PERF_SAMPLE_ADDR) - data->addr = 0; + if (filtered_sample_type & PERF_SAMPLE_ADDR) { + data->addr = 0; + data->sample_flags |= PERF_SAMPLE_ADDR; } - if (sample_type & PERF_SAMPLE_REGS_INTR) { + if (filtered_sample_type & PERF_SAMPLE_REGS_INTR) { /* regs dump ABI info */ int size = sizeof(u64); @@ -7667,19 +7704,22 @@ void perf_prepare_sample(struct perf_event_header *header, } data->dyn_size += size; + data->sample_flags |= PERF_SAMPLE_REGS_INTR; } - if (sample_type & PERF_SAMPLE_PHYS_ADDR && - filtered_sample_type & PERF_SAMPLE_PHYS_ADDR) + if (filtered_sample_type & PERF_SAMPLE_PHYS_ADDR) { data->phys_addr = perf_virt_to_phys(data->addr); + data->sample_flags |= PERF_SAMPLE_PHYS_ADDR; + } #ifdef CONFIG_CGROUP_PERF - if (sample_type & PERF_SAMPLE_CGROUP) { + if (filtered_sample_type & PERF_SAMPLE_CGROUP) { struct cgroup *cgrp; /* protected by RCU */ cgrp = task_css_check(current, perf_event_cgrp_id, 1)->cgroup; data->cgroup = cgroup_id(cgrp); + data->sample_flags |= PERF_SAMPLE_CGROUP; } #endif @@ -7688,13 +7728,17 @@ void perf_prepare_sample(struct perf_event_header *header, * require PERF_SAMPLE_ADDR, kernel implicitly retrieve the data->addr, * but the value will not dump to the userspace. */ - if (sample_type & PERF_SAMPLE_DATA_PAGE_SIZE) + if (filtered_sample_type & PERF_SAMPLE_DATA_PAGE_SIZE) { data->data_page_size = perf_get_page_size(data->addr); + data->sample_flags |= PERF_SAMPLE_DATA_PAGE_SIZE; + } - if (sample_type & PERF_SAMPLE_CODE_PAGE_SIZE) + if (filtered_sample_type & PERF_SAMPLE_CODE_PAGE_SIZE) { data->code_page_size = perf_get_page_size(data->ip); + data->sample_flags |= PERF_SAMPLE_CODE_PAGE_SIZE; + } - if (sample_type & PERF_SAMPLE_AUX) { + if (filtered_sample_type & PERF_SAMPLE_AUX) { u64 size; header->size += sizeof(u64); /* size */ @@ -7712,6 +7756,7 @@ void perf_prepare_sample(struct perf_event_header *header, WARN_ON_ONCE(size + header->size > U16_MAX); data->dyn_size += size + sizeof(u64); /* size above */ + data->sample_flags |= PERF_SAMPLE_AUX; } header->size += data->dyn_size; From patchwork Wed Jan 18 06:05:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Namhyung Kim X-Patchwork-Id: 44987 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2171333wrn; Tue, 17 Jan 2023 22:22:58 -0800 (PST) X-Google-Smtp-Source: AMrXdXuVYP2F6wIwDoMV/LpQIH9lbzCo5tT6Q+EwE61xCc3H58HF0q2qSGqTN6Gkcp3cf5DvvUrL X-Received: by 2002:a17:90b:698:b0:226:ad9f:92c6 with SMTP id m24-20020a17090b069800b00226ad9f92c6mr27775969pjz.32.1674022978290; Tue, 17 Jan 2023 22:22:58 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674022978; cv=none; d=google.com; s=arc-20160816; b=WMGTbDbyBdgcuEY33/c9fbvfF2yoCGHAnMGA4xc2pzXLe16LFOOWaIIEBW2qRoY+ai ZqQKaep3Ruu4l7DK9mEG1X84wm6+Rw/TiFDLjl9uK8JbOzruE0XoKMHrvw+T3jNTfzTM t7XYHSWoBbMCc7hsg/jq7jB7vYMVQImq0wnIMPlosAmwUOsFD9TD3TgY7a9VBSuDFa10 p2QxDi1/Lmtgm3TYJ9H8o585dXLs4Nz92z97lqhs0UYXDVQxD+9Z+cJlMji1fxjh6Kd0 xUvCfFZYuj8sBHNgKP5sF9yQfgmHsTrUR66ATYf7OW2nM/RlNNJ2ymK480UTccUKjCFC NgLQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=G8Zh73XBwb5RMlMiLWpYbNBMevrlhNdfAXS0S1zywoE=; b=mDV/pJszbT0sRJK+98dM/1Pm7KGdWKdSmNpaDtPvN4eq3rHvG0qMnErzwzfwpI9+R4 fBAA/46EThR3Yi9C4kkorhIJx56XhnWt+mvluW9TlMLwWqQparLvseQM/xFA/5jQu1eY 6sN5vZogJ8MXgjEvFXe5xOzRpJvosTLdiOFLP+krUoIstlcZJxAxI6Yx2mOKAyCYOE+l 7EWAmUOnutgC09CvkkSsV7LG/o9jGeoSk/SK+zisGv2jGf52s0iWf2SdOj9UQzxaUcha AC6ciQamvs2pm0d/8ryDCvlrH5laykQU3NFAeTlBpSGzMcFuKn+XMkKApYkcz9c80Cz9 hy6g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=BtwVgukY; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id d8-20020a17090ab30800b0021a005518d0si1205107pjr.50.2023.01.17.22.22.43; Tue, 17 Jan 2023 22:22:58 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=BtwVgukY; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229682AbjARGVh (ORCPT + 99 others); Wed, 18 Jan 2023 01:21:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54060 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229654AbjARGSW (ORCPT ); Wed, 18 Jan 2023 01:18:22 -0500 Received: from mail-pj1-x102b.google.com (mail-pj1-x102b.google.com [IPv6:2607:f8b0:4864:20::102b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4DE8A1D921; Tue, 17 Jan 2023 22:06:21 -0800 (PST) Received: by mail-pj1-x102b.google.com with SMTP id d8so3576686pjc.3; Tue, 17 Jan 2023 22:06:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=G8Zh73XBwb5RMlMiLWpYbNBMevrlhNdfAXS0S1zywoE=; b=BtwVgukY8p2esH90r+fFnX1sw/xNf3PzZDpleVXeJjNvYBea8dbKTD5knI16ZPYgok oWn2fv24+g5U8shpZzCUvHcGQU/61rPmRCn9+RfJ85ha/rMs3/KosTyQVgV7NtrmyCz9 k0NtzMhxCIPQA9rL3f3cKv4MdNW7MzBGeEZf6HWSeogLiXicdsFcOly6Ywmq6L1TuMJG NgEl5ocR60OBqguMRShxpuyN+LS7xyEX+uO4TYUyfI8n92g67XLuUVs/F5eWkkkkilZB V4p+vpsyyHfWexwQcDIf13zJQKJT3g/x0gCQyEZzkJfC/9eLVI6H25KIajRv/tGPCvFH y38w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=G8Zh73XBwb5RMlMiLWpYbNBMevrlhNdfAXS0S1zywoE=; b=LiCoUyz5/UQtCXdByEOVEyWfTrecc+QWrDluUWQ12AFSt/JzDj5v1X1ChROOfRTbYx Ba5CDXcMR1g4aDdOy1pmmTrqPbzKiJgh+sXJQSXFgLwknloF05RtwRziaq8wDTnCWvim rymU+tCsg695RMyTVR4QdKgozee418FiLakAaA+a/XlQoChgq7gklRL4A+wLqURrS7n0 moWjkNtUKh7Y0wCLnQu/ID4BkS9OKGNob0oi2D9TRBcgTm6YHj+hacY2wKDR6wzPpD47 6mc1wmRVFlUJgRzA+ctHSK7sMjkRLzKgyfgnjHpoUPrqXUTVi9Foo93R19DHjDXN00+X fzPA== X-Gm-Message-State: AFqh2kqIzgMK/HP5IdJIda5uwa+KkN8gnL9xABDqhkCMrliPPHi2MYfM ES1PNz1RwpM2wNxhm/5vy1dMjREWO+A= X-Received: by 2002:a05:6a20:b925:b0:9d:efbf:8156 with SMTP id fe37-20020a056a20b92500b0009defbf8156mr29465355pzb.31.1674021969212; Tue, 17 Jan 2023 22:06:09 -0800 (PST) Received: from balhae.hsd1.ca.comcast.net ([2601:647:6780:ff0:3749:9eb3:dfb5:f449]) by smtp.gmail.com with ESMTPSA id b126-20020a62cf84000000b00574e84ed847sm10990180pfg.24.2023.01.17.22.06.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Jan 2023 22:06:08 -0800 (PST) Sender: Namhyung Kim From: Namhyung Kim To: Peter Zijlstra , Ingo Molnar Cc: Mark Rutland , Alexander Shishkin , Arnaldo Carvalho de Melo , Jiri Olsa , LKML , Song Liu , bpf@vger.kernel.org Subject: [PATCH 6/8] perf/core: Do not pass header for sample id init Date: Tue, 17 Jan 2023 22:05:57 -0800 Message-Id: <20230118060559.615653-7-namhyung@kernel.org> X-Mailer: git-send-email 2.39.0.314.g84b9a713c41-goog In-Reply-To: <20230118060559.615653-1-namhyung@kernel.org> References: <20230118060559.615653-1-namhyung@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1755340318359218201?= X-GMAIL-MSGID: =?utf-8?q?1755340318359218201?= The only thing it does for header in __perf_event_header__init_id() is to update the header size with event->id_header_size. We can do this outside and get rid of the argument for the later change. Acked-by: Jiri Olsa Acked-by: Song Liu Tested-by: Jiri Olsa Signed-off-by: Namhyung Kim --- kernel/events/core.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/kernel/events/core.c b/kernel/events/core.c index 1384137a90f7..9cc55122188f 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -7061,14 +7061,12 @@ static void perf_aux_sample_output(struct perf_event *event, PERF_SAMPLE_ID | PERF_SAMPLE_STREAM_ID | \ PERF_SAMPLE_CPU | PERF_SAMPLE_IDENTIFIER) -static void __perf_event_header__init_id(struct perf_event_header *header, - struct perf_sample_data *data, +static void __perf_event_header__init_id(struct perf_sample_data *data, struct perf_event *event, u64 sample_type) { data->type = event->attr.sample_type; data->sample_flags |= data->type & PERF_SAMPLE_ID_ALL; - header->size += event->id_header_size; if (sample_type & PERF_SAMPLE_TID) { /* namespace issues */ @@ -7095,8 +7093,10 @@ void perf_event_header__init_id(struct perf_event_header *header, struct perf_sample_data *data, struct perf_event *event) { - if (event->attr.sample_id_all) - __perf_event_header__init_id(header, data, event, event->attr.sample_type); + if (event->attr.sample_id_all) { + header->size += event->id_header_size; + __perf_event_header__init_id(data, event, event->attr.sample_type); + } } static void __perf_event__output_id_sample(struct perf_output_handle *handle, @@ -7584,7 +7584,7 @@ void perf_prepare_sample(struct perf_event_header *header, u64 filtered_sample_type; header->type = PERF_RECORD_SAMPLE; - header->size = sizeof(*header) + event->header_size; + header->size = sizeof(*header) + event->header_size + event->id_header_size; header->misc = 0; header->misc |= perf_misc_flags(regs); @@ -7602,7 +7602,7 @@ void perf_prepare_sample(struct perf_event_header *header, PERF_SAMPLE_REGS_USER); filtered_sample_type &= ~data->sample_flags; - __perf_event_header__init_id(header, data, event, filtered_sample_type); + __perf_event_header__init_id(data, event, filtered_sample_type); if (filtered_sample_type & PERF_SAMPLE_IP) { data->ip = perf_instruction_pointer(regs); From patchwork Wed Jan 18 06:05:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Namhyung Kim X-Patchwork-Id: 45046 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2193098wrn; Tue, 17 Jan 2023 23:30:31 -0800 (PST) X-Google-Smtp-Source: AMrXdXurwEt1rVA+qXJ/Lut/vTaoBvTFlFEGTTxSS+I2xA3H8QBeKorPyCIBq+1UYZp00zO9OimI X-Received: by 2002:a17:90a:ab15:b0:21a:1b9b:139e with SMTP id m21-20020a17090aab1500b0021a1b9b139emr6491310pjq.8.1674027031153; Tue, 17 Jan 2023 23:30:31 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674027031; cv=none; d=google.com; s=arc-20160816; b=f/UpayxWpTyGoD9F5pFZ6lQXw0LZFF0nnMAvIb8dmJ20ZHWN8KiFDKw5xOtyyJL6+q Q3Fm75OEcjD0Yt2UDGbD0pZq1BXhEFiD3KtqWubT662XUG4cy/BalY4LLXmqJf7Sgpi3 CESOR4ZQ1bFkGn8rr1/KOC3bLFr2x1vzDblDULd8CnEP0soQZyJylwBXUmN9dJszCxCp FTIEnM5tExxRR1T0jd4mKw2zMK1BgPM1CG2pfNxXdU2LmlbxQhCQHxieWHsjnpCmCaxP y6nB3MN5JHYyyblJTzvJRY79UOdMjEpIjdch1fHgn571snIU1jrqApd11zgTY5EGby1G zhxQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=yYsRe09JkVPoqaZ4OGb+wtxVDI81dN8UGiocCLjkwaM=; b=X68f227W57GY4paukyy1GeLpenbIkBVXiZBRDrPt2YWJJzzmuM07xOWdqhJSLP1Bm2 BIOWh9/OjTJ7Gxpcw0RMpHzpXuhE2Ap5QcWXWRyds2qqixCjiws0/Qt4hPxXdvUFlr+j t+2lLNocRFyQwXpulXLjQbSf2z/lIwCcsYNUe7M14PuBaChlRpc4WMhtoXhuV/4MkR/5 1akcvMitplC23BfbP61oFGFEOPa7MhIkgn4R7RWrtNfEgSjVWRxP8SSYelPaR3YL1NQ8 m2RXnsiKkVq3oelVpIBaJuMy94UsG6qvyqR/Ok8SO0h/EKtoPPYXgV8KiEQmGIzSqhbe rM7w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=UuNDWiWk; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id y23-20020a17090aca9700b00208606154f7si1353247pjt.117.2023.01.17.23.30.17; Tue, 17 Jan 2023 23:30:31 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=UuNDWiWk; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229800AbjARGry (ORCPT + 99 others); Wed, 18 Jan 2023 01:47:54 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57036 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229801AbjARGZy (ORCPT ); Wed, 18 Jan 2023 01:25:54 -0500 Received: from mail-yw1-x1131.google.com (mail-yw1-x1131.google.com [IPv6:2607:f8b0:4864:20::1131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 73C12552A8; Tue, 17 Jan 2023 22:14:27 -0800 (PST) Received: by mail-yw1-x1131.google.com with SMTP id 00721157ae682-4d13cb4bbffso334309407b3.3; Tue, 17 Jan 2023 22:14:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=yYsRe09JkVPoqaZ4OGb+wtxVDI81dN8UGiocCLjkwaM=; b=UuNDWiWkCSC/tViFrw0EsvSVoiLmSFAmRZo8dL5cwLjHZKK1sSbLL+i18W7mW+xrAL nG09hM7nqHgkx6Jq7ZZxm7yXP33VUPXFenmn2XAwb7HkbRw4eif9G4eUFegzbaAF9B9d NwNw/ApRjUGIaPsKBeFyWciTBzceR1RJLNBM9aTGg8bPMp1prUrh8KL0xXqq8bBek9ZL om9Lku+XchTVTXd+FhRDQFKgA3U8aZoPK8ZNosZwInKuwawoD4ODga8s8Kp+rRADfLv5 Hx/NyyN4nrarPQ5TzGrTT6cMZ4sq3GJDRR1xvp1J0QoFaX5coQaVk/be9EFaoc6/2oSO qKdw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=yYsRe09JkVPoqaZ4OGb+wtxVDI81dN8UGiocCLjkwaM=; b=ZsFQJhTxP5m76XA+EFZKK1OcjR5G/MV4qaVQk9l4IhQi0Wj8kJNJdU8tPHjBnvsuYT CP0iTQIChrpOgM3ReYXWTLQyo529ZveJL2LWOKKUV5cSIGCYE1a45Wmb/nybj9f6EofA 2QJy2RJ8gZbMGcbUhxXNbGajy5iXoKpS6JNcL9OlJhjaJ12O9l9DE6C5MihXxmKQhCn8 p8t9uX3v7Gh/0KTPMURk17pPoN6LpK2/H/Iw84BWwTOawY+ZeurOVJr/+n1xlo/CW1Rr A04LceZw6xJ5NFhvZ7tu9zEZX0BibqX6FqNPcJ6YUeBh6UkmP/yveWtwzgFhFn6M4hZd A0aA== X-Gm-Message-State: AFqh2kpF40HMfYGz0iAYvfsSzAFF7/D6utcecxv00k+IdUc9muuINm3o t5zZVKxj/16tn1PRjQ+NzvGvH+5/xDc= X-Received: by 2002:a62:ab0b:0:b0:58b:46c9:a6b1 with SMTP id p11-20020a62ab0b000000b0058b46c9a6b1mr6485937pff.33.1674021970712; Tue, 17 Jan 2023 22:06:10 -0800 (PST) Received: from balhae.hsd1.ca.comcast.net ([2601:647:6780:ff0:3749:9eb3:dfb5:f449]) by smtp.gmail.com with ESMTPSA id b126-20020a62cf84000000b00574e84ed847sm10990180pfg.24.2023.01.17.22.06.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Jan 2023 22:06:10 -0800 (PST) Sender: Namhyung Kim From: Namhyung Kim To: Peter Zijlstra , Ingo Molnar Cc: Mark Rutland , Alexander Shishkin , Arnaldo Carvalho de Melo , Jiri Olsa , LKML , Song Liu , bpf@vger.kernel.org, linux-s390@vger.kernel.org, x86@kernel.org Subject: [PATCH 7/8] perf/core: Introduce perf_prepare_header() Date: Tue, 17 Jan 2023 22:05:58 -0800 Message-Id: <20230118060559.615653-8-namhyung@kernel.org> X-Mailer: git-send-email 2.39.0.314.g84b9a713c41-goog In-Reply-To: <20230118060559.615653-1-namhyung@kernel.org> References: <20230118060559.615653-1-namhyung@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1755344568572514524?= X-GMAIL-MSGID: =?utf-8?q?1755344568572514524?= Factor out perf_prepare_header() so that it can call perf_prepare_sample() without a header if not needed. Also it checks the filtered_sample_type to avoid duplicate work when perf_prepare_sample() is called twice (or more). Cc: linux-s390@vger.kernel.org Cc: x86@kernel.org Suggested-by: Peter Zijlstr Acked-by: Jiri Olsa Acked-by: Song Liu Tested-by: Jiri Olsa Signed-off-by: Namhyung Kim --- arch/s390/kernel/perf_cpum_sf.c | 3 ++- arch/x86/events/intel/ds.c | 3 ++- include/linux/perf_event.h | 16 +++++++++++++- kernel/events/core.c | 38 +++++++++++++++++++++------------ 4 files changed, 43 insertions(+), 17 deletions(-) diff --git a/arch/s390/kernel/perf_cpum_sf.c b/arch/s390/kernel/perf_cpum_sf.c index 332a49965130..fd02f8423243 100644 --- a/arch/s390/kernel/perf_cpum_sf.c +++ b/arch/s390/kernel/perf_cpum_sf.c @@ -671,7 +671,8 @@ static void cpumsf_output_event_pid(struct perf_event *event, /* Protect callchain buffers, tasks */ rcu_read_lock(); - perf_prepare_sample(&header, data, event, regs); + perf_prepare_sample(data, event, regs); + perf_prepare_header(&header, data, event, regs); if (perf_output_begin(&handle, data, event, header.size)) goto out; diff --git a/arch/x86/events/intel/ds.c b/arch/x86/events/intel/ds.c index 07c8a2cdc3ee..183efa914b99 100644 --- a/arch/x86/events/intel/ds.c +++ b/arch/x86/events/intel/ds.c @@ -807,7 +807,8 @@ int intel_pmu_drain_bts_buffer(void) * the sample. */ rcu_read_lock(); - perf_prepare_sample(&header, &data, event, ®s); + perf_prepare_sample(&data, event, ®s); + perf_prepare_header(&header, &data, event, ®s); if (perf_output_begin(&handle, &data, event, header.size * (top - base - skip))) diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h index 7db0e9cc2682..d5628a7b5eaa 100644 --- a/include/linux/perf_event.h +++ b/include/linux/perf_event.h @@ -1250,6 +1250,17 @@ static inline void perf_sample_save_brstack(struct perf_sample_data *data, data->sample_flags |= PERF_SAMPLE_BRANCH_STACK; } +static inline u32 perf_sample_data_size(struct perf_sample_data *data, + struct perf_event *event) +{ + u32 size = sizeof(struct perf_event_header); + + size += event->header_size + event->id_header_size; + size += data->dyn_size; + + return size; +} + /* * Clear all bitfields in the perf_branch_entry. * The to and from fields are not cleared because they are @@ -1271,7 +1282,10 @@ extern void perf_output_sample(struct perf_output_handle *handle, struct perf_event_header *header, struct perf_sample_data *data, struct perf_event *event); -extern void perf_prepare_sample(struct perf_event_header *header, +extern void perf_prepare_sample(struct perf_sample_data *data, + struct perf_event *event, + struct pt_regs *regs); +extern void perf_prepare_header(struct perf_event_header *header, struct perf_sample_data *data, struct perf_event *event, struct pt_regs *regs); diff --git a/kernel/events/core.c b/kernel/events/core.c index 9cc55122188f..73c40ce84c48 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -7575,20 +7575,13 @@ static __always_inline u64 __cond_set(u64 flags, u64 s, u64 d) return d * !!(flags & s); } -void perf_prepare_sample(struct perf_event_header *header, - struct perf_sample_data *data, +void perf_prepare_sample(struct perf_sample_data *data, struct perf_event *event, struct pt_regs *regs) { u64 sample_type = event->attr.sample_type; u64 filtered_sample_type; - header->type = PERF_RECORD_SAMPLE; - header->size = sizeof(*header) + event->header_size + event->id_header_size; - - header->misc = 0; - header->misc |= perf_misc_flags(regs); - /* * Add the sample flags that are dependent to others. And clear the * sample flags that have already been done by the PMU driver. @@ -7602,6 +7595,12 @@ void perf_prepare_sample(struct perf_event_header *header, PERF_SAMPLE_REGS_USER); filtered_sample_type &= ~data->sample_flags; + if (filtered_sample_type == 0) { + /* Make sure it has the correct data->type for output */ + data->type = event->attr.sample_type; + return; + } + __perf_event_header__init_id(data, event, filtered_sample_type); if (filtered_sample_type & PERF_SAMPLE_IP) { @@ -7653,9 +7652,10 @@ void perf_prepare_sample(struct perf_event_header *header, * up the rest of the sample size. */ u16 stack_size = event->attr.sample_stack_user; + u16 header_size = perf_sample_data_size(data, event); u16 size = sizeof(u64); - stack_size = perf_sample_ustack_size(stack_size, header->size, + stack_size = perf_sample_ustack_size(stack_size, header_size, data->regs_user.regs); /* @@ -7740,8 +7740,9 @@ void perf_prepare_sample(struct perf_event_header *header, if (filtered_sample_type & PERF_SAMPLE_AUX) { u64 size; + u16 header_size = perf_sample_data_size(data, event); - header->size += sizeof(u64); /* size */ + header_size += sizeof(u64); /* size */ /* * Given the 16bit nature of header::size, an AUX sample can @@ -7749,17 +7750,25 @@ void perf_prepare_sample(struct perf_event_header *header, * Make sure this doesn't happen by using up to U16_MAX bytes * per sample in total (rounded down to 8 byte boundary). */ - size = min_t(size_t, U16_MAX - header->size, + size = min_t(size_t, U16_MAX - header_size, event->attr.aux_sample_size); size = rounddown(size, 8); size = perf_prepare_sample_aux(event, data, size); - WARN_ON_ONCE(size + header->size > U16_MAX); + WARN_ON_ONCE(size + header_size > U16_MAX); data->dyn_size += size + sizeof(u64); /* size above */ data->sample_flags |= PERF_SAMPLE_AUX; } +} - header->size += data->dyn_size; +void perf_prepare_header(struct perf_event_header *header, + struct perf_sample_data *data, + struct perf_event *event, + struct pt_regs *regs) +{ + header->type = PERF_RECORD_SAMPLE; + header->size = perf_sample_data_size(data, event); + header->misc = perf_misc_flags(regs); /* * If you're adding more sample types here, you likely need to do @@ -7788,7 +7797,8 @@ __perf_event_output(struct perf_event *event, /* protect the callchain buffers */ rcu_read_lock(); - perf_prepare_sample(&header, data, event, regs); + perf_prepare_sample(data, event, regs); + perf_prepare_header(&header, data, event, regs); err = output_begin(&handle, data, event, header.size); if (err) From patchwork Wed Jan 18 06:05:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Namhyung Kim X-Patchwork-Id: 44989 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2171654wrn; Tue, 17 Jan 2023 22:23:56 -0800 (PST) X-Google-Smtp-Source: AMrXdXvfVRgEvBgzHrsgTcJywzOVnB46btP4+FFoP3Ts0tMSDreVn9RdY0kj0vW2ectQcNU5ub/o X-Received: by 2002:a17:902:9343:b0:194:4a2b:d7e4 with SMTP id g3-20020a170902934300b001944a2bd7e4mr6824317plp.17.1674023036195; Tue, 17 Jan 2023 22:23:56 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674023036; cv=none; d=google.com; s=arc-20160816; b=lYOZZ8A/THCYJYVPtJJlEidgGP+IdDfOXFIPSz87wm8ZbVtaYCjEacxsi2Nw6mXHdb hMy8TncPGG9tX+KFJ9m97XgDpktmciqbtlaXP3+xMcazu4D/1cTgtgplgnVrDbUiFvhR 7XUGUzZPBm+Ndxe1Nd66461bNKTXZca70oXticJsfGcUAC4Dpn3rUBFDK5BbqhEXovo7 tZc3uFEOEwlmP3dGi449NIsMtbzRbVpOXbUVKc4vu331kb/b1bBsH9H2TLHpalNXoKl0 1WLq+/oy5ra7hBoqX2Rvwjp75Z99AkXRvkkngsiRNkKgF6rEYtbc02yttka6x2PARNYR hSRw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=bUmpjVqBBhi1SGzypxOnduAfDJL57JgsOCLfCSRMLnM=; b=uxgiJFocoMAgEQpWbj+LAaEQB/2f4S/vu7vQS5uZnC/Wd3bdgb5P0wDCdi6ggqANPS kcBcISShHB6CqgVCY6XY+M0a/r4eBVTkF1Yn/4dQlGVzWkhR7pVA0BrPR5IOuDXrMSmT k9s7J+JIcYupXu+SC15tuqW5+x4A0C933VDnImEpY8r3gYKCDEIe2e7dFTeKA/qEY0NG KaZGTKJzY6XG6CyKsM4yiO4buJV51j7l6+wpVbTTqwr5VVY2R6k3QenMq8W7AeTtWECK HGSz3G9MNoY+svKb2Y0SrsUKQS9YvxsqK+u2COBXn8+YGHUNETWkhqwjtD3UQqCmGfSo edwA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=gMYlUezq; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id q23-20020a170902edd700b00189d0fa14d6si14939862plk.13.2023.01.17.22.23.43; Tue, 17 Jan 2023 22:23:56 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=gMYlUezq; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229552AbjARGXD (ORCPT + 99 others); Wed, 18 Jan 2023 01:23:03 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53484 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229727AbjARGTT (ORCPT ); Wed, 18 Jan 2023 01:19:19 -0500 Received: from mail-pf1-x42e.google.com (mail-pf1-x42e.google.com [IPv6:2607:f8b0:4864:20::42e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7CCF41E5C2; Tue, 17 Jan 2023 22:06:23 -0800 (PST) Received: by mail-pf1-x42e.google.com with SMTP id 200so19187999pfx.7; Tue, 17 Jan 2023 22:06:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=bUmpjVqBBhi1SGzypxOnduAfDJL57JgsOCLfCSRMLnM=; b=gMYlUezqD4qdnPjzfuEndSrnOUf+/3VDb7WsD0wt6b1eQ/7cmXmHEoMuaO8AJymobm VE2rRZgaEtg4NN4R2aR+6OBouY8AG0013mcrh0nOlrsF6i7c0epALpEKg3dAlzlC7ORP TGTmfKyyf0uKhOeV0jAhC3mzXc3ocd5P/oKZay0kHtTt1a0zQyWb2hsmfNp///3MPZo9 aXUm6e0H+VG5DmwjFhuQXBPuhLAUQ8ApezyyT7h3yd2hHy7msmnHingkccsYoZEWvjP4 myQROjtlZDFxXNlOd2k2OfSgZUESDLjXcuz7R4/xzFNWHr4hGpbZuU3dTtZnjMc4Y0AY yh/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=bUmpjVqBBhi1SGzypxOnduAfDJL57JgsOCLfCSRMLnM=; b=5iz+RMzBBME9gRJHdBJWKfdRDSFZehYUT8id016Ho+FJ2AfMBSVguEHcI+daxLKBNr 6pg2s3vOngMiKZX4cBBWxyKSDsDZrfx8rZ90Ii0dKOPW9C/zAGke6jLmVF/7U/8AhXY2 57LcQIJ+ChIkx/ocd2JLSSA5K4c9vF3QMT9C+LQG58QDPCVxTjJ9mDWcMkCTa8YK9E9a 50hDkCzaXPuPllHfVUnERZSz7qzyCnLJ2Nx7lOmG6GhoZBP7U/3NVocPh3r5xS69fUMA srw2HLZfS75WEGmSDkE6Baa9ZMB9qk5ES+VylIb2wnjDSe0IDcgwVbncMXEHRbX1v0Be vX6g== X-Gm-Message-State: AFqh2kqQnEMYDBoYx3z3nq8rkfQdiBzKPPHPl0Z4WBwweJuNr2+jTsf3 W7XjdVg4V+p5YXICrkOvMkcvNBeVAPk= X-Received: by 2002:a62:1615:0:b0:587:df0a:804d with SMTP id 21-20020a621615000000b00587df0a804dmr6672554pfw.27.1674021972075; Tue, 17 Jan 2023 22:06:12 -0800 (PST) Received: from balhae.hsd1.ca.comcast.net ([2601:647:6780:ff0:3749:9eb3:dfb5:f449]) by smtp.gmail.com with ESMTPSA id b126-20020a62cf84000000b00574e84ed847sm10990180pfg.24.2023.01.17.22.06.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Jan 2023 22:06:11 -0800 (PST) Sender: Namhyung Kim From: Namhyung Kim To: Peter Zijlstra , Ingo Molnar Cc: Mark Rutland , Alexander Shishkin , Arnaldo Carvalho de Melo , Jiri Olsa , LKML , Song Liu , bpf@vger.kernel.org Subject: [PATCH 8/8] perf/core: Call perf_prepare_sample() before running BPF Date: Tue, 17 Jan 2023 22:05:59 -0800 Message-Id: <20230118060559.615653-9-namhyung@kernel.org> X-Mailer: git-send-email 2.39.0.314.g84b9a713c41-goog In-Reply-To: <20230118060559.615653-1-namhyung@kernel.org> References: <20230118060559.615653-1-namhyung@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1755340379408810842?= X-GMAIL-MSGID: =?utf-8?q?1755340379408810842?= As BPF can access sample data, it needs to populate the data. Also remove the logic to get the callchain specifically as it's covered by the perf_prepare_sample() now. Cc: bpf@vger.kernel.org Acked-by: Jiri Olsa Acked-by: Song Liu Tested-by: Jiri Olsa Signed-off-by: Namhyung Kim --- kernel/events/core.c | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/kernel/events/core.c b/kernel/events/core.c index 73c40ce84c48..5af61d0292ab 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -10363,13 +10363,7 @@ static void bpf_overflow_handler(struct perf_event *event, rcu_read_lock(); prog = READ_ONCE(event->prog); if (prog) { - if (prog->call_get_stack && - (event->attr.sample_type & PERF_SAMPLE_CALLCHAIN) && - !(data->sample_flags & PERF_SAMPLE_CALLCHAIN)) { - data->callchain = perf_callchain(event, regs); - data->sample_flags |= PERF_SAMPLE_CALLCHAIN; - } - + perf_prepare_sample(data, event, regs); ret = bpf_prog_run(prog, &ctx); } rcu_read_unlock();