From patchwork Thu Jan 12 21:40:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Namhyung Kim X-Patchwork-Id: 42831 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp4121992wrt; Thu, 12 Jan 2023 13:51:10 -0800 (PST) X-Google-Smtp-Source: AMrXdXu8yrmvZqlaR8KaKRRRgSrZ9Y1nl4Yy0/hJtnQLqbAxdW+1+oFXHsit7QLZOVV4Sy1Mo2Gq X-Received: by 2002:a17:90b:358f:b0:229:1f66:97a with SMTP id mm15-20020a17090b358f00b002291f66097amr807932pjb.12.1673560270360; Thu, 12 Jan 2023 13:51:10 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1673560270; cv=none; d=google.com; s=arc-20160816; b=mAXH4xc5Sm5X1vYa11akCWlx3yjWMeEIBYkukmrxChtk3zW7eTsDobwTr8TCFTR7SS BkvhhfAmo6Jhgt1Vq2OPQMCF8PAuAhHV4MFvnS1F0jLgDPAIeOoL0etWPQRgo8x6KOVT iXXKGSDKL5Mb/Y2yj3bE7EzdqE2M1QK23ta0a9T44zmODVFyronSFlK6/LjAYCzo/fQH YQfUcfxYSYnz4xaw7UKHf1YY5dy2VOCm2CpDNypIvI6YOEHYX/rM6/DgJAQNu8ux0zN5 OMbonxyLXgwHvKzFaP4IOdgpzS426tDePZBqpKqlj1GwHZYv9J47vCboKOXuljlZgFvf U7fA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=6nuL0A6b/Cc0UYziUfDk8TnmjwdlAAeWvEhDoJKmGGg=; b=vXY/8FjoIGU+5yxdIooHXz66QB1YdVnDS3kJASchWk5GTCYTLRNLT2DyvcedOpQeg5 3wwThXDQloW7O0GSqLCkwA3yIOUMKbrfd4/AdV9Z7nYVgPoDCvIbW4ka/nqOLQoLq/8R UE0Rove8pcuRTx0ODoaz042C1HUxT/SDSUBOl2boEp9gU8n2vwuiBuTgYvecq0VRRFs/ NUmOmx2d0k1+k7AdTJV2Rsw7gbFy+DF2Waq8XfSMnAtZrVObAPJbA9BjSdDlVirBi76q s39fZVeepsEfiYNS48rmwOKJmaJvIbxwFDvEUY6/YG0q75yBuKjfdnrnwGyCSasT313y Hlmw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=h4yPXUpM; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id p19-20020a63e653000000b00478ed620d12si18354060pgj.777.2023.01.12.13.50.53; Thu, 12 Jan 2023 13:51:10 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=h4yPXUpM; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240372AbjALVsV (ORCPT + 99 others); Thu, 12 Jan 2023 16:48:21 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40728 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240345AbjALVrd (ORCPT ); Thu, 12 Jan 2023 16:47:33 -0500 Received: from mail-pf1-x435.google.com (mail-pf1-x435.google.com [IPv6:2607:f8b0:4864:20::435]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2693A95BF for ; Thu, 12 Jan 2023 13:40:21 -0800 (PST) Received: by mail-pf1-x435.google.com with SMTP id i65so11320783pfc.0 for ; Thu, 12 Jan 2023 13:40:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=6nuL0A6b/Cc0UYziUfDk8TnmjwdlAAeWvEhDoJKmGGg=; b=h4yPXUpM4r5Qaaf1l3vXl7ZzDwfTEguwSCBWDkCCeXlC7UoH/Hn6Dvbb/QNdZULD3P QpmWnNV89arn29Y8fZns3qK1XzdXdf5OS/BjNz/nv3aZU9fggTD01LCl0IP1OAuzLFvR iqpyQfB9mrPqy6RgLAw7RCE7xn1tLzfAdlMaBjIK6Fgy0RW9pIdgQF+LccZpOuRoBjFn 6ymS+Y9Vz42FBpgu24knYF+rAWaXMfIn/Ew8ixRgIHUWavQDR11RejJqUkZ+gP+MVs6Y QI7jNerOp3oGBGZSoM5jI68pDWHat25n0KVwuTSSUmLfKjH59KZxPS8NBMNOv27dZzsJ JlgQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=6nuL0A6b/Cc0UYziUfDk8TnmjwdlAAeWvEhDoJKmGGg=; b=UAstYBxOeR4OCWnRjWXPkOALlEk/EJ5Di5B9VjkDixCCe96Z9ccqn4NVYwayOCXNH7 rAYwd4XE1YNlxxlrj4QLWpJQLM3HG7PI5Wy96hYu/qOygl2f0TSeASC6iLlg9OTN9VrX kXJddvp2wZe/7jLMg1JTGOGDCq/fHrUgkDw1DvH8FTo61QdtSe9gb6vELqi/aoeaLti+ HO0JYsEleBGhbLCGZX5f2e5MaCCQ6HJk1zBYqCISDuDM3sDXOolha0ggh3SDYpoCMGBI Wp7qKYRpG/EP5p7Q+r/U9Hsu3gpyOAA5RCYoF/aiRsmOmDTZrOGHOdsMmPyDwYACmLTZ JaOg== X-Gm-Message-State: AFqh2krmBqL6IOJ7juG5zTdCk7nheKWpFkcLX6QVGlEDG/IwkoR11hTo HSj5aPjNwVi9UYRvhozUX6s= X-Received: by 2002:a05:6a00:1d03:b0:580:149a:5650 with SMTP id a3-20020a056a001d0300b00580149a5650mr79037670pfx.22.1673559620511; Thu, 12 Jan 2023 13:40:20 -0800 (PST) Received: from youngsil.svl.corp.google.com ([2620:15c:2d4:203:b71e:23d4:aa6a:47c0]) by smtp.gmail.com with ESMTPSA id q20-20020aa79834000000b0058134d2df41sm11344605pfl.146.2023.01.12.13.40.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 12 Jan 2023 13:40:20 -0800 (PST) Sender: Namhyung Kim From: Namhyung Kim To: Peter Zijlstra Cc: Ingo Molnar , Mark Rutland , Alexander Shishkin , Arnaldo Carvalho de Melo , Jiri Olsa , LKML , Song Liu Subject: [PATCH 1/8] perf/core: Save the dynamic parts of sample data size Date: Thu, 12 Jan 2023 13:40:08 -0800 Message-Id: <20230112214015.1014857-2-namhyung@kernel.org> X-Mailer: git-send-email 2.39.0.314.g84b9a713c41-goog In-Reply-To: <20230112214015.1014857-1-namhyung@kernel.org> References: <20230112214015.1014857-1-namhyung@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.7 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1754855133899452495?= X-GMAIL-MSGID: =?utf-8?q?1754855133899452495?= The perf sample data can be divided into parts. The event->header_size and event->id_header_size keep the static part of the sample data which is determined by the sample_type flags. But other parts like CALLCHAIN and BRANCH_STACK are changing dynamically so it needs to see the actual data. In preparation of handling repeated calls for perf_prepare_sample(), it can save the dynamic size to the perf sample data to avoid the duplicate work. Signed-off-by: Namhyung Kim Acked-by: Song Liu --- include/linux/perf_event.h | 2 ++ kernel/events/core.c | 17 ++++++++++------- 2 files changed, 12 insertions(+), 7 deletions(-) diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h index 03949d017ac9..16b980014449 100644 --- a/include/linux/perf_event.h +++ b/include/linux/perf_event.h @@ -1103,6 +1103,7 @@ struct perf_sample_data { */ u64 sample_flags; u64 period; + u64 dyn_size; /* * Fields commonly set by __perf_event_header__init_id(), @@ -1158,6 +1159,7 @@ static inline void perf_sample_data_init(struct perf_sample_data *data, /* remaining struct members initialized in perf_prepare_sample() */ data->sample_flags = PERF_SAMPLE_PERIOD; data->period = period; + data->dyn_size = 0; if (addr) { data->addr = addr; diff --git a/kernel/events/core.c b/kernel/events/core.c index eacc3702654d..8c8de26f04ab 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -7593,7 +7593,7 @@ void perf_prepare_sample(struct perf_event_header *header, size += data->callchain->nr; - header->size += size * sizeof(u64); + data->dyn_size += size * sizeof(u64); } if (sample_type & PERF_SAMPLE_RAW) { @@ -7619,7 +7619,7 @@ void perf_prepare_sample(struct perf_event_header *header, data->raw = NULL; } - header->size += size; + data->dyn_size += size; } if (sample_type & PERF_SAMPLE_BRANCH_STACK) { @@ -7631,7 +7631,7 @@ void perf_prepare_sample(struct perf_event_header *header, size += data->br_stack->nr * sizeof(struct perf_branch_entry); } - header->size += size; + data->dyn_size += size; } if (sample_type & (PERF_SAMPLE_REGS_USER | PERF_SAMPLE_STACK_USER)) @@ -7646,7 +7646,7 @@ void perf_prepare_sample(struct perf_event_header *header, size += hweight64(mask) * sizeof(u64); } - header->size += size; + data->dyn_size += size; } if (sample_type & PERF_SAMPLE_STACK_USER) { @@ -7671,7 +7671,7 @@ void perf_prepare_sample(struct perf_event_header *header, size += sizeof(u64) + stack_size; data->stack_user_size = stack_size; - header->size += size; + data->dyn_size += size; } if (filtered_sample_type & PERF_SAMPLE_WEIGHT_TYPE) @@ -7700,7 +7700,7 @@ void perf_prepare_sample(struct perf_event_header *header, size += hweight64(mask) * sizeof(u64); } - header->size += size; + data->dyn_size += size; } if (sample_type & PERF_SAMPLE_PHYS_ADDR && @@ -7745,8 +7745,11 @@ void perf_prepare_sample(struct perf_event_header *header, size = perf_prepare_sample_aux(event, data, size); WARN_ON_ONCE(size + header->size > U16_MAX); - header->size += size; + data->dyn_size += size + sizeof(u64); /* size above */ } + + header->size += data->dyn_size; + /* * If you're adding more sample types here, you likely need to do * something about the overflowing header::size, like repurpose the From patchwork Thu Jan 12 21:40:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Namhyung Kim X-Patchwork-Id: 42832 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp4121998wrt; Thu, 12 Jan 2023 13:51:12 -0800 (PST) X-Google-Smtp-Source: AMrXdXvhlt1Uf8x9O7/aoJ6UGDuNcQGa4o7LVlKJKly7qoqlLin3vin5K/pAiRtW8HG2Pfg2QhjV X-Received: by 2002:a17:902:d710:b0:194:6afa:ca with SMTP id w16-20020a170902d71000b001946afa00camr764492ply.56.1673560271802; Thu, 12 Jan 2023 13:51:11 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1673560271; cv=none; d=google.com; s=arc-20160816; b=kiddPPnan4mYggadigAxXOTuEgC9MGAARkXMKKSf3RNQyY32rWVj6RE9OwIRAg5/Jt QcIDmmWFldiN9blk4jr49uGiy7Mj/L0DDo4X3Z62O8qWNFxhmkB1/HEqbqKpDCxwLzLe LOh0Zv+84A3TGJeH0XT6E4TLXS6RREmiMU4LcTsd/G5E/rCRxAtLJwCGv0iUycG34eW2 PDlr8SfGs/UTJ9VEaHvOJct4L4P5boAT0+lN3T7a7g7iBT+X3WjPL4YZFgwg/Hh5U4Wn cWe+sUW0JnTzoLAZhg8Ws5+Z/dP30UL8GCFPOMZ3nc0zmxP/Wlm9q2JDma3/Q7bku07y 9Xdg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=4JJiMcoWlhZgKB78U5VEVvQdWOzu1ox9DCMJw9an4s8=; b=Ym+Rfxoh3VGG2DardDTe9wsbO8umENSCaXDVdtTm+SkMkTwjJiDO989oXFvU/O7fhM v3MwpGcQopopkdEFY+QGmT3Jl3/BiawF9VRorGs3ZqWpO6NbDHma8UymcB5YQZWDZw3q KZDkSDlPGVa6pYZ7hdahogYQEsgAFsor6cPahKi/PrR0c/o8wfrC0w1KcAHgN1rkH6cv eWBvLoPrncgXyhVrEtaLL6wlaOC8iW3kTM1Wx5ZHG1SBTlcYHMYFK/2sMboWeWqWhQKi NHRhK/Qw91iG6QEBJ9uWOs5miR5ZFgY79TQrSbiyA7VMSFAGSVAXuqTHKYXVuoizM0hc 4Itw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=kUvO4oov; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id g23-20020a63e617000000b004988e653842si17956242pgh.139.2023.01.12.13.50.56; Thu, 12 Jan 2023 13:51:11 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=kUvO4oov; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240729AbjALVsc (ORCPT + 99 others); Thu, 12 Jan 2023 16:48:32 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40730 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240727AbjALVre (ORCPT ); Thu, 12 Jan 2023 16:47:34 -0500 Received: from mail-pf1-x431.google.com (mail-pf1-x431.google.com [IPv6:2607:f8b0:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 915DA6DBB8 for ; Thu, 12 Jan 2023 13:40:22 -0800 (PST) Received: by mail-pf1-x431.google.com with SMTP id c26so10354478pfp.10 for ; Thu, 12 Jan 2023 13:40:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=4JJiMcoWlhZgKB78U5VEVvQdWOzu1ox9DCMJw9an4s8=; b=kUvO4oovmAHHSaIoJ2m3MMu7qTo+mucZEpq2qcSyu3sEdW2HrH2rrQ6oAmVESctwp5 i1Vx3qMpoWOWGqLS5b2DooVkqmswps4vNl+zOWnSXVpiD6sC6fWILS6iWG5+SGPHEy0K WP1a/qe6NEewQOfIgjqtHxScWwYQ9XOBAwL8CcTAK1XIDDCsq9UlCLnaw7j7www4RjQ9 w0Qx/xy1Xqm6pN8vB6jrhrWW0AUTWH590XaDQce6pwQIvPw7W7OkOUEEEz3w09mzaP/2 MsWlfXtGvcFfO+Al2MrB0WxxosBw7P5OKP8TNBO6YkYPFsxy22nZZKQOVD2JXiREknNl wIXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=4JJiMcoWlhZgKB78U5VEVvQdWOzu1ox9DCMJw9an4s8=; b=rpnkH8yLIYcnLieHfGx1DprTpBVHdKZTeZalSGLFXk5o5wamw5pw/8tDuztFelP1PC m1ZYPLo0MKDaSWQ1SnxUwBnMOYXvAlJKFrKfAFPo2Vy441zgJUr2EfpLDeW+gnij3ZPj haN6HN4vFuE4AsNmgRsbG1iPZE6FPtx8P2zMevU3dWXDle3U21NrL8Ux9AIVi+bGERyn YWmGoX22zeersZTB7qRcgLYSJsOXX55wjH5PCYrRXIH8NW7BWvwBg906DwuVPCHOmTCt KB2XUyaCqgfprKUTWj6WH2F4owq5OvE4EadD7ejAaapV8GH86Y58Dfoz2bkQ0ahL7P5J /ihA== X-Gm-Message-State: AFqh2kovPSDG03YOiGHJKpjH6UGuEj/IwiyknAYlnQUInKQ18pHPNsRA Gzo54z81vP7Hhc0/Uza4aNL4IidDkdA= X-Received: by 2002:a05:6a00:896:b0:586:35dd:91c3 with SMTP id q22-20020a056a00089600b0058635dd91c3mr25796905pfj.29.1673559621684; Thu, 12 Jan 2023 13:40:21 -0800 (PST) Received: from youngsil.svl.corp.google.com ([2620:15c:2d4:203:b71e:23d4:aa6a:47c0]) by smtp.gmail.com with ESMTPSA id q20-20020aa79834000000b0058134d2df41sm11344605pfl.146.2023.01.12.13.40.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 12 Jan 2023 13:40:21 -0800 (PST) Sender: Namhyung Kim From: Namhyung Kim To: Peter Zijlstra Cc: Ingo Molnar , Mark Rutland , Alexander Shishkin , Arnaldo Carvalho de Melo , Jiri Olsa , LKML , Song Liu , x86@kernel.org Subject: [PATCH 2/8] perf/core: Add perf_sample_save_callchain() helper Date: Thu, 12 Jan 2023 13:40:09 -0800 Message-Id: <20230112214015.1014857-3-namhyung@kernel.org> X-Mailer: git-send-email 2.39.0.314.g84b9a713c41-goog In-Reply-To: <20230112214015.1014857-1-namhyung@kernel.org> References: <20230112214015.1014857-1-namhyung@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.7 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1754855135174681828?= X-GMAIL-MSGID: =?utf-8?q?1754855135174681828?= When it saves the callchain to the perf sample data, it needs to update the sample flags and the dynamic size. To make sure this, add the perf_sample_save_callchain() helper and convert all call sites. Cc: x86@kernel.org Suggested-by: Peter Zijlstra Signed-off-by: Namhyung Kim Acked-by: Song Liu --- arch/x86/events/amd/ibs.c | 6 ++---- arch/x86/events/intel/ds.c | 12 ++++-------- include/linux/perf_event.h | 16 +++++++++++++++- kernel/events/core.c | 12 ++---------- 4 files changed, 23 insertions(+), 23 deletions(-) diff --git a/arch/x86/events/amd/ibs.c b/arch/x86/events/amd/ibs.c index da3f5ebac4e1..417c80bd3274 100644 --- a/arch/x86/events/amd/ibs.c +++ b/arch/x86/events/amd/ibs.c @@ -1122,10 +1122,8 @@ static int perf_ibs_handle_irq(struct perf_ibs *perf_ibs, struct pt_regs *iregs) * recorded as part of interrupt regs. Thus we need to use rip from * interrupt regs while unwinding call stack. */ - if (event->attr.sample_type & PERF_SAMPLE_CALLCHAIN) { - data.callchain = perf_callchain(event, iregs); - data.sample_flags |= PERF_SAMPLE_CALLCHAIN; - } + if (event->attr.sample_type & PERF_SAMPLE_CALLCHAIN) + perf_sample_save_callchain(&data, event, iregs); throttle = perf_event_overflow(event, &data, ®s); out: diff --git a/arch/x86/events/intel/ds.c b/arch/x86/events/intel/ds.c index 6ec326b47e2e..158cf845fc80 100644 --- a/arch/x86/events/intel/ds.c +++ b/arch/x86/events/intel/ds.c @@ -1617,10 +1617,8 @@ static void setup_pebs_fixed_sample_data(struct perf_event *event, * previous PMI context or an (I)RET happened between the record and * PMI. */ - if (sample_type & PERF_SAMPLE_CALLCHAIN) { - data->callchain = perf_callchain(event, iregs); - data->sample_flags |= PERF_SAMPLE_CALLCHAIN; - } + if (sample_type & PERF_SAMPLE_CALLCHAIN) + perf_sample_save_callchain(data, event, iregs); /* * We use the interrupt regs as a base because the PEBS record does not @@ -1795,10 +1793,8 @@ static void setup_pebs_adaptive_sample_data(struct perf_event *event, * previous PMI context or an (I)RET happened between the record and * PMI. */ - if (sample_type & PERF_SAMPLE_CALLCHAIN) { - data->callchain = perf_callchain(event, iregs); - data->sample_flags |= PERF_SAMPLE_CALLCHAIN; - } + if (sample_type & PERF_SAMPLE_CALLCHAIN) + perf_sample_save_callchain(data, event, iregs); *regs = *iregs; /* The ip in basic is EventingIP */ diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h index 16b980014449..a9419608402b 100644 --- a/include/linux/perf_event.h +++ b/include/linux/perf_event.h @@ -1095,6 +1095,8 @@ int perf_event_read_local(struct perf_event *event, u64 *value, extern u64 perf_event_read_value(struct perf_event *event, u64 *enabled, u64 *running); +extern struct perf_callchain_entry *perf_callchain(struct perf_event *event, struct pt_regs *regs); + struct perf_sample_data { /* @@ -1167,6 +1169,19 @@ static inline void perf_sample_data_init(struct perf_sample_data *data, } } +static inline void perf_sample_save_callchain(struct perf_sample_data *data, + struct perf_event *event, + struct pt_regs *regs) +{ + int size = 1; + + data->callchain = perf_callchain(event, regs); + size += data->callchain->nr; + + data->dyn_size += size * sizeof(u64); + data->sample_flags |= PERF_SAMPLE_CALLCHAIN; +} + /* * Clear all bitfields in the perf_branch_entry. * The to and from fields are not cleared because they are @@ -1408,7 +1423,6 @@ extern void perf_callchain_kernel(struct perf_callchain_entry_ctx *entry, struct extern struct perf_callchain_entry * get_perf_callchain(struct pt_regs *regs, u32 init_nr, bool kernel, bool user, u32 max_stack, bool crosstask, bool add_mark); -extern struct perf_callchain_entry *perf_callchain(struct perf_event *event, struct pt_regs *regs); extern int get_callchain_buffers(int max_stack); extern void put_callchain_buffers(void); extern struct perf_callchain_entry *get_callchain_entry(int *rctx); diff --git a/kernel/events/core.c b/kernel/events/core.c index 8c8de26f04ab..0fba98b9cd65 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -7585,16 +7585,8 @@ void perf_prepare_sample(struct perf_event_header *header, if (sample_type & (PERF_SAMPLE_IP | PERF_SAMPLE_CODE_PAGE_SIZE)) data->ip = perf_instruction_pointer(regs); - if (sample_type & PERF_SAMPLE_CALLCHAIN) { - int size = 1; - - if (filtered_sample_type & PERF_SAMPLE_CALLCHAIN) - data->callchain = perf_callchain(event, regs); - - size += data->callchain->nr; - - data->dyn_size += size * sizeof(u64); - } + if (filtered_sample_type & PERF_SAMPLE_CALLCHAIN) + perf_sample_save_callchain(data, event, regs); if (sample_type & PERF_SAMPLE_RAW) { struct perf_raw_record *raw = data->raw; From patchwork Thu Jan 12 21:40:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Namhyung Kim X-Patchwork-Id: 42828 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp4121911wrt; Thu, 12 Jan 2023 13:50:56 -0800 (PST) X-Google-Smtp-Source: AMrXdXuNmUyA9q2LWe4uIy9vtLHMCDf8GYadJRZ7nHAj4F+vqCFqdfvTrDZ7tmDgoCar6FRBG2xO X-Received: by 2002:a17:906:1398:b0:84d:47e3:fe49 with SMTP id f24-20020a170906139800b0084d47e3fe49mr15173364ejc.39.1673560256335; Thu, 12 Jan 2023 13:50:56 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1673560256; cv=none; d=google.com; s=arc-20160816; b=hiswglvdPJP8N9pINB30iV40y2RLO5nikeC0r/zwONGI2I4nGfck/1jdXYJA4Glf6O 9U9j+GtjFItZGiC7L4+NHK1KTwJLLGi0S/umcQ1hhYlA6+WMcFiWEj1J90P2U36veTy3 SqUjyXAo+AMT2EMxuX5+WYW/uriRB8eI+V+n0Pe1sJCfOd/kqX9WLhwm2AR++p91Ew+/ 7zD8ar7sqS1I8Ifif9KKQRvpeNBcaDiEJx1qYcjGNxMoMhUMO9Q1tfSdxEgHloIP1JtA WTQ8qYCqWVqOLtNluSxQWr0tjhsIaZytRQrCP0I5CJiUqJFUfqUcELRjhZX3S8AUjIcQ cwVw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=l3GEYOgHwVgvJBmBnAdUL8BdmY/9n1zanIfz8U25Xac=; b=xnClS8/j7h3OeVp6MWSdMvNNsU2KXmTD+Et7VCuXfxeE9oO64rAWVl5UB+SSe42Rva GVTaDelumkyUX+k9h3Bg2ayhpAWJc2SZKn+DAz65eRpzl7fCZX8bzktqVwqVIFby/tf7 BLxTEwv92oZIoDGAGUeq3cHMLinfT8Jeokugk42ho7NJs2xcrIWxIudjOKBeUdmPANiL 91RsxveTd3lkEfFS5C2GKDSRY7Yt0StZ82LQknRoSbqaKp1HWcNr2dwsCqRqbeWqa3PS e/TGlEgBiS/yVuaw1gBnGL6Wo8NkOR7BJSBjiS71a6fVc40Zsq+Y9AYGcf5BwmLU1y8T XlYA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=Gb11zOzI; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id ho12-20020a1709070e8c00b00859812aea43si9564933ejc.568.2023.01.12.13.50.31; Thu, 12 Jan 2023 13:50:56 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=Gb11zOzI; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240825AbjALVsf (ORCPT + 99 others); Thu, 12 Jan 2023 16:48:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40464 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240730AbjALVrg (ORCPT ); Thu, 12 Jan 2023 16:47:36 -0500 Received: from mail-pg1-x532.google.com (mail-pg1-x532.google.com [IPv6:2607:f8b0:4864:20::532]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D18686DBA4; Thu, 12 Jan 2023 13:40:23 -0800 (PST) Received: by mail-pg1-x532.google.com with SMTP id h192so13691860pgc.7; Thu, 12 Jan 2023 13:40:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=l3GEYOgHwVgvJBmBnAdUL8BdmY/9n1zanIfz8U25Xac=; b=Gb11zOzI9XeCl+/zCSz8IqtHBx1WB/Pd3fb3E2R32TTjj2OD1cwX9X8V6TSptQQv4i MaFNBKnV3reJc2C/aH3UjumX3hFZ0Y+WpC9LSBGH4NS9RDx2XCa0C0kpe98OHCg08b+/ vLC6CfHccT4mUKZHJicRNjOpCGAbGjIQ25ywc2nSHkbHLq1kWI2hN08wxujg6BURrs/9 e1H0brQCK9Enl3zWbMiwOWn9y0wJw21x/i7Obphk7dBJDd1ss8j4b0ewmTwLr0qXH2ok kdFo2Kw81t/lH4lGnlX9pNWssTvKD88pq4ExF1dQmxEjfktEhXm6pIh0VJ30507tC08b UpYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=l3GEYOgHwVgvJBmBnAdUL8BdmY/9n1zanIfz8U25Xac=; b=bArBiHr6srPrm8CNsUgDFDCyq4aOETUOkiiZC0WUT5vgFJXLsJkalil9r+CTLbYY4a e1gH79HYO1PW257VpREAa0hAL+yoh3rRRjN73DlO1DXERa8+q1KsJsRWkw9Im+w9/bTl 7UgZdWuMJgPKOuhc5M51nKclANnJiiQ9z4xvw3UvSqoIhPXG92ZV4NpPiQLqnGLIp2fl WAUMNdASvvEWvhRYygEFeVOcyCZbMferFSb9zn/hT/W502/zauLaSHxTJvjiBc3ESVN3 i9PrlA6r72C5I26p4dGhwG4tBVru0hC36t+MRYud0jiarXCia+79sncB5cM1kvvIqGVk MSPw== X-Gm-Message-State: AFqh2koxHwOpdW9oD3R6SSrOya05GDWwL/U/Y9c2AxQGTGVtRbRAofHy 3Kk/jEX1yKPNuwaJycg/TDA= X-Received: by 2002:a62:cd4e:0:b0:58a:d606:4258 with SMTP id o75-20020a62cd4e000000b0058ad6064258mr11338748pfg.10.1673559623278; Thu, 12 Jan 2023 13:40:23 -0800 (PST) Received: from youngsil.svl.corp.google.com ([2620:15c:2d4:203:b71e:23d4:aa6a:47c0]) by smtp.gmail.com with ESMTPSA id q20-20020aa79834000000b0058134d2df41sm11344605pfl.146.2023.01.12.13.40.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 12 Jan 2023 13:40:22 -0800 (PST) Sender: Namhyung Kim From: Namhyung Kim To: Peter Zijlstra Cc: Ingo Molnar , Mark Rutland , Alexander Shishkin , Arnaldo Carvalho de Melo , Jiri Olsa , LKML , Song Liu , linux-s390@vger.kernel.org, x86@kernel.org, bpf@vger.kernel.org Subject: [PATCH 3/8] perf/core: Add perf_sample_save_raw_data() helper Date: Thu, 12 Jan 2023 13:40:10 -0800 Message-Id: <20230112214015.1014857-4-namhyung@kernel.org> X-Mailer: git-send-email 2.39.0.314.g84b9a713c41-goog In-Reply-To: <20230112214015.1014857-1-namhyung@kernel.org> References: <20230112214015.1014857-1-namhyung@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.7 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1754855119388072820?= X-GMAIL-MSGID: =?utf-8?q?1754855119388072820?= When it saves the raw_data to the perf sample data, it needs to update the sample flags and the dynamic size. To make sure this, add the perf_sample_save_raw_data() helper and convert all call sites. Cc: linux-s390@vger.kernel.org Cc: x86@kernel.org Cc: bpf@vger.kernel.org Suggested-by: Peter Zijlstra Signed-off-by: Namhyung Kim --- arch/s390/kernel/perf_cpum_cf.c | 4 +--- arch/s390/kernel/perf_pai_crypto.c | 4 +--- arch/s390/kernel/perf_pai_ext.c | 4 +--- arch/x86/events/amd/ibs.c | 3 +-- include/linux/perf_event.h | 33 +++++++++++++++++++++++++----- kernel/events/core.c | 31 +++++----------------------- kernel/trace/bpf_trace.c | 6 ++---- 7 files changed, 39 insertions(+), 46 deletions(-) diff --git a/arch/s390/kernel/perf_cpum_cf.c b/arch/s390/kernel/perf_cpum_cf.c index f043a7ff220b..aa38649c7c27 100644 --- a/arch/s390/kernel/perf_cpum_cf.c +++ b/arch/s390/kernel/perf_cpum_cf.c @@ -662,9 +662,7 @@ static int cfdiag_push_sample(struct perf_event *event, if (event->attr.sample_type & PERF_SAMPLE_RAW) { raw.frag.size = cpuhw->usedss; raw.frag.data = cpuhw->stop; - raw.size = raw.frag.size; - data.raw = &raw; - data.sample_flags |= PERF_SAMPLE_RAW; + perf_sample_save_raw_data(&data, &raw); } overflow = perf_event_overflow(event, &data, ®s); diff --git a/arch/s390/kernel/perf_pai_crypto.c b/arch/s390/kernel/perf_pai_crypto.c index 985e243a2ed8..a7b339c4fd7c 100644 --- a/arch/s390/kernel/perf_pai_crypto.c +++ b/arch/s390/kernel/perf_pai_crypto.c @@ -362,9 +362,7 @@ static int paicrypt_push_sample(void) if (event->attr.sample_type & PERF_SAMPLE_RAW) { raw.frag.size = rawsize; raw.frag.data = cpump->save; - raw.size = raw.frag.size; - data.raw = &raw; - data.sample_flags |= PERF_SAMPLE_RAW; + perf_sample_save_raw_data(&data, &raw); } overflow = perf_event_overflow(event, &data, ®s); diff --git a/arch/s390/kernel/perf_pai_ext.c b/arch/s390/kernel/perf_pai_ext.c index 1138f57baae3..555597222bad 100644 --- a/arch/s390/kernel/perf_pai_ext.c +++ b/arch/s390/kernel/perf_pai_ext.c @@ -451,9 +451,7 @@ static int paiext_push_sample(void) if (event->attr.sample_type & PERF_SAMPLE_RAW) { raw.frag.size = rawsize; raw.frag.data = cpump->save; - raw.size = raw.frag.size; - data.raw = &raw; - data.sample_flags |= PERF_SAMPLE_RAW; + perf_sample_save_raw_data(&data, &raw); } overflow = perf_event_overflow(event, &data, ®s); diff --git a/arch/x86/events/amd/ibs.c b/arch/x86/events/amd/ibs.c index 417c80bd3274..64582954b5f6 100644 --- a/arch/x86/events/amd/ibs.c +++ b/arch/x86/events/amd/ibs.c @@ -1110,8 +1110,7 @@ static int perf_ibs_handle_irq(struct perf_ibs *perf_ibs, struct pt_regs *iregs) .data = ibs_data.data, }, }; - data.raw = &raw; - data.sample_flags |= PERF_SAMPLE_RAW; + perf_sample_save_raw_data(&data, &raw); } if (perf_ibs == &perf_ibs_op) diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h index a9419608402b..569dfac5887f 100644 --- a/include/linux/perf_event.h +++ b/include/linux/perf_event.h @@ -95,6 +95,11 @@ struct perf_raw_record { u32 size; }; +static __always_inline bool perf_raw_frag_last(const struct perf_raw_frag *frag) +{ + return frag->pad < sizeof(u64); +} + /* * branch stack layout: * nr: number of taken branches stored in entries[] @@ -1182,6 +1187,29 @@ static inline void perf_sample_save_callchain(struct perf_sample_data *data, data->sample_flags |= PERF_SAMPLE_CALLCHAIN; } +static inline void perf_sample_save_raw_data(struct perf_sample_data *data, + struct perf_raw_record *raw) +{ + struct perf_raw_frag *frag = &raw->frag; + u32 sum = 0; + int size; + + do { + sum += frag->size; + if (perf_raw_frag_last(frag)) + break; + frag = frag->next; + } while (1); + + size = round_up(sum + sizeof(u32), sizeof(u64)); + raw->size = size - sizeof(u32); + frag->pad = raw->size - sum; + + data->raw = raw; + data->dyn_size += size; + data->sample_flags |= PERF_SAMPLE_RAW; +} + /* * Clear all bitfields in the perf_branch_entry. * The to and from fields are not cleared because they are @@ -1690,11 +1718,6 @@ extern void perf_restore_debug_store(void); static inline void perf_restore_debug_store(void) { } #endif -static __always_inline bool perf_raw_frag_last(const struct perf_raw_frag *frag) -{ - return frag->pad < sizeof(u64); -} - #define perf_output_put(handle, x) perf_output_copy((handle), &(x), sizeof(x)) struct perf_pmu_events_attr { diff --git a/kernel/events/core.c b/kernel/events/core.c index 0fba98b9cd65..133894ae5e30 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -7588,30 +7588,10 @@ void perf_prepare_sample(struct perf_event_header *header, if (filtered_sample_type & PERF_SAMPLE_CALLCHAIN) perf_sample_save_callchain(data, event, regs); - if (sample_type & PERF_SAMPLE_RAW) { - struct perf_raw_record *raw = data->raw; - int size; - - if (raw && (data->sample_flags & PERF_SAMPLE_RAW)) { - struct perf_raw_frag *frag = &raw->frag; - u32 sum = 0; - - do { - sum += frag->size; - if (perf_raw_frag_last(frag)) - break; - frag = frag->next; - } while (1); - - size = round_up(sum + sizeof(u32), sizeof(u64)); - raw->size = size - sizeof(u32); - frag->pad = raw->size - sum; - } else { - size = sizeof(u64); - data->raw = NULL; - } - - data->dyn_size += size; + if (filtered_sample_type & PERF_SAMPLE_RAW) { + data->raw = NULL; + data->dyn_size += sizeof(u64); + data->sample_flags |= PERF_SAMPLE_RAW; } if (sample_type & PERF_SAMPLE_BRANCH_STACK) { @@ -10127,8 +10107,7 @@ void perf_tp_event(u16 event_type, u64 count, void *record, int entry_size, }; perf_sample_data_init(&data, 0, 0); - data.raw = &raw; - data.sample_flags |= PERF_SAMPLE_RAW; + perf_sample_save_raw_data(&data, &raw); perf_trace_buf_update(record, event_type); diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index 3bbd3f0c810c..ad37608afc35 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -687,8 +687,7 @@ BPF_CALL_5(bpf_perf_event_output, struct pt_regs *, regs, struct bpf_map *, map, } perf_sample_data_init(sd, 0, 0); - sd->raw = &raw; - sd->sample_flags |= PERF_SAMPLE_RAW; + perf_sample_save_raw_data(sd, &raw); err = __bpf_perf_event_output(regs, map, flags, sd); @@ -746,8 +745,7 @@ u64 bpf_event_output(struct bpf_map *map, u64 flags, void *meta, u64 meta_size, perf_fetch_caller_regs(regs); perf_sample_data_init(sd, 0, 0); - sd->raw = &raw; - sd->sample_flags |= PERF_SAMPLE_RAW; + perf_sample_save_raw_data(sd, &raw); ret = __bpf_perf_event_output(regs, map, flags, sd); out: From patchwork Thu Jan 12 21:40:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Namhyung Kim X-Patchwork-Id: 42827 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp4121893wrt; Thu, 12 Jan 2023 13:50:53 -0800 (PST) X-Google-Smtp-Source: AMrXdXt/Hq/b0ZM+fQ4kgS38cneMvvk5VkWAhs3VuN/SUFzUiRPWbZ0WIkVfYwqEjzkz8qUwCPPi X-Received: by 2002:a17:907:a789:b0:7c0:bbab:22e5 with SMTP id vx9-20020a170907a78900b007c0bbab22e5mr71140976ejc.16.1673560253669; Thu, 12 Jan 2023 13:50:53 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1673560253; cv=none; d=google.com; s=arc-20160816; b=eweqPhkcf8L/bY2l55zDBnIb4LuCfCHZ76Mv2Ct3O54g4FejV35O3JGTiBbW6iUAPb Y/m6Oex18odqK2fPs1Ljznolk2JiGKTJIkh4t4742yAS8H+ODJaOCbsTXyxQlr83UXjd aqNalHTEKk2tov7VIH3UcsE0ycTNg6LDALsx3zg5s8lR9+O+qDQ96aWaE2pGK2dXFzFi dg2oC562VipMIIfCXmKJ6tLWDT49gQeQBxwO4B/dC3zAx8y+cgHWEY5kRrHu/bcwueYG OYWLwjk30RnvVGFno48EBRNrCJljRkph08XZfuY4GzYKsjwp5piO8yjUFxdOoBBaOwon lPrg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=l2i4YuMUlvMifEh3T6Lz7Q3wr3VQ+FCqQlCzw4xiZBo=; b=JL51nQLk/2zO+OMv8y/8+mlAln2IO+6flg9U2b8WBW6BqKAMqmHKkcCsv7Xq1O3zKC UJHFV7Gyabbyz36+m/sWkV/Vu8bk2sIjdxHMQDAWP6aXQOpyfPaqSvIZnyAqU3DFo2fq mq1aNmOzfWMvGkfKRZ2Tb203L+tDcozY2agy6b23aLlYr1FGcLour6rMdYNgGrnwHxfc bBuRSxRIpPyvisieTxZuRgctyCH+uJjaYevLfDpy0mIKcO3GKsWq34Fkt9Dy9HdAZtcD LtYZ1mkc2BnvjTAp9AX1WW8P7SlzCHYNfsGzyWaGNVFIQOCFrtwrjjIuVeSswLp8HBzT 3f2g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=XVZvrxt5; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id hq17-20020a1709073f1100b0085c64af3ea9si7602926ejc.510.2023.01.12.13.50.27; Thu, 12 Jan 2023 13:50:53 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=XVZvrxt5; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240923AbjALVsi (ORCPT + 99 others); Thu, 12 Jan 2023 16:48:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40734 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240745AbjALVrg (ORCPT ); Thu, 12 Jan 2023 16:47:36 -0500 Received: from mail-pj1-x102a.google.com (mail-pj1-x102a.google.com [IPv6:2607:f8b0:4864:20::102a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6338A33D69 for ; Thu, 12 Jan 2023 13:40:25 -0800 (PST) Received: by mail-pj1-x102a.google.com with SMTP id w4-20020a17090ac98400b002186f5d7a4cso25060050pjt.0 for ; Thu, 12 Jan 2023 13:40:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=l2i4YuMUlvMifEh3T6Lz7Q3wr3VQ+FCqQlCzw4xiZBo=; b=XVZvrxt5WGZOPzCt8LDpJrjmv8glK3D9y2YETiG2e3W2V/XenVy70B5SsLUdUe6CJ8 6UgFVCsd19ClkBttPp76iWnpqwbRld6K+1rrvX+x9usSAFx1KN7utwRemqy3mMwTmp6i 5cIWZz8dg9F0eAoq/zcIdGiS3VORcbs8cCb4JomvOU9cgHxbaciaubTAB20K8GMEwTQq MJdb8q+EzilmfWBLgqGkIx7KJB29oIvaJikSM88/5xfyS5xqym3zjVixXkS7zTg9eFl7 o7akxkNjflSS2M4HHWEl0/ZiBHnHkOsFAevM0mBz/DuNeGqPICmdmB474dv8ZGASPWIl ryjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=l2i4YuMUlvMifEh3T6Lz7Q3wr3VQ+FCqQlCzw4xiZBo=; b=603cr7NjDr8QFhr9GZGpz1eW5uOp1eXKOUvt4nDfP9UE9oe4oQxoXqvAvaagjU/1ao BDV/ndEHHfwyrc/O9AJRPJUIQ4wwxn7CqaJSDF+QalyjYP3uGR+ELhFtjINLKwgk8IQb cj92fGXLuOxEpfy1lRYD7LA8kamCepEqgVGg2WSVlQRiULgatORZAfbjnk1Xu1dxfNy2 oek9dNxAummpY8AWQVAki/v07t5O+WVEYSRi2ZgfGqzDFZ+LHo+nYQpuWahz02AoxECz KqFGWqILOk4CQQU1Bmsl1FjLZ20LPCsbs8zCX+/kc4IJdxqWm6Ki0OROYqvuk6GWAKLk Umog== X-Gm-Message-State: AFqh2kpWiViZhhl40fW/fZ3fujjueot+b1XOQ2dERgeCApoOLa7auh2A wVECiwG1WzQ0FTuzZG6MXOw= X-Received: by 2002:a05:6a20:aa9f:b0:af:7762:3c29 with SMTP id ck31-20020a056a20aa9f00b000af77623c29mr9348181pzb.10.1673559624771; Thu, 12 Jan 2023 13:40:24 -0800 (PST) Received: from youngsil.svl.corp.google.com ([2620:15c:2d4:203:b71e:23d4:aa6a:47c0]) by smtp.gmail.com with ESMTPSA id q20-20020aa79834000000b0058134d2df41sm11344605pfl.146.2023.01.12.13.40.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 12 Jan 2023 13:40:24 -0800 (PST) Sender: Namhyung Kim From: Namhyung Kim To: Peter Zijlstra Cc: Ingo Molnar , Mark Rutland , Alexander Shishkin , Arnaldo Carvalho de Melo , Jiri Olsa , LKML , Song Liu , linuxppc-dev@lists.ozlabs.org, x86@kernel.org Subject: [PATCH 4/8] perf/core: Add perf_sample_save_brstack() helper Date: Thu, 12 Jan 2023 13:40:11 -0800 Message-Id: <20230112214015.1014857-5-namhyung@kernel.org> X-Mailer: git-send-email 2.39.0.314.g84b9a713c41-goog In-Reply-To: <20230112214015.1014857-1-namhyung@kernel.org> References: <20230112214015.1014857-1-namhyung@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.7 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1754855116534198709?= X-GMAIL-MSGID: =?utf-8?q?1754855116534198709?= When it saves the branch stack to the perf sample data, it needs to update the sample flags and the dynamic size. To make sure this, add the perf_sample_save_brstack() helper and convert all call sites. Cc: linuxppc-dev@lists.ozlabs.org Cc: x86@kernel.org Suggested-by: Peter Zijlstra Signed-off-by: Namhyung Kim --- arch/powerpc/perf/core-book3s.c | 3 +- arch/x86/events/amd/core.c | 6 +-- arch/x86/events/intel/core.c | 6 +-- arch/x86/events/intel/ds.c | 9 ++--- include/linux/perf_event.h | 66 ++++++++++++++++++++------------- kernel/events/core.c | 16 +++----- 6 files changed, 53 insertions(+), 53 deletions(-) diff --git a/arch/powerpc/perf/core-book3s.c b/arch/powerpc/perf/core-book3s.c index bf318dd9b709..8c1f7def596e 100644 --- a/arch/powerpc/perf/core-book3s.c +++ b/arch/powerpc/perf/core-book3s.c @@ -2313,8 +2313,7 @@ static void record_and_restart(struct perf_event *event, unsigned long val, struct cpu_hw_events *cpuhw; cpuhw = this_cpu_ptr(&cpu_hw_events); power_pmu_bhrb_read(event, cpuhw); - data.br_stack = &cpuhw->bhrb_stack; - data.sample_flags |= PERF_SAMPLE_BRANCH_STACK; + perf_sample_save_brstack(&data, event, &cpuhw->bhrb_stack); } if (event->attr.sample_type & PERF_SAMPLE_DATA_SRC && diff --git a/arch/x86/events/amd/core.c b/arch/x86/events/amd/core.c index d6f3703e4119..463f3eb8bbd7 100644 --- a/arch/x86/events/amd/core.c +++ b/arch/x86/events/amd/core.c @@ -928,10 +928,8 @@ static int amd_pmu_v2_handle_irq(struct pt_regs *regs) if (!x86_perf_event_set_period(event)) continue; - if (has_branch_stack(event)) { - data.br_stack = &cpuc->lbr_stack; - data.sample_flags |= PERF_SAMPLE_BRANCH_STACK; - } + if (has_branch_stack(event)) + perf_sample_save_brstack(&data, event, &cpuc->lbr_stack); if (perf_event_overflow(event, &data, regs)) x86_pmu_stop(event, 0); diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c index 29d2d0411caf..14f0a746257d 100644 --- a/arch/x86/events/intel/core.c +++ b/arch/x86/events/intel/core.c @@ -3036,10 +3036,8 @@ static int handle_pmi_common(struct pt_regs *regs, u64 status) perf_sample_data_init(&data, 0, event->hw.last_period); - if (has_branch_stack(event)) { - data.br_stack = &cpuc->lbr_stack; - data.sample_flags |= PERF_SAMPLE_BRANCH_STACK; - } + if (has_branch_stack(event)) + perf_sample_save_brstack(&data, event, &cpuc->lbr_stack); if (perf_event_overflow(event, &data, regs)) x86_pmu_stop(event, 0); diff --git a/arch/x86/events/intel/ds.c b/arch/x86/events/intel/ds.c index 158cf845fc80..07c8a2cdc3ee 100644 --- a/arch/x86/events/intel/ds.c +++ b/arch/x86/events/intel/ds.c @@ -1720,10 +1720,8 @@ static void setup_pebs_fixed_sample_data(struct perf_event *event, data->sample_flags |= PERF_SAMPLE_TIME; } - if (has_branch_stack(event)) { - data->br_stack = &cpuc->lbr_stack; - data->sample_flags |= PERF_SAMPLE_BRANCH_STACK; - } + if (has_branch_stack(event)) + perf_sample_save_brstack(data, event, &cpuc->lbr_stack); } static void adaptive_pebs_save_regs(struct pt_regs *regs, @@ -1883,8 +1881,7 @@ static void setup_pebs_adaptive_sample_data(struct perf_event *event, if (has_branch_stack(event)) { intel_pmu_store_pebs_lbrs(lbr); - data->br_stack = &cpuc->lbr_stack; - data->sample_flags |= PERF_SAMPLE_BRANCH_STACK; + perf_sample_save_brstack(data, event, &cpuc->lbr_stack); } } diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h index 569dfac5887f..7db0e9cc2682 100644 --- a/include/linux/perf_event.h +++ b/include/linux/perf_event.h @@ -1102,6 +1102,31 @@ extern u64 perf_event_read_value(struct perf_event *event, extern struct perf_callchain_entry *perf_callchain(struct perf_event *event, struct pt_regs *regs); +static inline bool branch_sample_no_flags(const struct perf_event *event) +{ + return event->attr.branch_sample_type & PERF_SAMPLE_BRANCH_NO_FLAGS; +} + +static inline bool branch_sample_no_cycles(const struct perf_event *event) +{ + return event->attr.branch_sample_type & PERF_SAMPLE_BRANCH_NO_CYCLES; +} + +static inline bool branch_sample_type(const struct perf_event *event) +{ + return event->attr.branch_sample_type & PERF_SAMPLE_BRANCH_TYPE_SAVE; +} + +static inline bool branch_sample_hw_index(const struct perf_event *event) +{ + return event->attr.branch_sample_type & PERF_SAMPLE_BRANCH_HW_INDEX; +} + +static inline bool branch_sample_priv(const struct perf_event *event) +{ + return event->attr.branch_sample_type & PERF_SAMPLE_BRANCH_PRIV_SAVE; +} + struct perf_sample_data { /* @@ -1210,6 +1235,21 @@ static inline void perf_sample_save_raw_data(struct perf_sample_data *data, data->sample_flags |= PERF_SAMPLE_RAW; } +static inline void perf_sample_save_brstack(struct perf_sample_data *data, + struct perf_event *event, + struct perf_branch_stack *brs) +{ + int size = sizeof(u64); /* nr */ + + if (branch_sample_hw_index(event)) + size += sizeof(u64); + size += brs->nr * sizeof(struct perf_branch_entry); + + data->br_stack = brs; + data->dyn_size += size; + data->sample_flags |= PERF_SAMPLE_BRANCH_STACK; +} + /* * Clear all bitfields in the perf_branch_entry. * The to and from fields are not cleared because they are @@ -1827,30 +1867,4 @@ static inline void perf_lopwr_cb(bool mode) } #endif -#ifdef CONFIG_PERF_EVENTS -static inline bool branch_sample_no_flags(const struct perf_event *event) -{ - return event->attr.branch_sample_type & PERF_SAMPLE_BRANCH_NO_FLAGS; -} - -static inline bool branch_sample_no_cycles(const struct perf_event *event) -{ - return event->attr.branch_sample_type & PERF_SAMPLE_BRANCH_NO_CYCLES; -} - -static inline bool branch_sample_type(const struct perf_event *event) -{ - return event->attr.branch_sample_type & PERF_SAMPLE_BRANCH_TYPE_SAVE; -} - -static inline bool branch_sample_hw_index(const struct perf_event *event) -{ - return event->attr.branch_sample_type & PERF_SAMPLE_BRANCH_HW_INDEX; -} - -static inline bool branch_sample_priv(const struct perf_event *event) -{ - return event->attr.branch_sample_type & PERF_SAMPLE_BRANCH_PRIV_SAVE; -} -#endif /* CONFIG_PERF_EVENTS */ #endif /* _LINUX_PERF_EVENT_H */ diff --git a/kernel/events/core.c b/kernel/events/core.c index 133894ae5e30..0218b6ffaf36 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -7317,7 +7317,7 @@ void perf_output_sample(struct perf_output_handle *handle, } if (sample_type & PERF_SAMPLE_BRANCH_STACK) { - if (data->sample_flags & PERF_SAMPLE_BRANCH_STACK) { + if (data->br_stack) { size_t size; size = data->br_stack->nr @@ -7594,16 +7594,10 @@ void perf_prepare_sample(struct perf_event_header *header, data->sample_flags |= PERF_SAMPLE_RAW; } - if (sample_type & PERF_SAMPLE_BRANCH_STACK) { - int size = sizeof(u64); /* nr */ - if (data->sample_flags & PERF_SAMPLE_BRANCH_STACK) { - if (branch_sample_hw_index(event)) - size += sizeof(u64); - - size += data->br_stack->nr - * sizeof(struct perf_branch_entry); - } - data->dyn_size += size; + if (filtered_sample_type & PERF_SAMPLE_BRANCH_STACK) { + data->br_stack = NULL; + data->dyn_size += sizeof(u64); + data->sample_flags |= PERF_SAMPLE_BRANCH_STACK; } if (sample_type & (PERF_SAMPLE_REGS_USER | PERF_SAMPLE_STACK_USER)) From patchwork Thu Jan 12 21:40:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Namhyung Kim X-Patchwork-Id: 42829 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp4121937wrt; Thu, 12 Jan 2023 13:51:01 -0800 (PST) X-Google-Smtp-Source: AMrXdXtJffZG4bDp2RMMJOmUnxRQZZY7+5FdUQPRgG3iFNrtlWIQquHxqPWarWgE587/tY4o2FYm X-Received: by 2002:a17:907:8e93:b0:7ae:bfec:74c7 with SMTP id tx19-20020a1709078e9300b007aebfec74c7mr71466565ejc.72.1673560261653; Thu, 12 Jan 2023 13:51:01 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1673560261; cv=none; d=google.com; s=arc-20160816; b=OSDFAUJ6xfyW4udy2w/NCLEv+13wCLxphT5idTrZMyiVmRl3moAxUr60RFCxL0j8lY DuRDrrgWL8V/heEbI9CGz+ljkB3MutiZknbyADDbamjfCPuItDrpPeXs4iK5LeYbjoP7 9mnQXTDOIjaXaHrW5VJk/0lf0+uWemQp2wNqqAkwTJX4nOKm+92UXk+W6IkrwRIcZUxm ZRgXCiOo5CdDal61+dM/oTs+H5D7c4MwledpnABCzTEi+w/yjx7nTBkprFiH9yme1iME LpPfMnSrIOjlmnEr2VeoXBEylLMV+Xdo2QexYbKC31udcrN9ycV3dN8B2l6TlHS8DJKl l4hA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=jvGUMU//A5tX7oB/uY4gCyETkJGZoTtytTEaxGlNFkw=; b=tflbGXHG3FvzLIyy3Cb2LLgBW9a1fgfF8pj2pOa2tmD9hbB7/UZ7uKvqITQ1KOiodD tfYTsZsqCd9eeD3eO7h8gZ6u5a0FYr786Hj2tpbHCNEZB37WPhw7ZEd7/OwdmKer1Ovo V90tVZR2Yw8g6EKqaVrfHOXiTsmC6Rc2hb9Et5GhAlK6zrRIvMxKjbZHlOljSA+RoWL9 9a1UW5zTyHWrdBqZW8ZdDVQktWnJJ7AvD45ra9TDj6zgFw7pvPUhZ43SDSrPXVfcaNwt g9sBWOa33GQ0oAWmYdnQOsty+d1c3mZU4W70LKEmb6DFeVTBg0Japa4IDDhd0hfMa5CL 4Bew== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=DbtSTDgD; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id go38-20020a1709070da600b0085224016f6dsi10632619ejc.242.2023.01.12.13.50.35; Thu, 12 Jan 2023 13:51:01 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=DbtSTDgD; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240997AbjALVsm (ORCPT + 99 others); Thu, 12 Jan 2023 16:48:42 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40472 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240737AbjALVrg (ORCPT ); Thu, 12 Jan 2023 16:47:36 -0500 Received: from mail-pg1-x536.google.com (mail-pg1-x536.google.com [IPv6:2607:f8b0:4864:20::536]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AD34834D77 for ; Thu, 12 Jan 2023 13:40:26 -0800 (PST) Received: by mail-pg1-x536.google.com with SMTP id g68so12597896pgc.11 for ; Thu, 12 Jan 2023 13:40:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=jvGUMU//A5tX7oB/uY4gCyETkJGZoTtytTEaxGlNFkw=; b=DbtSTDgDJTxKl+2n9fo09NLB/opC1JK/LEyqOK7lOQJv64J2oR1FoFfaFVtpS1s2/l wQSna2mI4B2LnQ8fuwmbeuHxnQQxpgdpQDHk3TSPSpd8LspQPFAQkHRubGw2Et/VbJGX MJJIA99ZwD4ZDb9hYQbjLlegbF54V93r0KFNykKo93ysNzjvUdjTeIs9ihmDy0yonkYv 0mnqYlRg87mYWV+N5Hgu3e2cCN4kmdosUGtjoENP0LtvK12aI1MUDYXhpR4YPK1zlSRV YFZS3MYhwbjeBIVoUvWrG8auaWZNmjYiQpvZvNb/LiVUZoQ+hmycMCeP0Haj2e75WQ6x dDpQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=jvGUMU//A5tX7oB/uY4gCyETkJGZoTtytTEaxGlNFkw=; b=KfV857IvupEjxmyx9IOqdR5Eq1MpZljaeMiKv3KEXGp0o4iFd7gC6pn0EQ1OHlvNpx 9yJo6/xHHWW4lllHsibv0j4npt/cLUjXIaAuaQ1EUlZY4nS8dQGHrG9rhP9zUL1AL+un zRamAAN81lKBT1QCnWde4s1Ux5BvhXMiSudulxoZ5eF+AEg7HWkQQKZRXKOx6Fch//c+ TTd97ThglJWVmwB+LHNjpxEh394xED1Pmpv5xAjryJx4iT8tETu61RTA/I8KtkWD22P7 FktrEGDvbGRjSuWPXdRR9R4gPYJh2IYfg8UNm/ro7CAuo6zNfKgpp6GiPzSIBF83nCKV wV9w== X-Gm-Message-State: AFqh2krgBehD8mn90laFjE7WrAuT/jxRQ0GiKEkjMJIe63eL/iDIjT0B LJK70kVgEC7S7lByEoGodU2uv6TWSO4= X-Received: by 2002:a62:6d07:0:b0:581:b3f4:21f8 with SMTP id i7-20020a626d07000000b00581b3f421f8mr49054158pfc.31.1673559626075; Thu, 12 Jan 2023 13:40:26 -0800 (PST) Received: from youngsil.svl.corp.google.com ([2620:15c:2d4:203:b71e:23d4:aa6a:47c0]) by smtp.gmail.com with ESMTPSA id q20-20020aa79834000000b0058134d2df41sm11344605pfl.146.2023.01.12.13.40.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 12 Jan 2023 13:40:25 -0800 (PST) Sender: Namhyung Kim From: Namhyung Kim To: Peter Zijlstra Cc: Ingo Molnar , Mark Rutland , Alexander Shishkin , Arnaldo Carvalho de Melo , Jiri Olsa , LKML , Song Liu Subject: [PATCH 5/8] perf/core: Set data->sample_flags in perf_prepare_sample() Date: Thu, 12 Jan 2023 13:40:12 -0800 Message-Id: <20230112214015.1014857-6-namhyung@kernel.org> X-Mailer: git-send-email 2.39.0.314.g84b9a713c41-goog In-Reply-To: <20230112214015.1014857-1-namhyung@kernel.org> References: <20230112214015.1014857-1-namhyung@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.7 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1754855125207131810?= X-GMAIL-MSGID: =?utf-8?q?1754855125207131810?= The perf_prepare_sample() sets the perf_sample_data according to the attr->sample_type before copying it to the ring buffer. But BPF also wants to access the sample data so it needs to prepare the sample even before the regular path. That means the perf_prepare_sample() can be called more than once. Set the data->sample_flags consistently so that it can indicate which fields are set already and skip them if sets. Also update the filtered_sample_type to have the dependent flags to reduce the number of branches. Suggested-by: Peter Zijlstra Signed-off-by: Namhyung Kim --- kernel/events/core.c | 85 +++++++++++++++++++++++++++++++++----------- 1 file changed, 65 insertions(+), 20 deletions(-) diff --git a/kernel/events/core.c b/kernel/events/core.c index 0218b6ffaf36..7c5555051662 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -7053,6 +7053,14 @@ static void perf_aux_sample_output(struct perf_event *event, ring_buffer_put(rb); } +/* + * A set of common sample data types saved even for non-sample records + * when event->attr.sample_id_all is set. + */ +#define PERF_SAMPLE_ID_ALL (PERF_SAMPLE_TID | PERF_SAMPLE_TIME | \ + PERF_SAMPLE_CPU | PERF_SAMPLE_ID | \ + PERF_SAMPLE_IDENTIFIER | PERF_SAMPLE_STREAM_ID) + static void __perf_event_header__init_id(struct perf_event_header *header, struct perf_sample_data *data, struct perf_event *event, @@ -7561,6 +7569,11 @@ perf_callchain(struct perf_event *event, struct pt_regs *regs) return callchain ?: &__empty_callchain; } +static __always_inline u64 __cond_set(u64 flags, u64 s, u64 d) +{ + return d * !!(flags & s); +} + void perf_prepare_sample(struct perf_event_header *header, struct perf_sample_data *data, struct perf_event *event, @@ -7576,14 +7589,25 @@ void perf_prepare_sample(struct perf_event_header *header, header->misc |= perf_misc_flags(regs); /* - * Clear the sample flags that have already been done by the - * PMU driver. + * Add the sample flags that are dependent to others. And clear the + * sample flags that have already been done by the PMU driver. */ - filtered_sample_type = sample_type & ~data->sample_flags; + filtered_sample_type = sample_type; + filtered_sample_type |= __cond_set(sample_type, PERF_SAMPLE_CODE_PAGE_SIZE, + PERF_SAMPLE_IP); + filtered_sample_type |= __cond_set(sample_type, PERF_SAMPLE_DATA_PAGE_SIZE | + PERF_SAMPLE_PHYS_ADDR, PERF_SAMPLE_ADDR); + filtered_sample_type |= __cond_set(sample_type, PERF_SAMPLE_STACK_USER, + PERF_SAMPLE_REGS_USER); + filtered_sample_type &= ~data->sample_flags; + __perf_event_header__init_id(header, data, event, filtered_sample_type); + data->sample_flags |= sample_type & PERF_SAMPLE_ID_ALL; - if (sample_type & (PERF_SAMPLE_IP | PERF_SAMPLE_CODE_PAGE_SIZE)) + if (filtered_sample_type & PERF_SAMPLE_IP) { data->ip = perf_instruction_pointer(regs); + data->sample_flags |= PERF_SAMPLE_IP; + } if (filtered_sample_type & PERF_SAMPLE_CALLCHAIN) perf_sample_save_callchain(data, event, regs); @@ -7600,10 +7624,15 @@ void perf_prepare_sample(struct perf_event_header *header, data->sample_flags |= PERF_SAMPLE_BRANCH_STACK; } - if (sample_type & (PERF_SAMPLE_REGS_USER | PERF_SAMPLE_STACK_USER)) + if (filtered_sample_type & PERF_SAMPLE_REGS_USER) perf_sample_regs_user(&data->regs_user, regs); - if (sample_type & PERF_SAMPLE_REGS_USER) { + /* + * It cannot use the filtered_sample_type here as REGS_USER can be set + * by STACK_USER (using __cond_set() above) and we don't want to update + * the dyn_size if it's not requested by users. + */ + if ((sample_type & ~data->sample_flags) & PERF_SAMPLE_REGS_USER) { /* regs dump ABI info */ int size = sizeof(u64); @@ -7613,9 +7642,10 @@ void perf_prepare_sample(struct perf_event_header *header, } data->dyn_size += size; + data->sample_flags |= PERF_SAMPLE_REGS_USER; } - if (sample_type & PERF_SAMPLE_STACK_USER) { + if (filtered_sample_type & PERF_SAMPLE_STACK_USER) { /* * Either we need PERF_SAMPLE_STACK_USER bit to be always * processed as the last one or have additional check added @@ -7638,23 +7668,30 @@ void perf_prepare_sample(struct perf_event_header *header, data->stack_user_size = stack_size; data->dyn_size += size; + data->sample_flags |= PERF_SAMPLE_STACK_USER; } - if (filtered_sample_type & PERF_SAMPLE_WEIGHT_TYPE) + if (filtered_sample_type & PERF_SAMPLE_WEIGHT_TYPE) { data->weight.full = 0; + data->sample_flags |= PERF_SAMPLE_WEIGHT_TYPE; + } - if (filtered_sample_type & PERF_SAMPLE_DATA_SRC) + if (filtered_sample_type & PERF_SAMPLE_DATA_SRC) { data->data_src.val = PERF_MEM_NA; + data->sample_flags |= PERF_SAMPLE_DATA_SRC; + } - if (filtered_sample_type & PERF_SAMPLE_TRANSACTION) + if (filtered_sample_type & PERF_SAMPLE_TRANSACTION) { data->txn = 0; + data->sample_flags |= PERF_SAMPLE_TRANSACTION; + } - if (sample_type & (PERF_SAMPLE_ADDR | PERF_SAMPLE_PHYS_ADDR | PERF_SAMPLE_DATA_PAGE_SIZE)) { - if (filtered_sample_type & PERF_SAMPLE_ADDR) - data->addr = 0; + if (filtered_sample_type & PERF_SAMPLE_ADDR) { + data->addr = 0; + data->sample_flags |= PERF_SAMPLE_ADDR; } - if (sample_type & PERF_SAMPLE_REGS_INTR) { + if (filtered_sample_type & PERF_SAMPLE_REGS_INTR) { /* regs dump ABI info */ int size = sizeof(u64); @@ -7667,19 +7704,22 @@ void perf_prepare_sample(struct perf_event_header *header, } data->dyn_size += size; + data->sample_flags |= PERF_SAMPLE_REGS_INTR; } - if (sample_type & PERF_SAMPLE_PHYS_ADDR && - filtered_sample_type & PERF_SAMPLE_PHYS_ADDR) + if (filtered_sample_type & PERF_SAMPLE_PHYS_ADDR) { data->phys_addr = perf_virt_to_phys(data->addr); + data->sample_flags |= PERF_SAMPLE_PHYS_ADDR; + } #ifdef CONFIG_CGROUP_PERF - if (sample_type & PERF_SAMPLE_CGROUP) { + if (filtered_sample_type & PERF_SAMPLE_CGROUP) { struct cgroup *cgrp; /* protected by RCU */ cgrp = task_css_check(current, perf_event_cgrp_id, 1)->cgroup; data->cgroup = cgroup_id(cgrp); + data->sample_flags |= PERF_SAMPLE_CGROUP; } #endif @@ -7688,13 +7728,17 @@ void perf_prepare_sample(struct perf_event_header *header, * require PERF_SAMPLE_ADDR, kernel implicitly retrieve the data->addr, * but the value will not dump to the userspace. */ - if (sample_type & PERF_SAMPLE_DATA_PAGE_SIZE) + if (filtered_sample_type & PERF_SAMPLE_DATA_PAGE_SIZE) { data->data_page_size = perf_get_page_size(data->addr); + data->sample_flags |= PERF_SAMPLE_DATA_PAGE_SIZE; + } - if (sample_type & PERF_SAMPLE_CODE_PAGE_SIZE) + if (filtered_sample_type & PERF_SAMPLE_CODE_PAGE_SIZE) { data->code_page_size = perf_get_page_size(data->ip); + data->sample_flags |= PERF_SAMPLE_CODE_PAGE_SIZE; + } - if (sample_type & PERF_SAMPLE_AUX) { + if (filtered_sample_type & PERF_SAMPLE_AUX) { u64 size; header->size += sizeof(u64); /* size */ @@ -7712,6 +7756,7 @@ void perf_prepare_sample(struct perf_event_header *header, WARN_ON_ONCE(size + header->size > U16_MAX); data->dyn_size += size + sizeof(u64); /* size above */ + data->sample_flags |= PERF_SAMPLE_AUX; } header->size += data->dyn_size; From patchwork Thu Jan 12 21:40:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Namhyung Kim X-Patchwork-Id: 42833 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp4122063wrt; Thu, 12 Jan 2023 13:51:19 -0800 (PST) X-Google-Smtp-Source: AMrXdXs8tmJyVTwBNACmVpR+P0wkqbo80YOORvhEJ2YG78kSQ2jbcrn8J9ZDD4YZwoaQOgGvcbOq X-Received: by 2002:a17:906:3e58:b0:7c0:fa2c:fc9b with SMTP id t24-20020a1709063e5800b007c0fa2cfc9bmr877941eji.55.1673560279223; Thu, 12 Jan 2023 13:51:19 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1673560279; cv=none; d=google.com; s=arc-20160816; b=tdqEv1JbU76bKW3De4XIBW7koDwfKYEtpLrAH/mEVXjl1HBgKzZnACJCHQ2+WPSpgi Ul+iAFwSeTykB7pd2GcK88G7ep7I8HVF8ZAnLGTLoYGb9WBgGyLY3Sts5Vb5iPPiRWoq Rkw3dzJQcbZVyII+g5TL3ZTuJ7V2kvdpe7zRQa3uUAmWercGpObU6ZyzyoAFNFxXZjMR JQ748FsFsd4sZ+ii1p2Aiy9QWw6yxpbBMTUpKXTD73nOBbQtwstDdqw6YZ4R284SIaYY 3bmpyx7w5Oub8RmbGd75GWORmFeh5woNcGpRONgDIPB0VdCL2R2t5GsefbkOcTwUF6Lu doQA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=N4u1T86SPW7EoRBLC3mtnjrxg1wEFO/gvDqxCBagRGw=; b=1BunP9KrOXEtq7JX3mniVbNwkV/+vW+q9gUyBn4ygZ72hVRfePCmhfgeUa7flyB7hB Tsa4CSApUcNXVF3WQe/7q4Z9bFd23MPAQfTabfzAWthOstMjYlwv/urSCdDGzowH6h5g lQNbP/OPwXlL7ksDRTD8mkBBvvbbqPGdQj4NYt53IafLnwChHyKAat0AaiuX3nG7jopN VjrjiHlPS5pwsMLbHPY80p+FpqxLOmNWm5+VTwXtJ7yh73C+FBChk1ljCuQ7Hh3UjXjG bNEBl3/APIsJ7ozDjwtWIIT0eZDb8l9ZDddbSqx060GREoEBDM5p8bk5n2cQbZAC4ZO0 89KA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=g2boBC2B; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id sh15-20020a1709076e8f00b008379ac04127si19782651ejc.199.2023.01.12.13.50.51; Thu, 12 Jan 2023 13:51:19 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=g2boBC2B; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230230AbjALVs7 (ORCPT + 99 others); Thu, 12 Jan 2023 16:48:59 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40740 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240753AbjALVrg (ORCPT ); Thu, 12 Jan 2023 16:47:36 -0500 Received: from mail-pg1-x52c.google.com (mail-pg1-x52c.google.com [IPv6:2607:f8b0:4864:20::52c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C14C01B9E1 for ; Thu, 12 Jan 2023 13:40:27 -0800 (PST) Received: by mail-pg1-x52c.google.com with SMTP id 78so13695745pgb.8 for ; Thu, 12 Jan 2023 13:40:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=N4u1T86SPW7EoRBLC3mtnjrxg1wEFO/gvDqxCBagRGw=; b=g2boBC2BF//Gt7s5eM0DpLOOf9GKDmw9dCXbKCVxn3Rc4ySTwhTOPDbYgzES96s/Ag Qy3QsA6+/d8B2HP2JNt5KCQK2yQAZXlbfIukJvdVSauX7qTxSWhtJSOD2dq+DmNy3ozc QVxY2k4m41lsnwWK6L3iwKL/50X1rA5WaGeDcKqWWb2gDtaMzdTaLnT7E3IGoCVQET6d i4fa6U+w93EnCxuIVWLPwfjzt0z62OJzRAyPEoghDGA90o5P6k9x4hXSmtXL37H13DZI 2+f0862alHk4m6hq0TaVdDma7By0OZr8vRac1Au8SEMUqJ6DyEPuQSo9TGATh1IbvcgN andA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=N4u1T86SPW7EoRBLC3mtnjrxg1wEFO/gvDqxCBagRGw=; b=nbOtx0/izdqL8J+U+8KknUOUOfFLcb4BZqZ/om5nOuaYZB/cwpCBic5GjvBO63u5i9 QU22P3v5vA9dZw6uyZeAuAcKeRYugFwG38/WAX86iMncaBbIyUWoZ+GRGTzEsOzkuvcH mAcBn2Q9DbgSjRBlkraJk1Jcljp8//5O+SSbYYUa6E1QZpojoWJPxiAeTB5Euq99mskA flOPze+iv/LYizMQXu6undUfcnOmiymr0TaGuloB6heVMQ/PRlpg1ai236Xwd2Sapxr6 czr6m0jV65OOpU0QIaGwS0iHZpT0OgG+sA7DFhTOQz6zX63Bgb5megXQfayT1hToAagt D3/g== X-Gm-Message-State: AFqh2kqk/0tHn1GtyiprZzN6muTnh+pupbBq0SlTueE363bQNPwEPSD4 2o5aTZj7g+uhqUOQKoajtSg= X-Received: by 2002:aa7:8649:0:b0:583:219a:670e with SMTP id a9-20020aa78649000000b00583219a670emr25677667pfo.30.1673559627191; Thu, 12 Jan 2023 13:40:27 -0800 (PST) Received: from youngsil.svl.corp.google.com ([2620:15c:2d4:203:b71e:23d4:aa6a:47c0]) by smtp.gmail.com with ESMTPSA id q20-20020aa79834000000b0058134d2df41sm11344605pfl.146.2023.01.12.13.40.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 12 Jan 2023 13:40:26 -0800 (PST) Sender: Namhyung Kim From: Namhyung Kim To: Peter Zijlstra Cc: Ingo Molnar , Mark Rutland , Alexander Shishkin , Arnaldo Carvalho de Melo , Jiri Olsa , LKML , Song Liu Subject: [PATCH 6/8] perf/core: Do not pass header for sample id init Date: Thu, 12 Jan 2023 13:40:13 -0800 Message-Id: <20230112214015.1014857-7-namhyung@kernel.org> X-Mailer: git-send-email 2.39.0.314.g84b9a713c41-goog In-Reply-To: <20230112214015.1014857-1-namhyung@kernel.org> References: <20230112214015.1014857-1-namhyung@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.7 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1754855143541870089?= X-GMAIL-MSGID: =?utf-8?q?1754855143541870089?= The only thing it does for header in __perf_event_header__init_id() is to update the header size with event->id_header_size. We can do this outside and get rid of the argument for the later change. Signed-off-by: Namhyung Kim Acked-by: Song Liu --- kernel/events/core.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/kernel/events/core.c b/kernel/events/core.c index 7c5555051662..127dddd20f93 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -7061,13 +7061,11 @@ static void perf_aux_sample_output(struct perf_event *event, PERF_SAMPLE_CPU | PERF_SAMPLE_ID | \ PERF_SAMPLE_IDENTIFIER | PERF_SAMPLE_STREAM_ID) -static void __perf_event_header__init_id(struct perf_event_header *header, - struct perf_sample_data *data, +static void __perf_event_header__init_id(struct perf_sample_data *data, struct perf_event *event, u64 sample_type) { data->type = event->attr.sample_type; - header->size += event->id_header_size; if (sample_type & PERF_SAMPLE_TID) { /* namespace issues */ @@ -7094,8 +7092,10 @@ void perf_event_header__init_id(struct perf_event_header *header, struct perf_sample_data *data, struct perf_event *event) { - if (event->attr.sample_id_all) - __perf_event_header__init_id(header, data, event, event->attr.sample_type); + if (event->attr.sample_id_all) { + header->size += event->id_header_size; + __perf_event_header__init_id(data, event, event->attr.sample_type); + } } static void __perf_event__output_id_sample(struct perf_output_handle *handle, @@ -7583,7 +7583,7 @@ void perf_prepare_sample(struct perf_event_header *header, u64 filtered_sample_type; header->type = PERF_RECORD_SAMPLE; - header->size = sizeof(*header) + event->header_size; + header->size = sizeof(*header) + event->header_size + event->id_header_size; header->misc = 0; header->misc |= perf_misc_flags(regs); @@ -7601,7 +7601,7 @@ void perf_prepare_sample(struct perf_event_header *header, PERF_SAMPLE_REGS_USER); filtered_sample_type &= ~data->sample_flags; - __perf_event_header__init_id(header, data, event, filtered_sample_type); + __perf_event_header__init_id(data, event, filtered_sample_type); data->sample_flags |= sample_type & PERF_SAMPLE_ID_ALL; if (filtered_sample_type & PERF_SAMPLE_IP) { From patchwork Thu Jan 12 21:40:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Namhyung Kim X-Patchwork-Id: 42830 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp4121963wrt; Thu, 12 Jan 2023 13:51:05 -0800 (PST) X-Google-Smtp-Source: AMrXdXvVs9/tILb3CQLC+6KmUocEioly/JgsE0CMVVAlVY+1IXofsc7Ett2H0IZ2yaC6Gmss3c8X X-Received: by 2002:a17:907:c018:b0:7c1:bb3:45e4 with SMTP id ss24-20020a170907c01800b007c10bb345e4mr65658624ejc.21.1673560264905; Thu, 12 Jan 2023 13:51:04 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1673560264; cv=none; d=google.com; s=arc-20160816; b=R8ybth3ZE/KYOYCd5kyfNcMjKK/9PPePBlO+rStKvYbArQb0FcqdcDZNue+tFxpVqb hclVRIPtKBq9R8tGGHK/lznDEJNZ3isdGmMx32fBPI+u9QxlomVxs+XzmeySw9AeWMCq 3hWioCa1W1JmA2UEfYe4Dw65q6KFIlgPrSpHOmwOYcc9hbK8CtMLqFVkF+upiqKDjQqp Up67+5EDje6LPT96b8eMGTSjhlloII0vGeVye+GdGWHjqDDZUMRlkjiMc8aAvIoVooki s3gwuK2/S46N2qPqeuwvO4IeUk9AhHzVdy7+Yxz4tY7L/NuwRNbN2QNmXo1YWtj5aVwh uWJg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=cFiEc4fnXcWosSJZXgNQt+xQQqIdoE277uZe1x+iaq4=; b=Q4dVzaEDC8lFeqWwGfjdcoA3HNZD0j9MOTjx1idLiw26TBSZ/Z47fYoHTbUGKPtOQH sNHco045M8S5QFtJ+z2ALTNSAeEBQGkzIsiBYfYSfaSEMvat6CZiHwyy2bfS6zQ2vdGh cAeSSLhnF3n0mK2ACglboSrkqQrZo0B5R9frGq3A5y0XIaYO6IfIYN7d2JDXj/9YSNdq AOfxv6HGXB3C+7q8G0XbdJgSkudRYwrdo1atQ4JdIn3lnSAVHJISsL2RUDlJS0Da006c OUJsjwh4Gs7/2HK85Te4N+0+SiUhLgnYxlReQMaXfKbsYnYta626ltR7HCozJuhiz5Dj SpdQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=bPETWCW5; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id mp3-20020a1709071b0300b007ba49ec6d3bsi20937649ejc.390.2023.01.12.13.50.37; Thu, 12 Jan 2023 13:51:04 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=bPETWCW5; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230298AbjALVs4 (ORCPT + 99 others); Thu, 12 Jan 2023 16:48:56 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40492 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240757AbjALVrg (ORCPT ); Thu, 12 Jan 2023 16:47:36 -0500 Received: from mail-pf1-x429.google.com (mail-pf1-x429.google.com [IPv6:2607:f8b0:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 17E231B9F3; Thu, 12 Jan 2023 13:40:29 -0800 (PST) Received: by mail-pf1-x429.google.com with SMTP id 20so9371786pfu.13; Thu, 12 Jan 2023 13:40:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=cFiEc4fnXcWosSJZXgNQt+xQQqIdoE277uZe1x+iaq4=; b=bPETWCW5IbBSYQyy6jMD+fldm74bgEgo3vBdHRXKU3fnmlEGP0/eT6B0u73M/2AaA8 IyHPPe4jHkK0Mqa+tZkgNBza/2P4NC6Gs+eRQbyk/BP4gXjedz4eShGS743BR2XKU8lJ o7gjuCg2rtuHjcWL/U75YpXufj1FHXlQWL/oAQtd9HqbWs1Ta3iy7sV184SDWfXKIp/O HZvKzFb3XErNYlGNTIkXj6qNYEbh8jGJVIlb6G1Es9Ay7KvwaKzA1h46lFeFd4tuIVFj dGu1fC7IMoBpIbuqTqFrL4o6DJZJwAZCPZ4mnJ7Dw6ioHTKeQNCX4V70AMNzlsKPHMD+ 5lUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=cFiEc4fnXcWosSJZXgNQt+xQQqIdoE277uZe1x+iaq4=; b=edBeMQHaJQEbrIDYmow1C+o8Gxv77S7kcr0Z02IHyMsZOl+rxQ8Kbj1hwpMt0XX67v 6mtHCadfLkTfYHmGltT2/52duYYGQB9AAHeryI8vPNRUdw/VzuZp3Kz1YvARsTJcFidl yNvCeAGTWGi6j210zynVmuW7CqxZLilP/Q/1v+zPyl+2UeCsWTtvXfVDhohPaXAdkKCL l0h1dJEHYq2jMusrpcLcwSQ+EY9vJ9FsvCiwAeiG0Hym03DPoLT9r0MMknRy9uLpk4ue lPJ7cSGPB3Cn2CwGiI09WdLt0gV6oSi4z4aS1vUa4+1YK602ygWwVNCrPVqnl8Z0WDOd vBRA== X-Gm-Message-State: AFqh2kqGS4FSTLWzk+Zcj5fJT49UR9K7i8+ULho25kIKhzkGNBVNtzql wKv1bbuBOE+sQcyhb/FW6Zc= X-Received: by 2002:aa7:9732:0:b0:580:d722:10e7 with SMTP id k18-20020aa79732000000b00580d72210e7mr71908290pfg.24.1673559628420; Thu, 12 Jan 2023 13:40:28 -0800 (PST) Received: from youngsil.svl.corp.google.com ([2620:15c:2d4:203:b71e:23d4:aa6a:47c0]) by smtp.gmail.com with ESMTPSA id q20-20020aa79834000000b0058134d2df41sm11344605pfl.146.2023.01.12.13.40.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 12 Jan 2023 13:40:28 -0800 (PST) Sender: Namhyung Kim From: Namhyung Kim To: Peter Zijlstra Cc: Ingo Molnar , Mark Rutland , Alexander Shishkin , Arnaldo Carvalho de Melo , Jiri Olsa , LKML , Song Liu , linux-s390@vger.kernel.org, x86@kernel.org Subject: [PATCH 7/8] perf/core: Introduce perf_prepare_header() Date: Thu, 12 Jan 2023 13:40:14 -0800 Message-Id: <20230112214015.1014857-8-namhyung@kernel.org> X-Mailer: git-send-email 2.39.0.314.g84b9a713c41-goog In-Reply-To: <20230112214015.1014857-1-namhyung@kernel.org> References: <20230112214015.1014857-1-namhyung@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.7 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1754855128594691958?= X-GMAIL-MSGID: =?utf-8?q?1754855128594691958?= Factor out perf_prepare_header() so that it can call perf_prepare_sample() without a header if not needed. Also it checks the filtered_sample_type to avoid duplicate work when perf_prepare_sample() is called twice (or more). Cc: linux-s390@vger.kernel.org Cc: x86@kernel.org Suggested-by: Peter Zijlstr Signed-off-by: Namhyung Kim Acked-by: Song Liu --- arch/s390/kernel/perf_cpum_sf.c | 3 ++- arch/x86/events/intel/ds.c | 3 ++- include/linux/perf_event.h | 16 +++++++++++++- kernel/events/core.c | 38 +++++++++++++++++++++------------ 4 files changed, 43 insertions(+), 17 deletions(-) diff --git a/arch/s390/kernel/perf_cpum_sf.c b/arch/s390/kernel/perf_cpum_sf.c index 332a49965130..fd02f8423243 100644 --- a/arch/s390/kernel/perf_cpum_sf.c +++ b/arch/s390/kernel/perf_cpum_sf.c @@ -671,7 +671,8 @@ static void cpumsf_output_event_pid(struct perf_event *event, /* Protect callchain buffers, tasks */ rcu_read_lock(); - perf_prepare_sample(&header, data, event, regs); + perf_prepare_sample(data, event, regs); + perf_prepare_header(&header, data, event, regs); if (perf_output_begin(&handle, data, event, header.size)) goto out; diff --git a/arch/x86/events/intel/ds.c b/arch/x86/events/intel/ds.c index 07c8a2cdc3ee..183efa914b99 100644 --- a/arch/x86/events/intel/ds.c +++ b/arch/x86/events/intel/ds.c @@ -807,7 +807,8 @@ int intel_pmu_drain_bts_buffer(void) * the sample. */ rcu_read_lock(); - perf_prepare_sample(&header, &data, event, ®s); + perf_prepare_sample(&data, event, ®s); + perf_prepare_header(&header, &data, event, ®s); if (perf_output_begin(&handle, &data, event, header.size * (top - base - skip))) diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h index 7db0e9cc2682..d5628a7b5eaa 100644 --- a/include/linux/perf_event.h +++ b/include/linux/perf_event.h @@ -1250,6 +1250,17 @@ static inline void perf_sample_save_brstack(struct perf_sample_data *data, data->sample_flags |= PERF_SAMPLE_BRANCH_STACK; } +static inline u32 perf_sample_data_size(struct perf_sample_data *data, + struct perf_event *event) +{ + u32 size = sizeof(struct perf_event_header); + + size += event->header_size + event->id_header_size; + size += data->dyn_size; + + return size; +} + /* * Clear all bitfields in the perf_branch_entry. * The to and from fields are not cleared because they are @@ -1271,7 +1282,10 @@ extern void perf_output_sample(struct perf_output_handle *handle, struct perf_event_header *header, struct perf_sample_data *data, struct perf_event *event); -extern void perf_prepare_sample(struct perf_event_header *header, +extern void perf_prepare_sample(struct perf_sample_data *data, + struct perf_event *event, + struct pt_regs *regs); +extern void perf_prepare_header(struct perf_event_header *header, struct perf_sample_data *data, struct perf_event *event, struct pt_regs *regs); diff --git a/kernel/events/core.c b/kernel/events/core.c index 127dddd20f93..5c4f3fa3d2b7 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -7574,20 +7574,13 @@ static __always_inline u64 __cond_set(u64 flags, u64 s, u64 d) return d * !!(flags & s); } -void perf_prepare_sample(struct perf_event_header *header, - struct perf_sample_data *data, +void perf_prepare_sample(struct perf_sample_data *data, struct perf_event *event, struct pt_regs *regs) { u64 sample_type = event->attr.sample_type; u64 filtered_sample_type; - header->type = PERF_RECORD_SAMPLE; - header->size = sizeof(*header) + event->header_size + event->id_header_size; - - header->misc = 0; - header->misc |= perf_misc_flags(regs); - /* * Add the sample flags that are dependent to others. And clear the * sample flags that have already been done by the PMU driver. @@ -7601,6 +7594,12 @@ void perf_prepare_sample(struct perf_event_header *header, PERF_SAMPLE_REGS_USER); filtered_sample_type &= ~data->sample_flags; + if (filtered_sample_type == 0) { + /* Make sure it has the correct data->type for output */ + data->type = event->attr.sample_type; + return; + } + __perf_event_header__init_id(data, event, filtered_sample_type); data->sample_flags |= sample_type & PERF_SAMPLE_ID_ALL; @@ -7653,9 +7652,10 @@ void perf_prepare_sample(struct perf_event_header *header, * up the rest of the sample size. */ u16 stack_size = event->attr.sample_stack_user; + u16 header_size = perf_sample_data_size(data, event); u16 size = sizeof(u64); - stack_size = perf_sample_ustack_size(stack_size, header->size, + stack_size = perf_sample_ustack_size(stack_size, header_size, data->regs_user.regs); /* @@ -7740,8 +7740,9 @@ void perf_prepare_sample(struct perf_event_header *header, if (filtered_sample_type & PERF_SAMPLE_AUX) { u64 size; + u16 header_size = perf_sample_data_size(data, event); - header->size += sizeof(u64); /* size */ + header_size += sizeof(u64); /* size */ /* * Given the 16bit nature of header::size, an AUX sample can @@ -7749,17 +7750,25 @@ void perf_prepare_sample(struct perf_event_header *header, * Make sure this doesn't happen by using up to U16_MAX bytes * per sample in total (rounded down to 8 byte boundary). */ - size = min_t(size_t, U16_MAX - header->size, + size = min_t(size_t, U16_MAX - header_size, event->attr.aux_sample_size); size = rounddown(size, 8); size = perf_prepare_sample_aux(event, data, size); - WARN_ON_ONCE(size + header->size > U16_MAX); + WARN_ON_ONCE(size + header_size > U16_MAX); data->dyn_size += size + sizeof(u64); /* size above */ data->sample_flags |= PERF_SAMPLE_AUX; } +} - header->size += data->dyn_size; +void perf_prepare_header(struct perf_event_header *header, + struct perf_sample_data *data, + struct perf_event *event, + struct pt_regs *regs) +{ + header->type = PERF_RECORD_SAMPLE; + header->size = perf_sample_data_size(data, event); + header->misc = perf_misc_flags(regs); /* * If you're adding more sample types here, you likely need to do @@ -7788,7 +7797,8 @@ __perf_event_output(struct perf_event *event, /* protect the callchain buffers */ rcu_read_lock(); - perf_prepare_sample(&header, data, event, regs); + perf_prepare_sample(data, event, regs); + perf_prepare_header(&header, data, event, regs); err = output_begin(&handle, data, event, header.size); if (err) From patchwork Thu Jan 12 21:40:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Namhyung Kim X-Patchwork-Id: 42834 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp4122076wrt; Thu, 12 Jan 2023 13:51:21 -0800 (PST) X-Google-Smtp-Source: AMrXdXvuTNGJdBL6vP0JoZm5EyTv1979324tsjnsiHnKK2V1RLlrKneX2m+nPydLBCjgyiCQ8ePA X-Received: by 2002:a05:6402:c44:b0:48e:ac4e:7bfa with SMTP id cs4-20020a0564020c4400b0048eac4e7bfamr15264316edb.2.1673560281429; Thu, 12 Jan 2023 13:51:21 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1673560281; cv=none; d=google.com; s=arc-20160816; b=fiXIG3z0fau+fadAhSmMz66S5BIAAtNVrAENbJh7c8zwp//8iuJm3ekAI+E4Er6l1t 8hEdkV81f/8FNM4cV+79ic3aTs1TBFBGCgzHazpsXBnF6l9S3qJ0zOQhmFZ7DBDXsOVv mfBAoureXUSG2YkGyR/Gtmr0Lmxt4oGq3xMdJf/K41npu65f+mIDpUGd1YjI3wTvWR+t 04KafW432mwmC4Fwb/hpUEAOrS1TmKi4vys9NpUBWT7PIaAC4kbfBEpQyjVbGr7ewl0+ WP9iFX31KvZDK3uZgVEe9T/b37Nm1OPDbnnPCq4qMZWTXPBUMns5n4rbFFpEG++KE0cq vxaw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=hw1JZT4caRKOkdMu9I0bZSzXZiCQYqiKX0Yt62R8CcM=; b=GuqlZbpRw1B/JmGrCJFEWpkuJD7SBPXosjzBvf0S7Hhogbrdia2FgO4wZUslJ/sh8g 4Ll3/ljO+d9ICDsAO1iiXNigdRnmpputNvUC5P6h3k2yPnxmjUvns71pEctIsdTkACqH C/xt1rF6O/HOjyevhaY6ss+8bpHw061VsjXzZW60eF0WavYHjTX/0G1TXIyLoMr0dbrH Bl78pjvPHfLrg0ZZVJBkC3mLL+Ml0BoQjcs12FLWiSej/8ESSuL6+tPB/msX3IU2Uxwn K7OMGAMsl9A4sDSR0TbjeV4OLMBJg4elBfuTzASQkCiSn5IZT8HHRqrCVIaIJ2w7ef/y Ad1A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=RcFJQebs; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id d8-20020a056402078800b0048a316518f9si8185412edy.320.2023.01.12.13.50.54; Thu, 12 Jan 2023 13:51:21 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=RcFJQebs; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234657AbjALVtI (ORCPT + 99 others); Thu, 12 Jan 2023 16:49:08 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41568 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240782AbjALVrh (ORCPT ); Thu, 12 Jan 2023 16:47:37 -0500 Received: from mail-pf1-x42f.google.com (mail-pf1-x42f.google.com [IPv6:2607:f8b0:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D223434756; Thu, 12 Jan 2023 13:40:29 -0800 (PST) Received: by mail-pf1-x42f.google.com with SMTP id a184so14781571pfa.9; Thu, 12 Jan 2023 13:40:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=hw1JZT4caRKOkdMu9I0bZSzXZiCQYqiKX0Yt62R8CcM=; b=RcFJQebsL1oo93IC8w4Oxta1bemGsbrJW5pMa303hyHmzRNvU7QBOBjeX85vkrLEMI Sk8rvxoUTPBaDeQKT680TOM72ra1sjElMwvEJgEJg0Ixa1MHLja3gUAB5dX2sqdGdS+B 2Ftg662VuXT/wTPCOj17BJlqy3JKqPcihaJF9+HMrbcRO0NM+nDcEmtBjx/IFreBBKcV hEYBr7jIEVgn3SgMffwhKIVJvoRsqwkc4pY6hlH3ftDgyMONnAebid7/JZlnztEwWDRL MZtAQtFWAI5VqoXoV9cSAyctDSmahTV6wIziVVoFz8qE/VCP3fHp+HqVMDhDeyg/RYwW YUJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=hw1JZT4caRKOkdMu9I0bZSzXZiCQYqiKX0Yt62R8CcM=; b=xZZErkY3MCSEchY1+UfF4tsyDHjS/ACCM/gc7dyESV00VoHFtHow9BV1NIDHJ4hY8R wuePVcJpNEtaEt2NxTW690K6XTba6a/War6jjz83vKpu9OgSd3H10L8Xa5eIV0J/rQ1Y r8UH8uBvw8RFoOmucELfMnDyvaPqiXfvu7fyNFWE1Mne85ENP/N59LWGnLYJjPuTRof/ d/KJTbIDG+wkzs19C4LC2MypVhFldICGf9jfDr09YpcsfY5o7lXCjDg672YIBWsPvYMC ADOdl8pITXyaKTbm15IU1YsrCLbimL0sAwlSu2mO6pfal0tOb1dTCwGhH+PtHpv13X1i azfg== X-Gm-Message-State: AFqh2kpq7KwigFDN92GKJ8ZYmFn20s9PNMV8JM6IVtJNGKlmIzZyjKoH G8t7U4V+paN23YwCzzpAel8= X-Received: by 2002:a62:f211:0:b0:581:68ea:50 with SMTP id m17-20020a62f211000000b0058168ea0050mr56865082pfh.26.1673559629520; Thu, 12 Jan 2023 13:40:29 -0800 (PST) Received: from youngsil.svl.corp.google.com ([2620:15c:2d4:203:b71e:23d4:aa6a:47c0]) by smtp.gmail.com with ESMTPSA id q20-20020aa79834000000b0058134d2df41sm11344605pfl.146.2023.01.12.13.40.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 12 Jan 2023 13:40:29 -0800 (PST) Sender: Namhyung Kim From: Namhyung Kim To: Peter Zijlstra Cc: Ingo Molnar , Mark Rutland , Alexander Shishkin , Arnaldo Carvalho de Melo , Jiri Olsa , LKML , Song Liu , bpf@vger.kernel.org Subject: [PATCH 8/8] perf/core: Call perf_prepare_sample() before running BPF Date: Thu, 12 Jan 2023 13:40:15 -0800 Message-Id: <20230112214015.1014857-9-namhyung@kernel.org> X-Mailer: git-send-email 2.39.0.314.g84b9a713c41-goog In-Reply-To: <20230112214015.1014857-1-namhyung@kernel.org> References: <20230112214015.1014857-1-namhyung@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.7 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1754855145650806758?= X-GMAIL-MSGID: =?utf-8?q?1754855145650806758?= As BPF can access sample data, it needs to populate the data. Also remove the logic to get the callchain specifically as it's covered by the perf_prepare_sample() now. Cc: Song Liu Cc: bpf@vger.kernel.org Signed-off-by: Namhyung Kim Acked-by: Song Liu --- kernel/events/core.c | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/kernel/events/core.c b/kernel/events/core.c index 5c4f3fa3d2b7..af8365fb639a 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -10363,13 +10363,7 @@ static void bpf_overflow_handler(struct perf_event *event, rcu_read_lock(); prog = READ_ONCE(event->prog); if (prog) { - if (prog->call_get_stack && - (event->attr.sample_type & PERF_SAMPLE_CALLCHAIN) && - !(data->sample_flags & PERF_SAMPLE_CALLCHAIN)) { - data->callchain = perf_callchain(event, regs); - data->sample_flags |= PERF_SAMPLE_CALLCHAIN; - } - + perf_prepare_sample(data, event, regs); ret = bpf_prog_run(prog, &ctx); } rcu_read_unlock();