From patchwork Fri Jan 19 00:13:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kyle Huey X-Patchwork-Id: 189365 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2bc4:b0:101:a8e8:374 with SMTP id hx4csp703184dyb; Thu, 18 Jan 2024 16:14:48 -0800 (PST) X-Google-Smtp-Source: AGHT+IEULIYY4+GxFB6MEIxzToHsGmgobqH/n0vCy0F8YeSPI9+wk+qCWedA6vBRVPYRfEevTTGZ X-Received: by 2002:ad4:5dcf:0:b0:681:55a5:ec55 with SMTP id m15-20020ad45dcf000000b0068155a5ec55mr1252895qvh.75.1705623288351; Thu, 18 Jan 2024 16:14:48 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1705623288; cv=pass; d=google.com; s=arc-20160816; b=RFksu/xyFzLp/klU3m9wKc+JmK5AzHcWtQ+pND+z1HfGGyJ1Is5p/HjAvk0A/zStbr U4YP47Vg99MgIlFoq3nK8dlolaSmzYXI0K4jrWfO/vU7yrtw2imDNC08ILiypb5EXzeb LiXURoVYu9sKGokUg/V6zU1vIJiqzB8yaVYtx1pENOGOr9wG4BkHtr5vGDQNudidZKpm 4oQEHKNVsB+thXNZRpso+b6JG8BPKB5W0k6n+Rx104zLP3AwZnZx/pKQBzin6QaE0H2+ ZARWJP7se3XToD3t+RplOhFDtl0CbelrI1BQV2xYY9SNCVHVO5/Zx2fx+rTdlvfVHDaG JhGA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=goCRlABDp65iNRuy7Fdk99c/XuzP9RSm/x9LjyCmqK8=; fh=vyLuMFPX7a5RxtKqZusW0ny9V2SiUBkKFuhqxl/vIvk=; b=yHeZ+CjG9sof6tzNMP3pw0Z7wywLnaHE2mHAe1y/5duC+pcX2WodUhIQSUuiX12K+P PIMMR+LWEbOCUjDYxU5ngHZz3GTxv7s3uj9KgFdaXIuoBuwe2to4s/8V6HvRENcCy1L9 B2zKFI3YKo+fa9KSM6za68h+teuEBs+DDPaMw4Fz+LS7NawWjNsbRSZpgKgRk0h8r+G4 zU++zwPqJzsB8SEE1s5CNVWHeVY6HV6bZa02PYCClLrvExo84/85+TeromFCxw+cBP5r NU6BmzEVtvFgXdvgJhWuip84BA/6pQ/bcXRGFl/NLzrMtDm1BVCgxcQ9TG8n+3ZnzkPG SLqQ== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kylehuey.com header.s=google header.b=g5vLseTw; arc=pass (i=1 spf=pass spfdomain=kylehuey.com dkim=pass dkdomain=kylehuey.com); spf=pass (google.com: domain of linux-kernel+bounces-30650-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-30650-ouuuleilei=gmail.com@vger.kernel.org" Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id g16-20020a05620a219000b007833e8da4fasi14513198qka.599.2024.01.18.16.14.48 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 18 Jan 2024 16:14:48 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-30650-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@kylehuey.com header.s=google header.b=g5vLseTw; arc=pass (i=1 spf=pass spfdomain=kylehuey.com dkim=pass dkdomain=kylehuey.com); spf=pass (google.com: domain of linux-kernel+bounces-30650-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-30650-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 1B1851C22A5A for ; Fri, 19 Jan 2024 00:14:48 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id CAB162117; Fri, 19 Jan 2024 00:14:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kylehuey.com header.i=@kylehuey.com header.b="g5vLseTw" Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 478D31111 for ; Fri, 19 Jan 2024 00:14:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705623255; cv=none; b=hOeXs8a6JCJGesJy4uADdQu6YOS+WDsquFWcqbY+dgyKsN0dex/gSJCaKa17bZNbaWg1LedwLVmCnuMM8zJkhvjxK+hEe0T/7/eP5+wQrfIMpKrq5HLT3lgMg6ZW8+1EmZGDTY0sxzk4qfq/eub7nkKTFHc9YFzUiQ0F2d16XlA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705623255; c=relaxed/simple; bh=BvdPhHhuuf/fRX3SqKFdqqVaVqqf5FQjd8NlByquZYU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Rwr/uAmv6YncLqVM0SCkBndWAhYIP93pUbRBM/pfei2k/Vb1PdenwuDK7UhUCjAVzhmsGCTDrZsS4Ihn1AHyQWlTyL5jnjC+2F9O0bRYsc2dlSnbnIASR5wcLlvERailhZMlJxURV+VM+Y/rvRDqC0lOOKrVB+bdcGEY7pL03AA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kylehuey.com; spf=pass smtp.mailfrom=kylehuey.com; dkim=pass (2048-bit key) header.d=kylehuey.com header.i=@kylehuey.com header.b=g5vLseTw; arc=none smtp.client-ip=209.85.214.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kylehuey.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kylehuey.com Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-1d5f1e0e32eso1395905ad.3 for ; Thu, 18 Jan 2024 16:14:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kylehuey.com; s=google; t=1705623253; x=1706228053; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=goCRlABDp65iNRuy7Fdk99c/XuzP9RSm/x9LjyCmqK8=; b=g5vLseTwxcL1+e/Y4krZOtr6EtaiOwYUQSZuxbGwOUr8uMWz/nICL/yTIYe+NmfkvY x0lJAgXKVhLVmjFFCntcSbWlM7htz8Hf8jgvRizk46PvmwYOExB+gglUH16J+xor7cWK zMRZVkDvkuTiRFwLXPKaqy/+BDIhqynGoWKpZ3BWwurBeWF8yPMB/XyqcKd7IRFd7SfT ySZYS+yFFx8GtAVLI0/eTK3Q4KP1MeOChv/suiKL+1Y3mH1Q02gLPRbQvQKQVMcPYyZk lz+BPo9Z2og9ZGxLWEERNuffyOYNgJ8NabEbVnG8/VatSP7wWAvrdEoFpd/OlUVYjTPo fFTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705623253; x=1706228053; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=goCRlABDp65iNRuy7Fdk99c/XuzP9RSm/x9LjyCmqK8=; b=HkYpKVxmjIhrTfAYFD7xxnjJuNc2p+CQ7ed33OaXwM+f88w8jLq7OYce/Tz9kVPBDM foDhZVg1xkK6vsKOr6UJynrBu/Z3M/xNbb3VWpvUtM42YmVv7tbIeX8rcVkgKGuzaQnT EJmrRtQ8UPvVZeLkKrtk985Qqa1Yur3rr8ILxf80Fw8J2bGNOS/1yVHiPYGH3bdCzBFE 7zZulErGIxzGJkRJSS5wSDLgEVM0e4mSlzqZA3i4JPhqJ0DOi0ubFH9BPSP35+ChC3oP +Uf64oJZWNLSmxj6VjDDBTyKWeOI2bKXedgveuqhhFNBvNfeHCZ5AgEgFpvF7SlH9I2+ bNLw== X-Gm-Message-State: AOJu0YzGDRu80alVO3vTmFbCKzkP4um7P8Q0KAQreWvdV3zZd3eS1qyL qLqNaYE0oZ8xOVksI8GOepZq7lv0Nys0oKlvfToPEQmVKM5BDHj/qJURRLA6rQ== X-Received: by 2002:a17:902:dac6:b0:1d4:441e:7377 with SMTP id q6-20020a170902dac600b001d4441e7377mr1612768plx.40.1705623253562; Thu, 18 Jan 2024 16:14:13 -0800 (PST) Received: from zhadum.home.kylehuey.com (c-76-126-33-191.hsd1.ca.comcast.net. [76.126.33.191]) by smtp.gmail.com with ESMTPSA id mj7-20020a1709032b8700b001d1d1ef8be6sm1921238plb.267.2024.01.18.16.14.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 18 Jan 2024 16:14:13 -0800 (PST) From: Kyle Huey X-Google-Original-From: Kyle Huey To: Kyle Huey , linux-kernel@vger.kernel.org, Andrii Nakryiko , Jiri Olsa , Namhyung Kim , Marco Elver , Yonghong Song , Song Liu Cc: Robert O'Callahan , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Mark Rutland , Alexander Shishkin , Ian Rogers , Adrian Hunter , linux-perf-users@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH v4 1/4] perf/bpf: Call bpf handler directly, not through overflow machinery Date: Thu, 18 Jan 2024 16:13:48 -0800 Message-Id: <20240119001352.9396-2-khuey@kylehuey.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240119001352.9396-1-khuey@kylehuey.com> References: <20240119001352.9396-1-khuey@kylehuey.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1788475645445403476 X-GMAIL-MSGID: 1788475645445403476 To ultimately allow bpf programs attached to perf events to completely suppress all of the effects of a perf event overflow (rather than just the sample output, as they do today), call bpf_overflow_handler() from __perf_event_overflow() directly rather than modifying struct perf_event's overflow_handler. Return the bpf program's return value from bpf_overflow_handler() so that __perf_event_overflow() knows how to proceed. Remove the now unnecessary orig_overflow_handler from struct perf_event. This patch is solely a refactoring and results in no behavior change. Signed-off-by: Kyle Huey Suggested-by: Namhyung Kim Acked-by: Song Liu --- include/linux/perf_event.h | 6 +----- kernel/events/core.c | 28 +++++++++++++++------------- 2 files changed, 16 insertions(+), 18 deletions(-) diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h index d2a15c0c6f8a..c7f54fd74d89 100644 --- a/include/linux/perf_event.h +++ b/include/linux/perf_event.h @@ -810,7 +810,6 @@ struct perf_event { perf_overflow_handler_t overflow_handler; void *overflow_handler_context; #ifdef CONFIG_BPF_SYSCALL - perf_overflow_handler_t orig_overflow_handler; struct bpf_prog *prog; u64 bpf_cookie; #endif @@ -1357,10 +1356,7 @@ __is_default_overflow_handler(perf_overflow_handler_t overflow_handler) #ifdef CONFIG_BPF_SYSCALL static inline bool uses_default_overflow_handler(struct perf_event *event) { - if (likely(is_default_overflow_handler(event))) - return true; - - return __is_default_overflow_handler(event->orig_overflow_handler); + return is_default_overflow_handler(event); } #else #define uses_default_overflow_handler(event) \ diff --git a/kernel/events/core.c b/kernel/events/core.c index f0f0f71213a1..24a718e7eb98 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -9548,6 +9548,12 @@ static inline bool sample_is_allowed(struct perf_event *event, struct pt_regs *r return true; } +#ifdef CONFIG_BPF_SYSCALL +static int bpf_overflow_handler(struct perf_event *event, + struct perf_sample_data *data, + struct pt_regs *regs); +#endif + /* * Generic event overflow handling, sampling. */ @@ -9617,7 +9623,10 @@ static int __perf_event_overflow(struct perf_event *event, irq_work_queue(&event->pending_irq); } - READ_ONCE(event->overflow_handler)(event, data, regs); +#ifdef CONFIG_BPF_SYSCALL + if (!(event->prog && !bpf_overflow_handler(event, data, regs))) +#endif + READ_ONCE(event->overflow_handler)(event, data, regs); if (*perf_event_fasync(event) && event->pending_kill) { event->pending_wakeup = 1; @@ -10427,9 +10436,9 @@ static void perf_event_free_filter(struct perf_event *event) } #ifdef CONFIG_BPF_SYSCALL -static void bpf_overflow_handler(struct perf_event *event, - struct perf_sample_data *data, - struct pt_regs *regs) +static int bpf_overflow_handler(struct perf_event *event, + struct perf_sample_data *data, + struct pt_regs *regs) { struct bpf_perf_event_data_kern ctx = { .data = data, @@ -10450,10 +10459,8 @@ static void bpf_overflow_handler(struct perf_event *event, rcu_read_unlock(); out: __this_cpu_dec(bpf_prog_active); - if (!ret) - return; - event->orig_overflow_handler(event, data, regs); + return ret; } static int perf_event_set_bpf_handler(struct perf_event *event, @@ -10489,8 +10496,6 @@ static int perf_event_set_bpf_handler(struct perf_event *event, event->prog = prog; event->bpf_cookie = bpf_cookie; - event->orig_overflow_handler = READ_ONCE(event->overflow_handler); - WRITE_ONCE(event->overflow_handler, bpf_overflow_handler); return 0; } @@ -10501,7 +10506,6 @@ static void perf_event_free_bpf_handler(struct perf_event *event) if (!prog) return; - WRITE_ONCE(event->overflow_handler, event->orig_overflow_handler); event->prog = NULL; bpf_prog_put(prog); } @@ -11975,13 +11979,11 @@ perf_event_alloc(struct perf_event_attr *attr, int cpu, overflow_handler = parent_event->overflow_handler; context = parent_event->overflow_handler_context; #if defined(CONFIG_BPF_SYSCALL) && defined(CONFIG_EVENT_TRACING) - if (overflow_handler == bpf_overflow_handler) { + if (parent_event->prog) { struct bpf_prog *prog = parent_event->prog; bpf_prog_inc(prog); event->prog = prog; - event->orig_overflow_handler = - parent_event->orig_overflow_handler; } #endif }