From patchwork Fri Oct 20 20:47:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Namhyung Kim X-Patchwork-Id: 156313 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2010:b0:403:3b70:6f57 with SMTP id fe16csp1316335vqb; Fri, 20 Oct 2023 13:47:49 -0700 (PDT) X-Google-Smtp-Source: AGHT+IF5T2GWM0t4Py/7BvSVEAEKAnJ8CprF7AiFyOf/og+W7IGlWjU4F0W6pnNeZV5brj1A36NG X-Received: by 2002:a17:90a:de8c:b0:27d:24d6:7343 with SMTP id n12-20020a17090ade8c00b0027d24d67343mr2728832pjv.19.1697834869102; Fri, 20 Oct 2023 13:47:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697834869; cv=none; d=google.com; s=arc-20160816; b=T7z7GTE6e/+qUnYifoR/9uDw4HfdMVt2dfL8P3bM+dSOF3jG0N9pgZDDlTOzQ18JI2 EcfuqmSF4kX8d2V4hSc/89cK2g0OzRBl6RtLjMqnT/0MvYivfCyB5dynzSYojdMfAiun TFhmXFpYmGaq55lqFfkv8avpr6FD3MPFCLTIbjtW0fOJP54uHytzSW4NDxG5aB6PoaQi 6eZj14l9N/YUjvKRrx3zcWHS39+T+SntDhK8EjUq9ouFhI5wMl/dzLp0SdjT4blbzOwE SdKq+S/nXmcUDGxuMZ+wVHx7lhj4TlG2Mr+7h5F1+06jHF1xTI+mkP34kgZG7S4T+Tud sZYg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=Mm7dtlx0Z3ORS3BpllS20dd9G/AciSYgKTckitgJskM=; fh=Eu3OfQ0ohnhl6u2myNPCo9midvMFa9y2PC9Dj+pizlY=; b=nUcP+NEVSSNHp+8s+zPGcfiE9/1xs6PK7sDo1p67N7E4zgPiOAOC7qyNu+jkumVXQ0 Kn1YkSTT2y+Lpbl82iX+wd/EuPsTCKfkdBX+wUD0wcJklZwe5V8bPc3luWqwo4qeytFv ikSRcc3YHIs6CKb0lUlAeMLUcjPRleUUlG7tkqhliiabtOo+a9bo8NN4h2KeJbCzxVy9 NxS/o8bQlYF7tKPSZmWxxtDC3wTX+63LCazH11sqEPNjuhxUstN66UvoeZtrt3sYUXIQ yjP8wCZoYGQz95hThE10aR2Ufi15DEaThSq75zMWOFyoQoqfB5V5pkUqUDrQbxJIBly8 p+4Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=gk2nIdCc; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id v14-20020a17090ac90e00b0026800178358si2549293pjt.144.2023.10.20.13.47.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Oct 2023 13:47:49 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=gk2nIdCc; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 5A786805A795; Fri, 20 Oct 2023 13:47:48 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230383AbjJTUrq (ORCPT + 26 others); Fri, 20 Oct 2023 16:47:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49696 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230114AbjJTUrp (ORCPT ); Fri, 20 Oct 2023 16:47:45 -0400 Received: from mail-pf1-x434.google.com (mail-pf1-x434.google.com [IPv6:2607:f8b0:4864:20::434]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EC491D51; Fri, 20 Oct 2023 13:47:43 -0700 (PDT) Received: by mail-pf1-x434.google.com with SMTP id d2e1a72fcca58-6b5cac99cfdso1157386b3a.2; Fri, 20 Oct 2023 13:47:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1697834863; x=1698439663; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:sender:from:to:cc:subject:date:message-id:reply-to; bh=Mm7dtlx0Z3ORS3BpllS20dd9G/AciSYgKTckitgJskM=; b=gk2nIdCc3be2ppxntJ0kb3coXmOqwnvKChPTqLAXDndxiomFoOV0T04nefrUJygfg4 W7tBAnLSlK66I1MCmXI+rxjqK0zxJ5EujjHeYWOlT1ZmNJNkeo994P63GxoP0mq1eg/Q 2oaGXuJbSGH2Q1g0kfP1vFsm75LFl+QoO3cu9/qgNy3NKsuEVuYt8S6tPMrrvz59zfp6 2mYqs0U/QUscZW5XM8GDJkYtkHiUhe7bZoR8MsLTXZVjem7BWK1n0d49vOPR3zDMq5Zu WLn5uhLUeRQqXfuxj8oY3TIDsJPg86xmh7pNZ29JPApr+B+VE5olcdrLP6dkYtTuFr7F bm8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697834863; x=1698439663; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:sender:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=Mm7dtlx0Z3ORS3BpllS20dd9G/AciSYgKTckitgJskM=; b=CpZkfAl61CEyQKgSwE2+mJyUUjM60uqrcdVjf/xPLk0vCNRO8hZEdTmPwf01csBRcK ohHydfs3F7ycxU7iK0Osf3abzVI+kmOO+uNvjlw8EWpT6XS7U8h3CnyWA7/5YrtBZmSI 8O9vI7AKsyhBkFvVKi2f2Ga0PRqR4kJ9xXmempiLEJJx3U5qNHVesdk69ExPvNFfuzsZ ShB2AslFM+wLj6ajy5ZTVZT1mr7VfEeWd6LLojCyuq0F9pgWc76XPiT8T30RJ+VYB4F+ 0h7H4ucW9acTrEylo/NEzlg/86mY4BSWoemhkJ5WpkaPTw9e9/zzQ76kZTVyAczE73mJ DUuQ== X-Gm-Message-State: AOJu0YxrS/xNkTKTyOT4FfMZwKThGxoR1p28iAsoi7yE7KOuKPKKCTgK NnhTndWjjxvOFYVvpVZg0YM= X-Received: by 2002:aa7:88d0:0:b0:68e:41e9:10be with SMTP id k16-20020aa788d0000000b0068e41e910bemr2696426pff.20.1697834863278; Fri, 20 Oct 2023 13:47:43 -0700 (PDT) Received: from bangji.hsd1.ca.comcast.net ([2601:647:6780:42e0:17e0:7ea9:fbc6:4c7d]) by smtp.gmail.com with ESMTPSA id r25-20020aa79639000000b00694f14a784bsm1971183pfg.52.2023.10.20.13.47.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Oct 2023 13:47:42 -0700 (PDT) Sender: Namhyung Kim From: Namhyung Kim To: Arnaldo Carvalho de Melo , Jiri Olsa Cc: Ian Rogers , Adrian Hunter , Peter Zijlstra , Ingo Molnar , LKML , linux-perf-users@vger.kernel.org, Song Liu , Hao Luo , bpf@vger.kernel.org Subject: [PATCH v3 1/3] perf lock contention: Clear lock addr after use Date: Fri, 20 Oct 2023 13:47:39 -0700 Message-ID: <20231020204741.1869520-1-namhyung@kernel.org> X-Mailer: git-send-email 2.42.0.655.g421f12c284-goog MIME-Version: 1.0 X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Fri, 20 Oct 2023 13:47:48 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780308895735061882 X-GMAIL-MSGID: 1780308895735061882 It checks the current lock to calculated the delta of contention time. The address is saved in the tstamp map which is allocated at begining of contention and released at end of contention. But it's possible for bpf_map_delete_elem() to fail. In that case, the element in the tstamp map kept for the current lock and it makes the next contention for the same lock tracked incorrectly. Specificially the next contention begin will see the existing element for the task and it'd just return. Then the next contention end will see the element and calculate the time using the timestamp for the previous begin. This can result in a large value for two small contentions happened from time to time. Let's clear the lock address so that it can be updated next time even if the bpf_map_delete_elem() failed. Signed-off-by: Namhyung Kim Acked-by: Ian Rogers --- tools/perf/util/bpf_skel/lock_contention.bpf.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/tools/perf/util/bpf_skel/lock_contention.bpf.c b/tools/perf/util/bpf_skel/lock_contention.bpf.c index 4900a5dfb4a4..b11179452e19 100644 --- a/tools/perf/util/bpf_skel/lock_contention.bpf.c +++ b/tools/perf/util/bpf_skel/lock_contention.bpf.c @@ -389,6 +389,7 @@ int contention_end(u64 *ctx) duration = bpf_ktime_get_ns() - pelem->timestamp; if ((__s64)duration < 0) { + pelem->lock = 0; bpf_map_delete_elem(&tstamp, &pid); __sync_fetch_and_add(&time_fail, 1); return 0; @@ -422,6 +423,7 @@ int contention_end(u64 *ctx) data = bpf_map_lookup_elem(&lock_stat, &key); if (!data) { if (data_map_full) { + pelem->lock = 0; bpf_map_delete_elem(&tstamp, &pid); __sync_fetch_and_add(&data_fail, 1); return 0; @@ -445,6 +447,7 @@ int contention_end(u64 *ctx) data_map_full = 1; __sync_fetch_and_add(&data_fail, 1); } + pelem->lock = 0; bpf_map_delete_elem(&tstamp, &pid); return 0; } @@ -458,6 +461,7 @@ int contention_end(u64 *ctx) if (data->min_time > duration) data->min_time = duration; + pelem->lock = 0; bpf_map_delete_elem(&tstamp, &pid); return 0; } From patchwork Fri Oct 20 20:47:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Namhyung Kim X-Patchwork-Id: 156314 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2010:b0:403:3b70:6f57 with SMTP id fe16csp1316406vqb; Fri, 20 Oct 2023 13:48:01 -0700 (PDT) X-Google-Smtp-Source: AGHT+IH6D8ixNa6ma3xl2Z6lj/CcdXOdjPkKt/aekhBOLApWE119fSyy8IDwZtUlL3/7W/kLBVJs X-Received: by 2002:a17:90b:1192:b0:261:685:95b6 with SMTP id gk18-20020a17090b119200b00261068595b6mr2905436pjb.13.1697834880743; Fri, 20 Oct 2023 13:48:00 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697834880; cv=none; d=google.com; s=arc-20160816; b=e59d+XtZFm5XWP5AfaC+5iJD0KS4r26ol6q7yJmayBDsPUzPRU/z1CFU1OrnJ6oElU sA53UVxpv+zCDhyuBVOyco75V9y+poRMxR5CVkToPNaKGfIIz4VW1e9QElLIKnx+Cdn+ YrBms4f+hPy0uT6HgFB/C40L2FchbfcGoo6VpUkuLKmtzfck2Kmy5/56LYotzq5ct1vn 3dfyNxwxA5V/+TCe7o32Qp+7JnfaWzaqmLfx8gpzbSrPR3lRlGUHxumeCvPLZjtUDE7y oOyds7o+awl0rtl8/4AfW2kmLGXqLrTb7KLAa3hcfYIuj3he+urG791RJTp6RW6izW6w dM+g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=s0ZIMRigGTwdVjAgf14DvEXgC3201/iiyAMjiEO7lCw=; fh=Eu3OfQ0ohnhl6u2myNPCo9midvMFa9y2PC9Dj+pizlY=; b=wZmJz0gsWoYYJUD+1yte2I9isHEgLjkPagKcVYhb7LwTZHGORt4azEy7QFzmnTjBYx f0bPCrE/Z+5wttzp2ZHw0waCGo9MeOHGKfF7C7ItkGeuxvPgPl6USS9+Igu4qPjpdPSR tOmPDR0p1CfwQUfwFgJsTCI7HbPLGz3JF9aFmtFP8a3VX6RhB5M9u6KIFd02KL33+EqK DgzAGHJiMe7MGQ2lV8QCk1YBnNY0ifDQ7UZryDWn2L45iZKo139xhyhEShxRCyZ/iU7E ILNgAVqnoi5eKwDXCvf0tA2/buyfGQoP1C3kHC5XAZ+g3wwwho6S4YH34DwKciHBMYFw i1+w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=YZbhJEo4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from pete.vger.email (pete.vger.email. [23.128.96.36]) by mx.google.com with ESMTPS id z11-20020a17090abd8b00b00273dfe0741csi4806971pjr.90.2023.10.20.13.48.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Oct 2023 13:48:00 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) client-ip=23.128.96.36; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=YZbhJEo4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by pete.vger.email (Postfix) with ESMTP id 1F042803D5A8; Fri, 20 Oct 2023 13:47:56 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at pete.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230430AbjJTUrs (ORCPT + 26 others); Fri, 20 Oct 2023 16:47:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49710 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230348AbjJTUrq (ORCPT ); Fri, 20 Oct 2023 16:47:46 -0400 Received: from mail-pf1-x42a.google.com (mail-pf1-x42a.google.com [IPv6:2607:f8b0:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 11CFCE8; Fri, 20 Oct 2023 13:47:45 -0700 (PDT) Received: by mail-pf1-x42a.google.com with SMTP id d2e1a72fcca58-6ba54c3ed97so1179124b3a.2; Fri, 20 Oct 2023 13:47:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1697834864; x=1698439664; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=s0ZIMRigGTwdVjAgf14DvEXgC3201/iiyAMjiEO7lCw=; b=YZbhJEo4UOjprrvjrvFewOSic0H1Fdd2aWy9RxOXWOyVWMm04JY6DG4ic6DUoK+POC pKveaFOEcJsIVoq4Pi1pl8Vj8SDGXrfiS7zWN2xQTH/Kg2vC67oMNzTfEzpSLkWIWrFR 5qIWBMOLTdhfDKSBy14laUy8ArK/n8vLhwg+lyRGZSrmVqTA074PJfJ0dFtPXDKhoedV ee6batgpo9oTJ0z9fWkqxneeYxjJthOdMCUF0Aei6x8ID/ZpmhMtxrL47kSxpAz/IHK7 cKmHa4jE4rHAZHfwhlq0iow/5vPkLZY5KhVHTEVy7XfGpzIA26XuYYe0GwBX5CY7/G1y v7Rg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697834864; x=1698439664; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=s0ZIMRigGTwdVjAgf14DvEXgC3201/iiyAMjiEO7lCw=; b=tW8GXgNNB0cmEV/D2NhWfCv4TWA2b8VK5aWXm71HVIE5i8AHjy8TPL2YPybWp+GAGq +HvcrER7wt5KC0Tfu0Z3kv8rxjvPP8IkMoI7pC77SPiqwG33b68AWylEMKYih1pvP0e2 t2ErIxYjHa3K5GqnSe0Wjv/1wNamYwh2FV1YWc1FhtDZVh2Xx+qsXYIi371KFd3dEHL+ IsQWop61xUMpiHgTlyPZzAqxuZV7fOGM7CElIyfoCigvHUB0rIZEeDgsdbE16Oqw7Ww0 LrEkji1QxBLChBfb7Nw9UlxehXGYKtqI5sJkTJQXJm22eBnNUw4AhLjR/kB5OEb0GZ9e lAwg== X-Gm-Message-State: AOJu0YzYoqzUk/5esNk0quk+nzY5HFTFjtIPtWPMqpkpdN4d5iPRrxZj VOhytdTCxWbeQ/2VjyKHzd4= X-Received: by 2002:a05:6a00:853:b0:6be:b79:eca with SMTP id q19-20020a056a00085300b006be0b790ecamr3278962pfk.32.1697834864411; Fri, 20 Oct 2023 13:47:44 -0700 (PDT) Received: from bangji.hsd1.ca.comcast.net ([2601:647:6780:42e0:17e0:7ea9:fbc6:4c7d]) by smtp.gmail.com with ESMTPSA id r25-20020aa79639000000b00694f14a784bsm1971183pfg.52.2023.10.20.13.47.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Oct 2023 13:47:44 -0700 (PDT) Sender: Namhyung Kim From: Namhyung Kim To: Arnaldo Carvalho de Melo , Jiri Olsa Cc: Ian Rogers , Adrian Hunter , Peter Zijlstra , Ingo Molnar , LKML , linux-perf-users@vger.kernel.org, Song Liu , Hao Luo , bpf@vger.kernel.org Subject: [PATCH v3 2/3] perf lock contention: Check race in tstamp elem creation Date: Fri, 20 Oct 2023 13:47:40 -0700 Message-ID: <20231020204741.1869520-2-namhyung@kernel.org> X-Mailer: git-send-email 2.42.0.655.g421f12c284-goog In-Reply-To: <20231020204741.1869520-1-namhyung@kernel.org> References: <20231020204741.1869520-1-namhyung@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.0 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Fri, 20 Oct 2023 13:47:56 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780308908046604282 X-GMAIL-MSGID: 1780308908046604282 When pelem is NULL, it'd create a new entry with zero data. But it might be preempted by IRQ/NMI just before calling bpf_map_update_elem() then there's a chance to call it twice for the same pid. So it'd be better to use BPF_NOEXIST flag and check the return value to prevent the race. Signed-off-by: Namhyung Kim Acked-by: Ian Rogers --- tools/perf/util/bpf_skel/lock_contention.bpf.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/tools/perf/util/bpf_skel/lock_contention.bpf.c b/tools/perf/util/bpf_skel/lock_contention.bpf.c index b11179452e19..69d31fd77cd0 100644 --- a/tools/perf/util/bpf_skel/lock_contention.bpf.c +++ b/tools/perf/util/bpf_skel/lock_contention.bpf.c @@ -328,7 +328,11 @@ int contention_begin(u64 *ctx) if (pelem == NULL) { struct tstamp_data zero = {}; - bpf_map_update_elem(&tstamp, &pid, &zero, BPF_ANY); + if (bpf_map_update_elem(&tstamp, &pid, &zero, BPF_NOEXIST) < 0) { + __sync_fetch_and_add(&task_fail, 1); + return 0; + } + pelem = bpf_map_lookup_elem(&tstamp, &pid); if (pelem == NULL) { __sync_fetch_and_add(&task_fail, 1); From patchwork Fri Oct 20 20:47:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Namhyung Kim X-Patchwork-Id: 156315 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2010:b0:403:3b70:6f57 with SMTP id fe16csp1316457vqb; Fri, 20 Oct 2023 13:48:10 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGq0uOL9cW8l3E3pU0hl4RIBtEklXNrxBONc/plVmk5whvnt8omglYmz8F26Md7Ek4A8TVe X-Received: by 2002:a05:6a00:194a:b0:6b9:f641:272e with SMTP id s10-20020a056a00194a00b006b9f641272emr3032623pfk.24.1697834890037; Fri, 20 Oct 2023 13:48:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697834890; cv=none; d=google.com; s=arc-20160816; b=oV08HkzjFxAlEhjlh2sjtiF3ImamIF5eIX+97O40gaUNc7sq1yITxV27A1KI1WxxqB +eQ1PJWgv+jy4Rip6jZ7uHOpZlPXXV6z9nq8Jxbj46A9W2+SWgKzZ2Sx7aOJvviRE+IO dpHUngPHYGEX7NMKtRRQlQH2weSEZ5CeWXqcoc6DAIEfNbaHO4gptgzSWeJI0OJYiToJ s0NupdMJUzKZPpgbCs9ZfjZRB+z1kEXNl0f+moJURb/8z4M/k5HGt/FpclwIc6P59vWU 14jBPiLuCoRZaTnHylTBluXpq+UIBmMN6ggjS7lMFdsKcjMEv4zkwNmsmx5+AchOGz8/ sBMg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=JOImYZdSx2m0PGY/kvmONhOIa9TTLkTXQlBLoR0LTq8=; fh=Eu3OfQ0ohnhl6u2myNPCo9midvMFa9y2PC9Dj+pizlY=; b=O/0ChCq5ipyTcgSU5EA3ud0TiakF5Iz1vy/bD7R1q7joj8DrnXfUkolz0AAgtcIf0u StgNSiOxsPmISnx37HCh23u3cBRopl83MDQGD/clF3lfdPAGR0A1ngaomMkQ0J55p6iP XCBqXPHVQZ0/CEDUuY15F4Glm/enXe8rPmO0D4ijnsPAzMIz+8f/Z9s14ujCW/UuE+mD ipcat0OeQpBOyxVpo6R6cJ338IG1yNsqOK+o2qMHpp5HtvLJ2G/oGA7I0ubtsMPCtbLb 5oFBkCzi7Q8vCiKIwZUKTmlzyovNN6PIPOeX4/+f7+9Xs1v5WmWmGgdSbLLtUFYX4LcQ oXwg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=fkrMbFRA; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from pete.vger.email (pete.vger.email. [2620:137:e000::3:6]) by mx.google.com with ESMTPS id f9-20020a056a0022c900b006bd0842622asi2698600pfj.239.2023.10.20.13.48.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Oct 2023 13:48:10 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) client-ip=2620:137:e000::3:6; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=fkrMbFRA; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by pete.vger.email (Postfix) with ESMTP id CEC9F80B8F68; Fri, 20 Oct 2023 13:48:06 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at pete.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231279AbjJTUrx (ORCPT + 26 others); Fri, 20 Oct 2023 16:47:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40536 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230522AbjJTUrs (ORCPT ); Fri, 20 Oct 2023 16:47:48 -0400 Received: from mail-pf1-x431.google.com (mail-pf1-x431.google.com [IPv6:2607:f8b0:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 70229D65; Fri, 20 Oct 2023 13:47:46 -0700 (PDT) Received: by mail-pf1-x431.google.com with SMTP id d2e1a72fcca58-6be840283ceso1163189b3a.3; Fri, 20 Oct 2023 13:47:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1697834866; x=1698439666; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=JOImYZdSx2m0PGY/kvmONhOIa9TTLkTXQlBLoR0LTq8=; b=fkrMbFRAqI1vUZkgOdAfiRdTiQsPUmyzXLXgcCkkysEYDVYfxsQiuUKYLRxgFP1vbe q6+swc7mnpxFDxy07AzB8DNQa38AHMj1rbW8BndpdRWnyQ1r52OQJ1/0huFeae1+riq7 eqQesvQvSe0+X65qgaUTpQCUkkPmiN46/Grsy7rj/umcjGQAOV/8OicMXwO2L6fEROcv R7evtYgSeoNgVJvaBBfCcQSZBC7dwVoaNnRNVNupAUB2APAch6mGjYxU98FCZhArLUhX tdRrLQ1pnClr7j0w0fDyOtO5cHnFItv9Na4366M4cugbGgNqXIhwMv9H8fFJwdXRHa5m Enqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697834866; x=1698439666; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=JOImYZdSx2m0PGY/kvmONhOIa9TTLkTXQlBLoR0LTq8=; b=N1gCvrqjVfR4Q+g/QIr8XcXhqdVjJZtPqTWCMqdKhlXr+kI7YgHTUgHvt+tnhhLRMA 1sXKpN+mXVgxLubwC9PQ0e7r0b6BNxYQHHmzu7YvLEvxWqHhtketc1NTuFQYWJMc0Bgn xYQuiaEp2yd6BvaYn/WTHbkVfzJtEnsAlzwv2SypqIYG+jVR3AajNe47PXMzUQ+PJDW3 Ut21pPNTGjm+JfTXpBLCXGwFIKBROlY+qTRiMG2ZDqbVOS8BINrGeTgpcGTycokJd34s OENRhAWu9uR58DpAe/Kj9onSrILmbics07ddf32+WOeOZTD+8JjraeHHZMBKztObaxVb ABmw== X-Gm-Message-State: AOJu0YzJIPF3ZLRBAul+jgyC4v2hkU13Ftcp0Z7QeNe/RWE7jxfBDhwP P7m0GC4Fk0myCyh5zlm0H47hYHMdxrU= X-Received: by 2002:a05:6a21:7989:b0:17b:8404:96d8 with SMTP id bh9-20020a056a21798900b0017b840496d8mr3002343pzc.41.1697834865589; Fri, 20 Oct 2023 13:47:45 -0700 (PDT) Received: from bangji.hsd1.ca.comcast.net ([2601:647:6780:42e0:17e0:7ea9:fbc6:4c7d]) by smtp.gmail.com with ESMTPSA id r25-20020aa79639000000b00694f14a784bsm1971183pfg.52.2023.10.20.13.47.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Oct 2023 13:47:45 -0700 (PDT) Sender: Namhyung Kim From: Namhyung Kim To: Arnaldo Carvalho de Melo , Jiri Olsa Cc: Ian Rogers , Adrian Hunter , Peter Zijlstra , Ingo Molnar , LKML , linux-perf-users@vger.kernel.org, Song Liu , Hao Luo , bpf@vger.kernel.org Subject: [PATCH v3 3/3] perf lock contention: Use per-cpu array map for spinlocks Date: Fri, 20 Oct 2023 13:47:41 -0700 Message-ID: <20231020204741.1869520-3-namhyung@kernel.org> X-Mailer: git-send-email 2.42.0.655.g421f12c284-goog In-Reply-To: <20231020204741.1869520-1-namhyung@kernel.org> References: <20231020204741.1869520-1-namhyung@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.0 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Fri, 20 Oct 2023 13:48:06 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780308917692080606 X-GMAIL-MSGID: 1780308917692080606 Currently lock contention timestamp is maintained in a hash map keyed by pid. That means it needs to get and release a map element (which is proctected by spinlock!) on each contention begin and end pair. This can impact on performance if there are a lot of contention (usually from spinlocks). It used to go with task local storage but it had an issue on memory allocation in some critical paths. Although it's addressed in recent kernels IIUC, the tool should support old kernels too. So it cannot simply switch to the task local storage at least for now. As spinlocks create lots of contention and they disabled preemption during the spinning, it can use per-cpu array to keep the timestamp to avoid overhead in hashmap update and delete. In contention_begin, it's easy to check the lock types since it can see the flags. But contention_end cannot see it. So let's try to per-cpu array first (unconditionally) if it has an active element (lock != 0). Then it should be used and per-task tstamp map should not be used until the per-cpu array element is cleared which means nested spinlock contention (if any) was finished and it nows see (the outer) lock. Signed-off-by: Namhyung Kim Acked-by: Ian Rogers --- .../perf/util/bpf_skel/lock_contention.bpf.c | 89 +++++++++++++++---- 1 file changed, 72 insertions(+), 17 deletions(-) diff --git a/tools/perf/util/bpf_skel/lock_contention.bpf.c b/tools/perf/util/bpf_skel/lock_contention.bpf.c index 69d31fd77cd0..95cd8414f6ef 100644 --- a/tools/perf/util/bpf_skel/lock_contention.bpf.c +++ b/tools/perf/util/bpf_skel/lock_contention.bpf.c @@ -42,6 +42,14 @@ struct { __uint(max_entries, MAX_ENTRIES); } tstamp SEC(".maps"); +/* maintain per-CPU timestamp at the beginning of contention */ +struct { + __uint(type, BPF_MAP_TYPE_PERCPU_ARRAY); + __uint(key_size, sizeof(__u32)); + __uint(value_size, sizeof(struct tstamp_data)); + __uint(max_entries, 1); +} tstamp_cpu SEC(".maps"); + /* actual lock contention statistics */ struct { __uint(type, BPF_MAP_TYPE_HASH); @@ -311,34 +319,57 @@ static inline __u32 check_lock_type(__u64 lock, __u32 flags) return 0; } -SEC("tp_btf/contention_begin") -int contention_begin(u64 *ctx) +static inline struct tstamp_data *get_tstamp_elem(__u32 flags) { __u32 pid; struct tstamp_data *pelem; - if (!enabled || !can_record(ctx)) - return 0; + /* Use per-cpu array map for spinlock and rwlock */ + if (flags == (LCB_F_SPIN | LCB_F_READ) || flags == LCB_F_SPIN || + flags == (LCB_F_SPIN | LCB_F_WRITE)) { + __u32 idx = 0; + + pelem = bpf_map_lookup_elem(&tstamp_cpu, &idx); + /* Do not update the element for nested locks */ + if (pelem && pelem->lock) + pelem = NULL; + return pelem; + } pid = bpf_get_current_pid_tgid(); pelem = bpf_map_lookup_elem(&tstamp, &pid); + /* Do not update the element for nested locks */ if (pelem && pelem->lock) - return 0; + return NULL; if (pelem == NULL) { struct tstamp_data zero = {}; if (bpf_map_update_elem(&tstamp, &pid, &zero, BPF_NOEXIST) < 0) { __sync_fetch_and_add(&task_fail, 1); - return 0; + return NULL; } pelem = bpf_map_lookup_elem(&tstamp, &pid); if (pelem == NULL) { __sync_fetch_and_add(&task_fail, 1); - return 0; + return NULL; } } + return pelem; +} + +SEC("tp_btf/contention_begin") +int contention_begin(u64 *ctx) +{ + struct tstamp_data *pelem; + + if (!enabled || !can_record(ctx)) + return 0; + + pelem = get_tstamp_elem(ctx[1]); + if (pelem == NULL) + return 0; pelem->timestamp = bpf_ktime_get_ns(); pelem->lock = (__u64)ctx[0]; @@ -377,24 +408,42 @@ int contention_begin(u64 *ctx) SEC("tp_btf/contention_end") int contention_end(u64 *ctx) { - __u32 pid; + __u32 pid = 0, idx = 0; struct tstamp_data *pelem; struct contention_key key = {}; struct contention_data *data; __u64 duration; + bool need_delete = false; if (!enabled) return 0; - pid = bpf_get_current_pid_tgid(); - pelem = bpf_map_lookup_elem(&tstamp, &pid); - if (!pelem || pelem->lock != ctx[0]) - return 0; + /* + * For spinlock and rwlock, it needs to get the timestamp for the + * per-cpu map. However, contention_end does not have the flags + * so it cannot know whether it reads percpu or hash map. + * + * Try per-cpu map first and check if there's active contention. + * If it is, do not read hash map because it cannot go to sleeping + * locks before releasing the spinning locks. + */ + pelem = bpf_map_lookup_elem(&tstamp_cpu, &idx); + if (pelem && pelem->lock) { + if (pelem->lock != ctx[0]) + return 0; + } else { + pid = bpf_get_current_pid_tgid(); + pelem = bpf_map_lookup_elem(&tstamp, &pid); + if (!pelem || pelem->lock != ctx[0]) + return 0; + need_delete = true; + } duration = bpf_ktime_get_ns() - pelem->timestamp; if ((__s64)duration < 0) { pelem->lock = 0; - bpf_map_delete_elem(&tstamp, &pid); + if (need_delete) + bpf_map_delete_elem(&tstamp, &pid); __sync_fetch_and_add(&time_fail, 1); return 0; } @@ -406,8 +455,11 @@ int contention_end(u64 *ctx) case LOCK_AGGR_TASK: if (lock_owner) key.pid = pelem->flags; - else + else { + if (!need_delete) + pid = bpf_get_current_pid_tgid(); key.pid = pid; + } if (needs_callstack) key.stack_id = pelem->stack_id; break; @@ -428,7 +480,8 @@ int contention_end(u64 *ctx) if (!data) { if (data_map_full) { pelem->lock = 0; - bpf_map_delete_elem(&tstamp, &pid); + if (need_delete) + bpf_map_delete_elem(&tstamp, &pid); __sync_fetch_and_add(&data_fail, 1); return 0; } @@ -452,7 +505,8 @@ int contention_end(u64 *ctx) __sync_fetch_and_add(&data_fail, 1); } pelem->lock = 0; - bpf_map_delete_elem(&tstamp, &pid); + if (need_delete) + bpf_map_delete_elem(&tstamp, &pid); return 0; } @@ -466,7 +520,8 @@ int contention_end(u64 *ctx) data->min_time = duration; pelem->lock = 0; - bpf_map_delete_elem(&tstamp, &pid); + if (need_delete) + bpf_map_delete_elem(&tstamp, &pid); return 0; }