From patchwork Thu Nov 2 17:57:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Rogers X-Patchwork-Id: 161034 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:8f47:0:b0:403:3b70:6f57 with SMTP id j7csp536642vqu; Thu, 2 Nov 2023 11:02:47 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEBPJmMJPWJwUF0hq97ltXtX1L61uMrBJXGSXozFiX1thGhHSlK0HM/ZJg0J/mJhVwSIVVV X-Received: by 2002:a17:90b:f06:b0:27d:694f:195d with SMTP id br6-20020a17090b0f0600b0027d694f195dmr17913641pjb.6.1698948167313; Thu, 02 Nov 2023 11:02:47 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1698948167; cv=none; d=google.com; s=arc-20160816; b=mDGgHrf6OEhvccNiWrxgjO7Y3as3ZK55rS9vVcH1tghURvzNKN1cuwmyKtHjeBaHiY qdlUQ7t8Ovj0YvoagQb3qP79ykO/fhfuHJ5ZwcZbuv92UQElb6YWC64CwlPid9Cd21Nq sW+MJnDto08RRqOLJzwMgJNEWrgQngEwJs8pwp6d19ofh1eQzOBXa3BXPciEXkQNtnl4 PrwehZMK3o8iMYgdslmq031ME3hVHWOXEujIlR7L7kc5TY78K93mmeXpxe61fz3zsAIw AmmDHp52dqdcLylyJiDQhdxsf4cAzs8TbPD5fWAGxS2brhiHNBJE6RY+09YI3tge+LAB 5N6w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:to:from:subject:references:mime-version :message-id:in-reply-to:date:dkim-signature; bh=gusYDjmzvPXlbpdRzaTtuVbqaUCwbO2hOJb/75bCX5U=; fh=z1KcSqUpYQ9oC4uLSeIkhAYTnJ2bvP0QbNW2xqV5NqA=; b=soRIpzNQiIwy5SO7e/YhrIuPYp4PCLwP/ppceTeI3XErNl5tUfQ9v6WpPT2XGAEoUU YNdjlJmAa15O63QLx8eSP5I6yDpLpmZsb78Au0IOmPxg9vPJ93hAhpls9EqHyGdmVL3p iZZQg+1VGK3xhONdTXdGBHxq1jelbKyS3SqZ8x3rmIZdRxBTE80nCZHazGhIfR6Hw/+2 bc8FCIbayoroqw9sF9UFWLHlDbPZcMdlR/JEPsPxg8aBM6itSJ0TT25/TFNcAUiesF8x DdPWghcVwA1tRcM2JBNE+fyFurhIQjqPYcv4sTkVWjCrmJz/4d95jtT5JlE+EQA+55Ab agwg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=eAJXjzFs; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id kk3-20020a17090b4a0300b0027763ca82e9si100966pjb.91.2023.11.02.11.02.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Nov 2023 11:02:47 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=eAJXjzFs; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 8243482B486F; Thu, 2 Nov 2023 11:01:48 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345134AbjKBSBZ (ORCPT + 35 others); Thu, 2 Nov 2023 14:01:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43852 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1347062AbjKBSAd (ORCPT ); Thu, 2 Nov 2023 14:00:33 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7C9AF1A5 for ; Thu, 2 Nov 2023 10:59:26 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-d9caf486775so1416080276.2 for ; Thu, 02 Nov 2023 10:59:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1698947965; x=1699552765; darn=vger.kernel.org; h=to:from:subject:references:mime-version:message-id:in-reply-to:date :from:to:cc:subject:date:message-id:reply-to; bh=gusYDjmzvPXlbpdRzaTtuVbqaUCwbO2hOJb/75bCX5U=; b=eAJXjzFsqtsEMOkBXu+FeQ9Cas8aZ2gx/VR+Bz9NeMqdcC4pFvtf0L01LgmegIdODt 4t1CnzrnkoLvYRdBcX59jRFG6ZipiVVBvyNeLvy30BfdysE0c8VGvxvUlEB9Fb8TlKgO u+ph5+rs/3AsxkmtCllFSFkMn7/OO6WApLbaAsbcAvyQXqyft0f29MPHgYsbweuE+W/g gE8uI8+kmDiwopgwQlpoJPCqE7gDtvT3Y0RYwYkFnfq1QbaDNqOQgARMEyMgil6oR9xf Fqg9nH+NTybtyg3lpvNdm1/jHv2uMogi8ImvQ3LaLuZHhq4wGTbqDWxUC33X2/D/W5vn o7Ig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698947965; x=1699552765; h=to:from:subject:references:mime-version:message-id:in-reply-to:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=gusYDjmzvPXlbpdRzaTtuVbqaUCwbO2hOJb/75bCX5U=; b=ABqSjrbP7inFUpOvv1qZB1Ym1HmTre7QjFrrGRujH3ZpoZ/CNnA9YHZQ7uIIrB3FWY iwzIKdZXHxebGRYnqNKcTziBgO3bSsIq/oqEFsfHW3kSN4N5bb+j/ptKLCAQS2G0VXYT ra1EPPt0Cu2cnuvtN7+Wo1D29/6L+YnUgXgBJdbhks1/WNW/MRqwmDYWhYYK1heKEAOw RlRABSUIWOlD7kcoh0JSg5//3SmmB5gVlbxHkWqxat3PgUVvHQEbjb/sh6JAHeNASgU+ rOWAA4BJ6EspQEKlKyz2uu8OWDBEUhlWkXPeW29jjBI0lUHVyLularRNWL/MW0sNLHBa z8nA== X-Gm-Message-State: AOJu0YzHe8uJ/XF9vh5LEmmIJ7Y57K7O97JC8lOlEl22Ou+FOqNi3HYq QGxkvK+/Evl5xiroVpdkQIvyI0kMCv3T X-Received: from irogers.svl.corp.google.com ([2620:15c:2a3:200:bb34:df9c:836c:afca]) (user=irogers job=sendgmr) by 2002:a25:3c3:0:b0:d9a:40ed:8d26 with SMTP id 186-20020a2503c3000000b00d9a40ed8d26mr406283ybd.0.1698947965524; Thu, 02 Nov 2023 10:59:25 -0700 (PDT) Date: Thu, 2 Nov 2023 10:57:21 -0700 In-Reply-To: <20231102175735.2272696-1-irogers@google.com> Message-Id: <20231102175735.2272696-40-irogers@google.com> Mime-Version: 1.0 References: <20231102175735.2272696-1-irogers@google.com> X-Mailer: git-send-email 2.42.0.869.gea05f2083d-goog Subject: [PATCH v4 39/53] perf threads: Switch from rbtree to hashmap From: Ian Rogers To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Mark Rutland , Alexander Shishkin , Jiri Olsa , Namhyung Kim , Ian Rogers , Adrian Hunter , Nick Terrell , Kan Liang , Andi Kleen , Kajol Jain , Athira Rajeev , Huacai Chen , Masami Hiramatsu , Vincent Whitchurch , "Steinar H. Gunderson" , Liam Howlett , Miguel Ojeda , Colin Ian King , Dmitrii Dolgov <9erthalion6@gmail.com>, Yang Jihong , Ming Wang , James Clark , K Prateek Nayak , Sean Christopherson , Leo Yan , Ravi Bangoria , German Gomez , Changbin Du , Paolo Bonzini , Li Dong , Sandipan Das , liuwenyu , linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Thu, 02 Nov 2023 11:01:48 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1781476273512144311 X-GMAIL-MSGID: 1781476273512144311 The rbtree provides a sorting on entries but this is unused. Switch to using hashmap for O(1) rather than O(log n) find/insert/remove complexity. Signed-off-by: Ian Rogers --- tools/perf/util/threads.c | 146 ++++++++++++-------------------------- tools/perf/util/threads.h | 6 +- 2 files changed, 47 insertions(+), 105 deletions(-) diff --git a/tools/perf/util/threads.c b/tools/perf/util/threads.c index d984ec939c7b..55923be53180 100644 --- a/tools/perf/util/threads.c +++ b/tools/perf/util/threads.c @@ -3,25 +3,30 @@ #include "machine.h" #include "thread.h" -struct thread_rb_node { - struct rb_node rb_node; - struct thread *thread; -}; - static struct threads_table_entry *threads__table(struct threads *threads, pid_t tid) { /* Cast it to handle tid == -1 */ return &threads->table[(unsigned int)tid % THREADS__TABLE_SIZE]; } +static size_t key_hash(long key, void *ctx __maybe_unused) +{ + /* The table lookup removes low bit entropy, but this is just ignored here. */ + return key; +} + +static bool key_equal(long key1, long key2, void *ctx __maybe_unused) +{ + return key1 == key2; +} + void threads__init(struct threads *threads) { for (int i = 0; i < THREADS__TABLE_SIZE; i++) { struct threads_table_entry *table = &threads->table[i]; - table->entries = RB_ROOT_CACHED; + hashmap__init(&table->shard, key_hash, key_equal, NULL); init_rwsem(&table->lock); - table->nr = 0; table->last_match = NULL; } } @@ -32,6 +37,7 @@ void threads__exit(struct threads *threads) for (int i = 0; i < THREADS__TABLE_SIZE; i++) { struct threads_table_entry *table = &threads->table[i]; + hashmap__clear(&table->shard); exit_rwsem(&table->lock); } } @@ -44,7 +50,7 @@ size_t threads__nr(struct threads *threads) struct threads_table_entry *table = &threads->table[i]; down_read(&table->lock); - nr += table->nr; + nr += hashmap__size(&table->shard); up_read(&table->lock); } return nr; @@ -86,28 +92,13 @@ static void threads_table_entry__set_last_match(struct threads_table_entry *tabl struct thread *threads__find(struct threads *threads, pid_t tid) { struct threads_table_entry *table = threads__table(threads, tid); - struct rb_node **p; - struct thread *res = NULL; + struct thread *res; down_read(&table->lock); res = __threads_table_entry__get_last_match(table, tid); - if (res) - return res; - - p = &table->entries.rb_root.rb_node; - while (*p != NULL) { - struct rb_node *parent = *p; - struct thread *th = rb_entry(parent, struct thread_rb_node, rb_node)->thread; - - if (thread__tid(th) == tid) { - res = thread__get(th); - break; - } - - if (tid < thread__tid(th)) - p = &(*p)->rb_left; - else - p = &(*p)->rb_right; + if (!res) { + if (hashmap__find(&table->shard, tid, &res)) + res = thread__get(res); } up_read(&table->lock); if (res) @@ -118,49 +109,25 @@ struct thread *threads__find(struct threads *threads, pid_t tid) struct thread *threads__findnew(struct threads *threads, pid_t pid, pid_t tid, bool *created) { struct threads_table_entry *table = threads__table(threads, tid); - struct rb_node **p; - struct rb_node *parent = NULL; struct thread *res = NULL; - struct thread_rb_node *nd; - bool leftmost = true; *created = false; down_write(&table->lock); - p = &table->entries.rb_root.rb_node; - while (*p != NULL) { - struct thread *th; - - parent = *p; - th = rb_entry(parent, struct thread_rb_node, rb_node)->thread; - - if (thread__tid(th) == tid) { - __threads_table_entry__set_last_match(table, th); - res = thread__get(th); - goto out_unlock; - } - - if (tid < thread__tid(th)) - p = &(*p)->rb_left; - else { - leftmost = false; - p = &(*p)->rb_right; - } - } - nd = malloc(sizeof(*nd)); - if (nd == NULL) - goto out_unlock; res = thread__new(pid, tid); - if (!res) - free(nd); - else { - *created = true; - nd->thread = thread__get(res); - rb_link_node(&nd->rb_node, parent, p); - rb_insert_color_cached(&nd->rb_node, &table->entries, leftmost); - ++table->nr; - __threads_table_entry__set_last_match(table, res); + if (res) { + if (hashmap__add(&table->shard, tid, res)) { + /* Add failed. Assume a race so find other entry. */ + thread__put(res); + res = NULL; + if (hashmap__find(&table->shard, tid, &res)) + res = thread__get(res); + } else { + res = thread__get(res); + *created = true; + } + if (res) + __threads_table_entry__set_last_match(table, res); } -out_unlock: up_write(&table->lock); return res; } @@ -169,57 +136,32 @@ void threads__remove_all_threads(struct threads *threads) { for (int i = 0; i < THREADS__TABLE_SIZE; i++) { struct threads_table_entry *table = &threads->table[i]; - struct rb_node *nd; + struct hashmap_entry *cur, *tmp; + size_t bkt; down_write(&table->lock); __threads_table_entry__set_last_match(table, NULL); - nd = rb_first_cached(&table->entries); - while (nd) { - struct thread_rb_node *trb = rb_entry(nd, struct thread_rb_node, rb_node); - - nd = rb_next(nd); - thread__put(trb->thread); - rb_erase_cached(&trb->rb_node, &table->entries); - RB_CLEAR_NODE(&trb->rb_node); - --table->nr; + hashmap__for_each_entry_safe((&table->shard), cur, tmp, bkt) { + struct thread *old_value; - free(trb); + hashmap__delete(&table->shard, cur->key, /*old_key=*/NULL, &old_value); + thread__put(old_value); } - assert(table->nr == 0); up_write(&table->lock); } } void threads__remove(struct threads *threads, struct thread *thread) { - struct rb_node **p; struct threads_table_entry *table = threads__table(threads, thread__tid(thread)); - pid_t tid = thread__tid(thread); + struct thread *old_value; down_write(&table->lock); if (table->last_match && RC_CHK_EQUAL(table->last_match, thread)) __threads_table_entry__set_last_match(table, NULL); - p = &table->entries.rb_root.rb_node; - while (*p != NULL) { - struct rb_node *parent = *p; - struct thread_rb_node *nd = rb_entry(parent, struct thread_rb_node, rb_node); - struct thread *th = nd->thread; - - if (RC_CHK_EQUAL(th, thread)) { - thread__put(nd->thread); - rb_erase_cached(&nd->rb_node, &table->entries); - RB_CLEAR_NODE(&nd->rb_node); - --table->nr; - free(nd); - break; - } - - if (tid < thread__tid(th)) - p = &(*p)->rb_left; - else - p = &(*p)->rb_right; - } + hashmap__delete(&table->shard, thread__tid(thread), /*old_key=*/NULL, &old_value); + thread__put(old_value); up_write(&table->lock); } @@ -229,11 +171,11 @@ int threads__for_each_thread(struct threads *threads, { for (int i = 0; i < THREADS__TABLE_SIZE; i++) { struct threads_table_entry *table = &threads->table[i]; - struct rb_node *nd; + struct hashmap_entry *cur; + size_t bkt; - for (nd = rb_first_cached(&table->entries); nd; nd = rb_next(nd)) { - struct thread_rb_node *trb = rb_entry(nd, struct thread_rb_node, rb_node); - int rc = fn(trb->thread, data); + hashmap__for_each_entry((&table->shard), cur, bkt) { + int rc = fn((struct thread *)cur->pvalue, data); if (rc != 0) return rc; diff --git a/tools/perf/util/threads.h b/tools/perf/util/threads.h index ed67de627578..d03bd91a7769 100644 --- a/tools/perf/util/threads.h +++ b/tools/perf/util/threads.h @@ -2,7 +2,7 @@ #ifndef __PERF_THREADS_H #define __PERF_THREADS_H -#include +#include "hashmap.h" #include "rwsem.h" struct thread; @@ -11,9 +11,9 @@ struct thread; #define THREADS__TABLE_SIZE (1 << THREADS__TABLE_BITS) struct threads_table_entry { - struct rb_root_cached entries; + /* Key is tid, value is struct thread. */ + struct hashmap shard; struct rw_semaphore lock; - unsigned int nr; struct thread *last_match; };