From patchwork Sat Nov 5 02:51:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ho-Ren (Jack) Chuang" X-Patchwork-Id: 15875 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp770932wru; Fri, 4 Nov 2022 19:54:40 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4dblJhJhRX9yT+OlupMjn8PpVLdSXj9CPvjRusJypTd7oTfq/tIwXzyKi1dUf4xmLnPUzK X-Received: by 2002:a17:907:2bf8:b0:7a9:ec45:1697 with SMTP id gv56-20020a1709072bf800b007a9ec451697mr37534987ejc.224.1667616880480; Fri, 04 Nov 2022 19:54:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667616880; cv=none; d=google.com; s=arc-20160816; b=qnkOUQm/3d9KBWLXbHLGv4SHb0XdKQMpqfVdISZinRVtxbXq3S7/49EUtwC7XtFRgA MA9qxUkxpdFZKQouLnlcesKM4IyYJtwwRV33PjFB4pWbMaL+qe4wjOXFMR3heOlp5Hrg A47UYn4Vr7Xhb5kna/x+YLmrOAXwp66MnNfZZaB+jmduCoZMcFWzG4XimLCYbE/D4oZA +5LwC2yxXYncQNrjRsDptBED63TF5gCLncqJ8lWj1dONp31VYoBxLhZsdgtEQuOj8QB/ 8eJTEtxQlgGf3QQ80hrNYlOZuKNf2+vNAfkfS6sX67gXseS5AxUYZ6lSPOQbvDRUTwmj qerw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=YXNmxis4Zkl54qmET11cnpJJGJ2VNVNuEs1M8Jg/V1A=; b=vaYyPf91d4xkX39P6YjACkRySZNQVcqzYAi1SAheZuLaOf6PnRf9Ic0w5OLP9XrDjV xm5FvIf/D7G+p8pmiBIp94C1ttBMT4b8/IB0Lq2dT0Bz06BPGxLWNL9+3F9JPsp3g50F erfBCuXKYpCds4HA3YxbBB2DtVIQO3Zsrct9yvCCk4kcOcpI3VogDZcSzdg2f6YWrLvi mH52CybTRoEZPylaWPMFa4NdeeKIygS3+4kjiyygM+gDYhou3HK37iDAFw+v3xgMUFHq BGDU1xkaIDnIHEAlA/C2XA8u/9AesBKKcCvV6dIuMCo/TRuZJh4ap6G6ZCDbHtNG0+e9 Q0pg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=J339RySF; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=bytedance.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id j8-20020aa7c0c8000000b00461e685ce72si1289539edp.116.2022.11.04.19.54.17; Fri, 04 Nov 2022 19:54:40 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=J339RySF; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=bytedance.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229608AbiKECwc (ORCPT + 99 others); Fri, 4 Nov 2022 22:52:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50870 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229562AbiKECw3 (ORCPT ); Fri, 4 Nov 2022 22:52:29 -0400 Received: from mail-qt1-x831.google.com (mail-qt1-x831.google.com [IPv6:2607:f8b0:4864:20::831]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 076943FB95 for ; Fri, 4 Nov 2022 19:52:28 -0700 (PDT) Received: by mail-qt1-x831.google.com with SMTP id x15so4353282qtv.9 for ; Fri, 04 Nov 2022 19:52:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=YXNmxis4Zkl54qmET11cnpJJGJ2VNVNuEs1M8Jg/V1A=; b=J339RySFJwqav1kH4OJqtPfBCpgE5RHngr1ZByyAiQz4+/YWBABUyUVASMJomr0SYu +/F5foKscg1gwwsCfTKfij2Ydw0jz3qiioA0bCS3Yd3p567ij7S7h5eEAWB0i6OqnisH L7ryIg5jjZMeBqOGXon4/bnjr57Whk/rXmGM7lbqG/sr5mUcQUAlmX0gdAvzcoXjLa4Z 01GPAoepaSNkr50uTKvqsOTte64oVvI8R31TLdS+WN2+y5fZKDowIVv+NMbmCndD0v2k mIzt1voNCiZGEDHKIoDSezNIbrhhg8qL/Y3pWi57CmgFApIhP99sZTHv/Oe44F/HHV8d HWBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YXNmxis4Zkl54qmET11cnpJJGJ2VNVNuEs1M8Jg/V1A=; b=YW1UxUSIJc7M+3+TxEvI0tSt7nVOl/BbpsAaIsrR0hrmw1fEAwwx0cM+1CEFPoPDLq rUb9MlTfFVFiWGMMse2caQdPm0C4Y7VLozKCiWrCn+DTASCQ2Ow2lozisn16BJqLiOWf uaqmubbgo+HpqpHV1SADE/UDV83yOYHWyVsqhBXSUMhnE5PGDK6D3qIQT1L+CcTDyeqI OBddjChiUoITGZ2Z00BfPxB+NueuepqxxMGO2wrOIiQhQjmaDGXMuSkZ44puMXTDB+9g AZhOefhmUhvtW1GJ5PJCjB7UKBfPuGsZEdo90hs8E24UBb4+jQyFKb4IcEOr6rNZUh3V 1NzQ== X-Gm-Message-State: ACrzQf1tS+Ylkp1TrlLfWN8TstIeoat+0PETqie20yiuIofUzRjsi2fY IEWCOGtMOZd0sULL1kOkWeA96g== X-Received: by 2002:a05:622a:5c8d:b0:3a5:1fcb:8326 with SMTP id ge13-20020a05622a5c8d00b003a51fcb8326mr26739483qtb.498.1667616747053; Fri, 04 Nov 2022 19:52:27 -0700 (PDT) Received: from 192-168-53-12.byted.org ([130.44.212.119]) by smtp.gmail.com with ESMTPSA id ay14-20020a05620a178e00b006bb366779a4sm805905qkb.6.2022.11.04.19.52.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Nov 2022 19:52:26 -0700 (PDT) From: "Ho-Ren (Jack) Chuang" To: Alexei Starovoitov , Alexei Starovoitov , Hao Luo , Jiri Olsa , Jiri Olsa , Andrii Nakryiko , Daniel Borkmann , John Fastabend , Martin KaFai Lau , Song Liu , Yonghong Song , KP Singh , Stanislav Fomichev , Quentin Monnet , Mykola Lysenko , Shuah Khan , Nathan Chancellor , Nick Desaulniers , Tom Rix , Joanne Koong , Kui-Feng Lee , Lorenzo Bianconi , Maxim Mikityanskiy , Hao Xiang , Punit Agrawal , Yifei Ma , Xiaoning Ding , bpf@vger.kernel.org Cc: Ho-Ren Chuang , Ho-Ren Chuang , linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, llvm@lists.linux.dev Subject: [PATCH bpf-next v1 1/4] bpf: Support reporting BPF htab map's used size for monitoring Date: Sat, 5 Nov 2022 02:51:43 +0000 Message-Id: <20221105025146.238209-2-horenchuang@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20221105025146.238209-1-horenchuang@bytedance.com> References: <20221105025146.238209-1-horenchuang@bytedance.com> MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748623037939772294?= X-GMAIL-MSGID: =?utf-8?q?1748623037939772294?= Expose BPF htab map's used size by counting accessed or allocated/freed elements to userspace. Leverage the htab->count value for both preallocated and dynamically allocated maps. Expose the value to a new field "used_entries" in a userspace struct bpf_map_info to allow monitoring. Support hash table type (BPF_MAP_TYPE_HASH). Signed-off-by: Ho-Ren (Jack) Chuang --- include/linux/bpf.h | 1 + include/uapi/linux/bpf.h | 1 + kernel/bpf/hashtab.c | 19 +++++++++++++++++++ kernel/bpf/syscall.c | 2 ++ 4 files changed, 23 insertions(+) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 9e7d46d16032..82ee14139b69 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -97,6 +97,7 @@ struct bpf_map_ops { int (*map_pop_elem)(struct bpf_map *map, void *value); int (*map_peek_elem)(struct bpf_map *map, void *value); void *(*map_lookup_percpu_elem)(struct bpf_map *map, void *key, u32 cpu); + u32 (*map_get_used_elem)(struct bpf_map *map); /* funcs called by prog_array and perf_event_array map */ void *(*map_fd_get_ptr)(struct bpf_map *map, struct file *map_file, diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 17f61338f8f8..63659368cf0e 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -6215,6 +6215,7 @@ struct bpf_map_info { __u32 id; __u32 key_size; __u32 value_size; + __u32 used_entries; __u32 max_entries; __u32 map_flags; char name[BPF_OBJ_NAME_LEN]; diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c index ed3f8a53603b..bc9c00b92e57 100644 --- a/kernel/bpf/hashtab.c +++ b/kernel/bpf/hashtab.c @@ -913,6 +913,7 @@ static void free_htab_elem(struct bpf_htab *htab, struct htab_elem *l) if (htab_is_prealloc(htab)) { check_and_free_fields(htab, l); __pcpu_freelist_push(&htab->freelist, &l->fnode); + dec_elem_count(htab); } else { dec_elem_count(htab); htab_elem_free(htab, l); @@ -994,6 +995,7 @@ static struct htab_elem *alloc_htab_elem(struct bpf_htab *htab, void *key, if (!l) return ERR_PTR(-E2BIG); l_new = container_of(l, struct htab_elem, fnode); + inc_elem_count(htab); } } else { if (is_map_full(htab)) @@ -2186,6 +2188,22 @@ static int bpf_for_each_hash_elem(struct bpf_map *map, bpf_callback_t callback_f return num_elems; } +u32 htab_map_get_used_elem(struct bpf_map *map) +{ + struct bpf_htab *htab = container_of(map, struct bpf_htab, map); + + /* The elem count may temporarily go beyond the max after + * inc_elem_count() but before dec_elem_count(). + */ + if (htab->use_percpu_counter) + return min_t(u32, htab->map.max_entries, + percpu_counter_sum(&htab->pcount) + + atomic_read(&htab->count)); + else + return min_t(u32, htab->map.max_entries, + atomic_read(&htab->count)); +} + BTF_ID_LIST_SINGLE(htab_map_btf_ids, struct, bpf_htab) const struct bpf_map_ops htab_map_ops = { .map_meta_equal = bpf_map_meta_equal, @@ -2202,6 +2220,7 @@ const struct bpf_map_ops htab_map_ops = { .map_seq_show_elem = htab_map_seq_show_elem, .map_set_for_each_callback_args = map_set_for_each_callback_args, .map_for_each_callback = bpf_for_each_hash_elem, + .map_get_used_elem = htab_map_get_used_elem, BATCH_OPS(htab), .map_btf_id = &htab_map_btf_ids[0], .iter_seq_info = &iter_seq_info, diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 7b373a5e861f..ea4828bb22ac 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -4203,6 +4203,8 @@ static int bpf_map_get_info_by_fd(struct file *file, info.map_flags = map->map_flags; info.map_extra = map->map_extra; memcpy(info.name, map->name, sizeof(map->name)); + if (map->ops->map_get_used_elem) + info.used_entries = map->ops->map_get_used_elem(map); if (map->btf) { info.btf_id = btf_obj_id(map->btf); From patchwork Sat Nov 5 02:51:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ho-Ren (Jack) Chuang" X-Patchwork-Id: 15873 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp770824wru; Fri, 4 Nov 2022 19:54:17 -0700 (PDT) X-Google-Smtp-Source: AMsMyM64uswY7Os17ACkr0m24brWe/ZzeC52ocWwGAbrkJDQGrzISwAMKUl+BlUsBsXPP1BiUDBD X-Received: by 2002:a17:907:6d9a:b0:7a0:d2d9:31cc with SMTP id sb26-20020a1709076d9a00b007a0d2d931ccmr37488261ejc.747.1667616857823; Fri, 04 Nov 2022 19:54:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667616857; cv=none; d=google.com; s=arc-20160816; b=TS4cvo9V5a0lVhJHwhkA3C9EvcecwXkN14d1gzH/5Q1/7Hl4ab1/pWEyYN/T/hYLV9 EnTV2NQqIzGO6N8DpasnVIvVGXe9SWGY4RDTqacNyDkUTedrpu+bq+tz0eI+rhU88WSU iti8hpjk+tCKZLvfYmtFh/yh8TnDeXpg/OIAFZr31abKRA/jYgymGcqgCj4BXqWbBbFH oBpXGIPCpFtYnc3EcTBu+mXEbuCCVP60lMLQA4SfTSTlkFS7TGeUlRKKTEDo5eYOShLY A4Znrz0xoDbgF6qzMsj0Nuwhb6t1K7wFE+bq3WkiM4AzF5fj7iF5MV92AISaTHBLJ8nB o5fQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=8AM+TWzrl1LA62BHe0rCl520VllgtghOrkiBcfVyQb4=; b=0LlTFuw3mpGc3/y0jn/dd56xC51gkw67uukNlTKzXA96aSHHu7uQxRH1yamElnpoLc 9kA4I/uBbRFjz/BetvM+nN+v8FOg13loMOcRdbXTCwFWNaeiQgDcQjaRzrHCpe3EFin4 IDH2z64Df0wUpUYxteAgU5KiyeJri6gE8DzENLElb3GEEo316AA58vihipESe782IYI6 F0JOHC8ltdBIews+VACihz6a3fZpLhFpyQKiB9brWTkR8x7/Kf5jcK6Kt8M3WZmjMNA1 SVrR6HWToxqFnrLDMyc5iAN74EQ7+er9Ok6UOTTTMNrg0plXdYDqCzmDJv49L7eV4RTk HJPg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=k0Jh0Xkl; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=bytedance.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id b15-20020a17090630cf00b0072ef52514f6si747328ejb.629.2022.11.04.19.53.55; Fri, 04 Nov 2022 19:54:17 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=k0Jh0Xkl; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=bytedance.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229635AbiKECwl (ORCPT + 99 others); Fri, 4 Nov 2022 22:52:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50882 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229582AbiKECwa (ORCPT ); Fri, 4 Nov 2022 22:52:30 -0400 Received: from mail-qk1-x729.google.com (mail-qk1-x729.google.com [IPv6:2607:f8b0:4864:20::729]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2B095419B8 for ; Fri, 4 Nov 2022 19:52:29 -0700 (PDT) Received: by mail-qk1-x729.google.com with SMTP id k4so4435242qkj.8 for ; Fri, 04 Nov 2022 19:52:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8AM+TWzrl1LA62BHe0rCl520VllgtghOrkiBcfVyQb4=; b=k0Jh0XkldmG8RANLS2K0+hj/5w/Kw+FdUYVrURoqA7WDh0Cm+uv9mOX10JkCNnS8NJ HnB4UGIL5WLZvvMH3MsOcD+Rf5i9kIMyTb0m/u7WvBBMjDxVshVSlBXaVUxbQYaorf4Z OLcdorfXmCYAdSAhlx0C89D8FytECOj9mSMNpOvT6YfMUIkDht2u5TNrPzYPjjFbeVDb v0mIxpscJB/OfPsp7LQOW2k22IH6mPkz5lPZgj4qFhS8ib1LgKpnXpkxSKLhoewt0uT9 iKao+31zZbLgnTvRyZWUSR+EzAhR2j5PPyZ1MVEAZt8LKQab3ub+qFmnTFYeJiF49ZJi RXZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8AM+TWzrl1LA62BHe0rCl520VllgtghOrkiBcfVyQb4=; b=U5AmWMmlkflynj0TTFU12R2XxNw3RyJYj6qPorJ49RzyPu/8cZJlCnwoMPfQ7jjDB5 7hYmILVtDeefJTg8ABKow+hxV9clWnNfSgxOyx4XXTfqDIwecLRJ/1kTF8kJRr4YuV7t FGZGaWH6enx5FaTmvHvWtW3fuRg8QrxbmIhlxgqavHL5UXoJWVrZMgeccHHd4Gs/Qr62 yo1VnfFj7yAvu3GVIdP05ptyQURTu6xtBTzQMAaT0UQ0ORmZw60cN29pEF8mL83/a+/A bnXERC2OgzveA3gcn1W38Omx1SwNZ0hLEHbSK25vFcBJNGs2DUTPq5LJgQzaXYe6jUsp 9g3w== X-Gm-Message-State: ACrzQf3gLU9oZDrWbAxIkvNI+O77z1+8lAHyrOnXLej9SlAzsUnORUp7 GEqazd8uCfA4yhtgOM6JsIo5NA== X-Received: by 2002:a05:620a:21c9:b0:6fa:3046:42c0 with SMTP id h9-20020a05620a21c900b006fa304642c0mr21689269qka.628.1667616747823; Fri, 04 Nov 2022 19:52:27 -0700 (PDT) Received: from 192-168-53-12.byted.org ([130.44.212.119]) by smtp.gmail.com with ESMTPSA id ay14-20020a05620a178e00b006bb366779a4sm805905qkb.6.2022.11.04.19.52.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Nov 2022 19:52:27 -0700 (PDT) From: "Ho-Ren (Jack) Chuang" To: Alexei Starovoitov , Alexei Starovoitov , Hao Luo , Jiri Olsa , Jiri Olsa , Andrii Nakryiko , Daniel Borkmann , John Fastabend , Martin KaFai Lau , Song Liu , Yonghong Song , KP Singh , Stanislav Fomichev , Quentin Monnet , Mykola Lysenko , Shuah Khan , Nathan Chancellor , Nick Desaulniers , Tom Rix , Joanne Koong , Kui-Feng Lee , Lorenzo Bianconi , Maxim Mikityanskiy , Hao Xiang , Punit Agrawal , Yifei Ma , Xiaoning Ding , bpf@vger.kernel.org Cc: Ho-Ren Chuang , Ho-Ren Chuang , linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, llvm@lists.linux.dev Subject: [PATCH bpf-next v1 2/4] bpftool: Add tools support to show BPF htab map's used size Date: Sat, 5 Nov 2022 02:51:44 +0000 Message-Id: <20221105025146.238209-3-horenchuang@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20221105025146.238209-1-horenchuang@bytedance.com> References: <20221105025146.238209-1-horenchuang@bytedance.com> MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748623014150223307?= X-GMAIL-MSGID: =?utf-8?q?1748623014150223307?= Add bpftool support for reporting the number of used entries of htab maps by leveraging the newly added "used_entries" field in struct bpf_map_info. It works with JSON as well. To better understand actual used memory size of a htab map, pre-allocated maps are now marked with "*" behind the "max_entries" size. Signed-off-by: Ho-Ren (Jack) Chuang --- tools/bpf/bpftool/map.c | 9 +++++++-- tools/include/uapi/linux/bpf.h | 1 + 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/tools/bpf/bpftool/map.c b/tools/bpf/bpftool/map.c index 9a6ca9f31133..0b07abae7309 100644 --- a/tools/bpf/bpftool/map.c +++ b/tools/bpf/bpftool/map.c @@ -475,6 +475,8 @@ static int show_map_close_json(int fd, struct bpf_map_info *info) jsonw_uint_field(json_wtr, "bytes_key", info->key_size); jsonw_uint_field(json_wtr, "bytes_value", info->value_size); jsonw_uint_field(json_wtr, "max_entries", info->max_entries); + if (info->type == BPF_MAP_TYPE_HASH) + jsonw_uint_field(json_wtr, "used_entries", info->used_entries); if (memlock) jsonw_int_field(json_wtr, "bytes_memlock", atoll(memlock)); @@ -561,8 +563,11 @@ static int show_map_close_plain(int fd, struct bpf_map_info *info) frozen_str = get_fdinfo(fd, "frozen"); show_map_header_plain(info); - printf("\tkey %uB value %uB max_entries %u", - info->key_size, info->value_size, info->max_entries); + printf("\tkey %uB value %uB max_entries %u%1s", + info->key_size, info->value_size, info->max_entries, + !(info->map_flags & BPF_F_NO_PREALLOC) ? "*" : ""); + if (info->type == BPF_MAP_TYPE_HASH) + printf(" used_entries %u", info->used_entries); if (memlock) printf(" memlock %sB", memlock); diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index 17f61338f8f8..63659368cf0e 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -6215,6 +6215,7 @@ struct bpf_map_info { __u32 id; __u32 key_size; __u32 value_size; + __u32 used_entries; __u32 max_entries; __u32 map_flags; char name[BPF_OBJ_NAME_LEN]; From patchwork Sat Nov 5 02:51:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ho-Ren (Jack) Chuang" X-Patchwork-Id: 15874 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp770830wru; Fri, 4 Nov 2022 19:54:19 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4jvuTsnRt2AVfZlE/bdPQ12oyxFadY4EmhImZFzZqcN5L4683+fpQlLQeOBAtBIDqgOVfj X-Received: by 2002:a17:906:fd8a:b0:75d:c79a:47c8 with SMTP id xa10-20020a170906fd8a00b0075dc79a47c8mr36451579ejb.389.1667616859063; Fri, 04 Nov 2022 19:54:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667616859; cv=none; d=google.com; s=arc-20160816; b=a4wmWioPtHxxJ3xzLebL22ZiQ/pwGc4OPstOIZXqBOee5L9cCK6FA7sNo7cgAbDJvE ZWqHAGpMHNjWhBTVr7w4ZowOSOcaunC10e5GKg1JuAvEmKR7NvaWjB9LdJUcdZ3ywWnK beACNP/bVMp0shDiWB3+K20gLyhYSX3feaniNoFvnaPTgc2e7wo6amzZE/bzG6PtaqEw z4Z1F1Teet8RL5Eq8/1RzxBs5i1GjtysCk3D8cGTJV6JBi11/Ix3iE3xpjLaxTcfb1Ha iHTokWDT0E+hb4p7P/FkXRretT68bj8KI45K47BxmlEGPPKh0R4uPv4TyyIpcQmUkrg8 zaqA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Wl8oQUpe8yXqSeuRtfUsJkpIGEQ98nwCnE5YcZtf2Pk=; b=1Aat4+Ree9frYSgrdT8emdxlE84rbzz7kSM2cgRMFr54gzpTo9QNKnDmH7qvYbx0/d jyfQkFOJiptKxVxxPeETiMgjCEn/4Wj2rqil5bV3t42KuZn78HjMqaBIbAad2SaWY3K8 7C/X7sFHl4OCufKc0Xjr0gmgSBp8rimye3zYg8dQThTQ6ABcWzjAiEuctlUZbOlcC5k+ 9p/n0SeyRiRyZflLEeSyiDTetE7tHM4ddi+tPy1oSF+oY/r5ykqLqm1iqdbZirc8iQKZ m/tW/bctFu9tZQUHpfT+gY1+cp9f/OItyBgZ/bWhuqZ/bnJWerWMPsHJxn82Z1j44gBc glSw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=D3Efmuf4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=bytedance.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id he44-20020a1709073dac00b007ae0e8f59a6si892277ejc.821.2022.11.04.19.53.56; Fri, 04 Nov 2022 19:54:19 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=D3Efmuf4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=bytedance.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229645AbiKECwm (ORCPT + 99 others); Fri, 4 Nov 2022 22:52:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50898 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229588AbiKECwa (ORCPT ); Fri, 4 Nov 2022 22:52:30 -0400 Received: from mail-qt1-x82f.google.com (mail-qt1-x82f.google.com [IPv6:2607:f8b0:4864:20::82f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 931773FB80 for ; Fri, 4 Nov 2022 19:52:29 -0700 (PDT) Received: by mail-qt1-x82f.google.com with SMTP id hh9so4335936qtb.13 for ; Fri, 04 Nov 2022 19:52:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Wl8oQUpe8yXqSeuRtfUsJkpIGEQ98nwCnE5YcZtf2Pk=; b=D3Efmuf4WRWk4XBr8PaHosbwLBamqCB1Qq1UsUIMQ1cYAddKnpMt9REEdYbmcxPJfv mvu8px+Pp135sgK1Qrx/XyhjPfeEKOZQ2bWTNKLeAtKePRk312c/yRWK//4J1WuP3Rx7 Flmzd2PuZ3ZkysxXlU87KxkQz3vaE0Ldi5tRP4jUNvfPSf/tipTYPsGVQ+r+llHYbL78 sQvnXCg8kpohgS26Uk8QPY7Xcnt3mIcczIEnjl42LVuhQM8RYGLOer04tSFCrfeT5qPK dMTz5O5LicnnSdKZxl/36cwUf6fpIzRTo69ZPQl+xsz9nxmWyNyAW0NWAztu2irqpF52 OCew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Wl8oQUpe8yXqSeuRtfUsJkpIGEQ98nwCnE5YcZtf2Pk=; b=uPH+ZqRDOP6OLXZHTzK8I8gH3iRuOqmuv8HiY9GlLuXu9cFDctT1Md4Mwk0LHboF9e 6hrbmMmmVbk2zg+BkDbdxODKJmbKG2WhhW3zOWEiBo24Z1seaxRRMpaoiQQz+1ak4EIC mkZUkpe+729o7qFn6yavQEuhMbW0ZZ1CK61fDfIrthA1ydnxNnbZmygBxXlGWSC1tk6j oHP5v9OAI75Lc0YYmpC5XSHPF6KFf11Qo0ueZ6NxKnBTlDMu2Mfi2IDpc0AGiT/uRCfS BicWwANsqKnJgyxr5NTGhtSVUyAdnF4kItAxb/hZ7sJ/cAQ+BBWsrTywh3BJsSXxHso0 GhAQ== X-Gm-Message-State: ACrzQf057c6muQCmOutbLdD7FaXB53JbD5g9/kbehMNtkSwsQ1s71B// g38dQBdbk/JCvbCRN84WytZFlQ== X-Received: by 2002:ac8:6e86:0:b0:3a5:27f8:f3db with SMTP id c6-20020ac86e86000000b003a527f8f3dbmr23498116qtv.97.1667616748585; Fri, 04 Nov 2022 19:52:28 -0700 (PDT) Received: from 192-168-53-12.byted.org ([130.44.212.119]) by smtp.gmail.com with ESMTPSA id ay14-20020a05620a178e00b006bb366779a4sm805905qkb.6.2022.11.04.19.52.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Nov 2022 19:52:28 -0700 (PDT) From: "Ho-Ren (Jack) Chuang" To: Alexei Starovoitov , Alexei Starovoitov , Hao Luo , Jiri Olsa , Jiri Olsa , Andrii Nakryiko , Daniel Borkmann , John Fastabend , Martin KaFai Lau , Song Liu , Yonghong Song , KP Singh , Stanislav Fomichev , Quentin Monnet , Mykola Lysenko , Shuah Khan , Nathan Chancellor , Nick Desaulniers , Tom Rix , Joanne Koong , Kui-Feng Lee , Lorenzo Bianconi , Maxim Mikityanskiy , Hao Xiang , Punit Agrawal , Yifei Ma , Xiaoning Ding , bpf@vger.kernel.org Cc: Ho-Ren Chuang , Ho-Ren Chuang , linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, llvm@lists.linux.dev Subject: [PATCH bpf-next v1 3/4] samples/bpf: Add concurrency testing for BPF htab map's used size Date: Sat, 5 Nov 2022 02:51:45 +0000 Message-Id: <20221105025146.238209-4-horenchuang@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20221105025146.238209-1-horenchuang@bytedance.com> References: <20221105025146.238209-1-horenchuang@bytedance.com> MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748623015305144199?= X-GMAIL-MSGID: =?utf-8?q?1748623015305144199?= Add htab map's used_size test cases for concurrency testing. Support hash table type (BPF_MAP_TYPE_HASH). Signed-off-by: Ho-Ren (Jack) Chuang --- samples/bpf/Makefile | 4 + samples/bpf/test_map_used_kern.c | 65 ++++++++++ samples/bpf/test_map_used_user.c | 204 +++++++++++++++++++++++++++++++ 3 files changed, 273 insertions(+) create mode 100644 samples/bpf/test_map_used_kern.c create mode 100644 samples/bpf/test_map_used_user.c diff --git a/samples/bpf/Makefile b/samples/bpf/Makefile index 727da3c5879b..8725d0d64a21 100644 --- a/samples/bpf/Makefile +++ b/samples/bpf/Makefile @@ -40,6 +40,7 @@ tprogs-y += tc_l2_redirect tprogs-y += lwt_len_hist tprogs-y += xdp_tx_iptunnel tprogs-y += test_map_in_map +tprogs-y += test_map_used tprogs-y += per_socket_stats_example tprogs-y += xdp_rxq_info tprogs-y += syscall_tp @@ -101,6 +102,7 @@ tc_l2_redirect-objs := tc_l2_redirect_user.o lwt_len_hist-objs := lwt_len_hist_user.o xdp_tx_iptunnel-objs := xdp_tx_iptunnel_user.o test_map_in_map-objs := test_map_in_map_user.o +test_map_used-objs := test_map_used_user.o per_socket_stats_example-objs := cookie_uid_helper_example.o xdp_rxq_info-objs := xdp_rxq_info_user.o syscall_tp-objs := syscall_tp_user.o @@ -153,6 +155,7 @@ always-y += sampleip_kern.o always-y += lwt_len_hist_kern.o always-y += xdp_tx_iptunnel_kern.o always-y += test_map_in_map_kern.o +always-y += test_map_used_kern.o always-y += tcp_synrto_kern.o always-y += tcp_rwnd_kern.o always-y += tcp_bufs_kern.o @@ -216,6 +219,7 @@ TPROGLDLIBS_xdp_router_ipv4 += -lm -pthread TPROGLDLIBS_tracex4 += -lrt TPROGLDLIBS_trace_output += -lrt TPROGLDLIBS_map_perf_test += -lrt +TPROGLDLIBS_test_map_used += -lrt TPROGLDLIBS_test_overhead += -lrt # Allows pointing LLC/CLANG to a LLVM backend with bpf support, redefine on cmdline: diff --git a/samples/bpf/test_map_used_kern.c b/samples/bpf/test_map_used_kern.c new file mode 100644 index 000000000000..e908593c1f09 --- /dev/null +++ b/samples/bpf/test_map_used_kern.c @@ -0,0 +1,65 @@ +/* Copyright (c) 2022 ByteDance + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of version 2 of the GNU General Public + * License as published by the Free Software Foundation. + */ +#include +#include +#include +#include +#include +#include +#include "trace_common.h" + +#define MAX_ENTRIES 1000 + +struct { + __uint(type, BPF_MAP_TYPE_HASH); + __type(key, u32); + __type(value, long); + __uint(max_entries, MAX_ENTRIES); + __uint(map_flags, BPF_F_NO_PREALLOC); +} touch_hash_no_prealloc SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_HASH); + __type(key, u32); + __type(value, long); + __uint(max_entries, MAX_ENTRIES); +} touch_hash_prealloc SEC(".maps"); + +SEC("kprobe/" SYSCALL(sys_mount)) +int stress_hmap_alloc(struct pt_regs *ctx) +{ + u32 key, i; + long init_val = bpf_get_current_pid_tgid(); + +#pragma clang loop unroll(full) + for (i = 0; i < MAX_ENTRIES; ++i) { + key = i; + bpf_map_update_elem(&touch_hash_no_prealloc, + &key, &init_val, BPF_ANY); + } + + return 0; +} + +SEC("kprobe/" SYSCALL(sys_umount)) +int stress_hmap_prealloc(struct pt_regs *ctx) +{ + u32 key, i; + long init_val = bpf_get_current_pid_tgid(); + +#pragma clang loop unroll(full) + for (i = 0; i < MAX_ENTRIES; ++i) { + key = i; + bpf_map_update_elem(&touch_hash_prealloc, + &key, &init_val, BPF_ANY); + } + + return 0; +} + +char _license[] SEC("license") = "GPL"; +u32 _version SEC("version") = LINUX_VERSION_CODE; diff --git a/samples/bpf/test_map_used_user.c b/samples/bpf/test_map_used_user.c new file mode 100644 index 000000000000..797f6ca7434d --- /dev/null +++ b/samples/bpf/test_map_used_user.c @@ -0,0 +1,204 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* Copyright (c) 2022 ByteDance + */ +#define _GNU_SOURCE +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +#define TEST_BIT(t) (1U << (t)) +#define MAX_NR_CPUS 1024 + +static __u64 time_get_ns(void) +{ + struct timespec ts; + + clock_gettime(CLOCK_MONOTONIC, &ts); + return ts.tv_sec * 1000000000ull + ts.tv_nsec; +} + +enum test_type { + HASH_TOUCH_PREALLOC, + HASH_TOUCH, + NR_TESTS, +}; + +const char *test_map_names[NR_TESTS] = { + [HASH_TOUCH_PREALLOC] = "hash_map", + [HASH_TOUCH] = "hash_map", +}; + +static int test_flags = ~0; +static __u32 num_map_entries; +static __u32 inner_lru_hash_size; +static __u32 max_cnt = 1000; + +static int check_test_flags(enum test_type t) +{ + return test_flags & TEST_BIT(t); +} + +static void test_hash_touch_prealloc(int cpu) +{ + __u64 start_time; + int i; + + start_time = time_get_ns(); + for (i = 0; i < max_cnt; i++) + syscall(__NR_umount2, NULL, 0); + printf("%d:hash_touch pre-alloc %lld touches per sec\n", + cpu, max_cnt * 1000000000ll / (time_get_ns() - start_time)); +} + +static void test_hash_touch(int cpu) +{ + __u64 start_time; + int i; + + start_time = time_get_ns(); + for (i = 0; i < max_cnt; i++) + syscall(__NR_mount, NULL, NULL, NULL, 0, NULL); + printf("%d:hash_touch %lld touchess per sec\n", + cpu, max_cnt * 1000000000ll * 64 / (time_get_ns() - start_time)); +} + +typedef void (*test_func)(int cpu); +const test_func test_funcs[] = { + [HASH_TOUCH_PREALLOC] = test_hash_touch_prealloc, + [HASH_TOUCH] = test_hash_touch, +}; + +static void loop(int cpu) +{ + cpu_set_t cpuset; + int i; + + CPU_ZERO(&cpuset); + CPU_SET(cpu, &cpuset); + sched_setaffinity(0, sizeof(cpuset), &cpuset); + + for (i = 0; i < NR_TESTS; i++) { + if (check_test_flags(i)) + test_funcs[i](cpu); + } +} + +static void run_perf_test(int tasks) +{ + pid_t pid[tasks]; + int i; + + for (i = 0; i < tasks; i++) { + pid[i] = fork(); + if (pid[i]) + printf("Spawn process #%d [%u]\n", i, pid[i]); + + if (pid[i] == 0) { + loop(i); + exit(0); + } else if (pid[i] == -1) { + printf("couldn't spawn #%d process\n", i); + exit(1); + } + } + for (i = 0; i < tasks; i++) { + int status; + + assert(waitpid(pid[i], &status, 0) == pid[i]); + assert(status == 0); + } +} + +static void fixup_map(struct bpf_object *obj) +{ + struct bpf_map *map; + int i; + + bpf_object__for_each_map(map, obj) { + const char *name = bpf_map__name(map); + + /* Only change the max_entries for the enabled test(s) */ + for (i = 0; i < NR_TESTS; i++) { + if (!strcmp(test_map_names[i], name) && + (check_test_flags(i))) { + bpf_map__set_max_entries(map, num_map_entries); + continue; + } + } + } + + inner_lru_hash_size = num_map_entries; +} + +int main(int argc, char **argv) +{ + int nr_cpus = sysconf(_SC_NPROCESSORS_ONLN); + struct bpf_link *links[8]; + struct bpf_program *prog; + struct bpf_object *obj; + char filename[256]; + int i = 0; + + if (argc > 1) + test_flags = atoi(argv[1]) ? : test_flags; + + if (argc > 2) + nr_cpus = atoi(argv[2]) ? : nr_cpus; + + if (argc > 3) + num_map_entries = atoi(argv[3]); + + if (argc > 4) + max_cnt = atoi(argv[4]); + + snprintf(filename, sizeof(filename), "%s_kern.o", argv[0]); + obj = bpf_object__open_file(filename, NULL); + if (libbpf_get_error(obj)) { + fprintf(stderr, "ERROR: opening BPF object file failed\n"); + return 0; + } + + /* resize BPF map prior to loading */ + if (num_map_entries > 0) + fixup_map(obj); + + /* load BPF program */ + if (bpf_object__load(obj)) { + fprintf(stderr, "ERROR: loading BPF object file failed\n"); + goto cleanup; + } + + bpf_object__for_each_program(prog, obj) { + links[i] = bpf_program__attach(prog); + if (libbpf_get_error(links[i])) { + fprintf(stderr, "ERROR: bpf_program__attach failed\n"); + links[i] = NULL; + goto cleanup; + } + i++; + } + + run_perf_test(nr_cpus); + +cleanup: + for (i--; i >= 0; i--) + bpf_link__destroy(links[i]); + + bpf_object__close(obj); + return 0; +} From patchwork Sat Nov 5 02:51:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ho-Ren (Jack) Chuang" X-Patchwork-Id: 15876 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp771006wru; Fri, 4 Nov 2022 19:54:57 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6YQ2/3epl6sSdfXHVmF9BjLEcWwSP7x5o+BLCo2IRyxTRn8dS3AzBYkG/fbPErmGqQpmOU X-Received: by 2002:a17:907:6d8c:b0:7ad:b45c:dd24 with SMTP id sb12-20020a1709076d8c00b007adb45cdd24mr34671028ejc.616.1667616897421; Fri, 04 Nov 2022 19:54:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667616897; cv=none; d=google.com; s=arc-20160816; b=J60h4ERgi3wEexWewwcHM78TDsaaK5ZeU57yM2nbz9kzbgx1uoMNalPzPhNnByj+4s o0K7Ub53GNFxb/sNd3J8PmQ2dDPRzK+ATUpHLaDCMJa4av+OyYjs54VV2C+Uuf7yeede ftlGLOKVTmTvnerVUH6JLwnBvnOT2sbFs0XSLtR6Ph47hBk/pvIeLEonPOrQ0metB29P uIEuuxDzmWM7gWMeRv9R8ADa00w4BUUMKbLr32hjvlCpxS+2UZSuChX6zJGf+Y0AiNAn a5pNR0WJTV2w3rj9frWh9thVhI/uABGyJBWl2cJh8L9QaMOHLO4x6pxRF7Z6GLodIUtP o9JA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=102zPobI2UCzZFYJfQ6W0YbWDm+eTOkX0gf/oCnQz2U=; b=uguIJZuC6RsDDg9wlEJDAP0uFIWkWeKBd6f5KSKHWnJmoFmsPpTthYEiPpRe1kXRtD tqjIKHAYMdBopIn/sUKN6IJsNzmPmaayj3RkX9ZS4M8UAGRtwc8xTLESZXrsV+dSJLbQ ikgIKMa82t3ai9Z/fcw7mLAp5j6n8fJzMIDjwcjTuOuW1pz1X9e1Tj6BkUADE1uwBynq ojl+nrt9jd4wbdJaUED9IZKeeOXlOb3pWDnoLLQyHrmRDFFs+TFyAFCUzDcel67BsWu2 XLtZfWtj0lMdrOpIvSeA3xVWKhxY/4VQCq4r/mxQ8h+jwpyXl3oRq9WgwIVL0W6kRPRS 56uQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=BhkDD2m+; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=bytedance.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id hb35-20020a170907162300b007ae26c33a6fsi1004728ejc.685.2022.11.04.19.54.34; Fri, 04 Nov 2022 19:54:57 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=BhkDD2m+; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=bytedance.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229676AbiKECwq (ORCPT + 99 others); Fri, 4 Nov 2022 22:52:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50882 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229610AbiKECwc (ORCPT ); Fri, 4 Nov 2022 22:52:32 -0400 Received: from mail-qk1-x72b.google.com (mail-qk1-x72b.google.com [IPv6:2607:f8b0:4864:20::72b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 764363FB95 for ; Fri, 4 Nov 2022 19:52:30 -0700 (PDT) Received: by mail-qk1-x72b.google.com with SMTP id z30so4410436qkz.13 for ; Fri, 04 Nov 2022 19:52:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=102zPobI2UCzZFYJfQ6W0YbWDm+eTOkX0gf/oCnQz2U=; b=BhkDD2m+dHxDTTE8PUg8n+8FDifwARJY/czahSUEz9t0qC/XYnI6eDSQvxa7uhHeGv l08NeotC1Xn9p98wFICv+UbKcRVQYXoIhe0yXVc0GWVq9rH7+pkF+jj1pIS7Hyd45Br2 3AQVwp/IB+SHvc+DhyFoH6lYq9HLAJIdT3/l96XmNmNmjusI9QuDKQj5kgDMOGfNHRM/ 6zL1O6bIOT2BnuZlVOhF1QyL0ZUpUs548iOfJWCDFE2zkoMVk9UXe2e6PMyGUvsEmzOJ yLKcRsOtw8NQfbgdQu5Ky4+h8EUjMm1hFDA7AUIanfg7gG+4tmQn65gWEb9jaSpN983A J/Fw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=102zPobI2UCzZFYJfQ6W0YbWDm+eTOkX0gf/oCnQz2U=; b=1i8BdziaDQv4woH6HMUILJWpW1y6OQ3PQPvfYYLcqpJ8m7AjNl6iRUYbLSD0qgpXXC pnMS0OiZdqx0XewfSvs+fe3H4gFqNneebe3Lskpz8a+GTkrEesCJ0l4KSppiL+o7pYLA AHNWaiSvZ5lpygajrsUg8MR+noegEw6xiL4uy5ogzGb1Nx9IZrgPLaBKdaWH0MKIm52z cauFSLxXSN7+xQghi+TykfaO7N+T1ABXc09TUEuPseBx7VwiDR9dqf3cyg7gcKyoRBCi 5yLwqQG3deczIc6Mw5gRf4A07u9Zz5cGr2f79IRd6X6nZlPYPOHlcTWHKbdAyt1jjMWh JNPQ== X-Gm-Message-State: ACrzQf2y5F96BVxL6dsXYsq5UqHC4slo9vQsK78S9esf8JeTNRYmBJ3q kWRxkCN4xt/3xCQ+AWFhC2k8PA== X-Received: by 2002:a37:69c5:0:b0:6fa:d6f:e848 with SMTP id e188-20020a3769c5000000b006fa0d6fe848mr27896590qkc.17.1667616749442; Fri, 04 Nov 2022 19:52:29 -0700 (PDT) Received: from 192-168-53-12.byted.org ([130.44.212.119]) by smtp.gmail.com with ESMTPSA id ay14-20020a05620a178e00b006bb366779a4sm805905qkb.6.2022.11.04.19.52.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Nov 2022 19:52:29 -0700 (PDT) From: "Ho-Ren (Jack) Chuang" To: Alexei Starovoitov , Alexei Starovoitov , Hao Luo , Jiri Olsa , Jiri Olsa , Andrii Nakryiko , Daniel Borkmann , John Fastabend , Martin KaFai Lau , Song Liu , Yonghong Song , KP Singh , Stanislav Fomichev , Quentin Monnet , Mykola Lysenko , Shuah Khan , Nathan Chancellor , Nick Desaulniers , Tom Rix , Joanne Koong , Kui-Feng Lee , Lorenzo Bianconi , Maxim Mikityanskiy , Hao Xiang , Punit Agrawal , Yifei Ma , Xiaoning Ding , bpf@vger.kernel.org Cc: Ho-Ren Chuang , Ho-Ren Chuang , linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, llvm@lists.linux.dev Subject: [PATCH bpf-next v1 4/4] selftests/bpf: Add unit tests for BPF htab map's used size Date: Sat, 5 Nov 2022 02:51:46 +0000 Message-Id: <20221105025146.238209-5-horenchuang@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20221105025146.238209-1-horenchuang@bytedance.com> References: <20221105025146.238209-1-horenchuang@bytedance.com> MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748623055521128448?= X-GMAIL-MSGID: =?utf-8?q?1748623055521128448?= Integrate with existing unit tests such as basic, read/write-only htab maps, and concurrency testings for testing htab map's used_entires. Signed-off-by: Ho-Ren (Jack) Chuang --- tools/testing/selftests/bpf/test_maps.c | 74 ++++++++++++++++++++++++- 1 file changed, 73 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/bpf/test_maps.c b/tools/testing/selftests/bpf/test_maps.c index b73152822aa2..3bd202d27563 100644 --- a/tools/testing/selftests/bpf/test_maps.c +++ b/tools/testing/selftests/bpf/test_maps.c @@ -38,6 +38,8 @@ static void test_hashmap(unsigned int task, void *data) { long long key, next_key, first_key, value; int fd; + struct bpf_map_info map_info = {}; + __u32 info_len = sizeof(map_info); fd = bpf_map_create(BPF_MAP_TYPE_HASH, NULL, sizeof(key), sizeof(value), 2, &map_opts); if (fd < 0) { @@ -50,16 +52,32 @@ static void test_hashmap(unsigned int task, void *data) /* Insert key=1 element. */ assert(bpf_map_update_elem(fd, &key, &value, BPF_ANY) == 0); + /* Check used_entires is now 1. */ + assert(bpf_obj_get_info_by_fd(fd, &map_info, &info_len) == 0); + assert(map_info.used_entries == 1); + value = 0; /* BPF_NOEXIST means add new element if it doesn't exist. */ assert(bpf_map_update_elem(fd, &key, &value, BPF_NOEXIST) < 0 && /* key=1 already exists. */ errno == EEXIST); + /* Check used_entires is still 1 because we are updating + * an existing element. + */ + assert(bpf_obj_get_info_by_fd(fd, &map_info, &info_len) == 0); + assert(map_info.used_entries == 1); + /* -1 is an invalid flag. */ assert(bpf_map_update_elem(fd, &key, &value, -1) < 0 && errno == EINVAL); + /* Check used_entires is still 1 because the last + * insertion was invalid. + */ + assert(bpf_obj_get_info_by_fd(fd, &map_info, &info_len) == 0); + assert(map_info.used_entries == 1); + /* Check that key=1 can be found. */ assert(bpf_map_lookup_elem(fd, &key, &value) == 0 && value == 1234); @@ -68,6 +86,10 @@ static void test_hashmap(unsigned int task, void *data) /* Insert key=2 element. */ assert(bpf_map_update_elem(fd, &key, &value, BPF_ANY) == 0); + /* Check used_entires is now 2. */ + assert(bpf_obj_get_info_by_fd(fd, &map_info, &info_len) == 0); + assert(map_info.used_entries == 2); + /* Check that key=2 matches the value and delete it */ assert(bpf_map_lookup_and_delete_elem(fd, &key, &value) == 0 && value == 1234); @@ -89,6 +111,10 @@ static void test_hashmap(unsigned int task, void *data) assert(bpf_map_update_elem(fd, &key, &value, BPF_NOEXIST) < 0 && errno == E2BIG); + /* Check used_entires is now 2. */ + assert(bpf_obj_get_info_by_fd(fd, &map_info, &info_len) == 0); + assert(map_info.used_entries == 2); + /* Update existing element, though the map is full. */ key = 1; assert(bpf_map_update_elem(fd, &key, &value, BPF_EXIST) == 0); @@ -102,6 +128,10 @@ static void test_hashmap(unsigned int task, void *data) key = 0; assert(bpf_map_delete_elem(fd, &key) < 0 && errno == ENOENT); + /* Check used_entires is now 2. */ + assert(bpf_obj_get_info_by_fd(fd, &map_info, &info_len) == 0); + assert(map_info.used_entries == 2); + /* Iterate over two elements. */ assert(bpf_map_get_next_key(fd, NULL, &first_key) == 0 && (first_key == 1 || first_key == 2)); @@ -127,6 +157,10 @@ static void test_hashmap(unsigned int task, void *data) assert(bpf_map_get_next_key(fd, &key, &next_key) < 0 && errno == ENOENT); + /* Check used_entires is now 0 because both elements were deleted. */ + assert(bpf_obj_get_info_by_fd(fd, &map_info, &info_len) == 0); + assert(map_info.used_entries == 0); + close(fd); } @@ -292,6 +326,8 @@ static void test_hashmap_walk(unsigned int task, void *data) int fd, i, max_entries = 10000; long long key, value[VALUE_SIZE], next_key; bool next_key_valid = true; + struct bpf_map_info map_info = {}; + __u32 info_len = sizeof(map_info); fd = helper_fill_hashmap(max_entries); @@ -302,6 +338,9 @@ static void test_hashmap_walk(unsigned int task, void *data) } assert(i == max_entries); + /* Check used_entires is now max_entries. */ + assert(bpf_obj_get_info_by_fd(fd, &map_info, &info_len) == 0); + assert(map_info.used_entries == max_entries); assert(bpf_map_get_next_key(fd, NULL, &key) == 0); for (i = 0; next_key_valid; i++) { @@ -313,6 +352,9 @@ static void test_hashmap_walk(unsigned int task, void *data) } assert(i == max_entries); + /* Check used_entires is now max_entries. */ + assert(bpf_obj_get_info_by_fd(fd, &map_info, &info_len) == 0); + assert(map_info.used_entries == max_entries); for (i = 0; bpf_map_get_next_key(fd, !i ? NULL : &key, &next_key) == 0; i++) { @@ -322,6 +364,9 @@ static void test_hashmap_walk(unsigned int task, void *data) } assert(i == max_entries); + /* Check used_entires is now max_entries. */ + assert(bpf_obj_get_info_by_fd(fd, &map_info, &info_len) == 0); + assert(map_info.used_entries == max_entries); close(fd); } @@ -1303,13 +1348,14 @@ static void test_map_in_map(void) static void test_map_large(void) { - struct bigkey { int a; char b[4096]; long long c; } key; int fd, i, value; + struct bpf_map_info map_info = {}; + __u32 info_len = sizeof(map_info); fd = bpf_map_create(BPF_MAP_TYPE_HASH, NULL, sizeof(key), sizeof(value), MAP_SIZE, &map_opts); @@ -1341,6 +1387,10 @@ static void test_map_large(void) key.a = 1; assert(bpf_map_lookup_elem(fd, &key, &value) < 0 && errno == ENOENT); + /* Check used_entires is now MAP_SIZE. */ + assert(bpf_obj_get_info_by_fd(fd, &map_info, &info_len) == 0); + assert(map_info.used_entries == MAP_SIZE); + close(fd); } @@ -1466,6 +1516,8 @@ static void test_map_parallel(void) { int i, fd, key = 0, value = 0, j = 0; int data[2]; + struct bpf_map_info map_info = {}; + __u32 info_len = sizeof(map_info); fd = bpf_map_create(BPF_MAP_TYPE_HASH, NULL, sizeof(key), sizeof(value), MAP_SIZE, &map_opts); @@ -1504,6 +1556,10 @@ static void test_map_parallel(void) value == key); } + /* Check used_entires is now MAP_SIZE. */ + assert(bpf_obj_get_info_by_fd(fd, &map_info, &info_len) == 0); + assert(map_info.used_entries == MAP_SIZE); + /* Now let's delete all elemenets in parallel. */ data[1] = DO_DELETE; run_parallel(TASKS, test_update_delete, data); @@ -1513,6 +1569,10 @@ static void test_map_parallel(void) assert(bpf_map_get_next_key(fd, NULL, &key) < 0 && errno == ENOENT); assert(bpf_map_get_next_key(fd, &key, &key) < 0 && errno == ENOENT); + /* Check used_entires is now 0. */ + assert(bpf_obj_get_info_by_fd(fd, &map_info, &info_len) == 0); + assert(map_info.used_entries == 0); + key = 0; bpf_map_delete_elem(fd, &key); if (j++ < 5) @@ -1524,6 +1584,8 @@ static void test_map_rdonly(void) { int fd, key = 0, value = 0; __u32 old_flags; + struct bpf_map_info map_info = {}; + __u32 info_len = sizeof(map_info); old_flags = map_opts.map_flags; map_opts.map_flags |= BPF_F_RDONLY; @@ -1546,6 +1608,10 @@ static void test_map_rdonly(void) assert(bpf_map_lookup_elem(fd, &key, &value) < 0 && errno == ENOENT); assert(bpf_map_get_next_key(fd, &key, &value) < 0 && errno == ENOENT); + /* Check used_entires is now 0. */ + assert(bpf_obj_get_info_by_fd(fd, &map_info, &info_len) == 0); + assert(map_info.used_entries == 0); + close(fd); } @@ -1553,6 +1619,8 @@ static void test_map_wronly_hash(void) { int fd, key = 0, value = 0; __u32 old_flags; + struct bpf_map_info map_info = {}; + __u32 info_len = sizeof(map_info); old_flags = map_opts.map_flags; map_opts.map_flags |= BPF_F_WRONLY; @@ -1574,6 +1642,10 @@ static void test_map_wronly_hash(void) assert(bpf_map_lookup_elem(fd, &key, &value) < 0 && errno == EPERM); assert(bpf_map_get_next_key(fd, &key, &value) < 0 && errno == EPERM); + /* Check used_entires is now 1. */ + assert(bpf_obj_get_info_by_fd(fd, &map_info, &info_len) == 0); + assert(map_info.used_entries == 1); + close(fd); }