From patchwork Wed Oct 18 06:17:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuyi Zhou X-Patchwork-Id: 154706 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp4595973vqb; Tue, 17 Oct 2023 23:23:07 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHgaH1YIIIzCCu55/jeISReqFVBtA4OAzacpjgVu00O6R/8jLVEalklaQbjEeDQHvWKs3rX X-Received: by 2002:a05:6870:6b8a:b0:1ea:1510:d8df with SMTP id ms10-20020a0568706b8a00b001ea1510d8dfmr4406492oab.4.1697610187139; Tue, 17 Oct 2023 23:23:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697610187; cv=none; d=google.com; s=arc-20160816; b=RGIaJMfiRCK5HGaIvsxReAZXkcOvUo5X11hWUNEZxsqbojJvhX9LfNg0Hjn903sqjp dSe7h/jndKQxEg89AWIxScR0q2/f8Lu6H5lKRaStZPxBDkPUox8BgqJMRZhSBFAFsRCU FlElAlQJIs5QKZOG5hcz4VHGy2ODYNbhqkdpO+UGIYB4Pj5BZmHAQUD0z5u5LO1xwTEd ZRGXpjvSpbX9sgIWADLEEio8+OGjdASHRLJMogS/649uzGto3M3KzQvmWec5PGQ1VZdO W2KyEBWQX7woyjnh59pl2JA37A6RiQAWRmfqyuclcWzVHZTxyipUjExeRWakuTm5WhJT rL8w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=FNW2oYGtelVvgczyk2A13Ij/ocoUNYFTCyV6b+P6gh8=; fh=DR7g1EcWKOXTEoooUPBSJXUaklSrDEYzv6YDdhz1CwE=; b=n0WX1/N76qFCs/bzNja9AKsm/U4CKrQ3SqJMumoLCs22foTjenH57HM55FQx+QVxQ4 Kj/1XFFIRPryI8MUcpqAP2Kicn7feqwLs8MaT/mV1dcWOV//n53/QeIsCaBt9j2I89Ny KlDX8hW6QpWrFbbRB+MfXVI5Enx6aOokAVqBigGizb+Q/IuLtJjW5LoCR1y0TySyNWRe DmKOogs83K94jgztc9IktpEFdiNIwiwpEF4NUie3F90pr0RZ/nq2fnkEpFwMNVorMArw djs+oxWGrIXaDBCzA+bbiawSzdSGA+W2+hStkEKdIaV6Upo0hZnsghlG8FMo164E4zI1 5SoA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=V4T7oq8V; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from pete.vger.email (pete.vger.email. [23.128.96.36]) by mx.google.com with ESMTPS id f22-20020a633816000000b0055c95e91f67si1485043pga.155.2023.10.17.23.23.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Oct 2023 23:23:07 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) client-ip=23.128.96.36; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=V4T7oq8V; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by pete.vger.email (Postfix) with ESMTP id 12B4B8097A67; Tue, 17 Oct 2023 23:22:53 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at pete.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234947AbjJRGUM (ORCPT + 23 others); Wed, 18 Oct 2023 02:20:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59672 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235028AbjJRGSn (ORCPT ); Wed, 18 Oct 2023 02:18:43 -0400 Received: from mail-pl1-x62c.google.com (mail-pl1-x62c.google.com [IPv6:2607:f8b0:4864:20::62c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9410AD58 for ; Tue, 17 Oct 2023 23:17:59 -0700 (PDT) Received: by mail-pl1-x62c.google.com with SMTP id d9443c01a7336-1c9e06f058bso44979305ad.0 for ; Tue, 17 Oct 2023 23:17:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1697609878; x=1698214678; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FNW2oYGtelVvgczyk2A13Ij/ocoUNYFTCyV6b+P6gh8=; b=V4T7oq8VpxNt6t2w14WbFy1JCSIyBZZb/8p38RNc0BIImJ1ucXOw68wWgX+Ywv7Nsi Vl3hVKY43Dz1IqRE5jkxkR0Y53AhOnCikPS1y5BDRZmrjmEwklA8FQQlhJHxGFUfHNdT ZFFm0GY8hrHCG4KF3hui+1/gn1zx6Md8MjOi0sTcHcSCjz6ngvwF4DJyRLCSrJv2C3H5 Z/JEAkBoQiRbZL0hHDVlgh1xznXAas4wpOiwHh4pO+Dh23KuUMKBEuf5TL7+TK8JsRvk DMMB3CuGfSC0TgkFT8r++RvrZAHkD3MCJz7FS7TdmRf7cP35y5hKp4zzfs4JA5Xh5F+H qbCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697609878; x=1698214678; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FNW2oYGtelVvgczyk2A13Ij/ocoUNYFTCyV6b+P6gh8=; b=vy4wLjC1T5nvJ+Ek7oZxFGJj9ZUPXBZVG+rMh7kXM9/A6sRfnDb9fbly4ZqmO817fD IbNsArMp7jlRa5OheLwjdLk6eubcVJg0vd9Btp7zL2CASq7XwjSbBB/O2mMnnqE31DCE BNcK/TpQItE8h9/2hyPF+TrdpY+ssR1ZP5KaSamzIFY7FsCzGWWAqXI62U8tsF/5RB3K VOtvY1jb90yBMdFpzTd0rNa+HBCBKTOFmcULmNQVnFLhAcpJ+rz8VXGBsgb8aHjaEsAq w4HYYmOB8/J9ChmOuVqPcJ8SneDh073MAX+YnF3DUeG3V8ogWZ5VVw8nuWm91UbKhptv 31MQ== X-Gm-Message-State: AOJu0Yw8iuatioCxul0DchhSrig/dFSPcqtQssF89QzVoiTQrxmd8PHD nYQB7ObYaySn5IRuIgBDGLBWXw== X-Received: by 2002:a17:903:41d1:b0:1c5:7d49:570e with SMTP id u17-20020a17090341d100b001c57d49570emr4978737ple.29.1697609877965; Tue, 17 Oct 2023 23:17:57 -0700 (PDT) Received: from n37-019-243.byted.org ([180.184.103.200]) by smtp.gmail.com with ESMTPSA id ix13-20020a170902f80d00b001c61acd5bd2sm2659116plb.112.2023.10.17.23.17.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Oct 2023 23:17:57 -0700 (PDT) From: Chuyi Zhou To: bpf@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@kernel.org, tj@kernel.org, linux-kernel@vger.kernel.org, Chuyi Zhou Subject: [RESEND PATCH bpf-next v6 1/8] cgroup: Prepare for using css_task_iter_*() in BPF Date: Wed, 18 Oct 2023 14:17:39 +0800 Message-Id: <20231018061746.111364-2-zhouchuyi@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20231018061746.111364-1-zhouchuyi@bytedance.com> References: <20231018061746.111364-1-zhouchuyi@bytedance.com> MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Tue, 17 Oct 2023 23:22:53 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780073299417031165 X-GMAIL-MSGID: 1780073299417031165 This patch makes some preparations for using css_task_iter_*() in BPF Program. 1. Flags CSS_TASK_ITER_* are #define-s and it's not easy for bpf prog to use them. Convert them to enum so bpf prog can take them from vmlinux.h. 2. In the next patch we will add css_task_iter_*() in common kfuncs which is not safe. Since css_task_iter_*() does spin_unlock_irq() which might screw up irq flags depending on the context where bpf prog is running. So we should use irqsave/irqrestore here and the switching is harmless. Suggested-by: Alexei Starovoitov Signed-off-by: Chuyi Zhou Acked-by: Tejun Heo --- include/linux/cgroup.h | 12 +++++------- kernel/cgroup/cgroup.c | 18 ++++++++++++------ 2 files changed, 17 insertions(+), 13 deletions(-) diff --git a/include/linux/cgroup.h b/include/linux/cgroup.h index b307013b9c6c..0ef0af66080e 100644 --- a/include/linux/cgroup.h +++ b/include/linux/cgroup.h @@ -40,13 +40,11 @@ struct kernel_clone_args; #define CGROUP_WEIGHT_DFL 100 #define CGROUP_WEIGHT_MAX 10000 -/* walk only threadgroup leaders */ -#define CSS_TASK_ITER_PROCS (1U << 0) -/* walk all threaded css_sets in the domain */ -#define CSS_TASK_ITER_THREADED (1U << 1) - -/* internal flags */ -#define CSS_TASK_ITER_SKIPPED (1U << 16) +enum { + CSS_TASK_ITER_PROCS = (1U << 0), /* walk only threadgroup leaders */ + CSS_TASK_ITER_THREADED = (1U << 1), /* walk all threaded css_sets in the domain */ + CSS_TASK_ITER_SKIPPED = (1U << 16), /* internal flags */ +}; /* a css_task_iter should be treated as an opaque object */ struct css_task_iter { diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c index 1fb7f562289d..b6d64f3b8888 100644 --- a/kernel/cgroup/cgroup.c +++ b/kernel/cgroup/cgroup.c @@ -4917,9 +4917,11 @@ static void css_task_iter_advance(struct css_task_iter *it) void css_task_iter_start(struct cgroup_subsys_state *css, unsigned int flags, struct css_task_iter *it) { + unsigned long irqflags; + memset(it, 0, sizeof(*it)); - spin_lock_irq(&css_set_lock); + spin_lock_irqsave(&css_set_lock, irqflags); it->ss = css->ss; it->flags = flags; @@ -4933,7 +4935,7 @@ void css_task_iter_start(struct cgroup_subsys_state *css, unsigned int flags, css_task_iter_advance(it); - spin_unlock_irq(&css_set_lock); + spin_unlock_irqrestore(&css_set_lock, irqflags); } /** @@ -4946,12 +4948,14 @@ void css_task_iter_start(struct cgroup_subsys_state *css, unsigned int flags, */ struct task_struct *css_task_iter_next(struct css_task_iter *it) { + unsigned long irqflags; + if (it->cur_task) { put_task_struct(it->cur_task); it->cur_task = NULL; } - spin_lock_irq(&css_set_lock); + spin_lock_irqsave(&css_set_lock, irqflags); /* @it may be half-advanced by skips, finish advancing */ if (it->flags & CSS_TASK_ITER_SKIPPED) @@ -4964,7 +4968,7 @@ struct task_struct *css_task_iter_next(struct css_task_iter *it) css_task_iter_advance(it); } - spin_unlock_irq(&css_set_lock); + spin_unlock_irqrestore(&css_set_lock, irqflags); return it->cur_task; } @@ -4977,11 +4981,13 @@ struct task_struct *css_task_iter_next(struct css_task_iter *it) */ void css_task_iter_end(struct css_task_iter *it) { + unsigned long irqflags; + if (it->cur_cset) { - spin_lock_irq(&css_set_lock); + spin_lock_irqsave(&css_set_lock, irqflags); list_del(&it->iters_node); put_css_set_locked(it->cur_cset); - spin_unlock_irq(&css_set_lock); + spin_unlock_irqrestore(&css_set_lock, irqflags); } if (it->cur_dcset) From patchwork Wed Oct 18 06:17:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuyi Zhou X-Patchwork-Id: 154701 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp4595143vqb; Tue, 17 Oct 2023 23:20:48 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHEyvWEtJ5Kn9vHrzQaOG3Mi/uH/bqR4vonIrhOP8KckCZfRs2kAylFyIn4vpl5gAVw/POb X-Received: by 2002:a05:6358:e927:b0:166:d9c3:941d with SMTP id gk39-20020a056358e92700b00166d9c3941dmr4713014rwb.3.1697610047999; Tue, 17 Oct 2023 23:20:47 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697610047; cv=none; d=google.com; s=arc-20160816; b=eIWttMkkeRggAycNP0cNZZ9/ds0iwFfz9qayepeOQt3qLtzLGfcQnQ2IFIIR5zplmC HXq8D7NaLIy/VEJzQ4AU13swSQFncY/0WGd8MtU/kzcAege2APqJNh14tb/gdknfIQX1 FukmU6PjBYx0x5vHbQF9h3shBf7fZy3fsJLfsnZgjyrpXlXAaZk/7B2Ln+0Ms48UjaxC aN/zoOOlps22EzWilxgH7XAJ9d0Y3NRuq3nJ+UkYyjE5DR5zCABJ/L6zD+n5lmuH+FLr uPLJooF698t4++wiC9maZt+7rXwq9s20o0K3lT8reuvtectmEnuzkkPbaH9WxpcUVz+D dlBA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=BbIp3xjsMf+vYU1+9/Ac/+6x9YCK8BhSfh7PegmcPCk=; fh=DR7g1EcWKOXTEoooUPBSJXUaklSrDEYzv6YDdhz1CwE=; b=qxFxKwF8UaosgdC5HVn65ipOlOqyRfqTcrYQU+CBf42YoDbV69RHqLt2MatO4/ssry ShHwlbvsuSwGu+Lhqc39VsJ+hxgpE/hHwdrIsb4mROdu+TQ6t4RrIYUtSiH9sI6788Wx vvShFMPMYHNWmih4/jCQ3Rkm5vhM19AA/o8Gvc6ArWjrk17Ab+9GwI+zUtxqs5+Z45QH mSDt1IWyDhk4GpmUWk9GzXYtMbu/FJJqF/c4Ji/xIBrOkiuNkZwjC9DHhyJg5yLlnM2g edegNBFcXuksFxCTEGtByOFAcpx/ruA6BaMm6kq/yiv9w/FMA1gjUJoZ6N3RIBTjaSyQ E09g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=P60L+SOx; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from howler.vger.email (howler.vger.email. [23.128.96.34]) by mx.google.com with ESMTPS id q22-20020a656a96000000b00577448019cbsi1422463pgu.841.2023.10.17.23.20.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Oct 2023 23:20:47 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) client-ip=23.128.96.34; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=P60L+SOx; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id 9DE8F80293FE; Tue, 17 Oct 2023 23:20:35 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344609AbjJRGUH (ORCPT + 23 others); Wed, 18 Oct 2023 02:20:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46318 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344641AbjJRGSq (ORCPT ); Wed, 18 Oct 2023 02:18:46 -0400 Received: from mail-pl1-x633.google.com (mail-pl1-x633.google.com [IPv6:2607:f8b0:4864:20::633]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EB50BD73 for ; Tue, 17 Oct 2023 23:18:03 -0700 (PDT) Received: by mail-pl1-x633.google.com with SMTP id d9443c01a7336-1c9b1e3a809so43299595ad.2 for ; Tue, 17 Oct 2023 23:18:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1697609881; x=1698214681; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=BbIp3xjsMf+vYU1+9/Ac/+6x9YCK8BhSfh7PegmcPCk=; b=P60L+SOxGIk94nEPVnzuCQHC3zDs9PjIbeKM3/GOID90zzbfc72OkNnJ2cBicTZbNz 2L6fF0ffUYiwZH41Il7pmXX6v8l7AMHcmhDOvB3M8Jhq0vgiyleUlcyRrPVpjKWNLOrx Cq4lPZohGdS+7EdMceaaM60pcLLMQ72HFz2k7ka2646lk+6Xw3ubIrel4XLOY2BkRMlx sx2WfF+44CjmP2YLJpg4/7ZgNLAsbP1n3H1ehiF4Hm0l2hOD+bf1OQ8vLIOhDDSqDAu0 2OSqrqduISXFK4Fu47FPWhENfyYvkCFYIaDPWaMOXeNZLz3Kr/pSvJX4F0hfT4IUNa2S +isg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697609881; x=1698214681; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BbIp3xjsMf+vYU1+9/Ac/+6x9YCK8BhSfh7PegmcPCk=; b=bJ9R1CCW3IY5RWOjoPB3itFlmEQy4tF48IGfL7f4Y9GqUVwULZo1ss1OchMWyEDgWN Z3tNLWszzSV+g96p0NqYJd4WfjRYSEUAlnEibRi5wMpk+fLJk2FNiU3qkA8SRVvC1sl/ WuIeX/7dAn1iAPy1jRxJno8ivr78h9mRFuQ7iYp/95XWLd2fIAsCDdu6Tu4jDLo7kh6S TH6Nd0/+L17SMrT5tMvhLABUm8vbqD+3961W1EDxTS8EqjiwKHgnuNMDsfl23a/wPE1i Y1go5e3giBNT2rGHwa7LBtxpGEAflaoeW0N9xn3EVA3lJnIDG9Rwdm9u4ivIIfx6QFup i8sg== X-Gm-Message-State: AOJu0Yy/IWNTbGfjRbL23sIxWGCxdvUp5kcWwXbL3283os5XJGXYQBD/ WTfJ3sGfE8jsT4L4oE1VF0HAdawT4lYm1LY/8nhgpA== X-Received: by 2002:a17:903:246:b0:1c7:5a63:43bb with SMTP id j6-20020a170903024600b001c75a6343bbmr5289084plh.8.1697609881516; Tue, 17 Oct 2023 23:18:01 -0700 (PDT) Received: from n37-019-243.byted.org ([180.184.103.200]) by smtp.gmail.com with ESMTPSA id ix13-20020a170902f80d00b001c61acd5bd2sm2659116plb.112.2023.10.17.23.17.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Oct 2023 23:18:01 -0700 (PDT) From: Chuyi Zhou To: bpf@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@kernel.org, tj@kernel.org, linux-kernel@vger.kernel.org, Chuyi Zhou Subject: [RESEND PATCH bpf-next v6 2/8] bpf: Introduce css_task open-coded iterator kfuncs Date: Wed, 18 Oct 2023 14:17:40 +0800 Message-Id: <20231018061746.111364-3-zhouchuyi@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20231018061746.111364-1-zhouchuyi@bytedance.com> References: <20231018061746.111364-1-zhouchuyi@bytedance.com> MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on howler.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Tue, 17 Oct 2023 23:20:35 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780073153550080473 X-GMAIL-MSGID: 1780073153550080473 This patch adds kfuncs bpf_iter_css_task_{new,next,destroy} which allow creation and manipulation of struct bpf_iter_css_task in open-coded iterator style. These kfuncs actually wrapps css_task_iter_{start,next, end}. BPF programs can use these kfuncs through bpf_for_each macro for iteration of all tasks under a css. css_task_iter_*() would try to get the global spin-lock *css_set_lock*, so the bpf side has to be careful in where it allows to use this iter. Currently we only allow it in bpf_lsm and bpf iter-s. Signed-off-by: Chuyi Zhou Acked-by: Tejun Heo --- kernel/bpf/helpers.c | 3 + kernel/bpf/task_iter.c | 58 +++++++++++++++++++ kernel/bpf/verifier.c | 23 ++++++++ .../testing/selftests/bpf/bpf_experimental.h | 8 +++ 4 files changed, 92 insertions(+) diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index 61f51dee8448..c01441db9fd5 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -2560,6 +2560,9 @@ BTF_ID_FLAGS(func, bpf_iter_num_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_iter_task_vma_new, KF_ITER_NEW | KF_RCU) BTF_ID_FLAGS(func, bpf_iter_task_vma_next, KF_ITER_NEXT | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_task_vma_destroy, KF_ITER_DESTROY) +BTF_ID_FLAGS(func, bpf_iter_css_task_new, KF_ITER_NEW | KF_TRUSTED_ARGS) +BTF_ID_FLAGS(func, bpf_iter_css_task_next, KF_ITER_NEXT | KF_RET_NULL) +BTF_ID_FLAGS(func, bpf_iter_css_task_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_dynptr_adjust) BTF_ID_FLAGS(func, bpf_dynptr_is_null) BTF_ID_FLAGS(func, bpf_dynptr_is_rdonly) diff --git a/kernel/bpf/task_iter.c b/kernel/bpf/task_iter.c index fef17628341f..e4126698cecf 100644 --- a/kernel/bpf/task_iter.c +++ b/kernel/bpf/task_iter.c @@ -894,6 +894,64 @@ __bpf_kfunc void bpf_iter_task_vma_destroy(struct bpf_iter_task_vma *it) __diag_pop(); +struct bpf_iter_css_task { + __u64 __opaque[1]; +} __attribute__((aligned(8))); + +struct bpf_iter_css_task_kern { + struct css_task_iter *css_it; +} __attribute__((aligned(8))); + +__diag_push(); +__diag_ignore_all("-Wmissing-prototypes", + "Global functions as their definitions will be in vmlinux BTF"); + +__bpf_kfunc int bpf_iter_css_task_new(struct bpf_iter_css_task *it, + struct cgroup_subsys_state *css, unsigned int flags) +{ + struct bpf_iter_css_task_kern *kit = (void *)it; + + BUILD_BUG_ON(sizeof(struct bpf_iter_css_task_kern) != sizeof(struct bpf_iter_css_task)); + BUILD_BUG_ON(__alignof__(struct bpf_iter_css_task_kern) != + __alignof__(struct bpf_iter_css_task)); + kit->css_it = NULL; + switch (flags) { + case CSS_TASK_ITER_PROCS | CSS_TASK_ITER_THREADED: + case CSS_TASK_ITER_PROCS: + case 0: + break; + default: + return -EINVAL; + } + + kit->css_it = bpf_mem_alloc(&bpf_global_ma, sizeof(struct css_task_iter)); + if (!kit->css_it) + return -ENOMEM; + css_task_iter_start(css, flags, kit->css_it); + return 0; +} + +__bpf_kfunc struct task_struct *bpf_iter_css_task_next(struct bpf_iter_css_task *it) +{ + struct bpf_iter_css_task_kern *kit = (void *)it; + + if (!kit->css_it) + return NULL; + return css_task_iter_next(kit->css_it); +} + +__bpf_kfunc void bpf_iter_css_task_destroy(struct bpf_iter_css_task *it) +{ + struct bpf_iter_css_task_kern *kit = (void *)it; + + if (!kit->css_it) + return; + css_task_iter_end(kit->css_it); + bpf_mem_free(&bpf_global_ma, kit->css_it); +} + +__diag_pop(); + DEFINE_PER_CPU(struct mmap_unlock_irq_work, mmap_unlock_work); static void do_mmap_read_unlock(struct irq_work *entry) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index bb58987e4844..974713185269 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -10472,6 +10472,7 @@ enum special_kfunc_type { KF_bpf_percpu_obj_new_impl, KF_bpf_percpu_obj_drop_impl, KF_bpf_throw, + KF_bpf_iter_css_task_new, }; BTF_SET_START(special_kfunc_set) @@ -10495,6 +10496,7 @@ BTF_ID(func, bpf_dynptr_clone) BTF_ID(func, bpf_percpu_obj_new_impl) BTF_ID(func, bpf_percpu_obj_drop_impl) BTF_ID(func, bpf_throw) +BTF_ID(func, bpf_iter_css_task_new) BTF_SET_END(special_kfunc_set) BTF_ID_LIST(special_kfunc_list) @@ -10520,6 +10522,7 @@ BTF_ID(func, bpf_dynptr_clone) BTF_ID(func, bpf_percpu_obj_new_impl) BTF_ID(func, bpf_percpu_obj_drop_impl) BTF_ID(func, bpf_throw) +BTF_ID(func, bpf_iter_css_task_new) static bool is_kfunc_ret_null(struct bpf_kfunc_call_arg_meta *meta) { @@ -11050,6 +11053,20 @@ static int process_kf_arg_ptr_to_rbtree_node(struct bpf_verifier_env *env, &meta->arg_rbtree_root.field); } +static bool check_css_task_iter_allowlist(struct bpf_verifier_env *env) +{ + enum bpf_prog_type prog_type = resolve_prog_type(env->prog); + + switch (prog_type) { + case BPF_PROG_TYPE_LSM: + return true; + case BPF_TRACE_ITER: + return env->prog->aux->sleepable; + default: + return false; + } +} + static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_arg_meta *meta, int insn_idx) { @@ -11300,6 +11317,12 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ break; } case KF_ARG_PTR_TO_ITER: + if (meta->func_id == special_kfunc_list[KF_bpf_iter_css_task_new]) { + if (!check_css_task_iter_allowlist(env)) { + verbose(env, "css_task_iter is only allowed in bpf_lsm and bpf iter-s\n"); + return -EINVAL; + } + } ret = process_iter_arg(env, regno, insn_idx, meta); if (ret < 0) return ret; diff --git a/tools/testing/selftests/bpf/bpf_experimental.h b/tools/testing/selftests/bpf/bpf_experimental.h index 2c8cb3f61529..6792ed2b45d7 100644 --- a/tools/testing/selftests/bpf/bpf_experimental.h +++ b/tools/testing/selftests/bpf/bpf_experimental.h @@ -458,4 +458,12 @@ extern void bpf_throw(u64 cookie) __ksym; __bpf_assert_op(LHS, <=, END, value, false); \ }) +struct bpf_iter_css_task; +struct cgroup_subsys_state; +extern int bpf_iter_css_task_new(struct bpf_iter_css_task *it, + struct cgroup_subsys_state *css, unsigned int flags) __weak __ksym; +extern struct task_struct *bpf_iter_css_task_next(struct bpf_iter_css_task *it) __weak __ksym; +extern void bpf_iter_css_task_destroy(struct bpf_iter_css_task *it) __weak __ksym; + + #endif From patchwork Wed Oct 18 06:17:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuyi Zhou X-Patchwork-Id: 154696 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp4594889vqb; Tue, 17 Oct 2023 23:20:06 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGU7GaF42nn/ykf69uM/zFoyftEIMjM9XmEtxu83T297ySHjvOFKJFAW/ElZOihPWadNaoe X-Received: by 2002:a05:6830:68c9:b0:6c6:473c:9ca0 with SMTP id cw9-20020a05683068c900b006c6473c9ca0mr4299677otb.3.1697610006411; Tue, 17 Oct 2023 23:20:06 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697610006; cv=none; d=google.com; s=arc-20160816; b=exAjw1wntiHDP89TucZUEvk6CLpLm4Lzs81VaYwbvpCm+UwH4YvtuC37AYPrgq+YyD uuDRhluydwdtDyLUODDKscpOWZZsRcV1yCxUBz4/KwbwBjLPo55Vzjlv5u3j4h9bjX1E 1sVepY6JueroQaRSRXqJd/SEr5IX1sbXhpj69Nsb24eLx4Ya1+1I6aON6/veV2aKzgtj 6EQNpTBIxCfNRuJ6IEC3rk6HRvbtoXZ56udEemakTopNUjLZyLPDMD7q4SQInTJMjn2L PBoFneFNGY7bGAFp8iEDDqXMIZciNRu2kbxdMSWLeRWIOX6LEKMmySXcFPuSY/pTlPMs uiTg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=6pA+uWgAid8oSLs2nLYNpXTwEjADAAqZra1pWMMUUeE=; fh=DR7g1EcWKOXTEoooUPBSJXUaklSrDEYzv6YDdhz1CwE=; b=TKsqXNQnvEXHp0NXlyxuh1ThUG7KNwuEfKRgwKDJJ21PUvv5bm655Dk8gN1KKRWZm7 fTCIe0oKWAtvDfI3atAa5CW3P9TOFdG9oqM+22LjMkbrMLHASiTFF5xQAo+8HuJQ0f5N 096PRvL9Fy8WO4PRJ7guncjcoXbl9SiwsTUCxFOip/tlVkRs8UgMrpgVB+qGTp1DJwQr zZTLgGjUU82Rm0lAcFTcbr+pp5vRinITVXZIi47YaPkfNBzA7fwq56nxO+I/w2CNMvMA XFd9lpT1z8o4ocVKQhJYSb+6hN2BdFrM3K3MC/HtukORpm/8HCqay2jKY8InPMiyzEYG lmBA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=GCUuEqUP; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from lipwig.vger.email (lipwig.vger.email. [2620:137:e000::3:3]) by mx.google.com with ESMTPS id ef24-20020a056a002c9800b0068fce4338bdsi3192897pfb.62.2023.10.17.23.20.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Oct 2023 23:20:06 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) client-ip=2620:137:e000::3:3; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=GCUuEqUP; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id A94268148BB0; Tue, 17 Oct 2023 23:20:03 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344559AbjJRGTy (ORCPT + 23 others); Wed, 18 Oct 2023 02:19:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46386 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234778AbjJRGTJ (ORCPT ); Wed, 18 Oct 2023 02:19:09 -0400 Received: from mail-pl1-x62e.google.com (mail-pl1-x62e.google.com [IPv6:2607:f8b0:4864:20::62e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D4E2610CA for ; Tue, 17 Oct 2023 23:18:05 -0700 (PDT) Received: by mail-pl1-x62e.google.com with SMTP id d9443c01a7336-1c87a85332bso55800255ad.2 for ; Tue, 17 Oct 2023 23:18:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1697609884; x=1698214684; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6pA+uWgAid8oSLs2nLYNpXTwEjADAAqZra1pWMMUUeE=; b=GCUuEqUPkuFXUrlH4jF65SsMpb0CT2u0W9yNLIkpXbxsj8hcMmcE+sgk+AT6VzYt24 wEd+1xAzig/a5u1Ah5HAF02eTqp090xRPCm2jvrDlMCt6H7s5h3a3SLdwOrIsOiY7PYK OsLOaiHdPhZqwXANd0BvSJISlwyefwCYKgBH215KN9Tbcv/V6gALShr30Jcg6rw/ayII fcYE2S/u1kx7XF7QdZyDVgQLQesPOVof5fufpZ3pRNDHNWHMMrcSYvixVvf/bufXgK7p zJ1woFrIc3OXdKRm+dwsS2CIOKCIbLhMKtqcjPMx93xIi4HIckOKGwD1oFdSbgUOZC/p 2n5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697609884; x=1698214684; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6pA+uWgAid8oSLs2nLYNpXTwEjADAAqZra1pWMMUUeE=; b=QbvePwM/P7RjMv6M1LBIWZkQCDrZG3HQdtxAmfT7O263jDKxwe46/cC3FTq/QI9ncX XmemEn3fS5EZokpBCsodtuGff/UT045eN5R/nUb43SYZ+93oFFzX7MVLOhEHKSuWy5/E sTfzcALUDFEKAqYAJZsbtnfEKPcQdOlbv1Hk1VDSLwBm7SSAaeVZgPdgpHOczi2FIDZl MqwIRqKp/RvZ5rPhG9Xh7SU7b/mK+oAgJ8wH4MZpCweHU6CdENR1eF3HjkdXVwTyql54 iibDl5AvToec7Qi95LsTMPPzArl2sMX0yizyQqM0DwpDtTpGiJ9BQLbwqPyq/DchZU+G jiHg== X-Gm-Message-State: AOJu0Yy+5W6RR/vTbKXnuHwNlDOQMggtpvxNf2KshZ/a0tlmhFcj0tC0 Bb829rm3KUmDaTAZdP6v29Sg4bosXYWN26o8PcABtA== X-Received: by 2002:a17:903:110c:b0:1c5:ec97:1718 with SMTP id n12-20020a170903110c00b001c5ec971718mr4519753plh.6.1697609883879; Tue, 17 Oct 2023 23:18:03 -0700 (PDT) Received: from n37-019-243.byted.org ([180.184.103.200]) by smtp.gmail.com with ESMTPSA id ix13-20020a170902f80d00b001c61acd5bd2sm2659116plb.112.2023.10.17.23.18.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Oct 2023 23:18:03 -0700 (PDT) From: Chuyi Zhou To: bpf@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@kernel.org, tj@kernel.org, linux-kernel@vger.kernel.org, Chuyi Zhou Subject: [RESEND PATCH bpf-next v6 3/8] bpf: Introduce task open coded iterator kfuncs Date: Wed, 18 Oct 2023 14:17:41 +0800 Message-Id: <20231018061746.111364-4-zhouchuyi@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20231018061746.111364-1-zhouchuyi@bytedance.com> References: <20231018061746.111364-1-zhouchuyi@bytedance.com> MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Tue, 17 Oct 2023 23:20:03 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780073110200741699 X-GMAIL-MSGID: 1780073110200741699 This patch adds kfuncs bpf_iter_task_{new,next,destroy} which allow creation and manipulation of struct bpf_iter_task in open-coded iterator style. BPF programs can use these kfuncs or through bpf_for_each macro to iterate all processes in the system. The API design keep consistent with SEC("iter/task"). bpf_iter_task_new() accepts a specific task and iterating type which allows: 1. iterating all process in the system (BPF_TASK_ITER_ALL_PROCS) 2. iterating all threads in the system (BPF_TASK_ITER_ALL_THREADS) 3. iterating all threads of a specific task (BPF_TASK_ITER_PROC_THREADS) Signed-off-by: Chuyi Zhou --- kernel/bpf/helpers.c | 3 + kernel/bpf/task_iter.c | 90 +++++++++++++++++++ .../testing/selftests/bpf/bpf_experimental.h | 5 ++ 3 files changed, 98 insertions(+) diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index c01441db9fd5..c25941531265 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -2563,6 +2563,9 @@ BTF_ID_FLAGS(func, bpf_iter_task_vma_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_iter_css_task_new, KF_ITER_NEW | KF_TRUSTED_ARGS) BTF_ID_FLAGS(func, bpf_iter_css_task_next, KF_ITER_NEXT | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_css_task_destroy, KF_ITER_DESTROY) +BTF_ID_FLAGS(func, bpf_iter_task_new, KF_ITER_NEW | KF_TRUSTED_ARGS) +BTF_ID_FLAGS(func, bpf_iter_task_next, KF_ITER_NEXT | KF_RET_NULL) +BTF_ID_FLAGS(func, bpf_iter_task_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_dynptr_adjust) BTF_ID_FLAGS(func, bpf_dynptr_is_null) BTF_ID_FLAGS(func, bpf_dynptr_is_rdonly) diff --git a/kernel/bpf/task_iter.c b/kernel/bpf/task_iter.c index e4126698cecf..faa1712c1df5 100644 --- a/kernel/bpf/task_iter.c +++ b/kernel/bpf/task_iter.c @@ -952,6 +952,96 @@ __bpf_kfunc void bpf_iter_css_task_destroy(struct bpf_iter_css_task *it) __diag_pop(); +struct bpf_iter_task { + __u64 __opaque[3]; +} __attribute__((aligned(8))); + +struct bpf_iter_task_kern { + struct task_struct *task; + struct task_struct *pos; + unsigned int flags; +} __attribute__((aligned(8))); + +enum { + /* all process in the system */ + BPF_TASK_ITER_ALL_PROCS, + /* all threads in the system */ + BPF_TASK_ITER_ALL_THREADS, + /* all threads of a specific process */ + BPF_TASK_ITER_PROC_THREADS +}; + +__diag_push(); +__diag_ignore_all("-Wmissing-prototypes", + "Global functions as their definitions will be in vmlinux BTF"); + +__bpf_kfunc int bpf_iter_task_new(struct bpf_iter_task *it, + struct task_struct *task, unsigned int flags) +{ + struct bpf_iter_task_kern *kit = (void *)it; + + BUILD_BUG_ON(sizeof(struct bpf_iter_task_kern) > sizeof(struct bpf_iter_task)); + BUILD_BUG_ON(__alignof__(struct bpf_iter_task_kern) != + __alignof__(struct bpf_iter_task)); + + kit->task = kit->pos = NULL; + switch (flags) { + case BPF_TASK_ITER_ALL_THREADS: + case BPF_TASK_ITER_ALL_PROCS: + case BPF_TASK_ITER_PROC_THREADS: + break; + default: + return -EINVAL; + } + + if (flags == BPF_TASK_ITER_PROC_THREADS) + kit->task = task; + else + kit->task = &init_task; + kit->pos = kit->task; + kit->flags = flags; + return 0; +} + +__bpf_kfunc struct task_struct *bpf_iter_task_next(struct bpf_iter_task *it) +{ + struct bpf_iter_task_kern *kit = (void *)it; + struct task_struct *pos; + unsigned int flags; + + flags = kit->flags; + pos = kit->pos; + + if (!pos) + return pos; + + if (flags == BPF_TASK_ITER_ALL_PROCS) + goto get_next_task; + + kit->pos = next_thread(kit->pos); + if (kit->pos == kit->task) { + if (flags == BPF_TASK_ITER_PROC_THREADS) { + kit->pos = NULL; + return pos; + } + } else + return pos; + +get_next_task: + kit->pos = next_task(kit->pos); + kit->task = kit->pos; + if (kit->pos == &init_task) + kit->pos = NULL; + + return pos; +} + +__bpf_kfunc void bpf_iter_task_destroy(struct bpf_iter_task *it) +{ +} + +__diag_pop(); + DEFINE_PER_CPU(struct mmap_unlock_irq_work, mmap_unlock_work); static void do_mmap_read_unlock(struct irq_work *entry) diff --git a/tools/testing/selftests/bpf/bpf_experimental.h b/tools/testing/selftests/bpf/bpf_experimental.h index 6792ed2b45d7..2f6c747aa874 100644 --- a/tools/testing/selftests/bpf/bpf_experimental.h +++ b/tools/testing/selftests/bpf/bpf_experimental.h @@ -465,5 +465,10 @@ extern int bpf_iter_css_task_new(struct bpf_iter_css_task *it, extern struct task_struct *bpf_iter_css_task_next(struct bpf_iter_css_task *it) __weak __ksym; extern void bpf_iter_css_task_destroy(struct bpf_iter_css_task *it) __weak __ksym; +struct bpf_iter_task; +extern int bpf_iter_task_new(struct bpf_iter_task *it, + struct task_struct *task, unsigned int flags) __weak __ksym; +extern struct task_struct *bpf_iter_task_next(struct bpf_iter_task *it) __weak __ksym; +extern void bpf_iter_task_destroy(struct bpf_iter_task *it) __weak __ksym; #endif From patchwork Wed Oct 18 06:17:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuyi Zhou X-Patchwork-Id: 154700 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp4595043vqb; Tue, 17 Oct 2023 23:20:33 -0700 (PDT) X-Google-Smtp-Source: AGHT+IH6EFtdxkbfwSbSLv+7/WyeotTfiX2jNbnEdenhR3z/V6i2lc9XAwJ93KdZ0XcLVaoAjXlF X-Received: by 2002:a05:6358:72a6:b0:166:d9c9:dbe with SMTP id w38-20020a05635872a600b00166d9c90dbemr4845971rwf.3.1697610032911; Tue, 17 Oct 2023 23:20:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697610032; cv=none; d=google.com; s=arc-20160816; b=AOTBbODJ/Ybn+ANrNkTrXsCt4d35cQWLf2ebsDAG8+RMVy9Q50+eUt3VIS6mN6uaFJ /aHMUZFR/g+eEWJ78obaNI/laM80aXGKUNkRbWEpezuxTBFq46BXq5kFXEPoWaTr/iwF 7Z/gUw6L2S5scZuBNn5P8P4v803ykR5PWdZnPwitpFSaasE+uVX0kpWkJrty3ImKIje0 Jv+C3mMWYfwACo3091XHY3/LHY8QgchRXkCa2EblhwdyjCqsQ/CxwZ1JAZDzS+HvYdIZ FRtNQDr69KzDC+Ya9o6jWlBsXxg2k9riPzUBDCuKtcmA56TkFsgOoyZbRUQxGk3XjBds AMUw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=AL8epgVpk2Z5NetWJuhkyFSgrHs5uVsKQ57V0KAcMww=; fh=DR7g1EcWKOXTEoooUPBSJXUaklSrDEYzv6YDdhz1CwE=; b=bc3vdnm/iBVpLjI7OeiWNOwngg8ojKBLxKUnakc4Uv9hIJosWHH3T2gkM4S0tFbZlF h9W3bM55s4vKXPLv//9xTcraCRTcIWOEX5rg+Hk6qUyJ05Da593aY7jJJui0kzxIlbEu DnSerYItD64UA+FgESZ9UTUL74V9qa0BJ4eGecOqPBQKvWpcTdpzFwdC37FzJp8p71P4 T6WQa0nGBYJqyS2WFnwGdWIPjuX4YqM2cU9hJtSTCrpHBfpCqOyzRAiZwy2rFhuuRpyq M9tNHHVkUDShKP82/p0PEXMn6vK28UQKIT9w2Y6azUbtgAKEFhasa1e/c3ArgVUPlbkW 68pg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=Yv2ZdPMd; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from lipwig.vger.email (lipwig.vger.email. [2620:137:e000::3:3]) by mx.google.com with ESMTPS id fc14-20020a056a002e0e00b006bec15892c9si377930pfb.141.2023.10.17.23.20.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Oct 2023 23:20:32 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) client-ip=2620:137:e000::3:3; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=Yv2ZdPMd; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id 93E098149739; Tue, 17 Oct 2023 23:20:30 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344621AbjJRGUR (ORCPT + 23 others); Wed, 18 Oct 2023 02:20:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40682 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344515AbjJRGTS (ORCPT ); Wed, 18 Oct 2023 02:19:18 -0400 Received: from mail-pl1-x62d.google.com (mail-pl1-x62d.google.com [IPv6:2607:f8b0:4864:20::62d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 16A5210E3 for ; Tue, 17 Oct 2023 23:18:07 -0700 (PDT) Received: by mail-pl1-x62d.google.com with SMTP id d9443c01a7336-1c0ecb9a075so44247805ad.2 for ; Tue, 17 Oct 2023 23:18:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1697609887; x=1698214687; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=AL8epgVpk2Z5NetWJuhkyFSgrHs5uVsKQ57V0KAcMww=; b=Yv2ZdPMdTZqcA++WVr52++fTcyXkS+b38DTPX4BzDYC42khJIP7Jy+/onhnjxsPH7K o2tBwTDJEQPE9RXZaoQI/KEyGnN45sY13GDrNIT3rdAmMDuV9yvWv6dawMm6BMGuu2Q0 ZWb2RKk+DbRjr10UCtZyAHAY4ch+V6JNfyZeH9eFKPZyRi0ln+XnImQJH1MYp5+SFIg0 6E8C7xremEzEZirBGQoh1J9lZX28MjpevIW3jnCLGe9gNwdqjndtfPtY3QZNveWn1jaP 3iFwdvT9xAdioQVo0u7/X3AW0MVj3ubPW3cRaXENJajE5HIugR1AyJgy4pO5wnk0Nr/F Xzew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697609887; x=1698214687; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AL8epgVpk2Z5NetWJuhkyFSgrHs5uVsKQ57V0KAcMww=; b=IYHNuubJwcXXIkV342bOTa/sOQmifbCXMywLamdhEvKwCTURX0kLdxer4WCHJtGsZ2 T6z5ivCMW+jSdEDCCXvT4iEddvjJO7K4OM8KxEy+5vDImqsvNIIsUW3rUG5dKW0Ijkgv RxGoPFOpeEXC8hkMziQYbLS2as8TNv5SnAlNkqZGzcc9pJa4ZW4zfXQlNMYAnAD0rUbl stw7qF8a/p10hnT0ZGclx1gNNKVWQgDkJzXTkRmp0/HBgACmWwgff3BxA9qgCxm99C9e HtZMRGP2ZZhVYCDSrzCtbM2oVVDYYoBMrYsiQIjutC7oMIZ2cfJ8dhn188A+AsZI0Jh5 989g== X-Gm-Message-State: AOJu0Yya+lDtTXzTMSJ3i+t84tFXa2UyFFgZAhz7ZK/fmWqD0UGgLOFF 7KfCvnyuLyZDAKeAMbXr4gdCOQ== X-Received: by 2002:a17:902:fa8e:b0:1c9:d90b:c3e4 with SMTP id lc14-20020a170902fa8e00b001c9d90bc3e4mr4246104plb.10.1697609886810; Tue, 17 Oct 2023 23:18:06 -0700 (PDT) Received: from n37-019-243.byted.org ([180.184.103.200]) by smtp.gmail.com with ESMTPSA id ix13-20020a170902f80d00b001c61acd5bd2sm2659116plb.112.2023.10.17.23.18.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Oct 2023 23:18:06 -0700 (PDT) From: Chuyi Zhou To: bpf@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@kernel.org, tj@kernel.org, linux-kernel@vger.kernel.org, Chuyi Zhou Subject: [RESEND PATCH bpf-next v6 4/8] bpf: Introduce css open-coded iterator kfuncs Date: Wed, 18 Oct 2023 14:17:42 +0800 Message-Id: <20231018061746.111364-5-zhouchuyi@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20231018061746.111364-1-zhouchuyi@bytedance.com> References: <20231018061746.111364-1-zhouchuyi@bytedance.com> MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Tue, 17 Oct 2023 23:20:30 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780073137848829999 X-GMAIL-MSGID: 1780073137848829999 This Patch adds kfuncs bpf_iter_css_{new,next,destroy} which allow creation and manipulation of struct bpf_iter_css in open-coded iterator style. These kfuncs actually wrapps css_next_descendant_{pre, post}. css_iter can be used to: 1) iterating a sepcific cgroup tree with pre/post/up order 2) iterating cgroup_subsystem in BPF Prog, like for_each_mem_cgroup_tree/cpuset_for_each_descendant_pre in kernel. The API design is consistent with cgroup_iter. bpf_iter_css_new accepts parameters defining iteration order and starting css. Here we also reuse BPF_CGROUP_ITER_DESCENDANTS_PRE, BPF_CGROUP_ITER_DESCENDANTS_POST, BPF_CGROUP_ITER_ANCESTORS_UP enums. Signed-off-by: Chuyi Zhou Acked-by: Tejun Heo --- kernel/bpf/cgroup_iter.c | 65 +++++++++++++++++++ kernel/bpf/helpers.c | 3 + .../testing/selftests/bpf/bpf_experimental.h | 6 ++ 3 files changed, 74 insertions(+) diff --git a/kernel/bpf/cgroup_iter.c b/kernel/bpf/cgroup_iter.c index 810378f04fbc..209e5135f9fb 100644 --- a/kernel/bpf/cgroup_iter.c +++ b/kernel/bpf/cgroup_iter.c @@ -294,3 +294,68 @@ static int __init bpf_cgroup_iter_init(void) } late_initcall(bpf_cgroup_iter_init); + +struct bpf_iter_css { + __u64 __opaque[3]; +} __attribute__((aligned(8))); + +struct bpf_iter_css_kern { + struct cgroup_subsys_state *start; + struct cgroup_subsys_state *pos; + unsigned int flags; +} __attribute__((aligned(8))); + +__diag_push(); +__diag_ignore_all("-Wmissing-prototypes", + "Global functions as their definitions will be in vmlinux BTF"); + +__bpf_kfunc int bpf_iter_css_new(struct bpf_iter_css *it, + struct cgroup_subsys_state *start, unsigned int flags) +{ + struct bpf_iter_css_kern *kit = (void *)it; + + BUILD_BUG_ON(sizeof(struct bpf_iter_css_kern) > sizeof(struct bpf_iter_css)); + BUILD_BUG_ON(__alignof__(struct bpf_iter_css_kern) != __alignof__(struct bpf_iter_css)); + + kit->start = NULL; + switch (flags) { + case BPF_CGROUP_ITER_DESCENDANTS_PRE: + case BPF_CGROUP_ITER_DESCENDANTS_POST: + case BPF_CGROUP_ITER_ANCESTORS_UP: + break; + default: + return -EINVAL; + } + + kit->start = start; + kit->pos = NULL; + kit->flags = flags; + return 0; +} + +__bpf_kfunc struct cgroup_subsys_state *bpf_iter_css_next(struct bpf_iter_css *it) +{ + struct bpf_iter_css_kern *kit = (void *)it; + + if (!kit->start) + return NULL; + + switch (kit->flags) { + case BPF_CGROUP_ITER_DESCENDANTS_PRE: + kit->pos = css_next_descendant_pre(kit->pos, kit->start); + break; + case BPF_CGROUP_ITER_DESCENDANTS_POST: + kit->pos = css_next_descendant_post(kit->pos, kit->start); + break; + case BPF_CGROUP_ITER_ANCESTORS_UP: + kit->pos = kit->pos ? kit->pos->parent : kit->start; + } + + return kit->pos; +} + +__bpf_kfunc void bpf_iter_css_destroy(struct bpf_iter_css *it) +{ +} + +__diag_pop(); \ No newline at end of file diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index c25941531265..b1d285ed4796 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -2566,6 +2566,9 @@ BTF_ID_FLAGS(func, bpf_iter_css_task_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_iter_task_new, KF_ITER_NEW | KF_TRUSTED_ARGS) BTF_ID_FLAGS(func, bpf_iter_task_next, KF_ITER_NEXT | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_task_destroy, KF_ITER_DESTROY) +BTF_ID_FLAGS(func, bpf_iter_css_new, KF_ITER_NEW | KF_TRUSTED_ARGS) +BTF_ID_FLAGS(func, bpf_iter_css_next, KF_ITER_NEXT | KF_RET_NULL) +BTF_ID_FLAGS(func, bpf_iter_css_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_dynptr_adjust) BTF_ID_FLAGS(func, bpf_dynptr_is_null) BTF_ID_FLAGS(func, bpf_dynptr_is_rdonly) diff --git a/tools/testing/selftests/bpf/bpf_experimental.h b/tools/testing/selftests/bpf/bpf_experimental.h index 2f6c747aa874..1386baf9ae4a 100644 --- a/tools/testing/selftests/bpf/bpf_experimental.h +++ b/tools/testing/selftests/bpf/bpf_experimental.h @@ -471,4 +471,10 @@ extern int bpf_iter_task_new(struct bpf_iter_task *it, extern struct task_struct *bpf_iter_task_next(struct bpf_iter_task *it) __weak __ksym; extern void bpf_iter_task_destroy(struct bpf_iter_task *it) __weak __ksym; +struct bpf_iter_css; +extern int bpf_iter_css_new(struct bpf_iter_css *it, + struct cgroup_subsys_state *start, unsigned int flags) __weak __ksym; +extern struct cgroup_subsys_state *bpf_iter_css_next(struct bpf_iter_css *it) __weak __ksym; +extern void bpf_iter_css_destroy(struct bpf_iter_css *it) __weak __ksym; + #endif From patchwork Wed Oct 18 06:17:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuyi Zhou X-Patchwork-Id: 154708 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp4596138vqb; Tue, 17 Oct 2023 23:23:37 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFE8ZgphhZmWCfMdWJukwvtqn17QJAMPmGRre1V2jJoCiOOpU/qYQXMs5VHWBKcybVjdEq7 X-Received: by 2002:a17:902:f092:b0:1bb:ac37:384b with SMTP id p18-20020a170902f09200b001bbac37384bmr4419091pla.6.1697610216756; Tue, 17 Oct 2023 23:23:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697610216; cv=none; d=google.com; s=arc-20160816; b=wu0S5iYcDI9VG2D7NLKdNg0YqHs46IMuxpqn5PnT0SCr71ych2yIIL9uzly+nymFvS FVCQkMZ0HPw9iLRRZn0zfERaUNvpuOm3jWIN4S5hjNfiw9NUY9Gb73UMDI68iHJWllK9 9JRb+73BxhdNdJL6qgjgGUVkdMVNfQZ7He86vbLJjt51Rba0QjeHOC387o6zEFDm/bcK ee8os7FDS7kmpPYde21pQuqZY91cRvaKkJZtfIQqydIgvwDOnP85uJ1CXMosq0bsTjNJ DdwZ0+YyMA3YXUz/dGjMSaMTV5dEfYvro8KQjZd8bpRpam6+Dvi/zhMcMVqE/X5K/wAo /q7Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=AQMAZzziueZcwnJjUGt9dx4cdF9gdrhIA4BtS7H5D4g=; fh=DR7g1EcWKOXTEoooUPBSJXUaklSrDEYzv6YDdhz1CwE=; b=Hoo2/xgL01ZEgqzVf5o4HQyixInJMdrLk2VaQFIkMj7hMyQTQCKZMGRDkMpdbdxfEi 1EUbzrK6efAlp5hpKjOvW1Um2v3pA8MyV62Bs2vJOi/mdfIXyRPZZVTnw+wf69asgiqs 3SUFUgrhAbP2f4kNbhdFDNCz99z0Sl3jdLlRswSqnExbkQxLd4kj/bPkGbLOVdAg8jcQ +qTT93I+uUaYe0UNLlcFUBeniDhEqqDkZO7dblsMx/oURvd7UR8jktDypTuvh9afrqAa bvYoNCWv0IixRQGR/k7ITfYNGJiwPC58Yyhw2Z74dCIRxm+UklnIes/18zSLGhiE09ku VeAg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=QSYq89qR; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from pete.vger.email (pete.vger.email. [23.128.96.36]) by mx.google.com with ESMTPS id p10-20020a170902e74a00b001c71eb782c0si3694782plf.464.2023.10.17.23.23.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Oct 2023 23:23:36 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) client-ip=23.128.96.36; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=QSYq89qR; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by pete.vger.email (Postfix) with ESMTP id 3213E8074CBB; Tue, 17 Oct 2023 23:23:28 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at pete.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235038AbjJRGWo (ORCPT + 23 others); Wed, 18 Oct 2023 02:22:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34668 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229700AbjJRGW2 (ORCPT ); Wed, 18 Oct 2023 02:22:28 -0400 Received: from mail-pl1-x62e.google.com (mail-pl1-x62e.google.com [IPv6:2607:f8b0:4864:20::62e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5946E1707 for ; Tue, 17 Oct 2023 23:18:11 -0700 (PDT) Received: by mail-pl1-x62e.google.com with SMTP id d9443c01a7336-1c9b7c234a7so55863775ad.3 for ; Tue, 17 Oct 2023 23:18:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1697609889; x=1698214689; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=AQMAZzziueZcwnJjUGt9dx4cdF9gdrhIA4BtS7H5D4g=; b=QSYq89qRGwdqkI88eFknudZLEHNL+FQlXBBsID2Ru6ED2VNBMRTILK7829IuBjqWcr yUlp40AZvkBaafA5LtjvUm5tyn1jQoYHB4WBaCyRtPnN3Jq4PjNTvGgfjuW+zK32H2kw y1/Wro4D4JBqHFgLgKYoFLPlBl4tW+W+YNvhEnF/x+6XgRsWkvkREA+9ikJW7psBnuut ALJPMPsA0CJcq66wfdLEzGgHOH+5wY/GTaQbs9hjL+wMvFfmmz8d1wcRWwpEeDOorZgF 025XcUdejgrj+g1k+qVVzAS8V8YTBF8pOL2l3WE50UwlZ++o2NSUdOFnuw03kay9kM8I FKQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697609889; x=1698214689; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AQMAZzziueZcwnJjUGt9dx4cdF9gdrhIA4BtS7H5D4g=; b=uPQy+fMeBa+X20vBFSdRAfIqs5+uazl/Uzog3fqPH9HrLOpKRS9ZHDAtuLTMFW1VVt NaMNLrHBV6q54CvQN1S7ebcPa5mvg94flJfyRqYpCN50R1M8OPHJiQVkKwILL6fp7DgP ty6Kq2SbHy4/i2dRJxtwdZbMw6Tn4M33SV8RDzWJ8gwEHIWFG9ZN8prouVWB3M7SE6+a QDtdeaxbaXFD56H6v9w7AfqcvuRIi8/v4jbPxNZuoZyx6YXqyy8CIvKLg3I7EvGAUoit NdhT6zSVf5wP78NRV3G7LfbMcGPW+Wpugb7vnEdX3Gq1098If2ifh+/0cRGbi7L0un9P GHOQ== X-Gm-Message-State: AOJu0YybXjtASh9IJnJAo7ZV6fy3ZRfcgad6LLEVVmjsX9B8oYayTaWM /sfNXZRle986LgAGYFH6YH/BbpGFfqF8QlJHCt37Ww== X-Received: by 2002:a17:903:249:b0:1ca:2743:bf79 with SMTP id j9-20020a170903024900b001ca2743bf79mr5399429plh.39.1697609889457; Tue, 17 Oct 2023 23:18:09 -0700 (PDT) Received: from n37-019-243.byted.org ([180.184.103.200]) by smtp.gmail.com with ESMTPSA id ix13-20020a170902f80d00b001c61acd5bd2sm2659116plb.112.2023.10.17.23.18.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Oct 2023 23:18:09 -0700 (PDT) From: Chuyi Zhou To: bpf@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@kernel.org, tj@kernel.org, linux-kernel@vger.kernel.org, Chuyi Zhou Subject: [RESEND PATCH bpf-next v6 5/8] bpf: teach the verifier to enforce css_iter and task_iter in RCU CS Date: Wed, 18 Oct 2023 14:17:43 +0800 Message-Id: <20231018061746.111364-6-zhouchuyi@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20231018061746.111364-1-zhouchuyi@bytedance.com> References: <20231018061746.111364-1-zhouchuyi@bytedance.com> MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Tue, 17 Oct 2023 23:23:28 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780073330604311409 X-GMAIL-MSGID: 1780073330604311409 css_iter and task_iter should be used in rcu section. Specifically, in sleepable progs explicit bpf_rcu_read_lock() is needed before use these iters. In normal bpf progs that have implicit rcu_read_lock(), it's OK to use them directly. This patch adds a new a KF flag KF_RCU_PROTECTED for bpf_iter_task_new and bpf_iter_css_new. It means the kfunc should be used in RCU CS. We check whether we are in rcu cs before we want to invoke this kfunc. If the rcu protection is guaranteed, we would let st->type = PTR_TO_STACK | MEM_RCU. Once user do rcu_unlock during the iteration, state MEM_RCU of regs would be cleared. is_iter_reg_valid_init() will reject if reg->type is UNTRUSTED. It is worth noting that currently, bpf_rcu_read_unlock does not clear the state of the STACK_ITER reg, since bpf_for_each_spilled_reg only considers STACK_SPILL. This patch also let bpf_for_each_spilled_reg search STACK_ITER. Signed-off-by: Chuyi Zhou Acked-by: Andrii Nakryiko --- include/linux/bpf_verifier.h | 19 ++++++++------ include/linux/btf.h | 1 + kernel/bpf/helpers.c | 4 +-- kernel/bpf/verifier.c | 50 ++++++++++++++++++++++++++++-------- 4 files changed, 53 insertions(+), 21 deletions(-) diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h index 94ec766432f5..e67cd45a85be 100644 --- a/include/linux/bpf_verifier.h +++ b/include/linux/bpf_verifier.h @@ -386,19 +386,18 @@ struct bpf_verifier_state { u32 jmp_history_cnt; }; -#define bpf_get_spilled_reg(slot, frame) \ +#define bpf_get_spilled_reg(slot, frame, mask) \ (((slot < frame->allocated_stack / BPF_REG_SIZE) && \ - (frame->stack[slot].slot_type[0] == STACK_SPILL)) \ + ((1 << frame->stack[slot].slot_type[0]) & (mask))) \ ? &frame->stack[slot].spilled_ptr : NULL) /* Iterate over 'frame', setting 'reg' to either NULL or a spilled register. */ -#define bpf_for_each_spilled_reg(iter, frame, reg) \ - for (iter = 0, reg = bpf_get_spilled_reg(iter, frame); \ +#define bpf_for_each_spilled_reg(iter, frame, reg, mask) \ + for (iter = 0, reg = bpf_get_spilled_reg(iter, frame, mask); \ iter < frame->allocated_stack / BPF_REG_SIZE; \ - iter++, reg = bpf_get_spilled_reg(iter, frame)) + iter++, reg = bpf_get_spilled_reg(iter, frame, mask)) -/* Invoke __expr over regsiters in __vst, setting __state and __reg */ -#define bpf_for_each_reg_in_vstate(__vst, __state, __reg, __expr) \ +#define bpf_for_each_reg_in_vstate_mask(__vst, __state, __reg, __mask, __expr) \ ({ \ struct bpf_verifier_state *___vstate = __vst; \ int ___i, ___j; \ @@ -410,7 +409,7 @@ struct bpf_verifier_state { __reg = &___regs[___j]; \ (void)(__expr); \ } \ - bpf_for_each_spilled_reg(___j, __state, __reg) { \ + bpf_for_each_spilled_reg(___j, __state, __reg, __mask) { \ if (!__reg) \ continue; \ (void)(__expr); \ @@ -418,6 +417,10 @@ struct bpf_verifier_state { } \ }) +/* Invoke __expr over regsiters in __vst, setting __state and __reg */ +#define bpf_for_each_reg_in_vstate(__vst, __state, __reg, __expr) \ + bpf_for_each_reg_in_vstate_mask(__vst, __state, __reg, 1 << STACK_SPILL, __expr) + /* linked list of verifier states used to prune search */ struct bpf_verifier_state_list { struct bpf_verifier_state state; diff --git a/include/linux/btf.h b/include/linux/btf.h index 928113a80a95..c2231c64d60b 100644 --- a/include/linux/btf.h +++ b/include/linux/btf.h @@ -74,6 +74,7 @@ #define KF_ITER_NEW (1 << 8) /* kfunc implements BPF iter constructor */ #define KF_ITER_NEXT (1 << 9) /* kfunc implements BPF iter next method */ #define KF_ITER_DESTROY (1 << 10) /* kfunc implements BPF iter destructor */ +#define KF_RCU_PROTECTED (1 << 11) /* kfunc should be protected by rcu cs when they are invoked */ /* * Tag marking a kernel function as a kfunc. This is meant to minimize the diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index b1d285ed4796..da058aead20c 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -2563,10 +2563,10 @@ BTF_ID_FLAGS(func, bpf_iter_task_vma_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_iter_css_task_new, KF_ITER_NEW | KF_TRUSTED_ARGS) BTF_ID_FLAGS(func, bpf_iter_css_task_next, KF_ITER_NEXT | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_css_task_destroy, KF_ITER_DESTROY) -BTF_ID_FLAGS(func, bpf_iter_task_new, KF_ITER_NEW | KF_TRUSTED_ARGS) +BTF_ID_FLAGS(func, bpf_iter_task_new, KF_ITER_NEW | KF_TRUSTED_ARGS | KF_RCU_PROTECTED) BTF_ID_FLAGS(func, bpf_iter_task_next, KF_ITER_NEXT | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_task_destroy, KF_ITER_DESTROY) -BTF_ID_FLAGS(func, bpf_iter_css_new, KF_ITER_NEW | KF_TRUSTED_ARGS) +BTF_ID_FLAGS(func, bpf_iter_css_new, KF_ITER_NEW | KF_TRUSTED_ARGS | KF_RCU_PROTECTED) BTF_ID_FLAGS(func, bpf_iter_css_next, KF_ITER_NEXT | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_css_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_dynptr_adjust) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 974713185269..fcdf2382153a 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -1173,7 +1173,12 @@ static bool is_dynptr_type_expected(struct bpf_verifier_env *env, struct bpf_reg static void __mark_reg_known_zero(struct bpf_reg_state *reg); +static bool in_rcu_cs(struct bpf_verifier_env *env); + +static bool is_kfunc_rcu_protected(struct bpf_kfunc_call_arg_meta *meta); + static int mark_stack_slots_iter(struct bpf_verifier_env *env, + struct bpf_kfunc_call_arg_meta *meta, struct bpf_reg_state *reg, int insn_idx, struct btf *btf, u32 btf_id, int nr_slots) { @@ -1194,6 +1199,12 @@ static int mark_stack_slots_iter(struct bpf_verifier_env *env, __mark_reg_known_zero(st); st->type = PTR_TO_STACK; /* we don't have dedicated reg type */ + if (is_kfunc_rcu_protected(meta)) { + if (in_rcu_cs(env)) + st->type |= MEM_RCU; + else + st->type |= PTR_UNTRUSTED; + } st->live |= REG_LIVE_WRITTEN; st->ref_obj_id = i == 0 ? id : 0; st->iter.btf = btf; @@ -1268,7 +1279,7 @@ static bool is_iter_reg_valid_uninit(struct bpf_verifier_env *env, return true; } -static bool is_iter_reg_valid_init(struct bpf_verifier_env *env, struct bpf_reg_state *reg, +static int is_iter_reg_valid_init(struct bpf_verifier_env *env, struct bpf_reg_state *reg, struct btf *btf, u32 btf_id, int nr_slots) { struct bpf_func_state *state = func(env, reg); @@ -1276,26 +1287,28 @@ static bool is_iter_reg_valid_init(struct bpf_verifier_env *env, struct bpf_reg_ spi = iter_get_spi(env, reg, nr_slots); if (spi < 0) - return false; + return -EINVAL; for (i = 0; i < nr_slots; i++) { struct bpf_stack_state *slot = &state->stack[spi - i]; struct bpf_reg_state *st = &slot->spilled_ptr; + if (st->type & PTR_UNTRUSTED) + return -EPROTO; /* only main (first) slot has ref_obj_id set */ if (i == 0 && !st->ref_obj_id) - return false; + return -EINVAL; if (i != 0 && st->ref_obj_id) - return false; + return -EINVAL; if (st->iter.btf != btf || st->iter.btf_id != btf_id) - return false; + return -EINVAL; for (j = 0; j < BPF_REG_SIZE; j++) if (slot->slot_type[j] != STACK_ITER) - return false; + return -EINVAL; } - return true; + return 0; } /* Check if given stack slot is "special": @@ -7640,15 +7653,24 @@ static int process_iter_arg(struct bpf_verifier_env *env, int regno, int insn_id return err; } - err = mark_stack_slots_iter(env, reg, insn_idx, meta->btf, btf_id, nr_slots); + err = mark_stack_slots_iter(env, meta, reg, insn_idx, meta->btf, btf_id, nr_slots); if (err) return err; } else { /* iter_next() or iter_destroy() expect initialized iter state*/ - if (!is_iter_reg_valid_init(env, reg, meta->btf, btf_id, nr_slots)) { + err = is_iter_reg_valid_init(env, reg, meta->btf, btf_id, nr_slots); + switch (err) { + case 0: + break; + case -EINVAL: verbose(env, "expected an initialized iter_%s as arg #%d\n", iter_type_str(meta->btf, btf_id), regno); - return -EINVAL; + return err; + case -EPROTO: + verbose(env, "expected an RCU CS when using %s\n", meta->func_name); + return err; + default: + return err; } spi = iter_get_spi(env, reg, nr_slots); @@ -10231,6 +10253,11 @@ static bool is_kfunc_rcu(struct bpf_kfunc_call_arg_meta *meta) return meta->kfunc_flags & KF_RCU; } +static bool is_kfunc_rcu_protected(struct bpf_kfunc_call_arg_meta *meta) +{ + return meta->kfunc_flags & KF_RCU_PROTECTED; +} + static bool __kfunc_param_match_suffix(const struct btf *btf, const struct btf_param *arg, const char *suffix) @@ -11582,6 +11609,7 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, if (env->cur_state->active_rcu_lock) { struct bpf_func_state *state; struct bpf_reg_state *reg; + u32 clear_mask = (1 << STACK_SPILL) | (1 << STACK_ITER); if (in_rbtree_lock_required_cb(env) && (rcu_lock || rcu_unlock)) { verbose(env, "Calling bpf_rcu_read_{lock,unlock} in unnecessary rbtree callback\n"); @@ -11592,7 +11620,7 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, verbose(env, "nested rcu read lock (kernel function %s)\n", func_name); return -EINVAL; } else if (rcu_unlock) { - bpf_for_each_reg_in_vstate(env->cur_state, state, reg, ({ + bpf_for_each_reg_in_vstate_mask(env->cur_state, state, reg, clear_mask, ({ if (reg->type & MEM_RCU) { reg->type &= ~(MEM_RCU | PTR_MAYBE_NULL); reg->type |= PTR_UNTRUSTED; From patchwork Wed Oct 18 06:17:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuyi Zhou X-Patchwork-Id: 154704 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp4595727vqb; Tue, 17 Oct 2023 23:22:24 -0700 (PDT) X-Google-Smtp-Source: AGHT+IF8BQjJAiNWJPsrko9SdSfGjJD5COCWorhGmcuAZCvSgTNM+lFJKwlMgqlkHFkQ0JLcUYBY X-Received: by 2002:a05:6359:5e20:b0:166:d93e:339 with SMTP id pw32-20020a0563595e2000b00166d93e0339mr4244124rwb.0.1697610144555; Tue, 17 Oct 2023 23:22:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697610144; cv=none; d=google.com; s=arc-20160816; b=xmDTXobwwEinRQ5sIEK57MYMulf0Upq2IWS+lIkAljrRFfMZb6XOuis64ZzErsQBve xftMiSS/WYTj+aaeWR0zBdERlayEpkXY6Nhn4VOawzkyXxT0fq2P9P4KB5WBwyBEBSVv GQhKCFnSwRLRFTsLtC/iooVoED6hn7YXG/l+IaZs5Y4m0H0jDlmkb+NsJi5KcTV5VAkh RJ4IxRqEeZ1mGVUHKVSxxuR0ZNSxAPH7lqeT0N77tRZ3pm5i9nXDPUFv/AkBWAS3yHU1 s7J/J4VjirUKRqTYGF7f0E/ImBO5A/4MNmJ5AwRiMiFJsPXF6LmbWiHBPisKRgclY6bA Dq9Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=pBjREUfdYjysteuQq+CF2MBmAUs5yTb/YGRZ4Q4shvo=; fh=DR7g1EcWKOXTEoooUPBSJXUaklSrDEYzv6YDdhz1CwE=; b=NmtEjyH6XTwpdFhYbI24RRPxw6z46Dmu3RWrI2dl070cgvhmM37METifPC1Qas5EFa g4bBUYLlxP2kgp1SV1p9/iU7LS5gNJlajc77zkaPUmYaC3Hg18y+EQRQFp7x5SbHTcPQ UNLmnWWh3Zw1GJRhkew7rD1jSqr2SW/+oeGGrUZNnVytadJhqTyq60We/tzix4nKZSbh b8sjOre7/nqWriWMpxLE7uB4UlPMW/B3g0oRmVq2df8rynYpyM7BLiJRkBSmAQcE6x0A RJixp44PIhN8JrHq3S6SZsDJ6psENWxi1BVgqNrh8UAO3D+QdsWg5lh1zZJECwQK+mkP dktQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=WW3aO8hM; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from morse.vger.email (morse.vger.email. [23.128.96.31]) by mx.google.com with ESMTPS id k200-20020a6284d1000000b006bd018a5bcfsi3181114pfd.340.2023.10.17.23.22.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Oct 2023 23:22:24 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) client-ip=23.128.96.31; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=WW3aO8hM; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by morse.vger.email (Postfix) with ESMTP id 0FC54807C76B; Tue, 17 Oct 2023 23:21:51 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at morse.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344614AbjJRGVR (ORCPT + 23 others); Wed, 18 Oct 2023 02:21:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37628 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344505AbjJRGVF (ORCPT ); Wed, 18 Oct 2023 02:21:05 -0400 Received: from mail-pl1-x633.google.com (mail-pl1-x633.google.com [IPv6:2607:f8b0:4864:20::633]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 48A9A1989 for ; Tue, 17 Oct 2023 23:18:17 -0700 (PDT) Received: by mail-pl1-x633.google.com with SMTP id d9443c01a7336-1c88b467ef8so45738055ad.0 for ; Tue, 17 Oct 2023 23:18:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1697609892; x=1698214692; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=pBjREUfdYjysteuQq+CF2MBmAUs5yTb/YGRZ4Q4shvo=; b=WW3aO8hMPq9dtwDyVMW8bvYBqA9W5ayBgjr8EyMbwXlENVQYLcT/O7MnckTrEdGeAg t3Ij9IMel+5V/0syQekAwBDa5pGTXJN50vM4JM/Ert0UO3OjadDmHxPrbzYIjEpp05YR 8fckf4LZZ3YcnWgZBC+D3ppUdF1DZUnLWGicx7U3Vl7O531PMA4dtGJfd3VHaivS4sCL p5ePlszhPZWiR4srszNmO/1Uyo7dfe9UCHwRp/6uIloSdMXEBFBCC/FprMW3sGKB6ZdI noD7+sMWbwoACGT/faOYyZyYCEjEPBME58+XZ6OBAoubQZav46qxr2N5exRNtKxLUJMh NScA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697609892; x=1698214692; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=pBjREUfdYjysteuQq+CF2MBmAUs5yTb/YGRZ4Q4shvo=; b=TT3bXaRpfCtpwW9v1an6NOHnFiAPgKJuYJU6h7II0YF6Kxcojlv7Hm1eOUV9Ffs/JD uF8UDMI7qsgE3YpuyZKhGmhT2u5GzRhnxQ6o9Z34WjU0lENMSZqLYQ+SkAGWLjGXViOa M+r/2rHpNiFOlv2E3jOHIcOJYsTTRyP9H/KjtNn+jq1olwD5kzr/e1PQVUPWnS9AlGeZ LFLwmDpR21qxzldqog3LwACL+fTPUWQccInZgsZcQspvNQBfZ+1cUAPhZShlWa2HQABW bysE9lbr8adnS+aW4aA17ZJ/Vj00BQnbcsKP9yhTvIMjh7Y0c7tFjHRtpaZDnFdEO1np tt3Q== X-Gm-Message-State: AOJu0Yw356dmOE0iJPvNYFnhdeiOZv0m8WJT2xnueA9XA/4A/DXfioO7 ywmBc2rl5ROlBdP6DWvq85OHKw== X-Received: by 2002:a17:90a:31c:b0:27d:1ce5:1eb6 with SMTP id 28-20020a17090a031c00b0027d1ce51eb6mr4382459pje.17.1697609891808; Tue, 17 Oct 2023 23:18:11 -0700 (PDT) Received: from n37-019-243.byted.org ([180.184.103.200]) by smtp.gmail.com with ESMTPSA id ix13-20020a170902f80d00b001c61acd5bd2sm2659116plb.112.2023.10.17.23.18.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Oct 2023 23:18:11 -0700 (PDT) From: Chuyi Zhou To: bpf@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@kernel.org, tj@kernel.org, linux-kernel@vger.kernel.org, Chuyi Zhou Subject: [RESEND PATCH bpf-next v6 6/8] bpf: Let bpf_iter_task_new accept null task ptr Date: Wed, 18 Oct 2023 14:17:44 +0800 Message-Id: <20231018061746.111364-7-zhouchuyi@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20231018061746.111364-1-zhouchuyi@bytedance.com> References: <20231018061746.111364-1-zhouchuyi@bytedance.com> MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on morse.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (morse.vger.email [0.0.0.0]); Tue, 17 Oct 2023 23:21:51 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780073255087414352 X-GMAIL-MSGID: 1780073255087414352 When using task_iter to iterate all threads of a specific task, we enforce that the user must pass a valid task pointer to ensure safety. However, when iterating all threads/process in the system, BPF verifier still require a valid ptr instead of "nullable" pointer, even though it's pointless, which is a kind of surprising from usability standpoint. It would be nice if we could let that kfunc accept a explicit null pointer when we are using BPF_TASK_ITER_ALL_{PROCS, THREADS} and a valid pointer when using BPF_TASK_ITER_THREAD. Given a trival kfunc: __bpf_kfunc void FN(struct TYPE_A *obj); BPF Prog would reject a nullptr for obj. The error info is: "arg#x pointer type xx xx must point to scalar, or struct with scalar" reported by get_kfunc_ptr_arg_type(). The reg->type is SCALAR_VALUE and the btf type of ref_t is not scalar or scalar_struct which leads to the rejection of get_kfunc_ptr_arg_type. This patch add "__nullable" annotation: __bpf_kfunc void FN(struct TYPE_A *obj__nullable); Here __nullable indicates obj can be optional, user can pass a explicit nullptr or a normal TYPE_A pointer. In get_kfunc_ptr_arg_type(), we will detect whether the current arg is optional and register is null, If so, return a new kfunc_ptr_arg_type KF_ARG_PTR_TO_NULL and skip to the next arg in check_kfunc_args(). Signed-off-by: Chuyi Zhou Acked-by: Andrii Nakryiko --- kernel/bpf/task_iter.c | 7 +++++-- kernel/bpf/verifier.c | 13 ++++++++++++- 2 files changed, 17 insertions(+), 3 deletions(-) diff --git a/kernel/bpf/task_iter.c b/kernel/bpf/task_iter.c index faa1712c1df5..59e747938bdb 100644 --- a/kernel/bpf/task_iter.c +++ b/kernel/bpf/task_iter.c @@ -976,7 +976,7 @@ __diag_ignore_all("-Wmissing-prototypes", "Global functions as their definitions will be in vmlinux BTF"); __bpf_kfunc int bpf_iter_task_new(struct bpf_iter_task *it, - struct task_struct *task, unsigned int flags) + struct task_struct *task__nullable, unsigned int flags) { struct bpf_iter_task_kern *kit = (void *)it; @@ -988,14 +988,17 @@ __bpf_kfunc int bpf_iter_task_new(struct bpf_iter_task *it, switch (flags) { case BPF_TASK_ITER_ALL_THREADS: case BPF_TASK_ITER_ALL_PROCS: + break; case BPF_TASK_ITER_PROC_THREADS: + if (!task__nullable) + return -EINVAL; break; default: return -EINVAL; } if (flags == BPF_TASK_ITER_PROC_THREADS) - kit->task = task; + kit->task = task__nullable; else kit->task = &init_task; kit->pos = kit->task; diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index fcdf2382153a..e9bc5d4a25a1 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -10332,6 +10332,11 @@ static bool is_kfunc_arg_refcounted_kptr(const struct btf *btf, const struct btf return __kfunc_param_match_suffix(btf, arg, "__refcounted_kptr"); } +static bool is_kfunc_arg_nullable(const struct btf *btf, const struct btf_param *arg) +{ + return __kfunc_param_match_suffix(btf, arg, "__nullable"); +} + static bool is_kfunc_arg_scalar_with_name(const struct btf *btf, const struct btf_param *arg, const char *name) @@ -10474,6 +10479,7 @@ enum kfunc_ptr_arg_type { KF_ARG_PTR_TO_CALLBACK, KF_ARG_PTR_TO_RB_ROOT, KF_ARG_PTR_TO_RB_NODE, + KF_ARG_PTR_TO_NULL, }; enum special_kfunc_type { @@ -10630,6 +10636,8 @@ get_kfunc_ptr_arg_type(struct bpf_verifier_env *env, if (is_kfunc_arg_callback(env, meta->btf, &args[argno])) return KF_ARG_PTR_TO_CALLBACK; + if (is_kfunc_arg_nullable(meta->btf, &args[argno]) && register_is_null(reg)) + return KF_ARG_PTR_TO_NULL; if (argno + 1 < nargs && (is_kfunc_arg_mem_size(meta->btf, &args[argno + 1], ®s[regno + 1]) || @@ -11180,7 +11188,8 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ } if ((is_kfunc_trusted_args(meta) || is_kfunc_rcu(meta)) && - (register_is_null(reg) || type_may_be_null(reg->type))) { + (register_is_null(reg) || type_may_be_null(reg->type)) && + !is_kfunc_arg_nullable(meta->btf, &args[i])) { verbose(env, "Possibly NULL pointer passed to trusted arg%d\n", i); return -EACCES; } @@ -11205,6 +11214,8 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ return kf_arg_type; switch (kf_arg_type) { + case KF_ARG_PTR_TO_NULL: + continue; case KF_ARG_PTR_TO_ALLOC_BTF_ID: case KF_ARG_PTR_TO_BTF_ID: if (!is_kfunc_trusted_args(meta) && !is_kfunc_rcu(meta)) From patchwork Wed Oct 18 06:17:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuyi Zhou X-Patchwork-Id: 154707 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp4596039vqb; Tue, 17 Oct 2023 23:23:16 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHwW2BDXtkhpU5yKBoiXZvtbfC9RJDsXvNAfxOU1TwsdsMutPqhzTXXaF4Q4ZHF/QeiJHH9 X-Received: by 2002:a17:90b:4c8b:b0:27d:2762:2728 with SMTP id my11-20020a17090b4c8b00b0027d27622728mr4762377pjb.0.1697610196470; Tue, 17 Oct 2023 23:23:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697610196; cv=none; d=google.com; s=arc-20160816; b=PCcvH0F7FUw2i3GuNnuqRC7XTPBalV3Edmad+4eJOw9wsZ05E2uTORZSQgVac7FMZE O5FdWxmeEBXGS5pDAz3DSM5fPnOxlbBk3AsEjyNTJVZVvJ4IZ7tn6j1hQAJCfmErDCME AZ0rIY346JbEID0DYmawmTuXEmPfD9eREFOZ27KaUqc40DtPmqeYiuPufJQNmBBEqpiN k0za7g5M/O+ZanFzPPFNspsoeXTV5pwS+b+xP1oy0yHbYm7VVX7/L5WPvh9BWlJcRck9 N2zx6g81mTCQd9Pud/AgkVDep3lvZYjAUHsZ/4oCTRwUru+NxrKXShY41eyCO65nQ0dz fzog== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=bQoEay4rhaVzIBrBvVhg11HsCpeS6HANzS4lkGBCuZ4=; fh=DR7g1EcWKOXTEoooUPBSJXUaklSrDEYzv6YDdhz1CwE=; b=wc2y7GEKB20LvLDlnQDWLTCWvkBlsp1tRegnQdd6aTvGDPBrqtqoK5qpevunIBTTk0 NmNXuErN5IBuMOZPP4HE51nKjtvhDFvyS2+RkPjLUKAqERs3yvEcBtxdWv3wMn3JZrnj gH+9NYu+NtlkoqkivSbavZezeSnmJUCbyrx8w2CaCLd/EIRNSBtJSBCNSIuh5Sajj5m4 qyF5ximPDH4Y5kJ7FKcKzgEdkLiK3NfnBJtEyECOdMTK+YaMAjGZyiXRoNGYpQJKZxwZ QoUEbuyRV5W6c33wFx1x/FOzP1PuLgtrUNyyPk6l7J84U2y4UvGOwsnHjjNiuaCS/FxW 7V6w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=FwW88olg; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:1 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from morse.vger.email (morse.vger.email. [2620:137:e000::3:1]) by mx.google.com with ESMTPS id bk23-20020a17090b081700b0027d1aed872dsi826319pjb.95.2023.10.17.23.23.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Oct 2023 23:23:16 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:1 as permitted sender) client-ip=2620:137:e000::3:1; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=FwW88olg; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:1 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by morse.vger.email (Postfix) with ESMTP id 66FEF8078641; Tue, 17 Oct 2023 23:23:14 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at morse.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235175AbjJRGXA (ORCPT + 23 others); Wed, 18 Oct 2023 02:23:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47848 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234819AbjJRGWl (ORCPT ); Wed, 18 Oct 2023 02:22:41 -0400 Received: from mail-pl1-x62e.google.com (mail-pl1-x62e.google.com [IPv6:2607:f8b0:4864:20::62e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DA1B019A8 for ; Tue, 17 Oct 2023 23:18:19 -0700 (PDT) Received: by mail-pl1-x62e.google.com with SMTP id d9443c01a7336-1c9bca1d96cso44216315ad.3 for ; Tue, 17 Oct 2023 23:18:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1697609894; x=1698214694; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bQoEay4rhaVzIBrBvVhg11HsCpeS6HANzS4lkGBCuZ4=; b=FwW88olgJ5EWUU25m6KzuQ6R858GYT0Ra2cPUGTZ1aXWYsCvDvhXDm8o8rVYoKTPBO cL1bp9RR1NrR4Yrmr6bAhWa65Jw5Lm/dqivydvj4sIBy6HKRei2Axp/3sNBo3uMGw+Mf VXEnX7jM7x4BWA+QJhPZBKJYq3Dp8omZWV+EA4YGlBljDXex3jqHyCuAg+YcVHzViJ27 hYXUj2SuaXJztjePmIJsU8CBievUhtZj7a3dcz4iI34lXhAco5vobbmRXjZ81hjgPuls TgFV6adRs5cqatgR68QofM/sD5ajefX8qR6RPEVDo1Xxe/jABtwLhJF2dSEMjeB6aNPb 4g9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697609894; x=1698214694; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bQoEay4rhaVzIBrBvVhg11HsCpeS6HANzS4lkGBCuZ4=; b=J/RI20cokehsRudnRtKgD2sXP4pmdOP0kagFTFfOLW09a+aHn0Arbrwv4I0EZV65SA g6I/KFFeklXT/4OG/b/UDz9ZEYLXfufZXjIl6zfYMkaCOwaxI3PIfOPR0KPJsFdijcSO G5Y0S07JkRBi8mOM1hLjOlORxpfzDHynxs3ksb43k7BQ1QkR5FWiHGFMnpxt75hTga1W otJW3TafrGJe2gdFFYvvcDF+fKICAG4l240o9i3ORQvrKOnU9gkOE3xT6PwerecVHq41 7fJzIodICw3MbEqFl7rF9u7c2xbC2HGoG9ji/ERIDaMvqfB4aaCet/yqVeSy52V0r3XO qjEw== X-Gm-Message-State: AOJu0YwlgB0S0jNRRPIv+lGlbNptdmot66b88whPRO4vGgSDoRyMPoCA BrIOsiIlaAP0BUtx+ZpdVp7xNg== X-Received: by 2002:a17:902:ecc9:b0:1ca:1af1:681d with SMTP id a9-20020a170902ecc900b001ca1af1681dmr4504438plh.67.1697609894426; Tue, 17 Oct 2023 23:18:14 -0700 (PDT) Received: from n37-019-243.byted.org ([180.184.103.200]) by smtp.gmail.com with ESMTPSA id ix13-20020a170902f80d00b001c61acd5bd2sm2659116plb.112.2023.10.17.23.18.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Oct 2023 23:18:14 -0700 (PDT) From: Chuyi Zhou To: bpf@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@kernel.org, tj@kernel.org, linux-kernel@vger.kernel.org, Chuyi Zhou Subject: [RESEND PATCH bpf-next v6 7/8] selftests/bpf: rename bpf_iter_task.c to bpf_iter_tasks.c Date: Wed, 18 Oct 2023 14:17:45 +0800 Message-Id: <20231018061746.111364-8-zhouchuyi@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20231018061746.111364-1-zhouchuyi@bytedance.com> References: <20231018061746.111364-1-zhouchuyi@bytedance.com> MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on morse.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (morse.vger.email [0.0.0.0]); Tue, 17 Oct 2023 23:23:14 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780073309015132323 X-GMAIL-MSGID: 1780073309015132323 The newly-added struct bpf_iter_task has a name collision with a selftest for the seq_file task iter's bpf skel, so the selftests/bpf/progs file is renamed in order to avoid the collision. Signed-off-by: Chuyi Zhou Acked-by: Andrii Nakryiko --- .../selftests/bpf/prog_tests/bpf_iter.c | 18 +++++++++--------- .../{bpf_iter_task.c => bpf_iter_tasks.c} | 0 2 files changed, 9 insertions(+), 9 deletions(-) rename tools/testing/selftests/bpf/progs/{bpf_iter_task.c => bpf_iter_tasks.c} (100%) diff --git a/tools/testing/selftests/bpf/prog_tests/bpf_iter.c b/tools/testing/selftests/bpf/prog_tests/bpf_iter.c index 41aba139b20b..e3498f607b49 100644 --- a/tools/testing/selftests/bpf/prog_tests/bpf_iter.c +++ b/tools/testing/selftests/bpf/prog_tests/bpf_iter.c @@ -7,7 +7,7 @@ #include "bpf_iter_ipv6_route.skel.h" #include "bpf_iter_netlink.skel.h" #include "bpf_iter_bpf_map.skel.h" -#include "bpf_iter_task.skel.h" +#include "bpf_iter_tasks.skel.h" #include "bpf_iter_task_stack.skel.h" #include "bpf_iter_task_file.skel.h" #include "bpf_iter_task_vmas.skel.h" @@ -215,12 +215,12 @@ static void *do_nothing_wait(void *arg) static void test_task_common_nocheck(struct bpf_iter_attach_opts *opts, int *num_unknown, int *num_known) { - struct bpf_iter_task *skel; + struct bpf_iter_tasks *skel; pthread_t thread_id; void *ret; - skel = bpf_iter_task__open_and_load(); - if (!ASSERT_OK_PTR(skel, "bpf_iter_task__open_and_load")) + skel = bpf_iter_tasks__open_and_load(); + if (!ASSERT_OK_PTR(skel, "bpf_iter_tasks__open_and_load")) return; ASSERT_OK(pthread_mutex_lock(&do_nothing_mutex), "pthread_mutex_lock"); @@ -239,7 +239,7 @@ static void test_task_common_nocheck(struct bpf_iter_attach_opts *opts, ASSERT_FALSE(pthread_join(thread_id, &ret) || ret != NULL, "pthread_join"); - bpf_iter_task__destroy(skel); + bpf_iter_tasks__destroy(skel); } static void test_task_common(struct bpf_iter_attach_opts *opts, int num_unknown, int num_known) @@ -307,10 +307,10 @@ static void test_task_pidfd(void) static void test_task_sleepable(void) { - struct bpf_iter_task *skel; + struct bpf_iter_tasks *skel; - skel = bpf_iter_task__open_and_load(); - if (!ASSERT_OK_PTR(skel, "bpf_iter_task__open_and_load")) + skel = bpf_iter_tasks__open_and_load(); + if (!ASSERT_OK_PTR(skel, "bpf_iter_tasks__open_and_load")) return; do_dummy_read(skel->progs.dump_task_sleepable); @@ -320,7 +320,7 @@ static void test_task_sleepable(void) ASSERT_GT(skel->bss->num_success_copy_from_user_task, 0, "num_success_copy_from_user_task"); - bpf_iter_task__destroy(skel); + bpf_iter_tasks__destroy(skel); } static void test_task_stack(void) diff --git a/tools/testing/selftests/bpf/progs/bpf_iter_task.c b/tools/testing/selftests/bpf/progs/bpf_iter_tasks.c similarity index 100% rename from tools/testing/selftests/bpf/progs/bpf_iter_task.c rename to tools/testing/selftests/bpf/progs/bpf_iter_tasks.c From patchwork Wed Oct 18 06:17:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuyi Zhou X-Patchwork-Id: 154703 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp4595470vqb; Tue, 17 Oct 2023 23:21:38 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFstx6B6pViNhrE+HhzGfI04iqI2rXpJO8uuNQN/5XqySkv4gqqDj8dMsSBhxhuXluAR87p X-Received: by 2002:a05:6808:2e90:b0:3ad:f525:52d5 with SMTP id gt16-20020a0568082e9000b003adf52552d5mr5486921oib.4.1697610098179; Tue, 17 Oct 2023 23:21:38 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697610098; cv=none; d=google.com; s=arc-20160816; b=s7lZVaFllRbbzPqqggyGnDjJFCrVTLqWERZSHHdSXojAv8+SyB9CNHLK1O8V7a5Mhh 7/0m5rX5ewwpbcEKdnghRhkKq1xWhbbeD+FC5aUg2re5iwS3P40R3+yMDs6ajHO9uhAn 07j5h/WnRzeM4kjVB+p25NFbIM5LzsWgYaNwni5RU32BPyVDsyriiABRzDqBKDbq4GhU gT4q+WsQhHcm+lxO/nz3T0FwkMojfIj5B9F2EOGZQ5o3L9s+PqJlE2Ygb6ZtBNgp3Lm1 lj8OTId7Tfyt8EhZDfHdcsz3hVGO7CUJUfWbi9Dy23bCqy95K0kKUatQ0Q3+uB0aFZEz USMQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=U8WV2XVvAQuJkK59o/8tW8GBGlk0HThG8biP3YZXvQU=; fh=DR7g1EcWKOXTEoooUPBSJXUaklSrDEYzv6YDdhz1CwE=; b=dQTXIvDeWzbdJHvySzrs9KWcTqEnKZ81iU/yFZ94oOaz2X/puMYM7cGkzYBXv/2EPe JqYf0wwZBgxGoNf7mug04+wQISOEvfe6b6NG+7Sb0c/eJE5wk8EVXCe/3MVSlhcSUV0l Kwu+DjvTOvrRN+DVWNKBMUUwx0BAX4CMZNgg70Pq2ZIy0A5j5WKPPJqPIy+WPpC9nd6i QZw/3Wyd0wwgQcUm46k7N4QAo30v+XZu5oern8xotLfJQy7SxHjsaXBsnJlgcqymhzXG zWCu12gZWGqU3RA3U1mym/gqFQaPMAaArrQUEB4qWgXjLMGZGy6UqSsgYPRmB85rWWG7 1BCw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b="go/gNRkW"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from lipwig.vger.email (lipwig.vger.email. [23.128.96.33]) by mx.google.com with ESMTPS id j6-20020a633c06000000b005890aa054fcsi1373105pga.400.2023.10.17.23.21.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Oct 2023 23:21:38 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) client-ip=23.128.96.33; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b="go/gNRkW"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id 72CF9814970B; Tue, 17 Oct 2023 23:21:34 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235102AbjJRGVE (ORCPT + 23 others); Wed, 18 Oct 2023 02:21:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55240 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229674AbjJRGU6 (ORCPT ); Wed, 18 Oct 2023 02:20:58 -0400 Received: from mail-pl1-x634.google.com (mail-pl1-x634.google.com [IPv6:2607:f8b0:4864:20::634]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 588A819B3 for ; Tue, 17 Oct 2023 23:18:20 -0700 (PDT) Received: by mail-pl1-x634.google.com with SMTP id d9443c01a7336-1c9e072472bso41269015ad.2 for ; Tue, 17 Oct 2023 23:18:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1697609897; x=1698214697; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=U8WV2XVvAQuJkK59o/8tW8GBGlk0HThG8biP3YZXvQU=; b=go/gNRkWlhHnC98EH3YpacP/SzqFJYjjIcR7IYtChqHS9L4DX0P7rR+L7BKli7ikYt QEZYYswE6H8n7yNSd8JRsbpsjeME2PtDKGHY1s33h9WtmzThwOxj0FCKLJNZzqrxOFkZ 3yeXRsqoiGz1B4tL8+EwZI/nNM2d/ScohstdCrgWmMC3XpKf/bqd5XKQT0psBroooFcS AG9YUm8/3SXWBGYY2t5/y+OZnh8ie2YTRue8j9P+w0lfbUm9gIv0zhrPrN6zmWwk4dYh Pk2Wx8bNaI0AMP9cSoDhDWTctkd4yU6HR1HesjQknGQ6Hy9lBiCHkSjOxOCQ92B6168C MGpQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697609897; x=1698214697; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=U8WV2XVvAQuJkK59o/8tW8GBGlk0HThG8biP3YZXvQU=; b=nb+4xncvJAB0SAnsUfvEcS407fGX/9KO0V0qZF8aI33uIg1SIKqDqsEmn4BzenHava 8ZGvBW+GxtpAF8VcpvzrsDukxvVMFr3zVsGbUTagEQ/LbIiIlkhFPapmFOrA95rZu3lo /FaS3sYYJW7TCbauRESz96ct1xn+BYmqb0zBcj1JVL34tpt9Hob2mswgAUp6blGzecdn +mhPBi2kgYnTvlFJ5FSfF1WwqkKiLbThBZNswKyQeq4Go7eNOyxjjPUlRq4c8W8GTnWm IPNnfSFy+md9ZlQ7/JkNPrOHTxcZlVYNf1M76be4ocZYWsPIMW9x9YItfvXEq6OVSa+s kSCg== X-Gm-Message-State: AOJu0YxoYWHAeiLRrC4v1rVvw63TQPVWaejOmkXFRwmv9EUAVSpbEdka /qz/zO+yFakHdNT76A+5MB1LpQ== X-Received: by 2002:a17:902:c40a:b0:1c3:a4f2:7c92 with SMTP id k10-20020a170902c40a00b001c3a4f27c92mr5260298plk.65.1697609897359; Tue, 17 Oct 2023 23:18:17 -0700 (PDT) Received: from n37-019-243.byted.org ([180.184.103.200]) by smtp.gmail.com with ESMTPSA id ix13-20020a170902f80d00b001c61acd5bd2sm2659116plb.112.2023.10.17.23.18.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Oct 2023 23:18:17 -0700 (PDT) From: Chuyi Zhou To: bpf@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@kernel.org, tj@kernel.org, linux-kernel@vger.kernel.org, Chuyi Zhou Subject: [RESEND PATCH bpf-next v6 8/8] selftests/bpf: Add tests for open-coded task and css iter Date: Wed, 18 Oct 2023 14:17:46 +0800 Message-Id: <20231018061746.111364-9-zhouchuyi@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20231018061746.111364-1-zhouchuyi@bytedance.com> References: <20231018061746.111364-1-zhouchuyi@bytedance.com> MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Tue, 17 Oct 2023 23:21:34 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780073205852344541 X-GMAIL-MSGID: 1780073205852344541 This patch adds 4 subtests to demonstrate these patterns and validating correctness. subtest1: 1) We use task_iter to iterate all process in the system and search for the current process with a given pid. 2) We create some threads in current process context, and use BPF_TASK_ITER_PROC_THREADS to iterate all threads of current process. As expected, we would find all the threads of current process. 3) We create some threads and use BPF_TASK_ITER_ALL_THREADS to iterate all threads in the system. As expected, we would find all the threads which was created. subtest2: We create a cgroup and add the current task to the cgroup. In the BPF program, we would use bpf_for_each(css_task, task, css) to iterate all tasks under the cgroup. As expected, we would find the current process. subtest3: 1) We create a cgroup tree. In the BPF program, we use bpf_for_each(css, pos, root, XXX) to iterate all descendant under the root with pre and post order. As expected, we would find all descendant and the last iterating cgroup in post-order is root cgroup, the first iterating cgroup in pre-order is root cgroup. 2) We wse BPF_CGROUP_ITER_ANCESTORS_UP to traverse the cgroup tree starting from leaf and root separately, and record the height. The diff of the hights would be the total tree-high - 1. subtest4: Add some failure testcase when using css_task, task and css iters, e.g, unlock when using task-iters to iterate tasks. Signed-off-by: Chuyi Zhou --- .../testing/selftests/bpf/prog_tests/iters.c | 150 ++++++++++++++++++ tools/testing/selftests/bpf/progs/iters_css.c | 72 +++++++++ .../selftests/bpf/progs/iters_css_task.c | 47 ++++++ .../testing/selftests/bpf/progs/iters_task.c | 41 +++++ .../selftests/bpf/progs/iters_task_failure.c | 105 ++++++++++++ 5 files changed, 415 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/iters_css.c create mode 100644 tools/testing/selftests/bpf/progs/iters_css_task.c create mode 100644 tools/testing/selftests/bpf/progs/iters_task.c create mode 100644 tools/testing/selftests/bpf/progs/iters_task_failure.c diff --git a/tools/testing/selftests/bpf/prog_tests/iters.c b/tools/testing/selftests/bpf/prog_tests/iters.c index b696873c5455..c2425791c923 100644 --- a/tools/testing/selftests/bpf/prog_tests/iters.c +++ b/tools/testing/selftests/bpf/prog_tests/iters.c @@ -1,7 +1,14 @@ // SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2023 Meta Platforms, Inc. and affiliates. */ +#include +#include +#include +#include +#include +#include #include +#include "cgroup_helpers.h" #include "iters.skel.h" #include "iters_state_safety.skel.h" @@ -9,6 +16,10 @@ #include "iters_num.skel.h" #include "iters_testmod_seq.skel.h" #include "iters_task_vma.skel.h" +#include "iters_task.skel.h" +#include "iters_css_task.skel.h" +#include "iters_css.skel.h" +#include "iters_task_failure.skel.h" static void subtest_num_iters(void) { @@ -146,6 +157,138 @@ static void subtest_task_vma_iters(void) iters_task_vma__destroy(skel); } +static pthread_mutex_t do_nothing_mutex; + +static void *do_nothing_wait(void *arg) +{ + pthread_mutex_lock(&do_nothing_mutex); + pthread_mutex_unlock(&do_nothing_mutex); + + pthread_exit(arg); +} + +#define thread_num 2 + +static void subtest_task_iters(void) +{ + struct iters_task *skel = NULL; + pthread_t thread_ids[thread_num]; + void *ret; + int err; + + skel = iters_task__open_and_load(); + if (!ASSERT_OK_PTR(skel, "open_and_load")) + goto cleanup; + skel->bss->target_pid = getpid(); + err = iters_task__attach(skel); + if (!ASSERT_OK(err, "iters_task__attach")) + goto cleanup; + pthread_mutex_lock(&do_nothing_mutex); + for (int i = 0; i < thread_num; i++) + ASSERT_OK(pthread_create(&thread_ids[i], NULL, &do_nothing_wait, NULL), + "pthread_create"); + + syscall(SYS_getpgid); + iters_task__detach(skel); + ASSERT_EQ(skel->bss->procs_cnt, 1, "procs_cnt"); + ASSERT_EQ(skel->bss->threads_cnt, thread_num + 1, "threads_cnt"); + ASSERT_EQ(skel->bss->proc_threads_cnt, thread_num + 1, "proc_threads_cnt"); + pthread_mutex_unlock(&do_nothing_mutex); + for (int i = 0; i < thread_num; i++) + ASSERT_OK(pthread_join(thread_ids[i], &ret), "pthread_join"); +cleanup: + iters_task__destroy(skel); +} + +extern int stack_mprotect(void); + +static void subtest_css_task_iters(void) +{ + struct iters_css_task *skel = NULL; + int err, cg_fd, cg_id; + const char *cgrp_path = "/cg1"; + + err = setup_cgroup_environment(); + if (!ASSERT_OK(err, "setup_cgroup_environment")) + goto cleanup; + cg_fd = create_and_get_cgroup(cgrp_path); + if (!ASSERT_GE(cg_fd, 0, "create_and_get_cgroup")) + goto cleanup; + cg_id = get_cgroup_id(cgrp_path); + err = join_cgroup(cgrp_path); + if (!ASSERT_OK(err, "join_cgroup")) + goto cleanup; + + skel = iters_css_task__open_and_load(); + if (!ASSERT_OK_PTR(skel, "open_and_load")) + goto cleanup; + + skel->bss->target_pid = getpid(); + skel->bss->cg_id = cg_id; + err = iters_css_task__attach(skel); + if (!ASSERT_OK(err, "iters_task__attach")) + goto cleanup; + err = stack_mprotect(); + if (!ASSERT_EQ(err, -1, "stack_mprotect") || + !ASSERT_EQ(errno, EPERM, "stack_mprotect")) + goto cleanup; + iters_css_task__detach(skel); + ASSERT_EQ(skel->bss->css_task_cnt, 1, "css_task_cnt"); + +cleanup: + cleanup_cgroup_environment(); + iters_css_task__destroy(skel); +} + +static void subtest_css_iters(void) +{ + struct iters_css *skel = NULL; + struct { + const char *path; + int fd; + } cgs[] = { + { "/cg1" }, + { "/cg1/cg2" }, + { "/cg1/cg2/cg3" }, + { "/cg1/cg2/cg3/cg4" }, + }; + int err, cg_nr = ARRAY_SIZE(cgs); + int i; + + err = setup_cgroup_environment(); + if (!ASSERT_OK(err, "setup_cgroup_environment")) + goto cleanup; + for (i = 0; i < cg_nr; i++) { + cgs[i].fd = create_and_get_cgroup(cgs[i].path); + if (!ASSERT_GE(cgs[i].fd, 0, "create_and_get_cgroup")) + goto cleanup; + } + + skel = iters_css__open_and_load(); + if (!ASSERT_OK_PTR(skel, "open_and_load")) + goto cleanup; + + skel->bss->target_pid = getpid(); + skel->bss->root_cg_id = get_cgroup_id(cgs[0].path); + skel->bss->leaf_cg_id = get_cgroup_id(cgs[cg_nr - 1].path); + err = iters_css__attach(skel); + + if (!ASSERT_OK(err, "iters_task__attach")) + goto cleanup; + + syscall(SYS_getpgid); + ASSERT_EQ(skel->bss->pre_order_cnt, cg_nr, "pre_order_cnt"); + ASSERT_EQ(skel->bss->first_cg_id, get_cgroup_id(cgs[0].path), "first_cg_id"); + + ASSERT_EQ(skel->bss->post_order_cnt, cg_nr, "post_order_cnt"); + ASSERT_EQ(skel->bss->last_cg_id, get_cgroup_id(cgs[0].path), "last_cg_id"); + ASSERT_EQ(skel->bss->tree_high, cg_nr - 1, "tree_high"); + iters_css__detach(skel); +cleanup: + cleanup_cgroup_environment(); + iters_css__destroy(skel); +} + void test_iters(void) { RUN_TESTS(iters_state_safety); @@ -161,4 +304,11 @@ void test_iters(void) subtest_testmod_seq_iters(); if (test__start_subtest("task_vma")) subtest_task_vma_iters(); + if (test__start_subtest("task")) + subtest_task_iters(); + if (test__start_subtest("css_task")) + subtest_css_task_iters(); + if (test__start_subtest("css")) + subtest_css_iters(); + RUN_TESTS(iters_task_failure); } diff --git a/tools/testing/selftests/bpf/progs/iters_css.c b/tools/testing/selftests/bpf/progs/iters_css.c new file mode 100644 index 000000000000..ec1f6c2f590b --- /dev/null +++ b/tools/testing/selftests/bpf/progs/iters_css.c @@ -0,0 +1,72 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (C) 2023 Chuyi Zhou */ + +#include "vmlinux.h" +#include +#include +#include "bpf_misc.h" +#include "bpf_experimental.h" + +char _license[] SEC("license") = "GPL"; + +pid_t target_pid; +u64 root_cg_id, leaf_cg_id; +u64 first_cg_id, last_cg_id; + +int pre_order_cnt, post_order_cnt, tree_high; + +struct cgroup *bpf_cgroup_from_id(u64 cgid) __ksym; +void bpf_cgroup_release(struct cgroup *p) __ksym; +void bpf_rcu_read_lock(void) __ksym; +void bpf_rcu_read_unlock(void) __ksym; + +SEC("fentry.s/" SYS_PREFIX "sys_getpgid") +int iter_css_for_each(const void *ctx) +{ + struct task_struct *cur_task = bpf_get_current_task_btf(); + struct cgroup_subsys_state *root_css, *leaf_css, *pos; + struct cgroup *root_cgrp, *leaf_cgrp, *cur_cgrp; + + if (cur_task->pid != target_pid) + return 0; + + root_cgrp = bpf_cgroup_from_id(root_cg_id); + + if (!root_cgrp) + return 0; + + leaf_cgrp = bpf_cgroup_from_id(leaf_cg_id); + + if (!leaf_cgrp) { + bpf_cgroup_release(root_cgrp); + return 0; + } + root_css = &root_cgrp->self; + leaf_css = &leaf_cgrp->self; + pre_order_cnt = post_order_cnt = tree_high = 0; + first_cg_id = last_cg_id = 0; + + bpf_rcu_read_lock(); + bpf_for_each(css, pos, root_css, BPF_CGROUP_ITER_DESCENDANTS_POST) { + cur_cgrp = pos->cgroup; + post_order_cnt++; + last_cg_id = cur_cgrp->kn->id; + } + + bpf_for_each(css, pos, root_css, BPF_CGROUP_ITER_DESCENDANTS_PRE) { + cur_cgrp = pos->cgroup; + pre_order_cnt++; + if (!first_cg_id) + first_cg_id = cur_cgrp->kn->id; + } + + bpf_for_each(css, pos, leaf_css, BPF_CGROUP_ITER_ANCESTORS_UP) + tree_high++; + + bpf_for_each(css, pos, root_css, BPF_CGROUP_ITER_ANCESTORS_UP) + tree_high--; + bpf_rcu_read_unlock(); + bpf_cgroup_release(root_cgrp); + bpf_cgroup_release(leaf_cgrp); + return 0; +} diff --git a/tools/testing/selftests/bpf/progs/iters_css_task.c b/tools/testing/selftests/bpf/progs/iters_css_task.c new file mode 100644 index 000000000000..5089ce384a1c --- /dev/null +++ b/tools/testing/selftests/bpf/progs/iters_css_task.c @@ -0,0 +1,47 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (C) 2023 Chuyi Zhou */ + +#include "vmlinux.h" +#include +#include +#include +#include "bpf_misc.h" +#include "bpf_experimental.h" + +char _license[] SEC("license") = "GPL"; + +struct cgroup *bpf_cgroup_from_id(u64 cgid) __ksym; +void bpf_cgroup_release(struct cgroup *p) __ksym; + +pid_t target_pid; +int css_task_cnt; +u64 cg_id; + +SEC("lsm/file_mprotect") +int BPF_PROG(iter_css_task_for_each, struct vm_area_struct *vma, + unsigned long reqprot, unsigned long prot, int ret) +{ + struct task_struct *cur_task = bpf_get_current_task_btf(); + struct cgroup_subsys_state *css; + struct task_struct *task; + struct cgroup *cgrp; + + if (cur_task->pid != target_pid) + return ret; + + cgrp = bpf_cgroup_from_id(cg_id); + + if (!cgrp) + return -EPERM; + + css = &cgrp->self; + css_task_cnt = 0; + + bpf_for_each(css_task, task, css, CSS_TASK_ITER_PROCS) + if (task->pid == target_pid) + css_task_cnt++; + + bpf_cgroup_release(cgrp); + + return -EPERM; +} diff --git a/tools/testing/selftests/bpf/progs/iters_task.c b/tools/testing/selftests/bpf/progs/iters_task.c new file mode 100644 index 000000000000..c9b4055cd410 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/iters_task.c @@ -0,0 +1,41 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (C) 2023 Chuyi Zhou */ + +#include "vmlinux.h" +#include +#include +#include "bpf_misc.h" +#include "bpf_experimental.h" + +char _license[] SEC("license") = "GPL"; + +pid_t target_pid; +int procs_cnt, threads_cnt, proc_threads_cnt; + +void bpf_rcu_read_lock(void) __ksym; +void bpf_rcu_read_unlock(void) __ksym; + +SEC("fentry.s/" SYS_PREFIX "sys_getpgid") +int iter_task_for_each_sleep(void *ctx) +{ + struct task_struct *cur_task = bpf_get_current_task_btf(); + struct task_struct *pos; + + if (cur_task->pid != target_pid) + return 0; + procs_cnt = threads_cnt = proc_threads_cnt = 0; + + bpf_rcu_read_lock(); + bpf_for_each(task, pos, NULL, BPF_TASK_ITER_ALL_PROCS) + if (pos->pid == target_pid) + procs_cnt++; + + bpf_for_each(task, pos, cur_task, BPF_TASK_ITER_PROC_THREADS) + proc_threads_cnt++; + + bpf_for_each(task, pos, NULL, BPF_TASK_ITER_ALL_THREADS) + if (pos->tgid == target_pid) + threads_cnt++; + bpf_rcu_read_unlock(); + return 0; +} diff --git a/tools/testing/selftests/bpf/progs/iters_task_failure.c b/tools/testing/selftests/bpf/progs/iters_task_failure.c new file mode 100644 index 000000000000..c3bf96a67dba --- /dev/null +++ b/tools/testing/selftests/bpf/progs/iters_task_failure.c @@ -0,0 +1,105 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (C) 2023 Chuyi Zhou */ + +#include "vmlinux.h" +#include +#include +#include "bpf_misc.h" +#include "bpf_experimental.h" + +char _license[] SEC("license") = "GPL"; + +struct cgroup *bpf_cgroup_from_id(u64 cgid) __ksym; +void bpf_cgroup_release(struct cgroup *p) __ksym; +void bpf_rcu_read_lock(void) __ksym; +void bpf_rcu_read_unlock(void) __ksym; + +SEC("?fentry.s/" SYS_PREFIX "sys_getpgid") +__failure __msg("expected an RCU CS when using bpf_iter_task_next") +int BPF_PROG(iter_tasks_without_lock) +{ + struct task_struct *pos; + + bpf_for_each(task, pos, NULL, BPF_TASK_ITER_ALL_PROCS) { + + } + return 0; +} + +SEC("?fentry.s/" SYS_PREFIX "sys_getpgid") +__failure __msg("expected an RCU CS when using bpf_iter_css_next") +int BPF_PROG(iter_css_without_lock) +{ + u64 cg_id = bpf_get_current_cgroup_id(); + struct cgroup *cgrp = bpf_cgroup_from_id(cg_id); + struct cgroup_subsys_state *root_css, *pos; + + if (!cgrp) + return 0; + root_css = &cgrp->self; + + bpf_for_each(css, pos, root_css, BPF_CGROUP_ITER_DESCENDANTS_POST) { + + } + bpf_cgroup_release(cgrp); + return 0; +} + +SEC("?fentry.s/" SYS_PREFIX "sys_getpgid") +__failure __msg("expected an RCU CS when using bpf_iter_task_next") +int BPF_PROG(iter_tasks_lock_and_unlock) +{ + struct task_struct *pos; + + bpf_rcu_read_lock(); + bpf_for_each(task, pos, NULL, BPF_TASK_ITER_ALL_PROCS) { + bpf_rcu_read_unlock(); + + bpf_rcu_read_lock(); + } + bpf_rcu_read_unlock(); + return 0; +} + +SEC("?fentry.s/" SYS_PREFIX "sys_getpgid") +__failure __msg("expected an RCU CS when using bpf_iter_css_next") +int BPF_PROG(iter_css_lock_and_unlock) +{ + u64 cg_id = bpf_get_current_cgroup_id(); + struct cgroup *cgrp = bpf_cgroup_from_id(cg_id); + struct cgroup_subsys_state *root_css, *pos; + + if (!cgrp) + return 0; + root_css = &cgrp->self; + + bpf_rcu_read_lock(); + bpf_for_each(css, pos, root_css, BPF_CGROUP_ITER_DESCENDANTS_POST) { + bpf_rcu_read_unlock(); + + bpf_rcu_read_lock(); + } + bpf_rcu_read_unlock(); + bpf_cgroup_release(cgrp); + return 0; +} + +SEC("?fentry.s/" SYS_PREFIX "sys_getpgid") +__failure __msg("css_task_iter is only allowed in bpf_lsm and bpf iter-s") +int BPF_PROG(iter_css_task_for_each) +{ + u64 cg_id = bpf_get_current_cgroup_id(); + struct cgroup *cgrp = bpf_cgroup_from_id(cg_id); + struct cgroup_subsys_state *css; + struct task_struct *task; + + if (cgrp == NULL) + return 0; + css = &cgrp->self; + + bpf_for_each(css_task, task, css, CSS_TASK_ITER_PROCS) { + + } + bpf_cgroup_release(cgrp); + return 0; +}