From patchwork Wed Oct 11 12:08:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuyi Zhou X-Patchwork-Id: 151341 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp490569vqb; Wed, 11 Oct 2023 05:10:43 -0700 (PDT) X-Google-Smtp-Source: AGHT+IG7/Odr8EjFye2SFTSAD/kQfkcuiGhKo8Z4FI6eEEAolAsMp+43+rnTnEcGECEk5NuelOwR X-Received: by 2002:a17:90b:4f4d:b0:27d:9b6:ec1b with SMTP id pj13-20020a17090b4f4d00b0027d09b6ec1bmr956161pjb.4.1697026242885; Wed, 11 Oct 2023 05:10:42 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697026242; cv=none; d=google.com; s=arc-20160816; b=HitApsfyK5MmWsD3pMhreq1MJNbkLYcqSIvDCWoqFxkENZhG9phyeApbFcxVABR8Ef 2nbdmQe3kOryuugbsHez+L/5KtsKBHNHauV7EoJEuLuy+hL1s7mmS7lwofzsjl96/nQm 5HcaJNl/dBMudSfGaaG/3VrnIdyad87Y/xt0wznOUmubnsrK0DDCNxtpb3h48CzzpN1H Jp7mFWGPm0MfQCIxd/+12lHftQqhd9FaIZDCeK612tRGFLVpQoD9pFg3X5X2pZaBplE9 GPLFq5iqwv0b5r41BRB4e7ZAKTBdZVNVyVIm4AiQrhSQzteYJRrl5FDMM0eVICNTX5bX u0nA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=FNW2oYGtelVvgczyk2A13Ij/ocoUNYFTCyV6b+P6gh8=; fh=DR7g1EcWKOXTEoooUPBSJXUaklSrDEYzv6YDdhz1CwE=; b=vfJyyS3mKTtS1fMoP45FFR52YBTLdaPCNfIlYMXimohY22Uss4na32ArKUgpDAPG4K xC77goQrI0ZIWadQ0FadqjiiJKLL731ngC6kQA5Icm50giQmpENZIwnxc9lm6LqHrN64 X62cYtE9xIPClwVGXRZL+WRH5pk+1CgveCC42dUagZRdsS7BI7fn4jbyDsBo7LH9i8jR pKaJwrnZQmyVD2XzL+i5OvE8QUl//8/iUKOvveVl0IPayQ5f6qKYkxflvmKA3f/QL4ST 5Zw3UxOQG646G3HHTRS0H92fHdjLIlfzA9FRY/ixfrPIYj/W3haC9nlouG6ToSv5dRMF GFLA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=Mmo0Lnn+; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from agentk.vger.email (agentk.vger.email. [23.128.96.32]) by mx.google.com with ESMTPS id kz12-20020a170902f9cc00b001c62d93585csi13470967plb.611.2023.10.11.05.10.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Oct 2023 05:10:42 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) client-ip=23.128.96.32; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=Mmo0Lnn+; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id 4028D809027E; Wed, 11 Oct 2023 05:10:14 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346781AbjJKMJy (ORCPT + 18 others); Wed, 11 Oct 2023 08:09:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56920 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346720AbjJKMJV (ORCPT ); Wed, 11 Oct 2023 08:09:21 -0400 Received: from mail-pf1-x431.google.com (mail-pf1-x431.google.com [IPv6:2607:f8b0:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 34260E5 for ; Wed, 11 Oct 2023 05:09:18 -0700 (PDT) Received: by mail-pf1-x431.google.com with SMTP id d2e1a72fcca58-68fb85afef4so5212654b3a.1 for ; Wed, 11 Oct 2023 05:09:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1697026158; x=1697630958; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FNW2oYGtelVvgczyk2A13Ij/ocoUNYFTCyV6b+P6gh8=; b=Mmo0Lnn+6H8KnO/eFtAmfbbl3hOGYqf/dJPe2Gl7jKVZA6i5IUgfYQmNNLIddHocA+ khn0amvjF094s7IG8TQIcf0740yn5BdlfLgsg+7Jdf0Y6u5Ch/xYr7wdfLd4UULrtYJO VGwe4GAU0n8qKA9iiqRGZFi8Dk5yEFpg54ZAU6s1HJ8y8dRg8yRWhrvk1SpyOfy++Z51 WCNs6OJGScGEa/qwzJJZiEjsbGHVgXOBtzqGjqumoSwQcaJA0hZ/vD/Pl9XSNl9Ke/pS 3YyynfVVqBpQ9rWrZuyp5KdApZQbZNHCU4c0FspeyoSMJ2o7kaJfsVx4c2s8RRDKB/kw YERw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697026158; x=1697630958; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FNW2oYGtelVvgczyk2A13Ij/ocoUNYFTCyV6b+P6gh8=; b=Ts7UQnRpyOVaCXHiFVuJangKHvQbNfWmYJU+CSEEMgThOsBspniZPv6w1yRzPl3wWu uB4w+hbEXCzgu2RSHhpK++lm7vot+peXsCN+jvsIb7rmifgDUgaQryU0ggYxjRkyb6Ee SdFiF6UWUofEBtKbDUB2UAf3l+vGEPNv/ybTpQbdbVLXYRO7ZGg4iEzpTL/6+2n9D+HL 1ppjFgJIDpg7OUzvQciua6x8tXFoWDHdtKr1Utae6OapP5V/c1Nm5z8+A0IT0T/91KoB aD6bSRKjAB+CAxd0WgOO6P9WdyfVPaWXzpU+E9ZKcxc4nfj1DtRvXTlciSvsgAFZYEg2 kZjA== X-Gm-Message-State: AOJu0Ywg2mA16wwcsjJCG70JtiZG1yvf6J+WiIa6EeMBx+8wUo+4Fa1K RNC7Md23amWp7qIKtLzaMb9Yyw== X-Received: by 2002:a05:6a00:4217:b0:697:4901:3d4d with SMTP id cd23-20020a056a00421700b0069749013d4dmr17449212pfb.5.1697026157840; Wed, 11 Oct 2023 05:09:17 -0700 (PDT) Received: from n37-019-243.byted.org ([180.184.51.142]) by smtp.gmail.com with ESMTPSA id u13-20020a62ed0d000000b006930db1e6cfsm9962769pfh.62.2023.10.11.05.09.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Oct 2023 05:09:17 -0700 (PDT) From: Chuyi Zhou To: bpf@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@kernel.org, tj@kernel.org, linux-kernel@vger.kernel.org, Chuyi Zhou Subject: [PATCH bpf-next v5 1/8] cgroup: Prepare for using css_task_iter_*() in BPF Date: Wed, 11 Oct 2023 20:08:50 +0800 Message-Id: <20231011120857.251943-2-zhouchuyi@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20231011120857.251943-1-zhouchuyi@bytedance.com> References: <20231011120857.251943-1-zhouchuyi@bytedance.com> MIME-Version: 1.0 X-Spam-Status: No, score=2.7 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, RCVD_IN_SBL_CSS,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Wed, 11 Oct 2023 05:10:14 -0700 (PDT) X-Spam-Level: ** X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779460989366121517 X-GMAIL-MSGID: 1779460989366121517 This patch makes some preparations for using css_task_iter_*() in BPF Program. 1. Flags CSS_TASK_ITER_* are #define-s and it's not easy for bpf prog to use them. Convert them to enum so bpf prog can take them from vmlinux.h. 2. In the next patch we will add css_task_iter_*() in common kfuncs which is not safe. Since css_task_iter_*() does spin_unlock_irq() which might screw up irq flags depending on the context where bpf prog is running. So we should use irqsave/irqrestore here and the switching is harmless. Suggested-by: Alexei Starovoitov Signed-off-by: Chuyi Zhou Acked-by: Tejun Heo --- include/linux/cgroup.h | 12 +++++------- kernel/cgroup/cgroup.c | 18 ++++++++++++------ 2 files changed, 17 insertions(+), 13 deletions(-) diff --git a/include/linux/cgroup.h b/include/linux/cgroup.h index b307013b9c6c..0ef0af66080e 100644 --- a/include/linux/cgroup.h +++ b/include/linux/cgroup.h @@ -40,13 +40,11 @@ struct kernel_clone_args; #define CGROUP_WEIGHT_DFL 100 #define CGROUP_WEIGHT_MAX 10000 -/* walk only threadgroup leaders */ -#define CSS_TASK_ITER_PROCS (1U << 0) -/* walk all threaded css_sets in the domain */ -#define CSS_TASK_ITER_THREADED (1U << 1) - -/* internal flags */ -#define CSS_TASK_ITER_SKIPPED (1U << 16) +enum { + CSS_TASK_ITER_PROCS = (1U << 0), /* walk only threadgroup leaders */ + CSS_TASK_ITER_THREADED = (1U << 1), /* walk all threaded css_sets in the domain */ + CSS_TASK_ITER_SKIPPED = (1U << 16), /* internal flags */ +}; /* a css_task_iter should be treated as an opaque object */ struct css_task_iter { diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c index 1fb7f562289d..b6d64f3b8888 100644 --- a/kernel/cgroup/cgroup.c +++ b/kernel/cgroup/cgroup.c @@ -4917,9 +4917,11 @@ static void css_task_iter_advance(struct css_task_iter *it) void css_task_iter_start(struct cgroup_subsys_state *css, unsigned int flags, struct css_task_iter *it) { + unsigned long irqflags; + memset(it, 0, sizeof(*it)); - spin_lock_irq(&css_set_lock); + spin_lock_irqsave(&css_set_lock, irqflags); it->ss = css->ss; it->flags = flags; @@ -4933,7 +4935,7 @@ void css_task_iter_start(struct cgroup_subsys_state *css, unsigned int flags, css_task_iter_advance(it); - spin_unlock_irq(&css_set_lock); + spin_unlock_irqrestore(&css_set_lock, irqflags); } /** @@ -4946,12 +4948,14 @@ void css_task_iter_start(struct cgroup_subsys_state *css, unsigned int flags, */ struct task_struct *css_task_iter_next(struct css_task_iter *it) { + unsigned long irqflags; + if (it->cur_task) { put_task_struct(it->cur_task); it->cur_task = NULL; } - spin_lock_irq(&css_set_lock); + spin_lock_irqsave(&css_set_lock, irqflags); /* @it may be half-advanced by skips, finish advancing */ if (it->flags & CSS_TASK_ITER_SKIPPED) @@ -4964,7 +4968,7 @@ struct task_struct *css_task_iter_next(struct css_task_iter *it) css_task_iter_advance(it); } - spin_unlock_irq(&css_set_lock); + spin_unlock_irqrestore(&css_set_lock, irqflags); return it->cur_task; } @@ -4977,11 +4981,13 @@ struct task_struct *css_task_iter_next(struct css_task_iter *it) */ void css_task_iter_end(struct css_task_iter *it) { + unsigned long irqflags; + if (it->cur_cset) { - spin_lock_irq(&css_set_lock); + spin_lock_irqsave(&css_set_lock, irqflags); list_del(&it->iters_node); put_css_set_locked(it->cur_cset); - spin_unlock_irq(&css_set_lock); + spin_unlock_irqrestore(&css_set_lock, irqflags); } if (it->cur_dcset) From patchwork Wed Oct 11 12:08:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuyi Zhou X-Patchwork-Id: 151345 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp490965vqb; Wed, 11 Oct 2023 05:11:18 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHEoU5D4e8rpid28q+g9vqIlskXNxCG3HlHPWQFWU2whhD10nCAMDx18uMqtFJEMMMK/Bmv X-Received: by 2002:a17:90a:6848:b0:26d:4ade:fcf0 with SMTP id e8-20020a17090a684800b0026d4adefcf0mr18537469pjm.4.1697026278590; Wed, 11 Oct 2023 05:11:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697026278; cv=none; d=google.com; s=arc-20160816; b=wn474GIyX6oV+H3pXvlw6dY20QDZfRuraeNhkClDX5pzEYMIS7BhdUuR7BMcYlfmuw xir5QxGqbqL7vW6bQ5QRGeYGlGbDKpJAYopks+kzguKs5nX5wMzjJ7nMOJSVLri+ktG2 g3FnoPaL/kjUiAh8H0a95IN3CoN6hTGaBlmTpEFL23yTx0+PfpUHdzQA0GOz1K8LzdDf yUJ4lqMUYUOfaRgdSbcBO44oW/nhpzX9rg0ZPOS6YndX00fihS2P9+Sg/97hiFtj59uB EJFZ5amwT8/bAZlueijlem9V4J80Grc/sW5mayFt7CvwzALf1VigbOPwehYwgniYY3Fn J0ng== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=k7iVnDvjE1E7HEpdBixICufOsht21DeND3qHD2AP2FY=; fh=DR7g1EcWKOXTEoooUPBSJXUaklSrDEYzv6YDdhz1CwE=; b=cOvpzxO4j+lHmZ3mTqVT4OPIle90omBHDlVRK0mkpyM+4AAmxArIJCmDzGgxJMwn1f wJYNCYqdLPYXApFgReove+eIcUVurXrJkAO/UAsH5ErM5v1G3BugSkeo16TAxcIRap+v 3WO/6mgF8JjHEs8+Ecc4rMoogBBSr6ht1GlMoCceFGxrgp5U52ya5YYf4UPQ/lgg4gQH O8Tyc5wTbRTmOoocmhgiA1rgnc4sfD9kvoyUPnonJJnOlMdGwzOgnERHashEVBfKp3Gp ot7ppZeX4LOWQpsftsQj6N6ijG93bihkIuKbIW/SGmzPr45dyx2plNmeCq2ID9FXdSUD tZZA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=lDgLyfF8; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from agentk.vger.email (agentk.vger.email. [2620:137:e000::3:2]) by mx.google.com with ESMTPS id l17-20020a17090aaa9100b0027cde7f5860si4411065pjq.30.2023.10.11.05.11.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Oct 2023 05:11:18 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) client-ip=2620:137:e000::3:2; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=lDgLyfF8; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id 88DE581354AC; Wed, 11 Oct 2023 05:11:15 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346767AbjJKMKf (ORCPT + 18 others); Wed, 11 Oct 2023 08:10:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57046 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346804AbjJKMJr (ORCPT ); Wed, 11 Oct 2023 08:09:47 -0400 Received: from mail-pf1-x42e.google.com (mail-pf1-x42e.google.com [IPv6:2607:f8b0:4864:20::42e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 34417F4 for ; Wed, 11 Oct 2023 05:09:23 -0700 (PDT) Received: by mail-pf1-x42e.google.com with SMTP id d2e1a72fcca58-690f7bf73ddso4826485b3a.2 for ; Wed, 11 Oct 2023 05:09:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1697026161; x=1697630961; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=k7iVnDvjE1E7HEpdBixICufOsht21DeND3qHD2AP2FY=; b=lDgLyfF8h2WKiMS/wqp25b4uqWaSOiTI7J03qOEzdt5h7wk46AGDGPASNHI3c0auXh 2xT8QLqUtmGM3KhvtHUJ/lLtFZyVKtYu8AfYnRckn5D4AquyEHIKCgH5Pwqz3zjLr/Oo BcqQgFqykZ4eQg8DEIpiKQA6WbmnOF/NnIafdboHFxlndZZOsAkoqw2e/9h7HRa/6yk3 too9s644/G3dTOe7S3oFs/sz/FAKk1+coT4bp6ZTIXoeqeLeN9B5mfMybjMbI9EVJBdU RLG3yX/8eMWnBXIq+TAA6+O87ysMtpQEgeB4GATjPGwUoOMdCJYSEeA3ySe/Kw7Dj42+ VCpQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697026161; x=1697630961; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=k7iVnDvjE1E7HEpdBixICufOsht21DeND3qHD2AP2FY=; b=Cj0gz60BJQTJkFItFcxJWBHmKcnoq6dOt3aK66ntuu9L/iJtBOqQmdGQGUsonyaN7e +cb21B5je6A6/VJmZbOETuU9lrrC/pGZ++p3YN6kg3Pn5l/GVOJPJGladd+9FCsXWjFt w+SzVS96cjX0U4WCyJlcJ5jzYSx2Jd9aLYUjb2qD9ICkVnOkb7b2WbW8wKtRL1vfOi8d Y2UeO0uGcPx18yvJ/HYg9IsX0XUab7wts525pmFNN9gd8jlCAP0956FQMDM0bl87z6S8 RGBhaNeakKa1/UpgxsBNuxHZfpBdrALPjCFRU68F4nfr0LJbWc2emeKPs4/e950TLiWi +Cvw== X-Gm-Message-State: AOJu0YzE6YdP34Nz7NeYrNWgph8Jp7/qklGxMzpEjSltv5SU1BDGBIin C7KmJw+5MfHCjy8sawn9pknjRw== X-Received: by 2002:a05:6a00:1825:b0:690:f6a1:ae62 with SMTP id y37-20020a056a00182500b00690f6a1ae62mr22054883pfa.23.1697026161464; Wed, 11 Oct 2023 05:09:21 -0700 (PDT) Received: from n37-019-243.byted.org ([180.184.51.142]) by smtp.gmail.com with ESMTPSA id u13-20020a62ed0d000000b006930db1e6cfsm9962769pfh.62.2023.10.11.05.09.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Oct 2023 05:09:21 -0700 (PDT) From: Chuyi Zhou To: bpf@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@kernel.org, tj@kernel.org, linux-kernel@vger.kernel.org, Chuyi Zhou Subject: [PATCH bpf-next v5 2/8] bpf: Introduce css_task open-coded iterator kfuncs Date: Wed, 11 Oct 2023 20:08:51 +0800 Message-Id: <20231011120857.251943-3-zhouchuyi@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20231011120857.251943-1-zhouchuyi@bytedance.com> References: <20231011120857.251943-1-zhouchuyi@bytedance.com> MIME-Version: 1.0 X-Spam-Status: No, score=2.7 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, RCVD_IN_SBL_CSS,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Wed, 11 Oct 2023 05:11:15 -0700 (PDT) X-Spam-Level: ** X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779461027061487408 X-GMAIL-MSGID: 1779461027061487408 This patch adds kfuncs bpf_iter_css_task_{new,next,destroy} which allow creation and manipulation of struct bpf_iter_css_task in open-coded iterator style. These kfuncs actually wrapps css_task_iter_{start,next, end}. BPF programs can use these kfuncs through bpf_for_each macro for iteration of all tasks under a css. css_task_iter_*() would try to get the global spin-lock *css_set_lock*, so the bpf side has to be careful in where it allows to use this iter. Currently we only allow it in bpf_lsm and bpf iter-s. Signed-off-by: Chuyi Zhou Acked-by: Tejun Heo --- kernel/bpf/helpers.c | 3 ++ kernel/bpf/task_iter.c | 53 +++++++++++++++++++ kernel/bpf/verifier.c | 23 ++++++++ .../testing/selftests/bpf/bpf_experimental.h | 8 +++ 4 files changed, 87 insertions(+) diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index d2840dd5b00d..cb24c4a916df 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -2552,6 +2552,9 @@ BTF_ID_FLAGS(func, bpf_dynptr_slice_rdwr, KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_num_new, KF_ITER_NEW) BTF_ID_FLAGS(func, bpf_iter_num_next, KF_ITER_NEXT | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_num_destroy, KF_ITER_DESTROY) +BTF_ID_FLAGS(func, bpf_iter_css_task_new, KF_ITER_NEW | KF_TRUSTED_ARGS) +BTF_ID_FLAGS(func, bpf_iter_css_task_next, KF_ITER_NEXT | KF_RET_NULL) +BTF_ID_FLAGS(func, bpf_iter_css_task_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_dynptr_adjust) BTF_ID_FLAGS(func, bpf_dynptr_is_null) BTF_ID_FLAGS(func, bpf_dynptr_is_rdonly) diff --git a/kernel/bpf/task_iter.c b/kernel/bpf/task_iter.c index 7473068ed313..2cfcb4dd8a37 100644 --- a/kernel/bpf/task_iter.c +++ b/kernel/bpf/task_iter.c @@ -7,6 +7,7 @@ #include #include #include +#include #include #include "mmap_unlock_work.h" @@ -803,6 +804,58 @@ const struct bpf_func_proto bpf_find_vma_proto = { .arg5_type = ARG_ANYTHING, }; +struct bpf_iter_css_task { + __u64 __opaque[1]; +} __attribute__((aligned(8))); + +struct bpf_iter_css_task_kern { + struct css_task_iter *css_it; +} __attribute__((aligned(8))); + +__bpf_kfunc int bpf_iter_css_task_new(struct bpf_iter_css_task *it, + struct cgroup_subsys_state *css, unsigned int flags) +{ + struct bpf_iter_css_task_kern *kit = (void *)it; + + BUILD_BUG_ON(sizeof(struct bpf_iter_css_task_kern) != sizeof(struct bpf_iter_css_task)); + BUILD_BUG_ON(__alignof__(struct bpf_iter_css_task_kern) != + __alignof__(struct bpf_iter_css_task)); + kit->css_it = NULL; + switch (flags) { + case CSS_TASK_ITER_PROCS | CSS_TASK_ITER_THREADED: + case CSS_TASK_ITER_PROCS: + case 0: + break; + default: + return -EINVAL; + } + + kit->css_it = bpf_mem_alloc(&bpf_global_ma, sizeof(struct css_task_iter)); + if (!kit->css_it) + return -ENOMEM; + css_task_iter_start(css, flags, kit->css_it); + return 0; +} + +__bpf_kfunc struct task_struct *bpf_iter_css_task_next(struct bpf_iter_css_task *it) +{ + struct bpf_iter_css_task_kern *kit = (void *)it; + + if (!kit->css_it) + return NULL; + return css_task_iter_next(kit->css_it); +} + +__bpf_kfunc void bpf_iter_css_task_destroy(struct bpf_iter_css_task *it) +{ + struct bpf_iter_css_task_kern *kit = (void *)it; + + if (!kit->css_it) + return; + css_task_iter_end(kit->css_it); + bpf_mem_free(&bpf_global_ma, kit->css_it); +} + DEFINE_PER_CPU(struct mmap_unlock_irq_work, mmap_unlock_work); static void do_mmap_read_unlock(struct irq_work *entry) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index eed7350e15f4..528d375c17ee 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -10450,6 +10450,7 @@ enum special_kfunc_type { KF_bpf_percpu_obj_new_impl, KF_bpf_percpu_obj_drop_impl, KF_bpf_throw, + KF_bpf_iter_css_task_new, }; BTF_SET_START(special_kfunc_set) @@ -10473,6 +10474,7 @@ BTF_ID(func, bpf_dynptr_clone) BTF_ID(func, bpf_percpu_obj_new_impl) BTF_ID(func, bpf_percpu_obj_drop_impl) BTF_ID(func, bpf_throw) +BTF_ID(func, bpf_iter_css_task_new) BTF_SET_END(special_kfunc_set) BTF_ID_LIST(special_kfunc_list) @@ -10498,6 +10500,7 @@ BTF_ID(func, bpf_dynptr_clone) BTF_ID(func, bpf_percpu_obj_new_impl) BTF_ID(func, bpf_percpu_obj_drop_impl) BTF_ID(func, bpf_throw) +BTF_ID(func, bpf_iter_css_task_new) static bool is_kfunc_ret_null(struct bpf_kfunc_call_arg_meta *meta) { @@ -11028,6 +11031,20 @@ static int process_kf_arg_ptr_to_rbtree_node(struct bpf_verifier_env *env, &meta->arg_rbtree_root.field); } +static bool check_css_task_iter_allowlist(struct bpf_verifier_env *env) +{ + enum bpf_prog_type prog_type = resolve_prog_type(env->prog); + + switch (prog_type) { + case BPF_PROG_TYPE_LSM: + return true; + case BPF_TRACE_ITER: + return env->prog->aux->sleepable; + default: + return false; + } +} + static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_arg_meta *meta, int insn_idx) { @@ -11278,6 +11295,12 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ break; } case KF_ARG_PTR_TO_ITER: + if (meta->func_id == special_kfunc_list[KF_bpf_iter_css_task_new]) { + if (!check_css_task_iter_allowlist(env)) { + verbose(env, "css_task_iter is only allowed in bpf_lsm and bpf iter-s\n"); + return -EINVAL; + } + } ret = process_iter_arg(env, regno, insn_idx, meta); if (ret < 0) return ret; diff --git a/tools/testing/selftests/bpf/bpf_experimental.h b/tools/testing/selftests/bpf/bpf_experimental.h index 9aa29564bd74..8b53537e0f27 100644 --- a/tools/testing/selftests/bpf/bpf_experimental.h +++ b/tools/testing/selftests/bpf/bpf_experimental.h @@ -450,4 +450,12 @@ extern void bpf_throw(u64 cookie) __ksym; __bpf_assert_op(LHS, <=, END, value, false); \ }) +struct bpf_iter_css_task; +struct cgroup_subsys_state; +extern int bpf_iter_css_task_new(struct bpf_iter_css_task *it, + struct cgroup_subsys_state *css, unsigned int flags) __weak __ksym; +extern struct task_struct *bpf_iter_css_task_next(struct bpf_iter_css_task *it) __weak __ksym; +extern void bpf_iter_css_task_destroy(struct bpf_iter_css_task *it) __weak __ksym; + + #endif From patchwork Wed Oct 11 12:08:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuyi Zhou X-Patchwork-Id: 151348 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp491152vqb; Wed, 11 Oct 2023 05:11:36 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEerA8Bl+wxLPBDRBhkP4u/RBvtMyYfVa6mWW4Lqnm6zMEK+7/sKcAletwH8IpfWwugyvdP X-Received: by 2002:a05:6a21:a587:b0:163:c167:964a with SMTP id gd7-20020a056a21a58700b00163c167964amr26315876pzc.1.1697026296488; Wed, 11 Oct 2023 05:11:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697026296; cv=none; d=google.com; s=arc-20160816; b=NIGkGgigNGzg6Db2DeGJrpRaAVYLGPN85VwkUa9EeClvY7zURe6B/S3//ofBVYl1zy eOZ/Pz2lSQ1S81kdkBFq1OlIDJNmsEOdhzH6Z0zcqXzMAEd5cBT7YSAMrC1N1+2IevQX DzF3TC2/uIOpUHVt5AZPaLTXkVnXRho9bebfDd9WLTNvI38mqzC5Wyea/8AM8q5cIRqy jySAC2SxlZNdJiZmROeOeqGe/oWUIGtWwALirVPRudyAlssmM+K97DxVp84801lbuDLU zZtjjWDAZYwzvtH4RC3oGoRVTBqmJiwi/gXsw5bbM5ea9GDXy0uJgKt3xdF9vnlz0cKb 99Gg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=+eGErJyAmlj72NVapjxnwiwBdTl0+C6Vctyi1OYX3+k=; fh=DR7g1EcWKOXTEoooUPBSJXUaklSrDEYzv6YDdhz1CwE=; b=FF7L0tmr82fNqHJfRhkhPGx0R6GVt8cpXNrS1XXbqBKuo5RqBMQNhMSHzhKSLUPwE7 PkLrUMYyvatV7+EwDekrt9cD1uCNUaLkanovJMBZNKxcuGdmV/siH3fgDXV3fZzvCbYd KKBULneASYtKUogQ/kGf9xiUzw6tpfIs/whEFMpHCTis3Wxnms6MTrWCLkYryn6zYyme XQAZPgpUlIhVA7CFa5qgFabWnpWq78ByQPz4W0yUi8xqo1xhEohggN3GkSyb638Gmthf n1vTFSxGORC7GxyH9mcpSUDIuCbZZ2W2DA86gTM3FGBMhckzyTZyBdLr6FtjK9/UX4YI HJkw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=Q75PKDfv; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from lipwig.vger.email (lipwig.vger.email. [2620:137:e000::3:3]) by mx.google.com with ESMTPS id h14-20020a170902680e00b001c9c10f0d44si2610421plk.8.2023.10.11.05.11.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Oct 2023 05:11:36 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) client-ip=2620:137:e000::3:3; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=Q75PKDfv; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id 63091824C4C1; Wed, 11 Oct 2023 05:11:28 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346860AbjJKMKq (ORCPT + 18 others); Wed, 11 Oct 2023 08:10:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56926 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346822AbjJKMJv (ORCPT ); Wed, 11 Oct 2023 08:09:51 -0400 Received: from mail-pf1-x433.google.com (mail-pf1-x433.google.com [IPv6:2607:f8b0:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EB3CCB7 for ; Wed, 11 Oct 2023 05:09:31 -0700 (PDT) Received: by mail-pf1-x433.google.com with SMTP id d2e1a72fcca58-690ba63891dso5161284b3a.2 for ; Wed, 11 Oct 2023 05:09:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1697026165; x=1697630965; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=+eGErJyAmlj72NVapjxnwiwBdTl0+C6Vctyi1OYX3+k=; b=Q75PKDfvOvXM4GmpyNYcTLyL865BRJGnxPG/0nEQPKGAln2BUNx0CLGfcuIrYrHh1x /24IrWqAKKxI4OYreihRAq0SU+j9ZBpzKrNVgfTAMViUn3tsjtwhPWpsC6MbADbWF40Z ic/qoMn+lHgV4KTE8YZOjO19uXeZllxslPZ07EemKRyV4UETu4/NdWr0Tg7y8v62Svml JCpUZSgOZwVzh+4NGFrcl3JW5ShACsUZt0d3tS95d3a97i6T3fYHkfoRrXHtcbqfC6lY DvipNl+CEAnea/UEviD1pbqXDBcIiFKSUSv5BD+pkiQ/8LuqMo9SSQVTRy1s3pqMICM8 QkHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697026165; x=1697630965; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+eGErJyAmlj72NVapjxnwiwBdTl0+C6Vctyi1OYX3+k=; b=VpiTzavW/CszEMQ8E4J5E9gNgwz6IH/8jhI+woCUQ44pE4M+Sz0m5NakLd7DVBCSQv YbzcPHiwuOCrwpryAPMatd+C6h0yansdrEwwbJABl840Tj9T0bicR7PWEm6GKGd1kNYA rTljDszMG31yeFD46cUfZivJEjFocJ5poHJ9UzAYpcgoSRnX478VWqzkW2XFeeY1TZ+y sVnMmHFbiJZ4V52RtmP7hJtJXnu4iEOuD6S629osDD1ql59aptkgtbN6Ox1NLhZEWB7u mCyiKlDWZHngb40rrJFxCsinOvLoTqhZlfTU3czZ7OrBRrEfLSvH2DnD9Smv7Y7UYkBR cFsw== X-Gm-Message-State: AOJu0YyvqZf/1YQZmO1/F2GxLINO41IugYoXWogIkdagLxUjLuDRBPPP SarF4qP8I3AZJhtvaIdgVB94Zw== X-Received: by 2002:a05:6a00:1790:b0:68f:c057:b567 with SMTP id s16-20020a056a00179000b0068fc057b567mr21438572pfg.26.1697026164800; Wed, 11 Oct 2023 05:09:24 -0700 (PDT) Received: from n37-019-243.byted.org ([180.184.51.142]) by smtp.gmail.com with ESMTPSA id u13-20020a62ed0d000000b006930db1e6cfsm9962769pfh.62.2023.10.11.05.09.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Oct 2023 05:09:24 -0700 (PDT) From: Chuyi Zhou To: bpf@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@kernel.org, tj@kernel.org, linux-kernel@vger.kernel.org, Chuyi Zhou Subject: [PATCH bpf-next v5 3/8] bpf: Introduce task open coded iterator kfuncs Date: Wed, 11 Oct 2023 20:08:52 +0800 Message-Id: <20231011120857.251943-4-zhouchuyi@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20231011120857.251943-1-zhouchuyi@bytedance.com> References: <20231011120857.251943-1-zhouchuyi@bytedance.com> MIME-Version: 1.0 X-Spam-Status: No, score=2.7 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, RCVD_IN_SBL_CSS,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Wed, 11 Oct 2023 05:11:28 -0700 (PDT) X-Spam-Level: ** X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779461046210792922 X-GMAIL-MSGID: 1779461046210792922 This patch adds kfuncs bpf_iter_task_{new,next,destroy} which allow creation and manipulation of struct bpf_iter_task in open-coded iterator style. BPF programs can use these kfuncs or through bpf_for_each macro to iterate all processes in the system. The API design keep consistent with SEC("iter/task"). bpf_iter_task_new() accepts a specific task and iterating type which allows: 1. iterating all process in the system(BPF_TASK_ITER_ALL_PROCS) 2. iterating all threads in the system(BPF_TASK_ITER_ALL_THREADS) 3. iterating all threads of a specific task(BPF_TASK_ITER_PROC_THREADS) Signed-off-by: Chuyi Zhou --- kernel/bpf/helpers.c | 3 + kernel/bpf/task_iter.c | 82 +++++++++++++++++++ .../testing/selftests/bpf/bpf_experimental.h | 5 ++ 3 files changed, 90 insertions(+) diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index cb24c4a916df..690763751f6e 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -2555,6 +2555,9 @@ BTF_ID_FLAGS(func, bpf_iter_num_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_iter_css_task_new, KF_ITER_NEW | KF_TRUSTED_ARGS) BTF_ID_FLAGS(func, bpf_iter_css_task_next, KF_ITER_NEXT | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_css_task_destroy, KF_ITER_DESTROY) +BTF_ID_FLAGS(func, bpf_iter_task_new, KF_ITER_NEW | KF_TRUSTED_ARGS) +BTF_ID_FLAGS(func, bpf_iter_task_next, KF_ITER_NEXT | KF_RET_NULL) +BTF_ID_FLAGS(func, bpf_iter_task_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_dynptr_adjust) BTF_ID_FLAGS(func, bpf_dynptr_is_null) BTF_ID_FLAGS(func, bpf_dynptr_is_rdonly) diff --git a/kernel/bpf/task_iter.c b/kernel/bpf/task_iter.c index 2cfcb4dd8a37..caeddad3d2f1 100644 --- a/kernel/bpf/task_iter.c +++ b/kernel/bpf/task_iter.c @@ -856,6 +856,88 @@ __bpf_kfunc void bpf_iter_css_task_destroy(struct bpf_iter_css_task *it) bpf_mem_free(&bpf_global_ma, kit->css_it); } +struct bpf_iter_task { + __u64 __opaque[3]; +} __attribute__((aligned(8))); + +struct bpf_iter_task_kern { + struct task_struct *task; + struct task_struct *pos; + unsigned int flags; +} __attribute__((aligned(8))); + +enum { + BPF_TASK_ITER_ALL_PROCS, + BPF_TASK_ITER_ALL_THREADS, + BPF_TASK_ITER_PROC_THREADS +}; + +__bpf_kfunc int bpf_iter_task_new(struct bpf_iter_task *it, + struct task_struct *task, unsigned int flags) +{ + struct bpf_iter_task_kern *kit = (void *)it; + + BUILD_BUG_ON(sizeof(struct bpf_iter_task_kern) > sizeof(struct bpf_iter_task)); + BUILD_BUG_ON(__alignof__(struct bpf_iter_task_kern) != + __alignof__(struct bpf_iter_task)); + + kit->task = kit->pos = NULL; + switch (flags) { + case BPF_TASK_ITER_ALL_THREADS: + case BPF_TASK_ITER_ALL_PROCS: + case BPF_TASK_ITER_PROC_THREADS: + break; + default: + return -EINVAL; + } + + if (flags == BPF_TASK_ITER_PROC_THREADS) + kit->task = task; + else + kit->task = &init_task; + kit->pos = kit->task; + kit->flags = flags; + return 0; +} + +__bpf_kfunc struct task_struct *bpf_iter_task_next(struct bpf_iter_task *it) +{ + struct bpf_iter_task_kern *kit = (void *)it; + struct task_struct *pos; + unsigned int flags; + + flags = kit->flags; + pos = kit->pos; + + if (!pos) + goto out; + + if (flags == BPF_TASK_ITER_ALL_PROCS) + goto get_next_task; + + kit->pos = next_thread(kit->pos); + if (kit->pos == kit->task) { + if (flags == BPF_TASK_ITER_PROC_THREADS) { + kit->pos = NULL; + goto out; + } + } else + goto out; + +get_next_task: + kit->pos = next_task(kit->pos); + kit->task = kit->pos; + if (kit->pos == &init_task) + kit->pos = NULL; + +out: + return pos; +} + +__bpf_kfunc void bpf_iter_task_destroy(struct bpf_iter_task *it) +{ +} + DEFINE_PER_CPU(struct mmap_unlock_irq_work, mmap_unlock_work); static void do_mmap_read_unlock(struct irq_work *entry) diff --git a/tools/testing/selftests/bpf/bpf_experimental.h b/tools/testing/selftests/bpf/bpf_experimental.h index 8b53537e0f27..1ec82997cce7 100644 --- a/tools/testing/selftests/bpf/bpf_experimental.h +++ b/tools/testing/selftests/bpf/bpf_experimental.h @@ -457,5 +457,10 @@ extern int bpf_iter_css_task_new(struct bpf_iter_css_task *it, extern struct task_struct *bpf_iter_css_task_next(struct bpf_iter_css_task *it) __weak __ksym; extern void bpf_iter_css_task_destroy(struct bpf_iter_css_task *it) __weak __ksym; +struct bpf_iter_task; +extern int bpf_iter_task_new(struct bpf_iter_task *it, + struct task_struct *task, unsigned int flags) __weak __ksym; +extern struct task_struct *bpf_iter_task_next(struct bpf_iter_task *it) __weak __ksym; +extern void bpf_iter_task_destroy(struct bpf_iter_task *it) __weak __ksym; #endif From patchwork Wed Oct 11 12:08:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuyi Zhou X-Patchwork-Id: 151362 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp491550vqb; Wed, 11 Oct 2023 05:12:07 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHqAF+hq57wSpAVCMed85w+0IgPCpMfbJqKGymgueGJcnsR+uiYynw/8Yu4vA3SwJ4cn3eT X-Received: by 2002:a05:6a21:a596:b0:163:d382:ba84 with SMTP id gd22-20020a056a21a59600b00163d382ba84mr25652901pzc.5.1697026327709; Wed, 11 Oct 2023 05:12:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697026327; cv=none; d=google.com; s=arc-20160816; b=E5yhFMPazuQgbGXhXixu2+N44GHvJMHRUV0HUYumvzi25ItM0vjyQKYGmC3DXNE6rj LKRZ937yGPGPMuKmkW/aajCxYDnaPcmbWWv7TNlshYqvPen478gM5EhaBpsMdeHwY8fx 9ooLtXZ5xDVU7B0TX7W1/t3wv2d7JVmyDGbCNC81wgi2gIYIGTzd0MBildItZCYryprs U3dJ+U5821DEFfLfrRJeYBD9+wbmkQ9Ku0J7JDPDoKl4Pbz4+9GD5IUXDxbiZP3ywqb4 qdPG8cqWPOjHOujJZ0HyFBp90HzSHUP+eOsJ+U2WoLshYBUQhD2Op6qy+PeztSVA60u9 wskA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=jb4XCkqj/mzqd+4TpsdA3MQ6n4XHCTZgLyR23iVy2IY=; fh=DR7g1EcWKOXTEoooUPBSJXUaklSrDEYzv6YDdhz1CwE=; b=QNIpnjRhBCFWy9NKy/Rng8W8o9Iyi8ASOnwCkGK2gFmCHh4mOnYhoDTEcnyFdFNk2y fVSro4nc/DR9Fx5XAG40Q/KLXBSJpxTZwY/p2ERrd7K8udl7FBLq9uyo4zXhJbdN4TZg tlt2uUQ5geBLn5Y+ewFWfsWs4NNZoLWtsVO+xlHfYdFGj/K1pqIWQw6Zd1cmdmqJe46E zhyrgJbL1tV6WxKUaLA8a0pZBWJ1+l2QPow7kEfYBUJfbZAjb5CUGP50dNHI39LPEke6 nNrHwgGVSFlY5zBiTFVN2afvG9FLd/3sgIxXU91SAUEIDnX/te7JIejSfFIE5Wn7kdEF m2rA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=NjZIAIq1; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:1 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from morse.vger.email (morse.vger.email. [2620:137:e000::3:1]) by mx.google.com with ESMTPS id b14-20020a170902d50e00b001bdc664cd69si4180671plg.153.2023.10.11.05.12.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Oct 2023 05:12:07 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:1 as permitted sender) client-ip=2620:137:e000::3:1; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=NjZIAIq1; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:1 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by morse.vger.email (Postfix) with ESMTP id 91C9B836D60F; Wed, 11 Oct 2023 05:12:00 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at morse.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346831AbjJKML1 (ORCPT + 18 others); Wed, 11 Oct 2023 08:11:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57030 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346906AbjJKMK1 (ORCPT ); Wed, 11 Oct 2023 08:10:27 -0400 Received: from mail-io1-xd33.google.com (mail-io1-xd33.google.com [IPv6:2607:f8b0:4864:20::d33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A3B401B1 for ; Wed, 11 Oct 2023 05:09:40 -0700 (PDT) Received: by mail-io1-xd33.google.com with SMTP id ca18e2360f4ac-79fce245bf6so266213139f.1 for ; Wed, 11 Oct 2023 05:09:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1697026168; x=1697630968; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jb4XCkqj/mzqd+4TpsdA3MQ6n4XHCTZgLyR23iVy2IY=; b=NjZIAIq1ZljMAE8nAyTpgqx6BMlmgdjyUoD6YzslAGbNKcKwkmkKpFoeLLPz0H13iG wYnnldAvGm9134va8/1857CPIYCPnGbfH7kJxgyJDUUKueEnNbi/8bJF7WDiEXW6Nn5L p+vbEOd70b6zW8T72V7psSHO8oRZiCBnqwGoJTDyNs2eRhkDbxBdjmbYBE4DXBkuVleT GfF6PxLz08/6dmS7iEbPThlBqKwjH84qTud/9/ElysYPxt85Q/HQv4QCDd03MvWmHJQ7 /3LyeBrrDobJOkmiydCmcSdoLbzenjbAzNqo2TaK+Z2HOoaCGMSFlMMvDYj4GqImyPpN fOqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697026168; x=1697630968; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jb4XCkqj/mzqd+4TpsdA3MQ6n4XHCTZgLyR23iVy2IY=; b=rL/k8HHGeYRnkyZO5oaY8EXyXw78HvCDXgx/QAxZEzyxPhoOLpZtqIlkCJVxRp9H6O hbeDI77MbMvvvQVbM1zW4P5XKwG8Q6lqroFJUqlgavngc7fTNjBBqW304JujT+w6IPpG ohq8FwY0nfYE7jCjbTtUyKcBhQ4j6JdpO7FehXcS+uatAQ0ysip+2JoxnfP2pszFvLiN wsP33YuMRpNkCLGpnIhAhhtTkoJDSsRDNISKSQB7OK700EWFqMS5JIfl5x0V61HiPDzD sn96c9HNAH6EUIzB6B5kBm5dDolaXRBNFtvkRmtv5gvsLvoPn0YABAr9fYxbsOwxpkNT 6TJw== X-Gm-Message-State: AOJu0YyUP5FOd4UIo8GsI6kNQMALFdJEVKD+bYKw3AqqyGeZfVrnl6n3 AGM8D469YTuTcWh1xkbxXOZnSbHTdCE9vuz72Ww= X-Received: by 2002:a05:6e02:1a4b:b0:352:6251:2364 with SMTP id u11-20020a056e021a4b00b0035262512364mr25805078ilv.11.1697026168227; Wed, 11 Oct 2023 05:09:28 -0700 (PDT) Received: from n37-019-243.byted.org ([180.184.51.142]) by smtp.gmail.com with ESMTPSA id u13-20020a62ed0d000000b006930db1e6cfsm9962769pfh.62.2023.10.11.05.09.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Oct 2023 05:09:27 -0700 (PDT) From: Chuyi Zhou To: bpf@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@kernel.org, tj@kernel.org, linux-kernel@vger.kernel.org, Chuyi Zhou Subject: [PATCH bpf-next v5 4/8] bpf: Introduce css open-coded iterator kfuncs Date: Wed, 11 Oct 2023 20:08:53 +0800 Message-Id: <20231011120857.251943-5-zhouchuyi@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20231011120857.251943-1-zhouchuyi@bytedance.com> References: <20231011120857.251943-1-zhouchuyi@bytedance.com> MIME-Version: 1.0 X-Spam-Status: No, score=2.7 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, RCVD_IN_SBL_CSS,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on morse.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (morse.vger.email [0.0.0.0]); Wed, 11 Oct 2023 05:12:00 -0700 (PDT) X-Spam-Level: ** X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779461078641185961 X-GMAIL-MSGID: 1779461078641185961 This Patch adds kfuncs bpf_iter_css_{new,next,destroy} which allow creation and manipulation of struct bpf_iter_css in open-coded iterator style. These kfuncs actually wrapps css_next_descendant_{pre, post}. css_iter can be used to: 1) iterating a sepcific cgroup tree with pre/post/up order 2) iterating cgroup_subsystem in BPF Prog, like for_each_mem_cgroup_tree/cpuset_for_each_descendant_pre in kernel. The API design is consistent with cgroup_iter. bpf_iter_css_new accepts parameters defining iteration order and starting css. Here we also reuse BPF_CGROUP_ITER_DESCENDANTS_PRE, BPF_CGROUP_ITER_DESCENDANTS_POST, BPF_CGROUP_ITER_ANCESTORS_UP enums. Signed-off-by: Chuyi Zhou Acked-by: Tejun Heo Signed-off-by: Chuyi Zhou --- kernel/bpf/cgroup_iter.c | 59 +++++++++++++++++++ kernel/bpf/helpers.c | 3 + .../testing/selftests/bpf/bpf_experimental.h | 6 ++ 3 files changed, 68 insertions(+) diff --git a/kernel/bpf/cgroup_iter.c b/kernel/bpf/cgroup_iter.c index 810378f04fbc..df2d6b6a5dd8 100644 --- a/kernel/bpf/cgroup_iter.c +++ b/kernel/bpf/cgroup_iter.c @@ -294,3 +294,62 @@ static int __init bpf_cgroup_iter_init(void) } late_initcall(bpf_cgroup_iter_init); + +struct bpf_iter_css { + __u64 __opaque[3]; +} __attribute__((aligned(8))); + +struct bpf_iter_css_kern { + struct cgroup_subsys_state *start; + struct cgroup_subsys_state *pos; + unsigned int flags; +} __attribute__((aligned(8))); + +__bpf_kfunc int bpf_iter_css_new(struct bpf_iter_css *it, + struct cgroup_subsys_state *start, unsigned int flags) +{ + struct bpf_iter_css_kern *kit = (void *)it; + + BUILD_BUG_ON(sizeof(struct bpf_iter_css_kern) > sizeof(struct bpf_iter_css)); + BUILD_BUG_ON(__alignof__(struct bpf_iter_css_kern) != __alignof__(struct bpf_iter_css)); + + kit->start = NULL; + switch (flags) { + case BPF_CGROUP_ITER_DESCENDANTS_PRE: + case BPF_CGROUP_ITER_DESCENDANTS_POST: + case BPF_CGROUP_ITER_ANCESTORS_UP: + break; + default: + return -EINVAL; + } + + kit->start = start; + kit->pos = NULL; + kit->flags = flags; + return 0; +} + +__bpf_kfunc struct cgroup_subsys_state *bpf_iter_css_next(struct bpf_iter_css *it) +{ + struct bpf_iter_css_kern *kit = (void *)it; + + if (!kit->start) + return NULL; + + switch (kit->flags) { + case BPF_CGROUP_ITER_DESCENDANTS_PRE: + kit->pos = css_next_descendant_pre(kit->pos, kit->start); + break; + case BPF_CGROUP_ITER_DESCENDANTS_POST: + kit->pos = css_next_descendant_post(kit->pos, kit->start); + break; + case BPF_CGROUP_ITER_ANCESTORS_UP: + kit->pos = kit->pos ? kit->pos->parent : kit->start; + } + + return kit->pos; +} + +__bpf_kfunc void bpf_iter_css_destroy(struct bpf_iter_css *it) +{ +} diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index 690763751f6e..6330e37ca8d5 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -2558,6 +2558,9 @@ BTF_ID_FLAGS(func, bpf_iter_css_task_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_iter_task_new, KF_ITER_NEW | KF_TRUSTED_ARGS) BTF_ID_FLAGS(func, bpf_iter_task_next, KF_ITER_NEXT | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_task_destroy, KF_ITER_DESTROY) +BTF_ID_FLAGS(func, bpf_iter_css_new, KF_ITER_NEW | KF_TRUSTED_ARGS) +BTF_ID_FLAGS(func, bpf_iter_css_next, KF_ITER_NEXT | KF_RET_NULL) +BTF_ID_FLAGS(func, bpf_iter_css_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_dynptr_adjust) BTF_ID_FLAGS(func, bpf_dynptr_is_null) BTF_ID_FLAGS(func, bpf_dynptr_is_rdonly) diff --git a/tools/testing/selftests/bpf/bpf_experimental.h b/tools/testing/selftests/bpf/bpf_experimental.h index 1ec82997cce7..9aab609f6edd 100644 --- a/tools/testing/selftests/bpf/bpf_experimental.h +++ b/tools/testing/selftests/bpf/bpf_experimental.h @@ -463,4 +463,10 @@ extern int bpf_iter_task_new(struct bpf_iter_task *it, extern struct task_struct *bpf_iter_task_next(struct bpf_iter_task *it) __weak __ksym; extern void bpf_iter_task_destroy(struct bpf_iter_task *it) __weak __ksym; +struct bpf_iter_css; +extern int bpf_iter_css_new(struct bpf_iter_css *it, + struct cgroup_subsys_state *start, unsigned int flags) __weak __ksym; +extern struct cgroup_subsys_state *bpf_iter_css_next(struct bpf_iter_css *it) __weak __ksym; +extern void bpf_iter_css_destroy(struct bpf_iter_css *it) __weak __ksym; + #endif From patchwork Wed Oct 11 12:08:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuyi Zhou X-Patchwork-Id: 151375 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp492415vqb; Wed, 11 Oct 2023 05:13:29 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGQ4qhllnR+97XI5cwKsH2Kk4BNSlHqkD4OoRlrCQdOpfsK229R1ewELDfKcEAkEtMRmZTZ X-Received: by 2002:a05:6358:f17:b0:14d:8792:1cec with SMTP id b23-20020a0563580f1700b0014d87921cecmr13461653rwj.1.1697026408797; Wed, 11 Oct 2023 05:13:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697026408; cv=none; d=google.com; s=arc-20160816; b=YHcPOeS74vQmyo6pkhRmpwid8cHz5SLSFvCCKrLcSaivat8OSCwKnirH139fbyGma/ IPw8PrbpEtHIVlX0HDvWl0nllm0UIM7+9ylb0jJXhCNlZo6EjYxXo2AQj8tqRzHHWa9I Np/IquEgMReurtHrteyc6dkRISF4uGpsKcaA8E9Kt/p/jJq2wXwTAin0MV6Q0+CpCHNa Rbo6UnY47OHLngVGvEGlZuqR7LsJzDvfhTPwiBstIWSSzThm0aqGk+pWoiFrYdpTV/aS o4OsUoJERwJwj9KQTkuC4mIpObXOHvhXwKvnhaWRi5b11oRhEUxu/OFisuAc8xtjJCop IS5A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=spK2veKjGKPWlL695PFbHPuUiEqdtYJXw3T2DQ3u9LI=; fh=DR7g1EcWKOXTEoooUPBSJXUaklSrDEYzv6YDdhz1CwE=; b=kQPbNZXARvzl5vM9xJS1K8GqIUSY/SANIF+/lYhNZAw8NgtKJx7A8JcoE4lzxxWOax lb1w73FcCbmlDnUHk4Mxjiv7Wlbq70T68OBhzhn2XAa/6fL/W2Sd5kiDccrZPYooxvSm BJF1XOW6MLr9qVI8ifCpsLpeaM4bti9au5QY0W6caeuHPf4JQfqdep55BQN4OPczjl0R 2OU619mLPm7EYM8lcTrGye9qzjKMPup1SIs6RbEv4kmo3wekw58Xmj3uhtnFMru9NYbx 761c7OVi3eKLw8fVoWM37WjriJ6HxaVw++5MgjscSYVFQMJIvrXf83kNIfnjfK95nIPN lFcQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=HNYpW+UB; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from pete.vger.email (pete.vger.email. [2620:137:e000::3:6]) by mx.google.com with ESMTPS id a70-20020a639049000000b00588fe3d83b3si14299704pge.408.2023.10.11.05.13.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Oct 2023 05:13:28 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) client-ip=2620:137:e000::3:6; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=HNYpW+UB; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by pete.vger.email (Postfix) with ESMTP id C2BD580DB9E9; Wed, 11 Oct 2023 05:13:10 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at pete.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346907AbjJKML5 (ORCPT + 18 others); Wed, 11 Oct 2023 08:11:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53704 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346854AbjJKMKp (ORCPT ); Wed, 11 Oct 2023 08:10:45 -0400 Received: from mail-pf1-x42a.google.com (mail-pf1-x42a.google.com [IPv6:2607:f8b0:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B8554D58 for ; Wed, 11 Oct 2023 05:09:43 -0700 (PDT) Received: by mail-pf1-x42a.google.com with SMTP id d2e1a72fcca58-694ed847889so5203357b3a.2 for ; Wed, 11 Oct 2023 05:09:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1697026172; x=1697630972; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=spK2veKjGKPWlL695PFbHPuUiEqdtYJXw3T2DQ3u9LI=; b=HNYpW+UBrqt1EsgOIjp8OlgFt+mMZYusVSuqLV+l/QNLFNSyFc8wDu52pcMSXvk8aN NTmBSuhCYHw1X6T7563JgBBENauR1pagGauCpBDUSWy5DcN6RR0aDrw36Lub/A3yU5jD wToRVraUczoB+pltLxsW56ixxIdV4uskoeyaIYSxmXCglA3hK0/TLBWcZHgNY9+P2tTX ODstqAamlpifAG4ZTg1TyS3WCouQytiQQg4UlBjsPQp3XA3fx3ncHvUSZXxybeTYlXD0 JaSwiC6P8aB09giMMPD64moeHD+fROMg1vMnNCwDVG4Dgis5ff/x1hhndpkqaE7XQcoC b18A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697026172; x=1697630972; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=spK2veKjGKPWlL695PFbHPuUiEqdtYJXw3T2DQ3u9LI=; b=nu5yy6QcjSK3XJICQM1jStejUzBtXG0PLYJOfOPiC/xXXuz89aa5rxcL1zoM3KwQX0 2ts8g4Xv6FK/aE3I50GIpF7VKSRN0a5smJK4OaYqCWcJ10cDwb93OHrNDbP7eVgU4W57 BVv+jcgRVn+PJN8YgOfYhEtqVN2so+Z7hC4ZNr33hydMZEarykpM7pvEd7DircGLOhJ1 fFAy/oSXR+8GJwls8jn5RdbyxagGTvBkuADNFthhRC497LvbqsdM/f9mtv4kPx+uttwS Q0QzkWMLVKN/XS14jhOHQ/pDFyy3unM4Vke7bWQ8pzd41NiMYLw0P2bZdy/N65D3lfT0 P9Ng== X-Gm-Message-State: AOJu0Yy76vnIvt+E71QAENQoWjLy1gtbUxPsQ+0wGDytfSudwYJhuVKW tAwmj0q6K9DX1MHOXkwqb/B5Og== X-Received: by 2002:a05:6a00:1943:b0:690:d4fa:d43d with SMTP id s3-20020a056a00194300b00690d4fad43dmr18270745pfk.6.1697026171738; Wed, 11 Oct 2023 05:09:31 -0700 (PDT) Received: from n37-019-243.byted.org ([180.184.51.142]) by smtp.gmail.com with ESMTPSA id u13-20020a62ed0d000000b006930db1e6cfsm9962769pfh.62.2023.10.11.05.09.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Oct 2023 05:09:31 -0700 (PDT) From: Chuyi Zhou To: bpf@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@kernel.org, tj@kernel.org, linux-kernel@vger.kernel.org, Chuyi Zhou Subject: [PATCH bpf-next v5 5/8] bpf: teach the verifier to enforce css_iter and task_iter in RCU CS Date: Wed, 11 Oct 2023 20:08:54 +0800 Message-Id: <20231011120857.251943-6-zhouchuyi@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20231011120857.251943-1-zhouchuyi@bytedance.com> References: <20231011120857.251943-1-zhouchuyi@bytedance.com> MIME-Version: 1.0 X-Spam-Status: No, score=2.7 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, RCVD_IN_SBL_CSS,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Wed, 11 Oct 2023 05:13:10 -0700 (PDT) X-Spam-Level: ** X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779461163494956700 X-GMAIL-MSGID: 1779461163494956700 css_iter and task_iter should be used in rcu section. Specifically, in sleepable progs explicit bpf_rcu_read_lock() is needed before use these iters. In normal bpf progs that have implicit rcu_read_lock(), it's OK to use them directly. This patch adds a new a KF flag KF_RCU_PROTECTED for bpf_iter_task_new and bpf_iter_css_new. It means the kfunc should be used in RCU CS. We check whether we are in rcu cs before we want to invoke this kfunc. If the rcu protection is guaranteed, we would let st->type = PTR_TO_STACK | MEM_RCU. Once user do rcu_unlock during the iteration, state MEM_RCU of regs would be cleared. is_iter_reg_valid_init() will reject if reg->type is UNTRUSTED. It is worth noting that currently, bpf_rcu_read_unlock does not clear the state of the STACK_ITER reg, since bpf_for_each_spilled_reg only considers STACK_SPILL. This patch also let bpf_for_each_spilled_reg search STACK_ITER. Signed-off-by: Chuyi Zhou Acked-by: Andrii Nakryiko --- include/linux/bpf_verifier.h | 19 ++++++++------ include/linux/btf.h | 1 + kernel/bpf/helpers.c | 4 +-- kernel/bpf/verifier.c | 50 ++++++++++++++++++++++++++++-------- 4 files changed, 53 insertions(+), 21 deletions(-) diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h index 94ec766432f5..e67cd45a85be 100644 --- a/include/linux/bpf_verifier.h +++ b/include/linux/bpf_verifier.h @@ -386,19 +386,18 @@ struct bpf_verifier_state { u32 jmp_history_cnt; }; -#define bpf_get_spilled_reg(slot, frame) \ +#define bpf_get_spilled_reg(slot, frame, mask) \ (((slot < frame->allocated_stack / BPF_REG_SIZE) && \ - (frame->stack[slot].slot_type[0] == STACK_SPILL)) \ + ((1 << frame->stack[slot].slot_type[0]) & (mask))) \ ? &frame->stack[slot].spilled_ptr : NULL) /* Iterate over 'frame', setting 'reg' to either NULL or a spilled register. */ -#define bpf_for_each_spilled_reg(iter, frame, reg) \ - for (iter = 0, reg = bpf_get_spilled_reg(iter, frame); \ +#define bpf_for_each_spilled_reg(iter, frame, reg, mask) \ + for (iter = 0, reg = bpf_get_spilled_reg(iter, frame, mask); \ iter < frame->allocated_stack / BPF_REG_SIZE; \ - iter++, reg = bpf_get_spilled_reg(iter, frame)) + iter++, reg = bpf_get_spilled_reg(iter, frame, mask)) -/* Invoke __expr over regsiters in __vst, setting __state and __reg */ -#define bpf_for_each_reg_in_vstate(__vst, __state, __reg, __expr) \ +#define bpf_for_each_reg_in_vstate_mask(__vst, __state, __reg, __mask, __expr) \ ({ \ struct bpf_verifier_state *___vstate = __vst; \ int ___i, ___j; \ @@ -410,7 +409,7 @@ struct bpf_verifier_state { __reg = &___regs[___j]; \ (void)(__expr); \ } \ - bpf_for_each_spilled_reg(___j, __state, __reg) { \ + bpf_for_each_spilled_reg(___j, __state, __reg, __mask) { \ if (!__reg) \ continue; \ (void)(__expr); \ @@ -418,6 +417,10 @@ struct bpf_verifier_state { } \ }) +/* Invoke __expr over regsiters in __vst, setting __state and __reg */ +#define bpf_for_each_reg_in_vstate(__vst, __state, __reg, __expr) \ + bpf_for_each_reg_in_vstate_mask(__vst, __state, __reg, 1 << STACK_SPILL, __expr) + /* linked list of verifier states used to prune search */ struct bpf_verifier_state_list { struct bpf_verifier_state state; diff --git a/include/linux/btf.h b/include/linux/btf.h index 928113a80a95..c2231c64d60b 100644 --- a/include/linux/btf.h +++ b/include/linux/btf.h @@ -74,6 +74,7 @@ #define KF_ITER_NEW (1 << 8) /* kfunc implements BPF iter constructor */ #define KF_ITER_NEXT (1 << 9) /* kfunc implements BPF iter next method */ #define KF_ITER_DESTROY (1 << 10) /* kfunc implements BPF iter destructor */ +#define KF_RCU_PROTECTED (1 << 11) /* kfunc should be protected by rcu cs when they are invoked */ /* * Tag marking a kernel function as a kfunc. This is meant to minimize the diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index 6330e37ca8d5..5081a4ac5b6d 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -2555,10 +2555,10 @@ BTF_ID_FLAGS(func, bpf_iter_num_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_iter_css_task_new, KF_ITER_NEW | KF_TRUSTED_ARGS) BTF_ID_FLAGS(func, bpf_iter_css_task_next, KF_ITER_NEXT | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_css_task_destroy, KF_ITER_DESTROY) -BTF_ID_FLAGS(func, bpf_iter_task_new, KF_ITER_NEW | KF_TRUSTED_ARGS) +BTF_ID_FLAGS(func, bpf_iter_task_new, KF_ITER_NEW | KF_TRUSTED_ARGS | KF_RCU_PROTECTED) BTF_ID_FLAGS(func, bpf_iter_task_next, KF_ITER_NEXT | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_task_destroy, KF_ITER_DESTROY) -BTF_ID_FLAGS(func, bpf_iter_css_new, KF_ITER_NEW | KF_TRUSTED_ARGS) +BTF_ID_FLAGS(func, bpf_iter_css_new, KF_ITER_NEW | KF_TRUSTED_ARGS | KF_RCU_PROTECTED) BTF_ID_FLAGS(func, bpf_iter_css_next, KF_ITER_NEXT | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_css_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_dynptr_adjust) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 528d375c17ee..3a60cc87520e 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -1173,7 +1173,12 @@ static bool is_dynptr_type_expected(struct bpf_verifier_env *env, struct bpf_reg static void __mark_reg_known_zero(struct bpf_reg_state *reg); +static bool in_rcu_cs(struct bpf_verifier_env *env); + +static bool is_kfunc_rcu_protected(struct bpf_kfunc_call_arg_meta *meta); + static int mark_stack_slots_iter(struct bpf_verifier_env *env, + struct bpf_kfunc_call_arg_meta *meta, struct bpf_reg_state *reg, int insn_idx, struct btf *btf, u32 btf_id, int nr_slots) { @@ -1194,6 +1199,12 @@ static int mark_stack_slots_iter(struct bpf_verifier_env *env, __mark_reg_known_zero(st); st->type = PTR_TO_STACK; /* we don't have dedicated reg type */ + if (is_kfunc_rcu_protected(meta)) { + if (in_rcu_cs(env)) + st->type |= MEM_RCU; + else + st->type |= PTR_UNTRUSTED; + } st->live |= REG_LIVE_WRITTEN; st->ref_obj_id = i == 0 ? id : 0; st->iter.btf = btf; @@ -1268,7 +1279,7 @@ static bool is_iter_reg_valid_uninit(struct bpf_verifier_env *env, return true; } -static bool is_iter_reg_valid_init(struct bpf_verifier_env *env, struct bpf_reg_state *reg, +static int is_iter_reg_valid_init(struct bpf_verifier_env *env, struct bpf_reg_state *reg, struct btf *btf, u32 btf_id, int nr_slots) { struct bpf_func_state *state = func(env, reg); @@ -1276,26 +1287,28 @@ static bool is_iter_reg_valid_init(struct bpf_verifier_env *env, struct bpf_reg_ spi = iter_get_spi(env, reg, nr_slots); if (spi < 0) - return false; + return -EINVAL; for (i = 0; i < nr_slots; i++) { struct bpf_stack_state *slot = &state->stack[spi - i]; struct bpf_reg_state *st = &slot->spilled_ptr; + if (st->type & PTR_UNTRUSTED) + return -EPROTO; /* only main (first) slot has ref_obj_id set */ if (i == 0 && !st->ref_obj_id) - return false; + return -EINVAL; if (i != 0 && st->ref_obj_id) - return false; + return -EINVAL; if (st->iter.btf != btf || st->iter.btf_id != btf_id) - return false; + return -EINVAL; for (j = 0; j < BPF_REG_SIZE; j++) if (slot->slot_type[j] != STACK_ITER) - return false; + return -EINVAL; } - return true; + return 0; } /* Check if given stack slot is "special": @@ -7618,15 +7631,24 @@ static int process_iter_arg(struct bpf_verifier_env *env, int regno, int insn_id return err; } - err = mark_stack_slots_iter(env, reg, insn_idx, meta->btf, btf_id, nr_slots); + err = mark_stack_slots_iter(env, meta, reg, insn_idx, meta->btf, btf_id, nr_slots); if (err) return err; } else { /* iter_next() or iter_destroy() expect initialized iter state*/ - if (!is_iter_reg_valid_init(env, reg, meta->btf, btf_id, nr_slots)) { + err = is_iter_reg_valid_init(env, reg, meta->btf, btf_id, nr_slots); + switch (err) { + case 0: + break; + case -EINVAL: verbose(env, "expected an initialized iter_%s as arg #%d\n", iter_type_str(meta->btf, btf_id), regno); - return -EINVAL; + return err; + case -EPROTO: + verbose(env, "expected an RCU CS when using %s\n", meta->func_name); + return err; + default: + return err; } spi = iter_get_spi(env, reg, nr_slots); @@ -10209,6 +10231,11 @@ static bool is_kfunc_rcu(struct bpf_kfunc_call_arg_meta *meta) return meta->kfunc_flags & KF_RCU; } +static bool is_kfunc_rcu_protected(struct bpf_kfunc_call_arg_meta *meta) +{ + return meta->kfunc_flags & KF_RCU_PROTECTED; +} + static bool __kfunc_param_match_suffix(const struct btf *btf, const struct btf_param *arg, const char *suffix) @@ -11560,6 +11587,7 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, if (env->cur_state->active_rcu_lock) { struct bpf_func_state *state; struct bpf_reg_state *reg; + u32 clear_mask = (1 << STACK_SPILL) | (1 << STACK_ITER); if (in_rbtree_lock_required_cb(env) && (rcu_lock || rcu_unlock)) { verbose(env, "Calling bpf_rcu_read_{lock,unlock} in unnecessary rbtree callback\n"); @@ -11570,7 +11598,7 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, verbose(env, "nested rcu read lock (kernel function %s)\n", func_name); return -EINVAL; } else if (rcu_unlock) { - bpf_for_each_reg_in_vstate(env->cur_state, state, reg, ({ + bpf_for_each_reg_in_vstate_mask(env->cur_state, state, reg, clear_mask, ({ if (reg->type & MEM_RCU) { reg->type &= ~(MEM_RCU | PTR_MAYBE_NULL); reg->type |= PTR_UNTRUSTED; From patchwork Wed Oct 11 12:08:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuyi Zhou X-Patchwork-Id: 151365 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp492019vqb; Wed, 11 Oct 2023 05:12:52 -0700 (PDT) X-Google-Smtp-Source: AGHT+IG5wwwR0grCeZM399CYOkPaM03RMopn6BvsQMUT4Bn+mkoI3CXjembWm1X09Va7NH6U7kOS X-Received: by 2002:a05:6a21:a587:b0:163:c167:964a with SMTP id gd7-20020a056a21a58700b00163c167964amr26320185pzc.1.1697026372235; Wed, 11 Oct 2023 05:12:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697026372; cv=none; d=google.com; s=arc-20160816; b=tNP35//+cIFlmscrma6olovh/xvZTYoOIpmlT6aRIevGSRqD2u5739gLSUI6qnv1z/ Gxynl0+SyPQiujcQNo7zuvleUyL+quJr31IHic524MmF8edoU/EJMqKHYg9nGI2anJjg g0TuObd2EN5b1Jw+isTIMEoKhLuJuYUYCxhRP0N7oFtFMgshnYg1m78+ZUu3wutknEVL 3bxCOfQixg3wmA3t8pslsNUESGrps16FKgBA/o16PUUJCso370J3AcbTJNRFqdbhlXId OpDxCLmoS3tDpCVGI0OTpmXs67QBvDnQiMnAF3TPeEBg+GT4Nc1fPCcFUddQbPtl/LoI QCFg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=OvtOLR/3l5/FirKuucsBQhHg+ePC25J/Wauwusn+tsY=; fh=DR7g1EcWKOXTEoooUPBSJXUaklSrDEYzv6YDdhz1CwE=; b=q/j+nDtJfZDPlBhbVk1VXoXgSJtP/Tx3XZ5baI08W0drh0v1zZqHRqt8zKHr1HaHCn QIB52Lyc0ybVEcRn6LDXY4vcKwd8xKyOWxFNXuq8Abxb+gcMTPQ2z3tCKBAe89x137XB 3E8GPbfM/XZBxpMZNQWSoz2tB5Ieoel1e3ocEHIjgo+ccWy43IElH6ayICn8YcFlWJhE IYc6l/+G3YB6AD7qEmiD0uOPddaYSiJjNBKoXZsCRfQXN8H8XkWaSgYd2SAws7IjO0Nq lmrjJeTKAfajyLWFFTyp1wly8ERI7UFwFQBjFHs75YPETgAi/mXy2DM1hWNUHNXH8rkI +Iqg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=M1H0DPw3; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from morse.vger.email (morse.vger.email. [23.128.96.31]) by mx.google.com with ESMTPS id f17-20020a631011000000b005637e28270fsi5372784pgl.243.2023.10.11.05.12.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Oct 2023 05:12:52 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) client-ip=23.128.96.31; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=M1H0DPw3; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by morse.vger.email (Postfix) with ESMTP id 78A4D836D622; Wed, 11 Oct 2023 05:12:47 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at morse.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346863AbjJKMLq (ORCPT + 18 others); Wed, 11 Oct 2023 08:11:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56742 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346923AbjJKMKj (ORCPT ); Wed, 11 Oct 2023 08:10:39 -0400 Received: from mail-ot1-x332.google.com (mail-ot1-x332.google.com [IPv6:2607:f8b0:4864:20::332]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 16127109 for ; Wed, 11 Oct 2023 05:09:49 -0700 (PDT) Received: by mail-ot1-x332.google.com with SMTP id 46e09a7af769-6c4bf619b57so4588856a34.1 for ; Wed, 11 Oct 2023 05:09:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1697026175; x=1697630975; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=OvtOLR/3l5/FirKuucsBQhHg+ePC25J/Wauwusn+tsY=; b=M1H0DPw3MpA19a7O02MHrJc+UyiAwU+KgsefNHBsHMQOveRmg6YoD+RXnPKSfLRWme vc+JbpGiyoAuS1d9csD9ctg9RBeOerKsybh3o8+idVJPTbAXTGhxMJO1Bbft6qLCN4nq kv5ZDjXroIinv4hpqidLqn7sGWQHV1Gv1XBaJ+i0FKd+jKhBERcFvAeKhSDZUML+1UN7 OnlvqF3/hS361IwM19/7L2RsvhiUGtzJsAAwNT7bgPbsxbIHJwfopfIeV4/ZvvxTFShD Wokj5ePoZ8oothX9MfhZ6Rnipr4lSavLFA9tCgWmLqrnAP3kmQMApQ7jr5j0DovQ1GLp UjcA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697026175; x=1697630975; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=OvtOLR/3l5/FirKuucsBQhHg+ePC25J/Wauwusn+tsY=; b=c1u5ewJbtnUBNz5xw/CIvCSMGfrf84cJXdAjdv+TzrYm9RFDA99hQrQ+rHalZioShT 6XXNa7Gh+w1DikzVihWgKbtYpsu1FwODmGSdeGCvbSEixsTjmNN8DV48Q6imwiTI37k3 ObiRhl4iSFXcV7fuU7yxRks2qJ5PdON7tVuMgy1bblS6T6Xe0Sqald3nwd1kgZ9o6C2h EV6WvvbF/rD3lVilTipsVSX6MjAozBckrFHjyO+KLhq2OSrdKAUIDvqyqwusXxt/g2i1 HuM1DEFO+D30sXIkQzMDhQt8L9iMeXiJq/3gTHWtUDLGGZUacmAMZ0Uw9HOZyTmfCEs2 c5WA== X-Gm-Message-State: AOJu0Yzp2YCqc/0JOLefRqn08qRxlTHvPF5z3LBYM7x2QgIuu1KWlM2X t4JyO4EliO/nKFc+zEVqaPKZ0Q== X-Received: by 2002:a05:6830:6685:b0:6c4:be02:635f with SMTP id cq5-20020a056830668500b006c4be02635fmr27430243otb.5.1697026175171; Wed, 11 Oct 2023 05:09:35 -0700 (PDT) Received: from n37-019-243.byted.org ([180.184.51.142]) by smtp.gmail.com with ESMTPSA id u13-20020a62ed0d000000b006930db1e6cfsm9962769pfh.62.2023.10.11.05.09.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Oct 2023 05:09:34 -0700 (PDT) From: Chuyi Zhou To: bpf@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@kernel.org, tj@kernel.org, linux-kernel@vger.kernel.org, Chuyi Zhou Subject: [PATCH bpf-next v5 6/8] bpf: Let bpf_iter_task_new accept null task ptr Date: Wed, 11 Oct 2023 20:08:55 +0800 Message-Id: <20231011120857.251943-7-zhouchuyi@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20231011120857.251943-1-zhouchuyi@bytedance.com> References: <20231011120857.251943-1-zhouchuyi@bytedance.com> MIME-Version: 1.0 X-Spam-Status: No, score=2.7 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, RCVD_IN_SBL_CSS,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on morse.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (morse.vger.email [0.0.0.0]); Wed, 11 Oct 2023 05:12:47 -0700 (PDT) X-Spam-Level: ** X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779461125546148394 X-GMAIL-MSGID: 1779461125546148394 When using task_iter to iterate all threads of a specific task, we enforce that the user must pass a valid task pointer to ensure safety. However, when iterating all threads/process in the system, BPF verifier still require a valid ptr instead of "nullable" pointer, even though it's pointless, which is a kind of surprising from usability standpoint. It would be nice if we could let that kfunc accept a explicit null pointer when we are using BPF_TASK_ITER_ALL_{PROCS, THREADS} and a valid pointer when using BPF_TASK_ITER_THREAD. Given a trival kfunc: __bpf_kfunc void FN(struct TYPE_A *obj); BPF Prog would reject a nullptr for obj. The error info is: "arg#x pointer type xx xx must point to scalar, or struct with scalar" reported by get_kfunc_ptr_arg_type(). The reg->type is SCALAR_VALUE and the btf type of ref_t is not scalar or scalar_struct which leads to the rejection of get_kfunc_ptr_arg_type. This patch add "__nullable" annotation: __bpf_kfunc void FN(struct TYPE_A *obj__nullable); Here __nullable indicates obj can be optional, user can pass a explicit nullptr or a normal TYPE_A pointer. In get_kfunc_ptr_arg_type(), we will detect whether the current arg is optional and register is null, If so, return a new kfunc_ptr_arg_type KF_ARG_PTR_TO_NULL and skip to the next arg in check_kfunc_args(). Signed-off-by: Chuyi Zhou Acked-by: Andrii Nakryiko --- kernel/bpf/task_iter.c | 7 +++++-- kernel/bpf/verifier.c | 13 ++++++++++++- 2 files changed, 17 insertions(+), 3 deletions(-) diff --git a/kernel/bpf/task_iter.c b/kernel/bpf/task_iter.c index caeddad3d2f1..0772545568f1 100644 --- a/kernel/bpf/task_iter.c +++ b/kernel/bpf/task_iter.c @@ -873,7 +873,7 @@ enum { }; __bpf_kfunc int bpf_iter_task_new(struct bpf_iter_task *it, - struct task_struct *task, unsigned int flags) + struct task_struct *task__nullable, unsigned int flags) { struct bpf_iter_task_kern *kit = (void *)it; @@ -885,14 +885,17 @@ __bpf_kfunc int bpf_iter_task_new(struct bpf_iter_task *it, switch (flags) { case BPF_TASK_ITER_ALL_THREADS: case BPF_TASK_ITER_ALL_PROCS: + break; case BPF_TASK_ITER_PROC_THREADS: + if (!task__nullable) + return -EINVAL; break; default: return -EINVAL; } if (flags == BPF_TASK_ITER_PROC_THREADS) - kit->task = task; + kit->task = task__nullable; else kit->task = &init_task; kit->pos = kit->task; diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 3a60cc87520e..d09697dbfd9c 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -10310,6 +10310,11 @@ static bool is_kfunc_arg_refcounted_kptr(const struct btf *btf, const struct btf return __kfunc_param_match_suffix(btf, arg, "__refcounted_kptr"); } +static bool is_kfunc_arg_nullable(const struct btf *btf, const struct btf_param *arg) +{ + return __kfunc_param_match_suffix(btf, arg, "__nullable"); +} + static bool is_kfunc_arg_scalar_with_name(const struct btf *btf, const struct btf_param *arg, const char *name) @@ -10452,6 +10457,7 @@ enum kfunc_ptr_arg_type { KF_ARG_PTR_TO_CALLBACK, KF_ARG_PTR_TO_RB_ROOT, KF_ARG_PTR_TO_RB_NODE, + KF_ARG_PTR_TO_NULL, }; enum special_kfunc_type { @@ -10608,6 +10614,8 @@ get_kfunc_ptr_arg_type(struct bpf_verifier_env *env, if (is_kfunc_arg_callback(env, meta->btf, &args[argno])) return KF_ARG_PTR_TO_CALLBACK; + if (is_kfunc_arg_nullable(meta->btf, &args[argno]) && register_is_null(reg)) + return KF_ARG_PTR_TO_NULL; if (argno + 1 < nargs && (is_kfunc_arg_mem_size(meta->btf, &args[argno + 1], ®s[regno + 1]) || @@ -11158,7 +11166,8 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ } if ((is_kfunc_trusted_args(meta) || is_kfunc_rcu(meta)) && - (register_is_null(reg) || type_may_be_null(reg->type))) { + (register_is_null(reg) || type_may_be_null(reg->type)) && + !is_kfunc_arg_nullable(meta->btf, &args[i])) { verbose(env, "Possibly NULL pointer passed to trusted arg%d\n", i); return -EACCES; } @@ -11183,6 +11192,8 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ return kf_arg_type; switch (kf_arg_type) { + case KF_ARG_PTR_TO_NULL: + continue; case KF_ARG_PTR_TO_ALLOC_BTF_ID: case KF_ARG_PTR_TO_BTF_ID: if (!is_kfunc_trusted_args(meta) && !is_kfunc_rcu(meta)) From patchwork Wed Oct 11 12:08:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuyi Zhou X-Patchwork-Id: 151373 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp492353vqb; Wed, 11 Oct 2023 05:13:24 -0700 (PDT) X-Google-Smtp-Source: AGHT+IG/QnugQIacbw4PVejeqUecRkY054YdISSC+v2N3uao/uA8dPTkaHbrMfdMpKts7+oNIsSn X-Received: by 2002:a17:902:dacd:b0:1b3:d8ac:8db3 with SMTP id q13-20020a170902dacd00b001b3d8ac8db3mr23387464plx.6.1697026403848; Wed, 11 Oct 2023 05:13:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697026403; cv=none; d=google.com; s=arc-20160816; b=QImDDycggLpQCkVeDex+2ojuYL+Ub41ZJtoxDyEpq1SFkMTLbgMMeSM9yaxd/OHQna fZA8r2O2XDzvys3Ab/kdRqPMy39ILKjRZiCTaitn8ka4NtnNUx4i88p+gF1D4FuAwRdn fVkc7701ntdSe/pFOJzaNmc2ittM0RuniV21wl/L9KWlw632le6jV2caMgkeWbITsW9W 1dhp+7DSeTBUT5L2NpyaO9PLCr8FdJ9xOJarfWknfy7MJcnaqmN9gXrzsPFjGLZWbcXS z5pvptcWiW+Nl5A0q6i8czAwrYVhGCE+Zzu4R1tZO2Zi66L8SECGBEOFqlAv+St5hNvD 3jYg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=aEInOcrGpkkZm6SAXLwwS2bVGzjJWfEHqZIWggTzZdM=; fh=DR7g1EcWKOXTEoooUPBSJXUaklSrDEYzv6YDdhz1CwE=; b=kXjZ1EdmACRNcRUcg0vbLOEsT+pYkfbqzo2tzhACkzKMCmM+PC1y6SisqZLDJLZ1eu 7KCCfo43540QNHiMBcBfJY7Qa0SyGjN9WKiszK1ZeZlMckdwBhSXEZuoW12HB1MjHVHC 1ybAToKmorvJ+JPw+SNAFr/rHcxjMzkWmPNzyBubZyiarQo1pcQuLbx9lmFY4QsoQ3To 0sI2Wx7h+waD6uzN5nDwZLjZQGePMK5rZIrKHUiMq4vFFs8AIqorzqib0upcW7fZMEzk kO6QQLgfdedn/1OqMg4511QPCQOhUZvvC1LTjjLdPhmnM7/qgP7cdevvSKfpSGhRI9Ht rqSw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b="NyCg/sEA"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from morse.vger.email (morse.vger.email. [23.128.96.31]) by mx.google.com with ESMTPS id l15-20020a170902f68f00b001c9b129a658si775285plg.122.2023.10.11.05.13.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Oct 2023 05:13:23 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) client-ip=23.128.96.31; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b="NyCg/sEA"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by morse.vger.email (Postfix) with ESMTP id 2468A836D62B; Wed, 11 Oct 2023 05:13:18 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at morse.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346730AbjJKMMx (ORCPT + 18 others); Wed, 11 Oct 2023 08:12:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38110 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346857AbjJKMMj (ORCPT ); Wed, 11 Oct 2023 08:12:39 -0400 Received: from mail-ot1-x32b.google.com (mail-ot1-x32b.google.com [IPv6:2607:f8b0:4864:20::32b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CC75110CB for ; Wed, 11 Oct 2023 05:09:55 -0700 (PDT) Received: by mail-ot1-x32b.google.com with SMTP id 46e09a7af769-6c64a3c4912so4760911a34.3 for ; Wed, 11 Oct 2023 05:09:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1697026178; x=1697630978; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=aEInOcrGpkkZm6SAXLwwS2bVGzjJWfEHqZIWggTzZdM=; b=NyCg/sEAhkgP6TXyPML7Nax8Il/sML12zlxaK/qC8giHbkI5CfSIOuibjrb+pWCz/D WFENhPPlipvvDODIrmad2gmdCgYgfvD9PE2XUwMQN+XPkeehu3P4zQ8fmLYiwIINRTs1 dnd07N7zX5M2B234H5csxFKKDNFtDvJdKkQlYKpQBXUuvE0ELpy9VptajXV0LeEaomPd DJoB80W0Q95NuQYlKa+XBsIRuh6yw9yYbvFpZHorJ/6Ixx2h/r0Z281367/ETWispVox 0pz7K0+P6TR7C6a8IpVfGoctJoNEGytFgwwX2KNX2TMOa4JLyq2hvVc9vwr484DzMCW7 Uamw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697026178; x=1697630978; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=aEInOcrGpkkZm6SAXLwwS2bVGzjJWfEHqZIWggTzZdM=; b=u4B62lrV69ca55/mb5VaFnkFzA7vLpqOFR8Wn2fLXIkzKQ9aQxPsOYPFdTT6iZHUKc Yx+G1D1EqOGY5KzjCXVzCTpqAmT4FbCk7dXfHLeMZGFLlP1XySDwWEntYe1+mWbEf+TD aF4k5rbCx8rFcDq9OIEYG+4yam+bgDXDfLIAzy9oPP8rCSLxxf5saVv/rkWuzaSKFnrN KNvdkS6DltTbcejpYXjLRfh4es790+V1vtMZiite+p7C07qKyulFb6mjhHj7g/P4yTS6 0Jgkdc8eEcZ0aF7IHhLob7zTybKqq5JgEemlDrLRjyhBopq46/KFkkskcqRV5sADmWzs FUcA== X-Gm-Message-State: AOJu0YxEVpMOub5IFpPMfpBTYy0pwb6CfZqv0Qgro9YWcOwUazGC3Vw+ G5AoD0td9ZIx2HcTN7QuEoYIpg== X-Received: by 2002:a9d:75da:0:b0:6bd:af4:274d with SMTP id c26-20020a9d75da000000b006bd0af4274dmr23662697otl.8.1697026178764; Wed, 11 Oct 2023 05:09:38 -0700 (PDT) Received: from n37-019-243.byted.org ([180.184.51.142]) by smtp.gmail.com with ESMTPSA id u13-20020a62ed0d000000b006930db1e6cfsm9962769pfh.62.2023.10.11.05.09.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Oct 2023 05:09:38 -0700 (PDT) From: Chuyi Zhou To: bpf@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@kernel.org, tj@kernel.org, linux-kernel@vger.kernel.org, Chuyi Zhou Subject: [PATCH bpf-next v5 7/8] selftests/bpf: rename bpf_iter_task.c to bpf_iter_tasks.c Date: Wed, 11 Oct 2023 20:08:56 +0800 Message-Id: <20231011120857.251943-8-zhouchuyi@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20231011120857.251943-1-zhouchuyi@bytedance.com> References: <20231011120857.251943-1-zhouchuyi@bytedance.com> MIME-Version: 1.0 X-Spam-Status: No, score=2.7 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, RCVD_IN_SBL_CSS,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on morse.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (morse.vger.email [0.0.0.0]); Wed, 11 Oct 2023 05:13:18 -0700 (PDT) X-Spam-Level: ** X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779461158343617630 X-GMAIL-MSGID: 1779461158343617630 The newly-added struct bpf_iter_task has a name collision with a selftest for the seq_file task iter's bpf skel, so the selftests/bpf/progs file is renamed in order to avoid the collision. Signed-off-by: Chuyi Zhou Acked-by: Andrii Nakryiko --- .../selftests/bpf/prog_tests/bpf_iter.c | 18 +++++++++--------- .../{bpf_iter_task.c => bpf_iter_tasks.c} | 0 2 files changed, 9 insertions(+), 9 deletions(-) rename tools/testing/selftests/bpf/progs/{bpf_iter_task.c => bpf_iter_tasks.c} (100%) diff --git a/tools/testing/selftests/bpf/prog_tests/bpf_iter.c b/tools/testing/selftests/bpf/prog_tests/bpf_iter.c index 1f02168103dd..dc60e8e125cd 100644 --- a/tools/testing/selftests/bpf/prog_tests/bpf_iter.c +++ b/tools/testing/selftests/bpf/prog_tests/bpf_iter.c @@ -7,7 +7,7 @@ #include "bpf_iter_ipv6_route.skel.h" #include "bpf_iter_netlink.skel.h" #include "bpf_iter_bpf_map.skel.h" -#include "bpf_iter_task.skel.h" +#include "bpf_iter_tasks.skel.h" #include "bpf_iter_task_stack.skel.h" #include "bpf_iter_task_file.skel.h" #include "bpf_iter_task_vma.skel.h" @@ -215,12 +215,12 @@ static void *do_nothing_wait(void *arg) static void test_task_common_nocheck(struct bpf_iter_attach_opts *opts, int *num_unknown, int *num_known) { - struct bpf_iter_task *skel; + struct bpf_iter_tasks *skel; pthread_t thread_id; void *ret; - skel = bpf_iter_task__open_and_load(); - if (!ASSERT_OK_PTR(skel, "bpf_iter_task__open_and_load")) + skel = bpf_iter_tasks__open_and_load(); + if (!ASSERT_OK_PTR(skel, "bpf_iter_tasks__open_and_load")) return; ASSERT_OK(pthread_mutex_lock(&do_nothing_mutex), "pthread_mutex_lock"); @@ -239,7 +239,7 @@ static void test_task_common_nocheck(struct bpf_iter_attach_opts *opts, ASSERT_FALSE(pthread_join(thread_id, &ret) || ret != NULL, "pthread_join"); - bpf_iter_task__destroy(skel); + bpf_iter_tasks__destroy(skel); } static void test_task_common(struct bpf_iter_attach_opts *opts, int num_unknown, int num_known) @@ -307,10 +307,10 @@ static void test_task_pidfd(void) static void test_task_sleepable(void) { - struct bpf_iter_task *skel; + struct bpf_iter_tasks *skel; - skel = bpf_iter_task__open_and_load(); - if (!ASSERT_OK_PTR(skel, "bpf_iter_task__open_and_load")) + skel = bpf_iter_tasks__open_and_load(); + if (!ASSERT_OK_PTR(skel, "bpf_iter_tasks__open_and_load")) return; do_dummy_read(skel->progs.dump_task_sleepable); @@ -320,7 +320,7 @@ static void test_task_sleepable(void) ASSERT_GT(skel->bss->num_success_copy_from_user_task, 0, "num_success_copy_from_user_task"); - bpf_iter_task__destroy(skel); + bpf_iter_tasks__destroy(skel); } static void test_task_stack(void) diff --git a/tools/testing/selftests/bpf/progs/bpf_iter_task.c b/tools/testing/selftests/bpf/progs/bpf_iter_tasks.c similarity index 100% rename from tools/testing/selftests/bpf/progs/bpf_iter_task.c rename to tools/testing/selftests/bpf/progs/bpf_iter_tasks.c From patchwork Wed Oct 11 12:08:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuyi Zhou X-Patchwork-Id: 151385 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp497923vqb; Wed, 11 Oct 2023 05:22:13 -0700 (PDT) X-Google-Smtp-Source: AGHT+IECtJTeZOnnjEQMQ25uFeiJ5U0HmMU1ruDBhq2V+ffzHEEeSwJQ03cv7MxomnQ+OL6JyjVj X-Received: by 2002:a05:6a20:8f01:b0:15a:2c0b:6c81 with SMTP id b1-20020a056a208f0100b0015a2c0b6c81mr25226225pzk.3.1697026933368; Wed, 11 Oct 2023 05:22:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697026933; cv=none; d=google.com; s=arc-20160816; b=tKqDfk04VBleD+EmD0pTwa/FGI8kUYYre/7qK0svBZUX0cuulxwV/OocHPcSSVeGd5 x9aKn6uU3yU4zuI/xaF5muC1d0PkMynwHFDs4cMRoPK/VSQP2kzjTL2Rke8zWropzPIM J8LGQPI2mtLZlB0iaOG0VZktiq8sNkGaYsb11WL7GCr2xJdMZsF4mr7Id47DkZQQjNiz /Apbdzhx2bOtxJX+fYNjXDu8OSZ7mVRQ1gWFSFUWa7MFXJjVbSIknrMU7rf0Y0aRrGC/ AlqpRPON3r8zMxSiTxAzaGGC9ifSOrSVUb8j7BUb8AO0H1Gsv4fgCrJDALHf/DatvJJL 8BZw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=nkM5A3bvgTaWcD7gBVleYNsoAnjxjxJ3YOdT/AtlgMM=; fh=DR7g1EcWKOXTEoooUPBSJXUaklSrDEYzv6YDdhz1CwE=; b=0o4uogW839OjPLpzfTuEqniEd6YB/iE4GEkRf+k+xCO8gUEFOvh/XOu3hY9Ehdqujo RAYudziifBW0cH8DOZ8pIM+CRIr67VjhHCHWDIvmLJN2wfOmOV+IhRrWL/sQ0c5UwIFF hVNpoz9I1iU0zp++lGelD5HnE6QBUDg87AmumhHWVYVpsRpldkPcydj0yT7tjKTa1hT7 ZZOgpbyE/1w84Ws0FjTJiYJg/TFgk/pTxW2SrL9/y8ELPQHmi9OnGc0UGd3zgmYq7lRm LkGiNlBL7jZBM2/0HrnY+S67UqiktrzNkfkXSGqv1rKCTR+L+KhONRx97axE55FZpo3o OI7g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=SqJABcBc; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from agentk.vger.email (agentk.vger.email. [23.128.96.32]) by mx.google.com with ESMTPS id lk6-20020a17090b33c600b00278ff770796si12103914pjb.88.2023.10.11.05.22.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Oct 2023 05:22:13 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) client-ip=23.128.96.32; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=SqJABcBc; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id 0A5F780A7CEF; Wed, 11 Oct 2023 05:22:10 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346135AbjJKMVr (ORCPT + 18 others); Wed, 11 Oct 2023 08:21:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46568 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346781AbjJKMU4 (ORCPT ); Wed, 11 Oct 2023 08:20:56 -0400 Received: from mail-oo1-xc34.google.com (mail-oo1-xc34.google.com [IPv6:2607:f8b0:4864:20::c34]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 30F8F170F for ; Wed, 11 Oct 2023 05:10:05 -0700 (PDT) Received: by mail-oo1-xc34.google.com with SMTP id 006d021491bc7-57bce354d94so3334279eaf.2 for ; Wed, 11 Oct 2023 05:10:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1697026183; x=1697630983; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=nkM5A3bvgTaWcD7gBVleYNsoAnjxjxJ3YOdT/AtlgMM=; b=SqJABcBcon8jwMk83neTKstQze+XerQ2bz5JpslL8DedbDcn+JvGAXvcv3QjOpn8D5 Vliv/BgSpOJ1PFmOViJLkBq9QdozyXZux45UaZKiuWw3w+vG4UUqGZCtBpRk6G2r+wv1 +9BNc8dnsnyDCO3//FdZTXUYsBNpsFdkww2gldOpdh1E/KIRxVWUJknLsGbR03Q/YMzG e2M0k/I1uP5Jr35K6lqtUHSHINqR8xuNRGOXbSifNLG4OpTLoXIVVWcgGK9bSx4vVDMm T8SYD25UqNiVL66eLstTmIKBW+ZzgtBA96u+jFm4gxkmkDL9Gj9fdZ4M+V/FB32d2pH4 Hi7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697026183; x=1697630983; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nkM5A3bvgTaWcD7gBVleYNsoAnjxjxJ3YOdT/AtlgMM=; b=SdyxNYkxp3GzK0YC/rzGRPjkUDuqCtiYshTn0864WPjyfta9QeAIwLLC+QoGx7Fhff gCRLQRogzNNyyOs5zPivNykRfVwFuIJnbkuCPCMgMqSpG0xCySEyQC41HP0r9KqH6F+T IX9c/YAXVjEEKont8WZkAo+8/OGSRLie3YIJ6OyUgz6LJxAStbzA/MNDnQGdG+aEFU4t AeblgD+ZeHsrEQ25/VrQQri/y/omunxRRs/8ERNXCO+EkeJ8XTXxpSXvTmDjpnMPI9U1 KOdzCQPo7JYsjhmEVyc0GHqPEjvd191OPM0JOLY84a40Vlse3gOJL+inKRg+OzlXF7k7 /bDA== X-Gm-Message-State: AOJu0YzxW0E2+G9hOjpW80gvWOmWcIzwi3/O8MMwY88mPjHJX62j6hga +KjprKNbRaIEa0bECiVW+8oRtQ== X-Received: by 2002:a05:6358:618c:b0:145:6560:d86d with SMTP id w12-20020a056358618c00b001456560d86dmr14328775rww.32.1697026182661; Wed, 11 Oct 2023 05:09:42 -0700 (PDT) Received: from n37-019-243.byted.org ([180.184.51.142]) by smtp.gmail.com with ESMTPSA id u13-20020a62ed0d000000b006930db1e6cfsm9962769pfh.62.2023.10.11.05.09.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Oct 2023 05:09:42 -0700 (PDT) From: Chuyi Zhou To: bpf@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@kernel.org, tj@kernel.org, linux-kernel@vger.kernel.org, Chuyi Zhou Subject: [PATCH bpf-next v5 8/8] selftests/bpf: Add tests for open-coded task and css iter Date: Wed, 11 Oct 2023 20:08:57 +0800 Message-Id: <20231011120857.251943-9-zhouchuyi@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20231011120857.251943-1-zhouchuyi@bytedance.com> References: <20231011120857.251943-1-zhouchuyi@bytedance.com> MIME-Version: 1.0 X-Spam-Status: No, score=2.7 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, RCVD_IN_SBL_CSS,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Wed, 11 Oct 2023 05:22:10 -0700 (PDT) X-Spam-Level: ** X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779461713879493807 X-GMAIL-MSGID: 1779461713879493807 This patch adds three subtests to demonstrate these patterns and validating correctness. subtest1: 1) We use task_iter to iterate all process in the system and search for the current process with a given pid. 2) We create some threads in current process context, and use BPF_TASK_ITER_PROC_THREADS to iterate all threads of current process. As expected, we would find all the threads of current process. 3) We create some threads and use BPF_TASK_ITER_ALL_THREADS to iterate all threads in the system. As expected, we would find all the threads which was created. subtest2: We create a cgroup and add the current task to the cgroup. In the BPF program, we would use bpf_for_each(css_task, task, css) to iterate all tasks under the cgroup. As expected, we would find the current process. subtest3: 1) We create a cgroup tree. In the BPF program, we use bpf_for_each(css, pos, root, XXX) to iterate all descendant under the root with pre and post order. As expected, we would find all descendant and the last iterating cgroup in post-order is root cgroup, the first iterating cgroup in pre-order is root cgroup. 2) We wse BPF_CGROUP_ITER_ANCESTORS_UP to traverse the cgroup tree starting from leaf and root separately, and record the height. The diff of the hights would be the total tree-high - 1. Signed-off-by: Chuyi Zhou --- .../testing/selftests/bpf/prog_tests/iters.c | 161 ++++++++++++++++++ tools/testing/selftests/bpf/progs/iters_css.c | 74 ++++++++ .../selftests/bpf/progs/iters_css_task.c | 42 +++++ .../testing/selftests/bpf/progs/iters_task.c | 41 +++++ .../selftests/bpf/progs/iters_task_failure.c | 105 ++++++++++++ 5 files changed, 423 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/iters_css.c create mode 100644 tools/testing/selftests/bpf/progs/iters_css_task.c create mode 100644 tools/testing/selftests/bpf/progs/iters_task.c create mode 100644 tools/testing/selftests/bpf/progs/iters_task_failure.c diff --git a/tools/testing/selftests/bpf/prog_tests/iters.c b/tools/testing/selftests/bpf/prog_tests/iters.c index 10804ae5ae97..8d7a7bef5c73 100644 --- a/tools/testing/selftests/bpf/prog_tests/iters.c +++ b/tools/testing/selftests/bpf/prog_tests/iters.c @@ -1,13 +1,24 @@ // SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2023 Meta Platforms, Inc. and affiliates. */ +#include +#include +#include +#include +#include +#include #include +#include "cgroup_helpers.h" #include "iters.skel.h" #include "iters_state_safety.skel.h" #include "iters_looping.skel.h" #include "iters_num.skel.h" #include "iters_testmod_seq.skel.h" +#include "iters_task.skel.h" +#include "iters_css_task.skel.h" +#include "iters_css.skel.h" +#include "iters_task_failure.skel.h" static void subtest_num_iters(void) { @@ -90,6 +101,149 @@ static void subtest_testmod_seq_iters(void) iters_testmod_seq__destroy(skel); } +static pthread_mutex_t do_nothing_mutex; + +static void *do_nothing_wait(void *arg) +{ + pthread_mutex_lock(&do_nothing_mutex); + pthread_mutex_unlock(&do_nothing_mutex); + + pthread_exit(arg); +} + +#define thread_num 2 + +static void subtest_task_iters(void) +{ + struct iters_task *skel = NULL; + pthread_t thread_ids[thread_num]; + void *ret; + int err; + + skel = iters_task__open(); + if (!ASSERT_OK_PTR(skel, "skel_open")) + goto cleanup; + err = iters_task__load(skel); + if (!ASSERT_OK(err, "skel_load")) + goto cleanup; + skel->bss->target_pid = getpid(); + err = iters_task__attach(skel); + if (!ASSERT_OK(err, "iters_task__attach")) + goto cleanup; + pthread_mutex_lock(&do_nothing_mutex); + for (int i = 0; i < thread_num; i++) + ASSERT_OK(pthread_create(&thread_ids[i], NULL, &do_nothing_wait, NULL), + "pthread_create"); + + syscall(SYS_getpgid); + iters_task__detach(skel); + ASSERT_EQ(skel->bss->process_cnt, 1, "process_cnt"); + ASSERT_EQ(skel->bss->thread_cnt, thread_num + 1, "thread_cnt"); + ASSERT_EQ(skel->bss->all_thread_cnt, thread_num + 1, "all_thread_cnt"); + pthread_mutex_unlock(&do_nothing_mutex); + for (int i = 0; i < thread_num; i++) + pthread_join(thread_ids[i], &ret); +cleanup: + iters_task__destroy(skel); +} + +extern int stack_mprotect(void); + +static void subtest_css_task_iters(void) +{ + struct iters_css_task *skel = NULL; + int err, cg_fd, cg_id; + const char *cgrp_path = "/cg1"; + + err = setup_cgroup_environment(); + if (!ASSERT_OK(err, "setup_cgroup_environment")) + goto cleanup; + cg_fd = create_and_get_cgroup(cgrp_path); + if (!ASSERT_GE(cg_fd, 0, "cg_create")) + goto cleanup; + cg_id = get_cgroup_id(cgrp_path); + err = join_cgroup(cgrp_path); + if (!ASSERT_OK(err, "setup_cgroup_environment")) + goto cleanup; + + skel = iters_css_task__open(); + if (!ASSERT_OK_PTR(skel, "skel_open")) + goto cleanup; + + err = iters_css_task__load(skel); + if (!ASSERT_OK(err, "skel_load")) + goto cleanup; + + skel->bss->target_pid = getpid(); + skel->bss->cg_id = cg_id; + err = iters_css_task__attach(skel); + + err = stack_mprotect(); + if (!ASSERT_OK(err, "iters_task__attach")) + goto cleanup; + + iters_css_task__detach(skel); + ASSERT_EQ(skel->bss->css_task_cnt, 1, "css_task_cnt"); + +cleanup: + cleanup_cgroup_environment(); + iters_css_task__destroy(skel); +} + +static void subtest_css_iters(void) +{ + struct iters_css *skel = NULL; + struct { + const char *path; + int fd; + } cgs[] = { + { "/cg1" }, + { "/cg1/cg2" }, + { "/cg1/cg2/cg3" }, + { "/cg1/cg2/cg3/cg4" }, + }; + int err, cg_nr = ARRAY_SIZE(cgs); + int i; + + err = setup_cgroup_environment(); + if (!ASSERT_OK(err, "setup_cgroup_environment")) + goto cleanup; + for (i = 0; i < cg_nr; i++) { + cgs[i].fd = create_and_get_cgroup(cgs[i].path); + if (!ASSERT_GE(cgs[i].fd, 0, "cg_create")) + goto cleanup; + } + + skel = iters_css__open(); + if (!ASSERT_OK_PTR(skel, "skel_open")) + goto cleanup; + err = iters_css__load(skel); + if (!ASSERT_OK(err, "skel_load")) + goto cleanup; + + skel->bss->target_pid = getpid(); + skel->bss->root_cg_id = get_cgroup_id(cgs[0].path); + skel->bss->leaf_cg_id = get_cgroup_id(cgs[cg_nr - 1].path); + err = iters_css__attach(skel); + + if (!ASSERT_OK(err, "iters_task__attach")) + goto cleanup; + + syscall(SYS_getpgid); + ASSERT_EQ(skel->bss->pre_css_dec_cnt, cg_nr, "pre order search dec count"); + ASSERT_EQ(skel->bss->first_cg_id, get_cgroup_id(cgs[0].path), + "pre order search first cgroup id"); + + ASSERT_EQ(skel->bss->post_css_dec_cnt, cg_nr, "post order search dec count"); + ASSERT_EQ(skel->bss->last_cg_id, get_cgroup_id(cgs[0].path), + "post order search last cgroup id"); + ASSERT_EQ(skel->bss->tree_high, cg_nr - 1, "tree high"); + iters_css__detach(skel); +cleanup: + cleanup_cgroup_environment(); + iters_css__destroy(skel); +} + void test_iters(void) { RUN_TESTS(iters_state_safety); @@ -103,4 +257,11 @@ void test_iters(void) subtest_num_iters(); if (test__start_subtest("testmod_seq")) subtest_testmod_seq_iters(); + if (test__start_subtest("task")) + subtest_task_iters(); + if (test__start_subtest("css_task")) + subtest_css_task_iters(); + if (test__start_subtest("css")) + subtest_css_iters(); + RUN_TESTS(iters_task_failure); } diff --git a/tools/testing/selftests/bpf/progs/iters_css.c b/tools/testing/selftests/bpf/progs/iters_css.c new file mode 100644 index 000000000000..1422a7956c44 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/iters_css.c @@ -0,0 +1,74 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (C) 2023 Chuyi Zhou */ + +#include "vmlinux.h" +#include +#include +#include "bpf_misc.h" +#include "bpf_experimental.h" + +char _license[] SEC("license") = "GPL"; + +pid_t target_pid = 0; +u64 root_cg_id; +u64 leaf_cg_id; + +u64 last_cg_id = 0; +u64 first_cg_id = 0; + +int post_css_dec_cnt = 0; +int pre_css_dec_cnt = 0; +int tree_high = 0; + +struct cgroup *bpf_cgroup_from_id(u64 cgid) __ksym; +void bpf_cgroup_release(struct cgroup *p) __ksym; +void bpf_rcu_read_lock(void) __ksym; +void bpf_rcu_read_unlock(void) __ksym; + +SEC("fentry.s/" SYS_PREFIX "sys_getpgid") +int iter_css_for_each(const void *ctx) +{ + struct task_struct *cur_task = bpf_get_current_task_btf(); + struct cgroup_subsys_state *root_css, *leaf_css, *pos; + struct cgroup *root_cgrp, *leaf_cgrp, *cur_cgrp; + + if (cur_task->pid != target_pid) + return 0; + + root_cgrp = bpf_cgroup_from_id(root_cg_id); + + if (!root_cgrp) + return 0; + + leaf_cgrp = bpf_cgroup_from_id(leaf_cg_id); + + if (!leaf_cgrp) { + bpf_cgroup_release(root_cgrp); + return 0; + } + root_css = &root_cgrp->self; + leaf_css = &leaf_cgrp->self; + bpf_rcu_read_lock(); + bpf_for_each(css, pos, root_css, BPF_CGROUP_ITER_DESCENDANTS_POST) { + cur_cgrp = pos->cgroup; + post_css_dec_cnt += 1; + last_cg_id = cur_cgrp->kn->id; + } + + bpf_for_each(css, pos, root_css, BPF_CGROUP_ITER_DESCENDANTS_PRE) { + cur_cgrp = pos->cgroup; + pre_css_dec_cnt += 1; + if (!first_cg_id) + first_cg_id = cur_cgrp->kn->id; + } + + bpf_for_each(css, pos, leaf_css, BPF_CGROUP_ITER_ANCESTORS_UP) + tree_high += 1; + + bpf_for_each(css, pos, root_css, BPF_CGROUP_ITER_ANCESTORS_UP) + tree_high -= 1; + bpf_rcu_read_unlock(); + bpf_cgroup_release(root_cgrp); + bpf_cgroup_release(leaf_cgrp); + return 0; +} diff --git a/tools/testing/selftests/bpf/progs/iters_css_task.c b/tools/testing/selftests/bpf/progs/iters_css_task.c new file mode 100644 index 000000000000..506a2755234e --- /dev/null +++ b/tools/testing/selftests/bpf/progs/iters_css_task.c @@ -0,0 +1,42 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (C) 2023 Chuyi Zhou */ + +#include "vmlinux.h" +#include +#include +#include "bpf_misc.h" +#include "bpf_experimental.h" + +char _license[] SEC("license") = "GPL"; + +struct cgroup *bpf_cgroup_from_id(u64 cgid) __ksym; +void bpf_cgroup_release(struct cgroup *p) __ksym; + +pid_t target_pid = 0; +int css_task_cnt = 0; +u64 cg_id; + +SEC("lsm/file_mprotect") +int BPF_PROG(iter_css_task_for_each) +{ + struct task_struct *cur_task = bpf_get_current_task_btf(); + struct cgroup_subsys_state *css; + struct task_struct *task; + struct cgroup *cgrp; + + if (cur_task->pid != target_pid) + return 0; + + cgrp = bpf_cgroup_from_id(cg_id); + + if (cgrp == NULL) + return 0; + css = &cgrp->self; + + bpf_for_each(css_task, task, css, CSS_TASK_ITER_PROCS) + if (task->pid == target_pid) + css_task_cnt += 1; + + bpf_cgroup_release(cgrp); + return 0; +} diff --git a/tools/testing/selftests/bpf/progs/iters_task.c b/tools/testing/selftests/bpf/progs/iters_task.c new file mode 100644 index 000000000000..bd6d4f7b5e59 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/iters_task.c @@ -0,0 +1,41 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (C) 2023 Chuyi Zhou */ + +#include "vmlinux.h" +#include +#include +#include "bpf_misc.h" +#include "bpf_experimental.h" + +char _license[] SEC("license") = "GPL"; + +pid_t target_pid = 0; +int process_cnt = 0; +int thread_cnt = 0; +int all_thread_cnt = 0; + +void bpf_rcu_read_lock(void) __ksym; +void bpf_rcu_read_unlock(void) __ksym; + +SEC("fentry.s/" SYS_PREFIX "sys_getpgid") +int iter_task_for_each_sleep(void *ctx) +{ + struct task_struct *cur_task = bpf_get_current_task_btf(); + struct task_struct *pos; + + if (cur_task->pid != target_pid) + return 0; + bpf_rcu_read_lock(); + bpf_for_each(task, pos, NULL, BPF_TASK_ITER_ALL_PROCS) + if (pos->pid == target_pid) + process_cnt += 1; + + bpf_for_each(task, pos, cur_task, BPF_TASK_ITER_PROC_THREADS) + thread_cnt += 1; + + bpf_for_each(task, pos, NULL, BPF_TASK_ITER_ALL_THREADS) + if (pos->tgid == target_pid) + all_thread_cnt += 1; + bpf_rcu_read_unlock(); + return 0; +} diff --git a/tools/testing/selftests/bpf/progs/iters_task_failure.c b/tools/testing/selftests/bpf/progs/iters_task_failure.c new file mode 100644 index 000000000000..c3bf96a67dba --- /dev/null +++ b/tools/testing/selftests/bpf/progs/iters_task_failure.c @@ -0,0 +1,105 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (C) 2023 Chuyi Zhou */ + +#include "vmlinux.h" +#include +#include +#include "bpf_misc.h" +#include "bpf_experimental.h" + +char _license[] SEC("license") = "GPL"; + +struct cgroup *bpf_cgroup_from_id(u64 cgid) __ksym; +void bpf_cgroup_release(struct cgroup *p) __ksym; +void bpf_rcu_read_lock(void) __ksym; +void bpf_rcu_read_unlock(void) __ksym; + +SEC("?fentry.s/" SYS_PREFIX "sys_getpgid") +__failure __msg("expected an RCU CS when using bpf_iter_task_next") +int BPF_PROG(iter_tasks_without_lock) +{ + struct task_struct *pos; + + bpf_for_each(task, pos, NULL, BPF_TASK_ITER_ALL_PROCS) { + + } + return 0; +} + +SEC("?fentry.s/" SYS_PREFIX "sys_getpgid") +__failure __msg("expected an RCU CS when using bpf_iter_css_next") +int BPF_PROG(iter_css_without_lock) +{ + u64 cg_id = bpf_get_current_cgroup_id(); + struct cgroup *cgrp = bpf_cgroup_from_id(cg_id); + struct cgroup_subsys_state *root_css, *pos; + + if (!cgrp) + return 0; + root_css = &cgrp->self; + + bpf_for_each(css, pos, root_css, BPF_CGROUP_ITER_DESCENDANTS_POST) { + + } + bpf_cgroup_release(cgrp); + return 0; +} + +SEC("?fentry.s/" SYS_PREFIX "sys_getpgid") +__failure __msg("expected an RCU CS when using bpf_iter_task_next") +int BPF_PROG(iter_tasks_lock_and_unlock) +{ + struct task_struct *pos; + + bpf_rcu_read_lock(); + bpf_for_each(task, pos, NULL, BPF_TASK_ITER_ALL_PROCS) { + bpf_rcu_read_unlock(); + + bpf_rcu_read_lock(); + } + bpf_rcu_read_unlock(); + return 0; +} + +SEC("?fentry.s/" SYS_PREFIX "sys_getpgid") +__failure __msg("expected an RCU CS when using bpf_iter_css_next") +int BPF_PROG(iter_css_lock_and_unlock) +{ + u64 cg_id = bpf_get_current_cgroup_id(); + struct cgroup *cgrp = bpf_cgroup_from_id(cg_id); + struct cgroup_subsys_state *root_css, *pos; + + if (!cgrp) + return 0; + root_css = &cgrp->self; + + bpf_rcu_read_lock(); + bpf_for_each(css, pos, root_css, BPF_CGROUP_ITER_DESCENDANTS_POST) { + bpf_rcu_read_unlock(); + + bpf_rcu_read_lock(); + } + bpf_rcu_read_unlock(); + bpf_cgroup_release(cgrp); + return 0; +} + +SEC("?fentry.s/" SYS_PREFIX "sys_getpgid") +__failure __msg("css_task_iter is only allowed in bpf_lsm and bpf iter-s") +int BPF_PROG(iter_css_task_for_each) +{ + u64 cg_id = bpf_get_current_cgroup_id(); + struct cgroup *cgrp = bpf_cgroup_from_id(cg_id); + struct cgroup_subsys_state *css; + struct task_struct *task; + + if (cgrp == NULL) + return 0; + css = &cgrp->self; + + bpf_for_each(css_task, task, css, CSS_TASK_ITER_PROCS) { + + } + bpf_cgroup_release(cgrp); + return 0; +}