From patchwork Tue Sep 12 07:01:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuyi Zhou X-Patchwork-Id: 140599 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp1233375vqi; Fri, 15 Sep 2023 11:11:40 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGRxwUYmZqd/0K7EmJTFCJB5dK8J4kcvRe2jCEDY447JtSNEBpsHPely/85IVtGzk9dIjGH X-Received: by 2002:a05:6a20:6a20:b0:132:cd2d:16fd with SMTP id p32-20020a056a206a2000b00132cd2d16fdmr3012698pzk.38.1694801499813; Fri, 15 Sep 2023 11:11:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694801499; cv=none; d=google.com; s=arc-20160816; b=CTMlgcE7vV+IoF8T03BYCWvPuNC7xSY5xi+oKHj8BhAPLSeSuhhnn+cL1LBGhqDhsn Af4wUQBmSXBDY8pNEXNAGuqI4Bmr+ErWf/+xIq7MBVWwi3WCSVRgulwky6jZSkBRlGT6 nNYqull1N91Lq4/tMxB6B39MO8A7lWO72hJN3nEFtzvx5vj18sTtplvm2XSDLzpZP6HP VpisQSQIsK906e2x4BwAy47zf4QSGj/iyJGlvxlHQtZOxrrzjZiJ25QiVsmkDMMxZt2R pvurGHbeDa+u4bbuvJujbf+sGgH/cfK6uiI+2/ZbgikxfQJTkxhnCnL3uCpyiMVANK5f gM1Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=HDkGVzbyVwp5CP2Bj0/gBikBLcGAUp0LBh7IxC9dYSw=; fh=DR7g1EcWKOXTEoooUPBSJXUaklSrDEYzv6YDdhz1CwE=; b=e9aVaY5/lyhHdw78Bv5tGKoxxh/fUfYm9vSSbkg1AW0HWCAEPhBoTWxwEbkh+zrwgK MSqsU0jt10ULtcXYX2yfC/QIFOi9CSTCzoGTps6/eo8u7RoK2RLp+qy4FO3VHfnK2Pil J1fxaNMxE0mghNMzswgNyEdmYWhKOhKP85j1ife1BERgdOteELysugRpa0VU2I+0BYKL qiFqsaqFw4m/JoVOM+fj3vnLMIbvuWriB1LkDN+stCzbZcWZnFLxDUB4a2JeCenaylTX 58aFR+cyoe5KtTPOqKkfWbKLkxONui6z217IUUrn1qt/Hmz8ccgNKUbTyGMUeYZGTFjs N/4Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=FuKgP0rd; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from howler.vger.email (howler.vger.email. [23.128.96.34]) by mx.google.com with ESMTPS id v64-20020a638943000000b005776040b2a5si3558672pgd.457.2023.09.15.11.11.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Sep 2023 11:11:39 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) client-ip=23.128.96.34; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=FuKgP0rd; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id 90F3E844BC77; Tue, 12 Sep 2023 00:02:37 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231299AbjILHCd (ORCPT + 39 others); Tue, 12 Sep 2023 03:02:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33000 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231328AbjILHCa (ORCPT ); Tue, 12 Sep 2023 03:02:30 -0400 Received: from mail-pl1-x62e.google.com (mail-pl1-x62e.google.com [IPv6:2607:f8b0:4864:20::62e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5195810EC for ; Tue, 12 Sep 2023 00:02:03 -0700 (PDT) Received: by mail-pl1-x62e.google.com with SMTP id d9443c01a7336-1bf6ea270b2so35558715ad.0 for ; Tue, 12 Sep 2023 00:02:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1694502123; x=1695106923; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HDkGVzbyVwp5CP2Bj0/gBikBLcGAUp0LBh7IxC9dYSw=; b=FuKgP0rdrCNA8BL2+5RQShyltCoZSh6tDMZSCrIaPkjDxlemdkC06K0GeEsnkYypnY 1P5q0ytA/Wa9uX9rOokYy3bVL7QhC4/mnOg5lGyldv6U9E3UXW94xGBr1iTzzLPwCzNp vznLxsYaTmrIn2+UPKGpAB3ua88fhLAR4+0r37N84H7yhRbk2Y8X8wnRkRlL7Blnn+vG V8N3ePT6EE0isFWsHU6pdYes5qSfPgtFNHjU3wgHZAimBWV7WSApZ0byr5HZCSU1p/qN Hb4+qGeskPP6wsbsiPvxmJNhKV8BG1yop2cson0Os3mCblUf0qc6FplKx6Wq5kZbdgSI 40ww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694502123; x=1695106923; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HDkGVzbyVwp5CP2Bj0/gBikBLcGAUp0LBh7IxC9dYSw=; b=gjbCGCmsE5VQXP4mh1yCyrCL4CFY8HoRmVym+osn7fGXEz96aykUR+dsjyuigc5J6J Ele/2vMiorNsYXrTuuk6K8+ZqCROQ2qGYW5dOoxb+qkgJvO3g9MsMzQ9pcRHBat4u3tZ ba6f3jFurtl+ZedMEYepaG6O5AkhwrXkXYVFOVS652uTBr1f9TSgAnkFfcKOgylDLkqk 4kMI8PI4nqoB/tjK2jMBOunNVqu5CHoBHASz8Xr+BUM86Le2hfEmu5wyUrGUGW9WDx11 jO8nTZsWibu1SdRDrPMe1z5Nrs+1Xnz2Ce1x0fRJ0LUaof6p90h2WaCKP4wC30+aRg8z HRCg== X-Gm-Message-State: AOJu0YzdFzlISD+Ts/TKcVaa8/mo0FarsW6F7HMbVD/eGzsuedq+Jbuu aGARLN1VaXt5lT/s+F6d6g8awg== X-Received: by 2002:a17:903:2443:b0:1c3:d864:d6b3 with SMTP id l3-20020a170903244300b001c3d864d6b3mr482903pls.57.1694502122788; Tue, 12 Sep 2023 00:02:02 -0700 (PDT) Received: from n37-019-243.byted.org ([180.184.84.173]) by smtp.gmail.com with ESMTPSA id b8-20020a170902d50800b001b8953365aesm7635401plg.22.2023.09.12.00.02.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Sep 2023 00:02:02 -0700 (PDT) From: Chuyi Zhou To: bpf@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@kernel.org, tj@kernel.org, linux-kernel@vger.kernel.org, Chuyi Zhou Subject: [PATCH bpf-next v2 1/6] cgroup: Prepare for using css_task_iter_*() in BPF Date: Tue, 12 Sep 2023 15:01:44 +0800 Message-Id: <20230912070149.969939-2-zhouchuyi@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230912070149.969939-1-zhouchuyi@bytedance.com> References: <20230912070149.969939-1-zhouchuyi@bytedance.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Tue, 12 Sep 2023 00:02:37 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777128177648151030 X-GMAIL-MSGID: 1777128177648151030 This patch makes some preparations for using css_task_iter_*() in BPF Program. 1. Flags CSS_TASK_ITER_* are #define-s and it's not easy for bpf prog to use them. Convert them to enum so bpf prog can take them from vmlinux.h. 2. In the next patch we will add css_task_iter_*() in common kfuncs which is not safe. Since css_task_iter_*() does spin_unlock_irq() which might screw up irq flags depending on the context where bpf prog is running. So we should use irqsave/irqrestore here and the switching is harmless. Signed-off-by: Chuyi Zhou Acked-by: Tejun Heo --- include/linux/cgroup.h | 12 +++++------- kernel/cgroup/cgroup.c | 18 ++++++++++++------ 2 files changed, 17 insertions(+), 13 deletions(-) diff --git a/include/linux/cgroup.h b/include/linux/cgroup.h index b307013b9c6c..0ef0af66080e 100644 --- a/include/linux/cgroup.h +++ b/include/linux/cgroup.h @@ -40,13 +40,11 @@ struct kernel_clone_args; #define CGROUP_WEIGHT_DFL 100 #define CGROUP_WEIGHT_MAX 10000 -/* walk only threadgroup leaders */ -#define CSS_TASK_ITER_PROCS (1U << 0) -/* walk all threaded css_sets in the domain */ -#define CSS_TASK_ITER_THREADED (1U << 1) - -/* internal flags */ -#define CSS_TASK_ITER_SKIPPED (1U << 16) +enum { + CSS_TASK_ITER_PROCS = (1U << 0), /* walk only threadgroup leaders */ + CSS_TASK_ITER_THREADED = (1U << 1), /* walk all threaded css_sets in the domain */ + CSS_TASK_ITER_SKIPPED = (1U << 16), /* internal flags */ +}; /* a css_task_iter should be treated as an opaque object */ struct css_task_iter { diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c index 1fb7f562289d..b6d64f3b8888 100644 --- a/kernel/cgroup/cgroup.c +++ b/kernel/cgroup/cgroup.c @@ -4917,9 +4917,11 @@ static void css_task_iter_advance(struct css_task_iter *it) void css_task_iter_start(struct cgroup_subsys_state *css, unsigned int flags, struct css_task_iter *it) { + unsigned long irqflags; + memset(it, 0, sizeof(*it)); - spin_lock_irq(&css_set_lock); + spin_lock_irqsave(&css_set_lock, irqflags); it->ss = css->ss; it->flags = flags; @@ -4933,7 +4935,7 @@ void css_task_iter_start(struct cgroup_subsys_state *css, unsigned int flags, css_task_iter_advance(it); - spin_unlock_irq(&css_set_lock); + spin_unlock_irqrestore(&css_set_lock, irqflags); } /** @@ -4946,12 +4948,14 @@ void css_task_iter_start(struct cgroup_subsys_state *css, unsigned int flags, */ struct task_struct *css_task_iter_next(struct css_task_iter *it) { + unsigned long irqflags; + if (it->cur_task) { put_task_struct(it->cur_task); it->cur_task = NULL; } - spin_lock_irq(&css_set_lock); + spin_lock_irqsave(&css_set_lock, irqflags); /* @it may be half-advanced by skips, finish advancing */ if (it->flags & CSS_TASK_ITER_SKIPPED) @@ -4964,7 +4968,7 @@ struct task_struct *css_task_iter_next(struct css_task_iter *it) css_task_iter_advance(it); } - spin_unlock_irq(&css_set_lock); + spin_unlock_irqrestore(&css_set_lock, irqflags); return it->cur_task; } @@ -4977,11 +4981,13 @@ struct task_struct *css_task_iter_next(struct css_task_iter *it) */ void css_task_iter_end(struct css_task_iter *it) { + unsigned long irqflags; + if (it->cur_cset) { - spin_lock_irq(&css_set_lock); + spin_lock_irqsave(&css_set_lock, irqflags); list_del(&it->iters_node); put_css_set_locked(it->cur_cset); - spin_unlock_irq(&css_set_lock); + spin_unlock_irqrestore(&css_set_lock, irqflags); } if (it->cur_dcset) From patchwork Tue Sep 12 07:01:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuyi Zhou X-Patchwork-Id: 138319 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9ecd:0:b0:3f2:4152:657d with SMTP id t13csp543133vqx; Tue, 12 Sep 2023 09:54:12 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFmKjMG9kSB3wJd49PMvV7DuWGcEpx5hvAxhs0OqNOWh41Z3w1LsrHJtRhAtrXOYyRzZass X-Received: by 2002:a05:6808:114c:b0:3a7:805:d42e with SMTP id u12-20020a056808114c00b003a70805d42emr480423oiu.15.1694537651994; Tue, 12 Sep 2023 09:54:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694537651; cv=none; d=google.com; s=arc-20160816; b=BIucJMVCz2WgqluFcy/oYrwKbzzw4I0bv7DN/HIQj+y8hurQBswotGyX3/mAbAzKWv B1GUZtTTDCG2vPCaBOft5RDfMeQ3FRqYqYVkiYrmZMACO3JcuPE9eCwJrpx21B74bybQ HOWns9L0ayvlmQbCuYVLv+6fO2QQfV7KWoxrR5JwSc6nJu88HJjBDWoR2WuhIWhsySfd wQqCvu4rEiFseByIgX5gCByQFp5eGES9knUb6ZgXDABBHeR/lAvyNdRSDh03EfEXvyXs 7QBGvbUGBW98r+WiD1/FD5uFnlFYkcaCct6unO6or2E35jjSblrmJlzugiglqaj8Qf6B LTIQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=FNQpGQe4u79p1mkZVJD58U2KIJEFL4MZuGQeXTwOQ8w=; fh=DR7g1EcWKOXTEoooUPBSJXUaklSrDEYzv6YDdhz1CwE=; b=qyJ/THEqbuFDqjq7pvmfGcXuwsasf4/f83dG8Fuh1h9N/HURr7LL0LQN9V2sTZRgCx 7R9yF9JR7u7ZLsluY1F47Pqf2fB+oCnN25uLh8PxzSPYOqfC1hxy5ajUw2gBGUsXtUay nGjjQGHBvy1STmCRl8MOk4LdzLtXNF9jcsAilU4i8tCRkOOvMPNWstugW2LFQzRQrdO7 ebP2t892Gq2HO1bpEP9i6lY96jEoO9Z70Vf/P51qGFQUcfX0vuN34RkRemk6YpndQNXW 8CXkrNa31f+5qSAFz/luZ+Pugu47To8LyrUy5ScxO1QObnoMxmh0Qwduvwf2ujvEvjUw 7E8g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=Dow5IJyK; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from howler.vger.email (howler.vger.email. [2620:137:e000::3:4]) by mx.google.com with ESMTPS id z19-20020a63e113000000b0057744d09d09si6101317pgh.130.2023.09.12.09.54.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Sep 2023 09:54:11 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) client-ip=2620:137:e000::3:4; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=Dow5IJyK; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id E949C8483174; Tue, 12 Sep 2023 00:02:40 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231248AbjILHCf (ORCPT + 39 others); Tue, 12 Sep 2023 03:02:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33022 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231327AbjILHCa (ORCPT ); Tue, 12 Sep 2023 03:02:30 -0400 Received: from mail-pf1-x431.google.com (mail-pf1-x431.google.com [IPv6:2607:f8b0:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B01B0E76 for ; Tue, 12 Sep 2023 00:02:06 -0700 (PDT) Received: by mail-pf1-x431.google.com with SMTP id d2e1a72fcca58-68fb46f38f9so1916831b3a.1 for ; Tue, 12 Sep 2023 00:02:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1694502126; x=1695106926; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FNQpGQe4u79p1mkZVJD58U2KIJEFL4MZuGQeXTwOQ8w=; b=Dow5IJyKwkOy28TWDR9DDnjAogp9ctHCGYgTIiopVm61CJBX4KtSrMGSU1tt3U8dZX t8jMpD60rg5/hDwiqUJeq9z4WKLQwBXfV7ALggUQeh7PqPCrti/rKseXZU4/2/ivg1/M t5mdw24PCiXIwotA7LSzjM5CpbpQpODL/Dn/30HR64J/9e4rJhIpvl9l4hPrgOJbqsfJ RkROP7y53WJoqvfKDUMZYjz0QnDf7EWX/eM2mjGuGXMgnfMuOkyDw0Xe/nQTbeKaVznR bLX1GuNfRU/ADul/SXbTD7ZuEl25jbAoJdvDAO7ZRl2/CLRyqKPi+dt4vKy/NLOY/Y8K a6lA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694502126; x=1695106926; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FNQpGQe4u79p1mkZVJD58U2KIJEFL4MZuGQeXTwOQ8w=; b=irHQftaPoXH9vhy8VehLJyMHtdI3khBIhmK9tK+lpibJLLJsFQxwTMGxnRvi1fmhO4 07MhtE0VnTTzHvsdUePg4AhjdMubAoitVGkZ4TQk3iMf/1+Q4a3CSWv/sc3UZez1qQAr Q+Oyr6sLh1kpcj2aei/4Y8Ieayl4iDWJb+/Yf6sfgIGvc3m4hwccrTSJvZUlrhXvXK1B sP2l/9/W/cg0VD6VX94n+AHfEeTXkrBqa5AzEL0ChUJoxNBhJugcYCH7gFNhdiUpi7rN C7HnGb5gr+DiqADJdt+a7tIrpPF4dDl5yD/uWD/AFkn0YpYV2b3k4ZEd8i3ZFKAjGTjY uR3Q== X-Gm-Message-State: AOJu0YyOI7V4N1DGe/XBiaiUs8PlqDXgtPtCkjBuO/OGFW+PG/Xb3cQB ZPYTg4ZzrAQZKszLEJqJ8CXdcw== X-Received: by 2002:a05:6a20:ce9f:b0:153:c9fa:4ed6 with SMTP id if31-20020a056a20ce9f00b00153c9fa4ed6mr8870801pzb.10.1694502126142; Tue, 12 Sep 2023 00:02:06 -0700 (PDT) Received: from n37-019-243.byted.org ([180.184.84.173]) by smtp.gmail.com with ESMTPSA id b8-20020a170902d50800b001b8953365aesm7635401plg.22.2023.09.12.00.02.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Sep 2023 00:02:05 -0700 (PDT) From: Chuyi Zhou To: bpf@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@kernel.org, tj@kernel.org, linux-kernel@vger.kernel.org, Chuyi Zhou Subject: [PATCH bpf-next v2 2/6] bpf: Introduce css_task open-coded iterator kfuncs Date: Tue, 12 Sep 2023 15:01:45 +0800 Message-Id: <20230912070149.969939-3-zhouchuyi@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230912070149.969939-1-zhouchuyi@bytedance.com> References: <20230912070149.969939-1-zhouchuyi@bytedance.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Tue, 12 Sep 2023 00:02:41 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1776851512796017396 X-GMAIL-MSGID: 1776851512796017396 This patch adds kfuncs bpf_iter_css_task_{new,next,destroy} which allow creation and manipulation of struct bpf_iter_css_task in open-coded iterator style. These kfuncs actually wrapps css_task_iter_{start,next, end}. BPF programs can use these kfuncs through bpf_for_each macro for iteration of all tasks under a css. css_task_iter_*() would try to get the global spin-lock *css_set_lock*, so the bpf side has to be careful in where it allows to use this iter. Currently we only allow it in bpf_lsm and bpf iter-s. Signed-off-by: Chuyi Zhou --- include/uapi/linux/bpf.h | 4 +++ kernel/bpf/helpers.c | 3 +++ kernel/bpf/task_iter.c | 48 ++++++++++++++++++++++++++++++++++ kernel/bpf/verifier.c | 23 ++++++++++++++++ tools/include/uapi/linux/bpf.h | 4 +++ tools/lib/bpf/bpf_helpers.h | 7 +++++ 6 files changed, 89 insertions(+) diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 73b155e52204..de02c0971428 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -7318,4 +7318,8 @@ struct bpf_iter_num { __u64 __opaque[1]; } __attribute__((aligned(8))); +struct bpf_iter_css_task { + __u64 __opaque[1]; +} __attribute__((aligned(8))); + #endif /* _UAPI__LINUX_BPF_H__ */ diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index b0a9834f1051..d6a16becfbb9 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -2504,6 +2504,9 @@ BTF_ID_FLAGS(func, bpf_dynptr_slice_rdwr, KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_num_new, KF_ITER_NEW) BTF_ID_FLAGS(func, bpf_iter_num_next, KF_ITER_NEXT | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_num_destroy, KF_ITER_DESTROY) +BTF_ID_FLAGS(func, bpf_iter_css_task_new, KF_ITER_NEW) +BTF_ID_FLAGS(func, bpf_iter_css_task_next, KF_ITER_NEXT | KF_RET_NULL) +BTF_ID_FLAGS(func, bpf_iter_css_task_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_dynptr_adjust) BTF_ID_FLAGS(func, bpf_dynptr_is_null) BTF_ID_FLAGS(func, bpf_dynptr_is_rdonly) diff --git a/kernel/bpf/task_iter.c b/kernel/bpf/task_iter.c index 7473068ed313..d8539cc05ffd 100644 --- a/kernel/bpf/task_iter.c +++ b/kernel/bpf/task_iter.c @@ -803,6 +803,54 @@ const struct bpf_func_proto bpf_find_vma_proto = { .arg5_type = ARG_ANYTHING, }; +struct bpf_iter_css_task_kern { + struct css_task_iter *css_it; +} __attribute__((aligned(8))); + +__bpf_kfunc int bpf_iter_css_task_new(struct bpf_iter_css_task *it, + struct cgroup_subsys_state *css, unsigned int flags) +{ + struct bpf_iter_css_task_kern *kit = (void *)it; + + BUILD_BUG_ON(sizeof(struct bpf_iter_css_task_kern) != sizeof(struct bpf_iter_css_task)); + BUILD_BUG_ON(__alignof__(struct bpf_iter_css_task_kern) != + __alignof__(struct bpf_iter_css_task)); + + switch (flags) { + case CSS_TASK_ITER_PROCS | CSS_TASK_ITER_THREADED: + case CSS_TASK_ITER_PROCS: + case 0: + break; + default: + return -EINVAL; + } + + kit->css_it = kzalloc(sizeof(struct css_task_iter), GFP_KERNEL); + if (!kit->css_it) + return -ENOMEM; + css_task_iter_start(css, flags, kit->css_it); + return 0; +} + +__bpf_kfunc struct task_struct *bpf_iter_css_task_next(struct bpf_iter_css_task *it) +{ + struct bpf_iter_css_task_kern *kit = (void *)it; + + if (!kit->css_it) + return NULL; + return css_task_iter_next(kit->css_it); +} + +__bpf_kfunc void bpf_iter_css_task_destroy(struct bpf_iter_css_task *it) +{ + struct bpf_iter_css_task_kern *kit = (void *)it; + + if (!kit->css_it) + return; + css_task_iter_end(kit->css_it); + kfree(kit->css_it); +} + DEFINE_PER_CPU(struct mmap_unlock_irq_work, mmap_unlock_work); static void do_mmap_read_unlock(struct irq_work *entry) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index dbba2b806017..2367483bf4c2 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -10332,6 +10332,7 @@ enum special_kfunc_type { KF_bpf_dynptr_clone, KF_bpf_percpu_obj_new_impl, KF_bpf_percpu_obj_drop_impl, + KF_bpf_iter_css_task_new, }; BTF_SET_START(special_kfunc_set) @@ -10354,6 +10355,7 @@ BTF_ID(func, bpf_dynptr_slice_rdwr) BTF_ID(func, bpf_dynptr_clone) BTF_ID(func, bpf_percpu_obj_new_impl) BTF_ID(func, bpf_percpu_obj_drop_impl) +BTF_ID(func, bpf_iter_css_task_new) BTF_SET_END(special_kfunc_set) BTF_ID_LIST(special_kfunc_list) @@ -10378,6 +10380,7 @@ BTF_ID(func, bpf_dynptr_slice_rdwr) BTF_ID(func, bpf_dynptr_clone) BTF_ID(func, bpf_percpu_obj_new_impl) BTF_ID(func, bpf_percpu_obj_drop_impl) +BTF_ID(func, bpf_iter_css_task_new) static bool is_kfunc_ret_null(struct bpf_kfunc_call_arg_meta *meta) { @@ -10902,6 +10905,20 @@ static int process_kf_arg_ptr_to_rbtree_node(struct bpf_verifier_env *env, &meta->arg_rbtree_root.field); } +static bool check_css_task_iter_allowlist(struct bpf_verifier_env *env) +{ + enum bpf_prog_type prog_type = resolve_prog_type(env->prog); + + switch (prog_type) { + case BPF_PROG_TYPE_LSM: + return true; + case BPF_TRACE_ITER: + return env->prog->aux->sleepable; + default: + return false; + } +} + static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_arg_meta *meta, int insn_idx) { @@ -11152,6 +11169,12 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ break; } case KF_ARG_PTR_TO_ITER: + if (meta->func_id == special_kfunc_list[KF_bpf_iter_css_task_new]) { + if (!check_css_task_iter_allowlist(env)) { + verbose(env, "css_task_iter is only allowed in bpf_lsm and bpf iter-s\n"); + return -EINVAL; + } + } ret = process_iter_arg(env, regno, insn_idx, meta); if (ret < 0) return ret; diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index 73b155e52204..de02c0971428 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -7318,4 +7318,8 @@ struct bpf_iter_num { __u64 __opaque[1]; } __attribute__((aligned(8))); +struct bpf_iter_css_task { + __u64 __opaque[1]; +} __attribute__((aligned(8))); + #endif /* _UAPI__LINUX_BPF_H__ */ diff --git a/tools/lib/bpf/bpf_helpers.h b/tools/lib/bpf/bpf_helpers.h index 77ceea575dc7..f48723c6c593 100644 --- a/tools/lib/bpf/bpf_helpers.h +++ b/tools/lib/bpf/bpf_helpers.h @@ -303,6 +303,13 @@ extern int bpf_iter_num_new(struct bpf_iter_num *it, int start, int end) __weak extern int *bpf_iter_num_next(struct bpf_iter_num *it) __weak __ksym; extern void bpf_iter_num_destroy(struct bpf_iter_num *it) __weak __ksym; +struct bpf_iter_css_task; +struct cgroup_subsys_state; +extern int bpf_iter_css_task_new(struct bpf_iter_css_task *it, + struct cgroup_subsys_state *css, unsigned int flags) __weak __ksym; +extern struct task_struct *bpf_iter_css_task_next(struct bpf_iter_css_task *it) __weak __ksym; +extern void bpf_iter_css_task_destroy(struct bpf_iter_css_task *it) __weak __ksym; + #ifndef bpf_for_each /* bpf_for_each(iter_type, cur_elem, args...) provides generic construct for * using BPF open-coded iterators without having to write mundane explicit From patchwork Tue Sep 12 07:01:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuyi Zhou X-Patchwork-Id: 138350 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9ecd:0:b0:3f2:4152:657d with SMTP id t13csp585616vqx; Tue, 12 Sep 2023 11:03:01 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEU5Bqj5h2QtQSVuM33ZBX8uWioxaXn2DbUJusGf8SvZYYsR6HuG2ULqv/FVPBpOL/1M23m X-Received: by 2002:a17:90a:9403:b0:267:f758:15d4 with SMTP id r3-20020a17090a940300b00267f75815d4mr9468057pjo.48.1694541781396; Tue, 12 Sep 2023 11:03:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694541781; cv=none; d=google.com; s=arc-20160816; b=nZVYYLRHDCk4/6OiURjNI1hMkQnEH4tyTeRrEjUlMTwZ83Ioa3XbkEfUPxdpOsZneu y5+LXvyh+10YGooVQ2Aea02jCMbsgTgoukYSutTbjsE2mZbcscpcaVGROorrMRFLSs2B o05gbZNPl37qS1pEkSiM+7IWOY9wlAvNfkuSUsLNWF9UukVE5IFwzZ+PKDH3bcRPIy4M EWFQ8j3PpIq9OgG7lcRNPEt7NzXx3qkhcLf8t+XHhpaYwC/phn6AcTh10C1guMM2hyOe 9Jd1Hbe5K92d26Cyd6CDaRAKQ24+hcPsa/EUYO/+CJbGbJ9DoJw3p+WulvK9VMDNnoiB ItUQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=+2pL38sdR1zkqvND06hEOkj6aCyDQM/YMk3Q7fUlrh4=; fh=DR7g1EcWKOXTEoooUPBSJXUaklSrDEYzv6YDdhz1CwE=; b=u09ih/ei582hXQn77L5tWG7zv7SQGvc9kfKzMvykHVVQWbgQ9s7AMmmOJ1+8/AjaSb VPBkEbUKAurmJcJ9RZ0iNIPQ2Y6fO/EJiYkiO7aRYK/gxZ9Fs60n4RFSeheRpExjF37v hPE2bwZIhozdY8d8OaJR0Iquh0cnkXYAzvjGaywx7na6hUXyAo/KmM66n00uq0tbPswg qCAFEKiE8sx9l0j44Pa4IESql9rIQtM0H0BIzGz2RhXoAEnxP8BfHeqZazQ6h07kMAFV YWFd+KjFFG60nWw+ED5v4IXih06FUxHfwx7eXNP08ebvf/HzyrqsFoz5yKRy3l1qFpob brbw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=M2IuHdH8; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from howler.vger.email (howler.vger.email. [2620:137:e000::3:4]) by mx.google.com with ESMTPS id w6-20020a17090a8a0600b0026b698fdda6si10208360pjn.98.2023.09.12.11.03.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Sep 2023 11:03:01 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) client-ip=2620:137:e000::3:4; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=M2IuHdH8; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id A1D91827D671; Tue, 12 Sep 2023 00:02:22 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231139AbjILHCQ (ORCPT + 39 others); Tue, 12 Sep 2023 03:02:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42904 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231216AbjILHCN (ORCPT ); Tue, 12 Sep 2023 03:02:13 -0400 Received: from mail-pl1-x62f.google.com (mail-pl1-x62f.google.com [IPv6:2607:f8b0:4864:20::62f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 02061E7C for ; Tue, 12 Sep 2023 00:02:10 -0700 (PDT) Received: by mail-pl1-x62f.google.com with SMTP id d9443c01a7336-1c0ecb9a075so37537735ad.2 for ; Tue, 12 Sep 2023 00:02:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1694502129; x=1695106929; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=+2pL38sdR1zkqvND06hEOkj6aCyDQM/YMk3Q7fUlrh4=; b=M2IuHdH8p6VL/T+r4KhufKVPTFU9Qdd3Vj8fRX16Ps/8Jex8HpP2UTkOUzS3bE7/oW p0lcRAwO7+0E3dt+h7uqMRwB2GqCI5g1PuSu6FZ9Ct0E7OnEX2gq5qyVLx3IIsoPkP6Q /2dD5jdLkzIsjH5bpUst9xbVHww97Jsxzgzlp5UJ2M4/+FHtUSdVd/tN+dxIcr/kq/bS GO2r9XE86jJyy7IkIzNpEanGMne3qAOb/KEHS2MlvGdiAEYv9VkI7cR7SuirsS019hcM DYFOZjhfY7g19waibSAbfQyTU+hR1nhYAnO2Bmms4oPTE+dMd4ikYAN0QIGgiINaHiuJ Vnzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694502129; x=1695106929; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+2pL38sdR1zkqvND06hEOkj6aCyDQM/YMk3Q7fUlrh4=; b=gu0csuK4shNgeEb8z7dGU+8cshjShm4OUeaO2UinoPJGvKajXjjZBOfaNTKaj9D60l epgXi+lP1UNRBKUKWKwM7ZEpKj/m2Os3pMwhCBWC+n2yRkylBNDJSKMTSZelFyUQM48E TfRpJS47uQDzab+7b9yti0M0xHDBZ8mLGCJtM35YfYFLoPtfydYt7pksEtzucwqGB4Sx zgmF7KwxT9dhXN4dApQVUXNX+A1JviKsr4JaMmJbvoGRv42zZtqVAcBGz8AaS9iZWFIv PmxRunnJ0g0knSfUXOhOJUx3GsLPOff5ghtxMhz67gLQb4rAz0+4G8mwVqMGaTErLYV2 FyCg== X-Gm-Message-State: AOJu0Yz+CrGjL0u+fFLhui7t384WpRDulcnx8fQPQn3VcT/XFCROTiHn xl+MwZMGjlCv5YExSCOuewE2Pg== X-Received: by 2002:a17:903:2302:b0:1bb:e74b:39ff with SMTP id d2-20020a170903230200b001bbe74b39ffmr12134008plh.0.1694502129564; Tue, 12 Sep 2023 00:02:09 -0700 (PDT) Received: from n37-019-243.byted.org ([180.184.84.173]) by smtp.gmail.com with ESMTPSA id b8-20020a170902d50800b001b8953365aesm7635401plg.22.2023.09.12.00.02.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Sep 2023 00:02:09 -0700 (PDT) From: Chuyi Zhou To: bpf@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@kernel.org, tj@kernel.org, linux-kernel@vger.kernel.org, Chuyi Zhou Subject: [PATCH bpf-next v2 3/6] bpf: Introduce process open coded iterator kfuncs Date: Tue, 12 Sep 2023 15:01:46 +0800 Message-Id: <20230912070149.969939-4-zhouchuyi@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230912070149.969939-1-zhouchuyi@bytedance.com> References: <20230912070149.969939-1-zhouchuyi@bytedance.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Tue, 12 Sep 2023 00:02:22 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1776855842822877231 X-GMAIL-MSGID: 1776855842822877231 This patch adds kfuncs bpf_iter_process_{new,next,destroy} which allow creation and manipulation of struct bpf_iter_process in open-coded iterator style. BPF programs can use these kfuncs or through bpf_for_each macro to iterate all processes in the system. Signed-off-by: Chuyi Zhou --- include/uapi/linux/bpf.h | 4 ++++ kernel/bpf/helpers.c | 3 +++ kernel/bpf/task_iter.c | 29 +++++++++++++++++++++++++++++ tools/include/uapi/linux/bpf.h | 4 ++++ tools/lib/bpf/bpf_helpers.h | 5 +++++ 5 files changed, 45 insertions(+) diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index de02c0971428..befa55b52e29 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -7322,4 +7322,8 @@ struct bpf_iter_css_task { __u64 __opaque[1]; } __attribute__((aligned(8))); +struct bpf_iter_process { + __u64 __opaque[1]; +} __attribute__((aligned(8))); + #endif /* _UAPI__LINUX_BPF_H__ */ diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index d6a16becfbb9..9b7d2c6f99d1 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -2507,6 +2507,9 @@ BTF_ID_FLAGS(func, bpf_iter_num_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_iter_css_task_new, KF_ITER_NEW) BTF_ID_FLAGS(func, bpf_iter_css_task_next, KF_ITER_NEXT | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_css_task_destroy, KF_ITER_DESTROY) +BTF_ID_FLAGS(func, bpf_iter_process_new, KF_ITER_NEW) +BTF_ID_FLAGS(func, bpf_iter_process_next, KF_ITER_NEXT | KF_RET_NULL) +BTF_ID_FLAGS(func, bpf_iter_process_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_dynptr_adjust) BTF_ID_FLAGS(func, bpf_dynptr_is_null) BTF_ID_FLAGS(func, bpf_dynptr_is_rdonly) diff --git a/kernel/bpf/task_iter.c b/kernel/bpf/task_iter.c index d8539cc05ffd..9d1927dc3a06 100644 --- a/kernel/bpf/task_iter.c +++ b/kernel/bpf/task_iter.c @@ -851,6 +851,35 @@ __bpf_kfunc void bpf_iter_css_task_destroy(struct bpf_iter_css_task *it) kfree(kit->css_it); } +struct bpf_iter_process_kern { + struct task_struct *tsk; +} __attribute__((aligned(8))); + +__bpf_kfunc int bpf_iter_process_new(struct bpf_iter_process *it) +{ + struct bpf_iter_process_kern *kit = (void *)it; + + BUILD_BUG_ON(sizeof(struct bpf_iter_process_kern) != sizeof(struct bpf_iter_process)); + BUILD_BUG_ON(__alignof__(struct bpf_iter_process_kern) != + __alignof__(struct bpf_iter_process)); + + kit->tsk = &init_task; + return 0; +} + +__bpf_kfunc struct task_struct *bpf_iter_process_next(struct bpf_iter_process *it) +{ + struct bpf_iter_process_kern *kit = (void *)it; + + kit->tsk = next_task(kit->tsk); + + return kit->tsk == &init_task ? NULL : kit->tsk; +} + +__bpf_kfunc void bpf_iter_process_destroy(struct bpf_iter_process *it) +{ +} + DEFINE_PER_CPU(struct mmap_unlock_irq_work, mmap_unlock_work); static void do_mmap_read_unlock(struct irq_work *entry) diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index de02c0971428..befa55b52e29 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -7322,4 +7322,8 @@ struct bpf_iter_css_task { __u64 __opaque[1]; } __attribute__((aligned(8))); +struct bpf_iter_process { + __u64 __opaque[1]; +} __attribute__((aligned(8))); + #endif /* _UAPI__LINUX_BPF_H__ */ diff --git a/tools/lib/bpf/bpf_helpers.h b/tools/lib/bpf/bpf_helpers.h index f48723c6c593..858252c2641c 100644 --- a/tools/lib/bpf/bpf_helpers.h +++ b/tools/lib/bpf/bpf_helpers.h @@ -310,6 +310,11 @@ extern int bpf_iter_css_task_new(struct bpf_iter_css_task *it, extern struct task_struct *bpf_iter_css_task_next(struct bpf_iter_css_task *it) __weak __ksym; extern void bpf_iter_css_task_destroy(struct bpf_iter_css_task *it) __weak __ksym; +struct bpf_iter_process; +extern int bpf_iter_process_new(struct bpf_iter_process *it) __weak __ksym; +extern struct task_struct *bpf_iter_process_next(struct bpf_iter_process *it) __weak __ksym; +extern void bpf_iter_process_destroy(struct bpf_iter_process *it) __weak __ksym; + #ifndef bpf_for_each /* bpf_for_each(iter_type, cur_elem, args...) provides generic construct for * using BPF open-coded iterators without having to write mundane explicit From patchwork Tue Sep 12 07:01:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuyi Zhou X-Patchwork-Id: 138675 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9ecd:0:b0:3f2:4152:657d with SMTP id t13csp901736vqx; Wed, 13 Sep 2023 00:08:18 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHLPgXZDuAHLtE8XySYm+amy1yy+E+NPiSWRWo09CgrA456B6kiOgm3yJlXpHOS024VXW3V X-Received: by 2002:a17:903:503:b0:1c1:f3f8:3949 with SMTP id jn3-20020a170903050300b001c1f3f83949mr2008637plb.1.1694588897832; Wed, 13 Sep 2023 00:08:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694588897; cv=none; d=google.com; s=arc-20160816; b=ydNdh+Pjv/kqrEN0F4Ayp46M5etak6pqfPP8W8i8r/HRF93e2Es7e5UD4bim1q5+a5 Nxw9kuOoz9hFva3ukdvGtW2+PM0fS2OEnzyIYWx6ie8Y2O1Br5JI2rYEr6hIJWoTr0Bm jsVgXKE8pNCn7VqZOQQXogzlps50g7kHObjCZyVoEbssT8kn5lYslO/jM24kDTLH7ySU Poqu9X2Bpng1j+50mdKZM7dtNq1N4ngyB8iWr5L7Krlpsio8F9cEP86RgDAKjZDlo/PT kocRvEbXtEBey3uRg2B/woKs6VeGt63B9fun5tvbNfrdhUfuhsAb3iZhabnUvLuHZbYM wUXA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=zZ7auXMwt2HG2eO79EZb79CGVaa+fy/IcLCkXgQTU2E=; fh=DR7g1EcWKOXTEoooUPBSJXUaklSrDEYzv6YDdhz1CwE=; b=wc9Bo+NinpCC7aR8RwLXonFMz/CtZvOCbDUzw/LzSVtj5UP27P5uGbdVwIxB33u1cp ahMLSxAkhhaB9gaQuq8A/YudY/wmplCkLTWmFMMYDwYJ6kNjJcZcMBJpV6lSZCfRybFk RiZZzPTtLK2VPW5RSqfzQIz7vCJcJcdcllPs72ZY4OauSCbhn1FphE5nqJEnujuiny5s sJan5xUM0ptCGjA5YFCcEBplkjk12ofGHeFSRko8c1yul/yrMP9jg1Oymy9DZqBwKt/T 1TRmFDiKD/rX7jEdDwcT8DohwkIIG6halAsGvQqx9nhTmeXzwSuh0jVZ4xpUF/vZAt3+ x1CQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=gJntY8cV; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from howler.vger.email (howler.vger.email. [23.128.96.34]) by mx.google.com with ESMTPS id h6-20020a170902f54600b001c1f27ea534si10077932plf.353.2023.09.13.00.08.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Sep 2023 00:08:17 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) client-ip=23.128.96.34; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=gJntY8cV; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id 9EE8B8483152; Tue, 12 Sep 2023 00:02:54 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231332AbjILHCt (ORCPT + 39 others); Tue, 12 Sep 2023 03:02:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51986 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231382AbjILHCl (ORCPT ); Tue, 12 Sep 2023 03:02:41 -0400 Received: from mail-pl1-x62e.google.com (mail-pl1-x62e.google.com [IPv6:2607:f8b0:4864:20::62e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B4347AF for ; Tue, 12 Sep 2023 00:02:16 -0700 (PDT) Received: by mail-pl1-x62e.google.com with SMTP id d9443c01a7336-1c35ee3b0d2so35295345ad.2 for ; Tue, 12 Sep 2023 00:02:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1694502136; x=1695106936; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zZ7auXMwt2HG2eO79EZb79CGVaa+fy/IcLCkXgQTU2E=; b=gJntY8cV6wapICpdZfEIKXOai/sUbNM/O2PEKnDVSxn2gbWVIXA5t6cb7ba/zTwtwZ WPKEc9Sb6BUUld9t3MOlZYSn36CNY7tArLtbbMwnTUTx3Mf5iaTfiZltVlVMV5xD4xQY X0FW9FZVCyndsjXUA8mh1czh/TU1HNC0/w83YQ7H09Et5h9x0ZmpeyHEFeji9O+0Vmxl xlJ0cfto8eS7a4We7zUxCHvM91RjLFelbmxD1yMP5jZN/ZD7K0XFwIEDw+SW7VfCMNOT Jep7/hQOIfpGe4m+m5bjhWqY0GvnE9Fwk1FcBmB1GqSj2HWUlw3/QSPGHW5JjFwi6oqI xDyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694502136; x=1695106936; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zZ7auXMwt2HG2eO79EZb79CGVaa+fy/IcLCkXgQTU2E=; b=Fv8wJ4ueJ6Hi+kcxAfo1obxMcUAFTShKxRFrUceNSO4yD8KEhlM5W2dE068gfoEDv+ +zy1oE6P8bx7uN2ea6/sDZFSK0/Jdt/ImmDpnIfxLMo/DBCjNp2KrS/htRbpgdck4M4q WCpdGLwlaZcgBo9DsqZFfR0bqO87z4xPN+qzxAjNpqeuN3nSTO8ZWKRgNHPltRGLQTlc L6L2s7tAaZBlieF2vyO0g/q+YGTUXzSb2Q9DXKQNT1DyVXC9uDVnXh/ugzqARLUMQtzB 6b/WRnsDsob326GQSf63scFzwoIdQ69Glbnfx1je65b1g9kTJh8PRcqLFSkg/+p7kw5h jVwQ== X-Gm-Message-State: AOJu0YwP9kbOJH7IfwreSUNeeZKPu/UuRoOhp2xT6q8RIde1AQ5lxl/X xKx/07lVmUGipamV+1JROkqJTw== X-Received: by 2002:a17:902:e883:b0:1bc:506a:58f2 with SMTP id w3-20020a170902e88300b001bc506a58f2mr10371670plg.46.1694502136153; Tue, 12 Sep 2023 00:02:16 -0700 (PDT) Received: from n37-019-243.byted.org ([180.184.84.173]) by smtp.gmail.com with ESMTPSA id b8-20020a170902d50800b001b8953365aesm7635401plg.22.2023.09.12.00.02.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Sep 2023 00:02:15 -0700 (PDT) From: Chuyi Zhou To: bpf@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@kernel.org, tj@kernel.org, linux-kernel@vger.kernel.org, Chuyi Zhou Subject: [PATCH bpf-next v2 5/6] bpf: teach the verifier to enforce css_iter and process_iter in RCU CS Date: Tue, 12 Sep 2023 15:01:48 +0800 Message-Id: <20230912070149.969939-6-zhouchuyi@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230912070149.969939-1-zhouchuyi@bytedance.com> References: <20230912070149.969939-1-zhouchuyi@bytedance.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Tue, 12 Sep 2023 00:02:54 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1776905248505348469 X-GMAIL-MSGID: 1776905248505348469 css_iter and process_iter should be used in rcu section. Specifically, in sleepable progs explicit bpf_rcu_read_lock() is needed before use these iters. In normal bpf progs that have implicit rcu_read_lock(), it's OK to use them directly. This patch checks whether we are in rcu cs before we want to invoke bpf_iter_process_new and bpf_iter_css_{pre, post}_new in mark_stack_slots_iter(). If the rcu protection is guaranteed, we would let st->type = PTR_TO_STACK | MEM_RCU. is_iter_reg_valid_init() will reject if reg->type is UNTRUSTED. Signed-off-by: Chuyi Zhou --- kernel/bpf/verifier.c | 30 ++++++++++++++++++++++++++++-- 1 file changed, 28 insertions(+), 2 deletions(-) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 2367483bf4c2..6a6827ba7a18 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -1172,7 +1172,13 @@ static bool is_dynptr_type_expected(struct bpf_verifier_env *env, struct bpf_reg static void __mark_reg_known_zero(struct bpf_reg_state *reg); +static bool in_rcu_cs(struct bpf_verifier_env *env); + +/* check whether we are using bpf_iter_process_*() or bpf_iter_css_*() */ +static bool is_iter_need_rcu(struct bpf_kfunc_call_arg_meta *meta); + static int mark_stack_slots_iter(struct bpf_verifier_env *env, + struct bpf_kfunc_call_arg_meta *meta, struct bpf_reg_state *reg, int insn_idx, struct btf *btf, u32 btf_id, int nr_slots) { @@ -1193,6 +1199,12 @@ static int mark_stack_slots_iter(struct bpf_verifier_env *env, __mark_reg_known_zero(st); st->type = PTR_TO_STACK; /* we don't have dedicated reg type */ + if (is_iter_need_rcu(meta)) { + if (in_rcu_cs(env)) + st->type |= MEM_RCU; + else + st->type |= PTR_UNTRUSTED; + } st->live |= REG_LIVE_WRITTEN; st->ref_obj_id = i == 0 ? id : 0; st->iter.btf = btf; @@ -1281,6 +1293,8 @@ static bool is_iter_reg_valid_init(struct bpf_verifier_env *env, struct bpf_reg_ struct bpf_stack_state *slot = &state->stack[spi - i]; struct bpf_reg_state *st = &slot->spilled_ptr; + if (st->type & PTR_UNTRUSTED) + return false; /* only main (first) slot has ref_obj_id set */ if (i == 0 && !st->ref_obj_id) return false; @@ -7503,13 +7517,13 @@ static int process_iter_arg(struct bpf_verifier_env *env, int regno, int insn_id return err; } - err = mark_stack_slots_iter(env, reg, insn_idx, meta->btf, btf_id, nr_slots); + err = mark_stack_slots_iter(env, meta, reg, insn_idx, meta->btf, btf_id, nr_slots); if (err) return err; } else { /* iter_next() or iter_destroy() expect initialized iter state*/ if (!is_iter_reg_valid_init(env, reg, meta->btf, btf_id, nr_slots)) { - verbose(env, "expected an initialized iter_%s as arg #%d\n", + verbose(env, "expected an initialized iter_%s as arg #%d or without bpf_rcu_read_lock()\n", iter_type_str(meta->btf, btf_id), regno); return -EINVAL; } @@ -10382,6 +10396,18 @@ BTF_ID(func, bpf_percpu_obj_new_impl) BTF_ID(func, bpf_percpu_obj_drop_impl) BTF_ID(func, bpf_iter_css_task_new) +BTF_SET_START(rcu_protect_kfuns_set) +BTF_ID(func, bpf_iter_process_new) +BTF_ID(func, bpf_iter_css_pre_new) +BTF_ID(func, bpf_iter_css_post_new) +BTF_SET_END(rcu_protect_kfuns_set) + +static inline bool is_iter_need_rcu(struct bpf_kfunc_call_arg_meta *meta) +{ + return btf_id_set_contains(&rcu_protect_kfuns_set, meta->func_id); +} + + static bool is_kfunc_ret_null(struct bpf_kfunc_call_arg_meta *meta) { if (meta->func_id == special_kfunc_list[KF_bpf_refcount_acquire_impl] && From patchwork Tue Sep 12 07:01:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuyi Zhou X-Patchwork-Id: 139868 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp652911vqi; Thu, 14 Sep 2023 15:06:45 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEt1xCe/v0EOnI8K+B7keAK3WYeQ3ZgGkOorrxX9XDa74LDdq+Lo4D5pgGvJ9+XeAfrgjSp X-Received: by 2002:a05:6a20:dd9a:b0:14c:512c:c138 with SMTP id kw26-20020a056a20dd9a00b0014c512cc138mr45433pzb.60.1694729204949; Thu, 14 Sep 2023 15:06:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694729204; cv=none; d=google.com; s=arc-20160816; b=is/lVneQn9kmqqv5aGrDW2uOlsvCOWlaErbvQFP7QhVw8zN+8qcILBZqH8vSaq3dZM C8ElydvGKVgrd6YZ0qJFbqb6OzK92eZMESMnjI54szCTNRGNjKxaz5ZfZJD3yAQ5Ozkh B+40rJ7zDhxTnub3Xz1FxSiizdSvJWwJfT+pCpXeFH/GjZadcYUGuOJa5sLT4IktRl7p 8/zNz9zzX+a6wDLitilPHgyX7LuVewxpeV27joEgHJ//pyduDhO/Ud00h4ybu29hWsAv E5GIJX4uWUvfo5XvSPhO+sBYMOCQUw6yjl7nOuJ/HL5EX4n28DOqI9fxIVTF/hIr3YVk ZKRQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=WHvChKGnnZi2VbMoC/1ghgca6iZmmMi6EKlv8J2G2fg=; fh=DR7g1EcWKOXTEoooUPBSJXUaklSrDEYzv6YDdhz1CwE=; b=dtgEWvBNd/LUWJ+X391sG/8k8rd5n1E2v0eBqt+lqcnRXf1JQNeBVJGXsi+7+I6cem pAApyJmebPPQmfQa/GzkJUMOUpstZXy9kwKYIwmSmh+L6sAnIWPCSm9NUDQtlM2ln09I kq8S1Lium47W66TwE7M7ZCGwsNMfF6a59EiZZhu1ohFYOF1nKYKiiH/2tV5UYECUfBYA 1M3yHspVNdGh8ugU5JXI7enfWh4yboOBtvA3zEkoHuzjBNOprxpQuXm3ttWQvUa/d418 5DYwvVpafyTEaYO8efHXQoUkNM7CfTOud1CV0Pb5xygHbEhzXd13ctEd3xKfRmjbc9Tm Pr3g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=Pz94yrZE; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from howler.vger.email (howler.vger.email. [2620:137:e000::3:4]) by mx.google.com with ESMTPS id ea13-20020a056a004c0d00b00686baf235easi2207582pfb.258.2023.09.14.15.06.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Sep 2023 15:06:44 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) client-ip=2620:137:e000::3:4; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=Pz94yrZE; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id 4723A84AC67C; Tue, 12 Sep 2023 00:03:00 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231416AbjILHCx (ORCPT + 39 others); Tue, 12 Sep 2023 03:02:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40522 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231392AbjILHCl (ORCPT ); Tue, 12 Sep 2023 03:02:41 -0400 Received: from mail-pl1-x631.google.com (mail-pl1-x631.google.com [IPv6:2607:f8b0:4864:20::631]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B42B910C7 for ; Tue, 12 Sep 2023 00:02:19 -0700 (PDT) Received: by mail-pl1-x631.google.com with SMTP id d9443c01a7336-1c336f3f449so45837165ad.3 for ; Tue, 12 Sep 2023 00:02:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1694502139; x=1695106939; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=WHvChKGnnZi2VbMoC/1ghgca6iZmmMi6EKlv8J2G2fg=; b=Pz94yrZEm1gAewdNBZNyHYGilUQYcv52N/6CM7ofREcH9Azf91rO5OrplltOLjm/vK 2IjyqTcsuiHZR2OZlAWJlsf3XCAkc07jw5hrHF3czY017l5o1m+shrQ07R8ALKrHbDsM sqOozaHKk1//v8czq1/Et6jCeIAuhLqU8nfj+qGH4NUWsHn0unUwp4tQAmEIH0PA7Oyg F4tDd+v3U65B5PQXAz+S65m2l8CQVvAKbW20jgwvksYdCtctjwy30kapQemgdGTTH4Bl e/siybWVV/ldUfRAqjdF0+W9DLCPMZtZ++C9BLwYUypljfkYj9xhPuaPoz0TD54DAI2V 4p5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694502139; x=1695106939; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WHvChKGnnZi2VbMoC/1ghgca6iZmmMi6EKlv8J2G2fg=; b=xBx6etKHWxG+Sb+SXLseIvGaY4W37g47KhQswVxq0vJFkCOfmK4vgKPJ/hMTXs3B7j Oks52tzHAYTLiVETbEzyZivcwHUxeLLpWmfSXzHSdZxW9+q74PCfz++El2CRPb3SetvV sRKl0QbEdNc508V0+YJTCBny/T5QcumD1rYFFMDh738fEF8XEBZ7b+GPCZ8T5EsXBDMi hY0/oevgU574Mj+Nx65Lg4Tf5it8DRV4mWd7OVMxeeyOcIRtd0Bs5cnzJKbO1hDpYsui LJTPdLfD38J1E1JdtyhMkVCeeENTtU2hXanPslIOOWL+2lZ+VK6rdvgRSo2e1oCVp0k1 rsiw== X-Gm-Message-State: AOJu0YzZ2qMq+i9S2eSYzEmmG5akluiePOst4fK/wfI0bPrAKj0lTRub 5uPiok79EMT/2T1gWGhap43cwQ== X-Received: by 2002:a17:902:b287:b0:1c3:7628:fcbb with SMTP id u7-20020a170902b28700b001c37628fcbbmr10807814plr.43.1694502139227; Tue, 12 Sep 2023 00:02:19 -0700 (PDT) Received: from n37-019-243.byted.org ([180.184.84.173]) by smtp.gmail.com with ESMTPSA id b8-20020a170902d50800b001b8953365aesm7635401plg.22.2023.09.12.00.02.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Sep 2023 00:02:19 -0700 (PDT) From: Chuyi Zhou To: bpf@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@kernel.org, tj@kernel.org, linux-kernel@vger.kernel.org, Chuyi Zhou Subject: [PATCH bpf-next v2 6/6] selftests/bpf: Add tests for open-coded task and css iter Date: Tue, 12 Sep 2023 15:01:49 +0800 Message-Id: <20230912070149.969939-7-zhouchuyi@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230912070149.969939-1-zhouchuyi@bytedance.com> References: <20230912070149.969939-1-zhouchuyi@bytedance.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Tue, 12 Sep 2023 00:03:00 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777052371121934074 X-GMAIL-MSGID: 1777052371121934074 This patch adds three tests to demonstrate these patterns and validating correctness. test1: We use bpf_for_each(process, task) to iterate all process in the system and search for the current process with a given pid. test2: We create a cgroup and add the current task to the cgroup. In the BPF program, we would use bpf_for_each(css_task, task, css) to iterate all tasks under the cgroup. As expected, we would find the current process. test3: We create a cgroup tree. In the BPF program, we use bpf_for_each(css_{pre,post}, pos, root) to iterate all descendant under the root with pre and post order. As expected, we would find all descendant and the last iterating cgroup in post-order is root cgroup, the first iterating cgroup in pre-order is root cgroup. Signed-off-by: Chuyi Zhou --- .../testing/selftests/bpf/prog_tests/iters.c | 138 ++++++++++++++++++ .../testing/selftests/bpf/progs/iters_task.c | 104 +++++++++++++ 2 files changed, 242 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/iters_task.c diff --git a/tools/testing/selftests/bpf/prog_tests/iters.c b/tools/testing/selftests/bpf/prog_tests/iters.c index 10804ae5ae97..f4e69a506509 100644 --- a/tools/testing/selftests/bpf/prog_tests/iters.c +++ b/tools/testing/selftests/bpf/prog_tests/iters.c @@ -1,13 +1,21 @@ // SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2023 Meta Platforms, Inc. and affiliates. */ +#include +#include +#include +#include +#include +#include #include +#include "cgroup_helpers.h" #include "iters.skel.h" #include "iters_state_safety.skel.h" #include "iters_looping.skel.h" #include "iters_num.skel.h" #include "iters_testmod_seq.skel.h" +#include "iters_task.skel.h" static void subtest_num_iters(void) { @@ -90,6 +98,130 @@ static void subtest_testmod_seq_iters(void) iters_testmod_seq__destroy(skel); } +static void subtest_process_iters(void) +{ + struct iters_task *skel; + int err; + + skel = iters_task__open(); + if (!ASSERT_OK_PTR(skel, "skel_open")) + goto cleanup; + bpf_program__set_autoload(skel->progs.iter_task_for_each_sleep, true); + err = iters_task__load(skel); + if (!ASSERT_OK(err, "skel_load")) + goto cleanup; + skel->bss->target_pid = getpid(); + err = iters_task__attach(skel); + if (!ASSERT_OK(err, "iters_task__attach")) + goto cleanup; + syscall(SYS_getpgid); + iters_task__detach(skel); + ASSERT_EQ(skel->bss->process_cnt, 1, "process_cnt"); + +cleanup: + iters_task__destroy(skel); +} + +extern int stack_mprotect(void); + +static void subtest_css_task_iters(void) +{ + struct iters_task *skel; + int err, cg_fd, cg_id; + const char *cgrp_path = "/cg1"; + + err = setup_cgroup_environment(); + if (!ASSERT_OK(err, "setup_cgroup_environment")) + goto cleanup; + cg_fd = create_and_get_cgroup(cgrp_path); + if (!ASSERT_GE(cg_fd, 0, "cg_create")) + goto cleanup; + cg_id = get_cgroup_id(cgrp_path); + err = join_cgroup(cgrp_path); + if (!ASSERT_OK(err, "setup_cgroup_environment")) + goto cleanup; + + skel = iters_task__open(); + if (!ASSERT_OK_PTR(skel, "skel_open")) + goto cleanup; + + bpf_program__set_autoload(skel->progs.iter_css_task_for_each, true); + err = iters_task__load(skel); + if (!ASSERT_OK(err, "skel_load")) + goto cleanup; + + skel->bss->target_pid = getpid(); + skel->bss->cg_id = cg_id; + err = iters_task__attach(skel); + + err = stack_mprotect(); + if (!ASSERT_OK(err, "iters_task__attach")) + goto cleanup; + + iters_task__detach(skel); + ASSERT_EQ(skel->bss->css_task_cnt, 1, "css_task_cnt"); + +cleanup: + cleanup_cgroup_environment(); + iters_task__destroy(skel); +} + +static void subtest_css_dec_iters(void) +{ + struct iters_task *skel; + struct { + const char *path; + int fd; + } cgs[] = { + { "/cg1" }, + { "/cg1/cg2" }, + { "/cg1/cg2/cg3" }, + { "/cg1/cg2/cg3/cg4" }, + }; + int err, cg_nr = ARRAY_SIZE(cgs); + int i; + + err = setup_cgroup_environment(); + if (!ASSERT_OK(err, "setup_cgroup_environment")) + goto cleanup; + for (i = 0; i < cg_nr; i++) { + cgs[i].fd = create_and_get_cgroup(cgs[i].path); + if (!ASSERT_GE(cgs[i].fd, 0, "cg_create")) + goto cleanup; + } + + skel = iters_task__open(); + if (!ASSERT_OK_PTR(skel, "skel_open")) + goto cleanup; + bpf_program__set_autoload(skel->progs.iter_css_dec_for_each, true); + err = iters_task__load(skel); + if (!ASSERT_OK(err, "skel_load")) + goto cleanup; + + skel->bss->target_pid = getpid(); + skel->bss->cg_id = get_cgroup_id(cgs[0].path); + + err = iters_task__attach(skel); + + if (!ASSERT_OK(err, "iters_task__attach")) + goto cleanup; + + syscall(SYS_getpgid); + ASSERT_EQ(skel->bss->css_dec_cnt, cg_nr, "pre order search dec count"); + ASSERT_EQ(skel->bss->first_cg_id, get_cgroup_id(cgs[0].path), + "pre order search first cgroup id"); + skel->bss->css_dec_cnt = 0; + skel->bss->is_post_order = 1; + syscall(SYS_getpgid); + ASSERT_EQ(skel->bss->css_dec_cnt, cg_nr, "post order search dec count"); + ASSERT_EQ(skel->bss->last_cg_id, get_cgroup_id(cgs[0].path), + "post order search last cgroup id"); + iters_task__detach(skel); +cleanup: + cleanup_cgroup_environment(); + iters_task__destroy(skel); +} + void test_iters(void) { RUN_TESTS(iters_state_safety); @@ -103,4 +235,10 @@ void test_iters(void) subtest_num_iters(); if (test__start_subtest("testmod_seq")) subtest_testmod_seq_iters(); + if (test__start_subtest("process")) + subtest_process_iters(); + if (test__start_subtest("css_task")) + subtest_css_task_iters(); + if (test__start_subtest("css_dec")) + subtest_css_dec_iters(); } diff --git a/tools/testing/selftests/bpf/progs/iters_task.c b/tools/testing/selftests/bpf/progs/iters_task.c new file mode 100644 index 000000000000..cf24a3b177f2 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/iters_task.c @@ -0,0 +1,104 @@ +// SPDX-License-Identifier: GPL-2.0 +#include "vmlinux.h" +#include +#include +#include "bpf_misc.h" +char _license[] SEC("license") = "GPL"; + +pid_t target_pid = 0; +int process_cnt = 0; +int css_task_cnt = 0; +int css_dec_cnt = 0; + +char is_post_order; +u64 cg_id; +u64 last_cg_id; +u64 first_cg_id; + +struct cgroup *bpf_cgroup_from_id(u64 cgid) __ksym; +struct cgroup *bpf_cgroup_acquire(struct cgroup *cgrp) __ksym; +void bpf_cgroup_release(struct cgroup *p) __ksym; +void bpf_rcu_read_lock(void) __ksym; +void bpf_rcu_read_unlock(void) __ksym; + +SEC("?fentry.s/" SYS_PREFIX "sys_getpgid") +int iter_task_for_each_sleep(void *ctx) +{ + struct task_struct *task; + struct task_struct *cur_task = bpf_get_current_task_btf(); + + if (cur_task->pid != target_pid) + return 0; + bpf_rcu_read_lock(); + bpf_for_each(process, task) { + if (task->pid == target_pid) + process_cnt += 1; + } + bpf_rcu_read_unlock(); + return 0; +} + + +SEC("?lsm/file_mprotect") +int BPF_PROG(iter_css_task_for_each) +{ + struct task_struct *task; + struct task_struct *cur_task = bpf_get_current_task_btf(); + + if (cur_task->pid != target_pid) + return 0; + + struct cgroup *cgrp = bpf_cgroup_from_id(cg_id); + + if (cgrp == NULL) + return 0; + struct cgroup_subsys_state *css = &cgrp->self; + + bpf_for_each(css_task, task, css, CSS_TASK_ITER_PROCS) { + if (!task) + continue; + if (task->pid == target_pid) + css_task_cnt += 1; + } + bpf_cgroup_release(cgrp); + return 0; +} + +SEC("?fentry.s/" SYS_PREFIX "sys_getpgid") +int iter_css_dec_for_each(const void *ctx) +{ + struct task_struct *cur_task = bpf_get_current_task_btf(); + + if (cur_task->pid != target_pid) + return 0; + + struct cgroup *cgrp = bpf_cgroup_from_id(cg_id); + + if (cgrp == NULL) + return 0; + struct cgroup_subsys_state *root = &cgrp->self; + struct cgroup_subsys_state *pos = NULL; + + bpf_rcu_read_lock(); + if (is_post_order) { + bpf_for_each(css_post, pos, root) { + struct cgroup *cur_cgrp = pos->cgroup; + + css_dec_cnt += 1; + if (cur_cgrp) + last_cg_id = cur_cgrp->kn->id; + } + } else { + bpf_for_each(css_pre, pos, root) { + struct cgroup *cur_cgrp = pos->cgroup; + + css_dec_cnt += 1; + if (cur_cgrp && !first_cg_id) + first_cg_id = cur_cgrp->kn->id; + } + } + bpf_rcu_read_unlock(); + bpf_cgroup_release(cgrp); + return 0; +} +