From patchwork Thu Feb 8 11:47:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexey Panov X-Patchwork-Id: 198336 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:50ea:b0:106:860b:bbdd with SMTP id r10csp106950dyd; Thu, 8 Feb 2024 03:49:54 -0800 (PST) X-Google-Smtp-Source: AGHT+IFr7PrbMjJRdN/WW+dXqu4/UQkQ/DRf9DLc+egIGX8ItKh5b9raS7w2bdq5sc4boFfV8Rup X-Received: by 2002:a05:6a20:3283:b0:19c:9eea:e731 with SMTP id g3-20020a056a20328300b0019c9eeae731mr5671011pzd.40.1707392994558; Thu, 08 Feb 2024 03:49:54 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707392994; cv=pass; d=google.com; s=arc-20160816; b=MhsjfrKTEMq1gFX4Tg/VGnRbvAnKIasjXCrsb4tzNkoeUkLISnOZuNYfwb3duytjQM G12FLy5zGjfsXBWxyQsBBFJnOsZJhtHrz32efaYqAn0ErqnqPsvBb+M5XJmJQWJkeEk5 83LzV5tpgEk+0PunRg1fJEIMj1QmNxajRxr0yDXTMvmT7E/JcGwMWfPskCIcEwrXa4Qg GfLbhcXnZHYH/6GlXZwLkE4aFseKoQRon+wizD9LJ1moR2iiFXuW3w6nG0WfulfuGaE2 8DDNfC3v/cFxdVbiSuANk2Kqi3iTCyVesZ95JT1MVcK757rCtz1K4rc/PNkwfKOjgs/s prSw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=gx3N0EV537Eg1XluR2QYd9uXobDSiQtLRJmXgTDReJ8=; fh=X80lnV/Ehu2S6GMC5171q4XlPtKTQWVZu7FAHmelwok=; b=BfnZw4T2Sdb/H3G3OYfnJAj33yJehK6jJr4F03qLMVCK7U/2ouFWsJDCjYQSij1qAY 7pza5YFTAI9nT+xBjFNyp2UyyegaRfvuDxY/ukVHObfwdhzRAlZNceIL9979/tkzyPp6 LmRhnOZ0FSeF9wP6uUxpwItSvBhN9f2/J9ikmWM4q4ncZVJ1ZUfVuqiBFAaj+UgWOs1z 7r3NLOQDrc8RJbSIx/LPl5hi+PpxujoU79dDE9XtBB5M7tVZ9gkYRJWeH+hWXIRC6jo2 mheEdOB/fQfAuzi2a9jyyWEjsTQrfH3+6rErd83jpkTTI9hgIP33OIip2Ulo7CjDqTkx URzw==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=astralinux.ru); spf=pass (google.com: domain of linux-kernel+bounces-57988-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-57988-ouuuleilei=gmail.com@vger.kernel.org" X-Forwarded-Encrypted: i=2; AJvYcCUq4/4kPSKbRJ+DzYwENkxhcGf/RZC4v0RyEUdskTupjsthnLKBDnG/5InkdomJMLAo5cpVNkrr1EmqmbqC+S8KLxL1Iw== Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id a9-20020a62d409000000b006e038c15681si3770839pfh.181.2024.02.08.03.49.54 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 08 Feb 2024 03:49:54 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-57988-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=astralinux.ru); spf=pass (google.com: domain of linux-kernel+bounces-57988-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-57988-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id CD2C62867E4 for ; Thu, 8 Feb 2024 11:49:42 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 278B6762C0; Thu, 8 Feb 2024 11:48:35 +0000 (UTC) Received: from new-mail.astralinux.ru (new-mail.astralinux.ru [51.250.53.244]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8FAAE6BFA7; Thu, 8 Feb 2024 11:48:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=51.250.53.244 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707392912; cv=none; b=EJY47f7NCaquddQRBUhpV8x1up1w3iBHRdZ916sOJmIQ+llPihmglTzCq5kVdOoh92OSg9amI3SlTI4jwWmepWxCMDxYpGmTCvBo/66zfJ0avIRlcriOGdt6v2g5QJxl0OGcTsDMYkkAukGQ4HToI0F4ZO29D7UcwMXE4LvGxYw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707392912; c=relaxed/simple; bh=lNnvn42Le0dxfnc9tfCKSniD850SfnLB8+PQT5qb2OY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=faH6LXU8SK1whr8nnapo24JFoFbPx1zLMzysWY9/EU919/8JooHlJpjqepigW8CwnYSOBsd2umTEJFsaszsgHexjw3HLlKt7L70dS0hWH5QYSp6FT8tnSYpYQNngQ5avjAgIRWeZMpf7HljRmjL5B3ySZQAuCWEuMqKj3DfxW3k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=astralinux.ru; spf=pass smtp.mailfrom=astralinux.ru; arc=none smtp.client-ip=51.250.53.244 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=astralinux.ru Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=astralinux.ru Received: from rbta-msk-lt-156703.astralinux.ru (unknown [10.177.233.118]) by new-mail.astralinux.ru (Postfix) with ESMTPA id 4TVwGr1lppzfZDr; Thu, 8 Feb 2024 14:48:20 +0300 (MSK) From: Alexey Panov To: stable@vger.kernel.org, Greg Kroah-Hartman Cc: Alexey Panov , Alexei Starovoitov , Daniel Borkmann , "David S. Miller" , Jakub Kicinski , Jesper Dangaard Brouer , John Fastabend , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , KP Singh , netdev@vger.kernel.org, bpf@vger.kernel.org, linux-kernel@vger.kernel.org, lvc-project@linuxtesting.org, Pu Lehui , Hou Tao , Sasha Levin Subject: [PATCH 5.10 1/2] bpf: cpumap: Fix memory leak in cpu_map_update_elem Date: Thu, 8 Feb 2024 14:47:59 +0300 Message-Id: <20240208114800.27676-2-apanov@astralinux.ru> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240208114800.27676-1-apanov@astralinux.ru> References: <20240208114800.27676-1-apanov@astralinux.ru> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-DrWeb-SpamScore: -100 X-DrWeb-SpamState: legit X-DrWeb-SpamDetail: gggruggvucftvghtrhhoucdtuddrgedvfedrvdehuddgtddvucetufdoteggodetrfcurfhrohhfihhlvgemucfftfghgfeunecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenucfjughrpefhvfevufffkffojghfggfgsedtkeertdertddtnecuhfhrohhmpeetlhgvgigvhicurfgrnhhovhcuoegrphgrnhhovhesrghsthhrrghlihhnuhigrdhruheqnecuggftrfgrthhtvghrnhepteeigfefledvueefgfdvieejgefgieeiiefggeekueejheefvdekvdefhfduudetnecuffhomhgrihhnpehkvghrnhgvlhdrohhrghenucfkphepuddtrddujeejrddvfeefrdduudeknecurfgrrhgrmhephhgvlhhopehrsghtrgdqmhhskhdqlhhtqdduheeijedtfedrrghsthhrrghlihhnuhigrdhruhdpihhnvghtpedutddrudejjedrvdeffedruddukeemgeefjedtvddpmhgrihhlfhhrohhmpegrphgrnhhovhesrghsthhrrghlihhnuhigrdhruhdpnhgspghrtghpthhtohepvddupdhrtghpthhtohepshhtrggslhgvsehvghgvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtohepghhrvghgkhhhsehlihhnuhigfhhouhhnuggrthhiohhnrdhorhhgpdhrtghpthhtoheprghprghnohhvsegrshhtrhgrlhhinhhugidrrhhupdhrtghpthhtoheprghstheskhgvrhhnvghlrdhorhhgpdhrtghpthhtohepuggrnhhivghlsehiohhgvggrrhgsohigrdhnvghtpd hrtghpthhtohepuggrvhgvmhesuggrvhgvmhhlohhfthdrnhgvthdprhgtphhtthhopehkuhgsrgeskhgvrhhnvghlrdhorhhgpdhrtghpthhtohephhgrfihksehkvghrnhgvlhdrohhrghdprhgtphhtthhopehjohhhnhdrfhgrshhtrggsvghnugesghhmrghilhdrtghomhdprhgtphhtthhopegrnhgurhhiiheskhgvrhhnvghlrdhorhhgpdhrtghpthhtohepkhgrfhgrihesfhgsrdgtohhmpdhrtghpthhtohepshhonhhglhhiuhgsrhgrvhhinhhgsehfsgdrtghomhdprhgtphhtthhopeihhhhssehfsgdrtghomhdprhgtphhtthhopehkphhsihhnghhhsehkvghrnhgvlhdrohhrghdprhgtphhtthhopehnvghtuggvvhesvhhgvghrrdhkvghrnhgvlhdrohhrghdprhgtphhtthhopegsphhfsehvghgvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtoheplhhinhhugidqkhgvrhhnvghlsehvghgvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtoheplhhvtgdqphhrohhjvggttheslhhinhhugihtvghsthhinhhgrdhorhhgpdhrtghpthhtohepphhulhgvhhhuiheshhhurgifvghirdgtohhmpdhrtghpthhtohephhhouhhtrghoudeshhhurgifvghirdgtohhmpdhrtghpthhtohepshgrshhhrghlsehkvghrnhgvlhdrohhrgh X-DrWeb-SpamVersion: Vade Retro 01.423.251#02 AS+AV+AP Profile: DRWEB; Bailout: 300 X-AntiVirus: Checked by Dr.Web [MailD: 11.1.19.2307031128, SE: 11.1.12.2210241838, Core engine: 7.00.61.08090, Virus records: 12345016, Updated: 2024-Feb-08 09:34:35 UTC] X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790331308008287321 X-GMAIL-MSGID: 1790331316866246842 From: Pu Lehui [ Upstream commit 4369016497319a9635702da010d02af1ebb1849d ] Syzkaller reported a memory leak as follows: BUG: memory leak unreferenced object 0xff110001198ef748 (size 192): comm "syz-executor.3", pid 17672, jiffies 4298118891 (age 9.906s) hex dump (first 32 bytes): 00 00 00 00 4a 19 00 00 80 ad e3 e4 fe ff c0 00 ....J........... 00 b2 d3 0c 01 00 11 ff 28 f5 8e 19 01 00 11 ff ........(....... backtrace: [] __cpu_map_entry_alloc+0xf7/0xb00 [] cpu_map_update_elem+0x2fe/0x3d0 [] bpf_map_update_value.isra.0+0x2bd/0x520 [] map_update_elem+0x4cb/0x720 [] __se_sys_bpf+0x8c3/0xb90 [] do_syscall_64+0x30/0x40 [] entry_SYSCALL_64_after_hwframe+0x61/0xc6 BUG: memory leak unreferenced object 0xff110001198ef528 (size 192): comm "syz-executor.3", pid 17672, jiffies 4298118891 (age 9.906s) hex dump (first 32 bytes): 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ backtrace: [] __cpu_map_entry_alloc+0x260/0xb00 [] cpu_map_update_elem+0x2fe/0x3d0 [] bpf_map_update_value.isra.0+0x2bd/0x520 [] map_update_elem+0x4cb/0x720 [] __se_sys_bpf+0x8c3/0xb90 [] do_syscall_64+0x30/0x40 [] entry_SYSCALL_64_after_hwframe+0x61/0xc6 BUG: memory leak unreferenced object 0xff1100010fd93d68 (size 8): comm "syz-executor.3", pid 17672, jiffies 4298118891 (age 9.906s) hex dump (first 8 bytes): 00 00 00 00 00 00 00 00 ........ backtrace: [] kvmalloc_node+0x11e/0x170 [] __cpu_map_entry_alloc+0x2f0/0xb00 [] cpu_map_update_elem+0x2fe/0x3d0 [] bpf_map_update_value.isra.0+0x2bd/0x520 [] map_update_elem+0x4cb/0x720 [] __se_sys_bpf+0x8c3/0xb90 [] do_syscall_64+0x30/0x40 [] entry_SYSCALL_64_after_hwframe+0x61/0xc6 In the cpu_map_update_elem flow, when kthread_stop is called before calling the threadfn of rcpu->kthread, since the KTHREAD_SHOULD_STOP bit of kthread has been set by kthread_stop, the threadfn of rcpu->kthread will never be executed, and rcpu->refcnt will never be 0, which will lead to the allocated rcpu, rcpu->queue and rcpu->queue->queue cannot be released. Calling kthread_stop before executing kthread's threadfn will return -EINTR. We can complete the release of memory resources in this state. Fixes: 6710e1126934 ("bpf: introduce new bpf cpu map type BPF_MAP_TYPE_CPUMAP") Signed-off-by: Pu Lehui Acked-by: Jesper Dangaard Brouer Acked-by: Hou Tao Link: https://lore.kernel.org/r/20230711115848.2701559-1-pulehui@huaweicloud.com Signed-off-by: Alexei Starovoitov Signed-off-by: Sasha Levin Signed-off-by: Alexey Panov --- kernel/bpf/cpumap.c | 40 ++++++++++++++++++++++++---------------- 1 file changed, 24 insertions(+), 16 deletions(-) diff --git a/kernel/bpf/cpumap.c b/kernel/bpf/cpumap.c index c61a23b564aa..2097ed9bd583 100644 --- a/kernel/bpf/cpumap.c +++ b/kernel/bpf/cpumap.c @@ -139,22 +139,6 @@ static void get_cpu_map_entry(struct bpf_cpu_map_entry *rcpu) atomic_inc(&rcpu->refcnt); } -/* called from workqueue, to workaround syscall using preempt_disable */ -static void cpu_map_kthread_stop(struct work_struct *work) -{ - struct bpf_cpu_map_entry *rcpu; - - rcpu = container_of(work, struct bpf_cpu_map_entry, kthread_stop_wq); - - /* Wait for flush in __cpu_map_entry_free(), via full RCU barrier, - * as it waits until all in-flight call_rcu() callbacks complete. - */ - rcu_barrier(); - - /* kthread_stop will wake_up_process and wait for it to complete */ - kthread_stop(rcpu->kthread); -} - static struct sk_buff *cpu_map_build_skb(struct xdp_frame *xdpf, struct sk_buff *skb) { @@ -225,6 +209,30 @@ static void put_cpu_map_entry(struct bpf_cpu_map_entry *rcpu) } } +/* called from workqueue, to workaround syscall using preempt_disable */ +static void cpu_map_kthread_stop(struct work_struct *work) +{ + struct bpf_cpu_map_entry *rcpu; + int err; + + rcpu = container_of(work, struct bpf_cpu_map_entry, kthread_stop_wq); + + /* Wait for flush in __cpu_map_entry_free(), via full RCU barrier, + * as it waits until all in-flight call_rcu() callbacks complete. + */ + rcu_barrier(); + + /* kthread_stop will wake_up_process and wait for it to complete */ + err = kthread_stop(rcpu->kthread); + if (err) { + /* kthread_stop may be called before cpu_map_kthread_run + * is executed, so we need to release the memory related + * to rcpu. + */ + put_cpu_map_entry(rcpu); + } +} + static int cpu_map_bpf_prog_run_xdp(struct bpf_cpu_map_entry *rcpu, void **frames, int n, struct xdp_cpumap_stats *stats) From patchwork Thu Feb 8 11:48:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexey Panov X-Patchwork-Id: 198335 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:50ea:b0:106:860b:bbdd with SMTP id r10csp106926dyd; Thu, 8 Feb 2024 03:49:52 -0800 (PST) X-Google-Smtp-Source: AGHT+IEDgZGVGPoMNpj2FDKkRfvpSyT4aYQUN+aP1lyHsgK1FkoeVhEiZGoSwXGQty84vEPczxRN X-Received: by 2002:a05:620a:1925:b0:785:35d3:405f with SMTP id bj37-20020a05620a192500b0078535d3405fmr9990228qkb.48.1707392992013; Thu, 08 Feb 2024 03:49:52 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707392992; cv=pass; d=google.com; s=arc-20160816; b=sue042K06HbXoIhanQ5UP9M5OesI/Ox4kTJaUSw8bCeryqLW5GbFyOKfQCFaFpI5gR BheEjj7ixsfulB0c42NxTnLUoQ08GhDWE9GIt9EcUvzYie6j3yOJfuHaOta0tunnFYmE xpdfS7yWX1HZiXjlyGuIb0WO6uUhA1raQNdALBcIbcdc600emgFu75EvTsDyXQppdK6e NC3zqlVnkr2viSwWd5fFww1NJmg5Dzo64ikGJY5UMa+eVGVHw5qdIMO8Yw/QCY9IEGo9 vZnrr4GnvRmrazOcztE2XPKczzo5Nt96UHTaNYE0cHs0ekv1kG237AYWJQt6pzKWpb2V u1xw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=yXjy3mZG/+6O/uq7mE1bb95IdsmLusZ7VGjAf9dSuAo=; fh=B3C/1Dv9aYMntopR3tdtqvgEPZ3jF17yObHTbsL2b4M=; b=lbUXyUlzLlHu/aXfqmsNjhHc+0HY/KFtrR4oeIft8OAM+d5kPq6BTeiNmkAjJCL8/m eZCIyLfh4hxf14Se6oNRSCyhicrhvMtLf/Y8gGfhGk4gQ5PKHtn/z1uaUiSTXzCYE4Ah 8F7c3jEA6dxw6Rhw4p/LNR5P3s7je4gabDiNtCKY9YAx3+Hpl+gmy7pfJ7xUqrIOA0P+ 3lAHbisqYpvZ4duTgPIrdm1CyFe1/dcGFodLzB/8RtWLvIFEjP9To8t2MIJVnOzmfqj6 OXbZVS90OhCc+boJTIR2+FNa4mytbiahFuYgZIwvppAmorZp0Fwv55LbjGvxCn1kKCQk nHdw==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=astralinux.ru); spf=pass (google.com: domain of linux-kernel+bounces-57989-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-57989-ouuuleilei=gmail.com@vger.kernel.org" X-Forwarded-Encrypted: i=2; AJvYcCWF6atRlKvYo0qFs0W2GqcfgQP11KdCbzI2fn7aJoW9hM8MWc9/WxPCVzPQgqA3TulnI/L8fP60MmB2C7FI3i+vhJ2QOw== Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id c12-20020a05620a164c00b007859c6997b0si3065309qko.214.2024.02.08.03.49.51 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 08 Feb 2024 03:49:51 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-57989-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=astralinux.ru); spf=pass (google.com: domain of linux-kernel+bounces-57989-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-57989-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id B86B31C2252E for ; Thu, 8 Feb 2024 11:49:51 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 2BF66762EA; Thu, 8 Feb 2024 11:48:36 +0000 (UTC) Received: from new-mail.astralinux.ru (new-mail.astralinux.ru [51.250.53.244]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DB34D6F095; Thu, 8 Feb 2024 11:48:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=51.250.53.244 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707392913; cv=none; b=en27UQYCmC2Yn2QlB7RJ62N6ncuvMWYWMp4rTBvqYxAfNTFyh0vmeIPBNTjxm8RnZox82jzxvFIW4pDWWiA6YKZmq4J7UU+7uAFrk2elTe8MPbTil1U8ZFAfOUapsfsWj8bi1Ubq3aWYjqyE0tlJTNzCLt5bNl7mxTjPlRrTNP0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707392913; c=relaxed/simple; bh=tIcxlaynASNxUtEmPx4UvmCBfpJO5p2cP96FiPLo3vU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Jmr8XTvANK5Wtyw3mlEOXmaTDNbeG4BZpzly9WQmbtzj80szs1Fdr2WvN5owodz/A7oQN0aeu65MY0tVd932utFxtJhVupze8YEj8zGmbn74liiV4o5qkG1fEn0stC0pe7kFcN+7X7pxlwc5oppr4VOMbIkN4PZlIKNdvSRVl/Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=astralinux.ru; spf=pass smtp.mailfrom=astralinux.ru; arc=none smtp.client-ip=51.250.53.244 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=astralinux.ru Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=astralinux.ru Received: from rbta-msk-lt-156703.astralinux.ru (unknown [10.177.233.118]) by new-mail.astralinux.ru (Postfix) with ESMTPA id 4TVwGs45C9zfZJT; Thu, 8 Feb 2024 14:48:21 +0300 (MSK) From: Alexey Panov To: stable@vger.kernel.org, Greg Kroah-Hartman Cc: Alexey Panov , Alexei Starovoitov , Daniel Borkmann , "David S. Miller" , Jakub Kicinski , Jesper Dangaard Brouer , John Fastabend , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , KP Singh , netdev@vger.kernel.org, bpf@vger.kernel.org, linux-kernel@vger.kernel.org, lvc-project@linuxtesting.org, Hou Tao , Pu Lehui , Martin KaFai Lau Subject: [PATCH 5.10 2/2] bpf, cpumap: Make sure kthread is running before map update returns Date: Thu, 8 Feb 2024 14:48:00 +0300 Message-Id: <20240208114800.27676-3-apanov@astralinux.ru> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240208114800.27676-1-apanov@astralinux.ru> References: <20240208114800.27676-1-apanov@astralinux.ru> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-DrWeb-SpamScore: -100 X-DrWeb-SpamState: legit X-DrWeb-SpamDetail: gggruggvucftvghtrhhoucdtuddrgedvfedrvdehuddgtddvucetufdoteggodetrfcurfhrohhfihhlvgemucfftfghgfeunecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenucfjughrpefhvfevufffkffojghfggfgsedtkeertdertddtnecuhfhrohhmpeetlhgvgigvhicurfgrnhhovhcuoegrphgrnhhovhesrghsthhrrghlihhnuhigrdhruheqnecuggftrfgrthhtvghrnhepteeigfefledvueefgfdvieejgefgieeiiefggeekueejheefvdekvdefhfduudetnecuffhomhgrihhnpehkvghrnhgvlhdrohhrghenucfkphepuddtrddujeejrddvfeefrdduudeknecurfgrrhgrmhephhgvlhhopehrsghtrgdqmhhskhdqlhhtqdduheeijedtfedrrghsthhrrghlihhnuhigrdhruhdpihhnvghtpedutddrudejjedrvdeffedruddukeemgeefjedtvddpmhgrihhlfhhrohhmpegrphgrnhhovhesrghsthhrrghlihhnuhigrdhruhdpnhgspghrtghpthhtohepvddupdhrtghpthhtohepshhtrggslhgvsehvghgvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtohepghhrvghgkhhhsehlihhnuhigfhhouhhnuggrthhiohhnrdhorhhgpdhrtghpthhtoheprghprghnohhvsegrshhtrhgrlhhinhhugidrrhhupdhrtghpthhtoheprghstheskhgvrhhnvghlrdhorhhgpdhrtghpthhtohepuggrnhhivghlsehiohhgvggrrhgsohigrdhnvghtpd hrtghpthhtohepuggrvhgvmhesuggrvhgvmhhlohhfthdrnhgvthdprhgtphhtthhopehkuhgsrgeskhgvrhhnvghlrdhorhhgpdhrtghpthhtohephhgrfihksehkvghrnhgvlhdrohhrghdprhgtphhtthhopehjohhhnhdrfhgrshhtrggsvghnugesghhmrghilhdrtghomhdprhgtphhtthhopegrnhgurhhiiheskhgvrhhnvghlrdhorhhgpdhrtghpthhtohepkhgrfhgrihesfhgsrdgtohhmpdhrtghpthhtohepshhonhhglhhiuhgsrhgrvhhinhhgsehfsgdrtghomhdprhgtphhtthhopeihhhhssehfsgdrtghomhdprhgtphhtthhopehkphhsihhnghhhsehkvghrnhgvlhdrohhrghdprhgtphhtthhopehnvghtuggvvhesvhhgvghrrdhkvghrnhgvlhdrohhrghdprhgtphhtthhopegsphhfsehvghgvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtoheplhhinhhugidqkhgvrhhnvghlsehvghgvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtoheplhhvtgdqphhrohhjvggttheslhhinhhugihtvghsthhinhhgrdhorhhgpdhrtghpthhtohephhhouhhtrghoudeshhhurgifvghirdgtohhmpdhrtghpthhtohepphhulhgvhhhuiheshhhurgifvghirdgtohhmpdhrtghpthhtohepmhgrrhhtihhnrdhlrghusehkvghrnhgvlhdrohhrgh X-DrWeb-SpamVersion: Vade Retro 01.423.251#02 AS+AV+AP Profile: DRWEB; Bailout: 300 X-AntiVirus: Checked by Dr.Web [MailD: 11.1.19.2307031128, SE: 11.1.12.2210241838, Core engine: 7.00.61.08090, Virus records: 12345016, Updated: 2024-Feb-08 09:34:35 UTC] X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790331314069321745 X-GMAIL-MSGID: 1790331314069321745 From: Hou Tao commit 640a604585aa30f93e39b17d4d6ba69fcb1e66c9 upstream. The following warning was reported when running stress-mode enabled xdp_redirect_cpu with some RT threads: ------------[ cut here ]------------ WARNING: CPU: 4 PID: 65 at kernel/bpf/cpumap.c:135 CPU: 4 PID: 65 Comm: kworker/4:1 Not tainted 6.5.0-rc2+ #1 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996) Workqueue: events cpu_map_kthread_stop RIP: 0010:put_cpu_map_entry+0xda/0x220 ...... Call Trace: ? show_regs+0x65/0x70 ? __warn+0xa5/0x240 ...... ? put_cpu_map_entry+0xda/0x220 cpu_map_kthread_stop+0x41/0x60 process_one_work+0x6b0/0xb80 worker_thread+0x96/0x720 kthread+0x1a5/0x1f0 ret_from_fork+0x3a/0x70 ret_from_fork_asm+0x1b/0x30 The root cause is the same as commit 436901649731 ("bpf: cpumap: Fix memory leak in cpu_map_update_elem"). The kthread is stopped prematurely by kthread_stop() in cpu_map_kthread_stop(), and kthread() doesn't call cpu_map_kthread_run() at all but XDP program has already queued some frames or skbs into ptr_ring. So when __cpu_map_ring_cleanup() checks the ptr_ring, it will find it was not emptied and report a warning. An alternative fix is to use __cpu_map_ring_cleanup() to drop these pending frames or skbs when kthread_stop() returns -EINTR, but it may confuse the user, because these frames or skbs have been handled correctly by XDP program. So instead of dropping these frames or skbs, just make sure the per-cpu kthread is running before __cpu_map_entry_alloc() returns. After apply the fix, the error handle for kthread_stop() will be unnecessary because it will always return 0, so just remove it. Fixes: 6710e1126934 ("bpf: introduce new bpf cpu map type BPF_MAP_TYPE_CPUMAP") Signed-off-by: Hou Tao Reviewed-by: Pu Lehui Acked-by: Jesper Dangaard Brouer Link: https://lore.kernel.org/r/20230729095107.1722450-2-houtao@huaweicloud.com Signed-off-by: Martin KaFai Lau Signed-off-by: Greg Kroah-Hartman Signed-off-by: Alexey Panov --- kernel/bpf/cpumap.c | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/kernel/bpf/cpumap.c b/kernel/bpf/cpumap.c index 2097ed9bd583..f54b7321a7bc 100644 --- a/kernel/bpf/cpumap.c +++ b/kernel/bpf/cpumap.c @@ -25,6 +25,7 @@ #include #include #include +#include #include #include /* netif_receive_skb_core */ @@ -69,6 +70,7 @@ struct bpf_cpu_map_entry { struct rcu_head rcu; struct work_struct kthread_stop_wq; + struct completion kthread_running; }; struct bpf_cpu_map { @@ -213,7 +215,6 @@ static void put_cpu_map_entry(struct bpf_cpu_map_entry *rcpu) static void cpu_map_kthread_stop(struct work_struct *work) { struct bpf_cpu_map_entry *rcpu; - int err; rcpu = container_of(work, struct bpf_cpu_map_entry, kthread_stop_wq); @@ -223,14 +224,7 @@ static void cpu_map_kthread_stop(struct work_struct *work) rcu_barrier(); /* kthread_stop will wake_up_process and wait for it to complete */ - err = kthread_stop(rcpu->kthread); - if (err) { - /* kthread_stop may be called before cpu_map_kthread_run - * is executed, so we need to release the memory related - * to rcpu. - */ - put_cpu_map_entry(rcpu); - } + kthread_stop(rcpu->kthread); } static int cpu_map_bpf_prog_run_xdp(struct bpf_cpu_map_entry *rcpu, @@ -308,6 +302,7 @@ static int cpu_map_kthread_run(void *data) { struct bpf_cpu_map_entry *rcpu = data; + complete(&rcpu->kthread_running); set_current_state(TASK_INTERRUPTIBLE); /* When kthread gives stop order, then rcpu have been disconnected @@ -462,6 +457,7 @@ __cpu_map_entry_alloc(struct bpf_cpumap_val *value, u32 cpu, int map_id) goto free_ptr_ring; /* Setup kthread */ + init_completion(&rcpu->kthread_running); rcpu->kthread = kthread_create_on_node(cpu_map_kthread_run, rcpu, numa, "cpumap/%d/map:%d", cpu, map_id); if (IS_ERR(rcpu->kthread)) @@ -474,6 +470,12 @@ __cpu_map_entry_alloc(struct bpf_cpumap_val *value, u32 cpu, int map_id) kthread_bind(rcpu->kthread, cpu); wake_up_process(rcpu->kthread); + /* Make sure kthread has been running, so kthread_stop() will not + * stop the kthread prematurely and all pending frames or skbs + * will be handled by the kthread before kthread_stop() returns. + */ + wait_for_completion(&rcpu->kthread_running); + return rcpu; free_prog: