From patchwork Wed Oct 19 08:32:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Greg KH X-Patchwork-Id: 4892 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4ac7:0:0:0:0:0 with SMTP id y7csp225511wrs; Wed, 19 Oct 2022 02:43:19 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6tJvqXxjG2A5kjDgJuOv79vIACs4nAheChVqI+cCNam9BLTMQ1W5u42W0z8m1lYQFMJwp/ X-Received: by 2002:a17:906:7952:b0:787:a14d:65a7 with SMTP id l18-20020a170906795200b00787a14d65a7mr6113382ejo.108.1666172599043; Wed, 19 Oct 2022 02:43:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666172599; cv=none; d=google.com; s=arc-20160816; b=v/dPJ/zVJm0Gr7MEzZ1IzWKNqabyjONSX34J6UUUvY1AHrWLfM6mp0w2y650pPQcm5 HPpvuhOl9ea7CuAQIgv0G/P1I+Sx/6S4yWESau3qS7wMovLAekFLndpuMoELE5ibMonl UgS+xXb2DUJRBdh2hunfDZSeksDI0Wpjyq1ItOwadVZZtcJ5PrR2dgRchy16SiCnyavf XWdIhZx/kkPK3E715uemz7pOdD9Nt/S883PTIs89l5qmXzh+TBeL9lxodbdBRAnOCUOD Yb0rf9X+5eyNdTFELFGaiDXteYpovlREOIhMbb4xc2QrrCyVNk5cIKiP4BLxjHIYTIPx t/jw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :user-agent:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=G8XAnjqPKg/5aTCwdOonfTc/QL7bk9dvwrxJiGYcME0=; b=ISisTKzz0Ewca8Bchhf909mH7EheEVfn9yypQeSAid+NgvSkqDToeNke+yBo9HfiQW sCQAWlSj1d1dTMR4+tgYXFjPvXf+1QfNz7T0ee5rVTPAJOm07AI+st6KJdN33KdZsJNn RdHFWm5WC+lJJqE45woGooWIpGFosdbRdimXWhbg4cnAyfuyhtNsn7ztVtexLBxgYCcc bzei+u3TTBCdHL2F/gI3WxD/Fg8G+26VptYxJflwudaTDR9pRd1f3Gn96EAb3LKrYxAQ V4c+ykJJpKZiR/rZubUXa2OAqlc8n0bU7JJ1Dr+ZATPTWthyxDkPbtIfh65XwkMSfciu mCZQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linuxfoundation.org header.s=korg header.b=jK84knL9; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linuxfoundation.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id hc32-20020a17090716a000b0078ddd8656d9si14806570ejc.9.2022.10.19.02.42.53; Wed, 19 Oct 2022 02:43:19 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linuxfoundation.org header.s=korg header.b=jK84knL9; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linuxfoundation.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233915AbiJSJeA (ORCPT + 99 others); Wed, 19 Oct 2022 05:34:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45342 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233771AbiJSJ3Y (ORCPT ); Wed, 19 Oct 2022 05:29:24 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E2C26C58BE; Wed, 19 Oct 2022 02:12:59 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 2CEC661886; Wed, 19 Oct 2022 09:08:17 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 405E4C43144; Wed, 19 Oct 2022 09:08:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1666170496; bh=CM0lsX2xNuDhqw6knoqtt6uLkfZ93Q02MCIk5faF5dE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jK84knL91kszEr50fLxA3xwU0z4n7zg4kM9luGiCByChmSXXjsWMmAdjlqFYLhSiQ leeWYjYcx19R3AepVzsLENSzb0igJ8JUsa1li2D/jO4GVGlnjvgSpYQ5Pywdde4a0s Xxgq2ObgCazgC+eYiksLYeSrY8lF6m6nN5CN9n10= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Nico Pache , Daniel Bristot de Oliveira , "Steven Rostedt (Google)" , Sasha Levin Subject: [PATCH 6.0 645/862] tracing/osnoise: Fix possible recursive locking in stop_per_cpu_kthreads Date: Wed, 19 Oct 2022 10:32:12 +0200 Message-Id: <20221019083318.425224404@linuxfoundation.org> X-Mailer: git-send-email 2.38.0 In-Reply-To: <20221019083249.951566199@linuxfoundation.org> References: <20221019083249.951566199@linuxfoundation.org> User-Agent: quilt/0.67 MIME-Version: 1.0 X-Spam-Status: No, score=-7.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747108599153121135?= X-GMAIL-MSGID: =?utf-8?q?1747108599153121135?= From: Nico Pache [ Upstream commit 99ee9317a1305cd5626736785c8cb38b0e47686c ] There is a recursive lock on the cpu_hotplug_lock. In kernel/trace/trace_osnoise.c:_per_cpu_kthreads: - start_per_cpu_kthreads calls cpus_read_lock() and if start_kthreads returns a error it will call stop_per_cpu_kthreads. - stop_per_cpu_kthreads then calls cpus_read_lock() again causing deadlock. Fix this by calling cpus_read_unlock() before calling stop_per_cpu_kthreads. This behavior can also be seen in commit f46b16520a08 ("trace/hwlat: Implement the per-cpu mode"). This error was noticed during the LTP ftrace-stress-test: WARNING: possible recursive locking detected -------------------------------------------- sh/275006 is trying to acquire lock: ffffffffb02f5400 (cpu_hotplug_lock){++++}-{0:0}, at: stop_per_cpu_kthreads but task is already holding lock: ffffffffb02f5400 (cpu_hotplug_lock){++++}-{0:0}, at: start_per_cpu_kthreads other info that might help us debug this: Possible unsafe locking scenario: CPU0 ---- lock(cpu_hotplug_lock); lock(cpu_hotplug_lock); *** DEADLOCK *** May be due to missing lock nesting notation 3 locks held by sh/275006: #0: ffff8881023f0470 (sb_writers#24){.+.+}-{0:0}, at: ksys_write #1: ffffffffb084f430 (trace_types_lock){+.+.}-{3:3}, at: rb_simple_write #2: ffffffffb02f5400 (cpu_hotplug_lock){++++}-{0:0}, at: start_per_cpu_kthreads Link: https://lkml.kernel.org/r/20220919144932.3064014-1-npache@redhat.com Fixes: c8895e271f79 ("trace/osnoise: Support hotplug operations") Signed-off-by: Nico Pache Acked-by: Daniel Bristot de Oliveira Signed-off-by: Steven Rostedt (Google) Signed-off-by: Sasha Levin --- kernel/trace/trace_osnoise.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/kernel/trace/trace_osnoise.c b/kernel/trace/trace_osnoise.c index 313439920a8c..78d536d3ff3d 100644 --- a/kernel/trace/trace_osnoise.c +++ b/kernel/trace/trace_osnoise.c @@ -1786,8 +1786,9 @@ static int start_per_cpu_kthreads(void) for_each_cpu(cpu, current_mask) { retval = start_kthread(cpu); if (retval) { + cpus_read_unlock(); stop_per_cpu_kthreads(); - break; + return retval; } }