From patchwork Sat Oct 22 07:26:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Greg KH X-Patchwork-Id: 7701 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4242:0:0:0:0:0 with SMTP id s2csp1109355wrr; Sat, 22 Oct 2022 01:48:11 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5LVgBXMsH2B6L4yBODlUPiv2rK9zh7xvkWwhXmZFHTlTta3V1d90eWLfkfUe/kU3SIxBAh X-Received: by 2002:aa7:de9a:0:b0:44d:8191:44c5 with SMTP id j26-20020aa7de9a000000b0044d819144c5mr20944881edv.232.1666428491216; Sat, 22 Oct 2022 01:48:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666428491; cv=none; d=google.com; s=arc-20160816; b=cFNkz4RMUHEs3K8ZiizCJ4F7B6gFqmJIW0QHFeD66Y7xbEd/DtRO9lD3BZ9Yxtopb+ 2wDvw4M9N4yVtnt3vVTnuMLWtkfCpnHdb06eq34MAR57kJGYHau4BwnO/VeQ1QIPON44 ZDcRfKVpjDEp/nVuNd5QR51EahZY7rcCs4iNYOB2M7nGJq5Mp0/aAOWHI2BNEc2MM0r9 r7vbOAT1+opUwx47ZcwB+7L6cDdPjtIr5pzsr9MgQORWV8wtAvx1Bl1SDXMML9y3uSdZ 0dKutTAW1SO5TMEdrCpBbHqJQ0qZO+A9VyaEY4/d0t4M505dstGKXrwY0CdSjG1O0uOu qoBg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :user-agent:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=G8XAnjqPKg/5aTCwdOonfTc/QL7bk9dvwrxJiGYcME0=; b=Ik0xzPkDQZceZ351CsfiS8mVap6FeFofc8Np0MbJUkEFz7bxtEj2Anx7ZY5rf+Au3+ hpldCwGt0jt4M8Vdav6P/cWbprE4erbSg523NPPs+CIN++LzJ71PM64wlb5+dGT+w3s1 yUYnF1Q53KnX/n+IQWjvi6rrafW/42EcGZkafU5IENitLubJAWpeaLH/jxR9UdtHKxTv b4HAJT6vfmxANEcmiCNZviVhkY9aBaauKGsyYOJ64bpYRD9K5EhpEA0JpLqvD/ZlyLbl 7NI1rUt4djCDwqMxsbi8eGHuOkSBAVe6L0e/LFemWUmgk7cR1NzUWmAJiGs/1egSEEcD kkEQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linuxfoundation.org header.s=korg header.b=xQdN2hfd; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linuxfoundation.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id t20-20020a50ab54000000b0045fd458991fsi6543520edc.428.2022.10.22.01.47.47; Sat, 22 Oct 2022 01:48:11 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linuxfoundation.org header.s=korg header.b=xQdN2hfd; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linuxfoundation.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234469AbiJVIrP (ORCPT + 99 others); Sat, 22 Oct 2022 04:47:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42352 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235138AbiJVIom (ORCPT ); Sat, 22 Oct 2022 04:44:42 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DDD8B2E9E04; Sat, 22 Oct 2022 01:08:35 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 36EEEB82DFA; Sat, 22 Oct 2022 08:00:12 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3C658C433D6; Sat, 22 Oct 2022 08:00:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1666425610; bh=CM0lsX2xNuDhqw6knoqtt6uLkfZ93Q02MCIk5faF5dE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=xQdN2hfdKHwuS2aAQ6flNLsTnWE1TX3KJOfdG7aS8m44B5pLou48sqP/5nS5rIzVK ilmcKhrcYgr16nvJUWf0YAlI7cxX21lH+KsLjVxDp6JOVkWe0A8SfZpUW7ABqKENMG pRfMINDxELonqmFgmnDR090A3vuY3ZLXtGVMnSt4= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Nico Pache , Daniel Bristot de Oliveira , "Steven Rostedt (Google)" , Sasha Levin Subject: [PATCH 5.19 527/717] tracing/osnoise: Fix possible recursive locking in stop_per_cpu_kthreads Date: Sat, 22 Oct 2022 09:26:46 +0200 Message-Id: <20221022072521.625085247@linuxfoundation.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221022072415.034382448@linuxfoundation.org> References: <20221022072415.034382448@linuxfoundation.org> User-Agent: quilt/0.67 MIME-Version: 1.0 X-Spam-Status: No, score=-7.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747376921368615739?= X-GMAIL-MSGID: =?utf-8?q?1747376921368615739?= From: Nico Pache [ Upstream commit 99ee9317a1305cd5626736785c8cb38b0e47686c ] There is a recursive lock on the cpu_hotplug_lock. In kernel/trace/trace_osnoise.c:_per_cpu_kthreads: - start_per_cpu_kthreads calls cpus_read_lock() and if start_kthreads returns a error it will call stop_per_cpu_kthreads. - stop_per_cpu_kthreads then calls cpus_read_lock() again causing deadlock. Fix this by calling cpus_read_unlock() before calling stop_per_cpu_kthreads. This behavior can also be seen in commit f46b16520a08 ("trace/hwlat: Implement the per-cpu mode"). This error was noticed during the LTP ftrace-stress-test: WARNING: possible recursive locking detected -------------------------------------------- sh/275006 is trying to acquire lock: ffffffffb02f5400 (cpu_hotplug_lock){++++}-{0:0}, at: stop_per_cpu_kthreads but task is already holding lock: ffffffffb02f5400 (cpu_hotplug_lock){++++}-{0:0}, at: start_per_cpu_kthreads other info that might help us debug this: Possible unsafe locking scenario: CPU0 ---- lock(cpu_hotplug_lock); lock(cpu_hotplug_lock); *** DEADLOCK *** May be due to missing lock nesting notation 3 locks held by sh/275006: #0: ffff8881023f0470 (sb_writers#24){.+.+}-{0:0}, at: ksys_write #1: ffffffffb084f430 (trace_types_lock){+.+.}-{3:3}, at: rb_simple_write #2: ffffffffb02f5400 (cpu_hotplug_lock){++++}-{0:0}, at: start_per_cpu_kthreads Link: https://lkml.kernel.org/r/20220919144932.3064014-1-npache@redhat.com Fixes: c8895e271f79 ("trace/osnoise: Support hotplug operations") Signed-off-by: Nico Pache Acked-by: Daniel Bristot de Oliveira Signed-off-by: Steven Rostedt (Google) Signed-off-by: Sasha Levin --- kernel/trace/trace_osnoise.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/kernel/trace/trace_osnoise.c b/kernel/trace/trace_osnoise.c index 313439920a8c..78d536d3ff3d 100644 --- a/kernel/trace/trace_osnoise.c +++ b/kernel/trace/trace_osnoise.c @@ -1786,8 +1786,9 @@ static int start_per_cpu_kthreads(void) for_each_cpu(cpu, current_mask) { retval = start_kthread(cpu); if (retval) { + cpus_read_unlock(); stop_per_cpu_kthreads(); - break; + return retval; } }