Message ID | 20230814093813.19152-2-hejunhao3@huawei.com |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b824:0:b0:3f2:4152:657d with SMTP id z4csp2655433vqi; Mon, 14 Aug 2023 03:47:20 -0700 (PDT) X-Google-Smtp-Source: AGHT+IENIT7s9T/ENaZIagu4/Rje7nts4GFX8XwwdhOQapn9sqsKxE2gv4yYbL/FSRgkiCtr2Si4 X-Received: by 2002:a17:90a:db0b:b0:268:5c3c:866f with SMTP id g11-20020a17090adb0b00b002685c3c866fmr6280156pjv.46.1692010040022; Mon, 14 Aug 2023 03:47:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692010040; cv=none; d=google.com; s=arc-20160816; b=PiVI3FpSNxdc5Z8C1s6AMIzkBDR6ifPsstfrTT419gbc20FJmsSNkRYVGer/PHHr5Q 8+arKeKEMp7rd7KSICdh23UFf1jZoqFxBHtnEM0yGGwSvEaW8Xfq//dJjkih72od7A+z EMf4PLsNbGtdVvG4J2It1zwDiLl98WUNz3rws23bqPtEWblC5AfywRNGWxUNO0a5De7c 3Tz1TaDW9RShBtJy0iQXpIVUosFg3ozB0DVXnFXwQHsY9xL+QK069matpbrwAo5WWmHg y+S3wo+OIOKmBPMbroPoxm4PpfG2ugX84R61vwqL1f+fI5Uz8kyJI6TY2ff9NNc7EFJl OUjQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=uH0gV4+AghvOGdZvAcoyHndwcwS1UhnnyuABjQbVs5U=; fh=qw43QZAXcNcl7h9B3MdyBVec7mySpMzzcuDhZUv6g+o=; b=kafT8evIYb+NPR4xJWY+hvPd8RmF/cV9mw7wtNFduPY7RPSWOd/Z2qx7pmdoO6TjAg CWPxNluccgZnsTP44l/ahw+ZGZLI9rTdeszi68Xa+FjhcAyRxlC3lXrLkfY4RMaHFoAx BItpHwLbMhsvrZVk4t5G8N+Jef/TfvCzlsKNpRJOGv4RdwXaevivsN6IuBUk9HtZnyEC imRD1uccAgCxbKE6jGnrcfvx+ITRPB4CiJIvRvjnUyappScR09tB/3W7VN6D+rIalq3N RH/S8dAdsAGtc8xUdTcDW2srOCQcMZqEKODgftdLx3bNqwlqarTUfwoVRo8He2S0wsG1 /5HA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id e65-20020a17090a6fc700b002636ac73ffdsi10273240pjk.63.2023.08.14.03.47.05; Mon, 14 Aug 2023 03:47:20 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233418AbjHNJlT (ORCPT <rfc822;274620705z@gmail.com> + 99 others); Mon, 14 Aug 2023 05:41:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49814 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234036AbjHNJlH (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Mon, 14 Aug 2023 05:41:07 -0400 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E6D8D109 for <linux-kernel@vger.kernel.org>; Mon, 14 Aug 2023 02:41:05 -0700 (PDT) Received: from dggpeml500002.china.huawei.com (unknown [172.30.72.53]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4RPTp15yTcztS0h; Mon, 14 Aug 2023 17:37:29 +0800 (CST) Received: from localhost.localdomain (10.69.192.56) by dggpeml500002.china.huawei.com (7.185.36.158) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Mon, 14 Aug 2023 17:41:04 +0800 From: Junhao He <hejunhao3@huawei.com> To: <suzuki.poulose@arm.com>, <mike.leach@linaro.org>, <leo.yan@linaro.org>, <anshuman.khandual@arm.com>, <jonathan.cameron@huawei.com> CC: <coresight@lists.linaro.org>, <linux-arm-kernel@lists.infradead.org>, <linux-kernel@vger.kernel.org>, <linuxarm@huawei.com>, <yangyicong@huawei.com>, <prime.zeng@hisilicon.com>, <hejunhao3@huawei.com> Subject: [PATCH 1/2] coresight: trbe: Fix TRBE potential sleep in atomic context Date: Mon, 14 Aug 2023 17:38:12 +0800 Message-ID: <20230814093813.19152-2-hejunhao3@huawei.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20230814093813.19152-1-hejunhao3@huawei.com> References: <20230814093813.19152-1-hejunhao3@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7BIT Content-Type: text/plain; charset=US-ASCII X-Originating-IP: [10.69.192.56] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To dggpeml500002.china.huawei.com (7.185.36.158) X-CFilter-Loop: Reflected X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1774201119773886078 X-GMAIL-MSGID: 1774201119773886078 |
Series |
Fix some issues with TRBE building as a module
|
|
Commit Message
hejunhao
Aug. 14, 2023, 9:38 a.m. UTC
smp_call_function_single() will allocate an IPI interrupt vector to
the target processor and send a function call request to the interrupt
vector. After the target processor receives the IPI interrupt, it will
execute arm_trbe_remove_coresight_cpu() call request in the interrupt
handler.
According to the device_unregister() stack information, if other process
is useing the device, the down_write() may sleep, and trigger deadlocks
or unexpected errors.
arm_trbe_remove_coresight_cpu
coresight_unregister
device_unregister
device_del
kobject_del
__kobject_del
sysfs_remove_dir
kernfs_remove
down_write ---------> it may sleep
Add a helper arm_trbe_disable_cpu() to disable TRBE precpu irq and reset
per TRBE.
Simply call arm_trbe_remove_coresight_cpu() directly without useing the
smp_call_function_single(), which is the same as registering the TRBE
coresight device.
Fixes: 3fbf7f011f24 ("coresight: sink: Add TRBE driver")
Signed-off-by: Junhao He <hejunhao3@huawei.com>
---
drivers/hwtracing/coresight/coresight-trbe.c | 35 +++++++++++---------
1 file changed, 20 insertions(+), 15 deletions(-)
Comments
Hi Junhao On 14/08/2023 10:38, Junhao He wrote: > smp_call_function_single() will allocate an IPI interrupt vector to > the target processor and send a function call request to the interrupt > vector. After the target processor receives the IPI interrupt, it will > execute arm_trbe_remove_coresight_cpu() call request in the interrupt > handler. > > According to the device_unregister() stack information, if other process > is useing the device, the down_write() may sleep, and trigger deadlocks > or unexpected errors. > > arm_trbe_remove_coresight_cpu > coresight_unregister > device_unregister > device_del > kobject_del > __kobject_del > sysfs_remove_dir > kernfs_remove > down_write ---------> it may sleep > > Add a helper arm_trbe_disable_cpu() to disable TRBE precpu irq and reset > per TRBE. > Simply call arm_trbe_remove_coresight_cpu() directly without useing the > smp_call_function_single(), which is the same as registering the TRBE > coresight device. > > Fixes: 3fbf7f011f24 ("coresight: sink: Add TRBE driver") > Signed-off-by: Junhao He <hejunhao3@huawei.com> > --- > drivers/hwtracing/coresight/coresight-trbe.c | 35 +++++++++++--------- > 1 file changed, 20 insertions(+), 15 deletions(-) > > diff --git a/drivers/hwtracing/coresight/coresight-trbe.c b/drivers/hwtracing/coresight/coresight-trbe.c > index 7720619909d6..ce1e6f537b8d 100644 > --- a/drivers/hwtracing/coresight/coresight-trbe.c > +++ b/drivers/hwtracing/coresight/coresight-trbe.c > @@ -1225,6 +1225,17 @@ static void arm_trbe_enable_cpu(void *info) > enable_percpu_irq(drvdata->irq, IRQ_TYPE_NONE); > } > > +static void arm_trbe_disable_cpu(void *info) > +{ > + struct trbe_drvdata *drvdata = info; > + struct trbe_cpudata *cpudata = this_cpu_ptr(drvdata->cpudata); > + > + disable_percpu_irq(drvdata->irq); > + trbe_reset_local(cpudata); > + cpudata->drvdata = NULL; > +} > + > + > static void arm_trbe_register_coresight_cpu(struct trbe_drvdata *drvdata, int cpu) > { > struct trbe_cpudata *cpudata = per_cpu_ptr(drvdata->cpudata, cpu); > @@ -1326,18 +1337,12 @@ static void arm_trbe_probe_cpu(void *info) > cpumask_clear_cpu(cpu, &drvdata->supported_cpus); > } > > -static void arm_trbe_remove_coresight_cpu(void *info) > +static void arm_trbe_remove_coresight_cpu(struct trbe_drvdata *drvdata, int cpu) > { > - int cpu = smp_processor_id(); > - struct trbe_drvdata *drvdata = info; > - struct trbe_cpudata *cpudata = per_cpu_ptr(drvdata->cpudata, cpu); > struct coresight_device *trbe_csdev = coresight_get_percpu_sink(cpu); > > - disable_percpu_irq(drvdata->irq); > - trbe_reset_local(cpudata); > if (trbe_csdev) { > coresight_unregister(trbe_csdev); > - cpudata->drvdata = NULL; > coresight_set_percpu_sink(cpu, NULL); I am a bit concerned about "resetting" the sink from a different CPU. Could we instead, schedule a delayed work to unregister the trbe_csdev? Suzuki > } > } > @@ -1366,8 +1371,12 @@ static int arm_trbe_remove_coresight(struct trbe_drvdata *drvdata) > { > int cpu; > > - for_each_cpu(cpu, &drvdata->supported_cpus) > - smp_call_function_single(cpu, arm_trbe_remove_coresight_cpu, drvdata, 1); > + for_each_cpu(cpu, &drvdata->supported_cpus) { > + if (cpumask_test_cpu(cpu, &drvdata->supported_cpus)) > + smp_call_function_single(cpu, arm_trbe_disable_cpu, drvdata, 1); > + if (cpumask_test_cpu(cpu, &drvdata->supported_cpus)) > + arm_trbe_remove_coresight_cpu(drvdata, cpu); > + } > free_percpu(drvdata->cpudata); > return 0; > } > @@ -1406,12 +1415,8 @@ static int arm_trbe_cpu_teardown(unsigned int cpu, struct hlist_node *node) > { > struct trbe_drvdata *drvdata = hlist_entry_safe(node, struct trbe_drvdata, hotplug_node); > > - if (cpumask_test_cpu(cpu, &drvdata->supported_cpus)) { > - struct trbe_cpudata *cpudata = per_cpu_ptr(drvdata->cpudata, cpu); > - > - disable_percpu_irq(drvdata->irq); > - trbe_reset_local(cpudata); > - } > + if (cpumask_test_cpu(cpu, &drvdata->supported_cpus)) > + arm_trbe_disable_cpu(drvdata); > return 0; > } >
Hi Suzuki On 2023/8/14 18:34, Suzuki K Poulose wrote: > Hi Junhao > > On 14/08/2023 10:38, Junhao He wrote: >> smp_call_function_single() will allocate an IPI interrupt vector to >> the target processor and send a function call request to the interrupt >> vector. After the target processor receives the IPI interrupt, it will >> execute arm_trbe_remove_coresight_cpu() call request in the interrupt >> handler. >> >> According to the device_unregister() stack information, if other process >> is useing the device, the down_write() may sleep, and trigger deadlocks >> or unexpected errors. >> >> arm_trbe_remove_coresight_cpu >> coresight_unregister >> device_unregister >> device_del >> kobject_del >> __kobject_del >> sysfs_remove_dir >> kernfs_remove >> down_write ---------> it may sleep >> >> Add a helper arm_trbe_disable_cpu() to disable TRBE precpu irq and reset >> per TRBE. >> Simply call arm_trbe_remove_coresight_cpu() directly without useing the >> smp_call_function_single(), which is the same as registering the TRBE >> coresight device. >> >> Fixes: 3fbf7f011f24 ("coresight: sink: Add TRBE driver") >> Signed-off-by: Junhao He <hejunhao3@huawei.com> >> --- >> drivers/hwtracing/coresight/coresight-trbe.c | 35 +++++++++++--------- >> 1 file changed, 20 insertions(+), 15 deletions(-) >> >> diff --git a/drivers/hwtracing/coresight/coresight-trbe.c >> b/drivers/hwtracing/coresight/coresight-trbe.c >> index 7720619909d6..ce1e6f537b8d 100644 >> --- a/drivers/hwtracing/coresight/coresight-trbe.c >> +++ b/drivers/hwtracing/coresight/coresight-trbe.c >> @@ -1225,6 +1225,17 @@ static void arm_trbe_enable_cpu(void *info) >> enable_percpu_irq(drvdata->irq, IRQ_TYPE_NONE); >> } >> +static void arm_trbe_disable_cpu(void *info) >> +{ >> + struct trbe_drvdata *drvdata = info; >> + struct trbe_cpudata *cpudata = this_cpu_ptr(drvdata->cpudata); >> + >> + disable_percpu_irq(drvdata->irq); >> + trbe_reset_local(cpudata); >> + cpudata->drvdata = NULL; >> +} >> + >> + >> static void arm_trbe_register_coresight_cpu(struct trbe_drvdata >> *drvdata, int cpu) >> { >> struct trbe_cpudata *cpudata = per_cpu_ptr(drvdata->cpudata, cpu); >> @@ -1326,18 +1337,12 @@ static void arm_trbe_probe_cpu(void *info) >> cpumask_clear_cpu(cpu, &drvdata->supported_cpus); >> } >> -static void arm_trbe_remove_coresight_cpu(void *info) >> +static void arm_trbe_remove_coresight_cpu(struct trbe_drvdata >> *drvdata, int cpu) >> { >> - int cpu = smp_processor_id(); >> - struct trbe_drvdata *drvdata = info; >> - struct trbe_cpudata *cpudata = per_cpu_ptr(drvdata->cpudata, cpu); >> struct coresight_device *trbe_csdev = >> coresight_get_percpu_sink(cpu); >> - disable_percpu_irq(drvdata->irq); >> - trbe_reset_local(cpudata); >> if (trbe_csdev) { >> coresight_unregister(trbe_csdev); >> - cpudata->drvdata = NULL; >> coresight_set_percpu_sink(cpu, NULL); > > I am a bit concerned about "resetting" the sink from a different CPU. > Could we instead, schedule a delayed work to unregister the trbe_csdev? Yes, I will try to do that. Sorry for my following questions. As you mean, do we need to take the same care when setting the percpu sink in the register trbe_csdev ? Best regards, Junhao. > > >> } >> } >> @@ -1366,8 +1371,12 @@ static int arm_trbe_remove_coresight(struct >> trbe_drvdata *drvdata) >> { >> int cpu; >> - for_each_cpu(cpu, &drvdata->supported_cpus) >> - smp_call_function_single(cpu, arm_trbe_remove_coresight_cpu, >> drvdata, 1); >> + for_each_cpu(cpu, &drvdata->supported_cpus) { >> + if (cpumask_test_cpu(cpu, &drvdata->supported_cpus)) >> + smp_call_function_single(cpu, arm_trbe_disable_cpu, >> drvdata, 1); >> + if (cpumask_test_cpu(cpu, &drvdata->supported_cpus)) >> + arm_trbe_remove_coresight_cpu(drvdata, cpu); >> + } >> free_percpu(drvdata->cpudata); >> return 0; >> } >> @@ -1406,12 +1415,8 @@ static int arm_trbe_cpu_teardown(unsigned int >> cpu, struct hlist_node *node) >> { >> struct trbe_drvdata *drvdata = hlist_entry_safe(node, struct >> trbe_drvdata, hotplug_node); >> - if (cpumask_test_cpu(cpu, &drvdata->supported_cpus)) { >> - struct trbe_cpudata *cpudata = per_cpu_ptr(drvdata->cpudata, >> cpu); >> - >> - disable_percpu_irq(drvdata->irq); >> - trbe_reset_local(cpudata); >> - } >> + if (cpumask_test_cpu(cpu, &drvdata->supported_cpus)) >> + arm_trbe_disable_cpu(drvdata); >> return 0; >> } > > > . >
On 14/08/2023 14:32, hejunhao wrote: > Hi Suzuki > > > On 2023/8/14 18:34, Suzuki K Poulose wrote: >> Hi Junhao >> >> On 14/08/2023 10:38, Junhao He wrote: >>> smp_call_function_single() will allocate an IPI interrupt vector to >>> the target processor and send a function call request to the interrupt >>> vector. After the target processor receives the IPI interrupt, it will >>> execute arm_trbe_remove_coresight_cpu() call request in the interrupt >>> handler. >>> >>> According to the device_unregister() stack information, if other process >>> is useing the device, the down_write() may sleep, and trigger deadlocks >>> or unexpected errors. >>> >>> arm_trbe_remove_coresight_cpu >>> coresight_unregister >>> device_unregister >>> device_del >>> kobject_del >>> __kobject_del >>> sysfs_remove_dir >>> kernfs_remove >>> down_write ---------> it may sleep >>> >>> Add a helper arm_trbe_disable_cpu() to disable TRBE precpu irq and reset >>> per TRBE. >>> Simply call arm_trbe_remove_coresight_cpu() directly without useing the >>> smp_call_function_single(), which is the same as registering the TRBE >>> coresight device. >>> >>> Fixes: 3fbf7f011f24 ("coresight: sink: Add TRBE driver") >>> Signed-off-by: Junhao He <hejunhao3@huawei.com> >>> --- >>> drivers/hwtracing/coresight/coresight-trbe.c | 35 +++++++++++--------- >>> 1 file changed, 20 insertions(+), 15 deletions(-) >>> >>> diff --git a/drivers/hwtracing/coresight/coresight-trbe.c >>> b/drivers/hwtracing/coresight/coresight-trbe.c >>> index 7720619909d6..ce1e6f537b8d 100644 >>> --- a/drivers/hwtracing/coresight/coresight-trbe.c >>> +++ b/drivers/hwtracing/coresight/coresight-trbe.c >>> @@ -1225,6 +1225,17 @@ static void arm_trbe_enable_cpu(void *info) >>> enable_percpu_irq(drvdata->irq, IRQ_TYPE_NONE); >>> } >>> +static void arm_trbe_disable_cpu(void *info) >>> +{ >>> + struct trbe_drvdata *drvdata = info; >>> + struct trbe_cpudata *cpudata = this_cpu_ptr(drvdata->cpudata); >>> + >>> + disable_percpu_irq(drvdata->irq); >>> + trbe_reset_local(cpudata); >>> + cpudata->drvdata = NULL; >>> +} >>> + >>> + >>> static void arm_trbe_register_coresight_cpu(struct trbe_drvdata >>> *drvdata, int cpu) >>> { >>> struct trbe_cpudata *cpudata = per_cpu_ptr(drvdata->cpudata, cpu); >>> @@ -1326,18 +1337,12 @@ static void arm_trbe_probe_cpu(void *info) >>> cpumask_clear_cpu(cpu, &drvdata->supported_cpus); >>> } >>> -static void arm_trbe_remove_coresight_cpu(void *info) >>> +static void arm_trbe_remove_coresight_cpu(struct trbe_drvdata >>> *drvdata, int cpu) >>> { >>> - int cpu = smp_processor_id(); >>> - struct trbe_drvdata *drvdata = info; >>> - struct trbe_cpudata *cpudata = per_cpu_ptr(drvdata->cpudata, cpu); >>> struct coresight_device *trbe_csdev = >>> coresight_get_percpu_sink(cpu); >>> - disable_percpu_irq(drvdata->irq); >>> - trbe_reset_local(cpudata); >>> if (trbe_csdev) { >>> coresight_unregister(trbe_csdev); >>> - cpudata->drvdata = NULL; >>> coresight_set_percpu_sink(cpu, NULL); >> >> I am a bit concerned about "resetting" the sink from a different CPU. >> Could we instead, schedule a delayed work to unregister the trbe_csdev? > > Yes, I will try to do that. > Sorry for my following questions. > As you mean, do we need to take the same care when setting the percpu sink > in the register trbe_csdev ? Apologies, having taken another look, we set the percpu_sink for a cpu outside smp_call_function(). So, I think your patch is fine. > > Best regards, > Junhao. > >> >> >>> } >>> } >>> @@ -1366,8 +1371,12 @@ static int arm_trbe_remove_coresight(struct >>> trbe_drvdata *drvdata) >>> { >>> int cpu; >>> - for_each_cpu(cpu, &drvdata->supported_cpus) >>> - smp_call_function_single(cpu, arm_trbe_remove_coresight_cpu, >>> drvdata, 1); >>> + for_each_cpu(cpu, &drvdata->supported_cpus) { >>> + if (cpumask_test_cpu(cpu, &drvdata->supported_cpus)) >>> + smp_call_function_single(cpu, arm_trbe_disable_cpu, >>> drvdata, 1); >>> + if (cpumask_test_cpu(cpu, &drvdata->supported_cpus)) >>> + arm_trbe_remove_coresight_cpu(drvdata, cpu); Do we need to test the cpu here in both places ? We already check that in the loop entry. The reason why we repeat the check during the probe, is to skip any CPUs that may have a TRBE not accessible. Suzuki >>> + } >>> free_percpu(drvdata->cpudata); >>> return 0; >>> } >>> @@ -1406,12 +1415,8 @@ static int arm_trbe_cpu_teardown(unsigned int >>> cpu, struct hlist_node *node) >>> { >>> struct trbe_drvdata *drvdata = hlist_entry_safe(node, struct >>> trbe_drvdata, hotplug_node); >>> - if (cpumask_test_cpu(cpu, &drvdata->supported_cpus)) { >>> - struct trbe_cpudata *cpudata = per_cpu_ptr(drvdata->cpudata, >>> cpu); >>> - >>> - disable_percpu_irq(drvdata->irq); >>> - trbe_reset_local(cpudata); >>> - } >>> + if (cpumask_test_cpu(cpu, &drvdata->supported_cpus)) >>> + arm_trbe_disable_cpu(drvdata); >>> return 0; >>> } >> >> >> . >> >
Hi Suzuki On 2023/8/15 6:57, Suzuki K Poulose wrote: > On 14/08/2023 14:32, hejunhao wrote: >> Hi Suzuki >> >> >> On 2023/8/14 18:34, Suzuki K Poulose wrote: >>> Hi Junhao >>> >>> On 14/08/2023 10:38, Junhao He wrote: >>>> smp_call_function_single() will allocate an IPI interrupt vector to >>>> the target processor and send a function call request to the interrupt >>>> vector. After the target processor receives the IPI interrupt, it will >>>> execute arm_trbe_remove_coresight_cpu() call request in the interrupt >>>> handler. >>>> >>>> According to the device_unregister() stack information, if other >>>> process >>>> is useing the device, the down_write() may sleep, and trigger >>>> deadlocks >>>> or unexpected errors. >>>> >>>> arm_trbe_remove_coresight_cpu >>>> coresight_unregister >>>> device_unregister >>>> device_del >>>> kobject_del >>>> __kobject_del >>>> sysfs_remove_dir >>>> kernfs_remove >>>> down_write ---------> it may sleep >>>> >>>> Add a helper arm_trbe_disable_cpu() to disable TRBE precpu irq and >>>> reset >>>> per TRBE. >>>> Simply call arm_trbe_remove_coresight_cpu() directly without useing >>>> the >>>> smp_call_function_single(), which is the same as registering the TRBE >>>> coresight device. >>>> >>>> Fixes: 3fbf7f011f24 ("coresight: sink: Add TRBE driver") >>>> Signed-off-by: Junhao He <hejunhao3@huawei.com> >>>> --- >>>> drivers/hwtracing/coresight/coresight-trbe.c | 35 >>>> +++++++++++--------- >>>> 1 file changed, 20 insertions(+), 15 deletions(-) >>>> >>>> diff --git a/drivers/hwtracing/coresight/coresight-trbe.c >>>> b/drivers/hwtracing/coresight/coresight-trbe.c >>>> index 7720619909d6..ce1e6f537b8d 100644 >>>> --- a/drivers/hwtracing/coresight/coresight-trbe.c >>>> +++ b/drivers/hwtracing/coresight/coresight-trbe.c >>>> @@ -1225,6 +1225,17 @@ static void arm_trbe_enable_cpu(void *info) >>>> enable_percpu_irq(drvdata->irq, IRQ_TYPE_NONE); >>>> } >>>> +static void arm_trbe_disable_cpu(void *info) >>>> +{ >>>> + struct trbe_drvdata *drvdata = info; >>>> + struct trbe_cpudata *cpudata = this_cpu_ptr(drvdata->cpudata); >>>> + >>>> + disable_percpu_irq(drvdata->irq); >>>> + trbe_reset_local(cpudata); >>>> + cpudata->drvdata = NULL; >>>> +} >>>> + >>>> + >>>> static void arm_trbe_register_coresight_cpu(struct trbe_drvdata >>>> *drvdata, int cpu) >>>> { >>>> struct trbe_cpudata *cpudata = per_cpu_ptr(drvdata->cpudata, >>>> cpu); >>>> @@ -1326,18 +1337,12 @@ static void arm_trbe_probe_cpu(void *info) >>>> cpumask_clear_cpu(cpu, &drvdata->supported_cpus); >>>> } >>>> -static void arm_trbe_remove_coresight_cpu(void *info) >>>> +static void arm_trbe_remove_coresight_cpu(struct trbe_drvdata >>>> *drvdata, int cpu) >>>> { >>>> - int cpu = smp_processor_id(); >>>> - struct trbe_drvdata *drvdata = info; >>>> - struct trbe_cpudata *cpudata = per_cpu_ptr(drvdata->cpudata, >>>> cpu); >>>> struct coresight_device *trbe_csdev = >>>> coresight_get_percpu_sink(cpu); >>>> - disable_percpu_irq(drvdata->irq); >>>> - trbe_reset_local(cpudata); >>>> if (trbe_csdev) { >>>> coresight_unregister(trbe_csdev); >>>> - cpudata->drvdata = NULL; >>>> coresight_set_percpu_sink(cpu, NULL); >>> >>> I am a bit concerned about "resetting" the sink from a different CPU. >>> Could we instead, schedule a delayed work to unregister the trbe_csdev? >> >> Yes, I will try to do that. >> Sorry for my following questions. >> As you mean, do we need to take the same care when setting the percpu >> sink >> in the register trbe_csdev ? > > Apologies, having taken another look, we set the percpu_sink for > a cpu outside smp_call_function(). So, I think your patch is fine. > > >> >> Best regards, >> Junhao. >> >>> >>> >>>> } >>>> } >>>> @@ -1366,8 +1371,12 @@ static int arm_trbe_remove_coresight(struct >>>> trbe_drvdata *drvdata) >>>> { >>>> int cpu; >>>> - for_each_cpu(cpu, &drvdata->supported_cpus) >>>> - smp_call_function_single(cpu, >>>> arm_trbe_remove_coresight_cpu, drvdata, 1); >>>> + for_each_cpu(cpu, &drvdata->supported_cpus) { >>>> + if (cpumask_test_cpu(cpu, &drvdata->supported_cpus)) >>>> + smp_call_function_single(cpu, arm_trbe_disable_cpu, >>>> drvdata, 1); >>>> + if (cpumask_test_cpu(cpu, &drvdata->supported_cpus)) >>>> + arm_trbe_remove_coresight_cpu(drvdata, cpu); > > Do we need to test the cpu here in both places ? We already check that > in the loop entry. The reason why we repeat the check during the probe, > is to skip any CPUs that may have a TRBE not accessible. > > Suzuki > Ok, Will fix in next version. Best regards, Junhao. > >>>> + } >>>> free_percpu(drvdata->cpudata); >>>> return 0; >>>> } >>>> @@ -1406,12 +1415,8 @@ static int arm_trbe_cpu_teardown(unsigned >>>> int cpu, struct hlist_node *node) >>>> { >>>> struct trbe_drvdata *drvdata = hlist_entry_safe(node, struct >>>> trbe_drvdata, hotplug_node); >>>> - if (cpumask_test_cpu(cpu, &drvdata->supported_cpus)) { >>>> - struct trbe_cpudata *cpudata = >>>> per_cpu_ptr(drvdata->cpudata, cpu); >>>> - >>>> - disable_percpu_irq(drvdata->irq); >>>> - trbe_reset_local(cpudata); >>>> - } >>>> + if (cpumask_test_cpu(cpu, &drvdata->supported_cpus)) >>>> + arm_trbe_disable_cpu(drvdata); >>>> return 0; >>>> } >>> >>> >>> . >>> >> > > > . >
diff --git a/drivers/hwtracing/coresight/coresight-trbe.c b/drivers/hwtracing/coresight/coresight-trbe.c index 7720619909d6..ce1e6f537b8d 100644 --- a/drivers/hwtracing/coresight/coresight-trbe.c +++ b/drivers/hwtracing/coresight/coresight-trbe.c @@ -1225,6 +1225,17 @@ static void arm_trbe_enable_cpu(void *info) enable_percpu_irq(drvdata->irq, IRQ_TYPE_NONE); } +static void arm_trbe_disable_cpu(void *info) +{ + struct trbe_drvdata *drvdata = info; + struct trbe_cpudata *cpudata = this_cpu_ptr(drvdata->cpudata); + + disable_percpu_irq(drvdata->irq); + trbe_reset_local(cpudata); + cpudata->drvdata = NULL; +} + + static void arm_trbe_register_coresight_cpu(struct trbe_drvdata *drvdata, int cpu) { struct trbe_cpudata *cpudata = per_cpu_ptr(drvdata->cpudata, cpu); @@ -1326,18 +1337,12 @@ static void arm_trbe_probe_cpu(void *info) cpumask_clear_cpu(cpu, &drvdata->supported_cpus); } -static void arm_trbe_remove_coresight_cpu(void *info) +static void arm_trbe_remove_coresight_cpu(struct trbe_drvdata *drvdata, int cpu) { - int cpu = smp_processor_id(); - struct trbe_drvdata *drvdata = info; - struct trbe_cpudata *cpudata = per_cpu_ptr(drvdata->cpudata, cpu); struct coresight_device *trbe_csdev = coresight_get_percpu_sink(cpu); - disable_percpu_irq(drvdata->irq); - trbe_reset_local(cpudata); if (trbe_csdev) { coresight_unregister(trbe_csdev); - cpudata->drvdata = NULL; coresight_set_percpu_sink(cpu, NULL); } } @@ -1366,8 +1371,12 @@ static int arm_trbe_remove_coresight(struct trbe_drvdata *drvdata) { int cpu; - for_each_cpu(cpu, &drvdata->supported_cpus) - smp_call_function_single(cpu, arm_trbe_remove_coresight_cpu, drvdata, 1); + for_each_cpu(cpu, &drvdata->supported_cpus) { + if (cpumask_test_cpu(cpu, &drvdata->supported_cpus)) + smp_call_function_single(cpu, arm_trbe_disable_cpu, drvdata, 1); + if (cpumask_test_cpu(cpu, &drvdata->supported_cpus)) + arm_trbe_remove_coresight_cpu(drvdata, cpu); + } free_percpu(drvdata->cpudata); return 0; } @@ -1406,12 +1415,8 @@ static int arm_trbe_cpu_teardown(unsigned int cpu, struct hlist_node *node) { struct trbe_drvdata *drvdata = hlist_entry_safe(node, struct trbe_drvdata, hotplug_node); - if (cpumask_test_cpu(cpu, &drvdata->supported_cpus)) { - struct trbe_cpudata *cpudata = per_cpu_ptr(drvdata->cpudata, cpu); - - disable_percpu_irq(drvdata->irq); - trbe_reset_local(cpudata); - } + if (cpumask_test_cpu(cpu, &drvdata->supported_cpus)) + arm_trbe_disable_cpu(drvdata); return 0; }