Message ID | 20230405203505.1343562-3-mcgrof@kernel.org |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp578048vqo; Wed, 5 Apr 2023 13:41:38 -0700 (PDT) X-Google-Smtp-Source: AKy350aWNt1Be3SouhcU35KysjNLv08wdv4pYpShmxPmIGsUd39xRZHkoR3YW++LAyoZo7kIiIKc X-Received: by 2002:a05:6402:44f:b0:502:a700:dc8b with SMTP id p15-20020a056402044f00b00502a700dc8bmr2659224edw.9.1680727297960; Wed, 05 Apr 2023 13:41:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1680727297; cv=none; d=google.com; s=arc-20160816; b=uNByk4/OhxedmmZZm8RX2KPAY/hB3QVJ8Gmej34txe/F3+sX15cwVWKKSLLH3/Anq/ wkreCcnXHu7kAGvz0k/7zdZiUNB0u6/cpWBBSIwh2ge1ATYChLB0etobqbijMLmIhwDq T3km2QjkWt03nxLVhBTDarjMK28HBwpULRDqbnKgITv/bJ+vziUft/pvkS8X/RjihbVA JUmr87Z7HbPG5aR85y2824vpsqe5W0qr1iiQqRpC+zznA59Da2g3Vdsu4oN2SBrPhUKZ rkz1Xfd06uUKHY8lpg6YAOTTf8kpk/DsjYDKtAAgAkuTeCKj4ZOfYNBFpckeBRIwPd5n 7SNA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=YNsOcxvyBsREks0tqN6x4wPg4m3MWZPixrfEIPIhg1U=; b=adXTpfHr6OhzAYw6Jq3T9IRSav+Wai2qSCDT4u68blk2MiPRRBwQtjevEFvehD2jMw XyX0TRAvbOebCcON/F9S26pI8vBJZDjM+qhJDpYjNNSyZ++3LB/j+fkSbZjDF/R98277 liEknVTvk2NzpKl1CCP6YgwN5LElkkU6CHV10ZBtkySyRLX8fBeb4meRnBkXquFDkRpp OI37WeR3vDOnD9A6d1ckfmXObIDyJLaps6xKRBWpr/kTFCZ+tIjQk1dmtTE7c4Zr+OgT ru8xaunGHlYovRuSRAMasHbgL7xlQSOM2fHMf5JpOp7PaYpAVm/ukRCuQaOogYtKN0hI r7Ng== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@infradead.org header.s=bombadil.20210309 header.b=gfUuvSKg; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id y10-20020a056402134a00b005002d902ae1si4040177edw.566.2023.04.05.13.41.13; Wed, 05 Apr 2023 13:41:37 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@infradead.org header.s=bombadil.20210309 header.b=gfUuvSKg; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233306AbjDEUfY (ORCPT <rfc822;lkml4gm@gmail.com> + 99 others); Wed, 5 Apr 2023 16:35:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52366 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233346AbjDEUfO (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Wed, 5 Apr 2023 16:35:14 -0400 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:3::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 702E16A72; Wed, 5 Apr 2023 13:35:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=YNsOcxvyBsREks0tqN6x4wPg4m3MWZPixrfEIPIhg1U=; b=gfUuvSKg4yYImkFYNGmsyBbM9/ 98+9D7ZfdVo5oy0g+qCsN+zRZAdeIcw42D6VL+3B4EETojpzrqXxf2fPwN5IrpuXCzNsChVc8ePOW bzq20BGd/mVGhBbLILr3CvX48WxYtJL9sRJ3Oe1bzyk6981tRGzBPh8znM++Pop3RGrpUwwgqpGNi hczxpBcuxOjTfJsezWUl6mrDXv6j4w6WsSccnsoXFIJ7rVVSqVCKi1gjncinIWeBtXqnM8IMFIUvz 5B5Z79iMSHvBm3Ns+7ZzHWRzMhw7m440onLHSzpBwnCLi0BRl520qUOiTOIWiYocYXHd/dnrb5ALb yTVa8mag==; Received: from mcgrof by bombadil.infradead.org with local (Exim 4.96 #2 (Red Hat Linux)) id 1pk9qT-005dWc-2p; Wed, 05 Apr 2023 20:35:05 +0000 From: Luis Chamberlain <mcgrof@kernel.org> To: david@redhat.com, patches@lists.linux.dev, linux-modules@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, pmladek@suse.com, petr.pavlu@suse.com, prarit@redhat.com, torvalds@linux-foundation.org, gregkh@linuxfoundation.org, rafael@kernel.org Cc: christophe.leroy@csgroup.eu, tglx@linutronix.de, peterz@infradead.org, song@kernel.org, rppt@kernel.org, dave@stgolabs.net, willy@infradead.org, vbabka@suse.cz, mhocko@suse.com, dave.hansen@linux.intel.com, colin.i.king@gmail.com, jim.cromie@gmail.com, catalin.marinas@arm.com, jbaron@akamai.com, rick.p.edgecombe@intel.com, mcgrof@kernel.org Subject: [PATCH v2 2/2] modules/kmod: replace implementation with a sempahore Date: Wed, 5 Apr 2023 13:35:05 -0700 Message-Id: <20230405203505.1343562-3-mcgrof@kernel.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20230405203505.1343562-1-mcgrof@kernel.org> References: <20230405203505.1343562-1-mcgrof@kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Sender: Luis Chamberlain <mcgrof@infradead.org> X-Spam-Status: No, score=-2.1 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762370307107327365?= X-GMAIL-MSGID: =?utf-8?q?1762370307107327365?= |
Series |
kmod: simplify with a semaphore
|
|
Commit Message
Luis Chamberlain
April 5, 2023, 8:35 p.m. UTC
Simplfy the concurrency delimiter we user for kmod with the semaphore.
I had used the kmod strategy to try to implement a similar concurrency
delimiter for the kernel_read*() calls from the finit_module() path
so to reduce vmalloc() memory pressure. That effort didn't provid yet
conclusive results, but one thing that did became clear is we can use
the suggested alternative solution with semaphores which Linus hinted
at instead of using the atomic / wait strategy.
I've stress tested this with kmod test 0008:
time /data/linux-next/tools/testing/selftests/kmod/kmod.sh -t 0008
And I get only a *slight* delay. That delay however is small, a few
seconds for a full test loop run that runs 150 times, for about ~30-40
seconds. The small delay is worth the simplfication IMHO.
Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
---
kernel/module/kmod.c | 26 +++++++-------------------
1 file changed, 7 insertions(+), 19 deletions(-)
Comments
In the title: s/sempahore/semaphore On Wed, 05 Apr 2023, Luis Chamberlain wrote: >Simplfy the concurrency delimiter we user for kmod with the semaphore. >I had used the kmod strategy to try to implement a similar concurrency >delimiter for the kernel_read*() calls from the finit_module() path >so to reduce vmalloc() memory pressure. That effort didn't provid yet >conclusive results, but one thing that did became clear is we can use >the suggested alternative solution with semaphores which Linus hinted >at instead of using the atomic / wait strategy. > >I've stress tested this with kmod test 0008: > >time /data/linux-next/tools/testing/selftests/kmod/kmod.sh -t 0008 > >And I get only a *slight* delay. That delay however is small, a few >seconds for a full test loop run that runs 150 times, for about ~30-40 >seconds. The small delay is worth the simplfication IMHO. Yes, code looks a lot nicer. Reviewed-by: Davidlohr Bueso <dave@stgolabs.net> >Signed-off-by: Luis Chamberlain <mcgrof@kernel.org> >--- > kernel/module/kmod.c | 26 +++++++------------------- > 1 file changed, 7 insertions(+), 19 deletions(-) > >diff --git a/kernel/module/kmod.c b/kernel/module/kmod.c >index b717134ebe17..925eb85b8346 100644 >--- a/kernel/module/kmod.c >+++ b/kernel/module/kmod.c >@@ -40,8 +40,7 @@ > * effect. Systems like these are very unlikely if modules are enabled. > */ > #define MAX_KMOD_CONCURRENT 50 >-static atomic_t kmod_concurrent_max = ATOMIC_INIT(MAX_KMOD_CONCURRENT); >-static DECLARE_WAIT_QUEUE_HEAD(kmod_wq); >+static DEFINE_SEMAPHORE(kmod_concurrent_max, MAX_KMOD_CONCURRENT); > > /* > * This is a restriction on having *all* MAX_KMOD_CONCURRENT threads >@@ -148,29 +147,18 @@ int __request_module(bool wait, const char *fmt, ...) > if (ret) > return ret; > >- if (atomic_dec_if_positive(&kmod_concurrent_max) < 0) { >- pr_warn_ratelimited("request_module: kmod_concurrent_max (%u) close to 0 (max_modprobes: %u), for module %s, throttling...", >- atomic_read(&kmod_concurrent_max), >- MAX_KMOD_CONCURRENT, module_name); >- ret = wait_event_killable_timeout(kmod_wq, >- atomic_dec_if_positive(&kmod_concurrent_max) >= 0, >- MAX_KMOD_ALL_BUSY_TIMEOUT * HZ); >- if (!ret) { >- pr_warn_ratelimited("request_module: modprobe %s cannot be processed, kmod busy with %d threads for more than %d seconds now", >- module_name, MAX_KMOD_CONCURRENT, MAX_KMOD_ALL_BUSY_TIMEOUT); >- return -ETIME; >- } else if (ret == -ERESTARTSYS) { >- pr_warn_ratelimited("request_module: sigkill sent for modprobe %s, giving up", module_name); >- return ret; >- } >+ ret = down_timeout(&kmod_concurrent_max, MAX_KMOD_ALL_BUSY_TIMEOUT); >+ if (ret) { >+ pr_warn_ratelimited("request_module: modprobe %s cannot be processed, kmod busy with %d threads for more than %d seconds now", >+ module_name, MAX_KMOD_CONCURRENT, MAX_KMOD_ALL_BUSY_TIMEOUT); >+ return ret; > } > > trace_module_request(module_name, wait, _RET_IP_); > > ret = call_modprobe(module_name, wait ? UMH_WAIT_PROC : UMH_WAIT_EXEC); > >- atomic_inc(&kmod_concurrent_max); >- wake_up(&kmod_wq); >+ up(&kmod_concurrent_max); > > return ret; > } >-- >2.39.2 >
On 05.04.23 22:35, Luis Chamberlain wrote: > Simplfy the concurrency delimiter we user for kmod with the semaphore. > I had used the kmod strategy to try to implement a similar concurrency > delimiter for the kernel_read*() calls from the finit_module() path > so to reduce vmalloc() memory pressure. That effort didn't provid yet > conclusive results, but one thing that did became clear is we can use > the suggested alternative solution with semaphores which Linus hinted > at instead of using the atomic / wait strategy. > > I've stress tested this with kmod test 0008: > > time /data/linux-next/tools/testing/selftests/kmod/kmod.sh -t 0008 > > And I get only a *slight* delay. That delay however is small, a few > seconds for a full test loop run that runs 150 times, for about ~30-40 > seconds. The small delay is worth the simplfication IMHO. > > Signed-off-by: Luis Chamberlain <mcgrof@kernel.org> > --- > kernel/module/kmod.c | 26 +++++++------------------- > 1 file changed, 7 insertions(+), 19 deletions(-) > > diff --git a/kernel/module/kmod.c b/kernel/module/kmod.c > index b717134ebe17..925eb85b8346 100644 > --- a/kernel/module/kmod.c > +++ b/kernel/module/kmod.c > @@ -40,8 +40,7 @@ > * effect. Systems like these are very unlikely if modules are enabled. > */ > #define MAX_KMOD_CONCURRENT 50 > -static atomic_t kmod_concurrent_max = ATOMIC_INIT(MAX_KMOD_CONCURRENT); > -static DECLARE_WAIT_QUEUE_HEAD(kmod_wq); > +static DEFINE_SEMAPHORE(kmod_concurrent_max, MAX_KMOD_CONCURRENT); > > /* > * This is a restriction on having *all* MAX_KMOD_CONCURRENT threads > @@ -148,29 +147,18 @@ int __request_module(bool wait, const char *fmt, ...) > if (ret) > return ret; > > - if (atomic_dec_if_positive(&kmod_concurrent_max) < 0) { > - pr_warn_ratelimited("request_module: kmod_concurrent_max (%u) close to 0 (max_modprobes: %u), for module %s, throttling...", > - atomic_read(&kmod_concurrent_max), > - MAX_KMOD_CONCURRENT, module_name); > - ret = wait_event_killable_timeout(kmod_wq, > - atomic_dec_if_positive(&kmod_concurrent_max) >= 0, > - MAX_KMOD_ALL_BUSY_TIMEOUT * HZ); > - if (!ret) { > - pr_warn_ratelimited("request_module: modprobe %s cannot be processed, kmod busy with %d threads for more than %d seconds now", > - module_name, MAX_KMOD_CONCURRENT, MAX_KMOD_ALL_BUSY_TIMEOUT); > - return -ETIME; > - } else if (ret == -ERESTARTSYS) { > - pr_warn_ratelimited("request_module: sigkill sent for modprobe %s, giving up", module_name); > - return ret; > - } > + ret = down_timeout(&kmod_concurrent_max, MAX_KMOD_ALL_BUSY_TIMEOUT); > + if (ret) { > + pr_warn_ratelimited("request_module: modprobe %s cannot be processed, kmod busy with %d threads for more than %d seconds now", > + module_name, MAX_KMOD_CONCURRENT, MAX_KMOD_ALL_BUSY_TIMEOUT); > + return ret; > } > > trace_module_request(module_name, wait, _RET_IP_); > > ret = call_modprobe(module_name, wait ? UMH_WAIT_PROC : UMH_WAIT_EXEC); > > - atomic_inc(&kmod_concurrent_max); > - wake_up(&kmod_wq); > + up(&kmod_concurrent_max); > > return ret; > } Much cleaner Reviewed-by: David Hildenbrand <david@redhat.com>
Hi, On Wed, 5 Apr 2023, Luis Chamberlain wrote: s/sempahore/semaphore/ in the subject > Simplfy the concurrency delimiter we user for kmod with the semaphore. "Simplify the concurrency delimiter we use for kmod with the semaphore." (two typos) > I had used the kmod strategy to try to implement a similar concurrency > delimiter for the kernel_read*() calls from the finit_module() path > so to reduce vmalloc() memory pressure. That effort didn't provid yet s/provid/provide/ > conclusive results, but one thing that did became clear is we can use s/did // (or s/became/become/) > the suggested alternative solution with semaphores which Linus hinted > at instead of using the atomic / wait strategy. > > I've stress tested this with kmod test 0008: > > time /data/linux-next/tools/testing/selftests/kmod/kmod.sh -t 0008 > > And I get only a *slight* delay. That delay however is small, a few > seconds for a full test loop run that runs 150 times, for about ~30-40 > seconds. The small delay is worth the simplfication IMHO. > > Signed-off-by: Luis Chamberlain <mcgrof@kernel.org> > --- > kernel/module/kmod.c | 26 +++++++------------------- > 1 file changed, 7 insertions(+), 19 deletions(-) > > diff --git a/kernel/module/kmod.c b/kernel/module/kmod.c > index b717134ebe17..925eb85b8346 100644 > --- a/kernel/module/kmod.c > +++ b/kernel/module/kmod.c > @@ -40,8 +40,7 @@ > * effect. Systems like these are very unlikely if modules are enabled. > */ > #define MAX_KMOD_CONCURRENT 50 > -static atomic_t kmod_concurrent_max = ATOMIC_INIT(MAX_KMOD_CONCURRENT); > -static DECLARE_WAIT_QUEUE_HEAD(kmod_wq); > +static DEFINE_SEMAPHORE(kmod_concurrent_max, MAX_KMOD_CONCURRENT); > > /* > * This is a restriction on having *all* MAX_KMOD_CONCURRENT threads > @@ -148,29 +147,18 @@ int __request_module(bool wait, const char *fmt, ...) > if (ret) > return ret; > > - if (atomic_dec_if_positive(&kmod_concurrent_max) < 0) { > - pr_warn_ratelimited("request_module: kmod_concurrent_max (%u) close to 0 (max_modprobes: %u), for module %s, throttling...", > - atomic_read(&kmod_concurrent_max), > - MAX_KMOD_CONCURRENT, module_name); > - ret = wait_event_killable_timeout(kmod_wq, > - atomic_dec_if_positive(&kmod_concurrent_max) >= 0, > - MAX_KMOD_ALL_BUSY_TIMEOUT * HZ); > - if (!ret) { > - pr_warn_ratelimited("request_module: modprobe %s cannot be processed, kmod busy with %d threads for more than %d seconds now", > - module_name, MAX_KMOD_CONCURRENT, MAX_KMOD_ALL_BUSY_TIMEOUT); > - return -ETIME; > - } else if (ret == -ERESTARTSYS) { > - pr_warn_ratelimited("request_module: sigkill sent for modprobe %s, giving up", module_name); > - return ret; > - } > + ret = down_timeout(&kmod_concurrent_max, MAX_KMOD_ALL_BUSY_TIMEOUT); MAX_KMOD_ALL_BUSY_TIMEOUT * HZ ? The simplification is very nice. Miroslav
On Thu, Apr 13, 2023 at 11:44:51AM +0200, Miroslav Benes wrote: > Hi, <spell checks> Thanks, fixed! > MAX_KMOD_ALL_BUSY_TIMEOUT * HZ ? Yes! In the end schedule_timeout() is used so yup, thanks! Luis
diff --git a/kernel/module/kmod.c b/kernel/module/kmod.c index b717134ebe17..925eb85b8346 100644 --- a/kernel/module/kmod.c +++ b/kernel/module/kmod.c @@ -40,8 +40,7 @@ * effect. Systems like these are very unlikely if modules are enabled. */ #define MAX_KMOD_CONCURRENT 50 -static atomic_t kmod_concurrent_max = ATOMIC_INIT(MAX_KMOD_CONCURRENT); -static DECLARE_WAIT_QUEUE_HEAD(kmod_wq); +static DEFINE_SEMAPHORE(kmod_concurrent_max, MAX_KMOD_CONCURRENT); /* * This is a restriction on having *all* MAX_KMOD_CONCURRENT threads @@ -148,29 +147,18 @@ int __request_module(bool wait, const char *fmt, ...) if (ret) return ret; - if (atomic_dec_if_positive(&kmod_concurrent_max) < 0) { - pr_warn_ratelimited("request_module: kmod_concurrent_max (%u) close to 0 (max_modprobes: %u), for module %s, throttling...", - atomic_read(&kmod_concurrent_max), - MAX_KMOD_CONCURRENT, module_name); - ret = wait_event_killable_timeout(kmod_wq, - atomic_dec_if_positive(&kmod_concurrent_max) >= 0, - MAX_KMOD_ALL_BUSY_TIMEOUT * HZ); - if (!ret) { - pr_warn_ratelimited("request_module: modprobe %s cannot be processed, kmod busy with %d threads for more than %d seconds now", - module_name, MAX_KMOD_CONCURRENT, MAX_KMOD_ALL_BUSY_TIMEOUT); - return -ETIME; - } else if (ret == -ERESTARTSYS) { - pr_warn_ratelimited("request_module: sigkill sent for modprobe %s, giving up", module_name); - return ret; - } + ret = down_timeout(&kmod_concurrent_max, MAX_KMOD_ALL_BUSY_TIMEOUT); + if (ret) { + pr_warn_ratelimited("request_module: modprobe %s cannot be processed, kmod busy with %d threads for more than %d seconds now", + module_name, MAX_KMOD_CONCURRENT, MAX_KMOD_ALL_BUSY_TIMEOUT); + return ret; } trace_module_request(module_name, wait, _RET_IP_); ret = call_modprobe(module_name, wait ? UMH_WAIT_PROC : UMH_WAIT_EXEC); - atomic_inc(&kmod_concurrent_max); - wake_up(&kmod_wq); + up(&kmod_concurrent_max); return ret; }