Message ID | 20221013175148.357646891@linuxfoundation.org |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4ac7:0:0:0:0:0 with SMTP id y7csp410857wrs; Thu, 13 Oct 2022 11:07:28 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7YY7qAlspfWcRSSncT9lGIgMtQGO/EtQnB5lO2hDw6FvwSdn8mZvaXW/R1ZUBZymN1Ere0 X-Received: by 2002:a17:902:900a:b0:178:77c7:aa28 with SMTP id a10-20020a170902900a00b0017877c7aa28mr1171337plp.3.1665684448518; Thu, 13 Oct 2022 11:07:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1665684448; cv=none; d=google.com; s=arc-20160816; b=hacDOTc6MpatJXPyTaBRgYddUZSSKsH4Z92WiElOTSLfCYjBOyGP4tMXd7IeHjvZ/a hjHbg8CEQY/jae6AIRrOE7g8ByadaHWh/prHkzLZPG5UH7qPP2bxGeCADq+BNBlstGs8 MqmsMpeZ6uuoR+rrcXdHF2h/wnkCfFUYQmCOOfeZ43m1va6wpTnT2VhYU46aWhwzAUho +GKourtoTfUGLCbHRuXWxMXHifq4NONPNPc/XUFtRNHXvufoRDc3SGdsGBZnacbqlkPT WB/XHpNsFAMiPVPmppDN/RXAwg3yKX3lMJCdXy+fr+k67cg/AhoItKcKKcs02IybicXY cslw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :user-agent:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=h9+XFfV4pVUmzYg8xezdnp8/XTzSNFyGzmah5maP6RM=; b=jSwcmK+pw8UOiF5i0k078siEQ5v90niFVubfciEDjVTQDDLtZyjbNyPmB6wT0U+rPR 8evd/CJutJHADteTnw+UxAWEUnWP07ZDVGc/0TRK0JHl0leIRJGRh2Y+8QCF5f2cvjvu OfJsFM8X7b8KtnUguCuNh7RzGkq8pI/dATrUHE7P2rgb1eN3Dfg7SEJUwwFhhefwDScS f1XX0hZamY0/KxvpJb5JUL9zd2lnY1CyM/zd7SxeJTBIVaR8Uoh0ienV80fmGrj+Cada c1bydTJvD9aBvmNfLQRDoCTxhbK53p/R880Kpxwpf9f29CHVfJc1wMpuPjZAwyHgJE+X iLyw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linuxfoundation.org header.s=korg header.b=EcjO2D3B; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linuxfoundation.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id x26-20020a63b21a000000b0043c9fcccbd9si18424pge.685.2022.10.13.11.07.15; Thu, 13 Oct 2022 11:07:28 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linuxfoundation.org header.s=korg header.b=EcjO2D3B; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linuxfoundation.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231523AbiJMSGo (ORCPT <rfc822;ouuuleilei@gmail.com> + 99 others); Thu, 13 Oct 2022 14:06:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36826 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231432AbiJMSEX (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Thu, 13 Oct 2022 14:04:23 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 696B115B334; Thu, 13 Oct 2022 11:04:10 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id C129F61927; Thu, 13 Oct 2022 17:57:18 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A3DABC433C1; Thu, 13 Oct 2022 17:57:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1665683838; bh=SOyq1/cvIKUT3o6f4I5CSNqOmITuqFONCBVLiyETKrw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=EcjO2D3BJlQ/qe6VFtvHIfB/NBTOBGT5pScuRyUOg7T238GAiwRaJxolWURrBFyuj 4Je6+yzYvy6lxJFwJqJtJQgvi/QxaJy3oJaiCB0jCw57wlcQAZ73MNAV7mm4eGpoaC 9S+xMYmD8F3ZIAjGPgdO4CMgDKfmbmVVRJE+sfM0= From: Greg Kroah-Hartman <gregkh@linuxfoundation.org> To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>, stable@vger.kernel.org, Sherry Yang <sherry.yang@oracle.com>, Paul Webb <paul.x.webb@oracle.com>, Phillip Goerl <phillip.goerl@oracle.com>, Jack Vogel <jack.vogel@oracle.com>, Nicky Veitch <nicky.veitch@oracle.com>, Colm Harrington <colm.harrington@oracle.com>, Ramanan Govindarajan <ramanan.govindarajan@oracle.com>, Sebastian Andrzej Siewior <bigeasy@linutronix.de>, Dominik Brodowski <linux@dominikbrodowski.net>, Tejun Heo <tj@kernel.org>, Sultan Alsawaf <sultan@kerneltoast.com>, "Jason A. Donenfeld" <Jason@zx2c4.com> Subject: [PATCH 5.10 42/54] random: use expired timer rather than wq for mixing fast pool Date: Thu, 13 Oct 2022 19:52:36 +0200 Message-Id: <20221013175148.357646891@linuxfoundation.org> X-Mailer: git-send-email 2.38.0 In-Reply-To: <20221013175147.337501757@linuxfoundation.org> References: <20221013175147.337501757@linuxfoundation.org> User-Agent: quilt/0.67 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1746596736376891398?= X-GMAIL-MSGID: =?utf-8?q?1746596736376891398?= |
Series |
None
|
|
Commit Message
Greg KH
Oct. 13, 2022, 5:52 p.m. UTC
From: Jason A. Donenfeld <Jason@zx2c4.com> commit 748bc4dd9e663f23448d8ad7e58c011a67ea1eca upstream. Previously, the fast pool was dumped into the main pool periodically in the fast pool's hard IRQ handler. This worked fine and there weren't problems with it, until RT came around. Since RT converts spinlocks into sleeping locks, problems cropped up. Rather than switching to raw spinlocks, the RT developers preferred we make the transformation from originally doing: do_some_stuff() spin_lock() do_some_other_stuff() spin_unlock() to doing: do_some_stuff() queue_work_on(some_other_stuff_worker) This is an ordinary pattern done all over the kernel. However, Sherry noticed a 10% performance regression in qperf TCP over a 40gbps InfiniBand card. Quoting her message: > MT27500 Family [ConnectX-3] cards: > Infiniband device 'mlx4_0' port 1 status: > default gid: fe80:0000:0000:0000:0010:e000:0178:9eb1 > base lid: 0x6 > sm lid: 0x1 > state: 4: ACTIVE > phys state: 5: LinkUp > rate: 40 Gb/sec (4X QDR) > link_layer: InfiniBand > > Cards are configured with IP addresses on private subnet for IPoIB > performance testing. > Regression identified in this bug is in TCP latency in this stack as reported > by qperf tcp_lat metric: > > We have one system listen as a qperf server: > [root@yourQperfServer ~]# qperf > > Have the other system connect to qperf server as a client (in this > case, it’s X7 server with Mellanox card): > [root@yourQperfClient ~]# numactl -m0 -N0 qperf 20.20.20.101 -v -uu -ub --time 60 --wait_server 20 -oo msg_size:4K:1024K:*2 tcp_lat Rather than incur the scheduling latency from queue_work_on, we can instead switch to running on the next timer tick, on the same core. This also batches things a bit more -- once per jiffy -- which is okay now that mix_interrupt_randomness() can credit multiple bits at once. Reported-by: Sherry Yang <sherry.yang@oracle.com> Tested-by: Paul Webb <paul.x.webb@oracle.com> Cc: Sherry Yang <sherry.yang@oracle.com> Cc: Phillip Goerl <phillip.goerl@oracle.com> Cc: Jack Vogel <jack.vogel@oracle.com> Cc: Nicky Veitch <nicky.veitch@oracle.com> Cc: Colm Harrington <colm.harrington@oracle.com> Cc: Ramanan Govindarajan <ramanan.govindarajan@oracle.com> Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Cc: Dominik Brodowski <linux@dominikbrodowski.net> Cc: Tejun Heo <tj@kernel.org> Cc: Sultan Alsawaf <sultan@kerneltoast.com> Cc: stable@vger.kernel.org Fixes: 58340f8e952b ("random: defer fast pool mixing to worker") Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> --- drivers/char/random.c | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-)
--- a/drivers/char/random.c +++ b/drivers/char/random.c @@ -898,17 +898,20 @@ struct fast_pool { unsigned long pool[4]; unsigned long last; unsigned int count; - struct work_struct mix; + struct timer_list mix; }; +static void mix_interrupt_randomness(struct timer_list *work); + static DEFINE_PER_CPU(struct fast_pool, irq_randomness) = { #ifdef CONFIG_64BIT #define FASTMIX_PERM SIPHASH_PERMUTATION - .pool = { SIPHASH_CONST_0, SIPHASH_CONST_1, SIPHASH_CONST_2, SIPHASH_CONST_3 } + .pool = { SIPHASH_CONST_0, SIPHASH_CONST_1, SIPHASH_CONST_2, SIPHASH_CONST_3 }, #else #define FASTMIX_PERM HSIPHASH_PERMUTATION - .pool = { HSIPHASH_CONST_0, HSIPHASH_CONST_1, HSIPHASH_CONST_2, HSIPHASH_CONST_3 } + .pool = { HSIPHASH_CONST_0, HSIPHASH_CONST_1, HSIPHASH_CONST_2, HSIPHASH_CONST_3 }, #endif + .mix = __TIMER_INITIALIZER(mix_interrupt_randomness, 0) }; /* @@ -950,7 +953,7 @@ int __cold random_online_cpu(unsigned in } #endif -static void mix_interrupt_randomness(struct work_struct *work) +static void mix_interrupt_randomness(struct timer_list *work) { struct fast_pool *fast_pool = container_of(work, struct fast_pool, mix); /* @@ -1004,10 +1007,11 @@ void add_interrupt_randomness(int irq) if (new_count < 1024 && !time_is_before_jiffies(fast_pool->last + HZ)) return; - if (unlikely(!fast_pool->mix.func)) - INIT_WORK(&fast_pool->mix, mix_interrupt_randomness); fast_pool->count |= MIX_INFLIGHT; - queue_work_on(raw_smp_processor_id(), system_highpri_wq, &fast_pool->mix); + if (!timer_pending(&fast_pool->mix)) { + fast_pool->mix.expires = jiffies; + add_timer_on(&fast_pool->mix, raw_smp_processor_id()); + } } EXPORT_SYMBOL_GPL(add_interrupt_randomness);