From patchwork Mon Oct 24 11:29:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Greg KH X-Patchwork-Id: 8421 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp400440wru; Mon, 24 Oct 2022 04:47:09 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5wfI0rz03kDFmfBJW3smbavfI8xkW/Tf76UViHTt8yaRDNE4LA2BMkZFDsY07EniScTBp/ X-Received: by 2002:a17:906:1e08:b0:73d:c724:4876 with SMTP id g8-20020a1709061e0800b0073dc7244876mr26450702ejj.62.1666612029451; Mon, 24 Oct 2022 04:47:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666612029; cv=none; d=google.com; s=arc-20160816; b=NqS9e/s7KGtkFuU5T2K4sTfdRCyt9AqegsziMxsczGFkMfVoxpT1eyt+jZGKIdzsUa BG4csygx0ulSeZXrC66EyWJGT44UZNwgca3A/sbSWkiMS6iQQ8lFtYjF2kiLskmUdnmy l2VVYvV+7u0+4oBnKQGWDnc6gwYx1XwwTv02T74EYm6w6lCHFyYXfN7pnHvjHplFT9pD rwtP5UxKWWLbQpQs7bu8XZbYlfWibin3UOX9mJ+lg5mx9vVKOYFjU+owpAx1bZ5kqSMN CeTkTH+1riTchSbE0VOq8Y7eRnTrd6pgNQH/HJLsbMfHMc3dpKlX7bCfKXJLbS9ET+1c Cj+w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :user-agent:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=XzkZKyz4+5Q6PgeNb5ZRt4CNe8NEzQRDN9B5v1bncHU=; b=aFYCPXWmgJlxXT78bi+8C3oyTjsno2Mcb/mhjRAGnpwLYrONNt6Bm4PiSvjcbYkQrX O7lpTndGKdomxoag5kOSwYSgVYaUtnLI9Kq+VVblwa3J/Z6vxyUzuA/6GhmI/RfITYqe 8tuGGNhFB3bs65KdLFXU15fjuvtINlcP/YRbDGnFze86OuKZ0jQQ9kEzMZ+1GK1BdkqF iXmjMPykXHoG4i8NF/lDMvVYfOU2IYS6ZEe6DS0yzKlL3+vsj0RAElWM71cNP8r77Qty AzFZhRuS4E1GuurRAAzy8PjI5smQFjSp4KBCvb4398PXVvj8w8Wbj7JDI96IfmwNMPo8 fsvQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linuxfoundation.org header.s=korg header.b=TjZg8daD; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linuxfoundation.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id eb13-20020a0564020d0d00b0044611122003si35567034edb.599.2022.10.24.04.46.45; Mon, 24 Oct 2022 04:47:09 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linuxfoundation.org header.s=korg header.b=TjZg8daD; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linuxfoundation.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231584AbiJXLqN (ORCPT + 99 others); Mon, 24 Oct 2022 07:46:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43456 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231726AbiJXLoV (ORCPT ); Mon, 24 Oct 2022 07:44:21 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1234A74352; Mon, 24 Oct 2022 04:41:27 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id A7BAC6127D; Mon, 24 Oct 2022 11:40:02 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 961B1C433D6; Mon, 24 Oct 2022 11:40:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1666611602; bh=owFmSv/S9DLDU2vakvxfkdD4+u8AbGA58BI/lh6hDm4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TjZg8daD7jty5py2r90oPRMx43IR64QFR4GOwhpH6I8Z8B6YbJG3XpFCfIpmMOr2C ydQsAuoev7xdHHqFj3u8HNhDtyOc5FtBGYj2/+2wZRZ6RTwHXvPrdSwspwdctRY9DK hjJSymyuoQTWQaLJYYMg9yjWvTgFjgGZLtQbpbxA= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Sherry Yang , Paul Webb , Phillip Goerl , Jack Vogel , Nicky Veitch , Colm Harrington , Ramanan Govindarajan , Sebastian Andrzej Siewior , Dominik Brodowski , Tejun Heo , Sultan Alsawaf , "Jason A. Donenfeld" Subject: [PATCH 4.9 040/159] random: use expired timer rather than wq for mixing fast pool Date: Mon, 24 Oct 2022 13:29:54 +0200 Message-Id: <20221024112950.838950804@linuxfoundation.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221024112949.358278806@linuxfoundation.org> References: <20221024112949.358278806@linuxfoundation.org> User-Agent: quilt/0.67 MIME-Version: 1.0 X-Spam-Status: No, score=-7.6 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747569375910602924?= X-GMAIL-MSGID: =?utf-8?q?1747569375910602924?= From: Jason A. Donenfeld commit 748bc4dd9e663f23448d8ad7e58c011a67ea1eca upstream. Previously, the fast pool was dumped into the main pool periodically in the fast pool's hard IRQ handler. This worked fine and there weren't problems with it, until RT came around. Since RT converts spinlocks into sleeping locks, problems cropped up. Rather than switching to raw spinlocks, the RT developers preferred we make the transformation from originally doing: do_some_stuff() spin_lock() do_some_other_stuff() spin_unlock() to doing: do_some_stuff() queue_work_on(some_other_stuff_worker) This is an ordinary pattern done all over the kernel. However, Sherry noticed a 10% performance regression in qperf TCP over a 40gbps InfiniBand card. Quoting her message: > MT27500 Family [ConnectX-3] cards: > Infiniband device 'mlx4_0' port 1 status: > default gid: fe80:0000:0000:0000:0010:e000:0178:9eb1 > base lid: 0x6 > sm lid: 0x1 > state: 4: ACTIVE > phys state: 5: LinkUp > rate: 40 Gb/sec (4X QDR) > link_layer: InfiniBand > > Cards are configured with IP addresses on private subnet for IPoIB > performance testing. > Regression identified in this bug is in TCP latency in this stack as reported > by qperf tcp_lat metric: > > We have one system listen as a qperf server: > [root@yourQperfServer ~]# qperf > > Have the other system connect to qperf server as a client (in this > case, it’s X7 server with Mellanox card): > [root@yourQperfClient ~]# numactl -m0 -N0 qperf 20.20.20.101 -v -uu -ub --time 60 --wait_server 20 -oo msg_size:4K:1024K:*2 tcp_lat Rather than incur the scheduling latency from queue_work_on, we can instead switch to running on the next timer tick, on the same core. This also batches things a bit more -- once per jiffy -- which is okay now that mix_interrupt_randomness() can credit multiple bits at once. Reported-by: Sherry Yang Tested-by: Paul Webb Cc: Sherry Yang Cc: Phillip Goerl Cc: Jack Vogel Cc: Nicky Veitch Cc: Colm Harrington Cc: Ramanan Govindarajan Cc: Sebastian Andrzej Siewior Cc: Dominik Brodowski Cc: Tejun Heo Cc: Sultan Alsawaf Cc: stable@vger.kernel.org Fixes: 58340f8e952b ("random: defer fast pool mixing to worker") Signed-off-by: Jason A. Donenfeld Signed-off-by: Greg Kroah-Hartman --- drivers/char/random.c | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) --- a/drivers/char/random.c +++ b/drivers/char/random.c @@ -894,7 +894,7 @@ struct fast_pool { unsigned long pool[4]; unsigned long last; unsigned int count; - struct work_struct mix; + struct timer_list mix; }; static DEFINE_PER_CPU(struct fast_pool, irq_randomness) = { @@ -946,9 +946,9 @@ int __cold random_online_cpu(unsigned in } #endif -static void mix_interrupt_randomness(struct work_struct *work) +static void mix_interrupt_randomness(unsigned long data) { - struct fast_pool *fast_pool = container_of(work, struct fast_pool, mix); + struct fast_pool *fast_pool = (struct fast_pool *)data; /* * The size of the copied stack pool is explicitly 2 longs so that we * only ever ingest half of the siphash output each time, retaining @@ -1000,10 +1000,14 @@ void add_interrupt_randomness(int irq) if (new_count < 1024 && !time_is_before_jiffies(fast_pool->last + HZ)) return; - if (unlikely(!fast_pool->mix.func)) - INIT_WORK(&fast_pool->mix, mix_interrupt_randomness); + if (unlikely(!fast_pool->mix.data)) + setup_timer(&fast_pool->mix, mix_interrupt_randomness, (unsigned long)fast_pool); + fast_pool->count |= MIX_INFLIGHT; - queue_work_on(raw_smp_processor_id(), system_highpri_wq, &fast_pool->mix); + if (!timer_pending(&fast_pool->mix)) { + fast_pool->mix.expires = jiffies; + add_timer_on(&fast_pool->mix, raw_smp_processor_id()); + } } EXPORT_SYMBOL_GPL(add_interrupt_randomness);