From patchwork Wed Jan 31 18:08:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Damato X-Patchwork-Id: 194893 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp2069751dyb; Wed, 31 Jan 2024 10:11:10 -0800 (PST) X-Google-Smtp-Source: AGHT+IFSM/fHFDqYym4rhYrMSJh0QHivVBelG2BGeDZbaoxIwDGGJEBwAFXyP2Dp7HIruFsD//WO X-Received: by 2002:ac8:4e56:0:b0:42a:a88b:9ba1 with SMTP id e22-20020ac84e56000000b0042aa88b9ba1mr84105qtw.21.1706724670169; Wed, 31 Jan 2024 10:11:10 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706724670; cv=pass; d=google.com; s=arc-20160816; b=CVfNKfU6u3cl1JmDJ/2jx3Vzu/LarADZAO/izqGpA9cJYpPhYPS6ff8glHH5hklKyH hpVroMNAdaLs0OQ61+G788cV+ZQSyy67sLY3Epvn/4b41gJiRTR+P9Ws5O8qv/5VN3T3 qJhD0SyM45u0Q1iu0rt4TWWrA820gnArm3YYCEhnpwcJHpG5aeDop9x8OIrM6Dd9dekG nGfvjCP/YqaU3CP2x98IP08CslPlfQvqbeh2QBeBIIdblpdtRVc0cVD1ndy4fZFqaIDA CNT+56R9wIzpf2oXHpKdnELuJAg5QJl3WpsKEmBo3p4VnGpDwBDTR/wWNv5ab4G/gdx+ ZZBg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=dis0Zk83NVRgFQ7j5PMchP4NSpK2O2E2OiTy+PShPHQ=; fh=VdM51TEBxh2WibK9su0aPW+z0JGEPgswLbdWq9srmro=; b=dpnIVxfJIWwFaZpQ7p5uBGVo9MAHTHmoGux1PfUuU1iOmuO3+8Hb1m3H4yiT3/WBI5 BYtzUT3hoztoGkxSfgtbow6DbhD8nbH/oz/Habc7tS7tpyZyySvACyMs/fCK58omMYER ftiGSLwPCXdPdXq7hf+g7USDfve4ITjMsOZeVxaRnZr1MNQUKzkmHR9QuPJ/2iS06wcc nKJsECr6kS4veL8fzlT5fkh+/+1m1iNeaZMMUHgmdqhac7aKXXlhwH94E6+z2VtNw29c TLWURKVgse2IGbHZEWH9sVdEDO3Y4VMelySZ/cvNX8LUIxG1Ao85dFfVIB3cY+b6B16+ R66w==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@fastly.com header.s=google header.b=KnHVwhWv; arc=pass (i=1 spf=pass spfdomain=fastly.com dkim=pass dkdomain=fastly.com dmarc=pass fromdomain=fastly.com); spf=pass (google.com: domain of linux-kernel+bounces-46955-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-46955-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=fastly.com X-Forwarded-Encrypted: i=1; AJvYcCWO5YPMim7/Tdb+igy2y44XWQ7lV0Os1WjsBJ63yJ47LcawWnTeBaLEXacwgW+aKtpIwXU5lP3egsO9FXbh4qKi9Kd4YA== Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id r19-20020ac85c93000000b0042a1138b665si12468990qta.783.2024.01.31.10.11.10 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Jan 2024 10:11:10 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-46955-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@fastly.com header.s=google header.b=KnHVwhWv; arc=pass (i=1 spf=pass spfdomain=fastly.com dkim=pass dkdomain=fastly.com dmarc=pass fromdomain=fastly.com); spf=pass (google.com: domain of linux-kernel+bounces-46955-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-46955-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=fastly.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id E49181C20C9F for ; Wed, 31 Jan 2024 18:11:09 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 476DF13BEAB; Wed, 31 Jan 2024 18:09:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b="KnHVwhWv" Received: from mail-oo1-f43.google.com (mail-oo1-f43.google.com [209.85.161.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D95FD1386D7 for ; Wed, 31 Jan 2024 18:09:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.161.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706724583; cv=none; b=k71CF0m5cmr1iCbaj+6rTgz0kB3HGeNw+v7FSqhHuXPAeZMMr2RPpzV/MpGWF2cNr9xw/s9PgdhYImmFhOzvmdwNezgOwKFP+eCL7QuF2chtR2Qw+hFvQ21KDlesKej7rvkiIyQZiJdk8jG1duXxOjYddcMdmko/oalybCGYf+M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706724583; c=relaxed/simple; bh=+NvwukKrlDHz6eylGdtOKZKD8hFQLLmD1KnjinUcTaA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=cnu+uQ/WDx+flwoPk03E2KJcuxp5Rbb5iwE6QHSoIICwdKoIRFk7TMEXnwxK1c5sk1xHwqnNPx3Elvs4bIY6603ufze96roi4fQ/tSIM90IpOWdZs11Ufm0No5GYs+58j3ZnfQCtDrzgHJk7xiAxnJ6aBWgkQiS3MTzhrLn2GaA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com; spf=pass smtp.mailfrom=fastly.com; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b=KnHVwhWv; arc=none smtp.client-ip=209.85.161.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=fastly.com Received: by mail-oo1-f43.google.com with SMTP id 006d021491bc7-598699c0f1eso52061eaf.2 for ; Wed, 31 Jan 2024 10:09:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; t=1706724580; x=1707329380; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=dis0Zk83NVRgFQ7j5PMchP4NSpK2O2E2OiTy+PShPHQ=; b=KnHVwhWvgJevhVlcnikP7QUDjrpfFLbgEZhFUoIjHIFCR/MNbfMrJXrQ4IkSM5TrqK 2UDPfy1N3PUya8cOMl+64EFU1OoCdR0ueVBFh3poG/6pfowuA0LDBbpd2CEe4mxyO5Ik 4vT9JFBioIWaYPE9QI3DKf+tA05SI0ri3vzZQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706724580; x=1707329380; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=dis0Zk83NVRgFQ7j5PMchP4NSpK2O2E2OiTy+PShPHQ=; b=n3lpDt61isn6LfJ3kpFLqjiQrKfK+G74K+tbGP8R9kZKRENJRzoF8W/qJ3E9wRiJlb lNqkgK8M0SnkUkWN7K6QyKDV1eI40Up+VZ+Y8IIvbRXkcq88g9rb/gTvG7HbOGzdQGQf eWVudv2ojM/t65bVwLitsXNekKEA+9apcciROhqRGz2TRYUzMyCwOb4XiV+lv9s63HFe hH1TS1LPHsJ1jJFCIhyDT4yy2ZyO4QtQP5ZQ2T+tuEmO9MSAOBU6H4gh3GTCbiHTnS9L cVAGgY+0fODos7IuuwXFlnZ42WuWnMenB9eYr4NyUHnorVkcXd30ZshkBqaNieKc7XHV i4AA== X-Gm-Message-State: AOJu0Yxjvq40F9uPTcFuyRyPOkqp+g/2A9W0Hc3Qz1c9lRYbOWEBwX6f 3XTzMQYoYaqgvwHGKZ9X0wvi+19iBZC9psKPOPVCQSX1f5HzOK2h7H9CyOYCGZUFIkAZ+9Cz7Ik iEcWGr/mIF7q32cTZAnSQSmlMHbzuvB2xOMR8TimKcCoD/EOrf9rJ9gk3ZkbLwOCB++t797evwm ZrBH/+GOf+IGaqjq+sjDutDfoSZOjn9EoEw+ePy5ao7IAb3A== X-Received: by 2002:a05:6358:4b4b:b0:176:915a:c7b7 with SMTP id ks11-20020a0563584b4b00b00176915ac7b7mr32979rwc.31.1706724580488; Wed, 31 Jan 2024 10:09:40 -0800 (PST) Received: from localhost.localdomain ([2620:11a:c018:0:ea8:be91:8d1:f59b]) by smtp.gmail.com with ESMTPSA id v24-20020a634818000000b005cfbf96c733sm10876004pga.30.2024.01.31.10.09.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Jan 2024 10:09:40 -0800 (PST) From: Joe Damato To: linux-kernel@vger.kernel.org, netdev@vger.kernel.org Cc: chuck.lever@oracle.com, jlayton@kernel.org, linux-api@vger.kernel.org, brauner@kernel.org, edumazet@google.com, davem@davemloft.net, alexander.duyck@gmail.com, sridhar.samudrala@intel.com, kuba@kernel.org, willemdebruijn.kernel@gmail.com, weiwan@google.com, David.Laight@ACULAB.COM, arnd@arndb.de, Joe Damato , Alexander Viro , Jan Kara , linux-fsdevel@vger.kernel.org (open list:FILESYSTEMS (VFS and infrastructure)) Subject: [PATCH net-next v5 2/3] eventpoll: Add per-epoll busy poll packet budget Date: Wed, 31 Jan 2024 18:08:04 +0000 Message-Id: <20240131180811.23566-3-jdamato@fastly.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240131180811.23566-1-jdamato@fastly.com> References: <20240131180811.23566-1-jdamato@fastly.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789630528046013767 X-GMAIL-MSGID: 1789630528046013767 When using epoll-based busy poll, the packet budget is hardcoded to BUSY_POLL_BUDGET (8). Users may desire larger busy poll budgets, which can potentially increase throughput when busy polling under high network load. Other busy poll methods allow setting the busy poll budget via SO_BUSY_POLL_BUDGET, but epoll-based busy polling uses a hardcoded value. Fix this edge case by adding support for a per-epoll context busy poll packet budget. If not specified, the default value (BUSY_POLL_BUDGET) is used. Signed-off-by: Joe Damato --- fs/eventpoll.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/fs/eventpoll.c b/fs/eventpoll.c index ce75189d46df..3985434df527 100644 --- a/fs/eventpoll.c +++ b/fs/eventpoll.c @@ -229,6 +229,8 @@ struct eventpoll { unsigned int napi_id; /* busy poll timeout */ u64 busy_poll_usecs; + /* busy poll packet budget */ + u16 busy_poll_budget; #endif #ifdef CONFIG_DEBUG_LOCK_ALLOC @@ -437,10 +439,14 @@ static bool ep_busy_loop_end(void *p, unsigned long start_time) static bool ep_busy_loop(struct eventpoll *ep, int nonblock) { unsigned int napi_id = READ_ONCE(ep->napi_id); + u16 budget = READ_ONCE(ep->busy_poll_budget); + + if (!budget) + budget = BUSY_POLL_BUDGET; if ((napi_id >= MIN_NAPI_ID) && ep_busy_loop_on(ep)) { napi_busy_loop(napi_id, nonblock ? NULL : ep_busy_loop_end, ep, false, - BUSY_POLL_BUDGET); + budget); if (ep_events_available(ep)) return true; /* @@ -2098,6 +2104,7 @@ static int do_epoll_create(int flags) } #ifdef CONFIG_NET_RX_BUSY_POLL ep->busy_poll_usecs = 0; + ep->busy_poll_budget = 0; #endif ep->file = file; fd_install(fd, file);