From patchwork Thu Jan 25 22:56:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Damato X-Patchwork-Id: 192329 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:e09d:b0:103:945f:af90 with SMTP id gm29csp313994dyb; Thu, 25 Jan 2024 15:01:01 -0800 (PST) X-Google-Smtp-Source: AGHT+IEikulBljK7VItB279ZvlKZHFE3HpOFbCa1q7tzS/dr04sN3Zr/JrpEQb2sg+ZpMdqMQy70 X-Received: by 2002:a17:902:ea0a:b0:1d7:233b:314d with SMTP id s10-20020a170902ea0a00b001d7233b314dmr427020plg.127.1706223661279; Thu, 25 Jan 2024 15:01:01 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706223661; cv=pass; d=google.com; s=arc-20160816; b=pAH4jww/S6LdQMFQQaD5fGvdnG/YafKRb0yL0D0692+xblsodX77N76SYD3wQmCp4m tUFdrXQ6JrHLq7JB2QGMC2cNwv23DgQ5F5l8MwfMmSuMk2Ph1fjhKi+hO1XZqtdxLU0N uV0IBHum8wwOvvDXSzFLrALhY+5CVA/FmVfVjm0Y4zEeUrtEtW4O4qr1V0RLELuZVcKE MYq2ZlotbIamwRarwH+GHUb1NxPNwCl/YqsntWzsaZkaVTihw5b9jYCYNWWWZH/F9U2s +ItZ1lTlCc3RoOowPJ0VtsKL8qVU9aMot4xVT4mUbcbE3ftalogaUCsvCQAeWOnz1+OF u4Ew== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=o95DHArKhHsVxZUrtiTfK3GJrg5R+5zGPrNbLEZjDzs=; fh=xXEYDfOQzkMJq5YQq0KYkBbOiZYpvo5c1OyUsvtKbLQ=; b=dHuM/2ZCeQD75IyzQyI9Fk/ZBsywYYsiJbXKBZLYj9t8JY7Cc1iBmMG+b7szhNG1qr 6d4+iMzIyYrAQzhPNODL+EEhelSkQ2TSRarq1y4FjbfZflx1sOsUSkbIPvJjTdm7zDJW IE/8Qi9K0y7AccWhf0yU0UuVEhRJtjkkmTjt/o32Pv5CRpwWZH5YfIyJUkC3osZl7HgO a/gvenrCwxo9I00FbyLnZjNCuM9ZRMC0/GDqSNZdsmArrr0qyvNnxP7Dk8JPEC7VXtMd lnDqAtA3oDf+MNNtLgGJmLDA6O7A5HAI+B4iSpJ/1r3gLd1jGdEgWahw3e+DLSqzp3Zl eveA== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@fastly.com header.s=google header.b="gK/gao8L"; arc=pass (i=1 spf=pass spfdomain=fastly.com dkim=pass dkdomain=fastly.com dmarc=pass fromdomain=fastly.com); spf=pass (google.com: domain of linux-kernel+bounces-39394-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-39394-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=fastly.com Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id s8-20020a170902ea0800b001d78f90818dsi24567plg.130.2024.01.25.15.01.01 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Jan 2024 15:01:01 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-39394-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; dkim=pass header.i=@fastly.com header.s=google header.b="gK/gao8L"; arc=pass (i=1 spf=pass spfdomain=fastly.com dkim=pass dkdomain=fastly.com dmarc=pass fromdomain=fastly.com); spf=pass (google.com: domain of linux-kernel+bounces-39394-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-39394-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=fastly.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id BDD0B29128C for ; Thu, 25 Jan 2024 23:00:49 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id B953F1B59C; Thu, 25 Jan 2024 22:57:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b="gK/gao8L" Received: from mail-il1-f180.google.com (mail-il1-f180.google.com [209.85.166.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 601EF15AF9 for ; Thu, 25 Jan 2024 22:57:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706223473; cv=none; b=MhVfCqUu/gfslf7lUB+71zHNJX+ZP6urkpyZ3KoT1TtOROVqqR9kLgz/cZ0J6FOnGp7O4OeLLiJIV2NtH34cg8E70TqlOZ0gXaGoLCpPGhRkxUibJdOVgjTMb61o4XJ6zZbtsWCoG2SQMouFUu+/s4Aos1/MFG2WFCTdjH34I2Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706223473; c=relaxed/simple; bh=3yteI2ybUubAH5CujHpgcS7qoJxKc8vJCs7EzjX9vMQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=d+1Oz9S4L/c2OspY03GRCI8UQolNcYdyo+nQnqee2yQlQk4/PERVP5Eb2MTqR0FYBoLFdiVHaFemXxTjBYXczgL1AepNGFKlXD1HGKEMfRc7gawAqCfn9w+g/ahN2ZTgLa/Az3FZ3neS4dzW2n0OV03/jsPJ/5gECjaGrwmUMvM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com; spf=pass smtp.mailfrom=fastly.com; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b=gK/gao8L; arc=none smtp.client-ip=209.85.166.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=fastly.com Received: by mail-il1-f180.google.com with SMTP id e9e14a558f8ab-3606f3f2f37so37727255ab.0 for ; Thu, 25 Jan 2024 14:57:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; t=1706223468; x=1706828268; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=o95DHArKhHsVxZUrtiTfK3GJrg5R+5zGPrNbLEZjDzs=; b=gK/gao8LZPQ14f+0l1Z/+ivgHpju3xt99PQTI/cOjX4zckT76Ao7/ofkfMZOM/dKY3 huhuUoAasEEsNHDBwxKIkVMhT1VBnxGfnbjiPJKJbswZYdFIkXci5IsVQOmx9poTHHD1 hWZFcA8fGZDbx+laWc3yhOV75Uj8m/jdqrEIM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706223468; x=1706828268; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=o95DHArKhHsVxZUrtiTfK3GJrg5R+5zGPrNbLEZjDzs=; b=wjCQj0YAQpIhD7OWs3WSAfC9IreUhk1sCkJFSGRxAB2gCGpXgR9iE43yhSoc9v+g+1 8fcalFX7vcSk0L7dWBQWVv9ABo8bvJ9NF4xV1S9xiu5EhdT/U7/1E2entDjouveErV7y 2OWt23x4pH8G59izuTAihzmUlcrI2xdMaYapSzHqBZBVChdEYw5gM6IeOcxe2d/e+Fn2 IsbeNF/JXEdhupBWtWTMCy0Zr6UoS7+D4f7FsXapmbL1Afi11kZdho80W0HY6+5FaEQW 1O9S2lWdRbEd+tPzQS0FizCOiUOWbytCyyG/yEdmt5SM43UAKRBlKYiBGSkHS+emhVwj yhXg== X-Gm-Message-State: AOJu0YziTTNUYl1oTWE+wInC37WHLvrUX8H5JZsT9jYukQqTGjzUVCJ2 FzEjVRYvpYQOmT4sSBOZuvU8fsoPh6xQTnnKsKS7kJ9hRbppI1C42xnQH3gtFU2JuvCPfgdk0NV QwZL0c2K3kvbHMgtNN4RVtzFIqEn5HozUbWIa5rzxaiuIUKH+NrWoaiOdKRr1d/MU36ryOic5Kl YMINgmSr3l05UfMNOks6l9+Gu5pMTWWOVZQPiv5pouWUsayg== X-Received: by 2002:a05:6e02:12ea:b0:360:76fb:a894 with SMTP id l10-20020a056e0212ea00b0036076fba894mr557277iln.62.1706223467756; Thu, 25 Jan 2024 14:57:47 -0800 (PST) Received: from localhost.localdomain ([2620:11a:c018:0:ea8:be91:8d1:f59b]) by smtp.gmail.com with ESMTPSA id z24-20020a631918000000b005d68962e1a7sm19948pgl.24.2024.01.25.14.57.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Jan 2024 14:57:47 -0800 (PST) From: Joe Damato To: linux-kernel@vger.kernel.org, netdev@vger.kernel.org Cc: chuck.lever@oracle.com, jlayton@kernel.org, linux-api@vger.kernel.org, brauner@kernel.org, edumazet@google.com, davem@davemloft.net, alexander.duyck@gmail.com, sridhar.samudrala@intel.com, kuba@kernel.org, willemdebruijn.kernel@gmail.com, weiwan@google.com, Joe Damato , Alexander Viro , Jan Kara , linux-fsdevel@vger.kernel.org (open list:FILESYSTEMS (VFS and infrastructure)) Subject: [PATCH net-next v3 2/3] eventpoll: Add per-epoll busy poll packet budget Date: Thu, 25 Jan 2024 22:56:58 +0000 Message-Id: <20240125225704.12781-3-jdamato@fastly.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240125225704.12781-1-jdamato@fastly.com> References: <20240125225704.12781-1-jdamato@fastly.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789105182035106085 X-GMAIL-MSGID: 1789105182035106085 When using epoll-based busy poll, the packet budget is hardcoded to BUSY_POLL_BUDGET (8). Users may desire larger busy poll budgets, which can potentially increase throughput when busy polling under high network load. Other busy poll methods allow setting the busy poll budget via SO_BUSY_POLL_BUDGET, but epoll-based busy polling uses a hardcoded value. Fix this edge case by adding support for a per-epoll context busy poll packet budget. If not specified, the default value (BUSY_POLL_BUDGET) is used. Signed-off-by: Joe Damato --- fs/eventpoll.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/fs/eventpoll.c b/fs/eventpoll.c index 4503fec01278..40bd97477b91 100644 --- a/fs/eventpoll.c +++ b/fs/eventpoll.c @@ -229,6 +229,8 @@ struct eventpoll { unsigned int napi_id; /* busy poll timeout */ u64 busy_poll_usecs; + /* busy poll packet budget */ + u16 busy_poll_budget; #endif #ifdef CONFIG_DEBUG_LOCK_ALLOC @@ -437,10 +439,14 @@ static bool ep_busy_loop_end(void *p, unsigned long start_time) static bool ep_busy_loop(struct eventpoll *ep, int nonblock) { unsigned int napi_id = READ_ONCE(ep->napi_id); + u16 budget = READ_ONCE(ep->busy_poll_budget); + + if (!budget) + budget = BUSY_POLL_BUDGET; if ((napi_id >= MIN_NAPI_ID) && ep_busy_loop_on(ep)) { napi_busy_loop(napi_id, nonblock ? NULL : ep_busy_loop_end, ep, false, - BUSY_POLL_BUDGET); + budget); if (ep_events_available(ep)) return true; /* @@ -2098,6 +2104,7 @@ static int do_epoll_create(int flags) } #ifdef CONFIG_NET_RX_BUSY_POLL ep->busy_poll_usecs = 0; + ep->busy_poll_budget = 0; #endif ep->file = file; fd_install(fd, file);