From patchwork Thu Jan 25 22:56:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Damato X-Patchwork-Id: 192328 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:e09d:b0:103:945f:af90 with SMTP id gm29csp313873dyb; Thu, 25 Jan 2024 15:00:50 -0800 (PST) X-Google-Smtp-Source: AGHT+IEb40H95/76t+SujkFiaW9ousm3fC/g5wVfeqkQAc16Q9P09XeS/ckZGVJA4hYtSyZPUpU7 X-Received: by 2002:a05:6102:6c6:b0:46b:1402:825d with SMTP id m6-20020a05610206c600b0046b1402825dmr589735vsg.29.1706223650445; Thu, 25 Jan 2024 15:00:50 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706223650; cv=pass; d=google.com; s=arc-20160816; b=FQdO4ssTxBE+naGr9KmO5vRo4TN+3joIK9xZZ17EtH80p81l6Qec5KkciU/HGXfnOZ 7YWOuhADJCGyFiT7HFiI+Dx757rQhnb/AqmUofVGmxiHv/z10hNt/FBkuEVHEWrfo9HO TUqmiPqlfsopaJ2tn6aE6Pe//YgO4kWMCgpiEeFW7ZZ7VPIbgiXp/wZBULPze76ocHet 25lJGnzpq8fB6bh4wTbAzkRrpD+1U+/cEjjOVVxR1ERRcXpBOzPGLWhNiscZfFZ4Awwq t+OfmP00Hn5qg3mvegMrD7zaoX445h4X+sJD3WUWZkWAZ9n09b0IYyWyznL+0/bqDWHL hukw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=VV39OAkRpCVAHLKUI3lU3BvBDLbvOa7qJKwdNWq/qLo=; fh=xXEYDfOQzkMJq5YQq0KYkBbOiZYpvo5c1OyUsvtKbLQ=; b=Q60DpsuQUOt2cg0IZt+9p55gYXRad5QO++rd/9K7AcNhvO4eekIDLnjUK1GLZ/dJQD iWCp1UAOZfrOCJn6iiVe1pd+JjVhtekF/AJyEj06ZB4rpqrP/WVtkE53bGsRhGNI4rOU aOjQBDGAdj9C+FVqqwNIrXsrGyGuSbVBavEmwoedaDts3MRcBsV60ct45zjhKA02bh8C MviFLsSNDfmzfk0FXgct8/Kx3O8P5E/IM/YuMNOvp7e+NZxyXfttsyWMPLK7eKBjyG2E AGttWf6efeKij0UIFJ6mCP5xOQiZ0YgUPGf8NdqbyL2Imr4EhqlM6hzSquhfdIV5lJ8j Za5w== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@fastly.com header.s=google header.b="LNteaX/L"; arc=pass (i=1 spf=pass spfdomain=fastly.com dkim=pass dkdomain=fastly.com dmarc=pass fromdomain=fastly.com); spf=pass (google.com: domain of linux-kernel+bounces-39393-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-39393-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=fastly.com Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id hy3-20020a67e7c3000000b00467e4fb0d00si2550vsb.0.2024.01.25.15.00.50 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Jan 2024 15:00:50 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-39393-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@fastly.com header.s=google header.b="LNteaX/L"; arc=pass (i=1 spf=pass spfdomain=fastly.com dkim=pass dkdomain=fastly.com dmarc=pass fromdomain=fastly.com); spf=pass (google.com: domain of linux-kernel+bounces-39393-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-39393-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=fastly.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 9B7501C244E4 for ; Thu, 25 Jan 2024 23:00:27 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 9CDB4199B0; Thu, 25 Jan 2024 22:57:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b="LNteaX/L" Received: from mail-ot1-f49.google.com (mail-ot1-f49.google.com [209.85.210.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DA0FB12E72 for ; Thu, 25 Jan 2024 22:57:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.49 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706223469; cv=none; b=fUk0QA7NN2AhxUJKUtV7uMLXTsz88GhgLLAml//W6EG4fAc5vtpaSSGpK+OlUfjXewfWZkIT2yIsfiyRcC17QjbjEfC312fh+hyhxQ6oGLFl/XJ2ob53MALE58XlCV2UMCMOOfUmLbf25CrHOU0/ZgeKbdkpkHeXCvGYebGAqQA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706223469; c=relaxed/simple; bh=qkngexUKlfn+hMPunN78J4XlCpnaY0JzFvOY2CvOwFw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Xr8YJYc2GvJnz/pEDbxkGX6OFfm43PCVVLPKcfJ8MhMAb2pU2gEYc1XIrnnbafQFyMNNoicx9UWScRJ2J2Okhwo5CqSO56XgLGgdWCrAV727ycRIk1nb1ekxBuiVBYlHa8So7BAGut3t7//vaAh5mwCzRxtKxS2HbQeCFTPEFH0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com; spf=pass smtp.mailfrom=fastly.com; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b=LNteaX/L; arc=none smtp.client-ip=209.85.210.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=fastly.com Received: by mail-ot1-f49.google.com with SMTP id 46e09a7af769-6dc20b4595bso71537a34.0 for ; Thu, 25 Jan 2024 14:57:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; t=1706223466; x=1706828266; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VV39OAkRpCVAHLKUI3lU3BvBDLbvOa7qJKwdNWq/qLo=; b=LNteaX/L+A7wlIGmFAxlvRJaShq9jtlCsS7lt6LO3BAt07Nb19rEMQjRcCbK+Uam8B AzOj6bZyX0A1KQiqGpZHWYCmB10uu9k3eS+FBEcf8d71RhKk3zsQkwoij+2OQbtvxmKg xwJNF9rgsMOWKsDTUZWkAu5aK+9qfLiWBUDco= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706223466; x=1706828266; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VV39OAkRpCVAHLKUI3lU3BvBDLbvOa7qJKwdNWq/qLo=; b=DKZudb/ViVq6RcDUlEoB40XlTxat/KNSGgswtiHK1uALt/QPZVV+/PkfL+xi+e1hnB e2XQsYTboQdaQaqD9vmmVEiSfqHxHeM8wQ7qmSEzea+5cSoVPKGEggExQOWM6cRI16Yr Bmb2x/Ao2RfPmAvSkGFRS4EiNcmxuWNncu/mD3b2mIbNaIysaMvZACNFlto6EFSni4hh bYOl7WKCSpk1fc8+YfnORb2kp5JclN9L5k4sZull3KStSRdWalO81qUimoVOAlzjEYwt rCcfCxLXI5duQt6mZu9c+2v/scQJ2fg8+L6QHAyJi7Am2aTqH8dIGb0KKerC6z7mdg+A JrJg== X-Gm-Message-State: AOJu0Yxpt3cGN6ekVT9YUZons7qpBt3WbzwVp0HZnX8EqHvACtuz2kN2 QBxqtA16c1g5dthJ5pzCyTDhxnbyIJwKXe2ItPMioq3zWkiuKlEPzVKKWyWg3GgZN/bTcu8eaUk MPa3kXhxw7Qbpep3jNNWZ/T+oCQvDmhAPD9Z98sBwdgDsf/Z3S7ONwIXd1QcPQJOgjRzk//rLHn 5zhsHx95dUSl9B9Poc8lXb4HxYVdZ4PF27z2e3vBU75fD66w== X-Received: by 2002:a05:6359:4588:b0:176:6270:8cb6 with SMTP id no8-20020a056359458800b0017662708cb6mr549318rwb.25.1706223466122; Thu, 25 Jan 2024 14:57:46 -0800 (PST) Received: from localhost.localdomain ([2620:11a:c018:0:ea8:be91:8d1:f59b]) by smtp.gmail.com with ESMTPSA id z24-20020a631918000000b005d68962e1a7sm19948pgl.24.2024.01.25.14.57.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Jan 2024 14:57:45 -0800 (PST) From: Joe Damato To: linux-kernel@vger.kernel.org, netdev@vger.kernel.org Cc: chuck.lever@oracle.com, jlayton@kernel.org, linux-api@vger.kernel.org, brauner@kernel.org, edumazet@google.com, davem@davemloft.net, alexander.duyck@gmail.com, sridhar.samudrala@intel.com, kuba@kernel.org, willemdebruijn.kernel@gmail.com, weiwan@google.com, Joe Damato , Alexander Viro , Jan Kara , linux-fsdevel@vger.kernel.org (open list:FILESYSTEMS (VFS and infrastructure)) Subject: [PATCH net-next v3 1/3] eventpoll: support busy poll per epoll instance Date: Thu, 25 Jan 2024 22:56:57 +0000 Message-Id: <20240125225704.12781-2-jdamato@fastly.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240125225704.12781-1-jdamato@fastly.com> References: <20240125225704.12781-1-jdamato@fastly.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789105170549373516 X-GMAIL-MSGID: 1789105170549373516 Allow busy polling on a per-epoll context basis. The per-epoll context usec timeout value is preferred, but the pre-existing system wide sysctl value is still supported if it specified. Note that this change uses an xor: either per epoll instance busy polling is enabled on the epoll instance or system wide epoll is enabled. Enabling both is disallowed. Signed-off-by: Joe Damato --- fs/eventpoll.c | 49 +++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 45 insertions(+), 4 deletions(-) diff --git a/fs/eventpoll.c b/fs/eventpoll.c index 3534d36a1474..4503fec01278 100644 --- a/fs/eventpoll.c +++ b/fs/eventpoll.c @@ -227,6 +227,8 @@ struct eventpoll { #ifdef CONFIG_NET_RX_BUSY_POLL /* used to track busy poll napi_id */ unsigned int napi_id; + /* busy poll timeout */ + u64 busy_poll_usecs; #endif #ifdef CONFIG_DEBUG_LOCK_ALLOC @@ -386,12 +388,44 @@ static inline int ep_events_available(struct eventpoll *ep) READ_ONCE(ep->ovflist) != EP_UNACTIVE_PTR; } +/** + * busy_loop_ep_timeout - check if busy poll has timed out. The timeout value + * from the epoll instance ep is preferred, but if it is not set fallback to + * the system-wide global via busy_loop_timeout. + * + * @start_time: The start time used to compute the remaining time until timeout. + * @ep: Pointer to the eventpoll context. + * + * Return: true if the timeout has expired, false otherwise. + */ +static inline bool busy_loop_ep_timeout(unsigned long start_time, struct eventpoll *ep) +{ +#ifdef CONFIG_NET_RX_BUSY_POLL + unsigned long bp_usec = READ_ONCE(ep->busy_poll_usecs); + + if (bp_usec) { + unsigned long end_time = start_time + bp_usec; + unsigned long now = busy_loop_current_time(); + + return time_after(now, end_time); + } else { + return busy_loop_timeout(start_time); + } +#endif + return true; +} + #ifdef CONFIG_NET_RX_BUSY_POLL +static bool ep_busy_loop_on(struct eventpoll *ep) +{ + return !!ep->busy_poll_usecs ^ net_busy_loop_on(); +} + static bool ep_busy_loop_end(void *p, unsigned long start_time) { struct eventpoll *ep = p; - return ep_events_available(ep) || busy_loop_timeout(start_time); + return ep_events_available(ep) || busy_loop_ep_timeout(start_time, ep); } /* @@ -404,7 +438,7 @@ static bool ep_busy_loop(struct eventpoll *ep, int nonblock) { unsigned int napi_id = READ_ONCE(ep->napi_id); - if ((napi_id >= MIN_NAPI_ID) && net_busy_loop_on()) { + if ((napi_id >= MIN_NAPI_ID) && ep_busy_loop_on(ep)) { napi_busy_loop(napi_id, nonblock ? NULL : ep_busy_loop_end, ep, false, BUSY_POLL_BUDGET); if (ep_events_available(ep)) @@ -430,7 +464,8 @@ static inline void ep_set_busy_poll_napi_id(struct epitem *epi) struct socket *sock; struct sock *sk; - if (!net_busy_loop_on()) + ep = epi->ep; + if (!ep_busy_loop_on(ep)) return; sock = sock_from_file(epi->ffd.file); @@ -442,7 +477,6 @@ static inline void ep_set_busy_poll_napi_id(struct epitem *epi) return; napi_id = READ_ONCE(sk->sk_napi_id); - ep = epi->ep; /* Non-NAPI IDs can be rejected * or @@ -466,6 +500,10 @@ static inline void ep_set_busy_poll_napi_id(struct epitem *epi) { } +static inline bool ep_busy_loop_on(struct eventpoll *ep) +{ + return false; +} #endif /* CONFIG_NET_RX_BUSY_POLL */ /* @@ -2058,6 +2096,9 @@ static int do_epoll_create(int flags) error = PTR_ERR(file); goto out_free_fd; } +#ifdef CONFIG_NET_RX_BUSY_POLL + ep->busy_poll_usecs = 0; +#endif ep->file = file; fd_install(fd, file); return fd; From patchwork Thu Jan 25 22:56:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Damato X-Patchwork-Id: 192329 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:e09d:b0:103:945f:af90 with SMTP id gm29csp313994dyb; Thu, 25 Jan 2024 15:01:01 -0800 (PST) X-Google-Smtp-Source: AGHT+IEikulBljK7VItB279ZvlKZHFE3HpOFbCa1q7tzS/dr04sN3Zr/JrpEQb2sg+ZpMdqMQy70 X-Received: by 2002:a17:902:ea0a:b0:1d7:233b:314d with SMTP id s10-20020a170902ea0a00b001d7233b314dmr427020plg.127.1706223661279; Thu, 25 Jan 2024 15:01:01 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706223661; cv=pass; d=google.com; s=arc-20160816; b=pAH4jww/S6LdQMFQQaD5fGvdnG/YafKRb0yL0D0692+xblsodX77N76SYD3wQmCp4m tUFdrXQ6JrHLq7JB2QGMC2cNwv23DgQ5F5l8MwfMmSuMk2Ph1fjhKi+hO1XZqtdxLU0N uV0IBHum8wwOvvDXSzFLrALhY+5CVA/FmVfVjm0Y4zEeUrtEtW4O4qr1V0RLELuZVcKE MYq2ZlotbIamwRarwH+GHUb1NxPNwCl/YqsntWzsaZkaVTihw5b9jYCYNWWWZH/F9U2s +ItZ1lTlCc3RoOowPJ0VtsKL8qVU9aMot4xVT4mUbcbE3ftalogaUCsvCQAeWOnz1+OF u4Ew== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=o95DHArKhHsVxZUrtiTfK3GJrg5R+5zGPrNbLEZjDzs=; fh=xXEYDfOQzkMJq5YQq0KYkBbOiZYpvo5c1OyUsvtKbLQ=; b=dHuM/2ZCeQD75IyzQyI9Fk/ZBsywYYsiJbXKBZLYj9t8JY7Cc1iBmMG+b7szhNG1qr 6d4+iMzIyYrAQzhPNODL+EEhelSkQ2TSRarq1y4FjbfZflx1sOsUSkbIPvJjTdm7zDJW IE/8Qi9K0y7AccWhf0yU0UuVEhRJtjkkmTjt/o32Pv5CRpwWZH5YfIyJUkC3osZl7HgO a/gvenrCwxo9I00FbyLnZjNCuM9ZRMC0/GDqSNZdsmArrr0qyvNnxP7Dk8JPEC7VXtMd lnDqAtA3oDf+MNNtLgGJmLDA6O7A5HAI+B4iSpJ/1r3gLd1jGdEgWahw3e+DLSqzp3Zl eveA== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@fastly.com header.s=google header.b="gK/gao8L"; arc=pass (i=1 spf=pass spfdomain=fastly.com dkim=pass dkdomain=fastly.com dmarc=pass fromdomain=fastly.com); spf=pass (google.com: domain of linux-kernel+bounces-39394-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-39394-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=fastly.com Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id s8-20020a170902ea0800b001d78f90818dsi24567plg.130.2024.01.25.15.01.01 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Jan 2024 15:01:01 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-39394-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; dkim=pass header.i=@fastly.com header.s=google header.b="gK/gao8L"; arc=pass (i=1 spf=pass spfdomain=fastly.com dkim=pass dkdomain=fastly.com dmarc=pass fromdomain=fastly.com); spf=pass (google.com: domain of linux-kernel+bounces-39394-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-39394-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=fastly.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id BDD0B29128C for ; Thu, 25 Jan 2024 23:00:49 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id B953F1B59C; Thu, 25 Jan 2024 22:57:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b="gK/gao8L" Received: from mail-il1-f180.google.com (mail-il1-f180.google.com [209.85.166.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 601EF15AF9 for ; Thu, 25 Jan 2024 22:57:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706223473; cv=none; b=MhVfCqUu/gfslf7lUB+71zHNJX+ZP6urkpyZ3KoT1TtOROVqqR9kLgz/cZ0J6FOnGp7O4OeLLiJIV2NtH34cg8E70TqlOZ0gXaGoLCpPGhRkxUibJdOVgjTMb61o4XJ6zZbtsWCoG2SQMouFUu+/s4Aos1/MFG2WFCTdjH34I2Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706223473; c=relaxed/simple; bh=3yteI2ybUubAH5CujHpgcS7qoJxKc8vJCs7EzjX9vMQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=d+1Oz9S4L/c2OspY03GRCI8UQolNcYdyo+nQnqee2yQlQk4/PERVP5Eb2MTqR0FYBoLFdiVHaFemXxTjBYXczgL1AepNGFKlXD1HGKEMfRc7gawAqCfn9w+g/ahN2ZTgLa/Az3FZ3neS4dzW2n0OV03/jsPJ/5gECjaGrwmUMvM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com; spf=pass smtp.mailfrom=fastly.com; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b=gK/gao8L; arc=none smtp.client-ip=209.85.166.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=fastly.com Received: by mail-il1-f180.google.com with SMTP id e9e14a558f8ab-3606f3f2f37so37727255ab.0 for ; Thu, 25 Jan 2024 14:57:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; t=1706223468; x=1706828268; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=o95DHArKhHsVxZUrtiTfK3GJrg5R+5zGPrNbLEZjDzs=; b=gK/gao8LZPQ14f+0l1Z/+ivgHpju3xt99PQTI/cOjX4zckT76Ao7/ofkfMZOM/dKY3 huhuUoAasEEsNHDBwxKIkVMhT1VBnxGfnbjiPJKJbswZYdFIkXci5IsVQOmx9poTHHD1 hWZFcA8fGZDbx+laWc3yhOV75Uj8m/jdqrEIM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706223468; x=1706828268; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=o95DHArKhHsVxZUrtiTfK3GJrg5R+5zGPrNbLEZjDzs=; b=wjCQj0YAQpIhD7OWs3WSAfC9IreUhk1sCkJFSGRxAB2gCGpXgR9iE43yhSoc9v+g+1 8fcalFX7vcSk0L7dWBQWVv9ABo8bvJ9NF4xV1S9xiu5EhdT/U7/1E2entDjouveErV7y 2OWt23x4pH8G59izuTAihzmUlcrI2xdMaYapSzHqBZBVChdEYw5gM6IeOcxe2d/e+Fn2 IsbeNF/JXEdhupBWtWTMCy0Zr6UoS7+D4f7FsXapmbL1Afi11kZdho80W0HY6+5FaEQW 1O9S2lWdRbEd+tPzQS0FizCOiUOWbytCyyG/yEdmt5SM43UAKRBlKYiBGSkHS+emhVwj yhXg== X-Gm-Message-State: AOJu0YziTTNUYl1oTWE+wInC37WHLvrUX8H5JZsT9jYukQqTGjzUVCJ2 FzEjVRYvpYQOmT4sSBOZuvU8fsoPh6xQTnnKsKS7kJ9hRbppI1C42xnQH3gtFU2JuvCPfgdk0NV QwZL0c2K3kvbHMgtNN4RVtzFIqEn5HozUbWIa5rzxaiuIUKH+NrWoaiOdKRr1d/MU36ryOic5Kl YMINgmSr3l05UfMNOks6l9+Gu5pMTWWOVZQPiv5pouWUsayg== X-Received: by 2002:a05:6e02:12ea:b0:360:76fb:a894 with SMTP id l10-20020a056e0212ea00b0036076fba894mr557277iln.62.1706223467756; Thu, 25 Jan 2024 14:57:47 -0800 (PST) Received: from localhost.localdomain ([2620:11a:c018:0:ea8:be91:8d1:f59b]) by smtp.gmail.com with ESMTPSA id z24-20020a631918000000b005d68962e1a7sm19948pgl.24.2024.01.25.14.57.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Jan 2024 14:57:47 -0800 (PST) From: Joe Damato To: linux-kernel@vger.kernel.org, netdev@vger.kernel.org Cc: chuck.lever@oracle.com, jlayton@kernel.org, linux-api@vger.kernel.org, brauner@kernel.org, edumazet@google.com, davem@davemloft.net, alexander.duyck@gmail.com, sridhar.samudrala@intel.com, kuba@kernel.org, willemdebruijn.kernel@gmail.com, weiwan@google.com, Joe Damato , Alexander Viro , Jan Kara , linux-fsdevel@vger.kernel.org (open list:FILESYSTEMS (VFS and infrastructure)) Subject: [PATCH net-next v3 2/3] eventpoll: Add per-epoll busy poll packet budget Date: Thu, 25 Jan 2024 22:56:58 +0000 Message-Id: <20240125225704.12781-3-jdamato@fastly.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240125225704.12781-1-jdamato@fastly.com> References: <20240125225704.12781-1-jdamato@fastly.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789105182035106085 X-GMAIL-MSGID: 1789105182035106085 When using epoll-based busy poll, the packet budget is hardcoded to BUSY_POLL_BUDGET (8). Users may desire larger busy poll budgets, which can potentially increase throughput when busy polling under high network load. Other busy poll methods allow setting the busy poll budget via SO_BUSY_POLL_BUDGET, but epoll-based busy polling uses a hardcoded value. Fix this edge case by adding support for a per-epoll context busy poll packet budget. If not specified, the default value (BUSY_POLL_BUDGET) is used. Signed-off-by: Joe Damato --- fs/eventpoll.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/fs/eventpoll.c b/fs/eventpoll.c index 4503fec01278..40bd97477b91 100644 --- a/fs/eventpoll.c +++ b/fs/eventpoll.c @@ -229,6 +229,8 @@ struct eventpoll { unsigned int napi_id; /* busy poll timeout */ u64 busy_poll_usecs; + /* busy poll packet budget */ + u16 busy_poll_budget; #endif #ifdef CONFIG_DEBUG_LOCK_ALLOC @@ -437,10 +439,14 @@ static bool ep_busy_loop_end(void *p, unsigned long start_time) static bool ep_busy_loop(struct eventpoll *ep, int nonblock) { unsigned int napi_id = READ_ONCE(ep->napi_id); + u16 budget = READ_ONCE(ep->busy_poll_budget); + + if (!budget) + budget = BUSY_POLL_BUDGET; if ((napi_id >= MIN_NAPI_ID) && ep_busy_loop_on(ep)) { napi_busy_loop(napi_id, nonblock ? NULL : ep_busy_loop_end, ep, false, - BUSY_POLL_BUDGET); + budget); if (ep_events_available(ep)) return true; /* @@ -2098,6 +2104,7 @@ static int do_epoll_create(int flags) } #ifdef CONFIG_NET_RX_BUSY_POLL ep->busy_poll_usecs = 0; + ep->busy_poll_budget = 0; #endif ep->file = file; fd_install(fd, file); From patchwork Thu Jan 25 22:56:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Damato X-Patchwork-Id: 192330 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:e09d:b0:103:945f:af90 with SMTP id gm29csp314226dyb; Thu, 25 Jan 2024 15:01:16 -0800 (PST) X-Google-Smtp-Source: AGHT+IGIs3qNP5HVW7UTbGrF9OPFucMDV+tRbjGJXF3wDZ0BSAW8RJMw+dpwqC8S1W9FLVB9NhSj X-Received: by 2002:a17:902:d48b:b0:1d4:25d5:5a75 with SMTP id c11-20020a170902d48b00b001d425d55a75mr516776plg.74.1706223676295; Thu, 25 Jan 2024 15:01:16 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706223676; cv=pass; d=google.com; s=arc-20160816; b=buSgSvouefEU9SAg7gq7Q3zXWaMBN8FQr7bELx8EwF0PFOmvWDMnXb6KOUbSmsSXWH nhaCNxfSiPxUAQSdRKNx/uEv3Cx//xqWLKf2qpFylvA+JZpzBchQY6W+JSpotSZ84B+0 IijK8SrmII31+l7H2OAeTS2KxIRjJxlsLKKZl5smtBiOWFgWZtYlxvRb1gRUEP9zb3Tu qpYWUDOB23xPFQkZ9yPP4cJ2znqh+c4YIvv27YLzG3GF9Ic/WJQszqU2TYuGmOCDcHFJ KiE7GNcJQXMYWoZVLM8NBqBCCiIqgsFbvzyHZyQzJGqRL4psL82uINBEKW7uNl0xWiLu IR/g== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=lGA3mbdBd/zeAlBJTaRnqUyMp6Mg1t+BBnoYkgoxqFQ=; fh=13ML0Ts+TAZCBK5Xu4Tytdb43Hj7j+tlNOBTtrs4qSk=; b=gT0alkIp3k0kvPLaQa49mkzd7tO2PFbXmNQL8Li8rX719iwd3C5ZGQHeiQzdzS89bU 6UcZ6A2yvM7YIuXQKTR1L/u66U5ZzolddVbz08txFwl1KNl4t5knWxahY2pO6FYeVUXm +kUQAupZWAhGwxg6ntS/jdUCrolGSFDWjuYpu7VCFwO1ujXcuj5tej+6zOlItAEGrQZA WPOB+Yrl17eW+HkhxG2lRcIoUul9h8Xr+EB4w8L/49NAHSUr0cx/u0lSax6aDhtwlbKc hFjHEr6PJTl8B2EWqVt3WFQclPlY5AsXp+JP3OnIrLvr8RKiDGDWatDezGgszpp2ZLOr IjdQ== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@fastly.com header.s=google header.b=fksUhXYA; arc=pass (i=1 spf=pass spfdomain=fastly.com dkim=pass dkdomain=fastly.com dmarc=pass fromdomain=fastly.com); spf=pass (google.com: domain of linux-kernel+bounces-39395-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-39395-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=fastly.com Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id ix6-20020a170902f80600b001d787d9d62esi13191plb.210.2024.01.25.15.01.16 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Jan 2024 15:01:16 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-39395-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; dkim=pass header.i=@fastly.com header.s=google header.b=fksUhXYA; arc=pass (i=1 spf=pass spfdomain=fastly.com dkim=pass dkdomain=fastly.com dmarc=pass fromdomain=fastly.com); spf=pass (google.com: domain of linux-kernel+bounces-39395-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-39395-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=fastly.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id A6CC728F738 for ; Thu, 25 Jan 2024 23:01:09 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id CD2041A726; Thu, 25 Jan 2024 22:57:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b="fksUhXYA" Received: from mail-oo1-f50.google.com (mail-oo1-f50.google.com [209.85.161.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D97241947F for ; Thu, 25 Jan 2024 22:57:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.161.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706223476; cv=none; b=DVztmlm0tu8XouYaCp/B4+H889OO1fylKwZ3w82UkVOCVeDBNWt9YKwex+7xNINxi4jFFuCkmB4M9kvZzPqwYoYMvZVdqxXW5ffWdgpMi0HQLAIcpsD6ZjFjoaBnt80eqLBmiZ3oCbiKhmgBqbR5OUaIiyimypVldW4yZSBFk6k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706223476; c=relaxed/simple; bh=BCSqrZ9DLkXBlfucIFiKDapTJzswZizsRiSDn8OrvBQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=AIkWXlN1pbVP9eVReuzVPEithW27sz/eUdnSRyEI+2efYvb5zzSAsf//0+8tlvxVKdy7eC+wBURiRdcE7Spj/T4+ZZOcyfn2n3v6X6KXcHhURp9o83rOUse1kA5r5Qdj9ZSDHuZvWhcsUd1Yd7WQ5rHXtXe9WUeYD0MkCOZGLLI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com; spf=pass smtp.mailfrom=fastly.com; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b=fksUhXYA; arc=none smtp.client-ip=209.85.161.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=fastly.com Received: by mail-oo1-f50.google.com with SMTP id 006d021491bc7-599ef8b1c35so27555eaf.0 for ; Thu, 25 Jan 2024 14:57:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; t=1706223473; x=1706828273; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=lGA3mbdBd/zeAlBJTaRnqUyMp6Mg1t+BBnoYkgoxqFQ=; b=fksUhXYAa2oMRpsDfRj3YHst/iyi9aJLZRzTEHZ++MaS/NLAOoBSKbwG2gZ6UKH4RL mD63+XN/vjWEf2f10Qm4OtbHDZgVd3nMJsoOEvzv/PdPpfsdTEH8fEe/myeHQ9go0cmV DPM9PbYHXWGRyJd89tPDG9zTkoViWNpLnM6DI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706223473; x=1706828273; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lGA3mbdBd/zeAlBJTaRnqUyMp6Mg1t+BBnoYkgoxqFQ=; b=sZ4tlV43uocpcwnpwkkXFmkxGMPUXc0D1ecQh50UfYuGYH0XVsRYsE1pmP2kInjG1U 2kTDCFaichmt19PPIbl1C/29iTiH1Z17WOk5eh0wX3ZiS5lz9yjodUWmzW8umqjshVnG d3wCpJR7du7/PxTMb9ML75Lo7rdag5s68wDuINOViMTEkLmoA2yCvByf+ApEqF6BlO+s iXzMKhKKb39YKdhOcoDpZdYkK+XHikv5r6RMsSu4D/JJT9Di3GPXuxXDQ+haMh0YTcRR RkCQ/4jqcwL/kJlfArCQ+emhuveMt+qCdE7vIuCixnSOVMu/t5m9nsgN6r1Axw8xX5PD tGlg== X-Gm-Message-State: AOJu0Yyk4Z0n9kpM2XiWqOfaKwq/TCXZm53ypD4PDyAAmaHvgluSQaWL J2WFrPSqzTUxtohZhaRrWThScamTt38ZyjplKQcURVUzLX6/e3sryWqmE2DHZfbZNIYU6b0J3Ad mLGKq9JtS6nWVsHQMTU9AfY3Tg1DOLzgOaBuWa7pmCR9Snn3z/8BWdBPNXCB1TX5S7HM5ekBjKN SmhIGQ/AUnQAqxyiAsOvw1XRwAZMZ0E7X93Y1argBM8kwatg== X-Received: by 2002:a05:6358:9497:b0:176:916b:462b with SMTP id i23-20020a056358949700b00176916b462bmr515749rwb.4.1706223473551; Thu, 25 Jan 2024 14:57:53 -0800 (PST) Received: from localhost.localdomain ([2620:11a:c018:0:ea8:be91:8d1:f59b]) by smtp.gmail.com with ESMTPSA id z24-20020a631918000000b005d68962e1a7sm19948pgl.24.2024.01.25.14.57.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Jan 2024 14:57:53 -0800 (PST) From: Joe Damato To: linux-kernel@vger.kernel.org, netdev@vger.kernel.org Cc: chuck.lever@oracle.com, jlayton@kernel.org, linux-api@vger.kernel.org, brauner@kernel.org, edumazet@google.com, davem@davemloft.net, alexander.duyck@gmail.com, sridhar.samudrala@intel.com, kuba@kernel.org, willemdebruijn.kernel@gmail.com, weiwan@google.com, Joe Damato , Jonathan Corbet , Alexander Viro , Jan Kara , Michael Ellerman , Greg Kroah-Hartman , Nathan Lynch , Steve French , Thomas Zimmermann , Jiri Slaby , Julien Panis , Arnd Bergmann , Andrew Waterman , Thomas Huth , Palmer Dabbelt , linux-doc@vger.kernel.org (open list:DOCUMENTATION), linux-fsdevel@vger.kernel.org (open list:FILESYSTEMS (VFS and infrastructure)) Subject: [PATCH net-next v3 3/3] eventpoll: Add epoll ioctl for epoll_params Date: Thu, 25 Jan 2024 22:56:59 +0000 Message-Id: <20240125225704.12781-4-jdamato@fastly.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240125225704.12781-1-jdamato@fastly.com> References: <20240125225704.12781-1-jdamato@fastly.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789105197533069733 X-GMAIL-MSGID: 1789105197533069733 Add an ioctl for getting and setting epoll_params. User programs can use this ioctl to get and set the busy poll usec time or packet budget params for a specific epoll context. Signed-off-by: Joe Damato --- .../userspace-api/ioctl/ioctl-number.rst | 1 + fs/eventpoll.c | 64 +++++++++++++++++++ include/uapi/linux/eventpoll.h | 12 ++++ 3 files changed, 77 insertions(+) diff --git a/Documentation/userspace-api/ioctl/ioctl-number.rst b/Documentation/userspace-api/ioctl/ioctl-number.rst index 457e16f06e04..b33918232f78 100644 --- a/Documentation/userspace-api/ioctl/ioctl-number.rst +++ b/Documentation/userspace-api/ioctl/ioctl-number.rst @@ -309,6 +309,7 @@ Code Seq# Include File Comments 0x89 0B-DF linux/sockios.h 0x89 E0-EF linux/sockios.h SIOCPROTOPRIVATE range 0x89 F0-FF linux/sockios.h SIOCDEVPRIVATE range +0x8A 00-1F linux/eventpoll.h 0x8B all linux/wireless.h 0x8C 00-3F WiNRADiO driver diff --git a/fs/eventpoll.c b/fs/eventpoll.c index 40bd97477b91..73ae886efb8a 100644 --- a/fs/eventpoll.c +++ b/fs/eventpoll.c @@ -6,6 +6,8 @@ * Davide Libenzi */ +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt + #include #include #include @@ -37,6 +39,7 @@ #include #include #include +#include #include /* @@ -495,6 +498,39 @@ static inline void ep_set_busy_poll_napi_id(struct epitem *epi) ep->napi_id = napi_id; } +static long ep_eventpoll_bp_ioctl(struct file *file, unsigned int cmd, + unsigned long arg) +{ + struct eventpoll *ep; + struct epoll_params epoll_params; + void __user *uarg = (void __user *) arg; + + ep = file->private_data; + + switch (cmd) { + case EPIOCSPARAMS: + if (copy_from_user(&epoll_params, uarg, sizeof(epoll_params))) + return -EFAULT; + + if (epoll_params.busy_poll_budget > NAPI_POLL_WEIGHT && + !capable(CAP_NET_ADMIN)) + return -EPERM; + + ep->busy_poll_usecs = epoll_params.busy_poll_usecs; + ep->busy_poll_budget = epoll_params.busy_poll_budget; + return 0; + case EPIOCGPARAMS: + memset(&epoll_params, 0, sizeof(epoll_params)); + epoll_params.busy_poll_usecs = ep->busy_poll_usecs; + epoll_params.busy_poll_budget = ep->busy_poll_budget; + if (copy_to_user(uarg, &epoll_params, sizeof(epoll_params))) + return -EFAULT; + return 0; + default: + return -ENOIOCTLCMD; + } +} + #else static inline bool ep_busy_loop(struct eventpoll *ep, int nonblock) @@ -510,6 +546,12 @@ static inline bool ep_busy_loop_on(struct eventpoll *ep) { return false; } + +static long ep_eventpoll_bp_ioctl(struct file *file, unsigned int cmd, + unsigned long arg) +{ + return -EOPNOTSUPP; +} #endif /* CONFIG_NET_RX_BUSY_POLL */ /* @@ -869,6 +911,26 @@ static void ep_clear_and_put(struct eventpoll *ep) ep_free(ep); } +static long ep_eventpoll_ioctl(struct file *file, unsigned int cmd, unsigned long arg) +{ + int ret; + + if (!is_file_epoll(file)) + return -EINVAL; + + switch (cmd) { + case EPIOCSPARAMS: + case EPIOCGPARAMS: + ret = ep_eventpoll_bp_ioctl(file, cmd, arg); + break; + default: + ret = -EINVAL; + break; + } + + return ret; +} + static int ep_eventpoll_release(struct inode *inode, struct file *file) { struct eventpoll *ep = file->private_data; @@ -975,6 +1037,8 @@ static const struct file_operations eventpoll_fops = { .release = ep_eventpoll_release, .poll = ep_eventpoll_poll, .llseek = noop_llseek, + .unlocked_ioctl = ep_eventpoll_ioctl, + .compat_ioctl = compat_ptr_ioctl, }; /* diff --git a/include/uapi/linux/eventpoll.h b/include/uapi/linux/eventpoll.h index cfbcc4cc49ac..8eb0fdbce995 100644 --- a/include/uapi/linux/eventpoll.h +++ b/include/uapi/linux/eventpoll.h @@ -85,4 +85,16 @@ struct epoll_event { __u64 data; } EPOLL_PACKED; +struct epoll_params { + u64 busy_poll_usecs; + u16 busy_poll_budget; + + /* for future fields */ + u8 data[118]; +} EPOLL_PACKED; + +#define EPOLL_IOC_TYPE 0x8A +#define EPIOCSPARAMS _IOW(EPOLL_IOC_TYPE, 0x01, struct epoll_params) +#define EPIOCGPARAMS _IOR(EPOLL_IOC_TYPE, 0x02, struct epoll_params) + #endif /* _UAPI_LINUX_EVENTPOLL_H */