From patchwork Tue Feb 13 06:16:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Damato X-Patchwork-Id: 200236 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp354035dyb; Mon, 12 Feb 2024 22:18:08 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWfLTa/FH26wG7/30Bssi5UTGL39qJQn2xcwvw/LWsgICnnqjGqKDKhUOl+VXNP1HkfQSmPjOdOTRmoqVrk+FIM3+9h5g== X-Google-Smtp-Source: AGHT+IGVbjNVH0FvnR40OagAIp1gwTvTtm4SRwR4/NIZPTeg9ED2xB1d4htbFKknaDWRsLWp0ZNr X-Received: by 2002:a05:622a:104c:b0:42d:aa92:ce2 with SMTP id f12-20020a05622a104c00b0042daa920ce2mr1993868qte.27.1707805088433; Mon, 12 Feb 2024 22:18:08 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707805088; cv=pass; d=google.com; s=arc-20160816; b=O/pu5SNsIuYsrSqxLWsus8V3DyMS8ZO/rGumGrYFdvoxzI0pDlX0lsQjOvS4QKJW6Y qgp7QE1+/zYJWyAhf4fTfvCL1YwEpoERQr9RQfeqDQL1KH7EM/tB9ncQtwhHVGz4Pbjh E0KQy2dBrTi66xmSrjpAL+0FcFXfWJ7n7W+Bqw9z6dH5rTKsOK6IvxDUFHQgWO4SxEgP ylknBVp0DpK1due/9IfOlklC2CaqaXhPen/uJ3y7UVFFAvJU8hf0+YICt1xwYrMJmUmq fFsb8gNOg4LONwNcNURlD6f1ci348gl57khd3yd6aA4w0TmUxK5cbvYdmDYvyJvVwbS7 6eqw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=TWK7Tcl4v78WKhBmDih4hp4aKoxAkEp5jIDo2VrMgEA=; fh=o1g3XBKqq7VqvTu4UTz/wBgemkH2gvk1M+raIP8exW4=; b=ATfhOsTCfsKYeq6ToFq5Lwwshji+kGZ5U9IXU/rVmTKqof0o0Gx3nWUj+1Imuy+YSS NVN9bnV7JWEI2xxBKKalEGH3uH1k5tj70G+SHA8MUPF442AlBEQE/yOtKqkFCHiOqp5Q PeQjrw+g32WGA7QNvyNbwwLKeUGFKgc6Gj1Kjw40sA8T5ZaY0kR+dzwWFA9sm+vOOvFP ufXAK5rw5Y3OHT3yF8FyygFvgc6lWW6o5KLU7i8wKcbDbATXu3zHXagefTYiAOW9Zzlw dIRe93btqHYY64Nq33vOv/vZU22k83DPgz+TC2SZI9wWhJpu+mgwQnWI00sHVudnPKDF HW8A==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@fastly.com header.s=google header.b=dZcIsh7s; arc=pass (i=1 spf=pass spfdomain=fastly.com dkim=pass dkdomain=fastly.com dmarc=pass fromdomain=fastly.com); spf=pass (google.com: domain of linux-kernel+bounces-62999-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-62999-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=fastly.com X-Forwarded-Encrypted: i=2; AJvYcCUxUMlfswo865bnICRBS9vKu2XZEKt3xYThm0AgiKwhqwuhXIkpzQ192ytZbqzgbQVfxIDc91+igjXH8RKrD4ztPmW4aQ== Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id x5-20020ac85385000000b0042cee77dd50si1961539qtp.238.2024.02.12.22.18.08 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Feb 2024 22:18:08 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-62999-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@fastly.com header.s=google header.b=dZcIsh7s; arc=pass (i=1 spf=pass spfdomain=fastly.com dkim=pass dkdomain=fastly.com dmarc=pass fromdomain=fastly.com); spf=pass (google.com: domain of linux-kernel+bounces-62999-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-62999-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=fastly.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 2F7D21C21BC3 for ; Tue, 13 Feb 2024 06:18:08 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 07D57179AE; Tue, 13 Feb 2024 06:17:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b="dZcIsh7s" Received: from mail-pf1-f172.google.com (mail-pf1-f172.google.com [209.85.210.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 18E06134DE for ; Tue, 13 Feb 2024 06:17:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707805023; cv=none; b=eaJPjKnxBd6K/p8CTHoA3NEleiKBRJonANhebfQFSkP+sRhP3miQ8QFFxosY3rVkz5rpVDa+bWe0+DwcaLGBaC9u6blcwm0xpP9caPaR08ZtDZvFXMEu8J/UgN+v4IwvBHEaC2nUNK4t970uc0pazYSJnf5BB6dFX5tTT0Cm4Fc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707805023; c=relaxed/simple; bh=+BdDiZtulZrbvLpADjlE62DUPuOp3KejqJWNkuE2SMM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=su5RvpiM7/KkoMHMvThPeuKj5u4xmj8r2qlzS1kq/UGCyc5uSG15AfIAgUZLELCWJ6J58/DbmKR6MKw/+ZX4nlLk2ELvNkVNCJNd3qcg74RIzP42IaOuZQ8f1AhanMyhGI39DezCvk2y9ooWqO8aNMJzi8qhIkuxeAxGES/tDcs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com; spf=pass smtp.mailfrom=fastly.com; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b=dZcIsh7s; arc=none smtp.client-ip=209.85.210.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=fastly.com Received: by mail-pf1-f172.google.com with SMTP id d2e1a72fcca58-6e0a4823881so1397279b3a.0 for ; Mon, 12 Feb 2024 22:17:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; t=1707805021; x=1708409821; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TWK7Tcl4v78WKhBmDih4hp4aKoxAkEp5jIDo2VrMgEA=; b=dZcIsh7sDKQJ8LRc01yzKn89AtehuAqcEwEA7cApStZTVvKuDvLarwxXCqdhW+DKT8 YLp4Q4Aay2K4aQwkFeTqnllQm0f3qLq9ajXahrohZlc5vAFUhyMo1XCZhVn3FLiJHcp0 NTTaFSvxWtlgMwV7Ap5IGyzgbVBd3cAQfzs/Q= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707805021; x=1708409821; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TWK7Tcl4v78WKhBmDih4hp4aKoxAkEp5jIDo2VrMgEA=; b=O5V1XZqPxbIsu+TW+3CwSr952N51ffWNx2i9aSm6HUgrAtsEohwnIPNfMtsLkMXEj5 9+tgzKvVhb+Us6hwFI+tzDzv/I+HShlAOFiW+C31l32nE2aIIixo3Ii4Yewa2YkuMVNL XoK8EaL0bjB2PcbXZ8N/LGYNDvPDstwFeu8KN8jB0ljFXmIMtlPmJrvkaFgM9Ee3OrPq xvsCBw/2DsE5rMsvHD0DgwO1J57OmcRSg1gWPLhr3D/ePumPzYcUWBS5Qhx7OhlSFLby CdY0dFXU6VmGEE6dqIHI1KKmbFLEttu1zi0MF44qQr2CMUCYF2ch7aPBOjNnO9Vn1zrf HTFA== X-Gm-Message-State: AOJu0YzUNq2EAZJ/tzgr+hsUGQEgy7lUuT1hNB/u2mMc6jOfrDx5sa0p vpfZW6Mxz6caVrR7RJZCxdmJ2ncYcRtyt9FEYZ1K0Trtprp8SSSE1653lN2bNlSowUocrPEUhxR 80IqzyVPpSXqTD9OmgkSwopTejY3woq46/2RmJLsh0vc3ki9JVNa22V/6qnpK5I+zIoGBQopRKp FaQGY260fibiy1Go5/3HKQnMxjZkinOuWG32RImQzxhT0= X-Received: by 2002:a05:6a00:bc6:b0:6e0:99d6:490 with SMTP id x6-20020a056a000bc600b006e099d60490mr7327725pfu.21.1707805020617; Mon, 12 Feb 2024 22:17:00 -0800 (PST) X-Forwarded-Encrypted: i=1; AJvYcCXmL7Vz3PAAYJ2YuBcOicTcHD7G3MPIBgpPVxgOqWx57ckQb0LQsp+zo+1xBYVJvVxX1xnFMSlslqEurFjhd8BN1Ask2EQZc6mdjdKDIMOrD2H0lpbgk0puZwOgN13OsNWKlrdGSpZQkn99PMb/TF7Oj1g/IwqZQq5GO+n3XY6NDDlRAtv12zlRd0iJIDQMrq7FkJSptx6nz6yJAAagghDmHZIA9mwS4SqtRXSu3uJqtwLYyikPd5fzSj0z4vZx78G9/mykhCZyiEGpwlQkKn63Jis6NDCbIJxOJsjVE0w5tj5tfF91tqmc3P213LPGsFXADlJRzdIJB//0Ulx206ft3k7pQWfiWaGLP5U4RECb/+vlQxJkkviTJuDmWX7O1JcYb4w5gdEt7RG4FG9CSrJ3JD7sSFtfiPLmpmKlPamtwsXW6cFbL/F06xzeuX0PLxAxn1O16eKif/KlZfe/lU/gZQfwRBIMdKexiAyTmgInf0DAvzL142wZqKm6vdLgmm4VbYXpnlpgP8utTiAXPZAKZzb2dnMUxVKYLzCP62FKYAYD0md7sF9dz4n2FLx1qDWBzGhXgJhHpdvg65qvOan1XyFJTk5wI0x9FBMjuUvRwkjUb2YMyMNjTcY9t34= Received: from localhost.localdomain ([2620:11a:c018:0:ea8:be91:8d1:f59b]) by smtp.gmail.com with ESMTPSA id n19-20020a638f13000000b005dc87f5dfcfsm342936pgd.78.2024.02.12.22.16.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Feb 2024 22:17:00 -0800 (PST) From: Joe Damato To: linux-kernel@vger.kernel.org, netdev@vger.kernel.org Cc: chuck.lever@oracle.com, jlayton@kernel.org, linux-api@vger.kernel.org, brauner@kernel.org, edumazet@google.com, davem@davemloft.net, alexander.duyck@gmail.com, sridhar.samudrala@intel.com, kuba@kernel.org, willemdebruijn.kernel@gmail.com, weiwan@google.com, David.Laight@ACULAB.COM, arnd@arndb.de, sdf@google.com, amritha.nambiar@intel.com, Joe Damato , Alexander Viro , Jan Kara , linux-fsdevel@vger.kernel.org (open list:FILESYSTEMS (VFS and infrastructure)) Subject: [PATCH net-next v8 1/4] eventpoll: support busy poll per epoll instance Date: Tue, 13 Feb 2024 06:16:42 +0000 Message-Id: <20240213061652.6342-2-jdamato@fastly.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240213061652.6342-1-jdamato@fastly.com> References: <20240213061652.6342-1-jdamato@fastly.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790763428244798157 X-GMAIL-MSGID: 1790763428244798157 Allow busy polling on a per-epoll context basis. The per-epoll context usec timeout value is preferred, but the pre-existing system wide sysctl value is still supported if it specified. busy_poll_usecs is a u32, but in a follow up patch the ioctl provided to the user only allows setting a value from 0 to S32_MAX. Signed-off-by: Joe Damato Acked-by: Stanislav Fomichev Reviewed-by: Eric Dumazet --- fs/eventpoll.c | 44 +++++++++++++++++++++++++++++++++++++++----- 1 file changed, 39 insertions(+), 5 deletions(-) diff --git a/fs/eventpoll.c b/fs/eventpoll.c index 3534d36a1474..401f865eced9 100644 --- a/fs/eventpoll.c +++ b/fs/eventpoll.c @@ -227,6 +227,8 @@ struct eventpoll { #ifdef CONFIG_NET_RX_BUSY_POLL /* used to track busy poll napi_id */ unsigned int napi_id; + /* busy poll timeout */ + u32 busy_poll_usecs; #endif #ifdef CONFIG_DEBUG_LOCK_ALLOC @@ -387,11 +389,41 @@ static inline int ep_events_available(struct eventpoll *ep) } #ifdef CONFIG_NET_RX_BUSY_POLL +/** + * busy_loop_ep_timeout - check if busy poll has timed out. The timeout value + * from the epoll instance ep is preferred, but if it is not set fallback to + * the system-wide global via busy_loop_timeout. + * + * @start_time: The start time used to compute the remaining time until timeout. + * @ep: Pointer to the eventpoll context. + * + * Return: true if the timeout has expired, false otherwise. + */ +static bool busy_loop_ep_timeout(unsigned long start_time, + struct eventpoll *ep) +{ + unsigned long bp_usec = READ_ONCE(ep->busy_poll_usecs); + + if (bp_usec) { + unsigned long end_time = start_time + bp_usec; + unsigned long now = busy_loop_current_time(); + + return time_after(now, end_time); + } else { + return busy_loop_timeout(start_time); + } +} + +static bool ep_busy_loop_on(struct eventpoll *ep) +{ + return !!ep->busy_poll_usecs || net_busy_loop_on(); +} + static bool ep_busy_loop_end(void *p, unsigned long start_time) { struct eventpoll *ep = p; - return ep_events_available(ep) || busy_loop_timeout(start_time); + return ep_events_available(ep) || busy_loop_ep_timeout(start_time, ep); } /* @@ -404,7 +436,7 @@ static bool ep_busy_loop(struct eventpoll *ep, int nonblock) { unsigned int napi_id = READ_ONCE(ep->napi_id); - if ((napi_id >= MIN_NAPI_ID) && net_busy_loop_on()) { + if (napi_id >= MIN_NAPI_ID && ep_busy_loop_on(ep)) { napi_busy_loop(napi_id, nonblock ? NULL : ep_busy_loop_end, ep, false, BUSY_POLL_BUDGET); if (ep_events_available(ep)) @@ -425,12 +457,12 @@ static bool ep_busy_loop(struct eventpoll *ep, int nonblock) */ static inline void ep_set_busy_poll_napi_id(struct epitem *epi) { - struct eventpoll *ep; + struct eventpoll *ep = epi->ep; unsigned int napi_id; struct socket *sock; struct sock *sk; - if (!net_busy_loop_on()) + if (!ep_busy_loop_on(ep)) return; sock = sock_from_file(epi->ffd.file); @@ -442,7 +474,6 @@ static inline void ep_set_busy_poll_napi_id(struct epitem *epi) return; napi_id = READ_ONCE(sk->sk_napi_id); - ep = epi->ep; /* Non-NAPI IDs can be rejected * or @@ -2058,6 +2089,9 @@ static int do_epoll_create(int flags) error = PTR_ERR(file); goto out_free_fd; } +#ifdef CONFIG_NET_RX_BUSY_POLL + ep->busy_poll_usecs = 0; +#endif ep->file = file; fd_install(fd, file); return fd; From patchwork Tue Feb 13 06:16:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Damato X-Patchwork-Id: 200237 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp354171dyb; Mon, 12 Feb 2024 22:18:35 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCV2bZGbwygKLV/WWDVcO8OZYz0Ww7aQKwKH/Xc/XGdnnwkvli2rex8E98iVH9lIIZE3ffwvn/M2dqZw2JFMYBvx0znrVA== X-Google-Smtp-Source: AGHT+IGSX8WI4t0Pc9wnpA1tqntDYTgdfiAODumTBq9Qc+hbxEZdHQUHPghnmmWyW7VTRHf3+i8R X-Received: by 2002:a05:6870:55c7:b0:21a:c53:2688 with SMTP id qk7-20020a05687055c700b0021a0c532688mr8226951oac.16.1707805115277; Mon, 12 Feb 2024 22:18:35 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707805115; cv=pass; d=google.com; s=arc-20160816; b=JxnaVblbfO8i2VHUST+u50XhW99ziIlxV14ZyiopLJtcvcYwE2goEsb9cb/QmblXMj AZGk6ZTnu8SAFoll6qH7hLKrg6bkFyjywpE2oWI+WCeFI+k5VPgq3kS+ka302QiqwaD0 +7pgjZF1b94XvxFkNDtmG/KJALCnQPWeKqoAIuzRDCydc39rPjGT7/XRqtYrTpTS62qP BHflYQ5avLf0i27X1NlaOD1KNJpVcOx4b2VgKANxFBDHse8Ykx9yue0D80gzfeQjm0g3 38iAYSL/uwL2VOn3qlbCbGNvkSW/Ia5NjBoOkPERozMOcv+ihklPNA9lUyonMLv+D3O9 uMxQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=JFThVNztewuogOwemulUSfgykwCuDZCM+kDTpLgPy0E=; fh=BynNj6jONmgXhbnJ5iqrs/6CmyS+iJl+wQUu8w/LQZw=; b=xj6p4YIWZ9efgGOwMqEEqPc9iyNMEuKsC1zIB5kd15R16fq7lAOLvFevfPTw5EK/p8 mehBslfxKSH/BPPfVCHqG3tF4c1Rc+YP1m7OLGmHccYqMjUOCknH2FSe9D0WQRl7uYPn 6tfgXlgwY9U3qL+8Q63/HCbvz7xUO7Mk/epWxYEtUDGlLD2KNmyLP08RDvGxr/PR3m72 40LrdDHv/jBKz93AAqDAxr+s4NN9xId3QLjuFGBcYFOm8iyvPLwwOp64BO5/ZwxLs5el S6KZzSLRXmQXXFCXPbqL6vF10luteuUlMN0x5sRGYZvJlSZAGhUYN2z1bXUY+JTXFfK8 Atgw==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@fastly.com header.s=google header.b=L6sEy3AH; arc=pass (i=1 spf=pass spfdomain=fastly.com dkim=pass dkdomain=fastly.com dmarc=pass fromdomain=fastly.com); spf=pass (google.com: domain of linux-kernel+bounces-63000-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63000-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=fastly.com X-Forwarded-Encrypted: i=2; AJvYcCULPoy67YpPiylVtlUHPd0oiKCEW+bck2VUcxvv7rvQbnQ52GcDrH6XwBsJXgHzOPXk1jgmUu3Ej5IdKoQ+LMA+Yj350g== Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id ca27-20020a056a02069b00b005dc854a1d02si648793pgb.230.2024.02.12.22.18.35 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Feb 2024 22:18:35 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-63000-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@fastly.com header.s=google header.b=L6sEy3AH; arc=pass (i=1 spf=pass spfdomain=fastly.com dkim=pass dkdomain=fastly.com dmarc=pass fromdomain=fastly.com); spf=pass (google.com: domain of linux-kernel+bounces-63000-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63000-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=fastly.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 07DB928657A for ; Tue, 13 Feb 2024 06:18:35 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 888701AAD0; Tue, 13 Feb 2024 06:17:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b="L6sEy3AH" Received: from mail-ot1-f52.google.com (mail-ot1-f52.google.com [209.85.210.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 01E9F17558 for ; Tue, 13 Feb 2024 06:17:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707805025; cv=none; b=gPuZkf5AjLwlGTBuaaguJM4UExqqp92CvmGD417QJzOBGX0jDF5T99xh3pwSA5b3mqrCtMW4OpoBf4/LROI/KwvAg86Pg9Z0Oy8Acl/OPj/7lewfT/YFi/MyZx6IrcOyjDgi5iWaybsWme8v0wMkzZpHiQOZ5N37ABLUoiqKSSQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707805025; c=relaxed/simple; bh=L4RwZXaFjOs7euXjMxwYiABkrXXZbZcKloyh3rbhzu8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=ZAB2VSrScpQOycKXV0iiLxH/Ecv6pgaKJtrC3X0yyNHBMprQ39ED/YNAYhuUxf786pNmn7HaVJkERBP8AUPZzS6cg6s/hrZrNYwy9ZnRHRr9wXbM6mBiXC7mWIMV+se2M3sCUbhGU0T2diW6LOHXpLc+v9cilYp34CZqj7vo4sE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com; spf=pass smtp.mailfrom=fastly.com; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b=L6sEy3AH; arc=none smtp.client-ip=209.85.210.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=fastly.com Received: by mail-ot1-f52.google.com with SMTP id 46e09a7af769-6dc8b280155so2801775a34.0 for ; Mon, 12 Feb 2024 22:17:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; t=1707805022; x=1708409822; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=JFThVNztewuogOwemulUSfgykwCuDZCM+kDTpLgPy0E=; b=L6sEy3AHB/9S1ETrXIdWeuDJTO+JItL1T8o3SFnZ4BKP4YiYwBRO37T/weSaBgPYhm YnVlQzuSsC9TTLPx1+OXMEASFzakzgHP6dAvRNAF426cdxfM5n8FUPQQ8PM94Qehbi8G KOZP76hVUY5LhIh5ExSs1xxMynX9QY3xhrz3s= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707805022; x=1708409822; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JFThVNztewuogOwemulUSfgykwCuDZCM+kDTpLgPy0E=; b=TizPtiskfKFrdQjVD5/V2oZZHl/Yywj8CN7ES/3S42d8cWpGyVFseLcBW8+VXJyzUt mHVR+toJbF9XQ9Fc3A26Fp2qO/qQWiaAU2AtN0fcVfMUcepb05zlQe4zu3DEalgZrRvX W26rl3WenEOpZlJNC35tGeIwPOParBR/MjbdfWQ+RlYLbC5V26vcicr0oy5sJDdKFtJ0 QiCaedeZmziHa0+qs9BaXtDHvGrCs5VL6tR5RC4qBLyt1G8WX7UTgrNygV1lEOCfZrDc IjUJLy4areApEkC1LEQ/upUCmi9AyxoJryjBTPDYPIrhvQ5L3ZTdEmpP8m/BLB7NWCaf DHMg== X-Gm-Message-State: AOJu0YzUyFwqhIbiu/ghVo+IU9A5yxyAF1pn2bIk+Kw/0qiAZNfmcdMy DLrCdyRXwqTS7BOQJyUnKh3ww/Yhg8IYpo+i4jB4ftFyPM0sEzkJ2PI74M13UeSpZK668x1FsBK Ane5Mj3E5ARwCwHDf/XBGLhX64Z/uH+0PcvxYC2ZgtKTwLr4Z9DLW7dLBNhBvraON7TOO84tkst h8xzUiAaIdklGcHo2fSoCIwGWL/wlzsBru/hz5EFwXlnE= X-Received: by 2002:a05:6830:1d54:b0:6e2:f1ce:4e88 with SMTP id p20-20020a0568301d5400b006e2f1ce4e88mr1965111oth.19.1707805022376; Mon, 12 Feb 2024 22:17:02 -0800 (PST) X-Forwarded-Encrypted: i=1; AJvYcCXYudH5/eqFbxhZ1hee68duq43gdfcl3tLV139OGz9rGgNuBXOhMoikoVe06ZLfc3dFF+s43Qx0hjnPO8pucyDy87wQCcQlRhWCz35HxPO/j+RWygD4yYXtpaPx4Ry+ENGYPR3CPJ4nXwnyppylY5YGFUCGsBx8jsV5vaRiLDcqmBexOI1QOv721rZxZ1DMDrvLnBfADBmQj8y7AWxcrPT2BAae2UZm9MgT57OUzg08Bpoig/7yjqVAr67cJoYvM0pkvmWBVPjtDRbZEVldDpc4wRAFKhdoAW69KcXlUxqHwKz28aeP1brgHx7bnggm8dbV7p2ou6y4ICaN9B1qZY3T+Oz/Ub1xuOWXj/ak+bIiTxLZj2uhqQ/z//l7/NBca1XZWp7adLGz9YFsacPO74y+4CpnIv3Q9NsYKc9aFuLMp6KRmdsqs1XcM4hAGW4PxL2FTf5u/TVpQQzenla9TIGqnms0q7AxBAGtEKfNJsH75WhhrROjOUIbthRqXHxZbRgh582zDFOEpAHrUQdjV0kfY+wh2sttc/xibp/6p4EVMsRPL2mfCAl2KBDc6EXfLntG06U6ivsUTG2u6Ao9qEJ4eIGSb4FmEe+tTx7EAp1ukNNH0JWWgbqgyb5ak7s= Received: from localhost.localdomain ([2620:11a:c018:0:ea8:be91:8d1:f59b]) by smtp.gmail.com with ESMTPSA id n19-20020a638f13000000b005dc87f5dfcfsm342936pgd.78.2024.02.12.22.17.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Feb 2024 22:17:01 -0800 (PST) From: Joe Damato To: linux-kernel@vger.kernel.org, netdev@vger.kernel.org Cc: chuck.lever@oracle.com, jlayton@kernel.org, linux-api@vger.kernel.org, brauner@kernel.org, edumazet@google.com, davem@davemloft.net, alexander.duyck@gmail.com, sridhar.samudrala@intel.com, kuba@kernel.org, willemdebruijn.kernel@gmail.com, weiwan@google.com, David.Laight@ACULAB.COM, arnd@arndb.de, sdf@google.com, amritha.nambiar@intel.com, Joe Damato , Alexander Viro , Jan Kara , linux-fsdevel@vger.kernel.org (open list:FILESYSTEMS (VFS and infrastructure)) Subject: [PATCH net-next v8 2/4] eventpoll: Add per-epoll busy poll packet budget Date: Tue, 13 Feb 2024 06:16:43 +0000 Message-Id: <20240213061652.6342-3-jdamato@fastly.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240213061652.6342-1-jdamato@fastly.com> References: <20240213061652.6342-1-jdamato@fastly.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790763456161815693 X-GMAIL-MSGID: 1790763456161815693 When using epoll-based busy poll, the packet budget is hardcoded to BUSY_POLL_BUDGET (8). Users may desire larger busy poll budgets, which can potentially increase throughput when busy polling under high network load. Other busy poll methods allow setting the busy poll budget via SO_BUSY_POLL_BUDGET, but epoll-based busy polling uses a hardcoded value. Fix this edge case by adding support for a per-epoll context busy poll packet budget. If not specified, the default value (BUSY_POLL_BUDGET) is used. Signed-off-by: Joe Damato Acked-by: Stanislav Fomichev Reviewed-by: Jakub Kicinski Reviewed-by: Eric Dumazet --- fs/eventpoll.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/fs/eventpoll.c b/fs/eventpoll.c index 401f865eced9..ed83ae33dd45 100644 --- a/fs/eventpoll.c +++ b/fs/eventpoll.c @@ -229,6 +229,8 @@ struct eventpoll { unsigned int napi_id; /* busy poll timeout */ u32 busy_poll_usecs; + /* busy poll packet budget */ + u16 busy_poll_budget; #endif #ifdef CONFIG_DEBUG_LOCK_ALLOC @@ -435,10 +437,14 @@ static bool ep_busy_loop_end(void *p, unsigned long start_time) static bool ep_busy_loop(struct eventpoll *ep, int nonblock) { unsigned int napi_id = READ_ONCE(ep->napi_id); + u16 budget = READ_ONCE(ep->busy_poll_budget); + + if (!budget) + budget = BUSY_POLL_BUDGET; if (napi_id >= MIN_NAPI_ID && ep_busy_loop_on(ep)) { napi_busy_loop(napi_id, nonblock ? NULL : ep_busy_loop_end, ep, false, - BUSY_POLL_BUDGET); + budget); if (ep_events_available(ep)) return true; /* @@ -2091,6 +2097,7 @@ static int do_epoll_create(int flags) } #ifdef CONFIG_NET_RX_BUSY_POLL ep->busy_poll_usecs = 0; + ep->busy_poll_budget = 0; #endif ep->file = file; fd_install(fd, file); From patchwork Tue Feb 13 06:16:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Damato X-Patchwork-Id: 200238 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp354237dyb; Mon, 12 Feb 2024 22:18:50 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCUWEebbY0B+W2K25GRebav0hY9w9fw4qFmTiPecCz7x8tV7JKtQTuF3l/mk8UimwMq6HBIbGf3wKxKGshn9f+uikxrtVA== X-Google-Smtp-Source: AGHT+IENzcRD602+cJNXwtLFhk3tiVpb9RLv4+By91lhr2yAe99xJ5H0O77F5K+pLksG4zA7C6rc X-Received: by 2002:a05:6e02:1a48:b0:363:d88e:e111 with SMTP id u8-20020a056e021a4800b00363d88ee111mr12977605ilv.30.1707805128306; Mon, 12 Feb 2024 22:18:48 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707805128; cv=pass; d=google.com; s=arc-20160816; b=eI+zR7oYhToo/4PtGFlCaCkYZtVBRGajwifPG164TVa2qDSYvVoUEjvt/GB2VPwbQA R1Y3tl13WYWuW4l/OI+EdDCFnvhqeXtHv/KnLB9Wuv2FHAf7TjKooilftxNlQOKyl2je VVB8rmOFzFfOMYpsUb9Oh7pHIpi8u4dzffotFvg3vlhf9Mf30yArQNSbW38UtEyRs42B 5F6NidKdB60tu/I0yybngWDMrLMpJOwg6sczYb8VToam+a7f3GuEHdweZPYiC/lsvSGR VqgRoXX8Ti/oRDW5AhdmqxAWj1nKCzWlWu5Z+dZjG13fdBvBu9aI5Nx98I0M6h6/PCkP IdXA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=G0NsVP2+DJYJlYo79AHfvXsFPLMqC3dLNa8zH3nE+As=; fh=S7oiRaKNSlSKBHiXq/7Gn+m77L2B7LItH0ka87YguBE=; b=VDJ0HP4GbU5Jhb14+aP70pJdd0RN5Y7zxh8UD+S41eA4JSkZBEuFz4jABs6ZVVEI0z DUNfB48w18b4iJPh/UJTMrML+Pwc9e36MRFDeBSyuVRsoFK/4uh5W8z/VAQUWlNtEM7o 2jaWYJCgfydBmhrNvUERd7UzON4i23UBaIAZJ2MfJ3kAb5aKv0bNQ83Zk3wcsVxS2roi iHuYDi8leboeR9caEV2lgVKMIIJmgXoB1Kz/m21DH55kWi63aOL7FdPTr0BwceqD2MX3 3fvo7UA9w5/Ij/7cWVpRzfh66w4yZUQPTUGnq7A1YldIOYdL2cRcN6ew/zFhGiQffCUm tDrw==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@fastly.com header.s=google header.b="WD72+/YR"; arc=pass (i=1 spf=pass spfdomain=fastly.com dkim=pass dkdomain=fastly.com dmarc=pass fromdomain=fastly.com); spf=pass (google.com: domain of linux-kernel+bounces-63001-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63001-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=fastly.com X-Forwarded-Encrypted: i=2; AJvYcCUe1VMNcUFZTFcVh8MizSvgIzUuNqe0PdVepIzO8QBc9QPaV0p3XZpf8lF2CjXPNJJXXSFznH2i0Ey4V9lY8LdPgykzGA== Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id f15-20020a63754f000000b005d7afbb31b3si1446930pgn.352.2024.02.12.22.18.48 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Feb 2024 22:18:48 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-63001-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; dkim=pass header.i=@fastly.com header.s=google header.b="WD72+/YR"; arc=pass (i=1 spf=pass spfdomain=fastly.com dkim=pass dkdomain=fastly.com dmarc=pass fromdomain=fastly.com); spf=pass (google.com: domain of linux-kernel+bounces-63001-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63001-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=fastly.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 140E728651B for ; Tue, 13 Feb 2024 06:18:48 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D1E2E1B7E8; Tue, 13 Feb 2024 06:17:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b="WD72+/YR" Received: from mail-oa1-f45.google.com (mail-oa1-f45.google.com [209.85.160.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 84DB017998 for ; Tue, 13 Feb 2024 06:17:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.160.45 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707805028; cv=none; b=um/ZGl+k3FQYyIncL/v44Ovh8u+Bm4yRDeArsjzlmGMkQadsNEifQl/SoGYHEtxz0a9XxHkz10ktxywS7B1SM3+fXAdM+FNlPASMQVyuPA7QAphpjlwnZwDRIQ1IhYJ/71ZrJabTQoYD2SKrTUOUf6EyGqNQde2SpMqxnPSPeoQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707805028; c=relaxed/simple; bh=CJVUUyQixO3GY5CVmRJIGygCqjREOllOxrIr3Eh94fc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=pUYTLnorv+2Q0P4YGk2d3hQW4IV1RxBL7rFIN5yPI/E1/PZoR91hxkT2TGrqhFbvs7hKrdQwTBLqr12Y7oP+OmFkUQcHOZShdTKkbTQD6Dm+sOAJ/gYGFnq1fQNYrlwcbXtoVLqGi4spxphY3ayXSAmduFXK4TZCqTzJbfH9Png= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com; spf=pass smtp.mailfrom=fastly.com; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b=WD72+/YR; arc=none smtp.client-ip=209.85.160.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=fastly.com Received: by mail-oa1-f45.google.com with SMTP id 586e51a60fabf-21a95258d42so126222fac.2 for ; Mon, 12 Feb 2024 22:17:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; t=1707805025; x=1708409825; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=G0NsVP2+DJYJlYo79AHfvXsFPLMqC3dLNa8zH3nE+As=; b=WD72+/YRyziBOOKYnxUAU4ncq/8VkvOZ2pbm0G8ifaJoOqgu2rhnZXjlbRw+CS5zyO 4PzlYVYsvdsmDAqH299dqfe4tnhbdlnFADjLcf07aPzClb40/NoPk6GpkOG+Osnn1811 z6d6PPASRtdJ1+jGV3QY7cvjNDhDgIrjaA09g= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707805025; x=1708409825; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=G0NsVP2+DJYJlYo79AHfvXsFPLMqC3dLNa8zH3nE+As=; b=wRK13gWEumUpSW2GkI5tuhXDFGbzQ5jhnEyJ7XgYco2sDqo6nn6d38pJBRobr3XbSk C3rvCzwwn8olwmxLoIkbGduJAaanujlxYfu1Zi47Crw6Me2Y2ylRrfdxJCXTE/ct16UW UmsjM/WKqWmW8v+G+lh8I9FEB0vVPv6veNz+QRyT4qnus+WRY/VEhwtcNnMNyT8YC+lL OMucHxrwkWIh8HA6a9ovlxm4Grqjj+YYPM/nRXIt5fVHLu1M+iZG22SVuDCgxQ3argUF W5as5btGbaqEqfQxbukTf/LqDyyNIu+QiN9GrbsaaFJWGITK/zY3dVhaCvhz6S0mrSrW extg== X-Gm-Message-State: AOJu0Yz4h80TXru/qaAqrIvVjWWvmC3WJKGHNtAld0MqzWeEPfMh4qT2 +W7zXWMSThOOVeFx++dkLsb6l8p2J12SyYmH3dNsbbeQnTELaXx+tbSOaZfU6fmdQAQfRDmrUMJ ts67SvLtlbLD+oejR0g9PJB1ch5vel6/g0rC9cn08blrM7RMqpobwFzVsbrhQYkje8IWURQsA2M 2EBeTAkKyimAx0d/8Ok3sOPMOzYMhhlfKLj+XCv3HU5PY= X-Received: by 2002:a05:6870:a70c:b0:21a:8e6c:26ea with SMTP id g12-20020a056870a70c00b0021a8e6c26eamr1009371oam.8.1707805024793; Mon, 12 Feb 2024 22:17:04 -0800 (PST) X-Forwarded-Encrypted: i=1; AJvYcCXkJlRyC/W6VSYAmeZ1aCoK74M/3UKA0+2neN2NqfNbLyNKu4MHpMMUr10S7ecIiGbjnW456R3jcLu6FwQBUI3Gb4QMYHa4mPbrF3curridEb+72102soyx7XqYNSu4FaBe4jAAB6nNo2BnRB8IjCjJiq2hRGMJ2yTa++suyZct7Iw1/9PjcL9nxbH9ZaeQwYIM+neBHpzAtj/YjQyo1jfytfYSEEqIc5PsojR4d05HO0PiY0uK3G5YuNz5H9RDhpyDFLbnCCpAmn5XrEhylbuPOBj1hmg7VLfQHFjO68YRZJtEjFk7DBGoGRabx+YFy7C4Of1xi+E6EX6wKq0VOCdPwGKGCqGAfBjcCCuZ+Uyj8hyktlF4cZmW3dKpYXQbYoUIwBu9nyrdujVcAVmcOxVRlE3oLNSIdRNKt2YeZhADYmmAbE2NsRvzmNjuljG3DDBIoM9Xnz4qkc4EIFCcSJThKY6zo3L29b0QUfBlYrKhvbkhQNSEROhXJUjiBiDIxIzVCoRfecHuI0PepA9BoDfvVatxwCh4VL+sI8ygu3NtdrKF+92STBO+F4xDNb8ZmI5mqkPp3jbL2TKPK31S/umYDVy89zuENeM6b8jsLXMfJYn+ffmBEfF0xANMUm8= Received: from localhost.localdomain ([2620:11a:c018:0:ea8:be91:8d1:f59b]) by smtp.gmail.com with ESMTPSA id n19-20020a638f13000000b005dc87f5dfcfsm342936pgd.78.2024.02.12.22.17.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Feb 2024 22:17:03 -0800 (PST) From: Joe Damato To: linux-kernel@vger.kernel.org, netdev@vger.kernel.org Cc: chuck.lever@oracle.com, jlayton@kernel.org, linux-api@vger.kernel.org, brauner@kernel.org, edumazet@google.com, davem@davemloft.net, alexander.duyck@gmail.com, sridhar.samudrala@intel.com, kuba@kernel.org, willemdebruijn.kernel@gmail.com, weiwan@google.com, David.Laight@ACULAB.COM, arnd@arndb.de, sdf@google.com, amritha.nambiar@intel.com, Joe Damato , Alexander Viro , Jan Kara , linux-fsdevel@vger.kernel.org (open list:FILESYSTEMS (VFS and infrastructure)) Subject: [PATCH net-next v8 3/4] eventpoll: Add per-epoll prefer busy poll option Date: Tue, 13 Feb 2024 06:16:44 +0000 Message-Id: <20240213061652.6342-4-jdamato@fastly.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240213061652.6342-1-jdamato@fastly.com> References: <20240213061652.6342-1-jdamato@fastly.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790763470539655869 X-GMAIL-MSGID: 1790763470539655869 When using epoll-based busy poll, the prefer_busy_poll option is hardcoded to false. Users may want to enable prefer_busy_poll to be used in conjunction with gro_flush_timeout and defer_hard_irqs_count to keep device IRQs masked. Other busy poll methods allow enabling or disabling prefer busy poll via SO_PREFER_BUSY_POLL, but epoll-based busy polling uses a hardcoded value. Fix this edge case by adding support for a per-epoll context prefer_busy_poll option. The default is false, as it was hardcoded before this change. Signed-off-by: Joe Damato Acked-by: Stanislav Fomichev Reviewed-by: Jakub Kicinski Reviewed-by: Eric Dumazet --- fs/eventpoll.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/fs/eventpoll.c b/fs/eventpoll.c index ed83ae33dd45..1b8d01af0c2c 100644 --- a/fs/eventpoll.c +++ b/fs/eventpoll.c @@ -231,6 +231,7 @@ struct eventpoll { u32 busy_poll_usecs; /* busy poll packet budget */ u16 busy_poll_budget; + bool prefer_busy_poll; #endif #ifdef CONFIG_DEBUG_LOCK_ALLOC @@ -438,13 +439,14 @@ static bool ep_busy_loop(struct eventpoll *ep, int nonblock) { unsigned int napi_id = READ_ONCE(ep->napi_id); u16 budget = READ_ONCE(ep->busy_poll_budget); + bool prefer_busy_poll = READ_ONCE(ep->prefer_busy_poll); if (!budget) budget = BUSY_POLL_BUDGET; if (napi_id >= MIN_NAPI_ID && ep_busy_loop_on(ep)) { - napi_busy_loop(napi_id, nonblock ? NULL : ep_busy_loop_end, ep, false, - budget); + napi_busy_loop(napi_id, nonblock ? NULL : ep_busy_loop_end, + ep, prefer_busy_poll, budget); if (ep_events_available(ep)) return true; /* @@ -2098,6 +2100,7 @@ static int do_epoll_create(int flags) #ifdef CONFIG_NET_RX_BUSY_POLL ep->busy_poll_usecs = 0; ep->busy_poll_budget = 0; + ep->prefer_busy_poll = false; #endif ep->file = file; fd_install(fd, file); From patchwork Tue Feb 13 06:16:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Damato X-Patchwork-Id: 200239 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp354349dyb; Mon, 12 Feb 2024 22:19:14 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWfuo/bpWHpIMW82YzgxnN4c6RY0CBjrVuYzOE/52P1upczh4VkYZxVDR0d2BOEOT8BgKhQnJ1OO2cYMH6SwGE3Xrv6Rw== X-Google-Smtp-Source: AGHT+IH/bA9/Eu7qUonhvvFEqjWXseOdDBgGKonOOyMhEzlYx5+6kakv1MbjYvYChTX0ZBpxo5ZG X-Received: by 2002:a05:620a:1212:b0:785:d7d1:1d8 with SMTP id u18-20020a05620a121200b00785d7d101d8mr4635361qkj.32.1707805154413; Mon, 12 Feb 2024 22:19:14 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707805154; cv=pass; d=google.com; s=arc-20160816; b=s9al3utk8DjdvBj0T+Jjwuuu1RU6bjSGtoec1v0oTSa3Rgzj+PZitcirOg2d5oDhDj PjATtT0sIupQmEdgT9Hk4EDfJayKsEXyIthScNGULoWRzrXeCT6968LWAZKD7UgEV5a9 3WbCXwYVdJ/onoKYkWzDZWWaL5zUD6aC0Eht1r8h2EhglhZ2Ede1Lh2qohQOpSGJg+5L 4WY+Faydvte1OnfT/FYeUUSN7cl2IaSPZ2TgA88ZKPvd39DDEwdHckdePLM/TAVyX5Lz ctWxdDOKHNmhTjBreBrlR15SGgMG7xsGV/WIkQzBIT+KGnCIHLbHvo8P7ZXvZnEyol3t kFzw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=NXoP2M02Y81OIisvdpEuqLXXw7KopR3yk8yAxza3E4Y=; fh=7UiSEtC8bFGBs5dllMvO6Y7bgh3Y+YNtOQJsH5Evc6k=; b=zhs6SobjLO1DxzRJEVrMhT21tqUS9WUa4qvdYtWN2zsb558844MOP6aTHBovGx0QnU cCLN9fney9VImv0fZnr9ChlQIlXH1SX55x2G3/R0FRGU7GmQMyLWczOPE1ykBxX0YnjE +8QpuLNRUnDjlNDtXVsIKcQ8YOP0ypQdsLCliJm9yWVH2drcGVnEMG56KIGUu0za10O+ SQvkeqQYCnObqJfIljfO/0Mk4ezIcSTVEL9kEf+e/WwX53i54WPxItgGu+neP87JIY2z 5gMo2qTPrcK1bJmsSX5y5cHY+c5gqS5INE7pelDddu6DzMNGTtEsvaASUUx7cGoOBRH1 0djQ==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@fastly.com header.s=google header.b=LIVlLi8u; arc=pass (i=1 spf=pass spfdomain=fastly.com dkim=pass dkdomain=fastly.com dmarc=pass fromdomain=fastly.com); spf=pass (google.com: domain of linux-kernel+bounces-63002-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63002-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=fastly.com X-Forwarded-Encrypted: i=2; AJvYcCXdTl57tWYbJvxsIvM7CCbYvrI/mp0pEUjBWMiOsTx2CtdV39DACVePQYIffX98U+fK40qn1AnirgSJTHfMRmwXpqHkuw== Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id e29-20020a05620a015d00b007871632c943si1942837qkn.57.2024.02.12.22.19.14 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Feb 2024 22:19:14 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-63002-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@fastly.com header.s=google header.b=LIVlLi8u; arc=pass (i=1 spf=pass spfdomain=fastly.com dkim=pass dkdomain=fastly.com dmarc=pass fromdomain=fastly.com); spf=pass (google.com: domain of linux-kernel+bounces-63002-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63002-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=fastly.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 28C7B1C21B16 for ; Tue, 13 Feb 2024 06:19:14 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 3854021340; Tue, 13 Feb 2024 06:17:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b="LIVlLi8u" Received: from mail-oi1-f170.google.com (mail-oi1-f170.google.com [209.85.167.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 36B7F1C687 for ; Tue, 13 Feb 2024 06:17:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707805033; cv=none; b=q8LFBPuXVRYHO8go73SLRral+w9E4xafIs/5OSGnTCiARHP/SXNyTwnrc8PLlX5tulG96zN3DwgUY5yyNoupr9kwoWplLY2HH63W/dia5kYUZU3fgmr43IIxCZD3CoJblkyFpJBNuaiHUZXqk+ZrVaka32sksXcSnOc6edhFREM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707805033; c=relaxed/simple; bh=keBus4r+KHdtvzkIagi6GmmLRoV/Iu5G57t/XEMaSDM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=fldGVYSvEmwzi9wAGulsGNm1LRFMp4oDgRosr/jhzHWvuOfs+RuAqnR6C+smj23cBzFMXRuJeVFwi080c1/kZxbd0M8QEp9LLcu7O+6w/AAtKKhd1vYxqQ9tLjMJm6ZqUbn0TGXErH7qXy3yQACFfmSAZKcPRiWXwrqBPYZABjw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com; spf=pass smtp.mailfrom=fastly.com; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b=LIVlLi8u; arc=none smtp.client-ip=209.85.167.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=fastly.com Received: by mail-oi1-f170.google.com with SMTP id 5614622812f47-3c031776e0cso2086148b6e.0 for ; Mon, 12 Feb 2024 22:17:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; t=1707805030; x=1708409830; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=NXoP2M02Y81OIisvdpEuqLXXw7KopR3yk8yAxza3E4Y=; b=LIVlLi8ujCIQNXEsW+nC9BqZydkEBMBSQ4dPeetnmCi4VHlXnjHCsjdkh+dtNm1Zv4 tH2ABtf/oZnqVric6WgGeJTflMsDNbeoiSC/hEf6zAuUxkwTSkMFqsf6UX1X9XP18fEB S41qXSOPxF4+X4xmR0RwHdGr9Y9oS6O9UKVRw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707805030; x=1708409830; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NXoP2M02Y81OIisvdpEuqLXXw7KopR3yk8yAxza3E4Y=; b=iIwnAiyL2cCKheaorUPO12qUeAWLjLeRxSyxa1tUXDTuQsa72GCnF8FYGuzfOYirsQ kJKDT/XS6K7yI/GHZxagMB6grIFGpbcbkix4tdhe4KLbN3yM1zj/mQ6X1bQFIOlao8Ye ZAPzV/RgwLnbMrcwvqZh+fDGfOReUBqF2VtsfDYqdwfPxmtQ5/Fc93tpNSxYY3vU8bSD zcXADsr64G/31yuqdUihCYuCvz2wIiDDh5ClOHieYnqhzWWREICssqe4BxxTzhrqXXmj 7jVaZ0eNKCzylIv5O3YXWffap0Lj/XiBbTlZGW8B2RA1iupT5AvKvKMCavIeTHYVnDo2 6ZsQ== X-Gm-Message-State: AOJu0Yw1LV5eFwQOZX1ieMwwcv7Ua984db+bHC316ZAduIbnKRlKjmd+ AIzeJjzCbydXPjt3+ygnOFDuPlboF65SMjuxcjGrSRNbAq7ALjcTRj0i/rXNlfNCaiphWUlbTT6 ytQTNVzFHLqbEsg+qFWolAylwia6F2vj2v6gmSZllydpvASQsK5+yP5s7z+ObeAxkmTsChizTh4 UBvrA39EFWr6+o4ehDpoat+/Y2vNv5vW2/5Y0c7xztHnuWjA== X-Received: by 2002:a05:6870:1681:b0:218:d8a0:69e2 with SMTP id j1-20020a056870168100b00218d8a069e2mr11392706oae.7.1707805030497; Mon, 12 Feb 2024 22:17:10 -0800 (PST) X-Forwarded-Encrypted: i=1; AJvYcCVzRDFeolDshuiT4chMcAX1qwSb4cjcuh+zAXZK3sWPgl+kSem+6ddHp0vng8xy2UBopU/rjhf1rVLKJMSidqIwajPleUxyxCKtTHsNgNrcG1pToS27rVvlKqgJqkCdLhxZvxvk3ThMRrwHeflrUnMyxgCIYW1xmDQQtVrRxOs9G4+b5eSfi1ZSpOfScm/NtOBbwxM54vN5gmG2LAQE5x5nkZwILfRVrp5v0PviODCADGfhQyQT1xtdtFBJwGkuEUi8I9Vt2viXCIpvHG7UPAvKCVDOpzyO5NuZHum6TSYLkjsiDWoj9eM/j9D2puhXpzC369Q8c+V9YE+Yu5k+jiOQLFo/xAqK2gDjM3D6XngcBlPMGkTRvaYKY9kLyZ0ca0ahtejbCDMmh6dihq5QFnrGQKbOIBLIAOEHfb5E7oMAuqhh17yM17r0+TOdUSeDi7bi1VQT4t97107D9EepePCdsQWQ/M2wSWDFsEakBkD9LpOs8X6aUfD9q5Z6XX2iAGHY07yABi/pSwAGwS6PggBhwXL9TIa0hxC2r7zw7PmJjz3X4Jg8jDU7p80dS44fiVB4sc9jF5ClkkJwBuLi7Te/iDQ3IUaKb/JalTRhtg+JvnRRoOj+7dJJ1CgEQbcHlGcdBfeSwAJzn3k3GQ/CWLbJTs7jHpWsAJIa2SkTaVvODllFsnSJ/J+LfDSUDjN2K4ubGLsrqpvkNYnqG2Npoe4sZBhU2mhdmoJ91ENqyw9V89o/Djvxk3c3zUAQUYPbU8k7B7X+Y4POAQrhvFlXIPlaskPMVRE+pxLICP6oSQOapxoMf1qWFB3gtUaUV82SGXXk4cWZKXv5n52EhvaHlZg4lbyqwMY2BVw6DHZQpP+lz6VMDGyF+tPOiJ7DhgRdHxPlPuF3WZ20LozSv7IqmRMz0UvRCuNmspCGqAk= Received: from localhost.localdomain ([2620:11a:c018:0:ea8:be91:8d1:f59b]) by smtp.gmail.com with ESMTPSA id n19-20020a638f13000000b005dc87f5dfcfsm342936pgd.78.2024.02.12.22.17.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Feb 2024 22:17:10 -0800 (PST) From: Joe Damato To: linux-kernel@vger.kernel.org, netdev@vger.kernel.org Cc: chuck.lever@oracle.com, jlayton@kernel.org, linux-api@vger.kernel.org, brauner@kernel.org, edumazet@google.com, davem@davemloft.net, alexander.duyck@gmail.com, sridhar.samudrala@intel.com, kuba@kernel.org, willemdebruijn.kernel@gmail.com, weiwan@google.com, David.Laight@ACULAB.COM, arnd@arndb.de, sdf@google.com, amritha.nambiar@intel.com, Joe Damato , Jiri Slaby , Jonathan Corbet , Alexander Viro , Jan Kara , Michael Ellerman , Greg Kroah-Hartman , Nathan Lynch , Thomas Zimmermann , Julien Panis , Steve French , Palmer Dabbelt , linux-doc@vger.kernel.org (open list:DOCUMENTATION), linux-fsdevel@vger.kernel.org (open list:FILESYSTEMS (VFS and infrastructure)) Subject: [PATCH net-next v8 4/4] eventpoll: Add epoll ioctl for epoll_params Date: Tue, 13 Feb 2024 06:16:45 +0000 Message-Id: <20240213061652.6342-5-jdamato@fastly.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240213061652.6342-1-jdamato@fastly.com> References: <20240213061652.6342-1-jdamato@fastly.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790763497771730435 X-GMAIL-MSGID: 1790763497771730435 Add an ioctl for getting and setting epoll_params. User programs can use this ioctl to get and set the busy poll usec time, packet budget, and prefer busy poll params for a specific epoll context. Parameters are limited: - busy_poll_usecs is limited to <= s32_max - busy_poll_budget is limited to <= NAPI_POLL_WEIGHT by unprivileged users (!capable(CAP_NET_ADMIN)) - prefer_busy_poll must be 0 or 1 - __pad must be 0 Signed-off-by: Joe Damato Acked-by: Stanislav Fomichev Reviewed-by: Jiri Slaby Reviewed-by: Eric Dumazet --- .../userspace-api/ioctl/ioctl-number.rst | 1 + fs/eventpoll.c | 73 +++++++++++++++++++ include/uapi/linux/eventpoll.h | 13 ++++ 3 files changed, 87 insertions(+) diff --git a/Documentation/userspace-api/ioctl/ioctl-number.rst b/Documentation/userspace-api/ioctl/ioctl-number.rst index 457e16f06e04..b33918232f78 100644 --- a/Documentation/userspace-api/ioctl/ioctl-number.rst +++ b/Documentation/userspace-api/ioctl/ioctl-number.rst @@ -309,6 +309,7 @@ Code Seq# Include File Comments 0x89 0B-DF linux/sockios.h 0x89 E0-EF linux/sockios.h SIOCPROTOPRIVATE range 0x89 F0-FF linux/sockios.h SIOCDEVPRIVATE range +0x8A 00-1F linux/eventpoll.h 0x8B all linux/wireless.h 0x8C 00-3F WiNRADiO driver diff --git a/fs/eventpoll.c b/fs/eventpoll.c index 1b8d01af0c2c..df2ed3af486e 100644 --- a/fs/eventpoll.c +++ b/fs/eventpoll.c @@ -37,6 +37,7 @@ #include #include #include +#include #include /* @@ -494,6 +495,49 @@ static inline void ep_set_busy_poll_napi_id(struct epitem *epi) ep->napi_id = napi_id; } +static long ep_eventpoll_bp_ioctl(struct file *file, unsigned int cmd, + unsigned long arg) +{ + struct eventpoll *ep = file->private_data; + void __user *uarg = (void __user *)arg; + struct epoll_params epoll_params; + + switch (cmd) { + case EPIOCSPARAMS: + if (copy_from_user(&epoll_params, uarg, sizeof(epoll_params))) + return -EFAULT; + + /* pad byte must be zero */ + if (epoll_params.__pad) + return -EINVAL; + + if (epoll_params.busy_poll_usecs > S32_MAX) + return -EINVAL; + + if (epoll_params.prefer_busy_poll > 1) + return -EINVAL; + + if (epoll_params.busy_poll_budget > NAPI_POLL_WEIGHT && + !capable(CAP_NET_ADMIN)) + return -EPERM; + + WRITE_ONCE(ep->busy_poll_usecs, epoll_params.busy_poll_usecs); + WRITE_ONCE(ep->busy_poll_budget, epoll_params.busy_poll_budget); + WRITE_ONCE(ep->prefer_busy_poll, epoll_params.prefer_busy_poll); + return 0; + case EPIOCGPARAMS: + memset(&epoll_params, 0, sizeof(epoll_params)); + epoll_params.busy_poll_usecs = READ_ONCE(ep->busy_poll_usecs); + epoll_params.busy_poll_budget = READ_ONCE(ep->busy_poll_budget); + epoll_params.prefer_busy_poll = READ_ONCE(ep->prefer_busy_poll); + if (copy_to_user(uarg, &epoll_params, sizeof(epoll_params))) + return -EFAULT; + return 0; + default: + return -ENOIOCTLCMD; + } +} + #else static inline bool ep_busy_loop(struct eventpoll *ep, int nonblock) @@ -505,6 +549,12 @@ static inline void ep_set_busy_poll_napi_id(struct epitem *epi) { } +static long ep_eventpoll_bp_ioctl(struct file *file, unsigned int cmd, + unsigned long arg) +{ + return -EOPNOTSUPP; +} + #endif /* CONFIG_NET_RX_BUSY_POLL */ /* @@ -864,6 +914,27 @@ static void ep_clear_and_put(struct eventpoll *ep) ep_free(ep); } +static long ep_eventpoll_ioctl(struct file *file, unsigned int cmd, + unsigned long arg) +{ + int ret; + + if (!is_file_epoll(file)) + return -EINVAL; + + switch (cmd) { + case EPIOCSPARAMS: + case EPIOCGPARAMS: + ret = ep_eventpoll_bp_ioctl(file, cmd, arg); + break; + default: + ret = -EINVAL; + break; + } + + return ret; +} + static int ep_eventpoll_release(struct inode *inode, struct file *file) { struct eventpoll *ep = file->private_data; @@ -970,6 +1041,8 @@ static const struct file_operations eventpoll_fops = { .release = ep_eventpoll_release, .poll = ep_eventpoll_poll, .llseek = noop_llseek, + .unlocked_ioctl = ep_eventpoll_ioctl, + .compat_ioctl = compat_ptr_ioctl, }; /* diff --git a/include/uapi/linux/eventpoll.h b/include/uapi/linux/eventpoll.h index cfbcc4cc49ac..4f4b948ef381 100644 --- a/include/uapi/linux/eventpoll.h +++ b/include/uapi/linux/eventpoll.h @@ -85,4 +85,17 @@ struct epoll_event { __u64 data; } EPOLL_PACKED; +struct epoll_params { + __u32 busy_poll_usecs; + __u16 busy_poll_budget; + __u8 prefer_busy_poll; + + /* pad the struct to a multiple of 64bits */ + __u8 __pad; +}; + +#define EPOLL_IOC_TYPE 0x8A +#define EPIOCSPARAMS _IOW(EPOLL_IOC_TYPE, 0x01, struct epoll_params) +#define EPIOCGPARAMS _IOR(EPOLL_IOC_TYPE, 0x02, struct epoll_params) + #endif /* _UAPI_LINUX_EVENTPOLL_H */