Message ID | ZdSAWAwUxc5R46NH@tpad |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel+bounces-72836-ouuuleilei=gmail.com@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:693c:2685:b0:108:e6aa:91d0 with SMTP id mn5csp310433dyc; Tue, 20 Feb 2024 02:36:04 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCUhf+PmRzytb4JrVouYhMHAYlqURwmDQJgAWsQZoBwsG2GYvElfFlP2+f0v1/ZH5V7L+QLhu6cqWHe44a7diYB+MoqHMw== X-Google-Smtp-Source: AGHT+IE0SSooH7Oul1IExZyvxELI7fv53wh1AY7FpMOrqeNWkov3w16j6OcGBaaOGs+e9R9eDE5f X-Received: by 2002:a17:902:d4d2:b0:1db:fc18:2da5 with SMTP id o18-20020a170902d4d200b001dbfc182da5mr6431200plg.30.1708425364615; Tue, 20 Feb 2024 02:36:04 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708425364; cv=pass; d=google.com; s=arc-20160816; b=nTFtC5Ev+/FV5CqpDbcGj+FkvMwQzU44COedVCekSLE7tZ/rcx2RXu+/jqw3cSPW0g gjtzhEoJTRineNG8aHhtJ4soDSVqC0y1KrKTesTbDGANRxOrzTVglqH2tsCgLt/HE/n3 krYsQeGWwEebiKcfS9wCZzROkMp22dFD2kDQdJodVa8ihrqLeppY4vLf5kk6ycpKbMyV z+1OSggGmueYPqW/Brv1eLPzl3l5LojfdQFzuQHnODLDR4KVUxg6X4o0QxI1nDi+HbiS 6ERPADnU7GGJVoRtDO7heSH1/ryJ/ym0aTIp1WzrouPPs1oBs5lqwAxM1N9dqFoAwM8i 3iCw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-disposition:mime-version:list-unsubscribe:list-subscribe :list-id:precedence:message-id:subject:cc:to:from:date :dkim-signature; bh=svGjrbyynWHu4RkuCMOXvlEyLxwGmnZthB0JJWKitFw=; fh=Aaqt4KOlqv9CVsm+BJ30ESOEQDu+OcH7DXfKb2uBDt0=; b=0GYuKoyjeftLxDBLVh6a7+hfhW+nCH2ZyHUH4ucmFI7yrO/bbjOIAvbxrPHTJoqsw2 07hh6oLYIJu5WM6pmvYqlxotq7FOyHLkGFoAQyVeUHE5SODda/A6jHKjPhqxrfJDGkka dXOW2+kI4qStk/tyrZALJywIUatEbCi/Ha/8DRzQ6iotvEq/u1o8xAm0EiiR3ek16SRu JLYL9ofNJXsEiW34WU6C6D/1CtQlh3zQFi8npRrFGs/kTwE61K3UPjhDmp0o7/ZGdysF 5a2zudhhbAsH7herk3sR05EFkUa5CYr74+phbnhR6BWfOupnL0I/UHJNVGeEuKpl1H4b K4hw==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=cXpCXeNw; arc=pass (i=1 spf=pass spfdomain=redhat.com dkim=pass dkdomain=redhat.com dmarc=pass fromdomain=redhat.com); spf=pass (google.com: domain of linux-kernel+bounces-72836-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-72836-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id kd16-20020a17090313d000b001d9b739a582si6198516plb.114.2024.02.20.02.36.04 for <ouuuleilei@gmail.com> (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Feb 2024 02:36:04 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-72836-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=cXpCXeNw; arc=pass (i=1 spf=pass spfdomain=redhat.com dkim=pass dkdomain=redhat.com dmarc=pass fromdomain=redhat.com); spf=pass (google.com: domain of linux-kernel+bounces-72836-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-72836-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 4E0B128205B for <ouuuleilei@gmail.com>; Tue, 20 Feb 2024 10:36:04 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id CDB876351D; Tue, 20 Feb 2024 10:35:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="cXpCXeNw" Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CF49760276 for <linux-kernel@vger.kernel.org>; Tue, 20 Feb 2024 10:35:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708425330; cv=none; b=FlEhoktSldFYdt9u9ssrsKpBIx6CTV9Cy3Eu/VC1me7JXWbncNnQ4udT56VGAKqlTr/Q4jkAAncSnoC6uCqc9Dcdf/1bzUuS2+GCBwz2LIPhFBZDNQ1NFXoFKHfbj9OLcY28MVN6iIsj16G8mlPCzLbRw+lVJPqaKjzOe3MFhoI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708425330; c=relaxed/simple; bh=HEvXUMThvUIqnt0LiI+bBVgaUe3iN2uphGAvjHQ1fog=; h=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type: Content-Disposition; b=munPU3d0RK0zPpP19DOC1EyHMmZ9AG2yO5CvytD6bJwHWZ/KiZ049U2FyeheEDyLfPpqkyt76/DsihSPm+osn7ZR+eFqM/FYyDWfUeB9Hfzcm+4yFOwk/qerG4i5CvucJHSv+ANH1g9V3G4hbG1AJuCvp98mGrwOSU9UuQvpBbY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=cXpCXeNw; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1708425327; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type; bh=svGjrbyynWHu4RkuCMOXvlEyLxwGmnZthB0JJWKitFw=; b=cXpCXeNwNBmT4/MyoIHdA7uxIZPw42T20mxRwjGlEy58KRStYfl9nEZHGUqu6Fu8KC0oOH gAsZDSq5XAHbuO6LKvhy/X6dzzapgBbmT4/aTx9EcsFMTmBy6aab392O91uCZkDoDXqIDK tppNSsiPPZKO8P3iupkR7ua/UEb1zA8= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-36-El1YA-VmOemMflwWPxauPA-1; Tue, 20 Feb 2024 05:35:25 -0500 X-MC-Unique: El1YA-VmOemMflwWPxauPA-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 0D93C106D020; Tue, 20 Feb 2024 10:35:25 +0000 (UTC) Received: from tpad.localdomain (unknown [10.96.133.2]) by smtp.corp.redhat.com (Postfix) with ESMTPS id D4979492BC6; Tue, 20 Feb 2024 10:35:24 +0000 (UTC) Received: by tpad.localdomain (Postfix, from userid 1000) id 3E04540194B36; Tue, 20 Feb 2024 07:35:04 -0300 (-03) Date: Tue, 20 Feb 2024 07:35:04 -0300 From: Marcelo Tosatti <mtosatti@redhat.com> To: netdev@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Jakub Kicinski <kuba@kernel.org>, Eric Dumazet <edumazet@google.com>, Frederic Weisbecker <frederic@kernel.org>, Valentin Schneider <vschneid@redhat.com> Subject: [PATCH] net/core/dev.c: enable timestamp static key if CPU isolation is configured Message-ID: <ZdSAWAwUxc5R46NH@tpad> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: <linux-kernel.vger.kernel.org> List-Subscribe: <mailto:linux-kernel+subscribe@vger.kernel.org> List-Unsubscribe: <mailto:linux-kernel+unsubscribe@vger.kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.9 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791413835220582064 X-GMAIL-MSGID: 1791413835220582064 |
Series |
net/core/dev.c: enable timestamp static key if CPU isolation is configured
|
|
Commit Message
Marcelo Tosatti
Feb. 20, 2024, 10:35 a.m. UTC
For systems that use CPU isolation (via nohz_full), creating or destroying
a socket with timestamping (SOF_TIMESTAMPING_OPT_TX_SWHW) might cause a
static key to be enabled/disabled. This in turn causes undesired
IPIs to isolated CPUs.
So enable the static key unconditionally, if CPU isolation is enabled,
thus avoiding the IPIs.
Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>
Comments
On Tue, 2024-02-20 at 07:35 -0300, Marcelo Tosatti wrote: > For systems that use CPU isolation (via nohz_full), creating or destroying > a socket with timestamping (SOF_TIMESTAMPING_OPT_TX_SWHW) might cause a > static key to be enabled/disabled. This in turn causes undesired > IPIs to isolated CPUs. > > So enable the static key unconditionally, if CPU isolation is enabled, > thus avoiding the IPIs. > > Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com> This looks like net-next material, but it does not apply cleanly to the net-next tree. Could you please rebase and repost including the 'net- next' tag into the patch prefix? Thanks! Paolo
Marcelo Tosatti wrote: > For systems that use CPU isolation (via nohz_full), creating or destroying > a socket with timestamping (SOF_TIMESTAMPING_OPT_TX_SWHW) might cause a > static key to be enabled/disabled. This in turn causes undesired > IPIs to isolated CPUs. This refers to SOF_TIMESTAMPING_RX_SOFTWARE, not SOF_TIMESTAMPING_OPT_TX_SWHW. See also sock_set_timestamping. > > So enable the static key unconditionally, if CPU isolation is enabled, > thus avoiding the IPIs. > > Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com> > > diff --git a/net/core/dev.c b/net/core/dev.c > index 0d548431f3fa..cc9a77b4aa4e 100644 > --- a/net/core/dev.c > +++ b/net/core/dev.c > @@ -153,6 +153,7 @@ > #include <linux/prandom.h> > #include <linux/once_lite.h> > #include <net/netdev_rx_queue.h> > +#include <linux/sched/isolation.h> > > #include "dev.h" > #include "net-sysfs.h" > @@ -11601,3 +11602,14 @@ static int __init net_dev_init(void) > } > > subsys_initcall(net_dev_init); > + > +static int __init net_dev_late_init(void) > +{ > + /* avoid static key IPIs to isolated CPUs */ > + if (housekeeping_enabled(HK_TYPE_MISC)) > + net_enable_timestamp(); > + > + return 0; > +} > + > +late_initcall(net_dev_late_init); >
On Thu, Feb 22, 2024 at 10:11:08AM -0500, Willem de Bruijn wrote: > Marcelo Tosatti wrote: > > For systems that use CPU isolation (via nohz_full), creating or destroying > > a socket with timestamping (SOF_TIMESTAMPING_OPT_TX_SWHW) might cause a > > static key to be enabled/disabled. This in turn causes undesired > > IPIs to isolated CPUs. > > This refers to SOF_TIMESTAMPING_RX_SOFTWARE, not SOF_TIMESTAMPING_OPT_TX_SWHW. > See also sock_set_timestamping. Willem, This test program does trigger the static key change: int main (void) { int option = SOF_TIMESTAMPING_OPT_TX_SWHW; int sock_fd; int ret; int pid_fd; pid_t pid; char buf[50]; .. /* set the timestamping option * this is to trigger the IPIs that notify all cpus of the change */ if (setsockopt(sock_fd, SOL_SOCKET, SO_TIMESTAMP, &option, sizeof (option)) < 0) { printf("Could not enable timestamping option %x", (unsigned int)option); close(sock_fd); return 0; } ..
Marcelo Tosatti wrote: > On Thu, Feb 22, 2024 at 10:11:08AM -0500, Willem de Bruijn wrote: > > Marcelo Tosatti wrote: > > > For systems that use CPU isolation (via nohz_full), creating or destroying > > > a socket with timestamping (SOF_TIMESTAMPING_OPT_TX_SWHW) might cause a > > > static key to be enabled/disabled. This in turn causes undesired > > > IPIs to isolated CPUs. > > > > This refers to SOF_TIMESTAMPING_RX_SOFTWARE, not SOF_TIMESTAMPING_OPT_TX_SWHW. > > See also sock_set_timestamping. > > Willem, > > This test program does trigger the static key change: > > int main (void) > { > int option = SOF_TIMESTAMPING_OPT_TX_SWHW; > int sock_fd; > int ret; > int pid_fd; > pid_t pid; > char buf[50]; > > ... > > /* set the timestamping option > * this is to trigger the IPIs that notify all cpus of the change > */ > if (setsockopt(sock_fd, SOL_SOCKET, SO_TIMESTAMP, &option, sizeof (option)) < 0) { > printf("Could not enable timestamping option %x", (unsigned int)option); > close(sock_fd); > return 0; > } > ... > That is because you call SO_TIMESTAMP, which interprets option as a boolean. The SOF_ flags apply to setsockopt SO_TIMESTAMPING.
diff --git a/net/core/dev.c b/net/core/dev.c index 0d548431f3fa..cc9a77b4aa4e 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -153,6 +153,7 @@ #include <linux/prandom.h> #include <linux/once_lite.h> #include <net/netdev_rx_queue.h> +#include <linux/sched/isolation.h> #include "dev.h" #include "net-sysfs.h" @@ -11601,3 +11602,14 @@ static int __init net_dev_init(void) } subsys_initcall(net_dev_init); + +static int __init net_dev_late_init(void) +{ + /* avoid static key IPIs to isolated CPUs */ + if (housekeeping_enabled(HK_TYPE_MISC)) + net_enable_timestamp(); + + return 0; +} + +late_initcall(net_dev_late_init);