From patchwork Sun Apr 30 13:15:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alvaro Karsz X-Patchwork-Id: 88913 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp2146666vqo; Sun, 30 Apr 2023 06:29:06 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4OxhbX/27Hfox69hSONWX5c0v6RUwd0I3m2Z2vtrCf8V017HRk9zAfxtjerO7Up6Oxmp3X X-Received: by 2002:a17:902:d48e:b0:1a9:aebc:d4ff with SMTP id c14-20020a170902d48e00b001a9aebcd4ffmr11718658plg.39.1682861346132; Sun, 30 Apr 2023 06:29:06 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1682861346; cv=pass; d=google.com; s=arc-20160816; b=hZqucyY+l+5vgsM4w8IcteI9b57CBgJktBpexQPWHoPvpK6k/JAKFPAGdn9bv/E9Va k4+1FgegELu8w46PN2ZbGmjnA0fYekZ8HofZUzVk8wF5Mwe67/Y8kNsJyGB38YsJn6fe 8O653zxq1vG/HT8c5y2kKyKNIuH2QYojOZD8Cd8FVj9MdyWlGABsnp4z1TpKNdygyq/L UkdkNRZyF4/g0K7YhPfepK2A0BVcQKyIle8HrrwuSLXXexF0N91EW/dVFHAv5XB9UoVO KrG2Q5lzQP2hi/rDK+Jz14fPFvEVpJFVfFTRI5IhabpIgdv6KOErqgV+vq5PW1iMmnDW dF7w== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:content-transfer-encoding :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=odkZ8HfHiza+jKPAfbqnFPQhDfN4CVkAQMeZK2QeXx4=; b=v2rf9tp40tejll0LeDKITmWAK4KOBDogHGL7Ofg06dZIJT6mwxfGUHCljTZa8FF0I8 +zzMF1vSTpsQdsx6ZOXjlSoDG9b56oATPUHn9FvDUsNVWmtHZPMlw+v2tYvZ52gYFqRA MOFdZHRMiD6QmQKRXKtD0pkx6LW5nJRvs1An6f0Bpx2EYEyv3iTfEnQg6s6oAtZd01qD AIUNVv31rMCRWXci4Go6AQpPXKvGrYpIKiFxKMbnoxWssFy5AguIzVal/cISzMsOSD3B xYkjv5gVpyVcOkD+Ct94YxifmjRrN3+zhswTBRVeMrO4/+9wnmSqqFkyDnKAlMEEgJAO 6ZJg== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@solidrn.onmicrosoft.com header.s=selector1-solidrn-onmicrosoft-com header.b=Sd0cycgm; arc=pass (i=1 spf=pass spfdomain=solid-run.com dkim=pass dkdomain=solid-run.com dmarc=pass fromdomain=solid-run.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id n2-20020a170902f60200b001a96124e47fsi21188561plg.348.2023.04.30.06.28.45; Sun, 30 Apr 2023 06:29:06 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@solidrn.onmicrosoft.com header.s=selector1-solidrn-onmicrosoft-com header.b=Sd0cycgm; arc=pass (i=1 spf=pass spfdomain=solid-run.com dkim=pass dkdomain=solid-run.com dmarc=pass fromdomain=solid-run.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230488AbjD3NPn (ORCPT + 99 others); Sun, 30 Apr 2023 09:15:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60532 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230336AbjD3NPe (ORCPT ); Sun, 30 Apr 2023 09:15:34 -0400 Received: from EUR02-AM0-obe.outbound.protection.outlook.com (mail-am0eur02on2066.outbound.protection.outlook.com [40.107.247.66]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DFB7E26A5; Sun, 30 Apr 2023 06:15:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FttUPltjV0RdnMnw5BP3akgr6owDfq6+bwnduM0TmM2LthljBqTJs5Nu32XfT8cmMCYufLj26DNSxkbuwe5Z0RckDAIbHxeCBIRoWR1A7U74E1ccrtxg8gRUTYKynLC/N0cwFm2+m7xST5fNB1/0VbHToNlT0ku38uT0lAwGnQCRA4450dp3kqNGpBAeetb39GVCRcj5LzW4k15jRYW1635+EKKVPxKarJsAbBgsKnBM25AsnOPUb6kDUfoxounifaigPVQH+h1v6OM8ltWJP+/dCPgBZ02D2r+N4/W0nmNtif8uRlTBGZoWax7+qp00PJsyznrizL1BJS4EbXedcA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=odkZ8HfHiza+jKPAfbqnFPQhDfN4CVkAQMeZK2QeXx4=; b=kH7o29vf0c0O17qwQAlGXkgM4lE4qMPH+pzxH9VnkxQI2WDngSzH4E0uBdQz9qCd7Ol5EPRDcoes8IO7Juu6pkhKJpJdBDCF1uIhDLXf1+3xUrx1WB3J4LU4hrxDTrMUI1CFYXfFMcvINoi9j+tGok4YHQn0spVR6PmRm90wcreK6dF/s0Y8IrYAAGfi1PUwhaqdrKj06ALf4silIVgvOKEZOjiDd6BL+u6HeWX98Z1jC4TYJkv7p/bYawJRMiRLhqhU3nWQ+KYLanfj5TIROajmB14halAiBRhAMXhXFZbURGCtf9rDINl10kdZ+Q/wv96tJW6FuR8DzmBKGL+4fQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=solid-run.com; dmarc=pass action=none header.from=solid-run.com; dkim=pass header.d=solid-run.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=solidrn.onmicrosoft.com; s=selector1-solidrn-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=odkZ8HfHiza+jKPAfbqnFPQhDfN4CVkAQMeZK2QeXx4=; b=Sd0cycgmlAu9ivKdsCQ0DjjTMcAbXfQR9QoAiPdX+1U84ak8QoG/bWLZy7aJbcqovD0nxu/dyjm5D6hKirSyN9DqZUw1Ga26sawNzX7ntfEYBKFT4DbSsSuAtOwROdq+HbJtlep4Z817ZGDeApjbI/QtzMvg7HsedEkSNoC/Vhk= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=solid-run.com; Received: from AM0PR04MB4723.eurprd04.prod.outlook.com (2603:10a6:208:c0::20) by AM7PR04MB7142.eurprd04.prod.outlook.com (2603:10a6:20b:113::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6340.26; Sun, 30 Apr 2023 13:15:29 +0000 Received: from AM0PR04MB4723.eurprd04.prod.outlook.com ([fe80::a5ea:bc1c:6fa6:8106]) by AM0PR04MB4723.eurprd04.prod.outlook.com ([fe80::a5ea:bc1c:6fa6:8106%5]) with mapi id 15.20.6340.026; Sun, 30 Apr 2023 13:15:29 +0000 From: Alvaro Karsz To: mst@redhat.com, jasowang@redhat.com Cc: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, xuanzhuo@linux.alibaba.com, Alvaro Karsz Subject: [RFC PATCH net 1/3] virtio: re-negotiate features if probe fails and features are blocked Date: Sun, 30 Apr 2023 16:15:16 +0300 Message-Id: <20230430131518.2708471-2-alvaro.karsz@solid-run.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230430131518.2708471-1-alvaro.karsz@solid-run.com> References: <20230430131518.2708471-1-alvaro.karsz@solid-run.com> X-ClientProxiedBy: ZR0P278CA0191.CHEP278.PROD.OUTLOOK.COM (2603:10a6:910:44::10) To AM0PR04MB4723.eurprd04.prod.outlook.com (2603:10a6:208:c0::20) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM0PR04MB4723:EE_|AM7PR04MB7142:EE_ X-MS-Office365-Filtering-Correlation-Id: 16b0bef8-0f73-4021-9fb6-08db497cf86e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: hXM2w5pYxEhvn+bOq/cQOEYlw+Q6qnc/93XLZFlVPQDTcpTdIDVyz4VFCvtLXGDc+eliqthmY1fthjXZTe4zZ8ZBmQVJDaZrQN3eul6w29yLYyqddIgOvyckUEILwlYst1ZGpEaaWeCAiVYNVNBntntl4XZ5CuGqDgcvjkdt2DGTEf52Yu590IE6ph16E1MfogvFKrEbGAYW8V1m7gOQ+B7/gJzgFfL/0Epzwr5LeCe8UYUifvWKXpIzdY1qCbrIlz4zVOYcpep4suGhKnsTZvXATEwhkz9ZYy5VpvjIPr6fPLsPzUjKYkblGK/j98OFtgQT3txQ/28pm4Ggf/VKpNfJJ7CZeNR9ORrNpyqw/8wV/DUmlF7ScJeWe1z8iA5aT8JO68hTaxccNWJ1EKNFya6YXlr8UqMPL4EzbUX3ZaJZLxwEtDYlvVnf+5hUjjkFm6yBL9ZM424ZwYRbFXSrwbCWZyHwxfhJJb4pG1A0YiZgv241QYIJjnaGR3IZTVj8q/oos6QJXp2fYBIuW9bUYWx2L+9Qrbg92RjS9U6wA4rb6i4vqB8+Xp08Q4mEuKvZXN+TSV8WZzBnS1JBvJWP+B2Pbk4Xt4gjNN4LJq6NkrJRHjtQGqJr99YR97BHw2al X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR04MB4723.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(376002)(366004)(136003)(346002)(396003)(39830400003)(451199021)(86362001)(36756003)(38100700002)(38350700002)(7416002)(5660300002)(15650500001)(4326008)(66946007)(316002)(66476007)(44832011)(8936002)(41300700001)(66556008)(8676002)(2906002)(83380400001)(2616005)(6666004)(52116002)(478600001)(6486002)(6506007)(1076003)(26005)(107886003)(6512007)(186003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 5ro8E+mwPuG9JhLh73ArBvdzQD10UpkLpXlcoLWxBfsgcazZdqBrqdZLOshZRiq6FKL4uyR/F6WxZMADPDJ7t/TqqPbm8XiDSbDZUNsBiT3WIulM2KZRIresieRTG5WgkPZKmtUWB8Met+EbKYYOx7yMXDe1X6pssXYKhnkDeH5JVPq1aVW9AgNv3sCK2vRthyHxyrQrzMG0qLsD33Mm30SWejCtfyasU4wFiW1o+YPebet+Rulk377kkA0ChM4fu25HvGmeYjMaSC+1iAomFJIlftmG/l47p7DZbuxNoYM055RM6y6Tp1bH6h+VHV0G3vDm4uMrrWA0LxvxFpBSabObfSdVvwHLOnUZF7pO6zf2HUs3x0rpYnuAt0T/45cPQ2EEGxG5G1U6n7uqyZk4ahNoV9nrjjbSO+dsPOez/bT3X+rGrRRXi+tA4M+AsynJl+CIiTUD+FQ8BQiKnEE2CoPbwjEtwY9qfOREyjY0J+I9yYlInWnkG+CgnywKmRWcWMB07BlpNKYjXbLsg+huu5xXRcsiX715E8qzt8Xdmd2v5FBu+ozkoXuhKJLa8UFEwXc+4LAYliYVAuMtTTyIx+5WVOY9dcRLJPNR4MYqUUkPqqZEIUGFMHCFwNHLJXO3NwAJ7z/lJwTSN/aPQlR9RIJV8PX2F/SzEBXvnfFR3izrF7TL3wKWUnZFvotB01P1gJp7KKPC1GffrryJP0ocX88zCL9D0F9hZpQFE4anuMc7X2ce/+vdKuEHTa9kJk3z68iOi7ia4wMKHXOfJIhADyxcIqc8gkMB0R+T66tG7gi4cwTXqaFERXUy1ltM38pNkPJzu8QtX1GVLF8wTZMXI3CO6torRmpoOmqHCfzmOrWgh7Putvxx53qzRfofHDQS1552g8wpWuplb4cE8SjDQL6ald3qhG4DGAI58asT2cCbt64lvbbomHBZPIZUPFD7EsByL5p/iLHjpPPlWtkOCfjKv/FlzqaWyetpfqBW5XBL39BzRNrLm8/YJTYzicmhBh+fcSdx9uKljeZHnB3fyZlytmf+MYFs9pHnzKAU7lShVhztUz2yBEOg4/nmNV7sZqXcIdvyZ5891wtYqbz7y+VkRbTKA0NDSk4KPaD/cO0xMSdqWz3f5eJwf09PKx6eTyY+9bbO1oZVHfPh7IAFQhSx5EC2wXxSfStWqX2kBpOS231c7QCsak/ceet8A0QjwpiRPond0nlegXhbwdzGhg/UAv4SFpqN3UFYgyDCtqjIvV3ZSx5jtDBsbAgZoVRatXokDrQ1S28VyOnArKxhCVE3dmq2lFvjtDcR1yg01bXgF0IJ1gQN6Ey6eSs6V+Mh0FlWwYcHYd9j1WOee9FhnwP6RIK/kZGlVasRcqm7Vo/S9FKjTLld2ziUez0J0TBwe2MzWuMFSUVAX5fBmTPK+7aZStryzo7i9BwrX2OchGW+s5JnLqTdYw60hHK8yD0GB6OBdvrcCoRfYnGp0a8PbAldEmvNcF9BNfZYWD3bWRGKUYhR+UX70MKT2gpM9lbbXR90S63bNj0nLBdeTqvSX3FVgvkwAa5BPPX5L7ny75G3Aej1ktsGhWCgeNghpyE/8lRIr3i4vTo8Y86zx8KWNA== X-OriginatorOrg: solid-run.com X-MS-Exchange-CrossTenant-Network-Message-Id: 16b0bef8-0f73-4021-9fb6-08db497cf86e X-MS-Exchange-CrossTenant-AuthSource: AM0PR04MB4723.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Apr 2023 13:15:29.7749 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a4a8aaf3-fd27-4e27-add2-604707ce5b82 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: KhOtNRtHXt4b7rdzP+o8KPj0Y8nGyVd3wie0hsm4+PEI69GK6c07ABlowNcMR0qfqI8T084Fr8D+2LPHjRQC7aV0U7xzKKEulhGmPIEHGis= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR04MB7142 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1764608018676507983?= X-GMAIL-MSGID: =?utf-8?q?1764608018676507983?= This patch exports a new virtio core function: virtio_block_feature. The function should be called during a virtio driver probe. If a virtio driver blocks features during probe and fails probe, virtio core will reset the device, try to re-negotiate the new features and probe again. Signed-off-by: Alvaro Karsz --- drivers/virtio/virtio.c | 73 ++++++++++++++++++++++++++++++----------- include/linux/virtio.h | 3 ++ 2 files changed, 56 insertions(+), 20 deletions(-) diff --git a/drivers/virtio/virtio.c b/drivers/virtio/virtio.c index 3893dc29eb2..eaad5b6a7a9 100644 --- a/drivers/virtio/virtio.c +++ b/drivers/virtio/virtio.c @@ -167,6 +167,13 @@ void virtio_add_status(struct virtio_device *dev, unsigned int status) } EXPORT_SYMBOL_GPL(virtio_add_status); +void virtio_block_feature(struct virtio_device *dev, unsigned int f) +{ + BUG_ON(f >= 64); + dev->blocked_features |= (1ULL << f); +} +EXPORT_SYMBOL_GPL(virtio_block_feature); + /* Do some validation, then set FEATURES_OK */ static int virtio_features_ok(struct virtio_device *dev) { @@ -234,17 +241,13 @@ void virtio_reset_device(struct virtio_device *dev) } EXPORT_SYMBOL_GPL(virtio_reset_device); -static int virtio_dev_probe(struct device *_d) +static int virtio_negotiate_features(struct virtio_device *dev) { - int err, i; - struct virtio_device *dev = dev_to_virtio(_d); struct virtio_driver *drv = drv_to_virtio(dev->dev.driver); u64 device_features; u64 driver_features; u64 driver_features_legacy; - - /* We have a driver! */ - virtio_add_status(dev, VIRTIO_CONFIG_S_DRIVER); + int i, ret; /* Figure out what features the device supports. */ device_features = dev->config->get_features(dev); @@ -279,30 +282,61 @@ static int virtio_dev_probe(struct device *_d) if (device_features & (1ULL << i)) __virtio_set_bit(dev, i); - err = dev->config->finalize_features(dev); - if (err) - goto err; + /* Remove blocked features */ + dev->features &= ~dev->blocked_features; + + ret = dev->config->finalize_features(dev); + if (ret) + goto exit; if (drv->validate) { u64 features = dev->features; - err = drv->validate(dev); - if (err) - goto err; + ret = drv->validate(dev); + if (ret) + goto exit; /* Did validation change any features? Then write them again. */ if (features != dev->features) { - err = dev->config->finalize_features(dev); - if (err) - goto err; + ret = dev->config->finalize_features(dev); + if (ret) + goto exit; } } - err = virtio_features_ok(dev); - if (err) - goto err; + ret = virtio_features_ok(dev); +exit: + return ret; +} + +static int virtio_dev_probe(struct device *_d) +{ + int err; + struct virtio_device *dev = dev_to_virtio(_d); + struct virtio_driver *drv = drv_to_virtio(dev->dev.driver); + u64 blocked_features; + bool renegotiate = true; + + /* We have a driver! */ + virtio_add_status(dev, VIRTIO_CONFIG_S_DRIVER); + + /* Store blocked features and attempt to negotiate features & probe. + * If the probe fails, we check if the driver has blocked any new features. + * If it has, we reset the device and try again with the new features. + */ + while (renegotiate) { + blocked_features = dev->blocked_features; + err = virtio_negotiate_features(dev); + if (err) + break; + + err = drv->probe(dev); + if (err && blocked_features != dev->blocked_features) + virtio_reset_device(dev); + else + renegotiate = false; + } - err = drv->probe(dev); if (err) goto err; @@ -319,7 +353,6 @@ static int virtio_dev_probe(struct device *_d) err: virtio_add_status(dev, VIRTIO_CONFIG_S_FAILED); return err; - } static void virtio_dev_remove(struct device *_d) diff --git a/include/linux/virtio.h b/include/linux/virtio.h index b93238db94e..2de9b2d3ca4 100644 --- a/include/linux/virtio.h +++ b/include/linux/virtio.h @@ -109,6 +109,7 @@ int virtqueue_resize(struct virtqueue *vq, u32 num, * @vringh_config: configuration ops for host vrings. * @vqs: the list of virtqueues for this device. * @features: the features supported by both driver and device. + * @blocked_features: the features blocked by the driver that can't be negotiated. * @priv: private pointer for the driver's use. */ struct virtio_device { @@ -124,6 +125,7 @@ struct virtio_device { const struct vringh_config_ops *vringh_config; struct list_head vqs; u64 features; + u64 blocked_features; void *priv; }; @@ -133,6 +135,7 @@ void virtio_add_status(struct virtio_device *dev, unsigned int status); int register_virtio_device(struct virtio_device *dev); void unregister_virtio_device(struct virtio_device *dev); bool is_virtio_device(struct device *dev); +void virtio_block_feature(struct virtio_device *dev, unsigned int f); void virtio_break_device(struct virtio_device *dev); void __virtio_unbreak_device(struct virtio_device *dev); From patchwork Sun Apr 30 13:15:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alvaro Karsz X-Patchwork-Id: 88917 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp2163478vqo; Sun, 30 Apr 2023 07:05:36 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5hkNGgBQVqCuPn7SNzEWFX0VJ0rEvkcAOpdwC2Os8inakNlP2o1EIMdEhuhm/VvWWooh4Y X-Received: by 2002:a05:6a20:9382:b0:f3:1c8b:ce54 with SMTP id x2-20020a056a20938200b000f31c8bce54mr14335709pzh.4.1682863535974; Sun, 30 Apr 2023 07:05:35 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1682863535; cv=pass; d=google.com; s=arc-20160816; b=FdPXs3SdN1uJWxLuNHasXFtvYxxQUDTFKoIDe9c/CUYIeJqt54VuQJ0pNfiGgcaIv7 YdTbFK9YN7i2ZdTWLaajhQOOEyS/4oIgj6G/1xOEkg5wMTEQDj+ZVZfsy6blDpa65PCw HLXZEf3hgPV8PFTHsEHe4tEZB4PzracI+yvRPfsWNIhIH5RiRx/jJL106k6rMnq4E3+0 hOeyICkUVSffJTSbY5dXYpEkwykrb+JJJKQlYrAK1P26iD2WiCWCn18ZzbVT2Oo4n9ki o/SoNFaaiqVxlw9tSv31WZT7CG37yoxsHhZF6iG8+WLBLk/rX8kEoNl8U40vZP+OZcdC Akiw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:content-transfer-encoding :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=T7aJHs4ZPbQLM+0r0hvl1cR057SHLuG2e2tFeWcuOjo=; b=HZ83VJOl349996epIhv1ZB0RkNU4EMzAlkMlilPOz5BDvfL/y9J6GkBG8UG2GyLZzf w1xL/hwIeOsESwjie+ERFWIJGl+kV1daBsUYfMks6lHeMYWChUtArzeyXcOqZXiwrB1V 0oRKHNPta3AZg2HviqxzbqnCODSfNCmaaDFo0BPaY8FNQEkhwjRplnxgBEyGMl9RH1cW BgAeH1MjSYe+B8HkJzDcP4Io8C0CuuHZpy4LyT6XIQFax78wUOxiwH1jh7dwNeyudqyK lcPNRDny1CaWA5HUFDf7GC/omYabRqu+1UTZ8hZZk8e+v0Xq68q8PJgpwuBHj1cwMkhl fPww== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@solidrn.onmicrosoft.com header.s=selector1-solidrn-onmicrosoft-com header.b=SgU4sjQc; arc=pass (i=1 spf=pass spfdomain=solid-run.com dkim=pass dkdomain=solid-run.com dmarc=pass fromdomain=solid-run.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id l6-20020a656806000000b005139e8ca7b1si27355194pgt.494.2023.04.30.07.05.13; Sun, 30 Apr 2023 07:05:35 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@solidrn.onmicrosoft.com header.s=selector1-solidrn-onmicrosoft-com header.b=SgU4sjQc; arc=pass (i=1 spf=pass spfdomain=solid-run.com dkim=pass dkdomain=solid-run.com dmarc=pass fromdomain=solid-run.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230432AbjD3NPk (ORCPT + 99 others); Sun, 30 Apr 2023 09:15:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60540 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230374AbjD3NPg (ORCPT ); Sun, 30 Apr 2023 09:15:36 -0400 Received: from EUR02-AM0-obe.outbound.protection.outlook.com (mail-am0eur02on2066.outbound.protection.outlook.com [40.107.247.66]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9F024269E; Sun, 30 Apr 2023 06:15:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=NH3gQD2q4yXWXan/mb8tHW87i2u2iEWh9VAEB5qIOsTu7C9spoJlLB25pqKinsbl7I0RTnn/uW7ouepwAk/LrNvxb+p8z2G8Kl3gBLZscHH1oCCmvEV/4au8IdoNaTGJ7FByZUhIF70q59SFT5jVLZmw7p1pw95QzhINgYgnDENqPLdr+KpW04zyN98Ts7coRG4HLZbO3adA8CfDuBmEB7I9+ORjwyYAGEGvrmDCbRLWtEuK+MoKyybHMrZOAc1POAdx8dha+vjEd8kJK7hyr3PZUQ5YY5/xQcKpK56uLG8Jt9oxIc2Sf+SDbDxRewCHfCoBy1+Di2ROafkqZQU8eA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=T7aJHs4ZPbQLM+0r0hvl1cR057SHLuG2e2tFeWcuOjo=; b=fYo3jf7V+004PIXfSWfIBVPeNBks7J3l2I8rWbndDoilu5Fv1x90l/pSbTw8boi9otN0C7ETelqD/Rq8ignYiACl/3Q+s8AM3P7u3SQXzvJpW6mOsP0fOSpqTtcxdA88NMsYkDPaXGmTI0q8Acdmr0xBg0B3yncqipDWWHDOm+r0K9Fl1enUHu/8eqTpbt+ctuz+312m4JHummKOVeJ1F7jnJHYm5hZA7V3IVbhDnBmV97qJiSvdPy6XqyJXQkkMgLuyKQFxnKJU3unEOUdb8cVBICjLFN+pe9Rd2HoFU3FbdcHpDmQbJtJ9LfpJe62tYEMmftxOp+rIbDT2ERhjMg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=solid-run.com; dmarc=pass action=none header.from=solid-run.com; dkim=pass header.d=solid-run.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=solidrn.onmicrosoft.com; s=selector1-solidrn-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=T7aJHs4ZPbQLM+0r0hvl1cR057SHLuG2e2tFeWcuOjo=; b=SgU4sjQc2/iebYT+sIaY7TBMvEK0MP2DKGAwb4kdttAp4Fn72O2KyJ7ypKO+rHuFVWvzf+Yn4Lq5Ji0eaXDdHu7caVE96ID058Eo2BjwpIIv3XNYQe6ljntytyJSbKhRTN0VIDekJwk5EdK9qz5NZ3r498hhlxyrPi7jzRNxK7s= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=solid-run.com; Received: from AM0PR04MB4723.eurprd04.prod.outlook.com (2603:10a6:208:c0::20) by AM7PR04MB7142.eurprd04.prod.outlook.com (2603:10a6:20b:113::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6340.26; Sun, 30 Apr 2023 13:15:31 +0000 Received: from AM0PR04MB4723.eurprd04.prod.outlook.com ([fe80::a5ea:bc1c:6fa6:8106]) by AM0PR04MB4723.eurprd04.prod.outlook.com ([fe80::a5ea:bc1c:6fa6:8106%5]) with mapi id 15.20.6340.026; Sun, 30 Apr 2023 13:15:31 +0000 From: Alvaro Karsz To: mst@redhat.com, jasowang@redhat.com Cc: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, xuanzhuo@linux.alibaba.com, Alvaro Karsz Subject: [RFC PATCH net 2/3] virtio-net: allow usage of vrings smaller than MAX_SKB_FRAGS + 2 Date: Sun, 30 Apr 2023 16:15:17 +0300 Message-Id: <20230430131518.2708471-3-alvaro.karsz@solid-run.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230430131518.2708471-1-alvaro.karsz@solid-run.com> References: <20230430131518.2708471-1-alvaro.karsz@solid-run.com> X-ClientProxiedBy: ZR0P278CA0191.CHEP278.PROD.OUTLOOK.COM (2603:10a6:910:44::10) To AM0PR04MB4723.eurprd04.prod.outlook.com (2603:10a6:208:c0::20) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM0PR04MB4723:EE_|AM7PR04MB7142:EE_ X-MS-Office365-Filtering-Correlation-Id: 5748f0f7-03c0-4712-ddf2-08db497cf974 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: TnD5wNTSfyEOeCdUUfS6WFkqe6eL3KiVU2jhRjBMKwyYIXQOyBnvm0BfLgleKQEH5wG7uMEy7h/kAJw2JX5m9is8oSfOfRFRMLqyTPEyf5VamWQ/n/OUUVRIXLAF78AJEISwdXAwqg4T34htm36mHsW0t8IAx2fgvpRA8atyDDezI1Vn0yz0l/uPJpHWCTHY+zeU0wUzJi9WDAQ0Djzcem1DyE7QxvWP29fwlhbMcVdrebTeKdRzL5y1LveTf1XpSZukQZiy9XCVqOPNir+lttYaYuDmqTfKsISZzXQanI6+2reF4IB455MkZVh9czILk8/bqjErBIXFUSZFt3zrrxoKImQsMGXNL2eI6R6dGPzzpiEPCVNzeqs0J+623bjrNJXxGggOm1pZ9RydMqmJx37OKUqImCgBmHJks1gsjsMjRM+EKzKbG/wMRZjbjEtnl1nJWBI+f1Omj5cjguffF4CeixHPVkvSBoEpaM4Y9Jluh50nPnFkdnFCNVgn6ztTjBPWbGAk9fVQFLxXGytr/FPrrCKHxDLEQ1LLlFupREhCoORWdCwU+VKr9xOrwH0vo36fxSP1NSotkoLgRjvofTRoLDunht2QewhIu6VVfrNFJGK3lYlJ76kmCOuupJOF X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR04MB4723.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(376002)(366004)(136003)(346002)(396003)(39830400003)(451199021)(86362001)(36756003)(38100700002)(38350700002)(7416002)(5660300002)(4326008)(66946007)(316002)(66476007)(44832011)(8936002)(41300700001)(66556008)(8676002)(2906002)(83380400001)(2616005)(6666004)(52116002)(478600001)(6486002)(6506007)(1076003)(26005)(107886003)(6512007)(186003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: RROlZnH3FJfpUbh1QyDF9W7VN3Vog9edRitDeQ5ro0zYr5+NdsmZ1u5bOrajdLtolGEcB2rLnMf0YiaJsRS6K6QW9CH+DfiY2UgikDUuaEVsE5TqV8i9fHruPvgabMZmibFJgq6GSgssIMv0B2TjAJFQVUberhvd+icBR7lrq0perMwbvivmnygQRs9uuh8F5TA11NPiUDeov2YRC/6+DYTutDNmZm5+8YyrtdrHF4NqEgPYCaORjy8d6uFDEsDjBrjAQgW7rP/3V/EpwUvEdXaNil3cNkNJNHsQgxhbh6r5VDnGt0JrmJYOhIp2WXedmiuscWiP3UKDRXl+crnnu0R2kXO3DZmWFqogEKDwwz82bkroGA1ZkYHRwqsyCHyFRvAjR6yvAg//A4s9plsSjAiCFSouG4oUwPZrPf/uISJ+JbJKxABCREmgb57pm6ZmTkHKrVw5k0NUmYAfQ8Q9kK1PhmClu71p4/yB5LeDuehCSlTiJqNjRbl0x2k+sVpECJeVM2m4UWnsHiDHbJ6EgV/9tEjeDJyO06UVQ/pahdtct3cewOrOp7o7i+mKT4JyqYGABBTjpaJNuI1hwQckHkrGtTkiB5tKpbtxejiF6ZGaTDSVaZ6yOFb5T3mZWSUSTwRaWNxyKX3PPRLjgFmeTzvVGnHFQ4D0hf5wDZUEqw8Utfdf+j1JJLlqRqG3gX9E6akpR1QQqaEGOBcEyVC3fl9NCTv86irD8uekyHa+BxeqUcf7PKYcOtWLCoGnavBIn1F03rrzE0Ol3PJKLvxymL989Pv369xRJOAGUMmizuJ37JRX4KUkYQikBuZB9A5ofQbKYDE2ZUa46DCK4foyp9qXY/0utDo5Qct77Se/DaeAKTCY9kA7ln7nNFFQ5B7jRnGRXnvatcFZWG4vmmi1rzkQCOMT7iiIxVqsGhaVvAYstxPV6WdSydl7caLVj84CdhFVIF2wPlNUDl1N9Prgw3MZlakvBTvdfy5ixo96AoofKsHX4BnkHkhmN6GQwHpXsXwB9zHERcq/HG+3Sm4JZAxAs+XQVbpZyZ8iEVvjiNSbkcg3HCiaCgADtqVayhzSYrOnjboGZTCgLlk+8Kar9Za/MEcAcTf+DIaNP37YLupxkcgxwLmstE3QtlvqNhsM/KtxS0ka7MbbeVSJgFylue5M/gwmxTg9CcRk9+42Kw8BlMgu7YH78R/VEyQULOOx5NgDHn9XNTVzrHjTUoHvwXtrfWVar0MDtERt5FHwG/yIQyf6vvkxAivFhvRTUl31DtxpsuJzpijDRKC0GDvRoa6lsKw0lh6+Tj6ZgTV5Dmf8DM9JHP6ZNGMQOKMJ/z46JYLtQw/MlTIlpHl5Qmf8HoYzRh9hU63TJy1Ek3CxaSIlDk0GgvwESyRPvPKqvBR2TOiylUBSIXAlFnFQGHFtOiWWHqW43cI+r0b9yzzsgdBJuXTAPOUkILnzHfyiU6XG21EbROtmki0sP+uLEt1Po87T33lM8sIMVr9KihaV39aP0jCJ5mQQI2Wpi1wkXVatlBJlKabBgSHO/TzlrZBG5csJR+m15FflBPwtJDM16/d5OKPPH2e/S3NRM6Nk6/GbWkK2/JhXH7ZFzwxXAn3Dxw== X-OriginatorOrg: solid-run.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5748f0f7-03c0-4712-ddf2-08db497cf974 X-MS-Exchange-CrossTenant-AuthSource: AM0PR04MB4723.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Apr 2023 13:15:31.4932 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a4a8aaf3-fd27-4e27-add2-604707ce5b82 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: tHWJRQlFn/vy4TC7MPc/nAy+wPpJVS5sJsj/ci5biCxHvA3GZ4KQCWoLvjFx6y31jatYnEX82XLAQuTrRkiExqJ7ar3DHbRf6N417CfFhjo= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR04MB7142 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1764610315251244758?= X-GMAIL-MSGID: =?utf-8?q?1764610315251244758?= At the moment, if a network device uses vrings with less than MAX_SKB_FRAGS + 2 entries, the device won't be functional. The following condition vq->num_free >= 2 + MAX_SKB_FRAGS will always evaluate to false, leading to TX timeouts. This patch introduces a new variable, single_pkt_max_descs, that holds the max number of descriptors we may need to handle a single packet. This patch also detects the small vring during probe, blocks some features that can't be used with small vrings, and fails probe, leading to a reset and features re-negotiation. Features that can't be used with small vrings: GRO features (VIRTIO_NET_F_GUEST_*): When we use small vrings, we may not have enough entries in the ring to chain page size buffers and form a 64K buffer. So we may need to allocate 64k of continuous memory, which may be too much when the system is stressed. This patch also fixes the MTU size in small vring cases to be up to the default one, 1500B. Signed-off-by: Alvaro Karsz --- drivers/net/virtio_net.c | 149 +++++++++++++++++++++++++++++++++++++-- 1 file changed, 144 insertions(+), 5 deletions(-) diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index 8d8038538fc..b4441d63890 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -103,6 +103,8 @@ struct virtnet_rq_stats { #define VIRTNET_SQ_STAT(m) offsetof(struct virtnet_sq_stats, m) #define VIRTNET_RQ_STAT(m) offsetof(struct virtnet_rq_stats, m) +#define IS_SMALL_VRING(size) ((size) < MAX_SKB_FRAGS + 2) + static const struct virtnet_stat_desc virtnet_sq_stats_desc[] = { { "packets", VIRTNET_SQ_STAT(packets) }, { "bytes", VIRTNET_SQ_STAT(bytes) }, @@ -268,6 +270,12 @@ struct virtnet_info { /* Does the affinity hint is set for virtqueues? */ bool affinity_hint_set; + /* How many ring descriptors we may need to transmit a single packet */ + u16 single_pkt_max_descs; + + /* Do we have virtqueues with small vrings? */ + bool svring; + /* CPU hotplug instances for online & dead */ struct hlist_node node; struct hlist_node node_dead; @@ -455,6 +463,7 @@ static struct sk_buff *page_to_skb(struct virtnet_info *vi, unsigned int copy, hdr_len, hdr_padded_len; struct page *page_to_free = NULL; int tailroom, shinfo_size; + u16 max_frags = MAX_SKB_FRAGS; char *p, *hdr_p, *buf; p = page_address(page) + offset; @@ -520,7 +529,10 @@ static struct sk_buff *page_to_skb(struct virtnet_info *vi, * tries to receive more than is possible. This is usually * the case of a broken device. */ - if (unlikely(len > MAX_SKB_FRAGS * PAGE_SIZE)) { + if (unlikely(vi->svring)) + max_frags = 1; + + if (unlikely(len > max_frags * PAGE_SIZE)) { net_dbg_ratelimited("%s: too much data\n", skb->dev->name); dev_kfree_skb(skb); return NULL; @@ -612,7 +624,7 @@ static void check_sq_full_and_disable(struct virtnet_info *vi, * Since most packets only take 1 or 2 ring slots, stopping the queue * early means 16 slots are typically wasted. */ - if (sq->vq->num_free < 2+MAX_SKB_FRAGS) { + if (sq->vq->num_free < vi->single_pkt_max_descs) { netif_stop_subqueue(dev, qnum); if (use_napi) { if (unlikely(!virtqueue_enable_cb_delayed(sq->vq))) @@ -620,7 +632,7 @@ static void check_sq_full_and_disable(struct virtnet_info *vi, } else if (unlikely(!virtqueue_enable_cb_delayed(sq->vq))) { /* More just got used, free them then recheck. */ free_old_xmit_skbs(sq, false); - if (sq->vq->num_free >= 2+MAX_SKB_FRAGS) { + if (sq->vq->num_free >= vi->single_pkt_max_descs) { netif_start_subqueue(dev, qnum); virtqueue_disable_cb(sq->vq); } @@ -1108,6 +1120,10 @@ static int virtnet_build_xdp_buff_mrg(struct net_device *dev, return 0; if (*num_buf > 1) { + /* Small vring - can't be more than 1 buffer */ + if (unlikely(vi->svring)) + return -EINVAL; + /* If we want to build multi-buffer xdp, we need * to specify that the flags of xdp_buff have the * XDP_FLAGS_HAS_FRAG bit. @@ -1828,7 +1844,7 @@ static void virtnet_poll_cleantx(struct receive_queue *rq) free_old_xmit_skbs(sq, true); } while (unlikely(!virtqueue_enable_cb_delayed(sq->vq))); - if (sq->vq->num_free >= 2 + MAX_SKB_FRAGS) + if (sq->vq->num_free >= vi->single_pkt_max_descs) netif_tx_wake_queue(txq); __netif_tx_unlock(txq); @@ -1919,7 +1935,7 @@ static int virtnet_poll_tx(struct napi_struct *napi, int budget) virtqueue_disable_cb(sq->vq); free_old_xmit_skbs(sq, true); - if (sq->vq->num_free >= 2 + MAX_SKB_FRAGS) + if (sq->vq->num_free >= vi->single_pkt_max_descs) netif_tx_wake_queue(txq); opaque = virtqueue_enable_cb_prepare(sq->vq); @@ -3862,6 +3878,15 @@ static bool virtnet_check_guest_gso(const struct virtnet_info *vi) virtio_has_feature(vi->vdev, VIRTIO_NET_F_GUEST_USO6)); } +static bool virtnet_check_host_gso(const struct virtnet_info *vi) +{ + return virtio_has_feature(vi->vdev, VIRTIO_NET_F_HOST_TSO4) || + virtio_has_feature(vi->vdev, VIRTIO_NET_F_HOST_TSO6) || + virtio_has_feature(vi->vdev, VIRTIO_NET_F_HOST_ECN) || + virtio_has_feature(vi->vdev, VIRTIO_NET_F_HOST_UFO) || + virtio_has_feature(vi->vdev, VIRTIO_NET_F_HOST_USO); +} + static void virtnet_set_big_packets(struct virtnet_info *vi, const int mtu) { bool guest_gso = virtnet_check_guest_gso(vi); @@ -3876,6 +3901,112 @@ static void virtnet_set_big_packets(struct virtnet_info *vi, const int mtu) } } +static u16 virtnet_calc_max_descs(struct virtnet_info *vi) +{ + if (vi->svring) { + if (virtnet_check_host_gso(vi)) + return 4; /* 1 fragment + linear part + virtio header + GSO header */ + else + return 3; /* 1 fragment + linear part + virtio header */ + } else { + return MAX_SKB_FRAGS + 2; + } +} + +static bool virtnet_uses_svring(struct virtnet_info *vi) +{ + u32 i; + + /* If a transmit/receive virtqueue is small, + * we cannot handle fragmented packets. + */ + for (i = 0; i < vi->max_queue_pairs; i++) { + if (IS_SMALL_VRING(virtqueue_get_vring_size(vi->sq[i].vq)) || + IS_SMALL_VRING(virtqueue_get_vring_size(vi->rq[i].vq))) + return true; + } + + return false; +} + +/* Function returns the number of features it blocked */ +static int virtnet_block_svring_unsupported(struct virtio_device *vdev) +{ + int cnt = 0; + /* Block Virtio guest GRO features. + * Asking Linux to allocate 64k of continuous memory is too much, + * specially when the system is stressed. + * + * If VIRTIO_NET_F_MRG_RXBUF is negotiated we can allcoate smaller + * buffers, but since the ring is small, the buffers can be quite big. + * + */ + if (virtio_has_feature(vdev, VIRTIO_NET_F_GUEST_TSO4)) { + virtio_block_feature(vdev, VIRTIO_NET_F_GUEST_TSO4); + cnt++; + } + if (virtio_has_feature(vdev, VIRTIO_NET_F_GUEST_TSO6)) { + virtio_block_feature(vdev, VIRTIO_NET_F_GUEST_TSO6); + cnt++; + } + if (virtio_has_feature(vdev, VIRTIO_NET_F_GUEST_ECN)) { + virtio_block_feature(vdev, VIRTIO_NET_F_GUEST_ECN); + cnt++; + } + if (virtio_has_feature(vdev, VIRTIO_NET_F_GUEST_UFO)) { + virtio_block_feature(vdev, VIRTIO_NET_F_GUEST_UFO); + cnt++; + } + if (virtio_has_feature(vdev, VIRTIO_NET_F_GUEST_USO4)) { + virtio_block_feature(vdev, VIRTIO_NET_F_GUEST_USO4); + cnt++; + } + if (virtio_has_feature(vdev, VIRTIO_NET_F_GUEST_USO6)) { + virtio_block_feature(vdev, VIRTIO_NET_F_GUEST_USO6); + cnt++; + } + + return cnt; +} + +static int virtnet_fixup_svring(struct virtnet_info *vi) +{ + int i; + /* Do we use small vrings? + * If not, nothing we need to do. + */ + vi->svring = virtnet_uses_svring(vi); + if (!vi->svring) + return 0; + + /* Some features can't be used with small vrings. + * Block those and return an error. + * This will trigger a reprobe without the blocked + * features. + */ + if (virtnet_block_svring_unsupported(vi->vdev)) + return -EOPNOTSUPP; + + /* Disable NETIF_F_SG */ + vi->dev->hw_features &= ~NETIF_F_SG; + + /* Don't use MTU bigger than default */ + if (vi->dev->max_mtu > ETH_DATA_LEN) + vi->dev->max_mtu = ETH_DATA_LEN; + if (vi->dev->mtu > ETH_DATA_LEN) + vi->dev->mtu = ETH_DATA_LEN; + + /* Don't use big packets */ + vi->big_packets = false; + vi->big_packets_num_skbfrags = 1; + + /* Fix min_buf_len for receive virtqueues */ + for (i = 0; i < vi->max_queue_pairs; i++) + vi->rq[i].min_buf_len = mergeable_min_buf_len(vi, vi->rq[i].vq); + + return 0; +} + static int virtnet_probe(struct virtio_device *vdev) { int i, err = -ENOMEM; @@ -4061,6 +4192,14 @@ static int virtnet_probe(struct virtio_device *vdev) if (err) goto free; + /* Do required fixups in case we are using small vrings */ + err = virtnet_fixup_svring(vi); + if (err) + goto free_vqs; + + /* Calculate the max. number of descriptors we may need to transmit a single packet */ + vi->single_pkt_max_descs = virtnet_calc_max_descs(vi); + #ifdef CONFIG_SYSFS if (vi->mergeable_rx_bufs) dev->sysfs_rx_queue_group = &virtio_net_mrg_rx_group; From patchwork Sun Apr 30 13:15:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alvaro Karsz X-Patchwork-Id: 88916 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp2163479vqo; Sun, 30 Apr 2023 07:05:36 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5jvsTFh+E6e+6ZUOcU7F7N1uMFAuqXfUGITuHcCR1zOmTa3mCX4wNhkAEz24RgV9gSxzAV X-Received: by 2002:a05:6a20:2201:b0:ef:bd:38 with SMTP id u1-20020a056a20220100b000ef00bd0038mr9857069pzb.55.1682863535960; Sun, 30 Apr 2023 07:05:35 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1682863535; cv=pass; d=google.com; s=arc-20160816; b=a3zaSoy4ZgPp1tMXRYvFHUOXsjS/sLNfibSYa8sJ1PuXXVd0fWZ4+2TcfsLyOSrMPp seySXS/6qQmxMDsL7gF8MsscbQmLFLqv/NUMamQJF6DVPDsAji7YSGt2XEbAsKdv7RS0 AAwR7aXzIQhpRnmJlK/saHubhIstqBR7MDWq5WTDrEfBlnS5TAWpE/2J4F5KJWOrIMVy t2NwKSR3qu/HG4gHJJdywTEsp0zlkidFCf6uJv257+J/oKWZtLoIb9YF0IDedD011Hpg 60rXsKgO7/rXTOSwW2x+Xt5eLbx+GQ63HCX70zjdo7zCuopCt2U7blyjhdGA2N0LnTOb DxSg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:content-transfer-encoding :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=3CkMn617NGcOIW82fXp6biqVBtJmQyqTZuuxdSUg1fE=; b=iX0yoo8OJohu4texGDZ9mAgYrmGLwQlOL6w+zke94A8dtRXC8+0hqwJRtSBtQd8thl HZJIHqIpAtaA7tdPCZ7p3h8eOWuNtch279vYPGHDTfMTFRTUHhdn2Btid5aYSwhIvFle CD7/OMftSyFH26Ob3D90igDc7dED4vecGvRzqksKeAkKLmcLTCk+ztBHCvqmmYeFCKSB zVR2PebQ9cGRoQp+/vaRtJvfMH71RQtxCag0CIDZZA2pasElh53b+4Rm8C3IRLo3ScK6 9YoYeZXhVC3jzhQ5AbRx2ZAUIPnl9Fhzb+c4jy2Uu5apLHU6wBivHyWw1ucPS9HT56O1 3fzg== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@solidrn.onmicrosoft.com header.s=selector1-solidrn-onmicrosoft-com header.b=dhX1SOch; arc=pass (i=1 spf=pass spfdomain=solid-run.com dkim=pass dkdomain=solid-run.com dmarc=pass fromdomain=solid-run.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id b29-20020a631b1d000000b0051f047acf17si24683377pgb.525.2023.04.30.07.05.13; Sun, 30 Apr 2023 07:05:35 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@solidrn.onmicrosoft.com header.s=selector1-solidrn-onmicrosoft-com header.b=dhX1SOch; arc=pass (i=1 spf=pass spfdomain=solid-run.com dkim=pass dkdomain=solid-run.com dmarc=pass fromdomain=solid-run.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230336AbjD3NPx (ORCPT + 99 others); Sun, 30 Apr 2023 09:15:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60546 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230391AbjD3NPh (ORCPT ); Sun, 30 Apr 2023 09:15:37 -0400 Received: from EUR02-AM0-obe.outbound.protection.outlook.com (mail-am0eur02on2066.outbound.protection.outlook.com [40.107.247.66]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A40D326AB; Sun, 30 Apr 2023 06:15:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=YglorN+m/uNV2tCqpeqx9vD+nAgucbhf6rYQaNjMM/ArjiprJ7Uurdy6VQhGstbN+bBR9g3R/dPSz5RBpSY23kI7VKEe6kRxT+PJldhLZEE3u1sjjL+rS5TC4a4dRIjF6theBaYtcBw8CkblMvKTd2t2amDBIuqJhn6B7E3zKE7BRbezMubzizaENYh1QJpNaqtHtoJp7t2py6wfzxcUOZJTbsWGxQycPwsFq/LOJieHSgUMQg5yBOcIJhOpT9ZzOszznN3VqV9ROAy51asYmViErlcJbSrW0vsBfNrqDjs9u/de2D5093WDGTWNsxbuckvQqK8VuZuxjrmDbZX4ug== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=3CkMn617NGcOIW82fXp6biqVBtJmQyqTZuuxdSUg1fE=; b=dZeOXkwnXsXtJ1/AZJVgtVg8rI0jfbSXN89NkSQ4MDoG7D7l+Pj1/Fs0GfCjK8KCB6t7E4WiXxTDjmzVEA5pgILXlVhYMvZp/XaERTYgaQLu17ChMKP8hEqzAtDHHd/VIx0KNzyhsN5ymtGvGqPasQXY16OjVqaeRyHsCegEAVXSjwuQoYGYHv525GeiGtRUE1+FI610Zipfdhcu2Ra5kBEnuB7yl/hYU9gaT6YnVVmRJ3ZYHPVbSe3rAFg4O7gKrIKaVpb/9LjaFxezRxDSRF4LkC5d524aE/yksfax8kzQDlKG1BbX0y+h9t1dMXJubZyruoSEnBdbChxWk6aoHA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=solid-run.com; dmarc=pass action=none header.from=solid-run.com; dkim=pass header.d=solid-run.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=solidrn.onmicrosoft.com; s=selector1-solidrn-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3CkMn617NGcOIW82fXp6biqVBtJmQyqTZuuxdSUg1fE=; b=dhX1SOchEQFpionO8As6uBA+yUxPwyY33dmENTGXwKtulv/3iSCPD3Xt6VDJPQylcX71Np1enPN8Q+lqtGsFD6U+6DC/a44B+7EztU5t2IkmQEqY7/GI9ro92qQ6tEeTuv4SBi1CM7rbd2kGxLoSb89bgmes5pAPY3aIVfH6A2U= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=solid-run.com; Received: from AM0PR04MB4723.eurprd04.prod.outlook.com (2603:10a6:208:c0::20) by AM7PR04MB7142.eurprd04.prod.outlook.com (2603:10a6:20b:113::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6340.26; Sun, 30 Apr 2023 13:15:33 +0000 Received: from AM0PR04MB4723.eurprd04.prod.outlook.com ([fe80::a5ea:bc1c:6fa6:8106]) by AM0PR04MB4723.eurprd04.prod.outlook.com ([fe80::a5ea:bc1c:6fa6:8106%5]) with mapi id 15.20.6340.026; Sun, 30 Apr 2023 13:15:33 +0000 From: Alvaro Karsz To: mst@redhat.com, jasowang@redhat.com Cc: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, xuanzhuo@linux.alibaba.com, Alvaro Karsz Subject: [RFC PATCH net 3/3] virtio-net: block ethtool from converting a ring to a small ring Date: Sun, 30 Apr 2023 16:15:18 +0300 Message-Id: <20230430131518.2708471-4-alvaro.karsz@solid-run.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230430131518.2708471-1-alvaro.karsz@solid-run.com> References: <20230430131518.2708471-1-alvaro.karsz@solid-run.com> X-ClientProxiedBy: ZR0P278CA0191.CHEP278.PROD.OUTLOOK.COM (2603:10a6:910:44::10) To AM0PR04MB4723.eurprd04.prod.outlook.com (2603:10a6:208:c0::20) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM0PR04MB4723:EE_|AM7PR04MB7142:EE_ X-MS-Office365-Filtering-Correlation-Id: f007aea1-0fad-4575-1c5f-08db497cfa7b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: kwIAWr7AprY5tyGqBsbUgR7d/QK7xAhnHHZnBY5xS795DxfPSodyYEev9CEC25zFU/WjuRhY6Jq/9vlqxS5HuNfb1GLm9e7TO14zwi3e9N8eoH0gc6JP5HIfKlxm5rfw/IVTBAKyiyXQctmrSyorJp42Oq+fbZw0pazluQ71j3wo2nuw88AVpEe/WYQdrF4CzKJIrGtsMuCE0bJXyUOOPIVre85mmNkaWDIGVOcnnu7Rl3g9djW18bvpvtsKe+Rz+I3ysg3tc/M0Qri9yF5tswrxfuhEx/lZfahJH6r0LOz2a1hmPOvFfoIGsDmGehJrhfK1TdJlI2NxCXbOXwOrcIeiAlpx8lYiOzUbyIM+QLSDbIPgYrxFJVRbT5i2IcxyJgeW+jbu16s2UiQJ9MtVWlxF7CuTUJM6QldWdYBQzEIzGWTKvrpUZQdDe5M00f1I3eyZ1Xv18CrVSdMhwYXG6+5meWp9iWOMd3rEmnf8CwP2gP0/PqEBnKQ/4+ZKUgA5XvXrryQtBKrqV+9lMNacCM44xS+RkhfhdKZBgO4yCzUlQ1fD2QXtNbzYkaKRVfKviPuEEkNH12FZY2+qUUfL5lNZ4iDs55pZpAWdUiz/yHWB6BiZgSfkv+jeDv4IatdS X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR04MB4723.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(376002)(366004)(136003)(346002)(396003)(39830400003)(451199021)(86362001)(36756003)(38100700002)(38350700002)(7416002)(5660300002)(4326008)(66946007)(316002)(66476007)(44832011)(8936002)(41300700001)(66556008)(8676002)(2906002)(2616005)(6666004)(52116002)(478600001)(6486002)(6506007)(1076003)(26005)(107886003)(6512007)(186003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: ijhI6mvsm7jz8ftjOh6UmrsHXhA9cNSBAw2tykjpxwCAUvORiv9kMFjbIN81P1X6ql03Ca7k2KIBbrWgLbzyf5vzwStx78teNw705K74kGDFW+wbfS/OZXLw6zNYL5jkTDL34v+js/7be0+nTYT74ZxFsrn95WbWOP8S+0+3EiDWMIMIJg0kMINtLVxD6PJR5iA11Rxaef2Iylfa/pWIK/+vguKhf/XOH+MsgqlOgMlpJVe4Jx3AU3EAwvy/vNXTcvz6IGFs6Ay1Gi9OCMsHfJFec5MGP4619XBVJX7O+DSzZsA9NSN0uv3sxTShxZKXmcysFehNBZLkPpd8cQelEFbyuh3nQlIZpJYsyt2niagNPBiWj0Mjl/072zflcyhgqcxbYHRrVZvLc5PvcESks+X3eN9k6cRbJ/PUrvC+7Tb4VUtDKa0SFrKUPdgC+SVUnRynzkZWQ592k/r38q8HcNFw3qAIvRvtR7neoFmWfg7wWvwD0NcGnY8t7tNe2GeTevmEM64RZgWeH6a9k4esQ4/x79IElHE+KpJHQZIvL3LAGOgpngUqFnWKqW3Yu2kXI4+CWYw2eHm0EcvQQIj3Qygfc+R1WX49+p6dGNSyldmi2UY2/FDVr1aRViAaMov0NH+vUlI77wXAUKncmAwPTOpKGSCjmnFDHXayxZALQpmKgG66AyDQF44bDQN1xglT1sNCHeqdZleyfk7feCh5cmqg9wrwerbfL2SnFVAyZeNv0yTMCEDQPCPEHDL3xzaqxlNwaKE+ZIFZtzxWsaalrkpeOh8yO9lcwFaxSz0R6p77UQ2xh5ytEnRQ7RSrXTfQE+TdMcNwt94CaPI/oMLdsPZxVgy2UZKBFHWGx4Sus7ULKfaDgvwWA91a7JefEejl7KN1eiICRAgNz+eNnQsfTfe2GXvVb4vcVqCKdMoRxO8nG2lfXImDxBVBHqEphsmaO6z4QZPl+IQ/e1+KP4fyTQ0R6WZQkfpga+r2XacPDpXlklEC00AyM0AOLifOzhPvShQZAlKkOmHuOHBvsc6aQ2LERa+2HWHqZT/HTbJ5Xb3QiKD366SWD5tbyJjodQxd5gef+NxtcG6mNM4wVGDsezGYc3FkOUDkoHIlDCTBkaw0lSv2WhKvl673sY8pgx/xii2k6y8qwRGar+pZ10SqDGqo9GDWAaRADLv/AE9RHqvjRk7WAIyMBAeqKQY9OfMGwvjW50fTOuZstvNEKbWm+2xJd1lZIJCejD5W6I0Tzvfqqi7OSU4hPY5UYgw69WbVMP+iu8ZfyE9/1ipP3lLEiwsvM21YDS/PGcJf/HkNq9gFV1OWtPJ/VJFUbKbZAjUm5wn3z96qZB9SIPV9CIqQvPb1Ch1e8C2FdWWuj+kiJH6Bl0jE5UAUhZwsFgJ9BKLIqvnfO4TAV7HG8cIxDlbcNWufZVn1Mg2sIGJJONFTkvGS1a27MCUGLW+qvlGgAVmbUQicq9gbIlDkIGaER0r3K8g2VDrYnLnCuC2dAtnSi7zw1sDt+g1Z/T5M338DcNfYOTlS7NqCOUbhNHhwIFMe3ipT0Fv+4ZjmyzfE2MNaWGqO2LT9rQ0DrDj27Ni2wqjVL6M6+mCgiwP5NC5IOdtCHA== X-OriginatorOrg: solid-run.com X-MS-Exchange-CrossTenant-Network-Message-Id: f007aea1-0fad-4575-1c5f-08db497cfa7b X-MS-Exchange-CrossTenant-AuthSource: AM0PR04MB4723.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Apr 2023 13:15:33.1993 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a4a8aaf3-fd27-4e27-add2-604707ce5b82 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: +C1LX+kRjfaIiRFASgP9tiBeyF3EPG9bIx6W3Uppy/6IP2+IEE8lgDmU4P3FNvj3AuPRYpRRsVxB5Z/h38v6NPZ4FmIinr7oH2pX24cpeAk= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR04MB7142 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1764610314982928339?= X-GMAIL-MSGID: =?utf-8?q?1764610314982928339?= Stop ethtool from resizing a TX/RX ring to size less than MAX_SKB_FRAGS + 2, if the ring was initialized with a bigger size. We cannot convert a "normal" ring to a "small" ring in runtime. Signed-off-by: Alvaro Karsz --- drivers/net/virtio_net.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index b4441d63890..b8238eaa1e4 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -2071,6 +2071,12 @@ static int virtnet_rx_resize(struct virtnet_info *vi, bool running = netif_running(vi->dev); int err, qindex; + /* We cannot convert a ring to a small vring */ + if (!vi->svring && IS_SMALL_VRING(ring_num)) { + netdev_err(vi->dev, "resize rx fail: size is too small..\n"); + return -EINVAL; + } + qindex = rq - vi->rq; if (running) @@ -2097,6 +2103,12 @@ static int virtnet_tx_resize(struct virtnet_info *vi, qindex = sq - vi->sq; + /* We cannot convert a ring to a small vring */ + if (!vi->svring && IS_SMALL_VRING(ring_num)) { + netdev_err(vi->dev, "resize tx fail: size is too small..\n"); + return -EINVAL; + } + if (running) virtnet_napi_tx_disable(&sq->napi);