From patchwork Thu Feb 29 21:22:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ziwei Xiao X-Patchwork-Id: 208527 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2097:b0:108:e6aa:91d0 with SMTP id gs23csp689412dyb; Thu, 29 Feb 2024 13:24:28 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCXe2pp7jmwf3gOTQVosSZT8TY5znZ3J4mzMnkqTQgdxRodzDXJqVzkqMJvIweusCZY+aE7LlrxYWNoqcYLij9hdjzBETQ== X-Google-Smtp-Source: AGHT+IFJlHx318gCEsAFb8OvWApomW7YnjDt6twm5NsrkUwncM8hLzCFIOwcY44QQvlOURsP325f X-Received: by 2002:a05:6402:a53:b0:565:df8c:6a27 with SMTP id bt19-20020a0564020a5300b00565df8c6a27mr91781edb.20.1709241868786; Thu, 29 Feb 2024 13:24:28 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1709241868; cv=pass; d=google.com; s=arc-20160816; b=KSgut0iXG5SjfzIJ5la1N8VjpThp5oXnf5zxz9bTiC2vkBr6/R6ZaxejoQvujACbj4 ME8IEasLWj2TxIe40yGFqJvfzwhXgAPE3pFw/mn7tuof+u0IsAzzq/Oo+4W1hteM+9Dy 5tIDcQL0sKI4kiImeJHGqU2orkGdKXDznUUAKF8X+21zrScbgdVfjsG2emRW/v5am/GY g1Czy0qgf3glBjS6RkL/Z7qxUR4Q4vyurDvNsuUgSahrEvKYTvE9O4Ms+tx9ul5+UJYd q9TRoUxHGtLdSLEn4wpPJK18AjyzKUA1vH6mhFbWUNwgREKSnZDamj4i744hlpTnFBpu rjoQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:from:subject:message-id:references:mime-version :list-unsubscribe:list-subscribe:list-id:precedence:in-reply-to:date :dkim-signature; bh=v8qmDvwC15IGXRa5Pi6idDC6aBrAS3VuKlnExrEFlW8=; fh=bwjWgx7We1gJosvGrUhVuqmcUnsMwQ9ToZzKT7TzpR0=; b=HO71+W2NS/QlcPD/FzIwAy6lNTbjWCA43NkRzqznZvz+aBoIvn/0DElv+uidFDVRVy rXcdVXMt99EM+Yzz/hwnO6aIok8E60+dmdg1S7cZV8afeVCNEsQMhLjZJfCXwxCJYRs1 IvHBjyiDrEzr+y1TJfbPabJEf063Ix9AAAX56Uwbm0eu/IrvEoHr98PGmPFJCujMtdRj Jq0vE3jPDW1ARVrb3sBDLoRFdvQsdeYWWF1XU42aIAC0M+tbvQKCqqnfxJNC5d1ORIBv WrF1QSp6lBiN9fio0BDdd9JHgkjnkM2e8ov4lSl9YevnDlUSYGlKitFW+m0mO14qc94i qjlQ==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=oDACDZVG; arc=pass (i=1 spf=pass spfdomain=flex--ziweixiao.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-87575-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-87575-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id v1-20020aa7d9c1000000b00564e0137b02si880695eds.483.2024.02.29.13.24.28 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Feb 2024 13:24:28 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-87575-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=oDACDZVG; arc=pass (i=1 spf=pass spfdomain=flex--ziweixiao.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-87575-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-87575-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 377451F26050 for ; Thu, 29 Feb 2024 21:24:28 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 423F674BE6; Thu, 29 Feb 2024 21:23:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="oDACDZVG" Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 12FDE16FF56 for ; Thu, 29 Feb 2024 21:23:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709241824; cv=none; b=c+V4gHZTQbDYXEAR13Qi3VpCKaksoCuBc+SafBkrH3XXjcOPeG+muQ/hPmDwid9PRchvD6VNukPTtawndCu02jGho+GUbGOb8sqwlv+rjP4oCyuMybh+1hSfLgG8Fard8HwvWNkUYwEzcbb0QvSIrK51A6bNf+p/Eve1WKfXP/U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709241824; c=relaxed/simple; bh=m0DOK2OvkQRSX7AzySErtR+aXRrKWap5v5cgIEElomo=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=RlO2MJXQOKevI3pRgZp79JQJFn6eYVLnPi84hk/nIKUnlRLIkev+rkgjm2nhpXU+EDnV26lyAqI7eP1ERNSkO/Tt1vp6CM86U2r7GaaweqHSK2zBYadXdJ/+rlNHuQVFAKscOp2OtXFTsc7F/t1+st0LcB4ceeYg5EXAgaOFRXU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ziweixiao.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=oDACDZVG; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ziweixiao.bounces.google.com Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2995baae8b4so1211264a91.0 for ; Thu, 29 Feb 2024 13:23:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1709241822; x=1709846622; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=v8qmDvwC15IGXRa5Pi6idDC6aBrAS3VuKlnExrEFlW8=; b=oDACDZVGHW5VvBQJLkOBzD83TYLzSmztEQQywzt6aY5onBYIGimJiA38+Tc35XUOvI aBB8o5XXY8blk+9lJo71dah9rFlMVWFz6JP5fNh5BqYcnFO7PLr3Kt1fb7mSl+kIF4zf ADhdbZjPgZo6OYsVpMNkOSpK10Fyiqw1LIJJRTeSKBaDP7out5QvG+zHh/MdfLWaCQak BOwNmDRvgdYlGdYzdwTUk9Cz3SwWNY+E/Jnn15D197OPUxm878llqlYIpNXUYipvhyPt kIr2kYfJC2neW30y+cwtx+cson96GKQfJVvt5vFVhpQJdvSWK1szSi7GjLUGMyzubtLu thqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709241822; x=1709846622; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=v8qmDvwC15IGXRa5Pi6idDC6aBrAS3VuKlnExrEFlW8=; b=GYnJhlQQEqjvmxabTa92cXlJP6AILc8tfpl3JK2wPdqFWIb1jBks6V4Gpr1DP6MK7R /v3SCdvOnXGjOwnHf7n4zEaZZCFgXeBOPUKN3mQemQrFVQSntL8+4LkjBcFdpngYawTE do+ivzILVsBsIIseK90JCc+T3s/kk/3sTtpEbU72IcYBTbZ0Vx6MSkX4PXiTUkCMNBmG 2M40oDkguGkaVPmL0lfdII27J8hTdS64LGpFM1vZ7o2wOq2LQwT6uJ2vdc/Lj0ch2aV/ KFeFkBgmFzSyrNWArIru9tbpxYc6D8BoYXpBCFKhmjTuYi1frUEE6fusmV8wbCiuV1tI wP9A== X-Forwarded-Encrypted: i=1; AJvYcCWOvZ7/jvCW0fTZ+2KVLJwM2YxeGd/wN2Atgee7HkeABo3mqS072sg0gL6gFjIM8HuvKYVeIjNR4Yvhjp9KMaB4JwxKMLYC9sZ+/EdN X-Gm-Message-State: AOJu0Ywtcg0VnMWCDee/v9pOXNAtiHVuBk1IreXQZGUNBtrkGWy9gSsU O17Um/U47rtj5c4r2bhmYk8ki/qOSALovOxB8WTtaSjwhErzMcHAJfYhiDK9iCDAgOG5fqhFl57 2iT+mQGxGv9hi2w== X-Received: from ziweixiao.sea.corp.google.com ([2620:15c:11c:202:43b5:fdf3:8395:a655]) (user=ziweixiao job=sendgmr) by 2002:a17:90b:38c7:b0:299:58b0:a4be with SMTP id nn7-20020a17090b38c700b0029958b0a4bemr626pjb.8.1709241822240; Thu, 29 Feb 2024 13:23:42 -0800 (PST) Date: Thu, 29 Feb 2024 13:22:34 -0800 In-Reply-To: <20240229212236.3152897-1-ziweixiao@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240229212236.3152897-1-ziweixiao@google.com> X-Mailer: git-send-email 2.44.0.rc1.240.g4c46232300-goog Message-ID: <20240229212236.3152897-2-ziweixiao@google.com> Subject: [PATCH net-next 1/3] gve: Add header split device option From: Ziwei Xiao To: netdev@vger.kernel.org Cc: jeroendb@google.com, pkaligineedi@google.com, shailend@google.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, willemb@google.com, rushilg@google.com, jfraker@google.com, jrkim@google.com, hramamurthy@google.com, ziweixiao@google.com, horms@kernel.org, linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1792270001733082306 X-GMAIL-MSGID: 1792270001733082306 From: Jeroen de Borst To enable header split via ethtool, we first need to query the device to get the max rx buffer size and header buffer size. Add a device option to get these values and store them in the driver. If the header buffer size received from the device is non-zero, it means header split is supported in the device. Currently the max rx buffer size will only be used when header split is enabled which will set the data_buffer_size_dqo to be the max rx buffer size. Also change the data_buffer_size_dqo from int to u16 since we are modifying it and making it to be consistent with max_rx_buffer_size. Co-developed-by: Ziwei Xiao Signed-off-by: Ziwei Xiao Signed-off-by: Jeroen de Borst Reviewed-by: Praveen Kaligineedi Reviewed-by: Harshitha Ramamurthy Reviewed-by: Willem de Bruijn --- drivers/net/ethernet/google/gve/gve.h | 9 +++- drivers/net/ethernet/google/gve/gve_adminq.c | 47 +++++++++++++++++--- drivers/net/ethernet/google/gve/gve_adminq.h | 20 ++++++++- drivers/net/ethernet/google/gve/gve_main.c | 8 +--- drivers/net/ethernet/google/gve/gve_rx_dqo.c | 2 +- 5 files changed, 70 insertions(+), 16 deletions(-) diff --git a/drivers/net/ethernet/google/gve/gve.h b/drivers/net/ethernet/google/gve/gve.h index fd290f3ad6ec..5305404516fc 100644 --- a/drivers/net/ethernet/google/gve/gve.h +++ b/drivers/net/ethernet/google/gve/gve.h @@ -51,12 +51,16 @@ #define GVE_DEFAULT_RX_BUFFER_SIZE 2048 +#define GVE_MAX_RX_BUFFER_SIZE 4096 + #define GVE_DEFAULT_RX_BUFFER_OFFSET 2048 #define GVE_XDP_ACTIONS 5 #define GVE_GQ_TX_MIN_PKT_DESC_BYTES 182 +#define GVE_DEFAULT_HEADER_BUFFER_SIZE 128 + #define DQO_QPL_DEFAULT_TX_PAGES 512 #define DQO_QPL_DEFAULT_RX_PAGES 2048 @@ -778,13 +782,16 @@ struct gve_priv { struct gve_ptype_lut *ptype_lut_dqo; /* Must be a power of two. */ - int data_buffer_size_dqo; + u16 data_buffer_size_dqo; + u16 max_rx_buffer_size; /* device limit */ enum gve_queue_format queue_format; /* Interrupt coalescing settings */ u32 tx_coalesce_usecs; u32 rx_coalesce_usecs; + + u16 header_buf_size; /* device configured, header-split supported if non-zero */ }; enum gve_service_task_flags_bit { diff --git a/drivers/net/ethernet/google/gve/gve_adminq.c b/drivers/net/ethernet/google/gve/gve_adminq.c index 12fbd723ecc6..e2c27bbb56e6 100644 --- a/drivers/net/ethernet/google/gve/gve_adminq.c +++ b/drivers/net/ethernet/google/gve/gve_adminq.c @@ -40,7 +40,8 @@ void gve_parse_device_option(struct gve_priv *priv, struct gve_device_option_gqi_qpl **dev_op_gqi_qpl, struct gve_device_option_dqo_rda **dev_op_dqo_rda, struct gve_device_option_jumbo_frames **dev_op_jumbo_frames, - struct gve_device_option_dqo_qpl **dev_op_dqo_qpl) + struct gve_device_option_dqo_qpl **dev_op_dqo_qpl, + struct gve_device_option_buffer_sizes **dev_op_buffer_sizes) { u32 req_feat_mask = be32_to_cpu(option->required_features_mask); u16 option_length = be16_to_cpu(option->option_length); @@ -147,6 +148,23 @@ void gve_parse_device_option(struct gve_priv *priv, } *dev_op_jumbo_frames = (void *)(option + 1); break; + case GVE_DEV_OPT_ID_BUFFER_SIZES: + if (option_length < sizeof(**dev_op_buffer_sizes) || + req_feat_mask != GVE_DEV_OPT_REQ_FEAT_MASK_BUFFER_SIZES) { + dev_warn(&priv->pdev->dev, GVE_DEVICE_OPTION_ERROR_FMT, + "Buffer Sizes", + (int)sizeof(**dev_op_buffer_sizes), + GVE_DEV_OPT_REQ_FEAT_MASK_BUFFER_SIZES, + option_length, req_feat_mask); + break; + } + + if (option_length > sizeof(**dev_op_buffer_sizes)) + dev_warn(&priv->pdev->dev, + GVE_DEVICE_OPTION_TOO_BIG_FMT, + "Buffer Sizes"); + *dev_op_buffer_sizes = (void *)(option + 1); + break; default: /* If we don't recognize the option just continue * without doing anything. @@ -164,7 +182,8 @@ gve_process_device_options(struct gve_priv *priv, struct gve_device_option_gqi_qpl **dev_op_gqi_qpl, struct gve_device_option_dqo_rda **dev_op_dqo_rda, struct gve_device_option_jumbo_frames **dev_op_jumbo_frames, - struct gve_device_option_dqo_qpl **dev_op_dqo_qpl) + struct gve_device_option_dqo_qpl **dev_op_dqo_qpl, + struct gve_device_option_buffer_sizes **dev_op_buffer_sizes) { const int num_options = be16_to_cpu(descriptor->num_device_options); struct gve_device_option *dev_opt; @@ -185,7 +204,7 @@ gve_process_device_options(struct gve_priv *priv, gve_parse_device_option(priv, descriptor, dev_opt, dev_op_gqi_rda, dev_op_gqi_qpl, dev_op_dqo_rda, dev_op_jumbo_frames, - dev_op_dqo_qpl); + dev_op_dqo_qpl, dev_op_buffer_sizes); dev_opt = next_opt; } @@ -755,7 +774,9 @@ static void gve_enable_supported_features(struct gve_priv *priv, const struct gve_device_option_jumbo_frames *dev_op_jumbo_frames, const struct gve_device_option_dqo_qpl - *dev_op_dqo_qpl) + *dev_op_dqo_qpl, + const struct gve_device_option_buffer_sizes + *dev_op_buffer_sizes) { /* Before control reaches this point, the page-size-capped max MTU from * the gve_device_descriptor field has already been stored in @@ -779,10 +800,22 @@ static void gve_enable_supported_features(struct gve_priv *priv, if (priv->rx_pages_per_qpl == 0) priv->rx_pages_per_qpl = DQO_QPL_DEFAULT_RX_PAGES; } + + if (dev_op_buffer_sizes && + (supported_features_mask & GVE_SUP_BUFFER_SIZES_MASK)) { + priv->max_rx_buffer_size = + be16_to_cpu(dev_op_buffer_sizes->packet_buffer_size); + priv->header_buf_size = + be16_to_cpu(dev_op_buffer_sizes->header_buffer_size); + dev_info(&priv->pdev->dev, + "BUFFER SIZES device option enabled with max_rx_buffer_size of %u, header_buf_size of %u.\n", + priv->max_rx_buffer_size, priv->header_buf_size); + } } int gve_adminq_describe_device(struct gve_priv *priv) { + struct gve_device_option_buffer_sizes *dev_op_buffer_sizes = NULL; struct gve_device_option_jumbo_frames *dev_op_jumbo_frames = NULL; struct gve_device_option_gqi_rda *dev_op_gqi_rda = NULL; struct gve_device_option_gqi_qpl *dev_op_gqi_qpl = NULL; @@ -816,7 +849,8 @@ int gve_adminq_describe_device(struct gve_priv *priv) err = gve_process_device_options(priv, descriptor, &dev_op_gqi_rda, &dev_op_gqi_qpl, &dev_op_dqo_rda, &dev_op_jumbo_frames, - &dev_op_dqo_qpl); + &dev_op_dqo_qpl, + &dev_op_buffer_sizes); if (err) goto free_device_descriptor; @@ -885,7 +919,8 @@ int gve_adminq_describe_device(struct gve_priv *priv) priv->default_num_queues = be16_to_cpu(descriptor->default_num_queues); gve_enable_supported_features(priv, supported_features_mask, - dev_op_jumbo_frames, dev_op_dqo_qpl); + dev_op_jumbo_frames, dev_op_dqo_qpl, + dev_op_buffer_sizes); free_device_descriptor: dma_pool_free(priv->adminq_pool, descriptor, descriptor_bus); diff --git a/drivers/net/ethernet/google/gve/gve_adminq.h b/drivers/net/ethernet/google/gve/gve_adminq.h index 5865ccdccbd0..5ac972e45ff8 100644 --- a/drivers/net/ethernet/google/gve/gve_adminq.h +++ b/drivers/net/ethernet/google/gve/gve_adminq.h @@ -125,6 +125,15 @@ struct gve_device_option_jumbo_frames { static_assert(sizeof(struct gve_device_option_jumbo_frames) == 8); +struct gve_device_option_buffer_sizes { + /* GVE_SUP_BUFFER_SIZES_MASK bit should be set */ + __be32 supported_features_mask; + __be16 packet_buffer_size; + __be16 header_buffer_size; +}; + +static_assert(sizeof(struct gve_device_option_buffer_sizes) == 8); + /* Terminology: * * RDA - Raw DMA Addressing - Buffers associated with SKBs are directly DMA @@ -140,6 +149,7 @@ enum gve_dev_opt_id { GVE_DEV_OPT_ID_DQO_RDA = 0x4, GVE_DEV_OPT_ID_DQO_QPL = 0x7, GVE_DEV_OPT_ID_JUMBO_FRAMES = 0x8, + GVE_DEV_OPT_ID_BUFFER_SIZES = 0xa, }; enum gve_dev_opt_req_feat_mask { @@ -149,10 +159,12 @@ enum gve_dev_opt_req_feat_mask { GVE_DEV_OPT_REQ_FEAT_MASK_DQO_RDA = 0x0, GVE_DEV_OPT_REQ_FEAT_MASK_JUMBO_FRAMES = 0x0, GVE_DEV_OPT_REQ_FEAT_MASK_DQO_QPL = 0x0, + GVE_DEV_OPT_REQ_FEAT_MASK_BUFFER_SIZES = 0x0, }; enum gve_sup_feature_mask { GVE_SUP_JUMBO_FRAMES_MASK = 1 << 2, + GVE_SUP_BUFFER_SIZES_MASK = 1 << 4, }; #define GVE_DEV_OPT_LEN_GQI_RAW_ADDRESSING 0x0 @@ -165,6 +177,7 @@ enum gve_driver_capbility { gve_driver_capability_dqo_qpl = 2, /* reserved for future use */ gve_driver_capability_dqo_rda = 3, gve_driver_capability_alt_miss_compl = 4, + gve_driver_capability_flexible_buffer_size = 5, }; #define GVE_CAP1(a) BIT((int)a) @@ -176,7 +189,8 @@ enum gve_driver_capbility { (GVE_CAP1(gve_driver_capability_gqi_qpl) | \ GVE_CAP1(gve_driver_capability_gqi_rda) | \ GVE_CAP1(gve_driver_capability_dqo_rda) | \ - GVE_CAP1(gve_driver_capability_alt_miss_compl)) + GVE_CAP1(gve_driver_capability_alt_miss_compl) | \ + GVE_CAP1(gve_driver_capability_flexible_buffer_size)) #define GVE_DRIVER_CAPABILITY_FLAGS2 0x0 #define GVE_DRIVER_CAPABILITY_FLAGS3 0x0 @@ -260,7 +274,9 @@ struct gve_adminq_create_rx_queue { __be16 packet_buffer_size; __be16 rx_buff_ring_size; u8 enable_rsc; - u8 padding[5]; + u8 padding1; + __be16 header_buffer_size; + u8 padding2[2]; }; static_assert(sizeof(struct gve_adminq_create_rx_queue) == 56); diff --git a/drivers/net/ethernet/google/gve/gve_main.c b/drivers/net/ethernet/google/gve/gve_main.c index db6d9ae7cd78..02d12aa50885 100644 --- a/drivers/net/ethernet/google/gve/gve_main.c +++ b/drivers/net/ethernet/google/gve/gve_main.c @@ -1448,12 +1448,6 @@ static int gve_queues_start(struct gve_priv *priv, if (err) goto reset; - if (!gve_is_gqi(priv)) { - /* Hard code this for now. This may be tuned in the future for - * performance. - */ - priv->data_buffer_size_dqo = GVE_DEFAULT_RX_BUFFER_SIZE; - } err = gve_create_rings(priv); if (err) goto reset; @@ -2511,6 +2505,8 @@ static int gve_probe(struct pci_dev *pdev, const struct pci_device_id *ent) priv->service_task_flags = 0x0; priv->state_flags = 0x0; priv->ethtool_flags = 0x0; + priv->data_buffer_size_dqo = GVE_DEFAULT_RX_BUFFER_SIZE; + priv->max_rx_buffer_size = GVE_DEFAULT_RX_BUFFER_SIZE; gve_set_probe_in_progress(priv); priv->gve_wq = alloc_ordered_workqueue("gve", 0); diff --git a/drivers/net/ethernet/google/gve/gve_rx_dqo.c b/drivers/net/ethernet/google/gve/gve_rx_dqo.c index 8e6aeb5b3ed4..c3ce819ee5ab 100644 --- a/drivers/net/ethernet/google/gve/gve_rx_dqo.c +++ b/drivers/net/ethernet/google/gve/gve_rx_dqo.c @@ -458,7 +458,7 @@ void gve_rx_post_buffers_dqo(struct gve_rx_ring *rx) static void gve_try_recycle_buf(struct gve_priv *priv, struct gve_rx_ring *rx, struct gve_rx_buf_state_dqo *buf_state) { - const int data_buffer_size = priv->data_buffer_size_dqo; + const u16 data_buffer_size = priv->data_buffer_size_dqo; int pagecount; /* Can't reuse if we only fit one buffer per page */ From patchwork Thu Feb 29 21:22:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ziwei Xiao X-Patchwork-Id: 208528 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2097:b0:108:e6aa:91d0 with SMTP id gs23csp689651dyb; Thu, 29 Feb 2024 13:25:00 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCU+BLJj0wpOHJHQp2OQzvSVCzru8jAjl4KfFT9CLhfFJVsRBj6diDnZ61lxBZwu5O/najrLtCPlpieRQVJAIyWoeNabKg== X-Google-Smtp-Source: AGHT+IFbUiK/LeAUdelOV8EcLHi9eGBPNpG6PJdEX2O2ZTCGarqB5VTkOy4n8VLCO37U0EatGYxI X-Received: by 2002:ac2:4c28:0:b0:513:238f:633a with SMTP id u8-20020ac24c28000000b00513238f633amr2219165lfq.46.1709241899833; Thu, 29 Feb 2024 13:24:59 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1709241899; cv=pass; d=google.com; s=arc-20160816; b=zCILU9c5wNulLCWlrDU29o3uMOurMjRe/4gXH9jSheQ+aJ8h8Ho/iU+sFSm863W9e7 Rf5dU6KgXlYQaNytNAv/2h73z8XxLMIo88MdBMhJLZVcwoDs+P9UCWj6EkYJojnw3dVh IoyU97y6cld1V0f3iEdkkGwLhE8ZNC98SHyiTtvQlDnDH5IgAT8hd84EWyjpV+OLORV5 +FxFQqDrPmPj1/ZCU/NfpBsWGph12hUGK7Vd5HZ5ptEv/vVNvNb9CJ1+XnwZ+FyaKER4 7kg9HkW7eWGkB0bKhCSGPtBwKwn/r0W26BfYSwG4XDrczw8VMSy0Eozwv7KuqtagPEZM XrRg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:from:subject:message-id:references:mime-version :list-unsubscribe:list-subscribe:list-id:precedence:in-reply-to:date :dkim-signature; bh=f9FlKgN0QCZV27xvHjQB5sdwQFtHd/7HtCh3o9a31n0=; fh=rf9pp3iJgm5BU8716Qw1CUFfmEBvrfF6H7NHQyecGRw=; b=HIbSxyXa2Z6JBA1vZovD1ugrQUuRMcuJos25Q1I0rPtJv8CVSCJnLoegboCsvdCUgu XARSyu6pz5e25Jzs420W0GjRSv7LiljZfrf67VUM5TTgG7tSqX4tB3aiRfO1GXsDbB+t NCZDKTkXu1VbfFPQk/7qeusmDFwTdmB62CKyfiKkd2gfjAWcOOSUtQyUnSqlo239t1v0 QffPVUB0trI7BOxfMgIuRSJfzRY+vFUu8GC912g7t8gMUUIPLgEkCCtA0BHLBgcPE8S/ fGQky37NQrZrqxrYKwb92HxrE8KnBfiNbKn3FhwJ6RYibqGK7pqjUDtHjVyk+PGFdWdV 120g==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=RfgaMYNw; arc=pass (i=1 spf=pass spfdomain=flex--ziweixiao.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-87576-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-87576-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id b15-20020aa7cd0f000000b0056452270979si868807edw.561.2024.02.29.13.24.59 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Feb 2024 13:24:59 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-87576-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=RfgaMYNw; arc=pass (i=1 spf=pass spfdomain=flex--ziweixiao.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-87576-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-87576-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 386C61F2627D for ; Thu, 29 Feb 2024 21:24:59 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 3409D6D525; Thu, 29 Feb 2024 21:23:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="RfgaMYNw" Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F01376D50E for ; Thu, 29 Feb 2024 21:23:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709241832; cv=none; b=Kjsrb+h7d+I8QndbfyOW13DI6sP3BtDufy6OqTGDhFlrAYpBP+/EMvzw1g22tHTjhm7KbJGwNRUi3c7sPw49ggKu+5i5gr29VBzEF5JPrIwAb0OnPa0r1ulHyXNkjKhXNMLG6R6g4BP6B9YACz6Ajw66eg65ulNaHo1FfgcTLUo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709241832; c=relaxed/simple; bh=rvYJHjF3TJa8UGPtWynrnvN5ETRyBHi+0lL/4xDeqKQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=WhwlCTPZCaZlStzFnkwmmyv3RK8ID9mumh+K4eRfIGQU7689fGTpxl/p6opKUBB2dZMBAdKxTACzspIszaiReOwcH5gZlMJ0iAxNCHA8gU4fjqnu1axf5UqbagUTMjnZV8yE7pXxgvdc+zD1VLWhCYlY4sAAmek2Q5hMtAwQGdM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ziweixiao.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=RfgaMYNw; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ziweixiao.bounces.google.com Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-299cae4f36bso1140245a91.1 for ; Thu, 29 Feb 2024 13:23:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1709241830; x=1709846630; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=f9FlKgN0QCZV27xvHjQB5sdwQFtHd/7HtCh3o9a31n0=; b=RfgaMYNwHx8myVSYWR8oZD/owVXz4iw8k9yGvIl5bL3Kadt2Q36HnbEYgWQr0Vlj6N 4+jNdvaly+T/mel2bwB8lhl8qPhzZtjg7JQnh39k7qsozMJtaO+wzOVCHuuTQzske4lT HlTKsEBf+IuwnTalzKGVwAH6bnq44mYh2DZwc5oI/P6Z482oD2mVCmsOuWpRIyza65dn FjEI3JGxA8mQ6+fcWqwmzvkDGNxLGursT+PiDG+iYIcJYB6GHjWsFDSFghdf6AKhCoHM 36v6EqKq3lwqmdwIvoqpzEN8VLD8ApgSczwK8MOYt0hqLioEzXOTpJwwUb4sRPjCba+p PKiw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709241830; x=1709846630; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=f9FlKgN0QCZV27xvHjQB5sdwQFtHd/7HtCh3o9a31n0=; b=gicbsK7ZNPu0+avlCQ2IXsNY/WcKaomkqt/xc1QOUPjrwmNQfIlRCmtMCOTrxTB35P H53hQhitGPcTQm6WU/cIhHLT7ajHHvZN3e5Oz+bqJe9PCZqeNF9MMUfgjOqpIX0/CYg8 dRnHb2mmu032UBb6SDSmBbvDvkDfwxUfY7RTcMXElBh1PhFCTdp8JreO3ziaWPQbtO0B 2IXP5btn2ChAbcUp4Essq5IlN6Bu2O8BKSbzCeS1pjnhFk5FKAzCwxTglQw8PMjMrAKD nytdWZGMjmK0R3rCqSv+DRYn50k+ueuJd4fwxpu+skVgAY8bLvAo1pATUEYBuAk00Jxq vovQ== X-Forwarded-Encrypted: i=1; AJvYcCWifgg8ZJVY8OH8NFHVkH8KKxqvW5WpeeDxQ2N404Ega7vj4yNLywlCg3jWIoISU6Cg3gblB91+7RLc+fbpm+sneJjbtD+kMzQ9lte0 X-Gm-Message-State: AOJu0Yyx/dam3NpR/6LiVAQ9gtn+Vnnqrf+RoWkdWnNKMAEqWT3XQm+o RNR7cCFb5nl1vqpVpBtPJHc6MAOC7kvvS7wbLfHhFvwPFLrW1eRArNcILfjkYcFquekxudBkUmH Gxn5RjhMzv0S9Dw== X-Received: from ziweixiao.sea.corp.google.com ([2620:15c:11c:202:43b5:fdf3:8395:a655]) (user=ziweixiao job=sendgmr) by 2002:a17:90a:b008:b0:299:dbe9:353b with SMTP id x8-20020a17090ab00800b00299dbe9353bmr800pjq.9.1709241829335; Thu, 29 Feb 2024 13:23:49 -0800 (PST) Date: Thu, 29 Feb 2024 13:22:35 -0800 In-Reply-To: <20240229212236.3152897-1-ziweixiao@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240229212236.3152897-1-ziweixiao@google.com> X-Mailer: git-send-email 2.44.0.rc1.240.g4c46232300-goog Message-ID: <20240229212236.3152897-3-ziweixiao@google.com> Subject: [PATCH net-next 2/3] gve: Add header split data path From: Ziwei Xiao To: netdev@vger.kernel.org Cc: jeroendb@google.com, pkaligineedi@google.com, shailend@google.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, willemb@google.com, rushilg@google.com, jfraker@google.com, jrkim@google.com, hramamurthy@google.com, ziweixiao@google.com, horms@kernel.org, linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1792270034487358082 X-GMAIL-MSGID: 1792270034487358082 From: Jeroen de Borst Add header buffers and ethtool support to enable header split via the tcp-data-split flag in ethtool's ringparam config. A coherent dma memory is allocated for the header buffers. There is one header buffer per ring entry by calculating the offset to the header-buffers starting address. The header buffer is always copied directly into the skb and payload is always added as frags. When there is a header buffer overflow or the header length is 0, the driver places the whole unsplit packet in frags. When toggling header split, the driver will call gve_adjust_config to set its queues appropriately. If header split is enabled by the user and the max packet buffer size is no less than 4KB, driver will set the packet buffer size as 4KB to support TCP_ZEROCOPY_RECEIVE. Otherwise the driver will use the default 2KB as the packet buffer size. `ethtool -G tcp-data-split on/off` is the command to toggle header split. `ethtool -g ` will show the status of header split with the field of `tcp-data-split`. Co-developed-by: Ziwei Xiao Signed-off-by: Ziwei Xiao Signed-off-by: Jeroen de Borst Reviewed-by: Praveen Kaligineedi Reviewed-by: Harshitha Ramamurthy Reviewed-by: Willem de Bruijn --- drivers/net/ethernet/google/gve/gve.h | 14 +++++ drivers/net/ethernet/google/gve/gve_adminq.c | 3 + drivers/net/ethernet/google/gve/gve_ethtool.c | 26 ++++++++- drivers/net/ethernet/google/gve/gve_main.c | 57 +++++++++++++++++++ drivers/net/ethernet/google/gve/gve_rx_dqo.c | 57 ++++++++++++++++++- drivers/net/ethernet/google/gve/gve_utils.c | 17 ++++-- drivers/net/ethernet/google/gve/gve_utils.h | 3 + 7 files changed, 169 insertions(+), 8 deletions(-) diff --git a/drivers/net/ethernet/google/gve/gve.h b/drivers/net/ethernet/google/gve/gve.h index 5305404516fc..5f02b87d7fea 100644 --- a/drivers/net/ethernet/google/gve/gve.h +++ b/drivers/net/ethernet/google/gve/gve.h @@ -9,6 +9,7 @@ #include #include +#include #include #include #include @@ -154,6 +155,11 @@ struct gve_rx_compl_queue_dqo { u32 mask; /* Mask for indices to the size of the ring */ }; +struct gve_header_buf { + u8 *data; + dma_addr_t addr; +}; + /* Stores state for tracking buffers posted to HW */ struct gve_rx_buf_state_dqo { /* The page posted to HW. */ @@ -256,6 +262,9 @@ struct gve_rx_ring { /* track number of used buffers */ u16 used_buf_states_cnt; + + /* Address info of the buffers for header-split */ + struct gve_header_buf hdr_bufs; } dqo; }; @@ -668,6 +677,7 @@ struct gve_rx_alloc_rings_cfg { struct gve_qpl_config *qpl_cfg; u16 ring_size; + u16 packet_buffer_size; bool raw_addressing; bool enable_header_split; @@ -792,6 +802,7 @@ struct gve_priv { u32 rx_coalesce_usecs; u16 header_buf_size; /* device configured, header-split supported if non-zero */ + bool header_split_enabled; /* True if the header split is enabled by the user */ }; enum gve_service_task_flags_bit { @@ -1129,6 +1140,9 @@ void gve_rx_free_rings_gqi(struct gve_priv *priv, struct gve_rx_alloc_rings_cfg *cfg); void gve_rx_start_ring_gqi(struct gve_priv *priv, int idx); void gve_rx_stop_ring_gqi(struct gve_priv *priv, int idx); +u16 gve_get_pkt_buf_size(const struct gve_priv *priv, bool enable_hplit); +bool gve_header_split_supported(const struct gve_priv *priv); +int gve_set_hsplit_config(struct gve_priv *priv, u8 tcp_data_split); /* Reset */ void gve_schedule_reset(struct gve_priv *priv); int gve_reset(struct gve_priv *priv, bool attempt_teardown); diff --git a/drivers/net/ethernet/google/gve/gve_adminq.c b/drivers/net/ethernet/google/gve/gve_adminq.c index e2c27bbb56e6..ae12ac38e18b 100644 --- a/drivers/net/ethernet/google/gve/gve_adminq.c +++ b/drivers/net/ethernet/google/gve/gve_adminq.c @@ -659,6 +659,9 @@ static int gve_adminq_create_rx_queue(struct gve_priv *priv, u32 queue_index) cpu_to_be16(rx_buff_ring_entries); cmd.create_rx_queue.enable_rsc = !!(priv->dev->features & NETIF_F_LRO); + if (priv->header_split_enabled) + cmd.create_rx_queue.header_buffer_size = + cpu_to_be16(priv->header_buf_size); } return gve_adminq_issue_cmd(priv, &cmd); diff --git a/drivers/net/ethernet/google/gve/gve_ethtool.c b/drivers/net/ethernet/google/gve/gve_ethtool.c index e5397aa1e48f..13c2901968e0 100644 --- a/drivers/net/ethernet/google/gve/gve_ethtool.c +++ b/drivers/net/ethernet/google/gve/gve_ethtool.c @@ -4,7 +4,6 @@ * Copyright (C) 2015-2021 Google, Inc. */ -#include #include #include "gve.h" #include "gve_adminq.h" @@ -480,6 +479,29 @@ static void gve_get_ringparam(struct net_device *netdev, cmd->tx_max_pending = priv->tx_desc_cnt; cmd->rx_pending = priv->rx_desc_cnt; cmd->tx_pending = priv->tx_desc_cnt; + + if (!gve_header_split_supported(priv)) + kernel_cmd->tcp_data_split = ETHTOOL_TCP_DATA_SPLIT_UNKNOWN; + else if (priv->header_split_enabled) + kernel_cmd->tcp_data_split = ETHTOOL_TCP_DATA_SPLIT_ENABLED; + else + kernel_cmd->tcp_data_split = ETHTOOL_TCP_DATA_SPLIT_DISABLED; +} + +static int gve_set_ringparam(struct net_device *netdev, + struct ethtool_ringparam *cmd, + struct kernel_ethtool_ringparam *kernel_cmd, + struct netlink_ext_ack *extack) +{ + struct gve_priv *priv = netdev_priv(netdev); + + if (priv->tx_desc_cnt != cmd->tx_pending || + priv->rx_desc_cnt != cmd->rx_pending) { + dev_info(&priv->pdev->dev, "Modify ring size is not supported.\n"); + return -EOPNOTSUPP; + } + + return gve_set_hsplit_config(priv, kernel_cmd->tcp_data_split); } static int gve_user_reset(struct net_device *netdev, u32 *flags) @@ -655,6 +677,7 @@ static int gve_set_coalesce(struct net_device *netdev, const struct ethtool_ops gve_ethtool_ops = { .supported_coalesce_params = ETHTOOL_COALESCE_USECS, + .supported_ring_params = ETHTOOL_RING_USE_TCP_DATA_SPLIT, .get_drvinfo = gve_get_drvinfo, .get_strings = gve_get_strings, .get_sset_count = gve_get_sset_count, @@ -667,6 +690,7 @@ const struct ethtool_ops gve_ethtool_ops = { .get_coalesce = gve_get_coalesce, .set_coalesce = gve_set_coalesce, .get_ringparam = gve_get_ringparam, + .set_ringparam = gve_set_ringparam, .reset = gve_user_reset, .get_tunable = gve_get_tunable, .set_tunable = gve_set_tunable, diff --git a/drivers/net/ethernet/google/gve/gve_main.c b/drivers/net/ethernet/google/gve/gve_main.c index 02d12aa50885..7b89b66adb53 100644 --- a/drivers/net/ethernet/google/gve/gve_main.c +++ b/drivers/net/ethernet/google/gve/gve_main.c @@ -1307,9 +1307,13 @@ static void gve_rx_get_curr_alloc_cfg(struct gve_priv *priv, cfg->qcfg = &priv->rx_cfg; cfg->qcfg_tx = &priv->tx_cfg; cfg->raw_addressing = !gve_is_qpl(priv); + cfg->enable_header_split = priv->header_split_enabled; cfg->qpls = priv->qpls; cfg->qpl_cfg = &priv->qpl_cfg; cfg->ring_size = priv->rx_desc_cnt; + cfg->packet_buffer_size = gve_is_gqi(priv) ? + GVE_DEFAULT_RX_BUFFER_SIZE : + priv->data_buffer_size_dqo; cfg->rx = priv->rx; } @@ -1448,6 +1452,9 @@ static int gve_queues_start(struct gve_priv *priv, if (err) goto reset; + priv->header_split_enabled = rx_alloc_cfg->enable_header_split; + priv->data_buffer_size_dqo = rx_alloc_cfg->packet_buffer_size; + err = gve_create_rings(priv); if (err) goto reset; @@ -2059,6 +2066,56 @@ static void gve_tx_timeout(struct net_device *dev, unsigned int txqueue) priv->tx_timeo_cnt++; } +u16 gve_get_pkt_buf_size(const struct gve_priv *priv, bool enable_hsplit) +{ + if (enable_hsplit && priv->max_rx_buffer_size >= GVE_MAX_RX_BUFFER_SIZE) + return GVE_MAX_RX_BUFFER_SIZE; + else + return GVE_DEFAULT_RX_BUFFER_SIZE; +} + +/* header-split is not supported on non-DQO_RDA yet even if device advertises it */ +bool gve_header_split_supported(const struct gve_priv *priv) +{ + return priv->header_buf_size && priv->queue_format == GVE_DQO_RDA_FORMAT; +} + +int gve_set_hsplit_config(struct gve_priv *priv, u8 tcp_data_split) +{ + struct gve_tx_alloc_rings_cfg tx_alloc_cfg = {0}; + struct gve_rx_alloc_rings_cfg rx_alloc_cfg = {0}; + struct gve_qpls_alloc_cfg qpls_alloc_cfg = {0}; + bool enable_hdr_split; + int err = 0; + + if (tcp_data_split == ETHTOOL_TCP_DATA_SPLIT_UNKNOWN) + return 0; + + if (!gve_header_split_supported(priv)) { + dev_err(&priv->pdev->dev, "Header-split not supported\n"); + return -EOPNOTSUPP; + } + + if (tcp_data_split == ETHTOOL_TCP_DATA_SPLIT_ENABLED) + enable_hdr_split = true; + else + enable_hdr_split = false; + + if (enable_hdr_split == priv->header_split_enabled) + return 0; + + gve_get_curr_alloc_cfgs(priv, &qpls_alloc_cfg, + &tx_alloc_cfg, &rx_alloc_cfg); + + rx_alloc_cfg.enable_header_split = enable_hdr_split; + rx_alloc_cfg.packet_buffer_size = gve_get_pkt_buf_size(priv, enable_hdr_split); + + if (netif_running(priv->dev)) + err = gve_adjust_config(priv, &qpls_alloc_cfg, + &tx_alloc_cfg, &rx_alloc_cfg); + return err; +} + static int gve_set_features(struct net_device *netdev, netdev_features_t features) { diff --git a/drivers/net/ethernet/google/gve/gve_rx_dqo.c b/drivers/net/ethernet/google/gve/gve_rx_dqo.c index c3ce819ee5ab..a12d776d4385 100644 --- a/drivers/net/ethernet/google/gve/gve_rx_dqo.c +++ b/drivers/net/ethernet/google/gve/gve_rx_dqo.c @@ -199,6 +199,18 @@ static int gve_alloc_page_dqo(struct gve_rx_ring *rx, return 0; } +static void gve_rx_free_hdr_bufs(struct gve_priv *priv, struct gve_rx_ring *rx) +{ + struct device *hdev = &priv->pdev->dev; + int buf_count = rx->dqo.bufq.mask + 1; + + if (rx->dqo.hdr_bufs.data) { + dma_free_coherent(hdev, priv->header_buf_size * buf_count, + rx->dqo.hdr_bufs.data, rx->dqo.hdr_bufs.addr); + rx->dqo.hdr_bufs.data = NULL; + } +} + void gve_rx_stop_ring_dqo(struct gve_priv *priv, int idx) { int ntfy_idx = gve_rx_idx_to_ntfy(priv, idx); @@ -258,9 +270,24 @@ static void gve_rx_free_ring_dqo(struct gve_priv *priv, struct gve_rx_ring *rx, kvfree(rx->dqo.buf_states); rx->dqo.buf_states = NULL; + gve_rx_free_hdr_bufs(priv, rx); + netif_dbg(priv, drv, priv->dev, "freed rx ring %d\n", idx); } +static int gve_rx_alloc_hdr_bufs(struct gve_priv *priv, struct gve_rx_ring *rx) +{ + struct device *hdev = &priv->pdev->dev; + int buf_count = rx->dqo.bufq.mask + 1; + + rx->dqo.hdr_bufs.data = dma_alloc_coherent(hdev, priv->header_buf_size * buf_count, + &rx->dqo.hdr_bufs.addr, GFP_KERNEL); + if (!rx->dqo.hdr_bufs.data) + return -ENOMEM; + + return 0; +} + void gve_rx_start_ring_dqo(struct gve_priv *priv, int idx) { int ntfy_idx = gve_rx_idx_to_ntfy(priv, idx); @@ -302,6 +329,11 @@ static int gve_rx_alloc_ring_dqo(struct gve_priv *priv, if (!rx->dqo.buf_states) return -ENOMEM; + /* Allocate header buffers for header-split */ + if (cfg->enable_header_split) + if (gve_rx_alloc_hdr_bufs(priv, rx)) + goto err; + /* Set up linked list of buffer IDs */ for (i = 0; i < rx->dqo.num_buf_states - 1; i++) rx->dqo.buf_states[i].next = i + 1; @@ -443,6 +475,10 @@ void gve_rx_post_buffers_dqo(struct gve_rx_ring *rx) desc->buf_id = cpu_to_le16(buf_state - rx->dqo.buf_states); desc->buf_addr = cpu_to_le64(buf_state->addr + buf_state->page_info.page_offset); + if (rx->dqo.hdr_bufs.data) + desc->header_buf_addr = + cpu_to_le64(rx->dqo.hdr_bufs.addr + + priv->header_buf_size * bufq->tail); bufq->tail = (bufq->tail + 1) & bufq->mask; complq->num_free_slots--; @@ -645,13 +681,16 @@ static int gve_rx_append_frags(struct napi_struct *napi, */ static int gve_rx_dqo(struct napi_struct *napi, struct gve_rx_ring *rx, const struct gve_rx_compl_desc_dqo *compl_desc, - int queue_idx) + u32 desc_idx, int queue_idx) { const u16 buffer_id = le16_to_cpu(compl_desc->buf_id); + const bool hbo = compl_desc->header_buffer_overflow; const bool eop = compl_desc->end_of_packet != 0; + const bool hsplit = compl_desc->split_header; struct gve_rx_buf_state_dqo *buf_state; struct gve_priv *priv = rx->gve; u16 buf_len; + u16 hdr_len; if (unlikely(buffer_id >= rx->dqo.num_buf_states)) { net_err_ratelimited("%s: Invalid RX buffer_id=%u\n", @@ -672,12 +711,26 @@ static int gve_rx_dqo(struct napi_struct *napi, struct gve_rx_ring *rx, } buf_len = compl_desc->packet_len; + hdr_len = compl_desc->header_len; /* Page might have not been used for awhile and was likely last written * by a different thread. */ prefetch(buf_state->page_info.page); + /* Copy the header into the skb in the case of header split */ + if (hsplit) { + if (hdr_len && !hbo) { + rx->ctx.skb_head = gve_rx_copy_data(priv->dev, napi, + rx->dqo.hdr_bufs.data + + desc_idx * priv->header_buf_size, + hdr_len); + if (unlikely(!rx->ctx.skb_head)) + goto error; + rx->ctx.skb_tail = rx->ctx.skb_head; + } + } + /* Sync the portion of dma buffer for CPU to read. */ dma_sync_single_range_for_cpu(&priv->pdev->dev, buf_state->addr, buf_state->page_info.page_offset, @@ -820,7 +873,7 @@ int gve_rx_poll_dqo(struct gve_notify_block *block, int budget) /* Do not read data until we own the descriptor */ dma_rmb(); - err = gve_rx_dqo(napi, rx, compl_desc, rx->q_num); + err = gve_rx_dqo(napi, rx, compl_desc, complq->head, rx->q_num); if (err < 0) { gve_rx_free_skb(rx); u64_stats_update_begin(&rx->statss); diff --git a/drivers/net/ethernet/google/gve/gve_utils.c b/drivers/net/ethernet/google/gve/gve_utils.c index 535b1796b91d..2349750075a5 100644 --- a/drivers/net/ethernet/google/gve/gve_utils.c +++ b/drivers/net/ethernet/google/gve/gve_utils.c @@ -64,11 +64,9 @@ void gve_rx_add_to_block(struct gve_priv *priv, int queue_idx) rx->ntfy_id = ntfy_idx; } -struct sk_buff *gve_rx_copy(struct net_device *dev, struct napi_struct *napi, - struct gve_rx_slot_page_info *page_info, u16 len) +struct sk_buff *gve_rx_copy_data(struct net_device *dev, struct napi_struct *napi, + u8 *data, u16 len) { - void *va = page_info->page_address + page_info->page_offset + - page_info->pad; struct sk_buff *skb; skb = napi_alloc_skb(napi, len); @@ -76,12 +74,21 @@ struct sk_buff *gve_rx_copy(struct net_device *dev, struct napi_struct *napi, return NULL; __skb_put(skb, len); - skb_copy_to_linear_data_offset(skb, 0, va, len); + skb_copy_to_linear_data_offset(skb, 0, data, len); skb->protocol = eth_type_trans(skb, dev); return skb; } +struct sk_buff *gve_rx_copy(struct net_device *dev, struct napi_struct *napi, + struct gve_rx_slot_page_info *page_info, u16 len) +{ + void *va = page_info->page_address + page_info->page_offset + + page_info->pad; + + return gve_rx_copy_data(dev, napi, va, len); +} + void gve_dec_pagecnt_bias(struct gve_rx_slot_page_info *page_info) { page_info->pagecnt_bias--; diff --git a/drivers/net/ethernet/google/gve/gve_utils.h b/drivers/net/ethernet/google/gve/gve_utils.h index 277921a629f7..bf2e9a0adb36 100644 --- a/drivers/net/ethernet/google/gve/gve_utils.h +++ b/drivers/net/ethernet/google/gve/gve_utils.h @@ -19,6 +19,9 @@ bool gve_rx_was_added_to_block(struct gve_priv *priv, int queue_idx); void gve_rx_remove_from_block(struct gve_priv *priv, int queue_idx); void gve_rx_add_to_block(struct gve_priv *priv, int queue_idx); +struct sk_buff *gve_rx_copy_data(struct net_device *dev, struct napi_struct *napi, + u8 *data, u16 len); + struct sk_buff *gve_rx_copy(struct net_device *dev, struct napi_struct *napi, struct gve_rx_slot_page_info *page_info, u16 len); From patchwork Thu Feb 29 21:22:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ziwei Xiao X-Patchwork-Id: 208532 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2097:b0:108:e6aa:91d0 with SMTP id gs23csp695925dyb; Thu, 29 Feb 2024 13:38:57 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWrrrBwwYI5Gp5RznWripYjfcW+cA16ZDrsaHlQ2TdFjDXWDxxvwvC3fJl/JUUekMLs0OB9Z645nm4LWc5rvvSTzM7J5g== X-Google-Smtp-Source: AGHT+IEPuqKnqCrTEmMDm2/WZCLIYxExnSgR0E7MaHp5213urWlsUXv9yu5HicfQqC+CutSycEeT X-Received: by 2002:a05:6a00:1709:b0:6e5:799b:17b3 with SMTP id h9-20020a056a00170900b006e5799b17b3mr275515pfc.21.1709242737631; Thu, 29 Feb 2024 13:38:57 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1709242737; cv=pass; d=google.com; s=arc-20160816; b=gkr5EMw20U9DyAzE3a3jn2NoHKCZ9f+1cudEbNszEUCi55tJ7bwk11EjzIVn8pnyQz NeApom3O/t8VMlmhLqa4s2e7dn36YGna2qX40B2OzEoxW8L75ywRZYCFw3p3VJ3O5UIG zNfccDd+CIMdoWjzfbs/WiK/qrCox/RXekMCasVNelsZOceTww4ptrGw6LjBWW9p0Li+ rzXc8V66/5YcWJafQ4uMrnM/aw2yDxJhOSILB8dPOM+V0JBxGUkziCoxNK9fXOc6ogHQ MM4ud4z0jXWUCDxkt6NjHGqm+UDPcREp/qvntPT8x4rRMs54s+nN9Im9FvYmEJBInezc Gt/A== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:from:subject:message-id:references:mime-version :list-unsubscribe:list-subscribe:list-id:precedence:in-reply-to:date :dkim-signature; bh=Xurr9IMa0IOArYf8o+P8T4vB1IGhQ03rSgtQlkyp0BA=; fh=Ddyi0j6W0VbEIf+1nbUSbZNw8qGYjM4BffZLoALgN34=; b=ebvQEZNHlQbADQFNjbtNF+GCTW1/UOZKxuRHKPcXSb7e91rkTIPk1aXIfPiJwzZbiX 9yiFu/FOB60ECb9woY1Tc6LnO8HMMh5p5DQu1iDkQ7dGiBllUye8NVih+7V9SCVAJNT2 EIpCsY9fmxZYurAZadPNNviGVTQzyau401iPVUNHx1v2RLDXRSOqdtF7b9MhSjJUftom WCZDXiuIEkxsch5Mu0C3XI7OUkJ92fClmRcWcUN6zcwornBZckoMzP4fwxV9BingXfhs 22AxGFVa0sdIPOJ3Tqkcf1AyUbhEwPiuFt49fdzJGstJtYHg5h2g34iVEvbuWYr2+Dq1 HaHg==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=H78YpcgH; arc=pass (i=1 spf=pass spfdomain=flex--ziweixiao.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-87577-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-87577-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id t36-20020a635364000000b005dcd64fe5dbsi2106505pgl.431.2024.02.29.13.38.57 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Feb 2024 13:38:57 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-87577-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=H78YpcgH; arc=pass (i=1 spf=pass spfdomain=flex--ziweixiao.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-87577-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-87577-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 15CE3B25545 for ; Thu, 29 Feb 2024 21:25:11 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 07F436D529; Thu, 29 Feb 2024 21:24:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="H78YpcgH" Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4694074BF2 for ; Thu, 29 Feb 2024 21:23:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709241836; cv=none; b=uqqdBNicEzLkr5zs/Gu1m1+xgRjC7ulFMawL4Xz+1ulL7JXCsky5MJqGgEAD+6XndoH3b/p9+6TUQ8uACq7q9CJWJlbRxXevM+WHQIiH0pfeMeMa8xae8j0ByQl3MJatKNovB7+58g2fCW7uBxd0OOklfnoiu88El3/W31q14ww= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709241836; c=relaxed/simple; bh=nNRYs43CNcjZoviggESxp1eEDgYULA6gB+/AuIQtI1c=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=uCZKVQlrknQ2XgHVtNsxgfg82/3YJ2yxNwFEdxf5xuGcDX6Fv9odsLTrIbPfgQcTfy/TjlWW/N5HYobn/6k4/AeMBysaMKR6KpKrKB/2hWv7akv++iC4jr+4KQOqX2oM+Sb1akzKMYALqrcl7IX5QT80pkuvsWyJB4o+jvogSeQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ziweixiao.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=H78YpcgH; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ziweixiao.bounces.google.com Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-1dcabec8fddso14927875ad.2 for ; Thu, 29 Feb 2024 13:23:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1709241835; x=1709846635; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Xurr9IMa0IOArYf8o+P8T4vB1IGhQ03rSgtQlkyp0BA=; b=H78YpcgHFoH4ndfLUAoWlauvyQHEotP4s7YBjdwDu16F5npy+5GwSQsR4sYhH+L9jb 849Bk7YgevtElm+2fhInoHzLI+2MwgbxD9F0X3eODveO/iowNjBpck90C+6z3sXuX3Wv 5jz/kn6OE80FMzjQys7icSC+oYCQ+qVd8fCN3z4CBGWvdn8ymES3NPoRVTosu2RuSDGP eSRFhJKA6jsUitOKA7A1pZpPeow6m7rNC+fkxlmJQqTp/flUwkFmUvnRtsji3ibExLvS IwqSSmq7N197x1Fo957JQVaLepuYizNpPJWfAywaosqXTLzHWgpOpH4HeWYDhwtsGQ/7 x2Pw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709241835; x=1709846635; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Xurr9IMa0IOArYf8o+P8T4vB1IGhQ03rSgtQlkyp0BA=; b=ex//xDPlqThCS6/qMxYP1oaQWcSocYGAcrQn5bIUtDuzy1degUlw6ndniLjPb3iSEL IfgjUJ7EDES9v7yGtaz6GB4YQiE4QsxAoi1jYdRgIHIAhjKbGcvQFwWfgU5AEE0J6Dx4 bGqVqZ0EjfOCOuTrXklOTGmENyua8jfurpAbqv0rDuJ4FgYWbk5JpicP/MB1yxgcmq3M +5u94IYKRdnewIIQcsu2BCK1zLioO+hH4csJd08LJi5EAJzIlnisf/y9mCkSHu5K86xI A9amUJs8P98fhmge5k2WnmxsOe/bIA1H3gETb6l9h6SdFjhJVUEKHumuGAS074naF5D9 oQ0Q== X-Forwarded-Encrypted: i=1; AJvYcCWR+OrtDpfLxpH4sBmrHHQNxidO7UMxi55P+kpbfOa4TuxQ/1L9HnCTlRKxP6IV6qzpmK3QwHpplwPn9cFgRY3UDVvML+3BcYOdE9oz X-Gm-Message-State: AOJu0Yxpq0n6RH6Z139OxMJsgtrUto/avP2ZbOvoQ/+yVIYKF15q62TS aR++dBPsatE2nXuAq/bZ8zcPj8JFlR3mWNsu1ZkHJRqOcUKeF3bxK52a1zMJz2T+/OMRKEjlrVD QP0OTnr6nnUZT4A== X-Received: from ziweixiao.sea.corp.google.com ([2620:15c:11c:202:43b5:fdf3:8395:a655]) (user=ziweixiao job=sendgmr) by 2002:a17:902:da8c:b0:1db:b8a6:dbb2 with SMTP id j12-20020a170902da8c00b001dbb8a6dbb2mr61212plx.13.1709241834615; Thu, 29 Feb 2024 13:23:54 -0800 (PST) Date: Thu, 29 Feb 2024 13:22:36 -0800 In-Reply-To: <20240229212236.3152897-1-ziweixiao@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240229212236.3152897-1-ziweixiao@google.com> X-Mailer: git-send-email 2.44.0.rc1.240.g4c46232300-goog Message-ID: <20240229212236.3152897-4-ziweixiao@google.com> Subject: [PATCH net-next 3/3] gve: Add header split ethtool stats From: Ziwei Xiao To: netdev@vger.kernel.org Cc: jeroendb@google.com, pkaligineedi@google.com, shailend@google.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, willemb@google.com, rushilg@google.com, jfraker@google.com, jrkim@google.com, hramamurthy@google.com, ziweixiao@google.com, horms@kernel.org, linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1792270912695129004 X-GMAIL-MSGID: 1792270912695129004 From: Jeroen de Borst To record the stats of header split packets, three stats are added in the driver's ethtool stats. - rx_hsplit_pkt is the split packets count with header split - rx_hsplit_bytes is the received header bytes count with header split - rx_hsplit_unsplit_pkt is the unsplit packet count due to header buffer overflow or zero header length when header split is enabled Currently, it's entering the stats_update critical section more than once per packet. We have plans to avoid that in the future change to let all the stats_update happen in one place at the end of `gve_rx_poll_dqo`. Co-developed-by: Ziwei Xiao Signed-off-by: Ziwei Xiao Signed-off-by: Jeroen de Borst Reviewed-by: Praveen Kaligineedi Reviewed-by: Harshitha Ramamurthy Reviewed-by: Willem de Bruijn --- drivers/net/ethernet/google/gve/gve.h | 4 +++ drivers/net/ethernet/google/gve/gve_ethtool.c | 36 +++++++++++++------ drivers/net/ethernet/google/gve/gve_rx_dqo.c | 9 +++++ 3 files changed, 38 insertions(+), 11 deletions(-) diff --git a/drivers/net/ethernet/google/gve/gve.h b/drivers/net/ethernet/google/gve/gve.h index 5f02b87d7fea..4814c96d5fe7 100644 --- a/drivers/net/ethernet/google/gve/gve.h +++ b/drivers/net/ethernet/google/gve/gve.h @@ -269,15 +269,19 @@ struct gve_rx_ring { }; u64 rbytes; /* free-running bytes received */ + u64 rx_hsplit_bytes; /* free-running header bytes received */ u64 rpackets; /* free-running packets received */ u32 cnt; /* free-running total number of completed packets */ u32 fill_cnt; /* free-running total number of descs and buffs posted */ u32 mask; /* masks the cnt and fill_cnt to the size of the ring */ + u64 rx_hsplit_pkt; /* free-running packets with headers split */ u64 rx_copybreak_pkt; /* free-running count of copybreak packets */ u64 rx_copied_pkt; /* free-running total number of copied packets */ u64 rx_skb_alloc_fail; /* free-running count of skb alloc fails */ u64 rx_buf_alloc_fail; /* free-running count of buffer alloc fails */ u64 rx_desc_err_dropped_pkt; /* free-running count of packets dropped by descriptor error */ + /* free-running count of unsplit packets due to header buffer overflow or hdr_len is 0 */ + u64 rx_hsplit_unsplit_pkt; u64 rx_cont_packet_cnt; /* free-running multi-fragment packets received */ u64 rx_frag_flip_cnt; /* free-running count of rx segments where page_flip was used */ u64 rx_frag_copy_cnt; /* free-running count of rx segments copied */ diff --git a/drivers/net/ethernet/google/gve/gve_ethtool.c b/drivers/net/ethernet/google/gve/gve_ethtool.c index 13c2901968e0..9aebfb843d9d 100644 --- a/drivers/net/ethernet/google/gve/gve_ethtool.c +++ b/drivers/net/ethernet/google/gve/gve_ethtool.c @@ -39,17 +39,18 @@ static u32 gve_get_msglevel(struct net_device *netdev) * as declared in enum xdp_action inside file uapi/linux/bpf.h . */ static const char gve_gstrings_main_stats[][ETH_GSTRING_LEN] = { - "rx_packets", "tx_packets", "rx_bytes", "tx_bytes", - "rx_dropped", "tx_dropped", "tx_timeouts", + "rx_packets", "rx_hsplit_pkt", "tx_packets", "rx_bytes", + "tx_bytes", "rx_dropped", "tx_dropped", "tx_timeouts", "rx_skb_alloc_fail", "rx_buf_alloc_fail", "rx_desc_err_dropped_pkt", + "rx_hsplit_unsplit_pkt", "interface_up_cnt", "interface_down_cnt", "reset_cnt", "page_alloc_fail", "dma_mapping_error", "stats_report_trigger_cnt", }; static const char gve_gstrings_rx_stats[][ETH_GSTRING_LEN] = { - "rx_posted_desc[%u]", "rx_completed_desc[%u]", "rx_consumed_desc[%u]", "rx_bytes[%u]", - "rx_cont_packet_cnt[%u]", "rx_frag_flip_cnt[%u]", "rx_frag_copy_cnt[%u]", - "rx_frag_alloc_cnt[%u]", + "rx_posted_desc[%u]", "rx_completed_desc[%u]", "rx_consumed_desc[%u]", + "rx_bytes[%u]", "rx_hsplit_bytes[%u]", "rx_cont_packet_cnt[%u]", + "rx_frag_flip_cnt[%u]", "rx_frag_copy_cnt[%u]", "rx_frag_alloc_cnt[%u]", "rx_dropped_pkt[%u]", "rx_copybreak_pkt[%u]", "rx_copied_pkt[%u]", "rx_queue_drop_cnt[%u]", "rx_no_buffers_posted[%u]", "rx_drops_packet_over_mru[%u]", "rx_drops_invalid_checksum[%u]", @@ -153,11 +154,13 @@ static void gve_get_ethtool_stats(struct net_device *netdev, struct ethtool_stats *stats, u64 *data) { - u64 tmp_rx_pkts, tmp_rx_bytes, tmp_rx_skb_alloc_fail, - tmp_rx_buf_alloc_fail, tmp_rx_desc_err_dropped_pkt, + u64 tmp_rx_pkts, tmp_rx_hsplit_pkt, tmp_rx_bytes, tmp_rx_hsplit_bytes, + tmp_rx_skb_alloc_fail, tmp_rx_buf_alloc_fail, + tmp_rx_desc_err_dropped_pkt, tmp_rx_hsplit_unsplit_pkt, tmp_tx_pkts, tmp_tx_bytes; - u64 rx_buf_alloc_fail, rx_desc_err_dropped_pkt, rx_pkts, - rx_skb_alloc_fail, rx_bytes, tx_pkts, tx_bytes, tx_dropped; + u64 rx_buf_alloc_fail, rx_desc_err_dropped_pkt, rx_hsplit_unsplit_pkt, + rx_pkts, rx_hsplit_pkt, rx_skb_alloc_fail, rx_bytes, tx_pkts, tx_bytes, + tx_dropped; int stats_idx, base_stats_idx, max_stats_idx; struct stats *report_stats; int *rx_qid_to_stats_idx; @@ -184,8 +187,10 @@ gve_get_ethtool_stats(struct net_device *netdev, kfree(rx_qid_to_stats_idx); return; } - for (rx_pkts = 0, rx_bytes = 0, rx_skb_alloc_fail = 0, - rx_buf_alloc_fail = 0, rx_desc_err_dropped_pkt = 0, ring = 0; + for (rx_pkts = 0, rx_bytes = 0, rx_hsplit_pkt = 0, + rx_skb_alloc_fail = 0, rx_buf_alloc_fail = 0, + rx_desc_err_dropped_pkt = 0, rx_hsplit_unsplit_pkt = 0, + ring = 0; ring < priv->rx_cfg.num_queues; ring++) { if (priv->rx) { do { @@ -194,18 +199,23 @@ gve_get_ethtool_stats(struct net_device *netdev, start = u64_stats_fetch_begin(&priv->rx[ring].statss); tmp_rx_pkts = rx->rpackets; + tmp_rx_hsplit_pkt = rx->rx_hsplit_pkt; tmp_rx_bytes = rx->rbytes; tmp_rx_skb_alloc_fail = rx->rx_skb_alloc_fail; tmp_rx_buf_alloc_fail = rx->rx_buf_alloc_fail; tmp_rx_desc_err_dropped_pkt = rx->rx_desc_err_dropped_pkt; + tmp_rx_hsplit_unsplit_pkt = + rx->rx_hsplit_unsplit_pkt; } while (u64_stats_fetch_retry(&priv->rx[ring].statss, start)); rx_pkts += tmp_rx_pkts; + rx_hsplit_pkt += tmp_rx_hsplit_pkt; rx_bytes += tmp_rx_bytes; rx_skb_alloc_fail += tmp_rx_skb_alloc_fail; rx_buf_alloc_fail += tmp_rx_buf_alloc_fail; rx_desc_err_dropped_pkt += tmp_rx_desc_err_dropped_pkt; + rx_hsplit_unsplit_pkt += tmp_rx_hsplit_unsplit_pkt; } } for (tx_pkts = 0, tx_bytes = 0, tx_dropped = 0, ring = 0; @@ -226,6 +236,7 @@ gve_get_ethtool_stats(struct net_device *netdev, i = 0; data[i++] = rx_pkts; + data[i++] = rx_hsplit_pkt; data[i++] = tx_pkts; data[i++] = rx_bytes; data[i++] = tx_bytes; @@ -237,6 +248,7 @@ gve_get_ethtool_stats(struct net_device *netdev, data[i++] = rx_skb_alloc_fail; data[i++] = rx_buf_alloc_fail; data[i++] = rx_desc_err_dropped_pkt; + data[i++] = rx_hsplit_unsplit_pkt; data[i++] = priv->interface_up_cnt; data[i++] = priv->interface_down_cnt; data[i++] = priv->reset_cnt; @@ -276,6 +288,7 @@ gve_get_ethtool_stats(struct net_device *netdev, start = u64_stats_fetch_begin(&priv->rx[ring].statss); tmp_rx_bytes = rx->rbytes; + tmp_rx_hsplit_bytes = rx->rx_hsplit_bytes; tmp_rx_skb_alloc_fail = rx->rx_skb_alloc_fail; tmp_rx_buf_alloc_fail = rx->rx_buf_alloc_fail; tmp_rx_desc_err_dropped_pkt = @@ -283,6 +296,7 @@ gve_get_ethtool_stats(struct net_device *netdev, } while (u64_stats_fetch_retry(&priv->rx[ring].statss, start)); data[i++] = tmp_rx_bytes; + data[i++] = tmp_rx_hsplit_bytes; data[i++] = rx->rx_cont_packet_cnt; data[i++] = rx->rx_frag_flip_cnt; data[i++] = rx->rx_frag_copy_cnt; diff --git a/drivers/net/ethernet/google/gve/gve_rx_dqo.c b/drivers/net/ethernet/google/gve/gve_rx_dqo.c index a12d776d4385..8e8071308aeb 100644 --- a/drivers/net/ethernet/google/gve/gve_rx_dqo.c +++ b/drivers/net/ethernet/google/gve/gve_rx_dqo.c @@ -720,6 +720,8 @@ static int gve_rx_dqo(struct napi_struct *napi, struct gve_rx_ring *rx, /* Copy the header into the skb in the case of header split */ if (hsplit) { + int unsplit = 0; + if (hdr_len && !hbo) { rx->ctx.skb_head = gve_rx_copy_data(priv->dev, napi, rx->dqo.hdr_bufs.data + @@ -728,7 +730,14 @@ static int gve_rx_dqo(struct napi_struct *napi, struct gve_rx_ring *rx, if (unlikely(!rx->ctx.skb_head)) goto error; rx->ctx.skb_tail = rx->ctx.skb_head; + } else { + unsplit = 1; } + u64_stats_update_begin(&rx->statss); + rx->rx_hsplit_pkt++; + rx->rx_hsplit_unsplit_pkt += unsplit; + rx->rx_hsplit_bytes += hdr_len; + u64_stats_update_end(&rx->statss); } /* Sync the portion of dma buffer for CPU to read. */