From patchwork Thu Nov 30 02:33:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Karan Tilak Kumar (kartilak)" X-Patchwork-Id: 171687 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp113244vqy; Wed, 29 Nov 2023 18:36:34 -0800 (PST) X-Google-Smtp-Source: AGHT+IHoyAxvHb9ZvEvoUMBzVsI+wCVtVgXLzpfud4WYCcqdS8yv78bFqYb/Uda1eKi9RbnUOQ8k X-Received: by 2002:a05:6a00:2d97:b0:6cb:8c5e:6bec with SMTP id fb23-20020a056a002d9700b006cb8c5e6becmr23025657pfb.9.1701311794432; Wed, 29 Nov 2023 18:36:34 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1701311794; cv=none; d=google.com; s=arc-20160816; b=vxovrIt6909fEUvNRwpVqAbhPTEu+Y7YwJQ4hg88giH8OCCrulq7RN5WCDcTFI2VnM VOWvTjhSrZ3EDMPwaIG/x55MhWpBwSktw9NoKu9on3kNE0xbaTzDabosflmNzsUJGrxw DqAIxlainyaNeagbDc9tyU6gT8Hh1bnuHWsXBEP16yc+vH+HO/BptJSrVW2B4L5077r3 7sH+m/tIttJd14ceoSMQ6gkF489Sj662BPPEQsxD/xbO+XKiHrKOXeObFxJsrGjZ+era KfF2K+iiz89slnl+0yst0Qj0jgGYUKltzvO4C/gZzZZS2BpSLCha0G8itttBjUWeAmnr E2+Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=yGV4XpeRlWLab4jEtJ9QdH5HcxDM5fJtqOUxJdoEe+E=; fh=bKSAdOD9sjDLFpuVEw2QY/iOhn17PgYAzgX/PkCBS1w=; b=VQriRhme/V2wat21Wy8WRF3lUO4hhdgnjDZIussH1NI/1aKSUNJnSo1b2Pje1gS6ts wY6JflVN6FTsjryEmwpZwUZ44f95wpP94AC35GcsAV+aLeJ1Rqqt+8uztNGbeELfcpj/ cv4nY6PDW+4zAZA9SRRkDItKMQ/sNeyjTJIuvUrHW1YDsY7dGAPJzXKIJoim44oBaVoj eHZK1PFBodUbwUm1FHI1EPbgW3xolZAHHuI771KUkyuyyteTTDZUq8pG80qjNfQHEflX g1G0EL7UIIpvq18Z8kGA5RqeBaGjit5Lt8SV8v8e9q21L4k8MM8w3xVvwxdwHUn4GlqM X6jw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@cisco.com header.s=iport header.b=dHZF8JYd; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=NONE dis=NONE) header.from=cisco.com Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id do11-20020a056a004a0b00b006cd853221easi156542pfb.362.2023.11.29.18.36.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 29 Nov 2023 18:36:34 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; dkim=pass header.i=@cisco.com header.s=iport header.b=dHZF8JYd; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=NONE dis=NONE) header.from=cisco.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 7B6B6802AF22; Wed, 29 Nov 2023 18:36:17 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344186AbjK3CgG (ORCPT + 99 others); Wed, 29 Nov 2023 21:36:06 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51034 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231644AbjK3CgC (ORCPT ); Wed, 29 Nov 2023 21:36:02 -0500 Received: from rcdn-iport-9.cisco.com (rcdn-iport-9.cisco.com [173.37.86.80]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B17C1D71; Wed, 29 Nov 2023 18:36:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=3195; q=dns/txt; s=iport; t=1701311767; x=1702521367; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=mcaSeiV3DCOZVPr6R0+RWxfvehrcdQLMAkhm3bXZX8w=; b=dHZF8JYdm3CrXixiPDlTMwB3Mwfau1t/0cIzeseu1m5Y7BImsabu3OKZ ec9lfNmCeyEOvFa+02d5BAhI4NkskzqXgF4HASj1Xl2V+oeyZ+iG+oa4I y3ABiz3umYb+k//aa9Au6A2Ovyux4RXsw8HHHkfaswJUgPWCFVL62SXbr k=; X-CSE-ConnectionGUID: 59cYfSnMSe2K5avB3PAwJA== X-CSE-MsgGUID: jeiqYSx+Qeeg79zJYnV+Iw== X-IronPort-AV: E=Sophos;i="6.04,237,1695686400"; d="scan'208";a="148823866" Received: from rcdn-core-9.cisco.com ([173.37.93.145]) by rcdn-iport-9.cisco.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Nov 2023 02:36:06 +0000 Received: from localhost.cisco.com ([10.193.101.253]) (authenticated bits=0) by rcdn-core-9.cisco.com (8.15.2/8.15.2) with ESMTPSA id 3AU2YA48007614 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 30 Nov 2023 02:36:06 GMT From: Karan Tilak Kumar To: sebaddel@cisco.com Cc: arulponn@cisco.com, djhawar@cisco.com, gcboffa@cisco.com, mkai2@cisco.com, satishkh@cisco.com, jejb@linux.ibm.com, martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linux-kernel@vger.kernel.org, Karan Tilak Kumar Subject: [PATCH v4 06/13] scsi: fnic: Refactor and redefine fnic.h for multiqueue Date: Wed, 29 Nov 2023 18:33:55 -0800 Message-Id: <20231130023402.802282-7-kartilak@cisco.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20231130023402.802282-1-kartilak@cisco.com> References: <20231130023402.802282-1-kartilak@cisco.com> MIME-Version: 1.0 X-Authenticated-User: kartilak@cisco.com X-Outbound-SMTP-Client: 10.193.101.253, [10.193.101.253] X-Outbound-Node: rcdn-core-9.cisco.com X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIMWL_WL_MED,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL, SPF_HELO_PASS,SPF_NONE,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Wed, 29 Nov 2023 18:36:17 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783954716375795648 X-GMAIL-MSGID: 1783954716375795648 Refactor and re-define values in fnic.h to implement multiqueue(MQ) functionality. VIC firmware allows fnic to create up to 64 copy workqueues. Update the copy workqueue max to 64. Modify the interrupt index to be in sync with the firmware to support MQ. Add irq number to the MSIX entry. Define a software workqueue table to track the status of io_reqs. Define a base for the copy workqueue. Changes between v2 and v3: Replace cpy_wq_base with copy_wq_base. Reviewed-by: Sesidhar Baddela Reviewed-by: Arulprabhu Ponnusamy Signed-off-by: Karan Tilak Kumar --- drivers/scsi/fnic/fnic.h | 30 ++++++++++++++++++++++-------- 1 file changed, 22 insertions(+), 8 deletions(-) diff --git a/drivers/scsi/fnic/fnic.h b/drivers/scsi/fnic/fnic.h index f4bd0e13203d..07d67fe903f2 100644 --- a/drivers/scsi/fnic/fnic.h +++ b/drivers/scsi/fnic/fnic.h @@ -167,12 +167,21 @@ do { \ #define FNIC_MAIN_NOTE(kern_level, host, fmt, args...) \ shost_printk(kern_level, host, fmt, ##args) +#define FNIC_WQ_COPY_MAX 64 +#define FNIC_WQ_MAX 1 +#define FNIC_RQ_MAX 1 +#define FNIC_CQ_MAX (FNIC_WQ_COPY_MAX + FNIC_WQ_MAX + FNIC_RQ_MAX) +#define FNIC_DFLT_IO_COMPLETIONS 256 + +#define FNIC_MQ_CQ_INDEX 2 + extern const char *fnic_state_str[]; enum fnic_intx_intr_index { FNIC_INTX_WQ_RQ_COPYWQ, - FNIC_INTX_ERR, + FNIC_INTX_DUMMY, FNIC_INTX_NOTIFY, + FNIC_INTX_ERR, FNIC_INTX_INTR_MAX, }; @@ -180,7 +189,7 @@ enum fnic_msix_intr_index { FNIC_MSIX_RQ, FNIC_MSIX_WQ, FNIC_MSIX_WQ_COPY, - FNIC_MSIX_ERR_NOTIFY, + FNIC_MSIX_ERR_NOTIFY = FNIC_MSIX_WQ_COPY + FNIC_WQ_COPY_MAX, FNIC_MSIX_INTR_MAX, }; @@ -189,6 +198,7 @@ struct fnic_msix_entry { char devname[IFNAMSIZ + 11]; irqreturn_t (*isr)(int, void *); void *devid; + int irq_num; }; enum fnic_state { @@ -198,12 +208,6 @@ enum fnic_state { FNIC_IN_ETH_TRANS_FC_MODE, }; -#define FNIC_WQ_COPY_MAX 1 -#define FNIC_WQ_MAX 1 -#define FNIC_RQ_MAX 1 -#define FNIC_CQ_MAX (FNIC_WQ_COPY_MAX + FNIC_WQ_MAX + FNIC_RQ_MAX) -#define FNIC_DFLT_IO_COMPLETIONS 256 - struct mempool; enum fnic_evt { @@ -218,6 +222,13 @@ struct fnic_event { enum fnic_evt event; }; +struct fnic_cpy_wq { + unsigned long hw_lock_flags; + u16 active_ioreq_count; + u16 ioreq_table_size; + ____cacheline_aligned struct fnic_io_req **io_req_table; +}; + /* Per-instance private data structure */ struct fnic { int fnic_num; @@ -287,6 +298,7 @@ struct fnic { mempool_t *io_sgl_pool[FNIC_SGL_NUM_CACHES]; spinlock_t io_req_lock[FNIC_IO_LOCKS]; /* locks for scsi cmnds */ + unsigned int copy_wq_base; struct work_struct link_work; struct work_struct frame_work; struct sk_buff_head frame_queue; @@ -306,6 +318,8 @@ struct fnic { /* copy work queue cache line section */ ____cacheline_aligned struct vnic_wq_copy hw_copy_wq[FNIC_WQ_COPY_MAX]; + ____cacheline_aligned struct fnic_cpy_wq sw_copy_wq[FNIC_WQ_COPY_MAX]; + /* completion queue cache line section */ ____cacheline_aligned struct vnic_cq cq[FNIC_CQ_MAX];