From patchwork Mon Dec 11 17:36:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Karan Tilak Kumar (kartilak)" X-Patchwork-Id: 176831 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp7223092vqy; Mon, 11 Dec 2023 09:39:01 -0800 (PST) X-Google-Smtp-Source: AGHT+IFnLicndTk0EKdpBJRPGgwiBm+eAzGLGqGXENRtGvPTxE+7EeghDfLYDKfoe/O8VfPfCpXq X-Received: by 2002:a17:90b:108d:b0:286:ceb9:97f1 with SMTP id gj13-20020a17090b108d00b00286ceb997f1mr1792541pjb.16.1702316341512; Mon, 11 Dec 2023 09:39:01 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702316341; cv=none; d=google.com; s=arc-20160816; b=eBe7ppue8ODyomS4+o3FGVy82zjREHGiWex1GEAS/iMt6D11wzyUY3aurV1bq03Bdo l7rFo3o9qY+NZAmkgRUZEsJ9lbaTXbBeEF2JByyESbgxuw2a4ONKciPODH85JiqBdLC4 ER5hC8og3j7Pq5r2XW333/DfvztpoLb4PFOeLiyhXZnwrOvNYhfVUeE7JhZjpjMGAR4M E89K+ItliaXZFoVGtjJQ5a7qupKnjqNMY5Ta2vfLqGR2wNBeuX+yBJc84Po/+yWcJ6TV jnWXrAvZU4SYhkLX0P1yNHlsGgNasO0t4wL7QzqJC8QC/O1LY036yMvmcjxzFhpXt0r6 WGPw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Q0xWd2ThYF+V4l8oL+rdzstG0URCVrdCbQqXz/xjPCs=; fh=bKSAdOD9sjDLFpuVEw2QY/iOhn17PgYAzgX/PkCBS1w=; b=oKQhqPW6AW0HcwR5WiUC0q1OWwZOyDw9Inc1CRHRAuLr8CIm5+fQ0TDwE6sRICLRGS shCjJYsfwNBhWwprbbg6RENr0g/Lf84pYqXmvJIxRwLY6MMqBCJRKxAIBvxWwbfgsJZ2 6gs0nMI+RsJuOQgNI0sOsZ57wXAtoaVx3cbT608IDzSAUE3vVEe3h48+qZGauaTlLo4n 3D2kD83+Zlw9tmedMJe0UIkRwPgjUQ2aZylvB6jioGx/FKHmEd2srJpuAQUNp4a4RSgN O4tGpLqHIuxK3hcubZNk6YEiL+suXvjN9MyOgQ+p1m5zuwlagD90lzEwQsbg6qvV3nq+ UY1w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@cisco.com header.s=iport header.b=b7rkc9iV; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=NONE dis=NONE) header.from=cisco.com Received: from fry.vger.email (fry.vger.email. [2620:137:e000::3:8]) by mx.google.com with ESMTPS id fz4-20020a17090b024400b002864f3ce7a4si6497088pjb.39.2023.12.11.09.39.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Dec 2023 09:39:01 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) client-ip=2620:137:e000::3:8; Authentication-Results: mx.google.com; dkim=pass header.i=@cisco.com header.s=iport header.b=b7rkc9iV; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=NONE dis=NONE) header.from=cisco.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by fry.vger.email (Postfix) with ESMTP id B622F805F430; Mon, 11 Dec 2023 09:38:57 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at fry.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235018AbjLKRit (ORCPT + 99 others); Mon, 11 Dec 2023 12:38:49 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55976 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235005AbjLKRis (ORCPT ); Mon, 11 Dec 2023 12:38:48 -0500 Received: from alln-iport-8.cisco.com (alln-iport-8.cisco.com [173.37.142.95]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 401B29B; Mon, 11 Dec 2023 09:38:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cisco.com; i=@cisco.com; l=3263; q=dns/txt; s=iport; t=1702316334; x=1703525934; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Q0xWd2ThYF+V4l8oL+rdzstG0URCVrdCbQqXz/xjPCs=; b=b7rkc9iVzhnrYw0Zp3EHf79x8GjmI5cUKYSlviNTUXsLrtmNGhxua/5q fhVExWTzJDOePvOg7utIxxZzsWRl5m1A40VWBPxdrB8FIoPsjzdgwBGP3 ZKV0wdIkJwiLEgC3vYtSoznE/BOWI+s4vt6Y2IzmQdo/MFiwbhm2HQI3b k=; X-CSE-ConnectionGUID: meCwM58qSiuwxEoMoyUcOg== X-CSE-MsgGUID: RDp+kURuSIOMJ2ZSI8CWFQ== X-IronPort-AV: E=Sophos;i="6.04,268,1695686400"; d="scan'208";a="193670948" Received: from rcdn-core-1.cisco.com ([173.37.93.152]) by alln-iport-8.cisco.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Dec 2023 17:38:53 +0000 Received: from localhost.cisco.com ([10.193.101.253]) (authenticated bits=0) by rcdn-core-1.cisco.com (8.15.2/8.15.2) with ESMTPSA id 3BBHaKr0009547 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 11 Dec 2023 17:38:52 GMT From: Karan Tilak Kumar To: sebaddel@cisco.com Cc: arulponn@cisco.com, djhawar@cisco.com, gcboffa@cisco.com, mkai2@cisco.com, satishkh@cisco.com, jejb@linux.ibm.com, martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linux-kernel@vger.kernel.org, Karan Tilak Kumar Subject: [PATCH v6 06/13] scsi: fnic: Refactor and redefine fnic.h for multiqueue Date: Mon, 11 Dec 2023 09:36:10 -0800 Message-Id: <20231211173617.932990-7-kartilak@cisco.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20231211173617.932990-1-kartilak@cisco.com> References: <20231211173617.932990-1-kartilak@cisco.com> MIME-Version: 1.0 X-Authenticated-User: kartilak@cisco.com X-Outbound-SMTP-Client: 10.193.101.253, [10.193.101.253] X-Outbound-Node: rcdn-core-1.cisco.com X-Spam-Status: No, score=-8.4 required=5.0 tests=DKIMWL_WL_HIGH,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE, USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on fry.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (fry.vger.email [0.0.0.0]); Mon, 11 Dec 2023 09:38:57 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785008060174451829 X-GMAIL-MSGID: 1785008060174451829 Refactor and re-define values in fnic.h to implement multiqueue(MQ) functionality. VIC firmware allows fnic to create up to 64 copy workqueues. Update the copy workqueue max to 64. Modify the interrupt index to be in sync with the firmware to support MQ. Add irq number to the MSIX entry. Define a software workqueue table to track the status of io_reqs. Define a base for the copy workqueue. Reviewed-by: Sesidhar Baddela Reviewed-by: Arulprabhu Ponnusamy Signed-off-by: Karan Tilak Kumar --- Changes between v4 and v5: Incorporate review comments from Martin: Modify patch commits to include a "---" separator. Changes between v2 and v3: Replace cpy_wq_base with copy_wq_base. --- drivers/scsi/fnic/fnic.h | 30 ++++++++++++++++++++++-------- 1 file changed, 22 insertions(+), 8 deletions(-) diff --git a/drivers/scsi/fnic/fnic.h b/drivers/scsi/fnic/fnic.h index 2e68209181a8..47173f1eec1e 100644 --- a/drivers/scsi/fnic/fnic.h +++ b/drivers/scsi/fnic/fnic.h @@ -167,12 +167,21 @@ do { \ #define FNIC_MAIN_NOTE(kern_level, host, fmt, args...) \ shost_printk(kern_level, host, fmt, ##args) +#define FNIC_WQ_COPY_MAX 64 +#define FNIC_WQ_MAX 1 +#define FNIC_RQ_MAX 1 +#define FNIC_CQ_MAX (FNIC_WQ_COPY_MAX + FNIC_WQ_MAX + FNIC_RQ_MAX) +#define FNIC_DFLT_IO_COMPLETIONS 256 + +#define FNIC_MQ_CQ_INDEX 2 + extern const char *fnic_state_str[]; enum fnic_intx_intr_index { FNIC_INTX_WQ_RQ_COPYWQ, - FNIC_INTX_ERR, + FNIC_INTX_DUMMY, FNIC_INTX_NOTIFY, + FNIC_INTX_ERR, FNIC_INTX_INTR_MAX, }; @@ -180,7 +189,7 @@ enum fnic_msix_intr_index { FNIC_MSIX_RQ, FNIC_MSIX_WQ, FNIC_MSIX_WQ_COPY, - FNIC_MSIX_ERR_NOTIFY, + FNIC_MSIX_ERR_NOTIFY = FNIC_MSIX_WQ_COPY + FNIC_WQ_COPY_MAX, FNIC_MSIX_INTR_MAX, }; @@ -189,6 +198,7 @@ struct fnic_msix_entry { char devname[IFNAMSIZ + 11]; irqreturn_t (*isr)(int, void *); void *devid; + int irq_num; }; enum fnic_state { @@ -198,12 +208,6 @@ enum fnic_state { FNIC_IN_ETH_TRANS_FC_MODE, }; -#define FNIC_WQ_COPY_MAX 1 -#define FNIC_WQ_MAX 1 -#define FNIC_RQ_MAX 1 -#define FNIC_CQ_MAX (FNIC_WQ_COPY_MAX + FNIC_WQ_MAX + FNIC_RQ_MAX) -#define FNIC_DFLT_IO_COMPLETIONS 256 - struct mempool; enum fnic_evt { @@ -218,6 +222,13 @@ struct fnic_event { enum fnic_evt event; }; +struct fnic_cpy_wq { + unsigned long hw_lock_flags; + u16 active_ioreq_count; + u16 ioreq_table_size; + ____cacheline_aligned struct fnic_io_req **io_req_table; +}; + /* Per-instance private data structure */ struct fnic { int fnic_num; @@ -289,6 +300,7 @@ struct fnic { mempool_t *io_sgl_pool[FNIC_SGL_NUM_CACHES]; spinlock_t io_req_lock[FNIC_IO_LOCKS]; /* locks for scsi cmnds */ + unsigned int copy_wq_base; struct work_struct link_work; struct work_struct frame_work; struct sk_buff_head frame_queue; @@ -308,6 +320,8 @@ struct fnic { /* copy work queue cache line section */ ____cacheline_aligned struct vnic_wq_copy hw_copy_wq[FNIC_WQ_COPY_MAX]; + ____cacheline_aligned struct fnic_cpy_wq sw_copy_wq[FNIC_WQ_COPY_MAX]; + /* completion queue cache line section */ ____cacheline_aligned struct vnic_cq cq[FNIC_CQ_MAX];