From patchwork Thu Dec 8 23:18:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Asutosh Das X-Patchwork-Id: 31524 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp472863wrr; Thu, 8 Dec 2022 15:20:48 -0800 (PST) X-Google-Smtp-Source: AA0mqf7n1jn5zy882UU7ej5pgP2wthqyorOoLaxEO6lJkY1MZxDoY8PSkWZqzk16eAPBmTBzq+ba X-Received: by 2002:a17:902:ebc2:b0:185:441e:58ae with SMTP id p2-20020a170902ebc200b00185441e58aemr3685174plg.17.1670541648539; Thu, 08 Dec 2022 15:20:48 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670541648; cv=none; d=google.com; s=arc-20160816; b=oxY+W5cMU3+xJHP14LZn/7ZMqay47mw+reswp5HfjU8fj5DWnfNUufgi4GdvAMo0Wk K6uHrAriegRtnw59rSq6WSBUfjMIZXzrlvb6yqU+VeACmGpy97VVnv6FFYb+7VEEiBNr 5WJ4GixJqVOl5RFzUS2ClfG8ciPKCL8maDVHvaZ9W4urzSshrZgFfAw9R3SMUr/iucTR smG6Z47fbNVQm6aZIGbVgr9lRSd8xPw0YtFQuM4rnuKPtF1SzkSKoXut4g7BDqjrDRZb AGhcK02EicwHjE8sECnHP4TnHFvzh7onSwFo+B0+bIvz+z4ny5jEZbvWmmQ1ZCZXtcde 4nSQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=ySeoHANVoUPzDPpnomiG5X9f91zkrUH1wy9mvylqHe0=; b=gSMeZX7CcTkqG30TH1BAHRzf/uKXQfxIdrYy8G1cwMnOo/SPg+wIXkQlQMkSq0cAOr 1THLDhJ5p3X8eZItk4WZLCblhgodooWhHU1AnawbGQpCE53WdkE71yj9yjCScHEcc091 SvoqpzRZPg2HAI6O/xJYuOD6VSnFgCEeBz+jQ7MeDq9PI8+xbhYZydqbeVIO7hOn61fI azPFOlD+RJrRJ/lyTnCpSanF+HEgf+nJZd72LHpQwnjHULw5pmMcfNzdSrB9wGNGyhrk K8i9Y9B5HNBKSy44kdIqEjzesLOSFOEXZUgzWJt1aXfBORP/VpFLYJ1hI/ue60F1Sv1L R5Cw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcdkim header.b=qdMGqaYa; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id t20-20020a170902e1d400b00188d6e1b831si22132052pla.298.2022.12.08.15.20.35; Thu, 08 Dec 2022 15:20:48 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcdkim header.b=qdMGqaYa; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229868AbiLHXTA (ORCPT + 99 others); Thu, 8 Dec 2022 18:19:00 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58768 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229656AbiLHXS7 (ORCPT ); Thu, 8 Dec 2022 18:18:59 -0500 Received: from alexa-out-sd-01.qualcomm.com (alexa-out-sd-01.qualcomm.com [199.106.114.38]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3EC3879C92; Thu, 8 Dec 2022 15:18:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1670541538; x=1702077538; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=ySeoHANVoUPzDPpnomiG5X9f91zkrUH1wy9mvylqHe0=; b=qdMGqaYa49KbQPzdKWuntnelIbcoHUhQ9vWI6dfyWJm4We1HPCfxHlez 5vwG2oe4pSVxrEW7EjFLY0c4k4m9Lef1t1ZIs2Yuc2wjxJwp+Ea+PL+1M JfkzP+GCx4X3qfb8Mtacem3kKxUE7gYCJ3RS6GPHFUyT2eOU263hGHdf1 Y=; Received: from unknown (HELO ironmsg04-sd.qualcomm.com) ([10.53.140.144]) by alexa-out-sd-01.qualcomm.com with ESMTP; 08 Dec 2022 15:18:54 -0800 X-QCInternal: smtphost Received: from unknown (HELO nasanex01a.na.qualcomm.com) ([10.52.223.231]) by ironmsg04-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Dec 2022 15:18:54 -0800 Received: from asutoshd-linux1.qualcomm.com (10.80.80.8) by nasanex01a.na.qualcomm.com (10.52.223.231) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Thu, 8 Dec 2022 15:18:54 -0800 From: Asutosh Das To: , , CC: , , , , , , , , , Asutosh Das , , Alim Akhtar , "James E.J. Bottomley" , Matthias Brugger , Jinyoung Choi , open list , "moderated list:ARM/Mediatek SoC support" , "moderated list:ARM/Mediatek SoC support" Subject: [PATCH v11 01/16] ufs: core: Optimize duplicate code to read extended feature Date: Thu, 8 Dec 2022 15:18:27 -0800 Message-ID: X-Mailer: git-send-email 2.7.4 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01b.na.qualcomm.com (10.46.141.250) To nasanex01a.na.qualcomm.com (10.52.223.231) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751689879895088031?= X-GMAIL-MSGID: =?utf-8?q?1751689879895088031?= The code to parse the extended feature is duplicated twice in the ufs core. Replace the duplicated code with a function. Signed-off-by: Asutosh Das Reviewed-by: Bart Van Assche Reviewed-by: Manivannan Sadhasivam Reviewed-by: Stanley Chu --- drivers/ufs/core/ufshcd.c | 21 +++++++++++++-------- 1 file changed, 13 insertions(+), 8 deletions(-) diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c index 2dbe249..6ea22b5 100644 --- a/drivers/ufs/core/ufshcd.c +++ b/drivers/ufs/core/ufshcd.c @@ -215,6 +215,17 @@ ufs_get_desired_pm_lvl_for_dev_link_state(enum ufs_dev_pwr_mode dev_state, return UFS_PM_LVL_0; } +static unsigned int ufs_get_ext_ufs_feature(struct ufs_hba *hba, + const u8 *desc_buf) +{ + if (hba->desc_size[QUERY_DESC_IDN_DEVICE] < + DEVICE_DESC_PARAM_EXT_UFS_FEATURE_SUP + 4) + return 0; + + return get_unaligned_be32(desc_buf + + DEVICE_DESC_PARAM_EXT_UFS_FEATURE_SUP); +} + static const struct ufs_dev_quirk ufs_fixups[] = { /* UFS cards deviations table */ { .wmanufacturerid = UFS_VENDOR_MICRON, @@ -7608,13 +7619,7 @@ static void ufshcd_wb_probe(struct ufs_hba *hba, const u8 *desc_buf) (hba->dev_quirks & UFS_DEVICE_QUIRK_SUPPORT_EXTENDED_FEATURES))) goto wb_disabled; - if (hba->desc_size[QUERY_DESC_IDN_DEVICE] < - DEVICE_DESC_PARAM_EXT_UFS_FEATURE_SUP + 4) - goto wb_disabled; - - ext_ufs_feature = get_unaligned_be32(desc_buf + - DEVICE_DESC_PARAM_EXT_UFS_FEATURE_SUP); - + ext_ufs_feature = ufs_get_ext_ufs_feature(hba, desc_buf); if (!(ext_ufs_feature & UFS_DEV_WRITE_BOOSTER_SUP)) goto wb_disabled; @@ -7668,7 +7673,7 @@ static void ufshcd_temp_notif_probe(struct ufs_hba *hba, const u8 *desc_buf) if (!(hba->caps & UFSHCD_CAP_TEMP_NOTIF) || dev_info->wspecversion < 0x300) return; - ext_ufs_feature = get_unaligned_be32(desc_buf + DEVICE_DESC_PARAM_EXT_UFS_FEATURE_SUP); + ext_ufs_feature = ufs_get_ext_ufs_feature(hba, desc_buf); if (ext_ufs_feature & UFS_DEV_LOW_TEMP_NOTIF) mask |= MASK_EE_TOO_LOW_TEMP; From patchwork Thu Dec 8 23:18:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Asutosh Das X-Patchwork-Id: 31529 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp473816wrr; Thu, 8 Dec 2022 15:23:28 -0800 (PST) X-Google-Smtp-Source: AA0mqf6Z5L/UerDeAhztx3W3oK8AtUrE1xSeY0dwHeXHOxMYVb01LB8HgqhIB9P2phtDCx6qBYMq X-Received: by 2002:a05:6a20:690c:b0:9d:efbe:a113 with SMTP id q12-20020a056a20690c00b0009defbea113mr5989374pzj.35.1670541807873; Thu, 08 Dec 2022 15:23:27 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670541807; cv=none; d=google.com; s=arc-20160816; b=OkhbIFLZImhIPY2xEx0v2E+37S/TPkw1TIfZ5u30pdJGBLdyQDe+VJDg6v6ynVTUfT td+eTXBYi6Qh+n0b618FQ2K7q/4lunl82dRKWakLEiVKMnw+VDfUcdxU/f2IMtwPN/GX RsunXXRcZ+LLFfSAZjEI7d2M7p6C0mns2MUYTyHdANsiGUxQazCiwBixeOnNkqQxbYIw I2Bj0QlNz/mM9h1B229U/hLK5swx1x+D004HD4KgD22EqjK8KQuhgCmbyUsKW/9AEImY PqgNWbwwy6bMtx0ppMxcpyH4/ONyVBGG5Oz4CSC9RGhSGtedlKUjToAErJ42OwCtYs5l CsLA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=+d0l14rTPCd2W++biZT6/aB/nfa1GzHun1cHII16ROs=; b=Js0iIn5QL6H1ulL2c6fJJroWewXriOAH11Vwr5LlEwOTmoVcrjKoL4Df7t2GaokLM4 uspGIihw5u9kY8WPI7hFFRP6cUuIIIRD2EDjp6NE7PXQsha/Dgo+PUfoqBopWdYC9Gfk CCpekM8ypj67fNS3+d7ifykQHLPSoDTE1vTugAKyTDsMzt7AAK7sSiHyXiKi1VmUbT6f rSoVSge0hG8ljJ8HHWy33OXccMAIjijiWre0TD6+oaA6DkRsd05TT7a0SLUSskDf1MZK pz7MpazJsfzjZaICDA8zS68rh2Rk88grTk59F/P1R7hdTOnMeIJ+VdZVTymXd9LoukvI RNqQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcdkim header.b=oMMseT5D; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id s14-20020a632c0e000000b00460a6ee5b53si24952235pgs.37.2022.12.08.15.23.14; Thu, 08 Dec 2022 15:23:27 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcdkim header.b=oMMseT5D; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230088AbiLHXTR (ORCPT + 99 others); Thu, 8 Dec 2022 18:19:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58864 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229811AbiLHXTG (ORCPT ); Thu, 8 Dec 2022 18:19:06 -0500 Received: from alexa-out-sd-02.qualcomm.com (alexa-out-sd-02.qualcomm.com [199.106.114.39]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 60B4579C27; Thu, 8 Dec 2022 15:19:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1670541545; x=1702077545; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=+d0l14rTPCd2W++biZT6/aB/nfa1GzHun1cHII16ROs=; b=oMMseT5DTNsMtzHwzTNrZRB9IoF+qTvnnB0DkeKg1fJpL2PzySpAYu9Q lZoJpZohDYINkTe6Y7sNsHg3V21h1J3kpG9GXrT1U+KwlYM03DSjcMMt5 LZRzIiHmSng0XKiDWn4fqVFTJvbjzHutsMhonssspNQaBYfvhDZpvujjw 0=; Received: from unknown (HELO ironmsg04-sd.qualcomm.com) ([10.53.140.144]) by alexa-out-sd-02.qualcomm.com with ESMTP; 08 Dec 2022 15:19:04 -0800 X-QCInternal: smtphost Received: from unknown (HELO nasanex01a.na.qualcomm.com) ([10.52.223.231]) by ironmsg04-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Dec 2022 15:19:04 -0800 Received: from asutoshd-linux1.qualcomm.com (10.80.80.8) by nasanex01a.na.qualcomm.com (10.52.223.231) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Thu, 8 Dec 2022 15:19:03 -0800 From: Asutosh Das To: , , CC: , , , , , , , , , Asutosh Das , , Alim Akhtar , "James E.J. Bottomley" , Matthias Brugger , Jinyoung Choi , Keoseong Park , Yoshihiro Shimoda , Kiwoong Kim , open list , "moderated list:ARM/Mediatek SoC support" , "moderated list:ARM/Mediatek SoC support" Subject: [PATCH v11 02/16] ufs: core: Probe for ext_iid support Date: Thu, 8 Dec 2022 15:18:28 -0800 Message-ID: X-Mailer: git-send-email 2.7.4 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01b.na.qualcomm.com (10.46.141.250) To nasanex01a.na.qualcomm.com (10.52.223.231) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751690046730019899?= X-GMAIL-MSGID: =?utf-8?q?1751690046730019899?= Task Tag is limited to 8 bits and this restricts the number of active IOs to 255. In Multi-circular queue mode, this may not be enough. The specification provides EXT_IID which can be used to increase the number of IOs if the UFS device and UFSHC support it. This patch adds support to probe for ext_iid support in ufs device and UFSHC. Co-developed-by: Can Guo Signed-off-by: Can Guo Signed-off-by: Asutosh Das Reviewed-by: Bart Van Assche Reviewed-by: Avri Altman Reviewed-by: Manivannan Sadhasivam Reviewed-by: Stanley Chu --- drivers/ufs/core/ufshcd.c | 31 +++++++++++++++++++++++++++++++ include/ufs/ufs.h | 4 ++++ include/ufs/ufshcd.h | 4 ++++ include/ufs/ufshci.h | 7 +++++++ 4 files changed, 46 insertions(+) diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c index 6ea22b5..595fd3c 100644 --- a/drivers/ufs/core/ufshcd.c +++ b/drivers/ufs/core/ufshcd.c @@ -2258,6 +2258,10 @@ static inline int ufshcd_hba_capabilities(struct ufs_hba *hba) if (err) dev_err(hba->dev, "crypto setup failed\n"); + hba->mcq_capabilities = ufshcd_readl(hba, REG_MCQCAP); + hba->ext_iid_sup = FIELD_GET(MASK_EXT_IID_SUPPORT, + hba->mcq_capabilities); + return err; } @@ -7687,6 +7691,30 @@ static void ufshcd_temp_notif_probe(struct ufs_hba *hba, const u8 *desc_buf) } } +static void ufshcd_ext_iid_probe(struct ufs_hba *hba, u8 *desc_buf) +{ + struct ufs_dev_info *dev_info = &hba->dev_info; + u32 ext_ufs_feature; + u32 ext_iid_en = 0; + int err; + + /* Only UFS-4.0 and above may support EXT_IID */ + if (dev_info->wspecversion < 0x400) + goto out; + + ext_ufs_feature = ufs_get_ext_ufs_feature(hba, desc_buf); + if (!(ext_ufs_feature & UFS_DEV_EXT_IID_SUP)) + goto out; + + err = ufshcd_query_attr_retry(hba, UPIU_QUERY_OPCODE_READ_ATTR, + QUERY_ATTR_IDN_EXT_IID_EN, 0, 0, &ext_iid_en); + if (err) + dev_err(hba->dev, "failed reading bEXTIIDEn. err = %d\n", err); + +out: + dev_info->b_ext_iid_en = ext_iid_en; +} + void ufshcd_fixup_dev_quirks(struct ufs_hba *hba, const struct ufs_dev_quirk *fixups) { @@ -7785,6 +7813,9 @@ static int ufs_get_device_desc(struct ufs_hba *hba) ufshcd_temp_notif_probe(hba, desc_buf); + if (hba->ext_iid_sup) + ufshcd_ext_iid_probe(hba, desc_buf); + /* * ufshcd_read_string_desc returns size of the string * reset the error value diff --git a/include/ufs/ufs.h b/include/ufs/ufs.h index 1bba3fe..ba2a1d8 100644 --- a/include/ufs/ufs.h +++ b/include/ufs/ufs.h @@ -165,6 +165,7 @@ enum attr_idn { QUERY_ATTR_IDN_AVAIL_WB_BUFF_SIZE = 0x1D, QUERY_ATTR_IDN_WB_BUFF_LIFE_TIME_EST = 0x1E, QUERY_ATTR_IDN_CURR_WB_BUFF_SIZE = 0x1F, + QUERY_ATTR_IDN_EXT_IID_EN = 0x2A, }; /* Descriptor idn for Query requests */ @@ -352,6 +353,7 @@ enum { UFS_DEV_EXT_TEMP_NOTIF = BIT(6), UFS_DEV_HPB_SUPPORT = BIT(7), UFS_DEV_WRITE_BOOSTER_SUP = BIT(8), + UFS_DEV_EXT_IID_SUP = BIT(16), }; #define UFS_DEV_HPB_SUPPORT_VERSION 0x310 @@ -601,6 +603,8 @@ struct ufs_dev_info { bool b_rpm_dev_flush_capable; u8 b_presrv_uspc_en; + /* UFS EXT_IID Enable */ + bool b_ext_iid_en; }; /* diff --git a/include/ufs/ufshcd.h b/include/ufs/ufshcd.h index 5cf81df..aec37cb9 100644 --- a/include/ufs/ufshcd.h +++ b/include/ufs/ufshcd.h @@ -747,6 +747,7 @@ struct ufs_hba_monitor { * @outstanding_lock: Protects @outstanding_reqs. * @outstanding_reqs: Bits representing outstanding transfer requests * @capabilities: UFS Controller Capabilities + * @mcq_capabilities: UFS Multi Circular Queue capabilities * @nutrs: Transfer Request Queue depth supported by controller * @nutmrs: Task Management Queue depth supported by controller * @reserved_slot: Used to submit device commands. Protected by @dev_cmd.lock. @@ -830,6 +831,7 @@ struct ufs_hba_monitor { * device * @complete_put: whether or not to call ufshcd_rpm_put() from inside * ufshcd_resume_complete() + * @ext_iid_sup: is EXT_IID is supported by UFSHC */ struct ufs_hba { void __iomem *mmio_base; @@ -871,6 +873,7 @@ struct ufs_hba { u32 capabilities; int nutrs; + u32 mcq_capabilities; int nutmrs; u32 reserved_slot; u32 ufs_version; @@ -978,6 +981,7 @@ struct ufs_hba { #endif u32 luns_avail; bool complete_put; + bool ext_iid_sup; }; /* Returns true if clocks can be gated. Otherwise false */ diff --git a/include/ufs/ufshci.h b/include/ufs/ufshci.h index f525566..4d4da06 100644 --- a/include/ufs/ufshci.h +++ b/include/ufs/ufshci.h @@ -22,6 +22,7 @@ enum { /* UFSHCI Registers */ enum { REG_CONTROLLER_CAPABILITIES = 0x00, + REG_MCQCAP = 0x04, REG_UFS_VERSION = 0x08, REG_CONTROLLER_DEV_ID = 0x10, REG_CONTROLLER_PROD_ID = 0x14, @@ -68,6 +69,12 @@ enum { MASK_OUT_OF_ORDER_DATA_DELIVERY_SUPPORT = 0x02000000, MASK_UIC_DME_TEST_MODE_SUPPORT = 0x04000000, MASK_CRYPTO_SUPPORT = 0x10000000, + MASK_MCQ_SUPPORT = 0x40000000, +}; + +/* MCQ capability mask */ +enum { + MASK_EXT_IID_SUPPORT = 0x00000400, }; #define UFS_MASK(mask, offset) ((mask) << (offset)) From patchwork Thu Dec 8 23:18:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Asutosh Das X-Patchwork-Id: 31525 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp473098wrr; Thu, 8 Dec 2022 15:21:24 -0800 (PST) X-Google-Smtp-Source: AA0mqf5vDcIkoTDTms6s0hUp+AQnFFMyCnrvU0nMVEi6EUf/I+azEkIIBYRAgvmgljknQKySTm1N X-Received: by 2002:a17:902:d044:b0:189:6ab3:9e57 with SMTP id l4-20020a170902d04400b001896ab39e57mr2961599pll.22.1670541684137; Thu, 08 Dec 2022 15:21:24 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670541684; cv=none; d=google.com; s=arc-20160816; b=SJULlvedXU7mRnqS1ir4Hcvm1oujYWMd6/08QUk5fqZ9gu9KdHWFqseqv3zLaPtkeI nfZBf/EaF4D2bI8VqqqFFXEQAP3onLjJcDWTC0u4JMZOALroh9uCwPuzdNN6ZMvdaMc8 +A79pjJnpjr9ApaJPgk9dgvLWZoMsI3hr08nSMBvvtv86Cuy+8v6S2x1KwUS8juGvMgN w86/BuiItWbcRFkrrlr9I6T7cr0rypdvTIkP0HY9MP8Rf7UGgIBj2Kn9rck5HZB2oDi7 jUkP1meNJTO4f/uCKbjfOPI7c2Nwja0MqeOndObU8kV+jcUtEVtIYx/UUksmXfOPbUDP dheQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=pdKIMJ/UrD6GGAEohrm/VtNxCd41dtCamf/awJloxGM=; b=KmE17N/tLp9m4G52BD3KdeU+rLHS0MkvyI4kbqBuGeut4fHbwHhWaGBqXiRR3QPgkN tq7tPagSTxu8szS9xlNfoMAOBH/KadQHV5FXshzALSfP2vhK9csI0LXhqJ316/xcNDv6 Eb8tWLKADHhYRA98C58tad4beO+VvgjfuLrCMY0n5wr95NJhePK7gScuqF/Agh7X7zZU dzRahwXKdHm9YDSxQAQguPkd1S8eNKXGTRuafFQFbp79mJupQsRb++B5/hry+Bqbgw/a xd/vwsQ3+P/U7kcNg+bq/Fb7YGnqsiaCYmW78z+5ZWSM/MZrywwkzNDxteX7N0DoC3Dm 08rg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=Udv5OdiR; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id e12-20020a17090301cc00b00189e9aca9b3si10784106plh.34.2022.12.08.15.21.11; Thu, 08 Dec 2022 15:21:24 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=Udv5OdiR; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230106AbiLHXTz (ORCPT + 99 others); Thu, 8 Dec 2022 18:19:55 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59452 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229895AbiLHXTj (ORCPT ); Thu, 8 Dec 2022 18:19:39 -0500 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 03170A0FB6; Thu, 8 Dec 2022 15:19:29 -0800 (PST) Received: from pps.filterd (m0279864.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2B8MlgbL007456; Thu, 8 Dec 2022 23:19:11 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=qcppdkim1; bh=pdKIMJ/UrD6GGAEohrm/VtNxCd41dtCamf/awJloxGM=; b=Udv5OdiRZGN4UYjnt0wkz4MGfyAoRX2VcHONH4atUKZDU+4NYFj6WKNcN8x4UsoXKK32 cVpcgfY/wYwDse0dE0ZZmAvcuDFNbf3Sf/qwlRw13LKpQVpmz5HAFC2+GeAnJpzvIWcc UFA2d5CdVCxIQwsdsmx45gmWavvBZqhqGXsR93RjUhe+HF5vs4MKlQdXO2NqiAizFmdS G69z526IjJe0xgJ1QWHo5V4dedDhT6yyBH+XMGKa3rOiUfDuQVg2YrnfUblt496Wyh/8 /TQKLcQCdAna6E6WZCswk3kcFMvOzUiT5xBVNOdexHFsd/ZyZatld21MKvrLKCJX/lpi EA== Received: from nasanppmta01.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3mbffs9ne4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 08 Dec 2022 23:19:11 +0000 Received: from nasanex01a.na.qualcomm.com (corens_vlan604_snip.qualcomm.com [10.53.140.1]) by NASANPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 2B8NJAML030391 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 8 Dec 2022 23:19:10 GMT Received: from asutoshd-linux1.qualcomm.com (10.80.80.8) by nasanex01a.na.qualcomm.com (10.52.223.231) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Thu, 8 Dec 2022 15:19:10 -0800 From: Asutosh Das To: , , CC: , , , , , , , , , Asutosh Das , , Alim Akhtar , "James E.J. Bottomley" , Matthias Brugger , Jinyoung Choi , Yoshihiro Shimoda , Keoseong Park , open list , "moderated list:ARM/Mediatek SoC support" , "moderated list:ARM/Mediatek SoC support" Subject: [PATCH v11 03/16] ufs: core: Introduce Multi-circular queue capability Date: Thu, 8 Dec 2022 15:18:29 -0800 Message-ID: <49921a49f2ad5b64a7c46535c485f281877baf7a.1670541364.git.quic_asutoshd@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01b.na.qualcomm.com (10.46.141.250) To nasanex01a.na.qualcomm.com (10.52.223.231) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: _b2Tx6UY_jpi75V805qr1CFuiC4t2SLp X-Proofpoint-GUID: _b2Tx6UY_jpi75V805qr1CFuiC4t2SLp X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-08_12,2022-12-08_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 mlxscore=0 priorityscore=1501 impostorscore=0 lowpriorityscore=0 mlxlogscore=999 bulkscore=0 spamscore=0 malwarescore=0 suspectscore=0 clxscore=1015 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000 definitions=main-2212080192 X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751689916948273683?= X-GMAIL-MSGID: =?utf-8?q?1751689916948273683?= Add support to check for MCQ capability in the UFSHC. Add a module parameter to disable MCQ if needed. Co-developed-by: Can Guo Signed-off-by: Can Guo Signed-off-by: Asutosh Das Reviewed-by: Bart Van Assche Reviewed-by: Manivannan Sadhasivam Reviewed-by: Stanley Chu --- drivers/ufs/core/ufshcd.c | 26 ++++++++++++++++++++++++++ include/ufs/ufshcd.h | 2 ++ 2 files changed, 28 insertions(+) diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c index 595fd3c..eca15b0 100644 --- a/drivers/ufs/core/ufshcd.c +++ b/drivers/ufs/core/ufshcd.c @@ -89,6 +89,28 @@ /* Polling time to wait for fDeviceInit */ #define FDEVICEINIT_COMPL_TIMEOUT 1500 /* millisecs */ +/* UFSHC 4.0 compliant HC support this mode, refer param_set_mcq_mode() */ +static bool use_mcq_mode = true; + +static int param_set_mcq_mode(const char *val, const struct kernel_param *kp) +{ + int ret; + + ret = param_set_bool(val, kp); + if (ret) + return ret; + + return 0; +} + +static const struct kernel_param_ops mcq_mode_ops = { + .set = param_set_mcq_mode, + .get = param_get_bool, +}; + +module_param_cb(use_mcq_mode, &mcq_mode_ops, &use_mcq_mode, 0644); +MODULE_PARM_DESC(use_mcq_mode, "Control MCQ mode for controllers starting from UFSHCI 4.0. 1 - enable MCQ, 0 - disable MCQ. MCQ is enabled by default"); + #define ufshcd_toggle_vreg(_dev, _vreg, _on) \ ({ \ int _ret; \ @@ -2258,6 +2280,10 @@ static inline int ufshcd_hba_capabilities(struct ufs_hba *hba) if (err) dev_err(hba->dev, "crypto setup failed\n"); + hba->mcq_sup = FIELD_GET(MASK_MCQ_SUPPORT, hba->capabilities); + if (!hba->mcq_sup) + return err; + hba->mcq_capabilities = ufshcd_readl(hba, REG_MCQCAP); hba->ext_iid_sup = FIELD_GET(MASK_EXT_IID_SUPPORT, hba->mcq_capabilities); diff --git a/include/ufs/ufshcd.h b/include/ufs/ufshcd.h index aec37cb9..70c0f9f 100644 --- a/include/ufs/ufshcd.h +++ b/include/ufs/ufshcd.h @@ -832,6 +832,7 @@ struct ufs_hba_monitor { * @complete_put: whether or not to call ufshcd_rpm_put() from inside * ufshcd_resume_complete() * @ext_iid_sup: is EXT_IID is supported by UFSHC + * @mcq_sup: is mcq supported by UFSHC */ struct ufs_hba { void __iomem *mmio_base; @@ -982,6 +983,7 @@ struct ufs_hba { u32 luns_avail; bool complete_put; bool ext_iid_sup; + bool mcq_sup; }; /* Returns true if clocks can be gated. Otherwise false */ From patchwork Thu Dec 8 23:18:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Asutosh Das X-Patchwork-Id: 31527 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp473437wrr; Thu, 8 Dec 2022 15:22:20 -0800 (PST) X-Google-Smtp-Source: AA0mqf6nsKYW+DGGjzeXCPpxabljNzNAHjc00924FByrLc4+sdI85ip1WVOVKlWVnT9n1718a9K6 X-Received: by 2002:a05:6a20:690d:b0:9d:efbe:e601 with SMTP id q13-20020a056a20690d00b0009defbee601mr5583636pzj.29.1670541739733; Thu, 08 Dec 2022 15:22:19 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670541739; cv=none; d=google.com; s=arc-20160816; b=jGAQ/Vm/5NipgymfFAh6jhyuwAf+n47cTsmq0loK9e6Dkb7ex1BceCFjXAG+WSWd32 mXUJhnzknEc81qVq6iCp7FfeoLEcti7mZm5jvWi++fEMP2keQrlOVc4dSujClTPz0OxK +2VJJ1zGLiNuAlnbU0/d326h3OPmufg4SxfL8wVI10qYKP+DqYe2eQLYR/wyvXVo7Zp5 9viOHBzY0wQN7txwIMRxOr/Bhw8Ea+yM4VzK5zsmYQ1F5JUmTYzFPQc8jR2xjMG124XB DyQDCUcYttvA33nSIVIRatsYDDpY3HHN2ZiWnm2X7WmCvJqoJVYAbQF4IpvaiY3MOKjH n3Vw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=XoJjdTNXumk8Uumod1/jhp8Wj2FhsoysZ80e4BVp310=; b=aAzXNqCjGb/e7MRe3K82w3wnrcjEt5IFJ6qW3j1NVnmsRqG2QfaNtG3jz5aBlzHKD1 jn2NqHspR+yBZoX1uayhSmAkepQ9ulEnU787l+jx6hS3ZN2d4tjxXSjZTaXzqKxSE6f+ wyIynbQNBPeZEdq0iLh28hGy2c/moYai/Q5njAqvpHn72pItQla++uHfZSPyYwd6w719 dhXWAgU0O3C4jONXCPiFMuP8bcdoDrlWXqKtllkJXEoigCCq5U9foLwHwRTCNNHszQrj Qhu0hrUOZnadLXWwt3/mNR49UihtLXRf3NZOLv1JuRx9KtEowHOH3iEremufU7Jq271o mLIQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcdkim header.b=x3l6ErR7; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id w63-20020a638242000000b00478bf0b11b1si11494634pgd.752.2022.12.08.15.22.06; Thu, 08 Dec 2022 15:22:19 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcdkim header.b=x3l6ErR7; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230097AbiLHXTq (ORCPT + 99 others); Thu, 8 Dec 2022 18:19:46 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58926 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230084AbiLHXTQ (ORCPT ); Thu, 8 Dec 2022 18:19:16 -0500 Received: from alexa-out-sd-01.qualcomm.com (alexa-out-sd-01.qualcomm.com [199.106.114.38]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 547E99E464; Thu, 8 Dec 2022 15:19:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1670541554; x=1702077554; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=XoJjdTNXumk8Uumod1/jhp8Wj2FhsoysZ80e4BVp310=; b=x3l6ErR761IQ8O9rJP/blvI0eL5/lUeT35/QzE9SESXdvKH22q4wnNQF YGnSa6pV5RHOv83jXO5gnS0E62gf5U5NzAOs9bWTurHeav8WbFB19VFz6 QX1eo5YLIMBBdg6kgdfKWAK8Fsy5KzD14dtu9QTFK4WHGNOEDjKGGlLTv k=; Received: from unknown (HELO ironmsg-SD-alpha.qualcomm.com) ([10.53.140.30]) by alexa-out-sd-01.qualcomm.com with ESMTP; 08 Dec 2022 15:19:14 -0800 X-QCInternal: smtphost Received: from unknown (HELO nasanex01a.na.qualcomm.com) ([10.52.223.231]) by ironmsg-SD-alpha.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Dec 2022 15:19:13 -0800 Received: from asutoshd-linux1.qualcomm.com (10.80.80.8) by nasanex01a.na.qualcomm.com (10.52.223.231) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Thu, 8 Dec 2022 15:19:13 -0800 From: Asutosh Das To: , , CC: , , , , , , , , , Asutosh Das , , Alim Akhtar , "James E.J. Bottomley" , Jinyoung Choi , open list Subject: [PATCH v11 04/16] ufs: core: Defer adding host to scsi if mcq is supported Date: Thu, 8 Dec 2022 15:18:30 -0800 Message-ID: <1e64cece4de5f01c05a3d157f7afcdffa5044e04.1670541364.git.quic_asutoshd@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01b.na.qualcomm.com (10.46.141.250) To nasanex01a.na.qualcomm.com (10.52.223.231) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751689975143994455?= X-GMAIL-MSGID: =?utf-8?q?1751689975143994455?= If MCQ support is present, enabling it after MCQ support has been configured would require reallocating tags and memory. It would also free up the already allocated memory in Single Doorbell Mode. So defer invoking scsi_add_host() until MCQ is configured. Co-developed-by: Can Guo Signed-off-by: Can Guo Signed-off-by: Asutosh Das Reviewed-by: Bart Van Assche Reviewed-by: Manivannan Sadhasivam --- drivers/ufs/core/ufshcd.c | 24 ++++++++++++++++++++---- 1 file changed, 20 insertions(+), 4 deletions(-) diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c index eca15b0..869e495 100644 --- a/drivers/ufs/core/ufshcd.c +++ b/drivers/ufs/core/ufshcd.c @@ -92,6 +92,11 @@ /* UFSHC 4.0 compliant HC support this mode, refer param_set_mcq_mode() */ static bool use_mcq_mode = true; +static bool is_mcq_supported(struct ufs_hba *hba) +{ + return hba->mcq_sup && use_mcq_mode; +} + static int param_set_mcq_mode(const char *val, const struct kernel_param *kp) { int ret; @@ -8227,6 +8232,7 @@ static int ufshcd_probe_hba(struct ufs_hba *hba, bool init_dev_params) int ret; unsigned long flags; ktime_t start = ktime_get(); + struct Scsi_Host *host = hba->host; hba->ufshcd_state = UFSHCD_STATE_RESET; @@ -8261,6 +8267,14 @@ static int ufshcd_probe_hba(struct ufs_hba *hba, bool init_dev_params) ret = ufshcd_device_params_init(hba); if (ret) goto out; + + if (is_mcq_supported(hba)) { + ret = scsi_add_host(host, hba->dev); + if (ret) { + dev_err(hba->dev, "scsi_add_host failed\n"); + goto out; + } + } } ufshcd_tune_unipro_params(hba); @@ -9857,10 +9871,12 @@ int ufshcd_init(struct ufs_hba *hba, void __iomem *mmio_base, unsigned int irq) hba->is_irq_enabled = true; } - err = scsi_add_host(host, hba->dev); - if (err) { - dev_err(hba->dev, "scsi_add_host failed\n"); - goto out_disable; + if (!is_mcq_supported(hba)) { + err = scsi_add_host(host, hba->dev); + if (err) { + dev_err(hba->dev, "scsi_add_host failed\n"); + goto out_disable; + } } hba->tmf_tag_set = (struct blk_mq_tag_set) { From patchwork Thu Dec 8 23:18:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Asutosh Das X-Patchwork-Id: 31528 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp473802wrr; Thu, 8 Dec 2022 15:23:24 -0800 (PST) X-Google-Smtp-Source: AA0mqf7Tcwn4wgTHL28ujIiso7X9SwEJtzCrkaVMqFYzg2ez7lMT6gbYwxz/REy+pXoW16PLN/mx X-Received: by 2002:a62:640f:0:b0:574:f1bc:bbee with SMTP id y15-20020a62640f000000b00574f1bcbbeemr3364397pfb.31.1670541803731; Thu, 08 Dec 2022 15:23:23 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670541803; cv=none; d=google.com; s=arc-20160816; b=skWHjjRngqlG6Ta8cXZDsuYwDBEENFI+grSaJAIBKYVWjzFMtFEsFARMTUf/B0yfgf MZ5dPhs8aee9NRydiSRVvNhj+N1X6oL5ndcwxuBt+RXnxWvjM9zgc+0s4qPXiHuDIN9R CuqV/DF5QRGvKGtRcZKMkw39GyxEEqZHsVl/T9wlvjgPso341dtZIJnzMpJxPPqN1aCQ q388XQYcr/05xrSSdwrRcRyqRmBxx7TSlH/z1rs3ceMeKGuwY1ADUzLAWVfIvw538Xnw bEMO6c64XSu6GWGJOzGl6Ny4foTQc0eacx6//Df+m0hAanJsZ/TRPBBLBnUPNGbmSfto B87w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=KUmDDQbEKYIrvM5vrZzKHPw9tSj7QBRyxDFFydUs+Yc=; b=NyECM2wI4s/VgCkjOGH2jbK+ywUis+NCMKcdDlUlUuRkPRlBnBLZbGF2iYpPvq2Oxk 3GVQilJBZb0mT1lEB+vMDFKfSyKXAOD+Z0Z0d0j8lsoW1hBDTUyD3zkbFmTp9OHOJIHu HQ09eP+heDYMWMZ1n9vHwR+awNBCUsYHwgYq8/d3kpXSriGQkgLU3huArD150c305sXp no2MvNoxWYX4eH0P0Vecgrog23DGRAnILes0MHcOszuc54MhGX2etcisPYuOQka+fvYO it+gAZPNE50dZlqtLY5ax7f8MhXJn8MoCflEOCD1KKIN3w5GjngNJ11qLhV/aYDAjB5g msDw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=NyzdBfli; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id a186-20020a624dc3000000b005397c4ad9e1si112814pfb.134.2022.12.08.15.23.06; Thu, 08 Dec 2022 15:23:23 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=NyzdBfli; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229843AbiLHXUP (ORCPT + 99 others); Thu, 8 Dec 2022 18:20:15 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59564 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230022AbiLHXTs (ORCPT ); Thu, 8 Dec 2022 18:19:48 -0500 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 59AA7A0F80; Thu, 8 Dec 2022 15:19:47 -0800 (PST) Received: from pps.filterd (m0279864.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2B8N479S014411; Thu, 8 Dec 2022 23:19:31 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=qcppdkim1; bh=KUmDDQbEKYIrvM5vrZzKHPw9tSj7QBRyxDFFydUs+Yc=; b=NyzdBflikwHJw0bAckp67jPSz0FKvMNlHbvVC2mKlJCC2X1Z0xy05zI5XGl4hTHAZoLs vZAf4Ss5pm6pmM5BqSzJES70JQqiqZpcWnOTH17H168jgCOmCzox5PZbkwH5Aebebw3h plXZAequ6jb4qbj9xmznvjHQ72XLcltIoInr5CiU4JF+5XuKa81c7HDti282pqYddkNz WXy/7OQn/O0Q47UR7oQm0itJe9I7b57fxR4EFPiDSn8da4Ef97rXWqX8urZzUSYzxCFK wO63edHzYiMc3+EeGyWM+rTl+n2UYS5lcr78MlxW2PyiJZkU0MAnotSFMkHPcI7cTFwn Rw== Received: from nasanppmta01.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3mbffs9nen-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 08 Dec 2022 23:19:31 +0000 Received: from nasanex01a.na.qualcomm.com (corens_vlan604_snip.qualcomm.com [10.53.140.1]) by NASANPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 2B8NJVv8030557 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 8 Dec 2022 23:19:31 GMT Received: from asutoshd-linux1.qualcomm.com (10.80.80.8) by nasanex01a.na.qualcomm.com (10.52.223.231) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Thu, 8 Dec 2022 15:19:30 -0800 From: Asutosh Das To: , , CC: , , , , , , , , , Asutosh Das , , Alim Akhtar , "James E.J. Bottomley" , Jinyoung Choi , Krzysztof Kozlowski , Arthur Simchaev , Yoshihiro Shimoda , Keoseong Park , open list Subject: [PATCH v11 05/16] ufs: core: mcq: Add support to allocate multiple queues Date: Thu, 8 Dec 2022 15:18:31 -0800 Message-ID: X-Mailer: git-send-email 2.7.4 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01b.na.qualcomm.com (10.46.141.250) To nasanex01a.na.qualcomm.com (10.52.223.231) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: xpEqI_AD9L6o5fnT-wmFP71vU5sKBOLK X-Proofpoint-GUID: xpEqI_AD9L6o5fnT-wmFP71vU5sKBOLK X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-08_12,2022-12-08_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 mlxscore=0 priorityscore=1501 impostorscore=0 lowpriorityscore=0 mlxlogscore=999 bulkscore=0 spamscore=0 malwarescore=0 suspectscore=0 clxscore=1015 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000 definitions=main-2212080192 X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751690042389655897?= X-GMAIL-MSGID: =?utf-8?q?1751690042389655897?= Multi-circular queue (MCQ) has been added in UFSHC v4.0 standard in addition to the Single Doorbell mode. The MCQ mode supports multiple submission and completion queues. Add support to allocate and configure the queues. Add module parameters support to configure the queues. Co-developed-by: Can Guo Signed-off-by: Can Guo Signed-off-by: Asutosh Das Reviewed-by: Bart Van Assche Reviewed-by: Manivannan Sadhasivam --- drivers/ufs/core/Makefile | 2 +- drivers/ufs/core/ufs-mcq.c | 124 +++++++++++++++++++++++++++++++++++++++++ drivers/ufs/core/ufshcd-priv.h | 1 + drivers/ufs/core/ufshcd.c | 12 ++++ include/ufs/ufshcd.h | 4 ++ 5 files changed, 142 insertions(+), 1 deletion(-) create mode 100644 drivers/ufs/core/ufs-mcq.c diff --git a/drivers/ufs/core/Makefile b/drivers/ufs/core/Makefile index 62f38c5..4d02e0f 100644 --- a/drivers/ufs/core/Makefile +++ b/drivers/ufs/core/Makefile @@ -1,7 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 obj-$(CONFIG_SCSI_UFSHCD) += ufshcd-core.o -ufshcd-core-y += ufshcd.o ufs-sysfs.o +ufshcd-core-y += ufshcd.o ufs-sysfs.o ufs-mcq.o ufshcd-core-$(CONFIG_DEBUG_FS) += ufs-debugfs.o ufshcd-core-$(CONFIG_SCSI_UFS_BSG) += ufs_bsg.o ufshcd-core-$(CONFIG_SCSI_UFS_CRYPTO) += ufshcd-crypto.o diff --git a/drivers/ufs/core/ufs-mcq.c b/drivers/ufs/core/ufs-mcq.c new file mode 100644 index 0000000..6ed3625 --- /dev/null +++ b/drivers/ufs/core/ufs-mcq.c @@ -0,0 +1,124 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022 Qualcomm Innovation Center. All rights reserved. + * + * Authors: + * Asutosh Das + * Can Guo + */ + +#include +#include +#include +#include +#include "ufshcd-priv.h" + +#define MAX_QUEUE_SUP GENMASK(7, 0) +#define UFS_MCQ_MIN_RW_QUEUES 2 +#define UFS_MCQ_MIN_READ_QUEUES 0 +#define UFS_MCQ_NUM_DEV_CMD_QUEUES 1 +#define UFS_MCQ_MIN_POLL_QUEUES 0 + +static int rw_queue_count_set(const char *val, const struct kernel_param *kp) +{ + return param_set_uint_minmax(val, kp, UFS_MCQ_MIN_RW_QUEUES, + num_possible_cpus()); +} + +static const struct kernel_param_ops rw_queue_count_ops = { + .set = rw_queue_count_set, + .get = param_get_uint, +}; + +static unsigned int rw_queues; +module_param_cb(rw_queues, &rw_queue_count_ops, &rw_queues, 0644); +MODULE_PARM_DESC(rw_queues, + "Number of interrupt driven I/O queues used for rw. Default value is nr_cpus"); + +static int read_queue_count_set(const char *val, const struct kernel_param *kp) +{ + return param_set_uint_minmax(val, kp, UFS_MCQ_MIN_READ_QUEUES, + num_possible_cpus()); +} + +static const struct kernel_param_ops read_queue_count_ops = { + .set = read_queue_count_set, + .get = param_get_uint, +}; + +static unsigned int read_queues; +module_param_cb(read_queues, &read_queue_count_ops, &read_queues, 0644); +MODULE_PARM_DESC(read_queues, + "Number of interrupt driven read queues used for read. Default value is 0"); + +static int poll_queue_count_set(const char *val, const struct kernel_param *kp) +{ + return param_set_uint_minmax(val, kp, UFS_MCQ_MIN_POLL_QUEUES, + num_possible_cpus()); +} + +static const struct kernel_param_ops poll_queue_count_ops = { + .set = poll_queue_count_set, + .get = param_get_uint, +}; + +static unsigned int poll_queues = 1; +module_param_cb(poll_queues, &poll_queue_count_ops, &poll_queues, 0644); +MODULE_PARM_DESC(poll_queues, + "Number of poll queues used for r/w. Default value is 1"); + +static int ufshcd_mcq_config_nr_queues(struct ufs_hba *hba) +{ + int i; + u32 hba_maxq, rem, tot_queues; + struct Scsi_Host *host = hba->host; + + hba_maxq = FIELD_GET(MAX_QUEUE_SUP, hba->mcq_capabilities); + + tot_queues = UFS_MCQ_NUM_DEV_CMD_QUEUES + read_queues + poll_queues + + rw_queues; + + if (hba_maxq < tot_queues) { + dev_err(hba->dev, "Total queues (%d) exceeds HC capacity (%d)\n", + tot_queues, hba_maxq); + return -EOPNOTSUPP; + } + + rem = hba_maxq - UFS_MCQ_NUM_DEV_CMD_QUEUES; + + if (rw_queues) { + hba->nr_queues[HCTX_TYPE_DEFAULT] = rw_queues; + rem -= hba->nr_queues[HCTX_TYPE_DEFAULT]; + } else { + rw_queues = num_possible_cpus(); + } + + if (poll_queues) { + hba->nr_queues[HCTX_TYPE_POLL] = poll_queues; + rem -= hba->nr_queues[HCTX_TYPE_POLL]; + } + + if (read_queues) { + hba->nr_queues[HCTX_TYPE_READ] = read_queues; + rem -= hba->nr_queues[HCTX_TYPE_READ]; + } + + if (!hba->nr_queues[HCTX_TYPE_DEFAULT]) + hba->nr_queues[HCTX_TYPE_DEFAULT] = min3(rem, rw_queues, + num_possible_cpus()); + + for (i = 0; i < HCTX_MAX_TYPES; i++) + host->nr_hw_queues += hba->nr_queues[i]; + + hba->nr_hw_queues = host->nr_hw_queues + UFS_MCQ_NUM_DEV_CMD_QUEUES; + return 0; +} + +int ufshcd_mcq_init(struct ufs_hba *hba) +{ + int ret; + + ret = ufshcd_mcq_config_nr_queues(hba); + + return ret; +} diff --git a/drivers/ufs/core/ufshcd-priv.h b/drivers/ufs/core/ufshcd-priv.h index a9e8e1f..9368ba2 100644 --- a/drivers/ufs/core/ufshcd-priv.h +++ b/drivers/ufs/core/ufshcd-priv.h @@ -61,6 +61,7 @@ int ufshcd_query_attr(struct ufs_hba *hba, enum query_opcode opcode, int ufshcd_query_flag(struct ufs_hba *hba, enum query_opcode opcode, enum flag_idn idn, u8 index, bool *flag_res); void ufshcd_auto_hibern8_update(struct ufs_hba *hba, u32 ahit); +int ufshcd_mcq_init(struct ufs_hba *hba); #define SD_ASCII_STD true #define SD_RAW false diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c index 869e495..4e2355b 100644 --- a/drivers/ufs/core/ufshcd.c +++ b/drivers/ufs/core/ufshcd.c @@ -8220,6 +8220,11 @@ static int ufshcd_add_lus(struct ufs_hba *hba) return ret; } +static int ufshcd_alloc_mcq(struct ufs_hba *hba) +{ + return ufshcd_mcq_init(hba); +} + /** * ufshcd_probe_hba - probe hba to detect device and initialize it * @hba: per-adapter instance @@ -8269,6 +8274,13 @@ static int ufshcd_probe_hba(struct ufs_hba *hba, bool init_dev_params) goto out; if (is_mcq_supported(hba)) { + ret = ufshcd_alloc_mcq(hba); + if (ret) { + /* Continue with SDB mode */ + use_mcq_mode = false; + dev_err(hba->dev, "MCQ mode is disabled, err=%d\n", + ret); + } ret = scsi_add_host(host, hba->dev); if (ret) { dev_err(hba->dev, "scsi_add_host failed\n"); diff --git a/include/ufs/ufshcd.h b/include/ufs/ufshcd.h index 70c0f9f..146b613 100644 --- a/include/ufs/ufshcd.h +++ b/include/ufs/ufshcd.h @@ -829,6 +829,8 @@ struct ufs_hba_monitor { * ee_ctrl_mask * @luns_avail: number of regular and well known LUNs supported by the UFS * device + * @nr_hw_queues: number of hardware queues configured + * @nr_queues: number of Queues of different queue types * @complete_put: whether or not to call ufshcd_rpm_put() from inside * ufshcd_resume_complete() * @ext_iid_sup: is EXT_IID is supported by UFSHC @@ -981,6 +983,8 @@ struct ufs_hba { u32 debugfs_ee_rate_limit_ms; #endif u32 luns_avail; + unsigned int nr_hw_queues; + unsigned int nr_queues[HCTX_MAX_TYPES]; bool complete_put; bool ext_iid_sup; bool mcq_sup; From patchwork Thu Dec 8 23:18:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Asutosh Das X-Patchwork-Id: 31526 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp473435wrr; Thu, 8 Dec 2022 15:22:19 -0800 (PST) X-Google-Smtp-Source: AA0mqf77/F3E72cjvScPP6e5YSe8n7F0pKOJTMmNY6NTUbrsyWVTTn4VIYZIhw6Z1Lbzwp2DqwM3 X-Received: by 2002:a05:6a20:8e06:b0:a7:187a:7b0d with SMTP id y6-20020a056a208e0600b000a7187a7b0dmr6499345pzj.18.1670541738960; Thu, 08 Dec 2022 15:22:18 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670541738; cv=none; d=google.com; s=arc-20160816; b=zR2wpaG6Xof57xoQT3SKMe2NpUaayo2pzvEqKmyu18w+D4awDNAAdSOcKrqpNGwNsM ahZkAUuDj6/eoPmd8Wh45WHlyOe6EfPlECxXZuG6xJRxU4zLFksmUmaOMDaTJOxPlgez IVnqthHygfd+d0IQGDcEdhvObkV7LSdSVX1hYuunoaTiiGgrXcawDuMfhBSMqIq7gk7I /1Bj9ElJSyGAQovB8Y1HflOdtO0t8KsUxgL8bdTgHGTV1GU32sLQqjJZz5X8zPcvASB8 1tYIw7YgXBzZk8zAOSo2gUU3EjOROzW8loRAaesYh/E+GS+YygNT3hSNd39wk3tHXMFR YRRg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=coMgtVIQV0KNJH47ZivlYLl4PuyO+OasqaGfgycFad8=; b=ebeCgz3DsvWMJwuvfonJxOQ+a+M1b1r7j5SpwwsvGq/s78huDBDz4dYD5D9egiJuO6 7Dhb2IhI9USnJmq2W6FVSr88jUpwkqf0i7I8CrCGYrgo1tD658BFJJioFbNCjxDDtzgk 1AfGF0+3dkaVw196JDphumDQ+jV9ZS2rtK2Vr/wXPXrqkeIbiMafbF9m0urRFulFNiZI 5r79o8+oqGPQYljLuIeY7CPCjgQ5UZ4JqJfSNIfdUPrO4il6bxLjha5Me6xCqoXUfx23 fplvw6rHHlvKaVOqRLUVbMHbzo/WXs3Rqaqn1c77SFb4mIUTVVno6+56j6C6+67VYYQC BvpQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=I8ZckVbO; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id q128-20020a632a86000000b00478fd997431si5317980pgq.872.2022.12.08.15.22.05; Thu, 08 Dec 2022 15:22:18 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=I8ZckVbO; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230163AbiLHXUf (ORCPT + 99 others); Thu, 8 Dec 2022 18:20:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59470 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230158AbiLHXUD (ORCPT ); Thu, 8 Dec 2022 18:20:03 -0500 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 35BBBA504D; Thu, 8 Dec 2022 15:20:01 -0800 (PST) Received: from pps.filterd (m0279867.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2B8Mx2OP013737; Thu, 8 Dec 2022 23:19:44 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=qcppdkim1; bh=coMgtVIQV0KNJH47ZivlYLl4PuyO+OasqaGfgycFad8=; b=I8ZckVbO0jE2QEeuPP3kpUHd8yln8Id5Sw32N5OGtsiA18i0dhLf+Rh8Oa8Evd5Q3RQF evDlf2NfbOeO6LuTGS0IGQQUhqO57tPcZ7ErmbEdNcawyorPjn2KF3qPWAJBGy8IEkVc Y1RuYBFjsScodGUF9G3YhBsQiUpEwzQZWs8P1fAh5cz+hb8BEXdLBvGWSedtPW+OxU49 5DB9N0r+bX+4RWasRNo8GsS0Xx2gcKT1M8HCeVuPPhf9cmbBt3UfEfP+2Kt+pb5cr8Dn pxlh07aAb7659pM50GBNYzwxZnjZD/CRs1mHz9QDpMilz6XLVI8UI4FMKkppM9pMk3U4 Ww== Received: from nasanppmta05.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3mbr0785jb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 08 Dec 2022 23:19:44 +0000 Received: from nasanex01a.na.qualcomm.com ([10.52.223.231]) by NASANPPMTA05.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 2B8NJhDZ029226 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 8 Dec 2022 23:19:43 GMT Received: from asutoshd-linux1.qualcomm.com (10.80.80.8) by nasanex01a.na.qualcomm.com (10.52.223.231) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Thu, 8 Dec 2022 15:19:42 -0800 From: Asutosh Das To: , , CC: , , , , , , , , , Asutosh Das , , Alim Akhtar , "James E.J. Bottomley" , "Andy Gross" , Bjorn Andersson , "Konrad Dybcio" , Krzysztof Kozlowski , Jinyoung Choi , Yoshihiro Shimoda , Keoseong Park , open list Subject: [PATCH v11 06/16] ufs: core: mcq: Configure resource regions Date: Thu, 8 Dec 2022 15:18:32 -0800 Message-ID: X-Mailer: git-send-email 2.7.4 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01b.na.qualcomm.com (10.46.141.250) To nasanex01a.na.qualcomm.com (10.52.223.231) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: hTpVBXImEQxUJPXokpeATUPjaMwGydIm X-Proofpoint-GUID: hTpVBXImEQxUJPXokpeATUPjaMwGydIm X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-08_12,2022-12-08_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 priorityscore=1501 impostorscore=0 malwarescore=0 adultscore=0 spamscore=0 suspectscore=0 bulkscore=0 mlxscore=0 clxscore=1015 lowpriorityscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000 definitions=main-2212080192 X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751689974509316854?= X-GMAIL-MSGID: =?utf-8?q?1751689974509316854?= Define the mcq resources and add support to ioremap the resource regions. Co-developed-by: Can Guo Signed-off-by: Can Guo Signed-off-by: Asutosh Das Reviewed-by: Manivannan Sadhasivam Reviewed-by: Bart Van Assche Reviewed-by: Stanley Chu --- drivers/ufs/core/ufs-mcq.c | 3 ++ drivers/ufs/core/ufshcd-priv.h | 8 ++++ drivers/ufs/host/ufs-qcom.c | 101 +++++++++++++++++++++++++++++++++++++++++ include/ufs/ufshcd.h | 30 ++++++++++++ 4 files changed, 142 insertions(+) diff --git a/drivers/ufs/core/ufs-mcq.c b/drivers/ufs/core/ufs-mcq.c index 6ed3625..65c0037 100644 --- a/drivers/ufs/core/ufs-mcq.c +++ b/drivers/ufs/core/ufs-mcq.c @@ -119,6 +119,9 @@ int ufshcd_mcq_init(struct ufs_hba *hba) int ret; ret = ufshcd_mcq_config_nr_queues(hba); + if (ret) + return ret; + ret = ufshcd_vops_mcq_config_resource(hba); return ret; } diff --git a/drivers/ufs/core/ufshcd-priv.h b/drivers/ufs/core/ufshcd-priv.h index 9368ba2..3e21242 100644 --- a/drivers/ufs/core/ufshcd-priv.h +++ b/drivers/ufs/core/ufshcd-priv.h @@ -227,6 +227,14 @@ static inline void ufshcd_vops_config_scaling_param(struct ufs_hba *hba, hba->vops->config_scaling_param(hba, p, data); } +static inline int ufshcd_vops_mcq_config_resource(struct ufs_hba *hba) +{ + if (hba->vops && hba->vops->mcq_config_resource) + return hba->vops->mcq_config_resource(hba); + + return -EOPNOTSUPP; +} + extern const struct ufs_pm_lvl_states ufs_pm_lvl_states[]; /** diff --git a/drivers/ufs/host/ufs-qcom.c b/drivers/ufs/host/ufs-qcom.c index 8ad1415..045f8ab 100644 --- a/drivers/ufs/host/ufs-qcom.c +++ b/drivers/ufs/host/ufs-qcom.c @@ -25,6 +25,12 @@ #define UFS_QCOM_DEFAULT_DBG_PRINT_EN \ (UFS_QCOM_DBG_PRINT_REGS_EN | UFS_QCOM_DBG_PRINT_TEST_BUS_EN) +#define MCQ_QCFGPTR_MASK GENMASK(7, 0) +#define MCQ_QCFGPTR_UNIT 0x200 +#define MCQ_SQATTR_OFFSET(c) \ + ((((c) >> 16) & MCQ_QCFGPTR_MASK) * MCQ_QCFGPTR_UNIT) +#define MCQ_QCFG_SIZE 0x40 + enum { TSTBUS_UAWM, TSTBUS_UARM, @@ -1424,6 +1430,100 @@ static void ufs_qcom_config_scaling_param(struct ufs_hba *hba, } #endif +/* Resources */ +static const struct ufshcd_res_info ufs_res_info[RES_MAX] = { + {.name = "ufs_mem",}, + {.name = "mcq",}, + /* Submission Queue DAO */ + {.name = "mcq_sqd",}, + /* Submission Queue Interrupt Status */ + {.name = "mcq_sqis",}, + /* Completion Queue DAO */ + {.name = "mcq_cqd",}, + /* Completion Queue Interrupt Status */ + {.name = "mcq_cqis",}, + /* MCQ vendor specific */ + {.name = "mcq_vs",}, +}; + +static int ufs_qcom_mcq_config_resource(struct ufs_hba *hba) +{ + struct platform_device *pdev = to_platform_device(hba->dev); + struct ufshcd_res_info *res; + struct resource *res_mem, *res_mcq; + int i, ret = 0; + + memcpy(hba->res, ufs_res_info, sizeof(ufs_res_info)); + + for (i = 0; i < RES_MAX; i++) { + res = &hba->res[i]; + res->resource = platform_get_resource_byname(pdev, + IORESOURCE_MEM, + res->name); + if (!res->resource) { + dev_info(hba->dev, "Resource %s not provided\n", res->name); + if (i == RES_UFS) + return -ENOMEM; + continue; + } else if (i == RES_UFS) { + res_mem = res->resource; + res->base = hba->mmio_base; + continue; + } + + res->base = devm_ioremap_resource(hba->dev, res->resource); + if (IS_ERR(res->base)) { + dev_err(hba->dev, "Failed to map res %s, err=%d\n", + res->name, (int)PTR_ERR(res->base)); + res->base = NULL; + ret = PTR_ERR(res->base); + return ret; + } + } + + /* MCQ resource provided in DT */ + res = &hba->res[RES_MCQ]; + /* Bail if MCQ resource is provided */ + if (res->base) + goto out; + + /* Explicitly allocate MCQ resource from ufs_mem */ + res_mcq = devm_kzalloc(hba->dev, sizeof(*res_mcq), GFP_KERNEL); + if (!res_mcq) + return ret; + + res_mcq->start = res_mem->start + + MCQ_SQATTR_OFFSET(hba->mcq_capabilities); + res_mcq->end = res_mcq->start + hba->nr_hw_queues * MCQ_QCFG_SIZE - 1; + res_mcq->flags = res_mem->flags; + res_mcq->name = "mcq"; + + ret = insert_resource(&iomem_resource, res_mcq); + if (ret) { + dev_err(hba->dev, "Failed to insert MCQ resource, err=%d\n", + ret); + goto insert_res_err; + } + + res->base = devm_ioremap_resource(hba->dev, res_mcq); + if (IS_ERR(res->base)) { + dev_err(hba->dev, "MCQ registers mapping failed, err=%d\n", + (int)PTR_ERR(res->base)); + ret = PTR_ERR(res->base); + goto ioremap_err; + } + +out: + hba->mcq_base = res->base; + return 0; +ioremap_err: + res->base = NULL; + remove_resource(res_mcq); +insert_res_err: + devm_kfree(hba->dev, res_mcq); + return ret; +} + /* * struct ufs_hba_qcom_vops - UFS QCOM specific variant operations * @@ -1447,6 +1547,7 @@ static const struct ufs_hba_variant_ops ufs_hba_qcom_vops = { .device_reset = ufs_qcom_device_reset, .config_scaling_param = ufs_qcom_config_scaling_param, .program_key = ufs_qcom_ice_program_key, + .mcq_config_resource = ufs_qcom_mcq_config_resource, }; /** diff --git a/include/ufs/ufshcd.h b/include/ufs/ufshcd.h index 146b613..0e21a6a 100644 --- a/include/ufs/ufshcd.h +++ b/include/ufs/ufshcd.h @@ -297,6 +297,7 @@ struct ufs_pwr_mode_info { * @config_scaling_param: called to configure clock scaling parameters * @program_key: program or evict an inline encryption key * @event_notify: called to notify important events + * @mcq_config_resource: called to configure MCQ platform resources */ struct ufs_hba_variant_ops { const char *name; @@ -335,6 +336,7 @@ struct ufs_hba_variant_ops { const union ufs_crypto_cfg_entry *cfg, int slot); void (*event_notify)(struct ufs_hba *hba, enum ufs_event_type evt, void *data); + int (*mcq_config_resource)(struct ufs_hba *hba); }; /* clock gating state */ @@ -724,6 +726,30 @@ struct ufs_hba_monitor { }; /** + * struct ufshcd_res_info_t - MCQ related resource regions + * + * @name: resource name + * @resource: pointer to resource region + * @base: register base address + */ +struct ufshcd_res_info { + const char *name; + struct resource *resource; + void __iomem *base; +}; + +enum ufshcd_res { + RES_UFS, + RES_MCQ, + RES_MCQ_SQD, + RES_MCQ_SQIS, + RES_MCQ_CQD, + RES_MCQ_CQIS, + RES_MCQ_VS, + RES_MAX, +}; + +/** * struct ufs_hba - per adapter private structure * @mmio_base: UFSHCI base register address * @ucdl_base_addr: UFS Command Descriptor base address @@ -835,6 +861,8 @@ struct ufs_hba_monitor { * ufshcd_resume_complete() * @ext_iid_sup: is EXT_IID is supported by UFSHC * @mcq_sup: is mcq supported by UFSHC + * @res: array of resource info of MCQ registers + * @mcq_base: Multi circular queue registers base address */ struct ufs_hba { void __iomem *mmio_base; @@ -988,6 +1016,8 @@ struct ufs_hba { bool complete_put; bool ext_iid_sup; bool mcq_sup; + struct ufshcd_res_info res[RES_MAX]; + void __iomem *mcq_base; }; /* Returns true if clocks can be gated. Otherwise false */ From patchwork Thu Dec 8 23:18:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Asutosh Das X-Patchwork-Id: 31531 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp474824wrr; Thu, 8 Dec 2022 15:26:36 -0800 (PST) X-Google-Smtp-Source: AA0mqf4MTFy9XS58BL/lpc+AVBygHmglTEc3eYMgtLng604SYlIrXBLcT5c+4q13lLxK3Ad0eZC9 X-Received: by 2002:a17:902:b68c:b0:188:63e1:bafc with SMTP id c12-20020a170902b68c00b0018863e1bafcmr3806232pls.0.1670541996160; Thu, 08 Dec 2022 15:26:36 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670541996; cv=none; d=google.com; s=arc-20160816; b=EXs/YmloyFiR/3AqTYaaB3Ug2Cx0T1fz2imN2fTJdiLYmAP4pACQS1Nvp/IRqc+Jtp rQiNGo98e+wYlqz2WMSKyTbkUCM2kbcHCF1uaLmGw3sklcxHqFBu7/zQbQHPFr8Yp2fy O5fV+7eujTw1YPtajaWW+AiGMoMgi91sxPXyZzdcbqk+W8rwxw69IKNpl/EzB2cWWKgH c4wyg+2x3m1mAEbUND8g/snG1sz7BOTpecXn7t5QBnN55glJ7AtGm7IG7/chcl67yugk KA5gpc9ZluJI3t32TdVVSvfzmda8GICXdoUut/WbfDspWn6FE6yKTFdLUGVwHzw3arkF B9rw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=Q4zhnEaUSNIwgRgYc8JFmnlUC0QDACvBhC+35B4jMXo=; b=d0xpZfRvfSj7tJ4RIx1RQOllPmESKK9hz3UhdY2hAk+4fBCk1hntmiJIc5PuedzMMK jCEhlPL2bn8khV6shjbfrMKyi2+B0fzQyeiD6TBWd7t3OQMzUGQTx5KKfZLx+HLicOy2 XvH8Dluh38pVuMuqN2e75mhf/El+JWG6Wf0cjcIXSSE/f3pdbtHsADZxqMTLvqov/4L/ yEtVOl7KsN3feylRb867besKlnozJgO0HToSMc2OAQLiSdS5nsRCnwZ5eqNbsZ4UEel+ ewE1vxp/AAAF2ECYp/vQrMnpkLtB+rBBSsY4iFIn04bZ1txNxzUkNiG9WuX6Lcc/q3m8 6Thg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=R52ED5Gl; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id u9-20020a170903124900b00187242da09asi28153642plh.265.2022.12.08.15.26.22; Thu, 08 Dec 2022 15:26:36 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=R52ED5Gl; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230144AbiLHXVQ (ORCPT + 99 others); Thu, 8 Dec 2022 18:21:16 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60022 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230138AbiLHXUv (ORCPT ); Thu, 8 Dec 2022 18:20:51 -0500 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 039A5A65BC; Thu, 8 Dec 2022 15:20:32 -0800 (PST) Received: from pps.filterd (m0279870.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2B8M54iG014501; Thu, 8 Dec 2022 23:20:01 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=qcppdkim1; bh=Q4zhnEaUSNIwgRgYc8JFmnlUC0QDACvBhC+35B4jMXo=; b=R52ED5GlaG87hnDafpLzNU8HQJD5XreDUXSjMx/aRSdmiXs8S8fy+cnIynWjZguDuQuL ZjVDcQ6naZa5v/MkEEWzzA4+87QxXhT6WqB2LYxGPf5ecTeo6P3+OcFILgG7r3NZQ2vw kLXXuM2zCu7HJmOe16bQamDd7ALQtSKx8xZOUGXA8wt76zkmpfp3z5bE1TorwBz5Nyja EOFB95p/sqnsdDlyU7d7hmeZUbWeeRZcS0ZHF4TIipqRSM4S9f5cg5jO27Dxup9ZEVij VI4Y0X6wYt9ltkbcDqsjm4q8hF6F3llQ4KOdx7FjOeAk/B4Z1Nmpp2+SJ5Hq0nMKwi1k EQ== Received: from nasanppmta01.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3mbr1hg50h-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 08 Dec 2022 23:20:01 +0000 Received: from nasanex01a.na.qualcomm.com (corens_vlan604_snip.qualcomm.com [10.53.140.1]) by NASANPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 2B8NK0Yi031194 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 8 Dec 2022 23:20:00 GMT Received: from asutoshd-linux1.qualcomm.com (10.80.80.8) by nasanex01a.na.qualcomm.com (10.52.223.231) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Thu, 8 Dec 2022 15:19:59 -0800 From: Asutosh Das To: , , CC: , , , , , , , , , Asutosh Das , , Alim Akhtar , "James E.J. Bottomley" , "Andy Gross" , Bjorn Andersson , "Konrad Dybcio" , Arthur Simchaev , Krzysztof Kozlowski , Jinyoung Choi , Yoshihiro Shimoda , Keoseong Park , Kiwoong Kim , open list Subject: [PATCH v11 07/16] ufs: core: mcq: Calculate queue depth Date: Thu, 8 Dec 2022 15:18:33 -0800 Message-ID: X-Mailer: git-send-email 2.7.4 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01b.na.qualcomm.com (10.46.141.250) To nasanex01a.na.qualcomm.com (10.52.223.231) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: NCOZ66GEQY8lBaf9cseCYtTNsKfgH65J X-Proofpoint-GUID: NCOZ66GEQY8lBaf9cseCYtTNsKfgH65J X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-08_12,2022-12-08_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 priorityscore=1501 mlxlogscore=999 clxscore=1015 spamscore=0 lowpriorityscore=0 bulkscore=0 mlxscore=0 suspectscore=0 impostorscore=0 phishscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000 definitions=main-2212080192 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751690244341653452?= X-GMAIL-MSGID: =?utf-8?q?1751690244341653452?= The ufs device defines the supported queuedepth by bqueuedepth which has a max value of 256. The HC defines MAC (Max Active Commands) that define the max number of commands that in flight to the ufs device. Calculate and configure the nutrs based on both these values. Co-developed-by: Can Guo Signed-off-by: Can Guo Signed-off-by: Asutosh Das Reviewed-by: Manivannan Sadhasivam Reviewed-by: Bart Van Assche Reviewed-by: Stanley Chu --- drivers/ufs/core/ufs-mcq.c | 35 +++++++++++++++++++++++++++++++++++ drivers/ufs/core/ufshcd-priv.h | 9 +++++++++ drivers/ufs/core/ufshcd.c | 17 ++++++++++++++++- drivers/ufs/host/ufs-qcom.c | 7 +++++++ drivers/ufs/host/ufs-qcom.h | 1 + include/ufs/ufs.h | 2 ++ include/ufs/ufshcd.h | 2 ++ include/ufs/ufshci.h | 1 + 8 files changed, 73 insertions(+), 1 deletion(-) diff --git a/drivers/ufs/core/ufs-mcq.c b/drivers/ufs/core/ufs-mcq.c index 65c0037..2f680ff 100644 --- a/drivers/ufs/core/ufs-mcq.c +++ b/drivers/ufs/core/ufs-mcq.c @@ -19,6 +19,9 @@ #define UFS_MCQ_NUM_DEV_CMD_QUEUES 1 #define UFS_MCQ_MIN_POLL_QUEUES 0 +#define MAX_DEV_CMD_ENTRIES 2 +#define MCQ_CFG_MAC_MASK GENMASK(16, 8) + static int rw_queue_count_set(const char *val, const struct kernel_param *kp) { return param_set_uint_minmax(val, kp, UFS_MCQ_MIN_RW_QUEUES, @@ -67,6 +70,38 @@ module_param_cb(poll_queues, &poll_queue_count_ops, &poll_queues, 0644); MODULE_PARM_DESC(poll_queues, "Number of poll queues used for r/w. Default value is 1"); +/** + * ufshcd_mcq_decide_queue_depth - decide the queue depth + * @hba - per adapter instance + * + * Returns queue-depth on success, non-zero on error + * + * MAC - Max. Active Command of the Host Controller (HC) + * HC wouldn't send more than this commands to the device. + * It is mandatory to implement get_hba_mac() to enable MCQ mode. + * Calculates and adjusts the queue depth based on the depth + * supported by the HC and ufs device. + */ +int ufshcd_mcq_decide_queue_depth(struct ufs_hba *hba) +{ + int mac; + + /* Mandatory to implement get_hba_mac() */ + mac = ufshcd_mcq_vops_get_hba_mac(hba); + if (mac < 0) { + dev_err(hba->dev, "Failed to get mac, err=%d\n", mac); + return mac; + } + + WARN_ON_ONCE(!hba->dev_info.bqueuedepth); + /* + * max. value of bqueuedepth = 256, mac is host dependent. + * It is mandatory for UFS device to define bQueueDepth if + * shared queuing architecture is enabled. + */ + return min_t(int, mac, hba->dev_info.bqueuedepth); +} + static int ufshcd_mcq_config_nr_queues(struct ufs_hba *hba) { int i; diff --git a/drivers/ufs/core/ufshcd-priv.h b/drivers/ufs/core/ufshcd-priv.h index 3e21242..d9b2087 100644 --- a/drivers/ufs/core/ufshcd-priv.h +++ b/drivers/ufs/core/ufshcd-priv.h @@ -62,6 +62,7 @@ int ufshcd_query_flag(struct ufs_hba *hba, enum query_opcode opcode, enum flag_idn idn, u8 index, bool *flag_res); void ufshcd_auto_hibern8_update(struct ufs_hba *hba, u32 ahit); int ufshcd_mcq_init(struct ufs_hba *hba); +int ufshcd_mcq_decide_queue_depth(struct ufs_hba *hba); #define SD_ASCII_STD true #define SD_RAW false @@ -235,6 +236,14 @@ static inline int ufshcd_vops_mcq_config_resource(struct ufs_hba *hba) return -EOPNOTSUPP; } +static inline int ufshcd_mcq_vops_get_hba_mac(struct ufs_hba *hba) +{ + if (hba->vops && hba->vops->get_hba_mac) + return hba->vops->get_hba_mac(hba); + + return -EOPNOTSUPP; +} + extern const struct ufs_pm_lvl_states ufs_pm_lvl_states[]; /** diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c index 4e2355b..8218b2a 100644 --- a/drivers/ufs/core/ufshcd.c +++ b/drivers/ufs/core/ufshcd.c @@ -7807,6 +7807,7 @@ static int ufs_get_device_desc(struct ufs_hba *hba) /* getting Specification Version in big endian format */ dev_info->wspecversion = desc_buf[DEVICE_DESC_PARAM_SPEC_VER] << 8 | desc_buf[DEVICE_DESC_PARAM_SPEC_VER + 1]; + dev_info->bqueuedepth = desc_buf[DEVICE_DESC_PARAM_Q_DPTH]; b_ufs_feature_sup = desc_buf[DEVICE_DESC_PARAM_UFS_FEAT]; model_index = desc_buf[DEVICE_DESC_PARAM_PRDCT_NAME]; @@ -8222,7 +8223,21 @@ static int ufshcd_add_lus(struct ufs_hba *hba) static int ufshcd_alloc_mcq(struct ufs_hba *hba) { - return ufshcd_mcq_init(hba); + int ret; + int old_nutrs = hba->nutrs; + + ret = ufshcd_mcq_decide_queue_depth(hba); + if (ret < 0) + return ret; + + hba->nutrs = ret; + ret = ufshcd_mcq_init(hba); + if (ret) { + hba->nutrs = old_nutrs; + return ret; + } + + return 0; } /** diff --git a/drivers/ufs/host/ufs-qcom.c b/drivers/ufs/host/ufs-qcom.c index 045f8ab..a52db10 100644 --- a/drivers/ufs/host/ufs-qcom.c +++ b/drivers/ufs/host/ufs-qcom.c @@ -1524,6 +1524,12 @@ static int ufs_qcom_mcq_config_resource(struct ufs_hba *hba) return ret; } +static int ufs_qcom_get_hba_mac(struct ufs_hba *hba) +{ + /* Qualcomm HC supports up to 64 */ + return MAX_SUPP_MAC; +} + /* * struct ufs_hba_qcom_vops - UFS QCOM specific variant operations * @@ -1548,6 +1554,7 @@ static const struct ufs_hba_variant_ops ufs_hba_qcom_vops = { .config_scaling_param = ufs_qcom_config_scaling_param, .program_key = ufs_qcom_ice_program_key, .mcq_config_resource = ufs_qcom_mcq_config_resource, + .get_hba_mac = ufs_qcom_get_hba_mac, }; /** diff --git a/drivers/ufs/host/ufs-qcom.h b/drivers/ufs/host/ufs-qcom.h index 44466a3..f86e532 100644 --- a/drivers/ufs/host/ufs-qcom.h +++ b/drivers/ufs/host/ufs-qcom.h @@ -16,6 +16,7 @@ #define HBRN8_POLL_TOUT_MS 100 #define DEFAULT_CLK_RATE_HZ 1000000 #define BUS_VECTOR_NAME_LEN 32 +#define MAX_SUPP_MAC 64 #define UFS_HW_VER_MAJOR_SHFT (28) #define UFS_HW_VER_MAJOR_MASK (0x000F << UFS_HW_VER_MAJOR_SHFT) diff --git a/include/ufs/ufs.h b/include/ufs/ufs.h index ba2a1d8..5112418 100644 --- a/include/ufs/ufs.h +++ b/include/ufs/ufs.h @@ -591,6 +591,8 @@ struct ufs_dev_info { u8 *model; u16 wspecversion; u32 clk_gating_wait_us; + /* Stores the depth of queue in UFS device */ + u8 bqueuedepth; /* UFS HPB related flag */ bool hpb_enabled; diff --git a/include/ufs/ufshcd.h b/include/ufs/ufshcd.h index 0e21a6a..9d7829a 100644 --- a/include/ufs/ufshcd.h +++ b/include/ufs/ufshcd.h @@ -298,6 +298,7 @@ struct ufs_pwr_mode_info { * @program_key: program or evict an inline encryption key * @event_notify: called to notify important events * @mcq_config_resource: called to configure MCQ platform resources + * @get_hba_mac: called to get vendor specific mac value, mandatory for mcq mode */ struct ufs_hba_variant_ops { const char *name; @@ -337,6 +338,7 @@ struct ufs_hba_variant_ops { void (*event_notify)(struct ufs_hba *hba, enum ufs_event_type evt, void *data); int (*mcq_config_resource)(struct ufs_hba *hba); + int (*get_hba_mac)(struct ufs_hba *hba); }; /* clock gating state */ diff --git a/include/ufs/ufshci.h b/include/ufs/ufshci.h index 4d4da06..67fcebd 100644 --- a/include/ufs/ufshci.h +++ b/include/ufs/ufshci.h @@ -57,6 +57,7 @@ enum { REG_UFS_CCAP = 0x100, REG_UFS_CRYPTOCAP = 0x104, + REG_UFS_MCQ_CFG = 0x380, UFSHCI_CRYPTO_REG_SPACE_SIZE = 0x400, }; From patchwork Thu Dec 8 23:18:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Asutosh Das X-Patchwork-Id: 31530 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp474400wrr; Thu, 8 Dec 2022 15:25:26 -0800 (PST) X-Google-Smtp-Source: AA0mqf7ZgfP2PlOpUtiQXvvxCT9QqqFiwbrY1BeFkum2LekFkjwkDWEN19eVpbK0XFS7P0oPkrAe X-Received: by 2002:a17:906:58a:b0:78d:f455:c3a7 with SMTP id 10-20020a170906058a00b0078df455c3a7mr2828284ejn.77.1670541926659; Thu, 08 Dec 2022 15:25:26 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670541926; cv=none; d=google.com; s=arc-20160816; b=PWylhkUnMyQO2mMS0aVJXqrHYaKohqOdOcGl3zAQVwGO3QlJgsMJfbf9oKExB7DCNa Tg3Tjk/LH0lGj5N1ED70HNXJowM0fVqSzdM66eyZVIKgQQG+bfq63rYqxiL3UzkHCQ27 eplxvz5Bo5T64SFS66rgR/nrrSIsHX5HBjsfoa3qfuqYH5nWw+YSvGFPX23Y4IHaJ0Fw JvGn1phRUZ7y5rS3hdCNClsoVBetjIo5Z8v/4yN1C79FLqezi4mKgu0VynzS2Aa/WyGF nk1NrtlEwaodxMqOskyejGI/EcugwS4FU0G3u4WynsUMDgzRhJxVk23TeB3YAWS6APPP z8Zw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=8EEHPJSMhn3hc7LJEQnlAQmMjnTVkHU5n4um4Y9V7Uk=; b=fuFn8PT+pktLUO0mUn/HxlZWntgQABH0NTxtGVMj4okIi6gDsik7glkvvDpxQ2wpIo Elzgp/VbAuiVtUhv4Fc2pQ+IqNMhzbSeNtVZ4f7XV9nqEPPPzEs1odlnlF4qrp2JcvvQ NLH3Nrae0inhN0dYwBSQoABQc2UMXLofHoNglUw/l70M3oYUenOlDkbm6fzeTkGaYDS1 7CQqDGcsk3oiTh9aJoZHdyCLvKb+GKEywOdFE5gXkLPXu/40w6YqSLBixssn37YgLD74 arfVwdlzflyXasWNhBLrrKEJ6gfU2RDt4p3hDLTCThN11KcKZnfpSrsISV63kirNt9q9 ObwQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=mzOBPZ5v; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id he18-20020a1709073d9200b0078dc5c888f1si18397950ejc.135.2022.12.08.15.25.03; Thu, 08 Dec 2022 15:25:26 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=mzOBPZ5v; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230223AbiLHXVL (ORCPT + 99 others); Thu, 8 Dec 2022 18:21:11 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60004 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230124AbiLHXUt (ORCPT ); Thu, 8 Dec 2022 18:20:49 -0500 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9EF98A504D; Thu, 8 Dec 2022 15:20:32 -0800 (PST) Received: from pps.filterd (m0279866.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2B8MnDcc024944; Thu, 8 Dec 2022 23:20:17 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=qcppdkim1; bh=8EEHPJSMhn3hc7LJEQnlAQmMjnTVkHU5n4um4Y9V7Uk=; b=mzOBPZ5v3RlR/SWyL15RAKnQ9MudNU6kKMC77Zq5No52+AlpyUKUk997TEP1ZhvRzN4J OH4P1PjbRIGj+83R2P9KGIdpd0klwwOSnYy8CW3JnM2Y+bbp/+7VZDG2C6yNGSYizrde wmWb4hY01g3PZH9u0TT/FOveUqrh2QGLkKGwGGYK8EnPIawC5HptHsDRKv3IzIuAjdFz +LZBwVRmB3rGPyKYWp5PODCY/QkIEyvPG7N9ZED0yS3fSe2wlUAWwgbtSp/o2xuAS5tt S8L9L3GbWgMZWdfkHdGxk+mYY/SExGfFId/gT6E3czP9WWuKnkrp1CHr8jxGvhVIGlv1 3Q== Received: from nasanppmta04.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3mbbd6tgda-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 08 Dec 2022 23:20:17 +0000 Received: from nasanex01a.na.qualcomm.com ([10.52.223.231]) by NASANPPMTA04.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 2B8NKGxX029466 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 8 Dec 2022 23:20:16 GMT Received: from asutoshd-linux1.qualcomm.com (10.80.80.8) by nasanex01a.na.qualcomm.com (10.52.223.231) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Thu, 8 Dec 2022 15:20:15 -0800 From: Asutosh Das To: , , CC: , , , , , , , , , Asutosh Das , , Alim Akhtar , "James E.J. Bottomley" , Jinyoung Choi , Krzysztof Kozlowski , Keoseong Park , Yoshihiro Shimoda , Kiwoong Kim , open list Subject: [PATCH v11 08/16] ufs: core: mcq: Allocate memory for mcq mode Date: Thu, 8 Dec 2022 15:18:34 -0800 Message-ID: X-Mailer: git-send-email 2.7.4 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01b.na.qualcomm.com (10.46.141.250) To nasanex01a.na.qualcomm.com (10.52.223.231) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: p8slfDqhRGq1QDq3coDQOBfbOHKcUaDU X-Proofpoint-GUID: p8slfDqhRGq1QDq3coDQOBfbOHKcUaDU X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-08_12,2022-12-08_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxlogscore=999 adultscore=0 mlxscore=0 clxscore=1015 suspectscore=0 malwarescore=0 bulkscore=0 phishscore=0 priorityscore=1501 spamscore=0 impostorscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000 definitions=main-2212080192 X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751690171071597143?= X-GMAIL-MSGID: =?utf-8?q?1751690171071597143?= To read the bqueuedepth, the device descriptor is fetched in Single Doorbell Mode. This allocated memory may not be enough for MCQ mode because the number of tags supported in MCQ mode may be larger than in SDB mode. Hence, release the memory allocated in SDB mode and allocate memory for MCQ mode operation. Define the ufs hardware queue and Completion Queue Entry. Co-developed-by: Can Guo Signed-off-by: Can Guo Signed-off-by: Asutosh Das Reviewed-by: Manivannan Sadhasivam Reviewed-by: Bart Van Assche --- drivers/ufs/core/ufs-mcq.c | 59 ++++++++++++++++++++++++++++++++++++++++-- drivers/ufs/core/ufshcd-priv.h | 1 + drivers/ufs/core/ufshcd.c | 48 +++++++++++++++++++++++++++++++--- include/ufs/ufshcd.h | 20 ++++++++++++++ include/ufs/ufshci.h | 22 ++++++++++++++++ 5 files changed, 145 insertions(+), 5 deletions(-) diff --git a/drivers/ufs/core/ufs-mcq.c b/drivers/ufs/core/ufs-mcq.c index 2f680ff..c77bc54 100644 --- a/drivers/ufs/core/ufs-mcq.c +++ b/drivers/ufs/core/ufs-mcq.c @@ -149,14 +149,69 @@ static int ufshcd_mcq_config_nr_queues(struct ufs_hba *hba) return 0; } +int ufshcd_mcq_memory_alloc(struct ufs_hba *hba) +{ + struct ufs_hw_queue *hwq; + size_t utrdl_size, cqe_size; + int i; + + for (i = 0; i < hba->nr_hw_queues; i++) { + hwq = &hba->uhq[i]; + + utrdl_size = sizeof(struct utp_transfer_req_desc) * + hwq->max_entries; + hwq->sqe_base_addr = dmam_alloc_coherent(hba->dev, utrdl_size, + &hwq->sqe_dma_addr, + GFP_KERNEL); + if (!hwq->sqe_dma_addr) { + dev_err(hba->dev, "SQE allocation failed\n"); + return -ENOMEM; + } + + cqe_size = sizeof(struct cq_entry) * hwq->max_entries; + hwq->cqe_base_addr = dmam_alloc_coherent(hba->dev, cqe_size, + &hwq->cqe_dma_addr, + GFP_KERNEL); + if (!hwq->cqe_dma_addr) { + dev_err(hba->dev, "CQE allocation failed\n"); + return -ENOMEM; + } + } + + return 0; +} + + int ufshcd_mcq_init(struct ufs_hba *hba) { - int ret; + struct ufs_hw_queue *hwq; + int ret, i; ret = ufshcd_mcq_config_nr_queues(hba); if (ret) return ret; ret = ufshcd_vops_mcq_config_resource(hba); - return ret; + if (ret) + return ret; + + hba->uhq = devm_kzalloc(hba->dev, + hba->nr_hw_queues * sizeof(struct ufs_hw_queue), + GFP_KERNEL); + if (!hba->uhq) { + dev_err(hba->dev, "ufs hw queue memory allocation failed\n"); + return -ENOMEM; + } + + for (i = 0; i < hba->nr_hw_queues; i++) { + hwq = &hba->uhq[i]; + hwq->max_entries = hba->nutrs; + } + + /* The very first HW queue serves device commands */ + hba->dev_cmd_queue = &hba->uhq[0]; + /* Give dev_cmd_queue the minimal number of entries */ + hba->dev_cmd_queue->max_entries = MAX_DEV_CMD_ENTRIES; + + return 0; } diff --git a/drivers/ufs/core/ufshcd-priv.h b/drivers/ufs/core/ufshcd-priv.h index d9b2087..93ebfec 100644 --- a/drivers/ufs/core/ufshcd-priv.h +++ b/drivers/ufs/core/ufshcd-priv.h @@ -63,6 +63,7 @@ int ufshcd_query_flag(struct ufs_hba *hba, enum query_opcode opcode, void ufshcd_auto_hibern8_update(struct ufs_hba *hba, u32 ahit); int ufshcd_mcq_init(struct ufs_hba *hba); int ufshcd_mcq_decide_queue_depth(struct ufs_hba *hba); +int ufshcd_mcq_memory_alloc(struct ufs_hba *hba); #define SD_ASCII_STD true #define SD_RAW false diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c index 8218b2a..f2571eb 100644 --- a/drivers/ufs/core/ufshcd.c +++ b/drivers/ufs/core/ufshcd.c @@ -3740,6 +3740,14 @@ static int ufshcd_memory_alloc(struct ufs_hba *hba) } /* + * Skip utmrdl allocation; it may have been + * allocated during first pass and not released during + * MCQ memory allocation. + * See ufshcd_release_sdb_queue() and ufshcd_config_mcq() + */ + if (hba->utmrdl_base_addr) + goto skip_utmrdl; + /* * Allocate memory for UTP Task Management descriptors * UFSHCI requires 1024 byte alignment of UTMRD */ @@ -3755,6 +3763,7 @@ static int ufshcd_memory_alloc(struct ufs_hba *hba) goto out; } +skip_utmrdl: /* Allocate memory for local reference block */ hba->lrb = devm_kcalloc(hba->dev, hba->nutrs, sizeof(struct ufshcd_lrb), @@ -8221,6 +8230,22 @@ static int ufshcd_add_lus(struct ufs_hba *hba) return ret; } +/* SDB - Single Doorbell */ +static void ufshcd_release_sdb_queue(struct ufs_hba *hba, int nutrs) +{ + size_t ucdl_size, utrdl_size; + + ucdl_size = sizeof(struct utp_transfer_cmd_desc) * nutrs; + dmam_free_coherent(hba->dev, ucdl_size, hba->ucdl_base_addr, + hba->ucdl_dma_addr); + + utrdl_size = sizeof(struct utp_transfer_req_desc) * nutrs; + dmam_free_coherent(hba->dev, utrdl_size, hba->utrdl_base_addr, + hba->utrdl_dma_addr); + + devm_kfree(hba->dev, hba->lrb); +} + static int ufshcd_alloc_mcq(struct ufs_hba *hba) { int ret; @@ -8232,12 +8257,29 @@ static int ufshcd_alloc_mcq(struct ufs_hba *hba) hba->nutrs = ret; ret = ufshcd_mcq_init(hba); - if (ret) { - hba->nutrs = old_nutrs; - return ret; + if (ret) + goto err; + + /* + * Previously allocated memory for nutrs may not be enough in MCQ mode. + * Number of supported tags in MCQ mode may be larger than SDB mode. + */ + if (hba->nutrs != old_nutrs) { + ufshcd_release_sdb_queue(hba, old_nutrs); + ret = ufshcd_memory_alloc(hba); + if (ret) + goto err; + ufshcd_host_memory_configure(hba); } + ret = ufshcd_mcq_memory_alloc(hba); + if (ret) + goto err; + return 0; +err: + hba->nutrs = old_nutrs; + return ret; } /** diff --git a/include/ufs/ufshcd.h b/include/ufs/ufshcd.h index 9d7829a..90461f43 100644 --- a/include/ufs/ufshcd.h +++ b/include/ufs/ufshcd.h @@ -865,6 +865,8 @@ enum ufshcd_res { * @mcq_sup: is mcq supported by UFSHC * @res: array of resource info of MCQ registers * @mcq_base: Multi circular queue registers base address + * @uhq: array of supported hardware queues + * @dev_cmd_queue: Queue for issuing device management commands */ struct ufs_hba { void __iomem *mmio_base; @@ -1020,6 +1022,24 @@ struct ufs_hba { bool mcq_sup; struct ufshcd_res_info res[RES_MAX]; void __iomem *mcq_base; + struct ufs_hw_queue *uhq; + struct ufs_hw_queue *dev_cmd_queue; +}; + +/** + * struct ufs_hw_queue - per hardware queue structure + * @sqe_base_addr: submission queue entry base address + * @sqe_dma_addr: submission queue dma address + * @cqe_base_addr: completion queue base address + * @cqe_dma_addr: completion queue dma address + * @max_entries: max number of slots in this hardware queue + */ +struct ufs_hw_queue { + void *sqe_base_addr; + dma_addr_t sqe_dma_addr; + struct cq_entry *cqe_base_addr; + dma_addr_t cqe_dma_addr; + u32 max_entries; }; /* Returns true if clocks can be gated. Otherwise false */ diff --git a/include/ufs/ufshci.h b/include/ufs/ufshci.h index 67fcebd..15d1ea2 100644 --- a/include/ufs/ufshci.h +++ b/include/ufs/ufshci.h @@ -486,6 +486,28 @@ struct utp_transfer_req_desc { __le16 prd_table_offset; }; +/* MCQ Completion Queue Entry */ +struct cq_entry { + /* DW 0-1 */ + __le64 command_desc_base_addr; + + /* DW 2 */ + __le16 response_upiu_length; + __le16 response_upiu_offset; + + /* DW 3 */ + __le16 prd_table_length; + __le16 prd_table_offset; + + /* DW 4 */ + __le32 status; + + /* DW 5-7 */ + __le32 reserved[3]; +}; + +static_assert(sizeof(struct cq_entry) == 32); + /* * UTMRD structure. */ From patchwork Thu Dec 8 23:18:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Asutosh Das X-Patchwork-Id: 31536 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp477651wrr; Thu, 8 Dec 2022 15:34:24 -0800 (PST) X-Google-Smtp-Source: AA0mqf7yMuIMjJE7SyMD8++lOlXy+9oYSHFXJN6+w2nrDShQiEaMREkQgzFvmsRM9osBlcUJ9kV8 X-Received: by 2002:a17:90a:ca16:b0:219:d636:dd82 with SMTP id x22-20020a17090aca1600b00219d636dd82mr3598908pjt.4.1670542463756; Thu, 08 Dec 2022 15:34:23 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670542463; cv=none; d=google.com; s=arc-20160816; b=zCmAEwGsHYo1u7yWexptOAwPVs/l1V0lxq4WW7QORoxlR8kvrEAadBLmtc84Ww0VCC cEVu7QF8x2Z1QDGJFa+RS1Titi9zU9ifAiYYQuJ7l6+3o2uvQKAfrKVU3fKmqEpxJo7l X38DhsbEhmXyyu3KX6ThyBdbQKrVyBFJbTHHR2A4R6UsdCcAdtCXJitQheNWEWk2JKsU lzuSHj6UezXbFz5rntbiSSgxiXOnxgcCvJnb+Jfqjng2nA3wO5crHhXQ/F05ZEQ24yiB aXlOh1GXf/1BzufvDUpGIsJJb2ISlcvo3OVaP9lHo856mM5slYYurjzckALokUiMvNi2 0TQQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=qccsNrb5j9fofD9FI295IKfJhufvTATYhOsqNDxKxjQ=; b=rn3o/JP2h+488RtnX3RRJTM3i27JlfoLdcXJNgMv5pHBa9NsEhR8Sg8K2MaTXjjBYv T1V4HlnkuNWzSUdtbShP1RaUOLGqVt9YfF8ydC6mcm2eDrVcgsN+v9E7WuKJY6DgaMNF 2hlN2kY87B96ZVMboVyNZi9d6/UZy/yW5iFpD0Ts/X0O8+cVM7EaITtHO2MxgETlKGRH ftWXfIvB5LnTKGicE1NXH3xbNOMLgxyDTqOg6QmXFkQ8q2KDqOzv+EvghSNm+4KWxXkJ KK4XYaYTUYTEhwJ5G5W2ETJ2wUGfAf8fQUxhVyyBC+U94oe4LHDZFdFPRihjARbP6+5V pc5A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=hFB5ZEAX; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id mw4-20020a17090b4d0400b0021a1b4ecfdbsi612400pjb.63.2022.12.08.15.34.01; Thu, 08 Dec 2022 15:34:23 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=hFB5ZEAX; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229996AbiLHXWA (ORCPT + 99 others); Thu, 8 Dec 2022 18:22:00 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58948 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230208AbiLHXVE (ORCPT ); Thu, 8 Dec 2022 18:21:04 -0500 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 84BADA0F8B; Thu, 8 Dec 2022 15:20:48 -0800 (PST) Received: from pps.filterd (m0279863.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2B8MpIvR005290; Thu, 8 Dec 2022 23:20:33 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=qcppdkim1; bh=qccsNrb5j9fofD9FI295IKfJhufvTATYhOsqNDxKxjQ=; b=hFB5ZEAXy4kEsHc/N2cZBLkjgA3lWxkpWfprqRZwPd/1jEEivl29S5P5P3OnNoQNExHv anZL39qtfdMF+2n4krFCxv6z6GzDS0gZnPXWD1GLDJ9rWkJ2J+zlw4eVo44OTccZ80cM /VliDD+Enq+9AEBM9UKPfKusaRMUApuWn52eTHQIybNy6cdpIDUrS2Z7BBf9IxeIAlzF JLe2C44Gc5z3W1hu+EOwyRfYsCAjFN8kvhRBvkdOUEaTchlzlTkXOgMcvz1Gs5K5sm5Q 5+00j2CLl99PnaEhGQTbaqZ+RbWQKOIeOiHIjnKBkxIBnUMhZ0a8EJvJzUF/I6Nz0fVs 1g== Received: from nasanppmta01.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3mavtf4y3r-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 08 Dec 2022 23:20:33 +0000 Received: from nasanex01a.na.qualcomm.com (corens_vlan604_snip.qualcomm.com [10.53.140.1]) by NASANPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 2B8NKWpW032183 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 8 Dec 2022 23:20:32 GMT Received: from asutoshd-linux1.qualcomm.com (10.80.80.8) by nasanex01a.na.qualcomm.com (10.52.223.231) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Thu, 8 Dec 2022 15:20:31 -0800 From: Asutosh Das To: , , CC: , , , , , , , , , Asutosh Das , , Alim Akhtar , "James E.J. Bottomley" , "Andy Gross" , Bjorn Andersson , "Konrad Dybcio" , Krzysztof Kozlowski , Jinyoung Choi , Yoshihiro Shimoda , Keoseong Park , Kiwoong Kim , open list Subject: [PATCH v11 09/16] ufs: core: mcq: Configure operation and runtime interface Date: Thu, 8 Dec 2022 15:18:35 -0800 Message-ID: <2110cd38478f5843841173be3f0da676cdd5ed31.1670541364.git.quic_asutoshd@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01b.na.qualcomm.com (10.46.141.250) To nasanex01a.na.qualcomm.com (10.52.223.231) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: MHGGor22kW_HDvM2iIwZ_nXzGXcCVBvt X-Proofpoint-ORIG-GUID: MHGGor22kW_HDvM2iIwZ_nXzGXcCVBvt X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-08_12,2022-12-08_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 lowpriorityscore=0 suspectscore=0 mlxscore=0 mlxlogscore=999 malwarescore=0 clxscore=1015 adultscore=0 spamscore=0 phishscore=0 bulkscore=0 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000 definitions=main-2212080192 X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751690734453173816?= X-GMAIL-MSGID: =?utf-8?q?1751690734453173816?= Runtime and operation registers are defined per Submission and Completion queue. The location of these registers is not defined in the spec; meaning the offsets and stride may vary for different HC vendors. Establish the stride, base address and doorbell address offsets from vendor host driver and program it. Co-developed-by: Can Guo Signed-off-by: Can Guo Signed-off-by: Asutosh Das Reviewed-by: Manivannan Sadhasivam Reviewed-by: Bart Van Assche --- drivers/ufs/core/ufs-mcq.c | 102 +++++++++++++++++++++++++++++++++++++++++ drivers/ufs/core/ufshcd-priv.h | 11 +++++ drivers/ufs/core/ufshcd.c | 27 +++++++++++ drivers/ufs/host/ufs-qcom.c | 24 ++++++++++ include/ufs/ufshcd.h | 52 +++++++++++++++++++++ include/ufs/ufshci.h | 31 +++++++++++++ 6 files changed, 247 insertions(+) diff --git a/drivers/ufs/core/ufs-mcq.c b/drivers/ufs/core/ufs-mcq.c index c77bc54..496e2b6 100644 --- a/drivers/ufs/core/ufs-mcq.c +++ b/drivers/ufs/core/ufs-mcq.c @@ -18,9 +18,13 @@ #define UFS_MCQ_MIN_READ_QUEUES 0 #define UFS_MCQ_NUM_DEV_CMD_QUEUES 1 #define UFS_MCQ_MIN_POLL_QUEUES 0 +#define QUEUE_EN_OFFSET 31 +#define QUEUE_ID_OFFSET 16 #define MAX_DEV_CMD_ENTRIES 2 #define MCQ_CFG_MAC_MASK GENMASK(16, 8) +#define MCQ_QCFG_SIZE 0x40 +#define MCQ_ENTRY_SIZE_IN_DWORD 8 static int rw_queue_count_set(const char *val, const struct kernel_param *kp) { @@ -71,6 +75,24 @@ MODULE_PARM_DESC(poll_queues, "Number of poll queues used for r/w. Default value is 1"); /** + * ufshcd_mcq_config_mac - Set the #Max Activ Cmds. + * @hba - per adapter instance + * @max_active_cmds - maximum # of active commands to the device at any time. + * + * The controller won't send more than the max_active_cmds to the device at + * any time. + */ +void ufshcd_mcq_config_mac(struct ufs_hba *hba, u32 max_active_cmds) +{ + u32 val; + + val = ufshcd_readl(hba, REG_UFS_MCQ_CFG); + val &= ~MCQ_CFG_MAC_MASK; + val |= FIELD_PREP(MCQ_CFG_MAC_MASK, max_active_cmds); + ufshcd_writel(hba, val, REG_UFS_MCQ_CFG); +} + +/** * ufshcd_mcq_decide_queue_depth - decide the queue depth * @hba - per adapter instance * @@ -182,6 +204,80 @@ int ufshcd_mcq_memory_alloc(struct ufs_hba *hba) } +/* Operation and runtime registers configuration */ +#define MCQ_CFG_n(r, i) ((r) + MCQ_QCFG_SIZE * (i)) +#define MCQ_OPR_OFFSET_n(p, i) \ + (hba->mcq_opr[(p)].offset + hba->mcq_opr[(p)].stride * (i)) + +static void __iomem *mcq_opr_base(struct ufs_hba *hba, + enum ufshcd_mcq_opr n, int i) +{ + struct ufshcd_mcq_opr_info_t *opr = &hba->mcq_opr[n]; + + return opr->base + opr->stride * i; +} + +void ufshcd_mcq_make_queues_operational(struct ufs_hba *hba) +{ + struct ufs_hw_queue *hwq; + u16 qsize; + int i; + + for (i = 0; i < hba->nr_hw_queues; i++) { + hwq = &hba->uhq[i]; + hwq->id = i; + qsize = hwq->max_entries * MCQ_ENTRY_SIZE_IN_DWORD - 1; + + /* Submission Queue Lower Base Address */ + ufsmcq_writelx(hba, lower_32_bits(hwq->sqe_dma_addr), + MCQ_CFG_n(REG_SQLBA, i)); + /* Submission Queue Upper Base Address */ + ufsmcq_writelx(hba, upper_32_bits(hwq->sqe_dma_addr), + MCQ_CFG_n(REG_SQUBA, i)); + /* Submission Queue Doorbell Address Offset */ + ufsmcq_writelx(hba, MCQ_OPR_OFFSET_n(OPR_SQD, i), + MCQ_CFG_n(REG_SQDAO, i)); + /* Submission Queue Interrupt Status Address Offset */ + ufsmcq_writelx(hba, MCQ_OPR_OFFSET_n(OPR_SQIS, i), + MCQ_CFG_n(REG_SQISAO, i)); + + /* Completion Queue Lower Base Address */ + ufsmcq_writelx(hba, lower_32_bits(hwq->cqe_dma_addr), + MCQ_CFG_n(REG_CQLBA, i)); + /* Completion Queue Upper Base Address */ + ufsmcq_writelx(hba, upper_32_bits(hwq->cqe_dma_addr), + MCQ_CFG_n(REG_CQUBA, i)); + /* Completion Queue Doorbell Address Offset */ + ufsmcq_writelx(hba, MCQ_OPR_OFFSET_n(OPR_CQD, i), + MCQ_CFG_n(REG_CQDAO, i)); + /* Completion Queue Interrupt Status Address Offset */ + ufsmcq_writelx(hba, MCQ_OPR_OFFSET_n(OPR_CQIS, i), + MCQ_CFG_n(REG_CQISAO, i)); + + /* Save the base addresses for quicker access */ + hwq->mcq_sq_head = mcq_opr_base(hba, OPR_SQD, i) + REG_SQHP; + hwq->mcq_sq_tail = mcq_opr_base(hba, OPR_SQD, i) + REG_SQTP; + hwq->mcq_cq_head = mcq_opr_base(hba, OPR_CQD, i) + REG_CQHP; + hwq->mcq_cq_tail = mcq_opr_base(hba, OPR_CQD, i) + REG_CQTP; + + /* Enable Tail Entry Push Status interrupt only for non-poll queues */ + if (i < hba->nr_hw_queues - hba->nr_queues[HCTX_TYPE_POLL]) + writel(1, mcq_opr_base(hba, OPR_CQIS, i) + REG_CQIE); + + /* Completion Queue Enable|Size to Completion Queue Attribute */ + ufsmcq_writel(hba, (1 << QUEUE_EN_OFFSET) | qsize, + MCQ_CFG_n(REG_CQATTR, i)); + + /* + * Submission Qeueue Enable|Size|Completion Queue ID to + * Submission Queue Attribute + */ + ufsmcq_writel(hba, (1 << QUEUE_EN_OFFSET) | qsize | + (i << QUEUE_ID_OFFSET), + MCQ_CFG_n(REG_SQATTR, i)); + } +} + int ufshcd_mcq_init(struct ufs_hba *hba) { struct ufs_hw_queue *hwq; @@ -195,6 +291,12 @@ int ufshcd_mcq_init(struct ufs_hba *hba) if (ret) return ret; + ret = ufshcd_mcq_vops_op_runtime_config(hba); + if (ret) { + dev_err(hba->dev, "Operation runtime config failed, ret=%d\n", + ret); + return ret; + } hba->uhq = devm_kzalloc(hba->dev, hba->nr_hw_queues * sizeof(struct ufs_hw_queue), GFP_KERNEL); diff --git a/drivers/ufs/core/ufshcd-priv.h b/drivers/ufs/core/ufshcd-priv.h index 93ebfec..0e35cfa 100644 --- a/drivers/ufs/core/ufshcd-priv.h +++ b/drivers/ufs/core/ufshcd-priv.h @@ -64,6 +64,9 @@ void ufshcd_auto_hibern8_update(struct ufs_hba *hba, u32 ahit); int ufshcd_mcq_init(struct ufs_hba *hba); int ufshcd_mcq_decide_queue_depth(struct ufs_hba *hba); int ufshcd_mcq_memory_alloc(struct ufs_hba *hba); +void ufshcd_mcq_make_queues_operational(struct ufs_hba *hba); +void ufshcd_mcq_config_mac(struct ufs_hba *hba, u32 max_active_cmds); +void ufshcd_mcq_select_mcq_mode(struct ufs_hba *hba); #define SD_ASCII_STD true #define SD_RAW false @@ -245,6 +248,14 @@ static inline int ufshcd_mcq_vops_get_hba_mac(struct ufs_hba *hba) return -EOPNOTSUPP; } +static inline int ufshcd_mcq_vops_op_runtime_config(struct ufs_hba *hba) +{ + if (hba->vops && hba->vops->op_runtime_config) + return hba->vops->op_runtime_config(hba); + + return -EOPNOTSUPP; +} + extern const struct ufs_pm_lvl_states ufs_pm_lvl_states[]; /** diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c index f2571eb..5d14d3f 100644 --- a/drivers/ufs/core/ufshcd.c +++ b/drivers/ufs/core/ufshcd.c @@ -43,6 +43,12 @@ #define UFSHCD_ENABLE_INTRS (UTP_TRANSFER_REQ_COMPL |\ UTP_TASK_REQ_COMPL |\ UFSHCD_ERROR_MASK) + +#define UFSHCD_ENABLE_MCQ_INTRS (UTP_TASK_REQ_COMPL |\ + UFSHCD_ERROR_MASK |\ + MCQ_CQ_EVENT_STATUS) + + /* UIC command timeout, unit: ms */ #define UIC_CMD_TIMEOUT 500 @@ -8282,6 +8288,20 @@ static int ufshcd_alloc_mcq(struct ufs_hba *hba) return ret; } +static void ufshcd_config_mcq(struct ufs_hba *hba) +{ + ufshcd_enable_intr(hba, UFSHCD_ENABLE_MCQ_INTRS); + ufshcd_mcq_make_queues_operational(hba); + ufshcd_mcq_config_mac(hba, hba->nutrs); + + hba->host->can_queue = hba->nutrs - UFSHCD_NUM_RESERVED; + hba->reserved_slot = hba->nutrs - UFSHCD_NUM_RESERVED; + dev_info(hba->dev, "MCQ configured, nr_queues=%d, io_queues=%d, read_queue=%d, poll_queues=%d, queue_depth=%d\n", + hba->nr_hw_queues, hba->nr_queues[HCTX_TYPE_DEFAULT], + hba->nr_queues[HCTX_TYPE_READ], hba->nr_queues[HCTX_TYPE_POLL], + hba->nutrs); +} + /** * ufshcd_probe_hba - probe hba to detect device and initialize it * @hba: per-adapter instance @@ -8311,6 +8331,10 @@ static int ufshcd_probe_hba(struct ufs_hba *hba, bool init_dev_params) /* UniPro link is active now */ ufshcd_set_link_active(hba); + /* Reconfigure MCQ upon reset */ + if (is_mcq_enabled(hba) && !init_dev_params) + ufshcd_config_mcq(hba); + /* Verify device initialization by sending NOP OUT UPIU */ ret = ufshcd_verify_dev_init(hba); if (ret) @@ -8343,6 +8367,9 @@ static int ufshcd_probe_hba(struct ufs_hba *hba, bool init_dev_params) dev_err(hba->dev, "scsi_add_host failed\n"); goto out; } + /* MCQ may be disabled if ufshcd_alloc_mcq() fails */ + if (use_mcq_mode) + ufshcd_config_mcq(hba); } } diff --git a/drivers/ufs/host/ufs-qcom.c b/drivers/ufs/host/ufs-qcom.c index a52db10..61bb1c8 100644 --- a/drivers/ufs/host/ufs-qcom.c +++ b/drivers/ufs/host/ufs-qcom.c @@ -1524,6 +1524,29 @@ static int ufs_qcom_mcq_config_resource(struct ufs_hba *hba) return ret; } +static int ufs_qcom_op_runtime_config(struct ufs_hba *hba) +{ + struct ufshcd_res_info *mem_res, *sqdao_res; + struct ufshcd_mcq_opr_info_t *opr; + int i; + + mem_res = &hba->res[RES_UFS]; + sqdao_res = &hba->res[RES_MCQ_SQD]; + + if (!mem_res->base || !sqdao_res->base) + return -EINVAL; + + for (i = 0; i < OPR_MAX; i++) { + opr = &hba->mcq_opr[i]; + opr->offset = sqdao_res->resource->start - + mem_res->resource->start + 0x40 * i; + opr->stride = 0x100; + opr->base = sqdao_res->base + 0x40 * i; + } + + return 0; +} + static int ufs_qcom_get_hba_mac(struct ufs_hba *hba) { /* Qualcomm HC supports up to 64 */ @@ -1555,6 +1578,7 @@ static const struct ufs_hba_variant_ops ufs_hba_qcom_vops = { .program_key = ufs_qcom_ice_program_key, .mcq_config_resource = ufs_qcom_mcq_config_resource, .get_hba_mac = ufs_qcom_get_hba_mac, + .op_runtime_config = ufs_qcom_op_runtime_config, }; /** diff --git a/include/ufs/ufshcd.h b/include/ufs/ufshcd.h index 90461f43..ac46d36 100644 --- a/include/ufs/ufshcd.h +++ b/include/ufs/ufshcd.h @@ -299,6 +299,7 @@ struct ufs_pwr_mode_info { * @event_notify: called to notify important events * @mcq_config_resource: called to configure MCQ platform resources * @get_hba_mac: called to get vendor specific mac value, mandatory for mcq mode + * @op_runtime_config: called to config Operation and runtime regs Pointers */ struct ufs_hba_variant_ops { const char *name; @@ -339,6 +340,7 @@ struct ufs_hba_variant_ops { enum ufs_event_type evt, void *data); int (*mcq_config_resource)(struct ufs_hba *hba); int (*get_hba_mac)(struct ufs_hba *hba); + int (*op_runtime_config)(struct ufs_hba *hba); }; /* clock gating state */ @@ -752,6 +754,27 @@ enum ufshcd_res { }; /** + * struct ufshcd_mcq_opr_info_t - Operation and Runtime registers + * + * @offset: Doorbell Address Offset + * @stride: Steps proportional to queue [0...31] + * @base: base address + */ +struct ufshcd_mcq_opr_info_t { + unsigned long offset; + unsigned long stride; + void __iomem *base; +}; + +enum ufshcd_mcq_opr { + OPR_SQD, + OPR_SQIS, + OPR_CQD, + OPR_CQIS, + OPR_MAX, +}; + +/** * struct ufs_hba - per adapter private structure * @mmio_base: UFSHCI base register address * @ucdl_base_addr: UFS Command Descriptor base address @@ -863,6 +886,7 @@ enum ufshcd_res { * ufshcd_resume_complete() * @ext_iid_sup: is EXT_IID is supported by UFSHC * @mcq_sup: is mcq supported by UFSHC + * @mcq_enabled: is mcq ready to accept requests * @res: array of resource info of MCQ registers * @mcq_base: Multi circular queue registers base address * @uhq: array of supported hardware queues @@ -1020,28 +1044,46 @@ struct ufs_hba { bool complete_put; bool ext_iid_sup; bool mcq_sup; + bool mcq_enabled; struct ufshcd_res_info res[RES_MAX]; void __iomem *mcq_base; struct ufs_hw_queue *uhq; struct ufs_hw_queue *dev_cmd_queue; + struct ufshcd_mcq_opr_info_t mcq_opr[OPR_MAX]; }; /** * struct ufs_hw_queue - per hardware queue structure + * @mcq_sq_head: base address of submission queue head pointer + * @mcq_sq_tail: base address of submission queue tail pointer + * @mcq_cq_head: base address of completion queue head pointer + * @mcq_cq_tail: base address of completion queue tail pointer * @sqe_base_addr: submission queue entry base address * @sqe_dma_addr: submission queue dma address * @cqe_base_addr: completion queue base address * @cqe_dma_addr: completion queue dma address * @max_entries: max number of slots in this hardware queue + * @id: hardware queue ID */ struct ufs_hw_queue { + void __iomem *mcq_sq_head; + void __iomem *mcq_sq_tail; + void __iomem *mcq_cq_head; + void __iomem *mcq_cq_tail; + void *sqe_base_addr; dma_addr_t sqe_dma_addr; struct cq_entry *cqe_base_addr; dma_addr_t cqe_dma_addr; u32 max_entries; + u32 id; }; +static inline bool is_mcq_enabled(struct ufs_hba *hba) +{ + return hba->mcq_enabled; +} + /* Returns true if clocks can be gated. Otherwise false */ static inline bool ufshcd_is_clkgating_allowed(struct ufs_hba *hba) { @@ -1097,6 +1139,16 @@ static inline bool ufshcd_enable_wb_if_scaling_up(struct ufs_hba *hba) return hba->caps & UFSHCD_CAP_WB_WITH_CLK_SCALING; } +#define ufsmcq_writel(hba, val, reg) \ + writel((val), (hba)->mcq_base + (reg)) +#define ufsmcq_readl(hba, reg) \ + readl((hba)->mcq_base + (reg)) + +#define ufsmcq_writelx(hba, val, reg) \ + writel_relaxed((val), (hba)->mcq_base + (reg)) +#define ufsmcq_readlx(hba, reg) \ + readl_relaxed((hba)->mcq_base + (reg)) + #define ufshcd_writel(hba, val, reg) \ writel((val), (hba)->mmio_base + (reg)) #define ufshcd_readl(hba, reg) \ diff --git a/include/ufs/ufshci.h b/include/ufs/ufshci.h index 15d1ea2..8784b88 100644 --- a/include/ufs/ufshci.h +++ b/include/ufs/ufshci.h @@ -57,6 +57,7 @@ enum { REG_UFS_CCAP = 0x100, REG_UFS_CRYPTOCAP = 0x104, + REG_UFS_MEM_CFG = 0x300, REG_UFS_MCQ_CFG = 0x380, UFSHCI_CRYPTO_REG_SPACE_SIZE = 0x400, }; @@ -78,6 +79,35 @@ enum { MASK_EXT_IID_SUPPORT = 0x00000400, }; +enum { + REG_SQATTR = 0x0, + REG_SQLBA = 0x4, + REG_SQUBA = 0x8, + REG_SQDAO = 0xC, + REG_SQISAO = 0x10, + + REG_CQATTR = 0x20, + REG_CQLBA = 0x24, + REG_CQUBA = 0x28, + REG_CQDAO = 0x2C, + REG_CQISAO = 0x30, +}; + +enum { + REG_SQHP = 0x0, + REG_SQTP = 0x4, +}; + +enum { + REG_CQHP = 0x0, + REG_CQTP = 0x4, +}; + +enum { + REG_CQIS = 0x0, + REG_CQIE = 0x4, +}; + #define UFS_MASK(mask, offset) ((mask) << (offset)) /* UFS Version 08h */ @@ -134,6 +164,7 @@ static inline u32 ufshci_version(u32 major, u32 minor) #define CONTROLLER_FATAL_ERROR 0x10000 #define SYSTEM_BUS_FATAL_ERROR 0x20000 #define CRYPTO_ENGINE_FATAL_ERROR 0x40000 +#define MCQ_CQ_EVENT_STATUS 0x100000 #define UFSHCD_UIC_HIBERN8_MASK (UIC_HIBERNATE_ENTER |\ UIC_HIBERNATE_EXIT) From patchwork Thu Dec 8 23:18:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Asutosh Das X-Patchwork-Id: 31532 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp474832wrr; Thu, 8 Dec 2022 15:26:40 -0800 (PST) X-Google-Smtp-Source: AA0mqf5ZIk6cZUUyn3OWwIjKse5m7gbOc4qvxfHzl53BnqTlHHw9jy1VL3n9tyZiQZnwmYAbSMyd X-Received: by 2002:a05:6a21:9007:b0:9d:efbf:48be with SMTP id tq7-20020a056a21900700b0009defbf48bemr3712700pzb.2.1670542000608; Thu, 08 Dec 2022 15:26:40 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670542000; cv=none; d=google.com; s=arc-20160816; b=bADnYiLdeumi07du64Yxp9d1tT+oaQUCPuF/SR6AyUusvFUBYqBYj1AlVJS9x5TIWL tQBgKXwmWj/9kaXgPrES5QkgOy1gUI2fMtdO2X8+yrotBMtPZvJ80MDNdnv4FEDz2fWa hI3tiMUGhh4KDuTsOJ8j/t1dzRy6OwS5lYeC3cUkQqH7Ki0yWzrZIx7hesxpKO1qfDg4 lwYlzYDc9jKmbv01go28ljHhMzA/1nALKRvUesBmF8jJRVD0WjXKYjXJukVVlXsnbKHt mUY8oIk3Sxv69zDKz7YeB6zpoSoLlVDATkqZaou8gkuMgYk5NYLOY9amiN+603nfgzzr Wpkw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=v/zRX5brnhFn2VsvDnEdtw/cPH4pi8o8AfKrkRUJDVY=; b=AFZRvqCdmYy5XYaOz24pLx7sqZBloJgh/0PkqcT7JM/aPw/JcRD3Zkk/RrqA4+W+6Y C07jcCOzlUJuppmzV++vGvfTri6+H1ujw+Die/kb2DDJxOsvvBvO4PJrVE0mipjL1wtY Bqbtcq44X+hoLE4HpTDv8w42UlLXDMzfzt9e2bZ+3eRy4p8vbP3NptYepOch3f9EXLWU C3TRCnqGQumYsO/iCMQnpItsO7II1/W3pVOIrfn1aLD8H4raCxqqKYP+S96DEnjFgcN1 dLGLurCVaY57obdOyx7iLjAZrbPKlejpPnIoKRVeP9FHOgdOnSWwhYmkXwP6jrF0e9Jb /h/Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=XOq23wt7; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id x84-20020a633157000000b00463fc72dae9si25661780pgx.144.2022.12.08.15.26.26; Thu, 08 Dec 2022 15:26:40 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=XOq23wt7; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230235AbiLHXVt (ORCPT + 99 others); Thu, 8 Dec 2022 18:21:49 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59482 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230162AbiLHXVC (ORCPT ); Thu, 8 Dec 2022 18:21:02 -0500 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1E07A9B29F; Thu, 8 Dec 2022 15:20:48 -0800 (PST) Received: from pps.filterd (m0279866.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2B8Mm1fc022152; Thu, 8 Dec 2022 23:20:38 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=qcppdkim1; bh=v/zRX5brnhFn2VsvDnEdtw/cPH4pi8o8AfKrkRUJDVY=; b=XOq23wt7FAQEFdewEGNKEyQYycgYtympGHPXb4McLHBcSq7Da1/kJdftN/DWf3mdPk9u /khhYQP6J5AzFsoMpdeoMqReRvZb2D4mgXPKMiBfYCgBa0VWksJV6g+uKx1luIKUX+yb /p1r8ZpdySzeGQm7Jm/gQPqmu/R2fFQpuPi6jL/TalxDeCOTYB3mbuHHEjUO7XIgvGk0 31DxuwHqUEj9D/+RSNqoqv8RUGagHWW3bflOOIGBmygzCFQLEoWF6D0TWn1CbWf2LW0n czQAUglfNjU0j+8MIemXCEn+igKHHGw7MxqHD1Cu7BYHjKC8ApEHzmxGlPCTb7FzY0J4 gw== Received: from nasanppmta05.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3mbbd6tgef-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 08 Dec 2022 23:20:38 +0000 Received: from nasanex01a.na.qualcomm.com ([10.52.223.231]) by NASANPPMTA05.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 2B8NKbwW030861 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 8 Dec 2022 23:20:37 GMT Received: from asutoshd-linux1.qualcomm.com (10.80.80.8) by nasanex01a.na.qualcomm.com (10.52.223.231) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Thu, 8 Dec 2022 15:20:37 -0800 From: Asutosh Das To: , , CC: , , , , , , , , , Asutosh Das , , Alim Akhtar , "James E.J. Bottomley" , Jinyoung Choi , open list Subject: [PATCH v11 10/16] ufs: core: mcq: Use shared tags for MCQ mode Date: Thu, 8 Dec 2022 15:18:36 -0800 Message-ID: X-Mailer: git-send-email 2.7.4 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01b.na.qualcomm.com (10.46.141.250) To nasanex01a.na.qualcomm.com (10.52.223.231) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: JskYg4pEOJr0L-_6Hz2VciF5H-oRP6en X-Proofpoint-GUID: JskYg4pEOJr0L-_6Hz2VciF5H-oRP6en X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-08_12,2022-12-08_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxlogscore=999 adultscore=0 mlxscore=0 clxscore=1015 suspectscore=0 malwarescore=0 bulkscore=0 phishscore=0 priorityscore=1501 spamscore=0 impostorscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000 definitions=main-2212080192 X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751690248736359575?= X-GMAIL-MSGID: =?utf-8?q?1751690248736359575?= Enable shared tags for MCQ. For UFS, this should not have a huge performance impact. It however simplifies the MCQ implementation and reuses most of the existing code in the issue and completion path. Also add multiple queue mapping to map_queue(). Co-developed-by: Can Guo Signed-off-by: Can Guo Signed-off-by: Asutosh Das Reviewed-by: Bart Van Assche Reviewed-by: Manivannan Sadhasivam Reviewed-by: Stanley Chu --- drivers/ufs/core/ufs-mcq.c | 2 ++ drivers/ufs/core/ufshcd.c | 28 ++++++++++++++++------------ 2 files changed, 18 insertions(+), 12 deletions(-) diff --git a/drivers/ufs/core/ufs-mcq.c b/drivers/ufs/core/ufs-mcq.c index 496e2b6..8bf222f 100644 --- a/drivers/ufs/core/ufs-mcq.c +++ b/drivers/ufs/core/ufs-mcq.c @@ -280,6 +280,7 @@ void ufshcd_mcq_make_queues_operational(struct ufs_hba *hba) int ufshcd_mcq_init(struct ufs_hba *hba) { + struct Scsi_Host *host = hba->host; struct ufs_hw_queue *hwq; int ret, i; @@ -315,5 +316,6 @@ int ufshcd_mcq_init(struct ufs_hba *hba) /* Give dev_cmd_queue the minimal number of entries */ hba->dev_cmd_queue->max_entries = MAX_DEV_CMD_ENTRIES; + host->host_tagset = 1; return 0; } diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c index 5d14d3f..21cfa37 100644 --- a/drivers/ufs/core/ufshcd.c +++ b/drivers/ufs/core/ufshcd.c @@ -2763,24 +2763,28 @@ static inline bool is_device_wlun(struct scsi_device *sdev) */ static void ufshcd_map_queues(struct Scsi_Host *shost) { - int i; + struct ufs_hba *hba = shost_priv(shost); + int i, queue_offset = 0; + + if (!is_mcq_supported(hba)) { + hba->nr_queues[HCTX_TYPE_DEFAULT] = 1; + hba->nr_queues[HCTX_TYPE_READ] = 0; + hba->nr_queues[HCTX_TYPE_POLL] = 1; + hba->nr_hw_queues = 1; + } for (i = 0; i < shost->nr_maps; i++) { struct blk_mq_queue_map *map = &shost->tag_set.map[i]; - switch (i) { - case HCTX_TYPE_DEFAULT: - case HCTX_TYPE_POLL: - map->nr_queues = 1; - break; - case HCTX_TYPE_READ: - map->nr_queues = 0; + map->nr_queues = hba->nr_queues[i]; + if (!map->nr_queues) continue; - default: - WARN_ON_ONCE(true); - } - map->queue_offset = 0; + map->queue_offset = queue_offset; + if (i == HCTX_TYPE_POLL && !is_mcq_supported(hba)) + map->queue_offset = 0; + blk_mq_map_queues(map); + queue_offset += map->nr_queues; } } From patchwork Thu Dec 8 23:18:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Asutosh Das X-Patchwork-Id: 31534 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp477009wrr; Thu, 8 Dec 2022 15:32:34 -0800 (PST) X-Google-Smtp-Source: AA0mqf6QAlYTAqb5cQb+UAo2SeufonXRjhZ6OXB8m1IfBZLT/Ial83dIyUBpprpGUPS0Q3sSJxG8 X-Received: by 2002:a05:6a21:8cca:b0:9d:efd3:66e5 with SMTP id ta10-20020a056a218cca00b0009defd366e5mr5025739pzb.44.1670542354650; Thu, 08 Dec 2022 15:32:34 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670542354; cv=none; d=google.com; s=arc-20160816; b=TJXkSxly1ceh544NyrwFIMf1kbLUMH1N/Ct4ffeUBiFj+DOymv7LNHPuQ8V6+4oJ7V hJKAEFwwektRNDas/gPMSo6jhyycs0woIc5icaBDwRwlPyaSvZGCR4k5V+F5Q9RLIOct onMuwxCuoCcCbuAbjez/wIyVKr7asihtfDW17x0sxp/pXNzGx04/6abuFdvFsRlqcKvm srKJeVa862V0GxTzW1d9a/7/EY26cYs/JBhHHjpmpQJxaZn14BZiDJqer/TT5WNEGwTY +caRI5PXUZaETiaVZBniru6bNQ0Xpy6X3wHb8RywUOhu3+Ku/FEaU5Zbn3rUbdyzbclE J5bA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=TtypK3xzSz0AV9H3azyDMJxhM0hKsRMwVxsRSfIsESE=; b=zRR8hiW8U7wflUm1AyBQjZK7waIjZapJb3A+g9qQvopfepSp9bQB0TwjfaR2TwcoOu Bh4IQNIxNfU8pNFkiono0eQR4lEJ6dCokgPa4sjmjuMHd7xzQd70v3QE6b6ZzNmNgzpj JcrwrzZxvQxjfqX4dJDOhI5ZWaCoRDfspIHj7CXUHSZiW+v4wuj5sSWOmqZqww3vZyXc zpwxOAN2mDWEZI/ZqTJ4pGwN+JmtLcN1Sqx5Jr6hKhYsVZ3PNwbBSDKIovSjzwSxzGld 0wSuHchdSuqph/GwhN1Pvbotf4Wvyjy/gxMfwUT2f+1tbWgyfAjfOv/G+ZyAjk6PKXlW dMPg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcdkim header.b=MijF0zpQ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id ng7-20020a17090b1a8700b0021a177ab3b9si119589pjb.178.2022.12.08.15.32.21; Thu, 08 Dec 2022 15:32:34 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcdkim header.b=MijF0zpQ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230110AbiLHXWE (ORCPT + 99 others); Thu, 8 Dec 2022 18:22:04 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59566 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230112AbiLHXVF (ORCPT ); Thu, 8 Dec 2022 18:21:05 -0500 Received: from alexa-out-sd-02.qualcomm.com (alexa-out-sd-02.qualcomm.com [199.106.114.39]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B8B589D8B3; Thu, 8 Dec 2022 15:20:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1670541652; x=1702077652; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=TtypK3xzSz0AV9H3azyDMJxhM0hKsRMwVxsRSfIsESE=; b=MijF0zpQNLMm6SPGPpdoDLwEEibfM9CJOB0FWsHIzJaYojroI85zWX5b /LL0NUi1Yqs09XeMq8Pll6qZ9GYCX3lR0mdl2o6R0FtRS35tdratO7ME5 LLrATn6XqiUMLq3d3Dmw7IgYVr256Dsxz/h1+FSHf+iRNaiguFK2e1Tbo I=; Received: from unknown (HELO ironmsg04-sd.qualcomm.com) ([10.53.140.144]) by alexa-out-sd-02.qualcomm.com with ESMTP; 08 Dec 2022 15:20:52 -0800 X-QCInternal: smtphost Received: from unknown (HELO nasanex01a.na.qualcomm.com) ([10.52.223.231]) by ironmsg04-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Dec 2022 15:20:52 -0800 Received: from asutoshd-linux1.qualcomm.com (10.80.80.8) by nasanex01a.na.qualcomm.com (10.52.223.231) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Thu, 8 Dec 2022 15:20:51 -0800 From: Asutosh Das To: , , CC: , , , , , , , , , Asutosh Das , , Alim Akhtar , "James E.J. Bottomley" , Jinyoung Choi , Krzysztof Kozlowski , Keoseong Park , Yoshihiro Shimoda , open list Subject: [PATCH v11 11/16] ufs: core: Prepare ufshcd_send_command for mcq Date: Thu, 8 Dec 2022 15:18:37 -0800 Message-ID: X-Mailer: git-send-email 2.7.4 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01b.na.qualcomm.com (10.46.141.250) To nasanex01a.na.qualcomm.com (10.52.223.231) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751690620350514808?= X-GMAIL-MSGID: =?utf-8?q?1751690620350514808?= Add support to send commands using multiple submission queues in MCQ mode. Modify the functions that use ufshcd_send_command(). Co-developed-by: Can Guo Signed-off-by: Can Guo Signed-off-by: Asutosh Das Reviewed-by: Bart Van Assche Reviewed-by: Manivannan Sadhasivam Reviewed-by: Stanley Chu --- drivers/ufs/core/ufs-mcq.c | 1 + drivers/ufs/core/ufshcd-priv.h | 10 ++++++++++ drivers/ufs/core/ufshcd.c | 36 ++++++++++++++++++++++++++---------- include/ufs/ufshcd.h | 5 +++++ 4 files changed, 42 insertions(+), 10 deletions(-) diff --git a/drivers/ufs/core/ufs-mcq.c b/drivers/ufs/core/ufs-mcq.c index 8bf222f..6815825 100644 --- a/drivers/ufs/core/ufs-mcq.c +++ b/drivers/ufs/core/ufs-mcq.c @@ -309,6 +309,7 @@ int ufshcd_mcq_init(struct ufs_hba *hba) for (i = 0; i < hba->nr_hw_queues; i++) { hwq = &hba->uhq[i]; hwq->max_entries = hba->nutrs; + spin_lock_init(&hwq->sq_lock); } /* The very first HW queue serves device commands */ diff --git a/drivers/ufs/core/ufshcd-priv.h b/drivers/ufs/core/ufshcd-priv.h index 0e35cfa..d295be4 100644 --- a/drivers/ufs/core/ufshcd-priv.h +++ b/drivers/ufs/core/ufshcd-priv.h @@ -332,4 +332,14 @@ static inline bool ufs_is_valid_unit_desc_lun(struct ufs_dev_info *dev_info, u8 return lun == UFS_UPIU_RPMB_WLUN || (lun < dev_info->max_lu_supported); } +static inline void ufshcd_inc_sq_tail(struct ufs_hw_queue *q) +{ + u32 mask = q->max_entries - 1; + u32 val; + + q->sq_tail_slot = (q->sq_tail_slot + 1) & mask; + val = q->sq_tail_slot * sizeof(struct utp_transfer_req_desc); + writel(val, q->mcq_sq_tail); +} + #endif /* _UFSHCD_PRIV_H_ */ diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c index 21cfa37..64fc95e 100644 --- a/drivers/ufs/core/ufshcd.c +++ b/drivers/ufs/core/ufshcd.c @@ -2182,9 +2182,11 @@ static void ufshcd_update_monitor(struct ufs_hba *hba, const struct ufshcd_lrb * * ufshcd_send_command - Send SCSI or device management commands * @hba: per adapter instance * @task_tag: Task tag of the command + * @hwq: pointer to hardware queue instance */ static inline -void ufshcd_send_command(struct ufs_hba *hba, unsigned int task_tag) +void ufshcd_send_command(struct ufs_hba *hba, unsigned int task_tag, + struct ufs_hw_queue *hwq) { struct ufshcd_lrb *lrbp = &hba->lrb[task_tag]; unsigned long flags; @@ -2198,12 +2200,24 @@ void ufshcd_send_command(struct ufs_hba *hba, unsigned int task_tag) if (unlikely(ufshcd_should_inform_monitor(hba, lrbp))) ufshcd_start_monitor(hba, lrbp); - spin_lock_irqsave(&hba->outstanding_lock, flags); - if (hba->vops && hba->vops->setup_xfer_req) - hba->vops->setup_xfer_req(hba, task_tag, !!lrbp->cmd); - __set_bit(task_tag, &hba->outstanding_reqs); - ufshcd_writel(hba, 1 << task_tag, REG_UTP_TRANSFER_REQ_DOOR_BELL); - spin_unlock_irqrestore(&hba->outstanding_lock, flags); + if (is_mcq_enabled(hba)) { + int utrd_size = sizeof(struct utp_transfer_req_desc); + + spin_lock(&hwq->sq_lock); + memcpy(hwq->sqe_base_addr + (hwq->sq_tail_slot * utrd_size), + lrbp->utr_descriptor_ptr, utrd_size); + ufshcd_inc_sq_tail(hwq); + spin_unlock(&hwq->sq_lock); + } else { + spin_lock_irqsave(&hba->outstanding_lock, flags); + if (hba->vops && hba->vops->setup_xfer_req) + hba->vops->setup_xfer_req(hba, lrbp->task_tag, + !!lrbp->cmd); + __set_bit(lrbp->task_tag, &hba->outstanding_reqs); + ufshcd_writel(hba, 1 << lrbp->task_tag, + REG_UTP_TRANSFER_REQ_DOOR_BELL); + spin_unlock_irqrestore(&hba->outstanding_lock, flags); + } } /** @@ -2822,6 +2836,7 @@ static int ufshcd_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *cmd) int tag = scsi_cmd_to_rq(cmd)->tag; struct ufshcd_lrb *lrbp; int err = 0; + struct ufs_hw_queue *hwq = NULL; WARN_ONCE(tag < 0 || tag >= hba->nutrs, "Invalid tag %d\n", tag); @@ -2906,7 +2921,7 @@ static int ufshcd_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *cmd) goto out; } - ufshcd_send_command(hba, tag); + ufshcd_send_command(hba, tag, hwq); out: rcu_read_unlock(); @@ -3101,10 +3116,11 @@ static int ufshcd_exec_dev_cmd(struct ufs_hba *hba, goto out; hba->dev_cmd.complete = &wait; + hba->dev_cmd.cqe = NULL; ufshcd_add_query_upiu_trace(hba, UFS_QUERY_SEND, lrbp->ucd_req_ptr); - ufshcd_send_command(hba, tag); + ufshcd_send_command(hba, tag, hba->dev_cmd_queue); err = ufshcd_wait_for_dev_cmd(hba, lrbp, timeout); ufshcd_add_query_upiu_trace(hba, err ? UFS_QUERY_ERR : UFS_QUERY_COMP, (struct utp_upiu_req *)lrbp->ucd_rsp_ptr); @@ -6952,7 +6968,7 @@ static int ufshcd_issue_devman_upiu_cmd(struct ufs_hba *hba, ufshcd_add_query_upiu_trace(hba, UFS_QUERY_SEND, lrbp->ucd_req_ptr); - ufshcd_send_command(hba, tag); + ufshcd_send_command(hba, tag, hba->dev_cmd_queue); /* * ignore the returning value here - ufshcd_check_query_response is * bound to fail since dev_cmd.query and dev_cmd.type were left empty. diff --git a/include/ufs/ufshcd.h b/include/ufs/ufshcd.h index ac46d36..ae20697 100644 --- a/include/ufs/ufshcd.h +++ b/include/ufs/ufshcd.h @@ -222,6 +222,7 @@ struct ufs_dev_cmd { struct mutex lock; struct completion *complete; struct ufs_query query; + struct cq_entry *cqe; }; /** @@ -1064,6 +1065,8 @@ struct ufs_hba { * @cqe_dma_addr: completion queue dma address * @max_entries: max number of slots in this hardware queue * @id: hardware queue ID + * @sq_tp_slot: current slot to which SQ tail pointer is pointing + * @sq_lock: serialize submission queue access */ struct ufs_hw_queue { void __iomem *mcq_sq_head; @@ -1077,6 +1080,8 @@ struct ufs_hw_queue { dma_addr_t cqe_dma_addr; u32 max_entries; u32 id; + u32 sq_tail_slot; + spinlock_t sq_lock; }; static inline bool is_mcq_enabled(struct ufs_hba *hba) From patchwork Thu Dec 8 23:18:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Asutosh Das X-Patchwork-Id: 31535 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp477435wrr; Thu, 8 Dec 2022 15:33:46 -0800 (PST) X-Google-Smtp-Source: AA0mqf6wIGp5tkDywfjnfROdSh7eTKExVfAZDv0ihkXWaNv5/ciGV75YQcYjM+dCd5fV8C4B+okt X-Received: by 2002:a62:4e96:0:b0:574:1d8a:d4f with SMTP id c144-20020a624e96000000b005741d8a0d4fmr3547948pfb.16.1670542426033; Thu, 08 Dec 2022 15:33:46 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670542426; cv=none; d=google.com; s=arc-20160816; b=xOr/MRz69xYu5atsLm32mBCx1JGA5fYYt5MiR8UgFTIucjwulQ6wmo9x78pWOf9Fgl 01m4/eo2iythwW7eM++hdHxaeKAqjyQZxS8vwsH37p0+WKwekeqekzcw7scE2Lf8EqCw h6QCKjEAyhD/6vLje5e2KcypRn9ZegDuYc3/zFY1Jm4u3YP3pg6lNBsnXjx349rOsDNF GAjX/7O+1ma3S+eHi6n1F0H29epM8nxG9EMlmvrGK4il4Fzmnnry54ER6OA4ojoH8Fh8 6yJp2HT0idachQDX53R9pYSPpJiX6jbTjOHif5Yx7Wts+D33UCjSKsjDee5ZgNaXT+xU +WYQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=yizNSbKBP5y6dEAI547lFI/knj727aVkgCZ/6hf5lM0=; b=VajK26x9+f0WUL0pvpzNrl/J2quBYmJsBKYUO2XMV+7I+94UTq63vT4zh6STMBYpk6 XFoYdLOWuQve2VhMKxbZh2eao83s4E9xCFbUauF/J3D9TGvTE5QoJf3ELvEJBMOo8bTw MrXG3qBxiPDVu6c1/2nfjMaCYhpPw4TXtLO4ZqKDsNhyuJoFUr2rwBEUPz/jdIstm2W4 cXXYnkZm2rFQgEDtBecB6BUJEcg/pK180b6g03XQywgLllRX867rw9oDhXsTU2QY6OnN IyBgVa2MuYPXzfQeHvHq/73lvlfOiuX4XVSwkje5Yr3opEKTUndvQEeWoccO9NaxQ53b LS3Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcdkim header.b=qbzT1oNL; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id bd33-20020a056a0027a100b00571e1bb9312si134096pfb.173.2022.12.08.15.33.32; Thu, 08 Dec 2022 15:33:46 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcdkim header.b=qbzT1oNL; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230112AbiLHXWI (ORCPT + 99 others); Thu, 8 Dec 2022 18:22:08 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60006 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230096AbiLHXVJ (ORCPT ); Thu, 8 Dec 2022 18:21:09 -0500 Received: from alexa-out-sd-01.qualcomm.com (alexa-out-sd-01.qualcomm.com [199.106.114.38]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B5F69A13E5; Thu, 8 Dec 2022 15:21:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1670541663; x=1702077663; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=yizNSbKBP5y6dEAI547lFI/knj727aVkgCZ/6hf5lM0=; b=qbzT1oNLYofVnBAE/s3ezpzmLpMAThIU2rRgaAkXL8vt9in+gtm60CDB Fw0vGMiyN6zq3it9oG6EYMwDb5HBoucTCJyrpPw1Fax+lDG8LgUiJgrt0 yv6iaUwdd4dHz4YRw0OS4y7uksGjs1BHo9pk4KZMvWFfV0hUv05FXJ7gi Q=; Received: from unknown (HELO ironmsg-SD-alpha.qualcomm.com) ([10.53.140.30]) by alexa-out-sd-01.qualcomm.com with ESMTP; 08 Dec 2022 15:21:03 -0800 X-QCInternal: smtphost Received: from unknown (HELO nasanex01a.na.qualcomm.com) ([10.52.223.231]) by ironmsg-SD-alpha.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Dec 2022 15:21:03 -0800 Received: from asutoshd-linux1.qualcomm.com (10.80.80.8) by nasanex01a.na.qualcomm.com (10.52.223.231) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Thu, 8 Dec 2022 15:21:02 -0800 From: Asutosh Das To: , , CC: , , , , , , , , , Asutosh Das , , Alim Akhtar , "James E.J. Bottomley" , Jinyoung Choi , Krzysztof Kozlowski , open list Subject: [PATCH v11 12/16] ufs: core: mcq: Find hardware queue to queue request Date: Thu, 8 Dec 2022 15:18:38 -0800 Message-ID: <7e9ea737bc52b9446263548405a7cd3da154444d.1670541364.git.quic_asutoshd@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01b.na.qualcomm.com (10.46.141.250) To nasanex01a.na.qualcomm.com (10.52.223.231) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751690694882329103?= X-GMAIL-MSGID: =?utf-8?q?1751690694882329103?= Adds support to find the hardware queue on which the request would be queued. Since the very first queue is to serve device commands, an offset of 1 is added to the index of the hardware queue. Co-developed-by: Can Guo Signed-off-by: Can Guo Signed-off-by: Asutosh Das Reviewed-by: Bart Van Assche Reviewed-by: Manivannan Sadhasivam --- drivers/ufs/core/ufs-mcq.c | 19 +++++++++++++++++++ drivers/ufs/core/ufshcd-priv.h | 3 +++ drivers/ufs/core/ufshcd.c | 3 +++ 3 files changed, 25 insertions(+) diff --git a/drivers/ufs/core/ufs-mcq.c b/drivers/ufs/core/ufs-mcq.c index 6815825..be84bce 100644 --- a/drivers/ufs/core/ufs-mcq.c +++ b/drivers/ufs/core/ufs-mcq.c @@ -93,6 +93,25 @@ void ufshcd_mcq_config_mac(struct ufs_hba *hba, u32 max_active_cmds) } /** + * ufshcd_mcq_req_to_hwq - find the hardware queue on which the + * request would be issued. + * @hba - per adapter instance + * @req - pointer to the request to be issued + * + * Returns the hardware queue instance on which the request would + * be queued. + */ +struct ufs_hw_queue *ufshcd_mcq_req_to_hwq(struct ufs_hba *hba, + struct request *req) +{ + u32 utag = blk_mq_unique_tag(req); + u32 hwq = blk_mq_unique_tag_to_hwq(utag); + + /* uhq[0] is used to serve device commands */ + return &hba->uhq[hwq + UFSHCD_MCQ_IO_QUEUE_OFFSET]; +} + +/** * ufshcd_mcq_decide_queue_depth - decide the queue depth * @hba - per adapter instance * diff --git a/drivers/ufs/core/ufshcd-priv.h b/drivers/ufs/core/ufshcd-priv.h index d295be4..a07f85e 100644 --- a/drivers/ufs/core/ufshcd-priv.h +++ b/drivers/ufs/core/ufshcd-priv.h @@ -67,7 +67,10 @@ int ufshcd_mcq_memory_alloc(struct ufs_hba *hba); void ufshcd_mcq_make_queues_operational(struct ufs_hba *hba); void ufshcd_mcq_config_mac(struct ufs_hba *hba, u32 max_active_cmds); void ufshcd_mcq_select_mcq_mode(struct ufs_hba *hba); +struct ufs_hw_queue *ufshcd_mcq_req_to_hwq(struct ufs_hba *hba, + struct request *req); +#define UFSHCD_MCQ_IO_QUEUE_OFFSET 1 #define SD_ASCII_STD true #define SD_RAW false int ufshcd_read_string_desc(struct ufs_hba *hba, u8 desc_index, diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c index 64fc95e..a7e613d 100644 --- a/drivers/ufs/core/ufshcd.c +++ b/drivers/ufs/core/ufshcd.c @@ -2921,6 +2921,9 @@ static int ufshcd_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *cmd) goto out; } + if (is_mcq_enabled(hba)) + hwq = ufshcd_mcq_req_to_hwq(hba, scsi_cmd_to_rq(cmd)); + ufshcd_send_command(hba, tag, hwq); out: From patchwork Thu Dec 8 23:18:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Asutosh Das X-Patchwork-Id: 31540 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp478294wrr; Thu, 8 Dec 2022 15:36:15 -0800 (PST) X-Google-Smtp-Source: AA0mqf69/UUPEAdbCGnZ/3XqrJodnuPOUaZrHb28uryxbniXZMmaaApdjenpKBEtC6Ymm6gNKy1V X-Received: by 2002:a05:6a20:9f8e:b0:a3:1ea9:dd13 with SMTP id mm14-20020a056a209f8e00b000a31ea9dd13mr4799585pzb.30.1670542575032; Thu, 08 Dec 2022 15:36:15 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670542575; cv=none; d=google.com; s=arc-20160816; b=bqFqaEV2mg+VpHi7FdXQvsC7Q+hPc9d6ZOJlP1n1IpQlQqQQer8yNHV69gqS6Ih3Ca v0kSMo0eXubIS1BD2ezyRbXt9hTKi5Nu+6POD2lsXhpVr0gdq0jxzOedwbq3LsGqWja0 0odHmMPQtQMl4uNyKnDBySoCHM5Qtpts/PJgRI2S25vnM6yA+MIOKMFhmjIY35898GU6 AvwPIhpp7PrOA48cGaiUSmSEAp8T2Tv8D9m5xcrVv9eukcswR2H+D2y/5GTVIgvic91y pycBtir9rohu40ukS2XZSHs8ZwjOtylPM8+7mrYShhQ9f8zDlxl3+4vngGOeFa/uh7MT 4Olg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=ZWh0XtmWAZIO7YQMk78LiilkuRluQTz7fx7cik9AWAc=; b=eU2DHXILEKdnJ7yWSI1g1Ckid6kZqHsAq7vAzsrHYLXSsET8htr78OXWK7m5hu2Qoe xeoO5Ukp+0tnGEZ2OUOWbMrBg8cTFpxD8CpvAaV0SWNqLaOZX6GtJS8FtfoWwN3hkv9u NwEqJdAmqdNUwe+vwTwmBZqxUAW55L0ImPYSYq1V/oQsSkOeoyDyaTuyUg1AEK/0Un6M p/9gbpKzLev//mkDSFPWgNdoW0S+w+7pgQFWQEO/XUhQv1PITnMNEIihVkoidFv0JwUO lWbNfPwFdK5vEpuvJZ5+RDc0XJPSLisOGq5Bvf1QjsfXR8LQvYiIe80Z0yoDYRhaG2Oc kbJQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=gT0eJbfd; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id h14-20020a63120e000000b0046ff26f21f0si24951522pgl.503.2022.12.08.15.36.00; Thu, 08 Dec 2022 15:36:15 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=gT0eJbfd; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230182AbiLHXWr (ORCPT + 99 others); Thu, 8 Dec 2022 18:22:47 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59136 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230205AbiLHXVs (ORCPT ); Thu, 8 Dec 2022 18:21:48 -0500 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3ED7CA65B9; Thu, 8 Dec 2022 15:21:25 -0800 (PST) Received: from pps.filterd (m0279870.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2B8NLCI5028099; Thu, 8 Dec 2022 23:21:12 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=qcppdkim1; bh=ZWh0XtmWAZIO7YQMk78LiilkuRluQTz7fx7cik9AWAc=; b=gT0eJbfdMQ914Fj7tbwxYXuFmrgvk6je6J2jIXZijIhXTIYFC/HOaUSmepIrs1pAjjm2 DVsVdJNMKilYam7RtJhfm4U7t79EKp46GNIn76MM6is/GgH6oo9Ljq7czDCt/D6N9N3C K2eNvmqdDTIvbZ2ynYbSokC9PXrolc/v3+L1ER/RsYdFbMog874oWHevYkRCEsPzkn2n pOSgJE1tHMjcgksiSB1wYvbSnGpzZIbLaE5aeCSkcqVoRoa/Zu8gtntFZ1IAvZsGT13R XDMrFRQ7/QasXcXRAlR7DShfSRIK1kaYU89gqcHbdZgvg6hbLV2dB4Hyf5HuSPXPST/e TQ== Received: from nasanppmta03.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3mbr1hg52r-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 08 Dec 2022 23:21:12 +0000 Received: from nasanex01a.na.qualcomm.com ([10.52.223.231]) by NASANPPMTA03.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 2B8NLBxj016774 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 8 Dec 2022 23:21:11 GMT Received: from asutoshd-linux1.qualcomm.com (10.80.80.8) by nasanex01a.na.qualcomm.com (10.52.223.231) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Thu, 8 Dec 2022 15:21:10 -0800 From: Asutosh Das To: , , CC: , , , , , , , , , Asutosh Das , , Alim Akhtar , "James E.J. Bottomley" , Arthur Simchaev , Krzysztof Kozlowski , Jinyoung Choi , open list Subject: [PATCH v11 13/16] ufs: core: Prepare for completion in mcq Date: Thu, 8 Dec 2022 15:18:39 -0800 Message-ID: X-Mailer: git-send-email 2.7.4 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01b.na.qualcomm.com (10.46.141.250) To nasanex01a.na.qualcomm.com (10.52.223.231) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: cgLznUDtAAGYf6fa7fnVReP0N_b_K8-K X-Proofpoint-GUID: cgLznUDtAAGYf6fa7fnVReP0N_b_K8-K X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-08_12,2022-12-08_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 priorityscore=1501 mlxlogscore=999 clxscore=1015 spamscore=0 lowpriorityscore=0 bulkscore=0 mlxscore=0 suspectscore=0 impostorscore=0 phishscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000 definitions=main-2212080192 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751690850879480153?= X-GMAIL-MSGID: =?utf-8?q?1751690850879480153?= Modify completion path APIs and add completion queue entry. Co-developed-by: Can Guo Signed-off-by: Can Guo Signed-off-by: Asutosh Das Reviewed-by: Bart Van Assche Reviewed-by: Manivannan Sadhasivam Reviewed-by: Stanley Chu --- drivers/ufs/core/ufshcd-priv.h | 2 ++ drivers/ufs/core/ufshcd.c | 80 ++++++++++++++++++++++++++---------------- 2 files changed, 51 insertions(+), 31 deletions(-) diff --git a/drivers/ufs/core/ufshcd-priv.h b/drivers/ufs/core/ufshcd-priv.h index a07f85e..a7af6aa 100644 --- a/drivers/ufs/core/ufshcd-priv.h +++ b/drivers/ufs/core/ufshcd-priv.h @@ -61,6 +61,8 @@ int ufshcd_query_attr(struct ufs_hba *hba, enum query_opcode opcode, int ufshcd_query_flag(struct ufs_hba *hba, enum query_opcode opcode, enum flag_idn idn, u8 index, bool *flag_res); void ufshcd_auto_hibern8_update(struct ufs_hba *hba, u32 ahit); +void ufshcd_compl_one_cqe(struct ufs_hba *hba, int task_tag, + struct cq_entry *cqe); int ufshcd_mcq_init(struct ufs_hba *hba); int ufshcd_mcq_decide_queue_depth(struct ufs_hba *hba); int ufshcd_mcq_memory_alloc(struct ufs_hba *hba); diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c index a7e613d..831da4f 100644 --- a/drivers/ufs/core/ufshcd.c +++ b/drivers/ufs/core/ufshcd.c @@ -784,12 +784,17 @@ static inline bool ufshcd_is_device_present(struct ufs_hba *hba) /** * ufshcd_get_tr_ocs - Get the UTRD Overall Command Status * @lrbp: pointer to local command reference block + * @cqe: pointer to the completion queue entry * * This function is used to get the OCS field from UTRD * Returns the OCS field in the UTRD */ -static enum utp_ocs ufshcd_get_tr_ocs(struct ufshcd_lrb *lrbp) +static enum utp_ocs ufshcd_get_tr_ocs(struct ufshcd_lrb *lrbp, + struct cq_entry *cqe) { + if (cqe) + return le32_to_cpu(cqe->status) & MASK_OCS; + return le32_to_cpu(lrbp->utr_descriptor_ptr->header.dword_2) & MASK_OCS; } @@ -3048,7 +3053,7 @@ static int ufshcd_wait_for_dev_cmd(struct ufs_hba *hba, * not trigger any race conditions. */ hba->dev_cmd.complete = NULL; - err = ufshcd_get_tr_ocs(lrbp); + err = ufshcd_get_tr_ocs(lrbp, hba->dev_cmd.cqe); if (!err) err = ufshcd_dev_cmd_completion(hba, lrbp); } else { @@ -5216,18 +5221,20 @@ ufshcd_scsi_cmd_status(struct ufshcd_lrb *lrbp, int scsi_status) * ufshcd_transfer_rsp_status - Get overall status of the response * @hba: per adapter instance * @lrbp: pointer to local reference block of completed command + * @cqe: pointer to the completion queue entry * * Returns result of the command to notify SCSI midlayer */ static inline int -ufshcd_transfer_rsp_status(struct ufs_hba *hba, struct ufshcd_lrb *lrbp) +ufshcd_transfer_rsp_status(struct ufs_hba *hba, struct ufshcd_lrb *lrbp, + struct cq_entry *cqe) { int result = 0; int scsi_status; enum utp_ocs ocs; /* overall command status of utrd */ - ocs = ufshcd_get_tr_ocs(lrbp); + ocs = ufshcd_get_tr_ocs(lrbp, cqe); if (hba->quirks & UFSHCD_QUIRK_BROKEN_OCS_FATAL_ERROR) { if (be32_to_cpu(lrbp->ucd_rsp_ptr->header.dword_1) & @@ -5392,42 +5399,53 @@ static void ufshcd_release_scsi_cmd(struct ufs_hba *hba, } /** - * __ufshcd_transfer_req_compl - handle SCSI and query command completion + * ufshcd_compl_one_cqe - handle a completion queue entry * @hba: per adapter instance - * @completed_reqs: bitmask that indicates which requests to complete + * @task_tag: the task tag of the request to be completed + * @cqe: pointer to the completion queue entry */ -static void __ufshcd_transfer_req_compl(struct ufs_hba *hba, - unsigned long completed_reqs) +void ufshcd_compl_one_cqe(struct ufs_hba *hba, int task_tag, + struct cq_entry *cqe) { struct ufshcd_lrb *lrbp; struct scsi_cmnd *cmd; - int index; - - for_each_set_bit(index, &completed_reqs, hba->nutrs) { - lrbp = &hba->lrb[index]; - lrbp->compl_time_stamp = ktime_get(); - lrbp->compl_time_stamp_local_clock = local_clock(); - cmd = lrbp->cmd; - if (cmd) { - if (unlikely(ufshcd_should_inform_monitor(hba, lrbp))) - ufshcd_update_monitor(hba, lrbp); - ufshcd_add_command_trace(hba, index, UFS_CMD_COMP); - cmd->result = ufshcd_transfer_rsp_status(hba, lrbp); - ufshcd_release_scsi_cmd(hba, lrbp); - /* Do not touch lrbp after scsi done */ - scsi_done(cmd); - } else if (lrbp->command_type == UTP_CMD_TYPE_DEV_MANAGE || - lrbp->command_type == UTP_CMD_TYPE_UFS_STORAGE) { - if (hba->dev_cmd.complete) { - ufshcd_add_command_trace(hba, index, - UFS_DEV_COMP); - complete(hba->dev_cmd.complete); - ufshcd_clk_scaling_update_busy(hba); - } + + lrbp = &hba->lrb[task_tag]; + lrbp->compl_time_stamp = ktime_get(); + cmd = lrbp->cmd; + if (cmd) { + if (unlikely(ufshcd_should_inform_monitor(hba, lrbp))) + ufshcd_update_monitor(hba, lrbp); + ufshcd_add_command_trace(hba, task_tag, UFS_CMD_COMP); + cmd->result = ufshcd_transfer_rsp_status(hba, lrbp, cqe); + ufshcd_release_scsi_cmd(hba, lrbp); + /* Do not touch lrbp after scsi done */ + scsi_done(cmd); + } else if (lrbp->command_type == UTP_CMD_TYPE_DEV_MANAGE || + lrbp->command_type == UTP_CMD_TYPE_UFS_STORAGE) { + if (hba->dev_cmd.complete) { + hba->dev_cmd.cqe = cqe; + ufshcd_add_command_trace(hba, task_tag, UFS_DEV_COMP); + complete(hba->dev_cmd.complete); + ufshcd_clk_scaling_update_busy(hba); } } } +/** + * __ufshcd_transfer_req_compl - handle SCSI and query command completion + * @hba: per adapter instance + * @completed_reqs: bitmask that indicates which requests to complete + */ +static void __ufshcd_transfer_req_compl(struct ufs_hba *hba, + unsigned long completed_reqs) +{ + int tag; + + for_each_set_bit(tag, &completed_reqs, hba->nutrs) + ufshcd_compl_one_cqe(hba, tag, NULL); +} + /* Any value that is not an existing queue number is fine for this constant. */ enum { UFSHCD_POLL_FROM_INTERRUPT_CONTEXT = -1 From patchwork Thu Dec 8 23:18:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Asutosh Das X-Patchwork-Id: 31537 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp477748wrr; Thu, 8 Dec 2022 15:34:40 -0800 (PST) X-Google-Smtp-Source: AA0mqf6OsqmCwA4+lwdGU7ioqQkg3G2BTsTu1IBE29Q/3Wecesgm6nIZjSYxaD7MmbcBZ6HVDTZP X-Received: by 2002:a05:6a20:3aa7:b0:9e:ae77:aad4 with SMTP id d39-20020a056a203aa700b0009eae77aad4mr4235187pzh.41.1670542479957; Thu, 08 Dec 2022 15:34:39 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670542479; cv=none; d=google.com; s=arc-20160816; b=jVsinaOcaal3knhcCYP33R+R5Ww2Igh5D1P75lR4onNhntG9Av9xt1ZMUkgULKtqDB C3gWhEvVrDPXkvUem4hgyuyrKHrkhhLRxtHnwZWVSSCdyRe19kUT6GxeTnVJFElecf8P DRWn/H+G44fXhCJZMVkCy84iiSVtTgFhuV9o9AcE4TWPDJpzDAjT3cL7LNbXK2b+l6Fb IQlQUHLkxHN6qLooPUZWcR4vDB4zpQOX2ZY9quuFyuHOE0/gKffoMtJSQ85rFeb5sXmp McK11RZa/EB5z/hG+Udn0Lk+oMVW/tuK/RqQX3x3Nlxl6gRNgRWj6HRaggAv5vOvjNhu aiQg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=YfLoq5vozXSD5hJSR9wdVTvKdruVf/IGIPjt3UGljSc=; b=cob+FurV+SfkFyYy7wCxsMqLbHXg65wU2w4/4u3iKcSR8gEHTte93lcaCVNSxdVXHt 4oZ1zAZCJ3hfQxZrlR3FsN0aDJHdEgwj5lbeXk4UIPsVx7THD7NKYNaBH+3rTche8RfD fqgBhm5xNJNCLHY+SjWnYOqs8COXOzm7RhTHF3LznMtdyiW+a/onZcA0qqoSGyXu96pP gk6xJmehULUqznQ04ZDQXe8OXNVEAn2ckmw1xpNSOa3/JaYUKThrYMY2XOPfWQOulOCz lGs5OrYVN4V79L2D3ZezEcEJ/RkhdDnNqEQhOEiDtP3XL+VOXvqCmjJ/4uKVbcZzhjt+ kJpg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=bjIxszlO; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id p17-20020a63c151000000b00477a10a91dcsi24396600pgi.695.2022.12.08.15.34.26; Thu, 08 Dec 2022 15:34:39 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=bjIxszlO; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230205AbiLHXX3 (ORCPT + 99 others); Thu, 8 Dec 2022 18:23:29 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59928 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230190AbiLHXWz (ORCPT ); Thu, 8 Dec 2022 18:22:55 -0500 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CDC3E9D8B1; Thu, 8 Dec 2022 15:21:49 -0800 (PST) Received: from pps.filterd (m0279872.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2B8NLSDT027672; Thu, 8 Dec 2022 23:21:28 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=qcppdkim1; bh=YfLoq5vozXSD5hJSR9wdVTvKdruVf/IGIPjt3UGljSc=; b=bjIxszlOUUOegrhoxTDh9ul8tVYha42Qmt9/KwNvZK3sLAC9vRf/UmSopZFL8RLxOcVS P9cmM8+rn7D2V2BEnctcvUPkS/2/6L0x99VuycYlXi6rpd/tNRwDGWc9RVgJuTmzlCnD MGh3Oi/tj8eV+1giJ0KZ4kEgOmS1Z18SeQnKotBnp9fJ0MbTU7hPe1T6sm19K211GVgB szRKt705RyhD4D0oXwpUZdBiucjTabqTYCo+9n4VarAZlO0Bs53xXlkDlERhILoNhdtY VG0FW2J4OzwyouJ2TXJCkqUPpZCt2F315wga1TwS0ow1vVAb3yOHw2/KISH2cGRAjhVM hA== Received: from nasanppmta02.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3map4a54p5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 08 Dec 2022 23:21:28 +0000 Received: from nasanex01a.na.qualcomm.com (corens_vlan604_snip.qualcomm.com [10.53.140.1]) by NASANPPMTA02.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 2B8NLRN5016429 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 8 Dec 2022 23:21:27 GMT Received: from asutoshd-linux1.qualcomm.com (10.80.80.8) by nasanex01a.na.qualcomm.com (10.52.223.231) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Thu, 8 Dec 2022 15:21:26 -0800 From: Asutosh Das To: , , CC: , , , , , , , , , Asutosh Das , , Alim Akhtar , "James E.J. Bottomley" , "Andy Gross" , Bjorn Andersson , "Konrad Dybcio" , Arthur Simchaev , Jinyoung Choi , Krzysztof Kozlowski , Keoseong Park , Yoshihiro Shimoda , Kiwoong Kim , "open list" Subject: [PATCH v11 14/16] ufs: mcq: Add completion support of a cqe Date: Thu, 8 Dec 2022 15:18:40 -0800 Message-ID: X-Mailer: git-send-email 2.7.4 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01b.na.qualcomm.com (10.46.141.250) To nasanex01a.na.qualcomm.com (10.52.223.231) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: LK6EIfvOtMqP_6fEFYiOvhKRkseIi4GX X-Proofpoint-GUID: LK6EIfvOtMqP_6fEFYiOvhKRkseIi4GX X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-08_12,2022-12-08_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 suspectscore=0 impostorscore=0 mlxscore=0 phishscore=0 priorityscore=1501 adultscore=0 mlxlogscore=999 spamscore=0 bulkscore=0 malwarescore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000 definitions=main-2212080192 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751690751698701018?= X-GMAIL-MSGID: =?utf-8?q?1751690751698701018?= Add support for completing requests from Completion Queue. Some host controllers support vendor specific registers that provide a bitmap of all CQ's which have at least one completed CQE. Add this support. The MCQ specification doesn't provide the Task Tag or its equivalent in the Completion Queue Entry. So use an indirect method to find the Task Tag from the Completion Queue Entry. Co-developed-by: Can Guo Signed-off-by: Can Guo Signed-off-by: Asutosh Das Reviewed-by: Bart Van Assche Reviewed-by: Manivannan Sadhasivam Reviewed-by: Stanley Chu --- drivers/ufs/core/ufs-mcq.c | 61 ++++++++++++++++++++++++++++++++++++++++++ drivers/ufs/core/ufshcd-priv.h | 43 +++++++++++++++++++++++++++++ drivers/ufs/core/ufshcd.c | 37 +++++++++++++++++++++++++ drivers/ufs/host/ufs-qcom.c | 14 ++++++++++ drivers/ufs/host/ufs-qcom.h | 4 +++ include/ufs/ufshcd.h | 7 +++++ include/ufs/ufshci.h | 3 +++ 7 files changed, 169 insertions(+) diff --git a/drivers/ufs/core/ufs-mcq.c b/drivers/ufs/core/ufs-mcq.c index be84bce..cd10d59 100644 --- a/drivers/ufs/core/ufs-mcq.c +++ b/drivers/ufs/core/ufs-mcq.c @@ -25,6 +25,7 @@ #define MCQ_CFG_MAC_MASK GENMASK(16, 8) #define MCQ_QCFG_SIZE 0x40 #define MCQ_ENTRY_SIZE_IN_DWORD 8 +#define CQE_UCD_BA GENMASK_ULL(63, 7) static int rw_queue_count_set(const char *val, const struct kernel_param *kp) { @@ -236,6 +237,63 @@ static void __iomem *mcq_opr_base(struct ufs_hba *hba, return opr->base + opr->stride * i; } +u32 ufshcd_mcq_read_cqis(struct ufs_hba *hba, int i) +{ + return readl(mcq_opr_base(hba, OPR_CQIS, i) + REG_CQIS); +} + +void ufshcd_mcq_write_cqis(struct ufs_hba *hba, u32 val, int i) +{ + writel(val, mcq_opr_base(hba, OPR_CQIS, i) + REG_CQIS); +} + +/* + * Current MCQ specification doesn't provide a Task Tag or its equivalent in + * the Completion Queue Entry. Find the Task Tag using an indirect method. + */ +static int ufshcd_mcq_get_tag(struct ufs_hba *hba, + struct ufs_hw_queue *hwq, + struct cq_entry *cqe) +{ + u64 addr; + + /* sizeof(struct utp_transfer_cmd_desc) must be a multiple of 128 */ + BUILD_BUG_ON(sizeof(struct utp_transfer_cmd_desc) & GENMASK(6, 0)); + + /* Bits 63:7 UCD base address, 6:5 are reserved, 4:0 is SQ ID */ + addr = (le64_to_cpu(cqe->command_desc_base_addr) & CQE_UCD_BA) - + hba->ucdl_dma_addr; + + return div_u64(addr, sizeof(struct utp_transfer_cmd_desc)); +} + +static void ufshcd_mcq_process_cqe(struct ufs_hba *hba, + struct ufs_hw_queue *hwq) +{ + struct cq_entry *cqe = ufshcd_mcq_cur_cqe(hwq); + int tag = ufshcd_mcq_get_tag(hba, hwq, cqe); + + ufshcd_compl_one_cqe(hba, tag, cqe); +} + +unsigned long ufshcd_mcq_poll_cqe_nolock(struct ufs_hba *hba, + struct ufs_hw_queue *hwq) +{ + unsigned long completed_reqs = 0; + + ufshcd_mcq_update_cq_tail_slot(hwq); + while (!ufshcd_mcq_is_cq_empty(hwq)) { + ufshcd_mcq_process_cqe(hba, hwq); + ufshcd_mcq_inc_cq_head_slot(hwq); + completed_reqs++; + } + + if (completed_reqs) + ufshcd_mcq_update_cq_head(hwq); + + return completed_reqs; +} + void ufshcd_mcq_make_queues_operational(struct ufs_hba *hba) { struct ufs_hw_queue *hwq; @@ -279,6 +337,9 @@ void ufshcd_mcq_make_queues_operational(struct ufs_hba *hba) hwq->mcq_cq_head = mcq_opr_base(hba, OPR_CQD, i) + REG_CQHP; hwq->mcq_cq_tail = mcq_opr_base(hba, OPR_CQD, i) + REG_CQTP; + /* Reinitializing is needed upon HC reset */ + hwq->sq_tail_slot = hwq->cq_tail_slot = hwq->cq_head_slot = 0; + /* Enable Tail Entry Push Status interrupt only for non-poll queues */ if (i < hba->nr_hw_queues - hba->nr_queues[HCTX_TYPE_POLL]) writel(1, mcq_opr_base(hba, OPR_CQIS, i) + REG_CQIE); diff --git a/drivers/ufs/core/ufshcd-priv.h b/drivers/ufs/core/ufshcd-priv.h index a7af6aa..70e3416 100644 --- a/drivers/ufs/core/ufshcd-priv.h +++ b/drivers/ufs/core/ufshcd-priv.h @@ -69,6 +69,10 @@ int ufshcd_mcq_memory_alloc(struct ufs_hba *hba); void ufshcd_mcq_make_queues_operational(struct ufs_hba *hba); void ufshcd_mcq_config_mac(struct ufs_hba *hba, u32 max_active_cmds); void ufshcd_mcq_select_mcq_mode(struct ufs_hba *hba); +u32 ufshcd_mcq_read_cqis(struct ufs_hba *hba, int i); +void ufshcd_mcq_write_cqis(struct ufs_hba *hba, u32 val, int i); +unsigned long ufshcd_mcq_poll_cqe_nolock(struct ufs_hba *hba, + struct ufs_hw_queue *hwq); struct ufs_hw_queue *ufshcd_mcq_req_to_hwq(struct ufs_hba *hba, struct request *req); @@ -261,6 +265,15 @@ static inline int ufshcd_mcq_vops_op_runtime_config(struct ufs_hba *hba) return -EOPNOTSUPP; } +static inline int ufshcd_vops_get_outstanding_cqs(struct ufs_hba *hba, + unsigned long *ocqs) +{ + if (hba->vops && hba->vops->get_outstanding_cqs) + return hba->vops->get_outstanding_cqs(hba, ocqs); + + return -EOPNOTSUPP; +} + extern const struct ufs_pm_lvl_states ufs_pm_lvl_states[]; /** @@ -347,4 +360,34 @@ static inline void ufshcd_inc_sq_tail(struct ufs_hw_queue *q) writel(val, q->mcq_sq_tail); } +static inline void ufshcd_mcq_update_cq_tail_slot(struct ufs_hw_queue *q) +{ + u32 val = readl(q->mcq_cq_tail); + + q->cq_tail_slot = val / sizeof(struct cq_entry); +} + +static inline bool ufshcd_mcq_is_cq_empty(struct ufs_hw_queue *q) +{ + return q->cq_head_slot == q->cq_tail_slot; +} + +static inline void ufshcd_mcq_inc_cq_head_slot(struct ufs_hw_queue *q) +{ + q->cq_head_slot++; + if (q->cq_head_slot == q->max_entries) + q->cq_head_slot = 0; +} + +static inline void ufshcd_mcq_update_cq_head(struct ufs_hw_queue *q) +{ + writel(q->cq_head_slot * sizeof(struct cq_entry), q->mcq_cq_head); +} + +static inline struct cq_entry *ufshcd_mcq_cur_cqe(struct ufs_hw_queue *q) +{ + struct cq_entry *cqe = q->cqe_base_addr; + + return cqe + q->cq_head_slot; +} #endif /* _UFSHCD_PRIV_H_ */ diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c index 831da4f..884dabb 100644 --- a/drivers/ufs/core/ufshcd.c +++ b/drivers/ufs/core/ufshcd.c @@ -6698,6 +6698,40 @@ static irqreturn_t ufshcd_tmc_handler(struct ufs_hba *hba) } /** + * ufshcd_handle_mcq_cq_events - handle MCQ completion queue events + * @hba: per adapter instance + * + * Returns IRQ_HANDLED if interrupt is handled + */ +static irqreturn_t ufshcd_handle_mcq_cq_events(struct ufs_hba *hba) +{ + struct ufs_hw_queue *hwq; + unsigned long outstanding_cqs; + unsigned int nr_queues; + int i, ret; + u32 events; + + ret = ufshcd_vops_get_outstanding_cqs(hba, &outstanding_cqs); + if (ret) + outstanding_cqs = (1U << hba->nr_hw_queues) - 1; + + /* Exclude the poll queues */ + nr_queues = hba->nr_hw_queues - hba->nr_queues[HCTX_TYPE_POLL]; + for_each_set_bit(i, &outstanding_cqs, nr_queues) { + hwq = &hba->uhq[i]; + + events = ufshcd_mcq_read_cqis(hba, i); + if (events) + ufshcd_mcq_write_cqis(hba, events, i); + + if (events & UFSHCD_MCQ_CQIS_TAIL_ENT_PUSH_STS) + ufshcd_mcq_poll_cqe_nolock(hba, hwq); + } + + return IRQ_HANDLED; +} + +/** * ufshcd_sl_intr - Interrupt service routine * @hba: per adapter instance * @intr_status: contains interrupts generated by the controller @@ -6722,6 +6756,9 @@ static irqreturn_t ufshcd_sl_intr(struct ufs_hba *hba, u32 intr_status) if (intr_status & UTP_TRANSFER_REQ_COMPL) retval |= ufshcd_transfer_req_compl(hba); + if (intr_status & MCQ_CQ_EVENT_STATUS) + retval |= ufshcd_handle_mcq_cq_events(hba); + return retval; } diff --git a/drivers/ufs/host/ufs-qcom.c b/drivers/ufs/host/ufs-qcom.c index 61bb1c8..379b213 100644 --- a/drivers/ufs/host/ufs-qcom.c +++ b/drivers/ufs/host/ufs-qcom.c @@ -1553,6 +1553,19 @@ static int ufs_qcom_get_hba_mac(struct ufs_hba *hba) return MAX_SUPP_MAC; } +static int ufs_qcom_get_outstanding_cqs(struct ufs_hba *hba, + unsigned long *ocqs) +{ + struct ufshcd_res_info *mcq_vs_res = &hba->res[RES_MCQ_VS]; + + if (!mcq_vs_res->base) + return -EINVAL; + + *ocqs = readl(mcq_vs_res->base + UFS_MEM_CQIS_VS); + + return 0; +} + /* * struct ufs_hba_qcom_vops - UFS QCOM specific variant operations * @@ -1579,6 +1592,7 @@ static const struct ufs_hba_variant_ops ufs_hba_qcom_vops = { .mcq_config_resource = ufs_qcom_mcq_config_resource, .get_hba_mac = ufs_qcom_get_hba_mac, .op_runtime_config = ufs_qcom_op_runtime_config, + .get_outstanding_cqs = ufs_qcom_get_outstanding_cqs, }; /** diff --git a/drivers/ufs/host/ufs-qcom.h b/drivers/ufs/host/ufs-qcom.h index f86e532..6912bdf 100644 --- a/drivers/ufs/host/ufs-qcom.h +++ b/drivers/ufs/host/ufs-qcom.h @@ -73,6 +73,10 @@ enum { UFS_UFS_DBG_RD_EDTL_RAM = 0x1900, }; +enum { + UFS_MEM_CQIS_VS = 0x8, +}; + #define UFS_CNTLR_2_x_x_VEN_REGS_OFFSET(x) (0x000 + x) #define UFS_CNTLR_3_x_x_VEN_REGS_OFFSET(x) (0x400 + x) diff --git a/include/ufs/ufshcd.h b/include/ufs/ufshcd.h index ae20697..8441c46 100644 --- a/include/ufs/ufshcd.h +++ b/include/ufs/ufshcd.h @@ -301,6 +301,7 @@ struct ufs_pwr_mode_info { * @mcq_config_resource: called to configure MCQ platform resources * @get_hba_mac: called to get vendor specific mac value, mandatory for mcq mode * @op_runtime_config: called to config Operation and runtime regs Pointers + * @get_outstanding_cqs: called to get outstanding completion queues */ struct ufs_hba_variant_ops { const char *name; @@ -342,6 +343,8 @@ struct ufs_hba_variant_ops { int (*mcq_config_resource)(struct ufs_hba *hba); int (*get_hba_mac)(struct ufs_hba *hba); int (*op_runtime_config)(struct ufs_hba *hba); + int (*get_outstanding_cqs)(struct ufs_hba *hba, + unsigned long *ocqs); }; /* clock gating state */ @@ -1067,6 +1070,8 @@ struct ufs_hba { * @id: hardware queue ID * @sq_tp_slot: current slot to which SQ tail pointer is pointing * @sq_lock: serialize submission queue access + * @cq_tail_slot: current slot to which CQ tail pointer is pointing + * @cq_head_slot: current slot to which CQ head pointer is pointing */ struct ufs_hw_queue { void __iomem *mcq_sq_head; @@ -1082,6 +1087,8 @@ struct ufs_hw_queue { u32 id; u32 sq_tail_slot; spinlock_t sq_lock; + u32 cq_tail_slot; + u32 cq_head_slot; }; static inline bool is_mcq_enabled(struct ufs_hba *hba) diff --git a/include/ufs/ufshci.h b/include/ufs/ufshci.h index 8784b88..1df8425 100644 --- a/include/ufs/ufshci.h +++ b/include/ufs/ufshci.h @@ -262,6 +262,9 @@ enum { /* UTMRLRSR - UTP Task Management Request Run-Stop Register 80h */ #define UTP_TASK_REQ_LIST_RUN_STOP_BIT 0x1 +/* CQISy - CQ y Interrupt Status Register */ +#define UFSHCD_MCQ_CQIS_TAIL_ENT_PUSH_STS 0x1 + /* UICCMD - UIC Command */ #define COMMAND_OPCODE_MASK 0xFF #define GEN_SELECTOR_INDEX_MASK 0xFFFF From patchwork Thu Dec 8 23:18:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Asutosh Das X-Patchwork-Id: 31533 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp476755wrr; Thu, 8 Dec 2022 15:32:00 -0800 (PST) X-Google-Smtp-Source: AA0mqf5XbX1yg9lwn/D1bveLkTiME3MFos1pbgENutbJu8jfaw3j+WQg7pUXC0LyypJZ8+N3Si20 X-Received: by 2002:a17:90b:2545:b0:213:113b:9085 with SMTP id nw5-20020a17090b254500b00213113b9085mr3651623pjb.23.1670542320145; Thu, 08 Dec 2022 15:32:00 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670542320; cv=none; d=google.com; s=arc-20160816; b=ZFigOAYwLkXBwQfAmTPRai3zJg3RLW9Eir3SMFCnBGAez6koHHWU/nuuIBcZ7C6Xlr PURMVsDhtC4YDr4Dd954sQzl/hs3m2otrYjagWQSPZCRGCbbE/5z6dsQ4vWjt8En/MOz lu5T5jcJINs06S9M9InkaSnOYJVOKbM4vlswMZSXRzaEIGSkr7CgHDPi2zo38VS0gxyK cVBsun838CT17olS8vTJLuW5QhQYvV5Ozg9513k2VTsHKORxqN0cSLn7+juYM/SUCnbs 4lR2XR2bFXMkswDfvrGC2CyN6li4KkCpWXKA65nCoLnyP+5OHgDLa7Su0lXWyqQkqmCA h5MQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=79R+NExqrZIZoVwsptvjNlEk8LlSSWuIaPAXQS6vKic=; b=a+W8MF4z8InO3edTkcn9X0GhrqfWXD5afYfEqKaLJbEFgt/vRngTbHUc1ilbqe5YH6 Ra7cRbGMkgRUqFw+E7WaMqncyLPSFXBrP2HQWs1yD+5Qu/VTykb/plyV3/paKFwAVjTv ryYWn1F3s5nkV+Oo36sK+LyLyV6XjnkMM8ida9e9kObgx89lFJfUdoL0EanNTHfoKY+9 O4XxcdmKDbkIUU8NnjZdtd5H0yWAAjG+6YIl8XXFRyUkdnwvSv+JAClhWYxaSFsfb9xh gPsdhjm7ucZhZWslkZy333Stzz4f9/WVDgY6QaVwxVWN856TyX2Od/ij7VxcJW12fvQx vDYA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcdkim header.b=Xw2zFrwA; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id t17-20020a63f351000000b0047815413184si24521671pgj.707.2022.12.08.15.31.45; Thu, 08 Dec 2022 15:32:00 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcdkim header.b=Xw2zFrwA; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230156AbiLHXXR (ORCPT + 99 others); Thu, 8 Dec 2022 18:23:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59472 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230139AbiLHXWj (ORCPT ); Thu, 8 Dec 2022 18:22:39 -0500 Received: from alexa-out-sd-02.qualcomm.com (alexa-out-sd-02.qualcomm.com [199.106.114.39]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B4B92A504B; Thu, 8 Dec 2022 15:21:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1670541701; x=1702077701; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=79R+NExqrZIZoVwsptvjNlEk8LlSSWuIaPAXQS6vKic=; b=Xw2zFrwAgCOcjUL28R0XFSfiPgWWWrRaFQlD2rzbf8Gqo4g59SYiQYQz Yd3bivp4CItgAkWcvRB9RKcjWyVBmyK3rd6mLwxiZPvIatiWkTi2x3S7i uGjfTeSObxqU/wlUsPvSfueX1HtSxlz1U+XSof8JA/AOiCzxeOaWjz/wX E=; Received: from unknown (HELO ironmsg04-sd.qualcomm.com) ([10.53.140.144]) by alexa-out-sd-02.qualcomm.com with ESMTP; 08 Dec 2022 15:21:41 -0800 X-QCInternal: smtphost Received: from unknown (HELO nasanex01a.na.qualcomm.com) ([10.52.223.231]) by ironmsg04-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Dec 2022 15:21:41 -0800 Received: from asutoshd-linux1.qualcomm.com (10.80.80.8) by nasanex01a.na.qualcomm.com (10.52.223.231) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Thu, 8 Dec 2022 15:21:41 -0800 From: Asutosh Das To: , , CC: , , , , , , , , , Asutosh Das , , Alim Akhtar , "James E.J. Bottomley" , Jinyoung Choi , Arthur Simchaev , Keoseong Park , Yoshihiro Shimoda , open list Subject: [PATCH v11 15/16] ufs: core: mcq: Add completion support in poll Date: Thu, 8 Dec 2022 15:18:41 -0800 Message-ID: <2d0824aaad6624aba41b3b5546e7473033c6b920.1670541364.git.quic_asutoshd@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01b.na.qualcomm.com (10.46.141.250) To nasanex01a.na.qualcomm.com (10.52.223.231) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751690583389776120?= X-GMAIL-MSGID: =?utf-8?q?1751690583389776120?= Complete cqe requests in poll. Assumption is that several poll completion may happen in different CPUs for the same completion queue. Hence a spin lock protection is added. Co-developed-by: Can Guo Signed-off-by: Can Guo Signed-off-by: Asutosh Das Reviewed-by: Bart Van Assche Reviewed-by: Manivannan Sadhasivam Reviewed-by: Stanley Chu --- drivers/ufs/core/ufs-mcq.c | 13 +++++++++++++ drivers/ufs/core/ufshcd-priv.h | 2 ++ drivers/ufs/core/ufshcd.c | 7 +++++++ include/ufs/ufshcd.h | 2 ++ 4 files changed, 24 insertions(+) diff --git a/drivers/ufs/core/ufs-mcq.c b/drivers/ufs/core/ufs-mcq.c index cd10d59..e710d19 100644 --- a/drivers/ufs/core/ufs-mcq.c +++ b/drivers/ufs/core/ufs-mcq.c @@ -294,6 +294,18 @@ unsigned long ufshcd_mcq_poll_cqe_nolock(struct ufs_hba *hba, return completed_reqs; } +unsigned long ufshcd_mcq_poll_cqe_lock(struct ufs_hba *hba, + struct ufs_hw_queue *hwq) +{ + unsigned long completed_reqs; + + spin_lock(&hwq->cq_lock); + completed_reqs = ufshcd_mcq_poll_cqe_nolock(hba, hwq); + spin_unlock(&hwq->cq_lock); + + return completed_reqs; +} + void ufshcd_mcq_make_queues_operational(struct ufs_hba *hba) { struct ufs_hw_queue *hwq; @@ -390,6 +402,7 @@ int ufshcd_mcq_init(struct ufs_hba *hba) hwq = &hba->uhq[i]; hwq->max_entries = hba->nutrs; spin_lock_init(&hwq->sq_lock); + spin_lock_init(&hwq->cq_lock); } /* The very first HW queue serves device commands */ diff --git a/drivers/ufs/core/ufshcd-priv.h b/drivers/ufs/core/ufshcd-priv.h index 70e3416..ff03aa5 100644 --- a/drivers/ufs/core/ufshcd-priv.h +++ b/drivers/ufs/core/ufshcd-priv.h @@ -75,6 +75,8 @@ unsigned long ufshcd_mcq_poll_cqe_nolock(struct ufs_hba *hba, struct ufs_hw_queue *hwq); struct ufs_hw_queue *ufshcd_mcq_req_to_hwq(struct ufs_hba *hba, struct request *req); +unsigned long ufshcd_mcq_poll_cqe_lock(struct ufs_hba *hba, + struct ufs_hw_queue *hwq); #define UFSHCD_MCQ_IO_QUEUE_OFFSET 1 #define SD_ASCII_STD true diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c index 884dabb..e42d642 100644 --- a/drivers/ufs/core/ufshcd.c +++ b/drivers/ufs/core/ufshcd.c @@ -5475,6 +5475,13 @@ static int ufshcd_poll(struct Scsi_Host *shost, unsigned int queue_num) struct ufs_hba *hba = shost_priv(shost); unsigned long completed_reqs, flags; u32 tr_doorbell; + struct ufs_hw_queue *hwq; + + if (is_mcq_enabled(hba)) { + hwq = &hba->uhq[queue_num + UFSHCD_MCQ_IO_QUEUE_OFFSET]; + + return ufshcd_mcq_poll_cqe_lock(hba, hwq); + } spin_lock_irqsave(&hba->outstanding_lock, flags); tr_doorbell = ufshcd_readl(hba, REG_UTP_TRANSFER_REQ_DOOR_BELL); diff --git a/include/ufs/ufshcd.h b/include/ufs/ufshcd.h index 8441c46..f20557b 100644 --- a/include/ufs/ufshcd.h +++ b/include/ufs/ufshcd.h @@ -1072,6 +1072,7 @@ struct ufs_hba { * @sq_lock: serialize submission queue access * @cq_tail_slot: current slot to which CQ tail pointer is pointing * @cq_head_slot: current slot to which CQ head pointer is pointing + * @cq_lock: Synchronize between multiple polling instances */ struct ufs_hw_queue { void __iomem *mcq_sq_head; @@ -1089,6 +1090,7 @@ struct ufs_hw_queue { spinlock_t sq_lock; u32 cq_tail_slot; u32 cq_head_slot; + spinlock_t cq_lock; }; static inline bool is_mcq_enabled(struct ufs_hba *hba) From patchwork Thu Dec 8 23:18:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Asutosh Das X-Patchwork-Id: 31538 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp478295wrr; Thu, 8 Dec 2022 15:36:15 -0800 (PST) X-Google-Smtp-Source: AA0mqf5EEftYeYcSiBrCPpvgFJhc7f1F4B0VN06e/OrF+lKbhTSHoIBq22rUjYVp3BR3EAS3h7jE X-Received: by 2002:a05:6a20:4db0:b0:9d:efbf:7856 with SMTP id gj48-20020a056a204db000b0009defbf7856mr4191651pzb.11.1670542575043; Thu, 08 Dec 2022 15:36:15 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670542575; cv=none; d=google.com; s=arc-20160816; b=rrHVtofcNUSdE2JBh9ZsX3Bcsw4enB+A26TsdyrR4BHtme+Ifq9vcEFC9tMWzYG0UY EpwQO/DJIumwuzL6T79lU/RTn/L7U2YDdBoqSE4E6etL123EtDQ2ASbiYkWBc5pQPnSH h0veO1CmQEGpIqXs7dQsd1n+b6sXpQQ5g8/kUDVu6uwmbCuafukAC18tMAlZcyWva6GD ftShFEsskT+r2FpN1zOpZDjGcH41/IUn6HrUhAHaUBHS+7s8t4S0trlp6W57iGA6aFfr I9J9ij0C+7L+Y1yL5tQlfK1oqyC0gpoSZMEuh9RVUQSPV/f03oOPngUOE9NnF9fHDSBE qg+w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=8SFDvbzQ1dlolm07vRWQ6nGPb415dRbXSJxfEmPGi4I=; b=qLcewyb2XfbLcgsZdYJylSkIEWBTTBxz1xcKtyRzunNKrTxsSW7nEuPMItLCGhjZKS amHqOum857aRWRCwGwX3VZpsuiCw+6QrAHKPq1fTMeSJs3BTmxdB9qn7vRnAUgtKc6Fg TSQ10ozk+WSoNmSpURYuYCsSALgorOS+QPbdiHNBCiEXkaZokI0AVrz2RQgkWBjaTvq6 sviALzSKqTAdgzRUWxU1ABsS/O230aY1HfTUkbLYnRTXAG0ifVF4XnwPjiQhyE+j+S3+ iOfZDFawjwTLEvyGzIpLZ6pKiVcRKqPx1uvNunYNZH/4nx6uX/h9RFrDUOv8Ah3yY007 11NQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b="JX9Yl/S3"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id l186-20020a6388c3000000b0046e96b9ecfdsi13040902pgd.801.2022.12.08.15.36.00; Thu, 08 Dec 2022 15:36:15 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b="JX9Yl/S3"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230266AbiLHXXi (ORCPT + 99 others); Thu, 8 Dec 2022 18:23:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59962 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230218AbiLHXW5 (ORCPT ); Thu, 8 Dec 2022 18:22:57 -0500 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3C2A9A504F; Thu, 8 Dec 2022 15:21:55 -0800 (PST) Received: from pps.filterd (m0279863.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2B8N7X3I015617; Thu, 8 Dec 2022 23:21:45 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=qcppdkim1; bh=8SFDvbzQ1dlolm07vRWQ6nGPb415dRbXSJxfEmPGi4I=; b=JX9Yl/S3uQf0i4ed0/klIrdU9A6WnqTbL4j2jZrskvQ/Q7PeieJ14gVOnXHrjPcWmkkF dsSV30pW/C2yYi/EMf5i2m1V/7p9Cy5B4f7NM9ax/Q+yihDwdVx9pyPFAUJn9TJFwEzh LgAiE+pg2JK27dCkh6aaUg85fZiTqddKRVqXNR/vMxBzGmQGh71Q1hVghP2hOjTvB9Mo APal3RyjSucj8rmjqZp+kLMMXzVjOgzv5OKpUjwbkfjPv+NiuohIX4o5Q0X7Vi8NOlUK 4LL2pMuuSTPsY3EJ9YeGlxGN5vgz7+IV4k1zJHcrKey96bZDfEgFR/8XAx/KuaAumZr+ CA== Received: from nasanppmta03.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3mavtf4y7c-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 08 Dec 2022 23:21:44 +0000 Received: from nasanex01a.na.qualcomm.com ([10.52.223.231]) by NASANPPMTA03.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 2B8NLiel017541 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 8 Dec 2022 23:21:44 GMT Received: from asutoshd-linux1.qualcomm.com (10.80.80.8) by nasanex01a.na.qualcomm.com (10.52.223.231) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Thu, 8 Dec 2022 15:21:43 -0800 From: Asutosh Das To: , , CC: , , , , , , , , , Asutosh Das , , Alim Akhtar , "James E.J. Bottomley" , Jinyoung Choi , open list Subject: [PATCH v11 16/16] ufs: core: mcq: Enable Multi Circular Queue Date: Thu, 8 Dec 2022 15:18:42 -0800 Message-ID: X-Mailer: git-send-email 2.7.4 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01b.na.qualcomm.com (10.46.141.250) To nasanex01a.na.qualcomm.com (10.52.223.231) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: xxePhSOhwBR-o6G3jTubTL-sbDvT-Oc_ X-Proofpoint-ORIG-GUID: xxePhSOhwBR-o6G3jTubTL-sbDvT-Oc_ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-08_12,2022-12-08_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 lowpriorityscore=0 suspectscore=0 mlxscore=0 mlxlogscore=999 malwarescore=0 clxscore=1015 adultscore=0 spamscore=0 phishscore=0 bulkscore=0 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000 definitions=main-2212080192 X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751690851078815291?= X-GMAIL-MSGID: =?utf-8?q?1751690851078815291?= Enable MCQ in the Host Controller. Signed-off-by: Asutosh Das Reviewed-by: Bart Van Assche Reviewed-by: Manivannan Sadhasivam Reviewed-by: Stanley Chu --- drivers/ufs/core/ufshcd.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c index e42d642..7f0bc10 100644 --- a/drivers/ufs/core/ufshcd.c +++ b/drivers/ufs/core/ufshcd.c @@ -8381,6 +8381,12 @@ static void ufshcd_config_mcq(struct ufs_hba *hba) hba->host->can_queue = hba->nutrs - UFSHCD_NUM_RESERVED; hba->reserved_slot = hba->nutrs - UFSHCD_NUM_RESERVED; + + /* Select MCQ mode */ + ufshcd_writel(hba, ufshcd_readl(hba, REG_UFS_MEM_CFG) | 0x1, + REG_UFS_MEM_CFG); + hba->mcq_enabled = true; + dev_info(hba->dev, "MCQ configured, nr_queues=%d, io_queues=%d, read_queue=%d, poll_queues=%d, queue_depth=%d\n", hba->nr_hw_queues, hba->nr_queues[HCTX_TYPE_DEFAULT], hba->nr_queues[HCTX_TYPE_READ], hba->nr_queues[HCTX_TYPE_POLL],