From patchwork Wed Nov 23 04:10:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Asutosh Das X-Patchwork-Id: 24704 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp2585625wrr; Tue, 22 Nov 2022 20:23:00 -0800 (PST) X-Google-Smtp-Source: AA0mqf7s5Pthz5pHo5n/PleX8r/Jyc7MqHjohQLcVJgCfu4jpALHLZRxo6LBMz90lC/hGXd2/d1Q X-Received: by 2002:a17:906:9702:b0:7aa:5e6c:4b59 with SMTP id k2-20020a170906970200b007aa5e6c4b59mr10292669ejx.231.1669177380433; Tue, 22 Nov 2022 20:23:00 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669177380; cv=none; d=google.com; s=arc-20160816; b=yWRIOXoHTo65cYj6Rm29rBLqpRNbv0V/FPxQsaHBKpg5Y8IU1JOaWdxYUB6XqM+Jeu BgQGhnudfy6Oa1cFV3FP5AYwp8d3a6be+USIy6idjZNt+Ymmro53ONRjAzeB90+GHes9 cTbIi1Hn18duslw+lXepvsaz00n6cnHrVTWzmu1kmuHLgxzuspwK/VqIQGOGsKXvjzxI u71LiMK1GZAfbVDIvogtt93gc7Qpj64CDsk/ncG2Mx8XkW/rfA8nsHd/TPflQaAG1Exo yGB8zYZ4smPHA0N9YZbnCIknI2k1qbmJ4hNSGjoIUayzd03EPgcOLB2e3LHAN6PoqJCk l4Pw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=2UB0M6tnPSwa+dbyzYwI6V7KgUvzZUfa+r3E/CZrKHU=; b=PGAug4mPHgtWclKLNbzgrJkO2Hl8S9DpGjfQrn9UtY6rNudzmAx+4+QTIYK0Ad3/Zv 64qLZYgeVbf82Nm0K2gO9bTZy8Y5LQfgQS9a7qXM/az9N1efJvo6YPj+UZ3eHf6UDHpr h+1+u8qHOc/APAQVwJkidlgrfcrNKRO8eJBcV1yZYBdLtRyxoRPCfTfDhVLTn67DQMB2 ime7obMeeOxXvCVNLrNWe3iEnCr97XI04cezI7m2zDqj9gQ8IZR+FOjR2SFR4+jl0SUt 1fw2juNL096n51CWouH5ScnngQuX1NB1Sp4V2RqkPFRdFfaaoSX8qU0vdsaROoA3vWR/ ZNhw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=WzxZwDaN; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id e15-20020a056402190f00b00469496d3bb0si10538203edz.208.2022.11.22.20.22.33; Tue, 22 Nov 2022 20:23:00 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=WzxZwDaN; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235932AbiKWESU (ORCPT + 99 others); Tue, 22 Nov 2022 23:18:20 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43760 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235919AbiKWERY (ORCPT ); Tue, 22 Nov 2022 23:17:24 -0500 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B6BB918B27; Tue, 22 Nov 2022 20:14:38 -0800 (PST) Received: from pps.filterd (m0279867.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2AN2hn60000474; Wed, 23 Nov 2022 04:14:26 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=qcppdkim1; bh=2UB0M6tnPSwa+dbyzYwI6V7KgUvzZUfa+r3E/CZrKHU=; b=WzxZwDaNtCVwBFq+xZoMdqIZU5sYOgh/c929+rPTdrllXYXZMmEAZjHbGwzWXFryCVcx V3oHLX32OPARbLknaBIN38irp5YkwpeZ4pto4P15sAMjSbgizi7oV9CpeoSKmWQQd2cL 7GnvqC0jyYtDXii9YDYkf/NsWAQC6sAG5Xn1lKnwW2iBrvJVCcN1RFC5HX8R9gCCJTVl QXR9UDS+yGGLefLgXgCPLo/zASvNYLqD3ECyUhkkDCz0MNq0yllG2l64DKY9b2BFyMBd CjbopSrs/914pXJt6ecROmhZBN6DL/kjJPWXcvJcUXf09KNYdtJXhW0GdY3HdYQO1XbA vQ== Received: from nasanppmta04.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3m0v0mjkfh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 23 Nov 2022 04:14:26 +0000 Received: from nasanex01a.na.qualcomm.com ([10.52.223.231]) by NASANPPMTA04.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 2AN4EPNW001257 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 23 Nov 2022 04:14:25 GMT Received: from asutoshd-linux1.qualcomm.com (10.80.80.8) by nasanex01a.na.qualcomm.com (10.52.223.231) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Tue, 22 Nov 2022 20:14:25 -0800 From: Asutosh Das To: , , CC: , , , , , , , , , Asutosh Das , , Alim Akhtar , "James E.J. Bottomley" , Jinyoung Choi , Arthur Simchaev , open list Subject: [PATCH v5 15/16] ufs: core: mcq: Add completion support in poll Date: Tue, 22 Nov 2022 20:10:28 -0800 Message-ID: <9d638d7921db39e48de30d4913b28076fe86b6dd.1669176158.git.quic_asutoshd@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01b.na.qualcomm.com (10.46.141.250) To nasanex01a.na.qualcomm.com (10.52.223.231) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: W3HnicobyDNx-z1iRnhD-b0CyCvo4IwV X-Proofpoint-ORIG-GUID: W3HnicobyDNx-z1iRnhD-b0CyCvo4IwV X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-11-23_01,2022-11-18_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 mlxlogscore=999 clxscore=1015 suspectscore=0 malwarescore=0 impostorscore=0 spamscore=0 bulkscore=0 adultscore=0 phishscore=0 mlxscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000 definitions=main-2211230030 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750259341211014321?= X-GMAIL-MSGID: =?utf-8?q?1750259341211014321?= Complete cqe requests in poll. Assumption is that several poll completion may happen in different CPUs for the same completion queue. Hence a spin lock protection is added. Co-developed-by: Can Guo Signed-off-by: Can Guo Signed-off-by: Asutosh Das Reviewed-by: Bart Van Assche Reviewed-by: Manivannan Sadhasivam --- drivers/ufs/core/ufs-mcq.c | 13 +++++++++++++ drivers/ufs/core/ufshcd-priv.h | 2 ++ drivers/ufs/core/ufshcd.c | 7 +++++++ include/ufs/ufshcd.h | 2 ++ 4 files changed, 24 insertions(+) diff --git a/drivers/ufs/core/ufs-mcq.c b/drivers/ufs/core/ufs-mcq.c index 365ad98..5311857 100644 --- a/drivers/ufs/core/ufs-mcq.c +++ b/drivers/ufs/core/ufs-mcq.c @@ -387,6 +387,18 @@ unsigned long ufshcd_mcq_poll_cqe_nolock(struct ufs_hba *hba, return completed_reqs; } +unsigned long ufshcd_mcq_poll_cqe_lock(struct ufs_hba *hba, + struct ufs_hw_queue *hwq) +{ + unsigned long completed_reqs; + + spin_lock(&hwq->cq_lock); + completed_reqs = ufshcd_mcq_poll_cqe_nolock(hba, hwq); + spin_unlock(&hwq->cq_lock); + + return completed_reqs; +} + void ufshcd_mcq_make_queues_operational(struct ufs_hba *hba) { struct ufs_hw_queue *hwq; @@ -483,6 +495,7 @@ int ufshcd_mcq_init(struct ufs_hba *hba) hwq = &hba->uhq[i]; hwq->max_entries = hba->nutrs; spin_lock_init(&hwq->sq_lock); + spin_lock_init(&hwq->cq_lock); } /* The very first HW queue serves device commands */ diff --git a/drivers/ufs/core/ufshcd-priv.h b/drivers/ufs/core/ufshcd-priv.h index c5b5bf3..73ce8a2 100644 --- a/drivers/ufs/core/ufshcd-priv.h +++ b/drivers/ufs/core/ufshcd-priv.h @@ -75,6 +75,8 @@ unsigned long ufshcd_mcq_poll_cqe_nolock(struct ufs_hba *hba, struct ufs_hw_queue *hwq); struct ufs_hw_queue *ufshcd_mcq_req_to_hwq(struct ufs_hba *hba, struct request *req); +unsigned long ufshcd_mcq_poll_cqe_lock(struct ufs_hba *hba, + struct ufs_hw_queue *hwq); #define UFSHCD_MCQ_IO_QUEUE_OFFSET 1 #define SD_ASCII_STD true diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c index 7fb7c5f..8416d42 100644 --- a/drivers/ufs/core/ufshcd.c +++ b/drivers/ufs/core/ufshcd.c @@ -5453,6 +5453,13 @@ static int ufshcd_poll(struct Scsi_Host *shost, unsigned int queue_num) struct ufs_hba *hba = shost_priv(shost); unsigned long completed_reqs, flags; u32 tr_doorbell; + struct ufs_hw_queue *hwq; + + if (is_mcq_enabled(hba)) { + hwq = &hba->uhq[queue_num + UFSHCD_MCQ_IO_QUEUE_OFFSET]; + + return ufshcd_mcq_poll_cqe_lock(hba, hwq); + } spin_lock_irqsave(&hba->outstanding_lock, flags); tr_doorbell = ufshcd_readl(hba, REG_UTP_TRANSFER_REQ_DOOR_BELL); diff --git a/include/ufs/ufshcd.h b/include/ufs/ufshcd.h index d5fde64..a709391 100644 --- a/include/ufs/ufshcd.h +++ b/include/ufs/ufshcd.h @@ -1069,6 +1069,7 @@ struct ufs_hba { * @sq_lock: serialize submission queue access * @cq_tail_slot: current slot to which CQ tail pointer is pointing * @cq_head_slot: current slot to which CQ head pointer is pointing + * @cq_lock: Synchronize between multiple polling instances */ struct ufs_hw_queue { void __iomem *mcq_sq_head; @@ -1086,6 +1087,7 @@ struct ufs_hw_queue { spinlock_t sq_lock; u32 cq_tail_slot; u32 cq_head_slot; + spinlock_t cq_lock; }; static inline bool is_mcq_enabled(struct ufs_hba *hba)