From patchwork Tue Nov 29 01:20:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Asutosh Das X-Patchwork-Id: 27019 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp57640wrr; Mon, 28 Nov 2022 17:28:16 -0800 (PST) X-Google-Smtp-Source: AA0mqf7ZVn7EHmUtW8tMk1+29yfOix6k82rg44vkyBBk63Y4E6Tto0rRqISfBTrOu6lwV9qpmnxf X-Received: by 2002:a63:4087:0:b0:477:15c8:ff67 with SMTP id n129-20020a634087000000b0047715c8ff67mr33154277pga.275.1669685295727; Mon, 28 Nov 2022 17:28:15 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669685295; cv=none; d=google.com; s=arc-20160816; b=juZq1hKrFlpNz1GdXNk4v+GztZD+hE/mPWFuhB9arwF+JNknxOpR6UBbxZnhj2GejI ztZ7dUC70QixnyYxfqhfcv0d1csLh9gkRU1hIJLSElskoFHsNpWDkBuSVb77R3Srok+m 4arA+5TIgrpSXBUyDaFuZmfuHZCEGYxKwykIg0/w8gokdb+5kK1SNOcSXQKd7jH4LnEY LKy6ZSp6hxTR3CoOz6ySfuxA5R8hmA0xzauaf6tVFA6WSUERgh166Rvg0XgXkbKR1SRw 2bP0LKGusJRuALRTBT9FxpBEv5nYE3nLdkKQ03JTh3EUv+6uGNRe3EAuox0jjVQACSt+ ld7g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=uTUVdg0qczYhFH0uvHZgDzDfpoKPLXX54elwW1V1w00=; b=vXrW0cUFyG5o085y5udA0YRAonsLAJLXdMT0VRC9t6uYVl+z7kOBxC/4xWoZ4XKAwI fx94vNq+uT53xb12nmKOt/KBvvCamxCAcLVvp7Je3aN3or9Y/FyfFCW5GdW7oIvxRZPb tF6JVm1dtIUocq0M5DnKXmsa7qEZKns/vfLfeoJjLjCAuspyasDWKWSBHUQmbQT81/A+ lnMwukEDMmerPkvFtuz4ZPd36f/bd+aYD63HzfcUuODf/pcs91x6kZM2HtLL8suxrepq Mqn0oy1gIfX7GxIVi4gkFHAPPpy8PKYLUQFDxwoQOL2cQ7ecnrGxGSq2rkIkP6Ks1som pvyg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=YmwAOcUi; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id q7-20020a63f947000000b00476caa56d0esi13482979pgk.739.2022.11.28.17.28.02; Mon, 28 Nov 2022 17:28:15 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=YmwAOcUi; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235121AbiK2BZe (ORCPT + 99 others); Mon, 28 Nov 2022 20:25:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53654 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235117AbiK2BZG (ORCPT ); Mon, 28 Nov 2022 20:25:06 -0500 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 64A0845A2F; Mon, 28 Nov 2022 17:24:22 -0800 (PST) Received: from pps.filterd (m0279870.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2AT0wtwJ001513; Tue, 29 Nov 2022 01:24:09 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=qcppdkim1; bh=uTUVdg0qczYhFH0uvHZgDzDfpoKPLXX54elwW1V1w00=; b=YmwAOcUiRFwrHg3xCoc5AmVl+Ud94u0XSHibDxvJ5SRS8qUWRXyemKJue09WMGJ2ibd7 BuCEXPyVLvnqZ5EFoEk+znRPi7qDdyetH0Y6RGHNkvU+RaidHCMpXfmOzfj6p9rKHNUz CQ5+i3eUrmIrYhJEWHkWkx00LzTQrHqMjYvJ7pnp/HoNmJRzAxXPzAGijZ/D0+xpImVA AK/SMGZiXvVuPppssrMapp0Ik3SJwGfLM5IPOjNaunH9E203T72VpIoNNoEZmvqYyiLo i/KC9CkxilFvnZhT16VRSbcgqVWmFRQuCxCGMD+MbTZALlQP2a2A7VFFwr+r0JuTfTtG 8A== Received: from nasanppmta01.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3m3aw35fuu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 29 Nov 2022 01:24:08 +0000 Received: from nasanex01a.na.qualcomm.com (corens_vlan604_snip.qualcomm.com [10.53.140.1]) by NASANPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 2AT1O7uH028814 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 29 Nov 2022 01:24:07 GMT Received: from asutoshd-linux1.qualcomm.com (10.80.80.8) by nasanex01a.na.qualcomm.com (10.52.223.231) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Mon, 28 Nov 2022 17:24:07 -0800 From: Asutosh Das To: , , CC: , , , , , , , , , Asutosh Das , , Alim Akhtar , "James E.J. Bottomley" , Jinyoung Choi , Krzysztof Kozlowski , open list Subject: [PATCH v6 15/16] ufs: core: mcq: Add completion support in poll Date: Mon, 28 Nov 2022 17:20:56 -0800 Message-ID: <7ead6678e8e32e70b5e33e1960af841e7e84532f.1669684648.git.quic_asutoshd@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01b.na.qualcomm.com (10.46.141.250) To nasanex01a.na.qualcomm.com (10.52.223.231) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: DTkTGgoPswnSa4YZm3C3UUfCqwdD_ICi X-Proofpoint-GUID: DTkTGgoPswnSa4YZm3C3UUfCqwdD_ICi X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-11-28_17,2022-11-28_02,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 priorityscore=1501 malwarescore=0 bulkscore=0 suspectscore=0 impostorscore=0 spamscore=0 mlxlogscore=999 mlxscore=0 phishscore=0 adultscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000 definitions=main-2211290006 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750791928755335465?= X-GMAIL-MSGID: =?utf-8?q?1750791928755335465?= Complete cqe requests in poll. Assumption is that several poll completion may happen in different CPUs for the same completion queue. Hence a spin lock protection is added. Co-developed-by: Can Guo Signed-off-by: Can Guo Signed-off-by: Asutosh Das Reviewed-by: Bart Van Assche Reviewed-by: Manivannan Sadhasivam --- drivers/ufs/core/ufs-mcq.c | 13 +++++++++++++ drivers/ufs/core/ufshcd-priv.h | 2 ++ drivers/ufs/core/ufshcd.c | 7 +++++++ include/ufs/ufshcd.h | 2 ++ 4 files changed, 24 insertions(+) diff --git a/drivers/ufs/core/ufs-mcq.c b/drivers/ufs/core/ufs-mcq.c index 68c4097..f99c912 100644 --- a/drivers/ufs/core/ufs-mcq.c +++ b/drivers/ufs/core/ufs-mcq.c @@ -294,6 +294,18 @@ unsigned long ufshcd_mcq_poll_cqe_nolock(struct ufs_hba *hba, return completed_reqs; } +unsigned long ufshcd_mcq_poll_cqe_lock(struct ufs_hba *hba, + struct ufs_hw_queue *hwq) +{ + unsigned long completed_reqs; + + spin_lock(&hwq->cq_lock); + completed_reqs = ufshcd_mcq_poll_cqe_nolock(hba, hwq); + spin_unlock(&hwq->cq_lock); + + return completed_reqs; +} + void ufshcd_mcq_make_queues_operational(struct ufs_hba *hba) { struct ufs_hw_queue *hwq; @@ -390,6 +402,7 @@ int ufshcd_mcq_init(struct ufs_hba *hba) hwq = &hba->uhq[i]; hwq->max_entries = hba->nutrs; spin_lock_init(&hwq->sq_lock); + spin_lock_init(&hwq->cq_lock); } /* The very first HW queue serves device commands */ diff --git a/drivers/ufs/core/ufshcd-priv.h b/drivers/ufs/core/ufshcd-priv.h index 4dcb7f9..d2bd9db 100644 --- a/drivers/ufs/core/ufshcd-priv.h +++ b/drivers/ufs/core/ufshcd-priv.h @@ -75,6 +75,8 @@ unsigned long ufshcd_mcq_poll_cqe_nolock(struct ufs_hba *hba, struct ufs_hw_queue *hwq); struct ufs_hw_queue *ufshcd_mcq_req_to_hwq(struct ufs_hba *hba, struct request *req); +unsigned long ufshcd_mcq_poll_cqe_lock(struct ufs_hba *hba, + struct ufs_hw_queue *hwq); #define UFSHCD_MCQ_IO_QUEUE_OFFSET 1 #define SD_ASCII_STD true diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c index 8d743c3..adf3597 100644 --- a/drivers/ufs/core/ufshcd.c +++ b/drivers/ufs/core/ufshcd.c @@ -5475,6 +5475,13 @@ static int ufshcd_poll(struct Scsi_Host *shost, unsigned int queue_num) struct ufs_hba *hba = shost_priv(shost); unsigned long completed_reqs, flags; u32 tr_doorbell; + struct ufs_hw_queue *hwq; + + if (is_mcq_enabled(hba)) { + hwq = &hba->uhq[queue_num + UFSHCD_MCQ_IO_QUEUE_OFFSET]; + + return ufshcd_mcq_poll_cqe_lock(hba, hwq); + } spin_lock_irqsave(&hba->outstanding_lock, flags); tr_doorbell = ufshcd_readl(hba, REG_UTP_TRANSFER_REQ_DOOR_BELL); diff --git a/include/ufs/ufshcd.h b/include/ufs/ufshcd.h index 8441c46..f20557b 100644 --- a/include/ufs/ufshcd.h +++ b/include/ufs/ufshcd.h @@ -1072,6 +1072,7 @@ struct ufs_hba { * @sq_lock: serialize submission queue access * @cq_tail_slot: current slot to which CQ tail pointer is pointing * @cq_head_slot: current slot to which CQ head pointer is pointing + * @cq_lock: Synchronize between multiple polling instances */ struct ufs_hw_queue { void __iomem *mcq_sq_head; @@ -1089,6 +1090,7 @@ struct ufs_hw_queue { spinlock_t sq_lock; u32 cq_tail_slot; u32 cq_head_slot; + spinlock_t cq_lock; }; static inline bool is_mcq_enabled(struct ufs_hba *hba)