From patchwork Tue Nov 29 18:53:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Asutosh Das X-Patchwork-Id: 27410 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp515078wrr; Tue, 29 Nov 2022 11:02:34 -0800 (PST) X-Google-Smtp-Source: AA0mqf4gtMH503oBylZu5dJvm2tG+8qRkMW1jiWCFJfL30Dy7xHujaoGcI5KJOQ0ECuijlsTS84f X-Received: by 2002:a17:902:cf4b:b0:186:7a1d:b6ee with SMTP id e11-20020a170902cf4b00b001867a1db6eemr43974722plg.67.1669748554344; Tue, 29 Nov 2022 11:02:34 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669748554; cv=none; d=google.com; s=arc-20160816; b=Ub3eMghexGLWOMrOenU71P0aAj+975+V+/25wiu9wDbibUURN5UueyxCHuRceyn85o J/wyBD+Ma608WBFE+vZswCQu6C48owB3J4C2wN4ktnQ8GcljlUw9Hp7U0Hyoz2vclhwC 5ngY33GxlU6hIBKDshIB8/R6YUBsbiMR/5WCFLLyv+sBjyOUY3OdFJIKiGiM11YGcNND Vkrnt5GxJiz97JDjNeCe9qyMTnz5ItkSSufbk06CxxIYjJ5kOUxj6BJ7FqgGlMm25eJI c9oesiJZuCbX2NzMlv21JijFy9dDJ/DydmjHbQj8C9vt5i7lDhRXrKaM48gQdJrZDOkD QQ0w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=uTUVdg0qczYhFH0uvHZgDzDfpoKPLXX54elwW1V1w00=; b=BaWBdqovweVqjBEG9JdCS/bTiMQ1XanerYFqv4l/DPkBY3LLIG0vyzVOaY7NYoomLC 7ImIUjX8Et3TODsOwq+cNTCP9E3oCFSWNV4vaoXalTqP03tVXQGilSkgVnMJaIYJ+QZ3 yjFWlDUHImmsR6VggvcfQYveWVPOmUPhIvfJzLtmNzq3/n6Ze0KsiJIYV9I96S+oXgUk P3kzKtRq0Pd/bMqIZovmjdAYNIOu3A0bqOITspu7Oiajk1/WYnPVY7q79vo6g2RTQrRN 7urRVaDSuInFuUOMhXx1gmhxPEeKuljrqkzlO/xlPn/FrwwuwUKPI7LZh59HoA3Eeb6J rcww== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=U2yAJ3zE; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id y30-20020a634b1e000000b00476d1591b76si13863538pga.119.2022.11.29.11.02.20; Tue, 29 Nov 2022 11:02:34 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=U2yAJ3zE; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236576AbiK2S7X (ORCPT + 99 others); Tue, 29 Nov 2022 13:59:23 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60932 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236457AbiK2S6x (ORCPT ); Tue, 29 Nov 2022 13:58:53 -0500 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A1C286C72A; Tue, 29 Nov 2022 10:57:14 -0800 (PST) Received: from pps.filterd (m0279869.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2ATIteCp018438; Tue, 29 Nov 2022 18:56:59 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=qcppdkim1; bh=uTUVdg0qczYhFH0uvHZgDzDfpoKPLXX54elwW1V1w00=; b=U2yAJ3zERH/Vtt/P9TUp0aIsCz88XDhYiaCffM6UuP/FGXMto0kELJcwim7wG+KfEklJ o2eFwPJvshjDtUXrh6EHugwOIDhqewMKrogB4Tuv5BCBfweyMSxt+ef2pv1aZyLQiGvF 7smFlgdK6Z6b9GPaCKhSNDt9ulxTIyoSKMY6UYSCsV/YZqQAjDxxmuylkX1jwcIkOEc1 sHgwQMDlmPU1NP9hDpI/u2Mzlcb2cWvnXqXMzQSLAOVj1UKTqkC5f3ZsjI8k6FaAAe+7 a830hbiwgtPTJ5p6SiynyaaRPrmUm78d0BOD7Kg9smG5fJ9P7AX9lKQd45hziP6ZYx3d Ng== Received: from nasanppmta05.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3m5bx0t4e6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 29 Nov 2022 18:56:59 +0000 Received: from nasanex01a.na.qualcomm.com ([10.52.223.231]) by NASANPPMTA05.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 2ATIuwcP012661 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 29 Nov 2022 18:56:58 GMT Received: from asutoshd-linux1.qualcomm.com (10.80.80.8) by nasanex01a.na.qualcomm.com (10.52.223.231) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Tue, 29 Nov 2022 10:56:57 -0800 From: Asutosh Das To: , , CC: , , , , , , , , , Asutosh Das , , Alim Akhtar , "James E.J. Bottomley" , Arthur Simchaev , Krzysztof Kozlowski , Jinyoung Choi , open list Subject: [PATCH v7 15/16] ufs: core: mcq: Add completion support in poll Date: Tue, 29 Nov 2022 10:53:21 -0800 Message-ID: <9b577733ecdb271301aa66176154f22b4d07b9b7.1669747235.git.quic_asutoshd@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01a.na.qualcomm.com (10.52.223.231) To nasanex01a.na.qualcomm.com (10.52.223.231) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: Dc77Dz2zwMVTPVlCHN-LyezgdpQPi_GF X-Proofpoint-ORIG-GUID: Dc77Dz2zwMVTPVlCHN-LyezgdpQPi_GF X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-11-29_11,2022-11-29_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 spamscore=0 bulkscore=0 malwarescore=0 mlxscore=0 lowpriorityscore=0 phishscore=0 clxscore=1015 mlxlogscore=999 suspectscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000 definitions=main-2211290108 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750858260053767109?= X-GMAIL-MSGID: =?utf-8?q?1750858260053767109?= Complete cqe requests in poll. Assumption is that several poll completion may happen in different CPUs for the same completion queue. Hence a spin lock protection is added. Co-developed-by: Can Guo Signed-off-by: Can Guo Signed-off-by: Asutosh Das Reviewed-by: Bart Van Assche Reviewed-by: Manivannan Sadhasivam --- drivers/ufs/core/ufs-mcq.c | 13 +++++++++++++ drivers/ufs/core/ufshcd-priv.h | 2 ++ drivers/ufs/core/ufshcd.c | 7 +++++++ include/ufs/ufshcd.h | 2 ++ 4 files changed, 24 insertions(+) diff --git a/drivers/ufs/core/ufs-mcq.c b/drivers/ufs/core/ufs-mcq.c index 68c4097..f99c912 100644 --- a/drivers/ufs/core/ufs-mcq.c +++ b/drivers/ufs/core/ufs-mcq.c @@ -294,6 +294,18 @@ unsigned long ufshcd_mcq_poll_cqe_nolock(struct ufs_hba *hba, return completed_reqs; } +unsigned long ufshcd_mcq_poll_cqe_lock(struct ufs_hba *hba, + struct ufs_hw_queue *hwq) +{ + unsigned long completed_reqs; + + spin_lock(&hwq->cq_lock); + completed_reqs = ufshcd_mcq_poll_cqe_nolock(hba, hwq); + spin_unlock(&hwq->cq_lock); + + return completed_reqs; +} + void ufshcd_mcq_make_queues_operational(struct ufs_hba *hba) { struct ufs_hw_queue *hwq; @@ -390,6 +402,7 @@ int ufshcd_mcq_init(struct ufs_hba *hba) hwq = &hba->uhq[i]; hwq->max_entries = hba->nutrs; spin_lock_init(&hwq->sq_lock); + spin_lock_init(&hwq->cq_lock); } /* The very first HW queue serves device commands */ diff --git a/drivers/ufs/core/ufshcd-priv.h b/drivers/ufs/core/ufshcd-priv.h index 4dcb7f9..d2bd9db 100644 --- a/drivers/ufs/core/ufshcd-priv.h +++ b/drivers/ufs/core/ufshcd-priv.h @@ -75,6 +75,8 @@ unsigned long ufshcd_mcq_poll_cqe_nolock(struct ufs_hba *hba, struct ufs_hw_queue *hwq); struct ufs_hw_queue *ufshcd_mcq_req_to_hwq(struct ufs_hba *hba, struct request *req); +unsigned long ufshcd_mcq_poll_cqe_lock(struct ufs_hba *hba, + struct ufs_hw_queue *hwq); #define UFSHCD_MCQ_IO_QUEUE_OFFSET 1 #define SD_ASCII_STD true diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c index 8d743c3..adf3597 100644 --- a/drivers/ufs/core/ufshcd.c +++ b/drivers/ufs/core/ufshcd.c @@ -5475,6 +5475,13 @@ static int ufshcd_poll(struct Scsi_Host *shost, unsigned int queue_num) struct ufs_hba *hba = shost_priv(shost); unsigned long completed_reqs, flags; u32 tr_doorbell; + struct ufs_hw_queue *hwq; + + if (is_mcq_enabled(hba)) { + hwq = &hba->uhq[queue_num + UFSHCD_MCQ_IO_QUEUE_OFFSET]; + + return ufshcd_mcq_poll_cqe_lock(hba, hwq); + } spin_lock_irqsave(&hba->outstanding_lock, flags); tr_doorbell = ufshcd_readl(hba, REG_UTP_TRANSFER_REQ_DOOR_BELL); diff --git a/include/ufs/ufshcd.h b/include/ufs/ufshcd.h index 8441c46..f20557b 100644 --- a/include/ufs/ufshcd.h +++ b/include/ufs/ufshcd.h @@ -1072,6 +1072,7 @@ struct ufs_hba { * @sq_lock: serialize submission queue access * @cq_tail_slot: current slot to which CQ tail pointer is pointing * @cq_head_slot: current slot to which CQ head pointer is pointing + * @cq_lock: Synchronize between multiple polling instances */ struct ufs_hw_queue { void __iomem *mcq_sq_head; @@ -1089,6 +1090,7 @@ struct ufs_hw_queue { spinlock_t sq_lock; u32 cq_tail_slot; u32 cq_head_slot; + spinlock_t cq_lock; }; static inline bool is_mcq_enabled(struct ufs_hba *hba)