From patchwork Thu Dec 8 23:18:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Asutosh Das X-Patchwork-Id: 31533 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp476755wrr; Thu, 8 Dec 2022 15:32:00 -0800 (PST) X-Google-Smtp-Source: AA0mqf5XbX1yg9lwn/D1bveLkTiME3MFos1pbgENutbJu8jfaw3j+WQg7pUXC0LyypJZ8+N3Si20 X-Received: by 2002:a17:90b:2545:b0:213:113b:9085 with SMTP id nw5-20020a17090b254500b00213113b9085mr3651623pjb.23.1670542320145; Thu, 08 Dec 2022 15:32:00 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670542320; cv=none; d=google.com; s=arc-20160816; b=ZFigOAYwLkXBwQfAmTPRai3zJg3RLW9Eir3SMFCnBGAez6koHHWU/nuuIBcZ7C6Xlr PURMVsDhtC4YDr4Dd954sQzl/hs3m2otrYjagWQSPZCRGCbbE/5z6dsQ4vWjt8En/MOz lu5T5jcJINs06S9M9InkaSnOYJVOKbM4vlswMZSXRzaEIGSkr7CgHDPi2zo38VS0gxyK cVBsun838CT17olS8vTJLuW5QhQYvV5Ozg9513k2VTsHKORxqN0cSLn7+juYM/SUCnbs 4lR2XR2bFXMkswDfvrGC2CyN6li4KkCpWXKA65nCoLnyP+5OHgDLa7Su0lXWyqQkqmCA h5MQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=79R+NExqrZIZoVwsptvjNlEk8LlSSWuIaPAXQS6vKic=; b=a+W8MF4z8InO3edTkcn9X0GhrqfWXD5afYfEqKaLJbEFgt/vRngTbHUc1ilbqe5YH6 Ra7cRbGMkgRUqFw+E7WaMqncyLPSFXBrP2HQWs1yD+5Qu/VTykb/plyV3/paKFwAVjTv ryYWn1F3s5nkV+Oo36sK+LyLyV6XjnkMM8ida9e9kObgx89lFJfUdoL0EanNTHfoKY+9 O4XxcdmKDbkIUU8NnjZdtd5H0yWAAjG+6YIl8XXFRyUkdnwvSv+JAClhWYxaSFsfb9xh gPsdhjm7ucZhZWslkZy333Stzz4f9/WVDgY6QaVwxVWN856TyX2Od/ij7VxcJW12fvQx vDYA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcdkim header.b=Xw2zFrwA; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id t17-20020a63f351000000b0047815413184si24521671pgj.707.2022.12.08.15.31.45; Thu, 08 Dec 2022 15:32:00 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcdkim header.b=Xw2zFrwA; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230156AbiLHXXR (ORCPT + 99 others); Thu, 8 Dec 2022 18:23:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59472 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230139AbiLHXWj (ORCPT ); Thu, 8 Dec 2022 18:22:39 -0500 Received: from alexa-out-sd-02.qualcomm.com (alexa-out-sd-02.qualcomm.com [199.106.114.39]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B4B92A504B; Thu, 8 Dec 2022 15:21:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1670541701; x=1702077701; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=79R+NExqrZIZoVwsptvjNlEk8LlSSWuIaPAXQS6vKic=; b=Xw2zFrwAgCOcjUL28R0XFSfiPgWWWrRaFQlD2rzbf8Gqo4g59SYiQYQz Yd3bivp4CItgAkWcvRB9RKcjWyVBmyK3rd6mLwxiZPvIatiWkTi2x3S7i uGjfTeSObxqU/wlUsPvSfueX1HtSxlz1U+XSof8JA/AOiCzxeOaWjz/wX E=; Received: from unknown (HELO ironmsg04-sd.qualcomm.com) ([10.53.140.144]) by alexa-out-sd-02.qualcomm.com with ESMTP; 08 Dec 2022 15:21:41 -0800 X-QCInternal: smtphost Received: from unknown (HELO nasanex01a.na.qualcomm.com) ([10.52.223.231]) by ironmsg04-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Dec 2022 15:21:41 -0800 Received: from asutoshd-linux1.qualcomm.com (10.80.80.8) by nasanex01a.na.qualcomm.com (10.52.223.231) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Thu, 8 Dec 2022 15:21:41 -0800 From: Asutosh Das To: , , CC: , , , , , , , , , Asutosh Das , , Alim Akhtar , "James E.J. Bottomley" , Jinyoung Choi , Arthur Simchaev , Keoseong Park , Yoshihiro Shimoda , open list Subject: [PATCH v11 15/16] ufs: core: mcq: Add completion support in poll Date: Thu, 8 Dec 2022 15:18:41 -0800 Message-ID: <2d0824aaad6624aba41b3b5546e7473033c6b920.1670541364.git.quic_asutoshd@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01b.na.qualcomm.com (10.46.141.250) To nasanex01a.na.qualcomm.com (10.52.223.231) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751690583389776120?= X-GMAIL-MSGID: =?utf-8?q?1751690583389776120?= Complete cqe requests in poll. Assumption is that several poll completion may happen in different CPUs for the same completion queue. Hence a spin lock protection is added. Co-developed-by: Can Guo Signed-off-by: Can Guo Signed-off-by: Asutosh Das Reviewed-by: Bart Van Assche Reviewed-by: Manivannan Sadhasivam Reviewed-by: Stanley Chu --- drivers/ufs/core/ufs-mcq.c | 13 +++++++++++++ drivers/ufs/core/ufshcd-priv.h | 2 ++ drivers/ufs/core/ufshcd.c | 7 +++++++ include/ufs/ufshcd.h | 2 ++ 4 files changed, 24 insertions(+) diff --git a/drivers/ufs/core/ufs-mcq.c b/drivers/ufs/core/ufs-mcq.c index cd10d59..e710d19 100644 --- a/drivers/ufs/core/ufs-mcq.c +++ b/drivers/ufs/core/ufs-mcq.c @@ -294,6 +294,18 @@ unsigned long ufshcd_mcq_poll_cqe_nolock(struct ufs_hba *hba, return completed_reqs; } +unsigned long ufshcd_mcq_poll_cqe_lock(struct ufs_hba *hba, + struct ufs_hw_queue *hwq) +{ + unsigned long completed_reqs; + + spin_lock(&hwq->cq_lock); + completed_reqs = ufshcd_mcq_poll_cqe_nolock(hba, hwq); + spin_unlock(&hwq->cq_lock); + + return completed_reqs; +} + void ufshcd_mcq_make_queues_operational(struct ufs_hba *hba) { struct ufs_hw_queue *hwq; @@ -390,6 +402,7 @@ int ufshcd_mcq_init(struct ufs_hba *hba) hwq = &hba->uhq[i]; hwq->max_entries = hba->nutrs; spin_lock_init(&hwq->sq_lock); + spin_lock_init(&hwq->cq_lock); } /* The very first HW queue serves device commands */ diff --git a/drivers/ufs/core/ufshcd-priv.h b/drivers/ufs/core/ufshcd-priv.h index 70e3416..ff03aa5 100644 --- a/drivers/ufs/core/ufshcd-priv.h +++ b/drivers/ufs/core/ufshcd-priv.h @@ -75,6 +75,8 @@ unsigned long ufshcd_mcq_poll_cqe_nolock(struct ufs_hba *hba, struct ufs_hw_queue *hwq); struct ufs_hw_queue *ufshcd_mcq_req_to_hwq(struct ufs_hba *hba, struct request *req); +unsigned long ufshcd_mcq_poll_cqe_lock(struct ufs_hba *hba, + struct ufs_hw_queue *hwq); #define UFSHCD_MCQ_IO_QUEUE_OFFSET 1 #define SD_ASCII_STD true diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c index 884dabb..e42d642 100644 --- a/drivers/ufs/core/ufshcd.c +++ b/drivers/ufs/core/ufshcd.c @@ -5475,6 +5475,13 @@ static int ufshcd_poll(struct Scsi_Host *shost, unsigned int queue_num) struct ufs_hba *hba = shost_priv(shost); unsigned long completed_reqs, flags; u32 tr_doorbell; + struct ufs_hw_queue *hwq; + + if (is_mcq_enabled(hba)) { + hwq = &hba->uhq[queue_num + UFSHCD_MCQ_IO_QUEUE_OFFSET]; + + return ufshcd_mcq_poll_cqe_lock(hba, hwq); + } spin_lock_irqsave(&hba->outstanding_lock, flags); tr_doorbell = ufshcd_readl(hba, REG_UTP_TRANSFER_REQ_DOOR_BELL); diff --git a/include/ufs/ufshcd.h b/include/ufs/ufshcd.h index 8441c46..f20557b 100644 --- a/include/ufs/ufshcd.h +++ b/include/ufs/ufshcd.h @@ -1072,6 +1072,7 @@ struct ufs_hba { * @sq_lock: serialize submission queue access * @cq_tail_slot: current slot to which CQ tail pointer is pointing * @cq_head_slot: current slot to which CQ head pointer is pointing + * @cq_lock: Synchronize between multiple polling instances */ struct ufs_hw_queue { void __iomem *mcq_sq_head; @@ -1089,6 +1090,7 @@ struct ufs_hw_queue { spinlock_t sq_lock; u32 cq_tail_slot; u32 cq_head_slot; + spinlock_t cq_lock; }; static inline bool is_mcq_enabled(struct ufs_hba *hba)