From patchwork Wed Nov 23 04:10:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Asutosh Das X-Patchwork-Id: 24692 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp2583393wrr; Tue, 22 Nov 2022 20:15:47 -0800 (PST) X-Google-Smtp-Source: AA0mqf7bXebuvlRw+N3csd4qim2NWLBGeRJIPQSbDccgmB0XIc10Ta4xid0FY7/U556eWL235DKw X-Received: by 2002:a17:906:80da:b0:7ae:5473:3c76 with SMTP id a26-20020a17090680da00b007ae54733c76mr6668973ejx.752.1669176946862; Tue, 22 Nov 2022 20:15:46 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669176946; cv=none; d=google.com; s=arc-20160816; b=o5CRio8rY6fY8wFKRVYoKF8DRYg2hSFS5C74CSXy4nRgfzpXZ+mg6e3ie2cVUnoo6J pxIDZot9DYx+mPmhqSZH9wUb80l5q3eFPVMt2ZbN8SBvJ3RUycZEFFRp0SFAyHjPkKba aHHtgeJtLSBNlgINqgVWPjq5VU+//DZLoiOHkfH1qtxbFOEQHS+zBru3SQkRQNFRGLuF tzRrtmtLlxMPrgK8OiEw4M4PYTfCH5uglQ75VgCnUErewQCRnvgfCJybh1+9NkNnNJB+ Uyo6hR10fn8dTu62E2MYRx25i70YprI74LZKP/YzWKLQb9Yk6CLhITvE+FoIUk6GlD9m Kwtw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=wWFQMmJySihT1fg9LQSFIRwajU37I/EdTWitLP2kyA8=; b=YgkDaXAt/7GC1mIGNRjciBjdT6ZL3OU6XXwmHrphvPiZzQnXVW/Y0UBXgvtU3u8Wd/ EJdH5wBdSH6IcEkY/IQ+cYL4Pq0ak+ABrUmKoKBOn1jqgrvsuIVcGzdaDX5rm5Q7ChFL +fxxozRAmGVA2EdOR1s7TFyxZYm18hzgbr7RugLjQCMWMrnuZe1jDkUeU2L0RmRBGoX3 Z0ruy0Ljd6Onj5B3zBscCNMhj9Qgu4RGFv5m6m74QSxPYecRBT0aNcn2eJdxvKaG998L DUJcWSA0tv9OMLyxS9Qo+ubGjqvjOXjhPmBGepH8Yt2wsT15MDshQhKMcsASr1SYNfLF KAVw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=Zlt6SBTN; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id xh13-20020a170906da8d00b0078e2828052asi13283914ejb.638.2022.11.22.20.15.21; Tue, 22 Nov 2022 20:15:46 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=Zlt6SBTN; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235733AbiKWEOY (ORCPT + 99 others); Tue, 22 Nov 2022 23:14:24 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39550 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235762AbiKWENu (ORCPT ); Tue, 22 Nov 2022 23:13:50 -0500 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6BC6966C9A; Tue, 22 Nov 2022 20:13:14 -0800 (PST) Received: from pps.filterd (m0279873.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2AN3pUjY031227; Wed, 23 Nov 2022 04:13:00 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=qcppdkim1; bh=wWFQMmJySihT1fg9LQSFIRwajU37I/EdTWitLP2kyA8=; b=Zlt6SBTNxuqNJ+jA0WK8PRCBkUvF6aKwYZx2ylKrViQuAYeGFMvGHzpGsPRS3z+RemUx EYWxR+WVEDXGz8deg2vnqJ0EvFFt3smw1bfGjj2xWSHc4UpW3a+wIhEGzgnJgIB+ptTf BiFPZlT5dV/YX/YBQ/mk1usTxGnpxJXSGamsV4xSKvoBxCtOsioRMGLy2+nVu4Z1iqrB lXN+ltH2UJokGD/FRTmAbrjKvm3d3EBNnOL+6ZuLt/sJ2NHg2psWkxZdfwb4zTlzdaeg IyjD2KiCR7MX4oimP9hFKw33fKaqcdRcoRgwBryqG3JI0yQPIM0U2cNu6HjcViffcb8i Nw== Received: from nasanppmta05.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3m11bu1de7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 23 Nov 2022 04:12:59 +0000 Received: from nasanex01a.na.qualcomm.com ([10.52.223.231]) by NASANPPMTA05.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 2AN4Cw2d023968 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 23 Nov 2022 04:12:58 GMT Received: from asutoshd-linux1.qualcomm.com (10.80.80.8) by nasanex01a.na.qualcomm.com (10.52.223.231) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Tue, 22 Nov 2022 20:12:58 -0800 From: Asutosh Das To: , , CC: , , , , , , , , , Asutosh Das , , Alim Akhtar , "James E.J. Bottomley" , Jinyoung Choi , Krzysztof Kozlowski , Kiwoong Kim , "open list" Subject: [PATCH v5 08/16] ufs: core: mcq: Allocate memory for mcq mode Date: Tue, 22 Nov 2022 20:10:21 -0800 Message-ID: X-Mailer: git-send-email 2.7.4 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01b.na.qualcomm.com (10.46.141.250) To nasanex01a.na.qualcomm.com (10.52.223.231) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: aTRy0eoLoe9ipb9Xrt5SfYTWbDt_0mAF X-Proofpoint-ORIG-GUID: aTRy0eoLoe9ipb9Xrt5SfYTWbDt_0mAF X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-11-22_13,2022-11-18_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 lowpriorityscore=0 mlxlogscore=999 spamscore=0 bulkscore=0 impostorscore=0 malwarescore=0 priorityscore=1501 phishscore=0 adultscore=0 clxscore=1015 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000 definitions=main-2211230030 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750258886202248239?= X-GMAIL-MSGID: =?utf-8?q?1750258886202248239?= To read the bqueuedepth, the device descriptor is fetched in Single Doorbell Mode. This allocated memory may not be enough for MCQ mode because the number of tags supported in MCQ mode may be larger than in SDB mode. Hence, release the memory allocated in SDB mode and allocate memory for MCQ mode operation. Define the ufs hardware queue and Completion Queue Entry. Co-developed-by: Can Guo Signed-off-by: Can Guo Signed-off-by: Asutosh Das Reviewed-by: Bart Van Assche --- drivers/ufs/core/ufs-mcq.c | 58 ++++++++++++++++++++++++++++++++++++++++-- drivers/ufs/core/ufshcd-priv.h | 1 + drivers/ufs/core/ufshcd.c | 42 +++++++++++++++++++++++++++--- include/ufs/ufshcd.h | 19 ++++++++++++++ include/ufs/ufshci.h | 22 ++++++++++++++++ 5 files changed, 137 insertions(+), 5 deletions(-) diff --git a/drivers/ufs/core/ufs-mcq.c b/drivers/ufs/core/ufs-mcq.c index e95f748..51f0e40 100644 --- a/drivers/ufs/core/ufs-mcq.c +++ b/drivers/ufs/core/ufs-mcq.c @@ -247,15 +247,69 @@ static int ufshcd_mcq_config_nr_queues(struct ufs_hba *hba) return 0; } +int ufshcd_mcq_memory_alloc(struct ufs_hba *hba) +{ + struct ufs_hw_queue *hwq; + size_t utrdl_size, cqe_size; + int i; + + for (i = 0; i < hba->nr_hw_queues; i++) { + hwq = &hba->uhq[i]; + + utrdl_size = sizeof(struct utp_transfer_req_desc) * + hwq->max_entries; + hwq->sqe_base_addr = dmam_alloc_coherent(hba->dev, utrdl_size, + &hwq->sqe_dma_addr, + GFP_KERNEL); + if (!hwq->sqe_dma_addr) { + dev_err(hba->dev, "SQE allocation failed\n"); + return -ENOMEM; + } + + cqe_size = sizeof(struct cq_entry) * hwq->max_entries; + hwq->cqe_base_addr = dmam_alloc_coherent(hba->dev, cqe_size, + &hwq->cqe_dma_addr, + GFP_KERNEL); + if (!hwq->cqe_dma_addr) { + dev_err(hba->dev, "CQE allocation failed\n"); + return -ENOMEM; + } + } + + return 0; +} + int ufshcd_mcq_init(struct ufs_hba *hba) { - int ret; + struct ufs_hw_queue *hwq; + int ret, i; ret = ufshcd_mcq_config_nr_queues(hba); if (ret) return ret; ret = ufshcd_mcq_config_resource(hba); - return ret; + if (ret) + return ret; + + hba->uhq = devm_kzalloc(hba->dev, + hba->nr_hw_queues * sizeof(struct ufs_hw_queue), + GFP_KERNEL); + if (!hba->uhq) { + dev_err(hba->dev, "ufs hw queue memory allocation failed\n"); + return -ENOMEM; + } + + for (i = 0; i < hba->nr_hw_queues; i++) { + hwq = &hba->uhq[i]; + hwq->max_entries = hba->nutrs; + } + + /* The very first HW queue serves device commands */ + hba->dev_cmd_queue = &hba->uhq[0]; + /* Give dev_cmd_queue the minimal number of entries */ + hba->dev_cmd_queue->max_entries = MAX_DEV_CMD_ENTRIES; + + return 0; } diff --git a/drivers/ufs/core/ufshcd-priv.h b/drivers/ufs/core/ufshcd-priv.h index 9f40fa5..4d2bde2 100644 --- a/drivers/ufs/core/ufshcd-priv.h +++ b/drivers/ufs/core/ufshcd-priv.h @@ -63,6 +63,7 @@ int ufshcd_query_flag(struct ufs_hba *hba, enum query_opcode opcode, void ufshcd_auto_hibern8_update(struct ufs_hba *hba, u32 ahit); int ufshcd_mcq_init(struct ufs_hba *hba); int ufshcd_mcq_decide_queue_depth(struct ufs_hba *hba); +int ufshcd_mcq_memory_alloc(struct ufs_hba *hba); #define SD_ASCII_STD true #define SD_RAW false diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c index ae065da..45686e8 100644 --- a/drivers/ufs/core/ufshcd.c +++ b/drivers/ufs/core/ufshcd.c @@ -3740,6 +3740,12 @@ static int ufshcd_memory_alloc(struct ufs_hba *hba) } /* + * Not freed if MCQ is configured see ufshcd_release_sdb_queue() and + * ufshcd_config_mcq() + */ + if (hba->utmrdl_base_addr) + goto skip_utmrdl; + /* * Allocate memory for UTP Task Management descriptors * UFSHCI requires 1024 byte alignment of UTMRD */ @@ -3755,6 +3761,7 @@ static int ufshcd_memory_alloc(struct ufs_hba *hba) goto out; } +skip_utmrdl: /* Allocate memory for local reference block */ hba->lrb = devm_kcalloc(hba->dev, hba->nutrs, sizeof(struct ufshcd_lrb), @@ -8197,6 +8204,22 @@ static int ufshcd_add_lus(struct ufs_hba *hba) return ret; } +/* SDB - Single Doorbell */ +static void ufshcd_release_sdb_queue(struct ufs_hba *hba, int nutrs) +{ + size_t ucdl_size, utrdl_size; + + ucdl_size = sizeof(struct utp_transfer_cmd_desc) * nutrs; + dmam_free_coherent(hba->dev, ucdl_size, hba->ucdl_base_addr, + hba->ucdl_dma_addr); + + utrdl_size = sizeof(struct utp_transfer_req_desc) * nutrs; + dmam_free_coherent(hba->dev, utrdl_size, hba->utrdl_base_addr, + hba->utrdl_dma_addr); + + devm_kfree(hba->dev, hba->lrb); +} + static int ufshcd_alloc_mcq(struct ufs_hba *hba) { int ret; @@ -8208,12 +8231,25 @@ static int ufshcd_alloc_mcq(struct ufs_hba *hba) hba->nutrs = ret; ret = ufshcd_mcq_init(hba); - if (ret) { - hba->nutrs = old_nutrs; - return ret; + if (ret) + goto err; + + if (hba->nutrs != old_nutrs) { + ufshcd_release_sdb_queue(hba, old_nutrs); + ret = ufshcd_memory_alloc(hba); + if (ret) + goto err; + ufshcd_host_memory_configure(hba); } + ret = ufshcd_mcq_memory_alloc(hba); + if (ret) + goto err; + return 0; +err: + hba->nutrs = old_nutrs; + return ret; } /** diff --git a/include/ufs/ufshcd.h b/include/ufs/ufshcd.h index e03b310..e478bab 100644 --- a/include/ufs/ufshcd.h +++ b/include/ufs/ufshcd.h @@ -863,6 +863,8 @@ enum ufshcd_res { * @nr_queues: number of Queues of different queue types * @res: array of resource info of MCQ registers * @mcq_base: Multi circular queue registers base address + * @uhq: array of supported hardware queues + * @dev_cmd_queue: Queue for issuing device management commands */ struct ufs_hba { void __iomem *mmio_base; @@ -1018,6 +1020,23 @@ struct ufs_hba { unsigned int nr_queues[HCTX_MAX_TYPES]; struct ufshcd_res_info res[RES_MAX]; void __iomem *mcq_base; + struct ufs_hw_queue *uhq; + struct ufs_hw_queue *dev_cmd_queue; +}; + +/** + * @sqe_base_addr: submission queue entry base address + * @sqe_dma_addr: submission queue dma address + * @cqe_base_addr: completion queue base address + * @cqe_dma_addr: completion queue dma address + * @max_entries: max number of slots in this hardware queue + */ +struct ufs_hw_queue { + void *sqe_base_addr; + dma_addr_t sqe_dma_addr; + struct cq_entry *cqe_base_addr; + dma_addr_t cqe_dma_addr; + u32 max_entries; }; /* Returns true if clocks can be gated. Otherwise false */ diff --git a/include/ufs/ufshci.h b/include/ufs/ufshci.h index 67fcebd..1aae5b2 100644 --- a/include/ufs/ufshci.h +++ b/include/ufs/ufshci.h @@ -486,6 +486,28 @@ struct utp_transfer_req_desc { __le16 prd_table_offset; }; +/* MCQ Completion Queue Entry */ +struct cq_entry { + /* DW 0-1 */ + __le64 command_desc_base_addr; + + /* DW 2 */ + __le16 response_upiu_length; + __le16 response_upiu_offset; + + /* DW 3 */ + __le16 prd_table_length; + __le16 prd_table_offset; + + /* DW 4 */ + __le32 status; + + /* DW 5-7 */ + u32 reserved[3]; +}; + +static_assert(sizeof(struct cq_entry) == 32); + /* * UTMRD structure. */