From patchwork Tue Oct 25 09:53:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: guanjun X-Patchwork-Id: 10694 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp913445wru; Tue, 25 Oct 2022 03:05:24 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7vD3I+16XzJvGbFU1uQGYHbmUQnV3+BPYiD+cAWwUthsj8YrksksJNY7WiqvhLPbRDJBe3 X-Received: by 2002:a50:9992:0:b0:461:6cde:ccdd with SMTP id m18-20020a509992000000b004616cdeccddmr16388978edb.402.1666692324202; Tue, 25 Oct 2022 03:05:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666692324; cv=none; d=google.com; s=arc-20160816; b=zl1goeFQWjomn45ohqUe4Hpg0LurO60p3st1GR8dpfRcXSCHGqtpO3U46Tr3Q9qJf/ 4PxusJzECD8jAZyfxJ20d6/TVHmYLiUlqmtgkd3nR9c4txgcaXoHkXn0KIexGT7IZt1U lEjUhBh6A/5FplLbVCrIp9mlG22uK/rDWzh0/B8dyIhart4HwY/Q8C+S5OjOg3vCj0jy ndXAGcj5c0cXimYp6biH3m/xVGX/8mwDvZYIciZctuH2YvIf8iKuy6ixyDFbrO+a7Nh/ vJW5j1pHL1t5XKokhvzm+s00zk8hXhDI8hMTucb1VRHE20ZiRRVugog5OFFTGvTiqM0k pL9g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from; bh=GbcE0PSRx/p+SsmHnLo3I7z8mEKB6OTuPMacGczxp84=; b=f+5l/thUvz/nlVN3a5E+BrkJxLXM5AzdCRJ15rsAKR7aj433l687h6tUyoiqzvCRXp L+hnTycmHbcp69vKj48Ln3RkAXKM8IJTW3Fh2Wyuwhrf98z3TJBPT1TIkJ9MA8HJqgJt 4ifauSnjWWmyVKyM9FvmPBZjt8tYPhW0V41bEvd1Ib0QbErvNewSnW5OLqbKCb05Cioi q71FzxttDrR37pAkA6wzSnog1LKiW5ZYJYbK+GsOu+hZsqvIFBeMIQ/zbEU3BGm2r9q2 uYJQOdE/B68G9i9kvKVRS8pWQc159tTTrMAnBOCjsNCV/jHp63+aM9xq6vRnH9H33P0b jm+A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id b40-20020a509f2b000000b0046154f0f866si2218491edf.159.2022.10.25.03.04.58; Tue, 25 Oct 2022 03:05:24 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231351AbiJYJ7r (ORCPT + 99 others); Tue, 25 Oct 2022 05:59:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41722 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232086AbiJYJ6q (ORCPT ); Tue, 25 Oct 2022 05:58:46 -0400 Received: from out30-42.freemail.mail.aliyun.com (out30-42.freemail.mail.aliyun.com [115.124.30.42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A1851188128; Tue, 25 Oct 2022 02:53:43 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R951e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046051;MF=guanjun@linux.alibaba.com;NM=1;PH=DS;RN=8;SR=0;TI=SMTPD_---0VT2U93L_1666691620; Received: from localhost(mailfrom:guanjun@linux.alibaba.com fp:SMTPD_---0VT2U93L_1666691620) by smtp.aliyun-inc.com; Tue, 25 Oct 2022 17:53:41 +0800 From: 'Guanjun' To: herbert@gondor.apana.org.au, elliott@hpe.com Cc: zelin.deng@linux.alibaba.com, artie.ding@linux.alibaba.com, guanjun@linux.alibaba.com, linux-crypto@vger.kernel.org, linux-kernel@vger.kernel.org, xuchun.shang@linux.alibaba.com Subject: [PATCH v3 4/9] crypto/ycc: Add device error handling support for ycc hw errors Date: Tue, 25 Oct 2022 17:53:31 +0800 Message-Id: <1666691616-69983-5-git-send-email-guanjun@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1666691616-69983-1-git-send-email-guanjun@linux.alibaba.com> References: <1666691616-69983-1-git-send-email-guanjun@linux.alibaba.com> X-Spam-Status: No, score=-9.9 required=5.0 tests=BAYES_00, ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2, SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747653570345073244?= X-GMAIL-MSGID: =?utf-8?q?1747653570345073244?= From: Zelin Deng Due to ycc hardware limitations, in REE ycc device cannot be reset to recover from fatal error (reset register is only valid in TEE and PCIE FLR only reset queue pointers but not ycc hw), regard all hw errors except queue error as fatal error. Signed-off-by: Zelin Deng --- drivers/crypto/ycc/ycc_isr.c | 93 +++++++++++++++++++++++++++++++++++++++++-- drivers/crypto/ycc/ycc_ring.c | 90 +++++++++++++++++++++++++++++++++++++++++ drivers/crypto/ycc/ycc_ring.h | 3 ++ 3 files changed, 183 insertions(+), 3 deletions(-) diff --git a/drivers/crypto/ycc/ycc_isr.c b/drivers/crypto/ycc/ycc_isr.c index a86c8d7..28f8918 100644 --- a/drivers/crypto/ycc/ycc_isr.c +++ b/drivers/crypto/ycc/ycc_isr.c @@ -15,6 +15,8 @@ #include "ycc_dev.h" #include "ycc_ring.h" +extern void ycc_clear_cmd_ring(struct ycc_ring *ring); +extern void ycc_clear_resp_ring(struct ycc_ring *ring); static irqreturn_t ycc_resp_isr(int irq, void *data) { @@ -24,11 +26,93 @@ static irqreturn_t ycc_resp_isr(int irq, void *data) return IRQ_HANDLED; } -/* - * TODO: will implement when ycc ring actually work. - */ +static void ycc_fatal_error(struct ycc_dev *ydev) +{ + struct ycc_ring *ring; + int i; + + for (i = 0; i < YCC_RINGPAIR_NUM; i++) { + ring = ydev->rings + i; + + if (ring->type != KERN_RING) + continue; + + spin_lock_bh(&ring->lock); + ycc_clear_cmd_ring(ring); + spin_unlock_bh(&ring->lock); + + ycc_clear_resp_ring(ring); + } +} + static void ycc_process_global_err(struct work_struct *work) { + struct ycc_dev *ydev = container_of(work, struct ycc_dev, work); + struct ycc_bar *cfg_bar = &ydev->ycc_bars[YCC_SEC_CFG_BAR]; + struct ycc_ring *ring; + u32 hclk_err, xclk_err; + u32 xclk_ecc_uncor_err_0, xclk_ecc_uncor_err_1; + u32 hclk_ecc_uncor_err; + int i; + + if (pci_wait_for_pending_transaction(ydev->pdev)) + pr_warn("Failed to pending transaction\n"); + + hclk_err = YCC_CSR_RD(cfg_bar->vaddr, REG_YCC_HCLK_INT_STATUS); + xclk_err = YCC_CSR_RD(cfg_bar->vaddr, REG_YCC_XCLK_INT_STATUS); + xclk_ecc_uncor_err_0 = YCC_CSR_RD(cfg_bar->vaddr, REG_YCC_XCLK_MEM_ECC_UNCOR_0); + xclk_ecc_uncor_err_1 = YCC_CSR_RD(cfg_bar->vaddr, REG_YCC_XCLK_MEM_ECC_UNCOR_1); + hclk_ecc_uncor_err = YCC_CSR_RD(cfg_bar->vaddr, REG_YCC_HCLK_MEM_ECC_UNCOR); + + if ((hclk_err & ~(YCC_HCLK_TRNG_ERR)) || xclk_err || hclk_ecc_uncor_err) { + pr_err("Got uncorrected error, must be reset\n"); + /* + * Fatal error, as ycc cannot be reset in REE, clear ring data. + */ + return ycc_fatal_error(ydev); + } + + if (xclk_ecc_uncor_err_0 || xclk_ecc_uncor_err_1) { + pr_err("Got algorithm ECC error: %x, %x\n", + xclk_ecc_uncor_err_0, xclk_ecc_uncor_err_1); + return ycc_fatal_error(ydev); + } + + /* This has to be queue error. Handling command rings. */ + for (i = 0; i < YCC_RINGPAIR_NUM; i++) { + ring = ydev->rings + i; + + if (ring->type != KERN_RING) + continue; + + ring->status = YCC_CSR_RD(ring->csr_vaddr, REG_RING_STATUS); + if (ring->status) { + pr_err("YCC: Dev: %d, Ring: %d got ring err: %x\n", + ydev->id, ring->ring_id, ring->status); + spin_lock_bh(&ring->lock); + ycc_clear_cmd_ring(ring); + spin_unlock_bh(&ring->lock); + } + } + + /* + * Give HW a chance to process all pending_cmds + * through recovering transactions. + */ + pci_set_master(ydev->pdev); + + for (i = 0; i < YCC_RINGPAIR_NUM; i++) { + ring = ydev->rings + i; + + if (ring->type != KERN_RING || !ring->status) + continue; + + ycc_clear_resp_ring(ring); + } + + ycc_g_err_unmask(cfg_bar->vaddr); + clear_bit(YDEV_STATUS_ERR, &ydev->status); + set_bit(YDEV_STATUS_READY, &ydev->status); } static irqreturn_t ycc_g_err_isr(int irq, void *data) @@ -45,6 +129,9 @@ static irqreturn_t ycc_g_err_isr(int irq, void *data) clear_bit(YDEV_STATUS_READY, &ydev->status); + /* Disable YCC mastering, no new transactions */ + pci_clear_master(ydev->pdev); + schedule_work(&ydev->work); return IRQ_HANDLED; } diff --git a/drivers/crypto/ycc/ycc_ring.c b/drivers/crypto/ycc/ycc_ring.c index d808054..f6f6e40 100644 --- a/drivers/crypto/ycc/ycc_ring.c +++ b/drivers/crypto/ycc/ycc_ring.c @@ -483,6 +483,24 @@ int ycc_enqueue(struct ycc_ring *ring, void *cmd) return ret; } +static void ycc_cancel_cmd(struct ycc_ring *ring, struct ycc_cmd_desc *desc) +{ + struct ycc_flags *aflag; + + dma_rmb(); + + aflag = (struct ycc_flags *)desc->private_ptr; + if (!aflag || (u64)aflag == CMD_INVALID_CONTENT_U64) { + pr_debug("YCC: Invalid aflag\n"); + return; + } + + aflag->ycc_done_callback(aflag->ptr, CMD_CANCELLED); + + memset(desc, CMD_INVALID_CONTENT_U8, sizeof(*desc)); + kfree(aflag); +} + static inline void ycc_check_cmd_state(u16 state) { switch (state) { @@ -560,3 +578,75 @@ void ycc_dequeue(struct ycc_ring *ring) if (cnt) YCC_CSR_WR(ring->csr_vaddr, REG_RING_RSP_RD_PTR, ring->resp_rd_ptr); } + +/* + * Clear incompletion cmds in command queue while rollback cmd_wr_ptr. + * + * Note: Make sure been invoked when error occurs in YCC internal and + * YCC status is not ready. + */ +void ycc_clear_cmd_ring(struct ycc_ring *ring) +{ + struct ycc_cmd_desc *desc = NULL; + + ring->cmd_rd_ptr = YCC_CSR_RD(ring->csr_vaddr, REG_RING_CMD_RD_PTR); + ring->cmd_wr_ptr = YCC_CSR_RD(ring->csr_vaddr, REG_RING_CMD_WR_PTR); + + while (ring->cmd_rd_ptr != ring->cmd_wr_ptr) { + desc = (struct ycc_cmd_desc *)ring->cmd_base_vaddr + + ring->cmd_rd_ptr; + ycc_cancel_cmd(ring, desc); + + if (--ring->cmd_wr_ptr == 0) + ring->cmd_wr_ptr = ring->max_desc; + } + + YCC_CSR_WR(ring->csr_vaddr, REG_RING_CMD_WR_PTR, ring->cmd_wr_ptr); +} + +/* + * Clear response queue + * + * Note: Make sure been invoked when error occurs in YCC internal and + * YCC status is not ready. + */ +void ycc_clear_resp_ring(struct ycc_ring *ring) +{ + struct ycc_resp_desc *resp; + int retry; + u32 pending_cmd; + + /* + * Check if the ring has been stopped. *stop* means no + * new transactions, No need to wait for pending_cmds + * been processed under this condition. + */ + retry = ycc_ring_stopped(ring) ? 0 : MAX_ERROR_RETRY; + pending_cmd = YCC_CSR_RD(ring->csr_vaddr, REG_RING_PENDING_CMD); + + ring->resp_wr_ptr = YCC_CSR_RD(ring->csr_vaddr, REG_RING_RSP_WR_PTR); + while (!ycc_ring_empty(ring) || (retry && pending_cmd)) { + if (!ycc_ring_empty(ring)) { + resp = (struct ycc_resp_desc *)ring->resp_base_vaddr + + ring->resp_rd_ptr; + resp->state = CMD_CANCELLED; + ycc_handle_resp(ring, resp); + + if (++ring->resp_rd_ptr == ring->max_desc) + ring->resp_rd_ptr = 0; + + YCC_CSR_WR(ring->csr_vaddr, REG_RING_RSP_RD_PTR, ring->resp_rd_ptr); + } else { + udelay(MAX_SLEEP_US_PER_CHECK); + retry--; + } + + pending_cmd = YCC_CSR_RD(ring->csr_vaddr, REG_RING_PENDING_CMD); + ring->resp_wr_ptr = YCC_CSR_RD(ring->csr_vaddr, REG_RING_RSP_WR_PTR); + } + + if (!retry && pending_cmd) + ring->type = INVAL_RING; + + ring->status = 0; +} diff --git a/drivers/crypto/ycc/ycc_ring.h b/drivers/crypto/ycc/ycc_ring.h index eb3e6f9..c3e7cbf 100644 --- a/drivers/crypto/ycc/ycc_ring.h +++ b/drivers/crypto/ycc/ycc_ring.h @@ -20,6 +20,9 @@ #define CMD_INVALID_CONTENT_U8 0x7f #define CMD_INVALID_CONTENT_U64 0x7f7f7f7f7f7f7f7fULL +#define MAX_SLEEP_US_PER_CHECK 100 /* every 100us to check register */ +#define MAX_ERROR_RETRY 10000 /* 1s in total */ + enum ring_type { FREE_RING, USER_RING,