From patchwork Wed Feb 7 18:40:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Jeffery X-Patchwork-Id: 198036 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:168b:b0:106:860b:bbdd with SMTP id ma11csp2432645dyb; Wed, 7 Feb 2024 10:43:49 -0800 (PST) X-Google-Smtp-Source: AGHT+IGcQKyhXcPbE5HsZBoQyRC9bBn1OB4ol62yZiMj2j5CT9tkSZiLIDbsI1+Lx9yCMng/DGKk X-Received: by 2002:ac8:5bd4:0:b0:42c:41bd:329c with SMTP id b20-20020ac85bd4000000b0042c41bd329cmr1816140qtb.0.1707331429551; Wed, 07 Feb 2024 10:43:49 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707331429; cv=pass; d=google.com; s=arc-20160816; b=v5F3tgzVJaAIXTZs3+ZB0cJKXp/ixjx2C1Mm+kkXVUyJATzQfEnnf3UwnOVPMHfj/l SQiiXl8W2na8rE3et7G/vYqnr2Ka0ovqrkID5Fm9BTGP4sSbp3V5SHKFs0d1y5Tl9W9f EVELgvqdQcQ3TLcxB0I+G610qtBelPNj+AzfyAXNpEISxwdlfCwjvK173aaNxLLT521a tOSp7t6y4J1oztBv70YOv93cnkpFeeCnXGi0HJS5IqbCHS2CpMPAnHuiV0Ts11wGWWun GuZqWMLO4+ZNCbrSBIE982GElL7kb1NGs6Vt6UT5irHhmvLsXbopYBx2m8aPBYmHFPcc v0NQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=cuv9rvzEed+0pQINLR4eZlmzvTUXMYlIRXYRIOt5NJQ=; fh=zmbN32ajG2gAW+W7IBul3Agq15dVgmSU7uvcp3FhAkA=; b=Fb7H4wJbjnVvp88CRr5K47BTUtojP4iFgMamcYbGxGUcZOphRdMS7a9RR3ffzi4SqM VjM3iCcfP5aMljOXNBm38TWr2I3mth879WNgutSuluLUDB2IAVwxczP3qvNl7OXXe26O 5JCPVhn8PAMHK9jTAftxlGgbFk6MNyVVtAFlIRqYZxRNQa+zea/RKrJ6s8K+0cclQNid 1bLPnEg3SsFJSQSTxxlwayvZhB0oHuPlz/3EGlfxOyoCQ968MNxsuzdPNsdhZe4sYUNt 01vOtzfHtk2JOdndKaLAVaksj89GXXMosRHlcdG1udmz6BIpBGInJInyhOeArY6Vwe1q mwgw==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=hQdUkb2A; arc=pass (i=1 spf=pass spfdomain=redhat.com dkim=pass dkdomain=redhat.com dmarc=pass fromdomain=redhat.com); spf=pass (google.com: domain of linux-kernel+bounces-56920-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-56920-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Forwarded-Encrypted: i=2; AJvYcCWVje1VK5KbrimT2sk+LYbvpcYeJ/02ho4RXuI/36cAmIDza5/3FrV1r9sxchn9diSaMRsgdo/uyXYBYX2jowB8P35BLQ== Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id g12-20020ac85d4c000000b0042c371969d2si1716667qtx.512.2024.02.07.10.43.49 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Feb 2024 10:43:49 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-56920-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=hQdUkb2A; arc=pass (i=1 spf=pass spfdomain=redhat.com dkim=pass dkdomain=redhat.com dmarc=pass fromdomain=redhat.com); spf=pass (google.com: domain of linux-kernel+bounces-56920-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-56920-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 4B75F1C2517C for ; Wed, 7 Feb 2024 18:43:49 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 4043C86157; Wed, 7 Feb 2024 18:42:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="hQdUkb2A" Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D080F8529D for ; Wed, 7 Feb 2024 18:42:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707331349; cv=none; b=cd4Ofj2aqgfSOlGqS3W2fRNEiAIa3gqJCnxT/pneZnlYb/1EUKVtdp/35EpVTpKEkiR8pfDNgWVxLg+O14pY7irl7Xav2Ub3Wzn0GdXqcQEp1A/12ZLtD1/MmqiJZgiHos3ddqwmLQnPgwF1iMtUNKV/a3gRZqS5Mwam8CJZQfw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707331349; c=relaxed/simple; bh=UeygM9npBEUq1MKQwn5tUh8m7ES9H0uWtUU1AqB2VWY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=c5OmLJjlz2QV79JOUa7V9l/3Km9ENHj+ASSc1qnZqqQBXekuFvdB+6CuJ4wtN3FBIxvO8yfqTLGtwhzOx3hBH45189UjfGiuY/0EqYlSXd8J0xLtvuI6Xoh1P7qghNS5mBjhtD4kCgFOHUrpi86Ex23sSmN25GiHv6B17yO5wUk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=hQdUkb2A; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1707331346; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cuv9rvzEed+0pQINLR4eZlmzvTUXMYlIRXYRIOt5NJQ=; b=hQdUkb2A4TuKqF5vVQFX2zuhjv55gntocFpadqGy3TIiMSHaalZkGxi8LQ918t2HkJpGNS xSQJGFIvP0krvy3h7ycQC9lRC/xSGmgmNzo1i+CtvUyffrU7B5rfVXIHTOA9k2Z4HgFF3u sGlpOV5VX5B4HmR1QJr8WzDFoCw9TQc= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-664-s5caMUuGOx6vwEyw6XmQ_A-1; Wed, 07 Feb 2024 13:42:25 -0500 X-MC-Unique: s5caMUuGOx6vwEyw6XmQ_A-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 1A62D29AC012; Wed, 7 Feb 2024 18:42:25 +0000 (UTC) Received: from fedora-work.redhat.com (unknown [10.22.32.236]) by smtp.corp.redhat.com (Postfix) with ESMTP id A15F140C9444; Wed, 7 Feb 2024 18:42:24 +0000 (UTC) From: David Jeffery To: linux-kernel@vger.kernel.org Cc: linux-nvme@lists.infradead.org, linux-scsi@vger.kernel.org, Greg Kroah-Hartman , "Rafael J . Wysocki" , David Jeffery , Laurence Oberman Subject: [RFC PATCH 4/6] pci nvme async shutdown support Date: Wed, 7 Feb 2024 13:40:58 -0500 Message-ID: <20240207184100.18066-5-djeffery@redhat.com> In-Reply-To: <20240207184100.18066-1-djeffery@redhat.com> References: <20240207184100.18066-1-djeffery@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.2 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790266761166543461 X-GMAIL-MSGID: 1790266761166543461 Alter nvme so that the pci nvme driver may shut down the hardware asynchronously, providing a large benefit to shutdown time with large numbers of nvme devices in some configurations. The functionality of nvme_disable_ctrl is split in separate functions for starting disabling the hardware and for waiting for the hardware to report success. A wrapper is created which provides the same interface for the other nvme types so they remain unchanged. And new field is added to nvme-pci's nvme_dev to track the nvme shutdown state so that the nvme async_shutdown_end call knows what if anything it still needs to do. Signed-off-by: David Jeffery Tested-by: Laurence Oberman --- drivers/nvme/host/core.c | 26 +++++++++++++++----- drivers/nvme/host/nvme.h | 2 ++ drivers/nvme/host/pci.c | 53 ++++++++++++++++++++++++++++++++++++++-- 3 files changed, 73 insertions(+), 8 deletions(-) diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index 85ab0fcf9e88..b24985a843a8 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -2242,20 +2242,21 @@ static int nvme_wait_ready(struct nvme_ctrl *ctrl, u32 mask, u32 val, return ret; } -int nvme_disable_ctrl(struct nvme_ctrl *ctrl, bool shutdown) +int nvme_disable_ctrl_send(struct nvme_ctrl *ctrl, bool shutdown) { - int ret; - ctrl->ctrl_config &= ~NVME_CC_SHN_MASK; if (shutdown) ctrl->ctrl_config |= NVME_CC_SHN_NORMAL; else ctrl->ctrl_config &= ~NVME_CC_ENABLE; - ret = ctrl->ops->reg_write32(ctrl, NVME_REG_CC, ctrl->ctrl_config); - if (ret) - return ret; + return ctrl->ops->reg_write32(ctrl, NVME_REG_CC, ctrl->ctrl_config); +} +EXPORT_SYMBOL_GPL(nvme_disable_ctrl_send); + +int nvme_disable_ctrl_wait(struct nvme_ctrl *ctrl, bool shutdown) +{ if (shutdown) { return nvme_wait_ready(ctrl, NVME_CSTS_SHST_MASK, NVME_CSTS_SHST_CMPLT, @@ -2266,6 +2267,19 @@ int nvme_disable_ctrl(struct nvme_ctrl *ctrl, bool shutdown) return nvme_wait_ready(ctrl, NVME_CSTS_RDY, 0, (NVME_CAP_TIMEOUT(ctrl->cap) + 1) / 2, "reset"); } +EXPORT_SYMBOL_GPL(nvme_disable_ctrl_wait); + + +int nvme_disable_ctrl(struct nvme_ctrl *ctrl, bool shutdown) +{ + int ret; + + ret = nvme_disable_ctrl_send(ctrl, shutdown); + if (!ret) + ret = nvme_disable_ctrl_wait(ctrl, shutdown); + + return ret; +} EXPORT_SYMBOL_GPL(nvme_disable_ctrl); int nvme_enable_ctrl(struct nvme_ctrl *ctrl) diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h index 030c80818240..5bdd862328d4 100644 --- a/drivers/nvme/host/nvme.h +++ b/drivers/nvme/host/nvme.h @@ -761,6 +761,8 @@ void nvme_cancel_admin_tagset(struct nvme_ctrl *ctrl); bool nvme_change_ctrl_state(struct nvme_ctrl *ctrl, enum nvme_ctrl_state new_state); int nvme_disable_ctrl(struct nvme_ctrl *ctrl, bool shutdown); +int nvme_disable_ctrl_send(struct nvme_ctrl *ctrl, bool shutdown); +int nvme_disable_ctrl_wait(struct nvme_ctrl *ctrl, bool shutdown); int nvme_enable_ctrl(struct nvme_ctrl *ctrl); int nvme_init_ctrl(struct nvme_ctrl *ctrl, struct device *dev, const struct nvme_ctrl_ops *ops, unsigned long quirks); diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index c1d6357ec98a..c2a7b3d28a56 100644 --- a/drivers/nvme/host/pci.c +++ b/drivers/nvme/host/pci.c @@ -112,6 +112,12 @@ static void nvme_dev_disable(struct nvme_dev *dev, bool shutdown); static void nvme_delete_io_queues(struct nvme_dev *dev); static void nvme_update_attrs(struct nvme_dev *dev); +enum nvme_disable_state { + NVME_DISABLE_START = 0, + NVME_DISABLE_WAIT = 1, + NVME_DISABLE_DONE = 2, +}; + /* * Represents an NVM Express device. Each nvme_dev is a PCI function. */ @@ -159,6 +165,7 @@ struct nvme_dev { unsigned int nr_allocated_queues; unsigned int nr_write_queues; unsigned int nr_poll_queues; + enum nvme_disable_state disable_state; }; static int io_queue_depth_set(const char *val, const struct kernel_param *kp) @@ -2574,12 +2581,14 @@ static bool nvme_pci_ctrl_is_dead(struct nvme_dev *dev) return (csts & NVME_CSTS_CFS) || !(csts & NVME_CSTS_RDY); } -static void nvme_dev_disable(struct nvme_dev *dev, bool shutdown) +static void nvme_dev_disable_start(struct nvme_dev *dev, bool shutdown) { enum nvme_ctrl_state state = nvme_ctrl_state(&dev->ctrl); struct pci_dev *pdev = to_pci_dev(dev->dev); bool dead; + dev->disable_state = NVME_DISABLE_START; + mutex_lock(&dev->shutdown_lock); dead = nvme_pci_ctrl_is_dead(dev); if (state == NVME_CTRL_LIVE || state == NVME_CTRL_RESETTING) { @@ -2597,7 +2606,20 @@ static void nvme_dev_disable(struct nvme_dev *dev, bool shutdown) if (!dead && dev->ctrl.queue_count > 0) { nvme_delete_io_queues(dev); - nvme_disable_ctrl(&dev->ctrl, shutdown); + nvme_disable_ctrl_send(&dev->ctrl, shutdown); + dev->disable_state = NVME_DISABLE_WAIT; + } +} + +static void nvme_dev_disable_end(struct nvme_dev *dev, bool shutdown) +{ + struct pci_dev *pdev = to_pci_dev(dev->dev); + + if (dev->disable_state == NVME_DISABLE_DONE) + return; + + if (dev->disable_state == NVME_DISABLE_WAIT) { + nvme_disable_ctrl_wait(&dev->ctrl, shutdown); nvme_poll_irqdisable(&dev->queues[0]); } nvme_suspend_io_queues(dev); @@ -2623,6 +2645,12 @@ static void nvme_dev_disable(struct nvme_dev *dev, bool shutdown) mutex_unlock(&dev->shutdown_lock); } +static void nvme_dev_disable(struct nvme_dev *dev, bool shutdown) +{ + nvme_dev_disable_start(dev, shutdown); + nvme_dev_disable_end(dev, shutdown); +} + static int nvme_disable_prepare_reset(struct nvme_dev *dev, bool shutdown) { if (!nvme_wait_reset(&dev->ctrl)) @@ -3120,6 +3148,25 @@ static void nvme_shutdown(struct pci_dev *pdev) nvme_disable_prepare_reset(dev, true); } +static void nvme_shutdown_start(struct pci_dev *pdev) +{ + struct nvme_dev *dev = pci_get_drvdata(pdev); + + if (!nvme_wait_reset(&dev->ctrl)) { + dev->disable_state = NVME_DISABLE_DONE; + return; + } + nvme_dev_disable_start(dev, true); +} + +static void nvme_shutdown_end(struct pci_dev *pdev) +{ + struct nvme_dev *dev = pci_get_drvdata(pdev); + + nvme_dev_disable_end(dev, true); +} + + /* * The driver's remove may be called on a device in a partially initialized * state. This function must not have any dependencies on the device state in @@ -3511,6 +3558,8 @@ static struct pci_driver nvme_driver = { .probe = nvme_probe, .remove = nvme_remove, .shutdown = nvme_shutdown, + .async_shutdown_start = nvme_shutdown_start, + .async_shutdown_end = nvme_shutdown_end, .driver = { .probe_type = PROBE_PREFER_ASYNCHRONOUS, #ifdef CONFIG_PM_SLEEP