Message ID | 20240124093941.2259199-1-hch@lst.de |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel+bounces-36730-ouuuleilei=gmail.com@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2553:b0:103:945f:af90 with SMTP id p19csp905726dyi; Wed, 24 Jan 2024 02:56:08 -0800 (PST) X-Google-Smtp-Source: AGHT+IEogmBHR9aEBJWetaE72QecIG9GDnk6RfC7aqQZAF/24Jh0gEwTvQezIGN0GmnCajMXuXv2 X-Received: by 2002:aa7:cd63:0:b0:55c:d502:c1bd with SMTP id ca3-20020aa7cd63000000b0055cd502c1bdmr171884edb.42.1706093767876; Wed, 24 Jan 2024 02:56:07 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706093767; cv=pass; d=google.com; s=arc-20160816; b=ewYo+/daXbLMoy6aSP1+GhfjZEK4pvp6DxOgIrE/XBkfcBm3S3jF1xphWlAE3eY/dP MEtG3gTAN0u81IExhGvHxBCf4HrAmNxiXR5SMXPPdZ7KXK4QawzC2ReDp5Y/tJudcjEv gveFFQNx0fHDtVZ8Q3XubP1aSnJ7w2OfrTHqBHBbcMGQslN6wnGh9fZfxIcaW3lxgVx9 fr2EYnojz9tnUpKYMxtHF8k+YEbWvR44MG+JI9CPuS+fU7OEBeze1OPuC7sjTi4qzwBm pprDf+LVQ0EXYM9BNHQ7BWwUvN53Ig/WWCDwOzDKsKrgUtE2T5hbblrNLqDNRUgxKLcy Hy5Q== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:message-id:date:subject:cc:to :from:dkim-signature; bh=iJqhfHE63yLibvofZGhJTgUGPFr97HkcGRUKyFZYBvw=; fh=7vEk1EnvGJbXctFKlS9RN748KTrnmCkbPLm1zdn8g80=; b=RInJyhMMMJ57F4S57YHvyWcJTmyOzOvd+v2/z+8rhwZCVYNkxHLCEIR988kiKQvYOE eTFUjQ5M7gU15FJDQXUzJ0MeBWzeCcV9/sJFiThJeXBzmSmxcBm3wvCykBXiJGZ+0dJ8 GesqxccTu6TZQF5v8Cwq1yRAF9ClBHDnDuYJ2xljg2rqGBRidusG7cCse7XFY4TwCUfM qctep9bVe2uMneOZZDF1+Ex0pyMVCSMeQEJ9o5sa0NcKqcHD7lrw+66jC6FC1ARQgU0j 4xRXX2tA6OhbFjTqmcaII4M9QSveDAtVXnCYCPg5VUfkPPEXZT/VoG4ahIQ3kJbSz9v3 m21g== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@infradead.org header.s=bombadil.20210309 header.b=J6FchOiY; arc=pass (i=1 dkim=pass dkdomain=infradead.org); spf=pass (google.com: domain of linux-kernel+bounces-36730-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-36730-ouuuleilei=gmail.com@vger.kernel.org" Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id x9-20020a056402414900b0055c15c5c7casi3955304eda.179.2024.01.24.02.56.07 for <ouuuleilei@gmail.com> (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jan 2024 02:56:07 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-36730-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@infradead.org header.s=bombadil.20210309 header.b=J6FchOiY; arc=pass (i=1 dkim=pass dkdomain=infradead.org); spf=pass (google.com: domain of linux-kernel+bounces-36730-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-36730-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id DFA7C1F299E7 for <ouuuleilei@gmail.com>; Wed, 24 Jan 2024 09:40:08 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id EC29A18059; Wed, 24 Jan 2024 09:39:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="J6FchOiY" Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B378818021; Wed, 24 Jan 2024 09:39:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.133 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706089189; cv=none; b=DpXzgEg20Xaq23W9IU3tt+URP8EUqCpNUUoHpBX3LZshAvRFKpR/QJyN+j2pYh+fTwML/zQiSMS88Rsn8x0Z76H+9cR7f6fzc2CVyTE8WMYHHCJp0fMjZqwTAI5dW63UBshDdhNVbh59r1EC5MDZj7Q0Hw4gOdjT9ekfpW4aCu8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706089189; c=relaxed/simple; bh=csxSFjKQHyJjxXdzXVfeqcnrqnGQMSacuTyh3j8G7gk=; h=From:To:Cc:Subject:Date:Message-Id:MIME-Version; b=fi7Zlx/Y301vX1Xa9HEwrc+0FaG+j6vquwa+0hj+X7wDhvtO3mJAjvhVLMDplAqKPcF0AWvb69qMYYbdpMknd/VaPGdSfVqwBwFJjTzoqqiOhuKkFe2fW48A9uO8YZFE1pnOKetRK8kaAPjbln8DtXFeyKNFJhfFhfJRkmkcJn8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de; spf=none smtp.mailfrom=bombadil.srs.infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=J6FchOiY; arc=none smtp.client-ip=198.137.202.133 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=bombadil.srs.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type: Content-ID:Content-Description:In-Reply-To:References; bh=iJqhfHE63yLibvofZGhJTgUGPFr97HkcGRUKyFZYBvw=; b=J6FchOiYBzu7hZPX+AZnNaaF3u TIfzf4T6K6vwvP1XXMXrAaxhkY48buvnSxR8ax0XKp7xcmlOwmEZbbJDMs4pTGBASt9lXaN/NOedj 6ih9qjAbrL6tgO6mbWBsujjvZUfhX/g6pWcBswfXyhfsIEpr55MpizgfVaeOdVRWBam4ibuhs5KlM GtWlYulU0k2QwY6P0GNmspXsTZeSJdSkXg+dZ6R1G46ED1oLkbOvhRNYJ+e097XxndJtIw4Hl2LDr +pFzsI3q3szln9220i8Zy22fNqE2v6MCDKXCZk0Ogl6prYxJsPslH9QKhv0+nEz5RxATI7VDArcm7 cGGngGYA==; Received: from [2001:4bb8:188:3f09:9c13:25f:1e5b:57f9] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1rSZjW-002I8t-0r; Wed, 24 Jan 2024 09:39:47 +0000 From: Christoph Hellwig <hch@lst.de> To: axboe@kernel.dk Cc: linux-block@vger.kernel.org, tj@kernel.org, jiangshanlai@gmail.com, linux-kernel@vger.kernel.org Subject: [PATCH, RFC] block: set noio context in submit_bio_noacct_nocheck Date: Wed, 24 Jan 2024 10:39:41 +0100 Message-Id: <20240124093941.2259199-1-hch@lst.de> X-Mailer: git-send-email 2.39.2 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: <linux-kernel.vger.kernel.org> List-Subscribe: <mailto:linux-kernel+subscribe@vger.kernel.org> List-Unsubscribe: <mailto:linux-kernel+unsubscribe@vger.kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by bombadil.infradead.org. See http://www.infradead.org/rpr.html X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1788968978928210332 X-GMAIL-MSGID: 1788968978928210332 |
Series |
[RFC] block: set noio context in submit_bio_noacct_nocheck
|
|
Commit Message
Christoph Hellwig
Jan. 24, 2024, 9:39 a.m. UTC
Make sure all in-line block layer submission runs in noio reclaim
context. This is a big step towards allowing GFP_NOIO, the other
one would be to have noio (and nofs for that matter) workqueues for
kblockd and driver internal workqueues.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
block/blk-core.c | 4 ++++
1 file changed, 4 insertions(+)
Comments
On 1/24/24 2:39 AM, Christoph Hellwig wrote: > Make sure all in-line block layer submission runs in noio reclaim > context. This is a big step towards allowing GFP_NOIO, the other > one would be to have noio (and nofs for that matter) workqueues for > kblockd and driver internal workqueues. I really don't like adding this for no good reason. Who's doing non NOIO allocations down from this path?
On Wed, Jan 24, 2024 at 08:40:28AM -0700, Jens Axboe wrote: > On 1/24/24 2:39 AM, Christoph Hellwig wrote: > > Make sure all in-line block layer submission runs in noio reclaim > > context. This is a big step towards allowing GFP_NOIO, the other > > one would be to have noio (and nofs for that matter) workqueues for > > kblockd and driver internal workqueues. > > I really don't like adding this for no good reason. Who's doing non NOIO > allocations down from this path? If there is a non-NOIO allocation right now that would be a bug, although I would not be surprised if we had a few of them. The reason to add this is a different one: The MM folks want to get rid of GFP_NOIO and GFP_NOFS and replace them by these context. And doing this in the submission path and kblockd will cover almost all of the noio context, with the rest probably covered by other workqueues. And this feels a lot less error prone than requiring every driver to annotate the context in their submission routines.
On 1/25/24 1:10 AM, Christoph Hellwig wrote: > On Wed, Jan 24, 2024 at 08:40:28AM -0700, Jens Axboe wrote: >> On 1/24/24 2:39 AM, Christoph Hellwig wrote: >>> Make sure all in-line block layer submission runs in noio reclaim >>> context. This is a big step towards allowing GFP_NOIO, the other >>> one would be to have noio (and nofs for that matter) workqueues for >>> kblockd and driver internal workqueues. >> >> I really don't like adding this for no good reason. Who's doing non NOIO >> allocations down from this path? > > If there is a non-NOIO allocation right now that would be a bug, > although I would not be surprised if we had a few of them. > > The reason to add this is a different one: The MM folks want to > get rid of GFP_NOIO and GFP_NOFS and replace them by these context. > > And doing this in the submission path and kblockd will cover almost > all of the noio context, with the rest probably covered by other > workqueues. And this feels a lot less error prone than requiring > every driver to annotate the context in their submission routines. I think it'd be much better to add a DEBUG protected aid that checks for violating allocations. Nothing that isn't buggy should trigger this, right now, and then we could catch problems if there are any. If we do the save/restore there and call it good, then we're going to be stuck with that forever. Regardless of whether it's actually needed or not.
On Thu, Jan 25, 2024 at 09:09:44AM -0700, Jens Axboe wrote: > On 1/25/24 1:10 AM, Christoph Hellwig wrote: > > On Wed, Jan 24, 2024 at 08:40:28AM -0700, Jens Axboe wrote: > >> On 1/24/24 2:39 AM, Christoph Hellwig wrote: > >>> Make sure all in-line block layer submission runs in noio reclaim > >>> context. This is a big step towards allowing GFP_NOIO, the other > >>> one would be to have noio (and nofs for that matter) workqueues for > >>> kblockd and driver internal workqueues. > >> > >> I really don't like adding this for no good reason. Who's doing non NOIO > >> allocations down from this path? > > > > If there is a non-NOIO allocation right now that would be a bug, > > although I would not be surprised if we had a few of them. > > > > The reason to add this is a different one: The MM folks want to > > get rid of GFP_NOIO and GFP_NOFS and replace them by these context. > > > > And doing this in the submission path and kblockd will cover almost > > all of the noio context, with the rest probably covered by other > > workqueues. And this feels a lot less error prone than requiring > > every driver to annotate the context in their submission routines. > > I think it'd be much better to add a DEBUG protected aid that checks for > violating allocations. Nothing that isn't buggy should trigger this, > right now, and then we could catch problems if there are any. If we do > the save/restore there and call it good, then we're going to be stuck > with that forever. Regardless of whether it's actually needed or not. Nono, you don't understand. The plan is to remove GFP_NOIO entirely. Allocations should be done with GFP_KERNEL while under a memalloc_noio_save().
On Thu, Jan 25, 2024 at 9:11?AM Matthew Wilcox <willy@infradead.org> wrote: > > On Thu, Jan 25, 2024 at 09:09:44AM -0700, Jens Axboe wrote: > > On 1/25/24 1:10 AM, Christoph Hellwig wrote: > > > On Wed, Jan 24, 2024 at 08:40:28AM -0700, Jens Axboe wrote: > > >> On 1/24/24 2:39 AM, Christoph Hellwig wrote: > > >>> Make sure all in-line block layer submission runs in noio reclaim > > >>> context. This is a big step towards allowing GFP_NOIO, the other > > >>> one would be to have noio (and nofs for that matter) workqueues for > > >>> kblockd and driver internal workqueues. > > >> > > >> I really don't like adding this for no good reason. Who's doing non NOIO > > >> allocations down from this path? > > > > > > If there is a non-NOIO allocation right now that would be a bug, > > > although I would not be surprised if we had a few of them. > > > > > > The reason to add this is a different one: The MM folks want to > > > get rid of GFP_NOIO and GFP_NOFS and replace them by these context. > > > > > > And doing this in the submission path and kblockd will cover almost > > > all of the noio context, with the rest probably covered by other > > > workqueues. And this feels a lot less error prone than requiring > > > every driver to annotate the context in their submission routines. > > > > I think it'd be much better to add a DEBUG protected aid that checks for > > violating allocations. Nothing that isn't buggy should trigger this, > > right now, and then we could catch problems if there are any. If we do > > the save/restore there and call it good, then we're going to be stuck > > with that forever. Regardless of whether it's actually needed or not. > > Nono, you don't understand. The plan is to remove GFP_NOIO > entirely. Allocations should be done with GFP_KERNEL while under a > memalloc_noio_save(). I do understand, but thanks for the vote of confidence. Place the save/restore higher up, most likely actual IO submission isn't going to be the only (or even major) allocation potentially needed for the IO.
On Thu, Jan 25, 2024 at 09:13:37AM -0700, Jens Axboe wrote: > > Nono, you don't understand. The plan is to remove GFP_NOIO > > entirely. Allocations should be done with GFP_KERNEL while under a > > memalloc_noio_save(). > > I do understand, but thanks for the vote of confidence. Place the > save/restore higher up, most likely actual IO submission isn't going to > be the only (or even major) allocation potentially needed for the IO. NOIO is defined as allocations that will not recurse into the I/O stack. So for anything block based, entering the block layer is literally the defined boundary where it should be used below. So no, wrapping every submit_bio into a context annotation doesn't make much sense.
diff --git a/block/blk-core.c b/block/blk-core.c index 11342af420d0c4..b85ef8a0fdf6a0 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -691,6 +691,8 @@ static void __submit_bio_noacct_mq(struct bio *bio) void submit_bio_noacct_nocheck(struct bio *bio) { + unsigned int noio_flag = memalloc_noio_save(); + blk_cgroup_bio_start(bio); blkcg_bio_issue_init(bio); @@ -715,6 +717,8 @@ void submit_bio_noacct_nocheck(struct bio *bio) __submit_bio_noacct_mq(bio); else __submit_bio_noacct(bio); + + memalloc_noio_restore(noio_flag); } /**