From patchwork Fri Jan 27 10:59:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dylan Yudaken X-Patchwork-Id: 49188 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp772978wrn; Fri, 27 Jan 2023 03:07:37 -0800 (PST) X-Google-Smtp-Source: AK7set+9swYlfGCU1RJeibm0C8emYe7WzhkTd/tzu+eFtpvF6Od7jV3SCx6yDRYPUONPFhy8cYdm X-Received: by 2002:a17:90b:4d04:b0:22c:f74:9415 with SMTP id mw4-20020a17090b4d0400b0022c0f749415mr9244788pjb.36.1674817657176; Fri, 27 Jan 2023 03:07:37 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674817657; cv=none; d=google.com; s=arc-20160816; b=fstX1Rl4sVKH1laBPCOEX4XK0MRvG0ZG1r007tPzpQFTQlG0HMuhbQsPXQaRQ2fBWY WOYdhZnF8bXkVdJkGK/wT4/EwIgYVDq74p856SjfqFw4h/7s3+WTeATcPHG859H/6Rwv d4PtfCyrDKINyYar/ERD3Kar9Em5XlvgJeMQLU98GsR0TQSWbFChRWp3SUUMNX0uPkzY xImCMtVoETIrQIfkKeyKHIE/4Gdny8xAxLzvzCTNhVk2EsRqU/A0rCrdViMZFpNJm4/F osKJ09FiAoA08tsKY+IUrsMATG3EyiNRCUcamXofSEngEJRN+u1rtZpQedpwsANRRNBB ifog== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=SFHN2U2WIAI6nW3KlEx2htg7L7yaTcuHQiYYjVO6IS0=; b=0TLIuBuCPcePfjG3OsXYmHxsRa+MIgwSp0B0/zvhzDySEs6EHT/C9yRQ7eyOigcs3m Jyx8qomDxctsPEPU0Z4WY4xd4l1IJAbymm0JaNVPubmTnMXhk+VGdjjxvlWTBPRg/ih6 KDMIpu64XV1jrXqeZIcGHB7Gdm4D5sU3YYYTmjCBMv2WIxxWIiBlesJ+h6EmAPL5YBSF m7yyFHHQgduOP5ytLq7J1pSoN63p0DieY6l0wSNCUiKZ4EnDdVoy6drxKdhOlkQFAfOC 74RES1yKFyU+NEChuRFAYTfNI3WbsINojtr+xfJEfbRR+M8rBtt+GVNIJR0Uby9xemT4 bk/w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@meta.com header.s=s2048-2021-q4 header.b=KoiwDmB9; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=meta.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id p1-20020a17090a4f0100b0022c19e2e734si5596593pjh.151.2023.01.27.03.07.24; Fri, 27 Jan 2023 03:07:37 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@meta.com header.s=s2048-2021-q4 header.b=KoiwDmB9; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=meta.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232603AbjA0K7h (ORCPT + 99 others); Fri, 27 Jan 2023 05:59:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41374 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232509AbjA0K7e (ORCPT ); Fri, 27 Jan 2023 05:59:34 -0500 Received: from mx0a-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3AF6E46A7 for ; Fri, 27 Jan 2023 02:59:33 -0800 (PST) Received: from pps.filterd (m0001303.ppops.net [127.0.0.1]) by m0001303.ppops.net (8.17.1.19/8.17.1.19) with ESMTP id 30R92UD9030685 for ; Fri, 27 Jan 2023 02:59:32 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=meta.com; h=from : to : cc : subject : date : message-id : mime-version : content-transfer-encoding : content-type; s=s2048-2021-q4; bh=SFHN2U2WIAI6nW3KlEx2htg7L7yaTcuHQiYYjVO6IS0=; b=KoiwDmB9/VUV+ch0S3JWRGjP86nd3iBEYTTPxXYXUnMUjf/auEww73baELTU8X2gnkvL TDPao7jatyvM5J6slVL3h3K1MWDe9HDsSkXWpEpoIWJXGUfGsggWCZGjDH0VQgcZ8QR4 Z0KmJLUWede963RP6rNILgXHojj9CHehW4Z4OHmT+dfxp3qHuAg87Ss1q3glJT5iwB0P PKRClnUJE6PAesWM6tGmNMFgnenV1KaVNbjgCJO0jW2D6XYD9cWauBPLcLfc+OXA4NJ0 r/NpoAO+iJ1QUDF7arNalxk0clNhnoYJnAruMdGGW0Mvclu3GwFkGt/PE9qs/QrSXJIg Sg== Received: from maileast.thefacebook.com ([163.114.130.16]) by m0001303.ppops.net (PPS) with ESMTPS id 3nbe902ru9-11 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Fri, 27 Jan 2023 02:59:32 -0800 Received: from twshared5320.05.ash8.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:83::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.17; Fri, 27 Jan 2023 02:59:29 -0800 Received: by devbig038.lla2.facebook.com (Postfix, from userid 572232) id 5BC63E9FEC07; Fri, 27 Jan 2023 02:59:15 -0800 (PST) From: Dylan Yudaken To: Jens Axboe , Pavel Begunkov CC: , , , Dylan Yudaken , Subject: [PATCH] io_uring: always prep_async for drain requests Date: Fri, 27 Jan 2023 02:59:11 -0800 Message-ID: <20230127105911.2420061-1-dylany@meta.com> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-ORIG-GUID: tl4_UXiNYPuQ85Chah9_aHM49gu-ieVt X-Proofpoint-GUID: tl4_UXiNYPuQ85Chah9_aHM49gu-ieVt X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.930,Hydra:6.0.562,FMLib:17.11.122.1 definitions=2023-01-27_06,2023-01-27_01,2022-06-22_01 X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756173599806886880?= X-GMAIL-MSGID: =?utf-8?q?1756173599806886880?= Drain requests all go through io_drain_req, which has a quick exit in case there is nothing pending (ie the drain is not useful). In that case it can run the issue the request immediately. However for safety it queues it through task work. The problem is that in this case the request is run asynchronously, but the async work has not been prepared through io_req_prep_async. This has not been a problem up to now, as the task work always would run before returning to userspace, and so the user would not have a chance to race with it. However - with IORING_SETUP_DEFER_TASKRUN - this is no longer the case and the work might be defered, giving userspace a chance to change data being referred to in the request. Instead _always_ prep_async for drain requests, which is simpler anyway and removes this issue. Cc: stable@vger.kernel.org Fixes: c0e0d6ba25f1 ("io_uring: add IORING_SETUP_DEFER_TASKRUN") Signed-off-by: Dylan Yudaken --- Hi, Worth pointing out in discussion with Pavel that we considered just removing the fast path in io_drain_req. That felt like more of an invasive change as it can add an extra kmalloc + io_prep_async_link(), but perhaps you feel that is a better approach. I'll send out a liburing test shortly. Dylan io_uring/io_uring.c | 18 ++++++++---------- 1 file changed, 8 insertions(+), 10 deletions(-) base-commit: b00c51ef8f72ced0965d021a291b98ff822c5337 diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c index 0a4efada9b3c..db623b3185c8 100644 --- a/io_uring/io_uring.c +++ b/io_uring/io_uring.c @@ -1765,17 +1765,12 @@ static __cold void io_drain_req(struct io_kiocb *req) } spin_unlock(&ctx->completion_lock); - ret = io_req_prep_async(req); - if (ret) { -fail: - io_req_defer_failed(req, ret); - return; - } io_prep_async_link(req); de = kmalloc(sizeof(*de), GFP_KERNEL); if (!de) { ret = -ENOMEM; - goto fail; + io_req_defer_failed(req, ret); + return; } spin_lock(&ctx->completion_lock); @@ -2048,13 +2043,16 @@ static void io_queue_sqe_fallback(struct io_kiocb *req) req->flags &= ~REQ_F_HARDLINK; req->flags |= REQ_F_LINK; io_req_defer_failed(req, req->cqe.res); - } else if (unlikely(req->ctx->drain_active)) { - io_drain_req(req); } else { int ret = io_req_prep_async(req); - if (unlikely(ret)) + if (unlikely(ret)) { io_req_defer_failed(req, ret); + return; + } + + if (unlikely(req->ctx->drain_active)) + io_drain_req(req); else io_queue_iowq(req, NULL); }