From patchwork Fri Mar 24 13:57:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 74555 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp647492vqo; Fri, 24 Mar 2023 07:02:27 -0700 (PDT) X-Google-Smtp-Source: AKy350Y1ezv3FUTk5D8g6C4AZsAeZnOWJnvnwG2KLPWujmCBYvrdZjJ3ZyArjRCMM0leCw/3CWCn X-Received: by 2002:a17:906:c791:b0:8f2:62a9:6159 with SMTP id cw17-20020a170906c79100b008f262a96159mr2779400ejb.2.1679666546905; Fri, 24 Mar 2023 07:02:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1679666546; cv=none; d=google.com; s=arc-20160816; b=IVvI1qUdhIm0qj98lK2tzuoaM1JGUjrutCNryJbjekFtnOQd9QKrUi1NahcTHvnazI d/Xu/FWQ+JLQJ6TYylAERE8PvBTYehkvk/dXSJ/9eyJgTIgjvcpUBhyjhuDcpzMD/fdr Vg+qBLRuglnpVgCfMfRrvM6sr7RJ/0/gvKV07zRCPVauAv4Vxkd4YLF/G27Xtg8Lsk4A 2Xzl5to32E2FBeyDt61yUbPtoed58/yvgaAwjjrnhG9WnCH2447D9T/g2ZKnnXFe6y4P MrI3d2Zhkv+d6mE5GImhmIayBZW43ByTVuN8Mxyn1xIUlVrS9fN4BDy+qYxEfVdIKkVh yRJQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=AOAiaCAkofmrjLYBJAozNfXVKYb+VVcnQ70rF3f1ifw=; b=fK12vnlr3mBa/iT3/buiRGwFpMXSQD//jijFDtSvlTHkoskIbVJSILECFBzMkRRYaI 7kJ980qKXLUyceOtJP78IayPF8s+QVMH5CINGrwo8WxWw4DKPhf0pu1ZoYP2ozwpp2kE 1Yt8OEAjEj3Rwl4vPTxQOQmYSMdEDvQcAUQD4MeX6HTEjK9YOG6Cen1zG1lKoIicurlk RENgjBEAwYBGpI0TiaxKQ54SyZpMBUg3WkMxx2sHzbe7zX6Nv3XgQG9iCFdyxQrFQZJ/ SASqSOJlkCthxE/u7+y2scRvYoj4Vgvl75cXMaPWALXpqxuYYZl6QfZ8PbRSnkxgXvwE XFTQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=HTvZwzRT; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id a13-20020a1709066d4d00b0092be4d3413asi20578837ejt.131.2023.03.24.07.01.58; Fri, 24 Mar 2023 07:02:26 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=HTvZwzRT; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232098AbjCXOAT (ORCPT + 99 others); Fri, 24 Mar 2023 10:00:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50664 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231950AbjCXOAF (ORCPT ); Fri, 24 Mar 2023 10:00:05 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E939E1ADC5 for ; Fri, 24 Mar 2023 06:58:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1679666327; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AOAiaCAkofmrjLYBJAozNfXVKYb+VVcnQ70rF3f1ifw=; b=HTvZwzRTzonwGKhCAwJPWeywi+vGKodv0jq1hyPQKyaqHbtE8/oINRH/6Z9r544oNqqK3g Ea5gkdBKi7dZAOZGiQzTX9NfJv1iyB3Sdo8Wizaek+9PF5nZ5EFeniPLOTb9HDs+gn8qpg pzyEwEdQMyAqwX/YKukNjK5qgxCIJKY= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-458-x23RE0-LOEyXfmhlZ1BRRw-1; Fri, 24 Mar 2023 09:58:40 -0400 X-MC-Unique: x23RE0-LOEyXfmhlZ1BRRw-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id CA70A18E0062; Fri, 24 Mar 2023 13:58:38 +0000 (UTC) Received: from localhost (ovpn-8-20.pek2.redhat.com [10.72.8.20]) by smtp.corp.redhat.com (Postfix) with ESMTP id 08AE34020C81; Fri, 24 Mar 2023 13:58:37 +0000 (UTC) From: Ming Lei To: Jens Axboe , io-uring@vger.kernel.org, linux-block@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Miklos Szeredi , ZiyangZhang , Xiaoguang Wang , Bernd Schubert , Pavel Begunkov , Stefan Hajnoczi , Ming Lei Subject: [PATCH V4 05/17] io_uring: support OP_SEND_ZC/OP_RECV for fused slave request Date: Fri, 24 Mar 2023 21:57:56 +0800 Message-Id: <20230324135808.855245-6-ming.lei@redhat.com> In-Reply-To: <20230324135808.855245-1-ming.lei@redhat.com> References: <20230324135808.855245-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 X-Spam-Status: No, score=-0.2 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1761258028920250320?= X-GMAIL-MSGID: =?utf-8?q?1761258028920250320?= Start to allow fused slave request to support OP_SEND_ZC/OP_RECV, and the buffer can be retrieved from master request. Once the slave request is completed, the master buffer will be returned back. Signed-off-by: Ming Lei --- io_uring/net.c | 30 ++++++++++++++++++++++++++++-- io_uring/opdef.c | 6 ++++++ 2 files changed, 34 insertions(+), 2 deletions(-) diff --git a/io_uring/net.c b/io_uring/net.c index b7f190ca528e..bc33f89f61b3 100644 --- a/io_uring/net.c +++ b/io_uring/net.c @@ -16,6 +16,7 @@ #include "net.h" #include "notif.h" #include "rsrc.h" +#include "fused_cmd.h" #if defined(CONFIG_NET) struct io_shutdown { @@ -68,6 +69,13 @@ struct io_sr_msg { struct io_kiocb *notif; }; +#define user_ptr_to_u64(x) ( \ +{ \ + typecheck(void __user *, (x)); \ + (u64)(unsigned long)(x); \ +} \ +) + static inline bool io_check_multishot(struct io_kiocb *req, unsigned int issue_flags) { @@ -378,7 +386,11 @@ int io_send(struct io_kiocb *req, unsigned int issue_flags) if (unlikely(!sock)) return -ENOTSOCK; - ret = import_ubuf(ITER_SOURCE, sr->buf, sr->len, &msg.msg_iter); + if (!(req->flags & REQ_F_FUSED_SLAVE)) + ret = import_ubuf(ITER_SOURCE, sr->buf, sr->len, &msg.msg_iter); + else + ret = io_import_buf_for_slave(user_ptr_to_u64(sr->buf), + sr->len, ITER_SOURCE, &msg.msg_iter, req); if (unlikely(ret)) return ret; @@ -869,7 +881,11 @@ int io_recv(struct io_kiocb *req, unsigned int issue_flags) sr->buf = buf; } - ret = import_ubuf(ITER_DEST, sr->buf, len, &msg.msg_iter); + if (!(req->flags & REQ_F_FUSED_SLAVE)) + ret = import_ubuf(ITER_DEST, sr->buf, len, &msg.msg_iter); + else + ret = io_import_buf_for_slave(user_ptr_to_u64(sr->buf), + sr->len, ITER_DEST, &msg.msg_iter, req); if (unlikely(ret)) goto out_free; @@ -983,6 +999,9 @@ int io_send_zc_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe) if (zc->flags & IORING_RECVSEND_FIXED_BUF) { unsigned idx = READ_ONCE(sqe->buf_index); + if (req->flags & REQ_F_FUSED_SLAVE) + return -EINVAL; + if (unlikely(idx >= ctx->nr_user_bufs)) return -EFAULT; idx = array_index_nospec(idx, ctx->nr_user_bufs); @@ -1119,8 +1138,15 @@ int io_send_zc(struct io_kiocb *req, unsigned int issue_flags) if (unlikely(ret)) return ret; msg.sg_from_iter = io_sg_from_iter; + } else if (req->flags & REQ_F_FUSED_SLAVE) { + ret = io_import_buf_for_slave(user_ptr_to_u64(zc->buf), + zc->len, ITER_SOURCE, &msg.msg_iter, req); + if (unlikely(ret)) + return ret; + msg.sg_from_iter = io_sg_from_iter; } else { io_notif_set_extended(zc->notif); + ret = import_ubuf(ITER_SOURCE, zc->buf, zc->len, &msg.msg_iter); if (unlikely(ret)) return ret; diff --git a/io_uring/opdef.c b/io_uring/opdef.c index 9b376df91abd..e7d75bf69c0f 100644 --- a/io_uring/opdef.c +++ b/io_uring/opdef.c @@ -273,6 +273,8 @@ const struct io_issue_def io_issue_defs[] = { .audit_skip = 1, .ioprio = 1, .manual_alloc = 1, + .fused_slave = 1, + .buf_dir = READ, #if defined(CONFIG_NET) .prep = io_sendmsg_prep, .issue = io_send, @@ -287,6 +289,8 @@ const struct io_issue_def io_issue_defs[] = { .buffer_select = 1, .audit_skip = 1, .ioprio = 1, + .fused_slave = 1, + .buf_dir = WRITE, #if defined(CONFIG_NET) .prep = io_recvmsg_prep, .issue = io_recv, @@ -413,6 +417,8 @@ const struct io_issue_def io_issue_defs[] = { .audit_skip = 1, .ioprio = 1, .manual_alloc = 1, + .fused_slave = 1, + .buf_dir = READ, #if defined(CONFIG_NET) .prep = io_send_zc_prep, .issue = io_send_zc,