From patchwork Wed Nov 15 15:49:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 16615 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b909:0:b0:403:3b70:6f57 with SMTP id t9csp2627947vqg; Wed, 15 Nov 2023 07:50:49 -0800 (PST) X-Google-Smtp-Source: AGHT+IHcarxRPnKbikNZxQgDiPPBNj3KY+icIHUqyeC6OqUtgwhOknSYx1kyaTR4FaYOO0Rr70WZ X-Received: by 2002:a05:6e02:156b:b0:359:3ac2:5123 with SMTP id k11-20020a056e02156b00b003593ac25123mr18385283ilu.23.1700063449007; Wed, 15 Nov 2023 07:50:49 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700063448; cv=none; d=google.com; s=arc-20160816; b=ZxFdEZt0QRlV/AkzoATcYRd0bBROgWRnfvQ6N7MJJCSpkePfvq9pGMEWXzpdwnsxC1 5fUQUEYsJDCfrA5mhT8xC87Y2SST5vhqgpEIABQBHRlYQ6WjdcD1Wj/TsRD2gofqdWZ1 QJ/afIx8guaWrtLrOLBa9RQgCZLVNbUK4nez81v9xwOX/g5gUcvAK1nYW6qcji4KFig/ C43YHmZOqIUEoDyagVvVTefXbavfbbKmDz8wbHLoR9dOuE/Zme8dbcMSEboeccdwIbsm LElAumMTVoqdyVcCF+dlnANR9WYU664SBY7uoa6kJ8rc2c6Cy4Ce2zI9UaBOQALiphpQ yMcg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=+63cPdnCPLjbKCS5/t308GAcV4bDVpzj1bwgDRa790U=; fh=PGlwbtmYJQXXiHHeo9Y560hs9IIi4KDu+s7LJ6xqrco=; b=zpoMymlyb/Z2zGsQAVBz7hmvxWBVyUyYxqld/ZFSORM2G0AqZVPGY5mqkxgJhbvKis nW9FdCZ5Qv63IBGZ7vlJX7E30nJjo94mI127feiWudSlzNJRXIOfwG8tKXLK8rHvL9nz H5cWUpqSD58f7Fr/8XehPmWurQ/3RJ/p/XtYgN4ASQUHZ2OHm2SsrG9CYXCfiZcb+y+2 0TERg99ekOwJpf63LoShK1xP2AsZ1qlNNjEhF/F4VD6YnjTOFiuqxfgWYLfCuZpi0r/2 d1kZhMB9JZjWrJrmelR7i5BbZ6kC5M8+eTIMgUcGgmu2iUfwo2HpgxDM5NdlYswNsj20 vdng== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b="HBXAnH0/"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from groat.vger.email (groat.vger.email. [23.128.96.35]) by mx.google.com with ESMTPS id bo7-20020a056a02038700b005acd3068202si10996072pgb.552.2023.11.15.07.50.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Nov 2023 07:50:48 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) client-ip=23.128.96.35; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b="HBXAnH0/"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by groat.vger.email (Postfix) with ESMTP id 43BEE80CFD01; Wed, 15 Nov 2023 07:50:34 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at groat.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231599AbjKOPuH (ORCPT + 29 others); Wed, 15 Nov 2023 10:50:07 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46260 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231702AbjKOPuG (ORCPT ); Wed, 15 Nov 2023 10:50:06 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7F284101 for ; Wed, 15 Nov 2023 07:50:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1700063399; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=+63cPdnCPLjbKCS5/t308GAcV4bDVpzj1bwgDRa790U=; b=HBXAnH0/QlzWXNmPHnqaCMTielLa+IMg+W+Ghw4rM7JUg90gylFV9aaUh9vfbcSAlQSK4V Ukb3N5J+XCz8zUhWDMQL12cM2eeNf2SRa6yMtGBFVp5OiPC6RIl//0TzFIw+zGC87Bfiwx LgiN/SMemNky6xdgvCAedDXI9pW34Vg= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-61-UGIGDn_hM6CdQuAGtRNCPw-1; Wed, 15 Nov 2023 10:49:54 -0500 X-MC-Unique: UGIGDn_hM6CdQuAGtRNCPw-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 86A62101A550; Wed, 15 Nov 2023 15:49:53 +0000 (UTC) Received: from warthog.procyon.org.com (unknown [10.42.28.16]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3EC981121306; Wed, 15 Nov 2023 15:49:51 +0000 (UTC) From: David Howells To: Christian Brauner Cc: David Howells , Jens Axboe , Al Viro , Linus Torvalds , Christoph Hellwig , David Laight , Matthew Wilcox , Brendan Higgins , David Gow , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-mm@kvack.org, netdev@vger.kernel.org, linux-kselftest@vger.kernel.org, kunit-dev@googlegroups.com, linux-kernel@vger.kernel.org Subject: [PATCH v3 00/10] iov_iter: kunit: Cleanup, abstraction and more tests Date: Wed, 15 Nov 2023 15:49:36 +0000 Message-ID: <20231115154946.3933808-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.3 X-Spam-Status: No, score=-1.0 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Wed, 15 Nov 2023 07:50:34 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1782645730950609846 X-GMAIL-MSGID: 1782645730950609846 Hi Christian, Can you take this through the filesystem tree? These patches make some changes to the kunit tests previously added for iov_iter testing, in particular adding testing of UBUF/IOVEC iterators and some benchmarking: (1) Clean up a couple of checkpatch style complaints. (2) Consolidate some repeated bits of code into helper functions and use the same struct to represent straight offset/address ranges and partial page lists. (3) Add a function to set up a userspace VM, attach the VM to the kunit testing thread, create an anonymous file, stuff some pages into the file and map the file into the VM to act as a buffer that can be used with UBUF/IOVEC iterators. I map an anonymous file with pages attached rather than using MAP_ANON so that I can check the pages obtained from iov_iter_extract_pages() without worrying about them changing due to swap, migrate, etc.. [?] Is this the best way to do things? Mirroring execve, it requires a number of extra core symbols to be exported. Should this be done in the core code? (4) Add tests for copying into and out of UBUF and IOVEC iterators. (5) Add tests for extracting pages from UBUF and IOVEC iterators. (6) Add tests to benchmark copying 256MiB to UBUF, IOVEC, KVEC, BVEC and XARRAY iterators. (7) Add a test to bencmark copying 256MiB from an xarray that gets decanted into 256-page BVEC iterators to model batching from the pagecache. (8) Add a test to benchmark copying 256MiB through dynamically allocated 256-page bvecs to simulate bio construction. Example benchmarks output: iov_kunit_benchmark_ubuf: avg 4474 uS, stddev 1340 uS iov_kunit_benchmark_iovec: avg 6619 uS, stddev 23 uS iov_kunit_benchmark_kvec: avg 2672 uS, stddev 14 uS iov_kunit_benchmark_bvec: avg 3189 uS, stddev 19 uS iov_kunit_benchmark_bvec_split: avg 3403 uS, stddev 8 uS iov_kunit_benchmark_xarray: avg 3709 uS, stddev 7 uS I've pushed the patches here also: https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git/log/?h=iov-kunit David Changes ======= ver #3) - #include to get READ_IMPLIES_EXEC. - Add a test to benchmark decanting an xarray into bio_vecs. ver #2) - Use MAP_ANON to make the user buffer if we don't want a list of pages. - KUNIT_ASSERT_NOT_ERR_OR_NULL() doesn't like __user pointers as the condition, so cast. - Make the UBUF benchmark loop, doing an iterator per page so that the overhead from the iterator code is not negligible. - Make the KVEC benchmark use an iovec per page so that the iteration is not not negligible. - Switch the benchmarking to use copy_from_iter() so that only a single page is needed in the userspace buffer (as it can be shared R/O), not 256MiB's worth. Link: https://lore.kernel.org/r/20230914221526.3153402-1-dhowells@redhat.com/ # v1 Link: https://lore.kernel.org/r/20230920130400.203330-1-dhowells@redhat.com/ # v2 Link: https://lore.kernel.org/r/20230922113038.1135236-1-dhowells@redhat.com/ # v3 David Howells (10): iov_iter: Fix some checkpatch complaints in kunit tests iov_iter: Consolidate some of the repeated code into helpers iov_iter: Consolidate the test vector struct in the kunit tests iov_iter: Consolidate bvec pattern checking iov_iter: Create a function to prepare userspace VM for UBUF/IOVEC tests iov_iter: Add copy kunit tests for ITER_UBUF and ITER_IOVEC iov_iter: Add extract kunit tests for ITER_UBUF and ITER_IOVEC iov_iter: Add benchmarking kunit tests iov_iter: Add kunit to benchmark decanting of xarray to bvec iov_iter: Add benchmarking kunit tests for UBUF/IOVEC arch/s390/kernel/vdso.c | 1 + fs/anon_inodes.c | 1 + kernel/fork.c | 2 + lib/kunit_iov_iter.c | 1317 +++++++++++++++++++++++++++++++++------ mm/mmap.c | 1 + mm/util.c | 3 + 6 files changed, 1139 insertions(+), 186 deletions(-)