From patchwork Wed Jul 19 22:23:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rae Moar X-Patchwork-Id: 12304 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:c923:0:b0:3e4:2afc:c1 with SMTP id j3csp2753466vqt; Wed, 19 Jul 2023 15:49:09 -0700 (PDT) X-Google-Smtp-Source: APBJJlFrdXcwDhq6j1IQzldAGPjN6pgloKvmwHC6PfwuXuAzzXh1nKtIxuwkegps5nCmvLMIlyEt X-Received: by 2002:a05:6a21:32a4:b0:132:cb97:83e2 with SMTP id yt36-20020a056a2132a400b00132cb9783e2mr18119223pzb.50.1689806949109; Wed, 19 Jul 2023 15:49:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1689806949; cv=none; d=google.com; s=arc-20160816; b=BYK4K4Cb5hZpupv8yXRtHuoKD88ZDkno0KeGmJeoIovnsxnKsunNY6RHef5tCtKSTh +WhJaJ58IwWZlLt+FJZ3Li8p++DveWG0o6f3KBEeoUeCrDsVYzOantqQcIjHeWxXe3sY ovpXBlYM2RZ9Lnp0G0QbDfvJYvybOI4dgpt/8e26JA9XQi+Hwyvvoeg74gvFvt076aoi ul1bUL/rRWr52zgUw/5EB+xaANlT1Px2g5mX20cSgqyaTjzmX7ycVvbfR4SA3j+KayAy DzX+kz1cP1pxdMwot8+STEJpdxcjteKIln4mmOEeNdx4yRbIeoWAPv+mI2pf9Q1COw/H kbCg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:mime-version:date :dkim-signature; bh=wQ30uDB639VGNW+T7EeVAUkSIhG2Jevrm/2Jy436reA=; fh=eSOGplXNKm0TQlVMiAPjma3ZxjxuDQ0l9LD5ETEQPUY=; b=THf3BVAB633x9w2yotrU5o0Tr1uJpwIAbd1vcXjjlbxwqc/vTxygdMs50ZOz14HdkV yDFjT/N4NHVl5vDOeUTQsFBn9cOLBN7U5sGHClLURhFd5HQboEe2zMdOErf7FyTDojIU W5xXxCOZ17vK5NZDdRChZutqguI3NriXtSwIc4KM54u0YTs2U85yrgbqCFqS34T9oZkc 8EKD98shewn5jQwrJwpF/QbLjp2v0QZ2e4ZKlvHpmoqjm/mUZ4sHUHQni+PrP8DLWTNz 5M3e442GuKj7XeST6NzpiUr7WKUWAfFtgrEcMudvv3jLGgaiuSoCReuHastDzqn/5Liy 6YHw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=jGBW4v5u; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id j20-20020a056a00235400b0064378bea76dsi4423947pfj.239.2023.07.19.15.48.56; Wed, 19 Jul 2023 15:49:09 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=jGBW4v5u; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230115AbjGSWXv (ORCPT + 99 others); Wed, 19 Jul 2023 18:23:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55164 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229936AbjGSWXu (ORCPT ); Wed, 19 Jul 2023 18:23:50 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 829E91FE6 for ; Wed, 19 Jul 2023 15:23:48 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-cac66969edaso113231276.3 for ; Wed, 19 Jul 2023 15:23:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1689805428; x=1692397428; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=wQ30uDB639VGNW+T7EeVAUkSIhG2Jevrm/2Jy436reA=; b=jGBW4v5uLMfOolbkZxtwRaLLhDb+BMh1OFRzrhGuOBXni+46bSXTF9cgl+VZEDO8Zi TrLvdFk/cyvWRxefzpSNGAOjgf6RjvfmnAO/RNmU2XbF19LVe4549pnH4RDAy4354SDL 9gr3n8hzyoRQqFH0nUeXAmfAl3J0bzU9aVdfrmQrYG6FjniIUtREGJUpiN+akhOB2Yfs 8V8GW9tAbO8kT0Xitj+p3G9GwiWPi4wxSsvL1f9o2OLllrYDR/o3RxkAJTNZKOIYoLhv 7/LJk9tRrF9kMjmXactWkHW9l1NDolQBeLzictRKTBB/5LojaVtXTRImzGTMfPsV1e7d htxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689805428; x=1692397428; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=wQ30uDB639VGNW+T7EeVAUkSIhG2Jevrm/2Jy436reA=; b=HvMgpNHuAH9dwTgNNSVz9VaSyfESXPBTLtyeWRgJRTwke2S0+UnnqVdpOjIFQNERIr AybBR7SBpKBYCsxYMuADH3bEpF0Xw53teNWaJ597W8JRu5FcsovahLnxz9qjvE+wrlSq OwgKz7gCHsIf2JW5/M7K0xrqxeKQ9gUnfyk9ERAHrIyUiFC5lu094/ec1ewAAHYN6ixY GRKJenauKJnuD04qyjABo8Ti8dOA6B8z1eAE86ypmKPGgFxS4/e60wOnj9kuVJFyzUH9 sQRZKioe8d4iikCARbf3E0QaSuECtxzsLu4ShVP2MP2ZAOIfFowHUuHIYrIxT+95bfYy 7Ttg== X-Gm-Message-State: ABy/qLZFDUSm3Z0c2AqXi3x4MGY9atDWFC5zG3SnNStMkjpeTzoOZ3Hx VkHOiB7dPCq5lj5k9a3Hj4hhgnoKwQ== X-Received: from rmoar-specialist.c.googlers.com ([fda3:e722:ac3:cc00:2b:7d90:c0a8:45d3]) (user=rmoar job=sendgmr) by 2002:a05:6902:1709:b0:cab:e42c:876b with SMTP id by9-20020a056902170900b00cabe42c876bmr34333ybb.3.1689805427820; Wed, 19 Jul 2023 15:23:47 -0700 (PDT) Date: Wed, 19 Jul 2023 22:23:27 +0000 Mime-Version: 1.0 X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog Message-ID: <20230719222338.259684-1-rmoar@google.com> Subject: [PATCH v1 0/9] kunit: Add test attributes API From: Rae Moar To: shuah@kernel.org, davidgow@google.com, dlatypov@google.com, brendan.higgins@linux.dev Cc: linux-kselftest@vger.kernel.org, kunit-dev@googlegroups.com, linux-kernel@vger.kernel.org, keescook@chromium.org, linux-hardening@vger.kernel.org, jstultz@google.com, tglx@linutronix.de, sboyd@kernel.org, Rae Moar X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED, USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1771891011738298923 X-GMAIL-MSGID: 1771891011738298923 Hello everyone, This patch series adds a test attributes framework to KUnit. There has been interest in filtering out "slow" KUnit tests. Most notably, a new config, CONFIG_MEMCPY_SLOW_KUNIT_TEST, has been added to exclude a particularly slow memcpy test (https://lore.kernel.org/all/20230118200653.give.574-kees@kernel.org/). This attributes framework can be used to save and access test associated data, including whether a test is slow. These attributes are reportable (via KTAP and command line output) and are also filterable. This framework is designed to allow for the addition of other attributes in the future. These attributes could include whether the test can be run concurrently, test file path, etc. To try out the framework I suggest running: "./tools/testing/kunit/kunit.py run --filter speed!=slow" This patch series was originally sent out as an RFC. Here is a link to the RFC v2: https://lore.kernel.org/all/20230707210947.1208717-1-rmoar@google.com/ Thanks! Rae Rae Moar (9): kunit: Add test attributes API structure kunit: Add speed attribute kunit: Add module attribute kunit: Add ability to filter attributes kunit: tool: Add command line interface to filter and report attributes kunit: memcpy: Mark tests as slow using test attributes kunit: time: Mark test as slow using test attributes kunit: add tests for filtering attributes kunit: Add documentation of KUnit test attributes .../dev-tools/kunit/running_tips.rst | 166 +++++++ include/kunit/attributes.h | 50 +++ include/kunit/test.h | 70 ++- kernel/time/time_test.c | 2 +- lib/Kconfig.debug | 3 + lib/kunit/Makefile | 3 +- lib/kunit/attributes.c | 421 ++++++++++++++++++ lib/kunit/executor.c | 115 ++++- lib/kunit/executor_test.c | 128 +++++- lib/kunit/kunit-example-test.c | 9 + lib/kunit/test.c | 27 +- lib/memcpy_kunit.c | 8 +- tools/testing/kunit/kunit.py | 70 ++- tools/testing/kunit/kunit_kernel.py | 8 +- tools/testing/kunit/kunit_parser.py | 11 +- tools/testing/kunit/kunit_tool_test.py | 39 +- 16 files changed, 1054 insertions(+), 76 deletions(-) create mode 100644 include/kunit/attributes.h create mode 100644 lib/kunit/attributes.c base-commit: 64bd4641310c41a1ecf07c13c67bc0ed61045dfd