From patchwork Tue Jul 25 21:25:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rae Moar X-Patchwork-Id: 12610 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:a985:0:b0:3e4:2afc:c1 with SMTP id t5csp34392vqo; Tue, 25 Jul 2023 15:03:49 -0700 (PDT) X-Google-Smtp-Source: APBJJlFnyRLTh7lJSOdyfIG/hwzxYrM5Y4VbPKlk1sCqNbJ4CNP+2+bzkuO0S6eZRl9X2rI9kHWh X-Received: by 2002:a17:902:ea06:b0:1b8:9552:ca with SMTP id s6-20020a170902ea0600b001b8955200camr412863plg.45.1690322629640; Tue, 25 Jul 2023 15:03:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690322629; cv=none; d=google.com; s=arc-20160816; b=rBeLSXTb9XgpmNYtb3O6LY9iwp3yeNnH8FZ0YIgAzecAcuWFYiOQ4Bd4LVuojtbDiB GomELn/5xjjsX1rRkxCWNGRVnNl6JJmMpDFhAHLcsx5uHTct5Y8rJkuaZWLPuSFoOrhh /3Ey8H1cLQE62kJHTpSnD7ByB/+e843hG+jC/rFuJrm1OxMa0aKImGKyp2jNs5pEilIh +GTKgJ1ycrOgZ9FFfMS+plhKA4N7gEFwhhsKhGhNXJFOlTVa7+P5iElXkAhNbCbTe2lZ DYgEdGLJOwmOurxrF+X3DoFCN0i+PF5hv1OF4esgbqtPlr1uExRqmxyhfhIg5BBukaYs TJ+w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:mime-version:date :dkim-signature; bh=1BncUN+zxsP5jL5qBPBFJiIzgW7UOfc4w8KYyzj3ScI=; fh=eSOGplXNKm0TQlVMiAPjma3ZxjxuDQ0l9LD5ETEQPUY=; b=EGntZMNKjTd1IbWVLXvc/JQjJwTbnSr9MhpOEIrURxqjZ1ZSiZY/z8omYoclnUcCJj FNpM0f5nm6e9m2xqfUf9AJ9e78kZzNbgE12ND2dsfTt1Tj0HEIagOIDmEGuhNWKnvWZ2 Zy+GzTSejk8ls4Z7MexT6+jSifqhR9zm9cauQ8d1uLVL+hqF7HqxWpJznMfhJG61itHx brDsDPWAnp+P1SjU/eAKSD1r+MhlvTs1iML9QjAMIo2i1AjGm9FsXmqF+XL086uWWRj4 Sm7EoNiMolbBjF3wYf+G3PHp0hPINSVanIMJKMFIwT/Zjtesuwm502hb3s//7O1Bqpc8 TrPw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=QnwBoAOn; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id j7-20020a170903024700b001b9e21bc14dsi12597659plh.652.2023.07.25.15.03.22; Tue, 25 Jul 2023 15:03:49 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=QnwBoAOn; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230403AbjGYVZe (ORCPT + 99 others); Tue, 25 Jul 2023 17:25:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56748 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229499AbjGYVZa (ORCPT ); Tue, 25 Jul 2023 17:25:30 -0400 Received: from mail-yw1-x114a.google.com (mail-yw1-x114a.google.com [IPv6:2607:f8b0:4864:20::114a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 109E51FD2 for ; Tue, 25 Jul 2023 14:25:29 -0700 (PDT) Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-583fe0f84a5so28041747b3.3 for ; Tue, 25 Jul 2023 14:25:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1690320328; x=1690925128; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=1BncUN+zxsP5jL5qBPBFJiIzgW7UOfc4w8KYyzj3ScI=; b=QnwBoAOnhYZ7R8UVvGHqcCPy1vPb+326cnbSsoW+eyzwAp1kyyFf2dWTCh301GsO0y eg+XQAfZkVmwh7hK6uLYgDQXtcr3qALDorbc86Num/iM/d2mLNdAQVjatQQu27QuonEO cHfDnO2FbYZUE5OwnTdjqTEsLvU1hpYSn1HdstuDvGnRToxW3zkg28CYdnVeus/tCktl 6nobV9L7yp7kfcq6xUDOq+zY4y3mD6u+KNDo+fKe0d41z7vbUppES+YSVmdNl+z8A5a2 Nkek+41sNJ6KZZPX43KtYO55nZvg0qAiKUVJ8VP2v2eIVQPJR8pKlMOclG5wqFGBdItB kpnw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690320328; x=1690925128; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=1BncUN+zxsP5jL5qBPBFJiIzgW7UOfc4w8KYyzj3ScI=; b=a1inEF7iW/J+a+gbGBlm0M62am5S1KSkQvyDJXI/9FhAb4UXkQ2BX3CQxNzA/pvsJU 18uTGr7rMIko6yCEto/61Lf9aGwCalGHbvb4b52y2hvNR5UhEnHepXeRBxsZ7Ux7Y8PG AcK2xbfWaTb9zDahqTy72rnXXuS+cYDRS/LMxNKKlqX8Qa7RG1OOhIUX6aerLkl1pjed yAVbYNqhAHuUM+YDc2L+hVaofLHOoXnRHpGY1WT+7rENLjrowXSSi7t5W3O7ibZA4/6f 8u6g3W0R0f93nHUmtzX7lbn79LAljlgWfREp06xJRz8frnUWenGGhtvL9nPQhgAqfbAB xz1Q== X-Gm-Message-State: ABy/qLZectAdnpep1OlGV6g7E72bpb0BKoNDOvawOhIHAJfUlGEXprMm +OXGwdYtCURyoXv8hQPRikwW+431XQ== X-Received: from rmoar-specialist.c.googlers.com ([fda3:e722:ac3:cc00:2b:7d90:c0a8:45d3]) (user=rmoar job=sendgmr) by 2002:a81:b71d:0:b0:579:e379:d5ab with SMTP id v29-20020a81b71d000000b00579e379d5abmr3453ywh.6.1690320328318; Tue, 25 Jul 2023 14:25:28 -0700 (PDT) Date: Tue, 25 Jul 2023 21:25:11 +0000 Mime-Version: 1.0 X-Mailer: git-send-email 2.41.0.487.g6d72f3e995-goog Message-ID: <20230725212522.1622716-1-rmoar@google.com> Subject: [PATCH v3 0/9] kunit: Add test attributes API From: Rae Moar To: shuah@kernel.org, davidgow@google.com, dlatypov@google.com, brendan.higgins@linux.dev Cc: linux-kselftest@vger.kernel.org, kunit-dev@googlegroups.com, linux-kernel@vger.kernel.org, keescook@chromium.org, linux-hardening@vger.kernel.org, jstultz@google.com, tglx@linutronix.de, sboyd@kernel.org, Rae Moar X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED, USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1772431741379199609 X-GMAIL-MSGID: 1772431741379199609 Hello everyone, This patch series adds a test attributes framework to KUnit. There has been interest in filtering out "slow" KUnit tests. Most notably, a new config, CONFIG_MEMCPY_SLOW_KUNIT_TEST, has been added to exclude a particularly slow memcpy test (https://lore.kernel.org/all/20230118200653.give.574-kees@kernel.org/). This attributes framework can be used to save and access test associated data, including whether a test is slow. These attributes are reportable (via KTAP and command line output) and are also filterable. This framework is designed to allow for the addition of other attributes in the future. These attributes could include whether the test can be run concurrently, test file path, etc. To try out the framework I suggest running: "./tools/testing/kunit/kunit.py run --filter speed!=slow" This patch series was originally sent out as an RFC. Here is a link to the RFC v2: https://lore.kernel.org/all/20230707210947.1208717-1-rmoar@google.com/ Thanks! Rae Rae Moar (9): kunit: Add test attributes API structure kunit: Add speed attribute kunit: Add module attribute kunit: Add ability to filter attributes kunit: tool: Add command line interface to filter and report attributes kunit: memcpy: Mark tests as slow using test attributes kunit: time: Mark test as slow using test attributes kunit: add tests for filtering attributes kunit: Add documentation of KUnit test attributes Documentation/dev-tools/kunit/run_wrapper.rst | 12 + .../dev-tools/kunit/running_tips.rst | 166 +++++++ include/kunit/attributes.h | 50 +++ include/kunit/test.h | 70 ++- kernel/time/time_test.c | 2 +- lib/Kconfig.debug | 3 + lib/kunit/Makefile | 3 +- lib/kunit/attributes.c | 418 ++++++++++++++++++ lib/kunit/executor.c | 114 ++++- lib/kunit/executor_test.c | 128 +++++- lib/kunit/kunit-example-test.c | 9 + lib/kunit/test.c | 27 +- lib/memcpy_kunit.c | 8 +- tools/testing/kunit/kunit.py | 70 ++- tools/testing/kunit/kunit_kernel.py | 8 +- tools/testing/kunit/kunit_parser.py | 11 +- tools/testing/kunit/kunit_tool_test.py | 39 +- 17 files changed, 1062 insertions(+), 76 deletions(-) create mode 100644 include/kunit/attributes.h create mode 100644 lib/kunit/attributes.c base-commit: 64bd4641310c41a1ecf07c13c67bc0ed61045dfd Reviewed-by: Arthur Grillo Reviewed-by: Arthur Grillo