From patchwork Mon Jul 24 16:28:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rae Moar X-Patchwork-Id: 12526 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9010:0:b0:3e4:2afc:c1 with SMTP id l16csp1926610vqg; Mon, 24 Jul 2023 09:57:37 -0700 (PDT) X-Google-Smtp-Source: APBJJlEl1mZhaBmvpxGJrS5RRUVf2/ZsX0qz6IhKyAFxUCCP9iHwKISFHwrZskf5r1ZuyPiD9ZBA X-Received: by 2002:a05:6e02:1152:b0:348:8b32:7ca3 with SMTP id o18-20020a056e02115200b003488b327ca3mr421829ill.5.1690217857602; Mon, 24 Jul 2023 09:57:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690217857; cv=none; d=google.com; s=arc-20160816; b=V3/yU7f09FnC/4SKUTYHP6eUEYoTc2g9x/K1I0K60RkhiNurcW1uVcIIKVOhw7eTiD 4qkEX+VGVMm1v9vJZAqdlnkIUXzKPg1PrtI962VPtTu1bufpbfmkPzdFtUMs2ywsSEbd nxoESFd4HbZrXhdOYfGR8SJ+reYkY8/GB/BYOT+6Khumtm49rOwzBKnwAuOy93kCKkPk fM2/GZac31yXQSI8nzPO51/IynNgPOiH6qr5pkMihHLFah7OrK9TOSAqTbXqxykz4xXm vVTjo7CX6FKiRb334dFSCNhbZjFBZOw5spPGJHHdwLgEAKISybh94au2JHOzsOPFrrjo u1aw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:mime-version:date :dkim-signature; bh=ELz6+MdFAwylMEzq57vnQLUyST7xKxQ3gFrQ4xet5oM=; fh=eSOGplXNKm0TQlVMiAPjma3ZxjxuDQ0l9LD5ETEQPUY=; b=tOIb/rlEeUl6jAXHOcgo3PasJi3eUUnMh1LSHw8cx4cxIbf8cPdMkPF4K86KPsIkud PzW8decDhFeDLrwbX0rLBr00PoF5c+3aXynXYz4hYf1vgtYu0lHngBJMIANFkyz4MMBy Rw63PC08wH2qPRYYtGGaGS5ACczBZJn870FKtXzDwX2zSWlr6UZ0xRARNgbF9Jyb+f55 US8XvVrfQR8LRmmSISrV82dyf5B0AkLyU5YvfkKfsUoWx+duOya+hipk9+Zj5bwsWP3g 0EWJWfCsvU8GBEwi1ilf5I62VYmcl8CuDGznoyFMC5ADEszvdGj/grgL9ZHtKlW7dVeV Ql3w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=BOuKpo28; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id be3-20020a656e43000000b00543a6ce8c33si9545337pgb.78.2023.07.24.09.57.22; Mon, 24 Jul 2023 09:57:37 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=BOuKpo28; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231210AbjGXQa3 (ORCPT + 99 others); Mon, 24 Jul 2023 12:30:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57018 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230173AbjGXQa0 (ORCPT ); Mon, 24 Jul 2023 12:30:26 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1FD63170E for ; Mon, 24 Jul 2023 09:30:14 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-cf4cb742715so5249642276.2 for ; Mon, 24 Jul 2023 09:30:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1690216213; x=1690821013; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=ELz6+MdFAwylMEzq57vnQLUyST7xKxQ3gFrQ4xet5oM=; b=BOuKpo28jYR0TEsF3UqCk6b9rkYbObdDcsJIOPy/T5B3QW/fTsV1V8pg6UsKAp4Auo tQct2po9F1GYuWOrgrBX78Slhb4boeo9R1H3dcxBBt+xbxgxQufkI15lvoAhW7QJQVNz Xgia7i3xcvOY3ceFl81PRkxmOsUXlox/ekcVwcgziBVgmT6X7s5nGzAvmvGIb5dBkCxF KeUOX5jQ+XsFJw9zhr7bi9viIsgkG6z0azx3U9Rle9Ndlf94FNikuAu1pMGMlKIE1AZL lRX/qFVIc5qxRf67DfVMy4hc5JXkD+OMHwpYJWInYQdyiJByR8eqL45UQemqnuPEN8r/ TC1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690216213; x=1690821013; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=ELz6+MdFAwylMEzq57vnQLUyST7xKxQ3gFrQ4xet5oM=; b=KQl82iMYtmMTb/FHA2Xp/LJ5DvkBzSOXdEfnuxwXrJCKfaOEHWG/lhuuL7SJ/oHkhS Eu4CqRLIDoXu4Gyu/nR9wd/IT13fS7deU5+klsMcvS44yqGgDpKRf4h3eeAv9SQOAu5R gHO7J1sDANU4RGz0x3s1zzpm52cfzNyhN6uyulGaCBg9o1EdeVC1a4q9pkw9WUMHq91/ 29CoQZ2+BJuKMnSyAfu+wlIXhBwsYCGAz3AFJKD5urdDKOwhPXJELvDncwC9yipNLyFo oXqCh0ozGAhwgWzbZC4dNPObbXmVRQHLfTYjRqV6MplJDFOi34C4xWTdqeuIoSfi0cdH 7/pg== X-Gm-Message-State: ABy/qLbbjUdLew4Pc1uJXGjTrRf+SWMQw01um4hAxlxn0QmzN7pmocSz 7qc2EmxpSd86dQE0w0SPczKaXn6hOg== X-Received: from rmoar-specialist.c.googlers.com ([fda3:e722:ac3:cc00:2b:7d90:c0a8:45d3]) (user=rmoar job=sendgmr) by 2002:a25:d887:0:b0:d12:d6e4:a08d with SMTP id p129-20020a25d887000000b00d12d6e4a08dmr12086ybg.7.1690216213261; Mon, 24 Jul 2023 09:30:13 -0700 (PDT) Date: Mon, 24 Jul 2023 16:28:20 +0000 Mime-Version: 1.0 X-Mailer: git-send-email 2.41.0.487.g6d72f3e995-goog Message-ID: <20230724162834.1354164-1-rmoar@google.com> Subject: [PATCH v2 0/9] kunit: Add test attributes API From: Rae Moar To: shuah@kernel.org, davidgow@google.com, dlatypov@google.com, brendan.higgins@linux.dev Cc: linux-kselftest@vger.kernel.org, kunit-dev@googlegroups.com, linux-kernel@vger.kernel.org, keescook@chromium.org, linux-hardening@vger.kernel.org, jstultz@google.com, tglx@linutronix.de, sboyd@kernel.org, Rae Moar X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1772321880366994766 X-GMAIL-MSGID: 1772321880366994766 Hello everyone, This patch series adds a test attributes framework to KUnit. There has been interest in filtering out "slow" KUnit tests. Most notably, a new config, CONFIG_MEMCPY_SLOW_KUNIT_TEST, has been added to exclude a particularly slow memcpy test (https://lore.kernel.org/all/20230118200653.give.574-kees@kernel.org/). This attributes framework can be used to save and access test associated data, including whether a test is slow. These attributes are reportable (via KTAP and command line output) and are also filterable. This framework is designed to allow for the addition of other attributes in the future. These attributes could include whether the test can be run concurrently, test file path, etc. To try out the framework I suggest running: "./tools/testing/kunit/kunit.py run --filter speed!=slow" This patch series was originally sent out as an RFC. Here is a link to the RFC v2: https://lore.kernel.org/all/20230707210947.1208717-1-rmoar@google.com/ Thanks! Rae Rae Moar (9): kunit: Add test attributes API structure kunit: Add speed attribute kunit: Add module attribute kunit: Add ability to filter attributes kunit: tool: Add command line interface to filter and report attributes kunit: memcpy: Mark tests as slow using test attributes kunit: time: Mark test as slow using test attributes kunit: add tests for filtering attributes kunit: Add documentation of KUnit test attributes .../dev-tools/kunit/running_tips.rst | 166 +++++++ include/kunit/attributes.h | 50 +++ include/kunit/test.h | 70 ++- kernel/time/time_test.c | 2 +- lib/Kconfig.debug | 3 + lib/kunit/Makefile | 3 +- lib/kunit/attributes.c | 418 ++++++++++++++++++ lib/kunit/executor.c | 115 ++++- lib/kunit/executor_test.c | 128 +++++- lib/kunit/kunit-example-test.c | 9 + lib/kunit/test.c | 27 +- lib/memcpy_kunit.c | 8 +- tools/testing/kunit/kunit.py | 70 ++- tools/testing/kunit/kunit_kernel.py | 8 +- tools/testing/kunit/kunit_parser.py | 11 +- tools/testing/kunit/kunit_tool_test.py | 39 +- 16 files changed, 1051 insertions(+), 76 deletions(-) create mode 100644 include/kunit/attributes.h create mode 100644 lib/kunit/attributes.c base-commit: 64bd4641310c41a1ecf07c13c67bc0ed61045dfd