From patchwork Fri Jul 28 15:44:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Janusz Krzysztofik X-Patchwork-Id: 127755 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:918b:0:b0:3e4:2afc:c1 with SMTP id s11csp572632vqg; Fri, 28 Jul 2023 10:04:49 -0700 (PDT) X-Google-Smtp-Source: APBJJlGi2770ER5jcSVqWEGrHu5djl6R2xmo8/K6hRKVMQnJAl/iWU+kMt48veC45YYPtPwRz0Bh X-Received: by 2002:a17:902:a510:b0:1bb:df62:777f with SMTP id s16-20020a170902a51000b001bbdf62777fmr1490136plq.26.1690563889428; Fri, 28 Jul 2023 10:04:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690563889; cv=none; d=google.com; s=arc-20160816; b=UBf/CJ6nOeBeW/vctUflOKaW+DDOj1rQSqlkz0+ABlJRS7aMwyHccUOEDXmDbi8QL4 hXkALTxVSxrs2RpcKEfGTNFfXk1RGjmIXofqEG8Mf7OoMQGhqd6rUkSVYixioO7/4UKX VWt9DGQutWguGv2XfkEXMK381BnPEqgUZj5ZXgj7NaQ7V8S1z+NWgELL0yvM4d/e3K/c rEMDu+j6gF+RXSUsxtHoicwaYJh6nEhAEKN2Pk6eQ68IRRuxUvX27Nmb8/ObLKEpGk0o Z1H2esOzv0FwVjXQkpAhopsTdbFQBzgeamPV//0nKBF2zK8CxIS5FI5sAMGZddCR/93q 5irg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=molosIvnLrvnltEaGVKzvi1qIz4BKcQV3nndhcgulbY=; fh=j6L+uVtzJZqXE2DwNxEp+xVr3d5QX+4t1Wa1bWrjFSE=; b=vNpsjtbuaSEAWroU3auGFQe43btOwFUWyrEzO4PN5ew7ggfgQZBNhBoVFNUEJn7xn1 mBmPSneGGoJ55zZUgaPMNkFtkazAW+gJRao5aDht6cMMqIJvb0zYvk1ygmwbNdko6teN kEtZZouexKviqLhmaUdg3Z0/t5X8nbe+DPjmstinNoF+14iz4jfhcEnmP4sMvAH4uo+/ nVNPTfiozasLOzVOw8KmnlEYzrr45WIxeVEfmTGifKYENFmgV+gyagqlxZh4LbzJJmEO VA1CJ7tGZhBJWgF94zvulKP2v7wB8tszwoVj9w/Jwmy2j3u4o65IvcXanFgvw39np07y lA8Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Tsoj3rOw; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id mn16-20020a17090b189000b00267f01101fesi3409851pjb.36.2023.07.28.10.04.35; Fri, 28 Jul 2023 10:04:49 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Tsoj3rOw; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237443AbjG1Ppf (ORCPT + 99 others); Fri, 28 Jul 2023 11:45:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34412 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236933AbjG1Ppc (ORCPT ); Fri, 28 Jul 2023 11:45:32 -0400 Received: from mgamail.intel.com (unknown [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 74613212B; Fri, 28 Jul 2023 08:45:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690559131; x=1722095131; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=xfWKTT2Wn3/IWe//m/LpBpFN0F6nm+3nkhbl/ibfElU=; b=Tsoj3rOw/uHMzjrKviGMAIs7AmcdV2ajQOvw3iAtuwzR7Yv2uVHWC62f XoYV++sPYQhx2kpO/KKfoILs2VlEl7oczdWUxVA6vKsKMHCFXd0Rnd5Bl fQUQ5tEL2WcOryBhaQZJjwvWQu9kEtmWY+l5RbCLS71krNpegvAUU5Twc eJztowFBfwfUyncwrcXqCpXw2U3piQXzgS8TJAdXM8J/jNHyv3EIO8ojY 0AqyumMwsDAGHH6Wsi2Mbi0V1MJEi5FlEkCGor21MpZBhsUWTUuW2VSRh aAwbLdjWYFahuUsTAY/HiM9rP2F170GvdGouMzGkF4cOhBm+V9pO6r1xV g==; X-IronPort-AV: E=McAfee;i="6600,9927,10784"; a="371323344" X-IronPort-AV: E=Sophos;i="6.01,237,1684825200"; d="scan'208";a="371323344" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Jul 2023 08:45:31 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10784"; a="817557440" X-IronPort-AV: E=Sophos;i="6.01,237,1684825200"; d="scan'208";a="817557440" Received: from jkrzyszt-mobl2.ger.corp.intel.com (HELO jkrzyszt-mobl2.lan) ([10.213.24.93]) by fmsmga003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Jul 2023 08:45:28 -0700 From: Janusz Krzysztofik To: Brendan Higgins , David Gow Cc: linux-kselftest@vger.kernel.org, kunit-dev@googlegroups.com, Mauro Carvalho Chehab , igt-dev@lists.freedesktop.org, intel-xe@lists.freedesktop.org, linux-kernel@vger.kernel.org Subject: [PATCH 1/3] kunit: Report the count of test suites in a module Date: Fri, 28 Jul 2023 17:44:21 +0200 Message-ID: <20230728154419.1810177-6-janusz.krzysztofik@linux.intel.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230728154419.1810177-5-janusz.krzysztofik@linux.intel.com> References: <20230728154419.1810177-5-janusz.krzysztofik@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1772684720898359081 X-GMAIL-MSGID: 1772684720898359081 According to KTAP specification[1], results should always start from a header that provides a TAP protocol version, followed by a test plan with a count of items to be executed. That pattern should be followed at each nesting level. In the current implementation of the top-most, i.e., test suite level, those rules apply only for test suites built into the kernel, executed and reported on boot. Results submitted to dmesg from kunit test modules loaded later are missing those top-level headers. As a consequence, if a kunit test module provides more than one test suite then, without the top level test plan, external tools that are parsing dmesg for kunit test output are not able to tell how many test suites should be expected and whether to continue parsing after complete output from the first test suite is collected. Submit the top-level headers also from the kunit test module notifier initialization callback. [1] https://docs.kernel.org/dev-tools/ktap.html# Signed-off-by: Janusz Krzysztofik --- lib/kunit/test.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/lib/kunit/test.c b/lib/kunit/test.c index 84e4666555c94..a29ca1acc4d81 100644 --- a/lib/kunit/test.c +++ b/lib/kunit/test.c @@ -729,6 +729,11 @@ EXPORT_SYMBOL_GPL(__kunit_test_suites_exit); #ifdef CONFIG_MODULES static void kunit_module_init(struct module *mod) { + if (mod->num_kunit_suites > 0) { + pr_info("KTAP version 1\n"); + pr_info("1..%d\n", mod->num_kunit_suites); + } + __kunit_test_suites_init(mod->kunit_suites, mod->num_kunit_suites); } From patchwork Fri Jul 28 15:44:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Janusz Krzysztofik X-Patchwork-Id: 127792 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:918b:0:b0:3e4:2afc:c1 with SMTP id s11csp591882vqg; Fri, 28 Jul 2023 10:37:07 -0700 (PDT) X-Google-Smtp-Source: APBJJlGSmftAgjP7Pf2lhv+Ley6QKLCQ+J2JgL24kvaHdupccwgx2NziLlBaSsPizNd7/lQPToAT X-Received: by 2002:a17:903:2654:b0:1b8:af7f:6331 with SMTP id je20-20020a170903265400b001b8af7f6331mr1965872plb.55.1690565827489; Fri, 28 Jul 2023 10:37:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690565827; cv=none; d=google.com; s=arc-20160816; b=pFnIPt17NHbK6ETmR52otI604fJ2yOh/dYdfJZ8/aWc7WF+ENsK+Cl3RKuWlS1gp25 dpHK3LDHXrGvLoRcAM5+VrbSbUco2rOxNe1oROabWpQQ8lNIY874P3uMgHhoqFAVtTF5 SzAz4M6D0pIcYdh0xupym/mcpkRZm3SLqKoHr7Ibav4QLvVm66fguB/Y/wHoHoeK2M99 qbBsnD5aVjWhTTnveDUkUh5Ku/M98nBn0SxQYkq1DZ8KuNXs04a1zfxnARTL2CPdVVeR ldJOeFpmA//aAfotMHxToEsuZGRrL26IteRLsXyJJwBnLUDzP3miq1t9tTu9t7uMmXUN itQQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=o6SPpnuYp5TdvUsII1iUiGuFXOwsgTone2aUnrAWS1Q=; fh=j6L+uVtzJZqXE2DwNxEp+xVr3d5QX+4t1Wa1bWrjFSE=; b=sg6ZPnktvSQD2DFrRA+y7bQ/2/eJM1lsZ8zE2YW5M0lCXiGBFfEVAWXAHVmAjPCB4w P2gbBVvLzzZrAU5g7FHNaF51pn4pClo/FZA+orqprEItf/V0O4C5XSJpQkQK7YcOMF0A uCz7kmhoI8R026jQbJzPHSlpMQpf+nEuWcljjXYo1zLUBJ+lmozNUsiuDvw3XFvxCynR ARPbj7/OiOkA1O1mv/le62swecgiRSts9bI/UV+Ht0cvhNtMONPUO6ZNxQ2eBjSskAuY ZK9HsWCAcEEoJD9r27s/oxicdM0IuFVokzDwa97kJr3dAK6dX1dwPSSwfTAE6phQHpQQ e7dg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=XLyPqrvj; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id ie7-20020a17090b400700b0026359a2e980si4845208pjb.21.2023.07.28.10.36.54; Fri, 28 Jul 2023 10:37:07 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=XLyPqrvj; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237460AbjG1Ppl (ORCPT + 99 others); Fri, 28 Jul 2023 11:45:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34418 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237442AbjG1Ppf (ORCPT ); Fri, 28 Jul 2023 11:45:35 -0400 Received: from mgamail.intel.com (unknown [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 647FBE0; Fri, 28 Jul 2023 08:45:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690559134; x=1722095134; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=C6julir46vLWT1EqiSA60X1VgzepuKjpvOpc4UuLp2Q=; b=XLyPqrvjx6vDB4BJi0BkpZPlR5vhYevY8hWRPxa7lYrQJr8g2WoV0UFd 0Wog5V1yWfQzZQ50ybH0JUHMlVZA3O6JcsofBCdehFO/gcd1X7KXqYG3Q Ke4eJdcnMCC2k8JG2QE1ScAfOrioNvKkKnyfyIebUXRo0m/Z2mDMBYv6t 2kVT6mU3SoEho2SMzLCzkwqfaKKqj+KXi2Tm0+bZz33iMHO/QBEQbTnQk NoRndbJkkDGTv7l8QxDer+wa/eElQ0J1ABootGzPD355q84qbcERQYSp1 BZRXbdfkXuE+U039ZkR7I/K26iNuFVRSKZJ3N9UcrnjrHZl6J5OV3NOJa Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10784"; a="371323398" X-IronPort-AV: E=Sophos;i="6.01,237,1684825200"; d="scan'208";a="371323398" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Jul 2023 08:45:33 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10784"; a="817557442" X-IronPort-AV: E=Sophos;i="6.01,237,1684825200"; d="scan'208";a="817557442" Received: from jkrzyszt-mobl2.ger.corp.intel.com (HELO jkrzyszt-mobl2.lan) ([10.213.24.93]) by fmsmga003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Jul 2023 08:45:31 -0700 From: Janusz Krzysztofik To: Brendan Higgins , David Gow Cc: linux-kselftest@vger.kernel.org, kunit-dev@googlegroups.com, Mauro Carvalho Chehab , igt-dev@lists.freedesktop.org, intel-xe@lists.freedesktop.org, linux-kernel@vger.kernel.org Subject: [PATCH 2/3] kunit: Make 'list' action available to kunit test modules Date: Fri, 28 Jul 2023 17:44:22 +0200 Message-ID: <20230728154419.1810177-7-janusz.krzysztofik@linux.intel.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230728154419.1810177-5-janusz.krzysztofik@linux.intel.com> References: <20230728154419.1810177-5-janusz.krzysztofik@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1772686752756013530 X-GMAIL-MSGID: 1772686752756013530 Results from kunit tests reported via dmesg may be interleaved with other kernel messages. When parsing dmesg for modular kunit results in real time, external tools, e.g., Intel GPU tools (IGT), may want to insert their own test name markers into dmesg at the start of each test, before any kernel message related to that test appears there, so existing upper level test result parsers have no doubt which test to blame for a specific kernel message. Unfortunately, kunit reports names of tests only at their completion (with the exeption of a not standarized "# Subtest: " header above a test plan of each test suite or parametrized test). External tools could be able to insert their own "start of the test" markers with test names included if they new those names in advance. Test names could be learned from a list if provided by a kunit test module. There exists a feature of listing kunit tests without actually executing them, but it is now limited to configurations with the kunit module built in and covers only built-in tests, already available at boot time. Moreover, switching from list to normal mode requires reboot. If that feature was also available when kunit is built as a module, userspace could load the module with action=list parameter, load some kunit test modules they are interested in and learn about the list of tests provided by those modules, then unload them, reload the kunit module in normal mode and execute the tests with their lists already known. Extend kunit module notifier initialization callback with a processing path for only listing the tests provided by a module if the kunit action parameter is set to "list". For ease of use, submit the list in the format of a standard KTAP report, with SKIP result from each test case, giving "list mode" as the reason for skipping. For each test suite provided by a kunit test module, make such list of its test cases also available via kunit debugfs for the lifetime of the module. For user convenience, make the kunit.action parameter visible in sysfs. Signed-off-by: Janusz Krzysztofik --- include/kunit/test.h | 1 + lib/kunit/executor.c | 19 +++++++++++++------ lib/kunit/test.c | 30 +++++++++++++++++++++++++++++- 3 files changed, 43 insertions(+), 7 deletions(-) diff --git a/include/kunit/test.h b/include/kunit/test.h index 23120d50499ef..6d693f21a4833 100644 --- a/include/kunit/test.h +++ b/include/kunit/test.h @@ -237,6 +237,7 @@ static inline void kunit_set_failure(struct kunit *test) } bool kunit_enabled(void); +const char *kunit_action(void); void kunit_init_test(struct kunit *test, const char *name, char *log); diff --git a/lib/kunit/executor.c b/lib/kunit/executor.c index 74982b83707ca..d1c0616569dfd 100644 --- a/lib/kunit/executor.c +++ b/lib/kunit/executor.c @@ -12,19 +12,26 @@ extern struct kunit_suite * const __kunit_suites_start[]; extern struct kunit_suite * const __kunit_suites_end[]; +static char *action_param; + +module_param_named(action, action_param, charp, 0400); +MODULE_PARM_DESC(action, + "Changes KUnit executor behavior, valid values are:\n" + ": run the tests like normal\n" + "'list' to list test names instead of running them.\n"); + +const char *kunit_action(void) +{ + return action_param; +} + #if IS_BUILTIN(CONFIG_KUNIT) static char *filter_glob_param; -static char *action_param; module_param_named(filter_glob, filter_glob_param, charp, 0); MODULE_PARM_DESC(filter_glob, "Filter which KUnit test suites/tests run at boot-time, e.g. list* or list*.*del_test"); -module_param_named(action, action_param, charp, 0); -MODULE_PARM_DESC(action, - "Changes KUnit executor behavior, valid values are:\n" - ": run the tests like normal\n" - "'list' to list test names instead of running them.\n"); /* glob_match() needs NULL terminated strings, so we need a copy of filter_glob_param. */ struct kunit_test_filter { diff --git a/lib/kunit/test.c b/lib/kunit/test.c index a29ca1acc4d81..413d9fd364a8d 100644 --- a/lib/kunit/test.c +++ b/lib/kunit/test.c @@ -674,6 +674,27 @@ int kunit_run_tests(struct kunit_suite *suite) } EXPORT_SYMBOL_GPL(kunit_run_tests); +static void kunit_list_suite(struct kunit_suite *suite) +{ + struct kunit_case *test_case; + + kunit_print_suite_start(suite); + + kunit_suite_for_each_test_case(suite, test_case) { + struct kunit test = { .param_value = NULL, .param_index = 0 }; + + kunit_init_test(&test, test_case->name, test_case->log); + + kunit_print_ok_not_ok(&test, true, KUNIT_SKIPPED, + kunit_test_case_num(suite, test_case), + test_case->name, "list mode"); + } + + kunit_print_ok_not_ok((void *)suite, false, KUNIT_SKIPPED, + kunit_suite_counter++, + suite->name, "list mode"); +} + static void kunit_init_suite(struct kunit_suite *suite) { kunit_debugfs_create_suite(suite); @@ -688,6 +709,7 @@ bool kunit_enabled(void) int __kunit_test_suites_init(struct kunit_suite * const * const suites, int num_suites) { + const char *action = kunit_action(); unsigned int i; if (!kunit_enabled() && num_suites > 0) { @@ -699,7 +721,13 @@ int __kunit_test_suites_init(struct kunit_suite * const * const suites, int num_ for (i = 0; i < num_suites; i++) { kunit_init_suite(suites[i]); - kunit_run_tests(suites[i]); + + if (!action) + kunit_run_tests(suites[i]); + else if (!strcmp(action, "list")) + kunit_list_suite(suites[i]); + else + pr_err("kunit: unknown action '%s'\n", action); } static_branch_dec(&kunit_running); From patchwork Fri Jul 28 15:44:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Janusz Krzysztofik X-Patchwork-Id: 127724 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:918b:0:b0:3e4:2afc:c1 with SMTP id s11csp529702vqg; Fri, 28 Jul 2023 08:57:59 -0700 (PDT) X-Google-Smtp-Source: APBJJlEMFOrurbYIXPysFdxxaQRVSQEp6K15VkHtSITepkT4qeKFMDvygvCsh0ub/7eqKukZ/MyT X-Received: by 2002:a05:6808:1701:b0:3a3:ed41:97d with SMTP id bc1-20020a056808170100b003a3ed41097dmr3679421oib.50.1690559879292; Fri, 28 Jul 2023 08:57:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690559879; cv=none; d=google.com; s=arc-20160816; b=mnWP/BBMQn9dQ7R0puaVT33IU9W8G1du6vPclb97BoFbmg0K2YTBZ9DMQ16GGeCfke a35t9PkRgZSUKoiwlm7+8sTOQqv5ZFvchJQMKcRhn+LsB9pbx2lMQRePXOgYXhj8ZMAf pA5MoBy+bEGCTrF+RKoWhLnpYrKaYFG+X2lJFjV8nB6Ke/GoDD6cF+Ox8czj0KCsLQP8 WkVuHI5+Ck/0chMa5Qs8Dr9w8itNpsXK581P3Y3sARurQ9vFggw6YIWb2YM5bn6lO2uL JOw2TR3JIesdX43EaE8dDobK5C2Fw1VCMBKDi5WT0k2X2CU6BtYWwygyEvmMKelv4Qq2 uN2A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=2/0gjSbdeVe7Y3sTNX4kvBnY1OBBEQUxsyY1FPT9VEw=; fh=j6L+uVtzJZqXE2DwNxEp+xVr3d5QX+4t1Wa1bWrjFSE=; b=v1F196L4kgSy4UmNZZ3yOEgSA0Rzb/lyA8lnauk0X56AJLVraqVca6WkShyjukyjC/ mm54Edo44qxzDJaLG6GDHJBnrsKWLFUw5RpWRcebHvKu22lcHf6OGCLv5nCOwqPD5tbH BCX2Hf0S4vdKIMYb06AHq6duyLFVIlH526xgUvwwdGUxvNbllVb2VJYvvMCcCU5OoVBD LjjYAvLDGCqibDNlvgmHP0xTkI8YBlQlO7NtWGyJ4G0+hQa3egTpj4VPPyG6GnyX5cKV pWyOXOua73FHSo7gb3jEQo19h1gSHgy4XZHCLEzkFeFZ+kdcjDpk+aGowPGwma3I2ZJt mHRQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=VPvkd9aV; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id n127-20020a632785000000b00563e6fb302esi3356023pgn.735.2023.07.28.08.57.46; Fri, 28 Jul 2023 08:57:59 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=VPvkd9aV; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237474AbjG1Ppn (ORCPT + 99 others); Fri, 28 Jul 2023 11:45:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34456 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237448AbjG1Ppi (ORCPT ); Fri, 28 Jul 2023 11:45:38 -0400 Received: from mgamail.intel.com (unknown [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CCD4426A0; Fri, 28 Jul 2023 08:45:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690559136; x=1722095136; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ZkxLWuOXNO9qLoZngSzFjUGnT0mXINuDPjbNOs/XV+4=; b=VPvkd9aVIY4z7NJCMkqXlEu6toqRZa47k83RQ19t6vjCf1LLIR8898JL UT3tVjEkB0zarE6dits06cWOuwkD74fTYqWZ+swZ7T5bFq8ZnNZybpy8G FettWoFa4/CzNW22QmF7nAIPbh/+YQonWlBz7/qzghlCqjZ12z6snGWkf gJMbvczsUxd0gRFGbXKO/KGtA26a49Abp8T8+hF01eezm5YjCt6PdhNt4 ofKMg4yZugQmn1KF2aGv/hpstQkjA8k2F0e72KUHQ/NwQQnlGuYwZ+q1s z/IO66x3RhibymztDBlUxvWW1VGA/6pCrU2VrzJzUAdqg+KJvU2jrR0tU w==; X-IronPort-AV: E=McAfee;i="6600,9927,10784"; a="371323424" X-IronPort-AV: E=Sophos;i="6.01,237,1684825200"; d="scan'208";a="371323424" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Jul 2023 08:45:36 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10784"; a="817557443" X-IronPort-AV: E=Sophos;i="6.01,237,1684825200"; d="scan'208";a="817557443" Received: from jkrzyszt-mobl2.ger.corp.intel.com (HELO jkrzyszt-mobl2.lan) ([10.213.24.93]) by fmsmga003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Jul 2023 08:45:33 -0700 From: Janusz Krzysztofik To: Brendan Higgins , David Gow Cc: linux-kselftest@vger.kernel.org, kunit-dev@googlegroups.com, Mauro Carvalho Chehab , igt-dev@lists.freedesktop.org, intel-xe@lists.freedesktop.org, linux-kernel@vger.kernel.org Subject: [PATCH 3/3] kunit: Allow kunit test modules to use test filtering Date: Fri, 28 Jul 2023 17:44:23 +0200 Message-ID: <20230728154419.1810177-8-janusz.krzysztofik@linux.intel.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230728154419.1810177-5-janusz.krzysztofik@linux.intel.com> References: <20230728154419.1810177-5-janusz.krzysztofik@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1772680516201515247 X-GMAIL-MSGID: 1772680516201515247 External tools, e.g., Intel GPU tools (IGT), support execution of individual selftests provided by kernel modules. That could be also applicable to kunit test modules if they provided test filtering. But test filtering is now possible only when kunit code is built into the kernel. Moreover, a filter can be specified only at boot time, then reboot is required each time a different filter is needed. Build the test filtering code also when kunit is configured as a module, expose test filtering functions to other kunit source files, and use them in kunit module notifier callback functions. Userspace can then reload the kunit module with a value of the filter_glob parameter tuned to a specific kunit test module every time it wants to limit the scope of tests executed on that module load. Make the kunit.filter_glob parameter visible in sysfs for user convenience. Signed-off-by: Janusz Krzysztofik --- include/kunit/test.h | 13 +++++++++++++ lib/kunit/executor.c | 36 ++++++++++++++++++------------------ lib/kunit/test.c | 22 ++++++++++++++++++++++ 3 files changed, 53 insertions(+), 18 deletions(-) diff --git a/include/kunit/test.h b/include/kunit/test.h index 6d693f21a4833..14ff12e72252a 100644 --- a/include/kunit/test.h +++ b/include/kunit/test.h @@ -190,6 +190,12 @@ struct kunit_suite { int suite_init_err; }; +/* Stores an array of suites, end points one past the end */ +struct kunit_suite_set { + struct kunit_suite * const *start; + struct kunit_suite * const *end; +}; + /** * struct kunit - represents a running instance of a test. * @@ -238,6 +244,7 @@ static inline void kunit_set_failure(struct kunit *test) bool kunit_enabled(void); const char *kunit_action(void); +const char *kunit_filter_glob(void); void kunit_init_test(struct kunit *test, const char *name, char *log); @@ -248,6 +255,12 @@ size_t kunit_suite_num_test_cases(struct kunit_suite *suite); unsigned int kunit_test_case_num(struct kunit_suite *suite, struct kunit_case *test_case); +struct kunit_suite_set +kunit_filter_suites(const struct kunit_suite_set *suite_set, + const char *filter_glob, + int *err); +void kunit_free_suite_set(struct kunit_suite_set suite_set); + int __kunit_test_suites_init(struct kunit_suite * const * const suites, int num_suites); void __kunit_test_suites_exit(struct kunit_suite **suites, int num_suites); diff --git a/lib/kunit/executor.c b/lib/kunit/executor.c index d1c0616569dfd..816e14f4fcb64 100644 --- a/lib/kunit/executor.c +++ b/lib/kunit/executor.c @@ -25,14 +25,17 @@ const char *kunit_action(void) return action_param; } -#if IS_BUILTIN(CONFIG_KUNIT) - static char *filter_glob_param; -module_param_named(filter_glob, filter_glob_param, charp, 0); +module_param_named(filter_glob, filter_glob_param, charp, 0400); MODULE_PARM_DESC(filter_glob, "Filter which KUnit test suites/tests run at boot-time, e.g. list* or list*.*del_test"); +const char *kunit_filter_glob(void) +{ + return filter_glob_param; +} + /* glob_match() needs NULL terminated strings, so we need a copy of filter_glob_param. */ struct kunit_test_filter { char *suite_glob; @@ -96,16 +99,7 @@ kunit_filter_tests(const struct kunit_suite *const suite, const char *test_glob) return copy; } -static char *kunit_shutdown; -core_param(kunit_shutdown, kunit_shutdown, charp, 0644); - -/* Stores an array of suites, end points one past the end */ -struct suite_set { - struct kunit_suite * const *start; - struct kunit_suite * const *end; -}; - -static void kunit_free_suite_set(struct suite_set suite_set) +void kunit_free_suite_set(struct kunit_suite_set suite_set) { struct kunit_suite * const *suites; @@ -114,13 +108,14 @@ static void kunit_free_suite_set(struct suite_set suite_set) kfree(suite_set.start); } -static struct suite_set kunit_filter_suites(const struct suite_set *suite_set, - const char *filter_glob, - int *err) +struct kunit_suite_set +kunit_filter_suites(const struct kunit_suite_set *suite_set, + const char *filter_glob, + int *err) { int i; struct kunit_suite **copy, *filtered_suite; - struct suite_set filtered; + struct kunit_suite_set filtered; struct kunit_test_filter filter; const size_t max = suite_set->end - suite_set->start; @@ -155,6 +150,11 @@ static struct suite_set kunit_filter_suites(const struct suite_set *suite_set, return filtered; } +#if IS_BUILTIN(CONFIG_KUNIT) + +static char *kunit_shutdown; +core_param(kunit_shutdown, kunit_shutdown, charp, 0644); + static void kunit_handle_shutdown(void) { if (!kunit_shutdown) @@ -169,7 +169,7 @@ static void kunit_handle_shutdown(void) } -static void kunit_exec_run_tests(struct suite_set *suite_set) +static void kunit_exec_run_tests(struct kunit_suite_set *suite_set) { size_t num_suites = suite_set->end - suite_set->start; diff --git a/lib/kunit/test.c b/lib/kunit/test.c index 413d9fd364a8d..bfc2f65bd1dae 100644 --- a/lib/kunit/test.c +++ b/lib/kunit/test.c @@ -757,6 +757,22 @@ EXPORT_SYMBOL_GPL(__kunit_test_suites_exit); #ifdef CONFIG_MODULES static void kunit_module_init(struct module *mod) { + struct kunit_suite_set suite_set = { + mod->kunit_suites, mod->kunit_suites + mod->num_kunit_suites, + }; + int err = 0; + + suite_set = kunit_filter_suites(&suite_set, + kunit_filter_glob() ?: "*.*", &err); + if (err) { + pr_err("kunit module: error filtering suites: %d\n", + err); + kfree(suite_set.start); + suite_set.start = NULL; + } + mod->kunit_suites = (struct kunit_suite **)suite_set.start; + mod->num_kunit_suites = suite_set.end - suite_set.start; + if (mod->num_kunit_suites > 0) { pr_info("KTAP version 1\n"); pr_info("1..%d\n", mod->num_kunit_suites); @@ -767,7 +783,13 @@ static void kunit_module_init(struct module *mod) static void kunit_module_exit(struct module *mod) { + struct kunit_suite_set suite_set = { + mod->kunit_suites, mod->kunit_suites + mod->num_kunit_suites, + }; + __kunit_test_suites_exit(mod->kunit_suites, mod->num_kunit_suites); + if (suite_set.start) + kunit_free_suite_set(suite_set); } static int kunit_module_notify(struct notifier_block *nb, unsigned long val,