From patchwork Mon Jul 31 14:10:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Janusz Krzysztofik X-Patchwork-Id: 128760 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:918b:0:b0:3e4:2afc:c1 with SMTP id s11csp2071887vqg; Mon, 31 Jul 2023 08:01:24 -0700 (PDT) X-Google-Smtp-Source: APBJJlFtwdwfWw0P87nW3iAqipNyHe3gjeVfHwARCIYpF+IGk3crLSNWi+laJ4aQcVeYdJQxBbWj X-Received: by 2002:a05:6a20:a104:b0:118:e70:6f7d with SMTP id q4-20020a056a20a10400b001180e706f7dmr10601846pzk.10.1690815684285; Mon, 31 Jul 2023 08:01:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690815684; cv=none; d=google.com; s=arc-20160816; b=hL8gzc7SNAI/fqhxzvIMmB5ATLDuq2G2X5Jgvz5Y4RbHqWhnFGJ2uE8B4vKX9l7FG5 92b75PjDwXdyC9OUczI9f0x+dMi1fK+/VRu+HknQO7oywRu1Czn2B1kIJ9xoTqrsw9C0 xiUswhANog7CabylBmJHG8IL7QI+ZJsLjCMoDpJBRztlGtFTVjwd5wVknLvqp4x/Q1g4 wRe3YQJxyS2rFsxgraDI6cY46PVpm5cx8Vg+LqOAuk4Dz+u27E6499BRhohPYnnJBU86 EzP/AD0PR6WAvQeOE1hTEwytrjG+q87GWxuy1krXJd2LKGMkashotoUr0dPwS/LTarZA gVhw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=molosIvnLrvnltEaGVKzvi1qIz4BKcQV3nndhcgulbY=; fh=2DascdEigk5YKkI7xsPhzXnEwnq6+gHGf5jSQyJPeBg=; b=Hk6D9uoSYh9OeqMTgvT2HDrAzVV0MipujYZzqqHZahzKr4P168TfwgG0OidulSG/I+ B9Vk1jdofsYuMCOt6NTaUjVxMpOGVHPjEOeoN/tEpp8X0aPvNb9ro+s7syqOD0lp1OeQ vVUscCFagINoZoPkmLGJtM8EU4etISzG4XAlW1PVqoPYu9YdI9WQEdUgtfEwqklxCsH/ TOQmhMcUThj5o/kodSpCXN8956PA2W1RHrUoiSpuYuJQ3PYJHrR48HmfevQvwxlQNFZ9 MR7NNHiZCwNl7Gstkou1xRE2F5+rTn0H1GJOwKOpzVx9CSpmL+43zOjp0W4t5Czuj+rg wB7g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=D7RoLEGS; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id i5-20020a636d05000000b005538c82b70fsi7356977pgc.166.2023.07.31.08.01.08; Mon, 31 Jul 2023 08:01:24 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=D7RoLEGS; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231773AbjGaONH (ORCPT + 99 others); Mon, 31 Jul 2023 10:13:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60112 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231867AbjGaOMr (ORCPT ); Mon, 31 Jul 2023 10:12:47 -0400 Received: from mgamail.intel.com (unknown [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 71F8AD8; Mon, 31 Jul 2023 07:12:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690812764; x=1722348764; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=xfWKTT2Wn3/IWe//m/LpBpFN0F6nm+3nkhbl/ibfElU=; b=D7RoLEGSSRXBJYSsWpoOe1BhF5oPVLCNW1iv8RkHovivM1L2XEWwT7J1 d67jcxsUMgYYZjJarVfsgD0cgFeA5VpPnNMSIfn16IGQd/SaCORIiWkAP ketO89XEp/tZy+ACCLfvti754Nt1YTVgS7C/rGcqNjhxXjMRjerTPPtlL dAJA6VK9Oi21pV671DsNsPAXJhAF7yAZge8pp+kJlp17Dc1OhlFA+3AsI JEqdGvdRBsc7JBJgvIG3AJ43A8gP8lFnHMcEH6Gpe+EKZxTggnf3QVyZp Kfla1iFrZfVbxNhtFEKTMWptSKG1fz7MyoNFrXrYK8wmzf9NB6QDkcGom A==; X-IronPort-AV: E=McAfee;i="6600,9927,10788"; a="455403626" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="455403626" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2023 07:12:43 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10788"; a="728324347" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="728324347" Received: from jkrzyszt-mobl2.ger.corp.intel.com (HELO jkrzyszt-mobl2.intranet) ([10.213.1.128]) by orsmga002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2023 07:12:41 -0700 From: Janusz Krzysztofik To: Brendan Higgins , David Gow Cc: linux-kselftest@vger.kernel.org, kunit-dev@googlegroups.com, Mauro Carvalho Chehab , igt-dev@lists.freedesktop.org, intel-xe@lists.freedesktop.org, linux-kernel@vger.kernel.org, Janusz Krzysztofik Subject: [PATCH v3 1/3] kunit: Report the count of test suites in a module Date: Mon, 31 Jul 2023 16:10:23 +0200 Message-ID: <20230731141021.2854827-6-janusz.krzysztofik@linux.intel.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230731141021.2854827-5-janusz.krzysztofik@linux.intel.com> References: <20230731141021.2854827-5-janusz.krzysztofik@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1772948747209634542 X-GMAIL-MSGID: 1772948747209634542 According to KTAP specification[1], results should always start from a header that provides a TAP protocol version, followed by a test plan with a count of items to be executed. That pattern should be followed at each nesting level. In the current implementation of the top-most, i.e., test suite level, those rules apply only for test suites built into the kernel, executed and reported on boot. Results submitted to dmesg from kunit test modules loaded later are missing those top-level headers. As a consequence, if a kunit test module provides more than one test suite then, without the top level test plan, external tools that are parsing dmesg for kunit test output are not able to tell how many test suites should be expected and whether to continue parsing after complete output from the first test suite is collected. Submit the top-level headers also from the kunit test module notifier initialization callback. [1] https://docs.kernel.org/dev-tools/ktap.html# Signed-off-by: Janusz Krzysztofik Tested-by: Rae Moar --- lib/kunit/test.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/lib/kunit/test.c b/lib/kunit/test.c index 84e4666555c94..a29ca1acc4d81 100644 --- a/lib/kunit/test.c +++ b/lib/kunit/test.c @@ -729,6 +729,11 @@ EXPORT_SYMBOL_GPL(__kunit_test_suites_exit); #ifdef CONFIG_MODULES static void kunit_module_init(struct module *mod) { + if (mod->num_kunit_suites > 0) { + pr_info("KTAP version 1\n"); + pr_info("1..%d\n", mod->num_kunit_suites); + } + __kunit_test_suites_init(mod->kunit_suites, mod->num_kunit_suites); } From patchwork Mon Jul 31 14:10:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Janusz Krzysztofik X-Patchwork-Id: 128766 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:918b:0:b0:3e4:2afc:c1 with SMTP id s11csp2082815vqg; Mon, 31 Jul 2023 08:15:49 -0700 (PDT) X-Google-Smtp-Source: APBJJlHW+ZPDwlqR4GGaOmEtJSGa0ff1i1Gimv7jiJHocKRJblqpuV+LLnaqjOXGa1Pa5CW6zsR+ X-Received: by 2002:a17:902:b58b:b0:1b6:a99b:ceb1 with SMTP id a11-20020a170902b58b00b001b6a99bceb1mr8439867pls.50.1690816549308; Mon, 31 Jul 2023 08:15:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690816549; cv=none; d=google.com; s=arc-20160816; b=Ef6WeleN9dglfyC1P0aOm4JGkT6Gz0f5Au7GZVSFo9Vs5UAD9aRoWxf1voKiCvTiUL 5QQbo/uKUWveya9+uDG1HEZVb9edowEzbsID/By62iFNjGWZd+81l8/6zREE7ipWBnXc DcrOAjJfrkbpEF62iZFOOtpI6vrg2GVhUh36XFXCgoB4SU185V9IwD6vTqlsxDmdkTa6 e5rTYKvpqgWkVjcZLM+nnOy+dc/XusBzN1N07mo/iQI+bL+kfN006Qy/Rew4u8UyMPG4 FxftYIfuqdhHcGBzppnY0TPHEZ/Bsx+0N94UeQzYBhRMHlY3+L/LThLnSRMcftuAuwxi LrPA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=o6SPpnuYp5TdvUsII1iUiGuFXOwsgTone2aUnrAWS1Q=; fh=2DascdEigk5YKkI7xsPhzXnEwnq6+gHGf5jSQyJPeBg=; b=ybD0gx55xzRLujTA6uUdXivBjrt7A1L5NhvMxEzjaGZ9wBDeX8F8Ex89edDbHbk0LD txqb1VB/z5pmTX86T7wJ9ckUG98BqJbcLK/0t2d0NhAWWWP1B5SBov81YbaYmPz9k5k0 Noz4h0TuzGC3skaNlSs+GHWXBr/SDrnwdjTImIs2yZrD2ZlmNr6idG4BHHgqHCkmt+B9 pjMg28bjFdy0URC7KD20o0DLFDYgLWaeAJZz/mlQBxwv03kE+/XezWxngTc2vGB8QH2t ApmGEOyoxgCmLUNVBdt4YSsv3ZOoMuJ42SbI1UxwrSaQJl28bU98LEjFiRt5fr6ZQ+Yq WIdQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=PfNsLwHQ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id r11-20020a1709028bcb00b001bba5626215si7142577plo.333.2023.07.31.08.15.35; Mon, 31 Jul 2023 08:15:49 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=PfNsLwHQ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232079AbjGaONM (ORCPT + 99 others); Mon, 31 Jul 2023 10:13:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60018 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231793AbjGaOMt (ORCPT ); Mon, 31 Jul 2023 10:12:49 -0400 Received: from mgamail.intel.com (unknown [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1F1AFB8; Mon, 31 Jul 2023 07:12:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690812767; x=1722348767; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=C6julir46vLWT1EqiSA60X1VgzepuKjpvOpc4UuLp2Q=; b=PfNsLwHQeh1rqjMmQpEztkJUdzZym0ymEL+BO4uZUNNnZqpRzuWuEw5F TrHAL+HfDzuTiDICRjiPzAQsN4GwlhUi/pFFrPnfwNMt+3M1wSKOgspFi DP8BgahMiCSlIe8t1zCQufuxNFoWMYVXX4Ca/LJJecppOcxrLh4ZNtwga fq5lDe7iUnJzNwfCFrpUO06w64CheWOlviwc6B4jgHaQLucytUp9lFOkc 1AyP53XlhAoqJ3MwRwP1AgTBc0NGDQgV0Sahf2aE5TAiZ3Ywa/KWP/nZ1 WQqlqI/Y9f6dCS7Z3cD4LT/bExFmLw3/zjM1E8WjFW1K04EvAWtw5YnON g==; X-IronPort-AV: E=McAfee;i="6600,9927,10788"; a="455403643" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="455403643" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2023 07:12:46 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10788"; a="728324355" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="728324355" Received: from jkrzyszt-mobl2.ger.corp.intel.com (HELO jkrzyszt-mobl2.intranet) ([10.213.1.128]) by orsmga002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2023 07:12:44 -0700 From: Janusz Krzysztofik To: Brendan Higgins , David Gow Cc: linux-kselftest@vger.kernel.org, kunit-dev@googlegroups.com, Mauro Carvalho Chehab , igt-dev@lists.freedesktop.org, intel-xe@lists.freedesktop.org, linux-kernel@vger.kernel.org, Janusz Krzysztofik Subject: [PATCH v3 2/3] kunit: Make 'list' action available to kunit test modules Date: Mon, 31 Jul 2023 16:10:24 +0200 Message-ID: <20230731141021.2854827-7-janusz.krzysztofik@linux.intel.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230731141021.2854827-5-janusz.krzysztofik@linux.intel.com> References: <20230731141021.2854827-5-janusz.krzysztofik@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1772949653712182956 X-GMAIL-MSGID: 1772949653712182956 Results from kunit tests reported via dmesg may be interleaved with other kernel messages. When parsing dmesg for modular kunit results in real time, external tools, e.g., Intel GPU tools (IGT), may want to insert their own test name markers into dmesg at the start of each test, before any kernel message related to that test appears there, so existing upper level test result parsers have no doubt which test to blame for a specific kernel message. Unfortunately, kunit reports names of tests only at their completion (with the exeption of a not standarized "# Subtest: " header above a test plan of each test suite or parametrized test). External tools could be able to insert their own "start of the test" markers with test names included if they new those names in advance. Test names could be learned from a list if provided by a kunit test module. There exists a feature of listing kunit tests without actually executing them, but it is now limited to configurations with the kunit module built in and covers only built-in tests, already available at boot time. Moreover, switching from list to normal mode requires reboot. If that feature was also available when kunit is built as a module, userspace could load the module with action=list parameter, load some kunit test modules they are interested in and learn about the list of tests provided by those modules, then unload them, reload the kunit module in normal mode and execute the tests with their lists already known. Extend kunit module notifier initialization callback with a processing path for only listing the tests provided by a module if the kunit action parameter is set to "list". For ease of use, submit the list in the format of a standard KTAP report, with SKIP result from each test case, giving "list mode" as the reason for skipping. For each test suite provided by a kunit test module, make such list of its test cases also available via kunit debugfs for the lifetime of the module. For user convenience, make the kunit.action parameter visible in sysfs. Signed-off-by: Janusz Krzysztofik --- include/kunit/test.h | 1 + lib/kunit/executor.c | 19 +++++++++++++------ lib/kunit/test.c | 30 +++++++++++++++++++++++++++++- 3 files changed, 43 insertions(+), 7 deletions(-) diff --git a/include/kunit/test.h b/include/kunit/test.h index 23120d50499ef..6d693f21a4833 100644 --- a/include/kunit/test.h +++ b/include/kunit/test.h @@ -237,6 +237,7 @@ static inline void kunit_set_failure(struct kunit *test) } bool kunit_enabled(void); +const char *kunit_action(void); void kunit_init_test(struct kunit *test, const char *name, char *log); diff --git a/lib/kunit/executor.c b/lib/kunit/executor.c index 74982b83707ca..d1c0616569dfd 100644 --- a/lib/kunit/executor.c +++ b/lib/kunit/executor.c @@ -12,19 +12,26 @@ extern struct kunit_suite * const __kunit_suites_start[]; extern struct kunit_suite * const __kunit_suites_end[]; +static char *action_param; + +module_param_named(action, action_param, charp, 0400); +MODULE_PARM_DESC(action, + "Changes KUnit executor behavior, valid values are:\n" + ": run the tests like normal\n" + "'list' to list test names instead of running them.\n"); + +const char *kunit_action(void) +{ + return action_param; +} + #if IS_BUILTIN(CONFIG_KUNIT) static char *filter_glob_param; -static char *action_param; module_param_named(filter_glob, filter_glob_param, charp, 0); MODULE_PARM_DESC(filter_glob, "Filter which KUnit test suites/tests run at boot-time, e.g. list* or list*.*del_test"); -module_param_named(action, action_param, charp, 0); -MODULE_PARM_DESC(action, - "Changes KUnit executor behavior, valid values are:\n" - ": run the tests like normal\n" - "'list' to list test names instead of running them.\n"); /* glob_match() needs NULL terminated strings, so we need a copy of filter_glob_param. */ struct kunit_test_filter { diff --git a/lib/kunit/test.c b/lib/kunit/test.c index a29ca1acc4d81..413d9fd364a8d 100644 --- a/lib/kunit/test.c +++ b/lib/kunit/test.c @@ -674,6 +674,27 @@ int kunit_run_tests(struct kunit_suite *suite) } EXPORT_SYMBOL_GPL(kunit_run_tests); +static void kunit_list_suite(struct kunit_suite *suite) +{ + struct kunit_case *test_case; + + kunit_print_suite_start(suite); + + kunit_suite_for_each_test_case(suite, test_case) { + struct kunit test = { .param_value = NULL, .param_index = 0 }; + + kunit_init_test(&test, test_case->name, test_case->log); + + kunit_print_ok_not_ok(&test, true, KUNIT_SKIPPED, + kunit_test_case_num(suite, test_case), + test_case->name, "list mode"); + } + + kunit_print_ok_not_ok((void *)suite, false, KUNIT_SKIPPED, + kunit_suite_counter++, + suite->name, "list mode"); +} + static void kunit_init_suite(struct kunit_suite *suite) { kunit_debugfs_create_suite(suite); @@ -688,6 +709,7 @@ bool kunit_enabled(void) int __kunit_test_suites_init(struct kunit_suite * const * const suites, int num_suites) { + const char *action = kunit_action(); unsigned int i; if (!kunit_enabled() && num_suites > 0) { @@ -699,7 +721,13 @@ int __kunit_test_suites_init(struct kunit_suite * const * const suites, int num_ for (i = 0; i < num_suites; i++) { kunit_init_suite(suites[i]); - kunit_run_tests(suites[i]); + + if (!action) + kunit_run_tests(suites[i]); + else if (!strcmp(action, "list")) + kunit_list_suite(suites[i]); + else + pr_err("kunit: unknown action '%s'\n", action); } static_branch_dec(&kunit_running); From patchwork Mon Jul 31 14:10:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Janusz Krzysztofik X-Patchwork-Id: 128799 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:918b:0:b0:3e4:2afc:c1 with SMTP id s11csp2148700vqg; Mon, 31 Jul 2023 10:09:03 -0700 (PDT) X-Google-Smtp-Source: APBJJlFblYO5I3NDjWK5QHrPq+fVVYdlK/Z4kaBCsZDiCh6SlNB2nZGBhPHS0iIqN0rWzOKKfQG0 X-Received: by 2002:a2e:8455:0:b0:2b9:c005:c1e5 with SMTP id u21-20020a2e8455000000b002b9c005c1e5mr387977ljh.34.1690823343486; Mon, 31 Jul 2023 10:09:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690823343; cv=none; d=google.com; s=arc-20160816; b=myudZpauwm9n49k6szFoCEJhNSlB1UYPCbgMShb9Drm9jMyvqk5im108n0+iROFmle Wz97peZTOuKnfuCq/X0vDg8uOj3Qg9aCKLgZDnQvVofGWo28jZmLn847zSYx/zRQRPXG IGvPivEegQx8lBYH5+A8kzHOYaJrW/YOOJegD+ykQIAGgz7w80qXQ/Gm/88xTJNXwPET oosBKcypGQzDFaxlbDOkhr4lfFp1Pkj9NIlQMOfwnEi3P1R59/qzoef5NLLasKbBoFrA g34Ih+NmbuYMl8OyHIVhegb9SiczB3C+lFf/pkc+xyLPv+NKCNbu71MfAngplZaiU5HB sarQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=WWP12Nwhdm0QraWVfUEQlOOqGKsBIy4XtJVScQA42zc=; fh=2DascdEigk5YKkI7xsPhzXnEwnq6+gHGf5jSQyJPeBg=; b=Hyo0JIF8kSXnUBb5JhPyYEp/j1D9MFpZUJNuG/2O0vP/Sybe3XZ4Vkie9hGr9yeYZ6 jBxDIKvhGZH00DEyGj8tMjZ/meMOR0eDZhFAmC1y310FS8E4r/Bv3CpZdPcFFQGBUi1J ii37IWSLjcaqCF+os8k624N34iQg0f+Kuud8N8AKYnWM9pU9JVUfbxtoXl5xHc0wYYoW UmFbpC/LqsJ0B4e2QeJBdLqN6oAOo+uQG5TJnrg1J2i8R7XXsMfqU3s/BSow7IjcwbZd Yyp+4cdXxIZLaztbeWMEa9uFpbuva5K4R0t/BY6g7ny30+MqOBsQAMSfuM0Vyy1UpXmE E9mA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=L5ykAgag; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id i6-20020a1709063c4600b0098d861f6513si6961129ejg.68.2023.07.31.10.08.37; Mon, 31 Jul 2023 10:09:03 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=L5ykAgag; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231793AbjGaONP (ORCPT + 99 others); Mon, 31 Jul 2023 10:13:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60130 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231679AbjGaOMv (ORCPT ); Mon, 31 Jul 2023 10:12:51 -0400 Received: from mgamail.intel.com (unknown [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 275DEB3; Mon, 31 Jul 2023 07:12:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690812770; x=1722348770; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Jt+Kf1a8TwG/spRuMs893nE/LfcM2cJ0n2VetEP0YVg=; b=L5ykAgagBzQ32N3xNp3RIZgcBk90UFYMga2AyqPoWmUmHg1IwEVSbhW6 G3Ja0AQjn6LpaKL3BeOm6sOkfO18lzfV+PBNIy8eW5ZMc2CBcHZ31S/dE dPXPZE6Em9oAn7lk9KF5ltRm0C2MEK3vwJl5j6BZO1wVJ1Hgjzhub7AN5 R/PyZCDOOywoFZU3BhFIl+R+gcv0bzR+z7WIIKkK2N3GMXWwHOnVvTvZu Zh8oCuHNVRBbu5xiBArXh4RQVJzrT0O99BKPhNMllG7G/ci3qo9OvpbMk 0YO0Zr/EK0ow6h2De1AolQBr9vrKDaKE2qJDD3QDODU7Tt/w5Sxds3H0f w==; X-IronPort-AV: E=McAfee;i="6600,9927,10788"; a="455403669" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="455403669" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2023 07:12:49 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10788"; a="728324359" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="728324359" Received: from jkrzyszt-mobl2.ger.corp.intel.com (HELO jkrzyszt-mobl2.intranet) ([10.213.1.128]) by orsmga002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2023 07:12:46 -0700 From: Janusz Krzysztofik To: Brendan Higgins , David Gow Cc: linux-kselftest@vger.kernel.org, kunit-dev@googlegroups.com, Mauro Carvalho Chehab , igt-dev@lists.freedesktop.org, intel-xe@lists.freedesktop.org, linux-kernel@vger.kernel.org, Janusz Krzysztofik Subject: [PATCH v3 3/3] kunit: Allow kunit test modules to use test filtering Date: Mon, 31 Jul 2023 16:10:25 +0200 Message-ID: <20230731141021.2854827-8-janusz.krzysztofik@linux.intel.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230731141021.2854827-5-janusz.krzysztofik@linux.intel.com> References: <20230731141021.2854827-5-janusz.krzysztofik@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1772956778340707442 X-GMAIL-MSGID: 1772956778340707442 External tools, e.g., Intel GPU tools (IGT), support execution of individual selftests provided by kernel modules. That could be also applicable to kunit test modules if they provided test filtering. But test filtering is now possible only when kunit code is built into the kernel. Moreover, a filter can be specified only at boot time, then reboot is required each time a different filter is needed. Build the test filtering code also when kunit is configured as a module, expose test filtering functions to other kunit source files, and use them in kunit module notifier callback functions. Userspace can then reload the kunit module with a value of the filter_glob parameter tuned to a specific kunit test module every time it wants to limit the scope of tests executed on that module load. Make the kunit.filter_glob parameter visible in sysfs for user convenience. v3: Fix CONFIG_GLOB, required by filtering fuctions, not selected when building as a module. v2: Fix new name of a structure moved to kunit namespace not updated across all uses. Signed-off-by: Janusz Krzysztofik --- include/kunit/test.h | 13 +++++++++++++ lib/kunit/Kconfig | 2 +- lib/kunit/executor.c | 42 ++++++++++++++++++++++-------------------- lib/kunit/test.c | 22 ++++++++++++++++++++++ 4 files changed, 58 insertions(+), 21 deletions(-) diff --git a/include/kunit/test.h b/include/kunit/test.h index 6d693f21a4833..14ff12e72252a 100644 --- a/include/kunit/test.h +++ b/include/kunit/test.h @@ -190,6 +190,12 @@ struct kunit_suite { int suite_init_err; }; +/* Stores an array of suites, end points one past the end */ +struct kunit_suite_set { + struct kunit_suite * const *start; + struct kunit_suite * const *end; +}; + /** * struct kunit - represents a running instance of a test. * @@ -238,6 +244,7 @@ static inline void kunit_set_failure(struct kunit *test) bool kunit_enabled(void); const char *kunit_action(void); +const char *kunit_filter_glob(void); void kunit_init_test(struct kunit *test, const char *name, char *log); @@ -248,6 +255,12 @@ size_t kunit_suite_num_test_cases(struct kunit_suite *suite); unsigned int kunit_test_case_num(struct kunit_suite *suite, struct kunit_case *test_case); +struct kunit_suite_set +kunit_filter_suites(const struct kunit_suite_set *suite_set, + const char *filter_glob, + int *err); +void kunit_free_suite_set(struct kunit_suite_set suite_set); + int __kunit_test_suites_init(struct kunit_suite * const * const suites, int num_suites); void __kunit_test_suites_exit(struct kunit_suite **suites, int num_suites); diff --git a/lib/kunit/Kconfig b/lib/kunit/Kconfig index 626719b95badd..68a6daec0aef1 100644 --- a/lib/kunit/Kconfig +++ b/lib/kunit/Kconfig @@ -4,7 +4,7 @@ menuconfig KUNIT tristate "KUnit - Enable support for unit tests" - select GLOB if KUNIT=y + select GLOB help Enables support for kernel unit tests (KUnit), a lightweight unit testing and mocking framework for the Linux kernel. These tests are diff --git a/lib/kunit/executor.c b/lib/kunit/executor.c index d1c0616569dfd..49fe40cc8f1af 100644 --- a/lib/kunit/executor.c +++ b/lib/kunit/executor.c @@ -25,14 +25,17 @@ const char *kunit_action(void) return action_param; } -#if IS_BUILTIN(CONFIG_KUNIT) - static char *filter_glob_param; -module_param_named(filter_glob, filter_glob_param, charp, 0); +module_param_named(filter_glob, filter_glob_param, charp, 0400); MODULE_PARM_DESC(filter_glob, "Filter which KUnit test suites/tests run at boot-time, e.g. list* or list*.*del_test"); +const char *kunit_filter_glob(void) +{ + return filter_glob_param; +} + /* glob_match() needs NULL terminated strings, so we need a copy of filter_glob_param. */ struct kunit_test_filter { char *suite_glob; @@ -96,16 +99,7 @@ kunit_filter_tests(const struct kunit_suite *const suite, const char *test_glob) return copy; } -static char *kunit_shutdown; -core_param(kunit_shutdown, kunit_shutdown, charp, 0644); - -/* Stores an array of suites, end points one past the end */ -struct suite_set { - struct kunit_suite * const *start; - struct kunit_suite * const *end; -}; - -static void kunit_free_suite_set(struct suite_set suite_set) +void kunit_free_suite_set(struct kunit_suite_set suite_set) { struct kunit_suite * const *suites; @@ -114,13 +108,14 @@ static void kunit_free_suite_set(struct suite_set suite_set) kfree(suite_set.start); } -static struct suite_set kunit_filter_suites(const struct suite_set *suite_set, - const char *filter_glob, - int *err) +struct kunit_suite_set +kunit_filter_suites(const struct kunit_suite_set *suite_set, + const char *filter_glob, + int *err) { int i; struct kunit_suite **copy, *filtered_suite; - struct suite_set filtered; + struct kunit_suite_set filtered; struct kunit_test_filter filter; const size_t max = suite_set->end - suite_set->start; @@ -155,6 +150,11 @@ static struct suite_set kunit_filter_suites(const struct suite_set *suite_set, return filtered; } +#if IS_BUILTIN(CONFIG_KUNIT) + +static char *kunit_shutdown; +core_param(kunit_shutdown, kunit_shutdown, charp, 0644); + static void kunit_handle_shutdown(void) { if (!kunit_shutdown) @@ -169,7 +169,7 @@ static void kunit_handle_shutdown(void) } -static void kunit_exec_run_tests(struct suite_set *suite_set) +static void kunit_exec_run_tests(struct kunit_suite_set *suite_set) { size_t num_suites = suite_set->end - suite_set->start; @@ -179,7 +179,7 @@ static void kunit_exec_run_tests(struct suite_set *suite_set) __kunit_test_suites_init(suite_set->start, num_suites); } -static void kunit_exec_list_tests(struct suite_set *suite_set) +static void kunit_exec_list_tests(struct kunit_suite_set *suite_set) { struct kunit_suite * const *suites; struct kunit_case *test_case; @@ -195,7 +195,9 @@ static void kunit_exec_list_tests(struct suite_set *suite_set) int kunit_run_all_tests(void) { - struct suite_set suite_set = {__kunit_suites_start, __kunit_suites_end}; + struct kunit_suite_set suite_set = { + __kunit_suites_start, __kunit_suites_end, + }; int err = 0; if (!kunit_enabled()) { pr_info("kunit: disabled\n"); diff --git a/lib/kunit/test.c b/lib/kunit/test.c index 413d9fd364a8d..bfc2f65bd1dae 100644 --- a/lib/kunit/test.c +++ b/lib/kunit/test.c @@ -757,6 +757,22 @@ EXPORT_SYMBOL_GPL(__kunit_test_suites_exit); #ifdef CONFIG_MODULES static void kunit_module_init(struct module *mod) { + struct kunit_suite_set suite_set = { + mod->kunit_suites, mod->kunit_suites + mod->num_kunit_suites, + }; + int err = 0; + + suite_set = kunit_filter_suites(&suite_set, + kunit_filter_glob() ?: "*.*", &err); + if (err) { + pr_err("kunit module: error filtering suites: %d\n", + err); + kfree(suite_set.start); + suite_set.start = NULL; + } + mod->kunit_suites = (struct kunit_suite **)suite_set.start; + mod->num_kunit_suites = suite_set.end - suite_set.start; + if (mod->num_kunit_suites > 0) { pr_info("KTAP version 1\n"); pr_info("1..%d\n", mod->num_kunit_suites); @@ -767,7 +783,13 @@ static void kunit_module_init(struct module *mod) static void kunit_module_exit(struct module *mod) { + struct kunit_suite_set suite_set = { + mod->kunit_suites, mod->kunit_suites + mod->num_kunit_suites, + }; + __kunit_test_suites_exit(mod->kunit_suites, mod->num_kunit_suites); + if (suite_set.start) + kunit_free_suite_set(suite_set); } static int kunit_module_notify(struct notifier_block *nb, unsigned long val,