From patchwork Thu Oct 26 08:59:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Ripard X-Patchwork-Id: 158461 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:d641:0:b0:403:3b70:6f57 with SMTP id cy1csp525766vqb; Thu, 26 Oct 2023 02:00:04 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGaz29U2DqvJYRqJCJO0QUJZRfCm5s38x5DvoYP45rcz6CaaL9fuX+vDWok0VGA/zy6hYYA X-Received: by 2002:a25:7602:0:b0:d9a:cd50:b99a with SMTP id r2-20020a257602000000b00d9acd50b99amr18140247ybc.12.1698310804678; Thu, 26 Oct 2023 02:00:04 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1698310804; cv=none; d=google.com; s=arc-20160816; b=0KevEqDEUyUIM4eHeNtf2BbOz/mZV+iXALpcpDRHVQfjPof4NZzxwuvTOQ2sAv4R8r nuZxmzT3b0UlG8zSqGQvXGdAth3IcxptcIsTRq7L4PndH5g5UPRX/RryXvkJ3XEyf4m0 ZTLKuHsI31WuzEmh95fiKPSptL7Pv625RAQ39VehxKvatakyoBy8409cfD7PaEcvWaO+ wdZapwr0EqAhLAVrJDyKSXdl7VMlbpZXEmtMdBz/KaO59jRdQ467u/hD22KI+KGyraZH ZVn6WS4g6OshN3ubkQVXHUWiXSPQ+rmYqPq7oVT/dAfFRRkoPbWkug0tBIH4HOYISDYY W/rg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=yj4a/lR5Ug3ljybIpgIa6/p81453bAqTYlJZiFPxSxg=; fh=TLCuMDloLp2earPKFEtMHjAcQwhiAhbmgvUtXmK/gO0=; b=tzeeNl2KDO5g48iG90AL2Jz5v3AhrxHcv5SB+oRTrOTPP4wnB6+r4tYXbkT2WEoKrc aQQhMzq0HZrmo8n3eHtucTqFdS+mezKRaHcEaNMq9lYjLeIkQshug+JtkVhwq67Lh6vj HyzfThZG/HIN2hLViDag6+MOzHMAqs0gemkxboW28rvwD0i3LwU5z5ToPZxq8auTAxGu u9A0t0MCxNGRHokFZoiNFiL4T0Lzvnip/uN7sXKgsBGeV5viKzPY0HuXSAA4/Yw+1OE1 4aR7ymGxX1bwbIyE+HH4hlDVsR3TFnbaCaesfbGguqbmzH/ZeVGwaOzU9gQbACfOCAH2 M0yg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b="g/65lgSS"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from lipwig.vger.email (lipwig.vger.email. [23.128.96.33]) by mx.google.com with ESMTPS id 23-20020a250d17000000b00da03c0ffe59si6961789ybn.455.2023.10.26.02.00.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Oct 2023 02:00:04 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) client-ip=23.128.96.33; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b="g/65lgSS"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id C368C8244E6E; Thu, 26 Oct 2023 02:00:01 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344406AbjJZI7j (ORCPT + 26 others); Thu, 26 Oct 2023 04:59:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44498 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229639AbjJZI7h (ORCPT ); Thu, 26 Oct 2023 04:59:37 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B3A2E10E; Thu, 26 Oct 2023 01:59:35 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 076DEC433C8; Thu, 26 Oct 2023 08:59:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1698310775; bh=ZxJyjl9iM4hVa2LaauAP/lo/cgON8YMCpqCM6LS+2Pk=; h=From:To:Cc:Subject:Date:From; b=g/65lgSSseLw3teMZhybtUHctnj8GOXAfGp6+CiWmSz6aX1JmvD8musYmiPdJen5c P+HOABcTBrWuuuT6zSFuplvDN4d1aUc+0IPrXv203nI2NGJwrnbzVhoVVikE3N/blF DBMnbhd6YTMIksT9qFh+Tu2kLBsZmJNa++YrfxbO91Lky/9TjrSKwT88bBNcRg0HiI 7QbGTESDQeUZSBics4ERjU9imaoKGy2ETJXtyt6+aeavkY6mDsqtZA2LuEb2XeIjJh KnRvucg7WE8mFBoNe6HjWkX7yZgeAeMhs34l8Ti7AyMvVQY5I8aKsZZv+fXRzDL8uF TKH5x8Px/2bpg== From: Maxime Ripard To: Brendan Higgins , David Gow Cc: Maxime Ripard , Jani Nikula , Rae Moar , linux-kselftest@vger.kernel.org, kunit-dev@googlegroups.com, linux-kernel@vger.kernel.org Subject: [PATCH v3] kunit: Warn if tests are slow Date: Thu, 26 Oct 2023 10:59:31 +0200 Message-ID: <20231026085931.883824-1-mripard@kernel.org> X-Mailer: git-send-email 2.41.0 MIME-Version: 1.0 X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Thu, 26 Oct 2023 02:00:01 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1776783063226660472 X-GMAIL-MSGID: 1780807950250032253 Kunit recently gained support to setup attributes, the first one being the speed of a given test, then allowing to filter out slow tests. A slow test is defined in the documentation as taking more than one second. There's an another speed attribute called "super slow" but whose definition is less clear. Add support to the test runner to check the test execution time, and report tests that should be marked as slow but aren't. Signed-off-by: Maxime Ripard Reviewed-by: David Gow --- To: Brendan Higgins To: David Gow Cc: Jani Nikula Cc: Rae Moar Cc: linux-kselftest@vger.kernel.org Cc: kunit-dev@googlegroups.com Cc: linux-kernel@vger.kernel.org Changes from v2: - Add defines and comments to make the warning reporting threshold more obvious - Switch the duration comparisons to timespec64_compare to be more accurate - Link: https://lore.kernel.org/all/20230920084903.1522728-1-mripard@kernel.org/ Changes from v1: - Split the patch out of the series - Change to trigger the warning only if the runtime is twice the threshold (Jani, Rae) - Split the speed check into a separate function (Rae) - Link: https://lore.kernel.org/all/20230911-kms-slow-tests-v1-0-d3800a69a1a1@kernel.org/ --- lib/kunit/test.c | 38 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/lib/kunit/test.c b/lib/kunit/test.c index 49698a168437..4b710c92340a 100644 --- a/lib/kunit/test.c +++ b/lib/kunit/test.c @@ -372,6 +372,36 @@ void kunit_init_test(struct kunit *test, const char *name, char *log) } EXPORT_SYMBOL_GPL(kunit_init_test); +/* Only warn when a test takes more than twice the threshold */ +#define KUNIT_SPEED_WARNING_MULTIPLIER 2 + +/* Slow tests are defined as taking more than 1s */ +#define KUNIT_SPEED_SLOW_THRESHOLD_S 1 + +#define KUNIT_SPEED_SLOW_WARNING_THRESHOLD_S \ + (KUNIT_SPEED_WARNING_MULTIPLIER * KUNIT_SPEED_SLOW_THRESHOLD_S) + +#define s_to_timespec64(s) ns_to_timespec64((s) * NSEC_PER_SEC) + +static void kunit_run_case_check_speed(struct kunit *test, + struct kunit_case *test_case, + struct timespec64 duration) +{ + struct timespec64 slow_thr = + s_to_timespec64(KUNIT_SPEED_SLOW_WARNING_THRESHOLD_S); + enum kunit_speed speed = test_case->attr.speed; + + if (timespec64_compare(&duration, &slow_thr) < 0) + return; + + if (speed == KUNIT_SPEED_VERY_SLOW || speed == KUNIT_SPEED_SLOW) + return; + + kunit_warn(test, + "Test should be marked slow (runtime: %lld.%09lds)", + duration.tv_sec, duration.tv_nsec); +} + /* * Initializes and runs test case. Does not clean up or do post validations. */ @@ -379,6 +409,8 @@ static void kunit_run_case_internal(struct kunit *test, struct kunit_suite *suite, struct kunit_case *test_case) { + struct timespec64 start, end; + if (suite->init) { int ret; @@ -390,7 +422,13 @@ static void kunit_run_case_internal(struct kunit *test, } } + ktime_get_ts64(&start); + test_case->run_case(test); + + ktime_get_ts64(&end); + + kunit_run_case_check_speed(test, test_case, timespec64_sub(end, start)); } static void kunit_case_internal_cleanup(struct kunit *test)