Message ID | 20221129001234.606653-1-dlatypov@google.com |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp25322wrr; Mon, 28 Nov 2022 16:17:24 -0800 (PST) X-Google-Smtp-Source: AA0mqf6tM1sKiRAsJE+OoZrTd1oct2m3LWKIg8K7ZW/kkdbR6aUyDLo3EX9HoWtWgBjRBY5giCzn X-Received: by 2002:a17:902:9681:b0:189:9d01:929d with SMTP id n1-20020a170902968100b001899d01929dmr150779plp.108.1669681043821; Mon, 28 Nov 2022 16:17:23 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669681043; cv=none; d=google.com; s=arc-20160816; b=gIYDgrj0ypO1aobi0GmCb4xqsS3H0n8fQuEyPWYoFry9kRLkffEk2YcFiD9doSEtVg gu1g5JEc6FrD7zqF9N1YYeVJ6c5CAavlj7TX/UQ+ohBOg1Do5K/ZN8S3UypaidQvZArP f2U3R2UWhhDMwmQOHuadLjlC505fq/Uxj2POHnqSUINh8khyUZRqhiCUm6szmlpKS4he fduhsD6lx1/3pCc9jMqRTwH7BXB6YjPWUwXLR+slNDEhZzGkB9FL3bbfHdPOdsTEiAWA kthYvX8cZO7uXElB3UyDtd9KyJmSvWyGLRLaVb+uNkm6bNOOZ2Bnsy+ESmUCq/uLdFWO nbVQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:mime-version:date :dkim-signature; bh=X881tlPDVXUlMgACZfZTZeyY92tnJLZ+/B1dNTIs8Gc=; b=B/yflMZqrhAsusy6fVnUp/ixK8xEkfNs93HiPU2lOV5PBYuZe1QZkiw8Yh/p4ab1n0 8LnTnG46zeoclOHSACS+iCJPbfRi6EeTfRIJniW5+iE+jnWmtU8dAwADelpl87Tx8K9y C9UQvoVjfyCtzP8BI+xAvMDGzNDQDUppjnsVggNcuIHqZFjIpTa3XcwzU5SBGlTyFdb/ t1xmLLd2V7YNoA6/Mq5wHNvwROhvKh9LHUP8R9PyNO4lTkO+Qg2mAqAQgZjtpLZA93xS hh+qU/EgqHLeTnemzeYvVx8u+EIEmyQoXfdGZT981IL67YJU58XZxe2CRd1yogbUl047 MsyA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=Oi0ZjD5s; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id w188-20020a6282c5000000b005736bd1a8acsi12273286pfd.335.2022.11.28.16.17.11; Mon, 28 Nov 2022 16:17:23 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=Oi0ZjD5s; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234607AbiK2ANF (ORCPT <rfc822;gah0developer@gmail.com> + 99 others); Mon, 28 Nov 2022 19:13:05 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44902 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234642AbiK2AMm (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Mon, 28 Nov 2022 19:12:42 -0500 Received: from mail-pj1-x1049.google.com (mail-pj1-x1049.google.com [IPv6:2607:f8b0:4864:20::1049]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 540376553 for <linux-kernel@vger.kernel.org>; Mon, 28 Nov 2022 16:12:40 -0800 (PST) Received: by mail-pj1-x1049.google.com with SMTP id on5-20020a17090b1d0500b0021821a07953so13074810pjb.4 for <linux-kernel@vger.kernel.org>; Mon, 28 Nov 2022 16:12:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=X881tlPDVXUlMgACZfZTZeyY92tnJLZ+/B1dNTIs8Gc=; b=Oi0ZjD5sM55sEXkn34mnC5VjQu6xKzBRPB+GDVOF2EnL4UyIrAVAj+ELyBOEjwwDGW s52xHTMYBS43CadhoSIW0qCfFLr9/dNXGbYqxREAcpO5ADcuQcbGGoVLhGcyFqOifAYz ojnWNbwTYlIyOmygoDbJnXy2VtgXqs/+yjKkqA2rMMM90MtJyW8+/OEsUGu7j28fquOe yOvSTpwtVyfqFbCa/Z053nvtEqBPgJ3feWv7yramCZQ80+/cUgNFfd/Hu0S7PmVXRryp Q/eyB0R27WV76y4VZNh25H39JHBqzfwP/Gr7dqgkyyzpXjcPx4HmPcDa9Lj/VZGtOJAy T8zA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=X881tlPDVXUlMgACZfZTZeyY92tnJLZ+/B1dNTIs8Gc=; b=g3xILbF4RjxVQBw5aZTaLIUZ4pbFPvvTA/rY/SNct1NuCvAkMas1uh+dwLQCbStjCQ RCdGWLfvUEpMfuO/lIUXk2oePlx+qTczL/DuRFlQjHpff43l1mgrc8nM+8dOlX0WauqK Uf7Ikx3C/+4Q3OiHne2akqs/X0MUvJ/zeEhEfLhKH/hLFJejTeYqA2ObfDYDXrHzOjYr e4IrakkewZs4dzEPLCS3pQjUZGhzAysAUdEP0r6IhySrSlXsuD9GkWSKmxqxarIXRuy7 BeANzWNkgilvF9RC2MbTlOo/5+kuaYOyDXgKg6vrsZlRVbhPW2j3SZELAveAf5BpEnMj EqQA== X-Gm-Message-State: ANoB5plyVbf8Vg4tLpZ6BIljh7TE7GQZPXrpLxhT/7TtzZ4ePcm2C0iP YvEG1bwvO4TcIrxXPoKe3SzLHKE813XMCQ== X-Received: from dlatypov-spec.c.googlers.com ([fda3:e722:ac3:cc00:24:72f4:c0a8:3f35]) (user=dlatypov job=sendgmr) by 2002:a05:6a00:408b:b0:56b:ca57:ba8c with SMTP id bw11-20020a056a00408b00b0056bca57ba8cmr35465110pfb.43.1669680759811; Mon, 28 Nov 2022 16:12:39 -0800 (PST) Date: Mon, 28 Nov 2022 16:12:34 -0800 Mime-Version: 1.0 X-Mailer: git-send-email 2.38.1.584.g0f3c55d4c2-goog Message-ID: <20221129001234.606653-1-dlatypov@google.com> Subject: [PATCH] kunit: tool: don't include KTAP headers and the like in the test log From: Daniel Latypov <dlatypov@google.com> To: brendanhiggins@google.com, davidgow@google.com Cc: rmoar@google.com, linux-kernel@vger.kernel.org, kunit-dev@googlegroups.com, linux-kselftest@vger.kernel.org, skhan@linuxfoundation.org, Daniel Latypov <dlatypov@google.com> Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750787470317097540?= X-GMAIL-MSGID: =?utf-8?q?1750787470317097540?= |
Series |
kunit: tool: don't include KTAP headers and the like in the test log
|
|
Commit Message
Daniel Latypov
Nov. 29, 2022, 12:12 a.m. UTC
We print the "test log" on failure.
This is meant to be all the kernel output that happened during the test.
But we also include the special KTAP lines in it, which are often
redundant.
E.g. we include the "not ok" line in the log, right before we print
that the test case failed...
[13:51:48] Expected 2 + 1 == 2, but
[13:51:48] 2 + 1 == 3 (0x3)
[13:51:48] not ok 1 example_simple_test
[13:51:48] [FAILED] example_simple_test
More full example after this patch:
[13:51:48] =================== example (4 subtests) ===================
[13:51:48] # example_simple_test: initializing
[13:51:48] # example_simple_test: EXPECTATION FAILED at lib/kunit/kunit-example-test.c:29
[13:51:48] Expected 2 + 1 == 2, but
[13:51:48] 2 + 1 == 3 (0x3)
[13:51:48] [FAILED] example_simple_test
Signed-off-by: Daniel Latypov <dlatypov@google.com>
---
tools/testing/kunit/kunit_parser.py | 8 ++++----
tools/testing/kunit/kunit_tool_test.py | 17 +++++++++++++++++
2 files changed, 21 insertions(+), 4 deletions(-)
base-commit: 11300092f6f4dc4103ac4bd950d62f94effc736a
Comments
On Tue, Nov 29, 2022 at 8:12 AM Daniel Latypov <dlatypov@google.com> wrote: > > We print the "test log" on failure. > This is meant to be all the kernel output that happened during the test. > > But we also include the special KTAP lines in it, which are often > redundant. > > E.g. we include the "not ok" line in the log, right before we print > that the test case failed... > [13:51:48] Expected 2 + 1 == 2, but > [13:51:48] 2 + 1 == 3 (0x3) > [13:51:48] not ok 1 example_simple_test > [13:51:48] [FAILED] example_simple_test > > More full example after this patch: > [13:51:48] =================== example (4 subtests) =================== > [13:51:48] # example_simple_test: initializing > [13:51:48] # example_simple_test: EXPECTATION FAILED at lib/kunit/kunit-example-test.c:29 > [13:51:48] Expected 2 + 1 == 2, but > [13:51:48] 2 + 1 == 3 (0x3) > [13:51:48] [FAILED] example_simple_test > > Signed-off-by: Daniel Latypov <dlatypov@google.com> > --- I totally agree we should skip these from the log. (Unless --raw_output is enabled, but that obviously doesn't apply.) Going forward, I think we should also probably disable kunit.stats_enabled when running via kunit.py, too (again, unless --raw_output is used.) In any case, this looks good and works well here. Reviewed-by: David Gow <davidgow@google.com> Cheers, -- David > tools/testing/kunit/kunit_parser.py | 8 ++++---- > tools/testing/kunit/kunit_tool_test.py | 17 +++++++++++++++++ > 2 files changed, 21 insertions(+), 4 deletions(-) > > diff --git a/tools/testing/kunit/kunit_parser.py b/tools/testing/kunit/kunit_parser.py > index 4cc2f8b7ecd0..99b8f058db40 100644 > --- a/tools/testing/kunit/kunit_parser.py > +++ b/tools/testing/kunit/kunit_parser.py > @@ -295,7 +295,7 @@ def parse_ktap_header(lines: LineStream, test: Test) -> bool: > check_version(version_num, TAP_VERSIONS, 'TAP', test) > else: > return False > - test.log.append(lines.pop()) > + lines.pop() > return True > > TEST_HEADER = re.compile(r'^# Subtest: (.*)$') > @@ -318,8 +318,8 @@ def parse_test_header(lines: LineStream, test: Test) -> bool: > match = TEST_HEADER.match(lines.peek()) > if not match: > return False > - test.log.append(lines.pop()) > test.name = match.group(1) > + lines.pop() > return True > > TEST_PLAN = re.compile(r'1\.\.([0-9]+)') > @@ -345,9 +345,9 @@ def parse_test_plan(lines: LineStream, test: Test) -> bool: > if not match: > test.expected_count = None > return False > - test.log.append(lines.pop()) > expected_count = int(match.group(1)) > test.expected_count = expected_count > + lines.pop() > return True > > TEST_RESULT = re.compile(r'^(ok|not ok) ([0-9]+) (- )?([^#]*)( # .*)?$') > @@ -409,7 +409,7 @@ def parse_test_result(lines: LineStream, test: Test, > # Check if line matches test result line format > if not match: > return False > - test.log.append(lines.pop()) > + lines.pop() > > # Set name of test object > if skip_match: > diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py > index d7f669cbf2a8..1ef921ac4331 100755 > --- a/tools/testing/kunit/kunit_tool_test.py > +++ b/tools/testing/kunit/kunit_tool_test.py > @@ -84,6 +84,10 @@ class KUnitParserTest(unittest.TestCase): > self.print_mock = mock.patch('kunit_printer.Printer.print').start() > self.addCleanup(mock.patch.stopall) > > + def noPrintCallContains(self, substr: str): > + for call in self.print_mock.mock_calls: > + self.assertNotIn(substr, call.args[0]) > + > def assertContains(self, needle: str, haystack: kunit_parser.LineStream): > # Clone the iterator so we can print the contents on failure. > copy, backup = itertools.tee(haystack) > @@ -327,6 +331,19 @@ class KUnitParserTest(unittest.TestCase): > result = kunit_parser.parse_run_tests(file.readlines()) > self.print_mock.assert_any_call(StrContains('suite (1 subtest)')) > > + def test_show_test_output_on_failure(self): > + output = """ > + KTAP version 1 > + 1..1 > + Test output. > + not ok 1 test1 > + """ > + result = kunit_parser.parse_run_tests(output.splitlines()) > + self.assertEqual(kunit_parser.TestStatus.FAILURE, result.status) > + > + self.print_mock.assert_any_call(StrContains('Test output.')) > + self.noPrintCallContains('not ok 1 test1') > + > def line_stream_from_strs(strs: Iterable[str]) -> kunit_parser.LineStream: > return kunit_parser.LineStream(enumerate(strs, start=1)) > > > base-commit: 11300092f6f4dc4103ac4bd950d62f94effc736a > -- > 2.38.1.584.g0f3c55d4c2-goog >
On Tue, Nov 29, 2022 at 12:31 AM David Gow <davidgow@google.com> wrote: > > On Tue, Nov 29, 2022 at 8:12 AM Daniel Latypov <dlatypov@google.com> wrote: > > > > We print the "test log" on failure. > > This is meant to be all the kernel output that happened during the test. > > > > But we also include the special KTAP lines in it, which are often > > redundant. > > > > E.g. we include the "not ok" line in the log, right before we print > > that the test case failed... > > [13:51:48] Expected 2 + 1 == 2, but > > [13:51:48] 2 + 1 == 3 (0x3) > > [13:51:48] not ok 1 example_simple_test > > [13:51:48] [FAILED] example_simple_test > > > > More full example after this patch: > > [13:51:48] =================== example (4 subtests) =================== > > [13:51:48] # example_simple_test: initializing > > [13:51:48] # example_simple_test: EXPECTATION FAILED at lib/kunit/kunit-example-test.c:29 > > [13:51:48] Expected 2 + 1 == 2, but > > [13:51:48] 2 + 1 == 3 (0x3) > > [13:51:48] [FAILED] example_simple_test > > > > Signed-off-by: Daniel Latypov <dlatypov@google.com> > > --- > > I totally agree we should skip these from the log. (Unless > --raw_output is enabled, but that obviously doesn't apply.) > > Going forward, I think we should also probably disable > kunit.stats_enabled when running via kunit.py, too (again, unless > --raw_output is used.) I considered including that as a patch 2/2 here. But changing the behavior like that felt a bit iffy. We've basically been telling people that looking at .kunit/test.log is logically equivalent to running with kunit.py run --raw_output. That would no longer be true after such a change. So I'm torn between that and automatically filtering them out in the parser side. Cons of tweaking args based on --raw_output * now more magic, harder to explain (see above) * people might find test counts useful when looking at test.log Cons of filtering out test counts in the parser * risks false positives: filtering out other lines besides test counts * when there's missing output, this is less debuggable * 99% of users are *not* going to dig into the python code * but IMO users are fairly likely to notice the extra kunit.stats_enabled=0 cmdline arg And overall, the benefit of hiding these is very small and cosmetic in nature. So that means making a tradeoff to do so feels more iffy. The hiding done in this patch seemed fine since there was no tradeoff, we just needed to stop including lines we've already recognized as KTAP directives. > > In any case, this looks good and works well here. > > Reviewed-by: David Gow <davidgow@google.com> > > Cheers, > -- David
On Tue, Nov 29, 2022 at 12:09 PM Daniel Latypov <dlatypov@google.com> wrote: > > On Tue, Nov 29, 2022 at 12:31 AM David Gow <davidgow@google.com> wrote: > > > > On Tue, Nov 29, 2022 at 8:12 AM Daniel Latypov <dlatypov@google.com> wrote: > > > > > > We print the "test log" on failure. > > > This is meant to be all the kernel output that happened during the test. > > > > > > But we also include the special KTAP lines in it, which are often > > > redundant. > > > > > > E.g. we include the "not ok" line in the log, right before we print > > > that the test case failed... > > > [13:51:48] Expected 2 + 1 == 2, but > > > [13:51:48] 2 + 1 == 3 (0x3) > > > [13:51:48] not ok 1 example_simple_test > > > [13:51:48] [FAILED] example_simple_test > > > > > > More full example after this patch: > > > [13:51:48] =================== example (4 subtests) =================== > > > [13:51:48] # example_simple_test: initializing > > > [13:51:48] # example_simple_test: EXPECTATION FAILED at lib/kunit/kunit-example-test.c:29 > > > [13:51:48] Expected 2 + 1 == 2, but > > > [13:51:48] 2 + 1 == 3 (0x3) > > > [13:51:48] [FAILED] example_simple_test > > > > > > Signed-off-by: Daniel Latypov <dlatypov@google.com> > > > --- > > > > I totally agree we should skip these from the log. (Unless > > --raw_output is enabled, but that obviously doesn't apply.) > > > > Going forward, I think we should also probably disable > > kunit.stats_enabled when running via kunit.py, too (again, unless > > --raw_output is used.) > > I considered including that as a patch 2/2 here. > But changing the behavior like that felt a bit iffy. > > We've basically been telling people that looking at .kunit/test.log is > logically equivalent to running with kunit.py run --raw_output. > That would no longer be true after such a change. > So I'm torn between that and automatically filtering them out in the > parser side. > > Cons of tweaking args based on --raw_output > * now more magic, harder to explain (see above) > * people might find test counts useful when looking at test.log > > Cons of filtering out test counts in the parser > * risks false positives: filtering out other lines besides test counts > * when there's missing output, this is less debuggable > * 99% of users are *not* going to dig into the python code > * but IMO users are fairly likely to notice the extra > kunit.stats_enabled=0 cmdline arg > > And overall, the benefit of hiding these is very small and cosmetic in nature. > So that means making a tradeoff to do so feels more iffy. > > The hiding done in this patch seemed fine since there was no tradeoff, > we just needed to stop including lines we've already recognized as > KTAP directives. > I tested and reviewed the changes and this all looks good to me. Nice to no longer print the KTAP version line or test result lines in the test logs. To comment on the above discussion: of the options discussed, I would prefer to either tweak the stats_enabled arg based on the raw_output flag or simply leave the output as is. I personally think that filtering out the test counts using the parser would be messy and does not match the precedent set by this patch to not include only recognized KTAP directives in the test log (with the exception of the "# Subtest" line, which I am planning to propose for KTAP v2). I agree with the cons to tweaking the arg based on the raw_output flag Daniel mentioned above. Additionally, I think including the test counts line in the log could actually be quite useful in the instance of combining module results. Since the parser does not include a test summary line with counts for failed subtests, this would save the time counting up the number of failed/skipped/passed subtests for a specific module. This would, in general, also apply to failing tests with a large number of subtests within them. That being said I would be ok with tweaking the stats_enabled arg based on the raw_output flag as well. It does look cleaner without the counts printed in the log, especially when there are only a few number of tests or the test summary line of the parser repeats the same information as the test counts line. Reviewed-by: Rae Moar <rmoar@google.com> Thanks! Rae > > > > In any case, this looks good and works well here. > > > > Reviewed-by: David Gow <davidgow@google.com> > > > > Cheers, > > -- David
diff --git a/tools/testing/kunit/kunit_parser.py b/tools/testing/kunit/kunit_parser.py index 4cc2f8b7ecd0..99b8f058db40 100644 --- a/tools/testing/kunit/kunit_parser.py +++ b/tools/testing/kunit/kunit_parser.py @@ -295,7 +295,7 @@ def parse_ktap_header(lines: LineStream, test: Test) -> bool: check_version(version_num, TAP_VERSIONS, 'TAP', test) else: return False - test.log.append(lines.pop()) + lines.pop() return True TEST_HEADER = re.compile(r'^# Subtest: (.*)$') @@ -318,8 +318,8 @@ def parse_test_header(lines: LineStream, test: Test) -> bool: match = TEST_HEADER.match(lines.peek()) if not match: return False - test.log.append(lines.pop()) test.name = match.group(1) + lines.pop() return True TEST_PLAN = re.compile(r'1\.\.([0-9]+)') @@ -345,9 +345,9 @@ def parse_test_plan(lines: LineStream, test: Test) -> bool: if not match: test.expected_count = None return False - test.log.append(lines.pop()) expected_count = int(match.group(1)) test.expected_count = expected_count + lines.pop() return True TEST_RESULT = re.compile(r'^(ok|not ok) ([0-9]+) (- )?([^#]*)( # .*)?$') @@ -409,7 +409,7 @@ def parse_test_result(lines: LineStream, test: Test, # Check if line matches test result line format if not match: return False - test.log.append(lines.pop()) + lines.pop() # Set name of test object if skip_match: diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py index d7f669cbf2a8..1ef921ac4331 100755 --- a/tools/testing/kunit/kunit_tool_test.py +++ b/tools/testing/kunit/kunit_tool_test.py @@ -84,6 +84,10 @@ class KUnitParserTest(unittest.TestCase): self.print_mock = mock.patch('kunit_printer.Printer.print').start() self.addCleanup(mock.patch.stopall) + def noPrintCallContains(self, substr: str): + for call in self.print_mock.mock_calls: + self.assertNotIn(substr, call.args[0]) + def assertContains(self, needle: str, haystack: kunit_parser.LineStream): # Clone the iterator so we can print the contents on failure. copy, backup = itertools.tee(haystack) @@ -327,6 +331,19 @@ class KUnitParserTest(unittest.TestCase): result = kunit_parser.parse_run_tests(file.readlines()) self.print_mock.assert_any_call(StrContains('suite (1 subtest)')) + def test_show_test_output_on_failure(self): + output = """ + KTAP version 1 + 1..1 + Test output. + not ok 1 test1 + """ + result = kunit_parser.parse_run_tests(output.splitlines()) + self.assertEqual(kunit_parser.TestStatus.FAILURE, result.status) + + self.print_mock.assert_any_call(StrContains('Test output.')) + self.noPrintCallContains('not ok 1 test1') + def line_stream_from_strs(strs: Iterable[str]) -> kunit_parser.LineStream: return kunit_parser.LineStream(enumerate(strs, start=1))