From patchwork Mon Jul 17 13:21:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthieu Baerts X-Patchwork-Id: 121364 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:c923:0:b0:3e4:2afc:c1 with SMTP id j3csp1146864vqt; Mon, 17 Jul 2023 07:28:17 -0700 (PDT) X-Google-Smtp-Source: APBJJlFcWYUo6k0BJdGR+fGhPcn0MrLYSxk0QJodJMnvYRvwe/Mz+5B90jg0pBmmh55FB+CdYtAO X-Received: by 2002:a17:90a:be0d:b0:263:eef:1465 with SMTP id a13-20020a17090abe0d00b002630eef1465mr9961466pjs.44.1689604097246; Mon, 17 Jul 2023 07:28:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1689604097; cv=none; d=google.com; s=arc-20160816; b=M2HtdSaT2UiXH+Oq+gkiARJ5FMrVew0bB8MyBYbQTXWdHpYB6IC6ZF/FhVOOgInn3S FfGE6tweYHI6HQM6b+Zq4/DeTSiPeI+apSjFwv8hZlhK3iKmkNmXfI1SP35Al1nAEsU/ Nth+FB/TfTahdXgnQ6ET6O6gj7gE+0yw3cUW3u17aUMZ2gcPMnFI0GsdrvWWiKJAFkMj lsI9cQgTDgXb7U8yEQ6KpGDePnp0+E/bXlemjiyMBQbLGOOAt0GT3eHvja90BB6QO85G EoF+Fg1D8c9rNIUZG3WnYPsU4OgyqD41NDLoDgu/eJaUXhTu+l084QYvTwNCLdezB/fy /aHw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:in-reply-to:references:message-id :content-transfer-encoding:mime-version:subject:date:from :dkim-signature; bh=xNpnmzuYMAKwrCxb2z/tH+HC3idxZwgWFXVOTSz9bCQ=; fh=9UgnUdzL30oFh+UbzKj4E7u1+buBgKOqdJJe+xVkJ8Q=; b=QmVxUdqcMtIbOeEtiwESyDs7+esckPvnTb0wZ2r+WQIM3GalL6kec4uswh7wCZu6tP NgYlLSwz7oKHK3obFwVmVY96QUiX39AIWOhT+uT2fUgw8kwBUZaczs+ufGDVzZ4E9wP/ /Psk6F0hxftEtxTaU1YgClODU2JKAH/PaUDjbYuUGhnY4nUG0avO/CmDMN9tGJBVUU8Q 1G9CmD6gdk9JUe9iV0+bOeKdLj6kqnFXmZ+lM0SbfrbZiS12t5NPKDXbc/od68og7xly A3G8OetpRkLMgl+AEFq52620aUUWEOTEajGaIjlADpR/xj/75J9wpCVTdWOcu7+hAD8A hzRg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@tessares.net header.s=google header.b=DmrHdWM4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=tessares.net Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id c3-20020a17090ad90300b0025be6419478si5709371pjv.92.2023.07.17.07.28.04; Mon, 17 Jul 2023 07:28:17 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@tessares.net header.s=google header.b=DmrHdWM4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=tessares.net Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231911AbjGQNW4 (ORCPT + 99 others); Mon, 17 Jul 2023 09:22:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59628 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231557AbjGQNV6 (ORCPT ); Mon, 17 Jul 2023 09:21:58 -0400 Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7C52E199A for ; Mon, 17 Jul 2023 06:21:46 -0700 (PDT) Received: by mail-wm1-x32b.google.com with SMTP id 5b1f17b1804b1-3fbea14706eso40916285e9.2 for ; Mon, 17 Jul 2023 06:21:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tessares.net; s=google; t=1689600105; x=1692192105; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=xNpnmzuYMAKwrCxb2z/tH+HC3idxZwgWFXVOTSz9bCQ=; b=DmrHdWM4SJ8p/7y3CQTgZ2PRhIeELaNzcJXZy1OyMUE8aTlE94YNVPnLC0hd31L2de plQxfClK/Oaey14A2/nTBllh+BFxmFray0H9WcaifLuE/ua4GacnI1NQKG/pndwgW4ne VlnrJ400vS24WbgtDFweb+5i0kLsaTO0X1IV4QUoFmbXU95KrJwNoDF5XNvZCRl+2XBy zhLyOpVAtk9SreRTvUE/06Ml9qGtPHRKh5TZ+lp8RdRip8iDOhmnPOhZ4x0Sg2hXPWP0 vtY54zsEIBW4ByhOeARixykxFQlRn2iohFFUXfC1iHAaXawZSlWzoc5NpBfNrI+BvpNm i/iQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689600105; x=1692192105; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xNpnmzuYMAKwrCxb2z/tH+HC3idxZwgWFXVOTSz9bCQ=; b=HnO8MWsJFvkU8X6fc07zaYAVjuz2eha0DBkWXRmXjucMNNE1u6MIQ15wCD1MztIBCf HZCNwcAcwOzV5o+46ZXOqhaDdAnh69RRArtZ2D36wIGHQDi+NsFsE8q5xq2xEDviaI+0 fSIiQS+QmqfPxoLokVfP5fxQZRbTDnnPH7gL0nkf1E9Wvo2JzggQwgtryMbxpx0Giasa i9ywmJ5Z4uLgip96zDoBW1LcL8s52FLN1bj45gxDaHGdb8fhcI9RUsoBIuU+JAejQfEH jV1lq9p6BYKUmJJ0nmBsda1ItEtJAeXTQxv0imPsRvNf2qWqpbC3IyfIo/8PBUcR7syN W6Kw== X-Gm-Message-State: ABy/qLZKQSYM8mC4+B/4slNS9O4wL2Hy4bt/oCgTwX9wA7PgWmt1bAwq /4heB2F7PnGFrp7EszRoqLKXQg== X-Received: by 2002:a5d:4b12:0:b0:313:f94f:3b15 with SMTP id v18-20020a5d4b12000000b00313f94f3b15mr10492925wrq.2.1689600104948; Mon, 17 Jul 2023 06:21:44 -0700 (PDT) Received: from vdi08.nix.tessares.net (static.219.156.76.144.clients.your-server.de. [144.76.156.219]) by smtp.gmail.com with ESMTPSA id r8-20020a056000014800b0030fa3567541sm19249836wrx.48.2023.07.17.06.21.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Jul 2023 06:21:44 -0700 (PDT) From: Matthieu Baerts Date: Mon, 17 Jul 2023 15:21:31 +0200 Subject: [PATCH net-next 11/13] selftests: mptcp: simult flows: format subtests results in TAP MIME-Version: 1.0 Message-Id: <20230717-upstream-net-next-20230712-selftests-mptcp-subtests-v1-11-695127e0ad83@tessares.net> References: <20230717-upstream-net-next-20230712-selftests-mptcp-subtests-v1-0-695127e0ad83@tessares.net> In-Reply-To: <20230717-upstream-net-next-20230712-selftests-mptcp-subtests-v1-0-695127e0ad83@tessares.net> To: mptcp@lists.linux.dev, Mat Martineau , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Shuah Khan Cc: netdev@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, Matthieu Baerts X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=2295; i=matthieu.baerts@tessares.net; h=from:subject:message-id; bh=omBnUHRU9N3QOZYPdUCMphvA7FerRHVuKdmTtTNH2r0=; b=owEBbQKS/ZANAwAIAfa3gk9CaaBzAcsmYgBktUBdfFA7VqtWidJmBkSXruI3mkrTSPePM+LSl Ny2sl4xKQiJAjMEAAEIAB0WIQToy4X3aHcFem4n93r2t4JPQmmgcwUCZLVAXQAKCRD2t4JPQmmg c2tzD/0aXD5RjfWLRekq1u49sOHl86tY4l5b0mdd4JI4Qe00RWSvhQrIZvfK6nRF+Rq/HVNoLwm cte2dIyBatY9PQKUrcBOiX5V5oA8vzwF9G7ZPS0+DJk1JvRn6pbqbm5/YDHX+t36aGRH5NRQMPE sdspTSAsAw/HyZeQNWc9Eh8pVglolalW2mwdku0XZA6gyYtcP/DcFZ+0W4t9L0G5J+yfE32TJYu TFUNcEeb73xb7eQfTaoY1xHQTKhTDtjwaShdSnXmtj3miKslhTpSNlNLFgchu4OlbPnaJMDVb3W kNawnAylF93fuDzunMmIf/PDzE8q99HiXIYO6sHkaNfSyX4ZfFUdKD2hx48E1qZZM4X27DWz1HV duXnF25va1i8F5ecaG0GBabr6whZd51nF6vpoZCovSmK7v5XakiaHjNhpWEI1uV0wOx+i6eTvIP eAXg8HQMlXOby8fG2kQMg396WRA2V4DW7R73KVqZP7O9lc8pymTDyH7VahDz7NFJGKDbblFxzjy 0QCczvpPBR8HDSCpsQ4/lmx+0p1YL401QkfqbczpHDB8KO59it4yWBsv/KZHMYCyogsY4wN0p79 VpucBK5T/Ptd6D7XQMbS4TlIk55LKNU3M4lTyt/CKm8VmXlD8VRVhBF9kvAU8n6FY0H9V/6mKQ+ xB6r9Dnf3cRIwgw== X-Developer-Key: i=matthieu.baerts@tessares.net; a=openpgp; fpr=E8CB85F76877057A6E27F77AF6B7824F4269A073 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1771678305666703592 X-GMAIL-MSGID: 1771678305666703592 The current selftests infrastructure formats the results in TAP 13. This version doesn't support subtests and only the end result of each selftest is taken into account. It means that a single issue in a subtest of a selftest containing multiple subtests forces the whole selftest to be marked as failed. It also means that subtests results are not tracked by CIs executing selftests. MPTCP selftests run hundreds of various subtests. It is then important to track each of them and not one result per selftest. It is particularly interesting to do that when validating stable kernels with the last version of the test suite: tests might fail because a feature is not supported but the test didn't skip that part. In this case, if subtests are not tracked, the whole selftest will be marked as failed making the other subtests useless because their results are ignored. This patch formats subtests results in TAP in simult_flows.sh selftest. Link: https://github.com/multipath-tcp/mptcp_net-next/issues/368 Acked-by: Paolo Abeni Signed-off-by: Matthieu Baerts --- tools/testing/selftests/net/mptcp/simult_flows.sh | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/tools/testing/selftests/net/mptcp/simult_flows.sh b/tools/testing/selftests/net/mptcp/simult_flows.sh index 36a3c9d92e20..ce9203b817f8 100755 --- a/tools/testing/selftests/net/mptcp/simult_flows.sh +++ b/tools/testing/selftests/net/mptcp/simult_flows.sh @@ -261,6 +261,7 @@ run_test() printf "%-60s" "$msg" do_transfer $small $large $time lret=$? + mptcp_lib_result_code "${lret}" "${msg}" if [ $lret -ne 0 ]; then ret=$lret [ $bail -eq 0 ] || exit $ret @@ -269,6 +270,7 @@ run_test() printf "%-60s" "$msg - reverse direction" do_transfer $large $small $time lret=$? + mptcp_lib_result_code "${lret}" "${msg}" if [ $lret -ne 0 ]; then ret=$lret [ $bail -eq 0 ] || exit $ret @@ -305,4 +307,6 @@ run_test 10 10 1 50 "balanced bwidth with unbalanced delay" run_test 30 10 0 0 "unbalanced bwidth" run_test 30 10 1 50 "unbalanced bwidth with unbalanced delay" run_test 30 10 50 1 "unbalanced bwidth with opposed, unbalanced delay" + +mptcp_lib_result_print_all_tap exit $ret