From patchwork Tue Dec 19 21:21:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 181266 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:24d3:b0:fb:cd0c:d3e with SMTP id r19csp2235369dyi; Tue, 19 Dec 2023 13:22:40 -0800 (PST) X-Google-Smtp-Source: AGHT+IHNooPW+NkKAplNz+uHWkGoa9eTfF8aov4F5/HBSIzVkeWviZJ33hxK+Kd7ED2uH1/Sn2bz X-Received: by 2002:a05:6214:2509:b0:67f:2b5:b96a with SMTP id gf9-20020a056214250900b0067f02b5b96amr17348781qvb.51.1703020960742; Tue, 19 Dec 2023 13:22:40 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703020960; cv=none; d=google.com; s=arc-20160816; b=uCJCgmsCIl6y8mGa+ssrO6rs168VWXyyCTXIj8+KYI+DC3qyb/ko7tq16RnLWC3IH/ 6xwvKP9MiTLg7iiQ10eoLXuDOrn43Jrf/BIYakWdklDvNoW1DmVupb5ZSN4cud1q5kVb SzWex3USvdrP+GO4ccDgbJNfsFG23ykD6T0Ypp/LQ46/fjlLkK1e7hW5w1vcdoKBc0h8 Z2j4OqOFyrv7+8H1xSJ9tvOZi0ekEwqlgapWkZDxynJW5UDAEREVAkIN2LZ+T0iwpPMQ H9IAvgJr6UQJs7XyX4+huP+g73VCHXGgFlw8vtC/SKbjUevs01uiLfbRIRmbfVVVjnzl UZyw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:list-unsubscribe:list-subscribe:list-id:precedence :subject:date:from:dkim-signature; bh=tCT0VI6Q84pYr8lVXh0H80KmAQcVgR3Qt0RMkYn4KNI=; fh=AWGRrYbdrZEzAbtHshVADdACQbgc7pZKSs/DT5CASU4=; b=0BbuGd17yQtlJYukZr+V5qNKu4ptDB4NXd1AWIBIgXFCX102p4UOvahwJn4v9Kz92z FLlAO77tBjjkstl9zGIZRRpZNypYQDS6L9nJPWls3KT+xn+8g7P4TSLyoRUFKQ7pOfSJ HxdlmyhPSqWJSplUhRbYQSeVRJPA5Af1X4uIqmuiw3faWT0HkL2SnM3+jJ5N0pyCvdK/ mITI5JBqTb9Uqo9fOgurAbgT0BToHfGLJth1yMjdXQcB3Oyy1HHnQUF8Ig04+nlGbE21 JWuFNtvICLi4uEREo73uUhETiAagvo/UMDrepILYwMx+Czc89WWC/HZDM+cA4OWSEUii XV4Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=OXdOslmS; spf=pass (google.com: domain of linux-kernel+bounces-5966-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-5966-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id a13-20020a0cb34d000000b0067f1b8febe2si10028552qvf.98.2023.12.19.13.22.40 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Dec 2023 13:22:40 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-5966-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=OXdOslmS; spf=pass (google.com: domain of linux-kernel+bounces-5966-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-5966-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 516181C2529A for ; Tue, 19 Dec 2023 21:22:40 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id F05273EA95; Tue, 19 Dec 2023 21:21:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="OXdOslmS" X-Original-To: linux-kernel@vger.kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DA3D03B795; Tue, 19 Dec 2023 21:21:33 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id CE7C7C433CB; Tue, 19 Dec 2023 21:21:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1703020893; bh=/6Du62khS6+mPSPQl4NojRywrcNucj3y1bs+UHh2N8s=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=OXdOslmSRGeSw/+OMUsivb/kKzI+9FouykNEE4GpU/0ODqi3Fwu+flD6SvKib0Wxt MjB5ErEmP5FlpAPBuwKVtfX5aOnbRxeyppFwWh7WjzLcxiwgiaSgwV/s8aS8zW1TKd lp2TZDj6XYAX6k1IYba7rcVmHdPr01IopVM+WUNdoaCxnRveIuUFGDpmmflm7/RC6Q zXukACwOe++PuEcJks89O39FxzxnMs4wu2TQdoNH/yHeawegU9VUNmk/LGZRw7eHSQ eOXlI3sYUN2Si5oEkPFXmfOcaG2Otz8YqO5IrupiTpHEsMz9n+t4TAd0iGaSLFCccK nYdILJ+G3PWaA== From: Mark Brown Date: Tue, 19 Dec 2023 21:21:19 +0000 Subject: [PATCH 1/2] kselftest/seccomp: Use kselftest output functions for benchmark Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20231219-b4-kselftest-seccomp-benchmark-ktap-v1-1-f99e228631b0@kernel.org> References: <20231219-b4-kselftest-seccomp-benchmark-ktap-v1-0-f99e228631b0@kernel.org> In-Reply-To: <20231219-b4-kselftest-seccomp-benchmark-ktap-v1-0-f99e228631b0@kernel.org> To: Kees Cook , Andy Lutomirski , Will Drewry , Shuah Khan Cc: linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, Mark Brown X-Mailer: b4 0.13-dev-5c066 X-Developer-Signature: v=1; a=openpgp-sha256; l=6298; i=broonie@kernel.org; h=from:subject:message-id; bh=/6Du62khS6+mPSPQl4NojRywrcNucj3y1bs+UHh2N8s=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlgglY3Xqud3QDQGsOnu6Bl17eNqabR/vKtLwyr5JZ uRGIzIeJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZYIJWAAKCRAk1otyXVSH0NSjB/ 9gD1x2kzPfLXrhhQNuWZ+Okq9MOR363rrV1tgbPP/4y6f84snqLvjABf5o2wTYSg6euIuuhcNUbYXH c314rrtmK+sVHptDlndr+pbYWZGZlsznLGOLo8rerX18RfQKsPykt04XlMigV00/DfUsSCIXdAnOev eeVqd1L/XVdbYwjNCqzZjLMWqgpoMQqkCJMS8BEmy+kGXh5PWJpO9XixtllYRpYRJpP4n8xreqtHgO homNQrHpG5fNtqqKCBEcYUf6UUtxjxTQFZMT6v2YyWjYf4+hOp+MsnVfhJjJe0jm/Vt1PDJ6ddE6Ed 0TtQEsx/kljB6lv/Hre3MQOgt7mf/u X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785746906482553028 X-GMAIL-MSGID: 1785746906482553028 In preparation for trying to output the test results themselves in TAP format rework all the prints in the benchmark to use the kselftest output functions. The uses of system() all produce single line output so we can avoid having to deal with fully managing the child process and continue to use system() by simply printing an empty message before we invoke system(). We also leave one printf() used to complete a line of output in place. Signed-off-by: Mark Brown --- .../testing/selftests/seccomp/seccomp_benchmark.c | 45 ++++++++++++---------- 1 file changed, 24 insertions(+), 21 deletions(-) diff --git a/tools/testing/selftests/seccomp/seccomp_benchmark.c b/tools/testing/selftests/seccomp/seccomp_benchmark.c index 5b5c9d558dee..93168dd2c1e3 100644 --- a/tools/testing/selftests/seccomp/seccomp_benchmark.c +++ b/tools/testing/selftests/seccomp/seccomp_benchmark.c @@ -38,10 +38,10 @@ unsigned long long timing(clockid_t clk_id, unsigned long long samples) i *= 1000000000ULL; i += finish.tv_nsec - start.tv_nsec; - printf("%lu.%09lu - %lu.%09lu = %llu (%.1fs)\n", - finish.tv_sec, finish.tv_nsec, - start.tv_sec, start.tv_nsec, - i, (double)i / 1000000000.0); + ksft_print_msg("%lu.%09lu - %lu.%09lu = %llu (%.1fs)\n", + finish.tv_sec, finish.tv_nsec, + start.tv_sec, start.tv_nsec, + i, (double)i / 1000000000.0); return i; } @@ -53,7 +53,7 @@ unsigned long long calibrate(void) pid_t pid, ret; int seconds = 15; - printf("Calibrating sample size for %d seconds worth of syscalls ...\n", seconds); + ksft_print_msg("Calibrating sample size for %d seconds worth of syscalls ...\n", seconds); samples = 0; pid = getpid(); @@ -102,14 +102,14 @@ long compare(const char *name_one, const char *name_eval, const char *name_two, { bool good; - printf("\t%s %s %s (%lld %s %lld): ", name_one, name_eval, name_two, - (long long)one, name_eval, (long long)two); + ksft_print_msg("\t%s %s %s (%lld %s %lld): ", name_one, name_eval, name_two, + (long long)one, name_eval, (long long)two); if (one > INT_MAX) { - printf("Miscalculation! Measurement went negative: %lld\n", (long long)one); + ksft_print_msg("Miscalculation! Measurement went negative: %lld\n", (long long)one); return 1; } if (two > INT_MAX) { - printf("Miscalculation! Measurement went negative: %lld\n", (long long)two); + ksft_print_msg("Miscalculation! Measurement went negative: %lld\n", (long long)two); return 1; } @@ -145,12 +145,15 @@ int main(int argc, char *argv[]) setbuf(stdout, NULL); - printf("Running on:\n"); + ksft_print_msg("Running on:\n"); + ksft_print_msg(""); system("uname -a"); - printf("Current BPF sysctl settings:\n"); + ksft_print_msg("Current BPF sysctl settings:\n"); /* Avoid using "sysctl" which may not be installed. */ + ksft_print_msg(""); system("grep -H . /proc/sys/net/core/bpf_jit_enable"); + ksft_print_msg(""); system("grep -H . /proc/sys/net/core/bpf_jit_harden"); if (argc > 1) @@ -158,11 +161,11 @@ int main(int argc, char *argv[]) else samples = calibrate(); - printf("Benchmarking %llu syscalls...\n", samples); + ksft_print_msg("Benchmarking %llu syscalls...\n", samples); /* Native call */ native = timing(CLOCK_PROCESS_CPUTIME_ID, samples) / samples; - printf("getpid native: %llu ns\n", native); + ksft_print_msg("getpid native: %llu ns\n", native); ret = prctl(PR_SET_NO_NEW_PRIVS, 1, 0, 0, 0); assert(ret == 0); @@ -172,33 +175,33 @@ int main(int argc, char *argv[]) assert(ret == 0); bitmap1 = timing(CLOCK_PROCESS_CPUTIME_ID, samples) / samples; - printf("getpid RET_ALLOW 1 filter (bitmap): %llu ns\n", bitmap1); + ksft_print_msg("getpid RET_ALLOW 1 filter (bitmap): %llu ns\n", bitmap1); /* Second filter resulting in a bitmap */ ret = prctl(PR_SET_SECCOMP, SECCOMP_MODE_FILTER, &bitmap_prog); assert(ret == 0); bitmap2 = timing(CLOCK_PROCESS_CPUTIME_ID, samples) / samples; - printf("getpid RET_ALLOW 2 filters (bitmap): %llu ns\n", bitmap2); + ksft_print_msg("getpid RET_ALLOW 2 filters (bitmap): %llu ns\n", bitmap2); /* Third filter, can no longer be converted to bitmap */ ret = prctl(PR_SET_SECCOMP, SECCOMP_MODE_FILTER, &prog); assert(ret == 0); filter1 = timing(CLOCK_PROCESS_CPUTIME_ID, samples) / samples; - printf("getpid RET_ALLOW 3 filters (full): %llu ns\n", filter1); + ksft_print_msg("getpid RET_ALLOW 3 filters (full): %llu ns\n", filter1); /* Fourth filter, can not be converted to bitmap because of filter 3 */ ret = prctl(PR_SET_SECCOMP, SECCOMP_MODE_FILTER, &bitmap_prog); assert(ret == 0); filter2 = timing(CLOCK_PROCESS_CPUTIME_ID, samples) / samples; - printf("getpid RET_ALLOW 4 filters (full): %llu ns\n", filter2); + ksft_print_msg("getpid RET_ALLOW 4 filters (full): %llu ns\n", filter2); /* Estimations */ #define ESTIMATE(fmt, var, what) do { \ var = (what); \ - printf("Estimated " fmt ": %llu ns\n", var); \ + ksft_print_msg("Estimated " fmt ": %llu ns\n", var); \ if (var > INT_MAX) \ goto more_samples; \ } while (0) @@ -218,7 +221,7 @@ int main(int argc, char *argv[]) ESTIMATE("seccomp per-filter overhead (filters / 4)", per_filter2, (filter2 - native - entry) / 4); - printf("Expectations:\n"); + ksft_print_msg("Expectations:\n"); ret |= compare("native", "≤", "1 bitmap", native, le, bitmap1); bits = compare("native", "≤", "1 filter", native, le, filter1); if (bits) @@ -230,7 +233,7 @@ int main(int argc, char *argv[]) bits = compare("1 bitmapped", "≈", "2 bitmapped", bitmap1 - native, approx, bitmap2 - native); if (bits) { - printf("Skipping constant action bitmap expectations: they appear unsupported.\n"); + ksft_print_msg("Skipping constant action bitmap expectations: they appear unsupported.\n"); goto out; } @@ -242,7 +245,7 @@ int main(int argc, char *argv[]) goto out; more_samples: - printf("Saw unexpected benchmark result. Try running again with more samples?\n"); + ksft_print_msg("Saw unexpected benchmark result. Try running again with more samples?\n"); out: return 0; } From patchwork Tue Dec 19 21:21:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 181267 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:24d3:b0:fb:cd0c:d3e with SMTP id r19csp2235428dyi; Tue, 19 Dec 2023 13:22:50 -0800 (PST) X-Google-Smtp-Source: AGHT+IGDDPAXFpicdPCQfk2/y5KgAeQRE9l/dNOWZzOLbiYHX2N/2VAnFWd785t8NHYFuiEdQd+p X-Received: by 2002:a05:620a:a4b:b0:781:102e:8a5 with SMTP id j11-20020a05620a0a4b00b00781102e08a5mr167933qka.8.1703020970440; Tue, 19 Dec 2023 13:22:50 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703020970; cv=none; d=google.com; s=arc-20160816; b=j5G1D92/1Fm6s3hZY5oyr4PCrUj6Wzeik2vtydDh786+fxuIiwq2BYtIRiMua/8z3o ERQSQ8IDXCdAWTVQY2B0uyEI3WnHah1TSKUVL3hjNamASWjNKxu5nXMj/GBBHKI+UVws jr7oGZtIFWXX1q3A8MMWkPm2Uk5xV1oKckab9pTbH5EnavNIVaxj8+Gsmm8y5mixPcWM 35oSReVLqD979qecMhzFTJHINGYts70F8yzOsO+uBsFyKyod7gC2y+qUv6UwYOr0jA12 xRMciVc6tJKgHwRFJTeqtiVa978SzryEgRDXLIwFKXpF6jNuHBIxzMSJf6EUa5fR12D3 W3HQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:list-unsubscribe:list-subscribe:list-id:precedence :subject:date:from:dkim-signature; bh=Vfjo03sItzjewQsVv8BrC4T9MYxP6tDeKdIUdaubxn8=; fh=AWGRrYbdrZEzAbtHshVADdACQbgc7pZKSs/DT5CASU4=; b=zJkGUX1JS6J2zn+qB4TLfOLZWI6eNCID7ZdZF9jne3W3Wvz8r9JmjiIzEbmcx0HPkY aZXVP7x97+oxqmrnqiottR/qqYvFAxqyy1tBQDx5ufxT74jCZNNM+dIbsBPUCMAgRPRU bT8edrFZHYmS5m2hdjVaeDF5XYBxtwIz376rtYX7O3NrMcleIBnzlR7HkJLshTcuuFHJ qHF6dFathRu7ag8J5gkI6EzlJeLv3bd7urYFhitx4ZZ4cOHU4iw7gbzFMkwd6vZhFv4Y /pzMxO/xR7brltJYF3Cmm0TUV9hwWVuQgmPHDcWg8YVmdYm6w4Esr6LYBNJ8FE9ZhbRI q0wQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=SHvpMbPs; spf=pass (google.com: domain of linux-kernel+bounces-5967-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-5967-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id or10-20020a05620a618a00b0077f6d73fac4si1258478qkn.475.2023.12.19.13.22.50 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Dec 2023 13:22:50 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-5967-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=SHvpMbPs; spf=pass (google.com: domain of linux-kernel+bounces-5967-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-5967-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 1C2821C25240 for ; Tue, 19 Dec 2023 21:22:50 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 903213FE32; Tue, 19 Dec 2023 21:21:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="SHvpMbPs" X-Original-To: linux-kernel@vger.kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B76EC3DB80; Tue, 19 Dec 2023 21:21:35 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B5B28C43395; Tue, 19 Dec 2023 21:21:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1703020895; bh=oA1pNNbJ63oV11EYi9Wh6Pn+MBONE49Y/f+8c5yfJbw=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=SHvpMbPsZ7dBP2btm3EjUSGBxL5/UhcQLmA6zmhiHyL9EIHspgoFFvMkIwZLDZW4F ALsXSzzekC86VG73Y51lOc7PItd71kADnawDQwRgTTutdGLDjBuTZzWAg9vqhzFDtn HlZ7k4DDV/H4UXpb7sim0H/O4PtYoL6JZvg8Xes0YrHv8+PwYvNiS0Ub2HaM2tg8YN zqWVRXelcW52Jhb2si/JCIVDYoSwPGoULyHW00PSH1/cFq3/mYwOARyBAEBiEHkEHV 7bbkvPaLRetDixMh4B8xPy8+nPqGpX5Pt9ABxAnfne2Tn/CC4IXIfv/j/wKNBmhb+a 6Iql2UVKSPZSg== From: Mark Brown Date: Tue, 19 Dec 2023 21:21:20 +0000 Subject: [PATCH 2/2] kselftest/seccomp: Report each expectation we assert as a KTAP test Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20231219-b4-kselftest-seccomp-benchmark-ktap-v1-2-f99e228631b0@kernel.org> References: <20231219-b4-kselftest-seccomp-benchmark-ktap-v1-0-f99e228631b0@kernel.org> In-Reply-To: <20231219-b4-kselftest-seccomp-benchmark-ktap-v1-0-f99e228631b0@kernel.org> To: Kees Cook , Andy Lutomirski , Will Drewry , Shuah Khan Cc: linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, Mark Brown X-Mailer: b4 0.13-dev-5c066 X-Developer-Signature: v=1; a=openpgp-sha256; l=4967; i=broonie@kernel.org; h=from:subject:message-id; bh=oA1pNNbJ63oV11EYi9Wh6Pn+MBONE49Y/f+8c5yfJbw=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlgglYESeKHvJrPrwFkWxo4TJbTH8YEb4zmiEYdAzx PebQDZ6JATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZYIJWAAKCRAk1otyXVSH0JoWB/ 9vgsFFxdsHgH+8hjqI3qPD2D1DdeuyCO8vYvasUYRqvkyMUZaOp0rQll2+nMWgYk+gNxu8Wc6pxbv1 xKlk1zYOiiHDb5o89SYoXdlZ918i1TY0B1V0RTii7OnrOhjZmH5QkmnEmsSOKmbCS/nP5iEePtZO0g ESdaVNBqjHxJjl20baGYT4UfAvMm663COAMA6+LdIv0gWL7JJiwKr0nBuLz6hDKhGb/RXAPFjU02oW CDerwPTxrzyqm3il2pXSqkf2A7DBtgNB1hVr8NO7eQQohL/6BrncR2HXEUa9yeiLqD2pvgtP8qzAxi CWlPpB+gM0R2zI7PYgEhN3nk3xXAU9 X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785746916806703018 X-GMAIL-MSGID: 1785746916806703018 The seccomp benchmark test makes a number of checks on the performance it measures and logs them to the output but does so in a custom format which none of the automated test runners understand meaning that the chances that anyone is paying attention are slim. Let's additionally log each result in KTAP format so that automated systems parsing the test output will see each comparison as a test case. The original logs are left in place since they provide the actual numbers for analysis. As part of this rework the flow for the main program so that when we skip tests we still log all the tests we skip, this is because the standard KTAP headers and footers include counts of the number of expected and run tests. --- .../testing/selftests/seccomp/seccomp_benchmark.c | 62 +++++++++++++++------- 1 file changed, 42 insertions(+), 20 deletions(-) diff --git a/tools/testing/selftests/seccomp/seccomp_benchmark.c b/tools/testing/selftests/seccomp/seccomp_benchmark.c index 93168dd2c1e3..436a527b8235 100644 --- a/tools/testing/selftests/seccomp/seccomp_benchmark.c +++ b/tools/testing/selftests/seccomp/seccomp_benchmark.c @@ -98,24 +98,36 @@ bool le(int i_one, int i_two) } long compare(const char *name_one, const char *name_eval, const char *name_two, - unsigned long long one, bool (*eval)(int, int), unsigned long long two) + unsigned long long one, bool (*eval)(int, int), unsigned long long two, + bool skip) { bool good; + if (skip) { + ksft_test_result_skip("%s %s %s\n", name_one, name_eval, + name_two); + return 0; + } + ksft_print_msg("\t%s %s %s (%lld %s %lld): ", name_one, name_eval, name_two, (long long)one, name_eval, (long long)two); if (one > INT_MAX) { ksft_print_msg("Miscalculation! Measurement went negative: %lld\n", (long long)one); - return 1; + good = false; + goto out; } if (two > INT_MAX) { ksft_print_msg("Miscalculation! Measurement went negative: %lld\n", (long long)two); - return 1; + good = false; + goto out; } good = eval(one, two); printf("%s\n", good ? "✔️" : "❌"); +out: + ksft_test_result(good, "%s %s %s\n", name_one, name_eval, name_two); + return good ? 0 : 1; } @@ -142,9 +154,13 @@ int main(int argc, char *argv[]) unsigned long long samples, calc; unsigned long long native, filter1, filter2, bitmap1, bitmap2; unsigned long long entry, per_filter1, per_filter2; + bool skip = false; setbuf(stdout, NULL); + ksft_print_header(); + ksft_set_plan(7); + ksft_print_msg("Running on:\n"); ksft_print_msg(""); system("uname -a"); @@ -202,8 +218,10 @@ int main(int argc, char *argv[]) #define ESTIMATE(fmt, var, what) do { \ var = (what); \ ksft_print_msg("Estimated " fmt ": %llu ns\n", var); \ - if (var > INT_MAX) \ - goto more_samples; \ + if (var > INT_MAX) { \ + skip = true; \ + ret |= 1; \ + } \ } while (0) ESTIMATE("total seccomp overhead for 1 bitmapped filter", calc, @@ -222,30 +240,34 @@ int main(int argc, char *argv[]) (filter2 - native - entry) / 4); ksft_print_msg("Expectations:\n"); - ret |= compare("native", "≤", "1 bitmap", native, le, bitmap1); - bits = compare("native", "≤", "1 filter", native, le, filter1); + ret |= compare("native", "≤", "1 bitmap", native, le, bitmap1, + skip); + bits = compare("native", "≤", "1 filter", native, le, filter1, + skip); if (bits) - goto more_samples; + skip = true; ret |= compare("per-filter (last 2 diff)", "≈", "per-filter (filters / 4)", - per_filter1, approx, per_filter2); + per_filter1, approx, per_filter2, skip); bits = compare("1 bitmapped", "≈", "2 bitmapped", - bitmap1 - native, approx, bitmap2 - native); + bitmap1 - native, approx, bitmap2 - native, skip); if (bits) { ksft_print_msg("Skipping constant action bitmap expectations: they appear unsupported.\n"); - goto out; + skip = true; } - ret |= compare("entry", "≈", "1 bitmapped", entry, approx, bitmap1 - native); - ret |= compare("entry", "≈", "2 bitmapped", entry, approx, bitmap2 - native); + ret |= compare("entry", "≈", "1 bitmapped", entry, approx, + bitmap1 - native, skip); + ret |= compare("entry", "≈", "2 bitmapped", entry, approx, + bitmap2 - native, skip); ret |= compare("native + entry + (per filter * 4)", "≈", "4 filters total", - entry + (per_filter1 * 4) + native, approx, filter2); - if (ret == 0) - goto out; + entry + (per_filter1 * 4) + native, approx, filter2, + skip); -more_samples: - ksft_print_msg("Saw unexpected benchmark result. Try running again with more samples?\n"); -out: - return 0; + if (ret) { + ksft_print_msg("Saw unexpected benchmark result. Try running again with more samples?\n"); + } + + ksft_finished(); }