From patchwork Tue May 30 10:56:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhangjin Wu X-Patchwork-Id: 100752 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp2093700vqr; Tue, 30 May 2023 04:04:53 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5p+FUM7xXaWjBHYttmCfzKMwXhY9N1RQBsRqBSh1iQceFx9F1Gyvu0TxGDAkIX5cz7DmI1 X-Received: by 2002:a05:6a20:1450:b0:10a:ba3a:42dd with SMTP id a16-20020a056a20145000b0010aba3a42ddmr2130081pzi.44.1685444693136; Tue, 30 May 2023 04:04:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685444693; cv=none; d=google.com; s=arc-20160816; b=pejoF1oMTNn5BTe4cvO0s1D+EN0+QsAjKRzJHXq0YpMmiaG3VNeminrvF6EXoctmhR M/N0+X0rX/ppdrMLAbainw4eqbWNdnvCp/pvnBSvtEbfg0L9npG0/aj/SNFdj0u2NciB mNxWZcELGIxwj+ZsYPKGzeB9LPB/90TaNzNYrvOwJcGQaVgoNMIg4AqwC4ScZAw5UtZB LN97BkzD59Wo0H7vE1sRVWXbUktNaDBagZupZfFsW7wGtked4/N+R9a6evC5Pk12yD3U 6J53BH8bjtVr7eXiroJWOeM64EOyVHxpThj+vUAcD9zbr4MkZr58+FvdQpRRUom6cE7u KGbg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:feedback-id:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from; bh=g12FJ5ZYTLzc3CPir9WNEzp9sko8pQzQva70aFdxjNc=; b=SvgjdijVmxVO9LfIEG1h+Mxi2VnVVr5Sev34S9g2/Rw9q5+s+aMvfqOyjHCkawANfH /h+ZeEUhWW3YO0Eoxzs1KI2p5s8Zs/nO6tyjg7TqWMbjJEU3Y9GYDAVhvWHQjUHNHGs+ 6lQfC8AsDu2pUBJ9Quyx7VgaxrmymESbKpAV+5p5zS5AFLaj+XRMvGTg6NwW1kTi45Ql m34Sx8nbOEIjX4fd6t0U7ROhLpc5yHpMFdR9ey+dD7CIkp45qjpFdkSmfQgDxsg8aNrM LToj+y4fsnLQpgxPuIk3tb9PsG+NCiYEmOublXS9GIWQyKkC7LaA/3abRYf4pcVwSpfn j+eA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 24-20020a631258000000b00520603e8e25si7803854pgs.67.2023.05.30.04.04.30; Tue, 30 May 2023 04:04:52 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229739AbjE3K4Y (ORCPT + 99 others); Tue, 30 May 2023 06:56:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44664 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231361AbjE3K4V (ORCPT ); Tue, 30 May 2023 06:56:21 -0400 Received: from bg4.exmail.qq.com (bg4.exmail.qq.com [43.154.221.58]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E598AF7; Tue, 30 May 2023 03:56:17 -0700 (PDT) X-QQ-mid: bizesmtp72t1685444172tvjqxqzn Received: from linux-lab-host.localdomain ( [119.123.130.226]) by bizesmtp.qq.com (ESMTP) with id ; Tue, 30 May 2023 18:56:11 +0800 (CST) X-QQ-SSF: 01200000000000D0V000000A0000000 X-QQ-FEAT: IV30oiqrgAYLeAVOtqPY/w4sV51ufOZgvH9veDK9Ln5rvQZwJVqfvQebBNkb0 rWdg0XWxELEr+D+JJDpmJRKXEGbTrqUmljotAwMSeKgbivwAh8Yc83kjym3D0j3G2Y/rfbT rmDMziIaF1RHSZ0VHwappE4cH+2ibJvPOUXuZeZixvII0+kcxU/CNsIAfWDAPnGiPA/X4Ry uhcwpAPuHw//KlJZuq0nj2KBV/mhrO1z6IjLwWZl6tEPenU5NHMlwsxmdxFUZgT39wwGWzW zUWW0R3DR8s36VqJKPbbMYeeSntiCd0UAIcDVkGpuo+W1S9x5EyFdJl4W5eKH6bam/lftYQ 9/imcDdRSqPSe6FQ4g46x2VQalG7yfX6OA+ka7s9XC8huDYjbWd2rJlVOqjkRRquGnxwI1c X-QQ-GoodBg: 0 X-BIZMAIL-ID: 16230640011585094560 From: Zhangjin Wu To: w@1wt.eu Cc: falcon@tinylab.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-riscv@lists.infradead.org, thomas@t-8ch.de Subject: [PATCH 1/4] selftests/nolibc: allow rerun with the same settings Date: Tue, 30 May 2023 18:56:10 +0800 Message-Id: <32d4ec1c3cabae5c1baabf85627b81d6ede01518.1685443199.git.falcon@tinylab.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 X-QQ-SENDSIZE: 520 Feedback-ID: bizesmtp:tinylab.org:qybglogicsvrsz:qybglogicsvrsz3a-3 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767316854892564659?= X-GMAIL-MSGID: =?utf-8?q?1767316854892564659?= Record the user settings from NOLIBC_TEST and allow reuse them in another run iteration. This allows to rerun the test cases with the same setting. Signed-off-by: Zhangjin Wu --- tools/testing/selftests/nolibc/nolibc-test.c | 46 ++++++++++++-------- 1 file changed, 29 insertions(+), 17 deletions(-) diff --git a/tools/testing/selftests/nolibc/nolibc-test.c b/tools/testing/selftests/nolibc/nolibc-test.c index fd7515f6b1d2..be718fa5dc86 100644 --- a/tools/testing/selftests/nolibc/nolibc-test.c +++ b/tools/testing/selftests/nolibc/nolibc-test.c @@ -46,6 +46,9 @@ char **environ; /* definition of a series of tests */ struct test { const char *name; /* test name */ + int min; + int max; + int run; int (*func)(int min, int max); /* handler */ }; @@ -940,12 +943,12 @@ int prepare(void) } /* This is the definition of known test names, with their functions */ -static const struct test test_names[] = { +static struct test test_names[] = { /* add new tests here */ - { .name = "syscall", .func = run_syscall }, - { .name = "stdlib", .func = run_stdlib }, - { .name = "vfprintf", .func = run_vfprintf }, - { .name = "protection", .func = run_protection }, + { .name = "syscall", .min = 0, .max = INT_MAX, .run = -1, .func = run_syscall }, + { .name = "stdlib", .min = 0, .max = INT_MAX, .run = -1, .func = run_stdlib }, + { .name = "vfprintf", .min = 0, .max = INT_MAX, .run = -1, .func = run_vfprintf }, + { .name = "protection", .min = 0, .max = INT_MAX, .run = -1, .func = run_protection }, { 0 } }; @@ -994,7 +997,11 @@ int main(int argc, char **argv, char **envp) break; } - if (test_names[idx].name) { + if (!test_names[idx].name) { + printf("Ignoring unknown test name '%s'\n", test); + } else { + test_names[idx].run = 1; + /* The test was named, it will be called at least * once. We may have an optional range at * here, which defaults to the full range. @@ -1022,27 +1029,32 @@ int main(int argc, char **argv, char **envp) value = colon; } - /* now's time to call the test */ - printf("Running test '%s'\n", test_names[idx].name); - err = test_names[idx].func(min, max); - ret += err; - printf("Errors during this test: %d\n\n", err); + test_names[idx].min = min; + test_names[idx].max = max; } while (colon && *colon); - } else - printf("Ignoring unknown test name '%s'\n", test); + } test = comma; } while (test && *test); - } else { - /* no test mentioned, run everything */ + + /* disable the left tests */ for (idx = 0; test_names[idx].name; idx++) { - printf("Running test '%s'\n", test_names[idx].name); - err = test_names[idx].func(min, max); + if (test_names[idx].run != 1) + test_names[idx].run = 0; + } + } + + /* run everything or the test mentioned */ + for (idx = 0; test_names[idx].name; idx++) { + if (test_names[idx].run != 0) { + printf("Running test '%s', from %d to %d\n", test_names[idx].name, test_names[idx].min, test_names[idx].max); + err = test_names[idx].func(test_names[idx].min, test_names[idx].max); ret += err; printf("Errors during this test: %d\n\n", err); } } + printf("Total number of errors: %d\n", ret); if (getpid() == 1) { From patchwork Tue May 30 11:03:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhangjin Wu X-Patchwork-Id: 100755 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp2099598vqr; Tue, 30 May 2023 04:15:01 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5cC9cGMiE7iKdiNvRSP2Hjt7gsj7MlK6OnEXSUnu6CMgSfLaOqZUhAD0OwgQxpzOQTCRiY X-Received: by 2002:a05:6a20:1611:b0:110:390f:6806 with SMTP id l17-20020a056a20161100b00110390f6806mr2215263pzj.57.1685445301088; Tue, 30 May 2023 04:15:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685445301; cv=none; d=google.com; s=arc-20160816; b=EDHxzO3eDgP5uWCG8tvAJhXLl/3jW959VC/asHsrhAFmxrw431Y8vahBlqwuYboSTr /PpOSLu1Bhh5T3eq7SdIq5XTUOqWuxXmu3F3QVk68PDcaDvoyZIr4E+ZL1FyLsUvYNBE +bfr+BEPQgwii2SSxubL2Q9nF8kGlMiN+IALxVPxnu6i42TOePrE2Qa4wp0rFs9r3ZZW kioDTN3m7pNKTHF4rLjcJYQryWJ4B/R7gQmMuwmjr/ZwUmHIb1LqLmE+rP2GxPkbg/3m 1cjFqlqsQymhodVM2ZYWeJfR6WEq6gk4g3EYzRgPNY9ideROPvrz0KSCTLKy9E//4zmV rZpQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:feedback-id:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from; bh=o4MzGiVumF+GHC28YeyNDCSmWjLSsJipRem8JBPPVMY=; b=xDdXEwdMdPnZQs94cjstIarLY0w3OzzkaO+JrOIUnKNYieSHD53SiJunARqLFiu+wf 0bUYNzYSctqYj8dTmKBwKuLGBelNjDicsX3/TibO0MS8i78CMkgfpAby5Q/fyf2LE8yW wcvmm+q6XJ8/c7qOI/x3yPhH6d93kvQiwKkOlNnyq1h3RteIj3gKlVLg46vfjDFt03lc KnRTe89+ofWHVMomWJ1pqg4K/EYp0WAaybCNaGNGj8bbKt7FSIdwxkO14Ekk/g4kCGKL EwIhbDdlKPCgegawRL+aSPZB6plgEkCc9+PL1dTZtZOz70VlclwlJVp++6rXeoqOTxAJ PODQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id y7-20020aa793c7000000b0064d32440768si1533028pff.138.2023.05.30.04.14.46; Tue, 30 May 2023 04:15:01 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231770AbjE3LDm (ORCPT + 99 others); Tue, 30 May 2023 07:03:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49318 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231696AbjE3LDj (ORCPT ); Tue, 30 May 2023 07:03:39 -0400 Received: from bg4.exmail.qq.com (bg4.exmail.qq.com [43.155.67.158]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C8B05D9; Tue, 30 May 2023 04:03:37 -0700 (PDT) X-QQ-mid: bizesmtp67t1685444612t08p4xww Received: from linux-lab-host.localdomain ( [119.123.130.226]) by bizesmtp.qq.com (ESMTP) with id ; Tue, 30 May 2023 19:03:31 +0800 (CST) X-QQ-SSF: 01200000000000D0V000000A0000000 X-QQ-FEAT: C46Rb8GPIEc160nmUqFUHiFICVb8yJ5e11pqzLjXmS2/y5C6aYAhQb0+Cg9Tj G3ora3iXuOFD2kdO18FYoRZpzosUJ632Ckb/qSW3VPNCJimOOJtBcHY61W3wIe1KHd8Ku/m SNaBXZkq2EDAvjm3Z2XXHd9yz9p9K+IWKqic4/ShSxQLPa+MhRk+KsuDEZwgII7hmu9g0Hj OD4MUc56yMOVYx8jF1UmiV4xnslxBAOf8TydDIQUBAo1c/pZb2cTMLZhdKOyDiqzb5R5A3t sM7sjtpkHDsR1PkkNnf2L64rsCpz9zPGGU8rWiCfNGmSBMcaIdRNSmIkwsJ1IDxPvn6f6Lv /tN5H40PTJIIHyFAN1kYgY95zls1dyh5pu+erFw4eSHkrjvLHL//EpzyhXCHPv1LNHhyht+ X-QQ-GoodBg: 0 X-BIZMAIL-ID: 17642228944967858726 From: Zhangjin Wu To: w@1wt.eu Cc: falcon@tinylab.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-riscv@lists.infradead.org, thomas@t-8ch.de Subject: [PATCH 2/4] selftests/nolibc: add rerun support Date: Tue, 30 May 2023 19:03:30 +0800 Message-Id: <247d4d69a5e229446fc43efa49cc4fa36ccd600a.1685443199.git.falcon@tinylab.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 X-QQ-SENDSIZE: 520 Feedback-ID: bizesmtp:tinylab.org:qybglogicsvrsz:qybglogicsvrsz3a-3 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767317491791177776?= X-GMAIL-MSGID: =?utf-8?q?1767317491791177776?= The global 'run' setting is added to allow specify the running iterations, for example: NOLIBC_TEST=run:5,syscall:1 This setting allows to run the first syscall for 5 times. Signed-off-by: Zhangjin Wu --- tools/testing/selftests/nolibc/nolibc-test.c | 53 ++++++++++++++------ 1 file changed, 38 insertions(+), 15 deletions(-) diff --git a/tools/testing/selftests/nolibc/nolibc-test.c b/tools/testing/selftests/nolibc/nolibc-test.c index be718fa5dc86..b8fd7fcf56a6 100644 --- a/tools/testing/selftests/nolibc/nolibc-test.c +++ b/tools/testing/selftests/nolibc/nolibc-test.c @@ -956,9 +956,12 @@ int main(int argc, char **argv, char **envp) { int min = 0; int max = INT_MAX; + int run = 1; + int selected = 0; int ret = 0; int err; int idx; + int i; char *test; environ = envp; @@ -974,7 +977,7 @@ int main(int argc, char **argv, char **envp) /* the definition of a series of tests comes from either argv[1] or the * "NOLIBC_TEST" environment variable. It's made of a comma-delimited * series of test names and optional ranges: - * syscall:5-15[:.*],stdlib:8-10 + * run:3,syscall:5-15[:.*],stdlib:8-10 */ test = argv[1]; if (!test) @@ -992,14 +995,27 @@ int main(int argc, char **argv, char **envp) if (colon) *(colon++) = '\0'; - for (idx = 0; test_names[idx].name; idx++) { - if (strcmp(test, test_names[idx].name) == 0) + if (strcmp(test, "run") != 0) { + for (idx = 0; test_names[idx].name; idx++) { + if (strcmp(test, test_names[idx].name) == 0) + break; + } + } else { + value = colon; + if (value && *value) + run = atoi(value); + + test = comma; + if (test && *test) + continue; + else break; } if (!test_names[idx].name) { printf("Ignoring unknown test name '%s'\n", test); } else { + selected++; test_names[idx].run = 1; /* The test was named, it will be called at least @@ -1038,24 +1054,31 @@ int main(int argc, char **argv, char **envp) } while (test && *test); /* disable the left tests */ - for (idx = 0; test_names[idx].name; idx++) { - if (test_names[idx].run != 1) - test_names[idx].run = 0; + if (selected != 0) { + for (idx = 0; test_names[idx].name; idx++) { + if (test_names[idx].run != 1) + test_names[idx].run = 0; + } } } /* run everything or the test mentioned */ - for (idx = 0; test_names[idx].name; idx++) { - if (test_names[idx].run != 0) { - printf("Running test '%s', from %d to %d\n", test_names[idx].name, test_names[idx].min, test_names[idx].max); - err = test_names[idx].func(test_names[idx].min, test_names[idx].max); - ret += err; - printf("Errors during this test: %d\n\n", err); - } - } + printf("Running iteration(s): %d\n\n", run); + for (i = 0; i < run; i++) { + printf("Current iteration: %d\n\n", i + 1); + ret = 0; + for (idx = 0; test_names[idx].name; idx++) { + if (test_names[idx].run != 0) { + printf("Running test '%s', from %d to %d\n", test_names[idx].name, test_names[idx].min, test_names[idx].max); + err = test_names[idx].func(test_names[idx].min, test_names[idx].max); + ret += err; + printf("Errors during this test: %d\n\n", err); + } + } - printf("Total number of errors: %d\n", ret); + printf("Total number of errors in the %d iteration(s): %d\n\n", i + 1, ret); + } if (getpid() == 1) { /* we're running as init, there's no other process on the From patchwork Tue May 30 11:05:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhangjin Wu X-Patchwork-Id: 100760 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp2102328vqr; Tue, 30 May 2023 04:19:59 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5c5tceNw6j3PsZkWarVeH3jUVdHOplFuyH7ZdWpTON3A55d/YI7xWcupX+I1+ibyE4W7VC X-Received: by 2002:a17:902:e84f:b0:1b0:43c2:b9f5 with SMTP id t15-20020a170902e84f00b001b043c2b9f5mr2345787plg.42.1685445599185; Tue, 30 May 2023 04:19:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685445599; cv=none; d=google.com; s=arc-20160816; b=Ww8V0ObF5m0GmJBJUavDFEgpa4YzulxeiPh3ROKIIumFwyTTFFEbnDUEJ3HvU6X+Pu P1tIm5w7EHGNl/vtb6L+0t5Ww2TeJ9ZncTvYnhmY7rBu1cEUnLm0qMtZnYDQfZLA4AZR Lphoqx1UzOw67XnxW809ESRaOQtP+KJemGBrA+VqlBbaYL70Xw4f5ghnGVEERkUS8ylV eXZhz+1sahZh05qp2A6bbDgqKe0jarTxWDgI3wIJ0HNDfa2Sh0Q+7LpRwhzXFNmUgm/1 YDujKFdLqYWUaPCSEb1f/Sog5yoN1rOpgtLu4j++8ts7pwy1YdkidApAP4pSdW6blhz+ Te5A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:feedback-id:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from; bh=e/N4BuaaWunZ27I6QhmcVTOSIXfB8nlHXu7DkxH8wxY=; b=I/uYEnPgfAAKWNobGJtcLlANqKebWaBifSI2WSkx6wfq7i/w5kKnPrbtPKnptMgvoY ShdOEJSKy/IyFV96dVT2Mki9UkUCYDXWyNrowCKYg8CW+e8HSFgnloV8y34YUmupYVFr suVLj5zcIeK/7NT72ejkGZhGJoWcYf0wMavQtR2UFBd1lNo3vOJU0euRrid2+iNpo/TS azZc9GeApRjBHb3BXTdBucrngtfEYQmEQtCA9HloCEKxpKaZ23OFsnF96dre9+cgtKt2 +HW0coA9tPE45V7wYGhD3YqZDbWgRWWEcXkyRmlBhe5gW8ySEOYdBZjQoQbhbkH325w4 0LpA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id k9-20020a170902c40900b001b04c74f2a2si2055453plk.85.2023.05.30.04.19.45; Tue, 30 May 2023 04:19:59 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231797AbjE3LFm (ORCPT + 99 others); Tue, 30 May 2023 07:05:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50688 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231794AbjE3LFk (ORCPT ); Tue, 30 May 2023 07:05:40 -0400 Received: from bg4.exmail.qq.com (bg4.exmail.qq.com [43.154.221.58]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DE56AE5; Tue, 30 May 2023 04:05:37 -0700 (PDT) X-QQ-mid: bizesmtp81t1685444733t0w0xo7y Received: from linux-lab-host.localdomain ( [119.123.130.226]) by bizesmtp.qq.com (ESMTP) with id ; Tue, 30 May 2023 19:05:31 +0800 (CST) X-QQ-SSF: 01200000000000D0V000000A0000000 X-QQ-FEAT: qTKqCPEPPMCW9B1ysKllWVaS1wh2y0szvr/5UcMtnZhWY6OZeO5K63HX3I6/W atmSrvFCqT6B3PWqAH4eaVk9KM8ij4PXLpSQf/GawIFbw+MFWjENtkYYCm+naDAuhSnhq4o +atjjVW3SRGVc0dHTJSWfZkCjNxM95CxppAWnLgiyovSI3Q2XLudaYo7w/UBTy1gAkDjVuH FmnLv8kj/vZr2bVMhpwWuT1SvZ9BvFhk0SjKzItfPgeavl68QSlRSlZ6kr5e2b3WIC+1VD3 4wJ6i6DNRmFbIikucPYfFvJ8S+y5NTMbkGbrrH0vFJWYwDwNgKKXZtTYdpP036OuHTi3Y5e E+xAC2HBsqZ+DRTj+RP7AcVM7Do3v5xQzuBIULzGLkcx+0utUD8sFgEwkcye24itOyMdb8z X-QQ-GoodBg: 0 X-BIZMAIL-ID: 1177987213565950779 From: Zhangjin Wu To: w@1wt.eu Cc: falcon@tinylab.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-riscv@lists.infradead.org, thomas@t-8ch.de Subject: [PATCH 3/4] selftests/nolibc: add user space efault handler Date: Tue, 30 May 2023 19:05:29 +0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 X-QQ-SENDSIZE: 520 Feedback-ID: bizesmtp:tinylab.org:qybglogicsvrsz:qybglogicsvrsz3a-3 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767317804603722585?= X-GMAIL-MSGID: =?utf-8?q?1767317804603722585?= Some hooks are added to record the test case and the test context, while traps on invalid data pointer access, try to continue next test if possible. Signed-off-by: Zhangjin Wu --- tools/testing/selftests/nolibc/nolibc-test.c | 151 ++++++++++++++++++- 1 file changed, 149 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/nolibc/nolibc-test.c b/tools/testing/selftests/nolibc/nolibc-test.c index b8fd7fcf56a6..9f9a09529a4f 100644 --- a/tools/testing/selftests/nolibc/nolibc-test.c +++ b/tools/testing/selftests/nolibc/nolibc-test.c @@ -114,6 +114,149 @@ const char *errorname(int err) } } +/* emulate EFAULT return in user space with isigaction/sigsetjmp/siglongjmp */ +#ifndef NOLIBC +#ifndef NO_USER_SPACE_EFAULT +#define USER_SPACE_EFAULT +#endif +#endif + +#ifdef USER_SPACE_EFAULT +#include + +static int next_test = 0; +static int test_llen = 0; +static int test_sig = 0; +static int expect_sig = 0; +static int test_idx = 0; +static int test_ret = 0; +static int test_iteration = 0; +static int test_iterations = 0; +static sigjmp_buf mark; + +static int pad_spc(int llen, int cnt, const char *fmt, ...); +static struct test test_names[]; +typedef int (*func_t)(int min, int max); +static func_t test_func = NULL; + +#define CASE_SIG(sig) \ + case sig: return #sig + +/* returns the signal name or the decimal value for less common ones. */ +const char *signame(int sig) +{ + switch (sig) { + CASE_SIG(SIGSEGV); + default: + return itoa(sig); + } +} + +static void record_test_context(int idx, int iteration, int iterations) +{ + test_idx = idx; + test_iteration = iteration; + test_iterations = iterations; +} + +static void record_test_case(int test, int llen, int ret, char *name) +{ + test_llen = llen - 1; + test_ret = ret; + next_test = test + 1; +} + +static void restore_from_trap(void) +{ + int idx; + int err; + int i; + int min = 0; + int max = INT_MAX; + + test_llen += printf(" ! %d %s ", test_sig, signame(test_sig)); + if (test_sig == expect_sig) + pad_spc(test_llen, 64, "[OK]\n"); + else { + test_ret++; + pad_spc(test_llen, 64, "[FAIL]\n"); + } + + if (next_test <= test_names[test_idx].max) { + test_func = test_names[test_idx].func; + err = test_func(next_test, test_names[test_idx].max); + test_ret += err; + printf("Errors during this test: %d\n\n", err); + } + + for (i = test_iteration; i < test_iterations; i++) { + /* for current iterations */ + if (i == test_iteration) { + idx = test_idx + 1; + } else { + printf("Current iteration: %d\n\n", i + 1); + + /* for left iterations */ + idx = 0; + test_ret = 0; + } + + for (; test_names[idx].name; idx++) { + if (test_names[idx].run != 0) { + printf("Running test '%s'\n", test_names[idx].name); + record_test_context(idx, i, test_iterations); + err = test_names[idx].func(test_names[idx].min, test_names[idx].max); + test_ret += err; + printf("Errors during this test: %d\n\n", err); + } + } + printf("Total number of errors in the %d iteration(s): %d\n\n", i + 1, test_ret); + } +} + +static void trap_handler(int sig, siginfo_t *si, void *p) +{ + test_sig = sig; + if (sig != SIGKILL) + siglongjmp(mark, -1); +} + +static void register_expect_trap(int experr1, int experr2) +{ + if (experr1 == EFAULT || experr2 == EFAULT) + expect_sig = SIGSEGV; + else + expect_sig = 0; +} + +static void register_trap_handler(void) +{ + int ret = 0; + + struct sigaction sa = {0}; + sa.sa_sigaction = trap_handler; + sa.sa_flags = SA_SIGINFO; + ret = sigaction(SIGSEGV, &sa, NULL); + if (ret == -1) { + perror("sigaction"); + exit(1); + } + + if (sigsetjmp(mark, 1) != 0) { + restore_from_trap(); + exit(0); + } +} + +#define has_user_space_efault() (1) +#else +#define record_test_context(idx, iteration, iterations) do { } while (0) +#define record_test_case(test, llen, name, ret) do { } while (0) +#define register_expect_trap(experr1, experr2) do { } while (0) +#define register_trap_handler() do { } while (0) +#define has_user_space_efault() (0) +#endif + static void putcharn(char c, size_t n) { char buf[64]; @@ -304,7 +447,7 @@ static int expect_sysne(int expr, int llen, int val) #define EXPECT_SYSER2(cond, expr, expret, experr1, experr2) \ - do { if (!cond) pad_spc(llen, 64, "[SKIPPED]\n"); else ret += expect_syserr2(expr, expret, experr1, experr2, llen); } while (0) + do { if (!cond) pad_spc(llen, 64, "[SKIPPED]\n"); else { register_expect_trap(experr1, experr2); ret += expect_syserr2(expr, expret, experr1, experr2, llen); } } while (0) #define EXPECT_SYSER(cond, expr, expret, experr) \ EXPECT_SYSER2(cond, expr, expret, experr, 0) @@ -439,7 +582,7 @@ static int expect_strne(const char *expr, int llen, const char *cmp) /* declare tests based on line numbers. There must be exactly one test per line. */ #define CASE_TEST(name) \ - case __LINE__: llen += printf("%d %s", test, #name); + case __LINE__: llen += printf("%d %s", test, #name); record_test_case(test, llen, ret, #name); /* used by some syscall tests below */ @@ -974,6 +1117,9 @@ int main(int argc, char **argv, char **envp) if (getpid() == 1) prepare(); + /* register exception restore support if enabled */ + register_trap_handler(); + /* the definition of a series of tests comes from either argv[1] or the * "NOLIBC_TEST" environment variable. It's made of a comma-delimited * series of test names and optional ranges: @@ -1071,6 +1217,7 @@ int main(int argc, char **argv, char **envp) for (idx = 0; test_names[idx].name; idx++) { if (test_names[idx].run != 0) { printf("Running test '%s', from %d to %d\n", test_names[idx].name, test_names[idx].min, test_names[idx].max); + record_test_context(idx, i, run); err = test_names[idx].func(test_names[idx].min, test_names[idx].max); ret += err; printf("Errors during this test: %d\n\n", err); From patchwork Tue May 30 11:08:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhangjin Wu X-Patchwork-Id: 100764 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp2107784vqr; Tue, 30 May 2023 04:30:23 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4Db0AsJVmnqcPSfJDQRuhgefxKfsrGTEMNX5u7SX/y02i874655xhiAiLlLOvhEZ5Peyrk X-Received: by 2002:a05:6a20:4326:b0:111:d03c:2bf9 with SMTP id h38-20020a056a20432600b00111d03c2bf9mr2542696pzk.12.1685446223159; Tue, 30 May 2023 04:30:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685446223; cv=none; d=google.com; s=arc-20160816; b=zxYf/hP49LShrTXkXDczg2aUxBWVF1mft6Ts8gredmHvfis6VLFdPulS1MA4vA7bTh iIzz52DZmoV9XyhSQqeXcZe4CSMoJ1QV4RoSMSgIm+dMert0oijuRT4wHUf4IKP4g0uy 0zOjvIAsSf7P9r5nDd+lN9QmidkjxkMaBnRHFWK7eU5rTJvu5Vo/NNUWKXK/J29DsuUc PC6nxGru9eJTGm5gU4LNK8upNN160PVmdkl4hxGV8jTsNTAvBFs85jkZoFbMTL8AW1op r94Zv46Ty5bXWvEkg3C5LuwZwzjJuJlXn8y4qxcXEus/UtHZWyJh1Jia3CKzYIeRYR0X kUvg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:feedback-id:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from; bh=y5inpmTXmcwz0uYNJXefT3hUU2pJf2XiAzg5qlh0xkk=; b=gkFrzV0skweZsIeFOGwy8ogHRdbIx6r8s6wAjdbd1EzeyzJbXQQkcw4/Mp0UGDmrYJ xDmog/+gP00pR3E6hVEo7ri83Uv8FQ7CKdT2JU6d5I0FEnU3PA3t9EbIhvk2MNuB9ND2 rYepZ0/Nc6I5s7e8629uwpmfmfOeKC2JTQZ+WFqLADETLnuVkM2gq+qTkM/R5Z9mZMKU y7dtfxmRXwHj3O/DKULhk3xMqGBNlRFORi2TZm4SfCETwdMOu40vjJtO+aa/r9aQV3T0 slpWVAVGSpzvnb/VzJi1i5DPyOE2wUrBTVmnikQueB/lfaosILEQULc8QDENvADGO48K Gjww== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id bu26-20020a63295a000000b00534dbd0d219si662273pgb.301.2023.05.30.04.30.10; Tue, 30 May 2023 04:30:23 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230109AbjE3LJA (ORCPT + 99 others); Tue, 30 May 2023 07:09:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53348 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229535AbjE3LI5 (ORCPT ); Tue, 30 May 2023 07:08:57 -0400 Received: from bg4.exmail.qq.com (bg4.exmail.qq.com [43.154.221.58]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0CB89115; Tue, 30 May 2023 04:08:51 -0700 (PDT) X-QQ-mid: bizesmtp80t1685444926t3zkui9c Received: from linux-lab-host.localdomain ( [119.123.130.226]) by bizesmtp.qq.com (ESMTP) with id ; Tue, 30 May 2023 19:08:45 +0800 (CST) X-QQ-SSF: 01200000000000D0V000000A0000000 X-QQ-FEAT: 3M0okmaRx3gelAiWP5RoGbm4vjCvwZfJGtFNG1quZBYLTu71sj9RkgohxK606 a+q+BffDHTGq7hGQggq49Yc7b00q60k9Z0+rpko1hv5Qylsrh5Ru5dO7bo9KIloRo5ONmwQ GxwAFUFc1x9e2wF81rgdmybkUrKQU2849rW+NYEvCIxyaTAiLxdNDZOpLSFnXpfxH1KVTqA kWZIjSdm5YfAh46ijTIDBYTyYWqbWGo7WiVpC7CQnofDv7czmI4JoOidGAbFsd3/zSmwz2M FtPx6NDMyIuosDDfYQm61GFTCo77Rlg8fJ/j2ZP4WDsV+0lFlJ8ptgtuyJp1957ue6tLVJi LhH20tTwUUWtfUWeW8whrClc1ayhenyHlIvd/B9P/KKnGSV0gae3vPZ63kNu3sDlcC/m0Qm X-QQ-GoodBg: 0 X-BIZMAIL-ID: 1574217579638465516 From: Zhangjin Wu To: w@1wt.eu Cc: falcon@tinylab.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-riscv@lists.infradead.org, thomas@t-8ch.de Subject: [PATCH 4/4] selftests/nolibc: add user-space efault restore test case Date: Tue, 30 May 2023 19:08:43 +0800 Message-Id: <69e9464e92fe8c60a421d6571a1139980103e8fd.1685443199.git.falcon@tinylab.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 X-QQ-SENDSIZE: 520 Feedback-ID: bizesmtp:tinylab.org:qybglogicsvrsz:qybglogicsvrsz3a-3 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767318458604940627?= X-GMAIL-MSGID: =?utf-8?q?1767318458604940627?= while the libc supports sigaction/sigsetjmp/siglongjump, it is able to restore next test after an invalid data pointer access, add such a test case for these libcs, otherwise, skip it. With glibc/musl: 29 efault_handler ! 11 SIGSEGV [OK] With current nolibc: 29 efault_handler [SKIPPED] Signed-off-by: Zhangjin Wu --- tools/testing/selftests/nolibc/nolibc-test.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/tools/testing/selftests/nolibc/nolibc-test.c b/tools/testing/selftests/nolibc/nolibc-test.c index 9f9a09529a4f..6b4ebe4be4d6 100644 --- a/tools/testing/selftests/nolibc/nolibc-test.c +++ b/tools/testing/selftests/nolibc/nolibc-test.c @@ -248,6 +248,15 @@ static void register_trap_handler(void) } } +static int test_efault(void) +{ + char *addr = (void *)1; + + *addr = 'a'; + + return -1; +} + #define has_user_space_efault() (1) #else #define record_test_context(idx, iteration, iterations) do { } while (0) @@ -255,6 +264,7 @@ static void register_trap_handler(void) #define register_expect_trap(experr1, experr2) do { } while (0) #define register_trap_handler() do { } while (0) #define has_user_space_efault() (0) +#define test_efault(addr) (-1) #endif static void putcharn(char c, size_t n) @@ -690,6 +700,7 @@ int run_syscall(int min, int max) struct stat stat_buf; int euid0; int proc; + int efault; int test; int tmp; int ret = 0; @@ -701,6 +712,9 @@ int run_syscall(int min, int max) /* this will be used to skip certain tests that can't be run unprivileged */ euid0 = geteuid() == 0; + /* user-space efault handler support */ + efault = has_user_space_efault(); + for (test = min; test >= 0 && test <= max; test++) { int llen = 0; /* line length */ @@ -737,6 +751,7 @@ int run_syscall(int min, int max) CASE_TEST(dup2_m1); tmp = dup2(-1, 100); EXPECT_SYSER(1, tmp, -1, EBADF); if (tmp != -1) close(tmp); break; CASE_TEST(dup3_0); tmp = dup3(0, 100, 0); EXPECT_SYSNE(1, tmp, -1); close(tmp); break; CASE_TEST(dup3_m1); tmp = dup3(-1, 100, 0); EXPECT_SYSER(1, tmp, -1, EBADF); if (tmp != -1) close(tmp); break; + CASE_TEST(efault_handler); EXPECT_SYSER(efault, test_efault(), -1, EFAULT); break; CASE_TEST(execve_root); EXPECT_SYSER(1, execve("/", (char*[]){ [0] = "/", [1] = NULL }, NULL), -1, EACCES); break; CASE_TEST(fork); EXPECT_SYSZR(1, test_fork()); break; CASE_TEST(getdents64_root); EXPECT_SYSNE(1, test_getdents64("/"), -1); break;