[3/3] selftests: cachestat: test for cachestat availability

  As cachestat is a new syscall, it won't be available on older kernels,
for instance those running on a build machine. In this case, a run
reports all tests as "not ok" at the moment.

Test for the cachestat syscall availability first, before doing further
tests, and bail out early with a TAP SKIP comment.

This also uses the opportunity to add the proper TAP headers, and add
one check for the syscall error handling (illegal file descriptor).

Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
 .../selftests/cachestat/test_cachestat.c      | 22 ++++++++++++++++++-
 1 file changed, 21 insertions(+), 1 deletion(-)
  

On Tue, 15 Aug 2023 16:25:54 -0700
Nhat Pham <nphamcs@gmail.com> wrote:

Hi Nhat,

many thanks for having a look!

> On Tue, Aug 15, 2023 at 8:56 AM Andre Przywara <andre.przywara@arm.com> wrote:
> >
> > As cachestat is a new syscall, it won't be available on older kernels,
> > for instance those running on a build machine. In this case, a run
> > reports all tests as "not ok" at the moment.  
> Interesting - I was under the assumption that if you backported the
> selftests for cachestat, you would also backport the syscall's implementation
> and wiring.

Well, just running the tests on the kernel you just built is only one
use case. I build the tests from latest git HEAD, then copy them to a
target system with some kernel running. Or I just build the tests and run
them for regression testing on my build system with a distro kernel, which
is Ubuntu's 5.15 flavour, in my case.
The documentation explicitly mentions that selftests should work on older
kernels (copying the normal userland compatibility requirements), check
the second paragraph of Documentation/dev-tools/kselftest.rst.

> But yeah, I guess if you build with !CONFIG_CACHESTAT_SYSCALL,
> these tests would fail.
> >
> > Test for the cachestat syscall availability first, before doing further
> > tests, and bail out early with a TAP SKIP comment.
> >
> > This also uses the opportunity to add the proper TAP headers, and add
> > one check for the syscall error handling (illegal file descriptor).  
> Thanks for the addition!
> >
> > Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> > ---
> >  .../selftests/cachestat/test_cachestat.c      | 22 ++++++++++++++++++-
> >  1 file changed, 21 insertions(+), 1 deletion(-)
> >
> > diff --git a/tools/testing/selftests/cachestat/test_cachestat.c b/tools/testing/selftests/cachestat/test_cachestat.c
> > index a5a4ac8dcb76c..77620e7ecf562 100644
> > --- a/tools/testing/selftests/cachestat/test_cachestat.c
> > +++ b/tools/testing/selftests/cachestat/test_cachestat.c
> > @@ -15,6 +15,8 @@
> >
> >  #include "../kselftest.h"
> >
> > +#define NR_TESTS       8
> > +
> >  static const char * const dev_files[] = {
> >         "/dev/zero", "/dev/null", "/dev/urandom",
> >         "/proc/version", "/proc"
> > @@ -235,7 +237,25 @@ bool test_cachestat_shmem(void)
> >
> >  int main(void)
> >  {
> > -       int ret = 0;
> > +       int ret;
> > +
> > +       ksft_print_header();
> > +
> > +       ret = syscall(__NR_cachestat, -1, NULL, NULL, 0);
> > +       if (ret == -1 && errno == ENOSYS) {  
> nit: if (ret && errno == ENOSYS) sounds cleaner, but up to you.

Do you ever return a positive value other than 0? I think technically
errno is only valid when the return value is -1, so in the error case,
which I wanted to test here explicitly.
Some syscall selftests (I checked landlock the other day) do very elaborate
testing of the error path, trying to carefully cover all corner cases:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/testing/selftests/landlock/base_test.c#n24

So this test was inspired by that, but I didn't want to go that far here
;-)

> > +               printf("1..0 # Skipped: cachestat syscall not available\n");  
> nit: perhaps ksft_print_msg()?

Ah, yes, of course!

> > +               return KSFT_SKIP;
> > +       }
> > +
> > +       ksft_set_plan(NR_TESTS);
> > +
> > +       if (ret == -1 && errno == EBADF) {
> > +               ksft_test_result_pass("bad file descriptor recognized\n");
> > +               ret = 0;
> > +       } else {
> > +               ksft_test_result_fail("bad file descriptor ignored\n");
> > +               ret = 1;
> > +       }  
> Nice!
> >
> >         for (int i = 0; i < 5; i++) {
> >                 const char *dev_filename = dev_files[i];
> > --
> > 2.25.1
> >  
> Nitpicking aside:
> Acked-by: Nhat Pham <nphamcs@gmail.com>

Thanks, I will send a v2 later, using ksft_print_msg(). But first I will
try if I can detect a tmpfs instance without boiling the ocean.

Cheers,
Andre
  

On Wed, 16 Aug 2023 11:11:49 -0600
Shuah Khan <skhan@linuxfoundation.org> wrote:

Hi,

> On 8/15/23 09:56, Andre Przywara wrote:
> > As cachestat is a new syscall, it won't be available on older kernels,
> > for instance those running on a build machine. In this case, a run
> > reports all tests as "not ok" at the moment.
> > 
> > Test for the cachestat syscall availability first, before doing further
> > tests, and bail out early with a TAP SKIP comment.
> > 
> > This also uses the opportunity to add the proper TAP headers, and add
> > one check for the syscall error handling (illegal file descriptor).
> > 
> > Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> > ---
> >   .../selftests/cachestat/test_cachestat.c      | 22 ++++++++++++++++++-
> >   1 file changed, 21 insertions(+), 1 deletion(-)
> > 
> > diff --git a/tools/testing/selftests/cachestat/test_cachestat.c b/tools/testing/selftests/cachestat/test_cachestat.c
> > index a5a4ac8dcb76c..77620e7ecf562 100644
> > --- a/tools/testing/selftests/cachestat/test_cachestat.c
> > +++ b/tools/testing/selftests/cachestat/test_cachestat.c
> > @@ -15,6 +15,8 @@
> >   
> >   #include "../kselftest.h"
> >   
> > +#define NR_TESTS	8
> > +
> >   static const char * const dev_files[] = {
> >   	"/dev/zero", "/dev/null", "/dev/urandom",
> >   	"/proc/version", "/proc"
> > @@ -235,7 +237,25 @@ bool test_cachestat_shmem(void)
> >   
> >   int main(void)
> >   {
> > -	int ret = 0;
> > +	int ret;
> > +
> > +	ksft_print_header();
> > +
> > +	ret = syscall(__NR_cachestat, -1, NULL, NULL, 0);
> > +	if (ret == -1 && errno == ENOSYS) {
> > +		printf("1..0 # Skipped: cachestat syscall not available\n");
> > +		return KSFT_SKIP;  
> What happens when other errors besides ENOSYS? The test shouldn't
> continue.

-1 is an illegal file descriptor, and this is checked below (still using
the same ret and errno), but reported using the normal framework.
This check above is done early, before we even announce the plan, so that
we can skip *all* of the tests, since they don't make any sense when the
syscall is not available at all.

Does that make sense?

Cheers,
Andre

> 
> > +	}
> > +
> > +	ksft_set_plan(NR_TESTS);
> > +
> > +	if (ret == -1 && errno == EBADF) {
> > +		ksft_test_result_pass("bad file descriptor recognized\n");
> > +		ret = 0;
> > +	} else {
> > +		ksft_test_result_fail("bad file descriptor ignored\n");
> > +		ret = 1;
> > +	}
> >   
> >   	for (int i = 0; i < 5; i++) {
> >   		const char *dev_filename = dev_files[i];  
> 
> thanks,
> -- Shuah