[RFC,v2,11/20] objtool: Flesh out warning related to pv_ops[] calls

  I had to look into objtool itself to understand what this warning was
about; make it more explicit.

Signed-off-by: Valentin Schneider <vschneid@redhat.com>
---
 tools/objtool/check.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
  

On Thu, Jul 20, 2023 at 05:30:47PM +0100, Valentin Schneider wrote:
> I had to look into objtool itself to understand what this warning was
> about; make it more explicit.
> 
> Signed-off-by: Valentin Schneider <vschneid@redhat.com>
> ---
>  tools/objtool/check.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tools/objtool/check.c b/tools/objtool/check.c
> index 8936a05f0e5ac..d308330f2910e 100644
> --- a/tools/objtool/check.c
> +++ b/tools/objtool/check.c
> @@ -3360,7 +3360,7 @@ static bool pv_call_dest(struct objtool_file *file, struct instruction *insn)
>  
>  	list_for_each_entry(target, &file->pv_ops[idx].targets, pv_target) {
>  		if (!target->sec->noinstr) {
> -			WARN("pv_ops[%d]: %s", idx, target->name);
> +			WARN("pv_ops[%d]: indirect call to %s() leaves .noinstr.text section", idx, target->name);
>  			file->pv_ops[idx].clean = false;

This is an improvement, though I think it still results in two warnings,
with the second not-so-useful warning happening in validate_call().

Ideally it would only show a single warning, I guess that would need a
little bit of restructuring the code.
  

On 28/07/23 10:33, Josh Poimboeuf wrote:
> On Thu, Jul 20, 2023 at 05:30:47PM +0100, Valentin Schneider wrote:
>> I had to look into objtool itself to understand what this warning was
>> about; make it more explicit.
>>
>> Signed-off-by: Valentin Schneider <vschneid@redhat.com>
>> ---
>>  tools/objtool/check.c | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/tools/objtool/check.c b/tools/objtool/check.c
>> index 8936a05f0e5ac..d308330f2910e 100644
>> --- a/tools/objtool/check.c
>> +++ b/tools/objtool/check.c
>> @@ -3360,7 +3360,7 @@ static bool pv_call_dest(struct objtool_file *file, struct instruction *insn)
>>
>>      list_for_each_entry(target, &file->pv_ops[idx].targets, pv_target) {
>>              if (!target->sec->noinstr) {
>> -			WARN("pv_ops[%d]: %s", idx, target->name);
>> +			WARN("pv_ops[%d]: indirect call to %s() leaves .noinstr.text section", idx, target->name);
>>                      file->pv_ops[idx].clean = false;
>
> This is an improvement, though I think it still results in two warnings,
> with the second not-so-useful warning happening in validate_call().
>
> Ideally it would only show a single warning, I guess that would need a
> little bit of restructuring the code.

You're quite right - fabricating an artificial warning with a call to __flush_tlb_local():

  vmlinux.o: warning: objtool: pv_ops[1]: indirect call to native_flush_tlb_local() leaves .noinstr.text section
  vmlinux.o: warning: objtool: __flush_tlb_all_noinstr+0x4: call to {dynamic}() leaves .noinstr.text section

Interestingly the second one doesn't seem to have triggered the "pv_ops"
bit of call_dest_name. Seems like any call to insn_reloc(NULL, x) will
return NULL.

Trickling down the file yields:

  vmlinux.o: warning: objtool: pv_ops[1]: indirect call to native_flush_tlb_local() leaves .noinstr.text section
  vmlinux.o: warning: objtool: __flush_tlb_all_noinstr+0x4: call to pv_ops[0]() leaves .noinstr.text section

In my case (!PARAVIRT_XXL) pv_ops should look like:
  [0]: .cpu.io_delay
  [1]: .mmu.flush_tlb_user()

so pv_ops[1] looks right. Seems like pv_call_dest() gets it right because
it uses arch_dest_reloc_offset().

If I use the above to fix up validate_call(), would we still need
pv_call_dest() & co?

>
> --
> Josh
  

On Mon, Jul 31, 2023 at 12:16:59PM +0100, Valentin Schneider wrote:
> You're quite right - fabricating an artificial warning with a call to __flush_tlb_local():
> 
>   vmlinux.o: warning: objtool: pv_ops[1]: indirect call to native_flush_tlb_local() leaves .noinstr.text section
>   vmlinux.o: warning: objtool: __flush_tlb_all_noinstr+0x4: call to {dynamic}() leaves .noinstr.text section
> 
> Interestingly the second one doesn't seem to have triggered the "pv_ops"
> bit of call_dest_name. Seems like any call to insn_reloc(NULL, x) will
> return NULL.

Yeah, that's weird.

> Trickling down the file yields:
> 
>   vmlinux.o: warning: objtool: pv_ops[1]: indirect call to native_flush_tlb_local() leaves .noinstr.text section
>   vmlinux.o: warning: objtool: __flush_tlb_all_noinstr+0x4: call to pv_ops[0]() leaves .noinstr.text section
> 
> In my case (!PARAVIRT_XXL) pv_ops should look like:
>   [0]: .cpu.io_delay
>   [1]: .mmu.flush_tlb_user()
> 
> so pv_ops[1] looks right. Seems like pv_call_dest() gets it right because
> it uses arch_dest_reloc_offset().
> 
> If I use the above to fix up validate_call(), would we still need
> pv_call_dest() & co?

The functionality in pv_call_dest() is still needed because it goes
through all the possible targets for the .mmu.flush_tlb_user() pointer
-- xen_flush_tlb() and native_flush_tlb_local() -- and makes sure
they're noinstr.

Ideally it would only print a single warning for this case, something
like:

  vmlinux.o: warning: objtool: __flush_tlb_all_noinstr+0x4: indirect call to native_flush_tlb_local() leaves .noinstr.text section

I left out "pv_ops[1]" because it's already long enough :-)

It would need a little bit of code shuffling.  But it's really a
preexisting problem so don't feel compelled to fix it with this patch
set.
  

On Mon, Jul 31, 2023 at 04:36:31PM -0500, Josh Poimboeuf wrote:
> On Mon, Jul 31, 2023 at 12:16:59PM +0100, Valentin Schneider wrote:
> > You're quite right - fabricating an artificial warning with a call to __flush_tlb_local():
> > 
> >   vmlinux.o: warning: objtool: pv_ops[1]: indirect call to native_flush_tlb_local() leaves .noinstr.text section
> >   vmlinux.o: warning: objtool: __flush_tlb_all_noinstr+0x4: call to {dynamic}() leaves .noinstr.text section
> > 
> > Interestingly the second one doesn't seem to have triggered the "pv_ops"
> > bit of call_dest_name. Seems like any call to insn_reloc(NULL, x) will
> > return NULL.
> 
> Yeah, that's weird.
> 
> > Trickling down the file yields:
> > 
> >   vmlinux.o: warning: objtool: pv_ops[1]: indirect call to native_flush_tlb_local() leaves .noinstr.text section
> >   vmlinux.o: warning: objtool: __flush_tlb_all_noinstr+0x4: call to pv_ops[0]() leaves .noinstr.text section
> > 
> > In my case (!PARAVIRT_XXL) pv_ops should look like:
> >   [0]: .cpu.io_delay
> >   [1]: .mmu.flush_tlb_user()
> > 
> > so pv_ops[1] looks right. Seems like pv_call_dest() gets it right because
> > it uses arch_dest_reloc_offset().
> > 
> > If I use the above to fix up validate_call(), would we still need
> > pv_call_dest() & co?
> 
> The functionality in pv_call_dest() is still needed because it goes
> through all the possible targets for the .mmu.flush_tlb_user() pointer
> -- xen_flush_tlb() and native_flush_tlb_local() -- and makes sure
> they're noinstr.
> 
> Ideally it would only print a single warning for this case, something
> like:
> 
>   vmlinux.o: warning: objtool: __flush_tlb_all_noinstr+0x4: indirect call to native_flush_tlb_local() leaves .noinstr.text section

But then what for the case where there are multiple implementations and
more than one isn't noinstr? IIRC that is where these double prints came
from. One is the callsite (always one) and the second is the offending
implementation (but there could be more).

> I left out "pv_ops[1]" because it's already long enough :-)

The index number is useful when also looking at the assembler, which
IIRC is an indexed indirect call.
  

On Mon, Jul 31, 2023 at 11:46:12PM +0200, Peter Zijlstra wrote:
> > Ideally it would only print a single warning for this case, something
> > like:
> > 
> >   vmlinux.o: warning: objtool: __flush_tlb_all_noinstr+0x4: indirect call to native_flush_tlb_local() leaves .noinstr.text section
> 
> But then what for the case where there are multiple implementations and
> more than one isn't noinstr?

The warning would be in the loop in pv_call_dest(), so it would
potentially print multiple warnings, one for each potential dest.

> IIRC that is where these double prints came from. One is the callsite
> (always one) and the second is the offending implementation (but there
> could be more).

It's confusing to warn about the call site and the destination in two
separate warnings.  That's why I'm proposing combining them into a
single warning (which still could end up as multiple warnings if there
are multiple affected dests).

> > I left out "pv_ops[1]" because it's already long enough :-)
> 
> The index number is useful when also looking at the assembler, which
> IIRC is an indexed indirect call.

Ok, so something like so?

  vmlinux.o: warning: objtool: __flush_tlb_all_noinstr+0x4: indirect call to pv_ops[1] (native_flush_tlb_local) leaves .noinstr.text section
  

On Tue, Aug 01, 2023 at 11:06:36AM -0500, Josh Poimboeuf wrote:
> On Mon, Jul 31, 2023 at 11:46:12PM +0200, Peter Zijlstra wrote:
> > > Ideally it would only print a single warning for this case, something
> > > like:
> > > 
> > >   vmlinux.o: warning: objtool: __flush_tlb_all_noinstr+0x4: indirect call to native_flush_tlb_local() leaves .noinstr.text section
> > 
> > But then what for the case where there are multiple implementations and
> > more than one isn't noinstr?
> 
> The warning would be in the loop in pv_call_dest(), so it would
> potentially print multiple warnings, one for each potential dest.
> 
> > IIRC that is where these double prints came from. One is the callsite
> > (always one) and the second is the offending implementation (but there
> > could be more).
> 
> It's confusing to warn about the call site and the destination in two
> separate warnings.  That's why I'm proposing combining them into a
> single warning (which still could end up as multiple warnings if there
> are multiple affected dests).
> 
> > > I left out "pv_ops[1]" because it's already long enough :-)
> > 
> > The index number is useful when also looking at the assembler, which
> > IIRC is an indexed indirect call.
> 
> Ok, so something like so?
> 
>   vmlinux.o: warning: objtool: __flush_tlb_all_noinstr+0x4: indirect call to pv_ops[1] (native_flush_tlb_local) leaves .noinstr.text section

Sure, that all would work I suppose.