diff mbox series

[v2,04/10] ftrace: Store direct called addresses in their ops

Message ID 20230207182135.2671106-5-revest@chromium.org
State New
Headers
Series Add ftrace direct call for arm64 |

Commit Message

Florent Revest Feb. 7, 2023, 6:21 p.m. UTC 
  All direct calls are now registered using the register_ftrace_direct API
so each ops can jump to only one direct-called trampoline.

By storing the direct called trampoline address directly in the ops we
can save one hashmap lookup in the direct call ops and implement arm64
direct calls on top of call ops.

Signed-off-by: Florent Revest <revest@chromium.org>
---
 include/linux/ftrace.h | 3 +++
 kernel/trace/ftrace.c  | 6 ++++--
 2 files changed, 7 insertions(+), 2 deletions(-)

Comments

Steven Rostedt March 15, 2023, 11:43 p.m. UTC | #1 
On Tue,  7 Feb 2023 19:21:29 +0100
Florent Revest <revest@chromium.org> wrote:

> @@ -5445,6 +5445,7 @@ __modify_ftrace_direct(struct ftrace_ops *ops, unsigned long addr)
>  	/* Enable the tmp_ops to have the same functions as the direct ops */
>  	ftrace_ops_init(&tmp_ops);
>  	tmp_ops.func_hash = ops->func_hash;
> +	tmp_ops.direct_call = addr;
>  
>  	err = register_ftrace_function_nolock(&tmp_ops);
>  	if (err)
> @@ -5466,6 +5467,7 @@ __modify_ftrace_direct(struct ftrace_ops *ops, unsigned long addr)
>  			entry->direct = addr;
>  		}
>  	}
> +	WRITE_ONCE(ops->direct_call, addr);

I'm curious about the use of WRITE_ONCE(). It should not go outside the
mutex barrier.

-- Steve

>  
>  	mutex_unlock(&ftrace_lock);
>
Florent Revest March 16, 2023, 3:40 p.m. UTC | #2 
On Thu, Mar 16, 2023 at 12:43 AM Steven Rostedt <rostedt@goodmis.org> wrote:
>
> On Tue,  7 Feb 2023 19:21:29 +0100
> Florent Revest <revest@chromium.org> wrote:
>
> > @@ -5445,6 +5445,7 @@ __modify_ftrace_direct(struct ftrace_ops *ops, unsigned long addr)
> >       /* Enable the tmp_ops to have the same functions as the direct ops */
> >       ftrace_ops_init(&tmp_ops);
> >       tmp_ops.func_hash = ops->func_hash;
> > +     tmp_ops.direct_call = addr;
> >
> >       err = register_ftrace_function_nolock(&tmp_ops);
> >       if (err)
> > @@ -5466,6 +5467,7 @@ __modify_ftrace_direct(struct ftrace_ops *ops, unsigned long addr)
> >                       entry->direct = addr;
> >               }
> >       }
> > +     WRITE_ONCE(ops->direct_call, addr);
>
> I'm curious about the use of WRITE_ONCE(). It should not go outside the
> mutex barrier.

This WRITE_ONCE was originally suggested by Mark here:
https://lore.kernel.org/all/Y9vW99htjOphDXqY@FVFF77S0Q05N.cambridge.arm.com/#t

My understanding is that it's not so much about avoiding re-ordering
but rather about avoiding store tearing since a ftrace_caller
trampoline could concurrently read ops->direct_call. Does that make
sense ?

> -- Steve
>
> >
> >       mutex_unlock(&ftrace_lock);
> >
Steven Rostedt March 16, 2023, 3:45 p.m. UTC | #3 
On Thu, 16 Mar 2023 16:40:48 +0100
Florent Revest <revest@chromium.org> wrote:

> > > @@ -5466,6 +5467,7 @@ __modify_ftrace_direct(struct ftrace_ops *ops, unsigned long addr)
> > >                       entry->direct = addr;
> > >               }
> > >       }
> > > +     WRITE_ONCE(ops->direct_call, addr);  
> >
> > I'm curious about the use of WRITE_ONCE(). It should not go outside the
> > mutex barrier.  
> 
> This WRITE_ONCE was originally suggested by Mark here:
> https://lore.kernel.org/all/Y9vW99htjOphDXqY@FVFF77S0Q05N.cambridge.arm.com/#t
> 
> My understanding is that it's not so much about avoiding re-ordering
> but rather about avoiding store tearing since a ftrace_caller
> trampoline could concurrently read ops->direct_call. Does that make
> sense ?

Yes, but a comment needs to be added:

     /* Prevent store tearing on some archs */
     WRITE_ONCE(ops->direct_call, addr);  

Or something to that affect. Otherwise I can see it confusing others in the
future. And probably me too, as I'll forget why it was a WRITE_ONCE() by
next month. ;-)

-- Steve
Florent Revest March 16, 2023, 4:15 p.m. UTC | #4 
On Thu, Mar 16, 2023 at 4:45 PM Steven Rostedt <rostedt@goodmis.org> wrote:
>
> On Thu, 16 Mar 2023 16:40:48 +0100
> Florent Revest <revest@chromium.org> wrote:
>
> > > > @@ -5466,6 +5467,7 @@ __modify_ftrace_direct(struct ftrace_ops *ops, unsigned long addr)
> > > >                       entry->direct = addr;
> > > >               }
> > > >       }
> > > > +     WRITE_ONCE(ops->direct_call, addr);
> > >
> > > I'm curious about the use of WRITE_ONCE(). It should not go outside the
> > > mutex barrier.
> >
> > This WRITE_ONCE was originally suggested by Mark here:
> > https://lore.kernel.org/all/Y9vW99htjOphDXqY@FVFF77S0Q05N.cambridge.arm.com/#t
> >
> > My understanding is that it's not so much about avoiding re-ordering
> > but rather about avoiding store tearing since a ftrace_caller
> > trampoline could concurrently read ops->direct_call. Does that make
> > sense ?
>
> Yes, but a comment needs to be added:
>
>      /* Prevent store tearing on some archs */
>      WRITE_ONCE(ops->direct_call, addr);
>
> Or something to that affect. Otherwise I can see it confusing others in the
> future. And probably me too, as I'll forget why it was a WRITE_ONCE() by
> next month. ;-)

Definitely :) I was myself confused after a few weeks of adding it so
I'll add a clarifying comment. Thanks!
diff mbox series

Patch

diff --git a/include/linux/ftrace.h b/include/linux/ftrace.h
index a7dbd307c3a4..84f717f8959e 100644
--- a/include/linux/ftrace.h
+++ b/include/linux/ftrace.h
@@ -321,6 +321,9 @@  struct ftrace_ops {
 	unsigned long			trampoline_size;
 	struct list_head		list;
 	ftrace_ops_func_t		ops_func;
+#ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS
+	unsigned long			direct_call;
+#endif
 #endif
 };
 
diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c
index cb77a0a208c7..dfa5f34ec320 100644
--- a/kernel/trace/ftrace.c
+++ b/kernel/trace/ftrace.c
@@ -2577,9 +2577,8 @@  ftrace_add_rec_direct(unsigned long ip, unsigned long addr,
 static void call_direct_funcs(unsigned long ip, unsigned long pip,
 			      struct ftrace_ops *ops, struct ftrace_regs *fregs)
 {
-	unsigned long addr;
+	unsigned long addr = ops->direct_call;
 
-	addr = ftrace_find_rec_direct(ip);
 	if (!addr)
 		return;
 
@@ -5375,6 +5374,7 @@  int register_ftrace_direct(struct ftrace_ops *ops, unsigned long addr)
 	ops->func = call_direct_funcs;
 	ops->flags = MULTI_FLAGS;
 	ops->trampoline = FTRACE_REGS_ADDR;
+	ops->direct_call = addr;
 
 	err = register_ftrace_function_nolock(ops);
 
@@ -5445,6 +5445,7 @@  __modify_ftrace_direct(struct ftrace_ops *ops, unsigned long addr)
 	/* Enable the tmp_ops to have the same functions as the direct ops */
 	ftrace_ops_init(&tmp_ops);
 	tmp_ops.func_hash = ops->func_hash;
+	tmp_ops.direct_call = addr;
 
 	err = register_ftrace_function_nolock(&tmp_ops);
 	if (err)
@@ -5466,6 +5467,7 @@  __modify_ftrace_direct(struct ftrace_ops *ops, unsigned long addr)
 			entry->direct = addr;
 		}
 	}
+	WRITE_ONCE(ops->direct_call, addr);
 
 	mutex_unlock(&ftrace_lock);