[3/5] tick/nohz: Don't shutdown the lowres tick from itself

  In lowres dynticks mode, just like in highres dynticks mode, when there
is no tick to program in the future, the tick eventually gets
deactivated either:

* From the idle loop if in idle mode.
* From the IRQ exit if in full dynticks mode.

Therefore there is no need to deactivate it from the tick itself. This
just just brings more overhead in the idle tick path for no reason.

Fixes: 62c1256d5447 ("timers/nohz: Switch to ONESHOT_STOPPED in the low-res handler when the tick is stopped")
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
---
 kernel/time/tick-sched.c | 24 +++++++++++++-----------
 1 file changed, 13 insertions(+), 11 deletions(-)
  

On Tue, Sep 12, 2023 at 6:44 AM Frederic Weisbecker <frederic@kernel.org> wrote:
>
> In lowres dynticks mode, just like in highres dynticks mode, when there
> is no tick to program in the future, the tick eventually gets
> deactivated either:
>
> * From the idle loop if in idle mode.
> * From the IRQ exit if in full dynticks mode.
>
> Therefore there is no need to deactivate it from the tick itself. This
> just just brings more overhead in the idle tick path for no reason.
>
> Fixes: 62c1256d5447 ("timers/nohz: Switch to ONESHOT_STOPPED in the low-res handler when the tick is stopped")
> Signed-off-by: Frederic Weisbecker <frederic@kernel.org>

If on some weird hardware, say  ts->next_tick = KTIME_MAX but a
spurious timer interrupt went off and tick_nohz_handler() did get
called (yeah weird hypothetical situation), then in
tick_nohz_stop_tick() we might early return from:

/* Skip reprogram of event if its not changed */
if (ts->tick_stopped && (expires == ts->next_tick))

without no "eventual" reprogramming.

Maybe we should also reprogram with KTIME_MAX in such a situation?
Then we can get rid of it from tick_nohz_handler() for the common case
as you are doing.

So for weird hardware, with this patch we are not doing an extra
tick_program_event(KTIME_MAX, 1); like Nick was doing. That makes me a
tad bit nervous.

Otherwise your patch looks correct to me (for hardware that tends not
to misbehave).

thanks,

 - Joel


> ---
>  kernel/time/tick-sched.c | 24 +++++++++++++-----------
>  1 file changed, 13 insertions(+), 11 deletions(-)
>
> diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
> index 95a8d1d118a2..8e9a9dcf60d5 100644
> --- a/kernel/time/tick-sched.c
> +++ b/kernel/time/tick-sched.c
> @@ -1403,18 +1403,16 @@ static void tick_nohz_lowres_handler(struct clock_event_device *dev)
>         tick_sched_do_timer(ts, now);
>         tick_sched_handle(ts, regs);
>
> -       if (unlikely(ts->tick_stopped)) {
> -               /*
> -                * The clockevent device is not reprogrammed, so change the
> -                * clock event device to ONESHOT_STOPPED to avoid spurious
> -                * interrupts on devices which might not be truly one shot.
> -                */
> -               tick_program_event(KTIME_MAX, 1);
> -               return;
> +       /*
> +        * In dynticks mode, tick reprogram is deferred:
> +        * - to the idle task if in dynticks-idle
> +        * - to IRQ exit if in full-dynticks.
> +        */
> +       if (likely(!ts->tick_stopped)) {
> +               hrtimer_forward(&ts->sched_timer, now, TICK_NSEC);
> +               tick_program_event(hrtimer_get_expires(&ts->sched_timer), 1);
>         }
>
> -       hrtimer_forward(&ts->sched_timer, now, TICK_NSEC);
> -       tick_program_event(hrtimer_get_expires(&ts->sched_timer), 1);
>  }
>
>  static inline void tick_nohz_activate(struct tick_sched *ts, int mode)
> @@ -1519,7 +1517,11 @@ static enum hrtimer_restart tick_nohz_highres_handler(struct hrtimer *timer)
>         else
>                 ts->next_tick = 0;
>
> -       /* No need to reprogram if we are in idle or full dynticks mode */
> +       /*
> +        * In dynticks mode, tick reprogram is deferred:
> +        * - to the idle task if in dynticks-idle
> +        * - to IRQ exit if in full-dynticks.
> +        */
>         if (unlikely(ts->tick_stopped))
>                 return HRTIMER_NORESTART;
>
> --
> 2.34.1
>
  

On Wed, Sep 13, 2023 at 09:17:21PM -0400, Joel Fernandes wrote:
> On Tue, Sep 12, 2023 at 6:44 AM Frederic Weisbecker <frederic@kernel.org> wrote:
> >
> > In lowres dynticks mode, just like in highres dynticks mode, when there
> > is no tick to program in the future, the tick eventually gets
> > deactivated either:
> >
> > * From the idle loop if in idle mode.
> > * From the IRQ exit if in full dynticks mode.
> >
> > Therefore there is no need to deactivate it from the tick itself. This
> > just just brings more overhead in the idle tick path for no reason.
> >
> > Fixes: 62c1256d5447 ("timers/nohz: Switch to ONESHOT_STOPPED in the low-res handler when the tick is stopped")
> > Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
> 
> If on some weird hardware, say  ts->next_tick = KTIME_MAX but a
> spurious timer interrupt went off and tick_nohz_handler() did get
> called (yeah weird hypothetical situation), then in
> tick_nohz_stop_tick() we might early return from:
> 
> /* Skip reprogram of event if its not changed */
> if (ts->tick_stopped && (expires == ts->next_tick))
> 
> without no "eventual" reprogramming.
> 
> Maybe we should also reprogram with KTIME_MAX in such a situation?
> Then we can get rid of it from tick_nohz_handler() for the common case
> as you are doing.
> 
> So for weird hardware, with this patch we are not doing an extra
> tick_program_event(KTIME_MAX, 1); like Nick was doing. That makes me a
> tad bit nervous.

So when a tick happens, ts->next_tick is reset to 0 (in tick_sched_handle()).
This way if a timer interrupt fires too early, and that includes also timer
interrupts when next_tick is KTIME_MAX, the timer is always reprogrammed upon
the next idle loop iteration. So this shouldn't happen.

Thanks.
  

On Thu, Sep 14, 2023 at 5:29 AM Frederic Weisbecker <frederic@kernel.org> wrote:
>
> On Wed, Sep 13, 2023 at 09:17:21PM -0400, Joel Fernandes wrote:
> > On Tue, Sep 12, 2023 at 6:44 AM Frederic Weisbecker <frederic@kernel.org> wrote:
> > >
> > > In lowres dynticks mode, just like in highres dynticks mode, when there
> > > is no tick to program in the future, the tick eventually gets
> > > deactivated either:
> > >
> > > * From the idle loop if in idle mode.
> > > * From the IRQ exit if in full dynticks mode.
> > >
> > > Therefore there is no need to deactivate it from the tick itself. This
> > > just just brings more overhead in the idle tick path for no reason.
> > >
> > > Fixes: 62c1256d5447 ("timers/nohz: Switch to ONESHOT_STOPPED in the low-res handler when the tick is stopped")
> > > Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
> >
> > If on some weird hardware, say  ts->next_tick = KTIME_MAX but a
> > spurious timer interrupt went off and tick_nohz_handler() did get
> > called (yeah weird hypothetical situation), then in
> > tick_nohz_stop_tick() we might early return from:
> >
> > /* Skip reprogram of event if its not changed */
> > if (ts->tick_stopped && (expires == ts->next_tick))
> >
> > without no "eventual" reprogramming.
> >
> > Maybe we should also reprogram with KTIME_MAX in such a situation?
> > Then we can get rid of it from tick_nohz_handler() for the common case
> > as you are doing.
> >
> > So for weird hardware, with this patch we are not doing an extra
> > tick_program_event(KTIME_MAX, 1); like Nick was doing. That makes me a
> > tad bit nervous.
>
> So when a tick happens, ts->next_tick is reset to 0 (in tick_sched_handle()).
> This way if a timer interrupt fires too early, and that includes also timer
> interrupts when next_tick is KTIME_MAX, the timer is always reprogrammed upon
> the next idle loop iteration. So this shouldn't happen.

Ah you are right, I missed that. So then I am good with it:
Reviewed-by: Joel Fernandes (Google) <joel@joelfernandes.org>

thanks,

 - Joel