genirq: Fix nested thread vs synchronize_hardirq() deadlock

Message ID 20230613-genirq-nested-v1-1-289dc15b7669@axis.com
State New
Headers
Series genirq: Fix nested thread vs synchronize_hardirq() deadlock |

Commit Message

Vincent Whitchurch June 15, 2023, 1:11 p.m. UTC
  There is a possibility of deadlock if synchronize_hardirq() is called
when the nested threaded interrupt is active.  The following scenario
was observed on a uniprocessor PREEMPT_NONE system:

 Thread 1                      Thread 2

 handle_nested_thread()
  Set INPROGRESS
  Call ->thread_fn()
   thread_fn goes to sleep

                              free_irq()
                               __synchronize_hardirq()
                                Busy-loop forever waiting for INPROGRESS
                                to be cleared

Since the purpose of the INPROGRESS flag seems to be for hard IRQ
handlers, remove its usage in the nested threaded interrupt case and
instead re-use the active_threads mechanism to wait for nested threaded
interrupts to complete.

Signed-off-by: Vincent Whitchurch <vincent.whitchurch@axis.com>
---
 kernel/irq/chip.c   | 5 +++--
 kernel/irq/manage.c | 4 ++++
 2 files changed, 7 insertions(+), 2 deletions(-)


---
base-commit: 858fd168a95c5b9669aac8db6c14a9aeab446375
change-id: 20230613-genirq-nested-625612a6fa05

Best regards,
  

Comments

Thomas Gleixner June 19, 2023, 5:37 p.m. UTC | #1
On Thu, Jun 15 2023 at 15:11, Vincent Whitchurch wrote:
> There is a possibility of deadlock if synchronize_hardirq() is called
> when the nested threaded interrupt is active.  The following scenario
> was observed on a uniprocessor PREEMPT_NONE system:
>
>  Thread 1                      Thread 2
>
>  handle_nested_thread()
>   Set INPROGRESS
>   Call ->thread_fn()
>    thread_fn goes to sleep
>
>                               free_irq()
>                                __synchronize_hardirq()
>                                 Busy-loop forever waiting for INPROGRESS
>                                 to be cleared

Duh. Right! Nice find.

> Since the purpose of the INPROGRESS flag seems to be for hard IRQ

seems? Either it is or it is not.

It's really for hard interrupt handlers only and does not make sense for
the nested threaded case.

> handlers, remove its usage in the nested threaded interrupt case and
> instead re-use the active_threads mechanism to wait for nested threaded
> interrupts to complete.

> Signed-off-by: Vincent Whitchurch <vincent.whitchurch@axis.com>
> ---
>  kernel/irq/chip.c   | 5 +++--
>  kernel/irq/manage.c | 4 ++++
>  2 files changed, 7 insertions(+), 2 deletions(-)
>
> diff --git a/kernel/irq/chip.c b/kernel/irq/chip.c
> index 49e7bc871fece..3e4b4c6de8195 100644
> --- a/kernel/irq/chip.c
> +++ b/kernel/irq/chip.c
> @@ -476,7 +476,7 @@ void handle_nested_irq(unsigned int irq)
>  	}
>  
>  	kstat_incr_irqs_this_cpu(desc);
> -	irqd_set(&desc->irq_data, IRQD_IRQ_INPROGRESS);
> +	atomic_inc(&desc->threads_active);
>  	raw_spin_unlock_irq(&desc->lock);
>  
>  	action_ret = IRQ_NONE;
> @@ -487,7 +487,8 @@ void handle_nested_irq(unsigned int irq)
>  		note_interrupt(desc, action_ret);
>  
>  	raw_spin_lock_irq(&desc->lock);
> -	irqd_clear(&desc->irq_data, IRQD_IRQ_INPROGRESS);
> +	if (atomic_dec_and_test(&desc->threads_active))
> +		wake_up(&desc->wait_for_threads);
>  
>  out_unlock:
>  	raw_spin_unlock_irq(&desc->lock);
> diff --git a/kernel/irq/manage.c b/kernel/irq/manage.c
> index d2742af0f0fd8..58dcc9df6d72c 100644
> --- a/kernel/irq/manage.c
> +++ b/kernel/irq/manage.c
> @@ -1977,6 +1977,10 @@ static struct irqaction *__free_irq(struct irq_desc *desc, void *dev_id)
>  		}
>  	}
>  
> +	/* Wait for any remaining nested threaded interrupts. */
> +	wait_event(desc->wait_for_threads,
> +		   !atomic_read(&desc->threads_active));

Which is together with the above _synchronize_hardirq(desc, true);
equivivalent to synchronize_irq(), no?

Thanks,

        tglx
  

Patch

diff --git a/kernel/irq/chip.c b/kernel/irq/chip.c
index 49e7bc871fece..3e4b4c6de8195 100644
--- a/kernel/irq/chip.c
+++ b/kernel/irq/chip.c
@@ -476,7 +476,7 @@  void handle_nested_irq(unsigned int irq)
 	}
 
 	kstat_incr_irqs_this_cpu(desc);
-	irqd_set(&desc->irq_data, IRQD_IRQ_INPROGRESS);
+	atomic_inc(&desc->threads_active);
 	raw_spin_unlock_irq(&desc->lock);
 
 	action_ret = IRQ_NONE;
@@ -487,7 +487,8 @@  void handle_nested_irq(unsigned int irq)
 		note_interrupt(desc, action_ret);
 
 	raw_spin_lock_irq(&desc->lock);
-	irqd_clear(&desc->irq_data, IRQD_IRQ_INPROGRESS);
+	if (atomic_dec_and_test(&desc->threads_active))
+		wake_up(&desc->wait_for_threads);
 
 out_unlock:
 	raw_spin_unlock_irq(&desc->lock);
diff --git a/kernel/irq/manage.c b/kernel/irq/manage.c
index d2742af0f0fd8..58dcc9df6d72c 100644
--- a/kernel/irq/manage.c
+++ b/kernel/irq/manage.c
@@ -1977,6 +1977,10 @@  static struct irqaction *__free_irq(struct irq_desc *desc, void *dev_id)
 		}
 	}
 
+	/* Wait for any remaining nested threaded interrupts. */
+	wait_event(desc->wait_for_threads,
+		   !atomic_read(&desc->threads_active));
+
 	/* Last action releases resources */
 	if (!desc->action) {
 		/*