[1/3] arm: Fix flush_dcache_page() for usage from irq context
Commit Message
Since at least kernel 6.1, flush_dcache_page() is called with IRQs
disabled, e.g. from aio_complete().
But the current implementation for flush_dcache_page() on ARM
unintentionally re-enables IRQs, which may lead to deadlocks.
Fix it by using xa_lock_irqsave() and xa_unlock_irqrestore()
for the flush_dcache_mmap_*lock() macros instead.
Cc: Russell King (Oracle) <linux@armlinux.org.uk>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: linux-arm-kernel@lists.infradead.org
Signed-off-by: Helge Deller <deller@gmx.de>
---
arch/arm/include/asm/cacheflush.h | 4 ++++
arch/arm/mm/flush.c | 5 +++--
2 files changed, 7 insertions(+), 2 deletions(-)
--
2.38.1
Comments
On Wed, May 24, 2023, at 17:26, Helge Deller wrote:
> Since at least kernel 6.1, flush_dcache_page() is called with IRQs
> disabled, e.g. from aio_complete().
>
> But the current implementation for flush_dcache_page() on ARM
> unintentionally re-enables IRQs, which may lead to deadlocks.
>
> Fix it by using xa_lock_irqsave() and xa_unlock_irqrestore()
> for the flush_dcache_mmap_*lock() macros instead.
>
> Cc: Russell King (Oracle) <linux@armlinux.org.uk>
> Cc: Arnd Bergmann <arnd@arndb.de>
> Cc: linux-arm-kernel@lists.infradead.org
> Signed-off-by: Helge Deller <deller@gmx.de>
Cc: stable@vger.kernel.org
Reviewed-by: Arnd Bergmann <arnd@arndb.de>
From what I can tell, the behavior in aio_complete has been
there for over 10 years, since 21b40200cfe96 ("aio: use
flush_dcache_page()"). Others may have done the same already
back then.
I also see you sent patches for nios2 and parisc, but not
for csky, which appears to need the same thing.
Arnd
On 5/24/23 22:00, Arnd Bergmann wrote:
> On Wed, May 24, 2023, at 17:26, Helge Deller wrote:
>> Since at least kernel 6.1, flush_dcache_page() is called with IRQs
>> disabled, e.g. from aio_complete().
>>
>> But the current implementation for flush_dcache_page() on ARM
>> unintentionally re-enables IRQs, which may lead to deadlocks.
>>
>> Fix it by using xa_lock_irqsave() and xa_unlock_irqrestore()
>> for the flush_dcache_mmap_*lock() macros instead.
>>
>> Cc: Russell King (Oracle) <linux@armlinux.org.uk>
>> Cc: Arnd Bergmann <arnd@arndb.de>
>> Cc: linux-arm-kernel@lists.infradead.org
>> Signed-off-by: Helge Deller <deller@gmx.de>
>
> Cc: stable@vger.kernel.org
> Reviewed-by: Arnd Bergmann <arnd@arndb.de>
Thanks!
I assume it's picked up in the arm git tree then.
> From what I can tell, the behavior in aio_complete has been
> there for over 10 years, since 21b40200cfe96 ("aio: use
> flush_dcache_page()").
Oh, then those arches are broken since then.
> Others may have done the same already back then.
>
> I also see you sent patches for nios2 and parisc, but not
> for csky, which appears to need the same thing.
csky doesn't use flush_dcache_mmap_lock() inside it's
flush_dcache_page() function, so I think it's not affected.
Helge
On 5/24/23 22:00, Arnd Bergmann wrote:
> On Wed, May 24, 2023, at 17:26, Helge Deller wrote:
>> Since at least kernel 6.1, flush_dcache_page() is called with IRQs
>> disabled, e.g. from aio_complete().
>>
>> But the current implementation for flush_dcache_page() on ARM
>> unintentionally re-enables IRQs, which may lead to deadlocks.
>>
>> Fix it by using xa_lock_irqsave() and xa_unlock_irqrestore()
>> for the flush_dcache_mmap_*lock() macros instead.
>>
>> Cc: Russell King (Oracle) <linux@armlinux.org.uk>
>> Cc: Arnd Bergmann <arnd@arndb.de>
>> Cc: linux-arm-kernel@lists.infradead.org
>> Signed-off-by: Helge Deller <deller@gmx.de>
>
> Cc: stable@vger.kernel.org
> Reviewed-by: Arnd Bergmann <arnd@arndb.de>
>
> From what I can tell, the behavior in aio_complete has been
> there for over 10 years, since 21b40200cfe96 ("aio: use
> flush_dcache_page()"). Others may have done the same already
> back then.
gentle ping...
I think this patch hasn't been picked up yet for arm.
Helge
@@ -315,6 +315,10 @@ static inline void flush_anon_page(struct vm_area_struct *vma,
#define flush_dcache_mmap_lock(mapping) xa_lock_irq(&mapping->i_pages)
#define flush_dcache_mmap_unlock(mapping) xa_unlock_irq(&mapping->i_pages)
+#define flush_dcache_mmap_lock_irqsave(mapping, flags) \
+ xa_lock_irqsave(&mapping->i_pages, flags)
+#define flush_dcache_mmap_unlock_irqrestore(mapping, flags) \
+ xa_unlock_irqrestore(&mapping->i_pages, flags)
/*
* We don't appear to need to do anything here. In fact, if we did, we'd
@@ -238,6 +238,7 @@ static void __flush_dcache_aliases(struct address_space *mapping, struct page *p
{
struct mm_struct *mm = current->active_mm;
struct vm_area_struct *mpnt;
+ unsigned long flags;
pgoff_t pgoff;
/*
@@ -248,7 +249,7 @@ static void __flush_dcache_aliases(struct address_space *mapping, struct page *p
*/
pgoff = page->index;
- flush_dcache_mmap_lock(mapping);
+ flush_dcache_mmap_lock_irqsave(mapping, flags);
vma_interval_tree_foreach(mpnt, &mapping->i_mmap, pgoff, pgoff) {
unsigned long offset;
@@ -262,7 +263,7 @@ static void __flush_dcache_aliases(struct address_space *mapping, struct page *p
offset = (pgoff - mpnt->vm_pgoff) << PAGE_SHIFT;
flush_cache_page(mpnt, mpnt->vm_start + offset, page_to_pfn(page));
}
- flush_dcache_mmap_unlock(mapping);
+ flush_dcache_mmap_unlock_irqrestore(mapping, flags);
}
#if __LINUX_ARM_ARCH__ >= 6