[mm] kasan: stop leaking stack trace handles
Commit Message
From: Andrey Konovalov <andreyknvl@google.com>
Commit 773688a6cb24 ("kasan: use stack_depot_put for Generic mode") added
support for stack trace eviction for Generic KASAN.
However, that commit didn't evict stack traces when the object is not put
into quarantine. As a result, some stack traces are never evicted from
the stack depot.
In addition, with the "kasan: save mempool stack traces" series, the
free stack traces for mempool objects are also not properly evicted from
the stack depot.
Fix both issues by:
1. Evicting all stack traces when an object if freed if it was not put
into quarantine;
2. Always evicting an existing free stack trace when a new one is saved.
Also do a few related clean-ups:
- Do not zero out free track when initializing/invalidating free meta:
set a value in shadow memory instead;
- Rename KASAN_SLAB_FREETRACK to KASAN_SLAB_FREE_META;
- Drop the kasan_init_cache_meta function as it's not used by KASAN;
- Add comments for the kasan_alloc_meta and kasan_free_meta structs.
Fixes: 773688a6cb24 ("kasan: use stack_depot_put for Generic mode")
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
---
Andrew, please put this as a separate patch on top of all KASAN patches
in mm.
---
mm/kasan/common.c | 27 +++++++++++++++---
mm/kasan/generic.c | 60 +++++++++++++++++++++++++++++++++------
mm/kasan/kasan.h | 25 ++++++++++++----
mm/kasan/quarantine.c | 20 +------------
mm/kasan/report_generic.c | 6 ++--
5 files changed, 97 insertions(+), 41 deletions(-)
Comments
Hi,
kernel test robot noticed the following build warnings:
[auto build test WARNING on akpm-mm/mm-everything]
[cannot apply to linus/master v6.7-rc7 next-20231222]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/andrey-konovalov-linux-dev/kasan-stop-leaking-stack-trace-handles/20231227-065314
base: https://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm.git mm-everything
patch link: https://lore.kernel.org/r/20231226225121.235865-1-andrey.konovalov%40linux.dev
patch subject: [PATCH mm] kasan: stop leaking stack trace handles
config: x86_64-randconfig-123-20231227 (https://download.01.org/0day-ci/archive/20231228/202312280213.6j147JJb-lkp@intel.com/config)
compiler: gcc-12 (Debian 12.2.0-14) 12.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20231228/202312280213.6j147JJb-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202312280213.6j147JJb-lkp@intel.com/
All warnings (new ones prefixed by >>):
>> mm/kasan/generic.c:506:6: warning: no previous prototype for 'release_alloc_meta' [-Wmissing-prototypes]
506 | void release_alloc_meta(struct kasan_alloc_meta *meta)
| ^~~~~~~~~~~~~~~~~~
>> mm/kasan/generic.c:517:6: warning: no previous prototype for 'release_free_meta' [-Wmissing-prototypes]
517 | void release_free_meta(const void *object, struct kasan_free_meta *meta)
| ^~~~~~~~~~~~~~~~~
vim +/release_alloc_meta +506 mm/kasan/generic.c
505
> 506 void release_alloc_meta(struct kasan_alloc_meta *meta)
507 {
508 /* Evict the stack traces from stack depot. */
509 stack_depot_put(meta->alloc_track.stack);
510 stack_depot_put(meta->aux_stack[0]);
511 stack_depot_put(meta->aux_stack[1]);
512
513 /* Zero out alloc meta to mark it as invalid. */
514 __memset(meta, 0, sizeof(*meta));
515 }
516
> 517 void release_free_meta(const void *object, struct kasan_free_meta *meta)
518 {
519 /* Check if free meta is valid. */
520 if (*(u8 *)kasan_mem_to_shadow(object) != KASAN_SLAB_FREE_META)
521 return;
522
523 /* Evict the stack trace from the stack depot. */
524 stack_depot_put(meta->free_track.stack);
525
526 /* Mark free meta as invalid. */
527 *(u8 *)kasan_mem_to_shadow(object) = KASAN_SLAB_FREE;
528 }
529
On Thu, 28 Dec 2023 02:19:51 +0800 kernel test robot <lkp@intel.com> wrote:
> Hi,
>
> kernel test robot noticed the following build warnings:
>
> [auto build test WARNING on akpm-mm/mm-everything]
> [cannot apply to linus/master v6.7-rc7 next-20231222]
> [If your patch is applied to the wrong git tree, kindly drop us a note.
> And when submitting patch, we suggest to use '--base' as documented in
> https://git-scm.com/docs/git-format-patch#_base_tree_information]
>
> url: https://github.com/intel-lab-lkp/linux/commits/andrey-konovalov-linux-dev/kasan-stop-leaking-stack-trace-handles/20231227-065314
> base: https://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm.git mm-everything
> patch link: https://lore.kernel.org/r/20231226225121.235865-1-andrey.konovalov%40linux.dev
> patch subject: [PATCH mm] kasan: stop leaking stack trace handles
> config: x86_64-randconfig-123-20231227 (https://download.01.org/0day-ci/archive/20231228/202312280213.6j147JJb-lkp@intel.com/config)
> compiler: gcc-12 (Debian 12.2.0-14) 12.2.0
> reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20231228/202312280213.6j147JJb-lkp@intel.com/reproduce)
>
> If you fix the issue in a separate patch/commit (i.e. not just a new version of
> the same patch/commit), kindly add following tags
> | Reported-by: kernel test robot <lkp@intel.com>
> | Closes: https://lore.kernel.org/oe-kbuild-all/202312280213.6j147JJb-lkp@intel.com/
>
> All warnings (new ones prefixed by >>):
>
> >> mm/kasan/generic.c:506:6: warning: no previous prototype for 'release_alloc_meta' [-Wmissing-prototypes]
> 506 | void release_alloc_meta(struct kasan_alloc_meta *meta)
> | ^~~~~~~~~~~~~~~~~~
> >> mm/kasan/generic.c:517:6: warning: no previous prototype for 'release_free_meta' [-Wmissing-prototypes]
> 517 | void release_free_meta(const void *object, struct kasan_free_meta *meta)
> | ^~~~~~~~~~~~~~~~~
Thanks, I added this fix:
--- a/mm/kasan/generic.c~kasan-stop-leaking-stack-trace-handles-fix
+++ a/mm/kasan/generic.c
@@ -503,7 +503,7 @@ void kasan_init_object_meta(struct kmem_
*/
}
-void release_alloc_meta(struct kasan_alloc_meta *meta)
+static void release_alloc_meta(struct kasan_alloc_meta *meta)
{
/* Evict the stack traces from stack depot. */
stack_depot_put(meta->alloc_track.stack);
@@ -514,7 +514,7 @@ void release_alloc_meta(struct kasan_all
__memset(meta, 0, sizeof(*meta));
}
-void release_free_meta(const void *object, struct kasan_free_meta *meta)
+static void release_free_meta(const void *object, struct kasan_free_meta *meta)
{
/* Check if free meta is valid. */
if (*(u8 *)kasan_mem_to_shadow(object) != KASAN_SLAB_FREE_META)
On Wed, Dec 27, 2023 at 10:23 PM Andrew Morton
<akpm@linux-foundation.org> wrote:
>
> Thanks, I added this fix:
>
> --- a/mm/kasan/generic.c~kasan-stop-leaking-stack-trace-handles-fix
> +++ a/mm/kasan/generic.c
> @@ -503,7 +503,7 @@ void kasan_init_object_meta(struct kmem_
> */
> }
>
> -void release_alloc_meta(struct kasan_alloc_meta *meta)
> +static void release_alloc_meta(struct kasan_alloc_meta *meta)
> {
> /* Evict the stack traces from stack depot. */
> stack_depot_put(meta->alloc_track.stack);
> @@ -514,7 +514,7 @@ void release_alloc_meta(struct kasan_all
> __memset(meta, 0, sizeof(*meta));
> }
>
> -void release_free_meta(const void *object, struct kasan_free_meta *meta)
> +static void release_free_meta(const void *object, struct kasan_free_meta *meta)
> {
> /* Check if free meta is valid. */
> if (*(u8 *)kasan_mem_to_shadow(object) != KASAN_SLAB_FREE_META)
> _
>
Could you mark them as "static inline" even?
I'll fix this if I end up sending v2.
Thank you, Andrew!
On Wed, 27 Dec 2023 22:42:40 +0100 Andrey Konovalov <andreyknvl@gmail.com> wrote:
> On Wed, Dec 27, 2023 at 10:23 PM Andrew Morton
> <akpm@linux-foundation.org> wrote:
> >
> > Thanks, I added this fix:
> >
> > --- a/mm/kasan/generic.c~kasan-stop-leaking-stack-trace-handles-fix
> > +++ a/mm/kasan/generic.c
> > @@ -503,7 +503,7 @@ void kasan_init_object_meta(struct kmem_
> > */
> > }
> >
> > -void release_alloc_meta(struct kasan_alloc_meta *meta)
> > +static void release_alloc_meta(struct kasan_alloc_meta *meta)
> > {
> > /* Evict the stack traces from stack depot. */
> > stack_depot_put(meta->alloc_track.stack);
> > @@ -514,7 +514,7 @@ void release_alloc_meta(struct kasan_all
> > __memset(meta, 0, sizeof(*meta));
> > }
> >
> > -void release_free_meta(const void *object, struct kasan_free_meta *meta)
> > +static void release_free_meta(const void *object, struct kasan_free_meta *meta)
> > {
> > /* Check if free meta is valid. */
> > if (*(u8 *)kasan_mem_to_shadow(object) != KASAN_SLAB_FREE_META)
> > _
> >
>
> Could you mark them as "static inline" even?
That's rather old-fashioned. Nowadays gcc is supposed to work out
whether or not to inline things, and we override that with noinline and
__always_inline.
Hi,
kernel test robot noticed the following build warnings:
[auto build test WARNING on akpm-mm/mm-everything]
[cannot apply to linus/master v6.7-rc7 next-20231222]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/andrey-konovalov-linux-dev/kasan-stop-leaking-stack-trace-handles/20231227-065314
base: https://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm.git mm-everything
patch link: https://lore.kernel.org/r/20231226225121.235865-1-andrey.konovalov%40linux.dev
patch subject: [PATCH mm] kasan: stop leaking stack trace handles
config: arm-randconfig-002-20231227 (https://download.01.org/0day-ci/archive/20231228/202312280603.WqS3sWfa-lkp@intel.com/config)
compiler: ClangBuiltLinux clang version 17.0.6 (https://github.com/llvm/llvm-project 6009708b4367171ccdbf4b5905cb6a803753fe18)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20231228/202312280603.WqS3sWfa-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202312280603.WqS3sWfa-lkp@intel.com/
All warnings (new ones prefixed by >>):
>> mm/kasan/generic.c:506:6: warning: no previous prototype for function 'release_alloc_meta' [-Wmissing-prototypes]
506 | void release_alloc_meta(struct kasan_alloc_meta *meta)
| ^
mm/kasan/generic.c:506:1: note: declare 'static' if the function is not intended to be used outside of this translation unit
506 | void release_alloc_meta(struct kasan_alloc_meta *meta)
| ^
| static
>> mm/kasan/generic.c:517:6: warning: no previous prototype for function 'release_free_meta' [-Wmissing-prototypes]
517 | void release_free_meta(const void *object, struct kasan_free_meta *meta)
| ^
mm/kasan/generic.c:517:1: note: declare 'static' if the function is not intended to be used outside of this translation unit
517 | void release_free_meta(const void *object, struct kasan_free_meta *meta)
| ^
| static
2 warnings generated.
vim +/release_alloc_meta +506 mm/kasan/generic.c
505
> 506 void release_alloc_meta(struct kasan_alloc_meta *meta)
507 {
508 /* Evict the stack traces from stack depot. */
509 stack_depot_put(meta->alloc_track.stack);
510 stack_depot_put(meta->aux_stack[0]);
511 stack_depot_put(meta->aux_stack[1]);
512
513 /* Zero out alloc meta to mark it as invalid. */
514 __memset(meta, 0, sizeof(*meta));
515 }
516
> 517 void release_free_meta(const void *object, struct kasan_free_meta *meta)
518 {
519 /* Check if free meta is valid. */
520 if (*(u8 *)kasan_mem_to_shadow(object) != KASAN_SLAB_FREE_META)
521 return;
522
523 /* Evict the stack trace from the stack depot. */
524 stack_depot_put(meta->free_track.stack);
525
526 /* Mark free meta as invalid. */
527 *(u8 *)kasan_mem_to_shadow(object) = KASAN_SLAB_FREE;
528 }
529
@@ -255,14 +255,33 @@ static inline bool poison_slab_object(struct kmem_cache *cache, void *object,
bool __kasan_slab_free(struct kmem_cache *cache, void *object,
unsigned long ip, bool init)
{
- bool buggy_object;
-
if (is_kfence_address(object))
return false;
- buggy_object = poison_slab_object(cache, object, ip, init);
+ /*
+ * If the object is buggy, do not let slab put the object onto the
+ * freelist. The object will thus never be allocated again and its
+ * metadata will never get released.
+ */
+ if (poison_slab_object(cache, object, ip, init))
+ return true;
+
+ /*
+ * If the object is put into quarantine, do not let slab put the object
+ * onto the freelist for now. The object's metadata is kept until the
+ * object gets evicted from quarantine.
+ */
+ if (kasan_quarantine_put(cache, object))
+ return true;
+
+ /*
+ * If the object is not put into quarantine, it will likely be quickly
+ * reallocated. Thus, release its metadata now.
+ */
+ kasan_release_object_meta(cache, object);
- return buggy_object ? true : kasan_quarantine_put(cache, object);
+ /* Let slab put the object onto the freelist. */
+ return false;
}
static inline bool check_page_allocation(void *ptr, unsigned long ip)
@@ -480,10 +480,10 @@ struct kasan_free_meta *kasan_get_free_meta(struct kmem_cache *cache,
void kasan_init_object_meta(struct kmem_cache *cache, const void *object)
{
struct kasan_alloc_meta *alloc_meta;
- struct kasan_free_meta *free_meta;
alloc_meta = kasan_get_alloc_meta(cache, object);
if (alloc_meta) {
+ /* Zero out alloc meta to mark it as invalid. */
__memset(alloc_meta, 0, sizeof(*alloc_meta));
/*
@@ -495,9 +495,50 @@ void kasan_init_object_meta(struct kmem_cache *cache, const void *object)
raw_spin_lock_init(&alloc_meta->aux_lock);
kasan_enable_current();
}
+
+ /*
+ * Explicitly marking free meta as invalid is not required: the shadow
+ * value for the first 8 bytes of a newly allocated object is not
+ * KASAN_SLAB_FREE_META.
+ */
+}
+
+void release_alloc_meta(struct kasan_alloc_meta *meta)
+{
+ /* Evict the stack traces from stack depot. */
+ stack_depot_put(meta->alloc_track.stack);
+ stack_depot_put(meta->aux_stack[0]);
+ stack_depot_put(meta->aux_stack[1]);
+
+ /* Zero out alloc meta to mark it as invalid. */
+ __memset(meta, 0, sizeof(*meta));
+}
+
+void release_free_meta(const void *object, struct kasan_free_meta *meta)
+{
+ /* Check if free meta is valid. */
+ if (*(u8 *)kasan_mem_to_shadow(object) != KASAN_SLAB_FREE_META)
+ return;
+
+ /* Evict the stack trace from the stack depot. */
+ stack_depot_put(meta->free_track.stack);
+
+ /* Mark free meta as invalid. */
+ *(u8 *)kasan_mem_to_shadow(object) = KASAN_SLAB_FREE;
+}
+
+void kasan_release_object_meta(struct kmem_cache *cache, const void *object)
+{
+ struct kasan_alloc_meta *alloc_meta;
+ struct kasan_free_meta *free_meta;
+
+ alloc_meta = kasan_get_alloc_meta(cache, object);
+ if (alloc_meta)
+ release_alloc_meta(alloc_meta);
+
free_meta = kasan_get_free_meta(cache, object);
if (free_meta)
- __memset(free_meta, 0, sizeof(*free_meta));
+ release_free_meta(object, free_meta);
}
size_t kasan_metadata_size(struct kmem_cache *cache, bool in_object)
@@ -573,11 +614,8 @@ void kasan_save_alloc_info(struct kmem_cache *cache, void *object, gfp_t flags)
if (!alloc_meta)
return;
- /* Evict previous stack traces (might exist for krealloc). */
- stack_depot_put(alloc_meta->alloc_track.stack);
- stack_depot_put(alloc_meta->aux_stack[0]);
- stack_depot_put(alloc_meta->aux_stack[1]);
- __memset(alloc_meta, 0, sizeof(*alloc_meta));
+ /* Evict previous stack traces (might exist for krealloc or mempool). */
+ release_alloc_meta(alloc_meta);
kasan_save_track(&alloc_meta->alloc_track, flags);
}
@@ -590,7 +628,11 @@ void kasan_save_free_info(struct kmem_cache *cache, void *object)
if (!free_meta)
return;
+ /* Evict previous stack trace (might exist for mempool). */
+ release_free_meta(object, free_meta);
+
kasan_save_track(&free_meta->free_track, 0);
- /* The object was freed and has free track set. */
- *(u8 *)kasan_mem_to_shadow(object) = KASAN_SLAB_FREETRACK;
+
+ /* Mark free meta as valid. */
+ *(u8 *)kasan_mem_to_shadow(object) = KASAN_SLAB_FREE_META;
}
@@ -156,7 +156,7 @@ static inline bool kasan_requires_meta(void)
#ifdef CONFIG_KASAN_GENERIC
-#define KASAN_SLAB_FREETRACK 0xFA /* freed slab object with free track */
+#define KASAN_SLAB_FREE_META 0xFA /* freed slab object with free meta */
#define KASAN_GLOBAL_REDZONE 0xF9 /* redzone for global variable */
/* Stack redzone shadow values. Compiler ABI, do not change. */
@@ -253,6 +253,15 @@ struct kasan_global {
#ifdef CONFIG_KASAN_GENERIC
+/*
+ * Alloc meta contains the allocation-related information about a slab object.
+ * Alloc meta is saved when an object is allocated and is kept until either the
+ * object returns to the slab freelist (leaves quarantine for quarantined
+ * objects or gets freed for the non-quarantined ones) or reallocated via
+ * krealloc or through a mempool.
+ * Alloc meta is stored inside of the object's redzone.
+ * Alloc meta is considered valid whenever it contains non-zero data.
+ */
struct kasan_alloc_meta {
struct kasan_track alloc_track;
/* Free track is stored in kasan_free_meta. */
@@ -278,8 +287,12 @@ struct qlist_node {
#define KASAN_NO_FREE_META INT_MAX
/*
- * Free meta is only used by Generic mode while the object is in quarantine.
- * After that, slab allocator stores the freelist pointer in the object.
+ * Free meta contains the freeing-related information about a slab object.
+ * Free meta is only kept for quarantined objects and for mempool objects until
+ * the object gets allocated again.
+ * Free meta is stored within the object's memory.
+ * Free meta is considered valid whenever the value of the shadow byte that
+ * corresponds to the first 8 bytes of the object is KASAN_SLAB_FREE_META.
*/
struct kasan_free_meta {
struct qlist_node quarantine_link;
@@ -380,15 +393,15 @@ void kasan_report_invalid_free(void *object, unsigned long ip, enum kasan_report
struct slab *kasan_addr_to_slab(const void *addr);
#ifdef CONFIG_KASAN_GENERIC
-void kasan_init_cache_meta(struct kmem_cache *cache, unsigned int *size);
-void kasan_init_object_meta(struct kmem_cache *cache, const void *object);
struct kasan_alloc_meta *kasan_get_alloc_meta(struct kmem_cache *cache,
const void *object);
struct kasan_free_meta *kasan_get_free_meta(struct kmem_cache *cache,
const void *object);
+void kasan_init_object_meta(struct kmem_cache *cache, const void *object);
+void kasan_release_object_meta(struct kmem_cache *cache, const void *object);
#else
-static inline void kasan_init_cache_meta(struct kmem_cache *cache, unsigned int *size) { }
static inline void kasan_init_object_meta(struct kmem_cache *cache, const void *object) { }
+static inline void kasan_release_object_meta(struct kmem_cache *cache, const void *object) { }
#endif
depot_stack_handle_t kasan_save_stack(gfp_t flags, depot_flags_t depot_flags);
@@ -143,22 +143,10 @@ static void *qlink_to_object(struct qlist_node *qlink, struct kmem_cache *cache)
static void qlink_free(struct qlist_node *qlink, struct kmem_cache *cache)
{
void *object = qlink_to_object(qlink, cache);
- struct kasan_alloc_meta *alloc_meta = kasan_get_alloc_meta(cache, object);
struct kasan_free_meta *free_meta = kasan_get_free_meta(cache, object);
unsigned long flags;
- if (alloc_meta) {
- stack_depot_put(alloc_meta->alloc_track.stack);
- stack_depot_put(alloc_meta->aux_stack[0]);
- stack_depot_put(alloc_meta->aux_stack[1]);
- __memset(alloc_meta, 0, sizeof(*alloc_meta));
- }
-
- if (free_meta &&
- *(u8 *)kasan_mem_to_shadow(object) == KASAN_SLAB_FREETRACK) {
- stack_depot_put(free_meta->free_track.stack);
- __memset(&free_meta->free_track, 0, sizeof(free_meta->free_track));
- }
+ kasan_release_object_meta(cache, object);
/*
* If init_on_free is enabled and KASAN's free metadata is stored in
@@ -170,12 +158,6 @@ static void qlink_free(struct qlist_node *qlink, struct kmem_cache *cache)
cache->kasan_info.free_meta_offset == 0)
memzero_explicit(free_meta, sizeof(*free_meta));
- /*
- * As the object now gets freed from the quarantine,
- * take note that its free track is no longer exists.
- */
- *(u8 *)kasan_mem_to_shadow(object) = KASAN_SLAB_FREE;
-
if (IS_ENABLED(CONFIG_SLAB))
local_irq_save(flags);
@@ -110,7 +110,7 @@ static const char *get_shadow_bug_type(struct kasan_report_info *info)
bug_type = "use-after-free";
break;
case KASAN_SLAB_FREE:
- case KASAN_SLAB_FREETRACK:
+ case KASAN_SLAB_FREE_META:
bug_type = "slab-use-after-free";
break;
case KASAN_ALLOCA_LEFT:
@@ -173,8 +173,8 @@ void kasan_complete_mode_report_info(struct kasan_report_info *info)
memcpy(&info->alloc_track, &alloc_meta->alloc_track,
sizeof(info->alloc_track));
- if (*(u8 *)kasan_mem_to_shadow(info->object) == KASAN_SLAB_FREETRACK) {
- /* Free meta must be present with KASAN_SLAB_FREETRACK. */
+ if (*(u8 *)kasan_mem_to_shadow(info->object) == KASAN_SLAB_FREE_META) {
+ /* Free meta must be present with KASAN_SLAB_FREE_META. */
free_meta = kasan_get_free_meta(info->cache, info->object);
memcpy(&info->free_track, &free_meta->free_track,
sizeof(info->free_track));