From patchwork Thu Apr 27 15:59:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Berg X-Patchwork-Id: 88313 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp390149vqo; Thu, 27 Apr 2023 09:28:50 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7lmQ8gsBmIhSd2Amsc4mwnL4uGqSwj9cjXa77fbCDxxnlz+BpAM50W2IZvHJJa1GO5w+A4 X-Received: by 2002:a05:6a21:3406:b0:f0:6567:92ba with SMTP id yn6-20020a056a21340600b000f0656792bamr2308108pzb.58.1682612929905; Thu, 27 Apr 2023 09:28:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682612929; cv=none; d=google.com; s=arc-20160816; b=ImKQ1JnCWi9wpHTQ6k61jzdTvhxyzPW4bGxhLYf8Ab1W7L0BZjanH5Y5eEQq5o32xA awUVRwmQx4irAwQBh1Klio1zaAMA4rNzhIOhhaWNvIZbxznHy81P3cT9yurx/s8VYAYo 0Ie2MEWk0y132wr7ASHk+/jsLa2ZXsWyx/s06q2dhDArQe006sCPcgY/M/QVGVqzDDHM NglAJxvhcLzAx+xhJcxzgu9Rlv2ZwuSUFRFVcapqR4Qgf+eedLMWA5NUWItYz//56AJb 4yd9Tk3TTCoikVJnnGB2joXEdOxm8e0EV4ebCwvjeaJy4Mx13aCuQF1ddhOnu/GY7EIk QgHQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=Cuucxv7D/U/64Bqs7uWjVu3OA9YWTZvXVvV+4QELO6Q=; b=Pdrq+9JCAytG7LsSaAMxBK2qB68H4kIM7vSTw//A5Koxg5tzY4YPtkTBbKWv2n1YFP zX+MPXbbOMnNKZ05KtgI6ArjAXTY1DLcDxaltitrR5CeOx1QUH63ZTg+eFwBQeFz4GeS ZUY8S/4B+GplTN9m53yqIDXQgKRrSQHZl5jitWtWDk4dAZFqCOuIrNLqYgyg/8BoOeFO yrvTP7WHYKwvG4d0ZtXjr+rDr1nxmvOYWEQNUUf0ci3UlS4PKIP8P08OZKdLNbjtGnKC phnlfOMWapUsUuL3v/y9g1cLXT0yqNlrcT0QpYJM7s4XDnyupDiTGNCr/Q+7SXPjBTxr ZgGg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@sipsolutions.net header.s=mail header.b=B5Binqu8; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=REJECT dis=NONE) header.from=sipsolutions.net Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id f11-20020a63e30b000000b004fb921d0184si19453656pgh.146.2023.04.27.09.28.36; Thu, 27 Apr 2023 09:28:49 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@sipsolutions.net header.s=mail header.b=B5Binqu8; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=REJECT dis=NONE) header.from=sipsolutions.net Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244192AbjD0P7n (ORCPT + 99 others); Thu, 27 Apr 2023 11:59:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42654 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243864AbjD0P7l (ORCPT ); Thu, 27 Apr 2023 11:59:41 -0400 Received: from sipsolutions.net (s3.sipsolutions.net [IPv6:2a01:4f8:191:4433::2]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 01FE9F9; Thu, 27 Apr 2023 08:59:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sipsolutions.net; s=mail; h=Content-Transfer-Encoding:MIME-Version: Message-Id:Date:Subject:Cc:To:From:Content-Type:Sender:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-To:Resent-Cc: Resent-Message-ID:In-Reply-To:References; bh=Cuucxv7D/U/64Bqs7uWjVu3OA9YWTZvXVvV+4QELO6Q=; t=1682611180; x=1683820780; b=B5Binqu88hbf3CBRFF0zAu3u6eA/cgIo5oaHBcOa8Sm1CmCAIrrgW1k9P+U+G8EZnom8VEF/1uj aP45T570CoJ+h/YnbGpC/C39gqrS0RixK/BAMN+59ZBMhNOzTd6KdfFeMu76wG3DsNhEqD/bmLjJv Wsi5114MfdzWF8ktrzBCXhT70Y+GbSWeJDihA/V0ZE6TXRWq4HRq1thHSZ+Pa13g45m4OlfAz1YjK mAN23mqHe3A8jhhghUfFQ6KiK8ia7jdjZbulFcbAbzAJDYiVKK+KnA8CGTabpEKHouDXp4ACZrbWM 8AepqrFXFj1Nx5SoaQ0vluz6cdlBwaOjRqOQ==; Received: by sipsolutions.net with esmtpsa (TLS1.3:ECDHE_X25519__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim 4.96) (envelope-from ) id 1ps41k-00AJvd-1D; Thu, 27 Apr 2023 17:59:24 +0200 From: Johannes Berg To: linux-trace-kernel@vger.kernel.org Cc: Steven Rostedt , Masami Hiramatsu , linux-kernel@vger.kernel.org, Johannes Berg Subject: [PATCH] tracing: ring-buffer: sync IRQ works before buffer destruction Date: Thu, 27 Apr 2023 17:59:20 +0200 Message-Id: <20230427175920.a76159263122.I8295e405c44362a86c995e9c2c37e3e03810aa56@changeid> X-Mailer: git-send-email 2.40.0 MIME-Version: 1.0 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,HEXHASH_WORD,SPF_HELO_PASS, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1764347535354873493?= X-GMAIL-MSGID: =?utf-8?q?1764347535354873493?= From: Johannes Berg If something was written to the buffer just before destruction, it may be possible (maybe not in a real system, but it did happen in ARCH=um with time-travel) to destroy the ringbuffer before the IRQ work ran, leading this KASAN report (or a crash without KASAN): BUG: KASAN: slab-use-after-free in irq_work_run_list+0x11a/0x13a Read of size 8 at addr 000000006d640a48 by task swapper/0 CPU: 0 PID: 0 Comm: swapper Tainted: G W O 6.3.0-rc1 #7 Stack: 60c4f20f 0c203d48 41b58ab3 60f224fc 600477fa 60f35687 60c4f20f 601273dd 00000008 6101eb00 6101eab0 615be548 Call Trace: [<60047a58>] show_stack+0x25e/0x282 [<60c609e0>] dump_stack_lvl+0x96/0xfd [<60c50d4c>] print_report+0x1a7/0x5a8 [<603078d3>] kasan_report+0xc1/0xe9 [<60308950>] __asan_report_load8_noabort+0x1b/0x1d [<60232844>] irq_work_run_list+0x11a/0x13a [<602328b4>] irq_work_tick+0x24/0x34 [<6017f9dc>] update_process_times+0x162/0x196 [<6019f335>] tick_sched_handle+0x1a4/0x1c3 [<6019fd9e>] tick_sched_timer+0x79/0x10c [<601812b9>] __hrtimer_run_queues.constprop.0+0x425/0x695 [<60182913>] hrtimer_interrupt+0x16c/0x2c4 [<600486a3>] um_timer+0x164/0x183 [...] Allocated by task 411: save_stack_trace+0x99/0xb5 stack_trace_save+0x81/0x9b kasan_save_stack+0x2d/0x54 kasan_set_track+0x34/0x3e kasan_save_alloc_info+0x25/0x28 ____kasan_kmalloc+0x8b/0x97 __kasan_kmalloc+0x10/0x12 __kmalloc+0xb2/0xe8 load_elf_phdrs+0xee/0x182 [...] The buggy address belongs to the object at 000000006d640800 which belongs to the cache kmalloc-1k of size 1024 The buggy address is located 584 bytes inside of freed 1024-byte region [000000006d640800, 000000006d640c00) Add the appropriate irq_work_sync() so the work finishes before the buffers are destroyed. Prior to the commit in the Fixes tag below, there was only a single global IRQ work, so this issue didn't exist. Fixes: 15693458c4bc ("tracing/ring-buffer: Move poll wake ups into ring buffer code") Signed-off-by: Johannes Berg --- kernel/trace/ring_buffer.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index af50d931b020..271997076029 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -1778,6 +1778,8 @@ static void rb_free_cpu_buffer(struct ring_buffer_per_cpu *cpu_buffer) struct list_head *head = cpu_buffer->pages; struct buffer_page *bpage, *tmp; + irq_work_sync(&cpu_buffer->irq_work.work); + free_buffer_page(cpu_buffer->reader_page); if (head) { @@ -1884,6 +1886,8 @@ ring_buffer_free(struct trace_buffer *buffer) cpuhp_state_remove_instance(CPUHP_TRACE_RB_PREPARE, &buffer->node); + irq_work_sync(&buffer->irq_work.work); + for_each_buffer_cpu(buffer, cpu) rb_free_cpu_buffer(buffer->buffers[cpu]);