From patchwork Wed Sep 27 18:41:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Mirsad Todorovac X-Patchwork-Id: 145537 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:cae8:0:b0:403:3b70:6f57 with SMTP id r8csp2833026vqu; Wed, 27 Sep 2023 11:54:09 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHvZnIgIMnb6U+9/JAgvl0SWzMnfVuS+L/rVTGIyLZs2KP8OmHs3gbtm8TdHSeKzBSAGomh X-Received: by 2002:a05:6358:881e:b0:13e:d853:cc22 with SMTP id hv30-20020a056358881e00b0013ed853cc22mr2928735rwb.4.1695840849014; Wed, 27 Sep 2023 11:54:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695840848; cv=none; d=google.com; s=arc-20160816; b=JNk/iMebJDShg5s/pJVCT8YSihcMuBlZMa9S508ef/oTIloAFIe9cN9KtrNJ0JdIu/ iQ/bb/x9jGPNCoEGZvka4y5hGIjf2crKqX3fZDVNGVwfHy3RnYXfDhHQkRPrmDJe9oRM q8lVJTy/2qt4w2lzN/5czUE1z7SOJjxj8jjiGxvOy/qSu7vPMAhq3i9FfCZhOnWFUglG VqWzSiQdD/WLwD6YxyWsFxNMRKs/g8rVS0iu5GvBjBU0zunhZkNTQQERYJ+fAPCeoebY /LyiXQ7BydajhGTJuKpHCsfR6uP+yyfyIsUe7Bj4pWhCDhQkYw/uU/fTKI0cjEJfI3ES Rfrg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature:dkim-signature; bh=coEohb0Sjq2Nfa3IGi19zcSwIvvpMEiL3KJ/gEw6CvM=; fh=7SsncGRbN74Z5AHCSqGaWZ8qQBPFU2OgoSp70ZMB+sA=; b=W9tzkHBi/6JPJNlQtba6BMpmfynwlI4X7qWzef1kho77S6KOiEPzwYGfChqZhbiKJx cEt/BJyV01/7TeLPE/pi1xUKlJTrS92mhslTryPyx4TnPff2uLWsVPj14UJZA9Ak2EPQ 0NOCx0T6OjjrIeIzoa6WcBV9EOmJcrdNrH/KkiQlKiLcofQDmomgax6WKfKIla+30xKT R8nYsLbeEGabKfr6cZg0T+2i+Cvx+85/DxvIH86Amb8QQh/ApZz514G6PAKk+b0ypr4T xQXLGk1c8I3nrdg/PA9lgdNJsK/SENkuudUBLlckHb0xK8dNMs0D/DwDXF7vzqhRECl3 abLA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@alu.unizg.hr header.s=mail header.b=sVT6Uh1g; dkim=fail header.i=@alu.unizg.hr header.s=mail header.b=EcAG9J8u; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alu.unizg.hr Received: from agentk.vger.email (agentk.vger.email. [23.128.96.32]) by mx.google.com with ESMTPS id f20-20020a637554000000b0057790a5a8e7si16973937pgn.190.2023.09.27.11.54.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Sep 2023 11:54:08 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) client-ip=23.128.96.32; Authentication-Results: mx.google.com; dkim=fail header.i=@alu.unizg.hr header.s=mail header.b=sVT6Uh1g; dkim=fail header.i=@alu.unizg.hr header.s=mail header.b=EcAG9J8u; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alu.unizg.hr Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id 9E913806AA18; Wed, 27 Sep 2023 11:45:13 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229542AbjI0Soz (ORCPT + 20 others); Wed, 27 Sep 2023 14:44:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42082 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229437AbjI0Sow (ORCPT ); Wed, 27 Sep 2023 14:44:52 -0400 Received: from domac.alu.hr (domac.alu.unizg.hr [IPv6:2001:b68:2:2800::3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 897088F; Wed, 27 Sep 2023 11:44:50 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by domac.alu.hr (Postfix) with ESMTP id 9A9F16017F; Wed, 27 Sep 2023 20:44:45 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=alu.unizg.hr; s=mail; t=1695840285; bh=1qH0GP+R/GxKsrMPBQpHw00MqLdFqpwgjdZGJ+r6ZEA=; h=From:To:Cc:Subject:Date:From; b=sVT6Uh1gy/o/m8pM+U/ZhbrAwq2iy1TGXvfKXNyvYPYC+gYcATKMmdMvCtfxPoHQu b9HTSyQY/zEdRNOeXmc0HQ0Def+mA0RpFQ7d+y20bmLIL+o/c9377mwSVaj55EYBlR ViRQdpa65yDgZhA4OjQIouthiX+J5SLHM3SnEjC5x6QQY7P4IwC3io7XoPkm9atdwM U/CwCv6+YSzaHbRDN1I/36LZWKKd8xcwpi5qewle1UmHAI3fzQ5Gz+lIfhO1umGkY0 x24x6pFBL5x2+SiiAveb4YsrjHIC6YXhnqWA+R2dUIOoxEB9sl19s5PHB1mhRh8FR7 uYMN0bB6g8flQ== X-Virus-Scanned: Debian amavisd-new at domac.alu.hr Received: from domac.alu.hr ([127.0.0.1]) by localhost (domac.alu.hr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yD9YtxdCfPDu; Wed, 27 Sep 2023 20:44:42 +0200 (CEST) Received: from defiant.home (78-2-200-97.adsl.net.t-com.hr [78.2.200.97]) by domac.alu.hr (Postfix) with ESMTPSA id CD65260174; Wed, 27 Sep 2023 20:44:41 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=alu.unizg.hr; s=mail; t=1695840282; bh=1qH0GP+R/GxKsrMPBQpHw00MqLdFqpwgjdZGJ+r6ZEA=; h=From:To:Cc:Subject:Date:From; b=EcAG9J8u9wKDk+Xcgc9K1XzL3IXinf4+1pXGuUc2emwv8uH1npi+6Z+cSmgyLcgKS geHdtDa6XP0B266Qr8IzwVts5iP0umKsyddtGukypt/O9iG5FkgvifN1oWj/meXifs z8so6vrK3OGZU/aUkep0ILj2NI31nIHPfvJoy8WgXxs2vLzyp2OL/gc9r5nG9492iv njsgG8doaX6aO0NRU2Web/GJhBa0QU3Q+ZC/nHQxfs7vA6RlDYr20SJP6aP3u84Pfb BLaFxMrB2Clg3woGJ92LQ871QeZ8OWE1BpgXdO9Mhd/x9I9ZvOYEM/R8QtWz9ZQJqm GPzxCTCkmUTFg== From: Mirsad Goran Todorovac To: Heiner Kallweit , netdev@vger.kernel.org, linux-kernel@vger.kernel.org Cc: nic_swsd@realtek.com, "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Mirsad Goran Todorovac , Marco Elver Subject: [PATCH RFC v1 1/3] r8169: fix the KCSAN reported data-race in rtl_tx() while reading tp->cur_tx Date: Wed, 27 Sep 2023 20:41:57 +0200 Message-Id: <20230927184158.243575-1-mirsad.todorovac@alu.unizg.hr> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-Spam-Status: No, score=-0.6 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Wed, 27 Sep 2023 11:45:13 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1778218014123270172 X-GMAIL-MSGID: 1778218014123270172 KCSAN reported the following data-race: ================================================================== BUG: KCSAN: data-race in rtl8169_poll [r8169] / rtl8169_start_xmit [r8169] write (marked) to 0xffff888102474b74 of 4 bytes by task 5358 on cpu 29: rtl8169_start_xmit (drivers/net/ethernet/realtek/r8169_main.c:4254) r8169 dev_hard_start_xmit (./include/linux/netdevice.h:4889 ./include/linux/netdevice.h:4903 net/core/dev.c:3544 net/core/dev.c:3560) sch_direct_xmit (net/sched/sch_generic.c:342) __dev_queue_xmit (net/core/dev.c:3817 net/core/dev.c:4306) ip_finish_output2 (./include/linux/netdevice.h:3082 ./include/net/neighbour.h:526 ./include/net/neighbour.h:540 net/ipv4/ip_output.c:233) __ip_finish_output (net/ipv4/ip_output.c:311 net/ipv4/ip_output.c:293) ip_finish_output (net/ipv4/ip_output.c:328) ip_output (net/ipv4/ip_output.c:435) ip_send_skb (./include/net/dst.h:458 net/ipv4/ip_output.c:127 net/ipv4/ip_output.c:1486) udp_send_skb (net/ipv4/udp.c:963) udp_sendmsg (net/ipv4/udp.c:1246) inet_sendmsg (net/ipv4/af_inet.c:840 (discriminator 4)) sock_sendmsg (net/socket.c:730 net/socket.c:753) __sys_sendto (net/socket.c:2177) __x64_sys_sendto (net/socket.c:2185) do_syscall_64 (arch/x86/entry/common.c:50 arch/x86/entry/common.c:80) entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:120) read to 0xffff888102474b74 of 4 bytes by interrupt on cpu 21: rtl8169_poll (drivers/net/ethernet/realtek/r8169_main.c:4397 drivers/net/ethernet/realtek/r8169_main.c:4581) r8169 __napi_poll (net/core/dev.c:6527) net_rx_action (net/core/dev.c:6596 net/core/dev.c:6727) __do_softirq (kernel/softirq.c:553) __irq_exit_rcu (kernel/softirq.c:427 kernel/softirq.c:632) irq_exit_rcu (kernel/softirq.c:647) common_interrupt (arch/x86/kernel/irq.c:247 (discriminator 14)) asm_common_interrupt (./arch/x86/include/asm/idtentry.h:636) cpuidle_enter_state (drivers/cpuidle/cpuidle.c:291) cpuidle_enter (drivers/cpuidle/cpuidle.c:390) call_cpuidle (kernel/sched/idle.c:135) do_idle (kernel/sched/idle.c:219 kernel/sched/idle.c:282) cpu_startup_entry (kernel/sched/idle.c:378 (discriminator 1)) start_secondary (arch/x86/kernel/smpboot.c:210 arch/x86/kernel/smpboot.c:294) secondary_startup_64_no_verify (arch/x86/kernel/head_64.S:433) value changed: 0x002f4815 -> 0x002f4816 Reported by Kernel Concurrency Sanitizer on: CPU: 21 PID: 0 Comm: swapper/21 Tainted: G L 6.6.0-rc2-kcsan-00143-gb5cbe7c00aa0 #41 Hardware name: ASRock X670E PG Lightning/X670E PG Lightning, BIOS 1.21 04/26/2023 ================================================================== The write side of drivers/net/ethernet/realtek/r8169_main.c is: ================== 4251 /* rtl_tx needs to see descriptor changes before updated tp->cur_tx */ 4252 smp_wmb(); 4253 → 4254 WRITE_ONCE(tp->cur_tx, tp->cur_tx + frags + 1); 4255 4256 stop_queue = !netif_subqueue_maybe_stop(dev, 0, rtl_tx_slots_avail(tp), 4257 R8169_TX_STOP_THRS, 4258 R8169_TX_START_THRS); The read side is the function rtl_tx(): 4355 static void rtl_tx(struct net_device *dev, struct rtl8169_private *tp, 4356 int budget) 4357 { 4358 unsigned int dirty_tx, bytes_compl = 0, pkts_compl = 0; 4359 struct sk_buff *skb; 4360 4361 dirty_tx = tp->dirty_tx; 4362 4363 while (READ_ONCE(tp->cur_tx) != dirty_tx) { 4364 unsigned int entry = dirty_tx % NUM_TX_DESC; 4365 u32 status; 4366 4367 status = le32_to_cpu(tp->TxDescArray[entry].opts1); 4368 if (status & DescOwn) 4369 break; 4370 4371 skb = tp->tx_skb[entry].skb; 4372 rtl8169_unmap_tx_skb(tp, entry); 4373 4374 if (skb) { 4375 pkts_compl++; 4376 bytes_compl += skb->len; 4377 napi_consume_skb(skb, budget); 4378 } 4379 dirty_tx++; 4380 } 4381 4382 if (tp->dirty_tx != dirty_tx) { 4383 dev_sw_netstats_tx_add(dev, pkts_compl, bytes_compl); 4384 WRITE_ONCE(tp->dirty_tx, dirty_tx); 4385 4386 netif_subqueue_completed_wake(dev, 0, pkts_compl, bytes_compl, 4387 rtl_tx_slots_avail(tp), 4388 R8169_TX_START_THRS); 4389 /* 4390 * 8168 hack: TxPoll requests are lost when the Tx packets are 4391 * too close. Let's kick an extra TxPoll request when a burst 4392 * of start_xmit activity is detected (if it is not detected, 4393 * it is slow enough). -- FR 4394 * If skb is NULL then we come here again once a tx irq is 4395 * triggered after the last fragment is marked transmitted. 4396 */ → 4397 if (tp->cur_tx != dirty_tx && skb) 4398 rtl8169_doorbell(tp); 4399 } 4400 } Obviously from the code, an earlier detected data-race for tp->cur_tx was fixed in the line 4363: 4363 while (READ_ONCE(tp->cur_tx) != dirty_tx) { but the same solution is required for protecting the other access to tp->cur_tx: → 4397 if (READ_ONCE(tp->cur_tx) != dirty_tx && skb) 4398 rtl8169_doorbell(tp); The write in the line 4254 is protected with WRITE_ONCE(), but the read in the line 4397 might have suffered read tearing under some compiler optimisations. The fix eliminated the KCSAN data-race report for this bug. It is yet to be evaluated what happens if tp->cur_tx changes between the test in line 4363 and line 4397. This test should certainly not be cached by the compiler in some register for such a long time, while asynchronous writes to tp->cur_tx might have occurred in line 4254 in the meantime. Fixes: 94d8a98e6235c ("r8169: reduce number of workaround doorbell rings") Cc: Heiner Kallweit Cc: nic_swsd@realtek.com Cc: "David S. Miller" Cc: Eric Dumazet Cc: Jakub Kicinski Cc: Paolo Abeni Cc: Marco Elver Cc: netdev@vger.kernel.org Link: https://lore.kernel.org/lkml/dc7fc8fa-4ea4-e9a9-30a6-7c83e6b53188@alu.unizg.hr/ Signed-off-by: Mirsad Goran Todorovac Acked-by: Marco Elver --- v1: the initial patch proposal. fixes the KCSAN warning. drivers/net/ethernet/realtek/r8169_main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/net/ethernet/realtek/r8169_main.c b/drivers/net/ethernet/realtek/r8169_main.c index 6351a2dc13bc..281aaa851847 100644 --- a/drivers/net/ethernet/realtek/r8169_main.c +++ b/drivers/net/ethernet/realtek/r8169_main.c @@ -4394,7 +4394,7 @@ static void rtl_tx(struct net_device *dev, struct rtl8169_private *tp, * If skb is NULL then we come here again once a tx irq is * triggered after the last fragment is marked transmitted. */ - if (tp->cur_tx != dirty_tx && skb) + if (READ_ONCE(tp->cur_tx) != dirty_tx && skb) rtl8169_doorbell(tp); } } From patchwork Wed Sep 27 18:41:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Mirsad Todorovac X-Patchwork-Id: 145538 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:cae8:0:b0:403:3b70:6f57 with SMTP id r8csp2833291vqu; Wed, 27 Sep 2023 11:54:44 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFF54H47dlqwK1iEuvfIscm6hyIWvoXuX38pq8rH49x3DPZo1EZj3MRa/MqLo3VnEuQWe9U X-Received: by 2002:a17:902:e80e:b0:1c2:193e:1126 with SMTP id u14-20020a170902e80e00b001c2193e1126mr4702842plg.28.1695840883875; Wed, 27 Sep 2023 11:54:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695840883; cv=none; d=google.com; s=arc-20160816; b=a2Om8iM7hZEjVo/dPasEVHPSE4LxAl9ZzWeoeg8wUUVGNvfVjCmIM4ObJGU7rE825u +D5D6fxFzZdPx3htPyNKPXeTtLAJa+yN0q2JQvB035gLiko3zOstvPOHpSTxcH5DeaV8 Ax7HcL7Q6MLSOD79mnnzukIFMzYMq9m1yHWPTEkqCR7lv31QzfE4jQfql1rn8gUCcngB kGT+0y8w7fXGFlunLKijma/EAw4XzYwwLxgBXuotnvMaLtFSrFffKsxqhdZ0863s6OOG JARosFeUMkFXokt9fwcjXJNJlLmwW8FG+sz0p28vhzFtxaUa8siW2sh/nrogLAuBu34J By7Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:dkim-signature; bh=/+orp7T6b6oFx/gh5swcg7kiZ72CI2uLAcmraU7PXLg=; fh=7SsncGRbN74Z5AHCSqGaWZ8qQBPFU2OgoSp70ZMB+sA=; b=qn1e0Mzxy4Shxv/kzfOsn+dfl+QGByMZlIsnUANCjQAwJmO5JRiTqRH5MAW22NfreF CBzHMfGSkZyJj4H57zdGjbvi3QlfmmH/WLEK5ou2VBbf9hJIvm294b8UUppHXCZfkt7l v12ZzQppQkHr2yhAkN2iYuhAtNaKXc2oAQ4kdBWmoS0qbxGQab+egDeBzbthNpf1avTg KKT5/Hg3/nV050lHYi80jjzhGIvnvpwWUppycPEKwvAu6aaO8IuPklTyzurogBb4UY5n EOHHAXyHANk7KRHr5CBaXrZxu2agtZnOlW/VipG/1naXS0V60C9hdW47S4Eaki0ctt+8 SOlw== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@alu.unizg.hr header.s=mail header.b="Vmm/E2i+"; dkim=fail header.i=@alu.unizg.hr header.s=mail header.b=Sz1VZjlP; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alu.unizg.hr Received: from agentk.vger.email (agentk.vger.email. [23.128.96.32]) by mx.google.com with ESMTPS id u12-20020a170902e5cc00b001c5c344a425si10376073plf.418.2023.09.27.11.54.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Sep 2023 11:54:43 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) client-ip=23.128.96.32; Authentication-Results: mx.google.com; dkim=fail header.i=@alu.unizg.hr header.s=mail header.b="Vmm/E2i+"; dkim=fail header.i=@alu.unizg.hr header.s=mail header.b=Sz1VZjlP; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alu.unizg.hr Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id B7E89801F112; Wed, 27 Sep 2023 11:46:10 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229568AbjI0Spk (ORCPT + 20 others); Wed, 27 Sep 2023 14:45:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47542 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229437AbjI0Spj (ORCPT ); Wed, 27 Sep 2023 14:45:39 -0400 Received: from domac.alu.hr (domac.alu.unizg.hr [IPv6:2001:b68:2:2800::3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D5B6FE6; Wed, 27 Sep 2023 11:45:37 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by domac.alu.hr (Postfix) with ESMTP id 8A2526017F; Wed, 27 Sep 2023 20:45:36 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=alu.unizg.hr; s=mail; t=1695840336; bh=jCUkxm8G1cRHqM0Z9NMo4Scij68gCyXJSLjxVEFYhK8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Vmm/E2i+UQIrYXgrkKdDbqQsXDSnRfM9A3xANbIIEZsUiZwC/kP5gBf9mdoTTKez/ xImMuP1G6mfUPVtBfPB0x/aDGALlS6Usstpc6SPIHbdv1jmzguxJ2msRneBd9vaDgk QJDDB8CeIghPo09ysR/Ow3d/xmlfFQFNM5WqJ3fWb14ZgLRU4DNIm5k20C+MxANl59 DEvT3p6v2lkzkC+56hSEpuGy2X6a2Yryn18wnD1cUn+5FqG0eqfh8JlpkgkvmSSXgi Kq4pYAYU4IuRFZfKncM7CDQ/gL98pG6jpppe5rLbULVFMMTIJHjyOPriWTNN50YQT2 qKT0pbB58jp9g== X-Virus-Scanned: Debian amavisd-new at domac.alu.hr Received: from domac.alu.hr ([127.0.0.1]) by localhost (domac.alu.hr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KK14IOTF3v0S; Wed, 27 Sep 2023 20:45:33 +0200 (CEST) Received: from defiant.home (78-2-200-97.adsl.net.t-com.hr [78.2.200.97]) by domac.alu.hr (Postfix) with ESMTPSA id BCCEB60174; Wed, 27 Sep 2023 20:45:33 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=alu.unizg.hr; s=mail; t=1695840333; bh=jCUkxm8G1cRHqM0Z9NMo4Scij68gCyXJSLjxVEFYhK8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Sz1VZjlP3pp7oLnmqR8DU+0cZdw/mXlESmiccRiDX26R1Fv7vKKOpJSoScsUvAlEz xzx7W/yGOvGMYUbSM+Gg5tknbsIX3k3SLNrc5X7Jhm4wuTLEfGJclYxu+4HxYJm986 /yqdAvWG6KeD14aEor2uro8zmRRsRB/we/b24UZRgEkDGuywvHU+tKDa/jQ4ccjYDY ZxLzeBgVIIrlCMqziExchoWNsPH15t4KkVA8KuIRiwasxnQ0gNtVAr4Y2552sPMom/ jT/t/Y6m2IU+NYMJR3GoOSoDWcp//qElYoZaqJjDSqhRe4NY7ecy8g8k7aBc9Qpf4l vnIZGa9fFe7GQ== From: Mirsad Goran Todorovac To: Heiner Kallweit , netdev@vger.kernel.org, linux-kernel@vger.kernel.org Cc: nic_swsd@realtek.com, "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Mirsad Goran Todorovac , Marco Elver Subject: [PATCH RFC v1 2/3] r8169: fix the KCSAN reported data race in rtl_rx while reading desc->opts1 Date: Wed, 27 Sep 2023 20:41:59 +0200 Message-Id: <20230927184158.243575-2-mirsad.todorovac@alu.unizg.hr> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230927184158.243575-1-mirsad.todorovac@alu.unizg.hr> References: <20230927184158.243575-1-mirsad.todorovac@alu.unizg.hr> MIME-Version: 1.0 X-Spam-Status: No, score=-0.6 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Wed, 27 Sep 2023 11:46:10 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1778218050966217899 X-GMAIL-MSGID: 1778218050966217899 KCSAN reported the following data-race bug: ================================================================== BUG: KCSAN: data-race in rtl8169_poll (drivers/net/ethernet/realtek/r8169_main.c:4430 drivers/net/ethernet/realtek/r8169_main.c:4583) r8169 race at unknown origin, with read to 0xffff888117e43510 of 4 bytes by interrupt on cpu 21: rtl8169_poll (drivers/net/ethernet/realtek/r8169_main.c:4430 drivers/net/ethernet/realtek/r8169_main.c:4583) r8169 __napi_poll (net/core/dev.c:6527) net_rx_action (net/core/dev.c:6596 net/core/dev.c:6727) __do_softirq (kernel/softirq.c:553) __irq_exit_rcu (kernel/softirq.c:427 kernel/softirq.c:632) irq_exit_rcu (kernel/softirq.c:647) sysvec_apic_timer_interrupt (arch/x86/kernel/apic/apic.c:1074 (discriminator 14)) asm_sysvec_apic_timer_interrupt (./arch/x86/include/asm/idtentry.h:645) cpuidle_enter_state (drivers/cpuidle/cpuidle.c:291) cpuidle_enter (drivers/cpuidle/cpuidle.c:390) call_cpuidle (kernel/sched/idle.c:135) do_idle (kernel/sched/idle.c:219 kernel/sched/idle.c:282) cpu_startup_entry (kernel/sched/idle.c:378 (discriminator 1)) start_secondary (arch/x86/kernel/smpboot.c:210 arch/x86/kernel/smpboot.c:294) secondary_startup_64_no_verify (arch/x86/kernel/head_64.S:433) value changed: 0x80003fff -> 0x3402805f Reported by Kernel Concurrency Sanitizer on: CPU: 21 PID: 0 Comm: swapper/21 Tainted: G L 6.6.0-rc2-kcsan-00143-gb5cbe7c00aa0 #41 Hardware name: ASRock X670E PG Lightning/X670E PG Lightning, BIOS 1.21 04/26/2023 ================================================================== drivers/net/ethernet/realtek/r8169_main.c: ========================================== 4429 → 4430 status = le32_to_cpu(desc->opts1); 4431 if (status & DescOwn) 4432 break; 4433 4434 /* This barrier is needed to keep us from reading 4435 * any other fields out of the Rx descriptor until 4436 * we know the status of DescOwn 4437 */ 4438 dma_rmb(); 4439 4440 if (unlikely(status & RxRES)) { 4441 if (net_ratelimit()) 4442 netdev_warn(dev, "Rx ERROR. status = %08x\n", Marco Elver explained that dma_rmb() doesn't prevent the compiler to tear up the access to desc->opts1 which can be written to concurrently. READ_ONCE() should prevent that from happening: 4429 → 4430 status = le32_to_cpu(READ_ONCE(desc->opts1)); 4431 if (status & DescOwn) 4432 break; 4433 As the consequence of this fix, this KCSAN warning was eliminated. Fixes: 6202806e7c03a ("r8169: drop member opts1_mask from struct rtl8169_private") Suggested-by: Marco Elver Cc: Heiner Kallweit Cc: nic_swsd@realtek.com Cc: "David S. Miller" Cc: Eric Dumazet Cc: Jakub Kicinski Cc: Paolo Abeni Cc: netdev@vger.kernel.org Link: https://lore.kernel.org/lkml/dc7fc8fa-4ea4-e9a9-30a6-7c83e6b53188@alu.unizg.hr/ Signed-off-by: Mirsad Goran Todorovac Signed-off-by: Mirsad Goran Todorovac --- v1: the initial patch proposal. fixes the KCSAN warning. drivers/net/ethernet/realtek/r8169_main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/net/ethernet/realtek/r8169_main.c b/drivers/net/ethernet/realtek/r8169_main.c index 281aaa851847..81be6085a480 100644 --- a/drivers/net/ethernet/realtek/r8169_main.c +++ b/drivers/net/ethernet/realtek/r8169_main.c @@ -4427,7 +4427,7 @@ static int rtl_rx(struct net_device *dev, struct rtl8169_private *tp, int budget dma_addr_t addr; u32 status; - status = le32_to_cpu(desc->opts1); + status = le32_to_cpu(READ_ONCE(desc->opts1)); if (status & DescOwn) break; From patchwork Wed Sep 27 18:42:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Mirsad Todorovac X-Patchwork-Id: 145539 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:cae8:0:b0:403:3b70:6f57 with SMTP id r8csp2833364vqu; Wed, 27 Sep 2023 11:54:53 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGMdOvi6jxi3b6VKGa7D17g57yR4h65TXAi/GYDfV3nMGTZ6u9q33Flhdsz5eP7hI4iOcbg X-Received: by 2002:a05:6a20:4306:b0:140:d536:d434 with SMTP id h6-20020a056a20430600b00140d536d434mr4802269pzk.10.1695840893003; Wed, 27 Sep 2023 11:54:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695840892; cv=none; d=google.com; s=arc-20160816; b=fAXMKGwbgjFZvzky0mp3ReqPf9MWoYZtcoY8i+6pR3BPTO/hHrNQOtHqaaxJOr6jsw 6GaRmnde/IjboaDy6eklZhRQuypSuYlmC5sXoelN6A64QUsxsGu52yJL1lGH3jjOhfTJ RvNOjPgGV3p3HA/05uH3IzTf8+Hu3aXoHIVfF39fGAKd0vleliwteiCCQ5MB2uJJpyvR ONTwrbuqV++jy1+N/DFTmoAV3rAdqgb0xIe+oA0HuXD3Ou5QV4MrSkbm62FovcqLHVND ymVWF1CGt9tOC65osRVm/ExgAiyETCtAwqY8WYneVW8St3NN1j/Yu2qkU/xXQjEBjXnC 3cXg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:dkim-signature; bh=dUqLSSdHEBJ+KA8+mv0OKPagXGgN2rO+oDpjT6foCzM=; fh=7SsncGRbN74Z5AHCSqGaWZ8qQBPFU2OgoSp70ZMB+sA=; b=0Eei9nBxbym7J7JnjDdkv1x4WTu1lIIaSALG8H0PnzMyyTAIODbh266Hnj0pBcFiHR upwhrBE/rgeZAH/XkvdPU/oT/QvF3mu4EVQPkdprk6zbLpMGEADizJl229fYUhrkt+YS sIrcdtL/uSCOlLOQYEmwWEJnVttcfH0K6q6Awu0A47icFMgqF7M3x5gQaZQKX71nevAY rNhw3Cln4Lvm9Dnv06d4W/PqVbHmUGbXv94XEZgQAdji4LP6IgaszSRInjXUFLjZC1mt uzDWJgsNs49niwZCI+o8PrG3fwUpqJMGEHIIykBTtvU8bs+noKN2p/mMwNNYcvwy7DdJ Bsiw== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@alu.unizg.hr header.s=mail header.b=LIVle9WZ; dkim=fail header.i=@alu.unizg.hr header.s=mail header.b=vtx2qKY2; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alu.unizg.hr Received: from agentk.vger.email (agentk.vger.email. [2620:137:e000::3:2]) by mx.google.com with ESMTPS id cm21-20020a056a00339500b006933e5559a3si1694477pfb.310.2023.09.27.11.54.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Sep 2023 11:54:52 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) client-ip=2620:137:e000::3:2; Authentication-Results: mx.google.com; dkim=fail header.i=@alu.unizg.hr header.s=mail header.b=LIVle9WZ; dkim=fail header.i=@alu.unizg.hr header.s=mail header.b=vtx2qKY2; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alu.unizg.hr Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id C75BF801D8CA; Wed, 27 Sep 2023 11:46:41 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229594AbjI0SqU (ORCPT + 20 others); Wed, 27 Sep 2023 14:46:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37962 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229437AbjI0SqT (ORCPT ); Wed, 27 Sep 2023 14:46:19 -0400 Received: from domac.alu.hr (domac.alu.unizg.hr [161.53.235.3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 497358F; Wed, 27 Sep 2023 11:46:18 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by domac.alu.hr (Postfix) with ESMTP id 6EB866017F; Wed, 27 Sep 2023 20:46:16 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=alu.unizg.hr; s=mail; t=1695840376; bh=eyWyK7TlkzBJCtBBJ/CXcDT6nfZD6J5YeGBFsunPCag=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LIVle9WZOWn6aD6dqLkakqPBGJI3QKKhpIvj4ohX77NetK7FZsmrMb0dh6jpz2/Nv 3Nm7eN698nE3Y6Zbv6KGJFMx98YL+m5uDMPUpC2KyWE096xMld/G5JQbYFyN2TJ1Bp kUHwV6O7ANKltpaV5MRxUgq2rNDb2r4jJBREpn9HTPfG4Ac5Nvt3tKR2fwy0ihogY1 OER1Bk9x5ZCM1/c3eLguY7jZhgXIw3jyLox0w0nlEKPTagloD6OY4OOXXAH40+M45P nNq43PrxSccmaV6Y5bfL9AkqMELa4o5p53WQA77sC3ehV2fljR7ICbJfe4FSeJ8NWy WGqcrejHSJwiA== X-Virus-Scanned: Debian amavisd-new at domac.alu.hr Received: from domac.alu.hr ([127.0.0.1]) by localhost (domac.alu.hr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UDcH-kqXjvO1; Wed, 27 Sep 2023 20:46:13 +0200 (CEST) Received: from defiant.home (78-2-200-97.adsl.net.t-com.hr [78.2.200.97]) by domac.alu.hr (Postfix) with ESMTPSA id C8ECA60174; Wed, 27 Sep 2023 20:46:13 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=alu.unizg.hr; s=mail; t=1695840373; bh=eyWyK7TlkzBJCtBBJ/CXcDT6nfZD6J5YeGBFsunPCag=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=vtx2qKY2k77KFOwaunjM9GBleUhuDJYnVeBrTA+d87zHihAMSc/XVd4H8fRvcXc4i 3vKDMaQ0IHXW3sXAjATPQZx6+/AAKSUOFRanT+X2LlQrh3Uqm2nOiaEF5bM/2A4v1q gZ2BOrinV6U4133GQ5s8VPGVrM1+6b8pAagJhitIZ+xKXSbCTZFlAXC5610A2Inj3H HsO1/P5oQK8nKaCgMvT4fyl2XeKValBfQDGjTq7BIxveynrL18qJ20GhWfZZEBssgt agUEzi9QruhXqaECETnvhZy5zaRQBzkGgX4NkKwNkvgtJsJMWhCPYaaxsgCIskMb3L SakgHwLQJuFIA== From: Mirsad Goran Todorovac To: Heiner Kallweit , netdev@vger.kernel.org, linux-kernel@vger.kernel.org Cc: nic_swsd@realtek.com, "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Mirsad Goran Todorovac , Marco Elver Subject: [PATCH RFC v1 3/3] r8169: fix the KCSAN reported data-race in rtl_tx while reading TxDescArray[entry].opts1 Date: Wed, 27 Sep 2023 20:42:01 +0200 Message-Id: <20230927184158.243575-3-mirsad.todorovac@alu.unizg.hr> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230927184158.243575-1-mirsad.todorovac@alu.unizg.hr> References: <20230927184158.243575-1-mirsad.todorovac@alu.unizg.hr> MIME-Version: 1.0 X-Spam-Status: No, score=-0.6 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Wed, 27 Sep 2023 11:46:41 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1778218060252046826 X-GMAIL-MSGID: 1778218060252046826 KCSAN reported the following data-race: ================================================================== BUG: KCSAN: data-race in rtl8169_poll (drivers/net/ethernet/realtek/r8169_main.c:4368 drivers/net/ethernet/realtek/r8169_main.c:4581) r8169 race at unknown origin, with read to 0xffff888140d37570 of 4 bytes by interrupt on cpu 21: rtl8169_poll (drivers/net/ethernet/realtek/r8169_main.c:4368 drivers/net/ethernet/realtek/r8169_main.c:4581) r8169 __napi_poll (net/core/dev.c:6527) net_rx_action (net/core/dev.c:6596 net/core/dev.c:6727) __do_softirq (kernel/softirq.c:553) __irq_exit_rcu (kernel/softirq.c:427 kernel/softirq.c:632) irq_exit_rcu (kernel/softirq.c:647) sysvec_apic_timer_interrupt (arch/x86/kernel/apic/apic.c:1074 (discriminator 14)) asm_sysvec_apic_timer_interrupt (./arch/x86/include/asm/idtentry.h:645) cpuidle_enter_state (drivers/cpuidle/cpuidle.c:291) cpuidle_enter (drivers/cpuidle/cpuidle.c:390) call_cpuidle (kernel/sched/idle.c:135) do_idle (kernel/sched/idle.c:219 kernel/sched/idle.c:282) cpu_startup_entry (kernel/sched/idle.c:378 (discriminator 1)) start_secondary (arch/x86/kernel/smpboot.c:210 arch/x86/kernel/smpboot.c:294) secondary_startup_64_no_verify (arch/x86/kernel/head_64.S:433) value changed: 0xb0000042 -> 0x00000000 Reported by Kernel Concurrency Sanitizer on: CPU: 21 PID: 0 Comm: swapper/21 Tainted: G L 6.6.0-rc2-kcsan-00143-gb5cbe7c00aa0 #41 Hardware name: ASRock X670E PG Lightning/X670E PG Lightning, BIOS 1.21 04/26/2023 ================================================================== The read side is in drivers/net/ethernet/realtek/r8169_main.c ========================================= 4355 static void rtl_tx(struct net_device *dev, struct rtl8169_private *tp, 4356 int budget) 4357 { 4358 unsigned int dirty_tx, bytes_compl = 0, pkts_compl = 0; 4359 struct sk_buff *skb; 4360 4361 dirty_tx = tp->dirty_tx; 4362 4363 while (READ_ONCE(tp->cur_tx) != dirty_tx) { 4364 unsigned int entry = dirty_tx % NUM_TX_DESC; 4365 u32 status; 4366 → 4367 status = le32_to_cpu(tp->TxDescArray[entry].opts1); 4368 if (status & DescOwn) 4369 break; 4370 4371 skb = tp->tx_skb[entry].skb; 4372 rtl8169_unmap_tx_skb(tp, entry); 4373 4374 if (skb) { 4375 pkts_compl++; 4376 bytes_compl += skb->len; 4377 napi_consume_skb(skb, budget); 4378 } 4379 dirty_tx++; 4380 } 4381 4382 if (tp->dirty_tx != dirty_tx) { 4383 dev_sw_netstats_tx_add(dev, pkts_compl, bytes_compl); 4384 WRITE_ONCE(tp->dirty_tx, dirty_tx); 4385 4386 netif_subqueue_completed_wake(dev, 0, pkts_compl, bytes_compl, 4387 rtl_tx_slots_avail(tp), 4388 R8169_TX_START_THRS); 4389 /* 4390 * 8168 hack: TxPoll requests are lost when the Tx packets are 4391 * too close. Let's kick an extra TxPoll request when a burst 4392 * of start_xmit activity is detected (if it is not detected, 4393 * it is slow enough). -- FR 4394 * If skb is NULL then we come here again once a tx irq is 4395 * triggered after the last fragment is marked transmitted. 4396 */ 4397 if (READ_ONCE(tp->cur_tx) != dirty_tx && skb) 4398 rtl8169_doorbell(tp); 4399 } 4400 } tp->TxDescArray[entry].opts1 is reported to have a data-race and READ_ONCE() fixes this KCSAN warning. 4366 → 4367 status = le32_to_cpu(READ_ONCE(tp->TxDescArray[entry].opts1)); 4368 if (status & DescOwn) 4369 break; 4370 Fixes: ^1da177e4c3f4 ("initial git repository build") Cc: Heiner Kallweit Cc: nic_swsd@realtek.com Cc: "David S. Miller" Cc: Eric Dumazet Cc: Jakub Kicinski Cc: Paolo Abeni Cc: Marco Elver Cc: netdev@vger.kernel.org Link: https://lore.kernel.org/lkml/dc7fc8fa-4ea4-e9a9-30a6-7c83e6b53188@alu.unizg.hr/ Signed-off-by: Mirsad Goran Todorovac --- v1: the initial patch proposal. fixes the KCSAN warning. drivers/net/ethernet/realtek/r8169_main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/net/ethernet/realtek/r8169_main.c b/drivers/net/ethernet/realtek/r8169_main.c index 81be6085a480..361b90007148 100644 --- a/drivers/net/ethernet/realtek/r8169_main.c +++ b/drivers/net/ethernet/realtek/r8169_main.c @@ -4364,7 +4364,7 @@ static void rtl_tx(struct net_device *dev, struct rtl8169_private *tp, unsigned int entry = dirty_tx % NUM_TX_DESC; u32 status; - status = le32_to_cpu(tp->TxDescArray[entry].opts1); + status = le32_to_cpu(READ_ONCE(tp->TxDescArray[entry].opts1)); if (status & DescOwn) break;