From patchwork Sun Oct 29 11:04:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mirsad Todorovac X-Patchwork-Id: 159353 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:d641:0:b0:403:3b70:6f57 with SMTP id cy1csp1630528vqb; Sun, 29 Oct 2023 04:12:05 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGS9mLLd4XZKILdMJuYHhPBXEGW0+pntU3ZXGR/38N78AqmjY6zxvWBXC7duMPz0Zf/U5bU X-Received: by 2002:a05:6a00:114c:b0:6be:59f:5172 with SMTP id b12-20020a056a00114c00b006be059f5172mr9176265pfm.19.1698577925148; Sun, 29 Oct 2023 04:12:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1698577925; cv=none; d=google.com; s=arc-20160816; b=RmUClQB3NPjfzRGDxMsqmD6sxz4uFgmlKuJ82fpSqurm4+JG1csnLlm1ZBpUlPNusw Q+eaCP/64NpQIrPScM0Bv9RD3rI9zx2qxTwOyPoIg3/RYjTmMnRViNUvtxydJFJ7JhmH xuEAxaa0wo5s1YkqIGP9ETQAIfgasp5Is2UcpwaUdwtaTnnrKZe74u86BIaG7oGS/VYD mW3L6/Ia1JUP/+UMbISNrS0gQuKSeabvwpS8QVPMEjb7fjyfH3MJqn7cPTm9JYeUwdns ucWtUIKm8e0mUE0smx6TIUGDWvtEv5uA9FBLO+dDrT7TlzdwMZRcsE07veg81uzxuBHf AnTw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature:dkim-signature; bh=Q7xgKmwRk0zslQJqz2h8+WxDnDmFQ8+lUeSUqDTSedg=; fh=SsNJs69sMYlYTeWQYB3Y6uFVnk1BYEiSEalBcmc5mfg=; b=G5fiKCX0xhVHW0yYt3k/e8aCdBhmdUZxr8rNrHjcDJsMbOOkw7tVuAY5fE2sXF9LO8 yc+157BxvC8iKhT6YNcYVrTYdNwRcm6HufEAoSFHyWVKiGDU8i4frA0yzDnZsd6LpgQA KhLsSVaGhOZir+ExBEOs8l+/PrRKwr4of/qJ0LCRIcv6c6N5QwexUuukQnJGI5U4sWAX S2YlETR1G/k1d+XzuRvjYlyOszbyZ0HLOrouXVR2ALEXcEvZ9h24hicoIBpIDFe473HM rSbMzXe8xiW/h8x5fs5XUJEThDsbKHA2dBbnzyxGAvnIHYuYxkkMArGU82kunNqabqc3 ktJw== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@alu.unizg.hr header.s=mail header.b=g7zJywWK; dkim=fail header.i=@alu.unizg.hr header.s=mail header.b=hnDTJdP5; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alu.unizg.hr Received: from groat.vger.email (groat.vger.email. [2620:137:e000::3:5]) by mx.google.com with ESMTPS id c11-20020a056a00248b00b006be322191desi3551298pfv.112.2023.10.29.04.12.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 29 Oct 2023 04:12:05 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) client-ip=2620:137:e000::3:5; Authentication-Results: mx.google.com; dkim=fail header.i=@alu.unizg.hr header.s=mail header.b=g7zJywWK; dkim=fail header.i=@alu.unizg.hr header.s=mail header.b=hnDTJdP5; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alu.unizg.hr Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by groat.vger.email (Postfix) with ESMTP id 6A9888087FE8; Sun, 29 Oct 2023 04:12:01 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at groat.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230050AbjJ2LLn (ORCPT + 30 others); Sun, 29 Oct 2023 07:11:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52542 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229529AbjJ2LLm (ORCPT ); Sun, 29 Oct 2023 07:11:42 -0400 Received: from domac.alu.hr (domac.alu.unizg.hr [161.53.235.3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A7DCDC0; Sun, 29 Oct 2023 04:11:38 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by domac.alu.hr (Postfix) with ESMTP id 3C06460182; Sun, 29 Oct 2023 12:11:36 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=alu.unizg.hr; s=mail; t=1698577896; bh=DY2xKmVncOybZOf1srDA6dHzjor+EqjADbirBoJ9WQI=; h=From:To:Cc:Subject:Date:From; b=g7zJywWKTkgUxWQWCdk8k6QlKXx8JheOwufuyZyZA/NzgWv5G6AzxAWA9F23KeMVW SccOSD8OMkYvuUWpczS5OHkYXHthlhPgjZ3RpFPTrLggqMSjlj9t3IFrLpij9bLWZz OAxjCMjpxZYbRmiKo53np1iVBenQ6eNBIRXgB+IRj/3ZBtuMcaVnc6SmdAaviEKou/ zpYT2lB92ot1Jy+LXPP+ZesAtX4DJb5lB54NMioA8qhaMrWU8mUhMLe6WUEFrssUKM Jc9nKj1yz+3Zxyw+cpGAIQ6FnqzNwNCwVOuqEKKP/TYMHbe3ex2cW2+ryBzPAdi7lG wMNRSM2IimfUw== X-Virus-Scanned: Debian amavisd-new at domac.alu.hr Received: from domac.alu.hr ([127.0.0.1]) by localhost (domac.alu.hr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hf1umPGTCaNK; Sun, 29 Oct 2023 12:11:33 +0100 (CET) Received: from defiant.home (78-3-40-247.adsl.net.t-com.hr [78.3.40.247]) by domac.alu.hr (Postfix) with ESMTPSA id 7742760173; Sun, 29 Oct 2023 12:11:30 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=alu.unizg.hr; s=mail; t=1698577893; bh=DY2xKmVncOybZOf1srDA6dHzjor+EqjADbirBoJ9WQI=; h=From:To:Cc:Subject:Date:From; b=hnDTJdP5KrlZ0+t9BybL+vDpfUkO07Z8v37kope2ILiVUpezVuIijgDL8qP3p6cIV 49kJ4nkWMcUFlJH/ZzNcbsx9S3s31ROnhK8mlRyvtgHdwziX0w4M5NNuxicGVhdSHn wZC3mjOD2xkbSnh9Ec/x4mW2w4ZAl6aLTKBltGSHt436KQbbu6TXy3/WQfKx5MLRy0 NVkt+MGdU+yKqWmfxLBLGXtS5YOPP6fPF5sxw6PyDH7Q7ey1cOcJ5CEXT+aT9Xtccz pFv/YG/gs7+PCATS5koYRNPsjOLLD+k1jZZVONqT3Zvbfih+mYXZIpQQ3g0dpw0wWJ mkTycYMNpmpRg== From: Mirsad Goran Todorovac To: Jason Gunthorpe , Joerg Roedel , Lu Baolu , iommu@lists.linux.dev, linux-kernel@vger.kernel.org, netdev@vger.kernel.org Cc: Joerg Roedel , Will Deacon , Robin Murphy , Heiner Kallweit , nic_swsd@realtek.com, "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Mirsad Goran Todorovac , Marco Elver Subject: [PATCH v4 1/5] r8169: Coalesce r8169_mac_ocp_write/modify calls to reduce spinlock stalls Date: Sun, 29 Oct 2023 12:04:39 +0100 Message-Id: <20231029110442.347448-1-mirsad.todorovac@alu.unizg.hr> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-Spam-Status: No, score=-0.6 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Sun, 29 Oct 2023 04:12:01 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1781088046550942752 X-GMAIL-MSGID: 1781088046550942752 A pair of new helpers r8168_mac_ocp_write_seq() and r8168_mac_ocp_modify_seq() are introduced. The motivation for these helpers was the locking overhead of 130 consecutive r8168_mac_ocp_write() calls in the RTL8411b reset after the NIC gets confused if the PHY is powered-down. To quote Heiner: On RTL8411b the RX unit gets confused if the PHY is powered-down. This was reported in [0] and confirmed by Realtek. Realtek provided a sequence to fix the RX unit after PHY wakeup. A series of about 130 r8168_mac_ocp_write() calls is performed to program the RTL registers for recovery, each doing an expensive spin_lock_irqsave() and spin_unlock_irqrestore(). Each mac ocp write is made of: static void __r8168_mac_ocp_write(struct rtl8169_private *tp, u32 reg, u32 data) { if (rtl_ocp_reg_failure(reg)) return; RTL_W32(tp, OCPDR, OCPAR_FLAG | (reg << 15) | data); } static void r8168_mac_ocp_write(struct rtl8169_private *tp, u32 reg, u32 data) { unsigned long flags; raw_spin_lock_irqsave(&tp->mac_ocp_lock, flags); __r8168_mac_ocp_write(tp, reg, data); raw_spin_unlock_irqrestore(&tp->mac_ocp_lock, flags); } Register programming is done through RTL_W32() macro which expands into #define RTL_W32(tp, reg, val32) writel((val32), tp->mmio_addr + (reg)) which is further (on Alpha): extern inline void writel(u32 b, volatile void __iomem *addr) { mb(); __raw_writel(b, addr); } or on i386/x86_64: #define build_mmio_write(name, size, type, reg, barrier) \ static inline void name(type val, volatile void __iomem *addr) \ { asm volatile("mov" size " %0,%1": :reg (val), \ "m" (*(volatile type __force *)addr) barrier); } build_mmio_write(writel, "l", unsigned int, "r", :"memory") This obviously involves iat least a compiler barrier. mb() expands into something like this i.e. on x86_64: #define mb() asm volatile("lock; addl $0,0(%%esp)" ::: "memory") This means a whole lot of memory bus stalls: for spin_lock_irqsave(), memory barrier, writel(), and spin_unlock_irqrestore(). With about 130 of these sequential calls to r8168_mac_ocp_write() this looks like a lock storm that will stall all of the cores and CPUs on the same memory controller for certain time I/O takes to finish. In a sequential case of RTL register programming, the writes to RTL registers can be coalesced under a same raw spinlock. This can dramatically decrease the number of bus stalls in a multicore or multi-CPU system. Macro helpers r8168_mac_ocp_write_seq() and r8168_mac_ocp_modify_seq() are provided to reduce lock contention: static void rtl_hw_start_8411_2(struct rtl8169_private *tp) { ... /* The following Realtek-provided magic fixes an issue with the RX unit * getting confused after the PHY having been powered-down. */ static const struct recover_8411b_info init_zero_seq[] = { { 0xFC28, 0x0000 }, { 0xFC2A, 0x0000 }, { 0xFC2C, 0x0000 }, ... }; ... r8168_mac_ocp_write_seq(tp, init_zero_seq); ... } The hex data is preserved intact through s/r8168_mac_ocp_write[(]tp,/{ / and s/[)];/ },/ functions that only changed the function names and the ending of the line, so the actual hex data is unchanged. To repeat, the reason for the introduction of the original commit was to enable recovery of the RX unit on the RTL8411b which was confused by the powered-down PHY. This sequence of r8168_mac_ocp_write() calls amplifies the problem into a series of about 500+ memory bus locks, most waiting for the main memory read, modify and write under a LOCK. The memory barrier in RTL_W32 should suffice for the programming sequence to reach RTL NIC registers. [0] https://bugzilla.redhat.com/show_bug.cgi?id=1692075 Fixes: fe4e8db0392a6 ("r8169: fix issue with confused RX unit after PHY power-down on RTL8411b") Fixes: 91c8643578a21 ("r8169: use spinlock to protect mac ocp register access") Fixes: d6c36cbc5e533 ("r8169: Use a raw_spinlock_t for the register locks.") Cc: Heiner Kallweit Cc: Marco Elver Cc: nic_swsd@realtek.com Cc: "David S. Miller" Cc: Eric Dumazet Cc: Jakub Kicinski Cc: Paolo Abeni Cc: netdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org Link: https://lore.kernel.org/lkml/20231028005153.2180411-1-mirsad.todorovac@alu.unizg.hr/ Link: https://lore.kernel.org/lkml/20231028110459.2644926-1-mirsad.todorovac@alu.unizg.hr/ Signed-off-by: Mirsad Goran Todorovac Reviewed-by: Jacob Keller --- v4: fixed complaints as advised by Heiner and checkpatch.pl. split the patch into five sections to be more easily manipulated and reviewed introduced r8168_mac_ocp_write_seq() applied coalescing of mac ocp writes/modifies for 8168H, 8125 and 8125B v3: removed register/mask pair array sentinels, so using ARRAY_SIZE(). avoided duplication of RTL_W32() call code as advised by Heiner. drivers/net/ethernet/realtek/r8169_main.c | 57 +++++++++++++++++++++++ 1 file changed, 57 insertions(+) diff --git a/drivers/net/ethernet/realtek/r8169_main.c b/drivers/net/ethernet/realtek/r8169_main.c index 361b90007148..df79fd95cf2d 100644 --- a/drivers/net/ethernet/realtek/r8169_main.c +++ b/drivers/net/ethernet/realtek/r8169_main.c @@ -939,6 +939,63 @@ static void r8168_mac_ocp_modify(struct rtl8169_private *tp, u32 reg, u16 mask, raw_spin_unlock_irqrestore(&tp->mac_ocp_lock, flags); } +struct e_info_regdata { + u32 reg; + u32 data; +}; + +struct e_info_regmaskset { + u32 reg; + u16 mask; + u16 set; +}; + +static void __r8168_mac_ocp_write_seqlen(struct rtl8169_private *tp, + const struct e_info_regdata *array, int len) +{ + struct e_info_regdata const *p; + + for (p = array; len--; p++) + __r8168_mac_ocp_write(tp, p->reg, p->data); +} + +static void r8168_mac_ocp_write_seqlen(struct rtl8169_private *tp, + const struct e_info_regdata *array, int len) +{ + unsigned long flags; + + raw_spin_lock_irqsave(&tp->mac_ocp_lock, flags); + __r8168_mac_ocp_write_seqlen(tp, array, len); + raw_spin_unlock_irqrestore(&tp->mac_ocp_lock, flags); +} + +static void __r8168_mac_ocp_modify_seqlen(struct rtl8169_private *tp, + const struct e_info_regmaskset *array, int len) +{ + struct e_info_regmaskset const *p; + u16 data; + + for (p = array; len--; p++) { + data = __r8168_mac_ocp_read(tp, p->reg); + __r8168_mac_ocp_write(tp, p->reg, (data & ~p->mask) | p->set); + } +} + +static void r8168_mac_ocp_modify_seqlen(struct rtl8169_private *tp, + const struct e_info_regmaskset *array, int len) +{ + unsigned long flags; + + raw_spin_lock_irqsave(&tp->mac_ocp_lock, flags); + __r8168_mac_ocp_modify_seqlen(tp, array, len); + raw_spin_unlock_irqrestore(&tp->mac_ocp_lock, flags); +} + +#define r8168_mac_ocp_write_seq(tp, a) r8168_mac_ocp_write_seqlen(tp, a, ARRAY_SIZE(a)) +#define r8168_mac_ocp_modify_seq(tp, a) r8168_mac_ocp_modify_seqlen(tp, a, ARRAY_SIZE(a)) +#define __r8168_mac_ocp_write_seq(tp, a) __r8168_mac_ocp_write_seqlen(tp, a, ARRAY_SIZE(a)) +#define __r8168_mac_ocp_modify_seq(tp, a) __r8168_mac_ocp_modify_seqlen(tp, a, ARRAY_SIZE(a)) + /* Work around a hw issue with RTL8168g PHY, the quirk disables * PHY MCU interrupts before PHY power-down. */ From patchwork Sun Oct 29 11:04:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mirsad Todorovac X-Patchwork-Id: 159354 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:d641:0:b0:403:3b70:6f57 with SMTP id cy1csp1630568vqb; Sun, 29 Oct 2023 04:12:09 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFsd4pDOOczMLwQr08eZzvXBAaI3TejfIK236pWYMoe02qm0BHaye5vcOgOwD19ZaeHOojx X-Received: by 2002:a05:6e02:1689:b0:351:4cdd:f533 with SMTP id f9-20020a056e02168900b003514cddf533mr11689899ila.4.1698577929479; Sun, 29 Oct 2023 04:12:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1698577929; cv=none; d=google.com; s=arc-20160816; b=s8Bx+VF2gXY6stHBUY9kHNdtzgruAVDlQi3APrqkzHtUzAxnz8dGJ5MrGLNneTkdNX i5OYXQjYEg/ll0pNzayxmwhPtiyDlqn8Mikhgmz/PURuw53yq1tPiK7qg0D0H+VLyqIG FhbEQU05IZn4WA8u/hBRqiL+hF8ntR9mzY5yK9jD/XTwQEw+blrnS+vm0QI5s2srxo2v SBqYfYomDKlYp9sOlbJByscj2N7Nji6rExS8cPSW36ShLkNmJPsFA5qYApeFU/ukXdgM Q+vs+y60pBBrF54Oir9T6Ot+dyipIlywoIZ4yfpM42Ly7TiP7+Won3WfFN+OJ0rliEvg n7DQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:dkim-signature; bh=vnNmO95NXRnEamu904lHrluWd/VP83V5ajFU565qrwY=; fh=SsNJs69sMYlYTeWQYB3Y6uFVnk1BYEiSEalBcmc5mfg=; b=0YjNeC0zuqCFARz4UPlY0Ek6iTtDkm7koFYoK6hAsItnxKxB37hIC8mWRmARW5c7NY 5Bb8CS5m9xH/XzjF1zMFqVab3aDrLqbqq6n+wylWHAQsyjGZsbAStbkpP1Fy2DT4C5mW LvZgGRxXsGhIZU1xrdFuvv34jgZw933I7654FHEri/pa/TcwRhmOniWb9/pMyxdmBpjz vXrh8b0sZPpNaB1ryku0HImo3C9D/7GbsnQAbBm5sAYNfkpJhM6khkpLIuDSbGq4/xFJ EAWyDz8w/y5R6jIIv616JZurr+6eJrS72cOVYfeiSNh8me+86wsa4czwPGHzhBIR+JDf FYjA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@alu.unizg.hr header.s=mail header.b=RrL696G8; dkim=fail header.i=@alu.unizg.hr header.s=mail header.b=CAjxRKyk; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alu.unizg.hr Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id q26-20020a631f5a000000b005b95e508b6bsi2192459pgm.58.2023.10.29.04.12.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 29 Oct 2023 04:12:09 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; dkim=fail header.i=@alu.unizg.hr header.s=mail header.b=RrL696G8; dkim=fail header.i=@alu.unizg.hr header.s=mail header.b=CAjxRKyk; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alu.unizg.hr Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id A471A8057E78; Sun, 29 Oct 2023 04:12:08 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230088AbjJ2LMF (ORCPT + 30 others); Sun, 29 Oct 2023 07:12:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43258 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230081AbjJ2LMD (ORCPT ); Sun, 29 Oct 2023 07:12:03 -0400 Received: from domac.alu.hr (domac.alu.unizg.hr [161.53.235.3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 28F8BF2; Sun, 29 Oct 2023 04:11:59 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by domac.alu.hr (Postfix) with ESMTP id B53DA60182; Sun, 29 Oct 2023 12:11:57 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=alu.unizg.hr; s=mail; t=1698577917; bh=3AKP/yrpF4vztL9Cpk3o5TLARUpCEVltlUZVu/KmPuc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RrL696G8C5C32G3Q7d58zqe35yRWyY6/reAEr8H2QdVS+yPYxcdMIfYuwSU1oV4sB kUekWLVtIfFEOHG4s3Hy3P8loK6LqrT7LLFqzJFCKoVTJcKz+hqD4UUE6u9AFXrbRd x1L66nmeJco5pkP9tmNzTQT8D8rqhtup20sW63yksZbC2juPU8zKmXoRPr3uhcQ0hF c3USfwXONMFrPdfqmj90M5j0Bwi0hVmlYDompS5gmxLxirG10gDhgwd+9mIbH+E+qP pTdFQsYvp6vdhw/G64yKU3Ryna3yxylHcLz1PaB375Yjk1DF6M3r/bC7rqSSaRSMUd kCrvk2WXp23gA== X-Virus-Scanned: Debian amavisd-new at domac.alu.hr Received: from domac.alu.hr ([127.0.0.1]) by localhost (domac.alu.hr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id duMJ6FyO0l5K; Sun, 29 Oct 2023 12:11:54 +0100 (CET) Received: from defiant.home (78-3-40-247.adsl.net.t-com.hr [78.3.40.247]) by domac.alu.hr (Postfix) with ESMTPSA id 94A6C60173; Sun, 29 Oct 2023 12:11:54 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=alu.unizg.hr; s=mail; t=1698577914; bh=3AKP/yrpF4vztL9Cpk3o5TLARUpCEVltlUZVu/KmPuc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CAjxRKykB0+1XbGi57d+Z1hzx1PlUGEvvQca3UQJtvUkPXXbm6iesBUgwwAu7w9IG dWNyqiIPuLAHWTPfJfl21V3mvxhjmRHRs4JvkNa/h05I/033xYVhjaYoQxLKIXt7zz SowG8tfJQoXvfqMzme2WIxtKbQaSXTzozDMCQfWMUjPfPAAXNF7+f+QwzG1IkNN6sQ ifpuRR/OxRIDGFIRZc1xVsUWsO5XwjYuXxMMy8X49Ho1E/QHGm801M/Kjh9CAujT21 39MhNiBhdT7G8kdaJjxE6ba/orxmVTL4aI2oqRrJ6M/5Xt/5jv9cpPeUS73iCumV8r zG3gQLsXpuyBw== From: Mirsad Goran Todorovac To: Jason Gunthorpe , Joerg Roedel , Lu Baolu , iommu@lists.linux.dev, linux-kernel@vger.kernel.org, netdev@vger.kernel.org Cc: Joerg Roedel , Will Deacon , Robin Murphy , Heiner Kallweit , nic_swsd@realtek.com, "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Mirsad Goran Todorovac , Marco Elver Subject: [PATCH v4 2/5] r8169: Coalesce RTL8411b PHY power-down recovery calls to reduce spinlock stalls Date: Sun, 29 Oct 2023 12:04:41 +0100 Message-Id: <20231029110442.347448-2-mirsad.todorovac@alu.unizg.hr> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231029110442.347448-1-mirsad.todorovac@alu.unizg.hr> References: <20231029110442.347448-1-mirsad.todorovac@alu.unizg.hr> MIME-Version: 1.0 X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Sun, 29 Oct 2023 04:12:08 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1781088051173313386 X-GMAIL-MSGID: 1781088051173313386 On RTL8411b the RX unit gets confused if the PHY is powered-down. This was reported in [0] and confirmed by Realtek. Realtek provided a sequence to fix the RX unit after PHY wakeup. A series of about 130 r8168_mac_ocp_write() calls is performed to program the RTL registers for recovery. With about 130 of these sequential calls to r8168_mac_ocp_write() this looks like a lock storm that will stall all of the cores and CPUs on the same memory controller for certain time I/O takes to finish. In a sequential case of RTL register programming, a sequence of writes to the RTL registers can be coalesced under a same raw spinlock. This can dramatically decrease the number of bus stalls in a multicore or multi-CPU system: static void rtl_hw_start_8411_2(struct rtl8169_private *tp) { ... /* The following Realtek-provided magic fixes an issue with the RX unit * getting confused after the PHY having been powered-down. */ static const struct recover_8411b_info init_zero_seq[] = { { 0xFC28, 0x0000 }, { 0xFC2A, 0x0000 }, { 0xFC2C, 0x0000 }, ... }; static const struct recover_8411b_info recover_seq[] = { { 0xF800, 0xE008 }, { 0xF802, 0xE00A }, { 0xF804, 0xE00C }, ... }; static const struct recover_8411b_info final_seq[] = { { 0xFC2A, 0x0743 }, { 0xFC2C, 0x0801 }, { 0xFC2E, 0x0BE9 }, ... }; r8168_mac_ocp_write_seq(tp, init_zero_seq); mdelay(3); r8168_mac_ocp_write(tp, 0xFC26, 0x0000); r8168_mac_ocp_write_seq(tp, recover_seq); r8168_mac_ocp_write(tp, 0xFC26, 0x8000); r8168_mac_ocp_write_seq(tp, final_seq); } The hex data is preserved intact through s/r8168_mac_ocp_write[(]tp,/{ / and s/[)];/ },/ functions that only changed the function names and the ending of the line, so the actual hex data is unchanged. Note that the reason for the introduction of the original commit was to enable recovery of the RX unit on the RTL8411b which was confused by the powered-down PHY. This sequence of r8168_mac_ocp_write() calls amplifies the problem into a series of about 500+ memory bus locks, most waiting for the main MMIO memory read-modify-write under a LOCK. The memory barrier in RTL_W32 should suffice for the programming sequence to reach RTL NIC registers. [0] https://bugzilla.redhat.com/show_bug.cgi?id=1692075 Fixes: fe4e8db0392a6 ("r8169: fix issue with confused RX unit after PHY power-down on RTL8411b") Cc: Heiner Kallweit Cc: Marco Elver Cc: nic_swsd@realtek.com Cc: "David S. Miller" Cc: Eric Dumazet Cc: Jakub Kicinski Cc: Paolo Abeni Cc: netdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org Link: https://lore.kernel.org/lkml/20231028005153.2180411-1-mirsad.todorovac@alu.unizg.hr/ Link: https://lore.kernel.org/lkml/20231028110459.2644926-1-mirsad.todorovac@alu.unizg.hr/ Signed-off-by: Mirsad Goran Todorovac Reviewed-by: Jacob Keller --- v4: fixed complaints as advised by Heiner and checkpatch.pl. split the patch into five sections to be more easily manipulated and reviewed introduced r8168_mac_ocp_write_seq() applied coalescing of mac ocp writes/modifies for 8168H, 8125 and 8125B v3: removed register/mask pair array sentinels, so using ARRAY_SIZE(). avoided duplication of RTL_W32() call code as advised by Heiner. drivers/net/ethernet/realtek/r8169_main.c | 173 ++++++---------------- 1 file changed, 46 insertions(+), 127 deletions(-) diff --git a/drivers/net/ethernet/realtek/r8169_main.c b/drivers/net/ethernet/realtek/r8169_main.c index df79fd95cf2d..3dae924a6ca3 100644 --- a/drivers/net/ethernet/realtek/r8169_main.c +++ b/drivers/net/ethernet/realtek/r8169_main.c @@ -3157,145 +3157,64 @@ static void rtl_hw_start_8411_2(struct rtl8169_private *tp) { 0x1d, 0x0000, 0x4000 }, }; - rtl_hw_start_8168g(tp); + static const struct e_info_regdata init_zero_seq[] = { + { 0xFC28, 0x0000 }, { 0xFC2A, 0x0000 }, { 0xFC2C, 0x0000 }, { 0xFC2E, 0x0000 }, + { 0xFC30, 0x0000 }, { 0xFC32, 0x0000 }, { 0xFC34, 0x0000 }, { 0xFC36, 0x0000 }, + }; + static const struct e_info_regdata recover_seq[] = { + { 0xF800, 0xE008 }, { 0xF802, 0xE00A }, { 0xF804, 0xE00C }, { 0xF806, 0xE00E }, + { 0xF808, 0xE027 }, { 0xF80A, 0xE04F }, { 0xF80C, 0xE05E }, { 0xF80E, 0xE065 }, + { 0xF810, 0xC602 }, { 0xF812, 0xBE00 }, { 0xF814, 0x0000 }, { 0xF816, 0xC502 }, + { 0xF818, 0xBD00 }, { 0xF81A, 0x074C }, { 0xF81C, 0xC302 }, { 0xF81E, 0xBB00 }, + { 0xF820, 0x080A }, { 0xF822, 0x6420 }, { 0xF824, 0x48C2 }, { 0xF826, 0x8C20 }, + { 0xF828, 0xC516 }, { 0xF82A, 0x64A4 }, { 0xF82C, 0x49C0 }, { 0xF82E, 0xF009 }, + { 0xF830, 0x74A2 }, { 0xF832, 0x8CA5 }, { 0xF834, 0x74A0 }, { 0xF836, 0xC50E }, + { 0xF838, 0x9CA2 }, { 0xF83A, 0x1C11 }, { 0xF83C, 0x9CA0 }, { 0xF83E, 0xE006 }, + { 0xF840, 0x74F8 }, { 0xF842, 0x48C4 }, { 0xF844, 0x8CF8 }, { 0xF846, 0xC404 }, + { 0xF848, 0xBC00 }, { 0xF84A, 0xC403 }, { 0xF84C, 0xBC00 }, { 0xF84E, 0x0BF2 }, + { 0xF850, 0x0C0A }, { 0xF852, 0xE434 }, { 0xF854, 0xD3C0 }, { 0xF856, 0x49D9 }, + { 0xF858, 0xF01F }, { 0xF85A, 0xC526 }, { 0xF85C, 0x64A5 }, { 0xF85E, 0x1400 }, + { 0xF860, 0xF007 }, { 0xF862, 0x0C01 }, { 0xF864, 0x8CA5 }, { 0xF866, 0x1C15 }, + { 0xF868, 0xC51B }, { 0xF86A, 0x9CA0 }, { 0xF86C, 0xE013 }, { 0xF86E, 0xC519 }, + { 0xF870, 0x74A0 }, { 0xF872, 0x48C4 }, { 0xF874, 0x8CA0 }, { 0xF876, 0xC516 }, + { 0xF878, 0x74A4 }, { 0xF87A, 0x48C8 }, { 0xF87C, 0x48CA }, { 0xF87E, 0x9CA4 }, + { 0xF880, 0xC512 }, { 0xF882, 0x1B00 }, { 0xF884, 0x9BA0 }, { 0xF886, 0x1B1C }, + { 0xF888, 0x483F }, { 0xF88A, 0x9BA2 }, { 0xF88C, 0x1B04 }, { 0xF88E, 0xC508 }, + { 0xF890, 0x9BA0 }, { 0xF892, 0xC505 }, { 0xF894, 0xBD00 }, { 0xF896, 0xC502 }, + { 0xF898, 0xBD00 }, { 0xF89A, 0x0300 }, { 0xF89C, 0x051E }, { 0xF89E, 0xE434 }, + { 0xF8A0, 0xE018 }, { 0xF8A2, 0xE092 }, { 0xF8A4, 0xDE20 }, { 0xF8A6, 0xD3C0 }, + { 0xF8A8, 0xC50F }, { 0xF8AA, 0x76A4 }, { 0xF8AC, 0x49E3 }, { 0xF8AE, 0xF007 }, + { 0xF8B0, 0x49C0 }, { 0xF8B2, 0xF103 }, { 0xF8B4, 0xC607 }, { 0xF8B6, 0xBE00 }, + { 0xF8B8, 0xC606 }, { 0xF8BA, 0xBE00 }, { 0xF8BC, 0xC602 }, { 0xF8BE, 0xBE00 }, + { 0xF8C0, 0x0C4C }, { 0xF8C2, 0x0C28 }, { 0xF8C4, 0x0C2C }, { 0xF8C6, 0xDC00 }, + { 0xF8C8, 0xC707 }, { 0xF8CA, 0x1D00 }, { 0xF8CC, 0x8DE2 }, { 0xF8CE, 0x48C1 }, + { 0xF8D0, 0xC502 }, { 0xF8D2, 0xBD00 }, { 0xF8D4, 0x00AA }, { 0xF8D6, 0xE0C0 }, + { 0xF8D8, 0xC502 }, { 0xF8DA, 0xBD00 }, { 0xF8DC, 0x0132 }, + }; + + static const struct e_info_regdata final_seq[] = { + { 0xFC2A, 0x0743 }, { 0xFC2C, 0x0801 }, { 0xFC2E, 0x0BE9 }, { 0xFC30, 0x02FD }, + { 0xFC32, 0x0C25 }, { 0xFC34, 0x00A9 }, { 0xFC36, 0x012D }, + }; + + rtl_hw_start_8168g(tp); rtl_ephy_init(tp, e_info_8411_2); /* The following Realtek-provided magic fixes an issue with the RX unit * getting confused after the PHY having been powered-down. */ - r8168_mac_ocp_write(tp, 0xFC28, 0x0000); - r8168_mac_ocp_write(tp, 0xFC2A, 0x0000); - r8168_mac_ocp_write(tp, 0xFC2C, 0x0000); - r8168_mac_ocp_write(tp, 0xFC2E, 0x0000); - r8168_mac_ocp_write(tp, 0xFC30, 0x0000); - r8168_mac_ocp_write(tp, 0xFC32, 0x0000); - r8168_mac_ocp_write(tp, 0xFC34, 0x0000); - r8168_mac_ocp_write(tp, 0xFC36, 0x0000); + + r8168_mac_ocp_write_seq(tp, init_zero_seq); mdelay(3); r8168_mac_ocp_write(tp, 0xFC26, 0x0000); - r8168_mac_ocp_write(tp, 0xF800, 0xE008); - r8168_mac_ocp_write(tp, 0xF802, 0xE00A); - r8168_mac_ocp_write(tp, 0xF804, 0xE00C); - r8168_mac_ocp_write(tp, 0xF806, 0xE00E); - r8168_mac_ocp_write(tp, 0xF808, 0xE027); - r8168_mac_ocp_write(tp, 0xF80A, 0xE04F); - r8168_mac_ocp_write(tp, 0xF80C, 0xE05E); - r8168_mac_ocp_write(tp, 0xF80E, 0xE065); - r8168_mac_ocp_write(tp, 0xF810, 0xC602); - r8168_mac_ocp_write(tp, 0xF812, 0xBE00); - r8168_mac_ocp_write(tp, 0xF814, 0x0000); - r8168_mac_ocp_write(tp, 0xF816, 0xC502); - r8168_mac_ocp_write(tp, 0xF818, 0xBD00); - r8168_mac_ocp_write(tp, 0xF81A, 0x074C); - r8168_mac_ocp_write(tp, 0xF81C, 0xC302); - r8168_mac_ocp_write(tp, 0xF81E, 0xBB00); - r8168_mac_ocp_write(tp, 0xF820, 0x080A); - r8168_mac_ocp_write(tp, 0xF822, 0x6420); - r8168_mac_ocp_write(tp, 0xF824, 0x48C2); - r8168_mac_ocp_write(tp, 0xF826, 0x8C20); - r8168_mac_ocp_write(tp, 0xF828, 0xC516); - r8168_mac_ocp_write(tp, 0xF82A, 0x64A4); - r8168_mac_ocp_write(tp, 0xF82C, 0x49C0); - r8168_mac_ocp_write(tp, 0xF82E, 0xF009); - r8168_mac_ocp_write(tp, 0xF830, 0x74A2); - r8168_mac_ocp_write(tp, 0xF832, 0x8CA5); - r8168_mac_ocp_write(tp, 0xF834, 0x74A0); - r8168_mac_ocp_write(tp, 0xF836, 0xC50E); - r8168_mac_ocp_write(tp, 0xF838, 0x9CA2); - r8168_mac_ocp_write(tp, 0xF83A, 0x1C11); - r8168_mac_ocp_write(tp, 0xF83C, 0x9CA0); - r8168_mac_ocp_write(tp, 0xF83E, 0xE006); - r8168_mac_ocp_write(tp, 0xF840, 0x74F8); - r8168_mac_ocp_write(tp, 0xF842, 0x48C4); - r8168_mac_ocp_write(tp, 0xF844, 0x8CF8); - r8168_mac_ocp_write(tp, 0xF846, 0xC404); - r8168_mac_ocp_write(tp, 0xF848, 0xBC00); - r8168_mac_ocp_write(tp, 0xF84A, 0xC403); - r8168_mac_ocp_write(tp, 0xF84C, 0xBC00); - r8168_mac_ocp_write(tp, 0xF84E, 0x0BF2); - r8168_mac_ocp_write(tp, 0xF850, 0x0C0A); - r8168_mac_ocp_write(tp, 0xF852, 0xE434); - r8168_mac_ocp_write(tp, 0xF854, 0xD3C0); - r8168_mac_ocp_write(tp, 0xF856, 0x49D9); - r8168_mac_ocp_write(tp, 0xF858, 0xF01F); - r8168_mac_ocp_write(tp, 0xF85A, 0xC526); - r8168_mac_ocp_write(tp, 0xF85C, 0x64A5); - r8168_mac_ocp_write(tp, 0xF85E, 0x1400); - r8168_mac_ocp_write(tp, 0xF860, 0xF007); - r8168_mac_ocp_write(tp, 0xF862, 0x0C01); - r8168_mac_ocp_write(tp, 0xF864, 0x8CA5); - r8168_mac_ocp_write(tp, 0xF866, 0x1C15); - r8168_mac_ocp_write(tp, 0xF868, 0xC51B); - r8168_mac_ocp_write(tp, 0xF86A, 0x9CA0); - r8168_mac_ocp_write(tp, 0xF86C, 0xE013); - r8168_mac_ocp_write(tp, 0xF86E, 0xC519); - r8168_mac_ocp_write(tp, 0xF870, 0x74A0); - r8168_mac_ocp_write(tp, 0xF872, 0x48C4); - r8168_mac_ocp_write(tp, 0xF874, 0x8CA0); - r8168_mac_ocp_write(tp, 0xF876, 0xC516); - r8168_mac_ocp_write(tp, 0xF878, 0x74A4); - r8168_mac_ocp_write(tp, 0xF87A, 0x48C8); - r8168_mac_ocp_write(tp, 0xF87C, 0x48CA); - r8168_mac_ocp_write(tp, 0xF87E, 0x9CA4); - r8168_mac_ocp_write(tp, 0xF880, 0xC512); - r8168_mac_ocp_write(tp, 0xF882, 0x1B00); - r8168_mac_ocp_write(tp, 0xF884, 0x9BA0); - r8168_mac_ocp_write(tp, 0xF886, 0x1B1C); - r8168_mac_ocp_write(tp, 0xF888, 0x483F); - r8168_mac_ocp_write(tp, 0xF88A, 0x9BA2); - r8168_mac_ocp_write(tp, 0xF88C, 0x1B04); - r8168_mac_ocp_write(tp, 0xF88E, 0xC508); - r8168_mac_ocp_write(tp, 0xF890, 0x9BA0); - r8168_mac_ocp_write(tp, 0xF892, 0xC505); - r8168_mac_ocp_write(tp, 0xF894, 0xBD00); - r8168_mac_ocp_write(tp, 0xF896, 0xC502); - r8168_mac_ocp_write(tp, 0xF898, 0xBD00); - r8168_mac_ocp_write(tp, 0xF89A, 0x0300); - r8168_mac_ocp_write(tp, 0xF89C, 0x051E); - r8168_mac_ocp_write(tp, 0xF89E, 0xE434); - r8168_mac_ocp_write(tp, 0xF8A0, 0xE018); - r8168_mac_ocp_write(tp, 0xF8A2, 0xE092); - r8168_mac_ocp_write(tp, 0xF8A4, 0xDE20); - r8168_mac_ocp_write(tp, 0xF8A6, 0xD3C0); - r8168_mac_ocp_write(tp, 0xF8A8, 0xC50F); - r8168_mac_ocp_write(tp, 0xF8AA, 0x76A4); - r8168_mac_ocp_write(tp, 0xF8AC, 0x49E3); - r8168_mac_ocp_write(tp, 0xF8AE, 0xF007); - r8168_mac_ocp_write(tp, 0xF8B0, 0x49C0); - r8168_mac_ocp_write(tp, 0xF8B2, 0xF103); - r8168_mac_ocp_write(tp, 0xF8B4, 0xC607); - r8168_mac_ocp_write(tp, 0xF8B6, 0xBE00); - r8168_mac_ocp_write(tp, 0xF8B8, 0xC606); - r8168_mac_ocp_write(tp, 0xF8BA, 0xBE00); - r8168_mac_ocp_write(tp, 0xF8BC, 0xC602); - r8168_mac_ocp_write(tp, 0xF8BE, 0xBE00); - r8168_mac_ocp_write(tp, 0xF8C0, 0x0C4C); - r8168_mac_ocp_write(tp, 0xF8C2, 0x0C28); - r8168_mac_ocp_write(tp, 0xF8C4, 0x0C2C); - r8168_mac_ocp_write(tp, 0xF8C6, 0xDC00); - r8168_mac_ocp_write(tp, 0xF8C8, 0xC707); - r8168_mac_ocp_write(tp, 0xF8CA, 0x1D00); - r8168_mac_ocp_write(tp, 0xF8CC, 0x8DE2); - r8168_mac_ocp_write(tp, 0xF8CE, 0x48C1); - r8168_mac_ocp_write(tp, 0xF8D0, 0xC502); - r8168_mac_ocp_write(tp, 0xF8D2, 0xBD00); - r8168_mac_ocp_write(tp, 0xF8D4, 0x00AA); - r8168_mac_ocp_write(tp, 0xF8D6, 0xE0C0); - r8168_mac_ocp_write(tp, 0xF8D8, 0xC502); - r8168_mac_ocp_write(tp, 0xF8DA, 0xBD00); - r8168_mac_ocp_write(tp, 0xF8DC, 0x0132); + r8168_mac_ocp_write_seq(tp, recover_seq); r8168_mac_ocp_write(tp, 0xFC26, 0x8000); - r8168_mac_ocp_write(tp, 0xFC2A, 0x0743); - r8168_mac_ocp_write(tp, 0xFC2C, 0x0801); - r8168_mac_ocp_write(tp, 0xFC2E, 0x0BE9); - r8168_mac_ocp_write(tp, 0xFC30, 0x02FD); - r8168_mac_ocp_write(tp, 0xFC32, 0x0C25); - r8168_mac_ocp_write(tp, 0xFC34, 0x00A9); - r8168_mac_ocp_write(tp, 0xFC36, 0x012D); + r8168_mac_ocp_write_seq(tp, final_seq); + } static void rtl_hw_start_8168h_1(struct rtl8169_private *tp) From patchwork Sun Oct 29 11:04:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mirsad Todorovac X-Patchwork-Id: 159355 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:d641:0:b0:403:3b70:6f57 with SMTP id cy1csp1630764vqb; Sun, 29 Oct 2023 04:12:36 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHnbJotXgCtesepJyoSOxBfgBPhNiaTwz4apv2cNnuTNUfvfKE+3UymkL0CnjhXQW+pPEqC X-Received: by 2002:a05:6a20:7349:b0:155:1710:664a with SMTP id v9-20020a056a20734900b001551710664amr8463371pzc.18.1698577956005; Sun, 29 Oct 2023 04:12:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1698577955; cv=none; d=google.com; s=arc-20160816; b=ghkj6/OvW6RZZul9bw4sYHdTnW6lstlMA0Sw/SKOE7llvwfdwprH+3KhAod2DOKWm8 vIeDiBgvdMpvhYsuz4TqdcpWXAU8GwmcbLjwiaPtNmaOKRoOaQUupQIp2Rv+SG2YFpxk nyiC7QURgWH++Ou4kDN/CzIe6z7qyk0voVXt6jP+Vd2jeFf76ltTJlbv1gqqy1vWvYD5 0AYhonOg7/8WjIPeQXzSBpZPiCJQwQLDMQzaw8PSkJ622zMzxHY8fJu53qRWfQYsR4Xr 6gRr0FQtKR37N+4QsO0+rWbVZapz/KPSwfbKk/NrIEinrmnuQqPRjyBft67R53fDsk0t LUiQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:dkim-signature; bh=HDY9iTysKaGAIJ+7JX1q3OY9xBeuDFmfGpGiX2YeDrI=; fh=SsNJs69sMYlYTeWQYB3Y6uFVnk1BYEiSEalBcmc5mfg=; b=mHsZetcLRzkbTPrypFXMWljPvflXFhtDRXm1Q76s65MnOqkhlDdozzIlDWAdHhc8fY Bg9nx7tqPhzwx8i9hVCbilnrSSQapDzZUEh6UaMmFHQJusfiz10e79OXbsNB1emgHfaK J+GNb5y7xA/v6vnd6Y6nKsFVd3OJmQl6ggsWBLrnS1SRA3QpUBkpqDzzgXy0E84PEKo/ nPhmjIzSwfamuS8eWA1aSfxA8M28XqdAXgLT+MTLu9ovLi5leG+KdofPXjK8YkAyWTXq YNzXjk99k45rdMLj6Y5LBSAEnV4yX+R7MPi1wpCbBM52++ekWpBXpGU+kC70AFol0Y6v aJkA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@alu.unizg.hr header.s=mail header.b="zFyX/nmh"; dkim=fail header.i=@alu.unizg.hr header.s=mail header.b=erTsa0mg; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alu.unizg.hr Received: from pete.vger.email (pete.vger.email. [23.128.96.36]) by mx.google.com with ESMTPS id b9-20020a056a000a8900b006c0fefd59eesi585536pfl.279.2023.10.29.04.12.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 29 Oct 2023 04:12:35 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) client-ip=23.128.96.36; Authentication-Results: mx.google.com; dkim=fail header.i=@alu.unizg.hr header.s=mail header.b="zFyX/nmh"; dkim=fail header.i=@alu.unizg.hr header.s=mail header.b=erTsa0mg; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alu.unizg.hr Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by pete.vger.email (Postfix) with ESMTP id 554FB806E5C8; Sun, 29 Oct 2023 04:12:33 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at pete.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230113AbjJ2LMX (ORCPT + 30 others); Sun, 29 Oct 2023 07:12:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39068 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229529AbjJ2LMW (ORCPT ); Sun, 29 Oct 2023 07:12:22 -0400 Received: from domac.alu.hr (domac.alu.unizg.hr [IPv6:2001:b68:2:2800::3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 62E31C6; Sun, 29 Oct 2023 04:12:19 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by domac.alu.hr (Postfix) with ESMTP id 178E060182; Sun, 29 Oct 2023 12:12:17 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=alu.unizg.hr; s=mail; t=1698577937; bh=Xx9ViACHZVW255zW2vq4zwUoDX0YuwgJy1XCvVN1gGU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=zFyX/nmhWu/BT6Q2NDpt/TVMG95IYlDRKls1iWo3iDkkJEHThDII5DDicb7Z3/Ce8 LyQzAGTxmcqoneW4J2+TQ7k7tiJae0TgmJe7/GvoRz4fin4JlSGGFErTbKdGBuZyhO zxRMP2n589mJPYQ1fmmuHW1B+lK5sjcqOjtzqVnYkkSn3JCwvCzYtac3vaw/gY9+B9 ZE8yt/yvj2sP7FPFpBG3j6ZlRcjlRcZPf62LxzubCbike0Lmv8sfy1k1OLMEkPbthe RkvQ+mAMZDhHfdgMXNdnsuLkotHosOxEEJW0TWAAlLnnQi5TRPJipJcIQciJw53r3r /JqjsAx5r1/Kg== X-Virus-Scanned: Debian amavisd-new at domac.alu.hr Received: from domac.alu.hr ([127.0.0.1]) by localhost (domac.alu.hr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2iQHhhvWbuPE; Sun, 29 Oct 2023 12:12:14 +0100 (CET) Received: from defiant.home (78-3-40-247.adsl.net.t-com.hr [78.3.40.247]) by domac.alu.hr (Postfix) with ESMTPSA id 91E9D60173; Sun, 29 Oct 2023 12:12:14 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=alu.unizg.hr; s=mail; t=1698577934; bh=Xx9ViACHZVW255zW2vq4zwUoDX0YuwgJy1XCvVN1gGU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=erTsa0mgD/HLpAcnUdMeqRZzwMc4NLw7pbv73Ubk8vgPlUAzwcSGu6C/zfa00WJms qS/o9Plhg4qL11m7oFxIBdUFOHU8cPLngPkiP4AhQXtHLU2zZ+V1HC5ei1HcAvIv18 /Gc7ctU5mjp6gNBoxVXhyGVgqyYfEoVL2YMqfFcjj4dqSAd0UERxQOXEPNI3f4Bmmn bXLtYuC+Sv2sEuBPd4SZx6w+vohes0B4u7VnDaqZjWSprcwDylgJHsT65ZFGUTbw1a XtXcxcktb5hd2H/jD5hfV6n0XR1cwsZ6AKnuEM+l+2zU9ySTFju7GYBsowE/UPj0PE sRcKclb3fjRYg== From: Mirsad Goran Todorovac To: Jason Gunthorpe , Joerg Roedel , Lu Baolu , iommu@lists.linux.dev, linux-kernel@vger.kernel.org, netdev@vger.kernel.org Cc: Joerg Roedel , Will Deacon , Robin Murphy , Heiner Kallweit , nic_swsd@realtek.com, "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Mirsad Goran Todorovac , Marco Elver Subject: [PATCH v4 3/5] r8169: Coalesce mac ocp write and modify for 8168H start to reduce spinlocks Date: Sun, 29 Oct 2023 12:04:43 +0100 Message-Id: <20231029110442.347448-3-mirsad.todorovac@alu.unizg.hr> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231029110442.347448-1-mirsad.todorovac@alu.unizg.hr> References: <20231029110442.347448-1-mirsad.todorovac@alu.unizg.hr> MIME-Version: 1.0 X-Spam-Status: No, score=-0.6 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Sun, 29 Oct 2023 04:12:33 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1781088078935938241 X-GMAIL-MSGID: 1781088078935938241 Repeated calls to r8168_mac_ocp_write() and r8168_mac_ocp_modify() in the startup of 8168H involve implicit spin_lock_irqsave() and spin_unlock_irqrestore() on each invocation. Coalesced with the corresponding helpers, r8168_mac_ocp_write_seq() and r8168_mac_ocp_modify_seq() with a sinqle lock/unlock, these calls reduce overall lock contention. Fixes: ef712ede3541d ("r8169: add helper r8168_mac_ocp_modify") Fixes: 6e1d0b8988188 ("r8169:add support for RTL8168H and RTL8107E") Cc: Heiner Kallweit Cc: Marco Elver Cc: nic_swsd@realtek.com Cc: "David S. Miller" Cc: Eric Dumazet Cc: Jakub Kicinski Cc: Paolo Abeni Cc: netdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org Link: https://lore.kernel.org/lkml/20231028005153.2180411-1-mirsad.todorovac@alu.unizg.hr/ Link: https://lore.kernel.org/lkml/20231028110459.2644926-1-mirsad.todorovac@alu.unizg.hr/ Signed-off-by: Mirsad Goran Todorovac Reviewed-by: Jacob Keller --- v4: fixed complaints as advised by Heiner and checkpatch.pl. split the patch into five sections to be more easily manipulated and reviewed introduced r8168_mac_ocp_write_seq() applied coalescing of mac ocp writes/modifies for 8168H, 8125 and 8125B v3: removed register/mask pair array sentinels, so using ARRAY_SIZE(). avoided duplication of RTL_W32() call code as advised by Heiner. drivers/net/ethernet/realtek/r8169_main.c | 26 +++++++++++++++-------- 1 file changed, 17 insertions(+), 9 deletions(-) diff --git a/drivers/net/ethernet/realtek/r8169_main.c b/drivers/net/ethernet/realtek/r8169_main.c index 3dae924a6ca3..cbaac4675bd2 100644 --- a/drivers/net/ethernet/realtek/r8169_main.c +++ b/drivers/net/ethernet/realtek/r8169_main.c @@ -3227,6 +3227,21 @@ static void rtl_hw_start_8168h_1(struct rtl8169_private *tp) { 0x04, 0xffff, 0x854a }, { 0x01, 0xffff, 0x068b } }; + + static const struct e_info_regmaskset e_info_regmaskset_8168h_1[] = { + { 0xe056, 0x00f0, 0x0070 }, + { 0xe052, 0x6000, 0x8008 }, + { 0xe0d6, 0x01ff, 0x017f }, + { 0xd420, 0x0fff, 0x047f }, + }; + + static const struct e_info_regdata e_info_regdata_8168h_1[] = { + { 0xe63e, 0x0001 }, + { 0xe63e, 0x0000 }, + { 0xc094, 0x0000 }, + { 0xc09e, 0x0000 }, + }; + int rg_saw_cnt; rtl_ephy_init(tp, e_info_8168h_1); @@ -3267,15 +3282,8 @@ static void rtl_hw_start_8168h_1(struct rtl8169_private *tp) r8168_mac_ocp_modify(tp, 0xd412, 0x0fff, sw_cnt_1ms_ini); } - r8168_mac_ocp_modify(tp, 0xe056, 0x00f0, 0x0070); - r8168_mac_ocp_modify(tp, 0xe052, 0x6000, 0x8008); - r8168_mac_ocp_modify(tp, 0xe0d6, 0x01ff, 0x017f); - r8168_mac_ocp_modify(tp, 0xd420, 0x0fff, 0x047f); - - r8168_mac_ocp_write(tp, 0xe63e, 0x0001); - r8168_mac_ocp_write(tp, 0xe63e, 0x0000); - r8168_mac_ocp_write(tp, 0xc094, 0x0000); - r8168_mac_ocp_write(tp, 0xc09e, 0x0000); + r8168_mac_ocp_modify_seq(tp, e_info_regmaskset_8168h_1); + r8168_mac_ocp_write_seq(tp, e_info_regdata_8168h_1); } static void rtl_hw_start_8168ep(struct rtl8169_private *tp) From patchwork Sun Oct 29 11:04:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mirsad Todorovac X-Patchwork-Id: 159356 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:d641:0:b0:403:3b70:6f57 with SMTP id cy1csp1631313vqb; Sun, 29 Oct 2023 04:13:58 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEA7rk1qn7bp5xly8WrkONBB39wo/rOQKtjc8NGdL/TSysVzzNDWb2Ly5+nI4+cTo6/F6xL X-Received: by 2002:a05:6602:2b86:b0:7a9:96aa:e01e with SMTP id r6-20020a0566022b8600b007a996aae01emr10715549iov.17.1698578038412; Sun, 29 Oct 2023 04:13:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1698578038; cv=none; d=google.com; s=arc-20160816; b=daA+HPICEm7HmNWN6DDAfPCoOH8wDef4LhMSYjxP8ughiqwyLvLGjOYUrXTWSW1DXY hgM2Zy8RvjBkMnQSGp9DefRWAVi5IETa+pQlXeK6dgxgYfj7Cu9D5OhwdJGOi8GC6kH4 Yho/I9PKf5QKMRILlZ7j1jx4f4sdL8mbVxnoDkJnCLqjF3YX36Eb7UzHtVIsBZksfbLe zP4ZhhCqLesYzmtuWsnqYsa+Q1WWRteWVgcORQqzxRwxnxU4zhQzEXIxIdBTkXrGFFzC jX+SxnGxpL4ZCwouVMSC2OjdhWtKmwJZL+jL+tzQcuQQQhbozLiUvKlJX6BqGM+MvjDV 3csQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:dkim-signature; bh=A9/kNI9b7RSIN3devPK7PHNaGj0voCSrUyIEM5SF+PU=; fh=SsNJs69sMYlYTeWQYB3Y6uFVnk1BYEiSEalBcmc5mfg=; b=PCKz7S7FQbXTdi9w3V9+VLpulPJjRqssLA6SJP0egYrAU7JMoFbz6EqnKZvMyEP9+4 44xIVjwCZey7HcVJdGDOhfqdWOdKRXCy6s9z28Q8tdljHksFTgWnXlWw6GrYG63Ak1A5 vAqKzYrTU1EgAtsmrK1/tHFrgYqtvx/cfoEF578yblR7fg8J82kH69xgCTbI74BNnrum bzlzD5fiBDRtTbxPsdGgwIk5lGwJWRjMjhjmBDrPXT4k3IOVkrpJTHGJ1nFqCzTTtAgR dkbqz/PuVtVYOtaHmvHp6lsv/FdhHqTmwFMd4Rftn4HH16TfmxOkftF4W/POKlVG2r9O W/+w== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@alu.unizg.hr header.s=mail header.b=SAJiupU5; dkim=fail header.i=@alu.unizg.hr header.s=mail header.b=RfercLSu; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alu.unizg.hr Received: from howler.vger.email (howler.vger.email. [23.128.96.34]) by mx.google.com with ESMTPS id k71-20020a62844a000000b006933caa6f03si3557952pfd.165.2023.10.29.04.13.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 29 Oct 2023 04:13:58 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) client-ip=23.128.96.34; Authentication-Results: mx.google.com; dkim=fail header.i=@alu.unizg.hr header.s=mail header.b=SAJiupU5; dkim=fail header.i=@alu.unizg.hr header.s=mail header.b=RfercLSu; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alu.unizg.hr Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id 4852D8060CA5; Sun, 29 Oct 2023 04:13:55 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229790AbjJ2LNi (ORCPT + 30 others); Sun, 29 Oct 2023 07:13:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58060 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229529AbjJ2LNh (ORCPT ); Sun, 29 Oct 2023 07:13:37 -0400 Received: from domac.alu.hr (domac.alu.unizg.hr [161.53.235.3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2A6BDC0; Sun, 29 Oct 2023 04:13:35 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by domac.alu.hr (Postfix) with ESMTP id DD1AE60182; Sun, 29 Oct 2023 12:13:33 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=alu.unizg.hr; s=mail; t=1698578013; bh=NtW5AA3iztqWChPJltkY4/MG+Vbb89cp9gCAty5mC2k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=SAJiupU5Lqr+a+cpvtLP1Ehji5uY8ahX5H0FXi86cRA8Z1urPmKYJbzGEuB/r9lAe Zq746u6NYHBfc2UlfY0QF73zJQncLb6Z8LZYJqpFj9ofD9sCJtTyI8MKrMamlbuKia SDEJFMk2moW9ksTsC2OQYRryloS5DZkqAYuM4UEcPCBE5/P3pZFRBiBrrZ7aD7g7bM yoSs5aS3f8eV/E4EPAAt0PjIjyNuAtOnG4S5big1qUHCgAmtrgUpiO2SkYz2Psz3AH pl5sEEfmIOvds17Q8LSE9UndM7leaKmcYDnmjLBZLqRniiGyUqO40tk3CMcVeNrmO6 VEOUnpa537J1Q== X-Virus-Scanned: Debian amavisd-new at domac.alu.hr Received: from domac.alu.hr ([127.0.0.1]) by localhost (domac.alu.hr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MUAMGmq2omC4; Sun, 29 Oct 2023 12:13:31 +0100 (CET) Received: from defiant.home (78-3-40-247.adsl.net.t-com.hr [78.3.40.247]) by domac.alu.hr (Postfix) with ESMTPSA id 3799A60173; Sun, 29 Oct 2023 12:13:31 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=alu.unizg.hr; s=mail; t=1698578011; bh=NtW5AA3iztqWChPJltkY4/MG+Vbb89cp9gCAty5mC2k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RfercLSu6F5ap+xDowIYoRw/jAzK6gD2ex1MxlxO/NKq6NYR+lj8XgELdp/SPEUzT V7Fdchlm6Ee3xKpUqwh74InU6qzjdRdWIGMUIuUCTXGMy9suAf2u3BYOuSUyOft0PG u/q+tubop6TOZv7qROvXDrAHH5rBenUKvIjg7TzAzGTCSy7CN/2/3+WWsTPK+cQ/te 39XKmlDmAC6koOLY4HZMQy93O46x/4xySgs+0EOAvXwTCW0kmMizOtPDGdjDGWHCSy iy0PCXKZ1bLlKad5Nv7cJcu1Y5WJaVTpg3QbxnodUht9ScvoBwlmLzjQTodhN/CaKE qys2jFnHWuFJg== From: Mirsad Goran Todorovac To: Jason Gunthorpe , Joerg Roedel , Lu Baolu , iommu@lists.linux.dev, linux-kernel@vger.kernel.org, netdev@vger.kernel.org Cc: Joerg Roedel , Will Deacon , Robin Murphy , Heiner Kallweit , nic_swsd@realtek.com, "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Mirsad Goran Todorovac , Marco Elver Subject: [PATCH v4 4/5] r8169: Coalesce mac ocp commands for 8125 and 8125B start to reduce spinlock contention Date: Sun, 29 Oct 2023 12:04:45 +0100 Message-Id: <20231029110442.347448-4-mirsad.todorovac@alu.unizg.hr> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231029110442.347448-1-mirsad.todorovac@alu.unizg.hr> References: <20231029110442.347448-1-mirsad.todorovac@alu.unizg.hr> MIME-Version: 1.0 X-Spam-Status: No, score=-0.6 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on howler.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Sun, 29 Oct 2023 04:13:55 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1781088165374460774 X-GMAIL-MSGID: 1781088165374460774 Repeated calls to r8168_mac_ocp_write() and r8168_mac_ocp_modify() in the startup of 8125 and 8125B involve implicit spin_lock_irqsave() and spin_unlock_irqrestore() on each invocation. Coalesced with the corresponding helpers r8168_mac_ocp_write_seq() and r8168_mac_ocp_modify_seq() into sequential write or modidy with a sinqle lock/unlock, these calls reduce overall lock contention. Fixes: f1bce4ad2f1ce ("r8169: add support for RTL8125") Fixes: 0439297be9511 ("r8169: add support for RTL8125B") Cc: Heiner Kallweit Cc: Marco Elver Cc: nic_swsd@realtek.com Cc: "David S. Miller" Cc: Eric Dumazet Cc: Jakub Kicinski Cc: Paolo Abeni Cc: netdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org Link: https://lore.kernel.org/lkml/20231028005153.2180411-1-mirsad.todorovac@alu.unizg.hr/ Link: https://lore.kernel.org/lkml/20231028110459.2644926-1-mirsad.todorovac@alu.unizg.hr/ Signed-off-by: Mirsad Goran Todorovac Reviewed-by: Jacob Keller --- v4: fixed complaints as advised by Heiner and checkpatch.pl. split the patch into five sections to be more easily manipulated and reviewed introduced r8168_mac_ocp_write_seq() applied coalescing of mac ocp writes/modifies for 8168H, 8125 and 8125B v3: removed register/mask pair array sentinels, so using ARRAY_SIZE(). avoided duplication of RTL_W32() call code as advised by Heiner. drivers/net/ethernet/realtek/r8169_main.c | 38 ++++++++++++++--------- 1 file changed, 23 insertions(+), 15 deletions(-) diff --git a/drivers/net/ethernet/realtek/r8169_main.c b/drivers/net/ethernet/realtek/r8169_main.c index cbaac4675bd2..dd65e0384ab3 100644 --- a/drivers/net/ethernet/realtek/r8169_main.c +++ b/drivers/net/ethernet/realtek/r8169_main.c @@ -3539,6 +3539,27 @@ DECLARE_RTL_COND(rtl_mac_ocp_e00e_cond) static void rtl_hw_start_8125_common(struct rtl8169_private *tp) { + + static const struct e_info_regmaskset e_info_8125_common_1[] = { + { 0xd3e2, 0x0fff, 0x03a9 }, + { 0xd3e4, 0x00ff, 0x0000 }, + { 0xe860, 0x0000, 0x0080 }, + }; + + static const struct e_info_regmaskset e_info_8125_common_2[] = { + { 0xc0b4, 0x0000, 0x000c }, + { 0xeb6a, 0x00ff, 0x0033 }, + { 0xeb50, 0x03e0, 0x0040 }, + { 0xe056, 0x00f0, 0x0030 }, + { 0xe040, 0x1000, 0x0000 }, + { 0xea1c, 0x0003, 0x0001 }, + { 0xe0c0, 0x4f0f, 0x4403 }, + { 0xe052, 0x0080, 0x0068 }, + { 0xd430, 0x0fff, 0x047f }, + { 0xea1c, 0x0004, 0x0000 }, + { 0xeb54, 0x0000, 0x0001 }, + }; + rtl_pcie_state_l2l3_disable(tp); RTL_W16(tp, 0x382, 0x221b); @@ -3553,9 +3574,7 @@ static void rtl_hw_start_8125_common(struct rtl8169_private *tp) r8168_mac_ocp_write(tp, 0xc140, 0xffff); r8168_mac_ocp_write(tp, 0xc142, 0xffff); - r8168_mac_ocp_modify(tp, 0xd3e2, 0x0fff, 0x03a9); - r8168_mac_ocp_modify(tp, 0xd3e4, 0x00ff, 0x0000); - r8168_mac_ocp_modify(tp, 0xe860, 0x0000, 0x0080); + r8168_mac_ocp_modify_seq(tp, e_info_8125_common_1); /* disable new tx descriptor format */ r8168_mac_ocp_modify(tp, 0xeb58, 0x0001, 0x0000); @@ -3570,18 +3589,7 @@ static void rtl_hw_start_8125_common(struct rtl8169_private *tp) else r8168_mac_ocp_modify(tp, 0xe63e, 0x0c30, 0x0020); - r8168_mac_ocp_modify(tp, 0xc0b4, 0x0000, 0x000c); - r8168_mac_ocp_modify(tp, 0xeb6a, 0x00ff, 0x0033); - r8168_mac_ocp_modify(tp, 0xeb50, 0x03e0, 0x0040); - r8168_mac_ocp_modify(tp, 0xe056, 0x00f0, 0x0030); - r8168_mac_ocp_modify(tp, 0xe040, 0x1000, 0x0000); - r8168_mac_ocp_modify(tp, 0xea1c, 0x0003, 0x0001); - r8168_mac_ocp_modify(tp, 0xe0c0, 0x4f0f, 0x4403); - r8168_mac_ocp_modify(tp, 0xe052, 0x0080, 0x0068); - r8168_mac_ocp_modify(tp, 0xd430, 0x0fff, 0x047f); - - r8168_mac_ocp_modify(tp, 0xea1c, 0x0004, 0x0000); - r8168_mac_ocp_modify(tp, 0xeb54, 0x0000, 0x0001); + r8168_mac_ocp_modify_seq(tp, e_info_8125_common_2); udelay(1); r8168_mac_ocp_modify(tp, 0xeb54, 0x0001, 0x0000); RTL_W16(tp, 0x1880, RTL_R16(tp, 0x1880) & ~0x0030); From patchwork Sun Oct 29 11:04:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mirsad Todorovac X-Patchwork-Id: 159357 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:d641:0:b0:403:3b70:6f57 with SMTP id cy1csp1631560vqb; Sun, 29 Oct 2023 04:14:39 -0700 (PDT) X-Google-Smtp-Source: AGHT+IF2e5/SaokvL3MXhB15IJAezj6w6z6EZnkATH1SjR9NBRqIAg6JazmGrY+a+ukEkfWOnWB4 X-Received: by 2002:a05:6a20:6a0d:b0:160:a752:59e with SMTP id p13-20020a056a206a0d00b00160a752059emr9036117pzk.40.1698578079224; Sun, 29 Oct 2023 04:14:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1698578079; cv=none; d=google.com; s=arc-20160816; b=h5FOor1d01xbAHR3q6Vu9i/GP6Q8uwzM0S+jcIiKiuUAViSA8xFRbNMqjIPNfzMshs 5N1CIC2T6OdnhjveBoVwRPKCCzDZGeRGSqdsJ2sUkGnVAAkMh3kyfXOVi2cfwzgYoC2j BjizJaLG8naTJ9ELyQqDnOmFNAo4/ZiC/NqYCKZNjQo9ZsTEL5kdZwi+ZoIgmkCPgNLV aMJ9Il4AdoMpfWNGJmur7xPBo/+O/TBiddi7Z0Ah2qy4fgPimkENoLYZoblic3YhngMj M8g0GMjUsRNiXzXUWmNELrxkEYF5AWnJOfo79uivaZmX2U04x6clHv/N45EWhZ63JQLY hLzw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:dkim-signature; bh=K6ZWdoNmwzeouGDrVVRD/RQmSEExKykrRSbeIVMOTyU=; fh=SsNJs69sMYlYTeWQYB3Y6uFVnk1BYEiSEalBcmc5mfg=; b=wq3EG/HPzGaZUT3WDdOIJ1BSmrZIyczvjA+rBJje4795BAE+fNcuz+S7CD0KwTl4pE ZOKzhciGEV4Cke8XXha9jVjamdJYeJPEXCC8LffY3et+2hLNoKcWseDy6vOcNu3wTEIP 4HaCLjJPJuV3qqo7V+2g3BBTRB/ATQTIwMRM3afVNdMNvljvxF8fADknyfXFtwNU1O0k p+8MSnZBBHAre7qFVHTFrAksLWZ2kNMa6RI5/wvTXu3yjc5Ck2GF0T7bW0mgPdKeN3kE y0LUox2J8IKFYyC4uTXUshpLXXQFBoxyZBMPrPoIgBgvayDShrMx5yvkT2V4p+q+66sS fCEQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@alu.unizg.hr header.s=mail header.b=uKov8Epv; dkim=fail header.i=@alu.unizg.hr header.s=mail header.b=c0S7U11d; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alu.unizg.hr Received: from agentk.vger.email (agentk.vger.email. [2620:137:e000::3:2]) by mx.google.com with ESMTPS id q20-20020a656854000000b005b7e3ee1820si3516590pgt.157.2023.10.29.04.14.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 29 Oct 2023 04:14:39 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) client-ip=2620:137:e000::3:2; Authentication-Results: mx.google.com; dkim=fail header.i=@alu.unizg.hr header.s=mail header.b=uKov8Epv; dkim=fail header.i=@alu.unizg.hr header.s=mail header.b=c0S7U11d; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alu.unizg.hr Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id 9ED33809D336; Sun, 29 Oct 2023 04:14:36 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230050AbjJ2LOU (ORCPT + 30 others); Sun, 29 Oct 2023 07:14:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43728 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229529AbjJ2LOS (ORCPT ); Sun, 29 Oct 2023 07:14:18 -0400 Received: from domac.alu.hr (domac.alu.unizg.hr [IPv6:2001:b68:2:2800::3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6BAA6AB; Sun, 29 Oct 2023 04:14:16 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by domac.alu.hr (Postfix) with ESMTP id 26A3C60182; Sun, 29 Oct 2023 12:14:15 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=alu.unizg.hr; s=mail; t=1698578055; bh=dD433RcfZMXumHsqDGuyG1IGuMidDi/U8xCKI0ffmTE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=uKov8Epvhl8Kxt60aShiUCdlHhiA5MYBeGSrMpoJ9kxoqqIaarwaU75cVWLKy6+Xd gtMdWqi2t3B54q2vyFRY4x35tS7/zbKe5cb6xFtA/Hl55bFUEzDVJdURlTeCK0mqm1 kM4f9ZbqdNajfyQnMDDUCfw4YnVqTtFa7FCrWZ/RcWNa4VxBoBPCVkURU3tn8leJzD Unz26dDpwbah/nCDge7ECepBXj0dPIGnLOANkcBrzY6HRqyfpHuQNlmPbvkmV9o54I PGoIdAU5xbHDMn2uKYgWWxUlrvKPgiVmVeb9XIAEavPOSU/ibRTyRmacHsappwD1UI S0xAOOWHV490A== X-Virus-Scanned: Debian amavisd-new at domac.alu.hr Received: from domac.alu.hr ([127.0.0.1]) by localhost (domac.alu.hr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id sKHw8ZPIZR9j; Sun, 29 Oct 2023 12:14:12 +0100 (CET) Received: from defiant.home (78-3-40-247.adsl.net.t-com.hr [78.3.40.247]) by domac.alu.hr (Postfix) with ESMTPSA id 68D7F60173; Sun, 29 Oct 2023 12:14:12 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=alu.unizg.hr; s=mail; t=1698578052; bh=dD433RcfZMXumHsqDGuyG1IGuMidDi/U8xCKI0ffmTE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=c0S7U11d+DXzFtzA+nyfBY+QzUPznh86Pv2iE543t7CyG3iR+Mm2pdQMzwmSqbNF1 93gHpqhYQrQjK2CuiXXCDDFBVrgHyw7EqvJzALlkI3VgzI3QxNH86f2hcjtLOhOOhk bIF99GeZN2QxKYa8RPG/UfJJNBvbZT16viauVYrOjWDwDs6RLmsMeHzm4TObjSip05 KK4VZR7QOTumOq/HanP+ZUUy48mvyov/Xa7bLADww4eFF3kNyRfDQ0uYh4L5EgaGBM 06oaJC2KEiQQMOA/m/BNY1z25mxkwLmgJr0aDdCmpWJIw9MNSJXFMAV65yY4OwHQx/ XqKhLhbVxNtcw== From: Mirsad Goran Todorovac To: Jason Gunthorpe , Joerg Roedel , Lu Baolu , iommu@lists.linux.dev, linux-kernel@vger.kernel.org, netdev@vger.kernel.org Cc: Joerg Roedel , Will Deacon , Robin Murphy , Heiner Kallweit , nic_swsd@realtek.com, "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Mirsad Goran Todorovac , Marco Elver Subject: [PATCH v4 5/5] r8169: Coalesce mac ocp commands for rtl_hw_init_8125 to reduce spinlocks Date: Sun, 29 Oct 2023 12:04:47 +0100 Message-Id: <20231029110442.347448-5-mirsad.todorovac@alu.unizg.hr> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231029110442.347448-1-mirsad.todorovac@alu.unizg.hr> References: <20231029110442.347448-1-mirsad.todorovac@alu.unizg.hr> MIME-Version: 1.0 X-Spam-Status: No, score=-0.6 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Sun, 29 Oct 2023 04:14:36 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1781088207683823204 X-GMAIL-MSGID: 1781088207683823204 Repeated calls to r8168_mac_ocp_write() and r8168_mac_ocp_modify() in the init sequence of the 8125 involve implicit spin_lock_irqsave() and spin_unlock_irqrestore() on each invocation. Coalesced with the corresponding helpers r8168_mac_ocp_write_seq() and r8168_mac_ocp_modify_seq() into sequential write or modidy with a sinqle lock/unlock, these calls reduce overall lock contention. Fixes: f1bce4ad2f1ce ("r8169: add support for RTL8125") Cc: Heiner Kallweit Cc: Marco Elver Cc: nic_swsd@realtek.com Cc: "David S. Miller" Cc: Eric Dumazet Cc: Jakub Kicinski Cc: Paolo Abeni Cc: netdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org Link: https://lore.kernel.org/lkml/20231028005153.2180411-1-mirsad.todorovac@alu.unizg.hr/ Link: https://lore.kernel.org/lkml/20231028110459.2644926-1-mirsad.todorovac@alu.unizg.hr/ Signed-off-by: Mirsad Goran Todorovac Reviewed-by: Jacob Keller --- v4: fixed complaints as advised by Heiner and checkpatch.pl. split the patch into five sections to be more easily manipulated and reviewed introduced r8168_mac_ocp_write_seq() applied coalescing of mac ocp writes/modifies for 8168H, 8125 and 8125B v3: removed register/mask pair array sentinels, so using ARRAY_SIZE(). avoided duplication of RTL_W32() call code as advised by Heiner. drivers/net/ethernet/realtek/r8169_main.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/drivers/net/ethernet/realtek/r8169_main.c b/drivers/net/ethernet/realtek/r8169_main.c index dd65e0384ab3..8dce08d90367 100644 --- a/drivers/net/ethernet/realtek/r8169_main.c +++ b/drivers/net/ethernet/realtek/r8169_main.c @@ -5070,6 +5070,12 @@ static void rtl_hw_init_8168g(struct rtl8169_private *tp) static void rtl_hw_init_8125(struct rtl8169_private *tp) { + static const struct e_info_regdata hw_init_8125_1[] = { + { 0xc0aa, 0x07d0 }, + { 0xc0a6, 0x0150 }, + { 0xc01e, 0x5555 }, + }; + rtl_enable_rxdvgate(tp); RTL_W8(tp, ChipCmd, RTL_R8(tp, ChipCmd) & ~(CmdTxEnb | CmdRxEnb)); @@ -5079,9 +5085,7 @@ static void rtl_hw_init_8125(struct rtl8169_private *tp) r8168_mac_ocp_modify(tp, 0xe8de, BIT(14), 0); r8168g_wait_ll_share_fifo_ready(tp); - r8168_mac_ocp_write(tp, 0xc0aa, 0x07d0); - r8168_mac_ocp_write(tp, 0xc0a6, 0x0150); - r8168_mac_ocp_write(tp, 0xc01e, 0x5555); + r8168_mac_ocp_write_seq(tp, hw_init_8125_1); r8168g_wait_ll_share_fifo_ready(tp); }