From patchwork Sun Sep 24 14:35:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Noah Goldstein X-Patchwork-Id: 144141 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:cae8:0:b0:403:3b70:6f57 with SMTP id r8csp838518vqu; Sun, 24 Sep 2023 14:05:44 -0700 (PDT) X-Google-Smtp-Source: AGHT+IF70r+xhM2j3ZdkaRycnEDDPL3+NpJ23znK7/JnTfKPFpw0Y9gQvVYqrUMC7izWMqYHGpVa X-Received: by 2002:a9d:7a48:0:b0:6b9:4155:7dbd with SMTP id z8-20020a9d7a48000000b006b941557dbdmr6298872otm.0.1695589544503; Sun, 24 Sep 2023 14:05:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695589544; cv=none; d=google.com; s=arc-20160816; b=aiacEkYfpirhP2PW6/ZMK+4YU7VcXaefciQsIUCBfL000G6UENkOx2kiUqjP5fN/gz 9aGW/siRM86hICTbCp+6XVndfVxGVOV9b5A7gzncKwWLfNtuNXze/uFX5NfMAy8mvhoB Vdz/uFvz7UFavlfazEAzyHCCSaUsk4vydMELX+8cILebXXtNv5kyZqINyRmQOJtSqvFQ LbbqtiXtkReiyOOkzupxpOPcd2d2/sQa/IQdxLJclAgwzm4BTIzjLYQd2mvL3HCkwv8x XZ5SgE+665XFbWECq0CFybPavbjiItI4il6RU8viVryu7eIhWh/60nBWYBkizxeUTNCM ABjQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=qKmidxGA713iO3U8UQepy4attClwHmpcYOKxNrg1rc4=; fh=vegKCTfi9EcwJmTLkNY/pt7pjUN00rPFpGjVwf8L/Y4=; b=KigdhhwmSEi+pBKTmooskw8Rf6DB9mJvjIwCTfXlyxYcK6Sujk2fSSDhzxjK8BZDry e8ohPchYWxkafafDKwQaDf+nylMyNJJSTdj953XBUwm2+U13OYumthaZk+3/wDWkLBGF J8QClqljOjrpK4bJBpLiXm2woOafCzagGULocR2gLvtoBWR8fYdTijst5YJCvfPY1qXF A9U28jvb7RRFHpTj2BsUZc/yadP0Gt0WuZwgr4Tfr/nb6trlKrvZkjopyEOwhrdaKBqp rkKZMhN8rdQNWnBGo6/LV5/NYGIV9wkjQxCtFVL7U/5KvVQ5a3X4dyZ2R9lRLu8L2Uqs AUqA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=HKXchKs2; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from lipwig.vger.email (lipwig.vger.email. [23.128.96.33]) by mx.google.com with ESMTPS id b17-20020a656691000000b005503f25aac2si8540811pgw.241.2023.09.24.14.05.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 24 Sep 2023 14:05:44 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) client-ip=23.128.96.33; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=HKXchKs2; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id 1A46B8145952; Sun, 24 Sep 2023 07:36:10 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229889AbjIXOgD (ORCPT + 30 others); Sun, 24 Sep 2023 10:36:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48070 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229795AbjIXOgC (ORCPT ); Sun, 24 Sep 2023 10:36:02 -0400 Received: from mail-qv1-xf29.google.com (mail-qv1-xf29.google.com [IPv6:2607:f8b0:4864:20::f29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7B9BBFA for ; Sun, 24 Sep 2023 07:35:53 -0700 (PDT) Received: by mail-qv1-xf29.google.com with SMTP id 6a1803df08f44-65af7e20f39so13001426d6.2 for ; Sun, 24 Sep 2023 07:35:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695566152; x=1696170952; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qKmidxGA713iO3U8UQepy4attClwHmpcYOKxNrg1rc4=; b=HKXchKs2E/3D1Ds0Q546E2AwbWrd81b7nxJNV+BhrLWUwilXk5radRJlLhgoaIMSVu QeqWbOKlCny0ydPSE2m88JdN41bM87u2zZJLzpTreSNWqZzbknJ2o2Oa07XuHYWhAbU6 I+0hoPW8c9WSX6oZrMq5ygja1qNaYOtpJa8vYOoU/c2AgenuoQcLxCJ0v/qD3yxEIxBB o8rsvkQJGKh+aKpbvpZfFFDIEldRjZ2W9Epmt+bFbVmMeM6K3A560y/wMA3UMef935ki cBcBz8p4xnhzzibOT9bPGu3ev4J1dqzqPgn5pES7RQgWx8nXnFWN2hLUreUsviGX3vpT J0eA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695566152; x=1696170952; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qKmidxGA713iO3U8UQepy4attClwHmpcYOKxNrg1rc4=; b=UT0sFrV5g4OfANO9cbqzAC7vvSqLiKenp0cN/CwOBl2Bzc4OlheMbYUJj+jkmOSYfr rMfzBxES4KixZI4XhdWzKOHsxdHpWwck/SF2Gic9x2G80KVuN027jm8Tks1yTPSSaQEN GJFcnYaOTuwdM718CaDa2dAKQrcp140qqCajIEN9VNrGGovRPORClXAjfQg+kUAw0wYm sLyYSReYLSTdqXsedP5mdFg02EB1yJIXSa/FaVeRdwZQhMosqd6DSbvVZ57H1F2zIYYy XBM5ArPq6HFHr68qsUliuDLSq5QHYLbFuCJ6X6xaBMU+MvgspAkhVOOIog/FopNqPktX +Bxg== X-Gm-Message-State: AOJu0Yy4SBsXdOqYdOc9QVO2Dk4cG+fJU3+dM3ru9k/54lcZF2Sw9vBb EG26YtiyLyFGurvh+G9dq0Q= X-Received: by 2002:a05:6214:4505:b0:656:3b4c:b98b with SMTP id oo5-20020a056214450500b006563b4cb98bmr5032160qvb.11.1695566152488; Sun, 24 Sep 2023 07:35:52 -0700 (PDT) Received: from noahgold-DESK.. (pool-100-37-150-153.nycmny.fios.verizon.net. [100.37.150.153]) by smtp.gmail.com with ESMTPSA id a11-20020a0cca8b000000b0063f88855ef2sm3254275qvk.101.2023.09.24.07.35.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 24 Sep 2023 07:35:51 -0700 (PDT) From: Noah Goldstein To: x86@kernel.org Cc: linux-kernel@vger.kernel.org, edumazet@google.com, tglx@linutronix.de, mingo@redhat.com, torvalds@linux-foundation.org, bp@alien8.de, dave.hansen@linux.intel.com, David.Laight@aculab.com, hpa@zytor.com, goldstein.w.n@gmail.com, David Laight Subject: x86/csum: Remove unnecessary odd handling Date: Sun, 24 Sep 2023 09:35:49 -0500 Message-Id: <20230924143549.539396-1-goldstein.w.n@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230628020657.957880-1-goldstein.w.n@gmail.com> References: <20230628020657.957880-1-goldstein.w.n@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-0.6 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Sun, 24 Sep 2023 07:36:10 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1769938355391134531 X-GMAIL-MSGID: 1777954502212229187 The special case for odd aligned buffers is unnecessary and mostly just adds overhead. Aligned buffers is the expectations, and even for unaligned buffer, the only case that was helped is if the buffer was 1-byte from word aligned which is ~1/7 of the cases. Overall it seems highly unlikely to be worth to extra branch. It was left in the previous perf improvement patch because I was erroneously comparing the exact output of `csum_partial(...)`, but really we only need `csum_fold(csum_partial(...))` to match so its safe to remove. All csum kunit tests pass. Signed-off-by: Noah Goldstein Reviewed-by: Eric Dumazet Reviewed-by: David Laight --- arch/x86/lib/csum-partial_64.c | 36 ++++------------------------------ 1 file changed, 4 insertions(+), 32 deletions(-) diff --git a/arch/x86/lib/csum-partial_64.c b/arch/x86/lib/csum-partial_64.c index cea25ca8b8cf..557e42ede68e 100644 --- a/arch/x86/lib/csum-partial_64.c +++ b/arch/x86/lib/csum-partial_64.c @@ -11,26 +11,9 @@ #include #include -static inline unsigned short from32to16(unsigned a) +static inline __wsum csum_finalize_sum(u64 temp64) { - unsigned short b = a >> 16; - asm("addw %w2,%w0\n\t" - "adcw $0,%w0\n" - : "=r" (b) - : "0" (b), "r" (a)); - return b; -} - -static inline __wsum csum_tail(u64 temp64, int odd) -{ - unsigned int result; - - result = add32_with_carry(temp64 >> 32, temp64 & 0xffffffff); - if (unlikely(odd)) { - result = from32to16(result); - result = ((result >> 8) & 0xff) | ((result & 0xff) << 8); - } - return (__force __wsum)result; + return (__force __wsum)((temp64 + ror64(temp64, 32)) >> 32); } /* @@ -47,17 +30,6 @@ static inline __wsum csum_tail(u64 temp64, int odd) __wsum csum_partial(const void *buff, int len, __wsum sum) { u64 temp64 = (__force u64)sum; - unsigned odd; - - odd = 1 & (unsigned long) buff; - if (unlikely(odd)) { - if (unlikely(len == 0)) - return sum; - temp64 = ror32((__force u32)sum, 8); - temp64 += (*(unsigned char *)buff << 8); - len--; - buff++; - } /* * len == 40 is the hot case due to IPv6 headers, but annotating it likely() @@ -73,7 +45,7 @@ __wsum csum_partial(const void *buff, int len, __wsum sum) "adcq $0,%[res]" : [res] "+r"(temp64) : [src] "r"(buff), "m"(*(const char(*)[40])buff)); - return csum_tail(temp64, odd); + return csum_finalize_sum(temp64); } if (unlikely(len >= 64)) { /* @@ -143,7 +115,7 @@ __wsum csum_partial(const void *buff, int len, __wsum sum) : [res] "+r"(temp64) : [trail] "r"(trail)); } - return csum_tail(temp64, odd); + return csum_finalize_sum(temp64); } EXPORT_SYMBOL(csum_partial);