From patchwork Thu Jan 25 14:59:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jisheng Zhang X-Patchwork-Id: 192127 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:e09d:b0:103:945f:af90 with SMTP id gm29csp42421dyb; Thu, 25 Jan 2024 07:13:24 -0800 (PST) X-Google-Smtp-Source: AGHT+IF+3rjSxDzQAZxuLme5QYB5LzlzT1Gmn39g6+VgmzdH2bO7YLI34yId3B0p9+fNUapIyu9q X-Received: by 2002:a05:620a:2411:b0:783:33df:dc6 with SMTP id d17-20020a05620a241100b0078333df0dc6mr1387441qkn.49.1706195603835; Thu, 25 Jan 2024 07:13:23 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706195603; cv=pass; d=google.com; s=arc-20160816; b=uiXAbdu5lwqSf6YrVL2FBLd8jv7Nm4ofVhE1gDJ3Pl4vWOe34RSlAFZtxR6khy9Hwf DHGyHv+5f8fVg1CsaLapEBmSGgZhEr9ED4dhVNpYeC6w3m2M2qUJNmRllwBARUF2MGyR LnIwcfRGsOOWzSGgDzN0ut2j0tbV8LUE6UxUVPempnmews0Q7P+Lak7HrmydT82X/PGx UuGu3QYhPKcf5ZyflbCE7XdoBkbFaRgxiQVs+o4sARc2Dz8CIEi5pdeXf0NW8oZF7gnm t0fLV+XCEm1JD0erwa86H42eIxnLDU0cNrFBIqhqNMztUB4bCGaytL5dJVCwjYXefccS kVSg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=mCN1N1vlfyYpuoFeuVzPmMowwKFrbo6J6bJRSI0N2bQ=; fh=q77r3SGtFcw5tD9IkA7hA6mq4WwiMsIO0yD7bT7A5oM=; b=RaPNVMk3OJXIcqmO2OhtvrmypqMuLbxNqxFlVP8XfO3IPpmAvEkVpbBbdNt+6ZqEnj aLq50ygyBHdlE7B8Kul0wFQq0VXzPTsGtNa3AKqbQV88rQUmazaMMZIO7iYg/ROt3qOl fqJ3+9K6O5YmlJZ3GS04T35FdcyjHOewwe2Rc8E2JKi48UbLDVz0XN0W/BRH8dRS7uPa oZOYHmXCMDj6rduYo2kfJ19yuRNJztw45NzaO2JRHYTLK7lRZ7BatYlNeiCPOndiIMXe RlbgvNvASqbn3d2pxfWOMcAtmzxtPTa7dPOW1L9+wT6R+RsWV6VaNGuzVc/9TB3R+lzL HWIw== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=nSXdWtBR; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-38803-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-38803-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id s17-20020a05620a255100b00781e1787c1csi14185440qko.507.2024.01.25.07.13.23 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Jan 2024 07:13:23 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-38803-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=nSXdWtBR; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-38803-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-38803-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 9D2361C20F5D for ; Thu, 25 Jan 2024 15:13:23 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C504C74E05; Thu, 25 Jan 2024 15:12:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="nSXdWtBR" Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1FC4C73175 for ; Thu, 25 Jan 2024 15:11:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706195519; cv=none; b=aJg/fH88/Cz3hmnRqk7ygPLaFfX78XyMcOld6VrGJKqF8Xu0qE/aUqiVpr1FX1YelGk8Az5XOMcYqIMSwUzDSLzteHwM0p24IKxp5iYqyt9ex/LDD1rsGBQZziNHuBvQyTq+FrVLK728M5rC57zdiK1PefCUfpyG/cQSqOhF5UE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706195519; c=relaxed/simple; bh=qaC0W89Ij+l6ejUX971u7DsedKgRmIMuV9AHu7Mxo8U=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=u6DukLqw02jP4JWaT2gkFSmqArklwz5O8v/Z4woQJWBZRiDsou/+POqZdf/0B4HzOnHr9MdjcyC2fyRGFbbTOH8PHRQJF71GGess7r1bLR4227Ph6o0cgdSHm7gwmq6lyOJcAZsi5Xzr/ENVcfzoo7NQ8THyQ8iXmAh1O6YuCOI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=nSXdWtBR; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0FBD0C43390; Thu, 25 Jan 2024 15:11:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706195518; bh=qaC0W89Ij+l6ejUX971u7DsedKgRmIMuV9AHu7Mxo8U=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nSXdWtBRn7k/oWkIm7gkawOUKqSN1OL4w9QeB24Fl8ZRHsLgAjoCIHhFWXd79Mp/k 3Ywru/xCQb2wY60k7l+yyuii5qOHoNUPxJY9OA4fDUB9tsBI4CKKLva81H4c5Npxko soBi2N+5Uw7UwkYTWpLpjdPKPQ0wBM3LM74JmxAgiScgs4je9bRGpVKtSOL7XfqcrF qK+tHIU3yLfbf+4jshsQqPu95ovDWv9J+0QvFX4IW5ZFrIZ/pWLLCKym8nubIjGZDb ItQ+fc24J9ME0sCIyGDYtuTdbnutgAEV9mcG2Qrx1ywShbrYa9g7szY7BW8Otb0SZs axAz+H2EwRvfA== From: Jisheng Zhang To: Paul Walmsley , Palmer Dabbelt , Albert Ou Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, Andrea Parri Subject: [PATCH v3 1/2] riscv: select ARCH_USE_CMPXCHG_LOCKREF Date: Thu, 25 Jan 2024 22:59:06 +0800 Message-ID: <20240125145908.968-2-jszhang@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240125145908.968-1-jszhang@kernel.org> References: <20240125145908.968-1-jszhang@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789075761503407793 X-GMAIL-MSGID: 1789075761503407793 Select ARCH_USE_CMPXCHG_LOCKREF to enable the cmpxchg-based lockless lockref implementation for riscv. Using Linus' test case[1] on TH1520 platform, I see a 11.2% improvement. On JH7110 platform, I see 12.0% improvement. Link: http://marc.info/?l=linux-fsdevel&m=137782380714721&w=4 [1] Signed-off-by: Jisheng Zhang Reviewed-by: Andrea Parri --- arch/riscv/Kconfig | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index fdd1a595ebd8..248cea3e2749 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -51,6 +51,7 @@ config RISCV select ARCH_SUPPORTS_PAGE_TABLE_CHECK if MMU select ARCH_SUPPORTS_PER_VMA_LOCK if MMU select ARCH_SUPPORTS_SHADOW_CALL_STACK if HAVE_SHADOW_CALL_STACK + select ARCH_USE_CMPXCHG_LOCKREF if 64BIT select ARCH_USE_MEMTEST select ARCH_USE_QUEUED_RWLOCKS select ARCH_USES_CFI_TRAPS if CFI_CLANG From patchwork Thu Jan 25 14:59:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jisheng Zhang X-Patchwork-Id: 192134 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:e09d:b0:103:945f:af90 with SMTP id gm29csp48672dyb; Thu, 25 Jan 2024 07:22:34 -0800 (PST) X-Google-Smtp-Source: AGHT+IEgpnmdWU0vF9uJATpA/GU9V1ePQZoCCDysGcO+RHa+nJAwoOXDEvnRUfRbg3kjaU8QCKnI X-Received: by 2002:a17:903:2498:b0:1d5:30fd:26b6 with SMTP id p24-20020a170903249800b001d530fd26b6mr967983plw.66.1706196154408; Thu, 25 Jan 2024 07:22:34 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706196154; cv=pass; d=google.com; s=arc-20160816; b=r0SIxF9pEBkcBEWYiDFTEvs3qg2YV86B5g8S6YsAS/LE/S7a67vFgU4DOK5a65kwJ3 3Pg2jlcSPc0qnvlZYMjXiKdm1TmkTjsH7cYUubXiwBPMeeiOo87Wb0CpwUb2dOT7uDEr 35xt3IhXXSt3JxzxqkoaADaxKwogDNyDDmRQtFR2N1JEewTtS2lfv9uhShRCwi/7WKui cx3xFXVPYrczMcbJ4DghNRSujg9pZITWs866kNGKID14c2jGgCob8o/9+XJ9zZST/qFh 1vv54OR05uwz9Z9c/2keba7Geik/EdGf1n91FVCG8gFihSFFuad+GSgzMppKe3W/Pg3l wLPQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=lCf+nc/wEr6+0xMMDe4hffYWrLInR/6BM8AmJX5LLIQ=; fh=q77r3SGtFcw5tD9IkA7hA6mq4WwiMsIO0yD7bT7A5oM=; b=FjiA8j9Ze78l7Ad/Sd+53xU/Mjas6Gm88aAPAXh598oOETJeGtMYvrwp8diZQf7bjv kyMio20wpMKEmzC8s0f7bpKDfI62rt8r17sZPO3/whG2D56fVJE+gSG0cOwYGkUaLMaE 3vp6DnrndTo1QTHPP4i44lsB5PQ7f5jAOfOca5Y0SibvFwyk+jnms/pxgxu5tX5KpCPP LAXxDovRmMqBkrcvAE5dmuoaBL7srp8XLAn8IG94Oxf/5KQTlS0X50tQ8eQ7O/C8cJ4M 7tEXmuKSL3hNgNu4T/JYG3k2hzCQ78jDdyO2zilF7LLGyrfKGVI22qzBg2xE4zk0jRuE E+Xg== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=DSvxk841; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-38805-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-38805-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id s15-20020a170902ea0f00b001d6f67afa3bsi10761243plg.113.2024.01.25.07.22.34 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Jan 2024 07:22:34 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-38805-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=DSvxk841; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-38805-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-38805-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id B8492295416 for ; Thu, 25 Jan 2024 15:13:43 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id B49727317C; Thu, 25 Jan 2024 15:12:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="DSvxk841" Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 067E0745F1 for ; Thu, 25 Jan 2024 15:12:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706195521; cv=none; b=V+DL+6UUxvv+sWOGPKTHCkjyCK5DvBuJpLcyfw2Sqp/A/9k/SYBQr+JSysIYHJsiqmrD96eFv4nXf9JRUxdYhhJ8/QfF2bF2zuiz0K+bkexuyGoXrnTR3d0zJlXUQAmVpyD1wHSdX0+pPuTUUr7EYPclo0rg5MqKp9oCYFWyTqA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706195521; c=relaxed/simple; bh=lQa90hyBkLuNQGPI6gmRovggw7RSn4HqbLkZoyCISio=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=WGTfZGtww+cVeYN0R8uSyQKIFLPzsGUJjF4QTktwmMRTJsyia6hm3EiZn2SiWeHlhr0Upp8mLFJXcPRIUpwhTMOdTLMshBa5r+WEKVfvBbdd2/QiZcxA8TM+jfe6I2oidQ97VXAfkmD8feJhpWDrdSZlE4YlbZxuaTRtKZROWMQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=DSvxk841; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 08165C433C7; Thu, 25 Jan 2024 15:11:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706195520; bh=lQa90hyBkLuNQGPI6gmRovggw7RSn4HqbLkZoyCISio=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DSvxk841kVJVmEf4He8NwMxVcYFLyFxsuI674Ebdd4b6L1RX4gAcaCYBnXUxFlaP7 p0a7XMaGgLwMu9eLSvy2e4HRngLmNotdK6jT+id/Ta+sSK5JumdHcVKChRiqAOTt7W i45ayymlz9m5T36KREF7ZYyUxFYddYN8TuQXbVVhaEJYUGz5x0fuPv/L8ChhkvbNRV gkI48DkJdAPmLS6vbaGRAx7xZRmzIzGt0AahTuW2S7d94uawJQm7xE5j0MJEHxhZt/ 7UBJc/rrtAChZ916eK0pwGYfJv2j3KVEpscpLDQy1kWrRBdzJ7lCT+WnTt5esCkQME Jr9iJ+lMf1/4A== From: Jisheng Zhang To: Paul Walmsley , Palmer Dabbelt , Albert Ou Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, Andrea Parri Subject: [PATCH v3 2/2] riscv: cmpxchg: implement arch_cmpxchg64_{relaxed|acquire|release} Date: Thu, 25 Jan 2024 22:59:07 +0800 Message-ID: <20240125145908.968-3-jszhang@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240125145908.968-1-jszhang@kernel.org> References: <20240125145908.968-1-jszhang@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789076339078225726 X-GMAIL-MSGID: 1789076339078225726 After selecting ARCH_USE_CMPXCHG_LOCKREF, one straight futher optimization is implementing the arch_cmpxchg64_relaxed() because the lockref code does not need the cmpxchg to have barrier semantics. At the same time, implement arch_cmpxchg64_acquire and arch_cmpxchg64_release as well. However, on both TH1520 and JH7110 platforms, I didn't see obvious performance improvement with Linus' test case [1]. IMHO, this may be related with the fence and lr.d/sc.d hw implementations. In theory, lr/sc without fence could give performance improvement over lr/sc plus fence, so add the code here to leave performance improvement room on newer HW platforms. Link: http://marc.info/?l=linux-fsdevel&m=137782380714721&w=4 [1] Signed-off-by: Jisheng Zhang Reviewed-by: Andrea Parri --- arch/riscv/include/asm/cmpxchg.h | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h index 2f4726d3cfcc..6318187f426f 100644 --- a/arch/riscv/include/asm/cmpxchg.h +++ b/arch/riscv/include/asm/cmpxchg.h @@ -360,4 +360,22 @@ arch_cmpxchg_relaxed((ptr), (o), (n)); \ }) +#define arch_cmpxchg64_relaxed(ptr, o, n) \ +({ \ + BUILD_BUG_ON(sizeof(*(ptr)) != 8); \ + arch_cmpxchg_relaxed((ptr), (o), (n)); \ +}) + +#define arch_cmpxchg64_acquire(ptr, o, n) \ +({ \ + BUILD_BUG_ON(sizeof(*(ptr)) != 8); \ + arch_cmpxchg_acquire((ptr), (o), (n)); \ +}) + +#define arch_cmpxchg64_release(ptr, o, n) \ +({ \ + BUILD_BUG_ON(sizeof(*(ptr)) != 8); \ + arch_cmpxchg_release((ptr), (o), (n)); \ +}) + #endif /* _ASM_RISCV_CMPXCHG_H */