From patchwork Wed Oct 26 10:40:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenyao Hai X-Patchwork-Id: 11183 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp183154wru; Wed, 26 Oct 2022 03:47:02 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6qMoiy4MjV76v6dYjtNg5yvZj4YiWu0TtK93pOJ5D4hyfMi7yy2uYY5tMzag0fykf85R6B X-Received: by 2002:a63:24d:0:b0:452:87c1:9781 with SMTP id 74-20020a63024d000000b0045287c19781mr36964736pgc.512.1666781222524; Wed, 26 Oct 2022 03:47:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666781222; cv=none; d=google.com; s=arc-20160816; b=tZZSmcXxTPo+30q5HD8S27XqJM5lluNgfmRI5AEAZ7k9kHYRECqnpxPX9mKUF3uUGb dnEwG1YHDk52FZcdo3aZkkwS0AFsfyriPDKb+37MYOvXQaRizrMjrasWh5k+y7er95Sb iCwTHmaRqlQ7G30TcuuDbFHHOo/ZDxOEKOhdVlYkKiQfnCWMY99EjJoTF/+qE9PGGqr1 rWlTHJgnWPUaZ9u/z20DTvKiimKdczkHy3bsTeFkI1wf1Wsyc8aYHL+FMH4GcaJCg9x8 ZMEqVy4CfqFLHADuiSlyM9ENbBxKnZYDpNo6xHLEU0v4mQTHA0XjArgjZ99pqhRoaykL uXTQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:feedback-id:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from; bh=4nsaFWfkZHRw0nksR7WqGvrRYtHQL21XPf2esmru0so=; b=kRskS0tbgzyYgicWfYyzX7IyerfidFLTCzDIvuMCGxrQhzWq9kQjFgCB+nj4DdgUcX Ch8YrCLUX1fJPo6HsmN6NVNmShihgtbcQYSIpeMtd+3he6vsKN6r2qob5N/cmkVEDksw Yeh3xhBzZRstQkISO67VBOeJViUTnTy8dn5ODLTCYtZNRPQfXo4LXsBakIpaHq7RSvN+ OYoI5eELHe6OK8JZodXK91shED2TUevpGTmRTucUNy4RpIOYfLCwW0J5rIsG6pvJg4zq RB301ebrP9VdpRMYWF5JE4m8kTHzzf/QGAyl9tJHdA6DxExCo5ko+OtBCI1x6r3Ii33R PslA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id pa2-20020a17090b264200b0021320088071si2159421pjb.175.2022.10.26.03.46.47; Wed, 26 Oct 2022 03:47:02 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233166AbiJZKla (ORCPT + 99 others); Wed, 26 Oct 2022 06:41:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53190 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233094AbiJZKl2 (ORCPT ); Wed, 26 Oct 2022 06:41:28 -0400 Received: from smtpbgau1.qq.com (smtpbgau1.qq.com [54.206.16.166]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3AE35A231F for ; Wed, 26 Oct 2022 03:41:23 -0700 (PDT) X-QQ-mid: bizesmtp72t1666780844t14jq9v9 Received: from localhost.localdomain ( [101.6.93.82]) by bizesmtp.qq.com (ESMTP) with id ; Wed, 26 Oct 2022 18:40:39 +0800 (CST) X-QQ-SSF: 0140000000000060B000000A0000000 X-QQ-FEAT: LG+NUo/f6sH/5cg3evHTTvo84aPoo8de19Y7L/kvt0DQWpVkBQ87B8w35Spxc r7yWzeAKHrTO4ar9iZQKg0C6tgnLQgquRZJz67RCahATJwXLarxPJku1iYNue5PPLvw6zI9 1DoJ3tDWf61LB3gsRazJpM2WTznKu5ba2scVA6PJ9B8ZAGNukYi8RfkfdU8NUIVMSUiZKlD yLs3sI7wmwg9G+fzeY0Y46u9uCUVeVi2pvx0KoZlNsSGPJ++SbjpxVM4UCAKp6VDALOFxq9 m5/wEGveIT0Gob4/F8lX3Mt4cLudhdq2q5S9STDbRaMpgWoFFwHTUw7HxEcptFhLXSoc2QS Mw8xXDFJRK2o28+/A9QL6fcL/quzThVhvuGafDRIVvSkLQvu4dwfngoCy38F+anzYHUw47T 3HKaaYKWF2o= X-QQ-GoodBg: 1 From: Wen Yao To: paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, dennis@kernel.org, tj@kernel.org, cl@linux.com Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Wen Yao Subject: [PATCH 1/2] riscv: percpu:Add riscv percpu operations Date: Wed, 26 Oct 2022 18:40:14 +0800 Message-Id: <20221026104015.565468-2-haiwenyao@uniontech.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221026104015.565468-1-haiwenyao@uniontech.com> References: <20221026104015.565468-1-haiwenyao@uniontech.com> MIME-Version: 1.0 X-QQ-SENDSIZE: 520 Feedback-ID: bizesmtp:uniontech.com:qybglogicsvr:qybglogicsvr2 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747746787359720429?= X-GMAIL-MSGID: =?utf-8?q?1747746787359720429?= This patch use riscv AMO(Atomic Memory Operation) instructions to optimise some this_cpu_and this_cpu_or this_cpu_add operations. It reuse cmpxchg_local() to impletment this_cpu_cmpxchg macros. It reuse xchg_relaxed() to impletment this_cpu_xchg macros. Signed-off-by: Wen Yao --- arch/riscv/include/asm/percpu.h | 101 ++++++++++++++++++++++++++++++++ 1 file changed, 101 insertions(+) create mode 100644 arch/riscv/include/asm/percpu.h diff --git a/arch/riscv/include/asm/percpu.h b/arch/riscv/include/asm/percpu.h new file mode 100644 index 000000000000..ae796e328442 --- /dev/null +++ b/arch/riscv/include/asm/percpu.h @@ -0,0 +1,101 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (C) 2020-2022 Union Tech Software Technology Corporation Limited + */ +#ifndef __ASM_PERCPU_H +#define __ASM_PERCPU_H + +#include + +#define PERCPU_OP(op, asm_op, c_op) \ + static inline unsigned long __percpu_##op(void *ptr, \ + unsigned long val, int size) \ + { \ + unsigned long ret; \ + switch (size) { \ + case 4: \ + __asm__ __volatile__( \ + "amo" #asm_op ".w" \ + " %[ret], %[val], %[ptr]\n" \ + : [ret] "=&r"(ret), [ptr] "+A"(*(u32 *)ptr) \ + : [val] "r"(val)); \ + break; \ + case 8: \ + __asm__ __volatile__( \ + "amo" #asm_op ".d" \ + " %[ret], %[val], %[ptr]\n" \ + : [ret] "=&r"(ret), [ptr] "+A"(*(u64 *)ptr) \ + : [val] "r"(val)); \ + break; \ + default: \ + ret = 0; \ + BUILD_BUG(); \ + } \ + \ + return ret c_op val; \ + } + +PERCPU_OP(add, add, +) +PERCPU_OP(and, and, &) +PERCPU_OP(or, or, |) +#undef PERCPU_OP + +/* this_cpu_xchg */ +#define _protect_xchg_local(pcp, val) \ + ({ \ + typeof(*raw_cpu_ptr(&(pcp))) __ret; \ + preempt_disable_notrace(); \ + __ret = xchg_relaxed(raw_cpu_ptr(&(pcp)), val); \ + preempt_enable_notrace(); \ + __ret; \ + }) + +/* this_cpu_cmpxchg */ +#define _protect_cmpxchg_local(pcp, o, n) \ + ({ \ + typeof(*raw_cpu_ptr(&(pcp))) __ret; \ + preempt_disable_notrace(); \ + __ret = cmpxchg_local(raw_cpu_ptr(&(pcp)), o, n); \ + preempt_enable_notrace(); \ + __ret; \ + }) + +#define _pcp_protect(operation, pcp, val) \ + ({ \ + typeof(pcp) __retval; \ + preempt_disable_notrace(); \ + __retval = (typeof(pcp))operation(raw_cpu_ptr(&(pcp)), (val), \ + sizeof(pcp)); \ + preempt_enable_notrace(); \ + __retval; \ + }) + +#define _percpu_add(pcp, val) _pcp_protect(__percpu_add, pcp, val) + +#define _percpu_add_return(pcp, val) _percpu_add(pcp, val) + +#define _percpu_and(pcp, val) _pcp_protect(__percpu_and, pcp, val) + +#define _percpu_or(pcp, val) _pcp_protect(__percpu_or, pcp, val) + +#define this_cpu_add_4(pcp, val) _percpu_add(pcp, val) +#define this_cpu_add_8(pcp, val) _percpu_add(pcp, val) + +#define this_cpu_add_return_4(pcp, val) _percpu_add_return(pcp, val) +#define this_cpu_add_return_8(pcp, val) _percpu_add_return(pcp, val) + +#define this_cpu_and_4(pcp, val) _percpu_and(pcp, val) +#define this_cpu_and_8(pcp, val) _percpu_and(pcp, val) + +#define this_cpu_or_4(pcp, val) _percpu_or(pcp, val) +#define this_cpu_or_8(pcp, val) _percpu_or(pcp, val) + +#define this_cpu_xchg_4(pcp, val) _protect_xchg_local(pcp, val) +#define this_cpu_xchg_8(pcp, val) _protect_xchg_local(pcp, val) + +#define this_cpu_cmpxchg_4(ptr, o, n) _protect_cmpxchg_local(ptr, o, n) +#define this_cpu_cmpxchg_8(ptr, o, n) _protect_cmpxchg_local(ptr, o, n) + +#include + +#endif /* __ASM_PERCPU_H */ From patchwork Wed Oct 26 10:40:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenyao Hai X-Patchwork-Id: 11184 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp183190wru; Wed, 26 Oct 2022 03:47:06 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5rI6dugyaudpE+Y5SsA5/5KRQWolRNmTiC9/KhCsxDb4e111UYHq3uG721VDX8a4NCebus X-Received: by 2002:a17:90b:3b8d:b0:20d:5c7a:abf1 with SMTP id pc13-20020a17090b3b8d00b0020d5c7aabf1mr3626554pjb.118.1666781225849; Wed, 26 Oct 2022 03:47:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666781225; cv=none; d=google.com; s=arc-20160816; b=HMIsu4QZ6m5l4VbqTEQamubSTP7AubK5u5yABK57OEoSpCfoofr2X45ieAarI63ho5 9xw5JbPu+g+btja0xfhhk7tfajCCJIIYvTqFlA5StSCcjBI6gjXHIlk37d/2N30ueLPx CguQiZwtAEta/AlBxkznK1jQbDPS08pJGIGW0dOWugkL6FFjRMqelOrpARuCebWaom7w ui5Um3YkwRNIjGPc5JaA/7rj5/SYNBTLuD9qPb1Vulzs2f22DBdCdbXRl4kQtlvCpW6E tdxCmJlcvKteSvFPfkpQEc9A5CxcA5y166R3wV3JndX3xziV1jwUSA3D/DTVKznd6+VC r/CA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:feedback-id:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from; bh=MXhXy3r55+UgmPfkrtBbtHf/rHUUGogeF5L7VF36P5c=; b=T135NZ6KTMXo4pj1AR3qxgvO9E9KLR26QM9iJXWPR5VO99Mq/+h9lQcHGzNUQ0qBxr uSVmUKSHH08rAod8PraHlRHyGbrW1A0HSrVdTjPblTZ/sxL4VDj8jrDsfGYNW/Uc5tKr HIguJd3naZLOl9iP9Y/RH3xNslwLk3fpiTXnu51/OEYIaNLT579yXCG/IGf1APe9gEWw PAdN6dJVR6Hv0tIZ8/l/G8h1QrUgbtAdbkeq6KQTcjnCX/lIXV62bxUN9cgPI80tXSJI 1qxjzOCP0S1Rx8tON3jPXdl9nF6Qo7+wFvGC+T5Im/+XL0rLfi74w7oi6VYK++EqFCmL SPNQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id r2-20020a056a00216200b0056353f0bd40si5904951pff.29.2022.10.26.03.46.51; Wed, 26 Oct 2022 03:47:05 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233396AbiJZKlg (ORCPT + 99 others); Wed, 26 Oct 2022 06:41:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53320 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233326AbiJZKld (ORCPT ); Wed, 26 Oct 2022 06:41:33 -0400 Received: from smtpbgau1.qq.com (smtpbgau1.qq.com [54.206.16.166]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5D8D6A033A for ; Wed, 26 Oct 2022 03:41:31 -0700 (PDT) X-QQ-mid: bizesmtp72t1666780860thlpjusd Received: from localhost.localdomain ( [101.6.93.82]) by bizesmtp.qq.com (ESMTP) with id ; Wed, 26 Oct 2022 18:40:46 +0800 (CST) X-QQ-SSF: 0140000000000060B000000A0000000 X-QQ-FEAT: rGm7xzoh3hl3ya5sdSHuKGIIU47rr/fjDyiOLrmcLON1wIvSddcUSf+DDWDU6 NACAphMLViC2kF9KRr/8fPaM9cG+p9AoXQtAfI/yZNMEcQt5G3zgMiOtXUG3OblLtxq2+b2 bs6Aen9/b7YT8b1MIlWAHuIFvvTZFlQ+gOED3tByPWzVdNFEkBTjQA+9fVV5g2RZBfciQEp hdm8N7btrbiOzOzHqflybP6uY01MZjrMchRycFsc/a5GbTfz6n4uZcQK2lWT2IxfJUYEt4E M1n+M1rT5VlnH0yy/F1nwzBtxiGlHz5Jw6enyciIzBCbkp2EPvEEiOLMW6wshtOOFQ2QncE 0x/NKiFIAsEJQtTllmhLBFWQpzN6QHVIYoRm269ADO9ovCNFc0FKtx431pFRzmFk3t9PR0+ wgJjvPN4/e4= X-QQ-GoodBg: 1 From: Wen Yao To: paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, dennis@kernel.org, tj@kernel.org, cl@linux.com Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Wen Yao Subject: [PATCH 2/2] riscv:kconfig:select HAVE_CMPXCHG_LOCAL Date: Wed, 26 Oct 2022 18:40:15 +0800 Message-Id: <20221026104015.565468-3-haiwenyao@uniontech.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221026104015.565468-1-haiwenyao@uniontech.com> References: <20221026104015.565468-1-haiwenyao@uniontech.com> MIME-Version: 1.0 X-QQ-SENDSIZE: 520 Feedback-ID: bizesmtp:uniontech.com:qybglogicsvr:qybglogicsvr2 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747746790283235855?= X-GMAIL-MSGID: =?utf-8?q?1747746790283235855?= We implement an optimised this_cpu_cmpxchg API,so select HAVE_CMPXCHG_LOCA to open cmpxchg-local feature. Signed-off-by: Wen Yao --- arch/riscv/Kconfig | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 6b48a3ae9843..8e19df5a8227 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -88,6 +88,7 @@ config RISCV select HAVE_ARCH_THREAD_STRUCT_WHITELIST select HAVE_ARCH_VMAP_STACK if MMU && 64BIT select HAVE_ASM_MODVERSIONS + select HAVE_CMPXCHG_LOCAL select HAVE_CONTEXT_TRACKING_USER select HAVE_DEBUG_KMEMLEAK select HAVE_DMA_CONTIGUOUS if MMU