From patchwork Tue Jan 2 22:00:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 184520 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:6f82:b0:100:9c79:88ff with SMTP id tb2csp4706355dyb; Tue, 2 Jan 2024 14:06:43 -0800 (PST) X-Google-Smtp-Source: AGHT+IFf4ocr1lcOgBHfgFYmUF2TsB6G5dL4kAUrrxUp0/nE+ssvWYun/P61gw7ldZhx1V1SCc33 X-Received: by 2002:a05:6512:2106:b0:50e:9767:1488 with SMTP id q6-20020a056512210600b0050e97671488mr1796486lfr.46.1704233203548; Tue, 02 Jan 2024 14:06:43 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1704233203; cv=none; d=google.com; s=arc-20160816; b=ZBvEMcqxcJZFhLu2ZD3YxaBnPENOSU360yFJekKWUWuH9eYNBKAOpIMR4eXLc6kNPS 9w2dfl1DMkBMHsu7rrZuhZ3480H5jVCxcuvv8I9Z1xELVWIR5QVmAx7j9P+1sQbdBFa/ reibLGAT+0GUC/PW42mioZcn7LRoezSnwCpxOJReBnDNaWfYJC6H2Seqf/xr5vxaFlF2 ql6Uu7i+jIHdE5SMr9Lm6HDXd6h22aoictqSoDWK0qN35gujVfRhRWezwX/n7KaLBeqh /c/EBp7nhWrI5nuvOVsu2zHAV4evCCOgCbcPFqIDJFfNRiNDCSt8nXyN3KoGeylAQ++z gCnw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=6c+wYTacIgINHvokNTIzxqzbFxSPL5G6VDrdebR45nU=; fh=dlotjpk72cYcmDz0+AxIEXTz48ooVBc8lhZ+ndNwuOg=; b=kEnxOCj19HFofDwCq+5hxwgPs3Z2o6Xl+aCOMIHMt9kGoR48SX45aAm8NTtmfJbSLb QtdEsWBiYjUfjQYXA1XFI/DG0ankfaeVLFrtCVDEvsLAJVd6mcbE3/EW/aIJ02+ksvgE cnkp+eTlEdd6aj7L6RRpzoAGEXy5cdRXmTfebt+BP5RTNPP65ei9m8a9C/hvVkljbbcD rnKl8p0AG7SrDVGft6G+MBLxoui12WQBj+VTd2YbrrAKy0fvyBFVVMjlzWCZ7kMMkZkT gZ28R0aP08eOiw6KXeyvlTWGUrn9yakXJYSPwfMmVWKLhJMtBjQeW1cX3tCf4a8b6JVz zNdA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@sifive.com header.s=google header.b=HP1wacbR; spf=pass (google.com: domain of linux-kernel+bounces-14916-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-14916-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=sifive.com Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id v20-20020a50a454000000b0054cd1a2460asi11815005edb.672.2024.01.02.14.06.43 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jan 2024 14:06:43 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-14916-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@sifive.com header.s=google header.b=HP1wacbR; spf=pass (google.com: domain of linux-kernel+bounces-14916-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-14916-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=sifive.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 00AE01F21AFF for ; Tue, 2 Jan 2024 22:06:43 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 05230199B6; Tue, 2 Jan 2024 22:01:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b="HP1wacbR" X-Original-To: linux-kernel@vger.kernel.org Received: from mail-pg1-f178.google.com (mail-pg1-f178.google.com [209.85.215.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 07A7418EA1 for ; Tue, 2 Jan 2024 22:01:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sifive.com Received: by mail-pg1-f178.google.com with SMTP id 41be03b00d2f7-53fa455cd94so2562418a12.2 for ; Tue, 02 Jan 2024 14:01:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1704232903; x=1704837703; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6c+wYTacIgINHvokNTIzxqzbFxSPL5G6VDrdebR45nU=; b=HP1wacbRDgJCzBv7AGUbFOLF9b1VWCqm1Dfy4j8RUI11g4HsIFZaPeN2dsxBhW64Oh HMp2yVVjhGcS3SVOsqJn6mI9NV+3gzBTAlo9QY9MowPlpIM0mq4/ft/JxoFh3xZ5Mrf7 EiR0pvh1HashwVTzQI5rlrk8FQVzG56V9eVwCjuMN2hsAS4Eqab5Rq+0S67UoBSD3tKr krCs+qnAjt16JeI5rs0ovGOpNiE5LlV/wNvluwlCzV78Ara8ksKlyKMDKmS/MuC/E7KS 6Y6vpwcTtP08CLaQ0EYhOk4N1nhM8ZGxMa1NLVETUNdPSEvD0WUXaaalr0VOCo88aPgU 5NZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1704232903; x=1704837703; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6c+wYTacIgINHvokNTIzxqzbFxSPL5G6VDrdebR45nU=; b=bElJ4hXujPMdCM+Feux5OoLSe3nZ7Ed12G60a4UM/o3+Dgh8uKVCPFoRRkWDtr3toh xul84wcIDyVZxhTRbv50gxomQFc2T0NV655IY2dRGfT0oY2pHJ1roErFcoFOw/Ne/7+L 30sctMeUCSqmPtj47GtWUSvqxJCcqoK0bGb5feOldS19cE78HmkqBAAHydX6Kxxjy4LC lPPKDow7FedXQTvwIObgiCAcpupLy/150T4GfV62WdU3s+XYHd8lrcbbcUhmghGALJRz JWi/JQ+q6cPgd5xR/9lt0ksA293Utr46Ve3kMop52cN6ryenzgNaqXP8WcuOexH5PKCE UZnA== X-Gm-Message-State: AOJu0YxMRd9rULtoF14/eQKo2By0JS3q9kmyNnKDW0vfaCkUFi3CzXj7 r3oIJiAECOY7d/9xfRJH+jKvGByx+eH1qQ== X-Received: by 2002:a17:902:d584:b0:1d4:5f9:d3d7 with SMTP id k4-20020a170902d58400b001d405f9d3d7mr8348439plh.127.1704232903384; Tue, 02 Jan 2024 14:01:43 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id r59-20020a17090a43c100b0028ce507cd7dsm101724pjg.55.2024.01.02.14.01.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jan 2024 14:01:43 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Samuel Holland Subject: [PATCH v4 06/12] riscv: Apply SiFive CIP-1200 workaround to single-ASID sfence.vma Date: Tue, 2 Jan 2024 14:00:43 -0800 Message-ID: <20240102220134.3229156-7-samuel.holland@sifive.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20240102220134.3229156-1-samuel.holland@sifive.com> References: <20240102220134.3229156-1-samuel.holland@sifive.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1787018035322620713 X-GMAIL-MSGID: 1787018035322620713 commit 3f1e782998cd ("riscv: add ASID-based tlbflushing methods") added calls to the sfence.vma instruction with rs2 != x0. These single-ASID instruction variants are also affected by SiFive errata CIP-1200. Until now, the errata workaround was not needed for the single-ASID sfence.vma variants, because they were only used when the ASID allocator was enabled, and the affected SiFive platforms do not support multiple ASIDs. However, we are going to start using those sfence.vma variants regardless of ASID support, so now we need alternatives covering them. Signed-off-by: Samuel Holland --- (no changes since v2) Changes in v2: - Rebase on Alexandre's "riscv: tlb flush improvements" series v5 arch/riscv/include/asm/errata_list.h | 12 +++++++++++- arch/riscv/include/asm/tlbflush.h | 19 ++++++++++++++++++- arch/riscv/mm/tlbflush.c | 23 ----------------------- 3 files changed, 29 insertions(+), 25 deletions(-) diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/errata_list.h index 83ed25e43553..6781460ae564 100644 --- a/arch/riscv/include/asm/errata_list.h +++ b/arch/riscv/include/asm/errata_list.h @@ -44,11 +44,21 @@ ALTERNATIVE(__stringify(RISCV_PTR do_page_fault), \ CONFIG_ERRATA_SIFIVE_CIP_453) #else /* !__ASSEMBLY__ */ -#define ALT_FLUSH_TLB_PAGE(x) \ +#define ALT_SFENCE_VMA_ASID(asid) \ +asm(ALTERNATIVE("sfence.vma x0, %0", "sfence.vma", SIFIVE_VENDOR_ID, \ + ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200) \ + : : "r" (asid) : "memory") + +#define ALT_SFENCE_VMA_ADDR(addr) \ asm(ALTERNATIVE("sfence.vma %0", "sfence.vma", SIFIVE_VENDOR_ID, \ ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200) \ : : "r" (addr) : "memory") +#define ALT_SFENCE_VMA_ADDR_ASID(addr, asid) \ +asm(ALTERNATIVE("sfence.vma %0, %1", "sfence.vma", SIFIVE_VENDOR_ID, \ + ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200) \ + : : "r" (addr), "r" (asid) : "memory") + /* * _val is marked as "will be overwritten", so need to set it to 0 * in the default case. diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index 7712ffe2f6c4..002c4c2620f3 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -22,10 +22,27 @@ static inline void local_flush_tlb_all(void) __asm__ __volatile__ ("sfence.vma" : : : "memory"); } +static inline void local_flush_tlb_all_asid(unsigned long asid) +{ + if (asid != FLUSH_TLB_NO_ASID) + ALT_SFENCE_VMA_ASID(asid); + else + local_flush_tlb_all(); +} + /* Flush one page from local TLB */ static inline void local_flush_tlb_page(unsigned long addr) { - ALT_FLUSH_TLB_PAGE(__asm__ __volatile__ ("sfence.vma %0" : : "r" (addr) : "memory")); + ALT_SFENCE_VMA_ADDR(addr); +} + +static inline void local_flush_tlb_page_asid(unsigned long addr, + unsigned long asid) +{ + if (asid != FLUSH_TLB_NO_ASID) + ALT_SFENCE_VMA_ADDR_ASID(addr, asid); + else + local_flush_tlb_page(addr); } void flush_tlb_all(void); diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 37b3c93e3c30..292d7cf3c4f6 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -7,29 +7,6 @@ #include #include -static inline void local_flush_tlb_all_asid(unsigned long asid) -{ - if (asid != FLUSH_TLB_NO_ASID) - __asm__ __volatile__ ("sfence.vma x0, %0" - : - : "r" (asid) - : "memory"); - else - local_flush_tlb_all(); -} - -static inline void local_flush_tlb_page_asid(unsigned long addr, - unsigned long asid) -{ - if (asid != FLUSH_TLB_NO_ASID) - __asm__ __volatile__ ("sfence.vma %0, %1" - : - : "r" (addr), "r" (asid) - : "memory"); - else - local_flush_tlb_page(addr); -} - /* * Flush entire TLB if number of entries to be flushed is greater * than the threshold below.