From patchwork Tue Oct 18 11:33:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Kirill A. Shutemov" X-Patchwork-Id: 4117 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4ac7:0:0:0:0:0 with SMTP id y7csp1910341wrs; Tue, 18 Oct 2022 04:39:48 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6FlAOTzqDRidJ6NPV+A3FwzXmopw4fb0xFa2zCUpgkVgKktCtBJlhcHfN+cwsHxt31Gwd3 X-Received: by 2002:a17:907:6ea1:b0:78d:4c16:a68b with SMTP id sh33-20020a1709076ea100b0078d4c16a68bmr2058823ejc.447.1666093187855; Tue, 18 Oct 2022 04:39:47 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666093187; cv=none; d=google.com; s=arc-20160816; b=psBNy0aIL1RS96UHXMt6DmzMfO0yxjKppeFI3a0BIGE6DThrarXz09YiYpaxbT8pg7 8yak/rvUP+rhzeL562igSLIlyZQLfYyhufBKg5cqsMCMGTLK6R88/TxAvb2V9p6Hi3Ol vCB8mkkV0g8PesgL4qaLDyuwN9UnXIpEqNToPan8fsL2UEircA01gpAuAxgKjvl91Pxn UJYJ2xzIlUBbkotI6AzDUp214+0PWOsfs4E33ox8/QjtVJ+TMUgWP3LStaVVeA8pUJ6h dCxygjRxDYx7n0JfdIxzptfu/bT9yo7ALdYIVQPLdDFjZSy2VyJSX++3O/1lCmBCrRye nxXg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=BGtjROjtrNGL89GNMOsNzSU9TtCi6Bm0JSMzAJygmWg=; b=eQLi1MCFMSYGzmRcTTqCbzr/bLQcpFlqaTmi07xv5wY/j461Fp9mqqw9eEzisksLs6 7nlGDvmzM5E4D7ZEFhonlRdsN+mHaz1gkiPLP1aev3oyJViHpJeCq+AuRBDqS7cBMyp6 vqch4oIMyXolvhkNs2rgn3hY9XTLanY02Lkf5LNObYR407kP4BG8U9mXP+Uj+8v+L50M NheFY+GLshOhGZSTpZK8e4llBU/kKhG7ClI0dfVzTb0JUHVbyWu7/J+8BfKaTAsr4NsY alsu2TJcpA8A1zpKpDICKrdotlBoCqw102yDmpJhtslgyA8kPIAAi1JcENhlsQOEAFjm 1dIw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=ie3iViYG; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id cn4-20020a0564020ca400b0045c4b1f4315si10467523edb.485.2022.10.18.04.39.22; Tue, 18 Oct 2022 04:39:47 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=ie3iViYG; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230261AbiJRLgf (ORCPT + 99 others); Tue, 18 Oct 2022 07:36:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60852 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229698AbiJRLfz (ORCPT ); Tue, 18 Oct 2022 07:35:55 -0400 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9373F4D4F3 for ; Tue, 18 Oct 2022 04:35:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1666092926; x=1697628926; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=40WXelYo9LYaLeJacp6rhf6Ge84QFQDH6MSN4czUvf8=; b=ie3iViYGe3Wrbuh5C520sMNyl5clkBGcbVbZSpNDtsdm5edWGHQWC0Xx Jgpf9Bzd4kNYqCVNsRBdKpVPmIwcTX5Xe08CWVp4CqIfMXBOe4Lp8v3xj Pg8YPBM0oLu2FxrlmYDfU4a8eDjvsnDV+Dk5RQaNL8Qfbomt6nV22Vx9H /G5jckVN3y695fbQixYJLP6eUtkjfjSf/5HR/srtaP2mbU3Glcdv5s6YO 1TTzqSwG2f78Hrg+BO8Zdjo24NneHpeAcp3NpNqYidcAYygVpRqn29UHZ g/dTCnXAqRthafQoW/pFfKIO+JIOhunlA2eO5Kkol6hRH+E9GSonZ0jk2 g==; X-IronPort-AV: E=McAfee;i="6500,9779,10503"; a="392382135" X-IronPort-AV: E=Sophos;i="5.95,193,1661842800"; d="scan'208";a="392382135" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Oct 2022 04:34:18 -0700 X-IronPort-AV: E=McAfee;i="6500,9779,10503"; a="661861186" X-IronPort-AV: E=Sophos;i="5.95,193,1661842800"; d="scan'208";a="661861186" Received: from vhavel-mobl.ger.corp.intel.com (HELO box.shutemov.name) ([10.252.51.115]) by orsmga001-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Oct 2022 04:34:13 -0700 Received: by box.shutemov.name (Postfix, from userid 1000) id 5A2A5104BA8; Tue, 18 Oct 2022 14:34:04 +0300 (+03) From: "Kirill A. Shutemov" To: Dave Hansen , Andy Lutomirski , Peter Zijlstra Cc: x86@kernel.org, Kostya Serebryany , Andrey Ryabinin , Andrey Konovalov , Alexander Potapenko , Taras Madan , Dmitry Vyukov , "H . J . Lu" , Andi Kleen , Rick Edgecombe , Bharata B Rao , Jacob Pan , Ashok Raj , linux-mm@kvack.org, linux-kernel@vger.kernel.org, "Kirill A. Shutemov" Subject: [PATCHv10 10/15] x86/mm, iommu/sva: Make LAM and SVM mutually exclusive Date: Tue, 18 Oct 2022 14:33:53 +0300 Message-Id: <20221018113358.7833-11-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.38.0 In-Reply-To: <20221018113358.7833-1-kirill.shutemov@linux.intel.com> References: <20221018113358.7833-1-kirill.shutemov@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747025330346102444?= X-GMAIL-MSGID: =?utf-8?q?1747025330346102444?= IOMMU and SVM-capable devices know nothing about LAM and only expect canonical addresses. Attempt to pass down tagged pointer will lead to address translation failure. By default do not allow to enable both LAM and use SVM in the same process. The new ARCH_FORCE_TAGGED_SVM arch_prctl() overrides the limitation. By using the arch_prctl() userspace takes responsibility to never pass tagged address to the device. Signed-off-by: Kirill A. Shutemov Reviewed-by: Ashok Raj --- arch/x86/include/asm/mmu.h | 6 ++++-- arch/x86/include/asm/mmu_context.h | 2 ++ arch/x86/include/uapi/asm/prctl.h | 1 + arch/x86/kernel/process_64.c | 13 +++++++++++++ drivers/iommu/iommu-sva-lib.c | 12 ++++++++++++ include/linux/mmu_context.h | 4 ++++ 6 files changed, 36 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/mmu.h b/arch/x86/include/asm/mmu.h index 2fdb390040b5..cce9b32b0d6d 100644 --- a/arch/x86/include/asm/mmu.h +++ b/arch/x86/include/asm/mmu.h @@ -9,9 +9,11 @@ #include /* Uprobes on this MM assume 32-bit code */ -#define MM_CONTEXT_UPROBE_IA32 BIT(0) +#define MM_CONTEXT_UPROBE_IA32 BIT(0) /* vsyscall page is accessible on this MM */ -#define MM_CONTEXT_HAS_VSYSCALL BIT(1) +#define MM_CONTEXT_HAS_VSYSCALL BIT(1) +/* Allow LAM and SVM coexisting */ +#define MM_CONTEXT_FORCE_TAGGED_SVM BIT(2) /* * x86 has arch-specific MMU state beyond what lives in mm_struct. diff --git a/arch/x86/include/asm/mmu_context.h b/arch/x86/include/asm/mmu_context.h index b0e9ea23758b..6b9ac2c60cec 100644 --- a/arch/x86/include/asm/mmu_context.h +++ b/arch/x86/include/asm/mmu_context.h @@ -113,6 +113,8 @@ static inline void mm_reset_untag_mask(struct mm_struct *mm) mm->context.untag_mask = -1UL; } +#define arch_pgtable_dma_compat(mm) \ + (!mm_lam_cr3_mask(mm) || (mm->context.flags & MM_CONTEXT_FORCE_TAGGED_SVM)) #else static inline unsigned long mm_lam_cr3_mask(struct mm_struct *mm) diff --git a/arch/x86/include/uapi/asm/prctl.h b/arch/x86/include/uapi/asm/prctl.h index a31e27b95b19..7bd22defb558 100644 --- a/arch/x86/include/uapi/asm/prctl.h +++ b/arch/x86/include/uapi/asm/prctl.h @@ -23,5 +23,6 @@ #define ARCH_GET_UNTAG_MASK 0x4001 #define ARCH_ENABLE_TAGGED_ADDR 0x4002 #define ARCH_GET_MAX_TAG_BITS 0x4003 +#define ARCH_FORCE_TAGGED_SVM 0x4004 #endif /* _ASM_X86_PRCTL_H */ diff --git a/arch/x86/kernel/process_64.c b/arch/x86/kernel/process_64.c index 9952e9f517ec..8faa8774bb93 100644 --- a/arch/x86/kernel/process_64.c +++ b/arch/x86/kernel/process_64.c @@ -783,6 +783,13 @@ static int prctl_enable_tagged_addr(struct mm_struct *mm, unsigned long nr_bits) goto out; } +#ifdef CONFIG_IOMMU_SVA + if (pasid_valid(mm->pasid) && + !(mm->context.flags & MM_CONTEXT_FORCE_TAGGED_SVM)) { + ret = -EBUSY; + goto out; + } +#endif if (!nr_bits) { ret = -EINVAL; goto out; @@ -893,6 +900,12 @@ long do_arch_prctl_64(struct task_struct *task, int option, unsigned long arg2) (unsigned long __user *)arg2); case ARCH_ENABLE_TAGGED_ADDR: return prctl_enable_tagged_addr(task->mm, arg2); + case ARCH_FORCE_TAGGED_SVM: + if (mmap_write_lock_killable(task->mm)) + return -EINTR; + task->mm->context.flags |= MM_CONTEXT_FORCE_TAGGED_SVM; + mmap_write_unlock(task->mm); + return 0; case ARCH_GET_MAX_TAG_BITS: if (!cpu_feature_enabled(X86_FEATURE_LAM)) return put_user(0, (unsigned long __user *)arg2); diff --git a/drivers/iommu/iommu-sva-lib.c b/drivers/iommu/iommu-sva-lib.c index 106506143896..593ae2472e2c 100644 --- a/drivers/iommu/iommu-sva-lib.c +++ b/drivers/iommu/iommu-sva-lib.c @@ -2,6 +2,8 @@ /* * Helpers for IOMMU drivers implementing SVA */ +#include +#include #include #include @@ -31,6 +33,15 @@ int iommu_sva_alloc_pasid(struct mm_struct *mm, ioasid_t min, ioasid_t max) min == 0 || max < min) return -EINVAL; + /* Serialize against address tagging enabling */ + if (mmap_write_lock_killable(mm)) + return -EINTR; + + if (!arch_pgtable_dma_compat(mm)) { + mmap_write_unlock(mm); + return -EBUSY; + } + mutex_lock(&iommu_sva_lock); /* Is a PASID already associated with this mm? */ if (pasid_valid(mm->pasid)) { @@ -46,6 +57,7 @@ int iommu_sva_alloc_pasid(struct mm_struct *mm, ioasid_t min, ioasid_t max) mm_pasid_set(mm, pasid); out: mutex_unlock(&iommu_sva_lock); + mmap_write_unlock(mm); return ret; } EXPORT_SYMBOL_GPL(iommu_sva_alloc_pasid); diff --git a/include/linux/mmu_context.h b/include/linux/mmu_context.h index b9b970f7ab45..115e2b518079 100644 --- a/include/linux/mmu_context.h +++ b/include/linux/mmu_context.h @@ -28,4 +28,8 @@ static inline void leave_mm(int cpu) { } # define task_cpu_possible(cpu, p) cpumask_test_cpu((cpu), task_cpu_possible_mask(p)) #endif +#ifndef arch_pgtable_dma_compat +#define arch_pgtable_dma_compat(mm) true +#endif + #endif