From patchwork Fri May 12 23:57:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Collingbourne X-Patchwork-Id: 93442 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp5471634vqo; Fri, 12 May 2023 17:12:58 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5+vMsvLwMSEuRVh7l7Vj9dQ3vaT4nXDkosmMkWtdWlGqsEi4CCyxv9qRS4Yw6r4ZBu5AQw X-Received: by 2002:a05:6a00:24d1:b0:63f:ffd:5360 with SMTP id d17-20020a056a0024d100b0063f0ffd5360mr40997762pfv.21.1683936778042; Fri, 12 May 2023 17:12:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683936778; cv=none; d=google.com; s=arc-20160816; b=RF4z8yNFW/WnwuX9wRDjcIpsvmEdaO014rsivMkni3KqNrnV62TIIlqUt3A2yrD29n N3+ejO3RE52d8RtrJnrPP96fAj7W+7QACANT1Bw2l+FJ6DMJ1WpQ4DA0GnNso8pDnasN McroDZD3RpqAln7odiL7d1ymh7RT8LtKMVgwNhMblrjI1zNptveUjkO99o7BG8Mdb9xk J0PstiFe10QMSWufXpQN2MVHvaPsQ+Sw2+U0V+k0JW5PAPDwk10ADkgwBUxITTi2uyBl UjPZwXgVNKz/MZ7nvl6DDTJR8bivkGX6oH9rWiAfBWtitaX76ImXKKroAfYwfqPzVJmm vOug== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:references:mime-version :message-id:in-reply-to:date:dkim-signature; bh=LVFc+Brp/nv5bcVUuCiFrcDkBMn4WnVlAUQLAdw9K1k=; b=Yra9JvXzC+HZgyLvvuCWi4vyu9MHdv2ApujQbrcvQNRTxBg6d9vJhi0XcSUNauaFhX fsosmtTqFp3gKtvmbpgt3UQz+euJTuhYs1u9M4nu3nug02YztELTWBPuNGvJuBqrogPw oYiB2iYRWTfynRGFY1sbaoD7cTF8m9UxAfrJFJOv0ugX0aL2zDwBzbc+Gi8sA61eOaTF gAIvNfIHGxsmDERtCxisULySqtPH2bapctKlesFH6y1bikKYIF51BgV849Mz7nB90N9I KSYthl+j2F5UntlQGGvZC497uWbfbLlcEhZuC0mYT2aHKihMYMXuqfvfhNKBhifB+7U8 IKQA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b="f5hclik/"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id s28-20020a63525c000000b00519c3475f12si10101623pgl.572.2023.05.12.17.12.43; Fri, 12 May 2023 17:12:58 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b="f5hclik/"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241169AbjELX6u (ORCPT + 99 others); Fri, 12 May 2023 19:58:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60850 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240973AbjELX6n (ORCPT ); Fri, 12 May 2023 19:58:43 -0400 Received: from mail-yw1-x114a.google.com (mail-yw1-x114a.google.com [IPv6:2607:f8b0:4864:20::114a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4EF2035A6 for ; Fri, 12 May 2023 16:58:16 -0700 (PDT) Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-560def4d06dso70288487b3.3 for ; Fri, 12 May 2023 16:58:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1683935895; x=1686527895; h=cc:to:from:subject:references:mime-version:message-id:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=LVFc+Brp/nv5bcVUuCiFrcDkBMn4WnVlAUQLAdw9K1k=; b=f5hclik/Ew6KC4whYAG5VntaAWwKmaBWXX4bCpgRdYjBpbSj5Apr3PkSoK7/QAJd0N RpHxTIRW/bLimRvcbUmZjobDE/Eutpj6tvgC/Rdg9nOt+ZvW2UeGjSSNnpNfchKgE9qV bfsrP2dthFHNMRT6c5aCQw0Ff/JTyx0DIAtRow1fvLqu2M2UZFqFRteLQ4IPrCBhdwYv ZUWPKO0rlZI6hiVGzQF5WVmuVs1EGnbfoUeE09/HLSnj37gvPWFw6+cmVnJFfHK5WbAH YWYFfgDih13LoVhaHD+4rYuds+BE6rKkLJ/AoCA0fp0U2j35styXF7xvUDDBUZNd5sjd vJuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683935895; x=1686527895; h=cc:to:from:subject:references:mime-version:message-id:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=LVFc+Brp/nv5bcVUuCiFrcDkBMn4WnVlAUQLAdw9K1k=; b=TbtE1shXhTgYQqgtGw3JnujLVp6gsuE93Ypym+2/MU6oI4QMHVk+Pj/lYhAMfxcKfC 0a277DIEk5MkdmXPrzRK/jBQ9Jgbzcyw+M4Vmtns5PHV1mu42vFq49oO349536OVeDey tfzSmfQ6x9JrVSR/6s+oteridDCpNfJtV18KeZrhnWYtXfnLDbDX3woiRKrEybp7QOzy TXzwQXTjMbJLodW7bm7rZctqW71jcWaqeTVfoRTTlb1q40TxT+LUw2J6PAr95GNJL16Q KQQAfV7GrIp/3ldwvrrdNabg8f+MDT7lQXccSjF04zYXstsuQaL4tbJXstILq4vzioJg k6fg== X-Gm-Message-State: AC+VfDyUX5PtylCgjMh/fE8IeE9mq1mCdTV7a5uD62RaB5YPwLGDRcn9 PrqyK/gJJyIbxhnFLx1dpGSOLoU= X-Received: from pcc-desktop.svl.corp.google.com ([2620:15c:2d3:205:ff6:108b:739d:6a1c]) (user=pcc job=sendgmr) by 2002:a81:b285:0:b0:559:f1b0:6eb with SMTP id q127-20020a81b285000000b00559f1b006ebmr16091480ywh.4.1683935894947; Fri, 12 May 2023 16:58:14 -0700 (PDT) Date: Fri, 12 May 2023 16:57:50 -0700 In-Reply-To: <20230512235755.1589034-1-pcc@google.com> Message-Id: <20230512235755.1589034-2-pcc@google.com> Mime-Version: 1.0 References: <20230512235755.1589034-1-pcc@google.com> X-Mailer: git-send-email 2.40.1.606.ga4b1b128d6-goog Subject: [PATCH 1/3] mm: Move arch_do_swap_page() call to before swap_free() From: Peter Collingbourne To: Catalin Marinas Cc: Peter Collingbourne , " =?utf-8?b?UXVuLXdlaSBMaW4gKA==?= =?utf-8?b?5p6X576k5bS0KQ==?= " , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, "surenb@google.com" , "david@redhat.com" , " =?utf-8?b?Q2hpbndlbiBDaGFuZyAo5by1?= =?utf-8?b?6Yym5paHKQ==?= " , "kasan-dev@googlegroups.com" , " =?utf-8?b?S3Vh?= =?utf-8?b?bi1ZaW5nIExlZSAo5p2O5Yag56mOKQ==?= " , " =?utf-8?b?Q2FzcGVyIExpICjmnY7kuK3mpq4p?= " , "gregkh@linuxfoundation.org" , vincenzo.frascino@arm.com, Alexandru Elisei , will@kernel.org, eugenis@google.com, Steven Price , stable@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED, USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1765735690992259019?= X-GMAIL-MSGID: =?utf-8?q?1765735690992259019?= Commit c145e0b47c77 ("mm: streamline COW logic in do_swap_page()") moved the call to swap_free() before the call to set_pte_at(), which meant that the MTE tags could end up being freed before set_pte_at() had a chance to restore them. One other possibility was to hook arch_do_swap_page(), but this had a number of problems: - The call to the hook was also after swap_free(). - The call to the hook was after the call to set_pte_at(), so there was a racy window where uninitialized metadata may be exposed to userspace. This likely also affects SPARC ADI, which implements this hook to restore tags. - As a result of commit 1eba86c096e3 ("mm: change page type prior to adding page table entry"), we were also passing the new PTE as the oldpte argument, preventing the hook from knowing the swap index. Fix all of these problems by moving the arch_do_swap_page() call before the call to free_page(), and ensuring that we do not set orig_pte until after the call. Signed-off-by: Peter Collingbourne Suggested-by: Catalin Marinas Link: https://linux-review.googlesource.com/id/I6470efa669e8bd2f841049b8c61020c510678965 Cc: # 6.1 Fixes: ca827d55ebaa ("mm, swap: Add infrastructure for saving page metadata on swap") Fixes: 1eba86c096e3 ("mm: change page type prior to adding page table entry") --- mm/memory.c | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 01a23ad48a04..83268d287ff1 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3914,19 +3914,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) } } - /* - * Remove the swap entry and conditionally try to free up the swapcache. - * We're already holding a reference on the page but haven't mapped it - * yet. - */ - swap_free(entry); - if (should_try_to_free_swap(folio, vma, vmf->flags)) - folio_free_swap(folio); - - inc_mm_counter(vma->vm_mm, MM_ANONPAGES); - dec_mm_counter(vma->vm_mm, MM_SWAPENTS); pte = mk_pte(page, vma->vm_page_prot); - /* * Same logic as in do_wp_page(); however, optimize for pages that are * certainly not shared either because we just allocated them without @@ -3946,8 +3934,21 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) pte = pte_mksoft_dirty(pte); if (pte_swp_uffd_wp(vmf->orig_pte)) pte = pte_mkuffd_wp(pte); + arch_do_swap_page(vma->vm_mm, vma, vmf->address, pte, vmf->orig_pte); vmf->orig_pte = pte; + /* + * Remove the swap entry and conditionally try to free up the swapcache. + * We're already holding a reference on the page but haven't mapped it + * yet. + */ + swap_free(entry); + if (should_try_to_free_swap(folio, vma, vmf->flags)) + folio_free_swap(folio); + + inc_mm_counter(vma->vm_mm, MM_ANONPAGES); + dec_mm_counter(vma->vm_mm, MM_SWAPENTS); + /* ksm created a completely new copy */ if (unlikely(folio != swapcache && swapcache)) { page_add_new_anon_rmap(page, vma, vmf->address); @@ -3959,7 +3960,6 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) VM_BUG_ON(!folio_test_anon(folio) || (pte_write(pte) && !PageAnonExclusive(page))); set_pte_at(vma->vm_mm, vmf->address, vmf->pte, pte); - arch_do_swap_page(vma->vm_mm, vma, vmf->address, pte, vmf->orig_pte); folio_unlock(folio); if (folio != swapcache && swapcache) { From patchwork Fri May 12 23:57:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Peter Collingbourne X-Patchwork-Id: 93440 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp5469231vqo; Fri, 12 May 2023 17:07:30 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6/YjzfR6eHW8iCnG7pYBgRtN/V+tNNAiLF6xpFE9pCOgpvd3bNtNJ65SNB8l7SLQzNiHgc X-Received: by 2002:a05:6a20:160a:b0:103:558c:513 with SMTP id l10-20020a056a20160a00b00103558c0513mr13891654pzj.57.1683936450619; Fri, 12 May 2023 17:07:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683936450; cv=none; d=google.com; s=arc-20160816; b=1IaVc+Hu9u7q8yrdaOTVJ1TzhgEGHDuIl3gE0lAp+Bt+/QToke1p//xxjtKHsScn/x btMqafAyxrmIlh10BJ/s03lmFyvNg3Xfz4IN4QV7uky9pdlu/3H9Kru+jDn8hrvFVhTA gj+KPQxuhlJy5qpLD4hLO3aXxK+VJWD6lLrYh3OXUY/ShDxHGLCU8MiuLEq+VS8mSDdy XKSBMd/hEqhStzwrgfBdj5MIRwgqabOeG0fDN+qmsvPr0SX5Wv0nN5qfMIUTblSYSy7B IW1jc52fx50y0hAJlcfi9kyhzX8+FZQwLotdBQU5HszrO8shTijw3hm7Tu6y6VsGH4Wt 7pxQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:cc:to:from:subject :references:mime-version:message-id:in-reply-to:date:dkim-signature; bh=XwMV83qzB00xUe1xka9Kex2X8Ec2jkmSilor+2INk6I=; b=CHU45eYHPgEGeNZcBGNBPN5CYsqze3wjNMMNeUQrsLaoEGRd3EF4TrwL6PHxoe3hCL zWuqtH60e9GirFWYYlPppCh9KgOq8Ho9EHZ+wwakT05k0EwE7UaKKhKs23QRctkA62EC v0MceNfjl0yZ+/Z4SI2L2qRPd6aC91K40xAq0yBVhaA03sSAKtLXaIzRSWxbVevSEXG0 tbn1sUEOK9ZFrL7File1aIMdt2OQIF2wSF1dw/3umL03xusA63x86YjDPiHg39XtmQxz MtyrHEjauLmK4UbglrzB9tPNtgeJFmaPVeSUZE3YRmhxrQfowL/WkB8ukOTtYtg/rhFe CZHg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=xk2Ibde3; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id b5-20020a63eb45000000b0052cc0b8d59dsi10340966pgk.798.2023.05.12.17.07.06; Fri, 12 May 2023 17:07:30 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=xk2Ibde3; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241139AbjELX7A (ORCPT + 99 others); Fri, 12 May 2023 19:59:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60718 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241005AbjELX6s (ORCPT ); Fri, 12 May 2023 19:58:48 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A69DB40F2 for ; Fri, 12 May 2023 16:58:22 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-b8f324b3ef8so13026032276.0 for ; Fri, 12 May 2023 16:58:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1683935897; x=1686527897; h=content-transfer-encoding:cc:to:from:subject:references :mime-version:message-id:in-reply-to:date:from:to:cc:subject:date :message-id:reply-to; bh=XwMV83qzB00xUe1xka9Kex2X8Ec2jkmSilor+2INk6I=; b=xk2Ibde3e8u/s1FXz5MjnaikjmFzTVuGbTz673xLExz96TfA9SY9JBEhrH4pDcD4kc lJcaoTYG0RcJgWFZFSCaY33HyZj2N01rZHV90M6ATudVtPlhyiI8LL94cSLE8hlDZsK1 ohmUhbm4cTQwXS7bDVKRWaW81vYw/Zs2XgvQtpjyOpS39KbdeTP0PlNP0LVOEJX0BiKd Gp/2PGuqyiqk4rIboKMupJ2lV3A59cCft+4hG9Pnr7dP/E1pUxUHHsAIktr3S1az7xY9 Y7C7SJ3J766Sa8Y3zSYkAveN2OM091uDtEnFY90+chuUo5dWOIKtYgSwr2SEMxwfaVJk l+/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683935897; x=1686527897; h=content-transfer-encoding:cc:to:from:subject:references :mime-version:message-id:in-reply-to:date:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=XwMV83qzB00xUe1xka9Kex2X8Ec2jkmSilor+2INk6I=; b=VSNaepKceXMIdZTLUTCkL6YsH4hLOKPwp9+Mks7qrlANAMWOJTogFdPKkB677t5mX7 dm+kGAk2E47sN2vt1VZDgKMVYWB9XUXaJ/WOugoJO1Ur5ie0zNMGJMMq54uvFH/wy3pS x5FrvZXF2ja5yx9L+ZITfn3wDwjPL/RjxPC6OFi0L9cTUakr/znkZugKa3bWk3zJMTw6 hqqWjZU4gRiK3SVDe/G/lxeeu2RIG1G/8ADrbsSCjskBRTzaaDjbY1maSHNnqvDdzLc3 iXZgQjCiID4F2VAbKBcgldhceFOsbs1+eyL+Sgn7QyRAi4/vXfTIakatbMAlS2VdbE6w PHpQ== X-Gm-Message-State: AC+VfDyx3sVa55KpTf76ADUOqF504xwKKndEPy3hYFMoHi+56+nXD94n 7vB6vsNwjOcEklW3XNu9+NWO+Hc= X-Received: from pcc-desktop.svl.corp.google.com ([2620:15c:2d3:205:ff6:108b:739d:6a1c]) (user=pcc job=sendgmr) by 2002:a25:d18a:0:b0:ba7:29a9:a471 with SMTP id i132-20020a25d18a000000b00ba729a9a471mr1565248ybg.0.1683935897117; Fri, 12 May 2023 16:58:17 -0700 (PDT) Date: Fri, 12 May 2023 16:57:51 -0700 In-Reply-To: <20230512235755.1589034-1-pcc@google.com> Message-Id: <20230512235755.1589034-3-pcc@google.com> Mime-Version: 1.0 References: <20230512235755.1589034-1-pcc@google.com> X-Mailer: git-send-email 2.40.1.606.ga4b1b128d6-goog Subject: [PATCH 2/3] mm: Call arch_swap_restore() from arch_do_swap_page() and deprecate the latter From: Peter Collingbourne To: Catalin Marinas Cc: Peter Collingbourne , " =?utf-8?b?UXVuLXdlaSBMaW4gKA==?= =?utf-8?b?5p6X576k5bS0KQ==?= " , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, "surenb@google.com" , "david@redhat.com" , " =?utf-8?b?Q2hpbndlbiBDaGFuZyAo5by1?= =?utf-8?b?6Yym5paHKQ==?= " , "kasan-dev@googlegroups.com" , " =?utf-8?b?S3Vh?= =?utf-8?b?bi1ZaW5nIExlZSAo5p2O5Yag56mOKQ==?= " , " =?utf-8?b?Q2FzcGVyIExpICjmnY7kuK3mpq4p?= " , "gregkh@linuxfoundation.org" , vincenzo.frascino@arm.com, Alexandru Elisei , will@kernel.org, eugenis@google.com, Steven Price , stable@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED, USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1765735347858755638?= X-GMAIL-MSGID: =?utf-8?q?1765735347858755638?= The previous patch made it possible for MTE to restore tags before they are freed by hooking arch_do_swap_page(). However, the arch_do_swap_page() hook API is incompatible with swap restoration in circumstances where we do not have an mm or a vma, such as swapoff with swapped out shmem, and I expect that ADI will currently fail to restore tags in these circumstances. This implies that arch-specific metadata stores ought to be indexed by swap index, as MTE does, rather than by mm and vma, as ADI does, and we should discourage hooking arch_do_swap_page(), preferring to hook arch_swap_restore() instead, as MTE already does. Therefore, instead of directly hooking arch_do_swap_page() for MTE, deprecate that hook, change its default implementation to call arch_swap_restore() and rely on the existing implementation of the latter for MTE. Fixes: c145e0b47c77 ("mm: streamline COW logic in do_swap_page()") Link: https://linux-review.googlesource.com/id/Id2f1ad76eaf606ae210e1d2dd0b7fe287e5f7d87 Signed-off-by: Peter Collingbourne Reported-by: Qun-wei Lin (林群崴) Link: https://lore.kernel.org/all/5050805753ac469e8d727c797c2218a9d780d434.camel@mediatek.com/ Cc: # 6.1 --- include/linux/pgtable.h | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h index c63cd44777ec..fc0259cf60fb 100644 --- a/include/linux/pgtable.h +++ b/include/linux/pgtable.h @@ -740,6 +740,12 @@ static inline int pgd_same(pgd_t pgd_a, pgd_t pgd_b) set_pgd(pgdp, pgd); \ }) +#ifndef __HAVE_ARCH_SWAP_RESTORE +static inline void arch_swap_restore(swp_entry_t entry, struct folio *folio) +{ +} +#endif + #ifndef __HAVE_ARCH_DO_SWAP_PAGE /* * Some architectures support metadata associated with a page. When a @@ -748,14 +754,14 @@ static inline int pgd_same(pgd_t pgd_a, pgd_t pgd_b) * processors support an ADI (Application Data Integrity) tag for the * page as metadata for the page. arch_do_swap_page() can restore this * metadata when a page is swapped back in. + * + * This hook is deprecated. Architectures should hook arch_swap_restore() + * instead, because this hook is not called on all code paths that can + * swap in a page, particularly those where mm and vma are not available + * (e.g. swapoff for shmem pages). */ -static inline void arch_do_swap_page(struct mm_struct *mm, - struct vm_area_struct *vma, - unsigned long addr, - pte_t pte, pte_t oldpte) -{ - -} +#define arch_do_swap_page(mm, vma, addr, pte, oldpte) \ + arch_swap_restore(pte_to_swp_entry(oldpte), page_folio(pte_page(pte))) #endif #ifndef __HAVE_ARCH_UNMAP_ONE @@ -798,12 +804,6 @@ static inline void arch_swap_invalidate_area(int type) } #endif -#ifndef __HAVE_ARCH_SWAP_RESTORE -static inline void arch_swap_restore(swp_entry_t entry, struct folio *folio) -{ -} -#endif - #ifndef __HAVE_ARCH_PGD_OFFSET_GATE #define pgd_offset_gate(mm, addr) pgd_offset(mm, addr) #endif From patchwork Fri May 12 23:57:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Collingbourne X-Patchwork-Id: 93441 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp5469985vqo; Fri, 12 May 2023 17:09:16 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6lQyCFsdpd2EUFS7IzTbKBweAqsXgGd7PeZiT6gqp0PhyBGs1djQMBaFYHyE+w8rKKCZg0 X-Received: by 2002:a05:6a00:1ad1:b0:648:fcfa:cc14 with SMTP id f17-20020a056a001ad100b00648fcfacc14mr12175412pfv.21.1683936556136; Fri, 12 May 2023 17:09:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683936556; cv=none; d=google.com; s=arc-20160816; b=iNRr5BB0X73lZIocl5Kx0b9/9yR79InBVaBhPuMvjlJh2eC9e8qAWJNJpn3ykKeqTE CdwrepXSv3VW43ZeF/30rWMVhqCzGzudGuHxQmeUL06VSjlHrBQBY+lL9M19UP83EbDV tnAYegIDd6TyhDwKoa+gUamEPtXkRjxrUs/pPdF9KVnDEhgBTpTMdI85J+E/J+cHxmd8 8y64NLYS8KRCsVmQvnIB8r/BaAYAU20Hdkk9we6PcANE3Nwqd0KobpfD7X9ihF5HZ0G7 BhmoiSLKoHFzdX6QZ9S/4PB4lgLvtJZlgs8yJike45LuTJq9W+SUvAPvFn965Wt16Nbd U4cg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:references:mime-version :message-id:in-reply-to:date:dkim-signature; bh=PFB1dkpGULUqWzGVGpWTfZD/wGkLnLUL2yYSj9UHJe8=; b=A7See4YKniZosKaIkHspjHtZmc0zEb/1P89svECDbtXBOshkEhyAQT7SHijmFdF5na XQAGhlFtJAtjAFzlCDhwGAW6rg9d167PbRoM+UNMYELKv8cHrbR9hU/F2mkLKgo5/Tzp XMd+qCZwd/55K99e9il7o/L0ENiuJ8vsoHWcEjxiJZUqYvVMTL2olfzh4DQpeXdGBb1r pn1fwte6qgqV2YhzktuF4oFdLs/1tFnAU9MTFBwVPJxke2UApItO6keccBHH8N97fl6I wT/3MMbTuatQausb7i5MoTE+shMI1Nn2mLxMPjYLhDhGSy9bYjmeDeM+xyL3sZ7YVh+B OOZg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=RfEuSDTL; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id b5-20020a63eb45000000b0052cc0b8d59dsi10340966pgk.798.2023.05.12.17.09.03; Fri, 12 May 2023 17:09:16 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=RfEuSDTL; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241085AbjELX7D (ORCPT + 99 others); Fri, 12 May 2023 19:59:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60818 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241010AbjELX6t (ORCPT ); Fri, 12 May 2023 19:58:49 -0400 Received: from mail-pg1-x54a.google.com (mail-pg1-x54a.google.com [IPv6:2607:f8b0:4864:20::54a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 05CBA35B6 for ; Fri, 12 May 2023 16:58:24 -0700 (PDT) Received: by mail-pg1-x54a.google.com with SMTP id 41be03b00d2f7-517bad1b8c5so9902145a12.0 for ; Fri, 12 May 2023 16:58:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1683935899; x=1686527899; h=cc:to:from:subject:references:mime-version:message-id:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=PFB1dkpGULUqWzGVGpWTfZD/wGkLnLUL2yYSj9UHJe8=; b=RfEuSDTLMnNvVhco/REROey0gb2x3qNZHS7QsUUEezpvjObhi40SJ4Ivhu9qrO+zn9 KaL54Or+583uthg14uV25Du+x+MjKeSsXpP0N8kbyqmOimQ32HKrIuItJhzMEe7Z7r00 H3xPmIPgJeHVZwbgHFPY4EK7ygVj2/eXjW5vntomlpbdw4u2rW1l7DywwTG+JbEDqlqW a2pVysEP1gPh7M/+7mMP5dvhKaBf8GtLDWx16ApQeX5HQjqiHJ5Sq4w6ZHe7QyWEHes7 xcFButTqpKqyg6mne6Xbpv8LV/kx6mv3uBW1s8FPhdpT8SGS/NjZYbwltNiI2b33sVlg VO5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683935899; x=1686527899; h=cc:to:from:subject:references:mime-version:message-id:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=PFB1dkpGULUqWzGVGpWTfZD/wGkLnLUL2yYSj9UHJe8=; b=gg2SjV1rTsfoz7ZEM/NsRYWXrmUUoZeUiip2wzc1roMpRYtrM0flLpMvCeJbqn9bo4 vNxzSp3GGjxQW5Ui1IfoXRrfGkKcczT/kG8bFlrJy+dnDrYe/iKlKU/oSFHk4ftnwepc /qMQ3ddwYwOGR75Knsd66IJ1b2JLmnC3QGGZMQHY62FCTRcImxejQJZnIUydQPAzkP6R pEu8BAb94WP2Kf3hagyMn5JWSvHElHiJUir/ij/urm212tcdK5lUop6WjXzt8F3F8tQD GAQ6iAQaUVtAoTDO7qWmGb3uD3Sd6rXZcbgiIZOu6BoK+CEjMwUjo5l8kbDwtK6UUaGG TBMw== X-Gm-Message-State: AC+VfDzz14Kk21F5Ez0x7KN/2wH9Faai3k3uo1IriTsFj152Pl5Aquu7 AsE+a+yTOMZiQbruaSTBxKl/bnI= X-Received: from pcc-desktop.svl.corp.google.com ([2620:15c:2d3:205:ff6:108b:739d:6a1c]) (user=pcc job=sendgmr) by 2002:a63:151:0:b0:52c:6149:f6be with SMTP id 78-20020a630151000000b0052c6149f6bemr7437419pgb.4.1683935899323; Fri, 12 May 2023 16:58:19 -0700 (PDT) Date: Fri, 12 May 2023 16:57:52 -0700 In-Reply-To: <20230512235755.1589034-1-pcc@google.com> Message-Id: <20230512235755.1589034-4-pcc@google.com> Mime-Version: 1.0 References: <20230512235755.1589034-1-pcc@google.com> X-Mailer: git-send-email 2.40.1.606.ga4b1b128d6-goog Subject: [PATCH 3/3] arm64: mte: Simplify swap tag restoration logic and fix uninitialized tag issue From: Peter Collingbourne To: Catalin Marinas Cc: Peter Collingbourne , " =?utf-8?b?UXVuLXdlaSBMaW4gKA==?= =?utf-8?b?5p6X576k5bS0KQ==?= " , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, "surenb@google.com" , "david@redhat.com" , " =?utf-8?b?Q2hpbndlbiBDaGFuZyAo5by1?= =?utf-8?b?6Yym5paHKQ==?= " , "kasan-dev@googlegroups.com" , " =?utf-8?b?S3Vh?= =?utf-8?b?bi1ZaW5nIExlZSAo5p2O5Yag56mOKQ==?= " , " =?utf-8?b?Q2FzcGVyIExpICjmnY7kuK3mpq4p?= " , "gregkh@linuxfoundation.org" , vincenzo.frascino@arm.com, Alexandru Elisei , will@kernel.org, eugenis@google.com, Steven Price , stable@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED, USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1765735458378243517?= X-GMAIL-MSGID: =?utf-8?q?1765735458378243517?= As a result of the previous two patches, there are no circumstances in which a swapped-in page is installed in a page table without first having arch_swap_restore() called on it. Therefore, we no longer need the logic in set_pte_at() that restores the tags, so remove it. Because we can now rely on the page being locked, we no longer need to handle the case where a page is having its tags restored by multiple tasks concurrently, so we can slightly simplify the logic in mte_restore_tags(). This patch also fixes an issue where a page can have PG_mte_tagged set with uninitialized tags. The issue is that the mte_sync_page_tags() function sets PG_mte_tagged if it initializes page tags. Then we return to mte_sync_tags(), which sets PG_mte_tagged again. At best, this is redundant. However, it is possible for mte_sync_page_tags() to return without having initialized tags for the page, i.e. in the case where check_swap is true (non-compound page), is_swap_pte(old_pte) is false and pte_is_tagged is false. So at worst, we set PG_mte_tagged on a page with uninitialized tags. This can happen if, for example, page migration causes a PTE for an untagged page to be replaced. If the userspace program subsequently uses mprotect() to enable PROT_MTE for that page, the uninitialized tags will be exposed to userspace. Signed-off-by: Peter Collingbourne Link: https://linux-review.googlesource.com/id/I8ad54476f3b2d0144ccd8ce0c1d7a2963e5ff6f3 Fixes: e059853d14ca ("arm64: mte: Fix/clarify the PG_mte_tagged semantics") Cc: # 6.1 --- The Fixes: tag (and the commit message in general) are written assuming that this patch is landed in a maintainer tree instead of "arm64: mte: Do not set PG_mte_tagged if tags were not initialized". arch/arm64/include/asm/mte.h | 4 ++-- arch/arm64/include/asm/pgtable.h | 14 ++------------ arch/arm64/kernel/mte.c | 32 +++----------------------------- arch/arm64/mm/mteswap.c | 7 +++---- 4 files changed, 10 insertions(+), 47 deletions(-) diff --git a/arch/arm64/include/asm/mte.h b/arch/arm64/include/asm/mte.h index 20dd06d70af5..dfea486a6a85 100644 --- a/arch/arm64/include/asm/mte.h +++ b/arch/arm64/include/asm/mte.h @@ -90,7 +90,7 @@ static inline bool try_page_mte_tagging(struct page *page) } void mte_zero_clear_page_tags(void *addr); -void mte_sync_tags(pte_t old_pte, pte_t pte); +void mte_sync_tags(pte_t pte); void mte_copy_page_tags(void *kto, const void *kfrom); void mte_thread_init_user(void); void mte_thread_switch(struct task_struct *next); @@ -122,7 +122,7 @@ static inline bool try_page_mte_tagging(struct page *page) static inline void mte_zero_clear_page_tags(void *addr) { } -static inline void mte_sync_tags(pte_t old_pte, pte_t pte) +static inline void mte_sync_tags(pte_t pte) { } static inline void mte_copy_page_tags(void *kto, const void *kfrom) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index b6ba466e2e8a..efdf48392026 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -337,18 +337,8 @@ static inline void __set_pte_at(struct mm_struct *mm, unsigned long addr, * don't expose tags (instruction fetches don't check tags). */ if (system_supports_mte() && pte_access_permitted(pte, false) && - !pte_special(pte)) { - pte_t old_pte = READ_ONCE(*ptep); - /* - * We only need to synchronise if the new PTE has tags enabled - * or if swapping in (in which case another mapping may have - * set tags in the past even if this PTE isn't tagged). - * (!pte_none() && !pte_present()) is an open coded version of - * is_swap_pte() - */ - if (pte_tagged(pte) || (!pte_none(old_pte) && !pte_present(old_pte))) - mte_sync_tags(old_pte, pte); - } + !pte_special(pte) && pte_tagged(pte)) + mte_sync_tags(pte); __check_safe_pte_update(mm, ptep, pte); diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c index f5bcb0dc6267..c40728046fed 100644 --- a/arch/arm64/kernel/mte.c +++ b/arch/arm64/kernel/mte.c @@ -35,41 +35,15 @@ DEFINE_STATIC_KEY_FALSE(mte_async_or_asymm_mode); EXPORT_SYMBOL_GPL(mte_async_or_asymm_mode); #endif -static void mte_sync_page_tags(struct page *page, pte_t old_pte, - bool check_swap, bool pte_is_tagged) -{ - if (check_swap && is_swap_pte(old_pte)) { - swp_entry_t entry = pte_to_swp_entry(old_pte); - - if (!non_swap_entry(entry)) - mte_restore_tags(entry, page); - } - - if (!pte_is_tagged) - return; - - if (try_page_mte_tagging(page)) { - mte_clear_page_tags(page_address(page)); - set_page_mte_tagged(page); - } -} - -void mte_sync_tags(pte_t old_pte, pte_t pte) +void mte_sync_tags(pte_t pte) { struct page *page = pte_page(pte); long i, nr_pages = compound_nr(page); - bool check_swap = nr_pages == 1; - bool pte_is_tagged = pte_tagged(pte); - - /* Early out if there's nothing to do */ - if (!check_swap && !pte_is_tagged) - return; /* if PG_mte_tagged is set, tags have already been initialised */ for (i = 0; i < nr_pages; i++, page++) { - if (!page_mte_tagged(page)) { - mte_sync_page_tags(page, old_pte, check_swap, - pte_is_tagged); + if (try_page_mte_tagging(page)) { + mte_clear_page_tags(page_address(page)); set_page_mte_tagged(page); } } diff --git a/arch/arm64/mm/mteswap.c b/arch/arm64/mm/mteswap.c index cd508ba80ab1..3a78bf1b1364 100644 --- a/arch/arm64/mm/mteswap.c +++ b/arch/arm64/mm/mteswap.c @@ -53,10 +53,9 @@ void mte_restore_tags(swp_entry_t entry, struct page *page) if (!tags) return; - if (try_page_mte_tagging(page)) { - mte_restore_page_tags(page_address(page), tags); - set_page_mte_tagged(page); - } + WARN_ON_ONCE(!try_page_mte_tagging(page)); + mte_restore_page_tags(page_address(page), tags); + set_page_mte_tagged(page); } void mte_invalidate_tags(int type, pgoff_t offset)