From patchwork Fri Nov 4 22:00:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 15811 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp659641wru; Fri, 4 Nov 2022 15:02:22 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5cWRfZqVcm9SdqOPlfduBxvU4M/zsp1V0u+KmYxSDJeg0VfsrXLqKdgIkEVfC/44ilMHDj X-Received: by 2002:a63:2f45:0:b0:457:dc63:68b4 with SMTP id v66-20020a632f45000000b00457dc6368b4mr32100534pgv.228.1667599342100; Fri, 04 Nov 2022 15:02:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667599342; cv=none; d=google.com; s=arc-20160816; b=E85HjYLQgCK55aQVXbuK6Vj4/jhIn/gkxHjLdackxj1+ul+4PdEp4P+c/kolyCiNuq TlV4RsitEW0AUu2S1QFm69HhDrf9hOcQVWUsJFdM9gqYvLteA+iWX6rH9XV6WLAjvzmP CzFqTAjWEX19x/3UC10Z3kuZNpiiZYb0iLzfcPb5Mz3bWfJ5+wH+ulp3jXr2YIuhu0Wr 6PxyECHiiakb+WkDoEtwrpxtkq5jdPpA3BloO0QA8XELnPra2ya5eCkhKrjWIy5eJk0S WoP+oiY9qcbIk04siL7/OQY4JVfQa+mFm8rhEPImMGSGtxJgp/1bRm9VYg8s2tjyF1U+ /j5w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:dkim-signature; bh=ORwfmj6kJY3MPYNwljMh0Ojz+sLVhyTdNLlUMLv1/pk=; b=l05vh/X4WuFouqtGqsakZLKUs+0MFIzlk6FcIOYprMwy9cA1VKPoDYINRbB5mjU1QD oQIP2bj7eKfKdPmikbd591TpOQtQRcrNMImQwHmUzOW3io7UAdmoyrEoO2xmHNBfDApE Zkr+o4XkU/BtiO+IGkGLFRT136im42e0c/XqPuza3cxDZxYQsPkagWZeAgCKxr0ic0su oC+XTwiO4iafKuj9+OKCUTpslUlcfUwQBJuhQh9mBnL1qURMHVWGwfHrEIVKt2595vPu WPKvF6RAy5EKzGnTvc/1vt5/cAQ+w97iFYNof4VbS9UwSdIyEdqbRLqELZGSTVO9OjzQ 7gLA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=PO78hPWL; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id x1-20020a056a00188100b0056bc14f1cd2si589223pfh.10.2022.11.04.15.02.08; Fri, 04 Nov 2022 15:02:22 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=PO78hPWL; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230140AbiKDWBQ (ORCPT + 99 others); Fri, 4 Nov 2022 18:01:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59102 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230075AbiKDWBE (ORCPT ); Fri, 4 Nov 2022 18:01:04 -0400 Received: from mail-pf1-x44a.google.com (mail-pf1-x44a.google.com [IPv6:2607:f8b0:4864:20::44a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EBC8548756 for ; Fri, 4 Nov 2022 15:01:02 -0700 (PDT) Received: by mail-pf1-x44a.google.com with SMTP id x8-20020aa79568000000b0056dd717e051so3049321pfq.11 for ; Fri, 04 Nov 2022 15:01:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=ORwfmj6kJY3MPYNwljMh0Ojz+sLVhyTdNLlUMLv1/pk=; b=PO78hPWLhGiPOdh3FKBDOyG8KHk37JCjEqyaVuWHpv31zowr7R3uAFNA1xtr4KmvX7 MNgLdzxinD3pD3F+Iy+FJRRYB6+BaIPkmAKMS8sMpLZvtYHqQ0Za+XlkPiLJGehQiKZD Ias/Me6yBT1a0kIMIsVRb9byR3i+C25+OfJMXPHOLtheGpsxMiwTrdYfDGgT7qe6YH0p VYlbRiW0TZ1PXbmeLJXvnAbdVEU6FaoijOh4HcfTQ+mLSmRNyTJbqm5Qx1KjIGGv3lW2 KyB/3dFcoWYX++Bzh/d18RvfnpRxTpexGp5tVGKeS09SDdPSUNgquKt+qz/3a71UO18U AzaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=ORwfmj6kJY3MPYNwljMh0Ojz+sLVhyTdNLlUMLv1/pk=; b=heu+Y+yHIH7PaGLre7J/K9qWlzHFPKAVZdozZw6K12yeqPeacesYNUDEhhOFiplI3K UBFknwddBKZysQhAJcyVbo0HrcD/9MPgYNXJ/y2+R/iYIpVX7lkNOE/tc7FAyKOqV+gz 3wVKjZv9RSq6uWCC6DUT/4EWh0vGSWEHN3HE4Wdqvb2farXOBo6Zn4ti+Omh5b/v4nNH XAZ6G/C4FWf0G2yOfldRPhIHYnHhhCRohjJ2lbksQ+Auk0mIsuji4ARuhP4eHq4KTBbK oyhftq1CP6H6g4Opnice4UomUOoDUWEpocEOAEj6Lzkp/+u6NlrfXmV1u/f39qeYTW7Q FVvg== X-Gm-Message-State: ACrzQf1J6WOEECqEQqAbVfdvhILR5hPWZiPQ8MqZ6tA7cI8JnNna5Y3F hlHJrszivwXD9s9u7GhIE8CMz8DvFAA= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:90b:1809:b0:213:aeae:64a1 with SMTP id lw9-20020a17090b180900b00213aeae64a1mr37290558pjb.226.1667599262508; Fri, 04 Nov 2022 15:01:02 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 4 Nov 2022 22:00:52 +0000 In-Reply-To: <20221104220053.1702977-1-seanjc@google.com> Mime-Version: 1.0 References: <20221104220053.1702977-1-seanjc@google.com> X-Mailer: git-send-email 2.38.1.431.g37b22c650d-goog Message-ID: <20221104220053.1702977-2-seanjc@google.com> Subject: [PATCH 1/2] x86/mm: Recompute physical address for every page of per-CPU CEA mapping From: Sean Christopherson To: Dave Hansen , Andy Lutomirski , Peter Zijlstra Cc: linux-kernel@vger.kernel.org, Andrey Ryabinin , Sean Christopherson , Dmitry Vyukov X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748604648018763033?= X-GMAIL-MSGID: =?utf-8?q?1748604648018763033?= Recompute the physical address for each per-CPU page in the CPU entry area, a recent commit inadvertantly modified cea_map_percpu_pages() such that every PTE is mapped to the physical address of the first page. Fixes: 9fd429c28073 ("x86/kasan: Map shadow for percpu pages on demand") Cc: Andrey Ryabinin Signed-off-by: Sean Christopherson --- arch/x86/mm/cpu_entry_area.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/mm/cpu_entry_area.c b/arch/x86/mm/cpu_entry_area.c index dff9001e5e12..d831aae94b41 100644 --- a/arch/x86/mm/cpu_entry_area.c +++ b/arch/x86/mm/cpu_entry_area.c @@ -97,7 +97,7 @@ cea_map_percpu_pages(void *cea_vaddr, void *ptr, int pages, pgprot_t prot) early_pfn_to_nid(PFN_DOWN(pa))); for ( ; pages; pages--, cea_vaddr+= PAGE_SIZE, ptr += PAGE_SIZE) - cea_set_pte(cea_vaddr, pa, prot); + cea_set_pte(cea_vaddr, per_cpu_ptr_to_phys(ptr), prot); } static void __init percpu_setup_debug_store(unsigned int cpu) From patchwork Fri Nov 4 22:00:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 15812 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp659677wru; Fri, 4 Nov 2022 15:02:27 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4AE1JNADlWEn5CjqHxMb1nBn/z4Gw/Uyh9vWt5YTKaT0HZTB/ois+Imr57/qEmkLhs4Z7N X-Received: by 2002:a17:902:6847:b0:186:61be:ef82 with SMTP id f7-20020a170902684700b0018661beef82mr37103938pln.135.1667599346742; Fri, 04 Nov 2022 15:02:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667599346; cv=none; d=google.com; s=arc-20160816; b=VIctZJM2QeTCFxF4DjC1iSuNUua0y2zJextKTo9F6xU7GZJXwZN4O4YCVEXbAc5u79 O6hOdNnop7LXSZwVNUCK/vSwztn3T3G/vvIvR/2fHwUmUyl3sHEsIjjICrO1nkB4dFxB clbPBjz6fbqljiXquMFEY68QihvQcVPlTH692U/ciCPZNJ+8/iNOLWVJvMR0AS5rKLkR WqZ103OAelfJp2/HkWhBM5YYy1uhWCJ2Pi34i0LglTc6p/AYmVpOXYBQ0MUq4guYyXH/ q/roAL1xf5tdIfzSTQTisGjDFgQa/d2eY0Hq91+kEyBQIyxMuZ2O3Oh1w7TbL8bnzjTk DH5g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:dkim-signature; bh=hFSxCFHJFYCR3Q5XQD/tcaVtxH3V8cegjhQxWvzfaqM=; b=eeP7v2s2It+mG1D2u1dIgT/c0MHQGUXV2DTjOuApQMeGFTK9ASjvFsX3xze8ewhDmI vSp1d68tpwWP7bTvc4afuFTriUhKT0MTNQBRMOekWnFmgzSKeeRD0yLK0NziT2NEb4ve AMJiTd7AwAktmgsAD3YHuca5xNPP5wP4OauowzIeFcVuF+844XleqaKhLEv/4Wx2ic0L 6ZqmHLTmOmtyGtpNd1O73Cgi5jC895LtKxb6tC8dwi5dbBe1+fQvsJx0FRpjiOUTNtJs PZzlNuFf/kX16riY8+RNgJYrpzx+iBRHHOYouRTCgfGg3+Q2Z5uDmp5wtIMGc7TzTx4J U0MQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=q0L8+bqn; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id az6-20020a056a02004600b004703815aa10si779863pgb.721.2022.11.04.15.02.13; Fri, 04 Nov 2022 15:02:26 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=q0L8+bqn; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230200AbiKDWBV (ORCPT + 99 others); Fri, 4 Nov 2022 18:01:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59082 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230108AbiKDWBK (ORCPT ); Fri, 4 Nov 2022 18:01:10 -0400 Received: from mail-pf1-x44a.google.com (mail-pf1-x44a.google.com [IPv6:2607:f8b0:4864:20::44a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DD4474E418 for ; Fri, 4 Nov 2022 15:01:04 -0700 (PDT) Received: by mail-pf1-x44a.google.com with SMTP id s11-20020a056a00178b00b0056cb4545c3fso3003261pfg.5 for ; Fri, 04 Nov 2022 15:01:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=hFSxCFHJFYCR3Q5XQD/tcaVtxH3V8cegjhQxWvzfaqM=; b=q0L8+bqnh53LPDpc0iuNUVddTLJoqw3lgeoQHAOoNMmJjbaq/RhZUEfKy65bQ2V4tx ERlv7E97fNBIVpYL+3G69Vj1ulBMJ0DL3rPkykV1dWoMXRXD5NyqnSBvGMSyjBEojso8 m6a5BX9oE/0gS/tTIbuRF2KkUS+AvoBLzXaU3IwHZevQo5anQo0hG2J/AUwEAG3Wh1dv Blmz68yvweRKLbBXaEw1fyIOLCCdFn2hfFu6QBTl8a4nIsVrR21EwF36hXNqsswJS558 Nob/GWI+YiXwC/4g88+MVWj+0dDFnEpBmDc/3RAyWL+C/sAuPgUoRuFlJPsJe7UhAoUy 8Q7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=hFSxCFHJFYCR3Q5XQD/tcaVtxH3V8cegjhQxWvzfaqM=; b=eG7lz7kA6SZuhT54L9ppsmAdzc6IStDjuYiV9UOfpf447dGMAir58FnN5JwrGrkR2k Sdy6qpRuBp39Au6+RX9EAXVtuws7NeRlzVV71vysBxWC1bv2DaySHGuCkAqU+8725CSK p9jW84FlW7V9sG4dcG+/jBWT6Hi0OR0qMbhkf8i7JGH72fogRnVefg+ftVxJHSGpd4h7 fK+sSZvaPoITmspfrEfqDzBWZu0R/kMJRECsHAHPNpUrQOhphmwfu1iOWUn6+w6BUccq PLs/WmwuckTJ6ZU0Tr430v48udQvhsPH8F+H7vOT+eQO7ovCNfOSer8Glhj1PeA5P1Li WRgg== X-Gm-Message-State: ACrzQf0knlXieM/JsaS/Ez1FqxYzXiDz/ddReHnMmDs+tVRQfIgobNZ1 TF3eiokT/kj5+U4fsI5z87WJS4By1AY= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:90b:11c1:b0:212:d3ec:632f with SMTP id gv1-20020a17090b11c100b00212d3ec632fmr381735pjb.43.1667599264346; Fri, 04 Nov 2022 15:01:04 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 4 Nov 2022 22:00:53 +0000 In-Reply-To: <20221104220053.1702977-1-seanjc@google.com> Mime-Version: 1.0 References: <20221104220053.1702977-1-seanjc@google.com> X-Mailer: git-send-email 2.38.1.431.g37b22c650d-goog Message-ID: <20221104220053.1702977-3-seanjc@google.com> Subject: [PATCH 2/2] x86/mm: Populate KASAN shadow for per-CPU DS buffers in CPU entry area From: Sean Christopherson To: Dave Hansen , Andy Lutomirski , Peter Zijlstra Cc: linux-kernel@vger.kernel.org, Andrey Ryabinin , Sean Christopherson , Dmitry Vyukov X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748604652972226757?= X-GMAIL-MSGID: =?utf-8?q?1748604652972226757?= Bounce through cea_map_percpu_pages() when setting the initial protections for per-CPU DS buffers so that KASAN populates a shadow for said mapping. Failure to populate the shadow will result in a not-present #PF during KASAN validation if DS buffers are activated later on. Fixes: 9fd429c28073 ("x86/kasan: Map shadow for percpu pages on demand") Cc: Andrey Ryabinin Cc: Dmitry Vyukov Signed-off-by: Sean Christopherson --- arch/x86/mm/cpu_entry_area.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/arch/x86/mm/cpu_entry_area.c b/arch/x86/mm/cpu_entry_area.c index d831aae94b41..64ae557ceb22 100644 --- a/arch/x86/mm/cpu_entry_area.c +++ b/arch/x86/mm/cpu_entry_area.c @@ -91,13 +91,12 @@ void cea_set_pte(void *cea_vaddr, phys_addr_t pa, pgprot_t flags) static void __init cea_map_percpu_pages(void *cea_vaddr, void *ptr, int pages, pgprot_t prot) { - phys_addr_t pa = per_cpu_ptr_to_phys(ptr); + int nid = ptr ? early_pfn_to_nid(PFN_DOWN(per_cpu_ptr_to_phys(ptr))) : 0; - kasan_populate_shadow_for_vaddr(cea_vaddr, pages * PAGE_SIZE, - early_pfn_to_nid(PFN_DOWN(pa))); + kasan_populate_shadow_for_vaddr(cea_vaddr, pages * PAGE_SIZE, nid); for ( ; pages; pages--, cea_vaddr+= PAGE_SIZE, ptr += PAGE_SIZE) - cea_set_pte(cea_vaddr, per_cpu_ptr_to_phys(ptr), prot); + cea_set_pte(cea_vaddr, ptr ? per_cpu_ptr_to_phys(ptr) : 0, prot); } static void __init percpu_setup_debug_store(unsigned int cpu) @@ -121,8 +120,7 @@ static void __init percpu_setup_debug_store(unsigned int cpu) * memory like debug store buffers. */ npages = sizeof(struct debug_store_buffers) / PAGE_SIZE; - for (; npages; npages--, cea += PAGE_SIZE) - cea_set_pte(cea, 0, PAGE_NONE); + cea_map_percpu_pages(cea, NULL, npages, PAGE_NONE); #endif }