From patchwork Mon Jul 31 07:48:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 128427 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:918b:0:b0:3e4:2afc:c1 with SMTP id s11csp1866882vqg; Mon, 31 Jul 2023 01:13:25 -0700 (PDT) X-Google-Smtp-Source: APBJJlHayw5aUM5su/DQWyDeUVCVfc+IY6IyTi27P3dR6yPc4ee+orVEnTIPvuKxpGZXatnKAcUe X-Received: by 2002:a05:6a21:7182:b0:133:b3a9:90d with SMTP id wq2-20020a056a21718200b00133b3a9090dmr8979337pzb.36.1690791205106; Mon, 31 Jul 2023 01:13:25 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690791205; cv=none; d=google.com; s=arc-20160816; b=eicH5oRuiKazNoWK/CUiRMCXiCKotASgXes2Tj3Xv7OUsyGMvtPVd5rHG1AoDTjK27 SoFdDpo3ZxUPYUTTGWAEsBmsW54OS3ZIW9vBJoyEUZwsJY2QNIMwnS9mb7HA5LRBe2yb MUsmkwuJdo/Gt0KGrlRoFH7kBbH7QsYg1Yl3RYt7Vwznnx9q3DJPeI0bkpYh0wa2yLay J29XPIo2w1+KyDVuzM4aMfeZi0Zan8zT8TvmD3ggUtoDwjpiZhYAxf8+62xNIcjtUmIA Tft/y1VIK6bUXBxQFr6knSGbqHHoPXqhFQSuPpDc4W87YwjcKTPR0i6wACuREvmZzbJB F8hA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=Yj4az2ALhhEyvky0TbaSWKAJzQx3pWTWZySo+ZOZ/q0=; fh=pWGPqOKfNGLee4HdbDJvfX9Bibz+qaOAq2a2PB8WHto=; b=i6JSJrMTG9PtywYeQco94G+IQxiYMykLtNNrQeVW392FGPNhg540Erzzo9BIP7/9pC bwANSn8jGCuFCXWLJ0pzD1EICcIy0+domIOhcSsZKzr5LUD/dcy0N9R5zSuysMYTOUdK zaj5kWQb6f0ZZjRNxWb13BpEcu5/uMIm9RYEGeHx1Q8GJeKViHvswgIC66N5lbayfidE vwkaUQtE3/1lr5xjsJLRsrLLnCZmWlYYymMr+l9jfeuXF7ie/tKjqn9boTxCQTc6/BKH FvyekU8yh/GG8fOHOe6g3sFi6KutHTanaDtD3Bog96h/cEadg7cEYPOmsS0O3D1jhDa/ +UWw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id fd28-20020a056a002e9c00b0067d12984ec5si3174383pfb.289.2023.07.31.01.13.12; Mon, 31 Jul 2023 01:13:25 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230458AbjGaHjY (ORCPT + 99 others); Mon, 31 Jul 2023 03:39:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47350 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229487AbjGaHjT (ORCPT ); Mon, 31 Jul 2023 03:39:19 -0400 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A8D8110C for ; Mon, 31 Jul 2023 00:39:17 -0700 (PDT) Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.57]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4RDqm627jmzNmP9; Mon, 31 Jul 2023 15:35:50 +0800 (CST) Received: from localhost.localdomain.localdomain (10.175.113.25) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Mon, 31 Jul 2023 15:39:14 +0800 From: Kefeng Wang To: Andrew Morton , Catalin Marinas , Will Deacon , Mike Kravetz , Muchun Song , Mina Almasry , , , , , CC: , , Kefeng Wang Subject: [PATCH 1/4] mm: hugetlb: use flush_hugetlb_tlb_range() in move_hugetlb_page_tables() Date: Mon, 31 Jul 2023 15:48:26 +0800 Message-ID: <20230731074829.79309-2-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230731074829.79309-1-wangkefeng.wang@huawei.com> References: <20230731074829.79309-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.113.25] X-ClientProxiedBy: dggems702-chm.china.huawei.com (10.3.19.179) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1772923078727086084 X-GMAIL-MSGID: 1772923078727086084 Archs may need to do special things when flushing hugepage tlb, so use the more applicable flush_hugetlb_tlb_range() instead of flush_tlb_range(). Fixes: 550a7d60bd5e ("mm, hugepages: add mremap() support for hugepage backed vma") Signed-off-by: Kefeng Wang Reviewed-by: Mike Kravetz Acked-by: Muchun Song --- mm/hugetlb.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 64a3239b6407..ac876bfba340 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -5281,9 +5281,9 @@ int move_hugetlb_page_tables(struct vm_area_struct *vma, } if (shared_pmd) - flush_tlb_range(vma, range.start, range.end); + flush_hugetlb_tlb_range(vma, range.start, range.end); else - flush_tlb_range(vma, old_end - len, old_end); + flush_hugetlb_tlb_range(vma, old_end - len, old_end); mmu_notifier_invalidate_range_end(&range); i_mmap_unlock_write(mapping); hugetlb_vma_unlock_write(vma); From patchwork Mon Jul 31 07:48:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 128426 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:918b:0:b0:3e4:2afc:c1 with SMTP id s11csp1866769vqg; Mon, 31 Jul 2023 01:13:06 -0700 (PDT) X-Google-Smtp-Source: APBJJlF0qkL/b8SnzV58jC+/Sn3wHlEviMMYxekOsJDx2t06ietl1IqFcZIIzi50vPQzKGSQaDiD X-Received: by 2002:a05:6a21:790a:b0:131:4808:d5a1 with SMTP id bg10-20020a056a21790a00b001314808d5a1mr7901571pzc.28.1690791186096; Mon, 31 Jul 2023 01:13:06 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690791186; cv=none; d=google.com; s=arc-20160816; b=JxsINs70SbB73Orw3tR8qzDfdNeK2dc/Qksdr8hG3PN7nOUcxmy7verYvxBF4IqfEr ndx/By/DtJYwJ7cGNnQR297MVcXG8tfrx5kW2Bau/G86kP1b0azrnKGWupU8xdApNytj s2CEhYFU8BjCpH2+IzjsuknxrBBMAT8eyBhJyM7QjXTi3OZcb8qCVX1sdjhbfir8Uv7J buTVfZLQv6OEC8Uu0iJx9cZtXYV4lsfVA+aJioi8COLgI7WEn+CB+kTNZ/vLG75cevOC KP7vNaQRwC/MFI91HaQjW8YOLKpjhNEfIAeyp/sM2nfAwUgRMErnh/qz8ig9vOgxp+Vv AWfQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=QnjtTek5fZ/4QtwvL41dNaJX0C5UFKPKxH+gNlzHWXo=; fh=pWGPqOKfNGLee4HdbDJvfX9Bibz+qaOAq2a2PB8WHto=; b=oIZ+DYLqkCSbJyzrv5QNTBuPpTvNXmgpZWkDw3xleuYvN6lsuGD1YxL5EZMRhwVOMm 2RMIUZWNiTu1dsCJmaS2NTcTbeXUhxUPiSHY95rU0Dfz5qFCF1ocsOxeAtBsAD14uyOI 2ivI9XliENn9hkFa1HqyTCfbYR68gDsjaVGvXCYwlFeS70NOckyc6j0xc9T8UIWy1//C Cdfhz0BEaguKGsSzJlpYp9FNtUogcY6qMovBbrdKpw8uh/THCbNHHpfZhJAxR4NsSZot wE6aFtlwpueZIucEZOmWrvb/FkVH7HXe+kcK5VqjqrW7VBkXx39ZFuGc6YpPr/l/PpJg OPJw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id q7-20020a170902a3c700b001bb162e944dsi3829295plb.51.2023.07.31.01.12.52; Mon, 31 Jul 2023 01:13:06 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231401AbjGaHjc (ORCPT + 99 others); Mon, 31 Jul 2023 03:39:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47352 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229722AbjGaHjU (ORCPT ); Mon, 31 Jul 2023 03:39:20 -0400 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B768C10E for ; Mon, 31 Jul 2023 00:39:17 -0700 (PDT) Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.54]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4RDqmD2YBwztQSD; Mon, 31 Jul 2023 15:35:56 +0800 (CST) Received: from localhost.localdomain.localdomain (10.175.113.25) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Mon, 31 Jul 2023 15:39:15 +0800 From: Kefeng Wang To: Andrew Morton , Catalin Marinas , Will Deacon , Mike Kravetz , Muchun Song , Mina Almasry , , , , , CC: , , Kefeng Wang Subject: [PATCH 2/4] mm: mremap: use flush_pmd_tlb_range() in move_normal_pmd() Date: Mon, 31 Jul 2023 15:48:27 +0800 Message-ID: <20230731074829.79309-3-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230731074829.79309-1-wangkefeng.wang@huawei.com> References: <20230731074829.79309-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.113.25] X-ClientProxiedBy: dggems702-chm.china.huawei.com (10.3.19.179) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Spam-Status: No, score=-5.2 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1772923058930469245 X-GMAIL-MSGID: 1772923058930469245 Archs may need to do special things when flushing thp tlb, so use the more applicable flush_pud_tlb_range() instead of flush_tlb_range(). Fixes: 2c91bd4a4e2e ("mm: speed up mremap by 20x on large regions") Signed-off-by: Kefeng Wang --- mm/mremap.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/mremap.c b/mm/mremap.c index 11e06e4ab33b..1883205fa22b 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -284,7 +284,7 @@ static bool move_normal_pmd(struct vm_area_struct *vma, unsigned long old_addr, VM_BUG_ON(!pmd_none(*new_pmd)); pmd_populate(mm, new_pmd, pmd_pgtable(pmd)); - flush_tlb_range(vma, old_addr, old_addr + PMD_SIZE); + flush_pmd_tlb_range(vma, old_addr, old_addr + PMD_SIZE); if (new_ptl != old_ptl) spin_unlock(new_ptl); spin_unlock(old_ptl); From patchwork Mon Jul 31 07:48:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 128436 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:918b:0:b0:3e4:2afc:c1 with SMTP id s11csp1867592vqg; Mon, 31 Jul 2023 01:15:12 -0700 (PDT) X-Google-Smtp-Source: APBJJlHAhAWn47kmWG1+wicSySRumOmOfRehBn5D04XAMWWHejmlMkVxcZsziB4HNz6zn/iVrGjh X-Received: by 2002:a17:90b:3908:b0:268:d1c:2421 with SMTP id ob8-20020a17090b390800b002680d1c2421mr8300069pjb.38.1690791312515; Mon, 31 Jul 2023 01:15:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690791312; cv=none; d=google.com; s=arc-20160816; b=QKFWaKxcpXZbPsjZH0KL0x0dqFBUuwWOcrfEXzuXi5wiRljflMAbkxATA3JSu2NS+K tZHwGZ46j29qcZH8Ogqx0IdDmYiH82IYbzjrS8KAjhdFBtMllw5C3JV0kSBXQ8oXs8ZW OJMM5xzBC83rEIEN6ppAGGaGO6l/YYp7H8LDnQYsJsLwzJITmrpIMFoHjJjeCRi62i3H WRehnNStIbn9vjgB+0XZZ8MinmJTSP62ByKX5ZlZ2TJT+Wrn44b66Kl1sEy1EaszhpEA JXTpsjRnRhx5X7Oh0cCbclW8BlFbux2SDpfbVzN+HJZ8dg1twqmZCFcrEtldPJki3XIy hPJA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=y2xwMHmp2+AF+LplUKlYJQ4KHQF98DXrKqXWA6DlewM=; fh=pWGPqOKfNGLee4HdbDJvfX9Bibz+qaOAq2a2PB8WHto=; b=lYlWu1bKZAk4xHh30/44u2PVnNRS6gE0c78WUanUrqT8nJjBaPBPqZzBZVBcgUoYDD P/oXL7m2wpUu/2u1yxWL8QjIVihbfHP8DI9nBV01jUpHOU0Yvq4d/4lAaV/qOxYeyiCH npjveQbEFXEmiDfH+dT5r8045WOfaissefGB3rQvpmHGRGRxCDalP/tVlKPY3EmpFAMV 7kb6FQ5yeaWX8tWrSD2cF4jlKWZxF3uZ2eR7ez0IcVpy3yE64UvgYuyrjJKk/Sg/qLJf rih+OajX1NK1OGAeUH9qtOh4K9XRnBuWpe2cSViyIrYSvYphu7Mh9/YWMR6O4P4+EW2F EEQA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 41-20020a17090a09ac00b00267e9ad4f40si5707847pjo.54.2023.07.31.01.14.59; Mon, 31 Jul 2023 01:15:12 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231468AbjGaHjf (ORCPT + 99 others); Mon, 31 Jul 2023 03:39:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47362 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229755AbjGaHjV (ORCPT ); Mon, 31 Jul 2023 03:39:21 -0400 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6EB7F115 for ; Mon, 31 Jul 2023 00:39:18 -0700 (PDT) Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.54]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4RDqm74d7YzNmbC; Mon, 31 Jul 2023 15:35:51 +0800 (CST) Received: from localhost.localdomain.localdomain (10.175.113.25) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Mon, 31 Jul 2023 15:39:15 +0800 From: Kefeng Wang To: Andrew Morton , Catalin Marinas , Will Deacon , Mike Kravetz , Muchun Song , Mina Almasry , , , , , CC: , , Kefeng Wang Subject: [PATCH 3/4] mm: mremap: use flush_pud_tlb_range in move_normal_pud() Date: Mon, 31 Jul 2023 15:48:28 +0800 Message-ID: <20230731074829.79309-4-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230731074829.79309-1-wangkefeng.wang@huawei.com> References: <20230731074829.79309-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.113.25] X-ClientProxiedBy: dggems702-chm.china.huawei.com (10.3.19.179) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1772923191518044652 X-GMAIL-MSGID: 1772923191518044652 Archs may need to do special things when flushing thp tlb, so use the more applicable flush_pud_tlb_range() instead of flush_tlb_range(). Fixes: c49dd3401802 ("mm: speedup mremap on 1GB or larger regions") Signed-off-by: Kefeng Wang --- mm/mremap.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/mremap.c b/mm/mremap.c index 1883205fa22b..25114e56901f 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -333,7 +333,7 @@ static bool move_normal_pud(struct vm_area_struct *vma, unsigned long old_addr, VM_BUG_ON(!pud_none(*new_pud)); pud_populate(mm, new_pud, pud_pgtable(pud)); - flush_tlb_range(vma, old_addr, old_addr + PUD_SIZE); + flush_pud_tlb_range(vma, old_addr, old_addr + PUD_SIZE); if (new_ptl != old_ptl) spin_unlock(new_ptl); spin_unlock(old_ptl); From patchwork Mon Jul 31 07:48:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 128418 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:918b:0:b0:3e4:2afc:c1 with SMTP id s11csp1864674vqg; Mon, 31 Jul 2023 01:07:50 -0700 (PDT) X-Google-Smtp-Source: APBJJlGSyXtVwLiFAIVUeOvHCnP9j5LDePzgoBd4Ce2xLVjuf0TU+/ylfGvsmVW5bSRM0MpfHqxi X-Received: by 2002:a17:902:e803:b0:1b6:6e3a:77fb with SMTP id u3-20020a170902e80300b001b66e3a77fbmr10742473plg.2.1690790869771; Mon, 31 Jul 2023 01:07:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690790869; cv=none; d=google.com; s=arc-20160816; b=bAOF/FsrG6PuplC1k94VJXPppWMNxl5pKVPH6nl/Y/I9cNxiWyPPpQcmPEIE8JE2A6 5NJfjgxQ+fKrQPdWQ1ux8h95/G59owNF1goPEXRj3rYYblv8iHuYMk1JRVQChcLTXPk8 yPdS5RJ5uVQ5SLUu2IL50g1LFzwmygCReHxnu7c1NhwXlnYfWFO37o1OmJuRsZXb2tAk Ch/vaF9NbEbaMDSIQeGDgttgmwgSbb4BsYa5lMWUIaK27GJ/W/cTkpS2iUxDowUdXJRC 0B7tL5jZA3NLy320/Je4p+3+rsJjwhzTo+smlhApeRH22HG8OJnM1WGpbOxtYiRh7dkL 1JXg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=Y3tb33D7Yj/stY7JznHb58iHPQ693NkVRx8ipzJMNtk=; fh=pWGPqOKfNGLee4HdbDJvfX9Bibz+qaOAq2a2PB8WHto=; b=PU7tF4To6wvyryd9grvKPu4xnFjRDhj4YQ6n9wn75sukJ08S9gwjTDO6tA51oB/+L4 TsC+bGgmVpz0fnTBG8x/5Szv1GR5ZBeB7iHGLT2L7xf8ewwDPK8rRX8UXFzaKKNo1ch1 McRdzBxGapAYkw2W3mRMn8iDY9rtgfMaS37OCzpq/UWb3/v2TBfDgQsWVYLGi4h3ys6f 4SwrNs0JgPGj8r/lQ0pp1wLa54/wDXd+Ca0fHeOljKu7i6GG29CEA3Qu41ZWkhTG8A2s gRdybiBklKrgSTr+QHu6Id12FOl7UVz2wn498ow69ujDFcKEI0yBb15Kc6DrsM+co1Nv yzHw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id le11-20020a170902fb0b00b001bbca0a8393si2772644plb.56.2023.07.31.01.07.35; Mon, 31 Jul 2023 01:07:49 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231634AbjGaHji (ORCPT + 99 others); Mon, 31 Jul 2023 03:39:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47376 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229680AbjGaHjV (ORCPT ); Mon, 31 Jul 2023 03:39:21 -0400 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 06E33125 for ; Mon, 31 Jul 2023 00:39:19 -0700 (PDT) Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.53]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4RDqp763FHzVjdw; Mon, 31 Jul 2023 15:37:35 +0800 (CST) Received: from localhost.localdomain.localdomain (10.175.113.25) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Mon, 31 Jul 2023 15:39:16 +0800 From: Kefeng Wang To: Andrew Morton , Catalin Marinas , Will Deacon , Mike Kravetz , Muchun Song , Mina Almasry , , , , , CC: , , Kefeng Wang Subject: [PATCH 4/4] arm64: tlb: set huge page size to stride for hugepage Date: Mon, 31 Jul 2023 15:48:29 +0800 Message-ID: <20230731074829.79309-5-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230731074829.79309-1-wangkefeng.wang@huawei.com> References: <20230731074829.79309-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.113.25] X-ClientProxiedBy: dggems702-chm.china.huawei.com (10.3.19.179) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1772922726820028505 X-GMAIL-MSGID: 1772922726820028505 It is better to use huge_page_size() for hugepage(HugeTLB) instead of PAGE_SIZE for stride, which has been done in flush_pmd/pud_tlb_range(), it could reduce the loop in __flush_tlb_range(). Signed-off-by: Kefeng Wang --- arch/arm64/include/asm/tlbflush.h | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/arch/arm64/include/asm/tlbflush.h b/arch/arm64/include/asm/tlbflush.h index 412a3b9a3c25..25e35e6f8093 100644 --- a/arch/arm64/include/asm/tlbflush.h +++ b/arch/arm64/include/asm/tlbflush.h @@ -360,16 +360,17 @@ static inline void __flush_tlb_range(struct vm_area_struct *vma, dsb(ish); } -static inline void flush_tlb_range(struct vm_area_struct *vma, - unsigned long start, unsigned long end) -{ - /* - * We cannot use leaf-only invalidation here, since we may be invalidating - * table entries as part of collapsing hugepages or moving page tables. - * Set the tlb_level to 0 because we can not get enough information here. - */ - __flush_tlb_range(vma, start, end, PAGE_SIZE, false, 0); -} +/* + * We cannot use leaf-only invalidation here, since we may be invalidating + * table entries as part of collapsing hugepages or moving page tables. + * Set the tlb_level to 0 because we can not get enough information here. + */ +#define flush_tlb_range(vma, start, end) \ + __flush_tlb_range(vma, start, end, \ + ((vma)->vm_flags & VM_HUGETLB) \ + ? huge_page_size(hstate_vma(vma)) \ + : PAGE_SIZE, false, 0) + static inline void flush_tlb_kernel_range(unsigned long start, unsigned long end) {