From patchwork Mon Sep 18 10:32:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 141350 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp2561264vqi; Mon, 18 Sep 2023 03:47:37 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFJH2IqZZEywST0jdB9abveD5Ft7kA3V5nI+FfDbaRCD37pS2ojXn8S0ZvOenBtNDA7xw0A X-Received: by 2002:a05:6a21:718b:b0:154:a9bc:12ca with SMTP id wq11-20020a056a21718b00b00154a9bc12camr7309060pzb.26.1695034056932; Mon, 18 Sep 2023 03:47:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695034056; cv=none; d=google.com; s=arc-20160816; b=Jt+Yab1CVjmbyLsl+HMQhXPEOseAbwxogC6J5wpDNFDJoCJXMofs4gJs2rI4/OdvyV cXE435IMnRlAwfJFnqTQ6apSjMv60Do7f8NKFoKouVCsMTVQkuJR4qvY3kQsxoemb1VM btV1DnxtfB+wfbogEk/4F1czkP4Y9ceg23g4nAhJ6195+nXOMXJHDGEsNhenb5l7HX4A oDvPuX6GuSgxwwxq3rhX3/jswoszcr1cpFrc8jeGK0RLE9zGoo+Th1i1LpZPoIRUMyc7 8+8uYHNNxBRIJoCcTtMDPvKIPrJUBWWYAPgNmacjPX8SIzh6E8h2rMMixIz1priGh3jW u1uw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=CMSHTaHrk6mPrRaj+nNWbQVpdzwUiaPbO4b5ojYmYnI=; fh=Je6qNyAM729b1goCtB6PkBk+yQrlQCIjJMRY6BFgKTo=; b=hoUyPaxlm2fIjR3osPpVjWp1OLcIegvvMd6pqt2yf30tmLboT8owQYKtNW6FqBaY6S pb2nFfZlHK4MZADckBdmuVR9HWZqipETfjhySqlhd1VxOkrcKTTlj/PA3xNfScIzyizF IeBWRpPaK49WUuEgnmmakUAcwnmRV8kyirDQQTm5a+7Q5c/Xdd8YIMJ1GyV96IuRfA/3 pvS2zdYyi6oLZKryhZ6qqHMzzepL8/ZBtGesgdRuZw8Hfn3qQYkRneqNi1e4iV0p+liU rO+XEPW4y+bvzhA7+dGCQYQDdqs/ILYNxjW2Zx03Bmoe7bKlNcWbRg0hLo80hQA5kH+a zwGg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id k2-20020a170902c40200b001b8921fbd87si8572484plk.490.2023.09.18.03.47.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Sep 2023 03:47:36 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id A073E818ABF9; Mon, 18 Sep 2023 03:34:07 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241369AbjIRKdn (ORCPT + 27 others); Mon, 18 Sep 2023 06:33:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38514 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241338AbjIRKdP (ORCPT ); Mon, 18 Sep 2023 06:33:15 -0400 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CBA31128 for ; Mon, 18 Sep 2023 03:32:50 -0700 (PDT) Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.55]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4Rq1HL6T1FzNnfq; Mon, 18 Sep 2023 18:29:02 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Mon, 18 Sep 2023 18:32:48 +0800 From: Kefeng Wang To: Andrew Morton CC: , , , , , Zi Yan , Mike Kravetz , , Kefeng Wang Subject: [PATCH 4/6] mm: memory: use a folio in do_numa_page() Date: Mon, 18 Sep 2023 18:32:11 +0800 Message-ID: <20230918103213.4166210-5-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20230918103213.4166210-1-wangkefeng.wang@huawei.com> References: <20230918103213.4166210-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Mon, 18 Sep 2023 03:34:07 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777372031304516144 X-GMAIL-MSGID: 1777372031304516144 Numa balancing only try to migrate non-compound page in do_numa_page(), use a folio in it to save several compound_head calls, note we use folio_estimated_sharers(), it is enough to check the folio sharers since only normal page is handled, if large folio numa balancing is supported, a precise folio sharers check would be used, no functional change intended. Signed-off-by: Kefeng Wang --- mm/memory.c | 33 ++++++++++++++++----------------- 1 file changed, 16 insertions(+), 17 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index ce7d9d9eddc4..ce3efe7255d2 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4737,8 +4737,8 @@ int numa_migrate_prep(struct folio *folio, struct vm_area_struct *vma, static vm_fault_t do_numa_page(struct vm_fault *vmf) { struct vm_area_struct *vma = vmf->vma; - struct page *page = NULL; - int page_nid = NUMA_NO_NODE; + struct folio *folio = NULL; + int nid = NUMA_NO_NODE; bool writable = false; int last_cpupid; int target_nid; @@ -4769,12 +4769,12 @@ static vm_fault_t do_numa_page(struct vm_fault *vmf) can_change_pte_writable(vma, vmf->address, pte)) writable = true; - page = vm_normal_page(vma, vmf->address, pte); - if (!page || is_zone_device_page(page)) + folio = vm_normal_folio(vma, vmf->address, pte); + if (!folio || folio_is_zone_device(folio)) goto out_map; /* TODO: handle PTE-mapped THP */ - if (PageCompound(page)) + if (folio_test_large(folio)) goto out_map; /* @@ -4789,34 +4789,33 @@ static vm_fault_t do_numa_page(struct vm_fault *vmf) flags |= TNF_NO_GROUP; /* - * Flag if the page is shared between multiple address spaces. This + * Flag if the folio is shared between multiple address spaces. This * is later used when determining whether to group tasks together */ - if (page_mapcount(page) > 1 && (vma->vm_flags & VM_SHARED)) + if (folio_estimated_sharers(folio) > 1 && (vma->vm_flags & VM_SHARED)) flags |= TNF_SHARED; - page_nid = page_to_nid(page); + nid = folio_nid(folio); /* * For memory tiering mode, cpupid of slow memory page is used * to record page access time. So use default value. */ if ((sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING) && - !node_is_toptier(page_nid)) + !node_is_toptier(nid)) last_cpupid = (-1 & LAST_CPUPID_MASK); else - last_cpupid = page_cpupid_last(page); - target_nid = numa_migrate_prep(page_folio(page), vma, vmf->address, - page_nid, &flags); + last_cpupid = page_cpupid_last(&folio->page); + target_nid = numa_migrate_prep(folio, vma, vmf->address, nid, &flags); if (target_nid == NUMA_NO_NODE) { - put_page(page); + folio_put(folio); goto out_map; } pte_unmap_unlock(vmf->pte, vmf->ptl); writable = false; /* Migrate to the requested node */ - if (migrate_misplaced_folio(page_folio(page), vma, target_nid)) { - page_nid = target_nid; + if (migrate_misplaced_folio(folio, vma, target_nid)) { + nid = target_nid; flags |= TNF_MIGRATED; } else { flags |= TNF_MIGRATE_FAIL; @@ -4832,8 +4831,8 @@ static vm_fault_t do_numa_page(struct vm_fault *vmf) } out: - if (page_nid != NUMA_NO_NODE) - task_numa_fault(last_cpupid, page_nid, 1, flags); + if (nid != NUMA_NO_NODE) + task_numa_fault(last_cpupid, nid, 1, flags); return 0; out_map: /*