Message ID | 20230206063313.635011-1-ying.huang@intel.com |
---|---|
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2081768wrn; Sun, 5 Feb 2023 22:34:50 -0800 (PST) X-Google-Smtp-Source: AK7set/2vsSjsfKGYAQynvJjoRQaAy0M3ZnhcAANnMg85lnX5lqYoo6fkUnlWJ1NH1X21HCjz1t3 X-Received: by 2002:a17:902:ecd1:b0:198:f45c:8554 with SMTP id a17-20020a170902ecd100b00198f45c8554mr8219296plh.55.1675665290538; Sun, 05 Feb 2023 22:34:50 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675665290; cv=none; d=google.com; s=arc-20160816; b=Q28chOKDUf960AhvQfnzT3MxgeGBbcOI0/uFOZ+8F5F0leNGOkmRw3k46Z4lEQCM+o pjjhqhqADiaISqYntgAos3EYwTKcTG1zs1dYVW3NbPknaYnx55keElFYBIhuF5kIftiS IU0yDxhXx43Ov+IFfhH2FFSazS8XeWSEL11grEZUbPaSUjILHdl83j/9Of1tByGcbv1h PotetOHdK4ZcpjUVJAepes3OpAYQxxUzyphZ0s4ufeQtdIBWb/BFUnNWSC1DaMgAv1LO u447GetjEjpLWTgXsDYfAXeW28Dy/i/JdRJPZ99WRUiKF0E7JefmGl6sK31irmPN/Vf3 ZDSA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=eQCGybRKLuo1lXH+q95g+s/s9WCuJaz4/3G88lpb+jY=; b=pw6fTOM7dAvyMuNTtCUxDlOVKN7gBJp26APm+67Vp4I7vB7tTr2OTOoaGUOMXTQuEG EvJPUtsGwvcf8zYCG6UYW4g+HXTKrRacKEsjVK/9FsgantniWBoMz55EiaIVemZPdjn6 32b2eI3qvvRbZ6/4P1lrbFCOnTIFxodTEXJPzuPycdmiJSI1ecCaYmJPPUXuQZK77xE0 TgRbqR3WakABnCC/QcQHiwIsSfEUZLX6VAAbsjt/3kZ571qw0I/6U5SuRjzRWT48cMHS bC0yJQ+wcNF7wZcNYJvbB+oazdpw2yCxueQfVnfTSPJKluB7TRIflfPeiFhn0U8Jg810 BfxA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=IAbdWdQu; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id x28-20020a63b21c000000b004e1821c0576si12161243pge.95.2023.02.05.22.34.38; Sun, 05 Feb 2023 22:34:50 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=IAbdWdQu; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229608AbjBFGdi (ORCPT <rfc822;kmanaouilinux@gmail.com> + 99 others); Mon, 6 Feb 2023 01:33:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36514 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229479AbjBFGdd (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Mon, 6 Feb 2023 01:33:33 -0500 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B2368193D4 for <linux-kernel@vger.kernel.org>; Sun, 5 Feb 2023 22:33:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675665208; x=1707201208; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=u2qsfcVM3St5WrkIXjYK8yM1e3KwimUpAfokCw818Xk=; b=IAbdWdQuAlRaiqYZTseoqrIrYn2ZONiCjMM0fgrtawKpIBnjOPKQDF/8 DKp3wnPJSwJBGYARYj5rgkqO5d7vtwa+zQaVojhk+F9lA4XKF/6RaZj01 Nvb8VvAmfauHDQxpZtJ+VIssPu6i6uf42UB6P7NSk9joUwHz/qWxZuEot nbpJkWm6bDcceDqBMScIq1zVHKvGHFaBJ64Ib/Vexx7Z87mArXdMzM99N hXghWmftF/Ty8gAzem/O26N66IxAkoSQheG1Yh/RZSMbI7i4t6Xc+X0IT AxzJjgiEbvOuM1th9fmgHJVku9UrbYMS/cmoEmiVLNEn6r0nnqvM0jGOK w==; X-IronPort-AV: E=McAfee;i="6500,9779,10612"; a="330432607" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="330432607" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2023 22:33:28 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10612"; a="659744606" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="659744606" Received: from baoyumen-mobl.ccr.corp.intel.com (HELO yhuang6-mobl2.smartont.net) ([10.255.30.227]) by orsmga007-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2023 22:33:22 -0800 From: Huang Ying <ying.huang@intel.com> To: Andrew Morton <akpm@linux-foundation.org> Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, "Huang, Ying" <ying.huang@intel.com>, Zi Yan <ziy@nvidia.com>, Yang Shi <shy828301@gmail.com>, Baolin Wang <baolin.wang@linux.alibaba.com>, Oscar Salvador <osalvador@suse.de>, Matthew Wilcox <willy@infradead.org>, Bharata B Rao <bharata@amd.com>, Alistair Popple <apopple@nvidia.com>, haoxin <xhao@linux.alibaba.com>, Minchan Kim <minchan@kernel.org>, Mike Kravetz <mike.kravetz@oracle.com>, Hyeonggon Yoo <42.hyeyoo@gmail.com> Subject: [PATCH -v4 0/9] migrate_pages(): batch TLB flushing Date: Mon, 6 Feb 2023 14:33:04 +0800 Message-Id: <20230206063313.635011-1-ying.huang@intel.com> X-Mailer: git-send-email 2.35.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757062407421383561?= X-GMAIL-MSGID: =?utf-8?q?1757062407421383561?= |
Series |
migrate_pages(): batch TLB flushing
|
|
Message
Huang, Ying
Feb. 6, 2023, 6:33 a.m. UTC
From: "Huang, Ying" <ying.huang@intel.com>
Now, migrate_pages() migrate folios one by one, like the fake code as
follows,
for each folio
unmap
flush TLB
copy
restore map
If multiple folios are passed to migrate_pages(), there are
opportunities to batch the TLB flushing and copying. That is, we can
change the code to something as follows,
for each folio
unmap
for each folio
flush TLB
for each folio
copy
for each folio
restore map
The total number of TLB flushing IPI can be reduced considerably. And
we may use some hardware accelerator such as DSA to accelerate the
folio copying.
So in this patch, we refactor the migrate_pages() implementation and
implement the TLB flushing batching. Base on this, hardware
accelerated folio copying can be implemented.
If too many folios are passed to migrate_pages(), in the naive batched
implementation, we may unmap too many folios at the same time. The
possibility for a task to wait for the migrated folios to be mapped
again increases. So the latency may be hurt. To deal with this
issue, the max number of folios be unmapped in batch is restricted to
no more than HPAGE_PMD_NR in the unit of page. That is, the influence
is at the same level of THP migration.
We use the following test to measure the performance impact of the
patchset,
On a 2-socket Intel server,
- Run pmbench memory accessing benchmark
- Run `migratepages` to migrate pages of pmbench between node 0 and
node 1 back and forth.
With the patch, the TLB flushing IPI reduces 99.1% during the test and
the number of pages migrated successfully per second increases 291.7%.
This patchset is based on v6.2-rc4.
Changes:
v4:
- Fixed another bug about non-LRU folio migration. Thanks Hyeonggon!
v3:
- Rebased on v6.2-rc4
- Fixed a bug about non-LRU folio migration. Thanks Mike!
- Fixed some comments. Thanks Baolin!
- Collected reviewed-by.
v2:
- Rebased on v6.2-rc3
- Fixed type force cast warning. Thanks Kees!
- Added more comments and cleaned up the code. Thanks Andrew, Zi, Alistair, Dan!
- Collected reviewed-by.
from rfc to v1:
- Rebased on v6.2-rc1
- Fix the deadlock issue caused by locking multiple pages synchronously
per Alistair's comments. Thanks!
- Fix the autonumabench panic per Rao's comments and fix. Thanks!
- Other minor fixes per comments. Thanks!
Best Regards,
Huang, Ying