Message ID | 20230618065824.1365750-1-yosryahmed@google.com |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp2341121vqr; Sun, 18 Jun 2023 00:50:37 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ43YKOqD+xWH0NQAlwkJQAoOzyJZZT9EXdRggfL15MPN9ASNlWcc0x8qvfoqeKNb5oqTJRA X-Received: by 2002:a05:6359:62a:b0:12f:91d6:3234 with SMTP id eh42-20020a056359062a00b0012f91d63234mr1798808rwb.14.1687074637151; Sun, 18 Jun 2023 00:50:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1687074637; cv=none; d=google.com; s=arc-20160816; b=UmqfTpmlpt3LKywO63GV/droVN7qdIwZctip9Ykrp7quiHIz/iZ+fpv/yi0U8FE5yU 7+G5SovqLLmXYyTdQX4xIU+P1EM6mfbiXZhX1YUZWd817me7Sd8aaMLN1UUB38OYbC7b Hz77xeXIz/irzMDfCbX8FjNQBpbI42aIRxpgQKcOoIFeAOo8v0g9L7R+9oZKBVbBMA97 ycXDQTuszKMdZxgU0qmQfsd8B59l/ep1LpAj++BIJNvmQItXf024FcyOvdpeRfxr9wB/ OCxm/9ZeIJ2lBxLx26MHFMFHUq2aBF5sQNi10B9zWsHFuzCHtt3Wsoi+Qu13/vE38qUd a5cA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:mime-version:date :dkim-signature; bh=h9MkEXCOECVy+F9lVXmcJqM8lpLvl13EAFxuWhnTpvI=; b=U3ZP59y22mqBe1t6t8Z+WlRCk6leY7GQqYlXSUB/Yv4ncKNbZf9Mt55VdMp+evaT6X mkDtAUQwvtra81zG7IgG4ebr5WcbEBNaQtpCdGzT0gWFrH3df4MkXb2rch1QOU0nNhMB PpnU/1HHl10d7MO+sIIs4qNEiEcGsBx1J6dQEJS4RNnB54/Sq7lZyqGik4rDEEoqvPYa ePLJLUBPCFd9rWCoM7JSD0YDAL6zXBjQmqGfhjZ0aL7oxU3LCwbcZImYB9HLQ4NxRZay PZkVu9puRAU8qFfPLQSlrK+1woF8bT+/r56NpJ+pHmSX9zsjclztPc+asK/IMsj5I2JX 9lWg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=Jmn08JRk; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id u14-20020a170902e5ce00b001b055bd62f6si2719138plf.431.2023.06.18.00.50.10; Sun, 18 Jun 2023 00:50:37 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=Jmn08JRk; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229529AbjFRG6o (ORCPT <rfc822;duw91626@gmail.com> + 99 others); Sun, 18 Jun 2023 02:58:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52584 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229555AbjFRG6i (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Sun, 18 Jun 2023 02:58:38 -0400 Received: from mail-pg1-x549.google.com (mail-pg1-x549.google.com [IPv6:2607:f8b0:4864:20::549]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F2C141707 for <linux-kernel@vger.kernel.org>; Sat, 17 Jun 2023 23:58:27 -0700 (PDT) Received: by mail-pg1-x549.google.com with SMTP id 41be03b00d2f7-517bad1b8c5so1961895a12.0 for <linux-kernel@vger.kernel.org>; Sat, 17 Jun 2023 23:58:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1687071507; x=1689663507; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=h9MkEXCOECVy+F9lVXmcJqM8lpLvl13EAFxuWhnTpvI=; b=Jmn08JRk9haBdTN7W4jKc018AW8wTh5/Vt4hWrG22Ce5WHHa5gH8+MFIzo5RR/XZDq /W6Qm/trFt2lkCHmg+/8aK4c3BVvqvUZGoipdn0DwQhEJG0zDsVcqfnYXrvXqWqFMSwB cD5AXD9nnEOim2ftXAtIhqlPxnHlllsa63199K3JNAxq5hr4CMKl6vYbDBMBcYoqGoJI TojGOAjg6KS0exeu21OEEbX34hPDKOcGV9jJ6KSr7JDko1JBsl94JVx3mVI4VmF+a4zE hOk3HCxA72ztJwA4CdUKzWKxOVZY+VQuQRRxrmIiJ5l2lm8iS7CGxZY5s5RVjVkMk7B+ 4BIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687071507; x=1689663507; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=h9MkEXCOECVy+F9lVXmcJqM8lpLvl13EAFxuWhnTpvI=; b=D8YxhZZllbc6hhbY9GL6VSt/PWsPqex6+wQUbLUeoVlcZoxJXbWLgEP0H7vMs3fg9V qx/lIjYNafpyPjruMuTLKYXKHXQ3iqznSDJNUlIK+oC2ulztZopCgjKjbjQz2IKadfMg 4EHy8xUKW5sCkHVpIY5j4ip+YUihNwMEahh7INDR0pO/olbnqE1wO+GcHoKvZ1eS6ux3 6TC35oWTH8/mINhbSzg8IRJBmgKsOXEYAna9G8XL9dwaCyxPE+kgtLU2ac8iwApHkFN+ fQG07QZYwXnYYgp7dNKfdn/Tr2x1HzGDg3x+sUTTw7831g02S5saPcn6z2+GAVsvUOB+ hNdQ== X-Gm-Message-State: AC+VfDwGmG23Nut34LTWvUDqAfFfAYtyk3WJUUae2KRlmRfsd5kngW1n TWTS4OSLBVTfC0FO+bZVXD5IxcXfHIw2WQNu X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a63:f106:0:b0:53f:2d21:5a16 with SMTP id f6-20020a63f106000000b0053f2d215a16mr450864pgi.10.1687071507411; Sat, 17 Jun 2023 23:58:27 -0700 (PDT) Date: Sun, 18 Jun 2023 06:58:24 +0000 Mime-Version: 1.0 X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230618065824.1365750-1-yosryahmed@google.com> Subject: [RFC PATCH 5/5] Revert "mm/migrate: __unmap_and_move() push good newpage to LRU" From: Yosry Ahmed <yosryahmed@google.com> To: Andrew Morton <akpm@linux-foundation.org> Cc: Yu Zhao <yuzhao@google.com>, "Jan Alexander Steffens (heftig)" <heftig@archlinux.org>, Steven Barrett <steven@liquorix.net>, Brian Geffon <bgeffon@google.com>, "T.J. Alumbaugh" <talumbau@google.com>, Gaosheng Cui <cuigaosheng1@huawei.com>, Suren Baghdasaryan <surenb@google.com>, "Matthew Wilcox (Oracle)" <willy@infradead.org>, "Liam R. Howlett" <Liam.Howlett@oracle.com>, David Hildenbrand <david@redhat.com>, Jason Gunthorpe <jgg@ziepe.ca>, Mathieu Desnoyers <mathieu.desnoyers@efficios.com>, David Howells <dhowells@redhat.com>, Hugh Dickins <hughd@google.com>, Greg Thelen <gthelen@google.com>, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yosry Ahmed <yosryahmed@google.com> Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1769025974546822305?= X-GMAIL-MSGID: =?utf-8?q?1769025974546822305?= |
Series |
[RFC,1/5] mm/mlock: rework mlock_count to use _mapcount for order-0 folios
|
|
Commit Message
Yosry Ahmed
June 18, 2023, 6:58 a.m. UTC
This reverts commit c3096e6782b733158bf34f6bbb4567808d4e0740.
That commit made sure we immediately add the new page to the LRU before
remove_migration_ptes() is called in migrate_move_folio() (used to be
__unmap_and_move() back then), such that the rmap walk will rebuild the
correct mlock_count for the page again. This was needed because the
mlock_count was lost when the page is isolated. This is no longer the
case since mlock_count no longer overlays page->lru.
Revert the commit (the code was foliated afterward the commit, so the
revert is updated as such).
Signed-off-by: Yosry Ahmed <yosryahmed@google.com>
---
mm/migrate.c | 24 +++++++++---------------
1 file changed, 9 insertions(+), 15 deletions(-)
Comments
Hi, Yosry, Yosry Ahmed <yosryahmed@google.com> writes: > This reverts commit c3096e6782b733158bf34f6bbb4567808d4e0740. > > That commit made sure we immediately add the new page to the LRU before > remove_migration_ptes() is called in migrate_move_folio() (used to be > __unmap_and_move() back then), such that the rmap walk will rebuild the > correct mlock_count for the page again. This was needed because the > mlock_count was lost when the page is isolated. This is no longer the > case since mlock_count no longer overlays page->lru. > > Revert the commit (the code was foliated afterward the commit, so the > revert is updated as such). > > Signed-off-by: Yosry Ahmed <yosryahmed@google.com> > --- > mm/migrate.c | 24 +++++++++--------------- > 1 file changed, 9 insertions(+), 15 deletions(-) > > diff --git a/mm/migrate.c b/mm/migrate.c > index 01cac26a3127..68f693731865 100644 > --- a/mm/migrate.c > +++ b/mm/migrate.c > @@ -1279,19 +1279,6 @@ static int migrate_folio_move(free_page_t put_new_page, unsigned long private, > if (unlikely(!is_lru)) > goto out_unlock_both; The patch itself looks good to me! Thanks! Reviewed-by: "Huang, Ying" <ying.huang@intel.com> And, it seems that we can remove the above 2 lines and "out_unlock_both" label now. That can make the code simpler a little. Right? Best Regards, Huang, Ying > - /* > - * When successful, push dst to LRU immediately: so that if it > - * turns out to be an mlocked page, remove_migration_ptes() will > - * automatically build up the correct dst->mlock_count for it. > - * > - * We would like to do something similar for the old page, when > - * unsuccessful, and other cases when a page has been temporarily > - * isolated from the unevictable LRU: but this case is the easiest. > - */ > - folio_add_lru(dst); > - if (page_was_mapped) > - lru_add_drain(); > - > if (page_was_mapped) > remove_migration_ptes(src, dst, false); > > @@ -1301,9 +1288,16 @@ static int migrate_folio_move(free_page_t put_new_page, unsigned long private, > /* > * If migration is successful, decrease refcount of dst, > * which will not free the page because new page owner increased > - * refcounter. > + * refcounter. As well, if it is LRU folio, add the folio to LRU > + * list in here. Use the old state of the isolated source folio to > + * determine if we migrated a LRU folio. dst was already unlocked > + * and possibly modified by its owner - don't rely on the folio > + * state. > */ > - folio_put(dst); > + if (unlikely(!is_lru)) > + folio_put(dst); > + else > + folio_putback_lru(dst); > > /* > * A folio that has been migrated has all references removed
On Sun, Jun 18, 2023 at 7:00 PM Huang, Ying <ying.huang@intel.com> wrote: > > Hi, Yosry, > > Yosry Ahmed <yosryahmed@google.com> writes: > > > This reverts commit c3096e6782b733158bf34f6bbb4567808d4e0740. > > > > That commit made sure we immediately add the new page to the LRU before > > remove_migration_ptes() is called in migrate_move_folio() (used to be > > __unmap_and_move() back then), such that the rmap walk will rebuild the > > correct mlock_count for the page again. This was needed because the > > mlock_count was lost when the page is isolated. This is no longer the > > case since mlock_count no longer overlays page->lru. > > > > Revert the commit (the code was foliated afterward the commit, so the > > revert is updated as such). > > > > Signed-off-by: Yosry Ahmed <yosryahmed@google.com> > > --- > > mm/migrate.c | 24 +++++++++--------------- > > 1 file changed, 9 insertions(+), 15 deletions(-) > > > > diff --git a/mm/migrate.c b/mm/migrate.c > > index 01cac26a3127..68f693731865 100644 > > --- a/mm/migrate.c > > +++ b/mm/migrate.c > > @@ -1279,19 +1279,6 @@ static int migrate_folio_move(free_page_t put_new_page, unsigned long private, > > if (unlikely(!is_lru)) > > goto out_unlock_both; > > The patch itself looks good to me! Thanks! > > Reviewed-by: "Huang, Ying" <ying.huang@intel.com> Thanks for taking a look! > > And, it seems that we can remove the above 2 lines and "out_unlock_both" > label now. That can make the code simpler a little. Right? I am not familiar with this code. If we remove the above condition then pages that have is_lru == 0 (i.e __PageMovable(src) is true) and page_was_mapped == 1 will call remove_migration_ptes(). This wouldn't happen without removing the above 2 lines. If this combination is impossible (is_lru == 0 && page_was_mapped == 1), then yeah we can remove the above condition. It looks like __SetPageMovable() is only called by zsmalloc, z3fold, and balloon_page_insert(). The former 2 will never have those pages mapped into userspace. I am not familiar with balloon_page_insert(), but my gut feeling is that those are pages used by the driver and are also not mapped into userspace. So I guess we can just remove the condition, but a confirmation for the above would be reassuring :) > > Best Regards, > Huang, Ying > > > - /* > > - * When successful, push dst to LRU immediately: so that if it > > - * turns out to be an mlocked page, remove_migration_ptes() will > > - * automatically build up the correct dst->mlock_count for it. > > - * > > - * We would like to do something similar for the old page, when > > - * unsuccessful, and other cases when a page has been temporarily > > - * isolated from the unevictable LRU: but this case is the easiest. > > - */ > > - folio_add_lru(dst); > > - if (page_was_mapped) > > - lru_add_drain(); > > - > > if (page_was_mapped) > > remove_migration_ptes(src, dst, false); > > > > @@ -1301,9 +1288,16 @@ static int migrate_folio_move(free_page_t put_new_page, unsigned long private, > > /* > > * If migration is successful, decrease refcount of dst, > > * which will not free the page because new page owner increased > > - * refcounter. > > + * refcounter. As well, if it is LRU folio, add the folio to LRU > > + * list in here. Use the old state of the isolated source folio to > > + * determine if we migrated a LRU folio. dst was already unlocked > > + * and possibly modified by its owner - don't rely on the folio > > + * state. > > */ > > - folio_put(dst); > > + if (unlikely(!is_lru)) > > + folio_put(dst); > > + else > > + folio_putback_lru(dst); > > > > /* > > * A folio that has been migrated has all references removed >
Yosry Ahmed <yosryahmed@google.com> writes: > On Sun, Jun 18, 2023 at 7:00 PM Huang, Ying <ying.huang@intel.com> wrote: >> >> Hi, Yosry, >> >> Yosry Ahmed <yosryahmed@google.com> writes: >> >> > This reverts commit c3096e6782b733158bf34f6bbb4567808d4e0740. >> > >> > That commit made sure we immediately add the new page to the LRU before >> > remove_migration_ptes() is called in migrate_move_folio() (used to be >> > __unmap_and_move() back then), such that the rmap walk will rebuild the >> > correct mlock_count for the page again. This was needed because the >> > mlock_count was lost when the page is isolated. This is no longer the >> > case since mlock_count no longer overlays page->lru. >> > >> > Revert the commit (the code was foliated afterward the commit, so the >> > revert is updated as such). >> > >> > Signed-off-by: Yosry Ahmed <yosryahmed@google.com> >> > --- >> > mm/migrate.c | 24 +++++++++--------------- >> > 1 file changed, 9 insertions(+), 15 deletions(-) >> > >> > diff --git a/mm/migrate.c b/mm/migrate.c >> > index 01cac26a3127..68f693731865 100644 >> > --- a/mm/migrate.c >> > +++ b/mm/migrate.c >> > @@ -1279,19 +1279,6 @@ static int migrate_folio_move(free_page_t put_new_page, unsigned long private, >> > if (unlikely(!is_lru)) >> > goto out_unlock_both; >> >> The patch itself looks good to me! Thanks! >> >> Reviewed-by: "Huang, Ying" <ying.huang@intel.com> > > Thanks for taking a look! > >> >> And, it seems that we can remove the above 2 lines and "out_unlock_both" >> label now. That can make the code simpler a little. Right? > > I am not familiar with this code. If we remove the above condition > then pages that have is_lru == 0 (i.e __PageMovable(src) is true) and > page_was_mapped == 1 will call remove_migration_ptes(). This wouldn't > happen without removing the above 2 lines. If this combination is > impossible (is_lru == 0 && page_was_mapped == 1), then yeah we can > remove the above condition. > > It looks like __SetPageMovable() is only called by zsmalloc, z3fold, > and balloon_page_insert(). The former 2 will never have those pages > mapped into userspace. I am not familiar with balloon_page_insert(), > but my gut feeling is that those are pages used by the driver and are > also not mapped into userspace. You can take a look at migrate_folio_unmap(), where "page_was_mapped" will not be set to 1 if !is_lru. Best Regards, Huang, Ying > So I guess we can just remove the condition, but a confirmation for > the above would be reassuring :) > >> >> Best Regards, >> Huang, Ying >> >> > - /* >> > - * When successful, push dst to LRU immediately: so that if it >> > - * turns out to be an mlocked page, remove_migration_ptes() will >> > - * automatically build up the correct dst->mlock_count for it. >> > - * >> > - * We would like to do something similar for the old page, when >> > - * unsuccessful, and other cases when a page has been temporarily >> > - * isolated from the unevictable LRU: but this case is the easiest. >> > - */ >> > - folio_add_lru(dst); >> > - if (page_was_mapped) >> > - lru_add_drain(); >> > - >> > if (page_was_mapped) >> > remove_migration_ptes(src, dst, false); >> > >> > @@ -1301,9 +1288,16 @@ static int migrate_folio_move(free_page_t put_new_page, unsigned long private, >> > /* >> > * If migration is successful, decrease refcount of dst, >> > * which will not free the page because new page owner increased >> > - * refcounter. >> > + * refcounter. As well, if it is LRU folio, add the folio to LRU >> > + * list in here. Use the old state of the isolated source folio to >> > + * determine if we migrated a LRU folio. dst was already unlocked >> > + * and possibly modified by its owner - don't rely on the folio >> > + * state. >> > */ >> > - folio_put(dst); >> > + if (unlikely(!is_lru)) >> > + folio_put(dst); >> > + else >> > + folio_putback_lru(dst); >> > >> > /* >> > * A folio that has been migrated has all references removed >>
On Sun, Jun 18, 2023 at 9:29 PM Huang, Ying <ying.huang@intel.com> wrote: > > Yosry Ahmed <yosryahmed@google.com> writes: > > > On Sun, Jun 18, 2023 at 7:00 PM Huang, Ying <ying.huang@intel.com> wrote: > >> > >> Hi, Yosry, > >> > >> Yosry Ahmed <yosryahmed@google.com> writes: > >> > >> > This reverts commit c3096e6782b733158bf34f6bbb4567808d4e0740. > >> > > >> > That commit made sure we immediately add the new page to the LRU before > >> > remove_migration_ptes() is called in migrate_move_folio() (used to be > >> > __unmap_and_move() back then), such that the rmap walk will rebuild the > >> > correct mlock_count for the page again. This was needed because the > >> > mlock_count was lost when the page is isolated. This is no longer the > >> > case since mlock_count no longer overlays page->lru. > >> > > >> > Revert the commit (the code was foliated afterward the commit, so the > >> > revert is updated as such). > >> > > >> > Signed-off-by: Yosry Ahmed <yosryahmed@google.com> > >> > --- > >> > mm/migrate.c | 24 +++++++++--------------- > >> > 1 file changed, 9 insertions(+), 15 deletions(-) > >> > > >> > diff --git a/mm/migrate.c b/mm/migrate.c > >> > index 01cac26a3127..68f693731865 100644 > >> > --- a/mm/migrate.c > >> > +++ b/mm/migrate.c > >> > @@ -1279,19 +1279,6 @@ static int migrate_folio_move(free_page_t put_new_page, unsigned long private, > >> > if (unlikely(!is_lru)) > >> > goto out_unlock_both; > >> > >> The patch itself looks good to me! Thanks! > >> > >> Reviewed-by: "Huang, Ying" <ying.huang@intel.com> > > > > Thanks for taking a look! > > > >> > >> And, it seems that we can remove the above 2 lines and "out_unlock_both" > >> label now. That can make the code simpler a little. Right? > > > > I am not familiar with this code. If we remove the above condition > > then pages that have is_lru == 0 (i.e __PageMovable(src) is true) and > > page_was_mapped == 1 will call remove_migration_ptes(). This wouldn't > > happen without removing the above 2 lines. If this combination is > > impossible (is_lru == 0 && page_was_mapped == 1), then yeah we can > > remove the above condition. > > > > It looks like __SetPageMovable() is only called by zsmalloc, z3fold, > > and balloon_page_insert(). The former 2 will never have those pages > > mapped into userspace. I am not familiar with balloon_page_insert(), > > but my gut feeling is that those are pages used by the driver and are > > also not mapped into userspace. > > You can take a look at migrate_folio_unmap(), where "page_was_mapped" > will not be set to 1 if !is_lru. Oh I was looking in the wrong place huh, thanks. Will remove it when I respin! > > Best Regards, > Huang, Ying > > > So I guess we can just remove the condition, but a confirmation for > > the above would be reassuring :) > > > >> > >> Best Regards, > >> Huang, Ying > >> > >> > - /* > >> > - * When successful, push dst to LRU immediately: so that if it > >> > - * turns out to be an mlocked page, remove_migration_ptes() will > >> > - * automatically build up the correct dst->mlock_count for it. > >> > - * > >> > - * We would like to do something similar for the old page, when > >> > - * unsuccessful, and other cases when a page has been temporarily > >> > - * isolated from the unevictable LRU: but this case is the easiest. > >> > - */ > >> > - folio_add_lru(dst); > >> > - if (page_was_mapped) > >> > - lru_add_drain(); > >> > - > >> > if (page_was_mapped) > >> > remove_migration_ptes(src, dst, false); > >> > > >> > @@ -1301,9 +1288,16 @@ static int migrate_folio_move(free_page_t put_new_page, unsigned long private, > >> > /* > >> > * If migration is successful, decrease refcount of dst, > >> > * which will not free the page because new page owner increased > >> > - * refcounter. > >> > + * refcounter. As well, if it is LRU folio, add the folio to LRU > >> > + * list in here. Use the old state of the isolated source folio to > >> > + * determine if we migrated a LRU folio. dst was already unlocked > >> > + * and possibly modified by its owner - don't rely on the folio > >> > + * state. > >> > */ > >> > - folio_put(dst); > >> > + if (unlikely(!is_lru)) > >> > + folio_put(dst); > >> > + else > >> > + folio_putback_lru(dst); > >> > > >> > /* > >> > * A folio that has been migrated has all references removed > >> >
On 19.06.23 05:59, Yosry Ahmed wrote: > On Sun, Jun 18, 2023 at 7:00 PM Huang, Ying <ying.huang@intel.com> wrote: >> >> Hi, Yosry, >> >> Yosry Ahmed <yosryahmed@google.com> writes: >> >>> This reverts commit c3096e6782b733158bf34f6bbb4567808d4e0740. >>> >>> That commit made sure we immediately add the new page to the LRU before >>> remove_migration_ptes() is called in migrate_move_folio() (used to be >>> __unmap_and_move() back then), such that the rmap walk will rebuild the >>> correct mlock_count for the page again. This was needed because the >>> mlock_count was lost when the page is isolated. This is no longer the >>> case since mlock_count no longer overlays page->lru. >>> >>> Revert the commit (the code was foliated afterward the commit, so the >>> revert is updated as such). >>> >>> Signed-off-by: Yosry Ahmed <yosryahmed@google.com> >>> --- >>> mm/migrate.c | 24 +++++++++--------------- >>> 1 file changed, 9 insertions(+), 15 deletions(-) >>> >>> diff --git a/mm/migrate.c b/mm/migrate.c >>> index 01cac26a3127..68f693731865 100644 >>> --- a/mm/migrate.c >>> +++ b/mm/migrate.c >>> @@ -1279,19 +1279,6 @@ static int migrate_folio_move(free_page_t put_new_page, unsigned long private, >>> if (unlikely(!is_lru)) >>> goto out_unlock_both; >> >> The patch itself looks good to me! Thanks! >> >> Reviewed-by: "Huang, Ying" <ying.huang@intel.com> > > Thanks for taking a look! > >> >> And, it seems that we can remove the above 2 lines and "out_unlock_both" >> label now. That can make the code simpler a little. Right? > > I am not familiar with this code. If we remove the above condition > then pages that have is_lru == 0 (i.e __PageMovable(src) is true) and > page_was_mapped == 1 will call remove_migration_ptes(). This wouldn't > happen without removing the above 2 lines. If this combination is > impossible (is_lru == 0 && page_was_mapped == 1), then yeah we can > remove the above condition. > > It looks like __SetPageMovable() is only called by zsmalloc, z3fold, > and balloon_page_insert(). The former 2 will never have those pages > mapped into userspace. I am not familiar with balloon_page_insert(), > but my gut feeling is that those are pages used by the driver and are > also not mapped into userspace. On XEN, there is xen_alloc_ballooned_pages(), which ends up mapping balloon-inflated pages into user space (for something like MMIO IIRC). But the XEN balloon does not use the balloon compaction framework, so __SetPageMovable() does not apply. The other balloon_page_insert() users (VMware balloon, CMM, virtio-balloon) shouldn't be doing something like that.
On 19.06.23 09:56, David Hildenbrand wrote: > On 19.06.23 05:59, Yosry Ahmed wrote: >> On Sun, Jun 18, 2023 at 7:00 PM Huang, Ying <ying.huang@intel.com> wrote: >>> >>> Hi, Yosry, >>> >>> Yosry Ahmed <yosryahmed@google.com> writes: >>> >>>> This reverts commit c3096e6782b733158bf34f6bbb4567808d4e0740. >>>> >>>> That commit made sure we immediately add the new page to the LRU before >>>> remove_migration_ptes() is called in migrate_move_folio() (used to be >>>> __unmap_and_move() back then), such that the rmap walk will rebuild the >>>> correct mlock_count for the page again. This was needed because the >>>> mlock_count was lost when the page is isolated. This is no longer the >>>> case since mlock_count no longer overlays page->lru. >>>> >>>> Revert the commit (the code was foliated afterward the commit, so the >>>> revert is updated as such). >>>> >>>> Signed-off-by: Yosry Ahmed <yosryahmed@google.com> >>>> --- >>>> mm/migrate.c | 24 +++++++++--------------- >>>> 1 file changed, 9 insertions(+), 15 deletions(-) >>>> >>>> diff --git a/mm/migrate.c b/mm/migrate.c >>>> index 01cac26a3127..68f693731865 100644 >>>> --- a/mm/migrate.c >>>> +++ b/mm/migrate.c >>>> @@ -1279,19 +1279,6 @@ static int migrate_folio_move(free_page_t put_new_page, unsigned long private, >>>> if (unlikely(!is_lru)) >>>> goto out_unlock_both; >>> >>> The patch itself looks good to me! Thanks! >>> >>> Reviewed-by: "Huang, Ying" <ying.huang@intel.com> >> >> Thanks for taking a look! >> >>> >>> And, it seems that we can remove the above 2 lines and "out_unlock_both" >>> label now. That can make the code simpler a little. Right? >> >> I am not familiar with this code. If we remove the above condition >> then pages that have is_lru == 0 (i.e __PageMovable(src) is true) and >> page_was_mapped == 1 will call remove_migration_ptes(). This wouldn't >> happen without removing the above 2 lines. If this combination is >> impossible (is_lru == 0 && page_was_mapped == 1), then yeah we can >> remove the above condition. >> >> It looks like __SetPageMovable() is only called by zsmalloc, z3fold, >> and balloon_page_insert(). The former 2 will never have those pages >> mapped into userspace. I am not familiar with balloon_page_insert(), >> but my gut feeling is that those are pages used by the driver and are >> also not mapped into userspace. > > On XEN, there is xen_alloc_ballooned_pages(), which ends up mapping > balloon-inflated pages into user space (for something like MMIO IIRC). > But the XEN balloon does not use the balloon compaction framework, so > __SetPageMovable() does not apply. > > The other balloon_page_insert() users (VMware balloon, CMM, > virtio-balloon) shouldn't be doing something like that. Ah, and I remember they even can't, because in balloon_page_insert() we also do a __SetPageOffline(). And such typed pages cannot be mapped into user space (because the type overlays the mapcount).
On Mon, Jun 19, 2023 at 12:58 AM David Hildenbrand <david@redhat.com> wrote: > > On 19.06.23 09:56, David Hildenbrand wrote: > > On 19.06.23 05:59, Yosry Ahmed wrote: > >> On Sun, Jun 18, 2023 at 7:00 PM Huang, Ying <ying.huang@intel.com> wrote: > >>> > >>> Hi, Yosry, > >>> > >>> Yosry Ahmed <yosryahmed@google.com> writes: > >>> > >>>> This reverts commit c3096e6782b733158bf34f6bbb4567808d4e0740. > >>>> > >>>> That commit made sure we immediately add the new page to the LRU before > >>>> remove_migration_ptes() is called in migrate_move_folio() (used to be > >>>> __unmap_and_move() back then), such that the rmap walk will rebuild the > >>>> correct mlock_count for the page again. This was needed because the > >>>> mlock_count was lost when the page is isolated. This is no longer the > >>>> case since mlock_count no longer overlays page->lru. > >>>> > >>>> Revert the commit (the code was foliated afterward the commit, so the > >>>> revert is updated as such). > >>>> > >>>> Signed-off-by: Yosry Ahmed <yosryahmed@google.com> > >>>> --- > >>>> mm/migrate.c | 24 +++++++++--------------- > >>>> 1 file changed, 9 insertions(+), 15 deletions(-) > >>>> > >>>> diff --git a/mm/migrate.c b/mm/migrate.c > >>>> index 01cac26a3127..68f693731865 100644 > >>>> --- a/mm/migrate.c > >>>> +++ b/mm/migrate.c > >>>> @@ -1279,19 +1279,6 @@ static int migrate_folio_move(free_page_t put_new_page, unsigned long private, > >>>> if (unlikely(!is_lru)) > >>>> goto out_unlock_both; > >>> > >>> The patch itself looks good to me! Thanks! > >>> > >>> Reviewed-by: "Huang, Ying" <ying.huang@intel.com> > >> > >> Thanks for taking a look! > >> > >>> > >>> And, it seems that we can remove the above 2 lines and "out_unlock_both" > >>> label now. That can make the code simpler a little. Right? > >> > >> I am not familiar with this code. If we remove the above condition > >> then pages that have is_lru == 0 (i.e __PageMovable(src) is true) and > >> page_was_mapped == 1 will call remove_migration_ptes(). This wouldn't > >> happen without removing the above 2 lines. If this combination is > >> impossible (is_lru == 0 && page_was_mapped == 1), then yeah we can > >> remove the above condition. > >> > >> It looks like __SetPageMovable() is only called by zsmalloc, z3fold, > >> and balloon_page_insert(). The former 2 will never have those pages > >> mapped into userspace. I am not familiar with balloon_page_insert(), > >> but my gut feeling is that those are pages used by the driver and are > >> also not mapped into userspace. > > > > On XEN, there is xen_alloc_ballooned_pages(), which ends up mapping > > balloon-inflated pages into user space (for something like MMIO IIRC). > > But the XEN balloon does not use the balloon compaction framework, so > > __SetPageMovable() does not apply. > > > > The other balloon_page_insert() users (VMware balloon, CMM, > > virtio-balloon) shouldn't be doing something like that. > > Ah, and I remember they even can't, because in balloon_page_insert() we > also do a __SetPageOffline(). And such typed pages cannot be mapped into > user space (because the type overlays the mapcount). Thanks David, good to know! I will remove the condition as Ying suggested in the next version then! > > -- > Cheers, > > David / dhildenb >
diff --git a/mm/migrate.c b/mm/migrate.c index 01cac26a3127..68f693731865 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -1279,19 +1279,6 @@ static int migrate_folio_move(free_page_t put_new_page, unsigned long private, if (unlikely(!is_lru)) goto out_unlock_both; - /* - * When successful, push dst to LRU immediately: so that if it - * turns out to be an mlocked page, remove_migration_ptes() will - * automatically build up the correct dst->mlock_count for it. - * - * We would like to do something similar for the old page, when - * unsuccessful, and other cases when a page has been temporarily - * isolated from the unevictable LRU: but this case is the easiest. - */ - folio_add_lru(dst); - if (page_was_mapped) - lru_add_drain(); - if (page_was_mapped) remove_migration_ptes(src, dst, false); @@ -1301,9 +1288,16 @@ static int migrate_folio_move(free_page_t put_new_page, unsigned long private, /* * If migration is successful, decrease refcount of dst, * which will not free the page because new page owner increased - * refcounter. + * refcounter. As well, if it is LRU folio, add the folio to LRU + * list in here. Use the old state of the isolated source folio to + * determine if we migrated a LRU folio. dst was already unlocked + * and possibly modified by its owner - don't rely on the folio + * state. */ - folio_put(dst); + if (unlikely(!is_lru)) + folio_put(dst); + else + folio_putback_lru(dst); /* * A folio that has been migrated has all references removed