From patchwork Wed Jun 28 07:17:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suren Baghdasaryan X-Patchwork-Id: 11369 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp8771931vqr; Wed, 28 Jun 2023 01:57:06 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6NC7eGVFG+b2nV55FdLRmoHBm5krWm+m2Szo4Fgvw8N52lCl8eZrFIFQY83AKjterzw6GN X-Received: by 2002:a05:6402:27ca:b0:514:ab8b:ee78 with SMTP id c10-20020a05640227ca00b00514ab8bee78mr29245791ede.3.1687942626668; Wed, 28 Jun 2023 01:57:06 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1687942626; cv=none; d=google.com; s=arc-20160816; b=fgCDSXHX2HLdYzn1m4b+xIUnCFHY3yExvd5hDgcARynFxd2a7XV8aJVZZ9wqWwGWv4 epAnTXQInmlsUq3e0QPMoHZtfgDBqpVyTHH4EhIkEsYwPllzASEBf7Tz4PUCeI0aJFuf GzKPfn8hjT9j8SMyGHhCgEJ8msjpkfYn9SecAx91Km8bGFrLK74UtMKNkZ9pPLMgM9JN 3pLbbegTuASTPoz/jOiqI3/L/t7V1Je9DdtAWU8kKKF2BZBUEYW11VFtGKiKfWcYBM4d rgU7Od3EyxwVdgLysEIkNYJYOd3NP84jgQ91ijl8yCNcDwmn+uoHG00YoE3ylbdjI1Tj zmxA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:mime-version:date :dkim-signature; bh=gPRmZ26IBsiMsbW2DFKvtKQmDaUZH2fHIbrEB332iB8=; fh=RMSYO7NH5mgRq+Q9Sd8uvtKjjFjspH3ZqZT6M3GUwas=; b=t/dBDQoHu7PWZMPRW8WpIwvI0oE4oc4ojBnGXx4Z0vchEXhPD0ys41ke9/+crC6jdg VD+TK1DpZQ3v7I35x3tX0rttQRgh+EHOESZCKOY7MBtE4+NVpCzXrBid0VWjVF1ewJwU MERzrIzXJUzbPxq3MCY33uOf8p4yqhvFigLgP729F++fI63Cw1orRH49B3a2MESOi6ay SqfkuwbGw8Cv+RXKBiIhLqj660YJ698Z2CSHQbfV/iTITCM1ap9L7x2AcKelsDW38qYF UGtmfTjYe6SEJlFscHSTdmqWMjEP0EfbA0uCwBJsU+a0eeYgizRd4iRrMPVsvCLslNL+ cOTQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=DUmaXirb; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id u5-20020aa7d985000000b0051db006e404si1450900eds.168.2023.06.28.01.56.42; Wed, 28 Jun 2023 01:57:06 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=DUmaXirb; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235503AbjF1IhH (ORCPT + 99 others); Wed, 28 Jun 2023 04:37:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41898 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234445AbjF1Ic7 (ORCPT ); Wed, 28 Jun 2023 04:32:59 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 957572D78 for ; Wed, 28 Jun 2023 01:25:11 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-c38c87ea9ceso34999276.1 for ; Wed, 28 Jun 2023 01:25:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1687940711; x=1690532711; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=gPRmZ26IBsiMsbW2DFKvtKQmDaUZH2fHIbrEB332iB8=; b=DUmaXirbxVAivZHig+mjQqvfHcbOAHAnca1Z72aGuEntgSr/iBnDcYpQgUMwRDEnvL mvVm/wjUKLqf4xCSzIzklR+aizu1YOUMXJeKLjG7jK9tB3/EF5vxoAHu2hB+InPJUlHq LIrlxKFhydvQMZ9P1yHUmsqgWV2QL+1Hl2QKWHZ2iNqnjU20S/J5tlpRsPM3xpLxDpTU WKtj34tLVgkhOVKe7kbqC2RIEGJXtURP/8lruoUYpJaHFlCf+tHZARjhDLEXyhzaeuAX e4RfJq29jVDofVaNTmJ9F84/2qGOHNlXTN1Pn4YHrgzLwkvI+M+M9aURQuA0XnXhLV9I 9TXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687940711; x=1690532711; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=gPRmZ26IBsiMsbW2DFKvtKQmDaUZH2fHIbrEB332iB8=; b=TJ9AnSG/GHI33MXGJNuDM1Jnm/IKITCtZ2phT1Sa6knFyUhnG4JxWX/lT5rxa6Na7p sT56Z7VLMLrY5dnmTBzMS8JdiZKy5EVXNp2gOpY4gW+2YKDLsG7nh4bp9ELxWw5MLZp1 qMUOaO8IrAiggzqStatNCf3i15ciAnzrN7x67BEULFr1DYyfVf+GJPXRMfIR9IK0XnIq uezLZHmXGn/fHnRHWDSjA5h5b7Gd+FxTneUkqLwkMJSU0gD4raRLRmrikwvU3obYFgtf MEp3qg6tUWTcENAwU22t6J8wHEvjvPDlsy8YGFHUF/M8lmetBNlXtDMW2f9LlSuz5sO0 IzhQ== X-Gm-Message-State: ABy/qLafcfK3PdSVW86b7w65ChHoPeHlEYQ93rT58sDAdXuG2MomnwEa GdVxRc3YEuYqYXf0rkTOaWSrd7xecq0= X-Received: from surenb-desktop.mtv.corp.google.com ([2620:15c:211:201:6664:8bd3:57fd:c83a]) (user=surenb job=sendgmr) by 2002:a05:690c:2f82:b0:56d:5db:2f07 with SMTP id ew2-20020a05690c2f8200b0056d05db2f07mr7658ywb.5.1687936684846; Wed, 28 Jun 2023 00:18:04 -0700 (PDT) Date: Wed, 28 Jun 2023 00:17:54 -0700 Mime-Version: 1.0 X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230628071800.544800-1-surenb@google.com> Subject: [PATCH v4 0/6] Per-VMA lock support for swap and userfaults From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: willy@infradead.org, hannes@cmpxchg.org, mhocko@suse.com, josef@toxicpanda.com, jack@suse.cz, ldufour@linux.ibm.com, laurent.dufour@fr.ibm.com, michel@lespinasse.org, liam.howlett@oracle.com, jglisse@google.com, vbabka@suse.cz, minchan@google.com, dave@stgolabs.net, punit.agrawal@bytedance.com, lstoakes@gmail.com, hdanton@sina.com, apopple@nvidia.com, peterx@redhat.com, ying.huang@intel.com, david@redhat.com, yuzhao@google.com, dhowells@redhat.com, hughd@google.com, viro@zeniv.linux.org.uk, brauner@kernel.org, pasha.tatashin@soleen.com, surenb@google.com, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, kernel-team@android.com X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1769936127665691853?= X-GMAIL-MSGID: =?utf-8?q?1769936127665691853?= When per-VMA locks were introduced in [1] several types of page faults would still fall back to mmap_lock to keep the patchset simple. Among them are swap and userfault pages. The main reason for skipping those cases was the fact that mmap_lock could be dropped while handling these faults and that required additional logic to be implemented. Implement the mechanism to allow per-VMA locks to be dropped for these cases. First, change handle_mm_fault to drop per-VMA locks when returning VM_FAULT_RETRY or VM_FAULT_COMPLETED to be consistent with the way mmap_lock is handled. Then change folio_lock_or_retry to accept vm_fault and return vm_fault_t which simplifies later patches. Finally allow swap and uffd page faults to be handled under per-VMA locks by dropping per-VMA and retrying, the same way it's done under mmap_lock. Naturally, once VMA lock is dropped that VMA should be assumed unstable and can't be used. Changes since v3 posted at [2] - Renamed folio_lock_or_retry back to folio_lock_fault, per Peter Xu - Moved per-VMA lock release to where VM_FAULT_RETRY is returned, per Peter Xu - Dropped FAULT_FLAG_LOCK_DROPPED usage, per Peter Xu - Introduced release_fault_lock() helper function, per Peter Xu - Dropped the patch releasing per-VMA lock before migration_entry_wait, per Peter Xu - Introduced assert_fault_locked() helper function, per Peter Xu - Added BUG_ON to prevent FAULT_FLAG_RETRY_NOWAIT usage with per-VMA locks Note: patch 3/8 will cause a trivial merge conflict in arch/arm64/mm/fault.c when applied over mm-unstable branch due to a patch from ARM64 tree [3] which is missing in mm-unstable. [1] https://lore.kernel.org/all/20230227173632.3292573-1-surenb@google.com/ [2] https://lore.kernel.org/all/20230627042321.1763765-1-surenb@google.com/ [3] https://lore.kernel.org/all/20230524131305.2808-1-jszhang@kernel.org/ Suren Baghdasaryan (6): swap: remove remnants of polling from read_swap_cache_async mm: add missing VM_FAULT_RESULT_TRACE name for VM_FAULT_COMPLETED mm: drop per-VMA lock when returning VM_FAULT_RETRY or VM_FAULT_COMPLETED mm: change folio_lock_or_retry to use vm_fault directly mm: handle swap page faults under per-VMA lock mm: handle userfaults under VMA lock arch/arm64/mm/fault.c | 3 ++- arch/powerpc/mm/fault.c | 3 ++- arch/s390/mm/fault.c | 3 ++- arch/x86/mm/fault.c | 3 ++- fs/userfaultfd.c | 39 ++++++++++++++++++--------------------- include/linux/mm.h | 39 +++++++++++++++++++++++++++++++++++++++ include/linux/mm_types.h | 3 ++- include/linux/pagemap.h | 9 ++++----- mm/filemap.c | 37 +++++++++++++++++++------------------ mm/madvise.c | 4 ++-- mm/memory.c | 38 ++++++++++++++++---------------------- mm/swap.h | 1 - mm/swap_state.c | 12 +++++------- 13 files changed, 113 insertions(+), 81 deletions(-)