From patchwork Tue Jan 2 23:32:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suren Baghdasaryan X-Patchwork-Id: 184539 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:6f82:b0:100:9c79:88ff with SMTP id tb2csp4737310dyb; Tue, 2 Jan 2024 15:33:17 -0800 (PST) X-Google-Smtp-Source: AGHT+IGMYxs2BPgQOJBRCsbBVDeUrooIdu5dM4p1exkOEbXkYa0yoCgmKTFgD6cESgVHfIE1WihR X-Received: by 2002:a05:620a:4153:b0:781:6627:c64c with SMTP id k19-20020a05620a415300b007816627c64cmr14061011qko.1.1704238397737; Tue, 02 Jan 2024 15:33:17 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1704238397; cv=none; d=google.com; s=arc-20160816; b=w3EMtSl1WPXVkJ+wko9Fc9KfMp5jQz//n8AAWV1JLb70ay9a31ModD2lvziiyw68k9 tp0XN5Lfa54Z4tSGk16C0kcZnwFt8qRQGAGkH1AIZRJMkHNoue06OehlyjTNFNETNfkJ TJycR6ZMKEHsDxmvd7a0kwF/fTkWMQYkLilv61WxdBXqzoGhWkSaelpCVdT5vpzEf3i/ s2y6/FMNMb8e2mfSvmqGxugGxSJYm2A0Q9bGIoPPO8eFTaqKmZJbrxjChaz1BjUIn78T IIujxY/mgQ4j0nCZJQpgktL2gBEOaEFHk+4V5pNXX7JW5qDfFhu8HGARUP/iJfTaTj18 4cxw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:from:subject:message-id:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:date:dkim-signature; bh=OrZsxOCrBJcDQG2F63wPu/oWesILjPUyL8XFF214j9w=; fh=L6siCmr5CdjcZAdYDK1kJZ2FvgUGOQXFZyHc5IS56hI=; b=fyun8cIMAcSMmS+IufvpaHrQib9WjCfDuIgsbDuQMnJ0K7RRP4D+jgxTFRKr42HQTS 5f4+3+/MDzWoY8lGM39wLTrJv50MQyujllKbq6nrJSMOmghKnaxTrOmZweYN9LrQYPog 7G2x/7dtbnr1x/h2mjjO6qBWXzM2ssjj2IKmHuwqrlrnE1DsTwihJ7dCNM9mwWOi27jq wiUSpmBeI1hjBahHm3unOf3siHBAgjYEPOOgBjO73HllcpTlXsOla8vDieJJMelBwkbS JlHYfyxQKPxjpQf5UUJz/ab01g/PYalPg1urNAvDqoQoLlnE00PLSsBRxmBFUkDbpHjF 8f3Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=wbEkI5us; spf=pass (google.com: domain of linux-kernel+bounces-14972-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-14972-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id bl34-20020a05620a1aa200b00781e4cd7879si732165qkb.479.2024.01.02.15.33.17 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jan 2024 15:33:17 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-14972-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=wbEkI5us; spf=pass (google.com: domain of linux-kernel+bounces-14972-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-14972-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 86CE31C21B03 for ; Tue, 2 Jan 2024 23:33:17 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 2938917995; Tue, 2 Jan 2024 23:33:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="wbEkI5us" X-Original-To: linux-kernel@vger.kernel.org Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 553FC17980 for ; Tue, 2 Jan 2024 23:33:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--surenb.bounces.google.com Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-dbe9dacc935so134236276.0 for ; Tue, 02 Jan 2024 15:33:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1704238379; x=1704843179; darn=vger.kernel.org; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=OrZsxOCrBJcDQG2F63wPu/oWesILjPUyL8XFF214j9w=; b=wbEkI5usZz2egk8Qhs2bsGoyngtDc57V7xwc5++UXaFTqmhNZfVl/lpS9bgoj6SrOu gORm049iIi/tbOltRAZtxJUCEsGnt5C2yTKcI5W1ujLeHn+jQgs6rz9bHlYUEil1XFfg zCCQbGIuKgdKX4yZ9v6j5nCuHlI9vlVSuvT3ep2ZQE42LpNcaLXATtENyZT+9LihZosF TbjE7GhD/hgctEtWUMBKoItVRk1fQscyBoxe6Is9zsB9K5YlF1ZNy/SfX9bFNfu9Q5aA xb3uRN6PhK32KEQtbXtbVWXCEQEtrdPz8WJy9cwQMNpkJ1gn4fks7QTe6S2dXKxsHL4j G9zQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1704238379; x=1704843179; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=OrZsxOCrBJcDQG2F63wPu/oWesILjPUyL8XFF214j9w=; b=aPRTD4dWKUFntfvT3DVm4TObfVgc8OV41HLH3raFOO0AVgM3s4u1tYAoR3YTkCHGZ0 5BZdzlsIrJlPHUHTkLPbTjFtA3YIj1/gVZY2BU2Y9OgXeZ0HLwJM9kipVix5Nd6zSdU4 E1QhGBMk9VZtgDiyn4D4PRPWhU3reej0SVQLJ+s7I0bIZhQK58DiLLkLc9TnsoRZGtQe mrEh8MysTd3QfIjzc/pdoos45ODLSdN0eZsnRmRHAN9aNBH0/QTm9fyjKJQtO4mkrcK+ dMZdjkJv6wkhAOgL51GDOFONdCNmClTs4LlhOYD6f83eEQ3WNmKsV/6NuS4MeVHLRxkb wWAQ== X-Gm-Message-State: AOJu0YzMjR4CYWz5fYkvin1oXDDEsA3BZGd4V1nv3vLV7p1jPjmKFMue WRzchOldbU2FTL++x4HUXRlCLXJtLqE8j88N/Q== X-Received: from surenb-desktop.mtv.corp.google.com ([2620:15c:211:201:7c15:8de:bd73:b2f6]) (user=surenb job=sendgmr) by 2002:a25:949:0:b0:dbc:1b8a:ea84 with SMTP id u9-20020a250949000000b00dbc1b8aea84mr417162ybm.7.1704238379350; Tue, 02 Jan 2024 15:32:59 -0800 (PST) Date: Tue, 2 Jan 2024 15:32:56 -0800 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 X-Mailer: git-send-email 2.43.0.472.g3155946c3a-goog Message-ID: <20240102233256.1077959-1-surenb@google.com> Subject: [PATCH v2 1/1] userfaultfd: fix move_pages_pte() splitting folio under RCU read lock From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: viro@zeniv.linux.org.uk, brauner@kernel.org, shuah@kernel.org, aarcange@redhat.com, lokeshgidra@google.com, peterx@redhat.com, david@redhat.com, ryan.roberts@arm.com, hughd@google.com, mhocko@suse.com, axelrasmussen@google.com, rppt@kernel.org, willy@infradead.org, Liam.Howlett@oracle.com, jannh@google.com, zhangpeng362@huawei.com, bgeffon@google.com, kaleshsingh@google.com, ngeoffray@google.com, jdduke@google.com, surenb@google.com, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, kernel-team@android.com X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1786673880881227452 X-GMAIL-MSGID: 1787023482127129503 While testing the split PMD path with lockdep enabled I've got an "Invalid wait context" error caused by split_huge_page_to_list() trying to lock anon_vma->rwsem while inside RCU read section. The issues is due to move_pages_pte() calling split_folio() under RCU read lock. Fix this by unmapping the PTEs and exiting RCU read section before splitting the folio and then retrying. The same retry pattern is used when locking the folio or anon_vma in this function. After splitting the large folio we unlock and release it because after the split the old folio might not be the one that contains the src_addr. Fixes: 94b01c885131 ("userfaultfd: UFFDIO_MOVE uABI") Signed-off-by: Suren Baghdasaryan Reviewed-by: Peter Xu --- Changes from v1 [1]: 1. Reset src_folio and src_folio_pte after folio is split, per Peter Xu [1] https://lore.kernel.org/all/20231230025607.2476912-1-surenb@google.com/ mm/userfaultfd.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index 5e718014e671..216ab4c8621f 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -1078,9 +1078,18 @@ static int move_pages_pte(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd, /* at this point we have src_folio locked */ if (folio_test_large(src_folio)) { + /* split_folio() can block */ + pte_unmap(&orig_src_pte); + pte_unmap(&orig_dst_pte); + src_pte = dst_pte = NULL; err = split_folio(src_folio); if (err) goto out; + /* have to reacquire the folio after it got split */ + folio_unlock(src_folio); + folio_put(src_folio); + src_folio = NULL; + goto retry; } if (!src_anon_vma) {