From patchwork Sun Oct 30 21:29:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 13059 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp1972620wru; Sun, 30 Oct 2022 14:32:39 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5LHJokbXyXL0GT7OEC6kwvcQbLaz0dgzqudrYncyVZzFyN4wphM6/FpVwCljZ1+ww+0iz6 X-Received: by 2002:a05:6a00:1145:b0:52b:78c:fa26 with SMTP id b5-20020a056a00114500b0052b078cfa26mr10805839pfm.27.1667165559250; Sun, 30 Oct 2022 14:32:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667165559; cv=none; d=google.com; s=arc-20160816; b=ZMQs0NTMrMID4uEWdiv2acw+PpeEaPiP8aR/T0qpVYsbqu30DRzeUsumJBpK9PUBnJ E3x2JZ3K9vY62uLlQxN1yNLMCFZdh+RrVlUtoRG72Be/mV1CKvGBj7hXuWCmGRvdo7nV nBt8u6m5h5x/k2fAEPA0liKIYiQKQRUwX11uZB/cWM6QEDWS/qzj6ipEUibLOFuRzufz AYFfT0PHKWBe5mkgDssFtB4MRqy9Du0XOT7G+fPMVOlnsHjoOvB9UEMR8+X7QvbarFRg GgiBPb3qcwwNWgxcVLRcRJV5G0rSAq0t+csyN5Y9sd4Nj9MVWuzC59C/QRUEmzQMInKS OCHg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=vgQM4vgy+7KnPWYFNhgrYbDAEMQY1XSSTWC+/NJ93CA=; b=Fjrg2+sGdsJqicjvvy0Biyv7EqiNNFuXIcGC5GiIEUKCyJnx4J9H1ongW4t5mMXPi6 m9xQgyLQwh4y7QkZgN8MumEFwML/cj6e9bYQLhKHIA0CEUD0W2jQW4Y9PW1g1gRQo6K9 ST0jJxWQ39aM9wnnk4qm84ybeWqrmV+m+d0t5iDBIUUKlShgZwGsb+N3M/8xk3+oNLvv quPZ3tk/HlibGbbACTMmaCcd5Zz0esKWvDzn5pSxLcLruWgVLEfSId7Y/5gUl69Tpkvk 3Bd2SanlE7alTMM/A3yVMok6zGS47FnGCRu8u+MUqHwOVPX7hmN1Lu8c54y5RCP54/mv i1Lw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=c2WcbyRM; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id ls14-20020a17090b350e00b00213defacf97si2069370pjb.163.2022.10.30.14.32.27; Sun, 30 Oct 2022 14:32:39 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=c2WcbyRM; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229714AbiJ3Vag (ORCPT + 99 others); Sun, 30 Oct 2022 17:30:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41848 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229651AbiJ3Vad (ORCPT ); Sun, 30 Oct 2022 17:30:33 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9CE91A190 for ; Sun, 30 Oct 2022 14:29:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1667165376; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vgQM4vgy+7KnPWYFNhgrYbDAEMQY1XSSTWC+/NJ93CA=; b=c2WcbyRMABi8Dplc0jXgTe7OJGJJMhuRRsv0NmtP5tbNj/uSou87A6leo4xWEuva1dMBvF hitqb5MaVttgCI4BuNlhzN2uzHkVjcy73vpoullBXdWTNgCEwlxWB6D6LsgnKc2p8QQ/T1 pb5f778ffOel+kznbPTpwpEb/VoyJOc= Received: from mail-qt1-f197.google.com (mail-qt1-f197.google.com [209.85.160.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-75-PM-fcrfCPQ-pwSPLlw_Csw-1; Sun, 30 Oct 2022 17:29:35 -0400 X-MC-Unique: PM-fcrfCPQ-pwSPLlw_Csw-1 Received: by mail-qt1-f197.google.com with SMTP id fb5-20020a05622a480500b003a525d52abcso655048qtb.10 for ; Sun, 30 Oct 2022 14:29:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vgQM4vgy+7KnPWYFNhgrYbDAEMQY1XSSTWC+/NJ93CA=; b=DYsghoWSS8o89aP8AF2ySpqs7uV7eTAqRh7e/eYLbSd9HEgCpkQ+9XWUaFEh6Q97qI Rev2/b6HTw9PcPgLjvGcUjohpQpBUuNCFeQ3zABsie2SdlipT+Ovwnw7iT0Jf+/qlmG2 82hx+Q32Ly8QNmjwxlLrA3J1qm5/rOBvJj31qL3XK0JJQN+ARFWfnzgYgmTM+46EQ5xi Nu9OfM1uR1PJwYbMJOZh1vQg+9M/EgbDVPXXR1SIrpC8gEdRr70nimLq6gHk1DwoCl3f UD0dxJWBVALWvwTsI1tRdlMSzK3KOvc8UznJq0rc3hCOlBNsqqLjpPBzX2QqpT3jOzgp R33Q== X-Gm-Message-State: ACrzQf3I27vqRc8mCWjOZELqEH6c2VcOKxUSVvK1rqdSTUZemGeypuZF z1rzBe8duTOfPIu1KIZq3roZUCuwLGDDc4pEV1YK/zBaTgrOPADUe62upRv5S4XB4B4FxfL+LRg EDK9vt4i9L3Vy2vtBqP7EVCzsv42BRbu7He4CSXLZTJ1XAcABIza7MqwEIG8yY6ncqsIgZAKUTQ == X-Received: by 2002:a05:620a:88a:b0:6f9:c32b:5fc6 with SMTP id b10-20020a05620a088a00b006f9c32b5fc6mr7043720qka.288.1667165374749; Sun, 30 Oct 2022 14:29:34 -0700 (PDT) X-Received: by 2002:a05:620a:88a:b0:6f9:c32b:5fc6 with SMTP id b10-20020a05620a088a00b006f9c32b5fc6mr7043701qka.288.1667165374468; Sun, 30 Oct 2022 14:29:34 -0700 (PDT) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id a1-20020ac81081000000b003a4b88b886esm2654781qtj.96.2022.10.30.14.29.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 30 Oct 2022 14:29:33 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Andrew Morton , James Houghton , Miaohe Lin , David Hildenbrand , Muchun Song , Andrea Arcangeli , Nadav Amit , Mike Kravetz , peterx@redhat.com, Rik van Riel Subject: [PATCH RFC 01/10] mm/hugetlb: Let vma_offset_start() to return start Date: Sun, 30 Oct 2022 17:29:20 -0400 Message-Id: <20221030212929.335473-2-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221030212929.335473-1-peterx@redhat.com> References: <20221030212929.335473-1-peterx@redhat.com> MIME-Version: 1.0 Content-type: text/plain X-Spam-Status: No, score=-3.2 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748149793050649237?= X-GMAIL-MSGID: =?utf-8?q?1748149793050649237?= Even though vma_offset_start() is named like that, it's not returning "the start address of the range" but rather the offset we should use to offset the vma->vm_start address. Make it return the real value of the start vaddr, and it also helps for all the callers because whenever the retval is used, it'll be ultimately added into the vma->vm_start anyway, so it's better. Signed-off-by: Peter Xu Reviewed-by: Mike Kravetz --- fs/hugetlbfs/inode.c | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c index 3ee84604e36d..ac3a69fe29c3 100644 --- a/fs/hugetlbfs/inode.c +++ b/fs/hugetlbfs/inode.c @@ -412,10 +412,12 @@ static bool hugetlb_vma_maps_page(struct vm_area_struct *vma, */ static unsigned long vma_offset_start(struct vm_area_struct *vma, pgoff_t start) { + unsigned long offset = 0; + if (vma->vm_pgoff < start) - return (start - vma->vm_pgoff) << PAGE_SHIFT; - else - return 0; + offset = (start - vma->vm_pgoff) << PAGE_SHIFT; + + return vma->vm_start + offset; } static unsigned long vma_offset_end(struct vm_area_struct *vma, pgoff_t end) @@ -457,7 +459,7 @@ static void hugetlb_unmap_file_folio(struct hstate *h, v_start = vma_offset_start(vma, start); v_end = vma_offset_end(vma, end); - if (!hugetlb_vma_maps_page(vma, vma->vm_start + v_start, page)) + if (!hugetlb_vma_maps_page(vma, v_start, page)) continue; if (!hugetlb_vma_trylock_write(vma)) { @@ -473,8 +475,8 @@ static void hugetlb_unmap_file_folio(struct hstate *h, break; } - unmap_hugepage_range(vma, vma->vm_start + v_start, v_end, - NULL, ZAP_FLAG_DROP_MARKER); + unmap_hugepage_range(vma, v_start, v_end, NULL, + ZAP_FLAG_DROP_MARKER); hugetlb_vma_unlock_write(vma); } @@ -507,10 +509,9 @@ static void hugetlb_unmap_file_folio(struct hstate *h, */ v_start = vma_offset_start(vma, start); v_end = vma_offset_end(vma, end); - if (hugetlb_vma_maps_page(vma, vma->vm_start + v_start, page)) - unmap_hugepage_range(vma, vma->vm_start + v_start, - v_end, NULL, - ZAP_FLAG_DROP_MARKER); + if (hugetlb_vma_maps_page(vma, v_start, page)) + unmap_hugepage_range(vma, v_start, v_end, NULL, + ZAP_FLAG_DROP_MARKER); kref_put(&vma_lock->refs, hugetlb_vma_lock_release); hugetlb_vma_unlock_write(vma); @@ -540,8 +541,7 @@ hugetlb_vmdelete_list(struct rb_root_cached *root, pgoff_t start, pgoff_t end, v_start = vma_offset_start(vma, start); v_end = vma_offset_end(vma, end); - unmap_hugepage_range(vma, vma->vm_start + v_start, v_end, - NULL, zap_flags); + unmap_hugepage_range(vma, v_start, v_end, NULL, zap_flags); /* * Note that vma lock only exists for shared/non-private From patchwork Sun Oct 30 21:29:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 13053 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp1972428wru; Sun, 30 Oct 2022 14:31:57 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6xkowMuCR0Zph2MkzgEFUrvdwBg7CuK4l3s4CaHlWrh6zHKudE9vl6n7PGEB1nu9c6HPXe X-Received: by 2002:a63:234b:0:b0:46b:208f:1e72 with SMTP id u11-20020a63234b000000b0046b208f1e72mr9325903pgm.581.1667165517175; Sun, 30 Oct 2022 14:31:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667165517; cv=none; d=google.com; s=arc-20160816; b=hnfWS+1Mv0E68hzUvGpZlf8tMvPkxU266N2Z7uR5rjMMmNRAs/9xtkI6sn5S+vFeVp Nl/6Ng8SNCa1c+fdYZXQr7UOiDHQxL3LVjcaSNjROpmYur8P7IWcXE/LfJo/sK7tAZIu +piuMCtfQNHqcBYLpZE+ywncdU3Y3yDRnFO4LUrdxZm4sSLZp4lzaZI2B5QJyIp/epJx DA0Fc14qIAu9Axp073YGG210qTtFj1sIrPO9hod2/35s+3HTvR7TuTJ4LaewhZC8RqOr jA0K5xjebNOlhbkADVZvbrEBhTPe1FAeHDHpsw8o/nKBAVK09bLEyGpZka5haU0Plhq5 fHfQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=TYAYVHGHmOdGIx+b1vNePbFs/fCGWl7iYc8XlHT8evo=; b=B8rPWDzmxouUzM7ygvgiSFcdXbzz8CVBkSYJ4tBn4vUgsxmv6y2GenC3vrij59s4na tNIIpcf/BbKQkASCXsKwZHXfWTids+1exUnw9tj8aaM2CmZZkYPNAACIxOUsKj92AWw4 /ovegH+0uGnKsOiQuoAlYi/JOwuMPv1iTYok4YU14uPf2ihFHE3S1YW6m6iFXzReQQl6 /4m6NHNR60i398+Opv6EbXsKTyQ5gYv7aoMpfUxBvbZwV1zzSimnUdACH6jlPrhYTjHY ljN2/dCKRNbehbmIu6u5dPZJu4aLoYMiDI28ynaZHS1TFgXaACiwBVdCQ0ke4l5T8MDu ctHA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=BES+zssW; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id k19-20020a056a00135300b0056bfa55ed99si7652266pfu.119.2022.10.30.14.31.44; Sun, 30 Oct 2022 14:31:57 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=BES+zssW; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229738AbiJ3Vax (ORCPT + 99 others); Sun, 30 Oct 2022 17:30:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41854 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229728AbiJ3Vam (ORCPT ); Sun, 30 Oct 2022 17:30:42 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E2197A1A0 for ; Sun, 30 Oct 2022 14:29:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1667165378; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TYAYVHGHmOdGIx+b1vNePbFs/fCGWl7iYc8XlHT8evo=; b=BES+zssWCvgN0DS018q5U/epVTPOt0UHr6iqWCZYluxZboIJ6S0n+GneXSWGNF4KqlWxX0 KQ3b6g8MCyr1WlTogppUggJ2Ay7Xo2+vmoaIq60rDDeGp8D0Pq+YQgB5WQS6U9ybAZydHY WK91CCrkqR9uMn9cLWmaGpPuSSN7BBg= Received: from mail-qt1-f200.google.com (mail-qt1-f200.google.com [209.85.160.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-672-sX22bnYeMpeOLp0Pgxeeig-1; Sun, 30 Oct 2022 17:29:37 -0400 X-MC-Unique: sX22bnYeMpeOLp0Pgxeeig-1 Received: by mail-qt1-f200.google.com with SMTP id ew11-20020a05622a514b00b003a524196d31so728276qtb.2 for ; Sun, 30 Oct 2022 14:29:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TYAYVHGHmOdGIx+b1vNePbFs/fCGWl7iYc8XlHT8evo=; b=3Hca9S2oLzquINCKV/71p5jdTuFNOeUxXDXmq3FGZcZnyTl55PETtYmeW75HqokSQK lwLBaT+VDmopeRM9lki48TvAYT2z2rSCg2SsLOdJGWaLP54IYQk3kGsgIXKHGWFzQ0OD hq9GUY6tbb52XdhJA0Z/67N29GfSrlzyitApbXLor7vA8eygfLtfqqRT+kjbh5z287qD AkqBtPceV35EczysvIIKyRbOU2JxFbsy/BRBUIs7UqDoSrHk+D2RpX2H6TECo0o/q06r A0SMGoZTlgOm+SBOvGHOSbOP5ATAg3X70u+n4ha6cq6GKbmS28JvZd3bBamnJUfZsAEB Zjmw== X-Gm-Message-State: ACrzQf0aQYHXjtTfklXOBOgv7SvBh7KhAgj8lJZADaX2zKPkBoBWPJ2a hqCnD8I28kfiuo5FttaoqLof7oPhMTXNrGrkeWteqRnf+qxshE3S8TpsQOTKLVzXQagBoBpQTU8 s3WcQB4gvrNZoEk0e+4eKqBnqDNdyO5rTEAbZPn6AgQfXLXuJAvpYgZNk4oSpQdlBp0L+ARt6Dg == X-Received: by 2002:a05:620a:284a:b0:6ab:9cc5:cb4c with SMTP id h10-20020a05620a284a00b006ab9cc5cb4cmr7121587qkp.609.1667165376171; Sun, 30 Oct 2022 14:29:36 -0700 (PDT) X-Received: by 2002:a05:620a:284a:b0:6ab:9cc5:cb4c with SMTP id h10-20020a05620a284a00b006ab9cc5cb4cmr7121563qkp.609.1667165375878; Sun, 30 Oct 2022 14:29:35 -0700 (PDT) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id a1-20020ac81081000000b003a4b88b886esm2654781qtj.96.2022.10.30.14.29.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 30 Oct 2022 14:29:35 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Andrew Morton , James Houghton , Miaohe Lin , David Hildenbrand , Muchun Song , Andrea Arcangeli , Nadav Amit , Mike Kravetz , peterx@redhat.com, Rik van Riel Subject: [PATCH RFC 02/10] mm/hugetlb: Comment huge_pte_offset() for its locking requirements Date: Sun, 30 Oct 2022 17:29:21 -0400 Message-Id: <20221030212929.335473-3-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221030212929.335473-1-peterx@redhat.com> References: <20221030212929.335473-1-peterx@redhat.com> MIME-Version: 1.0 Content-type: text/plain X-Spam-Status: No, score=-3.2 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748149749420661557?= X-GMAIL-MSGID: =?utf-8?q?1748149749420661557?= huge_pte_offset() is potentially a pgtable walker, looking up pte_t* for a hugetlb address. Normally, it's always safe to walk the pgtable as long as we're with the mmap lock held for either read or write, because that guarantees the pgtable pages will always be valid during the process. But it's not true for hugetlbfs: hugetlbfs has the pmd sharing feature, it means that even with mmap lock held, the PUD pgtable page can still go away from under us if pmd unsharing is possible during the walk. It's not always the case, e.g.: (1) If the mapping is private we're not prone to pmd sharing or unsharing, so it's okay. (2) If we're with the hugetlb vma lock held for either read/write, it's okay too because pmd unshare cannot happen at all. Document all these explicitly for huge_pte_offset(), because it's really not that obvious. This also tells all the callers on what it needs to guarantee huge_pte_offset() thread-safety. Signed-off-by: Peter Xu --- arch/arm64/mm/hugetlbpage.c | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 35e9a468d13e..0bf930c75d4b 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -329,6 +329,38 @@ pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma, return ptep; } +/* + * huge_pte_offset(): Walk the hugetlb pgtable until the last level PTE. + * Returns the pte_t* if found, or NULL if the address is not mapped. + * + * NOTE: since this function will walk all the pgtable pages (including not + * only high-level pgtable page, but also PUD that can be unshared + * concurrently for VM_SHARED), the caller of this function should be + * responsible of its thread safety. One can follow this rule: + * + * (1) For private mappings: pmd unsharing is not possible, so it'll + * always be safe if we're with the mmap sem for either read or write. + * This is normally always the case, so IOW we don't need to do + * anything special. + * + * (2) For shared mappings: pmd unsharing is possible (so the PUD-ranged + * pgtable page can go away from under us! It can be done by a pmd + * unshare with a follow up munmap() on the other process), then we + * need either: + * + * (2.1) hugetlb vma lock read or write held, to make sure pmd unshare + * won't happen upon the range (it also makes sure the pte_t we + * read is the right and stable one), or, + * + * (2.2) RCU read lock, to make sure even pmd unsharing happened, the + * old shared PUD page won't get freed from under us, so even of + * the pteval can be obsolete, at least it's still always safe to + * access the pgtable page (e.g., de-referencing pte_t* would not + * cause use-after-free). + * + * PS: from the regard of (2.2), it's the same logic of fast-gup being safe + * for generic mm, as long as RCU is used to free any pgtable page. + */ pte_t *huge_pte_offset(struct mm_struct *mm, unsigned long addr, unsigned long sz) { From patchwork Sun Oct 30 21:29:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 13054 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp1972462wru; Sun, 30 Oct 2022 14:32:05 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7UBc+bFArV1QrOqV2bBeHuEz99UljyBQd3fTImc7CY0TRhPXxl0vm8HNTBlPsSeK3lYXNH X-Received: by 2002:a63:350:0:b0:44c:2476:12ff with SMTP id 77-20020a630350000000b0044c247612ffmr10009402pgd.50.1667165525129; Sun, 30 Oct 2022 14:32:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667165525; cv=none; d=google.com; s=arc-20160816; b=t6aEKFNGahVj0KEOISoUG/tR0dI7dnnAHUtTsCLn+B+YIFsCxoNXclY9hs+JM2QMR3 QviQeVXzcIXfZrl5xl9w3CzEt3jZtllsBsPueC6F3OmeEr5UD8qCWouCWmL9+e+26oTM 88Qs1O3OT+tDdcOy3z7fOw4FoHrn3jJTPlDFGXkXw613e8jptd7oOnPUoqBr3TK/ZZ+2 peQGW1DnqiySyOqf3SUQfEVak3YTDXCVyb5ntK6Hdj9GKwOeoR4ysPyHrCZjVWwbrNet 1wzz5tXmg7ZuW8MPJ9wcoRxQOfu3oA4Hgxtlejput2cerhtv3MyuK+48yfsgy/EajBZi AW1Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=wagvClDr96VqW50v7EvOelntcB2vHhVhi8l6aUJsy+c=; b=NaQLytRjHLr3Vd7TLD4XTYlW132mIlQuaX2pm6iehABB4bLtD2YcEZh2VRsym5h2/J cmlwswtzmvGem6Hv4zCmaoWfjI4GTGBC1x0pIfWonKQ1Krg52mVKxrytcnp0C/y5leiC X7y8cluB1+EInO2iklIAvCq5OZmR5RySgN8fXtY6fMlBpa04MyMfPvjni/aOXKDiP0pV rkUfEgZ0d6VS/Lwno2nz/jmY1czZG0Unxpw9IR9D6Lx42Jx7eXzNTwoD35Lt4knHAttH yUH0pdML6zg1nECL7zQK1WmeO0mnipEniFhQ71SmoDe7MSSPNraZ93o3oFq4moMTzMcN Vqmg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=Ento34bf; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id pq5-20020a17090b3d8500b001fe1c9436b1si6533742pjb.86.2022.10.30.14.31.52; Sun, 30 Oct 2022 14:32:05 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=Ento34bf; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229770AbiJ3Van (ORCPT + 99 others); Sun, 30 Oct 2022 17:30:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41856 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229681AbiJ3Vag (ORCPT ); Sun, 30 Oct 2022 17:30:36 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C0CDEA1A2 for ; Sun, 30 Oct 2022 14:29:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1667165380; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wagvClDr96VqW50v7EvOelntcB2vHhVhi8l6aUJsy+c=; b=Ento34bfcoEHMbFUVUOCg31FX7uM/ax295P8Y1/3QXPs8X+T8nNyFqtijJAaJBm6onyiJK sSHA310fpO4p72xxZnGsD+d+P4G90nrKm7TZKEGIIsYKRRGu26jfXQrbu5+kMqyQqIh2B2 tJUUVVHoLWiCgLMgAdz1UoFSJmUvzZY= Received: from mail-qt1-f197.google.com (mail-qt1-f197.google.com [209.85.160.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-458-crETd6G8NlypahQZeKVAnA-1; Sun, 30 Oct 2022 17:29:38 -0400 X-MC-Unique: crETd6G8NlypahQZeKVAnA-1 Received: by mail-qt1-f197.google.com with SMTP id fp9-20020a05622a508900b003a503ff1d4cso5118129qtb.22 for ; Sun, 30 Oct 2022 14:29:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wagvClDr96VqW50v7EvOelntcB2vHhVhi8l6aUJsy+c=; b=lWvpk+lu9CsX+oW1BccnEs2RyoCD3LwUAZHETZlUh8dTC+s0vFVYdnXJmJvlpdG+n7 Pb6qaDfiVHCcpL1NegUalfncL+0fI3os3jK1HMeikkrJGSVTpwTRJiYtqGOtzoCaxAb4 g7VUQjiimYdm0saLsr+ABs/VLcVTii0b/7I+dbc1Y+710KFn8Hz8AoQxJ9WKKft3gc3V ghIvAR+mqtiNO0t96zCt+Rkg6vExa9KCmMv6cQtMF2Kdu0CVqtZoTWVgAR8v7fhg97Wo 3N8Tkr/4GZg7NxCe7rxgTwsSUydsnDZhOTT8KCR3UVJICgyiXwMaQxKwe7rdfR4Lhoo8 L7IQ== X-Gm-Message-State: ACrzQf3zBcLTPK3wQm6GyxAePxP+WpuwjNXxxZjenFsHwfc6PZ3aDsKE Zdm0MgGFzJxSaPwariv4YQyecMvpcApdYFfQ1Cd6qh3Lw7EkwoZAR3GLqbiyA9W06Hjf7AJ8ugc PwDPj2n0CX3S2ukgOuZxcbsWCl8negb0kx+e4E2+HCVK4kIk9N9m/AfAsGaJr/rTb5WDa2A8PKw == X-Received: by 2002:a05:620a:270e:b0:6df:7dba:f45a with SMTP id b14-20020a05620a270e00b006df7dbaf45amr6855707qkp.661.1667165377674; Sun, 30 Oct 2022 14:29:37 -0700 (PDT) X-Received: by 2002:a05:620a:270e:b0:6df:7dba:f45a with SMTP id b14-20020a05620a270e00b006df7dbaf45amr6855682qkp.661.1667165377401; Sun, 30 Oct 2022 14:29:37 -0700 (PDT) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id a1-20020ac81081000000b003a4b88b886esm2654781qtj.96.2022.10.30.14.29.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 30 Oct 2022 14:29:36 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Andrew Morton , James Houghton , Miaohe Lin , David Hildenbrand , Muchun Song , Andrea Arcangeli , Nadav Amit , Mike Kravetz , peterx@redhat.com, Rik van Riel Subject: [PATCH RFC 03/10] mm/hugetlb: Make hugetlb_vma_maps_page() RCU-safe Date: Sun, 30 Oct 2022 17:29:22 -0400 Message-Id: <20221030212929.335473-4-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221030212929.335473-1-peterx@redhat.com> References: <20221030212929.335473-1-peterx@redhat.com> MIME-Version: 1.0 Content-type: text/plain X-Spam-Status: No, score=-3.2 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748149757963094090?= X-GMAIL-MSGID: =?utf-8?q?1748149757963094090?= RCU makes sure the pte_t* won't go away from under us. Please refer to the comment above huge_pte_offset() for more information. Signed-off-by: Peter Xu --- fs/hugetlbfs/inode.c | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c index ac3a69fe29c3..b9e7825079c7 100644 --- a/fs/hugetlbfs/inode.c +++ b/fs/hugetlbfs/inode.c @@ -387,21 +387,26 @@ static bool hugetlb_vma_maps_page(struct vm_area_struct *vma, unsigned long addr, struct page *page) { pte_t *ptep, pte; + bool result = false; + + /* For huge_pte_offset() */ + rcu_read_lock(); ptep = huge_pte_offset(vma->vm_mm, addr, huge_page_size(hstate_vma(vma))); if (!ptep) - return false; + goto out; pte = huge_ptep_get(ptep); if (huge_pte_none(pte) || !pte_present(pte)) - return false; + goto out; if (pte_page(pte) == page) - return true; - - return false; + result = true; +out: + rcu_read_unlock(); + return result; } /* From patchwork Sun Oct 30 21:29:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 13055 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp1972501wru; Sun, 30 Oct 2022 14:32:14 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7pNqjW9OV83e1ad6iCAOGrsj3GgSHtwEJTrKTOoPLGaA/pup2Lil805d1VO5gHWjD6gS27 X-Received: by 2002:a17:902:c643:b0:186:9efc:67a1 with SMTP id s3-20020a170902c64300b001869efc67a1mr11001192pls.30.1667165533816; Sun, 30 Oct 2022 14:32:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667165533; cv=none; d=google.com; s=arc-20160816; b=ymFxVqNqP35AJ1HSXdQtRmHCyp/wVst28PCOTl/PUiZ95piaM13jLrKdS1BJQp7hYv wQjqos/bW8mZ77duJFyGYKJr5AYCxPzMheKouLhGtUl1cP0PxgJKEIZszzKhLl2jX6DJ /QXkE8ccqaHd5oBhonCls8ol13GyjQho8prL7/g7QHDgX1p2WY+ZhkNHoRDFNTV/Dh7S qdjkp7B+ygZedd7/SRtrI2kqECgGaW4Crohj9Kv5h+9QBSzf2QzGwdRkbEy7qWq/X8/+ JgLta8KLu4pC2gl2+cHu5FDt2yuJf37JjRwRtsmrYssBU30/gvu9UYxbNdDvt/SZ01hC z4ag== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=agXgQWAuByPHwxJXo49Hx7hkR/BEQEXX5BlO+c7HbVA=; b=umv6745ciJ1PT+3GbUnTStaPl73ZQW+9i3dICpJXvyBVxi4kVYeYwu5ZOUlBjkPZ3K DaS/QaxOX8fzDE3tlY8D5OacJ+/Z6n7t9/lPIrqM65KeV0SuxlOE1vSiUs8JWdzORKSR /ZGrBqDp3xImVuA3xzS5vmc7l9lnlWCOKlnLPa3PgRPDCUQtcX8Q+Qw8vL102sYufIxT EJfqtZtllqS78kktEuCw6uxxo5hSqg9O0kxl0fbwpBq/ochDiEKUc6tOfPSTxqFaqmMv Jhegfo9wx+a5PifEA1jO9E2mfxmf1DwWjjv/nTHvt4X8r/AvnuKblLuXXc4+34Sts0T0 Cjrg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=eULKnbz0; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id l9-20020a170902f68900b00186a3bc939esi6972583plg.211.2022.10.30.14.32.00; Sun, 30 Oct 2022 14:32:13 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=eULKnbz0; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229689AbiJ3Vaz (ORCPT + 99 others); Sun, 30 Oct 2022 17:30:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41866 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229767AbiJ3Van (ORCPT ); Sun, 30 Oct 2022 17:30:43 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D7DC3A44D for ; Sun, 30 Oct 2022 14:29:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1667165382; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=agXgQWAuByPHwxJXo49Hx7hkR/BEQEXX5BlO+c7HbVA=; b=eULKnbz0iRds+nPlupkJyRMKYcVP+cp5maTyzQDLF5wNoXctbhkLXZD9j0LTMMVj5TsfXd fR9mu1TxV1yDf1GtxdfgdkUv/AjHxoeNsHTdUcUehxHI9fAUO7TqMKB0YsbM6imdWHXonw 7qzTxx/IGWrnJcdxMEvS3aOEzhRmQCM= Received: from mail-qk1-f198.google.com (mail-qk1-f198.google.com [209.85.222.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-327-42NyM1FaOu-uVp8XgLv4dg-1; Sun, 30 Oct 2022 17:29:39 -0400 X-MC-Unique: 42NyM1FaOu-uVp8XgLv4dg-1 Received: by mail-qk1-f198.google.com with SMTP id de21-20020a05620a371500b006eed31abb72so7984366qkb.6 for ; Sun, 30 Oct 2022 14:29:39 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=agXgQWAuByPHwxJXo49Hx7hkR/BEQEXX5BlO+c7HbVA=; b=Gows9Re9yfGoUNgITLJKKq60Ec3WWjKhyE3fESjCRJ1M4s+i/aAQWIsKberUYj02m+ DOGeok46iH1gHOKw4v8adKiAu6DQfE0qsGNqg4r8CP6/5P+gBsHf7wYzFzpTUguXsvPD va6FSMEeILn2XkBa9+xcCR9D+vqqKt4qp6IRM/KG+ZUSr4J4aXmHqh8aw/799nf2oAnW h8pZYRPYObKXdjIgBnq41qnxfNq3Z1FfoMecFZUeMlMdsqLOiWqyeR9L9T96qlvEa/TQ /leSjIbXlIWSQ+/04O/YPu4pSNDDV9VN52ZtP5vLl9PksXy3YoT5P0U75xDn/UhZKqDQ wN1Q== X-Gm-Message-State: ACrzQf2zJgP/O9MmPQG4x7Yy6Lafr9zaGGFzL0DuWdcPPWQYp4RDjG8A q3lKNajO/mAu7alBdYdmQ4iZ1XGrQFfnB9SzST84JjRawaA+5Xz+0qD7m1IuZdEKc3l55dcCHUK RIavErLnjYVUesKn+fVQF2JvJRegX0+FNsPGCUNzvmaEbkGLskhCW/0mddEY+YNMN/9H54p94sw == X-Received: by 2002:a0c:f0d4:0:b0:4bb:6583:66e6 with SMTP id d20-20020a0cf0d4000000b004bb658366e6mr8560482qvl.123.1667165379140; Sun, 30 Oct 2022 14:29:39 -0700 (PDT) X-Received: by 2002:a0c:f0d4:0:b0:4bb:6583:66e6 with SMTP id d20-20020a0cf0d4000000b004bb658366e6mr8560462qvl.123.1667165378831; Sun, 30 Oct 2022 14:29:38 -0700 (PDT) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id a1-20020ac81081000000b003a4b88b886esm2654781qtj.96.2022.10.30.14.29.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 30 Oct 2022 14:29:38 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Andrew Morton , James Houghton , Miaohe Lin , David Hildenbrand , Muchun Song , Andrea Arcangeli , Nadav Amit , Mike Kravetz , peterx@redhat.com, Rik van Riel Subject: [PATCH RFC 04/10] mm/hugetlb: Make userfaultfd_huge_must_wait() RCU-safe Date: Sun, 30 Oct 2022 17:29:23 -0400 Message-Id: <20221030212929.335473-5-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221030212929.335473-1-peterx@redhat.com> References: <20221030212929.335473-1-peterx@redhat.com> MIME-Version: 1.0 Content-type: text/plain X-Spam-Status: No, score=-3.2 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748149767172885867?= X-GMAIL-MSGID: =?utf-8?q?1748149767172885867?= RCU makes sure the pte_t* won't go away from under us. Please refer to the comment above huge_pte_offset() for more information. Signed-off-by: Peter Xu --- fs/userfaultfd.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c index 07c81ab3fd4d..4e813e68e4f8 100644 --- a/fs/userfaultfd.c +++ b/fs/userfaultfd.c @@ -243,6 +243,9 @@ static inline bool userfaultfd_huge_must_wait(struct userfaultfd_ctx *ctx, mmap_assert_locked(mm); + /* For huge_pte_offset() */ + rcu_read_lock(); + ptep = huge_pte_offset(mm, address, vma_mmu_pagesize(vma)); if (!ptep) @@ -261,6 +264,7 @@ static inline bool userfaultfd_huge_must_wait(struct userfaultfd_ctx *ctx, if (!huge_pte_write(pte) && (reason & VM_UFFD_WP)) ret = true; out: + rcu_read_unlock(); return ret; } #else From patchwork Sun Oct 30 21:29:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 13061 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp1972738wru; Sun, 30 Oct 2022 14:33:03 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7CXx/lZ/wdiMU3+jqU/yzLBzDmDUD2YgIcGGQ8iyfenRJGJLY4i/hwYxEmGSm6OMdocKfv X-Received: by 2002:a17:902:e8cd:b0:186:9efc:6790 with SMTP id v13-20020a170902e8cd00b001869efc6790mr11035117plg.91.1667165583467; Sun, 30 Oct 2022 14:33:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667165583; cv=none; d=google.com; s=arc-20160816; b=Xp6nYKidhsV+Fro9crvur3z63Yvd5W2zo5hK8BGdSwSznmEfxxj1MmmDPmRxC93VRT +84VNyZyycc4nUIvQ5K02m1C3cwLGMO1YCzPlDvlu7snmVy3Pw5WE/H+dNkMzH8Mg+Ff vsdy8ab63kO6/jRwGTQgzDOA4bcw+qeRenAAJGmTHZZ1mBMooAitWM9527C61TrUYS1C rz5+TFn9f4Ymu+t2t4N0LII6kW51R8xOb+xFtbXbDx04EABz2GPe59ly/Xs08ILThQAl lfpu8n5otEJmBSxURE/Wlq5ZwPDtuq9WGQUDugPl8HdK+NpKGoa6aavy7HcEpTJZGS2O lNEg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=JcZdgMzXcUClzXwFwvMe9vX/XbvITV8H0Y6t4On5idY=; b=uO0YJtUI4EpqSyUbEVaZjAQc3DyG00zIk4c45Baqp1cLEZXg7e2ayKESSDmt2QEY4x MmbP4CntKqgWVg2++78x0y8BUrnC4ssXHnkj6jzDbIVZzWhfaW77fD0qflFg9ePK/v4s tThUFgB5ResIBslyR32LGZ8PS3t7JW8nkYXulukRBsuPBK16+JdrwN/m+fWmdrL4tP8u Zy0ZZTQi7Erah+yT5GsI8brhjmkQA8h+rdckVv8dc0ysVo9hSLPwd3sWvmIXrRfEetqU XiU8NrodY8rTa5hqBdOMmfQrjJdJVc/rUa+BWojIRlXcLuXVzNLNVvhWjWnLQ///bnPJ m6oQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=BvcrJES4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id e14-20020aa7980e000000b00565a581ecc0si5959864pfl.11.2022.10.30.14.32.51; Sun, 30 Oct 2022 14:33:03 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=BvcrJES4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229849AbiJ3Vbk (ORCPT + 99 others); Sun, 30 Oct 2022 17:31:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41896 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229718AbiJ3Vbb (ORCPT ); Sun, 30 Oct 2022 17:31:31 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 59241A45D for ; Sun, 30 Oct 2022 14:29:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1667165382; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JcZdgMzXcUClzXwFwvMe9vX/XbvITV8H0Y6t4On5idY=; b=BvcrJES4j0nILKfC76UaCHTgFjMycDrDv6w+C8IYnKZ4GiscdDrMn5DoIPp4LnPjV269aJ TNtaj4tsazvYdg4yX38rIlXNAIFg5xuOfXWzbFpuZs0RM2StwfpUiueU6YnzgGp1V+umc9 JAUSwPK7CFfnsHuhqM2Ou+fVRTtIwmo= Received: from mail-qk1-f198.google.com (mail-qk1-f198.google.com [209.85.222.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-363-U_Hwj77VOKiPjB-qBt7nWQ-1; Sun, 30 Oct 2022 17:29:41 -0400 X-MC-Unique: U_Hwj77VOKiPjB-qBt7nWQ-1 Received: by mail-qk1-f198.google.com with SMTP id o13-20020a05620a2a0d00b006cf9085682dso7978019qkp.7 for ; Sun, 30 Oct 2022 14:29:41 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JcZdgMzXcUClzXwFwvMe9vX/XbvITV8H0Y6t4On5idY=; b=wNE2bsds37kZFN/8ELmdkj1XCw4gVEtqSw5PynUFxEKuMRKOBQOzN5jxVAKpg9Ohmt +GGFs0YosZJyzhPIKYFGHbVLLFxLKSGsDODqgji78UCF0H9zw2eIIbELIClPDcXKj+J7 jHGmXa+Be1d7oQLhGjo0tUBVGLRkQVAr/EVB28cVimr+kH7aM6k/n3lyD+Nl5cPilYjg qqz38z/NGzhJ3TsRj4x8ZopsN4nUyCt3jib66k8q5jldQG5bLquY8htA/nL9w7qv9Eow LO0+a2ye4OKvu8XreLf8DzMkN9dbxB/YGzxPh8hXm0U9VlChPaqBl58FlABh0vkBJEuk e7Vw== X-Gm-Message-State: ACrzQf1TQTQAAvE0dOzp2A2eOgb2dM4EyoHZbaAm+QECb/Pc6CavpqqS 7EI/KIk4JdZHqVAZwdLADjUCpS6ZZpq6wtV2IKu+7nkOenLT3wo+ga4eTBbRVt32k2RP/2HlA+k appNSnWYmfti+io/mNR93+xR2SvhgRxDSWZSSKKLYYTOxbAJVmiFT32M26sE00iXZQDmeci2o9w == X-Received: by 2002:ad4:5c4e:0:b0:4bb:9fea:f52e with SMTP id a14-20020ad45c4e000000b004bb9feaf52emr8476681qva.51.1667165380631; Sun, 30 Oct 2022 14:29:40 -0700 (PDT) X-Received: by 2002:ad4:5c4e:0:b0:4bb:9fea:f52e with SMTP id a14-20020ad45c4e000000b004bb9feaf52emr8476659qva.51.1667165380375; Sun, 30 Oct 2022 14:29:40 -0700 (PDT) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id a1-20020ac81081000000b003a4b88b886esm2654781qtj.96.2022.10.30.14.29.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 30 Oct 2022 14:29:39 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Andrew Morton , James Houghton , Miaohe Lin , David Hildenbrand , Muchun Song , Andrea Arcangeli , Nadav Amit , Mike Kravetz , peterx@redhat.com, Rik van Riel Subject: [PATCH RFC 05/10] mm/hugetlb: Make walk_hugetlb_range() RCU-safe Date: Sun, 30 Oct 2022 17:29:24 -0400 Message-Id: <20221030212929.335473-6-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221030212929.335473-1-peterx@redhat.com> References: <20221030212929.335473-1-peterx@redhat.com> MIME-Version: 1.0 Content-type: text/plain X-Spam-Status: No, score=-3.2 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748149818914580191?= X-GMAIL-MSGID: =?utf-8?q?1748149818914580191?= RCU makes sure the pte_t* won't go away from under us. Please refer to the comment above huge_pte_offset() for more information. Signed-off-by: Peter Xu --- mm/pagewalk.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/mm/pagewalk.c b/mm/pagewalk.c index 7f1c9b274906..bbc71c750576 100644 --- a/mm/pagewalk.c +++ b/mm/pagewalk.c @@ -302,6 +302,9 @@ static int walk_hugetlb_range(unsigned long addr, unsigned long end, const struct mm_walk_ops *ops = walk->ops; int err = 0; + /* For huge_pte_offset() */ + rcu_read_lock(); + do { next = hugetlb_entry_end(h, addr, end); pte = huge_pte_offset(walk->mm, addr & hmask, sz); @@ -315,6 +318,8 @@ static int walk_hugetlb_range(unsigned long addr, unsigned long end, break; } while (addr = next, addr != end); + rcu_read_unlock(); + return err; } From patchwork Sun Oct 30 21:29:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 13057 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp1972550wru; Sun, 30 Oct 2022 14:32:23 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4qqUUjcYzQbqltuOeOLZUhzyYt0+AWb1yltEi2L/76OELWrmeb3rU6bj3xVcsf8FnjJm4t X-Received: by 2002:a63:4384:0:b0:43a:18ce:f98a with SMTP id q126-20020a634384000000b0043a18cef98amr9652141pga.273.1667165543046; Sun, 30 Oct 2022 14:32:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667165543; cv=none; d=google.com; s=arc-20160816; b=C8i5yiuNvVy8jrWOJAW3kY5jxWdC6RBxOHzDOoPWateF6i7c0ylmibBgsDxALx48ro 6EDKFfYnee/uQkr8XjkCrKgS/tid4w2VxTivWQyhhtv0VgP8MvC9KvEqnvOQ7t/a22Xj +70ABPnpFgiaBjOh4YRKiBOCzODVbObPdcFiYR09ExdHy/cwhGBFtGG3dKa5ZkTFMmJw Z873b0W3UL3yducuiS+MajxKmdSgDts4zx/YWYGSuG/ZSjZT8aTsm1xWpiPYYmNll7MV ETZKjAEWmTnBGrNpIJQMr/97USHOZqSrqytvrPhDhcfq7Sd/SPFuhAb3tFtv1Jtj+npi steQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=WBN8D9wCQW9wV7A+8u6L08Ik6RBdolsM1Jhl7A7rc8s=; b=tU4id6GiFBCiKyt63+3dpfmAguDveiMUjLQ+IHXaGjRVWdWNMC8JZlBlyQg/mViDiL MPINgJvvWqmS+/AUBgnHpDvx02TrXAzrwxrEgjCbZDPUs2LLtyEFlzWkVCWRBUbEj86b rKvJ7u8u1eQJ3qqRm4HiwHPmyOpP6BodPK8e6yT/HQNdnDRBlHLUpAjVLN119LjjZt0Z ftIdmt5E46F0NfbnFsEEYwjc+mZkTHzpDWTHKTon+0GpPtR5b/vgjvktFrmA5JXJUOoZ fntg3L1FNK/8KG0/LKVLoJn3Zhpi2KLjkeIYTSQRd1xdm4qfjIx/BsR2jm7fr43hgnet M5Xw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=ZhXgywEZ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id y7-20020a17090322c700b0016dc27fb9fbsi7586073plg.117.2022.10.30.14.32.10; Sun, 30 Oct 2022 14:32:23 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=ZhXgywEZ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229761AbiJ3Vbe (ORCPT + 99 others); Sun, 30 Oct 2022 17:31:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41900 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229556AbiJ3Vb1 (ORCPT ); Sun, 30 Oct 2022 17:31:27 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6B183A460 for ; Sun, 30 Oct 2022 14:29:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1667165391; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WBN8D9wCQW9wV7A+8u6L08Ik6RBdolsM1Jhl7A7rc8s=; b=ZhXgywEZckyh/r+RrUqDGLSPPPKpNRQzaTne4j2fbKtwcs6CTh9BRCFjKReRG+97gk/YN6 On14FKxHkryyMUxhP8tMJ1BXG8pF7Pw72O1f/11hDHwPwbKXX06vu2c2v1RdxizTlSHXJ5 AEu2lZcn9sMIS0Js8PRoFm5wEUwnw5o= Received: from mail-qt1-f197.google.com (mail-qt1-f197.google.com [209.85.160.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-329-11DEyvRzPSy2VC0YzWNAuQ-1; Sun, 30 Oct 2022 17:29:42 -0400 X-MC-Unique: 11DEyvRzPSy2VC0YzWNAuQ-1 Received: by mail-qt1-f197.google.com with SMTP id i4-20020ac813c4000000b003a5044a818cso4846994qtj.11 for ; Sun, 30 Oct 2022 14:29:42 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WBN8D9wCQW9wV7A+8u6L08Ik6RBdolsM1Jhl7A7rc8s=; b=cuUwJid9F63Y3MikCKhQEDEQIYvrgwoEpsPz965MHb8cwz13K2DsE+yUPRhz7eAXz1 nBkwnX1Q3/E0VJpya58Njr4QlwBlbphGUnOUU6bdMRNA74jiKoSg6sUkFOUi32Y8U1nP Huq7rBKL4DAmrsg+/3apcQcMED/daYjMGLZVQ/Gb603cGRursZsNEYpgokTcB7skzu4Q 2kQc9L7Wao0S0sVHRyQKQ9m0luW7USWWCDQZ+coGx8MrNyKHxix2blUnc24Uo6fOGGF+ xywS2xvKH/yoPEU9JvTZNve86Wyh7unKuGkosRokI8SZvAd2S70nlpRxVWdfykaT+1qH ZIWg== X-Gm-Message-State: ACrzQf0Qx+KKu+jdSjEPhf5TCbyKY9z7sCQexYfQpDKUFiDeoqVr1cJr 9B3Y89hK4CwjoEX6SMRfrtv7wqixLKIOHC1PwjilzYEIIuBnJgLMbyb+x4PcBuMX8fh6/bBHej3 y2r+ukztPKmrcFVO5oz1XeTY3az2gm8YimCVl4htRRSvJ1A5loGCl+osgkK7LefF7gHFKPk0B9Q == X-Received: by 2002:a05:620a:404d:b0:6da:4b51:2a8f with SMTP id i13-20020a05620a404d00b006da4b512a8fmr6987364qko.421.1667165381984; Sun, 30 Oct 2022 14:29:41 -0700 (PDT) X-Received: by 2002:a05:620a:404d:b0:6da:4b51:2a8f with SMTP id i13-20020a05620a404d00b006da4b512a8fmr6987344qko.421.1667165381708; Sun, 30 Oct 2022 14:29:41 -0700 (PDT) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id a1-20020ac81081000000b003a4b88b886esm2654781qtj.96.2022.10.30.14.29.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 30 Oct 2022 14:29:41 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Andrew Morton , James Houghton , Miaohe Lin , David Hildenbrand , Muchun Song , Andrea Arcangeli , Nadav Amit , Mike Kravetz , peterx@redhat.com, Rik van Riel Subject: [PATCH RFC 06/10] mm/hugetlb: Make page_vma_mapped_walk() RCU-safe Date: Sun, 30 Oct 2022 17:29:25 -0400 Message-Id: <20221030212929.335473-7-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221030212929.335473-1-peterx@redhat.com> References: <20221030212929.335473-1-peterx@redhat.com> MIME-Version: 1.0 Content-type: text/plain X-Spam-Status: No, score=-3.2 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748149776843270121?= X-GMAIL-MSGID: =?utf-8?q?1748149776843270121?= RCU makes sure the pte_t* won't go away from under us. Please refer to the comment above huge_pte_offset() for more information. Signed-off-by: Peter Xu --- include/linux/rmap.h | 3 +++ mm/page_vma_mapped.c | 7 ++++++- 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/include/linux/rmap.h b/include/linux/rmap.h index bd3504d11b15..d2c5e69a56f2 100644 --- a/include/linux/rmap.h +++ b/include/linux/rmap.h @@ -408,6 +408,9 @@ static inline void page_vma_mapped_walk_done(struct page_vma_mapped_walk *pvmw) pte_unmap(pvmw->pte); if (pvmw->ptl) spin_unlock(pvmw->ptl); + /* Hugetlb uses RCU lock for safe access of huge_pte_offset() */ + if (is_vm_hugetlb_page(pvmw->vma)) + rcu_read_unlock(); } bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw); diff --git a/mm/page_vma_mapped.c b/mm/page_vma_mapped.c index 93e13fc17d3c..513210a59d7b 100644 --- a/mm/page_vma_mapped.c +++ b/mm/page_vma_mapped.c @@ -169,10 +169,15 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw) if (pvmw->pte) return not_found(pvmw); + /* For huge_pte_offset() */ + rcu_read_lock(); + /* when pud is not present, pte will be NULL */ pvmw->pte = huge_pte_offset(mm, pvmw->address, size); - if (!pvmw->pte) + if (!pvmw->pte) { + rcu_read_unlock(); return false; + } pvmw->ptl = huge_pte_lock(hstate, mm, pvmw->pte); if (!check_pte(pvmw)) From patchwork Sun Oct 30 21:29:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 13056 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp1972505wru; Sun, 30 Oct 2022 14:32:15 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5M/8zrqN+X0xxppwL2AwftpdT3K/Nu+hMC8uSi7AP+YbNrAxCWyZM9wdUF/A+n7r1Or4Of X-Received: by 2002:a63:8bc2:0:b0:46f:5bbb:7370 with SMTP id j185-20020a638bc2000000b0046f5bbb7370mr9860622pge.70.1667165534784; Sun, 30 Oct 2022 14:32:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667165534; cv=none; d=google.com; s=arc-20160816; b=OVT3J5pYuxW5XEUMt6r+tL1YbpzrP90z0WGT9rHP8R99lIecddbkqcOqBhP2D1KNbJ L6iBBczLFJVZW1Bk/Xoci2AFWmzgBpFP9fI2E4qfgo0sqNpvCcYtyJCIHisHwcFZmjy3 DBItXEv3FtGAvhJ1WL6Rt5lzs1onO+N3l2kT+JfARbQwuk2BJ0Dqsh3pzJJ+Vv/byYFy lm8oZVQIYzC2qm+7D1xVPnnvZ47opzz7ajSG4+gDOIX3LfNwkklgFNE3bAfWyCSO77P+ //Na2w6d9Y4sY2zfyIIgq7kl9XvVZ+PF4dxyc5OoGurfPYUxeeE4QPbeqIsavVhp9SdX 036w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=6JldXIZ0VovVxqs2u1isjA5hu6yU0tGljewUccwkM0U=; b=zVczMGvMVlKfccLrBMvvSEC5ST4fSREo/6/jhsBzOVHJkTJWUxlv1aica4TMFYTI+s YZHUZb0myW5qUJlZUbMBtgyPT2I+oj7q52mL5zK/9s+UOO7Q+Zo1VBwpgKkFmPY/2z5j Dx32mQ3kZU2Nc1xmniWVxQT0OzKHbs+chDQ7pEXZSISRyZSMJufRbKjPAm/ANjuwh12w j2C8jUtmYQYCVJSe8fqaqOZZk+IYPh38CsvM85G3KXHT1XAY0JK+N2u1YQuGVelPM8Uh 2G6Np7XF5at9rsJ4RZ1jDqmPMj8dgxKDiF7SxEKQqhS21LhWEfa7xqC2rx+M1usChBya towg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=ZWfAMV31; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id e38-20020a631e26000000b0046f5808167asi6189926pge.812.2022.10.30.14.32.01; Sun, 30 Oct 2022 14:32:14 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=ZWfAMV31; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229781AbiJ3Vba (ORCPT + 99 others); Sun, 30 Oct 2022 17:31:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41898 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229785AbiJ3VbV (ORCPT ); Sun, 30 Oct 2022 17:31:21 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D263EA45F for ; Sun, 30 Oct 2022 14:29:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1667165388; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6JldXIZ0VovVxqs2u1isjA5hu6yU0tGljewUccwkM0U=; b=ZWfAMV31AnIrzdaOb4rmWP1Uph4sW7jnjx2J+58PQTS7/7ZqFZXefJV0ASj2Ln64JwH+W1 UVkTT9ozXnEqFZ9+i39uFuJ2bIIkodpQwxPrCnMgV+Mx4v1jisLwYsSh0WaNfFOgp3U/tC a1gaWA/8Nj7d0DFpcsgI9IMib91HWiY= Received: from mail-qt1-f200.google.com (mail-qt1-f200.google.com [209.85.160.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-403-avLD5dvBNOyjZvKtLIGT4g-1; Sun, 30 Oct 2022 17:29:45 -0400 X-MC-Unique: avLD5dvBNOyjZvKtLIGT4g-1 Received: by mail-qt1-f200.google.com with SMTP id cj6-20020a05622a258600b003a519d02f59so2054879qtb.5 for ; Sun, 30 Oct 2022 14:29:45 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6JldXIZ0VovVxqs2u1isjA5hu6yU0tGljewUccwkM0U=; b=Xy1GfrLV7UFXKExE+0SEL+isZXHJT/z0gZgnimV7rZsp5X0p8AxF1qmIkXoW4vzuuE bkqYPi2xJtAr+jfuV85kRxWg8EhQGy/BculYMlE3zMi9bGfDV6sX+gPZ2sD7qhPuqnfi YAOAIzmrPvF2oDEjB9nw6j2k2gzTxzYoRB+zBUlXSic1yVv2nLCXSHG73unyIBR+Hoe1 bBT/QTmi0it8ykulJ7KrdAVZjWy3cVM3Pk6ZV3Ll44799JBpCs3a2t7+I/vt+RbTJb6B f0xf2Y7jZ0k1VAniISk5zSRj3temz4R1adPLv+YusplnDjLesPacytoA16wo/MqiDEpj mzwQ== X-Gm-Message-State: ACrzQf399OaEj9q7NPh7iL4LEpn5NnJIQXBZJwYCzpZlGkJGF+3tZx+Q UXUZx+r/hyi129Kei7O/Fym39nJpST8Io5qqu5JsFPvlwTlF8Nm0IPK0ouFX0MQbfhg5hRAoal4 TBrxXWgAyVIcaCr8BKb9YD5z1Y9/W9AUp9MhMFXQNmcqymoCdH2bpwejCdSECQo2XjDLG/gmxbA == X-Received: by 2002:a05:6214:1c09:b0:4b7:f9f6:7d17 with SMTP id u9-20020a0562141c0900b004b7f9f67d17mr8720827qvc.22.1667165383360; Sun, 30 Oct 2022 14:29:43 -0700 (PDT) X-Received: by 2002:a05:6214:1c09:b0:4b7:f9f6:7d17 with SMTP id u9-20020a0562141c0900b004b7f9f67d17mr8720803qvc.22.1667165383128; Sun, 30 Oct 2022 14:29:43 -0700 (PDT) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id a1-20020ac81081000000b003a4b88b886esm2654781qtj.96.2022.10.30.14.29.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 30 Oct 2022 14:29:42 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Andrew Morton , James Houghton , Miaohe Lin , David Hildenbrand , Muchun Song , Andrea Arcangeli , Nadav Amit , Mike Kravetz , peterx@redhat.com, Rik van Riel Subject: [PATCH RFC 07/10] mm/hugetlb: Make hugetlb_follow_page_mask() RCU-safe Date: Sun, 30 Oct 2022 17:29:26 -0400 Message-Id: <20221030212929.335473-8-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221030212929.335473-1-peterx@redhat.com> References: <20221030212929.335473-1-peterx@redhat.com> MIME-Version: 1.0 Content-type: text/plain X-Spam-Status: No, score=-3.2 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748149767861881257?= X-GMAIL-MSGID: =?utf-8?q?1748149767861881257?= RCU makes sure the pte_t* won't go away from under us. Please refer to the comment above huge_pte_offset() for more information. Signed-off-by: Peter Xu --- mm/hugetlb.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 9869c12e6460..85214095fb85 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -6229,10 +6229,12 @@ struct page *hugetlb_follow_page_mask(struct vm_area_struct *vma, if (WARN_ON_ONCE(flags & FOLL_PIN)) return NULL; + /* For huge_pte_offset() */ + rcu_read_lock(); retry: pte = huge_pte_offset(mm, haddr, huge_page_size(h)); if (!pte) - return NULL; + goto out_rcu; ptl = huge_pte_lock(h, mm, pte); entry = huge_ptep_get(pte); @@ -6266,6 +6268,8 @@ struct page *hugetlb_follow_page_mask(struct vm_area_struct *vma, } out: spin_unlock(ptl); +out_rcu: + rcu_read_unlock(); return page; } From patchwork Sun Oct 30 21:30:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 13058 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp1972581wru; Sun, 30 Oct 2022 14:32:31 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6ka3KIgOHjM7AZ+G9jEA/8VqD55+5b1KJY/rO44tF9wrikn1bphvSo8KgZusjnptFOeN6V X-Received: by 2002:a63:64d:0:b0:46b:158f:102e with SMTP id 74-20020a63064d000000b0046b158f102emr9985060pgg.150.1667165550884; Sun, 30 Oct 2022 14:32:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667165550; cv=none; d=google.com; s=arc-20160816; b=xcCioeOFKHtiZ3FKrSa9LjqjQy8WcJsKmrsL/PZIrCvuUAHOuRkqOGs/ac3s4+oxHG qmvmIrIp8wTiVNheLTS1v2V9fAGDU+pH/c1hucRKMFY+HtbfA6QhazQUtCUiSFVgL8dH bn54d2PkgngB8XJJktQJj+CuWfkRpQTRyVe63JMsbgCiInz12SYot9+WZkpPDY25fKg5 lvXSO00TslUVJaJNiQLlThXm9adJzvLf69jKffo0BgRrzPGUPPP+Iy/cXikiByIESezS +y2TwHLEIzjpzEAz2KCrOQGGCKsRksC6xZT9aFJPrHNchA867MTX2MQqFPfwVO8rLt22 hn0w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=IiwOf4LgCWPWNLGKe48rMbCAvlS3n6l4UUSg+FeDzrg=; b=g3wXJ4xDKmtZspL+gJxW6StMH2/1lqRV2brEJxw6CgD3wz8qM/yCEW3hLA8RHErWTv LxUYGlS/MS0MRzDvgbhr/7MOjJNPZxcjF1nZPzpbXn8G0w4/4GfkeDXxeskkqjiinmrz kSQPttY398u479bwnQ+Zn9Q0VlaHv/IScZOYTa0otT8lWR7WJVteT+BmiSzOOM4r5Ipy ySp9XSRpFqOwXYc2GHXBFL67g2Fzhv07fmeFKW1jXWZz7sEz6N78z5w3W9rYO33cbPPC DilWzYeIw6MY6zxPfFPnnpUGjj2ydBvUaaTtWypoSWyAl03waZOMYWNfZKeqjydfuL5u TDWg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=idvhbcWR; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id bt3-20020a17090af00300b0020a8db1da7bsi5449648pjb.170.2022.10.30.14.32.18; Sun, 30 Oct 2022 14:32:30 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=idvhbcWR; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229802AbiJ3Vbo (ORCPT + 99 others); Sun, 30 Oct 2022 17:31:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41926 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229785AbiJ3Vbf (ORCPT ); Sun, 30 Oct 2022 17:31:35 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 83DEBA47B for ; Sun, 30 Oct 2022 14:30:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1667165438; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=IiwOf4LgCWPWNLGKe48rMbCAvlS3n6l4UUSg+FeDzrg=; b=idvhbcWR738L/aQ5ehZP1m92TgG1mhHVdofKWfsPj6/uLrpY7dqRMns5mzEyCnCvEZyZWM sXdbGlNO/rtvnnSObICra9kepZMwW2BWn1JjTDj0tQNiCpeZUphOzbvPd4V2+OCpxXUYH4 SVztPpxQYhTCboot/LVYDt8rhJ0z3bU= Received: from mail-qt1-f199.google.com (mail-qt1-f199.google.com [209.85.160.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-583-AGrdtJLdMoe3b1tBraq3Wg-1; Sun, 30 Oct 2022 17:30:37 -0400 X-MC-Unique: AGrdtJLdMoe3b1tBraq3Wg-1 Received: by mail-qt1-f199.google.com with SMTP id i13-20020ac8764d000000b003a4ec8693dcso6554819qtr.14 for ; Sun, 30 Oct 2022 14:30:37 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=IiwOf4LgCWPWNLGKe48rMbCAvlS3n6l4UUSg+FeDzrg=; b=iDgkPSmJokemggaz2DS9wVZYmi7SSps4YlhFpS42ilLy+LZdVJkEeetdD45uvgIBne V2LhsWk3y4u1U+r8oa1L0VQ4iizthcWEddUctXpOIoFB2ucKpMfOp/RLcRcYSRayONsG k+XF0VF5bieEYJKrmI8e9cYaML2f7T1bAE7s2FwDTmwwgKNiHhE+oZDFTzP1icuvR/KR ldsqu2CANh/Bjvl9oRToTRuOaiUnjGN2s8lt+d/pYdoc+du0OR4cDEwn63M3dVhase18 mw45kDktigbGB+jTqez48Cp/gq4kiGQeCFn65+UcLXV6Fw3ejpYh4Aq5f4EPVekJWHyd 6yNA== X-Gm-Message-State: ACrzQf1EmoK1+RPxXQ2KPL+HD9GP42rE8dyD5R2JwQPQi+i+LcxETpJM 3IHB8mKqgR0HE1L3hsOHggzplz/V+i56XKpCnwxdUSsrirGbsvlSa4AyCA/ewG6BXKZx+AG6Qcs S/E6k4XVnAkgYDkidXZODdVXD9JealPoX1zSp9mejF/BhQzdxhp6W+VhR6jjj69zMlNhQMGAbrg == X-Received: by 2002:a05:620a:16c7:b0:6fa:313b:f0e0 with SMTP id a7-20020a05620a16c700b006fa313bf0e0mr325886qkn.137.1667165434619; Sun, 30 Oct 2022 14:30:34 -0700 (PDT) X-Received: by 2002:a05:620a:16c7:b0:6fa:313b:f0e0 with SMTP id a7-20020a05620a16c700b006fa313bf0e0mr325867qkn.137.1667165434343; Sun, 30 Oct 2022 14:30:34 -0700 (PDT) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id bm29-20020a05620a199d00b006e42a8e9f9bsm3469443qkb.121.2022.10.30.14.30.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 30 Oct 2022 14:30:33 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Andrew Morton , Miaohe Lin , Muchun Song , Rik van Riel , James Houghton , Nadav Amit , Mike Kravetz , David Hildenbrand , Andrea Arcangeli , peterx@redhat.com Subject: [PATCH RFC 08/10] mm/hugetlb: Make follow_hugetlb_page RCU-safe Date: Sun, 30 Oct 2022 17:30:32 -0400 Message-Id: <20221030213032.335589-1-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221030212929.335473-1-peterx@redhat.com> References: <20221030212929.335473-1-peterx@redhat.com> MIME-Version: 1.0 Content-type: text/plain X-Spam-Status: No, score=-3.2 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748149784693715983?= X-GMAIL-MSGID: =?utf-8?q?1748149784693715983?= RCU makes sure the pte_t* won't go away from under us. Please refer to the comment above huge_pte_offset() for more information. Some small trick is used to release RCU slightly earlier, but that should be safe and just cleaner (with rich comment in code). Signed-off-by: Peter Xu --- mm/hugetlb.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 85214095fb85..5dc87e4e6780 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -6300,6 +6300,9 @@ long follow_hugetlb_page(struct mm_struct *mm, struct vm_area_struct *vma, break; } + /* For huge_pte_offset() */ + rcu_read_lock(); + /* * Some archs (sparc64, sh*) have multiple pte_ts to * each hugepage. We have to make sure we get the @@ -6324,6 +6327,7 @@ long follow_hugetlb_page(struct mm_struct *mm, struct vm_area_struct *vma, !hugetlbfs_pagecache_present(h, vma, vaddr)) { if (pte) spin_unlock(ptl); + rcu_read_unlock(); remainder = 0; break; } @@ -6345,6 +6349,8 @@ long follow_hugetlb_page(struct mm_struct *mm, struct vm_area_struct *vma, if (pte) spin_unlock(ptl); + rcu_read_unlock(); + if (flags & FOLL_WRITE) fault_flags |= FAULT_FLAG_WRITE; else if (unshare) @@ -6387,6 +6393,19 @@ long follow_hugetlb_page(struct mm_struct *mm, struct vm_area_struct *vma, continue; } + /* + * When reach here, it means the pteval is not absent, so + * anyone who wants to free and invalidate the pgtable page + * (aka, pte*) should need to first unmap the entries which + * relies on the pgtable lock. Since we're holding it, + * we're safe even without RCU anymore. + * + * We can also just release RCU after each unlock of + * pgtable below, but this is just much cleaner, and also + * smaller critical section. + */ + rcu_read_unlock(); + pfn_offset = (vaddr & ~huge_page_mask(h)) >> PAGE_SHIFT; page = pte_page(huge_ptep_get(pte)); From patchwork Sun Oct 30 21:30:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 13062 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp1972983wru; Sun, 30 Oct 2022 14:33:49 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5JvTEkEUVMzNd6HY9+iCOfdBepTXpKVe7fZpzdwlvVaUjMc+Z6hOV2ZyVVbA6sNtdeFZ2X X-Received: by 2002:a17:90b:782:b0:213:9c2c:d047 with SMTP id l2-20020a17090b078200b002139c2cd047mr11688692pjz.144.1667165628999; Sun, 30 Oct 2022 14:33:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667165628; cv=none; d=google.com; s=arc-20160816; b=EHePtTSLtRNYWZZXUV7eokYFi+4rWcsNDj2ruvi+7eqc59Ti+KFpnyY7ibwFVQynm9 nG/UJLINjskBAb84sjlKdTQuJMHEWBCAjDT6WBy2FGNd3Mn/zWXpHyzkEOgNB/oDghRG QkJjp1yl9m2Bsnzx2eIxZ1fJ4mIIIn9ovQ6bet7oZ2jk/Fdt02ZpFCRA8TrB+vpVAv02 1DMOSjZ8XkNpa/YjEWYTyvjc9ZO5gHnR+C/ejIF76VLjFM8LqBE2dNdoTsxInS7uwZ9f /l64ovEP3hTNhNBy4jTYZqhrvhmzPtK7PIHAunzt2x9wNB5rpUb6btGBCHN8c9KFa/3l AZvQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=18dXjMMHISRNlhGaFpCwLJgIneJr5paDcQBCMHn2vKI=; b=LIT9umymbgeDJsPEF3kshaoTVSDUcBWUon8sGNEBdhINI7hwV00nkl7RHHUAyaGnIw 3jD1aKslpGteRPcCFhqBZyJ+DJ0LDtERE6glwwaQwkERwswwj3g+Py10hVmv/xnAA/kJ bgOkXHt3yrv2Bq8eOmJ6cPJXyhsVZc0TpmS1OvfHJZG9SIZCdPW2AOwrn0udHKkUZTgG Wrh1Bwba0EjKLnCJSebwvs2xknKY7u/Vxuh5D2Ao+GF1iE1QQ+NiLuXe+xIT6DBY3+Ht FtNN/+PtUrECRcyb1v9kVjaT7DbsAnVF5yxI5c4IKeMpLIB+HFv/uv1lEsPNMO/zhQng y5iw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=RPpNh4jE; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id b6-20020a056a00114600b00563a0453c55si5640141pfm.268.2022.10.30.14.33.36; Sun, 30 Oct 2022 14:33:48 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=RPpNh4jE; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229528AbiJ3VcA (ORCPT + 99 others); Sun, 30 Oct 2022 17:32:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42336 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229925AbiJ3Vbs (ORCPT ); Sun, 30 Oct 2022 17:31:48 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BBE7BBC81 for ; Sun, 30 Oct 2022 14:30:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1667165447; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=18dXjMMHISRNlhGaFpCwLJgIneJr5paDcQBCMHn2vKI=; b=RPpNh4jEPEssaaPDA/aG59xRNzUMZNX1O+j+TdjkMqX5KZZp+pj+q25cxjNwdoQImZdZuU bl1R+dDKOrCpGLa/t3MfG5JJQkpqrwg4e+2TueDjQkyh7KGjCKJPyC87FgpN/uFz3p/GCr EmGqRHq3u0zyjL8oT4wcopwXxfHkMTw= Received: from mail-qk1-f200.google.com (mail-qk1-f200.google.com [209.85.222.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-338-IcsO8kabPgyEphmiN16L2A-1; Sun, 30 Oct 2022 17:30:46 -0400 X-MC-Unique: IcsO8kabPgyEphmiN16L2A-1 Received: by mail-qk1-f200.google.com with SMTP id bp10-20020a05620a458a00b006fa29f253dcso1142465qkb.11 for ; Sun, 30 Oct 2022 14:30:46 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=18dXjMMHISRNlhGaFpCwLJgIneJr5paDcQBCMHn2vKI=; b=EGBVSY7FfPD9nHOGgh9/tKMGMQkqc51AS14waXVwslDUJaqqhusQ+ZqUZT1jrbmv56 Rw3h86SQ/IpbED6pgH0SVZvKvREgwvKfjVafUY20dMT64bt8CrN/sQOMYzxY1h95ywOW bBl096OxQg16HQyDC8b2KGzrAE2ReQ16IDxEIky7bKPnlEO1AnTUnM7+yP6zScQer8yZ 81v8xxbkcR2X2IyPdwTjRQ4blWW/nuQ/k/0mT8SS/QcTP+jH9n/C8aRD/gGjCHI1ZXC6 1e0zujxCd41mhqOG3JgAJxe4oPkbHs034quBpXrvJuzNwhsnqULBW17ysv60Wkp578nw 7bpA== X-Gm-Message-State: ACrzQf0fiJ8n5prZLhfDlhPcNsIlHVE+F6Z35fJ1bvy5+V5bWl4BLdj2 SiYQNi3IRoawEFKXf8V7yHNCntBaC89t/2dicQcRQXp1x1Ld2WuUpn6hQwsdOtqX6faBOuL2X2y D9T8K/bj4dQXw/MmxeqfqytP2 X-Received: by 2002:a05:6214:262f:b0:4b6:89bc:3ba4 with SMTP id gv15-20020a056214262f00b004b689bc3ba4mr8361028qvb.44.1667165445900; Sun, 30 Oct 2022 14:30:45 -0700 (PDT) X-Received: by 2002:a05:6214:262f:b0:4b6:89bc:3ba4 with SMTP id gv15-20020a056214262f00b004b689bc3ba4mr8361010qvb.44.1667165445716; Sun, 30 Oct 2022 14:30:45 -0700 (PDT) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id i3-20020a05620a248300b006ee9d734479sm3533891qkn.33.2022.10.30.14.30.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 30 Oct 2022 14:30:45 -0700 (PDT) From: Peter Xu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: James Houghton , Mike Kravetz , David Hildenbrand , Andrea Arcangeli , Rik van Riel , peterx@redhat.com, Andrew Morton , Muchun Song , Miaohe Lin , Nadav Amit Subject: [PATCH RFC 09/10] mm/hugetlb: Make hugetlb_fault() RCU-safe Date: Sun, 30 Oct 2022 17:30:43 -0400 Message-Id: <20221030213043.335669-1-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221030212929.335473-1-peterx@redhat.com> References: <20221030212929.335473-1-peterx@redhat.com> MIME-Version: 1.0 Content-type: text/plain X-Spam-Status: No, score=-3.2 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748149866998355081?= X-GMAIL-MSGID: =?utf-8?q?1748149866998355081?= RCU makes sure the pte_t* won't go away from under us. Please refer to the comment above huge_pte_offset() for more information. Signed-off-by: Peter Xu --- mm/hugetlb.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 5dc87e4e6780..6d336d286394 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -5822,6 +5822,8 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, struct vm_area_struct *vma, int need_wait_lock = 0; unsigned long haddr = address & huge_page_mask(h); + /* For huge_pte_offset() */ + rcu_read_lock(); ptep = huge_pte_offset(mm, haddr, huge_page_size(h)); if (ptep) { /* @@ -5830,13 +5832,15 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, struct vm_area_struct *vma, * not actually modifying content here. */ entry = huge_ptep_get(ptep); + rcu_read_unlock(); if (unlikely(is_hugetlb_entry_migration(entry))) { migration_entry_wait_huge(vma, ptep); return 0; } else if (unlikely(is_hugetlb_entry_hwpoisoned(entry))) return VM_FAULT_HWPOISON_LARGE | VM_FAULT_SET_HINDEX(hstate_index(h)); - } + } else + rcu_read_unlock(); /* * Serialize hugepage allocation and instantiation, so that we don't From patchwork Sun Oct 30 21:30:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 13060 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp1972646wru; Sun, 30 Oct 2022 14:32:44 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4HneVZ0z2CwHuHAF0GkXiPN8Q9dB/LZPDCwwx3u4lsJySjP28QaXY/uVKHhhXdQNWWJq1w X-Received: by 2002:a17:903:50e:b0:182:631b:df6f with SMTP id jn14-20020a170903050e00b00182631bdf6fmr10968447plb.66.1667165564580; Sun, 30 Oct 2022 14:32:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667165564; cv=none; d=google.com; s=arc-20160816; b=NP+bdry6oobkD3NqEbWxPyyGAIe6sAkd6+fgtXdUvsjgzGzyyAORjnakXqOgcg9b5b Rhn2lBo81gAzONIwdRTT5XeXaF7QyxMMn9rOIeAxejZmUmlkev1DQEiqzDq4FNl45aip JGQDxLjcUlWO0cdsfI71eUiCJcyTtkk0e427y3JQGxetP47u4XkSPYieaqmZtdm4pFqC G5IcJeYNFlDZE1fpyA2Y05teR2q5AROUTGg5VQ9lKPZiYYXqIwpSlrr4NJBFq7qj5SyO XpvFlnM17df2IePkKjHRm+xzgJ+JmFarllFoqRBLib63r30WjCsedOmMofoHK2RHVjY1 g+1w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=MbdbMcNyE2RWt/K55AkSFMkKO4f3Jk1aXGE/HUSEbTg=; b=RA8Cehtz3/07Xx3m4BpEjVk5I803ADPNgQJqzH2BuV85YssCmNhJ+WY8eaHg5RP4k2 1laQEZPNfYqtvx7YWVYpZw76znh5A5xISLhq4f6pUOxHgZ2EZFJxai7nW34XUHqGLzsK fHrL+oVb+Pe6hmnEtmR2MRvjSsAvXhi/Xz7vl8YqBY+3jdslX9lmOLgrVDKwFq7V2Y7D rMX4IH+IPPP0rq28gJh9/0SBlmCO8lDl47Q4RVRX3KRxD9WDleIkE+QLnAwoVHv3+rIl 0QzMeODf2mpVpzEXZyXnLwl8MbcI/jysHRGtl5cxBx16MCIVCWWNEr0YeKvQGTCkOYMd gHQg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=BS6qw7MN; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id g8-20020a656cc8000000b0043a061cc079si7368778pgw.358.2022.10.30.14.32.32; Sun, 30 Oct 2022 14:32:44 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=BS6qw7MN; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229826AbiJ3VcJ (ORCPT + 99 others); Sun, 30 Oct 2022 17:32:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42324 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229885AbiJ3Vb6 (ORCPT ); Sun, 30 Oct 2022 17:31:58 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 067ADBCAB for ; Sun, 30 Oct 2022 14:30:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1667165450; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MbdbMcNyE2RWt/K55AkSFMkKO4f3Jk1aXGE/HUSEbTg=; b=BS6qw7MNn/JHJKrhXFZ2er9onLButcr1nAeCNucXNFEpX1Ei7MwOZ25D1P09ylrkCUxlWi P9R9yb2OQ8OzCFvu6cxF5EkJGxEgR0odij+gEECUXgo25XoxLIKDgB6nrQkLlLOFFpiB6x rYWvNEvTuMWqRnIP5UdtC0SCHXUkM+Q= Received: from mail-qk1-f199.google.com (mail-qk1-f199.google.com [209.85.222.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-617-Go1VYq51Ptymg0yEMlu4-Q-1; Sun, 30 Oct 2022 17:30:48 -0400 X-MC-Unique: Go1VYq51Ptymg0yEMlu4-Q-1 Received: by mail-qk1-f199.google.com with SMTP id j13-20020a05620a410d00b006e08208eb31so8010670qko.3 for ; Sun, 30 Oct 2022 14:30:48 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MbdbMcNyE2RWt/K55AkSFMkKO4f3Jk1aXGE/HUSEbTg=; b=vYPyoHtdpwqPYuIWghT0jHZACAu08BXGDGzzqpav3AKbK7KbwEqu09XxFkqdZbbQYr VK/BKs2SweK2iYhWujQ/EQWZYQg6gvZXv+Br21oUnkdNqQBhlCHdbOwpoMNnFAgHVN7o LFCiRZ/1z0O/QHbMp2rVcMb2o6WzbiEaEU9taS2riCu2wIj3soT9bI5bv51u4LnxmQhN vOwpKYcPnBsivKYDyj3bSxQipvYcSz5vKuDzOktNn2OCSlIx7xHIrMUBB2c+9yq+gDAu Q+ZZOYTvBvUg8wptp6YSn6VOZz1ZkMAIJBU0iikTUrBlQF2qnByqM6lCL+E2nAl1qOO8 jDqg== X-Gm-Message-State: ACrzQf3u42xVCrCYeDvkwEOwHscGIKA+lNFhmFEAlF7iaiD+VGqH5rjR TeyJ9bFdZa/hu4JrzyLIxWBq6kw9OQADGFzumR6x9ydo740X+f/b5NlUEZJENrv9GpouZtQxeMU KmTrgHq5AHL41SpNeYDDqJihd X-Received: by 2002:a0c:e44d:0:b0:4bb:6814:cacc with SMTP id d13-20020a0ce44d000000b004bb6814caccmr8623248qvm.73.1667165447960; Sun, 30 Oct 2022 14:30:47 -0700 (PDT) X-Received: by 2002:a0c:e44d:0:b0:4bb:6814:cacc with SMTP id d13-20020a0ce44d000000b004bb6814caccmr8623237qvm.73.1667165447749; Sun, 30 Oct 2022 14:30:47 -0700 (PDT) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id bk20-20020a05620a1a1400b006f3e6933bacsm3482823qkb.113.2022.10.30.14.30.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 30 Oct 2022 14:30:47 -0700 (PDT) From: Peter Xu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: James Houghton , Mike Kravetz , David Hildenbrand , Andrea Arcangeli , Rik van Riel , peterx@redhat.com, Andrew Morton , Muchun Song , Miaohe Lin , Nadav Amit Subject: [PATCH RFC 10/10] mm/hugetlb: Comment at rest huge_pte_offset() places Date: Sun, 30 Oct 2022 17:30:45 -0400 Message-Id: <20221030213045.335680-1-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221030212929.335473-1-peterx@redhat.com> References: <20221030212929.335473-1-peterx@redhat.com> MIME-Version: 1.0 Content-type: text/plain X-Spam-Status: No, score=-3.2 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748149799278426923?= X-GMAIL-MSGID: =?utf-8?q?1748149799278426923?= This makes sure that we're covering all the existing huge_pte_offset() callers and mention why they are safe regarding to pmd unsharing. Signed-off-by: Peter Xu --- mm/hugetlb.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 6d336d286394..270bfc578115 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -4822,6 +4822,7 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src, last_addr_mask = hugetlb_mask_last_page(h); for (addr = src_vma->vm_start; addr < src_vma->vm_end; addr += sz) { spinlock_t *src_ptl, *dst_ptl; + /* With vma lock held, safe without RCU */ src_pte = huge_pte_offset(src, addr, sz); if (!src_pte) { addr |= last_addr_mask; @@ -5026,6 +5027,7 @@ int move_hugetlb_page_tables(struct vm_area_struct *vma, hugetlb_vma_lock_write(vma); i_mmap_lock_write(mapping); for (; old_addr < old_end; old_addr += sz, new_addr += sz) { + /* With vma lock held, safe without RCU */ src_pte = huge_pte_offset(mm, old_addr, sz); if (!src_pte) { old_addr |= last_addr_mask; @@ -5097,6 +5099,7 @@ static void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct last_addr_mask = hugetlb_mask_last_page(h); address = start; for (; address < end; address += sz) { + /* With vma lock held, safe without RCU */ ptep = huge_pte_offset(mm, address, sz); if (!ptep) { address |= last_addr_mask; @@ -5402,6 +5405,7 @@ static vm_fault_t hugetlb_wp(struct mm_struct *mm, struct vm_area_struct *vma, mutex_lock(&hugetlb_fault_mutex_table[hash]); hugetlb_vma_lock_read(vma); spin_lock(ptl); + /* With vma lock held, safe without RCU */ ptep = huge_pte_offset(mm, haddr, huge_page_size(h)); if (likely(ptep && pte_same(huge_ptep_get(ptep), pte))) @@ -5440,6 +5444,7 @@ static vm_fault_t hugetlb_wp(struct mm_struct *mm, struct vm_area_struct *vma, * before the page tables are altered */ spin_lock(ptl); + /* With vma lock (and even pgtable lock) held, safe without RCU */ ptep = huge_pte_offset(mm, haddr, huge_page_size(h)); if (likely(ptep && pte_same(huge_ptep_get(ptep), pte))) { /* Break COW or unshare */ @@ -6511,6 +6516,7 @@ unsigned long hugetlb_change_protection(struct vm_area_struct *vma, last_addr_mask = hugetlb_mask_last_page(h); for (; address < end; address += psize) { spinlock_t *ptl; + /* With vma lock held, safe without RCU */ ptep = huge_pte_offset(mm, address, psize); if (!ptep) { address |= last_addr_mask; @@ -7060,7 +7066,14 @@ pte_t *huge_pmd_share(struct mm_struct *mm, struct vm_area_struct *vma, saddr = page_table_shareable(svma, vma, addr, idx); if (saddr) { + /* + * huge_pmd_share() (or say its solo caller, + * huge_pte_alloc()) always takes the hugetlb vma + * lock, so it's always safe to walk the pgtable of + * the process, even without RCU. + */ spte = huge_pte_offset(svma->vm_mm, saddr, + vma_mmu_pagesize(svma)); if (spte) { get_page(virt_to_page(spte)); @@ -7420,6 +7433,7 @@ void hugetlb_unshare_all_pmds(struct vm_area_struct *vma) hugetlb_vma_lock_write(vma); i_mmap_lock_write(vma->vm_file->f_mapping); for (address = start; address < end; address += PUD_SIZE) { + /* With vma lock held, safe without RCU */ ptep = huge_pte_offset(mm, address, sz); if (!ptep) continue;