From patchwork Fri Sep 15 18:36:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 14087 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp1440934vqi; Fri, 15 Sep 2023 19:33:43 -0700 (PDT) X-Google-Smtp-Source: AGHT+IF4c+QnX+wuZI5HK+1xGEyCRXYCv8kIT/SiIe53KHWK34WUc3p8D8eeuag70St9lV6qPSBf X-Received: by 2002:a17:903:2345:b0:1b2:5d5b:e871 with SMTP id c5-20020a170903234500b001b25d5be871mr3707554plh.59.1694831623439; Fri, 15 Sep 2023 19:33:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694831623; cv=none; d=google.com; s=arc-20160816; b=S6WdSuxaL/u1h9TW4n3Qb40p+Y5+uwkqXhtGpOmiedI1gV70XYsc1SaCaGSqpTrhk6 wKQuiBf4YQb/p5DFX+7iaIsToQOsHfzAGilB8SWkDm82kbFM6voOd7VrEea/3DCKerd5 HKoYg6KBMJ+hIJ1Z29ktAQtGHhFyxBHkukSFFhNyjqQ3VDCmzzmUY5AkPj+D/mf47KBS SV55t5wNqA1+jlW+b650gUzMOFnL4l6yE76UB0rGCBulyS10OZoSElD1LGzsnB2sFKjW tW9DrC9SSfD5+gwHVPvOgcE8wGKQZT1Bg7rWQCztMfTrfh/VsZUG7u08ipQh+iZOth05 /j1w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=JjdcCeARBetsZH137aFl33BJ3ii6NFAsip2SWa5M3tY=; fh=Exu8p2J0bt/MoYkEByA/Woj5EWTdUoRsvJYmK88z1hY=; b=B92KpjucobokENM+j+lNtAcUA6FdAYoEzmubm+uxoD/mOay9BpduL1aAC7Me5H9fth TCQXIQNN8MIgA8xoRUzgaTv6MsMOicTj2xH4+Ku5/s8hKwWcMcgQxwvvDt7Le5mzHZg0 a9J6Q/gpf+sToR6DdfKr1EayQ3gL8lnwOhhgO+T65BtI3wivbaWPlJl25mCk+76WnNC3 WnDvQXvEBH4Fv2Z84LW9m1UrsXwNJoiOOvE/vwOK+plSDAUg5PfsI1mAW7m7awJQJM4E osuHRmo4/UANIPjkq+5/AVUXHxEkn5b2TW12o3KFAXV1uzC39bd+cAFgBYfD+pbFd+Cs 7omg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@infradead.org header.s=casper.20170209 header.b=p17pTr3s; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from lipwig.vger.email (lipwig.vger.email. [2620:137:e000::3:3]) by mx.google.com with ESMTPS id u6-20020a170902e80600b001c383a64ebesi4344229plg.319.2023.09.15.19.33.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Sep 2023 19:33:43 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) client-ip=2620:137:e000::3:3; Authentication-Results: mx.google.com; dkim=pass header.i=@infradead.org header.s=casper.20170209 header.b=p17pTr3s; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id 2213582972FB; Fri, 15 Sep 2023 11:39:09 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236777AbjIOSiV (ORCPT + 30 others); Fri, 15 Sep 2023 14:38:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33916 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236544AbjIOShr (ORCPT ); Fri, 15 Sep 2023 14:37:47 -0400 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 736281BD3; Fri, 15 Sep 2023 11:37:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:In-Reply-To:References; bh=JjdcCeARBetsZH137aFl33BJ3ii6NFAsip2SWa5M3tY=; b=p17pTr3se0JO/f6M04cDpPtP+5 K/0tbeZ6PTE1lD28UmJtDp+MmcL/fkV6rpoqjooMTaFiT6A+wiZNO3LfFsTW0EJg0CtGJ3T2Htj7q 2vPlqyDoipOOe5joiIo8yawKYHtz72l/cRIJRH1Y7fKZZaXRjPnL96TuNG3d7I5exwb2fOz2bdJZx hrZHp1wcxwHcn37XwnLC4Q98HIXkwl0RW4mQtwsuzJZ06on3fs6Qb0d6uUwUI1wZ+FxH8pQT4w73L qS3WAWId/CoC7Q2weH7Kr/MQIOEamN7K1gZKEzWnitOC6RJa5VvnCInxTb738HTzJvu/bfYYBYciq GcGIN+Ig==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qhDgi-00BMIL-Vs; Fri, 15 Sep 2023 18:37:09 +0000 From: "Matthew Wilcox (Oracle)" To: linux-kernel@vger.kernel.org Cc: "Matthew Wilcox (Oracle)" , linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, torvalds@linux-foundation.org, Nicholas Piggin Subject: [PATCH 00/17] Add folio_end_read Date: Fri, 15 Sep 2023 19:36:50 +0100 Message-Id: <20230915183707.2707298-1-willy@infradead.org> X-Mailer: git-send-email 2.37.1 MIME-Version: 1.0 X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Fri, 15 Sep 2023 11:39:09 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777159764104687643 X-GMAIL-MSGID: 1777159764104687643 The core of this patchset is the new folio_end_read() call which filesystems can use when finishing a page cache read instead of separate calls to mark the folio uptodate and unlock it. As an illustration of its use, I converted ext4, iomap & mpage; more can be converted. I think that's useful by itself, but the interesting optimisation is that we can implement that with a single XOR instruction that sets the uptodate bit, clears the lock bit, tests the waiter bit and provides a write memory barrier. That removes one memory barrier and one atomic instruction from each page read, which seems worth doing. That's in patch 15. The last two patches could be a separate series, but basically we can do the same thing with the writeback flag that we do with the unlock flag; clear it and test the waiters bit at the same time. I don't have any performance numbers; I'm hoping Nick might provide some since PPC seems particularly unhappy with write-after-write hazards. Matthew Wilcox (Oracle) (17): iomap: Hold state_lock over call to ifs_set_range_uptodate() iomap: Protect read_bytes_pending with the state_lock mm: Add folio_end_read() ext4: Use folio_end_read() buffer: Use folio_end_read() iomap: Use folio_end_read() bitops: Add xor_unlock_is_negative_byte() alpha: Implement xor_unlock_is_negative_byte m68k: Implement xor_unlock_is_negative_byte mips: Implement xor_unlock_is_negative_byte powerpc: Implement arch_xor_unlock_is_negative_byte on 32-bit riscv: Implement xor_unlock_is_negative_byte s390: Implement arch_xor_unlock_is_negative_byte mm: Delete checks for xor_unlock_is_negative_byte() mm: Add folio_xor_flags_has_waiters() mm: Make __end_folio_writeback() return void mm: Use folio_xor_flags_has_waiters() in folio_end_writeback() arch/alpha/include/asm/bitops.h | 20 +++++ arch/m68k/include/asm/bitops.h | 13 ++++ arch/mips/include/asm/bitops.h | 25 +++++- arch/mips/lib/bitops.c | 14 ++++ arch/powerpc/include/asm/bitops.h | 21 ++--- arch/riscv/include/asm/bitops.h | 12 +++ arch/s390/include/asm/bitops.h | 10 +++ arch/x86/include/asm/bitops.h | 11 ++- fs/buffer.c | 16 +--- fs/ext4/readpage.c | 14 +--- fs/iomap/buffered-io.c | 55 ++++++++----- .../asm-generic/bitops/instrumented-lock.h | 28 ++++--- include/asm-generic/bitops/lock.h | 20 +---- include/linux/page-flags.h | 19 +++++ include/linux/pagemap.h | 1 + kernel/kcsan/kcsan_test.c | 9 +-- kernel/kcsan/selftest.c | 9 +-- mm/filemap.c | 77 ++++++++++--------- mm/kasan/kasan_test.c | 8 +- mm/page-writeback.c | 35 ++++----- 20 files changed, 248 insertions(+), 169 deletions(-)