From patchwork Wed May 17 16:09:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiaqi Yan X-Patchwork-Id: 95403 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1255460vqo; Wed, 17 May 2023 09:14:51 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ43MpoCHiSa23lYAi1KiwLuOo2a6PzfwNhrbqzz41+gk2an1xWfdVUz4hCKAqM7DakLooU3 X-Received: by 2002:a05:6a20:4411:b0:101:2ad0:134c with SMTP id ce17-20020a056a20441100b001012ad0134cmr40691956pzb.45.1684340091556; Wed, 17 May 2023 09:14:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684340091; cv=none; d=google.com; s=arc-20160816; b=eCRHvmFuBoYjgMVfVkYGTTdin6bE+dksLEQPFuuqiOvqV4ylnvmFmzZOP//4nhFZLy u4DbAsN9RgwDu7ej10eF0/swXbJmxBWuR9/ddhS8aYqrwtQmsGNclO/O9PcXULWx2WbB roiRbl3FZOYfJ7BEor/GGRRikkG8YP8rrbOhJKqCez/+E36VDQVGrLuVH8i7+D2Ps8dE WTPPTJuMPMTRHnvLbShH+3iUCCzPwy7EhLCY134qb/2lvlmLGKnqX2KxzSHUehy4xtUg F2yHR6r68EnjI2mSiaG/ZVAUG2ZI9+AnpsbIk3qrWHwPhofBu/uTbuSbkqUaAANXAu1H IX6w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=woRAdaWYV8FD8vab19hJA4MPyBeuK73rImQoSJfr/I0=; b=HS6WYS77LvXVbJJssDPEaA01160yBD4Q+9X/Ckl0gLYhtpIdVwO99TRk8MafPhZq/8 eWc9NYdUeiJccb1K54oI+gA7Gooosgq//cXWVlwyAl7T8Yu15OM4z2P9k1G3uQDXC8p4 gs71yVD/Ch9FJjTHHW3Y6HWfKh4NNpDnlh4PYvPwi+hQybeNqG1/5P3VRtKHPAxp5Grg G5h0jhjzQvzZuyqlQ9tc+L1A5rcBTw3cQXLob5qxWQK6MGnOZ1yYu0H7jiQH4RkJ0zxL zSGeLBf3PuvNa1/bkf/oqo+6ZDr0whS3p3V1LLTKmT5mqlzFJSNz2Fp3+rOnRtPsnt99 UBzw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b="YE1b/zym"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id f26-20020a63755a000000b0051b48085aaasi21779356pgn.858.2023.05.17.09.14.39; Wed, 17 May 2023 09:14:51 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b="YE1b/zym"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231891AbjEQQKH (ORCPT + 99 others); Wed, 17 May 2023 12:10:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37876 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231331AbjEQQKC (ORCPT ); Wed, 17 May 2023 12:10:02 -0400 Received: from mail-pl1-x64a.google.com (mail-pl1-x64a.google.com [IPv6:2607:f8b0:4864:20::64a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F1D24268C for ; Wed, 17 May 2023 09:10:01 -0700 (PDT) Received: by mail-pl1-x64a.google.com with SMTP id d9443c01a7336-1ae515ff6a9so976625ad.0 for ; Wed, 17 May 2023 09:10:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1684339801; x=1686931801; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=woRAdaWYV8FD8vab19hJA4MPyBeuK73rImQoSJfr/I0=; b=YE1b/zymZ9vc5qJrU2yZ6Y6Sk4upJCBRZPNMxGSUkSJ4NmR9re5sc7FtxyFCA6op5Z NYFItdPqItmJDGSzZeWAH3tLZk1Wm8Afz5r3LdakqUg7OhhCWIIdGn7UMn0FZjneF7a+ bisIrHgG+Aut0UuT0TMLzqNQqD7aRu51rTrV0uDZzw1l+caeFdr0GilMHPWzBABPv15M X4eJsAFnhsiuhrdvQNPjyZTVUft46X+7oX5ruBdVoJ1jwWqEudjOGdVLWk0A5g6E9K6h emxl51IemdopSKTESt1Ud6k+QmEevHfSkIuOBN+uVG3cweipz2hw8QLMUNEJwvHVbkK9 kVdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684339801; x=1686931801; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=woRAdaWYV8FD8vab19hJA4MPyBeuK73rImQoSJfr/I0=; b=V7I39NxV/JDfIKRP7L5B8O6aKctKzU03tqiUZpReGw4Ttpwm2YMXKTDPvwReIitHAn bnvsvjKZITUUyDRtw4fJ1lZeD5KyzqWpN1Y2IQMU6zEY3XeLDmswGwfuzj5dGWdnWU/8 FKyolGnQB9LprcjQifwYdFwr3/xBYGt2/G24K76ysGTYw2LCyqEr5g3ZQlu2NnqwooMY C0lC7E4VUL5fHmXrvgbVo2LVdE4lscbihOgN0IgTkPe/tQTYEYWS+VyFVPQAIploebub ZzKMX1egvfOoxNc6+hdkkmQGVmRFUpMUmkO5+fsJPsVnskziBaY9bENrWvIyqyhFVez+ iTbw== X-Gm-Message-State: AC+VfDw8L3KXjrqtayPVq7bob6eEa65tn6rXr+QCVPFtMIuj+xFvPuas 7wDz9d3DP+kfa5f7jOkFi/92digLXIcCvA== X-Received: from yjq3.c.googlers.com ([fda3:e722:ac3:cc00:24:72f4:c0a8:272f]) (user=jiaqiyan job=sendgmr) by 2002:a17:903:334f:b0:1ae:531f:366a with SMTP id ka15-20020a170903334f00b001ae531f366amr637380plb.5.1684339801269; Wed, 17 May 2023 09:10:01 -0700 (PDT) Date: Wed, 17 May 2023 16:09:46 +0000 In-Reply-To: <20230517160948.811355-1-jiaqiyan@google.com> Mime-Version: 1.0 References: <20230517160948.811355-1-jiaqiyan@google.com> X-Mailer: git-send-email 2.40.1.606.ga4b1b128d6-goog Message-ID: <20230517160948.811355-2-jiaqiyan@google.com> Subject: [PATCH v1 1/3] mm/hwpoison: find subpage in hugetlb HWPOISON list From: Jiaqi Yan To: mike.kravetz@oracle.com, songmuchun@bytedance.com, naoya.horiguchi@nec.com, shy828301@gmail.com, linmiaohe@huawei.com Cc: akpm@linux-foundation.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, duenwen@google.com, axelrasmussen@google.com, jthoughton@google.com, Jiaqi Yan X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1766158595691867385?= X-GMAIL-MSGID: =?utf-8?q?1766158595691867385?= Adds the functionality to search a subpage's corresponding raw_hwp_page in hugetlb page's HWPOISON list. This functionality can also tell if a subpage is a raw HWPOISON page. Exports this functionality to be immediately used in the read operation for hugetlbfs. Signed-off-by: Jiaqi Yan Signed-off-by: Mike Kravetz --- include/linux/mm.h | 23 +++++++++++++++++++++++ mm/memory-failure.c | 26 ++++++++++++++++---------- 2 files changed, 39 insertions(+), 10 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 27ce77080c79..f191a4119719 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3683,6 +3683,29 @@ enum mf_action_page_type { */ extern const struct attribute_group memory_failure_attr_group; +#ifdef CONFIG_HUGETLB_PAGE +/* + * Struct raw_hwp_page represents information about "raw error page", + * constructing singly linked list from ->_hugetlb_hwpoison field of folio. + */ +struct raw_hwp_page { + struct llist_node node; + struct page *page; +}; + +static inline struct llist_head *raw_hwp_list_head(struct folio *folio) +{ + return (struct llist_head *)&folio->_hugetlb_hwpoison; +} + +/* + * Given @subpage, a raw page in a hugepage, find its location in @folio's + * _hugetlb_hwpoison list. Return NULL if @subpage is not in the list. + */ +struct raw_hwp_page *find_raw_hwp_page(struct folio *folio, + struct page *subpage); +#endif + #if defined(CONFIG_TRANSPARENT_HUGEPAGE) || defined(CONFIG_HUGETLBFS) extern void clear_huge_page(struct page *page, unsigned long addr_hint, diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 5b663eca1f29..c49e6c2d1f07 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -1818,18 +1818,24 @@ EXPORT_SYMBOL_GPL(mf_dax_kill_procs); #endif /* CONFIG_FS_DAX */ #ifdef CONFIG_HUGETLB_PAGE -/* - * Struct raw_hwp_page represents information about "raw error page", - * constructing singly linked list from ->_hugetlb_hwpoison field of folio. - */ -struct raw_hwp_page { - struct llist_node node; - struct page *page; -}; -static inline struct llist_head *raw_hwp_list_head(struct folio *folio) +struct raw_hwp_page *find_raw_hwp_page(struct folio *folio, + struct page *subpage) { - return (struct llist_head *)&folio->_hugetlb_hwpoison; + struct llist_node *t, *tnode; + struct llist_head *raw_hwp_head = raw_hwp_list_head(folio); + struct raw_hwp_page *hwp_page = NULL; + struct raw_hwp_page *p; + + llist_for_each_safe(tnode, t, raw_hwp_head->first) { + p = container_of(tnode, struct raw_hwp_page, node); + if (subpage == p->page) { + hwp_page = p; + break; + } + } + + return hwp_page; } static unsigned long __folio_free_raw_hwp(struct folio *folio, bool move_flag) From patchwork Wed May 17 16:09:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiaqi Yan X-Patchwork-Id: 95406 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1257999vqo; Wed, 17 May 2023 09:18:12 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7lOEAzNqwC0aB2bdekJycoAZ9/CXFChCb4PWjZSvMCEVsdsRKybTShtnp6+gx/AywN2oJK X-Received: by 2002:a05:6a20:258e:b0:ff:ca91:68ee with SMTP id k14-20020a056a20258e00b000ffca9168eemr3617085pzd.9.1684340292588; Wed, 17 May 2023 09:18:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684340292; cv=none; d=google.com; s=arc-20160816; b=rFs+cYjDQyByraOnxqA/aGbVnBCeNAHhx1qZYNPTyIPP67MKTtJxAbdT5mU1wp/obn fkw9eRRv7GT93oeyU8ybWv2v4aKnnnRlKzn7QGIvq/5P0TayzyTF+aMTY30aPdYXNhGQ 1ouX6ky2PIo1TUTN3e7P9oKd6t4L5+ip3mtxanFKBXsYj/j20H0tFZSLfR+2oqg32TNX dn+eBn2RzVAeYoESlUc5j774Uz79c7O6qm4OaTjvxt7vKO3K7jZWivKmM5cuFdVYyRkT bC8WYy19FqdT7o5gc3fC6gYcLBjnPD+swyelrNRdTnNQM/YaGbkHzuka3M7Whglpa0Jt tTEg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=4afwjJMNRVLZWsRS4KvH1UWrNK7pSNXsAEU4xOupb6Y=; b=q9soSKKJIfb44wKgWN44e3tcqzBn8GG6rOGn0KiUgo+qe23QlY7qzxqUwLo4Wx1f49 1kgPHTmMqPtaym3JKvfWobaWQR4SQ82YL9W8Vox4Nb55qp8DZgcRmeaJUXeBdc3oEIF4 ZuffMoaJ0Ez4gwHehV+X+Hn4f0nWxN20qLfFltN+6losqI7Hi9JgfNfp7W/6HdzgmN4E 1+YK70CJX6a37XJu72ccL+dRSvkZZqbN3oJn9moyluEjEkxMNgOkNi+QpbtIvk+Rmv5e jsOJXPCHccKQpXqDy7MLCuuLrd3XY7nt9t0czahf7RAhSI6YE5V2JfPXfJQv6UK+gk7E yw4w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=axQ41qX1; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id c22-20020aa79536000000b00643994622e4si21905674pfp.98.2023.05.17.09.17.59; Wed, 17 May 2023 09:18:12 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=axQ41qX1; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232037AbjEQQKN (ORCPT + 99 others); Wed, 17 May 2023 12:10:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37888 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231819AbjEQQKF (ORCPT ); Wed, 17 May 2023 12:10:05 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CE8E9268C for ; Wed, 17 May 2023 09:10:03 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-ba83fed50a6so1875273276.0 for ; Wed, 17 May 2023 09:10:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1684339803; x=1686931803; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=4afwjJMNRVLZWsRS4KvH1UWrNK7pSNXsAEU4xOupb6Y=; b=axQ41qX1bwiHJbGU6rrt3d0A/FAXog8UjScM7eAjhvO/VZz1uMpCiXfvNRTVLE3/8S OZBGnYY6NkXRzHCcXWb2deoRCp4d/IhzJGUh0wL1C9VUN2hWUXoDFRR7iXDO/6Wl+fp6 uGwIan2D0pw5vLh98Q1bRnbHw7ENy/oETdEpQ8m8wE2UMa7QnS7eUD2E3CuA2ie5z6mc Q3h9U6cGrBmDrF/iHiQeKEXmB/AqVVSqZ2+ND4BoFj+Gq0VO1MO60ZY7L3bL5eaUrqOU INdMQsIXL5q34h0EXxp3baPOV9jVqICirZUX4wvP0xgrgao+63OSBIe1/yr8wrkYoNgo iHsA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684339803; x=1686931803; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=4afwjJMNRVLZWsRS4KvH1UWrNK7pSNXsAEU4xOupb6Y=; b=gTU8lhy78Fxb6pl31YdIN0kAxN3qB2r4wsr63/5erKEpJdalBKbMYGmvkDnKWGLJPj Vaj4l6LS2nHS4Zz25tebrEchJZlJKKBSfZ30xYHmzIky/LUyWiGpJ8xC23xwCA8kzLvz TWeINWo5Gc8HLowS30nk2v/X9EJUsBpL+YHuswFLl5kswPObo+H+ROnlI+sntHo/GTNx V5IUNiwDmZewFNGKoLYFF+dBbf50utgjL8Y6EdDlStx3H2YpJxHsFWS9jLrVnvPFr/WL IihoIiCwBjKV0AOUpBxNWwLDgstCdbKK00zvSBLJiomA+BFkYjSAhDkMebgFhhHObL1W Rd4Q== X-Gm-Message-State: AC+VfDwIILVy9u0wOG2FXGuGFL0HOWSPH/V5ctpz/Ga//6KQovxxt4iT ZIMydNVgToA+azzC5YB02Qw+8zGhT0+whw== X-Received: from yjq3.c.googlers.com ([fda3:e722:ac3:cc00:24:72f4:c0a8:272f]) (user=jiaqiyan job=sendgmr) by 2002:a25:ce12:0:b0:ba8:1fab:4f99 with SMTP id x18-20020a25ce12000000b00ba81fab4f99mr2620511ybe.9.1684339803118; Wed, 17 May 2023 09:10:03 -0700 (PDT) Date: Wed, 17 May 2023 16:09:47 +0000 In-Reply-To: <20230517160948.811355-1-jiaqiyan@google.com> Mime-Version: 1.0 References: <20230517160948.811355-1-jiaqiyan@google.com> X-Mailer: git-send-email 2.40.1.606.ga4b1b128d6-goog Message-ID: <20230517160948.811355-3-jiaqiyan@google.com> Subject: [PATCH v1 2/3] hugetlbfs: improve read HWPOISON hugepage From: Jiaqi Yan To: mike.kravetz@oracle.com, songmuchun@bytedance.com, naoya.horiguchi@nec.com, shy828301@gmail.com, linmiaohe@huawei.com Cc: akpm@linux-foundation.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, duenwen@google.com, axelrasmussen@google.com, jthoughton@google.com, Jiaqi Yan X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1766158806322133724?= X-GMAIL-MSGID: =?utf-8?q?1766158806322133724?= When a hugepage contains HWPOISON pages, read() fails to read any byte of the hugepage and returns -EIO, although many bytes in the HWPOISON hugepage are readable. Improve this by allowing hugetlbfs_read_iter returns as many bytes as possible. For a requested range [offset, offset + len) that contains HWPOISON page, return [offset, first HWPOISON page addr); the next read attempt will fail and return -EIO. Signed-off-by: Jiaqi Yan --- fs/hugetlbfs/inode.c | 62 +++++++++++++++++++++++++++++++++++++++----- 1 file changed, 56 insertions(+), 6 deletions(-) diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c index ecfdfb2529a3..1baa08ec679f 100644 --- a/fs/hugetlbfs/inode.c +++ b/fs/hugetlbfs/inode.c @@ -282,6 +282,46 @@ hugetlb_get_unmapped_area(struct file *file, unsigned long addr, } #endif +/* + * Someone wants to read @bytes from a HWPOISON hugetlb @page from @offset. + * Returns the maximum number of bytes one can read without touching the 1st raw + * HWPOISON subpage. + * + * The implementation borrows the iteration logic from copy_page_to_iter*. + */ +static size_t adjust_range_hwpoison(struct page *page, size_t offset, size_t bytes) +{ + size_t n = 0; + size_t res = 0; + struct folio *folio = page_folio(page); + + folio_lock(folio); + + /* First subpage to start the loop. */ + page += offset / PAGE_SIZE; + offset %= PAGE_SIZE; + while (1) { + if (find_raw_hwp_page(folio, page) != NULL) + break; + + /* Safe to read n bytes without touching HWPOISON subpage. */ + n = min(bytes, (size_t)PAGE_SIZE - offset); + res += n; + bytes -= n; + if (!bytes || !n) + break; + offset += n; + if (offset == PAGE_SIZE) { + page++; + offset = 0; + } + } + + folio_unlock(folio); + + return res; +} + /* * Support for read() - Find the page attached to f_mapping and copy out the * data. This provides functionality similar to filemap_read(). @@ -300,7 +340,7 @@ static ssize_t hugetlbfs_read_iter(struct kiocb *iocb, struct iov_iter *to) while (iov_iter_count(to)) { struct page *page; - size_t nr, copied; + size_t nr, copied, want; /* nr is the maximum number of bytes to copy from this page */ nr = huge_page_size(h); @@ -328,16 +368,26 @@ static ssize_t hugetlbfs_read_iter(struct kiocb *iocb, struct iov_iter *to) } else { unlock_page(page); - if (PageHWPoison(page)) { - put_page(page); - retval = -EIO; - break; + if (!PageHWPoison(page)) + want = nr; + else { + /* + * Adjust how many bytes safe to read without + * touching the 1st raw HWPOISON subpage after + * offset. + */ + want = adjust_range_hwpoison(page, offset, nr); + if (want == 0) { + put_page(page); + retval = -EIO; + break; + } } /* * We have the page, copy it to user space buffer. */ - copied = copy_page_to_iter(page, offset, nr, to); + copied = copy_page_to_iter(page, offset, want, to); put_page(page); } offset += copied; From patchwork Wed May 17 16:09:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiaqi Yan X-Patchwork-Id: 95404 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1256689vqo; Wed, 17 May 2023 09:16:23 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7pIe2/oXr9soMLOShBFZD8RreoJnSqbJ7shLQXBIwkQ/VSBRhlJYJltAYmR0uycXXfAxsC X-Received: by 2002:a05:6a00:2291:b0:64b:43d8:a575 with SMTP id f17-20020a056a00229100b0064b43d8a575mr168170pfe.25.1684340182950; Wed, 17 May 2023 09:16:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684340182; cv=none; d=google.com; s=arc-20160816; b=mbVcxHP2xIU8+FNbcdaO6Pi3gMTx8ENN1TA6n5Tr5Eur+97bsFI2eYopOs7JhFWIeN XOnuOAcz7mFJ0ElmH1wqQtHkk1CijPDFL4/bS17+NClxFlHtmLysDnfxRTxURytxbtVY G7uQcSmbXg6WZfHrfBl47Xi+4CZeyvm5Ku4ipofEltPASfUGsE3wohCxzDkWmTzfUbmg p1YvZQHEqprwDxMDwtHkPH4p1ytoNpGWhIQJjYMcN6TbSckeInm9ZUzTx5Let/F7mKi8 rnuEnyliH/PgjtKn8JN5S5Bwzu9M5gKKutU3W6T4Rr+gY8s076wFkkG+yZsAtB+cOfEo 5UHA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=kGxg+zhhm/tdTqkE4MZxwuMTE52MydtCt+SDG4bfPzg=; b=uZlLQ4RkuXqpFsrtKMgPyjAugJZLApYjP+gDB2KWWoYYgSC8OaYPENkQLnb5iR4Tiv xhpsGg67+Vsf3LMJ0Rf5an+zqbKGxvSOiXWu05UaCP/FbbsDjZqkIPeIMaxtysGJc3OF +UspypxKZfe1mykosv+ntezSLWpCtRXaK6HgtsJgjh30IjMJCM06dBPfbpGx6eUj5RKm KE7RyzIsTbxQ0+QjuCpdx0C81rk9qBSq8bqKqs2lDO3UrZkWezWv2aHnCDrel029SoOb /CjNIuaD6IMrcwfxOfTgKc+zwxRkx79QRv+BSZgAAOqfWceMZQ11cVszCbg5IGYP4zag fFoQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=rX0HksKs; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id r28-20020aa7963c000000b00646df7a783bsi21905695pfg.118.2023.05.17.09.16.10; Wed, 17 May 2023 09:16:22 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=rX0HksKs; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232052AbjEQQKR (ORCPT + 99 others); Wed, 17 May 2023 12:10:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37920 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231863AbjEQQKG (ORCPT ); Wed, 17 May 2023 12:10:06 -0400 Received: from mail-pj1-x104a.google.com (mail-pj1-x104a.google.com [IPv6:2607:f8b0:4864:20::104a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2FB635FC4 for ; Wed, 17 May 2023 09:10:05 -0700 (PDT) Received: by mail-pj1-x104a.google.com with SMTP id 98e67ed59e1d1-2505b80d5b3so98423a91.0 for ; Wed, 17 May 2023 09:10:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1684339804; x=1686931804; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=kGxg+zhhm/tdTqkE4MZxwuMTE52MydtCt+SDG4bfPzg=; b=rX0HksKsk0IJFI+DSWwGwnSF1zsM0Y/3NSgkTE4ZsUpxUBwgEnbw18PdOa4svGvURJ Z/ZtfKN7U1NEhdwFryCTbpt629Lk3erFtvp0ks79CR7fNs4DF9ZUJ7r3rymcKmX1jxLx 1/8VzjvNjUTrOOzM0zqxVIj6gBRb/Q1AyjBmfpBAPMAjRyv7K+bR6aS1jV2AcucecG09 PiV2VwOn0EFurlAEbZnmRq2zSQ+a5Lqd1uCwtPeNFlwqIyEkdfG3BejPJqOgGnQcijDm K4RmMrWMkOQF5LZYJhUk5lnzr6Fi4LUAi6tHOOgtS4eRQeDaORBjAtuvBswrm2NGd2m/ fGHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684339804; x=1686931804; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=kGxg+zhhm/tdTqkE4MZxwuMTE52MydtCt+SDG4bfPzg=; b=PpFCASIk1dtmCoEfETyJXRU3Q9Y2AW9xBqL3P68GDdqvPeYGrf9PKefw3FhKj/dSfS fpnTm2JAQhaZl02t4ssnF+APTdeyEavoTS6auMuhs/9IWvsnVBHJbYV1qHmXquG2o/if y0q1D/t5MuoCToHOn6rjngQS8ZvITMPkmkjRcgRWfPHRZ3fLXy9cGCQmlpT98KOnDTCN zHk/anA0BUOVCp1wFYo6DvxkZcgB7mYGg7Osi1ZlDOGNvZJsXc89j+gL+M9lLT6H4Bx+ PRTA1bp6FNds2dN6PO518Um0ke92CR+pfi1Rm56z3jPfrdRwhPaPI8yhjKPG9ewAKkl+ KcRg== X-Gm-Message-State: AC+VfDwn0hKadsoCUy0HhAaDZwZrb75e/lQUDOd8CkwBLL+2vu7Zs4Cz MsAK+lF5xwOFvyAV8l/kTiPFn7M2il9GXw== X-Received: from yjq3.c.googlers.com ([fda3:e722:ac3:cc00:24:72f4:c0a8:272f]) (user=jiaqiyan job=sendgmr) by 2002:a17:90b:89:b0:253:4800:438b with SMTP id bb9-20020a17090b008900b002534800438bmr7216pjb.2.1684339804718; Wed, 17 May 2023 09:10:04 -0700 (PDT) Date: Wed, 17 May 2023 16:09:48 +0000 In-Reply-To: <20230517160948.811355-1-jiaqiyan@google.com> Mime-Version: 1.0 References: <20230517160948.811355-1-jiaqiyan@google.com> X-Mailer: git-send-email 2.40.1.606.ga4b1b128d6-goog Message-ID: <20230517160948.811355-4-jiaqiyan@google.com> Subject: [PATCH v1 3/3] selftests/mm: add tests for HWPOISON hugetlbfs read From: Jiaqi Yan To: mike.kravetz@oracle.com, songmuchun@bytedance.com, naoya.horiguchi@nec.com, shy828301@gmail.com, linmiaohe@huawei.com Cc: akpm@linux-foundation.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, duenwen@google.com, axelrasmussen@google.com, jthoughton@google.com, Jiaqi Yan X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1766158691733283287?= X-GMAIL-MSGID: =?utf-8?q?1766158691733283287?= Add tests for the improvement made to read operations on HWPOISON hugetlb page with different read granularities. 0) Simple regression test on read. 1) Sequential read page by page should succeed until encounters the 1st raw HWPOISON subpage. 2) After skip raw HWPOISON subpage by lseek, read always succeeds. Signed-off-by: Jiaqi Yan --- tools/testing/selftests/mm/.gitignore | 1 + tools/testing/selftests/mm/Makefile | 1 + .../selftests/mm/hugetlb-read-hwpoison.c | 322 ++++++++++++++++++ 3 files changed, 324 insertions(+) create mode 100644 tools/testing/selftests/mm/hugetlb-read-hwpoison.c diff --git a/tools/testing/selftests/mm/.gitignore b/tools/testing/selftests/mm/.gitignore index 8917455f4f51..fe8224d2ee06 100644 --- a/tools/testing/selftests/mm/.gitignore +++ b/tools/testing/selftests/mm/.gitignore @@ -5,6 +5,7 @@ hugepage-mremap hugepage-shm hugepage-vmemmap hugetlb-madvise +hugetlb-read-hwpoison khugepaged map_hugetlb map_populate diff --git a/tools/testing/selftests/mm/Makefile b/tools/testing/selftests/mm/Makefile index 23af4633f0f4..6cc63668c50e 100644 --- a/tools/testing/selftests/mm/Makefile +++ b/tools/testing/selftests/mm/Makefile @@ -37,6 +37,7 @@ TEST_GEN_PROGS += compaction_test TEST_GEN_PROGS += gup_test TEST_GEN_PROGS += hmm-tests TEST_GEN_PROGS += hugetlb-madvise +TEST_GEN_PROGS += hugetlb-read-hwpoison TEST_GEN_PROGS += hugepage-mmap TEST_GEN_PROGS += hugepage-mremap TEST_GEN_PROGS += hugepage-shm diff --git a/tools/testing/selftests/mm/hugetlb-read-hwpoison.c b/tools/testing/selftests/mm/hugetlb-read-hwpoison.c new file mode 100644 index 000000000000..2f8e84eceb3d --- /dev/null +++ b/tools/testing/selftests/mm/hugetlb-read-hwpoison.c @@ -0,0 +1,322 @@ +// SPDX-License-Identifier: GPL-2.0 + +#define _GNU_SOURCE +#include +#include +#include + +#include +#include +#include +#include +#include + +#include "../kselftest.h" + +#define PREFIX " ... " +#define ERROR_PREFIX " !!! " + +#define MAX_WRITE_READ_CHUNK_SIZE (getpagesize() * 16) +#define MAX(a, b) (((a) > (b)) ? (a) : (b)) + +enum test_status { + TEST_PASSED = 0, + TEST_FAILED = 1, + TEST_SKIPPED = 2, +}; + +static char *status_to_str(enum test_status status) +{ + switch (status) { + case TEST_PASSED: + return "TEST_PASSED"; + case TEST_FAILED: + return "TEST_FAILED"; + case TEST_SKIPPED: + return "TEST_SKIPPED"; + default: + return "TEST_???"; + } +} + +static int setup_filemap(char *filemap, size_t len, size_t wr_chunk_size) +{ + char iter = 0; + + for (size_t offset = 0; offset < len; + offset += wr_chunk_size) { + iter++; + memset(filemap + offset, iter, wr_chunk_size); + } + + return 0; +} + +static bool verify_chunk(char *buf, size_t len, char val) +{ + size_t i; + + for (i = 0; i < len; ++i) { + if (buf[i] != val) { + printf(ERROR_PREFIX "check fail: buf[%lu] = %u != %u\n", + i, buf[i], val); + return false; + } + } + + return true; +} + +static bool seek_read_hugepage_filemap(int fd, size_t len, size_t wr_chunk_size, + off_t offset, size_t expected) +{ + char buf[MAX_WRITE_READ_CHUNK_SIZE]; + ssize_t ret_count = 0; + ssize_t total_ret_count = 0; + char val = offset / wr_chunk_size + offset % wr_chunk_size; + + printf(PREFIX "init val=%u with offset=0x%lx\n", val, offset); + printf(PREFIX "expect to read 0x%lx bytes of data in total\n", + expected); + if (lseek(fd, offset, SEEK_SET) < 0) { + perror(ERROR_PREFIX "seek failed"); + return false; + } + + while (offset + total_ret_count < len) { + ret_count = read(fd, buf, wr_chunk_size); + if (ret_count == 0) { + printf(PREFIX "read reach end of the file\n"); + break; + } else if (ret_count < 0) { + perror(ERROR_PREFIX "read failed"); + break; + } + ++val; + if (!verify_chunk(buf, ret_count, val)) + return false; + + total_ret_count += ret_count; + } + printf(PREFIX "actually read 0x%lx bytes of data in total\n", + total_ret_count); + + return total_ret_count == expected; +} + +static bool read_hugepage_filemap(int fd, size_t len, + size_t wr_chunk_size, size_t expected) +{ + char buf[MAX_WRITE_READ_CHUNK_SIZE]; + ssize_t ret_count = 0; + ssize_t total_ret_count = 0; + char val = 0; + + printf(PREFIX "expect to read 0x%lx bytes of data in total\n", + expected); + while (total_ret_count < len) { + ret_count = read(fd, buf, wr_chunk_size); + if (ret_count == 0) { + printf(PREFIX "read reach end of the file\n"); + break; + } else if (ret_count < 0) { + perror(ERROR_PREFIX "read failed"); + break; + } + ++val; + if (!verify_chunk(buf, ret_count, val)) + return false; + + total_ret_count += ret_count; + } + printf(PREFIX "actually read 0x%lx bytes of data in total\n", + total_ret_count); + + return total_ret_count == expected; +} + +static enum test_status +test_hugetlb_read(int fd, size_t len, size_t wr_chunk_size) +{ + enum test_status status = TEST_SKIPPED; + char *filemap = NULL; + + if (ftruncate(fd, len) < 0) { + perror(ERROR_PREFIX "ftruncate failed"); + return status; + } + + filemap = mmap(NULL, len, PROT_READ | PROT_WRITE, + MAP_SHARED | MAP_POPULATE, fd, 0); + if (filemap == MAP_FAILED) { + perror(ERROR_PREFIX "mmap for primary mapping failed"); + goto done; + } + + setup_filemap(filemap, len, wr_chunk_size); + status = TEST_FAILED; + + if (read_hugepage_filemap(fd, len, wr_chunk_size, len)) + status = TEST_PASSED; + + munmap(filemap, len); +done: + if (ftruncate(fd, 0) < 0) { + perror(ERROR_PREFIX "ftruncate back to 0 failed"); + status = TEST_FAILED; + } + + return status; +} + +static enum test_status +test_hugetlb_read_hwpoison(int fd, size_t len, size_t wr_chunk_size, + bool skip_hwpoison_page) +{ + enum test_status status = TEST_SKIPPED; + char *filemap = NULL; + char *hwp_addr = NULL; + const unsigned long pagesize = getpagesize(); + + if (ftruncate(fd, len) < 0) { + perror(ERROR_PREFIX "ftruncate failed"); + return status; + } + + filemap = mmap(NULL, len, PROT_READ | PROT_WRITE, + MAP_SHARED | MAP_POPULATE, fd, 0); + if (filemap == MAP_FAILED) { + perror(ERROR_PREFIX "mmap for primary mapping failed"); + goto done; + } + + setup_filemap(filemap, len, wr_chunk_size); + status = TEST_FAILED; + + /* + * Poisoned hugetlb page layout (assume hugepagesize=2MB): + * |<---------------------- 1MB ---------------------->| + * |<---- healthy page ---->|<---- HWPOISON page ----->| + * |<------------------- (1MB - 8KB) ----------------->| + */ + hwp_addr = filemap + len / 2 + pagesize; + if (madvise(hwp_addr, pagesize, MADV_HWPOISON) < 0) { + perror(ERROR_PREFIX "MADV_HWPOISON failed"); + goto unmap; + } + + if (!skip_hwpoison_page) { + /* + * Userspace should be able to read (1MB + 1 page) from + * the beginning of the HWPOISONed hugepage. + */ + if (read_hugepage_filemap(fd, len, wr_chunk_size, + len / 2 + pagesize)) + status = TEST_PASSED; + } else { + /* + * Userspace should be able to read (1MB - 2 pages) from + * HWPOISONed hugepage. + */ + if (seek_read_hugepage_filemap(fd, len, wr_chunk_size, + len / 2 + MAX(2 * pagesize, wr_chunk_size), + len / 2 - MAX(2 * pagesize, wr_chunk_size))) + status = TEST_PASSED; + } + +unmap: + munmap(filemap, len); +done: + if (ftruncate(fd, 0) < 0) { + perror(ERROR_PREFIX "ftruncate back to 0 failed"); + status = TEST_FAILED; + } + + return status; +} + +static int create_hugetlbfs_file(struct statfs *file_stat) +{ + int fd; + + fd = memfd_create("hugetlb_tmp", MFD_HUGETLB); + if (fd < 0) { + perror(ERROR_PREFIX "could not open hugetlbfs file"); + return -1; + } + + memset(file_stat, 0, sizeof(*file_stat)); + if (fstatfs(fd, file_stat)) { + perror(ERROR_PREFIX "fstatfs failed"); + goto close; + } + if (file_stat->f_type != HUGETLBFS_MAGIC) { + printf(ERROR_PREFIX "not hugetlbfs file\n"); + goto close; + } + + return fd; +close: + close(fd); + return -1; +} + +int main(void) +{ + int fd; + struct statfs file_stat; + enum test_status status; + /* Test read() in different granularity. */ + size_t wr_chunk_sizes[] = { + getpagesize() / 2, getpagesize(), + getpagesize() * 2, getpagesize() * 4 + }; + size_t i; + + for (i = 0; i < ARRAY_SIZE(wr_chunk_sizes); ++i) { + printf(PREFIX "Write/read chunk size=0x%lx\n", + wr_chunk_sizes[i]); + + fd = create_hugetlbfs_file(&file_stat); + if (fd < 0) + goto create_failure; + printf("HugeTLB read regression test...\n"); + status = test_hugetlb_read(fd, file_stat.f_bsize, + wr_chunk_sizes[i]); + printf("HugeTLB read regression test...%s\n", + status_to_str(status)); + close(fd); + if (status == TEST_FAILED) + return -1; + + fd = create_hugetlbfs_file(&file_stat); + if (fd < 0) + goto create_failure; + printf("HugeTLB read HWPOISON test...\n"); + status = test_hugetlb_read_hwpoison(fd, file_stat.f_bsize, + wr_chunk_sizes[i], false); + printf("HugeTLB read HWPOISON test...%s\n", + status_to_str(status)); + close(fd); + if (status == TEST_FAILED) + return -1; + + fd = create_hugetlbfs_file(&file_stat); + if (fd < 0) + goto create_failure; + printf("HugeTLB seek then read HWPOISON test...\n"); + status = test_hugetlb_read_hwpoison(fd, file_stat.f_bsize, + wr_chunk_sizes[i], true); + printf("HugeTLB seek then read HWPOISON test...%s\n", + status_to_str(status)); + close(fd); + if (status == TEST_FAILED) + return -1; + } + + return 0; + +create_failure: + printf(ERROR_PREFIX "Abort test: failed to create hugetlbfs file\n"); + return -1; +}