From patchwork Wed Oct 25 14:45:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 158145 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:d641:0:b0:403:3b70:6f57 with SMTP id cy1csp1310vqb; Wed, 25 Oct 2023 07:47:23 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGkefkcbG0e3bDbx0I/kVEpXl0wHtVeNTOz4Ry4pQUMXNJ28DTa6u8AGuDuGfLM9SNdw/+1 X-Received: by 2002:a0d:f285:0:b0:5ad:4975:c860 with SMTP id b127-20020a0df285000000b005ad4975c860mr4850532ywf.39.1698245235993; Wed, 25 Oct 2023 07:47:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1698245235; cv=none; d=google.com; s=arc-20160816; b=lCCfsG8+CapmYwb23MBvzk/ezJEeT/+I1VqEyHc5DBk8W7/nNbn+HR73nf5Mg89+GI S5nx6FXwFY2fYMYUlngyX+9cnWieY6/gnOriXbr6utLg/aJnX5PwdUX4Ce2YhkQch7Xe I0vPRTAYuibeXfm9rqyOxrLk/HdUHZYDe37aMl4FdKFkQs54IE9EKEoikF4rmCnOhDtl q0L2i/R9VFoQ8JkM4BpVDyBe3oX3vwn4Wm1yvwy61YSA9SsQyt9L/g/I+5fua51gbfAO kXaJEPV3y4ceDs6dS4PZaFwU/APDrQ4+qmImG0u1UoBxXT6e6r3g4kKauM3uV8cs3adf QvjQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=D4xovi6VfiUHc7H1WexA1wOCaWXMFwz43AMZCLiNoqA=; fh=odHZZLn5aUrUhssH6KSiJJlFsZ7wTjfMLFlHN8avkCo=; b=WlrTlK1NH89vkVVPPdgm7QZIu5Wf3bbf7rpxkGzcbs4pBocU7RKr4TDERtAx0pYvje tcrO4CtsPMyHa9E//EA4duE5So3N4IPehwY6xPPJgm1exd0i8JMgDSa+WQBuaLoHmeNv v0XPaKWmDAYRl87HmhtfJI/jpFQD1c1E+/JbqyX/N9Go1Y8DFJf4HaG3ZGcAq+odo26h Hrbjp9i1OJiX1ylhOtS9YKynLrT+A5C3F282l8/ylKeiMRvayeD5vfuTll/TpbX6nkIo mGX5h+Gr0ZzUwALn/hvxEG0eRdpRAEVc4YKIAJiN6to/nnqSNKDByf15dhbnQuoxofRK uymA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from howler.vger.email (howler.vger.email. [23.128.96.34]) by mx.google.com with ESMTPS id u136-20020a0deb8e000000b005a22a115e6csi11138536ywe.325.2023.10.25.07.47.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 Oct 2023 07:47:15 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) client-ip=23.128.96.34; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id 2399D802D0C0; Wed, 25 Oct 2023 07:47:05 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344142AbjJYOqL (ORCPT + 26 others); Wed, 25 Oct 2023 10:46:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32882 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1343659AbjJYOqE (ORCPT ); Wed, 25 Oct 2023 10:46:04 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 940F9B0 for ; Wed, 25 Oct 2023 07:46:01 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id EB8F71477; Wed, 25 Oct 2023 07:46:42 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.26]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id CC0443F64C; Wed, 25 Oct 2023 07:45:59 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , David Hildenbrand , Matthew Wilcox , Huang Ying , Gao Xiang , Yu Zhao , Yang Shi , Michal Hocko , Kefeng Wang Cc: Ryan Roberts , linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v3 3/4] mm: swap: Simplify ssd behavior when scanner steals entry Date: Wed, 25 Oct 2023 15:45:45 +0100 Message-Id: <20231025144546.577640-4-ryan.roberts@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231025144546.577640-1-ryan.roberts@arm.com> References: <20231025144546.577640-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on howler.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Wed, 25 Oct 2023 07:47:05 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780739196493443263 X-GMAIL-MSGID: 1780739196493443263 When a CPU fails to reserve a cluster (due to free list exhaustion), we revert to the scanner to find a free entry somewhere in the swap file. This might cause an entry to be stolen from another CPU's reserved cluster. Upon noticing this, the CPU with the stolen entry would previously scan forward to the end of the cluster trying to find a free entry to use. If there were none, it would try to reserve a new pre-cpu cluster and allocate from that. This scanning behavior does not scale well to high-order allocations, which will be introduced in a future patch since would need to scan for a contiguous area that was naturally aligned. Given stealing is a rare occurrence, let's remove the scanning behavior from the ssd allocator and simply drop the cluster and try to allocate a new one. Given the purpose of the per-cpu cluster is to ensure a given task's pages are sequential on disk to aid readahead, allocating a new cluster at this point makes most sense. Furthermore, si->max will always be greater than or equal to the end of the last cluster because any partial cluster will never be put on the free cluster list. Therefore we can simplify this logic too. These changes make it simpler to generalize scan_swap_map_try_ssd_cluster() to handle any allocation order. Signed-off-by: Ryan Roberts --- mm/swapfile.c | 21 +++++++++------------ 1 file changed, 9 insertions(+), 12 deletions(-) diff --git a/mm/swapfile.c b/mm/swapfile.c index 617e34b8cdbe..94f7cc225eb9 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -639,27 +639,24 @@ static bool scan_swap_map_try_ssd_cluster(struct swap_info_struct *si, /* * Other CPUs can use our cluster if they can't find a free cluster, - * check if there is still free entry in the cluster + * check if the expected entry is still free. If not, drop it and + * reserve a new cluster. */ - max = min_t(unsigned long, si->max, - ALIGN_DOWN(tmp, SWAPFILE_CLUSTER) + SWAPFILE_CLUSTER); - if (tmp < max) { - ci = lock_cluster(si, tmp); - while (tmp < max) { - if (!si->swap_map[tmp]) - break; - tmp++; - } + ci = lock_cluster(si, tmp); + if (si->swap_map[tmp]) { unlock_cluster(ci); - } - if (tmp >= max) { *cpu_next = SWAP_NEXT_NULL; goto new_cluster; } + unlock_cluster(ci); + *offset = tmp; *scan_base = tmp; + + max = ALIGN_DOWN(tmp, SWAPFILE_CLUSTER) + SWAPFILE_CLUSTER; tmp += 1; *cpu_next = tmp < max ? tmp : SWAP_NEXT_NULL; + return true; }