From patchwork Fri Oct 21 17:41:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Logan Gunthorpe X-Patchwork-Id: 6906 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4242:0:0:0:0:0 with SMTP id s2csp825892wrr; Fri, 21 Oct 2022 10:45:42 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6HOM2emAOB+Kk4TcQYutyeyiLFfa08D38R27/7X2tpgcg2RczDlDtRtb3y0TzlqkwmSUCW X-Received: by 2002:a05:6a00:23c5:b0:563:a0f0:e4d1 with SMTP id g5-20020a056a0023c500b00563a0f0e4d1mr20224918pfc.62.1666374341887; Fri, 21 Oct 2022 10:45:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666374341; cv=none; d=google.com; s=arc-20160816; b=nr+MVBRmSmrEUyLfLvx7d0f4sj7L2yV9V1SUXJX+mt95HD4idafwfq5z84N1e97+rN phrqjOZtGghcXVEMPYnfjGeIG0VEKmokIlGVWfGl4uXUrs7gIjTn+60cSi9mCGNm0aeC tu0MdyR5k8w0f3ovxIluCxjVFxsKs/wuiLQj6fZzVaO5/1f0Qlp0DCuaVfy0fCZme++5 NZJ11tugluonhaHlP6QB429Kgnh0zIhz52GEFMMpn6TYbyNVeFD1ErnqXDG8SlU3dL7D MAQBJSC117Ad8aggeMKhhFhmYo0Mi3gIM3Ba87K2uBg7fCtWYgfPSCJNxjcXi8il7YoW zhTg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:subject:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:cc:to:from:dkim-signature; bh=5j6xtapvwUdhGQNeqeFIKOu/eta4czgVaM1iHN6JS40=; b=ROkpQcD/YpFrgHQLhd2KpXTlv52m/TubJwm2z4KcHi7ShBqQIcXBHsEA2+4dmJQ3DM 2mjkdAERADt/y6EOp9qBcY45fvdSlxT9L3qmRB7xpxctbQihPcWzeZ9N5Mem9P37GM4e bUX49k6Y4UtDBJARb3aOaxHDdlNUnjGw88nGzPi/cDMPgPxxHGt6uDj9gwK+rh9zKdQs TLSOL6/qErQqnpCvr3oNy+TNG0ROILngwxCqfNw08DktKcZRALUXaxq0mZbnrDnJwUYC h/wr0N8tR1mqCdjYfkA8hdmNGRENvBtPo5GVu5URoSp0PFBB3PsQLsaTC5/GIbh/C6Xn LNsA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@deltatee.com header.s=20200525 header.b=SefADJT8; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=deltatee.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id m68-20020a625847000000b00562bb120d21si24661295pfb.165.2022.10.21.10.45.29; Fri, 21 Oct 2022 10:45:41 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@deltatee.com header.s=20200525 header.b=SefADJT8; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=deltatee.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230438AbiJURlr (ORCPT + 99 others); Fri, 21 Oct 2022 13:41:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59760 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229739AbiJURl3 (ORCPT ); Fri, 21 Oct 2022 13:41:29 -0400 Received: from ale.deltatee.com (ale.deltatee.com [204.191.154.188]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F3B4524AE1A; Fri, 21 Oct 2022 10:41:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=deltatee.com; s=20200525; h=Subject:MIME-Version:References:In-Reply-To: Message-Id:Date:Cc:To:From:content-disposition; bh=5j6xtapvwUdhGQNeqeFIKOu/eta4czgVaM1iHN6JS40=; b=SefADJT8EcX0hAmtPQ1R6W8xGe KAAj1gI1gcLhzb9ZUrK1mAndLYdIz1/sOp2f9PNdbV0eqKdhfQiD+7l8h7yu5nKN0bOAxU0gyriHT /WOxPRxOnDFNr+RUmaLIcAYIz4kJr8lVlpCEFBxT8tr10eGA4KGCwcjNjkziwHBPH4LrPgKoLauvr l3s1B+C/1okZjtAdHIea6v6X6JCjWmb9V/i4VzwPPd6BBPWz8iz6nrWwnpG31RbO2WDL3t8VoPyRK E4Nb1qteOxar2KTcF8H2OYO2Kkhn6/VMp9+qWQB4/GLfyo+CDFsdchrXsoxS19hYMoWs9hUvh/FMH kg1W/ilA==; Received: from cgy1-donard.priv.deltatee.com ([172.16.1.31]) by ale.deltatee.com with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1olw1L-00DoHy-8a; Fri, 21 Oct 2022 11:41:24 -0600 Received: from gunthorp by cgy1-donard.priv.deltatee.com with local (Exim 4.94.2) (envelope-from ) id 1olw1G-0001tI-1k; Fri, 21 Oct 2022 11:41:18 -0600 From: Logan Gunthorpe To: linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org, linux-pci@vger.kernel.org, linux-mm@kvack.org Cc: Christoph Hellwig , Greg Kroah-Hartman , Dan Williams , Jason Gunthorpe , =?utf-8?q?Christian_K=C3=B6nig?= , John Hubbard , Don Dutile , Matthew Wilcox , Daniel Vetter , Minturn Dave B , Jason Ekstrand , Dave Hansen , Xiong Jianxin , Bjorn Helgaas , Ira Weiny , Robin Murphy , Martin Oliveira , Chaitanya Kulkarni , Ralph Campbell , Stephen Bates , Logan Gunthorpe Date: Fri, 21 Oct 2022 11:41:12 -0600 Message-Id: <20221021174116.7200-6-logang@deltatee.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20221021174116.7200-1-logang@deltatee.com> References: <20221021174116.7200-1-logang@deltatee.com> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 172.16.1.31 X-SA-Exim-Rcpt-To: linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org, linux-block@vger.kernel.org, linux-pci@vger.kernel.org, linux-mm@kvack.org, hch@lst.de, gregkh@linuxfoundation.org, jgg@ziepe.ca, christian.koenig@amd.com, ddutile@redhat.com, willy@infradead.org, daniel.vetter@ffwll.ch, jason@jlekstrand.net, dave.hansen@linux.intel.com, helgaas@kernel.org, dan.j.williams@intel.com, dave.b.minturn@intel.com, jianxin.xiong@intel.com, ira.weiny@intel.com, robin.murphy@arm.com, martin.oliveira@eideticom.com, ckulkarnilinux@gmail.com, jhubbard@nvidia.com, rcampbell@nvidia.com, sbates@raithlin.com, logang@deltatee.com X-SA-Exim-Mail-From: gunthorp@deltatee.com X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_PASS,SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 Subject: [PATCH v11 5/9] lib/scatterlist: add check when merging zone device pages X-SA-Exim-Version: 4.2.1 (built Sat, 13 Feb 2021 17:57:42 +0000) X-SA-Exim-Scanned: Yes (on ale.deltatee.com) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747320141742450857?= X-GMAIL-MSGID: =?utf-8?q?1747320141742450857?= Consecutive zone device pages should not be merged into the same sgl or bvec segment with other types of pages or if they belong to different pgmaps. Otherwise getting the pgmap of a given segment is not possible without scanning the entire segment. This helper returns true either if both pages are not zone device pages or both pages are zone device pages with the same pgmap. Factor out the check for page mergability into a pages_are_mergable() helper and add a check with zone_device_pages_are_mergeable(). Signed-off-by: Logan Gunthorpe Reviewed-by: Christoph Hellwig Reviewed-by: Chaitanya Kulkarni --- lib/scatterlist.c | 25 +++++++++++++++---------- 1 file changed, 15 insertions(+), 10 deletions(-) diff --git a/lib/scatterlist.c b/lib/scatterlist.c index c8c3d675845c..a0ad2a7959b5 100644 --- a/lib/scatterlist.c +++ b/lib/scatterlist.c @@ -410,6 +410,15 @@ static struct scatterlist *get_next_sg(struct sg_append_table *table, return new_sg; } +static bool pages_are_mergeable(struct page *a, struct page *b) +{ + if (page_to_pfn(a) != page_to_pfn(b) + 1) + return false; + if (!zone_device_pages_have_same_pgmap(a, b)) + return false; + return true; +} + /** * sg_alloc_append_table_from_pages - Allocate and initialize an append sg * table from an array of pages @@ -447,6 +456,7 @@ int sg_alloc_append_table_from_pages(struct sg_append_table *sgt_append, unsigned int chunks, cur_page, seg_len, i, prv_len = 0; unsigned int added_nents = 0; struct scatterlist *s = sgt_append->prv; + struct page *last_pg; /* * The algorithm below requires max_segment to be aligned to PAGE_SIZE @@ -460,21 +470,17 @@ int sg_alloc_append_table_from_pages(struct sg_append_table *sgt_append, return -EOPNOTSUPP; if (sgt_append->prv) { - unsigned long paddr = - (page_to_pfn(sg_page(sgt_append->prv)) * PAGE_SIZE + - sgt_append->prv->offset + sgt_append->prv->length) / - PAGE_SIZE; - if (WARN_ON(offset)) return -EINVAL; /* Merge contiguous pages into the last SG */ prv_len = sgt_append->prv->length; - while (n_pages && page_to_pfn(pages[0]) == paddr) { + last_pg = sg_page(sgt_append->prv); + while (n_pages && pages_are_mergeable(last_pg, pages[0])) { if (sgt_append->prv->length + PAGE_SIZE > max_segment) break; sgt_append->prv->length += PAGE_SIZE; - paddr++; + last_pg = pages[0]; pages++; n_pages--; } @@ -488,7 +494,7 @@ int sg_alloc_append_table_from_pages(struct sg_append_table *sgt_append, for (i = 1; i < n_pages; i++) { seg_len += PAGE_SIZE; if (seg_len >= max_segment || - page_to_pfn(pages[i]) != page_to_pfn(pages[i - 1]) + 1) { + !pages_are_mergeable(pages[i], pages[i - 1])) { chunks++; seg_len = 0; } @@ -504,8 +510,7 @@ int sg_alloc_append_table_from_pages(struct sg_append_table *sgt_append, for (j = cur_page + 1; j < n_pages; j++) { seg_len += PAGE_SIZE; if (seg_len >= max_segment || - page_to_pfn(pages[j]) != - page_to_pfn(pages[j - 1]) + 1) + !pages_are_mergeable(pages[j], pages[j - 1])) break; }