From patchwork Fri Dec 16 20:16:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keith Busch X-Patchwork-Id: 34074 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:e747:0:0:0:0:0 with SMTP id c7csp1176281wrn; Fri, 16 Dec 2022 12:18:15 -0800 (PST) X-Google-Smtp-Source: AA0mqf58nlP0apwL9SUVfQHuMRGOndoYu7ZXSlWnJuQUKeXaZQ5Y/0tSpFd8KSfD4ptaV5tGBkqp X-Received: by 2002:a17:902:f64b:b0:186:de2c:97e3 with SMTP id m11-20020a170902f64b00b00186de2c97e3mr38669199plg.54.1671221895085; Fri, 16 Dec 2022 12:18:15 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1671221895; cv=none; d=google.com; s=arc-20160816; b=B6C8UNEFwMYo6gjjV8oJ1BWq6hrHyay0+jPUDl4CQULl7U7qCWqnH8DUsKJTDfIg7N zJcQ8KXeOOe2gPhfh7H02lRpa9ET27Jqcv3mKUihnrTxfPwoChhQhY/+Rldx7DYenmu5 mgZfkF2N7PeHim9XQQAo5T+EL1fKuDkPKyni1NCMhnmN46stpwLioZyFuBfY3xpMRs02 XbBURynM9N1y40+EG18x/WObYX224s22nhBecRwv5oZUtxLYCCqKX07iLixGByKblD2+ 0ktVlEfDRa15gfniC3Z3KBO8F7z1zCzhl2xajDYLKTTpwe/W6Ddat9lNBDK9jsfE0EcZ YXQQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:content-transfer-encoding :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=90Dk+65o6zL/IJhusSfca6Gtj59oaXmZv+zwCh9VWSM=; b=CxWfHzS/MY92ZLTihV92x+vCPWXDh7maW8lDx/cCmVetc1BNlISZvwBrDMX4T0OFyV JskrdwQ3DV8COOxeDfQXzR5WyDKP2YrETbOvNXRcihl8MuMl9483AuF+Y8Wo/x6qcMyR 07JhArw0yJfHCV8l0z8o/43fdg0g31F1Acb/9krzMknGAYa2IAV6jTjRJSa6vXdqCSSP tXG+l4IPiS4gvCBYJaUYjC5wL4T6LgF8hL4SxSU7n5GF1jrMcJA5TSazBJuAsfhWZksO AzltIsvnsrBz0H1Uc+vK62YRNm5EKC1W2rhCQYR4DPTxrBTGhDC7RAu03BDUpIMKJEbg Za5g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@meta.com header.s=s2048-2021-q4 header.b=jy9Xam41; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=meta.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id t10-20020a170902e84a00b00188e5c9e026si3679257plg.102.2022.12.16.12.18.02; Fri, 16 Dec 2022 12:18:15 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@meta.com header.s=s2048-2021-q4 header.b=jy9Xam41; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=meta.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231962AbiLPURV (ORCPT + 99 others); Fri, 16 Dec 2022 15:17:21 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58856 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231761AbiLPURR (ORCPT ); Fri, 16 Dec 2022 15:17:17 -0500 Received: from mx0b-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4234D6A768 for ; Fri, 16 Dec 2022 12:17:16 -0800 (PST) Received: from pps.filterd (m0109332.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2BGJx9Ra012075 for ; Fri, 16 Dec 2022 12:17:15 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=meta.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : content-transfer-encoding : mime-version; s=s2048-2021-q4; bh=90Dk+65o6zL/IJhusSfca6Gtj59oaXmZv+zwCh9VWSM=; b=jy9Xam41TnuZ7I7z0bKovbU95FicKE3dZCRm44FQ6DCEVup9aF4QpXQRvrl9nocbv1sg V/0dCd8DujYodoVlOGFOvuuBMab2QbNNAAgl3A3FyJK64qVtovK731gEy1OmI9zs6/1D W8CBRkE+GrQ+FUL0cXxMS7k8792Lp17sR0TiG3JOMmf7CSE1W6R0T0SmQ+fI2SuCth8X /gQroU/8mIo5P85+/B4nulgscA9X+y2p0LN+sNz5Zf3hsRkGwvRE6kCemUdgVATjHR0W XvuhS3GmCrRehaK/7aj8kaqM3nhWleUrJQntsUefC5MDMZ/p/yDK+zijL5VsmfUaCbdl Yg== Received: from mail.thefacebook.com ([163.114.132.120]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3mgm45uyfn-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Fri, 16 Dec 2022 12:17:15 -0800 Received: from twshared15216.17.frc2.facebook.com (2620:10d:c085:108::8) by mail.thefacebook.com (2620:10d:c085:11d::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Fri, 16 Dec 2022 12:17:13 -0800 Received: by devbig007.nao1.facebook.com (Postfix, from userid 544533) id E5D62D042AB1; Fri, 16 Dec 2022 12:17:09 -0800 (PST) From: Keith Busch To: , , Matthew Wilcox CC: Tony Battersby , Kernel Team , Keith Busch Subject: [PATCHv2 01/11] dmapool: add alloc/free performance test Date: Fri, 16 Dec 2022 12:16:15 -0800 Message-ID: <20221216201625.2362737-2-kbusch@meta.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20221216201625.2362737-1-kbusch@meta.com> References: <20221216201625.2362737-1-kbusch@meta.com> X-FB-Internal: Safe X-Proofpoint-ORIG-GUID: DmbyTOlhU_y47KGbqHdVDZ1PA_OVpTk7 X-Proofpoint-GUID: DmbyTOlhU_y47KGbqHdVDZ1PA_OVpTk7 X-Proofpoint-UnRewURL: 0 URL was un-rewritten MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-16_14,2022-12-15_02,2022-06-22_01 X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1752403169814972810?= X-GMAIL-MSGID: =?utf-8?q?1752403169814972810?= From: Keith Busch Provide a module that allocates and frees many blocks of various sizes and report how long it takes. This is intended to provide a consistent way to measure how changes to the dma_pool_alloc/free routines affect timing. Signed-off-by: Keith Busch --- v1->v2: Used appropriate dma_ops accessor, fixing: https://lore.kernel.org/all/202212061052.hMChRFTf-lkp@intel.com/ Added ability to test arbitrary alignments and boundaries and added an entry to test the previously failing test case: https://lore.kernel.org/oe-lkp/202212112301.ad0819f7-oliver.sang@intel.com/ mm/Kconfig | 9 +++ mm/Makefile | 1 + mm/dmapool_test.c | 146 ++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 156 insertions(+) create mode 100644 mm/dmapool_test.c diff --git a/mm/Kconfig b/mm/Kconfig index 34d36958b8ac9..3d71acc29c10f 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -1070,6 +1070,15 @@ config GUP_TEST comment "GUP_TEST needs to have DEBUG_FS enabled" depends on !GUP_TEST && !DEBUG_FS +config DMAPOOL_TEST + tristate "Enable a module to run time tests on dma_pool" + depends on HAS_DMA + help + Provides a module that will allocate and free many blocks of various + sizes and report how long it takes. This is intended to provide a + consistent way to measure how changes to the dma_pool_alloc/free + routines affect performance. + config GUP_GET_PTE_LOW_HIGH bool diff --git a/mm/Makefile b/mm/Makefile index 8e105e5b3e293..3a08f5d7b1782 100644 --- a/mm/Makefile +++ b/mm/Makefile @@ -103,6 +103,7 @@ obj-$(CONFIG_MEMCG) += swap_cgroup.o endif obj-$(CONFIG_CGROUP_HUGETLB) += hugetlb_cgroup.o obj-$(CONFIG_GUP_TEST) += gup_test.o +obj-$(CONFIG_DMAPOOL_TEST) += dmapool_test.o obj-$(CONFIG_MEMORY_FAILURE) += memory-failure.o obj-$(CONFIG_HWPOISON_INJECT) += hwpoison-inject.o obj-$(CONFIG_DEBUG_KMEMLEAK) += kmemleak.o diff --git a/mm/dmapool_test.c b/mm/dmapool_test.c new file mode 100644 index 0000000000000..c4292dfda3690 --- /dev/null +++ b/mm/dmapool_test.c @@ -0,0 +1,146 @@ +#include +#include +#include +#include +#include +#include +#include + +#define NR_TESTS (100) + +struct dma_pool_pair { + dma_addr_t dma; + void *v; +}; + +struct dmapool_parms { + size_t size; + size_t align; + size_t boundary; +}; + +static const struct dmapool_parms pool_parms[] = { + { .size = 16, .align = 16, .boundary = 0 }, + { .size = 64, .align = 64, .boundary = 0 }, + { .size = 256, .align = 256, .boundary = 0 }, + { .size = 1024, .align = 1024, .boundary = 0 }, + { .size = 4096, .align = 4096, .boundary = 0 }, + { .size = 68, .align = 32, .boundary = 4096 }, +}; + +static struct dma_pool *pool; +static struct device test_dev; +static u64 dma_mask; + +static inline int nr_blocks(int size) +{ + return clamp_t(int, (PAGE_SIZE / size) * 512, 1024, 8192); +} + +static int dmapool_test_alloc(struct dma_pool_pair *p, int blocks) +{ + int i; + + for (i = 0; i < blocks; i++) { + p[i].v = dma_pool_alloc(pool, GFP_KERNEL, + &p[i].dma); + if (!p[i].v) + goto pool_fail; + } + + for (i = 0; i < blocks; i++) + dma_pool_free(pool, p[i].v, p[i].dma); + + return 0; + +pool_fail: + for (--i; i >= 0; i--) + dma_pool_free(pool, p[i].v, p[i].dma); + return -ENOMEM; +} + +static int dmapool_test_block(const struct dmapool_parms *parms) +{ + int blocks = nr_blocks(parms->size); + ktime_t start_time, end_time; + struct dma_pool_pair *p; + int i, ret; + + p = kcalloc(blocks, sizeof(*p), GFP_KERNEL); + if (!p) + return -ENOMEM; + + pool = dma_pool_create("test pool", &test_dev, parms->size, + parms->align, parms->boundary); + if (!pool) { + ret = -ENOMEM; + goto free_pairs; + } + + start_time = ktime_get(); + for (i = 0; i < NR_TESTS; i++) { + ret = dmapool_test_alloc(p, blocks); + if (ret) + goto free_pool; + if (need_resched()) + cond_resched(); + } + end_time = ktime_get(); + + printk("dmapool test: size:%-4ld blocks:%-6d time:%llu\n", parms->size, + blocks, ktime_us_delta(end_time, start_time)); + +free_pool: + dma_pool_destroy(pool); +free_pairs: + kfree(p); + return ret; +} + +static void dmapool_test_release(struct device *dev) +{ +} + +static int dmapool_checks(void) +{ + int i, ret; + + ret = dev_set_name(&test_dev, "dmapool-test"); + if (ret) + return ret; + + ret = device_register(&test_dev); + if (ret) { + printk("%s: register failed:%d\n", __func__, ret); + goto put_device; + } + + test_dev.release = dmapool_test_release; + set_dma_ops(&test_dev, NULL); + test_dev.dma_mask = &dma_mask; + ret = dma_set_mask_and_coherent(&test_dev, DMA_BIT_MASK(64)); + if (ret) { + printk("%s: mask failed:%d\n", __func__, ret); + goto del_device; + } + + for (i = 0; i < ARRAY_SIZE(pool_parms); i++) { + ret = dmapool_test_block(&pool_parms[i]); + if (ret) + break; + } + +del_device: + device_del(&test_dev); +put_device: + put_device(&test_dev); + return ret; +} + +static void dmapool_exit(void) +{ +} + +module_init(dmapool_checks); +module_exit(dmapool_exit); +MODULE_LICENSE("GPL"); From patchwork Fri Dec 16 20:16:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keith Busch X-Patchwork-Id: 34072 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:e747:0:0:0:0:0 with SMTP id c7csp1176191wrn; Fri, 16 Dec 2022 12:18:01 -0800 (PST) X-Google-Smtp-Source: AA0mqf7Iv4WzjEwwQnUHmgqjTmdaX/zFSWKLm/zGohQaDlAJrOh2xeXMuCPYyCcDdmX3Q67SZJcR X-Received: by 2002:a17:902:c745:b0:189:9519:87b6 with SMTP id q5-20020a170902c74500b00189951987b6mr28367581plq.5.1671221881293; Fri, 16 Dec 2022 12:18:01 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1671221881; cv=none; d=google.com; s=arc-20160816; b=EeXVYJq0GjEQCxKJoQaG7cFYgzQJORNahho2b708eoxTcO2AlMxqkqcAXzkRS9nik7 1TI53UgMFOE3nMQWV6CONx72jmYxGijZMII/QQCPslZQgeNAWfVsw2/BLZo8ow0+Tn1Y Lbbhzer20nv96sHEDl+J9LHc19UF6jziimBtoqbxK8U2FtMWSrv7jPrWOd0rjdAExm+q zwXOmxZEUHfV5S7kc3UZ2xRhzT+8o4WnyRg+Pi+81YghjLVcuz7cM8vWI8KXIE5qcSob PYiziLqaRLSKq8P3M9zUgfA4LepjrkzbF5ilKIgeh+cSzobtPGRKEq5YXhRS/K3cnXFA 6QXA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=YU1RvihRnvDJ0EI7aPZRW/ndJTmtKABPXXQKd2XE/QM=; b=GhCEsNT8NqG0esFaqCEMyVeojUEeeTrZYNgTS5ow68q944Ka1a/ESDvuT8EaynRZze 5pa3xv1d9iil3JBELcvMtFQyEOUrwyQ3RPqjA6xRSjvX9cy1TNaB6x1XswImZBZX1SzG uUQphq3aiSFQdBbSF4Bkgq36EioJx1MhrCcZMuixsKUjDqj6VntrpD59qmjUBqWblOg4 iM5AAiTwjqfcBhOshgQ4JhT7XN855CWk2vfkH3rN+VTbDlVyi1clYMbSbNZIv7dVdqHN AjfR2cjAzWJlxFA1NJ+Rrf8koikC5baaDjim9iYByjWRm97SXBLNZcJ3xqR8LfEirqEu vZcQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@meta.com header.s=s2048-2021-q4 header.b=EOJULLWU; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=meta.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id n13-20020a6543cd000000b0041aaa841692si3763115pgp.108.2022.12.16.12.17.48; Fri, 16 Dec 2022 12:18:01 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@meta.com header.s=s2048-2021-q4 header.b=EOJULLWU; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=meta.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231782AbiLPUR0 (ORCPT + 99 others); Fri, 16 Dec 2022 15:17:26 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58868 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231961AbiLPURU (ORCPT ); Fri, 16 Dec 2022 15:17:20 -0500 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0326469A82 for ; Fri, 16 Dec 2022 12:17:20 -0800 (PST) Received: from pps.filterd (m0109333.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2BGJxOJF003511 for ; Fri, 16 Dec 2022 12:17:19 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=meta.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=s2048-2021-q4; bh=YU1RvihRnvDJ0EI7aPZRW/ndJTmtKABPXXQKd2XE/QM=; b=EOJULLWU5UiM1q04JbUAbiQd0PUKnll0UXbftHQzzxz7yZK3DjguaSKyqwL1JRpxtcFY wAx1DBpLWKk8quCYHZmll68+0uhWKQ8zorCor1mfyWh8+uQxvsPPKJ6cGtCLCJPWzS3Z XvsJIwOqeI78+CmYSJW5MwCtQEh+k4tqReLf8DhtOhAgCSlZrRxsCnNLVtQ/+o+mEpk/ +WbH97ifLShydcbG9gNi6mRAjulATneAj02jCdi98MZEeJt0CPGhbpcMMfhfOkBH53C3 qqn3mVOkSEFvFmogvHyvGN8xckvj1rDccte/q0tesXw7MBX/+d21ExQNAgu6S38gEzm0 Ng== Received: from mail.thefacebook.com ([163.114.132.120]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3mgdt16cym-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Fri, 16 Dec 2022 12:17:19 -0800 Received: from snc-exhub201.TheFacebook.com (2620:10d:c085:21d::7) by snc-exhub101.TheFacebook.com (2620:10d:c085:11d::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Fri, 16 Dec 2022 12:17:18 -0800 Received: from twshared21680.02.ash8.facebook.com (2620:10d:c085:108::8) by mail.thefacebook.com (2620:10d:c085:21d::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Fri, 16 Dec 2022 12:17:18 -0800 Received: by devbig007.nao1.facebook.com (Postfix, from userid 544533) id F41E3D042AB3; Fri, 16 Dec 2022 12:17:09 -0800 (PST) From: Keith Busch To: , , Matthew Wilcox CC: Tony Battersby , Kernel Team , Keith Busch Subject: [PATCHv2 02/11] dmapool: remove checks for dev == NULL Date: Fri, 16 Dec 2022 12:16:16 -0800 Message-ID: <20221216201625.2362737-3-kbusch@meta.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20221216201625.2362737-1-kbusch@meta.com> References: <20221216201625.2362737-1-kbusch@meta.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: oNdb3mq6zzu9N9N2iXltLETGh62m1Amd X-Proofpoint-ORIG-GUID: oNdb3mq6zzu9N9N2iXltLETGh62m1Amd X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-16_14,2022-12-15_02,2022-06-22_01 X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1752403154964843509?= X-GMAIL-MSGID: =?utf-8?q?1752403154964843509?= From: Tony Battersby dmapool originally tried to support pools without a device because dma_alloc_coherent() supports allocations without a device. But nobody ended up using dma pools without a device, and trying to do so will result in an oops. So remove the checks for pool->dev == NULL since they are unneeded bloat. Signed-off-by: Tony Battersby [added check for null dev on create] Signed-off-by: Keith Busch Reviewed-by: Christoph Hellwig --- mm/dmapool.c | 45 ++++++++++++++------------------------------- 1 file changed, 14 insertions(+), 31 deletions(-) diff --git a/mm/dmapool.c b/mm/dmapool.c index a7eb5d0eb2da7..559207e1c3339 100644 --- a/mm/dmapool.c +++ b/mm/dmapool.c @@ -134,6 +134,9 @@ struct dma_pool *dma_pool_create(const char *name, struct device *dev, size_t allocation; bool empty = false; + if (!dev) + return NULL; + if (align == 0) align = 1; else if (align & (align - 1)) @@ -275,7 +278,7 @@ void dma_pool_destroy(struct dma_pool *pool) mutex_lock(&pools_reg_lock); mutex_lock(&pools_lock); list_del(&pool->pools); - if (pool->dev && list_empty(&pool->dev->dma_pools)) + if (list_empty(&pool->dev->dma_pools)) empty = true; mutex_unlock(&pools_lock); if (empty) @@ -284,12 +287,8 @@ void dma_pool_destroy(struct dma_pool *pool) list_for_each_entry_safe(page, tmp, &pool->page_list, page_list) { if (is_page_busy(page)) { - if (pool->dev) - dev_err(pool->dev, "%s %s, %p busy\n", __func__, - pool->name, page->vaddr); - else - pr_err("%s %s, %p busy\n", __func__, - pool->name, page->vaddr); + dev_err(pool->dev, "%s %s, %p busy\n", __func__, + pool->name, page->vaddr); /* leak the still-in-use consistent memory */ list_del(&page->page_list); kfree(page); @@ -351,12 +350,8 @@ void *dma_pool_alloc(struct dma_pool *pool, gfp_t mem_flags, for (i = sizeof(page->offset); i < pool->size; i++) { if (data[i] == POOL_POISON_FREED) continue; - if (pool->dev) - dev_err(pool->dev, "%s %s, %p (corrupted)\n", - __func__, pool->name, retval); - else - pr_err("%s %s, %p (corrupted)\n", - __func__, pool->name, retval); + dev_err(pool->dev, "%s %s, %p (corrupted)\n", + __func__, pool->name, retval); /* * Dump the first 4 bytes even if they are not @@ -411,12 +406,8 @@ void dma_pool_free(struct dma_pool *pool, void *vaddr, dma_addr_t dma) page = pool_find_page(pool, dma); if (!page) { spin_unlock_irqrestore(&pool->lock, flags); - if (pool->dev) - dev_err(pool->dev, "%s %s, %p/%pad (bad dma)\n", - __func__, pool->name, vaddr, &dma); - else - pr_err("%s %s, %p/%pad (bad dma)\n", - __func__, pool->name, vaddr, &dma); + dev_err(pool->dev, "%s %s, %p/%pad (bad dma)\n", + __func__, pool->name, vaddr, &dma); return; } @@ -426,12 +417,8 @@ void dma_pool_free(struct dma_pool *pool, void *vaddr, dma_addr_t dma) #ifdef DMAPOOL_DEBUG if ((dma - page->dma) != offset) { spin_unlock_irqrestore(&pool->lock, flags); - if (pool->dev) - dev_err(pool->dev, "%s %s, %p (bad vaddr)/%pad\n", - __func__, pool->name, vaddr, &dma); - else - pr_err("%s %s, %p (bad vaddr)/%pad\n", - __func__, pool->name, vaddr, &dma); + dev_err(pool->dev, "%s %s, %p (bad vaddr)/%pad\n", + __func__, pool->name, vaddr, &dma); return; } { @@ -442,12 +429,8 @@ void dma_pool_free(struct dma_pool *pool, void *vaddr, dma_addr_t dma) continue; } spin_unlock_irqrestore(&pool->lock, flags); - if (pool->dev) - dev_err(pool->dev, "%s %s, dma %pad already free\n", - __func__, pool->name, &dma); - else - pr_err("%s %s, dma %pad already free\n", - __func__, pool->name, &dma); + dev_err(pool->dev, "%s %s, dma %pad already free\n", + __func__, pool->name, &dma); return; } } From patchwork Fri Dec 16 20:16:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keith Busch X-Patchwork-Id: 34082 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:e747:0:0:0:0:0 with SMTP id c7csp1181738wrn; Fri, 16 Dec 2022 12:31:17 -0800 (PST) X-Google-Smtp-Source: AA0mqf7kA4KKhQjgk1JjYAXhmtkO4fML6pN2Job+DyP7UarMXqKG0s3MD2ARrZG7JZxbhtVyomId X-Received: by 2002:a50:fe88:0:b0:467:5e9b:7df1 with SMTP id d8-20020a50fe88000000b004675e9b7df1mr27961400edt.24.1671222677540; Fri, 16 Dec 2022 12:31:17 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1671222677; cv=none; d=google.com; s=arc-20160816; b=PGrVh/hQwqQUE0al5AFxRnWR+nd0eVtzsm0SfZ7rdmC0Czg27bOt2ihxKS4fJLyqqC AzuxhRa7SzW+adOHXYQozn/sjz3koiBSdS85DWEb4cw2coa76U72AgCnt1P4Ahhim2Pz /KlwYwon4BukrnPjLiYYOarDFD+kCPfORe0FFJOjH/+RKpNJ9KMMxhscChJgTlc+NE2j e2eyrqNoHDX9ztPEw507agOrcOXfM0Hdvr/EBxZ2Qg9Z1aVGT+XdL1zykZ/tHQEkEwmA 4RMO/L87NtO2qfsTohB8aWqJ/LjKjh5lW1ORBaYhJ5LcKRz96WisI24PxS9nB979oITv yOdw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=MLWHDUS20C0jmHJ39hBO1H02jIUsWm5bZEXT2SOrcPk=; b=qpd9MQ7KaUmuqfItHx8snRyqZgp1gVeP815fqNrNruuWvoQ/Xc8v8JrBGOmxGd1zWE AGNmSu91TJXZ3i5IphHD3zkXN7p/NV6P0Vuf1qf+gi8uV9r5H30CqZBIeZF0TEMWlcpp LEE80gzCFepkmUdI0B6F3jLFyZoxX3cevGfY8exszfJ1SOpVEEWl2xmsBmjHYDnSxjKS MR5LNwWoDi+8D30Rk3JWH97yLUsjhL3XkF1XM017EZMzrYn6yWDb2k5JFCrrskf+SJUO GEnHLft7UI7boLzTv6M21qF8Zz9AY4062JVMHTjNFcTj0B8WSb9DnO4a0BVu7pvGcfdH XMIg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@meta.com header.s=s2048-2021-q4 header.b=mfMn7jRA; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=meta.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id mm8-20020a170906cc4800b007c15ca85481si2645383ejb.812.2022.12.16.12.30.51; Fri, 16 Dec 2022 12:31:17 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@meta.com header.s=s2048-2021-q4 header.b=mfMn7jRA; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=meta.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232071AbiLPUSS (ORCPT + 99 others); Fri, 16 Dec 2022 15:18:18 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59550 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232021AbiLPURs (ORCPT ); Fri, 16 Dec 2022 15:17:48 -0500 Received: from mx0a-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1921872601 for ; Fri, 16 Dec 2022 12:17:33 -0800 (PST) Received: from pps.filterd (m0001303.ppops.net [127.0.0.1]) by m0001303.ppops.net (8.17.1.19/8.17.1.19) with ESMTP id 2BGJxVB0024629 for ; Fri, 16 Dec 2022 12:17:33 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=meta.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=s2048-2021-q4; bh=MLWHDUS20C0jmHJ39hBO1H02jIUsWm5bZEXT2SOrcPk=; b=mfMn7jRAj1JALvXAGMrFFL2ynhUTkET2GZUP//Au3O898Gj3HgP6C83+obVffTaI9AEC LdH93rl62N9EjRD5+os0ye+85TMSG1HNxCqKKwVjcnoo9AYzjB9WKpmso7prETZ5tjzc 3HRP9xgLaR+p0coaQsxcCgS66JxPo99d6u33bkLP+D12zFt3zYgcMC02JjfrlJuH8gXo LYvJCI7ZTuPhY8i7TzUUu5iEU6bFUrUlcg/HAud+/dv1BAINn8D3AdXdX6uIibAi8w1U ySKVjeRciobFpyq04tF0c/gbUyK/rd7Wx3oq7ad63OgUBOIRgxK+2AB1dxOA/1eD9v5e Jg== Received: from mail.thefacebook.com ([163.114.132.120]) by m0001303.ppops.net (PPS) with ESMTPS id 3mg3hn2eff-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Fri, 16 Dec 2022 12:17:32 -0800 Received: from twshared15216.17.frc2.facebook.com (2620:10d:c085:208::f) by mail.thefacebook.com (2620:10d:c085:11d::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Fri, 16 Dec 2022 12:17:24 -0800 Received: by devbig007.nao1.facebook.com (Postfix, from userid 544533) id 143E1D042AB5; Fri, 16 Dec 2022 12:17:10 -0800 (PST) From: Keith Busch To: , , Matthew Wilcox CC: Tony Battersby , Kernel Team , Keith Busch Subject: [PATCHv2 03/11] dmapool: use sysfs_emit() instead of scnprintf() Date: Fri, 16 Dec 2022 12:16:17 -0800 Message-ID: <20221216201625.2362737-4-kbusch@meta.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20221216201625.2362737-1-kbusch@meta.com> References: <20221216201625.2362737-1-kbusch@meta.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: qfc2w-Px_3l_LSo_6fiQmORoywmOr1Ul X-Proofpoint-ORIG-GUID: qfc2w-Px_3l_LSo_6fiQmORoywmOr1Ul X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-16_14,2022-12-15_02,2022-06-22_01 X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1752403990667414447?= X-GMAIL-MSGID: =?utf-8?q?1752403990667414447?= From: Tony Battersby Use sysfs_emit instead of scnprintf, snprintf or sprintf. Signed-off-by: Tony Battersby Signed-off-by: Keith Busch Reviewed-by: Christoph Hellwig --- mm/dmapool.c | 23 +++++++---------------- 1 file changed, 7 insertions(+), 16 deletions(-) diff --git a/mm/dmapool.c b/mm/dmapool.c index 559207e1c3339..20616b760bb9c 100644 --- a/mm/dmapool.c +++ b/mm/dmapool.c @@ -64,18 +64,11 @@ static DEFINE_MUTEX(pools_reg_lock); static ssize_t pools_show(struct device *dev, struct device_attribute *attr, char *buf) { - unsigned temp; - unsigned size; - char *next; + int size; struct dma_page *page; struct dma_pool *pool; - next = buf; - size = PAGE_SIZE; - - temp = scnprintf(next, size, "poolinfo - 0.1\n"); - size -= temp; - next += temp; + size = sysfs_emit(buf, "poolinfo - 0.1\n"); mutex_lock(&pools_lock); list_for_each_entry(pool, &dev->dma_pools, pools) { @@ -90,16 +83,14 @@ static ssize_t pools_show(struct device *dev, struct device_attribute *attr, cha spin_unlock_irq(&pool->lock); /* per-pool info, no real statistics yet */ - temp = scnprintf(next, size, "%-16s %4u %4zu %4zu %2u\n", - pool->name, blocks, - pages * (pool->allocation / pool->size), - pool->size, pages); - size -= temp; - next += temp; + size += sysfs_emit_at(buf, size, "%-16s %4u %4zu %4zu %2u\n", + pool->name, blocks, + pages * (pool->allocation / pool->size), + pool->size, pages); } mutex_unlock(&pools_lock); - return PAGE_SIZE - size; + return size; } static DEVICE_ATTR_RO(pools); From patchwork Fri Dec 16 20:16:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keith Busch X-Patchwork-Id: 34073 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:e747:0:0:0:0:0 with SMTP id c7csp1176216wrn; Fri, 16 Dec 2022 12:18:06 -0800 (PST) X-Google-Smtp-Source: AA0mqf5ONkrlQgqcXMTyyR51O9F5hF8AsiMJ0lCv9/nlPIe9xMc2u8AGWpMLHQCcNhw48QZFyp6q X-Received: by 2002:a17:902:c183:b0:189:854e:93a3 with SMTP id d3-20020a170902c18300b00189854e93a3mr33169683pld.69.1671221886250; Fri, 16 Dec 2022 12:18:06 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1671221886; cv=none; d=google.com; s=arc-20160816; b=xGVxBMrUxK89VufWEsev948fidASnNd+KPsCnuf4dQDduvXQV6tsYYqkrnjiWnZpek 0OSPYqC4KbP2Fl9cOUdpeE0lIplYUnZhF4nNAeLJ+GXgNiDW/cROGuXxYLU1d+PsF4bQ Q6tPLp2vpFCmuFFWulNO7Kvd02fRAcxhQLO17mEDJWS31GqjZtsWkiVwIYIboOQyjADD inqmUAh8XinmjsoXjSkK/ZgFOBadpku4RCnu7rl55KyCW5QmVH2/JHo8XpqdW8in2FUQ HBESLZAnF5tU1XNHhBRJrLUQ9SGErGJtIA2QyrrH1Qzhaw0Zobtwsz79tV3aqxYc79W7 ii3Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=TsEKLROJI6bk+d/af34BtH3nHi+XT+AwCEfmZNUH3/Y=; b=SoZ/VQSUcyZ7UF2H0RGayisXFC3y2wT01Vrr9I/8Wj1w98dtlGoTpBswT1V8w0HgLX QG9/p31DhL7bMaVG7lyPe/tt0Ix345qUk30Bkry6332VCoPZlFcg8EO7pEIeE2mFNVRN v5EM1ZMLkLsXDXb16690o8ymNlY2CD1QgRy9DdsLkv/tG/zfqUkHgYOt+Q1bwhy+YlmG /mByacv7VpUA6wiv3OehpNXBYMdts+KtfMw3hxn7xfGg6YEkcZn5/R2QdchrmHJJb//Q aetYZnRWyKIBeLDVFbUxk1GlWgCg5IzRCFzSYKwuCkNg188f8okLMH2cgUT1uzxQDXaU NeQg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@meta.com header.s=s2048-2021-q4 header.b="eg/nESr6"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=meta.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 12-20020a170902ee4c00b00188763b35e1si3166459plo.424.2022.12.16.12.17.53; Fri, 16 Dec 2022 12:18:06 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@meta.com header.s=s2048-2021-q4 header.b="eg/nESr6"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=meta.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231975AbiLPURc (ORCPT + 99 others); Fri, 16 Dec 2022 15:17:32 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58882 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231963AbiLPURV (ORCPT ); Fri, 16 Dec 2022 15:17:21 -0500 Received: from mx0b-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CC6C56A741 for ; Fri, 16 Dec 2022 12:17:20 -0800 (PST) Received: from pps.filterd (m0109332.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2BGJx5ZI011799 for ; Fri, 16 Dec 2022 12:17:20 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=meta.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=s2048-2021-q4; bh=TsEKLROJI6bk+d/af34BtH3nHi+XT+AwCEfmZNUH3/Y=; b=eg/nESr6F77JzUbkWU0e8iE2X8/jNwrl9zRoWRDdZr9uOfV0IDl/ujcTuVTc15OXPMls qU20SlqzotONNGPF/XwPTBIXdMUmsF5JBreS/shSEouA+j7ohJczc3i9drTwXVqyWhF7 uzJGYjx4yBnaQLsTh/JEY3tCWGUdCWeiYeFclsfq/fjyvWuuXKDj+vPrgdxzm/RDEpzQ +DugdrXadzXsbN1rvNvYRc3gq07R5VlZ8ZMEfKFAi72eBjaPi13EDukrGm2vBM4HGWsT 0QBbcoQbnNkyhujIGVmQcO9VshkHIKrdtE0bV0512vMynv7F3sHtHUP0EpRQLVjo7UX3 LA== Received: from mail.thefacebook.com ([163.114.132.120]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3mgm45uyg4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Fri, 16 Dec 2022 12:17:19 -0800 Received: from twshared24004.14.frc2.facebook.com (2620:10d:c085:208::11) by mail.thefacebook.com (2620:10d:c085:11d::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Fri, 16 Dec 2022 12:17:18 -0800 Received: by devbig007.nao1.facebook.com (Postfix, from userid 544533) id 1A5B2D042AB7; Fri, 16 Dec 2022 12:17:10 -0800 (PST) From: Keith Busch To: , , Matthew Wilcox CC: Tony Battersby , Kernel Team , Keith Busch Subject: [PATCHv2 04/11] dmapool: cleanup integer types Date: Fri, 16 Dec 2022 12:16:18 -0800 Message-ID: <20221216201625.2362737-5-kbusch@meta.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20221216201625.2362737-1-kbusch@meta.com> References: <20221216201625.2362737-1-kbusch@meta.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-ORIG-GUID: vhSY8vOHsHmUmbV2-dPGRoPkjZHkTU1j X-Proofpoint-GUID: vhSY8vOHsHmUmbV2-dPGRoPkjZHkTU1j X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-16_14,2022-12-15_02,2022-06-22_01 X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1752403160629282108?= X-GMAIL-MSGID: =?utf-8?q?1752403160629282108?= From: Tony Battersby To represent the size of a single allocation, dmapool currently uses 'unsigned int' in some places and 'size_t' in other places. Standardize on 'unsigned int' to reduce overhead, but use 'size_t' when counting all the blocks in the entire pool. Signed-off-by: Tony Battersby Signed-off-by: Keith Busch Reviewed-by: Christoph Hellwig --- mm/dmapool.c | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/mm/dmapool.c b/mm/dmapool.c index 20616b760bb9c..ee993bb59fc27 100644 --- a/mm/dmapool.c +++ b/mm/dmapool.c @@ -43,10 +43,10 @@ struct dma_pool { /* the pool */ struct list_head page_list; spinlock_t lock; - size_t size; struct device *dev; - size_t allocation; - size_t boundary; + unsigned int size; + unsigned int allocation; + unsigned int boundary; char name[32]; struct list_head pools; }; @@ -73,7 +73,7 @@ static ssize_t pools_show(struct device *dev, struct device_attribute *attr, cha mutex_lock(&pools_lock); list_for_each_entry(pool, &dev->dma_pools, pools) { unsigned pages = 0; - unsigned blocks = 0; + size_t blocks = 0; spin_lock_irq(&pool->lock); list_for_each_entry(page, &pool->page_list, page_list) { @@ -83,9 +83,10 @@ static ssize_t pools_show(struct device *dev, struct device_attribute *attr, cha spin_unlock_irq(&pool->lock); /* per-pool info, no real statistics yet */ - size += sysfs_emit_at(buf, size, "%-16s %4u %4zu %4zu %2u\n", + size += sysfs_emit_at(buf, size, "%-16s %4zu %4zu %4u %2u\n", pool->name, blocks, - pages * (pool->allocation / pool->size), + (size_t) pages * + (pool->allocation / pool->size), pool->size, pages); } mutex_unlock(&pools_lock); @@ -133,7 +134,7 @@ struct dma_pool *dma_pool_create(const char *name, struct device *dev, else if (align & (align - 1)) return NULL; - if (size == 0) + if (size == 0 || size > INT_MAX) return NULL; else if (size < 4) size = 4; @@ -146,6 +147,8 @@ struct dma_pool *dma_pool_create(const char *name, struct device *dev, else if ((boundary < size) || (boundary & (boundary - 1))) return NULL; + boundary = min(boundary, allocation); + retval = kmalloc(sizeof(*retval), GFP_KERNEL); if (!retval) return retval; @@ -306,7 +309,7 @@ void *dma_pool_alloc(struct dma_pool *pool, gfp_t mem_flags, { unsigned long flags; struct dma_page *page; - size_t offset; + unsigned int offset; void *retval; might_alloc(mem_flags); From patchwork Fri Dec 16 20:16:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keith Busch X-Patchwork-Id: 34076 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:e747:0:0:0:0:0 with SMTP id c7csp1176431wrn; Fri, 16 Dec 2022 12:18:31 -0800 (PST) X-Google-Smtp-Source: AA0mqf6ShHmUwFzxFqDDLi5xGJ3IWB+cQys3YUTpkmFD9+XFu90jz22ROPyqIWBgbKULZ33Tnyr/ X-Received: by 2002:a17:902:e845:b0:189:f8d0:7b8f with SMTP id t5-20020a170902e84500b00189f8d07b8fmr40237199plg.63.1671221911386; Fri, 16 Dec 2022 12:18:31 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1671221911; cv=none; d=google.com; s=arc-20160816; b=KATrFJUuXoaRTDfq6qc65HAPbLYuh+Q5LzS9G2Y9V399rfweGk12ThJ3qko8wn75cg r7fj1MYDYsxg0ob+lfp6KTiVLmRXDl8la9Alp1/JAzl2YfAeVOqoB04TlwvZuFoS0UwF 4xfMoV1eaDqGl5Hg3MaXPZXXcAqmqtGyl7vqpwp04KB/6EYWaIrGgion1yLWOcO/BPOn LHlfEneybYi6OZw3oiBTAwdAU1VyjADq/OLLYSQhkvPcc5rlWgrnwIrGem4VMy4glNyF CjUbhwSe8h78IkU0vyNlpj1KUynl7nyp2i2tOTNtouF8pUaFWF/PpgapWhjoKDLbgHHJ Ewww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=P/KRpfmdlQUKxAAPrLlGtIIYCnA6qEIJS0aGmE3w/OM=; b=aD/ZdnPexsI8tlVEgzH7LlsHYbw9WlOZy5QIn/PCwY6wYQychHBoYiQ39WGneLVepD Sb6GtX9WzH6hm5M+sr50des1pw8WEDXLkxUBZkNeLdlD7HpEe+2p+aGr164s4uibaiKR j+LRl6Vg7bLR7m0SEKA8ela/Adshr5mUF+ik0wsNpNHtcpoh3qR9VK5190ACmXttbC1Y g4c27t816gP5GhXELLuuFQgf0aHHGUbjuZRDximBSKZTBVCGuewGbnhURTL3nlXkfW18 LJd6es/cZgZAKNlzgYybJezz7hj8ULxJLCy2HRWBUYdLGMdq4nMgKuJbYWmOWuMfIF73 qNmQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@meta.com header.s=s2048-2021-q4 header.b=MbCoiSgf; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=meta.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id i184-20020a6387c1000000b00478b771fa18si3490776pge.250.2022.12.16.12.18.18; Fri, 16 Dec 2022 12:18:31 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@meta.com header.s=s2048-2021-q4 header.b=MbCoiSgf; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=meta.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232011AbiLPURq (ORCPT + 99 others); Fri, 16 Dec 2022 15:17:46 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58886 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231965AbiLPURW (ORCPT ); Fri, 16 Dec 2022 15:17:22 -0500 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 480196A762 for ; Fri, 16 Dec 2022 12:17:21 -0800 (PST) Received: from pps.filterd (m0044010.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2BGJxe1t026562 for ; Fri, 16 Dec 2022 12:17:21 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=meta.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=s2048-2021-q4; bh=P/KRpfmdlQUKxAAPrLlGtIIYCnA6qEIJS0aGmE3w/OM=; b=MbCoiSgfMSiseLUY8bCAs3dbyFZQ3Rl0CLAubF9uzrqkfr/MvHkNbhvABzz3rPOT94Uw sTTUx6fDbStVDEpl/yLxoFGe5kd913/iZPC8NzP4KNck043HmxwxPnhDpEln4pd8i9jN 6Cw3Lal8BKx/kX+3JkIH1HhFzFzt7+aMQf/q36VH/uV+dYPUng6PyeXINbyO+CNAX5wU 9z2qD1wgvH7IFT9nDRQIKIlr5VhzMpcPlVILYm71FLYPu22UN5iWFdsEGlZAz1Kdfl3g vSiVkdpvfAUdtxLcTxFzUw2ZW8+WSkwiTrstgXw3VCJtgTgDtji/8m1fiSDv2xYl4U6n QQ== Received: from mail.thefacebook.com ([163.114.132.120]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3mgh7v57s3-7 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Fri, 16 Dec 2022 12:17:20 -0800 Received: from twshared8047.05.ash9.facebook.com (2620:10d:c085:208::11) by mail.thefacebook.com (2620:10d:c085:21d::4) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Fri, 16 Dec 2022 12:17:20 -0800 Received: by devbig007.nao1.facebook.com (Postfix, from userid 544533) id 289EAD042AB9; Fri, 16 Dec 2022 12:17:10 -0800 (PST) From: Keith Busch To: , , Matthew Wilcox CC: Tony Battersby , Kernel Team , Keith Busch Subject: [PATCHv2 05/11] dmapool: speedup DMAPOOL_DEBUG with init_on_alloc Date: Fri, 16 Dec 2022 12:16:19 -0800 Message-ID: <20221216201625.2362737-6-kbusch@meta.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20221216201625.2362737-1-kbusch@meta.com> References: <20221216201625.2362737-1-kbusch@meta.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: ZQVgAyTWC7jHrUwurblEzB0HFo65Zofu X-Proofpoint-ORIG-GUID: ZQVgAyTWC7jHrUwurblEzB0HFo65Zofu X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-16_14,2022-12-15_02,2022-06-22_01 X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1752403186740599190?= X-GMAIL-MSGID: =?utf-8?q?1752403186740599190?= From: Tony Battersby Avoid double-memset of the same allocated memory in dma_pool_alloc() when both DMAPOOL_DEBUG is enabled and init_on_alloc=1. Signed-off-by: Tony Battersby Signed-off-by: Keith Busch Reviewed-by: Christoph Hellwig --- mm/dmapool.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/dmapool.c b/mm/dmapool.c index ee993bb59fc27..eaed3ffb42aa8 100644 --- a/mm/dmapool.c +++ b/mm/dmapool.c @@ -356,7 +356,7 @@ void *dma_pool_alloc(struct dma_pool *pool, gfp_t mem_flags, break; } } - if (!(mem_flags & __GFP_ZERO)) + if (!want_init_on_alloc(mem_flags)) memset(retval, POOL_POISON_ALLOCATED, pool->size); #endif spin_unlock_irqrestore(&pool->lock, flags); From patchwork Fri Dec 16 20:16:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keith Busch X-Patchwork-Id: 34077 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:e747:0:0:0:0:0 with SMTP id c7csp1176462wrn; Fri, 16 Dec 2022 12:18:36 -0800 (PST) X-Google-Smtp-Source: AA0mqf7nMaaynzFevN3gPovp0CFuUxk/ehajGLIXpKsYQaYsyoToRUJK/iA5J77EfvoAk91aQoWn X-Received: by 2002:a17:902:ca14:b0:189:bbd3:a5a7 with SMTP id w20-20020a170902ca1400b00189bbd3a5a7mr27936030pld.51.1671221916134; Fri, 16 Dec 2022 12:18:36 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1671221916; cv=none; d=google.com; s=arc-20160816; b=ZK1gtPpQShZhwLpFAj1btKsRauY6xKCGboi6u3a4g8EDjNVVNgKunh46GQBsQ4L/FX lSTJ+bmTgcy86CGRMUJQAK2VRUIyML/DiL6xgPF/Rqq3nc02RBr9ih2a1BbGsRQXxv5q mPoh6ZGJ2PTCtDyg81B3J68SKVRk6Z60qh38ihAcfJGVDghD0pSbi4gc+E7/y3wGhueu yX9zrMhVuFt0OGG1nNexWlJAjNkQ0fXgm2tA/ZsYVxxro5JBiWK0NK0xUzaKw7m3Q+Ov XmqxipCiaHZKuwPyjTfVF2nPTsHMVDYpKyqFMRR9E8/oxtw+m6d6FhdTm67kzzrvM3j6 ri8Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=TZNkpjr8/3ibxb7IcC5QH+0+yx1Gj6xL4H95QeGVkb8=; b=elxCotnbty5TQjm98Yi2lZXvDOQnQC8rcC4RlXmY9lEBtDhPZQDogxMIkScoBfKs/2 AZPblVPdB8z0BPPJP8Z20ey/tJt9Molyh/xxmNZhZwWxEIJ9bcidY8HsU5jIU852xSuX ePlfyXMEGV49JtHAu5IJJxJwtRr8R+xsVjx51fHnc3cR3nzN8UcHC7OE/jjkp0p+LSCK CB1Ntw+w1I7JWr1Z0IgLZmiXozNM2FSyA6KXl1y4U5tyFjZTUb3ikz1hIuPXRxfEVyDx umST/4sMUx1ImTDPmMyTxOYuuLYwa6iiD13eb1Rd3UWoBKNZDqeepOb70/M1QRjE31Lo xshQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@meta.com header.s=s2048-2021-q4 header.b=Le2B0or9; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=meta.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id h11-20020a170902f2cb00b00186bc66d2cdsi3144986plc.75.2022.12.16.12.18.23; Fri, 16 Dec 2022 12:18:36 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@meta.com header.s=s2048-2021-q4 header.b=Le2B0or9; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=meta.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231982AbiLPURz (ORCPT + 99 others); Fri, 16 Dec 2022 15:17:55 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58960 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230389AbiLPURY (ORCPT ); Fri, 16 Dec 2022 15:17:24 -0500 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 679A672608 for ; Fri, 16 Dec 2022 12:17:23 -0800 (PST) Received: from pps.filterd (m0109334.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2BGJxCtl024349 for ; Fri, 16 Dec 2022 12:17:23 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=meta.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=s2048-2021-q4; bh=TZNkpjr8/3ibxb7IcC5QH+0+yx1Gj6xL4H95QeGVkb8=; b=Le2B0or9qIKN4/IDsOqRE2RNIBLBq5FZiadWiezcWuUvS9XFIoEKTu0GV7qP14BvcHD9 175I2MF7yY3OCK45Ei8UbmafeAbdxmSKGkTuMYg3ZCcDNpjeXSCUdAwaLbjTLSLhBZQ8 eCNcPvP0zWoOsXdvvixgXSEANHPKPAmprt0cjvG4m0nUBFGf5VTmXsT9OUT5g+OAuktR xVVhWVJ5UMh5q1td1i0SdeiS+c+hDUj0TUPkmScLzOAc7eDOWs72WgLcWIgL4WZMh0F2 bwVilxlvdyARmBjemREW31XWdNHSfUt279QMZ0rlaTSyF+Q5acEP7pzglKeVBsosc/sW JA== Received: from mail.thefacebook.com ([163.114.132.120]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3mgxjy8gcx-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Fri, 16 Dec 2022 12:17:23 -0800 Received: from snc-exhub201.TheFacebook.com (2620:10d:c085:21d::7) by snc-exhub103.TheFacebook.com (2620:10d:c085:11d::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Fri, 16 Dec 2022 12:17:22 -0800 Received: from twshared8047.05.ash9.facebook.com (2620:10d:c085:108::4) by mail.thefacebook.com (2620:10d:c085:21d::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Fri, 16 Dec 2022 12:17:22 -0800 Received: by devbig007.nao1.facebook.com (Postfix, from userid 544533) id 36E3ED042ABC; Fri, 16 Dec 2022 12:17:10 -0800 (PST) From: Keith Busch To: , , Matthew Wilcox CC: Tony Battersby , Kernel Team , Keith Busch Subject: [PATCHv2 06/11] dmapool: move debug code to own functions Date: Fri, 16 Dec 2022 12:16:20 -0800 Message-ID: <20221216201625.2362737-7-kbusch@meta.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20221216201625.2362737-1-kbusch@meta.com> References: <20221216201625.2362737-1-kbusch@meta.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: q6HuErhKedLs_TltBFut5biXvYNyFJwi X-Proofpoint-ORIG-GUID: q6HuErhKedLs_TltBFut5biXvYNyFJwi X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-16_14,2022-12-15_02,2022-06-22_01 X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1752403191816008412?= X-GMAIL-MSGID: =?utf-8?q?1752403191816008412?= From: Keith Busch Clean up the normal path by moving the debug code outside it. Signed-off-by: Keith Busch --- mm/dmapool.c | 96 +++++++++++++++++++++++++++++----------------------- 1 file changed, 54 insertions(+), 42 deletions(-) diff --git a/mm/dmapool.c b/mm/dmapool.c index eaed3ffb42aa8..8a7aa19e650a1 100644 --- a/mm/dmapool.c +++ b/mm/dmapool.c @@ -294,6 +294,32 @@ void dma_pool_destroy(struct dma_pool *pool) } EXPORT_SYMBOL(dma_pool_destroy); +static inline void pool_check_block(struct dma_pool *pool, void *retval, + unsigned int offset, gfp_t mem_flags) +{ +#ifdef DMAPOOL_DEBUG + int i; + u8 *data = retval; + /* page->offset is stored in first 4 bytes */ + for (i = sizeof(offset); i < pool->size; i++) { + if (data[i] == POOL_POISON_FREED) + continue; + dev_err(pool->dev, "%s %s, %p (corrupted)\n", + __func__, pool->name, retval); + + /* + * Dump the first 4 bytes even if they are not + * POOL_POISON_FREED + */ + print_hex_dump(KERN_ERR, "", DUMP_PREFIX_OFFSET, 16, 1, + data, pool->size, 1); + break; + } + if (!want_init_on_alloc(mem_flags)) + memset(retval, POOL_POISON_ALLOCATED, pool->size); +#endif +} + /** * dma_pool_alloc - get a block of consistent memory * @pool: dma pool that will produce the block @@ -336,29 +362,7 @@ void *dma_pool_alloc(struct dma_pool *pool, gfp_t mem_flags, page->offset = *(int *)(page->vaddr + offset); retval = offset + page->vaddr; *handle = offset + page->dma; -#ifdef DMAPOOL_DEBUG - { - int i; - u8 *data = retval; - /* page->offset is stored in first 4 bytes */ - for (i = sizeof(page->offset); i < pool->size; i++) { - if (data[i] == POOL_POISON_FREED) - continue; - dev_err(pool->dev, "%s %s, %p (corrupted)\n", - __func__, pool->name, retval); - - /* - * Dump the first 4 bytes even if they are not - * POOL_POISON_FREED - */ - print_hex_dump(KERN_ERR, "", DUMP_PREFIX_OFFSET, 16, 1, - data, pool->size, 1); - break; - } - } - if (!want_init_on_alloc(mem_flags)) - memset(retval, POOL_POISON_ALLOCATED, pool->size); -#endif + pool_check_block(pool, retval, offset, mem_flags); spin_unlock_irqrestore(&pool->lock, flags); if (want_init_on_alloc(mem_flags)) @@ -381,6 +385,32 @@ static struct dma_page *pool_find_page(struct dma_pool *pool, dma_addr_t dma) return NULL; } +static inline bool pool_page_err(struct dma_pool *pool, struct dma_page *page, + void *vaddr) +{ +#ifdef DMAPOOL_DEBUG + unsigned int chain = page->offset; + + if ((dma - page->dma) != offset) { + dev_err(pool->dev, "%s %s, %p (bad vaddr)/%pad\n", + __func__, pool->name, vaddr, &dma); + return true; + } + + while (chain < pool->allocation) { + if (chain != offset) { + chain = *(int *)(page->vaddr + chain); + continue; + } + dev_err(pool->dev, "%s %s, dma %pad already free\n", + __func__, pool->name, &dma); + return true; + } + memset(vaddr, POOL_POISON_FREED, pool->size); +#endif + return false; +} + /** * dma_pool_free - put block back into dma pool * @pool: the dma pool holding the block @@ -408,28 +438,10 @@ void dma_pool_free(struct dma_pool *pool, void *vaddr, dma_addr_t dma) offset = vaddr - page->vaddr; if (want_init_on_free()) memset(vaddr, 0, pool->size); -#ifdef DMAPOOL_DEBUG - if ((dma - page->dma) != offset) { + if (pool_page_err(pool, page, vaddr)) { spin_unlock_irqrestore(&pool->lock, flags); - dev_err(pool->dev, "%s %s, %p (bad vaddr)/%pad\n", - __func__, pool->name, vaddr, &dma); return; } - { - unsigned int chain = page->offset; - while (chain < pool->allocation) { - if (chain != offset) { - chain = *(int *)(page->vaddr + chain); - continue; - } - spin_unlock_irqrestore(&pool->lock, flags); - dev_err(pool->dev, "%s %s, dma %pad already free\n", - __func__, pool->name, &dma); - return; - } - } - memset(vaddr, POOL_POISON_FREED, pool->size); -#endif page->in_use--; *(int *)vaddr = page->offset; From patchwork Fri Dec 16 20:16:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keith Busch X-Patchwork-Id: 34075 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:e747:0:0:0:0:0 with SMTP id c7csp1176374wrn; Fri, 16 Dec 2022 12:18:27 -0800 (PST) X-Google-Smtp-Source: AA0mqf5JuH8CnxqTp9oOCaKpCww4CiwBGebIMdsW2XYqeLb8rEcIFdJgngCnm/Du1x4/plYjaFLC X-Received: by 2002:a17:903:2408:b0:185:441e:2d77 with SMTP id e8-20020a170903240800b00185441e2d77mr39343693plo.14.1671221906747; Fri, 16 Dec 2022 12:18:26 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1671221906; cv=none; d=google.com; s=arc-20160816; b=CiQe7F6ZnZs8fMQgiJ4CqP4OycV8gudPXhEYfLy9i1p5bMQYnsFYvaBBwJooovDisx cyWCgWke3H6HPMKBCsWJUoB+XL9wPNxSkoSTpq9UDSF9OI8Sz/vufXY2R5Dy6xS8kTPR cAvwRZAEcBheCK2QQSpmvUkp5SbrpslUkR1mBUYQ0h8U4zzI/ltYFXweTi3p2K9vIIrH 52H0w2dl/H8Hd2PIq2KuCBftjfQ5JD6smXBuaWTOljmfSxKBQz/yUxDIkyDVEoGpqRJS 54PzRyNfSKh3N+hXladtR5INKSFBL5vL/31t0bgeyRpTAWjN0kzBiGt5LTF5yV5sXREA TNzQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=ADvvFQUdwkTabb+mgOJxdznef25hLYACuBWcCNuRK/M=; b=imyDuJMntvBWkRTRo55Ojt4Nwr7cPDJUMgxBVWqDbApE0ESva2DbEQWaZ4+sVUKuC3 E1Sqq7bXO2efMN6UEXwbGPl+uJNnQMj6+8XapIsnJiRO0zfpyy77gb8ew5sfKVde8VdX r18vqbrZ8kbztEL85eJpHZujVe60lneOZVJdCU8CE7mUWC7uehNM8tYTfNXTyNkMTtAW gPXnMPVfb0gSflO7I4C996PvO5mveGkI+I8DIAmKI5h3sZxdqR9C06JR0HiegVrdJAbC 0sH/xFkVD90s82/RmvAVzTn4j+2nCoEy0xaTbVttGwArgLSuvtU4mp3yeaAg+IRWGVG6 kFyA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@meta.com header.s=s2048-2021-q4 header.b=STzCkNPe; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=meta.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id e5-20020a170902784500b00176b7b8cf2esi2979289pln.586.2022.12.16.12.18.14; Fri, 16 Dec 2022 12:18:26 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@meta.com header.s=s2048-2021-q4 header.b=STzCkNPe; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=meta.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231991AbiLPURl (ORCPT + 99 others); Fri, 16 Dec 2022 15:17:41 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58934 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231971AbiLPURX (ORCPT ); Fri, 16 Dec 2022 15:17:23 -0500 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5A2FF7260D for ; Fri, 16 Dec 2022 12:17:21 -0800 (PST) Received: from pps.filterd (m0148461.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2BGJxCtI024587 for ; Fri, 16 Dec 2022 12:17:21 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=meta.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=s2048-2021-q4; bh=ADvvFQUdwkTabb+mgOJxdznef25hLYACuBWcCNuRK/M=; b=STzCkNPeqfgrpppDPw5oSsN3vOKMm72rvWYfooQHmAohLWtRhPDzTrjukAr2+N4WppCo 4xtpDJWyYTkk3xoTbuVy/4pguJ5xy15tVj9S7KISqhZwQAx3iTegSd8mQeUl/u/y+rF0 XZXBOPIJKhpC/kwbwy8yu914EOgGhsLmuhgiTDhSz8agnvMAcnzD6UdgHs5N+y3xIdvt epoTvXQQEUwlW3RTfwNffUzxoTGfVYqDLWap3Neo+O5cuzgnnW8oY4Enx6FQBXxK5iYN EicOxa1Rl8qJk11jKwYXIlWB4z/Ugl3M8JHl7nIV/FtkKqoO3JDMrmsru82aItyBzHMr vA== Received: from mail.thefacebook.com ([163.114.132.120]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3mgwvu0uu8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Fri, 16 Dec 2022 12:17:21 -0800 Received: from twshared21680.02.ash8.facebook.com (2620:10d:c085:208::11) by mail.thefacebook.com (2620:10d:c085:11d::4) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Fri, 16 Dec 2022 12:17:20 -0800 Received: by devbig007.nao1.facebook.com (Postfix, from userid 544533) id 45158D042ABE; Fri, 16 Dec 2022 12:17:10 -0800 (PST) From: Keith Busch To: , , Matthew Wilcox CC: Tony Battersby , Kernel Team , Keith Busch Subject: [PATCHv2 07/11] dmapool: rearrange page alloc failure handling Date: Fri, 16 Dec 2022 12:16:21 -0800 Message-ID: <20221216201625.2362737-8-kbusch@meta.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20221216201625.2362737-1-kbusch@meta.com> References: <20221216201625.2362737-1-kbusch@meta.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: 1yp9UXgPFRP2qgbBLE18nxXddd1am-SZ X-Proofpoint-ORIG-GUID: 1yp9UXgPFRP2qgbBLE18nxXddd1am-SZ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-16_14,2022-12-15_02,2022-06-22_01 X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1752403181991384372?= X-GMAIL-MSGID: =?utf-8?q?1752403181991384372?= From: Keith Busch Handle the error in a condition so the good path can be in the normal flow. Signed-off-by: Keith Busch Reviewed-by: Christoph Hellwig --- mm/dmapool.c | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/mm/dmapool.c b/mm/dmapool.c index 8a7aa19e650a1..f77e2af4941de 100644 --- a/mm/dmapool.c +++ b/mm/dmapool.c @@ -222,17 +222,17 @@ static struct dma_page *pool_alloc_page(struct dma_pool *pool, gfp_t mem_flags) return NULL; page->vaddr = dma_alloc_coherent(pool->dev, pool->allocation, &page->dma, mem_flags); - if (page->vaddr) { -#ifdef DMAPOOL_DEBUG - memset(page->vaddr, POOL_POISON_FREED, pool->allocation); -#endif - pool_initialise_page(pool, page); - page->in_use = 0; - page->offset = 0; - } else { + if (!page->vaddr) { kfree(page); - page = NULL; + return NULL; } +#ifdef DMAPOOL_DEBUG + memset(page->vaddr, POOL_POISON_FREED, pool->allocation); +#endif + pool_initialise_page(pool, page); + page->in_use = 0; + page->offset = 0; + return page; } From patchwork Fri Dec 16 20:16:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keith Busch X-Patchwork-Id: 34079 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:e747:0:0:0:0:0 with SMTP id c7csp1177108wrn; Fri, 16 Dec 2022 12:19:48 -0800 (PST) X-Google-Smtp-Source: AA0mqf4IMXR0VF9govEY1vmfeGv2nCs51we6YwfSV0+UxModw55xruMMMK8/0WqpgIoY56hJPAPr X-Received: by 2002:a05:6a00:a07:b0:573:3de7:89a with SMTP id p7-20020a056a000a0700b005733de7089amr49062300pfh.4.1671221987967; Fri, 16 Dec 2022 12:19:47 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1671221987; cv=none; d=google.com; s=arc-20160816; b=t5TZzheyxVoIwrJu8jNYv8xCvtZ3beJVmm5B7iTaa+EKKLNi9d/OEdgSRqour0Ki29 AQo3vv3yhGDzWEZ9VuNRFhR16BVZvGNXlPBmqwljX+IERIaf+FCFqNhV2iZNsl5/+en9 kZCNQY2uEivfu6oqcEK5oUH8rlcyfPdpeJt4I3NTkBe9ceK9PnClfcf4NbIOCWcL89Az 21jIM2b0AOKdWjN2Lpeh6cr9uwkQPQv0oBbUgdeMbv6IVKc2blp/J+a4YDNzaUxIdkgV QYA4z05vF1pw1zpfkm3A+5tHy38wZ5ONRF3sjWwHV7eGPkH8GyCGhW0wgiVw7mAo4Cjy v+Bw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=5sHDLhRqo9/RufklnGKFVoQ7JnNipMp+QhHHPmz6D+s=; b=yKytgQHZQBkpxfQYn93YRxCC/3m1HwzIe7SNjBGRsZrMZnpQoHRAS7meKqzadXsDt2 CKqLxVWV7iMCFLD2pVCzFtOTJpBXOjsYdZNAxg2Bvcv/cYuDj5hBZRnKJxRr6cycducx mfoP9EAZXfIhRI95ZF+myGjhjaEPB2ySRovlTG9LCIErxbAduLZHsrEcLWG2U1I2TbnV Rtxf1XbAcff1suxIhMKhOFuoVnj4aSNtfhZKQo4XH1CD+PpD+LG2lvAFXwijtTOHqbVD F66cNjD3KD6xjddVOeM9NtwelFn+tYuCNkxbYFpZdrOY5Aco5u0FcVC5Fov90JjacZuR K6HA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@meta.com header.s=s2048-2021-q4 header.b=B3+PZwrA; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=meta.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id o2-20020a056a0015c200b00571ea18493dsi3727585pfu.175.2022.12.16.12.19.35; Fri, 16 Dec 2022 12:19:47 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@meta.com header.s=s2048-2021-q4 header.b=B3+PZwrA; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=meta.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232067AbiLPUSN (ORCPT + 99 others); Fri, 16 Dec 2022 15:18:13 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59554 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232020AbiLPURs (ORCPT ); Fri, 16 Dec 2022 15:17:48 -0500 Received: from mx0a-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 18B9E7262D for ; Fri, 16 Dec 2022 12:17:33 -0800 (PST) Received: from pps.filterd (m0001303.ppops.net [127.0.0.1]) by m0001303.ppops.net (8.17.1.19/8.17.1.19) with ESMTP id 2BGJxW12024664 for ; Fri, 16 Dec 2022 12:17:32 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=meta.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=s2048-2021-q4; bh=5sHDLhRqo9/RufklnGKFVoQ7JnNipMp+QhHHPmz6D+s=; b=B3+PZwrAtaqliipErF2lfKdvLX5EMW9rOh5+K5jLhFVOZTR6d84n31oJMOU7BQP/Jy7d D8kKSAzX4C+1S5wPBIyz0XG//m6IRbrBuVAH3NI3MrEBBRvOz3mNAk+YmZcR+ZAWWwFZ bhKgP5sfoRhbcJvhAx6SCWUm2KK+XBD6ktgWXKG2U3XXJLKkfcYUP9eJBEgC2lSKnM/I jR0zY/1WX2HYcwlCkv4At2tSXEmljOaozDVn9+FU7YNHCMW/NbzkkJECwOl0vVG2tw4Z ua5e9XR/e3zItHGcTMZEPYxkFX4QHySE465IScJgjoqOOWppX3osl/JdnwdWxLNcB07K 4A== Received: from maileast.thefacebook.com ([163.114.130.16]) by m0001303.ppops.net (PPS) with ESMTPS id 3mg3hn2egg-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Fri, 16 Dec 2022 12:17:32 -0800 Received: from twshared7043.05.ash9.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:82::e) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Fri, 16 Dec 2022 12:17:31 -0800 Received: by devbig007.nao1.facebook.com (Postfix, from userid 544533) id 4F484D042ACE; Fri, 16 Dec 2022 12:17:10 -0800 (PST) From: Keith Busch To: , , Matthew Wilcox CC: Tony Battersby , Kernel Team , Keith Busch Subject: [PATCHv2 08/11] dmapool: consolidate page initialization Date: Fri, 16 Dec 2022 12:16:22 -0800 Message-ID: <20221216201625.2362737-9-kbusch@meta.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20221216201625.2362737-1-kbusch@meta.com> References: <20221216201625.2362737-1-kbusch@meta.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: -0JmSdXK1FHllZ1u-Rup9TA3m6XpVZrI X-Proofpoint-ORIG-GUID: -0JmSdXK1FHllZ1u-Rup9TA3m6XpVZrI X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-16_14,2022-12-15_02,2022-06-22_01 X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1752403267652676848?= X-GMAIL-MSGID: =?utf-8?q?1752403267652676848?= From: Keith Busch Various fields of the dma pool are set in different places. Move it all to one function. Signed-off-by: Keith Busch --- mm/dmapool.c | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/mm/dmapool.c b/mm/dmapool.c index f77e2af4941de..33d20ceff18c5 100644 --- a/mm/dmapool.c +++ b/mm/dmapool.c @@ -202,6 +202,8 @@ static void pool_initialise_page(struct dma_pool *pool, struct dma_page *page) unsigned int offset = 0; unsigned int next_boundary = pool->boundary; + page->in_use = 0; + page->offset = 0; do { unsigned int next = offset + pool->size; if (unlikely((next + pool->size) >= next_boundary)) { @@ -211,6 +213,8 @@ static void pool_initialise_page(struct dma_pool *pool, struct dma_page *page) *(int *)(page->vaddr + offset) = next; offset = next; } while (offset < pool->allocation); + + list_add(&page->page_list, &pool->page_list); } static struct dma_page *pool_alloc_page(struct dma_pool *pool, gfp_t mem_flags) @@ -229,10 +233,6 @@ static struct dma_page *pool_alloc_page(struct dma_pool *pool, gfp_t mem_flags) #ifdef DMAPOOL_DEBUG memset(page->vaddr, POOL_POISON_FREED, pool->allocation); #endif - pool_initialise_page(pool, page); - page->in_use = 0; - page->offset = 0; - return page; } @@ -354,8 +354,7 @@ void *dma_pool_alloc(struct dma_pool *pool, gfp_t mem_flags, return NULL; spin_lock_irqsave(&pool->lock, flags); - - list_add(&page->page_list, &pool->page_list); + pool_initialise_page(pool, page); ready: page->in_use++; offset = page->offset; From patchwork Fri Dec 16 20:16:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keith Busch X-Patchwork-Id: 34078 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:e747:0:0:0:0:0 with SMTP id c7csp1176561wrn; Fri, 16 Dec 2022 12:18:47 -0800 (PST) X-Google-Smtp-Source: AA0mqf6vyLRTWcJQQl7T4Dsz8FpFvgg0D+dfOJJkAsXVqDyxfiTH/nqbFHLFJ/H6yczvZ6V4FNvs X-Received: by 2002:a17:90a:9742:b0:219:c2f2:f818 with SMTP id i2-20020a17090a974200b00219c2f2f818mr35012577pjw.42.1671221927675; Fri, 16 Dec 2022 12:18:47 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1671221927; cv=none; d=google.com; s=arc-20160816; b=SVE9ct9i+/WddmMg2xhb8LOtssIKVY5IeqoxkpVIE1OSLSNgqRtd/ZS6x7+cwOmcRk hqzP4nnvywqfFjMowNlvEYvz09CIOwWgrEYfYwuw7m5cWYZrFW8tKaOjtSWOudmrgnPl CqocE8LOhlI1Lrv7VRW9FzFBBB2o126gUbE3lPPniUfTvrtPzgZ+SjUehTIDADThmJRp OougqLiyVZ6cgBxGlaxb3xfroK0HlAtjOhRi7d05mBMhFmVotYLR1S/kf6wY2VGrD8GG Bzryr1qlVD6vgc84u1NV6AwK+Hj9i8y9gA9fsVQgBtuOaeYpSuqGfVae9JX9CJDELMO9 pM6Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=vVY03djUbGuw9RyCWd6Eexf1lAzlqKJXlcJv1cor+cc=; b=bHCbOW6WjoPoalKqmB3Cb/h6K3gpRmezHuFuojaA7x+epyv1pAKefMo3jQZfpGAWHZ Lq4UT7tjFGBMQpqPW9CvcCh+rHpTy8ztS+IvinxdrdS/y721OI+mIfGS762iGNB+h0q2 cM0sCxtfHkLFMvyxDXy7hb6e+8o/rSdeyrgSncd3qCPQyMTtic2LODLV6Oyj29WTBr+/ yOur6ZhRe7+uEwmB3YT1GiSVXLDgJAzDQfitldX34kzaccKDwwbiCS7k1tjCFCDAM1Pn D5SjORYPwluqs/1a5uqvXXG7JUhGlwRWASE5iOj0uZnQmQUXXp/R6eLtrR/hqSbHZX48 fcmA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@meta.com header.s=s2048-2021-q4 header.b=EKCA0B+h; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=meta.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id lr4-20020a17090b4b8400b00202c7f4631esi10093552pjb.19.2022.12.16.12.18.34; Fri, 16 Dec 2022 12:18:47 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@meta.com header.s=s2048-2021-q4 header.b=EKCA0B+h; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=meta.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232038AbiLPUSC (ORCPT + 99 others); Fri, 16 Dec 2022 15:18:02 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58962 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231600AbiLPURY (ORCPT ); Fri, 16 Dec 2022 15:17:24 -0500 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A898172605 for ; Fri, 16 Dec 2022 12:17:23 -0800 (PST) Received: from pps.filterd (m0109334.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2BGJxCtm024349 for ; Fri, 16 Dec 2022 12:17:23 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=meta.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=s2048-2021-q4; bh=vVY03djUbGuw9RyCWd6Eexf1lAzlqKJXlcJv1cor+cc=; b=EKCA0B+hrgWZGusugiQjREbG/siOJhUpzpm7y5E7Byc0U1PFytmCSSpP+757rRLlob3M /DAkqpdjkU+berglGccH1db6L+KkuuhcMEb28WPevJe8nz6PtsfX3n6J6UTZtxBkUd9x ohwjRsEQY5ex0hF2rJOrXt+rNPYYzoxLNkG7O4Qsfi0i3vvg294ITcoBm3af2cLcDuc5 3DUH6NPTjIJTd7qSu9VDmoSKBlPcb9liN8SaXpkox/+3HKnXZmlF4JayolAEbZiFonBN IHF3j3reEBigwDGFUZvdiP2zwtjbZzZW8hzOLipYwJbBJEZodYBTbhYztHun1U1wdfnz Og== Received: from mail.thefacebook.com ([163.114.132.120]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3mgxjy8gcx-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Fri, 16 Dec 2022 12:17:23 -0800 Received: from twshared8047.05.ash9.facebook.com (2620:10d:c085:108::4) by mail.thefacebook.com (2620:10d:c085:11d::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Fri, 16 Dec 2022 12:17:22 -0800 Received: by devbig007.nao1.facebook.com (Postfix, from userid 544533) id 619E0D042AD5; Fri, 16 Dec 2022 12:17:10 -0800 (PST) From: Keith Busch To: , , Matthew Wilcox CC: Tony Battersby , Kernel Team , Keith Busch Subject: [PATCHv2 09/11] dmapool: simplify freeing Date: Fri, 16 Dec 2022 12:16:23 -0800 Message-ID: <20221216201625.2362737-10-kbusch@meta.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20221216201625.2362737-1-kbusch@meta.com> References: <20221216201625.2362737-1-kbusch@meta.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: f6v824Ga87VXh6Ka0_PRSnOIlVsNosSW X-Proofpoint-ORIG-GUID: f6v824Ga87VXh6Ka0_PRSnOIlVsNosSW X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-16_14,2022-12-15_02,2022-06-22_01 X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1752403203958062988?= X-GMAIL-MSGID: =?utf-8?q?1752403203958062988?= From: Keith Busch The actions for busy and not busy are mostly the same, so combine these and remove the unnecessary function. Also, the pool is about to be freed so there's no need to poison the page data since we only check for poison on alloc, which can't be done on a freed pool. Signed-off-by: Keith Busch --- mm/dmapool.c | 29 ++++++----------------------- 1 file changed, 6 insertions(+), 23 deletions(-) diff --git a/mm/dmapool.c b/mm/dmapool.c index 33d20ceff18c5..44622f2bf4641 100644 --- a/mm/dmapool.c +++ b/mm/dmapool.c @@ -241,18 +241,6 @@ static inline bool is_page_busy(struct dma_page *page) return page->in_use != 0; } -static void pool_free_page(struct dma_pool *pool, struct dma_page *page) -{ - dma_addr_t dma = page->dma; - -#ifdef DMAPOOL_DEBUG - memset(page->vaddr, POOL_POISON_FREED, pool->allocation); -#endif - dma_free_coherent(pool->dev, pool->allocation, page->vaddr, dma); - list_del(&page->page_list); - kfree(page); -} - /** * dma_pool_destroy - destroys a pool of dma memory blocks. * @pool: dma pool that will be destroyed @@ -280,14 +268,14 @@ void dma_pool_destroy(struct dma_pool *pool) mutex_unlock(&pools_reg_lock); list_for_each_entry_safe(page, tmp, &pool->page_list, page_list) { - if (is_page_busy(page)) { + if (!is_page_busy(page)) + dma_free_coherent(pool->dev, pool->allocation, + page->vaddr, page->dma); + else dev_err(pool->dev, "%s %s, %p busy\n", __func__, pool->name, page->vaddr); - /* leak the still-in-use consistent memory */ - list_del(&page->page_list); - kfree(page); - } else - pool_free_page(pool, page); + list_del(&page->page_list); + kfree(page); } kfree(pool); @@ -445,11 +433,6 @@ void dma_pool_free(struct dma_pool *pool, void *vaddr, dma_addr_t dma) page->in_use--; *(int *)vaddr = page->offset; page->offset = offset; - /* - * Resist a temptation to do - * if (!is_page_busy(page)) pool_free_page(pool, page); - * Better have a few empty pages hang around. - */ spin_unlock_irqrestore(&pool->lock, flags); } EXPORT_SYMBOL(dma_pool_free); From patchwork Fri Dec 16 20:16:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keith Busch X-Patchwork-Id: 34081 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:e747:0:0:0:0:0 with SMTP id c7csp1180531wrn; Fri, 16 Dec 2022 12:28:12 -0800 (PST) X-Google-Smtp-Source: AA0mqf5jawtqSqviS4LjKBLby0cC+W8OHvhdyzjDnCrbqXvr45kWM27Uo5pPjgUyXOTkrSsf6eS6 X-Received: by 2002:a17:907:7704:b0:7c1:79ce:a431 with SMTP id kw4-20020a170907770400b007c179cea431mr19742032ejc.25.1671222492219; Fri, 16 Dec 2022 12:28:12 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1671222492; cv=none; d=google.com; s=arc-20160816; b=p2kNm11Gkven0TPTybyxAHTs7VSUTI3FE8qQIrgiH7ycRLTOmH8IsqEuAWq0BbvHfF Z5sQn5GDkwVxYwOfHc5uuifcK4pyf3RmqGsFyQUSxLpRXlib1znFauGLYUSR2CzQSRb9 LBsx5WcR1lalUCKbV5F3MS3jXKTQ0BSxJW/+PD108bAIaRH/87NloVBVHWRCZftft4pq yLUakBk6IlXiyy86WIupbCMr/NTtowXjy6v6GvUigCR0LicCrgK4hTKHVtaNaLvTYNIa Ygr8B7es3coxZ5crSbaWerM8NkA6UmoC5i2E1G943Q2JXcJOcVllxr0KyBWczkawh4aK KR5Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=F5ndiXKSIYwdaNgVa7CtE+9m7dnA1GNZSuFoZd2plto=; b=JekLCxmXem+qqAyHA7/h8sN7jIvXvaBTsEvqMLm9VGP/PMlYU/Xm/rzcrWxsNCH2Yn HKuGayB1O6r7Qr/Q1uJt0N/MEONYJAwnB8HWXNDDICvNDf/TTM/y7meezT0Lc0F5auu2 6w8dEZ35Qt/+BNRLa4LNANF7arh1ylEVJsM/T9djK5GDd53RZU94KRBeESSeZje6AHzp mVDExFXp/+QFPaYcPcnzcNUaBP1yLkzhFvkyEIyJ0L6kQxsDR9HpGFrWJRcYy4OAvx3w Z43vl1MxDmIXaVfNrR6JqeEoiqo8qyvXHMEARKfNQYklztdcqjQkAzzrQl7CYl0m2YiQ 6/gA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@meta.com header.s=s2048-2021-q4 header.b=Vz3TOfSo; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=meta.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id eg38-20020a05640228a600b0046ac13261a1si2651930edb.459.2022.12.16.12.27.46; Fri, 16 Dec 2022 12:28:12 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@meta.com header.s=s2048-2021-q4 header.b=Vz3TOfSo; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=meta.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232084AbiLPUSV (ORCPT + 99 others); Fri, 16 Dec 2022 15:18:21 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59738 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232029AbiLPURw (ORCPT ); Fri, 16 Dec 2022 15:17:52 -0500 Received: from mx0a-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2CB9472621 for ; Fri, 16 Dec 2022 12:17:34 -0800 (PST) Received: from pps.filterd (m0001303.ppops.net [127.0.0.1]) by m0001303.ppops.net (8.17.1.19/8.17.1.19) with ESMTP id 2BGJxW16024664 for ; Fri, 16 Dec 2022 12:17:34 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=meta.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=s2048-2021-q4; bh=F5ndiXKSIYwdaNgVa7CtE+9m7dnA1GNZSuFoZd2plto=; b=Vz3TOfSogj9YKRFU0FPVZV0gvOoQfhhRcsSA2dhXhwwH5EVElRZRqVcp+5Kia1PCjJpH PCiVZBhnC2ZHhHKTD99jCGXf65/gI1n25OjFKY2uMuHEkwFZItTEMf2nKCzxlNTr17hJ bwYX6YWAYRudAD9vyv02nU9aIyZMn+e75WlL0CgObA04Vh7/SawYqZyWVCcgPO+vJD5G VpKN6+BtPKllYrYXIEZ2U4j31ciACWiy8T8fdbD9o2DnxEeTxBHLFjh0qYndfkzxLOD5 8/XA8/lpO/Zv+0HF2a5k44B9b4kC8/n3JVtDEKzykCSu3oilrAxp0ZQDA5CJcIrqXwXF Cw== Received: from maileast.thefacebook.com ([163.114.130.16]) by m0001303.ppops.net (PPS) with ESMTPS id 3mg3hn2egg-6 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Fri, 16 Dec 2022 12:17:34 -0800 Received: from twshared7043.05.ash9.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:82::e) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Fri, 16 Dec 2022 12:17:31 -0800 Received: by devbig007.nao1.facebook.com (Postfix, from userid 544533) id 6DD18D042AD7; Fri, 16 Dec 2022 12:17:10 -0800 (PST) From: Keith Busch To: , , Matthew Wilcox CC: Tony Battersby , Kernel Team , Keith Busch Subject: [PATCHv2 10/11] dmapool: don't memset on free twice Date: Fri, 16 Dec 2022 12:16:24 -0800 Message-ID: <20221216201625.2362737-11-kbusch@meta.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20221216201625.2362737-1-kbusch@meta.com> References: <20221216201625.2362737-1-kbusch@meta.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: 7u5zG4jAVa0GYklJhRAMokqPf1kxQiva X-Proofpoint-ORIG-GUID: 7u5zG4jAVa0GYklJhRAMokqPf1kxQiva X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-16_14,2022-12-15_02,2022-06-22_01 X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1752403796052969172?= X-GMAIL-MSGID: =?utf-8?q?1752403796052969172?= From: Keith Busch If debug is enabled, dmapool will poison the range, so no need to clear it to 0 immediately before writing over it. Signed-off-by: Keith Busch Reviewed-by: Christoph Hellwig --- mm/dmapool.c | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/mm/dmapool.c b/mm/dmapool.c index 44622f2bf4641..f5b79c3268856 100644 --- a/mm/dmapool.c +++ b/mm/dmapool.c @@ -372,10 +372,10 @@ static struct dma_page *pool_find_page(struct dma_pool *pool, dma_addr_t dma) return NULL; } +#ifdef DMAPOOL_DEBUG static inline bool pool_page_err(struct dma_pool *pool, struct dma_page *page, void *vaddr) { -#ifdef DMAPOOL_DEBUG unsigned int chain = page->offset; if ((dma - page->dma) != offset) { @@ -394,9 +394,17 @@ static inline bool pool_page_err(struct dma_pool *pool, struct dma_page *page, return true; } memset(vaddr, POOL_POISON_FREED, pool->size); -#endif return false; } +#else +static inline bool pool_page_err(struct dma_pool *pool, struct dma_page *page, + void *vaddr) +{ + if (want_init_on_free()) + memset(vaddr, 0, pool->size); + return false; +} +#endif /** * dma_pool_free - put block back into dma pool @@ -423,8 +431,6 @@ void dma_pool_free(struct dma_pool *pool, void *vaddr, dma_addr_t dma) } offset = vaddr - page->vaddr; - if (want_init_on_free()) - memset(vaddr, 0, pool->size); if (pool_page_err(pool, page, vaddr)) { spin_unlock_irqrestore(&pool->lock, flags); return; From patchwork Fri Dec 16 20:16:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keith Busch X-Patchwork-Id: 34080 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:e747:0:0:0:0:0 with SMTP id c7csp1177111wrn; Fri, 16 Dec 2022 12:19:48 -0800 (PST) X-Google-Smtp-Source: AA0mqf5OYoyjkwR9ysbhEG3ZIH106IF+nndKGMiSGQtOX9t6UHIm0P3moAN1dSme5TpBDISfL50w X-Received: by 2002:a17:902:7407:b0:189:5b2b:3df4 with SMTP id g7-20020a170902740700b001895b2b3df4mr30144951pll.24.1671221988221; Fri, 16 Dec 2022 12:19:48 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1671221988; cv=none; d=google.com; s=arc-20160816; b=nbPmTqMP5wWxvWEmvZEop468tmG1ezYIJ8G0Lr4LhkmZrouQ2g49oR1z6vPCC3heWG 8AJTRmd3wzw4nQyTGwKsUdaB/giQeajfYwZKGuDA/RIEtioiDdBdedzZF1FoJJLReDlr cY6kpHSw/ibu5j2JFAzhqzA1cJy38am2Dm7sJttJTIEGcPW0DETPnzAeMBS4O4aJYQxV ftexQEf6X5hs8uuz9lQ9WcOcz6dhiPC5nMghEnoGfjjSHWkcfRHpLdjaIcoaLflxyvEy S97kc7+i7tiibhHXYg0L9C42beBgoAVlDYZRWkX8sEZvRmVnZR0I377Ib0P9hHsvn02R l4pw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=qfJz/DDH55XtUOlIDeacYqymxZSjqIq0VpHDCFV+y+Y=; b=xzMr52Z1/TZ7aS8OPN4jLA0zrd73NecdlFoAUQhvNMW6pE+MdJTtC6chYQOVIP9sDA VLgm6syNMSv6wJgXppvLilGFAOe2MnkhDFTtc0cIgZteZY6L8/RB7kwpyH+mz6SbV5hk rzZe9/i3srPK77F47ks3Oi47H/BcPDdj10li9pgL6r2wJmK5dPZI4OmxLpj6Uhs6c3I4 X30kKzeKeJ+fUu8yDn+h4Rga2vQr00zrtAtD0B/Gg+qwMDZajdnv6DrGg4OtuC6SeW/C cyYeGXNy8SyB8ZslMFiH42Byku6PImuRCb5JqY1BJfMjO/8Zk1Ja2yKpLw6JaQg4sF9Z epVg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@meta.com header.s=s2048-2021-q4 header.b=fsDFFrmJ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=meta.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id e16-20020a170902ef5000b0019103860c7asi2590895plx.530.2022.12.16.12.19.35; Fri, 16 Dec 2022 12:19:48 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@meta.com header.s=s2048-2021-q4 header.b=fsDFFrmJ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=meta.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232052AbiLPUSK (ORCPT + 99 others); Fri, 16 Dec 2022 15:18:10 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59434 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232016AbiLPURr (ORCPT ); Fri, 16 Dec 2022 15:17:47 -0500 Received: from mx0b-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 03D6872628 for ; Fri, 16 Dec 2022 12:17:32 -0800 (PST) Received: from pps.filterd (m0109331.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2BGJxPe9015102 for ; Fri, 16 Dec 2022 12:17:32 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=meta.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=s2048-2021-q4; bh=qfJz/DDH55XtUOlIDeacYqymxZSjqIq0VpHDCFV+y+Y=; b=fsDFFrmJ5GWo97CAumjaL4HFFX7L0ymtC5kLDdM+OZo/nXZeO36EX6kGCRIAKrsBRzAy FKvEW/4AIGNjsw8DneZl9cH2P/zDevWdXhh5Rq0pjpoLEjhZnI1It8J6KxJRs231JNIP OaXpUrXsOrQ8WfBZqmbDkBZu/oH1wr5v7tq3qBkXN89HpkfMnoQYhKDVD0yHrSqTJfP7 BsHBLciV/dnQo0NH5AtF/DPXlrgpO+3P6phBFI7TRTaWn+5cmr+rEBPfPHjE4tVY3TKm zqzbgN9QRJfzqgqn6fD8PqQc8f5ct0DbkjYA3Ixd2AXlvIXlcVvnXefJ9ZGIy4LNuP8W 1g== Received: from maileast.thefacebook.com ([163.114.130.16]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3mg3hj2dga-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Fri, 16 Dec 2022 12:17:32 -0800 Received: from twshared7043.05.ash9.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:83::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Fri, 16 Dec 2022 12:17:31 -0800 Received: by devbig007.nao1.facebook.com (Postfix, from userid 544533) id 79FDAD042AD9; Fri, 16 Dec 2022 12:17:10 -0800 (PST) From: Keith Busch To: , , Matthew Wilcox CC: Tony Battersby , Kernel Team , Keith Busch Subject: [PATCHv2 11/11] dmapool: link blocks across pages Date: Fri, 16 Dec 2022 12:16:25 -0800 Message-ID: <20221216201625.2362737-12-kbusch@meta.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20221216201625.2362737-1-kbusch@meta.com> References: <20221216201625.2362737-1-kbusch@meta.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: FB9HGyNsTxDLMbWAOhQW-7MSco66prtg X-Proofpoint-ORIG-GUID: FB9HGyNsTxDLMbWAOhQW-7MSco66prtg X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-16_14,2022-12-15_02,2022-06-22_01 X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1752403267532467684?= X-GMAIL-MSGID: =?utf-8?q?1752403267532467684?= From: Keith Busch The allocated dmapool pages are never freed for the lifetime of the pool. There is no need for the two level list+stack lookup for finding a free block since nothing is ever removed from the list. Just use a simple stack, reducing time complexity to constant. The implementation inserts the stack linking elements and the dma handle of the block within itself when freed. This means the smallest possible dmapool block is increased to at most 16 bytes to accomodate these fields, but there are no exisiting users requesting a dma pool smaller than that anyway. Removing the list has a significant change in performance. Using the kernel's micro-benchmarking self test: Before: # modprobe dmapool_test dmapool test: size:16 blocks:8192 time:57282 dmapool test: size:64 blocks:8192 time:172562 dmapool test: size:256 blocks:8192 time:789247 dmapool test: size:1024 blocks:2048 time:371823 dmapool test: size:4096 blocks:1024 time:362237 After: # modprobe dmapool_test dmapool test: size:16 blocks:8192 time:24997 dmapool test: size:64 blocks:8192 time:26584 dmapool test: size:256 blocks:8192 time:33542 dmapool test: size:1024 blocks:2048 time:9022 dmapool test: size:4096 blocks:1024 time:6045 The module test allocates quite a few blocks that may not accurately represent how these pools are used in real life. For a more marco level benchmark, running fio high-depth + high-batched on nvme, this patch shows submission and completion latency reduced by ~100usec each, 1% IOPs improvement, and perf record's time spent in dma_pool_alloc/free were reduced by half. Signed-off-by: Keith Busch --- v1->v2: Applied feedback comments from Tony: Updated data structure description comment Used consistent size_t for accounting variables Fixed block initialization for odd alignments mm/dmapool.c | 215 ++++++++++++++++++++++++++------------------------- 1 file changed, 109 insertions(+), 106 deletions(-) diff --git a/mm/dmapool.c b/mm/dmapool.c index f5b79c3268856..d26a0751dee63 100644 --- a/mm/dmapool.c +++ b/mm/dmapool.c @@ -15,7 +15,7 @@ * represented by the 'struct dma_pool' which keeps a doubly-linked list of * allocated pages. Each page in the page_list is split into blocks of at * least 'size' bytes. Free blocks are tracked in an unsorted singly-linked - * list of free blocks within the page. Used blocks aren't tracked, but we + * list of free blocks across all pages. Used blocks aren't tracked, but we * keep a count of how many are currently allocated from each page. */ @@ -40,13 +40,22 @@ #define DMAPOOL_DEBUG 1 #endif +struct dma_block { + struct dma_block *next_block; + dma_addr_t dma; +}; + struct dma_pool { /* the pool */ struct list_head page_list; spinlock_t lock; struct device *dev; + struct dma_block *next_block; unsigned int size; unsigned int allocation; unsigned int boundary; + size_t nr_blocks; + size_t nr_active; + size_t nr_pages; char name[32]; struct list_head pools; }; @@ -55,8 +64,6 @@ struct dma_page { /* cacheable header for 'allocation' bytes */ struct list_head page_list; void *vaddr; dma_addr_t dma; - unsigned int in_use; - unsigned int offset; }; static DEFINE_MUTEX(pools_lock); @@ -64,30 +71,18 @@ static DEFINE_MUTEX(pools_reg_lock); static ssize_t pools_show(struct device *dev, struct device_attribute *attr, char *buf) { - int size; - struct dma_page *page; struct dma_pool *pool; + unsigned size; size = sysfs_emit(buf, "poolinfo - 0.1\n"); mutex_lock(&pools_lock); list_for_each_entry(pool, &dev->dma_pools, pools) { - unsigned pages = 0; - size_t blocks = 0; - - spin_lock_irq(&pool->lock); - list_for_each_entry(page, &pool->page_list, page_list) { - pages++; - blocks += page->in_use; - } - spin_unlock_irq(&pool->lock); - /* per-pool info, no real statistics yet */ - size += sysfs_emit_at(buf, size, "%-16s %4zu %4zu %4u %2u\n", - pool->name, blocks, - (size_t) pages * - (pool->allocation / pool->size), - pool->size, pages); + size += sysfs_emit_at(buf, size, "%-16s %4ld %4ld %4u %2ld\n", + pool->name, pool->nr_active, + pool->nr_blocks, pool->size, + pool->nr_pages); } mutex_unlock(&pools_lock); @@ -96,6 +91,25 @@ static ssize_t pools_show(struct device *dev, struct device_attribute *attr, cha static DEVICE_ATTR_RO(pools); +static inline struct dma_block *pool_block_pop(struct dma_pool *pool) +{ + struct dma_block *block = pool->next_block; + + if (block) { + pool->next_block = block->next_block; + pool->nr_active++; + } + return block; +} + +static inline void pool_block_push(struct dma_pool *pool, struct dma_block *block, + dma_addr_t dma) +{ + block->dma = dma; + block->next_block = pool->next_block; + pool->next_block = block; +} + /** * dma_pool_create - Creates a pool of consistent memory blocks, for dma. * @name: name of pool, for diagnostics @@ -136,8 +150,8 @@ struct dma_pool *dma_pool_create(const char *name, struct device *dev, if (size == 0 || size > INT_MAX) return NULL; - else if (size < 4) - size = 4; + if (size < sizeof(struct dma_block)) + size = sizeof(struct dma_block); size = ALIGN(size, align); allocation = max_t(size_t, size, PAGE_SIZE); @@ -162,6 +176,10 @@ struct dma_pool *dma_pool_create(const char *name, struct device *dev, retval->size = size; retval->boundary = boundary; retval->allocation = allocation; + retval->nr_blocks = 0; + retval->nr_active = 0; + retval->nr_pages = 0; + retval->next_block = NULL; INIT_LIST_HEAD(&retval->pools); @@ -199,22 +217,24 @@ EXPORT_SYMBOL(dma_pool_create); static void pool_initialise_page(struct dma_pool *pool, struct dma_page *page) { - unsigned int offset = 0; - unsigned int next_boundary = pool->boundary; - - page->in_use = 0; - page->offset = 0; - do { - unsigned int next = offset + pool->size; - if (unlikely((next + pool->size) >= next_boundary)) { - next = next_boundary; + unsigned int next_boundary = pool->boundary, offset = 0; + struct dma_block *block; + + while (offset + pool->size <= pool->allocation) { + if (offset + pool->size > next_boundary) { + offset = next_boundary; next_boundary += pool->boundary; + continue; } - *(int *)(page->vaddr + offset) = next; - offset = next; - } while (offset < pool->allocation); + + block = page->vaddr + offset; + pool_block_push(pool, block, page->dma + offset); + offset += pool->size; + pool->nr_blocks++; + } list_add(&page->page_list, &pool->page_list); + pool->nr_pages++; } static struct dma_page *pool_alloc_page(struct dma_pool *pool, gfp_t mem_flags) @@ -236,11 +256,6 @@ static struct dma_page *pool_alloc_page(struct dma_pool *pool, gfp_t mem_flags) return page; } -static inline bool is_page_busy(struct dma_page *page) -{ - return page->in_use != 0; -} - /** * dma_pool_destroy - destroys a pool of dma memory blocks. * @pool: dma pool that will be destroyed @@ -252,7 +267,7 @@ static inline bool is_page_busy(struct dma_page *page) void dma_pool_destroy(struct dma_pool *pool) { struct dma_page *page, *tmp; - bool empty = false; + bool empty = false, busy = false; if (unlikely(!pool)) return; @@ -267,13 +282,15 @@ void dma_pool_destroy(struct dma_pool *pool) device_remove_file(pool->dev, &dev_attr_pools); mutex_unlock(&pools_reg_lock); + if (pool->nr_active) { + dev_err(pool->dev, "%s %s busy\n", __func__, pool->name); + busy = true; + } + list_for_each_entry_safe(page, tmp, &pool->page_list, page_list) { - if (!is_page_busy(page)) + if (!busy) dma_free_coherent(pool->dev, pool->allocation, page->vaddr, page->dma); - else - dev_err(pool->dev, "%s %s, %p busy\n", __func__, - pool->name, page->vaddr); list_del(&page->page_list); kfree(page); } @@ -282,18 +299,18 @@ void dma_pool_destroy(struct dma_pool *pool) } EXPORT_SYMBOL(dma_pool_destroy); -static inline void pool_check_block(struct dma_pool *pool, void *retval, - unsigned int offset, gfp_t mem_flags) +static inline void pool_check_block(struct dma_pool *pool, struct dma_block *block, + gfp_t mem_flags) { -#ifdef DMAPOOL_DEBUG +#ifdef DMAPOOL_DEBUG + u8 *data = (void *)block; int i; - u8 *data = retval; - /* page->offset is stored in first 4 bytes */ - for (i = sizeof(offset); i < pool->size; i++) { + + for (i = sizeof(struct dma_block); i < pool->size; i++) { if (data[i] == POOL_POISON_FREED) continue; - dev_err(pool->dev, "%s %s, %p (corrupted)\n", - __func__, pool->name, retval); + dev_err(pool->dev, "%s %s, %p (corrupted)\n", __func__, + pool->name, block); /* * Dump the first 4 bytes even if they are not @@ -303,8 +320,9 @@ static inline void pool_check_block(struct dma_pool *pool, void *retval, data, pool->size, 1); break; } + if (!want_init_on_alloc(mem_flags)) - memset(retval, POOL_POISON_ALLOCATED, pool->size); + memset(block, POOL_POISON_ALLOCATED, pool->size); #endif } @@ -321,44 +339,41 @@ static inline void pool_check_block(struct dma_pool *pool, void *retval, void *dma_pool_alloc(struct dma_pool *pool, gfp_t mem_flags, dma_addr_t *handle) { - unsigned long flags; + struct dma_block *block; struct dma_page *page; - unsigned int offset; - void *retval; + unsigned long flags; might_alloc(mem_flags); spin_lock_irqsave(&pool->lock, flags); - list_for_each_entry(page, &pool->page_list, page_list) { - if (page->offset < pool->allocation) - goto ready; - } - - /* pool_alloc_page() might sleep, so temporarily drop &pool->lock */ - spin_unlock_irqrestore(&pool->lock, flags); + block = pool_block_pop(pool); + if (!block) { + /* + * pool_alloc_page() might sleep, so temporarily drop + * &pool->lock + */ + spin_unlock_irqrestore(&pool->lock, flags); - page = pool_alloc_page(pool, mem_flags & (~__GFP_ZERO)); - if (!page) - return NULL; + page = pool_alloc_page(pool, mem_flags & (~__GFP_ZERO)); + if (!page) + return NULL; - spin_lock_irqsave(&pool->lock, flags); - pool_initialise_page(pool, page); - ready: - page->in_use++; - offset = page->offset; - page->offset = *(int *)(page->vaddr + offset); - retval = offset + page->vaddr; - *handle = offset + page->dma; - pool_check_block(pool, retval, offset, mem_flags); + spin_lock_irqsave(&pool->lock, flags); + pool_initialise_page(pool, page); + block = pool_block_pop(pool); + } spin_unlock_irqrestore(&pool->lock, flags); + *handle = block->dma; + pool_check_block(pool, block, mem_flags); if (want_init_on_alloc(mem_flags)) - memset(retval, 0, pool->size); + memset(block, 0, pool->size); - return retval; + return block; } EXPORT_SYMBOL(dma_pool_alloc); +#ifdef DMAPOOL_DEBUG static struct dma_page *pool_find_page(struct dma_pool *pool, dma_addr_t dma) { struct dma_page *page; @@ -372,33 +387,35 @@ static struct dma_page *pool_find_page(struct dma_pool *pool, dma_addr_t dma) return NULL; } -#ifdef DMAPOOL_DEBUG -static inline bool pool_page_err(struct dma_pool *pool, struct dma_page *page, - void *vaddr) +static inline bool pool_block_err(struct dma_pool *pool, void *vaddr, + dma_addr_t dma) { - unsigned int chain = page->offset; + struct dma_block *block = pool->next_block; + struct dma_page *page; - if ((dma - page->dma) != offset) { - dev_err(pool->dev, "%s %s, %p (bad vaddr)/%pad\n", + page = pool_find_page(pool, dma); + if (!page) { + dev_err(pool->dev, "%s %s, %p/%pad (bad dma)\n", __func__, pool->name, vaddr, &dma); return true; } - while (chain < pool->allocation) { - if (chain != offset) { - chain = *(int *)(page->vaddr + chain); + while (block) { + if (block != vaddr) { + block = block->next_block; continue; } dev_err(pool->dev, "%s %s, dma %pad already free\n", __func__, pool->name, &dma); return true; } + memset(vaddr, POOL_POISON_FREED, pool->size); return false; } #else -static inline bool pool_page_err(struct dma_pool *pool, struct dma_page *page, - void *vaddr) +static inline bool pool_block_err(struct dma_pool *pool, void *vaddr, + dma_addr_t dma) { if (want_init_on_free()) memset(vaddr, 0, pool->size); @@ -417,28 +434,14 @@ static inline bool pool_page_err(struct dma_pool *pool, struct dma_page *page, */ void dma_pool_free(struct dma_pool *pool, void *vaddr, dma_addr_t dma) { - struct dma_page *page; + struct dma_block *block = vaddr; unsigned long flags; - unsigned int offset; spin_lock_irqsave(&pool->lock, flags); - page = pool_find_page(pool, dma); - if (!page) { - spin_unlock_irqrestore(&pool->lock, flags); - dev_err(pool->dev, "%s %s, %p/%pad (bad dma)\n", - __func__, pool->name, vaddr, &dma); - return; + if (!pool_block_err(pool, vaddr, dma)) { + pool_block_push(pool, block, dma); + pool->nr_active--; } - - offset = vaddr - page->vaddr; - if (pool_page_err(pool, page, vaddr)) { - spin_unlock_irqrestore(&pool->lock, flags); - return; - } - - page->in_use--; - *(int *)vaddr = page->offset; - page->offset = offset; spin_unlock_irqrestore(&pool->lock, flags); } EXPORT_SYMBOL(dma_pool_free);