From patchwork Mon Jun 26 13:01:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Petr Tesarik X-Patchwork-Id: 112944 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp7468854vqr; Mon, 26 Jun 2023 06:07:37 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4wxtA9qgjEXUceAFT6eO+oE6vOiXAOggWlzzCqjAvvE3RDqWJZwbA9x7N9fu12xZma2DlJ X-Received: by 2002:a17:907:3ea3:b0:977:4b19:97a with SMTP id hs35-20020a1709073ea300b009774b19097amr27696053ejc.73.1687784856808; Mon, 26 Jun 2023 06:07:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1687784856; cv=none; d=google.com; s=arc-20160816; b=LPP0ICjwRr91VqLKsSueXhPlseRO4oxvC07T5QbASxDR9qt833vASFXMbtw5i8ypDe WT1U6Gpp38jBfJccF0EqQsHKU205ghDJMWx836wo/gZOdyRK76+ZmcdM3iodeuddlS6S E76+UUPmw2Lis2y7zvsOBDH1BD124g3yVFZ9dq3xqX3J7BhKuA/3yxntZS7owmc9qLbP deCzmFx9Qbl5CikkN3YWU+61CFTndq7Dhu2a98xSvqnRmgmPpxWh2vmBxCQ9UH2eVm8n tyo91TptAtYurMTK/FmzS2f2bRT1rqU8cWrDsg/lA34/M4sfdun6flj45dup2YKe25Cv e2vg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=vjMQDbflY2l7p53mbBYfRrkRy2Uk6s0zDSlMAwm0Tvs=; fh=Pm5qlDXZMCeI+BDi8trv5j6CRr9ePSM+XXpN7cigZa4=; b=H+UnPs5Y5tH8gRGZQMGaHy2iTpav/pV7diYaoXwwJJpLwiKyZViM7xFA50hDJYdvVZ /dN7J8WRIW8REoiYqKrs7NTDIVxUT7u2/E5LZn+wdPi92foyZUzu6js4dFqjG5f3Cvc+ sQDD5d7JCnpmVSo/0p5kf8udOwpTlZ1/pnXbfUxOS3zREGmSAl0SiwvdvdDxeP4UFKd1 lme/VDzSlrN8HuT63rdxavEUSUvEYAyjH7CrBt0PbTxIQmmQBSBpNAstsshy4CTiwVjq whm5v3xoQ674Hra0jDmN2DhxPMcU3cwr9plWOwAjXvnOOrlc8r2jaHYuqZ09HB1NhUaV DsNA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id z21-20020a1709067e5500b0098e39d98115si1787653ejr.174.2023.06.26.06.07.09; Mon, 26 Jun 2023 06:07:36 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230436AbjFZNCW (ORCPT + 99 others); Mon, 26 Jun 2023 09:02:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52162 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231310AbjFZNCK (ORCPT ); Mon, 26 Jun 2023 09:02:10 -0400 Received: from frasgout11.his.huawei.com (unknown [14.137.139.23]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7917FC2 for ; Mon, 26 Jun 2023 06:02:05 -0700 (PDT) Received: from mail02.huawei.com (unknown [172.18.147.228]) by frasgout11.his.huawei.com (SkyGuard) with ESMTP id 4QqSQB5PXjz9xFQ7 for ; Mon, 26 Jun 2023 20:51:14 +0800 (CST) Received: from A2101119013HW2.china.huawei.com (unknown [10.81.202.159]) by APP2 (Coremail) with SMTP id GxC2BwDXXWQljJlk1fGzAw--.36557S4; Mon, 26 Jun 2023 14:01:46 +0100 (CET) From: Petr Tesarik To: Christoph Hellwig , Marek Szyprowski , Robin Murphy , iommu@lists.linux.dev (open list:DMA MAPPING HELPERS), linux-kernel@vger.kernel.org (open list) Cc: Roberto Sassu , Kefeng Wang , petr@tesarici.cz Subject: [PATCH v1 2/2] swiotlb: Reduce the number of areas to match actual memory pool size Date: Mon, 26 Jun 2023 15:01:04 +0200 Message-Id: <097dffeb406cef6596499c8dbdafb28d05f7501b.1687784289.git.petr.tesarik.ext@huawei.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 X-CM-TRANSID: GxC2BwDXXWQljJlk1fGzAw--.36557S4 X-Coremail-Antispam: 1UD129KBjvJXoWxZw1xur4UKr1xCF18GFyrCrg_yoWrGFy3pr yfWa4UtFWFqFn7CFW2y3y8CFySka4vyrnFkF9xC3sxXw1UGF1SkrWqkFWUJFWrtF4kuF4f X3yrZF45uanxJ3DanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUBG14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_Jryl82xGYIkIc2 x26xkF7I0E14v26r4j6ryUM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2z4x0 Y4vE2Ix0cI8IcVAFwI0_Jr0_JF4l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr0_Cr1l84 ACjcxK6I8E87Iv67AKxVW8JVWxJwA2z4x0Y4vEx4A2jsIEc7CjxVAFwI0_Gr0_Gr1UM2AI xVAIcxkEcVAq07x20xvEncxIr21l5I8CrVACY4xI64kE6c02F40Ex7xfMcIj6xIIjxv20x vE14v26r1j6r18McIj6I8E87Iv67AKxVWUJVW8JwAm72CE4IkC6x0Yz7v_Jr0_Gr1lF7xv r2IYc2Ij64vIr41lF7I21c0EjII2zVCS5cI20VAGYxC7MxkF7I0Ew4C26cxK6c8Ij28Icw CF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E14v26r1j 6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI0_Jw0_GFylIxkGc2Ij64 vIr41lIxAIcVC0I7IYx2IY67AKxVWUJVWUCwCI42IY6xIIjxv20xvEc7CjxVAFwI0_Gr0_ Cr1lIxAIcVCF04k26cxKx2IYs7xG6r1j6r1xMIIF0xvEx4A2jsIE14v26r1j6r4UMIIF0x vEx4A2jsIEc7CjxVAFwI0_Gr0_Gr1UYxBIdaVFxhVjvjDU0xZFpf9x0pRLqXdUUUUU= X-CM-SenderInfo: hshw23xhvd2x3n6k3tpzhluzxrxghudrp/ X-CFilter-Loop: Reflected X-Spam-Status: No, score=-0.9 required=5.0 tests=BAYES_00,KHOP_HELO_FCRDNS, MAY_BE_FORGED,PDS_RDNS_DYNAMIC_FP,RDNS_DYNAMIC,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1769770694165080507?= X-GMAIL-MSGID: =?utf-8?q?1769770694165080507?= From: Petr Tesarik Although the desired size of the SWIOTLB memory pool is increased in swiotlb_adjust_nareas() to match the number of areas, the actual allocation may be smaller, which may require reducing the number of areas. For example, Xen uses swiotlb_init_late(), which in turn uses the page allocator. On x86, page size is 4 KiB and MAX_ORDER is 10 (1024 pages), resulting in a maximum memory pool size of 4 MiB. This corresponds to 2048 slots of 2 KiB each. The minimum area size is 128 (IO_TLB_SEGSIZE), allowing at most 2048 / 128 = 16 areas. If num_possible_cpus() is greater than the maximum number of areas, areas are smaller than IO_TLB_SEGSIZE and contiguous groups of free slots will span multiple areas. When allocating and freeing slots, only one area will be properly locked, causing race conditions on the unlocked slots and ultimately data corruption, kernel hangs and crashes. Fixes: 20347fca71a3 ("swiotlb: split up the global swiotlb lock") Signed-off-by: Petr Tesarik --- kernel/dma/swiotlb.c | 27 ++++++++++++++++++++++++--- 1 file changed, 24 insertions(+), 3 deletions(-) diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c index 16f53d8c51bc..079df5ad38d0 100644 --- a/kernel/dma/swiotlb.c +++ b/kernel/dma/swiotlb.c @@ -138,6 +138,23 @@ static void swiotlb_adjust_nareas(unsigned int nareas) (default_nslabs << IO_TLB_SHIFT) >> 20); } +/** + * limit_nareas() - get the maximum number of areas for a given memory pool size + * @nareas: Desired number of areas. + * @nslots: Total number of slots in the memory pool. + * + * Limit the number of areas to the maximum possible number of areas in + * a memory pool of the given size. + * + * Return: Maximum possible number of areas. + */ +static unsigned int limit_nareas(unsigned int nareas, unsigned long nslots) +{ + if (nslots < nareas * IO_TLB_SEGSIZE) + nareas = nslots / IO_TLB_SEGSIZE; + return nareas; +} + static int __init setup_io_tlb_npages(char *str) { @@ -297,6 +314,7 @@ void __init swiotlb_init_remap(bool addressing_limit, unsigned int flags, { struct io_tlb_mem *mem = &io_tlb_default_mem; unsigned long nslabs; + unsigned int nareas; size_t alloc_size; void *tlb; @@ -309,10 +327,12 @@ void __init swiotlb_init_remap(bool addressing_limit, unsigned int flags, swiotlb_adjust_nareas(num_possible_cpus()); nslabs = default_nslabs; + nareas = limit_nareas(default_nareas, nslabs); while ((tlb = swiotlb_memblock_alloc(nslabs, flags, remap)) == NULL) { if (nslabs <= IO_TLB_MIN_SLABS) return; nslabs = ALIGN(nslabs >> 1, IO_TLB_SEGSIZE); + nareas = limit_nareas(nareas, nslabs); } if (default_nslabs != nslabs) { @@ -358,6 +378,7 @@ int swiotlb_init_late(size_t size, gfp_t gfp_mask, { struct io_tlb_mem *mem = &io_tlb_default_mem; unsigned long nslabs = ALIGN(size >> IO_TLB_SHIFT, IO_TLB_SEGSIZE); + unsigned int nareas; unsigned char *vstart = NULL; unsigned int order, area_order; bool retried = false; @@ -403,8 +424,8 @@ int swiotlb_init_late(size_t size, gfp_t gfp_mask, (PAGE_SIZE << order) >> 20); } - area_order = get_order(array_size(sizeof(*mem->areas), - default_nareas)); + nareas = limit_nareas(default_nareas, nslabs); + area_order = get_order(array_size(sizeof(*mem->areas), nareas)); mem->areas = (struct io_tlb_area *) __get_free_pages(GFP_KERNEL | __GFP_ZERO, area_order); if (!mem->areas) @@ -418,7 +439,7 @@ int swiotlb_init_late(size_t size, gfp_t gfp_mask, set_memory_decrypted((unsigned long)vstart, (nslabs << IO_TLB_SHIFT) >> PAGE_SHIFT); swiotlb_init_io_tlb_mem(mem, virt_to_phys(vstart), nslabs, 0, true, - default_nareas); + nareas); swiotlb_print_info(); return 0;