From patchwork Wed Apr 19 10:03:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Petr Tesarik X-Patchwork-Id: 85303 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp274368vqo; Wed, 19 Apr 2023 03:24:24 -0700 (PDT) X-Google-Smtp-Source: AKy350b6/QPvAsc7JhQrMpWQEqWhDGTM0DxM7sDuv53hT4c7Tf6mEM+A5J8dZLHWALP4ygGDLbAW X-Received: by 2002:a05:6a00:2305:b0:63d:3411:f9eb with SMTP id h5-20020a056a00230500b0063d3411f9ebmr3386418pfh.1.1681899864386; Wed, 19 Apr 2023 03:24:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681899864; cv=none; d=google.com; s=arc-20160816; b=z99a9fyFO8CeVw3XnMDzBSf4hcJPxswowd01FkorJejtQRI2D7mlfkzmfvEALsImdO gR81W/R7/tJu7yHNa5tdCNenyhLWFdkityR3hJejYMRgXEHx3vaqfIjG7Cad69OKH0YE FGT6j93KnLJA6cYUAPIlkTIn5mKUFo7+2WHUclQ6cdjgROKujnKB6xIj2zDVE8aWsBF9 nVqcy6X69Rf0Mgt4kPUxSQQrJF6BzWt5/NhmVp3R8ihFXH37uWlF7HT6lhFhkjNacZ0A kjZrzsMDCESYxh8otU4TYePADOKQr5NL7Egi5raAYcp8frmhaELuuG/oMaoMat7fJh61 SvRQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=U9mWzwFUnbp/5A+eD/y/ICit7IFdi18a7mivtaU7+IM=; b=uNaQ1Pss+hMPDzpaDQCyz6dB1nn9CGL5aMtHxiPqfUV/Dn+azxT5xGrF7tZ1zTfn2r zl/qPu4BiXEj0KHZxRKw0Unor504EDnC3Lw/bgPtSx96PXizi/wXlVwwoFlpT2Waj2qN GapBi3SGZmDTtMUswksviqPsfAwb9BxOLb7w/22ilpUO+O6LIahplCZpjLeYR2LkNqBj HZQVeuFo+ilArlkk+E8Ak4j3rhkPRhJeNxOYnG/JZTaiE7AnhslXeXfseyg+K62LhjHi kkIxwwVVUgYL5z+IfRusTq3Gi9P1D6a9n4TEV7UI0eFuPvyZ/zI8jgWSSnTq/0ZBk8jh Q87Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id a25-20020aa795b9000000b0063d5fffd8b1si1033229pfk.228.2023.04.19.03.24.12; Wed, 19 Apr 2023 03:24:24 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233088AbjDSKIS (ORCPT + 99 others); Wed, 19 Apr 2023 06:08:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33572 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233065AbjDSKHe (ORCPT ); Wed, 19 Apr 2023 06:07:34 -0400 Received: from frasgout11.his.huawei.com (frasgout11.his.huawei.com [14.137.139.23]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 27A5F14443; Wed, 19 Apr 2023 03:07:26 -0700 (PDT) Received: from mail02.huawei.com (unknown [172.18.147.227]) by frasgout11.his.huawei.com (SkyGuard) with ESMTP id 4Q1bnX3fTjz9v7V0; Wed, 19 Apr 2023 17:57:52 +0800 (CST) Received: from A2101119013HW2.china.huawei.com (unknown [10.45.152.239]) by APP1 (Coremail) with SMTP id LxC2BwAXi_aYvD9k7TY6Ag--.1944S9; Wed, 19 Apr 2023 11:06:42 +0100 (CET) From: Petr Tesarik To: Jonathan Corbet , Greg Kroah-Hartman , "Rafael J. Wysocki" , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , David Airlie , Daniel Vetter , Christoph Hellwig , Marek Szyprowski , Robin Murphy , Borislav Petkov , "Paul E. McKenney" , Catalin Marinas , Will Deacon , Zhen Lei , Randy Dunlap , Damien Le Moal , Kim Phillips , "Steven Rostedt (Google)" , Muchun Song , Ondrej Zary , "Jason A. Donenfeld" , Petr Tesarik , Hans de Goede , Dan Williams , Andy Shevchenko , Kees Cook , Thomas Gleixner , Won Chung , linux-doc@vger.kernel.org (open list:DOCUMENTATION), linux-kernel@vger.kernel.org (open list), dri-devel@lists.freedesktop.org (open list:DRM DRIVERS), iommu@lists.linux.dev (open list:DMA MAPPING HELPERS) Cc: Roberto Sassu , Kefeng Wang , petr@tesarici.cz Subject: [PATCH v2 7/7] swiotlb: per-device flag if there are dynamically allocated buffers Date: Wed, 19 Apr 2023 12:03:59 +0200 Message-Id: <08ec2c2baa9a24d2eae5d5107868fd6df0f7dd1c.1681898595.git.petr.tesarik.ext@huawei.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 X-CM-TRANSID: LxC2BwAXi_aYvD9k7TY6Ag--.1944S9 X-Coremail-Antispam: 1UD129KBjvJXoWxWr1UWr47Jw1kZw4UKF48Zwb_yoW5uFW3pF y8uF98KF4qqFykZ3sF9w47uF47uw4q93y3CrWFgr1Fkry5X34rWF4kCry2y34rJr409F4x XryjvrWrAF17Ww7anT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUmEb4IE77IF4wAFF20E14v26rWj6s0DM7CY07I20VC2zVCF04k2 6cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M28IrcIa0xkI8VA2jI8067AKxVWUAV Cq3wA2048vs2IY020Ec7CjxVAFwI0_Xr0E3s1l8cAvFVAK0II2c7xJM28CjxkF64kEwVA0 rcxSw2x7M28EF7xvwVC0I7IYx2IY67AKxVW8JVW5JwA2z4x0Y4vE2Ix0cI8IcVCY1x0267 AKxVW8Jr0_Cr1UM28EF7xvwVC2z280aVAFwI0_Gr0_Cr1l84ACjcxK6I8E87Iv6xkF7I0E 14v26r4UJVWxJr1le2I262IYc4CY6c8Ij28IcVAaY2xG8wAqx4xG64xvF2IEw4CE5I8CrV C2j2WlYx0E2Ix0cI8IcVAFwI0_Jr0_Jr4lYx0Ex4A2jsIE14v26r1j6r4UMcvjeVCFs4IE 7xkEbVWUJVW8JwACjcxG0xvY0x0EwIxGrwACI402YVCY1x02628vn2kIc2xKxwCY1x0262 kKe7AKxVWUtVW8ZwCY1x0264kExVAvwVAq07x20xyl42xK82IYc2Ij64vIr41l4I8I3I0E 4IkC6x0Yz7v_Jr0_Gr1lx2IqxVAqx4xG67AKxVWUJVWUGwC20s026x8GjcxK67AKxVWUGV WUWwC2zVAF1VAY17CE14v26rWY6r4UJwCIc40Y0x0EwIxGrwCI42IY6xIIjxv20xvE14v2 6r1I6r4UMIIF0xvE2Ix0cI8IcVCY1x0267AKxVW8Jr0_Cr1UMIIF0xvE42xK8VAvwI8IcI k0rVWUJVWUCwCI42IY6I8E87Iv67AKxVW8JVWxJwCI42IY6I8E87Iv6xkF7I0E14v26r4U JVWxJrUvcSsGvfC2KfnxnUUI43ZEXa7IU8yxR3UUUUU== X-CM-SenderInfo: hshw23xhvd2x3n6k3tpzhluzxrxghudrp/ X-CFilter-Loop: Reflected X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1763599832306720935?= X-GMAIL-MSGID: =?utf-8?q?1763599832306720935?= From: Petr Tesarik Do not walk the list of dynamically allocated bounce buffers if the list is empty. This avoids taking dma_io_tlb_dyn_lock for devices which do not use any dynamically allocated bounce buffers. When unmapping the last dynamically allocated bounce buffer, the flag is set to false as soon as possible to allow skipping the spinlock even before the list itself is updated. Signed-off-by: Petr Tesarik --- include/linux/device.h | 4 ++++ include/linux/swiotlb.h | 6 +++++- kernel/dma/swiotlb.c | 6 ++++++ 3 files changed, 15 insertions(+), 1 deletion(-) diff --git a/include/linux/device.h b/include/linux/device.h index e12d6092bb9c..131b6db7fb3f 100644 --- a/include/linux/device.h +++ b/include/linux/device.h @@ -511,6 +511,9 @@ struct device_physical_location { * @dma_io_tlb_dyn_slots: * Dynamically allocated bounce buffers for this device. * Not for driver use. + * @dma_io_tlb_have_dyn: + * Does this device have any dynamically allocated bounce + * buffers? Not for driver use. * @archdata: For arch-specific additions. * @of_node: Associated device tree node. * @fwnode: Associated device node supplied by platform firmware. @@ -618,6 +621,7 @@ struct device { struct io_tlb_mem *dma_io_tlb_mem; spinlock_t dma_io_tlb_dyn_lock; struct list_head dma_io_tlb_dyn_slots; + bool dma_io_tlb_have_dyn; #endif /* arch specific additions */ struct dev_archdata archdata; diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h index e614aa0f4f64..9ca4812b5977 100644 --- a/include/linux/swiotlb.h +++ b/include/linux/swiotlb.h @@ -143,7 +143,11 @@ static inline bool is_swiotlb_buffer(struct device *dev, phys_addr_t paddr) return mem && (is_swiotlb_fixed(mem, paddr) || - (mem->allow_dyn && is_swiotlb_dyn(dev, paddr))); + /* Pairs with smp_store_release() in swiotlb_dyn_map() + * and swiotlb_dyn_unmap(). + */ + (smp_load_acquire(&dev->dma_io_tlb_have_dyn) && + is_swiotlb_dyn(dev, paddr))); } static inline bool is_swiotlb_force_bounce(struct device *dev) diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c index 4899fb0e4331..9b4faed7ef8f 100644 --- a/kernel/dma/swiotlb.c +++ b/kernel/dma/swiotlb.c @@ -685,6 +685,9 @@ static phys_addr_t swiotlb_dyn_map(struct device *dev, phys_addr_t orig_addr, spin_lock_irqsave(&dev->dma_io_tlb_dyn_lock, flags); list_add(&slot->node, &dev->dma_io_tlb_dyn_slots); + if (!dev->dma_io_tlb_have_dyn) + /* Pairs with smp_load_acquire() in is_swiotlb_buffer() */ + smp_store_release(&dev->dma_io_tlb_have_dyn, true); spin_unlock_irqrestore(&dev->dma_io_tlb_dyn_lock, flags); return page_to_phys(slot->page); @@ -711,6 +714,9 @@ static void swiotlb_dyn_unmap(struct device *dev, phys_addr_t tlb_addr, unsigned long flags; spin_lock_irqsave(&dev->dma_io_tlb_dyn_lock, flags); + if (list_is_singular(&dev->dma_io_tlb_dyn_slots)) + /* Pairs with smp_load_acquire() in is_swiotlb_buffer() */ + smp_store_release(&dev->dma_io_tlb_have_dyn, false); slot = lookup_dyn_slot_locked(dev, tlb_addr); list_del(&slot->node); spin_unlock_irqrestore(&dev->dma_io_tlb_dyn_lock, flags);