From patchwork Tue May 9 09:18:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Petr Tesarik X-Patchwork-Id: 9072 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp2738233vqo; Tue, 9 May 2023 02:25:20 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4TBGhaN2iLFMAAMZ+4hOkt9XhRSnzAA9BQn5SMPIMQ0R/j4oNFDbXcZtHGH5pMzMrsvkJF X-Received: by 2002:a17:903:22cb:b0:1ac:6e1f:d19c with SMTP id y11-20020a17090322cb00b001ac6e1fd19cmr9180229plg.41.1683624320072; Tue, 09 May 2023 02:25:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683624320; cv=none; d=google.com; s=arc-20160816; b=dQnUQQc1TFGQQvCnMJoG1HA4pxhIIy2N7w4cVo+B7bhkXFCAPkd4yjLkYVK0A/NkbR eqHNdWZt+qBlNAt0jzIssb0FiY2omMikywECuvG5yA7BwvlS92nNUHI27wzhY6YEej95 o7YZOEmka1vfiOgomQHARliSSMmMK2rwX7TKZXrfyFQmK7b6NGq0dJw13fiElj1vLScD 5dQMwc/PvaqnDs6rY0OG++FU6Rjh02qnwZvgoFkTmvjxgu9J4ekiOPrsO87sUvTznMW6 ZA88Wgdz3Jp8JhHYKGadx0gTB3Z25YzZYxy+hhG8YLvwozt4nIXxznKzF8pEM1si6dcd QwcA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from; bh=Z+gr864t1NeEPZVkyraRbNNpRxBJi321FpQnhXUDLmE=; b=VuQLTylThTTBnn3nH3bn+eTALU+wWR3JFjdJdw4VJx7ssaO+l5YwRYsRPBtAjtZfUa WQSM4ifS1aN8DSWReexOyBtHLQSwT9GUzSom5duO5f6zrgOz921MRCPulzI6JZFIyuew FFvlBYHCzEMr/ARdeVOwfSW8dZyyl56ypSqd+6ZB2RnCqksYHgEVy695vsYZaQ8WK3XC GUshGD8v2Aj2d0k82Lvg0l5SqZ69bLfNiikY7LT1OiyYNkuKwR1J9vg4s2dDRWSAn1fE hn9uWPNjDbGkxKpfipjnRcklFJr2UBoObwkLlS62tQyPdQIaqIfVZdE228w+BmgZLh5b bmPg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id jd4-20020a170903260400b001aaff31bcccsi1084711plb.124.2023.05.09.02.25.07; Tue, 09 May 2023 02:25:20 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234832AbjEIJTc (ORCPT + 99 others); Tue, 9 May 2023 05:19:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41518 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234569AbjEIJT3 (ORCPT ); Tue, 9 May 2023 05:19:29 -0400 Received: from frasgout12.his.huawei.com (frasgout12.his.huawei.com [14.137.139.154]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8651019A3; Tue, 9 May 2023 02:19:27 -0700 (PDT) Received: from mail02.huawei.com (unknown [172.18.147.229]) by frasgout12.his.huawei.com (SkyGuard) with ESMTP id 4QFsl72m3gz9xFrw; Tue, 9 May 2023 17:08:19 +0800 (CST) Received: from A2101119013HW2.china.huawei.com (unknown [10.81.212.219]) by APP2 (Coremail) with SMTP id GxC2BwDHdz7lD1pksN2OAg--.5S2; Tue, 09 May 2023 10:18:45 +0100 (CET) From: Petr Tesarik To: Jonathan Corbet , Greg Kroah-Hartman , "Rafael J. Wysocki" , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , David Airlie , Daniel Vetter , Christoph Hellwig , Marek Szyprowski , Robin Murphy , "Paul E. McKenney" , Borislav Petkov , Randy Dunlap , Catalin Marinas , Damien Le Moal , Kim Phillips , "Steven Rostedt (Google)" , Andy Shevchenko , Hans de Goede , Jason Gunthorpe , Kees Cook , Thomas Gleixner , linux-doc@vger.kernel.org (open list:DOCUMENTATION), linux-kernel@vger.kernel.org (open list), dri-devel@lists.freedesktop.org (open list:DRM DRIVERS), iommu@lists.linux.dev (open list:DMA MAPPING HELPERS) Cc: Roberto Sassu , Kefeng Wang , petr@tesarici.cz Subject: [PATCH v2 RESEND 0/7] Allow dynamic allocation of software IO TLB bounce buffers Date: Tue, 9 May 2023 11:18:12 +0200 Message-Id: X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-CM-TRANSID: GxC2BwDHdz7lD1pksN2OAg--.5S2 X-Coremail-Antispam: 1UD129KBjvJXoWxJw18WF1rGrW5WFyrWF1xKrg_yoW5GrWfpF WYk34jvrn8tryxu3yxCr4xWa4rGan5Zay3Ga9Yvrn5ZFW5Gr9FvwnrtrW5J3s8Cr4xXF4F qr1qvr15CFyfuaDanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUvSb4IE77IF4wAFF20E14v26rWj6s0DM7CY07I20VC2zVCF04k2 6cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M28lY4IEw2IIxxk0rwA2F7IY1VAKz4 vEj48ve4kI8wA2z4x0Y4vE2Ix0cI8IcVAFwI0_Jr0_JF4l84ACjcxK6xIIjxv20xvEc7Cj xVAFwI0_Gr0_Cr1l84ACjcxK6I8E87Iv67AKxVW8JVWxJwA2z4x0Y4vEx4A2jsIEc7CjxV AFwI0_Gr0_Gr1UM2AIxVAIcxkEcVAq07x20xvEncxIr21l5I8CrVACY4xI64kE6c02F40E x7xfMcIj6xIIjxv20xvE14v26r1j6r18McIj6I8E87Iv67AKxVWUJVW8JwAm72CE4IkC6x 0Yz7v_Jr0_Gr1lF7xvr2IYc2Ij64vIr41lFIxGxcIEc7CjxVA2Y2ka0xkIwI1lc7CjxVAK zI0EY4vE52x082I5MxAIw28IcxkI7VAKI48JMxC20s026xCaFVCjc4AY6r1j6r4UMI8I3I 0E5I8CrVAFwI0_Jr0_Jr4lx2IqxVCjr7xvwVAFwI0_JrI_JrWlx4CE17CEb7AF67AKxVWr XVW8Jr1lIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVWUJVWUCwCI42IY6xIIjxv20x vEc7CjxVAFwI0_Gr0_Cr1lIxAIcVCF04k26cxKx2IYs7xG6rW3Jr0E3s1lIxAIcVC2z280 aVAFwI0_Jr0_Gr1lIxAIcVC2z280aVCY1x0267AKxVW8JVW8JrUvcSsGvfC2KfnxnUUI43 ZEXa7IU86BT5UUUUU== X-CM-SenderInfo: hshw23xhvd2x3n6k3tpzhluzxrxghudrp/ X-CFilter-Loop: Reflected X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RCVD_IN_MSPIKE_H2, SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1765408055331775925?= X-GMAIL-MSGID: =?utf-8?q?1765408055331775925?= From: Petr Tesarik The goal of my work is to provide more flexibility in the sizing of SWIOTLB. The software IO TLB was designed with these assumptions: 1. It would not be used much, especially on 64-bit systems. 2. A small fixed memory area (64 MiB by default) is sufficient to handle the few cases which require a bounce buffer. 3. 64 MiB is little enough that it has no impact on the rest of the system. First, if SEV is active, all DMA must be done through shared unencrypted pages, and SWIOTLB is used to make this happen without changing device drivers. The software IO TLB size is increased to 6% of total memory in sev_setup_arch(), but that is more of an approximation. The actual requirements may vary depending on the amount of I/O and which drivers are used. These factors may not be know at boot time, i.e. when SWIOTLB is allocated. Second, other colleagues have noticed that they can reliably get rid of occasional OOM kills on an Arm embedded device by reducing the SWIOTLB size. This can be achieved with a kernel parameter, but determining the right value puts additional burden on pre-release testing, which could be avoided if SWIOTLB is allocated small and grows only when necessary. Changes from RFC: - Track dynamic buffers per device instead of per swiotlb - Use a linked list instead of a maple tree - Move initialization of swiotlb fields of struct device to a helper function - Rename __lookup_dyn_slot() to lookup_dyn_slot_locked() - Introduce per-device flag if dynamic buffers are in use - Add one more user of DMA_ATTR_MAY_SLEEP - Add kernel-doc comments for new (and some old) code - Properly escape '*' in dma-attributes.rst Petr Tesarik (7): swiotlb: Use a helper to initialize swiotlb fields in struct device swiotlb: Move code around in preparation for dynamic bounce buffers dma-mapping: introduce the DMA_ATTR_MAY_SLEEP attribute swiotlb: Dynamically allocated bounce buffers swiotlb: Add a boot option to enable dynamic bounce buffers drm: Use DMA_ATTR_MAY_SLEEP from process context swiotlb: per-device flag if there are dynamically allocated buffers .../admin-guide/kernel-parameters.txt | 6 +- Documentation/core-api/dma-attributes.rst | 10 + drivers/base/core.c | 4 +- drivers/gpu/drm/drm_gem_shmem_helper.c | 2 +- drivers/gpu/drm/drm_prime.c | 2 +- include/linux/device.h | 12 + include/linux/dma-mapping.h | 6 + include/linux/swiotlb.h | 54 ++- kernel/dma/swiotlb.c | 382 ++++++++++++++++-- 9 files changed, 443 insertions(+), 35 deletions(-)