From patchwork Wed Apr 19 10:03:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Petr Tesarik X-Patchwork-Id: 8418 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp264599vqo; Wed, 19 Apr 2023 03:05:54 -0700 (PDT) X-Google-Smtp-Source: AKy350afOazCpjWTuu8UMX7q4MHLM8R88UyHA2188HFf8G/+dVrPCe/rTatlW5iCb9vmqdsc1Edh X-Received: by 2002:a17:90a:bc8e:b0:249:6812:7ce1 with SMTP id x14-20020a17090abc8e00b0024968127ce1mr2178796pjr.25.1681898753926; Wed, 19 Apr 2023 03:05:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681898753; cv=none; d=google.com; s=arc-20160816; b=miwJTTmHNMBRNxzaSFaWQsgmT3hDx+zlrkkW7X2k82Waw30zljnBNyCxdoBfEJ+BUq Chyh/+L+0vaHHsPb+tWl/EHdVSbJF2FUf4NB2DEID7keSOS6NeZe4HwlJudbk4y92HMS xj8OESPQgwvRpHTWqZrAPhjTrJic97wVBjgw7xocAgOK+PqhUScm/XJUIl9I0k8m6B1b rBAHJqlD7SlJSRo0bGyiX9WwvQjp0cpVT6SksK1xvfYRCYijKwu+X9ctOrsqPC/i1jYg +Iw93FGgPjQ5a/V8eZnxn6xRwPOXfFjbHrZlmx81bJZKoW2+KlbcL1AB0vTi2PJ7MCyc TDQQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from; bh=65geUFpZ0MS4tM/4GeyYBoAg6MG8jijsWv9zwjZ5xaU=; b=cPTTwt4sahsOS378UoqS9konH5vLgTGObpAHUg6RQf7Vl2MuKkPBy2WsTgYnVZya80 8CVUxakAGVpq3J/Jrv76YQTp33zs3L7y5TLkuI+3lJMlpl/1R7n2clBCcGNSonYQfwfZ FU4fbwxiR+67NKSFRFbhER4+kUMhgl5zq/iyvq1wD1PPUaiy42Lygh9iE6NOUAJXee6g NCs0eDwphYn4HmqskpeBN6NhBAfk6XFP2QcYyIDcQoH3tEZ1fSAhMUMcHHwXCdxthdm/ 5FUi4vKLsK3TS5utSIw5ROPKeGIgBN7TXZEvDiAf++QY+qZ++JW1JohNFJRLpS7Pjkaa R7EQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id y17-20020a17090aa41100b0023f000be67dsi1486599pjp.13.2023.04.19.03.05.38; Wed, 19 Apr 2023 03:05:53 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231620AbjDSKFR (ORCPT + 99 others); Wed, 19 Apr 2023 06:05:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58998 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229850AbjDSKFP (ORCPT ); Wed, 19 Apr 2023 06:05:15 -0400 Received: from frasgout13.his.huawei.com (frasgout13.his.huawei.com [14.137.139.46]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8F542E7; Wed, 19 Apr 2023 03:05:13 -0700 (PDT) Received: from mail02.huawei.com (unknown [172.18.147.229]) by frasgout13.his.huawei.com (SkyGuard) with ESMTP id 4Q1bl13kmqz9xFHH; Wed, 19 Apr 2023 17:55:41 +0800 (CST) Received: from A2101119013HW2.china.huawei.com (unknown [10.45.152.239]) by APP1 (Coremail) with SMTP id LxC2BwAXi_aYvD9k7TY6Ag--.1944S2; Wed, 19 Apr 2023 11:04:29 +0100 (CET) From: Petr Tesarik To: Jonathan Corbet , Greg Kroah-Hartman , "Rafael J. Wysocki" , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , David Airlie , Daniel Vetter , Christoph Hellwig , Marek Szyprowski , Robin Murphy , Borislav Petkov , "Paul E. McKenney" , Catalin Marinas , Will Deacon , Zhen Lei , Randy Dunlap , Damien Le Moal , Kim Phillips , "Steven Rostedt (Google)" , Muchun Song , Ondrej Zary , "Jason A. Donenfeld" , Petr Tesarik , Hans de Goede , Dan Williams , Andy Shevchenko , Kees Cook , Thomas Gleixner , Won Chung , linux-doc@vger.kernel.org (open list:DOCUMENTATION), linux-kernel@vger.kernel.org (open list), dri-devel@lists.freedesktop.org (open list:DRM DRIVERS), iommu@lists.linux.dev (open list:DMA MAPPING HELPERS) Cc: Roberto Sassu , Kefeng Wang , petr@tesarici.cz Subject: [PATCH v2 0/7] Allow dynamic allocation of software IO TLB bounce buffers Date: Wed, 19 Apr 2023 12:03:52 +0200 Message-Id: X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-CM-TRANSID: LxC2BwAXi_aYvD9k7TY6Ag--.1944S2 X-Coremail-Antispam: 1UD129KBjvJXoWxJw18WF1rGrW5WFyrCry5CFg_yoW5Ww15pF Wak34jvrn8tryxu3yxCr4xWa4rGan5ZFW7Ga9Yvwn5ZrW5urn2vw12yrW3J3s8Gr4fXF4Y qF1qvr15CFyrur7anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUvlb4IE77IF4wAFF20E14v26rWj6s0DM7CY07I20VC2zVCF04k2 6cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M28lY4IEw2IIxxk0rwA2F7IY1VAKz4 vEj48ve4kI8wA2z4x0Y4vE2Ix0cI8IcVAFwI0_Jr0_JF4l84ACjcxK6xIIjxv20xvEc7Cj xVAFwI0_Cr0_Gr1UM28EF7xvwVC2z280aVAFwI0_Gr0_Cr1l84ACjcxK6I8E87Iv6xkF7I 0E14v26r4UJVWxJr1le2I262IYc4CY6c8Ij28IcVAaY2xG8wAqx4xG64xvF2IEw4CE5I8C rVC2j2WlYx0E2Ix0cI8IcVAFwI0_Jr0_Jr4lYx0Ex4A2jsIE14v26r1j6r4UMcvjeVCFs4 IE7xkEbVWUJVW8JwACjcxG0xvY0x0EwIxGrwACI402YVCY1x02628vn2kIc2xKxwCY1x02 64kExVAvwVAq07x20xyl42xK82IYc2Ij64vIr41l4I8I3I0E4IkC6x0Yz7v_Jr0_Gr1lx2 IqxVAqx4xG67AKxVWUJVWUGwC20s026x8GjcxK67AKxVWUGVWUWwC2zVAF1VAY17CE14v2 6rWY6r4UJwCIc40Y0x0EwIxGrwCI42IY6xIIjxv20xvE14v26r1j6r1xMIIF0xvE2Ix0cI 8IcVCY1x0267AKxVW8JVWxJwCI42IY6xAIw20EY4v20xvaj40_Wr1j6rW3Jr1lIxAIcVC2 z280aVAFwI0_Jr0_Gr1lIxAIcVC2z280aVCY1x0267AKxVW8Jr0_Cr1UYxBIdaVFxhVjvj DU0xZFpf9x07jeKZXUUUUU= X-CM-SenderInfo: hshw23xhvd2x3n6k3tpzhluzxrxghudrp/ X-CFilter-Loop: Reflected X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1763598667633717380?= X-GMAIL-MSGID: =?utf-8?q?1763598667633717380?= From: Petr Tesarik The goal of my work is to provide more flexibility in the sizing of SWIOTLB. The software IO TLB was designed with these assumptions: 1. It would not be used much, especially on 64-bit systems. 2. A small fixed memory area (64 MiB by default) is sufficient to handle the few cases which require a bounce buffer. 3. 64 MiB is little enough that it has no impact on the rest of the system. First, if SEV is active, all DMA must be done through shared unencrypted pages, and SWIOTLB is used to make this happen without changing device drivers. The software IO TLB size is increased to 6% of total memory in sev_setup_arch(), but that is more of an approximation. The actual requirements may vary depending on the amount of I/O and which drivers are used. These factors may not be know at boot time, i.e. when SWIOTLB is allocated. Second, other colleagues have noticed that they can reliably get rid of occasional OOM kills on an Arm embedded device by reducing the SWIOTLB size. This can be achieved with a kernel parameter, but determining the right value puts additional burden on pre-release testing, which could be avoided if SWIOTLB is allocated small and grows only when necessary. Changes from v1-devel-v7: - Add comments to acquire/release barriers - Fix whitespace issues reported by checkpatch.pl Changes from v1-devel-v6: - Provide long description of functions - Fix kernel-doc (Returns: to Return:) - Rename __lookup_dyn_slot() to lookup_dyn_slot_locked() Changes from RFC: - Track dynamic buffers per device instead of per swiotlb - Use a linked list instead of a maple tree - Move initialization of swiotlb fields of struct device to a helper function - Rename __lookup_dyn_slot() to lookup_dyn_slot_locked() - Introduce per-device flag if dynamic buffers are in use - Add one more user of DMA_ATTR_MAY_SLEEP - Add kernel-doc comments for new (and some old) code - Properly escape '*' in dma-attributes.rst Petr Tesarik (7): swiotlb: Use a helper to initialize swiotlb fields in struct device swiotlb: Move code around in preparation for dynamic bounce buffers dma-mapping: introduce the DMA_ATTR_MAY_SLEEP attribute swiotlb: Dynamically allocated bounce buffers swiotlb: Add a boot option to enable dynamic bounce buffers drm: Use DMA_ATTR_MAY_SLEEP from process context swiotlb: per-device flag if there are dynamically allocated buffers .../admin-guide/kernel-parameters.txt | 6 +- Documentation/core-api/dma-attributes.rst | 10 + drivers/base/core.c | 4 +- drivers/gpu/drm/drm_gem_shmem_helper.c | 2 +- drivers/gpu/drm/drm_prime.c | 2 +- include/linux/device.h | 12 + include/linux/dma-mapping.h | 6 + include/linux/swiotlb.h | 54 ++- kernel/dma/swiotlb.c | 382 ++++++++++++++++-- 9 files changed, 443 insertions(+), 35 deletions(-)