Message ID | tencent_0B319B215E8D487CC082C0DA5E8E46B86B08@qq.com |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel+bounces-27072-ouuuleilei=gmail.com@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:42cf:b0:101:a8e8:374 with SMTP id q15csp98460dye; Mon, 15 Jan 2024 23:25:44 -0800 (PST) X-Google-Smtp-Source: AGHT+IG6YWCbtvbkBUjMecsx6ampK+y5r0jMeCA/oDDFgPs2WGTVUZXNWgq8TxbZGqD/9AphHRRj X-Received: by 2002:a05:620a:424f:b0:783:55c3:351b with SMTP id w15-20020a05620a424f00b0078355c3351bmr4660570qko.127.1705389944668; Mon, 15 Jan 2024 23:25:44 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1705389944; cv=none; d=google.com; s=arc-20160816; b=fHb4szxTaQyEacBYY8AO7j/QdiFFV6AHdWdJeQhtWIYRyOBoZdWshAT4rEg1r6QqTg /2FSpBicvHcDjMDJBf1fBDTlO0JA3hNRsT5/5TMXNQw86Yk7wVHLH9tb8dzw3TZuu+nu kZjpRYtqSrmhckFYBVmyE0+MOIjg+n3Kp1Cg/aohXMI/DOKcMacaUwwKzplL+IYbdiMO w5NA26CsWETQ1iR+UrQ/3Qv+Hl8JA91B/yN3M8B1mTSnk0ncRAkgKdRfu3wTWH2sjkOu 6/DVqik/IZUnSk4EsohiHuw3pR6rVvVg9c1fVrz0BH+U1IJYnfUwJJtBoOnJY985VcTP qBgA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:date:subject:cc:to:from :message-id:dkim-signature; bh=ZTnSJxE23Nw64lBhpFMiDwgBba+3N2Q91LfEvH/xgy8=; fh=eNPPkSytXkAEcIG64fCslI60BSh5fBmlpk5FYOBkMGs=; b=CGfZFXg75qWQ67t+aaFt40x0k6DoxZWBudD8eDTBbg/2dKS8f3XJF7Eak8eFA5sJ3v tRGlMKM4TcYhZ/NJfJyHhllHmXe8nIHEy/9awXzixDOd/RWsY4P4IMbZ/V9lJm3kYsOY 6jpBZQ5piH8QdbGaUga8GoEQazmQRxm551NVCAkUmJiCiazqcU2fCss8Sx4fMSgVZCIV l4CWZy0EN6IRfP4tv3zuXOk1lXXeq5CrjkzFYTPhoNv9YAQ1unPOPA9I1BUuRmUpwot/ vkt7uRalj5GaOMWDTU9rNC+5nHkDRpeth7Of5J9nwXBfZvdX36YICyKFP7c1sbIIybQ0 PXCw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@qq.com header.s=s201512 header.b=G38fTUco; spf=pass (google.com: domain of linux-kernel+bounces-27072-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-27072-ouuuleilei=gmail.com@vger.kernel.org" Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id w2-20020a05620a094200b007834e8d218asi6415897qkw.305.2024.01.15.23.25.44 for <ouuuleilei@gmail.com> (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Jan 2024 23:25:44 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-27072-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@qq.com header.s=s201512 header.b=G38fTUco; spf=pass (google.com: domain of linux-kernel+bounces-27072-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-27072-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 741281C23038 for <ouuuleilei@gmail.com>; Tue, 16 Jan 2024 07:25:44 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 2844F111AC; Tue, 16 Jan 2024 07:25:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=qq.com header.i=@qq.com header.b="G38fTUco" Received: from out162-62-57-252.mail.qq.com (out162-62-57-252.mail.qq.com [162.62.57.252]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B64A811181 for <linux-kernel@vger.kernel.org>; Tue, 16 Jan 2024 07:25:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=cyyself.name Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=cyyself.name DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=qq.com; s=s201512; t=1705389924; bh=ZTnSJxE23Nw64lBhpFMiDwgBba+3N2Q91LfEvH/xgy8=; h=From:To:Cc:Subject:Date; b=G38fTUcoKNLSZd/QUBsWbCN/RnPFaiNfIYw18aeBSLdQUIRkuDB/RPJofM8J+aaI4 f85uc3qsOEtWcRmqRRFREBHHPH8w17O4/+SzyYhRHMo3Nka16AdWCpaFDdjR0OlMRE Z0mYZl5oxZbkU9So7i1fRlgeuWG0YR+70RmLvqbs= Received: from cyy-pc.lan ([2001:da8:c800:d084:c65a:644a:13d7:e72c]) by newxmesmtplogicsvrszc5-0.qq.com (NewEsmtp) with SMTP id 651A00DE; Tue, 16 Jan 2024 15:25:17 +0800 X-QQ-mid: xmsmtpt1705389917t93fy8ema Message-ID: <tencent_0B319B215E8D487CC082C0DA5E8E46B86B08@qq.com> X-QQ-XMAILINFO: Mk33QmgWauDsVhZvDrj7M3dssC3UeSG28JJBftbgZZxw/z6e0XiKTnW6dUZ7ub q4us9jOKT7MYrIpaxAp7A0JRfI0iTENFWvZCGJp7KoV736goBQFeEBtW6xmkl3opdSb9131K+bwo skscGb5PaYHkklbTch8emcjsdSZ/CEna1Pc04OlqWBJ7tKAaROnot02p2pUly3jCvr3mZFPujXtD MFK0TbmKyMFY7A2CaWEQVq1zx7l/E7gw4suN8RUFCIzJMJ3f905PGXWosBx57wbSGwXjZh4v9r+Z 0LkV/9avVPphtGQLt12EZz5W3+KoJ/0T+PqOPtkT20PuxKpKZRnjClVjzKzAd6uz3Xw2UfLgQw2B ZTVHqKKNjdLA7a/IEHfyfO2hS7bl+ZS4h+T4JgGoDS4WB81HMRqOG6MrURCRbM4NFV7XEWgpE7bp P1CIsQPY9YDCgRaqRWUAtrEck1Oi8m61cLmMUAflo1cfWN/peusc0YbjTX/NIuz1BgU0VXh/B7Mo SOqG4hK4+PqPCMr9tNxCXvMPNMGjc3WVXZAR+BB5Rxyks+D+ZjvIwu8VNQP8LVPqNTXSfArz3fUY d+AuLzIkQ6d/v6E6y14DGQvHRKx7z5tP7UVRI7M3cB/LJew4jHOnJCUC6aQwK33kpW5t2IDKWD2e 1ChYPRufeFC8NsiBXCXXW/8upYqOAKptW6mLfdKtJ4BdDYG1n69j9Ous5d7PqdDEZ8l3u5NeuycS UWPmJKH0TmPfxTTuYtTLly2tKN7URRCHpJTllYxSdAFVwgcZgj+RR19qr2FQFw802669848wQan2 /WavfQo3m+cd8g7q3aI0JHafClIwOoHx/iHpx6bddDst3HPrd+5fWdvwj3D6ehZG+nqDAHNHbhEm Z8QsRaxfcGI+i+/Yh4c4B9y5DVnA+mGrmtfCB5mdywA8FYD5v82QTJPi7T2ZnQEfNBk/EMye7dZ4 2v2E/t+guRkynBfXlF21eXMiOMfdjgd7mLxS0TqnTXOkq1OQvNcr1akSig9Pll X-QQ-XMRINFO: NyFYKkN4Ny6FSmKK/uo/jdU= From: Yangyu Chen <cyy@cyyself.name> To: dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org Cc: Christian Koenig <christian.koenig@amd.com>, Huang Rui <ray.huang@amd.com>, Maarten Lankhorst <maarten.lankhorst@linux.intel.com>, Maxime Ripard <mripard@kernel.org>, Thomas Zimmermann <tzimmermann@suse.de>, David Airlie <airlied@gmail.com>, Daniel Vetter <daniel@ffwll.ch>, Jiuyang Liu <liu@jiuyang.me>, Yichuan Gao <i@gycis.me>, Yangyu Chen <cyy@cyyself.name> Subject: [PATCH] drm/ttm: only use DMA32 if needed for dummy_read_page Date: Tue, 16 Jan 2024 15:24:06 +0800 X-OQ-MSGID: <20240116072406.107206-1-cyy@cyyself.name> X-Mailer: git-send-email 2.43.0 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: <linux-kernel.vger.kernel.org> List-Subscribe: <mailto:linux-kernel+subscribe@vger.kernel.org> List-Unsubscribe: <mailto:linux-kernel+unsubscribe@vger.kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1788230966524198462 X-GMAIL-MSGID: 1788230966524198462 |
Series |
drm/ttm: only use DMA32 if needed for dummy_read_page
|
|
Commit Message
Yangyu Chen
Jan. 16, 2024, 7:24 a.m. UTC
Some platforms may not have any memory in ZONE_DMA32 and use IOMMU to allow
32-bit-DMA-only device to work. Forcing GFP_DMA32 on dummy_read_page will
fail in such platforms. Only use DMA32 when it must to get the bug
resolved.
Signed-off-by: Yangyu Chen <cyy@cyyself.name>
---
drivers/gpu/drm/ttm/ttm_device.c | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
Comments
Am 16.01.24 um 08:24 schrieb Yangyu Chen: > Some platforms may not have any memory in ZONE_DMA32 and use IOMMU to allow > 32-bit-DMA-only device to work. Forcing GFP_DMA32 on dummy_read_page will > fail in such platforms. Only use DMA32 when it must to get the bug > resolved. Well that makes no sense. If a platform doesn't have a ZONE_DMA32 then GFP_DMA32 is just ignored as far as I know. This patch here won't work since the use_dma32 flag is a per device flag which can't be used for the global initialization. Otherwise this can randomly fail depending on if a DMA32 device initializes first or after some device with larger addressing capabilities. This configuration is quite common on older motherboards with both integrated and dedicated graphics. Regards, Christian. > > Signed-off-by: Yangyu Chen <cyy@cyyself.name> > --- > drivers/gpu/drm/ttm/ttm_device.c | 7 ++++--- > 1 file changed, 4 insertions(+), 3 deletions(-) > > diff --git a/drivers/gpu/drm/ttm/ttm_device.c b/drivers/gpu/drm/ttm/ttm_device.c > index d48b39132b32..62f16fb72428 100644 > --- a/drivers/gpu/drm/ttm/ttm_device.c > +++ b/drivers/gpu/drm/ttm/ttm_device.c > @@ -63,7 +63,7 @@ static void ttm_global_release(void) > mutex_unlock(&ttm_global_mutex); > } > > -static int ttm_global_init(void) > +static int ttm_global_init(bool use_dma32) > { > struct ttm_global *glob = &ttm_glob; > unsigned long num_pages, num_dma32; > @@ -95,7 +95,8 @@ static int ttm_global_init(void) > ttm_pool_mgr_init(num_pages); > ttm_tt_mgr_init(num_pages, num_dma32); > > - glob->dummy_read_page = alloc_page(__GFP_ZERO | GFP_DMA32); > + glob->dummy_read_page = use_dma32 ? alloc_page(__GFP_ZERO | GFP_DMA32) : > + alloc_page(__GFP_ZERO); > > if (unlikely(glob->dummy_read_page == NULL)) { > ret = -ENOMEM; > @@ -200,7 +201,7 @@ int ttm_device_init(struct ttm_device *bdev, const struct ttm_device_funcs *func > if (WARN_ON(vma_manager == NULL)) > return -EINVAL; > > - ret = ttm_global_init(); > + ret = ttm_global_init(use_dma32); > if (ret) > return ret; >
On 1/16/24 17:45, Christian König wrote: > Am 16.01.24 um 08:24 schrieb Yangyu Chen: >> Some platforms may not have any memory in ZONE_DMA32 and use IOMMU to >> allow >> 32-bit-DMA-only device to work. Forcing GFP_DMA32 on dummy_read_page will >> fail in such platforms. Only use DMA32 when it must to get the bug >> resolved. > > Well that makes no sense. > > If a platform doesn't have a ZONE_DMA32 then GFP_DMA32 is just ignored > as far as I know. > Well, but it only applies when such ISA has no ZONE_DMA32 such as x86-32 without PAE. It does not apply to some ISAs which has ZONE_DMA32 defined in the kernel but there is no memory below the 4GB address range. It happens on some RISC-V SoC chips and I know at least two chips have this issue. Below is some dmesg from my platform, using such RISC-V without ZONE_DMA32 with AMD RX550 GPU: [ 0.000000] Linux version 6.7.0-dirty (cyy@cyy-pc) (riscv64-linux-gnu-gcc (Debian 13.2.0-7) 13.2.0, GNU ld (GNU Binutils for Debian) 2.41.50.20231227) #12 SMP Tue Jan 16 18:55:13 CST 2024 [ 0.000000] Machine model: [ 0.000000] SBI specification v2.0 detected [ 0.000000] SBI implementation ID=0x1 Version=0x10004 [ 0.000000] SBI TIME extension detected [ 0.000000] SBI IPI extension detected [ 0.000000] SBI RFENCE extension detected [ 0.000000] efi: UEFI not found. [ 0.000000] OF: reserved mem: 0x0000002000000000..0x000000200003ffff (256 KiB) nomap non-reusable mmode_resv1@20,0 [ 0.000000] OF: reserved mem: 0x0000002000040000..0x000000200005ffff (128 KiB) nomap non-reusable mmode_resv0@20,40000 [ 0.000000] Zone ranges: [ 0.000000] DMA32 empty [ 0.000000] Normal [mem 0x0000002000000000-0x00000021ffffffff] [ 35.738619] [drm] amdgpu kernel modesetting enabled. [ 35.743916] [drm] initializing kernel modesetting (POLARIS12 0x1002:0x699F 0x1043:0x0513 0xC7). [ 35.752671] [drm] register mmio base: 0xA8100000 [ 35.763558] [drm] register mmio size: 262144 [ 35.768491] [drm] add ip block number 0 <vi_common> [ 35.773403] [drm] add ip block number 1 <gmc_v8_0> [ 35.773407] [drm] add ip block number 2 <tonga_ih> [ 35.773410] [drm] add ip block number 3 <gfx_v8_0> [ 35.773412] [drm] add ip block number 4 <sdma_v3_0> [ 35.773415] [drm] add ip block number 5 <powerplay> [ 35.773418] [drm] add ip block number 6 <dm> [ 35.806668] [drm] add ip block number 7 <uvd_v6_0> [ 35.806671] [drm] add ip block number 8 <vce_v3_0> [ 36.014036] amdgpu 0000:05:00.0: amdgpu: Fetched VBIOS from ROM BAR [ 36.020322] amdgpu: ATOM BIOS: 115-C994PI2-100 [ 36.043586] [drm] UVD is enabled in VM mode [ 36.047813] [drm] UVD ENC is enabled in VM mode [ 36.052370] [drm] VCE enabled in VM mode [ 36.052377] amdgpu 0000:05:00.0: amdgpu: Trusted Memory Zone (TMZ) feature not supported [ 36.064404] amdgpu 0000:05:00.0: amdgpu: PCIE atomic ops is not supported [ 36.071209] [drm] GPU posting now... [ 36.190029] [drm] vm size is 64 GB, 2 levels, block size is 10-bit, fragment size is 9-bit [ 36.199385] amdgpu 0000:05:00.0: amdgpu: VRAM: 4096M 0x000000F400000000 - 0x000000F4FFFFFFFF (4096M used) [ 36.208978] amdgpu 0000:05:00.0: amdgpu: GART: 256M 0x000000FF00000000 - 0x000000FF0FFFFFFF [ 36.217345] [drm] Detected VRAM RAM=4096M, BAR=256M [ 36.222226] [drm] RAM width 128bits GDDR5 [ 36.226288] warn_alloc: 8 callbacks suppressed [ 36.226290] (udev-worker): page allocation failure: order:0, mode:0x104(GFP_DMA32|__GFP_ZERO), nodemask=(null),cpuset=/,mems_allowed=0 [ 36.242837] CPU: 3 PID: 165 Comm: (udev-worker) Not tainted 6.7.0-dirty #12 [ 36.249796] Hardware name: (DT) [ 36.254670] Call Trace: [ 36.257113] [<ffffffff80006056>] dump_backtrace+0x1c/0x24 [ 36.262513] [<ffffffff8096b860>] show_stack+0x2c/0x38 [ 36.267562] [<ffffffff80977f14>] dump_stack_lvl+0x3c/0x54 [ 36.272958] [<ffffffff80977f40>] dump_stack+0x14/0x1c [ 36.278006] [<ffffffff8018e66c>] warn_alloc+0xee/0x16c [ 36.283143] [<ffffffff8018f17a>] __alloc_pages+0xa90/0xb40 [ 36.288627] [<ffffffff0174f5fe>] ttm_global_init+0x12a/0x1ac [ttm] [ 36.294830] [<ffffffff0174f6b2>] ttm_device_init+0x32/0x158 [ttm] [ 36.300935] [<ffffffff018448bc>] amdgpu_ttm_init+0x7a/0x638 [amdgpu] [ 36.310260] [<ffffffff01b4c8b8>] amdgpu_bo_init+0x76/0x82 [amdgpu] [ 36.319323] [<ffffffff018c2b6e>] gmc_v8_0_sw_init+0x36c/0x652 [amdgpu] [ 36.328710] [<ffffffff018328fa>] amdgpu_device_init+0x1648/0x20fa [amdgpu] [ 36.338395] [<ffffffff01834c1a>] amdgpu_driver_load_kms+0x1e/0x158 [amdgpu] [ 36.348164] [<ffffffff0182b9d4>] amdgpu_pci_probe+0x124/0x46c [amdgpu] [ 36.357543] [<ffffffff804ae3ae>] pci_device_probe+0x7a/0xf0 [ 36.363115] [<ffffffff8058cfb6>] really_probe+0x86/0x242 [ 36.368427] [<ffffffff8058d1ce>] __driver_probe_device+0x5c/0xda [ 36.374431] [<ffffffff8058d278>] driver_probe_device+0x2c/0xb2 [ 36.380260] [<ffffffff8058d3f8>] __driver_attach+0x6c/0x11a [ 36.385829] [<ffffffff8058b14c>] bus_for_each_dev+0x60/0xae [ 36.391398] [<ffffffff8058ca08>] driver_attach+0x1a/0x22 [ 36.396707] [<ffffffff8058c368>] bus_add_driver+0xd0/0x1ba [ 36.402189] [<ffffffff8058e046>] driver_register+0x3e/0xd8 [ 36.407671] [<ffffffff804ad04a>] __pci_register_driver+0x58/0x62 [ 36.413679] [<ffffffff0207e078>] amdgpu_init+0x78/0x1000 [amdgpu] [ 36.422616] [<ffffffff8000212c>] do_one_initcall+0x58/0x19c [ 36.428188] [<ffffffff80086d22>] do_init_module+0x4e/0x1b0 [ 36.433672] [<ffffffff80088456>] load_module+0x1374/0x1768 [ 36.439155] [<ffffffff80088a10>] init_module_from_file+0x76/0xaa [ 36.445158] [<ffffffff80088c2c>] __riscv_sys_finit_module+0x1cc/0x2ba [ 36.451597] [<ffffffff80978952>] do_trap_ecall_u+0xba/0x12e [ 36.457167] [<ffffffff809819f4>] ret_from_exception+0x0/0x64 [ 36.462847] Mem-Info: [ 36.465124] active_anon:41 inactive_anon:4533 isolated_anon:0 active_file:12928 inactive_file:24854 isolated_file:0 unevictable:0 dirty:176 writeback:0 slab_reclaimable:3390 slab_unreclaimable:3922 mapped:4609 shmem:333 pagetables:564 sec_pagetables:0 bounce:0 kernel_misc_reclaimable:0 free:1983561 free_pcp:0 free_cma:0 [ 36.490913] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB [ 36.508722] 38077 total pagecache pages [ 36.512560] 0 pages in swap cache [ 36.515875] Free swap = 0kB [ 36.518757] Total swap = 0kB [ 36.521633] 2097152 pages RAM [ 36.524601] 0 pages HighMem/MovableOnly [ 36.528438] 45269 pages reserved [ 36.531699] [drm:amdgpu_ttm_init [amdgpu]] *ERROR* failed initializing buffer object driver(-12). [ 36.543451] [drm:amdgpu_device_init [amdgpu]] *ERROR* sw_init of IP block <gmc_v8_0> failed -12 [ 36.555011] amdgpu 0000:05:00.0: amdgpu: amdgpu_device_ip_init failed [ 36.561458] amdgpu 0000:05:00.0: amdgpu: Fatal error during GPU init [ 36.567816] amdgpu 0000:05:00.0: amdgpu: amdgpu: finishing device. > This patch here won't work since the use_dma32 flag is a per device flag > which can't be used for the global initialization. > > Otherwise this can randomly fail depending on if a DMA32 device > initializes first or after some device with larger addressing > capabilities. This configuration is quite common on older motherboards > with both integrated and dedicated graphics. > > Regards, > Christian. > Thanks for your review. However, the bug I mentioned above still needs to be resolved. I want to submit patch v2 with a different solution that tries to allocate with GFP_DMA32 flag, if it fails, then try to allocate without GFP_DMA32 flag. Would this solution be acceptable? Thanks, Yangyu Chen >> >> Signed-off-by: Yangyu Chen <cyy@cyyself.name> >> --- >> drivers/gpu/drm/ttm/ttm_device.c | 7 ++++--- >> 1 file changed, 4 insertions(+), 3 deletions(-) >> >> diff --git a/drivers/gpu/drm/ttm/ttm_device.c >> b/drivers/gpu/drm/ttm/ttm_device.c >> index d48b39132b32..62f16fb72428 100644 >> --- a/drivers/gpu/drm/ttm/ttm_device.c >> +++ b/drivers/gpu/drm/ttm/ttm_device.c >> @@ -63,7 +63,7 @@ static void ttm_global_release(void) >> mutex_unlock(&ttm_global_mutex); >> } >> -static int ttm_global_init(void) >> +static int ttm_global_init(bool use_dma32) >> { >> struct ttm_global *glob = &ttm_glob; >> unsigned long num_pages, num_dma32; >> @@ -95,7 +95,8 @@ static int ttm_global_init(void) >> ttm_pool_mgr_init(num_pages); >> ttm_tt_mgr_init(num_pages, num_dma32); >> - glob->dummy_read_page = alloc_page(__GFP_ZERO | GFP_DMA32); >> + glob->dummy_read_page = use_dma32 ? alloc_page(__GFP_ZERO | >> GFP_DMA32) : >> + alloc_page(__GFP_ZERO); >> if (unlikely(glob->dummy_read_page == NULL)) { >> ret = -ENOMEM; >> @@ -200,7 +201,7 @@ int ttm_device_init(struct ttm_device *bdev, const >> struct ttm_device_funcs *func >> if (WARN_ON(vma_manager == NULL)) >> return -EINVAL; >> - ret = ttm_global_init(); >> + ret = ttm_global_init(use_dma32); >> if (ret) >> return ret; >
Am 16.01.24 um 12:27 schrieb Yangyu Chen: > On 1/16/24 17:45, Christian König wrote: >> Am 16.01.24 um 08:24 schrieb Yangyu Chen: >>> Some platforms may not have any memory in ZONE_DMA32 and use IOMMU >>> to allow >>> 32-bit-DMA-only device to work. Forcing GFP_DMA32 on dummy_read_page >>> will >>> fail in such platforms. Only use DMA32 when it must to get the bug >>> resolved. >> >> Well that makes no sense. >> >> If a platform doesn't have a ZONE_DMA32 then GFP_DMA32 is just >> ignored as far as I know. >> > > Well, but it only applies when such ISA has no ZONE_DMA32 such as x86-32 > without PAE. It does not apply to some ISAs which has ZONE_DMA32 defined > in the kernel but there is no memory below the 4GB address range. It > happens on some RISC-V SoC chips and I know at least two chips have this > issue. > > > Below is some dmesg from my platform, using such RISC-V without > ZONE_DMA32 with AMD RX550 GPU: > > [ 0.000000] Linux version 6.7.0-dirty (cyy@cyy-pc) > (riscv64-linux-gnu-gcc (Debian 13.2.0-7) 13.2.0, GNU ld (GNU Binutils > for Debian) 2.41.50.20231227) #12 SMP Tue Jan 16 18:55:13 CST 2024 > [ 0.000000] Machine model: > [ 0.000000] SBI specification v2.0 detected > [ 0.000000] SBI implementation ID=0x1 Version=0x10004 > [ 0.000000] SBI TIME extension detected > [ 0.000000] SBI IPI extension detected > [ 0.000000] SBI RFENCE extension detected > [ 0.000000] efi: UEFI not found. > [ 0.000000] OF: reserved mem: > 0x0000002000000000..0x000000200003ffff (256 KiB) nomap non-reusable > mmode_resv1@20,0 > [ 0.000000] OF: reserved mem: > 0x0000002000040000..0x000000200005ffff (128 KiB) nomap non-reusable > mmode_resv0@20,40000 > [ 0.000000] Zone ranges: > [ 0.000000] DMA32 empty > [ 0.000000] Normal [mem 0x0000002000000000-0x00000021ffffffff] > > [ 35.738619] [drm] amdgpu kernel modesetting enabled. > [ 35.743916] [drm] initializing kernel modesetting (POLARIS12 > 0x1002:0x699F 0x1043:0x0513 0xC7). > [ 35.752671] [drm] register mmio base: 0xA8100000 > [ 35.763558] [drm] register mmio size: 262144 > [ 35.768491] [drm] add ip block number 0 <vi_common> > [ 35.773403] [drm] add ip block number 1 <gmc_v8_0> > [ 35.773407] [drm] add ip block number 2 <tonga_ih> > [ 35.773410] [drm] add ip block number 3 <gfx_v8_0> > [ 35.773412] [drm] add ip block number 4 <sdma_v3_0> > [ 35.773415] [drm] add ip block number 5 <powerplay> > [ 35.773418] [drm] add ip block number 6 <dm> > [ 35.806668] [drm] add ip block number 7 <uvd_v6_0> > [ 35.806671] [drm] add ip block number 8 <vce_v3_0> > [ 36.014036] amdgpu 0000:05:00.0: amdgpu: Fetched VBIOS from ROM BAR > [ 36.020322] amdgpu: ATOM BIOS: 115-C994PI2-100 > [ 36.043586] [drm] UVD is enabled in VM mode > [ 36.047813] [drm] UVD ENC is enabled in VM mode > [ 36.052370] [drm] VCE enabled in VM mode > [ 36.052377] amdgpu 0000:05:00.0: amdgpu: Trusted Memory Zone (TMZ) > feature not supported > [ 36.064404] amdgpu 0000:05:00.0: amdgpu: PCIE atomic ops is not > supported > [ 36.071209] [drm] GPU posting now... > [ 36.190029] [drm] vm size is 64 GB, 2 levels, block size is 10-bit, > fragment size is 9-bit > [ 36.199385] amdgpu 0000:05:00.0: amdgpu: VRAM: 4096M > 0x000000F400000000 - 0x000000F4FFFFFFFF (4096M used) > [ 36.208978] amdgpu 0000:05:00.0: amdgpu: GART: 256M > 0x000000FF00000000 - 0x000000FF0FFFFFFF > [ 36.217345] [drm] Detected VRAM RAM=4096M, BAR=256M > [ 36.222226] [drm] RAM width 128bits GDDR5 > [ 36.226288] warn_alloc: 8 callbacks suppressed > [ 36.226290] (udev-worker): page allocation failure: order:0, > mode:0x104(GFP_DMA32|__GFP_ZERO), nodemask=(null),cpuset=/,mems_allowed=0 > [ 36.242837] CPU: 3 PID: 165 Comm: (udev-worker) Not tainted > 6.7.0-dirty #12 > [ 36.249796] Hardware name: (DT) > [ 36.254670] Call Trace: > [ 36.257113] [<ffffffff80006056>] dump_backtrace+0x1c/0x24 > [ 36.262513] [<ffffffff8096b860>] show_stack+0x2c/0x38 > [ 36.267562] [<ffffffff80977f14>] dump_stack_lvl+0x3c/0x54 > [ 36.272958] [<ffffffff80977f40>] dump_stack+0x14/0x1c > [ 36.278006] [<ffffffff8018e66c>] warn_alloc+0xee/0x16c > [ 36.283143] [<ffffffff8018f17a>] __alloc_pages+0xa90/0xb40 > [ 36.288627] [<ffffffff0174f5fe>] ttm_global_init+0x12a/0x1ac [ttm] > [ 36.294830] [<ffffffff0174f6b2>] ttm_device_init+0x32/0x158 [ttm] > [ 36.300935] [<ffffffff018448bc>] amdgpu_ttm_init+0x7a/0x638 [amdgpu] > [ 36.310260] [<ffffffff01b4c8b8>] amdgpu_bo_init+0x76/0x82 [amdgpu] > [ 36.319323] [<ffffffff018c2b6e>] gmc_v8_0_sw_init+0x36c/0x652 [amdgpu] > [ 36.328710] [<ffffffff018328fa>] amdgpu_device_init+0x1648/0x20fa > [amdgpu] > [ 36.338395] [<ffffffff01834c1a>] amdgpu_driver_load_kms+0x1e/0x158 > [amdgpu] > [ 36.348164] [<ffffffff0182b9d4>] amdgpu_pci_probe+0x124/0x46c [amdgpu] > [ 36.357543] [<ffffffff804ae3ae>] pci_device_probe+0x7a/0xf0 > [ 36.363115] [<ffffffff8058cfb6>] really_probe+0x86/0x242 > [ 36.368427] [<ffffffff8058d1ce>] __driver_probe_device+0x5c/0xda > [ 36.374431] [<ffffffff8058d278>] driver_probe_device+0x2c/0xb2 > [ 36.380260] [<ffffffff8058d3f8>] __driver_attach+0x6c/0x11a > [ 36.385829] [<ffffffff8058b14c>] bus_for_each_dev+0x60/0xae > [ 36.391398] [<ffffffff8058ca08>] driver_attach+0x1a/0x22 > [ 36.396707] [<ffffffff8058c368>] bus_add_driver+0xd0/0x1ba > [ 36.402189] [<ffffffff8058e046>] driver_register+0x3e/0xd8 > [ 36.407671] [<ffffffff804ad04a>] __pci_register_driver+0x58/0x62 > [ 36.413679] [<ffffffff0207e078>] amdgpu_init+0x78/0x1000 [amdgpu] > [ 36.422616] [<ffffffff8000212c>] do_one_initcall+0x58/0x19c > [ 36.428188] [<ffffffff80086d22>] do_init_module+0x4e/0x1b0 > [ 36.433672] [<ffffffff80088456>] load_module+0x1374/0x1768 > [ 36.439155] [<ffffffff80088a10>] init_module_from_file+0x76/0xaa > [ 36.445158] [<ffffffff80088c2c>] __riscv_sys_finit_module+0x1cc/0x2ba > [ 36.451597] [<ffffffff80978952>] do_trap_ecall_u+0xba/0x12e > [ 36.457167] [<ffffffff809819f4>] ret_from_exception+0x0/0x64 > [ 36.462847] Mem-Info: > [ 36.465124] active_anon:41 inactive_anon:4533 isolated_anon:0 > active_file:12928 inactive_file:24854 isolated_file:0 > unevictable:0 dirty:176 writeback:0 > slab_reclaimable:3390 slab_unreclaimable:3922 > mapped:4609 shmem:333 pagetables:564 > sec_pagetables:0 bounce:0 > kernel_misc_reclaimable:0 > free:1983561 free_pcp:0 free_cma:0 > [ 36.490913] Node 0 hugepages_total=0 hugepages_free=0 > hugepages_surp=0 hugepages_size=2048kB > [ 36.508722] 38077 total pagecache pages > [ 36.512560] 0 pages in swap cache > [ 36.515875] Free swap = 0kB > [ 36.518757] Total swap = 0kB > [ 36.521633] 2097152 pages RAM > [ 36.524601] 0 pages HighMem/MovableOnly > [ 36.528438] 45269 pages reserved > [ 36.531699] [drm:amdgpu_ttm_init [amdgpu]] *ERROR* failed > initializing buffer object driver(-12). > [ 36.543451] [drm:amdgpu_device_init [amdgpu]] *ERROR* sw_init of IP > block <gmc_v8_0> failed -12 > [ 36.555011] amdgpu 0000:05:00.0: amdgpu: amdgpu_device_ip_init failed > [ 36.561458] amdgpu 0000:05:00.0: amdgpu: Fatal error during GPU init > [ 36.567816] amdgpu 0000:05:00.0: amdgpu: amdgpu: finishing device. Interesting, I think I remember seeing something like this on ARM as well but we didn't run into any problems with GFP_DMA32 there. >> This patch here won't work since the use_dma32 flag is a per device >> flag which can't be used for the global initialization. >> >> Otherwise this can randomly fail depending on if a DMA32 device >> initializes first or after some device with larger addressing >> capabilities. This configuration is quite common on older >> motherboards with both integrated and dedicated graphics. >> >> Regards, >> Christian. >> > > Thanks for your review. However, the bug I mentioned above still needs > to be resolved. I want to submit patch v2 with a different solution > that tries to allocate with GFP_DMA32 flag, if it fails, then try to > allocate without GFP_DMA32 flag. Would this solution be acceptable? Yeah, that's probably a good idea. I mean in general we should avoid GFP_DMA32 altogether here, but for this we would need a function to get memory with the best possible accessibility for all devices. Regards, Christian. > > Thanks, > Yangyu Chen > >>> >>> Signed-off-by: Yangyu Chen <cyy@cyyself.name> >>> --- >>> drivers/gpu/drm/ttm/ttm_device.c | 7 ++++--- >>> 1 file changed, 4 insertions(+), 3 deletions(-) >>> >>> diff --git a/drivers/gpu/drm/ttm/ttm_device.c >>> b/drivers/gpu/drm/ttm/ttm_device.c >>> index d48b39132b32..62f16fb72428 100644 >>> --- a/drivers/gpu/drm/ttm/ttm_device.c >>> +++ b/drivers/gpu/drm/ttm/ttm_device.c >>> @@ -63,7 +63,7 @@ static void ttm_global_release(void) >>> mutex_unlock(&ttm_global_mutex); >>> } >>> -static int ttm_global_init(void) >>> +static int ttm_global_init(bool use_dma32) >>> { >>> struct ttm_global *glob = &ttm_glob; >>> unsigned long num_pages, num_dma32; >>> @@ -95,7 +95,8 @@ static int ttm_global_init(void) >>> ttm_pool_mgr_init(num_pages); >>> ttm_tt_mgr_init(num_pages, num_dma32); >>> - glob->dummy_read_page = alloc_page(__GFP_ZERO | GFP_DMA32); >>> + glob->dummy_read_page = use_dma32 ? alloc_page(__GFP_ZERO | >>> GFP_DMA32) : >>> + alloc_page(__GFP_ZERO); >>> if (unlikely(glob->dummy_read_page == NULL)) { >>> ret = -ENOMEM; >>> @@ -200,7 +201,7 @@ int ttm_device_init(struct ttm_device *bdev, >>> const struct ttm_device_funcs *func >>> if (WARN_ON(vma_manager == NULL)) >>> return -EINVAL; >>> - ret = ttm_global_init(); >>> + ret = ttm_global_init(use_dma32); >>> if (ret) >>> return ret; >> >
diff --git a/drivers/gpu/drm/ttm/ttm_device.c b/drivers/gpu/drm/ttm/ttm_device.c index d48b39132b32..62f16fb72428 100644 --- a/drivers/gpu/drm/ttm/ttm_device.c +++ b/drivers/gpu/drm/ttm/ttm_device.c @@ -63,7 +63,7 @@ static void ttm_global_release(void) mutex_unlock(&ttm_global_mutex); } -static int ttm_global_init(void) +static int ttm_global_init(bool use_dma32) { struct ttm_global *glob = &ttm_glob; unsigned long num_pages, num_dma32; @@ -95,7 +95,8 @@ static int ttm_global_init(void) ttm_pool_mgr_init(num_pages); ttm_tt_mgr_init(num_pages, num_dma32); - glob->dummy_read_page = alloc_page(__GFP_ZERO | GFP_DMA32); + glob->dummy_read_page = use_dma32 ? alloc_page(__GFP_ZERO | GFP_DMA32) : + alloc_page(__GFP_ZERO); if (unlikely(glob->dummy_read_page == NULL)) { ret = -ENOMEM; @@ -200,7 +201,7 @@ int ttm_device_init(struct ttm_device *bdev, const struct ttm_device_funcs *func if (WARN_ON(vma_manager == NULL)) return -EINVAL; - ret = ttm_global_init(); + ret = ttm_global_init(use_dma32); if (ret) return ret;