From patchwork Fri Mar 1 08:22:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ho-Ren (Jack) Chuang" X-Patchwork-Id: 21191 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2097:b0:108:e6aa:91d0 with SMTP id gs23csp934361dyb; Fri, 1 Mar 2024 00:24:19 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWJ5KydSlL6vd2223AnZphEAT21ToBqWZuWvXgXhkRaWSaT1gI/BiBRbpcgK76hi0EygLEJSV7ucbZgaeDmPZfO8wHJLw== X-Google-Smtp-Source: AGHT+IGy20PwWKxAHutrr4/G3ufLzkqIPOPJ8VDG+jPZi7ziCdUNKJZ5+dUbCNxzAH/Xd1YnJ+JE X-Received: by 2002:a05:6e02:1a26:b0:365:316b:23d3 with SMTP id g6-20020a056e021a2600b00365316b23d3mr1170475ile.19.1709281459158; Fri, 01 Mar 2024 00:24:19 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1709281459; cv=pass; d=google.com; s=arc-20160816; b=f4rFBX81GUA0B/nwzA26l8G9ngJ3RInCJtcLjHOmOZch8dArobwR6CFPPaghDpKf+m w2NlsZJkOhgzNNoh4/2QdscFxjiOgSau7dtcIkQ9poEAEuisSieIhCPHXjV1UjWMfHaC LggeKHCU13rKCbll1WjZ4uUkb2o7kaZb+/cUqQed22kSY1NseW0trE01PSWFLODE1k0G 60nddI23p7hMboxwBdsS5TwP9edVtW8SKFA8Z2vX7M14Pqj9rLOYZVmRBj3+vOaw1fb6 pQIMyiG2In5gADaJ7soS5Cga8Z/UKgAm1w9OXbdbK3l5h8n6PlnnUOSZos/5q/HwSuDf waFA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:message-id:date:subject:cc:to :from:dkim-signature; bh=jbrAz4lnEz25kwMLB527PTKTo91cgL6BvX6RpZZhxe0=; fh=wUfkwpcXbxgDqyicg9awwqJAydN8Kej5HP+RzKK1hGI=; b=QfSsUXugwOacQksF2k11SqOV3oSMR6c3djdk5txVaMv95fDvZUhT3X2+cMgIFp5Eu2 czgwKTkEec92L75VMv1JpEkckJGOGuaHtz8Ur81kIe0UGARJWiNwSnoEB638AFCj/Dh7 H1LVTc0/YmnOEAZXXNs3A7l2AiLUh5N2qgHjN3Lm5Eriu+p7DOP1UNnHkob38L7BSTVg LBy2z/0OPmS6jKitbr5yYIsjRKFU+wLnf/MOO2JK17jAuTOQ6/42tcvdPY71WZtIy3VZ JJG5/UtAQ0aPeRbviNMPC++kadE5So5JZKN3Ze/7TQe75dsBopHtMy5sKwoAa7IDcjZm 1PZQ==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=bvxrLYFh; arc=pass (i=1 spf=pass spfdomain=bytedance.com dkim=pass dkdomain=bytedance.com dmarc=pass fromdomain=bytedance.com); spf=pass (google.com: domain of linux-kernel+bounces-88069-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-88069-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id u15-20020a63f64f000000b005dbd518e89esi3084139pgj.761.2024.03.01.00.24.19 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Mar 2024 00:24:19 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-88069-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=bvxrLYFh; arc=pass (i=1 spf=pass spfdomain=bytedance.com dkim=pass dkdomain=bytedance.com dmarc=pass fromdomain=bytedance.com); spf=pass (google.com: domain of linux-kernel+bounces-88069-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-88069-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id EEA2028A818 for ; Fri, 1 Mar 2024 08:24:18 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D5FF169DF4; Fri, 1 Mar 2024 08:24:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="bvxrLYFh" Received: from mail-qk1-f176.google.com (mail-qk1-f176.google.com [209.85.222.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 17B072AF0D for ; Fri, 1 Mar 2024 08:24:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709281442; cv=none; b=cwe8Bi0MPnQ1lR8WmbCzfkwU6B7atyj+P+uwHAgki8GV8v+e26CIF4akbEglq26qfQLJ3EHHsv6DgiLz8RvqT/KRKFPmRMUpblR8GXkxNegkKkwP1RC4OsyOTZ+e3hksCj9Ks9Yo2FHxbepUccJMEs+KDNFSrrsC1e0shF8daqE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709281442; c=relaxed/simple; bh=4eat6mnFfXzoCJ3q1jO+074FtyB2bjfelHIkr8Ir38k=; h=From:To:Cc:Subject:Date:Message-Id:MIME-Version; b=Hb6tEMXWum2Kavk2UxxPqXHyK+3uUF/eiPXpG5zvL7ve/DZ3njwHtv36/Nc99hgaZ3vAUGBfk0SCg141YNpMwzdtPLAP8/3rNkzHGUb9uXvUID8cyZv59qtb+2QyT8S8XXCriVnjC+L2tduacVmIeiQyhF9uBigpthLpXMsZ5Mk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=bvxrLYFh; arc=none smtp.client-ip=209.85.222.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Received: by mail-qk1-f176.google.com with SMTP id af79cd13be357-787bb013422so106869885a.0 for ; Fri, 01 Mar 2024 00:24:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1709281440; x=1709886240; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=jbrAz4lnEz25kwMLB527PTKTo91cgL6BvX6RpZZhxe0=; b=bvxrLYFhGccIDpTcA8STZO/EWHDLnSss9a9yBaMr4dRvcfekUbs8s7vldrjrBTGnxX aKnIaCNVgzRJrn0uvxW+FEPk7GaW96lU4e1jzAUeXdf6oN6r8MbLq9/KHK6FJISywjvr caQ5a5mCbfmOdpvadUnvIDY4JK/DLOEbpqasEex5XHsVJ3DcZol56GYvbaiWoX0e7Ni3 5D8a8BpLb5wXlthMgwPOQPeZCqFldpPjgrJcUrxj6cOx1kA/mS4nuJHwQ9uKQdEoBLA+ lB68cbZF7P1iQ5KyxQFo31UwfBT2NVq4AhKRvT1ofpoNuffREWfsnn1bETYAW0dGDOWC gtLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709281440; x=1709886240; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=jbrAz4lnEz25kwMLB527PTKTo91cgL6BvX6RpZZhxe0=; b=gJ3OS6x6aBSfnUdEB3/NOC5zeTd/wJa2NF494aE3ljJYDgnSVlMkNefug7YejmRI3w 5VOHBE+Moy80tjeBsWZhXqpqrOUa6PO+FDAB1kziQrKAdUeUoBFzCRRdofEAUyHToNEm 025nhYMX+f2HaQNFuUwyQOEUagz+8qVWpSNxMVKK9eidaV9BLKznYVstWwYIrelA0oCV zAysLyF6A3uuQwwNr2LVsB24C9y2q5LiihK2AaGgXPfPcLh1PhKhTS3qx3tWdpUx4WdV OxfTSbU9u6ClZ7xgIkhujMhxCvofiNjBrEp0rFlynx4CH6MhG9R47fap3WAS+Yyl821N vJgw== X-Forwarded-Encrypted: i=1; AJvYcCVnx2a5bTmGhsXvGotYPdYa1CS0hPjO1/tW0AOQJd4k3WRZyRm+Jz4tto9C/LrtyHl4m6lpujCWgeAnSHybBB0JhFlAtwOhFPnUlfVO X-Gm-Message-State: AOJu0Yy3OPTI6+7a3JsqkYUEOPw6+ZgPGHlM3W+3VqjfKaKJ+aomCRyI wsncn7u/8DHjOj1/lal8u/1TrKi4+qqWb98sMp/aN8UhKXV/lvjc8nu04crEIj0= X-Received: by 2002:a0c:c20a:0:b0:68f:43f6:4834 with SMTP id l10-20020a0cc20a000000b0068f43f64834mr990776qvh.26.1709281440109; Fri, 01 Mar 2024 00:24:00 -0800 (PST) Received: from n231-228-171.byted.org ([130.44.215.123]) by smtp.gmail.com with ESMTPSA id y19-20020a0cd993000000b0068fc392f526sm1631907qvj.127.2024.03.01.00.23.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Mar 2024 00:23:59 -0800 (PST) From: "Ho-Ren (Jack) Chuang" To: "Hao Xiang" , "Gregory Price" , aneesh.kumar@linux.ibm.com, mhocko@suse.com, tj@kernel.org, john@jagalactic.com, "Eishan Mirakhur" , "Vinicius Tavares Petrucci" , "Ravis OpenSrc" , "Alistair Popple" , "Rafael J. Wysocki" , Len Brown , Andrew Morton , Dave Jiang , Dan Williams , Jonathan Cameron , Huang Ying , "Ho-Ren (Jack) Chuang" , linux-acpi@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: "Ho-Ren (Jack) Chuang" , "Ho-Ren (Jack) Chuang" , linux-cxl@vger.kernel.org, qemu-devel@nongnu.org Subject: [PATCH v1 0/1] Improved Memory Tier Creation for CPUless NUMA Nodes Date: Fri, 1 Mar 2024 08:22:44 +0000 Message-Id: <20240301082248.3456086-1-horenchuang@bytedance.com> X-Mailer: git-send-email 2.20.1 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1792311515212217568 X-GMAIL-MSGID: 1792311515212217568 The memory tiering component in the kernel is functionally useless for CPUless memory/non-DRAM devices like CXL1.1 type3 memory because the nodes are lumped together in the DRAM tier. https://lore.kernel.org/linux-mm/PH0PR08MB7955E9F08CCB64F23963B5C3A860A@PH0PR08MB7955.namprd08.prod.outlook.com/T/ This patchset automatically resolves the issues. It delays the initialization of memory tiers for CPUless NUMA nodes until they obtain HMAT information at boot time, eliminating the need for user intervention. If no HMAT specified, it falls back to using `default_dram_type`. Example usecase: We have CXL memory on the host, and we create VMs with a new system memory device backed by host CXL memory. We inject CXL memory performance attributes through QEMU, and the guest now sees memory nodes with performance attributes in HMAT. With this change, we enable the guest kernel to construct the correct memory tiering for the memory nodes. Ho-Ren (Jack) Chuang (1): memory tier: acpi/hmat: create CPUless memory tiers after obtaining HMAT info drivers/acpi/numa/hmat.c | 3 ++ include/linux/memory-tiers.h | 6 +++ mm/memory-tiers.c | 76 ++++++++++++++++++++++++++++++++---- 3 files changed, 77 insertions(+), 8 deletions(-)