Message ID | 20240206114852.8472-1-sumitg@nvidia.com |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel+bounces-54836-ouuuleilei=gmail.com@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:168b:b0:106:860b:bbdd with SMTP id ma11csp1488072dyb; Tue, 6 Feb 2024 04:04:24 -0800 (PST) X-Google-Smtp-Source: AGHT+IHpJJDblR2sVzmqAX4T7U/Jaxr8WuJQjaLxTwir/wGTHZqPGPRszA/Pmak3iqvUwty3Yx9x X-Received: by 2002:a05:6214:5009:b0:68c:92f4:9161 with SMTP id jo9-20020a056214500900b0068c92f49161mr2158359qvb.2.1707221064006; Tue, 06 Feb 2024 04:04:24 -0800 (PST) X-Forwarded-Encrypted: i=2; AJvYcCVBM6rMY9pjKBtMFUrM1Yygtot194xQl2WYPHZ0TIihLVsCcsHjz/2+9w1Xn0gwdICgsJicNa64IYTQcIlYONkymiKT5A== Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id c4-20020ad45ae4000000b0068cb1b44a1fsi1617037qvh.20.2024.02.06.04.04.23 for <ouuuleilei@gmail.com> (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 06 Feb 2024 04:04:23 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-54836-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=SbJYzPp2; arc=fail (signature failed); spf=pass (google.com: domain of linux-kernel+bounces-54836-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-54836-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id BC52E1C235BA for <ouuuleilei@gmail.com>; Tue, 6 Feb 2024 12:04:23 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id CAF96135A58; Tue, 6 Feb 2024 11:49:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="SbJYzPp2" Received: from NAM04-MW2-obe.outbound.protection.outlook.com (mail-mw2nam04on2082.outbound.protection.outlook.com [40.107.101.82]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 62B4F6DCF7; Tue, 6 Feb 2024 11:49:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.101.82 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707220161; cv=fail; b=Oto3bTIxuBnAFJKy7kXrF1JqHyOmX7eut5hcWp50riC58QOrESWdsf8McCNm+wGRBfIssRdobuimu5dZpj41rNrUZ5lB+/9xiBvj2x07/2lh1sVCYHnjXG2VbXJ52TvF59eHjoYr6fyKI28+EPyu90EnDu9w4cccovpXIccV6AA= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707220161; c=relaxed/simple; bh=OoDJ8yMQHJcg25QloJTbWWQ1elVKmK4prq6g1nkpF2U=; h=From:To:CC:Subject:Date:Message-ID:MIME-Version:Content-Type; b=Epv0cKCUcpT+z3ZvyLZx0GSUbUXlUlJGvX5+sE3gBQfaGDcukXhrdEaI352yPAHsl2p1dBu4X1xCd3Hs1FW8r8EFlZd2JT8CubC7Y7S7sl0MYUAtWw0Xsv/GTWE+GDgrUv7Muk9/pbRfGRRn8peIE/yENfJ83xa2s3dCIQj3+yI= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=SbJYzPp2; arc=fail smtp.client-ip=40.107.101.82 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SWHUZmjqX9oAxGzfeGBB6yQOLP1IQ09WqVjiH86siilIWAmLMcFcj40La8OTAPemUiMgScoABvqyS02HsPLMCu1PTxSzQK4uOutSjXWgJHOvpWbQs8QN95VnXh3/W8fGqsB68KXMSTiVD+o9FShnrjaMxFcfd+D82HFRwQWsufBR57ZpP3rr6OHePm2czwCdBEqwp9D1NRfZBPfnmZE0sdDKCZWa459k5h/9ympvkIU4Qy93im+kmeEau9VNYWlWlHOnlTUKOSMU3TdUiGOF5jDWkoSA71wbUMGzkDOaXAEqJjj52x4inM2DzQkpTXs236IPMeIgTsqIFCAsE1kKgQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=emTUHxJY0T+TFBlYA2byvT4Ib94x7/1m1l84cDhhjko=; b=dCe2R+5aaDOumAgzkkWHCkGLZQj0aqKNZgeT+HIJQlHcdPY5aq6+MPyEMhbzaQkA/seYNLuQvkacO+NB7twuE4o0CyBz44fYns+6C0PPBVGr0t9ArL2fMdK+hJl8m3WG6FVXxWhLWP5cK74c1AI26NypI3SqBW1irrlZyh6oWOsqbE06frQhc65DoTT9F6PL9f77VrkoAgU2xcukBJIo3T+NRpVVeNhx4w7qK+LnoqdDk4urEVynKvowQpaFDaKrClwO7AKT8x4GcPSYsTZYJbLqv9cmP2gF9y6PIxumyNRcqveXGgtCzptX/FpOJGRuZTp1RDfJfuE7hC0FaIqV1Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=linaro.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=emTUHxJY0T+TFBlYA2byvT4Ib94x7/1m1l84cDhhjko=; b=SbJYzPp2D2iK4AP7Go7M+j+MSY2B02WM8nDwzdAjW7x2FUNR86u7XaX6VWFJgtowDHtmjtSW/AuPREKcUF6zgZci83SaKPaFhNQZTiKAxKuB6rU9oaikj+wcWqPdgyAlSSGFRhhi25HnvX0Kny8FqLfGDxedrjgiOipdWxAkYcCgyywzOXF9bC/z4c8T688UVGARZgW6mwKgQk/6pDL9xjNX8lRQS+/Q531tmjWPe/l6YjFtf5ZwnqggiyBhwlLQq44n09PT+asUj+1IoyKFQUgTBS0tKETgq4G4rqtV78rdZuNFKL2gZEjiE8VfE7cRGjidGlCQ/F+XmKAERPav+g== Received: from CH5PR03CA0001.namprd03.prod.outlook.com (2603:10b6:610:1f1::29) by SJ2PR12MB7919.namprd12.prod.outlook.com (2603:10b6:a03:4cc::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.8; Tue, 6 Feb 2024 11:49:16 +0000 Received: from CH2PEPF00000099.namprd02.prod.outlook.com (2603:10b6:610:1f1:cafe::e5) by CH5PR03CA0001.outlook.office365.com (2603:10b6:610:1f1::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.36 via Frontend Transport; Tue, 6 Feb 2024 11:49:16 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by CH2PEPF00000099.mail.protection.outlook.com (10.167.244.20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.19 via Frontend Transport; Tue, 6 Feb 2024 11:49:16 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Tue, 6 Feb 2024 03:49:04 -0800 Received: from rnnvmail204.nvidia.com (10.129.68.6) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Tue, 6 Feb 2024 03:49:04 -0800 Received: from sumitg-l4t.nvidia.com (10.127.8.10) by mail.nvidia.com (10.129.68.6) with Microsoft SMTP Server id 15.2.986.41 via Frontend Transport; Tue, 6 Feb 2024 03:49:01 -0800 From: Sumit Gupta <sumitg@nvidia.com> To: <treding@nvidia.com>, <krzysztof.kozlowski@linaro.org>, <jonathanh@nvidia.com>, <maz@kernel.org>, <mark.rutland@arm.com>, <linux-kernel@vger.kernel.org>, <linux-tegra@vger.kernel.org> CC: <amhetre@nvidia.com>, <bbasu@nvidia.com>, <sumitg@nvidia.com> Subject: [Patch] memory: tegra: Skip SID override from Guest VM Date: Tue, 6 Feb 2024 17:18:52 +0530 Message-ID: <20240206114852.8472-1-sumitg@nvidia.com> X-Mailer: git-send-email 2.17.1 X-NVConfidentiality: public Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: <linux-kernel.vger.kernel.org> List-Subscribe: <mailto:linux-kernel+subscribe@vger.kernel.org> List-Unsubscribe: <mailto:linux-kernel+unsubscribe@vger.kernel.org> MIME-Version: 1.0 Content-Type: text/plain X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PEPF00000099:EE_|SJ2PR12MB7919:EE_ X-MS-Office365-Filtering-Correlation-Id: bf71aefd-7208-4bc4-6a52-08dc2709a569 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ggXTJdQgKpZS1qOOIJkmdDuMOMvRihPyxnVyS+qvJTaG9UIFwZ9BBj4X1Xn99yCILAyghzpxTE1gfS9qqYfC1dfau1g1V6Zb/n527EpCZeRtMleS79sHnaP5JtNVI0s/Zj4YhQRkio9W5/MjQPLoOqyalu8rxfPpCgfr4z4vkb0+Yuqx8CAI2aEmlgyE7e9buP5AsPMjrwOS/oVwsk5V3UnSFCIAAOmfJlJt7Sj2xBsw5PDd6zXnCsDk5GhXfXCdgggtSJIQ5l33ZYImHpKA/OeZgiUWGQybsNy2k80ZfwCMscLeIBIBP/oMhaFthFNdR2RhbbQxM+OW28FRNDazx85U0G5UvZVrpBk+yLOi90mh2zRDpUPt1D9EGnepTZD7DmMNBE4kFLBHklETHDbZbFg9nt9/Lvw5rrOC0m31zR0wxQpPO1XvXBDSoOB7FTN4ezPpzbjuDWkTqejApHBUbk4g4I4xHlAHi2AGEKSB5vq2e/GdhJHney8asMAkseMTmcC/9MnyJoGimzXw10I1UHar9MhU3hxG9lsAaw6HqGTfGIbnGbUMRV3M1GXrQTz5xnOUQyfm6uOuEFwDeCBhmkW/NRKWS09wg8U1o04gMqNFEiOZyVxV/GIGOm9DB34HNuaKnRVe1/w+JT1fuuN1xqTyMm9Vk/Vevm7lwf4dvyQaJnsNQqbEvlMLRLAX1f9FTfMPQ6aqRyqtlLsRBkFyUhBNoZIrI0yX0Vvc8Al+o8rCdi5aJmY2joEJItriYYvXLtKZ4MRlROR3k4aQ2d2xbw== X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230031)(4636009)(396003)(346002)(376002)(136003)(39860400002)(230922051799003)(1800799012)(186009)(451199024)(64100799003)(82310400011)(46966006)(40470700004)(36840700001)(40480700001)(1076003)(40460700003)(2616005)(26005)(36860700001)(41300700001)(336012)(356005)(107886003)(478600001)(54906003)(316002)(83380400001)(7696005)(47076005)(6666004)(426003)(7636003)(36756003)(110136005)(2906002)(70206006)(82740400003)(4326008)(8936002)(86362001)(8676002)(5660300002)(70586007)(2101003);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Feb 2024 11:49:16.1543 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: bf71aefd-7208-4bc4-6a52-08dc2709a569 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CH2PEPF00000099.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB7919 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790151034216905429 X-GMAIL-MSGID: 1790151034216905429 |
Series |
memory: tegra: Skip SID override from Guest VM
|
|
Commit Message
Sumit Gupta
Feb. 6, 2024, 11:48 a.m. UTC
MC SID register access is restricted for Guest VM.
So, skip the SID override programming from the Guest VM.
Signed-off-by: Sumit Gupta <sumitg@nvidia.com>
---
drivers/memory/tegra/tegra186.c | 11 +++++++++++
1 file changed, 11 insertions(+)
Comments
On 06/02/2024 12:48, Sumit Gupta wrote: > MC SID register access is restricted for Guest VM. > So, skip the SID override programming from the Guest VM. > > Signed-off-by: Sumit Gupta <sumitg@nvidia.com> > --- > drivers/memory/tegra/tegra186.c | 11 +++++++++++ > 1 file changed, 11 insertions(+) > > diff --git a/drivers/memory/tegra/tegra186.c b/drivers/memory/tegra/tegra186.c > index 1b3183951bfe..df441896b69d 100644 > --- a/drivers/memory/tegra/tegra186.c > +++ b/drivers/memory/tegra/tegra186.c > @@ -10,6 +10,7 @@ > #include <linux/of.h> > #include <linux/of_platform.h> > #include <linux/platform_device.h> > +#include <asm/virt.h> Are you sure this still compile tests? > > #include <soc/tegra/mc.h> > > @@ -118,6 +119,11 @@ static int tegra186_mc_probe_device(struct tegra_mc *mc, struct device *dev) > unsigned int i, index = 0; > u32 sid; > > + if (!is_kernel_in_hyp_mode()) { > + dev_dbg(mc->dev, "Register access not allowed\n"); Doesn't this depend on hypervisor? > + return 0; > + } > + > if (!tegra_dev_iommu_get_stream_id(dev, &sid)) > return 0; Best regards, Krzysztof
On Tue, Feb 06, 2024 at 05:18:52PM +0530, Sumit Gupta wrote: > MC SID register access is restricted for Guest VM. > So, skip the SID override programming from the Guest VM. > > Signed-off-by: Sumit Gupta <sumitg@nvidia.com> Surely this is probed from the DT? Why would the hypervisor put this in the guest's DT if that hypervisor isn't exposing it to the guest? Mark. > --- > drivers/memory/tegra/tegra186.c | 11 +++++++++++ > 1 file changed, 11 insertions(+) > > diff --git a/drivers/memory/tegra/tegra186.c b/drivers/memory/tegra/tegra186.c > index 1b3183951bfe..df441896b69d 100644 > --- a/drivers/memory/tegra/tegra186.c > +++ b/drivers/memory/tegra/tegra186.c > @@ -10,6 +10,7 @@ > #include <linux/of.h> > #include <linux/of_platform.h> > #include <linux/platform_device.h> > +#include <asm/virt.h> > > #include <soc/tegra/mc.h> > > @@ -118,6 +119,11 @@ static int tegra186_mc_probe_device(struct tegra_mc *mc, struct device *dev) > unsigned int i, index = 0; > u32 sid; > > + if (!is_kernel_in_hyp_mode()) { > + dev_dbg(mc->dev, "Register access not allowed\n"); > + return 0; > + } > + > if (!tegra_dev_iommu_get_stream_id(dev, &sid)) > return 0; > > @@ -146,6 +152,11 @@ static int tegra186_mc_resume(struct tegra_mc *mc) > #if IS_ENABLED(CONFIG_IOMMU_API) > unsigned int i; > > + if (!is_kernel_in_hyp_mode()) { > + dev_dbg(mc->dev, "Register access not allowed\n"); > + return 0; > + } > + > for (i = 0; i < mc->soc->num_clients; i++) { > const struct tegra_mc_client *client = &mc->soc->clients[i]; > > -- > 2.17.1 >
Hi Sumit, On Tue, 06 Feb 2024 11:48:52 +0000, Sumit Gupta <sumitg@nvidia.com> wrote: > > MC SID register access is restricted for Guest VM. > So, skip the SID override programming from the Guest VM. > > Signed-off-by: Sumit Gupta <sumitg@nvidia.com> > --- > drivers/memory/tegra/tegra186.c | 11 +++++++++++ > 1 file changed, 11 insertions(+) > > diff --git a/drivers/memory/tegra/tegra186.c b/drivers/memory/tegra/tegra186.c > index 1b3183951bfe..df441896b69d 100644 > --- a/drivers/memory/tegra/tegra186.c > +++ b/drivers/memory/tegra/tegra186.c > @@ -10,6 +10,7 @@ > #include <linux/of.h> > #include <linux/of_platform.h> > #include <linux/platform_device.h> > +#include <asm/virt.h> > > #include <soc/tegra/mc.h> > > @@ -118,6 +119,11 @@ static int tegra186_mc_probe_device(struct tegra_mc *mc, struct device *dev) > unsigned int i, index = 0; > u32 sid; > > + if (!is_kernel_in_hyp_mode()) { > + dev_dbg(mc->dev, "Register access not allowed\n"); > + return 0; > + } > + > if (!tegra_dev_iommu_get_stream_id(dev, &sid)) > return 0; > > @@ -146,6 +152,11 @@ static int tegra186_mc_resume(struct tegra_mc *mc) > #if IS_ENABLED(CONFIG_IOMMU_API) > unsigned int i; > > + if (!is_kernel_in_hyp_mode()) { > + dev_dbg(mc->dev, "Register access not allowed\n"); > + return 0; > + } > + > for (i = 0; i < mc->soc->num_clients; i++) { > const struct tegra_mc_client *client = &mc->soc->clients[i]; > This doesn't look right. Multiple reasons: - This helper really has nothing to do in a driver. This is architectural stuff that is not intended for use outside of arch core code. - My own tegra186 HW doesn't have VHE, since it is ARMv8.0, and this helper will always return 'false'. How could this result in something that still works? Can I get a free CPU upgrade? - If you assign this device to a VM and that the hypervisor doesn't correctly virtualise it, then it is a different device and you should simply advertise it something else. Or even better, fix your hypervisor. Thanks, M.
Hi Marc, On 06/02/2024 12:17, Marc Zyngier wrote: > Hi Sumit, > > On Tue, 06 Feb 2024 11:48:52 +0000, > Sumit Gupta <sumitg@nvidia.com> wrote: >> >> MC SID register access is restricted for Guest VM. >> So, skip the SID override programming from the Guest VM. >> >> Signed-off-by: Sumit Gupta <sumitg@nvidia.com> >> --- >> drivers/memory/tegra/tegra186.c | 11 +++++++++++ >> 1 file changed, 11 insertions(+) >> >> diff --git a/drivers/memory/tegra/tegra186.c b/drivers/memory/tegra/tegra186.c >> index 1b3183951bfe..df441896b69d 100644 >> --- a/drivers/memory/tegra/tegra186.c >> +++ b/drivers/memory/tegra/tegra186.c >> @@ -10,6 +10,7 @@ >> #include <linux/of.h> >> #include <linux/of_platform.h> >> #include <linux/platform_device.h> >> +#include <asm/virt.h> >> >> #include <soc/tegra/mc.h> >> >> @@ -118,6 +119,11 @@ static int tegra186_mc_probe_device(struct tegra_mc *mc, struct device *dev) >> unsigned int i, index = 0; >> u32 sid; >> >> + if (!is_kernel_in_hyp_mode()) { >> + dev_dbg(mc->dev, "Register access not allowed\n"); >> + return 0; >> + } >> + >> if (!tegra_dev_iommu_get_stream_id(dev, &sid)) >> return 0; >> >> @@ -146,6 +152,11 @@ static int tegra186_mc_resume(struct tegra_mc *mc) >> #if IS_ENABLED(CONFIG_IOMMU_API) >> unsigned int i; >> >> + if (!is_kernel_in_hyp_mode()) { >> + dev_dbg(mc->dev, "Register access not allowed\n"); >> + return 0; >> + } >> + >> for (i = 0; i < mc->soc->num_clients; i++) { >> const struct tegra_mc_client *client = &mc->soc->clients[i]; >> > > This doesn't look right. Multiple reasons: > > - This helper really has nothing to do in a driver. This is > architectural stuff that is not intended for use outside of arch > core code. We see a few other drivers using this ... drivers/perf/arm_pmuv3.c drivers/clocksource/arm_arch_timer.c drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h drivers/hwtracing/coresight/coresight-etm4x-core.c drivers/hwtracing/coresight/coresight-etm4x-core.c drivers/irqchip/irq-apple-aic.c We were looking for a way to determine if the OS is a guest OS or not. However, I can see that this is a ARM64 specific API and so probably the above are only compiled for ARM64. Interestingly, the AMD driver implements the following ... static inline bool is_virtual_machine(void) { #if defined(CONFIG_X86) return boot_cpu_has(X86_FEATURE_HYPERVISOR); #elif defined(CONFIG_ARM64) return !is_kernel_in_hyp_mode(); #else return false; #endif } > - My own tegra186 HW doesn't have VHE, since it is ARMv8.0, and this > helper will always return 'false'. How could this result in > something that still works? Can I get a free CPU upgrade? I thought this API just checks to see if we are in EL2? > - If you assign this device to a VM and that the hypervisor doesn't > correctly virtualise it, then it is a different device and you > should simply advertise it something else. Or even better, fix your > hypervisor. Sumit can add some more details on why we don't completely disable the device for guest OSs. Jon
Hi Jon, On Tue, 06 Feb 2024 12:28:27 +0000, Jon Hunter <jonathanh@nvidia.com> wrote: > > Hi Marc, > > On 06/02/2024 12:17, Marc Zyngier wrote: > > Hi Sumit, > > > > On Tue, 06 Feb 2024 11:48:52 +0000, > > Sumit Gupta <sumitg@nvidia.com> wrote: > >> > >> MC SID register access is restricted for Guest VM. > >> So, skip the SID override programming from the Guest VM. > >> > >> Signed-off-by: Sumit Gupta <sumitg@nvidia.com> > >> --- > >> drivers/memory/tegra/tegra186.c | 11 +++++++++++ > >> 1 file changed, 11 insertions(+) > >> > >> diff --git a/drivers/memory/tegra/tegra186.c b/drivers/memory/tegra/tegra186.c > >> index 1b3183951bfe..df441896b69d 100644 > >> --- a/drivers/memory/tegra/tegra186.c > >> +++ b/drivers/memory/tegra/tegra186.c > >> @@ -10,6 +10,7 @@ > >> #include <linux/of.h> > >> #include <linux/of_platform.h> > >> #include <linux/platform_device.h> > >> +#include <asm/virt.h> > >> #include <soc/tegra/mc.h> > >> @@ -118,6 +119,11 @@ static int tegra186_mc_probe_device(struct > >> tegra_mc *mc, struct device *dev) > >> unsigned int i, index = 0; > >> u32 sid; > >> + if (!is_kernel_in_hyp_mode()) { > >> + dev_dbg(mc->dev, "Register access not allowed\n"); > >> + return 0; > >> + } > >> + > >> if (!tegra_dev_iommu_get_stream_id(dev, &sid)) > >> return 0; > >> @@ -146,6 +152,11 @@ static int tegra186_mc_resume(struct > >> tegra_mc *mc) > >> #if IS_ENABLED(CONFIG_IOMMU_API) > >> unsigned int i; > >> + if (!is_kernel_in_hyp_mode()) { > >> + dev_dbg(mc->dev, "Register access not allowed\n"); > >> + return 0; > >> + } > >> + > >> for (i = 0; i < mc->soc->num_clients; i++) { > >> const struct tegra_mc_client *client = &mc->soc->clients[i]; > >> > > > > This doesn't look right. Multiple reasons: > > > > - This helper really has nothing to do in a driver. This is > > architectural stuff that is not intended for use outside of arch > > core code. > > We see a few other drivers using this ... > > drivers/perf/arm_pmuv3.c > drivers/clocksource/arm_arch_timer.c These two are definitely part of the CPU architecture. > drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h This is just a bug. Please don't copy this stuff. > drivers/hwtracing/coresight/coresight-etm4x-core.c > drivers/hwtracing/coresight/coresight-etm4x-core.c > drivers/irqchip/irq-apple-aic.c These are also part of the CPU architecture. > > We were looking for a way to determine if the OS is a guest OS or > not. However, I can see that this is a ARM64 specific API and so > probably the above are only compiled for ARM64. Interestingly, the AMD > driver implements the following ... > > static inline bool is_virtual_machine(void) > { > #if defined(CONFIG_X86) > return boot_cpu_has(X86_FEATURE_HYPERVISOR); > #elif defined(CONFIG_ARM64) > return !is_kernel_in_hyp_mode(); > #else > return false; > #endif > } This stuff should simply be ripped out and burned. Whoever wrote it didn't understand how this works. > > > - My own tegra186 HW doesn't have VHE, since it is ARMv8.0, and this > > helper will always return 'false'. How could this result in > > something that still works? Can I get a free CPU upgrade? > > I thought this API just checks to see if we are in EL2? It does. And that's the problem. On ARMv8.0, we run the Linux kernel at EL1. Tegra186 is ARMv8.0 (Denver + A57). So as written, this change breaks the very platform it intends to support. > > > - If you assign this device to a VM and that the hypervisor doesn't > > correctly virtualise it, then it is a different device and you > > should simply advertise it something else. Or even better, fix your > > hypervisor. > > Sumit can add some more details on why we don't completely disable the > device for guest OSs. It's not about disabling it. It is about correctly supporting it (providing full emulation for it), or advertising it as something different so that SW can handle it differently. Poking into the internals of how the kernel is booted for a driver that isn't tied to the core architecture (because it would need to access system registers, for example) is not an acceptable outcome. Thanks, M.
On Tue, 06 Feb 2024 12:51:35 +0000, Jon Hunter <jonathanh@nvidia.com> wrote: > > > On 06/02/2024 12:28, Jon Hunter wrote: > > ... > > >> - My own tegra186 HW doesn't have VHE, since it is ARMv8.0, and this > >> helper will always return 'false'. How could this result in > >> something that still works? Can I get a free CPU upgrade? > > > > I thought this API just checks to see if we are in EL2? > > > Sorry to add a bit more info, I see EL2 is used for hypervisor [0], > but on my Tegra186 with no hypervisor I see ... > > CPU: All CPU(s) started at EL2 Yes. and yet the kernel runs at EL1 (on ARMv8.0, we can't run the kernel at EL2 at all). M.
On Tue, 06 Feb 2024 14:07:10 +0000, "Thierry Reding" <thierry.reding@gmail.com> wrote: > > [1 <text/plain; UTF-8 (quoted-printable)>] > On Tue Feb 6, 2024 at 1:53 PM CET, Marc Zyngier wrote: > > On Tue, 06 Feb 2024 12:28:27 +0000, Jon Hunter <jonathanh@nvidia.com> wrote: > > > On 06/02/2024 12:17, Marc Zyngier wrote: > [...] > > > > - My own tegra186 HW doesn't have VHE, since it is ARMv8.0, and this > > > > helper will always return 'false'. How could this result in > > > > something that still works? Can I get a free CPU upgrade? > > > > > > I thought this API just checks to see if we are in EL2? > > > > It does. And that's the problem. On ARMv8.0, we run the Linux kernel > > at EL1. Tegra186 is ARMv8.0 (Denver + A57). So as written, this change > > breaks the very platform it intends to support. > > To clarify, the code that accesses these registers is shared across > Tegra186 and later chips. Tegra194 and later do support ARMv8.1 VHE. But even on these machines that are VHE-capable, not running at EL2 doesn't mean we're running as a guest. The user can force the kernel to stick to EL1, using a command-line option such as kvm-arm.mode=nvhe which will force the kernel to stay at EL1 while deploying KVM at EL2. > Granted, if it always returns false on Tegra186 that's not what we > want. I'm glad we agree here. > > > > - If you assign this device to a VM and that the hypervisor doesn't > > > > correctly virtualise it, then it is a different device and you > > > > should simply advertise it something else. Or even better, fix your > > > > hypervisor. > > > > > > Sumit can add some more details on why we don't completely disable the > > > device for guest OSs. > > > > It's not about disabling it. It is about correctly supporting it > > (providing full emulation for it), or advertising it as something > > different so that SW can handle it differently. > > It's really not a different device. It's exactly the same device except > that accessing some registers isn't permitted. We also can't easily > remove parts of the register region from device tree because these are > intermixed with other registers that we do want access to. But that's the definition of being a different device. It has a different programming interface, hence it is different. The fact that it is the same HW block mediated by a hypervisor doesn't really change that. > > Poking into the internals of how the kernel is booted for a driver > > that isn't tied to the core architecture (because it would need to > > access system registers, for example) is not an acceptable outcome. > > So what would be the better option? Use a different compatible string to > make the driver handle the device differently? Or adding a custom > property to the device tree node to mark this as running in a > virtualized environment? A different compatible string would be my preferred option. An extra property would work as well. As far as I am concerned, these two options are the right way to express the fact that you have something that isn't quite like the real thing. > Perhaps we can reuse the top-level hypervisor node? That seems to only > ever have been used for Xen on 32-bit ARM, so not sure if that'd still > be appropriate. I'd shy away from this. You would be deriving properties from a hypervisor implementation, instead of expressing those properties directly. In my experience, the direct method is always preferable. Thanks, M.
On Tue, 06 Feb 2024 17:08:42 +0000, "Thierry Reding" <thierry.reding@gmail.com> wrote: > > [1 <text/plain; UTF-8 (quoted-printable)>] > On Tue Feb 6, 2024 at 3:54 PM CET, Marc Zyngier wrote: > > On Tue, 06 Feb 2024 14:07:10 +0000, > > "Thierry Reding" <thierry.reding@gmail.com> wrote: > > > > > > [1 <text/plain; UTF-8 (quoted-printable)>] > > > On Tue Feb 6, 2024 at 1:53 PM CET, Marc Zyngier wrote: > > > > On Tue, 06 Feb 2024 12:28:27 +0000, Jon Hunter <jonathanh@nvidia.com> wrote: > > > > > On 06/02/2024 12:17, Marc Zyngier wrote: > > > [...] > > > > > > - My own tegra186 HW doesn't have VHE, since it is ARMv8.0, and this > > > > > > helper will always return 'false'. How could this result in > > > > > > something that still works? Can I get a free CPU upgrade? > > > > > > > > > > I thought this API just checks to see if we are in EL2? > > > > > > > > It does. And that's the problem. On ARMv8.0, we run the Linux kernel > > > > at EL1. Tegra186 is ARMv8.0 (Denver + A57). So as written, this change > > > > breaks the very platform it intends to support. > > > > > > To clarify, the code that accesses these registers is shared across > > > Tegra186 and later chips. Tegra194 and later do support ARMv8.1 VHE. > > > > But even on these machines that are VHE-capable, not running at EL2 > > doesn't mean we're running as a guest. The user can force the kernel > > to stick to EL1, using a command-line option such as kvm-arm.mode=nvhe > > which will force the kernel to stay at EL1 while deploying KVM at EL2. > > > > > Granted, if it always returns false on Tegra186 that's not what we > > > want. > > > > I'm glad we agree here. > > > > > > > > - If you assign this device to a VM and that the hypervisor doesn't > > > > > > correctly virtualise it, then it is a different device and you > > > > > > should simply advertise it something else. Or even better, fix your > > > > > > hypervisor. > > > > > > > > > > Sumit can add some more details on why we don't completely disable the > > > > > device for guest OSs. > > > > > > > > It's not about disabling it. It is about correctly supporting it > > > > (providing full emulation for it), or advertising it as something > > > > different so that SW can handle it differently. > > > > > > It's really not a different device. It's exactly the same device except > > > that accessing some registers isn't permitted. We also can't easily > > > remove parts of the register region from device tree because these are > > > intermixed with other registers that we do want access to. > > > > But that's the definition of being a different device. It has a > > different programming interface, hence it is different. The fact that > > it is the same HW block mediated by a hypervisor doesn't really change > > that. > > The programming model isn't really different in these cases, but rather > restricted. I think a compatible string is a suboptimal way to describe > this. It *is* different. If it wasn't different, you wouldn't need this patch. I'm puzzled that we have to argue on *that*. You can call it restricted, I call it broken. In both case, it is a *different* programming interface as you can't use existing SW for it. > > > > > Poking into the internals of how the kernel is booted for a driver > > > > that isn't tied to the core architecture (because it would need to > > > > access system registers, for example) is not an acceptable outcome. > > > > > > So what would be the better option? Use a different compatible string to > > > make the driver handle the device differently? Or adding a custom > > > property to the device tree node to mark this as running in a > > > virtualized environment? > > > > A different compatible string would be my preferred option. An extra > > property would work as well. As far as I am concerned, these two > > options are the right way to express the fact that you have something > > that isn't quite like the real thing. > > Coincidentally there's another discussion with a lot of similarities > regarding simulated platforms. For these it's usually less about the > register set being restricted and more about certain quirks that are > needed which will not ultimately be necessary for silicon. > > This could be a timeout that's longer in simulation, or it could be > certain programming that would be needed in silicon but isn't necessary > or functional in simulation (think I/O calibration, that sort of thing). > One could argue that these are also different devices when in simulation > but they really aren't. They're more like an approximation of the actual > device that will be in silicon chips. Simulation/DV environments are a very different kettle of fish. You generally treat passing time with a scaling factor, and you are likely to run very hacked-up SW stack anyway. In any case, this is not relevant to upstream stuff, unless you plan to ship your emulation environment. > Another problem that both of the cases have in common is that they are > parameters that usually apply to the entire system. For some devices it > is easier to parameterize via DT (for example for certain devices we > have bindings with special register regions that are only available in > host OS mode), but for others this may not be true. Adding extra > compatible strings for virtualization/simulation is going to get quite > complex very quickly if we need to differentiate between all of these > scenarios. That's the price you pay for these inconsistencies. If your "HW" has a lot of variability and that you can't discover its capabilities from SW, then it either badly designed, badly implemented, badly emulated, or any combination thereof. In any case, you get to keep the pieces. > > > > Perhaps we can reuse the top-level hypervisor node? That seems to only > > > ever have been used for Xen on 32-bit ARM, so not sure if that'd still > > > be appropriate. > > > > I'd shy away from this. You would be deriving properties from a > > hypervisor implementation, instead of expressing those properties > > directly. In my experience, the direct method is always preferable. > > I would generally agree. However, I think especially the compatible > string solution could turn very ugly for this. If we express these > properties via compatible strings we may very well end up with many > different compatible strings to cover all cases. > > Say you've got one hypervisor that changes the programming model in a > certain way and a second hypervisor that constrains in a different way. > Do we now need one compatible string for each hypervisor? Do we add > compatible strings for each restriction and have potentially very long > compatible string lists? Separate properties would work slightly better > for this. Again, the job of a hypervisor is to offer an architecturally correct view of some HW, emulated or not. If your hypervisors are implementing a large variety of diverging behaviours, SW needs to be able to distinguish between those. You can either add properties, use compat strings, or use a discovery protocol implemented by the device. In any case, each deviation needs to be uniquely identifiable, and be described either in FW or by the device itself, if only because Linux isn't the only game in town. > There are some cases where we can use register contents to determine > what the OS is allowed to do, but these registers don't exist for all HW > blocks. We may be able to get more added to new chips, but we obviously > can't retroactively add them for existing ones. > > A central node or property would at least allow broad parameterization. > I would hope that at least hypervisor implementations don't vary too > much in terms of what they restrict and what they don't, so perhaps it > wouldn't be that bad. Perhaps that's also overly optimistic. Top level properties are no good unless what they express is forever immutable and described upfront. Identifying a hypervisor doesn't do that, and most of the time there will be all sorts of *variable* properties that need to be further discovered by a mechanism or another. In my (surely very limited) experience at writing hypervisors for some time, this eventually becomes an unmaintainable mess. You are of course free to do that in the drivers you maintain as long as you don't break my own toys, but I'd urge you to reconsider this and explore other possibilities. Thanks, M.
diff --git a/drivers/memory/tegra/tegra186.c b/drivers/memory/tegra/tegra186.c index 1b3183951bfe..df441896b69d 100644 --- a/drivers/memory/tegra/tegra186.c +++ b/drivers/memory/tegra/tegra186.c @@ -10,6 +10,7 @@ #include <linux/of.h> #include <linux/of_platform.h> #include <linux/platform_device.h> +#include <asm/virt.h> #include <soc/tegra/mc.h> @@ -118,6 +119,11 @@ static int tegra186_mc_probe_device(struct tegra_mc *mc, struct device *dev) unsigned int i, index = 0; u32 sid; + if (!is_kernel_in_hyp_mode()) { + dev_dbg(mc->dev, "Register access not allowed\n"); + return 0; + } + if (!tegra_dev_iommu_get_stream_id(dev, &sid)) return 0; @@ -146,6 +152,11 @@ static int tegra186_mc_resume(struct tegra_mc *mc) #if IS_ENABLED(CONFIG_IOMMU_API) unsigned int i; + if (!is_kernel_in_hyp_mode()) { + dev_dbg(mc->dev, "Register access not allowed\n"); + return 0; + } + for (i = 0; i < mc->soc->num_clients; i++) { const struct tegra_mc_client *client = &mc->soc->clients[i];