From patchwork Thu Sep 28 16:45:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dragos Tatulea X-Patchwork-Id: 146286 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:cae8:0:b0:403:3b70:6f57 with SMTP id r8csp3572454vqu; Thu, 28 Sep 2023 13:13:56 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFPPy84XF9F+leciIWB8JFsL1GkCUF6/exrgY3hveRhoox1hJD4OpS0D2GEXrhxhqBcVU/S X-Received: by 2002:a05:6a21:4982:b0:152:cb38:5b47 with SMTP id ax2-20020a056a21498200b00152cb385b47mr2002179pzc.55.1695932036361; Thu, 28 Sep 2023 13:13:56 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1695932036; cv=pass; d=google.com; s=arc-20160816; b=KEYnoJoILWaNAAyHeNkCoksfpeKQ9J0aojRv0EtSLzlB5W84Aux96MSgT4xgD8OlQg 7Ep2nu+TaET53iUs74pXXO3irLqB0KlJCxo6awEHjoZ0aECKnzf929AmH2ThyOqsUPGl jmK1CusvpqZsTrFS5z4Tg0wmvKab3XiNdd1g+DpWbz/nhDLFslTwvtZcjz5bOzejmi17 /qkG3iqZHv0LR4ojnVhdddM/rwLVQdhX23B1MjLNZ4TtKApgz54pXWEG7fb8zbPWiXVs mF8tZWfLT7HVV70jg69beqbIHUXO7iGLR1unUIFXILlcGgHpUO9cYKGC2Bx3CZdrSNOo 3MnQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Ks0yqKFa9s2l5F9bXQdb3gSn8cxS7t2k+S80slwQQFY=; fh=jbL+pc5MLtbO3s2WLiLFfGFg2eVxPvE9Iic3Znyeju8=; b=0LSuuSpMbwIomc3sfAv964roq0z3t7Ks8bBG5r7H3Dr0nFdQClMYHAB3doYoC7NnU3 Pau/kRmqu17uTWl30x7Mn0JXeEPmcJElDAN0F1tDdFiugzP5iHqHPLKY5Sv38iNqey9B WSW1NlyfHVXjiAp/SVcjgmLZxWQAagthJ2NoOx33KpVDPI9v8WDzhzOD9q2EW9OVyg66 V4zLpFyWpYQNovPvIiALfuTgnm62q73ytHDMcJHjt99sDasTRsQlM5jhcER725kAi+XG w7FrZyMkHkhJ7yF6f1QfB0n4mBkcT8BfMoqL/5AzWsOq8wuef6lInOAPDQjTdD+G8xH9 FvLw== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=Df7yFrKv; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: from lipwig.vger.email (lipwig.vger.email. [23.128.96.33]) by mx.google.com with ESMTPS id kl6-20020a170903074600b001c3a05b0b67si11514137plb.566.2023.09.28.13.13.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 28 Sep 2023 13:13:56 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) client-ip=23.128.96.33; Authentication-Results: mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=Df7yFrKv; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id 2498582A88A2; Thu, 28 Sep 2023 09:50:06 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231689AbjI1Qtl (ORCPT + 21 others); Thu, 28 Sep 2023 12:49:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50360 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231796AbjI1Qtj (ORCPT ); Thu, 28 Sep 2023 12:49:39 -0400 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2040.outbound.protection.outlook.com [40.107.236.40]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 815CA1B5; Thu, 28 Sep 2023 09:49:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=G9Wsppf80vBMr8HzzP3DmQcWCTJR3jRl5xh/Rosbex7v18QH6uXVj9KRPrrC2a/Id3TP9KbaENngH41+Fj/qUckG32qquH1ff9gnsOqu2GjtQT57Y7m2MBAZ/EbcGeJYtYQHVCuKNysuF0qWa2v2Pctk1EdVsin6lKt4VXPgNJLBe85PhHbOZLp0hyxh9F4kbSf/tjiLSmrOc7G8IVGgK/vrdn8JHQio3oXbAPjJ/uU7YNLYohAvSUPq8bsBWpHOIJ+4ECb5pAmS076kPq6SxnRDtx8az9aBZgm9Utsx/vJeJe8jaSj2GIQ39sJTGhqaCBVVkhEUn+Nw7mPHiz5k0g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Ks0yqKFa9s2l5F9bXQdb3gSn8cxS7t2k+S80slwQQFY=; b=We0UjEtQZs3AWtOxR7dhfK3Cs21O+ZlQzPXEGZLY1/GQXXMWUUmcmH5Z+n6uCJg3hOvDAKLWXEkLuuD+tHKlxn5IgDy4vFzH+eRAf194com+Wa2hKOgbGESglnDULECoNstJxeTnkdsoQJ3bgYsvWfxOpWCfvI5TTdYGdpl3y6DyUjBBPPV8aQwhhcWg3h61l8F8dYVigQ7bXkVsphR1kdxpoH8QLRkxagX/Tsre3JM4eSidY7thMD5kL1TbFCWVh14XMg/+d/XgWl09MYQofP9GY3POckfInDxwMTtczNzzDUDFA5Z3utV21EFcEVKjubrPQt9UwM/0aKlq5n0clA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.232) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ks0yqKFa9s2l5F9bXQdb3gSn8cxS7t2k+S80slwQQFY=; b=Df7yFrKvLipqr23Zd8b6SwnWZIA/F/d4tru3Jo32Z/fxa247zNKkc6ERpkNFJCARltMrr9vhhTrPF+tNv59evEYu6npTQRwHepmwsybl+5/2VXkeXFMM4xtT0YYLlFBG98yvbBzXvmhy1QdpbM6KZ2k3AkdoQGneq87whdAmzViNgYPMomvrEEZfO9OnQQd4uQkmM80iMEhgJkZHTccQX3qHzc2nqS0vrcqDLHBu+cz+mk+KDdZb8yezKdZa/ONsfR+lGOTP56GS7TYG12nRrsBzt1pzj8bx0GSfRoL0QZ5LiT0fQyys5W+FcpGSMgyMESsNMqnkEmZYtGpG2be2fA== Received: from CY5PR15CA0110.namprd15.prod.outlook.com (2603:10b6:930:7::29) by PH0PR12MB7886.namprd12.prod.outlook.com (2603:10b6:510:26e::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.22; Thu, 28 Sep 2023 16:49:34 +0000 Received: from CY4PEPF0000E9D3.namprd03.prod.outlook.com (2603:10b6:930:7:cafe::36) by CY5PR15CA0110.outlook.office365.com (2603:10b6:930:7::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.25 via Frontend Transport; Thu, 28 Sep 2023 16:49:34 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.232) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.232 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.232; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.232) by CY4PEPF0000E9D3.mail.protection.outlook.com (10.167.241.146) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.15 via Frontend Transport; Thu, 28 Sep 2023 16:49:34 +0000 Received: from drhqmail202.nvidia.com (10.126.190.181) by mail.nvidia.com (10.127.129.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Thu, 28 Sep 2023 09:49:21 -0700 Received: from drhqmail202.nvidia.com (10.126.190.181) by drhqmail202.nvidia.com (10.126.190.181) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Thu, 28 Sep 2023 09:49:21 -0700 Received: from c-237-113-220-225.mtl.labs.mlnx (10.127.8.12) by mail.nvidia.com (10.126.190.181) with Microsoft SMTP Server id 15.2.986.41 via Frontend Transport; Thu, 28 Sep 2023 09:49:18 -0700 From: Dragos Tatulea To: , , "Michael S . Tsirkin" , Jason Wang , Saeed Mahameed , Leon Romanovsky CC: , Dragos Tatulea , , , Subject: [PATCH mlx5-next 01/16] vdpa/mlx5: Expose descriptor group mkey hw capability Date: Thu, 28 Sep 2023 19:45:12 +0300 Message-ID: <20230928164550.980832-3-dtatulea@nvidia.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230928164550.980832-2-dtatulea@nvidia.com> References: <20230928164550.980832-2-dtatulea@nvidia.com> MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000E9D3:EE_|PH0PR12MB7886:EE_ X-MS-Office365-Filtering-Correlation-Id: b6fe711b-33dc-4994-433b-08dbc042e4c9 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: dLzEG8kepgKj4K00FkPk+9JeOis6Msf1MlonDwB0XiGU9ifdbOAkm1qBHtnsyzlYfjb6L7e+rue+W+Stuxabv54PbT3YpqEBzsfiBqWyrGIDC5j1GHHZ0rkQrnZ4L6ZsHQ6u+YxqGaonfysrcPvfvwFEH7tgK+WQZYGjKfWAoONWdst8G/VRNliijgSnob9TkEWzUo9tatR3bhPo+y9CenkNmosejYza+BFsdXAkydamFj23qcLWPiS6dkspldIDvW3bBBBXMgWSzELQn/NhFvo59mOj6XkU6PyBRecTKrVjjPI/jZ9E9vU6HULjiZgBfBBtaUfBDZHMZDzcTUePaR6QnMeBPmQWbLonD0K+u92rRauo1TEvJoScfOVJx313FpDelEXtu6bhbDpr0f1Eg7VSIBp7EVcH60DxRYxE8jB+HqkjMzjHdmiV5N4xTURDV+YfWygB6pa5mDEyZHwPqEHtFdIdI7MFSXj45mVHxOYwfaYqI2qD/t9gfBoBqpyBmc12XluwQjbdRuhxZnF0oR48Nod6xrl3o/BBSTFsjFLkjwIluoc4E7APXpou5ZiJ0vEJHNlrK4+VVLgI+MZ13SdrLECQ1M5vbz+vUfh+lAb8Ky+1/4Px8cF56K30gQoviE+wnOuud+wA7XVp6mvMJDYqUaL2n5j2FNb4MGvQvh1IzOWuXqkFR5LqyA3keDLYuCdvddddzbNFmbrKpWOEXVGCteXS1OQL5J/lpbYF8J7WBLkxtqE3ZlrDmm9epjns X-Forefront-Antispam-Report: CIP:216.228.118.232;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc7edge1.nvidia.com;CAT:NONE;SFS:(13230031)(4636009)(346002)(396003)(376002)(39860400002)(136003)(230922051799003)(64100799003)(186009)(82310400011)(1800799009)(451199024)(46966006)(36840700001)(40470700004)(40460700003)(2906002)(7636003)(2616005)(82740400003)(70586007)(6666004)(1076003)(5660300002)(110136005)(86362001)(478600001)(47076005)(70206006)(54906003)(83380400001)(36860700001)(426003)(356005)(26005)(336012)(4326008)(36756003)(40480700001)(41300700001)(316002)(8936002)(8676002)(4744005);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2023 16:49:34.0733 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b6fe711b-33dc-4994-433b-08dbc042e4c9 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.118.232];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000E9D3.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR12MB7886 X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Thu, 28 Sep 2023 09:50:07 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1778313630881203259 X-GMAIL-MSGID: 1778313630881203259 Necessary for improved live migration flow. Actual support will be added in a downstream patch. Reviewed-by: Gal Pressman Signed-off-by: Dragos Tatulea --- include/linux/mlx5/mlx5_ifc.h | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/include/linux/mlx5/mlx5_ifc.h b/include/linux/mlx5/mlx5_ifc.h index dd8421d021cf..ec15330b970d 100644 --- a/include/linux/mlx5/mlx5_ifc.h +++ b/include/linux/mlx5/mlx5_ifc.h @@ -1231,7 +1231,13 @@ struct mlx5_ifc_virtio_emulation_cap_bits { u8 max_emulated_devices[0x8]; u8 max_num_virtio_queues[0x18]; - u8 reserved_at_a0[0x60]; + u8 reserved_at_a0[0x20]; + + u8 reserved_at_c0[0x13]; + u8 desc_group_mkey_supported[0x1]; + u8 reserved_at_d4[0xc]; + + u8 reserved_at_e0[0x20]; u8 umem_1_buffer_param_a[0x20]; From patchwork Thu Sep 28 16:45:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Dragos Tatulea X-Patchwork-Id: 146268 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:cae8:0:b0:403:3b70:6f57 with SMTP id r8csp3550488vqu; Thu, 28 Sep 2023 12:33:19 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEe3Gp1GYVubKfbToH2YAwOZ+15smHrsIPSHOP7p1IsrQQXuciWSs6yQuXQuSFNakLkULOW X-Received: by 2002:a17:902:ea0c:b0:1c6:3157:29f3 with SMTP id s12-20020a170902ea0c00b001c6315729f3mr2241803plg.36.1695929599327; Thu, 28 Sep 2023 12:33:19 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1695929599; cv=pass; d=google.com; s=arc-20160816; b=HCBVVp2NS4ntnRYF17CO5Thlcd6zXs2KUGYwaaA0Rn+cGOAUM6CaGTeJ1Q/yNVh57P 1MYw7lzz59Qk6cnyjIrzpD/EdFbSG5S/jQPTfqBZ6TFsWCZg2/zN0X9EgLN8npPSEWBm gc4C22pKzyJRajEDxxOmSHezlLTpm/TBqOY5Y1UgPgoqgR3EH+k21RCCja20x8vSUKKN G140e4OafDdbF+8iWUwAs+ApR2AGCEeoazGKJLaRHqGSjANTFciPdL+44F2datJKfF00 X2+juxxH73dPawci6tZNRHWDolzMTMCaH4e154C+vbM1Jj5eE5vMON0Gyp+JoD4GrT0T a+6Q== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Zs2p5Im8kifm/aJ1lEAfNpIB6nh0RoaB6CA/cNIHKDA=; fh=B9udZ+ud0HfwRADyyT9ngGhdamCmR0DEmmvQNJyQkXE=; b=m3Gnbgue8VWHz+SXVGCoIHL+yw1VS1te1x2eDSvMTBjKJYTNPJuDL4zjwp2bS1xBpT k3SU+KB1+oVoZ7KNgrkdUuEKc8SRHvDJp+NeI9Nb4t5/tkXuX26z+FIQHs4UboAIsKdE e0odbF0+kCWw22FUnpNL9Gb429tPskqWPtgxiNuDRTnS7peCEMz7Ll4cf9aiJ2v7+oE/ u1mz3Ff1Ma3BGn8N2LxrR5Lmf+Zianrhi6dgg9+8vfIqIU+/j6SnHFP5MOz0qZkf5R4m hlMWZoJSJkN+0Q3Ac+vSJt7QeQpw2A0oF8mTEqI9DNLXRR3KjTKbCTn22AR+HaCIpCyf w6mA== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=Tu7V1LR0; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id l9-20020a170902f68900b001c60ee79b6esi13276501plg.0.2023.09.28.12.33.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 28 Sep 2023 12:33:19 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=Tu7V1LR0; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 1E9CE81A334C; Thu, 28 Sep 2023 09:50:02 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231784AbjI1Qtu (ORCPT + 21 others); Thu, 28 Sep 2023 12:49:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50436 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231717AbjI1Qtn (ORCPT ); Thu, 28 Sep 2023 12:49:43 -0400 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2067.outbound.protection.outlook.com [40.107.93.67]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 384011A7 for ; Thu, 28 Sep 2023 09:49:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jDSIES7u2eQBZmpcDACVTbq6Ad8YqUj2GX9K/N2pPiqhOgU1arflO5FkFlubop2YVbX5fcvcfSOxjg9Z8S7bNu7qnOPgeKZ89Pr+iceiLmZ9seIqQhofnuu3Aj7IXo/h2rBz6J3eAfw9NEpxm2vC1y0/+J5JMdC6ulR1aX9Eo/99UrkEfL5Y/PhDmfnhby8IGUWT20pHhoD5N48i09tbP6IZGDuBGQgdIxsqSi1nlqtJErMLxYLtT9GA0Q0XH64tvD9Iwrp4w1zOKQzLBX/dn7WUG6Ff8RK9uBHvrVX41BdU+QHGXaL3XrwCf4b1j7pOJTAZpreI4mdxr+NkK8RbQw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Zs2p5Im8kifm/aJ1lEAfNpIB6nh0RoaB6CA/cNIHKDA=; b=Ksw1+PGEvR/xzuOYZuiO4owASpDRcbnJJqXayB6TlSQCN227jsveLc3mJH/Nu7Qw7xy4NSPnFB3XsWjF1uB2XrYhLLOVLA3sw1oC6j8keeJ3VSa5fuHfQZbCwhEdovIsG29qZrqn+AoMmwMWjjmaE64YASZkrU7nDySXD/RlUePvrmXVC+vpYoLKGWp2C1hQKTMNp4kPuaadb+nkTqBklQ7G7RfSDchGFObN5jhHFbLtcUmWMnL2+KWt9EzjMtiFqZv5LEX4R1dmgfHP3KhUKexjU0BDlHtSvHSW0aYq+0mu99gCE3Zo/4Rem7bfcShr9/wvZnhJGcyXvbXk4q7chg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.232) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Zs2p5Im8kifm/aJ1lEAfNpIB6nh0RoaB6CA/cNIHKDA=; b=Tu7V1LR0tp/7qvW+VaL73wYbd2BHqNvVm/1XJc15u+JkXOG4YVrl5VOwe7mO71MmNpkGrnF/cZ0z7x553emgbnITrYOzBnhPscXFxk3Vju+hwIiknHrhjuC0z84JXniGJCLAqW/rera0CcTaOGASSM8HBAF6PpgUjGeXU7/V3lE+mUe0cwfijZvvzYzPYg/t2Kt7Dj+oirDZi3EYkZv87RTWmfNy9liFHy3aP4lwQOQhWLVynbA5BOd93skJKWVsjz1PGl81cgGnc4XVkWgrc+RcS9eHK3d0/kj73Q6jZWhtewhORpjTRUrcEOD6FuMUh4jz9FingTXtoamaEvkENw== Received: from CYZPR14CA0028.namprd14.prod.outlook.com (2603:10b6:930:a0::7) by MW4PR12MB6803.namprd12.prod.outlook.com (2603:10b6:303:20e::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.21; Thu, 28 Sep 2023 16:49:38 +0000 Received: from CY4PEPF0000E9D2.namprd03.prod.outlook.com (2603:10b6:930:a0:cafe::17) by CYZPR14CA0028.outlook.office365.com (2603:10b6:930:a0::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.25 via Frontend Transport; Thu, 28 Sep 2023 16:49:38 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.232) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.232 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.232; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.232) by CY4PEPF0000E9D2.mail.protection.outlook.com (10.167.241.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.19 via Frontend Transport; Thu, 28 Sep 2023 16:49:38 +0000 Received: from drhqmail202.nvidia.com (10.126.190.181) by mail.nvidia.com (10.127.129.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Thu, 28 Sep 2023 09:49:29 -0700 Received: from drhqmail202.nvidia.com (10.126.190.181) by drhqmail202.nvidia.com (10.126.190.181) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Thu, 28 Sep 2023 09:49:28 -0700 Received: from c-237-113-220-225.mtl.labs.mlnx (10.127.8.12) by mail.nvidia.com (10.126.190.181) with Microsoft SMTP Server id 15.2.986.41 via Frontend Transport; Thu, 28 Sep 2023 09:49:26 -0700 From: Dragos Tatulea To: , , "Michael S . Tsirkin" , Jason Wang , Xuan Zhuo CC: , Si-Wei Liu , Subject: [PATCH vhost 02/16] vdpa: introduce dedicated descriptor group for virtqueue Date: Thu, 28 Sep 2023 19:45:13 +0300 Message-ID: <20230928164550.980832-4-dtatulea@nvidia.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230928164550.980832-2-dtatulea@nvidia.com> References: <20230928164550.980832-2-dtatulea@nvidia.com> MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000E9D2:EE_|MW4PR12MB6803:EE_ X-MS-Office365-Filtering-Correlation-Id: c7ad0b10-3665-4d65-9f2e-08dbc042e72e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: kj5BJN6WuunciC+HjKzFmr1Z1UbWRPaNZJJvkWG82YHBBxVX0EUY/J5eFG3LnP/nMCZGiKJ1pCDx14rNcjGiyC2DEgkAW6/KeIGdFdg2rCAlG4Cfz/Fz92ePJCFS7ENxwXk7BqkOUYMD7HHT05XbLmC1ZfYgqKlfqFiwWqIpdKs8VXpl5MQwbYPhrS6czKPKS8FVRhWm1jUSrLuCbdpbxpdaDnc9xkXS8tW9mZXm6RZbej/Oj158huyWPAa+9jmbI3etxCv8sN+7koqUparj8k7Ypw1qFeWjppPOsodCAjuoZKkBCCAF+rukH1/APuhtbfM6X4muFShzOrA7+Osi107nTk5TqKehCWAmtkJRvV/3qqabs+U29VR/J/UaTLtHvp1VLs8ooT/4loPlGCOH7fTwOrU6mmYdlR4fSVOHXJ13KaTSAm9NnFv4BRXEcfoZ+V5rjyHnhJ4pu5kL23zUCk9aY4rNugGeNTE2iBRjpMJO4QHRz3eaOTWcIQmNMtHfGQT/NdzYjCMJ1kyQSNp/blBFHLWFcAX1kppX4tezH7q977m1RZBiSgNl1LhzMpDT9RLQkeAXlEYeCJhn8rsgvT8a9dRHfi6DDPUlc0+u4k+wiGU4wh5+t/lfb2hZPseZ2F3zv8X/jtwKPswnEFCB7GIvpr4QZovNMEpu0gbXW4ARo64727jMYHbt3c/Pmn1/B/8UfXEpMHCR2sNrCvXSVdPjtaQuJbUxyxme4E3ZfQKwIlExJSOaPDyVx3FsUx9f X-Forefront-Antispam-Report: CIP:216.228.118.232;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc7edge1.nvidia.com;CAT:NONE;SFS:(13230031)(4636009)(136003)(376002)(396003)(346002)(39860400002)(230922051799003)(186009)(1800799009)(64100799003)(82310400011)(451199024)(46966006)(40470700004)(36840700001)(40460700003)(336012)(26005)(2906002)(66574015)(6666004)(1076003)(2616005)(426003)(83380400001)(478600001)(8676002)(36860700001)(4326008)(8936002)(5660300002)(47076005)(316002)(70586007)(70206006)(82740400003)(54906003)(7636003)(356005)(110136005)(41300700001)(40480700001)(86362001)(36756003);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2023 16:49:38.1086 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c7ad0b10-3665-4d65-9f2e-08dbc042e72e X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.118.232];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000E9D2.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB6803 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Thu, 28 Sep 2023 09:50:02 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1778311075447864737 X-GMAIL-MSGID: 1778311075447864737 From: Si-Wei Liu In some cases, the access to the virtqueue's descriptor area, device and driver areas (precluding indirect descriptor table in guest memory) may have to be confined to a different address space than where its buffers reside. Without loss of simplicity and generality with already established terminology, let's fold up these 3 areas and call them as a whole as descriptor table group, or descriptor group for short. Specifically, in case of split virtqueues, descriptor group consists of regions for Descriptor Table, Available Ring and Used Ring; for packed virtqueues layout, descriptor group contains Descriptor Ring, Driver and Device Event Suppression structures. The group ID for a dedicated descriptor group can be obtained through a new .get_vq_desc_group() op. If driver implements this op, it means that the descriptor, device and driver areas of the virtqueue may reside in a dedicated group than where its buffers reside, a.k.a the default virtqueue group through the .get_vq_group() op. In principle, the descriptor group may or may not have same group ID as the default group. Even if the descriptor group has a different ID, meaning the vq's descriptor group areas can optionally move to a separate address space than where guest memory resides, the descriptor group may still start from a default address space, same as where its buffers reside. To move the descriptor group to a different address space, .set_group_asid() has to be called to change the ASID binding for the group, which is no different than what needs to be done on any other virtqueue group. On the other hand, the .reset() semantics also applies on descriptor table group, meaning the device reset will clear all ASID bindings and move all virtqueue groups including descriptor group back to the default address space, i.e. in ASID 0. QEMU's shadow virtqueue is going to utilize dedicated descriptor group to speed up map and unmap operations, yielding tremendous downtime reduction by avoiding the full and slow remap cycle in SVQ switching. Signed-off-by: Si-Wei Liu Acked-by: Eugenio Pérez Acked-by: Jason Wang --- include/linux/vdpa.h | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/include/linux/vdpa.h b/include/linux/vdpa.h index 0e652026b776..d376309b99cf 100644 --- a/include/linux/vdpa.h +++ b/include/linux/vdpa.h @@ -204,6 +204,16 @@ struct vdpa_map_file { * @vdev: vdpa device * @idx: virtqueue index * Returns u32: group id for this virtqueue + * @get_vq_desc_group: Get the group id for the descriptor table of + * a specific virtqueue (optional) + * @vdev: vdpa device + * @idx: virtqueue index + * Returns u32: group id for the descriptor table + * portion of this virtqueue. Could be different + * than the one from @get_vq_group, in which case + * the access to the descriptor table can be + * confined to a separate asid, isolating from + * the virtqueue's buffer address access. * @get_device_features: Get virtio features supported by the device * @vdev: vdpa device * Returns the virtio features support by the @@ -360,6 +370,7 @@ struct vdpa_config_ops { /* Device ops */ u32 (*get_vq_align)(struct vdpa_device *vdev); u32 (*get_vq_group)(struct vdpa_device *vdev, u16 idx); + u32 (*get_vq_desc_group)(struct vdpa_device *vdev, u16 idx); u64 (*get_device_features)(struct vdpa_device *vdev); u64 (*get_backend_features)(const struct vdpa_device *vdev); int (*set_driver_features)(struct vdpa_device *vdev, u64 features); From patchwork Thu Sep 28 16:45:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Dragos Tatulea X-Patchwork-Id: 146325 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:cae8:0:b0:403:3b70:6f57 with SMTP id r8csp3633414vqu; Thu, 28 Sep 2023 15:15:48 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGMqi4ZxmEddeRChY7QLndrDP3AUPSMbinxjeDJiAMpd1ES/eQaY62u8rW7q0dVnZSoCjKX X-Received: by 2002:a05:6a00:1396:b0:690:2ad9:1436 with SMTP id t22-20020a056a00139600b006902ad91436mr3273125pfg.7.1695939348256; Thu, 28 Sep 2023 15:15:48 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1695939348; cv=pass; d=google.com; s=arc-20160816; b=SI8ATeOylk15oCDg/9dhRn1uNDYdxDNueoDcnkbMLjunohtXb+HGMyzOvcbCZzrIjA YyZJEYdDXC6PRpC7dWpxZvKSojlAkZiSEZ1xgDC+diL8xKD5Pbc95o5BkDwDsJ0ppNls 3dU2z/4QcDqhGrPEOxUZQ9e+WsA8T+L8zC1YfzjGPB912b7JN6AT1/3EjBJBCQx3/kLt pL3xr8H2jUjF5hGJmxXdttMdowTBkPk8XepRcZOlZ/T2LjlRy4DlmRVgF+Ve8YeoEkPg J/auJwhRz7J9GCqnQywCKPEgancC36tpwuwvnHqRMaVeiTw1J8NRRBUUtQ5qsLLv7CxZ YQyQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=ILKog4btT16IMu9HXAHH95fAm+8LcOyK/4mzDrViAvI=; fh=1xGpd5lngf1TyFM70qClJX4f2pPqDnlBc+KXaB3IJHU=; b=d++nio/wI2U46Vm2IH0kU0WO7Rr9d2arrjyDI4CTB3XjtMbvpGZWN2WMHEqyqMmKBu B3DIDjS6DGS2syorDyO8cKDbZE+aEsCLPXpimFQl1ydm/xISbEx0xJEMuXQUVidsYpeN HdvmwQp2FEwdF99UPCyxrmGQ/rWZM4imP847zVVqHKBJKxqW6wwUTIugAPTr0O6uvyrm uWpOVGG1yG+CJW7hcDXbA2PDKsA1vOe4yMXr6b7jCpGAhkDE/98kF5Azxn+4YKe0RkLG mm66Dh+Q5cIiqQY2a5lc8L0D6kl/c7k/vIya+kk9ZiIO0jMD7GHxcdKFque9kKxDzWAe oXvg== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b="R/ssvQl7"; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: from groat.vger.email (groat.vger.email. [23.128.96.35]) by mx.google.com with ESMTPS id bw8-20020a056a00408800b0068fb95e5401si19056669pfb.65.2023.09.28.15.15.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 28 Sep 2023 15:15:48 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) client-ip=23.128.96.35; Authentication-Results: mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b="R/ssvQl7"; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by groat.vger.email (Postfix) with ESMTP id 6DA6080A0E31; Thu, 28 Sep 2023 09:50:30 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at groat.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231845AbjI1Qty (ORCPT + 21 others); Thu, 28 Sep 2023 12:49:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34418 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231600AbjI1Qtu (ORCPT ); Thu, 28 Sep 2023 12:49:50 -0400 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2084.outbound.protection.outlook.com [40.107.94.84]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 219881B9; Thu, 28 Sep 2023 09:49:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=fSbhbXzUlV0ef70enOM6b5s+aIdlAyUei0cCdbpWfWNTsl+zuujZTXh51T+p9elsJpTu9a/tqPJzgHtSzbuGSmR9PJmKbt79FVf8FZ29WqDiWdiChXT8nWVFHhEryUAVvcD1WEQoUAc+FygmutXevcTrXziK3qTgJiIJr30NhxzMV9FsYBM2T8Kh7A3i860M9TlpNxaKIp5ACmEoN3qy81DKIZOQMPIReJIN1SlPpAJ3a9j00TFPzo8zP10rq0D32+JHRjsWmTj4Xcktve/BH1DkkRMiJ2hUn/lFl9WEIRZOi7sqfH9suMDlcoxbCFtjeFur2WQ4cOuVtEOd3x7BIA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ILKog4btT16IMu9HXAHH95fAm+8LcOyK/4mzDrViAvI=; b=SrxpFqgpFNbAoyQoFI9VNKxnxAFWRoWIkowCN70n4wiXuxYAUULcep0JJ+nXfFdFyVqMe1KWzQRCMNoWew7Ga04BZLQOUbj63G1TtYrNMMwcDbWooLdWhOGF8zR4RX+qL5T1LvLc8sljieR18bTn1x+pZBN2zv0BQbXno8M4AfxiwVnlJMvipasowkFEENNGx36dSKwZXfpTI+b9uq6dihGibFEWzrRIrAYRwJ6PpQtQFPcKGcOap4Q6da6IcoVhtmC/Y+edAlFkMoikPKcZ+Sd0aoTAoyvNA5HUBtGZmb+aUhO5z/AapGFwKdUNJ8dVNiXjvhSOZzacIfiO91lyjw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.233) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ILKog4btT16IMu9HXAHH95fAm+8LcOyK/4mzDrViAvI=; b=R/ssvQl7U7O+rKzpolGLdt7B++/mrp/DYGSWVGZqknhub6e3UFAbS0oVieJJ9dgWsSaxJPY9NO+MHjw4BNiQwTpsUPjx0k3w/0qux6b2IlLWju7/OXV0a58jYd5aHN++JYdZ/XpQ+y0BCDZ1bK0X2nrnZyKDhHOvBDjcYKiz0NTMH/Dg5xFucHOQWGtJpwPjOl2/RkUWEX9XA+qfLPowkC26tVjbiQKekzXql9IPXbmdkCmdFeQ+JooaK0rMAxXt78FyVZW/n1VwmMtlRznQiiZLugQ1z7zygUzVztBqUXQKFhlQhPW63InOC1SciBOHOoyDG/FLbOrzdXjxL4OEIQ== Received: from MW3PR06CA0007.namprd06.prod.outlook.com (2603:10b6:303:2a::12) by SJ0PR12MB6928.namprd12.prod.outlook.com (2603:10b6:a03:47a::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.25; Thu, 28 Sep 2023 16:49:41 +0000 Received: from CO1PEPF000044F8.namprd21.prod.outlook.com (2603:10b6:303:2a:cafe::ba) by MW3PR06CA0007.outlook.office365.com (2603:10b6:303:2a::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.22 via Frontend Transport; Thu, 28 Sep 2023 16:49:40 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.233) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.233 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.233; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.233) by CO1PEPF000044F8.mail.protection.outlook.com (10.167.241.198) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6863.9 via Frontend Transport; Thu, 28 Sep 2023 16:49:40 +0000 Received: from drhqmail203.nvidia.com (10.126.190.182) by mail.nvidia.com (10.127.129.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Thu, 28 Sep 2023 09:49:34 -0700 Received: from drhqmail202.nvidia.com (10.126.190.181) by drhqmail203.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Thu, 28 Sep 2023 09:49:33 -0700 Received: from c-237-113-220-225.mtl.labs.mlnx (10.127.8.12) by mail.nvidia.com (10.126.190.181) with Microsoft SMTP Server id 15.2.986.41 via Frontend Transport; Thu, 28 Sep 2023 09:49:31 -0700 From: Dragos Tatulea To: , , "Michael S . Tsirkin" , Jason Wang CC: , Si-Wei Liu , , , Subject: [PATCH vhost 03/16] vhost-vdpa: introduce descriptor group backend feature Date: Thu, 28 Sep 2023 19:45:14 +0300 Message-ID: <20230928164550.980832-5-dtatulea@nvidia.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230928164550.980832-2-dtatulea@nvidia.com> References: <20230928164550.980832-2-dtatulea@nvidia.com> MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000044F8:EE_|SJ0PR12MB6928:EE_ X-MS-Office365-Filtering-Correlation-Id: a8cf670a-480e-4e43-b516-08dbc042e8c8 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: S99Teqc4zP9Pv5ViUGNESMcnYVwrrMgtuNN0QU/rransQ+H2Zi/bImHZqwxnMVytCObHpEL4aqVs8Y/5cjWHfmDaemMEECldyghzp3RKIkQA4K7Rt8bC+cpe3grtVULanXFDFNFXoU2xrn0+DS+x9GizH0niV9nbsaSEEFyUpIgVRd8hrshXgWROpdGj2ULEheDe0pwtwH8MRXO2+kGUoHWvzEuskMrXiIuHIsTQldhPQlNfsreifFa2RQSQ+MuccWOUEpWWlLYJTfilIuDdg+aSgrouCVuIcVzkhmT6sC5qnXzfLjHxrYd/xR5ibz2S6DDTFLQUasBkZJizDq7/jcQlCpOFzD0oNxPBMIPyMLtoFhq6t/elqt9Uk1eqZorh6SgbD3wzrIH0TTrfXjZ7K95N2hq14pOTAqSPQtffg7xBV0Ezkpjuxay9Xoni5vKeNnEVI9CrNR1TYeWw9mRDB4fKdbCjWy4rNRNzKYpGzgYNHsvNPE3hPSEseSorK2emgfcaX6QHR9wCR+aW8zYBtlo67N++GR0oWvJZK3BUh3bp/qLFjvnNPX3IJ/UUOxbc1j9fqzJrGyPSSLd/XiWvhluT+NuzjV2FLgEqzmfsFxT/RjuB+z2BIalxkl660gEyYdaQWN0KnzJzM2rhIiHtc8G4uH2ei8f30Wl4gu3omZmzkhiFaAjSoNdb17yc7vBaa1vg8zZTN3XoJePv4OX/9Lv8tzG7g5G1w/CzQlDTCNT3JW+70wmPMA+34GIUAyfE X-Forefront-Antispam-Report: CIP:216.228.118.233;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc7edge2.nvidia.com;CAT:NONE;SFS:(13230031)(4636009)(396003)(136003)(346002)(39860400002)(376002)(230922051799003)(186009)(1800799009)(64100799003)(82310400011)(451199024)(36840700001)(40470700004)(46966006)(40460700003)(2906002)(6666004)(2616005)(1076003)(86362001)(110136005)(478600001)(70586007)(70206006)(54906003)(47076005)(36860700001)(336012)(82740400003)(426003)(66574015)(26005)(356005)(7636003)(83380400001)(36756003)(5660300002)(40480700001)(316002)(4326008)(41300700001)(8676002)(8936002);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2023 16:49:40.7961 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a8cf670a-480e-4e43-b516-08dbc042e8c8 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.118.233];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000044F8.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR12MB6928 X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Thu, 28 Sep 2023 09:50:30 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1778321298243751638 X-GMAIL-MSGID: 1778321298243751638 From: Si-Wei Liu Userspace knows if the device has dedicated descriptor group or not by checking this feature bit. It's only exposed if the vdpa driver backend implements the .get_vq_desc_group() operation callback. Userspace trying to negotiate this feature when it or the dependent _F_IOTLB_ASID feature hasn't been exposed will result in an error. Signed-off-by: Si-Wei Liu Acked-by: Eugenio Pérez Acked-by: Jason Wang --- drivers/vhost/vdpa.c | 17 +++++++++++++++++ include/uapi/linux/vhost_types.h | 5 +++++ 2 files changed, 22 insertions(+) diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c index 78379ffd2336..2f21798a37ee 100644 --- a/drivers/vhost/vdpa.c +++ b/drivers/vhost/vdpa.c @@ -389,6 +389,14 @@ static bool vhost_vdpa_can_resume(const struct vhost_vdpa *v) return ops->resume; } +static bool vhost_vdpa_has_desc_group(const struct vhost_vdpa *v) +{ + struct vdpa_device *vdpa = v->vdpa; + const struct vdpa_config_ops *ops = vdpa->config; + + return ops->get_vq_desc_group; +} + static long vhost_vdpa_get_features(struct vhost_vdpa *v, u64 __user *featurep) { struct vdpa_device *vdpa = v->vdpa; @@ -690,6 +698,7 @@ static long vhost_vdpa_unlocked_ioctl(struct file *filep, if (copy_from_user(&features, featurep, sizeof(features))) return -EFAULT; if (features & ~(VHOST_VDPA_BACKEND_FEATURES | + BIT_ULL(VHOST_BACKEND_F_DESC_ASID) | BIT_ULL(VHOST_BACKEND_F_SUSPEND) | BIT_ULL(VHOST_BACKEND_F_RESUME) | BIT_ULL(VHOST_BACKEND_F_ENABLE_AFTER_DRIVER_OK))) @@ -700,6 +709,12 @@ static long vhost_vdpa_unlocked_ioctl(struct file *filep, if ((features & BIT_ULL(VHOST_BACKEND_F_RESUME)) && !vhost_vdpa_can_resume(v)) return -EOPNOTSUPP; + if ((features & BIT_ULL(VHOST_BACKEND_F_DESC_ASID)) && + !(features & BIT_ULL(VHOST_BACKEND_F_IOTLB_ASID))) + return -EINVAL; + if ((features & BIT_ULL(VHOST_BACKEND_F_DESC_ASID)) && + !vhost_vdpa_has_desc_group(v)) + return -EOPNOTSUPP; vhost_set_backend_features(&v->vdev, features); return 0; } @@ -753,6 +768,8 @@ static long vhost_vdpa_unlocked_ioctl(struct file *filep, features |= BIT_ULL(VHOST_BACKEND_F_SUSPEND); if (vhost_vdpa_can_resume(v)) features |= BIT_ULL(VHOST_BACKEND_F_RESUME); + if (vhost_vdpa_has_desc_group(v)) + features |= BIT_ULL(VHOST_BACKEND_F_DESC_ASID); features |= vhost_vdpa_get_backend_features(v); if (copy_to_user(featurep, &features, sizeof(features))) r = -EFAULT; diff --git a/include/uapi/linux/vhost_types.h b/include/uapi/linux/vhost_types.h index 2d827d22cd99..18ad6ae7ab5c 100644 --- a/include/uapi/linux/vhost_types.h +++ b/include/uapi/linux/vhost_types.h @@ -185,5 +185,10 @@ struct vhost_vdpa_iova_range { * DRIVER_OK */ #define VHOST_BACKEND_F_ENABLE_AFTER_DRIVER_OK 0x6 +/* Device may expose the virtqueue's descriptor area, driver area and + * device area to a different group for ASID binding than where its + * buffers may reside. Requires VHOST_BACKEND_F_IOTLB_ASID. + */ +#define VHOST_BACKEND_F_DESC_ASID 0x7 #endif From patchwork Thu Sep 28 16:45:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Dragos Tatulea X-Patchwork-Id: 146370 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:cae8:0:b0:403:3b70:6f57 with SMTP id r8csp3684602vqu; Thu, 28 Sep 2023 17:12:31 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEqUeMWhhRQfX1+Wzh9VU+OjeUIutffrstELPF1J8vKaDAy0HV55VgUHPsxRyI7PbNZyNhr X-Received: by 2002:a17:90a:b94b:b0:273:dffd:664a with SMTP id f11-20020a17090ab94b00b00273dffd664amr2558443pjw.20.1695946351040; Thu, 28 Sep 2023 17:12:31 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1695946351; cv=pass; d=google.com; s=arc-20160816; b=UZpL0NIJPGr3154NnByYbCjMA1ig0MbcyOv+gocRwRziElD7oifKm3KDapPzrQj0HT 6QJD7wlZ/S2B1JLAcYrY12lvQ4h98LSbSKARAovyvIxShZq3NklT69ClpncGBELX9Xpq oK/LVx7ti5UiPkHZNfCEVQnUR6so5pjN47b2qZjuk61IZI4/cyVirQwzl1LRKLPY4oEs BUEKnM8xTAz5DnB2rkKIde5H8UnitWoHQ6xno4P3eu5AkDw+aEmZIL1xzDBboOe1W93j S6bMZWui/5Ke3fPPdPwnugTm2tlxUz1eh7WjJ53UyiCWwetCtdebmvsDeVFGOoOswl8c HBDg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=u64U1vHaUjCMSpg15sklK0flAB7mos1z6H94Kmb4gsM=; fh=1xGpd5lngf1TyFM70qClJX4f2pPqDnlBc+KXaB3IJHU=; b=r1nz/56a7OLeKHOtNQJrQt89ceNNwiBV9jrn5jFPuMOOboFRQUiiXN1nKc4SkyeNK3 kku8YXtIAWJTZF+EMTdv+gHGdqEpi/dnHxObeuSmIuoiq7YdhJSvqoZADGkn6xVG7+xR 9iXh4Bc+iMV1YD1VJzF9WTQQRUaZH3WX+VWBNSk1OGDiBm8PSeUYJt6zQo5bN5/rjqgz fy6xBdSl+uzNxToVen62KSKBhze7T1GiZ+MQlx+nD6fTMTfw0OALofX5c4svSLGJBV97 B3/SQn1IJ7mfHoSzTiHBT/B4A7zgNQhWpBd3KpuXDI5TJ0i/jcrYdbRx4uEov7TS9F+v pkMw== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=RMqUPGWV; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: from howler.vger.email (howler.vger.email. [2620:137:e000::3:4]) by mx.google.com with ESMTPS id cl13-20020a17090af68d00b00262f937bf93si306293pjb.77.2023.09.28.17.12.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 28 Sep 2023 17:12:31 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) client-ip=2620:137:e000::3:4; Authentication-Results: mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=RMqUPGWV; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id C4B2E85BD896; Thu, 28 Sep 2023 09:50:22 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231965AbjI1QuC (ORCPT + 21 others); Thu, 28 Sep 2023 12:50:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34486 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231842AbjI1Qtw (ORCPT ); Thu, 28 Sep 2023 12:49:52 -0400 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2047.outbound.protection.outlook.com [40.107.92.47]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5903BCE3; Thu, 28 Sep 2023 09:49:47 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RIaa/CxF69oMvfRMNfHd5QoO+LxTveHRSVYS+gmUD/hyf5myrZQxmqrMvUuuctGXIlIt6D/qugcr/h4Xbj2sU8sO6BMml6u8K1h90jw5UepdyWMN4NXQ+vdsoDvfBZnImoLaw1SbQQqzj3NB0J3j0LPROJ1YV/NBL6uVy5e6NBtW2S6JiIM682U6vZDCsR93N99w+ASaeABieHia8O41tk92FQZRmKssDNTuqIu3b7+o6wSZDdHBVG+1j3XhEBs0RQmIxGoUSS4gbkhJxb0tp8W6gZd4PxQlZ1p/nMl1PnjxEs4bjk0AwaDF94A7R5qR1sZkD4/Ci6jmk60I6xnzAw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=u64U1vHaUjCMSpg15sklK0flAB7mos1z6H94Kmb4gsM=; b=l89UZgGFsi4ekBIuMsrB9UkGtdT0o+L66g9GXTG9btPS0P+V/FRN446Oe1lIMcikx0UxS14NwVx2xWHBtuBSWbQ1mkXL+iggt2GSbRV2ovipetlNYfsB8Nl39iU7xRbfzbZ5luXmldwh/oz+n6YLgI3bdUa2oDJqbg7eNzzvFE5yJHkpx3wctwwuOmK7jpg5XORUyRpCX6OK/KK54nI58tczd9jAVw/EV3Yw3AY7zX+7JYIrKw1B5QWIyCSDAoZBxJfButBRg4xmlQRlzz/UDDpzx9jgnUUAvy84KNLj6o0QUgdjUAUlHPltOxop4VC+B0XYAXUKLgyD7uZbErrwmw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.233) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=u64U1vHaUjCMSpg15sklK0flAB7mos1z6H94Kmb4gsM=; b=RMqUPGWVQyzJMkle/oL496Jo2Q32omWBRHpGGcNY9nBgPtP6sJ2aIYFq0nJwH0/e7NMxyJZZh4LqicRFcPl/JcG7vAiD0XSgi7a+GrirRVwl/cjFsG412gFk8E7+bDreaod8CNr6dW25Dgvh09LykHCSteKD1Cgz7W/iRCqLs9FZlHkDOwqgn3iQoVlIA8eccRArMZ+LsWR2rQJd7uTkUJU9fdcoCCrtbPhQmAPeV/sBaGagMIbEf8+zcbipSIWAphVeMtdGKni1GUPAX4O1fCePKf8HVKxlFcvCvbzbwCzmn1J+rMqWkRTDmoWuqTRNFHSjWkcDlpga7XkGPjiQ3Q== Received: from MW3PR06CA0017.namprd06.prod.outlook.com (2603:10b6:303:2a::22) by DS0PR12MB7704.namprd12.prod.outlook.com (2603:10b6:8:130::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6813.21; Thu, 28 Sep 2023 16:49:45 +0000 Received: from CO1PEPF000044F8.namprd21.prod.outlook.com (2603:10b6:303:2a:cafe::69) by MW3PR06CA0017.outlook.office365.com (2603:10b6:303:2a::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.22 via Frontend Transport; Thu, 28 Sep 2023 16:49:44 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.233) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.233 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.233; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.233) by CO1PEPF000044F8.mail.protection.outlook.com (10.167.241.198) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6863.9 via Frontend Transport; Thu, 28 Sep 2023 16:49:44 +0000 Received: from drhqmail202.nvidia.com (10.126.190.181) by mail.nvidia.com (10.127.129.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Thu, 28 Sep 2023 09:49:37 -0700 Received: from drhqmail202.nvidia.com (10.126.190.181) by drhqmail202.nvidia.com (10.126.190.181) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Thu, 28 Sep 2023 09:49:37 -0700 Received: from c-237-113-220-225.mtl.labs.mlnx (10.127.8.12) by mail.nvidia.com (10.126.190.181) with Microsoft SMTP Server id 15.2.986.41 via Frontend Transport; Thu, 28 Sep 2023 09:49:35 -0700 From: Dragos Tatulea To: , , "Michael S . Tsirkin" , Jason Wang CC: , Si-Wei Liu , , , Subject: [PATCH vhost 04/16] vhost-vdpa: uAPI to get dedicated descriptor group id Date: Thu, 28 Sep 2023 19:45:15 +0300 Message-ID: <20230928164550.980832-6-dtatulea@nvidia.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230928164550.980832-2-dtatulea@nvidia.com> References: <20230928164550.980832-2-dtatulea@nvidia.com> MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000044F8:EE_|DS0PR12MB7704:EE_ X-MS-Office365-Filtering-Correlation-Id: bf69fef0-7f33-441a-5fd3-08dbc042eafb X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 3covqXVqV2bCvSqif165e1cjvj97aFhG3v7BAdqt8P+zGUeEV+8AJOweXKt1SlBDqE24q8CK7DPQcM/IXjmTyDYr+XnR7kGERe4ZaymPbkxNMqdPH7ZEwxwvDJd8jhmoWOtyH2NC135lk64Ui46dH9cHTHjsEq2NK9Z2x2nRepZQ5CIBhs4QqN2wU7qjC01Go3JB3dulhY+aXFmgeSzv9cRLuvioYAh5N0oBlh2qfQlr9nnsLzlyfeLJQkZg9Hvzqp+E+3L2shR9Y/1eBJL9vrM68mqcHyZk1TYnvaGbZu/l3eCzibYAxE7viTHL3l172YwKT6vvfrVZxlaBKeQ+lsaLgooAJusmhT4kG2uD9H8kvhAfyJSl8LckkzVg1TDlKabjlqSmPKEjSSgy9qkZCwsTkwFIZWiKSC5yxC8B+2OY6TXjpLdSw6IhWXmyeauCQJ+bzO9ZP2CHf7bh4uMWqu0fkCm0P1dneyc9zFBNcp7Wn5e4QlgmQMUXby7obd47REFBICS/PZG+D+iiw94ekl5W1UPY1Jo+fN+5bIWu95e2TP0kipNuxEFblhtzLabG4meLGSsAOosghpNgeVhZfHQ8zfU4HKDoLl2qEoHS4CZdjP+3puJ8XAlcwLCacSMcB8kPtfQ1RWvy7uxxQqe7ger6q8QGMWf6+q2WZOwZMMFbQiwkyQrIw/UfZB3GkAP6HkP0pZBMdNBsw0sA2/P12UgRKg/tSMkmLset4Jo3B2R13wq69x0MTdhXjEJ8Sqti X-Forefront-Antispam-Report: CIP:216.228.118.233;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc7edge2.nvidia.com;CAT:NONE;SFS:(13230031)(4636009)(396003)(346002)(39860400002)(136003)(376002)(230922051799003)(82310400011)(64100799003)(186009)(451199024)(1800799009)(36840700001)(40470700004)(46966006)(36756003)(40460700003)(336012)(6666004)(70206006)(356005)(82740400003)(41300700001)(478600001)(7636003)(1076003)(2906002)(26005)(110136005)(2616005)(70586007)(54906003)(47076005)(316002)(426003)(36860700001)(40480700001)(4326008)(83380400001)(5660300002)(86362001)(8676002)(8936002);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2023 16:49:44.4836 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: bf69fef0-7f33-441a-5fd3-08dbc042eafb X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.118.233];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000044F8.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB7704 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Thu, 28 Sep 2023 09:50:22 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1778328640904156666 X-GMAIL-MSGID: 1778328640904156666 From: Si-Wei Liu With _F_DESC_ASID backend feature, the device can now support the VHOST_VDPA_GET_VRING_DESC_GROUP ioctl, and it may expose the descriptor table (including avail and used ring) in a different group than the buffers it contains. This new uAPI will fetch the group ID of the descriptor table. Signed-off-by: Si-Wei Liu Acked-by: Eugenio Pérez Acked-by: Jason Wang --- drivers/vhost/vdpa.c | 10 ++++++++++ include/uapi/linux/vhost.h | 8 ++++++++ 2 files changed, 18 insertions(+) diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c index 2f21798a37ee..851535f57b95 100644 --- a/drivers/vhost/vdpa.c +++ b/drivers/vhost/vdpa.c @@ -613,6 +613,16 @@ static long vhost_vdpa_vring_ioctl(struct vhost_vdpa *v, unsigned int cmd, else if (copy_to_user(argp, &s, sizeof(s))) return -EFAULT; return 0; + case VHOST_VDPA_GET_VRING_DESC_GROUP: + if (!vhost_vdpa_has_desc_group(v)) + return -EOPNOTSUPP; + s.index = idx; + s.num = ops->get_vq_desc_group(vdpa, idx); + if (s.num >= vdpa->ngroups) + return -EIO; + else if (copy_to_user(argp, &s, sizeof(s))) + return -EFAULT; + return 0; case VHOST_VDPA_SET_GROUP_ASID: if (copy_from_user(&s, argp, sizeof(s))) return -EFAULT; diff --git a/include/uapi/linux/vhost.h b/include/uapi/linux/vhost.h index f5c48b61ab62..649560c685f1 100644 --- a/include/uapi/linux/vhost.h +++ b/include/uapi/linux/vhost.h @@ -219,4 +219,12 @@ */ #define VHOST_VDPA_RESUME _IO(VHOST_VIRTIO, 0x7E) +/* Get the group for the descriptor table including driver & device areas + * of a virtqueue: read index, write group in num. + * The virtqueue index is stored in the index field of vhost_vring_state. + * The group ID of the descriptor table for this specific virtqueue + * is returned via num field of vhost_vring_state. + */ +#define VHOST_VDPA_GET_VRING_DESC_GROUP _IOWR(VHOST_VIRTIO, 0x7F, \ + struct vhost_vring_state) #endif From patchwork Thu Sep 28 16:45:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dragos Tatulea X-Patchwork-Id: 146346 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:cae8:0:b0:403:3b70:6f57 with SMTP id r8csp3663032vqu; Thu, 28 Sep 2023 16:22:31 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGvnNYNr1pXz8/DvH1ON3qw9SwGU0u2ceG6ytbZgKcSDmoVwxFjegnRVGNUNzixJWYNyjxr X-Received: by 2002:a05:6a00:1a8a:b0:693:3851:bd3e with SMTP id e10-20020a056a001a8a00b006933851bd3emr2736887pfv.3.1695943351066; Thu, 28 Sep 2023 16:22:31 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1695943351; cv=pass; d=google.com; s=arc-20160816; b=i4lExscM5s4oM6pUHOGtBGdDcwnBAC/xfaG4CCyvRDOVlgWAklLmwanA2Ipmd0wGi1 9j0YxUq4EuobeiN04s7FJ981ph1Z6oK3E466E8F9QPsMgVYRHPWTRwPu4kFMh1dtHC0a CiOCPoT/OqEbwsTLHUQt6NxrI8XKuq+HSxFAoR9rgPwVYYsuV5ZDohbaJEjjC09KazTu WxlXW1kfY3TCbnjzdwl8M/56l62RNmtkuCAu/Mo1QU2Refvub6X9JpS+KdNIvEFC6TGC vMtrrMarYZ/IjTt30EQp7JA99OKybqmAgpSPphq01GXJmiaHGhOrBbJXZct86YJaHhK3 4Fcw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=0k7vZDkq8HIAIRoAeeLZ+cqnJ3LV329QdGvZjtWRFo4=; fh=4SYmPDx0sMrYC0lSOUfhTOMEzVMfG5iPEyPqlL4Bs2M=; b=HZdJp6wp2kAUuDvTAdAMGTonVt0Nlc2dlXkLmbbYWWKSlKkRn0HFTQMOYvPXCluvgq jbv97aV8JaRYY+ZKlVXrvFBwinNc0nFIstYzXXZfHJzyQvMXQqgal7ni5saYMTotTjc6 bqrotGfvDZ2yKH8jAo8MiJvHslybcoPVaWFy1vyw4zVuyCZyErhOHuoNojG1ItGiZD2D PDC77l4X07NKN0pC1Me0f9t6zckV689QuDI7GlAZI+VZMQgeM1Uv83sBorrSuLs2a6jl LWF2C+BkR4yysDK6XJn+YCJIT0cHTxOUU9I/LOK6xP+rhOUFh6mun7tegLhuK/AI8FXl evLw== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=Nb0CTbIL; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: from lipwig.vger.email (lipwig.vger.email. [23.128.96.33]) by mx.google.com with ESMTPS id v21-20020a056a00149500b0068e285eaa31si21937834pfu.259.2023.09.28.16.22.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 28 Sep 2023 16:22:31 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) client-ip=23.128.96.33; Authentication-Results: mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=Nb0CTbIL; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id E16158070D45; Thu, 28 Sep 2023 09:50:30 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231810AbjI1QuG (ORCPT + 21 others); Thu, 28 Sep 2023 12:50:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34330 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231802AbjI1Qty (ORCPT ); Thu, 28 Sep 2023 12:49:54 -0400 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2042.outbound.protection.outlook.com [40.107.93.42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BC42E1A3 for ; Thu, 28 Sep 2023 09:49:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ROC8YYVy3V8XPbi6MMBmzvJmexM3i4kMKUjUuVkkSI8zLKuCHrqR7Uzq4FHG9PanFvRFwR77ftSJ7jdeJaGp3ki8wykv1mjmSl/VgVXcHi9sPVKFyV1kpSi9/coZcXZzMQXO0/zV0V333kBwgDmuinonyisrXMOdwhWJO6uVHufcrPasevhAA5hm/b5rnB4h4q7ZJWjzshKeBrW/OSearrWeaNfGfdyBvvEg2hcD42TY84xIcOhTlN1guBy/MkezyORxh0+5TYi28I6UtaYBdzY9poliZL7Ef/yMGDa33sm/KcTN6p4eBD3YDue9kc1i8JNi7OjKY64wuI3ann5fTA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=0k7vZDkq8HIAIRoAeeLZ+cqnJ3LV329QdGvZjtWRFo4=; b=iS5Q1DIfsN0bjsm1v35ekfry3RVWfRMhn8BLaZp2I3rwFsMWXZXryxWflAkmuJEUhiQJzXBDc3g+nwB/2GH7nHszbsFgrVTSr+CSvyKhYKopuN9tED6hJJwc5fT1MAyL/3u7PSQkW63+XcfBUkSNOHIOKpcRWKcaUhq6M5r+hEWpEQzraf/SVAF0qvptpowbXrbfEdBfW/S11NV59uZWFjwh3FZE6RUbLD3cWbJf5H8Vp8r4Z5cMvKw2WBb0uFVsG6w9M2N212p4wlR6D0oxKPFZ1xI7SVh2tUuLI7pr1+kgMzConzJ6YW7TioQLVfwfxz4Xhs8XBcVpm9d0coyDEw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.232) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=0k7vZDkq8HIAIRoAeeLZ+cqnJ3LV329QdGvZjtWRFo4=; b=Nb0CTbILKnOWIZ9V2U19RVsBEKNqwlGeP787rjibv+YVcXPzdv7QGaj47oXCOuJMDEQapyGAxdF35xfG6Zh0D2mRb4o0sD91DBCa2rr59rhNSfo3c1bCzStHGwiWIpffBLYpPCMJgaiP84fOCI0TieVZkBJMHR/VA8E5XvnCRDrWgFl++jy8dUsTYvIOwrjca9c40X8UfiMepRETSIiGHj698pxjDQwY2RC0PqzvZx37PUny1MZwC3n1g4FSvc4sfK0ipE3lk0ixHFJX3S8Bm/VxOYxSO5L7gK2l+6XVU92BRN0XcJ203acji2DcZoxofCcE+56jJVg7ydx2BMORqA== Received: from CY5PR15CA0100.namprd15.prod.outlook.com (2603:10b6:930:7::20) by IA1PR12MB6332.namprd12.prod.outlook.com (2603:10b6:208:3e2::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.21; Thu, 28 Sep 2023 16:49:48 +0000 Received: from CY4PEPF0000E9D1.namprd03.prod.outlook.com (2603:10b6:930:7:cafe::10) by CY5PR15CA0100.outlook.office365.com (2603:10b6:930:7::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.25 via Frontend Transport; Thu, 28 Sep 2023 16:49:47 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.232) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.232 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.232; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.232) by CY4PEPF0000E9D1.mail.protection.outlook.com (10.167.241.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.19 via Frontend Transport; Thu, 28 Sep 2023 16:49:47 +0000 Received: from drhqmail202.nvidia.com (10.126.190.181) by mail.nvidia.com (10.127.129.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Thu, 28 Sep 2023 09:49:40 -0700 Received: from drhqmail202.nvidia.com (10.126.190.181) by drhqmail202.nvidia.com (10.126.190.181) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Thu, 28 Sep 2023 09:49:40 -0700 Received: from c-237-113-220-225.mtl.labs.mlnx (10.127.8.12) by mail.nvidia.com (10.126.190.181) with Microsoft SMTP Server id 15.2.986.41 via Frontend Transport; Thu, 28 Sep 2023 09:49:38 -0700 From: Dragos Tatulea To: , , "Michael S . Tsirkin" , Jason Wang , Xuan Zhuo CC: , Dragos Tatulea , Subject: [PATCH vhost 05/16] vdpa/mlx5: Create helper function for dma mappings Date: Thu, 28 Sep 2023 19:45:16 +0300 Message-ID: <20230928164550.980832-7-dtatulea@nvidia.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230928164550.980832-2-dtatulea@nvidia.com> References: <20230928164550.980832-2-dtatulea@nvidia.com> MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000E9D1:EE_|IA1PR12MB6332:EE_ X-MS-Office365-Filtering-Correlation-Id: 57d590aa-c920-40b0-f163-08dbc042ecfc X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: x/EyXN0uODAIzxJckL/PfSBec4e/irrVH4zNspuRSowCxdefpCeT+kowxMJ3qZrxdro/HaJz73dodKJXa8fYhsavtIbUSuIuwTWwIMTwMsYBeZOJg96NKReic407pqIrkN4kTmRVmh3mlKKGz6KfbsHUbIG9fCUbIZHa3EaYttG+/iS9h8IDYPKHMXdyfs5uVr7A2KPd5zh/P2SAvQjAhqoJnSb+G9n3f1JUZUCQG0N5+3MBVKWUIjACu2L1/EAU+IMBKOEZXoYZFHpXR+q7qwj+nT5e1OS0MnMAiQuMclb6LbnIO9Fo6m/oiRd4GHAMnp9oVJM7bQvY2yPCF093OPeUunJ2APbUcH81T5ZNTjWu+bV5gs/ISpyzSjSdreB4MJ18DDmkgWmiCmH+YTyv2BU0vE7V25wVrCXSj8SL18PN6xFiTk+4FntTiTUyZkj213BDTIds1fneGx4tJ5/DWE05IS99tDFHwr43YEbW1o6/B3oEwdULd4tBXlaDsVZ6ZKTS4t/Zaohklp3bffFpils1suw5+dUm3erC12k1LS/9BfvlmV7LcAlyb7CPD/LEDXLPw9qDE8At2smehoY3FEkP+8/JqbZkLMaZzBlvMG8gVb9lwLksVjbLZGKgVYnY+qiD9kFTf/gzXLB6N7n4rTqjOpMZr9CXhfwxCUKxqS/qWE4xOG/4TE1n+TgHMav8F9FrBmFcJPEehHse9Z2pQTmqJsBhVuM7Y11BHqQCt2Y= X-Forefront-Antispam-Report: CIP:216.228.118.232;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc7edge1.nvidia.com;CAT:NONE;SFS:(13230031)(4636009)(396003)(346002)(136003)(39860400002)(376002)(230922051799003)(64100799003)(186009)(82310400011)(451199024)(1800799009)(40470700004)(46966006)(36840700001)(4326008)(1076003)(70586007)(110136005)(70206006)(36860700001)(26005)(47076005)(8676002)(5660300002)(8936002)(2906002)(41300700001)(40480700001)(426003)(336012)(478600001)(6666004)(54906003)(40460700003)(82740400003)(36756003)(7636003)(356005)(2616005)(83380400001)(316002)(86362001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2023 16:49:47.8337 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 57d590aa-c920-40b0-f163-08dbc042ecfc X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.118.232];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000E9D1.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB6332 X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Thu, 28 Sep 2023 09:50:31 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1778325495001504210 X-GMAIL-MSGID: 1778325495001504210 Necessary for upcoming cvq separation from mr allocation. Acked-by: Jason Wang Signed-off-by: Dragos Tatulea --- drivers/vdpa/mlx5/core/mlx5_vdpa.h | 1 + drivers/vdpa/mlx5/core/mr.c | 5 +++++ drivers/vdpa/mlx5/net/mlx5_vnet.c | 4 ++-- 3 files changed, 8 insertions(+), 2 deletions(-) diff --git a/drivers/vdpa/mlx5/core/mlx5_vdpa.h b/drivers/vdpa/mlx5/core/mlx5_vdpa.h index ca56242972b3..3748f027cfe9 100644 --- a/drivers/vdpa/mlx5/core/mlx5_vdpa.h +++ b/drivers/vdpa/mlx5/core/mlx5_vdpa.h @@ -120,6 +120,7 @@ int mlx5_vdpa_create_mr(struct mlx5_vdpa_dev *mvdev, struct vhost_iotlb *iotlb, unsigned int asid); void mlx5_vdpa_destroy_mr(struct mlx5_vdpa_dev *mvdev); void mlx5_vdpa_destroy_mr_asid(struct mlx5_vdpa_dev *mvdev, unsigned int asid); +int mlx5_vdpa_create_dma_mr(struct mlx5_vdpa_dev *mvdev); #define mlx5_vdpa_warn(__dev, format, ...) \ dev_warn((__dev)->mdev->device, "%s:%d:(pid %d) warning: " format, __func__, __LINE__, \ diff --git a/drivers/vdpa/mlx5/core/mr.c b/drivers/vdpa/mlx5/core/mr.c index 5a1971fcd87b..7bd0883b8b25 100644 --- a/drivers/vdpa/mlx5/core/mr.c +++ b/drivers/vdpa/mlx5/core/mr.c @@ -619,3 +619,8 @@ int mlx5_vdpa_handle_set_map(struct mlx5_vdpa_dev *mvdev, struct vhost_iotlb *io return err; } + +int mlx5_vdpa_create_dma_mr(struct mlx5_vdpa_dev *mvdev) +{ + return mlx5_vdpa_create_mr(mvdev, NULL, 0); +} diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5_vnet.c index 40a03b08d7cf..65b6a54ad344 100644 --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c @@ -2836,7 +2836,7 @@ static int mlx5_vdpa_reset(struct vdpa_device *vdev) ++mvdev->generation; if (MLX5_CAP_GEN(mvdev->mdev, umem_uid_0)) { - if (mlx5_vdpa_create_mr(mvdev, NULL, 0)) + if (mlx5_vdpa_create_dma_mr(mvdev)) mlx5_vdpa_warn(mvdev, "create MR failed\n"); } up_write(&ndev->reslock); @@ -3441,7 +3441,7 @@ static int mlx5_vdpa_dev_add(struct vdpa_mgmt_dev *v_mdev, const char *name, goto err_mpfs; if (MLX5_CAP_GEN(mvdev->mdev, umem_uid_0)) { - err = mlx5_vdpa_create_mr(mvdev, NULL, 0); + err = mlx5_vdpa_create_dma_mr(mvdev); if (err) goto err_res; } From patchwork Thu Sep 28 16:45:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Dragos Tatulea X-Patchwork-Id: 146363 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:cae8:0:b0:403:3b70:6f57 with SMTP id r8csp3680410vqu; Thu, 28 Sep 2023 17:04:23 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGKm+JJBgeIU3x+wWYzVD+NUl/SgcQM+BEdJXASW94/OEX9xeWmjDUm+YgFFJHRMuTvSdUM X-Received: by 2002:a05:6358:94a9:b0:12b:e45b:3fac with SMTP id i41-20020a05635894a900b0012be45b3facmr3067894rwb.32.1695945863466; Thu, 28 Sep 2023 17:04:23 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1695945863; cv=pass; d=google.com; s=arc-20160816; b=KJ14IZ+a7g2VZInxt5xLNMphPrJlph7gvmgA9P2qo0buTfe1ghI1GEyG7CyMCOIQtS tHDjvnmsBnJ4UXTYevnwYcGCf2H27c/U6sDEJ0dkQhgJFXhSpJUIB1bMWyVMK8HhrBNr nmrdqI7CbXYiZwPfcNPlNMZrFDqJ9S6wTCTcmkNI1+agj0D72HFx4onLAsigq9O966Zj PhL1cutimKuYV0Vrbs3gio9JUi5byA2DaC9U/RUWL0J3t4y9xRO4/KAL6DemqEL/O+Ie EP2atVhx7mRaURA9zsEPV/CDlueyeZSzoi/td8OxJip4TENojO1BM7jNtfo4A/FzDUTV o9EA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=sN7TV4u8UPdAnQNWvbOBZvrtM3lU/XqKcHUxqYYSho4=; fh=4SYmPDx0sMrYC0lSOUfhTOMEzVMfG5iPEyPqlL4Bs2M=; b=bw9678c2lOf2rKSkrVrq0Occ6m01GsZ/kGXNrhn83YOfOCJGYIw9O67VIcU07Zctj5 pbyuHutt9r9hvjQf3XsLS77qS6eGku3x1OY6ruUEKVeJLluTIrZRP1YC7don2wmSaLbe psYlTh8fIJZ+4+5U3fxftdTT/XhW75Uc0G2ojCptHJlGNqo3zeGG6gO6iWFvA//i8PoG /EhJCuv2rmamkVDqOVy9/g8Doqi/Pmq8mRLhX3An07cbtsBHUb7DoJFUOTO/bZ2Ek6ue YEumJ5BSYDGchqkFa/kuAsG9CYdaxh8tS2SYLi4re1QaSLypMqjnHcHE4Mt8O3trKdkd gPFw== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=nMSevCR5; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id 82-20020a630155000000b00563f4e46f7esi19557909pgb.287.2023.09.28.17.04.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 28 Sep 2023 17:04:23 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=nMSevCR5; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id E099181CE130; Thu, 28 Sep 2023 09:50:28 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231873AbjI1QuQ (ORCPT + 21 others); Thu, 28 Sep 2023 12:50:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58122 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231987AbjI1QuD (ORCPT ); Thu, 28 Sep 2023 12:50:03 -0400 Received: from NAM04-MW2-obe.outbound.protection.outlook.com (mail-mw2nam04on2069.outbound.protection.outlook.com [40.107.101.69]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 91E7ECFF for ; Thu, 28 Sep 2023 09:49:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=TAGprFBk7gIsbgeslXqe4TpTVWGslgFkoV3zL/bNtbIXUGZDpPbf99VgNZ7/L2g0Z4dt33dwkmqGOFHsLYI3W+8QK168VWT28CpJs6jphxGIYiXwCKKZx2GlwxH048WzEa+m1xEumBiFe4DfRR3na+QBJkETlugKhmfTv13ePXO1Ps8Orgv/JbEZJKNt94KB4GJAEveGThKsvM+Nq/Jct8qMn/yyuFBNDIAro6AJxlzl51IN9ohOnAlBgqt267W2CNGmeAhVqZWYdcbXbnNrJ5qZl1hD/pH8m3u8IOX2I3n7CTNdauSZGJZf5FcP2eISK+dxX1unxc/llzVSO4IlaA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=sN7TV4u8UPdAnQNWvbOBZvrtM3lU/XqKcHUxqYYSho4=; b=O8RWE98dqA9mgwk4yz3x6glW9iVeEXyrGnMNzhaidw7kM15NTuMNfG22mXRMzxNQnDPwn5tWcT/zpQCHjw0yMxRCkmKXWHGulB9SmbOaWSaHedCyueCMyjRCVZrsGPhOm0pp7r6z87Nv4A7M9ASKkq6FYT/l6MbijNKrNlbxe++INkLSEnSfQjSZLiGsA9RCiHcT7nXv2lKEK+W5e7Pftvd/QkWVbnBRZgEtTC9Tw0QRFjWqgPygn6Y107WQPFQickR2BKnAe28TinLtKRFkphOIFm+aD5SrMvpW4AunOom9WpQqoGnvZ1IydlF0p5NuNLXZhmvxYOgn++rHz3yjkA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.233) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=sN7TV4u8UPdAnQNWvbOBZvrtM3lU/XqKcHUxqYYSho4=; b=nMSevCR5aDBbEgr/MlCByHxD2sVgptoDNK8Tf8I28Ozu0N904Zoe+ZzIwN1juEo48PWiVSO1lxICrxAQ6m8CziWJogPFr0vq6yvbuPaVPLczIhE2bvo6P68/rgjXsY7GRq3OFVQQKXDniXhKQMkyfJlAvvPe/jcxWKJJzPgsz/Yhc+Jb8y1l8M4ckzJzoeATsOE2MJX+kLuG/Pd/l38D8gJQfXTV5ZU2fu3P84rUgVUoNYbFFjTTuzizzajOp76XIZNbQ4nvurj6K+z8woEe3NHl5+Uyu/bXA/qFXgjh6bqgUhXvdBTPMDxQWRK8d9PG/tkckT9ybEA9RynQ8YoqhQ== Received: from MW3PR06CA0001.namprd06.prod.outlook.com (2603:10b6:303:2a::6) by BL1PR12MB5031.namprd12.prod.outlook.com (2603:10b6:208:31a::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.21; Thu, 28 Sep 2023 16:49:55 +0000 Received: from CO1PEPF000044F8.namprd21.prod.outlook.com (2603:10b6:303:2a:cafe::47) by MW3PR06CA0001.outlook.office365.com (2603:10b6:303:2a::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.24 via Frontend Transport; Thu, 28 Sep 2023 16:49:54 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.233) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.233 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.233; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.233) by CO1PEPF000044F8.mail.protection.outlook.com (10.167.241.198) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6863.9 via Frontend Transport; Thu, 28 Sep 2023 16:49:54 +0000 Received: from drhqmail203.nvidia.com (10.126.190.182) by mail.nvidia.com (10.127.129.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Thu, 28 Sep 2023 09:49:43 -0700 Received: from drhqmail202.nvidia.com (10.126.190.181) by drhqmail203.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Thu, 28 Sep 2023 09:49:43 -0700 Received: from c-237-113-220-225.mtl.labs.mlnx (10.127.8.12) by mail.nvidia.com (10.126.190.181) with Microsoft SMTP Server id 15.2.986.41 via Frontend Transport; Thu, 28 Sep 2023 09:49:41 -0700 From: Dragos Tatulea To: , , "Michael S . Tsirkin" , Jason Wang , Xuan Zhuo CC: , Dragos Tatulea , Subject: [PATCH vhost 06/16] vdpa/mlx5: Decouple cvq iotlb handling from hw mapping code Date: Thu, 28 Sep 2023 19:45:17 +0300 Message-ID: <20230928164550.980832-8-dtatulea@nvidia.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230928164550.980832-2-dtatulea@nvidia.com> References: <20230928164550.980832-2-dtatulea@nvidia.com> MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000044F8:EE_|BL1PR12MB5031:EE_ X-MS-Office365-Filtering-Correlation-Id: 98f56862-56c1-4a22-2a92-08dbc042f11c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: kt58hLNfr1uvXayMAhZ/vbKcOrLp+xjUPb64DJ7aDEWBDLnHsepY+UZ2PGaHUwg5HRQQ6vDxRyk7btCN3Wjn7lIyBKqsd8eR7XsqCm+CMkZu/3i6n93fd8EuN5/0KtM19ZCC6lLTHnIrPDuyLK7HXTdw9A3J+VqR2LtNsTJpWNcjNojUZNYy16rDXHscVL5Sffzx/bfVj2FDcf2aiA/m4rCVn63+4B55kUHf34lAbcdPsG0nnu4eylnFei33pgMPHHqqez1drp/3aFvGRvKRkc6KGkZE+ZLYtSLnpc+plfE3U3Ck4TpxgttDiygHUPhF7sy0kUmGEvRXw/wa4TjoMokLD12jFEJ7b9ARBQ8l56fB9WtwPQXSko77dHgimlpVuePKfj9zy6nm8GO/iyio7blQPOCKi+M3lp8dYB55grx8ERfKYCzfFksBaoXKrybGRQcnqY2hWD/YT+BEIg2w6tGco67AK5Ou3dAywmHqvIUgEAXTpzMqPvDBYqkfdmJrFaMATFqwSTT9EaVksmY2ktEnyxN+URrUvSL8PBTVQUxbJsQ2A8hfKbs4COadOnm6CYCN6UIZxcj/f6VduryXGCN4SzdakpT82rVjm/665K7hUHFt7VxiVx597Z+EPcAW67dI2hx7StjKlSSYlcZcei8syORp6N1+aqSkIrBijmQxASlaEcEuBsRZjJr2UjsjaH9Bmuapu7W63WBuw5XDyB9umcIHaP3gd1TkS7t8eAA= X-Forefront-Antispam-Report: CIP:216.228.118.233;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc7edge2.nvidia.com;CAT:NONE;SFS:(13230031)(4636009)(376002)(39860400002)(396003)(346002)(136003)(230922051799003)(1800799009)(82310400011)(186009)(64100799003)(451199024)(36840700001)(40470700004)(46966006)(2616005)(6666004)(478600001)(336012)(426003)(26005)(316002)(1076003)(83380400001)(41300700001)(110136005)(5660300002)(70206006)(70586007)(4326008)(8676002)(8936002)(54906003)(36756003)(2906002)(36860700001)(47076005)(86362001)(7636003)(356005)(82740400003)(40480700001)(40460700003);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2023 16:49:54.7024 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 98f56862-56c1-4a22-2a92-08dbc042f11c X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.118.233];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000044F8.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5031 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Thu, 28 Sep 2023 09:50:28 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1778328129769513805 X-GMAIL-MSGID: 1778328129769513805 The handling of the cvq iotlb is currently coupled with the creation and destruction of the hardware mkeys (mr). This patch moves cvq iotlb handling into its own function and shifts it to a scope that is not related to mr handling. As cvq handling is just a prune_iotlb + dup_iotlb cycle, put it all in the same "update" function. Finally, the destruction path is handled by directly pruning the iotlb. After this move is done the ASID mr code can be collapsed into a single function. Acked-by: Jason Wang Signed-off-by: Dragos Tatulea Acked-by: Eugenio Pérez --- drivers/vdpa/mlx5/core/mlx5_vdpa.h | 3 ++ drivers/vdpa/mlx5/core/mr.c | 57 +++++++++++------------------- drivers/vdpa/mlx5/net/mlx5_vnet.c | 7 ++-- 3 files changed, 28 insertions(+), 39 deletions(-) diff --git a/drivers/vdpa/mlx5/core/mlx5_vdpa.h b/drivers/vdpa/mlx5/core/mlx5_vdpa.h index 3748f027cfe9..554899a80241 100644 --- a/drivers/vdpa/mlx5/core/mlx5_vdpa.h +++ b/drivers/vdpa/mlx5/core/mlx5_vdpa.h @@ -120,6 +120,9 @@ int mlx5_vdpa_create_mr(struct mlx5_vdpa_dev *mvdev, struct vhost_iotlb *iotlb, unsigned int asid); void mlx5_vdpa_destroy_mr(struct mlx5_vdpa_dev *mvdev); void mlx5_vdpa_destroy_mr_asid(struct mlx5_vdpa_dev *mvdev, unsigned int asid); +int mlx5_vdpa_update_cvq_iotlb(struct mlx5_vdpa_dev *mvdev, + struct vhost_iotlb *iotlb, + unsigned int asid); int mlx5_vdpa_create_dma_mr(struct mlx5_vdpa_dev *mvdev); #define mlx5_vdpa_warn(__dev, format, ...) \ diff --git a/drivers/vdpa/mlx5/core/mr.c b/drivers/vdpa/mlx5/core/mr.c index 7bd0883b8b25..fcb6ae32e9ed 100644 --- a/drivers/vdpa/mlx5/core/mr.c +++ b/drivers/vdpa/mlx5/core/mr.c @@ -489,14 +489,6 @@ static void destroy_user_mr(struct mlx5_vdpa_dev *mvdev, struct mlx5_vdpa_mr *mr } } -static void _mlx5_vdpa_destroy_cvq_mr(struct mlx5_vdpa_dev *mvdev, unsigned int asid) -{ - if (mvdev->group2asid[MLX5_VDPA_CVQ_GROUP] != asid) - return; - - prune_iotlb(mvdev); -} - static void _mlx5_vdpa_destroy_dvq_mr(struct mlx5_vdpa_dev *mvdev, unsigned int asid) { struct mlx5_vdpa_mr *mr = &mvdev->mr; @@ -522,25 +514,14 @@ void mlx5_vdpa_destroy_mr_asid(struct mlx5_vdpa_dev *mvdev, unsigned int asid) mutex_lock(&mr->mkey_mtx); _mlx5_vdpa_destroy_dvq_mr(mvdev, asid); - _mlx5_vdpa_destroy_cvq_mr(mvdev, asid); mutex_unlock(&mr->mkey_mtx); } void mlx5_vdpa_destroy_mr(struct mlx5_vdpa_dev *mvdev) { - mlx5_vdpa_destroy_mr_asid(mvdev, mvdev->group2asid[MLX5_VDPA_CVQ_GROUP]); mlx5_vdpa_destroy_mr_asid(mvdev, mvdev->group2asid[MLX5_VDPA_DATAVQ_GROUP]); -} - -static int _mlx5_vdpa_create_cvq_mr(struct mlx5_vdpa_dev *mvdev, - struct vhost_iotlb *iotlb, - unsigned int asid) -{ - if (mvdev->group2asid[MLX5_VDPA_CVQ_GROUP] != asid) - return 0; - - return dup_iotlb(mvdev, iotlb); + prune_iotlb(mvdev); } static int _mlx5_vdpa_create_dvq_mr(struct mlx5_vdpa_dev *mvdev, @@ -572,22 +553,7 @@ static int _mlx5_vdpa_create_dvq_mr(struct mlx5_vdpa_dev *mvdev, static int _mlx5_vdpa_create_mr(struct mlx5_vdpa_dev *mvdev, struct vhost_iotlb *iotlb, unsigned int asid) { - int err; - - err = _mlx5_vdpa_create_dvq_mr(mvdev, iotlb, asid); - if (err) - return err; - - err = _mlx5_vdpa_create_cvq_mr(mvdev, iotlb, asid); - if (err) - goto out_err; - - return 0; - -out_err: - _mlx5_vdpa_destroy_dvq_mr(mvdev, asid); - - return err; + return _mlx5_vdpa_create_dvq_mr(mvdev, iotlb, asid); } int mlx5_vdpa_create_mr(struct mlx5_vdpa_dev *mvdev, struct vhost_iotlb *iotlb, @@ -620,7 +586,24 @@ int mlx5_vdpa_handle_set_map(struct mlx5_vdpa_dev *mvdev, struct vhost_iotlb *io return err; } +int mlx5_vdpa_update_cvq_iotlb(struct mlx5_vdpa_dev *mvdev, + struct vhost_iotlb *iotlb, + unsigned int asid) +{ + if (mvdev->group2asid[MLX5_VDPA_CVQ_GROUP] != asid) + return 0; + + prune_iotlb(mvdev); + return dup_iotlb(mvdev, iotlb); +} + int mlx5_vdpa_create_dma_mr(struct mlx5_vdpa_dev *mvdev) { - return mlx5_vdpa_create_mr(mvdev, NULL, 0); + int err; + + err = mlx5_vdpa_create_mr(mvdev, NULL, 0); + if (err) + return err; + + return mlx5_vdpa_update_cvq_iotlb(mvdev, NULL, 0); } diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5_vnet.c index 65b6a54ad344..aa4896662699 100644 --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c @@ -2884,10 +2884,13 @@ static int set_map_data(struct mlx5_vdpa_dev *mvdev, struct vhost_iotlb *iotlb, return err; } - if (change_map) + if (change_map) { err = mlx5_vdpa_change_map(mvdev, iotlb, asid); + if (err) + return err; + } - return err; + return mlx5_vdpa_update_cvq_iotlb(mvdev, iotlb, asid); } static int mlx5_vdpa_set_map(struct vdpa_device *vdev, unsigned int asid, From patchwork Thu Sep 28 16:45:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Dragos Tatulea X-Patchwork-Id: 146381 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:cae8:0:b0:403:3b70:6f57 with SMTP id r8csp3696775vqu; Thu, 28 Sep 2023 17:45:20 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEe5jZTFNJCsxbVNQGxZhcIuiUraWob5hJsJRTbXLIOiqtd4WzmQWnbdY1cVlEX6WN0di1h X-Received: by 2002:a17:902:f544:b0:1c5:efd1:82b4 with SMTP id h4-20020a170902f54400b001c5efd182b4mr4556200plf.29.1695948320167; Thu, 28 Sep 2023 17:45:20 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1695948320; cv=pass; d=google.com; s=arc-20160816; b=m2zIXB/Hm0QH9O07sVOmw9WbXS53K57SqtsTnOqP2De4JqtxhE8DGTHtijKYGej9Dq ESyYhN4ervGUoco6Yj5hrNq+BSB4ol/5r9et0bxQrmotC0UhDZmZo4I/5Vk/AHV4i8Kh aVUsSAGjLwsl1qSbqGWs8JpRIZMD3Dt+Nw7RQN9FtDbZzyOitvByuU4qvfppWt/2HEIh TQdIfx2EpaUf/s+Q2u2pmL3KZAp+dQxR/GM5m40SrLESXsIo/2FOSkBnMnnduG/t9V7i X+VgAHgRgPtnKhpyjuY0b+LJ8C1uJXlWRcKdfRXM07K++j7mn11bcTEIdFloKX0mlLdR A/mA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=qnJQ5tPt3zRoEBpIfXlFPOb7QvzypqNGxupaS7vWJDc=; fh=4SYmPDx0sMrYC0lSOUfhTOMEzVMfG5iPEyPqlL4Bs2M=; b=hLorCw+U5g5zF1gYjXDFB+t5zXDFMV33/eAGek51nMoG341bjYRrTTiL+Z7GuHSXUv QaP/DeykSwWQUsUhw+BzozSChaL8R0LGcqNb1zm2Xs38sbZPz0PgRnju1HLPDsESxYpz CvIqOBSAFHs/uC/JyV72sj/qgvfMz2o+/c7a/b0LKnvotj6BojbWQNl0Yeep8lMwsz8U 6SHgflO8AFYZ7NXkBS6xwSQrjRQ+CwiwVc/1SrF8S30/RaJ99JieP3EihNM1UE9pkF16 /v6LSiYSz1FBe6/5JH8/G7Nz6sK7Mj7bcAU30v5TS3NZa77ADOdg0eeWrHzfdZwbVyZa gNig== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b="Lgtsk/BS"; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: from howler.vger.email (howler.vger.email. [2620:137:e000::3:4]) by mx.google.com with ESMTPS id m4-20020a170902db0400b001c7358e8cf3si3740029plx.542.2023.09.28.17.45.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 28 Sep 2023 17:45:20 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) client-ip=2620:137:e000::3:4; Authentication-Results: mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b="Lgtsk/BS"; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id E69F785BE6C2; Thu, 28 Sep 2023 09:50:38 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231958AbjI1Qud (ORCPT + 21 others); Thu, 28 Sep 2023 12:50:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58070 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231866AbjI1QuM (ORCPT ); Thu, 28 Sep 2023 12:50:12 -0400 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2063.outbound.protection.outlook.com [40.107.223.63]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E49F510F2 for ; Thu, 28 Sep 2023 09:50:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=NwQXQlBTiFxwVkAgE+XtiEpQRYwspaBbRPvoumGu6u2UENjPBnqhF8tH4EcNu4XnbusOxOR6tITy7gDUB50LyPmR+hekA/zc5wie0NPsY1qKPYyrD8fxzm04L55fdOGjsW8urkEUyauTObBEGKR53EIxg8fGGRm2VzTg046vYju7MTX9ouFi7KNVliu/KcdGo/5IUFzYERxd+n6Q9e4tUA5uAzDngNx7vmKQzJwJrebpjiBmhaNYoMy2cZ4lYmY5MpKfi6zuvoiw/bx5uIR4l4XTUGIHTSxgGy/JhBNotx72hGPFrGJO1OwtXD+eFkZ6iiQIvoEMXxfTVr7aYeYWBw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=qnJQ5tPt3zRoEBpIfXlFPOb7QvzypqNGxupaS7vWJDc=; b=kSmKSsBorHuX8gV+MRk5Nzk6n2Xak+xuiffHEnvh38JRStl7hDg5WRbvoHu0YVG5yaVxdO4CoYg5ztUvoYvIbL/2PA7MEKzwTk94tT5qV3KSraBHzmpJ95Wn8hiEc6GRafBlJ+b8sAYY6FdQuvPfmkrJX4d7LEsFARfinb90FLOMiM1rVZWVcJSmOS+gIZR4B+n2crjRHuTBeN2+hsSi3puS/ekfCzldaobaCWCWoMq5BulNHUyQYxKMWHShaTygKlwLWaSu/scuzUngdMs39i0EgSU8ROShiDzCq9XLQrS1RVOjryz5nxgtMWGS8Ha+n3I4btiEgkcIKjFJslvLPw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.233) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qnJQ5tPt3zRoEBpIfXlFPOb7QvzypqNGxupaS7vWJDc=; b=Lgtsk/BSTQpGh1geAwyXe/ipNtYrlgXhWQDXY1hDDJFB7qQ/LGHrUvMUAnGh4cb1xteTzx1aErl59mkTxHLVmwdG+vTKd28RHRF/Anfqs/TkcHkT9D/girZe5gGPOOMVFt48NWKiYAeOJiY322tY0XSg5QOlR4G9k+hUipKsZYdHqb73Z2uJk1FgZ0eaSBISeIB7CInj9hwddrUhA++UNWx7nUBcQ9d5BAQwHporcw3ueScWD/UymA52dyDxuTKeiryrVCp9nlq45hs24Toy+POJ+cRl14jjtdkY5XloqBaQcSiCKiXXjcQ+W5I4xBdfyk1LHv+toPYV0PVK6rxmlA== Received: from MW4P223CA0018.NAMP223.PROD.OUTLOOK.COM (2603:10b6:303:80::23) by CH3PR12MB8211.namprd12.prod.outlook.com (2603:10b6:610:125::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.21; Thu, 28 Sep 2023 16:49:59 +0000 Received: from CO1PEPF000044FC.namprd21.prod.outlook.com (2603:10b6:303:80:cafe::d3) by MW4P223CA0018.outlook.office365.com (2603:10b6:303:80::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.25 via Frontend Transport; Thu, 28 Sep 2023 16:49:59 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.233) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.233 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.233; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.233) by CO1PEPF000044FC.mail.protection.outlook.com (10.167.241.202) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6863.9 via Frontend Transport; Thu, 28 Sep 2023 16:49:59 +0000 Received: from drhqmail203.nvidia.com (10.126.190.182) by mail.nvidia.com (10.127.129.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Thu, 28 Sep 2023 09:49:46 -0700 Received: from drhqmail202.nvidia.com (10.126.190.181) by drhqmail203.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Thu, 28 Sep 2023 09:49:46 -0700 Received: from c-237-113-220-225.mtl.labs.mlnx (10.127.8.12) by mail.nvidia.com (10.126.190.181) with Microsoft SMTP Server id 15.2.986.41 via Frontend Transport; Thu, 28 Sep 2023 09:49:44 -0700 From: Dragos Tatulea To: , , "Michael S . Tsirkin" , Jason Wang , Xuan Zhuo CC: , Dragos Tatulea , Subject: [PATCH vhost 07/16] vdpa/mlx5: Take cvq iotlb lock during refresh Date: Thu, 28 Sep 2023 19:45:18 +0300 Message-ID: <20230928164550.980832-9-dtatulea@nvidia.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230928164550.980832-2-dtatulea@nvidia.com> References: <20230928164550.980832-2-dtatulea@nvidia.com> MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000044FC:EE_|CH3PR12MB8211:EE_ X-MS-Office365-Filtering-Correlation-Id: a9ea5f04-db78-4564-ceaf-08dbc042f3f2 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: NaZ8iRDNrNMysAHH2DSBaK2oVPK/yeQPunLj9MybVEOqzfxuqqhNzzO7QacTJEcOV3/OFzwERfrZSu8cAPTYq9pZuo1lyr3fh7GJ0Ol/PuyprrPDP8EnqW683a6xAiUn6cNIojUVfBeWpNNgMK7JQsTY+TvmrNZDwqo7gE4otx1bluj8Ms4orGgqN3Ps3JljMk6YXdHiuuZySeRD11HXpiWd/wOBFx6KEOeCwbpc0oFohESihouMDkzSPpotFLs6abLS2/V0OAiX4tvIgMZJyHDi2Xrijk5QCZoMOBhRrWoNvewQr0HcpZAGYIC9WSr0Ym/iybpSOIcbghWHtdr4OMQmHo76lGqqYv42OXvYJJHikVQU0zbdjxGgQOR6IyRQDre1AJnduSkEV28rFy/S/mFmt/9npE6cNyhzptAdFKuxMST9WkuDJsmeAb6Rs+naF2tehTITwyHyQzoHfA+mxNN+afdP318rKSk60BJ8PPAojUMRnXiLj1NjXjG0in13niWG2/v2u4Byf0zxURXkWK7Bg4cmwqBnG9u2u4EvPc9gc1uaUWYTpOUK7zdykGN+V2GfZv5wi615VbOQa6xY57eL5MBIK8gPWvRZCWB7qcpcxjE6qEbpfOdBytonK9+FpjnKH8EI9ne6e0hTdavItyrf/0rIHBbNpR6hUjcL4mFByiph2rYVDjCPtSFpd6+Onoll6DUUABj1Lgn+/6nEMSLpGf5aDa3qwi0mgUK/Hsaxtxq/dTEwSYMZjhiEMvzdLC2jgjuFrfE4VqkPYaQU538FubY+h7qZBbjQGyJDhnc= X-Forefront-Antispam-Report: CIP:216.228.118.233;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc7edge2.nvidia.com;CAT:NONE;SFS:(13230031)(4636009)(39860400002)(346002)(376002)(136003)(396003)(230922051799003)(64100799003)(1800799009)(451199024)(186009)(82310400011)(36840700001)(40470700004)(46966006)(83380400001)(2616005)(478600001)(36860700001)(86362001)(36756003)(356005)(7636003)(82740400003)(40480700001)(336012)(26005)(66574015)(2906002)(47076005)(110136005)(966005)(6666004)(40460700003)(1076003)(4326008)(8936002)(8676002)(5660300002)(426003)(41300700001)(316002)(70206006)(54906003)(70586007);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2023 16:49:59.5426 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a9ea5f04-db78-4564-ceaf-08dbc042f3f2 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.118.233];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000044FC.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB8211 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Thu, 28 Sep 2023 09:50:40 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1778330705631757884 X-GMAIL-MSGID: 1778330705631757884 The reslock is taken while refresh is called but iommu_lock is more specific to this resource. So take the iommu_lock during cvq iotlb refresh. Based on Eugenio's patch [0]. [0] https://lore.kernel.org/lkml/20230112142218.725622-4-eperezma@redhat.com/ Acked-by: Jason Wang Suggested-by: Eugenio Pérez Signed-off-by: Dragos Tatulea Reviewed-by: Eugenio Pérez --- drivers/vdpa/mlx5/core/mr.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/drivers/vdpa/mlx5/core/mr.c b/drivers/vdpa/mlx5/core/mr.c index fcb6ae32e9ed..587300e7c18e 100644 --- a/drivers/vdpa/mlx5/core/mr.c +++ b/drivers/vdpa/mlx5/core/mr.c @@ -590,11 +590,19 @@ int mlx5_vdpa_update_cvq_iotlb(struct mlx5_vdpa_dev *mvdev, struct vhost_iotlb *iotlb, unsigned int asid) { + int err; + if (mvdev->group2asid[MLX5_VDPA_CVQ_GROUP] != asid) return 0; + spin_lock(&mvdev->cvq.iommu_lock); + prune_iotlb(mvdev); - return dup_iotlb(mvdev, iotlb); + err = dup_iotlb(mvdev, iotlb); + + spin_unlock(&mvdev->cvq.iommu_lock); + + return err; } int mlx5_vdpa_create_dma_mr(struct mlx5_vdpa_dev *mvdev) From patchwork Thu Sep 28 16:45:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Dragos Tatulea X-Patchwork-Id: 146249 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:cae8:0:b0:403:3b70:6f57 with SMTP id r8csp3505877vqu; Thu, 28 Sep 2023 11:12:49 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEiwh47CdGA/B1xpOTUxospntbpgZYnhPj5WYxWzgHAMdjTqkyK58iNxFeNSy98tTvCdgtE X-Received: by 2002:a05:6a00:2490:b0:690:c79c:cc9c with SMTP id c16-20020a056a00249000b00690c79ccc9cmr2317650pfv.14.1695924768747; Thu, 28 Sep 2023 11:12:48 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1695924768; cv=pass; d=google.com; s=arc-20160816; b=LErNH9YZXRzRH3GC0uDSrbglFyLcww8FfI5hcH9oRAommzdRSbmumAP4Z6PfXd8I/B sISDGb+C9NLlOQIWyeyIDUk5RTquEXLEfFnOWQKJfFbVNXdMQuNuEzUnmogEo7G1LSMX m9ZjlGLBAK+KogRhcyYB1ccUd2dXLcIyQQNYo2RbUAUsl66PwfwRWOSCoCzka/VBh4gU +WCjOIB/5eA4HQ6Uu2k/uUYSMn2GaCsJHd7/txj1eJfpVMS+vUP5jNvUuIP5Isuu69fn spIgXNf0vtpI7OL+K2kZPCo/8ybDEPaotZ1ZGqqfnwz1S33yjjXJf5IRytL4jHfhGMdd e8hg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=vD/9ODOQ8spLLT2XMllWp5GS6vJBa2gpMx9/5yXJbBI=; fh=4SYmPDx0sMrYC0lSOUfhTOMEzVMfG5iPEyPqlL4Bs2M=; b=O45yFLWi3V0U83vs/gUPPydOz8ADpLqGZqK/pju7QqxFXAHueNuTF9yD9eARsMNtIX JIVeqhRoEySs1xedMpLhDW6YodiT8jIag9MUeur7vNM+BAixwE0q92i9oa2T9Twrl+0r HnlZHddHBvJaInzXaA3H5DdcQbLHXZHFpblpfFrrhswEtBDjTFNONwliYOXPzo0OTNJy w5KcQFlnbrQwGngW2AYwFkK5DIJ2xgOUyBj1ouWxaY+DmSoeDgxsvXo9/todAiAfRDeV TkpBfLkdIlOHgzulTv5gBP+FHne0o5UwDlNwBkIYEgQdf4DlgHkaT5+3wtQhmPZLEJBq 8jhA== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=kp1LG4gm; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id bx37-20020a056a02052500b00570979732e4si22360411pgb.392.2023.09.28.11.12.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 28 Sep 2023 11:12:48 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=kp1LG4gm; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 2C51581E14E0; Thu, 28 Sep 2023 09:50:50 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231796AbjI1Qug (ORCPT + 21 others); Thu, 28 Sep 2023 12:50:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34574 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231949AbjI1QuN (ORCPT ); Thu, 28 Sep 2023 12:50:13 -0400 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2081.outbound.protection.outlook.com [40.107.244.81]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A08D6CF7 for ; Thu, 28 Sep 2023 09:50:04 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WgREUROKxv54f26aqI223bwY5/Y/YrFNa8OL2AbXu4dUSRHEySgTz1WN2yjYjePxyjnvZdeS2naogDxgsPrgb51P8gRWpDkVgdt585e33HzqNsL2r+1I1BAPzrHF1pIKwQELldMAVPefKuSRUTgbVVYZVAyINggJLwNd+rMq0Qy9EQm8L6/Lex3Q6/z3NzZxzOc1SH1oDGkF6/01q8HByKIyzhhFSkWxrxg6SX7qfrVhLCzMBdJDXyq7ORJrz73TFeyOa8q0ovv2ollyfEos3j6moann1G/+fNwbfQlntYJU5qwTMDYUfQh4Y7lux0lFFUlz7EAyjiD2uNqxkqGzBg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=vD/9ODOQ8spLLT2XMllWp5GS6vJBa2gpMx9/5yXJbBI=; b=IQVDCc7HZn2Zz1z91LMv8DZPV0O8ta9v4CkHpMPvQoi9BSwICCyz4j9kZLjhxk3VgcWDWYxsMbmC6mlMtIAAKLM4xnFOf+Leo0GarBaCWN4dEezucjH3vy1U4N8Lv2Z8zeyfks3UcQmvPnzwL03/IzGmygkjvlA9e2R3NsEfooVvygbC9r1mj/Ln5AYID2YZPuIO+SkVtpGBNB9mc5TQmsTo3Qho/cqHU0XxIO8XUL53H4IbEIY5D7O8uJ8eqJFTVizENIuafs67kyLvhbtTWFlJLoT9oQUiqjJ50WpcYkY2QyyQsgHpkhvoYTUNpmTKr2JeNya8iTV+4IgzgTW2Ag== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.233) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vD/9ODOQ8spLLT2XMllWp5GS6vJBa2gpMx9/5yXJbBI=; b=kp1LG4gmwsoCUzrmFugGPyqMds7q7JSOf0/jW9Gz/Sba5FMSzkWMObKqGO0LGDea90gCtG7NWeM5QbyExZp/9qR3jDIdCIUzG4x9I2dNuZtqTscLIA7nOSTC+W8xN99ryejbNT4qZG061ycn08NFUNT4NxECVjBwA8sh3xQ8XUdZNo+x3ydnUxg7bajLlH9MmBGlNKYsTvnocyX+GMUWqcDVUQulE17MPIukaiypDyQhb+XF8vDvfK287Lp5HDo9macBZtZgMk1BXcPUtpizg9AHOQoQtcZxGKrwYL8MIK9TlIsqhaGSvaABZu2K0qnEwIabIuHfFcEX0af+vKAmOA== Received: from MW4P223CA0027.NAMP223.PROD.OUTLOOK.COM (2603:10b6:303:80::32) by IA1PR12MB7685.namprd12.prod.outlook.com (2603:10b6:208:423::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6813.25; Thu, 28 Sep 2023 16:50:01 +0000 Received: from CO1PEPF000044FC.namprd21.prod.outlook.com (2603:10b6:303:80:cafe::ea) by MW4P223CA0027.outlook.office365.com (2603:10b6:303:80::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.24 via Frontend Transport; Thu, 28 Sep 2023 16:50:01 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.233) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.233 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.233; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.233) by CO1PEPF000044FC.mail.protection.outlook.com (10.167.241.202) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6863.9 via Frontend Transport; Thu, 28 Sep 2023 16:50:01 +0000 Received: from drhqmail201.nvidia.com (10.126.190.180) by mail.nvidia.com (10.127.129.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Thu, 28 Sep 2023 09:49:49 -0700 Received: from drhqmail202.nvidia.com (10.126.190.181) by drhqmail201.nvidia.com (10.126.190.180) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Thu, 28 Sep 2023 09:49:49 -0700 Received: from c-237-113-220-225.mtl.labs.mlnx (10.127.8.12) by mail.nvidia.com (10.126.190.181) with Microsoft SMTP Server id 15.2.986.41 via Frontend Transport; Thu, 28 Sep 2023 09:49:47 -0700 From: Dragos Tatulea To: , , "Michael S . Tsirkin" , Jason Wang , Xuan Zhuo CC: , Dragos Tatulea , Subject: [PATCH vhost 08/16] vdpa/mlx5: Collapse "dvq" mr add/delete functions Date: Thu, 28 Sep 2023 19:45:19 +0300 Message-ID: <20230928164550.980832-10-dtatulea@nvidia.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230928164550.980832-2-dtatulea@nvidia.com> References: <20230928164550.980832-2-dtatulea@nvidia.com> MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000044FC:EE_|IA1PR12MB7685:EE_ X-MS-Office365-Filtering-Correlation-Id: f2cac3b4-4ea4-4208-250b-08dbc042f4ed X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: a6WjB0N6Im6iWAEj2BeqkumZvEi3rqsmxCtknIwSktltuKSRJouJMtihX2INI25wSMRRVM4rrEhX7MNLgBo3A6qUaDAwV8lp5EZMUoKADeqDn8S/vZDIhYbZaPG2ufG9fMas6eGbRzAFfGrDX0zFh02lzu7om4C2voiU8mntdPGHpsUfFJQtL1oe9tz9Si0RElKLdZpD+ZMZDnfW6s1g6InC7ScYT+hVvMq0TR36eO7EnF/PZnUhu4bh2+8FhTr8J4nZ5ylG++zd71iEq9pEAaJ9C8j3zQ1cju02kJRj922AUHWtJB7XWn54muB+5IG/SY5TdmRri2a+3IXasOzg/burEiUmFzR2f91f4T6GGtfj1dwaZTyaBggo34D0Iuj1Y2cNtToJCzbjQpO7Bh2cO2mk/485H7MIcUxGBojuAkdPDl3c11DEHIbdCbfzYbJlFyOtnZvaoY4dkJgVlAny2txNz9TwGN0zZdz4A6/GtHLwOBOx9xtBNmpIWor7QInVZOpHD9pGsm+rX8VeEL2OyLxDio99YBdMKFnt2TyNyjFfQ5COdJ1oeX4FuaGp9uEBbD2If799fCI+rCPyg+kSyqH7u0wODOrp0c5dowQFhmK373WhWEscZzZoX2dMUjCUXEzaRmX3bxQuNy+vinXlOt812Qux7Fu42RZjn19Vyv5S/ZHFYsR/pTz26JTCJ/1/S9x7pbQkEXDifawNoN/jBDur7jQ0wGH3gzQaPmlGnsDq47wU5R3If54mb8cdtkcd X-Forefront-Antispam-Report: CIP:216.228.118.233;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc7edge2.nvidia.com;CAT:NONE;SFS:(13230031)(4636009)(346002)(376002)(136003)(39860400002)(396003)(230922051799003)(64100799003)(186009)(82310400011)(1800799009)(451199024)(36840700001)(46966006)(40470700004)(36756003)(40460700003)(86362001)(40480700001)(8936002)(26005)(70206006)(2616005)(426003)(54906003)(5660300002)(70586007)(8676002)(336012)(4326008)(41300700001)(316002)(110136005)(1076003)(36860700001)(2906002)(83380400001)(6666004)(478600001)(47076005)(82740400003)(7636003)(356005);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2023 16:50:01.1677 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f2cac3b4-4ea4-4208-250b-08dbc042f4ed X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.118.233];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000044FC.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB7685 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Thu, 28 Sep 2023 09:50:50 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1778306010263402547 X-GMAIL-MSGID: 1778306010263402547 Now that the cvq code is out of mlx5_vdpa_create/destroy_mr, the "dvq" functions can be folded into their callers. Having "dvq" in the naming will no longer be accurate in the downstream patches. Acked-by: Jason Wang Signed-off-by: Dragos Tatulea Acked-by: Eugenio Pérez --- drivers/vdpa/mlx5/core/mr.c | 16 +++++----------- 1 file changed, 5 insertions(+), 11 deletions(-) diff --git a/drivers/vdpa/mlx5/core/mr.c b/drivers/vdpa/mlx5/core/mr.c index 587300e7c18e..fde00497f4ad 100644 --- a/drivers/vdpa/mlx5/core/mr.c +++ b/drivers/vdpa/mlx5/core/mr.c @@ -489,7 +489,7 @@ static void destroy_user_mr(struct mlx5_vdpa_dev *mvdev, struct mlx5_vdpa_mr *mr } } -static void _mlx5_vdpa_destroy_dvq_mr(struct mlx5_vdpa_dev *mvdev, unsigned int asid) +static void _mlx5_vdpa_destroy_mr(struct mlx5_vdpa_dev *mvdev, unsigned int asid) { struct mlx5_vdpa_mr *mr = &mvdev->mr; @@ -513,7 +513,7 @@ void mlx5_vdpa_destroy_mr_asid(struct mlx5_vdpa_dev *mvdev, unsigned int asid) mutex_lock(&mr->mkey_mtx); - _mlx5_vdpa_destroy_dvq_mr(mvdev, asid); + _mlx5_vdpa_destroy_mr(mvdev, asid); mutex_unlock(&mr->mkey_mtx); } @@ -524,9 +524,9 @@ void mlx5_vdpa_destroy_mr(struct mlx5_vdpa_dev *mvdev) prune_iotlb(mvdev); } -static int _mlx5_vdpa_create_dvq_mr(struct mlx5_vdpa_dev *mvdev, - struct vhost_iotlb *iotlb, - unsigned int asid) +static int _mlx5_vdpa_create_mr(struct mlx5_vdpa_dev *mvdev, + struct vhost_iotlb *iotlb, + unsigned int asid) { struct mlx5_vdpa_mr *mr = &mvdev->mr; int err; @@ -550,12 +550,6 @@ static int _mlx5_vdpa_create_dvq_mr(struct mlx5_vdpa_dev *mvdev, return 0; } -static int _mlx5_vdpa_create_mr(struct mlx5_vdpa_dev *mvdev, - struct vhost_iotlb *iotlb, unsigned int asid) -{ - return _mlx5_vdpa_create_dvq_mr(mvdev, iotlb, asid); -} - int mlx5_vdpa_create_mr(struct mlx5_vdpa_dev *mvdev, struct vhost_iotlb *iotlb, unsigned int asid) { From patchwork Thu Sep 28 16:45:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Dragos Tatulea X-Patchwork-Id: 146229 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:cae8:0:b0:403:3b70:6f57 with SMTP id r8csp3478722vqu; Thu, 28 Sep 2023 10:27:41 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEoU+/Lkjre+XlnLhdt4UHWIJ+22dlzTrR9wuDrNIc2wkVJZCTBwhxtb5KGNZYh/O1FiaIS X-Received: by 2002:a05:6870:230d:b0:1bb:a912:9339 with SMTP id w13-20020a056870230d00b001bba9129339mr1999425oao.7.1695922060985; Thu, 28 Sep 2023 10:27:40 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1695922060; cv=pass; d=google.com; s=arc-20160816; b=vhregqFhxVzU3T2XeDiWv40WWF8UtG2LwYWxF4MCz4VDCz2bUf9nYJCLVcJJDfiK2i 7rifwn0NrQr1k9osJ/ZKjAS+3IzRXzubUjgHtbe5lC/w/nHnSP2NVtAvWzXWDzC0g1Wr 3CGsHABlGSeUNJEHwTc6gK2snh3jf1jmUI6cQa43gCCdOiPtGwrWEEcfLj9ntWk/YixA +E+nBcBLxD3v4rO6GsiIZd38t27Sd63TajtWOaxMUp5cw3Z831Own6DqvUmpb12/JkoP PXl19cCDE0/J52xUpY1LWM+6AxKG0eMihUQD31qFjYGaVUeZy1cqWoESthENipuUAP/A gItw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=eLEYp8FG8MYA2AcSAX//hEYnU7j56KnewdXwaWn6gPc=; fh=4SYmPDx0sMrYC0lSOUfhTOMEzVMfG5iPEyPqlL4Bs2M=; b=LvgwP44KNErhXcpCfAGbHr0THf/iZ7j58I9d70YYoW0iDDntGyfc9X1uaYyCShATiU kso05pmFyEHurRibVO7JBDevi+7sYnBxm3umUFOgEwFS58i96C70r0IoTsp954sBGJX/ tpSXaeVWCK8DbA48049WmMo1rcf3cBfrYj5MOUuhK2wfpRm2DfKsclE2BzT01D03hYDv 4d2Py43m+6TNgMH77wUZ24E4ncabd0gh4sk5OaLi5Wa4b4O/Mm4Q9Gc5KesdYoVduGux f//K5zMMSO8mmTBDe6YRSrMCXuphIr5b6hLu8nDeXtjBGfqpmbFccEEIkBokt8+k4VCz UppA== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=UEjRnEcq; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id k67-20020a633d46000000b0057c2f614756si17890178pga.212.2023.09.28.10.27.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 28 Sep 2023 10:27:40 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=UEjRnEcq; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 4358381D2AC2; Thu, 28 Sep 2023 09:50:30 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232031AbjI1QuU (ORCPT + 21 others); Thu, 28 Sep 2023 12:50:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56580 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231822AbjI1QuG (ORCPT ); Thu, 28 Sep 2023 12:50:06 -0400 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2085.outbound.protection.outlook.com [40.107.93.85]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7AFCE1B3 for ; Thu, 28 Sep 2023 09:50:00 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Kas8/nFuK27ABHt5Kg9sZHisGSgMpSOF6WXjLx1LzkaEWfVw0N2BImzvNZsCAxbcQJlWu8JJYt7wh49vHkS3lf3r8kCIfr/updAVnbBjeYXB6bjuH7wsY5vFMlSfDvYH9zKX+XknZ/bE/sjy3XMQXBqkaoD+0wi2pmaBRFxm6BHo8I/8Qr4ukavY7jdcP0IuDrb9nC6yApVYuqC0UwQ7MA6dh3kBE1TzSNQV3rH23TvMqn9/xGBZs1LqtpMTPiPUW0OEV8xXlr7tZt6Ayhh06NjRSRuJj6KXZttfwtz4JsFEACo3sZ/sZp+AHZ93mP7oGZ5ux1A2LcGACuufbzzSJw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=eLEYp8FG8MYA2AcSAX//hEYnU7j56KnewdXwaWn6gPc=; b=dYTePof3hdLi57FkxGiJV8pAqZqsy0Mbx1a4fv9f5JjuPexFB5CWCudWQ1zqYvKoNT472n+t8caDmbGdEAiNMjq0hhDdpnjuNmNpVopVu/Ka6n7V6yaaxrco74BBtVtAlDZFtmbUPIfG3fxLyqWVIgDrSAfP7z8grLcrSHRTL8SGqp2xemWe7W64ppIGIJbn8p9maFfUQeMnP69IOAlEoJ0SWvYc8dmSZzTu8REXoLekakMY1Zgz/6kIt2x/wlrTdEyXcwKC8zSWw+meAxrZismv767oXT8sQwmpyCoFxo0AR5i5dyLVIMzFXkKnKRMmUxAdv8uQeNmzUuovvrocGA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.232) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=eLEYp8FG8MYA2AcSAX//hEYnU7j56KnewdXwaWn6gPc=; b=UEjRnEcqM0y2T+YtWJCBFQQZgSNV/q6Utio6hqf+iKpMloS+oANiEOJpJ/7WdaOknsSVWFdWTbFZ0CrZvMz3Up4EFwlF4pNqHkvt9vj0s4DRsjlrp/UtNGRZvTadSrzZhiQj0/KR//B+pzgsrPKquH2Ikgk3l145eSJd9XkhB72r9uHtnzwQfVEiYqt8hSk29l0gVupII8/wt9hPnGMSHwItNPvYAJLVrNF9lazGqruB0vLEke4rKJgF44eEEB7W5uG/KvsHOorH76z8lq99YpewxbV8x2yJJQw0K/pX3e12cGAkNDpCarramXewUNEKlPKvG8S3lPQ77Pi/mdm3TQ== Received: from CY5PR17CA0060.namprd17.prod.outlook.com (2603:10b6:930:12::27) by SN7PR12MB7250.namprd12.prod.outlook.com (2603:10b6:806:2aa::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6813.28; Thu, 28 Sep 2023 16:49:57 +0000 Received: from CY4PEPF0000E9CF.namprd03.prod.outlook.com (2603:10b6:930:12:cafe::55) by CY5PR17CA0060.outlook.office365.com (2603:10b6:930:12::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.22 via Frontend Transport; Thu, 28 Sep 2023 16:49:57 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.232) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.232 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.232; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.232) by CY4PEPF0000E9CF.mail.protection.outlook.com (10.167.241.142) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.19 via Frontend Transport; Thu, 28 Sep 2023 16:49:57 +0000 Received: from drhqmail201.nvidia.com (10.126.190.180) by mail.nvidia.com (10.127.129.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Thu, 28 Sep 2023 09:49:52 -0700 Received: from drhqmail202.nvidia.com (10.126.190.181) by drhqmail201.nvidia.com (10.126.190.180) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Thu, 28 Sep 2023 09:49:52 -0700 Received: from c-237-113-220-225.mtl.labs.mlnx (10.127.8.12) by mail.nvidia.com (10.126.190.181) with Microsoft SMTP Server id 15.2.986.41 via Frontend Transport; Thu, 28 Sep 2023 09:49:50 -0700 From: Dragos Tatulea To: , , "Michael S . Tsirkin" , Jason Wang , Xuan Zhuo CC: , Dragos Tatulea , Subject: [PATCH vhost 09/16] vdpa/mlx5: Rename mr destroy functions Date: Thu, 28 Sep 2023 19:45:20 +0300 Message-ID: <20230928164550.980832-11-dtatulea@nvidia.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230928164550.980832-2-dtatulea@nvidia.com> References: <20230928164550.980832-2-dtatulea@nvidia.com> MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000E9CF:EE_|SN7PR12MB7250:EE_ X-MS-Office365-Filtering-Correlation-Id: 275dd05c-ab6c-4c9c-3364-08dbc042f2b8 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: e7o7g7fIWbhi3/hkxirxgCjhmityxNha3ESQQZjsQZUmZgTxECh1deUEjWdH9oul+eilzE0NOy1Pr0QUAynCW3rFLpYl9q6losZdlZLH3lhw4lmVneQtrI2GlRHTtZX1eRI2kAVGk6j/giH3RyMXbrszVs4KKRWwbUZrvihyNoGCBhumDQLB346+vRf5LK16lwGznqs0uGKY5TgkmAocomqa31w1AKcFVM0VIHjsf4bRExqxb2B067N5af08m/7mwdD+2PxpNDkwpFEs/i8MPh1hKStHEIxjQie32J5gnLQCEIYDXz3NWBWZUbry7faSPuogbfPD+dONDLLaxE5gEpsowsZydw620mrgxXH3lYgWKW/riWIGwe/6TJPlpiP4JTudwdU76P4MQtHXyZ4oEfwtTZtfOzalkVDZf674i6Pqt9vnm6TJDWwj7pj2kEwlP0sfNAQn69sTGZW4C627G7roG23azxVdizAVPXHnp1VaKfnPZvbgGIbwGGmFq3gWzEBBumOZSF79ciFsDdX6JWjRLyoZ+v3dhCFMXQQ+0KLbgEwkHdyXmTHI3NIB77FA3b618hWP/ItcLgSn8qD+X3tE0dArNc5SgkCmnhBbjtfR+/QzEf1i4KEYzj9EMnJOFwcedTCSCMDUFvbcB6XL4HNrXvxkRrWczkPp7CppZSV+ZSbHOmq/ax9v5ccoh/Lwf6vp47hEMzOUxa74mpFi4asWn9Mbbx37uO06kSxgGh0DL5Mudv7aNXpN/LKsx8zU X-Forefront-Antispam-Report: CIP:216.228.118.232;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc7edge1.nvidia.com;CAT:NONE;SFS:(13230031)(4636009)(376002)(136003)(396003)(346002)(39860400002)(230922051799003)(82310400011)(64100799003)(1800799009)(186009)(451199024)(40470700004)(46966006)(36840700001)(26005)(6666004)(1076003)(478600001)(2616005)(36860700001)(7636003)(426003)(36756003)(336012)(86362001)(356005)(5660300002)(83380400001)(82740400003)(47076005)(40460700003)(4326008)(40480700001)(8936002)(2906002)(8676002)(41300700001)(54906003)(316002)(70206006)(110136005)(70586007);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2023 16:49:57.4652 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 275dd05c-ab6c-4c9c-3364-08dbc042f2b8 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.118.232];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000E9CF.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB7250 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Thu, 28 Sep 2023 09:50:30 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1778303170978257933 X-GMAIL-MSGID: 1778303170978257933 Make mlx5_destroy_mr symmetric to mlx5_create_mr. Acked-by: Jason Wang Signed-off-by: Dragos Tatulea Acked-by: Eugenio Pérez --- drivers/vdpa/mlx5/core/mlx5_vdpa.h | 4 ++-- drivers/vdpa/mlx5/core/mr.c | 6 +++--- drivers/vdpa/mlx5/net/mlx5_vnet.c | 12 ++++++------ 3 files changed, 11 insertions(+), 11 deletions(-) diff --git a/drivers/vdpa/mlx5/core/mlx5_vdpa.h b/drivers/vdpa/mlx5/core/mlx5_vdpa.h index 554899a80241..e1e6e7aba50e 100644 --- a/drivers/vdpa/mlx5/core/mlx5_vdpa.h +++ b/drivers/vdpa/mlx5/core/mlx5_vdpa.h @@ -118,8 +118,8 @@ int mlx5_vdpa_handle_set_map(struct mlx5_vdpa_dev *mvdev, struct vhost_iotlb *io bool *change_map, unsigned int asid); int mlx5_vdpa_create_mr(struct mlx5_vdpa_dev *mvdev, struct vhost_iotlb *iotlb, unsigned int asid); -void mlx5_vdpa_destroy_mr(struct mlx5_vdpa_dev *mvdev); -void mlx5_vdpa_destroy_mr_asid(struct mlx5_vdpa_dev *mvdev, unsigned int asid); +void mlx5_vdpa_destroy_mr_resources(struct mlx5_vdpa_dev *mvdev); +void mlx5_vdpa_destroy_mr(struct mlx5_vdpa_dev *mvdev, unsigned int asid); int mlx5_vdpa_update_cvq_iotlb(struct mlx5_vdpa_dev *mvdev, struct vhost_iotlb *iotlb, unsigned int asid); diff --git a/drivers/vdpa/mlx5/core/mr.c b/drivers/vdpa/mlx5/core/mr.c index fde00497f4ad..00dcce190a1f 100644 --- a/drivers/vdpa/mlx5/core/mr.c +++ b/drivers/vdpa/mlx5/core/mr.c @@ -507,7 +507,7 @@ static void _mlx5_vdpa_destroy_mr(struct mlx5_vdpa_dev *mvdev, unsigned int asid mr->initialized = false; } -void mlx5_vdpa_destroy_mr_asid(struct mlx5_vdpa_dev *mvdev, unsigned int asid) +void mlx5_vdpa_destroy_mr(struct mlx5_vdpa_dev *mvdev, unsigned int asid) { struct mlx5_vdpa_mr *mr = &mvdev->mr; @@ -518,9 +518,9 @@ void mlx5_vdpa_destroy_mr_asid(struct mlx5_vdpa_dev *mvdev, unsigned int asid) mutex_unlock(&mr->mkey_mtx); } -void mlx5_vdpa_destroy_mr(struct mlx5_vdpa_dev *mvdev) +void mlx5_vdpa_destroy_mr_resources(struct mlx5_vdpa_dev *mvdev) { - mlx5_vdpa_destroy_mr_asid(mvdev, mvdev->group2asid[MLX5_VDPA_DATAVQ_GROUP]); + mlx5_vdpa_destroy_mr(mvdev, mvdev->group2asid[MLX5_VDPA_DATAVQ_GROUP]); prune_iotlb(mvdev); } diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5_vnet.c index aa4896662699..ab196c43694c 100644 --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c @@ -2644,7 +2644,7 @@ static int mlx5_vdpa_change_map(struct mlx5_vdpa_dev *mvdev, goto err_mr; teardown_driver(ndev); - mlx5_vdpa_destroy_mr_asid(mvdev, asid); + mlx5_vdpa_destroy_mr(mvdev, asid); err = mlx5_vdpa_create_mr(mvdev, iotlb, asid); if (err) goto err_mr; @@ -2660,7 +2660,7 @@ static int mlx5_vdpa_change_map(struct mlx5_vdpa_dev *mvdev, return 0; err_setup: - mlx5_vdpa_destroy_mr_asid(mvdev, asid); + mlx5_vdpa_destroy_mr(mvdev, asid); err_mr: return err; } @@ -2797,7 +2797,7 @@ static void mlx5_vdpa_set_status(struct vdpa_device *vdev, u8 status) err_driver: unregister_link_notifier(ndev); err_setup: - mlx5_vdpa_destroy_mr(&ndev->mvdev); + mlx5_vdpa_destroy_mr_resources(&ndev->mvdev); ndev->mvdev.status |= VIRTIO_CONFIG_S_FAILED; err_clear: up_write(&ndev->reslock); @@ -2824,7 +2824,7 @@ static int mlx5_vdpa_reset(struct vdpa_device *vdev) unregister_link_notifier(ndev); teardown_driver(ndev); clear_vqs_ready(ndev); - mlx5_vdpa_destroy_mr(&ndev->mvdev); + mlx5_vdpa_destroy_mr_resources(&ndev->mvdev); ndev->mvdev.status = 0; ndev->mvdev.suspended = false; ndev->cur_num_vqs = 0; @@ -2944,7 +2944,7 @@ static void mlx5_vdpa_free(struct vdpa_device *vdev) ndev = to_mlx5_vdpa_ndev(mvdev); free_resources(ndev); - mlx5_vdpa_destroy_mr(mvdev); + mlx5_vdpa_destroy_mr_resources(mvdev); if (!is_zero_ether_addr(ndev->config.mac)) { pfmdev = pci_get_drvdata(pci_physfn(mvdev->mdev->pdev)); mlx5_mpfs_del_mac(pfmdev, ndev->config.mac); @@ -3474,7 +3474,7 @@ static int mlx5_vdpa_dev_add(struct vdpa_mgmt_dev *v_mdev, const char *name, err_res2: free_resources(ndev); err_mr: - mlx5_vdpa_destroy_mr(mvdev); + mlx5_vdpa_destroy_mr_resources(mvdev); err_res: mlx5_vdpa_free_resources(&ndev->mvdev); err_mpfs: From patchwork Thu Sep 28 16:45:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Dragos Tatulea X-Patchwork-Id: 146390 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:cae8:0:b0:403:3b70:6f57 with SMTP id r8csp3719127vqu; Thu, 28 Sep 2023 18:43:45 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFgDni25yv7fUmZ+pSBsGNlT6FfOZrxnprEJwh873HBkcC7k10+Pb802nea0IwJvroMMW77 X-Received: by 2002:a17:902:a417:b0:1b7:e86f:7631 with SMTP id p23-20020a170902a41700b001b7e86f7631mr2557217plq.19.1695951824673; Thu, 28 Sep 2023 18:43:44 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1695951824; cv=pass; d=google.com; s=arc-20160816; b=Pisf2Mpv1KRLiK58YWh+UAK36KvPq1EcrC5j8G4XbGH+53Ofqu54ZRmIrcVLe3ycyB l3AWdhnSe0CU1iZmerSAap2aQ+tnidRgSEOR7OgVg9nPa4okvD5fFv6cON7UikcDOEyZ slk9/vvl8D7X9myhp6liD9b8QDjkXCsjRej2MjuTz1AY4tRhIip75AnozHF/98zQhRSe xPP3ZzqTz4KYx2bPfu86vqKhfXIccfaUpidPF08vFrEJz4BAZPj8zXasamGy+h6zlF7Q gtGLdxOwVmm2GbpYagEu6TIsHOZsEQhjwo3k/8QppA5fqlmXmcFZ4fs+z+T5RYce8TBF wtKA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=1nqpxFPlC0nJlt8MPwoegjK6o2hj1U1JGXxph7oPUDY=; fh=4SYmPDx0sMrYC0lSOUfhTOMEzVMfG5iPEyPqlL4Bs2M=; b=hJf4r57pfEcK9fMNtZ2lNh8PDo9WEsmu/Q04o2d3WSHwpS6laONHzHzvCN6JLjYeM7 6z3G1VDDEmWtAGYkp0tEjuumbOBWyd7elOYaLRdTaMU42w2a1fQA0c2+R+xNZBMDgkBP qrY7mz3lXwfKiJDSErGw/b/7Pof459Tl4fJ7lzM3SZeXfvPnpyThNR3YA0zKEgpmjkNM PqkvTwImgenhQNdN0ZMMhDqI/CVCL5tNWeOG5insLZLbBcTNQnYv4pceqeBFxMsipQuW dPvbHdbEkUPDFRTydyB6b0WLceGQVqBp/LEmHsCVgpZWXtie9UgfGWr7AGoDl+uo+EVB R2Zw== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=qa6z+RQy; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: from lipwig.vger.email (lipwig.vger.email. [2620:137:e000::3:3]) by mx.google.com with ESMTPS id b1-20020a170902d50100b001baff05d890si7183074plg.313.2023.09.28.18.43.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 28 Sep 2023 18:43:44 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) client-ip=2620:137:e000::3:3; Authentication-Results: mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=qa6z+RQy; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id 860968194580; Thu, 28 Sep 2023 09:50:59 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232152AbjI1Quj (ORCPT + 21 others); Thu, 28 Sep 2023 12:50:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34596 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231974AbjI1QuQ (ORCPT ); Thu, 28 Sep 2023 12:50:16 -0400 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2086.outbound.protection.outlook.com [40.107.94.86]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CA55E10C8 for ; Thu, 28 Sep 2023 09:50:06 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Z0NryE7mczykT7tc2m5fAw4Ke4lOZhevG3v3Yh2G4vQF5L84uWDLpAZl/KlKQiMrsYRjpE24FcR1SWaOweTCSm4ytOrTv0L5nxKoaF7lxhdu7fXVAggPT9K9dp6u/PqcD8pZWWmqjZ4akDW5uGdt17YCv3xT6qtTDGa0y/GGJepg0OJtGbDKYlH6IYsvjkzuOgP0zuushmlPDW0VZKPkldnvkruA4jlOD7wq3xvLMMtagxKUrKpR1/VkrS6X8Mf2tjSYTeCv5OLz5p7LB6MeKAN//r6YAZV9QSvoLNirMBct+rbSucazmxDGVPmY5+Ov2lAC3dCdryMrZJcyJqMusw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=1nqpxFPlC0nJlt8MPwoegjK6o2hj1U1JGXxph7oPUDY=; b=kA5BsEi4w6QrtvGv8cPZ0pW1SZCaq3n/M+DkLqt9Ozv2V1xGpFx5qXZxhWq0At7Bi7orVTcUAGsudqSqKuH2LIVeGUDdKyb9/BDLQF8ip1Re9wgketytTAPTRWr/9rKOpGVWLRG43EsHbE+KVJ/PQIb3hXG+xP9GwUip04eZp2S0pJDq+eTF378aXebWv+UN2BncoRnMtTWfJ14yDKAq1rMPkacXrLU3A0aAz2eRAyIorKW0yAO7NgMvhgV4mrjty3fzRblfc2PlkNCRetXYUCjyvgmHOrgrAaksNGFxiNG6wn8b0IViMGQzSs7R0wnzsvNcpBBPdWBdHsHgNZG9/g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.232) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1nqpxFPlC0nJlt8MPwoegjK6o2hj1U1JGXxph7oPUDY=; b=qa6z+RQyOQq2fBLfSQ+s8OB3aUirLzZT1jB7JcaeHGi4lcUBFoZLGU9JLReNiXiVgzBRCglSMcfwMXppC5S4FO3q7RKgUhjaC5yCGUNm4klEd+aL4uYISDdLLTLKJ+L3SY1UsNh8W4DQvxP/ClEOPzpdkQEbIKT4SkvJ+M3++0h9fbMW/L8Ki8rnkBQNvG229enM0reYsdFPB8UHjrrdEpiW8yO9BPy+NiqxL+yx8OOTOZ6+DtgxwcPLMFgbfOUSb1fMbqvn049/Tta4naMAUPyROrYCpF4WenYRLEY5yZ2UIGvNi4oS3nb8T4dwI6fFNuXovLyitZ9putMoOhI9PA== Received: from CY5PR17CA0049.namprd17.prod.outlook.com (2603:10b6:930:12::33) by MW3PR12MB4508.namprd12.prod.outlook.com (2603:10b6:303:5b::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.25; Thu, 28 Sep 2023 16:50:01 +0000 Received: from CY4PEPF0000E9CF.namprd03.prod.outlook.com (2603:10b6:930:12:cafe::29) by CY5PR17CA0049.outlook.office365.com (2603:10b6:930:12::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.25 via Frontend Transport; Thu, 28 Sep 2023 16:50:00 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.232) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.232 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.232; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.232) by CY4PEPF0000E9CF.mail.protection.outlook.com (10.167.241.142) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.19 via Frontend Transport; Thu, 28 Sep 2023 16:50:00 +0000 Received: from drhqmail203.nvidia.com (10.126.190.182) by mail.nvidia.com (10.127.129.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Thu, 28 Sep 2023 09:49:55 -0700 Received: from drhqmail202.nvidia.com (10.126.190.181) by drhqmail203.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Thu, 28 Sep 2023 09:49:55 -0700 Received: from c-237-113-220-225.mtl.labs.mlnx (10.127.8.12) by mail.nvidia.com (10.126.190.181) with Microsoft SMTP Server id 15.2.986.41 via Frontend Transport; Thu, 28 Sep 2023 09:49:53 -0700 From: Dragos Tatulea To: , , "Michael S . Tsirkin" , Jason Wang , Xuan Zhuo CC: , Dragos Tatulea , Subject: [PATCH vhost 10/16] vdpa/mlx5: Allow creation/deletion of any given mr struct Date: Thu, 28 Sep 2023 19:45:21 +0300 Message-ID: <20230928164550.980832-12-dtatulea@nvidia.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230928164550.980832-2-dtatulea@nvidia.com> References: <20230928164550.980832-2-dtatulea@nvidia.com> MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000E9CF:EE_|MW3PR12MB4508:EE_ X-MS-Office365-Filtering-Correlation-Id: 3f9f4de9-b8bb-437d-f44a-08dbc042f4bf X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 79pJobIvEd0XNzl3PsD0vpi9HBZaXighEZZpWN02bCfqY93749oOEsdSuh84xRFfPixcyxXmqlJXYiUp151f+osipueRjjnDQZ9OaWiK3JAc9VP8biii75ZnF0ltJUk8vd4TS7MlO9cxRqXrKJoZHCqsqxoKF8KnDzkwNomhFyzqPnmHTNOWd9aiDJeqLt963B4cIEv18D79rhe8wi2Ic3EGvn378nrAn3MumCjJcxmLGQpwktWbhDM5B/yCZ6y65DgmeJQ+gMqIfy7tXFiJ5xVEZhZ2KdwMRQXI8OrG1D/BtPfCLH4TVn4xHE54fnBF69JSFp57BV2+lm3jOnAd52zYM/uISEaKBq2OPfnpvj/krunMXR08fqZf+rkSXQxj5AjGiO7gLoFCaIjAFKFHzIXabVYZImwc0Y/bNBr84XuaKQQ9bBCH3kipw0dQZ4SwYayILdjKyWXgTgCsHXtExfPm+CreSFfd9z/hxbAvFaF/ebov7D1iWaWvn9CiCfq/hlno/gHPpHz56hHJNTkyhdbykwh+Xbe3rnvn4i816j0WBm0Q69hYYVwBFQcIP8DV72rpVKrEEl2XIWSllNimHzLLic/mqpgwr7PwekK13VyK1S75P3fOciKNpVE0FnGrTWgq5vgvaT6oYY9h0fUlJ0ubq2I2ArnWJ/U9c0CeY2n4DU3XublBDozdiuwTZxj59fgcFbdeaMkXTBV0GSDTQ7AbgEaJrMlZwZhyJYFjqOx2IXaw2KVeQzTVt2xP4QbY X-Forefront-Antispam-Report: CIP:216.228.118.232;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc7edge1.nvidia.com;CAT:NONE;SFS:(13230031)(4636009)(376002)(346002)(136003)(39860400002)(396003)(230922051799003)(82310400011)(1800799009)(186009)(451199024)(64100799003)(36840700001)(46966006)(40470700004)(478600001)(40460700003)(5660300002)(7636003)(6666004)(40480700001)(41300700001)(54906003)(70206006)(70586007)(110136005)(316002)(8936002)(4326008)(8676002)(36756003)(86362001)(83380400001)(47076005)(336012)(426003)(2906002)(36860700001)(356005)(82740400003)(26005)(1076003)(2616005);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2023 16:50:00.8558 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 3f9f4de9-b8bb-437d-f44a-08dbc042f4bf X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.118.232];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000E9CF.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW3PR12MB4508 X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Thu, 28 Sep 2023 09:50:59 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1778334380657627177 X-GMAIL-MSGID: 1778334380657627177 This patch adapts the mr creation/deletion code to be able to work with any given mr struct pointer. All the APIs are adapted to take an extra parameter for the mr. mlx5_vdpa_create/delete_mr doesn't need a ASID parameter anymore. The check is done in the caller instead (mlx5_set_map). This change is needed for a followup patch which will introduce an additional mr for the vq descriptor data. Signed-off-by: Dragos Tatulea Acked-by: Eugenio Pérez --- drivers/vdpa/mlx5/core/mlx5_vdpa.h | 8 +++-- drivers/vdpa/mlx5/core/mr.c | 53 ++++++++++++++---------------- drivers/vdpa/mlx5/net/mlx5_vnet.c | 10 ++++-- 3 files changed, 36 insertions(+), 35 deletions(-) diff --git a/drivers/vdpa/mlx5/core/mlx5_vdpa.h b/drivers/vdpa/mlx5/core/mlx5_vdpa.h index e1e6e7aba50e..01d4ee58ccb1 100644 --- a/drivers/vdpa/mlx5/core/mlx5_vdpa.h +++ b/drivers/vdpa/mlx5/core/mlx5_vdpa.h @@ -116,10 +116,12 @@ int mlx5_vdpa_create_mkey(struct mlx5_vdpa_dev *mvdev, u32 *mkey, u32 *in, int mlx5_vdpa_destroy_mkey(struct mlx5_vdpa_dev *mvdev, u32 mkey); int mlx5_vdpa_handle_set_map(struct mlx5_vdpa_dev *mvdev, struct vhost_iotlb *iotlb, bool *change_map, unsigned int asid); -int mlx5_vdpa_create_mr(struct mlx5_vdpa_dev *mvdev, struct vhost_iotlb *iotlb, - unsigned int asid); +int mlx5_vdpa_create_mr(struct mlx5_vdpa_dev *mvdev, + struct mlx5_vdpa_mr *mr, + struct vhost_iotlb *iotlb); void mlx5_vdpa_destroy_mr_resources(struct mlx5_vdpa_dev *mvdev); -void mlx5_vdpa_destroy_mr(struct mlx5_vdpa_dev *mvdev, unsigned int asid); +void mlx5_vdpa_destroy_mr(struct mlx5_vdpa_dev *mvdev, + struct mlx5_vdpa_mr *mr); int mlx5_vdpa_update_cvq_iotlb(struct mlx5_vdpa_dev *mvdev, struct vhost_iotlb *iotlb, unsigned int asid); diff --git a/drivers/vdpa/mlx5/core/mr.c b/drivers/vdpa/mlx5/core/mr.c index 00dcce190a1f..6f29e8eaabb1 100644 --- a/drivers/vdpa/mlx5/core/mr.c +++ b/drivers/vdpa/mlx5/core/mr.c @@ -301,10 +301,13 @@ static void unmap_direct_mr(struct mlx5_vdpa_dev *mvdev, struct mlx5_vdpa_direct sg_free_table(&mr->sg_head); } -static int add_direct_chain(struct mlx5_vdpa_dev *mvdev, u64 start, u64 size, u8 perm, +static int add_direct_chain(struct mlx5_vdpa_dev *mvdev, + struct mlx5_vdpa_mr *mr, + u64 start, + u64 size, + u8 perm, struct vhost_iotlb *iotlb) { - struct mlx5_vdpa_mr *mr = &mvdev->mr; struct mlx5_vdpa_direct_mr *dmr; struct mlx5_vdpa_direct_mr *n; LIST_HEAD(tmp); @@ -354,9 +357,10 @@ static int add_direct_chain(struct mlx5_vdpa_dev *mvdev, u64 start, u64 size, u8 * indirect memory key that provides access to the enitre address space given * by iotlb. */ -static int create_user_mr(struct mlx5_vdpa_dev *mvdev, struct vhost_iotlb *iotlb) +static int create_user_mr(struct mlx5_vdpa_dev *mvdev, + struct mlx5_vdpa_mr *mr, + struct vhost_iotlb *iotlb) { - struct mlx5_vdpa_mr *mr = &mvdev->mr; struct mlx5_vdpa_direct_mr *dmr; struct mlx5_vdpa_direct_mr *n; struct vhost_iotlb_map *map; @@ -384,7 +388,7 @@ static int create_user_mr(struct mlx5_vdpa_dev *mvdev, struct vhost_iotlb *iotlb LOG_MAX_KLM_SIZE); mr->num_klms += nnuls; } - err = add_direct_chain(mvdev, ps, pe - ps, pperm, iotlb); + err = add_direct_chain(mvdev, mr, ps, pe - ps, pperm, iotlb); if (err) goto err_chain; } @@ -393,7 +397,7 @@ static int create_user_mr(struct mlx5_vdpa_dev *mvdev, struct vhost_iotlb *iotlb pperm = map->perm; } } - err = add_direct_chain(mvdev, ps, pe - ps, pperm, iotlb); + err = add_direct_chain(mvdev, mr, ps, pe - ps, pperm, iotlb); if (err) goto err_chain; @@ -489,13 +493,8 @@ static void destroy_user_mr(struct mlx5_vdpa_dev *mvdev, struct mlx5_vdpa_mr *mr } } -static void _mlx5_vdpa_destroy_mr(struct mlx5_vdpa_dev *mvdev, unsigned int asid) +static void _mlx5_vdpa_destroy_mr(struct mlx5_vdpa_dev *mvdev, struct mlx5_vdpa_mr *mr) { - struct mlx5_vdpa_mr *mr = &mvdev->mr; - - if (mvdev->group2asid[MLX5_VDPA_DATAVQ_GROUP] != asid) - return; - if (!mr->initialized) return; @@ -507,38 +506,33 @@ static void _mlx5_vdpa_destroy_mr(struct mlx5_vdpa_dev *mvdev, unsigned int asid mr->initialized = false; } -void mlx5_vdpa_destroy_mr(struct mlx5_vdpa_dev *mvdev, unsigned int asid) +void mlx5_vdpa_destroy_mr(struct mlx5_vdpa_dev *mvdev, + struct mlx5_vdpa_mr *mr) { - struct mlx5_vdpa_mr *mr = &mvdev->mr; - mutex_lock(&mr->mkey_mtx); - _mlx5_vdpa_destroy_mr(mvdev, asid); + _mlx5_vdpa_destroy_mr(mvdev, mr); mutex_unlock(&mr->mkey_mtx); } void mlx5_vdpa_destroy_mr_resources(struct mlx5_vdpa_dev *mvdev) { - mlx5_vdpa_destroy_mr(mvdev, mvdev->group2asid[MLX5_VDPA_DATAVQ_GROUP]); + mlx5_vdpa_destroy_mr(mvdev, &mvdev->mr); prune_iotlb(mvdev); } static int _mlx5_vdpa_create_mr(struct mlx5_vdpa_dev *mvdev, - struct vhost_iotlb *iotlb, - unsigned int asid) + struct mlx5_vdpa_mr *mr, + struct vhost_iotlb *iotlb) { - struct mlx5_vdpa_mr *mr = &mvdev->mr; int err; - if (mvdev->group2asid[MLX5_VDPA_DATAVQ_GROUP] != asid) - return 0; - if (mr->initialized) return 0; if (iotlb) - err = create_user_mr(mvdev, iotlb); + err = create_user_mr(mvdev, mr, iotlb); else err = create_dma_mr(mvdev, mr); @@ -550,13 +544,14 @@ static int _mlx5_vdpa_create_mr(struct mlx5_vdpa_dev *mvdev, return 0; } -int mlx5_vdpa_create_mr(struct mlx5_vdpa_dev *mvdev, struct vhost_iotlb *iotlb, - unsigned int asid) +int mlx5_vdpa_create_mr(struct mlx5_vdpa_dev *mvdev, + struct mlx5_vdpa_mr *mr, + struct vhost_iotlb *iotlb) { int err; mutex_lock(&mvdev->mr.mkey_mtx); - err = _mlx5_vdpa_create_mr(mvdev, iotlb, asid); + err = _mlx5_vdpa_create_mr(mvdev, mr, iotlb); mutex_unlock(&mvdev->mr.mkey_mtx); return err; } @@ -574,7 +569,7 @@ int mlx5_vdpa_handle_set_map(struct mlx5_vdpa_dev *mvdev, struct vhost_iotlb *io *change_map = true; } if (!*change_map) - err = _mlx5_vdpa_create_mr(mvdev, iotlb, asid); + err = _mlx5_vdpa_create_mr(mvdev, mr, iotlb); mutex_unlock(&mr->mkey_mtx); return err; @@ -603,7 +598,7 @@ int mlx5_vdpa_create_dma_mr(struct mlx5_vdpa_dev *mvdev) { int err; - err = mlx5_vdpa_create_mr(mvdev, NULL, 0); + err = mlx5_vdpa_create_mr(mvdev, &mvdev->mr, NULL); if (err) return err; diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5_vnet.c index ab196c43694c..256fdd80c321 100644 --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c @@ -2644,8 +2644,8 @@ static int mlx5_vdpa_change_map(struct mlx5_vdpa_dev *mvdev, goto err_mr; teardown_driver(ndev); - mlx5_vdpa_destroy_mr(mvdev, asid); - err = mlx5_vdpa_create_mr(mvdev, iotlb, asid); + mlx5_vdpa_destroy_mr(mvdev, &mvdev->mr); + err = mlx5_vdpa_create_mr(mvdev, &mvdev->mr, iotlb); if (err) goto err_mr; @@ -2660,7 +2660,7 @@ static int mlx5_vdpa_change_map(struct mlx5_vdpa_dev *mvdev, return 0; err_setup: - mlx5_vdpa_destroy_mr(mvdev, asid); + mlx5_vdpa_destroy_mr(mvdev, &mvdev->mr); err_mr: return err; } @@ -2878,6 +2878,9 @@ static int set_map_data(struct mlx5_vdpa_dev *mvdev, struct vhost_iotlb *iotlb, bool change_map; int err; + if (mvdev->group2asid[MLX5_VDPA_DATAVQ_GROUP] != asid) + goto end; + err = mlx5_vdpa_handle_set_map(mvdev, iotlb, &change_map, asid); if (err) { mlx5_vdpa_warn(mvdev, "set map failed(%d)\n", err); @@ -2890,6 +2893,7 @@ static int set_map_data(struct mlx5_vdpa_dev *mvdev, struct vhost_iotlb *iotlb, return err; } +end: return mlx5_vdpa_update_cvq_iotlb(mvdev, iotlb, asid); } From patchwork Thu Sep 28 16:45:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Dragos Tatulea X-Patchwork-Id: 146294 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:cae8:0:b0:403:3b70:6f57 with SMTP id r8csp3581870vqu; Thu, 28 Sep 2023 13:32:19 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHgF2hU5ii9Pb3YmU9KNKBCxIWjWB1zrvjCqqOwTLU0tPmkE7cajG+SnLjy7ADpfV7nRmm1 X-Received: by 2002:a17:902:ecd2:b0:1c4:65d5:34ce with SMTP id a18-20020a170902ecd200b001c465d534cemr4238100plh.31.1695933138998; Thu, 28 Sep 2023 13:32:18 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1695933138; cv=pass; d=google.com; s=arc-20160816; b=PFhZv4tcP148JgolzbrB5aX6RsJMqVngrWsCU5j+Uq7RaoK03UXkcwtCv56Z59K0td 7thPVeQjcEcgO8xL60pjJVgt29f1lqAUt95VDw5Co//6upvupzPAa30GQv1H2lyJp+a5 z/DyVfabcoS+KSyQlTm5SfYKCI3vRxPgVViwzM8DRw8fVQwevJsRcQCE78AiYgcJPNBH Gx7Z8/QKFEkZ+f08ADEw6EiF5cIlUh+ojhX2HLgS/IeeTBuczsBYo6+Xb8pe1/KRWBwg YYefbpP7tzWRKlY/k4QJ4f1/1Rz234w0HbrKgwH3dBmDC3Y6SYytlTqAoPQ1+NYUWeM/ tHHA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=pugKhXn1yyJ2eySbGpZefWpS3s+uIziF5TphRI58E5w=; fh=4SYmPDx0sMrYC0lSOUfhTOMEzVMfG5iPEyPqlL4Bs2M=; b=BYkySk4ROExmMKsfyLmqQOjhELPwq19FLdGOB7l843ZicSGzUyO0HM1mEsV8BWBHaj bN/Z9BSkJUBiERlCKAzCl77Jwk//6ce8MHJy1CtwaJwQ77b4m1FI5WlP6U7ZEA7rA6Aj O+pqDIC31KDdzqXvLmxqdTvC60ntqjufZ6C20p8hi+xmXl8Gx8KEzNUq9qlWlfaZU4YC GdqgA5HJ2ZyU9ehpFcJgTupbsqtfwp/Vb340Xe8WGCWE4Mly9qvBEdWOVdJcs3DvZJSU 92nThm0nqZwwlp9J5TSzuljqOBttEQRPlUQJyNgCWiw06qNhVZfDt/8ZqYII3QbfQUfy jldg== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=cjBZTUUu; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: from howler.vger.email (howler.vger.email. [23.128.96.34]) by mx.google.com with ESMTPS id f20-20020a637554000000b00578b785d46csi19940602pgn.193.2023.09.28.13.32.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 28 Sep 2023 13:32:18 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) client-ip=23.128.96.34; Authentication-Results: mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=cjBZTUUu; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id F21DE801BF64; Thu, 28 Sep 2023 09:51:12 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232091AbjI1Quv (ORCPT + 21 others); Thu, 28 Sep 2023 12:50:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58044 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232070AbjI1Qua (ORCPT ); Thu, 28 Sep 2023 12:50:30 -0400 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2088.outbound.protection.outlook.com [40.107.244.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7D92DCFD for ; Thu, 28 Sep 2023 09:50:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=AvgYBte1eOu0UBDgUh+8ODgy48ddEN03y+E9wEDZLwa+BlnqD4pSqCSWgGgH1rEBGKfVVySRspvVCn2ufSAadOjL5XHthg3vSlHvqkkwGc5MNLSNHv7MpQZdyg11sVXorjOC4ctNCYf3egNn5fQHOqQJuH8PjekPybt0QaDIFtP3FCIYaGXgkzYpZehVKXjtOed7bGIQb5LBzvTVfL2s9rNkrCpD6Uzwi4eL9HXkDySstpz0AWdh7RFZuYS64Pc+qkdcAmm3NEwWlvowKPAKpiWNQkxSKKaeGUWCVUZuAVCGM3eDFF+V92noJPpIPuEyAxIhDCzyNfR3kMjmGmpe0Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=pugKhXn1yyJ2eySbGpZefWpS3s+uIziF5TphRI58E5w=; b=YHyvHb+8frVH2OJZnrRW+ysYrZu1OP3+kVsAFpbzJ+Zvkd1xK3yv6B6M02M8R/prerV0DMYUuyQdAhWKzTxS9iYq5hJHsdZtb1txx85ob513wEHa9blzdfzgz/qYhhbbObT7jw07U7nCghXr79MlV+5AxShtW1JlTj6/B8+61VzKoNAYnGkx+poo0kzBfTnLZ9YKcr0aeAy8QF7w5PjGuB5g1GrYf+3cyures4rYGBm/1zQ2UPbOJosB0re4gXL1SufK+Pqee5mcQnsU8iGRKQISeHSjpF2PHc9l6TwRJ6JMZ43C5pJ2E5W9EgPmmIlfgb3Vo3H3/XUB6a+AeqFYpQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.233) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=pugKhXn1yyJ2eySbGpZefWpS3s+uIziF5TphRI58E5w=; b=cjBZTUUusX34zRWyErUREsLDQOigKqUQnirQrYCZMVucfQbJ5uLXqhCeCPzXyRmPPtTj8u08s1bkDOe6rcHbIiBn3G6pbfAq4f/byCePafn7tT3wAPARzvzzEm+5jxvvKudmFeKPZw87izA4QDKVPnBbzwwdaOr08iHZ8nb+DIGJHu9XUSkstvIJ760sZb/hJ8DEdhc1GiAvjrH7t98NDHu4o/18Zgwbt2c3fbU8fpJD+Cq4+7uN4D799Bo77N7L8ioGxZRfqNezazkgXpTRDfRh/s4/YQk6HoQOCtcf9Gvvm0PiISA6bFz59+/YmaAjsJmslBz/CobRRH0JPO/STg== Received: from MW4P223CA0023.NAMP223.PROD.OUTLOOK.COM (2603:10b6:303:80::28) by DS0PR12MB8766.namprd12.prod.outlook.com (2603:10b6:8:14e::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6813.28; Thu, 28 Sep 2023 16:50:09 +0000 Received: from CO1PEPF000044FC.namprd21.prod.outlook.com (2603:10b6:303:80:cafe::83) by MW4P223CA0023.outlook.office365.com (2603:10b6:303:80::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.25 via Frontend Transport; Thu, 28 Sep 2023 16:50:09 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.233) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.233 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.233; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.233) by CO1PEPF000044FC.mail.protection.outlook.com (10.167.241.202) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6863.9 via Frontend Transport; Thu, 28 Sep 2023 16:50:09 +0000 Received: from drhqmail201.nvidia.com (10.126.190.180) by mail.nvidia.com (10.127.129.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Thu, 28 Sep 2023 09:49:58 -0700 Received: from drhqmail202.nvidia.com (10.126.190.181) by drhqmail201.nvidia.com (10.126.190.180) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Thu, 28 Sep 2023 09:49:58 -0700 Received: from c-237-113-220-225.mtl.labs.mlnx (10.127.8.12) by mail.nvidia.com (10.126.190.181) with Microsoft SMTP Server id 15.2.986.41 via Frontend Transport; Thu, 28 Sep 2023 09:49:56 -0700 From: Dragos Tatulea To: , , "Michael S . Tsirkin" , Jason Wang , Xuan Zhuo CC: , Dragos Tatulea , Subject: [PATCH vhost 11/16] vdpa/mlx5: Move mr mutex out of mr struct Date: Thu, 28 Sep 2023 19:45:22 +0300 Message-ID: <20230928164550.980832-13-dtatulea@nvidia.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230928164550.980832-2-dtatulea@nvidia.com> References: <20230928164550.980832-2-dtatulea@nvidia.com> MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000044FC:EE_|DS0PR12MB8766:EE_ X-MS-Office365-Filtering-Correlation-Id: cd71045f-783c-486a-fd41-08dbc042f9b8 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 4otg5Ue9/XaMs9MDhxqT3Rx0GB80C8kjBAHDhgtCEIQ8Jwds2w2XyMPTewtcHuv+fyNtvH8+UU+PnP2c/6c0Mtrx+HiaIyoH+Pg5XBLO64/8YrqkCpgKsWPzIp1l3i0GGd3Xdk7RssFeOBQJeLSbu+CzICp3kori9NtTp9lJrBuCuZoI/LTu/ynPuk5xqDjyolVhGQUs+wXZBnngNYnm0KvY5xrMJhVhmP82KpwVXakcv0+qeHMPJCxwqbI5To/FdAmrOszQW6D4aLdzgq6emg1Jm/lwOwPwZ48Ru5ZOrCaLcDbCHs4vR+bEAgfIRG6VB7zeqiSOG/DqfMbB/bFFv46qU2jL4kIOsH94gOx/V9w0ZD9Q4RCUPcgwHyvFJyJm55kaa6Kxa75LECTF/sC/SC99js/T+VRzue8knnhqMoe0OfKNpZPCl2JT4G82WJy3EZr13bBrirqlHCLc4ok98PQs28ILDj7LJ34o6nO5MyGAHowiIvnpcY1nGMigAiWoIrk4dp7GixYoIgbcKoZNK/4Z/hm9m380OST0w7+/kqJ5TbptJ23l/LQkmO47IWf0ym8dEwuZud/tnQMxusAiDI5RFr98lJtRQjHKrz8XGSBjSR4EnY0T5c1H7UIWuI9tBZ8/5a13Yf24KOjMlxLqWigx0X/qlP246KhU4ltgi4bgwNj/urgkR/0/cFqWPAamriYinYXM/jzvznzN0ZntNIgxdLZzYsRD/PF9Qfxuh7/cFYiXgAh/b3LrkREW5tVm X-Forefront-Antispam-Report: CIP:216.228.118.233;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc7edge2.nvidia.com;CAT:NONE;SFS:(13230031)(4636009)(376002)(346002)(136003)(39860400002)(396003)(230922051799003)(82310400011)(64100799003)(1800799009)(451199024)(186009)(36840700001)(46966006)(40470700004)(426003)(47076005)(40460700003)(356005)(2906002)(66899024)(86362001)(5660300002)(82740400003)(7636003)(6666004)(36756003)(83380400001)(478600001)(40480700001)(2616005)(4326008)(1076003)(8936002)(41300700001)(8676002)(110136005)(336012)(26005)(54906003)(70586007)(316002)(36860700001)(70206006);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2023 16:50:09.2144 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: cd71045f-783c-486a-fd41-08dbc042f9b8 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.118.233];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000044FC.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB8766 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Thu, 28 Sep 2023 09:51:13 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1778314787307415275 X-GMAIL-MSGID: 1778314787307415275 The mutex is named like it is supposed to protect only the mkey but in reality it is a global lock for all mr resources. Shift the mutex to it's rightful location (struct mlx5_vdpa_dev) and give it a more appropriate name. Signed-off-by: Dragos Tatulea Acked-by: Eugenio Pérez --- drivers/vdpa/mlx5/core/mlx5_vdpa.h | 4 ++-- drivers/vdpa/mlx5/core/mr.c | 13 +++++++------ drivers/vdpa/mlx5/core/resources.c | 6 +++--- 3 files changed, 12 insertions(+), 11 deletions(-) diff --git a/drivers/vdpa/mlx5/core/mlx5_vdpa.h b/drivers/vdpa/mlx5/core/mlx5_vdpa.h index 01d4ee58ccb1..9c6ac42c21e1 100644 --- a/drivers/vdpa/mlx5/core/mlx5_vdpa.h +++ b/drivers/vdpa/mlx5/core/mlx5_vdpa.h @@ -34,8 +34,6 @@ struct mlx5_vdpa_mr { /* state of dvq mr */ bool initialized; - /* serialize mkey creation and destruction */ - struct mutex mkey_mtx; bool user_mr; }; @@ -94,6 +92,8 @@ struct mlx5_vdpa_dev { u32 generation; struct mlx5_vdpa_mr mr; + /* serialize mr access */ + struct mutex mr_mtx; struct mlx5_control_vq cvq; struct workqueue_struct *wq; unsigned int group2asid[MLX5_VDPA_NUMVQ_GROUPS]; diff --git a/drivers/vdpa/mlx5/core/mr.c b/drivers/vdpa/mlx5/core/mr.c index 6f29e8eaabb1..abd6a6fb122f 100644 --- a/drivers/vdpa/mlx5/core/mr.c +++ b/drivers/vdpa/mlx5/core/mr.c @@ -509,11 +509,11 @@ static void _mlx5_vdpa_destroy_mr(struct mlx5_vdpa_dev *mvdev, struct mlx5_vdpa_ void mlx5_vdpa_destroy_mr(struct mlx5_vdpa_dev *mvdev, struct mlx5_vdpa_mr *mr) { - mutex_lock(&mr->mkey_mtx); + mutex_lock(&mvdev->mr_mtx); _mlx5_vdpa_destroy_mr(mvdev, mr); - mutex_unlock(&mr->mkey_mtx); + mutex_unlock(&mvdev->mr_mtx); } void mlx5_vdpa_destroy_mr_resources(struct mlx5_vdpa_dev *mvdev) @@ -550,9 +550,10 @@ int mlx5_vdpa_create_mr(struct mlx5_vdpa_dev *mvdev, { int err; - mutex_lock(&mvdev->mr.mkey_mtx); + mutex_lock(&mvdev->mr_mtx); err = _mlx5_vdpa_create_mr(mvdev, mr, iotlb); - mutex_unlock(&mvdev->mr.mkey_mtx); + mutex_unlock(&mvdev->mr_mtx); + return err; } @@ -563,14 +564,14 @@ int mlx5_vdpa_handle_set_map(struct mlx5_vdpa_dev *mvdev, struct vhost_iotlb *io int err = 0; *change_map = false; - mutex_lock(&mr->mkey_mtx); + mutex_lock(&mvdev->mr_mtx); if (mr->initialized) { mlx5_vdpa_info(mvdev, "memory map update\n"); *change_map = true; } if (!*change_map) err = _mlx5_vdpa_create_mr(mvdev, mr, iotlb); - mutex_unlock(&mr->mkey_mtx); + mutex_unlock(&mvdev->mr_mtx); return err; } diff --git a/drivers/vdpa/mlx5/core/resources.c b/drivers/vdpa/mlx5/core/resources.c index d5a59c9035fb..5c5a41b64bfc 100644 --- a/drivers/vdpa/mlx5/core/resources.c +++ b/drivers/vdpa/mlx5/core/resources.c @@ -256,7 +256,7 @@ int mlx5_vdpa_alloc_resources(struct mlx5_vdpa_dev *mvdev) mlx5_vdpa_warn(mvdev, "resources already allocated\n"); return -EINVAL; } - mutex_init(&mvdev->mr.mkey_mtx); + mutex_init(&mvdev->mr_mtx); res->uar = mlx5_get_uars_page(mdev); if (IS_ERR(res->uar)) { err = PTR_ERR(res->uar); @@ -301,7 +301,7 @@ int mlx5_vdpa_alloc_resources(struct mlx5_vdpa_dev *mvdev) err_uctx: mlx5_put_uars_page(mdev, res->uar); err_uars: - mutex_destroy(&mvdev->mr.mkey_mtx); + mutex_destroy(&mvdev->mr_mtx); return err; } @@ -318,6 +318,6 @@ void mlx5_vdpa_free_resources(struct mlx5_vdpa_dev *mvdev) dealloc_pd(mvdev, res->pdn, res->uid); destroy_uctx(mvdev, res->uid); mlx5_put_uars_page(mvdev->mdev, res->uar); - mutex_destroy(&mvdev->mr.mkey_mtx); + mutex_destroy(&mvdev->mr_mtx); res->valid = false; } From patchwork Thu Sep 28 16:45:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dragos Tatulea X-Patchwork-Id: 146215 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:cae8:0:b0:403:3b70:6f57 with SMTP id r8csp3456427vqu; Thu, 28 Sep 2023 09:56:41 -0700 (PDT) X-Google-Smtp-Source: AGHT+IE6Tfmo/RnFkn+iXcfDM0PQZAmKS3hFinJxgBgSxn09vo5j1KnA5sEsLEbTmdh04BSXf02S X-Received: by 2002:a05:6808:2392:b0:3ae:16b6:6338 with SMTP id bp18-20020a056808239200b003ae16b66338mr2277137oib.3.1695920201721; Thu, 28 Sep 2023 09:56:41 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1695920201; cv=pass; d=google.com; s=arc-20160816; b=yCo0dg+Z4k+aoL6SD3+7gc22SzvvafRGwzV+jsBAKQsi7wnkfJm+uG1VeikBkrZbDj Os5R8uFMjgOdofMX4xa3ziSZ9wUFpzn/A4KhhtXViXb+pX/nkeU5LEXJDpdXjB3S+tKu jh1KShLrr6nTB5Fp2qwkPZULXoKKG6dpUUGUQq0JgodblV44kRKDuhe6l0HG3wmKU9sp t8zJuxy9jQyKN4Yc3cDiBZEXzBtsOTcXULyj7URyzXj3HeCZfyi+U7crYIiroflgvlsX ipVLQSovmihJyuJesHGHo0+ePa9dk4Fz45f2XWSSwfE/FgODxHivU5mJv3uXq8xxQoyr td4Q== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=i2mTBNgSWLtaFjwqt2Roqjc72lDAUiHhmvP2L+fzVOU=; fh=4SYmPDx0sMrYC0lSOUfhTOMEzVMfG5iPEyPqlL4Bs2M=; b=CWAzMpugHtTawIdbTiPc/m6MWzEo8WHlaVam+7nag8nqBLfqsn7EZUMUot3y+V4TrE N0qkttxqm5UwZmMw7zkQ7oidM1TuW5i0w/9K5Bwy9JaTfvS0iD1S4stb451cB98+ajVr ttwLddCNJPo5osJimRxF5g9Ufase2WjJp+XcltGCrS8GkIjJVdcbt+aT14JWxo0rjshb kL4uP8sKYTgovBahp6LrLI6qQi0FHCWldXKUJCnc+6ERw/N6lY6FRn24iViCLKntWlu+ 4Ep3OAeEMPI61JWpk1Kf7rA2CUgcB4AaxLa0FRqPsdertk4TyRXcCXvEtwMU4CGq7+2C Aeag== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=cajw7OMG; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: from pete.vger.email (pete.vger.email. [2620:137:e000::3:6]) by mx.google.com with ESMTPS id eb17-20020a056a004c9100b00690158afc78si19212151pfb.284.2023.09.28.09.56.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 28 Sep 2023 09:56:41 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) client-ip=2620:137:e000::3:6; Authentication-Results: mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=cajw7OMG; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by pete.vger.email (Postfix) with ESMTP id 2B7D6806311E; Thu, 28 Sep 2023 09:51:35 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at pete.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232237AbjI1Qun (ORCPT + 21 others); Thu, 28 Sep 2023 12:50:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58200 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231765AbjI1QuT (ORCPT ); Thu, 28 Sep 2023 12:50:19 -0400 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2040.outbound.protection.outlook.com [40.107.236.40]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4B9FC10FE for ; Thu, 28 Sep 2023 09:50:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=k/l+9K8LvaoNXa39odT++g3nvrsV22QC/feAI65h2Z6BbbMdu+vYttBu5fpcXvBAOMj8/bmuG3iUW1dlmiO4eFtQQs54lVk2SLjfDmMkXkTmVQIoPIrcTmo22CARgkN/KXg1xb2HQEOuIWkYbRUjoshcBZBl59TCskGYZCEOEuy/2eAT6HQ2WKz7VQgRNI9YJO9pF6esjB9eEKn+20GvYHSvgAK7ORj5HGoFzG9o7l7T5IyIAqxJPCAAp0VMVPYByMuGg7sfDG24F8n85m1wQtEDTay0C4zJI48TkrTN0s8GO7y/3VsHDDn+udSVeLexUEtnQkn12SpDwZqplMKNuA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=i2mTBNgSWLtaFjwqt2Roqjc72lDAUiHhmvP2L+fzVOU=; b=LGExCxPgfyrY/hZVlkXn5Xh7MfiZsrWBiJ7ZCtCCYQLSKzWYza0eZhwTlmYEasjTpawoqPd5zq0s7LeZ7qkAyCMgo6B59plQ7ePe4To+cp8ob+lyIV4OSIHf68hQb1OkMlihDU9gKhCHFzYVmmalUfhvMTeaNY4JeOrDDJ7vbDP3WwWLypOYkv1LXg3fKZzJzjrr3Vgvk4ko+4lkcgqdb5v33ZykCaM43HI/THzBgWVOjp+vby3T6d3EAUU4XcFFT2Lm9MV/FVoX0mYB22c/gi4TTrHNP9M/t5ynw9foD2zQi+VUU1HNj0oz4l7bZpgXjD8ZE7Cexe2ERlK+hNY8iQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.232) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=i2mTBNgSWLtaFjwqt2Roqjc72lDAUiHhmvP2L+fzVOU=; b=cajw7OMGyJcw8Egp8GFuBj+K45Xlrw9CMTNjxVF7wjMNvpL9V2PIScbIC8w9aGEy2inbleDNzrJ0VKpeSbAJjC/8EK+5VR+w8Kinl5YgibPRYMCgnHtuCAjiX/nSohKX7msWCDZ3YM7fuskyBs4fH9P+Q5xbPiJb5bEkttfixE7WK2F3Rp/EDWPCJHm7Ph4WDya/r2dpDx9Br3+AYTZUqIfanDaMCs4MdXKFTHnU1ArxXyXtFvjCrcwuRhyK6ztOSm5i3GjaY0l/BNZhX2wHVxjmPY322kaI9Z4O0z4SDkQ0SUb1/ylwpZgYT1oXLeu+fufaNf4zpTLkpukeQ6eD7A== Received: from CY5PR15CA0114.namprd15.prod.outlook.com (2603:10b6:930:7::28) by DM4PR12MB6566.namprd12.prod.outlook.com (2603:10b6:8:8d::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.25; Thu, 28 Sep 2023 16:50:06 +0000 Received: from CY4PEPF0000E9D1.namprd03.prod.outlook.com (2603:10b6:930:7:cafe::19) by CY5PR15CA0114.outlook.office365.com (2603:10b6:930:7::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.25 via Frontend Transport; Thu, 28 Sep 2023 16:50:06 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.232) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.232 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.232; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.232) by CY4PEPF0000E9D1.mail.protection.outlook.com (10.167.241.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.19 via Frontend Transport; Thu, 28 Sep 2023 16:50:06 +0000 Received: from drhqmail202.nvidia.com (10.126.190.181) by mail.nvidia.com (10.127.129.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Thu, 28 Sep 2023 09:50:02 -0700 Received: from drhqmail202.nvidia.com (10.126.190.181) by drhqmail202.nvidia.com (10.126.190.181) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Thu, 28 Sep 2023 09:50:01 -0700 Received: from c-237-113-220-225.mtl.labs.mlnx (10.127.8.12) by mail.nvidia.com (10.126.190.181) with Microsoft SMTP Server id 15.2.986.41 via Frontend Transport; Thu, 28 Sep 2023 09:49:59 -0700 From: Dragos Tatulea To: , , "Michael S . Tsirkin" , Jason Wang , Xuan Zhuo CC: , Dragos Tatulea , Subject: [PATCH vhost 12/16] vdpa/mlx5: Improve mr update flow Date: Thu, 28 Sep 2023 19:45:23 +0300 Message-ID: <20230928164550.980832-14-dtatulea@nvidia.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230928164550.980832-2-dtatulea@nvidia.com> References: <20230928164550.980832-2-dtatulea@nvidia.com> MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000E9D1:EE_|DM4PR12MB6566:EE_ X-MS-Office365-Filtering-Correlation-Id: b9abee80-1ca4-40df-4775-08dbc042f838 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: HNrjvrGhpT/gMATDE0GN9mANWhx5b2aIejWVHDFTm3I0Xe2FPFc0hhLd/IDNo50OusdaDiJ4VakyN4krypz50RsEAKEL+h5i0CMgaczWc6WGer5q5LWUtxDPliteFNNC1a14TSeF1XmJ6qST3uvO2AXjnqTOU4wIq4qYMnTXjQcILybs5cgKr4JUA5iSDnAMAu+gR6T7NpVRWYu0BLYzGzVwpRR4pVIg4vzSlxtMuI7MIDITS5beqcusKpZGIrSGeOgAkILapjhxZXbkrtkcihqEq7QlfPf++pDYTpQxOSX0xhRGea1IlyWYwb1Fsv0qofx9pEXkqHh1gNlTNhoReNbxrwyHsn6xPrdZHfk8q4tx+B8qaBSlukwKfLgvolUM5XCWOWXmDEEAwZLi79YcY2kA3YInJtLnA+K8A2rt6IIZv5TrnARUT7kad06V9BEg5xfYPBFQeB6GhtHUa0mpre1dabEI5nq8QVnL4CZoJjH3dE62tmA6dq2Wau9YxF96cGt6bPWGnWLtKtY98H2Q4tVNftbYEZBHPVc/EymQBSllDFJZVuBO9apOEOxjhDjMdFXUAebV54UfRl0ReoVcrHasvr0uYbXm5781nK0yBcn//01Kg0n6OWMJXXdWcKgK7XNyc5Q+UoWVp4CtDHteo1XikozE41483zjQ0BAfTwd7YD8AkYhVgzSmC+jLCVtHOEFjcr94qFIWKLKtogwuWOTkcXmDeaSFg/2tWLf33X3gx8vUuZwWekqdPitok0ZuLNBm3HR4XnpwhIoi8Ilmig== X-Forefront-Antispam-Report: CIP:216.228.118.232;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc7edge1.nvidia.com;CAT:NONE;SFS:(13230031)(4636009)(346002)(396003)(39860400002)(376002)(136003)(230922051799003)(82310400011)(1800799009)(451199024)(64100799003)(186009)(40470700004)(36840700001)(46966006)(2906002)(40460700003)(6666004)(2616005)(1076003)(478600001)(86362001)(70586007)(110136005)(54906003)(70206006)(83380400001)(36860700001)(47076005)(426003)(336012)(26005)(82740400003)(356005)(7636003)(5660300002)(36756003)(4326008)(40480700001)(316002)(41300700001)(8936002)(15650500001)(8676002)(334744004);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2023 16:50:06.6932 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b9abee80-1ca4-40df-4775-08dbc042f838 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.118.232];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000E9D1.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB6566 X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Thu, 28 Sep 2023 09:51:35 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1778301220948841033 X-GMAIL-MSGID: 1778301220948841033 The current flow for updating an mr works directly on mvdev->mr which makes it cumbersome to handle multiple new mr structs. This patch makes the flow more straightforward by having mlx5_vdpa_create_mr return a new mr which will update the old mr (if any). The old mr will be deleted and unlinked from mvdev. This change paves the way for adding mrs for different ASIDs. The initialized bool is no longer needed as mr is now a pointer in the mlx5_vdpa_dev struct which will be NULL when not initialized. Signed-off-by: Dragos Tatulea --- drivers/vdpa/mlx5/core/mlx5_vdpa.h | 14 +++-- drivers/vdpa/mlx5/core/mr.c | 87 ++++++++++++++++-------------- drivers/vdpa/mlx5/net/mlx5_vnet.c | 46 ++++++++-------- 3 files changed, 76 insertions(+), 71 deletions(-) diff --git a/drivers/vdpa/mlx5/core/mlx5_vdpa.h b/drivers/vdpa/mlx5/core/mlx5_vdpa.h index 9c6ac42c21e1..bbe4335106bd 100644 --- a/drivers/vdpa/mlx5/core/mlx5_vdpa.h +++ b/drivers/vdpa/mlx5/core/mlx5_vdpa.h @@ -31,8 +31,6 @@ struct mlx5_vdpa_mr { struct list_head head; unsigned long num_directs; unsigned long num_klms; - /* state of dvq mr */ - bool initialized; bool user_mr; }; @@ -91,7 +89,7 @@ struct mlx5_vdpa_dev { u16 max_idx; u32 generation; - struct mlx5_vdpa_mr mr; + struct mlx5_vdpa_mr *mr; /* serialize mr access */ struct mutex mr_mtx; struct mlx5_control_vq cvq; @@ -114,14 +112,14 @@ void mlx5_vdpa_free_resources(struct mlx5_vdpa_dev *mvdev); int mlx5_vdpa_create_mkey(struct mlx5_vdpa_dev *mvdev, u32 *mkey, u32 *in, int inlen); int mlx5_vdpa_destroy_mkey(struct mlx5_vdpa_dev *mvdev, u32 mkey); -int mlx5_vdpa_handle_set_map(struct mlx5_vdpa_dev *mvdev, struct vhost_iotlb *iotlb, - bool *change_map, unsigned int asid); -int mlx5_vdpa_create_mr(struct mlx5_vdpa_dev *mvdev, - struct mlx5_vdpa_mr *mr, - struct vhost_iotlb *iotlb); +struct mlx5_vdpa_mr *mlx5_vdpa_create_mr(struct mlx5_vdpa_dev *mvdev, + struct vhost_iotlb *iotlb); void mlx5_vdpa_destroy_mr_resources(struct mlx5_vdpa_dev *mvdev); void mlx5_vdpa_destroy_mr(struct mlx5_vdpa_dev *mvdev, struct mlx5_vdpa_mr *mr); +void mlx5_vdpa_update_mr(struct mlx5_vdpa_dev *mvdev, + struct mlx5_vdpa_mr *mr, + unsigned int asid); int mlx5_vdpa_update_cvq_iotlb(struct mlx5_vdpa_dev *mvdev, struct vhost_iotlb *iotlb, unsigned int asid); diff --git a/drivers/vdpa/mlx5/core/mr.c b/drivers/vdpa/mlx5/core/mr.c index abd6a6fb122f..00eff5a07152 100644 --- a/drivers/vdpa/mlx5/core/mr.c +++ b/drivers/vdpa/mlx5/core/mr.c @@ -495,30 +495,51 @@ static void destroy_user_mr(struct mlx5_vdpa_dev *mvdev, struct mlx5_vdpa_mr *mr static void _mlx5_vdpa_destroy_mr(struct mlx5_vdpa_dev *mvdev, struct mlx5_vdpa_mr *mr) { - if (!mr->initialized) - return; - if (mr->user_mr) destroy_user_mr(mvdev, mr); else destroy_dma_mr(mvdev, mr); - - mr->initialized = false; } void mlx5_vdpa_destroy_mr(struct mlx5_vdpa_dev *mvdev, struct mlx5_vdpa_mr *mr) { + if (!mr) + return; + mutex_lock(&mvdev->mr_mtx); _mlx5_vdpa_destroy_mr(mvdev, mr); + if (mvdev->mr == mr) + mvdev->mr = NULL; + + mutex_unlock(&mvdev->mr_mtx); + + kfree(mr); +} + +void mlx5_vdpa_update_mr(struct mlx5_vdpa_dev *mvdev, + struct mlx5_vdpa_mr *new_mr, + unsigned int asid) +{ + struct mlx5_vdpa_mr *old_mr = mvdev->mr; + + mutex_lock(&mvdev->mr_mtx); + + mvdev->mr = new_mr; + if (old_mr) { + _mlx5_vdpa_destroy_mr(mvdev, old_mr); + kfree(old_mr); + } + mutex_unlock(&mvdev->mr_mtx); + } void mlx5_vdpa_destroy_mr_resources(struct mlx5_vdpa_dev *mvdev) { - mlx5_vdpa_destroy_mr(mvdev, &mvdev->mr); + mlx5_vdpa_destroy_mr(mvdev, mvdev->mr); prune_iotlb(mvdev); } @@ -528,52 +549,36 @@ static int _mlx5_vdpa_create_mr(struct mlx5_vdpa_dev *mvdev, { int err; - if (mr->initialized) - return 0; - if (iotlb) err = create_user_mr(mvdev, mr, iotlb); else err = create_dma_mr(mvdev, mr); - if (err) - return err; - - mr->initialized = true; - - return 0; + return err; } -int mlx5_vdpa_create_mr(struct mlx5_vdpa_dev *mvdev, - struct mlx5_vdpa_mr *mr, - struct vhost_iotlb *iotlb) +struct mlx5_vdpa_mr *mlx5_vdpa_create_mr(struct mlx5_vdpa_dev *mvdev, + struct vhost_iotlb *iotlb) { + struct mlx5_vdpa_mr *mr; int err; + mr = kzalloc(sizeof(*mr), GFP_KERNEL); + if (!mr) + return ERR_PTR(-ENOMEM); + mutex_lock(&mvdev->mr_mtx); err = _mlx5_vdpa_create_mr(mvdev, mr, iotlb); mutex_unlock(&mvdev->mr_mtx); - return err; -} - -int mlx5_vdpa_handle_set_map(struct mlx5_vdpa_dev *mvdev, struct vhost_iotlb *iotlb, - bool *change_map, unsigned int asid) -{ - struct mlx5_vdpa_mr *mr = &mvdev->mr; - int err = 0; + if (err) + goto out_err; - *change_map = false; - mutex_lock(&mvdev->mr_mtx); - if (mr->initialized) { - mlx5_vdpa_info(mvdev, "memory map update\n"); - *change_map = true; - } - if (!*change_map) - err = _mlx5_vdpa_create_mr(mvdev, mr, iotlb); - mutex_unlock(&mvdev->mr_mtx); + return mr; - return err; +out_err: + kfree(mr); + return ERR_PTR(err); } int mlx5_vdpa_update_cvq_iotlb(struct mlx5_vdpa_dev *mvdev, @@ -597,11 +602,13 @@ int mlx5_vdpa_update_cvq_iotlb(struct mlx5_vdpa_dev *mvdev, int mlx5_vdpa_create_dma_mr(struct mlx5_vdpa_dev *mvdev) { - int err; + struct mlx5_vdpa_mr *mr; - err = mlx5_vdpa_create_mr(mvdev, &mvdev->mr, NULL); - if (err) - return err; + mr = mlx5_vdpa_create_mr(mvdev, NULL); + if (IS_ERR(mr)) + return PTR_ERR(mr); + + mlx5_vdpa_update_mr(mvdev, mr, 0); return mlx5_vdpa_update_cvq_iotlb(mvdev, NULL, 0); } diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5_vnet.c index 256fdd80c321..4a87f9119fca 100644 --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c @@ -873,7 +873,7 @@ static int create_virtqueue(struct mlx5_vdpa_net *ndev, struct mlx5_vdpa_virtque MLX5_SET64(virtio_q, vq_ctx, desc_addr, mvq->desc_addr); MLX5_SET64(virtio_q, vq_ctx, used_addr, mvq->device_addr); MLX5_SET64(virtio_q, vq_ctx, available_addr, mvq->driver_addr); - MLX5_SET(virtio_q, vq_ctx, virtio_q_mkey, ndev->mvdev.mr.mkey); + MLX5_SET(virtio_q, vq_ctx, virtio_q_mkey, ndev->mvdev.mr->mkey); MLX5_SET(virtio_q, vq_ctx, umem_1_id, mvq->umem1.id); MLX5_SET(virtio_q, vq_ctx, umem_1_size, mvq->umem1.size); MLX5_SET(virtio_q, vq_ctx, umem_2_id, mvq->umem2.id); @@ -2633,7 +2633,7 @@ static void restore_channels_info(struct mlx5_vdpa_net *ndev) } static int mlx5_vdpa_change_map(struct mlx5_vdpa_dev *mvdev, - struct vhost_iotlb *iotlb, unsigned int asid) + struct mlx5_vdpa_mr *new_mr, unsigned int asid) { struct mlx5_vdpa_net *ndev = to_mlx5_vdpa_ndev(mvdev); int err; @@ -2641,27 +2641,18 @@ static int mlx5_vdpa_change_map(struct mlx5_vdpa_dev *mvdev, suspend_vqs(ndev); err = save_channels_info(ndev); if (err) - goto err_mr; + return err; teardown_driver(ndev); - mlx5_vdpa_destroy_mr(mvdev, &mvdev->mr); - err = mlx5_vdpa_create_mr(mvdev, &mvdev->mr, iotlb); - if (err) - goto err_mr; + + mlx5_vdpa_update_mr(mvdev, new_mr, asid); if (!(mvdev->status & VIRTIO_CONFIG_S_DRIVER_OK) || mvdev->suspended) - goto err_mr; + return 0; restore_channels_info(ndev); err = setup_driver(mvdev); - if (err) - goto err_setup; - - return 0; -err_setup: - mlx5_vdpa_destroy_mr(mvdev, &mvdev->mr); -err_mr: return err; } @@ -2875,26 +2866,35 @@ static u32 mlx5_vdpa_get_generation(struct vdpa_device *vdev) static int set_map_data(struct mlx5_vdpa_dev *mvdev, struct vhost_iotlb *iotlb, unsigned int asid) { - bool change_map; + struct mlx5_vdpa_mr *new_mr; int err; if (mvdev->group2asid[MLX5_VDPA_DATAVQ_GROUP] != asid) goto end; - err = mlx5_vdpa_handle_set_map(mvdev, iotlb, &change_map, asid); - if (err) { - mlx5_vdpa_warn(mvdev, "set map failed(%d)\n", err); + new_mr = mlx5_vdpa_create_mr(mvdev, iotlb); + if (IS_ERR(new_mr)) { + err = PTR_ERR(new_mr); + mlx5_vdpa_warn(mvdev, "create map failed(%d)\n", err); return err; } - if (change_map) { - err = mlx5_vdpa_change_map(mvdev, iotlb, asid); - if (err) - return err; + if (!mvdev->mr) { + mlx5_vdpa_update_mr(mvdev, new_mr, asid); + } else { + err = mlx5_vdpa_change_map(mvdev, new_mr, asid); + if (err) { + mlx5_vdpa_warn(mvdev, "change map failed(%d)\n", err); + goto out_err; + } } end: return mlx5_vdpa_update_cvq_iotlb(mvdev, iotlb, asid); + +out_err: + mlx5_vdpa_destroy_mr(mvdev, new_mr); + return err; } static int mlx5_vdpa_set_map(struct vdpa_device *vdev, unsigned int asid, From patchwork Thu Sep 28 16:45:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Dragos Tatulea X-Patchwork-Id: 146389 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:cae8:0:b0:403:3b70:6f57 with SMTP id r8csp3718388vqu; Thu, 28 Sep 2023 18:41:23 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGBwiQ9zQvK8wrx+g5k84yPAcocyaSRXiHIc6e7Cml2Z+hDiyz4mV+5LlZYXe2zVKDsLvLS X-Received: by 2002:a05:6a00:cce:b0:68f:f0bb:7de1 with SMTP id b14-20020a056a000cce00b0068ff0bb7de1mr3198394pfv.11.1695951683147; Thu, 28 Sep 2023 18:41:23 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1695951683; cv=pass; d=google.com; s=arc-20160816; b=ijFgVph+kAY4CbKumP+KxdC8Yp2Szr7j+x5Ylk2N4JtbyBgQj1BHv8oJYTXArJ+0hU qp+U5zBaCIbBIAk9IlajdSkdnwpqc3WjfZkasdhDASVQ+FVM6Prdlu+gTSdOl+M1QIbB cgNVyF4oaauyG8Pat/OE60Em4F/H/qFbdKHkkTxEcAFqEaDt8AW610AxTIt0w56EBAFQ DzAEiIUdTode0cKL4OwsX9YubuOXChiaDgMmwZP4BpB8zVCLYaC5qYA2FaTuA/DMpPdR RxsJeAjM6dzG3n8yT2/3hw6j1EBJSVKFlV23u1f+y9LW+qbmjHOoBmKBF4VLa8NXGBxH iNDg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=VxT2lbBZR73KweaHMkpPsjB3qleSM5PAGG5aYNlriAY=; fh=4SYmPDx0sMrYC0lSOUfhTOMEzVMfG5iPEyPqlL4Bs2M=; b=DYweClZYQZBr7YohBlYXB67OKdfKgVvZe3Q5q5LEpT3OlLtm3g+knPOdj4DeUL04U+ q5UF4bLAYDMtGwmSGKdxq85pWTrXmyGN4NViAgM4DzVNxnWR6vlHbuX474PdpmoAH2uW q5rBHfQbsGLV7ZMaGOopOjhvVtQf6BCoutE7VD65bVS1wsjh0J6a5P8TE+BfSPYtyElC n7m4ZkoQYRHRtGQR23YEGcFieELpiCjx2E4wVcd8LnMgXvXJcy2kGVI5fb2EgzK5TkgX WS9rmbI8yFFloputvKbiC/TJQm90+02+AibJKBRRp5QnM7N4iugExiz4kIkXEHixv9qj R9rA== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=ihvYW9iv; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: from groat.vger.email (groat.vger.email. [2620:137:e000::3:5]) by mx.google.com with ESMTPS id cq12-20020a056a00330c00b0068a54cfcd74si19624226pfb.192.2023.09.28.18.41.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 28 Sep 2023 18:41:23 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) client-ip=2620:137:e000::3:5; Authentication-Results: mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=ihvYW9iv; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by groat.vger.email (Postfix) with ESMTP id 5150E80BB558; Thu, 28 Sep 2023 09:52:02 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at groat.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231377AbjI1Quz (ORCPT + 21 others); Thu, 28 Sep 2023 12:50:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34596 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232120AbjI1Qud (ORCPT ); Thu, 28 Sep 2023 12:50:33 -0400 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2070.outbound.protection.outlook.com [40.107.220.70]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EEC6110D9 for ; Thu, 28 Sep 2023 09:50:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Xw42mUSJNNgVYQ/bi01EsfU7A8j4TKqbyiuEnlYHnYTlu0EZ3duGNA9QJiLlISEixziF+2VtJzt5GRjBxq2+Y157WqkxiOWSZm75HwgJjgYydRy3fBZOgTKymgcjAfAEjLZL/DxZMlfGP4StWwAMAtgzuKPCFBH4nehHfiKTjJ4FkZW76TNOm2LDKPKzzGoQjWj+h+KC2JsGiafGzwyyoduz/ReptUXnuIzMe90qCUmA1qNuVYHguDilCZnekCCvDx8Z/RLbcgigeM5FZEl+nyqVeVX5WbfxySlZP26BS4MUNNd9SC8uHp/S1zmUFJQukAVFlqi51L5Q3WslhP2Inw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=VxT2lbBZR73KweaHMkpPsjB3qleSM5PAGG5aYNlriAY=; b=DaMlR5yyhw3WXMtjejVAm9C2cx5ftyA5GoAjERKze4Ju+7Nm0Z7vF6Y2sjhKUHrndykTGHCe+dXHTSuW8jRNsHAnaRI2L88fnDPpzWjlR1uZfLH1OLGkNxgcqOX7XbxGctKpwgawpe9sU5qqleAn3mk8kk/ncNpW42kFiEdV/sY5ZoErnx0F5t+z+fuoziA59aLPsMui42VTwKi5RCArh7IClX5bM1lxtEyBYb0b2jryC5BQSdeQZKERgiPYdnIv3B7UbGbz0IoxkhhuXIpl4eXYdAdTvLj2V1XHWVHm3kIehV/U3Jw5WUy1EQ6+ItC2GUGOwMgQTQ98bOwsn7At8w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.233) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=VxT2lbBZR73KweaHMkpPsjB3qleSM5PAGG5aYNlriAY=; b=ihvYW9iv+2FhQqNyOPkQCjkWkX/cg4e1CnLoUw/CDwH1ykldy5dhE13t7De46Wa+Uic6Qhdtm3e+te9EDvhnJq8tmncG2fHTydyx5EoaOTB0jwYkvK3htsltJxvXkfdUsRpiMQr2xoIsxjP+YLdT7Ltg87LirzzZxqNB8b36Kz7HHcnOo7vR2MU3ovKMrJ1MlwqVZ+UgtMfunduTtGHsQyxToTA7Dtb8dOR/MjsMNMiaNNlKKytLbCJUW4JRxlaScVYpMv5+xuNF6EM08xZ+SWy7YsktewNwHCdhPlWVyiyRgyOM66pUJ9+/eBmZMKLT0CyXMrOuHCzDHVPG/2mEzA== Received: from MW4P223CA0019.NAMP223.PROD.OUTLOOK.COM (2603:10b6:303:80::24) by IA1PR12MB8287.namprd12.prod.outlook.com (2603:10b6:208:3f5::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6813.35; Thu, 28 Sep 2023 16:50:14 +0000 Received: from CO1PEPF000044FC.namprd21.prod.outlook.com (2603:10b6:303:80:cafe::d1) by MW4P223CA0019.outlook.office365.com (2603:10b6:303:80::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.25 via Frontend Transport; Thu, 28 Sep 2023 16:50:14 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.233) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.233 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.233; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.233) by CO1PEPF000044FC.mail.protection.outlook.com (10.167.241.202) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6863.9 via Frontend Transport; Thu, 28 Sep 2023 16:50:14 +0000 Received: from drhqmail202.nvidia.com (10.126.190.181) by mail.nvidia.com (10.127.129.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Thu, 28 Sep 2023 09:50:05 -0700 Received: from drhqmail202.nvidia.com (10.126.190.181) by drhqmail202.nvidia.com (10.126.190.181) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Thu, 28 Sep 2023 09:50:04 -0700 Received: from c-237-113-220-225.mtl.labs.mlnx (10.127.8.12) by mail.nvidia.com (10.126.190.181) with Microsoft SMTP Server id 15.2.986.41 via Frontend Transport; Thu, 28 Sep 2023 09:50:02 -0700 From: Dragos Tatulea To: , , "Michael S . Tsirkin" , Jason Wang , Xuan Zhuo CC: , Dragos Tatulea , Subject: [PATCH vhost 13/16] vdpa/mlx5: Introduce mr for vq descriptor Date: Thu, 28 Sep 2023 19:45:24 +0300 Message-ID: <20230928164550.980832-15-dtatulea@nvidia.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230928164550.980832-2-dtatulea@nvidia.com> References: <20230928164550.980832-2-dtatulea@nvidia.com> MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000044FC:EE_|IA1PR12MB8287:EE_ X-MS-Office365-Filtering-Correlation-Id: 4ecf0c06-e719-424a-38b0-08dbc042fce1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: UGfB6kt+n/NBuZZrqQxjQTV1vO90NXoCMEG8REVsg04Pwa23K3yV3SgJ3LfpNBKgTCFauyphR7QIuUnINjWo8YDc67lKTGkny9czrOAKvKEajQ877o808vRIDRcj/4lOiQfITbeG4hajaLi6oicSnI/hK9r1C+6LHELS1ysNo/CdCtANFodqq6UW2nQRGEO99u5yJN4WrOGHS59UOjmOmusY81eF524jwSOsGheKtHRoOz9QFywOdE91ts3EOy0zu3Ei2Heb9J4blc1+7xKIzQFWqgHT3dulAWz/7ir5CORfR+bzO4pcrmsDBzgHcZDkb/LFBPbKzhDFRO/WiQe9WqaU4wOZnNlo9W8+yEyvK6/4UvotTrn9QKo+X0hrgzOZbXRPUK4sRoPakNmsBLeRhRUMmho0Nxca/jBZjgxVfOwS+HH6HUSCwZfoSKF5h2xrBsmaqx1LTn5kt0eZ4KI86ksBEGrh4kPOW+Y+ozUXjbf2Qf2Ej7oE4R8ugxpBCMRot615gDiSVwZrZLCo9qhKnjmQBmccKaKipwTPIY9PdjeXP8avDt/DIpwdmNATOl9OQvVTLd2OzDqrDtJ/lOBV7B0W3S/07TyV8JocNGUVkE0RqW2q27rYluazYVyUvHJLn0X7hIP2ara5l/6vCgqqVdfGE7vO9WsOjzZUIMk5y7QPRQe59l1c6U+yzieuMhiKTk3UBpUVjw72rFM2g5KWxMowCjQ4Zej/qQkxao4nYAY= X-Forefront-Antispam-Report: CIP:216.228.118.233;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc7edge2.nvidia.com;CAT:NONE;SFS:(13230031)(4636009)(136003)(396003)(39860400002)(376002)(346002)(230922051799003)(186009)(82310400011)(1800799009)(64100799003)(451199024)(36840700001)(46966006)(40470700004)(6666004)(478600001)(426003)(336012)(83380400001)(2616005)(26005)(1076003)(2906002)(41300700001)(8936002)(5660300002)(70586007)(316002)(70206006)(54906003)(4326008)(8676002)(110136005)(36756003)(36860700001)(47076005)(356005)(7636003)(86362001)(82740400003)(40480700001)(40460700003);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2023 16:50:14.5269 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 4ecf0c06-e719-424a-38b0-08dbc042fce1 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.118.233];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000044FC.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB8287 X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Thu, 28 Sep 2023 09:52:02 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1778334232453126257 X-GMAIL-MSGID: 1778334232453126257 Introduce the vq descriptor group and ASID 1. Until now .set_map on ASID 1 was only updating the cvq iotlb. From now on it also creates a mkey for it. The current patch doesn't use it but follow-up patches will add hardware support for mapping the vq descriptors. Signed-off-by: Dragos Tatulea Acked-by: Eugenio Pérez --- drivers/vdpa/mlx5/core/mlx5_vdpa.h | 5 +++-- drivers/vdpa/mlx5/core/mr.c | 14 +++++++++----- drivers/vdpa/mlx5/net/mlx5_vnet.c | 20 +++++++++++++------- 3 files changed, 25 insertions(+), 14 deletions(-) diff --git a/drivers/vdpa/mlx5/core/mlx5_vdpa.h b/drivers/vdpa/mlx5/core/mlx5_vdpa.h index bbe4335106bd..ae09296f4270 100644 --- a/drivers/vdpa/mlx5/core/mlx5_vdpa.h +++ b/drivers/vdpa/mlx5/core/mlx5_vdpa.h @@ -70,11 +70,12 @@ struct mlx5_vdpa_wq_ent { enum { MLX5_VDPA_DATAVQ_GROUP, MLX5_VDPA_CVQ_GROUP, + MLX5_VDPA_DATAVQ_DESC_GROUP, MLX5_VDPA_NUMVQ_GROUPS }; enum { - MLX5_VDPA_NUM_AS = MLX5_VDPA_NUMVQ_GROUPS + MLX5_VDPA_NUM_AS = 2 }; struct mlx5_vdpa_dev { @@ -89,7 +90,7 @@ struct mlx5_vdpa_dev { u16 max_idx; u32 generation; - struct mlx5_vdpa_mr *mr; + struct mlx5_vdpa_mr *mr[MLX5_VDPA_NUM_AS]; /* serialize mr access */ struct mutex mr_mtx; struct mlx5_control_vq cvq; diff --git a/drivers/vdpa/mlx5/core/mr.c b/drivers/vdpa/mlx5/core/mr.c index 00eff5a07152..3dee6d9bed6b 100644 --- a/drivers/vdpa/mlx5/core/mr.c +++ b/drivers/vdpa/mlx5/core/mr.c @@ -511,8 +511,10 @@ void mlx5_vdpa_destroy_mr(struct mlx5_vdpa_dev *mvdev, _mlx5_vdpa_destroy_mr(mvdev, mr); - if (mvdev->mr == mr) - mvdev->mr = NULL; + for (int i = 0; i < MLX5_VDPA_NUM_AS; i++) { + if (mvdev->mr[i] == mr) + mvdev->mr[i] = NULL; + } mutex_unlock(&mvdev->mr_mtx); @@ -523,11 +525,11 @@ void mlx5_vdpa_update_mr(struct mlx5_vdpa_dev *mvdev, struct mlx5_vdpa_mr *new_mr, unsigned int asid) { - struct mlx5_vdpa_mr *old_mr = mvdev->mr; + struct mlx5_vdpa_mr *old_mr = mvdev->mr[asid]; mutex_lock(&mvdev->mr_mtx); - mvdev->mr = new_mr; + mvdev->mr[asid] = new_mr; if (old_mr) { _mlx5_vdpa_destroy_mr(mvdev, old_mr); kfree(old_mr); @@ -539,7 +541,9 @@ void mlx5_vdpa_update_mr(struct mlx5_vdpa_dev *mvdev, void mlx5_vdpa_destroy_mr_resources(struct mlx5_vdpa_dev *mvdev) { - mlx5_vdpa_destroy_mr(mvdev, mvdev->mr); + for (int i = 0; i < MLX5_VDPA_NUM_AS; i++) + mlx5_vdpa_destroy_mr(mvdev, mvdev->mr[i]); + prune_iotlb(mvdev); } diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5_vnet.c index 4a87f9119fca..25bd2c324f5b 100644 --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c @@ -821,6 +821,8 @@ static int create_virtqueue(struct mlx5_vdpa_net *ndev, struct mlx5_vdpa_virtque { int inlen = MLX5_ST_SZ_BYTES(create_virtio_net_q_in); u32 out[MLX5_ST_SZ_DW(create_virtio_net_q_out)] = {}; + struct mlx5_vdpa_dev *mvdev = &ndev->mvdev; + struct mlx5_vdpa_mr *vq_mr; void *obj_context; u16 mlx_features; void *cmd_hdr; @@ -873,7 +875,9 @@ static int create_virtqueue(struct mlx5_vdpa_net *ndev, struct mlx5_vdpa_virtque MLX5_SET64(virtio_q, vq_ctx, desc_addr, mvq->desc_addr); MLX5_SET64(virtio_q, vq_ctx, used_addr, mvq->device_addr); MLX5_SET64(virtio_q, vq_ctx, available_addr, mvq->driver_addr); - MLX5_SET(virtio_q, vq_ctx, virtio_q_mkey, ndev->mvdev.mr->mkey); + vq_mr = mvdev->mr[mvdev->group2asid[MLX5_VDPA_DATAVQ_GROUP]]; + if (vq_mr) + MLX5_SET(virtio_q, vq_ctx, virtio_q_mkey, vq_mr->mkey); MLX5_SET(virtio_q, vq_ctx, umem_1_id, mvq->umem1.id); MLX5_SET(virtio_q, vq_ctx, umem_1_size, mvq->umem1.size); MLX5_SET(virtio_q, vq_ctx, umem_2_id, mvq->umem2.id); @@ -2633,7 +2637,8 @@ static void restore_channels_info(struct mlx5_vdpa_net *ndev) } static int mlx5_vdpa_change_map(struct mlx5_vdpa_dev *mvdev, - struct mlx5_vdpa_mr *new_mr, unsigned int asid) + struct mlx5_vdpa_mr *new_mr, + unsigned int asid) { struct mlx5_vdpa_net *ndev = to_mlx5_vdpa_ndev(mvdev); int err; @@ -2652,8 +2657,10 @@ static int mlx5_vdpa_change_map(struct mlx5_vdpa_dev *mvdev, restore_channels_info(ndev); err = setup_driver(mvdev); + if (err) + return err; - return err; + return 0; } /* reslock must be held for this function */ @@ -2869,8 +2876,8 @@ static int set_map_data(struct mlx5_vdpa_dev *mvdev, struct vhost_iotlb *iotlb, struct mlx5_vdpa_mr *new_mr; int err; - if (mvdev->group2asid[MLX5_VDPA_DATAVQ_GROUP] != asid) - goto end; + if (asid >= MLX5_VDPA_NUM_AS) + return -EINVAL; new_mr = mlx5_vdpa_create_mr(mvdev, iotlb); if (IS_ERR(new_mr)) { @@ -2879,7 +2886,7 @@ static int set_map_data(struct mlx5_vdpa_dev *mvdev, struct vhost_iotlb *iotlb, return err; } - if (!mvdev->mr) { + if (!mvdev->mr[asid]) { mlx5_vdpa_update_mr(mvdev, new_mr, asid); } else { err = mlx5_vdpa_change_map(mvdev, new_mr, asid); @@ -2889,7 +2896,6 @@ static int set_map_data(struct mlx5_vdpa_dev *mvdev, struct vhost_iotlb *iotlb, } } -end: return mlx5_vdpa_update_cvq_iotlb(mvdev, iotlb, asid); out_err: From patchwork Thu Sep 28 16:45:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Dragos Tatulea X-Patchwork-Id: 146297 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:cae8:0:b0:403:3b70:6f57 with SMTP id r8csp3591591vqu; Thu, 28 Sep 2023 13:52:14 -0700 (PDT) X-Google-Smtp-Source: AGHT+IH9E0dvnOAFGFFccBlw1LfBSDGILEUNTheYML+wzOzTS4J+u2t9dnvoQN3wXLuDnYwQKJ+1 X-Received: by 2002:a05:6a21:a58c:b0:135:1af6:9a01 with SMTP id gd12-20020a056a21a58c00b001351af69a01mr3007721pzc.8.1695934334286; Thu, 28 Sep 2023 13:52:14 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1695934334; cv=pass; d=google.com; s=arc-20160816; b=bPl7rO89p/LshyiKHX6/rEppF4vkzl/YL91W373yZKhSBIe7aCKgqABUmCKw9vaFU0 HDnPKuGn7blwUj0wQHlD288t4l9WuqL4dYViY8zsRFYRtnf+Pie1f0/J2bqdhT5sxzAN pWTk+Mmbe847iMGWSXGC/PL8a2LVZCfSKi1DJQCgJUSIygqez1919YYmPhd4l/g65TB9 DOxTMP1rDKR9DMCMe3fawDwOkPN3FmAEsulZ07Bz1f7Oak+RrAE7HD08Hmj8ZKlM1q/s 1JGMh2/JjopPde5tgYJ2Bgzdew2IAKxnFDYqaMHdwE3wgsErfsGbRzIGDT6o/X4IyhV3 70zA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=T7XjABeOssKQEWLk+sMbqrIPh+wZaqt2PP7FPEk1w5k=; fh=BiTOBIKuYKWXCTG6yQc65Fd9Ws4doRwAwAOh7UE9fe4=; b=aU9zS8MPSAzuVpBB4ZCqeaDMJtfixO6gQ9AzL4JaBBmiZLrAPmv01ZQM1Y91s1lTXK k3kHVATT2JAg7x4sE9C+Obre3H69m9ydWvhQPjziW8Gy29Lqxp7rlJ2eWoXr2sA08kjE KGadKL8anGWIOj+xsYK0z9PHQg3YWAbnt4rrL3/3PanvzpjvKP2dyOZ4EV3CUPVFfdXk 6X93OedjS+temYbkHDNbQskSYhUNZUM1O0v5u/NGg2B2fwwA0QGhML0Fhm4iKQeYCFYM IASgMgUy1pCeTtEAOJkf+88RLkFQrPq+4JKjV02NcSdgp+4e72Pq2uhzrwi7LDLBHdF/ LwRw== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=G0wJWeiB; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: from howler.vger.email (howler.vger.email. [2620:137:e000::3:4]) by mx.google.com with ESMTPS id o23-20020a17090ac71700b002777081181fsi9784131pjt.51.2023.09.28.13.52.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 28 Sep 2023 13:52:14 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) client-ip=2620:137:e000::3:4; Authentication-Results: mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=G0wJWeiB; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id EBB5C822DA06; Thu, 28 Sep 2023 09:51:20 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231689AbjI1QvC (ORCPT + 21 others); Thu, 28 Sep 2023 12:51:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39578 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232203AbjI1Quk (ORCPT ); Thu, 28 Sep 2023 12:50:40 -0400 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2066.outbound.protection.outlook.com [40.107.92.66]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0DF9E1B5; Thu, 28 Sep 2023 09:50:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LRq5k5CBUYYtCUyl8jazo45SZfs2jRFLaOnqnJB2kCEp5/yXFDM9ek9wT5SMv3OjtcQ+eAoF46DPblGazyrF9FUS2a3NsXnYf5UbMevRJbRYT1SKsPAZYY/06SaBCZUBnaIboqF6mCdG7UluW/jR3bi6zv6T5ZLB04QR1rS43Rzp+/DBhBMMSgYQx6HvbTZCNkNbzlLM/tUep0S6qcxmKOXmCSYFVzrp1F29vfWQqnhn0VVDgn6E+glFhKHojx/gyZYwxQAWebCEzhyoEiFZS+YVYVeQC3uevpI3H920DwSFtajLUuTJiHE1LP5uVp1X9oZelGSlgWsPG9okfCvJBQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=T7XjABeOssKQEWLk+sMbqrIPh+wZaqt2PP7FPEk1w5k=; b=OKTjMX5MqF5uC7EVDFt2zferi2ZuImmxb8kYEglC+veFaAf0AhMWJ/cVh7Ys6KG/jhbhL4EtR4cuFYGd8hf1H9+U3YThocFXZ4yTQsrrrLAbkZ1HLrfer6LOfxTGI0dbEaQq0uJWapPUF9H+6O3cA2NTIoPOekx+NXSf6rNjXuWt4uZvYEE3fK5tsdha+jm9FEoV+HSwukykfE/JTowW39Bh8RBJcrv3RNSH09hE5jJ/4uXrBJt9FAobgqWo3fSCImJcznm7dagnUJo0MtLa0gqreTH9SQb+XOSFxWNC/iOegcI18BOCjyh3vUTd6ZuZCu6xh+l4fkpoZ/GjdFZ9Qw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.233) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=T7XjABeOssKQEWLk+sMbqrIPh+wZaqt2PP7FPEk1w5k=; b=G0wJWeiBIREas4vW7f/0t739y9vgwYUGJgU8DysCXhDBY3R4EqdsNpDgDm1nBS7m+wpu78joVY0xykDXpZ4wfTALLRECJFr7dhDMnjtJg5xoYq1Ocu4DwjbJbkZVPLjPY+dav83Lp33ahSsAKhVX2kO9luKWc/lOv7BFfxNbFlWsE6G7ZUkdzI11YZZpBzT3ZAhhqDE6LVn0bWd8AIcvz3ir1bV6JNm2o6HAvVWPrvSaYj4MWMR4/p+IE7P0U+oAP2hza4AUeN6Ta3lCOcXS7x9tyuYcCyaclXStgPxyX9ZkNgyztMsedkhQ5pQuOKdTZ11JHsIW9QxprsiSEhQtXA== Received: from MW4PR03CA0047.namprd03.prod.outlook.com (2603:10b6:303:8e::22) by CH2PR12MB4279.namprd12.prod.outlook.com (2603:10b6:610:af::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.25; Thu, 28 Sep 2023 16:50:18 +0000 Received: from CO1PEPF000044FA.namprd21.prod.outlook.com (2603:10b6:303:8e:cafe::7e) by MW4PR03CA0047.outlook.office365.com (2603:10b6:303:8e::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.22 via Frontend Transport; Thu, 28 Sep 2023 16:50:18 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.233) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.233 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.233; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.233) by CO1PEPF000044FA.mail.protection.outlook.com (10.167.241.200) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6863.9 via Frontend Transport; Thu, 28 Sep 2023 16:50:18 +0000 Received: from drhqmail201.nvidia.com (10.126.190.180) by mail.nvidia.com (10.127.129.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Thu, 28 Sep 2023 09:50:08 -0700 Received: from drhqmail202.nvidia.com (10.126.190.181) by drhqmail201.nvidia.com (10.126.190.180) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Thu, 28 Sep 2023 09:50:08 -0700 Received: from c-237-113-220-225.mtl.labs.mlnx (10.127.8.12) by mail.nvidia.com (10.126.190.181) with Microsoft SMTP Server id 15.2.986.41 via Frontend Transport; Thu, 28 Sep 2023 09:50:05 -0700 From: Dragos Tatulea To: , , "Michael S . Tsirkin" , Jason Wang , Xuan Zhuo , Leon Romanovsky , "Saeed Mahameed" CC: , Dragos Tatulea , , , Subject: [PATCH vhost 14/16] vdpa/mlx5: Enable hw support for vq descriptor mapping Date: Thu, 28 Sep 2023 19:45:25 +0300 Message-ID: <20230928164550.980832-16-dtatulea@nvidia.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230928164550.980832-2-dtatulea@nvidia.com> References: <20230928164550.980832-2-dtatulea@nvidia.com> MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000044FA:EE_|CH2PR12MB4279:EE_ X-MS-Office365-Filtering-Correlation-Id: 87dc1fef-c389-46c2-ca05-08dbc042ff11 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Yhotgiu8Gcj++CnZr5z6D/r06CpAS7Hm4nyOHG+FODr0/RHwZhkQ8BH8L+aqJoYQcCqmfxr7RJ+0e/nkHt5P30avZkAlmxUZs8UhzEoojnCLL0hmVf61KsnIk7Tl6m7a9n+bzF7YOsWzsMCiA6ZChhtgGBX3Fbe8NO5kqT2Yy28uaGDQVUDxij11ZUsqy7D8QzmcrcZ0cWAvrlYpiTIGtOLZkLsGSrdhDmLRQ5m6DLsoVgFd9aCawJv+RjvlVxG5pj/XCc1gDsXGJpEd4lJhPRtkgKw6aI7smMHA4XUYP4icCjyf48efhHmHoM5Qf4y40Stm0vp9+0EGwC0EJhY6K1G1LvV9ij+EVkerZsxklNIjPZqoRdHHTY0SMzOtSxyL8MSg1+u0HcaV2PG5OXyE2GY61rY/yQ5QdXaacY/P2L8nt/oHKRtnnvHsO4M0TJhiG9yIywzEA4094ltPjzrPJ3OFpQVHntbOJzgDMyNMbMo5YdPTTsaUHMzHcUPY2Nv+wgVPcdQRBiw9myhFfF/dZIVp0nLuMSbZssFJ33Ea7I84dbtneAcIp0YtpdJHWJRxAilPRDm1fGPXUkQ9W/ECZISsZ7vjb5MbBjUG1MNacZcNQZzcFxdKlxIbxSTh8hhpLGSfuAyXAm6nGEN2dTx19mqpBIS1Qt27wBohn93X7uyFfe17GCmGACiZmdHiUpqyqMnBrEBzmfWfQKkQsfPpVRs3U5ieUfmIyPCfofO0qJ6MWnW9hE1iZwvQwk11nXDV X-Forefront-Antispam-Report: CIP:216.228.118.233;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc7edge2.nvidia.com;CAT:NONE;SFS:(13230031)(4636009)(39860400002)(396003)(346002)(136003)(376002)(230922051799003)(451199024)(64100799003)(1800799009)(82310400011)(186009)(40470700004)(46966006)(36840700001)(36860700001)(2906002)(336012)(426003)(36756003)(83380400001)(47076005)(86362001)(82740400003)(26005)(2616005)(1076003)(356005)(7636003)(6666004)(40460700003)(478600001)(5660300002)(6636002)(316002)(8676002)(8936002)(4326008)(40480700001)(54906003)(70206006)(110136005)(70586007)(41300700001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2023 16:50:18.1850 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 87dc1fef-c389-46c2-ca05-08dbc042ff11 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.118.233];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000044FA.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR12MB4279 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Thu, 28 Sep 2023 09:51:21 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1778316040167422064 X-GMAIL-MSGID: 1778316040167422064 Vq descriptor mappings are supported in hardware by filling in an additional mkey which contains the descriptor mappings to the hw vq. A previous patch in this series added support for hw mkey (mr) creation for ASID 1. This patch fills in both the vq data and vq descriptor mkeys based on group ASID mapping. The feature is signaled to the vdpa core through the presence of the .get_vq_desc_group op. Signed-off-by: Dragos Tatulea Acked-by: Eugenio Pérez --- drivers/vdpa/mlx5/net/mlx5_vnet.c | 26 ++++++++++++++++++++++++-- include/linux/mlx5/mlx5_ifc_vdpa.h | 7 ++++++- 2 files changed, 30 insertions(+), 3 deletions(-) diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5_vnet.c index 25bd2c324f5b..46441e41892c 100644 --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c @@ -823,6 +823,7 @@ static int create_virtqueue(struct mlx5_vdpa_net *ndev, struct mlx5_vdpa_virtque u32 out[MLX5_ST_SZ_DW(create_virtio_net_q_out)] = {}; struct mlx5_vdpa_dev *mvdev = &ndev->mvdev; struct mlx5_vdpa_mr *vq_mr; + struct mlx5_vdpa_mr *vq_desc_mr; void *obj_context; u16 mlx_features; void *cmd_hdr; @@ -878,6 +879,11 @@ static int create_virtqueue(struct mlx5_vdpa_net *ndev, struct mlx5_vdpa_virtque vq_mr = mvdev->mr[mvdev->group2asid[MLX5_VDPA_DATAVQ_GROUP]]; if (vq_mr) MLX5_SET(virtio_q, vq_ctx, virtio_q_mkey, vq_mr->mkey); + + vq_desc_mr = mvdev->mr[mvdev->group2asid[MLX5_VDPA_DATAVQ_DESC_GROUP]]; + if (vq_desc_mr) + MLX5_SET(virtio_q, vq_ctx, desc_group_mkey, vq_desc_mr->mkey); + MLX5_SET(virtio_q, vq_ctx, umem_1_id, mvq->umem1.id); MLX5_SET(virtio_q, vq_ctx, umem_1_size, mvq->umem1.size); MLX5_SET(virtio_q, vq_ctx, umem_2_id, mvq->umem2.id); @@ -2265,6 +2271,16 @@ static u32 mlx5_vdpa_get_vq_group(struct vdpa_device *vdev, u16 idx) return MLX5_VDPA_DATAVQ_GROUP; } +static u32 mlx5_vdpa_get_vq_desc_group(struct vdpa_device *vdev, u16 idx) +{ + struct mlx5_vdpa_dev *mvdev = to_mvdev(vdev); + + if (is_ctrl_vq_idx(mvdev, idx)) + return MLX5_VDPA_CVQ_GROUP; + + return MLX5_VDPA_DATAVQ_DESC_GROUP; +} + static u64 mlx_to_vritio_features(u16 dev_features) { u64 result = 0; @@ -3139,7 +3155,7 @@ static int mlx5_set_group_asid(struct vdpa_device *vdev, u32 group, { struct mlx5_vdpa_dev *mvdev = to_mvdev(vdev); - if (group >= MLX5_VDPA_NUMVQ_GROUPS) + if (group >= MLX5_VDPA_NUMVQ_GROUPS || asid >= MLX5_VDPA_NUM_AS) return -EINVAL; mvdev->group2asid[group] = asid; @@ -3160,6 +3176,7 @@ static const struct vdpa_config_ops mlx5_vdpa_ops = { .get_vq_irq = mlx5_get_vq_irq, .get_vq_align = mlx5_vdpa_get_vq_align, .get_vq_group = mlx5_vdpa_get_vq_group, + .get_vq_desc_group = mlx5_vdpa_get_vq_desc_group, /* Op disabled if not supported. */ .get_device_features = mlx5_vdpa_get_device_features, .set_driver_features = mlx5_vdpa_set_driver_features, .get_driver_features = mlx5_vdpa_get_driver_features, @@ -3258,6 +3275,7 @@ struct mlx5_vdpa_mgmtdev { struct vdpa_mgmt_dev mgtdev; struct mlx5_adev *madev; struct mlx5_vdpa_net *ndev; + struct vdpa_config_ops vdpa_ops; }; static int config_func_mtu(struct mlx5_core_dev *mdev, u16 mtu) @@ -3371,7 +3389,7 @@ static int mlx5_vdpa_dev_add(struct vdpa_mgmt_dev *v_mdev, const char *name, max_vqs = 2; } - ndev = vdpa_alloc_device(struct mlx5_vdpa_net, mvdev.vdev, mdev->device, &mlx5_vdpa_ops, + ndev = vdpa_alloc_device(struct mlx5_vdpa_net, mvdev.vdev, mdev->device, &mgtdev->vdpa_ops, MLX5_VDPA_NUMVQ_GROUPS, MLX5_VDPA_NUM_AS, name, false); if (IS_ERR(ndev)) return PTR_ERR(ndev); @@ -3546,6 +3564,10 @@ static int mlx5v_probe(struct auxiliary_device *adev, MLX5_CAP_DEV_VDPA_EMULATION(mdev, max_num_virtio_queues) + 1; mgtdev->mgtdev.supported_features = get_supported_features(mdev); mgtdev->madev = madev; + mgtdev->vdpa_ops = mlx5_vdpa_ops; + + if (!MLX5_CAP_DEV_VDPA_EMULATION(mdev, desc_group_mkey_supported)) + mgtdev->vdpa_ops.get_vq_desc_group = NULL; err = vdpa_mgmtdev_register(&mgtdev->mgtdev); if (err) diff --git a/include/linux/mlx5/mlx5_ifc_vdpa.h b/include/linux/mlx5/mlx5_ifc_vdpa.h index 9becdc3fa503..b86d51a855f6 100644 --- a/include/linux/mlx5/mlx5_ifc_vdpa.h +++ b/include/linux/mlx5/mlx5_ifc_vdpa.h @@ -74,7 +74,11 @@ struct mlx5_ifc_virtio_q_bits { u8 reserved_at_320[0x8]; u8 pd[0x18]; - u8 reserved_at_340[0xc0]; + u8 reserved_at_340[0x20]; + + u8 desc_group_mkey[0x20]; + + u8 reserved_at_380[0x80]; }; struct mlx5_ifc_virtio_net_q_object_bits { @@ -141,6 +145,7 @@ enum { MLX5_VIRTQ_MODIFY_MASK_STATE = (u64)1 << 0, MLX5_VIRTQ_MODIFY_MASK_DIRTY_BITMAP_PARAMS = (u64)1 << 3, MLX5_VIRTQ_MODIFY_MASK_DIRTY_BITMAP_DUMP_ENABLE = (u64)1 << 4, + MLX5_VIRTQ_MODIFY_MASK_DESC_GROUP_MKEY = (u64)1 << 14, }; enum { From patchwork Thu Sep 28 16:45:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Dragos Tatulea X-Patchwork-Id: 146366 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:cae8:0:b0:403:3b70:6f57 with SMTP id r8csp3683084vqu; Thu, 28 Sep 2023 17:09:14 -0700 (PDT) X-Google-Smtp-Source: AGHT+IE+R4e0eeCPRv7DNXJ15LM95YGUbfny8WaT5i2jbLbemlz2Q9HfDQeQr91NRBrWTxpUAsub X-Received: by 2002:a05:6358:9195:b0:143:9827:5f71 with SMTP id j21-20020a056358919500b0014398275f71mr3332806rwa.8.1695946153927; Thu, 28 Sep 2023 17:09:13 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1695946153; cv=pass; d=google.com; s=arc-20160816; b=VxepTUPlehV3D7WYaFBptOFPv5zx4wRHxCXq70e183wEH91eKjwkm6fSvQ9cd+Ko58 anj4CjWJ1chS5iS0cDSJXejyb4Jwgym7z0fPlLNeMLZWAdIfPjRGdTsBxTgImTUefilT wcN3/wEyeHj0sNT0uCnekvmGsEszkVapdRqpD+C5urv60TZk+lwZS+TFZqatYgGUKkls lpaUZmWM7ejcarZu3wXxJEMj+SNigpdkorQk1NFMM4mNGP68lkvjP+pO9+PVFB6GOVUe /kfQ1MRnHf3DwjcvCFJ86C5Qas2khXtX3ZZ5wBVBgHq92GsOeShXXU4lpMrN4+YkBbW7 d4oQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=KNPxs3nbrHeoqWz3cXqfucMTiEexjBAyJD5k6GXMPaE=; fh=4SYmPDx0sMrYC0lSOUfhTOMEzVMfG5iPEyPqlL4Bs2M=; b=XT9JDDdsgaS6FDUKn8gOGzD2f3WyYoAW+0zIh0PdOtX5aDCb96qTwrF51cmn5bAnOV n5+cRAAtBg5msvw3Nv8z7FBrwUkRGU951BcMnIlM3ijWIAauCWCfR3J1zKPeGoJAu268 xjI7OizJFujRvcbCPUYizIc1nH2Qkws3YD6viS3gVrMCDeN4c7pfKNW88j/L9nfTEUqi dalBhYgOCRsgNqi7Eejk8osvaNSjg//Tyv/O5LD1yY1evlKSqfqF/jBrF4WlGfJ07O/R +aYWizcQikVAWXojfgRPQgISCopDlN8JGf14uqa1uOB9SMm+2xy//c62jxLyoxvA7eAB cqYw== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=KNfQt0I+; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: from howler.vger.email (howler.vger.email. [23.128.96.34]) by mx.google.com with ESMTPS id g18-20020a631112000000b00563d9ff5158si19408145pgl.350.2023.09.28.17.09.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 28 Sep 2023 17:09:13 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) client-ip=23.128.96.34; Authentication-Results: mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=KNfQt0I+; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id 1648B822F8DC; Thu, 28 Sep 2023 09:51:30 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232179AbjI1QvJ (ORCPT + 21 others); Thu, 28 Sep 2023 12:51:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58066 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232078AbjI1Quu (ORCPT ); Thu, 28 Sep 2023 12:50:50 -0400 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2050.outbound.protection.outlook.com [40.107.220.50]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CD63BBF for ; Thu, 28 Sep 2023 09:50:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SgWpIvUBTyxteUWy+VA6NKTDWCYwwp1creD3HKqStyUiVGfPj6e1DAq2VV8zncZ9+7KcpDnY0QRAdq0zTfmg1mgNWqabyqX3s0qkFA04JHzDX+SFRnv3aTmXTwsx554q9HmSzFHtGkOBKyrVe+GsMhJ7AXBjie+nOvloNHT5bZP3z0DsePvyhqwLx4cx+v8f+oMYFxmj9xZl5IdiOfjp0tt5hBL0rXv/PfsiEuY/AGwcFtpY80v5TPpJE+ryf9BxhG734AghEYCekMDrKFDusOtl81li/fjdJutEkR3vdyrQ/mjpFNLvQi/eUVZiH4UPtXsUYMJLjPXKtSu8oOpMbg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=KNPxs3nbrHeoqWz3cXqfucMTiEexjBAyJD5k6GXMPaE=; b=IDLlrPndrfpZWlM5ieyWukfYrmymyW8hSgfwBqrdf18c4t9wbGBACFmQi3j/xU3Jt/3Xe2112g2SGl2wxpJ9qAQtwit5Fe+/tgAemxFgqa9Qe9u9YZmgb8KncV9Gz48Fv2wY1ZJ1fRmzOXv5k+bp7OqGemmrequ8ooHcLpGUNhUkyHpkiDuNfNoLJyxIjfPgm3eEln0YiT8rNaJb9gRyWZkRRc8O4e4kfTdFppsm9lOgpKDczDTJskAC864izPNv0XYcNcJIDTBcz7GXPgZ/lf24Ulzn9C1Ls68lGYPA0qBwHQ1NOX4kCXae04FhiOr/KPcB3cPQOOUAZ49WWSk6Jw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.232) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KNPxs3nbrHeoqWz3cXqfucMTiEexjBAyJD5k6GXMPaE=; b=KNfQt0I+larDqjbytSvXdMSl695UqZ/MKkVlETQQnHB0lA/OG7hK6/c9notNVIFePvdh6dKdEQHEfsX/sq7r+4GACybwa6kQ0qy1NQCFa7XfpXkwOyXlJjh1IpEJxAaGhIr1mM/povfzT5yZnCI9iJUcnhD0loDg0nIPIg8XUfcpgQdR4VbDZW7NsKATZC+obitNI5NVM7H5QmpkdS8h4N30vsVZFaH19JPrEYAHXwVPtrsOAbYYc3aRA/dP+3aY2DNXDP988vxGeUN/QqV+goClIKbK8pBFgVYd04FOiImmYLiAO0HtPNyXUq7suwsc6gAPMb0j3qJJXFyGSUy1AQ== Received: from CY5PR17CA0056.namprd17.prod.outlook.com (2603:10b6:930:12::10) by SN7PR12MB7323.namprd12.prod.outlook.com (2603:10b6:806:29a::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6813.28; Thu, 28 Sep 2023 16:50:26 +0000 Received: from CY4PEPF0000E9CF.namprd03.prod.outlook.com (2603:10b6:930:12:cafe::6) by CY5PR17CA0056.outlook.office365.com (2603:10b6:930:12::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.25 via Frontend Transport; Thu, 28 Sep 2023 16:50:26 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.232) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.232 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.232; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.232) by CY4PEPF0000E9CF.mail.protection.outlook.com (10.167.241.142) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.19 via Frontend Transport; Thu, 28 Sep 2023 16:50:25 +0000 Received: from drhqmail201.nvidia.com (10.126.190.180) by mail.nvidia.com (10.127.129.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Thu, 28 Sep 2023 09:50:11 -0700 Received: from drhqmail202.nvidia.com (10.126.190.181) by drhqmail201.nvidia.com (10.126.190.180) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Thu, 28 Sep 2023 09:50:11 -0700 Received: from c-237-113-220-225.mtl.labs.mlnx (10.127.8.12) by mail.nvidia.com (10.126.190.181) with Microsoft SMTP Server id 15.2.986.41 via Frontend Transport; Thu, 28 Sep 2023 09:50:09 -0700 From: Dragos Tatulea To: , , "Michael S . Tsirkin" , Jason Wang , Xuan Zhuo CC: , Dragos Tatulea , Subject: [PATCH vhost 15/16] vdpa/mlx5: Make iotlb helper functions more generic Date: Thu, 28 Sep 2023 19:45:26 +0300 Message-ID: <20230928164550.980832-17-dtatulea@nvidia.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230928164550.980832-2-dtatulea@nvidia.com> References: <20230928164550.980832-2-dtatulea@nvidia.com> MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000E9CF:EE_|SN7PR12MB7323:EE_ X-MS-Office365-Filtering-Correlation-Id: 16e316bc-4671-4347-5a6d-08dbc04303b7 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: l4xhs/AtuERVwoWpggppHkpw4U5o0dBF9K2N0Lwmi55OPZygque8UMt4HNKILQDIbhhbHu/4FvsXqkCXtIhlPkzrjjZGhI1e7PgFX6PrJ+daycqKdmVcQY8rkvXHNr/d5Vxv7twmRyVXjPO0jNvtXjmvTD1iPcfGVL3I9e7JE6vwF/VxQMhqGXIyqhLI6QcNBSRhX5q6VfV8Rf8Vkrw1MWqIWGkSCrc2ZY5u/4IwkeahrN1QAErhMk96274qKPA3nAO+f0uoRzGaqms/5G9bb44tXZ1ayKpImEjFJDTTfcWIM4KbsxLqq9OQXSYNi1wCBu5a4HdMTMuu/U6XZEiudkhI8QyBV/7BOoDZ4R+vtOs8MLwrbF0HQPGHsN/NlYDjK40Khnhc/gow43V3GkyG2ISrrvQDIyUXhsPaOPX95CBz5lqRzWlNZk/tFd1RFQ/jPfQMrDCU713bscVb+VfR8vU7bGBZYp+35JJo03U8WCuWCWYcCXWBFEFd1MMY6U7EYGzz4yknWukmAyWjwpe+Kxv/ctsFnzAc6WruBGVu3T96zGesbdBEJpkKu+DchSLKetEkbEleCX89sBfotzA89DYhRv4gd9GuJvkFX4T+nimIFJGg0FwcrN+z9Wen4LGDWG5rf7rEavGv0ciEGiacVy7ejzZpGicHfOQtPA4xDWDMDlKgNMomXiNtRinSUriJuTFSFvWsVA6zCGJBsJ0k8TrpuT5t9ypsvnziay9M3LalhQOPlq3jhw5nnN0HfiO0WSkjt+oxK1Dn7hFdfWkQBw== X-Forefront-Antispam-Report: CIP:216.228.118.232;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc7edge1.nvidia.com;CAT:NONE;SFS:(13230031)(4636009)(376002)(396003)(39860400002)(346002)(136003)(230922051799003)(64100799003)(451199024)(186009)(1800799009)(82310400011)(46966006)(36840700001)(40470700004)(2906002)(110136005)(5660300002)(54906003)(6666004)(70586007)(478600001)(70206006)(1076003)(2616005)(336012)(426003)(26005)(316002)(8676002)(8936002)(40460700003)(4326008)(83380400001)(41300700001)(82740400003)(356005)(7636003)(36756003)(86362001)(40480700001)(36860700001)(47076005)(41533002);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2023 16:50:25.9809 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 16e316bc-4671-4347-5a6d-08dbc04303b7 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.118.232];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000E9CF.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB7323 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Thu, 28 Sep 2023 09:51:30 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1778328433990585388 X-GMAIL-MSGID: 1778328433990585388 They will be used in a followup patch. Signed-off-by: Dragos Tatulea Acked-by: Eugenio Pérez --- drivers/vdpa/mlx5/core/mr.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/drivers/vdpa/mlx5/core/mr.c b/drivers/vdpa/mlx5/core/mr.c index 3dee6d9bed6b..a4135c16b5bf 100644 --- a/drivers/vdpa/mlx5/core/mr.c +++ b/drivers/vdpa/mlx5/core/mr.c @@ -454,20 +454,20 @@ static void destroy_dma_mr(struct mlx5_vdpa_dev *mvdev, struct mlx5_vdpa_mr *mr) mlx5_vdpa_destroy_mkey(mvdev, mr->mkey); } -static int dup_iotlb(struct mlx5_vdpa_dev *mvdev, struct vhost_iotlb *src) +static int dup_iotlb(struct vhost_iotlb *iotlb, struct vhost_iotlb *src) { struct vhost_iotlb_map *map; u64 start = 0, last = ULLONG_MAX; int err; if (!src) { - err = vhost_iotlb_add_range(mvdev->cvq.iotlb, start, last, start, VHOST_ACCESS_RW); + err = vhost_iotlb_add_range(iotlb, start, last, start, VHOST_ACCESS_RW); return err; } for (map = vhost_iotlb_itree_first(src, start, last); map; map = vhost_iotlb_itree_next(map, start, last)) { - err = vhost_iotlb_add_range(mvdev->cvq.iotlb, map->start, map->last, + err = vhost_iotlb_add_range(iotlb, map->start, map->last, map->addr, map->perm); if (err) return err; @@ -475,9 +475,9 @@ static int dup_iotlb(struct mlx5_vdpa_dev *mvdev, struct vhost_iotlb *src) return 0; } -static void prune_iotlb(struct mlx5_vdpa_dev *mvdev) +static void prune_iotlb(struct vhost_iotlb *iotlb) { - vhost_iotlb_del_range(mvdev->cvq.iotlb, 0, ULLONG_MAX); + vhost_iotlb_del_range(iotlb, 0, ULLONG_MAX); } static void destroy_user_mr(struct mlx5_vdpa_dev *mvdev, struct mlx5_vdpa_mr *mr) @@ -544,7 +544,7 @@ void mlx5_vdpa_destroy_mr_resources(struct mlx5_vdpa_dev *mvdev) for (int i = 0; i < MLX5_VDPA_NUM_AS; i++) mlx5_vdpa_destroy_mr(mvdev, mvdev->mr[i]); - prune_iotlb(mvdev); + prune_iotlb(mvdev->cvq.iotlb); } static int _mlx5_vdpa_create_mr(struct mlx5_vdpa_dev *mvdev, @@ -596,8 +596,8 @@ int mlx5_vdpa_update_cvq_iotlb(struct mlx5_vdpa_dev *mvdev, spin_lock(&mvdev->cvq.iommu_lock); - prune_iotlb(mvdev); - err = dup_iotlb(mvdev, iotlb); + prune_iotlb(mvdev->cvq.iotlb); + err = dup_iotlb(mvdev->cvq.iotlb, iotlb); spin_unlock(&mvdev->cvq.iommu_lock); From patchwork Thu Sep 28 16:45:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Dragos Tatulea X-Patchwork-Id: 146250 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:cae8:0:b0:403:3b70:6f57 with SMTP id r8csp3506800vqu; Thu, 28 Sep 2023 11:14:19 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHHjtE0IqSa/Q79LWUbGh7NCiYxU/91wyybkFx5p5lt2zfW5kQiNUKLr6aEu3WzO1r/sAAt X-Received: by 2002:a05:6a20:5611:b0:159:c918:1016 with SMTP id ir17-20020a056a20561100b00159c9181016mr1779763pzc.49.1695924859274; Thu, 28 Sep 2023 11:14:19 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1695924859; cv=pass; d=google.com; s=arc-20160816; b=i6S77uZb+/D+3ZPUOb52r8yAnz/lf09M/5ghWEKQzqGKrZjZDdz/+gjslFsiEA0cEg 0dlFQZJHNBfUe2+x/bHhsZcMd+4r9YmD96RtA6ByVKeeEY5Uf64L/0cYR1yGzGqPinpx RUXa0I8OvML0mmHkXQR7WqXa6VogiCmhVMcZaUSvH8gLN9iqAtHbSBm0P+193NZ4utpI YZL88qwF7gT45b5tDcDh8oXKKR/OZRDftI8VpL3bkTbXDBWLTvM9ztPILzyaMDAKJUGd O46fJvj7wvuNCnopNm8O1VUii1BG/N+pvb8vvnDbEhjzXVh36gxUSHOOYA/PseKWvX3I 6yuA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=gqXzNtuzmep7ULt81tvKIba4FBPF+U2bzirjqh6BoMg=; fh=4SYmPDx0sMrYC0lSOUfhTOMEzVMfG5iPEyPqlL4Bs2M=; b=CMvD7/zL+lE+TSK8m+kbe9W6h5dYyGTGafmzPSehaNDw+L6yh5fi+Kb619jQpGmHPk O+3yK6CjSD8VqDGuGNCvQzgyQnzl2E26oNUqkcaSrR4WCY/Ydlc6dnTkkVkyTGOZYZxG 5i6WCdorgT4bNkjNTgFLlnjoSdr7CWFjBOzr3KNjIeyFUS/PYPJc7LlE2PyZ3+wHcBK4 JkFwV8drboFNNt4RMvQ5zxPsmuwEhuFIGiEAyOrHKXzYbye7PZ84dbLfGukRaWX0TodM Z8uFLIadM5vYjmBkK6YBCAKmPjvsRIKfn6rEEBATsoHP7JzcauLKfPYk886J9FcO+OXr xmgQ== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=VGJivgmA; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: from howler.vger.email (howler.vger.email. [23.128.96.34]) by mx.google.com with ESMTPS id q18-20020a17090311d200b001b89551a392si20516971plh.113.2023.09.28.11.14.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 28 Sep 2023 11:14:19 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) client-ip=23.128.96.34; Authentication-Results: mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=VGJivgmA; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id B1AE38233640; Thu, 28 Sep 2023 09:51:36 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231310AbjI1QvN (ORCPT + 21 others); Thu, 28 Sep 2023 12:51:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58108 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232106AbjI1Quw (ORCPT ); Thu, 28 Sep 2023 12:50:52 -0400 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2063.outbound.protection.outlook.com [40.107.237.63]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8BA55CC7 for ; Thu, 28 Sep 2023 09:50:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=mwQ42FeyZpcmqr4QpH3J3h2hpvWbbbdMi07REVGf/xClGq/sKh7Qpwya9dkPNuPBjC0LKFvP/RviwaWyk2SokC/BFGzXd7bGAOJ8qxw4NETCuqLUUfdTglyiDPFUc7T+EWkf8lEdMSLZf3UooyZkrnlyGs8WSTKTY9qncMcIwQvpHngTt5cJ+NbaMx7xkafNdiWzt8LMgUtQjpvRSvQ/b5DDpG4m4YfNGjzmy67m/PE13Umb35p0uSa+oRiyVhZxxGdhmxcFMBRA04Z39vNwvg6ZolfCAuCqDwCLDr9QSOaexr5gnq84wFWPz0XVBJ+hpqjPAEJeB0j3Z9aMTuVT0g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=gqXzNtuzmep7ULt81tvKIba4FBPF+U2bzirjqh6BoMg=; b=OMjKggyhn5CLKZ7DDp5DT0DpoAZh8kTLi6BoTzEI1Z//smRrg9ryEsYAJvCC+ciB4cEoG5eMsoKPTOX5tSYe0tKonZgDJqUZWpRAmvNrZ2v71X7q3HlaYsE9cLPThUnF0QA+3FEHx/bMvMfclIGAZbGspJuzToK3aVFNLsapPvifoZPIQlQmelf8waSrT/BSgVTEFstkaojyHeuFy81NJbtHC8X/uhxQA2jq0pX0k6VMRVDO4OwsjGKMkvyOohptNnojmhR0aU+y4RV9M8UGmfDbqxPf5MywgUN1NpSusTunINWudSVIw8TrFMdezhxy3f+bDbjPN8pJcc8DnS58ng== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.233) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=gqXzNtuzmep7ULt81tvKIba4FBPF+U2bzirjqh6BoMg=; b=VGJivgmAX5RifnPykfy9/28ffOOanjqIoILCSuTQOKgjXQpj58vSL44WrekW0SkjZ8UEzTrhZcp1v+PUu6ccI89Hh/jl2eOrzLOIENWgnpR8RdCpEhvuycstWs8UJv6/8bL4nvTtMfWb6LacCQkSOyWE2YmuobuDDpqAFM6UcZm9sJPJzq+7/se0s/HRdLtLA/DODY/X3c4Racamqjk6xkb4mGR4SUxbtCOmXNA9VGcHQDp7tK2pTjxxJcGjtRxI6sGWoeMrkO3UBj6RSy+fXK+x7k6D8Z0LqjX34Ha335w4XJ6aB0AFoKto2ALM4ojuyQdXmUBDtPpbA6fQuZ65XQ== Received: from MW2PR16CA0035.namprd16.prod.outlook.com (2603:10b6:907::48) by DM4PR12MB7646.namprd12.prod.outlook.com (2603:10b6:8:106::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6745.18; Thu, 28 Sep 2023 16:50:29 +0000 Received: from CO1PEPF000044F9.namprd21.prod.outlook.com (2603:10b6:907:0:cafe::85) by MW2PR16CA0035.outlook.office365.com (2603:10b6:907::48) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.25 via Frontend Transport; Thu, 28 Sep 2023 16:50:28 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.233) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.233 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.233; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.233) by CO1PEPF000044F9.mail.protection.outlook.com (10.167.241.199) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6863.9 via Frontend Transport; Thu, 28 Sep 2023 16:50:28 +0000 Received: from drhqmail201.nvidia.com (10.126.190.180) by mail.nvidia.com (10.127.129.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Thu, 28 Sep 2023 09:50:14 -0700 Received: from drhqmail202.nvidia.com (10.126.190.181) by drhqmail201.nvidia.com (10.126.190.180) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Thu, 28 Sep 2023 09:50:14 -0700 Received: from c-237-113-220-225.mtl.labs.mlnx (10.127.8.12) by mail.nvidia.com (10.126.190.181) with Microsoft SMTP Server id 15.2.986.41 via Frontend Transport; Thu, 28 Sep 2023 09:50:12 -0700 From: Dragos Tatulea To: , , "Michael S . Tsirkin" , Jason Wang , Xuan Zhuo CC: , Dragos Tatulea , Subject: [PATCH vhost 16/16] vdpa/mlx5: Update cvq iotlb mapping on ASID change Date: Thu, 28 Sep 2023 19:45:27 +0300 Message-ID: <20230928164550.980832-18-dtatulea@nvidia.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230928164550.980832-2-dtatulea@nvidia.com> References: <20230928164550.980832-2-dtatulea@nvidia.com> MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000044F9:EE_|DM4PR12MB7646:EE_ X-MS-Office365-Filtering-Correlation-Id: 6df76dc9-3965-44b2-e1bf-08dbc043056d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 5wiL0Vb8XEAa5WYLQvRQw9gJJryeUhdQE6JegoxvSi/5hDP0lgt1DavRqoEc0RkLjAHhD6dielmyvDtlfKERUUbP+xh+UPyQNendkrfdiuDIXdHlKmXY4NVzSCJW2zyamtNbAyys10pX6wt8+OR7TAqgUs9fw3n7wkgr8Yeh+Q1tfcX8XUbCcmgSH7n8lmElwsYTclayIgbyVav3LMWzVCRKUKqhzEfVWxo1p0H4InZgqn3NOLyNGc8B62qA93cZ//RnUZStZVLgjd2/XBy4ctLqCFmWjYK3/x+2B42QhdtMJ9smrhdbkVsHvMygrqI3dkJHgWG4P7Hqbq6ksQZJxtD8dkBlzfBWahyAW5faau8w9H4TjDl85G+vlSddc9uQgHr59lrxAz9AgjmV7TsNR8lNRk66MdEpZVSvW1Nk66sN6O1HnNrQc1OQlmLqgXR4PcJky9dE+ZjO3EsHo/FyvsUqpLzQD5cbg7fff9cO/507a9N7SKfraCttcM3CdWpNCLF0fquDMGFs4GoFKJvUG7QLOR18L5VncDhK+ftBmLT7OABNY7OwKSzH0nP+9t5telBmvgxWJ3J5cg5VkW/5b5SjD5pB7jAY1jHzPD/M1gYfFpxVr4/0Wiky3wY/paRXh9OX7UDMFpOP97r/PIegTIe2l08FdNJW6Ol/Mbj04e8MF3rRdzzcVQH0IBGyrFAWPqbf5FOuRtjyEfXhXVKGGHudmhNlILMpeY17SSPOwCZVql/OwOdGfOCwuAqq9rsv X-Forefront-Antispam-Report: CIP:216.228.118.233;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc7edge2.nvidia.com;CAT:NONE;SFS:(13230031)(4636009)(346002)(39860400002)(376002)(396003)(136003)(230922051799003)(64100799003)(451199024)(186009)(82310400011)(1800799009)(40470700004)(46966006)(36840700001)(7636003)(336012)(426003)(47076005)(1076003)(36860700001)(2616005)(86362001)(26005)(41300700001)(36756003)(83380400001)(40460700003)(8936002)(4326008)(40480700001)(356005)(5660300002)(70206006)(8676002)(82740400003)(110136005)(2906002)(6666004)(70586007)(316002)(54906003)(478600001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2023 16:50:28.8548 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 6df76dc9-3965-44b2-e1bf-08dbc043056d X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.118.233];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000044F9.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB7646 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Thu, 28 Sep 2023 09:51:36 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1778306105124340390 X-GMAIL-MSGID: 1778306105124340390 For the following sequence: - cvq group is in ASID 0 - .set_map(1, cvq_iotlb) - .set_group_asid(cvq_group, 1) ... the cvq mapping from ASID 0 will be used. This is not always correct behaviour. This patch adds support for the above mentioned flow by saving the iotlb on each .set_map and updating the cvq iotlb with it on a cvq group change. Signed-off-by: Dragos Tatulea Acked-by: Eugenio Pérez --- drivers/vdpa/mlx5/core/mlx5_vdpa.h | 2 ++ drivers/vdpa/mlx5/core/mr.c | 26 ++++++++++++++++++++++++++ drivers/vdpa/mlx5/net/mlx5_vnet.c | 9 ++++++++- 3 files changed, 36 insertions(+), 1 deletion(-) diff --git a/drivers/vdpa/mlx5/core/mlx5_vdpa.h b/drivers/vdpa/mlx5/core/mlx5_vdpa.h index ae09296f4270..db988ced5a5d 100644 --- a/drivers/vdpa/mlx5/core/mlx5_vdpa.h +++ b/drivers/vdpa/mlx5/core/mlx5_vdpa.h @@ -32,6 +32,8 @@ struct mlx5_vdpa_mr { unsigned long num_directs; unsigned long num_klms; + struct vhost_iotlb *iotlb; + bool user_mr; }; diff --git a/drivers/vdpa/mlx5/core/mr.c b/drivers/vdpa/mlx5/core/mr.c index a4135c16b5bf..403c08271489 100644 --- a/drivers/vdpa/mlx5/core/mr.c +++ b/drivers/vdpa/mlx5/core/mr.c @@ -499,6 +499,8 @@ static void _mlx5_vdpa_destroy_mr(struct mlx5_vdpa_dev *mvdev, struct mlx5_vdpa_ destroy_user_mr(mvdev, mr); else destroy_dma_mr(mvdev, mr); + + vhost_iotlb_free(mr->iotlb); } void mlx5_vdpa_destroy_mr(struct mlx5_vdpa_dev *mvdev, @@ -558,6 +560,30 @@ static int _mlx5_vdpa_create_mr(struct mlx5_vdpa_dev *mvdev, else err = create_dma_mr(mvdev, mr); + if (err) + return err; + + mr->iotlb = vhost_iotlb_alloc(0, 0); + if (!mr->iotlb) { + err = -ENOMEM; + goto err_mr; + } + + err = dup_iotlb(mr->iotlb, iotlb); + if (err) + goto err_iotlb; + + return 0; + +err_iotlb: + vhost_iotlb_free(mr->iotlb); + +err_mr: + if (iotlb) + destroy_user_mr(mvdev, mr); + else + destroy_dma_mr(mvdev, mr); + return err; } diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5_vnet.c index 46441e41892c..fc5d6b989a5a 100644 --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c @@ -3154,12 +3154,19 @@ static int mlx5_set_group_asid(struct vdpa_device *vdev, u32 group, unsigned int asid) { struct mlx5_vdpa_dev *mvdev = to_mvdev(vdev); + int err = 0; if (group >= MLX5_VDPA_NUMVQ_GROUPS || asid >= MLX5_VDPA_NUM_AS) return -EINVAL; mvdev->group2asid[group] = asid; - return 0; + + mutex_lock(&mvdev->mr_mtx); + if (group == MLX5_VDPA_CVQ_GROUP && mvdev->mr[asid]) + err = mlx5_vdpa_update_cvq_iotlb(mvdev, mvdev->mr[asid]->iotlb, asid); + mutex_unlock(&mvdev->mr_mtx); + + return err; } static const struct vdpa_config_ops mlx5_vdpa_ops = {