From patchwork Tue Jan 3 10:51:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Boeuf, Sebastien" X-Patchwork-Id: 38371 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp4548255wrt; Tue, 3 Jan 2023 02:53:09 -0800 (PST) X-Google-Smtp-Source: AMrXdXtvdPuqov9INHU+YUn0cHBoFz2JD1EGkKMiPaKbgbu5ygivX5sC3s9jWbCA3+/SBZ7a1Rti X-Received: by 2002:a05:6a20:2a96:b0:b2:5cf9:817b with SMTP id v22-20020a056a202a9600b000b25cf9817bmr72104604pzh.5.1672743188904; Tue, 03 Jan 2023 02:53:08 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1672743188; cv=none; d=google.com; s=arc-20160816; b=oek2c849s4E++8NStJVMbGJNYlZrOCAS3iiPqR7sshLhzAWoZ87TPFupRnhandbdtc o2n7TokYQAust9tppeqrU6aQ679k5EYs/1/iavKQdT9OrQrPKDmHWUYq+pOLCIbRqvKn lJlSXkzA3x9ybHPYocqaCgrxHIN5O2wY/6Yh8H0zeq0yMOvXX0/+5wZB2h6rOe68vjvR m1KIPJVLyFZdpXHwXBVqNTvfLdz7ie88OJIqR4/9BveKsW4268ZZXvxufRiGsBqlJJfN GkUzk1+R2q5oSGyBxyLPZc3wb1FAkSsr662GV7AM2xMbv81P2aOuCJfOHXKPk4mxYN1h 9JYA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=QLNHna+NJ8rzmbixv6xgH1RchkjZLS4f6MLf1dXszbQ=; b=bxo92uhzKcw/AV/XalGB3b0XP817wkf3EAT3yzcUdRrCPrzgpNSbxDK9/tVopSXsdR 47nnY9NOLnDlp8poxALEeKvUp230nE3jNnccPyVZQmEjVoEFD+DINq6+pvAOiccuVIfM IGc4qM5sjoDrwkWvjvyCEVg5CUWcll4+bxVqaJRnSFDdQKL/Ie42qzZ0T96sFXGwJ87p jevg/xKremsSND0Mqdf+vT4NQTTEPPjvv8Fw1Q64HhzUYBEVQUhmui6mzrsIRwv+Bd8p Cd8kFcWl2pHrIcfaglu+3jLhLHigDVlolu7lPEOoWTe89rtGFNUo4/+1Uh08vxXyu/uZ UghA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Zfx7MmkX; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id x10-20020a63db4a000000b00463e9f26c48si32745373pgi.177.2023.01.03.02.52.53; Tue, 03 Jan 2023 02:53:08 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Zfx7MmkX; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237322AbjACKwE (ORCPT + 99 others); Tue, 3 Jan 2023 05:52:04 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39468 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237108AbjACKvc (ORCPT ); Tue, 3 Jan 2023 05:51:32 -0500 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 05AD0DF3 for ; Tue, 3 Jan 2023 02:51:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1672743092; x=1704279092; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=zukvlO7ayucX1PyDrSkH1p1yDhKXLf7pZ9ViBv268Sg=; b=Zfx7MmkXB6gpkVmIK0PPGoas+8IDi5c5KLumkTL15In6DhDp1zjNISWW w2CTV/CbL5mjgMnlgEHDoSxCQita9fcfuLDoZDLJ5X1g8r/8TtB7Mx/Km 9yKwiuiAccEYcBXW4L8KSo5IjHlegswPC/xAM1B361wRuy323i/1gMDHX mwlGjaGSnBAqxj5/w5ImVRmYybvUOvy0b6RPod3dTHpDsGGVlNY8RjKUH j+muTGxsEmm26BZiLFF7H+L+dMEEouenc/Ch2PncspqeW5hRcN0Qf+U84 g/6iajpHY0xdZ3cFv5xoEDRdIW32mmFodu97PrdI8oWF3ZSMOl4gNCkMn A==; X-IronPort-AV: E=McAfee;i="6500,9779,10578"; a="301318433" X-IronPort-AV: E=Sophos;i="5.96,296,1665471600"; d="scan'208";a="301318433" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Jan 2023 02:51:31 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10578"; a="604772656" X-IronPort-AV: E=Sophos;i="5.96,296,1665471600"; d="scan'208";a="604772656" Received: from isobansk-mobl1.ger.corp.intel.com (HELO sboeuf-mobl.home) ([10.252.24.246]) by orsmga003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Jan 2023 02:51:29 -0800 From: sebastien.boeuf@intel.com To: linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org Cc: mst@redhat.com, jasowang@redhat.com, eperezma@redhat.com, sebastien.boeuf@intel.com Subject: [PATCH v6 1/4] vdpa: Add resume operation Date: Tue, 3 Jan 2023 11:51:05 +0100 Message-Id: <6e05c4b31b47f3e29cb2bd7ebd56c81f84b8f48a.1672742878.git.sebastien.boeuf@intel.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,LOTS_OF_MONEY, RCVD_IN_DNSWL_MED,SPF_HELO_PASS,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1753998358004978159?= X-GMAIL-MSGID: =?utf-8?q?1753998361990250274?= From: Sebastien Boeuf Add a new operation to allow a vDPA device to be resumed after it has been suspended. Trying to resume a device that wasn't suspended will result in a no-op. This operation is optional. If it's not implemented, the associated backend feature bit will not be exposed. And if the feature bit is not exposed, invoking this operation will return an error. Acked-by: Jason Wang Signed-off-by: Sebastien Boeuf Reviewed-by: Stefano Garzarella --- include/linux/vdpa.h | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/include/linux/vdpa.h b/include/linux/vdpa.h index 6d0f5e4e82c2..96d308cbf97b 100644 --- a/include/linux/vdpa.h +++ b/include/linux/vdpa.h @@ -219,7 +219,10 @@ struct vdpa_map_file { * @reset: Reset device * @vdev: vdpa device * Returns integer: success (0) or error (< 0) - * @suspend: Suspend or resume the device (optional) + * @suspend: Suspend the device (optional) + * @vdev: vdpa device + * Returns integer: success (0) or error (< 0) + * @resume: Resume the device (optional) * @vdev: vdpa device * Returns integer: success (0) or error (< 0) * @get_config_size: Get the size of the configuration space includes @@ -324,6 +327,7 @@ struct vdpa_config_ops { void (*set_status)(struct vdpa_device *vdev, u8 status); int (*reset)(struct vdpa_device *vdev); int (*suspend)(struct vdpa_device *vdev); + int (*resume)(struct vdpa_device *vdev); size_t (*get_config_size)(struct vdpa_device *vdev); void (*get_config)(struct vdpa_device *vdev, unsigned int offset, void *buf, unsigned int len); From patchwork Tue Jan 3 10:51:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Boeuf, Sebastien" X-Patchwork-Id: 38373 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp4548345wrt; Tue, 3 Jan 2023 02:53:21 -0800 (PST) X-Google-Smtp-Source: AMrXdXsi+MpRC8L2VkjeVuG3bTMQd4VPfzFZ11v3noDpMp+/GHS+P6aPJcBD07PKJVeqtFkvxqKk X-Received: by 2002:a17:90a:1b82:b0:219:70f1:cef with SMTP id w2-20020a17090a1b8200b0021970f10cefmr47603233pjc.46.1672743201211; Tue, 03 Jan 2023 02:53:21 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1672743201; cv=none; d=google.com; s=arc-20160816; b=XHOvL9VLtOr5f1rpRxwyN8dTEY5pt98zkjvmVBHkZQh04I3ytE+SwqFpAjBLsAm9Sl DB9A1FUEjTYiQQ+oO+Ucl4KK++av18nf2BhuvET2UTdBVjzmPk21tR9t7arBjJEto8mZ qKZb6FrxjQHYfcGsO6LN9W7cuDlUIr8iEQK/uvTlhhqsmimNTgWJNfFNPhVd4JVczyAE AIudz1fT++le1LymQs7jFdZidzrnAU9BkeSYXjigSN44UvWhzlOAs4WpZe8z/RQOH3oW LunCRufEo0xx0SjoHCmppq0PQQS5Zwjy4e4RYiWNebbDWG0plPkLA5ooCaobX2XiIPZl xIIQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=E5hYtJLaLYD8nZjC1AlQ3IMgWvuOvSL0616tPGzzVqE=; b=ifIqBucoGKtniQtOFPSNWmycW5TRjJbtXQ14vNhEU42mQslb4cNj65Nr9DFcpitXE2 We/tVZYVE0/8uBv/rvlNrLoIoRzVIYjkUawUzne8tvTkiWScBaRuWteD1FCuQ9aIUmkT 53SH1H3+U5zbwxon2PUcvYpq1TrrnNytuCg+MW8UI5aSVnB++X6sneBI+U723CdO7AcB SvC3RyzVATtOSNrLrALncXJ5erU3i3KkDiKpEuw+xgFJwG8gCU2+5EiYt1YvB6tuitaZ 2NEgPFpYkV8iGZXb636JEB+5GWrwwT4bTXnXzF7iM1dI/Tfs9QtvZBDIu4mOIY13vGXy uoPQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=PpLoTiiA; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id lp5-20020a17090b4a8500b00218a9f90590si38317536pjb.2.2023.01.03.02.53.04; Tue, 03 Jan 2023 02:53:21 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=PpLoTiiA; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237429AbjACKwK (ORCPT + 99 others); Tue, 3 Jan 2023 05:52:10 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39504 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237193AbjACKvi (ORCPT ); Tue, 3 Jan 2023 05:51:38 -0500 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 765602FA for ; Tue, 3 Jan 2023 02:51:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1672743097; x=1704279097; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=wwAlVm/7I8ndsWi60dF17HSTriSq8epoPrbPLSohnLw=; b=PpLoTiiAXvPV6Y0VKIgd581nm9AxugWU/N8sDhTYFo0TI/aIuE2wbShk ORfQmkUbA7DKmB/cFQYW9tvBGQi4AoPo11ezZAyzzEZrpxSHrJStaWjJx qKtD9zHglvyKLCFrlElVLMvLiWuHDQlPOMY/cmADgpT/43Wjh6VUrrIS1 dDzYe1CbDVmttZApxjr1njzECMyoeM/gxCSbe2YIyyf5cZEEG/YnFlnMe ecSDYTUNqHvkWOCxEieiuUwDLXCv97W/0IAPeSE9qAuF/cYDngEYzZw1Y OwMRNepXuZF0OlYhHhQNGTuUiDVb4up5vf/i0y7jLVznumuwWhv6Sg4MQ w==; X-IronPort-AV: E=McAfee;i="6500,9779,10578"; a="301318451" X-IronPort-AV: E=Sophos;i="5.96,296,1665471600"; d="scan'208";a="301318451" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Jan 2023 02:51:37 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10578"; a="604772689" X-IronPort-AV: E=Sophos;i="5.96,296,1665471600"; d="scan'208";a="604772689" Received: from isobansk-mobl1.ger.corp.intel.com (HELO sboeuf-mobl.home) ([10.252.24.246]) by orsmga003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Jan 2023 02:51:35 -0800 From: sebastien.boeuf@intel.com To: linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org Cc: mst@redhat.com, jasowang@redhat.com, eperezma@redhat.com, sebastien.boeuf@intel.com Subject: [PATCH v6 2/4] vhost-vdpa: Introduce RESUME backend feature bit Date: Tue, 3 Jan 2023 11:51:06 +0100 Message-Id: X-Mailer: git-send-email 2.37.2 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,LOTS_OF_MONEY, RCVD_IN_DNSWL_MED,SPF_HELO_PASS,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1753998374919548804?= X-GMAIL-MSGID: =?utf-8?q?1753998374919548804?= From: Sebastien Boeuf Userspace knows if the device can be resumed or not by checking this feature bit. It's only exposed if the vdpa driver backend implements the resume() operation callback. Userspace trying to negotiate this feature when it hasn't been exposed will result in an error. Acked-by: Jason Wang Signed-off-by: Sebastien Boeuf Reviewed-by: Stefano Garzarella --- drivers/vhost/vdpa.c | 16 +++++++++++++++- include/uapi/linux/vhost_types.h | 2 ++ 2 files changed, 17 insertions(+), 1 deletion(-) diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c index 166044642fd5..833617d00ef6 100644 --- a/drivers/vhost/vdpa.c +++ b/drivers/vhost/vdpa.c @@ -355,6 +355,14 @@ static bool vhost_vdpa_can_suspend(const struct vhost_vdpa *v) return ops->suspend; } +static bool vhost_vdpa_can_resume(const struct vhost_vdpa *v) +{ + struct vdpa_device *vdpa = v->vdpa; + const struct vdpa_config_ops *ops = vdpa->config; + + return ops->resume; +} + static long vhost_vdpa_get_features(struct vhost_vdpa *v, u64 __user *featurep) { struct vdpa_device *vdpa = v->vdpa; @@ -602,11 +610,15 @@ static long vhost_vdpa_unlocked_ioctl(struct file *filep, if (copy_from_user(&features, featurep, sizeof(features))) return -EFAULT; if (features & ~(VHOST_VDPA_BACKEND_FEATURES | - BIT_ULL(VHOST_BACKEND_F_SUSPEND))) + BIT_ULL(VHOST_BACKEND_F_SUSPEND) | + BIT_ULL(VHOST_BACKEND_F_RESUME))) return -EOPNOTSUPP; if ((features & BIT_ULL(VHOST_BACKEND_F_SUSPEND)) && !vhost_vdpa_can_suspend(v)) return -EOPNOTSUPP; + if ((features & BIT_ULL(VHOST_BACKEND_F_RESUME)) && + !vhost_vdpa_can_resume(v)) + return -EOPNOTSUPP; vhost_set_backend_features(&v->vdev, features); return 0; } @@ -658,6 +670,8 @@ static long vhost_vdpa_unlocked_ioctl(struct file *filep, features = VHOST_VDPA_BACKEND_FEATURES; if (vhost_vdpa_can_suspend(v)) features |= BIT_ULL(VHOST_BACKEND_F_SUSPEND); + if (vhost_vdpa_can_resume(v)) + features |= BIT_ULL(VHOST_BACKEND_F_RESUME); if (copy_to_user(featurep, &features, sizeof(features))) r = -EFAULT; break; diff --git a/include/uapi/linux/vhost_types.h b/include/uapi/linux/vhost_types.h index 53601ce2c20a..c5690a8992d8 100644 --- a/include/uapi/linux/vhost_types.h +++ b/include/uapi/linux/vhost_types.h @@ -163,5 +163,7 @@ struct vhost_vdpa_iova_range { #define VHOST_BACKEND_F_IOTLB_ASID 0x3 /* Device can be suspended */ #define VHOST_BACKEND_F_SUSPEND 0x4 +/* Device can be resumed */ +#define VHOST_BACKEND_F_RESUME 0x5 #endif From patchwork Tue Jan 3 10:51:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Boeuf, Sebastien" X-Patchwork-Id: 38374 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp4548497wrt; Tue, 3 Jan 2023 02:53:47 -0800 (PST) X-Google-Smtp-Source: AMrXdXu29tu05tGOy40HiZ55bP98tSYVS4q3w0al3V0Y5X78I8IdvLF/RGVJt4H9vC7OaeXyUbN0 X-Received: by 2002:a62:1642:0:b0:580:c548:9910 with SMTP id 63-20020a621642000000b00580c5489910mr33004597pfw.7.1672743227695; Tue, 03 Jan 2023 02:53:47 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1672743227; cv=none; d=google.com; s=arc-20160816; b=zzKWAm5sw3yyh+7eAO+AAYInqi5gmpTo2lo+P96s+aKF1Tvo622lZYAOBOxlNIAqRD lQRnbz1BFN2UFwxo0xfVPUXtOr1Xu3j5ZsBkZ+OtnNUGQz41piLlNqEzL24yCP47QfSp pPK6C1mM83s5+Yz2uTmYcNCYPpkTRF4Ba9r6lKRzGQj5yI5yrUZu4D/ujea3uj5CE4Mg J9AWcrsJNqlX3lj1jQKW3ZmzYMsNxx+fl/O5iyh6Cisu8YrRvS81KEJl0WLkkf2Lyyu9 LcNmKhN3XbjUAgZK9xxQ+65yNe57uvWrIedGJCs3XhbvsANx6vtMlq9LxsWH/9OxOFa6 cf9A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=l4fCEsiLzm45J7O+K4Y88a/Guop8UyNFm/AVdJja+oc=; b=NasenVg6F8qoXCNWtNGnFYTDERayrTt+KxXT0LGkxPvo/cSVztvJgiGW6kgpoXCTQr cezQF9EwyvClZyJvP+QosjpApoV4VWyVk/gF4OZAxk7g729r1etYad5dtqi0GniSxwBS Ro9FiEeYFdNZ71rkj/Rxp7lcI1UX1l6QKJyN3oWIj7zI/34pF3yPgp9blGTTdPXfkhLh 6ClSCBJHZg/GiF7hjOA+mVlAm0sRLzs55WOSGKzPjNLtCDuban4jZp0TXn/Ns/tI0FMg gHHrR0inXrmNa+z3VE9jqCLlq2UwHnZhlKAsFgD49Mqh8XBwLml/tP9zS3EDuuaKBcvM kjig== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=P9+4tBXn; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id j14-20020a62e90e000000b005775394c99fsi31004735pfh.52.2023.01.03.02.53.34; Tue, 03 Jan 2023 02:53:47 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=P9+4tBXn; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237440AbjACKwM (ORCPT + 99 others); Tue, 3 Jan 2023 05:52:12 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39558 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237304AbjACKvm (ORCPT ); Tue, 3 Jan 2023 05:51:42 -0500 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 784FB2AE1 for ; Tue, 3 Jan 2023 02:51:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1672743101; x=1704279101; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Wpp2BiRmPtCrq2dRBOAMounsOwKlI+2jikZIpNAzISg=; b=P9+4tBXnraxk6gOQE34Ei6+MCf4EUe6dmRVgVf0LcBe8DiWMvMeD0qlx zUxxTF0e5sZsLB3Jd139wpn0Gic6Zf+ObMqIeSHPpB8tOOrm7lYOvXPfD KfvqYliwPrD+awOebQwWYFkL3kXfNEDej+ybREW3a8H0mBVVKurq+UA5r qxvUh5WFJPdqXdTnJyOmJ20OKv8wkigFMiKBFE7bBWP8ZqUZ7e1JvxuNn lA0gYI2xhUvSFNUgHkdgbjwcE8FNxPOr/adHAhBjpM7Bdrx0yAZJ1zdvX 772Um0xuvbJqDbEXfgD7ZHj7tT4b105v2cfkOKJwpWFo7Bxbd2giy6+WB A==; X-IronPort-AV: E=McAfee;i="6500,9779,10578"; a="301318463" X-IronPort-AV: E=Sophos;i="5.96,296,1665471600"; d="scan'208";a="301318463" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Jan 2023 02:51:41 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10578"; a="604772706" X-IronPort-AV: E=Sophos;i="5.96,296,1665471600"; d="scan'208";a="604772706" Received: from isobansk-mobl1.ger.corp.intel.com (HELO sboeuf-mobl.home) ([10.252.24.246]) by orsmga003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Jan 2023 02:51:39 -0800 From: sebastien.boeuf@intel.com To: linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org Cc: mst@redhat.com, jasowang@redhat.com, eperezma@redhat.com, sebastien.boeuf@intel.com Subject: [PATCH v6 3/4] vhost-vdpa: uAPI to resume the device Date: Tue, 3 Jan 2023 11:51:07 +0100 Message-Id: <73b75fb87d25cff59768b4955a81fe7ffe5b4770.1672742878.git.sebastien.boeuf@intel.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,LOTS_OF_MONEY, RCVD_IN_DNSWL_MED,SPF_HELO_PASS,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1753998402830005203?= X-GMAIL-MSGID: =?utf-8?q?1753998402830005203?= From: Sebastien Boeuf This new ioctl adds support for resuming the device from userspace. This is required when trying to restore the device in a functioning state after it's been suspended. It is already possible to reset a suspended device, but that means the device must be reconfigured and all the IOMMU/IOTLB mappings must be recreated. This new operation allows the device to be resumed without going through a full reset. This is particularly useful when trying to perform offline migration of a virtual machine (also known as snapshot/restore) as it allows the VMM to resume the virtual machine back to a running state after the snapshot is performed. Acked-by: Jason Wang Signed-off-by: Sebastien Boeuf Reviewed-by: Stefano Garzarella --- drivers/vhost/vdpa.c | 18 ++++++++++++++++++ include/uapi/linux/vhost.h | 8 ++++++++ 2 files changed, 26 insertions(+) diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c index 833617d00ef6..1db7bd39fb63 100644 --- a/drivers/vhost/vdpa.c +++ b/drivers/vhost/vdpa.c @@ -502,6 +502,21 @@ static long vhost_vdpa_suspend(struct vhost_vdpa *v) return ops->suspend(vdpa); } +/* After a successful return of this ioctl the device resumes processing + * virtqueue descriptors. The device becomes fully operational the same way it + * was before it was suspended. + */ +static long vhost_vdpa_resume(struct vhost_vdpa *v) +{ + struct vdpa_device *vdpa = v->vdpa; + const struct vdpa_config_ops *ops = vdpa->config; + + if (!ops->resume) + return -EOPNOTSUPP; + + return ops->resume(vdpa); +} + static long vhost_vdpa_vring_ioctl(struct vhost_vdpa *v, unsigned int cmd, void __user *argp) { @@ -687,6 +702,9 @@ static long vhost_vdpa_unlocked_ioctl(struct file *filep, case VHOST_VDPA_SUSPEND: r = vhost_vdpa_suspend(v); break; + case VHOST_VDPA_RESUME: + r = vhost_vdpa_resume(v); + break; default: r = vhost_dev_ioctl(&v->vdev, cmd, argp); if (r == -ENOIOCTLCMD) diff --git a/include/uapi/linux/vhost.h b/include/uapi/linux/vhost.h index f9f115a7c75b..92e1b700b51c 100644 --- a/include/uapi/linux/vhost.h +++ b/include/uapi/linux/vhost.h @@ -180,4 +180,12 @@ */ #define VHOST_VDPA_SUSPEND _IO(VHOST_VIRTIO, 0x7D) +/* Resume a device so it can resume processing virtqueue requests + * + * After the return of this ioctl the device will have restored all the + * necessary states and it is fully operational to continue processing the + * virtqueue descriptors. + */ +#define VHOST_VDPA_RESUME _IO(VHOST_VIRTIO, 0x7E) + #endif From patchwork Tue Jan 3 10:51:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Boeuf, Sebastien" X-Patchwork-Id: 38372 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp4548347wrt; Tue, 3 Jan 2023 02:53:21 -0800 (PST) X-Google-Smtp-Source: AMrXdXsHM4VOT3I9uNmiIhNhV989V7A/lw64p65++9h/4NqWvzyZvD6XhAXVlo1ufSCHjlxmYtUC X-Received: by 2002:a17:90a:f614:b0:219:672a:42db with SMTP id bw20-20020a17090af61400b00219672a42dbmr53310422pjb.19.1672743201304; Tue, 03 Jan 2023 02:53:21 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1672743201; cv=none; d=google.com; s=arc-20160816; b=dUbu6iz50acaY4Mvca5ngagzN0xCYg/qCqt9Z00Yab2pi2m1aVd9gwJYgK8XBwl8Qf u6uDD6sOSRRz+Hiaq0kVyuVTBHwtjOGT2kxGpOxrVmPcvnHvlXALo5GuOR3VBK5s3+Hu 63z4KiNR+9IubFYre6vzDSCCKuDrIxv4/EeluFQt8rjPioxDG9Z7E6lCmcz+GxXo9tJA D3TtP4Em2RnWIJy0ixgCyT3mMWwOiUtl2+KxPeE3j9MXrZtaQ+a6ojRX6xrFBWLsverE i+iL4XvPI7DbwKJ67CaEIi9bUUJf2TX0sCknQcNiDb42PHcHt+dfS+ZPo2zV/BOQh/al M0lg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=32OSybmrWfqPTJ78P0BNvontY2SJxNux0lTZ7d3rQww=; b=IC3cCkF8bU5hk/rZFoDtazQXlxFtizpNjp1bnkTpYWBuX31MbwFLs0yFBVqkP0dE+R OpqEdJNlbHcPBa7oQjkWtB6TigXzQB/3QT0yy6zMRUnCVlOed6iv7j6qlSqEiMtgdj5E VVs6gFnKf+EdCYL8mORb3aD3vRfQMNI8lKTh5viyZEL2bGQ9kKzmrxq67TTzrL11Xtau 9rZMGOFNhJh1Vnj/5KvzFC2P509XrZOnwyormj/JZZw1fLMPlTE/gG0Fh+fsjmGv0qP8 rRoMhjMVTR9sGWXKx4vWrySvbge0TI2ZykyI/rzxheSWdWVtkerdGH9ajWOsIvFvGD9j hFIA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=ddfmerVj; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id k23-20020a63ba17000000b004626957c3c7si33882568pgf.193.2023.01.03.02.53.05; Tue, 03 Jan 2023 02:53:21 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=ddfmerVj; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237449AbjACKwO (ORCPT + 99 others); Tue, 3 Jan 2023 05:52:14 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39584 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237350AbjACKvq (ORCPT ); Tue, 3 Jan 2023 05:51:46 -0500 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C67B5FFC for ; Tue, 3 Jan 2023 02:51:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1672743105; x=1704279105; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=lmnHqI3tKK38BSUrN2DgjBvLt72f0TuIKDBVQU/rvhw=; b=ddfmerVjrE7+0suayq7UkX2tHTfDp5ezOWMIB5e1Rye1oVg2/Q7+Uv1M k/Ja6KmucxqmJd1UheCeOHBnDGeG1Al4jXcUBxgZnbmBh2qK8DrpMOzJt YC5hm3WbPhIHXGfQd9lTK2/9Zt69CXnAvIvCzxOqk1ClEzIo6NILmdkbG Y3oNjACCZQKRGFYWIzOavOG90Vl04T0FhHSOo9F3d+/RapiWkA4AxiPa0 nftFlUt6R3ei3B3VSOW1YrqgtdFuzPaa8Z4QA1dE8r/+cPYQRjaoNM3eh +pmFdPXyu07VUMy+Q84pucO2g2bnjVrxdO7ux26102vAsLQnQYh+HnHUX A==; X-IronPort-AV: E=McAfee;i="6500,9779,10578"; a="301318478" X-IronPort-AV: E=Sophos;i="5.96,296,1665471600"; d="scan'208";a="301318478" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Jan 2023 02:51:45 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10578"; a="604772731" X-IronPort-AV: E=Sophos;i="5.96,296,1665471600"; d="scan'208";a="604772731" Received: from isobansk-mobl1.ger.corp.intel.com (HELO sboeuf-mobl.home) ([10.252.24.246]) by orsmga003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Jan 2023 02:51:43 -0800 From: sebastien.boeuf@intel.com To: linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org Cc: mst@redhat.com, jasowang@redhat.com, eperezma@redhat.com, sebastien.boeuf@intel.com Subject: [PATCH v6 4/4] vdpa_sim: Implement resume vdpa op Date: Tue, 3 Jan 2023 11:51:08 +0100 Message-Id: <15a4566826033c5dd9a2167e5cfb0ef4d90cea49.1672742878.git.sebastien.boeuf@intel.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,LOTS_OF_MONEY, RCVD_IN_DNSWL_MED,SPF_HELO_PASS,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1753998375183538014?= X-GMAIL-MSGID: =?utf-8?q?1753998375183538014?= From: Sebastien Boeuf Implement resume operation for vdpa_sim devices, so vhost-vdpa will offer that backend feature and userspace can effectively resume the device. Signed-off-by: Sebastien Boeuf Acked-by: Jason Wang --- drivers/vdpa/vdpa_sim/vdpa_sim.c | 29 +++++++++++++++++++++++++++++ drivers/vdpa/vdpa_sim/vdpa_sim.h | 1 + 2 files changed, 30 insertions(+) diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.c b/drivers/vdpa/vdpa_sim/vdpa_sim.c index b071f0d842fb..756a5db0109c 100644 --- a/drivers/vdpa/vdpa_sim/vdpa_sim.c +++ b/drivers/vdpa/vdpa_sim/vdpa_sim.c @@ -357,6 +357,12 @@ static void vdpasim_kick_vq(struct vdpa_device *vdpa, u16 idx) struct vdpasim *vdpasim = vdpa_to_sim(vdpa); struct vdpasim_virtqueue *vq = &vdpasim->vqs[idx]; + if (!vdpasim->running && + (vdpasim->status & VIRTIO_CONFIG_S_DRIVER_OK)) { + vdpasim->pending_kick = true; + return; + } + if (vq->ready) schedule_work(&vdpasim->work); } @@ -527,6 +533,27 @@ static int vdpasim_suspend(struct vdpa_device *vdpa) return 0; } +static int vdpasim_resume(struct vdpa_device *vdpa) +{ + struct vdpasim *vdpasim = vdpa_to_sim(vdpa); + int i; + + spin_lock(&vdpasim->lock); + vdpasim->running = true; + + if (vdpasim->pending_kick) { + /* Process pending descriptors */ + for (i = 0; i < vdpasim->dev_attr.nvqs; ++i) + vdpasim_kick_vq(vdpa, i); + + vdpasim->pending_kick = false; + } + + spin_unlock(&vdpasim->lock); + + return 0; +} + static size_t vdpasim_get_config_size(struct vdpa_device *vdpa) { struct vdpasim *vdpasim = vdpa_to_sim(vdpa); @@ -717,6 +744,7 @@ static const struct vdpa_config_ops vdpasim_config_ops = { .set_status = vdpasim_set_status, .reset = vdpasim_reset, .suspend = vdpasim_suspend, + .resume = vdpasim_resume, .get_config_size = vdpasim_get_config_size, .get_config = vdpasim_get_config, .set_config = vdpasim_set_config, @@ -750,6 +778,7 @@ static const struct vdpa_config_ops vdpasim_batch_config_ops = { .set_status = vdpasim_set_status, .reset = vdpasim_reset, .suspend = vdpasim_suspend, + .resume = vdpasim_resume, .get_config_size = vdpasim_get_config_size, .get_config = vdpasim_get_config, .set_config = vdpasim_set_config, diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.h b/drivers/vdpa/vdpa_sim/vdpa_sim.h index 0e78737dcc16..a745605589e2 100644 --- a/drivers/vdpa/vdpa_sim/vdpa_sim.h +++ b/drivers/vdpa/vdpa_sim/vdpa_sim.h @@ -67,6 +67,7 @@ struct vdpasim { u64 features; u32 groups; bool running; + bool pending_kick; /* spinlock to synchronize iommu table */ spinlock_t iommu_lock; };