From patchwork Wed Nov 23 11:32:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shameerali Kolothum Thodi X-Patchwork-Id: 24927 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp2740725wrr; Wed, 23 Nov 2022 03:57:15 -0800 (PST) X-Google-Smtp-Source: AA0mqf5fiahyxSUSkvvcVR7Usw5jlKD/nF2a/6nksCXUdogEFXTD6ee+3sK4t6H4EcuhZSFMgUIZ X-Received: by 2002:a63:5409:0:b0:476:e3bb:2340 with SMTP id i9-20020a635409000000b00476e3bb2340mr7097922pgb.530.1669204635180; Wed, 23 Nov 2022 03:57:15 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669204635; cv=none; d=google.com; s=arc-20160816; b=Ww6HP8flN4fbZzDhVTusokQn8xZJIxWHY3nCyMUx6XZTcAg+U5QCffgJTpFYnB3A9Y ntW6WvQCNTkXOKf4a/VZRJNess1sHqYo3+hzxalV3a7I+4PsQRpo1ujrN1IflNocHt4f aKhtQKlaR0E/S1Ye8e9314rwQl62YWdNjWceeZTAwfTx6BxGdoxEqITBW3bsCtN5Xv87 Db6MEYsp8Z8i8hc1Ot4VfTNgGczsxsOSivqk4Y1gPfEVwmTiOLmgPl+31mAVvDwzRNLF TvTspUQo07PIQfuTomIweGiI/CsLx3jInmPprZGQfMpsCA/+dolMnzy0AWN3jN5OFCrU U1HA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from; bh=+RAqInh9xHaZvhZLR219kgbCB+hyiVnaCO5wdSGafOw=; b=umQdumeOJ/tNI9U1TMOkdGwEpBj+lipsqeR6N89aVqe17gp23mbysd/6IVGjFpwMhX RsTb/dnV6eL1RbSANj1hD9h13rPqfZOWoRLHY5K01hCBicXGrCf+JRZ2HrwhTnGjbVp8 Zo8UnRMTl9mhK5ui0B7npmSMQl8QmPXsquspX6OZs0tKugx7enQNn5pfI+4mew6ZCR4J k2ew9B0WdFW8boMRQceijaME9oashuDnSsjn3q8AfZsoCwmRs4juaDPMtdrJOGaT/fju swahYyGGW67Pe2eBhoDzgyU7Npe6Jmnzq/IUZmL9x+RxJ5cVo7uYYYsFb7Pi/yoUBKLz awJw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id w7-20020a639347000000b0046ec0715bd9si4521569pgm.818.2022.11.23.03.57.01; Wed, 23 Nov 2022 03:57:15 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236972AbiKWLdd (ORCPT + 99 others); Wed, 23 Nov 2022 06:33:33 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42824 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237204AbiKWLdT (ORCPT ); Wed, 23 Nov 2022 06:33:19 -0500 Received: from frasgout.his.huawei.com (frasgout.his.huawei.com [185.176.79.56]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F0A8A5CD28; Wed, 23 Nov 2022 03:33:14 -0800 (PST) Received: from frapeml100006.china.huawei.com (unknown [172.18.147.200]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4NHJpQ460Dz67JVF; Wed, 23 Nov 2022 19:30:38 +0800 (CST) Received: from lhrpeml500005.china.huawei.com (7.191.163.240) by frapeml100006.china.huawei.com (7.182.85.201) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Wed, 23 Nov 2022 12:33:13 +0100 Received: from A2006125610.china.huawei.com (10.202.227.178) by lhrpeml500005.china.huawei.com (7.191.163.240) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Wed, 23 Nov 2022 11:33:07 +0000 From: Shameer Kolothum To: , CC: , , , , , , Subject: [PATCH 1/4] hisi_acc_vfio_pci: Add support for precopy IOCTL Date: Wed, 23 Nov 2022 11:32:33 +0000 Message-ID: <20221123113236.896-2-shameerali.kolothum.thodi@huawei.com> X-Mailer: git-send-email 2.12.0.windows.1 In-Reply-To: <20221123113236.896-1-shameerali.kolothum.thodi@huawei.com> References: <20221123113236.896-1-shameerali.kolothum.thodi@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.202.227.178] X-ClientProxiedBy: dggems702-chm.china.huawei.com (10.3.19.179) To lhrpeml500005.china.huawei.com (7.191.163.240) X-CFilter-Loop: Reflected X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750287919592217510?= X-GMAIL-MSGID: =?utf-8?q?1750287919592217510?= PRECOPY IOCTL in the case of HiSiIicon ACC driver can be used to perform the device compatibility check earlier during migration. Signed-off-by: Shameer Kolothum --- .../vfio/pci/hisilicon/hisi_acc_vfio_pci.c | 52 +++++++++++++++++++ .../vfio/pci/hisilicon/hisi_acc_vfio_pci.h | 1 + 2 files changed, 53 insertions(+) diff --git a/drivers/vfio/pci/hisilicon/hisi_acc_vfio_pci.c b/drivers/vfio/pci/hisilicon/hisi_acc_vfio_pci.c index 0c0c0c7f0521..f3b74a06edb6 100644 --- a/drivers/vfio/pci/hisilicon/hisi_acc_vfio_pci.c +++ b/drivers/vfio/pci/hisilicon/hisi_acc_vfio_pci.c @@ -764,9 +764,58 @@ hisi_acc_vf_pci_resume(struct hisi_acc_vf_core_device *hisi_acc_vdev) stream_open(migf->filp->f_inode, migf->filp); mutex_init(&migf->lock); + migf->hisi_acc_vdev = hisi_acc_vdev; return migf; } +static long hisi_acc_vf_precopy_ioctl(struct file *filp, + unsigned int cmd, unsigned long arg) +{ + struct hisi_acc_vf_migration_file *migf = filp->private_data; + struct hisi_acc_vf_core_device *hisi_acc_vdev = migf->hisi_acc_vdev; + loff_t *pos = &filp->f_pos; + struct vfio_precopy_info info; + unsigned long minsz; + int ret; + + if (cmd != VFIO_MIG_GET_PRECOPY_INFO) + return -ENOTTY; + + minsz = offsetofend(struct vfio_precopy_info, dirty_bytes); + + if (copy_from_user(&info, (void __user *)arg, minsz)) + return -EFAULT; + if (info.argsz < minsz) + return -EINVAL; + + mutex_lock(&hisi_acc_vdev->state_mutex); + if (hisi_acc_vdev->mig_state != VFIO_DEVICE_STATE_PRE_COPY) { + mutex_unlock(&hisi_acc_vdev->state_mutex); + return -EINVAL; + } + + mutex_lock(&migf->lock); + + if (migf->disabled) { + ret = -ENODEV; + goto out; + } + + if (*pos > migf->total_length) { + ret = -EINVAL; + goto out; + } + + info.dirty_bytes = 0; + info.initial_bytes = migf->total_length - *pos; + + ret = copy_to_user((void __user *)arg, &info, minsz) ? -EFAULT : 0; +out: + mutex_unlock(&migf->lock); + mutex_unlock(&hisi_acc_vdev->state_mutex); + return ret; +} + static ssize_t hisi_acc_vf_save_read(struct file *filp, char __user *buf, size_t len, loff_t *pos) { @@ -807,6 +856,8 @@ static ssize_t hisi_acc_vf_save_read(struct file *filp, char __user *buf, size_t static const struct file_operations hisi_acc_vf_save_fops = { .owner = THIS_MODULE, .read = hisi_acc_vf_save_read, + .unlocked_ioctl = hisi_acc_vf_precopy_ioctl, + .compat_ioctl = compat_ptr_ioctl, .release = hisi_acc_vf_release_file, .llseek = no_llseek, }; @@ -832,6 +883,7 @@ hisi_acc_vf_stop_copy(struct hisi_acc_vf_core_device *hisi_acc_vdev) stream_open(migf->filp->f_inode, migf->filp); mutex_init(&migf->lock); + migf->hisi_acc_vdev = hisi_acc_vdev; ret = vf_qm_state_save(hisi_acc_vdev, migf); if (ret) { diff --git a/drivers/vfio/pci/hisilicon/hisi_acc_vfio_pci.h b/drivers/vfio/pci/hisilicon/hisi_acc_vfio_pci.h index 67343325b320..11d51345f5b5 100644 --- a/drivers/vfio/pci/hisilicon/hisi_acc_vfio_pci.h +++ b/drivers/vfio/pci/hisilicon/hisi_acc_vfio_pci.h @@ -91,6 +91,7 @@ struct hisi_acc_vf_migration_file { struct mutex lock; bool disabled; + struct hisi_acc_vf_core_device *hisi_acc_vdev; struct acc_vf_data vf_data; size_t total_length; }; From patchwork Wed Nov 23 11:32:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shameerali Kolothum Thodi X-Patchwork-Id: 24925 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp2735975wrr; Wed, 23 Nov 2022 03:43:48 -0800 (PST) X-Google-Smtp-Source: AA0mqf6vkgiSIZmlTheNsoupRAWHCXimIipyeZW6Ga/TGIauDkh0C3h5OBo893sgjFINOrvmIvR6 X-Received: by 2002:a63:b16:0:b0:46e:c052:a6b0 with SMTP id 22-20020a630b16000000b0046ec052a6b0mr7149431pgl.91.1669203828316; Wed, 23 Nov 2022 03:43:48 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669203828; cv=none; d=google.com; s=arc-20160816; b=ACqVkQf82mmRisVG+0EiaPP7v+pR9LdVzX54G4cGKVzJ8jQ1QfR2cWItmPYqGtDyaC LEAkz0ioJX+TQ/h0BYRDkHhl3QnPL1k9/fOobkS02V6pwJYwMaOHThKqdfI73cpr0svo BSlcm9g3mS0Ni1wZg28A87CO+F5Gbn335bwND4k9HzFThzsrXfTA1pKd6EgaQkJmdq8l 8i7ojGPsYfmlzp1BxY8IUBD2XU2v/z1y2lJJxCnBRovD4STQkuSv7OWHSxDNvvWkapxX vFait97Ew2BmpZICSvNit0vVqWoRrD6FnEJssVeWwVjsXKEO11L5ZmaA/+HEETBN7YSX FE/g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from; bh=9qc4QD+xTuJ8ctFKrkmV4JgdqX4vb85R8NVaIlD75M0=; b=MU9QZBA5MFI3xBgfFLRIKj+FDY1I9XwtNHo73QqrlWTMAkEKJaURH0aI3p2f8pZi9A 6qFkMpMtpS06/B/ytxYoG0xj/wLzuYL3KC0ttBLwvuJ5w7IzdFjjYzTTUad6zZpE1Eu2 uCZjMGiIsTzHJqDhMrW2yNTh3NtRAIrFX2Xy0IfJ7pFEIRKrMXcINLgYEYUGZ7BpXHoZ 5TBjnrtLuk4Lof12zniIYnD/7syyBfeWGxvS9nBPI59si5/8shJn+6Jcsocw5/C1XUGH vSGGKvdvmscNIii++hLR8jR0KCIhpd2ylQ7sMQX4g1TfJqBfRMMEIl95jwFwSuYOlQLx U8aQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id e11-20020a170902cf4b00b0017d2bb81273si16345860plg.600.2022.11.23.03.43.34; Wed, 23 Nov 2022 03:43:48 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237076AbiKWLdi (ORCPT + 99 others); Wed, 23 Nov 2022 06:33:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42746 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236693AbiKWLdW (ORCPT ); Wed, 23 Nov 2022 06:33:22 -0500 Received: from frasgout.his.huawei.com (frasgout.his.huawei.com [185.176.79.56]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0C6CC27CD1; Wed, 23 Nov 2022 03:33:22 -0800 (PST) Received: from frapeml100005.china.huawei.com (unknown [172.18.147.200]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4NHJpY4mrpz6889H; Wed, 23 Nov 2022 19:30:45 +0800 (CST) Received: from lhrpeml500005.china.huawei.com (7.191.163.240) by frapeml100005.china.huawei.com (7.182.85.132) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Wed, 23 Nov 2022 12:33:20 +0100 Received: from A2006125610.china.huawei.com (10.202.227.178) by lhrpeml500005.china.huawei.com (7.191.163.240) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Wed, 23 Nov 2022 11:33:15 +0000 From: Shameer Kolothum To: , CC: , , , , , , Subject: [PATCH 2/4] hisi_acc_vfio_pci: Introduce support for PRE_COPY state transitions Date: Wed, 23 Nov 2022 11:32:34 +0000 Message-ID: <20221123113236.896-3-shameerali.kolothum.thodi@huawei.com> X-Mailer: git-send-email 2.12.0.windows.1 In-Reply-To: <20221123113236.896-1-shameerali.kolothum.thodi@huawei.com> References: <20221123113236.896-1-shameerali.kolothum.thodi@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.202.227.178] X-ClientProxiedBy: dggems702-chm.china.huawei.com (10.3.19.179) To lhrpeml500005.china.huawei.com (7.191.163.240) X-CFilter-Loop: Reflected X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750287073789914505?= X-GMAIL-MSGID: =?utf-8?q?1750287073789914505?= The saving_migf is open in PRE_COPY state if it is supported and reads initial device match data. hisi_acc_vf_stop_copy() is refactored to make use of common code. Signed-off-by: Shameer Kolothum --- .../vfio/pci/hisilicon/hisi_acc_vfio_pci.c | 74 ++++++++++++++++++- 1 file changed, 71 insertions(+), 3 deletions(-) diff --git a/drivers/vfio/pci/hisilicon/hisi_acc_vfio_pci.c b/drivers/vfio/pci/hisilicon/hisi_acc_vfio_pci.c index f3b74a06edb6..c8658636a84c 100644 --- a/drivers/vfio/pci/hisilicon/hisi_acc_vfio_pci.c +++ b/drivers/vfio/pci/hisilicon/hisi_acc_vfio_pci.c @@ -863,7 +863,7 @@ static const struct file_operations hisi_acc_vf_save_fops = { }; static struct hisi_acc_vf_migration_file * -hisi_acc_vf_stop_copy(struct hisi_acc_vf_core_device *hisi_acc_vdev) +hisi_acc_open_saving_migf(struct hisi_acc_vf_core_device *hisi_acc_vdev) { struct hisi_acc_vf_migration_file *migf; int ret; @@ -885,7 +885,7 @@ hisi_acc_vf_stop_copy(struct hisi_acc_vf_core_device *hisi_acc_vdev) mutex_init(&migf->lock); migf->hisi_acc_vdev = hisi_acc_vdev; - ret = vf_qm_state_save(hisi_acc_vdev, migf); + ret = vf_qm_get_match_data(hisi_acc_vdev, &migf->vf_data); if (ret) { fput(migf->filp); return ERR_PTR(ret); @@ -894,6 +894,44 @@ hisi_acc_vf_stop_copy(struct hisi_acc_vf_core_device *hisi_acc_vdev) return migf; } +static struct hisi_acc_vf_migration_file * +hisi_acc_vf_pre_copy(struct hisi_acc_vf_core_device *hisi_acc_vdev) +{ + struct hisi_acc_vf_migration_file *migf; + + migf = hisi_acc_open_saving_migf(hisi_acc_vdev); + if (IS_ERR(migf)) + return migf; + + migf->total_length = QM_MATCH_SIZE; + return migf; +} + +static struct hisi_acc_vf_migration_file * +hisi_acc_vf_stop_copy(struct hisi_acc_vf_core_device *hisi_acc_vdev, bool open) +{ + int ret; + struct hisi_acc_vf_migration_file *migf = NULL; + + if (open) { + /* + * Userspace didn't use PRECOPY support. Hence saving_migf + * is not opened yet. + */ + migf = hisi_acc_open_saving_migf(hisi_acc_vdev); + if (IS_ERR(migf)) + return migf; + } else { + migf = hisi_acc_vdev->saving_migf; + } + + ret = vf_qm_state_save(hisi_acc_vdev, migf); + if (ret) + return ERR_PTR(ret); + + return open ? migf : NULL; +} + static int hisi_acc_vf_stop_device(struct hisi_acc_vf_core_device *hisi_acc_vdev) { struct device *dev = &hisi_acc_vdev->vf_dev->dev; @@ -921,6 +959,31 @@ hisi_acc_vf_set_device_state(struct hisi_acc_vf_core_device *hisi_acc_vdev, u32 cur = hisi_acc_vdev->mig_state; int ret; + if (cur == VFIO_DEVICE_STATE_RUNNING && new == VFIO_DEVICE_STATE_PRE_COPY) { + struct hisi_acc_vf_migration_file *migf; + + migf = hisi_acc_vf_pre_copy(hisi_acc_vdev); + if (IS_ERR(migf)) + return ERR_CAST(migf); + get_file(migf->filp); + hisi_acc_vdev->saving_migf = migf; + return migf->filp; + } + + if (cur == VFIO_DEVICE_STATE_PRE_COPY && new == VFIO_DEVICE_STATE_STOP_COPY) { + struct hisi_acc_vf_migration_file *migf; + + ret = hisi_acc_vf_stop_device(hisi_acc_vdev); + if (ret) + return ERR_PTR(ret); + + migf = hisi_acc_vf_stop_copy(hisi_acc_vdev, false); + if (IS_ERR(migf)) + return ERR_CAST(migf); + + return NULL; + } + if (cur == VFIO_DEVICE_STATE_RUNNING && new == VFIO_DEVICE_STATE_STOP) { ret = hisi_acc_vf_stop_device(hisi_acc_vdev); if (ret) @@ -931,7 +994,7 @@ hisi_acc_vf_set_device_state(struct hisi_acc_vf_core_device *hisi_acc_vdev, if (cur == VFIO_DEVICE_STATE_STOP && new == VFIO_DEVICE_STATE_STOP_COPY) { struct hisi_acc_vf_migration_file *migf; - migf = hisi_acc_vf_stop_copy(hisi_acc_vdev); + migf = hisi_acc_vf_stop_copy(hisi_acc_vdev, true); if (IS_ERR(migf)) return ERR_CAST(migf); get_file(migf->filp); @@ -963,6 +1026,11 @@ hisi_acc_vf_set_device_state(struct hisi_acc_vf_core_device *hisi_acc_vdev, return NULL; } + if (cur == VFIO_DEVICE_STATE_PRE_COPY && new == VFIO_DEVICE_STATE_RUNNING) { + hisi_acc_vf_disable_fds(hisi_acc_vdev); + return NULL; + } + if (cur == VFIO_DEVICE_STATE_STOP && new == VFIO_DEVICE_STATE_RUNNING) { hisi_acc_vf_start_device(hisi_acc_vdev); return NULL; From patchwork Wed Nov 23 11:32:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shameerali Kolothum Thodi X-Patchwork-Id: 24928 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp2741457wrr; Wed, 23 Nov 2022 03:59:23 -0800 (PST) X-Google-Smtp-Source: AA0mqf7Bsu1ZNbH67+nmuIRw1rDg1wWywgqChY/R2P045iJ8sRV0yc7VEnDlSr4qih3GyIeeCblJ X-Received: by 2002:a17:902:ebcc:b0:181:896c:2337 with SMTP id p12-20020a170902ebcc00b00181896c2337mr8393737plg.74.1669204763578; Wed, 23 Nov 2022 03:59:23 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669204763; cv=none; d=google.com; s=arc-20160816; b=DOpPpQRRDUlnaP22hzH65p5q/ly+miaTfM+CP5rkU9xA0CTSNDlLEid4LMdgQVTiMX vlnCjzaaPs67I2jvB5aqB3ExkBSKe44hpRES3E3N+1QHYqeWdZ7241BXSWwLuu9Yp3qM FtULsvj8tpnE/8i67AGkaBDYauyfAHFb5O6ORIQsLUo+NjzmGqnlzsJK8qyo8QuHyW4H 2HskTmV+MYkKBfcBYVLaV0Gze1YZm1gP9P/NJuteAIoQviW4yBcbpaHkUD/2j5jctq8q NKKuYflLOyaT81gQRM3Ax4ofda06jvLw2GUBYHU1uNGnHwDIhSLzHYaTDWgFBNEigl6T tE+w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from; bh=Vzi+NYxIMaJhY+FHrJ8CyI80sP1IWC95O/RT02lAzRE=; b=DbioFs+6wW1MHgl1EBx6Niue9ChkhavobdULxf+m3IH21ENun8h6WinCtHD/kvih1y 8ecyh49f8zrolS3tOKPEvEt5JL0VOgCLxOmV6FPshvifqcSqTgaxFHsHmEAhJePbFHJe D+nS+dINW6K03EGEQ2IRvGQUpFRmxP0YzskmDt2tQH+DTw8zOs1ueaGWvqcso4psm0cF cNtp/oGODWvq3MmujM8fFV4nrlxoNIUEtV6i5D9FhrOG5jQPVJNGdDnQO5HG/Su2WBt+ qiK1TaX2D4IjUQn64VtOfbalYFXB2+q9i2tt/UUCPKx3VeT6VzeXU6IXxIcoel7TO5je zMQg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id t11-20020a63eb0b000000b00476bfca3909si725279pgh.792.2022.11.23.03.59.09; Wed, 23 Nov 2022 03:59:23 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237244AbiKWLdn (ORCPT + 99 others); Wed, 23 Nov 2022 06:33:43 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43128 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236962AbiKWLdb (ORCPT ); Wed, 23 Nov 2022 06:33:31 -0500 Received: from frasgout.his.huawei.com (frasgout.his.huawei.com [185.176.79.56]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9F62285A1D; Wed, 23 Nov 2022 03:33:29 -0800 (PST) Received: from frapeml100002.china.huawei.com (unknown [172.18.147.200]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4NHJpj19r3z6H7DX; Wed, 23 Nov 2022 19:30:53 +0800 (CST) Received: from lhrpeml500005.china.huawei.com (7.191.163.240) by frapeml100002.china.huawei.com (7.182.85.26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Wed, 23 Nov 2022 12:33:27 +0100 Received: from A2006125610.china.huawei.com (10.202.227.178) by lhrpeml500005.china.huawei.com (7.191.163.240) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Wed, 23 Nov 2022 11:33:22 +0000 From: Shameer Kolothum To: , CC: , , , , , , Subject: [PATCH 3/4] hisi_acc_vfio_pci: Move the dev compatibility tests for early check Date: Wed, 23 Nov 2022 11:32:35 +0000 Message-ID: <20221123113236.896-4-shameerali.kolothum.thodi@huawei.com> X-Mailer: git-send-email 2.12.0.windows.1 In-Reply-To: <20221123113236.896-1-shameerali.kolothum.thodi@huawei.com> References: <20221123113236.896-1-shameerali.kolothum.thodi@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.202.227.178] X-ClientProxiedBy: dggems702-chm.china.huawei.com (10.3.19.179) To lhrpeml500005.china.huawei.com (7.191.163.240) X-CFilter-Loop: Reflected X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750288053770790797?= X-GMAIL-MSGID: =?utf-8?q?1750288053770790797?= Instead of waiting till data transfer is complete to perform dev compatibility, do it as soon as we have enough data to perform the check. This will be useful when we enable the support for PRE_COPY. Signed-off-by: Shameer Kolothum --- .../vfio/pci/hisilicon/hisi_acc_vfio_pci.c | 19 +++++++------------ .../vfio/pci/hisilicon/hisi_acc_vfio_pci.h | 1 + 2 files changed, 8 insertions(+), 12 deletions(-) diff --git a/drivers/vfio/pci/hisilicon/hisi_acc_vfio_pci.c b/drivers/vfio/pci/hisilicon/hisi_acc_vfio_pci.c index c8658636a84c..9a51f41e1d2a 100644 --- a/drivers/vfio/pci/hisilicon/hisi_acc_vfio_pci.c +++ b/drivers/vfio/pci/hisilicon/hisi_acc_vfio_pci.c @@ -360,8 +360,8 @@ static int vf_qm_check_match(struct hisi_acc_vf_core_device *hisi_acc_vdev, u32 que_iso_state; int ret; - if (migf->total_length < QM_MATCH_SIZE) - return -EINVAL; + if (migf->total_length < QM_MATCH_SIZE || hisi_acc_vdev->match_done) + return 0; if (vf_data->acc_magic != ACC_DEV_MAGIC) { dev_err(dev, "failed to match ACC_DEV_MAGIC\n"); @@ -406,6 +406,7 @@ static int vf_qm_check_match(struct hisi_acc_vf_core_device *hisi_acc_vdev, } hisi_acc_vdev->vf_qm_state = vf_data->vf_qm_state; + hisi_acc_vdev->match_done = true; return 0; } @@ -493,10 +494,6 @@ static int vf_qm_state_save(struct hisi_acc_vf_core_device *hisi_acc_vdev, struct device *dev = &vf_qm->pdev->dev; int ret; - ret = vf_qm_get_match_data(hisi_acc_vdev, vf_data); - if (ret) - return ret; - if (unlikely(qm_wait_dev_not_ready(vf_qm))) { /* Update state and return with match data */ vf_data->vf_qm_state = QM_NOT_READY; @@ -673,12 +670,6 @@ static int hisi_acc_vf_load_state(struct hisi_acc_vf_core_device *hisi_acc_vdev) struct hisi_acc_vf_migration_file *migf = hisi_acc_vdev->resuming_migf; int ret; - /* Check dev compatibility */ - ret = vf_qm_check_match(hisi_acc_vdev, migf); - if (ret) { - dev_err(dev, "failed to match the VF!\n"); - return ret; - } /* Recover data to VF */ ret = vf_qm_load_data(hisi_acc_vdev, migf); if (ret) { @@ -732,6 +723,10 @@ static ssize_t hisi_acc_vf_resume_write(struct file *filp, const char __user *bu *pos += len; done = len; migf->total_length += len; + + ret = vf_qm_check_match(migf->hisi_acc_vdev, migf); + if (ret) + done = -EFAULT; out_unlock: mutex_unlock(&migf->lock); return done; diff --git a/drivers/vfio/pci/hisilicon/hisi_acc_vfio_pci.h b/drivers/vfio/pci/hisilicon/hisi_acc_vfio_pci.h index 11d51345f5b5..dcabfeec6ca1 100644 --- a/drivers/vfio/pci/hisilicon/hisi_acc_vfio_pci.h +++ b/drivers/vfio/pci/hisilicon/hisi_acc_vfio_pci.h @@ -98,6 +98,7 @@ struct hisi_acc_vf_migration_file { struct hisi_acc_vf_core_device { struct vfio_pci_core_device core_device; + u8 match_done:1; u8 deferred_reset:1; /* For migration state */ struct mutex state_mutex; From patchwork Wed Nov 23 11:32:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shameerali Kolothum Thodi X-Patchwork-Id: 24926 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp2736661wrr; Wed, 23 Nov 2022 03:45:44 -0800 (PST) X-Google-Smtp-Source: AA0mqf4WubDuLp02VZTPYhGPQtlBZqwP+hqINGzs4W3OVt5aRRYH/aI6pj8Cg6Job0BRXRVoDHdI X-Received: by 2002:a17:902:ba89:b0:189:385f:b7f5 with SMTP id k9-20020a170902ba8900b00189385fb7f5mr5734436pls.69.1669203944484; Wed, 23 Nov 2022 03:45:44 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669203944; cv=none; d=google.com; s=arc-20160816; b=tvMeP15D2ZuEmNqxNEaLxz9qu19fR9IHde/3PHw2FduMtFOFPMTgjMrfDHXKnc9fcd mUXV34rtOkHDv22NeCeAK3d1GfR9VNprvV8WLIpOMoidMhVqp/iXITSsVP3CeJ2Yh6bw 6n9RKVptLaj7YCBBcM3qDn4ZxAn/2liGvmJxYel6MKWsrHbgb3lqDv+S6iO87sAe1RMv +siB2XQwzLVwZoBbVMGMGgWPGYqnK9dNZuNkR5tUPS66ojGKu1NqSJ8VmycguXuEK6I/ A9llruA/FsuwK3sKOV8MTEavmXr4AIfco/YLOVtWhjjhn0RMDFSzrCsZKUQPVgkzeqgJ FjQQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from; bh=NVB0uSvDiCnb5Fh4Dnx0NSlAlx7e+Hyz9NlOSGnCKYo=; b=b70ivip3jwryc61HtDMAG9nF0CSQKExj/U9LPEa3o4Nabb2ldcvPxBHdgsf/+gR6r+ GSdF0FrZVcHFp1vAGmNH42HYYhIHBc0lX9b4It+/bYWitP1Gku8vdxxld3JU1H0mKf7M vYVo9YHcxbnOFp+THqF4fv1R9KHRjVXnJpoK4FDk6x1F3xc0uMqys+o4v2da9SnrTaZI PdlTWcDzWM90dWwc7lurTGkK0veVmASKwF+Qd5HyRzQhp+GcZlshc15iira6VRK7WNGA uEG4l3KsDrWRPqRciGjXiYHbmou7UiBOh1kyd7XgT67rExDz1aLIlmXmXNvG6TkiY18d JPSg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id g19-20020a056a0023d300b0056beba0b31dsi18344253pfc.8.2022.11.23.03.45.30; Wed, 23 Nov 2022 03:45:44 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237444AbiKWLeA (ORCPT + 99 others); Wed, 23 Nov 2022 06:34:00 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44372 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237201AbiKWLdv (ORCPT ); Wed, 23 Nov 2022 06:33:51 -0500 Received: from frasgout.his.huawei.com (frasgout.his.huawei.com [185.176.79.56]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6B39011DA22; Wed, 23 Nov 2022 03:33:37 -0800 (PST) Received: from fraeml743-chm.china.huawei.com (unknown [172.18.147.200]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4NHJph0vRSz6HJYL; Wed, 23 Nov 2022 19:30:52 +0800 (CST) Received: from lhrpeml500005.china.huawei.com (7.191.163.240) by fraeml743-chm.china.huawei.com (10.206.15.224) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Wed, 23 Nov 2022 12:33:35 +0100 Received: from A2006125610.china.huawei.com (10.202.227.178) by lhrpeml500005.china.huawei.com (7.191.163.240) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Wed, 23 Nov 2022 11:33:30 +0000 From: Shameer Kolothum To: , CC: , , , , , , Subject: [PATCH 4/4] hisi_acc_vfio_pci: Enable PRE_COPY flag Date: Wed, 23 Nov 2022 11:32:36 +0000 Message-ID: <20221123113236.896-5-shameerali.kolothum.thodi@huawei.com> X-Mailer: git-send-email 2.12.0.windows.1 In-Reply-To: <20221123113236.896-1-shameerali.kolothum.thodi@huawei.com> References: <20221123113236.896-1-shameerali.kolothum.thodi@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.202.227.178] X-ClientProxiedBy: dggems702-chm.china.huawei.com (10.3.19.179) To lhrpeml500005.china.huawei.com (7.191.163.240) X-CFilter-Loop: Reflected X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750287195528609280?= X-GMAIL-MSGID: =?utf-8?q?1750287195528609280?= Now that we have everything to support the PRE_COPY state, enable it. Signed-off-by: Shameer Kolothum --- drivers/vfio/pci/hisilicon/hisi_acc_vfio_pci.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/vfio/pci/hisilicon/hisi_acc_vfio_pci.c b/drivers/vfio/pci/hisilicon/hisi_acc_vfio_pci.c index 9a51f41e1d2a..51941bb4f31f 100644 --- a/drivers/vfio/pci/hisilicon/hisi_acc_vfio_pci.c +++ b/drivers/vfio/pci/hisilicon/hisi_acc_vfio_pci.c @@ -1351,7 +1351,7 @@ static int hisi_acc_vfio_pci_migrn_init_dev(struct vfio_device *core_vdev) hisi_acc_vdev->vf_dev = pdev; mutex_init(&hisi_acc_vdev->state_mutex); - core_vdev->migration_flags = VFIO_MIGRATION_STOP_COPY; + core_vdev->migration_flags = VFIO_MIGRATION_STOP_COPY | VFIO_MIGRATION_PRE_COPY; core_vdev->mig_ops = &hisi_acc_vfio_pci_migrn_state_ops; return vfio_pci_core_init_dev(core_vdev);