From patchwork Wed Jan 18 16:43:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 45388 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2443184wrn; Wed, 18 Jan 2023 08:50:34 -0800 (PST) X-Google-Smtp-Source: AMrXdXs6kFWxyEi1n2z4GNIzn5iocED1yJr4r+1XjPx2zKiOqKEute0HsDCR0V0TCT/nEJ1LGi40 X-Received: by 2002:a17:902:9891:b0:192:7c38:4842 with SMTP id s17-20020a170902989100b001927c384842mr28442629plp.53.1674060633811; Wed, 18 Jan 2023 08:50:33 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674060633; cv=none; d=google.com; s=arc-20160816; b=qZ+NArQJVbMsXvFVRqWGSt0QG+36hhIQAs1mAtKSrSxnsF3vUouqJfOvkA2Nd0v/zl oGBkTX6NO153MXPlkUhlRt5cH1xtmSNJN8YnSKFiOj6/DOaaU1mMQEd553Qj/RQAiixZ /ormxvwYsv5RBP+moFcqotQk6CyBPLgHI+J8dTdgaYVTlESvUhPNLVAclzRaqG7MCic7 iDM7uJ6Gqt2ebQR7jKA2EmvJeeZw7TO6xmMagbVjKb2y4uHYSDo2XoSZydRdhYXiNdVA J8wjy/W86h+4KGYUdYg/WJzxLsNZXVNfBXW3TiA2YPdfIKWbFQhokzc+KfP+BaehIF9v HHEQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=fY7ZFKCSeA1/hO616eCT/ffy6ZcVRsbG6pt4HMxlg40=; b=iNoRemITcpVrKijJzTWaURMxu/TD85Nt/ZBlHVqyWwLSvIpPwgQ5WzhnVew5cEBXrA T0AbKxLruINgcwMyHFScukZDq9IEFWt+34Op34lURJtIp1IPerxF/UX+4u877j7KMEVo vs3zsisVuUSKnh8MhNJCU0/ialwL+klJYQM8tWeHBxC/3n0sO7pbyoFEFtBF817GI4n7 FKIXbqU18vuYq+2YY1yKT67N3V5jimeGsS/R4GcrWAZxSkz8hXrsCfoJXvmfZm7oRQXy mnUDgniDobYGqmFiBpNd4GY9VKdewv6QidQlfsjS0BcytdOFZ56xCUuwCunoufeBGJmk E8SA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=L8o2q5Pt; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id k7-20020a170902ce0700b00192dbc36b89si10403553plg.232.2023.01.18.08.50.22; Wed, 18 Jan 2023 08:50:33 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=L8o2q5Pt; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230062AbjARQqH (ORCPT + 99 others); Wed, 18 Jan 2023 11:46:07 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48826 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229614AbjARQpg (ORCPT ); Wed, 18 Jan 2023 11:45:36 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7AFC358661 for ; Wed, 18 Jan 2023 08:44:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674060254; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fY7ZFKCSeA1/hO616eCT/ffy6ZcVRsbG6pt4HMxlg40=; b=L8o2q5PtRBhRqyTubSx1OHweuwc5hXIBUqsGUHU2TnVHC8+aNeNkqAS4ICU7WDD/ZwTx1U YIzukUvWq2HGsqREWSv7Mb7Du1k4GL4RuaWVReVA7k5rImSuQy/lB456rurd11fBE/gD+q +/87dZp3FoCRW5IuRvL/XYeoLHQznmw= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-211-CsVoZYk3O1W7FhPQcrc4nQ-1; Wed, 18 Jan 2023 11:44:08 -0500 X-MC-Unique: CsVoZYk3O1W7FhPQcrc4nQ-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 2287B1871DA4; Wed, 18 Jan 2023 16:44:06 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.141]) by smtp.corp.redhat.com (Postfix) with ESMTP id E7DC2140EBF6; Wed, 18 Jan 2023 16:44:03 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: mst@redhat.com Cc: leiyang@redhat.com, Laurent Vivier , sgarzare@redhat.com, jasowang@redhat.com, Zhu Lingshan , virtualization@lists.linux-foundation.org, si-wei.liu@oracle.com, linux-kernel@vger.kernel.org, lulu@redhat.com, Gautam Dawar , alvaro.karsz@solid-run.com Subject: [PATCH 1/2] vdpa_sim: not reset state in vdpasim_queue_ready Date: Wed, 18 Jan 2023 17:43:58 +0100 Message-Id: <20230118164359.1523760-2-eperezma@redhat.com> In-Reply-To: <20230118164359.1523760-1-eperezma@redhat.com> References: <20230118164359.1523760-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1755379803286728445?= X-GMAIL-MSGID: =?utf-8?q?1755379803286728445?= vdpasim_queue_ready calls vringh_init_iotlb, which resets split indexes. But it can be called after setting a ring base with vdpasim_set_vq_state. Fix it by stashing them. They're still resetted in vdpasim_vq_reset. This was discovered and tested live migrating the vdpa_sim_net device. Fixes: 2c53d0f64c06 ("vdpasim: vDPA device simulator") Signed-off-by: Eugenio Pérez Acked-by: Jason Wang Tested-by: Lei Yang --- drivers/vdpa/vdpa_sim/vdpa_sim.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.c b/drivers/vdpa/vdpa_sim/vdpa_sim.c index cb88891b44a8..8839232a3fcb 100644 --- a/drivers/vdpa/vdpa_sim/vdpa_sim.c +++ b/drivers/vdpa/vdpa_sim/vdpa_sim.c @@ -66,6 +66,7 @@ static void vdpasim_vq_notify(struct vringh *vring) static void vdpasim_queue_ready(struct vdpasim *vdpasim, unsigned int idx) { struct vdpasim_virtqueue *vq = &vdpasim->vqs[idx]; + uint16_t last_avail_idx = vq->vring.last_avail_idx; vringh_init_iotlb(&vq->vring, vdpasim->features, vq->num, false, (struct vring_desc *)(uintptr_t)vq->desc_addr, @@ -74,6 +75,7 @@ static void vdpasim_queue_ready(struct vdpasim *vdpasim, unsigned int idx) (struct vring_used *) (uintptr_t)vq->device_addr); + vq->vring.last_avail_idx = last_avail_idx; vq->vring.notify = vdpasim_vq_notify; } From patchwork Wed Jan 18 16:43:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 45387 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2443147wrn; Wed, 18 Jan 2023 08:50:30 -0800 (PST) X-Google-Smtp-Source: AMrXdXuCAXN9odfXcOM4argHUrcL8/WQtEXAkjWCIL4o+3rOGIASj5Cq4gg4NsQ/x53g/KhEoqvA X-Received: by 2002:a17:90b:1209:b0:229:472d:af43 with SMTP id gl9-20020a17090b120900b00229472daf43mr7836726pjb.44.1674060630549; Wed, 18 Jan 2023 08:50:30 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674060630; cv=none; d=google.com; s=arc-20160816; b=mVa23kIv9sQv9vZwR86VNJF29gXHd1l3XjQCF/OJCpGXINjA12H7RukxYPrxehv5Dh 2/61ofmhz5ShLeOA62wnFtnAW4YGB1/0i+xigJwKN8JR8ZQdTnFl5B+mmqxiFmnJ57Z4 fXVE6LJd5sud8B5Rpawq9Nxem9bV4zG45Q8mfX5TsmsCIwS9Mf7PbateKxI7YVHMiIxS KybrbFb+gwflRD+/KuJVPaRGdsmzon9jPOUEcTpIvDN97SfZZWo9NsrcncrhdTcFoT1i SX2dhhlH3ir5Kz8z++kHhTIDqWqTScODP0V4jMEPydef4SNFobbMgzHeKkyKND036T+K 83XQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=wHqIpzQSkuEl19TQY/3XES0k5HPtR7PzJw91HkmdGrE=; b=BLktwu/IlRTQGPiNA1cg+K7NS4HV7X2x8W8qPS4DDhUNU+us5F7FZklNBXFj8ITn+q BQMygxxYfClA3kDQMTefpEGViXxUw38ZEPlUjjlZ++2EktFdaCBsR4sMepfsPHrgEPnt tci3x4kdeSTdZUNToz5IvGwc2CKUrj/HVDcIk2Jx0QptUkGJo7WRaeZb3oQBB7ECt2gO 31/0Esf5j4fZXqRlA4E7c3rX6ivHz6FQn/vcjhUpLvXfNoX5krXAtAAZ5W303e83puvR cRi4ROmqck6gIR+eRFRoS6737JsFNRn7JroFmOnqkk4aDHN/IUvKneWVZOvHg2j4iSZi FPQA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=UK3L84TP; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id br3-20020a17090b0f0300b002265fe84432si2614925pjb.44.2023.01.18.08.50.18; Wed, 18 Jan 2023 08:50:30 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=UK3L84TP; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229917AbjARQqA (ORCPT + 99 others); Wed, 18 Jan 2023 11:46:00 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48764 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229986AbjARQpf (ORCPT ); Wed, 18 Jan 2023 11:45:35 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6685D582AA for ; Wed, 18 Jan 2023 08:44:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674060253; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wHqIpzQSkuEl19TQY/3XES0k5HPtR7PzJw91HkmdGrE=; b=UK3L84TPIeY/Ia5ttYrHvRvXpk44Ep/TNNwL1+4BqEOdGMXerU3ce6WjGRI0lTHbgmZAxb KxKAYBprKc8q6tTnxd76l26gQdoxF5xRGH2t2CvJSXCwhLDRdSG7VRldN63zAV39JwYpF0 b6DlrpbkZevbOF+rscsaKqzlovpnrmM= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-264-qnHa-CyiOPyF5XEt1aLMuQ-1; Wed, 18 Jan 2023 11:44:12 -0500 X-MC-Unique: qnHa-CyiOPyF5XEt1aLMuQ-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 979113804527; Wed, 18 Jan 2023 16:44:08 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.141]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6EE30140EBF6; Wed, 18 Jan 2023 16:44:06 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: mst@redhat.com Cc: leiyang@redhat.com, Laurent Vivier , sgarzare@redhat.com, jasowang@redhat.com, Zhu Lingshan , virtualization@lists.linux-foundation.org, si-wei.liu@oracle.com, linux-kernel@vger.kernel.org, lulu@redhat.com, Gautam Dawar , alvaro.karsz@solid-run.com Subject: [PATCH 2/2] vringh: fetch used_idx from vring at vringh_init_iotlb Date: Wed, 18 Jan 2023 17:43:59 +0100 Message-Id: <20230118164359.1523760-3-eperezma@redhat.com> In-Reply-To: <20230118164359.1523760-1-eperezma@redhat.com> References: <20230118164359.1523760-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1755379799988378420?= X-GMAIL-MSGID: =?utf-8?q?1755379799988378420?= Starting from an used_idx different than 0 is needed in use cases like virtual machine migration. Not doing so and letting the caller set an avail idx different than 0 causes destination device to try to use old buffers that source driver already recover and are not available anymore. While callers like vdpa_sim set avail_idx directly it does not set used_idx. Instead of let the caller do the assignment, fetch it from the guest at initialization like vhost-kernel do. To perform the same at vring_kernel_init and vring_user_init is left for the future. Signed-off-by: Eugenio Pérez --- drivers/vhost/vringh.c | 25 +++++++++++++++++++++++-- 1 file changed, 23 insertions(+), 2 deletions(-) diff --git a/drivers/vhost/vringh.c b/drivers/vhost/vringh.c index 33eb941fcf15..0eed825197f2 100644 --- a/drivers/vhost/vringh.c +++ b/drivers/vhost/vringh.c @@ -1301,6 +1301,17 @@ static inline int putused_iotlb(const struct vringh *vrh, return 0; } +/** + * vringh_update_used_idx - fetch used idx from driver's used split vring + * @vrh: The vring. + * + * Returns -errno or 0. + */ +static inline int vringh_update_used_idx(struct vringh *vrh) +{ + return getu16_iotlb(vrh, &vrh->last_used_idx, &vrh->vring.used->idx); +} + /** * vringh_init_iotlb - initialize a vringh for a ring with IOTLB. * @vrh: the vringh to initialize. @@ -1319,8 +1330,18 @@ int vringh_init_iotlb(struct vringh *vrh, u64 features, struct vring_avail *avail, struct vring_used *used) { - return vringh_init_kern(vrh, features, num, weak_barriers, - desc, avail, used); + int r = vringh_init_kern(vrh, features, num, weak_barriers, desc, + avail, used); + + if (r != 0) + return r; + + /* Consider the ring not initialized */ + if ((void *)desc == used) + return 0; + + return vringh_update_used_idx(vrh); + } EXPORT_SYMBOL(vringh_init_iotlb);