From patchwork Mon Feb 27 09:18:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Longpeng(Mike)" X-Patchwork-Id: 61886 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp2319122wrd; Mon, 27 Feb 2023 01:30:06 -0800 (PST) X-Google-Smtp-Source: AK7set91KCyCpanFH4m8Chcds8WyN69HLY1mgH7lraCLj8BWOdgaBLmRzQf1cSlrBhpP/0ccDW5s X-Received: by 2002:a17:906:8d0a:b0:8f2:62a9:6159 with SMTP id rv10-20020a1709068d0a00b008f262a96159mr14001562ejc.2.1677490206563; Mon, 27 Feb 2023 01:30:06 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1677490206; cv=none; d=google.com; s=arc-20160816; b=EiQeeiFHvyuqwc1rFlX6cNIWq+cqzFnz8W49x8vWIwXyxtY8OsBYyzZ8nOMqqmW2bn StzyaVC1t7pQsdu9Ur/oAy2EKPxzJVUjf+eovEsj0XjwQ0j0LxRpVKyKZuS0t/h8r6gp FHJiCog0fETMVZDF70CxLeF+8PCmSavot9GCp8zHjcUSeLbx3D1XLer81+WrQrahJsW0 2NyvOCLuAbBiLB6o6l3nbo6iBaGddJzPinBDQG/4EZRTbXK8POFtAcCFv5dgqIBJrReV qPWRMSg1Mjx5X+gD0j4PvwHGaPQx5SK1hpMTd34l4fwYHyyH/Z9V5JarUDPIRkh70ANf KvRQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=Xxg2u3nKc8g3Ma+mnjQ/63eGqQJSZ0S6BDYk9pBtDck=; b=fNeh27VH0mkX0rckUgOt3rLSldDb0yx2mWH8n0aiM+ooCSZLccO7ucPP1FmVwpicCi UA914kUwrQLTzTDKeWIBq4+e0l0G5wY5RAv9etM9vt5VTBrud8r1jCaMs2CorRFTeVjm jIQ9ebMziiM51XxfhIkZN+ZAI4uPgAGXW59gacPKeRDPrzAlINumH5EgbWl9rIKY4rPd LAo09arHhn/OG2P5/hVFSOGhkCczUQY1Ss1BT6lS5v7j/RD0s7GHApoG9DvQIL2GNf1Q ReyaEJ3c1PcTH1ytxHNGijdh5HtTJ22U2ILOsJzp91+XU7N0XPgk29zFcCrUwin8Ixqb YkGw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id t23-20020aa7d717000000b00487ceac6d34si8433437edq.137.2023.02.27.01.29.43; Mon, 27 Feb 2023 01:30:06 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229590AbjB0JVs (ORCPT + 99 others); Mon, 27 Feb 2023 04:21:48 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56052 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229795AbjB0JVX (ORCPT ); Mon, 27 Feb 2023 04:21:23 -0500 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8B7C75269 for ; Mon, 27 Feb 2023 01:19:33 -0800 (PST) Received: from kwepemi100025.china.huawei.com (unknown [172.30.72.55]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4PQFL662XnzrSMw; Mon, 27 Feb 2023 17:18:54 +0800 (CST) Received: from DESKTOP-27KDQMV.china.huawei.com (10.174.148.223) by kwepemi100025.china.huawei.com (7.221.188.158) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.21; Mon, 27 Feb 2023 17:19:01 +0800 From: "Longpeng(Mike)" To: , CC: , , , , , , , , Longpeng Subject: [PATCH v4 2/2] vdpasim: support doorbell mapping Date: Mon, 27 Feb 2023 17:18:57 +0800 Message-ID: <20230227091857.2406-3-longpeng2@huawei.com> X-Mailer: git-send-email 2.25.0.windows.1 In-Reply-To: <20230227091857.2406-1-longpeng2@huawei.com> References: <20230227091857.2406-1-longpeng2@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.174.148.223] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To kwepemi100025.china.huawei.com (7.221.188.158) X-CFilter-Loop: Reflected X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1758975970905997795?= X-GMAIL-MSGID: =?utf-8?q?1758975970905997795?= From: Longpeng Support doorbell mapping for vdpasim devices, then we can test the notify passthrough feature even if there's no real hardware on hand. Allocates a dummy page which is used to emulate the notify page of the device, all VQs share the same notify register that initiated to 0xffff. A periodic work will check whether there're requests need to process ( the value of the notify register is 0xffff or not ). We can test on QEMU with: -device vhost-vdpa-device-pci,vhostdev=/dev/vhost-vdpa-0,page-per-vq=on Signed-off-by: Longpeng Acked-by: Jason Wang --- drivers/vdpa/vdpa_sim/vdpa_sim.c | 67 ++++++++++++++++++++++++++++++++ drivers/vdpa/vdpa_sim/vdpa_sim.h | 3 ++ 2 files changed, 70 insertions(+) diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.c b/drivers/vdpa/vdpa_sim/vdpa_sim.c index cb88891b44a8..5a8c820b179f 100644 --- a/drivers/vdpa/vdpa_sim/vdpa_sim.c +++ b/drivers/vdpa/vdpa_sim/vdpa_sim.c @@ -39,6 +39,8 @@ MODULE_PARM_DESC(max_iotlb_entries, #define VDPASIM_QUEUE_ALIGN PAGE_SIZE #define VDPASIM_QUEUE_MAX 256 #define VDPASIM_VENDOR_ID 0 +#define VDPASIM_VRING_POLL_PERIOD 100 /* ms */ +#define VDPASIM_NOTIFY_DEFVAL 0xffff static struct vdpasim *vdpa_to_sim(struct vdpa_device *vdpa) { @@ -245,6 +247,28 @@ static const struct dma_map_ops vdpasim_dma_ops = { static const struct vdpa_config_ops vdpasim_config_ops; static const struct vdpa_config_ops vdpasim_batch_config_ops; +static void vdpasim_notify_work(struct work_struct *work) +{ + struct vdpasim *vdpasim; + u16 *val; + + vdpasim = container_of(work, struct vdpasim, notify_work.work); + + if (!(vdpasim->status & VIRTIO_CONFIG_S_DRIVER_OK)) + goto out; + + if (!vdpasim->running) + goto out; + + val = (u16 *)vdpasim->notify; + if (xchg(val, VDPASIM_NOTIFY_DEFVAL) != VDPASIM_NOTIFY_DEFVAL) + schedule_work(&vdpasim->work); + +out: + schedule_delayed_work(&vdpasim->notify_work, + msecs_to_jiffies(VDPASIM_VRING_POLL_PERIOD)); +} + struct vdpasim *vdpasim_create(struct vdpasim_dev_attr *dev_attr, const struct vdpa_dev_set_config *config) { @@ -286,6 +310,13 @@ struct vdpasim *vdpasim_create(struct vdpasim_dev_attr *dev_attr, set_dma_ops(dev, &vdpasim_dma_ops); vdpasim->vdpa.mdev = dev_attr->mgmt_dev; + INIT_DELAYED_WORK(&vdpasim->notify_work, vdpasim_notify_work); + + vdpasim->notify = (u16 *)__get_free_page(GFP_KERNEL | __GFP_ZERO); + if (!vdpasim->notify) + goto err_iommu; + WRITE_ONCE(*vdpasim->notify, VDPASIM_NOTIFY_DEFVAL); + vdpasim->config = kzalloc(dev_attr->config_size, GFP_KERNEL); if (!vdpasim->config) goto err_iommu; @@ -320,6 +351,13 @@ struct vdpasim *vdpasim_create(struct vdpasim_dev_attr *dev_attr, vdpasim->vdpa.dma_dev = dev; + /* + * Start periodic (every 100ms) notify work, it won't introduce + * any overhead if the device is not started. + */ + schedule_delayed_work(&vdpasim->notify_work, + msecs_to_jiffies(VDPASIM_VRING_POLL_PERIOD)); + return vdpasim; err_iommu: @@ -671,11 +709,34 @@ static int vdpasim_dma_unmap(struct vdpa_device *vdpa, unsigned int asid, return 0; } +static pgprot_t vdpasim_get_vq_notification_pgprot(struct vdpa_device *vdpa, + u16 qid, pgprot_t prot) +{ + /* + * We use normal RAM pages to emulate the vq notification area, so + * just keep the pgprot as it mmaped. + */ + return prot; +} + +static struct vdpa_notification_area +vdpasim_get_vq_notification(struct vdpa_device *vdpa, u16 qid) +{ + struct vdpasim *vdpasim = vdpa_to_sim(vdpa); + struct vdpa_notification_area notify; + + notify.addr = virt_to_phys((void *)vdpasim->notify); + notify.size = PAGE_SIZE; + + return notify; +} + static void vdpasim_free(struct vdpa_device *vdpa) { struct vdpasim *vdpasim = vdpa_to_sim(vdpa); int i; + cancel_delayed_work_sync(&vdpasim->notify_work); cancel_work_sync(&vdpasim->work); for (i = 0; i < vdpasim->dev_attr.nvqs; i++) { @@ -694,6 +755,8 @@ static void vdpasim_free(struct vdpa_device *vdpa) kfree(vdpasim->iommu); kfree(vdpasim->vqs); kfree(vdpasim->config); + if (vdpasim->notify) + free_page((unsigned long)vdpasim->notify); } static const struct vdpa_config_ops vdpasim_config_ops = { @@ -705,6 +768,8 @@ static const struct vdpa_config_ops vdpasim_config_ops = { .get_vq_ready = vdpasim_get_vq_ready, .set_vq_state = vdpasim_set_vq_state, .get_vq_state = vdpasim_get_vq_state, + .get_vq_notification = vdpasim_get_vq_notification, + .get_vq_notification_pgprot = vdpasim_get_vq_notification_pgprot, .get_vq_align = vdpasim_get_vq_align, .get_vq_group = vdpasim_get_vq_group, .get_device_features = vdpasim_get_device_features, @@ -738,6 +803,8 @@ static const struct vdpa_config_ops vdpasim_batch_config_ops = { .get_vq_ready = vdpasim_get_vq_ready, .set_vq_state = vdpasim_set_vq_state, .get_vq_state = vdpasim_get_vq_state, + .get_vq_notification = vdpasim_get_vq_notification, + .get_vq_notification_pgprot = vdpasim_get_vq_notification_pgprot, .get_vq_align = vdpasim_get_vq_align, .get_vq_group = vdpasim_get_vq_group, .get_device_features = vdpasim_get_device_features, diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.h b/drivers/vdpa/vdpa_sim/vdpa_sim.h index 0e78737dcc16..786e902de0b2 100644 --- a/drivers/vdpa/vdpa_sim/vdpa_sim.h +++ b/drivers/vdpa/vdpa_sim/vdpa_sim.h @@ -69,6 +69,9 @@ struct vdpasim { bool running; /* spinlock to synchronize iommu table */ spinlock_t iommu_lock; + /* dummy notify page */ + u16 *notify; + struct delayed_work notify_work; }; struct vdpasim *vdpasim_create(struct vdpasim_dev_attr *attr,