From patchwork Sun Jan 29 02:50:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Longpeng(Mike)" X-Patchwork-Id: 49932 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp1588187wrn; Sat, 28 Jan 2023 19:02:59 -0800 (PST) X-Google-Smtp-Source: AK7set8iu9mAORCxCmC9EZ8b+V9hRN9N4NnWlfO36UH/EeGlkouDDCDBPnIVj5Tkqj1YZd1uOY8W X-Received: by 2002:aa7:8f16:0:b0:593:b491:40b8 with SMTP id x22-20020aa78f16000000b00593b49140b8mr262591pfr.4.1674961379187; Sat, 28 Jan 2023 19:02:59 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674961379; cv=none; d=google.com; s=arc-20160816; b=ePsxl8EZRyejbidynNIkhjSPt1JSIil5ysVN6NeyfcpF9FFUROEgZgPHFX1i0ZcjGR 4Hh7eN2yW56xZlGyhaDPRZ8IJR9hHBGH+uSb9QKblmt9Yozd6EEeBLFKs9bzqzW4xDf3 SeO0IKmIwEDqNLtfeuP7Q5ctWMhNJYdES3adaz+f5/07gReDFWDF3ZX+ePeIMsVuhUaT xNN8iHn/gzpAuKzpBKoAfdZw8wTITFHBNcbF1iVAPy9rZaA9++/mWONm0hEwCStlK8wV 2Nbu3ht9It53BHh/sYbHmLUTrwXCLRR3/d3sVpEoAi/4yX4R+eXYDFGuVl7gMUTJuflk htzQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=BP1jSVzxrgksQs+95NEJhJsuniEFEackCzl696n0iHU=; b=dmanflnJ8gc3+p854SfSgx/z7dZpf1TnIkLytfg0fFrB14hJqvWn3LSBvTUGw8dE92 vviAL9eH3YwzC/QiTN/7FdXkLGNHrO+oeNuFWPzE4O1jt3Q64OlQfEHGWNxd+In3yfkQ P9VqNm3f2Vd4/M4ccAOkIO91s8OAHmW/zVM7E3Yl1if22gkj4K21oVIetdkLUrDGOl0P uYqXbXzLWVOWfFv35+naknNVDDnd/+nyG9uWhoQ9rVam2Z5H8VUQszYcmY35J02id/zo a3uu42AAwgGDzzxrBSNW/4BDPpAna5KTLQUdWAvqHemmcsKm9y1eFy5FUDJ8enx5haQ6 SYIQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id i22-20020aa78d96000000b00571a5c5eaa5si9171003pfr.150.2023.01.28.19.02.45; Sat, 28 Jan 2023 19:02:59 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234327AbjA2Cup (ORCPT + 99 others); Sat, 28 Jan 2023 21:50:45 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34868 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231946AbjA2Cuo (ORCPT ); Sat, 28 Jan 2023 21:50:44 -0500 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 154AF1B575 for ; Sat, 28 Jan 2023 18:50:42 -0800 (PST) Received: from kwepemi100025.china.huawei.com (unknown [172.30.72.55]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4P4G5N519fzfZ4C; Sun, 29 Jan 2023 10:50:32 +0800 (CST) Received: from DESKTOP-27KDQMV.china.huawei.com (10.174.148.223) by kwepemi100025.china.huawei.com (7.221.188.158) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Sun, 29 Jan 2023 10:50:38 +0800 From: "Longpeng(Mike)" To: , CC: , , , , , , , , Longpeng Subject: [PATCH v3 1/2] vdpa: support specify the pgprot of vq notification area Date: Sun, 29 Jan 2023 10:50:33 +0800 Message-ID: <20230129025034.2000-2-longpeng2@huawei.com> X-Mailer: git-send-email 2.25.0.windows.1 In-Reply-To: <20230129025034.2000-1-longpeng2@huawei.com> References: <20230129025034.2000-1-longpeng2@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.174.148.223] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To kwepemi100025.china.huawei.com (7.221.188.158) X-CFilter-Loop: Reflected X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756324303269554956?= X-GMAIL-MSGID: =?utf-8?q?1756324303269554956?= From: Longpeng Adds get_vq_notification_pgprot operation to vdpa_config_ops to support specify the pgprot of vq norification area. It's an optional operation, the vdpa framework will treat the pgprot of vq notification area as noncached as default as usual. Acked-by: Jason Wang --- drivers/vhost/vdpa.c | 4 +++- include/linux/vdpa.h | 9 +++++++++ 2 files changed, 12 insertions(+), 1 deletion(-) diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c index 166044642fd5..036fe88425c8 100644 --- a/drivers/vhost/vdpa.c +++ b/drivers/vhost/vdpa.c @@ -1263,7 +1263,9 @@ static vm_fault_t vhost_vdpa_fault(struct vm_fault *vmf) notify = ops->get_vq_notification(vdpa, index); - vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot); + vma->vm_page_prot = ops->get_vq_notification_pgprot ? + ops->get_vq_notification_pgprot(vdpa, index, vma->vm_page_prot) : + pgprot_noncached(vma->vm_page_prot); if (remap_pfn_range(vma, vmf->address & PAGE_MASK, PFN_DOWN(notify.addr), PAGE_SIZE, vma->vm_page_prot)) diff --git a/include/linux/vdpa.h b/include/linux/vdpa.h index 6d0f5e4e82c2..07fcf5e6abc8 100644 --- a/include/linux/vdpa.h +++ b/include/linux/vdpa.h @@ -169,6 +169,12 @@ struct vdpa_map_file { * @vdev: vdpa device * @idx: virtqueue index * Returns the notifcation area + * @get_vq_notification_pgprot: Get the pgprot of the vq's notification area (optional) + * @vdev: vdpa device + * @idx: virtqueue index + * @prot: original page protection value of the + * notification area + * Returns pgprot_t: the pgprot of the notification area * @get_vq_irq: Get the irq number of a virtqueue (optional, * but must implemented if require vq irq offloading) * @vdev: vdpa device @@ -305,6 +311,9 @@ struct vdpa_config_ops { struct netlink_ext_ack *extack); struct vdpa_notification_area (*get_vq_notification)(struct vdpa_device *vdev, u16 idx); + pgprot_t (*get_vq_notification_pgprot)(struct vdpa_device *vdev, + u16 idx, + pgprot_t prot); /* vq irq is not expected to be changed once DRIVER_OK is set */ int (*get_vq_irq)(struct vdpa_device *vdev, u16 idx); From patchwork Sun Jan 29 02:50:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Longpeng(Mike)" X-Patchwork-Id: 49933 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp1588487wrn; Sat, 28 Jan 2023 19:03:56 -0800 (PST) X-Google-Smtp-Source: AMrXdXvTCBSsW/pq645WtSRpGVVTIiB+G2deTD43jAkZADIPNAORw0GheCutUm+G9+ej9kBUJdAm X-Received: by 2002:a05:6a20:4295:b0:b8:9c66:cd64 with SMTP id o21-20020a056a20429500b000b89c66cd64mr59756687pzj.14.1674961436149; Sat, 28 Jan 2023 19:03:56 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674961436; cv=none; d=google.com; s=arc-20160816; b=GvRXqDfOmu2gIbltTRl+iDDkZ507oyxrbxsh/pAbckd9YXQsKSJdvPL3VnKYyT/K7h rZfXHV+WO7+XRHMFsY/Vj51y5ItwVwC6mV4/ErTszoEFACfb+G3AvsmcXGLWfyjjnaUm Vzkd1J3WX0V2yFY/qXRjG4tTVD5J0xRo5qEudgnJ++YaGf7CnIfbuygdk5XbOHOCJ53C NcM6f9PtqeCCcAFjIbZHtZKe0QBKkPVfeaHfF6fpn1ZFWT5qfLx02cL89t2wEiofrSBF lNywuXFB2emDd/EYh8b1Qv8mpQP/yXg4CHi8LFtAD/jYWtwUe80uWU+jYPb2fVKz+fJy JibA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=83tmXNkiVpsXWixw1kjuFHYtfsHYB/4LIMNF9bSGfa0=; b=PPV1SrGKfaTgrKYX2CPTG9HGhJ8xgELNd92c/5PoiWFZUdC7HpgrhJwrhEwBxYaf0I MrIN6KfWc8aA4GwyC6jCo/MGwGUn69dmx8wK8gBTslbEQDEaOaQ8Wwj6XB03dAu/RQNk FEL7N/TIIxPyWuz12s3dFjDXUq3B3qLq5MVDQgmGj7w3cFAIb6dd42T1M/nzKM9rWGby 6/ruRL60zoWGXKk+aBS/CtK7GqAjs9z3cd/a3OqltmE5qZJu8q6BvkPgGJr7g9KwcjU2 Up06XX7K+a5bvUpAaWWuk9ShIc6Twv9rnvRvqQ7kWCXLQIoou7Rt4FnW2wulMvN0Vpvt vKtA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id n5-20020a622705000000b00580d9a62ab8si8569652pfn.184.2023.01.28.19.03.43; Sat, 28 Jan 2023 19:03:56 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234639AbjA2Cuu (ORCPT + 99 others); Sat, 28 Jan 2023 21:50:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34866 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230247AbjA2Cuo (ORCPT ); Sat, 28 Jan 2023 21:50:44 -0500 Received: from szxga03-in.huawei.com (szxga03-in.huawei.com [45.249.212.189]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C23561B571 for ; Sat, 28 Jan 2023 18:50:41 -0800 (PST) Received: from kwepemi100025.china.huawei.com (unknown [172.30.72.56]) by szxga03-in.huawei.com (SkyGuard) with ESMTP id 4P4G0Q09JzzJq3x; Sun, 29 Jan 2023 10:46:14 +0800 (CST) Received: from DESKTOP-27KDQMV.china.huawei.com (10.174.148.223) by kwepemi100025.china.huawei.com (7.221.188.158) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Sun, 29 Jan 2023 10:50:39 +0800 From: "Longpeng(Mike)" To: , CC: , , , , , , , , Longpeng Subject: [PATCH v3 2/2] vdpasim: support doorbell mapping Date: Sun, 29 Jan 2023 10:50:34 +0800 Message-ID: <20230129025034.2000-3-longpeng2@huawei.com> X-Mailer: git-send-email 2.25.0.windows.1 In-Reply-To: <20230129025034.2000-1-longpeng2@huawei.com> References: <20230129025034.2000-1-longpeng2@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.174.148.223] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To kwepemi100025.china.huawei.com (7.221.188.158) X-CFilter-Loop: Reflected X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756324220501905159?= X-GMAIL-MSGID: =?utf-8?q?1756324362821970026?= From: Longpeng Support doorbell mapping for vdpasim devices, then we can test the notify passthrough feature even if there's no real hardware on hand. Allocates a dummy page which is used to emulate the notify page of the device, all VQs share the same notify register that initiated to 0xffff. A periodic work will check whether there're requests need to process ( the value of the notify register is 0xffff or not ). --- drivers/vdpa/vdpa_sim/vdpa_sim.c | 65 ++++++++++++++++++++++++++++++++ drivers/vdpa/vdpa_sim/vdpa_sim.h | 3 ++ 2 files changed, 68 insertions(+) diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.c b/drivers/vdpa/vdpa_sim/vdpa_sim.c index b071f0d842fb..4fcfeb6e2fb8 100644 --- a/drivers/vdpa/vdpa_sim/vdpa_sim.c +++ b/drivers/vdpa/vdpa_sim/vdpa_sim.c @@ -39,6 +39,8 @@ MODULE_PARM_DESC(max_iotlb_entries, #define VDPASIM_QUEUE_ALIGN PAGE_SIZE #define VDPASIM_QUEUE_MAX 256 #define VDPASIM_VENDOR_ID 0 +#define VDPASIM_VRING_POLL_PERIOD 100 /* ms */ +#define VDPASIM_NOTIFY_DEFVAL 0xffff static struct vdpasim *vdpa_to_sim(struct vdpa_device *vdpa) { @@ -246,6 +248,28 @@ static const struct dma_map_ops vdpasim_dma_ops = { static const struct vdpa_config_ops vdpasim_config_ops; static const struct vdpa_config_ops vdpasim_batch_config_ops; +static void vdpasim_notify_work(struct work_struct *work) +{ + struct vdpasim *vdpasim; + u16 *val; + + vdpasim = container_of(work, struct vdpasim, notify_work.work); + + if (!(vdpasim->status & VIRTIO_CONFIG_S_DRIVER_OK)) + goto out; + + if (!vdpasim->running) + goto out; + + val = (u16 *)vdpasim->notify; + if (xchg(val, VDPASIM_NOTIFY_DEFVAL) != VDPASIM_NOTIFY_DEFVAL) + schedule_work(&vdpasim->work); + +out: + schedule_delayed_work(&vdpasim->notify_work, + msecs_to_jiffies(VDPASIM_VRING_POLL_PERIOD)); +} + struct vdpasim *vdpasim_create(struct vdpasim_dev_attr *dev_attr, const struct vdpa_dev_set_config *config) { @@ -287,6 +311,13 @@ struct vdpasim *vdpasim_create(struct vdpasim_dev_attr *dev_attr, set_dma_ops(dev, &vdpasim_dma_ops); vdpasim->vdpa.mdev = dev_attr->mgmt_dev; + INIT_DELAYED_WORK(&vdpasim->notify_work, vdpasim_notify_work); + + vdpasim->notify = __get_free_page(GFP_KERNEL | __GFP_ZERO); + if (!vdpasim->notify) + goto err_iommu; + *(u16 *)vdpasim->notify = VDPASIM_NOTIFY_DEFVAL; + vdpasim->config = kzalloc(dev_attr->config_size, GFP_KERNEL); if (!vdpasim->config) goto err_iommu; @@ -498,16 +529,21 @@ static u8 vdpasim_get_status(struct vdpa_device *vdpa) static void vdpasim_set_status(struct vdpa_device *vdpa, u8 status) { struct vdpasim *vdpasim = vdpa_to_sim(vdpa); + bool started = vdpasim->status & VIRTIO_CONFIG_S_DRIVER_OK; spin_lock(&vdpasim->lock); vdpasim->status = status; spin_unlock(&vdpasim->lock); + if (!started && (status & VIRTIO_CONFIG_S_DRIVER_OK)) + schedule_delayed_work(&vdpasim->notify_work, + msecs_to_jiffies(VDPASIM_VRING_POLL_PERIOD)); } static int vdpasim_reset(struct vdpa_device *vdpa) { struct vdpasim *vdpasim = vdpa_to_sim(vdpa); + cancel_delayed_work_sync(&vdpasim->notify_work); spin_lock(&vdpasim->lock); vdpasim->status = 0; vdpasim_do_reset(vdpasim); @@ -672,11 +708,34 @@ static int vdpasim_dma_unmap(struct vdpa_device *vdpa, unsigned int asid, return 0; } +static pgprot_t vdpasim_get_vq_notification_pgprot(struct vdpa_device *vdpa, + u16 qid, pgprot_t prot) +{ + /* + * We use normal RAM pages to emulate the vq notification area, so + * just keep the pgprot as it mmaped. + */ + return prot; +} + +static struct vdpa_notification_area +vdpasim_get_vq_notification(struct vdpa_device *vdpa, u16 qid) +{ + struct vdpasim *vdpasim = vdpa_to_sim(vdpa); + struct vdpa_notification_area notify; + + notify.addr = virt_to_phys((void *)vdpasim->notify); + notify.size = PAGE_SIZE; + + return notify; +} + static void vdpasim_free(struct vdpa_device *vdpa) { struct vdpasim *vdpasim = vdpa_to_sim(vdpa); int i; + cancel_delayed_work_sync(&vdpasim->notify_work); cancel_work_sync(&vdpasim->work); for (i = 0; i < vdpasim->dev_attr.nvqs; i++) { @@ -693,6 +752,8 @@ static void vdpasim_free(struct vdpa_device *vdpa) vhost_iotlb_free(vdpasim->iommu); kfree(vdpasim->vqs); kfree(vdpasim->config); + if (vdpasim->notify) + free_page(vdpasim->notify); } static const struct vdpa_config_ops vdpasim_config_ops = { @@ -704,6 +765,8 @@ static const struct vdpa_config_ops vdpasim_config_ops = { .get_vq_ready = vdpasim_get_vq_ready, .set_vq_state = vdpasim_set_vq_state, .get_vq_state = vdpasim_get_vq_state, + .get_vq_notification = vdpasim_get_vq_notification, + .get_vq_notification_pgprot = vdpasim_get_vq_notification_pgprot, .get_vq_align = vdpasim_get_vq_align, .get_vq_group = vdpasim_get_vq_group, .get_device_features = vdpasim_get_device_features, @@ -737,6 +800,8 @@ static const struct vdpa_config_ops vdpasim_batch_config_ops = { .get_vq_ready = vdpasim_get_vq_ready, .set_vq_state = vdpasim_set_vq_state, .get_vq_state = vdpasim_get_vq_state, + .get_vq_notification = vdpasim_get_vq_notification, + .get_vq_notification_pgprot = vdpasim_get_vq_notification_pgprot, .get_vq_align = vdpasim_get_vq_align, .get_vq_group = vdpasim_get_vq_group, .get_device_features = vdpasim_get_device_features, diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.h b/drivers/vdpa/vdpa_sim/vdpa_sim.h index 0e78737dcc16..0769ccbd3911 100644 --- a/drivers/vdpa/vdpa_sim/vdpa_sim.h +++ b/drivers/vdpa/vdpa_sim/vdpa_sim.h @@ -69,6 +69,9 @@ struct vdpasim { bool running; /* spinlock to synchronize iommu table */ spinlock_t iommu_lock; + /* dummy notify page */ + unsigned long notify; + struct delayed_work notify_work; }; struct vdpasim *vdpasim_create(struct vdpasim_dev_attr *attr,