From patchwork Thu Jan 19 06:15:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Wang X-Patchwork-Id: 4218 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp169714wrn; Wed, 18 Jan 2023 22:21:43 -0800 (PST) X-Google-Smtp-Source: AMrXdXuT4JGO5hRkty1QPihn+UhhYbDiYDakBs3uJYzD9h7jctWNQ1yvOiuycjtwHa4Bsbed53em X-Received: by 2002:a17:907:c713:b0:78d:f454:ba3d with SMTP id ty19-20020a170907c71300b0078df454ba3dmr6679157ejc.60.1674109303032; Wed, 18 Jan 2023 22:21:43 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674109303; cv=none; d=google.com; s=arc-20160816; b=XRITwh1f/Y+NdpHZE35mK2x78q76mhf+LTWfR9GPvpbGjTyjAu3UMJbAUSXwWDz8Ca s7WN1/Y1U2EXSmhZYdL4Y5AeDZWAPoZj45iaPJrV7JR4WPrmQV2XFHmN5pQi3iW0ZVZH Skhlp3JpOZP3/5FaL2pUXaf5unVEKCu12ObPXC4CRAWO2zp4pSHF1rSJVa51wfmAH5s9 sMF2twxGCVK89gFjqqehumMdgydJcoyRnSK5y7bw2glyPe8vfMo+OhtK19TuG2pso/mi Kmi0t4m0ICt2aXJchriwRUj6jdtNbUVfsLaDikB5D87YhDe11+Eqniqtqgfl24MPpkMt iU1g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=DINlqTBfPPkQSaUFJZpbcOWI3MVsDFM2A5IzYzWW+XQ=; b=wn9ubFKrPT0DlfhcBwY9k1l29oG50Z5Js64FOybRRWtcTglKeBunk4fW9Ak5lQMqH3 +tSpDlvOczsYYY+VprfM4g+7Q96VlVDIaqqKGLAdfIA8dvtdgpgbjbc6T54MmCFs9jpK MdZrHRd4XuZfnHXPkmCsWdOt1gwOoR3DCQRxghPv+DR6cO7vvC14qza4Es4c8dEoqGrC tH1RACQ+HO8Cff8QUa/5TNm9MWTDOUDF2lHPDBUC8QJUm+ZmvNqc/KeqCIOVMgsRgciH fIfpv4moyazncAze5pchbblFt0G569ZzMg8T/YxsJdIcIwEbq5uCCPKQTwWQZwYg/OW5 76GA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=R7lfGLth; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id gs42-20020a1709072d2a00b007c170f6b32bsi10115176ejc.266.2023.01.18.22.21.19; Wed, 18 Jan 2023 22:21:43 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=R7lfGLth; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229734AbjASGQr (ORCPT + 99 others); Thu, 19 Jan 2023 01:16:47 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48634 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229544AbjASGQj (ORCPT ); Thu, 19 Jan 2023 01:16:39 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 996BA65EF1 for ; Wed, 18 Jan 2023 22:15:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674108947; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=DINlqTBfPPkQSaUFJZpbcOWI3MVsDFM2A5IzYzWW+XQ=; b=R7lfGLth/MwkskECuNHwyEFCfpCQfTi2oESodGPXuiWJwX/BWjMjgTymw+RSuQ4RjK6PNt RhMSID28lA369BzP8Z/B1zxmnG0ILzN8h+cc5AAUwtd6+Y4Av7yV5COBwY1M/6T/Bupqq2 1ojKRGJSWOGozH7Yku9H6dOdqeuyXn4= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-544-FZ9LurrSPMSHkxASg_7zXw-1; Thu, 19 Jan 2023 01:15:33 -0500 X-MC-Unique: FZ9LurrSPMSHkxASg_7zXw-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 1B3C88A010C; Thu, 19 Jan 2023 06:15:33 +0000 (UTC) Received: from localhost.localdomain (ovpn-13-97.pek2.redhat.com [10.72.13.97]) by smtp.corp.redhat.com (Postfix) with ESMTP id A4D901121315; Thu, 19 Jan 2023 06:15:29 +0000 (UTC) From: Jason Wang To: mst@redhat.com, jasowang@redhat.com Cc: elic@nvidia.com, gdawar@amd.com, tanuj.kamde@amd.com, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org Subject: [PATCH V2 0/5] virtio_ring: per virtqueue DMA device Date: Thu, 19 Jan 2023 14:15:20 +0800 Message-Id: <20230119061525.75068-1-jasowang@redhat.com> MIME-Version: 1.0 Content-type: text/plain X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1755430836809239633?= X-GMAIL-MSGID: =?utf-8?q?1755430836809239633?= Hi All: In some cases, the virtqueue could be backed by different devices. One example is that in the case of vDPA some parent may emualte virtqueue via vringh. In this case, it would be wrong if we stick with the physical DMA device for software emulated device, since there's no easy way for vringh to know about the hardware IOMMU mappings. So this series tries to introduce per virtqueue DMA device, then software virtqueues can utilize the transport specific method to assign appropirate DMA device. This fixes the crash of mlx5_vdpa + virtio_vdpa when platform IOMMU is enabled but not in the passthrough mode. The reason for the crash is that the virito_ring tries to map the control virtqueue into platform IOMMU but the vringh assumes a direct mapping (PA as IOVA). This is fixed by advetise the vDPA device that doesnt do DMA (without a DMA ops). So DMA API can go with the direct mapping then the vringh will be happy since mlx5_vdpa assuems a direct/identical mapping by default. Please review. Changes since V1: - make mlx5_get_vq_dma_dev() static Jason Wang (5): virtio_ring: per virtqueue dma device vdpa: introduce get_vq_dma_device() virtio-vdpa: support per vq dma device vdpa: set dma mask for vDPA device vdpa: mlx5: support per virtqueue dma device drivers/vdpa/mlx5/net/mlx5_vnet.c | 11 +++ drivers/vdpa/vdpa.c | 5 ++ drivers/virtio/virtio_ring.c | 133 +++++++++++++++++++++--------- drivers/virtio/virtio_vdpa.c | 13 ++- include/linux/vdpa.h | 6 ++ include/linux/virtio_ring.h | 16 ++++ 6 files changed, 141 insertions(+), 43 deletions(-)