From patchwork Thu Mar 2 11:34:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefano Garzarella X-Patchwork-Id: 63363 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp4184761wrd; Thu, 2 Mar 2023 03:38:05 -0800 (PST) X-Google-Smtp-Source: AK7set/eWUZK345CHfMlRk17EpvYr4C+UT9WVIFXq05UQGSDxJXcoSL42zr16cBOqcE/y//hec2v X-Received: by 2002:aa7:cf17:0:b0:4bc:ad70:62ad with SMTP id a23-20020aa7cf17000000b004bcad7062admr4863474edy.19.1677757084973; Thu, 02 Mar 2023 03:38:04 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1677757084; cv=none; d=google.com; s=arc-20160816; b=uw45OXd/c68qmQHMp45G4Vjet6uZie0tVMlh7B6jaLAuUfnTC0TjCdyO7xj5ghnEOK AN31ov3AOXyA1Sv7HDzc6eZ/H4BmqHcqe0+WQGyn+wyCI/y7ZD2ML5cEpaY0OA4wMTLS XyxlrMYtibo5jcU0N+TKrl+t3CfNba1uIJAIqgDTDi7wb67UDCnnvG9Uj+iXAI/39IPF /c+F/RAQCl3JYT1NgKduvAT/5VrZfbzoWRDWemmdAoGa8zQ+AsxXluRCUap9G7wksyQ1 NSxZ0bqUgrlZXtwjnWXS1nPaFjWEBqo1f4VwSpv9gDwwig506aobxVSMUkmOIHen4zDo R77g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=zd9PLJS6vpKJ6CPmR9YHtESYhzkUvkwsWDxsXaqlwQ8=; b=qrFxOz5VbPMaxT9HYMJihCHhlAQnpMCMGvHW7W03BE4lk+L2CGYmlt6uHkdhp7+0Ya GjhWsDuQ8k6ZXUiO0Qtevf+8JNCUIeyU5G8KFSJyhWE8m+y+I+wj94/LXmzo3Htc91p4 bxtf8RXYhrfY8jqvNI+e1+kG5aQFNmFRvQwPr1oafr5/Av8ZSExmay8zeRt6dHtKejgJ kFSFxMU7eSdO2vnYE/p9twOgYZ30KWKZJhfP2GCHCYtFmE8JQBjfujGTEcLDtJn03Vtx cuq/j3r4S9CCiOSZ1aCFiqKHbtDQWJ+4B4qnoYEmN3OmSYd8/zBGp/4V/sviGM3cQb2l tPJQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=KBPxhpEZ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id v15-20020aa7d64f000000b004ada4ddbbf6si986613edr.426.2023.03.02.03.37.42; Thu, 02 Mar 2023 03:38:04 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=KBPxhpEZ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229814AbjCBLfj (ORCPT + 99 others); Thu, 2 Mar 2023 06:35:39 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55106 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229779AbjCBLfZ (ORCPT ); Thu, 2 Mar 2023 06:35:25 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F23D8199E1 for ; Thu, 2 Mar 2023 03:34:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1677756873; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zd9PLJS6vpKJ6CPmR9YHtESYhzkUvkwsWDxsXaqlwQ8=; b=KBPxhpEZ/FO1f2jdraRNtjAM4oXlKEWSC3qkaxMTzF7kXTUXp+a3WEs+v8R7BbU78Xd+iG xBzhuoAk3eU3W4ksdhs9KDiqIzekmxvaLOJpnZPF90MXrTwwB8aHC/OMa64QyhMlffI+xv s7JHfdWcK3NfP42WHzABqgl6tCT/bRA= Received: from mail-qk1-f200.google.com (mail-qk1-f200.google.com [209.85.222.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-226-Mc2WA9EfO2em5QzhmsZCrg-1; Thu, 02 Mar 2023 06:34:32 -0500 X-MC-Unique: Mc2WA9EfO2em5QzhmsZCrg-1 Received: by mail-qk1-f200.google.com with SMTP id dm13-20020a05620a1d4d00b00742a22c4239so7998870qkb.1 for ; Thu, 02 Mar 2023 03:34:31 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1677756871; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zd9PLJS6vpKJ6CPmR9YHtESYhzkUvkwsWDxsXaqlwQ8=; b=EX+/AYKnegEzkG2ezZMqO0wxExlTQw2Nj7m2bhDHSLsIXn36HUtJ6JhazEEh9ob3XZ 9CmFjp31Je7nhVIH/GVyzvjp0y4XcVUuzcqeIojau3qzBLoeDVR4fOOAq9dVHkU2KZ5n DJp0tvW5EqWtIp8Wwnyk350qcszA9bNNkylXoqGnk+2qrBgyeZU4YXxenAxdchVwbET4 TMt1ph+XcypoEj7a2jyWoUugUUfGcVOjYjXokH2KE8rpX1EzDSowUwg5SSh/iqz5OvNG jT262eR0UhWUxBQAHVmyduyKyj27QV/imSXEttOrjzdPtn28UZt+wG/IvubU/rgeO5oV notg== X-Gm-Message-State: AO0yUKWK5km27SxqUu8IPVjSz9HytrmxwJNzRKBEqv1m92Hw5NLsQK3e rxa8ZdDigtMbLJ0hS8MHsUuLVK4r/+qru1I9gfQ2K/IObEexUtW9+Q0qb0mS41Zu2uwLJgFf2kc VZFU+IqZXaJ1ybT8wdq8TJY3w X-Received: by 2002:a05:622a:14a:b0:3bf:d0b1:433d with SMTP id v10-20020a05622a014a00b003bfd0b1433dmr17172913qtw.60.1677756871625; Thu, 02 Mar 2023 03:34:31 -0800 (PST) X-Received: by 2002:a05:622a:14a:b0:3bf:d0b1:433d with SMTP id v10-20020a05622a014a00b003bfd0b1433dmr17172896qtw.60.1677756871371; Thu, 02 Mar 2023 03:34:31 -0800 (PST) Received: from step1.redhat.com (c-115-213.cust-q.wadsl.it. [212.43.115.213]) by smtp.gmail.com with ESMTPSA id o12-20020ac8698c000000b003ba19e53e43sm10084156qtq.25.2023.03.02.03.34.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Mar 2023 03:34:30 -0800 (PST) From: Stefano Garzarella To: virtualization@lists.linux-foundation.org Cc: Andrey Zhadchenko , eperezma@redhat.com, netdev@vger.kernel.org, stefanha@redhat.com, linux-kernel@vger.kernel.org, Jason Wang , "Michael S. Tsirkin" , kvm@vger.kernel.org, Stefano Garzarella Subject: [PATCH v2 1/8] vdpa: add bind_mm/unbind_mm callbacks Date: Thu, 2 Mar 2023 12:34:14 +0100 Message-Id: <20230302113421.174582-2-sgarzare@redhat.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230302113421.174582-1-sgarzare@redhat.com> References: <20230302113421.174582-1-sgarzare@redhat.com> MIME-Version: 1.0 Content-type: text/plain X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1759255813035844949?= X-GMAIL-MSGID: =?utf-8?q?1759255813035844949?= These new optional callbacks is used to bind/unbind the device to a specific address space so the vDPA framework can use VA when these callbacks are implemented. Suggested-by: Jason Wang Signed-off-by: Stefano Garzarella --- Notes: v2: - removed `struct task_struct *owner` param (unused for now, maybe useful to support cgroups) [Jason] - add unbind_mm callback [Jason] include/linux/vdpa.h | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/include/linux/vdpa.h b/include/linux/vdpa.h index 43f59ef10cc9..369c21394284 100644 --- a/include/linux/vdpa.h +++ b/include/linux/vdpa.h @@ -290,6 +290,14 @@ struct vdpa_map_file { * @vdev: vdpa device * @idx: virtqueue index * Returns pointer to structure device or error (NULL) + * @bind_mm: Bind the device to a specific address space + * so the vDPA framework can use VA when this + * callback is implemented. (optional) + * @vdev: vdpa device + * @mm: address space to bind + * @unbind_mm: Unbind the device from the address space + * bound using the bind_mm callback. (optional) + * @vdev: vdpa device * @free: Free resources that belongs to vDPA (optional) * @vdev: vdpa device */ @@ -351,6 +359,8 @@ struct vdpa_config_ops { int (*set_group_asid)(struct vdpa_device *vdev, unsigned int group, unsigned int asid); struct device *(*get_vq_dma_dev)(struct vdpa_device *vdev, u16 idx); + int (*bind_mm)(struct vdpa_device *vdev, struct mm_struct *mm); + void (*unbind_mm)(struct vdpa_device *vdev); /* Free device resources */ void (*free)(struct vdpa_device *vdev); From patchwork Thu Mar 2 11:34:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefano Garzarella X-Patchwork-Id: 63362 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp4184592wrd; Thu, 2 Mar 2023 03:37:39 -0800 (PST) X-Google-Smtp-Source: AK7set8j3P/nLVuHyN4BWiG2+e7SHqRLoI4EUzVKFLHEUSPpz6TdAZF0haQkW+W8DiSegdRMoY8V X-Received: by 2002:a17:906:4887:b0:8af:54d2:36af with SMTP id v7-20020a170906488700b008af54d236afmr9246189ejq.76.1677757059417; Thu, 02 Mar 2023 03:37:39 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1677757059; cv=none; d=google.com; s=arc-20160816; b=ZciHXauRrgJflKTTdpSy+G7Cz/s0LBa7YwzCcTKy1AeOzDrCvsx018J7YMCsGnfM09 miLrAiQTjRPThmqcVAFmQTzn3wBDU4dqyc7SjXK3sQhgaYqdTQjHdXNmLtctzAVsR2U4 W9Bvdwx4r1aQSSHFAdTFWGVrR+Wn6QLPvl+5oDGU8wQ5X3WLDJ0jU7fHnb4eUyIEmre7 gPQOKDrCdDaQIin6eKe0iQRZSVR6gMpSSOiuUeAZI19SrfmIkm/ivQ48dTVdxF8AIypY dx7AMhCxMbsj0sR1/P+uTt1aYbkOeQ68vu7KBttKjiqj55OX8WngMbkakvC+wslNlSYS 6/pg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=iYIRvQhreJmamPTgF10vjkMAMOhZQaOaX6sUzVAfpfw=; b=A5wOJ7wFvsgAdFoZuNv7Ju8dkuCoP8DUahfXiE0W60ggBcoLMpmH3dCbP9C15IlK/N Ub6DZHdA2QG94pnfQyiUCgBOpt2TrNsUnUlDECrrNtAJb42B4vCbFrue/3x6bBiU4wpJ gSR0waEh5wKxh/imnXnmhJOhOrx4tZQdAZKstsVaahPMsJN9zzDyAPjVD8/S0j18OWxG Iwg8g4afcXN/8pMCyBIjttXXlag6woBmIC+SSlT7FvkcP41kw9Pua33QEnz0x4pI96Cq Tmz4R9WyjFkpHKNylKHDuEXgK7U61uqingOK+4rWq2oVT1XYPQOturb3rlNyzV/a5GuO MNbg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=hQManXAS; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id j12-20020a170906830c00b008b17e6556c3si15960021ejx.587.2023.03.02.03.37.16; Thu, 02 Mar 2023 03:37:39 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=hQManXAS; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229899AbjCBLfm (ORCPT + 99 others); Thu, 2 Mar 2023 06:35:42 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55108 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229792AbjCBLf1 (ORCPT ); Thu, 2 Mar 2023 06:35:27 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 94E3E227AF for ; Thu, 2 Mar 2023 03:34:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1677756876; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iYIRvQhreJmamPTgF10vjkMAMOhZQaOaX6sUzVAfpfw=; b=hQManXAScAKWA8mgFefT9Uv+XU0ZH8HiL2J6k7qE72pVd/cT+KFy5MDd3ttUqAL4aNRLM9 OHLmtGUnwE8v1YrbAdkK4dILv8AvlxcTjr2i2Lqlgkl4+G0zR09KWgDTWH2OTyBoI+fcuL u+KYgUX7mWzqoyosTbdovsMBY49gLdQ= Received: from mail-qt1-f200.google.com (mail-qt1-f200.google.com [209.85.160.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-150-1EnjUKfBP8y_cjlR2BTFIQ-1; Thu, 02 Mar 2023 06:34:35 -0500 X-MC-Unique: 1EnjUKfBP8y_cjlR2BTFIQ-1 Received: by mail-qt1-f200.google.com with SMTP id o10-20020a05622a138a00b003bfdabf3b89so6594020qtk.13 for ; Thu, 02 Mar 2023 03:34:35 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1677756875; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=iYIRvQhreJmamPTgF10vjkMAMOhZQaOaX6sUzVAfpfw=; b=6n6aUiJpk1ioK7488R2e83v0uX1TO6cQRukeI0IG3nrKxc3vXXf6XVqIYvFFRr742P QEhIhKxo0v5cTTD5CDnqTJz4H97vvhq0Df/XkxokGGDkZr4fmxWWUfvc+5Bf3mp2JNTu JEOFd2uNh2a5LCLYrDJ/EDC76gT4cXgC0RnkWet7dXdHIFCFdWJzj0rQaGfRg5VxZKrD ZoHKteAMCnjuVM4uJ3oPT0SUZ+YsQm8ELxABp9kEBQUXngn+zLPxBUjK2Kd6htk5nzgb F0Y1dzUlwFeBIkyKoAds2g+2RtDnGP2BgJTMCdwoRzk6clV8l68jihEJLC2kfzbVECw2 UbxQ== X-Gm-Message-State: AO0yUKVIoqZmkGBadCO86GAGPECDI20QVPKASOOqsnoHURymXeRgYKt+ 1SJPJJ6k8Wt2AunjNkmQUOaUB1eg7sppYlWUefqSfEvAK8CbXieOrdXZTr53zZViy05kckKG64F 9EHhH7Wal4IJJE4BRVmI6OZMa X-Received: by 2002:a05:622a:186:b0:3bf:d4c3:365d with SMTP id s6-20020a05622a018600b003bfd4c3365dmr17450754qtw.14.1677756875320; Thu, 02 Mar 2023 03:34:35 -0800 (PST) X-Received: by 2002:a05:622a:186:b0:3bf:d4c3:365d with SMTP id s6-20020a05622a018600b003bfd4c3365dmr17450741qtw.14.1677756875094; Thu, 02 Mar 2023 03:34:35 -0800 (PST) Received: from step1.redhat.com (c-115-213.cust-q.wadsl.it. [212.43.115.213]) by smtp.gmail.com with ESMTPSA id o12-20020ac8698c000000b003ba19e53e43sm10084156qtq.25.2023.03.02.03.34.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Mar 2023 03:34:34 -0800 (PST) From: Stefano Garzarella To: virtualization@lists.linux-foundation.org Cc: Andrey Zhadchenko , eperezma@redhat.com, netdev@vger.kernel.org, stefanha@redhat.com, linux-kernel@vger.kernel.org, Jason Wang , "Michael S. Tsirkin" , kvm@vger.kernel.org, Stefano Garzarella Subject: [PATCH v2 2/8] vhost-vdpa: use bind_mm/unbind_mm device callbacks Date: Thu, 2 Mar 2023 12:34:15 +0100 Message-Id: <20230302113421.174582-3-sgarzare@redhat.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230302113421.174582-1-sgarzare@redhat.com> References: <20230302113421.174582-1-sgarzare@redhat.com> MIME-Version: 1.0 Content-type: text/plain X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1759255786046129006?= X-GMAIL-MSGID: =?utf-8?q?1759255786046129006?= When the user call VHOST_SET_OWNER ioctl and the vDPA device has `use_va` set to true, let's call the bind_mm callback. In this way we can bind the device to the user address space and directly use the user VA. The unbind_mm callback is called during the release after stopping the device. Signed-off-by: Stefano Garzarella --- Notes: v2: - call the new unbind_mm callback during the release [Jason] - avoid to call bind_mm callback after the reset, since the device is not detaching it now during the reset drivers/vhost/vdpa.c | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c index dc12dbd5b43b..1ab89fccd825 100644 --- a/drivers/vhost/vdpa.c +++ b/drivers/vhost/vdpa.c @@ -219,6 +219,28 @@ static int vhost_vdpa_reset(struct vhost_vdpa *v) return vdpa_reset(vdpa); } +static long vhost_vdpa_bind_mm(struct vhost_vdpa *v) +{ + struct vdpa_device *vdpa = v->vdpa; + const struct vdpa_config_ops *ops = vdpa->config; + + if (!vdpa->use_va || !ops->bind_mm) + return 0; + + return ops->bind_mm(vdpa, v->vdev.mm); +} + +static void vhost_vdpa_unbind_mm(struct vhost_vdpa *v) +{ + struct vdpa_device *vdpa = v->vdpa; + const struct vdpa_config_ops *ops = vdpa->config; + + if (!vdpa->use_va || !ops->unbind_mm) + return; + + ops->unbind_mm(vdpa); +} + static long vhost_vdpa_get_device_id(struct vhost_vdpa *v, u8 __user *argp) { struct vdpa_device *vdpa = v->vdpa; @@ -711,6 +733,13 @@ static long vhost_vdpa_unlocked_ioctl(struct file *filep, break; default: r = vhost_dev_ioctl(&v->vdev, cmd, argp); + if (!r && cmd == VHOST_SET_OWNER) { + r = vhost_vdpa_bind_mm(v); + if (r) { + vhost_dev_reset_owner(&v->vdev, NULL); + break; + } + } if (r == -ENOIOCTLCMD) r = vhost_vdpa_vring_ioctl(v, cmd, argp); break; @@ -1285,6 +1314,7 @@ static int vhost_vdpa_release(struct inode *inode, struct file *filep) vhost_vdpa_clean_irq(v); vhost_vdpa_reset(v); vhost_dev_stop(&v->vdev); + vhost_vdpa_unbind_mm(v); vhost_vdpa_free_domain(v); vhost_vdpa_config_put(v); vhost_vdpa_cleanup(v); From patchwork Thu Mar 2 11:34:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefano Garzarella X-Patchwork-Id: 63367 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp4185272wrd; Thu, 2 Mar 2023 03:39:18 -0800 (PST) X-Google-Smtp-Source: AK7set8qPYKzyT+hTno7YCNCYO9J4sFihhY0VNKpL4bSLXwZHbOOWvOJlVO3tUsvrtwt/6IbV8f0 X-Received: by 2002:aa7:c498:0:b0:474:a583:2e1a with SMTP id m24-20020aa7c498000000b00474a5832e1amr9090270edq.12.1677757158287; Thu, 02 Mar 2023 03:39:18 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1677757158; cv=none; d=google.com; s=arc-20160816; b=qX5tk/lIgX2YkvxmXUO2y4DN6ciTwSo0QHMllf25YuhIrDMqwCNyaj4MvkT7T2tYif Q0Wc6H1Lc/lRh6HPbajPwHENfd4acV9Dp4sjiec1PTw86bpLtE8o587V+YaRd5ebvj8O 7xednDEQ35sp5XWH5gtAgIFoyOea80/b12JNI0sGF3FMQ0xY4Rz/xEsYnXTniZO6CSXW sNjRA2yJcs/JQ2EOkJqd/gvBliR2adfSsjE8U/us0aNzdcSi9hbBvZujM7tUcWBUQpXq KV9J0sIDFcWbFD5Gfdzwahw2syPHecOrFbTQhMWLXo+ojCmFj5ddmDAZW1Q0t8/Ml1Vf nZTw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=nMUvg2hEs0iMx2dm3/Jg9b5hv2Rlumo7TQ+47EWr+7I=; b=TUppjhVY5c2snKKeHTepvb+Xnvl7R2BTGc5HxnV01PvpyAI6Nsamf6aFkkMS280ClE bUG/JGNOziVXgfZrSonnKrnuFCRamcVuCnypB7OYkXPXEtIAWv4ezYcA40NoJq0+xYKU tglz0iwU2SvW2RqP2JFmQ3sC0ZazoT1bEuHiBPteNJYlYTYnhgJmytOvUhMixW10Kf/G /iwS8E/hW6qwGZzNxydmC357AEbY0qzPLvJNhjSZIArjWXLrM2MncpF3sJPz55RpDOlt HXQEbl9ori9fpEt2S5S+y8cC8j2rNvusOdWHP2YKCy5mUynoOQf1R41FoJkKuJENMMzF WQVg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=eqM4MKD4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id e18-20020a056402089200b004aaa653a74asi7525832edy.596.2023.03.02.03.38.55; Thu, 02 Mar 2023 03:39:18 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=eqM4MKD4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229592AbjCBLfr (ORCPT + 99 others); Thu, 2 Mar 2023 06:35:47 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55214 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229831AbjCBLfe (ORCPT ); Thu, 2 Mar 2023 06:35:34 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9509B1517F for ; Thu, 2 Mar 2023 03:34:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1677756880; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nMUvg2hEs0iMx2dm3/Jg9b5hv2Rlumo7TQ+47EWr+7I=; b=eqM4MKD4lbhRF16c0egtl+jxhsbdRBQsXRI4OHQkwG+VozXKm3UxGokgCw0Zq5uPLEPCfu NXl5vgLB/aDOCv+Zsp8zMHfkgbP7oZcEWgWf/q/o3GiDfb2GTqItxf4LidNxasFa+rhfYz 2ej4Im32XE4zOHKneIupZ+1GCn7w7rM= Received: from mail-qv1-f69.google.com (mail-qv1-f69.google.com [209.85.219.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-634-6luZiEUZPIePmsJYjtSOYg-1; Thu, 02 Mar 2023 06:34:39 -0500 X-MC-Unique: 6luZiEUZPIePmsJYjtSOYg-1 Received: by mail-qv1-f69.google.com with SMTP id y3-20020a0cec03000000b0056ee5b123bbso8740724qvo.7 for ; Thu, 02 Mar 2023 03:34:39 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1677756879; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nMUvg2hEs0iMx2dm3/Jg9b5hv2Rlumo7TQ+47EWr+7I=; b=Ulh+wiMhKXTnEjeOJLA/7uXFxjN8HBI+5E/aMRxir6f8kh9FgPchZy+AQw8htRF+WV +co6DCjbWvxpW+KP+QbTp6Jae7mFWHxG4yVstkVpA9trLJ5pOlZdOxScmn0ceLQ8cx85 nL20Mp07tqzyBVnr3DLgie4+MWhhQkmp9RdA6UxSYyFfYImeaNzOUcHJ84ixGyRhFRDM doiB6T8yC6uBIFHUuEFrMibW/Yzskzclu2HIPgsRfaBlwOnmf9yW1QsyD/rjqUoY+5dX jqFGAWOtjg7EkpD20UuHG8J52OU4RwulQpWHUJNMfaPoI+xJ6zS2B76P7NaQEujXzrDy k6Xw== X-Gm-Message-State: AO0yUKW3PejyuFnr4P57wWa4Exk01yf59/w58wFk386xA9VT9wfOVp0n wI/AelEh1KBCUlAOI3LAShk9HKDEnCVNfwVkKhHoCA1vHVmrJjLI+qRIA8oZ3LzKV2JXf9dZCld N6WDJgHyZVw/xDZviFsEIjRgX X-Received: by 2002:ac8:5850:0:b0:3bf:d6ad:5236 with SMTP id h16-20020ac85850000000b003bfd6ad5236mr15655317qth.32.1677756879136; Thu, 02 Mar 2023 03:34:39 -0800 (PST) X-Received: by 2002:ac8:5850:0:b0:3bf:d6ad:5236 with SMTP id h16-20020ac85850000000b003bfd6ad5236mr15655295qth.32.1677756878884; Thu, 02 Mar 2023 03:34:38 -0800 (PST) Received: from step1.redhat.com (c-115-213.cust-q.wadsl.it. [212.43.115.213]) by smtp.gmail.com with ESMTPSA id o12-20020ac8698c000000b003ba19e53e43sm10084156qtq.25.2023.03.02.03.34.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Mar 2023 03:34:38 -0800 (PST) From: Stefano Garzarella To: virtualization@lists.linux-foundation.org Cc: Andrey Zhadchenko , eperezma@redhat.com, netdev@vger.kernel.org, stefanha@redhat.com, linux-kernel@vger.kernel.org, Jason Wang , "Michael S. Tsirkin" , kvm@vger.kernel.org, Stefano Garzarella Subject: [PATCH v2 3/8] vringh: replace kmap_atomic() with kmap_local_page() Date: Thu, 2 Mar 2023 12:34:16 +0100 Message-Id: <20230302113421.174582-4-sgarzare@redhat.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230302113421.174582-1-sgarzare@redhat.com> References: <20230302113421.174582-1-sgarzare@redhat.com> MIME-Version: 1.0 Content-type: text/plain X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1759255890085488115?= X-GMAIL-MSGID: =?utf-8?q?1759255890085488115?= kmap_atomic() is deprecated in favor of kmap_local_page(). With kmap_local_page() the mappings are per thread, CPU local, can take page-faults, and can be called from any context (including interrupts). Furthermore, the tasks can be preempted and, when they are scheduled to run again, the kernel virtual addresses are restored and still valid. kmap_atomic() is implemented like a kmap_local_page() which also disables page-faults and preemption (the latter only for !PREEMPT_RT kernels, otherwise it only disables migration). The code within the mappings/un-mappings in getu16_iotlb() and putu16_iotlb() don't depend on the above-mentioned side effects of kmap_atomic(), so that mere replacements of the old API with the new one is all that is required (i.e., there is no need to explicitly add calls to pagefault_disable() and/or preempt_disable()). Signed-off-by: Stefano Garzarella --- Notes: v2: - added this patch since checkpatch.pl complained about deprecation of kmap_atomic() touched by next patch drivers/vhost/vringh.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/drivers/vhost/vringh.c b/drivers/vhost/vringh.c index a1e27da54481..0ba3ef809e48 100644 --- a/drivers/vhost/vringh.c +++ b/drivers/vhost/vringh.c @@ -1220,10 +1220,10 @@ static inline int getu16_iotlb(const struct vringh *vrh, if (ret < 0) return ret; - kaddr = kmap_atomic(iov.bv_page); + kaddr = kmap_local_page(iov.bv_page); from = kaddr + iov.bv_offset; *val = vringh16_to_cpu(vrh, READ_ONCE(*(__virtio16 *)from)); - kunmap_atomic(kaddr); + kunmap_local(kaddr); return 0; } @@ -1241,10 +1241,10 @@ static inline int putu16_iotlb(const struct vringh *vrh, if (ret < 0) return ret; - kaddr = kmap_atomic(iov.bv_page); + kaddr = kmap_local_page(iov.bv_page); to = kaddr + iov.bv_offset; WRITE_ONCE(*(__virtio16 *)to, cpu_to_vringh16(vrh, val)); - kunmap_atomic(kaddr); + kunmap_local(kaddr); return 0; } From patchwork Thu Mar 2 11:34:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Stefano Garzarella X-Patchwork-Id: 63369 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp4185636wrd; Thu, 2 Mar 2023 03:40:06 -0800 (PST) X-Google-Smtp-Source: AK7set8WLFpSBqT+zWXHL1xvXwYAYex3p1Hc2DIJL85vN1sKh2G1S6K2khTs4ppah93Grm+4UupG X-Received: by 2002:a05:6402:6c1:b0:4bf:1989:9e87 with SMTP id n1-20020a05640206c100b004bf19899e87mr3902424edy.20.1677757206133; Thu, 02 Mar 2023 03:40:06 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1677757206; cv=none; d=google.com; s=arc-20160816; b=PxN04kSXW5C8sRvlyqF/WVjmuEHIKBzC780r6YfaIEvRaFrbXSI7/KoOKI33tu1X8U KJB/2+ePY6EH4vcKVRhcAQGRK9nVNyOFivjvnQRiq+Azv9pRQQj3zJjswn02SV3ailgd aPEJJs6vpQ+LzCowowCbgIKpmj4wGY8Vrxx/keJ24yvr0QZFCd2CZhXg3W9+Ii+TRMD1 fnWtJNHuM5I6boJRJqC+i9X3wlliA6TEF2H5lGcqdQunLVv5kdgw5QK2HgzkL+0822X4 c/s/KLQg8i9bCJjDj3k4PD2N6W0ocy3BMBy+BjZosnwiiIhEYiZIWqxfozC/pVBSLmiG laSA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Wr7wXnFCQpY1zpniPAgMpMPx/fjhLmfIKoC/NtXm0MU=; b=rxPeL1vmGnkMYrBvOhOkbBeDvlLmDV+9ItaXYdUKfqJlEJfpCFeqY3tGaGQxUpS8p1 s9etWblmUiqvPBJ7Km1QpDQlka1d6WOzonLg10xoaZJoEU52oEWKV2a0GPXmIrMdVONG 7+AoWISnfeyn032pwBl7tsAsucXqdxcxwX/yY+LoF3UWkKV9CSurtfj81kWytH4DKnoT ymQPTQ79Yv/4IFrM93ClGiSsKnTvp3hozQf/Y2RZuuLMu9a+aox4mxuCXRBOl54ji9xy 54szwRlNYSstQg4qNIriyP3ohVPiXlvt71XOKsqGnP6uOaJ+ZZbVMMIVX7TYJWmxpUqD HLFw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=a5MJvr3w; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id y14-20020aa7d50e000000b004c10ad4e6aesi677863edq.145.2023.03.02.03.39.43; Thu, 02 Mar 2023 03:40:06 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=a5MJvr3w; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229986AbjCBLg3 (ORCPT + 99 others); Thu, 2 Mar 2023 06:36:29 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55402 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229977AbjCBLgB (ORCPT ); Thu, 2 Mar 2023 06:36:01 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 702BA977C for ; Thu, 2 Mar 2023 03:35:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1677756908; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Wr7wXnFCQpY1zpniPAgMpMPx/fjhLmfIKoC/NtXm0MU=; b=a5MJvr3wg9+1nqth+sk2jp4+vv1BAKm/j2EBHz7OTK42R6sygI+5yJzC3OgTjv5y2dI6Gv IXAl7J3CZXIAWWkq/cN0OO32CFOsQ60c3+aUMS4gQnH4KkC/tRtfJMD96znM6lqdtXPps+ EQQDo9oqZH+M8t89zJz+PKREW1urnqs= Received: from mail-qv1-f70.google.com (mail-qv1-f70.google.com [209.85.219.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-258-RwKLwvlAPMmFzQWt_QD1cQ-1; Thu, 02 Mar 2023 06:35:07 -0500 X-MC-Unique: RwKLwvlAPMmFzQWt_QD1cQ-1 Received: by mail-qv1-f70.google.com with SMTP id l13-20020ad44d0d000000b004c74bbb0affso8636417qvl.21 for ; Thu, 02 Mar 2023 03:35:07 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1677756905; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Wr7wXnFCQpY1zpniPAgMpMPx/fjhLmfIKoC/NtXm0MU=; b=hrqzMJZJdzWOBNmRYR6NEqHUxfy3EwPEmqrkPmRkhlk5PWB20fua/KhSydXhJ82+KP tmhcHgeQuL8tqejFJInaPAlueI6EEqS2H7M+7OWiFaVd5XRW0T1ATSsbAExAF4gOCZoD o0js7X46+s6WOpeIBra9AO6orQIaSm6TvKG4IuttynADbSIlLG5TUeBxrVYMaCIiZvi3 50APeZrNW5io1bpFi+Zv/IlnEPzOEyWDG/4fDNEMRPQkGBVh1xqRrwE5ovTsVpBhDNJm J8VjjWdYDtvDoJhSYHQTlHojRjx9b/7IOQHNUpV/Uxop9zi/+f6btSHli+feR0tHfkEy dkiw== X-Gm-Message-State: AO0yUKWJzxiwfT69Q0YB1r06jvbs0G3KeRkFdC2hSlJF7kfJ1CqkehmY 6kcUs6nYGbeeAaKIoj9QYHPEMvU4YjOYwltigoK2Mf2Xww6X1Zr4oE8luE4lvZeMdGRjPLynL5y Cn/DnYbbYiYlitY3Y0uxSGibW X-Received: by 2002:a05:6214:20c1:b0:56e:a556:f28f with SMTP id 1-20020a05621420c100b0056ea556f28fmr18887338qve.34.1677756905280; Thu, 02 Mar 2023 03:35:05 -0800 (PST) X-Received: by 2002:a05:6214:20c1:b0:56e:a556:f28f with SMTP id 1-20020a05621420c100b0056ea556f28fmr18887295qve.34.1677756904884; Thu, 02 Mar 2023 03:35:04 -0800 (PST) Received: from step1.redhat.com (c-115-213.cust-q.wadsl.it. [212.43.115.213]) by smtp.gmail.com with ESMTPSA id p12-20020a37420c000000b007426ec97253sm8383058qka.111.2023.03.02.03.35.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Mar 2023 03:35:04 -0800 (PST) From: Stefano Garzarella To: virtualization@lists.linux-foundation.org Cc: Andrey Zhadchenko , eperezma@redhat.com, netdev@vger.kernel.org, stefanha@redhat.com, linux-kernel@vger.kernel.org, Jason Wang , "Michael S. Tsirkin" , kvm@vger.kernel.org, Stefano Garzarella Subject: [PATCH v2 4/8] vringh: support VA with iotlb Date: Thu, 2 Mar 2023 12:34:17 +0100 Message-Id: <20230302113421.174582-5-sgarzare@redhat.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230302113421.174582-1-sgarzare@redhat.com> References: <20230302113421.174582-1-sgarzare@redhat.com> MIME-Version: 1.0 Content-type: text/plain X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1759255940227734387?= X-GMAIL-MSGID: =?utf-8?q?1759255940227734387?= vDPA supports the possibility to use user VA in the iotlb messages. So, let's add support for user VA in vringh to use it in the vDPA simulators. Signed-off-by: Stefano Garzarella Acked-by: Eugenio Pérez Martin --- Notes: v2: - replace kmap_atomic() with kmap_local_page() [see previous patch] - fix cast warnings when build with W=1 C=1 include/linux/vringh.h | 5 +- drivers/vdpa/mlx5/net/mlx5_vnet.c | 2 +- drivers/vdpa/vdpa_sim/vdpa_sim.c | 4 +- drivers/vhost/vringh.c | 247 ++++++++++++++++++++++++------ 4 files changed, 205 insertions(+), 53 deletions(-) diff --git a/include/linux/vringh.h b/include/linux/vringh.h index 1991a02c6431..d39b9f2dcba0 100644 --- a/include/linux/vringh.h +++ b/include/linux/vringh.h @@ -32,6 +32,9 @@ struct vringh { /* Can we get away with weak barriers? */ bool weak_barriers; + /* Use user's VA */ + bool use_va; + /* Last available index we saw (ie. where we're up to). */ u16 last_avail_idx; @@ -279,7 +282,7 @@ void vringh_set_iotlb(struct vringh *vrh, struct vhost_iotlb *iotlb, spinlock_t *iotlb_lock); int vringh_init_iotlb(struct vringh *vrh, u64 features, - unsigned int num, bool weak_barriers, + unsigned int num, bool weak_barriers, bool use_va, struct vring_desc *desc, struct vring_avail *avail, struct vring_used *used); diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5_vnet.c index 3a0e721aef05..babc8dd171a6 100644 --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c @@ -2537,7 +2537,7 @@ static int setup_cvq_vring(struct mlx5_vdpa_dev *mvdev) if (mvdev->actual_features & BIT_ULL(VIRTIO_NET_F_CTRL_VQ)) err = vringh_init_iotlb(&cvq->vring, mvdev->actual_features, - MLX5_CVQ_MAX_ENT, false, + MLX5_CVQ_MAX_ENT, false, false, (struct vring_desc *)(uintptr_t)cvq->desc_addr, (struct vring_avail *)(uintptr_t)cvq->driver_addr, (struct vring_used *)(uintptr_t)cvq->device_addr); diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.c b/drivers/vdpa/vdpa_sim/vdpa_sim.c index 6a0a65814626..481eb156658b 100644 --- a/drivers/vdpa/vdpa_sim/vdpa_sim.c +++ b/drivers/vdpa/vdpa_sim/vdpa_sim.c @@ -60,7 +60,7 @@ static void vdpasim_queue_ready(struct vdpasim *vdpasim, unsigned int idx) struct vdpasim_virtqueue *vq = &vdpasim->vqs[idx]; uint16_t last_avail_idx = vq->vring.last_avail_idx; - vringh_init_iotlb(&vq->vring, vdpasim->features, vq->num, true, + vringh_init_iotlb(&vq->vring, vdpasim->features, vq->num, true, false, (struct vring_desc *)(uintptr_t)vq->desc_addr, (struct vring_avail *) (uintptr_t)vq->driver_addr, @@ -81,7 +81,7 @@ static void vdpasim_vq_reset(struct vdpasim *vdpasim, vq->cb = NULL; vq->private = NULL; vringh_init_iotlb(&vq->vring, vdpasim->dev_attr.supported_features, - VDPASIM_QUEUE_MAX, false, NULL, NULL, NULL); + VDPASIM_QUEUE_MAX, false, false, NULL, NULL, NULL); vq->vring.notify = NULL; } diff --git a/drivers/vhost/vringh.c b/drivers/vhost/vringh.c index 0ba3ef809e48..61c79cea44ca 100644 --- a/drivers/vhost/vringh.c +++ b/drivers/vhost/vringh.c @@ -1094,15 +1094,99 @@ EXPORT_SYMBOL(vringh_need_notify_kern); #if IS_REACHABLE(CONFIG_VHOST_IOTLB) -static int iotlb_translate(const struct vringh *vrh, - u64 addr, u64 len, u64 *translated, - struct bio_vec iov[], - int iov_size, u32 perm) +static int iotlb_translate_va(const struct vringh *vrh, + u64 addr, u64 len, u64 *translated, + struct iovec iov[], + int iov_size, u32 perm) { struct vhost_iotlb_map *map; struct vhost_iotlb *iotlb = vrh->iotlb; + u64 s = 0, last = addr + len - 1; int ret = 0; + + spin_lock(vrh->iotlb_lock); + + while (len > s) { + u64 size; + + if (unlikely(ret >= iov_size)) { + ret = -ENOBUFS; + break; + } + + map = vhost_iotlb_itree_first(iotlb, addr, last); + if (!map || map->start > addr) { + ret = -EINVAL; + break; + } else if (!(map->perm & perm)) { + ret = -EPERM; + break; + } + + size = map->size - addr + map->start; + iov[ret].iov_len = min(len - s, size); + iov[ret].iov_base = (void __user *)(unsigned long) + (map->addr + addr - map->start); + s += size; + addr += size; + ++ret; + } + + spin_unlock(vrh->iotlb_lock); + + if (translated) + *translated = min(len, s); + + return ret; +} + +static inline int copy_from_va(const struct vringh *vrh, void *dst, void *src, + u64 len, u64 *translated) +{ + struct iovec iov[16]; + struct iov_iter iter; + int ret; + + ret = iotlb_translate_va(vrh, (u64)(uintptr_t)src, len, translated, iov, + ARRAY_SIZE(iov), VHOST_MAP_RO); + if (ret == -ENOBUFS) + ret = ARRAY_SIZE(iov); + else if (ret < 0) + return ret; + + iov_iter_init(&iter, ITER_SOURCE, iov, ret, *translated); + + return copy_from_iter(dst, *translated, &iter); +} + +static inline int copy_to_va(const struct vringh *vrh, void *dst, void *src, + u64 len, u64 *translated) +{ + struct iovec iov[16]; + struct iov_iter iter; + int ret; + + ret = iotlb_translate_va(vrh, (u64)(uintptr_t)dst, len, translated, iov, + ARRAY_SIZE(iov), VHOST_MAP_WO); + if (ret == -ENOBUFS) + ret = ARRAY_SIZE(iov); + else if (ret < 0) + return ret; + + iov_iter_init(&iter, ITER_DEST, iov, ret, *translated); + + return copy_to_iter(src, *translated, &iter); +} + +static int iotlb_translate_pa(const struct vringh *vrh, + u64 addr, u64 len, u64 *translated, + struct bio_vec iov[], + int iov_size, u32 perm) +{ + struct vhost_iotlb_map *map; + struct vhost_iotlb *iotlb = vrh->iotlb; u64 s = 0, last = addr + len - 1; + int ret = 0; spin_lock(vrh->iotlb_lock); @@ -1141,28 +1225,61 @@ static int iotlb_translate(const struct vringh *vrh, return ret; } +static inline int copy_from_pa(const struct vringh *vrh, void *dst, void *src, + u64 len, u64 *translated) +{ + struct bio_vec iov[16]; + struct iov_iter iter; + int ret; + + ret = iotlb_translate_pa(vrh, (u64)(uintptr_t)src, len, translated, iov, + ARRAY_SIZE(iov), VHOST_MAP_RO); + if (ret == -ENOBUFS) + ret = ARRAY_SIZE(iov); + else if (ret < 0) + return ret; + + iov_iter_bvec(&iter, ITER_SOURCE, iov, ret, *translated); + + return copy_from_iter(dst, *translated, &iter); +} + +static inline int copy_to_pa(const struct vringh *vrh, void *dst, void *src, + u64 len, u64 *translated) +{ + struct bio_vec iov[16]; + struct iov_iter iter; + int ret; + + ret = iotlb_translate_pa(vrh, (u64)(uintptr_t)dst, len, translated, iov, + ARRAY_SIZE(iov), VHOST_MAP_WO); + if (ret == -ENOBUFS) + ret = ARRAY_SIZE(iov); + else if (ret < 0) + return ret; + + iov_iter_bvec(&iter, ITER_DEST, iov, ret, *translated); + + return copy_to_iter(src, *translated, &iter); +} + static inline int copy_from_iotlb(const struct vringh *vrh, void *dst, void *src, size_t len) { u64 total_translated = 0; while (total_translated < len) { - struct bio_vec iov[16]; - struct iov_iter iter; u64 translated; int ret; - ret = iotlb_translate(vrh, (u64)(uintptr_t)src, - len - total_translated, &translated, - iov, ARRAY_SIZE(iov), VHOST_MAP_RO); - if (ret == -ENOBUFS) - ret = ARRAY_SIZE(iov); - else if (ret < 0) - return ret; - - iov_iter_bvec(&iter, ITER_SOURCE, iov, ret, translated); + if (vrh->use_va) { + ret = copy_from_va(vrh, dst, src, + len - total_translated, &translated); + } else { + ret = copy_from_pa(vrh, dst, src, + len - total_translated, &translated); + } - ret = copy_from_iter(dst, translated, &iter); if (ret < 0) return ret; @@ -1180,22 +1297,17 @@ static inline int copy_to_iotlb(const struct vringh *vrh, void *dst, u64 total_translated = 0; while (total_translated < len) { - struct bio_vec iov[16]; - struct iov_iter iter; u64 translated; int ret; - ret = iotlb_translate(vrh, (u64)(uintptr_t)dst, - len - total_translated, &translated, - iov, ARRAY_SIZE(iov), VHOST_MAP_WO); - if (ret == -ENOBUFS) - ret = ARRAY_SIZE(iov); - else if (ret < 0) - return ret; - - iov_iter_bvec(&iter, ITER_DEST, iov, ret, translated); + if (vrh->use_va) { + ret = copy_to_va(vrh, dst, src, + len - total_translated, &translated); + } else { + ret = copy_to_pa(vrh, dst, src, + len - total_translated, &translated); + } - ret = copy_to_iter(src, translated, &iter); if (ret < 0) return ret; @@ -1210,20 +1322,37 @@ static inline int copy_to_iotlb(const struct vringh *vrh, void *dst, static inline int getu16_iotlb(const struct vringh *vrh, u16 *val, const __virtio16 *p) { - struct bio_vec iov; - void *kaddr, *from; int ret; /* Atomic read is needed for getu16 */ - ret = iotlb_translate(vrh, (u64)(uintptr_t)p, sizeof(*p), NULL, - &iov, 1, VHOST_MAP_RO); - if (ret < 0) - return ret; + if (vrh->use_va) { + struct iovec iov; + __virtio16 tmp; + + ret = iotlb_translate_va(vrh, (u64)(uintptr_t)p, sizeof(*p), + NULL, &iov, 1, VHOST_MAP_RO); + if (ret < 0) + return ret; - kaddr = kmap_local_page(iov.bv_page); - from = kaddr + iov.bv_offset; - *val = vringh16_to_cpu(vrh, READ_ONCE(*(__virtio16 *)from)); - kunmap_local(kaddr); + ret = __get_user(tmp, (__virtio16 __user *)iov.iov_base); + if (ret) + return ret; + + *val = vringh16_to_cpu(vrh, tmp); + } else { + struct bio_vec iov; + void *kaddr, *from; + + ret = iotlb_translate_pa(vrh, (u64)(uintptr_t)p, sizeof(*p), + NULL, &iov, 1, VHOST_MAP_RO); + if (ret < 0) + return ret; + + kaddr = kmap_local_page(iov.bv_page); + from = kaddr + iov.bv_offset; + *val = vringh16_to_cpu(vrh, READ_ONCE(*(__virtio16 *)from)); + kunmap_local(kaddr); + } return 0; } @@ -1231,20 +1360,37 @@ static inline int getu16_iotlb(const struct vringh *vrh, static inline int putu16_iotlb(const struct vringh *vrh, __virtio16 *p, u16 val) { - struct bio_vec iov; - void *kaddr, *to; int ret; /* Atomic write is needed for putu16 */ - ret = iotlb_translate(vrh, (u64)(uintptr_t)p, sizeof(*p), NULL, - &iov, 1, VHOST_MAP_WO); - if (ret < 0) - return ret; + if (vrh->use_va) { + struct iovec iov; + __virtio16 tmp; - kaddr = kmap_local_page(iov.bv_page); - to = kaddr + iov.bv_offset; - WRITE_ONCE(*(__virtio16 *)to, cpu_to_vringh16(vrh, val)); - kunmap_local(kaddr); + ret = iotlb_translate_va(vrh, (u64)(uintptr_t)p, sizeof(*p), + NULL, &iov, 1, VHOST_MAP_RO); + if (ret < 0) + return ret; + + tmp = cpu_to_vringh16(vrh, val); + + ret = __put_user(tmp, (__virtio16 __user *)iov.iov_base); + if (ret) + return ret; + } else { + struct bio_vec iov; + void *kaddr, *to; + + ret = iotlb_translate_pa(vrh, (u64)(uintptr_t)p, sizeof(*p), NULL, + &iov, 1, VHOST_MAP_WO); + if (ret < 0) + return ret; + + kaddr = kmap_local_page(iov.bv_page); + to = kaddr + iov.bv_offset; + WRITE_ONCE(*(__virtio16 *)to, cpu_to_vringh16(vrh, val)); + kunmap_local(kaddr); + } return 0; } @@ -1306,6 +1452,7 @@ static inline int putused_iotlb(const struct vringh *vrh, * @features: the feature bits for this ring. * @num: the number of elements. * @weak_barriers: true if we only need memory barriers, not I/O. + * @use_va: true if IOTLB contains user VA * @desc: the userpace descriptor pointer. * @avail: the userpace avail pointer. * @used: the userpace used pointer. @@ -1313,11 +1460,13 @@ static inline int putused_iotlb(const struct vringh *vrh, * Returns an error if num is invalid. */ int vringh_init_iotlb(struct vringh *vrh, u64 features, - unsigned int num, bool weak_barriers, + unsigned int num, bool weak_barriers, bool use_va, struct vring_desc *desc, struct vring_avail *avail, struct vring_used *used) { + vrh->use_va = use_va; + return vringh_init_kern(vrh, features, num, weak_barriers, desc, avail, used); } From patchwork Thu Mar 2 11:34:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Stefano Garzarella X-Patchwork-Id: 63364 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp4185056wrd; Thu, 2 Mar 2023 03:38:46 -0800 (PST) X-Google-Smtp-Source: AK7set+V+oDlkpXzisb6AvBmK410hby10TjTgZuWiD4i0EHNW6tbQ1qLcx6CB0hhJoLdE+0UcZE5 X-Received: by 2002:a17:907:7898:b0:8b1:7857:2331 with SMTP id ku24-20020a170907789800b008b178572331mr9133727ejc.65.1677757126225; Thu, 02 Mar 2023 03:38:46 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1677757126; cv=none; d=google.com; s=arc-20160816; b=lgqRjgMNBzUVmhMvwNlXcjKJlUD3FQeVbvWkTKIhL0CWxMDV9ztfstqPuWA2OIOdod EJ6qzoWuk4wSuHJz3RN4xkv45JB8kZDuresvY9TJlO/rR1OLXl4nuLuPPIYnOP43tziF AFKLpPkobRZqwv7j/VlsNCSZhMe04EiHV4qpS/QAx47oA4bYoomH8Kqm+6+VIKEAAar4 xLIwO2GaiLZytdMBenT6SDbAl+MBAXvzj1nATQSqZUodn4sd7AcOE38B3TsAIMZGiwiO rTaMSnOo69Aajx+CIsSQNJS4vEdNa6OrOcSTQKWcFhfq09zHN94NX79EpfrudBMcgm6k 0GuQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=eLgIJ3s/Tx97mhsfweK6+28YKiuYY5rTS0gdVJoPPN4=; b=tS1QR+JObrc9XQw1h3AxUhJNUFRKESUTuG/XQiiT1dzaG0X4x5PesJZN1WuLhc2DcN S7OQociyyuYwaIsoxbUgx0k2l4MaXQEYNzEFPb1MV8uKAZ3APl4V6LKHBiK3N/LjjQ4V JdsSXp4Ge62cpdrDugX0JDTLVcmW/fipxnXeMDiFHvsyILXxcF228/kMX6ZXHMuppyxw UkGMO7QDLfdndTGV2XpbuNdRSTtudQ2Faz3GmtUvBa0hPREg0hwGK9ccLOXEp5OTeiEN ubt1qFI1Q3obBCvDs1pzQNDM8xDNnK9tFhrvD8ANX5HNNr/OfsIlpysApeTL9US7Y2T8 2IdA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=MUrPE4Q6; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id sc27-20020a1709078a1b00b008d399f4f54asi10065566ejc.750.2023.03.02.03.38.23; Thu, 02 Mar 2023 03:38:46 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=MUrPE4Q6; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230083AbjCBLgj (ORCPT + 99 others); Thu, 2 Mar 2023 06:36:39 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55624 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229914AbjCBLgE (ORCPT ); Thu, 2 Mar 2023 06:36:04 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5E77917146 for ; Thu, 2 Mar 2023 03:35:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1677756918; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=eLgIJ3s/Tx97mhsfweK6+28YKiuYY5rTS0gdVJoPPN4=; b=MUrPE4Q6uW1IREFaxtQU0JtXX4iD2sYmhQ51EZoClzBqE4ec9bN+n9XVXmSwo91ADoSRAw sqaGkLwZ/PgSKQZ5UfVt/rQbBds8N9rPWxzJDDqCix5465vLI8EJ8Hk7rZee7Vj0irDlY8 mjlhU8FkHdD8ioUXtrfD9UsDOcM66wk= Received: from mail-qk1-f197.google.com (mail-qk1-f197.google.com [209.85.222.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-467-wnJEaStbOJ2Sd_G8HyOxCA-1; Thu, 02 Mar 2023 06:35:09 -0500 X-MC-Unique: wnJEaStbOJ2Sd_G8HyOxCA-1 Received: by mail-qk1-f197.google.com with SMTP id eb13-20020a05620a480d00b00742be8a1a17so6496076qkb.18 for ; Thu, 02 Mar 2023 03:35:09 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1677756909; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=eLgIJ3s/Tx97mhsfweK6+28YKiuYY5rTS0gdVJoPPN4=; b=acMXwvKgnjf6Fd/dk6eL9NQIPZPftOckq5Ay1e3BEbko5qMQwp1s/xJK50ZnHuDF2j U7d1YpH7+IgR8AKx/87voM5nT2vRnsHy8hav/LDsUawGRfxwkzY8PDH7ApmP3MQDyb6b 5dY4iPqH65bsxkZxZdZRmR9qIjUYlvzUCaazS+4GdcSIn8nmosCOzRy2h0tZ8Bzx+vgl m4TB5yRnHha5Db6AcI5ALqi11hzQuH6BmDMZw3ah9/fdXd24yJjoKGGtDeackiO0Hwvx WxeXgf0E35rzuIERDd/zHGWrLtBBgeRhqnDSMkDvttvh0K0zhO5z1JxUe4NQ1cXg3LKS a5Vw== X-Gm-Message-State: AO0yUKU6EBrLjzcpwWArpzHMWY8fw1TYsdf6YF13bIYYPfqyO/cEmb7Y 9uLrahqAkzIjtFi9ZCFskzRhZ9+GNufpuRNquDqQYFPCBuQxgtLAirmi2kQcnCRu/C+NBQYpDDz DezI4i9HP75Mwt1/Ls6kQqA0w X-Received: by 2002:a05:622a:1746:b0:3bf:dbd3:a014 with SMTP id l6-20020a05622a174600b003bfdbd3a014mr17136701qtk.45.1677756909325; Thu, 02 Mar 2023 03:35:09 -0800 (PST) X-Received: by 2002:a05:622a:1746:b0:3bf:dbd3:a014 with SMTP id l6-20020a05622a174600b003bfdbd3a014mr17136689qtk.45.1677756909086; Thu, 02 Mar 2023 03:35:09 -0800 (PST) Received: from step1.redhat.com (c-115-213.cust-q.wadsl.it. [212.43.115.213]) by smtp.gmail.com with ESMTPSA id p12-20020a37420c000000b007426ec97253sm8383058qka.111.2023.03.02.03.35.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Mar 2023 03:35:08 -0800 (PST) From: Stefano Garzarella To: virtualization@lists.linux-foundation.org Cc: Andrey Zhadchenko , eperezma@redhat.com, netdev@vger.kernel.org, stefanha@redhat.com, linux-kernel@vger.kernel.org, Jason Wang , "Michael S. Tsirkin" , kvm@vger.kernel.org, Stefano Garzarella Subject: [PATCH v2 5/8] vdpa_sim: make devices agnostic for work management Date: Thu, 2 Mar 2023 12:34:18 +0100 Message-Id: <20230302113421.174582-6-sgarzare@redhat.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230302113421.174582-1-sgarzare@redhat.com> References: <20230302113421.174582-1-sgarzare@redhat.com> MIME-Version: 1.0 Content-type: text/plain X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1759255856616062972?= X-GMAIL-MSGID: =?utf-8?q?1759255856616062972?= Let's move work management inside the vdpa_sim core. This way we can easily change how we manage the works, without having to change the devices each time. Signed-off-by: Stefano Garzarella Acked-by: Eugenio Pérez Martin Acked-by: Jason Wang --- drivers/vdpa/vdpa_sim/vdpa_sim.h | 3 ++- drivers/vdpa/vdpa_sim/vdpa_sim.c | 17 +++++++++++++++-- drivers/vdpa/vdpa_sim/vdpa_sim_blk.c | 6 ++---- drivers/vdpa/vdpa_sim/vdpa_sim_net.c | 6 ++---- 4 files changed, 21 insertions(+), 11 deletions(-) diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.h b/drivers/vdpa/vdpa_sim/vdpa_sim.h index 144858636c10..acee20faaf6a 100644 --- a/drivers/vdpa/vdpa_sim/vdpa_sim.h +++ b/drivers/vdpa/vdpa_sim/vdpa_sim.h @@ -45,7 +45,7 @@ struct vdpasim_dev_attr { u32 ngroups; u32 nas; - work_func_t work_fn; + void (*work_fn)(struct vdpasim *vdpasim); void (*get_config)(struct vdpasim *vdpasim, void *config); void (*set_config)(struct vdpasim *vdpasim, const void *config); int (*get_stats)(struct vdpasim *vdpasim, u16 idx, @@ -78,6 +78,7 @@ struct vdpasim { struct vdpasim *vdpasim_create(struct vdpasim_dev_attr *attr, const struct vdpa_dev_set_config *config); +void vdpasim_schedule_work(struct vdpasim *vdpasim); /* TODO: cross-endian support */ static inline bool vdpasim_is_little_endian(struct vdpasim *vdpasim) diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.c b/drivers/vdpa/vdpa_sim/vdpa_sim.c index 481eb156658b..a6ee830efc38 100644 --- a/drivers/vdpa/vdpa_sim/vdpa_sim.c +++ b/drivers/vdpa/vdpa_sim/vdpa_sim.c @@ -116,6 +116,13 @@ static void vdpasim_do_reset(struct vdpasim *vdpasim) static const struct vdpa_config_ops vdpasim_config_ops; static const struct vdpa_config_ops vdpasim_batch_config_ops; +static void vdpasim_work_fn(struct work_struct *work) +{ + struct vdpasim *vdpasim = container_of(work, struct vdpasim, work); + + vdpasim->dev_attr.work_fn(vdpasim); +} + struct vdpasim *vdpasim_create(struct vdpasim_dev_attr *dev_attr, const struct vdpa_dev_set_config *config) { @@ -152,7 +159,7 @@ struct vdpasim *vdpasim_create(struct vdpasim_dev_attr *dev_attr, vdpasim = vdpa_to_sim(vdpa); vdpasim->dev_attr = *dev_attr; - INIT_WORK(&vdpasim->work, dev_attr->work_fn); + INIT_WORK(&vdpasim->work, vdpasim_work_fn); spin_lock_init(&vdpasim->lock); spin_lock_init(&vdpasim->iommu_lock); @@ -203,6 +210,12 @@ struct vdpasim *vdpasim_create(struct vdpasim_dev_attr *dev_attr, } EXPORT_SYMBOL_GPL(vdpasim_create); +void vdpasim_schedule_work(struct vdpasim *vdpasim) +{ + schedule_work(&vdpasim->work); +} +EXPORT_SYMBOL_GPL(vdpasim_schedule_work); + static int vdpasim_set_vq_address(struct vdpa_device *vdpa, u16 idx, u64 desc_area, u64 driver_area, u64 device_area) @@ -237,7 +250,7 @@ static void vdpasim_kick_vq(struct vdpa_device *vdpa, u16 idx) } if (vq->ready) - schedule_work(&vdpasim->work); + vdpasim_schedule_work(vdpasim); } static void vdpasim_set_vq_cb(struct vdpa_device *vdpa, u16 idx, diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim_blk.c b/drivers/vdpa/vdpa_sim/vdpa_sim_blk.c index 5117959bed8a..eb4897c8541e 100644 --- a/drivers/vdpa/vdpa_sim/vdpa_sim_blk.c +++ b/drivers/vdpa/vdpa_sim/vdpa_sim_blk.c @@ -11,7 +11,6 @@ #include #include #include -#include #include #include #include @@ -286,9 +285,8 @@ static bool vdpasim_blk_handle_req(struct vdpasim *vdpasim, return handled; } -static void vdpasim_blk_work(struct work_struct *work) +static void vdpasim_blk_work(struct vdpasim *vdpasim) { - struct vdpasim *vdpasim = container_of(work, struct vdpasim, work); bool reschedule = false; int i; @@ -326,7 +324,7 @@ static void vdpasim_blk_work(struct work_struct *work) spin_unlock(&vdpasim->lock); if (reschedule) - schedule_work(&vdpasim->work); + vdpasim_schedule_work(vdpasim); } static void vdpasim_blk_get_config(struct vdpasim *vdpasim, void *config) diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim_net.c b/drivers/vdpa/vdpa_sim/vdpa_sim_net.c index 862f405362de..e61a9ecbfafe 100644 --- a/drivers/vdpa/vdpa_sim/vdpa_sim_net.c +++ b/drivers/vdpa/vdpa_sim/vdpa_sim_net.c @@ -11,7 +11,6 @@ #include #include #include -#include #include #include #include @@ -192,9 +191,8 @@ static void vdpasim_handle_cvq(struct vdpasim *vdpasim) u64_stats_update_end(&net->cq_stats.syncp); } -static void vdpasim_net_work(struct work_struct *work) +static void vdpasim_net_work(struct vdpasim *vdpasim) { - struct vdpasim *vdpasim = container_of(work, struct vdpasim, work); struct vdpasim_virtqueue *txq = &vdpasim->vqs[1]; struct vdpasim_virtqueue *rxq = &vdpasim->vqs[0]; struct vdpasim_net *net = sim_to_net(vdpasim); @@ -260,7 +258,7 @@ static void vdpasim_net_work(struct work_struct *work) vdpasim_net_complete(rxq, write); if (tx_pkts > 4) { - schedule_work(&vdpasim->work); + vdpasim_schedule_work(vdpasim); goto out; } } From patchwork Thu Mar 2 11:34:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefano Garzarella X-Patchwork-Id: 63365 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp4185068wrd; Thu, 2 Mar 2023 03:38:47 -0800 (PST) X-Google-Smtp-Source: AK7set9mikggTdCRSxrbGKK6EprG3+xpIBcebieKOEfuiAJByYZNsGwPryPfzKDB5EGBVsXmeTKo X-Received: by 2002:aa7:c713:0:b0:4af:70cf:d142 with SMTP id i19-20020aa7c713000000b004af70cfd142mr1526577edq.17.1677757127760; Thu, 02 Mar 2023 03:38:47 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1677757127; cv=none; d=google.com; s=arc-20160816; b=FGd7ryeUN1PrGoy69toAyUlZvhdsLC4TkAndb/8SdXUBIlS0BAsO98lS79i2Q1devQ HSa1heDM3q62H3qLCsm93erkISTFduclqURblwQ1HgAbEm2ZipoWR4D+22xR6wIG5JuZ vRFaXv7gTZRo66Jpmd5Oiek6eGrLQ5JBd8cW0JuRsN/9z7+tg5ea/XLhMA3ufGfvI2Lu ZUK8eZLOFNOiBzRU1zuyfeMoH4uaYxQ9J0axZTyHjCTnIsyCHOHVIEE+VW7G6JH0VAcF J+4U0c4a3SaLJ+OTzcTJjYbR9bVAYHVSulNa1pQpHyF9xnFgycPXTZUYabtO0aUzsvWN WNGw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=garxf7u2PYyPkA/AdwQgG8IfzxLSGUACcilXDazWE4o=; b=XZlAY/EjFNHqdK7u93Y1u2zUZMaWiGTIHR/EyVWfWtY/WLL6/RURWpH6HFLNFFE2KJ j7U5z2kRtnROwQfhX62Hefb5Xf1lr5ezakWA6ljvF92G7dFdHkasoeKbDsGv68POYfAO sjoiBBDThBmlIlSi0WgxEX8B3qqC+NCsgrsBNlaJUyRxjgGz+7hTBIvpRzm7Z7YpPzqO 0XPoLGappUkTqyvBc7TJuiH6zwmLcN5vOAvq0FBqXpCSTWTdaJA+wK7BYt5HYKHgvFi3 LsIzx9PI+SclTwEfgfMY+CspN3O2EdLB5a3i/ZTNupuvOKg9L6iWjTjpkH69Ik7B92WL N+5Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=eYfmIggr; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id k11-20020a05640212cb00b004ad71106d26si17144217edx.60.2023.03.02.03.38.24; Thu, 02 Mar 2023 03:38:47 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=eYfmIggr; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229793AbjCBLgo (ORCPT + 99 others); Thu, 2 Mar 2023 06:36:44 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55558 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229913AbjCBLgE (ORCPT ); Thu, 2 Mar 2023 06:36:04 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A3C07136F2 for ; Thu, 2 Mar 2023 03:35:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1677756914; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=garxf7u2PYyPkA/AdwQgG8IfzxLSGUACcilXDazWE4o=; b=eYfmIggrFzs+QlJskU3P1wY5oLW8cnrT8NCG7jz8PNEMfCDXvtrKKLTBkqD5HXUNTdbB/N 1594+iYYehYrAJGWUffQ+yIXPAnIwgD3KkiRDPx9Ov6wIytgpWrkr8+TwIaPmwDi3O4RKq rUK1tysPzyQmgQR9189E3Y4G1g29Tvc= Received: from mail-qv1-f70.google.com (mail-qv1-f70.google.com [209.85.219.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-550-4ce3jHClP6C2VQ-cDkbndw-1; Thu, 02 Mar 2023 06:35:13 -0500 X-MC-Unique: 4ce3jHClP6C2VQ-cDkbndw-1 Received: by mail-qv1-f70.google.com with SMTP id pz4-20020ad45504000000b0056f060452adso8571212qvb.6 for ; Thu, 02 Mar 2023 03:35:13 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1677756913; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=garxf7u2PYyPkA/AdwQgG8IfzxLSGUACcilXDazWE4o=; b=lRq97CYN9pb7TXkjxK8SkfNF4u3cmd/YnNnm+0tYzmbQkNfplbXooLny4BRbYu99BF E49t1sbHSpgzv8xt4uRsdEcDClcsR0D9QW4/bnZnMUWmZUJTFq2bp9GLllG5OXmZv25f cxYdCUizCoRiVAbTvavB7Qrm3Y0g9xcGWbiifDQadNnvEUje9PM1NMRWSe31dqiNv7IC dl3zjiGcWc6SIEDtJ5N7sFAN6r+oTQKVVq2BvYRGbobKazC/Ti0c+R6GiPsfnkBPTqOF HgQAG+Co8I+z3MeOyRq19b3uAhG2szOgEDBXAc6y9iZA21VLepyO6weAX/G9NJs+8J/O ucvA== X-Gm-Message-State: AO0yUKVgm8e74Ndn4+RRg3A2zoV/EInImFVdGVGrxZ5uqNEYFGXMFBtq ssbj4Kf66z9qZxR7PrfncI6SlZzQbcsi0d/c1IQ3oXvu2zHsqAkx8e/949Ozlhl1xlxDaSBibmB pZBaOEBymYAVnwJiD2HxV86X54AVZww== X-Received: by 2002:a05:6214:ca2:b0:56e:a620:7b39 with SMTP id s2-20020a0562140ca200b0056ea6207b39mr2884830qvs.4.1677756913180; Thu, 02 Mar 2023 03:35:13 -0800 (PST) X-Received: by 2002:a05:6214:ca2:b0:56e:a620:7b39 with SMTP id s2-20020a0562140ca200b0056ea6207b39mr2884814qvs.4.1677756912950; Thu, 02 Mar 2023 03:35:12 -0800 (PST) Received: from step1.redhat.com (c-115-213.cust-q.wadsl.it. [212.43.115.213]) by smtp.gmail.com with ESMTPSA id p12-20020a37420c000000b007426ec97253sm8383058qka.111.2023.03.02.03.35.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Mar 2023 03:35:12 -0800 (PST) From: Stefano Garzarella To: virtualization@lists.linux-foundation.org Cc: Andrey Zhadchenko , eperezma@redhat.com, netdev@vger.kernel.org, stefanha@redhat.com, linux-kernel@vger.kernel.org, Jason Wang , "Michael S. Tsirkin" , kvm@vger.kernel.org, Stefano Garzarella Subject: [PATCH v2 6/8] vdpa_sim: use kthread worker Date: Thu, 2 Mar 2023 12:34:19 +0100 Message-Id: <20230302113421.174582-7-sgarzare@redhat.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230302113421.174582-1-sgarzare@redhat.com> References: <20230302113421.174582-1-sgarzare@redhat.com> MIME-Version: 1.0 Content-type: text/plain X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1759255857931791328?= X-GMAIL-MSGID: =?utf-8?q?1759255857931791328?= Let's use our own kthread to run device jobs. This allows us more flexibility, especially we can attach the kthread to the user address space when vDPA uses user's VA. Signed-off-by: Stefano Garzarella Acked-by: Jason Wang --- drivers/vdpa/vdpa_sim/vdpa_sim.h | 3 ++- drivers/vdpa/vdpa_sim/vdpa_sim.c | 17 ++++++++++++----- 2 files changed, 14 insertions(+), 6 deletions(-) diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.h b/drivers/vdpa/vdpa_sim/vdpa_sim.h index acee20faaf6a..ce83f9130a5d 100644 --- a/drivers/vdpa/vdpa_sim/vdpa_sim.h +++ b/drivers/vdpa/vdpa_sim/vdpa_sim.h @@ -57,7 +57,8 @@ struct vdpasim_dev_attr { struct vdpasim { struct vdpa_device vdpa; struct vdpasim_virtqueue *vqs; - struct work_struct work; + struct kthread_worker *worker; + struct kthread_work work; struct vdpasim_dev_attr dev_attr; /* spinlock to synchronize virtqueue state */ spinlock_t lock; diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.c b/drivers/vdpa/vdpa_sim/vdpa_sim.c index a6ee830efc38..6feb29726c2a 100644 --- a/drivers/vdpa/vdpa_sim/vdpa_sim.c +++ b/drivers/vdpa/vdpa_sim/vdpa_sim.c @@ -11,8 +11,8 @@ #include #include #include +#include #include -#include #include #include #include @@ -116,7 +116,7 @@ static void vdpasim_do_reset(struct vdpasim *vdpasim) static const struct vdpa_config_ops vdpasim_config_ops; static const struct vdpa_config_ops vdpasim_batch_config_ops; -static void vdpasim_work_fn(struct work_struct *work) +static void vdpasim_work_fn(struct kthread_work *work) { struct vdpasim *vdpasim = container_of(work, struct vdpasim, work); @@ -159,7 +159,13 @@ struct vdpasim *vdpasim_create(struct vdpasim_dev_attr *dev_attr, vdpasim = vdpa_to_sim(vdpa); vdpasim->dev_attr = *dev_attr; - INIT_WORK(&vdpasim->work, vdpasim_work_fn); + + kthread_init_work(&vdpasim->work, vdpasim_work_fn); + vdpasim->worker = kthread_create_worker(0, "vDPA sim worker: %s", + dev_attr->name); + if (IS_ERR(vdpasim->worker)) + goto err_iommu; + spin_lock_init(&vdpasim->lock); spin_lock_init(&vdpasim->iommu_lock); @@ -212,7 +218,7 @@ EXPORT_SYMBOL_GPL(vdpasim_create); void vdpasim_schedule_work(struct vdpasim *vdpasim) { - schedule_work(&vdpasim->work); + kthread_queue_work(vdpasim->worker, &vdpasim->work); } EXPORT_SYMBOL_GPL(vdpasim_schedule_work); @@ -612,7 +618,8 @@ static void vdpasim_free(struct vdpa_device *vdpa) struct vdpasim *vdpasim = vdpa_to_sim(vdpa); int i; - cancel_work_sync(&vdpasim->work); + kthread_cancel_work_sync(&vdpasim->work); + kthread_destroy_worker(vdpasim->worker); for (i = 0; i < vdpasim->dev_attr.nvqs; i++) { vringh_kiov_cleanup(&vdpasim->vqs[i].out_iov); From patchwork Thu Mar 2 11:34:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefano Garzarella X-Patchwork-Id: 63366 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp4185227wrd; Thu, 2 Mar 2023 03:39:11 -0800 (PST) X-Google-Smtp-Source: AK7set9+tvq/I2Nc4sU9g3lGEEj3JYITR0aAU1mDHGjooy+W2Duk3z059RTXMfUX+BTG8aW9OuRW X-Received: by 2002:a17:906:ecec:b0:8ec:4334:fe with SMTP id qt12-20020a170906ecec00b008ec433400femr10230841ejb.26.1677757151092; Thu, 02 Mar 2023 03:39:11 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1677757151; cv=none; d=google.com; s=arc-20160816; b=hTyRNqyFt83WXLBWZ2lISDs1JKb6LqezQ/yFqB7JnqTeJUq1em0q4EbFfjofpg5Fbf RBoH2CKfrZEOfJnKY+EZ/WAuLVUImokkHjJqKXQFUIwznA0rBvnmVh/5/stO3LCCIGsL fJKFd9d082dGA4G04gqs1TcUnrpm2y+1mIHJcY3yqwOwfTI6W/CZB91FtICzJk4PEoiI QZh7RI5JuzmAmwDZQUifrR9qodQybtJxlO9cG0hNQ18xp5ctDUGHfr5s4w79OIqssyaM kN0nf0MqcdGJCXfIQWX/K8EEqf4XjqvDxw4CPf7wGn+CqcrJh6YB5V+gRL5knPxIJxGh DoGQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Gj3dXUcrOW7GQbHC3bzCmguoR9a2asK1vNem232OL7c=; b=xzVxlUL1PKYFhUAUh30knefptYfkinVlToAqnvQyS1YXVgo8i6bo/F0YGZFlSIGeOB V+FT5ePS9JXlGQwOz5oXc1u7j/zCtQ7pMwHmDG2Bhnh7Kg7uUktMApcQuUgHFsmkWL99 6Ns3A1z6ymbIQyOG3n/PogCsae3gyKY8zKMhHplDCK53qugWS1fbtBM9orbAUoHpcYg4 VpKK5BvFY4WKWLv0ahOYZNNTiZky17i8EUEqGBVyIJJdAHREaTqNARt3SskD59youON/ fI3/r+ddd6fLchNQ/dHzQNYSFq3/6GQtzI2twvjYHIiFHLZGv/NcSZgTeyJo5qVPt0k+ 0hNQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=LpHHUkYU; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id jw8-20020a17090776a800b008d77c083ac6si7010843ejc.691.2023.03.02.03.38.48; Thu, 02 Mar 2023 03:39:11 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=LpHHUkYU; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229913AbjCBLgt (ORCPT + 99 others); Thu, 2 Mar 2023 06:36:49 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55628 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229811AbjCBLgI (ORCPT ); Thu, 2 Mar 2023 06:36:08 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5F4A71ABC8 for ; Thu, 2 Mar 2023 03:35:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1677756918; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Gj3dXUcrOW7GQbHC3bzCmguoR9a2asK1vNem232OL7c=; b=LpHHUkYUuZjSnP377+4wqhPPWp9ul9x3fPMBvM1LzaZTktdtq5gLPJkTMnEiit1aenrVLY egBJXdD4OFIkfIpjyfkZw21dMaqMi4TBMZsbpSIBdAUkm2qqCDJ34Adi6wenQdnnimFJAN NNYpRihr1kb9ZoNJ/Ztj6cHqNOp5Ruc= Received: from mail-qk1-f200.google.com (mail-qk1-f200.google.com [209.85.222.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-642-Rg8IE5EVMd2D_b-msRISuA-1; Thu, 02 Mar 2023 06:35:17 -0500 X-MC-Unique: Rg8IE5EVMd2D_b-msRISuA-1 Received: by mail-qk1-f200.google.com with SMTP id pc36-20020a05620a842400b00742c715894bso6029316qkn.21 for ; Thu, 02 Mar 2023 03:35:17 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1677756917; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Gj3dXUcrOW7GQbHC3bzCmguoR9a2asK1vNem232OL7c=; b=5iPvOTNev2sICXIheZ0sFmn71Q/8t/N8+MMXpRn4LygnOOcNTypaz5yEmjIBe/CB6Q yAfwrSHkRIpAC4gRP+LNQslOolggEsH94MdH8dSXSmc74Igg8PHY9cEW5ypImPEbmdse nIHNBm8xO1dTmj/EKOvAcIl/xnIT/uSiBKXQb9Tovd9PXgZpsMThkBs+K2rsqmyDXKU1 X7UBUUIEPCQqeeHQ81LOdAo5eCi0fl8rsYZdRoduwqbzg1prm96h/KjOpaYO/OG1U+nH Zm8tIr5e5Qu2sgv7mg9q+3cOkMvT6XU+T6qt/sSXXbgOK0mwhai1fGDThtvxhivBYkny tMCw== X-Gm-Message-State: AO0yUKVfDDcoafKXyLM1P2fkmnLPiypVcuXQvCpCXkmx8P1z27rZF5+n VCA/5d35W3grkeWcJ/uXrkhfEZl7jq9bnlfxBrPLflDvWFHgJV2Jx/grP2h9p6Hh0H62+MPyDDm viF1Wt/KRb2m2SFk25y00A0lt X-Received: by 2002:ac8:594f:0:b0:3a7:e9a2:4f4a with SMTP id 15-20020ac8594f000000b003a7e9a24f4amr16171736qtz.8.1677756916955; Thu, 02 Mar 2023 03:35:16 -0800 (PST) X-Received: by 2002:ac8:594f:0:b0:3a7:e9a2:4f4a with SMTP id 15-20020ac8594f000000b003a7e9a24f4amr16171717qtz.8.1677756916710; Thu, 02 Mar 2023 03:35:16 -0800 (PST) Received: from step1.redhat.com (c-115-213.cust-q.wadsl.it. [212.43.115.213]) by smtp.gmail.com with ESMTPSA id p12-20020a37420c000000b007426ec97253sm8383058qka.111.2023.03.02.03.35.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Mar 2023 03:35:16 -0800 (PST) From: Stefano Garzarella To: virtualization@lists.linux-foundation.org Cc: Andrey Zhadchenko , eperezma@redhat.com, netdev@vger.kernel.org, stefanha@redhat.com, linux-kernel@vger.kernel.org, Jason Wang , "Michael S. Tsirkin" , kvm@vger.kernel.org, Stefano Garzarella Subject: [PATCH v2 7/8] vdpa_sim: replace the spinlock with a mutex to protect the state Date: Thu, 2 Mar 2023 12:34:20 +0100 Message-Id: <20230302113421.174582-8-sgarzare@redhat.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230302113421.174582-1-sgarzare@redhat.com> References: <20230302113421.174582-1-sgarzare@redhat.com> MIME-Version: 1.0 Content-type: text/plain X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1759255882452103236?= X-GMAIL-MSGID: =?utf-8?q?1759255882452103236?= The spinlock we use to protect the state of the simulator is sometimes held for a long time (for example, when devices handle requests). This also prevents us from calling functions that might sleep (such as kthread_flush_work() in the next patch), and thus having to release and retake the lock. For these reasons, let's replace the spinlock with a mutex that gives us more flexibility. Suggested-by: Jason Wang Signed-off-by: Stefano Garzarella Acked-by: Jason Wang --- drivers/vdpa/vdpa_sim/vdpa_sim.h | 4 ++-- drivers/vdpa/vdpa_sim/vdpa_sim.c | 34 ++++++++++++++-------------- drivers/vdpa/vdpa_sim/vdpa_sim_blk.c | 4 ++-- drivers/vdpa/vdpa_sim/vdpa_sim_net.c | 4 ++-- 4 files changed, 23 insertions(+), 23 deletions(-) diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.h b/drivers/vdpa/vdpa_sim/vdpa_sim.h index ce83f9130a5d..4774292fba8c 100644 --- a/drivers/vdpa/vdpa_sim/vdpa_sim.h +++ b/drivers/vdpa/vdpa_sim/vdpa_sim.h @@ -60,8 +60,8 @@ struct vdpasim { struct kthread_worker *worker; struct kthread_work work; struct vdpasim_dev_attr dev_attr; - /* spinlock to synchronize virtqueue state */ - spinlock_t lock; + /* mutex to synchronize virtqueue state */ + struct mutex mutex; /* virtio config according to device type */ void *config; struct vhost_iotlb *iommu; diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.c b/drivers/vdpa/vdpa_sim/vdpa_sim.c index 6feb29726c2a..a28103a67ae7 100644 --- a/drivers/vdpa/vdpa_sim/vdpa_sim.c +++ b/drivers/vdpa/vdpa_sim/vdpa_sim.c @@ -166,7 +166,7 @@ struct vdpasim *vdpasim_create(struct vdpasim_dev_attr *dev_attr, if (IS_ERR(vdpasim->worker)) goto err_iommu; - spin_lock_init(&vdpasim->lock); + mutex_init(&vdpasim->mutex); spin_lock_init(&vdpasim->iommu_lock); dev = &vdpasim->vdpa.dev; @@ -275,13 +275,13 @@ static void vdpasim_set_vq_ready(struct vdpa_device *vdpa, u16 idx, bool ready) struct vdpasim_virtqueue *vq = &vdpasim->vqs[idx]; bool old_ready; - spin_lock(&vdpasim->lock); + mutex_lock(&vdpasim->mutex); old_ready = vq->ready; vq->ready = ready; if (vq->ready && !old_ready) { vdpasim_queue_ready(vdpasim, idx); } - spin_unlock(&vdpasim->lock); + mutex_unlock(&vdpasim->mutex); } static bool vdpasim_get_vq_ready(struct vdpa_device *vdpa, u16 idx) @@ -299,9 +299,9 @@ static int vdpasim_set_vq_state(struct vdpa_device *vdpa, u16 idx, struct vdpasim_virtqueue *vq = &vdpasim->vqs[idx]; struct vringh *vrh = &vq->vring; - spin_lock(&vdpasim->lock); + mutex_lock(&vdpasim->mutex); vrh->last_avail_idx = state->split.avail_index; - spin_unlock(&vdpasim->lock); + mutex_unlock(&vdpasim->mutex); return 0; } @@ -398,9 +398,9 @@ static u8 vdpasim_get_status(struct vdpa_device *vdpa) struct vdpasim *vdpasim = vdpa_to_sim(vdpa); u8 status; - spin_lock(&vdpasim->lock); + mutex_lock(&vdpasim->mutex); status = vdpasim->status; - spin_unlock(&vdpasim->lock); + mutex_unlock(&vdpasim->mutex); return status; } @@ -409,19 +409,19 @@ static void vdpasim_set_status(struct vdpa_device *vdpa, u8 status) { struct vdpasim *vdpasim = vdpa_to_sim(vdpa); - spin_lock(&vdpasim->lock); + mutex_lock(&vdpasim->mutex); vdpasim->status = status; - spin_unlock(&vdpasim->lock); + mutex_unlock(&vdpasim->mutex); } static int vdpasim_reset(struct vdpa_device *vdpa) { struct vdpasim *vdpasim = vdpa_to_sim(vdpa); - spin_lock(&vdpasim->lock); + mutex_lock(&vdpasim->mutex); vdpasim->status = 0; vdpasim_do_reset(vdpasim); - spin_unlock(&vdpasim->lock); + mutex_unlock(&vdpasim->mutex); return 0; } @@ -430,9 +430,9 @@ static int vdpasim_suspend(struct vdpa_device *vdpa) { struct vdpasim *vdpasim = vdpa_to_sim(vdpa); - spin_lock(&vdpasim->lock); + mutex_lock(&vdpasim->mutex); vdpasim->running = false; - spin_unlock(&vdpasim->lock); + mutex_unlock(&vdpasim->mutex); return 0; } @@ -442,7 +442,7 @@ static int vdpasim_resume(struct vdpa_device *vdpa) struct vdpasim *vdpasim = vdpa_to_sim(vdpa); int i; - spin_lock(&vdpasim->lock); + mutex_lock(&vdpasim->mutex); vdpasim->running = true; if (vdpasim->pending_kick) { @@ -453,7 +453,7 @@ static int vdpasim_resume(struct vdpa_device *vdpa) vdpasim->pending_kick = false; } - spin_unlock(&vdpasim->lock); + mutex_unlock(&vdpasim->mutex); return 0; } @@ -525,14 +525,14 @@ static int vdpasim_set_group_asid(struct vdpa_device *vdpa, unsigned int group, iommu = &vdpasim->iommu[asid]; - spin_lock(&vdpasim->lock); + mutex_lock(&vdpasim->mutex); for (i = 0; i < vdpasim->dev_attr.nvqs; i++) if (vdpasim_get_vq_group(vdpa, i) == group) vringh_set_iotlb(&vdpasim->vqs[i].vring, iommu, &vdpasim->iommu_lock); - spin_unlock(&vdpasim->lock); + mutex_unlock(&vdpasim->mutex); return 0; } diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim_blk.c b/drivers/vdpa/vdpa_sim/vdpa_sim_blk.c index eb4897c8541e..568119e1553f 100644 --- a/drivers/vdpa/vdpa_sim/vdpa_sim_blk.c +++ b/drivers/vdpa/vdpa_sim/vdpa_sim_blk.c @@ -290,7 +290,7 @@ static void vdpasim_blk_work(struct vdpasim *vdpasim) bool reschedule = false; int i; - spin_lock(&vdpasim->lock); + mutex_lock(&vdpasim->mutex); if (!(vdpasim->status & VIRTIO_CONFIG_S_DRIVER_OK)) goto out; @@ -321,7 +321,7 @@ static void vdpasim_blk_work(struct vdpasim *vdpasim) } } out: - spin_unlock(&vdpasim->lock); + mutex_unlock(&vdpasim->mutex); if (reschedule) vdpasim_schedule_work(vdpasim); diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim_net.c b/drivers/vdpa/vdpa_sim/vdpa_sim_net.c index e61a9ecbfafe..7ab434592bfe 100644 --- a/drivers/vdpa/vdpa_sim/vdpa_sim_net.c +++ b/drivers/vdpa/vdpa_sim/vdpa_sim_net.c @@ -201,7 +201,7 @@ static void vdpasim_net_work(struct vdpasim *vdpasim) u64 rx_drops = 0, rx_overruns = 0, rx_errors = 0, tx_errors = 0; int err; - spin_lock(&vdpasim->lock); + mutex_lock(&vdpasim->mutex); if (!vdpasim->running) goto out; @@ -264,7 +264,7 @@ static void vdpasim_net_work(struct vdpasim *vdpasim) } out: - spin_unlock(&vdpasim->lock); + mutex_unlock(&vdpasim->mutex); u64_stats_update_begin(&net->tx_stats.syncp); net->tx_stats.pkts += tx_pkts; From patchwork Thu Mar 2 11:34:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefano Garzarella X-Patchwork-Id: 63368 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp4185454wrd; Thu, 2 Mar 2023 03:39:41 -0800 (PST) X-Google-Smtp-Source: AK7set/JfsjsDSkrDvIN13vyJcIovK9uqGqHtZLMre4GV4v5/C4bY7ivHbxE8MHz6XIZfVGghevX X-Received: by 2002:a05:6402:189:b0:4ab:1d33:69ba with SMTP id r9-20020a056402018900b004ab1d3369bamr11625405edv.16.1677757180933; Thu, 02 Mar 2023 03:39:40 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1677757180; cv=none; d=google.com; s=arc-20160816; b=MZ1h7j69+I2CPxFKvHln/E8X3E8PWqAVOqXSgUUEOMZ8xXxX/+dEI7nmaWZFEDLJ2j vkzRL2+E+UKDB7CMNIVOr7c6vDQEnplkmysq23ma6eIF+XNVQqTuzPBXwe/65yjg2pzP 7F1UpTYjRdfTTTHFSHIFOP9ttyNuWXQWm8QL7mPvwBPw/5F5jpiRgI9xF7iz3jfi57P+ Zz/T8FFYpjkvNRrtgYz0hdHX8gjAItAAozPyWVfWwQt/dNhms3auGO2kZKux4M5c+DRD jezOIOcr/p1scnvMUPDoCrg0mz1mXBfHq5ftoD3hY6BDuDXFtNLgEL9UQ4w0RdLYMDli 7Mzw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=8ei7n2l78e4NFmvUi+bOVTMUuSnDAXEzTgyerL27l10=; b=d0fX9MIUI+hJkhKmkBLk1TJIvDl0qWcs7n5DpjPXIovEn61JeGWBrE4igJQTISqUnh JOeLuO/dPlreKBGSAsFQDTBdL961S9i6eg6tmpl4+VZqoo+O66DQ4VsAAmywUVn/VCPE 3pv3LHj+qnF6sTBqdJBWbGTXCdd7trHbFhIvTYd0j36B4CGhvmJPT7lKcwG3qtK36G2Y RxMbvajPqo6HgVeh2HKEk+dx4zq0qLSoRfvyg7ZdwbNFrdEwaDRMjfcymvp23TeU41Mz gN8bD5L0liTXyo2xtRdzHKw6rnRhycSaBuyYT5xDRsy8OVVLFMaXQDdXBvyMA/wtLshd fvyg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=SiKWFQEs; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id n5-20020a170906840500b008cb6722b1edsi6519945ejx.454.2023.03.02.03.39.18; Thu, 02 Mar 2023 03:39:40 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=SiKWFQEs; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229921AbjCBLhZ (ORCPT + 99 others); Thu, 2 Mar 2023 06:37:25 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55196 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230037AbjCBLhI (ORCPT ); Thu, 2 Mar 2023 06:37:08 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4EA33474D9 for ; Thu, 2 Mar 2023 03:35:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1677756946; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8ei7n2l78e4NFmvUi+bOVTMUuSnDAXEzTgyerL27l10=; b=SiKWFQEsjux9pg7VcflKRjwXsMx2t4MEgSQjLUbocfexN4n7yUTyFIwasVmDxJk7tWjTSE vfsCID+KZCOf3eTyu0KKix0o1V3uCBb+3jEAvXLriy85RpzoTFSyVpP39qdFtbcuA+uTE1 Opol4lLm/zofmnMhSnlIl8IFoZBfo68= Received: from mail-qt1-f197.google.com (mail-qt1-f197.google.com [209.85.160.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-74-ufYSVbWwPNOQgmGCMqmw5g-1; Thu, 02 Mar 2023 06:35:45 -0500 X-MC-Unique: ufYSVbWwPNOQgmGCMqmw5g-1 Received: by mail-qt1-f197.google.com with SMTP id w16-20020ac843d0000000b003bfe50a4105so5301830qtn.10 for ; Thu, 02 Mar 2023 03:35:45 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1677756944; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8ei7n2l78e4NFmvUi+bOVTMUuSnDAXEzTgyerL27l10=; b=iVc2oSCiFGdxzp5ikjbbVOoD7glF4yPxosIV82bvo9OQI3Fk1efF4CU0sgrgmDZhy6 EwgEMhnM6U4lacNRHIFH6Ai+36MnChTQEtbcNYi2KfcukEIgMm6jD+FoKv8P0TPoI3pj J2gtqC58u5qYkCJe4xg3lQcItvmrnqXN9L/JBFQoFqbgrCKLPIxfFE6Qzqvk6J1joD61 dj/X6YzTugsX7mGAj3Cn0wals4u83ciuQxFbCPF9BBtACaay7ZbGKi2ef3HdSX9NS7mf LtXXprh44OSzBelEou0MvGkitKfAEdLzdZFcwPqZ5ShxL7NMSqGg6/7N4AknHmrkalmp 72oA== X-Gm-Message-State: AO0yUKU2u0V4tA3RQlP0DFRr43NLP2E5kyukSDB83s8cveYKI0T0kb/a T1PKEs+1Dm89SiDWpwQPmwmpc5w6RlqXhgyaQMajNwMCEUwt/TBun64AKYPwixWNmOG17mfmLbV x3iTsh5c9lrpqdgVzK0CJp3vU X-Received: by 2002:a05:6214:23cc:b0:56f:796e:c3a5 with SMTP id hr12-20020a05621423cc00b0056f796ec3a5mr15695877qvb.4.1677756944514; Thu, 02 Mar 2023 03:35:44 -0800 (PST) X-Received: by 2002:a05:6214:23cc:b0:56f:796e:c3a5 with SMTP id hr12-20020a05621423cc00b0056f796ec3a5mr15695849qvb.4.1677756944236; Thu, 02 Mar 2023 03:35:44 -0800 (PST) Received: from step1.redhat.com (c-115-213.cust-q.wadsl.it. [212.43.115.213]) by smtp.gmail.com with ESMTPSA id m26-20020ae9e01a000000b00741d87eb5d1sm10630925qkk.105.2023.03.02.03.35.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Mar 2023 03:35:43 -0800 (PST) From: Stefano Garzarella To: virtualization@lists.linux-foundation.org Cc: Andrey Zhadchenko , eperezma@redhat.com, netdev@vger.kernel.org, stefanha@redhat.com, linux-kernel@vger.kernel.org, Jason Wang , "Michael S. Tsirkin" , kvm@vger.kernel.org, Stefano Garzarella Subject: [PATCH v2 8/8] vdpa_sim: add support for user VA Date: Thu, 2 Mar 2023 12:34:21 +0100 Message-Id: <20230302113421.174582-9-sgarzare@redhat.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230302113421.174582-1-sgarzare@redhat.com> References: <20230302113421.174582-1-sgarzare@redhat.com> MIME-Version: 1.0 Content-type: text/plain X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1759255867636234036?= X-GMAIL-MSGID: =?utf-8?q?1759255913253351719?= The new "use_va" module parameter (default: false) is used in vdpa_alloc_device() to inform the vDPA framework that the device supports VA. vringh is initialized to use VA only when "use_va" is true and the user's mm has been bound. So, only when the bus supports user VA (e.g. vhost-vdpa). vdpasim_mm_work_fn work is used to attach the kthread to the user address space when the .bind_mm callback is invoked, and to detach it when the .unbind_mm callback is invoked. Signed-off-by: Stefano Garzarella --- Notes: v2: - `use_va` set to true by default [Eugenio] - supported the new unbind_mm callback [Jason] - removed the unbind_mm call in vdpasim_do_reset() [Jason] - avoided to release the lock while call kthread_flush_work() since we are now using a mutex to protect the device state drivers/vdpa/vdpa_sim/vdpa_sim.h | 1 + drivers/vdpa/vdpa_sim/vdpa_sim.c | 98 +++++++++++++++++++++++++++++++- 2 files changed, 97 insertions(+), 2 deletions(-) diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.h b/drivers/vdpa/vdpa_sim/vdpa_sim.h index 4774292fba8c..3a42887d05d9 100644 --- a/drivers/vdpa/vdpa_sim/vdpa_sim.h +++ b/drivers/vdpa/vdpa_sim/vdpa_sim.h @@ -59,6 +59,7 @@ struct vdpasim { struct vdpasim_virtqueue *vqs; struct kthread_worker *worker; struct kthread_work work; + struct mm_struct *mm_bound; struct vdpasim_dev_attr dev_attr; /* mutex to synchronize virtqueue state */ struct mutex mutex; diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.c b/drivers/vdpa/vdpa_sim/vdpa_sim.c index a28103a67ae7..eda26bc33df5 100644 --- a/drivers/vdpa/vdpa_sim/vdpa_sim.c +++ b/drivers/vdpa/vdpa_sim/vdpa_sim.c @@ -35,10 +35,77 @@ module_param(max_iotlb_entries, int, 0444); MODULE_PARM_DESC(max_iotlb_entries, "Maximum number of iotlb entries for each address space. 0 means unlimited. (default: 2048)"); +static bool use_va = true; +module_param(use_va, bool, 0444); +MODULE_PARM_DESC(use_va, "Enable/disable the device's ability to use VA"); + #define VDPASIM_QUEUE_ALIGN PAGE_SIZE #define VDPASIM_QUEUE_MAX 256 #define VDPASIM_VENDOR_ID 0 +struct vdpasim_mm_work { + struct kthread_work work; + struct mm_struct *mm; + bool bind; + int ret; +}; + +static void vdpasim_mm_work_fn(struct kthread_work *work) +{ + struct vdpasim_mm_work *mm_work = + container_of(work, struct vdpasim_mm_work, work); + + mm_work->ret = 0; + + if (mm_work->bind) { + kthread_use_mm(mm_work->mm); + //TODO: should we attach the cgroup of the mm owner? + } else { + kthread_unuse_mm(mm_work->mm); + } +} + +static void vdpasim_worker_queue_mm(struct vdpasim *vdpasim, + struct vdpasim_mm_work *mm_work) +{ + struct kthread_work *work = &mm_work->work; + + kthread_init_work(work, vdpasim_mm_work_fn); + kthread_queue_work(vdpasim->worker, work); + + kthread_flush_work(work); +} + +static int vdpasim_worker_bind_mm(struct vdpasim *vdpasim, + struct mm_struct *new_mm) +{ + struct vdpasim_mm_work mm_work; + + mm_work.mm = new_mm; + mm_work.bind = true; + + vdpasim_worker_queue_mm(vdpasim, &mm_work); + + if (!mm_work.ret) + vdpasim->mm_bound = new_mm; + + return mm_work.ret; +} + +static void vdpasim_worker_unbind_mm(struct vdpasim *vdpasim) +{ + struct vdpasim_mm_work mm_work; + + if (!vdpasim->mm_bound) + return; + + mm_work.mm = vdpasim->mm_bound; + mm_work.bind = false; + + vdpasim_worker_queue_mm(vdpasim, &mm_work); + + vdpasim->mm_bound = NULL; +} static struct vdpasim *vdpa_to_sim(struct vdpa_device *vdpa) { return container_of(vdpa, struct vdpasim, vdpa); @@ -59,8 +126,10 @@ static void vdpasim_queue_ready(struct vdpasim *vdpasim, unsigned int idx) { struct vdpasim_virtqueue *vq = &vdpasim->vqs[idx]; uint16_t last_avail_idx = vq->vring.last_avail_idx; + bool va_enabled = use_va && vdpasim->mm_bound; - vringh_init_iotlb(&vq->vring, vdpasim->features, vq->num, true, false, + vringh_init_iotlb(&vq->vring, vdpasim->features, vq->num, true, + va_enabled, (struct vring_desc *)(uintptr_t)vq->desc_addr, (struct vring_avail *) (uintptr_t)vq->driver_addr, @@ -151,7 +220,7 @@ struct vdpasim *vdpasim_create(struct vdpasim_dev_attr *dev_attr, vdpa = __vdpa_alloc_device(NULL, ops, dev_attr->ngroups, dev_attr->nas, dev_attr->alloc_size, - dev_attr->name, false); + dev_attr->name, use_va); if (IS_ERR(vdpa)) { ret = PTR_ERR(vdpa); goto err_alloc; @@ -571,6 +640,27 @@ static int vdpasim_set_map(struct vdpa_device *vdpa, unsigned int asid, return ret; } +static int vdpasim_bind_mm(struct vdpa_device *vdpa, struct mm_struct *mm) +{ + struct vdpasim *vdpasim = vdpa_to_sim(vdpa); + int ret; + + mutex_lock(&vdpasim->mutex); + ret = vdpasim_worker_bind_mm(vdpasim, mm); + mutex_unlock(&vdpasim->mutex); + + return ret; +} + +static void vdpasim_unbind_mm(struct vdpa_device *vdpa) +{ + struct vdpasim *vdpasim = vdpa_to_sim(vdpa); + + mutex_lock(&vdpasim->mutex); + vdpasim_worker_unbind_mm(vdpasim); + mutex_unlock(&vdpasim->mutex); +} + static int vdpasim_dma_map(struct vdpa_device *vdpa, unsigned int asid, u64 iova, u64 size, u64 pa, u32 perm, void *opaque) @@ -667,6 +757,8 @@ static const struct vdpa_config_ops vdpasim_config_ops = { .set_group_asid = vdpasim_set_group_asid, .dma_map = vdpasim_dma_map, .dma_unmap = vdpasim_dma_unmap, + .bind_mm = vdpasim_bind_mm, + .unbind_mm = vdpasim_unbind_mm, .free = vdpasim_free, }; @@ -701,6 +793,8 @@ static const struct vdpa_config_ops vdpasim_batch_config_ops = { .get_iova_range = vdpasim_get_iova_range, .set_group_asid = vdpasim_set_group_asid, .set_map = vdpasim_set_map, + .bind_mm = vdpasim_bind_mm, + .unbind_mm = vdpasim_unbind_mm, .free = vdpasim_free, };