From patchwork Thu Apr 27 17:35:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Reinette Chatre X-Patchwork-Id: 88329 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp430242vqo; Thu, 27 Apr 2023 10:39:37 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4ZD/E3hRD4YkZrSelz1Y1DCaVZGikPDSn9bF7b0uk2RSompwswADaovhib24F5RrlBtX4F X-Received: by 2002:a17:903:2341:b0:1a2:8924:2230 with SMTP id c1-20020a170903234100b001a289242230mr8867099plh.27.1682617177470; Thu, 27 Apr 2023 10:39:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682617177; cv=none; d=google.com; s=arc-20160816; b=evPVqEzh+4qAk2GyJqpl0ReOcvNJ8HzVx5GoU2g0sg7C7Ky0YRQ9WoEZ9F5kPKWAjh XrH/wwp04sjVMAfw/MATN7d90agHWxc1srxQW/Ixjmfzx9MMBXPnGmK6OIYBs438wbCp ZlUoa3HeOrPYPKTQlZK/AKV0+uyjLilueASks/TSns10rwbtaR9IFCGOyQ2CfsqnPA6O rpPysdvoEwMKkKfAzzqcRLPYLtn1dy+t9mDY6zl1JMHWxAVhw4gzViv9t7W2J5Ze6DDY B4eZWqIcnBG/6eIvXNOpp47dLNmqndtfgr74j8Kidbg55ewr1Wt3KOMqhkB41E0lpNRA FX1g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=4ulh1A+8iZD4YqBFAp5REgvIZtLSou8CsI8b/KwCdjU=; b=W4Byq40UkUlU/tAStxfx1BVYVV02/4mKK+qopIvSen7FIUMMsfGb34xSJRnLumMm3G icXDRYaVm88AxcQjaFBQK/+dJWTEEVNqYysZ3L5Z6wtAYyhb3VVS9Lj2dPEdLYUzpcYo wRLoWqgPRcStk6++iQyN+vHlQ3wmtuRR7nuEwzdPrtfrCgd9PxAtnORfrhgdxmyCfO3g o4F7UF6UAAu4fnHjSkmm+vL2pqapWtkpnkdu1sMbDAShsmQ8eQ6LXINVYYpauL8syjzw GMsRBazJ7W1BgIgefC2NKJsrSVBZcsM/RJq8olBbY7hAnw7wt3qG27TXX5Enxdw2K5zZ VvLg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=D2JMoQQ1; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id f11-20020a170902684b00b001a229e52c19si19114994pln.91.2023.04.27.10.39.23; Thu, 27 Apr 2023 10:39:37 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=D2JMoQQ1; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244600AbjD0RhH (ORCPT + 99 others); Thu, 27 Apr 2023 13:37:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41014 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244416AbjD0RgY (ORCPT ); Thu, 27 Apr 2023 13:36:24 -0400 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 022CD3C15; Thu, 27 Apr 2023 10:36:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1682616982; x=1714152982; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=dECZEsHgNKDAg2NJhsGmsEPmmLEkwgLOFPBHSm8iUiM=; b=D2JMoQQ1r7w/LhPl6fC9iexEYTO0RSC1iJy5E2BL1FviDlTVtchgUTdB FNSP18bLuphvVg4Sf1fDh61USLB2xyiQqjIrYI1wAtsKqrUhqoI0lmduO LC6sncdr7SouXoE0pj7O8mUuJ202OAuQx3u3TNTmKfMOrdcNUcSvw4pCV OmnTdw9kmKce/AfV/l/Tl4v3U23vkJFFeF73XzG3wET9AsOonKTDZr8K2 2UHQzZkV8/MlbnPeHOxsPFoKKQRrwivrerT4KinL4F2gdI7RviMrqqg2+ A1YaAT6nSbaFMJVtnB3V8bOyxbwgro4ABkHeSlWN5AMZzmtqgVdJNIRix g==; X-IronPort-AV: E=McAfee;i="6600,9927,10693"; a="349496871" X-IronPort-AV: E=Sophos;i="5.99,232,1677571200"; d="scan'208";a="349496871" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Apr 2023 10:36:20 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10693"; a="697172962" X-IronPort-AV: E=Sophos;i="5.99,232,1677571200"; d="scan'208";a="697172962" Received: from rchatre-ws.ostc.intel.com ([10.54.69.144]) by fmsmga007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Apr 2023 10:36:19 -0700 From: Reinette Chatre To: jgg@nvidia.com, yishaih@nvidia.com, shameerali.kolothum.thodi@huawei.com, kevin.tian@intel.com, alex.williamson@redhat.com Cc: tglx@linutronix.de, darwi@linutronix.de, kvm@vger.kernel.org, dave.jiang@intel.com, jing2.liu@intel.com, ashok.raj@intel.com, fenghua.yu@intel.com, tom.zanussi@linux.intel.com, reinette.chatre@intel.com, linux-kernel@vger.kernel.org Subject: [PATCH V4 01/11] vfio/pci: Consolidate irq cleanup on MSI/MSI-X disable Date: Thu, 27 Apr 2023 10:35:58 -0700 Message-Id: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1764351989470259606?= X-GMAIL-MSGID: =?utf-8?q?1764351989470259606?= vfio_msi_disable() releases all previously allocated state associated with each interrupt before disabling MSI/MSI-X. vfio_msi_disable() iterates twice over the interrupt state: first directly with a for loop to do virqfd cleanup, followed by another for loop within vfio_msi_set_block() that removes the interrupt handler and its associated state using vfio_msi_set_vector_signal(). Simplify interrupt cleanup by iterating over allocated interrupts once. Signed-off-by: Reinette Chatre Reviewed-by: Kevin Tian --- No changes since V3. Changes since V2: - Improve accuracy of changelog. drivers/vfio/pci/vfio_pci_intrs.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_intrs.c b/drivers/vfio/pci/vfio_pci_intrs.c index bffb0741518b..6a9c6a143cc3 100644 --- a/drivers/vfio/pci/vfio_pci_intrs.c +++ b/drivers/vfio/pci/vfio_pci_intrs.c @@ -426,10 +426,9 @@ static void vfio_msi_disable(struct vfio_pci_core_device *vdev, bool msix) for (i = 0; i < vdev->num_ctx; i++) { vfio_virqfd_disable(&vdev->ctx[i].unmask); vfio_virqfd_disable(&vdev->ctx[i].mask); + vfio_msi_set_vector_signal(vdev, i, -1, msix); } - vfio_msi_set_block(vdev, 0, vdev->num_ctx, NULL, msix); - cmd = vfio_pci_memory_lock_and_enable(vdev); pci_free_irq_vectors(pdev); vfio_pci_memory_unlock_and_restore(vdev, cmd); From patchwork Thu Apr 27 17:35:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Reinette Chatre X-Patchwork-Id: 88342 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp434033vqo; Thu, 27 Apr 2023 10:48:01 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4jcIOJaL+Nhpk6XB4S+8RsM8mQap+KtTmSK3kLYiIFgvqU88NLan3xX87qQN2Jc9M7Xvia X-Received: by 2002:a17:903:41d2:b0:1a9:b91f:63fc with SMTP id u18-20020a17090341d200b001a9b91f63fcmr2545320ple.12.1682617681203; Thu, 27 Apr 2023 10:48:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682617681; cv=none; d=google.com; s=arc-20160816; b=W0Dv8nM9ziQ1YNH4M1aJq5j9G3QAn5IfOik4rqsE/YCTlSbgP9rROiEWZVCu9H0/A4 ggKtsZU7AmieL0qY7OcVGqK3qxMaKVR6onYM7yzNJjKpYMAmI13jGqeUQk1+zkdxcVF6 jWf6mYhOVwbK+SPT5XRmzl3y2pJMDiAuLl2oGvxSAHHG6FGiD+1vXaX85/7y0rTG+Wyb 4btF3hNbezSj+BCqJdGFQNpz0idgdjVRTzVmrYhki6zoBcer76G7bErvcD5TIjdeScnr wzguYLR/fO8QdXC55MparII952ev3cuO3+DYN4/xDtj9c8R6U/IMySWAl8wXu0TBWEiy YgXA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=nQl+3O+AyXHpAWbTtLz2D9nxDsGEfKIia9SFa2Tu8qg=; b=h9SX1+JvXT3oGb4MsPnGBgdHNbQEHBu8uyV21rfmNS2ZClK8AwBnuQUuUi0Eqy3z0S HOS8xa0BBSiz3h0rdlRrjat+zND9m7WDzokC6B/9B5slQ85KGMobTnsW1ersATmszUW0 S203bTJKop5n//XU9QPgO02Yi+bM+yR6JXGhvB+uHYamc66AsobPsHg0qWcjdXyIUkSC IAJ53J+yvj63fC/Cfc7UL0sfEw43Q2dLPkfXu0jgJ2BPcEtu9LwHAmJ1UevB4P8s3N+i 2DZAjC0EnyIuGmrwLogF9hc8kgR8DRHiWIcn69MJkq8DZyiC2X28ELeQGVRyMhC4BkUT dgcw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=gS3DBvMM; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id j5-20020a170903024500b001a55571febesi21347328plh.277.2023.04.27.10.47.48; Thu, 27 Apr 2023 10:48:01 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=gS3DBvMM; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244490AbjD0Rgv (ORCPT + 99 others); Thu, 27 Apr 2023 13:36:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41030 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244420AbjD0RgY (ORCPT ); Thu, 27 Apr 2023 13:36:24 -0400 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CBF113C39; Thu, 27 Apr 2023 10:36:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1682616982; x=1714152982; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=FPa5Zz9lQ8mt2gPp5QecV+WcSW7u6oYI/fcqZRu8FDI=; b=gS3DBvMMQKoyIY9awQDkhPrmaXDSt4aQzTk6fzGt5WUPcohBZda++swy O8TZzK9zcKgbpzgMzuhhfwPtk9Wris68B5R/IWeFuVP/qO1NGroAWbEYq BJzrYW0SD7RQhCpUgJV+eBgO4BDvj1uHf0flqSxRE52tPd7Tnp+ourEf/ HMVbuislvLEUOsHHduHUBd4ElzsbTQDYKsVgz/s3xrlO19t6pjUssT1xK jJnPTVKe3yosdI0SXdS7+C/HeOzBX9KMinuXKl3WUV7/27WvsLWmO2iS3 W9Gr6CwzqrsB6LZfWM2fZA+NXBAwmkj+ybbMAvFc8lv1x2zA6aeX/qd36 Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10693"; a="349496878" X-IronPort-AV: E=Sophos;i="5.99,232,1677571200"; d="scan'208";a="349496878" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Apr 2023 10:36:20 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10693"; a="697172966" X-IronPort-AV: E=Sophos;i="5.99,232,1677571200"; d="scan'208";a="697172966" Received: from rchatre-ws.ostc.intel.com ([10.54.69.144]) by fmsmga007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Apr 2023 10:36:20 -0700 From: Reinette Chatre To: jgg@nvidia.com, yishaih@nvidia.com, shameerali.kolothum.thodi@huawei.com, kevin.tian@intel.com, alex.williamson@redhat.com Cc: tglx@linutronix.de, darwi@linutronix.de, kvm@vger.kernel.org, dave.jiang@intel.com, jing2.liu@intel.com, ashok.raj@intel.com, fenghua.yu@intel.com, tom.zanussi@linux.intel.com, reinette.chatre@intel.com, linux-kernel@vger.kernel.org Subject: [PATCH V4 02/11] vfio/pci: Remove negative check on unsigned vector Date: Thu, 27 Apr 2023 10:35:59 -0700 Message-Id: <5add301d11d4a566c29c487a78b4227ae383f11d.1682615447.git.reinette.chatre@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1764352517835178972?= X-GMAIL-MSGID: =?utf-8?q?1764352517835178972?= User space provides the vector as an unsigned int that is checked early for validity (vfio_set_irqs_validate_and_prepare()). A later negative check of the provided vector is not necessary. Remove the negative check and ensure the type used for the vector is consistent as an unsigned int. Signed-off-by: Reinette Chatre Reviewed-by: Kevin Tian --- No changes since V3. Changes since V2: - Rework unwind loop within vfio_msi_set_block() that required j to be an int. Rework results in both i and j used for vector, both moved to be unsigned int. (Alex) drivers/vfio/pci/vfio_pci_intrs.c | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_intrs.c b/drivers/vfio/pci/vfio_pci_intrs.c index 6a9c6a143cc3..258de57ef956 100644 --- a/drivers/vfio/pci/vfio_pci_intrs.c +++ b/drivers/vfio/pci/vfio_pci_intrs.c @@ -317,14 +317,14 @@ static int vfio_msi_enable(struct vfio_pci_core_device *vdev, int nvec, bool msi } static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, - int vector, int fd, bool msix) + unsigned int vector, int fd, bool msix) { struct pci_dev *pdev = vdev->pdev; struct eventfd_ctx *trigger; int irq, ret; u16 cmd; - if (vector < 0 || vector >= vdev->num_ctx) + if (vector >= vdev->num_ctx) return -EINVAL; irq = pci_irq_vector(pdev, vector); @@ -399,7 +399,8 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, static int vfio_msi_set_block(struct vfio_pci_core_device *vdev, unsigned start, unsigned count, int32_t *fds, bool msix) { - int i, j, ret = 0; + unsigned int i, j; + int ret = 0; if (start >= vdev->num_ctx || start + count > vdev->num_ctx) return -EINVAL; @@ -410,8 +411,8 @@ static int vfio_msi_set_block(struct vfio_pci_core_device *vdev, unsigned start, } if (ret) { - for (--j; j >= (int)start; j--) - vfio_msi_set_vector_signal(vdev, j, -1, msix); + for (i = start; i < j; i++) + vfio_msi_set_vector_signal(vdev, i, -1, msix); } return ret; @@ -420,7 +421,7 @@ static int vfio_msi_set_block(struct vfio_pci_core_device *vdev, unsigned start, static void vfio_msi_disable(struct vfio_pci_core_device *vdev, bool msix) { struct pci_dev *pdev = vdev->pdev; - int i; + unsigned int i; u16 cmd; for (i = 0; i < vdev->num_ctx; i++) { @@ -542,7 +543,7 @@ static int vfio_pci_set_msi_trigger(struct vfio_pci_core_device *vdev, unsigned index, unsigned start, unsigned count, uint32_t flags, void *data) { - int i; + unsigned int i; bool msix = (index == VFIO_PCI_MSIX_IRQ_INDEX) ? true : false; if (irq_is(vdev, index) && !count && (flags & VFIO_IRQ_SET_DATA_NONE)) { From patchwork Thu Apr 27 17:36:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Reinette Chatre X-Patchwork-Id: 88333 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp433583vqo; Thu, 27 Apr 2023 10:47:03 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7836JWOPgurHmRvtvk3fTh5xieiiCX8WXZ/wtDyOXgs8O4Db6/0iTYYBHG1qrmWdSsDLKc X-Received: by 2002:a05:6a20:3c9f:b0:f2:45fd:1de2 with SMTP id b31-20020a056a203c9f00b000f245fd1de2mr8448838pzj.19.1682617622784; Thu, 27 Apr 2023 10:47:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682617622; cv=none; d=google.com; s=arc-20160816; b=uGwOPgmcxb3v5b86RogTychoNXAvyhTDxdIX0YXB+iqKCKUUeA2EdZEeKgVqO0Wes7 XyTrM+ENvZ4RwULX3uX9Kh+fUD7OSCsM4UdqSkoebLcrzXuVCxRAJyfwgjzKfReq46Ad YzplCm6fG7pKLfDC5KasLWhEqxm72Y069bJqaAFd5fgK9MJRTLkxGIa2uoBV9GP4aLyZ jjbdFCb5/yAuOOxJ+RA6tOnBCsDB0PuFSMez8OHDh4w+BZVdCBgBYJREuY/oZzn0cgHo HR5v8/Z987SNBUxeBdb3N6+JhhjRf21nifhnfZPKgtKveUfRPjZTI4pt1BNYyE069WTs 0i5A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=kQejDBtpIcTHMuAMpGDGh5cSCk6Gx1vRWIzEcgaXm5Y=; b=ZDhDxNYxWvYktS/I87xHtrm8dpKAxiKytc0cbZbUtPuAdjx9pqQVegb2eXDNrmu2fl qexjNiO4cLSVLtr//QWYL2MxbNl6jfGeHX2c/ORpRof6mcOzUZM5BY+5TJu9AcDfp0ak pFGEdQf7EQ0Uvj4Dm0WqXok5Zy7kSE707HsViDOCXKLwNav3kMMVS6qEUdIwSVp5t3YF e3SNnMTQHS5npWD5cSVvtTvlaoE8T942Pzi6+uba8H1HUPvaD9mBmtlyv0r4BvMxBa2Q 7H02+z9eSAubuTFEN3W8Kkr+C6aAxPUW8Fdn0+UPbfQmJ/4m8omDmh+qV8mGOLAo74R8 MQdA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=nqs+Wz+M; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 63-20020a630142000000b00520b3928be3si19356681pgb.329.2023.04.27.10.46.47; Thu, 27 Apr 2023 10:47:02 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=nqs+Wz+M; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244521AbjD0Rgy (ORCPT + 99 others); Thu, 27 Apr 2023 13:36:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41058 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244428AbjD0Rg0 (ORCPT ); Thu, 27 Apr 2023 13:36:26 -0400 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0DBFB3584; Thu, 27 Apr 2023 10:36:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1682616983; x=1714152983; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=md25LnaVmBGc4U+FZeYEKgswv71ook8yDdmIU9PpL6c=; b=nqs+Wz+MKkE4iVw/urx69U6zprBo1tp3dhAa6EY2bNBLRV7XQF0hT5Ea CN2TOiJGDezM5iZVAaBnofYulevBKJjjUEh8wtaqBjlmzp2KPg6XZJmfa VzOByh4/Epm0GtM/h4C+sEVyFILXDjK+ZjLPLtBbOJGHemiqtpo2dOXdH zxr59xvAwhAS1kR6ZdBP0r6TprZHH5lVc3tw+HLB7GeWjzDRYnZRDTH2l DXu6pXcs/fx8pdDAlQJzL5WPCfYy97VgA2halrnyHI1bURFOOr/TauXU2 cH6cN8GnTr1KRG3ml73szvHGtcVUVDnH/MSymB1N1wJGunwvthkZo+WFK w==; X-IronPort-AV: E=McAfee;i="6600,9927,10693"; a="349496886" X-IronPort-AV: E=Sophos;i="5.99,232,1677571200"; d="scan'208";a="349496886" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Apr 2023 10:36:20 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10693"; a="697172970" X-IronPort-AV: E=Sophos;i="5.99,232,1677571200"; d="scan'208";a="697172970" Received: from rchatre-ws.ostc.intel.com ([10.54.69.144]) by fmsmga007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Apr 2023 10:36:20 -0700 From: Reinette Chatre To: jgg@nvidia.com, yishaih@nvidia.com, shameerali.kolothum.thodi@huawei.com, kevin.tian@intel.com, alex.williamson@redhat.com Cc: tglx@linutronix.de, darwi@linutronix.de, kvm@vger.kernel.org, dave.jiang@intel.com, jing2.liu@intel.com, ashok.raj@intel.com, fenghua.yu@intel.com, tom.zanussi@linux.intel.com, reinette.chatre@intel.com, linux-kernel@vger.kernel.org Subject: [PATCH V4 03/11] vfio/pci: Prepare for dynamic interrupt context storage Date: Thu, 27 Apr 2023 10:36:00 -0700 Message-Id: <6fcd4019e22931a97d962b6e657e74d6fd1049ba.1682615447.git.reinette.chatre@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1764352456177099495?= X-GMAIL-MSGID: =?utf-8?q?1764352456177099495?= Interrupt context storage is statically allocated at the time interrupts are allocated. Following allocation, the interrupt context is managed by directly accessing the elements of the array using the vector as index. It is possible to allocate additional MSI-X vectors after MSI-X has been enabled. Dynamic storage of interrupt context is needed to support adding new MSI-X vectors after initial allocation. Replace direct access of array elements with pointers to the array elements. Doing so reduces impact of moving to a new data structure. Move interactions with the array to helpers to mostly contain changes needed to transition to a dynamic data structure. No functional change intended. Signed-off-by: Reinette Chatre --- No changes since V2. Changes since RFC V1: - Improve accuracy of changelog. drivers/vfio/pci/vfio_pci_intrs.c | 206 ++++++++++++++++++++---------- 1 file changed, 140 insertions(+), 66 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_intrs.c b/drivers/vfio/pci/vfio_pci_intrs.c index 258de57ef956..b664fbb6d2f2 100644 --- a/drivers/vfio/pci/vfio_pci_intrs.c +++ b/drivers/vfio/pci/vfio_pci_intrs.c @@ -48,6 +48,31 @@ static bool is_irq_none(struct vfio_pci_core_device *vdev) vdev->irq_type == VFIO_PCI_MSIX_IRQ_INDEX); } +static +struct vfio_pci_irq_ctx *vfio_irq_ctx_get(struct vfio_pci_core_device *vdev, + unsigned long index) +{ + if (index >= vdev->num_ctx) + return NULL; + return &vdev->ctx[index]; +} + +static void vfio_irq_ctx_free_all(struct vfio_pci_core_device *vdev) +{ + kfree(vdev->ctx); +} + +static int vfio_irq_ctx_alloc_num(struct vfio_pci_core_device *vdev, + unsigned long num) +{ + vdev->ctx = kcalloc(num, sizeof(struct vfio_pci_irq_ctx), + GFP_KERNEL_ACCOUNT); + if (!vdev->ctx) + return -ENOMEM; + + return 0; +} + /* * INTx */ @@ -55,17 +80,28 @@ static void vfio_send_intx_eventfd(void *opaque, void *unused) { struct vfio_pci_core_device *vdev = opaque; - if (likely(is_intx(vdev) && !vdev->virq_disabled)) - eventfd_signal(vdev->ctx[0].trigger, 1); + if (likely(is_intx(vdev) && !vdev->virq_disabled)) { + struct vfio_pci_irq_ctx *ctx; + + ctx = vfio_irq_ctx_get(vdev, 0); + if (!ctx) + return; + eventfd_signal(ctx->trigger, 1); + } } /* Returns true if the INTx vfio_pci_irq_ctx.masked value is changed. */ bool vfio_pci_intx_mask(struct vfio_pci_core_device *vdev) { struct pci_dev *pdev = vdev->pdev; + struct vfio_pci_irq_ctx *ctx; unsigned long flags; bool masked_changed = false; + ctx = vfio_irq_ctx_get(vdev, 0); + if (!ctx) + return masked_changed; + spin_lock_irqsave(&vdev->irqlock, flags); /* @@ -77,7 +113,7 @@ bool vfio_pci_intx_mask(struct vfio_pci_core_device *vdev) if (unlikely(!is_intx(vdev))) { if (vdev->pci_2_3) pci_intx(pdev, 0); - } else if (!vdev->ctx[0].masked) { + } else if (!ctx->masked) { /* * Can't use check_and_mask here because we always want to * mask, not just when something is pending. @@ -87,7 +123,7 @@ bool vfio_pci_intx_mask(struct vfio_pci_core_device *vdev) else disable_irq_nosync(pdev->irq); - vdev->ctx[0].masked = true; + ctx->masked = true; masked_changed = true; } @@ -105,9 +141,14 @@ static int vfio_pci_intx_unmask_handler(void *opaque, void *unused) { struct vfio_pci_core_device *vdev = opaque; struct pci_dev *pdev = vdev->pdev; + struct vfio_pci_irq_ctx *ctx; unsigned long flags; int ret = 0; + ctx = vfio_irq_ctx_get(vdev, 0); + if (!ctx) + return ret; + spin_lock_irqsave(&vdev->irqlock, flags); /* @@ -117,7 +158,7 @@ static int vfio_pci_intx_unmask_handler(void *opaque, void *unused) if (unlikely(!is_intx(vdev))) { if (vdev->pci_2_3) pci_intx(pdev, 1); - } else if (vdev->ctx[0].masked && !vdev->virq_disabled) { + } else if (ctx->masked && !vdev->virq_disabled) { /* * A pending interrupt here would immediately trigger, * but we can avoid that overhead by just re-sending @@ -129,7 +170,7 @@ static int vfio_pci_intx_unmask_handler(void *opaque, void *unused) } else enable_irq(pdev->irq); - vdev->ctx[0].masked = (ret > 0); + ctx->masked = (ret > 0); } spin_unlock_irqrestore(&vdev->irqlock, flags); @@ -146,18 +187,23 @@ void vfio_pci_intx_unmask(struct vfio_pci_core_device *vdev) static irqreturn_t vfio_intx_handler(int irq, void *dev_id) { struct vfio_pci_core_device *vdev = dev_id; + struct vfio_pci_irq_ctx *ctx; unsigned long flags; int ret = IRQ_NONE; + ctx = vfio_irq_ctx_get(vdev, 0); + if (!ctx) + return ret; + spin_lock_irqsave(&vdev->irqlock, flags); if (!vdev->pci_2_3) { disable_irq_nosync(vdev->pdev->irq); - vdev->ctx[0].masked = true; + ctx->masked = true; ret = IRQ_HANDLED; - } else if (!vdev->ctx[0].masked && /* may be shared */ + } else if (!ctx->masked && /* may be shared */ pci_check_and_mask_intx(vdev->pdev)) { - vdev->ctx[0].masked = true; + ctx->masked = true; ret = IRQ_HANDLED; } @@ -171,15 +217,24 @@ static irqreturn_t vfio_intx_handler(int irq, void *dev_id) static int vfio_intx_enable(struct vfio_pci_core_device *vdev) { + struct vfio_pci_irq_ctx *ctx; + int ret; + if (!is_irq_none(vdev)) return -EINVAL; if (!vdev->pdev->irq) return -ENODEV; - vdev->ctx = kzalloc(sizeof(struct vfio_pci_irq_ctx), GFP_KERNEL_ACCOUNT); - if (!vdev->ctx) - return -ENOMEM; + ret = vfio_irq_ctx_alloc_num(vdev, 1); + if (ret) + return ret; + + ctx = vfio_irq_ctx_get(vdev, 0); + if (!ctx) { + vfio_irq_ctx_free_all(vdev); + return -EINVAL; + } vdev->num_ctx = 1; @@ -189,9 +244,9 @@ static int vfio_intx_enable(struct vfio_pci_core_device *vdev) * here, non-PCI-2.3 devices will have to wait until the * interrupt is enabled. */ - vdev->ctx[0].masked = vdev->virq_disabled; + ctx->masked = vdev->virq_disabled; if (vdev->pci_2_3) - pci_intx(vdev->pdev, !vdev->ctx[0].masked); + pci_intx(vdev->pdev, !ctx->masked); vdev->irq_type = VFIO_PCI_INTX_IRQ_INDEX; @@ -202,41 +257,46 @@ static int vfio_intx_set_signal(struct vfio_pci_core_device *vdev, int fd) { struct pci_dev *pdev = vdev->pdev; unsigned long irqflags = IRQF_SHARED; + struct vfio_pci_irq_ctx *ctx; struct eventfd_ctx *trigger; unsigned long flags; int ret; - if (vdev->ctx[0].trigger) { + ctx = vfio_irq_ctx_get(vdev, 0); + if (!ctx) + return -EINVAL; + + if (ctx->trigger) { free_irq(pdev->irq, vdev); - kfree(vdev->ctx[0].name); - eventfd_ctx_put(vdev->ctx[0].trigger); - vdev->ctx[0].trigger = NULL; + kfree(ctx->name); + eventfd_ctx_put(ctx->trigger); + ctx->trigger = NULL; } if (fd < 0) /* Disable only */ return 0; - vdev->ctx[0].name = kasprintf(GFP_KERNEL_ACCOUNT, "vfio-intx(%s)", - pci_name(pdev)); - if (!vdev->ctx[0].name) + ctx->name = kasprintf(GFP_KERNEL_ACCOUNT, "vfio-intx(%s)", + pci_name(pdev)); + if (!ctx->name) return -ENOMEM; trigger = eventfd_ctx_fdget(fd); if (IS_ERR(trigger)) { - kfree(vdev->ctx[0].name); + kfree(ctx->name); return PTR_ERR(trigger); } - vdev->ctx[0].trigger = trigger; + ctx->trigger = trigger; if (!vdev->pci_2_3) irqflags = 0; ret = request_irq(pdev->irq, vfio_intx_handler, - irqflags, vdev->ctx[0].name, vdev); + irqflags, ctx->name, vdev); if (ret) { - vdev->ctx[0].trigger = NULL; - kfree(vdev->ctx[0].name); + ctx->trigger = NULL; + kfree(ctx->name); eventfd_ctx_put(trigger); return ret; } @@ -246,7 +306,7 @@ static int vfio_intx_set_signal(struct vfio_pci_core_device *vdev, int fd) * disable_irq won't. */ spin_lock_irqsave(&vdev->irqlock, flags); - if (!vdev->pci_2_3 && vdev->ctx[0].masked) + if (!vdev->pci_2_3 && ctx->masked) disable_irq_nosync(pdev->irq); spin_unlock_irqrestore(&vdev->irqlock, flags); @@ -255,12 +315,17 @@ static int vfio_intx_set_signal(struct vfio_pci_core_device *vdev, int fd) static void vfio_intx_disable(struct vfio_pci_core_device *vdev) { - vfio_virqfd_disable(&vdev->ctx[0].unmask); - vfio_virqfd_disable(&vdev->ctx[0].mask); + struct vfio_pci_irq_ctx *ctx; + + ctx = vfio_irq_ctx_get(vdev, 0); + if (ctx) { + vfio_virqfd_disable(&ctx->unmask); + vfio_virqfd_disable(&ctx->mask); + } vfio_intx_set_signal(vdev, -1); vdev->irq_type = VFIO_PCI_NUM_IRQS; vdev->num_ctx = 0; - kfree(vdev->ctx); + vfio_irq_ctx_free_all(vdev); } /* @@ -284,10 +349,9 @@ static int vfio_msi_enable(struct vfio_pci_core_device *vdev, int nvec, bool msi if (!is_irq_none(vdev)) return -EINVAL; - vdev->ctx = kcalloc(nvec, sizeof(struct vfio_pci_irq_ctx), - GFP_KERNEL_ACCOUNT); - if (!vdev->ctx) - return -ENOMEM; + ret = vfio_irq_ctx_alloc_num(vdev, nvec); + if (ret) + return ret; /* return the number of supported vectors if we can't get all: */ cmd = vfio_pci_memory_lock_and_enable(vdev); @@ -296,7 +360,7 @@ static int vfio_msi_enable(struct vfio_pci_core_device *vdev, int nvec, bool msi if (ret > 0) pci_free_irq_vectors(pdev); vfio_pci_memory_unlock_and_restore(vdev, cmd); - kfree(vdev->ctx); + vfio_irq_ctx_free_all(vdev); return ret; } vfio_pci_memory_unlock_and_restore(vdev, cmd); @@ -320,6 +384,7 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, unsigned int vector, int fd, bool msix) { struct pci_dev *pdev = vdev->pdev; + struct vfio_pci_irq_ctx *ctx; struct eventfd_ctx *trigger; int irq, ret; u16 cmd; @@ -327,33 +392,33 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, if (vector >= vdev->num_ctx) return -EINVAL; + ctx = vfio_irq_ctx_get(vdev, vector); + if (!ctx) + return -EINVAL; irq = pci_irq_vector(pdev, vector); - if (vdev->ctx[vector].trigger) { - irq_bypass_unregister_producer(&vdev->ctx[vector].producer); + if (ctx->trigger) { + irq_bypass_unregister_producer(&ctx->producer); cmd = vfio_pci_memory_lock_and_enable(vdev); - free_irq(irq, vdev->ctx[vector].trigger); + free_irq(irq, ctx->trigger); vfio_pci_memory_unlock_and_restore(vdev, cmd); - - kfree(vdev->ctx[vector].name); - eventfd_ctx_put(vdev->ctx[vector].trigger); - vdev->ctx[vector].trigger = NULL; + kfree(ctx->name); + eventfd_ctx_put(ctx->trigger); + ctx->trigger = NULL; } if (fd < 0) return 0; - vdev->ctx[vector].name = kasprintf(GFP_KERNEL_ACCOUNT, - "vfio-msi%s[%d](%s)", - msix ? "x" : "", vector, - pci_name(pdev)); - if (!vdev->ctx[vector].name) + ctx->name = kasprintf(GFP_KERNEL_ACCOUNT, "vfio-msi%s[%d](%s)", + msix ? "x" : "", vector, pci_name(pdev)); + if (!ctx->name) return -ENOMEM; trigger = eventfd_ctx_fdget(fd); if (IS_ERR(trigger)) { - kfree(vdev->ctx[vector].name); + kfree(ctx->name); return PTR_ERR(trigger); } @@ -372,26 +437,25 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, pci_write_msi_msg(irq, &msg); } - ret = request_irq(irq, vfio_msihandler, 0, - vdev->ctx[vector].name, trigger); + ret = request_irq(irq, vfio_msihandler, 0, ctx->name, trigger); vfio_pci_memory_unlock_and_restore(vdev, cmd); if (ret) { - kfree(vdev->ctx[vector].name); + kfree(ctx->name); eventfd_ctx_put(trigger); return ret; } - vdev->ctx[vector].producer.token = trigger; - vdev->ctx[vector].producer.irq = irq; - ret = irq_bypass_register_producer(&vdev->ctx[vector].producer); + ctx->producer.token = trigger; + ctx->producer.irq = irq; + ret = irq_bypass_register_producer(&ctx->producer); if (unlikely(ret)) { dev_info(&pdev->dev, "irq bypass producer (token %p) registration fails: %d\n", - vdev->ctx[vector].producer.token, ret); + ctx->producer.token, ret); - vdev->ctx[vector].producer.token = NULL; + ctx->producer.token = NULL; } - vdev->ctx[vector].trigger = trigger; + ctx->trigger = trigger; return 0; } @@ -421,13 +485,17 @@ static int vfio_msi_set_block(struct vfio_pci_core_device *vdev, unsigned start, static void vfio_msi_disable(struct vfio_pci_core_device *vdev, bool msix) { struct pci_dev *pdev = vdev->pdev; + struct vfio_pci_irq_ctx *ctx; unsigned int i; u16 cmd; for (i = 0; i < vdev->num_ctx; i++) { - vfio_virqfd_disable(&vdev->ctx[i].unmask); - vfio_virqfd_disable(&vdev->ctx[i].mask); - vfio_msi_set_vector_signal(vdev, i, -1, msix); + ctx = vfio_irq_ctx_get(vdev, i); + if (ctx) { + vfio_virqfd_disable(&ctx->unmask); + vfio_virqfd_disable(&ctx->mask); + vfio_msi_set_vector_signal(vdev, i, -1, msix); + } } cmd = vfio_pci_memory_lock_and_enable(vdev); @@ -443,7 +511,7 @@ static void vfio_msi_disable(struct vfio_pci_core_device *vdev, bool msix) vdev->irq_type = VFIO_PCI_NUM_IRQS; vdev->num_ctx = 0; - kfree(vdev->ctx); + vfio_irq_ctx_free_all(vdev); } /* @@ -463,14 +531,18 @@ static int vfio_pci_set_intx_unmask(struct vfio_pci_core_device *vdev, if (unmask) vfio_pci_intx_unmask(vdev); } else if (flags & VFIO_IRQ_SET_DATA_EVENTFD) { + struct vfio_pci_irq_ctx *ctx = vfio_irq_ctx_get(vdev, 0); int32_t fd = *(int32_t *)data; + + if (!ctx) + return -EINVAL; if (fd >= 0) return vfio_virqfd_enable((void *) vdev, vfio_pci_intx_unmask_handler, vfio_send_intx_eventfd, NULL, - &vdev->ctx[0].unmask, fd); + &ctx->unmask, fd); - vfio_virqfd_disable(&vdev->ctx[0].unmask); + vfio_virqfd_disable(&ctx->unmask); } return 0; @@ -543,6 +615,7 @@ static int vfio_pci_set_msi_trigger(struct vfio_pci_core_device *vdev, unsigned index, unsigned start, unsigned count, uint32_t flags, void *data) { + struct vfio_pci_irq_ctx *ctx; unsigned int i; bool msix = (index == VFIO_PCI_MSIX_IRQ_INDEX) ? true : false; @@ -577,14 +650,15 @@ static int vfio_pci_set_msi_trigger(struct vfio_pci_core_device *vdev, return -EINVAL; for (i = start; i < start + count; i++) { - if (!vdev->ctx[i].trigger) + ctx = vfio_irq_ctx_get(vdev, i); + if (!ctx || !ctx->trigger) continue; if (flags & VFIO_IRQ_SET_DATA_NONE) { - eventfd_signal(vdev->ctx[i].trigger, 1); + eventfd_signal(ctx->trigger, 1); } else if (flags & VFIO_IRQ_SET_DATA_BOOL) { uint8_t *bools = data; if (bools[i - start]) - eventfd_signal(vdev->ctx[i].trigger, 1); + eventfd_signal(ctx->trigger, 1); } } return 0; From patchwork Thu Apr 27 17:36:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Reinette Chatre X-Patchwork-Id: 88325 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp429833vqo; Thu, 27 Apr 2023 10:38:38 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6y0KpZtVyfFZchgJvvtPT8N1I97ej+53x0hTSOQEiNPgm5jQuXH0rRwHo07uN711UoHvqt X-Received: by 2002:a05:6a20:9189:b0:f3:5cf7:581b with SMTP id v9-20020a056a20918900b000f35cf7581bmr2992493pzd.28.1682617118286; Thu, 27 Apr 2023 10:38:38 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682617118; cv=none; d=google.com; s=arc-20160816; b=EYhlWSynPbdJz1w+qD+kqeAyJzz9WbZ4ECMXhrOg+i1NgmUuJ9nZCvUMJa67Ko0czJ 4rlmcky48PcPzyDt0Gqm4apc2wiFSsZ4GqIXoMqrWkaDjb4XfE8gBZx0onh9UVb7lq1N tD47kun9re9dE1oiBr5z9Ga1OZHGfiTCTYfrQIvEHPOWJSfnwhB/5NZLhoy+7wo18Zwr 4fF7hjMytrd5iHDBFmUX8kcvx61IUf1l6GtNupqCwTxY2SE5GL00AUzgxg3KupMmea0q 5MecJTCPVaBZd7f7fiu8hIc65GumY1cPLr3K95rvvtun0p9RuUbgciw+l15+IKVYIeK3 ZrKw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=TeK7rQwJISA/4Sel6S+3kVTI1S47lGVTs8JyC+k5P3E=; b=xf2+wT3zLYEm6YIktc9uvjuN+KcY8Z1ZrF4alaektCPi3ss9oNeElK7W3o8HYWo4DV DpHNQPvmPVU52bf6Z2f23DKaytnehe4MEyy2hIuxilqgmzwvxUp1sWsf/cFXwvE7n8+6 yFN5oW0BElf4O9q7IXGYVo9+4+pqwpJB40RGcdck8aSMaJeg5w7zu3zA526S6PWF8iTh YAEOBS7uuCoyIjaAkhNl+VU3/gCxYYY8GIITebiNBtRZgI0U84tjktbflAk5V0oTsV1o JnAFWgIlUVKSm+aOLIHHnZPGiGLD0BniYnSP9+yZVXWLPW4hl+CV07ACkq+WsoJ4luNo g2Gw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=OjmGPtE9; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id o3-20020a63fb03000000b0050c0549c4dbsi20177258pgh.702.2023.04.27.10.38.24; Thu, 27 Apr 2023 10:38:38 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=OjmGPtE9; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244545AbjD0Rg5 (ORCPT + 99 others); Thu, 27 Apr 2023 13:36:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41040 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244423AbjD0RgZ (ORCPT ); Thu, 27 Apr 2023 13:36:25 -0400 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 47B3E3A82; Thu, 27 Apr 2023 10:36:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1682616984; x=1714152984; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=1ZvtNhskQo1n54fwtOBEZ3264EZTpF7b30t34MsMeZM=; b=OjmGPtE94Pq78FToZkKDeMmKsaJloN5N5r5QUF1+MMPengZ8DXkGufQI mNts4AfJTHvymMtsxaKF6YBknAKldp/yzdw8GEe3K5DOvTyvk+G/bC1/3 XKUF50nLWePAH37jbaEGLjf4LQZTJddUY+zoBjgBxtoNx0I8bZU7KVvW7 p5u1e4PsXaaSNEXNyQ3d5Djr3pJi2A+TlsASXIbUWIxr10oLKm8yrjMVA 97AnwSDx4oFl34vzx1PSgNX9/fUnZHl+YNjLwNscJl28BbL7zax9JAg15 1nE7ms3kVF3ciD1B+VHLGSz/lTP6O2H3yG8awniR4oM97excSpie+ipfR g==; X-IronPort-AV: E=McAfee;i="6600,9927,10693"; a="349496895" X-IronPort-AV: E=Sophos;i="5.99,232,1677571200"; d="scan'208";a="349496895" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Apr 2023 10:36:20 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10693"; a="697172973" X-IronPort-AV: E=Sophos;i="5.99,232,1677571200"; d="scan'208";a="697172973" Received: from rchatre-ws.ostc.intel.com ([10.54.69.144]) by fmsmga007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Apr 2023 10:36:20 -0700 From: Reinette Chatre To: jgg@nvidia.com, yishaih@nvidia.com, shameerali.kolothum.thodi@huawei.com, kevin.tian@intel.com, alex.williamson@redhat.com Cc: tglx@linutronix.de, darwi@linutronix.de, kvm@vger.kernel.org, dave.jiang@intel.com, jing2.liu@intel.com, ashok.raj@intel.com, fenghua.yu@intel.com, tom.zanussi@linux.intel.com, reinette.chatre@intel.com, linux-kernel@vger.kernel.org Subject: [PATCH V4 04/11] vfio/pci: Move to single error path Date: Thu, 27 Apr 2023 10:36:01 -0700 Message-Id: <521fd5184ef052ff768a90bbe670cfc4e375eff9.1682615447.git.reinette.chatre@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1764351927668702136?= X-GMAIL-MSGID: =?utf-8?q?1764351927668702136?= Enabling and disabling of an interrupt involves several steps that can fail. Cleanup after failure is done when the error is encountered, resulting in some repetitive code. Support for dynamic contexts will introduce more steps during interrupt enabling and disabling. Transition to centralized exit path in preparation for dynamic contexts to eliminate duplicate error handling code. Signed-off-by: Reinette Chatre Reviewed-by: Kevin Tian --- No changes since V3. Changes since V2: - Move patch to earlier in series in support of the change to dynamic context management. - Do not add the "ctx->name = NULL" in error path. It is not done in baseline and will not be needed when transitioning to dynamic context management. - Update changelog to not make this change specific to dynamic MSI-X. Changes since RFC V1: - Improve changelog. drivers/vfio/pci/vfio_pci_intrs.c | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_intrs.c b/drivers/vfio/pci/vfio_pci_intrs.c index b664fbb6d2f2..9e17e59a4d60 100644 --- a/drivers/vfio/pci/vfio_pci_intrs.c +++ b/drivers/vfio/pci/vfio_pci_intrs.c @@ -418,8 +418,8 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, trigger = eventfd_ctx_fdget(fd); if (IS_ERR(trigger)) { - kfree(ctx->name); - return PTR_ERR(trigger); + ret = PTR_ERR(trigger); + goto out_free_name; } /* @@ -439,11 +439,8 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, ret = request_irq(irq, vfio_msihandler, 0, ctx->name, trigger); vfio_pci_memory_unlock_and_restore(vdev, cmd); - if (ret) { - kfree(ctx->name); - eventfd_ctx_put(trigger); - return ret; - } + if (ret) + goto out_put_eventfd_ctx; ctx->producer.token = trigger; ctx->producer.irq = irq; @@ -458,6 +455,12 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, ctx->trigger = trigger; return 0; + +out_put_eventfd_ctx: + eventfd_ctx_put(trigger); +out_free_name: + kfree(ctx->name); + return ret; } static int vfio_msi_set_block(struct vfio_pci_core_device *vdev, unsigned start, From patchwork Thu Apr 27 17:36:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Reinette Chatre X-Patchwork-Id: 88328 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp430113vqo; Thu, 27 Apr 2023 10:39:17 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5hcHHeNlEOgFmnNpP+SHuNf6E+ACz1OQK+MSBJD8oePJMX8Sfw/CG3WV/8FJ1unyFTiKht X-Received: by 2002:a05:6a00:2350:b0:63b:8eeb:77b8 with SMTP id j16-20020a056a00235000b0063b8eeb77b8mr3532987pfj.13.1682617157283; Thu, 27 Apr 2023 10:39:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682617157; cv=none; d=google.com; s=arc-20160816; b=o0K7V/ML/efWjDfxIu6KVG1Fb2OV/xEYRMoZbm5h37Zrp+WqC/ecESx2d3Wmc8TweF c4jplSmfAMcPOqVYPkVWMKvMpT5A78EJVNcrj1yUnSDZFYazNFZ5yMLqo0To3xilQPfU qECozcXgd07vzZM9IpVBGkW7gmd4JCv+QI3QRCuLmjgl82XRg+Autg/1gqDuAf20tSbe 4/9DPjGNXNv45x1qxI4y0JJxC3OuCwE/ox4D+WWoPLV+U3y10R7KBEsFHb/Hnc/xeGXo YoU//dpy+iM/Ac4fc1fAdjhbrKsjRUO3EJmZZ3dTBJGlvrNi1qB7TF2xJOY+vDx9ujS6 4OoQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=I0NB21bon+UwNvfsL3wPr5EqE0OI7IKez8Q4Hmyn06g=; b=bQxLMfCjfRqmZsdjpZmu8d5Znc/Equx3TofDxVY7taQ64FzUIwsulGpifjYD2G6AXE QZtCXs+T9JcfS3vBVip58MF+u75P9QTpnxtsHj0dHaGKo5bZbbL0QjAZu9/2WzTy64iX pkW2yeIOjVeuJrKrlmUBmqg9afNJqZD96qQnYsA8/lecnbXfzayWxmGW2hN/1Tzyc8gB Y4lNM/uCEvJz9ov3bdLk1BXnoFmdqhU3cK4xyC5/5ywf+bc1BPLo5ANvcgZksPhpZeJE 0PzxnHAh6NNFYaXexYgtXWzxVRzgMuK0U5SjRRlR4lFF+UqbrogYPrW63ozDxlog5OlF uAXw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=U9iKTlv8; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id q23-20020aa79837000000b0063b648880adsi18658416pfl.239.2023.04.27.10.39.03; Thu, 27 Apr 2023 10:39:17 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=U9iKTlv8; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244462AbjD0Rgr (ORCPT + 99 others); Thu, 27 Apr 2023 13:36:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41060 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244430AbjD0Rg0 (ORCPT ); Thu, 27 Apr 2023 13:36:26 -0400 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BE23330EE; Thu, 27 Apr 2023 10:36:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1682616984; x=1714152984; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=m9u9MJu3UaAOcMVDIbUZV6NTYDxJcYlI8ihIF3NktQI=; b=U9iKTlv8DXDFICbOd6TbNR3X68s5hkxoar50MhCoRdjxqVcDRL8KO2WI Th8kMwfW9Yatl+yePIKVPh1lxNmLUtN1uNTrclrVMUGR5s5ywlmcfhIIS 0vFzwcroA5L6IttuErB0uMvoCtvzJCrmnkJWttJ3IQPwyo+oFvoTNU1mK bocfRgvu7MGJL1RQgMrtfHkFOGw2QpsZ21Z1S4/KlsSLdcfJCjhG0V7mF zoo3b7xvYkPKt8LGKH4UwVgV/MPt9c+MIT5DAQWv9dL0j77vc4CCValOq QG4kzLoVnymrci5vN6aa8GartIwqJ24p3X4OePxgBgBHS1voXcJbJ964o w==; X-IronPort-AV: E=McAfee;i="6600,9927,10693"; a="349496903" X-IronPort-AV: E=Sophos;i="5.99,232,1677571200"; d="scan'208";a="349496903" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Apr 2023 10:36:21 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10693"; a="697172978" X-IronPort-AV: E=Sophos;i="5.99,232,1677571200"; d="scan'208";a="697172978" Received: from rchatre-ws.ostc.intel.com ([10.54.69.144]) by fmsmga007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Apr 2023 10:36:20 -0700 From: Reinette Chatre To: jgg@nvidia.com, yishaih@nvidia.com, shameerali.kolothum.thodi@huawei.com, kevin.tian@intel.com, alex.williamson@redhat.com Cc: tglx@linutronix.de, darwi@linutronix.de, kvm@vger.kernel.org, dave.jiang@intel.com, jing2.liu@intel.com, ashok.raj@intel.com, fenghua.yu@intel.com, tom.zanussi@linux.intel.com, reinette.chatre@intel.com, linux-kernel@vger.kernel.org Subject: [PATCH V4 05/11] vfio/pci: Use xarray for interrupt context storage Date: Thu, 27 Apr 2023 10:36:02 -0700 Message-Id: <78182c9cd770885b6d354f114ba157c7024c8b39.1682615447.git.reinette.chatre@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1764351968640231411?= X-GMAIL-MSGID: =?utf-8?q?1764351968640231411?= Interrupt context is statically allocated at the time interrupts are allocated. Following allocation, the context is managed by directly accessing the elements of the array using the vector as index. The storage is released when interrupts are disabled. It is possible to dynamically allocate a single MSI-X interrupt after MSI-X is enabled. A dynamic storage for interrupt context is needed to support this. Replace the interrupt context array with an xarray (similar to what the core uses as store for MSI descriptors) that can support the dynamic expansion while maintaining the custom that uses the vector as index. With a dynamic storage it is no longer required to pre-allocate interrupt contexts at the time the interrupts are allocated. MSI and MSI-X interrupt contexts are only used when interrupts are enabled. Their allocation can thus be delayed until interrupt enabling. Only enabled interrupts will have associated interrupt contexts. Whether an interrupt has been allocated (a Linux irq number exists for it) becomes the criteria for whether an interrupt can be enabled. Signed-off-by: Reinette Chatre Link: https://lore.kernel.org/lkml/20230404122444.59e36a99.alex.williamson@redhat.com/ Reviewed-by: Kevin Tian --- No changes since V3. Changes since V2: - Only allocate contexts as they are used, or "active". (Alex) - Move vfio_irq_ctx_free() from later patch to prevent open-coding the same within vfio_irq_ctx_free_all(). This evolved into vfio_irq_ctx_free() used for dynamic context allocation and vfio_irq_ctx_free_all() removed because of it. (Alex) - With vfio_irq_ctx_alloc_num() removed, rename vfio_irq_ctx_alloc_single() to vfio_irq_ctx_alloc(). Changes since RFC V1: - Let vfio_irq_ctx_alloc_single() return pointer to allocated context. (Alex) - Transition INTx allocation to simplified vfio_irq_ctx_alloc_single(). - Improve accuracy of changelog. drivers/vfio/pci/vfio_pci_core.c | 1 + drivers/vfio/pci/vfio_pci_intrs.c | 91 ++++++++++++++++--------------- include/linux/vfio_pci_core.h | 2 +- 3 files changed, 48 insertions(+), 46 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c index a5ab416cf476..ae0e161c7fc9 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -2102,6 +2102,7 @@ int vfio_pci_core_init_dev(struct vfio_device *core_vdev) INIT_LIST_HEAD(&vdev->vma_list); INIT_LIST_HEAD(&vdev->sriov_pfs_item); init_rwsem(&vdev->memory_lock); + xa_init(&vdev->ctx); return 0; } diff --git a/drivers/vfio/pci/vfio_pci_intrs.c b/drivers/vfio/pci/vfio_pci_intrs.c index 9e17e59a4d60..117cd384b3ad 100644 --- a/drivers/vfio/pci/vfio_pci_intrs.c +++ b/drivers/vfio/pci/vfio_pci_intrs.c @@ -52,25 +52,33 @@ static struct vfio_pci_irq_ctx *vfio_irq_ctx_get(struct vfio_pci_core_device *vdev, unsigned long index) { - if (index >= vdev->num_ctx) - return NULL; - return &vdev->ctx[index]; + return xa_load(&vdev->ctx, index); } -static void vfio_irq_ctx_free_all(struct vfio_pci_core_device *vdev) +static void vfio_irq_ctx_free(struct vfio_pci_core_device *vdev, + struct vfio_pci_irq_ctx *ctx, unsigned long index) { - kfree(vdev->ctx); + xa_erase(&vdev->ctx, index); + kfree(ctx); } -static int vfio_irq_ctx_alloc_num(struct vfio_pci_core_device *vdev, - unsigned long num) +static struct vfio_pci_irq_ctx * +vfio_irq_ctx_alloc(struct vfio_pci_core_device *vdev, unsigned long index) { - vdev->ctx = kcalloc(num, sizeof(struct vfio_pci_irq_ctx), - GFP_KERNEL_ACCOUNT); - if (!vdev->ctx) - return -ENOMEM; + struct vfio_pci_irq_ctx *ctx; + int ret; - return 0; + ctx = kzalloc(sizeof(*ctx), GFP_KERNEL_ACCOUNT); + if (!ctx) + return NULL; + + ret = xa_insert(&vdev->ctx, index, ctx, GFP_KERNEL_ACCOUNT); + if (ret) { + kfree(ctx); + return NULL; + } + + return ctx; } /* @@ -218,7 +226,6 @@ static irqreturn_t vfio_intx_handler(int irq, void *dev_id) static int vfio_intx_enable(struct vfio_pci_core_device *vdev) { struct vfio_pci_irq_ctx *ctx; - int ret; if (!is_irq_none(vdev)) return -EINVAL; @@ -226,15 +233,9 @@ static int vfio_intx_enable(struct vfio_pci_core_device *vdev) if (!vdev->pdev->irq) return -ENODEV; - ret = vfio_irq_ctx_alloc_num(vdev, 1); - if (ret) - return ret; - - ctx = vfio_irq_ctx_get(vdev, 0); - if (!ctx) { - vfio_irq_ctx_free_all(vdev); - return -EINVAL; - } + ctx = vfio_irq_ctx_alloc(vdev, 0); + if (!ctx) + return -ENOMEM; vdev->num_ctx = 1; @@ -325,7 +326,7 @@ static void vfio_intx_disable(struct vfio_pci_core_device *vdev) vfio_intx_set_signal(vdev, -1); vdev->irq_type = VFIO_PCI_NUM_IRQS; vdev->num_ctx = 0; - vfio_irq_ctx_free_all(vdev); + vfio_irq_ctx_free(vdev, ctx, 0); } /* @@ -349,10 +350,6 @@ static int vfio_msi_enable(struct vfio_pci_core_device *vdev, int nvec, bool msi if (!is_irq_none(vdev)) return -EINVAL; - ret = vfio_irq_ctx_alloc_num(vdev, nvec); - if (ret) - return ret; - /* return the number of supported vectors if we can't get all: */ cmd = vfio_pci_memory_lock_and_enable(vdev); ret = pci_alloc_irq_vectors(pdev, 1, nvec, flag); @@ -360,7 +357,6 @@ static int vfio_msi_enable(struct vfio_pci_core_device *vdev, int nvec, bool msi if (ret > 0) pci_free_irq_vectors(pdev); vfio_pci_memory_unlock_and_restore(vdev, cmd); - vfio_irq_ctx_free_all(vdev); return ret; } vfio_pci_memory_unlock_and_restore(vdev, cmd); @@ -392,12 +388,13 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, if (vector >= vdev->num_ctx) return -EINVAL; - ctx = vfio_irq_ctx_get(vdev, vector); - if (!ctx) - return -EINVAL; irq = pci_irq_vector(pdev, vector); + if (irq < 0) + return -EINVAL; - if (ctx->trigger) { + ctx = vfio_irq_ctx_get(vdev, vector); + + if (ctx) { irq_bypass_unregister_producer(&ctx->producer); cmd = vfio_pci_memory_lock_and_enable(vdev); @@ -405,16 +402,22 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, vfio_pci_memory_unlock_and_restore(vdev, cmd); kfree(ctx->name); eventfd_ctx_put(ctx->trigger); - ctx->trigger = NULL; + vfio_irq_ctx_free(vdev, ctx, vector); } if (fd < 0) return 0; + ctx = vfio_irq_ctx_alloc(vdev, vector); + if (!ctx) + return -ENOMEM; + ctx->name = kasprintf(GFP_KERNEL_ACCOUNT, "vfio-msi%s[%d](%s)", msix ? "x" : "", vector, pci_name(pdev)); - if (!ctx->name) - return -ENOMEM; + if (!ctx->name) { + ret = -ENOMEM; + goto out_free_ctx; + } trigger = eventfd_ctx_fdget(fd); if (IS_ERR(trigger)) { @@ -460,6 +463,8 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, eventfd_ctx_put(trigger); out_free_name: kfree(ctx->name); +out_free_ctx: + vfio_irq_ctx_free(vdev, ctx, vector); return ret; } @@ -489,16 +494,13 @@ static void vfio_msi_disable(struct vfio_pci_core_device *vdev, bool msix) { struct pci_dev *pdev = vdev->pdev; struct vfio_pci_irq_ctx *ctx; - unsigned int i; + unsigned long i; u16 cmd; - for (i = 0; i < vdev->num_ctx; i++) { - ctx = vfio_irq_ctx_get(vdev, i); - if (ctx) { - vfio_virqfd_disable(&ctx->unmask); - vfio_virqfd_disable(&ctx->mask); - vfio_msi_set_vector_signal(vdev, i, -1, msix); - } + xa_for_each(&vdev->ctx, i, ctx) { + vfio_virqfd_disable(&ctx->unmask); + vfio_virqfd_disable(&ctx->mask); + vfio_msi_set_vector_signal(vdev, i, -1, msix); } cmd = vfio_pci_memory_lock_and_enable(vdev); @@ -514,7 +516,6 @@ static void vfio_msi_disable(struct vfio_pci_core_device *vdev, bool msix) vdev->irq_type = VFIO_PCI_NUM_IRQS; vdev->num_ctx = 0; - vfio_irq_ctx_free_all(vdev); } /* @@ -654,7 +655,7 @@ static int vfio_pci_set_msi_trigger(struct vfio_pci_core_device *vdev, for (i = start; i < start + count; i++) { ctx = vfio_irq_ctx_get(vdev, i); - if (!ctx || !ctx->trigger) + if (!ctx) continue; if (flags & VFIO_IRQ_SET_DATA_NONE) { eventfd_signal(ctx->trigger, 1); diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h index 367fd79226a3..61d7873a3973 100644 --- a/include/linux/vfio_pci_core.h +++ b/include/linux/vfio_pci_core.h @@ -59,7 +59,7 @@ struct vfio_pci_core_device { struct perm_bits *msi_perm; spinlock_t irqlock; struct mutex igate; - struct vfio_pci_irq_ctx *ctx; + struct xarray ctx; int num_ctx; int irq_type; int num_regions; From patchwork Thu Apr 27 17:36:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Reinette Chatre X-Patchwork-Id: 88339 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp433786vqo; Thu, 27 Apr 2023 10:47:30 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6afWm7AFHn1McQCA6DKMRUjyNpND/iPwHQrTfByIv/uqNgbtVlH2sHAHhQ14MUUkzbwpVF X-Received: by 2002:a17:902:e549:b0:1a2:8866:e8a4 with SMTP id n9-20020a170902e54900b001a28866e8a4mr3179015plf.1.1682617649988; Thu, 27 Apr 2023 10:47:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682617649; cv=none; d=google.com; s=arc-20160816; b=H/V4g6mL3f9BAHniK52CMHz2OUgWoNAOu4valW7GYN3auWMnBC4TNqwtaYQ8uJ/x73 GKuOs5Jdyft+wCIwPKMz1MUOJ2Epy95SU7a5BR2G+fpXjVzgsS5rcnnC8o5FWklkF/fZ Mtelc3ZXOxTlsxoPi6dJjAF8HjqPwDC4betGnTGDhRKr0+KpPMhPs7veAKLgM3kb3WFE xQG6txuQfFfy80bCRXICKfM414I9o+O/tShzljeTLep9hXjNQaPeswui/j9g1BSbEM8p +Bu5KQKhvwap8JrD4RMYZOfACMruaXceQXsU6mataiD24Pb5EQTZ4z0+q23tmut/wH2O JfSQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=5xtYPrIvLAJhpOfrqQNu19puN4Ple75YIublu9GzB+M=; b=NCGDQSQebxY6dgGBvgdB7E8qneiYdSFsXMf83JHm086TcI5HAWgdRLtESFsuZdUOzg j3GZKpep4pBatHKLGmifRXXyOfZNte7MheYCru93lGhRf03cDZL9uC+eDydz1zTCJ3Lu cdJp3eo/Nw8oynKurMpMzpIWyvF7BZVf5/3yu4TZXvKMCFRnUtInVf5uhyi8Vp8uzI75 tyZ3LrPHcI8BBbKlk2sYm1wm9Yu8+ahLxiWxJk6NyCzRtWa8jnBz0v3uke7U5Hj/FBYw 1StXvfxnbu67AE5aClEWqoetBwnpV49OFtizbD3/JR1GJzrwUxnI71YVTdXoQHX6jerh kRnQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=B+4QC7d0; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id j5-20020a170903024500b0019ca54e71e4si20281582plh.224.2023.04.27.10.47.16; Thu, 27 Apr 2023 10:47:29 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=B+4QC7d0; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244560AbjD0Rg7 (ORCPT + 99 others); Thu, 27 Apr 2023 13:36:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41072 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243520AbjD0Rg1 (ORCPT ); Thu, 27 Apr 2023 13:36:27 -0400 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9B1403C21; Thu, 27 Apr 2023 10:36:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1682616985; x=1714152985; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Pn9lW2UtIuohqa4cfdAN2Wg7lcE/Ap5BLuKRV+7PaWI=; b=B+4QC7d0jSIEgWRadXKXKZnqLV9eoAkw1Cwh9jpn8ZwFus3wrEr0ev3w 4GjP0gQ9au8rBi2+I8onboNOpLXY2ssIBZew6tkkebLdQo+S6n5r4k2p1 Zs3wcbtBC8MZ9OAX5DsizeRL46KDRRpfYn3v3qMoXjKv9xp2NSpwWIdCp OS6A+LGXXeESEEZ/2iUf1jmeS4dSub5XllsRFO00cf9odqkzqH9E3M5Y7 HwJL7jrhasZVW3T3QpPTjPQpSD7+1OrkWYshZejgumCdSxNcBEjwqF7jO bFaaRKCoCs4AMRXLlper4Oer/vp30sVyL4NJhIcEm21X3sDS39JKjMq9c w==; X-IronPort-AV: E=McAfee;i="6600,9927,10693"; a="349496910" X-IronPort-AV: E=Sophos;i="5.99,232,1677571200"; d="scan'208";a="349496910" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Apr 2023 10:36:21 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10693"; a="697172982" X-IronPort-AV: E=Sophos;i="5.99,232,1677571200"; d="scan'208";a="697172982" Received: from rchatre-ws.ostc.intel.com ([10.54.69.144]) by fmsmga007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Apr 2023 10:36:21 -0700 From: Reinette Chatre To: jgg@nvidia.com, yishaih@nvidia.com, shameerali.kolothum.thodi@huawei.com, kevin.tian@intel.com, alex.williamson@redhat.com Cc: tglx@linutronix.de, darwi@linutronix.de, kvm@vger.kernel.org, dave.jiang@intel.com, jing2.liu@intel.com, ashok.raj@intel.com, fenghua.yu@intel.com, tom.zanussi@linux.intel.com, reinette.chatre@intel.com, linux-kernel@vger.kernel.org Subject: [PATCH V4 06/11] vfio/pci: Remove interrupt context counter Date: Thu, 27 Apr 2023 10:36:03 -0700 Message-Id: <056fbd6c7c5161fb912d60b3f75e379ab3255d75.1682615447.git.reinette.chatre@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1764352484938654891?= X-GMAIL-MSGID: =?utf-8?q?1764352484938654891?= struct vfio_pci_core_device::num_ctx counts how many interrupt contexts have been allocated. When all interrupt contexts are allocated simultaneously num_ctx provides the upper bound of all vectors that can be used as indices into the interrupt context array. With the upcoming support for dynamic MSI-X the number of interrupt contexts does not necessarily span the range of allocated interrupts. Consequently, num_ctx is no longer a trusted upper bound for valid indices. Stop using num_ctx to determine if a provided vector is valid. Use the existence of allocated interrupt. This changes behavior on the error path when user space provides an invalid vector range. Behavior changes from early exit without any modifications to possible modifications to valid vectors within the invalid range. This is acceptable considering that an invalid range is not a valid scenario, see link to discussion. The checks that ensure that user space provides a range of vectors that is valid for the device are untouched. Signed-off-by: Reinette Chatre Link: https://lore.kernel.org/lkml/20230316155646.07ae266f.alex.williamson@redhat.com/ Reviewed-by: Kevin Tian --- No changes since V3. Changes since V2: - Update changelog to reflect change in policy that existence of allocated interrupt is validity check, not existence of context (which is now dynamically allocated). Changes since RFC V1: - Remove vfio_irq_ctx_range_allocated(). (Alex and Kevin). drivers/vfio/pci/vfio_pci_intrs.c | 13 +------------ include/linux/vfio_pci_core.h | 1 - 2 files changed, 1 insertion(+), 13 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_intrs.c b/drivers/vfio/pci/vfio_pci_intrs.c index 117cd384b3ad..5e3de004f4cb 100644 --- a/drivers/vfio/pci/vfio_pci_intrs.c +++ b/drivers/vfio/pci/vfio_pci_intrs.c @@ -237,8 +237,6 @@ static int vfio_intx_enable(struct vfio_pci_core_device *vdev) if (!ctx) return -ENOMEM; - vdev->num_ctx = 1; - /* * If the virtual interrupt is masked, restore it. Devices * supporting DisINTx can be masked at the hardware level @@ -325,7 +323,6 @@ static void vfio_intx_disable(struct vfio_pci_core_device *vdev) } vfio_intx_set_signal(vdev, -1); vdev->irq_type = VFIO_PCI_NUM_IRQS; - vdev->num_ctx = 0; vfio_irq_ctx_free(vdev, ctx, 0); } @@ -361,7 +358,6 @@ static int vfio_msi_enable(struct vfio_pci_core_device *vdev, int nvec, bool msi } vfio_pci_memory_unlock_and_restore(vdev, cmd); - vdev->num_ctx = nvec; vdev->irq_type = msix ? VFIO_PCI_MSIX_IRQ_INDEX : VFIO_PCI_MSI_IRQ_INDEX; @@ -385,9 +381,6 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, int irq, ret; u16 cmd; - if (vector >= vdev->num_ctx) - return -EINVAL; - irq = pci_irq_vector(pdev, vector); if (irq < 0) return -EINVAL; @@ -474,9 +467,6 @@ static int vfio_msi_set_block(struct vfio_pci_core_device *vdev, unsigned start, unsigned int i, j; int ret = 0; - if (start >= vdev->num_ctx || start + count > vdev->num_ctx) - return -EINVAL; - for (i = 0, j = start; i < count && !ret; i++, j++) { int fd = fds ? fds[i] : -1; ret = vfio_msi_set_vector_signal(vdev, j, fd, msix); @@ -515,7 +505,6 @@ static void vfio_msi_disable(struct vfio_pci_core_device *vdev, bool msix) pci_intx(pdev, 0); vdev->irq_type = VFIO_PCI_NUM_IRQS; - vdev->num_ctx = 0; } /* @@ -650,7 +639,7 @@ static int vfio_pci_set_msi_trigger(struct vfio_pci_core_device *vdev, return ret; } - if (!irq_is(vdev, index) || start + count > vdev->num_ctx) + if (!irq_is(vdev, index)) return -EINVAL; for (i = start; i < start + count; i++) { diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h index 61d7873a3973..148fd1ae6c1c 100644 --- a/include/linux/vfio_pci_core.h +++ b/include/linux/vfio_pci_core.h @@ -60,7 +60,6 @@ struct vfio_pci_core_device { spinlock_t irqlock; struct mutex igate; struct xarray ctx; - int num_ctx; int irq_type; int num_regions; struct vfio_pci_region *region; From patchwork Thu Apr 27 17:36:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Reinette Chatre X-Patchwork-Id: 88330 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp430287vqo; Thu, 27 Apr 2023 10:39:42 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4z3D3yGIpSz5JWvJdyQbAxUj5EHaRxgKr5uTyQGaHx01MUoSrTxjDNbKlu2bpubfMziFs2 X-Received: by 2002:aa7:88d0:0:b0:63d:4407:b6c with SMTP id k16-20020aa788d0000000b0063d44070b6cmr3236861pff.7.1682617181722; Thu, 27 Apr 2023 10:39:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682617181; cv=none; d=google.com; s=arc-20160816; b=Fztzs6pwMiAWp4PqRw0NjRUTOYL4hzHyWDxYedZui5EyJj9lnheavQfAN9r/ZevZiK B1DIPIuRh/DhtikjZruamdlY7YIcGbR22eQ8LCCMU+ta5BaaR06AQ7lePj07vbb5mkR1 GelsiyYzjx9fRiWxWX4XLyVYecbyyZOgElU029Yi8U0HJZramzQstCS6vfalgN74tu7h CKYEe43qG1fLAFNkJ3DVDkaYImNPrRT4rGmk5znR92/JAlEy97QUEg0cAAeP/Ox2ZL8R soMWY6Qs+p0KU9pbyfXZj+5O3e2UUTCBNOhocI/uFFhuMtgO3AxxPd6+ytasuCjX5894 oTNg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=I4uQ9UTyPM3LJS9eUpiq8DpmM3I+Q67FNUIpKVHyv+E=; b=nkz5NV42NLq8CjcCWiUqP1G3TJ7v5HWUbfdrvx9714pFsX1z7zM22lMT2gCjSQn6AE Z5CClRoai4bTTAiffHIj8dYlN+TZwfscF+EZDVcw9rOf9GXJ4douZtIqSl8UUmscUqwg fMSYoP3D+3OKo+n3bFZarSNYG/Q1HwaG96wqVA4jFrUtJx15GO6C7e4b1YiD56UlWNtn +T8G1W4PLDLpZVmMquhK23WSKLQ1qGdYjl+94a6pliQMHCj31RzMX1eXVa3r/jUYE3rq gwwfISiHAJirTOKemA/21NVVTx3Y1ElfNFIbNLuGEUf64sgi+c4rbUv49t8CBulxu4Fp HNQA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=AsvZtfkh; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id d69-20020a621d48000000b0063b7bc5f3a8si19300548pfd.86.2023.04.27.10.39.27; Thu, 27 Apr 2023 10:39:41 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=AsvZtfkh; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244607AbjD0RhI (ORCPT + 99 others); Thu, 27 Apr 2023 13:37:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41080 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244436AbjD0Rg1 (ORCPT ); Thu, 27 Apr 2023 13:36:27 -0400 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 78D9035A4; Thu, 27 Apr 2023 10:36:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1682616986; x=1714152986; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=cSgLlGdyxaXFMa3vrlcEIL4XkCkVpS7t5FiSBXiFMow=; b=AsvZtfkhjWhYkIOzSUUTF8NOt5+p58ne09Qs98uPd/DtS/hTsEakcDjg Ta/csH/pS0i5YfvXvFV5GeTjnCsqTHdWN+cERCpImxXO6f0atJDN+nXgO c5Yfn2o8FERX6Xo0i9Ua1b9k450+u8zL0QiJsvBe9jEZzs9vBCf7Lejlb VRzbejDlazqAuYKYvSrVsb+tMMSS+7ts9zeNv6Q6TtEShkLNXFUxoLxTp W0ImXyZfOA5YYgV9w/ZJZlH5okkeSvDJFt55H1wBJ0dLFT3npBrm/QRAz klTQJzPo3Q+2vXYrm3+b41Cv9NuiHMBKwZnhcF97dvOnLty2WqYJ9KLrS Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10693"; a="349496916" X-IronPort-AV: E=Sophos;i="5.99,232,1677571200"; d="scan'208";a="349496916" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Apr 2023 10:36:21 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10693"; a="697172986" X-IronPort-AV: E=Sophos;i="5.99,232,1677571200"; d="scan'208";a="697172986" Received: from rchatre-ws.ostc.intel.com ([10.54.69.144]) by fmsmga007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Apr 2023 10:36:21 -0700 From: Reinette Chatre To: jgg@nvidia.com, yishaih@nvidia.com, shameerali.kolothum.thodi@huawei.com, kevin.tian@intel.com, alex.williamson@redhat.com Cc: tglx@linutronix.de, darwi@linutronix.de, kvm@vger.kernel.org, dave.jiang@intel.com, jing2.liu@intel.com, ashok.raj@intel.com, fenghua.yu@intel.com, tom.zanussi@linux.intel.com, reinette.chatre@intel.com, linux-kernel@vger.kernel.org Subject: [PATCH V4 07/11] vfio/pci: Update stale comment Date: Thu, 27 Apr 2023 10:36:04 -0700 Message-Id: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1764351994388998824?= X-GMAIL-MSGID: =?utf-8?q?1764351994388998824?= In preparation for surrounding code change it is helpful to ensure that existing comments are accurate. Remove inaccurate comment about direct access and update the rest of the comment to reflect the purpose of writing the cached MSI message to the device. Suggested-by: Alex Williamson Link: https://lore.kernel.org/lkml/20230330164050.0069e2a5.alex.williamson@redhat.com/ Signed-off-by: Reinette Chatre --- No changes since V3. Changes since V2: - New patch. drivers/vfio/pci/vfio_pci_intrs.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_intrs.c b/drivers/vfio/pci/vfio_pci_intrs.c index 5e3de004f4cb..bdda7f46c2be 100644 --- a/drivers/vfio/pci/vfio_pci_intrs.c +++ b/drivers/vfio/pci/vfio_pci_intrs.c @@ -419,11 +419,9 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, } /* - * The MSIx vector table resides in device memory which may be cleared - * via backdoor resets. We don't allow direct access to the vector - * table so even if a userspace driver attempts to save/restore around - * such a reset it would be unsuccessful. To avoid this, restore the - * cached value of the message prior to enabling. + * If the vector was previously allocated, refresh the on-device + * message data before enabling in case it had been cleared or + * corrupted since writing. */ cmd = vfio_pci_memory_lock_and_enable(vdev); if (msix) { From patchwork Thu Apr 27 17:36:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Reinette Chatre X-Patchwork-Id: 88334 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp433615vqo; Thu, 27 Apr 2023 10:47:06 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5Avu8S1kZoekDXu5lk1yMB1Eja/PUJ/nfbcRMr+krXeNyYIyGIaBYi5atz8zRR7LeZTMc+ X-Received: by 2002:a05:6a00:803:b0:63b:7fa1:f1b1 with SMTP id m3-20020a056a00080300b0063b7fa1f1b1mr3802356pfk.26.1682617626601; Thu, 27 Apr 2023 10:47:06 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682617626; cv=none; d=google.com; s=arc-20160816; b=NBTB3GjFm4fDJkzm0UpqLnckscdT4bfYka9ryiw9tlLpbFF+JE9O7hJLf8k0k+qJmN 8Kt0AEfrXuJS7yNKh/3PG8fe6moXaYHMIHldQCLAcN5JUfHrohAAijCRHZnHe4r8Kyf7 sVIJkLF5yjvG6q5fwuy6L5NXnjjfC717hBXWMFq8XHFRc74YkXT6ncGijG0iBhbnaXfE 2tpdUyOuicvXOVLw/2UNOO/W4tFed+mVkZp0COX2lQ4USCB42mwwBF796iUPUU22A558 BbVFdbPhn/THXPKuotfXVT908MFv7+8nETIRqxK8sFXgFS1ADV1fgy8fNx1s0MsQDbOQ derA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=E8E+BevFTewNyLWW6S0RdjDgnnBxxeZMmDWlP6PlZvA=; b=lW3JE+Y0N/Fqg/8GfsJxPEz3TAHXyH2ZwjGrFUWDSLGzlSr1njLNQyVbrhd3MLsop6 owrzHXluJNXxfIBZs3yVlV00DmFIRRzCrfZGNF4aLS/syGZ4mQxxHCTLgqNGdeoaFU90 D9r1gpGBbF/EgP8Wmgau2ST3nnfzCRAPQW9Nssb88MgIzYOMI4XG9S1M6FxS1HBvF2o1 LusLMDZSfO9sbgils86tEkyfqmIMIPQZHN+YwHQmPCV6EjL2nLH8cbuUG2bNI5JRRfRw GJih/+Vt99M/xQdn1LaJtJ0vtR25iLuqLDrIrRODrzJwdfyrTG99cnUctjirj0iJ2AEv UIsw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Nrv9pMP6; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id x29-20020aa7957d000000b00624eb57b45dsi19277438pfq.74.2023.04.27.10.46.51; Thu, 27 Apr 2023 10:47:06 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Nrv9pMP6; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242835AbjD0Rgo (ORCPT + 99 others); Thu, 27 Apr 2023 13:36:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41082 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244437AbjD0Rg1 (ORCPT ); Thu, 27 Apr 2023 13:36:27 -0400 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 78E823A82; Thu, 27 Apr 2023 10:36:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1682616986; x=1714152986; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=UvHuLF1vNIYR0OWCHchmVeakk9WbW07/wvEcKt6JI3M=; b=Nrv9pMP6yyThauizXPZLNMh8cN+fq78e70v4b50fOSOV3hrcMJMLPOAL GzfhSLTOgfOMPxx8Rp174w0qzglV5NduGZaaZkfVFTNm7cYt98vs6vhzt uo7S2UoN2p4yCZCFGzM2l2uzZLZgfniYd92H6vPfTwxXhlWtqb+ySKNYf uhb0RLE9yhAveMbXnpZjAOAru8EnxxNF7hs2d0kvu7+2kbc2mP6bXRo3z rYO1861IgqMQ3MX0d35m+4gJYT5kRHvAmSCnOsKFSs+f6qb1HPosbFQlH hwW9XpHkdUYmPMgx0+X5XG+98UadK/ucD9t6n2PZX0fDTCjx2oBFIfErd g==; X-IronPort-AV: E=McAfee;i="6600,9927,10693"; a="349496922" X-IronPort-AV: E=Sophos;i="5.99,232,1677571200"; d="scan'208";a="349496922" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Apr 2023 10:36:22 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10693"; a="697172990" X-IronPort-AV: E=Sophos;i="5.99,232,1677571200"; d="scan'208";a="697172990" Received: from rchatre-ws.ostc.intel.com ([10.54.69.144]) by fmsmga007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Apr 2023 10:36:21 -0700 From: Reinette Chatre To: jgg@nvidia.com, yishaih@nvidia.com, shameerali.kolothum.thodi@huawei.com, kevin.tian@intel.com, alex.williamson@redhat.com Cc: tglx@linutronix.de, darwi@linutronix.de, kvm@vger.kernel.org, dave.jiang@intel.com, jing2.liu@intel.com, ashok.raj@intel.com, fenghua.yu@intel.com, tom.zanussi@linux.intel.com, reinette.chatre@intel.com, linux-kernel@vger.kernel.org Subject: [PATCH V4 08/11] vfio/pci: Use bitfield for struct vfio_pci_core_device flags Date: Thu, 27 Apr 2023 10:36:05 -0700 Message-Id: <42397f8cd0419694797c6c5cf65ed715117fc760.1682615447.git.reinette.chatre@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1764352460058639464?= X-GMAIL-MSGID: =?utf-8?q?1764352460058639464?= struct vfio_pci_core_device contains eleven boolean flags. Boolean flags clearly indicate their usage but space usage starts to be a concern when there are many. An upcoming change adds another boolean flag to struct vfio_pci_core_device, thereby increasing the concern that the boolean flags are consuming unnecessary space. Transition the boolean flags to use bitfields. On a system that uses one byte per boolean this reduces the space consumed by existing flags from 11 bytes to 2 bytes with room for a few more flags without increasing the structure's size. Suggested-by: Jason Gunthorpe Signed-off-by: Reinette Chatre Reviewed-by: Kevin Tian --- Changes since V3: - New patch. (Jason) include/linux/vfio_pci_core.h | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h index 148fd1ae6c1c..adb47e2914d7 100644 --- a/include/linux/vfio_pci_core.h +++ b/include/linux/vfio_pci_core.h @@ -68,17 +68,17 @@ struct vfio_pci_core_device { u16 msix_size; u32 msix_offset; u32 rbar[7]; - bool pci_2_3; - bool virq_disabled; - bool reset_works; - bool extended_caps; - bool bardirty; - bool has_vga; - bool needs_reset; - bool nointx; - bool needs_pm_restore; - bool pm_intx_masked; - bool pm_runtime_engaged; + bool pci_2_3:1; + bool virq_disabled:1; + bool reset_works:1; + bool extended_caps:1; + bool bardirty:1; + bool has_vga:1; + bool needs_reset:1; + bool nointx:1; + bool needs_pm_restore:1; + bool pm_intx_masked:1; + bool pm_runtime_engaged:1; struct pci_saved_state *pci_saved_state; struct pci_saved_state *pm_save; int ioeventfds_nr; From patchwork Thu Apr 27 17:36:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Reinette Chatre X-Patchwork-Id: 88327 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp429936vqo; Thu, 27 Apr 2023 10:38:56 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5LJSVTjbYrpYycjKfIYmeWl6MmjUT9SiHVelQtuIG6CzdUlT+qtx7ci4cq8n6cnRKLz5/B X-Received: by 2002:a17:902:ea09:b0:1a9:79e7:2ba with SMTP id s9-20020a170902ea0900b001a979e702bamr2858832plg.23.1682617136699; Thu, 27 Apr 2023 10:38:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682617136; cv=none; d=google.com; s=arc-20160816; b=kGKCjl+GQwTdIWFfdZI09qCxPZUHtkoCF7FD4HuU4tIoYOGACpHOf7xjUpMpae8aje mRev3apUnzK4P7Bw/muIM4kk3+cEgQGLgdlMxenhRqp++CX+askap9ZAvfpAWX1CVDX9 LPePMsxfuUq/vvDn3n7s1XBzpyZlfJqStF/NZSNSgz9958Cxwhju86PGk5tbbirOmtl5 T6dCe3DEM2nFdly8QPFHc4iFsYKxUOFa/lW1Md9TZphTvQsvQup7ZZM9DUsQwrYwxjHs 0JPiwQCvLMOXUiqurFovL4u47n3ufKV+e8r6sSAPH6d/Uz7QLUZKQkFwZ99mpN+sca97 ph+A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=66HvSEnfyOdqfJcM2KWKZlHzijCZ2XuTFP+G/vUoqIw=; b=mc6U7es26B/7ALYSCDkEd+s0zPyAh6qyWoT707nPN6js/c/giT3Pke3EakeXUl17mL 8RdhZDmFhqPPXXRm/AdjqluZJfhoQoiUc+/hLWOY6LqSISIZzmN0kHlHwitefSnUC2nC e2BSiZFMCYFZjQVTKKEtIXTCIlkpOpI+I4T4cIhiT1VlKB9BcdeQ9/s0/6Nq3ETVAetK W9yBLBZI3xFqdzzdlsPQhXkNxNck9MCn5dhm7U35q9NAuJ0rZeZngL9hcjXmWTY2BFr4 0MIBD0DZqHSXPiFbmXHc3M8oY2glo5dCARvcb5jsCvUWc5ToLbpdJwZJg88d+KPR+nC2 97FA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=SGlm86QL; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id d10-20020a170902ceca00b001a68986a3d6si18187301plg.408.2023.04.27.10.38.41; Thu, 27 Apr 2023 10:38:56 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=SGlm86QL; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244586AbjD0RhE (ORCPT + 99 others); Thu, 27 Apr 2023 13:37:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41096 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244438AbjD0Rg2 (ORCPT ); Thu, 27 Apr 2023 13:36:28 -0400 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4BA033584; Thu, 27 Apr 2023 10:36:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1682616987; x=1714152987; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=XwAG5SNcH6mS98HHUVIWrH5rxfGqTd4FpZ6wJI2ZqM0=; b=SGlm86QLZl96RbZukkNnEm2Z+qAXh0v/Pgo6/lYpG1d+qvJAvZfVKsVi h7+nXt6IxFWaz0h0sw90BrZrihcwDFOCSVm/jmFlV8A4hdVivMjG2dLn4 mvxT8aW5wbGzhYYfwJR2yyGX2qSKAQOlJeFeOM7smFat2dq1hGuRYo1xL fAUdeejxw81+4pes9YIT4/xIR3SkEWyuJ8kqKJXwBRfEidWhuZ1raPy+y gkP/tUX2V5NbMGZfGtAO1BBFYRpG+34MZqEV8NorLtFN75fw7ys/jYmbt zXmOE1d0Gzc6k8lCvrUmdqHB1uB0HDZv6ZvkDFs8IfTnhZ3kM/wuqShBW w==; X-IronPort-AV: E=McAfee;i="6600,9927,10693"; a="349496928" X-IronPort-AV: E=Sophos;i="5.99,232,1677571200"; d="scan'208";a="349496928" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Apr 2023 10:36:22 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10693"; a="697172994" X-IronPort-AV: E=Sophos;i="5.99,232,1677571200"; d="scan'208";a="697172994" Received: from rchatre-ws.ostc.intel.com ([10.54.69.144]) by fmsmga007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Apr 2023 10:36:21 -0700 From: Reinette Chatre To: jgg@nvidia.com, yishaih@nvidia.com, shameerali.kolothum.thodi@huawei.com, kevin.tian@intel.com, alex.williamson@redhat.com Cc: tglx@linutronix.de, darwi@linutronix.de, kvm@vger.kernel.org, dave.jiang@intel.com, jing2.liu@intel.com, ashok.raj@intel.com, fenghua.yu@intel.com, tom.zanussi@linux.intel.com, reinette.chatre@intel.com, linux-kernel@vger.kernel.org Subject: [PATCH V4 09/11] vfio/pci: Probe and store ability to support dynamic MSI-X Date: Thu, 27 Apr 2023 10:36:06 -0700 Message-Id: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1764351946725597181?= X-GMAIL-MSGID: =?utf-8?q?1764351946725597181?= Not all MSI-X devices support dynamic MSI-X allocation. Whether a device supports dynamic MSI-X should be queried using pci_msix_can_alloc_dyn(). Instead of scattering code with pci_msix_can_alloc_dyn(), probe this ability once and store it as a property of the virtual device. Suggested-by: Alex Williamson Signed-off-by: Reinette Chatre Reviewed-by: Kevin Tian --- Changes since V3: - Move field to improve structure layout. (Alex) - Use bitfield. (Jason) Changes since V2: - New patch. (Alex) drivers/vfio/pci/vfio_pci_core.c | 5 ++++- include/linux/vfio_pci_core.h | 1 + 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c index ae0e161c7fc9..a3635a8e54c8 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -530,8 +530,11 @@ int vfio_pci_core_enable(struct vfio_pci_core_device *vdev) vdev->msix_bar = table & PCI_MSIX_TABLE_BIR; vdev->msix_offset = table & PCI_MSIX_TABLE_OFFSET; vdev->msix_size = ((flags & PCI_MSIX_FLAGS_QSIZE) + 1) * 16; - } else + vdev->has_dyn_msix = pci_msix_can_alloc_dyn(pdev); + } else { vdev->msix_bar = 0xFF; + vdev->has_dyn_msix = false; + } if (!vfio_vga_disabled() && vfio_pci_is_vga(pdev)) vdev->has_vga = true; diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h index adb47e2914d7..562e8754869d 100644 --- a/include/linux/vfio_pci_core.h +++ b/include/linux/vfio_pci_core.h @@ -68,6 +68,7 @@ struct vfio_pci_core_device { u16 msix_size; u32 msix_offset; u32 rbar[7]; + bool has_dyn_msix:1; bool pci_2_3:1; bool virq_disabled:1; bool reset_works:1; From patchwork Thu Apr 27 17:36:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Reinette Chatre X-Patchwork-Id: 88326 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp429866vqo; Thu, 27 Apr 2023 10:38:46 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ58JcLVPuiSFySdkZJHYnBwVRBZLf7e+cEFeii4EmfFFUFyPG8Zt9rmDgYD/g1hOBCRrN5X X-Received: by 2002:a05:6a20:6a07:b0:ef:cb4c:c23e with SMTP id p7-20020a056a206a0700b000efcb4cc23emr3035553pzk.29.1682617125904; Thu, 27 Apr 2023 10:38:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682617125; cv=none; d=google.com; s=arc-20160816; b=wga76dugRf0hx+YgXYg0VNlpVgC/rsxfYU4fszwAvIZDPkaKQr2R7JSFbEuuTFCQIW DFWv85ma7XIkCB/smwv2nZqLCzKdViCE9XA6KFAXb+grGU5Nqg6deNfXJCkKCqL+Xe5E HC4DSkHA4Wa8A9fCVlwiVpuhs0utBLLOrf/qiFYkDVYclehUoTriX7o9pkDJ9ND7oW4E 4aYCqxEpD+OKzcEFZRuOyq/MRUWIfuV5KPpgtmGeUy6vtQZBA//DO4Y+BGgiBDeMlJQ+ Qc4PKvc5vdaz0khy11gpbdGVJbAEa/QvTe6oBRWQ9ET3/AOir+OodraBG2aoMpwynCnH 5vDA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Wc4cy6/FnULjJBm50CezazWXPL+MCJQ518e9PPqOypo=; b=XyiL5WTDXiZtxyMIE6NeUO6MzHWp/FYL7Gp4HRuMoKJBHxcX0yz1go2COhmnmNiVR9 AHgYf/SdFhOLD7PjVkSw7w8MtT2kiXIXAVT02cE6INyMebHdtT+RAGa49egMlKaozy9o Jr3/rFcAt7OCZmNNdIwiUIECVzE4t0c99Dx6QdfYZbJMnL44cA1Vkt0EpGfYZa2AyW+w S9iZRx0pJ77dUnwqYRboGFNxmXc6T0km5ycmne/I84MMBJOcbMFOvskj8Lu08kd5emTT FAjanePbTZ444DFxJY/dhD9PVe8ElNSPdv8WK+VqiMymtt80LV7GtjpdRhZJ4P7M+Jcs VUDA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=c43hC4sS; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id gt18-20020a17090af2d200b002473f9e8a19si21568356pjb.14.2023.04.27.10.38.32; Thu, 27 Apr 2023 10:38:45 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=c43hC4sS; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244572AbjD0RhB (ORCPT + 99 others); Thu, 27 Apr 2023 13:37:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41108 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244443AbjD0Rg2 (ORCPT ); Thu, 27 Apr 2023 13:36:28 -0400 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 751393A91; Thu, 27 Apr 2023 10:36:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1682616987; x=1714152987; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Sngpy7QSRRwSxWcsLL2pb0sag7VSeHhyYvrzzFIvniE=; b=c43hC4sSIXLnF+iIj3AQkVtVJVcdXEn5QXXgisGepKmg9OIz/S+oqKwz /KxR0CtVLT2TudIouddtoLsyi++eMJlB6ug0y1uv7FKSGxw4z2V/zmXd9 6XcB5lFR6EBkNvSGD+xr39DYBpu5ypQUA0NeavqY6ay0620S2bzrQn7Cm q7BDun374/geJXs5bOTZsv/63fEJiWg7sRf3wlm0dCLpRka952ycdPF8m welnTj+wPEbBaxHRZtk+lbz8+fXoCNq+Y7OVSYphHAkBLWpopCA33MC7P UrJeP6XQDxzt6DktFoBzdpOQmbMWgMAbZiup5YFJfScSPLD9wiZg1YuUi w==; X-IronPort-AV: E=McAfee;i="6600,9927,10693"; a="349496937" X-IronPort-AV: E=Sophos;i="5.99,232,1677571200"; d="scan'208";a="349496937" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Apr 2023 10:36:22 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10693"; a="697172998" X-IronPort-AV: E=Sophos;i="5.99,232,1677571200"; d="scan'208";a="697172998" Received: from rchatre-ws.ostc.intel.com ([10.54.69.144]) by fmsmga007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Apr 2023 10:36:22 -0700 From: Reinette Chatre To: jgg@nvidia.com, yishaih@nvidia.com, shameerali.kolothum.thodi@huawei.com, kevin.tian@intel.com, alex.williamson@redhat.com Cc: tglx@linutronix.de, darwi@linutronix.de, kvm@vger.kernel.org, dave.jiang@intel.com, jing2.liu@intel.com, ashok.raj@intel.com, fenghua.yu@intel.com, tom.zanussi@linux.intel.com, reinette.chatre@intel.com, linux-kernel@vger.kernel.org Subject: [PATCH V4 10/11] vfio/pci: Support dynamic MSI-X Date: Thu, 27 Apr 2023 10:36:07 -0700 Message-Id: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1764351934928351479?= X-GMAIL-MSGID: =?utf-8?q?1764351934928351479?= pci_msix_alloc_irq_at() enables an individual MSI-X interrupt to be allocated after MSI-X enabling. Use dynamic MSI-X (if supported by the device) to allocate an interrupt after MSI-X is enabled. An MSI-X interrupt is dynamically allocated at the time a valid eventfd is assigned. This is different behavior from a range provided during MSI-X enabling where interrupts are allocated for the entire range whether a valid eventfd is provided for each interrupt or not. The PCI-MSIX API requires that some number of irqs are allocated for an initial set of vectors when enabling MSI-X on the device. When dynamic MSIX allocation is not supported, the vector table, and thus the allocated irq set can only be resized by disabling and re-enabling MSI-X with a different range. In that case the irq allocation is essentially a cache for configuring vectors within the previously allocated vector range. When dynamic MSI-X allocation is supported, the API still requires some initial set of irqs to be allocated, but also supports allocating and freeing specific irq vectors both within and beyond the initially allocated range. For consistency between modes, as well as to reduce latency and improve reliability of allocations, and also simplicity, this implementation only releases irqs via pci_free_irq_vectors() when either the interrupt mode changes or the device is released. Signed-off-by: Reinette Chatre Link: https://lore.kernel.org/lkml/20230403211841.0e206b67.alex.williamson@redhat.com/ --- Changes since V3: - Remove vfio_msi_free_irq(). (Alex) - Rework changelog. (Alex) Changes since V2: - Move vfio_irq_ctx_free() to earlier in series to support earlier usage. (Alex) - Use consistent terms in changelog: MSI-x changed to MSI-X. - Make dynamic interrupt context creation generic across all MSI/MSI-X interrupts. This resulted in code moving to earlier in series as part of xarray introduction patch. (Alex) - Remove the local allow_dyn_alloc and direct calling of pci_msix_can_alloc_dyn(), use the new vdev->has_dyn_msix introduced earlier instead. (Alex) - Stop tracking new allocations (remove "new_ctx"). (Alex) - Introduce new wrapper that returns Linux interrupt number or dynamically allocate a new interrupt. Wrapper can be used for all interrupt cases. (Alex) - Only free dynamic MSI-X interrupts on MSI-X teardown. (Alex) Changes since RFC V1: - Add pointer to interrupt context as function parameter to vfio_irq_ctx_free(). (Alex) - Initialize new_ctx to false. (Dan Carpenter) - Only support dynamic allocation if device supports it. (Alex) drivers/vfio/pci/vfio_pci_intrs.c | 47 +++++++++++++++++++++++++++---- 1 file changed, 41 insertions(+), 6 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_intrs.c b/drivers/vfio/pci/vfio_pci_intrs.c index bdda7f46c2be..8340135b09fa 100644 --- a/drivers/vfio/pci/vfio_pci_intrs.c +++ b/drivers/vfio/pci/vfio_pci_intrs.c @@ -372,27 +372,56 @@ static int vfio_msi_enable(struct vfio_pci_core_device *vdev, int nvec, bool msi return 0; } +/* + * Return Linux IRQ number of an MSI or MSI-X device interrupt vector. + * If a Linux IRQ number is not available then a new interrupt will be + * allocated if dynamic MSI-X is supported. + */ +static int vfio_msi_alloc_irq(struct vfio_pci_core_device *vdev, + unsigned int vector, bool msix) +{ + struct pci_dev *pdev = vdev->pdev; + struct msi_map map; + int irq; + u16 cmd; + + irq = pci_irq_vector(pdev, vector); + if (irq > 0 || !msix || !vdev->has_dyn_msix) + return irq; + + cmd = vfio_pci_memory_lock_and_enable(vdev); + map = pci_msix_alloc_irq_at(pdev, vector, NULL); + vfio_pci_memory_unlock_and_restore(vdev, cmd); + + return map.index < 0 ? map.index : map.virq; +} + +/* + * Where is vfio_msi_free_irq() ? + * + * Allocated interrupts are maintained, essentially forming a cache that + * subsequent allocations can draw from. Interrupts are freed using + * pci_free_irq_vectors() when MSI/MSI-X is disabled. + */ + static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, unsigned int vector, int fd, bool msix) { struct pci_dev *pdev = vdev->pdev; struct vfio_pci_irq_ctx *ctx; struct eventfd_ctx *trigger; - int irq, ret; + int irq = -EINVAL, ret; u16 cmd; - irq = pci_irq_vector(pdev, vector); - if (irq < 0) - return -EINVAL; - ctx = vfio_irq_ctx_get(vdev, vector); if (ctx) { irq_bypass_unregister_producer(&ctx->producer); - + irq = pci_irq_vector(pdev, vector); cmd = vfio_pci_memory_lock_and_enable(vdev); free_irq(irq, ctx->trigger); vfio_pci_memory_unlock_and_restore(vdev, cmd); + /* Interrupt stays allocated, will be freed at MSI-X disable. */ kfree(ctx->name); eventfd_ctx_put(ctx->trigger); vfio_irq_ctx_free(vdev, ctx, vector); @@ -401,6 +430,12 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, if (fd < 0) return 0; + if (irq == -EINVAL) { + irq = vfio_msi_alloc_irq(vdev, vector, msix); + if (irq < 0) + return irq; + } + ctx = vfio_irq_ctx_alloc(vdev, vector); if (!ctx) return -ENOMEM; From patchwork Thu Apr 27 17:36:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Reinette Chatre X-Patchwork-Id: 88324 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp429504vqo; Thu, 27 Apr 2023 10:37:53 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7FPFaLzoDgdiY9SxyjaE6xC+R12/bJi+gjtVXBZBUw0glCcpaHPSRtLvH9Sv8Z3gHZN0eY X-Received: by 2002:a05:6a20:729e:b0:f2:ae03:4020 with SMTP id o30-20020a056a20729e00b000f2ae034020mr2919681pzk.54.1682617073009; Thu, 27 Apr 2023 10:37:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682617072; cv=none; d=google.com; s=arc-20160816; b=tM48bMyxNB8qq8qhG/sp8E/4oGoIvQArSCjPkMMc5CnaZWoeHxat7lgRHNIETuUkhb uCGRJw6pMjBkdHrlXHIrsE5azTvJYqop9c4yQRzTuDk1WmmozEQtYCBoTXCw+WZMSOaL phMswhhKqQc6BPXlacYmzaG7t69g4F90mLMo1su9SZuEbQUB1p2O00StmgFSUtuDqSzW h7i8bk+Zdygq/fuwR0F8awvu3i6bL7tio6NNQYeS3Slass6GMn3O7SpSf9oiXWygE1n4 hhUWRGHfOmfFVsPS+fTv7kpEsJbmst1Vqj+AombOIizBbDMDGHJRkDRDz9H6JZgZVe0r +4Lg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=VTuZgaNOEY1r2jc12mMufZEVC9zSOXBc8YPaYezEBe4=; b=WVnfzMSQOjrFSoMXg0cgiLpnmptxhsh2bQ9Kn+0Z1EuHb1OGZY5Om7CGqT3Kdc6jew TSnrdti/cALRCO1EImuB59wbbsckZGqcBWt/mzsOyKM6vNqc7JyaJhwAT1Cs8AAfpO0f +AzcpOhJ5mGMI9incXaSACKXUNXTH5AkG7t0bmFOu563OdwpdXzBHKjC4EJ9DcXy3ERI rhH9/cpPX3eyeFm2fIBl6E2dSBcs++uWxb4ejPrQD/m9uJWHAMex//1ohXy1VEjD1F6o +VliyEOIY+7Vm95V37CkZtPRzwfW/ndGsQ52KNZedNxfMTrNFPAJZTdi6C8L/bfUCiuv Y9zQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=JD3Ff0xx; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id k27-20020a63561b000000b0051b52bacc82si18448518pgb.243.2023.04.27.10.37.40; Thu, 27 Apr 2023 10:37:52 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=JD3Ff0xx; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244399AbjD0Rgl (ORCPT + 99 others); Thu, 27 Apr 2023 13:36:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41102 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244439AbjD0Rg2 (ORCPT ); Thu, 27 Apr 2023 13:36:28 -0400 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 773773AB1; Thu, 27 Apr 2023 10:36:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1682616987; x=1714152987; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=uWriv/HW8rBdAww/p0pd10UnWNQnafdmDWtqvgZfv90=; b=JD3Ff0xxIRtx/3+es2oE8szYBgehF9eGe44xPaAquzkrCxr2Rzc6UZb9 IFNV9RfWgIUvHVA0SXYjJiqCfxPMv5NeFhKCkm4mNVvzrs8k0a55EAE59 Iq80C4mCMOna2CokpWJ5r3NVvPs39NFQ3NyZ3qWI8MEda7Uma1wZpoN9x e+nXQGlUnlf3q6jM1qlImUNIQv8Z/bvDpGeUq4H0E2qC+Fz1RCw2fLTYj 4gFsY3/pBsOZZ/nnHZ+WsNK0ZG4OX4vq/PHkoZJR4iM+qKkHj0GVv4SnL usoXJy7z6AsoE95pJD0hT4aE1396Yyoh+cT/MD9Weoi0aRRT8ZuUloen2 g==; X-IronPort-AV: E=McAfee;i="6600,9927,10693"; a="349496947" X-IronPort-AV: E=Sophos;i="5.99,232,1677571200"; d="scan'208";a="349496947" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Apr 2023 10:36:22 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10693"; a="697173004" X-IronPort-AV: E=Sophos;i="5.99,232,1677571200"; d="scan'208";a="697173004" Received: from rchatre-ws.ostc.intel.com ([10.54.69.144]) by fmsmga007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Apr 2023 10:36:22 -0700 From: Reinette Chatre To: jgg@nvidia.com, yishaih@nvidia.com, shameerali.kolothum.thodi@huawei.com, kevin.tian@intel.com, alex.williamson@redhat.com Cc: tglx@linutronix.de, darwi@linutronix.de, kvm@vger.kernel.org, dave.jiang@intel.com, jing2.liu@intel.com, ashok.raj@intel.com, fenghua.yu@intel.com, tom.zanussi@linux.intel.com, reinette.chatre@intel.com, linux-kernel@vger.kernel.org Subject: [PATCH V4 11/11] vfio/pci: Clear VFIO_IRQ_INFO_NORESIZE for MSI-X Date: Thu, 27 Apr 2023 10:36:08 -0700 Message-Id: <2370aaabd9c006747233df6678eed1b51ccca426.1682615447.git.reinette.chatre@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1764351879728662478?= X-GMAIL-MSGID: =?utf-8?q?1764351879728662478?= Dynamic MSI-X is supported. Clear VFIO_IRQ_INFO_NORESIZE to provide guidance to user space. Signed-off-by: Reinette Chatre Reviewed-by: Kevin Tian --- Changes since V3: - Remove unnecessary test from condition. (Alex) Changes since V2: - Use new vdev->has_dyn_msix property instead of calling pci_msix_can_alloc_dyn() directly. (Alex) Changes since RFC V1: - Only advertise VFIO_IRQ_INFO_NORESIZE for MSI-X devices that can actually support dynamic allocation. (Alex) drivers/vfio/pci/vfio_pci_core.c | 2 +- include/uapi/linux/vfio.h | 3 +++ 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c index a3635a8e54c8..ec7e662de033 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -1114,7 +1114,7 @@ static int vfio_pci_ioctl_get_irq_info(struct vfio_pci_core_device *vdev, if (info.index == VFIO_PCI_INTX_IRQ_INDEX) info.flags |= (VFIO_IRQ_INFO_MASKABLE | VFIO_IRQ_INFO_AUTOMASKED); - else + else if (info.index != VFIO_PCI_MSIX_IRQ_INDEX || !vdev->has_dyn_msix) info.flags |= VFIO_IRQ_INFO_NORESIZE; return copy_to_user(arg, &info, minsz) ? -EFAULT : 0; diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h index 0552e8dcf0cb..1a36134cae5c 100644 --- a/include/uapi/linux/vfio.h +++ b/include/uapi/linux/vfio.h @@ -511,6 +511,9 @@ struct vfio_region_info_cap_nvlink2_lnkspd { * then add and unmask vectors, it's up to userspace to make the decision * whether to allocate the maximum supported number of vectors or tear * down setup and incrementally increase the vectors as each is enabled. + * Absence of the NORESIZE flag indicates that vectors can be enabled + * and disabled dynamically without impacting other vectors within the + * index. */ struct vfio_irq_info { __u32 argsz;