From patchwork Mon Oct 16 13:13:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= X-Patchwork-Id: 153390 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp3451626vqb; Mon, 16 Oct 2023 06:14:40 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFPxInIXz9+eDTDzo1xGzKaQbfklVmXHtoDUOfuDlk1gUFuJWqumziE2MI6TlSLxgsX/ak6 X-Received: by 2002:a17:902:f686:b0:1c7:5581:f9c with SMTP id l6-20020a170902f68600b001c755810f9cmr37545135plg.0.1697462079918; Mon, 16 Oct 2023 06:14:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697462079; cv=none; d=google.com; s=arc-20160816; b=g320r7Qydy+BhCH45G2+RIEFUzTfpW5P5LsU1M3PQQ+FBSP0ZXWCTW4kM7XQrFtAqA 5Pvcl5tPd2KKfbuDeSyK3NYaUp63bzSkq4Qiid7t7IAYUY5VL0OXRNHcfYHB956Q0H0J 72eznrIdJyqCqLK2iywBLvixotOn+3Erye/yLGlaLFoEB4s50ci/KM7ug4wl/Tie5qjh ZvNyAGlTP/P5StzkxjA2X4O17yR7AMAxt80JwvPaguhkPotbAe0l7gXkTz/vjxsuVt2I rMjTCuYaq4NkridD7fWitvn7o/b4UBN26V8t1KcesAmXlHS2vfIsQzzUQfhV9fDszAaX 8S4w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:feedback-id:dkim-signature :dkim-signature; bh=4l799Etjsd61etoKzKWXHhkW7EpAA6Zrh8WZJFoVuBk=; fh=3BkUVCq5M7mV+4pc2fqV123bC6dmJxXywOJporACPjE=; b=g0mqplVtjAeLxXLQZ2J+hJ/XDVqRvSEUikUfgJFZ9bZzrQ9nTT2yapCYtuObqTgyLb cWosvkeJdELQuE8xQyG6ymGCmB3ZGI/TePJ61u9EY0VM5rBdM1bM4kE5kCsZkL8TyML/ 6nJ4XpJ/7zMjKsz50IGIgdZLh+yCZsaw7LDV8EUPYf7nTCXipjKHEbOlqOpGIWfpAExa CwdRd2TDXH3ac4Yy3ZyXhuHpa77iUvfiAlIc3gewdz8L9iu0Q7LvCcjyd8mvMMXmmMJR ucYj1VhJJbbmXUcRdZl+2Ka5d3XkfKx6SZtLr5QTMk4zuldv6ZndeSOabgpiY7uEeiJS d6tw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@invisiblethingslab.com header.s=fm2 header.b=Q4ad2P67; dkim=pass header.i=@messagingengine.com header.s=fm3 header.b=ZkdZbrGO; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from groat.vger.email (groat.vger.email. [23.128.96.35]) by mx.google.com with ESMTPS id k10-20020a170902c40a00b001c9fb2acff5si8166976plk.526.2023.10.16.06.14.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 16 Oct 2023 06:14:39 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) client-ip=23.128.96.35; Authentication-Results: mx.google.com; dkim=pass header.i=@invisiblethingslab.com header.s=fm2 header.b=Q4ad2P67; dkim=pass header.i=@messagingengine.com header.s=fm3 header.b=ZkdZbrGO; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by groat.vger.email (Postfix) with ESMTP id 676F08084959; Mon, 16 Oct 2023 06:14:35 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at groat.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233095AbjJPNOJ (ORCPT + 18 others); Mon, 16 Oct 2023 09:14:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41926 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229501AbjJPNOE (ORCPT ); Mon, 16 Oct 2023 09:14:04 -0400 Received: from out4-smtp.messagingengine.com (out4-smtp.messagingengine.com [66.111.4.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 597509B for ; Mon, 16 Oct 2023 06:14:02 -0700 (PDT) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.nyi.internal (Postfix) with ESMTP id 96AE65C039B; Mon, 16 Oct 2023 09:14:01 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Mon, 16 Oct 2023 09:14:01 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= invisiblethingslab.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:from:from:in-reply-to :message-id:mime-version:reply-to:sender:subject:subject:to:to; s=fm2; t=1697462041; x=1697548441; bh=4l799Etjsd61etoKzKWXHhkW7 EpAA6Zrh8WZJFoVuBk=; b=Q4ad2P67UxYuPBTo+McVbi1NM1+QQZpwwIceSxqRR HtB2bTO7YjQQYTJiHVC7pQZNxbG9WS/HBiXpyU2E7+eyS4+y+aCjMJ97Uz+0zQHP LasIi4tteOstoIvBDNqWcLNaZ5zk8DeoKcW9vgESSbLmwJJBGutYv+zACr9vmqvy urYPWe0OhjdByauDyJUjgqONAziO3rLhNd9bZWaj4AEBmTesN0XITspPxU2tLW33 DV4e4IG1NPb8bu5oRpUHz6zaxl+ZdbvgljoysS4gj/yDQs9NfUHWsnmJM/lCByDr E2gSWMP3ldSoCB9QRBicaJad1KL5mECSA9RW1/6uGCoig== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:message-id:mime-version:reply-to:sender :subject:subject:to:to:x-me-proxy:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm3; t=1697462041; x=1697548441; bh=4 l799Etjsd61etoKzKWXHhkW7EpAA6Zrh8WZJFoVuBk=; b=ZkdZbrGOYWZUq6T5u GHuv6ICTTiXmHuNqH7nRx9uW0XPkeKVN6JQfDRTjV5MANp+xMF0kycL5hEekwCTE PUU43Pm/f9Y8e+i57OKNcyRWfLxyQSWpoGhYGswME3y3D5bWwPHDmuUbPlyKlTNz PehThgfGxNiqxVaS6dMDsUQFvm1f3Gexf91UWb/ZReXz42T2gJ5DpeEzqpkSNdbs zvjuPgMaUANMVo6SU68AAxYukshJ9va2zsDaGscCr+FlwVMRFczgFswtF+kgHWj/ 4aNABTRgcMKh5RwQxxCY5N19m5dHMCllZVi7KvWdKj24ax4ZCz5lxFxRHm8rZphT KgYvA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrjedtgdehlecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvfevufffkffogggtgfesthekredtredtjeenucfhrhhomhepofgrrhgvkhcu ofgrrhgtiiihkhhofihskhhiqdfikphrvggtkhhiuceomhgrrhhmrghrvghksehinhhvih hsihgslhgvthhhihhnghhslhgrsgdrtghomheqnecuggftrfgrthhtvghrnhepleekhfdu leetleelleetteevfeefteffkeetteejheelgfegkeelgeehhfdthedvnecuvehluhhsth gvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepmhgrrhhmrghrvghksehi nhhvihhsihgslhgvthhhihhnghhslhgrsgdrtghomh X-ME-Proxy: Feedback-ID: i1568416f:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 16 Oct 2023 09:14:00 -0400 (EDT) From: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= To: linux-kernel@vger.kernel.org Cc: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Jan Beulich , xen-devel@lists.xenproject.org (moderated list:XEN HYPERVISOR INTERFACE) Subject: [PATCH v4] xen-pciback: Consider INTx disabled when MSI/MSI-X is enabled Date: Mon, 16 Oct 2023 15:13:25 +0200 Message-ID: <20231016131348.1734721-1-marmarek@invisiblethingslab.com> X-Mailer: git-send-email 2.41.0 MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Mon, 16 Oct 2023 06:14:35 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1749850511751236812 X-GMAIL-MSGID: 1779917997934033260 Linux enables MSI-X before disabling INTx, but keeps MSI-X masked until the table is filled. Then it disables INTx just before clearing MASKALL bit. Currently this approach is rejected by xen-pciback. According to the PCIe spec, device cannot use INTx when MSI/MSI-X is enabled (in other words: enabling MSI/MSI-X implicitly disables INTx). Change the logic to consider INTx disabled if MSI/MSI-X is enabled. This applies to three places: - checking currently enabled interrupts type, - transition to MSI/MSI-X - where INTx would be implicitly disabled, - clearing INTx disable bit - which can be allowed even if MSI/MSI-X is enabled, as device should consider INTx disabled anyway in that case Fixes: 5e29500eba2a ("xen-pciback: Allow setting PCI_MSIX_FLAGS_MASKALL too") Signed-off-by: Marek Marczykowski-Górecki Acked-by: Juergen Gross --- Changes in v4: - update comment (retain A-b) Changes in v3: - allow clearing INTx regardless of MSI/MSI-X state, to be consistent with enabling MSI/MSI-X Changes in v2: - restructure the patch to consider not only MASKALL bit, but enabling MSI/MSI-X generally, without explicitly disabling INTx first --- drivers/xen/xen-pciback/conf_space.c | 19 +++++++++++------ .../xen/xen-pciback/conf_space_capability.c | 8 ++++++- drivers/xen/xen-pciback/conf_space_header.c | 21 +++---------------- 3 files changed, 23 insertions(+), 25 deletions(-) diff --git a/drivers/xen/xen-pciback/conf_space.c b/drivers/xen/xen-pciback/conf_space.c index 059de92aea7d..d47eee6c5143 100644 --- a/drivers/xen/xen-pciback/conf_space.c +++ b/drivers/xen/xen-pciback/conf_space.c @@ -288,12 +288,6 @@ int xen_pcibk_get_interrupt_type(struct pci_dev *dev) u16 val; int ret = 0; - err = pci_read_config_word(dev, PCI_COMMAND, &val); - if (err) - return err; - if (!(val & PCI_COMMAND_INTX_DISABLE)) - ret |= INTERRUPT_TYPE_INTX; - /* * Do not trust dev->msi(x)_enabled here, as enabling could be done * bypassing the pci_*msi* functions, by the qemu. @@ -316,6 +310,19 @@ int xen_pcibk_get_interrupt_type(struct pci_dev *dev) if (val & PCI_MSIX_FLAGS_ENABLE) ret |= INTERRUPT_TYPE_MSIX; } + + /* + * PCIe spec says device cannot use INTx if MSI/MSI-X is enabled, + * so check for INTx only when both are disabled. + */ + if (!ret) { + err = pci_read_config_word(dev, PCI_COMMAND, &val); + if (err) + return err; + if (!(val & PCI_COMMAND_INTX_DISABLE)) + ret |= INTERRUPT_TYPE_INTX; + } + return ret ?: INTERRUPT_TYPE_NONE; } diff --git a/drivers/xen/xen-pciback/conf_space_capability.c b/drivers/xen/xen-pciback/conf_space_capability.c index 097316a74126..1948a9700c8f 100644 --- a/drivers/xen/xen-pciback/conf_space_capability.c +++ b/drivers/xen/xen-pciback/conf_space_capability.c @@ -236,10 +236,16 @@ static int msi_msix_flags_write(struct pci_dev *dev, int offset, u16 new_value, return PCIBIOS_SET_FAILED; if (new_value & field_config->enable_bit) { - /* don't allow enabling together with other interrupt types */ + /* + * Don't allow enabling together with other interrupt type, but do + * allow enabling MSI(-X) while INTx is still active to please Linuxes + * MSI(-X) startup sequence. It is safe to do, as according to PCI + * spec, device with enabled MSI(-X) shouldn't use INTx. + */ int int_type = xen_pcibk_get_interrupt_type(dev); if (int_type == INTERRUPT_TYPE_NONE || + int_type == INTERRUPT_TYPE_INTX || int_type == field_config->int_type) goto write; return PCIBIOS_SET_FAILED; diff --git a/drivers/xen/xen-pciback/conf_space_header.c b/drivers/xen/xen-pciback/conf_space_header.c index 981435103af1..fc0332645966 100644 --- a/drivers/xen/xen-pciback/conf_space_header.c +++ b/drivers/xen/xen-pciback/conf_space_header.c @@ -104,24 +104,9 @@ static int command_write(struct pci_dev *dev, int offset, u16 value, void *data) pci_clear_mwi(dev); } - if (dev_data && dev_data->allow_interrupt_control) { - if ((cmd->val ^ value) & PCI_COMMAND_INTX_DISABLE) { - if (value & PCI_COMMAND_INTX_DISABLE) { - pci_intx(dev, 0); - } else { - /* Do not allow enabling INTx together with MSI or MSI-X. */ - switch (xen_pcibk_get_interrupt_type(dev)) { - case INTERRUPT_TYPE_NONE: - pci_intx(dev, 1); - break; - case INTERRUPT_TYPE_INTX: - break; - default: - return PCIBIOS_SET_FAILED; - } - } - } - } + if (dev_data && dev_data->allow_interrupt_control && + ((cmd->val ^ value) & PCI_COMMAND_INTX_DISABLE)) + pci_intx(dev, !(value & PCI_COMMAND_INTX_DISABLE)); cmd->val = value;