From patchwork Thu Dec 22 07:26:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xinghui Li X-Patchwork-Id: 35635 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:e747:0:0:0:0:0 with SMTP id c7csp184979wrn; Wed, 21 Dec 2022 23:32:01 -0800 (PST) X-Google-Smtp-Source: AMrXdXvWmUcQXZJBdSAMbyQMpc9eWB4rRmIRNZwjjCRXdWVoiRYx3M0s80vGnTCaDg9eZEKHhbSa X-Received: by 2002:a17:90a:744b:b0:223:2aa8:aad3 with SMTP id o11-20020a17090a744b00b002232aa8aad3mr5361392pjk.0.1671694320868; Wed, 21 Dec 2022 23:32:00 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1671694320; cv=none; d=google.com; s=arc-20160816; b=vSLAh9D9tL6JoGfuIg4CzF/aOdNWX+BAEQeMf7hd1J/43tesP2uPLo6E9ByhTnb/NH Zko5FWMXDeDucvqlkKc4Lx9JiNKnjNAXlCv7xuSkPo4/yd2ycHgElZXOWCG3IO533WHC LxwoaVAAK8j7n8RX5k26ug6nhy/MISPK3v9Zdk/w2MXUdl4MOwOya2jdUwnipMVk1XSC JnwqdK7xLx+tHdOXY+R2dX2kDiaVyeZF/RkQ9nqL4OpTDZpHHaccLsu0jnYL6dAJjyXb r7vRZMXFKuTquZgwVWWZ65Qz9zYMg6jlGI5XMVQZHrWx3xXzCdfmMkfx/k75S0Q2mr4Y Fzqw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=C1M6sI8nGjn1AQZW7MjYI4O6nfdLnBf5J3+wPQ6ghb0=; b=yv+lZ9Kh7W7kYdT4NmPFOa/o5Km3zKKcduRgDZxdHkrFsWEMuTCHI+y6dt1mHgd+jt BD5Hj8YyToQEsWNidE55f1QycF6Vdu3nzATgft0LpUYIfYFnDHM6Tw2ukDfn9w2eCBFp L6I3I2MOB+bhuW8ihcyhNT3x+hqKaFfWWblD99A38NVmBHwvKmo3AimLLMted82twQgJ St3Rmbm6DH7gcqLg2SSLj7/ZSqPXeWr/lvwRlMFWEtpz7tZq7Jz8hLP9tubUgoQb1MOf 6FZ0aUTg0I52PnoDFA0zXONY9HFhKvnzsRalaToQYpHrkWfV5c8bHpwMN8lxvCLldLnG JhEw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=KZRpJQE2; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id p30-20020a63951e000000b00476f3f74928si129228pgd.59.2022.12.21.23.31.46; Wed, 21 Dec 2022 23:32:00 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=KZRpJQE2; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229862AbiLVH0N (ORCPT + 99 others); Thu, 22 Dec 2022 02:26:13 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41240 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229608AbiLVH0I (ORCPT ); Thu, 22 Dec 2022 02:26:08 -0500 Received: from mail-pf1-x442.google.com (mail-pf1-x442.google.com [IPv6:2607:f8b0:4864:20::442]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 96A8538B4; Wed, 21 Dec 2022 23:26:07 -0800 (PST) Received: by mail-pf1-x442.google.com with SMTP id x66so650596pfx.3; Wed, 21 Dec 2022 23:26:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=C1M6sI8nGjn1AQZW7MjYI4O6nfdLnBf5J3+wPQ6ghb0=; b=KZRpJQE2dMcoZOUADI+9CK3UIfA+yeOJRhVV06WaoBzBZFtg5CdZP5hSE68WLOOdXm rG8tFq14LpuMpsAzqBqXWMdbGZYMyGIQka9dtDTu9Z0iNl1g7AiYDphbg44aUydjiiaS TuK6DrJjGI1cOwkQxLoSaKogyEP3yeyP+rasQspiBvXFLV8sNUIRPw7oj6UtTpLhfUV6 7xz/qHNDbXb8f2Zqsoz+dIcgYoWolASTcAcCD70/oNGbes+I0+4ZkgcXp+k9QUtqaAUC NyLYQmrC52cU6+XUGtmFw5AxyCxtKkWHiAUMIQXHDjVZZb2Z4YPqvLKlXoylD3mbxDCh OgJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=C1M6sI8nGjn1AQZW7MjYI4O6nfdLnBf5J3+wPQ6ghb0=; b=13v8N6MhXyuD7XNvmql05xK8xp49um2Fe+wFzsCpFkHwRl2WTl7sbCCeM1IvCZmMBb Ki09o5wldhZi/tmEsg2o17jRNpdQYskdJAcIMaX+KQTOnOi9I83NUKmC02+kcFJbrJRn 9rQy1ybCwk/gfGmDfsyeaUd5yqiEehjit0i5AyGVM9EfenmI1rlQLhreYKLMR4rysegU InMw/qls+m1yfO1aKzQ5bNH7MEfKeBUiUVUL9TNnm8PP7OODUe6f1suLYyye8JfGVTzq 6KN3LeFtjGhDvArsdigzokOJB/qRlHv6LMYyO4UYWdhKIyWfvoLv90Zm0GgpgFwnW5Kx r4Fw== X-Gm-Message-State: AFqh2kq0605eupuvKR9z/keIBpsThISVg/8XBKZ93bvBIMqVAzMtaEHK hD1Ev/k7AaraZoYZjnSq9MI= X-Received: by 2002:a05:6a00:3691:b0:577:6264:9d0f with SMTP id dw17-20020a056a00369100b0057762649d0fmr5425480pfb.6.1671693967081; Wed, 21 Dec 2022 23:26:07 -0800 (PST) Received: from localhost.localdomain ([43.132.141.4]) by smtp.gmail.com with ESMTPSA id j186-20020a62c5c3000000b00573eb4a9a66sm11815117pfg.2.2022.12.21.23.26.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Dec 2022 23:26:06 -0800 (PST) From: korantwork@gmail.com To: nirmal.patel@linux.intel.com, jonathan.derrick@linux.dev, lpieralisi@kernel.org Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, Xinghui Li Subject: [PATCH] PCI: vmd: Do not disable MSI-X remapping in VMD 28C0 controller Date: Thu, 22 Dec 2022 15:26:03 +0800 Message-Id: <20221222072603.1175248-1-korantwork@gmail.com> X-Mailer: git-send-email 2.31.1 MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1752898544409354228?= X-GMAIL-MSGID: =?utf-8?q?1752898544409354228?= From: Xinghui Li Commit ee81ee84f873("PCI: vmd: Disable MSI-X remapping when possible") disable the vmd MSI-X remapping for optimizing pci performance.However, this feature severely negatively optimized performance in multi-disk situations. In FIO 4K random test, we test 1 disk in the 1 CPU when disable MSI-X remapping: read: IOPS=1183k, BW=4622MiB/s (4847MB/s)(1354GiB/300001msec) READ: bw=4622MiB/s (4847MB/s), 4622MiB/s-4622MiB/s (4847MB/s-4847MB/s), io=1354GiB (1454GB), run=300001-300001msec When not disable MSI-X remapping: read: IOPS=1171k, BW=4572MiB/s (4795MB/s)(1340GiB/300001msec) READ: bw=4572MiB/s (4795MB/s), 4572MiB/s-4572MiB/s (4795MB/s-4795MB/s), io=1340GiB (1438GB), run=300001-300001msec However, the bypass mode could increase the interrupts costs in CPU. We test 12 disks in the 6 CPU, When disable MSI-X remapping: read: IOPS=562k, BW=2197MiB/s (2304MB/s)(644GiB/300001msec) READ: bw=2197MiB/s (2304MB/s), 2197MiB/s-2197MiB/s (2304MB/s-2304MB/s), io=644GiB (691GB), run=300001-300001msec When not disable MSI-X remapping: read: IOPS=1144k, BW=4470MiB/s (4687MB/s)(1310GiB/300005msec) READ: bw=4470MiB/s (4687MB/s), 4470MiB/s-4470MiB/s (4687MB/s-4687MB/s), io=1310GiB (1406GB), run=300005-300005msec Signed-off-by: Xinghui Li --- drivers/pci/controller/vmd.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/drivers/pci/controller/vmd.c b/drivers/pci/controller/vmd.c index e06e9f4fc50f..9f6e9324d67d 100644 --- a/drivers/pci/controller/vmd.c +++ b/drivers/pci/controller/vmd.c @@ -998,8 +998,7 @@ static const struct pci_device_id vmd_ids[] = { .driver_data = VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP,}, {PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_VMD_28C0), .driver_data = VMD_FEAT_HAS_MEMBAR_SHADOW | - VMD_FEAT_HAS_BUS_RESTRICTIONS | - VMD_FEAT_CAN_BYPASS_MSI_REMAP,}, + VMD_FEAT_HAS_BUS_RESTRICTIONS,}, {PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0x467f), .driver_data = VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP | VMD_FEAT_HAS_BUS_RESTRICTIONS |