From patchwork Thu Jan 25 15:04:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lennert Buytenhek X-Patchwork-Id: 192121 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:e09d:b0:103:945f:af90 with SMTP id gm29csp35527dyb; Thu, 25 Jan 2024 07:05:13 -0800 (PST) X-Google-Smtp-Source: AGHT+IEdg5YCcirXIDxg8SMc+RoRn/Uut5xLWm9cgZ16lBzPkD5PL9z4L2ClgZy8FOz25RYcGv88 X-Received: by 2002:a05:622a:150:b0:42a:384a:78a6 with SMTP id v16-20020a05622a015000b0042a384a78a6mr1177276qtw.30.1706195113546; Thu, 25 Jan 2024 07:05:13 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706195113; cv=pass; d=google.com; s=arc-20160816; b=DBQPB7WH520ueEhx7RLiPuT4wLcXoPwBl19unWPsi5zGOhnvpNJM6Dmgr9iaWFE57/ I4M9XMz3D0vZP/sC52z9r2zk3JBPsMJYBZf17g/U/aCT1UaXahaXStwyWilWU486gDmK vp71BOgnMHcX64Hu0RpMgMkxIh+T806C1Zx+lEg/NLW0Fmdf1fTeoAtHs5ljOsV8LHe7 Eqj1cJIYNMANm3aD9woboNYHRlNO/c029jQz0Ao5pnd4qudICxNENzIXeytEE0cx7sld 1vPWY1dNBy6UEs+swmKippkHmnD7YH/Jbb5p3zI2rLdIhwlL8VKpyDitbh3zvHJSWeYQ fV2g== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-disposition:mime-version:list-unsubscribe:list-subscribe :list-id:precedence:message-id:subject:cc:to:from:date; bh=F5JF+BGx9oGiYbEJ6DQYuvfRUlgaFaqkSCXldML8ezw=; fh=gIKVftY6bhzdnpRA4n3C2BmeGIMP9hSE+c2lyBWzykM=; b=iM2Flo9yWkQJUhTGdnL5yof9/CHKKqaFKvOLCv4wzl8LDX40LKjy2SKAKJHaGiy+a3 XHaZc7sMUJDFJdgwbVKn730H41ddr4rf+pbLXrGPMH3CkmItZxRzMvssC7amka/Mdfi5 1DEqW+g3Dip72nMqIK5/VQnNtsju4LMbwK55BSTAz2Bvc+YSQyTTZZ9MUlQgmDR50XBv sCA6JjYF5Ym4zOOU2oYodYyDuPtsf+JcjM00ukR2CJe7bzlC61RVQfLXoL0OtTtv64hd WjHAK+qmQzi70pCKdslNwN6eidGYob403Ug4jCdh1XR0uspO6WqiRbywDgA4G4T+VYfH Z44w== ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=wantstofly.org); spf=pass (google.com: domain of linux-kernel+bounces-38792-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-38792-ouuuleilei=gmail.com@vger.kernel.org" Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id t38-20020a05622a182600b0042a35e9b068si11779877qtc.349.2024.01.25.07.05.13 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Jan 2024 07:05:13 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-38792-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=wantstofly.org); spf=pass (google.com: domain of linux-kernel+bounces-38792-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-38792-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 0CC2D1C24E73 for ; Thu, 25 Jan 2024 15:04:41 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 4E85E73174; Thu, 25 Jan 2024 15:04:12 +0000 (UTC) Received: from mail.wantstofly.org (hmm.wantstofly.org [213.239.204.108]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 83E906E2C9; Thu, 25 Jan 2024 15:04:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.239.204.108 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706195050; cv=none; b=uf/9BmTdN1qZfyK6u2hJaGil6ASf7TfIMELMPlELshZ2Nlu+hSrUxbZ4aJMRpsHQOgmXibc32TDQKseTiVPdqEtfoPBGc9v8rbOOWtBGQptQDQoL06a2MK/7XWXqZn6ZInv+ZgkBzsuCLWaOGjn3vKp6Zb92fgc5e9i153NIGwI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706195050; c=relaxed/simple; bh=Uj12xY5O7bvmsb6dk74jYQu7SJ0BQuZaMSiAXqwFNfE=; h=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type: Content-Disposition; b=B7Oau1ypOd/SoE+AJmEt1KF0ls2nORnMGBFWaZxti0Of2oe14mXK6NgacvxijZD8SRvmu6AWFu5LJqVQdrStweCvqS6mJVkej79hV8a2l5gC4/P1ZZ1vafvutxub+dc5aZT2Rt6ns+OQSPOLWYoVqOtmdsj6E2iNFqRkflmODPo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=wantstofly.org; spf=pass smtp.mailfrom=wantstofly.org; arc=none smtp.client-ip=213.239.204.108 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=wantstofly.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=wantstofly.org Received: by mail.wantstofly.org (Postfix, from userid 1000) id B45547F712; Thu, 25 Jan 2024 17:04:01 +0200 (EET) Date: Thu, 25 Jan 2024 17:04:01 +0200 From: Lennert Buytenhek To: Niklas Cassel , Damien Le Moal , linux-ide@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Robin Murphy , John Garry , Joerg Roedel Subject: [PATCH v2] ahci: add 43-bit DMA address quirk for ASMedia ASM1061 controllers Message-ID: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789075247674187942 X-GMAIL-MSGID: 1789075247674187942 With one of the on-board ASM1061 AHCI controllers (1b21:0612) on an ASUSTeK Pro WS WRX80E-SAGE SE WIFI mainboard, a controller hang was observed that was immediately preceded by the following kernel messages: [Thu Jan 4 23:12:54 2024] ahci 0000:28:00.0: Using 64-bit DMA addresses [Thu Jan 4 23:12:54 2024] ahci 0000:28:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0035 address=0x7fffff00000 flags=0x0000] [Thu Jan 4 23:12:54 2024] ahci 0000:28:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0035 address=0x7fffff00300 flags=0x0000] [Thu Jan 4 23:12:54 2024] ahci 0000:28:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0035 address=0x7fffff00380 flags=0x0000] [Thu Jan 4 23:12:54 2024] ahci 0000:28:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0035 address=0x7fffff00400 flags=0x0000] [Thu Jan 4 23:12:54 2024] ahci 0000:28:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0035 address=0x7fffff00680 flags=0x0000] [Thu Jan 4 23:12:54 2024] ahci 0000:28:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0035 address=0x7fffff00700 flags=0x0000] The first message is produced by code in drivers/iommu/dma-iommu.c which is accompanied by the following comment that seems to apply: /* * Try to use all the 32-bit PCI addresses first. The original SAC vs. * DAC reasoning loses relevance with PCIe, but enough hardware and * firmware bugs are still lurking out there that it's safest not to * venture into the 64-bit space until necessary. * * If your device goes wrong after seeing the notice then likely either * its driver is not setting DMA masks accurately, the hardware has * some inherent bug in handling >32-bit addresses, or not all the * expected address bits are wired up between the device and the IOMMU. */ Asking the ASM1061 on a discrete PCIe card to DMA from I/O virtual address 0xffffffff00000000 produces the following I/O page faults: [Tue Jan 23 21:31:55 2024] vfio-pci 0000:07:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0021 address=0x7ff00000000 flags=0x0010] [Tue Jan 23 21:31:55 2024] vfio-pci 0000:07:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0021 address=0x7ff00000500 flags=0x0010] Note that the upper 21 bits of the logged DMA address are zero. (When asking a different PCIe device in the same PCIe slot to DMA to the same I/O virtual address, we do see all the upper 32 bits of the DMA address as 1, so this is not an issue with the chipset or IOMMU configuration on the test system.) Also, hacking libahci to always set the upper 21 bits of all DMA addresses to 1 produces no discernible effect on the behavior of the ASM1061, and mkfs/mount/scrub/etc work as without this hack. This all strongly suggests that the ASM1061 has a 43 bit DMA address limit, and this commit therefore adds a quirk to deal with this limit. This issue probably applies to (some of) the other supported ASMedia parts as well, but we limit it to the PCI IDs known to refer to ASM1061 parts, as that's the only part we know for sure to be affected by this issue at this point. Link: https://lore.kernel.org/linux-ide/ZaZ2PIpEId-rl6jv@wantstofly.org/ Signed-off-by: Lennert Buytenhek --- Changes in v2: - Make quirk be for ASM1061 only, requested by Niklas. Also: - Rename the board type from board_ahci_asm106x to board_ahci_43bit_dma, as the quirk no longer applies to all ASM106x chipsets, and it's a bit clearer to indicate the specific quirk in the board name. - Tweak commit description accordingly. - Make AHCI_HFLAG_43BIT_ONLY handling logic clearer, requested by Niklas. Link to v1: https://lore.kernel.org/linux-ide/ZbEWsp4KT-V9YUaC@wantstofly.org/ --- drivers/ata/ahci.c | 29 +++++++++++++++++++++++------ drivers/ata/ahci.h | 1 + 2 files changed, 24 insertions(+), 6 deletions(-) diff --git a/drivers/ata/ahci.c b/drivers/ata/ahci.c index 08745e7db820..253bfa8aeb51 100644 --- a/drivers/ata/ahci.c +++ b/drivers/ata/ahci.c @@ -48,6 +48,7 @@ enum { enum board_ids { /* board IDs by feature in alphabetical order */ board_ahci, + board_ahci_43bit_dma, board_ahci_ign_iferr, board_ahci_low_power, board_ahci_no_debounce_delay, @@ -128,6 +129,13 @@ static const struct ata_port_info ahci_port_info[] = { .udma_mask = ATA_UDMA6, .port_ops = &ahci_ops, }, + [board_ahci_43bit_dma] = { + AHCI_HFLAGS (AHCI_HFLAG_43BIT_ONLY), + .flags = AHCI_FLAG_COMMON, + .pio_mask = ATA_PIO4, + .udma_mask = ATA_UDMA6, + .port_ops = &ahci_ops, + }, [board_ahci_ign_iferr] = { AHCI_HFLAGS (AHCI_HFLAG_IGN_IRQ_IF_ERR), .flags = AHCI_FLAG_COMMON, @@ -596,11 +604,11 @@ static const struct pci_device_id ahci_pci_tbl[] = { { PCI_VDEVICE(PROMISE, 0x3f20), board_ahci }, /* PDC42819 */ { PCI_VDEVICE(PROMISE, 0x3781), board_ahci }, /* FastTrak TX8660 ahci-mode */ - /* Asmedia */ + /* ASMedia */ { PCI_VDEVICE(ASMEDIA, 0x0601), board_ahci }, /* ASM1060 */ { PCI_VDEVICE(ASMEDIA, 0x0602), board_ahci }, /* ASM1060 */ - { PCI_VDEVICE(ASMEDIA, 0x0611), board_ahci }, /* ASM1061 */ - { PCI_VDEVICE(ASMEDIA, 0x0612), board_ahci }, /* ASM1062 */ + { PCI_VDEVICE(ASMEDIA, 0x0611), board_ahci_43bit_dma }, /* ASM1061 */ + { PCI_VDEVICE(ASMEDIA, 0x0612), board_ahci_43bit_dma }, /* ASM1061/1062 */ { PCI_VDEVICE(ASMEDIA, 0x0621), board_ahci }, /* ASM1061R */ { PCI_VDEVICE(ASMEDIA, 0x0622), board_ahci }, /* ASM1062R */ { PCI_VDEVICE(ASMEDIA, 0x0624), board_ahci }, /* ASM1062+JMB575 */ @@ -943,11 +951,20 @@ static int ahci_pci_device_resume(struct device *dev) #endif /* CONFIG_PM */ -static int ahci_configure_dma_masks(struct pci_dev *pdev, int using_dac) +static int ahci_configure_dma_masks(struct pci_dev *pdev, + struct ahci_host_priv *hpriv) { - const int dma_bits = using_dac ? 64 : 32; + int dma_bits; int rc; + if (hpriv->cap & HOST_CAP_64) { + dma_bits = 64; + if (hpriv->flags & AHCI_HFLAG_43BIT_ONLY) + dma_bits = 43; + } else { + dma_bits = 32; + } + /* * If the device fixup already set the dma_mask to some non-standard * value, don't extend it here. This happens on STA2X11, for example. @@ -1920,7 +1937,7 @@ static int ahci_init_one(struct pci_dev *pdev, const struct pci_device_id *ent) ahci_gtf_filter_workaround(host); /* initialize adapter */ - rc = ahci_configure_dma_masks(pdev, hpriv->cap & HOST_CAP_64); + rc = ahci_configure_dma_masks(pdev, hpriv); if (rc) return rc; diff --git a/drivers/ata/ahci.h b/drivers/ata/ahci.h index 4bae95b06ae3..df8f8a1a3a34 100644 --- a/drivers/ata/ahci.h +++ b/drivers/ata/ahci.h @@ -247,6 +247,7 @@ enum { AHCI_HFLAG_SUSPEND_PHYS = BIT(26), /* handle PHYs during suspend/resume */ AHCI_HFLAG_NO_SXS = BIT(28), /* SXS not supported */ + AHCI_HFLAG_43BIT_ONLY = BIT(29), /* 43bit DMA addr limit */ /* ap->flags bits */