From patchwork Thu Feb 1 21:05:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Georgi Djakov X-Patchwork-Id: 195463 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:9bc1:b0:106:209c:c626 with SMTP id op1csp23638dyc; Thu, 1 Feb 2024 13:07:37 -0800 (PST) X-Google-Smtp-Source: AGHT+IH1W9qoruObTHMWeMRCXrn4Zo3QoucwRx5a18zQAV9bMBH3/UOpFjRhGguDFRB2y6mTmLU9 X-Received: by 2002:ac8:5e46:0:b0:42b:eca2:ee7b with SMTP id i6-20020ac85e46000000b0042beca2ee7bmr526537qtx.2.1706821657377; Thu, 01 Feb 2024 13:07:37 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706821657; cv=pass; d=google.com; s=arc-20160816; b=oboi0GQ4oXSOGYngvlHujtiRT0WfTssXLAqjCsTam71Ng5CMoFkulx6Lkad2SqXXeM cNoI6eij+8vGLBEqyAnEDBe7bvJy6nRvpL4GCNZDLDwr2nzXmKFAce/6vpkqBJY2qKyC PcbbGE2dtS3o242oHcv5uDZL/UQm/lwTtydtcJjBtDf06Dz10FVcHuTRISHNOw1SPUfk +/BfanJLhUTsKvbcCHAskn2KG/+Wc86K+yjEVOW6Y5YL32HHqPYy9ZoN/xcMi0gC7VAt UbuyDfUfDnjsVs74g8X7DBJrZPHOjSe9RJpAGV8PyrR/K59PBxbFym04T8T+EVm+Dg+h /tgw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:list-unsubscribe:list-subscribe:list-id:precedence :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=HuuOBXSg+8tIdBcl9fCjzuCorwFa6mDmTJdQXVi8HVE=; fh=ASGTh/upSf1Vy8gjXSjvbk7plXzzp97v32FVI5GbZJs=; b=tu4olZSwzuVYyaZ6Ix6nM2rNlkAkVJ/m5+3YdvEUvnCuk+84ds3XU/NvDrqEIIqooN j8/3lD4fH0iX5v2DxVjHY5CwHAImB0lEVq/jDMs8BGsV8Cth/kOssV77W89wOEuFEWUR mM/e9n1Fg1Zqvryltiyjv5lINhcroKnQC72WznX0/dpWSXHnBBK+/RtWK+mXnGfoihGD Zcf2RaPs7BFDVQJqSZb71EIt8IZDvteCD7GpCo405QNIHa2MCkn2df5kUleGbt5bsm5Y 2cbATnf9gJ76M51ZC+Kt5f5adTF46kWN3U45/fhRYPV9o+RnNG9u1Kx0ZPL4TFCKTJWS Vkkw==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=nLv1QCse; arc=pass (i=1 spf=pass spfdomain=quicinc.com dkim=pass dkdomain=quicinc.com dmarc=pass fromdomain=quicinc.com); spf=pass (google.com: domain of linux-kernel+bounces-48848-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-48848-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com X-Forwarded-Encrypted: i=1; AJvYcCVPK9DQDKUqeun0IGGeBTOPq6wbV3KLKgX0eswxOcOnvzlmFqEd10JAs18zB0x2weKhRkChgiwLaMx7AYeyac4+SiftHw== Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id y20-20020a05622a005400b0042be1aeae59si358160qtw.358.2024.02.01.13.07.37 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 01 Feb 2024 13:07:37 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-48848-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=nLv1QCse; arc=pass (i=1 spf=pass spfdomain=quicinc.com dkim=pass dkdomain=quicinc.com dmarc=pass fromdomain=quicinc.com); spf=pass (google.com: domain of linux-kernel+bounces-48848-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-48848-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id A4C191C23383 for ; Thu, 1 Feb 2024 21:07:33 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 26A774177B; Thu, 1 Feb 2024 21:06:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="nLv1QCse" Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6FA267472; Thu, 1 Feb 2024 21:05:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.168.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706821561; cv=none; b=bDIpMJtWwyZn8pz/i4zHMpO+UHW0wFTe9KLcx4uBSz3Ho8WoWAn1vqgeuyYHFHZLw8r7+rJ01QEQTEiwS8GhKv4etWJHnVyJlk81dyICxivZTZ70u1M6lyUEt+UUrzwkBJlrTbWJyuzRmTAET1+74DpagLD+0Jf0ajDaAYgDLGc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706821561; c=relaxed/simple; bh=gxm9PRp9PCNBWyTzKX0mhIMd7sNVCmI8xEnEwkiBKhg=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=NUfxoXw/moz3meIEB8NcgzynB/4RNrEvYdsr5aj/bJ+k8Eh6CHSFlnfun/HyI4N+p6RMXXVsJ+1i95va+Y4WJM/LKgtp0ZauVFMn0HpgMd6FKh63bEJhgncYa4ploHJEZ71KtGH/5ri83DwXIppy7GxVTdTdlh5ux0hi6uMYfVQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com; spf=pass smtp.mailfrom=quicinc.com; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b=nLv1QCse; arc=none smtp.client-ip=205.220.168.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=quicinc.com Received: from pps.filterd (m0279864.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 411FDs2Y017314; Thu, 1 Feb 2024 21:05:43 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-type; s=qcppdkim1; bh=HuuOBXSg+8tIdBcl9fCj zuCorwFa6mDmTJdQXVi8HVE=; b=nLv1QCse1x0LsNJdNDktgHXb7n3kuNk4Z0ya diwvP88eTbAfNz7oNnas75ubNkDXVZrKfMV3Nly+ccS37QHeCqQjYuaUTskZ2j+V yBV15RROAdHj38Ixa9UQgN4VVawcaDQC4CYApOOBUVWuyHomiUMXOP/pRuaDncXH E+Fhw72VXBFJy3G5339a1X4nxcCzVNmvoXm7P7O1S2Y9COXAybAmzRLbWgt/xLPE WJKFkTpXDlAUynKEcnhrkwHyR+YsV2HBDhbLeIEOkOtjzGaWSqFpHFPNobeGQtU5 au8InX3MvA9g5b0M47GJOH+xlK9XTiVKWNBrfKtYvPSDKLWoxA== Received: from nasanppmta03.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3w0b4y16vm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 01 Feb 2024 21:05:43 +0000 (GMT) Received: from nasanex01a.na.qualcomm.com (nasanex01a.na.qualcomm.com [10.52.223.231]) by NASANPPMTA03.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 411L5gtu015593 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 1 Feb 2024 21:05:42 GMT Received: from hu-c-gdjako-lv.qualcomm.com (10.49.16.6) by nasanex01a.na.qualcomm.com (10.52.223.231) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.40; Thu, 1 Feb 2024 13:05:42 -0800 From: Georgi Djakov To: , , , , , , CC: , , , , , , , , , , , Subject: [PATCH v4 06/10] iommu/arm-smmu-qcom: Use a custom context fault handler for sdm845 Date: Thu, 1 Feb 2024 13:05:25 -0800 Message-ID: <20240201210529.7728-7-quic_c_gdjako@quicinc.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240201210529.7728-1-quic_c_gdjako@quicinc.com> References: <20240201210529.7728-1-quic_c_gdjako@quicinc.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: nalasex01a.na.qualcomm.com (10.47.209.196) To nasanex01a.na.qualcomm.com (10.52.223.231) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: OixCnjCIb8XWwA5o1X_uFV5xxhuD-x3- X-Proofpoint-GUID: OixCnjCIb8XWwA5o1X_uFV5xxhuD-x3- X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-02-01_06,2024-01-31_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 malwarescore=0 bulkscore=0 priorityscore=1501 impostorscore=0 suspectscore=0 lowpriorityscore=0 spamscore=0 adultscore=0 phishscore=0 mlxscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2401190000 definitions=main-2402010163 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789732226301825112 X-GMAIL-MSGID: 1789732226301825112 The sdm845 platform now supports TBUs, so let's get additional debug info from the TBUs when a context fault occurs. Implement a custom context fault handler that does both software + hardware page table walks and TLB Invalidate All. Signed-off-by: Georgi Djakov --- .../iommu/arm/arm-smmu/arm-smmu-qcom-tbu.c | 130 ++++++++++++++++++ drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c | 4 + 2 files changed, 134 insertions(+) diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu-qcom-tbu.c b/drivers/iommu/arm/arm-smmu/arm-smmu-qcom-tbu.c index c00f89f223bd..167e3616f021 100644 --- a/drivers/iommu/arm/arm-smmu/arm-smmu-qcom-tbu.c +++ b/drivers/iommu/arm/arm-smmu/arm-smmu-qcom-tbu.c @@ -301,6 +301,136 @@ static phys_addr_t qsmmuv500_iova_to_phys(struct arm_smmu_domain *smmu_domain, return phys; } +static phys_addr_t qcom_smmu_iova_to_phys_hard(struct arm_smmu_domain *smmu_domain, dma_addr_t iova) +{ + struct arm_smmu_device *smmu = smmu_domain->smmu; + int idx = smmu_domain->cfg.cbndx; + u32 frsynra; + u16 sid; + + frsynra = arm_smmu_gr1_read(smmu, ARM_SMMU_GR1_CBFRSYNRA(idx)); + sid = FIELD_GET(ARM_SMMU_CBFRSYNRA_SID, frsynra); + + return qsmmuv500_iova_to_phys(smmu_domain, iova, sid); +} + +static phys_addr_t qcom_smmu_verify_fault(struct arm_smmu_domain *smmu_domain, dma_addr_t iova, u32 fsr) +{ + struct io_pgtable *iop = io_pgtable_ops_to_pgtable(smmu_domain->pgtbl_ops); + struct arm_smmu_device *smmu = smmu_domain->smmu; + phys_addr_t phys_post_tlbiall; + phys_addr_t phys; + + phys = qcom_smmu_iova_to_phys_hard(smmu_domain, iova); + io_pgtable_tlb_flush_all(iop); + phys_post_tlbiall = qcom_smmu_iova_to_phys_hard(smmu_domain, iova); + + if (phys != phys_post_tlbiall) { + dev_err(smmu->dev, + "ATOS results differed across TLBIALL... (before: %pa after: %pa)\n", + &phys, &phys_post_tlbiall); + } + + return (phys == 0 ? phys_post_tlbiall : phys); +} + +irqreturn_t qcom_smmu_context_fault(int irq, void *dev) +{ + struct arm_smmu_domain *smmu_domain = dev; + struct io_pgtable_ops *ops = smmu_domain->pgtbl_ops; + struct arm_smmu_device *smmu = smmu_domain->smmu; + u32 fsr, fsynr, cbfrsynra, resume = 0; + int idx = smmu_domain->cfg.cbndx; + phys_addr_t phys_soft; + unsigned long iova; + int ret, tmp; + + static DEFINE_RATELIMIT_STATE(_rs, + DEFAULT_RATELIMIT_INTERVAL, + DEFAULT_RATELIMIT_BURST); + + fsr = arm_smmu_cb_read(smmu, idx, ARM_SMMU_CB_FSR); + if (!(fsr & ARM_SMMU_FSR_FAULT)) + return IRQ_NONE; + + fsynr = arm_smmu_cb_read(smmu, idx, ARM_SMMU_CB_FSYNR0); + iova = arm_smmu_cb_readq(smmu, idx, ARM_SMMU_CB_FAR); + cbfrsynra = arm_smmu_gr1_read(smmu, ARM_SMMU_GR1_CBFRSYNRA(idx)); + + phys_soft = ops->iova_to_phys(ops, iova); + + tmp = report_iommu_fault(&smmu_domain->domain, NULL, iova, + fsynr & ARM_SMMU_FSYNR0_WNR ? IOMMU_FAULT_WRITE : IOMMU_FAULT_READ); + if (!tmp || tmp == -EBUSY) { + dev_dbg(smmu->dev, + "Context fault handled by client: iova=0x%08lx, fsr=0x%x, fsynr=0x%x, cb=%d\n", + iova, fsr, fsynr, idx); + dev_dbg(smmu->dev, "soft iova-to-phys=%pa\n", &phys_soft); + ret = IRQ_HANDLED; + resume = ARM_SMMU_RESUME_TERMINATE; + } else { + phys_addr_t phys_atos = qcom_smmu_verify_fault(smmu_domain, iova, fsr); + + if (__ratelimit(&_rs)) { + dev_err(smmu->dev, + "Unhandled context fault: fsr=0x%x, iova=0x%08lx, fsynr=0x%x, cbfrsynra=0x%x, cb=%d\n", + fsr, iova, fsynr, cbfrsynra, idx); + dev_err(smmu->dev, + "FSR = %08x [%s%s%s%s%s%s%s%s%s], SID=0x%x\n", + fsr, + (fsr & 0x02) ? "TF " : "", + (fsr & 0x04) ? "AFF " : "", + (fsr & 0x08) ? "PF " : "", + (fsr & 0x10) ? "EF " : "", + (fsr & 0x20) ? "TLBMCF " : "", + (fsr & 0x40) ? "TLBLKF " : "", + (fsr & 0x80) ? "MHF " : "", + (fsr & 0x40000000) ? "SS " : "", + (fsr & 0x80000000) ? "MULTI " : "", + cbfrsynra); + + dev_err(smmu->dev, + "soft iova-to-phys=%pa\n", &phys_soft); + if (!phys_soft) + dev_err(smmu->dev, + "SOFTWARE TABLE WALK FAILED! Looks like %s accessed an unmapped address!\n", + dev_name(smmu->dev)); + if (phys_atos) + dev_err(smmu->dev, "hard iova-to-phys (ATOS)=%pa\n", + &phys_atos); + else + dev_err(smmu->dev, "hard iova-to-phys (ATOS) failed\n"); + } + ret = IRQ_NONE; + resume = ARM_SMMU_RESUME_TERMINATE; + } + + /* + * If the client returns -EBUSY, do not clear FSR and do not RESUME + * if stalled. This is required to keep the IOMMU client stalled on + * the outstanding fault. This gives the client a chance to take any + * debug action and then terminate the stalled transaction. + * So, the sequence in case of stall on fault should be: + * 1) Do not clear FSR or write to RESUME here + * 2) Client takes any debug action + * 3) Client terminates the stalled transaction and resumes the IOMMU + * 4) Client clears FSR. The FSR should only be cleared after 3) and + * not before so that the fault remains outstanding. This ensures + * SCTLR.HUPCF has the desired effect if subsequent transactions also + * need to be terminated. + */ + if (tmp != -EBUSY) { + /* Clear the faulting FSR */ + arm_smmu_cb_write(smmu, idx, ARM_SMMU_CB_FSR, fsr); + + /* Retry or terminate any stalled transactions */ + if (fsr & ARM_SMMU_FSR_SS) + arm_smmu_cb_write(smmu, idx, ARM_SMMU_CB_RESUME, resume); + } + + return ret; +} + static int qsmmuv500_tbu_probe(struct platform_device *pdev) { struct device_node *np = pdev->dev.of_node; diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c b/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c index ca806644e6eb..2635f5b31455 100644 --- a/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c +++ b/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c @@ -423,6 +423,10 @@ static const struct arm_smmu_impl sdm845_smmu_500_impl = { .reset = qcom_sdm845_smmu500_reset, .write_s2cr = qcom_smmu_write_s2cr, .tlb_sync = qcom_smmu_tlb_sync, +#ifdef CONFIG_ARM_SMMU_QCOM_TBU + .context_fault = qcom_smmu_context_fault, + .context_fault_needs_threaded_irq = true, +#endif }; static const struct arm_smmu_impl qcom_adreno_smmu_v2_impl = {