From patchwork Mon Feb 5 15:32:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Robin Murphy X-Patchwork-Id: 196877 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:168b:b0:106:860b:bbdd with SMTP id ma11csp960775dyb; Mon, 5 Feb 2024 07:48:26 -0800 (PST) X-Google-Smtp-Source: AGHT+IFwJWWECtef0orccGzE/M0DmWdhTmOZSDz8xoi7SdHiUKD4+EBmODJ6SNwXn2KvPOXoyXZu X-Received: by 2002:a05:6a00:8cb:b0:6df:ee93:570 with SMTP id s11-20020a056a0008cb00b006dfee930570mr8988486pfu.6.1707148105942; Mon, 05 Feb 2024 07:48:25 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707148105; cv=pass; d=google.com; s=arc-20160816; b=hvy9g7vUXQwAuORPpx8J8Q4GmnwoT9MXUKsRpWQ899tCLa4eKSN23/yrupArGDhwa/ oAe3pQ72ywK7HeBsHP4amt2rgj1VMv6cVyOC9cu8tibyfHrtFwCSeiWXYr2LD+u0mMf5 ZaFSGM5lu/ilRGjqTY4FmpLk4PQASOFHnd/ozJ+Y/IphLNiS0ByPeTzE+jGTRzljLJo8 DoadOYYYTvCCDI4DgUhgIrUtBpMC6ATwmjk7nZYA6ndgPhDqkzNtiyt4kJL5rWKupLvc boqyWlcq4v4QfS4cegKhSixvJsKX3D/wj1b6rLa8LpOxTMY/5hFoTbi89V8MP539F8Nk aTjQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=yY+4M7kMt2BVJso2Lto96zv05XHdud4jM/u756+6AO4=; fh=LHA1qWhKFI4HOaWg5TOkQurzOc6cLnn87URmahmvLAU=; b=LsjRMINPMqYj3NjnibNUKjO+70YJLgyXHM+nqjZRU+UwW0yVE5yUUQkqj+sfHXD9u/ EBu/Ey6iQEZ5oEuAg4hIGjgmv5fOu8F1sV5QwTT8q3uVnXkk86hRqA2jC67LieUvS6KU DgVPlxa25+msXlcygahHA6hn38ba2NaEYOhQbGj4xn5jtWeXkYKc1vDGJRVMpk/Necu/ eb8CQ4y0QgplaI4+7gym1AAV5eOzffEB4APf7BJ7VnEg4rb3pXyzHzgXO6rusVXXa/D/ 2zUkPzN11Bs2laThhNrm3FiXUxeTvwCfLx2e/csvbOLO8CWhluq57OEZkzzKmrdKep1Z imhw==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-52916-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-52916-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com X-Forwarded-Encrypted: i=1; AJvYcCX8/CLJnyEhqDG9H4Day+85FvvFlI9DXv0G/WUvMSy+gpOmSDuW6BUR8pyt4+gitKyAdkkSrsKIEnHwYmPZ1y/LMw0Qig== Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id q23-20020aa79837000000b006e02da43aa0si588706pfl.281.2024.02.05.07.48.25 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Feb 2024 07:48:25 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-52916-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-52916-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-52916-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 6E121289D77 for ; Mon, 5 Feb 2024 15:33:59 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 342E03FE44; Mon, 5 Feb 2024 15:32:52 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 082503A1DE for ; Mon, 5 Feb 2024 15:32:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707147170; cv=none; b=r8plPvrbMbw4u1EXr9V3ij9UIw5YiZD9z0W0UyTDg/+d0M5hPCw0yreJN9t1OYjjbTCDXJRMlhOOlfMfpZC68IMv0DRouMl+Rwb4+qbxac4Ujr6zVWeOy3qQYftdrxNCt2uIX1gs/ocb1uW49NDBr7nmcsU7Sq9bxg1S7CAMpmg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707147170; c=relaxed/simple; bh=M6yxaufXQSaUctlg9rkP73rx1vbqTh2nUd3NBrS+Se8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=NkBfw+TqNdEqpsRDcbhiMFp6W5CPPvkHrm+jhUEwuhyt6vsdfcIBT6UvYB5xlqB4T7zan4gd1WuLL/HIEdrTE3Z2gPkIDAZTLWZKdsLJUjRQcy9ndSG/guk0544IoknHaOdFeJOjJIFMimB1aaLztC1fa+XFZR4thiIf0rDrzRU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 36CBDDA7; Mon, 5 Feb 2024 07:33:30 -0800 (PST) Received: from e121345-lin.cambridge.arm.com (e121345-lin.cambridge.arm.com [10.1.196.40]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 6A7DF3F5A1; Mon, 5 Feb 2024 07:32:46 -0800 (PST) From: Robin Murphy To: joro@8bytes.org Cc: will@kernel.org, pasha.tatashin@soleen.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-mm@kvack.org, rientjes@google.com, yosryahmed@google.com, john.g.garry@oracle.com Subject: [PATCH v3 1/3] iommu/iova: Tidy up iova_cache_get() failure Date: Mon, 5 Feb 2024 15:32:39 +0000 Message-Id: X-Mailer: git-send-email 2.39.2.101.g768bb238c484.dirty In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790074532491459155 X-GMAIL-MSGID: 1790074532491459155 Failure handling in iova_cache_get() is a little messy, and we'd like to add some more to it, so let's tidy up a bit first. By leaving the hotplug handler until last we can take advantage of kmem_cache_destroy() being NULL-safe to have a single cleanup label. We can also improve the error reporting, noting that kmem_cache_create() already screams if it fails, so that one is redundant. Signed-off-by: Robin Murphy Acked-by: David Rientjes Reviewed-by: John Garry Reviewed-by: Jerry Snitselaar --- drivers/iommu/iova.c | 33 ++++++++++++++++----------------- 1 file changed, 16 insertions(+), 17 deletions(-) diff --git a/drivers/iommu/iova.c b/drivers/iommu/iova.c index d30e453d0fb4..cf95001d85c0 100644 --- a/drivers/iommu/iova.c +++ b/drivers/iommu/iova.c @@ -254,26 +254,20 @@ static void free_iova_mem(struct iova *iova) int iova_cache_get(void) { + int err = -ENOMEM; + mutex_lock(&iova_cache_mutex); if (!iova_cache_users) { - int ret; + iova_cache = kmem_cache_create("iommu_iova", sizeof(struct iova), 0, + SLAB_HWCACHE_ALIGN, NULL); + if (!iova_cache) + goto out_err; - ret = cpuhp_setup_state_multi(CPUHP_IOMMU_IOVA_DEAD, "iommu/iova:dead", NULL, - iova_cpuhp_dead); - if (ret) { - mutex_unlock(&iova_cache_mutex); - pr_err("Couldn't register cpuhp handler\n"); - return ret; - } - - iova_cache = kmem_cache_create( - "iommu_iova", sizeof(struct iova), 0, - SLAB_HWCACHE_ALIGN, NULL); - if (!iova_cache) { - cpuhp_remove_multi_state(CPUHP_IOMMU_IOVA_DEAD); - mutex_unlock(&iova_cache_mutex); - pr_err("Couldn't create iova cache\n"); - return -ENOMEM; + err = cpuhp_setup_state_multi(CPUHP_IOMMU_IOVA_DEAD, "iommu/iova:dead", + NULL, iova_cpuhp_dead); + if (err) { + pr_err("IOVA: Couldn't register cpuhp handler: %pe\n", ERR_PTR(err)); + goto out_err; } } @@ -281,6 +275,11 @@ int iova_cache_get(void) mutex_unlock(&iova_cache_mutex); return 0; + +out_err: + kmem_cache_destroy(iova_cache); + mutex_unlock(&iova_cache_mutex); + return err; } EXPORT_SYMBOL_GPL(iova_cache_get); From patchwork Mon Feb 5 15:32:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Robin Murphy X-Patchwork-Id: 196864 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:168b:b0:106:860b:bbdd with SMTP id ma11csp952278dyb; Mon, 5 Feb 2024 07:34:10 -0800 (PST) X-Google-Smtp-Source: AGHT+IG3aKvL82UyIyTggaDHu+PMPUjucZ+i1ukZzUReja4m1Q/8/R7V7GjV5YmDm9PaMj04JLH/ X-Received: by 2002:ad4:5aec:0:b0:68c:6a65:895f with SMTP id c12-20020ad45aec000000b0068c6a65895fmr8434014qvh.10.1707147250574; Mon, 05 Feb 2024 07:34:10 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707147250; cv=pass; d=google.com; s=arc-20160816; b=QtQl6mmQiwd0DJZRnqTFdTO51JpwqTC6DlPFEsdcuR0m+kA1GTIwFJO2W1p2lQVIB8 JTuD5HZpqA2N06yVkmzjbTbnyRiuvtCm6nYN/jYrwwKu8FYtVzO0WrFb+EZ2XYOfXRI4 ySTf1jchhPrTRJQEInoJoAjP43QXOB409osqv/k3J2G4fovt2l1em90tZjY4y943IqpF Og3r8idAZGzNIYEHZwHMFXJTFXW+m6kiqvvTXbI3jYQtkuPNL6LfRdOtuphstYjD8k3R VTZ4jdX5HW7eTuqXCwCURgC5FajSCkKuJkK5GvJO8ObVnS9aNHGASXf2EWIiwLFLzUMq 5QQQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=iYOjIPp5YF4pHRIdEAilCHpp38B3kyUgFoGeW/y2Mfw=; fh=5nJXcSr+yUDxpUf2vKXptyTvA5EBncXbqhSbPq3WIrI=; b=zmP8E5lAyKG85G06Anx0zdqqN4DpEDtCfr9H/4bt43H0DpYmmMce1WTi2Y4qYvQzgf 5UWMQ6+w70TEMYb1hupjc/XiXtuPjf6AdNNNb7bxrTf3n8t6GQJcYUFa8AvZKb8YGdg3 0HmU+0YH5HZhN6xuY30XIruKgIJyJow33K1fuULBuHGt/Q0XPWBJKDn18x7gJ5hCqEZC r/lqT9nuHRx/MPV9EAtNZpNCjw/VGtKWSuIufgYjlEqDBldqppF3r+P52rrBDWto11XU zz9E+JtJzYCgpOiUQ8yaClsxF3kO+7fC2TEmFXCEwJaxT48JxKp5MAjqGF9dQ8EcKW+S 38/A==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-52917-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-52917-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com X-Forwarded-Encrypted: i=1; AJvYcCXH3EPmwFpLT4xsG8cBWT7o64yNZ8UxmErHXRbWS7SFabvqsDzDylaz8obCe0zcb35YEjptvvTXenUDuY6eDnVXnEoMiw== Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id s15-20020ad4524f000000b0067f49d274besi114302qvq.428.2024.02.05.07.34.10 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Feb 2024 07:34:10 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-52917-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-52917-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-52917-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 512241C21C45 for ; Mon, 5 Feb 2024 15:34:10 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 378083EA9C; Mon, 5 Feb 2024 15:32:53 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 66F813B18D for ; Mon, 5 Feb 2024 15:32:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707147171; cv=none; b=TBSw+UYmpDo9qTAe3ZmoTSCA5ysPNNH5HtnUAVT8hALpj6P+WpydEQv/QCyCJdOklNk3gRnl2BPQPmZJTx4s/i0314B3B1q5vqOqMyrIZ2QcdEwc34Kl2kNnqK1rt8qZ+6br2V0E59EeAImM3Q+feVuSi8CkX1+BJHRrPN7EhoY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707147171; c=relaxed/simple; bh=r1hkzYJmSuuIn0H8ddW+ei/JSVISOgRUUYc6h1WMsCA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Iasucun0u9DQNhGSiSncUAe76rYj+gju9hNoezNLNcXmE5i69DqAavWTmMOePaI2azUH0mAeW/ysOqx6nogVsk+WW9/+hcyAZUT9Qn5+I5KFk0oAj4N/P35/1t7xq4nrLQmNF7x84pL+Pwrj9ydbG/jKKBf2tpZ5yTYMDLDLOGc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9287C12FC; Mon, 5 Feb 2024 07:33:31 -0800 (PST) Received: from e121345-lin.cambridge.arm.com (e121345-lin.cambridge.arm.com [10.1.196.40]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id C60753F5A1; Mon, 5 Feb 2024 07:32:47 -0800 (PST) From: Robin Murphy To: joro@8bytes.org Cc: will@kernel.org, pasha.tatashin@soleen.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-mm@kvack.org, rientjes@google.com, yosryahmed@google.com, john.g.garry@oracle.com Subject: [PATCH v3 2/3] iommu/iova: Reorganise some code Date: Mon, 5 Feb 2024 15:32:40 +0000 Message-Id: X-Mailer: git-send-email 2.39.2.101.g768bb238c484.dirty In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790073635284699505 X-GMAIL-MSGID: 1790073635284699505 The iova_cache_{get,put}() calls really represent top-level lifecycle management for the whole IOVA library, so it's long been rather confusing to have them buried right in the middle of the allocator implementation details. Move them to a more expected position at the end of the file, where it will then also be easier to expand them. With this, we can also move the rcache hotplug handler (plus another stray function) into the rcache portion of the file. Signed-off-by: Robin Murphy Acked-by: David Rientjes Reviewed-by: John Garry Reviewed-by: Jerry Snitselaar --- drivers/iommu/iova.c | 128 +++++++++++++++++++++---------------------- 1 file changed, 64 insertions(+), 64 deletions(-) diff --git a/drivers/iommu/iova.c b/drivers/iommu/iova.c index cf95001d85c0..b5de865ee50b 100644 --- a/drivers/iommu/iova.c +++ b/drivers/iommu/iova.c @@ -24,24 +24,8 @@ static bool iova_rcache_insert(struct iova_domain *iovad, static unsigned long iova_rcache_get(struct iova_domain *iovad, unsigned long size, unsigned long limit_pfn); -static void free_cpu_cached_iovas(unsigned int cpu, struct iova_domain *iovad); static void free_iova_rcaches(struct iova_domain *iovad); - -unsigned long iova_rcache_range(void) -{ - return PAGE_SIZE << (IOVA_RANGE_CACHE_MAX_SIZE - 1); -} - -static int iova_cpuhp_dead(unsigned int cpu, struct hlist_node *node) -{ - struct iova_domain *iovad; - - iovad = hlist_entry_safe(node, struct iova_domain, cpuhp_dead); - - free_cpu_cached_iovas(cpu, iovad); - return 0; -} - +static void free_cpu_cached_iovas(unsigned int cpu, struct iova_domain *iovad); static void free_global_cached_iovas(struct iova_domain *iovad); static struct iova *to_iova(struct rb_node *node) @@ -252,53 +236,6 @@ static void free_iova_mem(struct iova *iova) kmem_cache_free(iova_cache, iova); } -int iova_cache_get(void) -{ - int err = -ENOMEM; - - mutex_lock(&iova_cache_mutex); - if (!iova_cache_users) { - iova_cache = kmem_cache_create("iommu_iova", sizeof(struct iova), 0, - SLAB_HWCACHE_ALIGN, NULL); - if (!iova_cache) - goto out_err; - - err = cpuhp_setup_state_multi(CPUHP_IOMMU_IOVA_DEAD, "iommu/iova:dead", - NULL, iova_cpuhp_dead); - if (err) { - pr_err("IOVA: Couldn't register cpuhp handler: %pe\n", ERR_PTR(err)); - goto out_err; - } - } - - iova_cache_users++; - mutex_unlock(&iova_cache_mutex); - - return 0; - -out_err: - kmem_cache_destroy(iova_cache); - mutex_unlock(&iova_cache_mutex); - return err; -} -EXPORT_SYMBOL_GPL(iova_cache_get); - -void iova_cache_put(void) -{ - mutex_lock(&iova_cache_mutex); - if (WARN_ON(!iova_cache_users)) { - mutex_unlock(&iova_cache_mutex); - return; - } - iova_cache_users--; - if (!iova_cache_users) { - cpuhp_remove_multi_state(CPUHP_IOMMU_IOVA_DEAD); - kmem_cache_destroy(iova_cache); - } - mutex_unlock(&iova_cache_mutex); -} -EXPORT_SYMBOL_GPL(iova_cache_put); - /** * alloc_iova - allocates an iova * @iovad: - iova domain in question @@ -653,6 +590,11 @@ struct iova_rcache { struct delayed_work work; }; +unsigned long iova_rcache_range(void) +{ + return PAGE_SIZE << (IOVA_RANGE_CACHE_MAX_SIZE - 1); +} + static struct iova_magazine *iova_magazine_alloc(gfp_t flags) { struct iova_magazine *mag; @@ -989,5 +931,63 @@ static void free_global_cached_iovas(struct iova_domain *iovad) spin_unlock_irqrestore(&rcache->lock, flags); } } + +static int iova_cpuhp_dead(unsigned int cpu, struct hlist_node *node) +{ + struct iova_domain *iovad; + + iovad = hlist_entry_safe(node, struct iova_domain, cpuhp_dead); + + free_cpu_cached_iovas(cpu, iovad); + return 0; +} + +int iova_cache_get(void) +{ + int err = -ENOMEM; + + mutex_lock(&iova_cache_mutex); + if (!iova_cache_users) { + iova_cache = kmem_cache_create("iommu_iova", sizeof(struct iova), 0, + SLAB_HWCACHE_ALIGN, NULL); + if (!iova_cache) + goto out_err; + + err = cpuhp_setup_state_multi(CPUHP_IOMMU_IOVA_DEAD, "iommu/iova:dead", + NULL, iova_cpuhp_dead); + if (err) { + pr_err("IOVA: Couldn't register cpuhp handler: %pe\n", ERR_PTR(err)); + goto out_err; + } + } + + iova_cache_users++; + mutex_unlock(&iova_cache_mutex); + + return 0; + +out_err: + kmem_cache_destroy(iova_cache); + mutex_unlock(&iova_cache_mutex); + return err; +} +EXPORT_SYMBOL_GPL(iova_cache_get); + +void iova_cache_put(void) +{ + mutex_lock(&iova_cache_mutex); + if (WARN_ON(!iova_cache_users)) { + mutex_unlock(&iova_cache_mutex); + return; + } + iova_cache_users--; + if (!iova_cache_users) { + cpuhp_remove_multi_state(CPUHP_IOMMU_IOVA_DEAD); + kmem_cache_destroy(iova_cache); + } + mutex_unlock(&iova_cache_mutex); +} +EXPORT_SYMBOL_GPL(iova_cache_put); + MODULE_AUTHOR("Anil S Keshavamurthy "); MODULE_LICENSE("GPL");