From patchwork Tue Jan 30 02:09:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Haitao Huang X-Patchwork-Id: 193876 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp1068030dyb; Tue, 30 Jan 2024 00:11:21 -0800 (PST) X-Google-Smtp-Source: AGHT+IEFkJLsu5/h3FQu+poM9XfVeeZ9R6xtJizZDSQH8nIbn6qvPM9CuFO81wXSDGBoCZTVmf1C X-Received: by 2002:a05:6402:a44:b0:55f:4095:950f with SMTP id bt4-20020a0564020a4400b0055f4095950fmr549249edb.13.1706602280867; Tue, 30 Jan 2024 00:11:20 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706602280; cv=pass; d=google.com; s=arc-20160816; b=abgKJRjoyP4um9r68GDfHWpuoy+G3//2cbUmdFPhdK2uXLXSTdmaBHcOfNjVXdpuBR slHSk6PGpPypLSJyaBKrsMcO0dd9owWbxIxbwxVZTBZJdjCCHRwM1aVV0gz5PtYQDHkO u8gc7j8gR3OU+7NIr+BHUjdpV8xTlfGltaVGLmS6DDAXeX9kquydgU0Y0GGgKvjLQiw8 GEQptOlTRqZk9x9bX2Y6Er0qTK1XFnEdDRykxVVYv6xhckqFQxA5UvTk6HStBHz2dYj3 Q9xX2q0Ebo4VwwcdW3FQDSDog2j4qh9O04/sUs+32F/EQkhHiom57uVysiW+ejzNsMYk hJkw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=ElFC2ehhELeEvC6JJ4a7k/iXjRXopkrfts3ldeJEX1E=; fh=dtGru4wMzkfh5N4HjO7hjgtPmzQheD7FyA8qGNiECvw=; b=itdSibEr2civH+umByRk34tXv5k2oh25il+PnrAcGpAo6D33WzqDovssfUDWRuN55g Nyd+FxVi8MltVEsUfFzRAXndDhn6B/wunNHDPG5HJs7AfPDobQcGfYGw69ALXWnua3XF O4Qa4JcysMxIj3rGpEAGH6zjMTaSbMdYmkKC1rr/HN9zI+UgrG6RLPJu/tFOw1U9Za7F /qY26heUW7XHV4t4B5Vgmraq1TZ3rCUnM8W0xDi8TANV9aOmdPcF4KJc0omt3AriNfZr 8r33dp/Kz9DnfnUeH+uihNuI1mfyeNXGFn8iuw4J0UUe6StVT1PxLu77zb1/aQR7lgIw /Urg== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=DmUyRlQ9; arc=pass (i=1 dkim=pass dkdomain=intel.com dmarc=pass fromdomain=linux.intel.com); spf=pass (google.com: domain of linux-kernel+bounces-43787-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43787-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id t36-20020a056402242400b0055efb4604fdsi1962837eda.676.2024.01.30.00.11.20 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 30 Jan 2024 00:11:20 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-43787-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=DmUyRlQ9; arc=pass (i=1 dkim=pass dkdomain=intel.com dmarc=pass fromdomain=linux.intel.com); spf=pass (google.com: domain of linux-kernel+bounces-43787-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43787-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 8921E1F260FD for ; Tue, 30 Jan 2024 02:12:56 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 07F2254F92; Tue, 30 Jan 2024 02:09:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="DmUyRlQ9" Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6DC683716E; Tue, 30 Jan 2024 02:09:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.10 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706580586; cv=none; b=oDTk0+GYdQmfhH5AIWq4lHNYZN0eWfzsFpDxThOifLV3WUWPZ4u20ju+IzLAJPbQMEUiDBO/p0DcLGQ3LRyGeQrm3FHARxUp28PATqmpB2G7ZF/bAtHkBrREmjkVw+3qNc3UENy7uq2KDSTZ6wDzOILfKCALAQgaRn2MXfOMZaw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706580586; c=relaxed/simple; bh=Il+dSnIOspaqoTr4+vmcB6/3FTgzodB3r6fRZncdtbI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=CK0SO2paeTtuor9BK9xFFKTxCCMKot+1AlKNwfe+xWTnOhxB+EaKlQuIucfu0UhSsfI7iPFA0jq9LrdmYOcPuuoNEKX45Ra6fqDnHxl2v+tEhZbFEnYoTiw8Hgg8J5lZUp+2/w1PMavU1grGUpGrv+MqohAWGWVjCZGP00jQrUQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=DmUyRlQ9; arc=none smtp.client-ip=198.175.65.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706580584; x=1738116584; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Il+dSnIOspaqoTr4+vmcB6/3FTgzodB3r6fRZncdtbI=; b=DmUyRlQ9SVyWZF04VJ5BtEgjqOlxfdISu8Z0rt/ZUXa0ydPhmPVgHpGv JUrGJi7AlufpbTzWJqSnBkouELjGXu8hZxEif25ycg4L8wt/LaYm2/Kho c0bokJrsYoSm2XkkJVrxVcED3dco9eAdmIXx3jdM74nFyN5qCERYpI/08 nmc9n2T8ux5LijXqZ7LqlcT0v8R1u65J77Xy7ElglfxslDApjVVmChXhX 2DLPuDWdyWM2Fx8rRZdCpwH4y5FJdyXVpU3QOWTWFIZvAmMAPZ9Aa1ttT kt3X0XwznfqHH9bE7lQDUKzL2VnQS7ZqZi0w+vWD2u084G1JjgJpmFd0Y w==; X-IronPort-AV: E=McAfee;i="6600,9927,10968"; a="16530950" X-IronPort-AV: E=Sophos;i="6.05,707,1701158400"; d="scan'208";a="16530950" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Jan 2024 18:09:40 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10968"; a="822042333" X-IronPort-AV: E=Sophos;i="6.05,707,1701158400"; d="scan'208";a="822042333" Received: from b4969161e530.jf.intel.com ([10.165.56.46]) by orsmga001.jf.intel.com with ESMTP; 29 Jan 2024 18:09:39 -0800 From: Haitao Huang To: jarkko@kernel.org, dave.hansen@linux.intel.com, tj@kernel.org, mkoutny@suse.com, linux-kernel@vger.kernel.org, linux-sgx@vger.kernel.org, x86@kernel.org, cgroups@vger.kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, sohil.mehta@intel.com Cc: zhiquan1.li@intel.com, kristen@linux.intel.com, seanjc@google.com, zhanb@microsoft.com, anakrish@microsoft.com, mikko.ylinen@linux.intel.com, yangjie@microsoft.com, chrisyan@microsoft.com Subject: [PATCH v8 05/15] x86/sgx: Add sgx_epc_lru_list to encapsulate LRU list Date: Mon, 29 Jan 2024 18:09:28 -0800 Message-Id: <20240130020938.10025-6-haitao.huang@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240130020938.10025-1-haitao.huang@linux.intel.com> References: <20240130020938.10025-1-haitao.huang@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789502193030143292 X-GMAIL-MSGID: 1789502193030143292 From: Sean Christopherson Introduce a data structure to wrap the existing reclaimable list and its spinlock. Each cgroup later will have one instance of this structure to track EPC pages allocated for processes associated with the same cgroup. Just like the global SGX reclaimer (ksgxd), an EPC cgroup reclaims pages from the reclaimable list in this structure when its usage reaches near its limit. Use this structure to encapsulate the LRU list and its lock used by the global reclaimer. Signed-off-by: Sean Christopherson Co-developed-by: Kristen Carlson Accardi Signed-off-by: Kristen Carlson Accardi Co-developed-by: Haitao Huang Signed-off-by: Haitao Huang Cc: Sean Christopherson Reviewed-by: Jarkko Sakkinen --- V6: - removed introduction to unreclaimables in commit message. V4: - Removed unneeded comments for the spinlock and the non-reclaimables. (Kai, Jarkko) - Revised the commit to add introduction comments for unreclaimables and multiple LRU lists.(Kai) - Reordered the patches: delay all changes for unreclaimables to later, and this one becomes the first change in the SGX subsystem. V3: - Removed the helper functions and revised commit messages. --- arch/x86/kernel/cpu/sgx/main.c | 39 +++++++++++++++++----------------- arch/x86/kernel/cpu/sgx/sgx.h | 15 +++++++++++++ 2 files changed, 35 insertions(+), 19 deletions(-) diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c index c32f18b70c73..912959c7ecc9 100644 --- a/arch/x86/kernel/cpu/sgx/main.c +++ b/arch/x86/kernel/cpu/sgx/main.c @@ -28,10 +28,9 @@ static DEFINE_XARRAY(sgx_epc_address_space); /* * These variables are part of the state of the reclaimer, and must be accessed - * with sgx_reclaimer_lock acquired. + * with sgx_global_lru.lock acquired. */ -static LIST_HEAD(sgx_active_page_list); -static DEFINE_SPINLOCK(sgx_reclaimer_lock); +static struct sgx_epc_lru_list sgx_global_lru; static atomic_long_t sgx_nr_free_pages = ATOMIC_LONG_INIT(0); @@ -306,13 +305,13 @@ static void sgx_reclaim_pages(void) int ret; int i; - spin_lock(&sgx_reclaimer_lock); + spin_lock(&sgx_global_lru.lock); for (i = 0; i < SGX_NR_TO_SCAN; i++) { - if (list_empty(&sgx_active_page_list)) + epc_page = list_first_entry_or_null(&sgx_global_lru.reclaimable, + struct sgx_epc_page, list); + if (!epc_page) break; - epc_page = list_first_entry(&sgx_active_page_list, - struct sgx_epc_page, list); list_del_init(&epc_page->list); encl_page = epc_page->owner; @@ -324,7 +323,7 @@ static void sgx_reclaim_pages(void) */ epc_page->flags &= ~SGX_EPC_PAGE_RECLAIMER_TRACKED; } - spin_unlock(&sgx_reclaimer_lock); + spin_unlock(&sgx_global_lru.lock); for (i = 0; i < cnt; i++) { epc_page = chunk[i]; @@ -347,9 +346,9 @@ static void sgx_reclaim_pages(void) continue; skip: - spin_lock(&sgx_reclaimer_lock); - list_add_tail(&epc_page->list, &sgx_active_page_list); - spin_unlock(&sgx_reclaimer_lock); + spin_lock(&sgx_global_lru.lock); + list_add_tail(&epc_page->list, &sgx_global_lru.reclaimable); + spin_unlock(&sgx_global_lru.lock); kref_put(&encl_page->encl->refcount, sgx_encl_release); @@ -380,7 +379,7 @@ static void sgx_reclaim_pages(void) static bool sgx_should_reclaim(unsigned long watermark) { return atomic_long_read(&sgx_nr_free_pages) < watermark && - !list_empty(&sgx_active_page_list); + !list_empty(&sgx_global_lru.reclaimable); } /* @@ -432,6 +431,8 @@ static bool __init sgx_page_reclaimer_init(void) ksgxd_tsk = tsk; + sgx_lru_init(&sgx_global_lru); + return true; } @@ -507,10 +508,10 @@ struct sgx_epc_page *__sgx_alloc_epc_page(void) */ void sgx_mark_page_reclaimable(struct sgx_epc_page *page) { - spin_lock(&sgx_reclaimer_lock); + spin_lock(&sgx_global_lru.lock); page->flags |= SGX_EPC_PAGE_RECLAIMER_TRACKED; - list_add_tail(&page->list, &sgx_active_page_list); - spin_unlock(&sgx_reclaimer_lock); + list_add_tail(&page->list, &sgx_global_lru.reclaimable); + spin_unlock(&sgx_global_lru.lock); } /** @@ -525,18 +526,18 @@ void sgx_mark_page_reclaimable(struct sgx_epc_page *page) */ int sgx_unmark_page_reclaimable(struct sgx_epc_page *page) { - spin_lock(&sgx_reclaimer_lock); + spin_lock(&sgx_global_lru.lock); if (page->flags & SGX_EPC_PAGE_RECLAIMER_TRACKED) { /* The page is being reclaimed. */ if (list_empty(&page->list)) { - spin_unlock(&sgx_reclaimer_lock); + spin_unlock(&sgx_global_lru.lock); return -EBUSY; } list_del(&page->list); page->flags &= ~SGX_EPC_PAGE_RECLAIMER_TRACKED; } - spin_unlock(&sgx_reclaimer_lock); + spin_unlock(&sgx_global_lru.lock); return 0; } @@ -578,7 +579,7 @@ struct sgx_epc_page *sgx_alloc_epc_page(void *owner, bool reclaim) break; } - if (list_empty(&sgx_active_page_list)) { + if (list_empty(&sgx_global_lru.reclaimable)) { page = ERR_PTR(-ENOMEM); break; } diff --git a/arch/x86/kernel/cpu/sgx/sgx.h b/arch/x86/kernel/cpu/sgx/sgx.h index a898d86dead0..0e99e9ae3a67 100644 --- a/arch/x86/kernel/cpu/sgx/sgx.h +++ b/arch/x86/kernel/cpu/sgx/sgx.h @@ -88,6 +88,21 @@ static inline void *sgx_get_epc_virt_addr(struct sgx_epc_page *page) return section->virt_addr + index * PAGE_SIZE; } +/* + * Contains EPC pages tracked by the global reclaimer (ksgxd) or an EPC + * cgroup. + */ +struct sgx_epc_lru_list { + spinlock_t lock; + struct list_head reclaimable; +}; + +static inline void sgx_lru_init(struct sgx_epc_lru_list *lru) +{ + spin_lock_init(&lru->lock); + INIT_LIST_HEAD(&lru->reclaimable); +} + struct sgx_epc_page *__sgx_alloc_epc_page(void); void sgx_free_epc_page(struct sgx_epc_page *page);