From patchwork Fri Dec 2 18:36:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kristen Carlson Accardi X-Patchwork-Id: 29067 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp1016294wrr; Fri, 2 Dec 2022 10:44:10 -0800 (PST) X-Google-Smtp-Source: AA0mqf7J8y8aXMvavvmYsV5C+OshwJqpgfUB09dPAlYYXCgrfy4I5VE2wXS6gNci5r+3TG6vgoX+ X-Received: by 2002:aa7:8e54:0:b0:574:fddf:9946 with SMTP id d20-20020aa78e54000000b00574fddf9946mr30511117pfr.79.1670006650522; Fri, 02 Dec 2022 10:44:10 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670006650; cv=none; d=google.com; s=arc-20160816; b=cWKDoZBmU6x57p62I00RRB0Z4nSFsdnUOhQT9EvOXlY8Lg6noJmlGIeMuxexTnOiOp muU9cXBANUgeolkBLDc5ODl3wfkql6VAAwtm5Wd6a8gtjW6FrVxaV9HqSuxjRwsU5hft CzIbkdGcwhcZInuCEAANHf98fBfZkfNqymJv2AYqC/i+eqThCFRNbBbei/twx18MYL2U 6YBcLt5vNDpALNlj+towNoWRTkmsRygfyoE2uu9Trl9ES/9du4bkl2rjrfi2vQJBgNt9 fQf1VOI/V1C/N0HXYQsoRXdFcOS35LgyyROYPAwyZPAsgO9xyQbjZRd+rzm3GKFI9Vcf +mnQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=mIZFKrsrQcPoditvd0MGS/zSv9NQfildL2QqFktbnjc=; b=qYxASy/aE0vxyynLXV1jJUXBcvxSbvZSuWE3mn0g9ugTjo5aUaVGI/toZe/t84WYvR 4chtRjAAEiVmaenBPO7W8TK+LTFzBQVpRBTTubGWuv6043JMDvcgH5MXg3qEaqVXcb46 mijehy956X/XhbNzyHeIe57LNJDoAkT3SfuCIx8SmalSvhOdIQcW94zDWIn6zHxPZksN YrEV4mz4lF40bOIswkeNXJ0CPtMRKjUI/v6J3Py6hR0GHcWRAH19ql13pvK4g5srUFr/ BohCl0RpU1+UGVAf1OGadzDCVuokT8QFhxEBGZzW2iEXeu+8vLoV3EpGUE4QLLqPE3LF rDfg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=f9t+qXj6; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 1-20020a630601000000b0045b22d763a3si8732919pgg.483.2022.12.02.10.43.57; Fri, 02 Dec 2022 10:44:10 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=f9t+qXj6; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234490AbiLBSiS (ORCPT + 99 others); Fri, 2 Dec 2022 13:38:18 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45432 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234563AbiLBSht (ORCPT ); Fri, 2 Dec 2022 13:37:49 -0500 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CC26FEBC9E; Fri, 2 Dec 2022 10:37:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1670006244; x=1701542244; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=r4FfaMTO7MJPJ0OHQyW69TD4X8I66ysGOSc1/fqmusM=; b=f9t+qXj6fIxfKIXvNV2C2mxUBRCHfJhXKY+oyEQB/gLa37IA7oOFyXHA L+chKoy0S52DrdDRUArO9tVRfK3EpJgCZnbq/YzJk6TaBOqNYuqtChca9 Yyko7JIC5Ehpwz7eIZlnyeZ783EDb9tUUHVa7+olyQsXG6UktC3EdlLp4 l+c+1lZtqjSP3egM9fjibOWL3hBAcOT/ND0Lcwlzg0jPVFDj9y65zRoJu IBmJyqFsIdZfxXwJK4ayP26BfDPiOOzUVPAN1wsvA6Elq1Swal9MMrace 943JvaJKnYo0hFHBUPo1v6o6TGbxGsi2Ni8mYK6m9n21CV75EELz0BfJ/ w==; X-IronPort-AV: E=McAfee;i="6500,9779,10549"; a="314724609" X-IronPort-AV: E=Sophos;i="5.96,213,1665471600"; d="scan'208";a="314724609" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Dec 2022 10:37:24 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10549"; a="713717500" X-IronPort-AV: E=Sophos;i="5.96,213,1665471600"; d="scan'208";a="713717500" Received: from kcaskeyx-mobl1.amr.corp.intel.com (HELO kcaccard-desk.amr.corp.intel.com) ([10.251.1.207]) by fmsmga004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Dec 2022 10:37:22 -0800 From: Kristen Carlson Accardi To: jarkko@kernel.org, dave.hansen@linux.intel.com, tj@kernel.org, linux-kernel@vger.kernel.org, linux-sgx@vger.kernel.org, cgroups@vger.kernel.org, Thomas Gleixner , Ingo Molnar , Borislav Petkov , x86@kernel.org, "H. Peter Anvin" Cc: zhiquan1.li@intel.com, Kristen Carlson Accardi , Sean Christopherson Subject: [PATCH v2 10/18] x86/sgx: Add option to ignore age of page during EPC reclaim Date: Fri, 2 Dec 2022 10:36:46 -0800 Message-Id: <20221202183655.3767674-11-kristen@linux.intel.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221202183655.3767674-1-kristen@linux.intel.com> References: <20221202183655.3767674-1-kristen@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751128893709649685?= X-GMAIL-MSGID: =?utf-8?q?1751128893709649685?= From: Sean Christopherson Add a flag to sgx_reclaim_pages() to instruct it to ignore the age of page, i.e. reclaim the page even if it's young. The EPC cgroup will use the flag to enforce its limits by draining the reclaimable lists before resorting to other measures, e.g. forcefully reclaimable "unreclaimable" pages by killing enclaves. Signed-off-by: Sean Christopherson Signed-off-by: Kristen Carlson Accardi Cc: Sean Christopherson --- arch/x86/kernel/cpu/sgx/main.c | 46 +++++++++++++++++++++------------- 1 file changed, 29 insertions(+), 17 deletions(-) diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c index a4a65eadfb79..db96483e2e74 100644 --- a/arch/x86/kernel/cpu/sgx/main.c +++ b/arch/x86/kernel/cpu/sgx/main.c @@ -31,6 +31,10 @@ static DEFINE_XARRAY(sgx_epc_address_space); * with sgx_global_lru.lock acquired. */ static struct sgx_epc_lru_lists sgx_global_lru; +static inline struct sgx_epc_lru_lists *sgx_lru_lists(struct sgx_epc_page *epc_page) +{ + return &sgx_global_lru; +} static atomic_long_t sgx_nr_free_pages = ATOMIC_LONG_INIT(0); @@ -278,6 +282,7 @@ static void sgx_reclaimer_write(struct sgx_epc_page *epc_page, /** * sgx_reclaim_pages() - Reclaim EPC pages from the consumers * @nr_to_scan: Number of EPC pages to scan for reclaim + * @ignore_age: Reclaim a page even if it is young * * Take a fixed number of pages from the head of the active page pool and * reclaim them to the enclave's private shmem files. Skip the pages, which have @@ -291,11 +296,12 @@ static void sgx_reclaimer_write(struct sgx_epc_page *epc_page, * problematic as it would increase the lock contention too much, which would * halt forward progress. */ -static int __sgx_reclaim_pages(int nr_to_scan) +static int __sgx_reclaim_pages(int nr_to_scan, bool ignore_age) { struct sgx_backing backing[SGX_MAX_NR_TO_RECLAIM]; struct sgx_epc_page *epc_page, *tmp; struct sgx_encl_page *encl_page; + struct sgx_epc_lru_lists *lru; pgoff_t page_index; LIST_HEAD(iso); int ret; @@ -332,7 +338,8 @@ static int __sgx_reclaim_pages(int nr_to_scan) list_for_each_entry_safe(epc_page, tmp, &iso, list) { encl_page = epc_page->encl_owner; - if (i == SGX_MAX_NR_TO_RECLAIM || !sgx_reclaimer_age(epc_page)) + if (i == SGX_MAX_NR_TO_RECLAIM || + (!ignore_age && !sgx_reclaimer_age(epc_page))) goto skip; page_index = PFN_DOWN(encl_page->desc - encl_page->encl->base); @@ -350,10 +357,11 @@ static int __sgx_reclaim_pages(int nr_to_scan) continue; skip: - spin_lock(&sgx_global_lru.lock); + lru = sgx_lru_lists(epc_page); + spin_lock(&lru->lock); epc_page->flags &= ~SGX_EPC_PAGE_RECLAIM_IN_PROGRESS; - sgx_epc_move_reclaimable(&sgx_global_lru, epc_page); - spin_unlock(&sgx_global_lru.lock); + sgx_epc_move_reclaimable(lru, epc_page); + spin_unlock(&lru->lock); kref_put(&encl_page->encl->refcount, sgx_encl_release); } @@ -377,11 +385,11 @@ static int __sgx_reclaim_pages(int nr_to_scan) return i; } -static int sgx_reclaim_pages(int nr_to_scan) +static int sgx_reclaim_pages(int nr_to_scan, bool ignore_age) { int ret; - ret = __sgx_reclaim_pages(nr_to_scan); + ret = __sgx_reclaim_pages(nr_to_scan, ignore_age); cond_resched(); return ret; } @@ -402,7 +410,7 @@ static bool sgx_should_reclaim(unsigned long watermark) void sgx_reclaim_direct(void) { if (sgx_should_reclaim(SGX_NR_LOW_PAGES)) - __sgx_reclaim_pages(SGX_NR_TO_SCAN); + __sgx_reclaim_pages(SGX_NR_TO_SCAN, false); } static int ksgxd(void *p) @@ -428,7 +436,7 @@ static int ksgxd(void *p) sgx_should_reclaim(SGX_NR_HIGH_PAGES)); if (sgx_should_reclaim(SGX_NR_HIGH_PAGES)) - sgx_reclaim_pages(SGX_NR_TO_SCAN); + sgx_reclaim_pages(SGX_NR_TO_SCAN, false); } return 0; @@ -522,15 +530,17 @@ struct sgx_epc_page *__sgx_alloc_epc_page(void) */ void sgx_record_epc_page(struct sgx_epc_page *page, unsigned long flags) { - spin_lock(&sgx_global_lru.lock); + struct sgx_epc_lru_lists *lru = sgx_lru_lists(page); + + spin_lock(&lru->lock); WARN_ON(page->flags & (SGX_EPC_PAGE_RECLAIMER_TRACKED | SGX_EPC_PAGE_RECLAIM_IN_PROGRESS)); page->flags |= flags; if (flags & SGX_EPC_PAGE_RECLAIMER_TRACKED) - sgx_epc_push_reclaimable(&sgx_global_lru, page); + sgx_epc_push_reclaimable(lru, page); else - sgx_epc_push_unreclaimable(&sgx_global_lru, page); - spin_unlock(&sgx_global_lru.lock); + sgx_epc_push_unreclaimable(lru, page); + spin_unlock(&lru->lock); } /** @@ -545,14 +555,16 @@ void sgx_record_epc_page(struct sgx_epc_page *page, unsigned long flags) */ int sgx_drop_epc_page(struct sgx_epc_page *page) { - spin_lock(&sgx_global_lru.lock); + struct sgx_epc_lru_lists *lru = sgx_lru_lists(page); + + spin_lock(&lru->lock); if ((page->flags & SGX_EPC_PAGE_RECLAIMER_TRACKED) && (page->flags & SGX_EPC_PAGE_RECLAIM_IN_PROGRESS)) { - spin_unlock(&sgx_global_lru.lock); + spin_unlock(&lru->lock); return -EBUSY; } list_del(&page->list); - spin_unlock(&sgx_global_lru.lock); + spin_unlock(&lru->lock); page->flags &= ~(SGX_EPC_PAGE_RECLAIMER_TRACKED | SGX_EPC_PAGE_RECLAIM_IN_PROGRESS | @@ -607,7 +619,7 @@ struct sgx_epc_page *sgx_alloc_epc_page(void *owner, bool reclaim) break; } - sgx_reclaim_pages(SGX_NR_TO_SCAN); + sgx_reclaim_pages(SGX_NR_TO_SCAN, false); } if (sgx_should_reclaim(SGX_NR_LOW_PAGES))