From patchwork Sat Sep 23 03:06:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Haitao Huang X-Patchwork-Id: 143840 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:cae8:0:b0:403:3b70:6f57 with SMTP id r8csp41942vqu; Sat, 23 Sep 2023 00:52:28 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEmsI1ZDVK5agVehMTgfcy9dDQ+/V1oEYy/ka969B3w44BhOvs1NDQJThWFuVsjrv4SEXJi X-Received: by 2002:a05:6e02:152f:b0:34f:d822:baab with SMTP id i15-20020a056e02152f00b0034fd822baabmr2147486ilu.12.1695455547963; Sat, 23 Sep 2023 00:52:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695455547; cv=none; d=google.com; s=arc-20160816; b=ySNhCBZWN2LYonmILTP0YKnaXIMu2W/oWmtQUZ37bUhO5Eswo7mSS2+gDR33ppOECD vsU24KxlZ35UAn+vQICSPytzVOBuLXbFaAOrgF6EuRtcRob2a4DGpf5prWS9eaNMZAZe Pc7l5rLgTBzqvd1nRm9qQBr3xnX8bgrrrf/M38Zc1CNSBVbRRtOUc48uFumLPhUoMiHl Qww+KhmMaEq1rLpn4y0IOEiLzy8HAr5ph7z0Gn5e27Do1J7bWcKB896LX6Y8qOoOA3no JwWN1jVJSWGHTA2bcU2ZLLKELzfFoZ1qPceJ6bpX4y9HfXsG7Rjtp9GnlGyHkilKnHOT FcBw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=8xNt2mBCODQ/BN42IHkMkLmmPDVQ62QgiBUGRzocNjg=; fh=j8PE345l5Ydlo3KwK7JeWnjqRgjiq4AteUoOZeOwa0I=; b=wEprssmBJ6IbPUoTgzS9jhKzuCgPvLkfDG05H42bQUz5Om9+KL3NB2rReggLfaBXWk HBtO+mg3cuinN9g6rsZaODeNgW4FU5y6RhFAUPPs7VIAWCQOOWBEHgPiqXKgyDNnGsbH DLYm9GJKLa605oQ+lxEXx89FNm1KkAaDQLfjC6HiX+BKpDd2VA0YAnKuXo/ThRNj3TLD XoLz60K3FyFLng7Jn3WdxEIuav9s72fLf9ss0bpziGKw1MlEoEERDqCqOuyjGqgaf5Xe k6yKR35HeJKT4GNQWo/R/GeTFjCNhNL55lM17JozbjyOAdZc5Dwb0WwMabUrYmQjU+gt pcqA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="m0yMOlB/"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from fry.vger.email (fry.vger.email. [23.128.96.38]) by mx.google.com with ESMTPS id r203-20020a632bd4000000b00573fc6a17dbsi5552483pgr.435.2023.09.23.00.52.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 23 Sep 2023 00:52:27 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) client-ip=23.128.96.38; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="m0yMOlB/"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by fry.vger.email (Postfix) with ESMTP id AD05883AD1FE; Fri, 22 Sep 2023 20:08:36 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at fry.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229899AbjIWDI3 (ORCPT + 28 others); Fri, 22 Sep 2023 23:08:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58460 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230418AbjIWDHj (ORCPT ); Fri, 22 Sep 2023 23:07:39 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8F13CCF6; Fri, 22 Sep 2023 20:07:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1695438434; x=1726974434; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=dZ4dr09FZCb8PLkHgNm+LEZeNQ2boJ7dLsvuHqxpIsY=; b=m0yMOlB/eV24uty8sh2g+qB/nyeDiDeJdkwLRFqUw0uvwLH2QOf+fsnh jaiFFfrpz4Gk3ZA9IX4AgVF3HMhPR33Pz6oGdqugzO8LZruHnsd3nGyxg fU9KkVbDW6alVgIx6J7Na+Q7IUUHhTESaiksasagIoSwX0MPCelEywxcT qcyZ4xhJgNgBO36vBQ3Fgh4PfHCZwAk8pJtz4679vzGRjp2pMIueFm408 /9/3IAhIYAHXlTOdMlvrpYN5lXx7dgXFmsO1lHDh1RGOnQKuCv22NYvvJ T9Op0P0IkPdMbC6WWV80yUn93aB4Co9Qemxx3nZ+Qi7uF0l0u/HhWbpM0 w==; X-IronPort-AV: E=McAfee;i="6600,9927,10841"; a="447466868" X-IronPort-AV: E=Sophos;i="6.03,169,1694761200"; d="scan'208";a="447466868" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Sep 2023 20:07:11 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10841"; a="891048591" X-IronPort-AV: E=Sophos;i="6.03,169,1694761200"; d="scan'208";a="891048591" Received: from b4969161e530.jf.intel.com ([10.165.56.46]) by fmsmga001.fm.intel.com with ESMTP; 22 Sep 2023 20:06:15 -0700 From: Haitao Huang To: jarkko@kernel.org, dave.hansen@linux.intel.com, tj@kernel.org, linux-kernel@vger.kernel.org, linux-sgx@vger.kernel.org, x86@kernel.org, cgroups@vger.kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, sohil.mehta@intel.com Cc: zhiquan1.li@intel.com, kristen@linux.intel.com, seanjc@google.com, zhanb@microsoft.com, anakrish@microsoft.com, mikko.ylinen@linux.intel.com, yangjie@microsoft.com Subject: [PATCH v5 18/18] selftests/sgx: Add scripts for EPC cgroup testing Date: Fri, 22 Sep 2023 20:06:57 -0700 Message-Id: <20230923030657.16148-19-haitao.huang@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230923030657.16148-1-haitao.huang@linux.intel.com> References: <20230923030657.16148-1-haitao.huang@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=2.8 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, RCVD_IN_SBL_CSS,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on fry.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (fry.vger.email [0.0.0.0]); Fri, 22 Sep 2023 20:08:36 -0700 (PDT) X-Spam-Level: ** X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777813997075648265 X-GMAIL-MSGID: 1777813997075648265 The scripts rely on cgroup-tools package from libcgroup [1]. To run selftests for epc cgroup: sudo ./run_epc_cg_selftests.sh With different cgroups, the script starts one or multiple concurrent SGX selftests, each to run one unclobbered_vdso_oversubscribed test. Each of such test tries to load an enclave of size equal to the EPC capacity available on the platform. The script checks results against the expectation set for each cgroup and report success or failure. The script creates 3 different cgroups at the beginning with following expectations: 1) SMALL - intentionally small enough to fail the test loading an enclave of size equal to the capacity. 2) LARGE - large enough to run up to 4 concurrent tests but fail some if more than 4 concurrent tests are run. The script start 4 expecting at least one test to pass, and then starts 5 expecting at least one test to fail. 3) LARGER - limit is the same as capacity, large enough to run lots of concurrent tests. The script starts 10 of them and expects all pass. To watch misc cgroup 'current' changes during testing, run this in a separate terminal: ./watch_misc_for_tests.sh current [1] https://github.com/libcgroup/libcgroup/blob/main/README Signed-off-by: Haitao Huang --- V5: - Added script with automatic results checking, remove the interactive script. - The script can run independent from the series below. V4: Note: Need to apply on top of this series previously reviewed: https://lore.kernel.org/linux-sgx/20220905020411.17290-1-jarkko@kernel.org/ --- .../selftests/sgx/run_epc_cg_selftests.sh | 147 ++++++++++++++++++ .../selftests/sgx/watch_misc_for_tests.sh | 13 ++ 2 files changed, 160 insertions(+) create mode 100755 tools/testing/selftests/sgx/run_epc_cg_selftests.sh create mode 100755 tools/testing/selftests/sgx/watch_misc_for_tests.sh diff --git a/tools/testing/selftests/sgx/run_epc_cg_selftests.sh b/tools/testing/selftests/sgx/run_epc_cg_selftests.sh new file mode 100755 index 000000000000..410c97ee6e18 --- /dev/null +++ b/tools/testing/selftests/sgx/run_epc_cg_selftests.sh @@ -0,0 +1,147 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright(c) 2023 Intel Corporation. + +TEST_ROOT_CG=selftest +cgcreate -g misc:$TEST_ROOT_CG +if [ $? -ne 0 ]; then + echo "# Please make sure cgroup-tools is installed, and misc cgroup is mounted." + exit 1 +fi +TEST_CG_SUB1=$TEST_ROOT_CG/test1 +TEST_CG_SUB2=$TEST_ROOT_CG/test2 +TEST_CG_SUB3=$TEST_ROOT_CG/test1/test3 +TEST_CG_SUB4=$TEST_ROOT_CG/test4 + +cgcreate -g misc:$TEST_CG_SUB1 +cgcreate -g misc:$TEST_CG_SUB2 +cgcreate -g misc:$TEST_CG_SUB3 +cgcreate -g misc:$TEST_CG_SUB4 + +# Default to V2 +CG_ROOT=/sys/fs/cgroup +if [ ! -d "/sys/fs/cgroup/misc" ]; then + echo "# cgroup V2 is in use." +else + echo "# cgroup V1 is in use." + CG_ROOT=/sys/fs/cgroup/misc +fi + +CAPACITY=$(grep "sgx_epc" "$CG_ROOT/misc.capacity" | awk '{print $2}') +# This is below number of VA pages needed for enclave of capacity size. So +# should fail oversubscribed cases +SMALL=$(( CAPACITY / 512 )) + +# At least load one enclave of capacity size successfully, maybe up to 4. +# But some may fail if we run more than 4 concurrent enclaves of capacity size. +LARGE=$(( SMALL * 4 )) + +# Load lots of enclaves +LARGER=$CAPACITY +echo "# Setting up limits." +echo "sgx_epc $SMALL" | tee $CG_ROOT/$TEST_CG_SUB1/misc.max +echo "sgx_epc $LARGE" | tee $CG_ROOT/$TEST_CG_SUB2/misc.max +echo "sgx_epc $LARGER" | tee $CG_ROOT/$TEST_CG_SUB4/misc.max + +timestamp=$(date +%Y%m%d_%H%M%S) + +test_cmd="./test_sgx -t unclobbered_vdso_oversubscribed" + +echo "# Start unclobbered_vdso_oversubscribed with SMALL limit, expecting failure..." +# Always use leaf node of misc cgroups so it works for both v1 and v2 +# these may fail on OOM +cgexec -g misc:$TEST_CG_SUB3 $test_cmd >cgtest_small_$timestamp.log 2>&1 +if [[ $? -eq 0 ]]; then + echo "# Fail on SMALL limit, not expecting any test passes." + cgdelete -r -g misc:$TEST_ROOT_CG + exit 1 +else + echo "# Test failed as expected." +fi + +echo "# PASSED SMALL limit." + +echo "# Start 4 concurrent unclobbered_vdso_oversubscribed tests with LARGE limit, expecting at least one success...." +pids=() +for i in {1..4}; do + ( + cgexec -g misc:$TEST_CG_SUB2 $test_cmd >cgtest_large_positive_$timestamp.$i.log 2>&1 + ) & + pids+=($!) +done + +any_success=0 +for pid in "${pids[@]}"; do + wait "$pid" + status=$? + if [[ $status -eq 0 ]]; then + any_success=1 + echo "# Process $pid returned successfully." + fi +done + +if [[ $any_success -eq 0 ]]; then + echo "# Failed on LARGE limit positive testing, no test passes." + cgdelete -r -g misc:$TEST_ROOT_CG + exit 1 +fi + +echo "# PASSED LARGE limit positive testing." + +echo "# Start 5 concurrent unclobbered_vdso_oversubscribed tests with LARGE limit, expecting at least one failure...." +pids=() +for i in {1..5}; do + ( + cgexec -g misc:$TEST_CG_SUB2 $test_cmd >cgtest_large_negative_$timestamp.$i.log 2>&1 + ) & + pids+=($!) +done + +any_failure=0 +for pid in "${pids[@]}"; do + wait "$pid" + status=$? + if [[ $status -ne 0 ]]; then + echo "# Process $pid returned failure." + any_failure=1 + fi +done + +if [[ $any_failure -eq 0 ]]; then + echo "# Failed on LARGE limit negative testing, no test fails." + cgdelete -r -g misc:$TEST_ROOT_CG + exit 1 +fi + +echo "# PASSED LARGE limit negative testing." + +echo "# Start 10 concurrent unclobbered_vdso_oversubscribed tests with LARGER limit, expecting no failure...." +pids=() +for i in {1..10}; do + ( + cgexec -g misc:$TEST_CG_SUB4 $test_cmd >cgtest_larger_$timestamp.$i.log 2>&1 + ) & + pids+=($!) +done + +any_failure=0 +for pid in "${pids[@]}"; do + wait "$pid" + status=$? + if [[ $status -ne 0 ]]; then + echo "# Process $pid returned failure." + any_failure=1 + fi +done + +if [[ $any_failure -ne 0 ]]; then + echo "# Failed on LARGER limit, at least one test fails." + cgdelete -r -g misc:$TEST_ROOT_CG + exit 1 +fi + +echo "# PASSED LARGER limit tests." + +echo "# PASSED ALL cgroup limit tests, cleanup cgroups..." +cgdelete -r -g misc:$TEST_ROOT_CG +echo "# done." diff --git a/tools/testing/selftests/sgx/watch_misc_for_tests.sh b/tools/testing/selftests/sgx/watch_misc_for_tests.sh new file mode 100755 index 000000000000..dbd38f346e7b --- /dev/null +++ b/tools/testing/selftests/sgx/watch_misc_for_tests.sh @@ -0,0 +1,13 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright(c) 2023 Intel Corporation. + +if [ -z "$1" ] + then + echo "No argument supplied, please provide 'max', 'current' or 'events'" + exit 1 +fi + +watch -n 1 "find /sys/fs/cgroup -wholename */test*/misc.$1 -exec sh -c \ + 'echo \"\$1:\"; cat \"\$1\"' _ {} \;" +