From patchwork Fri Jun 9 16:26:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wang, Weilin" X-Patchwork-Id: 10563 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:6358:3046:b0:115:7a1d:dabb with SMTP id p6csp1125276rwl; Fri, 9 Jun 2023 09:40:59 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4D8tWTwZZtgkoSAIj5oLTZyPb8YhPTZdZ8uWfujZLgHNCWYjAGieS1lzv5HkwavrnpfcFa X-Received: by 2002:a17:902:f691:b0:1a6:b971:faf6 with SMTP id l17-20020a170902f69100b001a6b971faf6mr1713123plg.35.1686328858922; Fri, 09 Jun 2023 09:40:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686328858; cv=none; d=google.com; s=arc-20160816; b=NQOaKxnYaaClqffc/V/xPaPbNteYi2jyAQKlUZOWiT1TZsIt9YKki1ix3wG0I4zuWI 61rAzrzro2GLk97bLpfLURM2eXL3RlZYiqfb+sBZM3Q+tJyYWfFivrG0TKsUNc5zZvQe 2cfGxJ0qFCjNV4sPelbw5mSHhUdxffcBnOiZSa8sBZQhsqAHd8eN2JxcHS93ewgynjTl Oy63RmH9+V8YqRZ7BPIy8IRYFAHov+Wv1dBBOKLYA2+D2ZQwsa/ET3DLEqdKGMjeZchA QzUv+P1ayaW069kCtpFeUF0L6BBZ/jUMe3lzV6oGBfl50r6PM3OCPeIicTA714/DDHhu nqxg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=m2avo6caYelmx030UrdjtqZDcaQFBWM6YJyscVXjBSg=; b=VIxGw12scNY8cHeMXKBNfOjFFOW+sFDxdWFHJzEQ0Ote2j1Bd9Yr2wgIOQHjMADDYB lp1I74jp2F6S8MsapLCL0bYeiXz0RcIgnurgoGv5KeYtgkOjmfu3GY6aowAqy7Ny3h2e 4cwVUHp6StYsR7yoD2O/Zrt2T4SM2TLm2kKzLPqE9VhIBfWYT7KFMxxLyQPVGv1RsRC7 8YBiPBBvj6PI4+jPw87DPJ86w8fxSCVIqejtOpZXYyCWwIKaSwoyGadVdsgc7mwqyZua jtec78lzSmXO9oB3eOUc8XR20xZ6kClQxhICE3389NfIfZ03NBNdbo0vFvpmgv2TeW9f 23+A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Qiu0aR19; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id kh12-20020a170903064c00b001ab259969e7si2936816plb.31.2023.06.09.09.40.45; Fri, 09 Jun 2023 09:40:58 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Qiu0aR19; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241801AbjFIQ0j (ORCPT + 99 others); Fri, 9 Jun 2023 12:26:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39846 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242110AbjFIQ0f (ORCPT ); Fri, 9 Jun 2023 12:26:35 -0400 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 939543A97; Fri, 9 Jun 2023 09:26:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1686327993; x=1717863993; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=GtBv/al6+wYc44j7F60+PSlt5v4CWvv4McB3U6J07gk=; b=Qiu0aR19QB+op7RuuskSxSsccqLUgqotPsLpTS5KFjKRl3M8jMjEVvRA UuPNkgUvXYk+TC+2LsY/r0+hw+rkhk1bq+Bdy5rRyy0N82RbjlQOGQdxY 70r8jl1wGqummqD93MIvftm1EW1VE8PeUqEpBY47s5bmB33JDTgMBkHvI 5VsMyXphWmUZjLGqAfHYYs/tTow2Eg4XzHdc4+rQbQ6mKSZTos10bPRCc QTqxKxymtqt3uC7c3ljLkDSqoSzoql8SeGmU82qK89qw3dYdromH7bsYn p0a24Vk8D7NIRK4DoWFdrsrAfIMKS9fnAtcYSLXv6+NH9QBachzYPQwi+ w==; X-IronPort-AV: E=McAfee;i="6600,9927,10736"; a="355123847" X-IronPort-AV: E=Sophos;i="6.00,229,1681196400"; d="scan'208";a="355123847" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Jun 2023 09:26:33 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10736"; a="800269655" X-IronPort-AV: E=Sophos;i="6.00,229,1681196400"; d="scan'208";a="800269655" Received: from a0cec87da3f2.jf.intel.com (HELO worker-node-1.jf.intel.com) ([10.165.55.163]) by FMSMGA003.fm.intel.com with ESMTP; 09 Jun 2023 09:26:32 -0700 From: Weilin Wang To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Jiri Olsa , Namhyung Kim , Adrian Hunter , Ian Rogers , linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Weilin Wang , Kan Liang , Samantha Alt , Perry Taylor , Caleb Biggers Subject: [PATCH v2 0/3] Add metric value validation test Date: Fri, 9 Jun 2023 09:26:22 -0700 Message-Id: <20230609162625.100897-1-weilin.wang@intel.com> X-Mailer: git-send-email 2.39.1 MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768243969698363647?= X-GMAIL-MSGID: =?utf-8?q?1768243969698363647?= This is the second version of metric value validation tests. We made the following changes from v1 to v2: - Add python setting check [Ian] - Skip non-Intel architectures [Ian] - Rename allowlist to skiplist [Ian] v1: https://lore.kernel.org/lkml/20230606202421.2628401-1-weilin.wang@intel.com/ Weilin Wang (3): perf test: Add metric value validation test perf test: Add skip list for metrics known would fail perf test: Rerun failed metrics with longer workload .../tests/shell/lib/perf_metric_validation.py | 574 ++++++++++++++++++ .../lib/perf_metric_validation_rules.json | 398 ++++++++++++ tools/perf/tests/shell/stat_metrics_values.sh | 30 + 3 files changed, 1002 insertions(+) create mode 100644 tools/perf/tests/shell/lib/perf_metric_validation.py create mode 100644 tools/perf/tests/shell/lib/perf_metric_validation_rules.json create mode 100755 tools/perf/tests/shell/stat_metrics_values.sh base-commit: 7cdda6998ee55140e64894e25048df7157344fc9