From patchwork Wed May 31 04:04:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anshuman Khandual X-Patchwork-Id: 101225 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp2630694vqr; Tue, 30 May 2023 21:28:08 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6SHzN17AZFCWSKVy8Wy2NdoJC4r6UziCQ1njRAn+EG0y8T1mvqoE8U0wiSFRT9xZEWPQ6P X-Received: by 2002:a05:6808:3090:b0:395:f4fd:9fe9 with SMTP id bl16-20020a056808309000b00395f4fd9fe9mr3631235oib.50.1685507288155; Tue, 30 May 2023 21:28:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685507288; cv=none; d=google.com; s=arc-20160816; b=hHErH/CgHouMHETje8IYGylAEkSiV8cmeaLznwwb4mRT/tVOioBwhvCQ2Dkn6zyMu1 32Kt7GgGs0L0FbXGSP2T2O+RHRSIcISYcOVpCBHg2PA7a6LlfJs4dY/gi7l9JrryUUGX W9H1UwuY2QOqSMk4TD0PV4O8Td6sahm7XKhbMtQPkiTpPWdCX6Fw4Sre+LcUzBJ8D5vv wWpGqew2pVLYlWCkuMDDnetbT9PdImxPXM/4c6hLNGfPRZjDCqFJFj5GaQgkdk7mBI7d ol+ed9OCnE2KOFlcZYDql2u2c5tmsHeIGrPqRL5fcP8lXMCObghujadrJZz+2h1Xi1IU 0Irg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=qPuXWntLaW802UGQUeuE9f22fbA5mHyRl93Q7mxtCP4=; b=J8f7QnypWch1mGCmbWQ7ABdhhOVifqBvrWFTOQc4WBlHSW5xMDnpOF5+jspqnxejSk wjdL/UjuEWk8cKuHktDp7TT+r1KGhwAnpibQGOSqrYKj8OPyP+IzOGx1N8zMs/GzLCp1 OIEYSh9NdjFtzGjlY+INW11D8Lr0YJKohUmA4g7F2IMVEL5U+xJN7+GDXVH/tvFeCUCP oSFFSBq+Gaf0+hwVLDq4Bibwmmu9B4c+gDRz4826L/qqMCdDFupDdGDNh0NFw0UDp9kh wA7FDTRQbhlguzmH832XJFrmTUOB+LemdqR06Kyy4/k0Vo8gpg8e0kOIqqVlB5DmWYjf 3weQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id k64-20020a638443000000b0052c6200e5e1si273624pgd.649.2023.05.30.21.27.56; Tue, 30 May 2023 21:28:08 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234277AbjEaEG5 (ORCPT + 99 others); Wed, 31 May 2023 00:06:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55598 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234243AbjEaEGQ (ORCPT ); Wed, 31 May 2023 00:06:16 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 58B891B1; Tue, 30 May 2023 21:05:50 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6C14E165C; Tue, 30 May 2023 21:06:35 -0700 (PDT) Received: from a077893.arm.com (unknown [10.163.73.163]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 9795D3F6C4; Tue, 30 May 2023 21:05:44 -0700 (PDT) From: Anshuman Khandual To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com Cc: Anshuman Khandual , Mark Brown , James Clark , Rob Herring , Marc Zyngier , Suzuki Poulose , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , linux-perf-users@vger.kernel.org Subject: [PATCH V11 09/10] arm64/perf: Implement branch records save on task sched out Date: Wed, 31 May 2023 09:34:27 +0530 Message-Id: <20230531040428.501523-10-anshuman.khandual@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230531040428.501523-1-anshuman.khandual@arm.com> References: <20230531040428.501523-1-anshuman.khandual@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767382490195424651?= X-GMAIL-MSGID: =?utf-8?q?1767382490195424651?= This modifies current armv8pmu_sched_task(), to implement a branch records save mechanism via armv8pmu_branch_save() when a task scheds out of a cpu. BRBE is paused and disabled for all exception levels before branch records get captured, which then get concatenated with all existing stored records present in the task context maintaining the contiguity. Although the final length of the concatenated buffer does not exceed implemented BRBE length. Cc: Catalin Marinas Cc: Will Deacon Cc: Mark Rutland Cc: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org Tested-by: James Clark Signed-off-by: Anshuman Khandual --- arch/arm64/include/asm/perf_event.h | 2 ++ drivers/perf/arm_brbe.c | 30 +++++++++++++++++++++++++++++ drivers/perf/arm_pmuv3.c | 14 ++++++++++++-- 3 files changed, 44 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/perf_event.h b/arch/arm64/include/asm/perf_event.h index f071d629c0cf..c81b768cd172 100644 --- a/arch/arm64/include/asm/perf_event.h +++ b/arch/arm64/include/asm/perf_event.h @@ -40,6 +40,7 @@ void armv8pmu_branch_probe(struct arm_pmu *arm_pmu); void armv8pmu_branch_reset(void); int armv8pmu_private_alloc(struct arm_pmu *arm_pmu); void armv8pmu_private_free(struct arm_pmu *arm_pmu); +void armv8pmu_branch_save(struct arm_pmu *arm_pmu, void *ctx); #else static inline void armv8pmu_branch_read(struct pmu_hw_events *cpuc, struct perf_event *event) { @@ -66,6 +67,7 @@ static inline void armv8pmu_branch_probe(struct arm_pmu *arm_pmu) { } static inline void armv8pmu_branch_reset(void) { } static inline int armv8pmu_private_alloc(struct arm_pmu *arm_pmu) { return 0; } static inline void armv8pmu_private_free(struct arm_pmu *arm_pmu) { } +static inline void armv8pmu_branch_save(struct arm_pmu *arm_pmu, void *ctx) { } #endif #endif #endif diff --git a/drivers/perf/arm_brbe.c b/drivers/perf/arm_brbe.c index 759db681d673..0678ebf0a896 100644 --- a/drivers/perf/arm_brbe.c +++ b/drivers/perf/arm_brbe.c @@ -207,6 +207,36 @@ static int stitch_stored_live_entries(struct brbe_regset *stored, return nr_last; } +static int brbe_branch_save(struct brbe_hw_attr *brbe_attr, struct brbe_regset *live) +{ + u64 brbfcr = read_sysreg_s(SYS_BRBFCR_EL1); + int nr_live; + + write_sysreg_s(brbfcr | BRBFCR_EL1_PAUSED, SYS_BRBFCR_EL1); + isb(); + + nr_live = capture_brbe_regset(brbe_attr, live); + + write_sysreg_s(brbfcr & ~BRBFCR_EL1_PAUSED, SYS_BRBFCR_EL1); + isb(); + + return nr_live; +} + +void armv8pmu_branch_save(struct arm_pmu *arm_pmu, void *ctx) +{ + struct brbe_hw_attr *brbe_attr = (struct brbe_hw_attr *)arm_pmu->private; + struct arm64_perf_task_context *task_ctx = ctx; + struct brbe_regset live[BRBE_MAX_ENTRIES]; + int nr_live, nr_store; + + nr_live = brbe_branch_save(brbe_attr, live); + nr_store = task_ctx->nr_brbe_records; + nr_store = stitch_stored_live_entries(task_ctx->store, live, nr_store, + nr_live, brbe_attr->brbe_nr); + task_ctx->nr_brbe_records = nr_store; +} + /* * Generic perf branch filters supported on BRBE * diff --git a/drivers/perf/arm_pmuv3.c b/drivers/perf/arm_pmuv3.c index 29672ff20026..9725a53d6799 100644 --- a/drivers/perf/arm_pmuv3.c +++ b/drivers/perf/arm_pmuv3.c @@ -907,9 +907,19 @@ static int armv8pmu_user_event_idx(struct perf_event *event) static void armv8pmu_sched_task(struct perf_event_pmu_context *pmu_ctx, bool sched_in) { struct arm_pmu *armpmu = to_arm_pmu(pmu_ctx->pmu); + void *task_ctx = pmu_ctx ? pmu_ctx->task_ctx_data : NULL; - if (sched_in && arm_pmu_branch_stack_supported(armpmu)) - armv8pmu_branch_reset(); + if (arm_pmu_branch_stack_supported(armpmu)) { + /* Save branch records in task_ctx on sched out */ + if (task_ctx && !sched_in) { + armv8pmu_branch_save(armpmu, task_ctx); + return; + } + + /* Reset branch records on sched in */ + if (sched_in) + armv8pmu_branch_reset(); + } } /*