From patchwork Thu Jun 15 13:32:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anshuman Khandual X-Patchwork-Id: 108539 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp653592vqr; Thu, 15 Jun 2023 06:58:16 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5AuOsDeE3z9gubCqSKgIGR+2VSYgYhmrE/CFiAeoTq20jdwgyeo5H9tiuP5KIYpk1OQKtb X-Received: by 2002:a05:6830:d2:b0:6ab:bec:592f with SMTP id x18-20020a05683000d200b006ab0bec592fmr16211955oto.17.1686837496480; Thu, 15 Jun 2023 06:58:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686837496; cv=none; d=google.com; s=arc-20160816; b=xnUq7202UwWbhTVSx0j4LsfKrG8+l3myGlmNgc/03p3I7JLvsBACpzFg/cQfE9Iese TlDUF/sKO6/a59PVLl7t3wE5gBIJdpogfKjJWCWGj9l92EZrSFoPb+i2IEpi4A6BEqV8 7FluN5foDy5fGQ8+nX7X/xJMO4VaqEPwMykVi/vvxNwuYW+dIszLYxWC6YnrqT+CFQNT M8fmFqUGMXbER6UmJI/88X7CqEpujJq+eMTJjpyUPhFeApdhOzIgZTpPobA8t/fptXKb mSjeyQ5LRXFS2tgjDBokB37YpF4DY2oSMBKp+XZfd/dsnwe6aMc0a7fwaUqgseReVY9v vo3g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=WNcTBBWzq8q+4Chkv5TPiII9TR9oGxsBG5KN2XU1hvY=; b=likHONpdzwo7fTBOqGhkHp66SNR3T9ojWO84N0x+aroBW7d1KNvMBu6NaHps8gX1Ta /y1JEIXz5DCxHnPWBkckvLEZeKyHtuQCGkefOOSMKoXReLM01Yu4hvW8nQBuFYIax3Ct korI3W7tL62Hzzwi1fpMtWQx4k2POOKLFMtX0O4uarVfaDQspzpQk1AS8blh/BlVNl9R GhMpWtWYZZiHOUPlo9lDjZTSbQBgPcbqdbsmTn34xFno4xwNjPWbSGKb2MngVFmZ+M/s pDU9QHT5k8j+L1SCWZg9YVl87kzVDrE3flvRvfbgchMXpBINcTmw9sxHqFzbUthqwifP RW4Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id z189-20020a6333c6000000b00553361ff1a9si553639pgz.746.2023.06.15.06.58.01; Thu, 15 Jun 2023 06:58:16 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245753AbjFONeb (ORCPT + 99 others); Thu, 15 Jun 2023 09:34:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44812 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344173AbjFONeB (ORCPT ); Thu, 15 Jun 2023 09:34:01 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id B042B2969; Thu, 15 Jun 2023 06:33:43 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7FDEB1FB; Thu, 15 Jun 2023 06:34:27 -0700 (PDT) Received: from a077893.arm.com (unknown [10.163.46.86]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 852AD3F64C; Thu, 15 Jun 2023 06:33:37 -0700 (PDT) From: Anshuman Khandual To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com Cc: Anshuman Khandual , Mark Brown , James Clark , Rob Herring , Marc Zyngier , Suzuki Poulose , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , linux-perf-users@vger.kernel.org Subject: [PATCH V12 09/10] arm64/perf: Implement branch records save on task sched out Date: Thu, 15 Jun 2023 19:02:38 +0530 Message-Id: <20230615133239.442736-10-anshuman.khandual@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230615133239.442736-1-anshuman.khandual@arm.com> References: <20230615133239.442736-1-anshuman.khandual@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768777314711407770?= X-GMAIL-MSGID: =?utf-8?q?1768777314711407770?= This modifies current armv8pmu_sched_task(), to implement a branch records save mechanism via armv8pmu_branch_save() when a task scheds out of a cpu. BRBE is paused and disabled for all exception levels before branch records get captured, which then get concatenated with all existing stored records present in the task context maintaining the contiguity. Although the final length of the concatenated buffer does not exceed implemented BRBE length. Cc: Catalin Marinas Cc: Will Deacon Cc: Mark Rutland Cc: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org Tested-by: James Clark Signed-off-by: Anshuman Khandual Acked-by: Mark Rutland --- arch/arm64/include/asm/perf_event.h | 2 ++ drivers/perf/arm_brbe.c | 30 +++++++++++++++++++++++++++++ drivers/perf/arm_pmuv3.c | 14 ++++++++++++-- 3 files changed, 44 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/perf_event.h b/arch/arm64/include/asm/perf_event.h index b0c12a5882df..36e7dfb466a6 100644 --- a/arch/arm64/include/asm/perf_event.h +++ b/arch/arm64/include/asm/perf_event.h @@ -40,6 +40,7 @@ void armv8pmu_branch_probe(struct arm_pmu *arm_pmu); void armv8pmu_branch_reset(void); int armv8pmu_task_ctx_cache_alloc(struct arm_pmu *arm_pmu); void armv8pmu_task_ctx_cache_free(struct arm_pmu *arm_pmu); +void armv8pmu_branch_save(struct arm_pmu *arm_pmu, void *ctx); #else static inline void armv8pmu_branch_read(struct pmu_hw_events *cpuc, struct perf_event *event) { @@ -66,6 +67,7 @@ static inline void armv8pmu_branch_probe(struct arm_pmu *arm_pmu) { } static inline void armv8pmu_branch_reset(void) { } static inline int armv8pmu_task_ctx_cache_alloc(struct arm_pmu *arm_pmu) { return 0; } static inline void armv8pmu_task_ctx_cache_free(struct arm_pmu *arm_pmu) { } +static inline void armv8pmu_branch_save(struct arm_pmu *arm_pmu, void *ctx) { } #endif #endif #endif diff --git a/drivers/perf/arm_brbe.c b/drivers/perf/arm_brbe.c index f6693699fade..3bb17ced2b1d 100644 --- a/drivers/perf/arm_brbe.c +++ b/drivers/perf/arm_brbe.c @@ -171,6 +171,36 @@ static int stitch_stored_live_entries(struct brbe_regset *stored, return min(nr_live + nr_stored, nr_max); } +static int brbe_branch_save(int nr_hw_entries, struct brbe_regset *live) +{ + u64 brbfcr = read_sysreg_s(SYS_BRBFCR_EL1); + int nr_live; + + write_sysreg_s(brbfcr | BRBFCR_EL1_PAUSED, SYS_BRBFCR_EL1); + isb(); + + nr_live = capture_brbe_regset(nr_hw_entries, live); + + write_sysreg_s(brbfcr & ~BRBFCR_EL1_PAUSED, SYS_BRBFCR_EL1); + isb(); + + return nr_live; +} + +void armv8pmu_branch_save(struct arm_pmu *arm_pmu, void *ctx) +{ + struct arm64_perf_task_context *task_ctx = ctx; + struct brbe_regset live[BRBE_MAX_ENTRIES]; + int nr_live, nr_store, nr_hw_entries; + + nr_hw_entries = brbe_get_numrec(arm_pmu->reg_brbidr); + nr_live = brbe_branch_save(nr_hw_entries, live); + nr_store = task_ctx->nr_brbe_records; + nr_store = stitch_stored_live_entries(task_ctx->store, live, nr_store, + nr_live, nr_hw_entries); + task_ctx->nr_brbe_records = nr_store; +} + /* * Generic perf branch filters supported on BRBE * diff --git a/drivers/perf/arm_pmuv3.c b/drivers/perf/arm_pmuv3.c index 3c079051a63a..53f404618891 100644 --- a/drivers/perf/arm_pmuv3.c +++ b/drivers/perf/arm_pmuv3.c @@ -907,9 +907,19 @@ static int armv8pmu_user_event_idx(struct perf_event *event) static void armv8pmu_sched_task(struct perf_event_pmu_context *pmu_ctx, bool sched_in) { struct arm_pmu *armpmu = to_arm_pmu(pmu_ctx->pmu); + void *task_ctx = pmu_ctx ? pmu_ctx->task_ctx_data : NULL; - if (sched_in && armpmu->has_branch_stack) - armv8pmu_branch_reset(); + if (armpmu->has_branch_stack) { + /* Save branch records in task_ctx on sched out */ + if (task_ctx && !sched_in) { + armv8pmu_branch_save(armpmu, task_ctx); + return; + } + + /* Reset branch records on sched in */ + if (sched_in) + armv8pmu_branch_reset(); + } } /*