From patchwork Thu Dec 15 17:00:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Atish Patra X-Patchwork-Id: 3002 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:e747:0:0:0:0:0 with SMTP id c7csp482246wrn; Thu, 15 Dec 2022 09:03:11 -0800 (PST) X-Google-Smtp-Source: AA0mqf43TTHk9LNeku7T+dMwpEDG6aszM+J9DS7nlQMF8Xf28D7de5H6thpBarZhRAquWsx1NYgE X-Received: by 2002:a17:906:7754:b0:78d:f455:b5c1 with SMTP id o20-20020a170906775400b0078df455b5c1mr24593122ejn.1.1671123791289; Thu, 15 Dec 2022 09:03:11 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1671123791; cv=none; d=google.com; s=arc-20160816; b=K1btbeoYsPVAmfaC1oe3AFXEIX6mqsBs7M1jCl6lY8r3MYyGTTRUw5zWOr1Umhx/PK kwb2AqB00YXxPgPl5PBXFQ6/EQhlpiOAtw+xwQF4PCSwzvFH2KXIcWSSbroh/CpuWBUs XBxlJVphUBiXvB8SMC6aVUuWGNxOsn0+OmnDpiR+ZnbI2TgMSXlq973C/gMDueJyf7Mb Z4YcQArRGx+S+hxgBX7ri8LlZxmW+c630ESrb3zLCh4oIvcdws1ujjP2C04oETjNyk9l IulsEKASK4mxLPsU+q4SS2FYTdCkhF4yoaTCQp58nVlXlam06ieVuZ8BCDvHFEzlBAjQ X0lQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=Qx+V0HqoGlECsFSjn8+kKGYij8Unq8RFOTIoeLkQqAQ=; b=h3HSwtPvm6OzrvaZICAtUDD/D+ebE7bNy3/y4kaxIcU4u+ZkLsmoeu73ADKyGA8g7N mQeAKjZNKR3SeueOVZ9BcQ8S1SjmSMaUlUDdezxkFE0L6Ngs4x31/0UWHwXNKURM7qQP c4xd40oi8mDM69jtna0Tw50b4M0H+mQ7GLcVE7jn+Xuy79Y6jJCAwqmIsFOKwuLtEUkz eSpG6IWtrzAVH/NTCTvUyy2s2wJvHy6lTu1+i99qoMHASHCkJYaisB19zDKyTmY9xrjS NrCm/fGwjF0S6ZznoqQ73aqk+/4cpQVlJ8oqOTIYTFdaa84HYIQ1ynZ653JBG3v/sGpe 1gow== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@rivosinc-com.20210112.gappssmtp.com header.s=20210112 header.b=WZQXttZy; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id hb35-20020a170907162300b007c1727f7c65si11390739ejc.470.2022.12.15.09.02.46; Thu, 15 Dec 2022 09:03:11 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@rivosinc-com.20210112.gappssmtp.com header.s=20210112 header.b=WZQXttZy; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230364AbiLORBS (ORCPT + 99 others); Thu, 15 Dec 2022 12:01:18 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51972 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230177AbiLORBP (ORCPT ); Thu, 15 Dec 2022 12:01:15 -0500 Received: from mail-pj1-x1036.google.com (mail-pj1-x1036.google.com [IPv6:2607:f8b0:4864:20::1036]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 115891A069 for ; Thu, 15 Dec 2022 09:01:14 -0800 (PST) Received: by mail-pj1-x1036.google.com with SMTP id u15-20020a17090a3fcf00b002191825cf02so3278396pjm.2 for ; Thu, 15 Dec 2022 09:01:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=Qx+V0HqoGlECsFSjn8+kKGYij8Unq8RFOTIoeLkQqAQ=; b=WZQXttZyILm2FtfOQTdSxF4+l7eAT9OOLhvMed/ku0pv/3kGzRZB4o8NNG2v02zWFN fagBkxAG6+UvXOnYtV8Jgm7HB8xik9lQ+iJAncmqh2VjKjrieJMFXPagnVPu8dnzqbr+ 0eAx2hqQe3Y/tbMeBggbIPePlmUu8ApFm+/hCgi+FBpmlFlyVvlEKTrGbBa727FdLn2f UFCBwNTAonW6bcQXv6dc0+G4xpmoj5YzygeHPlRXo1xx3eKosznF7obEewjhjocmxrPl vmt/2erJr3c8CrnPGS3R/FoejHxONTpgeuVvPa+B3CGqPdxjnr6NC5BByjCQOmRiSpVF MhOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Qx+V0HqoGlECsFSjn8+kKGYij8Unq8RFOTIoeLkQqAQ=; b=aYm0eT/ngV3kFksLs/K0MBoWqE+0EwnlsKsKIw5F4OU7i7fGOPPmz8N8neeb53KbFc MhPiTueY88qui1SEDuhEahh2WGdLyGm+74iQCyT7jxPu4HTZFqofqDDtDp4+DOnMzdbq Lw5SHXC1hhCXjIG/i8Y/BkbVWsAjhEeWbtwn3XZ2t0tWYYKYzW+RSdYsu/tFpnHVh5wB N8laAs6ZsY7Xm5zKiOpLd+rEVyAsoByKE9vvkQK7pv1ksr8Pa95KIMAPfIj8tRrMGxoS 8hnu+or3/+FveqfJkY9sMlYbt4i7aN61uJf2dCEaMJG8QWxSpB++0BZbWjnoi73D8wa4 96bQ== X-Gm-Message-State: ANoB5pluKcE+kvt1slbrDYcipbXHo/EEG+K1lJyIJGm1F7zJeHcA+g3N IO5dTGZygz24V8vft6uUuSnUWXjEqn5+iQgK X-Received: by 2002:a17:902:b083:b0:18c:d9df:a4b5 with SMTP id p3-20020a170902b08300b0018cd9dfa4b5mr26407596plr.7.1671123673271; Thu, 15 Dec 2022 09:01:13 -0800 (PST) Received: from atishp.ba.rivosinc.com ([66.220.2.162]) by smtp.gmail.com with ESMTPSA id p10-20020a170902780a00b001897bfc9800sm4067449pll.53.2022.12.15.09.01.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 15 Dec 2022 09:01:12 -0800 (PST) From: Atish Patra To: linux-kernel@vger.kernel.org Cc: Atish Patra , Anup Patel , Andrew Jones , Atish Patra , Guo Ren , kvm-riscv@lists.infradead.org, kvm@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Rutland , Palmer Dabbelt , Paul Walmsley , Sergey Matyukevich , Eric Lin , Will Deacon Subject: [PATCH v2 00/11] KVM perf support Date: Thu, 15 Dec 2022 09:00:35 -0800 Message-Id: <20221215170046.2010255-1-atishp@rivosinc.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1752300300597889518?= X-GMAIL-MSGID: =?utf-8?q?1752300300597889518?= This series extends perf support for KVM. The KVM implementation relies on the SBI PMU extension and trap n emulation of hpmcounter CSRs. The KVM implementation exposes the virtual counters to the guest and internally manage the counters using kernel perf counters. This series doesn't support the counter overflow as the Sscofpmf extension doesn't allow trap & emulation mechanism of scountovf CSR yet. The required changes to allow that are being under discussions. Supporting overflow interrupt also requires AIA interrupt filtering support. 1. PATCH1-4 are generic KVM/PMU driver improvements. 2. PATCH8 disables hpmcounter for now. It will be enabled to maintain ABI requirement once the ONE reg interface is settled. perf stat works in kvm guests with this series. Here is example of running perf stat in a guest running in KVM. =========================================================================== / # /host/apps/perf stat -e instructions -e cycles -e r8000000000000005 \ > -e r8000000000000006 -e r8000000000000007 -e r8000000000000008 \ > -e r800000000000000a perf bench sched messaging -g 10 -l 10 # Running 'sched/messaging' benchmark: # 20 sender and receiver processes per group # 10 groups == 400 processes run Total time: 7.769 [sec] Performance counter stats for 'perf bench sched messaging -g 10 -l 10': 73556259604 cycles 73387266056 instructions # 1.00 insn per cycle 0 dTLB-store-misses 0 iTLB-load-misses 0 r8000000000000005 2595 r8000000000000006 2272 r8000000000000007 10 r8000000000000008 0 r800000000000000a 12.173720400 seconds time elapsed 1.002716000 seconds user 21.931047000 seconds sys Note: The SBI_PMU_FW_SET_TIMER (eventid : r8000000000000005) is zero as kvm guest supports sstc now. This series can be found here as well. https://github.com/atishp04/linux/tree/kvm_perf_v2 TODO: 1. Add sscofpmf support. 2. Add One reg interface for the following operations: 1. Enable/Disable PMU (should it at VM level rather than vcpu ?) 2. Number of hpmcounter and width of the counters 3. Init PMU 4. Allow guest user to access cycle & instret without trapping Changes from v1->v2: 1. Addressed comments from Andrew. 2. Removed kvpmu sanity check. 3. Added a kvm pmu init flag and the sanity check to probe function. 4. Improved the linux vs sbi error code handling. Atish Patra (11): RISC-V: Define helper functions expose hpm counter width and count RISC-V: KVM: Define a probe function for SBI extension data structures RISC-V: KVM: Return correct code for hsm stop function RISC-V: KVM: Modify SBI extension handler to return SBI error code RISC-V: KVM: Improve privilege mode filtering for perf RISC-V: KVM: Add skeleton support for perf RISC-V: KVM: Add SBI PMU extension support RISC-V: KVM: Disable all hpmcounter access for VS/VU mode RISC-V: KVM: Implement trap & emulate for hpmcounters RISC-V: KVM: Implement perf support without sampling RISC-V: KVM: Implement firmware events arch/riscv/include/asm/kvm_host.h | 3 + arch/riscv/include/asm/kvm_vcpu_pmu.h | 108 +++++ arch/riscv/include/asm/kvm_vcpu_sbi.h | 13 +- arch/riscv/include/asm/sbi.h | 2 +- arch/riscv/kvm/Makefile | 1 + arch/riscv/kvm/main.c | 3 +- arch/riscv/kvm/tlb.c | 6 +- arch/riscv/kvm/vcpu.c | 5 + arch/riscv/kvm/vcpu_insn.c | 4 +- arch/riscv/kvm/vcpu_pmu.c | 585 ++++++++++++++++++++++++++ arch/riscv/kvm/vcpu_sbi.c | 56 ++- arch/riscv/kvm/vcpu_sbi_base.c | 45 +- arch/riscv/kvm/vcpu_sbi_hsm.c | 22 +- arch/riscv/kvm/vcpu_sbi_pmu.c | 86 ++++ arch/riscv/kvm/vcpu_sbi_replace.c | 51 ++- drivers/perf/riscv_pmu_sbi.c | 62 ++- include/linux/perf/riscv_pmu.h | 5 + 17 files changed, 963 insertions(+), 94 deletions(-) create mode 100644 arch/riscv/include/asm/kvm_vcpu_pmu.h create mode 100644 arch/riscv/kvm/vcpu_pmu.c create mode 100644 arch/riscv/kvm/vcpu_sbi_pmu.c --- 2.25.1