From patchwork Tue Feb 14 02:56:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: zhaotianrui X-Patchwork-Id: 5368 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2723432wrn; Mon, 13 Feb 2023 18:58:38 -0800 (PST) X-Google-Smtp-Source: AK7set8YD0S4MrbHtxNaxoo1ptkA9CSTs/tmqHvy9/ZMdXVlZLEGXhZyXo0PF42g67b5gFeuauTi X-Received: by 2002:a17:90b:1649:b0:233:e4f6:43dd with SMTP id il9-20020a17090b164900b00233e4f643ddmr716694pjb.18.1676343517719; Mon, 13 Feb 2023 18:58:37 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1676343517; cv=none; d=google.com; s=arc-20160816; b=yhstMcynU3qFLyatJ0dlJcGZcGWEaWyRth/RpYMXGJ00jT5UYtEW+MC+ljwWo3JQBK QW3xf/XU8uxPYcTCuibN+agC/gvWCeeZr6ciKQHLQGVRqMXkXZtQCTm+GUIor6T9BNEK Ua0LDBdww8b+Hzl8m6mNuBcatU8KwymIZ/NMmtWcTjOhTndgBdaFBmR5zj0jQwhUpNDL dGO9UfxwOhClsygwJAlJ3VXm22Ohhkx62u8pXchIT55GFyhrhH0a5PDSV3ihUlJml9tr hHpTtxE+eoQ4i35aiq6cf8DpMpd/rdI4QZysP2ETV3RaX+fvHtFBGycB7HiHASntkSg6 87lg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from; bh=WgXXhuv+NBLl3tc1fE4pvnPxWpJo+EgdsTSdUCfygpc=; b=j+TfgTG7GTV1hL3Ri3R5hZqZAeWUdnz/b03DI63KWQmYCR12ELtcU3sfaHsvY4xn4L w/9rxFZG3uXrZgM6yFK14+mIZr0nEuJgTR2lckAXzo/KZSYxr1ws2XGovyE1dR/7Kx1k 04qODVr7hIkP/Qk1u76gi8t4+GKBp45O2ouRQMOGTl82kd9Dk/x0et//JJWXX3XXxS7E /c1jNgOb74zDiTc4D4x55NQtN1t5DdUqUwQWOzyrgceWJqWSacB9eyRTtImIEVVSyO+9 EzLPL5Yn+TV4RtiUQj3gE+Us04uxM4MOwVUXTKQCCeAWl8yT5thPLiEtq+Z3HS88mdx0 PYxw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id in2-20020a17090b438200b002340d8ef64csi2943017pjb.108.2023.02.13.18.58.24; Mon, 13 Feb 2023 18:58:37 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229707AbjBNC5E (ORCPT + 99 others); Mon, 13 Feb 2023 21:57:04 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54696 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230044AbjBNC44 (ORCPT ); Mon, 13 Feb 2023 21:56:56 -0500 Received: from loongson.cn (mail.loongson.cn [114.242.206.163]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 0852118A92; Mon, 13 Feb 2023 18:56:53 -0800 (PST) Received: from loongson.cn (unknown [10.2.5.185]) by gateway (Coremail) with SMTP id _____8Dxldh0+OpjO1cAAA--.369S3; Tue, 14 Feb 2023 10:56:52 +0800 (CST) Received: from localhost.localdomain (unknown [10.2.5.185]) by localhost.localdomain (Coremail) with SMTP id AQAAf8Axeb1w+OpjmZwyAA--.28802S2; Tue, 14 Feb 2023 10:56:48 +0800 (CST) From: Tianrui Zhao To: Paolo Bonzini Cc: Huacai Chen , WANG Xuerui , Greg Kroah-Hartman , loongarch@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Jens Axboe , Mark Brown , Alex Deucher Subject: [PATCH v1 00/24] Add KVM LoongArch support Date: Tue, 14 Feb 2023 10:56:24 +0800 Message-Id: <20230214025648.1898508-1-zhaotianrui@loongson.cn> X-Mailer: git-send-email 2.31.1 MIME-Version: 1.0 X-CM-TRANSID: AQAAf8Axeb1w+OpjmZwyAA--.28802S2 X-CM-SenderInfo: p2kd03xldq233l6o00pqjv00gofq/ X-Coremail-Antispam: 1Uk129KBjvJXoWxKFWkZFWrGr4kGry7Jr1DKFg_yoWxXFWkpF W3urn8Gr4UGrZ3X3yvq3s8Zwn0vF1xGryag3WavFy8CrW2qry8ZrykKr9FvFy3AaykJr10 qr1rKw1ag3WUJaDanT9S1TB71UUUUjJqnTZGkaVYY2UrUUUUj1kv1TuYvTs0mT0YCTnIWj qI5I8CrVACY4xI64kE6c02F40Ex7xfYxn0WfASr-VFAUDa7-sFnT9fnUUIcSsGvfJTRUUU bcAFc2x0x2IEx4CE42xK8VAvwI8IcIk0rVWrJVCq3wA2ocxC64kIII0Yj41l84x0c7CEw4 AK67xGY2AK021l84ACjcxK6xIIjxv20xvE14v26ryj6F1UM28EF7xvwVC0I7IYx2IY6xkF 7I0E14v26r4j6F4UM28EF7xvwVC2z280aVAFwI0_Gr1j6F4UJwA2z4x0Y4vEx4A2jsIEc7 CjxVAFwI0_Gr1j6F4UJwAaw2AFwI0_JF0_Jw1le2I262IYc4CY6c8Ij28IcVAaY2xG8wAq jxCEc2xF0cIa020Ex4CE44I27wAqx4xG64xvF2IEw4CE5I8CrVC2j2WlYx0E74AGY7Cv6c x26rWlOx8S6xCaFVCjc4AY6r1j6r4UM4x0Y48IcxkI7VAKI48JMxkF7I0En4kS14v26r12 6r1DMxAIw28IcxkI7VAKI48JMxAIw28IcVCjz48v1sIEY20_WwCFx2IqxVCFs4IE7xkEbV WUJVW8JwCFI7km07C267AKxVWUAVWUtwC20s026c02F40E14v26r1j6r18MI8I3I0E7480 Y4vE14v26r106r1rMI8E67AF67kF1VAFwI0_Jw0_GFylIxkGc2Ij64vIr41lIxAIcVC0I7 IYx2IY67AKxVW5JVW7JwCI42IY6xIIjxv20xvEc7CjxVAFwI0_Gr0_Cr1lIxAIcVCF04k2 6cxKx2IYs7xG6r1j6r1xMIIF0xvEx4A2jsIE14v26F4j6r4UJwCI42IY6I8E87Iv6xkF7I 0E14v26r4j6r4UJbIYCTnIWIevJa73UjIFyTuYvj4RKpBTUUUUU X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,SPF_HELO_PASS, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757773580454671092?= X-GMAIL-MSGID: =?utf-8?q?1757773580454671092?= This series adds KVM LoongArch support. Loongson 3A5000 supports hardware assisted virtualization. With cpu virtualization, there are separate hw-supported user mode and kernel mode in guest mode. With memory virtualization, there are two-level hw mmu table for guest mode and host mode. Also there is separate hw cpu timer with consant frequency in guest mode, so that vm can migrate between hosts with different freq. Currently, we are able to boot LoongArch Linux Guests. Few key aspects of KVM LoongArch added by this series are: 1. Enable kvm hardware function when kvm module is loaded. 2. Implement VM and vcpu related ioctl interface such as vcpu create, vcpu run etc. GET_ONE_REG/SET_ONE_REG ioctl commands are use to get general registers one by one; KVM_GET_CSRS can be used to get system registers with batch mode similar with KVM_GET_MSRS in x86. 3. Hardware access about MMU, timer and csr are emulated in kernel. 4. Hardwares such as mmio and iocsr device are emulated in user space such as APIC, IPI, pci devices etc. The running environment of LoongArch virt machine: 1. Cross tools to build kernel and uefi: $ wget https://github.com/loongson/build-tools/releases/download/2022.09.06/loongarch64-clfs-6.3-cross-tools-gcc-glibc.tar.xz tar -vxf loongarch64-clfs-6.3-cross-tools-gcc-glibc.tar.xz -C /opt export PATH=/opt/cross-tools/bin:$PATH export LD_LIBRARY_PATH=/opt/cross-tools/lib:$LD_LIBRARY_PATH export LD_LIBRARY_PATH=/opt/cross-tools/loongarch64-unknown-linux-gnu/lib/:$LD_LIBRARY_PATH 2. This series is based on the linux source code: https://github.com/loongson/linux-loongarch-kvm Build command: git checkout kvm-loongarch make ARCH=loongarch CROSS_COMPILE=loongarch64-unknown-linux-gnu- loongson3_defconfig make ARCH=loongarch CROSS_COMPILE=loongarch64-unknown-linux-gnu- 3. QEMU hypervisor with LoongArch supported: https://github.com/loongson/qemu Build command: git checkout kvm-loongarch ./configure --target-list="loongarch64-softmmu" --enable-kvm make 4. Uefi bios of LoongArch virt machine: Reference: https://github.com/tianocore/edk2-platforms/tree/master/Platform/Loongson/LoongArchQemuPkg#readme 5. you can also access the binary files we have already build: https://github.com/yangxiaojuan-loongson/qemu-binary The command to boot loongarch virt machine: $ qemu-system-loongarch64 -machine virt -m 4G -cpu la464 \ -smp 1 -bios QEMU_EFI.fd -kernel vmlinuz.efi -initrd ramdisk \ -serial stdio -monitor telnet:localhost:4495,server,nowait \ -append "root=/dev/ram rdinit=/sbin/init console=ttyS0,115200" \ --nographic Tianrui Zhao (24): LoongArch: KVM: Implement kvm module related interface LoongArch: KVM: Implement VM related functions LoongArch: KVM: Implement vcpu create,run,destroy operations. LoongArch: KVM: Implement vcpu get, vcpu set registers LoongArch: KVM: Implement vcpu ENABLE_CAP, CHECK_EXTENSION ioctl interface LoongArch: KVM: Implement fpu related operations for vcpu LoongArch: KVM: Implement vcpu interrupt operations LoongArch: KVM: Implement misc vcpu related interfaces LoongArch: KVM: Implement vcpu load and vcpu put operations LoongArch: KVM: Implement vcpu status description LoongArch: KVM: Implement update VM id function LoongArch: KVM: Implement virtual machine tlb operations LoongArch: KVM: Implement vcpu timer operations LoongArch: KVM: Implement kvm mmu operations LoongArch: KVM: Implement handle csr excption LoongArch: KVM: Implement handle iocsr exception LoongArch: KVM: Implement handle idle exception LoongArch: KVM: Implement handle gspr exception LoongArch: KVM: Implement handle mmio exception LoongArch: KVM: Implement handle fpu exception LoongArch: KVM: Implement kvm exception vector LoongArch: KVM: Implement vcpu world switch LoongArch: KVM: Implement probe virtualization when loongarch cpu init LoongArch: KVM: Enable kvm config and add the makefile arch/loongarch/Kbuild | 1 + arch/loongarch/Kconfig | 2 + arch/loongarch/configs/loongson3_defconfig | 2 + arch/loongarch/include/asm/cpu-features.h | 22 + arch/loongarch/include/asm/cpu-info.h | 13 + arch/loongarch/include/asm/inst.h | 16 + arch/loongarch/include/asm/kvm_csr.h | 89 ++ arch/loongarch/include/asm/kvm_host.h | 257 +++++ arch/loongarch/include/asm/kvm_types.h | 11 + arch/loongarch/include/asm/kvm_vcpu.h | 112 ++ arch/loongarch/include/asm/loongarch.h | 195 +++- arch/loongarch/include/uapi/asm/kvm.h | 121 ++ arch/loongarch/kernel/asm-offsets.c | 32 + arch/loongarch/kernel/cpu-probe.c | 53 + arch/loongarch/kvm/Kconfig | 38 + arch/loongarch/kvm/Makefile | 21 + arch/loongarch/kvm/exit.c | 702 ++++++++++++ arch/loongarch/kvm/interrupt.c | 126 +++ arch/loongarch/kvm/main.c | 152 +++ arch/loongarch/kvm/mmu.c | 821 ++++++++++++++ arch/loongarch/kvm/switch.S | 327 ++++++ arch/loongarch/kvm/timer.c | 266 +++++ arch/loongarch/kvm/tlb.c | 31 + arch/loongarch/kvm/trace.h | 137 +++ arch/loongarch/kvm/vcpu.c | 1185 ++++++++++++++++++++ arch/loongarch/kvm/vm.c | 85 ++ arch/loongarch/kvm/vmid.c | 64 ++ include/uapi/linux/kvm.h | 15 + 28 files changed, 4890 insertions(+), 6 deletions(-) create mode 100644 arch/loongarch/include/asm/kvm_csr.h create mode 100644 arch/loongarch/include/asm/kvm_host.h create mode 100644 arch/loongarch/include/asm/kvm_types.h create mode 100644 arch/loongarch/include/asm/kvm_vcpu.h create mode 100644 arch/loongarch/include/uapi/asm/kvm.h create mode 100644 arch/loongarch/kvm/Kconfig create mode 100644 arch/loongarch/kvm/Makefile create mode 100644 arch/loongarch/kvm/exit.c create mode 100644 arch/loongarch/kvm/interrupt.c create mode 100644 arch/loongarch/kvm/main.c create mode 100644 arch/loongarch/kvm/mmu.c create mode 100644 arch/loongarch/kvm/switch.S create mode 100644 arch/loongarch/kvm/timer.c create mode 100644 arch/loongarch/kvm/tlb.c create mode 100644 arch/loongarch/kvm/trace.h create mode 100644 arch/loongarch/kvm/vcpu.c create mode 100644 arch/loongarch/kvm/vm.c create mode 100644 arch/loongarch/kvm/vmid.c