From patchwork Tue Feb 13 09:32:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 200284 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp423544dyb; Tue, 13 Feb 2024 01:34:21 -0800 (PST) X-Google-Smtp-Source: AGHT+IEyb+jbUc15l6JKnBfHyD9mYGVMrs/Zjy55sBDeZKkrvGep5ALiEe2KkSjXmcUW+nL5Trgv X-Received: by 2002:aa7:ca4c:0:b0:55f:1c0a:5c35 with SMTP id j12-20020aa7ca4c000000b0055f1c0a5c35mr6340770edt.4.1707816860814; Tue, 13 Feb 2024 01:34:20 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707816860; cv=pass; d=google.com; s=arc-20160816; b=eK5H37kJ5/hDQCkmEeWX7ptmaqPkzJ7JWD6fT4fwJkiWwFadFLbxv09+7nv2rzv9Ky Zf6bnvE05vhhpkJ5wNTv1vZWZOSgDiuq7ge7N+ls/jor47FuwrR3Xoqvhufj4QbBDsCy edlD97IUkzi7kHlekg4nERKtP4eiF1GSNmVzTzEZy2a9BRyX0eznSn0W8qPoD0HHJyeu hIR+cKPDC9KBxKcXctA5ESDkR/P2dKFyoVRatqSSifjK0y/c3iIlyb9dJg6n3XHIuKd/ BmZt6LvkP6D6onF8rvlF5Wpy00hxfCa5MTBQ+bPIzak61TcJ44dNbgjBySxuc8UNVhfM fQbQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=cyUnvVECpp/8iTljvXmQPWzrvEO1ZBGnoQKIdU51DBU=; fh=otbepu7hgHdWBIxFXTxBpgdEW8ltUHkOIsBddvXJxco=; b=rCaUAZOz3uqBqW6yhTGQQlBt5ueuIxquCK2ada62eI4ArDLmBRjbKQZvG3UN/Hem03 +l1CcT1LDR/KO+68JMuU/qr0XAkJ2fkNw7l661JnF3+J8y6FV6XBe8tk9mDjVujWUlJU yEN3dK19YuBGjhD7tSLS5H5AkYiLUPRl6it5DEwXBKvPl6Ki/tIdWWOx9cT9zRGWdviK apl9MZrt74Ji7Vf2W3BewV8LM3sJS+I+HMawHwsuiOxLvEz0b9qcxfPmdFsUJbTlBPeT 4DT2aloyhW8w5JuxzAlzghci+GUlUUTtpdGqGDM2NnwAtpv0sDl8zDAUW0zB+XjmzU8K kmiw==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=vo8RB2qf; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63206-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63206-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev X-Forwarded-Encrypted: i=2; AJvYcCUKbVQn7xe4rPyp5yvuhWZD22iUTfjk0HXFO54r/3SpLNpHOBDnvcj1X1t9gYRgLImXZ5ZmHP+0+v0JnZYX9BhuQTA0nw== Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id u6-20020a50c046000000b00561ee2f8c3dsi642099edd.646.2024.02.13.01.34.20 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 01:34:20 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-63206-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=vo8RB2qf; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63206-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63206-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 43A621F2557C for ; Tue, 13 Feb 2024 09:34:20 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 8B8E824A11; Tue, 13 Feb 2024 09:33:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="vo8RB2qf" Received: from out-184.mta1.migadu.com (out-184.mta1.migadu.com [95.215.58.184]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4E4051B273 for ; Tue, 13 Feb 2024 09:33:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.184 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707816799; cv=none; b=UHETzXCigOmKBHrSZWxwQM0L5p19IvdO47Z89G4fFsqBiattHvbPsfR2cde2b0lZAUq34qv/MNQrPGS0RtI+CxqJ4Gk1DMh6ebZxmWH6x+QbeJWO48eukVv0Idp4ZDno2J+tQoHK7MrZg6SfnJcNh7u0osMEG8e64R23i6CKm98= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707816799; c=relaxed/simple; bh=bwAh/HjIrt1hqUnlvYVrZwrrMVLgoGSwee3tzVmP50s=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=OG2muiUO6/q1zmWw9Bii7hocnIxsWn1D5Ucadbe08R4FZ1X9qzXfjuO0+GuMrtgJAZ3DxGj6OeU7J22oa56O03DGWu4gHSjhUTH/a8+cmiRctTKE3jlvrJ/jhSctm5IjnQtPE27F9OI2qSp0K8i0OuEMU2dgBMrqSAxjDElWmNs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=vo8RB2qf; arc=none smtp.client-ip=95.215.58.184 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1707816794; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cyUnvVECpp/8iTljvXmQPWzrvEO1ZBGnoQKIdU51DBU=; b=vo8RB2qfGqEonVTQPePSakr2QqTty/7P7NdKqNQyptvl5bDfkijfU1kXX6vvVTY6dFU+c3 HogKV3EfdrrPBOPunsoemPi/WzmFuHTnAflQrWwQm5WlW5x/RrSrKna+v5kRUktY0CUQnw ARsEvSc++Dh03U+obakDaYrOC/LfaT8= From: Oliver Upton To: kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , linux-kernel@vger.kernel.org, Oliver Upton Subject: [PATCH v2 01/23] KVM: arm64: Add tracepoints + stats for LPI cache effectiveness Date: Tue, 13 Feb 2024 09:32:38 +0000 Message-ID: <20240213093250.3960069-2-oliver.upton@linux.dev> In-Reply-To: <20240213093250.3960069-1-oliver.upton@linux.dev> References: <20240213093250.3960069-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790775772876606776 X-GMAIL-MSGID: 1790775772876606776 LPI translation and injection has been shown to have a significant impact on the performance of VM workloads, so it probably makes sense to add some signals in this area. Introduce the concept of a KVM tracepoint that associates with a VM stat and use it for the LPI translation cache tracepoints. It isn't too uncommon for a kernel hacker to attach to tracepoints, while at the same time userspace may open a 'binary stats' FD to peek at the corresponding VM stats. Signed-off-by: Oliver Upton --- arch/arm64/include/asm/kvm_host.h | 3 ++ arch/arm64/kvm/guest.c | 5 ++- arch/arm64/kvm/vgic/trace.h | 66 +++++++++++++++++++++++++++++++ arch/arm64/kvm/vgic/vgic-its.c | 14 ++++++- include/linux/kvm_host.h | 4 ++ 5 files changed, 89 insertions(+), 3 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 21c57b812569..6f88b76373a5 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -966,6 +966,9 @@ static inline bool __vcpu_write_sys_reg_to_cpu(u64 val, int reg) struct kvm_vm_stat { struct kvm_vm_stat_generic generic; + u64 vgic_its_trans_cache_hit; + u64 vgic_its_trans_cache_miss; + u64 vgic_its_trans_cache_victim; }; struct kvm_vcpu_stat { diff --git a/arch/arm64/kvm/guest.c b/arch/arm64/kvm/guest.c index aaf1d4939739..354d67251fc2 100644 --- a/arch/arm64/kvm/guest.c +++ b/arch/arm64/kvm/guest.c @@ -30,7 +30,10 @@ #include "trace.h" const struct _kvm_stats_desc kvm_vm_stats_desc[] = { - KVM_GENERIC_VM_STATS() + KVM_GENERIC_VM_STATS(), + STATS_DESC_COUNTER(VM, vgic_its_trans_cache_hit), + STATS_DESC_COUNTER(VM, vgic_its_trans_cache_miss), + STATS_DESC_COUNTER(VM, vgic_its_trans_cache_victim) }; const struct kvm_stats_header kvm_vm_stats_header = { diff --git a/arch/arm64/kvm/vgic/trace.h b/arch/arm64/kvm/vgic/trace.h index 83c64401a7fc..ff6423f22c91 100644 --- a/arch/arm64/kvm/vgic/trace.h +++ b/arch/arm64/kvm/vgic/trace.h @@ -27,6 +27,72 @@ TRACE_EVENT(vgic_update_irq_pending, __entry->vcpu_id, __entry->irq, __entry->level) ); +TRACE_EVENT(vgic_its_trans_cache_hit, + TP_PROTO(__u64 db_addr, __u32 device_id, __u32 event_id, __u32 intid), + TP_ARGS(db_addr, device_id, event_id, intid), + + TP_STRUCT__entry( + __field( __u64, db_addr ) + __field( __u32, device_id ) + __field( __u32, event_id ) + __field( __u32, intid ) + ), + + TP_fast_assign( + __entry->db_addr = db_addr; + __entry->device_id = device_id; + __entry->event_id = event_id; + __entry->intid = intid; + ), + + TP_printk("DB: %016llx, device_id %u, event_id %u, intid %u", + __entry->db_addr, __entry->device_id, __entry->event_id, + __entry->intid) +); + +TRACE_EVENT(vgic_its_trans_cache_miss, + TP_PROTO(__u64 db_addr, __u32 device_id, __u32 event_id), + TP_ARGS(db_addr, device_id, event_id), + + TP_STRUCT__entry( + __field( __u64, db_addr ) + __field( __u32, device_id ) + __field( __u32, event_id ) + ), + + TP_fast_assign( + __entry->db_addr = db_addr; + __entry->device_id = device_id; + __entry->event_id = event_id; + ), + + TP_printk("DB: %016llx, device_id %u, event_id %u", + __entry->db_addr, __entry->device_id, __entry->event_id) +); + +TRACE_EVENT(vgic_its_trans_cache_victim, + TP_PROTO(__u64 db_addr, __u32 device_id, __u32 event_id, __u32 intid), + TP_ARGS(db_addr, device_id, event_id, intid), + + TP_STRUCT__entry( + __field( __u64, db_addr ) + __field( __u32, device_id ) + __field( __u32, event_id ) + __field( __u32, intid ) + ), + + TP_fast_assign( + __entry->db_addr = db_addr; + __entry->device_id = device_id; + __entry->event_id = event_id; + __entry->intid = intid; + ), + + TP_printk("DB: %016llx, device_id %u, event_id %u, intid %u", + __entry->db_addr, __entry->device_id, __entry->event_id, + __entry->intid) +); + #endif /* _TRACE_VGIC_H */ #undef TRACE_INCLUDE_PATH diff --git a/arch/arm64/kvm/vgic/vgic-its.c b/arch/arm64/kvm/vgic/vgic-its.c index e2764d0ffa9f..59179268ac2d 100644 --- a/arch/arm64/kvm/vgic/vgic-its.c +++ b/arch/arm64/kvm/vgic/vgic-its.c @@ -20,6 +20,7 @@ #include #include +#include "trace.h" #include "vgic.h" #include "vgic-mmio.h" @@ -636,8 +637,11 @@ static void vgic_its_cache_translation(struct kvm *kvm, struct vgic_its *its, * to the interrupt, so drop the potential reference on what * was in the cache, and increment it on the new interrupt. */ - if (cte->irq) + if (cte->irq) { + KVM_VM_TRACE_EVENT(kvm, vgic_its_trans_cache_victim, cte->db, + cte->devid, cte->eventid, cte->irq->intid); __vgic_put_lpi_locked(kvm, cte->irq); + } vgic_get_irq_kref(irq); @@ -767,8 +771,14 @@ int vgic_its_inject_cached_translation(struct kvm *kvm, struct kvm_msi *msi) db = (u64)msi->address_hi << 32 | msi->address_lo; irq = vgic_its_check_cache(kvm, db, msi->devid, msi->data); - if (!irq) + if (!irq) { + KVM_VM_TRACE_EVENT(kvm, vgic_its_trans_cache_miss, db, msi->devid, + msi->data); return -EWOULDBLOCK; + } + + KVM_VM_TRACE_EVENT(kvm, vgic_its_trans_cache_hit, db, msi->devid, + msi->data, irq->intid); raw_spin_lock_irqsave(&irq->irq_lock, flags); irq->pending_latch = true; diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 7e7fd25b09b3..846b447b6798 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1927,6 +1927,10 @@ struct _kvm_stats_desc { HALT_POLL_HIST_COUNT), \ STATS_DESC_IBOOLEAN(VCPU_GENERIC, blocking) +#define KVM_VM_TRACE_EVENT(vm, event, ...) \ + ((vm)->stat.event)++; trace_## event(__VA_ARGS__) + + extern struct dentry *kvm_debugfs_dir; ssize_t kvm_stats_read(char *id, const struct kvm_stats_header *header, From patchwork Tue Feb 13 09:32:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 200285 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp423651dyb; Tue, 13 Feb 2024 01:34:33 -0800 (PST) X-Google-Smtp-Source: AGHT+IFkp2BSk7Js6982un7hOKg1yTvfaIGTDr/yT7EJa73bYXD7C1SrbQsetuONt/Y8iQvu/Yyv X-Received: by 2002:a05:6402:797:b0:561:e4aa:ef2c with SMTP id d23-20020a056402079700b00561e4aaef2cmr1665019edy.32.1707816873212; Tue, 13 Feb 2024 01:34:33 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707816873; cv=pass; d=google.com; s=arc-20160816; b=oEAtdGnNeYdi/4SaEA3t4kN+ESVFbpNvIC8Zagsw9cyKlgGWPt45db5rZlmeHrN0i0 l+vGKTUsRNpT7FmwtcPDXVrvEEoBOx1MssSwzzws7PW5p3FEmmA89/kwsQ+6lsh1aHui X6GobHG3c2bKXm9A8oPA3E2UVfivLvaQMWEOGnX0s8eaQm08+yxzUoJ3vQhCIpR6QoWC pGkDz3WwWwPCCQdO5/2rgv9eqal0Te5NPNLAVZvu/ZKO0Lta+iIqU1d1XlnZLaXsgh23 stW46RyaFE1tocfhLEX5O/GgrrsvrJ5j2iI7+2SY9W2zRV222poAbCi1nPX5MsT6IBk7 mtxw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=fnPoXibpf2eubnu4qas8Q67UZMLIaNGzfiD65c36g0Y=; fh=9qsDl80Ah6hGGVcyRYMeGASCZsA3ANHxoWkqy6EnK2Y=; b=RihQppPU3diM1GK9smivUSlMHK+J09TAqC6firxG1/yLTjddxhfrc5Pz8UNMD07/ad kBwIDtyoChSzIHbOgA/bkEG1V8aJ1JS2GCXwWUiEWnofJPhwSytZk9BHjJRbz4biR5DN PY2p3p89nvgE0H0D+u1SrwV6aMF4jk16/QGiqrPoQlQViHtu00fuBuufOxa2tqElWo1h xJAL9fv8JPOL//vsG1iTAqnyc4GXKx2V4FCdivxcwoEqsigIOymoL4K9+uda6TdShwNq cwRqF8h5MgRru0SYJi0EUS9IEuF4AFIO9V6f3kIa22KU68iZ6/1yBYgc8O0oa1LRrfgL dORQ==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=vaKssv7Z; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63207-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63207-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev X-Forwarded-Encrypted: i=2; AJvYcCVmCL9ptBFF6DUNfE5F4BtjIlSiK/5kGXN4cDzFWhW60ENPLe1UiDxJFQ/tWXbj4xPSbJa7EDjrqQo46pyru2pBYij9Bg== Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id m22-20020aa7d356000000b00560027fbed5si3648528edr.619.2024.02.13.01.34.33 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 01:34:33 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-63207-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=vaKssv7Z; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63207-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63207-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id D12451F2523E for ; Tue, 13 Feb 2024 09:34:32 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 5B845286AD; Tue, 13 Feb 2024 09:33:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="vaKssv7Z" Received: from out-175.mta1.migadu.com (out-175.mta1.migadu.com [95.215.58.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C72F422334 for ; Tue, 13 Feb 2024 09:33:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707816800; cv=none; b=usveOC/Cr/uy0PS9/Ldjyl89FJgbtVSMfOENv5rNzIWdK1q7zpNUeeVEjXKvxCUcZRsNcjxkmQh1Yb+DYmGhilJJT2i/uAdESqIOo+uIjnLhWDSB4XE1Y1j+oTLyRqTjGZiYuZD+TyvZmdOvY0npEuUfo1UYi7oz2ndVIZQIar8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707816800; c=relaxed/simple; bh=H7EPwNAHhjpAAN+1sCd20hc3wyAtZ3lgAHFoSyw3Q5g=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=dd9PE2ExHZAiAui1Cx+h8fgqqKHoCqlS6mwlm68ATJu9n01/elOhXUe5WSTo1MQmF7wk4LnSrvtR1ztvgURNhQs5ANZP3iQ0hpGlUaQv+V1YuK19FmeMRRapxvRVTzOtlmQVIspz/T26NMHui4FdxoXxfXuwNEs5X4sQidLx1MA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=vaKssv7Z; arc=none smtp.client-ip=95.215.58.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1707816796; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fnPoXibpf2eubnu4qas8Q67UZMLIaNGzfiD65c36g0Y=; b=vaKssv7ZX0b92LyxSE2pWJosQl7s9miduOitoV/mhpSghGsXb9f3a1INqHUg7rWIm7QU0+ pAWzKGNYijFdtUoxI/hac1dBCqgGANwd3LvAv72hBapQ435bqhAwQNAB+mpCNs1wRmuezW y3dxjGfE0Qf/iJWDD1ynrlLkHHYxdrs= From: Oliver Upton To: kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , linux-kernel@vger.kernel.org, Oliver Upton Subject: [PATCH v2 02/23] KVM: arm64: vgic: Store LPIs in an xarray Date: Tue, 13 Feb 2024 09:32:39 +0000 Message-ID: <20240213093250.3960069-3-oliver.upton@linux.dev> In-Reply-To: <20240213093250.3960069-1-oliver.upton@linux.dev> References: <20240213093250.3960069-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790775785901624392 X-GMAIL-MSGID: 1790775785901624392 Using a linked-list for LPIs is less than ideal as it of course requires iterative searches to find a particular entry. An xarray is a better data structure for this use case, as it provides faster searches and can still handle a potentially sparse range of INTID allocations. Start by storing LPIs in an xarray, punting usage of the xarray to a subsequent change. Signed-off-by: Oliver Upton --- arch/arm64/kvm/vgic/vgic-init.c | 3 +++ arch/arm64/kvm/vgic/vgic-its.c | 16 ++++++++++++++++ arch/arm64/kvm/vgic/vgic.c | 1 + include/kvm/arm_vgic.h | 2 ++ 4 files changed, 22 insertions(+) diff --git a/arch/arm64/kvm/vgic/vgic-init.c b/arch/arm64/kvm/vgic/vgic-init.c index e949e1d0fd9f..411719053107 100644 --- a/arch/arm64/kvm/vgic/vgic-init.c +++ b/arch/arm64/kvm/vgic/vgic-init.c @@ -56,6 +56,7 @@ void kvm_vgic_early_init(struct kvm *kvm) INIT_LIST_HEAD(&dist->lpi_list_head); INIT_LIST_HEAD(&dist->lpi_translation_cache); raw_spin_lock_init(&dist->lpi_list_lock); + xa_init_flags(&dist->lpi_xa, XA_FLAGS_LOCK_IRQ); } /* CREATION */ @@ -366,6 +367,8 @@ static void kvm_vgic_dist_destroy(struct kvm *kvm) if (vgic_supports_direct_msis(kvm)) vgic_v4_teardown(kvm); + + xa_destroy(&dist->lpi_xa); } static void __kvm_vgic_vcpu_destroy(struct kvm_vcpu *vcpu) diff --git a/arch/arm64/kvm/vgic/vgic-its.c b/arch/arm64/kvm/vgic/vgic-its.c index 59179268ac2d..0265cd1f2d6e 100644 --- a/arch/arm64/kvm/vgic/vgic-its.c +++ b/arch/arm64/kvm/vgic/vgic-its.c @@ -53,6 +53,12 @@ static struct vgic_irq *vgic_add_lpi(struct kvm *kvm, u32 intid, if (!irq) return ERR_PTR(-ENOMEM); + ret = xa_reserve_irq(&dist->lpi_xa, intid, GFP_KERNEL_ACCOUNT); + if (ret) { + kfree(irq); + return ERR_PTR(ret); + } + INIT_LIST_HEAD(&irq->lpi_list); INIT_LIST_HEAD(&irq->ap_list); raw_spin_lock_init(&irq->irq_lock); @@ -87,10 +93,20 @@ static struct vgic_irq *vgic_add_lpi(struct kvm *kvm, u32 intid, goto out_unlock; } + ret = xa_err(xa_store(&dist->lpi_xa, intid, irq, 0)); + if (ret) { + xa_release(&dist->lpi_xa, intid); + kfree(irq); + goto out_unlock; + } + list_add_tail(&irq->lpi_list, &dist->lpi_list_head); dist->lpi_list_count++; out_unlock: + if (ret) + return ERR_PTR(ret); + raw_spin_unlock_irqrestore(&dist->lpi_list_lock, flags); /* diff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c index db2a95762b1b..c126014f8395 100644 --- a/arch/arm64/kvm/vgic/vgic.c +++ b/arch/arm64/kvm/vgic/vgic.c @@ -131,6 +131,7 @@ void __vgic_put_lpi_locked(struct kvm *kvm, struct vgic_irq *irq) return; list_del(&irq->lpi_list); + xa_erase(&dist->lpi_xa, irq->intid); dist->lpi_list_count--; kfree(irq); diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h index 8cc38e836f54..795b35656b54 100644 --- a/include/kvm/arm_vgic.h +++ b/include/kvm/arm_vgic.h @@ -13,6 +13,7 @@ #include #include #include +#include #include #include #include @@ -275,6 +276,7 @@ struct vgic_dist { /* Protects the lpi_list and the count value below. */ raw_spinlock_t lpi_list_lock; + struct xarray lpi_xa; struct list_head lpi_list_head; int lpi_list_count; From patchwork Tue Feb 13 09:32:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 200287 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp423990dyb; Tue, 13 Feb 2024 01:35:34 -0800 (PST) X-Google-Smtp-Source: AGHT+IHytzSH4xyggtyNessiXAUCUDZi6DxaLucLJgLKkElYwfi6i91nKZQoToWLc29/ENUfiVw6 X-Received: by 2002:a05:6871:729:b0:219:3447:6ba6 with SMTP id f41-20020a056871072900b0021934476ba6mr12474335oap.47.1707816933847; Tue, 13 Feb 2024 01:35:33 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707816933; cv=pass; d=google.com; s=arc-20160816; b=E9+qXPIwsSPHLjI9Rm39S1ahP3eUVYLAw4B2Ck/fUfOFCfxT9Tx+Olb7yuNVPpoHm/ NSqxCB/fLcHfqBbNNCA4gMBG9SYzIqWO6uhZfBKLVNGCv730bAZVLhFBJockYztYLvi0 uwAiHqVg79yjb+wXs/kKskqIi1XGnoPgjT0xh9Sqct7mpv1X6FBRJP/xCiEJZvSr+v3Y I/XJD+E7N3zAuPKk2NNP0asbC3cSPi3Naj4m4ac0CPLDTYOahGMpBjOib1gGSMfhoYxt k5ZMtmCcgzTX8xxH+VZgWNtReMHW0dJdOh8Pf1fVmtAg2c58kpgTIBa8XxYPAB5FEqSQ 0euQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=MN4YT9WjubbgV9vwymCrG2zbA0P41iVcrtZBfJr5OD0=; fh=uA7GYO7f+twxVA127KUUkobskh0ga5Sb6rstslx7gBw=; b=kcpsJ6ffQ+NBJc9AbudJPzy6olAXiuAZett9VWLnnnTTWKyv0G6KBoEC5W63ptUBhs P6oZkOz3lXsOqglQZxKwObx1kXEIN2lnlUdZJg8k2OpnGVu9HrUtQPE3Awv7tt2zYT5u q/FaB/VKmb82JyaTk+ni+eSAPdX08oG7mduQ5ule9q27SJ1y03sO2xpEY+AU71IqPvr9 7Qns+5aJGaZITuGomwiHQ5oPJUro7YsHOeZw28fuNMvlEZwmuZ9Cx/JFSj6wuKsiV//f fMEg9ynmFBITnN+fGDm2xRdK9QjwA6IM46xPgHrwmFAgb4nRkq1plHerdZPsl0M8NpM7 8tMw==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=C7HSnkXR; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63208-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63208-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev X-Forwarded-Encrypted: i=2; AJvYcCXuBdleyNQq6kb1Dsd4exK7vW/bbzs0DE6Qv8y56Pd0ZawLfraMKT9Jgkz8LLb7doBJSPUCTX9AQ25IDLnmLVXamp2sdw== Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id bq13-20020a056a02044d00b005d7a13d0be6si1837518pgb.232.2024.02.13.01.35.33 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 01:35:33 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-63208-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=C7HSnkXR; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63208-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63208-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 37A3228700D for ; Tue, 13 Feb 2024 09:35:14 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 94E7334550; Tue, 13 Feb 2024 09:33:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="C7HSnkXR" Received: from out-183.mta1.migadu.com (out-183.mta1.migadu.com [95.215.58.183]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1DC85225D7 for ; Tue, 13 Feb 2024 09:33:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.183 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707816803; cv=none; b=kTneMpRHw8jja/y4W4rZeaSVY0nF5qk1miMMwi4sPsE4XQOJTrMEpN4vRh05mPJ+dalxPxbFfFV+qrnw0DoaZVrVl8WRICw0LsVwrr6RcXWui3XxKA0GMs3HypR0vLQ2bLZyVDr9WrQMW5PkXdrV5EcBJ7scxQR9yPIIaIOX6SE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707816803; c=relaxed/simple; bh=4brcHvDAVkgAUVv+EhHLpJGf3bgOIUhxLJMTpEy7pkA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Q8sLUztp2/o2s22TI83H0hxi8J0aoF2kOsJhiiqjdVYJOdK97c/2MnKqAr9GtbDYMvNp2kVZws/wW1h/vMMThu3v5jKEPIRKlnSf2Hkpu/guB1OBrYSZ9NZ+phi6xb6DXiMTCaFJ8t0KiEKAc7vv1Zq58IoLx5rTVAJP9JgGV0g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=C7HSnkXR; arc=none smtp.client-ip=95.215.58.183 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1707816798; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MN4YT9WjubbgV9vwymCrG2zbA0P41iVcrtZBfJr5OD0=; b=C7HSnkXROz1+UoFZtjeg4W2NSgOZRroDA00Cdsa98sNQuumqq4yrXfLeNfcXS5vMpujcKT OmC+dNgbyNxY5y311yrx0WTver+2ok75zp+yixGCV9iajwZaZBpvbGmw7RTG9c6dzlLqgO 71HwURum7tLyeuMsoVyAtjOEU1Opz+Y= From: Oliver Upton To: kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , linux-kernel@vger.kernel.org, Oliver Upton Subject: [PATCH v2 03/23] KVM: arm64: vgic: Use xarray to find LPI in vgic_get_lpi() Date: Tue, 13 Feb 2024 09:32:40 +0000 Message-ID: <20240213093250.3960069-4-oliver.upton@linux.dev> In-Reply-To: <20240213093250.3960069-1-oliver.upton@linux.dev> References: <20240213093250.3960069-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790775848827148553 X-GMAIL-MSGID: 1790775848827148553 Iterating over the LPI linked-list is less than ideal when the desired index is already known. Use the INTID to index the LPI xarray instead. Signed-off-by: Oliver Upton --- arch/arm64/kvm/vgic/vgic.c | 19 +++++-------------- 1 file changed, 5 insertions(+), 14 deletions(-) diff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c index c126014f8395..d90c42ff051d 100644 --- a/arch/arm64/kvm/vgic/vgic.c +++ b/arch/arm64/kvm/vgic/vgic.c @@ -54,8 +54,9 @@ struct vgic_global kvm_vgic_global_state __ro_after_init = { */ /* - * Iterate over the VM's list of mapped LPIs to find the one with a - * matching interrupt ID and return a reference to the IRQ structure. + * Index the VM's xarray of mapped LPIs and return a reference to the IRQ + * structure. The caller is expected to call vgic_put_irq() later once it's + * finished with the IRQ. */ static struct vgic_irq *vgic_get_lpi(struct kvm *kvm, u32 intid) { @@ -65,20 +66,10 @@ static struct vgic_irq *vgic_get_lpi(struct kvm *kvm, u32 intid) raw_spin_lock_irqsave(&dist->lpi_list_lock, flags); - list_for_each_entry(irq, &dist->lpi_list_head, lpi_list) { - if (irq->intid != intid) - continue; - - /* - * This increases the refcount, the caller is expected to - * call vgic_put_irq() later once it's finished with the IRQ. - */ + irq = xa_load(&dist->lpi_xa, intid); + if (irq) vgic_get_irq_kref(irq); - goto out_unlock; - } - irq = NULL; -out_unlock: raw_spin_unlock_irqrestore(&dist->lpi_list_lock, flags); return irq; From patchwork Tue Feb 13 09:32:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 200286 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp423903dyb; Tue, 13 Feb 2024 01:35:17 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCV++upSMmuSmwa9E/EQROQobNx2IfXHICFsTpQVhoGlycWdENOjEsIhOX+/5scMnWxB/MWwm8KdObyoCG6Eade4yL6uqA== X-Google-Smtp-Source: AGHT+IFG8OJL33i3CZMyYil9iEuJ/DpFnAgnacmgwsNvEonSfmYfC6TsMAQlYeVuUmZAcn9Lotpp X-Received: by 2002:a17:906:7fd7:b0:a38:916e:a4fb with SMTP id r23-20020a1709067fd700b00a38916ea4fbmr7390648ejs.69.1707816917592; Tue, 13 Feb 2024 01:35:17 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707816917; cv=pass; d=google.com; s=arc-20160816; b=Ks7+3skO22y0E9t7ovnZosGOGugCDPjg8B8TQlNZfgH7j5xvQsquSnoFYj1TV2F8il xQFUC6courgu57QCYRfz6AnpMjUJvUpC2JqVwk46DGBxi2Z/ttffHuNXmEsCCEFiv9Ok 7tzw6Sg+TrhbQEWAaqAUrHSPElbH8rYw+mHsmZE7JS/LAlqVzlruOIERDcYUFR/M8d+Z dxLFXTvOKL/KrCDJNvu+yydU0VOyTzjAkgslCw/xOG68vMRXyx1n6EuwJSS8W0u8eOCA Ue3RR5DHhta63t5dKgB+0XO7tlVteSZrSp2PbIShzusnyyOWAFUqPfZicaSqP1a6M8qW a+qQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=vHotxBfo61Yf7VQTBp6voheJpujlHBDNO8X85Tkwjh0=; fh=wFACL8wzc6lWd12BgWZoLxWpvtllGvFLJCbEcf8Etq4=; b=sXGp8IrKADQDbn48WNegBwoeV0n3fH2BnFrEbSZiPzq0bwXC5vQt81Ry5JSBuxKLID 30yVCZQNsh3XvZ9TJN7pgxgcnbzP/kSPQbpLT3zXh+Y41B3Nuf3qEmNWYYqmgOXfcGGP jR1ZTv3NG/0A3CHPZGRiwdoUqKJgm+G2RUUXwyj8H9t0f5yTONt2gCtJsKBMIgs3Gx6S VHi8s5VrOD2iuYIwAB97dwWSfD/zFFuEpNScg1y0iSKAzPO8JOupXzTwBTBKnFV1aSHX WB0vgk6sD1+ipKDVXjzZ2ouc9zSla0oAu8Uzc+F3zGNSfIQ3Ei/ogpU3uYkenmDhhoLl +ONA==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=Ycl6rikb; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63209-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63209-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev X-Forwarded-Encrypted: i=2; AJvYcCUexo03VdrCiVJnCO52IzVHf2NgWMh94A0v4cLvALn/ofcvTn66pZuwBaddv/HPDfdSmzhXfufGLZhmcBbYLRcXeGk6hQ== Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id g25-20020a170906199900b00a3c476d7d61si1023808ejd.528.2024.02.13.01.35.17 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 01:35:17 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-63209-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=Ycl6rikb; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63209-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63209-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 360901F254CB for ; Tue, 13 Feb 2024 09:35:17 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id B22A9364B3; Tue, 13 Feb 2024 09:33:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="Ycl6rikb" Received: from out-174.mta1.migadu.com (out-174.mta1.migadu.com [95.215.58.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9841123774 for ; Tue, 13 Feb 2024 09:33:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707816803; cv=none; b=clduAflXqjQFB7ljz0PKZxONJZYRkds6xdMNsUsFnMPfsK98aHxCbXbdMoV2gTBRs5jUYmX18n2nbdUkHInfjr6U2bpT7pktdtcubTEeHCKkTji4C4o84gM8S87ldY9Bj5+u+iPsnEDtYni2PJ5/wT07tLNh9+RiMjcqavs8qtI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707816803; c=relaxed/simple; bh=42EjqTJnQjPHCRZ1CTq/hh9Jo/BrHMISVzmWYzrxSIE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=dUI4K+v8ZyLXLoshpiJoRTiKhp9lth6+Isu5ISxRcG5i7PfOwXSx1hJYLmlPurOBiZdjbdv6aQ54iYxNrRqHwS123Tn8nwV1tu7QhlXuGBNOx35DFtVtgY0yxFkvvc6gHvE8ef/4WPuUMy/Pp+tPQGYXcU0ybi5pfPIIRt7JBrw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=Ycl6rikb; arc=none smtp.client-ip=95.215.58.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1707816800; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vHotxBfo61Yf7VQTBp6voheJpujlHBDNO8X85Tkwjh0=; b=Ycl6rikbVAanKCNVreZMOwzDdQAIGD1dS+LPBIe7vQ1Sq5OcJDWhFjgF6M0OGCEBYC8Y7K ur7EsSz3WhyVCtrbPd2w9FWmNVdhshL4P9hcDsTwqyO5b4Che0EBVeFD9PRcxND2ZPnec+ cRTQI5/mj2t502UjVNydFjb1qWzhh50= From: Oliver Upton To: kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , linux-kernel@vger.kernel.org, Oliver Upton Subject: [PATCH v2 04/23] KVM: arm64: vgic-v3: Iterate the xarray to find pending LPIs Date: Tue, 13 Feb 2024 09:32:41 +0000 Message-ID: <20240213093250.3960069-5-oliver.upton@linux.dev> In-Reply-To: <20240213093250.3960069-1-oliver.upton@linux.dev> References: <20240213093250.3960069-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790775832134204168 X-GMAIL-MSGID: 1790775832134204168 Start walking the LPI xarray to find pending LPIs in preparation for the removal of the LPI linked-list. Note that the 'basic' iterator is chosen here as each iteration needs to drop the xarray read lock (RCU) as reads/writes to guest memory can potentially block. Signed-off-by: Oliver Upton --- arch/arm64/kvm/vgic/vgic-v3.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kvm/vgic/vgic-v3.c b/arch/arm64/kvm/vgic/vgic-v3.c index 9465d3706ab9..4ea3340786b9 100644 --- a/arch/arm64/kvm/vgic/vgic-v3.c +++ b/arch/arm64/kvm/vgic/vgic-v3.c @@ -380,6 +380,7 @@ int vgic_v3_save_pending_tables(struct kvm *kvm) struct vgic_irq *irq; gpa_t last_ptr = ~(gpa_t)0; bool vlpi_avail = false; + unsigned long index; int ret = 0; u8 val; @@ -396,7 +397,7 @@ int vgic_v3_save_pending_tables(struct kvm *kvm) vlpi_avail = true; } - list_for_each_entry(irq, &dist->lpi_list_head, lpi_list) { + xa_for_each(&dist->lpi_xa, index, irq) { int byte_offset, bit_nr; struct kvm_vcpu *vcpu; gpa_t pendbase, ptr; From patchwork Tue Feb 13 09:32:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 200288 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp424151dyb; Tue, 13 Feb 2024 01:35:53 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCUTtMsukBYRpvQFhdTz99z3THDEiICanzi74h75ril/dPWs5uxDbj6DGL8uyjz668m9sYYT+lKt2k50KU8lj2m+drZmDQ== X-Google-Smtp-Source: AGHT+IEH5n8dDIl6HDgdtCGvxvwBO0gbHqRB4OoQeAEvYQ9Y5xCFY/IGY8sK4+yzWXJLh+kHkPzw X-Received: by 2002:a81:48c5:0:b0:5df:4993:4371 with SMTP id v188-20020a8148c5000000b005df49934371mr5814806ywa.39.1707816953079; Tue, 13 Feb 2024 01:35:53 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707816953; cv=pass; d=google.com; s=arc-20160816; b=MpE1YUAPlmq7WhMxC+rj2dH2MWrRbjyoazkT7aqan1z4g9DzXfxYKDmsHFbahx+J1y KYe+PTYbHydFZPOTeTtdMyJ6G839GBv5diwKLgAMpPw5XZ0mwn0amYVE5dOBo3v5xpPQ LQ7JjnLnGnVjI/KLvc30edFIByeqSOMe4kU6e+zWe6FS6bklU6sU0LElkgh3GPQAuhRT UTrMpQypH6nJIJCIUwsSlbgbV7IHnOq2Gw8daG0Do6qeSIhFp1rbgrHFrxUDtK2L0yTt JqY1JVxxsHbGLvp+tjpDOtgzWJaxX7hLEiMk20/FdVRbsoIItVplbLU/enwl8zmgT/Ue ESEw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=D7DiWlMgiNfAagj80AqXky3JLRxK4Eu9kOYloGceSKs=; fh=ol1R5JcNH1rQ60M5cactUf+O5Se/SHhFCWafidUC/NE=; b=Pkj0DJ8XSMRRLXGxszsMMBFrlkWjnl6VY0dRlqdbKVVqRHwpjotbbQWoV7nlGdhEtu naZFtjQ1yr29CWmStM9rAdqxGO7NCnOAE1U/S7Hk1OWA7YcBJGWsuTYnF7pzoHL9BI0i pEf+FVlaBbwYKAn8xMWQxpNDZ3fFoxwF8hzRE/1hYDpepre4jh+L4Z1ghJCeDN38rRIf dsPwpGKgdKHnc9ZLzAD5ylb5Phhj2LtG7Veelb6fujEZ9DPU8FNNUGRG4sNUh/6Gm+Zg oE0R+yU2ISFEL7qOv2PZR+Van8wNzgSQdxadc1GlJT8xgn/mJs+t4jQ0LrHaHAgH0czO sa1g==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=tHo0sLNa; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63210-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63210-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev X-Forwarded-Encrypted: i=2; AJvYcCV+jMQ/sprSErrPBUbhKZ4br8K6oNTPDXuHjr5+L2j6zmiVjajr7oaczrqQ2+qZozGKbk3K/NDj7LvL/Dt6mQRR5xF3gQ== Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id j14-20020ac8664e000000b0042c5d1612a1si2328955qtp.152.2024.02.13.01.35.52 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 01:35:53 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-63210-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=tHo0sLNa; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63210-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63210-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id D85A11C21FB4 for ; Tue, 13 Feb 2024 09:35:52 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 2D7B5383BA; Tue, 13 Feb 2024 09:33:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="tHo0sLNa" Received: from out-170.mta1.migadu.com (out-170.mta1.migadu.com [95.215.58.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4F16F1B273 for ; Tue, 13 Feb 2024 09:33:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707816806; cv=none; b=iWr79amWeJOiuuobV4vXnmdZSGgR4PmNSrWT0ZVKOtVfjA5Ens7R8YsT2r7U0FPCRWG65aDkbUkzcZhBJ2Lsxv71ccxlncfODPJ0ohCpB1vun6TgwZnODoV3nfd/VAaURmW7X9nEzXTABXc3k+Hx667go/OzD1EzKY6tX78MIho= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707816806; c=relaxed/simple; bh=7gCAtwCAzOxOThnwdigYSe669AuhD7GmWusCH1zdeDk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=eOqw+WmP7d0mvb/zkPDsw5LnFHIg4HYP8pGOWdC2OQQHyhpb/8RwVlKdbNJQhINqXyUwsM6QVAs7O4yyu6+53D2UxzQ/pZW7/i7Xh3XB+Um/U2HUEi8CY6junaITtwihh23VUop5uJylNN/csJiF4PTULOh4ilTSelIRZMsnoHE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=tHo0sLNa; arc=none smtp.client-ip=95.215.58.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1707816801; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=D7DiWlMgiNfAagj80AqXky3JLRxK4Eu9kOYloGceSKs=; b=tHo0sLNacpdzPwUWnZwC01vVZXsuonkEzA+krHUVpDXM46Dk5TLIcH3L7wPG0wRCGJl+OF l3em5KbB5DskRHC5tKe+SwvpKj7CqjkXoLykd9bUfbnijUkt/J79Nw1w0B2/1VsduejFDq DXas+JptrfVm6c8f9wt1uhKnDhAFIqQ= From: Oliver Upton To: kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , linux-kernel@vger.kernel.org, Oliver Upton Subject: [PATCH v2 05/23] KVM: arm64: vgic-its: Walk the LPI xarray in vgic_copy_lpi_list() Date: Tue, 13 Feb 2024 09:32:42 +0000 Message-ID: <20240213093250.3960069-6-oliver.upton@linux.dev> In-Reply-To: <20240213093250.3960069-1-oliver.upton@linux.dev> References: <20240213093250.3960069-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790775869233964329 X-GMAIL-MSGID: 1790775869233964329 Start iterating the LPI xarray in anticipation of removing the LPI linked-list. Signed-off-by: Oliver Upton --- arch/arm64/kvm/vgic/vgic-its.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kvm/vgic/vgic-its.c b/arch/arm64/kvm/vgic/vgic-its.c index 0265cd1f2d6e..aefe0794b71c 100644 --- a/arch/arm64/kvm/vgic/vgic-its.c +++ b/arch/arm64/kvm/vgic/vgic-its.c @@ -336,6 +336,7 @@ static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq, int vgic_copy_lpi_list(struct kvm *kvm, struct kvm_vcpu *vcpu, u32 **intid_ptr) { struct vgic_dist *dist = &kvm->arch.vgic; + XA_STATE(xas, &dist->lpi_xa, GIC_LPI_OFFSET); struct vgic_irq *irq; unsigned long flags; u32 *intids; @@ -354,7 +355,9 @@ int vgic_copy_lpi_list(struct kvm *kvm, struct kvm_vcpu *vcpu, u32 **intid_ptr) return -ENOMEM; raw_spin_lock_irqsave(&dist->lpi_list_lock, flags); - list_for_each_entry(irq, &dist->lpi_list_head, lpi_list) { + rcu_read_lock(); + + xas_for_each(&xas, irq, INTERRUPT_ID_BITS_ITS) { if (i == irq_count) break; /* We don't need to "get" the IRQ, as we hold the list lock. */ @@ -362,6 +365,8 @@ int vgic_copy_lpi_list(struct kvm *kvm, struct kvm_vcpu *vcpu, u32 **intid_ptr) continue; intids[i++] = irq->intid; } + + rcu_read_unlock(); raw_spin_unlock_irqrestore(&dist->lpi_list_lock, flags); *intid_ptr = intids; From patchwork Tue Feb 13 09:32:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 200296 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp427002dyb; Tue, 13 Feb 2024 01:43:30 -0800 (PST) X-Google-Smtp-Source: AGHT+IFl4ttsobEWgBehXqDeN6kto9UpUicVIvKna+SZbMphAsuWpHqJ5TZCsZmhPWTYRMnX4pWm X-Received: by 2002:a92:d3c1:0:b0:363:cc7d:102c with SMTP id c1-20020a92d3c1000000b00363cc7d102cmr10482028ilh.10.1707817410433; Tue, 13 Feb 2024 01:43:30 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707817410; cv=pass; d=google.com; s=arc-20160816; b=VWBYGrBBoCtds46M+iD2jFhgoRRJFkvQoMpM3LKZY98b5JObaOlqhGdal5O8C6H+Rr XT9CVfsWycq3AwSAcVwQiNoD92hsC+u0pby3zwK8jp1M5A9gQaQc6TOOgOi7MAeJBijY Vrz+uHg7pIFEnspx/wt4Go5PmEYTl1VeDfy1rlZsBI6yyFTf41bZHB6L782ZyRdWNW0j y2izadPXSs/YTc/Mm7A6YfRA10IuJ253rfeTmQDIQSQ7hKYTNxMA1tQD7dn8mQSK03qV BORmA1e8SH1aRXWxFmOMW3db3gbWz+MbQZI8c96u3lu0+MuPD4yY8i5iu+nKmuFKFERt 2Qkg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=d0pU/1urqxYilVj53Ip1KYtHI35wj+PNtuLJSjy+uYo=; fh=Qu6hlNfzN49lffehpLlumtI2GzgZ0M5+jMg9OoN6y4w=; b=M7faJ3TkXpZtrQ4P2yNqmD8AX4OCPwPTmPsVUqKIFgo2bHCc9zHvD1pyo1EltJicEU 2V2iGj81gDG7Is/NvJkBsMw0IdNlOXDXz9h9LF0GNPyPaw+4VRJHhJuffV6mjf8bCATf 2w4zcxzElk4AAHR3a/23dFtDcTPDhHn5y3/r2chsIzHRZBoQJx8LiQz60bQgOH4vEs4z /4zRDvMIITWWUSNtHNZzxVH57IAqFwXbYF8UNp/t+Q47ODXa+HkmmjCMhghkTyFab4FT YJ4501HOnyxkbrJNROz1Nm+VN7BMqR8iIOBsQvHv9Cc8XIuF+yjoRvKhE01IzK0z77O8 eR4A==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=vMfTEbmx; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63211-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63211-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev X-Forwarded-Encrypted: i=2; AJvYcCWHFs2uXDn8QQ8UWw/jnxh1dleplUL75KsV6u99Lys1nse7BtuRxrj1muDuISNbiMKr5Qxlcmitz2ogfTFzNY7x/XRkgA== Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id i10-20020a65484a000000b005dc5070efecsi1726265pgs.825.2024.02.13.01.43.30 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 01:43:30 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-63211-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=vMfTEbmx; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63211-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63211-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 1F515B25ECD for ; Tue, 13 Feb 2024 09:36:06 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id E6A4638FB9; Tue, 13 Feb 2024 09:33:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="vMfTEbmx" Received: from out-179.mta1.migadu.com (out-179.mta1.migadu.com [95.215.58.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2CAB62BD00 for ; Tue, 13 Feb 2024 09:33:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707816807; cv=none; b=ax8DMvdfX6WGEJVLqIk0pTHKII6Z1YSQpqONBFMgI4xL6l3dTlVgxIe/k+/ht7wwyf6A3xxnRWMjVB3t9iqRrStbly3NE5ckjzrUww9NEEZHLgI73dBs5/O5thZnjxQ82uQXfxjU1/NSFwmR5Udl5Ftofl7KaG6Wu3NmFLnmWqo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707816807; c=relaxed/simple; bh=x1S98sHSYhxkabprmCbrd2K4IpsVQD88YKDf/wPFgQo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=EpSuRjmjLV+rSKzdCJAe1BqmDQX3Ha0fzckhM9rXfdRnPR2ipFN0fulXbD3E2m+LSOpiAtFL7NkTTW6QFkODr/TH/xftfPZ5hrVJEoPGXJk2vm6sd5lwW9DnLXQww+CbgNxUxYVS249YVFM1N3FHdomNkWL2bIzsSlL8pZt929s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=vMfTEbmx; arc=none smtp.client-ip=95.215.58.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1707816803; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=d0pU/1urqxYilVj53Ip1KYtHI35wj+PNtuLJSjy+uYo=; b=vMfTEbmx3n8BntxZAxz615n3jMiuLC40HjUl/AUUTd+3ex7zjkcBSO+0xpSIr4MS6fyp6m dsXQMZ5FGNAts1rcmS2qQIZvdQn+8psj5XEPX6TVwyOUjVyx2/eDof6AD+yywnBWA3Gvhd ZtEh0zYClbkXgAGauoANm3irPT7JgbY= From: Oliver Upton To: kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , linux-kernel@vger.kernel.org, Oliver Upton Subject: [PATCH v2 06/23] KVM: arm64: vgic: Get rid of the LPI linked-list Date: Tue, 13 Feb 2024 09:32:43 +0000 Message-ID: <20240213093250.3960069-7-oliver.upton@linux.dev> In-Reply-To: <20240213093250.3960069-1-oliver.upton@linux.dev> References: <20240213093250.3960069-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790776348726367631 X-GMAIL-MSGID: 1790776348726367631 All readers of LPI configuration have been transitioned to use the LPI xarray. Get rid of the linked-list altogether. Signed-off-by: Oliver Upton --- arch/arm64/kvm/vgic/vgic-init.c | 1 - arch/arm64/kvm/vgic/vgic-its.c | 8 ++------ arch/arm64/kvm/vgic/vgic.c | 1 - include/kvm/arm_vgic.h | 2 -- 4 files changed, 2 insertions(+), 10 deletions(-) diff --git a/arch/arm64/kvm/vgic/vgic-init.c b/arch/arm64/kvm/vgic/vgic-init.c index 411719053107..e25672d6e846 100644 --- a/arch/arm64/kvm/vgic/vgic-init.c +++ b/arch/arm64/kvm/vgic/vgic-init.c @@ -53,7 +53,6 @@ void kvm_vgic_early_init(struct kvm *kvm) { struct vgic_dist *dist = &kvm->arch.vgic; - INIT_LIST_HEAD(&dist->lpi_list_head); INIT_LIST_HEAD(&dist->lpi_translation_cache); raw_spin_lock_init(&dist->lpi_list_lock); xa_init_flags(&dist->lpi_xa, XA_FLAGS_LOCK_IRQ); diff --git a/arch/arm64/kvm/vgic/vgic-its.c b/arch/arm64/kvm/vgic/vgic-its.c index aefe0794b71c..c68164d6cba0 100644 --- a/arch/arm64/kvm/vgic/vgic-its.c +++ b/arch/arm64/kvm/vgic/vgic-its.c @@ -59,7 +59,6 @@ static struct vgic_irq *vgic_add_lpi(struct kvm *kvm, u32 intid, return ERR_PTR(ret); } - INIT_LIST_HEAD(&irq->lpi_list); INIT_LIST_HEAD(&irq->ap_list); raw_spin_lock_init(&irq->irq_lock); @@ -75,10 +74,8 @@ static struct vgic_irq *vgic_add_lpi(struct kvm *kvm, u32 intid, * There could be a race with another vgic_add_lpi(), so we need to * check that we don't add a second list entry with the same LPI. */ - list_for_each_entry(oldirq, &dist->lpi_list_head, lpi_list) { - if (oldirq->intid != intid) - continue; - + oldirq = xa_load(&dist->lpi_xa, intid); + if (oldirq) { /* Someone was faster with adding this LPI, lets use that. */ kfree(irq); irq = oldirq; @@ -100,7 +97,6 @@ static struct vgic_irq *vgic_add_lpi(struct kvm *kvm, u32 intid, goto out_unlock; } - list_add_tail(&irq->lpi_list, &dist->lpi_list_head); dist->lpi_list_count++; out_unlock: diff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c index d90c42ff051d..e58ce68e325c 100644 --- a/arch/arm64/kvm/vgic/vgic.c +++ b/arch/arm64/kvm/vgic/vgic.c @@ -121,7 +121,6 @@ void __vgic_put_lpi_locked(struct kvm *kvm, struct vgic_irq *irq) if (!kref_put(&irq->refcount, vgic_irq_release)) return; - list_del(&irq->lpi_list); xa_erase(&dist->lpi_xa, irq->intid); dist->lpi_list_count--; diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h index 795b35656b54..aeff363e3ba6 100644 --- a/include/kvm/arm_vgic.h +++ b/include/kvm/arm_vgic.h @@ -117,7 +117,6 @@ struct irq_ops { struct vgic_irq { raw_spinlock_t irq_lock; /* Protects the content of the struct */ - struct list_head lpi_list; /* Used to link all LPIs together */ struct list_head ap_list; struct kvm_vcpu *vcpu; /* SGIs and PPIs: The VCPU @@ -277,7 +276,6 @@ struct vgic_dist { /* Protects the lpi_list and the count value below. */ raw_spinlock_t lpi_list_lock; struct xarray lpi_xa; - struct list_head lpi_list_head; int lpi_list_count; /* LPI translation cache */ From patchwork Tue Feb 13 09:32:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 200297 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp427107dyb; Tue, 13 Feb 2024 01:43:49 -0800 (PST) X-Google-Smtp-Source: AGHT+IHuFIaSOP26iRAF+z5INgGMKA94WHYmSYCwbsvLE8AqNm0Dcqtw41AZuy2ycQd7YJ+Re6US X-Received: by 2002:a17:90b:4ece:b0:298:a3e4:d6b3 with SMTP id ta14-20020a17090b4ece00b00298a3e4d6b3mr1170798pjb.5.1707817428974; Tue, 13 Feb 2024 01:43:48 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707817428; cv=pass; d=google.com; s=arc-20160816; b=gTQW7P0ILD9TSypQwzyzSKTbphrDRVJ1Ms9E6f/eHXgMlRhfQFmEG0NumeSWcrGdra rvQDI90Z6kQmTsDkvVLz6tLQg1S4+JSMMGrlJjSGKdIouPp8EaJMu04MaQL1vHzMw4Ch sLtYQB0M002BifAL1NKaEwDAD4MMW9cUfMPwfb+JrqtgKb6TJfRNOGY+uBGIalotUuTp SUnopn5YbASdjlgnw5VvV4nQH2KcFDMAFs333r3ItsDeA4wcYYRyC3WcCniyp+I8Neqp hH/5wL7SgTjePk/h+lKifbRSQyd2KMbgHD14h6VO+dRI4bOI/84LU5kMScdEGtQMlyc4 dQ4A== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=nQ22xtb57Hhh6ukA9UKpHbyaiDwLVQOEd0N6U2bxudQ=; fh=8KOlWUfB6ku5hkQYwHxYJ/+bS4dRLEh4Bb52VJQgfCE=; b=fv25fQKyWXDmr16aSnz8XNLIihNTcA+RW9SyMJuIKrKuH+WeFdfaBSU+dMG82gAxtg mp9+AaOKW2vrrAkY8AKkQkmlWXiNQWXS8rQLUt7eejiCWcn3YhmiPjSrUFeJwjX1NzFN XY2tb8I4Y/N8nhi2gQrPJrIjAqe8aIZjlDLpJ6qA5B97wAJpkIqToSKPILaw4KkK+UDi xFMITjWbfUFpfqUPnWYaq5+8m0CgLi51mPcILvVLwgDnIESBYZ4Ex9diVPDepBSIKm81 p+EFWmcTf1wXnuRhtQ17N9mJTigMQ8ExYI1LIL1oWjmMKtNEFjcxaxIvK/e20YRwHwlR LGvQ==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=lQLrteem; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63212-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63212-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev X-Forwarded-Encrypted: i=2; AJvYcCWLJN4ZuM4sAc91gOpy9XKt7zj3UqcrtmY63Dg/eU2AiEbfn0gv3isWI1N7ZS7C5wc37sgD/1ysEFBhEBJj35fQxWXrIA== Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [147.75.48.161]) by mx.google.com with ESMTPS id q1-20020a656a81000000b005dc833ef4e6si1034548pgu.75.2024.02.13.01.43.48 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 01:43:48 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-63212-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=lQLrteem; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63212-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63212-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 2E90AB25F6A for ; Tue, 13 Feb 2024 09:36:22 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 2B8223D3A4; Tue, 13 Feb 2024 09:33:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="lQLrteem" Received: from out-185.mta1.migadu.com (out-185.mta1.migadu.com [95.215.58.185]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EF7E036AE4 for ; Tue, 13 Feb 2024 09:33:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.185 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707816808; cv=none; b=X3xDu4K4T1D6l3CJlFycWvw+v7oegHBvQFI6B7mWNdMXjqxZbZQaFJuKyD7MorNLxW+PeinKMXxGW0X350v+xEwn5cMst+KV/izvywtvPbLd7Dz/itbLQ+HXxr4y2mXLRo/7f42BfBeAbiVg9duochqDuJOSLBmyHFk284ZeKS8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707816808; c=relaxed/simple; bh=K1BXK5oYZl6+dA5mTViFGwIhpj458PXsoVl/Nu1w2ig=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qof9YXqfGUgtU+f1na1YDq15ChU0T9Q/aUcMvGzGs8gd83+a5QFVeUflS1MEgvY1pxCyKUHTc5TwaC1YmpPhJSRcbEPjIGBlfUGjyocJwZb7aPoS9w1lCEBa0emIHdxe3LUfX32HmUIiDjp7rXVOEkzphdC2WFn344f8VjQJQtQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=lQLrteem; arc=none smtp.client-ip=95.215.58.185 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1707816805; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nQ22xtb57Hhh6ukA9UKpHbyaiDwLVQOEd0N6U2bxudQ=; b=lQLrteem+/oNETa88tCfy93s3hWpOQjhHY9udjC6jjluC/ugre068aS/g+cCHk38wmFIsT llzxysWLqD6W5XXtdWNc3eO5DfxFGCce105uksr2XMisqueJ+4opUXNBfUW4IqJp1zCYUn AluLUWq2PU2m0JaY0lo+9urNUaCPJVg= From: Oliver Upton To: kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , linux-kernel@vger.kernel.org, Oliver Upton Subject: [PATCH v2 07/23] KVM: arm64: vgic: Use atomics to count LPIs Date: Tue, 13 Feb 2024 09:32:44 +0000 Message-ID: <20240213093250.3960069-8-oliver.upton@linux.dev> In-Reply-To: <20240213093250.3960069-1-oliver.upton@linux.dev> References: <20240213093250.3960069-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790776368403555513 X-GMAIL-MSGID: 1790776368403555513 Switch to using atomics for LPI accounting, allowing vgic_irq references to be dropped in parallel. Signed-off-by: Oliver Upton --- arch/arm64/kvm/vgic/vgic-debug.c | 2 +- arch/arm64/kvm/vgic/vgic-its.c | 4 ++-- arch/arm64/kvm/vgic/vgic.c | 2 +- include/kvm/arm_vgic.h | 4 ++-- 4 files changed, 6 insertions(+), 6 deletions(-) diff --git a/arch/arm64/kvm/vgic/vgic-debug.c b/arch/arm64/kvm/vgic/vgic-debug.c index 85606a531dc3..389025ce7749 100644 --- a/arch/arm64/kvm/vgic/vgic-debug.c +++ b/arch/arm64/kvm/vgic/vgic-debug.c @@ -149,7 +149,7 @@ static void print_dist_state(struct seq_file *s, struct vgic_dist *dist) seq_printf(s, "vgic_model:\t%s\n", v3 ? "GICv3" : "GICv2"); seq_printf(s, "nr_spis:\t%d\n", dist->nr_spis); if (v3) - seq_printf(s, "nr_lpis:\t%d\n", dist->lpi_list_count); + seq_printf(s, "nr_lpis:\t%d\n", atomic_read(&dist->lpi_count)); seq_printf(s, "enabled:\t%d\n", dist->enabled); seq_printf(s, "\n"); diff --git a/arch/arm64/kvm/vgic/vgic-its.c b/arch/arm64/kvm/vgic/vgic-its.c index c68164d6cba0..048226812974 100644 --- a/arch/arm64/kvm/vgic/vgic-its.c +++ b/arch/arm64/kvm/vgic/vgic-its.c @@ -97,7 +97,7 @@ static struct vgic_irq *vgic_add_lpi(struct kvm *kvm, u32 intid, goto out_unlock; } - dist->lpi_list_count++; + atomic_inc(&dist->lpi_count); out_unlock: if (ret) @@ -345,7 +345,7 @@ int vgic_copy_lpi_list(struct kvm *kvm, struct kvm_vcpu *vcpu, u32 **intid_ptr) * command). If coming from another path (such as enabling LPIs), * we must be careful not to overrun the array. */ - irq_count = READ_ONCE(dist->lpi_list_count); + irq_count = atomic_read(&dist->lpi_count); intids = kmalloc_array(irq_count, sizeof(intids[0]), GFP_KERNEL_ACCOUNT); if (!intids) return -ENOMEM; diff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c index e58ce68e325c..5988d162b765 100644 --- a/arch/arm64/kvm/vgic/vgic.c +++ b/arch/arm64/kvm/vgic/vgic.c @@ -122,7 +122,7 @@ void __vgic_put_lpi_locked(struct kvm *kvm, struct vgic_irq *irq) return; xa_erase(&dist->lpi_xa, irq->intid); - dist->lpi_list_count--; + atomic_dec(&dist->lpi_count); kfree(irq); } diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h index aeff363e3ba6..71e9d719533b 100644 --- a/include/kvm/arm_vgic.h +++ b/include/kvm/arm_vgic.h @@ -273,10 +273,10 @@ struct vgic_dist { */ u64 propbaser; - /* Protects the lpi_list and the count value below. */ + /* Protects the lpi_list. */ raw_spinlock_t lpi_list_lock; struct xarray lpi_xa; - int lpi_list_count; + atomic_t lpi_count; /* LPI translation cache */ struct list_head lpi_translation_cache; From patchwork Tue Feb 13 09:32:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 200289 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp424462dyb; Tue, 13 Feb 2024 01:36:45 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWUWiW3x3CI8mlpjZwL694RxAOnv5g43KySdq9FKXuTj6oTwVeiGoIB8RCHt281X7RbxVJ0AnJnzGVoWpPuL/jnWLS+bg== X-Google-Smtp-Source: AGHT+IF/zEkTKznxSG0R3YU1oCgvI9SNUt9STNqWv2Q/KEXC6n6qM/fbu0mAbbp2pivEpzeoaILj X-Received: by 2002:a17:906:5798:b0:a3d:1fc:c5f7 with SMTP id k24-20020a170906579800b00a3d01fcc5f7mr687167ejq.52.1707817005672; Tue, 13 Feb 2024 01:36:45 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707817005; cv=pass; d=google.com; s=arc-20160816; b=znLRWJOPzLjgEihautYA9IQy6iMVA9Hnf3MkZ+GEL9QDNR0d2LQg9+zpjzQfzBCJh6 YK2slpw4Kn4gdfhttuw6GzM5Dna7aVAJ2EXk01LTZRfI3m7qXd3/yHKsRdffssYmyY7/ 5aq/aNuy6OPUWnG4MQl7TKqHfS3kXgkZnYBn/a2koKKBzkuBd/5Qmln6wCasaFmC/TeM Kq9SpQxAAu8ejF88z18XahAQQEjKZCfeR41F2UEfBpgvpzTUeRPKdDOpswzZxFZv+MhB xfd9K7lIjXNFRrE2SkmNc1+MngT1aZr6eTFRf9jKqmKZTSflAuMLaW1mBupyGgb/uXY7 tvnA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=TtUC9LwCBXa/InKHa3MSglqgjCcySjFJ8GApzq67bXk=; fh=bxDVlu76AACwTGl/BZc5HrEL3eHjK/J0aSXRc/93tWs=; b=XRYUVDIflleAnXB3J8eQ+lAApdtbbVCU6ht1Tlu+t91m6kK8rvilT7YTuP0Y4S0XWu e9Q989iRqKlG8XbvICgUVk+3cZ+w4F0dlK42eTZ6CiOxeWGOXPXoG0x7vL/UE4skiLUQ ZE91m0rqp/Pug3BHApFy8tc0WxqzIjdWKS8IVKHWVKJF76bbr19En2KOHWZwSrav7iXF yPgZhB+XgjNuGuKKFfH5zQS8InQtU8u7FFk4/WnDRdzSmixmcx3wTK87XBpajonJqRX7 5JnJ1K5PxM50gVwfNxjJYYxwAnB4gSvoA4NCGpx0OJgt6bcH1YGFwTyhRliOJoLzjGhA Mv0w==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=JxeGbdEa; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63213-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63213-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev X-Forwarded-Encrypted: i=2; AJvYcCUzo0/G9So4EyIt87vxOYdBXfuIcKgLbWIeLdzZWrkiB460/SNf7W9mDwEY0iAK0TJuU4+9b5ciNHmKz76K98XG1PUJTw== Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id o11-20020a1709062e8b00b00a3c2bfc9086si1025493eji.624.2024.02.13.01.36.45 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 01:36:45 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-63213-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=JxeGbdEa; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63213-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63213-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 478CB1F2177C for ; Tue, 13 Feb 2024 09:36:45 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 1710F43154; Tue, 13 Feb 2024 09:33:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="JxeGbdEa" Received: from out-189.mta1.migadu.com (out-189.mta1.migadu.com [95.215.58.189]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A6C1737715 for ; Tue, 13 Feb 2024 09:33:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.189 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707816810; cv=none; b=CVXiAfzSM3tX2AOElAR8Z5fO4u2RDx0NwroRswkgkd/b4CCkRNzCqHCpDmwgnBfwJyfSVzxvMY0NmBSDNB4cSCQvFdZXEv+lGMB0l2PzYjsAoNlNmJQCPTOPIkTbH8PcUacTJx/9GyCse4CGrOOov9wEUAInqO8U9joXfeHVRXo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707816810; c=relaxed/simple; bh=Xl8MN8nPeESY7C6rMo2OigN0MetDVjtmFaMrPpaUx6g=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=CrTjiOhLiP2Btp+WzDlYKDVuOucDWdkKafEqDbPJs5rP5jTYVMM6zzg6LpoO99GDwOLvKu5GkQjCS1p0fCKMdTz1QiK3XklNKA0FPrwBr6CqvI4bj0z5q1Rhu9kPmER3gnSrPaOCEGCwzYkgs56K25/99JKrmH4QyQVJ432r398= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=JxeGbdEa; arc=none smtp.client-ip=95.215.58.189 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1707816807; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TtUC9LwCBXa/InKHa3MSglqgjCcySjFJ8GApzq67bXk=; b=JxeGbdEawMgFP4lolaW83sT6rOVtn3Jr5awIE+IJufW0O6N7HRGIYnCCAdSYU1n3+2eSFJ 4Lal0qXRYPH/zdJ9lHg+ZsUA6r4ZMDf3+VNDIH4rnpGq1vpnwRkOQFCGELItFtMp80Tt9B QEiMbQvQ+M6qEBCO8OHFpR4xtsc+py4= From: Oliver Upton To: kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , linux-kernel@vger.kernel.org, Oliver Upton Subject: [PATCH v2 08/23] KVM: arm64: vgic: Free LPI vgic_irq structs in an RCU-safe manner Date: Tue, 13 Feb 2024 09:32:45 +0000 Message-ID: <20240213093250.3960069-9-oliver.upton@linux.dev> In-Reply-To: <20240213093250.3960069-1-oliver.upton@linux.dev> References: <20240213093250.3960069-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790775924873374519 X-GMAIL-MSGID: 1790775924873374519 Free the vgic_irq structs in an RCU-safe manner to allow reads of the LPI configuration data to happen in parallel with the release of LPIs. Signed-off-by: Oliver Upton --- arch/arm64/kvm/vgic/vgic.c | 2 +- include/kvm/arm_vgic.h | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c index 5988d162b765..be3ed4c5e1fa 100644 --- a/arch/arm64/kvm/vgic/vgic.c +++ b/arch/arm64/kvm/vgic/vgic.c @@ -124,7 +124,7 @@ void __vgic_put_lpi_locked(struct kvm *kvm, struct vgic_irq *irq) xa_erase(&dist->lpi_xa, irq->intid); atomic_dec(&dist->lpi_count); - kfree(irq); + kfree_rcu(irq, rcu); } void vgic_put_irq(struct kvm *kvm, struct vgic_irq *irq) diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h index 71e9d719533b..47035946648e 100644 --- a/include/kvm/arm_vgic.h +++ b/include/kvm/arm_vgic.h @@ -117,6 +117,7 @@ struct irq_ops { struct vgic_irq { raw_spinlock_t irq_lock; /* Protects the content of the struct */ + struct rcu_head rcu; struct list_head ap_list; struct kvm_vcpu *vcpu; /* SGIs and PPIs: The VCPU From patchwork Tue Feb 13 09:32:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 200290 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp424566dyb; Tue, 13 Feb 2024 01:37:01 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCV3+OH+Y7mk+uvqUaOgJ7bn0H+Rv42EUbPwbgb6VM3dZSwHgJtjnA/LoS8ptMjqLmlvesipgSCWuUYREaSWMUs22j0OFQ== X-Google-Smtp-Source: AGHT+IGzA7xQDqm9PyceWomMg8T8+v8KJCGDnzPjzIOmLL/40uU6wCeQm5HJVjmR2i6EJVCozRV5 X-Received: by 2002:a05:6102:c8d:b0:46e:c78e:3389 with SMTP id f13-20020a0561020c8d00b0046ec78e3389mr3381563vst.2.1707817021172; Tue, 13 Feb 2024 01:37:01 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707817021; cv=pass; d=google.com; s=arc-20160816; b=0kNouoZdKdgoqmiMSLN8LQAqTpSGlcMJ5W4ku1mv0yXNGnqP3yq/8xouIL6GHoarK3 +GsQ8q05dUlQ5n7s5yOLTNdgL/hDtG3VHH2FBcl12+BzFdU4Bu8aXykGE/AS9s/c+pFu NAUZQJuI/YRDd58WCihWyZri7RAa3HDHLu3vMRVQt5YGMBgNhChbCmZy4ZJ6eeLvluvF tUnf+EILF/A0GtEhUUViTyFNQI6isB+PiVmQfctarVZzppo3xnq+onaECBovIOOryNlT A87kAek9AiJHHB+tSFnlWOawtQltAL0+1vZg7X1l5umJhOKYhhm3SUoCDS3FrX0KYmTH anzw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=+sE7IWb8cyeSXhM2SoQx6feQUCncZFMKiDi8i8kQuog=; fh=ob//CYUHY8bJliEaSzyn3JgEK5Y9G5O5D0ocsH8rQC0=; b=Nnfjgq3+JRobU90900gkfQyuGWEwHkp2ce8J8xLblYVof7t14xpdQAi+ocud5QFKqA H9ThneGN+p3N6TQvnHxgLQ3Bc17P3Dc3WM4/taqoG5kzrfXolbXPREpUEHAHT2JF64aC xKompWLUnLwXEd3nYNfFj6h6k/CDZXSbZ4Fo70Uch9ZEg3G+RnnVZGa7cXo7qykvI48J FvW/7eWEQXYA0HbhyS8SwhhuGHjfYs3N22cIaSVr0dBUfiqLBSHXuacBpDz65CfO+CnW 6kU+h5qrSeWcS+wlXa26SQdBfrmO8ZobECdz/vsioPcM8Hg+4/9/ciZ/0tw3tgwaCifX kBFg==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=fIbqeW1Z; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63214-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63214-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev X-Forwarded-Encrypted: i=2; AJvYcCXxdrqw9euO9zAwunuIgoGvI5IHZNTOPKr47w5Tk5MpCaGCbivuS+xE8LsXWiKcd6z+fzuonL+1TrJWadDEc7w5HO7oWA== Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id q16-20020ac84510000000b0042c49437b5csi2278242qtn.39.2024.02.13.01.37.01 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 01:37:01 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-63214-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=fIbqeW1Z; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63214-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63214-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id F18AB1C2112D for ; Tue, 13 Feb 2024 09:37:00 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 7FA994DA0A; Tue, 13 Feb 2024 09:33:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="fIbqeW1Z" Received: from out-187.mta1.migadu.com (out-187.mta1.migadu.com [95.215.58.187]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 38FB939AF0 for ; Tue, 13 Feb 2024 09:33:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.187 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707816811; cv=none; b=AUfY8G2jaoI/zZAwVL9nVQ09pnVee5P9vmMDGGcOeosEBKBN+kYCLpcIT5ICPdrmlpU7dZH+qhzPJwVplDBGVX/BkF5JGOR136CRn9R1/hZ04jouH35o/++sHc1JEqg8GPBz0sPD62LvdrW5UqNHdlS+wEc5NL55TaGBFF3XvCs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707816811; c=relaxed/simple; bh=0T2ru6MMuxoS0lW+UfmGEtA3Xi3gFhDd6hkRlTBsx60=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Cdf+tNhNKjKiBvX5k0IxNJPymGQ1swQOEpdpPKQFS0SU6MKfeHr27S7OgF+Ouozyd2cTOwpAxgV8RFi+5weaxxK1ztpr8WHxMhIUTPMk7994vh0e2K6n+/muLDOf0qpaD/wj79lrs6XUHuYV8YdlOvOdVp0QZZxjb8+Kp4TAJyI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=fIbqeW1Z; arc=none smtp.client-ip=95.215.58.187 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1707816808; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+sE7IWb8cyeSXhM2SoQx6feQUCncZFMKiDi8i8kQuog=; b=fIbqeW1ZtUZ68xNmBl0T2oKjxtonR+jlZ0zOZVCkvVUyb9G8iY3hNpGjdaSwnBzT25HlJu aTq0LUmN+pDqwuA4EwzAsaCusVYF+CmVzZ7Cy6egs0n+XFNiTk5KKapqAkptc3AoQoEGOs MFPfZbxRvSMakWrZbH07OLbdPfSuMRY= From: Oliver Upton To: kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , linux-kernel@vger.kernel.org, Oliver Upton Subject: [PATCH v2 09/23] KVM: arm64: vgic: Rely on RCU protection in vgic_get_lpi() Date: Tue, 13 Feb 2024 09:32:46 +0000 Message-ID: <20240213093250.3960069-10-oliver.upton@linux.dev> In-Reply-To: <20240213093250.3960069-1-oliver.upton@linux.dev> References: <20240213093250.3960069-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790775940561116729 X-GMAIL-MSGID: 1790775940561116729 Stop acquiring the lpi_list_lock in favor of RCU for protecting the read-side critical section in vgic_get_lpi(). In order for this to be safe, we also need to be careful not to take a reference on an irq with a refcount of 0, as it is about to be freed. Signed-off-by: Oliver Upton --- arch/arm64/kvm/vgic/vgic.c | 9 ++++----- arch/arm64/kvm/vgic/vgic.h | 14 +++++++++++--- 2 files changed, 15 insertions(+), 8 deletions(-) diff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c index be3ed4c5e1fa..128ae53a0a55 100644 --- a/arch/arm64/kvm/vgic/vgic.c +++ b/arch/arm64/kvm/vgic/vgic.c @@ -62,15 +62,14 @@ static struct vgic_irq *vgic_get_lpi(struct kvm *kvm, u32 intid) { struct vgic_dist *dist = &kvm->arch.vgic; struct vgic_irq *irq = NULL; - unsigned long flags; - raw_spin_lock_irqsave(&dist->lpi_list_lock, flags); + rcu_read_lock(); irq = xa_load(&dist->lpi_xa, intid); - if (irq) - vgic_get_irq_kref(irq); + if (!vgic_try_get_irq_kref(irq)) + irq = NULL; - raw_spin_unlock_irqrestore(&dist->lpi_list_lock, flags); + rcu_read_unlock(); return irq; } diff --git a/arch/arm64/kvm/vgic/vgic.h b/arch/arm64/kvm/vgic/vgic.h index 8d134569d0a1..f874b9932c5a 100644 --- a/arch/arm64/kvm/vgic/vgic.h +++ b/arch/arm64/kvm/vgic/vgic.h @@ -220,12 +220,20 @@ void vgic_v2_vmcr_sync(struct kvm_vcpu *vcpu); void vgic_v2_save_state(struct kvm_vcpu *vcpu); void vgic_v2_restore_state(struct kvm_vcpu *vcpu); -static inline void vgic_get_irq_kref(struct vgic_irq *irq) +static inline bool vgic_try_get_irq_kref(struct vgic_irq *irq) { + if (!irq) + return false; + if (irq->intid < VGIC_MIN_LPI) - return; + return true; - kref_get(&irq->refcount); + return kref_get_unless_zero(&irq->refcount); +} + +static inline void vgic_get_irq_kref(struct vgic_irq *irq) +{ + WARN_ON_ONCE(!vgic_try_get_irq_kref(irq)); } void vgic_v3_fold_lr_state(struct kvm_vcpu *vcpu); From patchwork Tue Feb 13 09:32:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 200292 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp424666dyb; Tue, 13 Feb 2024 01:37:19 -0800 (PST) X-Google-Smtp-Source: AGHT+IG6UIfIOgQrHFmajjNduNM4Ji6262hPo/Q+qU/rYsQQK37VtbgPpZOM5jXrYXIy5GM2z5LR X-Received: by 2002:aa7:d951:0:b0:561:aa3:f9e6 with SMTP id l17-20020aa7d951000000b005610aa3f9e6mr6238030eds.15.1707817039392; Tue, 13 Feb 2024 01:37:19 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707817039; cv=pass; d=google.com; s=arc-20160816; b=cbk4WNkI8Rr4ZJiJrsUUzTBngi7yy+h3fgVdX53e4tVzm1AsDWinE/2GgKcDXvB89E zvoznLM9OidZjEvmXMQ71F35fZzYjW7PigjcA/h92LpiLGxR1Hr7+j84tdBbbVcG3Eqc Pu54KRJSZfUWs3Vg4Kdod28OeKo2Is6O02W9jQu8/caZXU9oA7kv1vKPmi+TFi1cSwrV dx3gr3dpEz9+PAjMjXcNViWwF5EmyUV+AzrAtEgXD9QfToHf0VxR2KJ1O/BYcmSZVHoo nYMP3tQ+APo2NSJvDGIzrJj9LyAIaiecXTxa+cY/Jq/F8BiU1SgWiNnyI1lUPml6ioJ5 k1KQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=Tw7Zntu0i50/45BH/WcBGp2LvuWwU+qxBI8D9y8HogY=; fh=v12+Dl3EATS/WNwhqufzzBMByhnp9nvJItkZ6yyZR2c=; b=yj92+Y92T8vS0TNLRZMwHCskujTvCPYkLc5UmcgKmRy2CZU0OaQHJ6NW22eFW8w1GH i2jMjr2UDBEuYH1aVvE0kZ4sVNKoBTpwCl3CDo3vfQ7f5IiVCI+dy/LsDr/ILz7v6y7C 1o+dEtWakPNB0b0W8/JzAUJTitatrIw8ruQhhM4+1o54eRlHXDroHk32pw7saggp4N8d 54+kMyG9stcLy9pd8I6FQ7YyTLtaiKG+7ce00o36EqGqTjMLOnN/QpUExLFd9NVMTmLs cYPyuK6L+MvgDkJEMaozmQhgieTBcqQB8lHbN02DW/QETD9r7yBFS4G2KU0D3CNQ7P2Y 6SCg==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=jazLFqqx; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63215-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63215-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev X-Forwarded-Encrypted: i=2; AJvYcCURCN3UpGegK7T+LpGN79StNHQQlsj4VI1ND75lnhmoUuzBJUJMPflPfJCWTKfLbLqnDl0l5uFdjVFDMOL57klsBwwNiw== Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id r6-20020a05640251c600b00561d07b6c4csi1226706edd.657.2024.02.13.01.37.19 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 01:37:19 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-63215-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=jazLFqqx; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63215-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63215-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 08D781F2340A for ; Tue, 13 Feb 2024 09:37:19 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 64EC7224E7; Tue, 13 Feb 2024 09:33:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="jazLFqqx" Received: from out-189.mta1.migadu.com (out-189.mta1.migadu.com [95.215.58.189]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1BD1B3FB39 for ; Tue, 13 Feb 2024 09:33:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.189 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707816813; cv=none; b=tvPZmk/k9Geyn6b159DR7bIZDLFtibP/T2xZNZlHTfNf0JSBzkK26UN9lg0U5YDyYWSgveLh6pcC/QecdG5GWNpzzEtRFnQ5GfHZ/5c4pBCbWPbz1NWwUcaYRTkTwUtvyE/HIYmeA4Gf26ofQ3GpIvSPWYC1/YCftxgobqpfFXc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707816813; c=relaxed/simple; bh=vk23VxPinLQAfnq8uTzaCqEThe7jwZl09foYlDKigqw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=YCu0ARBnyL/Wes+Jz1XEEWihW9gk7cTJDoebUqL2EubS5Tq0QLVgr/qFr3QdU6aUDjNYgBObtEqdNvHfwqGWYOaS2BE8XWOI8DejU5vYc+olL+/4uc9DuGxo5EigonCPB2MWoEEX7cgy8PSBlF8mxnokHXt38RBfYpw1dXTijl8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=jazLFqqx; arc=none smtp.client-ip=95.215.58.189 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1707816810; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Tw7Zntu0i50/45BH/WcBGp2LvuWwU+qxBI8D9y8HogY=; b=jazLFqqxgtjzkcxj1dc01Z4YBDKx8CvNw9zqpJpY6FjkPxtrbnhpejwlDxidZ7Kq8tXaXb 91T712fapp4jgvLZo/42Dhp1S3DBz/2cpv3OwOdQj0XnYmVZ0aaLyrJYTAA/3WZmPadSn2 l5iY+4M2h6Rg4VWGdqFUqZVxtbdv30g= From: Oliver Upton To: kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , linux-kernel@vger.kernel.org, Oliver Upton Subject: [PATCH v2 10/23] KVM: arm64: vgic: Ensure the irq refcount is nonzero when taking a ref Date: Tue, 13 Feb 2024 09:32:47 +0000 Message-ID: <20240213093250.3960069-11-oliver.upton@linux.dev> In-Reply-To: <20240213093250.3960069-1-oliver.upton@linux.dev> References: <20240213093250.3960069-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790775960357556837 X-GMAIL-MSGID: 1790775960357556837 It will soon be possible for get() and put() calls to happen in parallel, which means in most cases we must ensure the refcount is nonzero when taking a new reference. Switch to using vgic_try_get_irq_kref() where necessary, and document the few conditions where an IRQ's refcount is guaranteed to be nonzero. Signed-off-by: Oliver Upton --- arch/arm64/kvm/vgic/vgic-its.c | 18 ++++++++---------- arch/arm64/kvm/vgic/vgic.c | 3 ++- 2 files changed, 10 insertions(+), 11 deletions(-) diff --git a/arch/arm64/kvm/vgic/vgic-its.c b/arch/arm64/kvm/vgic/vgic-its.c index 048226812974..6b9634cec77f 100644 --- a/arch/arm64/kvm/vgic/vgic-its.c +++ b/arch/arm64/kvm/vgic/vgic-its.c @@ -75,18 +75,11 @@ static struct vgic_irq *vgic_add_lpi(struct kvm *kvm, u32 intid, * check that we don't add a second list entry with the same LPI. */ oldirq = xa_load(&dist->lpi_xa, intid); - if (oldirq) { + if (vgic_try_get_irq_kref(oldirq)) { /* Someone was faster with adding this LPI, lets use that. */ kfree(irq); irq = oldirq; - /* - * This increases the refcount, the caller is expected to - * call vgic_put_irq() on the returned pointer once it's - * finished with the IRQ. - */ - vgic_get_irq_kref(irq); - goto out_unlock; } @@ -610,8 +603,8 @@ static struct vgic_irq *vgic_its_check_cache(struct kvm *kvm, phys_addr_t db, raw_spin_lock_irqsave(&dist->lpi_list_lock, flags); irq = __vgic_its_check_cache(dist, db, devid, eventid); - if (irq) - vgic_get_irq_kref(irq); + if (!vgic_try_get_irq_kref(irq)) + irq = NULL; raw_spin_unlock_irqrestore(&dist->lpi_list_lock, flags); @@ -660,6 +653,11 @@ static void vgic_its_cache_translation(struct kvm *kvm, struct vgic_its *its, __vgic_put_lpi_locked(kvm, cte->irq); } + /* + * The irq refcount is guaranteed to be nonzero while holding the + * its_lock, as the ITE (and the reference it holds) cannot be freed. + */ + lockdep_assert_held(&its->its_lock); vgic_get_irq_kref(irq); cte->db = db; diff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c index 128ae53a0a55..2a288d6c0be7 100644 --- a/arch/arm64/kvm/vgic/vgic.c +++ b/arch/arm64/kvm/vgic/vgic.c @@ -394,7 +394,8 @@ bool vgic_queue_irq_unlock(struct kvm *kvm, struct vgic_irq *irq, /* * Grab a reference to the irq to reflect the fact that it is - * now in the ap_list. + * now in the ap_list. This is safe as the caller must already hold a + * reference on the irq. */ vgic_get_irq_kref(irq); list_add_tail(&irq->ap_list, &vcpu->arch.vgic_cpu.ap_list_head); From patchwork Tue Feb 13 09:32:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 200293 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp424808dyb; Tue, 13 Feb 2024 01:37:44 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCW5a5SUAahU3mJb6crH7RgM0/pQO9hoTQQy0pC3LvsWZvjgKzqglBRMFCRawgBZDK9dSePRxzVO22FeiKz6Fuxxq5EYkQ== X-Google-Smtp-Source: AGHT+IGOozcAUPtGpy7uEzx6IBWv/B7VTDBXzNWXjaWcm5pH+MwQElh5OHX+kt4XbgsrKbRHoXzx X-Received: by 2002:a50:ee8b:0:b0:560:799:f802 with SMTP id f11-20020a50ee8b000000b005600799f802mr5456305edr.21.1707817064117; Tue, 13 Feb 2024 01:37:44 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707817064; cv=pass; d=google.com; s=arc-20160816; b=YNiY6PJnieEWusESPDFaAMhUxQvaZleFFusmRMVsWFNVnj8ZiQaLVYN3KtVTWCbACU sZCNrwLZWhOPPaGCg8VV5OEmSgjm7eA3kkvgLy5jb8wN4M6z/lRHqFO3+2h4n8XKGPoo K3qc07CCMdUvatMpdpKWbGoc0CclU2N+DsM0SNIZQwGmHJ9oForqGpGvW26l0zRmd1F4 xAx1jyPN6VF81K5YtK29Q6BEklnABdLlxEWjPuVwSpjj0OUG+j63JKYvj8e2odLd4KrA bNhpzT06+sNDjo5xTO0TfZosJMvIXLuWm+AdQynRQHuPPsbKrh2CwgF9oMlghZa8gxEW X/Gg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=/NKiyUPOAHZp+4seCuTeFdtHdSqB+Sryj+FB70LZ6SE=; fh=Cu0vbI7dT0j1msF8y22bJmmP2K9Hyp3x9Aq81evchps=; b=Wl6nopOLLPCN+a7CIRT2CgLKiUwhU6qa9zo4LJiHHVHmZ7lbCGzClrQz2fmjoHE5qr ga72eRKz2BHTwUI78hS2rDY3ktsuuE+VDBlpThL1GOaJmOgJ0EBPiFJpD0M1mX4RD7fx 5+xvGT0dfd0VEJNp6/uesGxmGzyyW98Vdvd0QhABqBw8TTxT6E0b2WOnpFbDjzijj33I +69bUbglthx66wmGaxE5fADA2Pz+eKf2CFkqk3jrT9Pd0oJ81d0OC/NfPZui18PCYX8o OYQEvIpZtMkFhsbUK4lmPEQDg+oGuQAPsUAn0JEpldeXwLgLBKFZMnFXrN/P952l1urW cRQg==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=m39rFo3b; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63216-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63216-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev X-Forwarded-Encrypted: i=2; AJvYcCVuj+Sw5HIWQrncyx+HO1LD92YCN/Gs6EM2RiXEAKvfwARQNB43ULjVM0E+9ANJK6Yxke9JeUs5G3hHDWodWMCaJW5bpQ== Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id i7-20020a05640242c700b00562003baaafsi228363edc.611.2024.02.13.01.37.43 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 01:37:44 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-63216-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=m39rFo3b; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63216-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63216-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id B74321F25582 for ; Tue, 13 Feb 2024 09:37:43 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 206914F88D; Tue, 13 Feb 2024 09:33:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="m39rFo3b" Received: from out-176.mta1.migadu.com (out-176.mta1.migadu.com [95.215.58.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2EA9D487B6 for ; Tue, 13 Feb 2024 09:33:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707816816; cv=none; b=bz854QMqMne67P48FemRx/txDZLmCoLc3rsLD0UM4NEu5Sf3s9S1Y6N3L8hHPLFqRlxRuxR3qq6Chq4mnRXqGq5CXanRENSYQzhP5N8JCugOVexatPMfhaQ0E8Rsg0Zd346xumoSmR/cZG+MonB6p3liI5DgSRhfJAYBbmMvEjM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707816816; c=relaxed/simple; bh=AMfprxM0/ut1EJb/0zCW90BuRmzW+XprxTWpT1Elo/s=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=afnPKU/osgFWl03695mo+7B/YxVh9ig/Io4FtlR2ksFxOdKxHJPhrsvnL+vqygBrCK6Druc6xhAmu7VO2zK+kjJw9fpLDuQDucMyRhAFLiqipSyHsVjvBheij9uCs6xGNiz2EHN2ggYKnpoSF6OpJOO5t1tyWxg4xpaE9l/79f8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=m39rFo3b; arc=none smtp.client-ip=95.215.58.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1707816812; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/NKiyUPOAHZp+4seCuTeFdtHdSqB+Sryj+FB70LZ6SE=; b=m39rFo3bZXhcOOKgVqA1OtqG/2d1pFKH3ESY4PlhbvCQIcBN54j3NLFr67rLVEi8hX3kPX +q0pAZguXIU08yRGInEPzoZMrySX58rpD1cqqK0rjeppMroPaI6tLWSaEP3gEVq7FbS382 U6z1mj8LlFCVXClGb2NJU7VJE0DhZlw= From: Oliver Upton To: kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , linux-kernel@vger.kernel.org, Oliver Upton Subject: [PATCH v2 11/23] KVM: arm64: vgic: Don't acquire the lpi_list_lock in vgic_put_irq() Date: Tue, 13 Feb 2024 09:32:48 +0000 Message-ID: <20240213093250.3960069-12-oliver.upton@linux.dev> In-Reply-To: <20240213093250.3960069-1-oliver.upton@linux.dev> References: <20240213093250.3960069-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790775986066906690 X-GMAIL-MSGID: 1790775986066906690 The LPI xarray's xa_lock is sufficient for synchronizing writers when freeing a given LPI. Furthermore, readers can only take a new reference on an IRQ if it was already nonzero. Stop taking the lpi_list_lock unnecessarily and get rid of __vgic_put_lpi_locked(). Signed-off-by: Oliver Upton --- arch/arm64/kvm/vgic/vgic-its.c | 4 ++-- arch/arm64/kvm/vgic/vgic.c | 21 ++++----------------- arch/arm64/kvm/vgic/vgic.h | 1 - 3 files changed, 6 insertions(+), 20 deletions(-) diff --git a/arch/arm64/kvm/vgic/vgic-its.c b/arch/arm64/kvm/vgic/vgic-its.c index 6b9634cec77f..50a9addebeed 100644 --- a/arch/arm64/kvm/vgic/vgic-its.c +++ b/arch/arm64/kvm/vgic/vgic-its.c @@ -650,7 +650,7 @@ static void vgic_its_cache_translation(struct kvm *kvm, struct vgic_its *its, if (cte->irq) { KVM_VM_TRACE_EVENT(kvm, vgic_its_trans_cache_victim, cte->db, cte->devid, cte->eventid, cte->irq->intid); - __vgic_put_lpi_locked(kvm, cte->irq); + vgic_put_irq(kvm, cte->irq); } /* @@ -688,7 +688,7 @@ void vgic_its_invalidate_cache(struct kvm *kvm) if (!cte->irq) break; - __vgic_put_lpi_locked(kvm, cte->irq); + vgic_put_irq(kvm, cte->irq); cte->irq = NULL; } diff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c index 2a288d6c0be7..c8208f81fdc8 100644 --- a/arch/arm64/kvm/vgic/vgic.c +++ b/arch/arm64/kvm/vgic/vgic.c @@ -110,13 +110,13 @@ static void vgic_irq_release(struct kref *ref) { } -/* - * Drop the refcount on the LPI. Must be called with lpi_list_lock held. - */ -void __vgic_put_lpi_locked(struct kvm *kvm, struct vgic_irq *irq) +void vgic_put_irq(struct kvm *kvm, struct vgic_irq *irq) { struct vgic_dist *dist = &kvm->arch.vgic; + if (irq->intid < VGIC_MIN_LPI) + return; + if (!kref_put(&irq->refcount, vgic_irq_release)) return; @@ -126,19 +126,6 @@ void __vgic_put_lpi_locked(struct kvm *kvm, struct vgic_irq *irq) kfree_rcu(irq, rcu); } -void vgic_put_irq(struct kvm *kvm, struct vgic_irq *irq) -{ - struct vgic_dist *dist = &kvm->arch.vgic; - unsigned long flags; - - if (irq->intid < VGIC_MIN_LPI) - return; - - raw_spin_lock_irqsave(&dist->lpi_list_lock, flags); - __vgic_put_lpi_locked(kvm, irq); - raw_spin_unlock_irqrestore(&dist->lpi_list_lock, flags); -} - void vgic_flush_pending_lpis(struct kvm_vcpu *vcpu) { struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu; diff --git a/arch/arm64/kvm/vgic/vgic.h b/arch/arm64/kvm/vgic/vgic.h index f874b9932c5a..0c2b82de8fa3 100644 --- a/arch/arm64/kvm/vgic/vgic.h +++ b/arch/arm64/kvm/vgic/vgic.h @@ -180,7 +180,6 @@ vgic_get_mmio_region(struct kvm_vcpu *vcpu, struct vgic_io_device *iodev, gpa_t addr, int len); struct vgic_irq *vgic_get_irq(struct kvm *kvm, struct kvm_vcpu *vcpu, u32 intid); -void __vgic_put_lpi_locked(struct kvm *kvm, struct vgic_irq *irq); void vgic_put_irq(struct kvm *kvm, struct vgic_irq *irq); bool vgic_get_phys_line_level(struct vgic_irq *irq); void vgic_irq_set_phys_pending(struct vgic_irq *irq, bool pending); From patchwork Tue Feb 13 09:32:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 200294 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp425009dyb; Tue, 13 Feb 2024 01:38:13 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWkB1sWKPdwIFoAsLFAULi2d/XhzxyV+PSL9Edsg07qzGAG4TIanbunIYD3YLh74VSUTD3Vb6vVP6N+cyTCzIclHLK+6w== X-Google-Smtp-Source: AGHT+IFbHeQEIbW0f3NcAlgaNle1DuRuY9/IDHuF0ToRhqUXsDig05CaWfVUvOjlH3j/GHjvairj X-Received: by 2002:a05:6e02:1d03:b0:363:8877:81af with SMTP id i3-20020a056e021d0300b00363887781afmr12162785ila.16.1707817093078; Tue, 13 Feb 2024 01:38:13 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707817093; cv=pass; d=google.com; s=arc-20160816; b=pjJIDARX3ztIogntKjvmCawYjWJmBbyppNdcjfCW4D6lmDvGhnqC2SjuiZ7/yRrHBr AHwFciZkRnCW+fftKjhcpqopDpuwtlXGv4zixr5wfmIAB/IF+WweuoA5XTd9wl9WogMb DFLOrT98DFWXEi6L/VahMowe1J7lkAtCyJMcw4eQpsgFfZHBLmgmEb1z4TEpPWq7mh+K fjjzGazV+ULqe4DmCRLNl5mwOLsrqagksNOG2U9/YK+X9W5rJhRsU9iwDEdCArjGPrZa ow8W6P943nYj4q3HvTujpb84a74UPBcl6rb0fC+Af2aSYjyGZFEr4/r/ZKrI2yWfm5LH t0YQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=Y+lMHsCzsaKIbkTveh0NrGUV6sBBBNjPrb67/o/BF7Y=; fh=XB9sD1znGn7GjNS3YNO58k6Qa9Vu3SBKnfh+5exzlOo=; b=Hg2Q6yCGQfIQOCUP2E+H7l19n2TGkDufs76XYaKdm3evyAVxWPvQ5xKXc9ynoSwqIl Go8iKlRcW+ynzIDEcKmWO8BXwj9gvBkL55kR9NPlSd/LvcDvHCOVTNSGjyuPo8tzzblc H5yjKt1FQ98mjTUCFRSOuIdaVleR1AzyQoIiWJA3AUIttxs8knATeO1HyI3XqeGqvOQv 0r0u7QLDc121RSt2IgSeI8lPoQK0bq12A5gCt3bwYGv+EWSr6/DtLLXEqy/h+Ov/eL7A SVNVouInoc+8mHUnYuNTv1xKWQn06BfyhmZ9egFK2ON2uQS6frugfervLe2Ibr2Ccy2U JZIQ==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=e7t7TNz2; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63217-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63217-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev X-Forwarded-Encrypted: i=2; AJvYcCV+b9Z+09QyNVGqGyK9ExZS39DpfJWmB6psW2d0S4ERtggLvqx5BAy9qYQ4Z1oGnsvJQSzg6I41539JM9Eh5zBxd+G6gA== Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id e16-20020a656890000000b005cdb499ac69si1726470pgt.53.2024.02.13.01.38.12 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 01:38:13 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-63217-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=e7t7TNz2; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63217-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63217-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id CD4DB280C29 for ; Tue, 13 Feb 2024 09:38:12 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 41C7B51C44; Tue, 13 Feb 2024 09:33:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="e7t7TNz2" Received: from out-189.mta1.migadu.com (out-189.mta1.migadu.com [95.215.58.189]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 54325224DE for ; Tue, 13 Feb 2024 09:33:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.189 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707816818; cv=none; b=PSddTWeVmfv74LU5N6OqO13mRpm18bLytD78/dLCoFAgPownBecjMpmwHGW/7O4SccJ4jm92Yas5LTUsGvkNr2MQKDhTPBNvurHXqhCmbldKmgzf6MhAB2BpVM5He0RXb+99Z4fWXXr22ET3W4hwfxYZFwVZJq8+Cq18evfMMrw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707816818; c=relaxed/simple; bh=u654twAWO/lOhw5G+s6F5ghriBPc9GHm8+B9VReBiXc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=oQYULhLAIb8297QsAJF20LpprEK8jUm9ULpSj+62BArw7Ze9kFfZbD1iA0Kpj4I+KMJxGb0ORm4Z1LtvYzit8yqliIjo2V17LV7krx8Em6KSitBTGoZB1MmMDlz+LMg0iFNPnubVXBlOvENZf4BiBGdCAnq9x3Jfaewo/0A9Ou4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=e7t7TNz2; arc=none smtp.client-ip=95.215.58.189 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1707816814; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Y+lMHsCzsaKIbkTveh0NrGUV6sBBBNjPrb67/o/BF7Y=; b=e7t7TNz2VtcfBlVd5Mjzyd6gJKVWSV0ZzpRHdbUml7x9/IMMNdWc32TLtbmJenkRNzWM9J gUzOaUy8lrCvUK+H25jbrbFVoeSONJHlCVz7Jfs5LveG9YomxxWmOfqm3lI6uD4l6WwE3s /z4mrZYrT42YMUsJUS1OXlW36Q421Ic= From: Oliver Upton To: kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , linux-kernel@vger.kernel.org, Oliver Upton Subject: [PATCH v2 12/23] KVM: arm64: vgic-its: Lazily allocate LPI translation cache Date: Tue, 13 Feb 2024 09:32:49 +0000 Message-ID: <20240213093250.3960069-13-oliver.upton@linux.dev> In-Reply-To: <20240213093250.3960069-1-oliver.upton@linux.dev> References: <20240213093250.3960069-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790776015741081311 X-GMAIL-MSGID: 1790776015741081311 Reusing translation cache entries within a read-side critical section is fundamentally incompatible with an rculist. As such, we need to allocate a new entry to replace an eviction and free the removed entry afterwards. Take this as an opportunity to remove the eager allocation of translation cache entries altogether in favor of a lazy allocation model on cache miss. Signed-off-by: Oliver Upton --- arch/arm64/kvm/vgic/vgic-init.c | 3 -- arch/arm64/kvm/vgic/vgic-its.c | 96 +++++++++++++++------------------ include/kvm/arm_vgic.h | 1 + 3 files changed, 45 insertions(+), 55 deletions(-) diff --git a/arch/arm64/kvm/vgic/vgic-init.c b/arch/arm64/kvm/vgic/vgic-init.c index e25672d6e846..660d5ce3b610 100644 --- a/arch/arm64/kvm/vgic/vgic-init.c +++ b/arch/arm64/kvm/vgic/vgic-init.c @@ -305,9 +305,6 @@ int vgic_init(struct kvm *kvm) } } - if (vgic_has_its(kvm)) - vgic_lpi_translation_cache_init(kvm); - /* * If we have GICv4.1 enabled, unconditionnaly request enable the * v4 support so that we get HW-accelerated vSGIs. Otherwise, only diff --git a/arch/arm64/kvm/vgic/vgic-its.c b/arch/arm64/kvm/vgic/vgic-its.c index 50a9addebeed..a7ba20b57264 100644 --- a/arch/arm64/kvm/vgic/vgic-its.c +++ b/arch/arm64/kvm/vgic/vgic-its.c @@ -611,12 +611,20 @@ static struct vgic_irq *vgic_its_check_cache(struct kvm *kvm, phys_addr_t db, return irq; } +/* Default is 16 cached LPIs per vcpu */ +#define LPI_DEFAULT_PCPU_CACHE_SIZE 16 + +static unsigned int vgic_its_max_cache_size(struct kvm *kvm) +{ + return atomic_read(&kvm->online_vcpus) * LPI_DEFAULT_PCPU_CACHE_SIZE; +} + static void vgic_its_cache_translation(struct kvm *kvm, struct vgic_its *its, u32 devid, u32 eventid, struct vgic_irq *irq) { + struct vgic_translation_cache_entry *new, *victim = NULL; struct vgic_dist *dist = &kvm->arch.vgic; - struct vgic_translation_cache_entry *cte; unsigned long flags; phys_addr_t db; @@ -624,10 +632,11 @@ static void vgic_its_cache_translation(struct kvm *kvm, struct vgic_its *its, if (irq->hw) return; - raw_spin_lock_irqsave(&dist->lpi_list_lock, flags); + new = kzalloc(sizeof(*new), GFP_KERNEL_ACCOUNT); + if (!new) + return; - if (unlikely(list_empty(&dist->lpi_translation_cache))) - goto out; + raw_spin_lock_irqsave(&dist->lpi_list_lock, flags); /* * We could have raced with another CPU caching the same @@ -635,22 +644,17 @@ static void vgic_its_cache_translation(struct kvm *kvm, struct vgic_its *its, * already */ db = its->vgic_its_base + GITS_TRANSLATER; - if (__vgic_its_check_cache(dist, db, devid, eventid)) + if (__vgic_its_check_cache(dist, db, devid, eventid)) { + kfree(new); goto out; + } - /* Always reuse the last entry (LRU policy) */ - cte = list_last_entry(&dist->lpi_translation_cache, - typeof(*cte), entry); - - /* - * Caching the translation implies having an extra reference - * to the interrupt, so drop the potential reference on what - * was in the cache, and increment it on the new interrupt. - */ - if (cte->irq) { - KVM_VM_TRACE_EVENT(kvm, vgic_its_trans_cache_victim, cte->db, - cte->devid, cte->eventid, cte->irq->intid); - vgic_put_irq(kvm, cte->irq); + if (dist->lpi_cache_count >= vgic_its_max_cache_size(kvm)) { + /* Always reuse the last entry (LRU policy) */ + victim = list_last_entry(&dist->lpi_translation_cache, + typeof(*cte), entry); + list_del(&victim->entry); + dist->lpi_cache_count--; } /* @@ -660,16 +664,33 @@ static void vgic_its_cache_translation(struct kvm *kvm, struct vgic_its *its, lockdep_assert_held(&its->its_lock); vgic_get_irq_kref(irq); - cte->db = db; - cte->devid = devid; - cte->eventid = eventid; - cte->irq = irq; + new->db = db; + new->devid = devid; + new->eventid = eventid; + new->irq = irq; /* Move the new translation to the head of the list */ - list_move(&cte->entry, &dist->lpi_translation_cache); + list_add(&new->entry, &dist->lpi_translation_cache); + dist->lpi_cache_count++; out: raw_spin_unlock_irqrestore(&dist->lpi_list_lock, flags); + + if (!victim) + return; + + /* + * Caching the translation implies having an extra reference + * to the interrupt, so drop the potential reference on what + * was in the cache, and increment it on the new interrupt. + */ + if (victim->irq) { + KVM_VM_TRACE_EVENT(kvm, vgic_its_trans_cache_victim, victim->db, + victim->devid, victim->eventid, victim->irq->intid); + vgic_put_irq(kvm, victim->irq); + } + + kfree(victim); } void vgic_its_invalidate_cache(struct kvm *kvm) @@ -1917,33 +1938,6 @@ static int vgic_register_its_iodev(struct kvm *kvm, struct vgic_its *its, return ret; } -/* Default is 16 cached LPIs per vcpu */ -#define LPI_DEFAULT_PCPU_CACHE_SIZE 16 - -void vgic_lpi_translation_cache_init(struct kvm *kvm) -{ - struct vgic_dist *dist = &kvm->arch.vgic; - unsigned int sz; - int i; - - if (!list_empty(&dist->lpi_translation_cache)) - return; - - sz = atomic_read(&kvm->online_vcpus) * LPI_DEFAULT_PCPU_CACHE_SIZE; - - for (i = 0; i < sz; i++) { - struct vgic_translation_cache_entry *cte; - - /* An allocation failure is not fatal */ - cte = kzalloc(sizeof(*cte), GFP_KERNEL_ACCOUNT); - if (WARN_ON(!cte)) - break; - - INIT_LIST_HEAD(&cte->entry); - list_add(&cte->entry, &dist->lpi_translation_cache); - } -} - void vgic_lpi_translation_cache_destroy(struct kvm *kvm) { struct vgic_dist *dist = &kvm->arch.vgic; @@ -1990,8 +1984,6 @@ static int vgic_its_create(struct kvm_device *dev, u32 type) kfree(its); return ret; } - - vgic_lpi_translation_cache_init(dev->kvm); } mutex_init(&its->its_lock); diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h index 47035946648e..431d05c01a53 100644 --- a/include/kvm/arm_vgic.h +++ b/include/kvm/arm_vgic.h @@ -281,6 +281,7 @@ struct vgic_dist { /* LPI translation cache */ struct list_head lpi_translation_cache; + unsigned int lpi_cache_count; /* used by vgic-debug */ struct vgic_state_iter *iter; From patchwork Tue Feb 13 09:32:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 200295 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp425070dyb; Tue, 13 Feb 2024 01:38:21 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCUCrLOT5CfQ7VODmjq8dgB88b/xEEm4s1Vh0C10gq3zB0R7g8oyYOt6zPT+hMjuua5JZgjbF5v4bR8L7dPpLkR4OxGggw== X-Google-Smtp-Source: AGHT+IEya+hG0Jb0Nxuh4W95uUuagp9P3BT+R1rp5wDe3dlIiQudgdKimFKSNxjy5HsEPZtQUPtm X-Received: by 2002:a17:906:e287:b0:a3c:8cf8:9383 with SMTP id gg7-20020a170906e28700b00a3c8cf89383mr4292165ejb.37.1707817101035; Tue, 13 Feb 2024 01:38:21 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707817101; cv=pass; d=google.com; s=arc-20160816; b=mgCByr0Igi6HinwQ4E9b4a9d8tIAv+K5ZpYsnNN4iOV5uot05K8YHM++KikRw9dt5L NS61Ut+go33FzRSawz4rsBre4W4Axs1PDXWpk+afVhqoI9F9o3UxzM3THMHaUL2q86km bxBlA5O//zJuB0NTtmPFJ1SOyWGV7p1ha4dNBxkkk4vUDTDmyaMcbPXT67JEJrTPjAZP 6cj5XbN1U19sr2wHw41xWAZuJqUaP7D4FDryNw9QN41BtiefGn6YHsT7QP+J51pD6oNY XQf3b68pEZO/F8opogGRxZ2IJ0FHqWqjDy7qHQzgB+ykqRWG0OW8ohJc25cHUEekKhLW iUxA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=BKECb1eTQQJ/f0GKxMxXWqmjCFHiATDCrLFETxE2qgY=; fh=PzNE90v4iM9Zvw5WFDr6s/X7fBoWF7du8zJvDHj7fqM=; b=hUXqE/VKOpYeaBswfZjU8LhdFcGx4742gjvlU08FIdfL8hVF6/dE2TulfEkepKIeWz +FPjaZQY3UmRAS1PgTjPq2dLw/AHCKGxlId0J+P6JtORw3LWvoD/saNzFAGpu7beG8Dm 2Xt7v3RvyzxlOi9PxXCD2pd5c7/wUo+Xq11r74Fjvy/XIHoFo2vlxsQTXfW5IqABRFNv BuA4TGqSu49dJwlBokqBdOIe7d36MXHgDq+pwMyUlUp/VQo8UdPpSTw6XwdzbFd39DnC aCTTRXpkES3e1rOxfV5lrV59bz85uTVZNeBxEJxaKtW5QXrfdpScQAbHARKgR4QfNEew VCYA==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=Kv8VMoYw; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63218-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63218-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev X-Forwarded-Encrypted: i=2; AJvYcCVYkIz9Cf9+GC3noeUmUvyrErbG0NitTahvmFRM1KhbroRIuQoJl+vCF7Ftl7AogQ7z1rnqEr2XYYUiswZX7BvZMJQtSg== Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id j15-20020a170906050f00b00a3d1bc0c9b6si40575eja.291.2024.02.13.01.38.20 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 01:38:21 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-63218-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=Kv8VMoYw; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63218-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63218-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 78B451F23649 for ; Tue, 13 Feb 2024 09:38:20 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 2297F51C5C; Tue, 13 Feb 2024 09:33:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="Kv8VMoYw" Received: from out-175.mta1.migadu.com (out-175.mta1.migadu.com [95.215.58.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E8955364DB for ; Tue, 13 Feb 2024 09:33:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707816819; cv=none; b=fATo7uoaTe19HtlBR5iiHFN2rXhwj3RfresGf8MwZbhwZVCJca+SKJCPwpbsvtM+Qx8hFptpul7Fh1H32A6M/Uf0HDD8piN4OggAjrvtnv9dB2dfusAqZlSGfZDHcNaDO9+UqHdsTLIcTUBmhEu5mtXL+J1EUv9xJeNJaLfYBMY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707816819; c=relaxed/simple; bh=Qi8rU6NjOkVcx9TX1/O9p73F+ULC5MIdl/I9CDfvIew=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=hsmiMYczPi3RM7gIZISGgDedbmy/i0i4dQEX8qHoeyRXAjNmDUuQZLyglgG1PTmoJmXYi3ngMEdvDN2gIuuBV7NY46/jiKrTNSGV00TpBwh6aIPM04mTrKZ6DRj2q1fxmHlO9+Alu8cXuSZx9RBwfbSFTWuGn/6x0N/1tteANxs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=Kv8VMoYw; arc=none smtp.client-ip=95.215.58.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1707816816; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BKECb1eTQQJ/f0GKxMxXWqmjCFHiATDCrLFETxE2qgY=; b=Kv8VMoYwDOw5o/4ifDv6LsdWhzI+zrP+hXZ/dyggg6voxrWhMxjS3n2slHUN6XjTEoqdYA dQuL0Hu9I6PuXqEFLSppH2l3M8sTV/qJmjOHGnCDT3mXHmHde1DdJLi7H/mz2ONCtgqeki OIZDsSupblvzEtuDnC7WHJDgthL8+mY= From: Oliver Upton To: kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , linux-kernel@vger.kernel.org, Oliver Upton Subject: [PATCH v2 13/23] KVM: arm64: vgic-its: Pick cache victim based on usage count Date: Tue, 13 Feb 2024 09:32:50 +0000 Message-ID: <20240213093250.3960069-14-oliver.upton@linux.dev> In-Reply-To: <20240213093250.3960069-1-oliver.upton@linux.dev> References: <20240213093250.3960069-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790776024824972848 X-GMAIL-MSGID: 1790776024824972848 To date the translation cache LRU policy relies on the ordering of the linked-list to pick the victim, as entries are moved to the head of the list on every cache hit. These sort of transformations are incompatible with an rculist, necessitating a different strategy for recording usage in-place. Count the number of cache hits since the last translation cache miss for every entry. The preferences for selecting a victim are as follows: - Invalid entries over valid entries - Valid entry with the lowest usage count - In the case of a tie, pick the entry closest to the tail (oldest) Signed-off-by: Oliver Upton --- arch/arm64/kvm/vgic/vgic-its.c | 46 ++++++++++++++++++++++++++-------- 1 file changed, 36 insertions(+), 10 deletions(-) diff --git a/arch/arm64/kvm/vgic/vgic-its.c b/arch/arm64/kvm/vgic/vgic-its.c index a7ba20b57264..35926d5ae561 100644 --- a/arch/arm64/kvm/vgic/vgic-its.c +++ b/arch/arm64/kvm/vgic/vgic-its.c @@ -157,6 +157,7 @@ struct vgic_translation_cache_entry { u32 devid; u32 eventid; struct vgic_irq *irq; + atomic64_t usage_count; }; /** @@ -580,13 +581,7 @@ static struct vgic_irq *__vgic_its_check_cache(struct vgic_dist *dist, cte->eventid != eventid) continue; - /* - * Move this entry to the head, as it is the most - * recently used. - */ - if (!list_is_first(&cte->entry, &dist->lpi_translation_cache)) - list_move(&cte->entry, &dist->lpi_translation_cache); - + atomic64_inc(&cte->usage_count); return cte->irq; } @@ -619,6 +614,36 @@ static unsigned int vgic_its_max_cache_size(struct kvm *kvm) return atomic_read(&kvm->online_vcpus) * LPI_DEFAULT_PCPU_CACHE_SIZE; } +static struct vgic_translation_cache_entry *vgic_its_cache_victim(struct vgic_dist *dist, + s64 *max_usage) +{ + struct vgic_translation_cache_entry *cte, *victim = NULL; + s64 min, tmp, max = S64_MIN; + + /* + * Find the least used cache entry since the last cache miss, preferring + * older entries in the case of a tie. Return the max usage count seen + * during the scan to initialize the new cache entry. + */ + list_for_each_entry(cte, &dist->lpi_translation_cache, entry) { + tmp = atomic64_read(&cte->usage_count); + max = max(max, tmp); + + if (!cte->irq) { + victim = cte; + break; + } + + if (!victim || tmp <= min) { + victim = cte; + min = tmp; + } + } + + *max_usage = max; + return victim; +} + static void vgic_its_cache_translation(struct kvm *kvm, struct vgic_its *its, u32 devid, u32 eventid, struct vgic_irq *irq) @@ -627,6 +652,7 @@ static void vgic_its_cache_translation(struct kvm *kvm, struct vgic_its *its, struct vgic_dist *dist = &kvm->arch.vgic; unsigned long flags; phys_addr_t db; + s64 usage = 0; /* Do not cache a directly injected interrupt */ if (irq->hw) @@ -650,9 +676,8 @@ static void vgic_its_cache_translation(struct kvm *kvm, struct vgic_its *its, } if (dist->lpi_cache_count >= vgic_its_max_cache_size(kvm)) { - /* Always reuse the last entry (LRU policy) */ - victim = list_last_entry(&dist->lpi_translation_cache, - typeof(*cte), entry); + victim = vgic_its_cache_victim(dist, &usage); + list_del(&victim->entry); dist->lpi_cache_count--; } @@ -664,6 +689,7 @@ static void vgic_its_cache_translation(struct kvm *kvm, struct vgic_its *its, lockdep_assert_held(&its->its_lock); vgic_get_irq_kref(irq); + atomic64_set(&new->usage_count, usage); new->db = db; new->devid = devid; new->eventid = eventid; From patchwork Tue Feb 13 09:37:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 200316 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp434073dyb; Tue, 13 Feb 2024 02:01:41 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWatQhW0QKUhpXkIwFeha/mKkUgaYaEthKO3fAf4v6CQWtrwwhKnnoFhNn/BTe8etGxedAJ7qvNDN6gb1MCy60qR0xlAg== X-Google-Smtp-Source: AGHT+IGnoWSyDsCx1aFDTNty/sTNm2ZO8SYpApgyWXbTiH5iwYfdlaJRpJj1QeqiXI46FxEGZvmM X-Received: by 2002:a17:903:1250:b0:1d9:8bb5:7497 with SMTP id u16-20020a170903125000b001d98bb57497mr13398589plh.40.1707818501343; Tue, 13 Feb 2024 02:01:41 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707818501; cv=pass; d=google.com; s=arc-20160816; b=JRHDa/EQLWNoSOZkvz3WTT5n3OeJC9Yjt7fIciJODKTR1mR3QTKWCd8GqIcx/H+KjN KkXBaONDMsJU8dtQ90yKjTZtEehDLLa3+mTqfxw7nDngxcFbVGcwG/W+3Dp1CEHYzibd WSqO0V0gnptr8kdqDTbJUXcE0St0nvPuaeo3WlRY+ciEekygQQnuPI5G9UrVqmkKYHQ0 0B9cX9OnMXBVQ8VqRGaWf0Vlxxr5bipTGe0NamwQ4UMoSmorYHduiIyUDA2uISuSUP6r MuyeCnz2+v9KO8cyi1jTAQWcVXwzvn2sLhCMtK0q0lSrcLQK1KHEmW21HVLLhhwdIjlH mTXw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=tzXQU++wkXrnCgv+wzrV4DGhnpKBtk4FI4+/K2Mli/0=; fh=46tEZY4hU8dnQeKDViYTjZxrnhfJu68wrnPWow+HF5Q=; b=0/pqNpuQ9jLPp3XhIn9yEUGot3LN3UzZYxlO62lE+Qtdqjfvp41wUARmH/LTfqWyoN 7+k/dXVcDqVnelavkNA9mf8vZjLVQl/evlzi2zJriy6QJLq9MvF1J3WsXxzNDYVnFo74 uexChwcIL5eUhdZknm/1tGOPBBh2NXdqAJkJCCzgP9DkQvkenMxMd52t8XXrrmJPWWoQ MRLaU+sJBwpAlcFWqDqry4Jfn1v6PzgeGBK7tbRPl4T8I6nHKlyOIR6aDv0v0eCoUQRD h0WNmY7UobBsr9b2t4hMt6mogIV8b0A0VZNoLvYFk5daiHvPCsstc3ooFNZMAzGBKmBL mqdQ==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=dZHpG1G4; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63237-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63237-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev X-Forwarded-Encrypted: i=2; AJvYcCWjf4CxLKr8hAcI/iP0XEA1HZ2KEOZcV2kI3ApurQZiS03UAS40H9kHobvVsxv1n/XzNgD8VwAllMM/NTuc5akSQvflXg== Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id z16-20020a170903019000b001d8f10a6a0asi1830228plg.353.2024.02.13.02.01.40 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 02:01:41 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-63237-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=dZHpG1G4; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63237-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63237-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 12AF2B23B4E for ; Tue, 13 Feb 2024 09:44:15 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 7F3E8524A8; Tue, 13 Feb 2024 09:38:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="dZHpG1G4" Received: from out-173.mta1.migadu.com (out-173.mta1.migadu.com [95.215.58.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5067251025 for ; Tue, 13 Feb 2024 09:38:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707817116; cv=none; b=QG80NMsCFGw0XzODEef7qLo9WnQJh6bx29/d9xu0Yq10C1WHTCkcFwA5ugyCMc/uHEulOs9Wioa7CgBrupavwJg+VGaND6QDOPriWnzytWiulOefmBRkuEuZ5GDgQbMX5ckkllmI0b/TGamnD6dG1gNE507pMS/3AAP8eRaxodw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707817116; c=relaxed/simple; bh=iYPy+Djc7SztwYSE8UN83cn8sXnT9Xu1HwT7l4kGpr8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Tq0RTUvzPPIMH8N1kP02GICcGU0U0LDx7eQWwk2+3nR8qr+T6CPLsXpIUBRpkOIDHJYYCi7vERnykLa7nd3KkwYzT92TlObJ5ZBYML2fcIqcRFGfAZYSMWUFAf8M8YlPHHRX1tSm4FKveMlgXZCxITclHKeFWQqjnXnrPM8iFjs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=dZHpG1G4; arc=none smtp.client-ip=95.215.58.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1707817113; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tzXQU++wkXrnCgv+wzrV4DGhnpKBtk4FI4+/K2Mli/0=; b=dZHpG1G41vsXh+Cd0nri7mL23uzEzEJ70EEubkmZ69krO3yyr+812ExcHaaBN0tAiYO/1j mWbk+z49WB6RF7XK+zWtdsHBak3IzYRe5ZZ6A+c6lVvY4iSgqVimAHVnBcZnoQY/JcJ+jy rWrWEvDJuv4QY3UsXTWBGjkZRgjx9ik= From: Oliver Upton To: kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , linux-kernel@vger.kernel.org, Oliver Upton Subject: [PATCH v2 14/23] KVM: arm64: vgic-its: Protect cached vgic_irq pointers with RCU Date: Tue, 13 Feb 2024 09:37:38 +0000 Message-ID: <20240213093747.3960866-1-oliver.upton@linux.dev> In-Reply-To: <20240213093250.3960069-1-oliver.upton@linux.dev> References: <20240213093250.3960069-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790777493129284799 X-GMAIL-MSGID: 1790777493129284799 RCU readers of the LPI translation cache will be able to run in parallel with a cache invalidation, which clears the RCU pointer. Start using RCU protection on the cached irq pointer in light of this. Signed-off-by: Oliver Upton --- arch/arm64/kvm/vgic/vgic-its.c | 28 ++++++++++++++++++---------- 1 file changed, 18 insertions(+), 10 deletions(-) diff --git a/arch/arm64/kvm/vgic/vgic-its.c b/arch/arm64/kvm/vgic/vgic-its.c index 35926d5ae561..99042ecc9c85 100644 --- a/arch/arm64/kvm/vgic/vgic-its.c +++ b/arch/arm64/kvm/vgic/vgic-its.c @@ -156,7 +156,7 @@ struct vgic_translation_cache_entry { phys_addr_t db; u32 devid; u32 eventid; - struct vgic_irq *irq; + struct vgic_irq __rcu *irq; atomic64_t usage_count; }; @@ -574,7 +574,7 @@ static struct vgic_irq *__vgic_its_check_cache(struct vgic_dist *dist, * If we hit a NULL entry, there is nothing after this * point. */ - if (!cte->irq) + if (!rcu_access_pointer(cte->irq)) break; if (cte->db != db || cte->devid != devid || @@ -582,7 +582,7 @@ static struct vgic_irq *__vgic_its_check_cache(struct vgic_dist *dist, continue; atomic64_inc(&cte->usage_count); - return cte->irq; + return rcu_dereference(cte->irq); } return NULL; @@ -629,7 +629,7 @@ static struct vgic_translation_cache_entry *vgic_its_cache_victim(struct vgic_di tmp = atomic64_read(&cte->usage_count); max = max(max, tmp); - if (!cte->irq) { + if (!rcu_access_pointer(cte->irq)) { victim = cte; break; } @@ -663,6 +663,7 @@ static void vgic_its_cache_translation(struct kvm *kvm, struct vgic_its *its, return; raw_spin_lock_irqsave(&dist->lpi_list_lock, flags); + rcu_read_lock(); /* * We could have raced with another CPU caching the same @@ -693,27 +694,32 @@ static void vgic_its_cache_translation(struct kvm *kvm, struct vgic_its *its, new->db = db; new->devid = devid; new->eventid = eventid; - new->irq = irq; + rcu_assign_pointer(new->irq, irq); /* Move the new translation to the head of the list */ list_add(&new->entry, &dist->lpi_translation_cache); dist->lpi_cache_count++; out: + rcu_read_unlock(); raw_spin_unlock_irqrestore(&dist->lpi_list_lock, flags); if (!victim) return; + synchronize_rcu(); + /* * Caching the translation implies having an extra reference * to the interrupt, so drop the potential reference on what - * was in the cache, and increment it on the new interrupt. + * was in the cache. */ if (victim->irq) { + struct vgic_irq *irq = rcu_dereference_raw(victim->irq); + KVM_VM_TRACE_EVENT(kvm, vgic_its_trans_cache_victim, victim->db, - victim->devid, victim->eventid, victim->irq->intid); - vgic_put_irq(kvm, victim->irq); + victim->devid, victim->eventid, irq->intid); + vgic_put_irq(kvm, irq); } kfree(victim); @@ -726,19 +732,21 @@ void vgic_its_invalidate_cache(struct kvm *kvm) unsigned long flags; raw_spin_lock_irqsave(&dist->lpi_list_lock, flags); + rcu_read_lock(); list_for_each_entry(cte, &dist->lpi_translation_cache, entry) { /* * If we hit a NULL entry, there is nothing after this * point. */ - if (!cte->irq) + if (!rcu_access_pointer(cte->irq)) break; vgic_put_irq(kvm, cte->irq); - cte->irq = NULL; + rcu_assign_pointer(cte->irq, NULL); } + rcu_read_unlock(); raw_spin_unlock_irqrestore(&dist->lpi_list_lock, flags); } From patchwork Tue Feb 13 09:39:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 200299 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp427499dyb; Tue, 13 Feb 2024 01:44:56 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCXkGMLWBzuwuh0qTNAza7j9z+FTXZSGgInSPUVw97xEidZCdT7v8r2QZcwEy/gDLs1N/6fDO0ZfWzCJV0MsCN269US0wQ== X-Google-Smtp-Source: AGHT+IHZ5FtQH/QRGVImLkC3Hm1NN6tU46FQVsaN7suQOlr35MPS5rQ7vg1JWY5EXPBcTjBj6SeZ X-Received: by 2002:a17:906:e54:b0:a3d:1cbd:67f7 with SMTP id q20-20020a1709060e5400b00a3d1cbd67f7mr48637eji.0.1707817496327; Tue, 13 Feb 2024 01:44:56 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707817496; cv=pass; d=google.com; s=arc-20160816; b=UB+vmIU1/5kpf8xclLvYDsk4FA9CXwD1PDVju+7qxBShbSoP0keg/M419eintl53S+ KXSLkUmM+Ed4zLdNnBQFOf7B2euqUhj2tkr0HnUOgP+NgluM/S8Kx4uFdyk9u04/AUfY 0YXa25IBh3GifmeYBq5AcCsQqSMf6P65ko/3kX6dYNd13lhpgeTu2tnS7DywkK4NRteN eMRr5F55bwqLwWeOynGQd6tWzgUSp99rffjefQHz/9LG1DOUh9nd68v1D8qs3UO9P+G+ 0+61cym6iOC5tnxJRC6c8X5reek1G8RYw2hFrYjRNSEOaDbAmqjIQsT5zWvhiJu6VgEU qS1Q== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=iVvQT5/XmikRWyk9sSNpoveNMbrL0Bk4ezrgyqLcu50=; fh=4yf6dbrwQUco0mKdczPUbfU+wYoXAD6Bmw8DsrhxRgU=; b=kacVI33wyPjRmTVKfd81BtTD88n/iUshkbdDf3bLQ9xl8ZynsUd5Hq9fmWaAp5Z3x2 Lj1R7UOlIEAV4iHsnyy1qYyT0ph70InCPX6bVRkZZyc5RqAgRh2d67ya6+ZRbuUmOpX0 BKO8b6YaH7wCL3CMoZecEYmiGhohaXdhLG0svKAVaQQLQGq3dIzhSAb97QhRbUKoBGRX +yXQSE4+ZG84NRW0wvUBRUXZiS5anH2HArru1wwJ734g2X0n+s/v1Scc0L6DDjiDk/+M jQx6K9y2uj+/Yf+Fk8VEnb6TQLoYr386wRn+v1ER/RDkXb/RObhse1gyMDI6Tw7XphEW 3pgQ==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=tspZWTRQ; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63239-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63239-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev X-Forwarded-Encrypted: i=2; AJvYcCUPtNKCxQU3y+4hNxPPkiDrllY6aqvqabgWRXx79CbhP1nPvoQbA8OlZKBIrm8HKK93mIeRbzzsmZY+QHPgFYNEbLLkxA== Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id li20-20020a170906f99400b00a3d00af0cf5si455912ejb.1047.2024.02.13.01.44.56 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 01:44:56 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-63239-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=tspZWTRQ; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63239-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63239-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id EA69A1F27FC9 for ; Tue, 13 Feb 2024 09:44:55 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id A1EE9249EA; Tue, 13 Feb 2024 09:40:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="tspZWTRQ" Received: from out-177.mta0.migadu.com (out-177.mta0.migadu.com [91.218.175.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4DDA1224E7 for ; Tue, 13 Feb 2024 09:40:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707817208; cv=none; b=toJQPLY64OXzYRql+Qbgm/z0HARN+CJHM7SoF4RAgwFizB7zPjbzdvfAQOq0FlqQkWOPwLGNvizDoZ/pXEcuoPHSX3Wa4RxKT27MU+ThYzTcq3kckgsSrllPNatHpQ5vNcxRijl+dG1P3JhBOTIIbj3zVZkCIcd+S9v5GXn8h0Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707817208; c=relaxed/simple; bh=1X3y/+7kwIA1bPfJOn0Y/WTdRQy52Y2ta/su6GXyAHE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=L4rYMITm10FpKs/GPx5JVnDVK0T7r1t0e/2ocCBGrQSjWSqnc1goP7xgNqzx8hDkD29JHH3JGPsflWHV10lmLQT01FjU7SbZKR8EYPz8d4JFhwvECRt1VFwRrLR+q+c7T/xE/CMESdvbO1yR0jCV7KdzxVHMOs+sgDI3O81rzCE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=tspZWTRQ; arc=none smtp.client-ip=91.218.175.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1707817203; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iVvQT5/XmikRWyk9sSNpoveNMbrL0Bk4ezrgyqLcu50=; b=tspZWTRQsY42Ipgi4nXkPCl77D6pxRdd0dv7qUb22qs3s6nvFQYwES8NqQumcyZj8EpM7d iKxxt1GENHJpHFumHr5BW/iasdQ/dj3jNfR6WStKJNTjpZOCt6a9eSUX5s+CGWxAk4tptc 2LrvdkzycAeYPES6xrPyBLmIlqc7/4c= From: Oliver Upton To: kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , linux-kernel@vger.kernel.org, Oliver Upton Subject: [PATCH v2 15/23] KVM: arm64: vgic-its: Treat the LPI translation cache as an rculist Date: Tue, 13 Feb 2024 09:39:54 +0000 Message-ID: <20240213093954.3961389-1-oliver.upton@linux.dev> In-Reply-To: <20240213093250.3960069-1-oliver.upton@linux.dev> References: <20240213093250.3960069-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790776439187995327 X-GMAIL-MSGID: 1790776439187995327 Convert the LPI translation cache to an rculist such that readers can walk it while only holding the RCU read lock. Signed-off-by: Oliver Upton --- arch/arm64/kvm/vgic/vgic-its.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/arch/arm64/kvm/vgic/vgic-its.c b/arch/arm64/kvm/vgic/vgic-its.c index 99042ecc9c85..55463eb84763 100644 --- a/arch/arm64/kvm/vgic/vgic-its.c +++ b/arch/arm64/kvm/vgic/vgic-its.c @@ -569,7 +569,7 @@ static struct vgic_irq *__vgic_its_check_cache(struct vgic_dist *dist, { struct vgic_translation_cache_entry *cte; - list_for_each_entry(cte, &dist->lpi_translation_cache, entry) { + list_for_each_entry_rcu(cte, &dist->lpi_translation_cache, entry) { /* * If we hit a NULL entry, there is nothing after this * point. @@ -625,7 +625,7 @@ static struct vgic_translation_cache_entry *vgic_its_cache_victim(struct vgic_di * older entries in the case of a tie. Return the max usage count seen * during the scan to initialize the new cache entry. */ - list_for_each_entry(cte, &dist->lpi_translation_cache, entry) { + list_for_each_entry_rcu(cte, &dist->lpi_translation_cache, entry) { tmp = atomic64_read(&cte->usage_count); max = max(max, tmp); @@ -679,7 +679,7 @@ static void vgic_its_cache_translation(struct kvm *kvm, struct vgic_its *its, if (dist->lpi_cache_count >= vgic_its_max_cache_size(kvm)) { victim = vgic_its_cache_victim(dist, &usage); - list_del(&victim->entry); + list_del_rcu(&victim->entry); dist->lpi_cache_count--; } @@ -697,7 +697,7 @@ static void vgic_its_cache_translation(struct kvm *kvm, struct vgic_its *its, rcu_assign_pointer(new->irq, irq); /* Move the new translation to the head of the list */ - list_add(&new->entry, &dist->lpi_translation_cache); + list_add_rcu(&new->entry, &dist->lpi_translation_cache); dist->lpi_cache_count++; out: @@ -734,7 +734,7 @@ void vgic_its_invalidate_cache(struct kvm *kvm) raw_spin_lock_irqsave(&dist->lpi_list_lock, flags); rcu_read_lock(); - list_for_each_entry(cte, &dist->lpi_translation_cache, entry) { + list_for_each_entry_rcu(cte, &dist->lpi_translation_cache, entry) { /* * If we hit a NULL entry, there is nothing after this * point. @@ -1981,7 +1981,7 @@ void vgic_lpi_translation_cache_destroy(struct kvm *kvm) list_for_each_entry_safe(cte, tmp, &dist->lpi_translation_cache, entry) { - list_del(&cte->entry); + list_del_rcu(&cte->entry); kfree(cte); } } From patchwork Tue Feb 13 09:40:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 200303 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp428054dyb; Tue, 13 Feb 2024 01:46:28 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCU7G+QWCmdPsT42//8qrinAXhFlidVJXcijXDREw0mMb9tPioQwJ0dL7/9M+r8S3Gl4bwpz24SEUY027HRY0j8Ad19wdw== X-Google-Smtp-Source: AGHT+IFHEf9MACYdJu7NcecVQWO8MiBmxg4+n16s1hgOb1ic3Uqo2Cio9USocnNLtLaQM1JKL9n1 X-Received: by 2002:a0c:e254:0:b0:68c:cb5b:25a0 with SMTP id x20-20020a0ce254000000b0068ccb5b25a0mr2756206qvl.26.1707817588045; Tue, 13 Feb 2024 01:46:28 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707817588; cv=pass; d=google.com; s=arc-20160816; b=SfoAydazsR2ojP7R3IddsD3YSNF4fMMk9xvrUeQVBYjO0SwJOvrf2CbN36MLIbMTAW Zv3UYihd4COeysbTcpxmd66xrEUmXbgr7AEXC1fmyvHj9wOMuoZyZ9/kOm4bJw80FWFY pDG+AQllkmrMTmtQyAcRiup7xQfkXTT7rddtgUCbIY/TVyt/NhW2A5npJs4YuWtm2L+4 zEA/NfvmIqfsKPWHPSsq+C8ePBEwJ12bzZQVREaZjakOVxGh/pK0ZLihUpGsEdlxx6GY KBoEQXdPvTB9g3PTcoCRojfSmAr3NWW7937Jmq/ci+pSh9ViCnHsjq/msjZmVRRe+8lh 7gAg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=OvMkuTAmZ2Y4I5mIeurgoX7p1vggpBUJZeCNdIuhqjI=; fh=dLHm0KwAywrebbLhXAB9MrbXzDvW3d37esTQcgIg0bs=; b=QsEFKX64WiQRLtXaJ9ZINf6V/RvUnCloyJQPciyoihPXpfe7W2GNXhtEebsoAoIIH5 PK8tLCuNsTmcWJ8I0HygfQmoadN3JqAOmd0IXeKeIrTNv6kjZ557EtzWKkOf03ieoMIa 4P/roUzkrYV5i7sawKcL6NP9OVVH3tPSc8TSuecl2NyIDZoxyOTqVNxq9g//pFMezAMB KfoAbEvA9LyRy1ZHXDn3oHUriX8j0tMSEdwCZOpirMvQFmbNumnBvjMXdOUrI7h9w1to krszqVKANVGe4u7NYZTYgn3YtvMvZLYBSWd0k/HNL26b268O261JBw0WI636VQFyQvW5 WQHg==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b="A9/0Pdap"; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63240-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63240-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev X-Forwarded-Encrypted: i=2; AJvYcCWqwTpialHOPScgbX7tP4dsu7byNOOsKDjfrYkmUPSPysx288ra4mrEnW/eKhNwOWM5n9N+JM5KV18XMVln0SccDleTBg== Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id jf11-20020a0562142a4b00b00680b0f40fb4si2502581qvb.206.2024.02.13.01.46.27 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 01:46:28 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-63240-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b="A9/0Pdap"; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63240-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63240-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 2BE021C2630B for ; Tue, 13 Feb 2024 09:45:13 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 6B3C225564; Tue, 13 Feb 2024 09:40:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="A9/0Pdap" Received: from out-171.mta0.migadu.com (out-171.mta0.migadu.com [91.218.175.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 591361E4A2 for ; Tue, 13 Feb 2024 09:40:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707817224; cv=none; b=eXC8o1sKJp3j1+h3EHAzFi5vihfgD27Yg/5e3sRH8ahjPmeicWtRKUvpT2YuNePOx7s9QjwM26iUZ4GIhFr1fkrhsbR1Ym7/26qEM4i9VueezdPYOV3HCaKLRaq25gMkMsMegrD8ZWM16W6EWIdsH2ttmogYr9OATSfwVBHBBfo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707817224; c=relaxed/simple; bh=Hw7vNI2Kb24vC0nkaK7/5I/kAuFRQ3NHTOq4CoOcbhQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=SRcsZD2IKNWwjzUeFb6So1dU5+RjAEsoqBvuz2Rzm12bITGpajaqHA7tlgMsNsb/ooBy+R2CaVkS/XhpJyGiHZ9vOqH9Tf6jEdTZI9HBmGiuACD60jQqf1djZ38IUA1RiUgO8nQGU7q/8/+zJlO+JgvEZQEHl1XIfqv6hrc9FFc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=A9/0Pdap; arc=none smtp.client-ip=91.218.175.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1707817221; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OvMkuTAmZ2Y4I5mIeurgoX7p1vggpBUJZeCNdIuhqjI=; b=A9/0Pdapfde4UMXWBuOSYEnsYs3XzXeTFT8/HtvInZYMXQGfoyHtP9GTyIEbV2KV6N5ZJf 1GzoPWM9GzgXptD4kKnxk5qSHY2zanY52NPwmDaJzKgsZSISUa26dRM3mo7RTWUFOcRMKa pXPEbI09FCmKKPQNM2lUj+U/uXTIovE= From: Oliver Upton To: kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , linux-kernel@vger.kernel.org, Oliver Upton Subject: [PATCH v2 16/23] KVM: arm64: vgic-its: Rely on RCU to protect translation cache reads Date: Tue, 13 Feb 2024 09:40:12 +0000 Message-ID: <20240213094012.3961447-1-oliver.upton@linux.dev> In-Reply-To: <20240213093250.3960069-1-oliver.upton@linux.dev> References: <20240213093250.3960069-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790776535015395701 X-GMAIL-MSGID: 1790776535015395701 Stop taking the lpi_list_lock when reading the LPI translation cache. Signed-off-by: Oliver Upton --- arch/arm64/kvm/vgic/vgic-its.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/arch/arm64/kvm/vgic/vgic-its.c b/arch/arm64/kvm/vgic/vgic-its.c index 55463eb84763..0e449f8c1f01 100644 --- a/arch/arm64/kvm/vgic/vgic-its.c +++ b/arch/arm64/kvm/vgic/vgic-its.c @@ -593,15 +593,14 @@ static struct vgic_irq *vgic_its_check_cache(struct kvm *kvm, phys_addr_t db, { struct vgic_dist *dist = &kvm->arch.vgic; struct vgic_irq *irq; - unsigned long flags; - raw_spin_lock_irqsave(&dist->lpi_list_lock, flags); + rcu_read_lock(); irq = __vgic_its_check_cache(dist, db, devid, eventid); if (!vgic_try_get_irq_kref(irq)) irq = NULL; - raw_spin_unlock_irqrestore(&dist->lpi_list_lock, flags); + rcu_read_unlock(); return irq; } From patchwork Tue Feb 13 09:40:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 200300 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp427727dyb; Tue, 13 Feb 2024 01:45:35 -0800 (PST) X-Google-Smtp-Source: AGHT+IHiN88LJFhIpgI4ucfBSKI7ShxeXzTLbvgxJGk5tL2ydRB88zgBpwXqwgNObuAgekd2sGiY X-Received: by 2002:a17:906:5615:b0:a3c:2da5:93b2 with SMTP id f21-20020a170906561500b00a3c2da593b2mr5485126ejq.1.1707817535148; Tue, 13 Feb 2024 01:45:35 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707817535; cv=pass; d=google.com; s=arc-20160816; b=hminfgc6inGbnzxxQRHsYiCfYjhRJ0f3FHYMX7EuA7MIhMzSaMYj5Hp8NU0o3haOeO SxtYpfwgi3xYZP9oxLxmIHIYsViyhYIqZEs8zcJQNwXTfgbVrmB0L0XhNW775Pr47Poq 8ieR8lwCDCjuCaNlTlysvFMWaEwifwsM2JoVUEiBcDZn08UAzy5SlGOTJ+8EvNQB0UZy zYrL3cxGKCIwCOWtxe6KTufWHsCOjMxHieHUfUvsUD6btrOK1xI9AFYeqHC+a4UytV/L zEATJrpky2yrPs+AeQLP5Ac7vypFeQ72oNNkncXOAXhEECi43FQ7W2K8FWZLrt++ulfE 4aqw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=vQ5hY5omSH31iPaOd7xWz0jMZ8mQtyKDXoozVP52xcM=; fh=Ui9e/LecTOPcFKhO7/9rY+cU2EdXB8gHSSkdtKg8Fos=; b=yIf5ktmiXKTXoGoxV/cpz+kklkbEwlvRlLo7IrvMgps3GGejY2KA8QU4sylqNt2Eme OmQ6n87Ofy5OvEleB9lJzM4PKqdOfDWeLpzlo3/Khg44gxXMDUTAx/oSaA6GUKIQ6LlS baeVSukbElkAArhIYpXJaZjvVenn4Bky44U1HZtI3K5FDQ0RNo9HfV9nbAHM2Dih0h6e 15z8oVc16nFDBpJ87PlNZLxrBphhkEvSQpueLuC5nt0ocPaR52Thp9KCcCTUfYnNI5Ik JajbdaN5xXd52rpkvUDxkx1yiTVi8gAY3l1E2y44/8C12Bj9EAq0hT/kyIkbC3Vu8oQE k/kA==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=I5sE7uuY; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63241-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63241-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev X-Forwarded-Encrypted: i=2; AJvYcCXDBu0Klsp9CL1dcs7VguT2MigJFdtADstWXGGeWNSpJvoQ35taTma4bfHDPA1xPdpEgKfq3QinpZGffXzq0NvYf9zLjQ== Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id l17-20020a1709065a9100b00a3d18bc7aa8si101525ejq.527.2024.02.13.01.45.34 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 01:45:35 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-63241-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=I5sE7uuY; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63241-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63241-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 67CB81F27FBA for ; Tue, 13 Feb 2024 09:45:34 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C0F702BB10; Tue, 13 Feb 2024 09:40:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="I5sE7uuY" Received: from out-184.mta0.migadu.com (out-184.mta0.migadu.com [91.218.175.184]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E42F024A00 for ; Tue, 13 Feb 2024 09:40:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.184 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707817247; cv=none; b=UnyFbYdcmLbSfMplVFzxnW55dKphWn469szFFjqhm2V4Jpc9eeveM49nV8mVDhk6ydWLTVftVXAh0NcS3f7CLv7PNalaXxgaGNKzUZwQb+f1DuF8tMZ7iORDguF0EYhk7wkoA3T5Wh8S7inZ+ssUFvc6heW/D92iMq0VI3u+3Yg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707817247; c=relaxed/simple; bh=+RFQwI3+7dStn2MORPKtaD6XfanOleuuamNkdQRiV8o=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=tTwcWTlH+nd8u9iU6gP6XCwhVeyj45TZs2yQaLLniTCFgLGrmH0Wfb7ODBedamXymA+bz3LFp7p3EL9wq1/VRDKYaYbfYUsSSoP+VGRvxqAPmB+LMG5udV85J1ooAEUsQXj4GUTJEcsToCeu1bBjliYwjB2r107bXC94aWdoBHA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=I5sE7uuY; arc=none smtp.client-ip=91.218.175.184 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1707817243; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vQ5hY5omSH31iPaOd7xWz0jMZ8mQtyKDXoozVP52xcM=; b=I5sE7uuYCAGNowEs0MD09extGnJuRcTdkNU2cXScCRvPRFwRIBVt5t7OR4tTLmnzCOr/lG 08wutY+CegBhu1BhjCjHynaaqem9KODOaU65BMipnTqOVAVkEHKHNBALzbv6MvtKS05ANj 7DJFDh0OOl3LTVyCT55Q5P/iOqgmi8M= From: Oliver Upton To: kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , linux-kernel@vger.kernel.org, Oliver Upton Subject: [PATCH v2 17/23] KVM: selftests: Align with kernel's GIC definitions Date: Tue, 13 Feb 2024 09:40:29 +0000 Message-ID: <20240213094029.3961571-1-oliver.upton@linux.dev> In-Reply-To: <20240213093250.3960069-1-oliver.upton@linux.dev> References: <20240213093250.3960069-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790776479990772531 X-GMAIL-MSGID: 1790776479990772531 There are a few subtle incongruencies between the GIC definitions used by the kernel and selftests. Furthermore, the selftests header blends implementation detail (e.g. default priority) with the architectural definitions. This is all rather annoying, since bulk imports of the kernel header is not possible. Move selftests-specific definitions out of the offending header and realign tests on the canonical definitions for things like sysregs. Finally, haul in a fresh copy of the gicv3 header to enable a forthcoming ITS selftest. Signed-off-by: Oliver Upton --- .../testing/selftests/kvm/aarch64/vgic_irq.c | 4 +- .../selftests/kvm/include/aarch64/gic_v3.h | 586 +++++++++++++++++- .../selftests/kvm/lib/aarch64/gic_v3.c | 13 +- 3 files changed, 568 insertions(+), 35 deletions(-) diff --git a/tools/testing/selftests/kvm/aarch64/vgic_irq.c b/tools/testing/selftests/kvm/aarch64/vgic_irq.c index 2e64b4856e38..d61a6302f467 100644 --- a/tools/testing/selftests/kvm/aarch64/vgic_irq.c +++ b/tools/testing/selftests/kvm/aarch64/vgic_irq.c @@ -152,7 +152,7 @@ static void reset_stats(void) static uint64_t gic_read_ap1r0(void) { - uint64_t reg = read_sysreg_s(SYS_ICV_AP1R0_EL1); + uint64_t reg = read_sysreg_s(SYS_ICC_AP1R0_EL1); dsb(sy); return reg; @@ -160,7 +160,7 @@ static uint64_t gic_read_ap1r0(void) static void gic_write_ap1r0(uint64_t val) { - write_sysreg_s(val, SYS_ICV_AP1R0_EL1); + write_sysreg_s(val, SYS_ICC_AP1R0_EL1); isb(); } diff --git a/tools/testing/selftests/kvm/include/aarch64/gic_v3.h b/tools/testing/selftests/kvm/include/aarch64/gic_v3.h index ba0886e8a2bb..a76615fa39a1 100644 --- a/tools/testing/selftests/kvm/include/aarch64/gic_v3.h +++ b/tools/testing/selftests/kvm/include/aarch64/gic_v3.h @@ -1,82 +1,604 @@ -/* SPDX-License-Identifier: GPL-2.0 */ +/* SPDX-License-Identifier: GPL-2.0-only */ /* - * ARM Generic Interrupt Controller (GIC) v3 specific defines + * Copyright (C) 2013, 2014 ARM Limited, All Rights Reserved. + * Author: Marc Zyngier */ - -#ifndef SELFTEST_KVM_GICV3_H -#define SELFTEST_KVM_GICV3_H - -#include +#ifndef __SELFTESTS_GIC_V3_H +#define __SELFTESTS_GIC_V3_H /* - * Distributor registers + * Distributor registers. We assume we're running non-secure, with ARE + * being set. Secure-only and non-ARE registers are not described. */ #define GICD_CTLR 0x0000 #define GICD_TYPER 0x0004 +#define GICD_IIDR 0x0008 +#define GICD_TYPER2 0x000C +#define GICD_STATUSR 0x0010 +#define GICD_SETSPI_NSR 0x0040 +#define GICD_CLRSPI_NSR 0x0048 +#define GICD_SETSPI_SR 0x0050 +#define GICD_CLRSPI_SR 0x0058 #define GICD_IGROUPR 0x0080 #define GICD_ISENABLER 0x0100 #define GICD_ICENABLER 0x0180 #define GICD_ISPENDR 0x0200 #define GICD_ICPENDR 0x0280 -#define GICD_ICACTIVER 0x0380 #define GICD_ISACTIVER 0x0300 +#define GICD_ICACTIVER 0x0380 #define GICD_IPRIORITYR 0x0400 #define GICD_ICFGR 0x0C00 +#define GICD_IGRPMODR 0x0D00 +#define GICD_NSACR 0x0E00 +#define GICD_IGROUPRnE 0x1000 +#define GICD_ISENABLERnE 0x1200 +#define GICD_ICENABLERnE 0x1400 +#define GICD_ISPENDRnE 0x1600 +#define GICD_ICPENDRnE 0x1800 +#define GICD_ISACTIVERnE 0x1A00 +#define GICD_ICACTIVERnE 0x1C00 +#define GICD_IPRIORITYRnE 0x2000 +#define GICD_ICFGRnE 0x3000 +#define GICD_IROUTER 0x6000 +#define GICD_IROUTERnE 0x8000 +#define GICD_IDREGS 0xFFD0 +#define GICD_PIDR2 0xFFE8 + +#define ESPI_BASE_INTID 4096 /* - * The assumption is that the guest runs in a non-secure mode. - * The following bits of GICD_CTLR are defined accordingly. + * Those registers are actually from GICv2, but the spec demands that they + * are implemented as RES0 if ARE is 1 (which we do in KVM's emulated GICv3). */ +#define GICD_ITARGETSR 0x0800 +#define GICD_SGIR 0x0F00 +#define GICD_CPENDSGIR 0x0F10 +#define GICD_SPENDSGIR 0x0F20 + #define GICD_CTLR_RWP (1U << 31) #define GICD_CTLR_nASSGIreq (1U << 8) +#define GICD_CTLR_DS (1U << 6) #define GICD_CTLR_ARE_NS (1U << 4) #define GICD_CTLR_ENABLE_G1A (1U << 1) #define GICD_CTLR_ENABLE_G1 (1U << 0) +#define GICD_IIDR_IMPLEMENTER_SHIFT 0 +#define GICD_IIDR_IMPLEMENTER_MASK (0xfff << GICD_IIDR_IMPLEMENTER_SHIFT) +#define GICD_IIDR_REVISION_SHIFT 12 +#define GICD_IIDR_REVISION_MASK (0xf << GICD_IIDR_REVISION_SHIFT) +#define GICD_IIDR_VARIANT_SHIFT 16 +#define GICD_IIDR_VARIANT_MASK (0xf << GICD_IIDR_VARIANT_SHIFT) +#define GICD_IIDR_PRODUCT_ID_SHIFT 24 +#define GICD_IIDR_PRODUCT_ID_MASK (0xff << GICD_IIDR_PRODUCT_ID_SHIFT) + + +/* + * In systems with a single security state (what we emulate in KVM) + * the meaning of the interrupt group enable bits is slightly different + */ +#define GICD_CTLR_ENABLE_SS_G1 (1U << 1) +#define GICD_CTLR_ENABLE_SS_G0 (1U << 0) + +#define GICD_TYPER_RSS (1U << 26) +#define GICD_TYPER_LPIS (1U << 17) +#define GICD_TYPER_MBIS (1U << 16) +#define GICD_TYPER_ESPI (1U << 8) + +#define GICD_TYPER_ID_BITS(typer) ((((typer) >> 19) & 0x1f) + 1) +#define GICD_TYPER_NUM_LPIS(typer) ((((typer) >> 11) & 0x1f) + 1) #define GICD_TYPER_SPIS(typer) ((((typer) & 0x1f) + 1) * 32) -#define GICD_INT_DEF_PRI_X4 0xa0a0a0a0 +#define GICD_TYPER_ESPIS(typer) \ + (((typer) & GICD_TYPER_ESPI) ? GICD_TYPER_SPIS((typer) >> 27) : 0) + +#define GICD_TYPER2_nASSGIcap (1U << 8) +#define GICD_TYPER2_VIL (1U << 7) +#define GICD_TYPER2_VID GENMASK(4, 0) + +#define GICD_IROUTER_SPI_MODE_ONE (0U << 31) +#define GICD_IROUTER_SPI_MODE_ANY (1U << 31) + +#define GIC_PIDR2_ARCH_MASK 0xf0 +#define GIC_PIDR2_ARCH_GICv3 0x30 +#define GIC_PIDR2_ARCH_GICv4 0x40 + +#define GIC_V3_DIST_SIZE 0x10000 + +#define GIC_PAGE_SIZE_4K 0ULL +#define GIC_PAGE_SIZE_16K 1ULL +#define GIC_PAGE_SIZE_64K 2ULL +#define GIC_PAGE_SIZE_MASK 3ULL /* - * Redistributor registers + * Re-Distributor registers, offsets from RD_base */ -#define GICR_CTLR 0x000 -#define GICR_WAKER 0x014 +#define GICR_CTLR GICD_CTLR +#define GICR_IIDR 0x0004 +#define GICR_TYPER 0x0008 +#define GICR_STATUSR GICD_STATUSR +#define GICR_WAKER 0x0014 +#define GICR_SETLPIR 0x0040 +#define GICR_CLRLPIR 0x0048 +#define GICR_PROPBASER 0x0070 +#define GICR_PENDBASER 0x0078 +#define GICR_INVLPIR 0x00A0 +#define GICR_INVALLR 0x00B0 +#define GICR_SYNCR 0x00C0 +#define GICR_IDREGS GICD_IDREGS +#define GICR_PIDR2 GICD_PIDR2 + +#define GICR_CTLR_ENABLE_LPIS (1UL << 0) +#define GICR_CTLR_CES (1UL << 1) +#define GICR_CTLR_IR (1UL << 2) +#define GICR_CTLR_RWP (1UL << 3) -#define GICR_CTLR_RWP (1U << 3) +#define GICR_TYPER_CPU_NUMBER(r) (((r) >> 8) & 0xffff) + +#define EPPI_BASE_INTID 1056 + +#define GICR_TYPER_NR_PPIS(r) \ + ({ \ + unsigned int __ppinum = ((r) >> 27) & 0x1f; \ + unsigned int __nr_ppis = 16; \ + if (__ppinum == 1 || __ppinum == 2) \ + __nr_ppis += __ppinum * 32; \ + \ + __nr_ppis; \ + }) #define GICR_WAKER_ProcessorSleep (1U << 1) #define GICR_WAKER_ChildrenAsleep (1U << 2) +#define GIC_BASER_CACHE_nCnB 0ULL +#define GIC_BASER_CACHE_SameAsInner 0ULL +#define GIC_BASER_CACHE_nC 1ULL +#define GIC_BASER_CACHE_RaWt 2ULL +#define GIC_BASER_CACHE_RaWb 3ULL +#define GIC_BASER_CACHE_WaWt 4ULL +#define GIC_BASER_CACHE_WaWb 5ULL +#define GIC_BASER_CACHE_RaWaWt 6ULL +#define GIC_BASER_CACHE_RaWaWb 7ULL +#define GIC_BASER_CACHE_MASK 7ULL +#define GIC_BASER_NonShareable 0ULL +#define GIC_BASER_InnerShareable 1ULL +#define GIC_BASER_OuterShareable 2ULL +#define GIC_BASER_SHAREABILITY_MASK 3ULL + +#define GIC_BASER_CACHEABILITY(reg, inner_outer, type) \ + (GIC_BASER_CACHE_##type << reg##_##inner_outer##_CACHEABILITY_SHIFT) + +#define GIC_BASER_SHAREABILITY(reg, type) \ + (GIC_BASER_##type << reg##_SHAREABILITY_SHIFT) + +/* encode a size field of width @w containing @n - 1 units */ +#define GIC_ENCODE_SZ(n, w) (((unsigned long)(n) - 1) & GENMASK_ULL(((w) - 1), 0)) + +#define GICR_PROPBASER_SHAREABILITY_SHIFT (10) +#define GICR_PROPBASER_INNER_CACHEABILITY_SHIFT (7) +#define GICR_PROPBASER_OUTER_CACHEABILITY_SHIFT (56) +#define GICR_PROPBASER_SHAREABILITY_MASK \ + GIC_BASER_SHAREABILITY(GICR_PROPBASER, SHAREABILITY_MASK) +#define GICR_PROPBASER_INNER_CACHEABILITY_MASK \ + GIC_BASER_CACHEABILITY(GICR_PROPBASER, INNER, MASK) +#define GICR_PROPBASER_OUTER_CACHEABILITY_MASK \ + GIC_BASER_CACHEABILITY(GICR_PROPBASER, OUTER, MASK) +#define GICR_PROPBASER_CACHEABILITY_MASK GICR_PROPBASER_INNER_CACHEABILITY_MASK + +#define GICR_PROPBASER_InnerShareable \ + GIC_BASER_SHAREABILITY(GICR_PROPBASER, InnerShareable) + +#define GICR_PROPBASER_nCnB GIC_BASER_CACHEABILITY(GICR_PROPBASER, INNER, nCnB) +#define GICR_PROPBASER_nC GIC_BASER_CACHEABILITY(GICR_PROPBASER, INNER, nC) +#define GICR_PROPBASER_RaWt GIC_BASER_CACHEABILITY(GICR_PROPBASER, INNER, RaWt) +#define GICR_PROPBASER_RaWb GIC_BASER_CACHEABILITY(GICR_PROPBASER, INNER, RaWb) +#define GICR_PROPBASER_WaWt GIC_BASER_CACHEABILITY(GICR_PROPBASER, INNER, WaWt) +#define GICR_PROPBASER_WaWb GIC_BASER_CACHEABILITY(GICR_PROPBASER, INNER, WaWb) +#define GICR_PROPBASER_RaWaWt GIC_BASER_CACHEABILITY(GICR_PROPBASER, INNER, RaWaWt) +#define GICR_PROPBASER_RaWaWb GIC_BASER_CACHEABILITY(GICR_PROPBASER, INNER, RaWaWb) + +#define GICR_PROPBASER_IDBITS_MASK (0x1f) +#define GICR_PROPBASER_ADDRESS(x) ((x) & GENMASK_ULL(51, 12)) +#define GICR_PENDBASER_ADDRESS(x) ((x) & GENMASK_ULL(51, 16)) + +#define GICR_PENDBASER_SHAREABILITY_SHIFT (10) +#define GICR_PENDBASER_INNER_CACHEABILITY_SHIFT (7) +#define GICR_PENDBASER_OUTER_CACHEABILITY_SHIFT (56) +#define GICR_PENDBASER_SHAREABILITY_MASK \ + GIC_BASER_SHAREABILITY(GICR_PENDBASER, SHAREABILITY_MASK) +#define GICR_PENDBASER_INNER_CACHEABILITY_MASK \ + GIC_BASER_CACHEABILITY(GICR_PENDBASER, INNER, MASK) +#define GICR_PENDBASER_OUTER_CACHEABILITY_MASK \ + GIC_BASER_CACHEABILITY(GICR_PENDBASER, OUTER, MASK) +#define GICR_PENDBASER_CACHEABILITY_MASK GICR_PENDBASER_INNER_CACHEABILITY_MASK + +#define GICR_PENDBASER_InnerShareable \ + GIC_BASER_SHAREABILITY(GICR_PENDBASER, InnerShareable) + +#define GICR_PENDBASER_nCnB GIC_BASER_CACHEABILITY(GICR_PENDBASER, INNER, nCnB) +#define GICR_PENDBASER_nC GIC_BASER_CACHEABILITY(GICR_PENDBASER, INNER, nC) +#define GICR_PENDBASER_RaWt GIC_BASER_CACHEABILITY(GICR_PENDBASER, INNER, RaWt) +#define GICR_PENDBASER_RaWb GIC_BASER_CACHEABILITY(GICR_PENDBASER, INNER, RaWb) +#define GICR_PENDBASER_WaWt GIC_BASER_CACHEABILITY(GICR_PENDBASER, INNER, WaWt) +#define GICR_PENDBASER_WaWb GIC_BASER_CACHEABILITY(GICR_PENDBASER, INNER, WaWb) +#define GICR_PENDBASER_RaWaWt GIC_BASER_CACHEABILITY(GICR_PENDBASER, INNER, RaWaWt) +#define GICR_PENDBASER_RaWaWb GIC_BASER_CACHEABILITY(GICR_PENDBASER, INNER, RaWaWb) + +#define GICR_PENDBASER_PTZ BIT_ULL(62) + /* - * Redistributor registers, offsets from SGI base + * Re-Distributor registers, offsets from SGI_base */ #define GICR_IGROUPR0 GICD_IGROUPR #define GICR_ISENABLER0 GICD_ISENABLER #define GICR_ICENABLER0 GICD_ICENABLER #define GICR_ISPENDR0 GICD_ISPENDR +#define GICR_ICPENDR0 GICD_ICPENDR #define GICR_ISACTIVER0 GICD_ISACTIVER #define GICR_ICACTIVER0 GICD_ICACTIVER -#define GICR_ICENABLER GICD_ICENABLER -#define GICR_ICACTIVER GICD_ICACTIVER #define GICR_IPRIORITYR0 GICD_IPRIORITYR +#define GICR_ICFGR0 GICD_ICFGR +#define GICR_IGRPMODR0 GICD_IGRPMODR +#define GICR_NSACR GICD_NSACR + +#define GICR_TYPER_PLPIS (1U << 0) +#define GICR_TYPER_VLPIS (1U << 1) +#define GICR_TYPER_DIRTY (1U << 2) +#define GICR_TYPER_DirectLPIS (1U << 3) +#define GICR_TYPER_LAST (1U << 4) +#define GICR_TYPER_RVPEID (1U << 7) +#define GICR_TYPER_COMMON_LPI_AFF GENMASK_ULL(25, 24) +#define GICR_TYPER_AFFINITY GENMASK_ULL(63, 32) + +#define GICR_INVLPIR_INTID GENMASK_ULL(31, 0) +#define GICR_INVLPIR_VPEID GENMASK_ULL(47, 32) +#define GICR_INVLPIR_V GENMASK_ULL(63, 63) + +#define GICR_INVALLR_VPEID GICR_INVLPIR_VPEID +#define GICR_INVALLR_V GICR_INVLPIR_V + +#define GIC_V3_REDIST_SIZE 0x20000 + +#define LPI_PROP_GROUP1 (1 << 1) +#define LPI_PROP_ENABLED (1 << 0) + +/* + * Re-Distributor registers, offsets from VLPI_base + */ +#define GICR_VPROPBASER 0x0070 + +#define GICR_VPROPBASER_IDBITS_MASK 0x1f + +#define GICR_VPROPBASER_SHAREABILITY_SHIFT (10) +#define GICR_VPROPBASER_INNER_CACHEABILITY_SHIFT (7) +#define GICR_VPROPBASER_OUTER_CACHEABILITY_SHIFT (56) + +#define GICR_VPROPBASER_SHAREABILITY_MASK \ + GIC_BASER_SHAREABILITY(GICR_VPROPBASER, SHAREABILITY_MASK) +#define GICR_VPROPBASER_INNER_CACHEABILITY_MASK \ + GIC_BASER_CACHEABILITY(GICR_VPROPBASER, INNER, MASK) +#define GICR_VPROPBASER_OUTER_CACHEABILITY_MASK \ + GIC_BASER_CACHEABILITY(GICR_VPROPBASER, OUTER, MASK) +#define GICR_VPROPBASER_CACHEABILITY_MASK \ + GICR_VPROPBASER_INNER_CACHEABILITY_MASK + +#define GICR_VPROPBASER_InnerShareable \ + GIC_BASER_SHAREABILITY(GICR_VPROPBASER, InnerShareable) + +#define GICR_VPROPBASER_nCnB GIC_BASER_CACHEABILITY(GICR_VPROPBASER, INNER, nCnB) +#define GICR_VPROPBASER_nC GIC_BASER_CACHEABILITY(GICR_VPROPBASER, INNER, nC) +#define GICR_VPROPBASER_RaWt GIC_BASER_CACHEABILITY(GICR_VPROPBASER, INNER, RaWt) +#define GICR_VPROPBASER_RaWb GIC_BASER_CACHEABILITY(GICR_VPROPBASER, INNER, RaWb) +#define GICR_VPROPBASER_WaWt GIC_BASER_CACHEABILITY(GICR_VPROPBASER, INNER, WaWt) +#define GICR_VPROPBASER_WaWb GIC_BASER_CACHEABILITY(GICR_VPROPBASER, INNER, WaWb) +#define GICR_VPROPBASER_RaWaWt GIC_BASER_CACHEABILITY(GICR_VPROPBASER, INNER, RaWaWt) +#define GICR_VPROPBASER_RaWaWb GIC_BASER_CACHEABILITY(GICR_VPROPBASER, INNER, RaWaWb) + +/* + * GICv4.1 VPROPBASER reinvention. A subtle mix between the old + * VPROPBASER and ITS_BASER. Just not quite any of the two. + */ +#define GICR_VPROPBASER_4_1_VALID (1ULL << 63) +#define GICR_VPROPBASER_4_1_ENTRY_SIZE GENMASK_ULL(61, 59) +#define GICR_VPROPBASER_4_1_INDIRECT (1ULL << 55) +#define GICR_VPROPBASER_4_1_PAGE_SIZE GENMASK_ULL(54, 53) +#define GICR_VPROPBASER_4_1_Z (1ULL << 52) +#define GICR_VPROPBASER_4_1_ADDR GENMASK_ULL(51, 12) +#define GICR_VPROPBASER_4_1_SIZE GENMASK_ULL(6, 0) + +#define GICR_VPENDBASER 0x0078 + +#define GICR_VPENDBASER_SHAREABILITY_SHIFT (10) +#define GICR_VPENDBASER_INNER_CACHEABILITY_SHIFT (7) +#define GICR_VPENDBASER_OUTER_CACHEABILITY_SHIFT (56) +#define GICR_VPENDBASER_SHAREABILITY_MASK \ + GIC_BASER_SHAREABILITY(GICR_VPENDBASER, SHAREABILITY_MASK) +#define GICR_VPENDBASER_INNER_CACHEABILITY_MASK \ + GIC_BASER_CACHEABILITY(GICR_VPENDBASER, INNER, MASK) +#define GICR_VPENDBASER_OUTER_CACHEABILITY_MASK \ + GIC_BASER_CACHEABILITY(GICR_VPENDBASER, OUTER, MASK) +#define GICR_VPENDBASER_CACHEABILITY_MASK \ + GICR_VPENDBASER_INNER_CACHEABILITY_MASK + +#define GICR_VPENDBASER_NonShareable \ + GIC_BASER_SHAREABILITY(GICR_VPENDBASER, NonShareable) + +#define GICR_VPENDBASER_InnerShareable \ + GIC_BASER_SHAREABILITY(GICR_VPENDBASER, InnerShareable) + +#define GICR_VPENDBASER_nCnB GIC_BASER_CACHEABILITY(GICR_VPENDBASER, INNER, nCnB) +#define GICR_VPENDBASER_nC GIC_BASER_CACHEABILITY(GICR_VPENDBASER, INNER, nC) +#define GICR_VPENDBASER_RaWt GIC_BASER_CACHEABILITY(GICR_VPENDBASER, INNER, RaWt) +#define GICR_VPENDBASER_RaWb GIC_BASER_CACHEABILITY(GICR_VPENDBASER, INNER, RaWb) +#define GICR_VPENDBASER_WaWt GIC_BASER_CACHEABILITY(GICR_VPENDBASER, INNER, WaWt) +#define GICR_VPENDBASER_WaWb GIC_BASER_CACHEABILITY(GICR_VPENDBASER, INNER, WaWb) +#define GICR_VPENDBASER_RaWaWt GIC_BASER_CACHEABILITY(GICR_VPENDBASER, INNER, RaWaWt) +#define GICR_VPENDBASER_RaWaWb GIC_BASER_CACHEABILITY(GICR_VPENDBASER, INNER, RaWaWb) + +#define GICR_VPENDBASER_Dirty (1ULL << 60) +#define GICR_VPENDBASER_PendingLast (1ULL << 61) +#define GICR_VPENDBASER_IDAI (1ULL << 62) +#define GICR_VPENDBASER_Valid (1ULL << 63) + +/* + * GICv4.1 VPENDBASER, used for VPE residency. On top of these fields, + * also use the above Valid, PendingLast and Dirty. + */ +#define GICR_VPENDBASER_4_1_DB (1ULL << 62) +#define GICR_VPENDBASER_4_1_VGRP0EN (1ULL << 59) +#define GICR_VPENDBASER_4_1_VGRP1EN (1ULL << 58) +#define GICR_VPENDBASER_4_1_VPEID GENMASK_ULL(15, 0) + +#define GICR_VSGIR 0x0080 + +#define GICR_VSGIR_VPEID GENMASK(15, 0) + +#define GICR_VSGIPENDR 0x0088 + +#define GICR_VSGIPENDR_BUSY (1U << 31) +#define GICR_VSGIPENDR_PENDING GENMASK(15, 0) + +/* + * ITS registers, offsets from ITS_base + */ +#define GITS_CTLR 0x0000 +#define GITS_IIDR 0x0004 +#define GITS_TYPER 0x0008 +#define GITS_MPIDR 0x0018 +#define GITS_CBASER 0x0080 +#define GITS_CWRITER 0x0088 +#define GITS_CREADR 0x0090 +#define GITS_BASER 0x0100 +#define GITS_IDREGS_BASE 0xffd0 +#define GITS_PIDR0 0xffe0 +#define GITS_PIDR1 0xffe4 +#define GITS_PIDR2 GICR_PIDR2 +#define GITS_PIDR4 0xffd0 +#define GITS_CIDR0 0xfff0 +#define GITS_CIDR1 0xfff4 +#define GITS_CIDR2 0xfff8 +#define GITS_CIDR3 0xfffc + +#define GITS_TRANSLATER 0x10040 + +#define GITS_SGIR 0x20020 + +#define GITS_SGIR_VPEID GENMASK_ULL(47, 32) +#define GITS_SGIR_VINTID GENMASK_ULL(3, 0) + +#define GITS_CTLR_ENABLE (1U << 0) +#define GITS_CTLR_ImDe (1U << 1) +#define GITS_CTLR_ITS_NUMBER_SHIFT 4 +#define GITS_CTLR_ITS_NUMBER (0xFU << GITS_CTLR_ITS_NUMBER_SHIFT) +#define GITS_CTLR_QUIESCENT (1U << 31) + +#define GITS_TYPER_PLPIS (1UL << 0) +#define GITS_TYPER_VLPIS (1UL << 1) +#define GITS_TYPER_ITT_ENTRY_SIZE_SHIFT 4 +#define GITS_TYPER_ITT_ENTRY_SIZE GENMASK_ULL(7, 4) +#define GITS_TYPER_IDBITS_SHIFT 8 +#define GITS_TYPER_DEVBITS_SHIFT 13 +#define GITS_TYPER_DEVBITS GENMASK_ULL(17, 13) +#define GITS_TYPER_PTA (1UL << 19) +#define GITS_TYPER_HCC_SHIFT 24 +#define GITS_TYPER_HCC(r) (((r) >> GITS_TYPER_HCC_SHIFT) & 0xff) +#define GITS_TYPER_VMOVP (1ULL << 37) +#define GITS_TYPER_VMAPP (1ULL << 40) +#define GITS_TYPER_SVPET GENMASK_ULL(42, 41) -/* CPU interface registers */ -#define SYS_ICC_PMR_EL1 sys_reg(3, 0, 4, 6, 0) -#define SYS_ICC_IAR1_EL1 sys_reg(3, 0, 12, 12, 0) -#define SYS_ICC_EOIR1_EL1 sys_reg(3, 0, 12, 12, 1) -#define SYS_ICC_DIR_EL1 sys_reg(3, 0, 12, 11, 1) -#define SYS_ICC_CTLR_EL1 sys_reg(3, 0, 12, 12, 4) -#define SYS_ICC_SRE_EL1 sys_reg(3, 0, 12, 12, 5) -#define SYS_ICC_GRPEN1_EL1 sys_reg(3, 0, 12, 12, 7) +#define GITS_IIDR_REV_SHIFT 12 +#define GITS_IIDR_REV_MASK (0xf << GITS_IIDR_REV_SHIFT) +#define GITS_IIDR_REV(r) (((r) >> GITS_IIDR_REV_SHIFT) & 0xf) +#define GITS_IIDR_PRODUCTID_SHIFT 24 -#define SYS_ICV_AP1R0_EL1 sys_reg(3, 0, 12, 9, 0) +#define GITS_CBASER_VALID (1ULL << 63) +#define GITS_CBASER_SHAREABILITY_SHIFT (10) +#define GITS_CBASER_INNER_CACHEABILITY_SHIFT (59) +#define GITS_CBASER_OUTER_CACHEABILITY_SHIFT (53) +#define GITS_CBASER_SHAREABILITY_MASK \ + GIC_BASER_SHAREABILITY(GITS_CBASER, SHAREABILITY_MASK) +#define GITS_CBASER_INNER_CACHEABILITY_MASK \ + GIC_BASER_CACHEABILITY(GITS_CBASER, INNER, MASK) +#define GITS_CBASER_OUTER_CACHEABILITY_MASK \ + GIC_BASER_CACHEABILITY(GITS_CBASER, OUTER, MASK) +#define GITS_CBASER_CACHEABILITY_MASK GITS_CBASER_INNER_CACHEABILITY_MASK -#define ICC_PMR_DEF_PRIO 0xf0 +#define GITS_CBASER_InnerShareable \ + GIC_BASER_SHAREABILITY(GITS_CBASER, InnerShareable) +#define GITS_CBASER_nCnB GIC_BASER_CACHEABILITY(GITS_CBASER, INNER, nCnB) +#define GITS_CBASER_nC GIC_BASER_CACHEABILITY(GITS_CBASER, INNER, nC) +#define GITS_CBASER_RaWt GIC_BASER_CACHEABILITY(GITS_CBASER, INNER, RaWt) +#define GITS_CBASER_RaWb GIC_BASER_CACHEABILITY(GITS_CBASER, INNER, RaWb) +#define GITS_CBASER_WaWt GIC_BASER_CACHEABILITY(GITS_CBASER, INNER, WaWt) +#define GITS_CBASER_WaWb GIC_BASER_CACHEABILITY(GITS_CBASER, INNER, WaWb) +#define GITS_CBASER_RaWaWt GIC_BASER_CACHEABILITY(GITS_CBASER, INNER, RaWaWt) +#define GITS_CBASER_RaWaWb GIC_BASER_CACHEABILITY(GITS_CBASER, INNER, RaWaWb) + +#define GITS_CBASER_ADDRESS(cbaser) ((cbaser) & GENMASK_ULL(51, 12)) + +#define GITS_BASER_NR_REGS 8 + +#define GITS_BASER_VALID (1ULL << 63) +#define GITS_BASER_INDIRECT (1ULL << 62) + +#define GITS_BASER_INNER_CACHEABILITY_SHIFT (59) +#define GITS_BASER_OUTER_CACHEABILITY_SHIFT (53) +#define GITS_BASER_INNER_CACHEABILITY_MASK \ + GIC_BASER_CACHEABILITY(GITS_BASER, INNER, MASK) +#define GITS_BASER_CACHEABILITY_MASK GITS_BASER_INNER_CACHEABILITY_MASK +#define GITS_BASER_OUTER_CACHEABILITY_MASK \ + GIC_BASER_CACHEABILITY(GITS_BASER, OUTER, MASK) +#define GITS_BASER_SHAREABILITY_MASK \ + GIC_BASER_SHAREABILITY(GITS_BASER, SHAREABILITY_MASK) + +#define GITS_BASER_nCnB GIC_BASER_CACHEABILITY(GITS_BASER, INNER, nCnB) +#define GITS_BASER_nC GIC_BASER_CACHEABILITY(GITS_BASER, INNER, nC) +#define GITS_BASER_RaWt GIC_BASER_CACHEABILITY(GITS_BASER, INNER, RaWt) +#define GITS_BASER_RaWb GIC_BASER_CACHEABILITY(GITS_BASER, INNER, RaWb) +#define GITS_BASER_WaWt GIC_BASER_CACHEABILITY(GITS_BASER, INNER, WaWt) +#define GITS_BASER_WaWb GIC_BASER_CACHEABILITY(GITS_BASER, INNER, WaWb) +#define GITS_BASER_RaWaWt GIC_BASER_CACHEABILITY(GITS_BASER, INNER, RaWaWt) +#define GITS_BASER_RaWaWb GIC_BASER_CACHEABILITY(GITS_BASER, INNER, RaWaWb) + +#define GITS_BASER_TYPE_SHIFT (56) +#define GITS_BASER_TYPE(r) (((r) >> GITS_BASER_TYPE_SHIFT) & 7) +#define GITS_BASER_ENTRY_SIZE_SHIFT (48) +#define GITS_BASER_ENTRY_SIZE(r) ((((r) >> GITS_BASER_ENTRY_SIZE_SHIFT) & 0x1f) + 1) +#define GITS_BASER_ENTRY_SIZE_MASK GENMASK_ULL(52, 48) +#define GITS_BASER_PHYS_52_to_48(phys) \ + (((phys) & GENMASK_ULL(47, 16)) | (((phys) >> 48) & 0xf) << 12) +#define GITS_BASER_ADDR_48_to_52(baser) \ + (((baser) & GENMASK_ULL(47, 16)) | (((baser) >> 12) & 0xf) << 48) + +#define GITS_BASER_SHAREABILITY_SHIFT (10) +#define GITS_BASER_InnerShareable \ + GIC_BASER_SHAREABILITY(GITS_BASER, InnerShareable) +#define GITS_BASER_PAGE_SIZE_SHIFT (8) +#define __GITS_BASER_PSZ(sz) (GIC_PAGE_SIZE_ ## sz << GITS_BASER_PAGE_SIZE_SHIFT) +#define GITS_BASER_PAGE_SIZE_4K __GITS_BASER_PSZ(4K) +#define GITS_BASER_PAGE_SIZE_16K __GITS_BASER_PSZ(16K) +#define GITS_BASER_PAGE_SIZE_64K __GITS_BASER_PSZ(64K) +#define GITS_BASER_PAGE_SIZE_MASK __GITS_BASER_PSZ(MASK) +#define GITS_BASER_PAGES_MAX 256 +#define GITS_BASER_PAGES_SHIFT (0) +#define GITS_BASER_NR_PAGES(r) (((r) & 0xff) + 1) + +#define GITS_BASER_TYPE_NONE 0 +#define GITS_BASER_TYPE_DEVICE 1 +#define GITS_BASER_TYPE_VCPU 2 +#define GITS_BASER_TYPE_RESERVED3 3 +#define GITS_BASER_TYPE_COLLECTION 4 +#define GITS_BASER_TYPE_RESERVED5 5 +#define GITS_BASER_TYPE_RESERVED6 6 +#define GITS_BASER_TYPE_RESERVED7 7 + +#define GITS_LVL1_ENTRY_SIZE (8UL) + +/* + * ITS commands + */ +#define GITS_CMD_MAPD 0x08 +#define GITS_CMD_MAPC 0x09 +#define GITS_CMD_MAPTI 0x0a +#define GITS_CMD_MAPI 0x0b +#define GITS_CMD_MOVI 0x01 +#define GITS_CMD_DISCARD 0x0f +#define GITS_CMD_INV 0x0c +#define GITS_CMD_MOVALL 0x0e +#define GITS_CMD_INVALL 0x0d +#define GITS_CMD_INT 0x03 +#define GITS_CMD_CLEAR 0x04 +#define GITS_CMD_SYNC 0x05 + +/* + * GICv4 ITS specific commands + */ +#define GITS_CMD_GICv4(x) ((x) | 0x20) +#define GITS_CMD_VINVALL GITS_CMD_GICv4(GITS_CMD_INVALL) +#define GITS_CMD_VMAPP GITS_CMD_GICv4(GITS_CMD_MAPC) +#define GITS_CMD_VMAPTI GITS_CMD_GICv4(GITS_CMD_MAPTI) +#define GITS_CMD_VMOVI GITS_CMD_GICv4(GITS_CMD_MOVI) +#define GITS_CMD_VSYNC GITS_CMD_GICv4(GITS_CMD_SYNC) +/* VMOVP, VSGI and INVDB are the odd ones, as they dont have a physical counterpart */ +#define GITS_CMD_VMOVP GITS_CMD_GICv4(2) +#define GITS_CMD_VSGI GITS_CMD_GICv4(3) +#define GITS_CMD_INVDB GITS_CMD_GICv4(0xe) + +/* + * ITS error numbers + */ +#define E_ITS_MOVI_UNMAPPED_INTERRUPT 0x010107 +#define E_ITS_MOVI_UNMAPPED_COLLECTION 0x010109 +#define E_ITS_INT_UNMAPPED_INTERRUPT 0x010307 +#define E_ITS_CLEAR_UNMAPPED_INTERRUPT 0x010507 +#define E_ITS_MAPD_DEVICE_OOR 0x010801 +#define E_ITS_MAPD_ITTSIZE_OOR 0x010802 +#define E_ITS_MAPC_PROCNUM_OOR 0x010902 +#define E_ITS_MAPC_COLLECTION_OOR 0x010903 +#define E_ITS_MAPTI_UNMAPPED_DEVICE 0x010a04 +#define E_ITS_MAPTI_ID_OOR 0x010a05 +#define E_ITS_MAPTI_PHYSICALID_OOR 0x010a06 +#define E_ITS_INV_UNMAPPED_INTERRUPT 0x010c07 +#define E_ITS_INVALL_UNMAPPED_COLLECTION 0x010d09 +#define E_ITS_MOVALL_PROCNUM_OOR 0x010e01 +#define E_ITS_DISCARD_UNMAPPED_INTERRUPT 0x010f07 + +/* + * CPU interface registers + */ +#define ICC_CTLR_EL1_EOImode_SHIFT (1) +#define ICC_CTLR_EL1_EOImode_drop_dir (0U << ICC_CTLR_EL1_EOImode_SHIFT) +#define ICC_CTLR_EL1_EOImode_drop (1U << ICC_CTLR_EL1_EOImode_SHIFT) +#define ICC_CTLR_EL1_EOImode_MASK (1 << ICC_CTLR_EL1_EOImode_SHIFT) +#define ICC_CTLR_EL1_CBPR_SHIFT 0 +#define ICC_CTLR_EL1_CBPR_MASK (1 << ICC_CTLR_EL1_CBPR_SHIFT) +#define ICC_CTLR_EL1_PMHE_SHIFT 6 +#define ICC_CTLR_EL1_PMHE_MASK (1 << ICC_CTLR_EL1_PMHE_SHIFT) +#define ICC_CTLR_EL1_PRI_BITS_SHIFT 8 +#define ICC_CTLR_EL1_PRI_BITS_MASK (0x7 << ICC_CTLR_EL1_PRI_BITS_SHIFT) +#define ICC_CTLR_EL1_ID_BITS_SHIFT 11 +#define ICC_CTLR_EL1_ID_BITS_MASK (0x7 << ICC_CTLR_EL1_ID_BITS_SHIFT) +#define ICC_CTLR_EL1_SEIS_SHIFT 14 +#define ICC_CTLR_EL1_SEIS_MASK (0x1 << ICC_CTLR_EL1_SEIS_SHIFT) +#define ICC_CTLR_EL1_A3V_SHIFT 15 +#define ICC_CTLR_EL1_A3V_MASK (0x1 << ICC_CTLR_EL1_A3V_SHIFT) +#define ICC_CTLR_EL1_RSS (0x1 << 18) +#define ICC_CTLR_EL1_ExtRange (0x1 << 19) +#define ICC_PMR_EL1_SHIFT 0 +#define ICC_PMR_EL1_MASK (0xff << ICC_PMR_EL1_SHIFT) +#define ICC_BPR0_EL1_SHIFT 0 +#define ICC_BPR0_EL1_MASK (0x7 << ICC_BPR0_EL1_SHIFT) +#define ICC_BPR1_EL1_SHIFT 0 +#define ICC_BPR1_EL1_MASK (0x7 << ICC_BPR1_EL1_SHIFT) +#define ICC_IGRPEN0_EL1_SHIFT 0 +#define ICC_IGRPEN0_EL1_MASK (1 << ICC_IGRPEN0_EL1_SHIFT) +#define ICC_IGRPEN1_EL1_SHIFT 0 +#define ICC_IGRPEN1_EL1_MASK (1 << ICC_IGRPEN1_EL1_SHIFT) +#define ICC_SRE_EL1_DIB (1U << 2) +#define ICC_SRE_EL1_DFB (1U << 1) #define ICC_SRE_EL1_SRE (1U << 0) -#define ICC_IGRPEN1_EL1_ENABLE (1U << 0) +/* These are for GICv2 emulation only */ +#define GICH_LR_VIRTUALID (0x3ffUL << 0) +#define GICH_LR_PHYSID_CPUID_SHIFT (10) +#define GICH_LR_PHYSID_CPUID (7UL << GICH_LR_PHYSID_CPUID_SHIFT) + +#define ICC_IAR1_EL1_SPURIOUS 0x3ff + +#define ICC_SRE_EL2_SRE (1 << 0) +#define ICC_SRE_EL2_ENABLE (1 << 3) -#define GICV3_MAX_CPUS 512 +#define ICC_SGI1R_TARGET_LIST_SHIFT 0 +#define ICC_SGI1R_TARGET_LIST_MASK (0xffff << ICC_SGI1R_TARGET_LIST_SHIFT) +#define ICC_SGI1R_AFFINITY_1_SHIFT 16 +#define ICC_SGI1R_AFFINITY_1_MASK (0xff << ICC_SGI1R_AFFINITY_1_SHIFT) +#define ICC_SGI1R_SGI_ID_SHIFT 24 +#define ICC_SGI1R_SGI_ID_MASK (0xfULL << ICC_SGI1R_SGI_ID_SHIFT) +#define ICC_SGI1R_AFFINITY_2_SHIFT 32 +#define ICC_SGI1R_AFFINITY_2_MASK (0xffULL << ICC_SGI1R_AFFINITY_2_SHIFT) +#define ICC_SGI1R_IRQ_ROUTING_MODE_BIT 40 +#define ICC_SGI1R_RS_SHIFT 44 +#define ICC_SGI1R_RS_MASK (0xfULL << ICC_SGI1R_RS_SHIFT) +#define ICC_SGI1R_AFFINITY_3_SHIFT 48 +#define ICC_SGI1R_AFFINITY_3_MASK (0xffULL << ICC_SGI1R_AFFINITY_3_SHIFT) -#endif /* SELFTEST_KVM_GICV3_H */ +#endif diff --git a/tools/testing/selftests/kvm/lib/aarch64/gic_v3.c b/tools/testing/selftests/kvm/lib/aarch64/gic_v3.c index 263bf3ed8fd5..cd8f0e209599 100644 --- a/tools/testing/selftests/kvm/lib/aarch64/gic_v3.c +++ b/tools/testing/selftests/kvm/lib/aarch64/gic_v3.c @@ -9,9 +9,20 @@ #include "processor.h" #include "delay.h" +#include "gic.h" #include "gic_v3.h" #include "gic_private.h" +#define GICV3_MAX_CPUS 512 + +#define GICD_INT_DEF_PRI 0xa0 +#define GICD_INT_DEF_PRI_X4 ((GICD_INT_DEF_PRI << 24) |\ + (GICD_INT_DEF_PRI << 16) |\ + (GICD_INT_DEF_PRI << 8) |\ + GICD_INT_DEF_PRI) + +#define ICC_PMR_DEF_PRIO 0xf0 + struct gicv3_data { void *dist_base; void *redist_base[GICV3_MAX_CPUS]; @@ -320,7 +331,7 @@ static void gicv3_cpu_init(unsigned int cpu, void *redist_base) write_sysreg_s(ICC_PMR_DEF_PRIO, SYS_ICC_PMR_EL1); /* Enable non-secure Group-1 interrupts */ - write_sysreg_s(ICC_IGRPEN1_EL1_ENABLE, SYS_ICC_GRPEN1_EL1); + write_sysreg_s(ICC_IGRPEN1_EL1_MASK, SYS_ICC_IGRPEN1_EL1); gicv3_data.redist_base[cpu] = redist_base_cpu; } From patchwork Tue Feb 13 09:40:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 200301 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp427856dyb; Tue, 13 Feb 2024 01:45:55 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCUCoIwyIp08ZF9YaE67PQtRMXFyKwEoZ2F9u2sGYvPZAm7259zrEW2ge0AzokNig2ZbfVIid8u4b3Vffz9/E/7DkVE8Hw== X-Google-Smtp-Source: AGHT+IF8cz4gap9j3N0WOJnYoTRZ+SOLF8zam8mVsNb1R0PWgafa0M/movaEU7cb7QS4WQuXxhe/ X-Received: by 2002:a17:906:af94:b0:a2d:a6a7:b3bc with SMTP id mj20-20020a170906af9400b00a2da6a7b3bcmr6831071ejb.4.1707817555437; Tue, 13 Feb 2024 01:45:55 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707817555; cv=pass; d=google.com; s=arc-20160816; b=zBUST9kloz1W1ew4RpJsjimYCNUIc8hm7k5PMteUF8VuKdh9U52oPE4BpvR1Onz31v waaz5mnsG0k3N72+j80D1tyQ192PGQb7qAmdh1eepnjTu7x/sCBXvAPYx4EdTLBsIUHc u9jGmuZfq1giIy2GHRBhInwFix+shFyF5YhZDg9ZQC/Odw1dUtDJoNWeU1XGLq3hI1TP PJnvBdwNWb4z9+4G+oCtYeO8JWPPbBP2fZNjcB78jPl6l/97eGN5PDoaqxTmDv4ZT4TD p/QqDVs/eG/7rM+ZyxtWSSzMJ8EmRE8LmfScgHdUd2ZokSTcVyuUogsim+MaRgqdlW4D iszw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=VKRKz9PHINTeNQtF9BS6pYUMzjC2ZlxpnobRHX3hMdw=; fh=Pur+uBJSjpcHXDRIxONqORSOainmJ51v7OA3j4HDa9c=; b=xuGmjkg4L1W89Q/5Gyk/6KytEGGmQmxlXlYauSo14Zlp9Z+ptPptbvY7mL6r2ppi3y 6izjnb4x9Yj71VG3LFZU/3sVip8I4dj2YNNPEjg/ZMEgipbV6jxXfrkzsJaZXArOHnWp PqCVbn2/SQJ/JuENE/4UwtYEbU6uOSCvI8BMMoi7INQHuLQfvv8Sa0IgDaiOrvidb6Eh W+FfwMBtLl4ZDQJHsiGmNyYI0T9tSKFontaAcfKd6FtPUD9OPIVPO9ZIvesN5qS/K4Yv cTzCtrqSxpuOuYZNFtQZoNKwp9rxtkOEj1sIGUFiulBsIOzQ4m/hR2jCJiDqbKHuiIU2 Gg3Q==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=h3OoyDDi; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63242-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63242-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev X-Forwarded-Encrypted: i=2; AJvYcCWtbIqnSOTRdJDKTZc09IHoPcA3uBXyjyR6MTSQjKxpD/ZwRFZ/UVLgvhdv9/N053saQNaOkSVrfb/amAJHYxQNOOl2Gw== Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id gz16-20020a170906f2d000b00a37da6faeeesi1078634ejb.334.2024.02.13.01.45.55 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 01:45:55 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-63242-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=h3OoyDDi; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63242-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63242-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id D54E61F27FD7 for ; Tue, 13 Feb 2024 09:45:54 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 3F99D364C2; Tue, 13 Feb 2024 09:41:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="h3OoyDDi" Received: from out-179.mta0.migadu.com (out-179.mta0.migadu.com [91.218.175.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1154C2BB1A; Tue, 13 Feb 2024 09:41:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707817268; cv=none; b=J9qxUOSUJUxAcVy1c397NhiDXPrWlYodIO4xwwKVvg2INX2N/Dc8xckm261T/N7rs8zSOIEIw7ar+PrBiujWh6anfRXVwNi6Ay/v1o9cVjK2wQKbY5lrCrhrlZgRtpxzbO8pCfXKFXx2GvbAPs41Gt65ETCeZ8Aj0j4IIB3dHtI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707817268; c=relaxed/simple; bh=8TVb02k9Dj6zPDEtr5f/OpqyZHmFQzODOugGavUF2U8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=DObUyRcYPzhsNLgea2YlB67Wek6y8B6l5q+Au3UOHZCX5SCHVFyn/bJDWmBLOHm4qULvUq2z2uxM8ACZCgDfhpigSwxG7DItx6wVvD3VoaWclGNaY0vmsNTASKwDPaSXZIwLOp+6xmCZzXnK1jgTYsehkfH9cfwh+Fkh/2Z4f04= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=h3OoyDDi; arc=none smtp.client-ip=91.218.175.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1707817264; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VKRKz9PHINTeNQtF9BS6pYUMzjC2ZlxpnobRHX3hMdw=; b=h3OoyDDiD6QEuRgNK/4ALlVm7AbyYwtw6xXgvcY1h84XkSvJ5ttORg/x19CfKC82uRxpGU iWAlw7qAcbX0J/HTDI8c5ZaPiMmAt/ok7dgNqvsGUu8ut6tXzNGTqQaTjU41PQWohBHrBd 8SRGANdavhihMcSEYBzhDdvhbsvSkEI= From: Oliver Upton To: kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , linux-kernel@vger.kernel.org, Oliver Upton , Colton Lewis Subject: [PATCH v2 18/23] KVM: selftests: Standardise layout of GIC frames Date: Tue, 13 Feb 2024 09:40:54 +0000 Message-ID: <20240213094054.3961627-1-oliver.upton@linux.dev> In-Reply-To: <20240213093250.3960069-1-oliver.upton@linux.dev> References: <20240213093250.3960069-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790776500979663627 X-GMAIL-MSGID: 1790776500979663627 It would appear that all of the selftests are using the same exact layout for the GIC frames. Fold this back into the library implementation to avoid defining magic values all over the selftests. This is an extension of Colton's change, ripping out parameterization of from the library internals in addition to the public interfaces. Co-developed-by: Colton Lewis Signed-off-by: Colton Lewis Signed-off-by: Oliver Upton --- .../selftests/kvm/aarch64/arch_timer.c | 8 +-- .../testing/selftests/kvm/aarch64/vgic_irq.c | 11 +--- .../kvm/aarch64/vpmu_counter_access.c | 6 +- .../selftests/kvm/dirty_log_perf_test.c | 5 +- .../selftests/kvm/include/aarch64/gic.h | 10 +++- .../selftests/kvm/include/aarch64/vgic.h | 3 +- tools/testing/selftests/kvm/lib/aarch64/gic.c | 18 +++--- .../selftests/kvm/lib/aarch64/gic_private.h | 4 +- .../selftests/kvm/lib/aarch64/gic_v3.c | 56 +++++++++---------- .../testing/selftests/kvm/lib/aarch64/vgic.c | 18 +++--- 10 files changed, 57 insertions(+), 82 deletions(-) diff --git a/tools/testing/selftests/kvm/aarch64/arch_timer.c b/tools/testing/selftests/kvm/aarch64/arch_timer.c index 274b8465b42a..f5101898c46a 100644 --- a/tools/testing/selftests/kvm/aarch64/arch_timer.c +++ b/tools/testing/selftests/kvm/aarch64/arch_timer.c @@ -59,9 +59,6 @@ static struct test_args test_args = { #define msecs_to_usecs(msec) ((msec) * 1000LL) -#define GICD_BASE_GPA 0x8000000ULL -#define GICR_BASE_GPA 0x80A0000ULL - enum guest_stage { GUEST_STAGE_VTIMER_CVAL = 1, GUEST_STAGE_VTIMER_TVAL, @@ -204,8 +201,7 @@ static void guest_code(void) local_irq_disable(); - gic_init(GIC_V3, test_args.nr_vcpus, - (void *)GICD_BASE_GPA, (void *)GICR_BASE_GPA); + gic_init(GIC_V3, test_args.nr_vcpus); timer_set_ctl(VIRTUAL, CTL_IMASK); timer_set_ctl(PHYSICAL, CTL_IMASK); @@ -391,7 +387,7 @@ static struct kvm_vm *test_vm_create(void) vcpu_init_descriptor_tables(vcpus[i]); test_init_timer_irq(vm); - gic_fd = vgic_v3_setup(vm, nr_vcpus, 64, GICD_BASE_GPA, GICR_BASE_GPA); + gic_fd = vgic_v3_setup(vm, nr_vcpus, 64); __TEST_REQUIRE(gic_fd >= 0, "Failed to create vgic-v3"); /* Make all the test's cmdline args visible to the guest */ diff --git a/tools/testing/selftests/kvm/aarch64/vgic_irq.c b/tools/testing/selftests/kvm/aarch64/vgic_irq.c index d61a6302f467..a51dbd2a5f84 100644 --- a/tools/testing/selftests/kvm/aarch64/vgic_irq.c +++ b/tools/testing/selftests/kvm/aarch64/vgic_irq.c @@ -19,9 +19,6 @@ #include "gic_v3.h" #include "vgic.h" -#define GICD_BASE_GPA 0x08000000ULL -#define GICR_BASE_GPA 0x080A0000ULL - /* * Stores the user specified args; it's passed to the guest and to every test * function. @@ -49,9 +46,6 @@ struct test_args { #define IRQ_DEFAULT_PRIO (LOWEST_PRIO - 1) #define IRQ_DEFAULT_PRIO_REG (IRQ_DEFAULT_PRIO << KVM_PRIO_SHIFT) /* 0xf0 */ -static void *dist = (void *)GICD_BASE_GPA; -static void *redist = (void *)GICR_BASE_GPA; - /* * The kvm_inject_* utilities are used by the guest to ask the host to inject * interrupts (e.g., using the KVM_IRQ_LINE ioctl). @@ -478,7 +472,7 @@ static void guest_code(struct test_args *args) bool level_sensitive = args->level_sensitive; struct kvm_inject_desc *f, *inject_fns; - gic_init(GIC_V3, 1, dist, redist); + gic_init(GIC_V3, 1); for (i = 0; i < nr_irqs; i++) gic_irq_enable(i); @@ -764,8 +758,7 @@ static void test_vgic(uint32_t nr_irqs, bool level_sensitive, bool eoi_split) memcpy(addr_gva2hva(vm, args_gva), &args, sizeof(args)); vcpu_args_set(vcpu, 1, args_gva); - gic_fd = vgic_v3_setup(vm, 1, nr_irqs, - GICD_BASE_GPA, GICR_BASE_GPA); + gic_fd = vgic_v3_setup(vm, 1, nr_irqs); __TEST_REQUIRE(gic_fd >= 0, "Failed to create vgic-v3, skipping"); vm_install_exception_handler(vm, VECTOR_IRQ_CURRENT, diff --git a/tools/testing/selftests/kvm/aarch64/vpmu_counter_access.c b/tools/testing/selftests/kvm/aarch64/vpmu_counter_access.c index 9d51b5691349..496a7a3c615b 100644 --- a/tools/testing/selftests/kvm/aarch64/vpmu_counter_access.c +++ b/tools/testing/selftests/kvm/aarch64/vpmu_counter_access.c @@ -420,9 +420,6 @@ static void guest_code(uint64_t expected_pmcr_n) GUEST_DONE(); } -#define GICD_BASE_GPA 0x8000000ULL -#define GICR_BASE_GPA 0x80A0000ULL - /* Create a VM that has one vCPU with PMUv3 configured. */ static void create_vpmu_vm(void *guest_code) { @@ -454,8 +451,7 @@ static void create_vpmu_vm(void *guest_code) init.features[0] |= (1 << KVM_ARM_VCPU_PMU_V3); vpmu_vm.vcpu = aarch64_vcpu_add(vpmu_vm.vm, 0, &init, guest_code); vcpu_init_descriptor_tables(vpmu_vm.vcpu); - vpmu_vm.gic_fd = vgic_v3_setup(vpmu_vm.vm, 1, 64, - GICD_BASE_GPA, GICR_BASE_GPA); + vpmu_vm.gic_fd = vgic_v3_setup(vpmu_vm.vm, 1, 64); __TEST_REQUIRE(vpmu_vm.gic_fd >= 0, "Failed to create vgic-v3, skipping"); diff --git a/tools/testing/selftests/kvm/dirty_log_perf_test.c b/tools/testing/selftests/kvm/dirty_log_perf_test.c index d374dbcf9a53..53c9d7a2611d 100644 --- a/tools/testing/selftests/kvm/dirty_log_perf_test.c +++ b/tools/testing/selftests/kvm/dirty_log_perf_test.c @@ -22,9 +22,6 @@ #ifdef __aarch64__ #include "aarch64/vgic.h" -#define GICD_BASE_GPA 0x8000000ULL -#define GICR_BASE_GPA 0x80A0000ULL - static int gic_fd; static void arch_setup_vm(struct kvm_vm *vm, unsigned int nr_vcpus) @@ -33,7 +30,7 @@ static void arch_setup_vm(struct kvm_vm *vm, unsigned int nr_vcpus) * The test can still run even if hardware does not support GICv3, as it * is only an optimization to reduce guest exits. */ - gic_fd = vgic_v3_setup(vm, nr_vcpus, 64, GICD_BASE_GPA, GICR_BASE_GPA); + gic_fd = vgic_v3_setup(vm, nr_vcpus, 64); } static void arch_cleanup_vm(struct kvm_vm *vm) diff --git a/tools/testing/selftests/kvm/include/aarch64/gic.h b/tools/testing/selftests/kvm/include/aarch64/gic.h index b217ea17cac5..16d944486e9c 100644 --- a/tools/testing/selftests/kvm/include/aarch64/gic.h +++ b/tools/testing/selftests/kvm/include/aarch64/gic.h @@ -11,6 +11,13 @@ enum gic_type { GIC_TYPE_MAX, }; +#define GICD_BASE_GPA 0x8000000ULL +#define GICR_BASE_GPA (GICD_BASE_GPA + SZ_64K) + +/* The GIC is identity-mapped into the guest at the time of setup. */ +#define GICD_BASE_GVA ((void *)GICD_BASE_GPA) +#define GICR_BASE_GVA ((void *)GICR_BASE_GPA) + #define MIN_SGI 0 #define MIN_PPI 16 #define MIN_SPI 32 @@ -21,8 +28,7 @@ enum gic_type { #define INTID_IS_PPI(intid) (MIN_PPI <= (intid) && (intid) < MIN_SPI) #define INTID_IS_SPI(intid) (MIN_SPI <= (intid) && (intid) <= MAX_SPI) -void gic_init(enum gic_type type, unsigned int nr_cpus, - void *dist_base, void *redist_base); +void gic_init(enum gic_type type, unsigned int nr_cpus); void gic_irq_enable(unsigned int intid); void gic_irq_disable(unsigned int intid); unsigned int gic_get_and_ack_irq(void); diff --git a/tools/testing/selftests/kvm/include/aarch64/vgic.h b/tools/testing/selftests/kvm/include/aarch64/vgic.h index 0ac6f05c63f9..ce19aa0a8360 100644 --- a/tools/testing/selftests/kvm/include/aarch64/vgic.h +++ b/tools/testing/selftests/kvm/include/aarch64/vgic.h @@ -16,8 +16,7 @@ ((uint64_t)(flags) << 12) | \ index) -int vgic_v3_setup(struct kvm_vm *vm, unsigned int nr_vcpus, uint32_t nr_irqs, - uint64_t gicd_base_gpa, uint64_t gicr_base_gpa); +int vgic_v3_setup(struct kvm_vm *vm, unsigned int nr_vcpus, uint32_t nr_irqs); #define VGIC_MAX_RESERVED 1023 diff --git a/tools/testing/selftests/kvm/lib/aarch64/gic.c b/tools/testing/selftests/kvm/lib/aarch64/gic.c index 55668631d546..7abbf8866512 100644 --- a/tools/testing/selftests/kvm/lib/aarch64/gic.c +++ b/tools/testing/selftests/kvm/lib/aarch64/gic.c @@ -17,13 +17,12 @@ static const struct gic_common_ops *gic_common_ops; static struct spinlock gic_lock; -static void gic_cpu_init(unsigned int cpu, void *redist_base) +static void gic_cpu_init(unsigned int cpu) { - gic_common_ops->gic_cpu_init(cpu, redist_base); + gic_common_ops->gic_cpu_init(cpu); } -static void -gic_dist_init(enum gic_type type, unsigned int nr_cpus, void *dist_base) +static void gic_dist_init(enum gic_type type, unsigned int nr_cpus) { const struct gic_common_ops *gic_ops = NULL; @@ -40,7 +39,7 @@ gic_dist_init(enum gic_type type, unsigned int nr_cpus, void *dist_base) GUEST_ASSERT(gic_ops); - gic_ops->gic_init(nr_cpus, dist_base); + gic_ops->gic_init(nr_cpus); gic_common_ops = gic_ops; /* Make sure that the initialized data is visible to all the vCPUs */ @@ -49,18 +48,15 @@ gic_dist_init(enum gic_type type, unsigned int nr_cpus, void *dist_base) spin_unlock(&gic_lock); } -void gic_init(enum gic_type type, unsigned int nr_cpus, - void *dist_base, void *redist_base) +void gic_init(enum gic_type type, unsigned int nr_cpus) { uint32_t cpu = guest_get_vcpuid(); GUEST_ASSERT(type < GIC_TYPE_MAX); - GUEST_ASSERT(dist_base); - GUEST_ASSERT(redist_base); GUEST_ASSERT(nr_cpus); - gic_dist_init(type, nr_cpus, dist_base); - gic_cpu_init(cpu, redist_base); + gic_dist_init(type, nr_cpus); + gic_cpu_init(cpu); } void gic_irq_enable(unsigned int intid) diff --git a/tools/testing/selftests/kvm/lib/aarch64/gic_private.h b/tools/testing/selftests/kvm/lib/aarch64/gic_private.h index 75d07313c893..d24e9ecc96c6 100644 --- a/tools/testing/selftests/kvm/lib/aarch64/gic_private.h +++ b/tools/testing/selftests/kvm/lib/aarch64/gic_private.h @@ -8,8 +8,8 @@ #define SELFTEST_KVM_GIC_PRIVATE_H struct gic_common_ops { - void (*gic_init)(unsigned int nr_cpus, void *dist_base); - void (*gic_cpu_init)(unsigned int cpu, void *redist_base); + void (*gic_init)(unsigned int nr_cpus); + void (*gic_cpu_init)(unsigned int cpu); void (*gic_irq_enable)(unsigned int intid); void (*gic_irq_disable)(unsigned int intid); uint64_t (*gic_read_iar)(void); diff --git a/tools/testing/selftests/kvm/lib/aarch64/gic_v3.c b/tools/testing/selftests/kvm/lib/aarch64/gic_v3.c index cd8f0e209599..de53d193d0be 100644 --- a/tools/testing/selftests/kvm/lib/aarch64/gic_v3.c +++ b/tools/testing/selftests/kvm/lib/aarch64/gic_v3.c @@ -24,8 +24,6 @@ #define ICC_PMR_DEF_PRIO 0xf0 struct gicv3_data { - void *dist_base; - void *redist_base[GICV3_MAX_CPUS]; unsigned int nr_cpus; unsigned int nr_spis; }; @@ -46,17 +44,23 @@ static void gicv3_gicd_wait_for_rwp(void) { unsigned int count = 100000; /* 1s */ - while (readl(gicv3_data.dist_base + GICD_CTLR) & GICD_CTLR_RWP) { + while (readl(GICD_BASE_GVA + GICD_CTLR) & GICD_CTLR_RWP) { GUEST_ASSERT(count--); udelay(10); } } -static void gicv3_gicr_wait_for_rwp(void *redist_base) +static inline void *gicr_base_cpu(uint32_t cpu) +{ + /* Align all the redistributors sequentially */ + return GICR_BASE_GVA + cpu * SZ_64K * 2; +} + +static void gicv3_gicr_wait_for_rwp(uint32_t cpu) { unsigned int count = 100000; /* 1s */ - while (readl(redist_base + GICR_CTLR) & GICR_CTLR_RWP) { + while (readl(gicr_base_cpu(cpu) + GICR_CTLR) & GICR_CTLR_RWP) { GUEST_ASSERT(count--); udelay(10); } @@ -67,7 +71,7 @@ static void gicv3_wait_for_rwp(uint32_t cpu_or_dist) if (cpu_or_dist & DIST_BIT) gicv3_gicd_wait_for_rwp(); else - gicv3_gicr_wait_for_rwp(gicv3_data.redist_base[cpu_or_dist]); + gicv3_gicr_wait_for_rwp(cpu_or_dist); } static enum gicv3_intid_range get_intid_range(unsigned int intid) @@ -127,15 +131,15 @@ static void gicv3_set_eoi_split(bool split) uint32_t gicv3_reg_readl(uint32_t cpu_or_dist, uint64_t offset) { - void *base = cpu_or_dist & DIST_BIT ? gicv3_data.dist_base - : sgi_base_from_redist(gicv3_data.redist_base[cpu_or_dist]); + void *base = cpu_or_dist & DIST_BIT ? GICD_BASE_GVA + : sgi_base_from_redist(gicr_base_cpu(cpu_or_dist)); return readl(base + offset); } void gicv3_reg_writel(uint32_t cpu_or_dist, uint64_t offset, uint32_t reg_val) { - void *base = cpu_or_dist & DIST_BIT ? gicv3_data.dist_base - : sgi_base_from_redist(gicv3_data.redist_base[cpu_or_dist]); + void *base = cpu_or_dist & DIST_BIT ? GICD_BASE_GVA + : sgi_base_from_redist(gicr_base_cpu(cpu_or_dist)); writel(reg_val, base + offset); } @@ -289,13 +293,7 @@ static void gicv3_enable_redist(void *redist_base) } } -static inline void *gicr_base_cpu(void *redist_base, uint32_t cpu) -{ - /* Align all the redistributors sequentially */ - return redist_base + cpu * SZ_64K * 2; -} - -static void gicv3_cpu_init(unsigned int cpu, void *redist_base) +static void gicv3_cpu_init(unsigned int cpu) { void *sgi_base; unsigned int i; @@ -303,7 +301,7 @@ static void gicv3_cpu_init(unsigned int cpu, void *redist_base) GUEST_ASSERT(cpu < gicv3_data.nr_cpus); - redist_base_cpu = gicr_base_cpu(redist_base, cpu); + redist_base_cpu = gicr_base_cpu(cpu); sgi_base = sgi_base_from_redist(redist_base_cpu); gicv3_enable_redist(redist_base_cpu); @@ -321,7 +319,7 @@ static void gicv3_cpu_init(unsigned int cpu, void *redist_base) writel(GICD_INT_DEF_PRI_X4, sgi_base + GICR_IPRIORITYR0 + i); - gicv3_gicr_wait_for_rwp(redist_base_cpu); + gicv3_gicr_wait_for_rwp(cpu); /* Enable the GIC system register (ICC_*) access */ write_sysreg_s(read_sysreg_s(SYS_ICC_SRE_EL1) | ICC_SRE_EL1_SRE, @@ -332,17 +330,14 @@ static void gicv3_cpu_init(unsigned int cpu, void *redist_base) /* Enable non-secure Group-1 interrupts */ write_sysreg_s(ICC_IGRPEN1_EL1_MASK, SYS_ICC_IGRPEN1_EL1); - - gicv3_data.redist_base[cpu] = redist_base_cpu; } static void gicv3_dist_init(void) { - void *dist_base = gicv3_data.dist_base; unsigned int i; /* Disable the distributor until we set things up */ - writel(0, dist_base + GICD_CTLR); + writel(0, GICD_BASE_GVA + GICD_CTLR); gicv3_gicd_wait_for_rwp(); /* @@ -350,33 +345,32 @@ static void gicv3_dist_init(void) * Also, deactivate and disable them. */ for (i = 32; i < gicv3_data.nr_spis; i += 32) { - writel(~0, dist_base + GICD_IGROUPR + i / 8); - writel(~0, dist_base + GICD_ICACTIVER + i / 8); - writel(~0, dist_base + GICD_ICENABLER + i / 8); + writel(~0, GICD_BASE_GVA + GICD_IGROUPR + i / 8); + writel(~0, GICD_BASE_GVA + GICD_ICACTIVER + i / 8); + writel(~0, GICD_BASE_GVA + GICD_ICENABLER + i / 8); } /* Set a default priority for all the SPIs */ for (i = 32; i < gicv3_data.nr_spis; i += 4) writel(GICD_INT_DEF_PRI_X4, - dist_base + GICD_IPRIORITYR + i); + GICD_BASE_GVA + GICD_IPRIORITYR + i); /* Wait for the settings to sync-in */ gicv3_gicd_wait_for_rwp(); /* Finally, enable the distributor globally with ARE */ writel(GICD_CTLR_ARE_NS | GICD_CTLR_ENABLE_G1A | - GICD_CTLR_ENABLE_G1, dist_base + GICD_CTLR); + GICD_CTLR_ENABLE_G1, GICD_BASE_GVA + GICD_CTLR); gicv3_gicd_wait_for_rwp(); } -static void gicv3_init(unsigned int nr_cpus, void *dist_base) +static void gicv3_init(unsigned int nr_cpus) { GUEST_ASSERT(nr_cpus <= GICV3_MAX_CPUS); gicv3_data.nr_cpus = nr_cpus; - gicv3_data.dist_base = dist_base; gicv3_data.nr_spis = GICD_TYPER_SPIS( - readl(gicv3_data.dist_base + GICD_TYPER)); + readl(GICD_BASE_GVA + GICD_TYPER)); if (gicv3_data.nr_spis > 1020) gicv3_data.nr_spis = 1020; diff --git a/tools/testing/selftests/kvm/lib/aarch64/vgic.c b/tools/testing/selftests/kvm/lib/aarch64/vgic.c index b5f28d21a947..ac55b6c2e915 100644 --- a/tools/testing/selftests/kvm/lib/aarch64/vgic.c +++ b/tools/testing/selftests/kvm/lib/aarch64/vgic.c @@ -19,8 +19,6 @@ * Input args: * vm - KVM VM * nr_vcpus - Number of vCPUs supported by this VM - * gicd_base_gpa - Guest Physical Address of the Distributor region - * gicr_base_gpa - Guest Physical Address of the Redistributor region * * Output args: None * @@ -30,11 +28,10 @@ * redistributor regions of the guest. Since it depends on the number of * vCPUs for the VM, it must be called after all the vCPUs have been created. */ -int vgic_v3_setup(struct kvm_vm *vm, unsigned int nr_vcpus, uint32_t nr_irqs, - uint64_t gicd_base_gpa, uint64_t gicr_base_gpa) +int vgic_v3_setup(struct kvm_vm *vm, unsigned int nr_vcpus, uint32_t nr_irqs) { int gic_fd; - uint64_t redist_attr; + uint64_t attr; struct list_head *iter; unsigned int nr_gic_pages, nr_vcpus_created = 0; @@ -60,18 +57,19 @@ int vgic_v3_setup(struct kvm_vm *vm, unsigned int nr_vcpus, uint32_t nr_irqs, kvm_device_attr_set(gic_fd, KVM_DEV_ARM_VGIC_GRP_CTRL, KVM_DEV_ARM_VGIC_CTRL_INIT, NULL); + attr = GICD_BASE_GPA; kvm_device_attr_set(gic_fd, KVM_DEV_ARM_VGIC_GRP_ADDR, - KVM_VGIC_V3_ADDR_TYPE_DIST, &gicd_base_gpa); + KVM_VGIC_V3_ADDR_TYPE_DIST, &attr); nr_gic_pages = vm_calc_num_guest_pages(vm->mode, KVM_VGIC_V3_DIST_SIZE); - virt_map(vm, gicd_base_gpa, gicd_base_gpa, nr_gic_pages); + virt_map(vm, GICD_BASE_GPA, GICD_BASE_GPA, nr_gic_pages); /* Redistributor setup */ - redist_attr = REDIST_REGION_ATTR_ADDR(nr_vcpus, gicr_base_gpa, 0, 0); + attr = REDIST_REGION_ATTR_ADDR(nr_vcpus, GICR_BASE_GPA, 0, 0); kvm_device_attr_set(gic_fd, KVM_DEV_ARM_VGIC_GRP_ADDR, - KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION, &redist_attr); + KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION, &attr); nr_gic_pages = vm_calc_num_guest_pages(vm->mode, KVM_VGIC_V3_REDIST_SIZE * nr_vcpus); - virt_map(vm, gicr_base_gpa, gicr_base_gpa, nr_gic_pages); + virt_map(vm, GICR_BASE_GPA, GICR_BASE_GPA, nr_gic_pages); kvm_device_attr_set(gic_fd, KVM_DEV_ARM_VGIC_GRP_CTRL, KVM_DEV_ARM_VGIC_CTRL_INIT, NULL); From patchwork Tue Feb 13 09:41:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 200319 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp436416dyb; Tue, 13 Feb 2024 02:05:58 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCUeJW45gYpEulPo9hGWsJChIqixyIle5IH0aqbSY7DsPsh/XfjQ00o2v1kpB7XVOOXgyio3wBzTnps1I510dJ7g5/+5Hw== X-Google-Smtp-Source: AGHT+IE1Ae6nfmRBwajH9js9FPYZntztNWdcstmn2F3nF+rGF1QexfB8Kng91W9AxDnJusdlmcXx X-Received: by 2002:a17:90a:b94b:b0:296:e219:b338 with SMTP id f11-20020a17090ab94b00b00296e219b338mr6407045pjw.46.1707818758507; Tue, 13 Feb 2024 02:05:58 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707818758; cv=pass; d=google.com; s=arc-20160816; b=bd0G4pv5+BjtO6bY0URE85gMtYyvhK5UuOqDNg8DSqOoY1kVltuVwytplSwmtxQpIQ uLD6LsTN6t4ruoKHT4UOaGZfw0I0gjS2qMa3uSD0y/qx4Xd1bzI0IUaK1Zyl5kzxncAw psGx7/8pFnbJ9XuBCR1ELud2WwPz5Dc9OPGbP7ez+zEdx+/e7eSMhoshHLa0zMxOtqfh sBOBIC8oBjA7am15xCVA2PLeYktwVTmwYailS6PelymtbYWgnaWQGJhuBvIaI5sBgylw Dg7CL2GeCh4ECixVasSt/FYfNHZtvW8e0bUlOgJN8IDXP0gD/Kuz4P4EXsUbDvw9NLVk +DwA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=OMPdGXoxf8WnGOh2+4/hLw9TzTmbExY6hN0KedYmhJk=; fh=Tosgz2huRsuVEMSCYI82QGqIz33zcSD+sCRK4jo0yRM=; b=wpzLL+ch/ntHThIymidutQxk1Wfp8w5i96hrVu/UaN2P4vl9j8ncZsec+7uoyfETRk QxIIEktFzTYw4LgdTsWvk0l4yLN7U5DitYpemf/xVNVWToXL4aOfEEikVvzJQx7rX4Kk f/IhOeNr4AkXwzIr0Mk2fthgTDJ2o3KL4W855ZwHycDq3F3YeVDL+N/GOYRCkXife83P cMRgVs4DRuGW4ysrPicbBgM0qx7CyJ6Z5VYCgVbn+gvN8GOZI0KgPEl1Cn2AMp6u0aU+ UtRt9YR4sI8ws1Ala4qTtuNaPd+Rg7zszILM8unLv6C7BRxytQd3GQdJA/255sa2O3ln wYYQ==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=CRwbn6KQ; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63243-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63243-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev X-Forwarded-Encrypted: i=2; AJvYcCUJ04jT8j9KfGW1CBdpEZYvvG8nbi+7e7DFo651Q6GJkI4oRWeGFVVBmbKtI677terp5LQSvoLyK+K+wfi6NVqz2o6PFw== Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id cm12-20020a056a020a0c00b005d5d32bf0cbsi1892734pgb.463.2024.02.13.02.05.58 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 02:05:58 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-63243-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=CRwbn6KQ; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63243-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63243-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 1F0EFB242FB for ; Tue, 13 Feb 2024 09:46:15 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C6763381A1; Tue, 13 Feb 2024 09:41:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="CRwbn6KQ" Received: from out-185.mta0.migadu.com (out-185.mta0.migadu.com [91.218.175.185]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3909936AEB for ; Tue, 13 Feb 2024 09:41:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.185 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707817291; cv=none; b=WCHkMdOJMKcUoEJ/Jh5P7BMoh2JMNJ3acwaihHvAw/saons3ZiAFkvzSjWJk4m8SWYFxSGst9YAQjWbT9Dm+D9lL5THwt1KNZpCrvTnfffPiz2+pyMNhedMYGRjgQoryqrPOEz3vkBy3l2wqlehMIU90yu+yPP1mKtsaYvKpphA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707817291; c=relaxed/simple; bh=NhGrZyxkIuZQWCcTYldapP56WiHf0mmlItW4rC91NEY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=VRod+gjrWOYvWsfQ67Fy1Mtq4+mQ329CHCHCgz+FDgAYavqXqiWatfiUKxGBDkMUwh15sJxR9o4c1zBurcaJgHmWgLd/bbnD7WYiFKXmnZyweLLr3QEkdU825/mtZBU8j/UyGdSCP6ZkQW89vb1PzhjzY3dn8VXFK9pTv9iCBk4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=CRwbn6KQ; arc=none smtp.client-ip=91.218.175.185 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1707817287; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OMPdGXoxf8WnGOh2+4/hLw9TzTmbExY6hN0KedYmhJk=; b=CRwbn6KQF18IkX38dkKxlBxMgn+o6HKW3LBuGcNnDDhTK4YWSYwa/N5KOPkI5MELb7MNxk lejmLVorOSCqjxkDGoKe79nlEZbQPziXzd9QczzrMkx1y8ImIqWoiYbtve1oD/GCr1uAy9 ugLbbF93vmM0VnXAJ1aJcH5GQxei1yQ= From: Oliver Upton To: kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , linux-kernel@vger.kernel.org, Oliver Upton Subject: [PATCH v2 19/23] KVM: selftests: Add a minimal library for interacting with an ITS Date: Tue, 13 Feb 2024 09:41:14 +0000 Message-ID: <20240213094114.3961683-1-oliver.upton@linux.dev> In-Reply-To: <20240213093250.3960069-1-oliver.upton@linux.dev> References: <20240213093250.3960069-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790777762329301651 X-GMAIL-MSGID: 1790777762329301651 A prerequisite of testing LPI injection performance is of course instantiating an ITS for the guest. Add a small library for creating an ITS and interacting with it *from userspace*. Yep, you read that right. KVM unintentionally allows userspace to send commands to the virtual ITS via the command queue. Besides adding test coverage for an elusive UAPI, interacting with the ITS in userspace simplifies the handling of commands that need to allocate memory, like a MAPD command with an ITT. Signed-off-by: Oliver Upton --- .../selftests/kvm/include/aarch64/gic.h | 7 +- .../selftests/kvm/include/aarch64/vgic.h | 20 ++ .../testing/selftests/kvm/lib/aarch64/vgic.c | 241 ++++++++++++++++++ 3 files changed, 267 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/kvm/include/aarch64/gic.h b/tools/testing/selftests/kvm/include/aarch64/gic.h index 16d944486e9c..abb41d67880c 100644 --- a/tools/testing/selftests/kvm/include/aarch64/gic.h +++ b/tools/testing/selftests/kvm/include/aarch64/gic.h @@ -11,7 +11,12 @@ enum gic_type { GIC_TYPE_MAX, }; -#define GICD_BASE_GPA 0x8000000ULL +/* + * Note that the redistributor frames are at the end, as the range scales + * with the number of vCPUs in the VM. + */ +#define GITS_BASE_GPA 0x8000000ULL +#define GICD_BASE_GPA (GITS_BASE_GPA + SZ_128K) #define GICR_BASE_GPA (GICD_BASE_GPA + SZ_64K) /* The GIC is identity-mapped into the guest at the time of setup. */ diff --git a/tools/testing/selftests/kvm/include/aarch64/vgic.h b/tools/testing/selftests/kvm/include/aarch64/vgic.h index ce19aa0a8360..d45b2902439d 100644 --- a/tools/testing/selftests/kvm/include/aarch64/vgic.h +++ b/tools/testing/selftests/kvm/include/aarch64/vgic.h @@ -32,4 +32,24 @@ void kvm_irq_write_isactiver(int gic_fd, uint32_t intid, struct kvm_vcpu *vcpu); #define KVM_IRQCHIP_NUM_PINS (1020 - 32) +struct vgic_its { + int its_fd; + void *cmdq_hva; + size_t cmdq_size; +}; + +struct vgic_its *vgic_its_setup(struct kvm_vm *vm, + vm_paddr_t coll_tbl, size_t coll_tbl_sz, + vm_paddr_t device_tbl, size_t device_tbl_sz, + vm_paddr_t cmdq, size_t cmdq_size); +void vgic_its_destroy(struct vgic_its *its); + +void vgic_its_send_mapd_cmd(struct vgic_its *its, u32 device_id, + vm_paddr_t itt_base, size_t itt_size, bool valid); +void vgic_its_send_mapc_cmd(struct vgic_its *its, struct kvm_vcpu *vcpu, + u32 collection_id, bool valid); +void vgic_its_send_mapti_cmd(struct vgic_its *its, u32 device_id, + u32 event_id, u32 collection_id, u32 intid); +void vgic_its_send_invall_cmd(struct vgic_its *its, u32 collection_id); + #endif // SELFTEST_KVM_VGIC_H diff --git a/tools/testing/selftests/kvm/lib/aarch64/vgic.c b/tools/testing/selftests/kvm/lib/aarch64/vgic.c index ac55b6c2e915..fc7b4fbe6453 100644 --- a/tools/testing/selftests/kvm/lib/aarch64/vgic.c +++ b/tools/testing/selftests/kvm/lib/aarch64/vgic.c @@ -12,6 +12,7 @@ #include "vgic.h" #include "gic.h" #include "gic_v3.h" +#include "processor.h" /* * vGIC-v3 default host setup @@ -166,3 +167,243 @@ void kvm_irq_write_isactiver(int gic_fd, uint32_t intid, struct kvm_vcpu *vcpu) { vgic_poke_irq(gic_fd, intid, vcpu, GICD_ISACTIVER); } + +static u64 vgic_its_read_reg(int its_fd, unsigned long offset) +{ + u64 attr; + + kvm_device_attr_get(its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS, + offset, &attr); + return attr; +} + +static void vgic_its_write_reg(int its_fd, unsigned long offset, u64 val) +{ + kvm_device_attr_set(its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS, + offset, &val); +} + +static unsigned long vgic_its_find_baser(int its_fd, unsigned int type) +{ + int i; + + for (i = 0; i < GITS_BASER_NR_REGS; i++) { + u64 baser; + unsigned long offset = GITS_BASER + (i * sizeof(baser)); + + baser = vgic_its_read_reg(its_fd, offset); + if (GITS_BASER_TYPE(baser) == type) + return offset; + } + + TEST_FAIL("Couldn't find an ITS BASER of type %u", type); + return -1; +} + +static void vgic_its_install_table(int its_fd, unsigned int type, vm_paddr_t base, + size_t size) +{ + unsigned long offset = vgic_its_find_baser(its_fd, type); + u64 baser; + + baser = ((size / SZ_64K) - 1) | + GITS_BASER_PAGE_SIZE_64K | + GITS_BASER_InnerShareable | + base | + GITS_BASER_RaWaWb | + GITS_BASER_VALID; + + vgic_its_write_reg(its_fd, offset, baser); +} + +static void vgic_its_install_cmdq(int its_fd, vm_paddr_t base, size_t size) +{ + u64 cbaser; + + cbaser = ((size / SZ_4K) - 1) | + GITS_CBASER_InnerShareable | + base | + GITS_CBASER_RaWaWb | + GITS_CBASER_VALID; + + vgic_its_write_reg(its_fd, GITS_CBASER, cbaser); +} + +struct vgic_its *vgic_its_setup(struct kvm_vm *vm, + vm_paddr_t coll_tbl, size_t coll_tbl_sz, + vm_paddr_t device_tbl, size_t device_tbl_sz, + vm_paddr_t cmdq, size_t cmdq_size) +{ + int its_fd = kvm_create_device(vm, KVM_DEV_TYPE_ARM_VGIC_ITS); + struct vgic_its *its = malloc(sizeof(struct vgic_its)); + u64 attr, ctlr; + + attr = GITS_BASE_GPA; + kvm_device_attr_set(its_fd, KVM_DEV_ARM_VGIC_GRP_ADDR, + KVM_VGIC_ITS_ADDR_TYPE, &attr); + + kvm_device_attr_set(its_fd, KVM_DEV_ARM_VGIC_GRP_CTRL, + KVM_DEV_ARM_VGIC_CTRL_INIT, NULL); + + vgic_its_install_table(its_fd, GITS_BASER_TYPE_COLLECTION, coll_tbl, + coll_tbl_sz); + vgic_its_install_table(its_fd, GITS_BASER_TYPE_DEVICE, device_tbl, + device_tbl_sz); + + vgic_its_install_cmdq(its_fd, cmdq, cmdq_size); + + ctlr = vgic_its_read_reg(its_fd, GITS_CTLR); + ctlr |= GITS_CTLR_ENABLE; + vgic_its_write_reg(its_fd, GITS_CTLR, ctlr); + + *its = (struct vgic_its) { + .its_fd = its_fd, + .cmdq_hva = addr_gpa2hva(vm, cmdq), + .cmdq_size = cmdq_size, + }; + + return its; +} + +void vgic_its_destroy(struct vgic_its *its) +{ + close(its->its_fd); + free(its); +} + +struct its_cmd_block { + union { + u64 raw_cmd[4]; + __le64 raw_cmd_le[4]; + }; +}; + +static inline void its_fixup_cmd(struct its_cmd_block *cmd) +{ + /* Let's fixup BE commands */ + cmd->raw_cmd_le[0] = cpu_to_le64(cmd->raw_cmd[0]); + cmd->raw_cmd_le[1] = cpu_to_le64(cmd->raw_cmd[1]); + cmd->raw_cmd_le[2] = cpu_to_le64(cmd->raw_cmd[2]); + cmd->raw_cmd_le[3] = cpu_to_le64(cmd->raw_cmd[3]); +} + +static void its_mask_encode(u64 *raw_cmd, u64 val, int h, int l) +{ + u64 mask = GENMASK_ULL(h, l); + *raw_cmd &= ~mask; + *raw_cmd |= (val << l) & mask; +} + +static void its_encode_cmd(struct its_cmd_block *cmd, u8 cmd_nr) +{ + its_mask_encode(&cmd->raw_cmd[0], cmd_nr, 7, 0); +} + +static void its_encode_devid(struct its_cmd_block *cmd, u32 devid) +{ + its_mask_encode(&cmd->raw_cmd[0], devid, 63, 32); +} + +static void its_encode_event_id(struct its_cmd_block *cmd, u32 id) +{ + its_mask_encode(&cmd->raw_cmd[1], id, 31, 0); +} + +static void its_encode_phys_id(struct its_cmd_block *cmd, u32 phys_id) +{ + its_mask_encode(&cmd->raw_cmd[1], phys_id, 63, 32); +} + +static void its_encode_size(struct its_cmd_block *cmd, u8 size) +{ + its_mask_encode(&cmd->raw_cmd[1], size, 4, 0); +} + +static void its_encode_itt(struct its_cmd_block *cmd, u64 itt_addr) +{ + its_mask_encode(&cmd->raw_cmd[2], itt_addr >> 8, 51, 8); +} + +static void its_encode_valid(struct its_cmd_block *cmd, int valid) +{ + its_mask_encode(&cmd->raw_cmd[2], !!valid, 63, 63); +} + +static void its_encode_target(struct its_cmd_block *cmd, u64 target_addr) +{ + its_mask_encode(&cmd->raw_cmd[2], target_addr >> 16, 51, 16); +} + +static void its_encode_collection(struct its_cmd_block *cmd, u16 col) +{ + its_mask_encode(&cmd->raw_cmd[2], col, 15, 0); +} + +static void vgic_its_send_cmd(struct vgic_its *its, struct its_cmd_block *cmd) +{ + u64 cwriter = vgic_its_read_reg(its->its_fd, GITS_CWRITER); + struct its_cmd_block *dst = its->cmdq_hva + cwriter; + u64 next; + + its_fixup_cmd(cmd); + + WRITE_ONCE(*dst, *cmd); + dsb(ishst); + + next = (cwriter + sizeof(*cmd)) % its->cmdq_size; + vgic_its_write_reg(its->its_fd, GITS_CWRITER, next); + + TEST_ASSERT(vgic_its_read_reg(its->its_fd, GITS_CREADR) == next, + "ITS didn't process command at offset: %lu\n", cwriter); +} + +void vgic_its_send_mapd_cmd(struct vgic_its *its, u32 device_id, + vm_paddr_t itt_base, size_t itt_size, bool valid) +{ + struct its_cmd_block cmd = {}; + + its_encode_cmd(&cmd, GITS_CMD_MAPD); + its_encode_devid(&cmd, device_id); + its_encode_size(&cmd, ilog2(itt_size) - 1); + its_encode_itt(&cmd, itt_base); + its_encode_valid(&cmd, valid); + + vgic_its_send_cmd(its, &cmd); +} + +void vgic_its_send_mapc_cmd(struct vgic_its *its, struct kvm_vcpu *vcpu, + u32 collection_id, bool valid) +{ + struct its_cmd_block cmd = {}; + + its_encode_cmd(&cmd, GITS_CMD_MAPC); + its_encode_collection(&cmd, collection_id); + its_encode_target(&cmd, vcpu->id); + its_encode_valid(&cmd, valid); + + vgic_its_send_cmd(its, &cmd); +} + +void vgic_its_send_mapti_cmd(struct vgic_its *its, u32 device_id, + u32 event_id, u32 collection_id, u32 intid) +{ + struct its_cmd_block cmd = {}; + + its_encode_cmd(&cmd, GITS_CMD_MAPTI); + its_encode_devid(&cmd, device_id); + its_encode_event_id(&cmd, event_id); + its_encode_phys_id(&cmd, intid); + its_encode_collection(&cmd, collection_id); + + vgic_its_send_cmd(its, &cmd); +} + +void vgic_its_send_invall_cmd(struct vgic_its *its, u32 collection_id) +{ + struct its_cmd_block cmd = {}; + + its_encode_cmd(&cmd, GITS_CMD_INVALL); + its_encode_collection(&cmd, collection_id); + + vgic_its_send_cmd(its, &cmd); +} From patchwork Tue Feb 13 09:41:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 200328 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp438076dyb; Tue, 13 Feb 2024 02:09:46 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCXFGyqIzCMe3t9GKQ9/deGjY7TuiD0wLzTwP5ZGTPk3rQz/qRLqcbRxL8Rr5uwH+AWyi331/xeeD8wFohiOPpefT3gnKg== X-Google-Smtp-Source: AGHT+IH5y4KcuF9MsIvY9c5EkRf9fE/mBfgPTvfksHlUzD+HrM5HbJE5UesaykMa9SewZXEvPYGH X-Received: by 2002:a05:6a20:9f98:b0:19c:9d37:ec59 with SMTP id mm24-20020a056a209f9800b0019c9d37ec59mr8830535pzb.28.1707818985997; Tue, 13 Feb 2024 02:09:45 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707818985; cv=pass; d=google.com; s=arc-20160816; b=iAbESJnj/8IT+MeGNTwbBNoi1SoDaKpc3ZnpdNMptPU5It914seaLYKublZAXattZ2 AfB7eV30LxCjXffkHJeQTScmUiCNB2ycqUeNtRUqtZmSCR8hXzGIuSHn112jrs6wzrTQ IFXsmECElMLSMhpWGOX6rmSW5usAANBptLC8MXSQ3yeCPXc06f50FVOBXKmtQW5ALVKh +8dfg6p0IP5aqkOLa1JOQzNU4RhCcyflMf5iWJapFFzURzS9RfP5x1+NnSycmHUuWX8V h53M1SkCQmE2Td+XL9eT4zC9RXJnHS3PK2u7EibKXGxGUBZIm0ahkkysszqr3q7W0Woc VNKQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=cnCi4BYIqMn8+Kc3y6O2Ph5n/B+/QtuSdPhWS/W9G2A=; fh=YWthtnI+0NKr19c04v/zQ6P6zqSydSqPe3k2q88b84s=; b=kQnhZjxID39y6el1XK8sgW3vIHX4qxLT35g+OMp/J2C3nt3bc8CdR9BPsKrhmDaeFY 63++HaWPhWvT1Bwo8t458D9weIdV4OTxB5k4KHX1cujrpmZGapTBdvdgjNJ/50oWPo+f MVGU4ebKCsxvA1kIGKgCUGgOzhyLfPSjQA7MY5tIm5IhdE7Vm96LQHMNm6eI2FY5AyoD UqwWmttukn75Y6c9N+j3YOljgWEMD2sNhH1g1L/V7yzQ2HBbzaF5dLFb313end8pZvlF HDSqng552gDI87QHxGnuGqxQccdspVhOy1cJBvga49aexDHePmgEOiMZ3YukGqXdkcDJ cZGg==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=fnhsyHji; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63253-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63253-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev X-Forwarded-Encrypted: i=2; AJvYcCVOgOMkf/YYrbI4EUkmXib0eF44cuVS0RMUO4ImAgBYSQ+JZwmYkFhjVTrrk+RsLCvkwLLBo//NbW6yDIVHAeFMncBLSA== Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id c185-20020a6335c2000000b005dc4bdd113fsi1766254pga.168.2024.02.13.02.09.45 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 02:09:45 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-63253-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=fnhsyHji; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63253-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63253-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 2C847B21D71 for ; Tue, 13 Feb 2024 09:49:24 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id B5CED56440; Tue, 13 Feb 2024 09:41:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="fnhsyHji" Received: from out-174.mta1.migadu.com (out-174.mta1.migadu.com [95.215.58.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4E5035578B for ; Tue, 13 Feb 2024 09:41:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707817315; cv=none; b=pPDkw50rPRj+v0yVsBrGkploosjEG0sYwfzrjuYejgp6zCay9a2x+RdWVpOT/8nCX0tDllcsCOAr7TspexOZ8nYs4cUtSV+BOu+uuayBbv2otI+mBrskaLPGASUkt83QhPG7yjaRqstl5Lg50jW129IJjTcc87a/sXpze5ngU3A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707817315; c=relaxed/simple; bh=LfIizmE1ae4sFNPaSEw/Hx7kqnSpkrUJGFdC2oe/wBw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=k/Qh3m+DjW3MbjVaqGPgtl149zyDjMbNjySoF7zLQbGMfmxWAP2v28YApigOBbgDTr8hFtQBnIJCxj/RhPc2A1Am4i/wcgeLdrVY/+7d6100Bn5w0mPvN4GbLFO8AYBwRHH25jt75p0q5u6mJ9MMArOA5Ngc6CPUdJVfhjGmsGY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=fnhsyHji; arc=none smtp.client-ip=95.215.58.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1707817311; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cnCi4BYIqMn8+Kc3y6O2Ph5n/B+/QtuSdPhWS/W9G2A=; b=fnhsyHjiUGadzULajmuLUrF0hDvKZSO4srOYrhf2IeX3GRiXL//klQxXw/rurnNZhNr9nP jj7nNFXEKEB2AIDKUw/llicgcNS7k/2TxnKtp4W2cdIf4xryaW7szE89B6sa3URnxcXgKm 9R4Wb9I7kXQOYhAUbbVdKvesvny1ADg= From: Oliver Upton To: kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , linux-kernel@vger.kernel.org, Oliver Upton Subject: [PATCH v2 20/23] KVM: selftests: Add helper for enabling LPIs on a redistributor Date: Tue, 13 Feb 2024 09:41:41 +0000 Message-ID: <20240213094141.3962026-1-oliver.upton@linux.dev> In-Reply-To: <20240213093250.3960069-1-oliver.upton@linux.dev> References: <20240213093250.3960069-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790778000899907291 X-GMAIL-MSGID: 1790778000899907291 The selftests GIC library presently does not support LPIs. Add a userspace helper for configuring a redistributor for LPIs, installing an LPI configuration table and LPI pending table. Signed-off-by: Oliver Upton --- .../selftests/kvm/include/aarch64/vgic.h | 4 + .../testing/selftests/kvm/lib/aarch64/vgic.c | 78 +++++++++++++++++++ 2 files changed, 82 insertions(+) diff --git a/tools/testing/selftests/kvm/include/aarch64/vgic.h b/tools/testing/selftests/kvm/include/aarch64/vgic.h index d45b2902439d..ae6a54453c62 100644 --- a/tools/testing/selftests/kvm/include/aarch64/vgic.h +++ b/tools/testing/selftests/kvm/include/aarch64/vgic.h @@ -32,6 +32,10 @@ void kvm_irq_write_isactiver(int gic_fd, uint32_t intid, struct kvm_vcpu *vcpu); #define KVM_IRQCHIP_NUM_PINS (1020 - 32) +void vgic_rdist_enable_lpis(int gic_fd, struct kvm_vcpu *vcpu, + vm_paddr_t cfg_table, size_t cfg_table_size, + vm_paddr_t pend_table); + struct vgic_its { int its_fd; void *cmdq_hva; diff --git a/tools/testing/selftests/kvm/lib/aarch64/vgic.c b/tools/testing/selftests/kvm/lib/aarch64/vgic.c index fc7b4fbe6453..2e1e0aa8cabc 100644 --- a/tools/testing/selftests/kvm/lib/aarch64/vgic.c +++ b/tools/testing/selftests/kvm/lib/aarch64/vgic.c @@ -3,8 +3,10 @@ * ARM Generic Interrupt Controller (GIC) v3 host support */ +#include #include #include +#include #include #include @@ -168,6 +170,82 @@ void kvm_irq_write_isactiver(int gic_fd, uint32_t intid, struct kvm_vcpu *vcpu) vgic_poke_irq(gic_fd, intid, vcpu, GICD_ISACTIVER); } +#define VGIC_AFFINITY_0_SHIFT 0 +#define VGIC_AFFINITY_1_SHIFT 8 +#define VGIC_AFFINITY_2_SHIFT 16 +#define VGIC_AFFINITY_3_SHIFT 24 + +#define MPIDR_TO_VGIC_LEVEL(mpidr, level) \ + ((((mpidr) >> MPIDR_LEVEL_SHIFT(level)) & MPIDR_LEVEL_MASK) << \ + VGIC_AFFINITY_## level ##_SHIFT) + +#define MPIDR_TO_VGIC(mpidr) \ + ((MPIDR_TO_VGIC_LEVEL(mpidr, 0) | \ + MPIDR_TO_VGIC_LEVEL(mpidr, 1) | \ + MPIDR_TO_VGIC_LEVEL(mpidr, 2) | \ + MPIDR_TO_VGIC_LEVEL(mpidr, 3)) << 32) + +static u32 vgic_rdist_read_reg(int gic_fd, struct kvm_vcpu *vcpu, + unsigned long offset) +{ + u64 mpidr, attr; + u32 val; + + vcpu_get_reg(vcpu, KVM_ARM64_SYS_REG(SYS_MPIDR_EL1), &mpidr); + + attr = MPIDR_TO_VGIC(mpidr) | offset; + kvm_device_attr_get(gic_fd, KVM_DEV_ARM_VGIC_GRP_REDIST_REGS, + attr, &val); + + return val; +} + +static void vgic_rdist_write_reg(int gic_fd, struct kvm_vcpu *vcpu, + unsigned long offset, u32 val) +{ + u64 mpidr, attr; + + vcpu_get_reg(vcpu, KVM_ARM64_SYS_REG(SYS_MPIDR_EL1), &mpidr); + + attr = MPIDR_TO_VGIC(mpidr) | offset; + kvm_device_attr_set(gic_fd, KVM_DEV_ARM_VGIC_GRP_REDIST_REGS, + attr, &val); +} + +static void vgic_rdist_write_baser(int gic_fd, struct kvm_vcpu *vcpu, + unsigned long offset, u64 val) +{ + u32 attr = val; + + vgic_rdist_write_reg(gic_fd, vcpu, offset, attr); + + attr = val >> 32; + vgic_rdist_write_reg(gic_fd, vcpu, offset + 4, attr); +} + +void vgic_rdist_enable_lpis(int gic_fd, struct kvm_vcpu *vcpu, + vm_paddr_t cfg_table, size_t cfg_table_size, + vm_paddr_t pend_table) +{ + u32 ctlr; + u64 val; + + val = (cfg_table | + GICR_PROPBASER_InnerShareable | + GICR_PROPBASER_RaWaWb | + ((ilog2(cfg_table_size) - 1) & GICR_PROPBASER_IDBITS_MASK)); + vgic_rdist_write_baser(gic_fd, vcpu, GICR_PROPBASER, val); + + val = (pend_table | + GICR_PENDBASER_InnerShareable | + GICR_PENDBASER_RaWaWb); + vgic_rdist_write_baser(gic_fd, vcpu, GICR_PENDBASER, val); + + ctlr = vgic_rdist_read_reg(gic_fd, vcpu, GICR_CTLR); + ctlr |= GICR_CTLR_ENABLE_LPIS; + vgic_rdist_write_reg(gic_fd, vcpu, GICR_CTLR, ctlr); +} + static u64 vgic_its_read_reg(int its_fd, unsigned long offset) { u64 attr; From patchwork Tue Feb 13 09:42:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 200306 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp429104dyb; Tue, 13 Feb 2024 01:49:43 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWXYIwOr03mwagyl4NxlbtXag+RavQsdO7WnPPLeVG/q1G127/G1G+3N3e+HI4k/BXYaGVFmJzldfFBi1GBA5UWEo6qog== X-Google-Smtp-Source: AGHT+IG/jmHn9vdDylvBgNmTbA5ZU4VH4yihZVBbTXPHd2t8gGqwv8Pk2xw+e+Rg2IY5ZjXSNZIi X-Received: by 2002:a05:620a:5659:b0:785:c9fc:29b5 with SMTP id vw25-20020a05620a565900b00785c9fc29b5mr3227530qkn.5.1707817783362; Tue, 13 Feb 2024 01:49:43 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707817783; cv=pass; d=google.com; s=arc-20160816; b=IU4JCNqY+/ItvnwXXqel05vS8uSxhdo0nfN6nNGYK7tivPYPcM0mMEIgLYtA8qZ8l+ y3L2LCkG1L5yy3w0X/C5zEun69GECpDWfOsaoUhiFkNTEo38eiRzN3jDa7Ksyf82qrbo NV9r9ccSAYg4HcCng58mTkKTzQlvHPB2l0zPfI7RoGvpzoFKhWL+n47p2Gzgvj6mPv4z yZ/Xp9aSN4neoZEQ7kQY1nCBdokg4DEPrF4zJP7CC3rZvqESmG/Zxlc4jT9hUaaum5rJ AH4auJkgxwfaWTHWdQPUZlfe7w8TuzRmrYEyq9yqFjygfpxrYYGyKpi/hb2FDV4xfy9R eVug== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=i+nhnpV9cTa5T5zRVWJVPTwjU5zRG8qikjP97n6jlxI=; fh=fg8BUbepVm6phOz8WMuChFWwXJSQOTCPHnFOc0brygo=; b=IVopwTPWy25b6Ir0k397A35w1SVZvkdgULTEnddTwlpnWcWyOEffZI12yxDRM5BXQg Sitn0fe2dJWsVEG4yp3LYfj3hvkEiMSf8hGO+WFX/1HBdZVq4H0M96R5vaPzgdoPfe5O p4RvD4/g0CEbOQEmo+e4f+O/SGxusakttJu9Zy30xbiYihixodYg8SlXioohb7FuYsbC z9zII5rluD9s9jvAfj0GRSswdv8bFzFWWoVestcRH0bDHpdu+Fdo00Umu2EHctUUc7Bk f1zdyrsF/7XIueD9F9BOsFNd1H4LvNshpF1M1rrR/tobcnJQZKGHZfYn3dH3Y4GNXQ12 zPlQ==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=ue3reT56; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63254-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63254-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev X-Forwarded-Encrypted: i=2; AJvYcCU8Tn0IP19ZcTkFoLK0yHYz4gCpvu06JJZmvUcGF0m3IvIeUQGiHbnMj0/iaZqm6SB7AW7/LjfDu4CBbRp0HL0Erl7IjQ== Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id 2-20020a05620a04c200b00785af741c35si7944162qks.618.2024.02.13.01.49.43 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 01:49:43 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-63254-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=ue3reT56; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63254-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63254-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 17C041C2113D for ; Tue, 13 Feb 2024 09:49:43 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 5938C4DA0A; Tue, 13 Feb 2024 09:42:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="ue3reT56" Received: from out-175.mta0.migadu.com (out-175.mta0.migadu.com [91.218.175.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1661D224D6 for ; Tue, 13 Feb 2024 09:42:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707817347; cv=none; b=RsnhS5SFbeLmofj71WDyN67R3bCeN2Fq8yltcGDG1GgKerc+KnLzUopXPFmux0dfmz8L5Txv3psX97oLHs6MWwup/mUObglXVYvnBwhUV2jJbV1wC34TKl2v9Yw1EJhMaPJCzGFQn2s+hVkEpOXlzhGCp9s8y1VSEut0n1rAJGY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707817347; c=relaxed/simple; bh=BBRquBRon3PszAnzHobEjBuxkFIZiY9f9MB60l15p2A=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Ui4jVmAQEkOUrh0khVFMBvNc8mVEByoRjPKDlW0Hf/YTLjkgZaw1AIOvRSofBEKUVxDfkIMSpqXScqCRIRXoiWQ66sby/FkjcFIFmD0M0iLR/YFFGeOeNksbT4Baqqfssz5xDzBO6CLup9tSO/qDHlZVpuOtwZH2H4QCUb00pxk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=ue3reT56; arc=none smtp.client-ip=91.218.175.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1707817344; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=i+nhnpV9cTa5T5zRVWJVPTwjU5zRG8qikjP97n6jlxI=; b=ue3reT56fVXCKJyglvSSqFfaWxf6VnGHRUKrI4CNwKYTCq1mx3ZedLMj7I6j/No9Pmtoct y0ceCEmMeibOZUMIqToQlUb7zPn2sD5RK/QSx6aHLGR6hA7LeX8Ui5+aZUt6X1AUtdctXB 7F83XFPIhmAamNtx76XTi0OBsk3QeSg= From: Oliver Upton To: kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , linux-kernel@vger.kernel.org, Oliver Upton Subject: [PATCH v2 21/23] KVM: selftests: Use MPIDR_HWID_BITMASK from cputype.h Date: Tue, 13 Feb 2024 09:42:02 +0000 Message-ID: <20240213094202.3962084-1-oliver.upton@linux.dev> In-Reply-To: <20240213093250.3960069-1-oliver.upton@linux.dev> References: <20240213093250.3960069-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790776740281525230 X-GMAIL-MSGID: 1790776740281525230 No need for a home-rolled definition, just rely on the common header. Signed-off-by: Oliver Upton --- tools/testing/selftests/kvm/aarch64/psci_test.c | 2 ++ tools/testing/selftests/kvm/include/aarch64/processor.h | 2 -- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/kvm/aarch64/psci_test.c b/tools/testing/selftests/kvm/aarch64/psci_test.c index 9b004905d1d3..9fa3578d47d5 100644 --- a/tools/testing/selftests/kvm/aarch64/psci_test.c +++ b/tools/testing/selftests/kvm/aarch64/psci_test.c @@ -13,7 +13,9 @@ #define _GNU_SOURCE +#include #include +#include #include "kvm_util.h" #include "processor.h" diff --git a/tools/testing/selftests/kvm/include/aarch64/processor.h b/tools/testing/selftests/kvm/include/aarch64/processor.h index cf20e44e86f2..94e8f1892754 100644 --- a/tools/testing/selftests/kvm/include/aarch64/processor.h +++ b/tools/testing/selftests/kvm/include/aarch64/processor.h @@ -58,8 +58,6 @@ MAIR_ATTRIDX(MAIR_ATTR_NORMAL, MT_NORMAL) | \ MAIR_ATTRIDX(MAIR_ATTR_NORMAL_WT, MT_NORMAL_WT)) -#define MPIDR_HWID_BITMASK (0xff00fffffful) - void aarch64_vcpu_setup(struct kvm_vcpu *vcpu, struct kvm_vcpu_init *init); struct kvm_vcpu *aarch64_vcpu_add(struct kvm_vm *vm, uint32_t vcpu_id, struct kvm_vcpu_init *init, void *guest_code); From patchwork Tue Feb 13 09:43:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 200314 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp432673dyb; Tue, 13 Feb 2024 01:59:23 -0800 (PST) X-Google-Smtp-Source: AGHT+IFhmNRtLLcSBALE6NiVxGxhXCsShwM4aAOEU7oyxdw/U0k5VydIzOLfjLmH003kGcn6uotb X-Received: by 2002:a17:902:784e:b0:1d9:6dc6:6616 with SMTP id e14-20020a170902784e00b001d96dc66616mr7333320pln.68.1707818363339; Tue, 13 Feb 2024 01:59:23 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707818363; cv=pass; d=google.com; s=arc-20160816; b=ga2thcC2bHffcn5YJni5CKNiUYOzb0j556BBEL4uF361a7+aGk1kr1TYA5J4WW4s9r c6QUuydNF9bp6r0gsaMYRvm6GVK5GfhhgVk6H3X07cwQwZd6aaSfPpGu/w4j2szw9/vs w61S3fWqehx5OlmXo3yBRHQ/VU3uXioSYW7O402MINRHXTdPhThAu4Itd7R813I7X0E+ c/nXFHL4oIk1pYXP6xLNc5ogJ/QXFgQg4aAa9nzHxHVLGytjS3LgJiDy0aACioz9Z9P1 1IB+QGXK1CL5SIoQcUVNPd3IQ5fsxEl/AVUttfWQala3ktMW8nP4z9rLQGjYWbwUu+Lt q3kw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=TIA8t073Y1mWOrzngK+gZq8Rv9Ku8mU359Al0v0ZlVo=; fh=d7jZ/AhbVu4uIWoDJRtUGj0vgU+Jw5ZwG+gNq9cilKM=; b=o/xILE6i+1/gqPt/mIoC6XSoKPjL68y7zLjjlmnP+wuglRP/cqY+/g/l6wucu3on6u CdDk7OgxphGdlxKDNnh+W7TtN+l2dDKmQ01LEDrQfYjmRnT42gE2xsETpfgxbCU/87NJ Z/GWU3YW28XSFCwG1G8QSKlH6GiBeGngYcg3k4isG35UmTu1lKyDZNf7jxyA8trPaYSX o20QQoPHZUR4FWZmXwvscbkl0yF/IRgUO3nmu8owlOx0HgJM+WQ+QwzX7yKbgBRkY3zZ NK2C/HaBV+IorUQTsWBKU2LlNj95ui1OCx4dyFzCqCk/l+oOgfGHwHiQEg994zp1PYQ8 DYdQ==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=pDp7fdCA; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63258-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63258-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev X-Forwarded-Encrypted: i=2; AJvYcCUnxe5kzpU8TfrpK9xdqsAiQxhxVbIgspyeNffc+Tmphc/G8AXs47YQJHZSkUCKe9VvwRyPHWg0OmeqnCVXF+Ed+DYvwQ== Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id g6-20020a170902740600b001d8ecf3d0fbsi1740453pll.511.2024.02.13.01.59.23 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 01:59:23 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-63258-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=pDp7fdCA; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63258-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63258-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 9E7562885F0 for ; Tue, 13 Feb 2024 09:51:11 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 2CC8856468; Tue, 13 Feb 2024 09:43:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="pDp7fdCA" Received: from out-174.mta0.migadu.com (out-174.mta0.migadu.com [91.218.175.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7C6E2535BD for ; Tue, 13 Feb 2024 09:43:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707817399; cv=none; b=OQYHbgtvvuSc6WZxma5Dabp8AIjNUahdqeRV0wFye8Dd0KJkHHpLoC6J7K5lHhLP5NjayqgFrT6V4tQIJ7xpIfFuEIp4bnk8eoLwFfVWOKTly2onE1e5hieA6udDZV1QGcV0FGMQEgWQCkaQf2W9qgMwcekhI9bkCA6humRLKag= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707817399; c=relaxed/simple; bh=Sv6AJeH0Hx+RUsbmtZwuRTSHKGXUdMSy3gQCRocBC3I=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=desjWchIvPuMZHVHF1u1xW0hJ2O/YWADHriHZhI8heHuctNFzPyPXlTM4mxhm+/FlWfZD1JALBdZ6dN1COIOOvP8LNz2Hem8iSTGVe/pUMzyTTPQJYXOPQfZTZm2kdvfdKnI7Fz9CoYVWYS78hWQ1Gm77DVTf8ycXw/Kt+H8YLk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=pDp7fdCA; arc=none smtp.client-ip=91.218.175.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1707817395; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TIA8t073Y1mWOrzngK+gZq8Rv9Ku8mU359Al0v0ZlVo=; b=pDp7fdCAMJlgD8V76p/DKAEGmrfeT7I2UsWQvoLbAaKNLUT9y01KmaCeMZdTpFeS02R0Le JVNt7gKRp/mnkRddNuxjDZBgW/UEcqz04+uryRzYc036wRnehgDSRuyyN6abFiNmhdWvYG pcDSvuPC0fY3JGpzB25sF9KUkpP1R6w= From: Oliver Upton To: kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , linux-kernel@vger.kernel.org, Oliver Upton Subject: [PATCH v2 22/23] KVM: selftests: Hack in support for aligned page allocations Date: Tue, 13 Feb 2024 09:43:03 +0000 Message-ID: <20240213094303.3962318-1-oliver.upton@linux.dev> In-Reply-To: <20240213093250.3960069-1-oliver.upton@linux.dev> References: <20240213093250.3960069-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790777348207972072 X-GMAIL-MSGID: 1790777348207972072 Many of the memory allocations handed off to a GIC ITS require 64K alignment. One would think that requesting 64K of pages would follow natural alignment in selftests (much like alloc_pages() in the kernel), however that is unfortunately not the case. Add a new helper to allow a caller to decide if they want naturally-aligned page allocations. Deliberately avoid making this the default in case this subtly breaks assumptions or memory overheads in other selftests. Signed-off-by: Oliver Upton --- .../selftests/kvm/include/kvm_util_base.h | 2 ++ tools/testing/selftests/kvm/lib/kvm_util.c | 27 ++++++++++++++++--- 2 files changed, 25 insertions(+), 4 deletions(-) diff --git a/tools/testing/selftests/kvm/include/kvm_util_base.h b/tools/testing/selftests/kvm/include/kvm_util_base.h index 9e5afc472c14..750638cfa849 100644 --- a/tools/testing/selftests/kvm/include/kvm_util_base.h +++ b/tools/testing/selftests/kvm/include/kvm_util_base.h @@ -830,6 +830,8 @@ vm_paddr_t vm_phy_page_alloc(struct kvm_vm *vm, vm_paddr_t paddr_min, vm_paddr_t vm_phy_pages_alloc(struct kvm_vm *vm, size_t num, vm_paddr_t paddr_min, uint32_t memslot); vm_paddr_t vm_alloc_page_table(struct kvm_vm *vm); +vm_paddr_t vm_phy_pages_alloc_aligned(struct kvm_vm *vm, size_t num, + vm_paddr_t paddr_min, uint32_t memslot); /* * ____vm_create() does KVM_CREATE_VM and little else. __vm_create() also diff --git a/tools/testing/selftests/kvm/lib/kvm_util.c b/tools/testing/selftests/kvm/lib/kvm_util.c index e066d584c656..60948a004012 100644 --- a/tools/testing/selftests/kvm/lib/kvm_util.c +++ b/tools/testing/selftests/kvm/lib/kvm_util.c @@ -1984,8 +1984,9 @@ const char *exit_reason_str(unsigned int exit_reason) * and their base address is returned. A TEST_ASSERT failure occurs if * not enough pages are available at or above paddr_min. */ -vm_paddr_t vm_phy_pages_alloc(struct kvm_vm *vm, size_t num, - vm_paddr_t paddr_min, uint32_t memslot) +static vm_paddr_t __vm_phy_pages_alloc(struct kvm_vm *vm, size_t num, + vm_paddr_t paddr_min, uint32_t memslot, + bool aligned) { struct userspace_mem_region *region; sparsebit_idx_t pg, base; @@ -1997,14 +1998,20 @@ vm_paddr_t vm_phy_pages_alloc(struct kvm_vm *vm, size_t num, " paddr_min: 0x%lx page_size: 0x%x", paddr_min, vm->page_size); + TEST_ASSERT(!aligned || (paddr_min % vm->page_size * num) == 0, + "Min physical address isn't naturally aligned.\n" + " paddr_min: 0x%lx page_size: 0x%x num: %lu", + paddr_min, vm->page_size, num); + region = memslot2region(vm, memslot); base = pg = paddr_min >> vm->page_shift; do { for (; pg < base + num; ++pg) { if (!sparsebit_is_set(region->unused_phy_pages, pg)) { - base = pg = sparsebit_next_set(region->unused_phy_pages, pg); - break; + do { + base = pg = sparsebit_next_set(region->unused_phy_pages, pg); + } while (aligned && ((pg % num) != 0)); } } } while (pg && pg != base + num); @@ -2024,6 +2031,12 @@ vm_paddr_t vm_phy_pages_alloc(struct kvm_vm *vm, size_t num, return base * vm->page_size; } +vm_paddr_t vm_phy_pages_alloc(struct kvm_vm *vm, size_t num, + vm_paddr_t paddr_min, uint32_t memslot) +{ + return __vm_phy_pages_alloc(vm, num, paddr_min, memslot, false); +} + vm_paddr_t vm_phy_page_alloc(struct kvm_vm *vm, vm_paddr_t paddr_min, uint32_t memslot) { @@ -2036,6 +2049,12 @@ vm_paddr_t vm_alloc_page_table(struct kvm_vm *vm) vm->memslots[MEM_REGION_PT]); } +vm_paddr_t vm_phy_pages_alloc_aligned(struct kvm_vm *vm, size_t num, + vm_paddr_t paddr_min, uint32_t memslot) +{ + return __vm_phy_pages_alloc(vm, num, paddr_min, memslot, true); +} + /* * Address Guest Virtual to Host Virtual * From patchwork Tue Feb 13 09:43:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 200331 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp439128dyb; Tue, 13 Feb 2024 02:12:21 -0800 (PST) X-Google-Smtp-Source: AGHT+IHAcn+6THn+zSEIZFkhwk26HQXHDgn9BvGNXAYfDsvIXp7MZ+hR42GkZufFycnmZM3NIB85 X-Received: by 2002:a05:6a20:ce4c:b0:19e:5bcb:8485 with SMTP id id12-20020a056a20ce4c00b0019e5bcb8485mr8825733pzb.26.1707819141078; Tue, 13 Feb 2024 02:12:21 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707819140; cv=pass; d=google.com; s=arc-20160816; b=EkxwXf1yAB1SJcaMyzwI+9ue8UDW/M+8/rKbPsSuYf72Vz2464GfUQjhC5F5ZhSnku YR030D3b7STKcHmr1EmpkiSMWvlfw/Iq5iirQ7kGPhAcS6c+0m7WyODH2eksMKiUE6tB YFVeaQeJoAKh3zi2wbOYMhVhYjSEsGoMjYgTt6KcYJYyzvPiMqgLgSRI5bOW8yDe9ttZ sRqaAHvE1e6/A0bnsJnqldW0uk3QouzGn8/ksrwdxkFBZqgN/FT0U9QFJcUQ+DV++Ar3 kt8buMPTN59njF0GvstcpqAwNtRz22WCWaXpZ1YsKf7/WlTTd1BdlDBiiCAgtQ0ICQBh 3sSQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=H13Cz6CLHkaFtkIfr3ij1xpNFb/J5ZQsFY/+Jqrv/6c=; fh=AQeOaXJ8gBunfAiAALc26wiuSwtEHx4UQnfvJbrOrDE=; b=PqICzFKa4qbpgaqvA4QJ9j99UNmXrccc3b5yjq9uU8IezNLrzbtXe6Cm1BVnbKGK/q E4lSSnZ3EzmHioXQxassbcPB5uJC+oYPSWADk1syVZkxi5jJBXJf7zyGDVzgkFeTiXmj 3KqN9rN2BKJ9+0foiTUK8r2qK7v6QIUC9afvOVlXC85sZMI3cqe4jT003U19xizQQrxs 7gcgtz0dRWAkZqZW0prnbOvIoO5/2Eyu41cjzRGp25xCuPFN5R2m+VMJtcMfr8rbKfgS 6438JGtB/lgPLzuGeEFbxXIT7a0ELqodfPmPScEJ0v0OGpvlQX2zIi7ah91xumKr0Vns 89FA==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=lYJQgbKH; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63259-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63259-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev X-Forwarded-Encrypted: i=2; AJvYcCXPS6ZK1AwjrII+9y7QbZ5rupsEdpV9wm/Eowk4NPJly8+ssKJ/exsE8aAN9D3MfsQx88PqZxkXjjgbnJDcX0rYUMMP6Q== Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id cq26-20020a056a00331a00b006e0848c3318si6523623pfb.81.2024.02.13.02.12.19 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 02:12:20 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-63259-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=lYJQgbKH; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-63259-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63259-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 7B9BBB26CAC for ; Tue, 13 Feb 2024 09:51:34 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 5EA2556B98; Tue, 13 Feb 2024 09:43:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="lYJQgbKH" Received: from out-178.mta0.migadu.com (out-178.mta0.migadu.com [91.218.175.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 72A9B5646F for ; Tue, 13 Feb 2024 09:43:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707817433; cv=none; b=MeZWl1FkBmeJ5jQmMAychI8qzg9bp6TAX54D/hMCwgns85szFeC3uJcuDWdoAnncgvMFpG9sdP7wweW9Qb8I/qjoPYl5RIn80GR/QgNoWnLN7Ix0kl8429i/x0oudpAskaIYtgrmROeliv6zu+wsEoWThzzskmz99W2Ohqc48ao= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707817433; c=relaxed/simple; bh=EV24iNdzDT2nEj7Q1vml0ardEk2Nqgw3CUGt3h9w/3I=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=YRdSkxOY3GUnR6H4ibDOtvz/xtUDjbUxgG3jT3bKLxrygAlvm7aOddcR2pRN8ZODWBK/fUJnxwTtj7Yy8c9ShRodNNPY1Hfz62eeEZvrB7MCFkKGlJruSD5gKmTsxAEXKr+51RqNuHsGnh2NAl/nfcwTz6JurbIF4Y1qJ4nZAfI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=lYJQgbKH; arc=none smtp.client-ip=91.218.175.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1707817429; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=H13Cz6CLHkaFtkIfr3ij1xpNFb/J5ZQsFY/+Jqrv/6c=; b=lYJQgbKHGs5rTKG8xSw+PjdRNhoA6IMdh3A+K7IvmYSODoqaXsQ3KOr34W0wdEhnkfNsRc M3S6tIxLTOfED1QPyMkXT0QWLO6H1zFIRS8Jsfsv5rQMOtH/+xkp5BJjLY5x9riqVLUbi+ S3gze4o6ow2WHJXQgwJgbUG8TyMJiQk= From: Oliver Upton To: kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , linux-kernel@vger.kernel.org, Oliver Upton Subject: [PATCH v2 23/23] KVM: selftests: Add stress test for LPI injection Date: Tue, 13 Feb 2024 09:43:34 +0000 Message-ID: <20240213094334.3963630-1-oliver.upton@linux.dev> In-Reply-To: <20240213093250.3960069-1-oliver.upton@linux.dev> References: <20240213093250.3960069-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790778162957094516 X-GMAIL-MSGID: 1790778162957094516 Now that all the infrastructure is in place, add a test to stress KVM's LPI injection. Keep a 1:1 mapping of device IDs to signalling threads, allowing the user to scale up/down the sender side of an LPI. Make use of the new VM stats for the translation cache to estimate the translation hit rate. Since the primary focus of the test is on performance, you'll notice that the guest code is not pedantic about the LPIs it receives. Counting the number of LPIs would require synchronization between the device and vCPU threads to avoid coalescing and would get in the way of performance numbers. Signed-off-by: Oliver Upton --- tools/testing/selftests/kvm/Makefile | 1 + .../selftests/kvm/aarch64/vgic_lpi_stress.c | 388 ++++++++++++++++++ 2 files changed, 389 insertions(+) create mode 100644 tools/testing/selftests/kvm/aarch64/vgic_lpi_stress.c diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile index 492e937fab00..8a240d20ec5e 100644 --- a/tools/testing/selftests/kvm/Makefile +++ b/tools/testing/selftests/kvm/Makefile @@ -153,6 +153,7 @@ TEST_GEN_PROGS_aarch64 += aarch64/smccc_filter TEST_GEN_PROGS_aarch64 += aarch64/vcpu_width_config TEST_GEN_PROGS_aarch64 += aarch64/vgic_init TEST_GEN_PROGS_aarch64 += aarch64/vgic_irq +TEST_GEN_PROGS_aarch64 += aarch64/vgic_lpi_stress TEST_GEN_PROGS_aarch64 += aarch64/vpmu_counter_access TEST_GEN_PROGS_aarch64 += access_tracking_perf_test TEST_GEN_PROGS_aarch64 += demand_paging_test diff --git a/tools/testing/selftests/kvm/aarch64/vgic_lpi_stress.c b/tools/testing/selftests/kvm/aarch64/vgic_lpi_stress.c new file mode 100644 index 000000000000..d557d907728a --- /dev/null +++ b/tools/testing/selftests/kvm/aarch64/vgic_lpi_stress.c @@ -0,0 +1,388 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * vgic_lpi_stress - Stress test for KVM's ITS emulation + * + * Copyright (c) 2024 Google LLC + */ + +#include +#include +#include + +#include "kvm_util.h" +#include "gic.h" +#include "gic_v3.h" +#include "processor.h" +#include "ucall.h" +#include "vgic.h" + +#define TEST_MEMSLOT_INDEX 1 + +#define GIC_LPI_OFFSET 8192 + +static u32 nr_vcpus = 1; +static u32 nr_devices = 1; +static u32 nr_event_ids = 16; +static size_t nr_iterations = 1000; +static vm_paddr_t gpa_base; + +static struct kvm_vm *vm; +static struct kvm_vcpu **vcpus; +static struct vgic_its *its; +static int gic_fd; + +static bool request_vcpus_stop; + +static void guest_irq_handler(struct ex_regs *regs) +{ + u32 intid = gic_get_and_ack_irq(); + + if (intid == IAR_SPURIOUS) + return; + + GUEST_ASSERT(intid >= GIC_LPI_OFFSET); + gic_set_eoi(intid); +} + +static void guest_code(size_t nr_lpis) +{ + gic_init(GIC_V3, nr_vcpus); + + GUEST_SYNC(0); + + /* + * Don't use WFI here to avoid blocking the vCPU thread indefinitely and + * never getting the stop singal. + */ + while (!READ_ONCE(request_vcpus_stop)) + cpu_relax(); + + GUEST_DONE(); +} + +static void setup_memslot(void) +{ + size_t pages; + size_t sz; + + /* + * For the ITS: + * - A single level device table + * - A single level collection table + * - The command queue + * - An ITT for each device + */ + sz = (3 + nr_devices) * SZ_64K; + + /* + * For the redistributors: + * - A shared LPI configuration table + * - An LPI pending table for each vCPU + */ + sz += (1 + nr_vcpus) * SZ_64K; + + pages = sz / vm->page_size; + gpa_base = ((vm_compute_max_gfn(vm) + 1) * vm->page_size) - sz; + vm_userspace_mem_region_add(vm, VM_MEM_SRC_ANONYMOUS, gpa_base, + TEST_MEMSLOT_INDEX, pages, 0); +} + +#define LPI_PROP_DEFAULT_PRIO 0xa0 + +static void configure_lpis(vm_paddr_t prop_table) +{ + u8 *tbl = addr_gpa2hva(vm, prop_table); + size_t i; + + for (i = 0; i < (nr_devices * nr_event_ids); i++) { + tbl[i] = LPI_PROP_DEFAULT_PRIO | + LPI_PROP_GROUP1 | + LPI_PROP_ENABLED; + } +} + +static void setup_gic(void) +{ + vm_paddr_t coll_table, device_table, cmdq_base; + + gic_fd = vgic_v3_setup(vm, nr_vcpus, 64); + __TEST_REQUIRE(gic_fd >= 0, "Failed to create GICv3"); + + coll_table = vm_phy_pages_alloc_aligned(vm, SZ_64K / vm->page_size, + gpa_base, TEST_MEMSLOT_INDEX); + device_table = vm_phy_pages_alloc_aligned(vm, SZ_64K / vm->page_size, + gpa_base, TEST_MEMSLOT_INDEX); + cmdq_base = vm_phy_pages_alloc_aligned(vm, SZ_64K / vm->page_size, + gpa_base, TEST_MEMSLOT_INDEX); + + its = vgic_its_setup(vm, coll_table, SZ_64K, + device_table, SZ_64K, cmdq_base, SZ_64K); +} + +static void setup_its_mappings(void) +{ + u32 coll_id, device_id, event_id, intid = GIC_LPI_OFFSET; + + for (coll_id = 0; coll_id < nr_vcpus; coll_id++) + vgic_its_send_mapc_cmd(its, vcpus[coll_id], coll_id, true); + + /* Round-robin the LPIs to all of the vCPUs in the VM */ + coll_id = 0; + for (device_id = 0; device_id < nr_devices; device_id++) { + vm_paddr_t itt_base = vm_phy_pages_alloc_aligned(vm, SZ_64K / vm->page_size, + gpa_base, TEST_MEMSLOT_INDEX); + + vgic_its_send_mapd_cmd(its, device_id, itt_base, SZ_64K, true); + + for (event_id = 0; event_id < nr_event_ids; event_id++) { + vgic_its_send_mapti_cmd(its, device_id, event_id, coll_id, + intid++); + + coll_id = (coll_id + 1) % nr_vcpus; + } + } +} + +static void setup_rdists_for_lpis(void) +{ + size_t i; + + vm_paddr_t prop_table = vm_phy_pages_alloc_aligned(vm, SZ_64K / vm->page_size, + gpa_base, TEST_MEMSLOT_INDEX); + + configure_lpis(prop_table); + + for (i = 0; i < nr_vcpus; i++) { + vm_paddr_t pend_table; + + pend_table = vm_phy_pages_alloc_aligned(vm, SZ_64K / vm->page_size, + gpa_base, TEST_MEMSLOT_INDEX); + + vgic_rdist_enable_lpis(gic_fd, vcpus[i], prop_table, SZ_64K, pend_table); + } +} + +static void invalidate_all_rdists(void) +{ + int i; + + for (i = 0; i < nr_vcpus; i++) + vgic_its_send_invall_cmd(its, i); +} + +static void signal_lpi(u32 device_id, u32 event_id) +{ + vm_paddr_t db_addr = GITS_BASE_GPA + GITS_TRANSLATER; + + struct kvm_msi msi = { + .address_lo = db_addr, + .address_hi = db_addr >> 32, + .data = event_id, + .devid = device_id, + .flags = KVM_MSI_VALID_DEVID, + }; + + /* + * KVM_SIGNAL_MSI returns 1 if the MSI wasn't 'blocked' by the VM, + * which for arm64 implies having a valid translation in the ITS. + */ + TEST_ASSERT(__vm_ioctl(vm, KVM_SIGNAL_MSI, &msi) == 1, + "KVM_SIGNAL_MSI ioctl failed"); +} + +static pthread_barrier_t test_setup_barrier; +static pthread_barrier_t test_start_barrier; + +static void *lpi_worker_thread(void *data) +{ + u32 device_id = (size_t)data; + u32 event_id; + size_t i; + + pthread_barrier_wait(&test_start_barrier); + + for (i = 0; i < nr_iterations; i++) + for (event_id = 0; event_id < nr_event_ids; event_id++) + signal_lpi(device_id, event_id); + + return NULL; +} + +static void *vcpu_worker_thread(void *data) +{ + struct kvm_vcpu *vcpu = data; + struct ucall uc; + + while (true) { + vcpu_run(vcpu); + + switch (get_ucall(vcpu, &uc)) { + case UCALL_SYNC: + /* + * Tell the main thread to complete its last bit of + * setup and wait for the signal to start the test. + */ + pthread_barrier_wait(&test_setup_barrier); + pthread_barrier_wait(&test_start_barrier); + break; + case UCALL_DONE: + return NULL; + case UCALL_ABORT: + REPORT_GUEST_ASSERT(uc); + break; + default: + TEST_FAIL("Unknown ucall: %lu", uc.cmd); + } + } + + return NULL; +} + +static void report_stats(struct timespec delta) +{ + u64 cache_hits, cache_misses, cache_accesses; + double nr_lpis; + double time; + + nr_lpis = nr_devices * nr_event_ids * nr_iterations; + + time = delta.tv_sec; + time += ((double)delta.tv_nsec) / NSEC_PER_SEC; + + pr_info("Rate: %.2f LPIs/sec\n", nr_lpis / time); + + __vm_get_stat(vm, "vgic_its_trans_cache_hit", &cache_hits, 1); + __vm_get_stat(vm, "vgic_its_trans_cache_miss", &cache_misses, 1); + + cache_accesses = cache_hits + cache_misses; + + pr_info("Translation Cache\n"); + pr_info(" %lu hits\n", cache_hits); + pr_info(" %lu misses\n", cache_misses); + pr_info(" %.2f%% hit rate\n", 100 * (((double)cache_hits) / cache_accesses)); +} + +static void run_test(void) +{ + pthread_t *lpi_threads = malloc(nr_devices * sizeof(pthread_t)); + pthread_t *vcpu_threads = malloc(nr_vcpus * sizeof(pthread_t)); + struct timespec start, delta; + size_t i; + + TEST_ASSERT(lpi_threads && vcpu_threads, "Failed to allocate pthread arrays"); + + /* Only the vCPU threads need to do setup before starting the VM. */ + pthread_barrier_init(&test_setup_barrier, NULL, nr_vcpus + 1); + pthread_barrier_init(&test_start_barrier, NULL, nr_devices + nr_vcpus + 1); + + for (i = 0; i < nr_vcpus; i++) + pthread_create(&vcpu_threads[i], NULL, vcpu_worker_thread, vcpus[i]); + + for (i = 0; i < nr_devices; i++) + pthread_create(&lpi_threads[i], NULL, lpi_worker_thread, (void *)i); + + pthread_barrier_wait(&test_setup_barrier); + + /* + * Setup LPIs for the VM after the guest has initialized the GIC. Yes, + * this is weird to be doing in userspace, but creating ITS translations + * requires allocating an ITT for every device. + */ + setup_rdists_for_lpis(); + setup_its_mappings(); + invalidate_all_rdists(); + + clock_gettime(CLOCK_MONOTONIC, &start); + pthread_barrier_wait(&test_start_barrier); + + for (i = 0; i < nr_devices; i++) + pthread_join(lpi_threads[i], NULL); + + delta = timespec_elapsed(start); + write_guest_global(vm, request_vcpus_stop, true); + + for (i = 0; i < nr_vcpus; i++) + pthread_join(vcpu_threads[i], NULL); + + report_stats(delta); +} + +static void setup_vm(void) +{ + int i; + + vcpus = malloc(nr_vcpus * sizeof(struct kvm_vcpu)); + TEST_ASSERT(vcpus, "Failed to allocate vCPU array"); + + vm = vm_create_with_vcpus(nr_vcpus, guest_code, vcpus); + + vm_init_descriptor_tables(vm); + for (i = 0; i < nr_vcpus; i++) + vcpu_init_descriptor_tables(vcpus[i]); + + vm_install_exception_handler(vm, VECTOR_IRQ_CURRENT, guest_irq_handler); + + setup_memslot(); + + setup_gic(); + + /* gic_init() demands the number of vCPUs in the VM */ + sync_global_to_guest(vm, nr_vcpus); +} + +static void destroy_vm(void) +{ + vgic_its_destroy(its); + close(gic_fd); + kvm_vm_free(vm); + free(vcpus); +} + +static void pr_usage(const char *name) +{ + pr_info("%s [-v NR_VCPUS] [-d NR_DEVICES] [-e NR_EVENTS] [-i ITERS] -h\n", name); + pr_info(" -v:\tnumber of vCPUs (default: %u)\n", nr_vcpus); + pr_info(" -d:\tnumber of devices (default: %u)\n", nr_devices); + pr_info(" -e:\tnumber of event IDs per device (default: %u)\n", nr_event_ids); + pr_info(" -i:\tnumber of iterations (default: %lu)\n", nr_iterations); +} + +int main(int argc, char **argv) +{ + u32 nr_threads; + int c; + + while ((c = getopt(argc, argv, "hv:d:e:i:")) != -1) { + switch (c) { + case 'v': + nr_vcpus = atoi(optarg); + break; + case 'd': + nr_devices = atoi(optarg); + break; + case 'e': + nr_event_ids = atoi(optarg); + break; + case 'i': + nr_iterations = strtoul(optarg, NULL, 0); + break; + case 'h': + default: + pr_usage(argv[0]); + return 1; + } + } + + nr_threads = nr_vcpus + nr_devices; + if (nr_threads > get_nprocs()) + pr_info("WARNING: running %u threads on %d CPUs; performance is degraded.\n", + nr_threads, get_nprocs()); + + setup_vm(); + + run_test(); + + destroy_vm(); + + return 0; +}