From patchwork Wed Feb 21 05:42:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 203931 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:693c:2685:b0:108:e6aa:91d0 with SMTP id mn5csp849581dyc; Tue, 20 Feb 2024 21:44:28 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCUTJS7IlosFQBjdrbWCg9DTOCcx745sirlN12TG2VPRl0BsOxl/nC3CrHnRi1Y3HuTpmf4Ee5G8fQF5h7lBSuiAuqWuWw== X-Google-Smtp-Source: AGHT+IHXcH2ZTdx/aTdtml0cq/rdbRaiyQ/7mpfWDhwIJfcQmSMX+2g8E6JA9Op19J4X5QOr/DPR X-Received: by 2002:a05:622a:13cc:b0:42c:755a:2365 with SMTP id p12-20020a05622a13cc00b0042c755a2365mr23536262qtk.67.1708494268434; Tue, 20 Feb 2024 21:44:28 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708494268; cv=pass; d=google.com; s=arc-20160816; b=gN8ybyKcko15eH7EIQq9AoSm9XuKe1TXwiPqEUkojHS7k/1wQJyAdS5Nm8jVrWCKaz +1A5V0+VqgmVI243RwQFBXpdzYYiVr2NRDbltogcA9TJ5khwQ1t3xjI4B3pVKgV1wSlh xaiAik0vQ7IddaoxlKc+orMW8tnBuoEe2D8OBPqRehA7/G/l/zxnb+H0OpAeVE7zwpqv 2hEN85Iy7iH693sp1wBwEd9WJ4BrRAxowmaGrMWDj7S4yIe4g692PI1xCAWClBGBRQWZ A/pjYxVBhT9aCTqMgBOy7XOz/a407OduqkZEjR1H3xFt2I3/dGjRyAgEaydyPtOdiHt7 tURg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=et9uSczKRmO7T/9bBY6kYRtqxJPq32jgEhjumGlg3NA=; fh=kCIhDCwfhVa+1moYcBlLNuY7yX/BfELZCoRk1wSf3jY=; b=YtAV6ZqHHIb35vfA6/O2RUzDY00ORjopbL8oPIKzVo+IyIv8Ja1T/U4SX1vdeyMIc5 0JlXnPMFpFHl8M04gJ7qy+W2AVrAKq8tHIw248HRM5Yph2n4UNrSRbcgiWPgVcbJGg2w bE2yo+KO4FTYGgH8jro6Ofy93XESFjYAQ/sTsunz5Ge1o22ZIrrsHOF5mXO2cN4jbHzX 4+WzpHt/mg8Juxa+kopmPsKat4VYosl8LWahqckkqEEt0W4WcWqdrDs5JCGcvGMJgvtM qwnzSHDS3xPn9NGAMMtBJ8Jv5sYr3gTaKx/mWQ/48g6U8nhqmJzLdYSF27zamgr9I61u m2Iw==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=fYgh58Q8; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-74099-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-74099-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id c11-20020a05620a200b00b00785d923c30fsi10167823qka.368.2024.02.20.21.44.28 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Feb 2024 21:44:28 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-74099-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=fYgh58Q8; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-74099-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-74099-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 3CDBF1C23290 for ; Wed, 21 Feb 2024 05:44:28 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 678923B2AD; Wed, 21 Feb 2024 05:43:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="fYgh58Q8" Received: from out-188.mta0.migadu.com (out-188.mta0.migadu.com [91.218.175.188]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6BFF839FC3 for ; Wed, 21 Feb 2024 05:43:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.188 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708494204; cv=none; b=qs+Q49YEpFaAHHuUWVXJi12MrFxeSj+6xj9M5VMW5Ajw8Q0mjdCZBDB3upFpcB5t2HqALTRB9z3kbWDFJ0S4c5tvDwSchgu2GZY/4UZuDHRMq+Y1aGtZfaall0lfmjLgmd+KnAedkP3JlskUQU926mVU6fzqdI0VjuGjKisWGO4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708494204; c=relaxed/simple; bh=ZmOXn9U2kRyvale593aVJAMMtGmI9I8PRxYMMaYh3Hg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ifMRX4ODAoZtxcUU2iWz7LRJrQVYPwtOaYWbTFizoU0/f//1SE0FYJRYwJWIrWe3LyX7hk6kTytRZxqDt1NJi9k49U6xCOZWYDxE698Nt+UE/C28rX1tkJVHXvSYm1+piM9/2Na+A2WvPp0kgsz4r2qahIGjIcvXrLGfKeJKDy8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=fYgh58Q8; arc=none smtp.client-ip=91.218.175.188 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1708494200; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=et9uSczKRmO7T/9bBY6kYRtqxJPq32jgEhjumGlg3NA=; b=fYgh58Q8CQGX1i5n2girseOonhB0gU5edl8fQi1hJyftVFoeOzvXH8l1LV7JLbwVC9SaTm uhdw/yl5tVkIRDRK3aavw40TR/wba9NxQ7Az0YFPNvpMk9WWupqsLLCOCRFcdozxDrNSHW ajsCY9K/5csZvg4mikj3wx/4XhXfPK8= From: Oliver Upton To: kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , linux-kernel@vger.kernel.org, Eric Auger , Oliver Upton Subject: [PATCH v4 01/10] KVM: arm64: vgic: Store LPIs in an xarray Date: Wed, 21 Feb 2024 05:42:44 +0000 Message-ID: <20240221054253.3848076-2-oliver.upton@linux.dev> In-Reply-To: <20240221054253.3848076-1-oliver.upton@linux.dev> References: <20240221054253.3848076-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791486086273667425 X-GMAIL-MSGID: 1791486086273667425 Using a linked-list for LPIs is less than ideal as it of course requires iterative searches to find a particular entry. An xarray is a better data structure for this use case, as it provides faster searches and can still handle a potentially sparse range of INTID allocations. Start by storing LPIs in an xarray, punting usage of the xarray to a subsequent change. The observant among you will notice that we added yet another lock to the chain of locking order rules; document the ordering of the xa_lock. Don't worry, we'll get rid of the lpi_list_lock one day... Signed-off-by: Oliver Upton --- arch/arm64/kvm/vgic/vgic-init.c | 3 +++ arch/arm64/kvm/vgic/vgic-its.c | 16 ++++++++++++++++ arch/arm64/kvm/vgic/vgic.c | 4 +++- include/kvm/arm_vgic.h | 2 ++ 4 files changed, 24 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kvm/vgic/vgic-init.c b/arch/arm64/kvm/vgic/vgic-init.c index e949e1d0fd9f..411719053107 100644 --- a/arch/arm64/kvm/vgic/vgic-init.c +++ b/arch/arm64/kvm/vgic/vgic-init.c @@ -56,6 +56,7 @@ void kvm_vgic_early_init(struct kvm *kvm) INIT_LIST_HEAD(&dist->lpi_list_head); INIT_LIST_HEAD(&dist->lpi_translation_cache); raw_spin_lock_init(&dist->lpi_list_lock); + xa_init_flags(&dist->lpi_xa, XA_FLAGS_LOCK_IRQ); } /* CREATION */ @@ -366,6 +367,8 @@ static void kvm_vgic_dist_destroy(struct kvm *kvm) if (vgic_supports_direct_msis(kvm)) vgic_v4_teardown(kvm); + + xa_destroy(&dist->lpi_xa); } static void __kvm_vgic_vcpu_destroy(struct kvm_vcpu *vcpu) diff --git a/arch/arm64/kvm/vgic/vgic-its.c b/arch/arm64/kvm/vgic/vgic-its.c index e2764d0ffa9f..fb2d3c356984 100644 --- a/arch/arm64/kvm/vgic/vgic-its.c +++ b/arch/arm64/kvm/vgic/vgic-its.c @@ -52,6 +52,12 @@ static struct vgic_irq *vgic_add_lpi(struct kvm *kvm, u32 intid, if (!irq) return ERR_PTR(-ENOMEM); + ret = xa_reserve_irq(&dist->lpi_xa, intid, GFP_KERNEL_ACCOUNT); + if (ret) { + kfree(irq); + return ERR_PTR(ret); + } + INIT_LIST_HEAD(&irq->lpi_list); INIT_LIST_HEAD(&irq->ap_list); raw_spin_lock_init(&irq->irq_lock); @@ -86,12 +92,22 @@ static struct vgic_irq *vgic_add_lpi(struct kvm *kvm, u32 intid, goto out_unlock; } + ret = xa_err(xa_store(&dist->lpi_xa, intid, irq, 0)); + if (ret) { + xa_release(&dist->lpi_xa, intid); + kfree(irq); + goto out_unlock; + } + list_add_tail(&irq->lpi_list, &dist->lpi_list_head); dist->lpi_list_count++; out_unlock: raw_spin_unlock_irqrestore(&dist->lpi_list_lock, flags); + if (ret) + return ERR_PTR(ret); + /* * We "cache" the configuration table entries in our struct vgic_irq's. * However we only have those structs for mapped IRQs, so we read in diff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c index db2a95762b1b..92e4d2fea365 100644 --- a/arch/arm64/kvm/vgic/vgic.c +++ b/arch/arm64/kvm/vgic/vgic.c @@ -30,7 +30,8 @@ struct vgic_global kvm_vgic_global_state __ro_after_init = { * its->its_lock (mutex) * vgic_cpu->ap_list_lock must be taken with IRQs disabled * kvm->lpi_list_lock must be taken with IRQs disabled - * vgic_irq->irq_lock must be taken with IRQs disabled + * vgic_dist->lpi_xa.xa_lock must be taken with IRQs disabled + * vgic_irq->irq_lock must be taken with IRQs disabled * * As the ap_list_lock might be taken from the timer interrupt handler, * we have to disable IRQs before taking this lock and everything lower @@ -131,6 +132,7 @@ void __vgic_put_lpi_locked(struct kvm *kvm, struct vgic_irq *irq) return; list_del(&irq->lpi_list); + xa_erase(&dist->lpi_xa, irq->intid); dist->lpi_list_count--; kfree(irq); diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h index 8cc38e836f54..795b35656b54 100644 --- a/include/kvm/arm_vgic.h +++ b/include/kvm/arm_vgic.h @@ -13,6 +13,7 @@ #include #include #include +#include #include #include #include @@ -275,6 +276,7 @@ struct vgic_dist { /* Protects the lpi_list and the count value below. */ raw_spinlock_t lpi_list_lock; + struct xarray lpi_xa; struct list_head lpi_list_head; int lpi_list_count; From patchwork Wed Feb 21 05:42:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 203934 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:693c:2685:b0:108:e6aa:91d0 with SMTP id mn5csp850171dyc; Tue, 20 Feb 2024 21:46:31 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCVkAL3TB/0vgoc+z8gXAWliwpuyfQvvcmOJ6NocF7b0S3YbcX0Ki3/vnZDeQkTHzV2c9Qhc65xlaALpN6i9ZiThd7D4SA== X-Google-Smtp-Source: AGHT+IGbycoCJ0WlUyOvDtyMUQgEORjQ5zbWCUdGtFz5+XF+r/xgnXMLe5M5mfKy++henk3JsJru X-Received: by 2002:a17:90a:f190:b0:299:3f16:184 with SMTP id bv16-20020a17090af19000b002993f160184mr13495891pjb.34.1708494391616; Tue, 20 Feb 2024 21:46:31 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708494391; cv=pass; d=google.com; s=arc-20160816; b=E81vE/nnzslzclZCVNfR1EeTYAnV2YY2XefMH2TZ2M3ei07ggdo6jUDtSIoPFPD/8q PY2e5U5UP2lzhqtxTFXgFJmOvglpKkLKPUEC8BaSNPiM5wGmaqbujGTENtbdaZok300M 0zUB0gh3s/mq84528Xsy4pkSQUTUkB/aHBsrK9SZkOZLdJJveQZElFeQtGwHRMc3ZWk8 0D4FDiQITMssOthUD+YWgri1b6rj1DREKOgWQn2GqXmle8gpWJ7YB6ScsyNGg8So9lc/ MHsHmP3yty4kSem7Aad90wh/R5q5D+x94bCB5ZvxMpN/AGyY0hRrmKN5RcE9778Wl4RW IYlA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=mj321azO2g3V6z75I7i/wkEQoxgT4KRGdjlI33lsjxQ=; fh=kCIhDCwfhVa+1moYcBlLNuY7yX/BfELZCoRk1wSf3jY=; b=R8j0Ml4+FE2Ae2y4/zpbhynmqoIhErqLm98dfo/VxMKKd+8XgCYsf58u1YNBEpwTbV vsFCoj160BaMhASHxXnoS8UlVBU0febdNuFfxXrA21LWbq0XVJC8ymeB3+sKGopAgpo9 tvi5jdGG9e7QwF0HdgT1jJ/mdRTo7PVlwbsxUo3rSL8JOPZTXqHiPNcVZj1/8ltPmf+2 t14Ykh/m5gY7DiiE4qoj/q3BoRLHcKidF+MUMb7nazzMEYUJJAHOvs79PDZdaONQzMAv RUa9ZP+AEz7FtE2pL83BWDEysa2zUUfJKrvLUgFd7ZyJDFAnx2d/Y4A8Zj1cmbZgw7dN 1kyA==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=mbz+iDp+; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-74100-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-74100-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [147.75.48.161]) by mx.google.com with ESMTPS id np6-20020a17090b4c4600b0029a1aaefe30si362159pjb.188.2024.02.20.21.46.31 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Feb 2024 21:46:31 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-74100-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=mbz+iDp+; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-74100-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-74100-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 2A7B0B233B0 for ; Wed, 21 Feb 2024 05:44:43 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C3CFF3A1AF; Wed, 21 Feb 2024 05:43:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="mbz+iDp+" Received: from out-188.mta0.migadu.com (out-188.mta0.migadu.com [91.218.175.188]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 309B13A8E5 for ; Wed, 21 Feb 2024 05:43:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.188 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708494205; cv=none; b=IeOEdz1BPKJNez76JfVC+kfLZSRE0tsnhM6fdepxHP2EG5k7T9XlfHMKr13iorHrLTzNvK8gWTPf0jqJBX/YCqjuPFEulUgnZNoVz2K96vVuzBltiAWb1WH5GxUu0CyBxh6yDQSAwVu53eD0/q9CTdxsNpoZ8cbhJ4GHfPedfjo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708494205; c=relaxed/simple; bh=PdEyg5rDXT1t31oqeD150AOaybowNPAh6qErSbtdqXs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=u2eIgXgj0pew8GYGXYsi+0DSSjpSK83x/ZFm9tH3GAnt0TSbqtTd0C3YVKRBXoRWVk3cMpV6TcgFIba+mj+AbI9z5CiWouuvvQAUi/COdU+8bis85RA1SIGYF2bHM9ahvdBcerkI1+dPS2ZYZ874bqHBp2i/quz7nvKDso8Vj0Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=mbz+iDp+; arc=none smtp.client-ip=91.218.175.188 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1708494202; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mj321azO2g3V6z75I7i/wkEQoxgT4KRGdjlI33lsjxQ=; b=mbz+iDp+sw32XYqQJv3DxB3aud7oUieMyg8J1rhMwXBs5ROt8iTNNC8UxSHnazQzP+3+B0 twdyKRXR7PasSyIRSD+4uYe2Gx9jg4AdcRBXUONHgEXY4DGuz5EnLw/Za+daWMM7sKgN9V KaOcci7KlpsTEBdUrUkbceIT/gbzOPE= From: Oliver Upton To: kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , linux-kernel@vger.kernel.org, Eric Auger , Oliver Upton Subject: [PATCH v4 02/10] KVM: arm64: vgic: Use xarray to find LPI in vgic_get_lpi() Date: Wed, 21 Feb 2024 05:42:45 +0000 Message-ID: <20240221054253.3848076-3-oliver.upton@linux.dev> In-Reply-To: <20240221054253.3848076-1-oliver.upton@linux.dev> References: <20240221054253.3848076-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791486214977382930 X-GMAIL-MSGID: 1791486214977382930 Iterating over the LPI linked-list is less than ideal when the desired index is already known. Use the INTID to index the LPI xarray instead. Signed-off-by: Oliver Upton --- arch/arm64/kvm/vgic/vgic.c | 19 +++++-------------- 1 file changed, 5 insertions(+), 14 deletions(-) diff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c index 92e4d2fea365..7d17dbc8f5dc 100644 --- a/arch/arm64/kvm/vgic/vgic.c +++ b/arch/arm64/kvm/vgic/vgic.c @@ -55,8 +55,9 @@ struct vgic_global kvm_vgic_global_state __ro_after_init = { */ /* - * Iterate over the VM's list of mapped LPIs to find the one with a - * matching interrupt ID and return a reference to the IRQ structure. + * Index the VM's xarray of mapped LPIs and return a reference to the IRQ + * structure. The caller is expected to call vgic_put_irq() later once it's + * finished with the IRQ. */ static struct vgic_irq *vgic_get_lpi(struct kvm *kvm, u32 intid) { @@ -66,20 +67,10 @@ static struct vgic_irq *vgic_get_lpi(struct kvm *kvm, u32 intid) raw_spin_lock_irqsave(&dist->lpi_list_lock, flags); - list_for_each_entry(irq, &dist->lpi_list_head, lpi_list) { - if (irq->intid != intid) - continue; - - /* - * This increases the refcount, the caller is expected to - * call vgic_put_irq() later once it's finished with the IRQ. - */ + irq = xa_load(&dist->lpi_xa, intid); + if (irq) vgic_get_irq_kref(irq); - goto out_unlock; - } - irq = NULL; -out_unlock: raw_spin_unlock_irqrestore(&dist->lpi_list_lock, flags); return irq; From patchwork Wed Feb 21 05:42:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 203936 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:693c:2685:b0:108:e6aa:91d0 with SMTP id mn5csp850246dyc; Tue, 20 Feb 2024 21:46:43 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCVSp8B95W7/7n4KB/D2wPLWLx8vAqVc1A/PMXBznyO2vWW8+6e0b47IalNayyldIKJgJrezwkYS9dnO8b1/HQYOolo5lQ== X-Google-Smtp-Source: AGHT+IFy865+gQf0LuVlVlH/OwZsPGaY8lgcKAYpmkMA9PD+wTMOjFHrBeuQu+8lliTOr9+MmLUB X-Received: by 2002:a05:6358:5719:b0:17b:56f6:ab32 with SMTP id a25-20020a056358571900b0017b56f6ab32mr1888140rwf.4.1708494403648; Tue, 20 Feb 2024 21:46:43 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708494403; cv=pass; d=google.com; s=arc-20160816; b=NXrdrc/yVNdmiHR5TlbIn+z3Nk1jf+feNvrRUxE1F6Z2MTkKuqXCIeSnDPkdFwYJDH hropihzQRpsURRbYy3EdCVCO4stasr+k9FNRQ7jTvtV3WspU83VIpcyGvUFdBpjt103Y K5ejM8JzgrfsD8Qnt+d1XsaZUufl9BzA8FMjUIEdXnkKLq8JvP9Xi0JscA3W4BW6Ta6Z GXihs5p4VXb5MiMa15nkuQ0friXCUSW6VdiTUMmLek2kOhsr33i2r12xL3O9P0DHcwzg Y+ah3w4oHN1pmERm77+bfzPg0o0/nHs0puRiMkV0eP+zr0GzGxHiWfpiWC3C8YORePQR NYKg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=p7/gBwWxFbIZwFC+F2czZFaZoLMKZvcePyaMRonjua4=; fh=kCIhDCwfhVa+1moYcBlLNuY7yX/BfELZCoRk1wSf3jY=; b=p31mSahdbp62qPyb2LWcp64M7c1SxXVwj45HsdbeTL7ffPcut3H48ri9tkLvcbYhjL Cae/8PxFT9dtVlFWNbbZ3RzwzckcJgOg3b/wK2J3CXBsIHAQQyRWTnFaTLjP1hOj00MC xL0ZZoQsJcUorq1KgYsu9eoQafuVXko0+hJARLgwQ/451hIZSORaOW14CLaF5DfTbMj+ FCEc3Ykk5tE8CiOcrT+6FGAv9TldlwuaE0WZxr4prfEKb/5BZnh9/jKwHpnJGwDuTQ5A pnoqthfdXf36N63yb/WTDzj7gvc59eC11cpl/RCUQQHhw1aO3oFnOIjXXautc01V2JEX ZmAA==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=GL2D5v0S; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-74101-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-74101-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id fj36-20020a056a003a2400b006d3c35096dcsi7486996pfb.320.2024.02.20.21.46.43 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Feb 2024 21:46:43 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-74101-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=GL2D5v0S; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-74101-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-74101-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 7D8FFB24220 for ; Wed, 21 Feb 2024 05:45:01 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C33C43C068; Wed, 21 Feb 2024 05:43:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="GL2D5v0S" Received: from out-182.mta0.migadu.com (out-182.mta0.migadu.com [91.218.175.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2882E3B193 for ; Wed, 21 Feb 2024 05:43:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708494208; cv=none; b=EOXS9Ctp/1boTXuUXevKyKHaCiJGS/XEVqz7gRAyAxTCopRvYeijz/ZIqXJc56S2GRFxFq1E1RTjRiXaLhAT5rGPOD828BQkowkzWHDYtaWrH+FsKHVxVozM43ftjx3Ey95zizCKXzQI1Hncau6Lb831HTdd497awuvMKRL5NfI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708494208; c=relaxed/simple; bh=RxF9w33LgnXLTuXV5O9RLdzK9dXUqE5MSbsOmdxxwVA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=LCtyuJ9AtPEq0qAr2LfMpcymzPEBhIb8DTh11NsRtvGBErFjvtc0LLrOwikdEdJEcr5pYOZIH8AIMDWh/87g2K6hL3Lsl+fwo02TWh7PlL6UI0ozWHMTF8ZeJY9SvJTICvfDey8912UdZLwLlukhiFhkK2SYw5U2sWV2iTNBVPE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=GL2D5v0S; arc=none smtp.client-ip=91.218.175.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1708494204; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=p7/gBwWxFbIZwFC+F2czZFaZoLMKZvcePyaMRonjua4=; b=GL2D5v0SC8vLCvny+w0lysq1H7+eVYaJPjgBXHDAX31SpwfBHeusS509F4eACvmVOuX2oW YZQOwMa3/zFkcg+V/vPyOoDP+9UuhCmyIp2XJVxwtXOjQ3Cs4Ip3o/VYERU1asvwz4SbZF cnqOWEIcewTFCBKCM25xdIFUmrmg9AE= From: Oliver Upton To: kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , linux-kernel@vger.kernel.org, Eric Auger , Oliver Upton Subject: [PATCH v4 03/10] KVM: arm64: vgic-v3: Iterate the xarray to find pending LPIs Date: Wed, 21 Feb 2024 05:42:46 +0000 Message-ID: <20240221054253.3848076-4-oliver.upton@linux.dev> In-Reply-To: <20240221054253.3848076-1-oliver.upton@linux.dev> References: <20240221054253.3848076-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791486228254562381 X-GMAIL-MSGID: 1791486228254562381 Start walking the LPI xarray to find pending LPIs in preparation for the removal of the LPI linked-list. Note that the 'basic' iterator is chosen here as each iteration needs to drop the xarray read lock (RCU) as reads/writes to guest memory can potentially block. Signed-off-by: Oliver Upton --- arch/arm64/kvm/vgic/vgic-v3.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kvm/vgic/vgic-v3.c b/arch/arm64/kvm/vgic/vgic-v3.c index 9465d3706ab9..4ea3340786b9 100644 --- a/arch/arm64/kvm/vgic/vgic-v3.c +++ b/arch/arm64/kvm/vgic/vgic-v3.c @@ -380,6 +380,7 @@ int vgic_v3_save_pending_tables(struct kvm *kvm) struct vgic_irq *irq; gpa_t last_ptr = ~(gpa_t)0; bool vlpi_avail = false; + unsigned long index; int ret = 0; u8 val; @@ -396,7 +397,7 @@ int vgic_v3_save_pending_tables(struct kvm *kvm) vlpi_avail = true; } - list_for_each_entry(irq, &dist->lpi_list_head, lpi_list) { + xa_for_each(&dist->lpi_xa, index, irq) { int byte_offset, bit_nr; struct kvm_vcpu *vcpu; gpa_t pendbase, ptr; From patchwork Wed Feb 21 05:42:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 203932 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:693c:2685:b0:108:e6aa:91d0 with SMTP id mn5csp849798dyc; Tue, 20 Feb 2024 21:45:12 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCVO6D3c+i5NGtWsQwcSwtUCN2GGW2JxwNwJO/2/qlMWXKaRqDeN2Kfw+Hnp3EVW/JcH7xDXpJKnYiftof/QnBgANodxUg== X-Google-Smtp-Source: AGHT+IFixKmbhwTXu7mUZyT3Q6z4V9fbimwcY8FafWaX/hMvfE53FIQgq4IGvxdL5BB0pehxjnhg X-Received: by 2002:a05:6402:26c8:b0:564:c704:ccb6 with SMTP id x8-20020a05640226c800b00564c704ccb6mr4141239edd.29.1708494312494; Tue, 20 Feb 2024 21:45:12 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708494312; cv=pass; d=google.com; s=arc-20160816; b=zO0RrL85Ian7sFNzUAxMdlkX7Ar7hTyF6N9UvkvCk3L4mol0uEajpHyG/kq4jSNpsa tANJ/akZY2LpSBc74MP4CCwGOHB0yw8GjRQGBOJ44JbYvhRTCjL1wKClkLwo83XJ6XC1 jS+OpWkmjOZGao6K232h51fRJsNabdTEaKjZEFUDD3Y/TewmLPWHTWprXHz+79C3w4xz ds6YO7HW1ypvlCqn1Op/ub1B9kGRhTSrvfgU9vPfRysbVxXKVrrWad5hN7bSOqykw8NI GQ/sB9x2XD1m/GiivAY/WJHVBx1PRjN3swGjmJklSNCY7t91qLkOwM71oLAoZ/c6Xq25 ltJw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=hwGBWvx+wwsJa7xNRdwDH7MJ0Q05E6x3UjwcqNXt4/0=; fh=kCIhDCwfhVa+1moYcBlLNuY7yX/BfELZCoRk1wSf3jY=; b=ZBxy7qiAvq2iNnDVqntunSD7OktYJLoy0ukPtzSwqFn1cUGtLTCkv4jhow3LOO/gat 6U0NhvSiAMSjyzhdnD8q6l/LaIumnmR5gl+dUYQadW8snRjzgl6tO1elg6/fSMskvm3L tLnVvxKcedEUXs34L0mPrKCbMmaT8tQoZplcKNNZRu/oAeoZapqeq8+DgmI0GjSTmW7Z uxISeTJWMnwSl51b35q8k5xrNOmTHb3lFhFFyffLOcVR6kMR66a+88z0vaHkbM/qhSzl 43P6JyycED/cIuj6IJBHtHpsf5XJIM7HVzaWJHEyDowJwsi3PYtR9ELlm+mlxfyIFkSy OOhA==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=MGYDRQmp; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-74102-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-74102-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id e21-20020a056402149500b00564266593b2si3611218edv.629.2024.02.20.21.45.12 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Feb 2024 21:45:12 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-74102-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=MGYDRQmp; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-74102-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-74102-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 217F01F24DEE for ; Wed, 21 Feb 2024 05:45:12 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 5AAC43C47C; Wed, 21 Feb 2024 05:43:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="MGYDRQmp" Received: from out-176.mta0.migadu.com (out-176.mta0.migadu.com [91.218.175.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 17CCA39FC9 for ; Wed, 21 Feb 2024 05:43:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708494209; cv=none; b=CE3pPC6e7oWRuydgyYrWxt8u0OreFbrDqazgkhcoF18dVTzRwy4UbR/96/Ukqtgr+DzkqeBZ6zVASoqUaLdI635jQFVV9/spFWD5P/bIV1l0gxav7sR1Rxkai/7twZyOFPx2G1m/sOKnCTtDFVjH+2ofXP6YgVKF+E7Vn/YEL/g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708494209; c=relaxed/simple; bh=egvxRC64pImQWpYJ5H/66l//0W0vYIRaEzxh49R2ggA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=JDXUzHkJbLySEteSPqxuw1cm8XrLhGPDirgTvZrUpUaeWGoZgtaoAAngyUFttoKvE/lcuo9AEK7x8q7l6QIwpc2bfOTun7NJsuJx+KXzkBJvmnv/huXgxt061/Pf6t4Oq5d8ncwhcu2niZ5Js+uZOpgbs50CNFmRhAlQ5khQJpo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=MGYDRQmp; arc=none smtp.client-ip=91.218.175.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1708494206; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hwGBWvx+wwsJa7xNRdwDH7MJ0Q05E6x3UjwcqNXt4/0=; b=MGYDRQmpErICmMJRKOaSwPhNZwcGWG9i3eBvE7S9eAoTsD13nFyL/UaN2lmj6158Yql47k AE2cr3rXzXv4faz9wb7vFc2SwzJUyIy/dLvzFP+PJ69TdIyorSkUwcQq8Iej9FIRHfuPhI PvHsPUfiiZp1eoJHZjh3RHF+HbDCms4= From: Oliver Upton To: kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , linux-kernel@vger.kernel.org, Eric Auger , Oliver Upton Subject: [PATCH v4 04/10] KVM: arm64: vgic-its: Walk the LPI xarray in vgic_copy_lpi_list() Date: Wed, 21 Feb 2024 05:42:47 +0000 Message-ID: <20240221054253.3848076-5-oliver.upton@linux.dev> In-Reply-To: <20240221054253.3848076-1-oliver.upton@linux.dev> References: <20240221054253.3848076-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791486132600152504 X-GMAIL-MSGID: 1791486132600152504 Start iterating the LPI xarray in anticipation of removing the LPI linked-list. Signed-off-by: Oliver Upton --- arch/arm64/kvm/vgic/vgic-its.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kvm/vgic/vgic-its.c b/arch/arm64/kvm/vgic/vgic-its.c index fb2d3c356984..b9874dc04608 100644 --- a/arch/arm64/kvm/vgic/vgic-its.c +++ b/arch/arm64/kvm/vgic/vgic-its.c @@ -327,6 +327,8 @@ static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq, return 0; } +#define GIC_LPI_MAX_INTID ((1 << INTERRUPT_ID_BITS_ITS) - 1) + /* * Create a snapshot of the current LPIs targeting @vcpu, so that we can * enumerate those LPIs without holding any lock. @@ -335,6 +337,7 @@ static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq, int vgic_copy_lpi_list(struct kvm *kvm, struct kvm_vcpu *vcpu, u32 **intid_ptr) { struct vgic_dist *dist = &kvm->arch.vgic; + XA_STATE(xas, &dist->lpi_xa, GIC_LPI_OFFSET); struct vgic_irq *irq; unsigned long flags; u32 *intids; @@ -353,7 +356,9 @@ int vgic_copy_lpi_list(struct kvm *kvm, struct kvm_vcpu *vcpu, u32 **intid_ptr) return -ENOMEM; raw_spin_lock_irqsave(&dist->lpi_list_lock, flags); - list_for_each_entry(irq, &dist->lpi_list_head, lpi_list) { + rcu_read_lock(); + + xas_for_each(&xas, irq, GIC_LPI_MAX_INTID) { if (i == irq_count) break; /* We don't need to "get" the IRQ, as we hold the list lock. */ @@ -361,6 +366,8 @@ int vgic_copy_lpi_list(struct kvm *kvm, struct kvm_vcpu *vcpu, u32 **intid_ptr) continue; intids[i++] = irq->intid; } + + rcu_read_unlock(); raw_spin_unlock_irqrestore(&dist->lpi_list_lock, flags); *intid_ptr = intids; From patchwork Wed Feb 21 05:42:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 203939 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:693c:2685:b0:108:e6aa:91d0 with SMTP id mn5csp850320dyc; Tue, 20 Feb 2024 21:47:04 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCU9szld3mWADYPioO3aXuBsQ/s9JUuYqddv/T994iS0/442+A1ArXgOyfAaphpGBNJ+kvwr5IpS1accZPJKUR9VkwV68g== X-Google-Smtp-Source: AGHT+IGG5LoI2/5H2BI6UNo5fCY07STqhgSXOJnMDJ6QTvNtO7uNYfpDQZKR6MXXEf5wCirgixp2 X-Received: by 2002:a05:6a20:9f0a:b0:1a0:a091:ee97 with SMTP id mk10-20020a056a209f0a00b001a0a091ee97mr11033815pzb.19.1708494424589; Tue, 20 Feb 2024 21:47:04 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708494424; cv=pass; d=google.com; s=arc-20160816; b=TW2C353XOgE8Gr6BJs8cSNsUKg+gfO8F7pj2fCypiInsln2O0MK5q3fPryEqtGGahU o2D39RH+mkR2MQQv8feMC7v78UTLMa6SbWNhgsjlF4SYsKIpTIikUdYf53V7rD/hYBP0 cwV0xOp3ZFrnUWo5rxauSW4MyL0JoyvBOfGEa1+Em8luDxpAJwCeUkQpr4yc3T4U3p4d WMrs/Ap7B7uWJ5/g3tsXOkeHG/Tq2Sv65E2d7oG1AClc9WLJq/na55v0f1KPgee4DicF g99CV+Fj5MorM4tUAvRRml+tEyzachPqRy5rvJdH2YaxXfwM0OofAl84nBd2yw7uymC/ xCSw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=8nd2SXFQ1C1LlktYqUUc4CI4OG59iyq71GzSyfZOU20=; fh=kCIhDCwfhVa+1moYcBlLNuY7yX/BfELZCoRk1wSf3jY=; b=leb1GL2Pb6Y+XdHcqloDm3/7ZseaRpDkhYWZBTqc2scO2MKIej/FO+dgFGi5gRsw0o VyFHNpQmkaDh7PWArkekkm6GfquyhOkORmo/XH/of98jQSk1nVH0m5j2A8AHn2rbmtJU pMYnoNnHQrIB+QVqVNfZJiswBlAntBa/dwYO1k6sL2cAZNeukc1rxGte+GjncCIvjkvd 3riwAbxrL2DtDFH1xS/ReS0cfxBq83vT2T1bX/Oiaeq+fycEFm62M6mHfUrmdPJnU3AA b1ztD2q5jQI6jQwU6bNRDX2fXSLJLVvz8ornnbFN6hNIhcEtICE2MV8k6nEz7ldL1U+P qxng==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=kWM4CnAI; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-74103-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-74103-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id e14-20020a65678e000000b005dc81a6b2absi7374462pgr.792.2024.02.20.21.47.04 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Feb 2024 21:47:04 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-74103-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=kWM4CnAI; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-74103-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-74103-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 06165B2444D for ; Wed, 21 Feb 2024 05:45:30 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C80133C6B3; Wed, 21 Feb 2024 05:43:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="kWM4CnAI" Received: from out-185.mta0.migadu.com (out-185.mta0.migadu.com [91.218.175.185]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 483B93BB47 for ; Wed, 21 Feb 2024 05:43:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.185 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708494212; cv=none; b=UJq65WB7uYXm0IzJTvggq2sytX4SrVfB9Vx15mCFbaU/oYPg75U/Fs95lM1Pls+oYxK17XorP71SorJwUotXcU/cV5Vy2gB9h5VjK0qxlB4TqiN+k+fBV6qbHe32VUUf240eTOU2qdydrteK8DNKUzZVfJ9L299bh5+Sk+lMFPs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708494212; c=relaxed/simple; bh=2nkDTHcHmB33a/QQu7UDTZHWtSxxacimi3tTiZ9a1sQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qXFcI0h8Kid3AswO8iAgnQgGdW1nIz/Mxp4Cn1ELEEfNjmYJpZQcW8sbH9uu2Rvah300aqkoXPAPZuhkBXUuKcVl4PRLpcvgihu3L+tCY800DG+6OI12uKKTPW31HkuJbep3tyuqRbEaGq7hgExbEDGUOoUXsRKF5fG1CBkzdns= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=kWM4CnAI; arc=none smtp.client-ip=91.218.175.185 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1708494208; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8nd2SXFQ1C1LlktYqUUc4CI4OG59iyq71GzSyfZOU20=; b=kWM4CnAIhzzDn6hcA5UWEWtiSn+E/I2Euqe4c3srTEF4gOULTNs5ApWtlcLfPBFH8QZZ7I 3Zvffct7NXAKoZXaEN6FPXMNl+/PinjhM/iFTxpVAUNvJCWvLZv9aJMYYAbWSXFDcfTRsh dCaDPDNwqHphtjV5ajguseu6IlopyMY= From: Oliver Upton To: kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , linux-kernel@vger.kernel.org, Eric Auger , Oliver Upton Subject: [PATCH v4 05/10] KVM: arm64: vgic: Get rid of the LPI linked-list Date: Wed, 21 Feb 2024 05:42:48 +0000 Message-ID: <20240221054253.3848076-6-oliver.upton@linux.dev> In-Reply-To: <20240221054253.3848076-1-oliver.upton@linux.dev> References: <20240221054253.3848076-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791486250017625668 X-GMAIL-MSGID: 1791486250017625668 All readers of LPI configuration have been transitioned to use the LPI xarray. Get rid of the linked-list altogether. Signed-off-by: Oliver Upton --- arch/arm64/kvm/vgic/vgic-init.c | 1 - arch/arm64/kvm/vgic/vgic-its.c | 8 ++------ arch/arm64/kvm/vgic/vgic.c | 1 - include/kvm/arm_vgic.h | 2 -- 4 files changed, 2 insertions(+), 10 deletions(-) diff --git a/arch/arm64/kvm/vgic/vgic-init.c b/arch/arm64/kvm/vgic/vgic-init.c index 411719053107..e25672d6e846 100644 --- a/arch/arm64/kvm/vgic/vgic-init.c +++ b/arch/arm64/kvm/vgic/vgic-init.c @@ -53,7 +53,6 @@ void kvm_vgic_early_init(struct kvm *kvm) { struct vgic_dist *dist = &kvm->arch.vgic; - INIT_LIST_HEAD(&dist->lpi_list_head); INIT_LIST_HEAD(&dist->lpi_translation_cache); raw_spin_lock_init(&dist->lpi_list_lock); xa_init_flags(&dist->lpi_xa, XA_FLAGS_LOCK_IRQ); diff --git a/arch/arm64/kvm/vgic/vgic-its.c b/arch/arm64/kvm/vgic/vgic-its.c index b9874dc04608..3d0208162bcd 100644 --- a/arch/arm64/kvm/vgic/vgic-its.c +++ b/arch/arm64/kvm/vgic/vgic-its.c @@ -58,7 +58,6 @@ static struct vgic_irq *vgic_add_lpi(struct kvm *kvm, u32 intid, return ERR_PTR(ret); } - INIT_LIST_HEAD(&irq->lpi_list); INIT_LIST_HEAD(&irq->ap_list); raw_spin_lock_init(&irq->irq_lock); @@ -74,10 +73,8 @@ static struct vgic_irq *vgic_add_lpi(struct kvm *kvm, u32 intid, * There could be a race with another vgic_add_lpi(), so we need to * check that we don't add a second list entry with the same LPI. */ - list_for_each_entry(oldirq, &dist->lpi_list_head, lpi_list) { - if (oldirq->intid != intid) - continue; - + oldirq = xa_load(&dist->lpi_xa, intid); + if (oldirq) { /* Someone was faster with adding this LPI, lets use that. */ kfree(irq); irq = oldirq; @@ -99,7 +96,6 @@ static struct vgic_irq *vgic_add_lpi(struct kvm *kvm, u32 intid, goto out_unlock; } - list_add_tail(&irq->lpi_list, &dist->lpi_list_head); dist->lpi_list_count++; out_unlock: diff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c index 7d17dbc8f5dc..6240faab0127 100644 --- a/arch/arm64/kvm/vgic/vgic.c +++ b/arch/arm64/kvm/vgic/vgic.c @@ -122,7 +122,6 @@ void __vgic_put_lpi_locked(struct kvm *kvm, struct vgic_irq *irq) if (!kref_put(&irq->refcount, vgic_irq_release)) return; - list_del(&irq->lpi_list); xa_erase(&dist->lpi_xa, irq->intid); dist->lpi_list_count--; diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h index 795b35656b54..aeff363e3ba6 100644 --- a/include/kvm/arm_vgic.h +++ b/include/kvm/arm_vgic.h @@ -117,7 +117,6 @@ struct irq_ops { struct vgic_irq { raw_spinlock_t irq_lock; /* Protects the content of the struct */ - struct list_head lpi_list; /* Used to link all LPIs together */ struct list_head ap_list; struct kvm_vcpu *vcpu; /* SGIs and PPIs: The VCPU @@ -277,7 +276,6 @@ struct vgic_dist { /* Protects the lpi_list and the count value below. */ raw_spinlock_t lpi_list_lock; struct xarray lpi_xa; - struct list_head lpi_list_head; int lpi_list_count; /* LPI translation cache */ From patchwork Wed Feb 21 05:42:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 203933 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:693c:2685:b0:108:e6aa:91d0 with SMTP id mn5csp850103dyc; Tue, 20 Feb 2024 21:46:17 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCX6qCHf1vo7gwSo7F4JcsMdJ+OIWxN0FPYoABdXpWcEzi6ODaYKvxuUiEGssfxfWvq1UgGrxCtO5HpoKHGLOgajpuAIyg== X-Google-Smtp-Source: AGHT+IFBXN6//8J33tE1aw97Kx54YIFL5/XHkAeDfN8XWOxo9YFFkc9A6dcuOgZ6XDZcXjKMnhBT X-Received: by 2002:a5b:e88:0:b0:dcc:58ed:6ecc with SMTP id z8-20020a5b0e88000000b00dcc58ed6eccmr16523660ybr.41.1708494377316; Tue, 20 Feb 2024 21:46:17 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708494377; cv=pass; d=google.com; s=arc-20160816; b=j0PqbR/5jCwKcuK7P93JuBOw1lrDRVCE53vOnio5QX8InSgZleMzWl0gJw+0FpKQQ6 pD5dTI8KkTV0Nx0yUdWTJX+mUSEoXJcc22DuY6Sa+IjHxtp25HkFOjAurqExZti9f0KV R/183TKQC5fgqsO7scDeXR1XGOtlflYdYrSCKUoRoRLUKH5jjuamH5mE5n9sLhcZFYwS vsaF3knvnugZgM9IgRjFm1G0BUtTh7afKz/hk9QPHCUk/WUQvFEKtVisDJ3p/Yv/P5oN Esi1dhG/OHwkm/0m1XiqyK1oGKqhfY5XIbxnqkjH/uxYofNUiwaZsxm990a7m9Xkgc6p FymA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=OSxryl4GJz7jglvRnwO6HB5bazTdcs2fEdVdtMmeB8s=; fh=kCIhDCwfhVa+1moYcBlLNuY7yX/BfELZCoRk1wSf3jY=; b=aaTNW13wltCVorj1AXFvlF5nHgRA65YZwddHWTW+8Jxbl+OyxkVETeGLbyyM8jsVM1 LBgsUX5RN5o3Eq0ArNrhETrKCV61c68lYJPPbsvOsDmkatvWc3cB0Rz1W4XrrnnWOiIv Ma+FQPCd3G7xbK90agMOiDFC6GbpEmVP8bsbxHnf5l97ps8UWQ7duLmTRjbsH8x9BCDe dFIME5set6rbTlU3ZmPmyltv3m01BLYFf5ImK7xFrCB27DOnzfPkxJznpS3jnT0pgQ+s 3x703q6B3ys4/Hmsfp8eu5CwZJF+tBUwrBeC5JnCTRRz/PyqlQAsP0h/UNl7UBRMCB1N K4bw==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=JOdXlq8W; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-74104-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-74104-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id t185-20020a6381c2000000b005e0c8027182si6571626pgd.831.2024.02.20.21.46.17 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Feb 2024 21:46:17 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-74104-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=JOdXlq8W; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-74104-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-74104-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 483C028215D for ; Wed, 21 Feb 2024 05:45:40 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 191BE3CF60; Wed, 21 Feb 2024 05:43:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="JOdXlq8W" Received: from out-180.mta0.migadu.com (out-180.mta0.migadu.com [91.218.175.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DE9663A8E5 for ; Wed, 21 Feb 2024 05:43:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708494213; cv=none; b=d+Z4cmhnQhNpublbzzYzu0larUrgdDx1Mib9O0iLJk/I9PWPbOlI6H0pHBJNGR2Avm3GNCjf+vMeVPjvv5FteY+5QnuxwDZnsW64zta9npmz9E/FUqDAU/60dIZ+Tg/T4w+UgXz9fHJOmfafv60S44gt5etiaxtuPp3qtqupjy0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708494213; c=relaxed/simple; bh=i1Z17y1L2zZUqDCMRJDQXDFZa8iLbDZzznncMJH6hZQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=j6csHFl7jWTbDhq2ulvowNe5WvT97dqwLDH3NwDJB/DGm17RnLCxF7ZKaHXoYPkWkuPjK/uboNgPTlVpapS2pWQkPavODgzyeDUak/QU2WHUC1nIbKBUbj4HhDqMaLkl2zkGP9VHXKJRdjRdCiK/fbyRZD2J0x7VZaYuv6jBlwc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=JOdXlq8W; arc=none smtp.client-ip=91.218.175.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1708494210; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OSxryl4GJz7jglvRnwO6HB5bazTdcs2fEdVdtMmeB8s=; b=JOdXlq8WUZfIK0n17nGijHUe5mPeuDqw2JyGKCciNDJMECFQT4kEAYdfH3vwhxjHvWm+Qb pb7a+3tuYFEeojv3/zOuWw6jNPAcD0AK+UZkb4eA8f0unN752nf2as6SvuFCUq9uK7Omv5 pcjH9bPZKe2pu2gtm7P8FjZrjXkjBb0= From: Oliver Upton To: kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , linux-kernel@vger.kernel.org, Eric Auger , Oliver Upton Subject: [PATCH v4 06/10] KVM: arm64: vgic: Use atomics to count LPIs Date: Wed, 21 Feb 2024 05:42:49 +0000 Message-ID: <20240221054253.3848076-7-oliver.upton@linux.dev> In-Reply-To: <20240221054253.3848076-1-oliver.upton@linux.dev> References: <20240221054253.3848076-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791486199900938233 X-GMAIL-MSGID: 1791486199900938233 Switch to using atomics for LPI accounting, allowing vgic_irq references to be dropped in parallel. Signed-off-by: Oliver Upton --- arch/arm64/kvm/vgic/vgic-debug.c | 2 +- arch/arm64/kvm/vgic/vgic-its.c | 4 ++-- arch/arm64/kvm/vgic/vgic.c | 2 +- include/kvm/arm_vgic.h | 4 ++-- 4 files changed, 6 insertions(+), 6 deletions(-) diff --git a/arch/arm64/kvm/vgic/vgic-debug.c b/arch/arm64/kvm/vgic/vgic-debug.c index 85606a531dc3..389025ce7749 100644 --- a/arch/arm64/kvm/vgic/vgic-debug.c +++ b/arch/arm64/kvm/vgic/vgic-debug.c @@ -149,7 +149,7 @@ static void print_dist_state(struct seq_file *s, struct vgic_dist *dist) seq_printf(s, "vgic_model:\t%s\n", v3 ? "GICv3" : "GICv2"); seq_printf(s, "nr_spis:\t%d\n", dist->nr_spis); if (v3) - seq_printf(s, "nr_lpis:\t%d\n", dist->lpi_list_count); + seq_printf(s, "nr_lpis:\t%d\n", atomic_read(&dist->lpi_count)); seq_printf(s, "enabled:\t%d\n", dist->enabled); seq_printf(s, "\n"); diff --git a/arch/arm64/kvm/vgic/vgic-its.c b/arch/arm64/kvm/vgic/vgic-its.c index 3d0208162bcd..0be3c33676c3 100644 --- a/arch/arm64/kvm/vgic/vgic-its.c +++ b/arch/arm64/kvm/vgic/vgic-its.c @@ -96,7 +96,7 @@ static struct vgic_irq *vgic_add_lpi(struct kvm *kvm, u32 intid, goto out_unlock; } - dist->lpi_list_count++; + atomic_inc(&dist->lpi_count); out_unlock: raw_spin_unlock_irqrestore(&dist->lpi_list_lock, flags); @@ -346,7 +346,7 @@ int vgic_copy_lpi_list(struct kvm *kvm, struct kvm_vcpu *vcpu, u32 **intid_ptr) * command). If coming from another path (such as enabling LPIs), * we must be careful not to overrun the array. */ - irq_count = READ_ONCE(dist->lpi_list_count); + irq_count = atomic_read(&dist->lpi_count); intids = kmalloc_array(irq_count, sizeof(intids[0]), GFP_KERNEL_ACCOUNT); if (!intids) return -ENOMEM; diff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c index 6240faab0127..15dbd17b3a9e 100644 --- a/arch/arm64/kvm/vgic/vgic.c +++ b/arch/arm64/kvm/vgic/vgic.c @@ -123,7 +123,7 @@ void __vgic_put_lpi_locked(struct kvm *kvm, struct vgic_irq *irq) return; xa_erase(&dist->lpi_xa, irq->intid); - dist->lpi_list_count--; + atomic_dec(&dist->lpi_count); kfree(irq); } diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h index aeff363e3ba6..71e9d719533b 100644 --- a/include/kvm/arm_vgic.h +++ b/include/kvm/arm_vgic.h @@ -273,10 +273,10 @@ struct vgic_dist { */ u64 propbaser; - /* Protects the lpi_list and the count value below. */ + /* Protects the lpi_list. */ raw_spinlock_t lpi_list_lock; struct xarray lpi_xa; - int lpi_list_count; + atomic_t lpi_count; /* LPI translation cache */ struct list_head lpi_translation_cache; From patchwork Wed Feb 21 05:42:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 203935 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:693c:2685:b0:108:e6aa:91d0 with SMTP id mn5csp850182dyc; Tue, 20 Feb 2024 21:46:33 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCV8Z6EQgkb7sTuamLsGhvAcxQnSq53UngWifmjXsqOV+ACblpHrDwFmbgAaZHwmv/SWn9RxKes9GEj5kuc6gIMg4N9V2w== X-Google-Smtp-Source: AGHT+IH3KUfUDZVJPDffafGVp2O3gfMtTIo1PuXOlBFTJLCsAD4GWDEiN17lUMVXMuNowBlw4hSR X-Received: by 2002:a05:6a20:d38f:b0:19e:9a97:cb12 with SMTP id iq15-20020a056a20d38f00b0019e9a97cb12mr22078770pzb.54.1708494393010; Tue, 20 Feb 2024 21:46:33 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708494392; cv=pass; d=google.com; s=arc-20160816; b=m+vmv3CjYapAKDI8IkZ70C0XHyXbGlwrF3JYSBJmkR94fQcjtC9eNCBz63Zi5+jRhU wT3/+sVFlU34fgtI4icZXKEGzue+nkpQi59C90PjkSUFJo+OlmguyA3tswISteoVhwuy 7LX7L9DQWjBMcmbgtGHKGoMnE/XUVQnZ8pzOU1WVMJYOFU1A8kQEIBhTk+HjjK3Mpak9 0slA5yjL6PDXv+j3T1WHZUpE8WtXp3r4nFym70aPEU/9Njta7NALLGsQz18TPf/MSM4E Yrs/8nVXWSqdpe0SKGGFb5H+LxNdASOJIdFrkEKZAH+zuQoOkQBGuKDNPdJOcMDOdi7P f/Wg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=w2vP8fjlFG48ofYu0phwr6P5tTp39+9IxC+FJcXQ4f8=; fh=kCIhDCwfhVa+1moYcBlLNuY7yX/BfELZCoRk1wSf3jY=; b=oAGqvpUnfY1GrHaU9QwFKI0o0WWhhb4ds3PZls22EYrNgLpn5QCLiCu0PY3M0/7uMN 7CrJbVXRERk7MiEbgqNhvMbcOq859dT7Kp0H/gJQ7tdVW8T/yIDb6aoyORAcutmLdUMW H+alVfjFRQd0WI+K2UFgYZeQf1HvaF7UatisHzWgZFNtQin2u554vR24D+qcipL0wtnm 2AJvBjlkTmK3MD2gVroBrF1HJEIPVh+cyXWjqhSxdGKM+4KSK8tuhTYktjS2TJEdqrCT 6bjePH7HslzTrsuXaDiwbqx6iTRCLr24/s6++TBMbnWPNHxEK2D6E980NgqwF8K+d7M0 tCAg==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=sdaejefD; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-74105-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-74105-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id sn15-20020a17090b2e8f00b002992db5a4f6si7519604pjb.101.2024.02.20.21.46.32 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Feb 2024 21:46:32 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-74105-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=sdaejefD; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-74105-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-74105-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id DA82D285EF4 for ; Wed, 21 Feb 2024 05:45:57 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 7642A3D0C9; Wed, 21 Feb 2024 05:43:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="sdaejefD" Received: from out-189.mta0.migadu.com (out-189.mta0.migadu.com [91.218.175.189]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 670C63A1C7; Wed, 21 Feb 2024 05:43:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.189 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708494215; cv=none; b=Pu56ponqHgCc/vwDKL68Pc6nfyyNN+6UpJzCgahjr2CEs9A4VmkUgUa8D2u5PTcIHZl75RLPY1p2qMD8JAXSvBdqVBs8GsFw8N4ZilsBvFFv1YrjeXBWqcJ7r2wH14QHdg06ta9iUq8BsKO5L1BWSYq7lTKRmEc5NbB4hiaDa74= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708494215; c=relaxed/simple; bh=iPMx6to6nitv27V003CljDT9TCjGzeZ9T5PREHKTqXY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=u1nNYgTX7iawtHklM1McCvSnuuRPf7V4kCinMjdCbinlHOTybJ+3VINq6ryu0xLK7uTNbxsRStm+9TCiFfo2ucq5Iangpw9TA0YtU0/jpDgPimcXfSskyTg6ZLXHbJELK6TVqu2sbCwv8NKWoqqkMxmQ+9uhlYsYUSXqNHiGMuo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=sdaejefD; arc=none smtp.client-ip=91.218.175.189 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1708494212; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=w2vP8fjlFG48ofYu0phwr6P5tTp39+9IxC+FJcXQ4f8=; b=sdaejefD5Jik4CTUWmfbhv0AYU/Q05Y8Op4W9bykUArkL/dz+jcAyW8mywmef8uZMcnfDU UIRvZ/bQ86t1OfXpl0cxcOBySejSehv3d6tkIZpwMvuPSuTYx748F88BSGe7BdJWFS/uzi AIMGLQHhX8DNtbIksD7DZa4YvK5oWhs= From: Oliver Upton To: kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , linux-kernel@vger.kernel.org, Eric Auger , Oliver Upton Subject: [PATCH v4 07/10] KVM: arm64: vgic: Free LPI vgic_irq structs in an RCU-safe manner Date: Wed, 21 Feb 2024 05:42:50 +0000 Message-ID: <20240221054253.3848076-8-oliver.upton@linux.dev> In-Reply-To: <20240221054253.3848076-1-oliver.upton@linux.dev> References: <20240221054253.3848076-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791486216822888142 X-GMAIL-MSGID: 1791486216822888142 Free the vgic_irq structs in an RCU-safe manner to allow reads of the LPI configuration data to happen in parallel with the release of LPIs. Signed-off-by: Oliver Upton --- arch/arm64/kvm/vgic/vgic.c | 2 +- include/kvm/arm_vgic.h | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c index 15dbd17b3a9e..3fedc58e663a 100644 --- a/arch/arm64/kvm/vgic/vgic.c +++ b/arch/arm64/kvm/vgic/vgic.c @@ -125,7 +125,7 @@ void __vgic_put_lpi_locked(struct kvm *kvm, struct vgic_irq *irq) xa_erase(&dist->lpi_xa, irq->intid); atomic_dec(&dist->lpi_count); - kfree(irq); + kfree_rcu(irq, rcu); } void vgic_put_irq(struct kvm *kvm, struct vgic_irq *irq) diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h index 71e9d719533b..47035946648e 100644 --- a/include/kvm/arm_vgic.h +++ b/include/kvm/arm_vgic.h @@ -117,6 +117,7 @@ struct irq_ops { struct vgic_irq { raw_spinlock_t irq_lock; /* Protects the content of the struct */ + struct rcu_head rcu; struct list_head ap_list; struct kvm_vcpu *vcpu; /* SGIs and PPIs: The VCPU From patchwork Wed Feb 21 05:42:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 203938 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:693c:2685:b0:108:e6aa:91d0 with SMTP id mn5csp850257dyc; Tue, 20 Feb 2024 21:46:46 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCW/N0mQYaJLjkomDJ3ME0gbg958ZKWKuO/rfMPVijK+imevEOuKY2hsLw8CntrxaZ+F2hyh0AcYUxBrKU+UjAyHyJwOZw== X-Google-Smtp-Source: AGHT+IEpk2hyxlpliuaJm6w9onGMmZWY7sa4o+c2XHIGGrT9DSIje7fB4jlO/84Du1oNPzNptTxA X-Received: by 2002:a17:902:e74d:b0:1db:e600:4585 with SMTP id p13-20020a170902e74d00b001dbe6004585mr10887594plf.19.1708494406781; Tue, 20 Feb 2024 21:46:46 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708494406; cv=pass; d=google.com; s=arc-20160816; b=I7aG8deu8kt7LE252uSAgRQLDEaiDkFDoZarZtJBVyZ8PBCvgFjW1ZdWbBEzywgWYd qfFdXt9D5ddKo8gOVwzcvHs965COK5NDebBtuAtEQDGEYNeKjRDuyDFAEsK6xw2G3HXH MgDZ+JpIS2tY3YlYgLOIiIQL9bcN4NdjzmSIvqm/wOeeNaUjHHl7gy2x0jBBJq0RH9hm pCasAp719d+n5g28Xo+cTyR7L0CpuWNaoB7AvklN0U3l3PFLITTVE+f0I61qfAJAoaY6 0iVvR7Bn5ehiySGvWyGG2HllOuRx3Wj+ViArzxShQNj3wgGkrbTeh/IbLvtuYEmpl60l ZnLw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=BJ4oP9M3Q0IijgdCm1rTjm/2Zq88g2zw/bm3B6utCQw=; fh=kCIhDCwfhVa+1moYcBlLNuY7yX/BfELZCoRk1wSf3jY=; b=g7P9fgKzlSt5xt9zuw7TuE7GKgba4UqubFwxBfyCspecfl3pHAEWC1k74R51IQsFCh BfxPKLGRLISoQuAxUL6KXsSUDzphtqopnx//DSZ85vaTD773tPWRflz2peCe3hRUbFFS QKG4difqFcvsFzV4zFmTC+go2GsREkh8yZlV/IazzvZGxNU43NhvWhhZgY1fkol5upwi Ctbjp0RGY9hBFwuan9IEQe07oq7JaedAh4SfMZdDATbE7wPNC7KmDEw6ji71Au6CYjsF F5EJsqU8fSuwqnN17mF/PveWYdAIAEVEkCyotB6ss3Q2J7KpCyqmRP8L5jP85xxEcBxU R1LQ==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=UhQFNMtJ; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-74106-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-74106-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id a2-20020a170902ecc200b001da22d9e810si7658948plh.219.2024.02.20.21.46.46 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Feb 2024 21:46:46 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-74106-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=UhQFNMtJ; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-74106-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-74106-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 44D91281214 for ; Wed, 21 Feb 2024 05:46:11 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 8CCCB3D3BA; Wed, 21 Feb 2024 05:43:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="UhQFNMtJ" Received: from out-174.mta0.migadu.com (out-174.mta0.migadu.com [91.218.175.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 58A7C3CF6C for ; Wed, 21 Feb 2024 05:43:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708494217; cv=none; b=KPk0fnRD6QW2lI94H2bLGjgGhe793t4vI3/wCUotTyVqW/e+Hb+ILHRkQs6ifd3143Mhw9ub5RTlvlwmYiKc60uZHzOsHLmEpKwDlQ3/LIB5T/bWJiptRnqczq6ozqocWb5ZRoMGY3NyYlawfJ2V+qjYqRld0dpDDnCPV2vRCm8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708494217; c=relaxed/simple; bh=w2mKAUpjDefBH8ZRTYVpf6wUaoHL7tLPKvgNGeP/7Pc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ZTsEGKeBol5R8edRpHguD+Dyn314/eV+OKyEho6cmk/OBc8s2Lcrdgp4DZ8PXBFe9r5+v6dIpxho1mT0Xsrh/nU9JFKIag/N0GLWViu9Xhwv1lORw7hbjaQR9zQvvsoXZjvh5DOkUqfL0LLpOMITJpahYhF4APrynYq5XU3jA2w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=UhQFNMtJ; arc=none smtp.client-ip=91.218.175.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1708494214; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BJ4oP9M3Q0IijgdCm1rTjm/2Zq88g2zw/bm3B6utCQw=; b=UhQFNMtJMJzvESlIIp0h2831VVZci4irnCeQlhWnqIopdwDfPiffDEsl1FdQD/dUj82J4M ai73hWEqta6WhHvDHXn6FLl1i57BCdfuXEx6luFE3soK+eVjBFfYjSCh9j3BoY4wgq0fDC cziW4XfD063WKUDH4KIEB9TmWsv5rVM= From: Oliver Upton To: kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , linux-kernel@vger.kernel.org, Eric Auger , Oliver Upton Subject: [PATCH v4 08/10] KVM: arm64: vgic: Rely on RCU protection in vgic_get_lpi() Date: Wed, 21 Feb 2024 05:42:51 +0000 Message-ID: <20240221054253.3848076-9-oliver.upton@linux.dev> In-Reply-To: <20240221054253.3848076-1-oliver.upton@linux.dev> References: <20240221054253.3848076-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791486231186140456 X-GMAIL-MSGID: 1791486231186140456 Stop acquiring the lpi_list_lock in favor of RCU for protecting the read-side critical section in vgic_get_lpi(). In order for this to be safe, we also need to be careful not to take a reference on an irq with a refcount of 0, as it is about to be freed. Signed-off-by: Oliver Upton --- arch/arm64/kvm/vgic/vgic.c | 9 ++++----- arch/arm64/kvm/vgic/vgic.h | 14 +++++++++++--- 2 files changed, 15 insertions(+), 8 deletions(-) diff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c index 3fedc58e663a..76abf3d946fe 100644 --- a/arch/arm64/kvm/vgic/vgic.c +++ b/arch/arm64/kvm/vgic/vgic.c @@ -63,15 +63,14 @@ static struct vgic_irq *vgic_get_lpi(struct kvm *kvm, u32 intid) { struct vgic_dist *dist = &kvm->arch.vgic; struct vgic_irq *irq = NULL; - unsigned long flags; - raw_spin_lock_irqsave(&dist->lpi_list_lock, flags); + rcu_read_lock(); irq = xa_load(&dist->lpi_xa, intid); - if (irq) - vgic_get_irq_kref(irq); + if (!vgic_try_get_irq_kref(irq)) + irq = NULL; - raw_spin_unlock_irqrestore(&dist->lpi_list_lock, flags); + rcu_read_unlock(); return irq; } diff --git a/arch/arm64/kvm/vgic/vgic.h b/arch/arm64/kvm/vgic/vgic.h index 8d134569d0a1..f874b9932c5a 100644 --- a/arch/arm64/kvm/vgic/vgic.h +++ b/arch/arm64/kvm/vgic/vgic.h @@ -220,12 +220,20 @@ void vgic_v2_vmcr_sync(struct kvm_vcpu *vcpu); void vgic_v2_save_state(struct kvm_vcpu *vcpu); void vgic_v2_restore_state(struct kvm_vcpu *vcpu); -static inline void vgic_get_irq_kref(struct vgic_irq *irq) +static inline bool vgic_try_get_irq_kref(struct vgic_irq *irq) { + if (!irq) + return false; + if (irq->intid < VGIC_MIN_LPI) - return; + return true; - kref_get(&irq->refcount); + return kref_get_unless_zero(&irq->refcount); +} + +static inline void vgic_get_irq_kref(struct vgic_irq *irq) +{ + WARN_ON_ONCE(!vgic_try_get_irq_kref(irq)); } void vgic_v3_fold_lr_state(struct kvm_vcpu *vcpu); From patchwork Wed Feb 21 05:42:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 203940 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:693c:2685:b0:108:e6aa:91d0 with SMTP id mn5csp850515dyc; Tue, 20 Feb 2024 21:47:52 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWMDDQXk84/vdUwczIUUQehdIlNjeNRORm+V39XuW/Cpnk9xaP1mZQe6KdbRZwfGW72CkZBBhj2VMyN2w8z+oTGYH126A== X-Google-Smtp-Source: AGHT+IExwuFnynA913JIj0AElII8dwLneqfDrwapGdQqenzsamkqgZDVGp+T+MlF3nsfsa5OqSV4 X-Received: by 2002:a05:6a21:9101:b0:19e:ccb2:fd80 with SMTP id tn1-20020a056a21910100b0019eccb2fd80mr16356825pzb.8.1708494472176; Tue, 20 Feb 2024 21:47:52 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708494472; cv=pass; d=google.com; s=arc-20160816; b=Kb+QowBqbpzw6fLwoytEoaBesNao3K5U+tYOVI2uSJ2PUBBND07XUnusSmYxzZZAvU +3Kr+kvgzP9wMsOD2rvpKGIYkw78clYLKhV3qjbri3lCGpPATeZayBNJJufK5Ozpa0F/ 4mb1QHJA0JLH7JjiqcHQFqkhRVyXBctv1G3ZMVF2g+Vjcyf4LCJFsUqzxMHkzUkzyoUG xQ4UmyoYij03peHSxGg//I5poB4rFfnVMk0bPe6eyrC9UrD4x/mVdGym06dJZpPH6JUk QyCyRbWgWiCVfphi/zuByYnV4QYEpAuhXOFj2mP2XhNV8xNJiHiVrdNJNwVGqjxSaDhe R31w== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=QeMz6ixCWY/1TUllIvS9OwAED3X7tPlGIFKged/im1A=; fh=kCIhDCwfhVa+1moYcBlLNuY7yX/BfELZCoRk1wSf3jY=; b=TFUGIowZNhYs3UIsT6aZVatK1WBJLe9UiJKrIGC4gCosKwLj1/k9ZnsaF5hVdmLJl0 qhmQ2sncMl8jHhoxJ/tjLr85MCeWtlUf+ZmihhTRmFJRC/thG3946OYHZIjo1SAwaqfl E2BpWmrhIX4vczAr0RGgrLj3f7FO0b3xlwS+h3Qpf2TYsN76bagGVydoAXW+yl1jJoPF gIqOEEGFQ0/FjYU/s2AJmAJOT6UcD9Ok78h3yma1MkGg9Bmi63sLvAxrgkRk3Hjloh9A wYSLaxsgdOOyovOMAR53Mor0xMdifOocqB8AaMl6HSU2+z6R4N82Gb7FzT0CbU3D5FmH R8nQ==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=oJYMuUDa; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-74107-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-74107-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [147.75.48.161]) by mx.google.com with ESMTPS id np6-20020a17090b4c4600b0029a1aaefe30si362159pjb.188.2024.02.20.21.47.51 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Feb 2024 21:47:52 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-74107-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=oJYMuUDa; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-74107-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-74107-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id D14ACB23303 for ; Wed, 21 Feb 2024 05:46:33 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C90533D578; Wed, 21 Feb 2024 05:43:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="oJYMuUDa" Received: from out-176.mta0.migadu.com (out-176.mta0.migadu.com [91.218.175.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5F2E23D0C6 for ; Wed, 21 Feb 2024 05:43:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708494220; cv=none; b=nts3WQmI7Hj10QJ6wCKaxaQ+hjeZ6bfINOQG8WNOHH3GYfGC6aFZNE47RoQpsucsRgnlbabHlFgRLGxOmTR+yvRBHbbs72Q2kEwIXwxQaE7u0rlMR2E4iSrE8LNfBGtiUD4oVcv5oaIjW3qyC8Bs/PWiLoiw6TFoaAG5vZkZjGM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708494220; c=relaxed/simple; bh=ohZ5HNh3+AbhZtUPEYzsTEwYsRO0b4zg7BkWRCA1M6o=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=LX6dUhjfgPEjgsnJRgOhiwDVvh7Nmfr3pJVG5uhLSZSdNx3nFjOdUtCyz6FQMF5dixsjDMtElFmrZl+plircMN6wEXUDHTGaPVyd2ET8bvWRU/rlX8DSUDIfo7pwQU1YVu/Lwr+BCIOyOShSluQQ1a0XwtgLj9V61S6WmAOmevU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=oJYMuUDa; arc=none smtp.client-ip=91.218.175.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1708494216; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QeMz6ixCWY/1TUllIvS9OwAED3X7tPlGIFKged/im1A=; b=oJYMuUDa4lorjzjLyP4W/y7ThUhnzjPTg1eB4MUsxt75hClMvcOy7W5XhtgPpwI/Z4vK9f Q2cvCYDT3VIRIO3EH6bfRd2UeplzRCqfbqNE5BAQq8jYAUnSfYsZMCdvCrM4hYZDZyHB3Q QI7HCZ7WCRhcFu35cRe8lW/4RIxoxpY= From: Oliver Upton To: kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , linux-kernel@vger.kernel.org, Eric Auger , Oliver Upton Subject: [PATCH v4 09/10] KVM: arm64: vgic: Ensure the irq refcount is nonzero when taking a ref Date: Wed, 21 Feb 2024 05:42:52 +0000 Message-ID: <20240221054253.3848076-10-oliver.upton@linux.dev> In-Reply-To: <20240221054253.3848076-1-oliver.upton@linux.dev> References: <20240221054253.3848076-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791486299573923559 X-GMAIL-MSGID: 1791486299573923559 It will soon be possible for get() and put() calls to happen in parallel, which means in most cases we must ensure the refcount is nonzero when taking a new reference. Switch to using vgic_try_get_irq_kref() where necessary, and document the few conditions where an IRQ's refcount is guaranteed to be nonzero. Signed-off-by: Oliver Upton --- arch/arm64/kvm/vgic/vgic-its.c | 18 ++++++++---------- arch/arm64/kvm/vgic/vgic.c | 3 ++- 2 files changed, 10 insertions(+), 11 deletions(-) diff --git a/arch/arm64/kvm/vgic/vgic-its.c b/arch/arm64/kvm/vgic/vgic-its.c index 0be3c33676c3..dad6f0ee7c49 100644 --- a/arch/arm64/kvm/vgic/vgic-its.c +++ b/arch/arm64/kvm/vgic/vgic-its.c @@ -74,18 +74,11 @@ static struct vgic_irq *vgic_add_lpi(struct kvm *kvm, u32 intid, * check that we don't add a second list entry with the same LPI. */ oldirq = xa_load(&dist->lpi_xa, intid); - if (oldirq) { + if (vgic_try_get_irq_kref(oldirq)) { /* Someone was faster with adding this LPI, lets use that. */ kfree(irq); irq = oldirq; - /* - * This increases the refcount, the caller is expected to - * call vgic_put_irq() on the returned pointer once it's - * finished with the IRQ. - */ - vgic_get_irq_kref(irq); - goto out_unlock; } @@ -611,8 +604,8 @@ static struct vgic_irq *vgic_its_check_cache(struct kvm *kvm, phys_addr_t db, raw_spin_lock_irqsave(&dist->lpi_list_lock, flags); irq = __vgic_its_check_cache(dist, db, devid, eventid); - if (irq) - vgic_get_irq_kref(irq); + if (!vgic_try_get_irq_kref(irq)) + irq = NULL; raw_spin_unlock_irqrestore(&dist->lpi_list_lock, flags); @@ -658,6 +651,11 @@ static void vgic_its_cache_translation(struct kvm *kvm, struct vgic_its *its, if (cte->irq) __vgic_put_lpi_locked(kvm, cte->irq); + /* + * The irq refcount is guaranteed to be nonzero while holding the + * its_lock, as the ITE (and the reference it holds) cannot be freed. + */ + lockdep_assert_held(&its->its_lock); vgic_get_irq_kref(irq); cte->db = db; diff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c index 76abf3d946fe..df9e1aa1956c 100644 --- a/arch/arm64/kvm/vgic/vgic.c +++ b/arch/arm64/kvm/vgic/vgic.c @@ -395,7 +395,8 @@ bool vgic_queue_irq_unlock(struct kvm *kvm, struct vgic_irq *irq, /* * Grab a reference to the irq to reflect the fact that it is - * now in the ap_list. + * now in the ap_list. This is safe as the caller must already hold a + * reference on the irq. */ vgic_get_irq_kref(irq); list_add_tail(&irq->ap_list, &vcpu->arch.vgic_cpu.ap_list_head); From patchwork Wed Feb 21 05:42:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 203937 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:693c:2685:b0:108:e6aa:91d0 with SMTP id mn5csp850253dyc; Tue, 20 Feb 2024 21:46:46 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWtsF9DSyMdQ04c+xH/DgfVJkLK3l2PX2gsRnZoDLUk2JclvBVvMXV3GTCH/Ry3/xxb6vv6jeqAToMS+neb4ucsx6+NJQ== X-Google-Smtp-Source: AGHT+IE//I+Q76PR+KH7Y0CCO/Y52iiBpdh8BJXug+Dvgtdg7w/Jrz7B66S+yrUhV0hp0bpmVSvD X-Received: by 2002:a17:906:cb9a:b0:a3e:9ce3:1b2 with SMTP id mf26-20020a170906cb9a00b00a3e9ce301b2mr5011755ejb.5.1708494406039; Tue, 20 Feb 2024 21:46:46 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708494406; cv=pass; d=google.com; s=arc-20160816; b=k7W15JfG+mvsjKWb7PHilcDqL/uEwihCIfDVgBmUsXTYWl3GnSAPmuyTfZ8BUUvsT4 /C5cWDkcnEDdPZwJFlbCB2Re3a8GGfhvX140VY0ybl37Up+I2TzqNAjL/zYK0hiWM9z0 fnudFyyrJF5+eMBvSrJFTrjYReYWnoi39FQo8wL0uWK4tFJbCPwTdDis7YrobRZU86B5 0ILsM1eyD9CwE9naLDWt76+6SL9bc4AmKNoy/iqAvPw/QRniqFgce3bO4FfPjqUG+/0S +1qEsz+1cqlGPqkjBPDpu+mlJOlH2xEv5eeloT6303ccFu3ivQvxEATQbUeEulYi4zVW mfoQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=7uZnxcLXKLqyPCNevo5l7NpGLhMeejXcsQuYRK9QxO0=; fh=kCIhDCwfhVa+1moYcBlLNuY7yX/BfELZCoRk1wSf3jY=; b=q2NU/4+4c6RYbFX934tU+V8QwhevNpY4e1wLJiweKzJR7VsePy6Fl1X3RuQo5VG/GX ktBtnQ7cC6LQ0QkozDsHYDCQNKf6H9MOiU9dHF4t4cCc8cCSkOzNTfsbZN5U2BZBdiah LUc4id0KmO5AzHrm7q7jo7OrSQhzRoycn0w3K+OWaclsqX9HN9IBiIaiNdHUumIiyNO+ ici2J8sXrgCW8Po4gFfqEJnpNCt2zppHipz2xpw7DjL4bpKNPXX2NEAStqgMQ0IxhHyX b4k8MHuvPVtXUTBMup2fdPPaDYY0zKxV+R1RRt5a4T2aW5Ht+kcn5j0gTaYV/lF4gKOg 2p2Q==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b="RK/jSFpK"; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-74108-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-74108-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id ko1-20020a170907986100b00a3e770045b0si2705836ejc.226.2024.02.20.21.46.45 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Feb 2024 21:46:46 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-74108-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b="RK/jSFpK"; arc=pass (i=1 spf=pass spfdomain=linux.dev dkim=pass dkdomain=linux.dev dmarc=pass fromdomain=linux.dev); spf=pass (google.com: domain of linux-kernel+bounces-74108-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-74108-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id A5A211F21228 for ; Wed, 21 Feb 2024 05:46:45 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id A0A7D3D978; Wed, 21 Feb 2024 05:43:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="RK/jSFpK" Received: from out-170.mta0.migadu.com (out-170.mta0.migadu.com [91.218.175.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8C6953A268 for ; Wed, 21 Feb 2024 05:43:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708494222; cv=none; b=rkxGO46GGPCPs0qzZGOiNODYZZx22qMabdPc4HwP0dOQ/hbrBWLzelgJAT5vrwUwnV5vEZfUsyRCkNnG7c9iesz3Ortpl1BfpEIatNrco/koE9xibaMFi8FmrvNjoZGGxrfv1nnPtkoFJM3PdI7fQTx6A+1iLrS3OZtjbG8OYJ8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708494222; c=relaxed/simple; bh=huN1PqJtmqytpL0vyqUMZqGEJFXVuxAo91FZdlEJvZw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ANrTd1w0r8MGB6wJu2cg3bWDC+q9JOtrD3YWHLUpRLR0z3pFx2XNV1lsOrqeUIvX1kAFTC/W1pHOmTvCu18afoZXnj+y5ToRTlGN9sDxP+ImaX8uGcgLSNaMfrPlmtrtHWntasxslp7zkZ1QUbiZoNxgGUDar+H67Rj9F2GYWmg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=RK/jSFpK; arc=none smtp.client-ip=91.218.175.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1708494218; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7uZnxcLXKLqyPCNevo5l7NpGLhMeejXcsQuYRK9QxO0=; b=RK/jSFpKYJqpDZ2ouKFiZ2I/rdOcsLx5lKza9bI7O/DEvmkWAWOwRt7MBFuz/AgkDXmv1z 1DsW4AldOgmJ72zxrmxbd18hjzeuCR1L2nPxHUhyTOjfZ4E7Nnk5ArYF9wBFEWr2ZBsot/ q64aIxAUJxhWVqMsKVznWHHqpDmDvVA= From: Oliver Upton To: kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , linux-kernel@vger.kernel.org, Eric Auger , Oliver Upton Subject: [PATCH v4 10/10] KVM: arm64: vgic: Don't acquire the lpi_list_lock in vgic_put_irq() Date: Wed, 21 Feb 2024 05:42:53 +0000 Message-ID: <20240221054253.3848076-11-oliver.upton@linux.dev> In-Reply-To: <20240221054253.3848076-1-oliver.upton@linux.dev> References: <20240221054253.3848076-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791486230344982597 X-GMAIL-MSGID: 1791486230344982597 The LPI xarray's xa_lock is sufficient for synchronizing writers when freeing a given LPI. Furthermore, readers can only take a new reference on an IRQ if it was already nonzero. Stop taking the lpi_list_lock unnecessarily and get rid of __vgic_put_lpi_locked(). Signed-off-by: Oliver Upton --- arch/arm64/kvm/vgic/vgic-its.c | 4 ++-- arch/arm64/kvm/vgic/vgic.c | 28 +++++++++------------------- arch/arm64/kvm/vgic/vgic.h | 1 - 3 files changed, 11 insertions(+), 22 deletions(-) diff --git a/arch/arm64/kvm/vgic/vgic-its.c b/arch/arm64/kvm/vgic/vgic-its.c index dad6f0ee7c49..f6025886071c 100644 --- a/arch/arm64/kvm/vgic/vgic-its.c +++ b/arch/arm64/kvm/vgic/vgic-its.c @@ -649,7 +649,7 @@ static void vgic_its_cache_translation(struct kvm *kvm, struct vgic_its *its, * was in the cache, and increment it on the new interrupt. */ if (cte->irq) - __vgic_put_lpi_locked(kvm, cte->irq); + vgic_put_irq(kvm, cte->irq); /* * The irq refcount is guaranteed to be nonzero while holding the @@ -686,7 +686,7 @@ void vgic_its_invalidate_cache(struct kvm *kvm) if (!cte->irq) break; - __vgic_put_lpi_locked(kvm, cte->irq); + vgic_put_irq(kvm, cte->irq); cte->irq = NULL; } diff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c index df9e1aa1956c..f963f410788a 100644 --- a/arch/arm64/kvm/vgic/vgic.c +++ b/arch/arm64/kvm/vgic/vgic.c @@ -111,22 +111,6 @@ static void vgic_irq_release(struct kref *ref) { } -/* - * Drop the refcount on the LPI. Must be called with lpi_list_lock held. - */ -void __vgic_put_lpi_locked(struct kvm *kvm, struct vgic_irq *irq) -{ - struct vgic_dist *dist = &kvm->arch.vgic; - - if (!kref_put(&irq->refcount, vgic_irq_release)) - return; - - xa_erase(&dist->lpi_xa, irq->intid); - atomic_dec(&dist->lpi_count); - - kfree_rcu(irq, rcu); -} - void vgic_put_irq(struct kvm *kvm, struct vgic_irq *irq) { struct vgic_dist *dist = &kvm->arch.vgic; @@ -135,9 +119,15 @@ void vgic_put_irq(struct kvm *kvm, struct vgic_irq *irq) if (irq->intid < VGIC_MIN_LPI) return; - raw_spin_lock_irqsave(&dist->lpi_list_lock, flags); - __vgic_put_lpi_locked(kvm, irq); - raw_spin_unlock_irqrestore(&dist->lpi_list_lock, flags); + if (!kref_put(&irq->refcount, vgic_irq_release)) + return; + + xa_lock_irqsave(&dist->lpi_xa, flags); + __xa_erase(&dist->lpi_xa, irq->intid); + xa_unlock_irqrestore(&dist->lpi_xa, flags); + + atomic_dec(&dist->lpi_count); + kfree_rcu(irq, rcu); } void vgic_flush_pending_lpis(struct kvm_vcpu *vcpu) diff --git a/arch/arm64/kvm/vgic/vgic.h b/arch/arm64/kvm/vgic/vgic.h index f874b9932c5a..0c2b82de8fa3 100644 --- a/arch/arm64/kvm/vgic/vgic.h +++ b/arch/arm64/kvm/vgic/vgic.h @@ -180,7 +180,6 @@ vgic_get_mmio_region(struct kvm_vcpu *vcpu, struct vgic_io_device *iodev, gpa_t addr, int len); struct vgic_irq *vgic_get_irq(struct kvm *kvm, struct kvm_vcpu *vcpu, u32 intid); -void __vgic_put_lpi_locked(struct kvm *kvm, struct vgic_irq *irq); void vgic_put_irq(struct kvm *kvm, struct vgic_irq *irq); bool vgic_get_phys_line_level(struct vgic_irq *irq); void vgic_irq_set_phys_pending(struct vgic_irq *irq, bool pending);