From patchwork Fri Dec 8 16:22:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Krowiak X-Patchwork-Id: 175921 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp5565151vqy; Fri, 8 Dec 2023 08:24:03 -0800 (PST) X-Google-Smtp-Source: AGHT+IFxXkKFG+55wLG6OY7NZePL9n1xuolm8BELCxMuoLUanbWS8bBAGXEp0gyGvEnKdPr0gbLT X-Received: by 2002:a17:902:dacd:b0:1cf:ce82:95ec with SMTP id q13-20020a170902dacd00b001cfce8295ecmr309128plx.2.1702052643053; Fri, 08 Dec 2023 08:24:03 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702052643; cv=none; d=google.com; s=arc-20160816; b=RboWuCQzPm9tf/akNtpXH1m7hT/biExo2V4q+3TsUxamts5SSDkL7ebqLMPwvOjXKu cWoakKFdwkrbtJR+VX+ZR0DTvc89dOM6Y5ZemOwYSPdfdOq75ybkWRbkwskTmcJ8YjS5 gABdGrKZ/K8XqLd4Uo5PV5uCpHKzyhjQ6QeDnWvf2tesSZlTacVYUIVwkBEXP1c0Hn3X 7f13Z2mozN8dzw0ahjgYW/BjYCZ+kyceRi9R77uIaNAtkVRqoens6jbokjUpbEkeSe4i yG7hL6lXTKlYC7Zppkquvw1mid29mOwRD9jE8PF0O/KAN3K18GyRJoOh4xa2MJlBRWYn C0cA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=mFpPMJfnCOgbZgoSRbq1Ejegj71ZF0EY6P3+BMfH/yI=; fh=89dtS/PwPJUSahQCrBwvdQ9UYtztJf8P1g5jTeyUoxY=; b=puxFPJDqW2O/RXU43BQL8+9ITnNYIQE+8zuGkpCVDwk6Ze2KJI8AopFF/ZOr9i3q9l H8eIqcuSzUIZpsJfxi+Zfxqxwl+XiT4wvKpAAN3IXSetOeZsA/xA7iRklvG+hpzf1zvC z5/sg+u4oBHBqT0WncmP7kSVRkR26dRGZJ3B1mIKCMYS1fOO8qQHPpZKDG4nqaAzE+Vw gK7GLIzO/SQ1ijGV7N/E8m808DOXw7djhOi/t3F9OxJ3PlTLQRHJP5WAmJU6T5tpZAPW g0BbjrbM5ei8SQ8rTL+N09uij4mM3jRLKdqk2QBYRPd05/huKi6X0o68bSJM2Iy6MDi3 QTgQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@ibm.com header.s=pp1 header.b=YiTCR0gd; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:1 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=NONE dis=NONE) header.from=ibm.com Received: from morse.vger.email (morse.vger.email. [2620:137:e000::3:1]) by mx.google.com with ESMTPS id t18-20020a170902e85200b001d0b5954eb1si1813441plg.143.2023.12.08.08.23.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Dec 2023 08:24:03 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:1 as permitted sender) client-ip=2620:137:e000::3:1; Authentication-Results: mx.google.com; dkim=pass header.i=@ibm.com header.s=pp1 header.b=YiTCR0gd; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:1 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=NONE dis=NONE) header.from=ibm.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by morse.vger.email (Postfix) with ESMTP id 75362837E679; Fri, 8 Dec 2023 08:23:48 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at morse.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1573862AbjLHQX3 (ORCPT + 99 others); Fri, 8 Dec 2023 11:23:29 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48056 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1574052AbjLHQXP (ORCPT ); Fri, 8 Dec 2023 11:23:15 -0500 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D73EC1998; Fri, 8 Dec 2023 08:23:12 -0800 (PST) Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 3B8FuFvH023033; Fri, 8 Dec 2023 16:23:11 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=mFpPMJfnCOgbZgoSRbq1Ejegj71ZF0EY6P3+BMfH/yI=; b=YiTCR0gdu45KNQrh5fSW+9Y8nUJ7pPJcXUPQjcFclnyW/nqaJ+WcObXCaDRKtSf298ve 6HuAAzqNkpQau7UHgPvEJj+xUSt0VVUlMS7z1cadYLJXTCBTMiMzcODtylHaEqNpxFOQ LDQyKTkCN+zaimyJvfmLSHHVa5pLpgAEJvEqsEqTBRKoBPv1SYJ1xD5YVxEqjhC0FXPd pPrav9KNSkIi0tfqAMAB8/pWd0ab7imrV5/Sb3IrEV8Zqi9xGzWuIhFa65VK3aCj8Kfx vqRe/Oq9QOMpUjpBODj7SiXSx8ZfI3NmlBw7E0daRFiI9i7MzqMCvNbejfOJtMY1cw2b aQ== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3uv61610nn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 08 Dec 2023 16:23:10 +0000 Received: from m0353729.ppops.net (m0353729.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3B8FuC8k022881; Fri, 8 Dec 2023 16:23:09 GMT Received: from ppma22.wdc07v.mail.ibm.com (5c.69.3da9.ip4.static.sl-reverse.com [169.61.105.92]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3uv61610my-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 08 Dec 2023 16:23:09 +0000 Received: from pps.filterd (ppma22.wdc07v.mail.ibm.com [127.0.0.1]) by ppma22.wdc07v.mail.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 3B8GLjU3001561; Fri, 8 Dec 2023 16:23:08 GMT Received: from smtprelay02.dal12v.mail.ibm.com ([172.16.1.4]) by ppma22.wdc07v.mail.ibm.com (PPS) with ESMTPS id 3utav2tjfe-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 08 Dec 2023 16:23:08 +0000 Received: from smtpav06.wdc07v.mail.ibm.com (smtpav06.wdc07v.mail.ibm.com [10.39.53.233]) by smtprelay02.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 3B8GN6gw36897106 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 8 Dec 2023 16:23:06 GMT Received: from smtpav06.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 75BE458054; Fri, 8 Dec 2023 16:23:06 +0000 (GMT) Received: from smtpav06.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 399F95803F; Fri, 8 Dec 2023 16:23:05 +0000 (GMT) Received: from li-2c1e724c-2c76-11b2-a85c-ae42eaf3cb3d.ibm.com.com (unknown [9.61.47.9]) by smtpav06.wdc07v.mail.ibm.com (Postfix) with ESMTP; Fri, 8 Dec 2023 16:23:05 +0000 (GMT) From: Tony Krowiak To: linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: jjherne@linux.ibm.com, borntraeger@de.ibm.com, pasic@linux.ibm.com, pbonzini@redhat.com, frankja@linux.ibm.com, imbrenda@linux.ibm.com, alex.williamson@redhat.com, kwankhede@nvidia.com, stable@vger.kernel.org Subject: [PATCH v1 5/6] s390/vfio-ap: reset queues associated with adapter for queue unbound from driver Date: Fri, 8 Dec 2023 11:22:50 -0500 Message-ID: <20231208162256.10633-6-akrowiak@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20231208162256.10633-1-akrowiak@linux.ibm.com> References: <20231208162256.10633-1-akrowiak@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: cfbNrAZ8MJ_VAunspm8OnWwJv-XtJDeq X-Proofpoint-ORIG-GUID: O38ENwsBah_BZQf4s-7WbQC_HcGC9KKw X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-08_11,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 phishscore=0 mlxscore=0 spamscore=0 bulkscore=0 lowpriorityscore=0 mlxlogscore=731 impostorscore=0 priorityscore=1501 clxscore=1015 malwarescore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2312080135 X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on morse.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (morse.vger.email [0.0.0.0]); Fri, 08 Dec 2023 08:23:48 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1784731552110107182 X-GMAIL-MSGID: 1784731552110107182 When a queue is unbound from the vfio_ap device driver, if that queue is assigned to a guest's AP configuration, its associated adapter is removed because queues are defined to a guest via a matrix of adapters and domains; so, it is not possible to remove a single queue. If an adapter is removed from the guest's AP configuration, all associated queues must be reset to prevent leaking crypto data should any of them be assigned to a different guest or device driver. The one caveat is that if the queue is being removed because the adapter or domain has been removed from the host's AP configuration, then an attempt to reset the queue will fail with response code 01, AP-queue number not valid; so resetting these queues should be skipped. Signed-off-by: Tony Krowiak Fixes: 09d31ff78793 ("s390/vfio-ap: hot plug/unplug of AP devices when probed/removed") Cc: --- drivers/s390/crypto/vfio_ap_ops.c | 39 ++++++++++++++++++++++++------- 1 file changed, 30 insertions(+), 9 deletions(-) diff --git a/drivers/s390/crypto/vfio_ap_ops.c b/drivers/s390/crypto/vfio_ap_ops.c index f08321385058..5db11d50b4b0 100644 --- a/drivers/s390/crypto/vfio_ap_ops.c +++ b/drivers/s390/crypto/vfio_ap_ops.c @@ -2187,6 +2187,23 @@ int vfio_ap_mdev_probe_queue(struct ap_device *apdev) return ret; } +static void reset_queues_for_apid(struct ap_matrix_mdev *matrix_mdev, + unsigned long apid) +{ + DECLARE_BITMAP(apm_reset, AP_DEVICES); + + /* + * If the adapter is not in the host's AP configuration, then resetting + * any queue for that adapter will fail with response code 01, (APQN not + * valid). + */ + if (test_bit_inv(apid, (unsigned long *)matrix_dev->info.apm)) { + bitmap_clear(apm_reset, 0, AP_DEVICES); + set_bit_inv(apid, apm_reset); + reset_queues_for_apids(matrix_mdev, apm_reset); + } +} + void vfio_ap_mdev_remove_queue(struct ap_device *apdev) { unsigned long apid, apqi; @@ -2199,24 +2216,28 @@ void vfio_ap_mdev_remove_queue(struct ap_device *apdev) matrix_mdev = q->matrix_mdev; if (matrix_mdev) { - vfio_ap_unlink_queue_fr_mdev(q); - - apid = AP_QID_CARD(q->apqn); - apqi = AP_QID_QUEUE(q->apqn); - - /* - * If the queue is assigned to the guest's APCB, then remove - * the adapter's APID from the APCB and hot it into the guest. - */ + /* If the queue is assigned to the guest's AP configuration */ if (test_bit_inv(apid, matrix_mdev->shadow_apcb.apm) && test_bit_inv(apqi, matrix_mdev->shadow_apcb.aqm)) { + /* + * Since the queues are defined via a matrix of adapters + * and domains, it is not possible to hot unplug a + * single queue; so, let's unplug the adapter. + */ clear_bit_inv(apid, matrix_mdev->shadow_apcb.apm); vfio_ap_mdev_update_guest_apcb(matrix_mdev); + reset_queues_for_apid(matrix_mdev, apid); + goto done; } } vfio_ap_mdev_reset_queue(q); flush_work(&q->reset_work); + +done: + if (matrix_mdev) + vfio_ap_unlink_queue_fr_mdev(q); + dev_set_drvdata(&apdev->device, NULL); kfree(q); release_update_locks_for_mdev(matrix_mdev);