From patchwork Thu Sep 14 19:15:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Brett Creeley X-Patchwork-Id: 139862 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp647373vqi; Thu, 14 Sep 2023 14:56:26 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFGhKN3ag4ElS6Z/3luZXY0O8/bPUHAPnZdsD7aLZox5gi7V/7qxiIeSn4OnOGizZ5YkRaq X-Received: by 2002:a17:90a:4417:b0:26b:535e:fc12 with SMTP id s23-20020a17090a441700b0026b535efc12mr4483778pjg.7.1694728585767; Thu, 14 Sep 2023 14:56:25 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1694728585; cv=pass; d=google.com; s=arc-20160816; b=m7uBWXa9qgbSVTeUTFThb/0RtCBqB48fN9PzNoGQaPhX/v12a5AV1ncn/7NIvoxNhK SPemtvSsq7Dd/n6otwHovuw6dR5nLEp4jHHYXXAtSXyaHkcpQsPa3SZ0A3YaX1RnR4eH p7UbwVGD1dIOon4pDgq3vNZ0WQDDSPEQKdZTx7OfLM6quuj/kpvqUC4h5e8t2+mXA8Ar 3h3xzw90shfWB+RZSPK1CJ4CP0rE6U2DUYCq/aAw+vAjKoHmOCmLbQ26szb0p+xKJche q/4Ai2EaZZ9bsr0AU94FyUeHmwLudNB9kIgEageKgvW1rCm8FeaWggZ6G++FHu9vcQ3U 7mLA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=NG7Jyj8R8P2UlL7YSF1hhHIULyx5NwgQGf+S9OQuyHI=; fh=GbNd0iuihik10v4ZTZpqkH2SVSESc+5QskJdSEY04s4=; b=xR8djRQcJU5fdNJuP5ILNWSkE2lUrabhGmgiOgD+8I0JXbQDIE1W62jtbIM8hHGi5L zItIXTHrPd58LugFYdOFhQVFUYQZpgRe1GMunbbjX2q/UJ/cRQl4aZeqrQDfXViCVKUz tvfMR8GEkRhigBwQMveH9YpGwRPEfbS7i1/ApttBN4BMhdugHx3NPDchu3i26EHxH2YI 1WaCczc9ZpUqRVht1MG4dJaoT/gjquOdqylnA1QowS0Ahmw7bCJTszzmU2VqQ6KQlohB FIbZh540Ddtsgyisz30Vi8485MyAnO/F/ijCYuieJ0S46jfSsSySJYyZgdulO2wNe9/T lgbA== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b=F8ExpzR5; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Received: from howler.vger.email (howler.vger.email. [2620:137:e000::3:4]) by mx.google.com with ESMTPS id m14-20020a17090a34ce00b00273fc50cf8dsi4335811pjf.156.2023.09.14.14.56.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Sep 2023 14:56:25 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) client-ip=2620:137:e000::3:4; Authentication-Results: mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b=F8ExpzR5; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id A46C28112980; Thu, 14 Sep 2023 12:16:08 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241809AbjINTQF (ORCPT + 33 others); Thu, 14 Sep 2023 15:16:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42426 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241731AbjINTQD (ORCPT ); Thu, 14 Sep 2023 15:16:03 -0400 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2041.outbound.protection.outlook.com [40.107.93.41]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B1ECA26B3; Thu, 14 Sep 2023 12:15:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JJsh3fj+bLvs86UoTr6P08712Y5S9BtOHcEwFXxbzbIsLjE0nH6VAJ9ZZiy/dxPoRCOw2M0A2WrJT6MzpoHc8e2+Pjp7hQfMB5wvd9r0joDj9PsyYRlI3kLS/DM1+aVs5F1HrgREfSlG1Fb+kDF83h7h3hjKZuLSYy8t6kiUFOoaHnjIaU5dcIzxk/gLF29sjokllAwycDiH3uBaaL9t4Gpj9po+kO+Qpxg6CIH0sIchU4nFBY3kOmBYn2Nv7JDoIU1dFmTA1bfTTtiJaqZ4MQ1vDgIFuJ4hF8sJYdrAOz89W1ho+MSWrbat/7W0DfrAvgh4DXe0wn5jlkpPrEvDdw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=NG7Jyj8R8P2UlL7YSF1hhHIULyx5NwgQGf+S9OQuyHI=; b=QflN4crE7tUjfC5eK4IdQJfVXpGlcclJ+lXynJCMI4hYzyATJBRcg7GUB5Vfni4UgA/NdtG2dgVQJeUiTX3AyOdjze0vfFPkv8YtUkOvR7m+C+WaBjBcI1mW4umTi7THNILhJk4qTp/VJAiomBxfffLtKcdIwzgkGhPepd2EF0+FfH3wZigNwfYecaO13/aoiUdMP4QW4zofCPJBW2keGoi8xnt6pG0CJmJitk4UsoC0ZKzRa36mIy8rReBGx99O1yuIFRn6mPua2bd/P/Y9FvIYI3On53GJzZP7E7SptwSd08NCpdZeywAOHIFLJFDVsHzxkwmBPsQgpOfxz186mg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=ziepe.ca smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NG7Jyj8R8P2UlL7YSF1hhHIULyx5NwgQGf+S9OQuyHI=; b=F8ExpzR5BWKWs6YryP6jFQDcaQfV4RGp6aZC+f50aeJ5KRFvpZx4N8DduvVUoqF+2r7H+otm1h7jqE/8iDcSwMLaXQV9tQtnuqGWqGDIsAVah8apOODxPj5XdiGwaFsgJdPmPnMlrV+Dk6rbbx/SSB28Xu3XVMYZ2vOxCKSMWvQ= Received: from SN7PR18CA0003.namprd18.prod.outlook.com (2603:10b6:806:f3::23) by BL1PR12MB5240.namprd12.prod.outlook.com (2603:10b6:208:319::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6792.20; Thu, 14 Sep 2023 19:15:55 +0000 Received: from SN1PEPF0002636C.namprd02.prod.outlook.com (2603:10b6:806:f3:cafe::74) by SN7PR18CA0003.outlook.office365.com (2603:10b6:806:f3::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6792.20 via Frontend Transport; Thu, 14 Sep 2023 19:15:54 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by SN1PEPF0002636C.mail.protection.outlook.com (10.167.241.137) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6792.20 via Frontend Transport; Thu, 14 Sep 2023 19:15:54 +0000 Received: from driver-dev1.pensando.io (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Thu, 14 Sep 2023 14:15:53 -0500 From: Brett Creeley To: , , , , , CC: , , , Subject: [PATCH vfio 1/3] pds/vfio: Fix spinlock bad magic BUG Date: Thu, 14 Sep 2023 12:15:38 -0700 Message-ID: <20230914191540.54946-2-brett.creeley@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230914191540.54946-1-brett.creeley@amd.com> References: <20230914191540.54946-1-brett.creeley@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN1PEPF0002636C:EE_|BL1PR12MB5240:EE_ X-MS-Office365-Filtering-Correlation-Id: 5e988d62-59f0-4c17-bfae-08dbb55704b8 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: hEIC0XHe7ol3BRzRMyLW0cUEqTso4OXPTGBPC83wIpCmSt5T3eOiOaOkZOpJXW2zPGc+wEQbe2v+cs72dh2zF3tMaKSH7/ew/oN9PLTG6SQhjP7tQz3OOqyxYgnp3WfmBoKjdUowEs+3Z5I0h6m0VDdGVqMQWdbTZ2sNK921q6gl51AN4kxd6wEga22u9FZtVaCBxv075ifkaB3DQR4NcOKbHa+7apq4ktiiqlnW1GfAA0+BiSKgaPs5XlTFqh0GRzRJIYa0LleaLDPqdZTrip3Qzb0qu2rEhi9JOXVpkkMPN7T/gCEEpGolXLGtnaoyD+HZpO1gfpN5a+nyPb/6HuIsnNw+JS5TYTjtVuiTeUCOEAllfDzbn3PXUS5+uxC5X/sumF0L7umtQBGEZ93S9XxR1k+SYeFpryoGMJ5LKlqSEybbCs0vMN6vMJV65lrT5JwZ1rVpThx1YzFgXOVs4vj2gOGJ+/P8M0MfsnvKBRk9cGQuLh0ZkHvMyZq8c69w63B0KyBH77xr/BfRRGY9w4Vwq6VVB2X4HeqDXPhBMrJliIPn2JLtkHZjAoJsthlOjID/zSiXiJlxerFNHib0GhvyO/Kkti8I/ssN3/mWShaxWS4Adw7gBL8mWj5Xf6HrsUBZMg3Y+Q3EB2SPAamAiU2+0N44eWSx9ZKrRtPh/K5Hyh5/Hl5m+3KCZPKqfpxcYzvY46yI93n0SBnM5WeVL99NEtUHZvSJqP0AF14LWcE0mOWI0qbE6oE5QClc5Ezjlqv9sJktQMG43a4FyI9rsg== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(39860400002)(376002)(396003)(136003)(346002)(82310400011)(186009)(1800799009)(451199024)(40470700004)(46966006)(36840700001)(40460700003)(5660300002)(26005)(16526019)(8936002)(8676002)(2616005)(4326008)(1076003)(2906002)(83380400001)(86362001)(81166007)(356005)(82740400003)(36860700001)(47076005)(36756003)(426003)(336012)(40480700001)(44832011)(6666004)(110136005)(54906003)(70206006)(70586007)(478600001)(41300700001)(316002)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Sep 2023 19:15:54.8587 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 5e988d62-59f0-4c17-bfae-08dbb55704b8 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SN1PEPF0002636C.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5240 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Thu, 14 Sep 2023 12:16:08 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777051721775201340 X-GMAIL-MSGID: 1777051721775201340 The following BUG was found when running on a kernel with CONFIG_DEBUG_SPINLOCK=y set: [ 675.116953] BUG: spinlock bad magic on CPU#2, bash/2481 [ 675.116966] lock: 0xffff8d6052a88f50, .magic: 00000000, .owner: /-1, .owner_cpu: 0 [ 675.116978] CPU: 2 PID: 2481 Comm: bash Tainted: G S 6.6.0-rc1-next-20230911 #1 [ 675.116986] Hardware name: HPE ProLiant DL360 Gen10/ProLiant DL360 Gen10, BIOS U32 01/23/2021 [ 675.116991] Call Trace: [ 675.116997] [ 675.117002] dump_stack_lvl+0x36/0x50 [ 675.117014] do_raw_spin_lock+0x79/0xc0 [ 675.117032] pds_vfio_reset+0x1d/0x60 [pds_vfio_pci] [ 675.117049] pci_reset_function+0x4b/0x70 [ 675.117061] reset_store+0x5b/0xa0 [ 675.117074] kernfs_fop_write_iter+0x137/0x1d0 [ 675.117087] vfs_write+0x2de/0x410 [ 675.117101] ksys_write+0x5d/0xd0 [ 675.117111] do_syscall_64+0x3b/0x90 [ 675.117122] entry_SYSCALL_64_after_hwframe+0x6e/0xd8 [ 675.117135] RIP: 0033:0x7f9ebbd1fa28 [ 675.117141] Code: 89 02 48 c7 c0 ff ff ff ff eb b3 0f 1f 80 00 00 00 00 f3 0f 1e fa 48 8d 05 15 4d 2a 00 8b 00 85 c0 75 17 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 58 c3 0f 1f 80 00 00 00 00 41 54 49 89 d4 55 [ 675.117148] RSP: 002b:00007ffdff410728 EFLAGS: 00000246 ORIG_RAX: 0000000000000001 [ 675.117156] RAX: ffffffffffffffda RBX: 0000000000000002 RCX: 00007f9ebbd1fa28 [ 675.117161] RDX: 0000000000000002 RSI: 000055ffc5fdf7c0 RDI: 0000000000000001 [ 675.117166] RBP: 000055ffc5fdf7c0 R08: 000000000000000a R09: 00007f9ebbd7fae0 [ 675.117170] R10: 000000000000000a R11: 0000000000000246 R12: 00007f9ebbfc06e0 [ 675.117174] R13: 0000000000000002 R14: 00007f9ebbfbb860 R15: 0000000000000002 [ 675.117180] As shown, the .magic: 00000000, does not match the expected value. This is because spin_lock_init() is never called for the reset_lock. Fix this by calling spin_lock_init(&pds_vfio->reset_lock) when initializing the device. Signed-off-by: Brett Creeley Reviewed-by: Shannon Nelson --- drivers/vfio/pci/pds/vfio_dev.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/vfio/pci/pds/vfio_dev.c b/drivers/vfio/pci/pds/vfio_dev.c index b46174f5eb09..147a543a7c39 100644 --- a/drivers/vfio/pci/pds/vfio_dev.c +++ b/drivers/vfio/pci/pds/vfio_dev.c @@ -155,6 +155,8 @@ static int pds_vfio_init_device(struct vfio_device *vdev) pds_vfio->vf_id = vf_id; + spin_lock_init(&pds_vfio->reset_lock); + vdev->migration_flags = VFIO_MIGRATION_STOP_COPY | VFIO_MIGRATION_P2P; vdev->mig_ops = &pds_vfio_lm_ops; vdev->log_ops = &pds_vfio_log_ops; From patchwork Thu Sep 14 19:15:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Brett Creeley X-Patchwork-Id: 139907 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp692998vqi; Thu, 14 Sep 2023 16:41:00 -0700 (PDT) X-Google-Smtp-Source: AGHT+IE5UNIDoMF0wfkzY4lMd1NAJpKIEMxEqwNZ+LbBipItRy80n5lEhhG1riufrZTvW9nbMfWi X-Received: by 2002:a05:6870:3906:b0:1d5:b2ba:bc93 with SMTP id b6-20020a056870390600b001d5b2babc93mr201344oap.13.1694734860442; Thu, 14 Sep 2023 16:41:00 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1694734860; cv=pass; d=google.com; s=arc-20160816; b=q7W7X22anpuAdCdieOku2NyH1eAFK/+gXobr4YzoL4dx3u1BlCWkw/5jA8vnzTTujZ J7LGz002o3hLW8Bd4nITb5qOqLIFAy4VVWx5Cl5oXa8X8Rx1n4Oi8VwMZpdUyru8Pqc/ on8Nir4Qd1P4/D3R0DD29XMCb8tWP+PwSVoLJay6qEsXmyj4JEGzvKRx+MGMzYEoH9Yg K0DtVcQoCO3ADZbJyoM75au34nUali+75vA/QS66W9C39doGpIRh2UU2WntsPTgoSLLe UEXPqzT77G0idwfxslosqkfQXd8TUocdF5oJTe1jilHU3DEavUrUUTEYzpKGrjzTs4TE UJ4A== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=GT8A/jR6oeASNo8i5J/kdagx5lvvur+pmWDzd3FILDw=; fh=GbNd0iuihik10v4ZTZpqkH2SVSESc+5QskJdSEY04s4=; b=y455fJau+iu/wSX7Kved9zrOCJqxhEHlg7Hp2IO1JIOPdvBPQsWLSmFTW2Bb/8h3ED Cu387fda2o5GFO7bZ6hRi9Zc2mfUY0WGEcH7UOaBhDZgys8JS0EUA3uOJguM84alT5Yv Z33ViPZUZkWcwMW2UqkYAltxXMtEQ0MkrxTlWtMD7VFm8Z6OajDceYjvPU5C/Oc2Z9j/ teswYFhAKv9hrhPc1v/oLkdifbMxUhflchaBsD/rWkfCHcIhdC6Ge7OWbVTBsIxNbMBX 3HzT2Cgc0mwXxCZT9BwGjH4wE01qNfvsf/EJE0z7vowxZfcB/FB1As9VoweCXeLnW9MQ P+/A== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b=CFHjA8SF; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Received: from lipwig.vger.email (lipwig.vger.email. [23.128.96.33]) by mx.google.com with ESMTPS id w184-20020a6382c1000000b0057779cabc13si2171077pgd.674.2023.09.14.16.40.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Sep 2023 16:41:00 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) client-ip=23.128.96.33; Authentication-Results: mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b=CFHjA8SF; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id 37DDF8053613; Thu, 14 Sep 2023 12:16:43 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241941AbjINTQM (ORCPT + 33 others); Thu, 14 Sep 2023 15:16:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42528 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241838AbjINTQG (ORCPT ); Thu, 14 Sep 2023 15:16:06 -0400 Received: from NAM04-MW2-obe.outbound.protection.outlook.com (mail-mw2nam04on2085.outbound.protection.outlook.com [40.107.101.85]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0555526BC; Thu, 14 Sep 2023 12:16:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=NEr/KN3tm7qEknDJjLO7AKq1oAoTysUwjw4kj6pT+9IR4gcbNKqkn/e2+dgnmqtY6CjfU7eNdYCVW/RlNn7Po2pI4Iu+pn6goD/HiJ0PJIIkjevX2SQBkXtcfgYLsjwZj6UOYxoHHAXhxkB1dgaw+Aod4A2mqmp9TL3HZj7SKBsFoIfDz3frn4gOSeZ33dl4GJLZq6q8MxB7WMJ+EqtSKmeSm+lppj0YpP68bpY34gbpeuMaSnifPIXXNrNNCrV8nZaDbPgejYBNbqXBW9hjZS5zuiSUkEcgml0vj+Ys6a5wDBAJhp32xFnTZzdOe2pIdAyiufAs4pKHjo4im3zH/A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=GT8A/jR6oeASNo8i5J/kdagx5lvvur+pmWDzd3FILDw=; b=TOZ/mDKUy4ap22XqW4pF9e94vixWNukmCZawAddFKgRs6OSnEgOtgQ7LCiC8PgflnjEyFPsZgI9cSzEglXAl5SCkJs9ZyPiZ7VXi7Ya88djR5kHTSulPqssWalDFY3O2QpNY776f9/ow7M7ZGTiwr8chBP0hDt1aamcYLfZkMKUN8TJM+GC2iOrCrbTx8TryCWJ5mD1gkCbTsYsI4mGtnQDWAycd4p1y+2SQH+idy0ABMce9lxEoV55leIi3kIbDONGeUERW6k2tlq/MMCg7CAlJbVZomT+LQlY1S/PiFbRb4xEnFJmCqEPkzC5urD/yy3idOFTuHK3pMMUIl0PQeg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=ziepe.ca smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=GT8A/jR6oeASNo8i5J/kdagx5lvvur+pmWDzd3FILDw=; b=CFHjA8SF89Q83ueGtSlneSO021FBOLJX+/8dDKl6P1hSrYJkl+szboyoIOQnfhDYgR0Fsd4+C7HKIJmT82HmIlegqvpfNdZWZkD74PRGhAEGTGXLpWJOz8FxLWTq1rJFIDM4maC8LabHgvdP7/Yp/guIdOQrY+X5KLF3f3FvyQQ= Received: from SN6PR16CA0053.namprd16.prod.outlook.com (2603:10b6:805:ca::30) by MW3PR12MB4569.namprd12.prod.outlook.com (2603:10b6:303:57::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6792.19; Thu, 14 Sep 2023 19:15:58 +0000 Received: from SN1PEPF0002636B.namprd02.prod.outlook.com (2603:10b6:805:ca:cafe::99) by SN6PR16CA0053.outlook.office365.com (2603:10b6:805:ca::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6792.21 via Frontend Transport; Thu, 14 Sep 2023 19:15:58 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by SN1PEPF0002636B.mail.protection.outlook.com (10.167.241.136) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6792.19 via Frontend Transport; Thu, 14 Sep 2023 19:15:58 +0000 Received: from driver-dev1.pensando.io (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Thu, 14 Sep 2023 14:15:55 -0500 From: Brett Creeley To: , , , , , CC: , , , Subject: [PATCH vfio 2/3] pds/vfio: Fix mutex lock->magic != lock warning Date: Thu, 14 Sep 2023 12:15:39 -0700 Message-ID: <20230914191540.54946-3-brett.creeley@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230914191540.54946-1-brett.creeley@amd.com> References: <20230914191540.54946-1-brett.creeley@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN1PEPF0002636B:EE_|MW3PR12MB4569:EE_ X-MS-Office365-Filtering-Correlation-Id: 12e20de4-aaaa-4dc2-c25a-08dbb55706ad X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: HBiLJxniyV8IMp3ZgaBo7UPfUCZdFvZJY4r+rd9fQnEV8Rtq+Pbvxa5UsSg+ct387DM0xDhCOtJsHF20Efy3sK+UkUD1MWae5rDthtR1/BV7gmMlROUs1u1xj/t/2qMZywFa74ZyjZifTvq96fBXYBooBSim7N+h5BbPM0Xsx3lZY7n/cwJ+3PzeQ1feDhVf3o+7296CdsPsUjpgnuVtoLel6rMIGqPkit3bx3TtHUv+4u9wX7Dl4MV48O4C3KAV+JsWW1j/Q+3jMU5eJKVYtrbweJyPobvSCEmUhh6SAyGnTr3kCuSVqgzMXN3ZLjJbnxMKhSXyOkSqypLufPLdHVtACv43A07ZuB1GdalALWZn8H3wX7c7WRm3N0Sow4CHp0clGduLK0TjbmXFAlM2ONn0u0EiHp1fd7gJJfvjYXlfwR7x7OUW9siz+17dFNUsh8F0PdnW7YOTvTkQd1hs0JP2NB5tqld0ehZhrQwc18r04zXb/FT7c+kEJJLZnIC/Eipw8dxJmbKVvZz91bLBGfKF/bL1AKWXYihVad39D4gGqtjsm30tQ1CIxZGFLCGIJQqyRma3FgIImUUDd23kKp3VEy5e7ZKl1Mjaitpx1POLPrVIe5dM+8Wf8laUNeoUsTrWApn58jz4RJVnOY/rMyz19zw1O68q3VaJsV9RHy5yWXzttzhV5+gGIN1sFfaxcRO7tWmO3L1WDE4x7tWHhxwpN7O9MyOvsyjjPLgkaFjPK71UN9YDC0K0oYRo+oTyFlXBTNXAg+bWJCaWaNmKvA== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(346002)(396003)(39860400002)(136003)(376002)(1800799009)(451199024)(186009)(82310400011)(36840700001)(46966006)(40470700004)(36860700001)(40460700003)(2906002)(47076005)(36756003)(86362001)(82740400003)(81166007)(40480700001)(356005)(4326008)(8936002)(1076003)(8676002)(16526019)(5660300002)(26005)(54906003)(70586007)(110136005)(316002)(2616005)(70206006)(6666004)(41300700001)(45080400002)(478600001)(83380400001)(44832011)(426003)(336012)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Sep 2023 19:15:58.1265 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 12e20de4-aaaa-4dc2-c25a-08dbb55706ad X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SN1PEPF0002636B.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW3PR12MB4569 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Thu, 14 Sep 2023 12:16:43 -0700 (PDT) X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777058300966160125 X-GMAIL-MSGID: 1777058300966160125 The following BUG was found when running on a kernel with CONFIG_DEBUG_MUTEXES=y set: [ 502.794510] DEBUG_LOCKS_WARN_ON(lock->magic != lock) [ 502.794519] WARNING: CPU: 16 PID: 2571 at kernel/locking/mutex.c:1085 mutex_trylock+0x10d/0x120 [ 502.794537] Modules linked in: pds_vfio_pci(OE) pds_core xt_CHECKSUM xt_MASQUERADE xt_conntrack ipt_REJECT nft_compat nf_nat_tftp nf_conntrack_tftp bridge stp llc nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set nf_tables nfnetlink intel_rapl_msr intel_rapl_common isst_if_common nfit libnvdimm x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm crct10dif_pclmul crc32_pclmul ghash_clmulni_intel rapl mei_me ses intel_cstate ipmi_ssif sunrpc enclosure mei nvme intel_uncore nvme_core hpwdt hpilo ioatdma acpi_tad acpi_ipmi pcspkr lpc_ich intel_pch_thermal acpi_power_meter ipmi_si xfs libcrc32c sd_mod t10_pi crc64_rocksoft crc64 sg mgag200 drm_kms_helper drm_shmem_helper crc32c_intel serio_raw igb smartpqi dca drm scsi_transport_sas i2c_algo_bit ionic wmi dm_mirror dm_region_hash dm_log dm_mod ipmi_devintf ipmi_msghandler fuse [last unloaded: pds_core] [ 502.794673] CPU: 16 PID: 2571 Comm: bash Tainted: G S OE 6.6.0-rc1-next-20230911 #1 [ 502.794676] Hardware name: HPE ProLiant DL360 Gen10/ProLiant DL360 Gen10, BIOS U32 01/23/2021 [ 502.794678] RIP: 0010:mutex_trylock+0x10d/0x120 [ 502.794681] Code: ff 85 c0 0f 84 40 ff ff ff 8b 3d f2 35 eb 00 85 ff 0f 85 32 ff ff ff 48 c7 c6 47 ff d3 bc 48 c7 c7 80 69 d3 bc e8 33 c7 4a ff <0f> 0b e9 18 ff ff ff b8 01 00 00 00 e9 5c ff ff ff 66 90 90 90 90 [ 502.794684] RSP: 0018:ffffabadc750fda8 EFLAGS: 00010282 [ 502.794686] RAX: 0000000000000000 RBX: ffff930000451000 RCX: 0000000000000027 [ 502.794688] RDX: 0000000000000027 RSI: ffffabadc750fca0 RDI: ffff930f7f920948 [ 502.794690] RBP: ffff930000451708 R08: 0000000000000000 R09: c0000000ffff7fff [ 502.794692] R10: 0000000000000001 R11: ffffabadc750fc40 R12: 0000000000000000 [ 502.794694] R13: fffffffffffffff2 R14: ffffabadc750fea0 R15: ffff92ffc85dc520 [ 502.794695] FS: 00007f3d8e322740(0000) GS:ffff930f7f900000(0000) knlGS:0000000000000000 [ 502.794697] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 502.794698] CR2: 000055cc4f6d9310 CR3: 0000000109894005 CR4: 00000000007706e0 [ 502.794700] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 502.794701] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 502.794703] PKRU: 55555554 [ 502.794704] Call Trace: [ 502.794707] [ 502.794710] ? __warn+0x85/0x140 [ 502.794718] ? mutex_trylock+0x10d/0x120 [ 502.794720] ? report_bug+0xfc/0x1e0 [ 502.794727] ? handle_bug+0x3f/0x70 [ 502.794732] ? exc_invalid_op+0x17/0x70 [ 502.794735] ? asm_exc_invalid_op+0x1a/0x20 [ 502.794742] ? mutex_trylock+0x10d/0x120 [ 502.794744] ? mutex_trylock+0x10d/0x120 [ 502.794748] pds_vfio_reset+0x3a/0x60 [pds_vfio_pci] [ 502.794756] pci_reset_function+0x4b/0x70 [ 502.794763] reset_store+0x5b/0xa0 [ 502.794770] kernfs_fop_write_iter+0x137/0x1d0 [ 502.794776] vfs_write+0x2de/0x410 [ 502.794784] ksys_write+0x5d/0xd0 [ 502.794787] do_syscall_64+0x3b/0x90 [ 502.794790] entry_SYSCALL_64_after_hwframe+0x6e/0xd8 [ 502.794794] RIP: 0033:0x7f3d8d51fa28 [ 502.794796] Code: 89 02 48 c7 c0 ff ff ff ff eb b3 0f 1f 80 00 00 00 00 f3 0f 1e fa 48 8d 05 15 4d 2a 00 8b 00 85 c0 75 17 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 58 c3 0f 1f 80 00 00 00 00 41 54 49 89 d4 55 [ 502.794799] RSP: 002b:00007ffc7b6c76f8 EFLAGS: 00000246 ORIG_RAX: 0000000000000001 [ 502.794801] RAX: ffffffffffffffda RBX: 0000000000000002 RCX: 00007f3d8d51fa28 [ 502.794803] RDX: 0000000000000002 RSI: 000055cc4f742aa0 RDI: 0000000000000001 [ 502.794804] RBP: 000055cc4f742aa0 R08: 000000000000000a R09: 00007f3d8d57fae0 [ 502.794805] R10: 000000000000000a R11: 0000000000000246 R12: 00007f3d8d7c06e0 [ 502.794807] R13: 0000000000000002 R14: 00007f3d8d7bb860 R15: 0000000000000002 [ 502.794809] As shown, lock->magic != lock. This is because mutex_init(&pds_vfio->state_mutex) is called in the VFIO open path. So, if a reset is initiated before the VFIO device is opened the mutex will have never been initialized. Fix this by calling mutex_init(&pds_vfio->state_mutex) in the VFIO init path. Signed-off-by: Brett Creeley Reviewed-by: Shannon Nelson --- drivers/vfio/pci/pds/vfio_dev.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/vfio/pci/pds/vfio_dev.c b/drivers/vfio/pci/pds/vfio_dev.c index 147a543a7c39..9db5f2c8f1ea 100644 --- a/drivers/vfio/pci/pds/vfio_dev.c +++ b/drivers/vfio/pci/pds/vfio_dev.c @@ -155,6 +155,7 @@ static int pds_vfio_init_device(struct vfio_device *vdev) pds_vfio->vf_id = vf_id; + mutex_init(&pds_vfio->state_mutex); spin_lock_init(&pds_vfio->reset_lock); vdev->migration_flags = VFIO_MIGRATION_STOP_COPY | VFIO_MIGRATION_P2P; @@ -181,7 +182,6 @@ static int pds_vfio_open_device(struct vfio_device *vdev) if (err) return err; - mutex_init(&pds_vfio->state_mutex); pds_vfio->state = VFIO_DEVICE_STATE_RUNNING; pds_vfio->deferred_reset_state = VFIO_DEVICE_STATE_RUNNING; From patchwork Thu Sep 14 19:15:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Brett Creeley X-Patchwork-Id: 140116 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp785950vqi; Thu, 14 Sep 2023 20:52:00 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEg8cL9u+XvuKNh7fM7IUkK648KH5H5FnMYlq5Ho55CfT48pqTHSqI5aamnuBOQJodfAqKw X-Received: by 2002:a17:90a:6fa6:b0:263:72c5:9ac6 with SMTP id e35-20020a17090a6fa600b0026372c59ac6mr375144pjk.11.1694749920179; Thu, 14 Sep 2023 20:52:00 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1694749920; cv=pass; d=google.com; s=arc-20160816; b=kgzEGm40aTyus61JoiSzlVJ18b4SIf4AryvH1lkz03Te5bdoLIB/uJkWsgmYw+6luD J3ithOOLfAuGCkI+uUHcIJrssqqv+jlHa0ONm0tGTJbxYqSwiV+3PhDrFyD3r6Y+wJm3 WPp8gyILSvTIl4VtvTeH47EBHgWeA6nr9Ei5cFQzr9ezgCLM9IBg9AYNrjidVScjWj8I dXnFPceW3dfdPfnE2ctX/vMF47U3t0sJS2Jx8Dgo8jPZX2L4x/yiC/xBUYt+8trxmDKh e6aOi/sDOwnbgs7QArH98lyU+kjCPWTJb0qgvyH8eky96tc4wBbWR9lq3GnOxU294sO6 orRg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=Pp3QuZ8rzxJqRmpkeTin7yKP+tcMvozhs31JUFgV0oA=; fh=GbNd0iuihik10v4ZTZpqkH2SVSESc+5QskJdSEY04s4=; b=f5klxEQ++20t9gYDMBbQhydkQOIn00Gk2XmLc/3HxTx0zDxoTNg7SLVbH2jaTplt+m TBjQ0AfF01/5uXVE7n9juXlfw1cX1ncUOoNLwQCflZ+gtyrpVIywSAx2koAEhLPGLMh2 OpYYCvWalE/mB/uldUlpiIby9bnvhen3xNwz9lmTxSi76ZV5Sax9F96rMSssa4aGgZoe 2XFy7JCygBJHx+tE98UdHxtjrMS1o9WcSS3ZCxpeEVfdf36w/9fmP18Y7seDL/omHX0G ygNicqfb5GSEhv8n6QiW/elbgPC0EQTOap43iZJI5vbIPQAP/feaGQu+RmRcDg/HKxqK RUSg== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b=Ge3K3Oo6; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Received: from agentk.vger.email (agentk.vger.email. [23.128.96.32]) by mx.google.com with ESMTPS id v7-20020a17090a088700b002741437d773si4843673pjc.176.2023.09.14.20.51.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Sep 2023 20:52:00 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) client-ip=23.128.96.32; Authentication-Results: mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b=Ge3K3Oo6; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id 26D8980B7AC5; Thu, 14 Sep 2023 12:16:48 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241925AbjINTQK (ORCPT + 33 others); Thu, 14 Sep 2023 15:16:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42542 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241840AbjINTQG (ORCPT ); Thu, 14 Sep 2023 15:16:06 -0400 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2082.outbound.protection.outlook.com [40.107.244.82]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A023326BB; Thu, 14 Sep 2023 12:16:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=I1Z5Up1XvfsXAr0FpUmk7tvgPXqm/YQZuaRdKC1Z23OAIkfdfXNIO4Gx5OR299GwzhPg9QeI/HJ1LwVwsiRrnSGkMyr90e6JDAgn/JNAYDlry+OOLin65s4DVpm9ZSTRswnmjPX188OM4R1TgXYlI3Wh+R7gq7PLlL/uV+ETS68Htre9vjStkGu02wBgI0zFX7D+uPh6vT53pkxxYZOaJMdzuJG5YvSlyMOo2+GSv9aVd7HsU8ROsw9v15zQDiZlLfeI14pWB5NQvCLaE3zVCPweZt9DY6MxWaPrYvkLd04RPD2imm+SE7/86jeU0s/1ccSr8tcYNr9dJGaEvPEHmw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Pp3QuZ8rzxJqRmpkeTin7yKP+tcMvozhs31JUFgV0oA=; b=RpU2UIeCn2QpCxgdm5s0KU6gLFnseasDIwMeVnMnkTxeaBpGjlhpZ/uV1MWfeH90PF4fo7MZhRPtrfufqIgZKqVnruR99/DrEzQBxU7dkuZPESSJt2IbYYV5PeHV2RbAWKFsd+nZTE932RiBDtga191iytBwMzd22dTSo+YCTcG31EPDiJxLyXLJakZvQcf0jrq98vLHGa1e3BZ1ZBz5r18Vuyx8EFmOJZDNO/Apipr6v5oPPfdNYnnxSVd0jl7h90smWnlK6kvvh2v5sb9j/3T/ZI1CtyhNiuJvWdjG2WJL/qoivuMD2iZpWYOT5ickA8dE2c67Y6kKS6+zeVi3FQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=ziepe.ca smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Pp3QuZ8rzxJqRmpkeTin7yKP+tcMvozhs31JUFgV0oA=; b=Ge3K3Oo6/CmKeP3eYTye3qM7M9a/QQzOttVObW68sjrPyWjeQXKvfVxxKe+gYEqJDGdXvrNjMYYtuWJVvI9fnTVuZBlp5K/Bp1bTyGbo222cJu/GIXBJ4oAjjIf1lIqXSTK6UZsLr3GXMFlP9FGAm3XL/XHqAmh9tj1hdJ7WlDQ= Received: from SN6PR16CA0037.namprd16.prod.outlook.com (2603:10b6:805:ca::14) by DM4PR12MB7575.namprd12.prod.outlook.com (2603:10b6:8:10d::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6768.31; Thu, 14 Sep 2023 19:15:59 +0000 Received: from SN1PEPF0002636B.namprd02.prod.outlook.com (2603:10b6:805:ca:cafe::3d) by SN6PR16CA0037.outlook.office365.com (2603:10b6:805:ca::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6792.20 via Frontend Transport; Thu, 14 Sep 2023 19:15:59 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by SN1PEPF0002636B.mail.protection.outlook.com (10.167.241.136) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6792.19 via Frontend Transport; Thu, 14 Sep 2023 19:15:59 +0000 Received: from driver-dev1.pensando.io (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Thu, 14 Sep 2023 14:15:57 -0500 From: Brett Creeley To: , , , , , CC: , , , Subject: [PATCH vfio 3/3] pds/vfio: Fix possible sleep while in atomic context Date: Thu, 14 Sep 2023 12:15:40 -0700 Message-ID: <20230914191540.54946-4-brett.creeley@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230914191540.54946-1-brett.creeley@amd.com> References: <20230914191540.54946-1-brett.creeley@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN1PEPF0002636B:EE_|DM4PR12MB7575:EE_ X-MS-Office365-Filtering-Correlation-Id: bf444e5a-1ba0-4fc1-0ab4-08dbb5570738 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: XYU2Dnxcgyu+tiRtTBowcmQfUAPs0KgnXXSuJgc8x95f8HJGP5ulpSRHbhlocYjL9HxjaF4DfvBgjSqRzoI1fgFkUhGWx8SyJswZDNqEtuXqsUsl3jIkUsMlCdacOxk/MaeSu00LmLPStBex4Y7o86+bz9+rEVd9TdPx8XjSJuDxBT6IzQV8ECZ1QJ8mptc0aPdietsLZ4lvc2wXYD5G94FcJ3ue6wYFyrkWUhE2JKOMbPgp7vikQPr3D4ck8giBSnJhlgAy3FmxMKB21OOFmqDgSCcF0+ellISkShaAKEn1LszIRxzvH0PLW+MtiXA1BFGl4WPcVcKrSnkeZJRhSSLvRKgU/MKpRadSCkQ73iSmGDFYnxY+OpJitvm1sPSvaxA20QqO/LPHU+F+PH940HBlBlbcv+9Hu/0hCC6dEk8gDNEJ4x/BbSVoxjXueDytuL4PuFcXM0/LarPtjn5Uz16EjIKfQUnEDNc62+YSzkVuwaHDG05AF00IXBh6ZyNbT0BjH78JjqLIRVtBfmAr3dOtMHnu1Qz55Rbs+9gMTVBDpbMOVnT4Vlx+/WYU5DT/cjWJZWmJCsHhG8t10TCT8lYZJGmwIwKOgvd/8rcPHf47V7wnq6l2QNOWBiLgnfilTu/lkt2QLNEARU2puUQKWGHpxC7EcCGVwOo+duy9q5nQy9f1+rOl+DxxRuZKy4dSarStKuXiSL2uoTL+V4YB3h+U1T93ztOYWoiNrn4WHIuiS3kvPk/uxqKX2JA7Ui7lwgYMzfGLOkDw7OWhO298qQ== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(136003)(39860400002)(396003)(376002)(346002)(82310400011)(451199024)(1800799009)(186009)(46966006)(40470700004)(36840700001)(26005)(16526019)(426003)(40480700001)(316002)(41300700001)(110136005)(44832011)(54906003)(70586007)(70206006)(478600001)(5660300002)(966005)(2906002)(336012)(4326008)(8676002)(8936002)(1076003)(40460700003)(6666004)(36756003)(36860700001)(47076005)(2616005)(86362001)(82740400003)(83380400001)(81166007)(356005)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Sep 2023 19:15:59.0327 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: bf444e5a-1ba0-4fc1-0ab4-08dbb5570738 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SN1PEPF0002636B.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB7575 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Thu, 14 Sep 2023 12:16:48 -0700 (PDT) X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777074092297745267 X-GMAIL-MSGID: 1777074092297745267 The driver could possibly sleep while in atomic context resulting in the following call trace while CONFIG_DEBUG_ATOMIC_SLEEP=y is set: [ 227.229806] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:283 [ 227.229818] in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 2817, name: bash [ 227.229824] preempt_count: 1, expected: 0 [ 227.229827] RCU nest depth: 0, expected: 0 [ 227.229832] CPU: 5 PID: 2817 Comm: bash Tainted: G S OE 6.6.0-rc1-next-20230911 #1 [ 227.229839] Hardware name: HPE ProLiant DL360 Gen10/ProLiant DL360 Gen10, BIOS U32 01/23/2021 [ 227.229843] Call Trace: [ 227.229848] [ 227.229853] dump_stack_lvl+0x36/0x50 [ 227.229865] __might_resched+0x123/0x170 [ 227.229877] mutex_lock+0x1e/0x50 [ 227.229891] pds_vfio_put_lm_file+0x1e/0xa0 [pds_vfio_pci] [ 227.229909] pds_vfio_put_save_file+0x19/0x30 [pds_vfio_pci] [ 227.229923] pds_vfio_state_mutex_unlock+0x2e/0x80 [pds_vfio_pci] [ 227.229937] pci_reset_function+0x4b/0x70 [ 227.229948] reset_store+0x5b/0xa0 [ 227.229959] kernfs_fop_write_iter+0x137/0x1d0 [ 227.229972] vfs_write+0x2de/0x410 [ 227.229986] ksys_write+0x5d/0xd0 [ 227.229996] do_syscall_64+0x3b/0x90 [ 227.230004] entry_SYSCALL_64_after_hwframe+0x6e/0xd8 [ 227.230017] RIP: 0033:0x7fb202b1fa28 [ 227.230023] Code: 89 02 48 c7 c0 ff ff ff ff eb b3 0f 1f 80 00 00 00 00 f3 0f 1e fa 48 8d 05 15 4d 2a 00 8b 00 85 c0 75 17 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 58 c3 0f 1f 80 00 00 00 00 41 54 49 89 d4 55 [ 227.230028] RSP: 002b:00007fff6915fbd8 EFLAGS: 00000246 ORIG_RAX: 0000000000000001 [ 227.230036] RAX: ffffffffffffffda RBX: 0000000000000002 RCX: 00007fb202b1fa28 [ 227.230040] RDX: 0000000000000002 RSI: 000055f3834d5aa0 RDI: 0000000000000001 [ 227.230044] RBP: 000055f3834d5aa0 R08: 000000000000000a R09: 00007fb202b7fae0 [ 227.230047] R10: 000000000000000a R11: 0000000000000246 R12: 00007fb202dc06e0 [ 227.230050] R13: 0000000000000002 R14: 00007fb202dbb860 R15: 0000000000000002 [ 227.230056] This can happen if pds_vfio_put_restore_file() and/or pds_vfio_put_save_file() grab the mutex_lock(&lm_file->lock) while the spin_lock(&pds_vfio->reset_lock) is held, which can happen during while calling pds_vfio_state_mutex_unlock(). Fix this by releasing the spin_unlock(&pds_vfio->reset_lock) before calling pds_vfio_put_restore_file() and pds_vfio_put_save_file() and re-acquiring spin_lock(&pds_vfio->reset_lock) after the previously mentioned functions are called to protect setting the subsequent state/deferred reset settings. The only possible concerns are other threads that may call pds_vfio_put_restore_file() and/or pds_vfio_put_save_file(). However, those paths are already protected by the state mutex_lock(). Reported-by: Dan Carpenter Closes: https://lore.kernel.org/kvm/1f9bc27b-3de9-4891-9687-ba2820c1b390@moroto.mountain/ Signed-off-by: Brett Creeley Reviewed-by: Shannon Nelson --- drivers/vfio/pci/pds/vfio_dev.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/vfio/pci/pds/vfio_dev.c b/drivers/vfio/pci/pds/vfio_dev.c index 9db5f2c8f1ea..6e664cb05dd1 100644 --- a/drivers/vfio/pci/pds/vfio_dev.c +++ b/drivers/vfio/pci/pds/vfio_dev.c @@ -33,8 +33,10 @@ void pds_vfio_state_mutex_unlock(struct pds_vfio_pci_device *pds_vfio) if (pds_vfio->deferred_reset) { pds_vfio->deferred_reset = false; if (pds_vfio->state == VFIO_DEVICE_STATE_ERROR) { + spin_unlock(&pds_vfio->reset_lock); pds_vfio_put_restore_file(pds_vfio); pds_vfio_put_save_file(pds_vfio); + spin_lock(&pds_vfio->reset_lock); pds_vfio_dirty_disable(pds_vfio, false); } pds_vfio->state = pds_vfio->deferred_reset_state;