From patchwork Tue Dec 13 12:11:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xiubo Li X-Patchwork-Id: 32790 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:e747:0:0:0:0:0 with SMTP id c7csp83200wrn; Tue, 13 Dec 2022 04:19:38 -0800 (PST) X-Google-Smtp-Source: AA0mqf6LrnAWsON/ZdEDXY8LTLcx2QiUZLRXaAxiylsTvdYDRe7S8XDb9ML8IYswdjbQd1ODX3YW X-Received: by 2002:a17:907:8d0c:b0:7c1:37f3:e982 with SMTP id tc12-20020a1709078d0c00b007c137f3e982mr19661785ejc.12.1670933977857; Tue, 13 Dec 2022 04:19:37 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670933977; cv=none; d=google.com; s=arc-20160816; b=DBe7ckyflpA6fVavLiMouFBZcL2DdB0DV5Ll3uojMCmXrA5Ulskc9cIQvHQnRl5uQ6 LbeRh5ZcH13T1I0DbALNvYzxfn60pUZRS99qL2jZqgXJhxKLKA+XVWJ8myOfmjGHG3lw hj3gF4xo/C3ct3EVayU/PXL8Jfm1xsh5o/9RpAu13BDbZqlSgfanM8m+HQhJly1LbUpX hm7oMX4jszCAvuz/Aeo+GbHjCLJu+Ufvow9xa+tUzzLwNnojQKxjF+4RCdC2PjiLFIje JgJlmPSZrvkVcUblx/h8nQgtPJ6vHTF8rkwFBJFM5Up1/lPqiat4PPqx+lfSnG/ZVpQr A2jw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=kL29FbHgsP8X987LWjgI5myMeBq1Iws5MCaohqIReFk=; b=llzDS+PJDAt4pa2XhjHdYC+XPtkJrGgVe3fcuhiv+9XdS06Vr6USaPSztb7Mh7Ukkb pprPBuoDkzTmSArfME4631bUcuXbbuTAoTD7O93gFLt9ehtHdPECIlsN7T4qqYyzUmRI wC4UmNxBQ7ATx7gVnd7rpq9wn6oDhRM3VSkLeNm/b1tpau/rS3RPZ1lK8l0p/1WlrG/W myWqxl1jjmGDSamqe/3OOPFRtzcmri3AKXsgHyRspV/C758XB9yHTMJPL4RTmXy+Ctjf 2rMw7ybOl3e36btiBMHsWCJZZUxDkwuhkxl1rrX0pDv2QrsEWKZEC53P5q0J+8aSGlbJ bHSA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=eHsbyYG9; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id hv14-20020a17090760ce00b0078e063fc78csi9650194ejc.576.2022.12.13.04.19.13; Tue, 13 Dec 2022 04:19:37 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=eHsbyYG9; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235577AbiLMMM1 (ORCPT + 99 others); Tue, 13 Dec 2022 07:12:27 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60530 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235513AbiLMMMK (ORCPT ); Tue, 13 Dec 2022 07:12:10 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 874F614036 for ; Tue, 13 Dec 2022 04:11:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1670933482; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kL29FbHgsP8X987LWjgI5myMeBq1Iws5MCaohqIReFk=; b=eHsbyYG9TihM1VQr7wJNEXYOFxzHLykNc3bWGnXq7xqBrsXGRDjia1TSgOiYen4xMiB/hz j2vFY7CDNj0NxQkDPVeGRVKa63XdSbrkBbN1j6529vBEj1E+tW2kOULDBKy8Muz2VnST93 nosfVWv+kamdItJ/FJd+b9klumdShVw= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-664-naIg6KinOoWMmFXX1YzMlA-1; Tue, 13 Dec 2022 07:11:17 -0500 X-MC-Unique: naIg6KinOoWMmFXX1YzMlA-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 02ACA3810D25; Tue, 13 Dec 2022 12:11:17 +0000 (UTC) Received: from lxbceph1.gsslab.pek2.redhat.com (unknown [10.72.47.117]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3252F40AE1E9; Tue, 13 Dec 2022 12:11:12 +0000 (UTC) From: xiubli@redhat.com To: jlayton@kernel.org, idryomov@gmail.com, ceph-devel@vger.kernel.org Cc: mchangir@redhat.com, lhenriques@suse.de, viro@zeniv.linux.org.uk, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, Xiubo Li , stable@vger.kernel.org Subject: [PATCH v4 1/2] ceph: switch to vfs_inode_has_locks() to fix file lock bug Date: Tue, 13 Dec 2022 20:11:02 +0800 Message-Id: <20221213121103.213631-2-xiubli@redhat.com> In-Reply-To: <20221213121103.213631-1-xiubli@redhat.com> References: <20221213121103.213631-1-xiubli@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1752101266740150053?= X-GMAIL-MSGID: =?utf-8?q?1752101266740150053?= From: Xiubo Li For the POSIX locks they are using the same owner, which is the thread id. And multiple POSIX locks could be merged into single one, so when checking whether the 'file' has locks may fail. For a file where some openers use locking and others don't is a really odd usage pattern though. Locks are like stoplights -- they only work if everyone pays attention to them. Just switch ceph_get_caps() to check whether any locks are set on the inode. If there are POSIX/OFD/FLOCK locks on the file at the time, we should set CHECK_FILELOCK, regardless of what fd was used to set the lock. Cc: stable@vger.kernel.org Fixes: ff5d913dfc71 ("ceph: return -EIO if read/write against filp that lost file locks") Cc: Jeff Layton Signed-off-by: Xiubo Li --- fs/ceph/caps.c | 2 +- fs/ceph/locks.c | 4 ---- fs/ceph/super.h | 1 - 3 files changed, 1 insertion(+), 6 deletions(-) diff --git a/fs/ceph/caps.c b/fs/ceph/caps.c index 065e9311b607..948136f81fc8 100644 --- a/fs/ceph/caps.c +++ b/fs/ceph/caps.c @@ -2964,7 +2964,7 @@ int ceph_get_caps(struct file *filp, int need, int want, loff_t endoff, int *got while (true) { flags &= CEPH_FILE_MODE_MASK; - if (atomic_read(&fi->num_locks)) + if (vfs_inode_has_locks(inode)) flags |= CHECK_FILELOCK; _got = 0; ret = try_get_cap_refs(inode, need, want, endoff, diff --git a/fs/ceph/locks.c b/fs/ceph/locks.c index 3e2843e86e27..b191426bf880 100644 --- a/fs/ceph/locks.c +++ b/fs/ceph/locks.c @@ -32,18 +32,14 @@ void __init ceph_flock_init(void) static void ceph_fl_copy_lock(struct file_lock *dst, struct file_lock *src) { - struct ceph_file_info *fi = dst->fl_file->private_data; struct inode *inode = file_inode(dst->fl_file); atomic_inc(&ceph_inode(inode)->i_filelock_ref); - atomic_inc(&fi->num_locks); } static void ceph_fl_release_lock(struct file_lock *fl) { - struct ceph_file_info *fi = fl->fl_file->private_data; struct inode *inode = file_inode(fl->fl_file); struct ceph_inode_info *ci = ceph_inode(inode); - atomic_dec(&fi->num_locks); if (atomic_dec_and_test(&ci->i_filelock_ref)) { /* clear error when all locks are released */ spin_lock(&ci->i_ceph_lock); diff --git a/fs/ceph/super.h b/fs/ceph/super.h index 14454f464029..e7662ff6f149 100644 --- a/fs/ceph/super.h +++ b/fs/ceph/super.h @@ -804,7 +804,6 @@ struct ceph_file_info { struct list_head rw_contexts; u32 filp_gen; - atomic_t num_locks; }; struct ceph_dir_file_info { From patchwork Tue Dec 13 12:11:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xiubo Li X-Patchwork-Id: 32791 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:e747:0:0:0:0:0 with SMTP id c7csp83214wrn; Tue, 13 Dec 2022 04:19:41 -0800 (PST) X-Google-Smtp-Source: AA0mqf5OvMCDoboe6pW3UGYXgiRD5eOuYCn796ISjydYBH29QR/Rz9ZUztl3/fF9ES9t7tnk4MtV X-Received: by 2002:a17:906:af6a:b0:7c1:12ef:bf52 with SMTP id os10-20020a170906af6a00b007c112efbf52mr15795848ejb.3.1670933981239; Tue, 13 Dec 2022 04:19:41 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670933981; cv=none; d=google.com; s=arc-20160816; b=JHTCD6nmFf3NgySE0lRoZ9iuXfZOEQRPHUy5adblPu5IXATdJ641lA3mBueIxRcMqt FJOjFL4X8cXEbZLQ1R50eo9W2kER3PuD1s/LYTXmb8Ere8ULULmkNkSJkj7R3axU6fD7 lbq9DwVhIxzypVtdlq2U1h2VYtKSiUrwvd6dhykfmS8WWzqSX78aRIrlE8uJeBGhX9eY UAoNCM21JdaLgJy2Ep+Cq7YQaKp/cQX7VoS2WmYwDvkUs43rdC2mMOLYtUMSRM7YRG3H v/McOaGBaqlH9EZYUo6A3jX31Ih2dWaACjWsyTCHi+ZncJirlaU7CSAk+chJGXkVevQA a+yQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=RAI9mgdskd1ADV22NmxESjpEH55uCsxgWZtJxV8d/1E=; b=eG857eNRi0dwpb4Fy+1yc6yINNeF2Z6mhHNfKxdBnCDV/3FLN2ftxOhM/9FTLx8l0Y r4POicLO1tVQsom5AhZf9vspObeWcoABcLuk1f2E3oyfQ7ND20v7Y3YXWhBf7LdLM6d7 xtECpHYxp8CDjPr6ON0DvF+MGotPaMKJA8mIm8XtPmjYWLZT3OqR9NUttUC7lins4Yej Y5jDmnBCqUznNt/3KtfBoH2BjkoLUCKcOSVYTOt6m8Rf0YJ9w4i/hf7lrJGgWcq69AiR GWIfG78IFiyv5zmvNAq+Ghk6shPdVvJkQPH2raht0LGfl7IPG5rj01TurIpWW1TxoO7a 4CtQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=JZkAan4J; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id gn33-20020a1709070d2100b007c0b61436efsi9068791ejc.1008.2022.12.13.04.19.17; Tue, 13 Dec 2022 04:19:41 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=JZkAan4J; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235625AbiLMMNK (ORCPT + 99 others); Tue, 13 Dec 2022 07:13:10 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60712 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235595AbiLMMMr (ORCPT ); Tue, 13 Dec 2022 07:12:47 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B081F16485 for ; Tue, 13 Dec 2022 04:11:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1670933487; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RAI9mgdskd1ADV22NmxESjpEH55uCsxgWZtJxV8d/1E=; b=JZkAan4J/TA51ebtiapW/NAgOjvRrO94U5QFGsmr71CD6X+uzs3nMBteUHe8fsO6p7pPZP B7XToX6renyfaEDSxu89bdjIeG9UU8/wNGXqPMo+ekl7jNocRcft8bDTppZtWr7IBCrMb4 wmHfHZlGUP1LUt+ULGp6TUzBSi6HqIY= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-267-dvOezT1eOcaie-ZF4hHeYw-1; Tue, 13 Dec 2022 07:11:22 -0500 X-MC-Unique: dvOezT1eOcaie-ZF4hHeYw-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 6CBB23C0F425; Tue, 13 Dec 2022 12:11:21 +0000 (UTC) Received: from lxbceph1.gsslab.pek2.redhat.com (unknown [10.72.47.117]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9CC3640AE1F4; Tue, 13 Dec 2022 12:11:17 +0000 (UTC) From: xiubli@redhat.com To: jlayton@kernel.org, idryomov@gmail.com, ceph-devel@vger.kernel.org Cc: mchangir@redhat.com, lhenriques@suse.de, viro@zeniv.linux.org.uk, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, Xiubo Li , stable@vger.kernel.org Subject: [PATCH v4 2/2] ceph: add ceph specific member support for file_lock Date: Tue, 13 Dec 2022 20:11:03 +0800 Message-Id: <20221213121103.213631-3-xiubli@redhat.com> In-Reply-To: <20221213121103.213631-1-xiubli@redhat.com> References: <20221213121103.213631-1-xiubli@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1752101270628393367?= X-GMAIL-MSGID: =?utf-8?q?1752101270628393367?= From: Xiubo Li When ceph releasing the file_lock it will try to get the inode pointer from the fl->fl_file, which the memory could already be released by another thread in filp_close(). Because in VFS layer the fl->fl_file doesn't increase the file's reference counter. Will switch to use ceph dedicate lock info to track the inode. And in ceph_fl_release_lock() we should skip all the operations if the fl->fl_u.ceph_fl.fl_inode is not set, which should come from the request file_lock. And we will set fl->fl_u.ceph_fl.fl_inode when inserting it to the inode lock list, which is when copying the lock. Cc: stable@vger.kernel.org Cc: Jeff Layton URL: https://tracker.ceph.com/issues/57986 Signed-off-by: Xiubo Li --- fs/ceph/locks.c | 20 ++++++++++++++++++-- include/linux/fs.h | 3 +++ 2 files changed, 21 insertions(+), 2 deletions(-) diff --git a/fs/ceph/locks.c b/fs/ceph/locks.c index b191426bf880..cf78608a3f9a 100644 --- a/fs/ceph/locks.c +++ b/fs/ceph/locks.c @@ -34,18 +34,34 @@ static void ceph_fl_copy_lock(struct file_lock *dst, struct file_lock *src) { struct inode *inode = file_inode(dst->fl_file); atomic_inc(&ceph_inode(inode)->i_filelock_ref); + dst->fl_u.ceph.fl_inode = igrab(inode); } +/* + * Do not use the 'fl->fl_file' in release function, which + * is possibly already released by another thread. + */ static void ceph_fl_release_lock(struct file_lock *fl) { - struct inode *inode = file_inode(fl->fl_file); - struct ceph_inode_info *ci = ceph_inode(inode); + struct inode *inode = fl->fl_u.ceph.fl_inode; + struct ceph_inode_info *ci; + + /* + * If inode is NULL it should be a request file_lock, + * nothing we can do. + */ + if (!inode) + return; + + ci = ceph_inode(inode); if (atomic_dec_and_test(&ci->i_filelock_ref)) { /* clear error when all locks are released */ spin_lock(&ci->i_ceph_lock); ci->i_ceph_flags &= ~CEPH_I_ERROR_FILELOCK; spin_unlock(&ci->i_ceph_lock); } + fl->fl_u.ceph.fl_inode = NULL; + iput(inode); } static const struct file_lock_operations ceph_fl_lock_ops = { diff --git a/include/linux/fs.h b/include/linux/fs.h index 7b52fdfb6da0..6106374f5257 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -1119,6 +1119,9 @@ struct file_lock { int state; /* state of grant or error if -ve */ unsigned int debug_id; } afs; + struct { + struct inode *fl_inode; + } ceph; } fl_u; } __randomize_layout;