From patchwork Fri Nov 18 02:06:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xiubo Li X-Patchwork-Id: 22060 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp734971wrr; Thu, 17 Nov 2022 18:15:02 -0800 (PST) X-Google-Smtp-Source: AA0mqf6io+p6UgMm4i3zWRImF1Lht/WEjzB0dyO9HyIA5sZm0Trk8zicrhVaUSSwEFaHCBnrtgyF X-Received: by 2002:a05:6402:b81:b0:45c:a651:8849 with SMTP id cf1-20020a0564020b8100b0045ca6518849mr4470564edb.209.1668737702603; Thu, 17 Nov 2022 18:15:02 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1668737702; cv=none; d=google.com; s=arc-20160816; b=fl0qSvpYeVv0+7msnIboSRZrloXeTT3HpxPzQJvpsVaOvYeQjG+7oQCnDWp2ROwqSV UtAu6zD570FgNqfTbfvk2RNJwtUx9o5SSjHMjCUmxVDK1k/2HsuZ0BczC6zEbeCs+sSN bxT0coyI+6B3O2tJJl83+2SqruWcn9fXU0Kl5FIa4YOvsBWanZPWcQY/XV4ZVhO5lqbK gSPEROz+0aEdZcriOkhwb5TQcjTpACOFzKhDsvqQfTFRva1bHoFOKsq6o6XpfBiFptsd F+vMf5w8B9umMNbrz29vtLJ4IpPSePI5iXyVImwkbZyCuLRdZcT5OM/PJ4Hvj7IoBlMb 4Cvg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=E50ouKsSOSWKlmi2VnrMWYeqEQ0C3pYJnQUUUioyHwY=; b=KCyLkR7enV3221kv6giMKzmiRkvrCLPKTlZJc03CX1InaD2X7qJTPr94/f3e2mUCSb cOKr3RAmDXsShzGGzHOY9kkZYVmI8e3EU9vt7YVAMFX/BjBXJ5INNrZhC8UBSKBXPEIf dKDvvWnGkbYnj03mNuzcH1Pqd1bc2S50ghaKaSD4BJ2jTbNJLz9iNYOIjojbYEVDaPAp u1beLN+iLqioqOV7b9m46/pwL5X0MUVGtNR6F2Ydp76V2CFD3uLFYs3l95QSjalq5OXA fXoEVVk9HMdeedi+8pBu2XCZO0vJQxeARn5VJQQZvlSn5+kuGnAW3AKXKGi54kN2iejf oQVA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=ZnKPmedu; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id u1-20020a1709064ac100b007ae21bbb010si1622035ejt.462.2022.11.17.18.14.37; Thu, 17 Nov 2022 18:15:02 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=ZnKPmedu; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241053AbiKRCIK (ORCPT + 99 others); Thu, 17 Nov 2022 21:08:10 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38490 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240957AbiKRCH6 (ORCPT ); Thu, 17 Nov 2022 21:07:58 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5585287A77 for ; Thu, 17 Nov 2022 18:06:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1668737215; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=E50ouKsSOSWKlmi2VnrMWYeqEQ0C3pYJnQUUUioyHwY=; b=ZnKPmedutEI0o7LyWvPebiH6H454Kbro5IWcrLclQhKyicGoBTNdbHVmnKE4XvpJ9vxqYg l0NarrK8Zo2btx7i4mzE8xDeV6Q1zVjU2q94+VhMYH5xZtMbYHmZWqfqykBB/vz50Rrp+0 p3tNdW0o6qzlQGzns/Mu9B/UsMa1BTk= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-85-6ZsLk-XWP5GCXGHwuIlnHA-1; Thu, 17 Nov 2022 21:06:54 -0500 X-MC-Unique: 6ZsLk-XWP5GCXGHwuIlnHA-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 6A51F101A54E; Fri, 18 Nov 2022 02:06:53 +0000 (UTC) Received: from lxbceph1.gsslab.pek2.redhat.com (unknown [10.72.47.117]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9B18FC158CF; Fri, 18 Nov 2022 02:06:49 +0000 (UTC) From: xiubli@redhat.com To: ceph-devel@vger.kernel.org, jlayton@kernel.org, idryomov@gmail.com Cc: lhenriques@suse.de, mchangir@redhat.com, viro@zeniv.linux.org.uk, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, Xiubo Li , stable@vger.kernel.org Subject: [PATCH 1/2 v3] ceph: switch to vfs_inode_has_locks() to fix file lock bug Date: Fri, 18 Nov 2022 10:06:41 +0800 Message-Id: <20221118020642.472484-2-xiubli@redhat.com> In-Reply-To: <20221118020642.472484-1-xiubli@redhat.com> References: <20221118020642.472484-1-xiubli@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1749798305142697215?= X-GMAIL-MSGID: =?utf-8?q?1749798305142697215?= From: Xiubo Li For the POSIX locks they are using the same owner, which is the thread id. And multiple POSIX locks could be merged into single one, so when checking whether the 'file' has locks may fail. For a file where some openers use locking and others don't is a really odd usage pattern though. Locks are like stoplights -- they only work if everyone pays attention to them. Just switch ceph_get_caps() to check whether any locks are set on the inode. If there are POSIX/OFD/FLOCK locks on the file at the time, we should set CHECK_FILELOCK, regardless of what fd was used to set the lock. Cc: stable@vger.kernel.org Cc: Jeff Layton Fixes: ff5d913dfc71 ("ceph: return -EIO if read/write against filp that lost file locks") URL: https://tracker.ceph.com/issues/57986 Signed-off-by: Xiubo Li --- fs/ceph/caps.c | 2 +- fs/ceph/locks.c | 4 ---- fs/ceph/super.h | 1 - 3 files changed, 1 insertion(+), 6 deletions(-) diff --git a/fs/ceph/caps.c b/fs/ceph/caps.c index 065e9311b607..948136f81fc8 100644 --- a/fs/ceph/caps.c +++ b/fs/ceph/caps.c @@ -2964,7 +2964,7 @@ int ceph_get_caps(struct file *filp, int need, int want, loff_t endoff, int *got while (true) { flags &= CEPH_FILE_MODE_MASK; - if (atomic_read(&fi->num_locks)) + if (vfs_inode_has_locks(inode)) flags |= CHECK_FILELOCK; _got = 0; ret = try_get_cap_refs(inode, need, want, endoff, diff --git a/fs/ceph/locks.c b/fs/ceph/locks.c index 3e2843e86e27..b191426bf880 100644 --- a/fs/ceph/locks.c +++ b/fs/ceph/locks.c @@ -32,18 +32,14 @@ void __init ceph_flock_init(void) static void ceph_fl_copy_lock(struct file_lock *dst, struct file_lock *src) { - struct ceph_file_info *fi = dst->fl_file->private_data; struct inode *inode = file_inode(dst->fl_file); atomic_inc(&ceph_inode(inode)->i_filelock_ref); - atomic_inc(&fi->num_locks); } static void ceph_fl_release_lock(struct file_lock *fl) { - struct ceph_file_info *fi = fl->fl_file->private_data; struct inode *inode = file_inode(fl->fl_file); struct ceph_inode_info *ci = ceph_inode(inode); - atomic_dec(&fi->num_locks); if (atomic_dec_and_test(&ci->i_filelock_ref)) { /* clear error when all locks are released */ spin_lock(&ci->i_ceph_lock); diff --git a/fs/ceph/super.h b/fs/ceph/super.h index 7b75a84ba48d..87dc55c866e9 100644 --- a/fs/ceph/super.h +++ b/fs/ceph/super.h @@ -803,7 +803,6 @@ struct ceph_file_info { struct list_head rw_contexts; u32 filp_gen; - atomic_t num_locks; }; struct ceph_dir_file_info { From patchwork Fri Nov 18 02:06:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xiubo Li X-Patchwork-Id: 22061 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp734987wrr; Thu, 17 Nov 2022 18:15:05 -0800 (PST) X-Google-Smtp-Source: AA0mqf7pEHUJU6WHaMOsdxQ8h52o6U+4n7UnDVTXUpTutOo2fJfum8ZFDY6Lc7je0A3BkFHsmSAs X-Received: by 2002:a17:906:6417:b0:7ae:937f:2c38 with SMTP id d23-20020a170906641700b007ae937f2c38mr4388499ejm.201.1668737705628; Thu, 17 Nov 2022 18:15:05 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1668737705; cv=none; d=google.com; s=arc-20160816; b=q7j/MG9aAMDqyDZH9rx8KTah0lEwuykPb+lhWCaY4LhiJIgSHssOtTW5Qm/gKVsat1 Sdo0ZmnLFqiYO8OFHQW6OBjCt1MHciZhtYRuCXUDmMXd1g5WBHhCEBvupzW2SfAzYQpR iXD6V2f650n6gmYvaDcWwLVFQGSz3UwYBMY7TM5eDMyYLj8lxYRUpOK//fszFNF08f1F 4jv0OrTAAuduxVQzvSqedaRZyJjK4NwXhSZiP4h7lh0Im6r4ZYe4Z2GFf7ByScNB9zX0 JUoSYOux4wmZvB/tdMsF+QuYStBPiPrd3quxJX/a71DZ/wvu6avOYD34mdkLAsRpC/N6 zkJQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Nmr3tXYC2BQhdEeIC85P8Dx88dykoP9+tJN2ppJbsMQ=; b=UoSEVn+7td9xUNSS6gNGK1FT2CHJfbs1jBDVzTRnzXLRcJ9b/M+5tgvbZ7eJDE91u9 c2oYmine0FOpEuzrQIlnaU6HUPxfhhB1Uh9iPq1W6jNxqMGoXDuC+uFTAtEGACB8iaTS QHsMlODSmnadwQ6Dib+di6qE41TY3A3eSuF0gzSK8/Q/XSrszUS/EUgjZqiWyiFiqS3e KyjSYrnMisbsMNKkIKGhaxTzjn07ehAzOwNat+FF8CbBHorBBtAxL9giQfTdO/mrU4cd BQp5WOPd9DmxRKxoK9bh6x1OLioAwHG0kvJo/6rB1oD+EzxCfeW4yMqwlt+c5FezPg5f ePaQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=EOrlzgEG; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id ht21-20020a170907609500b007b299050723si2077374ejc.410.2022.11.17.18.14.41; Thu, 17 Nov 2022 18:15:05 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=EOrlzgEG; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240822AbiKRCIv (ORCPT + 99 others); Thu, 17 Nov 2022 21:08:51 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38474 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240570AbiKRCIs (ORCPT ); Thu, 17 Nov 2022 21:08:48 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4F5B7898D7 for ; Thu, 17 Nov 2022 18:07:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1668737221; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Nmr3tXYC2BQhdEeIC85P8Dx88dykoP9+tJN2ppJbsMQ=; b=EOrlzgEGggllOnCtO5QfHnUdVgXLWz4oEUJ0gdMjHgzd0oPkFmcLdW1M4SpPimvlEMoGsj zoea6XZWylM5JJlgH/YRj092GyfdUrwmrUBgICaglchRsD0eguNMXmzz6lOwaX5td7zhZ8 z0iGxO1E7Y6CIQZiHhS60RjQnxqTrjI= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-509-NN580cw-NYiGpmNXvyLozA-1; Thu, 17 Nov 2022 21:06:58 -0500 X-MC-Unique: NN580cw-NYiGpmNXvyLozA-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id DBB8B101A52A; Fri, 18 Nov 2022 02:06:57 +0000 (UTC) Received: from lxbceph1.gsslab.pek2.redhat.com (unknown [10.72.47.117]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0E23FC158CF; Fri, 18 Nov 2022 02:06:53 +0000 (UTC) From: xiubli@redhat.com To: ceph-devel@vger.kernel.org, jlayton@kernel.org, idryomov@gmail.com Cc: lhenriques@suse.de, mchangir@redhat.com, viro@zeniv.linux.org.uk, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, Xiubo Li , stable@vger.kernel.org Subject: [PATCH 2/2 v3] ceph: add ceph_lock_info support for file_lock Date: Fri, 18 Nov 2022 10:06:42 +0800 Message-Id: <20221118020642.472484-3-xiubli@redhat.com> In-Reply-To: <20221118020642.472484-1-xiubli@redhat.com> References: <20221118020642.472484-1-xiubli@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1749798308411878299?= X-GMAIL-MSGID: =?utf-8?q?1749798308411878299?= From: Xiubo Li When ceph releasing the file_lock it will try to get the inode pointer from the fl->fl_file, which the memory could already be released by another thread in filp_close(). Because in VFS layer the fl->fl_file doesn't increase the file's reference counter. Will switch to use ceph dedicate lock info to track the inode. And in ceph_fl_release_lock() we should skip all the operations if the fl->fl_u.ceph_fl.fl_inode is not set, which should come from the request file_lock. And we will set fl->fl_u.ceph_fl.fl_inode when inserting it to the inode lock list, which is when copying the lock. Cc: stable@vger.kernel.org Cc: Jeff Layton URL: https://tracker.ceph.com/issues/57986 Signed-off-by: Xiubo Li --- fs/ceph/locks.c | 20 ++++++++++++++++++-- include/linux/ceph/ceph_fs_fl.h | 17 +++++++++++++++++ include/linux/fs.h | 2 ++ 3 files changed, 37 insertions(+), 2 deletions(-) create mode 100644 include/linux/ceph/ceph_fs_fl.h diff --git a/fs/ceph/locks.c b/fs/ceph/locks.c index b191426bf880..621f38f10a88 100644 --- a/fs/ceph/locks.c +++ b/fs/ceph/locks.c @@ -34,18 +34,34 @@ static void ceph_fl_copy_lock(struct file_lock *dst, struct file_lock *src) { struct inode *inode = file_inode(dst->fl_file); atomic_inc(&ceph_inode(inode)->i_filelock_ref); + dst->fl_u.ceph_fl.fl_inode = igrab(inode); } +/* + * Do not use the 'fl->fl_file' in release function, which + * is possibly already released by another thread. + */ static void ceph_fl_release_lock(struct file_lock *fl) { - struct inode *inode = file_inode(fl->fl_file); - struct ceph_inode_info *ci = ceph_inode(inode); + struct inode *inode = fl->fl_u.ceph_fl.fl_inode; + struct ceph_inode_info *ci; + + /* + * If inode is NULL it should be a request file_lock, + * nothing we can do. + */ + if (!inode) + return; + + ci = ceph_inode(inode); if (atomic_dec_and_test(&ci->i_filelock_ref)) { /* clear error when all locks are released */ spin_lock(&ci->i_ceph_lock); ci->i_ceph_flags &= ~CEPH_I_ERROR_FILELOCK; spin_unlock(&ci->i_ceph_lock); } + fl->fl_u.ceph_fl.fl_inode = NULL; + iput(inode); } static const struct file_lock_operations ceph_fl_lock_ops = { diff --git a/include/linux/ceph/ceph_fs_fl.h b/include/linux/ceph/ceph_fs_fl.h new file mode 100644 index 000000000000..ad1cf96329f9 --- /dev/null +++ b/include/linux/ceph/ceph_fs_fl.h @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * ceph_fs_fl.h - Ceph lock info + * + * LGPL2 + */ + +#ifndef CEPH_FS_FL_H +#define CEPH_FS_FL_H + +#include + +struct ceph_lock_info { + struct inode *fl_inode; +}; + +#endif diff --git a/include/linux/fs.h b/include/linux/fs.h index d6cb42b7e91c..2b03d5e375d7 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -1066,6 +1066,7 @@ bool opens_in_grace(struct net *); /* that will die - we need it for nfs_lock_info */ #include +#include /* * struct file_lock represents a generic "file lock". It's used to represent @@ -1119,6 +1120,7 @@ struct file_lock { int state; /* state of grant or error if -ve */ unsigned int debug_id; } afs; + struct ceph_lock_info ceph_fl; } fl_u; } __randomize_layout;