From patchwork Wed Dec 14 03:35:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xiubo Li X-Patchwork-Id: 33039 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:e747:0:0:0:0:0 with SMTP id c7csp518548wrn; Tue, 13 Dec 2022 19:45:08 -0800 (PST) X-Google-Smtp-Source: AA0mqf4FtI5sifrFtQmfQDI4QW/4CEVR+7Pb/4060iucLJ2lcvoXjuBW45vja+seSuM9mwl7qxMA X-Received: by 2002:a05:6402:3609:b0:461:8be6:1ac5 with SMTP id el9-20020a056402360900b004618be61ac5mr18346484edb.3.1670989508442; Tue, 13 Dec 2022 19:45:08 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670989508; cv=none; d=google.com; s=arc-20160816; b=OH1h0ABPO5cjK5IOsD9d1810ur2P4nCHYar3t42iB+TmMw89bYg1/rwEMmRxTnG7gG oIjWy0MRyUi6BY6IDAG35G6Zon0FQ5InqLT/OB2s9sXl73Vjs9FTsMM99kwa6XrQywT2 aUPIHG81rmPvsszE3gBUj8zSiTOu7YcQB6AUfm00DG0OPwc3776FtqK6fiL4EJ1tlh3N J08RJCQgEnfRJm4vi7+enDgcgkxi6AfkamtbKMwM/JsrL+sSAxVIzYeSK8XMcvmKsBdP IsY2yHnUZuJO3nY+2EDhlFS7qr41RXvhPerYVxG3Fru/lOKamh9cXRovkNPGkJG9tFNH In5Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=+M3hztkvVAWIthOGbgW5wWDevbGWjmaD7x7zJ7tY33A=; b=YWeIS5uflmLD5Gy3cAhQUbDY8zjHQKcqK5BOmHW8JiPWnCeEMuM9Jx5Jwb0+K8tPTH zpc+ibyB8xkI2CdrGb3Tapww3eXHqz3F1HuOxBUMNun//cq/mOMg8S3rdEahChm49nTc AB7tsvL4ke/EF115nGr+8DP/u9NkEirgQYBvnrOIeQxLijX34TpEwjZgLH8h86znMh+v nRormQ9zVu9kO0uWPER1i+GQM+7Xahl0bDyUd74f9FI4jpGAOQoiZJeoNbKnK3bmvjOt 8LznYY9HziFJaomsheIY1xi+XFRZWwXJd0ummOVfQqugr1qMmaW/bGloxP326m0cIo8E cdSA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=hR8YwEXQ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id n21-20020aa7c455000000b00459b51c2b25si10625602edr.438.2022.12.13.19.44.29; Tue, 13 Dec 2022 19:45:08 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=hR8YwEXQ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237315AbiLNDgY (ORCPT + 99 others); Tue, 13 Dec 2022 22:36:24 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38978 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237301AbiLNDgV (ORCPT ); Tue, 13 Dec 2022 22:36:21 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4B6F71EC66 for ; Tue, 13 Dec 2022 19:35:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1670988935; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+M3hztkvVAWIthOGbgW5wWDevbGWjmaD7x7zJ7tY33A=; b=hR8YwEXQ8hWoNmB6JM33As1hQIUVT0yO0NPDmJJkYTiYOwyGN7h+OY7DSm+9BmrmefnIfj 9ok4EPagjF0U0a0ufoo0UYwjDCpTERhWiDLi1HV2Wgdh9a6XV9CVHTLCTRnsO1NEdpHhox PZWX8My/kM+3qFRjkp+uL/Vr0TSyVmc= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-550-XOsTcXL_O4CUOb-7CFHS6w-1; Tue, 13 Dec 2022 22:35:32 -0500 X-MC-Unique: XOsTcXL_O4CUOb-7CFHS6w-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id CD5BC381078D; Wed, 14 Dec 2022 03:35:19 +0000 (UTC) Received: from lxbceph1.gsslab.pek2.redhat.com (unknown [10.72.47.117]) by smtp.corp.redhat.com (Postfix) with ESMTP id D25252166B26; Wed, 14 Dec 2022 03:35:27 +0000 (UTC) From: xiubli@redhat.com To: jlayton@kernel.org, idryomov@gmail.com, ceph-devel@vger.kernel.org Cc: mchangir@redhat.com, lhenriques@suse.de, viro@zeniv.linux.org.uk, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, Xiubo Li , stable@vger.kernel.org Subject: [PATCH v5 1/2] ceph: switch to vfs_inode_has_locks() to fix file lock bug Date: Wed, 14 Dec 2022 11:35:11 +0800 Message-Id: <20221214033512.659913-2-xiubli@redhat.com> In-Reply-To: <20221214033512.659913-1-xiubli@redhat.com> References: <20221214033512.659913-1-xiubli@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1752159494740713186?= X-GMAIL-MSGID: =?utf-8?q?1752159494740713186?= From: Xiubo Li For the POSIX locks they are using the same owner, which is the thread id. And multiple POSIX locks could be merged into single one, so when checking whether the 'file' has locks may fail. For a file where some openers use locking and others don't is a really odd usage pattern though. Locks are like stoplights -- they only work if everyone pays attention to them. Just switch ceph_get_caps() to check whether any locks are set on the inode. If there are POSIX/OFD/FLOCK locks on the file at the time, we should set CHECK_FILELOCK, regardless of what fd was used to set the lock. Cc: stable@vger.kernel.org Cc: Jeff Layton Fixes: ff5d913dfc71 ("ceph: return -EIO if read/write against filp that lost file locks") Reviewed-by: Jeff Layton Signed-off-by: Xiubo Li --- fs/ceph/caps.c | 2 +- fs/ceph/locks.c | 4 ---- fs/ceph/super.h | 1 - 3 files changed, 1 insertion(+), 6 deletions(-) diff --git a/fs/ceph/caps.c b/fs/ceph/caps.c index 065e9311b607..948136f81fc8 100644 --- a/fs/ceph/caps.c +++ b/fs/ceph/caps.c @@ -2964,7 +2964,7 @@ int ceph_get_caps(struct file *filp, int need, int want, loff_t endoff, int *got while (true) { flags &= CEPH_FILE_MODE_MASK; - if (atomic_read(&fi->num_locks)) + if (vfs_inode_has_locks(inode)) flags |= CHECK_FILELOCK; _got = 0; ret = try_get_cap_refs(inode, need, want, endoff, diff --git a/fs/ceph/locks.c b/fs/ceph/locks.c index 3e2843e86e27..b191426bf880 100644 --- a/fs/ceph/locks.c +++ b/fs/ceph/locks.c @@ -32,18 +32,14 @@ void __init ceph_flock_init(void) static void ceph_fl_copy_lock(struct file_lock *dst, struct file_lock *src) { - struct ceph_file_info *fi = dst->fl_file->private_data; struct inode *inode = file_inode(dst->fl_file); atomic_inc(&ceph_inode(inode)->i_filelock_ref); - atomic_inc(&fi->num_locks); } static void ceph_fl_release_lock(struct file_lock *fl) { - struct ceph_file_info *fi = fl->fl_file->private_data; struct inode *inode = file_inode(fl->fl_file); struct ceph_inode_info *ci = ceph_inode(inode); - atomic_dec(&fi->num_locks); if (atomic_dec_and_test(&ci->i_filelock_ref)) { /* clear error when all locks are released */ spin_lock(&ci->i_ceph_lock); diff --git a/fs/ceph/super.h b/fs/ceph/super.h index 14454f464029..e7662ff6f149 100644 --- a/fs/ceph/super.h +++ b/fs/ceph/super.h @@ -804,7 +804,6 @@ struct ceph_file_info { struct list_head rw_contexts; u32 filp_gen; - atomic_t num_locks; }; struct ceph_dir_file_info { From patchwork Wed Dec 14 03:35:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xiubo Li X-Patchwork-Id: 33040 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:e747:0:0:0:0:0 with SMTP id c7csp519528wrn; Tue, 13 Dec 2022 19:48:15 -0800 (PST) X-Google-Smtp-Source: AA0mqf5WlDYMU1M1HajGB9BqlTTUmB9epndU1Cqx1MF6w0l/J5i5dmNAUkps1CdPCiYlU5cBIXVm X-Received: by 2002:a17:902:7e86:b0:189:eaae:a19c with SMTP id z6-20020a1709027e8600b00189eaaea19cmr21387374pla.30.1670989695204; Tue, 13 Dec 2022 19:48:15 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670989695; cv=none; d=google.com; s=arc-20160816; b=mhC8aiAHVnkDqCAWzlgU7YsZVr/ysLvUG426ZLMPhqTTiqRsxmbBDsJcE8XE1pc+Z1 Z8Mj7v4pWdC9ZAQVAkjmYF3uQwWMuoCEn41c/Scc+odeIPCSk3Fl7JlP6lpuaExthbYK LE+DEHm5CEv71hm0cBLux4QiOIj29fp2nik1SKaspWGQqRY8+xdPVaCGkb7cMNLQCCRW UfrjLo/0cHhem5J4uWvASZYUElrtMNexPmkTzAU1tMaFE5SmNkeJv0x58M8y/pvjuQIH 6NdquLd2rk0RvB1dOp501fXQIyNmKvEypdjYHMUTTARpWu98tSDMYPn/S4gIL4Dixjma Sa/Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=BDTRgN4BLYRjIi6g7D7REKB901ral4G3k6EuzHldn1Q=; b=wUZD52YocQKTnqhk6+qxD2LuiHpSDG+xTtnpU5l/M3Gvg2OQBHQC8cmubzU7VhFrbj 27L2ZZbDmGLzo+c1Pn4SHyB4hAeF6cS2ceO8raJf2TJ48QYrUdDinQhKdC1MrCKv9Pou eSUaHXSM14GlJ38LqmlsnBDnPACm75BA61f3QPn2OA3Km3FjUecSWJZxhthgwkinQWAH pe0usClj9HRNcj4yym+ZMCLE1Rcw3MMTB6MYoKptQnV/f1bMG86J7luwxUxcZpts9qOb ltjT0ExToK8JX35pjU1DDl1VcLWCgU1AkdDE+m4lSYT2FZ1n0+daJDzAh1kYEcpMu3t9 M/2g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=XtTyCfZs; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id f11-20020a170902ce8b00b00189274c157bsi1764160plg.518.2022.12.13.19.48.00; Tue, 13 Dec 2022 19:48:15 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=XtTyCfZs; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237170AbiLNDhM (ORCPT + 99 others); Tue, 13 Dec 2022 22:37:12 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39032 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237308AbiLNDhH (ORCPT ); Tue, 13 Dec 2022 22:37:07 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C61E52792D for ; Tue, 13 Dec 2022 19:35:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1670988942; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BDTRgN4BLYRjIi6g7D7REKB901ral4G3k6EuzHldn1Q=; b=XtTyCfZsAJq5VAEn052DnhiuefcWYlHVYWTR3B8+995ZBk+aJQ/M8Wrx/num+7rx+RMmuz tssy30kUakuhNx3vJPxh9gWw12LjaiY0kko16R9O4lOnvi/LjER73Ll0wCzcCkx3vnbzed pt1fLixMRcA7wcRWxdpr8xJE9y8QBNE= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-629-Et1anJN0P7irZgzriR91Ww-1; Tue, 13 Dec 2022 22:35:36 -0500 X-MC-Unique: Et1anJN0P7irZgzriR91Ww-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 42484380673B; Wed, 14 Dec 2022 03:35:24 +0000 (UTC) Received: from lxbceph1.gsslab.pek2.redhat.com (unknown [10.72.47.117]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4D7382166B26; Wed, 14 Dec 2022 03:35:32 +0000 (UTC) From: xiubli@redhat.com To: jlayton@kernel.org, idryomov@gmail.com, ceph-devel@vger.kernel.org Cc: mchangir@redhat.com, lhenriques@suse.de, viro@zeniv.linux.org.uk, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, Xiubo Li , stable@vger.kernel.org Subject: [PATCH v5 2/2] ceph: add ceph specific member support for file_lock Date: Wed, 14 Dec 2022 11:35:12 +0800 Message-Id: <20221214033512.659913-3-xiubli@redhat.com> In-Reply-To: <20221214033512.659913-1-xiubli@redhat.com> References: <20221214033512.659913-1-xiubli@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1752159690400576080?= X-GMAIL-MSGID: =?utf-8?q?1752159690400576080?= From: Xiubo Li When ceph releasing the file_lock it will try to get the inode pointer from the fl->fl_file, which the memory could already be released by another thread in filp_close(). Because in VFS layer the fl->fl_file doesn't increase the file's reference counter. Will switch to use ceph dedicate lock info to track the inode. And in ceph_fl_release_lock() we should skip all the operations if the fl->fl_u.ceph_fl.fl_inode is not set, which should come from the request file_lock. And we will set fl->fl_u.ceph_fl.fl_inode when inserting it to the inode lock list, which is when copying the lock. Cc: stable@vger.kernel.org Cc: Jeff Layton URL: https://tracker.ceph.com/issues/57986 Reviewed-by: Jeff Layton Signed-off-by: Xiubo Li --- fs/ceph/locks.c | 20 ++++++++++++++++++-- include/linux/fs.h | 3 +++ 2 files changed, 21 insertions(+), 2 deletions(-) diff --git a/fs/ceph/locks.c b/fs/ceph/locks.c index b191426bf880..d31955f710f2 100644 --- a/fs/ceph/locks.c +++ b/fs/ceph/locks.c @@ -34,18 +34,34 @@ static void ceph_fl_copy_lock(struct file_lock *dst, struct file_lock *src) { struct inode *inode = file_inode(dst->fl_file); atomic_inc(&ceph_inode(inode)->i_filelock_ref); + dst->fl_u.ceph.inode = igrab(inode); } +/* + * Do not use the 'fl->fl_file' in release function, which + * is possibly already released by another thread. + */ static void ceph_fl_release_lock(struct file_lock *fl) { - struct inode *inode = file_inode(fl->fl_file); - struct ceph_inode_info *ci = ceph_inode(inode); + struct inode *inode = fl->fl_u.ceph.inode; + struct ceph_inode_info *ci; + + /* + * If inode is NULL it should be a request file_lock, + * nothing we can do. + */ + if (!inode) + return; + + ci = ceph_inode(inode); if (atomic_dec_and_test(&ci->i_filelock_ref)) { /* clear error when all locks are released */ spin_lock(&ci->i_ceph_lock); ci->i_ceph_flags &= ~CEPH_I_ERROR_FILELOCK; spin_unlock(&ci->i_ceph_lock); } + fl->fl_u.ceph.inode = NULL; + iput(inode); } static const struct file_lock_operations ceph_fl_lock_ops = { diff --git a/include/linux/fs.h b/include/linux/fs.h index 7b52fdfb6da0..c92c62f1e964 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -1119,6 +1119,9 @@ struct file_lock { int state; /* state of grant or error if -ve */ unsigned int debug_id; } afs; + struct { + struct inode *inode; + } ceph; } fl_u; } __randomize_layout;