Message ID | 20230316181413.26916-1-lhenriques@suse.de |
---|---|
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:604a:0:0:0:0:0 with SMTP id j10csp640538wrt; Thu, 16 Mar 2023 11:40:45 -0700 (PDT) X-Google-Smtp-Source: AK7set8/yK0t2bB6cYf9bK0E6PSYksVdrNZ0RDvMVuPnEW2n6XbKkrm8RpC96l3jtqot9Pw8UgsR X-Received: by 2002:a05:6a20:4413:b0:cd:9db3:a7c1 with SMTP id ce19-20020a056a20441300b000cd9db3a7c1mr5712463pzb.44.1678992045495; Thu, 16 Mar 2023 11:40:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1678992045; cv=none; d=google.com; s=arc-20160816; b=Xw4dN9ghTIsPsWrSZCn2A4l1Y+hHqhCwKT1mNsH2Dfky7ez3BwziK1T4UGMmjtH3td +phfC1KUaZRy7JBz0tDsj1lVR+u8vLXJ50fb86yPULG1oPt+qEEiFYOepninD/sTKoCG k18PvKUgPEDvtWVTlLQkWqKB350PnKRLiNmVc6nlWVuxZknot2GsAJKe0wDk3w+7OM8Z 1UrijWtiOgTSE5DZnHR8lIh3T9T7WfELqFlaUwnIbUXjikVpcwmyLIF1JxCwWGdw1nJ1 l+7e54GfvlDJkXm4S0rnaDmtvMxg7+zex/fIO7XUbdRf91z2xBMSj01DMDqp4zwTJrYI uAuw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature:dkim-signature; bh=wPYxUl/V0E0fQjPzR03hgTUP8YMge8ZIO2XP7gX1C3s=; b=eA7gTXFI5Y8P/V3HD+mLgovXxsABrtsyMHWDVLsK/ze8fMF8rLgdVhOrGZGM0Po7Zv evs/PLMBSSDTH/MDe7kNT2gg0MjA/4A0T4meXRzKcqClDxfqrzk//6ot6dvpgkmxyyB5 6YZGGa13+Q+OvIfPo+v3I293nj40Ks1aFWeroVSPjf5/Rxa5a4D0Bh1c6k7MeokUtWRT EWt5HoVa1G+nRQOvrSL3Tt3yq9aNaDhTdofP25tGJDcNvJkdkFB5XJAr8IQihtRZZM/4 vY3vfhgqq0RVAR6rCVi33UJnmYhMTlnKrTK4myxjntaFo+4IMJIdACEtLgwNiwRS0bGa WtCQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@suse.de header.s=susede2_rsa header.b="S6Cs9pO/"; dkim=neutral (no key) header.i=@suse.de header.b=ylKYUdoH; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=suse.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id z19-20020a63e113000000b0050308edbd50si8685708pgh.453.2023.03.16.11.40.32; Thu, 16 Mar 2023 11:40:45 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@suse.de header.s=susede2_rsa header.b="S6Cs9pO/"; dkim=neutral (no key) header.i=@suse.de header.b=ylKYUdoH; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=suse.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230489AbjCPSOX (ORCPT <rfc822;pwkd43@gmail.com> + 99 others); Thu, 16 Mar 2023 14:14:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36374 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229621AbjCPSOV (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Thu, 16 Mar 2023 14:14:21 -0400 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 52D8D3C29; Thu, 16 Mar 2023 11:14:20 -0700 (PDT) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id B984021A3B; Thu, 16 Mar 2023 18:14:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1678990458; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=wPYxUl/V0E0fQjPzR03hgTUP8YMge8ZIO2XP7gX1C3s=; b=S6Cs9pO/cWfbFKvIjeuhYXDHc+oVPLHQVLSFtRwJGAXesXJBfftVsm6nHeA/uCRhuTPQK4 Kgx1W9qnKxmufPDz1+3LpF4AAg2CS0nuO6OnFcDAEJbkpcWlMVThVPg0+b1yoLkJHk5wTr 5i5hUQigWt43VgmNVR+dYg+zTKYkGXQ= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1678990458; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=wPYxUl/V0E0fQjPzR03hgTUP8YMge8ZIO2XP7gX1C3s=; b=ylKYUdoHK6tYD29ZZc7kL6KnlL1I54rv2q+e0f/LpOy42rFxpHgCT5eR7QAjXMZ+Ax61sL ovSqcNwIHtqn/TBw== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 1D18C133E0; Thu, 16 Mar 2023 18:14:18 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id SSvEA3pcE2RtXAAAMHmgww (envelope-from <lhenriques@suse.de>); Thu, 16 Mar 2023 18:14:18 +0000 Received: from localhost (brahms.olymp [local]) by brahms.olymp (OpenSMTPD) with ESMTPA id 36427925; Thu, 16 Mar 2023 18:14:14 +0000 (UTC) From: =?utf-8?q?Lu=C3=ADs_Henriques?= <lhenriques@suse.de> To: Eric Biggers <ebiggers@kernel.org>, Xiubo Li <xiubli@redhat.com>, Jeff Layton <jlayton@kernel.org> Cc: "Theodore Y. Ts'o" <tytso@mit.edu>, Jaegeuk Kim <jaegeuk@kernel.org>, Ilya Dryomov <idryomov@gmail.com>, linux-fscrypt@vger.kernel.org, ceph-devel@vger.kernel.org, linux-kernel@vger.kernel.org, =?utf-8?q?Lu?= =?utf-8?q?=C3=ADs_Henriques?= <lhenriques@suse.de> Subject: [PATCH v3 0/3] ceph: fscrypt: fix atomic open bug for encrypted directories Date: Thu, 16 Mar 2023 18:14:10 +0000 Message-Id: <20230316181413.26916-1-lhenriques@suse.de> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1760550762725495698?= X-GMAIL-MSGID: =?utf-8?q?1760550762725495698?= |
Series |
ceph: fscrypt: fix atomic open bug for encrypted directories
|
|
Message
Luis Henriques
March 16, 2023, 6:14 p.m. UTC
Hi! I started seeing fstest generic/123 failing in ceph fscrypt, when running it with 'test_dummy_encryption'. This test is quite simple: 1. Creates a directory with write permissions for root only 2. Writes into a file in that directory 3. Uses 'su' to try to modify that file as a different user, and gets -EPERM All the test steps succeed, but the test fails to cleanup: 'rm -rf <dir>' will fail with -ENOTEMPTY. 'strace' shows that calling unlinkat() to remove the file got a -ENOENT and then -ENOTEMPTY for the directory. This is because 'su' does a drop_caches ('su (874): drop_caches: 2' in dmesg), and ceph's atomic open will do: if (IS_ENCRYPTED(dir)) { set_bit(CEPH_MDS_R_FSCRYPT_FILE, &req->r_req_flags); if (!fscrypt_has_encryption_key(dir)) { spin_lock(&dentry->d_lock); dentry->d_flags |= DCACHE_NOKEY_NAME; spin_unlock(&dentry->d_lock); } } Although 'dir' has the encryption key available, fscrypt_has_encryption_key() will return 'false' because fscrypt info isn't yet set after the cache cleanup. The first patch will add a new helper for the atomic_open that will force the fscrypt info to be loaded into an inode that has been evicted recently but for which the key is still available. The second patch switches ceph atomic_open to use the new fscrypt helper. Cheers, -- Luís Changes since v2: - Make helper more generic and to be used both in lookup and atomic open operations - Modify ceph_lookup (patch 0002) and ceph_atomic_open (patch 0003) to use the new helper Changes since v1: - Dropped IS_ENCRYPTED() from helper function because kerneldoc says already that it applies to encrypted directories and, most importantly, because it would introduce a different behaviour for CONFIG_FS_ENCRYPTION and !CONFIG_FS_ENCRYPTION. - Rephrased helper kerneldoc Changes since initial RFC (after Eric's review): - Added kerneldoc comments to the new fscrypt helper - Dropped '__' from helper name (now fscrypt_prepare_atomic_open()) - Added IS_ENCRYPTED() check in helper - DCACHE_NOKEY_NAME is not set if fscrypt_get_encryption_info() returns an error - Fixed helper for !CONFIG_FS_ENCRYPTION (now defined 'static inline') Luís Henriques (3): fscrypt: new helper function - fscrypt_prepare_lookup_partial() ceph: switch ceph_open() to use new fscrypt helper ceph: switch ceph_open_atomic() to use the new fscrypt helper fs/ceph/dir.c | 13 +++++++------ fs/ceph/file.c | 8 +++----- fs/crypto/hooks.c | 37 +++++++++++++++++++++++++++++++++++++ include/linux/fscrypt.h | 7 +++++++ 4 files changed, 54 insertions(+), 11 deletions(-)
Comments
On 17/03/2023 02:14, Luís Henriques wrote: > Hi! > > I started seeing fstest generic/123 failing in ceph fscrypt, when running it > with 'test_dummy_encryption'. This test is quite simple: > > 1. Creates a directory with write permissions for root only > 2. Writes into a file in that directory > 3. Uses 'su' to try to modify that file as a different user, and > gets -EPERM > > All the test steps succeed, but the test fails to cleanup: 'rm -rf <dir>' > will fail with -ENOTEMPTY. 'strace' shows that calling unlinkat() to remove > the file got a -ENOENT and then -ENOTEMPTY for the directory. > > This is because 'su' does a drop_caches ('su (874): drop_caches: 2' in > dmesg), and ceph's atomic open will do: > > if (IS_ENCRYPTED(dir)) { > set_bit(CEPH_MDS_R_FSCRYPT_FILE, &req->r_req_flags); > if (!fscrypt_has_encryption_key(dir)) { > spin_lock(&dentry->d_lock); > dentry->d_flags |= DCACHE_NOKEY_NAME; > spin_unlock(&dentry->d_lock); > } > } > > Although 'dir' has the encryption key available, fscrypt_has_encryption_key() > will return 'false' because fscrypt info isn't yet set after the cache > cleanup. > > The first patch will add a new helper for the atomic_open that will force > the fscrypt info to be loaded into an inode that has been evicted recently > but for which the key is still available. > > The second patch switches ceph atomic_open to use the new fscrypt helper. > > Cheers, > -- > Luís > > Changes since v2: > - Make helper more generic and to be used both in lookup and atomic open > operations > - Modify ceph_lookup (patch 0002) and ceph_atomic_open (patch 0003) to use > the new helper > > Changes since v1: > - Dropped IS_ENCRYPTED() from helper function because kerneldoc says > already that it applies to encrypted directories and, most importantly, > because it would introduce a different behaviour for > CONFIG_FS_ENCRYPTION and !CONFIG_FS_ENCRYPTION. > - Rephrased helper kerneldoc > > Changes since initial RFC (after Eric's review): > - Added kerneldoc comments to the new fscrypt helper > - Dropped '__' from helper name (now fscrypt_prepare_atomic_open()) > - Added IS_ENCRYPTED() check in helper > - DCACHE_NOKEY_NAME is not set if fscrypt_get_encryption_info() returns an > error > - Fixed helper for !CONFIG_FS_ENCRYPTION (now defined 'static inline') This series looks good to me. And I have run the test locally and worked well. > Luís Henriques (3): > fscrypt: new helper function - fscrypt_prepare_lookup_partial() Eric, If possible I we can pick this together to ceph repo and need your ack about this. Or you can pick it to the crypto repo then please feel free to add: Tested-by: Xiubo Li <xiubli@redhat.com> and Reviewed-by: Xiubo Li <xiubli@redhat.com> Thanks - Xiubo > ceph: switch ceph_open() to use new fscrypt helper > ceph: switch ceph_open_atomic() to use the new fscrypt helper > > fs/ceph/dir.c | 13 +++++++------ > fs/ceph/file.c | 8 +++----- > fs/crypto/hooks.c | 37 +++++++++++++++++++++++++++++++++++++ > include/linux/fscrypt.h | 7 +++++++ > 4 files changed, 54 insertions(+), 11 deletions(-) >
On Mon, Mar 20, 2023 at 2:07 AM Xiubo Li <xiubli@redhat.com> wrote: > > > On 17/03/2023 02:14, Luís Henriques wrote: > > Hi! > > > > I started seeing fstest generic/123 failing in ceph fscrypt, when running it > > with 'test_dummy_encryption'. This test is quite simple: > > > > 1. Creates a directory with write permissions for root only > > 2. Writes into a file in that directory > > 3. Uses 'su' to try to modify that file as a different user, and > > gets -EPERM > > > > All the test steps succeed, but the test fails to cleanup: 'rm -rf <dir>' > > will fail with -ENOTEMPTY. 'strace' shows that calling unlinkat() to remove > > the file got a -ENOENT and then -ENOTEMPTY for the directory. > > > > This is because 'su' does a drop_caches ('su (874): drop_caches: 2' in > > dmesg), and ceph's atomic open will do: > > > > if (IS_ENCRYPTED(dir)) { > > set_bit(CEPH_MDS_R_FSCRYPT_FILE, &req->r_req_flags); > > if (!fscrypt_has_encryption_key(dir)) { > > spin_lock(&dentry->d_lock); > > dentry->d_flags |= DCACHE_NOKEY_NAME; > > spin_unlock(&dentry->d_lock); > > } > > } > > > > Although 'dir' has the encryption key available, fscrypt_has_encryption_key() > > will return 'false' because fscrypt info isn't yet set after the cache > > cleanup. > > > > The first patch will add a new helper for the atomic_open that will force > > the fscrypt info to be loaded into an inode that has been evicted recently > > but for which the key is still available. > > > > The second patch switches ceph atomic_open to use the new fscrypt helper. > > > > Cheers, > > -- > > Luís > > > > Changes since v2: > > - Make helper more generic and to be used both in lookup and atomic open > > operations > > - Modify ceph_lookup (patch 0002) and ceph_atomic_open (patch 0003) to use > > the new helper > > > > Changes since v1: > > - Dropped IS_ENCRYPTED() from helper function because kerneldoc says > > already that it applies to encrypted directories and, most importantly, > > because it would introduce a different behaviour for > > CONFIG_FS_ENCRYPTION and !CONFIG_FS_ENCRYPTION. > > - Rephrased helper kerneldoc > > > > Changes since initial RFC (after Eric's review): > > - Added kerneldoc comments to the new fscrypt helper > > - Dropped '__' from helper name (now fscrypt_prepare_atomic_open()) > > - Added IS_ENCRYPTED() check in helper > > - DCACHE_NOKEY_NAME is not set if fscrypt_get_encryption_info() returns an > > error > > - Fixed helper for !CONFIG_FS_ENCRYPTION (now defined 'static inline') > > This series looks good to me. > > And I have run the test locally and worked well. > > > > Luís Henriques (3): > > fscrypt: new helper function - fscrypt_prepare_lookup_partial() > > Eric, > > If possible I we can pick this together to ceph repo and need your ack > about this. Or you can pick it to the crypto repo then please feel free > to add: > > Tested-by: Xiubo Li <xiubli@redhat.com> and Reviewed-by: Xiubo Li > <xiubli@redhat.com> I would prefer the fscrypt helper to go through the fscrypt tree. Thanks, Ilya
On 20/03/2023 19:20, Ilya Dryomov wrote: > On Mon, Mar 20, 2023 at 2:07 AM Xiubo Li <xiubli@redhat.com> wrote: >> >> On 17/03/2023 02:14, Luís Henriques wrote: >>> Hi! >>> >>> I started seeing fstest generic/123 failing in ceph fscrypt, when running it >>> with 'test_dummy_encryption'. This test is quite simple: >>> >>> 1. Creates a directory with write permissions for root only >>> 2. Writes into a file in that directory >>> 3. Uses 'su' to try to modify that file as a different user, and >>> gets -EPERM >>> >>> All the test steps succeed, but the test fails to cleanup: 'rm -rf <dir>' >>> will fail with -ENOTEMPTY. 'strace' shows that calling unlinkat() to remove >>> the file got a -ENOENT and then -ENOTEMPTY for the directory. >>> >>> This is because 'su' does a drop_caches ('su (874): drop_caches: 2' in >>> dmesg), and ceph's atomic open will do: >>> >>> if (IS_ENCRYPTED(dir)) { >>> set_bit(CEPH_MDS_R_FSCRYPT_FILE, &req->r_req_flags); >>> if (!fscrypt_has_encryption_key(dir)) { >>> spin_lock(&dentry->d_lock); >>> dentry->d_flags |= DCACHE_NOKEY_NAME; >>> spin_unlock(&dentry->d_lock); >>> } >>> } >>> >>> Although 'dir' has the encryption key available, fscrypt_has_encryption_key() >>> will return 'false' because fscrypt info isn't yet set after the cache >>> cleanup. >>> >>> The first patch will add a new helper for the atomic_open that will force >>> the fscrypt info to be loaded into an inode that has been evicted recently >>> but for which the key is still available. >>> >>> The second patch switches ceph atomic_open to use the new fscrypt helper. >>> >>> Cheers, >>> -- >>> Luís >>> >>> Changes since v2: >>> - Make helper more generic and to be used both in lookup and atomic open >>> operations >>> - Modify ceph_lookup (patch 0002) and ceph_atomic_open (patch 0003) to use >>> the new helper >>> >>> Changes since v1: >>> - Dropped IS_ENCRYPTED() from helper function because kerneldoc says >>> already that it applies to encrypted directories and, most importantly, >>> because it would introduce a different behaviour for >>> CONFIG_FS_ENCRYPTION and !CONFIG_FS_ENCRYPTION. >>> - Rephrased helper kerneldoc >>> >>> Changes since initial RFC (after Eric's review): >>> - Added kerneldoc comments to the new fscrypt helper >>> - Dropped '__' from helper name (now fscrypt_prepare_atomic_open()) >>> - Added IS_ENCRYPTED() check in helper >>> - DCACHE_NOKEY_NAME is not set if fscrypt_get_encryption_info() returns an >>> error >>> - Fixed helper for !CONFIG_FS_ENCRYPTION (now defined 'static inline') >> This series looks good to me. >> >> And I have run the test locally and worked well. >> >> >>> Luís Henriques (3): >>> fscrypt: new helper function - fscrypt_prepare_lookup_partial() >> Eric, >> >> If possible I we can pick this together to ceph repo and need your ack >> about this. Or you can pick it to the crypto repo then please feel free >> to add: >> >> Tested-by: Xiubo Li <xiubli@redhat.com> and Reviewed-by: Xiubo Li >> <xiubli@redhat.com> > I would prefer the fscrypt helper to go through the fscrypt tree. Sure. This also LGTM. Thanks - Xiubo > Thanks, > > Ilya >
Xiubo Li <xiubli@redhat.com> writes: > On 17/03/2023 02:14, Luís Henriques wrote: >> Hi! >> >> I started seeing fstest generic/123 failing in ceph fscrypt, when running it >> with 'test_dummy_encryption'. This test is quite simple: >> >> 1. Creates a directory with write permissions for root only >> 2. Writes into a file in that directory >> 3. Uses 'su' to try to modify that file as a different user, and >> gets -EPERM >> >> All the test steps succeed, but the test fails to cleanup: 'rm -rf <dir>' >> will fail with -ENOTEMPTY. 'strace' shows that calling unlinkat() to remove >> the file got a -ENOENT and then -ENOTEMPTY for the directory. >> >> This is because 'su' does a drop_caches ('su (874): drop_caches: 2' in >> dmesg), and ceph's atomic open will do: >> >> if (IS_ENCRYPTED(dir)) { >> set_bit(CEPH_MDS_R_FSCRYPT_FILE, &req->r_req_flags); >> if (!fscrypt_has_encryption_key(dir)) { >> spin_lock(&dentry->d_lock); >> dentry->d_flags |= DCACHE_NOKEY_NAME; >> spin_unlock(&dentry->d_lock); >> } >> } >> >> Although 'dir' has the encryption key available, fscrypt_has_encryption_key() >> will return 'false' because fscrypt info isn't yet set after the cache >> cleanup. >> >> The first patch will add a new helper for the atomic_open that will force >> the fscrypt info to be loaded into an inode that has been evicted recently >> but for which the key is still available. >> >> The second patch switches ceph atomic_open to use the new fscrypt helper. >> >> Cheers, >> -- >> Luís >> >> Changes since v2: >> - Make helper more generic and to be used both in lookup and atomic open >> operations >> - Modify ceph_lookup (patch 0002) and ceph_atomic_open (patch 0003) to use >> the new helper >> >> Changes since v1: >> - Dropped IS_ENCRYPTED() from helper function because kerneldoc says >> already that it applies to encrypted directories and, most importantly, >> because it would introduce a different behaviour for >> CONFIG_FS_ENCRYPTION and !CONFIG_FS_ENCRYPTION. >> - Rephrased helper kerneldoc >> >> Changes since initial RFC (after Eric's review): >> - Added kerneldoc comments to the new fscrypt helper >> - Dropped '__' from helper name (now fscrypt_prepare_atomic_open()) >> - Added IS_ENCRYPTED() check in helper >> - DCACHE_NOKEY_NAME is not set if fscrypt_get_encryption_info() returns an >> error >> - Fixed helper for !CONFIG_FS_ENCRYPTION (now defined 'static inline') > > This series looks good to me. > > And I have run the test locally and worked well. Awesome, thanks a lot Xiubo. I've been testing it locally as well and I haven't observed any breakage either. Cheers,
On Mon, Mar 20, 2023 at 08:47:18PM +0800, Xiubo Li wrote: > > On 20/03/2023 19:20, Ilya Dryomov wrote: > > On Mon, Mar 20, 2023 at 2:07 AM Xiubo Li <xiubli@redhat.com> wrote: > > > > > > On 17/03/2023 02:14, Luís Henriques wrote: > > > > Hi! > > > > > > > > I started seeing fstest generic/123 failing in ceph fscrypt, when running it > > > > with 'test_dummy_encryption'. This test is quite simple: > > > > > > > > 1. Creates a directory with write permissions for root only > > > > 2. Writes into a file in that directory > > > > 3. Uses 'su' to try to modify that file as a different user, and > > > > gets -EPERM > > > > > > > > All the test steps succeed, but the test fails to cleanup: 'rm -rf <dir>' > > > > will fail with -ENOTEMPTY. 'strace' shows that calling unlinkat() to remove > > > > the file got a -ENOENT and then -ENOTEMPTY for the directory. > > > > > > > > This is because 'su' does a drop_caches ('su (874): drop_caches: 2' in > > > > dmesg), and ceph's atomic open will do: > > > > > > > > if (IS_ENCRYPTED(dir)) { > > > > set_bit(CEPH_MDS_R_FSCRYPT_FILE, &req->r_req_flags); > > > > if (!fscrypt_has_encryption_key(dir)) { > > > > spin_lock(&dentry->d_lock); > > > > dentry->d_flags |= DCACHE_NOKEY_NAME; > > > > spin_unlock(&dentry->d_lock); > > > > } > > > > } > > > > > > > > Although 'dir' has the encryption key available, fscrypt_has_encryption_key() > > > > will return 'false' because fscrypt info isn't yet set after the cache > > > > cleanup. > > > > > > > > The first patch will add a new helper for the atomic_open that will force > > > > the fscrypt info to be loaded into an inode that has been evicted recently > > > > but for which the key is still available. > > > > > > > > The second patch switches ceph atomic_open to use the new fscrypt helper. > > > > > > > > Cheers, > > > > -- > > > > Luís > > > > > > > > Changes since v2: > > > > - Make helper more generic and to be used both in lookup and atomic open > > > > operations > > > > - Modify ceph_lookup (patch 0002) and ceph_atomic_open (patch 0003) to use > > > > the new helper > > > > > > > > Changes since v1: > > > > - Dropped IS_ENCRYPTED() from helper function because kerneldoc says > > > > already that it applies to encrypted directories and, most importantly, > > > > because it would introduce a different behaviour for > > > > CONFIG_FS_ENCRYPTION and !CONFIG_FS_ENCRYPTION. > > > > - Rephrased helper kerneldoc > > > > > > > > Changes since initial RFC (after Eric's review): > > > > - Added kerneldoc comments to the new fscrypt helper > > > > - Dropped '__' from helper name (now fscrypt_prepare_atomic_open()) > > > > - Added IS_ENCRYPTED() check in helper > > > > - DCACHE_NOKEY_NAME is not set if fscrypt_get_encryption_info() returns an > > > > error > > > > - Fixed helper for !CONFIG_FS_ENCRYPTION (now defined 'static inline') > > > This series looks good to me. > > > > > > And I have run the test locally and worked well. > > > > > > > > > > Luís Henriques (3): > > > > fscrypt: new helper function - fscrypt_prepare_lookup_partial() > > > Eric, > > > > > > If possible I we can pick this together to ceph repo and need your ack > > > about this. Or you can pick it to the crypto repo then please feel free > > > to add: > > > > > > Tested-by: Xiubo Li <xiubli@redhat.com> and Reviewed-by: Xiubo Li > > > <xiubli@redhat.com> > > I would prefer the fscrypt helper to go through the fscrypt tree. > > Sure. This also LGTM. > > Thanks > I've applied it to https://git.kernel.org/pub/scm/fs/fscrypt/linux.git/log/?h=for-next But I ended up reworking the comment a bit and moving the function to be just below __fscrypt_prepare_lookup(). So I sent out v4 that matches what I applied. BTW, I'm wondering if anyone has had any thoughts about the race condition I described at https://lore.kernel.org/r/ZBC1P4Gn6eAKD61+@sol.localdomain/. In particular, I'm wondering whether this helper function will need to be changed or not. Maybe not, because ceph could look at DCACHE_NOKEY_NAME to determine whether the name should be treated as a no-key name or not, instead of checking fscrypt_has_encryption_key() again (as I think it is doing currently)? - Eric
Eric Biggers <ebiggers@kernel.org> writes: > On Mon, Mar 20, 2023 at 08:47:18PM +0800, Xiubo Li wrote: >> >> On 20/03/2023 19:20, Ilya Dryomov wrote: >> > On Mon, Mar 20, 2023 at 2:07 AM Xiubo Li <xiubli@redhat.com> wrote: >> > > >> > > On 17/03/2023 02:14, Luís Henriques wrote: >> > > > Hi! >> > > > >> > > > I started seeing fstest generic/123 failing in ceph fscrypt, when running it >> > > > with 'test_dummy_encryption'. This test is quite simple: >> > > > >> > > > 1. Creates a directory with write permissions for root only >> > > > 2. Writes into a file in that directory >> > > > 3. Uses 'su' to try to modify that file as a different user, and >> > > > gets -EPERM >> > > > >> > > > All the test steps succeed, but the test fails to cleanup: 'rm -rf <dir>' >> > > > will fail with -ENOTEMPTY. 'strace' shows that calling unlinkat() to remove >> > > > the file got a -ENOENT and then -ENOTEMPTY for the directory. >> > > > >> > > > This is because 'su' does a drop_caches ('su (874): drop_caches: 2' in >> > > > dmesg), and ceph's atomic open will do: >> > > > >> > > > if (IS_ENCRYPTED(dir)) { >> > > > set_bit(CEPH_MDS_R_FSCRYPT_FILE, &req->r_req_flags); >> > > > if (!fscrypt_has_encryption_key(dir)) { >> > > > spin_lock(&dentry->d_lock); >> > > > dentry->d_flags |= DCACHE_NOKEY_NAME; >> > > > spin_unlock(&dentry->d_lock); >> > > > } >> > > > } >> > > > >> > > > Although 'dir' has the encryption key available, fscrypt_has_encryption_key() >> > > > will return 'false' because fscrypt info isn't yet set after the cache >> > > > cleanup. >> > > > >> > > > The first patch will add a new helper for the atomic_open that will force >> > > > the fscrypt info to be loaded into an inode that has been evicted recently >> > > > but for which the key is still available. >> > > > >> > > > The second patch switches ceph atomic_open to use the new fscrypt helper. >> > > > >> > > > Cheers, >> > > > -- >> > > > Luís >> > > > >> > > > Changes since v2: >> > > > - Make helper more generic and to be used both in lookup and atomic open >> > > > operations >> > > > - Modify ceph_lookup (patch 0002) and ceph_atomic_open (patch 0003) to use >> > > > the new helper >> > > > >> > > > Changes since v1: >> > > > - Dropped IS_ENCRYPTED() from helper function because kerneldoc says >> > > > already that it applies to encrypted directories and, most importantly, >> > > > because it would introduce a different behaviour for >> > > > CONFIG_FS_ENCRYPTION and !CONFIG_FS_ENCRYPTION. >> > > > - Rephrased helper kerneldoc >> > > > >> > > > Changes since initial RFC (after Eric's review): >> > > > - Added kerneldoc comments to the new fscrypt helper >> > > > - Dropped '__' from helper name (now fscrypt_prepare_atomic_open()) >> > > > - Added IS_ENCRYPTED() check in helper >> > > > - DCACHE_NOKEY_NAME is not set if fscrypt_get_encryption_info() returns an >> > > > error >> > > > - Fixed helper for !CONFIG_FS_ENCRYPTION (now defined 'static inline') >> > > This series looks good to me. >> > > >> > > And I have run the test locally and worked well. >> > > >> > > >> > > > Luís Henriques (3): >> > > > fscrypt: new helper function - fscrypt_prepare_lookup_partial() >> > > Eric, >> > > >> > > If possible I we can pick this together to ceph repo and need your ack >> > > about this. Or you can pick it to the crypto repo then please feel free >> > > to add: >> > > >> > > Tested-by: Xiubo Li <xiubli@redhat.com> and Reviewed-by: Xiubo Li >> > > <xiubli@redhat.com> >> > I would prefer the fscrypt helper to go through the fscrypt tree. >> >> Sure. This also LGTM. >> >> Thanks >> > > I've applied it to > https://git.kernel.org/pub/scm/fs/fscrypt/linux.git/log/?h=for-next > > But I ended up reworking the comment a bit and moving the function to be just > below __fscrypt_prepare_lookup(). So I sent out v4 that matches what I applied. Awesome, thanks a lot, Eric. > BTW, I'm wondering if anyone has had any thoughts about the race condition I > described at https://lore.kernel.org/r/ZBC1P4Gn6eAKD61+@sol.localdomain/. In > particular, I'm wondering whether this helper function will need to be changed > or not. Maybe not, because ceph could look at DCACHE_NOKEY_NAME to determine > whether the name should be treated as a no-key name or not, instead of checking > fscrypt_has_encryption_key() again (as I think it is doing currently)? I started looking into that but, to be honest, I haven't yet reached any conclusion. It looks like the ceph code that handles filenames *may* have this race too (I'm looking at ceph_fill_trace()) but I'm still not 100% sure. In any case, I think that an eventual fix for this race (if it does indeed exist!) will likely be restricted to the ceph code and won't touch the generic fscrypt code. But I'm still looking... Cheers,