From patchwork Fri Dec 8 03:27:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: NeilBrown X-Patchwork-Id: 175557 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp5222860vqy; Thu, 7 Dec 2023 19:39:58 -0800 (PST) X-Google-Smtp-Source: AGHT+IE+2pogBkM/pmrdMO1bpeQliC6+AISr3rsAjSL2+fvP34rMO4/UXfJJNRVCTKXEyuEL2D3U X-Received: by 2002:a05:6a00:10cb:b0:6ce:379f:5ea0 with SMTP id d11-20020a056a0010cb00b006ce379f5ea0mr315030pfu.1.1702006798596; Thu, 07 Dec 2023 19:39:58 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702006798; cv=none; d=google.com; s=arc-20160816; b=AeXLrTsBLEUFn4TRe0Gdr6DNABNgnonxm2cKWqKQRTeHzQQ01XSswqmLoBPTPkmnkt U2IfjafTxddRP8O08arNAS/ym+1cAFEs5sZqSCkzvm06IN0/hYJmiJaXuEUdH3NufDZm bJS4F0Fa1ijyieLO0eKpyKWVBhVg6Fy9U+EuQFk28FsKYBOCBi0YDkTCG5ms0MAe8VHI heAEsN7WC6T7a2CeGq+4mYgzNCmwOnz4aVcQ/w1QYVg2wvOGXOlWVGVr4J+MAyKpOGIZ 9GY8njnXnh3mJcX+nQaN1JtmarXciztjX2XH/h19LV3XDCmpL+C+Qx6PRIFVU37Mj1ha J0jQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:dkim-signature; bh=Vu248dzuKJlGI9r2nLRgAmjct9Ps79FDgE6mGoaCwWU=; fh=I45k8M33HW2Wqj1OlgncPyvUt41clxPTDAFs+9Wisus=; b=AVVX87Oi9D988TXV61m8DoOIMT2wVZqf8l+RbdnygjRUJ/CGjCFPJ4R+RQWlP7Bi6i UBZeRCLcnWFy26COFK9AU82ZA1A4Qv/0RUrBXjUcVKodbJZRfj9F5oz3BfRg+MRUd9Ih QPp1jAgp0rwTB/mGQgzVQzdcE0q3b4efbTCHiGdEr7ZFnv9tADMSmDMamlCTHIgIUQBm rQLnoSoEQMFg+FPGVhc+DmOHn60gJ4LXlAkRW7RM0qBhaT8/SYKvwJ+yfDJBJEew3+Cr zV7AEkiUiPhKnDw9M2CFqBmJ9+WCmbvFepQcLOrLPTR6OgWZXw39580gHMRyHW/4+MuE B+3g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@suse.de header.s=susede2_rsa header.b=anmxB+fk; dkim=neutral (no key) header.i=@suse.de header.s=susede2_ed25519 header.b=RVlBKXdB; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=suse.de Received: from groat.vger.email (groat.vger.email. [2620:137:e000::3:5]) by mx.google.com with ESMTPS id q12-20020a056a00088c00b006cddc343ceesi808507pfj.181.2023.12.07.19.39.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 Dec 2023 19:39:58 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) client-ip=2620:137:e000::3:5; Authentication-Results: mx.google.com; dkim=pass header.i=@suse.de header.s=susede2_rsa header.b=anmxB+fk; dkim=neutral (no key) header.i=@suse.de header.s=susede2_ed25519 header.b=RVlBKXdB; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=suse.de Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by groat.vger.email (Postfix) with ESMTP id 9AEF38087E0E; Thu, 7 Dec 2023 19:39:52 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at groat.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231892AbjLHDjn (ORCPT + 99 others); Thu, 7 Dec 2023 22:39:43 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37500 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229531AbjLHDjn (ORCPT ); Thu, 7 Dec 2023 22:39:43 -0500 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 21E8D10E6; Thu, 7 Dec 2023 19:39:49 -0800 (PST) Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id AD09022102; Fri, 8 Dec 2023 03:39:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1702006787; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Vu248dzuKJlGI9r2nLRgAmjct9Ps79FDgE6mGoaCwWU=; b=anmxB+fkH2fqhNie3MRW+XjNo0iXCABg60BSQw5Gsayu+iRy7OkPLqu3TWa1mTc05JFt4b PB64CHduam7aN+CxqIAk22QBDh8iM9NqskeG90Tsq/T3SdNhJ7iBncCsTGKBzfB+NjIlQb YV+pMG2fSEtB7Ejcs7hIWeF6ddVnbHY= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1702006787; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Vu248dzuKJlGI9r2nLRgAmjct9Ps79FDgE6mGoaCwWU=; b=RVlBKXdBjGA2gqTyBfMjuQ3JTo5Idgr/yQkZVdm/OFHxFJVi1kDjDn+oWJNh1MzyNl0vx4 h92N1tS4/CTf9XCQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 1DB3C13725; Fri, 8 Dec 2023 03:30:33 +0000 (UTC) Received: from dovecot-director2.suse.de ([10.150.64.162]) by imap1.dmz-prg2.suse.org with ESMTPSA id h/XQKNmNcmX+MAAAD6G6ig (envelope-from ); Fri, 08 Dec 2023 03:30:33 +0000 From: NeilBrown To: Al Viro , Christian Brauner , Jens Axboe , Oleg Nesterov , Chuck Lever , Jeff Layton Cc: linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-nfs@vger.kernel.org Subject: [PATCH 1/3] nfsd: use __fput_sync() to avoid delayed closing of files. Date: Fri, 8 Dec 2023 14:27:26 +1100 Message-ID: <20231208033006.5546-2-neilb@suse.de> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20231208033006.5546-1-neilb@suse.de> References: <20231208033006.5546-1-neilb@suse.de> MIME-Version: 1.0 X-Spamd-Result: default: False [10.00 / 50.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; BAYES_SPAM(5.10)[100.00%]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_MISSING_CHARSET(2.50)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; BROKEN_CONTENT_TYPE(1.50)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; RCPT_COUNT_SEVEN(0.00)[9]; MID_CONTAINS_FROM(1.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:email]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_ALL(0.00)[] Authentication-Results: smtp-out1.suse.de; none X-Spam-Score: 10.00 X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Thu, 07 Dec 2023 19:39:53 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1784683480706825206 X-GMAIL-MSGID: 1784683480706825206 Calling fput() directly or though filp_close() from a kernel thread like nfsd causes the final __fput() (if necessary) to be called from a workqueue. This means that nfsd is not forced to wait for any work to complete. If the ->release of ->destroy_inode function is slow for any reason, this can result in nfsd closing files more quickly than the workqueue can complete the close and the queue of pending closes can grow without bounces (30 million has been seen at one customer site, though this was in part due to a slowness in xfs which has since been fixed). nfsd does not need this. This quite appropriate and safe for nfsd to do its own close work. There is now reason that close should ever wait for nfsd, so no deadlock can occur. So change all fput() calls to __fput_sync(), and convert filp_close() to the sequence get_file();filp_close();__fput_sync(). This ensure that no fput work is queued to the workqueue. Note that this removes the only in-module use of flush_fput_queue(). Signed-off-by: NeilBrown --- fs/nfsd/filecache.c | 3 ++- fs/nfsd/lockd.c | 2 +- fs/nfsd/nfs4proc.c | 4 ++-- fs/nfsd/nfs4recover.c | 2 +- fs/nfsd/vfs.c | 12 ++++++------ 5 files changed, 12 insertions(+), 11 deletions(-) diff --git a/fs/nfsd/filecache.c b/fs/nfsd/filecache.c index ef063f93fde9..e9734c7451b5 100644 --- a/fs/nfsd/filecache.c +++ b/fs/nfsd/filecache.c @@ -283,7 +283,9 @@ nfsd_file_free(struct nfsd_file *nf) nfsd_file_mark_put(nf->nf_mark); if (nf->nf_file) { nfsd_file_check_write_error(nf); + get_file(nf->nf_file); filp_close(nf->nf_file, NULL); + __fput_sync(nf->nf_file); } /* @@ -631,7 +633,6 @@ nfsd_file_close_inode_sync(struct inode *inode) list_del_init(&nf->nf_lru); nfsd_file_free(nf); } - flush_delayed_fput(); } /** diff --git a/fs/nfsd/lockd.c b/fs/nfsd/lockd.c index 46a7f9b813e5..f9d1059096a4 100644 --- a/fs/nfsd/lockd.c +++ b/fs/nfsd/lockd.c @@ -60,7 +60,7 @@ nlm_fopen(struct svc_rqst *rqstp, struct nfs_fh *f, struct file **filp, static void nlm_fclose(struct file *filp) { - fput(filp); + __fput_sync(filp); } static const struct nlmsvc_binding nfsd_nlm_ops = { diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c index 6f2d4aa4970d..20d60823d530 100644 --- a/fs/nfsd/nfs4proc.c +++ b/fs/nfsd/nfs4proc.c @@ -629,7 +629,7 @@ nfsd4_open(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, nn->somebody_reclaimed = true; out: if (open->op_filp) { - fput(open->op_filp); + __fput_sync(open->op_filp); open->op_filp = NULL; } if (resfh && resfh != &cstate->current_fh) { @@ -1546,7 +1546,7 @@ nfsd4_cleanup_inter_ssc(struct nfsd4_ssc_umount_item *nsui, struct file *filp, long timeout = msecs_to_jiffies(nfsd4_ssc_umount_timeout); nfs42_ssc_close(filp); - fput(filp); + __fput_sync(filp); spin_lock(&nn->nfsd_ssc_lock); list_del(&nsui->nsui_list); diff --git a/fs/nfsd/nfs4recover.c b/fs/nfsd/nfs4recover.c index 3509e73abe1f..f8f0112fd9f5 100644 --- a/fs/nfsd/nfs4recover.c +++ b/fs/nfsd/nfs4recover.c @@ -561,7 +561,7 @@ nfsd4_shutdown_recdir(struct net *net) if (!nn->rec_file) return; - fput(nn->rec_file); + __fput_sync(nn->rec_file); nn->rec_file = NULL; } diff --git a/fs/nfsd/vfs.c b/fs/nfsd/vfs.c index fbbea7498f02..15a811229211 100644 --- a/fs/nfsd/vfs.c +++ b/fs/nfsd/vfs.c @@ -879,7 +879,7 @@ __nfsd_open(struct svc_rqst *rqstp, struct svc_fh *fhp, umode_t type, host_err = ima_file_check(file, may_flags); if (host_err) { - fput(file); + __fput_sync(file); goto out; } @@ -1884,10 +1884,10 @@ nfsd_rename(struct svc_rqst *rqstp, struct svc_fh *ffhp, char *fname, int flen, fh_drop_write(ffhp); /* - * If the target dentry has cached open files, then we need to try to - * close them prior to doing the rename. Flushing delayed fput - * shouldn't be done with locks held however, so we delay it until this - * point and then reattempt the whole shebang. + * If the target dentry has cached open files, then we need to + * try to close them prior to doing the rename. Final fput + * shouldn't be done with locks held however, so we delay it + * until this point and then reattempt the whole shebang. */ if (close_cached) { close_cached = false; @@ -2141,7 +2141,7 @@ nfsd_readdir(struct svc_rqst *rqstp, struct svc_fh *fhp, loff_t *offsetp, if (err == nfserr_eof || err == nfserr_toosmall) err = nfs_ok; /* can still be found in ->err */ out_close: - fput(file); + __fput_sync(file); out: return err; }