From patchwork Wed Dec 6 06:05:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Chinner X-Patchwork-Id: 174309 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp3907568vqy; Tue, 5 Dec 2023 22:06:57 -0800 (PST) X-Google-Smtp-Source: AGHT+IEMpTxSJDJR2FDAe71VQCHQkV0fy20bR1qUN1tDolHswns7mu3qqEm+C7F0dsXapT7Y7vgW X-Received: by 2002:a05:6808:191b:b0:3b8:b063:5d6d with SMTP id bf27-20020a056808191b00b003b8b0635d6dmr664664oib.84.1701842817475; Tue, 05 Dec 2023 22:06:57 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1701842817; cv=none; d=google.com; s=arc-20160816; b=HktNEZoVz/Jge72rQN1qeKEnLv9P2upwm40zg6KOt0e//E7p93rpAS4/x8atfAHaZJ xcEua8CsKTg5I75N7NU0gkIF1y1Suocodje3NPypBRCU89vyi2D8V1G+ASUJxIEwKewq lGSNxFOL4hwVxyaMZ9rgII5+Kg/aKp8A7KewBXF2bhXBR3OJ+wSojEKhc/fIK8Jg1DZv cE5Ux3byyFwQVfFmk4geYOwKQvJLegvODnxwCfLZbuCd+I32wPKSj8a+W12uHj6l+srV c8oPB3sAXE/h9PzUL/UMyGWzZCHlx0bhGIinb5jZlkfumEzo8mcFyR1T915TjirgDlpW ewAQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=I06elknjTKgnu51XMDSSIqaWSXzJw01Bc0WUn6qADWU=; fh=CW5pnDTDkJR8Vn9NaUbLrrp28PUVapFEjMCn+HkS0lE=; b=DRtWJWccDP2+08b7u1sl/bpyGz89izVzy8zTAteilvhILQSzwF2AIZ/FCNEibYdtWy y2OK4haTiCfHLoImKPN3bSE0TzdsPpnQ39qqd3iK1Myk1j+mPwLMMABg7IaFdHF1jDml EAqIYJfMKZDRtnI8Zp0xCJCvfAguWZvZMnxSFotouUAsxQUxpFd6lDPhE4HQNGyBNoDO BP91hXoeMhLQ4IwtxMRuZ09NOWsNN61c8NeJTLhcTLXBF9TcIYfjVfkZlRqS8B8lq9oR erGwvICJWLgT58CJTRqd8eJYp2RzJ6nijZBLYUHQID7Waou7f6Fxj6fCI7qlhc/RnBS0 a8Ew== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@fromorbit-com.20230601.gappssmtp.com header.s=20230601 header.b=z1+QzgS4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=fromorbit.com Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id e3-20020a63d943000000b005aebc9096d4si10732683pgj.150.2023.12.05.22.06.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Dec 2023 22:06:57 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; dkim=pass header.i=@fromorbit-com.20230601.gappssmtp.com header.s=20230601 header.b=z1+QzgS4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=fromorbit.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 1D4B9802F309; Tue, 5 Dec 2023 22:06:56 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1376924AbjLFGGl (ORCPT + 99 others); Wed, 6 Dec 2023 01:06:41 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49868 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1376893AbjLFGGc (ORCPT ); Wed, 6 Dec 2023 01:06:32 -0500 Received: from mail-oi1-x234.google.com (mail-oi1-x234.google.com [IPv6:2607:f8b0:4864:20::234]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A8697D4F for ; Tue, 5 Dec 2023 22:06:36 -0800 (PST) Received: by mail-oi1-x234.google.com with SMTP id 5614622812f47-3b8b463c7c6so2594462b6e.3 for ; Tue, 05 Dec 2023 22:06:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fromorbit-com.20230601.gappssmtp.com; s=20230601; t=1701842796; x=1702447596; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=I06elknjTKgnu51XMDSSIqaWSXzJw01Bc0WUn6qADWU=; b=z1+QzgS4/Sbur45/4z83hrhThmyzPewSii5IGNWBp6GTVMHXan1ZgtwY9whhev+mue K2ZHkgYYfm6+aYG0XuukPWn5UueTuQw+vOUYlV80QZ+lUDS2TfQykJDh7HtNJsxIKz7n SpQtBqTgUjl8bpz5bxP9mVB+b9Hs80aof8fSZcuFhp2tLKDxIFIW3xsRkXS9AbzM0iK3 iRZWOATKwtohJNwKyzPUz7t0hfb4AgHI9uvqRSrtknZnOo43KE/OM4nvkcLkJlDTwx7/ hgSRB/D5Cu4dVXEoD7xo9XGHzuEhM7KIKsX0Wc1kHbOpmaAGbWLjo9U/Vw9B5ehH1kU1 M6pQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701842796; x=1702447596; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=I06elknjTKgnu51XMDSSIqaWSXzJw01Bc0WUn6qADWU=; b=MnpIFFvVvtTvp8TAu8oU9sFppIoBUyRCUWOb08LBE3IKgUn8ZpDYR8eREbiOBm12tT VXF194IML2SgdmkaKXJRByNVDSROPRR5mWDW++RkRa3LuTH+AuzSF/Jt3zSYbhKE8l2E U6vYLYMljd/AKSs6FnjkIqUCnoUATqsAL0znGDjxNtnRbSTdhG7NzackBtdzmBOTaPnn thSafvymOQT4QXnce093RfG/cb0lQ4MvD49gUPkjiLErioKPX7WT/mJbhJYkhXSaRmJl ERtl1veaGwPFFu8QfV08cuZp9uKQ/7Bq5LBNu/h7WHi7cLKVZwa/j1baTS74dK0Ijals eYOw== X-Gm-Message-State: AOJu0YwXTyBkIEulBN1oV4aVTK82OfxTyl2yV99c8GEF5s1gtG8zeinB WsRpZlOBSL/jH62DBW45hNpZ0Q== X-Received: by 2002:a05:6808:6507:b0:3b8:b063:5d82 with SMTP id fm7-20020a056808650700b003b8b0635d82mr684790oib.105.1701842795753; Tue, 05 Dec 2023 22:06:35 -0800 (PST) Received: from dread.disaster.area (pa49-180-125-5.pa.nsw.optusnet.com.au. [49.180.125.5]) by smtp.gmail.com with ESMTPSA id m18-20020a056a00081200b006ce64ebd2a0sm2297147pfk.99.2023.12.05.22.06.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Dec 2023 22:06:34 -0800 (PST) Received: from [192.168.253.23] (helo=devoid.disaster.area) by dread.disaster.area with esmtp (Exim 4.96) (envelope-from ) id 1rAl3I-004VOr-0C; Wed, 06 Dec 2023 17:06:31 +1100 Received: from dave by devoid.disaster.area with local (Exim 4.97-RC0) (envelope-from ) id 1rAl3H-0000000BrVL-2XOR; Wed, 06 Dec 2023 17:06:31 +1100 From: Dave Chinner To: linux-fsdevel@vger.kernel.org Cc: linux-block@vger.kernel.org, linux-cachefs@redhat.com, dhowells@redhat.com, gfs2@lists.linux.dev, dm-devel@lists.linux.dev, linux-security-module@vger.kernel.org, selinux@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 05/11] selinux: use dlist for isec inode list Date: Wed, 6 Dec 2023 17:05:34 +1100 Message-ID: <20231206060629.2827226-6-david@fromorbit.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231206060629.2827226-1-david@fromorbit.com> References: <20231206060629.2827226-1-david@fromorbit.com> MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Tue, 05 Dec 2023 22:06:56 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1784511533889637423 X-GMAIL-MSGID: 1784511533889637423 From: Dave Chinner Because it's a horrible point of lock contention under heavily concurrent directory traversals... - 12.14% d_instantiate - 12.06% security_d_instantiate - 12.13% selinux_d_instantiate - 12.16% inode_doinit_with_dentry - 15.45% _raw_spin_lock - do_raw_spin_lock 14.68% __pv_queued_spin_lock_slowpath Signed-off-by: Dave Chinner Acked-by: Paul Moore --- include/linux/dlock-list.h | 9 ++++ security/selinux/hooks.c | 72 +++++++++++++++---------------- security/selinux/include/objsec.h | 6 +-- 3 files changed, 47 insertions(+), 40 deletions(-) diff --git a/include/linux/dlock-list.h b/include/linux/dlock-list.h index 327cb9edc7e3..7ad933b8875d 100644 --- a/include/linux/dlock-list.h +++ b/include/linux/dlock-list.h @@ -132,6 +132,15 @@ extern void dlock_lists_add(struct dlock_list_node *node, struct dlock_list_heads *dlist); extern void dlock_lists_del(struct dlock_list_node *node); +static inline void +dlock_list_del_iter(struct dlock_list_iter *iter, + struct dlock_list_node *node) +{ + WARN_ON_ONCE((iter->entry != READ_ONCE(node->head))); + list_del_init(&node->list); + WRITE_ONCE(node->head, NULL); +} + /* * Find the first entry of the next available list. */ diff --git a/security/selinux/hooks.c b/security/selinux/hooks.c index feda711c6b7b..0358d7c66949 100644 --- a/security/selinux/hooks.c +++ b/security/selinux/hooks.c @@ -340,26 +340,11 @@ static struct inode_security_struct *backing_inode_security(struct dentry *dentr static void inode_free_security(struct inode *inode) { struct inode_security_struct *isec = selinux_inode(inode); - struct superblock_security_struct *sbsec; if (!isec) return; - sbsec = selinux_superblock(inode->i_sb); - /* - * As not all inode security structures are in a list, we check for - * empty list outside of the lock to make sure that we won't waste - * time taking a lock doing nothing. - * - * The list_del_init() function can be safely called more than once. - * It should not be possible for this function to be called with - * concurrent list_add(), but for better safety against future changes - * in the code, we use list_empty_careful() here. - */ - if (!list_empty_careful(&isec->list)) { - spin_lock(&sbsec->isec_lock); - list_del_init(&isec->list); - spin_unlock(&sbsec->isec_lock); - } + if (!list_empty(&isec->list.list)) + dlock_lists_del(&isec->list); } struct selinux_mnt_opts { @@ -547,6 +532,8 @@ static int sb_finish_set_opts(struct super_block *sb) struct superblock_security_struct *sbsec = selinux_superblock(sb); struct dentry *root = sb->s_root; struct inode *root_inode = d_backing_inode(root); + struct dlock_list_iter iter; + struct inode_security_struct *isec, *n; int rc = 0; if (sbsec->behavior == SECURITY_FS_USE_XATTR) { @@ -570,27 +557,28 @@ static int sb_finish_set_opts(struct super_block *sb) /* Initialize the root inode. */ rc = inode_doinit_with_dentry(root_inode, root); - /* Initialize any other inodes associated with the superblock, e.g. - inodes created prior to initial policy load or inodes created - during get_sb by a pseudo filesystem that directly - populates itself. */ - spin_lock(&sbsec->isec_lock); - while (!list_empty(&sbsec->isec_head)) { - struct inode_security_struct *isec = - list_first_entry(&sbsec->isec_head, - struct inode_security_struct, list); + /* + * Initialize any other inodes associated with the superblock, e.g. + * inodes created prior to initial policy load or inodes created during + * get_sb by a pseudo filesystem that directly populates itself. + */ + init_dlock_list_iter(&iter, &sbsec->isec_head); + dlist_for_each_entry_safe(isec, n, &iter, list) { struct inode *inode = isec->inode; - list_del_init(&isec->list); - spin_unlock(&sbsec->isec_lock); + + dlock_list_del_iter(&iter, &isec->list); + dlock_list_unlock(&iter); + inode = igrab(inode); if (inode) { if (!IS_PRIVATE(inode)) inode_doinit_with_dentry(inode, NULL); iput(inode); } - spin_lock(&sbsec->isec_lock); + + dlock_list_relock(&iter); } - spin_unlock(&sbsec->isec_lock); + WARN_ON_ONCE(!dlock_lists_empty(&sbsec->isec_head)); return rc; } @@ -1428,10 +1416,8 @@ static int inode_doinit_with_dentry(struct inode *inode, struct dentry *opt_dent /* Defer initialization until selinux_complete_init, after the initial policy is loaded and the security server is ready to handle calls. */ - spin_lock(&sbsec->isec_lock); - if (list_empty(&isec->list)) - list_add(&isec->list, &sbsec->isec_head); - spin_unlock(&sbsec->isec_lock); + if (list_empty(&isec->list.list)) + dlock_lists_add(&isec->list, &sbsec->isec_head); goto out_unlock; } @@ -2548,9 +2534,10 @@ static int selinux_sb_alloc_security(struct super_block *sb) { struct superblock_security_struct *sbsec = selinux_superblock(sb); + if (alloc_dlock_list_heads(&sbsec->isec_head)) + return -ENOMEM; + mutex_init(&sbsec->lock); - INIT_LIST_HEAD(&sbsec->isec_head); - spin_lock_init(&sbsec->isec_lock); sbsec->sid = SECINITSID_UNLABELED; sbsec->def_sid = SECINITSID_FILE; sbsec->mntpoint_sid = SECINITSID_UNLABELED; @@ -2558,6 +2545,15 @@ static int selinux_sb_alloc_security(struct super_block *sb) return 0; } +static void selinux_sb_free_security(struct super_block *sb) +{ + struct superblock_security_struct *sbsec = selinux_superblock(sb); + + if (!sbsec) + return; + free_dlock_list_heads(&sbsec->isec_head); +} + static inline int opt_len(const char *s) { bool open_quote = false; @@ -2841,7 +2837,7 @@ static int selinux_inode_alloc_security(struct inode *inode) u32 sid = current_sid(); spin_lock_init(&isec->lock); - INIT_LIST_HEAD(&isec->list); + init_dlock_list_node(&isec->list); isec->inode = inode; isec->sid = SECINITSID_UNLABELED; isec->sclass = SECCLASS_FILE; @@ -2920,6 +2916,7 @@ static int selinux_inode_init_security(struct inode *inode, struct inode *dir, if (rc) return rc; + /* Possibly defer initialization to selinux_complete_init. */ if (sbsec->flags & SE_SBINITIALIZED) { struct inode_security_struct *isec = selinux_inode(inode); @@ -7215,6 +7212,7 @@ static struct security_hook_list selinux_hooks[] __ro_after_init = { selinux_msg_queue_alloc_security), LSM_HOOK_INIT(shm_alloc_security, selinux_shm_alloc_security), LSM_HOOK_INIT(sb_alloc_security, selinux_sb_alloc_security), + LSM_HOOK_INIT(sb_free_security, selinux_sb_free_security), LSM_HOOK_INIT(inode_alloc_security, selinux_inode_alloc_security), LSM_HOOK_INIT(sem_alloc_security, selinux_sem_alloc_security), LSM_HOOK_INIT(secid_to_secctx, selinux_secid_to_secctx), diff --git a/security/selinux/include/objsec.h b/security/selinux/include/objsec.h index 8159fd53c3de..e0709f429c56 100644 --- a/security/selinux/include/objsec.h +++ b/security/selinux/include/objsec.h @@ -24,6 +24,7 @@ #include #include #include +#include #include #include "flask.h" #include "avc.h" @@ -45,7 +46,7 @@ enum label_initialized { struct inode_security_struct { struct inode *inode; /* back pointer to inode object */ - struct list_head list; /* list of inode_security_struct */ + struct dlock_list_node list; /* list of inode_security_struct */ u32 task_sid; /* SID of creating task */ u32 sid; /* SID of this object */ u16 sclass; /* security class of this object */ @@ -67,8 +68,7 @@ struct superblock_security_struct { unsigned short behavior; /* labeling behavior */ unsigned short flags; /* which mount options were specified */ struct mutex lock; - struct list_head isec_head; - spinlock_t isec_lock; + struct dlock_list_heads isec_head; }; struct msg_security_struct {