From patchwork Thu Dec 1 04:54:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kemeng Shi X-Patchwork-Id: 28198 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp75984wrr; Wed, 30 Nov 2022 20:55:38 -0800 (PST) X-Google-Smtp-Source: AA0mqf7x6xXi4gK+N3gRn2yKgiAKZCZP6ckSKDOrMSQ3L1Vqvit8/lMFwImtffpppjBzqGDbmR8i X-Received: by 2002:a05:6402:360b:b0:468:f365:dca with SMTP id el11-20020a056402360b00b00468f3650dcamr21998777edb.41.1669870538498; Wed, 30 Nov 2022 20:55:38 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669870538; cv=none; d=google.com; s=arc-20160816; b=D4lj0MI7lu0gJLT1zmHNNxSkyJ2gNGj1VeMLOgEsAj9TnuRmRhTwFz6WkDio/vR4QL 8Bjmu+/B9Zhjyh0dKHEcDGtOqaI1nmJkqsYRRtzUR4bORLJ5TipKAcqTcOYIgumRwX/l 76NaO7d8pmnL52a/M+SmsLZBb6pRhqYHrh5WV/ov/k2oldPt6bUNVrVN0g0NA7+IoVae AXw5qoPzTQOa8Qgknn9WJTM/O9dh2FPPJA7YeHnbTYRs9/oWc/SadUmIcUcXdPO433sd i8vLdvsx9kLjDFDjmrYyRP2wuc40wF8ful80j32St4+++siymaI9n81y2/vGEusmdD/T pkvw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from; bh=e2NThQ1/viC7V74zQA9nfcZrj3Re3dxAsOqRN0Rg0FM=; b=MRXlpvsM2ojZzbkecuCF+u3qncf5IY/Yvpc7sE5vxU1gWQ6h4PsB4xoXHR1GND5n1b SdVD6h6Yq7rMycXRwJHxXvn79KYbpjUxpVr0c3Oufvchbvhh1HyOcekDyTZ4yAFY302R DTFWDMtJyz4jV+cyONmUli91sx/ySaTB17Y0Sh99qbnpQmzBZGHmbUDNizBmXtU/yx80 vjFVNgRVKRc9/wC4khXpvREJ/+C0VRLAzTKS97EmSw0JMvDFHxUT6xsxwUI2Ha/1/GEN Rk+U1aPCGQjN31oChzz7LU5b5Z25Z2b1UnJAHgasZydz9dv5RJdNQGb3zYBcado2jVsb BmQA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id q5-20020a056402248500b004676b7d42d7si2927045eda.243.2022.11.30.20.55.14; Wed, 30 Nov 2022 20:55:38 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229461AbiLAEzB (ORCPT + 99 others); Wed, 30 Nov 2022 23:55:01 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54834 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229728AbiLAEy5 (ORCPT ); Wed, 30 Nov 2022 23:54:57 -0500 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A58FA9FEC1; Wed, 30 Nov 2022 20:54:56 -0800 (PST) Received: from kwepemi500016.china.huawei.com (unknown [172.30.72.54]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4NN3cg3ZLbzHwMr; Thu, 1 Dec 2022 12:53:39 +0800 (CST) Received: from huawei.com (10.174.178.129) by kwepemi500016.china.huawei.com (7.221.188.220) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Thu, 1 Dec 2022 12:54:10 +0800 From: Kemeng Shi To: CC: , , , , Subject: [PATCH 1/5] sbitmap: don't consume nr for inactive waitqueue to avoid lost wakeups Date: Thu, 1 Dec 2022 12:54:04 +0800 Message-ID: <20221201045408.21908-2-shikemeng@huawei.com> X-Mailer: git-send-email 2.14.1.windows.1 In-Reply-To: <20221201045408.21908-1-shikemeng@huawei.com> References: <20221201045408.21908-1-shikemeng@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.174.178.129] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To kwepemi500016.china.huawei.com (7.221.188.220) X-CFilter-Loop: Reflected X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750986169610360402?= X-GMAIL-MSGID: =?utf-8?q?1750986169610360402?= If we decremented queue without waiters, we should not decremente freed bits number "nr", or all "nr" could be consumed in a empty queue and no wakeup will be called. Currently, for case "wait_cnt > 0", "nr" will not be decremented if we decremented queue without watiers and retry is returned to avoid lost wakeups. However for case "wait_cnt == 0", "nr" will be decremented unconditionally and maybe decremented to zero. Although retry is returned by active state of queue, it's not actually executed for "nr" is zero. Fix this by only decrementing "nr" for active queue when "wait_cnt == 0". After this fix, "nr" will always be non-zero when we decremented inactive queue for case "wait_cnt == 0", so the need to retry could be returned by "nr" and active state of waitqueue returned for the same purpose is not needed. Signed-off-by: Kemeng Shi --- lib/sbitmap.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/lib/sbitmap.c b/lib/sbitmap.c index 7280ae8ca88c..e40759bcf821 100644 --- a/lib/sbitmap.c +++ b/lib/sbitmap.c @@ -604,7 +604,6 @@ static bool __sbq_wake_up(struct sbitmap_queue *sbq, int *nr) struct sbq_wait_state *ws; unsigned int wake_batch; int wait_cnt, cur, sub; - bool ret; if (*nr <= 0) return false; @@ -632,15 +631,15 @@ static bool __sbq_wake_up(struct sbitmap_queue *sbq, int *nr) if (wait_cnt > 0) return !waitqueue_active(&ws->wait); - *nr -= sub; - /* * When wait_cnt == 0, we have to be particularly careful as we are * responsible to reset wait_cnt regardless whether we've actually - * woken up anybody. But in case we didn't wakeup anybody, we still - * need to retry. + * woken up anybody. But in case we didn't wakeup anybody, we should + * not consume nr and need to retry to avoid lost wakeups. */ - ret = !waitqueue_active(&ws->wait); + if (waitqueue_active(&ws->wait)) + *nr -= sub; + wake_batch = READ_ONCE(sbq->wake_batch); /* @@ -669,7 +668,7 @@ static bool __sbq_wake_up(struct sbitmap_queue *sbq, int *nr) sbq_index_atomic_inc(&sbq->wake_index); atomic_set(&ws->wait_cnt, wake_batch); - return ret || *nr; + return *nr; } void sbitmap_queue_wake_up(struct sbitmap_queue *sbq, int nr) From patchwork Thu Dec 1 04:54:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kemeng Shi X-Patchwork-Id: 28199 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp76381wrr; Wed, 30 Nov 2022 20:56:46 -0800 (PST) X-Google-Smtp-Source: AA0mqf5DYr3PiykFpPOZfD7gzok3iWI97k9WwJbRo08+EQDqFRjZFR3gCgbszHFVhgovlrwNfH9R X-Received: by 2002:aa7:d816:0:b0:458:4868:814 with SMTP id v22-20020aa7d816000000b0045848680814mr58763846edq.190.1669870606219; Wed, 30 Nov 2022 20:56:46 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669870606; cv=none; d=google.com; s=arc-20160816; b=ocLsAYEMmsUKsg2LQnIfcRyysyj5CLjfBySwuBJv7oAsTD4rNM0MOg34bjIqHmkx0s osZOXfEeyUkNG9oLfV0b5r2iXoZRdMLNhHNUgcstEfBr46cMpS/BHM/xfzbEjycDlVNp EAZ590ibwmbZCy/vuQhTjCRR+d2i9d8uC8c8hVUy69uHidQUeRrkZ8lAjko+I2kT+7pp dww6il45RscoalDoeyUGwS9t6qHdxoMH95MAm3fFPJ5vWgvppM8gaRO8NVmcR5yQ565e sNCXg6APdxhKSLQXo0kHSxiuHRbHJf8BuvCZetptxZV9/zIPu+6FnrD1rbHbgzpqQqiG /7Ig== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from; bh=czWF03TrqHKlZO+kjYtNTQVHPQwBDeQEezHL5a8a8N0=; b=GGxQUB7vARLcn7pFAMJIDjl09U4uA21fHAXF9r/4J8MKOKm+FqvaOVg81CkLkmKruE 86KydFtXHIUZRee+xJhZcUwMo/UOGA5CLLjugx9lh+XCeK1m0TMv5QmJ4sZj1Ojj4MbI IHCc48cLdoTvtroaz0H3n2a5MutjDs5cxKOGSDVZK1cyJCFl4Vb4/sZAbXvx/vrzrxVS 1X5ggymaeEjMC37IYPlTYhTPBcv7i3gshdty4i9beOy/a82rXuuaMx/Zkn3RhsmCb2w1 PEyI+5Upm3r8/Zkcnh+dO5vJIr8YGgnW4mjErBeCiGbvG2vv4bHHanUkoHr7EaWqjeE5 +OOQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id qw26-20020a1709066a1a00b007829f6fed9dsi3100268ejc.232.2022.11.30.20.56.23; Wed, 30 Nov 2022 20:56:46 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229778AbiLAEzE (ORCPT + 99 others); Wed, 30 Nov 2022 23:55:04 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54856 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229744AbiLAEy5 (ORCPT ); Wed, 30 Nov 2022 23:54:57 -0500 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CABE89FEE2; Wed, 30 Nov 2022 20:54:56 -0800 (PST) Received: from kwepemi500016.china.huawei.com (unknown [172.30.72.54]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4NN3cg6K3SzHwN6; Thu, 1 Dec 2022 12:53:39 +0800 (CST) Received: from huawei.com (10.174.178.129) by kwepemi500016.china.huawei.com (7.221.188.220) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Thu, 1 Dec 2022 12:54:11 +0800 From: Kemeng Shi To: CC: , , , , Subject: [PATCH 2/5] sbitmap: remove unnecessary calculation of alloc_hint in __sbitmap_get_shallow Date: Thu, 1 Dec 2022 12:54:05 +0800 Message-ID: <20221201045408.21908-3-shikemeng@huawei.com> X-Mailer: git-send-email 2.14.1.windows.1 In-Reply-To: <20221201045408.21908-1-shikemeng@huawei.com> References: <20221201045408.21908-1-shikemeng@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.174.178.129] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To kwepemi500016.china.huawei.com (7.221.188.220) X-CFilter-Loop: Reflected X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750986240708972793?= X-GMAIL-MSGID: =?utf-8?q?1750986240708972793?= We calculate allow_hint in next word as following: /* low sb->shift bit of alloc_hint will be 0 after this shift */ alloc_hint = index << sb->shift; /* get low sb->shift bit of alloc_hit */ SB_NR_TO_BIT(sb, alloc_hint) So alloc_hit in next word will always be zero. Simpfy alloc_hit calculation in __sbitmap_get_shallow according to the alloc_hit calculation in __sbitmap_get. Signed-off-by: Kemeng Shi --- lib/sbitmap.c | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git a/lib/sbitmap.c b/lib/sbitmap.c index e40759bcf821..791edf5ea4ca 100644 --- a/lib/sbitmap.c +++ b/lib/sbitmap.c @@ -243,6 +243,7 @@ static int __sbitmap_get_shallow(struct sbitmap *sb, int nr = -1; index = SB_NR_TO_INDEX(sb, alloc_hint); + alloc_hint = SB_NR_TO_BIT(sb, alloc_hint); for (i = 0; i < sb->map_nr; i++) { again: @@ -250,7 +251,7 @@ static int __sbitmap_get_shallow(struct sbitmap *sb, min_t(unsigned int, __map_depth(sb, index), shallow_depth), - SB_NR_TO_BIT(sb, alloc_hint), true); + alloc_hint, true); if (nr != -1) { nr += index << sb->shift; break; @@ -260,13 +261,9 @@ static int __sbitmap_get_shallow(struct sbitmap *sb, goto again; /* Jump to next index. */ - index++; - alloc_hint = index << sb->shift; - - if (index >= sb->map_nr) { + alloc_hint = 0; + if (++index >= sb->map_nr) index = 0; - alloc_hint = 0; - } } return nr; From patchwork Thu Dec 1 04:54:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kemeng Shi X-Patchwork-Id: 28200 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp76393wrr; Wed, 30 Nov 2022 20:56:47 -0800 (PST) X-Google-Smtp-Source: AA0mqf4WUEgJ1XFlgiitd8yUFKp2v+90HLWIEwPeXb0P8wX5jqwtZT3aPeFikTvjLvlieW4MWzMu X-Received: by 2002:a05:6402:2993:b0:462:845:ba98 with SMTP id eq19-20020a056402299300b004620845ba98mr3478837edb.12.1669870607438; Wed, 30 Nov 2022 20:56:47 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669870607; cv=none; d=google.com; s=arc-20160816; b=wWan30EsF8wNvKJPHFUZ+mP9Wmewfmp6GI0GaFrR+e9hzzvoUc0dpU5FpHS0kOS0KW bzcM/+gUf4MSw+znWU/nfzsfz8yexhq3ZcVxqXLtjp1nvf9K5VaCDLY2tKw7ahejVqvi 9BFwZCbs1CS7+DQp1rRNFCX7bXanKBsk7KkTz+dcpWF2JZ17yk5A5fn1YGCvqm9d3qOZ uip/M6UcUBP3y0nUFt8WfypBA6kP3glGrsBP7wG7Al+9IdpKhhaPwnbWVPyyTXGDQYMA ZyDHGCpWPaDyR07Gxm066/u8sQdodc8JnhBRffMv1TCqdhr9C56IV16MCuYU+J7I9HbS 6/tA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from; bh=+qYHcydYynGpb5+uiW7pbsJZIAgZUgq1sTrJtP5PI+A=; b=T6mJRfLrrb7J+vpXu4FFxRBGzouycm2vL37/I/s4PA0bTfRBaolGcwyaKjdHgh7/Vp brRbXb/0LnS2zx0tFl/Mx8xFHr2Z62IcPQw9qu4avsKZ4hIvlLWl0R6fdlsVDYbwF11Z Rs+bxwejl4USsGfzzzObyYE2pBqJKuyTeoDWtpVDJWcEEvE5GlxS1c08QtoUuiGrWCYx dptRMy5gfiZwnB5T7Q+2flylW9tx8pRdFtQ9bKhhEk3TXQGhFEzebZCPCyE/ilLFUNPC 7lzcrMVLJaXSQsvQsq750JyXhs1OpqSIOCnlysUJgE0Jmsv7pJxQAHZV/5nabElBoYXw BNlg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id d8-20020aa7d688000000b0046b58b0497asi2767744edr.314.2022.11.30.20.56.24; Wed, 30 Nov 2022 20:56:47 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229810AbiLAEzI (ORCPT + 99 others); Wed, 30 Nov 2022 23:55:08 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54854 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229747AbiLAEy5 (ORCPT ); Wed, 30 Nov 2022 23:54:57 -0500 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B065F9FEC7; Wed, 30 Nov 2022 20:54:56 -0800 (PST) Received: from kwepemi500016.china.huawei.com (unknown [172.30.72.54]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4NN3ch0VwhzHwN9; Thu, 1 Dec 2022 12:53:40 +0800 (CST) Received: from huawei.com (10.174.178.129) by kwepemi500016.china.huawei.com (7.221.188.220) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Thu, 1 Dec 2022 12:54:11 +0800 From: Kemeng Shi To: CC: , , , , Subject: [PATCH 3/5] sbitmap: remove redundant check in __sbitmap_queue_get_batch Date: Thu, 1 Dec 2022 12:54:06 +0800 Message-ID: <20221201045408.21908-4-shikemeng@huawei.com> X-Mailer: git-send-email 2.14.1.windows.1 In-Reply-To: <20221201045408.21908-1-shikemeng@huawei.com> References: <20221201045408.21908-1-shikemeng@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.174.178.129] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To kwepemi500016.china.huawei.com (7.221.188.220) X-CFilter-Loop: Reflected X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750986242508182689?= X-GMAIL-MSGID: =?utf-8?q?1750986242508182689?= Commit fbb564a557809 ("lib/sbitmap: Fix invalid loop in __sbitmap_queue_get_batch()") mentioned that "Checking free bits when setting the target bits. Otherwise, it may reuse the busying bits." This commit add check to make sure all masked bits in word before cmpxchg is zero. Then the existing check after cmpxchg to check any zero bit is existing in masked bits in word is redundant. Actually, old value of word before cmpxchg is stored in val and we will filter out busy bits in val by "(get_mask & ~val)" after cmpxchg. So we will not reuse busy bits methioned in commit fbb564a557809 ("lib/sbitmap: Fix invalid loop in __sbitmap_queue_get_batch()"). Revert new-added check to remove redundant check. Signed-off-by: Kemeng Shi --- lib/sbitmap.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/lib/sbitmap.c b/lib/sbitmap.c index 791edf5ea4ca..70b62b1e40a1 100644 --- a/lib/sbitmap.c +++ b/lib/sbitmap.c @@ -534,11 +534,9 @@ unsigned long __sbitmap_queue_get_batch(struct sbitmap_queue *sbq, int nr_tags, get_mask = ((1UL << nr_tags) - 1) << nr; val = READ_ONCE(map->word); - do { - if ((val & ~get_mask) != val) - goto next; - } while (!atomic_long_try_cmpxchg(ptr, &val, - get_mask | val)); + while (!atomic_long_try_cmpxchg(ptr, &val, + get_mask | val)) + ; get_mask = (get_mask & ~val) >> nr; if (get_mask) { *offset = nr + (index << sb->shift); From patchwork Thu Dec 1 04:54:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kemeng Shi X-Patchwork-Id: 28201 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp77104wrr; Wed, 30 Nov 2022 20:58:32 -0800 (PST) X-Google-Smtp-Source: AA0mqf50eihTtSGDPho26VRAdlOHyVh1zRiZR9VfYtx4H6Unv0UtyTAQyDzO39lqdX5KdNlebUVH X-Received: by 2002:a50:fc10:0:b0:464:2afe:ae18 with SMTP id i16-20020a50fc10000000b004642afeae18mr59965583edr.183.1669870711804; Wed, 30 Nov 2022 20:58:31 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669870711; cv=none; d=google.com; s=arc-20160816; b=Er1BhId6J32hpsBeIH6d8/kwd+HCnKAPTL0U5AcZKwApAsPtZr8+0NXxLWrmFVTrb5 At8gseKPcqrT9Q/19tw1rX3pKNS6hXtaEMh8ZX4CnzT2DPBFOJOI+0sxyLliPPxu3v5y 4es+8Jm63mpl8iiEblhrwMG8vEWfMKcElZHgHb1HKsIYYRW6LMUtIz/v/WZLp3ufeR8H WtZwwHxwbfbLZHwazPV8V2YR3qTqyV8k2Ew6UuG6dWHAApOeRDgczHCkyZuoLMDLRBSh N9zzOgMMhESbI7YMqD5n+zA2npAcg9Faja4JNWuVaDyw75CcwQ/Pq3DcQ3JKlpkAdbIT wuZQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from; bh=PDZI7FQmIUVnjrF2VMRezi48adWtXOe5Xf8F6u755y0=; b=PZst7AEk4ZVvAiz1MTckMMtsAPfGxQFqhYyTqXBjXKdXt2gvr0OIzz/M4Xc4TeD/q9 mwAVwOHnNRb7x7bJPd0MDzfALpc4Nl/yJqYl8j/6w5PmGpck+YAp5fenYs/qLkJHHZjf MbU/AyC3otLz2w9bKeSxiCBPTmFh/l97UV8cvHmcyIVFc3jAVEfW5JyaUl2ycbfquQK7 ly17K++7NjcE6X+bcwTBs8ekkwaULFqv+MFIJViSBlIqkj9D9XNaPiyTYsApPAq8QuPj a9lYbAjWzlJsmYt7g3ppOOkrEXpcwczhP6uz2pXYfShBlCYS4u0pjD+yEoN0rK55uNzh 487g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id ds15-20020a170907724f00b007aec7f879basi3539101ejc.22.2022.11.30.20.58.08; Wed, 30 Nov 2022 20:58:31 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229838AbiLAEzN (ORCPT + 99 others); Wed, 30 Nov 2022 23:55:13 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54934 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229755AbiLAEy7 (ORCPT ); Wed, 30 Nov 2022 23:54:59 -0500 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A8C82A0576; Wed, 30 Nov 2022 20:54:57 -0800 (PST) Received: from kwepemi500016.china.huawei.com (unknown [172.30.72.54]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4NN3ch3FKyzHwJt; Thu, 1 Dec 2022 12:53:40 +0800 (CST) Received: from huawei.com (10.174.178.129) by kwepemi500016.china.huawei.com (7.221.188.220) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Thu, 1 Dec 2022 12:54:12 +0800 From: Kemeng Shi To: CC: , , , , Subject: [PATCH 4/5] sbitmap: rewrite sbitmap_find_bit_in_index to reduce repeat code Date: Thu, 1 Dec 2022 12:54:07 +0800 Message-ID: <20221201045408.21908-5-shikemeng@huawei.com> X-Mailer: git-send-email 2.14.1.windows.1 In-Reply-To: <20221201045408.21908-1-shikemeng@huawei.com> References: <20221201045408.21908-1-shikemeng@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.174.178.129] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To kwepemi500016.china.huawei.com (7.221.188.220) X-CFilter-Loop: Reflected X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750986351399033079?= X-GMAIL-MSGID: =?utf-8?q?1750986351399033079?= Rewrite sbitmap_find_bit_in_index as following: 1. Rename sbitmap_find_bit_in_index to sbitmap_find_bit_in_word 2. Accept "struct sbitmap_word *" directly instead of accepting "struct sbitmap *" and "int index" to get "struct sbitmap_word *". 3. Accept depth/shallow_depth and wrap for __sbitmap_get_word from caller to support need of both __sbitmap_get_shallow and __sbitmap_get. With helper function sbitmap_find_bit_in_word, we can remove repeat code in __sbitmap_get_shallow to find bit considring deferred clear. Signed-off-by: Kemeng Shi --- lib/sbitmap.c | 29 ++++++++++++++--------------- 1 file changed, 14 insertions(+), 15 deletions(-) diff --git a/lib/sbitmap.c b/lib/sbitmap.c index 70b62b1e40a1..b6a2cdb9bdaf 100644 --- a/lib/sbitmap.c +++ b/lib/sbitmap.c @@ -167,15 +167,16 @@ static int __sbitmap_get_word(unsigned long *word, unsigned long depth, return nr; } -static int sbitmap_find_bit_in_index(struct sbitmap *sb, int index, - unsigned int alloc_hint) +static int sbitmap_find_bit_in_word(struct sbitmap_word *map, + unsigned int depth, + unsigned int alloc_hint, + bool wrap) { - struct sbitmap_word *map = &sb->map[index]; int nr; do { - nr = __sbitmap_get_word(&map->word, __map_depth(sb, index), - alloc_hint, !sb->round_robin); + nr = __sbitmap_get_word(&map->word, depth, + alloc_hint, wrap); if (nr != -1) break; if (!sbitmap_deferred_clear(map)) @@ -203,7 +204,8 @@ static int __sbitmap_get(struct sbitmap *sb, unsigned int alloc_hint) alloc_hint = 0; for (i = 0; i < sb->map_nr; i++) { - nr = sbitmap_find_bit_in_index(sb, index, alloc_hint); + nr = sbitmap_find_bit_in_word(&sb->map[index], __map_depth(sb, index), + alloc_hint, !sb->round_robin); if (nr != -1) { nr += index << sb->shift; break; @@ -246,20 +248,17 @@ static int __sbitmap_get_shallow(struct sbitmap *sb, alloc_hint = SB_NR_TO_BIT(sb, alloc_hint); for (i = 0; i < sb->map_nr; i++) { -again: - nr = __sbitmap_get_word(&sb->map[index].word, - min_t(unsigned int, - __map_depth(sb, index), - shallow_depth), - alloc_hint, true); + nr = sbitmap_find_bit_in_word(&sb->map[index], + min_t(unsigned int, + __map_depth(sb, index), + shallow_depth), + alloc_hint, true); + if (nr != -1) { nr += index << sb->shift; break; } - if (sbitmap_deferred_clear(&sb->map[index])) - goto again; - /* Jump to next index. */ alloc_hint = 0; if (++index >= sb->map_nr) From patchwork Thu Dec 1 04:54:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kemeng Shi X-Patchwork-Id: 28202 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp82502wrr; Wed, 30 Nov 2022 21:13:07 -0800 (PST) X-Google-Smtp-Source: AA0mqf6IxktO6/HQpAyAu+YwWhDilGw4LR7x1ut+ZpABNcQMf0NHxu3nM+LOdNeF57Xhe9rtC1Tl X-Received: by 2002:aa7:d2d5:0:b0:469:9951:ac3a with SMTP id k21-20020aa7d2d5000000b004699951ac3amr43989835edr.339.1669871587444; Wed, 30 Nov 2022 21:13:07 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669871587; cv=none; d=google.com; s=arc-20160816; b=1KPu/QehaSOVzGybfP4SieNrsGgiIMUFb5oOb15zie2Y9KQr5q2nsEbW6JTeYJRd4d 5+k9CY0RKj9hnuHDKcekadwmVKkvaKC+vCH2ZQKcIBV+KhKl41ndMxWT5A8KXKFR8KrG 5Wur5s+k8FLxBHsdIby7UMfQUqndb3AQXD17UHircohT+NvxUqw2qFjjyXL1w7JvnZOi bpy7BvmseVtS/1TI7qUIgERKPAHNpnMq+UNH5B1dCdr6yxgCgEecmexLIyUgL71Zx+GQ Y5VMpZMSP/+/Jj/p2dpb7xvUhfHnadHbUjcZaNd+IvsWmdApZgC2cGILn9PmpLZn2nNf s3gQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from; bh=lwmm2IrOpYnOVuf4f5I96S4VhD1i6DDs6fDK4Etm4to=; b=aF7hEyGPX2kYjrZ9FMIHy9vf9d8KuOIHkZoC0x/JsbekjqAUbtdG7Kx8esAgw/LK3g J0vhKd4Xm6KwDifA2E0jR3P+SkKbPIQc2ODjKIouDlwcOc2VIELzBQd5Z3PmrLo3rIdJ 3PsEjwaKN1UPUcOl3eD1iCCpPDqlGtkDX+9GLC1vXD3O9PAezyWY0DcJV6+O2UJ3elib uNa5Rvl+eNdKdivH61MzJqNOn3O0lyj6gp6z+L5tSPRZv5JOD1Q97yHmMgUC8zbOXZA9 pKUsLPzV4ywLan637U/gAZmwC+MUd0YtRN4ijomfvnAU3DNgIjzOU2IuopUmvVjjJr7o u2Hg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id hd13-20020a170907968d00b007bff09aec2csi3306076ejc.1008.2022.11.30.21.12.37; Wed, 30 Nov 2022 21:13:07 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229820AbiLAEzL (ORCPT + 99 others); Wed, 30 Nov 2022 23:55:11 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54924 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229754AbiLAEy7 (ORCPT ); Wed, 30 Nov 2022 23:54:59 -0500 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 647BFA055A; Wed, 30 Nov 2022 20:54:57 -0800 (PST) Received: from kwepemi500016.china.huawei.com (unknown [172.30.72.54]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4NN3cj6nwbzHwNG; Thu, 1 Dec 2022 12:53:41 +0800 (CST) Received: from huawei.com (10.174.178.129) by kwepemi500016.china.huawei.com (7.221.188.220) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Thu, 1 Dec 2022 12:54:13 +0800 From: Kemeng Shi To: CC: , , , , Subject: [PATCH 5/5] sbitmap: add sbitmap_find_bit to remove repeat code in __sbitmap_get/__sbitmap_get_shallow Date: Thu, 1 Dec 2022 12:54:08 +0800 Message-ID: <20221201045408.21908-6-shikemeng@huawei.com> X-Mailer: git-send-email 2.14.1.windows.1 In-Reply-To: <20221201045408.21908-1-shikemeng@huawei.com> References: <20221201045408.21908-1-shikemeng@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.174.178.129] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To kwepemi500016.china.huawei.com (7.221.188.220) X-CFilter-Loop: Reflected X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750987269909478633?= X-GMAIL-MSGID: =?utf-8?q?1750987269909478633?= There are three differences between __sbitmap_get and __sbitmap_get_shallow when searching free bit: 1. __sbitmap_get_shallow limit number of bit to search per word. __sbitmap_get has no such limit. 2. __sbitmap_get_shallow always searches with wrap set. __sbitmap_get set wrap according to round_robin. 3. __sbitmap_get_shallow always searches from first bit in first word. __sbitmap_get searches from first bit when round_robin is not set otherwise searches from SB_NR_TO_BIT(sb, alloc_hint). Add helper function sbitmap_find_bit function to do common search while accept "limit depth per word", "wrap flag" and "first bit to search" from caller to support the need of both __sbitmap_get and __sbitmap_get_shallow. Signed-off-by: Kemeng Shi --- lib/sbitmap.c | 72 +++++++++++++++++++++++++-------------------------- 1 file changed, 35 insertions(+), 37 deletions(-) diff --git a/lib/sbitmap.c b/lib/sbitmap.c index b6a2cdb9bdaf..93e7db484b96 100644 --- a/lib/sbitmap.c +++ b/lib/sbitmap.c @@ -186,26 +186,22 @@ static int sbitmap_find_bit_in_word(struct sbitmap_word *map, return nr; } -static int __sbitmap_get(struct sbitmap *sb, unsigned int alloc_hint) +static int sbitmap_find_bit(struct sbitmap *sb, + unsigned int depth, + unsigned int index, + unsigned int alloc_hint, + bool wrap) { - unsigned int i, index; + unsigned int i; int nr = -1; - index = SB_NR_TO_INDEX(sb, alloc_hint); - - /* - * Unless we're doing round robin tag allocation, just use the - * alloc_hint to find the right word index. No point in looping - * twice in find_next_zero_bit() for that case. - */ - if (sb->round_robin) - alloc_hint = SB_NR_TO_BIT(sb, alloc_hint); - else - alloc_hint = 0; - for (i = 0; i < sb->map_nr; i++) { - nr = sbitmap_find_bit_in_word(&sb->map[index], __map_depth(sb, index), - alloc_hint, !sb->round_robin); + nr = sbitmap_find_bit_in_word(&sb->map[index], + min_t(unsigned int, + __map_depth(sb, index), + depth), + alloc_hint, wrap); + if (nr != -1) { nr += index << sb->shift; break; @@ -215,11 +211,32 @@ static int __sbitmap_get(struct sbitmap *sb, unsigned int alloc_hint) alloc_hint = 0; if (++index >= sb->map_nr) index = 0; + } return nr; } +static int __sbitmap_get(struct sbitmap *sb, unsigned int alloc_hint) +{ + unsigned int index; + + index = SB_NR_TO_INDEX(sb, alloc_hint); + + /* + * Unless we're doing round robin tag allocation, just use the + * alloc_hint to find the right word index. No point in looping + * twice in find_next_zero_bit() for that case. + */ + if (sb->round_robin) + alloc_hint = SB_NR_TO_BIT(sb, alloc_hint); + else + alloc_hint = 0; + + return sbitmap_find_bit(sb, UINT_MAX, index, alloc_hint, + !sb->round_robin); +} + int sbitmap_get(struct sbitmap *sb) { int nr; @@ -241,31 +258,12 @@ static int __sbitmap_get_shallow(struct sbitmap *sb, unsigned int alloc_hint, unsigned long shallow_depth) { - unsigned int i, index; - int nr = -1; + unsigned int index; index = SB_NR_TO_INDEX(sb, alloc_hint); alloc_hint = SB_NR_TO_BIT(sb, alloc_hint); - for (i = 0; i < sb->map_nr; i++) { - nr = sbitmap_find_bit_in_word(&sb->map[index], - min_t(unsigned int, - __map_depth(sb, index), - shallow_depth), - alloc_hint, true); - - if (nr != -1) { - nr += index << sb->shift; - break; - } - - /* Jump to next index. */ - alloc_hint = 0; - if (++index >= sb->map_nr) - index = 0; - } - - return nr; + return sbitmap_find_bit(sb, shallow_depth, index, alloc_hint, true); } int sbitmap_get_shallow(struct sbitmap *sb, unsigned long shallow_depth)