From patchwork Mon Nov 20 17:46:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 167313 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2b07:b0:403:3b70:6f57 with SMTP id io7csp105894vqb; Mon, 20 Nov 2023 09:48:22 -0800 (PST) X-Google-Smtp-Source: AGHT+IFGXI7f2DSVGQB7SBXZqzf/m8NVbW34JEDur4GjY44/uXl8pZ/9D52POpipgCYnitGEa6iL X-Received: by 2002:a17:902:c40a:b0:1cc:3fce:8aa8 with SMTP id k10-20020a170902c40a00b001cc3fce8aa8mr6956260plk.6.1700502502092; Mon, 20 Nov 2023 09:48:22 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700502502; cv=none; d=google.com; s=arc-20160816; b=dV0OlPKjdHnpZMx9ZnhxEEuRt3tgcIUxES9kb8vMTeC+vjyNmHrkSrwmGAZQbRq0dv yiRMpgSnoc9R+VRYHEfi79T8IT72TVRRQbElKIb0hUdiHcJH4ok07Qx4SKSDX+1WY/sC RWtZKI5sJOZjWW1KaaX86lfx3k78zOZ7/ARaa7/sMzPLeJK++ly5rbaswk6DgKCZ+eIB VJ3khijAQmqtMUjJ0AdLIalasjqpz6hjBeh+QjFlBHH4g6f2BuaZaGBs7blL2JOm2LcY VwE43ueNtqNKX3hPDOfWacJgW+ilD0CZAiAoZBhe5BL3/TOB7PARFDuHVDUY4SqyxfOR f1Ug== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=9mwqfN/pqk9HRQ931HoPPP333OlumxbY++YPCTeY76U=; fh=qWoxmPN1zRqnLSoCWGpfFsDtzJsx/SGdqx98lbP+Uho=; b=r7CtdSe/dU2lZc8YazwtBgKKJH8tgdQAJC2q1uIfwOCqnwqLmd9V1USMC4YauQyouB N73uafLHjwZAYRXHVvCQ2/R7/dHyXAtPbMomvHucIQntIZM21Q0XzCd7V7ZIkcIDJs6E WcoK0zNc3owcKNhj59lGwED/9q7d0FN9R2HHfWp/kV3c/klqjj0qv79gK6v+vg6rv9r2 ycrmzxW/5JFNHYg+Pou0dDHHLgLAnftUKFzYWGwkdk5x2XCdSlV5FZtwvW7QgaQrI36l 95TeaW/jLSdUFcj69YpnwEOozUvLE5Tfy6BNBsdyuvTHu5IPIEL6IrVU6wohN10oWvHv ya3Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b="bR/I30d5"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from fry.vger.email (fry.vger.email. [2620:137:e000::3:8]) by mx.google.com with ESMTPS id a4-20020a170902ee8400b001c61bd9db5csi8099594pld.317.2023.11.20.09.48.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Nov 2023 09:48:22 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) client-ip=2620:137:e000::3:8; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b="bR/I30d5"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by fry.vger.email (Postfix) with ESMTP id 38AC3804ADBF; Mon, 20 Nov 2023 09:48:11 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at fry.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232871AbjKTRre (ORCPT + 27 others); Mon, 20 Nov 2023 12:47:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51242 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232359AbjKTRrc (ORCPT ); Mon, 20 Nov 2023 12:47:32 -0500 Received: from out-189.mta1.migadu.com (out-189.mta1.migadu.com [IPv6:2001:41d0:203:375::bd]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1FB29BA for ; Mon, 20 Nov 2023 09:47:29 -0800 (PST) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1700502447; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9mwqfN/pqk9HRQ931HoPPP333OlumxbY++YPCTeY76U=; b=bR/I30d5ng/lYBtNQqgdoGfH5paO2CtcP21wojJGbtHFG33q9OSF6VODY4kRaiORiokLQo iZpd9D4Xa29yBgsAA1WjhOAcCg0lSOCGSH3njrApHXcLDY08VxXeaX0yqJ6yzjAbR1DXy7 0Hwm3iz+UTEkMpPA3RFCPkvRL0wjaJM= From: andrey.konovalov@linux.dev To: Andrew Morton Cc: Andrey Konovalov , Marco Elver , Alexander Potapenko , Dmitry Vyukov , Vlastimil Babka , kasan-dev@googlegroups.com, Evgenii Stepanov , Oscar Salvador , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v4 01/22] lib/stackdepot: print disabled message only if truly disabled Date: Mon, 20 Nov 2023 18:46:59 +0100 Message-Id: <73a25c5fff29f3357cd7a9330e85e09bc8da2cbe.1700502145.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on fry.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (fry.vger.email [0.0.0.0]); Mon, 20 Nov 2023 09:48:11 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783106111733371074 X-GMAIL-MSGID: 1783106111733371074 From: Andrey Konovalov Currently, if stack_depot_disable=off is passed to the kernel command-line after stack_depot_disable=on, stack depot prints a message that it is disabled, while it is actually enabled. Fix this by moving printing the disabled message to stack_depot_early_init. Place it before the __stack_depot_early_init_requested check, so that the message is printed even if early stack depot init has not been requested. Also drop the stack_table = NULL assignment from disable_stack_depot, as stack_table is NULL by default. Fixes: e1fdc403349c ("lib: stackdepot: add support to disable stack depot") Reviewed-by: Marco Elver Signed-off-by: Andrey Konovalov Reviewed-by: Oscar Salvador --- lib/stackdepot.c | 24 +++++++++++++++--------- 1 file changed, 15 insertions(+), 9 deletions(-) diff --git a/lib/stackdepot.c b/lib/stackdepot.c index 2f5aa851834e..0eeaef4f2523 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -101,14 +101,7 @@ static int next_pool_required = 1; static int __init disable_stack_depot(char *str) { - int ret; - - ret = kstrtobool(str, &stack_depot_disabled); - if (!ret && stack_depot_disabled) { - pr_info("disabled\n"); - stack_table = NULL; - } - return 0; + return kstrtobool(str, &stack_depot_disabled); } early_param("stack_depot_disable", disable_stack_depot); @@ -130,6 +123,15 @@ int __init stack_depot_early_init(void) return 0; __stack_depot_early_init_passed = true; + /* + * Print disabled message even if early init has not been requested: + * stack_depot_init() will not print one. + */ + if (stack_depot_disabled) { + pr_info("disabled\n"); + return 0; + } + /* * If KASAN is enabled, use the maximum order: KASAN is frequently used * in fuzzing scenarios, which leads to a large number of different @@ -138,7 +140,11 @@ int __init stack_depot_early_init(void) if (kasan_enabled() && !stack_bucket_number_order) stack_bucket_number_order = STACK_BUCKET_NUMBER_ORDER_MAX; - if (!__stack_depot_early_init_requested || stack_depot_disabled) + /* + * Check if early init has been requested after setting + * stack_bucket_number_order: stack_depot_init() uses its value. + */ + if (!__stack_depot_early_init_requested) return 0; /* From patchwork Mon Nov 20 17:47:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 167317 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2b07:b0:403:3b70:6f57 with SMTP id io7csp106198vqb; Mon, 20 Nov 2023 09:48:55 -0800 (PST) X-Google-Smtp-Source: AGHT+IHeKzrj9j/PakqatzHwTVhkv0MSIjFcCMdfvkjuFD6TMdkm+ZcITqChOCMRDy7gt3TntijC X-Received: by 2002:a05:6a20:918e:b0:187:e3a5:b35d with SMTP id v14-20020a056a20918e00b00187e3a5b35dmr235509pzd.13.1700502534924; Mon, 20 Nov 2023 09:48:54 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700502534; cv=none; d=google.com; s=arc-20160816; b=KNdPC1AetPzMKC6+8H/vPo9fCbCIkw+mONxFbMDmE59C20GvPplw5zp/Xe8Fwe4p9G AxkuYKg3Ray8S2GQjzFE9GT/PWWm8WMRZoWQ915gEEH2eX4sXUNOYPbP77t940x8bUZ4 JcFxbZydP8MI4dAYESgyZDy7076jOMgj9Qp0ZKw++pKN/oaItBjrkczUwrC+AdN3cw88 ZNp19V1ryEASj5w+HDyZSZ6Qtvn9mni52JweTP6513/nZvIynf6PKypTOoWCx4bccwdp 5Xs2vRFoHOV/8AWLGWkGzyyiGMjfbqj2UZEslptf56JIOncA3o5zLukFhsExDY8COWkI sbdw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=4GD92F9WT2lJ15sdFM57D7OnzHXCVz8Opyiuivl9/aY=; fh=qWoxmPN1zRqnLSoCWGpfFsDtzJsx/SGdqx98lbP+Uho=; b=zjNWfFH4LXv5MxpyYLosUdl3XU2Cyi/TJMVKe5ZkesuMavq4QUBWIpVBx+rgXqacMk 1HSefcoags/VViHOovEX7FmRmQZmgLyCgJIpaOP34YZQ+H9O9QGQydfLRpbTaVXOVtz1 z51PvaDPuJtm1TkOxcI/KXVhzWp/USQ5+5yIep38ZJ3TUK5Gl47s4TjoUz+Jxxixf/mB uH8iiDQyn5i9+GlPExR5URYFAxc3v/OxhY6+fB9YVahkOQ6cO1FL7IZxzT86Bj8SpOWR trPueeWsVNfiMCE7npuPBOCWI1aX73A5iZ2lsSAXqo3xanufyvfXD+xQmHNgkon9DG8s NsWw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=qyhdPL9y; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from groat.vger.email (groat.vger.email. [23.128.96.35]) by mx.google.com with ESMTPS id x28-20020aa78f1c000000b006be08015248si8366325pfr.294.2023.11.20.09.48.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Nov 2023 09:48:54 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) client-ip=23.128.96.35; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=qyhdPL9y; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by groat.vger.email (Postfix) with ESMTP id A248A80A8BB8; Mon, 20 Nov 2023 09:48:52 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at groat.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233862AbjKTRrk (ORCPT + 27 others); Mon, 20 Nov 2023 12:47:40 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51254 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232547AbjKTRrd (ORCPT ); Mon, 20 Nov 2023 12:47:33 -0500 Received: from out-187.mta1.migadu.com (out-187.mta1.migadu.com [95.215.58.187]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B871510E for ; Mon, 20 Nov 2023 09:47:29 -0800 (PST) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1700502447; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4GD92F9WT2lJ15sdFM57D7OnzHXCVz8Opyiuivl9/aY=; b=qyhdPL9yHGZwgbX97BEztXeCz1Zt+rIj3RmbbGcjPTQsTwh98G4ATjxhFfnwka3CKqD1b6 x9kwKDWmny/BO2cvVbgqEwg2/wpgt3BzZx3yBf1KxUz7o5wnkBkYXhX4a/sfMby5QWyf2c 4HelafeGlMBXbFW+ItTdiGsKvqfzEZg= From: andrey.konovalov@linux.dev To: Andrew Morton Cc: Andrey Konovalov , Marco Elver , Alexander Potapenko , Dmitry Vyukov , Vlastimil Babka , kasan-dev@googlegroups.com, Evgenii Stepanov , Oscar Salvador , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v4 02/22] lib/stackdepot: check disabled flag when fetching Date: Mon, 20 Nov 2023 18:47:00 +0100 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Mon, 20 Nov 2023 09:48:52 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783106145974898764 X-GMAIL-MSGID: 1783106145974898764 From: Andrey Konovalov Do not try fetching a stack trace from the stack depot if the stack_depot_disabled flag is enabled. Reviewed-by: Alexander Potapenko Signed-off-by: Andrey Konovalov Reviewed-by: Oscar Salvador --- lib/stackdepot.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/stackdepot.c b/lib/stackdepot.c index 0eeaef4f2523..f8a8033e1dc8 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -483,7 +483,7 @@ unsigned int stack_depot_fetch(depot_stack_handle_t handle, */ kmsan_unpoison_memory(entries, sizeof(*entries)); - if (!handle) + if (!handle || stack_depot_disabled) return 0; if (parts.pool_index > pool_index_cached) { From patchwork Mon Nov 20 17:47:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 167311 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2b07:b0:403:3b70:6f57 with SMTP id io7csp105872vqb; Mon, 20 Nov 2023 09:48:20 -0800 (PST) X-Google-Smtp-Source: AGHT+IGpWpZMi0mVX0TEmWn/fvXFubrMJACZB+wKfxQF++8yI/Ga0UY38yMlTCylF4ua4c1d+tFK X-Received: by 2002:a05:6a20:12ca:b0:187:1165:c665 with SMTP id v10-20020a056a2012ca00b001871165c665mr6694755pzg.58.1700502499915; Mon, 20 Nov 2023 09:48:19 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700502499; cv=none; d=google.com; s=arc-20160816; b=gwqNOyXn1uMX6dimio/qSrfJu+1JTzh5NBStmVyoiqF3qtLLvt8iAOrlZpih66EIKB DVNBfK+zTZ+2mUCMcZAGByyTGujnRZDBsM1KmG9JHBKm6Nuzbt0cvCI4uc/kp42Y/hpD JLQcG92xnj5+jLCkS2VC49GZHhwcaf5/NMoc1d0eiyQKG7SAIeC+Z5EfffGShFLDz1H+ HZgJLlr+z31vicT2xoPB/WJ4qWkwDOP4S+AFDPILNf9mZ1SH6tZHTh8mNZYARZhp1AJj TfQrjpwsTTgOkn91XvJ4CnqCAvv20tIu3hq97W15bqFYlsCTb+bS+L9IiBSyGwKRpY6C reiA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=VQKv8ipVyqmLqtwmF3tWarfk+ElO5xq6wgwn4qj22hY=; fh=qWoxmPN1zRqnLSoCWGpfFsDtzJsx/SGdqx98lbP+Uho=; b=Y73Ivh/LoyfPCDmAV0H9fNIpvESBU8z+pJKa2nYKIxRPeWnBS2/9WrKRO5Wy2Z7JFW GFvkcOx9cakZtfDn/JsH4A9GU15rhMJFfknLJraaDgU+E9ZX+IUfet+LjMsHn1YhGj8C gozbxz3b7NrLY8/97TvnyFx0teyQYEG765Nxmwz8uCcU1ZsrSRcQFih4is4269ppQyLZ dh8QC0FIYQ9Ftn+ObHlFwbh06j8hnO62Ma4fBXF7qqKTJSVHf14KHh4F1vodHWKTPZka ui2lTURYel/1HMhGUy5ABqEdEQPdwC2YvG+QEXIBRKHv14UVQuZdR98htsWtPZUSw+BA nCnA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=DpwTRKlG; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id fd23-20020a056a002e9700b0068ff741579fsi8981499pfb.318.2023.11.20.09.48.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Nov 2023 09:48:19 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=DpwTRKlG; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 7E2C080DECF0; Mon, 20 Nov 2023 09:47:55 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233971AbjKTRrn (ORCPT + 27 others); Mon, 20 Nov 2023 12:47:43 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51272 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232734AbjKTRrd (ORCPT ); Mon, 20 Nov 2023 12:47:33 -0500 Received: from out-172.mta1.migadu.com (out-172.mta1.migadu.com [IPv6:2001:41d0:203:375::ac]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3E65411C for ; Mon, 20 Nov 2023 09:47:30 -0800 (PST) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1700502448; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VQKv8ipVyqmLqtwmF3tWarfk+ElO5xq6wgwn4qj22hY=; b=DpwTRKlGqKrBB4M8buYXFUbh5C4t1DeCOJCZmUpqezNSRGsyD+Wdylo613HdhfnrTQ5s+e pjLcUIFVsXiRTj7skGIcX0r4YxbEBcZF9ntx+1uASDYWt1MOFzRn7mamvzav2NVLFYw6Tg zTiWDgpYpFtnG/OSWJ9T8vQPvfnqvc0= From: andrey.konovalov@linux.dev To: Andrew Morton Cc: Andrey Konovalov , Marco Elver , Alexander Potapenko , Dmitry Vyukov , Vlastimil Babka , kasan-dev@googlegroups.com, Evgenii Stepanov , Oscar Salvador , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v4 03/22] lib/stackdepot: simplify __stack_depot_save Date: Mon, 20 Nov 2023 18:47:01 +0100 Message-Id: <3b0763c8057a1cf2f200ff250a5f9580ee36a28c.1700502145.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Mon, 20 Nov 2023 09:47:55 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783106109720650604 X-GMAIL-MSGID: 1783106109720650604 From: Andrey Konovalov The retval local variable in __stack_depot_save has the union type handle_parts, but the function never uses anything but the union's handle field. Define retval simply as depot_stack_handle_t to simplify the code. Reviewed-by: Alexander Potapenko Signed-off-by: Andrey Konovalov Reviewed-by: Oscar Salvador --- lib/stackdepot.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/lib/stackdepot.c b/lib/stackdepot.c index f8a8033e1dc8..3e71c8f61c7d 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -366,7 +366,7 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, gfp_t alloc_flags, bool can_alloc) { struct stack_record *found = NULL, **bucket; - union handle_parts retval = { .handle = 0 }; + depot_stack_handle_t handle = 0; struct page *page = NULL; void *prealloc = NULL; unsigned long flags; @@ -383,7 +383,7 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, nr_entries = filter_irq_stacks(entries, nr_entries); if (unlikely(nr_entries == 0) || stack_depot_disabled) - goto fast_exit; + return 0; hash = hash_stack(entries, nr_entries); bucket = &stack_table[hash & stack_hash_mask]; @@ -449,9 +449,8 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, free_pages((unsigned long)prealloc, DEPOT_POOL_ORDER); } if (found) - retval.handle = found->handle.handle; -fast_exit: - return retval.handle; + handle = found->handle.handle; + return handle; } EXPORT_SYMBOL_GPL(__stack_depot_save); From patchwork Mon Nov 20 17:47:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 167312 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2b07:b0:403:3b70:6f57 with SMTP id io7csp105873vqb; Mon, 20 Nov 2023 09:48:20 -0800 (PST) X-Google-Smtp-Source: AGHT+IGPoCWGOmR7gu81cuFNl416bUyN3OU/f7UGiD7vBvjNC/efWo++Tz825WApLq6IiD8z9lJ0 X-Received: by 2002:a17:90b:4aca:b0:27d:3ecb:3cbb with SMTP id mh10-20020a17090b4aca00b0027d3ecb3cbbmr6359310pjb.37.1700502499913; Mon, 20 Nov 2023 09:48:19 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700502499; cv=none; d=google.com; s=arc-20160816; b=eAnXcCfMutHOJLNWfnxN2BGw4yAQnd6zQ4APafpyGM5s/2911EHqO70QT/llLyALRU 6OzcKPZPAM8DBBGK3Je7dWI+cM34fUt2tCGExxc5nLLpsTwhmPjTNg8h/CgZnL3D3S4c D8fiTAYcXLu04p8rBohyxLdmb8hclcJQMGt9/1SpnetwETJ7WIQOJrb1LKTFmDG624R/ aDdusc3/WZ9uOhXK0XvWbAvTmlikPRqYZ7SiJOR5RKeEcBgpvvbvSvTmOqIAe+65WOj7 IsUeL9Pq3L1F6lmhcJe6mwbuFqKTz7yUhnUBoMooFEb+6CkhRotEAYovbC4xEgky4LkF MBag== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=lmg5zTuKt1w9oWZyNpXv7HY69NJCp78Dr+tP0crczmk=; fh=qWoxmPN1zRqnLSoCWGpfFsDtzJsx/SGdqx98lbP+Uho=; b=yaGeq2Qj1Csqj+6G7DjQZgobJEA63rWVcuPGS0Dtfx0pd65kNFvFJa3dysB6oa0xhv X/BF3nzejudsktD/oImCWrgF0eoqEtHRUQ+710Ge/PoaS3SySTmOPzgxIkRaaVr7y2xu UXkCWXzOLOAXpdKeiiUkDT6e0nC/RWPW1JTW0Ht3fAh0i30Eq51IwVTqj6IMRsM3uq1X hkqKH0vUyJu+hT9pBDmcqCB4yUHDWET7iwm6NHQMg3wlnGuoqg+8aXd1S+ujh2G0qOv/ tigAydYMyP8jQqdL+dxaYAeVlYEvd6DYRMZ4XYHW2NytNbUL+BRvxBlMQeAtYT67lZNH SYYA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=sig9IVvl; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id bk14-20020a17090b080e00b00283a251c077si6235706pjb.94.2023.11.20.09.48.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Nov 2023 09:48:19 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=sig9IVvl; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id A41FC8059E73; Mon, 20 Nov 2023 09:47:55 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234078AbjKTRrp (ORCPT + 27 others); Mon, 20 Nov 2023 12:47:45 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51280 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232803AbjKTRre (ORCPT ); Mon, 20 Nov 2023 12:47:34 -0500 Received: from out-187.mta1.migadu.com (out-187.mta1.migadu.com [95.215.58.187]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7D5D5BA for ; Mon, 20 Nov 2023 09:47:30 -0800 (PST) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1700502449; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lmg5zTuKt1w9oWZyNpXv7HY69NJCp78Dr+tP0crczmk=; b=sig9IVvlHqVaQ7yAjHcKyEBZSXKZVvVyW0FFm1WUvwt3/VhlR8ohBHYAlXrR7cabYtXtPi 1x7T5BIS3RF5oH+C0wW4PoHF0fJj3mAq0pXEuDuyXI6LIJomNt8wyDqJivUeLox2pQT6re fzBBmv3FWure1h0tN/rWvxM6gmS3lDg= From: andrey.konovalov@linux.dev To: Andrew Morton Cc: Andrey Konovalov , Marco Elver , Alexander Potapenko , Dmitry Vyukov , Vlastimil Babka , kasan-dev@googlegroups.com, Evgenii Stepanov , Oscar Salvador , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v4 04/22] lib/stackdepot: drop valid bit from handles Date: Mon, 20 Nov 2023 18:47:02 +0100 Message-Id: <34969bba2ca6e012c6ad071767197dee64dc5723.1700502145.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Mon, 20 Nov 2023 09:47:55 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783106109693357693 X-GMAIL-MSGID: 1783106109693357693 From: Andrey Konovalov Stack depot doesn't use the valid bit in handles in any way, so drop it. Reviewed-by: Alexander Potapenko Signed-off-by: Andrey Konovalov Reviewed-by: Oscar Salvador --- lib/stackdepot.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/lib/stackdepot.c b/lib/stackdepot.c index 3e71c8f61c7d..46a422d31c1f 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -32,13 +32,12 @@ #define DEPOT_HANDLE_BITS (sizeof(depot_stack_handle_t) * 8) -#define DEPOT_VALID_BITS 1 #define DEPOT_POOL_ORDER 2 /* Pool size order, 4 pages */ #define DEPOT_POOL_SIZE (1LL << (PAGE_SHIFT + DEPOT_POOL_ORDER)) #define DEPOT_STACK_ALIGN 4 #define DEPOT_OFFSET_BITS (DEPOT_POOL_ORDER + PAGE_SHIFT - DEPOT_STACK_ALIGN) -#define DEPOT_POOL_INDEX_BITS (DEPOT_HANDLE_BITS - DEPOT_VALID_BITS - \ - DEPOT_OFFSET_BITS - STACK_DEPOT_EXTRA_BITS) +#define DEPOT_POOL_INDEX_BITS (DEPOT_HANDLE_BITS - DEPOT_OFFSET_BITS - \ + STACK_DEPOT_EXTRA_BITS) #define DEPOT_POOLS_CAP 8192 #define DEPOT_MAX_POOLS \ (((1LL << (DEPOT_POOL_INDEX_BITS)) < DEPOT_POOLS_CAP) ? \ @@ -50,7 +49,6 @@ union handle_parts { struct { u32 pool_index : DEPOT_POOL_INDEX_BITS; u32 offset : DEPOT_OFFSET_BITS; - u32 valid : DEPOT_VALID_BITS; u32 extra : STACK_DEPOT_EXTRA_BITS; }; }; @@ -309,7 +307,6 @@ depot_alloc_stack(unsigned long *entries, int size, u32 hash, void **prealloc) stack->size = size; stack->handle.pool_index = pool_index; stack->handle.offset = pool_offset >> DEPOT_STACK_ALIGN; - stack->handle.valid = 1; stack->handle.extra = 0; memcpy(stack->entries, entries, flex_array_size(stack, entries, size)); pool_offset += required_size; From patchwork Mon Nov 20 17:47:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 167315 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2b07:b0:403:3b70:6f57 with SMTP id io7csp105915vqb; Mon, 20 Nov 2023 09:48:23 -0800 (PST) X-Google-Smtp-Source: AGHT+IFAqy6dAtR3rtMPvILDxhH9KVbJD4hqi8O4AvdNLvKiRmQCXdeYveo+xRboYvwLw/TKJNtQ X-Received: by 2002:a17:90b:4b04:b0:281:37b0:4b0d with SMTP id lx4-20020a17090b4b0400b0028137b04b0dmr5637146pjb.7.1700502503544; Mon, 20 Nov 2023 09:48:23 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700502503; cv=none; d=google.com; s=arc-20160816; b=FpKpZebPCBsG/5Csmx2AFk+d1k5LRZfDmrjusR4hlyNWgWZvW6kpwNP0OoHZBhCAeo DFIdER3XySBQ1Y7iA9UttxFCWGR4Fk4kRlwq3FLGb1s1tf6p8uUyvmaUJN1mTxFQJVsT Od7BlylNcVF+inlYPZQ3knPksvi66kku4zIdYGQk6ySiZmFVWkXtMSdLbUIyFRCEaoVn 13hi8M/QB0lTCUlnPwjbdOj+OOB/rctcGfqFO//DzZvLK6Zk7/FhlNluDTKimBLFIgwB cNA7Feg5C//OPNnEXDZe02UrF3A6p38Z0NHw0cThU8sahjACzqOccsuTh6XJS5Tf7Kmh C6RQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=4+Y9MGkz9f3yZSUENZCgBAvLs1CfO+kuipQPdiyYS+c=; fh=qWoxmPN1zRqnLSoCWGpfFsDtzJsx/SGdqx98lbP+Uho=; b=0NarAsASvTFgpgKFA9inVtxLJoCT7O8RLFo5BuA7Nniv33kmb6/09fC0fsuwuJKlwG AsFRi2f+0aZPCs5omgbO0W8iXKbcTftjYDqwsDUqfScWBwz+4fnBJ0IUyciiVHbqFEVO fGFxjd6ZsQQ42DrBHzo8i2MxXsr6+iNemSELe/4j8hUC8HXNg+VZ6ZRVJZ69Lk0El7Jt VciaxIo78Tg4zeqSzrTiofd9Qc5Gz5piHAEZmtzUBo2Uirx/A+UUI5cBequ8xlQ5Qj0r 3mrr3XHN7RFz5omEbXPi8NvoN+hGMSbTKoMNptCCdO+4jn7rW+ER/fNN7OGS+N5yECld osdg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=Ql85St5S; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id x7-20020a17090a530700b00277517b42dasi8327427pjh.35.2023.11.20.09.48.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Nov 2023 09:48:23 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=Ql85St5S; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id CF18B80EE0CF; Mon, 20 Nov 2023 09:48:03 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234157AbjKTRrs (ORCPT + 27 others); Mon, 20 Nov 2023 12:47:48 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51284 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232999AbjKTRrf (ORCPT ); Mon, 20 Nov 2023 12:47:35 -0500 Received: from out-170.mta1.migadu.com (out-170.mta1.migadu.com [IPv6:2001:41d0:203:375::aa]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7F70A9E for ; Mon, 20 Nov 2023 09:47:31 -0800 (PST) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1700502449; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4+Y9MGkz9f3yZSUENZCgBAvLs1CfO+kuipQPdiyYS+c=; b=Ql85St5SjYl9dK6UKQo/wZBfC05xwZ8kz2f2txy485DwN0EC3gx7bgEAGlRsR3amTQZM6i Nep6MT9W4kXFWBOfaYnIaeXGDBapToM5EgH6SXSUwG03GDKgl7MtqpodPgiSEzEwBXoD0L 1m7pwV0+V5c6VKCswrtvLTavWAO/Q/M= From: andrey.konovalov@linux.dev To: Andrew Morton Cc: Andrey Konovalov , Marco Elver , Alexander Potapenko , Dmitry Vyukov , Vlastimil Babka , kasan-dev@googlegroups.com, Evgenii Stepanov , Oscar Salvador , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v4 05/22] lib/stackdepot: add depot_fetch_stack helper Date: Mon, 20 Nov 2023 18:47:03 +0100 Message-Id: <170d8c202f29dc8e3d5491ee074d1e9e029a46db.1700502145.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Mon, 20 Nov 2023 09:48:03 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783106113392383305 X-GMAIL-MSGID: 1783106113392383305 From: Andrey Konovalov Add a helper depot_fetch_stack function that fetches the pointer to a stack record. With this change, all static depot_* functions now operate on stack pools and the exported stack_depot_* functions operate on the hash table. Reviewed-by: Alexander Potapenko Signed-off-by: Andrey Konovalov Reviewed-by: Oscar Salvador --- Changes v1->v2: - Minor comment fix as suggested by Alexander. --- lib/stackdepot.c | 45 ++++++++++++++++++++++++++++----------------- 1 file changed, 28 insertions(+), 17 deletions(-) diff --git a/lib/stackdepot.c b/lib/stackdepot.c index 46a422d31c1f..e41713983cac 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -310,6 +310,7 @@ depot_alloc_stack(unsigned long *entries, int size, u32 hash, void **prealloc) stack->handle.extra = 0; memcpy(stack->entries, entries, flex_array_size(stack, entries, size)); pool_offset += required_size; + /* * Let KMSAN know the stored stack record is initialized. This shall * prevent false positive reports if instrumented code accesses it. @@ -319,6 +320,32 @@ depot_alloc_stack(unsigned long *entries, int size, u32 hash, void **prealloc) return stack; } +static struct stack_record *depot_fetch_stack(depot_stack_handle_t handle) +{ + union handle_parts parts = { .handle = handle }; + /* + * READ_ONCE pairs with potential concurrent write in + * depot_alloc_stack(). + */ + int pool_index_cached = READ_ONCE(pool_index); + void *pool; + size_t offset = parts.offset << DEPOT_STACK_ALIGN; + struct stack_record *stack; + + if (parts.pool_index > pool_index_cached) { + WARN(1, "pool index %d out of bounds (%d) for stack id %08x\n", + parts.pool_index, pool_index_cached, handle); + return NULL; + } + + pool = stack_pools[parts.pool_index]; + if (!pool) + return NULL; + + stack = pool + offset; + return stack; +} + /* Calculates the hash for a stack. */ static inline u32 hash_stack(unsigned long *entries, unsigned int size) { @@ -462,14 +489,6 @@ EXPORT_SYMBOL_GPL(stack_depot_save); unsigned int stack_depot_fetch(depot_stack_handle_t handle, unsigned long **entries) { - union handle_parts parts = { .handle = handle }; - /* - * READ_ONCE pairs with potential concurrent write in - * depot_alloc_stack. - */ - int pool_index_cached = READ_ONCE(pool_index); - void *pool; - size_t offset = parts.offset << DEPOT_STACK_ALIGN; struct stack_record *stack; *entries = NULL; @@ -482,15 +501,7 @@ unsigned int stack_depot_fetch(depot_stack_handle_t handle, if (!handle || stack_depot_disabled) return 0; - if (parts.pool_index > pool_index_cached) { - WARN(1, "pool index %d out of bounds (%d) for stack id %08x\n", - parts.pool_index, pool_index_cached, handle); - return 0; - } - pool = stack_pools[parts.pool_index]; - if (!pool) - return 0; - stack = pool + offset; + stack = depot_fetch_stack(handle); *entries = stack->entries; return stack->size; From patchwork Mon Nov 20 17:47:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 167326 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2b07:b0:403:3b70:6f57 with SMTP id io7csp106999vqb; Mon, 20 Nov 2023 09:50:22 -0800 (PST) X-Google-Smtp-Source: AGHT+IHRaLv+nyVzwgDi1wBDp8CUB/v87Dj3Lf5LPoDudh5hJOQpVMUtA0dcpA9jJ8S+oyy3zykx X-Received: by 2002:a17:902:da87:b0:1cf:5711:b8e6 with SMTP id j7-20020a170902da8700b001cf5711b8e6mr4852152plx.17.1700502621919; Mon, 20 Nov 2023 09:50:21 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700502621; cv=none; d=google.com; s=arc-20160816; b=g18e5XMUXh3IuTvSYVC7Y1HHOrFWonuEKltqqIP/+nxF7sE0X7BHrnWwDFTlmkU9dD 98py2Ebz0IH/uO/uQjZZl1MvKvt+4Wdh7NhgaxY6WMEHiTX5cMBXoNehsRbstV5TXmnr HRRMVviahI+0oCPwzPwZCkTe0BxfncCn6L/t0qJqQKWMCVOMRSYnL7nW9lQrjMOsqhrG Og+mfbbjPH6H3WiSXu3L2YQCrM20NlVUShR3FE6qgBamOWpjjNlJsA2oFMNUsNnp93iW 8LUKsRRbu3zGrhHFcwlaashUppFM9lNO4GDXeOek5cjTvdXg+Acg/HOczzURqcmn0D2m nXEg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=pu1XFZGC+3mxYm/tNoBY6t9qKRkAeXfQVUMTHQNQJK8=; fh=qWoxmPN1zRqnLSoCWGpfFsDtzJsx/SGdqx98lbP+Uho=; b=y8Nidw7GavgzBAxxvAnZLcSSZME8mP/ZQmd5RsR13dKVBSA7vaW9+MsVjEL5ORR9Zi L3QHvoh/CEpbu99PkiQu6BJ1osANSuiRC6ZAFrKdfFtN330VUX67TUmo6br5MBfor8Bj GKrlDbLBS47meb+/H1lAnacQD5aCNEDSZUO1D2kLvSjcHLP9MprcRt0oOUpwW3AJRBNY O057lamd1Of3IWsGLA61oNSPd2NXFa60pIQAVZJIESE/FfKPwOxqXYVmt0VHAt+gUDS+ KdYVRr/N5Uty2p+XC6Q8vpxjLi+j4DKUewolw9p5M/Z8D2ScljrD6r0LtInprZH6yYmq +mVA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=v29OoaCr; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from agentk.vger.email (agentk.vger.email. [2620:137:e000::3:2]) by mx.google.com with ESMTPS id n12-20020a170902f60c00b001cc55bca838si9122314plg.576.2023.11.20.09.50.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Nov 2023 09:50:21 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) client-ip=2620:137:e000::3:2; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=v29OoaCr; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id 60D8B803D8CA; Mon, 20 Nov 2023 09:49:47 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233871AbjKTRt2 (ORCPT + 27 others); Mon, 20 Nov 2023 12:49:28 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49276 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234106AbjKTRtR (ORCPT ); Mon, 20 Nov 2023 12:49:17 -0500 Received: from out-173.mta0.migadu.com (out-173.mta0.migadu.com [IPv6:2001:41d0:1004:224b::ad]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B743CD2 for ; Mon, 20 Nov 2023 09:48:34 -0800 (PST) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1700502512; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pu1XFZGC+3mxYm/tNoBY6t9qKRkAeXfQVUMTHQNQJK8=; b=v29OoaCrQAFVYXx8qE1accX78dCRXzOu4rgaHGTPoImCDsSRXsXJW7zty4lVMppXjkqcg3 cICBXUXycfKNBxolMNR9vSSUR3Qdi4ReyKaR9Gp00MSY7TEAkMubulEyKPVq2YOWSy/vMK Csf5CwyODxYj8dOCpaCpXMz38YUJonc= From: andrey.konovalov@linux.dev To: Andrew Morton Cc: Andrey Konovalov , Marco Elver , Alexander Potapenko , Dmitry Vyukov , Vlastimil Babka , kasan-dev@googlegroups.com, Evgenii Stepanov , Oscar Salvador , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v4 06/22] lib/stackdepot: use fixed-sized slots for stack records Date: Mon, 20 Nov 2023 18:47:04 +0100 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Mon, 20 Nov 2023 09:49:47 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783106237448721817 X-GMAIL-MSGID: 1783106237448721817 From: Andrey Konovalov Instead of storing stack records in stack depot pools one right after another, use fixed-sized slots. Add a new Kconfig option STACKDEPOT_MAX_FRAMES that allows to select the size of the slot in frames. Use 64 as the default value, which is the maximum stack trace size both KASAN and KMSAN use right now. Also add descriptions for other stack depot Kconfig options. This is preparatory patch for implementing the eviction of stack records from the stack depot. Signed-off-by: Andrey Konovalov Reviewed-by: Oscar Salvador --- Changes v2->v3: - Keep previously existing Kconfig options not configurable by users. Changes v1->v2: - Add and use STACKDEPOT_MAX_FRAMES Kconfig option. --- lib/Kconfig | 10 ++++++++++ lib/stackdepot.c | 13 +++++++++---- 2 files changed, 19 insertions(+), 4 deletions(-) diff --git a/lib/Kconfig b/lib/Kconfig index 3ea1c830efab..5ddda7c2ed9b 100644 --- a/lib/Kconfig +++ b/lib/Kconfig @@ -713,10 +713,20 @@ config ARCH_STACKWALK config STACKDEPOT bool select STACKTRACE + help + Stack depot: stack trace storage that avoids duplication config STACKDEPOT_ALWAYS_INIT bool select STACKDEPOT + help + Always initialize stack depot during early boot + +config STACKDEPOT_MAX_FRAMES + int "Maximum number of frames in trace saved in stack depot" + range 1 256 + default 64 + depends on STACKDEPOT config REF_TRACKER bool diff --git a/lib/stackdepot.c b/lib/stackdepot.c index e41713983cac..682497dbe081 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -58,9 +58,12 @@ struct stack_record { u32 hash; /* Hash in the hash table */ u32 size; /* Number of stored frames */ union handle_parts handle; - unsigned long entries[]; /* Variable-sized array of frames */ + unsigned long entries[CONFIG_STACKDEPOT_MAX_FRAMES]; /* Frames */ }; +#define DEPOT_STACK_RECORD_SIZE \ + ALIGN(sizeof(struct stack_record), 1 << DEPOT_STACK_ALIGN) + static bool stack_depot_disabled; static bool __stack_depot_early_init_requested __initdata = IS_ENABLED(CONFIG_STACKDEPOT_ALWAYS_INIT); static bool __stack_depot_early_init_passed __initdata; @@ -264,9 +267,7 @@ static struct stack_record * depot_alloc_stack(unsigned long *entries, int size, u32 hash, void **prealloc) { struct stack_record *stack; - size_t required_size = struct_size(stack, entries, size); - - required_size = ALIGN(required_size, 1 << DEPOT_STACK_ALIGN); + size_t required_size = DEPOT_STACK_RECORD_SIZE; /* Check if there is not enough space in the current pool. */ if (unlikely(pool_offset + required_size > DEPOT_POOL_SIZE)) { @@ -301,6 +302,10 @@ depot_alloc_stack(unsigned long *entries, int size, u32 hash, void **prealloc) if (stack_pools[pool_index] == NULL) return NULL; + /* Limit number of saved frames to CONFIG_STACKDEPOT_MAX_FRAMES. */ + if (size > CONFIG_STACKDEPOT_MAX_FRAMES) + size = CONFIG_STACKDEPOT_MAX_FRAMES; + /* Save the stack trace. */ stack = stack_pools[pool_index] + pool_offset; stack->hash = hash; From patchwork Mon Nov 20 17:47:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 167323 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2b07:b0:403:3b70:6f57 with SMTP id io7csp106924vqb; Mon, 20 Nov 2023 09:50:14 -0800 (PST) X-Google-Smtp-Source: AGHT+IG19pL5k9VhffC0bssiKMgq8Vuqirc/hTt7kuRdclh8KgJNKaGttK/0B9hjmr1tGwEJZZ3W X-Received: by 2002:a05:6a21:1a0:b0:174:373b:4381 with SMTP id le32-20020a056a2101a000b00174373b4381mr7408470pzb.49.1700502614545; Mon, 20 Nov 2023 09:50:14 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700502614; cv=none; d=google.com; s=arc-20160816; b=xFzbRiLwU9e/V5igsAO2iT1sNM+IOENkFvinPT/Yuu0InUdFH/24cNFzWOwm2m5dcu SEq228z+EAWVnN5toKJJkR6OWm7sg3GVZRh2BzKcD/qquh97pyCUWGie6ZfuHHYxev1Z tfN1ol4s4AZ1477TKg4E/mxnvdbC+9zITT3YF+YOa6PuJoAEIyF/JerVTZD7ZoLun3bE d6MyziR4f0yX7NwcV2ZW2aXc7qDgNmnWFseMWU2woE848nvwtXSgUw3LvF0gfRYv5mPs I5GnxmGNJW/YmGgbuF0dZ32cY8PYwuOr7emMaJxxBWGI1xvNzz9VALg0RTMCVMdsO/a4 PlrA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=ul/nKw7kj0oOeSlF9NlnEwH7YAqJQYzQqOqaJ7LWVgY=; fh=qWoxmPN1zRqnLSoCWGpfFsDtzJsx/SGdqx98lbP+Uho=; b=Z8bLPaVRkv9f4F7fBwpzELrA4z1jH4BAigBKkXqWLC0XVrz0ElhfAsHFlGQB6dWX8j /p/iBqpLmMJEOuyv6KqyRnrmJW5ox2yNQaELiz5d2ABFk6OiUOC1ftGCk3d7yZC6Oi0g t42zCBYSbpoOWtLyPr49NZlXp7+E5KDO8+4Emp9+RCwpjOv+p1Z/A+SHxlAHev9lrcKw qQuFJ/2ohwWCqZmD4CGWH2x7EFmR7467r1jJw0akkS/WdvWtn5Bwz0kGdwl4E32K7SRs mOnD0xh7ZjPftxutheU0Ns+Ywd+DqByaz6LLJdVoJhijxwFsx13o/CQ/E6Pu4YGXwnAr pltw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=ukcrau9r; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from lipwig.vger.email (lipwig.vger.email. [2620:137:e000::3:3]) by mx.google.com with ESMTPS id b2-20020a056a000cc200b0068a85532963si8782972pfv.160.2023.11.20.09.49.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Nov 2023 09:50:14 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) client-ip=2620:137:e000::3:3; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=ukcrau9r; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id 500CC8032C7D; Mon, 20 Nov 2023 09:49:46 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234110AbjKTRtb (ORCPT + 27 others); Mon, 20 Nov 2023 12:49:31 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49802 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234159AbjKTRtR (ORCPT ); Mon, 20 Nov 2023 12:49:17 -0500 Received: from out-176.mta0.migadu.com (out-176.mta0.migadu.com [IPv6:2001:41d0:1004:224b::b0]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E83C9D5B for ; Mon, 20 Nov 2023 09:48:34 -0800 (PST) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1700502513; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ul/nKw7kj0oOeSlF9NlnEwH7YAqJQYzQqOqaJ7LWVgY=; b=ukcrau9rBSSL43tgTPUpFjw/XmDDuuV7b5Rf6ugJdDIx3eaVAx8V/9PFiPuIgxA8HOVqDI FOv6Ks7qLmHGEak24eJMUo0HQta3weJK4BNrKPY6LbsCGCWMiUh2CgVaukTzBdLG8nj0JF 0fqdYiwEAxqEdM4YGbvK075MNLH3F5M= From: andrey.konovalov@linux.dev To: Andrew Morton Cc: Andrey Konovalov , Marco Elver , Alexander Potapenko , Dmitry Vyukov , Vlastimil Babka , kasan-dev@googlegroups.com, Evgenii Stepanov , Oscar Salvador , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v4 07/22] lib/stackdepot: fix and clean-up atomic annotations Date: Mon, 20 Nov 2023 18:47:05 +0100 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Mon, 20 Nov 2023 09:49:46 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783106229718050522 X-GMAIL-MSGID: 1783106229718050522 From: Andrey Konovalov Drop smp_load_acquire from next_pool_required in depot_init_pool, as both depot_init_pool and the all smp_store_release's to this variable are executed under the stack depot lock. Also simplify and clean up comments accompanying the use of atomic accesses in the stack depot code. Reviewed-by: Alexander Potapenko Signed-off-by: Andrey Konovalov --- This patch is not strictly required, as the atomic accesses are fully removed in one of the latter patches. However, I decided to keep the patch just in case we end up needing these atomics in the following iterations of this series. Changes v2->v3: - Keep parentheses when referring to functions in comments. - Add comment that explains why depot_init_pool reads next_pool_required non-atomically. Changes v1->v2: - Minor comment fix as suggested by Marco. - Drop READ_ONCE marking for next_pool_required. --- lib/stackdepot.c | 29 ++++++++++++++--------------- 1 file changed, 14 insertions(+), 15 deletions(-) diff --git a/lib/stackdepot.c b/lib/stackdepot.c index 682497dbe081..cfa3c6c7cc2e 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -231,10 +231,10 @@ static void depot_init_pool(void **prealloc) /* * If the next pool is already initialized or the maximum number of * pools is reached, do not use the preallocated memory. - * smp_load_acquire() here pairs with smp_store_release() below and - * in depot_alloc_stack(). + * Access next_pool_required non-atomically, as there are no concurrent + * write accesses to this variable. */ - if (!smp_load_acquire(&next_pool_required)) + if (!next_pool_required) return; /* Check if the current pool is not yet allocated. */ @@ -255,8 +255,8 @@ static void depot_init_pool(void **prealloc) * At this point, either the next pool is initialized or the * maximum number of pools is reached. In either case, take * note that initializing another pool is not required. - * This smp_store_release pairs with smp_load_acquire() above - * and in stack_depot_save(). + * smp_store_release() pairs with smp_load_acquire() in + * stack_depot_save(). */ smp_store_release(&next_pool_required, 0); } @@ -279,7 +279,7 @@ depot_alloc_stack(unsigned long *entries, int size, u32 hash, void **prealloc) /* * Move on to the next pool. - * WRITE_ONCE pairs with potential concurrent read in + * WRITE_ONCE() pairs with potential concurrent read in * stack_depot_fetch(). */ WRITE_ONCE(pool_index, pool_index + 1); @@ -287,8 +287,8 @@ depot_alloc_stack(unsigned long *entries, int size, u32 hash, void **prealloc) /* * If the maximum number of pools is not reached, take note * that the next pool needs to initialized. - * smp_store_release() here pairs with smp_load_acquire() in - * stack_depot_save() and depot_init_pool(). + * smp_store_release() pairs with smp_load_acquire() in + * stack_depot_save(). */ if (pool_index + 1 < DEPOT_MAX_POOLS) smp_store_release(&next_pool_required, 1); @@ -329,7 +329,7 @@ static struct stack_record *depot_fetch_stack(depot_stack_handle_t handle) { union handle_parts parts = { .handle = handle }; /* - * READ_ONCE pairs with potential concurrent write in + * READ_ONCE() pairs with potential concurrent write in * depot_alloc_stack(). */ int pool_index_cached = READ_ONCE(pool_index); @@ -419,8 +419,7 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, /* * Fast path: look the stack trace up without locking. - * The smp_load_acquire() here pairs with smp_store_release() to - * |bucket| below. + * smp_load_acquire() pairs with smp_store_release() to |bucket| below. */ found = find_stack(smp_load_acquire(bucket), entries, nr_entries, hash); if (found) @@ -430,8 +429,8 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, * Check if another stack pool needs to be initialized. If so, allocate * the memory now - we won't be able to do that under the lock. * - * The smp_load_acquire() here pairs with smp_store_release() to - * |next_pool_inited| in depot_alloc_stack() and depot_init_pool(). + * smp_load_acquire() pairs with smp_store_release() in + * depot_alloc_stack() and depot_init_pool(). */ if (unlikely(can_alloc && smp_load_acquire(&next_pool_required))) { /* @@ -457,8 +456,8 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, if (new) { new->next = *bucket; /* - * This smp_store_release() pairs with - * smp_load_acquire() from |bucket| above. + * smp_store_release() pairs with smp_load_acquire() + * from |bucket| above. */ smp_store_release(bucket, new); found = new; From patchwork Mon Nov 20 17:47:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 167318 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2b07:b0:403:3b70:6f57 with SMTP id io7csp106844vqb; Mon, 20 Nov 2023 09:50:07 -0800 (PST) X-Google-Smtp-Source: AGHT+IHC3kDFDQc13zlX5iV9rrVbQajHTFm2HKR5wpLNvffkyoS9lA1X7C9UrNOBEVUGeBN3AsVU X-Received: by 2002:a05:6808:f94:b0:3b2:e3d0:863f with SMTP id o20-20020a0568080f9400b003b2e3d0863fmr8095865oiw.13.1700502606729; Mon, 20 Nov 2023 09:50:06 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700502606; cv=none; d=google.com; s=arc-20160816; b=uHAEXD6NvET98w4Rn8d8LKhnPrUAfWTM5xSv9RrvEZu+/EM9CDQMJqchbyYRhh3oD4 LD8gDudEzuX8ycsYq+ABXjFxI766V1oh96d5UtO2tnPGiDjHkLPowy1YT2p0CaH9u+5M lz9lY4CZTEhKYjogUm1YtnuNupyy8zlebxJl5mSaYzw2IM9iEg711qrGxi7IkMDgyN9Z FeCGZRlGHMPiI89+cOgeJ8oxxocHPe5f4BGOgZW8U5y3wz6qtht/L8DIJYItZThb5yoc BgATbSK38sUh4NeoqQNC+ohGHnmWOVhZ/l5AoAdrtrxU39DXberFguy+84iW9XVyfO4C HaCQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=9IYHxZGEq/qB2Qlddv235LQK4ccHh5XyxORYKM2z5Wc=; fh=qWoxmPN1zRqnLSoCWGpfFsDtzJsx/SGdqx98lbP+Uho=; b=lcWO89fRy3HWpA5qxEnduEOQNr9U43/5zWHmLXLEk+BQw3IfKfqiYPWKJANsrXEh5z onjJ8zURuL7a3CzMiToGPLHXLYeOPFr+YwObOyISdKjo9CRGd9YbP3Rh9XqYYSr4amhD zE/B89RbOOAYKN7tFLESCcXxeEiF2rbMMqfDpqQ+coLZk0LFmEEU1WVrA0Swmjw5QhAB 16Sc7uzRp/3jjFDM25RaD6wuIOGhBLGgdb9rk8bQiwboQQ2XgbvWzrHVY73M+6sUdoBp p0lEsba37DAQ85P0li+GGI3ae/6lplzAxCID2dwUFykCGcqO0NEhhhL9MV3GBVIp7xjG U9jw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=Kt5p6zSc; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id g17-20020a63fa51000000b005b902b61cbbsi8446578pgk.125.2023.11.20.09.49.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Nov 2023 09:50:06 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=Kt5p6zSc; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 7DE1180EE0DE; Mon, 20 Nov 2023 09:49:41 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234255AbjKTRtf (ORCPT + 27 others); Mon, 20 Nov 2023 12:49:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53418 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234496AbjKTRtR (ORCPT ); Mon, 20 Nov 2023 12:49:17 -0500 Received: from out-181.mta0.migadu.com (out-181.mta0.migadu.com [IPv6:2001:41d0:1004:224b::b5]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 757AA12A for ; Mon, 20 Nov 2023 09:48:35 -0800 (PST) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1700502513; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9IYHxZGEq/qB2Qlddv235LQK4ccHh5XyxORYKM2z5Wc=; b=Kt5p6zScvUKaXsHar2QlUefWEFtJ6Uws7kbkfGgrB5cxpVMGSn2WXhKYFJ5MhB50RhntDQ K7N0E2IbssHoPp6AeWgnF1sPLT3aDEF/nGgRuJ546AoCXVl6clQWGAmNHZLEBxlZNtF5g9 xiIqlvFkZnYs+J6Hza0Jtp/FsRG4z5s= From: andrey.konovalov@linux.dev To: Andrew Morton Cc: Andrey Konovalov , Marco Elver , Alexander Potapenko , Dmitry Vyukov , Vlastimil Babka , kasan-dev@googlegroups.com, Evgenii Stepanov , Oscar Salvador , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v4 08/22] lib/stackdepot: rework helpers for depot_alloc_stack Date: Mon, 20 Nov 2023 18:47:06 +0100 Message-Id: <71fb144d42b701fcb46708d7f4be6801a4a8270e.1700502145.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Mon, 20 Nov 2023 09:49:41 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783106221320280108 X-GMAIL-MSGID: 1783106221320280108 From: Andrey Konovalov Split code in depot_alloc_stack and depot_init_pool into 3 functions: 1. depot_keep_next_pool that keeps preallocated memory for the next pool if required. 2. depot_update_pools that moves on to the next pool if there's no space left in the current pool, uses preallocated memory for the new current pool if required, and calls depot_keep_next_pool otherwise. 3. depot_alloc_stack that calls depot_update_pools and then allocates a stack record as before. This makes it somewhat easier to follow the logic of depot_alloc_stack and also serves as a preparation for implementing the eviction of stack records from the stack depot. Reviewed-by: Alexander Potapenko Signed-off-by: Andrey Konovalov Reviewed-by: Oscar Salvador --- Changes v2->v3: - Add parentheses when referring to function calls in comments. --- lib/stackdepot.c | 86 +++++++++++++++++++++++++++--------------------- 1 file changed, 49 insertions(+), 37 deletions(-) diff --git a/lib/stackdepot.c b/lib/stackdepot.c index cfa3c6c7cc2e..b3af868627f4 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -225,11 +225,11 @@ int stack_depot_init(void) } EXPORT_SYMBOL_GPL(stack_depot_init); -/* Uses preallocated memory to initialize a new stack depot pool. */ -static void depot_init_pool(void **prealloc) +/* Keeps the preallocated memory to be used for the next stack depot pool. */ +static void depot_keep_next_pool(void **prealloc) { /* - * If the next pool is already initialized or the maximum number of + * If the next pool is already saved or the maximum number of * pools is reached, do not use the preallocated memory. * Access next_pool_required non-atomically, as there are no concurrent * write accesses to this variable. @@ -237,44 +237,34 @@ static void depot_init_pool(void **prealloc) if (!next_pool_required) return; - /* Check if the current pool is not yet allocated. */ - if (stack_pools[pool_index] == NULL) { - /* Use the preallocated memory for the current pool. */ - stack_pools[pool_index] = *prealloc; + /* + * Use the preallocated memory for the next pool + * as long as we do not exceed the maximum number of pools. + */ + if (pool_index + 1 < DEPOT_MAX_POOLS) { + stack_pools[pool_index + 1] = *prealloc; *prealloc = NULL; - } else { - /* - * Otherwise, use the preallocated memory for the next pool - * as long as we do not exceed the maximum number of pools. - */ - if (pool_index + 1 < DEPOT_MAX_POOLS) { - stack_pools[pool_index + 1] = *prealloc; - *prealloc = NULL; - } - /* - * At this point, either the next pool is initialized or the - * maximum number of pools is reached. In either case, take - * note that initializing another pool is not required. - * smp_store_release() pairs with smp_load_acquire() in - * stack_depot_save(). - */ - smp_store_release(&next_pool_required, 0); } + + /* + * At this point, either the next pool is kept or the maximum + * number of pools is reached. In either case, take note that + * keeping another pool is not required. + * smp_store_release() pairs with smp_load_acquire() in + * stack_depot_save(). + */ + smp_store_release(&next_pool_required, 0); } -/* Allocates a new stack in a stack depot pool. */ -static struct stack_record * -depot_alloc_stack(unsigned long *entries, int size, u32 hash, void **prealloc) +/* Updates references to the current and the next stack depot pools. */ +static bool depot_update_pools(size_t required_size, void **prealloc) { - struct stack_record *stack; - size_t required_size = DEPOT_STACK_RECORD_SIZE; - /* Check if there is not enough space in the current pool. */ if (unlikely(pool_offset + required_size > DEPOT_POOL_SIZE)) { /* Bail out if we reached the pool limit. */ if (unlikely(pool_index + 1 >= DEPOT_MAX_POOLS)) { WARN_ONCE(1, "Stack depot reached limit capacity"); - return NULL; + return false; } /* @@ -284,9 +274,10 @@ depot_alloc_stack(unsigned long *entries, int size, u32 hash, void **prealloc) */ WRITE_ONCE(pool_index, pool_index + 1); pool_offset = 0; + /* * If the maximum number of pools is not reached, take note - * that the next pool needs to initialized. + * that the next pool needs to be initialized. * smp_store_release() pairs with smp_load_acquire() in * stack_depot_save(). */ @@ -294,9 +285,30 @@ depot_alloc_stack(unsigned long *entries, int size, u32 hash, void **prealloc) smp_store_release(&next_pool_required, 1); } - /* Assign the preallocated memory to a pool if required. */ + /* Check if the current pool is not yet allocated. */ + if (*prealloc && stack_pools[pool_index] == NULL) { + /* Use the preallocated memory for the current pool. */ + stack_pools[pool_index] = *prealloc; + *prealloc = NULL; + return true; + } + + /* Otherwise, try using the preallocated memory for the next pool. */ if (*prealloc) - depot_init_pool(prealloc); + depot_keep_next_pool(prealloc); + return true; +} + +/* Allocates a new stack in a stack depot pool. */ +static struct stack_record * +depot_alloc_stack(unsigned long *entries, int size, u32 hash, void **prealloc) +{ + struct stack_record *stack; + size_t required_size = DEPOT_STACK_RECORD_SIZE; + + /* Update current and next pools if required and possible. */ + if (!depot_update_pools(required_size, prealloc)) + return NULL; /* Check if we have a pool to save the stack trace. */ if (stack_pools[pool_index] == NULL) @@ -330,7 +342,7 @@ static struct stack_record *depot_fetch_stack(depot_stack_handle_t handle) union handle_parts parts = { .handle = handle }; /* * READ_ONCE() pairs with potential concurrent write in - * depot_alloc_stack(). + * depot_update_pools(). */ int pool_index_cached = READ_ONCE(pool_index); void *pool; @@ -430,7 +442,7 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, * the memory now - we won't be able to do that under the lock. * * smp_load_acquire() pairs with smp_store_release() in - * depot_alloc_stack() and depot_init_pool(). + * depot_update_pools() and depot_keep_next_pool(). */ if (unlikely(can_alloc && smp_load_acquire(&next_pool_required))) { /* @@ -467,7 +479,7 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, * Stack depot already contains this stack trace, but let's * keep the preallocated memory for the future. */ - depot_init_pool(&prealloc); + depot_keep_next_pool(&prealloc); } raw_spin_unlock_irqrestore(&pool_lock, flags); From patchwork Mon Nov 20 17:47:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 167319 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2b07:b0:403:3b70:6f57 with SMTP id io7csp106868vqb; Mon, 20 Nov 2023 09:50:10 -0800 (PST) X-Google-Smtp-Source: AGHT+IHLr+6xEmu+G/WdVG+kGYdQmXMn+QL4m3gUHzzW6bTHksS8pzE704/GVVInOBNkGGZWT2kQ X-Received: by 2002:a17:902:e5ca:b0:1ce:6600:39e6 with SMTP id u10-20020a170902e5ca00b001ce660039e6mr10955697plf.0.1700502610359; Mon, 20 Nov 2023 09:50:10 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700502610; cv=none; d=google.com; s=arc-20160816; b=jF+clBKne7070pbz4u193A3bvB8jVgM1UylrsAfJZoiz93GjpErspdCC5zc25zKahv R76wvzbyzLHdtdmPurbaDh18AecSjeUAjQaY0PMud0YlmMBCE0ATHXsP8Rm4PwF3yeoO PUTO/vs+Isxikxyjy8isw/KUX4J3Gws0jfI7pPhROzhov8gyaZLqeREpS5UyHfI7L1XD k3ZxXxzYYbXwKJVHbx3IJ5YceSy3UDSm/TA3dZXeRbblgy05Qg7Fz+L5I9fHoGdHLMYL Ay8rQeEk8DxvYruJCpjGz8VCExK8OkKcO+U8AQ9i/AbtgRK3PNm1mN5TwpTpYgc7D/VF nzbw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=hhwH9Rj+7SlwckI0c6mMcIkIslfacxpde7X6neQ7UWA=; fh=qWoxmPN1zRqnLSoCWGpfFsDtzJsx/SGdqx98lbP+Uho=; b=fQQWelzfkuOdMSAySBbJ11CECuV4FGVjJZiPMU96xSL4086zdtNuuzz4cG4OYVJYmi g2M0msaXM6pgOMMUmnj4JcGCfAYfUXQmwxG78tsxNTnz1wlwO6AaRW93dyvFAbtugmxJ GFXZl/wWi+I71yXrelbzKQFQSlNVZCzj2sdB13ZTx8VElS7IMx8V8No7/PGOBj7xA6GH eMQ/7D2WGbEurV/7MEZonCP/NCh7Q1b0U/5rM+Bcxp3ycMevpQbgfp7gD9F6htmXyIwj 5glgFtas23cT7B3HV3xs+AN2aq63tGHiL3M3p7bPnN0KWnHWoN2mMfvBN0vPWQXhzofg zKqw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=tZ0IiLou; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id a24-20020a170902b59800b001b8a8f359fcsi8319613pls.208.2023.11.20.09.49.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Nov 2023 09:50:10 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=tZ0IiLou; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 9035280EE0F2; Mon, 20 Nov 2023 09:49:46 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234446AbjKTRth (ORCPT + 27 others); Mon, 20 Nov 2023 12:49:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49824 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234249AbjKTRtR (ORCPT ); Mon, 20 Nov 2023 12:49:17 -0500 Received: from out-176.mta0.migadu.com (out-176.mta0.migadu.com [IPv6:2001:41d0:1004:224b::b0]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0D23AD77 for ; Mon, 20 Nov 2023 09:48:35 -0800 (PST) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1700502514; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hhwH9Rj+7SlwckI0c6mMcIkIslfacxpde7X6neQ7UWA=; b=tZ0IiLouwcQL+2MAJN48oud52EurhrFZ0EVdLm9w8bTgwIXFM6wmInvK6PvSCrc9NfsCH+ mZJSfVlEyctgDWkqrhpIIt5iHEnorbr/fJGIrTZlLY3MGFvfhwPt5aeKYygAHMY4xY+vIV lxaS4PU+oIsToSR3HDnVIiG5H7/UWHw= From: andrey.konovalov@linux.dev To: Andrew Morton Cc: Andrey Konovalov , Marco Elver , Alexander Potapenko , Dmitry Vyukov , Vlastimil Babka , kasan-dev@googlegroups.com, Evgenii Stepanov , Oscar Salvador , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v4 09/22] lib/stackdepot: rename next_pool_required to new_pool_required Date: Mon, 20 Nov 2023 18:47:07 +0100 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Mon, 20 Nov 2023 09:49:46 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783106224817746627 X-GMAIL-MSGID: 1783106224817746627 From: Andrey Konovalov Rename next_pool_required to new_pool_required. This a purely code readability change: the following patch will change stack depot to store the pointer to the new pool in a separate variable, and "new" seems like a more logical name. Reviewed-by: Alexander Potapenko Signed-off-by: Andrey Konovalov Reviewed-by: Oscar Salvador --- lib/stackdepot.c | 49 ++++++++++++++++++++++++------------------------ 1 file changed, 24 insertions(+), 25 deletions(-) diff --git a/lib/stackdepot.c b/lib/stackdepot.c index b3af868627f4..a38661beab97 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -93,12 +93,11 @@ static size_t pool_offset; static DEFINE_RAW_SPINLOCK(pool_lock); /* * Stack depot tries to keep an extra pool allocated even before it runs out - * of space in the currently used pool. - * This flag marks that this next extra pool needs to be allocated and - * initialized. It has the value 0 when either the next pool is not yet - * initialized or the limit on the number of pools is reached. + * of space in the currently used pool. This flag marks whether this extra pool + * needs to be allocated. It has the value 0 when either an extra pool is not + * yet allocated or if the limit on the number of pools is reached. */ -static int next_pool_required = 1; +static int new_pool_required = 1; static int __init disable_stack_depot(char *str) { @@ -225,20 +224,20 @@ int stack_depot_init(void) } EXPORT_SYMBOL_GPL(stack_depot_init); -/* Keeps the preallocated memory to be used for the next stack depot pool. */ -static void depot_keep_next_pool(void **prealloc) +/* Keeps the preallocated memory to be used for a new stack depot pool. */ +static void depot_keep_new_pool(void **prealloc) { /* - * If the next pool is already saved or the maximum number of + * If a new pool is already saved or the maximum number of * pools is reached, do not use the preallocated memory. - * Access next_pool_required non-atomically, as there are no concurrent + * Access new_pool_required non-atomically, as there are no concurrent * write accesses to this variable. */ - if (!next_pool_required) + if (!new_pool_required) return; /* - * Use the preallocated memory for the next pool + * Use the preallocated memory for the new pool * as long as we do not exceed the maximum number of pools. */ if (pool_index + 1 < DEPOT_MAX_POOLS) { @@ -247,13 +246,13 @@ static void depot_keep_next_pool(void **prealloc) } /* - * At this point, either the next pool is kept or the maximum + * At this point, either a new pool is kept or the maximum * number of pools is reached. In either case, take note that * keeping another pool is not required. * smp_store_release() pairs with smp_load_acquire() in * stack_depot_save(). */ - smp_store_release(&next_pool_required, 0); + smp_store_release(&new_pool_required, 0); } /* Updates references to the current and the next stack depot pools. */ @@ -268,7 +267,7 @@ static bool depot_update_pools(size_t required_size, void **prealloc) } /* - * Move on to the next pool. + * Move on to the new pool. * WRITE_ONCE() pairs with potential concurrent read in * stack_depot_fetch(). */ @@ -277,12 +276,12 @@ static bool depot_update_pools(size_t required_size, void **prealloc) /* * If the maximum number of pools is not reached, take note - * that the next pool needs to be initialized. + * that yet another new pool needs to be allocated. * smp_store_release() pairs with smp_load_acquire() in * stack_depot_save(). */ if (pool_index + 1 < DEPOT_MAX_POOLS) - smp_store_release(&next_pool_required, 1); + smp_store_release(&new_pool_required, 1); } /* Check if the current pool is not yet allocated. */ @@ -293,9 +292,9 @@ static bool depot_update_pools(size_t required_size, void **prealloc) return true; } - /* Otherwise, try using the preallocated memory for the next pool. */ + /* Otherwise, try using the preallocated memory for a new pool. */ if (*prealloc) - depot_keep_next_pool(prealloc); + depot_keep_new_pool(prealloc); return true; } @@ -306,7 +305,7 @@ depot_alloc_stack(unsigned long *entries, int size, u32 hash, void **prealloc) struct stack_record *stack; size_t required_size = DEPOT_STACK_RECORD_SIZE; - /* Update current and next pools if required and possible. */ + /* Update current and new pools if required and possible. */ if (!depot_update_pools(required_size, prealloc)) return NULL; @@ -438,13 +437,13 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, goto exit; /* - * Check if another stack pool needs to be initialized. If so, allocate - * the memory now - we won't be able to do that under the lock. + * Check if another stack pool needs to be allocated. If so, allocate + * the memory now: we won't be able to do that under the lock. * * smp_load_acquire() pairs with smp_store_release() in - * depot_update_pools() and depot_keep_next_pool(). + * depot_update_pools() and depot_keep_new_pool(). */ - if (unlikely(can_alloc && smp_load_acquire(&next_pool_required))) { + if (unlikely(can_alloc && smp_load_acquire(&new_pool_required))) { /* * Zero out zone modifiers, as we don't have specific zone * requirements. Keep the flags related to allocation in atomic @@ -477,9 +476,9 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, } else if (prealloc) { /* * Stack depot already contains this stack trace, but let's - * keep the preallocated memory for the future. + * keep the preallocated memory for future. */ - depot_keep_next_pool(&prealloc); + depot_keep_new_pool(&prealloc); } raw_spin_unlock_irqrestore(&pool_lock, flags); From patchwork Mon Nov 20 17:47:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 167320 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2b07:b0:403:3b70:6f57 with SMTP id io7csp106879vqb; Mon, 20 Nov 2023 09:50:12 -0800 (PST) X-Google-Smtp-Source: AGHT+IF3S96YAPz9fkfkHMw0hHQkb8YIjdAr+ccCJyd/KTHj0OGqjDt5SNP8AOXa62t4HJvnR05M X-Received: by 2002:a05:6808:23d0:b0:3ae:5e0e:1671 with SMTP id bq16-20020a05680823d000b003ae5e0e1671mr12255556oib.4.1700502611830; Mon, 20 Nov 2023 09:50:11 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700502611; cv=none; d=google.com; s=arc-20160816; b=Pi+8ZPg5RTvNTt2qDU95bvwu40Jj1CyD4viOjOFsGW+44FMq5j6XolBkfnMeFxc4ZZ imXEMGYJqRS+9c+/hCqEuBYzJDVunQW2e7g48WDBIUOFoJt4xrvqFh2QSiCADhHgXVUR j8owdqajaeiVAt2XMHMny6OJkNuya3GF345AvDbEzY738wCiLaZeddGc9jdv9bYeOnn8 VUHkrPT/eIFN35qOKLc2Cc/EtnAFEZFkLzxbEMLOidJyt6lWR2nb8nB5P1CP/wSu3nR0 KX/1xpYUjg560XytQd9zd+vnzGaX0YLWil6WTDGZbrq+O+MgAAsIhaMEloc8KFbmEodg BZZA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Wo0lOk4QF0V4/CSryrBtBcTGMr2NPIzDQs2c2iiME6Q=; fh=qWoxmPN1zRqnLSoCWGpfFsDtzJsx/SGdqx98lbP+Uho=; b=w02MFeCzmkV7MuJv91aQ0vV6KnrwickPELAs4Dd3ZGzOQmN9hVyQckUAxYNSjsstAr ITmJEIG/JvsHTKz2HmpHb1ty73DjQ1P+KZtGs47sxsOL9iztH/Rxo9GbFz/6pee8uene Su8yjwJynvwlSL/8I4HeL+OmfYqXrQDMrhnhHv98l4HOF+8XWUkJsqfK0D1orr6hzBqO QXnn9aOWc2fx/XxXyCOzzxhcsA2/vVB3oHEb+khdmUAGGTkXIeRIfyLRAXic72R6yc0r v4OeTxXqkmfgjrTWP8iffI9Qvv+pKKbrp1v7IsXHDPa9u9UmR3Bh8Q/d7VSDRRN6A21h Y0HQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=mVlXzudg; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id b19-20020a63d313000000b005c1faa82bfesi8387724pgg.857.2023.11.20.09.49.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Nov 2023 09:50:11 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=mVlXzudg; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 41D8880EE0FF; Mon, 20 Nov 2023 09:49:48 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233752AbjKTRtk (ORCPT + 27 others); Mon, 20 Nov 2023 12:49:40 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53448 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234254AbjKTRtS (ORCPT ); Mon, 20 Nov 2023 12:49:18 -0500 Received: from out-184.mta0.migadu.com (out-184.mta0.migadu.com [91.218.175.184]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 291A6D7D for ; Mon, 20 Nov 2023 09:48:37 -0800 (PST) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1700502515; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Wo0lOk4QF0V4/CSryrBtBcTGMr2NPIzDQs2c2iiME6Q=; b=mVlXzudgWPREwFPbPjEzWKl0pkuPrGrBfXgV/o3hu2wbrexzrKyl7c+k+ldkpF3cqIoiwI u8DO0T3MhAJP+5xhUN4eko0SgcoGz172qfdNSCMVSIN1yVWwfrjX7hnhKKWOfY2ZNwgi4e RtBwt9lIOYU3HUK95YFtEZQHsBHOLqY= From: andrey.konovalov@linux.dev To: Andrew Morton Cc: Andrey Konovalov , Marco Elver , Alexander Potapenko , Dmitry Vyukov , Vlastimil Babka , kasan-dev@googlegroups.com, Evgenii Stepanov , Oscar Salvador , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v4 10/22] lib/stackdepot: store next pool pointer in new_pool Date: Mon, 20 Nov 2023 18:47:08 +0100 Message-Id: <448bc18296c16bef95cb3167697be6583dcc8ce3.1700502145.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Mon, 20 Nov 2023 09:49:48 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783106226762000273 X-GMAIL-MSGID: 1783106226762000273 From: Andrey Konovalov Instead of using the last pointer in stack_pools for storing the pointer to a new pool (which does not yet store any stack records), use a new new_pool variable. This a purely code readability change: it seems more logical to store the pointer to a pool with a special meaning in a dedicated variable. Reviewed-by: Alexander Potapenko Signed-off-by: Andrey Konovalov Reviewed-by: Oscar Salvador --- lib/stackdepot.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/lib/stackdepot.c b/lib/stackdepot.c index a38661beab97..68c1ac9aa916 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -85,6 +85,8 @@ static unsigned int stack_hash_mask; /* Array of memory regions that store stack traces. */ static void *stack_pools[DEPOT_MAX_POOLS]; +/* Newly allocated pool that is not yet added to stack_pools. */ +static void *new_pool; /* Currently used pool in stack_pools. */ static int pool_index; /* Offset to the unused space in the currently used pool. */ @@ -241,7 +243,7 @@ static void depot_keep_new_pool(void **prealloc) * as long as we do not exceed the maximum number of pools. */ if (pool_index + 1 < DEPOT_MAX_POOLS) { - stack_pools[pool_index + 1] = *prealloc; + new_pool = *prealloc; *prealloc = NULL; } @@ -272,6 +274,8 @@ static bool depot_update_pools(size_t required_size, void **prealloc) * stack_depot_fetch(). */ WRITE_ONCE(pool_index, pool_index + 1); + stack_pools[pool_index] = new_pool; + new_pool = NULL; pool_offset = 0; /* From patchwork Mon Nov 20 17:47:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 167321 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2b07:b0:403:3b70:6f57 with SMTP id io7csp106882vqb; Mon, 20 Nov 2023 09:50:12 -0800 (PST) X-Google-Smtp-Source: AGHT+IHMw1woukzjkNedJWQgfTgceiCjLTHubiRmnn1bqFiZa1HPPvDfSrT0gA7L6vbJdbtMUWUF X-Received: by 2002:a05:6a00:148b:b0:6be:5a1a:3bb8 with SMTP id v11-20020a056a00148b00b006be5a1a3bb8mr10779735pfu.28.1700502611959; Mon, 20 Nov 2023 09:50:11 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700502611; cv=none; d=google.com; s=arc-20160816; b=NUwZf/qCaqv6RIc2Im1P7bTZzlsPyLnrY8nDLv3Z0JyF4csgjb00K/ebwksjPwsAfq twit3AuxXDALB+jIykJMR6Vquu+AgAk/mVWRyoM065HL8J1vSj/0i2SDe1H54HsKajQ9 PFO7t6uLUXkoCdHEEZPl7B2+IUmAuX2CphgWsZ6hEV7OFWYMfukMOeGVO75W7RDnnama QoEQo+HP0ggawDhvcWzQZr6OtIDG1GiGdApY62fnFwBB5Qc1vGwdhsGquccZj8lnT0+q b+l5CqYuS+auvyLJChZGZFM9M8wuaXTdgGxHxVkOFGJd3jEVxzd2PLTHJdapk3Zj+vEc LHBg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=+N1Q7uFgicZ/Pf0RFILDijpppowVMY0ExiNLLtkdgaQ=; fh=qWoxmPN1zRqnLSoCWGpfFsDtzJsx/SGdqx98lbP+Uho=; b=BM8CuB9iQ27SQFWdyn1Ie33AyUecI980MehpZ7QXhrQLJRHlO3+M6s1KXPYGYOXhit qbgWdn8srbEjPFTovxdrX35vUu4ilXcpmJYMZ7/RXoD+CkgN16t2Dl0ICxIg4niGkAM5 2Uuq/so1NJaop5R3kfMQWGD7c9EwSk+WuIq9aPDJiu/5UMPU/OBSPrJeO7wr2DNHdEZu EwG22v28NQwKGDAMKmO5zP4ClWZM6f2+GtGa0fIbEJIxR7NcxawVkGMZXpco0r1Cc3XD kKpmRL0tQezyhN+cydzOtda/S0Y2QdbcdUhDCpHvPdUIAabdFS22fLQ5Z01fGhftzC7N xARQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b="RxqnWJ/W"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id q10-20020a056a00150a00b006c338d18b6dsi8801316pfu.13.2023.11.20.09.49.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Nov 2023 09:50:11 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b="RxqnWJ/W"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id A838180F9F9B; Mon, 20 Nov 2023 09:49:48 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234581AbjKTRtn (ORCPT + 27 others); Mon, 20 Nov 2023 12:49:43 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49834 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234261AbjKTRtS (ORCPT ); Mon, 20 Nov 2023 12:49:18 -0500 Received: from out-175.mta0.migadu.com (out-175.mta0.migadu.com [91.218.175.175]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8DFEE171D for ; Mon, 20 Nov 2023 09:48:37 -0800 (PST) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1700502515; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+N1Q7uFgicZ/Pf0RFILDijpppowVMY0ExiNLLtkdgaQ=; b=RxqnWJ/WbNov6GNDVKKl98NdJigJDFyuUx5XrfpgY0Y/FIFkxZ8cb2F/DaXDPkvN4oYnyI w7lMzmrfXQRH2KCXJzK07x0GGzxp0h1cjOV+z4h6XLtv4ZtoYRFtRsvvF70WP3/hsS5GYv gtRARyq+ePtSURhjvSTg/rB+k7qi5sM= From: andrey.konovalov@linux.dev To: Andrew Morton Cc: Andrey Konovalov , Marco Elver , Alexander Potapenko , Dmitry Vyukov , Vlastimil Babka , kasan-dev@googlegroups.com, Evgenii Stepanov , Oscar Salvador , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v4 11/22] lib/stackdepot: store free stack records in a freelist Date: Mon, 20 Nov 2023 18:47:09 +0100 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Mon, 20 Nov 2023 09:49:48 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783106226477500654 X-GMAIL-MSGID: 1783106226477500654 From: Andrey Konovalov Instead of using the global pool_offset variable to find a free slot when storing a new stack record, mainlain a freelist of free slots within the allocated stack pools. A global next_stack variable is used as the head of the freelist, and the next field in the stack_record struct is reused as freelist link (when the record is not in the freelist, this field is used as a link in the hash table). This is preparatory patch for implementing the eviction of stack records from the stack depot. Reviewed-by: Alexander Potapenko Signed-off-by: Andrey Konovalov --- Changes v2->v3: - Add parentheses when referring to function calls in comments. Changes v1->v2: - Fix out-of-bounds when initializing a pool. --- lib/stackdepot.c | 131 +++++++++++++++++++++++++++++------------------ 1 file changed, 82 insertions(+), 49 deletions(-) diff --git a/lib/stackdepot.c b/lib/stackdepot.c index 68c1ac9aa916..a5eff165c0d5 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -54,8 +54,8 @@ union handle_parts { }; struct stack_record { - struct stack_record *next; /* Link in the hash table */ - u32 hash; /* Hash in the hash table */ + struct stack_record *next; /* Link in hash table or freelist */ + u32 hash; /* Hash in hash table */ u32 size; /* Number of stored frames */ union handle_parts handle; unsigned long entries[CONFIG_STACKDEPOT_MAX_FRAMES]; /* Frames */ @@ -87,10 +87,10 @@ static unsigned int stack_hash_mask; static void *stack_pools[DEPOT_MAX_POOLS]; /* Newly allocated pool that is not yet added to stack_pools. */ static void *new_pool; -/* Currently used pool in stack_pools. */ -static int pool_index; -/* Offset to the unused space in the currently used pool. */ -static size_t pool_offset; +/* Number of pools in stack_pools. */ +static int pools_num; +/* Next stack in the freelist of stack records within stack_pools. */ +static struct stack_record *next_stack; /* Lock that protects the variables above. */ static DEFINE_RAW_SPINLOCK(pool_lock); /* @@ -226,6 +226,42 @@ int stack_depot_init(void) } EXPORT_SYMBOL_GPL(stack_depot_init); +/* Initializes a stack depol pool. */ +static void depot_init_pool(void *pool) +{ + const int records_in_pool = DEPOT_POOL_SIZE / DEPOT_STACK_RECORD_SIZE; + int i, offset; + + /* Initialize handles and link stack records to each other. */ + for (i = 0, offset = 0; + offset <= DEPOT_POOL_SIZE - DEPOT_STACK_RECORD_SIZE; + i++, offset += DEPOT_STACK_RECORD_SIZE) { + struct stack_record *stack = pool + offset; + + stack->handle.pool_index = pools_num; + stack->handle.offset = offset >> DEPOT_STACK_ALIGN; + stack->handle.extra = 0; + + if (i < records_in_pool - 1) + stack->next = (void *)stack + DEPOT_STACK_RECORD_SIZE; + else + stack->next = NULL; + } + + /* Link stack records into the freelist. */ + WARN_ON(next_stack); + next_stack = pool; + + /* Save reference to the pool to be used by depot_fetch_stack(). */ + stack_pools[pools_num] = pool; + + /* + * WRITE_ONCE() pairs with potential concurrent read in + * depot_fetch_stack(). + */ + WRITE_ONCE(pools_num, pools_num + 1); +} + /* Keeps the preallocated memory to be used for a new stack depot pool. */ static void depot_keep_new_pool(void **prealloc) { @@ -242,7 +278,7 @@ static void depot_keep_new_pool(void **prealloc) * Use the preallocated memory for the new pool * as long as we do not exceed the maximum number of pools. */ - if (pool_index + 1 < DEPOT_MAX_POOLS) { + if (pools_num < DEPOT_MAX_POOLS) { new_pool = *prealloc; *prealloc = NULL; } @@ -258,45 +294,42 @@ static void depot_keep_new_pool(void **prealloc) } /* Updates references to the current and the next stack depot pools. */ -static bool depot_update_pools(size_t required_size, void **prealloc) +static bool depot_update_pools(void **prealloc) { - /* Check if there is not enough space in the current pool. */ - if (unlikely(pool_offset + required_size > DEPOT_POOL_SIZE)) { - /* Bail out if we reached the pool limit. */ - if (unlikely(pool_index + 1 >= DEPOT_MAX_POOLS)) { - WARN_ONCE(1, "Stack depot reached limit capacity"); - return false; - } + /* Check if we still have objects in the freelist. */ + if (next_stack) + goto out_keep_prealloc; - /* - * Move on to the new pool. - * WRITE_ONCE() pairs with potential concurrent read in - * stack_depot_fetch(). - */ - WRITE_ONCE(pool_index, pool_index + 1); - stack_pools[pool_index] = new_pool; + /* Check if we have a new pool saved and use it. */ + if (new_pool) { + depot_init_pool(new_pool); new_pool = NULL; - pool_offset = 0; - /* - * If the maximum number of pools is not reached, take note - * that yet another new pool needs to be allocated. - * smp_store_release() pairs with smp_load_acquire() in - * stack_depot_save(). - */ - if (pool_index + 1 < DEPOT_MAX_POOLS) + /* Take note that we might need a new new_pool. */ + if (pools_num < DEPOT_MAX_POOLS) smp_store_release(&new_pool_required, 1); + + /* Try keeping the preallocated memory for new_pool. */ + goto out_keep_prealloc; + } + + /* Bail out if we reached the pool limit. */ + if (unlikely(pools_num >= DEPOT_MAX_POOLS)) { + WARN_ONCE(1, "Stack depot reached limit capacity"); + return false; } - /* Check if the current pool is not yet allocated. */ - if (*prealloc && stack_pools[pool_index] == NULL) { - /* Use the preallocated memory for the current pool. */ - stack_pools[pool_index] = *prealloc; + /* Check if we have preallocated memory and use it. */ + if (*prealloc) { + depot_init_pool(*prealloc); *prealloc = NULL; return true; } - /* Otherwise, try using the preallocated memory for a new pool. */ + return false; + +out_keep_prealloc: + /* Keep the preallocated memory for a new pool if required. */ if (*prealloc) depot_keep_new_pool(prealloc); return true; @@ -307,35 +340,35 @@ static struct stack_record * depot_alloc_stack(unsigned long *entries, int size, u32 hash, void **prealloc) { struct stack_record *stack; - size_t required_size = DEPOT_STACK_RECORD_SIZE; /* Update current and new pools if required and possible. */ - if (!depot_update_pools(required_size, prealloc)) + if (!depot_update_pools(prealloc)) return NULL; - /* Check if we have a pool to save the stack trace. */ - if (stack_pools[pool_index] == NULL) + /* Check if we have a stack record to save the stack trace. */ + stack = next_stack; + if (!stack) return NULL; + /* Advance the freelist. */ + next_stack = stack->next; + /* Limit number of saved frames to CONFIG_STACKDEPOT_MAX_FRAMES. */ if (size > CONFIG_STACKDEPOT_MAX_FRAMES) size = CONFIG_STACKDEPOT_MAX_FRAMES; /* Save the stack trace. */ - stack = stack_pools[pool_index] + pool_offset; + stack->next = NULL; stack->hash = hash; stack->size = size; - stack->handle.pool_index = pool_index; - stack->handle.offset = pool_offset >> DEPOT_STACK_ALIGN; - stack->handle.extra = 0; + /* stack->handle is already filled in by depot_init_pool(). */ memcpy(stack->entries, entries, flex_array_size(stack, entries, size)); - pool_offset += required_size; /* * Let KMSAN know the stored stack record is initialized. This shall * prevent false positive reports if instrumented code accesses it. */ - kmsan_unpoison_memory(stack, required_size); + kmsan_unpoison_memory(stack, DEPOT_STACK_RECORD_SIZE); return stack; } @@ -345,16 +378,16 @@ static struct stack_record *depot_fetch_stack(depot_stack_handle_t handle) union handle_parts parts = { .handle = handle }; /* * READ_ONCE() pairs with potential concurrent write in - * depot_update_pools(). + * depot_init_pool(). */ - int pool_index_cached = READ_ONCE(pool_index); + int pools_num_cached = READ_ONCE(pools_num); void *pool; size_t offset = parts.offset << DEPOT_STACK_ALIGN; struct stack_record *stack; - if (parts.pool_index > pool_index_cached) { + if (parts.pool_index > pools_num_cached) { WARN(1, "pool index %d out of bounds (%d) for stack id %08x\n", - parts.pool_index, pool_index_cached, handle); + parts.pool_index, pools_num_cached, handle); return NULL; } From patchwork Mon Nov 20 17:47:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 167322 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2b07:b0:403:3b70:6f57 with SMTP id io7csp106900vqb; Mon, 20 Nov 2023 09:50:13 -0800 (PST) X-Google-Smtp-Source: AGHT+IEA0inY35yAE4JJbMFfd4QWn+RP+Qtsy4i7b/fwavitHNBRag3JFFUB+XmKKw59xH2fMsQy X-Received: by 2002:a05:6808:222a:b0:3b6:cb7f:8315 with SMTP id bd42-20020a056808222a00b003b6cb7f8315mr12291462oib.11.1700502613086; Mon, 20 Nov 2023 09:50:13 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700502613; cv=none; d=google.com; s=arc-20160816; b=rA1VYmlJshxtUXFsO9AkSFIS8fstmlNCEoHFB7+DE8geDLKRb0Kz8bY3wmBFE+AbI9 mJhUKJ7guNndvsYRbUjjTarsoacuUC46c4fxfgZ9hsUUu5FZ/ja8DovPKOymArsiFYFk 3G3IdfsK7mj1g9re8/4P+SwK9n7BfcJ9q91PsXZyA50sQvoHOoZteDIyPG5w/ihzhOdT FZ0ETry8ducDCYHivnmOjAMOGpaMNE8Dbe/G7y3TmWOUNUtZz3dzG9KhfbM8//4LBuW1 ry5HDH+V36ycdBHzSGXg03BJVlQV76ZcQNQ40wIC1GVbeiOFNVEdBiWjY9hdF2fhcRee CCKw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=K4xViYAWVaV6VQ5HtL4BYhsbr4MB9DPwoOHOaowHM8k=; fh=qWoxmPN1zRqnLSoCWGpfFsDtzJsx/SGdqx98lbP+Uho=; b=fV5ltOPq8GqSKGlW8yCz4gfAnE87DQIA4Xr5z9J4+HvLS120OtwXZXyqz4tl1SaQbl ESqzOTG6K6jl1YPQlYbDYxr2zHAwCVwA1xCAdVFMh92lZC8fYQMgL0e1mg/CZstX291u zTdOoEUcXMhOBv/GSzrtwly1qpWJTNVmTyb6LVzsR2eZ744Gb7ou2OmTuCo1I+Z3HF5f p4vCD6aA4YqSbBXyFDU9dX5c0zRPOTEs+Rjc2IniByTSKsJPCvvAaOsywsDag1rxA0dx WHxsGAadeiQx4T/8FRPCNSPBQOhbeCjoDOJCDHcXYICxAaROy+tFHwq7WnzrspBNYTCB QEyA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=IaUsjEmo; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id 12-20020a630b0c000000b005b92fb731a9si8091078pgl.834.2023.11.20.09.50.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Nov 2023 09:50:13 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=IaUsjEmo; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 4870D80EE0DE; Mon, 20 Nov 2023 09:50:12 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234274AbjKTRuI (ORCPT + 27 others); Mon, 20 Nov 2023 12:50:08 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50006 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234592AbjKTRts (ORCPT ); Mon, 20 Nov 2023 12:49:48 -0500 Received: from out-189.mta0.migadu.com (out-189.mta0.migadu.com [91.218.175.189]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 69A7210D4 for ; Mon, 20 Nov 2023 09:49:39 -0800 (PST) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1700502577; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=K4xViYAWVaV6VQ5HtL4BYhsbr4MB9DPwoOHOaowHM8k=; b=IaUsjEmoPRLDB8+q+N1/FFx4BQ3dNpy2/8UK1Gk7f8jxcPQzpMGtk0vCNWAd3ep43VVaLY YSfB7RGeC4krK+GbjgpVSoqyBiDp1Yja1RT2FgG6vH1HpRdUzo+USs4eje1zfRoqc8P41c 5glC0pVcsGIq8X4efr6ATYrkrC3nrgc= From: andrey.konovalov@linux.dev To: Andrew Morton Cc: Andrey Konovalov , Marco Elver , Alexander Potapenko , Dmitry Vyukov , Vlastimil Babka , kasan-dev@googlegroups.com, Evgenii Stepanov , Oscar Salvador , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v4 12/22] lib/stackdepot: use read/write lock Date: Mon, 20 Nov 2023 18:47:10 +0100 Message-Id: <9f81ffcc4bb422ebb6326a65a770bf1918634cbb.1700502145.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Mon, 20 Nov 2023 09:50:12 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783106228011295791 X-GMAIL-MSGID: 1783106228011295791 From: Andrey Konovalov Currently, stack depot uses the following locking scheme: 1. Lock-free accesses when looking up a stack record, which allows to have multiple users to look up records in parallel; 2. Spinlock for protecting the stack depot pools and the hash table when adding a new record. For implementing the eviction of stack traces from stack depot, the lock-free approach is not going to work anymore, as we will need to be able to also remove records from the hash table. Convert the spinlock into a read/write lock, and drop the atomic accesses, as they are no longer required. Looking up stack traces is now protected by the read lock and adding new records - by the write lock. One of the following patches will add a new function for evicting stack records, which will be protected by the write lock as well. With this change, multiple users can still look up records in parallel. This is preparatory patch for implementing the eviction of stack records from the stack depot. Reviewed-by: Alexander Potapenko Signed-off-by: Andrey Konovalov Reviewed-by: Oscar Salvador Signed-off-by: Marco Elver --- Changed v2->v3: - Use lockdep_assert_held_read annotation in depot_fetch_stack. Changes v1->v2: - Add lockdep_assert annotations. --- lib/stackdepot.c | 87 +++++++++++++++++++++++++----------------------- 1 file changed, 46 insertions(+), 41 deletions(-) diff --git a/lib/stackdepot.c b/lib/stackdepot.c index a5eff165c0d5..8378b32b5310 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -23,6 +23,7 @@ #include #include #include +#include #include #include #include @@ -91,15 +92,15 @@ static void *new_pool; static int pools_num; /* Next stack in the freelist of stack records within stack_pools. */ static struct stack_record *next_stack; -/* Lock that protects the variables above. */ -static DEFINE_RAW_SPINLOCK(pool_lock); /* * Stack depot tries to keep an extra pool allocated even before it runs out * of space in the currently used pool. This flag marks whether this extra pool * needs to be allocated. It has the value 0 when either an extra pool is not * yet allocated or if the limit on the number of pools is reached. */ -static int new_pool_required = 1; +static bool new_pool_required = true; +/* Lock that protects the variables above. */ +static DEFINE_RWLOCK(pool_rwlock); static int __init disable_stack_depot(char *str) { @@ -232,6 +233,8 @@ static void depot_init_pool(void *pool) const int records_in_pool = DEPOT_POOL_SIZE / DEPOT_STACK_RECORD_SIZE; int i, offset; + lockdep_assert_held_write(&pool_rwlock); + /* Initialize handles and link stack records to each other. */ for (i = 0, offset = 0; offset <= DEPOT_POOL_SIZE - DEPOT_STACK_RECORD_SIZE; @@ -254,22 +257,17 @@ static void depot_init_pool(void *pool) /* Save reference to the pool to be used by depot_fetch_stack(). */ stack_pools[pools_num] = pool; - - /* - * WRITE_ONCE() pairs with potential concurrent read in - * depot_fetch_stack(). - */ - WRITE_ONCE(pools_num, pools_num + 1); + pools_num++; } /* Keeps the preallocated memory to be used for a new stack depot pool. */ static void depot_keep_new_pool(void **prealloc) { + lockdep_assert_held_write(&pool_rwlock); + /* * If a new pool is already saved or the maximum number of * pools is reached, do not use the preallocated memory. - * Access new_pool_required non-atomically, as there are no concurrent - * write accesses to this variable. */ if (!new_pool_required) return; @@ -287,15 +285,15 @@ static void depot_keep_new_pool(void **prealloc) * At this point, either a new pool is kept or the maximum * number of pools is reached. In either case, take note that * keeping another pool is not required. - * smp_store_release() pairs with smp_load_acquire() in - * stack_depot_save(). */ - smp_store_release(&new_pool_required, 0); + new_pool_required = false; } /* Updates references to the current and the next stack depot pools. */ static bool depot_update_pools(void **prealloc) { + lockdep_assert_held_write(&pool_rwlock); + /* Check if we still have objects in the freelist. */ if (next_stack) goto out_keep_prealloc; @@ -307,7 +305,7 @@ static bool depot_update_pools(void **prealloc) /* Take note that we might need a new new_pool. */ if (pools_num < DEPOT_MAX_POOLS) - smp_store_release(&new_pool_required, 1); + new_pool_required = true; /* Try keeping the preallocated memory for new_pool. */ goto out_keep_prealloc; @@ -341,6 +339,8 @@ depot_alloc_stack(unsigned long *entries, int size, u32 hash, void **prealloc) { struct stack_record *stack; + lockdep_assert_held_write(&pool_rwlock); + /* Update current and new pools if required and possible. */ if (!depot_update_pools(prealloc)) return NULL; @@ -376,18 +376,15 @@ depot_alloc_stack(unsigned long *entries, int size, u32 hash, void **prealloc) static struct stack_record *depot_fetch_stack(depot_stack_handle_t handle) { union handle_parts parts = { .handle = handle }; - /* - * READ_ONCE() pairs with potential concurrent write in - * depot_init_pool(). - */ - int pools_num_cached = READ_ONCE(pools_num); void *pool; size_t offset = parts.offset << DEPOT_STACK_ALIGN; struct stack_record *stack; - if (parts.pool_index > pools_num_cached) { + lockdep_assert_held_read(&pool_rwlock); + + if (parts.pool_index > pools_num) { WARN(1, "pool index %d out of bounds (%d) for stack id %08x\n", - parts.pool_index, pools_num_cached, handle); + parts.pool_index, pools_num, handle); return NULL; } @@ -429,6 +426,8 @@ static inline struct stack_record *find_stack(struct stack_record *bucket, { struct stack_record *found; + lockdep_assert_held(&pool_rwlock); + for (found = bucket; found; found = found->next) { if (found->hash == hash && found->size == size && @@ -446,6 +445,7 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, depot_stack_handle_t handle = 0; struct page *page = NULL; void *prealloc = NULL; + bool need_alloc = false; unsigned long flags; u32 hash; @@ -465,22 +465,26 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, hash = hash_stack(entries, nr_entries); bucket = &stack_table[hash & stack_hash_mask]; - /* - * Fast path: look the stack trace up without locking. - * smp_load_acquire() pairs with smp_store_release() to |bucket| below. - */ - found = find_stack(smp_load_acquire(bucket), entries, nr_entries, hash); - if (found) + read_lock_irqsave(&pool_rwlock, flags); + + /* Fast path: look the stack trace up without full locking. */ + found = find_stack(*bucket, entries, nr_entries, hash); + if (found) { + read_unlock_irqrestore(&pool_rwlock, flags); goto exit; + } + + /* Take note if another stack pool needs to be allocated. */ + if (new_pool_required) + need_alloc = true; + + read_unlock_irqrestore(&pool_rwlock, flags); /* - * Check if another stack pool needs to be allocated. If so, allocate - * the memory now: we won't be able to do that under the lock. - * - * smp_load_acquire() pairs with smp_store_release() in - * depot_update_pools() and depot_keep_new_pool(). + * Allocate memory for a new pool if required now: + * we won't be able to do that under the lock. */ - if (unlikely(can_alloc && smp_load_acquire(&new_pool_required))) { + if (unlikely(can_alloc && need_alloc)) { /* * Zero out zone modifiers, as we don't have specific zone * requirements. Keep the flags related to allocation in atomic @@ -494,7 +498,7 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, prealloc = page_address(page); } - raw_spin_lock_irqsave(&pool_lock, flags); + write_lock_irqsave(&pool_rwlock, flags); found = find_stack(*bucket, entries, nr_entries, hash); if (!found) { @@ -503,11 +507,7 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, if (new) { new->next = *bucket; - /* - * smp_store_release() pairs with smp_load_acquire() - * from |bucket| above. - */ - smp_store_release(bucket, new); + *bucket = new; found = new; } } else if (prealloc) { @@ -518,7 +518,7 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, depot_keep_new_pool(&prealloc); } - raw_spin_unlock_irqrestore(&pool_lock, flags); + write_unlock_irqrestore(&pool_rwlock, flags); exit: if (prealloc) { /* Stack depot didn't use this memory, free it. */ @@ -542,6 +542,7 @@ unsigned int stack_depot_fetch(depot_stack_handle_t handle, unsigned long **entries) { struct stack_record *stack; + unsigned long flags; *entries = NULL; /* @@ -553,8 +554,12 @@ unsigned int stack_depot_fetch(depot_stack_handle_t handle, if (!handle || stack_depot_disabled) return 0; + read_lock_irqsave(&pool_rwlock, flags); + stack = depot_fetch_stack(handle); + read_unlock_irqrestore(&pool_rwlock, flags); + *entries = stack->entries; return stack->size; } From patchwork Mon Nov 20 17:47:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 167325 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2b07:b0:403:3b70:6f57 with SMTP id io7csp106978vqb; Mon, 20 Nov 2023 09:50:19 -0800 (PST) X-Google-Smtp-Source: AGHT+IEpYBsERedkgfgj65NV/09FvRzucG5TYJIk5oyYELQyhUlf2HwrfQNr52HAvE8/vcjDAQvH X-Received: by 2002:a05:6a20:9589:b0:189:f8bc:71a3 with SMTP id iu9-20020a056a20958900b00189f8bc71a3mr4827283pzb.37.1700502619366; Mon, 20 Nov 2023 09:50:19 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700502619; cv=none; d=google.com; s=arc-20160816; b=EYjI1Kj9E5qWo4OEDP5OoP1h1Ukvwp/bV2vTVbHu4QeJvgfBPHNUIUcSpFwhjwlMQg Jcq6W6Tr7ddyPYljvk8Nhp3XDJaBAU1A5WgIWSvhj4RzvvvK+sfVAXKvR3kEB8ghuHe9 n9EJFNzcNWW6XRVyl7d2o+fMJDaZSP6wTdSp1w0QUYnQM/OeY5ytRW7Q1ReCTq+3GVb2 fzeN3xRqI0HeyarZVMHxfccitczU+xewuxypkrzEInGtPh6Ip3ELtzuNX93rshaVYa5P MUl+NEnHTDS90zTI249WFklHujn7oue2RMDuucoIpdwkhpa5dwz4Fp42V2fwU1NYFld7 hkzA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=JJPmOktxCpdhFxDsq8sCddRT3Qn7jxR2DhkERqO3NMk=; fh=qWoxmPN1zRqnLSoCWGpfFsDtzJsx/SGdqx98lbP+Uho=; b=YxLFUvKhAnLno6iqiJ34UYLNPstmk3BVCWN/q/Gg0iASA7Y1pm+mT9J5mNxB9DY260 VMmMJETacjp3lVA3AXIxMeWqzMtB2a1OUhRylJfkLU86Fx24nM+ucRaixBMurrfh6Bqm iUQ+f91Q7mdxTw7rAM1k92KybBBTT9eehTQceBNcxSCIyPQlYHLBkbEMr5pwkNcQu4DZ dZEL3zesNcEjcpQrHezmQE3vip6THFj14qKlwYBSjn8U+zS3hxdPxoAXRhwez7x2G7m2 WUlNVVNNeq6S4C3rwXSJpKIJY1XdPDxqQK5jPWDeu5RzUSLW4T3aDY28SSsDrNdndh/W jDrQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=Ru67pJEW; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id d10-20020a056a00244a00b006c4d1f10b8asi8670760pfj.99.2023.11.20.09.50.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Nov 2023 09:50:19 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=Ru67pJEW; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 71F9180FA85B; Mon, 20 Nov 2023 09:50:18 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234545AbjKTRuN (ORCPT + 27 others); Mon, 20 Nov 2023 12:50:13 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49320 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234599AbjKTRtt (ORCPT ); Mon, 20 Nov 2023 12:49:49 -0500 Received: from out-189.mta0.migadu.com (out-189.mta0.migadu.com [IPv6:2001:41d0:1004:224b::bd]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8E2E610D5 for ; Mon, 20 Nov 2023 09:49:39 -0800 (PST) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1700502578; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JJPmOktxCpdhFxDsq8sCddRT3Qn7jxR2DhkERqO3NMk=; b=Ru67pJEW7nzqb9rpO9yuZYxURTiycM1daQ55M4avqe1G86/dIuZUOw3dlKb408oXGg6x5O JLJSygolUyzo1EhZYq8z5/W03AxWgm/1aMH2K0UPV7/E3PTSKcGiP2gUzwgoDhjawwRIqO osUPMvkhvTwmI4K9sg+oiHZAmIKPW1E= From: andrey.konovalov@linux.dev To: Andrew Morton Cc: Andrey Konovalov , Marco Elver , Alexander Potapenko , Dmitry Vyukov , Vlastimil Babka , kasan-dev@googlegroups.com, Evgenii Stepanov , Oscar Salvador , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v4 13/22] lib/stackdepot: use list_head for stack record links Date: Mon, 20 Nov 2023 18:47:11 +0100 Message-Id: <4787d9a584cd33433d9ee1846b17fa3d3e1987ad.1700502145.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Mon, 20 Nov 2023 09:50:18 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783106234606128386 X-GMAIL-MSGID: 1783106234606128386 From: Andrey Konovalov Switch stack_record to use list_head for links in the hash table and in the freelist. This will allow removing entries from the hash table buckets. This is preparatory patch for implementing the eviction of stack records from the stack depot. Signed-off-by: Andrey Konovalov --- Changes v2->v3: - Use the proper number of entries for initializing the stack table when alloc_large_system_hash() auto-calculates the number. Changes v1->v2: - Use list_head instead of open-coding backward links. --- lib/stackdepot.c | 87 ++++++++++++++++++++++++++++-------------------- 1 file changed, 50 insertions(+), 37 deletions(-) diff --git a/lib/stackdepot.c b/lib/stackdepot.c index 8378b32b5310..4bb0af423f82 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -18,6 +18,7 @@ #include #include #include +#include #include #include #include @@ -55,7 +56,7 @@ union handle_parts { }; struct stack_record { - struct stack_record *next; /* Link in hash table or freelist */ + struct list_head list; /* Links in hash table or freelist */ u32 hash; /* Hash in hash table */ u32 size; /* Number of stored frames */ union handle_parts handle; @@ -77,21 +78,21 @@ static bool __stack_depot_early_init_passed __initdata; /* Initial seed for jhash2. */ #define STACK_HASH_SEED 0x9747b28c -/* Hash table of pointers to stored stack traces. */ -static struct stack_record **stack_table; +/* Hash table of stored stack records. */ +static struct list_head *stack_table; /* Fixed order of the number of table buckets. Used when KASAN is enabled. */ static unsigned int stack_bucket_number_order; /* Hash mask for indexing the table. */ static unsigned int stack_hash_mask; -/* Array of memory regions that store stack traces. */ +/* Array of memory regions that store stack records. */ static void *stack_pools[DEPOT_MAX_POOLS]; /* Newly allocated pool that is not yet added to stack_pools. */ static void *new_pool; /* Number of pools in stack_pools. */ static int pools_num; -/* Next stack in the freelist of stack records within stack_pools. */ -static struct stack_record *next_stack; +/* Freelist of stack records within stack_pools. */ +static LIST_HEAD(free_stacks); /* * Stack depot tries to keep an extra pool allocated even before it runs out * of space in the currently used pool. This flag marks whether this extra pool @@ -116,6 +117,15 @@ void __init stack_depot_request_early_init(void) __stack_depot_early_init_requested = true; } +/* Initialize list_head's within the hash table. */ +static void init_stack_table(unsigned long entries) +{ + unsigned long i; + + for (i = 0; i < entries; i++) + INIT_LIST_HEAD(&stack_table[i]); +} + /* Allocates a hash table via memblock. Can only be used during early boot. */ int __init stack_depot_early_init(void) { @@ -152,16 +162,16 @@ int __init stack_depot_early_init(void) /* * If stack_bucket_number_order is not set, leave entries as 0 to rely - * on the automatic calculations performed by alloc_large_system_hash. + * on the automatic calculations performed by alloc_large_system_hash(). */ if (stack_bucket_number_order) entries = 1UL << stack_bucket_number_order; pr_info("allocating hash table via alloc_large_system_hash\n"); stack_table = alloc_large_system_hash("stackdepot", - sizeof(struct stack_record *), + sizeof(struct list_head), entries, STACK_HASH_TABLE_SCALE, - HASH_EARLY | HASH_ZERO, + HASH_EARLY, NULL, &stack_hash_mask, 1UL << STACK_BUCKET_NUMBER_ORDER_MIN, @@ -171,6 +181,14 @@ int __init stack_depot_early_init(void) stack_depot_disabled = true; return -ENOMEM; } + if (!entries) { + /* + * Obtain the number of entries that was calculated by + * alloc_large_system_hash(). + */ + entries = stack_hash_mask + 1; + } + init_stack_table(entries); return 0; } @@ -211,7 +229,7 @@ int stack_depot_init(void) entries = 1UL << STACK_BUCKET_NUMBER_ORDER_MAX; pr_info("allocating hash table of %lu entries via kvcalloc\n", entries); - stack_table = kvcalloc(entries, sizeof(struct stack_record *), GFP_KERNEL); + stack_table = kvcalloc(entries, sizeof(struct list_head), GFP_KERNEL); if (!stack_table) { pr_err("hash table allocation failed, disabling\n"); stack_depot_disabled = true; @@ -219,6 +237,7 @@ int stack_depot_init(void) goto out_unlock; } stack_hash_mask = entries - 1; + init_stack_table(entries); out_unlock: mutex_unlock(&stack_depot_init_mutex); @@ -230,31 +249,24 @@ EXPORT_SYMBOL_GPL(stack_depot_init); /* Initializes a stack depol pool. */ static void depot_init_pool(void *pool) { - const int records_in_pool = DEPOT_POOL_SIZE / DEPOT_STACK_RECORD_SIZE; - int i, offset; + int offset; lockdep_assert_held_write(&pool_rwlock); - /* Initialize handles and link stack records to each other. */ - for (i = 0, offset = 0; - offset <= DEPOT_POOL_SIZE - DEPOT_STACK_RECORD_SIZE; - i++, offset += DEPOT_STACK_RECORD_SIZE) { + WARN_ON(!list_empty(&free_stacks)); + + /* Initialize handles and link stack records into the freelist. */ + for (offset = 0; offset <= DEPOT_POOL_SIZE - DEPOT_STACK_RECORD_SIZE; + offset += DEPOT_STACK_RECORD_SIZE) { struct stack_record *stack = pool + offset; stack->handle.pool_index = pools_num; stack->handle.offset = offset >> DEPOT_STACK_ALIGN; stack->handle.extra = 0; - if (i < records_in_pool - 1) - stack->next = (void *)stack + DEPOT_STACK_RECORD_SIZE; - else - stack->next = NULL; + list_add(&stack->list, &free_stacks); } - /* Link stack records into the freelist. */ - WARN_ON(next_stack); - next_stack = pool; - /* Save reference to the pool to be used by depot_fetch_stack(). */ stack_pools[pools_num] = pool; pools_num++; @@ -295,7 +307,7 @@ static bool depot_update_pools(void **prealloc) lockdep_assert_held_write(&pool_rwlock); /* Check if we still have objects in the freelist. */ - if (next_stack) + if (!list_empty(&free_stacks)) goto out_keep_prealloc; /* Check if we have a new pool saved and use it. */ @@ -346,19 +358,18 @@ depot_alloc_stack(unsigned long *entries, int size, u32 hash, void **prealloc) return NULL; /* Check if we have a stack record to save the stack trace. */ - stack = next_stack; - if (!stack) + if (list_empty(&free_stacks)) return NULL; - /* Advance the freelist. */ - next_stack = stack->next; + /* Get and unlink the first entry from the freelist. */ + stack = list_first_entry(&free_stacks, struct stack_record, list); + list_del(&stack->list); /* Limit number of saved frames to CONFIG_STACKDEPOT_MAX_FRAMES. */ if (size > CONFIG_STACKDEPOT_MAX_FRAMES) size = CONFIG_STACKDEPOT_MAX_FRAMES; /* Save the stack trace. */ - stack->next = NULL; stack->hash = hash; stack->size = size; /* stack->handle is already filled in by depot_init_pool(). */ @@ -420,15 +431,17 @@ int stackdepot_memcmp(const unsigned long *u1, const unsigned long *u2, } /* Finds a stack in a bucket of the hash table. */ -static inline struct stack_record *find_stack(struct stack_record *bucket, +static inline struct stack_record *find_stack(struct list_head *bucket, unsigned long *entries, int size, u32 hash) { + struct list_head *pos; struct stack_record *found; lockdep_assert_held(&pool_rwlock); - for (found = bucket; found; found = found->next) { + list_for_each(pos, bucket) { + found = list_entry(pos, struct stack_record, list); if (found->hash == hash && found->size == size && !stackdepot_memcmp(entries, found->entries, size)) @@ -441,7 +454,8 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, unsigned int nr_entries, gfp_t alloc_flags, bool can_alloc) { - struct stack_record *found = NULL, **bucket; + struct list_head *bucket; + struct stack_record *found = NULL; depot_stack_handle_t handle = 0; struct page *page = NULL; void *prealloc = NULL; @@ -468,7 +482,7 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, read_lock_irqsave(&pool_rwlock, flags); /* Fast path: look the stack trace up without full locking. */ - found = find_stack(*bucket, entries, nr_entries, hash); + found = find_stack(bucket, entries, nr_entries, hash); if (found) { read_unlock_irqrestore(&pool_rwlock, flags); goto exit; @@ -500,14 +514,13 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, write_lock_irqsave(&pool_rwlock, flags); - found = find_stack(*bucket, entries, nr_entries, hash); + found = find_stack(bucket, entries, nr_entries, hash); if (!found) { struct stack_record *new = depot_alloc_stack(entries, nr_entries, hash, &prealloc); if (new) { - new->next = *bucket; - *bucket = new; + list_add(&new->list, bucket); found = new; } } else if (prealloc) { From patchwork Mon Nov 20 17:47:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 167324 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2b07:b0:403:3b70:6f57 with SMTP id io7csp106952vqb; Mon, 20 Nov 2023 09:50:17 -0800 (PST) X-Google-Smtp-Source: AGHT+IHoyQObsXEWKO70l2u6IE1LCUlSAD3tTVPIT414LxJVaoJjw6bu3B3ECRCao1K0CIbn9vdm X-Received: by 2002:a05:6a21:a58f:b0:187:63b1:94c with SMTP id gd15-20020a056a21a58f00b0018763b1094cmr7232883pzc.56.1700502617379; Mon, 20 Nov 2023 09:50:17 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700502617; cv=none; d=google.com; s=arc-20160816; b=dfU9DhRVCrGCfG3PP8T+X629sEsI9VNgHmuu3BemuwAoPIyfVP8J3DnxM10meAIq40 6yI0VAnzjEpsTgwPccOOjj7pK/dW1qoPIZNWHRMGDfsrAyAuHhEEfI2cszox4FSC3EO8 XUXF/dpzihX//aGIr4+asHjJ+6YkoRw6aGJ4eDa8spdvOq0XdlSp6h6ZSWF5jgIys0zn xP96kZS3b8bAHP9vnvDgTA6fCk/cvlyMo/xLPr4bi13iptep32zQxD1uk6jt1uyd2PtG H50Mk0GQcpYIcK6LzliYsbO/nbYGE4XpHOY/bUvCjHX88JgrvwFh88YZ9Rgyr+a1flQk fJYQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=AmubiAZbFJ0d/i3HqBc+VhVD2T8fru9oX5DWKdU3rlA=; fh=qWoxmPN1zRqnLSoCWGpfFsDtzJsx/SGdqx98lbP+Uho=; b=vnjCVTK/429sikC6eCQz6OO2zdSm74Hyum/fX0RyWgtjWY8iAORYZXJD4BqqAwcwc+ Ggd5Dx106SWo0kJ1kUTOjVhmZzvFXtz/DyHwW+iPMiUhQ4/Dc5oKq3IOfNr/SLXuYrpb UcDRobK7B0VNw40GcSgQxk3gZBsRGIa61vK0BGPvU0Nr+9kj4Jr7EKw+hqCDM0q/4uy7 kPBfgZu2xxOKzEKFrSy7Y68RJoVy1JRazqbVpPqVDaCW8y9gnA/t6baO+POMsGDoGm3m N1hydKFUJjDrirCrYHXQtf8NRFsU4vQcYhXBdYS/k3nLBSI6PII9CawdnlqTwm0rEKL6 rlvw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=b2OIEP3B; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id w6-20020a634906000000b005c2157e8483si6649167pga.231.2023.11.20.09.50.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Nov 2023 09:50:17 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=b2OIEP3B; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 737C280FA854; Mon, 20 Nov 2023 09:50:16 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233838AbjKTRuL (ORCPT + 27 others); Mon, 20 Nov 2023 12:50:11 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49546 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233605AbjKTRts (ORCPT ); Mon, 20 Nov 2023 12:49:48 -0500 Received: from out-184.mta0.migadu.com (out-184.mta0.migadu.com [IPv6:2001:41d0:1004:224b::b8]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A9CEE10DD for ; Mon, 20 Nov 2023 09:49:40 -0800 (PST) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1700502578; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AmubiAZbFJ0d/i3HqBc+VhVD2T8fru9oX5DWKdU3rlA=; b=b2OIEP3BJYVimjqmep3gX4pnHU69uhP3rz4BbIU0wDtcOHhahRQdj/mDShPRiA2HDFVZhX ObfN7eBTcN91SgmffyLxHsXabiv3RbporAg2HdC9O1DTTM3xtlUsATOXQ2XYASwgncbBwR 7K4umBHx7gSfcQI4eOh8RKYS2iCumFg= From: andrey.konovalov@linux.dev To: Andrew Morton Cc: Andrey Konovalov , Marco Elver , Alexander Potapenko , Dmitry Vyukov , Vlastimil Babka , kasan-dev@googlegroups.com, Evgenii Stepanov , Oscar Salvador , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v4 14/22] kmsan: use stack_depot_save instead of __stack_depot_save Date: Mon, 20 Nov 2023 18:47:12 +0100 Message-Id: <18092240699efdc6acd78b51e41ea782953e6c8d.1700502145.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Mon, 20 Nov 2023 09:50:16 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783106232731393576 X-GMAIL-MSGID: 1783106232731393576 From: Andrey Konovalov Make KMSAN use stack_depot_save instead of __stack_depot_save, as it always passes true to __stack_depot_save as the last argument. Reviewed-by: Alexander Potapenko Signed-off-by: Andrey Konovalov --- Changes v1->v2: - This is a new patch. --- mm/kmsan/core.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/mm/kmsan/core.c b/mm/kmsan/core.c index c19f47af0424..cf2d70e9c9a5 100644 --- a/mm/kmsan/core.c +++ b/mm/kmsan/core.c @@ -76,7 +76,7 @@ depot_stack_handle_t kmsan_save_stack_with_flags(gfp_t flags, /* Don't sleep. */ flags &= ~(__GFP_DIRECT_RECLAIM | __GFP_KSWAPD_RECLAIM); - handle = __stack_depot_save(entries, nr_entries, flags, true); + handle = stack_depot_save(entries, nr_entries, flags); return stack_depot_set_extra_bits(handle, extra); } @@ -185,11 +185,10 @@ depot_stack_handle_t kmsan_internal_chain_origin(depot_stack_handle_t id) /* * @entries is a local var in non-instrumented code, so KMSAN does not * know it is initialized. Explicitly unpoison it to avoid false - * positives when __stack_depot_save() passes it to instrumented code. + * positives when stack_depot_save() passes it to instrumented code. */ kmsan_internal_unpoison_memory(entries, sizeof(entries), false); - handle = __stack_depot_save(entries, ARRAY_SIZE(entries), __GFP_HIGH, - true); + handle = stack_depot_save(entries, ARRAY_SIZE(entries), __GFP_HIGH); return stack_depot_set_extra_bits(handle, extra_bits); } From patchwork Mon Nov 20 17:47:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 167327 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2b07:b0:403:3b70:6f57 with SMTP id io7csp107022vqb; Mon, 20 Nov 2023 09:50:24 -0800 (PST) X-Google-Smtp-Source: AGHT+IHeZ3AMptGpfYqOKZYpEocqaCiAYSfO7i+Bc/I+9E36o4gvy4zFXff0xP1qsKWQ5S1VSe3x X-Received: by 2002:a17:902:e749:b0:1cc:5db8:7eb1 with SMTP id p9-20020a170902e74900b001cc5db87eb1mr11288099plf.51.1700502623674; Mon, 20 Nov 2023 09:50:23 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700502623; cv=none; d=google.com; s=arc-20160816; b=QzbonkiV77mSXzWWsEectkxXUz0eSEThgAoFgHfmQ4A/M3/H6/tOW7v+ZQatBf/R/j Kc5YWjT45u+B8DlabzUSFdEXIJByRIEx3t0tHaBnyaz1qMwLFuElKTYoh1YHNZnj9HHA /ij166pmhMipavOW5KdlsubWnB2szZzNgcw+vdBWTCUTn3sQl3WPvkFy9YuviJkKQ9yP aF3eDZgaoUOIxxd0JdHfqYSSX/rmYSZE+8xgS7eJ0lGYQStAGDir94oYbnOn+aBQqQPN zBBWQN16Mib6N+VPG9PAj6GK2h8TZEuLhf/bLDAWLyeusTRVDSASQwF6NrOLcK3JIQLq y6eg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=aXTF/ovgO7tSDGaC/rtVyVWMqky+xGAg2LPsWIiIw38=; fh=qWoxmPN1zRqnLSoCWGpfFsDtzJsx/SGdqx98lbP+Uho=; b=1K9Fp7C5R4d4PLpMtrbFlzeaQLrkg2clZRMClFtKxbmhmET6sGIBvmZLxL597HKeP+ pzFyn2s/vL8HAohQr/6ECVgMI63eEekq893w4lnMvgpreATy32+RuaCft0DDOb6d73Fp omVWVDlcF1Ie5o8kVjOhJrzvkKNPFq21SuYYv9svUdVkJjOOH5glDHlI7tjYHysTeTYs QWqz1pUaP3Ys8XEQ3PH2uTBCkdo2OBIEUAJmpBBXqs/D6TgWGstwhbYfMf15LPSy8T+l aZE8KbQd7Kz4p29uwe2yyYByOU+XeIcxr4N1oRV7/YWhHsGKhDS8lxbed2aV9S7hEvdi iutQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=HqbyNSHD; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id ik12-20020a170902ab0c00b001bf741231f7si8261530plb.570.2023.11.20.09.50.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Nov 2023 09:50:23 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=HqbyNSHD; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id C89F580EE0DC; Mon, 20 Nov 2023 09:50:22 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233624AbjKTRuP (ORCPT + 27 others); Mon, 20 Nov 2023 12:50:15 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49314 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234309AbjKTRtu (ORCPT ); Mon, 20 Nov 2023 12:49:50 -0500 Received: from out-176.mta0.migadu.com (out-176.mta0.migadu.com [91.218.175.176]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4B36A10E6 for ; Mon, 20 Nov 2023 09:49:41 -0800 (PST) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1700502579; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aXTF/ovgO7tSDGaC/rtVyVWMqky+xGAg2LPsWIiIw38=; b=HqbyNSHD4r9vYA816UIOJaXgIqcjElz6erC2e41iS8jUsO6pNaIMU40yawD+G8CERk8/mB qdv7+Ypo8jZ0ZcLL7hUCGbmu0MDX0b6jRsskfd3hDjPvU8Dgx5Nmli1NgQFC70MO8T/FHR /N3gcwtihjoZvi89Omg8LoW5wxj9aNk= From: andrey.konovalov@linux.dev To: Andrew Morton Cc: Andrey Konovalov , Marco Elver , Alexander Potapenko , Dmitry Vyukov , Vlastimil Babka , kasan-dev@googlegroups.com, Evgenii Stepanov , Oscar Salvador , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v4 15/22] lib/stackdepot, kasan: add flags to __stack_depot_save and rename Date: Mon, 20 Nov 2023 18:47:13 +0100 Message-Id: <645fa15239621eebbd3a10331e5864b718839512.1700502145.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Mon, 20 Nov 2023 09:50:22 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783106238811228339 X-GMAIL-MSGID: 1783106238811228339 From: Andrey Konovalov Change the bool can_alloc argument of __stack_depot_save to a u32 argument that accepts a set of flags. The following patch will add another flag to stack_depot_save_flags besides the existing STACK_DEPOT_FLAG_CAN_ALLOC. Also rename the function to stack_depot_save_flags, as __stack_depot_save is a cryptic name, Reviewed-by: Alexander Potapenko Signed-off-by: Andrey Konovalov --- Changes v2->v3: - WARN_ON invalid flags in stack_depot_save_flags. Changes v1->v2: - This is a new patch. --- include/linux/stackdepot.h | 36 +++++++++++++++++++++++++----------- lib/stackdepot.c | 16 +++++++++++----- mm/kasan/common.c | 7 ++++--- mm/kasan/generic.c | 9 +++++---- mm/kasan/kasan.h | 2 +- mm/kasan/tags.c | 3 ++- 6 files changed, 48 insertions(+), 25 deletions(-) diff --git a/include/linux/stackdepot.h b/include/linux/stackdepot.h index e58306783d8e..0b262e14144e 100644 --- a/include/linux/stackdepot.h +++ b/include/linux/stackdepot.h @@ -32,6 +32,17 @@ typedef u32 depot_stack_handle_t; */ #define STACK_DEPOT_EXTRA_BITS 5 +typedef u32 depot_flags_t; + +/* + * Flags that can be passed to stack_depot_save_flags(); see the comment next + * to its declaration for more details. + */ +#define STACK_DEPOT_FLAG_CAN_ALLOC ((depot_flags_t)0x0001) + +#define STACK_DEPOT_FLAGS_NUM 1 +#define STACK_DEPOT_FLAGS_MASK ((depot_flags_t)((1 << STACK_DEPOT_FLAGS_NUM) - 1)) + /* * Using stack depot requires its initialization, which can be done in 3 ways: * @@ -69,31 +80,34 @@ static inline int stack_depot_early_init(void) { return 0; } #endif /** - * __stack_depot_save - Save a stack trace to stack depot + * stack_depot_save_flags - Save a stack trace to stack depot * * @entries: Pointer to the stack trace * @nr_entries: Number of frames in the stack * @alloc_flags: Allocation GFP flags - * @can_alloc: Allocate stack pools (increased chance of failure if false) + * @depot_flags: Stack depot flags + * + * Saves a stack trace from @entries array of size @nr_entries. * - * Saves a stack trace from @entries array of size @nr_entries. If @can_alloc is - * %true, stack depot can replenish the stack pools in case no space is left - * (allocates using GFP flags of @alloc_flags). If @can_alloc is %false, avoids - * any allocations and fails if no space is left to store the stack trace. + * If STACK_DEPOT_FLAG_CAN_ALLOC is set in @depot_flags, stack depot can + * replenish the stack pools in case no space is left (allocates using GFP + * flags of @alloc_flags). Otherwise, stack depot avoids any allocations and + * fails if no space is left to store the stack trace. * * If the provided stack trace comes from the interrupt context, only the part * up to the interrupt entry is saved. * - * Context: Any context, but setting @can_alloc to %false is required if + * Context: Any context, but setting STACK_DEPOT_FLAG_CAN_ALLOC is required if * alloc_pages() cannot be used from the current context. Currently * this is the case for contexts where neither %GFP_ATOMIC nor * %GFP_NOWAIT can be used (NMI, raw_spin_lock). * * Return: Handle of the stack struct stored in depot, 0 on failure */ -depot_stack_handle_t __stack_depot_save(unsigned long *entries, - unsigned int nr_entries, - gfp_t gfp_flags, bool can_alloc); +depot_stack_handle_t stack_depot_save_flags(unsigned long *entries, + unsigned int nr_entries, + gfp_t gfp_flags, + depot_flags_t depot_flags); /** * stack_depot_save - Save a stack trace to stack depot @@ -103,7 +117,7 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, * @alloc_flags: Allocation GFP flags * * Context: Contexts where allocations via alloc_pages() are allowed. - * See __stack_depot_save() for more details. + * See stack_depot_save_flags() for more details. * * Return: Handle of the stack trace stored in depot, 0 on failure */ diff --git a/lib/stackdepot.c b/lib/stackdepot.c index 4bb0af423f82..59d61d5c09a7 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -450,19 +450,24 @@ static inline struct stack_record *find_stack(struct list_head *bucket, return NULL; } -depot_stack_handle_t __stack_depot_save(unsigned long *entries, - unsigned int nr_entries, - gfp_t alloc_flags, bool can_alloc) +depot_stack_handle_t stack_depot_save_flags(unsigned long *entries, + unsigned int nr_entries, + gfp_t alloc_flags, + depot_flags_t depot_flags) { struct list_head *bucket; struct stack_record *found = NULL; depot_stack_handle_t handle = 0; struct page *page = NULL; void *prealloc = NULL; + bool can_alloc = depot_flags & STACK_DEPOT_FLAG_CAN_ALLOC; bool need_alloc = false; unsigned long flags; u32 hash; + if (WARN_ON(depot_flags & ~STACK_DEPOT_FLAGS_MASK)) + return 0; + /* * If this stack trace is from an interrupt, including anything before * interrupt entry usually leads to unbounded stack depot growth. @@ -541,13 +546,14 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, handle = found->handle.handle; return handle; } -EXPORT_SYMBOL_GPL(__stack_depot_save); +EXPORT_SYMBOL_GPL(stack_depot_save_flags); depot_stack_handle_t stack_depot_save(unsigned long *entries, unsigned int nr_entries, gfp_t alloc_flags) { - return __stack_depot_save(entries, nr_entries, alloc_flags, true); + return stack_depot_save_flags(entries, nr_entries, alloc_flags, + STACK_DEPOT_FLAG_CAN_ALLOC); } EXPORT_SYMBOL_GPL(stack_depot_save); diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 256930da578a..825a0240ec02 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -22,6 +22,7 @@ #include #include #include +#include #include #include #include @@ -37,19 +38,19 @@ struct slab *kasan_addr_to_slab(const void *addr) return NULL; } -depot_stack_handle_t kasan_save_stack(gfp_t flags, bool can_alloc) +depot_stack_handle_t kasan_save_stack(gfp_t flags, depot_flags_t depot_flags) { unsigned long entries[KASAN_STACK_DEPTH]; unsigned int nr_entries; nr_entries = stack_trace_save(entries, ARRAY_SIZE(entries), 0); - return __stack_depot_save(entries, nr_entries, flags, can_alloc); + return stack_depot_save_flags(entries, nr_entries, flags, depot_flags); } void kasan_set_track(struct kasan_track *track, gfp_t flags) { track->pid = current->pid; - track->stack = kasan_save_stack(flags, true); + track->stack = kasan_save_stack(flags, STACK_DEPOT_FLAG_CAN_ALLOC); } #if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) diff --git a/mm/kasan/generic.c b/mm/kasan/generic.c index 4d837ab83f08..5d168c9afb32 100644 --- a/mm/kasan/generic.c +++ b/mm/kasan/generic.c @@ -25,6 +25,7 @@ #include #include #include +#include #include #include #include @@ -472,7 +473,7 @@ size_t kasan_metadata_size(struct kmem_cache *cache, bool in_object) sizeof(struct kasan_free_meta) : 0); } -static void __kasan_record_aux_stack(void *addr, bool can_alloc) +static void __kasan_record_aux_stack(void *addr, depot_flags_t depot_flags) { struct slab *slab = kasan_addr_to_slab(addr); struct kmem_cache *cache; @@ -489,17 +490,17 @@ static void __kasan_record_aux_stack(void *addr, bool can_alloc) return; alloc_meta->aux_stack[1] = alloc_meta->aux_stack[0]; - alloc_meta->aux_stack[0] = kasan_save_stack(0, can_alloc); + alloc_meta->aux_stack[0] = kasan_save_stack(0, depot_flags); } void kasan_record_aux_stack(void *addr) { - return __kasan_record_aux_stack(addr, true); + return __kasan_record_aux_stack(addr, STACK_DEPOT_FLAG_CAN_ALLOC); } void kasan_record_aux_stack_noalloc(void *addr) { - return __kasan_record_aux_stack(addr, false); + return __kasan_record_aux_stack(addr, 0); } void kasan_save_alloc_info(struct kmem_cache *cache, void *object, gfp_t flags) diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 8b06bab5c406..b29d46b83d1f 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -368,7 +368,7 @@ static inline void kasan_init_cache_meta(struct kmem_cache *cache, unsigned int static inline void kasan_init_object_meta(struct kmem_cache *cache, const void *object) { } #endif -depot_stack_handle_t kasan_save_stack(gfp_t flags, bool can_alloc); +depot_stack_handle_t kasan_save_stack(gfp_t flags, depot_flags_t depot_flags); void kasan_set_track(struct kasan_track *track, gfp_t flags); void kasan_save_alloc_info(struct kmem_cache *cache, void *object, gfp_t flags); void kasan_save_free_info(struct kmem_cache *cache, void *object); diff --git a/mm/kasan/tags.c b/mm/kasan/tags.c index 7dcfe341d48e..4fd32121b0fd 100644 --- a/mm/kasan/tags.c +++ b/mm/kasan/tags.c @@ -13,6 +13,7 @@ #include #include #include +#include #include #include #include @@ -101,7 +102,7 @@ static void save_stack_info(struct kmem_cache *cache, void *object, struct kasan_stack_ring_entry *entry; void *old_ptr; - stack = kasan_save_stack(gfp_flags, true); + stack = kasan_save_stack(gfp_flags, STACK_DEPOT_FLAG_CAN_ALLOC); /* * Prevent save_stack_info() from modifying stack ring From patchwork Mon Nov 20 17:47:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 167328 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2b07:b0:403:3b70:6f57 with SMTP id io7csp107477vqb; Mon, 20 Nov 2023 09:51:11 -0800 (PST) X-Google-Smtp-Source: AGHT+IGMgnDg4F80FkaOcRYeEb2M8N8Qnav1sz3nn2gytbAjo0gzhzt4IWsLRXCYEnMUjBYBDtyw X-Received: by 2002:a05:6a00:3012:b0:68f:c078:b0c9 with SMTP id ay18-20020a056a00301200b0068fc078b0c9mr268246pfb.11.1700502670702; Mon, 20 Nov 2023 09:51:10 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700502670; cv=none; d=google.com; s=arc-20160816; b=u8dyWzsEozLaNe3qTCEE+PHHrmFXLDY8sXdVxrYtY8wJiCbCKVN5/fmpm4LY1SEGfs iwZaoav9ftDS7hsuZ1h2cIligYzYGgM+ModfLq+nWjYbptE0LNGct2lFEYy41Ec/MngU chinDNQHpIn03NCAFVpqCcnsky0UBvI0BbH7SXBHjhLE+pSuxZyVeHF9dCe03GLOdqlb tB23PC0neBUWIq6rTLw+AbrsbEW19jRKjN6yCTpNZus6kzJAcobcdBnm2LgmSmFlAyLy /6s8sj2px9YXDWMM7K9Fb2U95rSfLrSFhhSQCShjfgJ8bSE0iHmbnj0/PrX1GzG5gqOw Z8CQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=91QxaYvk4RHHEJwI4Sc7Oo7z2rlFVPgOoXOer3eAY+g=; fh=qWoxmPN1zRqnLSoCWGpfFsDtzJsx/SGdqx98lbP+Uho=; b=ICzDhzF4sMXQYv0ydbHz5yeKLUuCApdVz0YvPbuP0CvALyC0oRQHaFncIrlNAVLYIb fvoxWYDyC2lVvA1ZH5eR8Oj6E5upCD1fHSvUbNxDsMjjcPsjUaQSYOEzgpSPDsX3Trl5 WJeXyYzP9Oe/EhkM69HKTRabRVRC+ioOFOLMX1ZzqwMwxrcRS7Szb8ez4H4fjB4Wv9ww u/hoXo9pmzelnGZBkAUIGZmKeI3dDyCZM7bbuYcTVo3XdnuqMb422tUKBF+fJnh9xBY5 qnE0Xd4FSVMOG/WoK+rDoaRZ7eeJegaDI9wZO4zsDLsFaWhHQ72A0kLaIEfYal/WxMB0 pxbg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b="rZ/C2Hcs"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from lipwig.vger.email (lipwig.vger.email. [2620:137:e000::3:3]) by mx.google.com with ESMTPS id u24-20020a63df18000000b005c1ccaca5c4si8554619pgg.873.2023.11.20.09.51.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Nov 2023 09:51:10 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) client-ip=2620:137:e000::3:3; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b="rZ/C2Hcs"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id 41439808A41E; Mon, 20 Nov 2023 09:50:30 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232158AbjKTRuT (ORCPT + 27 others); Mon, 20 Nov 2023 12:50:19 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49182 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234647AbjKTRtw (ORCPT ); Mon, 20 Nov 2023 12:49:52 -0500 Received: from out-181.mta0.migadu.com (out-181.mta0.migadu.com [91.218.175.181]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DA9B110F6 for ; Mon, 20 Nov 2023 09:49:41 -0800 (PST) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1700502580; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=91QxaYvk4RHHEJwI4Sc7Oo7z2rlFVPgOoXOer3eAY+g=; b=rZ/C2HcsNzvMwpX5O9VOx1lNSC2GHR0kSTsgEzMvFKq6y+hKBKp3/9Ua+zCoBCJmYeieov wMbZyfFLCbQY6lOshDlPQuKUW5//srMlo9hBj2CFZiAjPs39TBjkS5qlASK6p+8fCOmgfm q8MQh7TtmnvvOGnA2P/0s6mRGFe8vjU= From: andrey.konovalov@linux.dev To: Andrew Morton Cc: Andrey Konovalov , Marco Elver , Alexander Potapenko , Dmitry Vyukov , Vlastimil Babka , kasan-dev@googlegroups.com, Evgenii Stepanov , Oscar Salvador , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v4 16/22] lib/stackdepot: add refcount for records Date: Mon, 20 Nov 2023 18:47:14 +0100 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Mon, 20 Nov 2023 09:50:30 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783106288482045912 X-GMAIL-MSGID: 1783106288482045912 From: Andrey Konovalov Add a reference counter for how many times a stack records has been added to stack depot. Add a new STACK_DEPOT_FLAG_GET flag to stack_depot_save_flags that instructs the stack depot to increment the refcount. Do not yet decrement the refcount; this is implemented in one of the following patches. Do not yet enable any users to use the flag to avoid overflowing the refcount. This is preparatory patch for implementing the eviction of stack records from the stack depot. Reviewed-by: Alexander Potapenko Signed-off-by: Andrey Konovalov --- Changes v1->v2: - Add forgotten refcount_inc() under write lock. - Add STACK_DEPOT_FLAG_GET flag for stack_depot_save_flags. --- include/linux/stackdepot.h | 13 ++++++++++--- lib/stackdepot.c | 12 ++++++++++-- 2 files changed, 20 insertions(+), 5 deletions(-) diff --git a/include/linux/stackdepot.h b/include/linux/stackdepot.h index 0b262e14144e..611716702d73 100644 --- a/include/linux/stackdepot.h +++ b/include/linux/stackdepot.h @@ -39,8 +39,9 @@ typedef u32 depot_flags_t; * to its declaration for more details. */ #define STACK_DEPOT_FLAG_CAN_ALLOC ((depot_flags_t)0x0001) +#define STACK_DEPOT_FLAG_GET ((depot_flags_t)0x0002) -#define STACK_DEPOT_FLAGS_NUM 1 +#define STACK_DEPOT_FLAGS_NUM 2 #define STACK_DEPOT_FLAGS_MASK ((depot_flags_t)((1 << STACK_DEPOT_FLAGS_NUM) - 1)) /* @@ -94,6 +95,9 @@ static inline int stack_depot_early_init(void) { return 0; } * flags of @alloc_flags). Otherwise, stack depot avoids any allocations and * fails if no space is left to store the stack trace. * + * If STACK_DEPOT_FLAG_GET is set in @depot_flags, stack depot will increment + * the refcount on the saved stack trace if it already exists in stack depot. + * * If the provided stack trace comes from the interrupt context, only the part * up to the interrupt entry is saved. * @@ -116,8 +120,11 @@ depot_stack_handle_t stack_depot_save_flags(unsigned long *entries, * @nr_entries: Number of frames in the stack * @alloc_flags: Allocation GFP flags * - * Context: Contexts where allocations via alloc_pages() are allowed. - * See stack_depot_save_flags() for more details. + * Does not increment the refcount on the saved stack trace; see + * stack_depot_save_flags() for more details. + * + * Context: Contexts where allocations via alloc_pages() are allowed; + * see stack_depot_save_flags() for more details. * * Return: Handle of the stack trace stored in depot, 0 on failure */ diff --git a/lib/stackdepot.c b/lib/stackdepot.c index 59d61d5c09a7..911dee11bf39 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -23,6 +23,7 @@ #include #include #include +#include #include #include #include @@ -60,6 +61,7 @@ struct stack_record { u32 hash; /* Hash in hash table */ u32 size; /* Number of stored frames */ union handle_parts handle; + refcount_t count; unsigned long entries[CONFIG_STACKDEPOT_MAX_FRAMES]; /* Frames */ }; @@ -373,6 +375,7 @@ depot_alloc_stack(unsigned long *entries, int size, u32 hash, void **prealloc) stack->hash = hash; stack->size = size; /* stack->handle is already filled in by depot_init_pool(). */ + refcount_set(&stack->count, 1); memcpy(stack->entries, entries, flex_array_size(stack, entries, size)); /* @@ -489,6 +492,8 @@ depot_stack_handle_t stack_depot_save_flags(unsigned long *entries, /* Fast path: look the stack trace up without full locking. */ found = find_stack(bucket, entries, nr_entries, hash); if (found) { + if (depot_flags & STACK_DEPOT_FLAG_GET) + refcount_inc(&found->count); read_unlock_irqrestore(&pool_rwlock, flags); goto exit; } @@ -528,12 +533,15 @@ depot_stack_handle_t stack_depot_save_flags(unsigned long *entries, list_add(&new->list, bucket); found = new; } - } else if (prealloc) { + } else { + if (depot_flags & STACK_DEPOT_FLAG_GET) + refcount_inc(&found->count); /* * Stack depot already contains this stack trace, but let's * keep the preallocated memory for future. */ - depot_keep_new_pool(&prealloc); + if (prealloc) + depot_keep_new_pool(&prealloc); } write_unlock_irqrestore(&pool_rwlock, flags); From patchwork Mon Nov 20 17:47:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 167329 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2b07:b0:403:3b70:6f57 with SMTP id io7csp108004vqb; Mon, 20 Nov 2023 09:52:12 -0800 (PST) X-Google-Smtp-Source: AGHT+IGJUrN0zun7q25OnEUt3m+5J0XGQhnILc/xR1HluLY7yhbnJfBKeRQKmhAIgoSfl+T7RMRj X-Received: by 2002:a17:90b:1bcc:b0:27d:75f2:a3ee with SMTP id oa12-20020a17090b1bcc00b0027d75f2a3eemr5679671pjb.10.1700502732362; Mon, 20 Nov 2023 09:52:12 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700502732; cv=none; d=google.com; s=arc-20160816; b=hpQJQWvsXOWfRhnRxz9JUJJZTbhktXrSIySEFhvoezWQM6VF2VduGjIDKFG2Cf6mIL 47Wvk+AVtQihMP+x3YoilVAo07Ntk3zMeqNDR4pmCvp3lT278dDegE8XwBgSvH9Bk8TZ f7GO86xFGxBdOIgRkI7rOvptucPtHhsAxpZP7l50Y0/7SCiDfv7VTBSg8JUZPAbWfXbF Zmx7fbnQHcSWXA1zMJULqFBXlFh7sHV9HmDI15Bh7hgAeyN8lcpku1ENIh7aRxT76Cvr 6VAnNTH2aYTB8RIcislVVChK0s0Ds4kakdMJTrI+9ztpeHUa7zdRpshG4aWHhPfTUOCq qOEQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=taQhm6+sqq7C99vuNmST7zKoEFBrlbi2ArfSWbY49ok=; fh=qWoxmPN1zRqnLSoCWGpfFsDtzJsx/SGdqx98lbP+Uho=; b=H4pgyyHkRM8/YEd0ikhJD9AawhOFDRwYbOCZ5X/GVJqy71WWwM88NgGyfknCUHBK8E m9NjuDCL7SgI1UcsW5z3+ShSmZJnVoc2MtEnqimBKOgbtm/BHNe8ESQ4Pm69koxl4sXj z6hL7dmxvnj5KH1foHRniIsLd3X4E+90l0GYnrsHpGil8zW8lLSiNeQrGkKAGi7RExRG /ZQ7rDYONruAYsXAZ8tWfR/NSRDvsS4tPjSBkismQHb67W0RbSX8Ib81zQqRmUkUpMUo DYsWkZW1XnAnWYX5+m6x9/4BzH7yQujlp0DzBvzLNsnRcrmt+jIHiLgW7v3vE1boi67Z Xtdw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=DLuotPQZ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from fry.vger.email (fry.vger.email. [2620:137:e000::3:8]) by mx.google.com with ESMTPS id q2-20020a17090a304200b00283982a4be5si8030930pjl.106.2023.11.20.09.52.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Nov 2023 09:52:12 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) client-ip=2620:137:e000::3:8; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=DLuotPQZ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by fry.vger.email (Postfix) with ESMTP id 52DAF8045666; Mon, 20 Nov 2023 09:51:56 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at fry.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234694AbjKTRu0 (ORCPT + 27 others); Mon, 20 Nov 2023 12:50:26 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39782 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234651AbjKTRtw (ORCPT ); Mon, 20 Nov 2023 12:49:52 -0500 Received: from out-170.mta0.migadu.com (out-170.mta0.migadu.com [IPv6:2001:41d0:1004:224b::aa]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C6E621708 for ; Mon, 20 Nov 2023 09:49:42 -0800 (PST) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1700502580; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=taQhm6+sqq7C99vuNmST7zKoEFBrlbi2ArfSWbY49ok=; b=DLuotPQZ2ya2ze/Uk9qdks6Velp5g/ufNOpAnZ3F0CTG4bBuTmqBJZX+8HrA4Dqfb+h/XH cXG4e43rc7TxqEGmM94O/sylDTL5wKtysxQVD2NH08lAvJQHMJZz91DKB754x+wRKfj68K mQIu1A/BqCWkTls4JakGGQmnxTCxoCg= From: andrey.konovalov@linux.dev To: Andrew Morton Cc: Andrey Konovalov , Marco Elver , Alexander Potapenko , Dmitry Vyukov , Vlastimil Babka , kasan-dev@googlegroups.com, Evgenii Stepanov , Oscar Salvador , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v4 17/22] lib/stackdepot: allow users to evict stack traces Date: Mon, 20 Nov 2023 18:47:15 +0100 Message-Id: <1d1ad5692ee43d4fc2b3fd9d221331d30b36123f.1700502145.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on fry.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (fry.vger.email [0.0.0.0]); Mon, 20 Nov 2023 09:51:56 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783106353463515163 X-GMAIL-MSGID: 1783106353463515163 From: Andrey Konovalov Add stack_depot_put, a function that decrements the reference counter on a stack record and removes it from the stack depot once the counter reaches 0. Internally, when removing a stack record, the function unlinks it from the hash table bucket and returns to the freelist. With this change, the users of stack depot can call stack_depot_put when keeping a stack trace in the stack depot is not needed anymore. This allows avoiding polluting the stack depot with irrelevant stack traces and thus have more space to store the relevant ones before the stack depot reaches its capacity. Signed-off-by: Andrey Konovalov --- Changes v1->v2: - Comments fixes as suggested by Marco. - Add lockdep_assert annotation. - Adapt to using list_head's. - Rename stack_depot_evict to stack_depot_put. --- include/linux/stackdepot.h | 14 ++++++++++++++ lib/stackdepot.c | 37 ++++++++++++++++++++++++++++++++++++- 2 files changed, 50 insertions(+), 1 deletion(-) diff --git a/include/linux/stackdepot.h b/include/linux/stackdepot.h index 611716702d73..a6796f178913 100644 --- a/include/linux/stackdepot.h +++ b/include/linux/stackdepot.h @@ -97,6 +97,8 @@ static inline int stack_depot_early_init(void) { return 0; } * * If STACK_DEPOT_FLAG_GET is set in @depot_flags, stack depot will increment * the refcount on the saved stack trace if it already exists in stack depot. + * Users of this flag must also call stack_depot_put() when keeping the stack + * trace is no longer required to avoid overflowing the refcount. * * If the provided stack trace comes from the interrupt context, only the part * up to the interrupt entry is saved. @@ -162,6 +164,18 @@ void stack_depot_print(depot_stack_handle_t stack); int stack_depot_snprint(depot_stack_handle_t handle, char *buf, size_t size, int spaces); +/** + * stack_depot_put - Drop a reference to a stack trace from stack depot + * + * @handle: Stack depot handle returned from stack_depot_save() + * + * The stack trace is evicted from stack depot once all references to it have + * been dropped (once the number of stack_depot_evict() calls matches the + * number of stack_depot_save_flags() calls with STACK_DEPOT_FLAG_GET set for + * this stack trace). + */ +void stack_depot_put(depot_stack_handle_t handle); + /** * stack_depot_set_extra_bits - Set extra bits in a stack depot handle * diff --git a/lib/stackdepot.c b/lib/stackdepot.c index 911dee11bf39..c1b31160f4b4 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -394,7 +394,7 @@ static struct stack_record *depot_fetch_stack(depot_stack_handle_t handle) size_t offset = parts.offset << DEPOT_STACK_ALIGN; struct stack_record *stack; - lockdep_assert_held_read(&pool_rwlock); + lockdep_assert_held(&pool_rwlock); if (parts.pool_index > pools_num) { WARN(1, "pool index %d out of bounds (%d) for stack id %08x\n", @@ -410,6 +410,14 @@ static struct stack_record *depot_fetch_stack(depot_stack_handle_t handle) return stack; } +/* Links stack into the freelist. */ +static void depot_free_stack(struct stack_record *stack) +{ + lockdep_assert_held_write(&pool_rwlock); + + list_add(&stack->list, &free_stacks); +} + /* Calculates the hash for a stack. */ static inline u32 hash_stack(unsigned long *entries, unsigned int size) { @@ -592,6 +600,33 @@ unsigned int stack_depot_fetch(depot_stack_handle_t handle, } EXPORT_SYMBOL_GPL(stack_depot_fetch); +void stack_depot_put(depot_stack_handle_t handle) +{ + struct stack_record *stack; + unsigned long flags; + + if (!handle || stack_depot_disabled) + return; + + write_lock_irqsave(&pool_rwlock, flags); + + stack = depot_fetch_stack(handle); + if (WARN_ON(!stack)) + goto out; + + if (refcount_dec_and_test(&stack->count)) { + /* Unlink stack from the hash table. */ + list_del(&stack->list); + + /* Free stack. */ + depot_free_stack(stack); + } + +out: + write_unlock_irqrestore(&pool_rwlock, flags); +} +EXPORT_SYMBOL_GPL(stack_depot_put); + void stack_depot_print(depot_stack_handle_t stack) { unsigned long *entries; From patchwork Mon Nov 20 17:47:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 167332 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2b07:b0:403:3b70:6f57 with SMTP id io7csp108276vqb; Mon, 20 Nov 2023 09:52:42 -0800 (PST) X-Google-Smtp-Source: AGHT+IFpDAQm0yD93T37h+XzL9NsP6Wt8ryC+VqF8JeQvl55JtxFJKIiLH8ZWB5oUwa5U89poNw0 X-Received: by 2002:a17:90b:4a49:b0:27c:f1ca:f8a6 with SMTP id lb9-20020a17090b4a4900b0027cf1caf8a6mr9255521pjb.30.1700502762334; Mon, 20 Nov 2023 09:52:42 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700502762; cv=none; d=google.com; s=arc-20160816; b=Om022wfvCeWtgQX8uUQX0tqFZdDMOiNj2jC4BaSPlViH0XtW3jNr+L4a1kuWuiREsz oOT+l/+LaY4AqKPOf/6xYTVxfz1Magr9heKUeRFv0uBbISWjIVOjyPV0ECEmQArSBrQP r6wVRtGpw1GryPFq2n2Buz8Skg0bqFlBns16FUcQfzjbICLslh9/Fx15tMdQDVixRKCD K+zayAOEGxutCHmkAlZlkt72PErGFJ34CW3Qg+tfslyJ4UxMRjjdcEh0bIxRbsk+5eaA 1RJ2A2+DiQcYH5rFI0Wk/LzLr0ndYZUWlFjMK9IFFp9oEK6hhyO5fk7vbOl+LEZ9PKsZ A6Mw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=h9lSbnFVvjj2GZw0LrViVCVgXWgHHPxOTEIrM79fNvQ=; fh=qWoxmPN1zRqnLSoCWGpfFsDtzJsx/SGdqx98lbP+Uho=; b=Wck+gV0TcUjC4t1khSEhPFlT3jqRh81SFcmK7XXGEEbYMTTJMIt0f1JJt1Gwn37IdN eYL2x28pEt9B8xDseU/YxMPaqtkSgeXGlWYRsdg0UJzLSu7mlEq2htOoL+OXyt1UJD9D znw/B5g/Ka7A1Kf5P29PS4Nd3dqiv/fYOOd4MJWTu5tiuBSPJt8hbRLdGes7esszvHC+ 2x+1EitWR5gYSDsE1EHC51q8tA7scFlqNMEcCuzVem0WFwxojxVVPy8aO0i6ANhF6qAg QVN51Aj5bc5tCNnOW+xWbc7Hi4EYoQhOFGRJsXneHOE/PDQXdiBLekeZekThJ5m61tbZ WHsg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b="altZ3Y/X"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from fry.vger.email (fry.vger.email. [23.128.96.38]) by mx.google.com with ESMTPS id a19-20020a17090a8c1300b0028307e08316si8317628pjo.156.2023.11.20.09.52.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Nov 2023 09:52:42 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) client-ip=23.128.96.38; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b="altZ3Y/X"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by fry.vger.email (Postfix) with ESMTP id 7A6AB807F49A; Mon, 20 Nov 2023 09:52:35 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at fry.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234249AbjKTRvv (ORCPT + 27 others); Mon, 20 Nov 2023 12:51:51 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50018 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234261AbjKTRvW (ORCPT ); Mon, 20 Nov 2023 12:51:22 -0500 Received: from out-177.mta1.migadu.com (out-177.mta1.migadu.com [IPv6:2001:41d0:203:375::b1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 07FB5E3 for ; Mon, 20 Nov 2023 09:50:44 -0800 (PST) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1700502642; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=h9lSbnFVvjj2GZw0LrViVCVgXWgHHPxOTEIrM79fNvQ=; b=altZ3Y/XI4JhIu5hrnCRXA2Lh0EqKL+D8A/MAbLwDHf+SQx3XdzFWHdK500Z2nrgBAdsMJ CIl8i/FRs6CobGUwDDY8obdvF4axa9Jsyq1WLFLLkkvZ3UoyLWEgpJBac7eWWb07xxbTHj /N4UEr5XJVfvRHr9tfyWkhDs5sCw/8o= From: andrey.konovalov@linux.dev To: Andrew Morton Cc: Andrey Konovalov , Marco Elver , Alexander Potapenko , Dmitry Vyukov , Vlastimil Babka , kasan-dev@googlegroups.com, Evgenii Stepanov , Oscar Salvador , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v4 18/22] kasan: remove atomic accesses to stack ring entries Date: Mon, 20 Nov 2023 18:47:16 +0100 Message-Id: <29f59126d9845c5257b6c29cd7ad113b16f19f47.1700502145.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on fry.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (fry.vger.email [0.0.0.0]); Mon, 20 Nov 2023 09:52:35 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783106384513361822 X-GMAIL-MSGID: 1783106384513361822 From: Andrey Konovalov Remove the atomic accesses to entry fields in save_stack_info and kasan_complete_mode_report_info for tag-based KASAN modes. These atomics are not required, as the read/write lock prevents the entries from being read (in kasan_complete_mode_report_info) while being written (in save_stack_info) and the try_cmpxchg prevents the same entry from being rewritten (in save_stack_info) in the unlikely case of wrapping during writing. Reviewed-by: Alexander Potapenko Signed-off-by: Andrey Konovalov --- Changes v1->v2: - This is a new patch. --- mm/kasan/report_tags.c | 25 +++++++------------------ mm/kasan/tags.c | 13 +++++-------- 2 files changed, 12 insertions(+), 26 deletions(-) diff --git a/mm/kasan/report_tags.c b/mm/kasan/report_tags.c index 8b8bfdb3cfdb..78abdcde5da9 100644 --- a/mm/kasan/report_tags.c +++ b/mm/kasan/report_tags.c @@ -31,10 +31,6 @@ void kasan_complete_mode_report_info(struct kasan_report_info *info) unsigned long flags; u64 pos; struct kasan_stack_ring_entry *entry; - void *ptr; - u32 pid; - depot_stack_handle_t stack; - bool is_free; bool alloc_found = false, free_found = false; if ((!info->cache || !info->object) && !info->bug_type) { @@ -61,18 +57,11 @@ void kasan_complete_mode_report_info(struct kasan_report_info *info) entry = &stack_ring.entries[i % stack_ring.size]; - /* Paired with smp_store_release() in save_stack_info(). */ - ptr = (void *)smp_load_acquire(&entry->ptr); - - if (kasan_reset_tag(ptr) != info->object || - get_tag(ptr) != get_tag(info->access_addr)) + if (kasan_reset_tag(entry->ptr) != info->object || + get_tag(entry->ptr) != get_tag(info->access_addr)) continue; - pid = READ_ONCE(entry->pid); - stack = READ_ONCE(entry->stack); - is_free = READ_ONCE(entry->is_free); - - if (is_free) { + if (entry->is_free) { /* * Second free of the same object. * Give up on trying to find the alloc entry. @@ -80,8 +69,8 @@ void kasan_complete_mode_report_info(struct kasan_report_info *info) if (free_found) break; - info->free_track.pid = pid; - info->free_track.stack = stack; + info->free_track.pid = entry->pid; + info->free_track.stack = entry->stack; free_found = true; /* @@ -95,8 +84,8 @@ void kasan_complete_mode_report_info(struct kasan_report_info *info) if (alloc_found) break; - info->alloc_track.pid = pid; - info->alloc_track.stack = stack; + info->alloc_track.pid = entry->pid; + info->alloc_track.stack = entry->stack; alloc_found = true; /* diff --git a/mm/kasan/tags.c b/mm/kasan/tags.c index 4fd32121b0fd..b6c017e670d8 100644 --- a/mm/kasan/tags.c +++ b/mm/kasan/tags.c @@ -121,15 +121,12 @@ static void save_stack_info(struct kmem_cache *cache, void *object, if (!try_cmpxchg(&entry->ptr, &old_ptr, STACK_RING_BUSY_PTR)) goto next; /* Busy slot. */ - WRITE_ONCE(entry->size, cache->object_size); - WRITE_ONCE(entry->pid, current->pid); - WRITE_ONCE(entry->stack, stack); - WRITE_ONCE(entry->is_free, is_free); + entry->size = cache->object_size; + entry->pid = current->pid; + entry->stack = stack; + entry->is_free = is_free; - /* - * Paired with smp_load_acquire() in kasan_complete_mode_report_info(). - */ - smp_store_release(&entry->ptr, (s64)object); + entry->ptr = object; read_unlock_irqrestore(&stack_ring.lock, flags); } From patchwork Mon Nov 20 17:47:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 167331 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2b07:b0:403:3b70:6f57 with SMTP id io7csp108221vqb; Mon, 20 Nov 2023 09:52:37 -0800 (PST) X-Google-Smtp-Source: AGHT+IFGD5wkc4x+CdHA+HOkHW9fUWCbpww5kV6hi31m+EbL1enaOkrURonFKd3h1hfXQ9IT4xv9 X-Received: by 2002:a05:6a21:6d9b:b0:187:5302:4b20 with SMTP id wl27-20020a056a216d9b00b0018753024b20mr7445334pzb.41.1700502756692; Mon, 20 Nov 2023 09:52:36 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700502756; cv=none; d=google.com; s=arc-20160816; b=iPLiD/lwddy12OVWKhkSaORyRJigELyvez+ICitrhIANRbQadv/++L7hEBpzLmMsJG hYHhw3GNuE8zVcsvx/P9VagF2R1SxaL+jpaiayA1+e22A6xN8PnNaDnn2Oxqm2vd1ayF 85h9rJNkeCogKUNXMepGfgvMLsYXFHPE9tcjM6gI6sJrFyVUu0GJOfOdKt1s9dHqV+AJ 1iZQr/w6VasGNR3cw8RJ4ZHzGXZ+aJqHE2NXjUIH54WmS6wvCE+P6KlMwEjt7wFqWJ+c 20SQRHtnVTALVYu9TG9t/wHgGRiDK0kFez0GHNgMouwSdCSn/l0sA6G8bU2MbG0voxJs q3sg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=uhMrVBrzgAoT1gwViC8KxLkH9OMZhix/uhvoEc70W98=; fh=qWoxmPN1zRqnLSoCWGpfFsDtzJsx/SGdqx98lbP+Uho=; b=SGF/aka9fYd6MctDvjFktR/ZpMnm6vAAODKUe0sdtWWBw0coXv+8i8sjvlNa5Wj6b1 //WUezRXNsOEksV/Tnx6VeTTo3cfeiELI4ma3YgqaKMAV3zIk5BAS6LGYsbSksKy49eq oizZgg7e1Sc4AsWFQY3dr8tg1gXvXXkMA/ZlolDstyCz6ab07klt/AfzvMC5Zmxpn/No 7Ay1Hp/WYKp57tcTD2yEbQd1aHPCMsb+xuJfGYDoCmslDG7P6XNayF0QO/4fmirwhV3w 8cdjMt6SVsAwhRdlx0JsTlOXe92PBkRWBysKuyw4aaD25W3669AX1JlWHE+unewoXvXw O3Xg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=lATFUJLN; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from agentk.vger.email (agentk.vger.email. [23.128.96.32]) by mx.google.com with ESMTPS id u128-20020a627986000000b006935df301a3si8213236pfc.8.2023.11.20.09.52.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Nov 2023 09:52:36 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) client-ip=23.128.96.32; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=lATFUJLN; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id 38CE8803D8C8; Mon, 20 Nov 2023 09:52:32 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231132AbjKTRv4 (ORCPT + 27 others); Mon, 20 Nov 2023 12:51:56 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49594 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234295AbjKTRvX (ORCPT ); Mon, 20 Nov 2023 12:51:23 -0500 Received: from out-182.mta1.migadu.com (out-182.mta1.migadu.com [95.215.58.182]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 73EDED45 for ; Mon, 20 Nov 2023 09:50:45 -0800 (PST) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1700502643; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uhMrVBrzgAoT1gwViC8KxLkH9OMZhix/uhvoEc70W98=; b=lATFUJLNYsof+QZ6G2sBXsnC9Q+wIKDyc8lFVJDjx9RK2nGA9FoeQs6kQO+6hjM8HBefIP eURADnY2Rf+gcAsebe+mW3WuHxDxbtNbYaSjhKQqLHNAk7mVow4Kb9CTZdG5zssygETF+r BwY3IBxnt51rdnlRs7VhwlO0TNcSVR8= From: andrey.konovalov@linux.dev To: Andrew Morton Cc: Andrey Konovalov , Marco Elver , Alexander Potapenko , Dmitry Vyukov , Vlastimil Babka , kasan-dev@googlegroups.com, Evgenii Stepanov , Oscar Salvador , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v4 19/22] kasan: check object_size in kasan_complete_mode_report_info Date: Mon, 20 Nov 2023 18:47:17 +0100 Message-Id: <68c6948175aadd7e7e7deea61725103d64a4528f.1700502145.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Mon, 20 Nov 2023 09:52:32 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783106378400463305 X-GMAIL-MSGID: 1783106378400463305 From: Andrey Konovalov Check the object size when looking up entries in the stack ring. If the size of the object for which a report is being printed does not match the size of the object for which a stack trace has been saved in the stack ring, the saved stack trace is irrelevant. Reviewed-by: Alexander Potapenko Signed-off-by: Andrey Konovalov --- Changes v2->v3: - Added missing "../slab.h" include for accessing a kmem_cache field. Changes v1->v2: - This is a new patch. --- mm/kasan/report_tags.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/mm/kasan/report_tags.c b/mm/kasan/report_tags.c index 78abdcde5da9..55154743f915 100644 --- a/mm/kasan/report_tags.c +++ b/mm/kasan/report_tags.c @@ -7,6 +7,7 @@ #include #include "kasan.h" +#include "../slab.h" extern struct kasan_stack_ring stack_ring; @@ -58,7 +59,8 @@ void kasan_complete_mode_report_info(struct kasan_report_info *info) entry = &stack_ring.entries[i % stack_ring.size]; if (kasan_reset_tag(entry->ptr) != info->object || - get_tag(entry->ptr) != get_tag(info->access_addr)) + get_tag(entry->ptr) != get_tag(info->access_addr) || + info->cache->object_size != entry->size) continue; if (entry->is_free) { From patchwork Mon Nov 20 17:47:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 167334 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2b07:b0:403:3b70:6f57 with SMTP id io7csp108423vqb; Mon, 20 Nov 2023 09:52:58 -0800 (PST) X-Google-Smtp-Source: AGHT+IER6xh4w1JMi8MSKOIJmrBu82Ny0jtEWvt9A4hOL/ByYYVQAQYYBTYDgaVzKNoEtZC5TMPx X-Received: by 2002:a05:6a20:5d8f:b0:188:75c:e6a2 with SMTP id km15-20020a056a205d8f00b00188075ce6a2mr4568234pzb.1.1700502777838; Mon, 20 Nov 2023 09:52:57 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700502777; cv=none; d=google.com; s=arc-20160816; b=Gbb1CJPwD/Z2szQoaJrhTzy5vxhja/dt2jwReKmirthKZ8Z62RtyROQ1kbP86PNcoR Fb+rzE60N4I3Z/07ymlFXfaaFWdSpB1eGQurYVzjFyb4xiLYRYGp5AfL6f7UIkP8CnSR 6AYo1ytkjNGzKF86aH8QTcOmejKy9J6yyU5C55AJlbGh4bKCJiIF/ieShAeaNPE66VRQ 7dOFl3w9P0QXNB04NR+iyPouG4BhRnlw1qYNP70adVP9LrYiSqGQi1i6GFCg+k+k03KX Di5BnatLlU0RZ2XGVlkYKf9RDx2Ave1DSzzSxk0FnuBQSY1OdpOTCZ4Y7NTDkD2FlXE9 krXA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Qc2DDhk7eSddbt2Q3kjJLSbCNGbLZWOl9JsGZF81sG4=; fh=qWoxmPN1zRqnLSoCWGpfFsDtzJsx/SGdqx98lbP+Uho=; b=pHrEKmxirucO4qlDOva53QbNIbGjs7WIgFCxX2FWJ3NQVgNCI/qhwSVk2t4aWgO1+G 5t+3C0lW9ULqPzAsl5L4VheSXU8t33N0qVJPPUkeugMUragxya/B2yBCEqGltHNM3nXE tZcx/ETSUp8nEYy2/rnp6SnM1upSrcdMVfQGOeJZX5VEh/YvPdxfLajiBd1xMSDk4RQE 7G6ncZSq+ihR6exe+c9iBpFEpYj3T/Xtl9ARL1rsfj8T+tpPDKdcslrZ+KX/Tsx9J8sd MeauVZ20OzG8P9A4XXXmTm+IwHDsSFl3cr/Fr/Al6h8pwoxbodpIe+KUccYhfBTeSXpl l0wQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=SjlBdcCR; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from fry.vger.email (fry.vger.email. [23.128.96.38]) by mx.google.com with ESMTPS id c26-20020a630d1a000000b00565f0e9cfbbsi8513662pgl.382.2023.11.20.09.52.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Nov 2023 09:52:57 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) client-ip=23.128.96.38; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=SjlBdcCR; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by fry.vger.email (Postfix) with ESMTP id A66DA8082902; Mon, 20 Nov 2023 09:52:55 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at fry.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233858AbjKTRvx (ORCPT + 27 others); Mon, 20 Nov 2023 12:51:53 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49786 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234384AbjKTRvY (ORCPT ); Mon, 20 Nov 2023 12:51:24 -0500 Received: from out-183.mta1.migadu.com (out-183.mta1.migadu.com [95.215.58.183]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 10C69D5F for ; Mon, 20 Nov 2023 09:50:45 -0800 (PST) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1700502644; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Qc2DDhk7eSddbt2Q3kjJLSbCNGbLZWOl9JsGZF81sG4=; b=SjlBdcCRByh5PrG2MOhc8aFxhKI3iaDpNaxTHUEIaJyLo5jT4ZNFXt6Bt3sd9u50OfOKSg CaLcy9JTN+VHkT3VWRZiUR5iYfkV9u6uxcxdsNTqWTVW/b0iVcXorrCGAHL4XcSBww8rJU PD/hyomJcMBAWfjY4j+0VCD4sMN1M4g= From: andrey.konovalov@linux.dev To: Andrew Morton Cc: Andrey Konovalov , Marco Elver , Alexander Potapenko , Dmitry Vyukov , Vlastimil Babka , kasan-dev@googlegroups.com, Evgenii Stepanov , Oscar Salvador , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v4 20/22] kasan: use stack_depot_put for tag-based modes Date: Mon, 20 Nov 2023 18:47:18 +0100 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on fry.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (fry.vger.email [0.0.0.0]); Mon, 20 Nov 2023 09:52:55 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783106400596003958 X-GMAIL-MSGID: 1783106400596003958 From: Andrey Konovalov Make tag-based KASAN modes evict stack traces from the stack depot once they are evicted from the stack ring. Internally, pass STACK_DEPOT_FLAG_GET to stack_depot_save_flags (via kasan_save_stack) to increment the refcount when saving a new entry to stack ring and call stack_depot_put when removing an entry from stack ring. Reviewed-by: Alexander Potapenko Signed-off-by: Andrey Konovalov --- Changes v1->v2: - Adapt to the stack depot API change. - Drop READ_ONCE when reading entry->stack. --- mm/kasan/tags.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/mm/kasan/tags.c b/mm/kasan/tags.c index b6c017e670d8..739ae997463d 100644 --- a/mm/kasan/tags.c +++ b/mm/kasan/tags.c @@ -97,12 +97,13 @@ static void save_stack_info(struct kmem_cache *cache, void *object, gfp_t gfp_flags, bool is_free) { unsigned long flags; - depot_stack_handle_t stack; + depot_stack_handle_t stack, old_stack; u64 pos; struct kasan_stack_ring_entry *entry; void *old_ptr; - stack = kasan_save_stack(gfp_flags, STACK_DEPOT_FLAG_CAN_ALLOC); + stack = kasan_save_stack(gfp_flags, + STACK_DEPOT_FLAG_CAN_ALLOC | STACK_DEPOT_FLAG_GET); /* * Prevent save_stack_info() from modifying stack ring @@ -121,6 +122,8 @@ static void save_stack_info(struct kmem_cache *cache, void *object, if (!try_cmpxchg(&entry->ptr, &old_ptr, STACK_RING_BUSY_PTR)) goto next; /* Busy slot. */ + old_stack = entry->stack; + entry->size = cache->object_size; entry->pid = current->pid; entry->stack = stack; @@ -129,6 +132,9 @@ static void save_stack_info(struct kmem_cache *cache, void *object, entry->ptr = object; read_unlock_irqrestore(&stack_ring.lock, flags); + + if (old_stack) + stack_depot_put(old_stack); } void kasan_save_alloc_info(struct kmem_cache *cache, void *object, gfp_t flags) From patchwork Mon Nov 20 17:47:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 167330 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2b07:b0:403:3b70:6f57 with SMTP id io7csp108082vqb; Mon, 20 Nov 2023 09:52:21 -0800 (PST) X-Google-Smtp-Source: AGHT+IHM0KFIaZ6UxRSkIqWQ9Mhl1YEIxkfbfJtVh+biV9J+xtOY1D5pQB0HKFH4/zgmIQA/HSau X-Received: by 2002:a05:6a20:8409:b0:181:1fc8:c5de with SMTP id c9-20020a056a20840900b001811fc8c5demr8755003pzd.43.1700502741451; Mon, 20 Nov 2023 09:52:21 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700502741; cv=none; d=google.com; s=arc-20160816; b=bmKKZcqUDuu02+FW/9Mc0PMQ8soQ9HBbWhKBE6H0l+xVuoZzagMg9EVUDl4Nu7xbv4 EmjZMZBVepYvO/ei0slNW6yJ4iCbROdqe4dCcehfnUGjkz+lcf5Vo/fuYipkLNvtWk7S sB2DJMpfRYjwi9KOyLdxlbddlYuN7hezLgRCOqxgRy2iZ3MMz4ZzY+wLo5i43/P2zPni qdoeAJWJJbZWunmkpYcTRaGI+xoR2Y/OGRfgNXn5dsRXYQbQ6Ufn0uv12IQs9IVwmIp6 V1jAXLbJ2bL64SVO6Jjh0yPhDa2hYuFj7CR69cKi0CrUjuCpRG1qtOOyebd8LR4rZVeg Rw8g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=RvoTVlNH/9QJKGjG7ts3ES/7xJIaYyT0JIUdEPtk5OM=; fh=qWoxmPN1zRqnLSoCWGpfFsDtzJsx/SGdqx98lbP+Uho=; b=nBI5A/FX4NSIUK6qkAB/wTCcBVTelLUr/4ES4Iqq3cjl4fV/mFt3D473IGA8/fjJ4Q kL5zMnkfs/0/U1sfoA4N/uFQeS4ZttPe8P3iuqDRRX8e1gF2UI86q1CLXt+USw5nzdff 6Xz0dwcUmzKPf6518Hs0ir55SWupesY4kU3Lz7bFEoKayS9cR/JkjvRRPS90LFeEL1gs Zx1O/N814W5w7UIxpRq6Ylv4EToR4LZRfeAfZCNZFISG3HpsT2OwegYGbb+o/DU21Z82 2w/caE74YNh+Zi5rU1F2XjIGbkbQPpvKMfdZbUUY4fz5ORPtkUlNcpJqeiZzCw+639R1 16MQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=DNgy9uYy; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from lipwig.vger.email (lipwig.vger.email. [23.128.96.33]) by mx.google.com with ESMTPS id l10-20020a6542ca000000b0057755b2f032si8675966pgp.542.2023.11.20.09.52.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Nov 2023 09:52:21 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) client-ip=23.128.96.33; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=DNgy9uYy; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id 21A25802FAB8; Mon, 20 Nov 2023 09:52:19 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234316AbjKTRv7 (ORCPT + 27 others); Mon, 20 Nov 2023 12:51:59 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39782 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234461AbjKTRv0 (ORCPT ); Mon, 20 Nov 2023 12:51:26 -0500 Received: from out-173.mta1.migadu.com (out-173.mta1.migadu.com [95.215.58.173]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 919A4D70 for ; Mon, 20 Nov 2023 09:50:46 -0800 (PST) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1700502644; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RvoTVlNH/9QJKGjG7ts3ES/7xJIaYyT0JIUdEPtk5OM=; b=DNgy9uYyYrfck71UIDDMmeBlP2Bik/eXdRS59wwnt2UBuC9bsjPHLcrxPmB7E98KHzUlfS OnFiyTSIdvZJFlR6bvb819vsoHa3zDRv09EFK0vHzurXdv2QeiuJeuOu21i9cfXyy/okM9 r0Wi2UF4LVMIbr0H8VW9kEyPJyGdVBE= From: andrey.konovalov@linux.dev To: Andrew Morton Cc: Andrey Konovalov , Marco Elver , Alexander Potapenko , Dmitry Vyukov , Vlastimil Babka , kasan-dev@googlegroups.com, Evgenii Stepanov , Oscar Salvador , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v4 21/22] kasan: use stack_depot_put for Generic mode Date: Mon, 20 Nov 2023 18:47:19 +0100 Message-Id: <5cef104d9b842899489b4054fe8d1339a71acee0.1700502145.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Mon, 20 Nov 2023 09:52:19 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783106362403211825 X-GMAIL-MSGID: 1783106362403211825 From: Andrey Konovalov Evict alloc/free stack traces from the stack depot for Generic KASAN once they are evicted from the quaratine. For auxiliary stack traces, evict the oldest stack trace once a new one is saved (KASAN only keeps references to the last two). Also evict all saved stack traces on krealloc. To avoid double-evicting and mis-evicting stack traces (in case KASAN's metadata was corrupted), reset KASAN's per-object metadata that stores stack depot handles when the object is initialized and when it's evicted from the quarantine. Note that stack_depot_put is no-op if the handle is 0. Reviewed-by: Marco Elver Signed-off-by: Andrey Konovalov --- mm/kasan/common.c | 3 ++- mm/kasan/generic.c | 22 ++++++++++++++++++---- mm/kasan/quarantine.c | 26 ++++++++++++++++++++------ 3 files changed, 40 insertions(+), 11 deletions(-) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 825a0240ec02..b5d8bd26fced 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -50,7 +50,8 @@ depot_stack_handle_t kasan_save_stack(gfp_t flags, depot_flags_t depot_flags) void kasan_set_track(struct kasan_track *track, gfp_t flags) { track->pid = current->pid; - track->stack = kasan_save_stack(flags, STACK_DEPOT_FLAG_CAN_ALLOC); + track->stack = kasan_save_stack(flags, + STACK_DEPOT_FLAG_CAN_ALLOC | STACK_DEPOT_FLAG_GET); } #if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) diff --git a/mm/kasan/generic.c b/mm/kasan/generic.c index 5d168c9afb32..50cc519e23f4 100644 --- a/mm/kasan/generic.c +++ b/mm/kasan/generic.c @@ -449,10 +449,14 @@ struct kasan_free_meta *kasan_get_free_meta(struct kmem_cache *cache, void kasan_init_object_meta(struct kmem_cache *cache, const void *object) { struct kasan_alloc_meta *alloc_meta; + struct kasan_free_meta *free_meta; alloc_meta = kasan_get_alloc_meta(cache, object); if (alloc_meta) __memset(alloc_meta, 0, sizeof(*alloc_meta)); + free_meta = kasan_get_free_meta(cache, object); + if (free_meta) + __memset(free_meta, 0, sizeof(*free_meta)); } size_t kasan_metadata_size(struct kmem_cache *cache, bool in_object) @@ -489,18 +493,20 @@ static void __kasan_record_aux_stack(void *addr, depot_flags_t depot_flags) if (!alloc_meta) return; + stack_depot_put(alloc_meta->aux_stack[1]); alloc_meta->aux_stack[1] = alloc_meta->aux_stack[0]; alloc_meta->aux_stack[0] = kasan_save_stack(0, depot_flags); } void kasan_record_aux_stack(void *addr) { - return __kasan_record_aux_stack(addr, STACK_DEPOT_FLAG_CAN_ALLOC); + return __kasan_record_aux_stack(addr, + STACK_DEPOT_FLAG_CAN_ALLOC | STACK_DEPOT_FLAG_GET); } void kasan_record_aux_stack_noalloc(void *addr) { - return __kasan_record_aux_stack(addr, 0); + return __kasan_record_aux_stack(addr, STACK_DEPOT_FLAG_GET); } void kasan_save_alloc_info(struct kmem_cache *cache, void *object, gfp_t flags) @@ -508,8 +514,16 @@ void kasan_save_alloc_info(struct kmem_cache *cache, void *object, gfp_t flags) struct kasan_alloc_meta *alloc_meta; alloc_meta = kasan_get_alloc_meta(cache, object); - if (alloc_meta) - kasan_set_track(&alloc_meta->alloc_track, flags); + if (!alloc_meta) + return; + + /* Evict previous stack traces (might exist for krealloc). */ + stack_depot_put(alloc_meta->alloc_track.stack); + stack_depot_put(alloc_meta->aux_stack[0]); + stack_depot_put(alloc_meta->aux_stack[1]); + __memset(alloc_meta, 0, sizeof(*alloc_meta)); + + kasan_set_track(&alloc_meta->alloc_track, flags); } void kasan_save_free_info(struct kmem_cache *cache, void *object) diff --git a/mm/kasan/quarantine.c b/mm/kasan/quarantine.c index ca4529156735..265ca2bbe2dd 100644 --- a/mm/kasan/quarantine.c +++ b/mm/kasan/quarantine.c @@ -143,11 +143,22 @@ static void *qlink_to_object(struct qlist_node *qlink, struct kmem_cache *cache) static void qlink_free(struct qlist_node *qlink, struct kmem_cache *cache) { void *object = qlink_to_object(qlink, cache); - struct kasan_free_meta *meta = kasan_get_free_meta(cache, object); + struct kasan_alloc_meta *alloc_meta = kasan_get_alloc_meta(cache, object); + struct kasan_free_meta *free_meta = kasan_get_free_meta(cache, object); unsigned long flags; - if (IS_ENABLED(CONFIG_SLAB)) - local_irq_save(flags); + if (alloc_meta) { + stack_depot_put(alloc_meta->alloc_track.stack); + stack_depot_put(alloc_meta->aux_stack[0]); + stack_depot_put(alloc_meta->aux_stack[1]); + __memset(alloc_meta, 0, sizeof(*alloc_meta)); + } + + if (free_meta && + *(u8 *)kasan_mem_to_shadow(object) == KASAN_SLAB_FREETRACK) { + stack_depot_put(free_meta->free_track.stack); + free_meta->free_track.stack = 0; + } /* * If init_on_free is enabled and KASAN's free metadata is stored in @@ -157,14 +168,17 @@ static void qlink_free(struct qlist_node *qlink, struct kmem_cache *cache) */ if (slab_want_init_on_free(cache) && cache->kasan_info.free_meta_offset == 0) - memzero_explicit(meta, sizeof(*meta)); + memzero_explicit(free_meta, sizeof(*free_meta)); /* - * As the object now gets freed from the quarantine, assume that its - * free track is no longer valid. + * As the object now gets freed from the quarantine, + * take note that its free track is no longer exists. */ *(u8 *)kasan_mem_to_shadow(object) = KASAN_SLAB_FREE; + if (IS_ENABLED(CONFIG_SLAB)) + local_irq_save(flags); + ___cache_free(cache, object, _THIS_IP_); if (IS_ENABLED(CONFIG_SLAB)) From patchwork Mon Nov 20 17:47:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 167333 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2b07:b0:403:3b70:6f57 with SMTP id io7csp108288vqb; Mon, 20 Nov 2023 09:52:43 -0800 (PST) X-Google-Smtp-Source: AGHT+IFEM2GOJACOZFv0+ySvNIVAB1WYc+eXJ19MjvwMyym5SK8R15ucyqmAcCxURbyDEst+4+x/ X-Received: by 2002:a05:6e02:156f:b0:357:a71f:8310 with SMTP id k15-20020a056e02156f00b00357a71f8310mr12156983ilu.10.1700502763426; Mon, 20 Nov 2023 09:52:43 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700502763; cv=none; d=google.com; s=arc-20160816; b=ZBlbqTd4m9h4O7PxFAQ7LC23z6eZHID5fCj+IODAuNZdH6EuxYhZe421NT2cb23pe6 KvqYLRA7MjSOGYblvX7FbVLhELm/Dey8WlcoEdJyyQvPxeoy9xwPqdXqpnY1Bo0ep3CF J9pX5n4zHIUo5fyAbXpEwwroZLVLRBMbz1ho0EEIJiZyS9PT+NGQz78xR1PUIpxwrgU1 YrgXKB3pZc83OArY4HJlyPH9K2dd8ta7u5sOEKbQLXyRvJzOsZ42Qtz3c8hpr+vPsxH8 TJJOVkcYowtLP+5Yo79R7vFuvO1pzPVfImZ8AF5fehOZ5V10AJlEkMSNVJSoLL4hl6ft yeRQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=mWJyEm6VyaOWhREL8AAhhvtbKxcDTX0iHOdq2I7biaE=; fh=qWoxmPN1zRqnLSoCWGpfFsDtzJsx/SGdqx98lbP+Uho=; b=HbWIVL/HDlC211t/sAq77U3AH65ea2Jxe4XLIuP/2FoqxL9P+Hzr8og7hGcmiKxFCR PFSdTYaqe2UgzeIT1arg/BXtPTc+Vmh2/tMpPhHqRbOOmbaWL0xYqtl3hL/WF5C6mn24 3Fu8CqkBQNlcCjqpS+IioZFtGaFphPRs7M0sGl0evAmh8VXnxPjSiyRb6PElmoOoHwbE gZer9i/hWiTmDO458god0+6SnxgoRAID+8ZFG7DNzxsEzcGcZmYPQsPgjjGUYcnsdqvW AHvXyXkWO4qGS+0o7XnQeb/uS4gOT9w9b4ifSLYx17icyv66FNOWWa/E2VsVuUBZkZ1i +pAg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=WmK6Qhek; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from agentk.vger.email (agentk.vger.email. [2620:137:e000::3:2]) by mx.google.com with ESMTPS id e12-20020a65678c000000b00563d9ff5157si8433277pgr.547.2023.11.20.09.52.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Nov 2023 09:52:43 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) client-ip=2620:137:e000::3:2; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=WmK6Qhek; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id 4E3A0803D8CC; Mon, 20 Nov 2023 09:52:41 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234473AbjKTRwC (ORCPT + 27 others); Mon, 20 Nov 2023 12:52:02 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49162 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234457AbjKTRvZ (ORCPT ); Mon, 20 Nov 2023 12:51:25 -0500 Received: from out-188.mta1.migadu.com (out-188.mta1.migadu.com [95.215.58.188]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 542FA1988 for ; Mon, 20 Nov 2023 09:50:47 -0800 (PST) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1700502645; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mWJyEm6VyaOWhREL8AAhhvtbKxcDTX0iHOdq2I7biaE=; b=WmK6QhekX7FInJyHV2uIR20pRmvpYax2GIBAaZMlPJyA0fa28yyqkU2wHc9EZkcNKhaGVw rpPS5IdkJL5crjQMbxbFNxa8MnpB9QR4IgbAId2GDpPOic2y+TwrC0P2SZxzr/nb1ApKtj eY3o0dSnWGXhfqV2/RcUKsx+woR/BOo= From: andrey.konovalov@linux.dev To: Andrew Morton Cc: Andrey Konovalov , Marco Elver , Alexander Potapenko , Dmitry Vyukov , Vlastimil Babka , kasan-dev@googlegroups.com, Evgenii Stepanov , Oscar Salvador , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v4 22/22] lib/stackdepot: adjust DEPOT_POOLS_CAP for KMSAN Date: Mon, 20 Nov 2023 18:47:20 +0100 Message-Id: <301a115cf7ce8ddb42ef6de9151c2bb76ba728fc.1700502145.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Mon, 20 Nov 2023 09:52:41 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783106385430841998 X-GMAIL-MSGID: 1783106385430841998 From: Andrey Konovalov KMSAN is frequently used in fuzzing scenarios and thus saves a lot of stack traces. As KMSAN does not support evicting stack traces from the stack depot, the stack depot capacity might be reached quickly with large stack records. Adjust the maximum number of stack depot pools for this case. The average size of a stack trace saved into the stack depot is ~16 frames. Thus, adjust the maximum pools number accordingly to keep the maximum number of stack traces that can be saved into the stack depot similar to the one that was allowed before the stack trace eviction changes. Signed-off-by: Andrey Konovalov --- lib/stackdepot.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/lib/stackdepot.c b/lib/stackdepot.c index c1b31160f4b4..870cce2f4cbd 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -41,7 +41,17 @@ #define DEPOT_OFFSET_BITS (DEPOT_POOL_ORDER + PAGE_SHIFT - DEPOT_STACK_ALIGN) #define DEPOT_POOL_INDEX_BITS (DEPOT_HANDLE_BITS - DEPOT_OFFSET_BITS - \ STACK_DEPOT_EXTRA_BITS) +#if IS_ENABLED(CONFIG_KMSAN) && CONFIG_STACKDEPOT_MAX_FRAMES >= 32 +/* + * KMSAN is frequently used in fuzzing scenarios and thus saves a lot of stack + * traces. As KMSAN does not support evicting stack traces from the stack + * depot, the stack depot capacity might be reached quickly with large stack + * records. Adjust the maximum number of stack depot pools for this case. + */ +#define DEPOT_POOLS_CAP (8192 * (CONFIG_STACKDEPOT_MAX_FRAMES / 16)) +#else #define DEPOT_POOLS_CAP 8192 +#endif #define DEPOT_MAX_POOLS \ (((1LL << (DEPOT_POOL_INDEX_BITS)) < DEPOT_POOLS_CAP) ? \ (1LL << (DEPOT_POOL_INDEX_BITS)) : DEPOT_POOLS_CAP)