From patchwork Mon Oct 23 16:22:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 156973 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:ce89:0:b0:403:3b70:6f57 with SMTP id p9csp1408387vqx; Mon, 23 Oct 2023 09:23:16 -0700 (PDT) X-Google-Smtp-Source: AGHT+IG01KKz9jPv/HrWuFTRT7RepY6K8296FINTmLcqa1LQHHeYozwM7FI1lD53ztDw+wKhD/x8 X-Received: by 2002:a17:90a:7047:b0:27c:df02:88b3 with SMTP id f65-20020a17090a704700b0027cdf0288b3mr13918946pjk.8.1698078195729; Mon, 23 Oct 2023 09:23:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1698078195; cv=none; d=google.com; s=arc-20160816; b=bPII5a5ntu0ajvafOjlJnNRQ7U1q+l8iRBisnua9tpaU6py9UA/GtT5j5gEhW+Pwcx DVTTQ5rmVF/gNdJdnl1qIkt8iR6kP02reKl3VHxLdA1jizdoo2A5IvmYne3j0MEIeBfZ MGrIPZ2XJQHWiLPMTPdoKT15oLUrodwApJtxBvalsW+qQQpm8uMEkZOSmlv/PMX/v9W8 +akSCTwNCqpC3XVUPQ/j/glQeW0oOx9HY42hwrEy6PE2IUzd7V8fNJiZnj4tjdwL0n/o arY836wRdVc/xemFY/zr2hMeS+Pbq8ar5lH18MpQ/svR7MbMwIUKGQXlbi9Qa5YUvAYB ilUA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=dA6zDrMISQ9ixV+61d0CbJSFq6N+6qPUyMdB5Zjx2Sw=; fh=nfQSbTp1dWHt2Ier1Up8UhVNdXOqoJJLNvTrpT3jJEk=; b=HhZHXxPFy+2abR7mGVlrX1rq0Jb2H95SwNHeVEaFrfhrmQ+HaCPuQ2dUCJM5RoAjVH 4GIYUIYx8DcCfqGa90MW6YDCLn+vm77SvLU9iElFz4ap+fTGV63BzxplEJzuSfTORerw 4ztqRJxjdPHHhkDz01CFB1lBocQp27EHIiYtawmEPcLIg+fxt/Jssvkb/XrgeDaOJNsN U2eLRuYWWHgTiRuaYqijX0QXRifSihLOdljGGihutRxX+YRH+8upjiVttxUpJ46M9mxv QHT5h6OE7sG0idfUZNdM2aaHJ4AgTaXdjGr6TZWg3d+oKSdUcijqqREs9RrsECEI1XLM xhVw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=CkRusZIQ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id nm10-20020a17090b19ca00b0027909685905si6627050pjb.149.2023.10.23.09.23.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Oct 2023 09:23:15 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=CkRusZIQ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id D9636804E72F; Mon, 23 Oct 2023 09:23:14 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233119AbjJWQXJ (ORCPT + 27 others); Mon, 23 Oct 2023 12:23:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38392 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230255AbjJWQXF (ORCPT ); Mon, 23 Oct 2023 12:23:05 -0400 Received: from out-208.mta0.migadu.com (out-208.mta0.migadu.com [91.218.175.208]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4CFE810C for ; Mon, 23 Oct 2023 09:23:00 -0700 (PDT) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1698078178; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dA6zDrMISQ9ixV+61d0CbJSFq6N+6qPUyMdB5Zjx2Sw=; b=CkRusZIQKT1eG4Gl5cCrRUWt5DUG+IA1Am7cKqEhxJvtxvEOtNHOGfa8oOTjOvRtsLmZuh DIpreTOPrJBHIpWysqvlzeB2EOS4cbEOWO+vFGBjLCZ+m4G6B8FIZg7ByX7aEIEwmgDxIB IhsqPDQSxHpXeKy5blW2EyaMIQyiiKs= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko Cc: Andrey Konovalov , Dmitry Vyukov , Vlastimil Babka , kasan-dev@googlegroups.com, Evgenii Stepanov , Oscar Salvador , Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v3 01/19] lib/stackdepot: check disabled flag when fetching Date: Mon, 23 Oct 2023 18:22:32 +0200 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Mon, 23 Oct 2023 09:23:14 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780564042285358623 X-GMAIL-MSGID: 1780564042285358623 From: Andrey Konovalov Do not try fetching a stack trace from the stack depot if the stack_depot_disabled flag is enabled. Reviewed-by: Alexander Potapenko Signed-off-by: Andrey Konovalov --- lib/stackdepot.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/stackdepot.c b/lib/stackdepot.c index 2f5aa851834e..3a945c7206f3 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -477,7 +477,7 @@ unsigned int stack_depot_fetch(depot_stack_handle_t handle, */ kmsan_unpoison_memory(entries, sizeof(*entries)); - if (!handle) + if (!handle || stack_depot_disabled) return 0; if (parts.pool_index > pool_index_cached) { From patchwork Mon Oct 23 16:22:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 156974 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:ce89:0:b0:403:3b70:6f57 with SMTP id p9csp1408455vqx; Mon, 23 Oct 2023 09:23:22 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHYtcrP4Fx7+U/VLoQW+dql5Hv821nY54wcqTyO6o0wK//zj2tZzNOsavQL+idKb2OD80nk X-Received: by 2002:a17:90b:4f82:b0:274:77b3:64d0 with SMTP id qe2-20020a17090b4f8200b0027477b364d0mr16934882pjb.17.1698078202207; Mon, 23 Oct 2023 09:23:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1698078202; cv=none; d=google.com; s=arc-20160816; b=pPTj63AURUx3FEqVumrw9dc79tx6aqmZu6WaLpdQw1y7EODOMBgsoTUEnAkTi4WarK JhQUo6okhhO2xOb7TZ+v2fj3oeYqgJfGTNdmeYyVQDfpjOJKLuRRAvtxrKbGnAU9jM5L fJmlYcwteY6NwLzF6WyD0U1MQV1SAiMQTT6bw8yaVWQkH9l0uqTmxHxR8AvaN3ncdC73 XCXr3oyxHhgMgN2moOHHyLrgqJNoDN+In/zxjimDj7adYDyrlr3Ic7EsWbVJH48Vm2So htI0MDmaHld1Tkl3a1HSqNzjBsjtfJbSrIpZhT95OK1A6olCyos8Q2o8YMRu6fdufX3Q YzvQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=hlykGlfL2an/VgWBLR95t+wE6CpOLlFaVT0cHbSVLN0=; fh=nfQSbTp1dWHt2Ier1Up8UhVNdXOqoJJLNvTrpT3jJEk=; b=oFxbNloWov0XAJ31wA+XPULPBlAMQJj4aQr/ZpRNTNa8zKI3hFgNYDYlj9fxnza62h dqXiGRwiP2rEufJqhYVo4/xr/x8ECXUikH2zFILAte+cLb8x+L/d3oiA8Nt6KSDSzisv owYQq9WX8IoGdEY8oAK88j8pPgiSqgn9RtIR62IUaW6RonR5J6IvAhXGwcRx1Yd/qE+o sFYeLl79l89jT3nn8Qu776lFjvToGaoCwLyvnX9kRKetYq11wPU7hj4zXkMZlkXXsuuV KySQcaWEUVGm/zRafx2khMFkGPRYbhROIKiKT31TV7gNDWhG0Lt0ECMzJ2+C0ris4nAV ybZA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=BwPbMNBq; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from agentk.vger.email (agentk.vger.email. [2620:137:e000::3:2]) by mx.google.com with ESMTPS id m5-20020a170902db8500b001b025aba9f2si6286686pld.22.2023.10.23.09.23.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Oct 2023 09:23:22 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) client-ip=2620:137:e000::3:2; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=BwPbMNBq; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id 455F6805AA3C; Mon, 23 Oct 2023 09:23:18 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232426AbjJWQXH (ORCPT + 27 others); Mon, 23 Oct 2023 12:23:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38406 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230329AbjJWQXF (ORCPT ); Mon, 23 Oct 2023 12:23:05 -0400 Received: from out-205.mta0.migadu.com (out-205.mta0.migadu.com [91.218.175.205]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4D03E10E for ; Mon, 23 Oct 2023 09:23:00 -0700 (PDT) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1698078178; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hlykGlfL2an/VgWBLR95t+wE6CpOLlFaVT0cHbSVLN0=; b=BwPbMNBqQr8eCi/2bzTTF8duZP4VRRfQUHs2ug+CAfTzdfxW5c7tGWmaNI5EFjq6IxzJZq SfjwshdOrp6OB9NwpUh9zr0qnIhgsa5b547AMebg2LpMrGL6e3ElCdDIR06kr2JMooKxRy iI329x7c9WbbQOKgKwFWyqcU/WVnUL4= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko Cc: Andrey Konovalov , Dmitry Vyukov , Vlastimil Babka , kasan-dev@googlegroups.com, Evgenii Stepanov , Oscar Salvador , Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v3 02/19] lib/stackdepot: simplify __stack_depot_save Date: Mon, 23 Oct 2023 18:22:33 +0200 Message-Id: <6ff0d1e89e50ba74618eed30fd3170dc78decea3.1698077459.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Mon, 23 Oct 2023 09:23:18 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780564048989443980 X-GMAIL-MSGID: 1780564048989443980 From: Andrey Konovalov The retval local variable in __stack_depot_save has the union type handle_parts, but the function never uses anything but the union's handle field. Define retval simply as depot_stack_handle_t to simplify the code. Reviewed-by: Alexander Potapenko Signed-off-by: Andrey Konovalov --- lib/stackdepot.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/lib/stackdepot.c b/lib/stackdepot.c index 3a945c7206f3..0772125efe8a 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -360,7 +360,7 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, gfp_t alloc_flags, bool can_alloc) { struct stack_record *found = NULL, **bucket; - union handle_parts retval = { .handle = 0 }; + depot_stack_handle_t handle = 0; struct page *page = NULL; void *prealloc = NULL; unsigned long flags; @@ -377,7 +377,7 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, nr_entries = filter_irq_stacks(entries, nr_entries); if (unlikely(nr_entries == 0) || stack_depot_disabled) - goto fast_exit; + return 0; hash = hash_stack(entries, nr_entries); bucket = &stack_table[hash & stack_hash_mask]; @@ -443,9 +443,8 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, free_pages((unsigned long)prealloc, DEPOT_POOL_ORDER); } if (found) - retval.handle = found->handle.handle; -fast_exit: - return retval.handle; + handle = found->handle.handle; + return handle; } EXPORT_SYMBOL_GPL(__stack_depot_save); From patchwork Mon Oct 23 16:22:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 156977 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:ce89:0:b0:403:3b70:6f57 with SMTP id p9csp1408680vqx; Mon, 23 Oct 2023 09:23:46 -0700 (PDT) X-Google-Smtp-Source: AGHT+IH84NcS5epxhOUYPfmJODWPPk+WmO5R8rC2eJtuK/gOmye+M436yKwVN/y9rcc+Bkzzlu7w X-Received: by 2002:a17:903:18a:b0:1c9:cf26:8d91 with SMTP id z10-20020a170903018a00b001c9cf268d91mr8209867plg.8.1698078226071; Mon, 23 Oct 2023 09:23:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1698078226; cv=none; d=google.com; s=arc-20160816; b=z0HdeSLURYucf9mJyJ+rFq54xlwVV7v6yd+igR7wJ9HvLbnM3FnUye+ZJo1afS0eMJ NJj0ERXQv7BV5M9p1onxTcLtKqCttRCWgv/5nPNazcSCo0wfXGTXyXBs69XK976uNuiN BdR0BXqQjzPPDXMp64jBkKlhOZ2NUv+qrJ2fCGanCabC50pY5cSNQCGYS+Zd/aD33ZCZ xwspoaAN8cFy+WI9J2tdBjOR02SdapIbC3sdfKUCwF/qqZ8+KjIB432Lp9ouZV/5SE+p LqIi5T6SvTJjT6pkhdtgCTIORLMXN/sHJ2sjbt/AK2H6ZqwwxqPRoqB1b+tO6znY9SyB ZMww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=eQriSK20/8KwwHOVdQKhz9C/zLOim8sff0QLVit0F7g=; fh=nfQSbTp1dWHt2Ier1Up8UhVNdXOqoJJLNvTrpT3jJEk=; b=nG+LC0UBPf42ZRoJpAQ40L89D9P4IT51y4qgC93SIzgd2JK3E8sd2Wt+sp1jdtl9hA PLmAoamtwFcDwJ8iSA/NAfXn3DQgWBOvH83sCh8ndN8PgoaiJ+s19RrNfcGFec9ZkRrW MhE8ll9tl/wBvcLyu2KWTchVA1XPrICTMZ1W0Gkah2pTnLFJ5SRMQCBLOTUk2Riiy0Cv KVXBhCESsHMBDCCtwIa71LgoLq4LcawegJdGnYLu0bDhFGyabMWJrVw0+cPbXV/NOGqY GKu+9Ek5j8tIRTgk+GoydEVL94V0aHP13Wqpu+u+2gKELo7lkUED46yI1FsXR857Qp+A bQTw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b="S/cthfh4"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from lipwig.vger.email (lipwig.vger.email. [23.128.96.33]) by mx.google.com with ESMTPS id t23-20020a1709028c9700b001c3fa95ca03si6520661plo.9.2023.10.23.09.23.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Oct 2023 09:23:46 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) client-ip=23.128.96.33; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b="S/cthfh4"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id 0AF9480B1230; Mon, 23 Oct 2023 09:23:44 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231827AbjJWQXX (ORCPT + 27 others); Mon, 23 Oct 2023 12:23:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38438 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232207AbjJWQXG (ORCPT ); Mon, 23 Oct 2023 12:23:06 -0400 Received: from out-208.mta0.migadu.com (out-208.mta0.migadu.com [91.218.175.208]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EDFF1D79 for ; Mon, 23 Oct 2023 09:23:00 -0700 (PDT) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1698078179; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=eQriSK20/8KwwHOVdQKhz9C/zLOim8sff0QLVit0F7g=; b=S/cthfh4emQrjcVoGLkWqQT0OvtdGJAoIvLTqkDDKVckvqP8bUpXYF9LHT/z69ET7396wu VHNZOBPwORg5a49jPPMsPr93gE7IJt9XyYCBaDWrIFujMNX6hp01vXkIeQlmRKoXZZfyEN tUKjyZoUBaxEoZLynpfz1I7/Mg/7kMg= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko Cc: Andrey Konovalov , Dmitry Vyukov , Vlastimil Babka , kasan-dev@googlegroups.com, Evgenii Stepanov , Oscar Salvador , Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v3 03/19] lib/stackdepot: drop valid bit from handles Date: Mon, 23 Oct 2023 18:22:34 +0200 Message-Id: <5e251a589cb3142607ec5af8fcb904d424702a14.1698077459.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Mon, 23 Oct 2023 09:23:44 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780564074174941443 X-GMAIL-MSGID: 1780564074174941443 From: Andrey Konovalov Stack depot doesn't use the valid bit in handles in any way, so drop it. Reviewed-by: Alexander Potapenko Signed-off-by: Andrey Konovalov --- lib/stackdepot.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/lib/stackdepot.c b/lib/stackdepot.c index 0772125efe8a..482eac40791e 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -32,13 +32,12 @@ #define DEPOT_HANDLE_BITS (sizeof(depot_stack_handle_t) * 8) -#define DEPOT_VALID_BITS 1 #define DEPOT_POOL_ORDER 2 /* Pool size order, 4 pages */ #define DEPOT_POOL_SIZE (1LL << (PAGE_SHIFT + DEPOT_POOL_ORDER)) #define DEPOT_STACK_ALIGN 4 #define DEPOT_OFFSET_BITS (DEPOT_POOL_ORDER + PAGE_SHIFT - DEPOT_STACK_ALIGN) -#define DEPOT_POOL_INDEX_BITS (DEPOT_HANDLE_BITS - DEPOT_VALID_BITS - \ - DEPOT_OFFSET_BITS - STACK_DEPOT_EXTRA_BITS) +#define DEPOT_POOL_INDEX_BITS (DEPOT_HANDLE_BITS - DEPOT_OFFSET_BITS - \ + STACK_DEPOT_EXTRA_BITS) #define DEPOT_POOLS_CAP 8192 #define DEPOT_MAX_POOLS \ (((1LL << (DEPOT_POOL_INDEX_BITS)) < DEPOT_POOLS_CAP) ? \ @@ -50,7 +49,6 @@ union handle_parts { struct { u32 pool_index : DEPOT_POOL_INDEX_BITS; u32 offset : DEPOT_OFFSET_BITS; - u32 valid : DEPOT_VALID_BITS; u32 extra : STACK_DEPOT_EXTRA_BITS; }; }; @@ -303,7 +301,6 @@ depot_alloc_stack(unsigned long *entries, int size, u32 hash, void **prealloc) stack->size = size; stack->handle.pool_index = pool_index; stack->handle.offset = pool_offset >> DEPOT_STACK_ALIGN; - stack->handle.valid = 1; stack->handle.extra = 0; memcpy(stack->entries, entries, flex_array_size(stack, entries, size)); pool_offset += required_size; From patchwork Mon Oct 23 16:22:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 156976 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:ce89:0:b0:403:3b70:6f57 with SMTP id p9csp1408593vqx; Mon, 23 Oct 2023 09:23:37 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEeLhJ9xmZ7fDMY2BQ0AexWjjD6lpQnvMqRMbARwN6aHSQaQD1BFbFtcYBh6u9ApjjhRPYh X-Received: by 2002:a05:6a20:8e0e:b0:129:d944:2e65 with SMTP id y14-20020a056a208e0e00b00129d9442e65mr120651pzj.13.1698078217287; Mon, 23 Oct 2023 09:23:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1698078217; cv=none; d=google.com; s=arc-20160816; b=th9FBuGCO1yBKyvHOqZ7FU8OFsCMeu1S0gu/HFnZuh3L9N112UrgjgZ6o9JfcMaIpU TkjcCxpCNfHiwE7iheDr3p1yCqTXwGsauuIf89SjHVDChwDQXUY3YAAPG0/c7Qgh9q6W xpXCObuqjS2GbSisVQ8TuMwHRaHgB14enSrnjorCB/PwPT1fbevnOpS3yJE52Y/3eb57 qT9rnMn6HoZLqpWmnfbKR0uumHCx+CKsOnccbyfgXKAEBqQCthnXPbYvzwZv4/yxSpUU No12eaL5843leuDIgi+TKiL28JbMEslaT/UwqaFwRvbkLMoiHQdgy3vuEGRJocwz8Ncc Nm6Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=9Tw+dU2o5sfOuuXbX4FJVP/5BAKc6p0/nQ7cBmUzv5Q=; fh=nfQSbTp1dWHt2Ier1Up8UhVNdXOqoJJLNvTrpT3jJEk=; b=pTJzKtBJAnE0w63cM8XPbko1atLxTV39GFICpNws6L7V6wpq8nFVWuvwQnZy8piCx3 RZXn+9hOwjLQCo9kcWlyapRooV5wB0N8NRheOMxT4KBWQ4WOEBTOMj3mg2jWBiainY2i YRiMnQCK4/yg0MTH9zYCdJPOyU9Lei3KY4n+ocO6ZcXoy9tuYqGxpjC5+SRzFpZPrQ1z RD9Jhy9r8WF8zTAIq/IF5S9xNfdoq+5xQlk3a88MtP7YgzB6X86XtL6sCEq/GO8lHPAX AOs5QMWtb9pdySDFVRYMVEC/7oItKBvfvpEOwdyWH+5NOKnqfColnk7BgMUGPI6nkppm UGLA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=MDW7AbX4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from agentk.vger.email (agentk.vger.email. [23.128.96.32]) by mx.google.com with ESMTPS id f11-20020a056a00228b00b0068fe810e8a0si6927021pfe.197.2023.10.23.09.23.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Oct 2023 09:23:37 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) client-ip=23.128.96.32; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=MDW7AbX4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id 24E98805AA3B; Mon, 23 Oct 2023 09:23:32 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233529AbjJWQXT (ORCPT + 27 others); Mon, 23 Oct 2023 12:23:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38378 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232212AbjJWQXG (ORCPT ); Mon, 23 Oct 2023 12:23:06 -0400 Received: from out-191.mta0.migadu.com (out-191.mta0.migadu.com [91.218.175.191]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AD7F3FF for ; Mon, 23 Oct 2023 09:23:01 -0700 (PDT) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1698078180; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9Tw+dU2o5sfOuuXbX4FJVP/5BAKc6p0/nQ7cBmUzv5Q=; b=MDW7AbX4eL0/TS7YLtRKEDg9+Xdw06lVwYB5nqof1k6w+UTAbp/tBlDVRi4NGfguchUae+ /lNPyD5G/IVsNQ4ZhW+fz7iuCkHqlbzWf5vQUi0pyT0Wn1XU8dxqiYv/rpGzHXYqU6FP/D 216IpW+w8WvbWqzOU6usNeJ95Qnlubo= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko Cc: Andrey Konovalov , Dmitry Vyukov , Vlastimil Babka , kasan-dev@googlegroups.com, Evgenii Stepanov , Oscar Salvador , Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v3 04/19] lib/stackdepot: add depot_fetch_stack helper Date: Mon, 23 Oct 2023 18:22:35 +0200 Message-Id: <48b71b2ff972088aacb3466d4de5afd46b6aa7e5.1698077459.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Mon, 23 Oct 2023 09:23:32 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780564064843367993 X-GMAIL-MSGID: 1780564064843367993 From: Andrey Konovalov Add a helper depot_fetch_stack function that fetches the pointer to a stack record. With this change, all static depot_* functions now operate on stack pools and the exported stack_depot_* functions operate on the hash table. Reviewed-by: Alexander Potapenko Signed-off-by: Andrey Konovalov --- Changes v1->v2: - Minor comment fix as suggested by Alexander. --- lib/stackdepot.c | 45 ++++++++++++++++++++++++++++----------------- 1 file changed, 28 insertions(+), 17 deletions(-) diff --git a/lib/stackdepot.c b/lib/stackdepot.c index 482eac40791e..9a004f15f59d 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -304,6 +304,7 @@ depot_alloc_stack(unsigned long *entries, int size, u32 hash, void **prealloc) stack->handle.extra = 0; memcpy(stack->entries, entries, flex_array_size(stack, entries, size)); pool_offset += required_size; + /* * Let KMSAN know the stored stack record is initialized. This shall * prevent false positive reports if instrumented code accesses it. @@ -313,6 +314,32 @@ depot_alloc_stack(unsigned long *entries, int size, u32 hash, void **prealloc) return stack; } +static struct stack_record *depot_fetch_stack(depot_stack_handle_t handle) +{ + union handle_parts parts = { .handle = handle }; + /* + * READ_ONCE pairs with potential concurrent write in + * depot_alloc_stack(). + */ + int pool_index_cached = READ_ONCE(pool_index); + void *pool; + size_t offset = parts.offset << DEPOT_STACK_ALIGN; + struct stack_record *stack; + + if (parts.pool_index > pool_index_cached) { + WARN(1, "pool index %d out of bounds (%d) for stack id %08x\n", + parts.pool_index, pool_index_cached, handle); + return NULL; + } + + pool = stack_pools[parts.pool_index]; + if (!pool) + return NULL; + + stack = pool + offset; + return stack; +} + /* Calculates the hash for a stack. */ static inline u32 hash_stack(unsigned long *entries, unsigned int size) { @@ -456,14 +483,6 @@ EXPORT_SYMBOL_GPL(stack_depot_save); unsigned int stack_depot_fetch(depot_stack_handle_t handle, unsigned long **entries) { - union handle_parts parts = { .handle = handle }; - /* - * READ_ONCE pairs with potential concurrent write in - * depot_alloc_stack. - */ - int pool_index_cached = READ_ONCE(pool_index); - void *pool; - size_t offset = parts.offset << DEPOT_STACK_ALIGN; struct stack_record *stack; *entries = NULL; @@ -476,15 +495,7 @@ unsigned int stack_depot_fetch(depot_stack_handle_t handle, if (!handle || stack_depot_disabled) return 0; - if (parts.pool_index > pool_index_cached) { - WARN(1, "pool index %d out of bounds (%d) for stack id %08x\n", - parts.pool_index, pool_index_cached, handle); - return 0; - } - pool = stack_pools[parts.pool_index]; - if (!pool) - return 0; - stack = pool + offset; + stack = depot_fetch_stack(handle); *entries = stack->entries; return stack->size; From patchwork Mon Oct 23 16:22:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 156975 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:ce89:0:b0:403:3b70:6f57 with SMTP id p9csp1408541vqx; Mon, 23 Oct 2023 09:23:33 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFvMOV7eDSsMVniKhxrJK1ikBvGJiHizGV8WkSaV+IQZ67/traa7YmKjI9V7FAxNDGaU+eC X-Received: by 2002:a05:6359:6243:b0:168:e4e6:19bb with SMTP id sd3-20020a056359624300b00168e4e619bbmr1819492rwb.5.1698078212955; Mon, 23 Oct 2023 09:23:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1698078212; cv=none; d=google.com; s=arc-20160816; b=TSLduOCewsE9HLAl4eIu41Pykk2ULHCHeKo+wkUvcOZ4QDUu91o7w940aeyUcnyekU QnWZ4C6CQdhznpQZNUQa8jlfJa2TREJGYGLi3dXA6e9cfSRfNi4Bx3Z/UzleFV9iQcvg QinVGEPOywYIWO71jIsO51IRdI/b9FTvOPw4mjIB8bWdZ0Knw0QjxjlBiQthiKGON/3K bfsrEQclhpitr5y4nPBF4XwEMcQZzZOJn3ENR1gSyh8h45VWjmALMUmlA6AVwgTB6TY9 qu6GzYp/czUUIqXhzH+F0cYAv8Byl/HkpkalnbCP4TQYdD2I6xLOWUJQ70qeibBMOjgN jksQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=bMe0IukHdtXUzp2BGVWle8stWPjWZ8/o4+6QYAoowVk=; fh=nfQSbTp1dWHt2Ier1Up8UhVNdXOqoJJLNvTrpT3jJEk=; b=XE0/tqu3Sj4494mBYa7c1aoKf89dAPq4qyBg20p/GMK98uWy3/0Z0ct5MnPbBTGOmx ODcVPJeXya4g0HXThAOnzPjrNqTvIoYNuPpr9xBciatpbtBdkUKZPfG/U1X5kDyQX5Mb lgx0MzKIi/l7me5Tb/0lYAl3CtkRrgr+e7+scKxt4vQ3NWQ6VFJ9QlbC/izSfiKcaY6I OpEQHU9CDfU65DbTF/T92iAbDXxtIXtlmoU4xO7wiH8NB68osHUr5q91K02vqGjzZMJ4 m2CVwYDBCKhWiLBJsJ3rqIofPO3n/wMYhorEfp3QRTplCp5y7+6VaemY7gCyNiVvGoho fvuA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=rkq81ZIH; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id e1-20020a630f01000000b00578b8d202b0si6490589pgl.536.2023.10.23.09.23.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Oct 2023 09:23:32 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=rkq81ZIH; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 36DB0804E73C; Mon, 23 Oct 2023 09:23:31 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232406AbjJWQXZ (ORCPT + 27 others); Mon, 23 Oct 2023 12:23:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47210 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233406AbjJWQXK (ORCPT ); Mon, 23 Oct 2023 12:23:10 -0400 Received: from out-197.mta0.migadu.com (out-197.mta0.migadu.com [91.218.175.197]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 267C6FF for ; Mon, 23 Oct 2023 09:23:07 -0700 (PDT) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1698078180; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bMe0IukHdtXUzp2BGVWle8stWPjWZ8/o4+6QYAoowVk=; b=rkq81ZIHIptQkPMqAJh4zbGsz6Tf/Km8HtCA+J1L5FmEVUWRXZFPtR3mYiLwbXzLU1ve+m ipP9E4g9fFYH2AhD85YBltaGTWSFkAJKRvJkJTSAujCAdfb7pv+ffb4OXZf2EtpweYEmBM QWsHiUJVDHj2Hr+JhC9QlyMPSRnLRJ0= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko Cc: Andrey Konovalov , Dmitry Vyukov , Vlastimil Babka , kasan-dev@googlegroups.com, Evgenii Stepanov , Oscar Salvador , Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v3 05/19] lib/stackdepot: use fixed-sized slots for stack records Date: Mon, 23 Oct 2023 18:22:36 +0200 Message-Id: <4340f57fa82fde81e00f64f98d69ee4a5d5ed9a8.1698077459.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Mon, 23 Oct 2023 09:23:31 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780564059969573792 X-GMAIL-MSGID: 1780564059969573792 From: Andrey Konovalov Instead of storing stack records in stack depot pools one right after another, use fixed-sized slots. Add a new Kconfig option STACKDEPOT_MAX_FRAMES that allows to select the size of the slot in frames. Use 64 as the default value, which is the maximum stack trace size both KASAN and KMSAN use right now. Also add descriptions for other stack depot Kconfig options. This is preparatory patch for implementing the eviction of stack records from the stack depot. Signed-off-by: Andrey Konovalov --- Changes v2->v3: - Keep previously existing Kconfig options not configurable by users. Changes v1->v2: - Add and use STACKDEPOT_MAX_FRAMES Kconfig option. --- lib/Kconfig | 10 ++++++++++ lib/stackdepot.c | 13 +++++++++---- 2 files changed, 19 insertions(+), 4 deletions(-) diff --git a/lib/Kconfig b/lib/Kconfig index c686f4adc124..5f3fa3659fa9 100644 --- a/lib/Kconfig +++ b/lib/Kconfig @@ -710,10 +710,20 @@ config ARCH_STACKWALK config STACKDEPOT bool select STACKTRACE + help + Stack depot: stack trace storage that avoids duplication config STACKDEPOT_ALWAYS_INIT bool select STACKDEPOT + help + Always initialize stack depot during early boot + +config STACKDEPOT_MAX_FRAMES + int "Maximum number of frames in trace saved in stack depot" + range 1 256 + default 64 + depends on STACKDEPOT config REF_TRACKER bool diff --git a/lib/stackdepot.c b/lib/stackdepot.c index 9a004f15f59d..128ece21afe9 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -58,9 +58,12 @@ struct stack_record { u32 hash; /* Hash in the hash table */ u32 size; /* Number of stored frames */ union handle_parts handle; - unsigned long entries[]; /* Variable-sized array of frames */ + unsigned long entries[CONFIG_STACKDEPOT_MAX_FRAMES]; /* Frames */ }; +#define DEPOT_STACK_RECORD_SIZE \ + ALIGN(sizeof(struct stack_record), 1 << DEPOT_STACK_ALIGN) + static bool stack_depot_disabled; static bool __stack_depot_early_init_requested __initdata = IS_ENABLED(CONFIG_STACKDEPOT_ALWAYS_INIT); static bool __stack_depot_early_init_passed __initdata; @@ -258,9 +261,7 @@ static struct stack_record * depot_alloc_stack(unsigned long *entries, int size, u32 hash, void **prealloc) { struct stack_record *stack; - size_t required_size = struct_size(stack, entries, size); - - required_size = ALIGN(required_size, 1 << DEPOT_STACK_ALIGN); + size_t required_size = DEPOT_STACK_RECORD_SIZE; /* Check if there is not enough space in the current pool. */ if (unlikely(pool_offset + required_size > DEPOT_POOL_SIZE)) { @@ -295,6 +296,10 @@ depot_alloc_stack(unsigned long *entries, int size, u32 hash, void **prealloc) if (stack_pools[pool_index] == NULL) return NULL; + /* Limit number of saved frames to CONFIG_STACKDEPOT_MAX_FRAMES. */ + if (size > CONFIG_STACKDEPOT_MAX_FRAMES) + size = CONFIG_STACKDEPOT_MAX_FRAMES; + /* Save the stack trace. */ stack = stack_pools[pool_index] + pool_offset; stack->hash = hash; From patchwork Mon Oct 23 16:22:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 156987 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:ce89:0:b0:403:3b70:6f57 with SMTP id p9csp1412052vqx; Mon, 23 Oct 2023 09:29:54 -0700 (PDT) X-Google-Smtp-Source: AGHT+IF40OtxQ3Pk9rAbIzxyXyPUs4pEfQttFtxocycjyUqqkd/kss7LLh8SuU4KsqC9ksk8AC8P X-Received: by 2002:a05:6a20:6a1f:b0:17c:c278:bcb8 with SMTP id p31-20020a056a206a1f00b0017cc278bcb8mr86606pzk.39.1698078594164; Mon, 23 Oct 2023 09:29:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1698078594; cv=none; d=google.com; s=arc-20160816; b=ukNVALwk8KmERNrs/vxoTBzCOF0mmPQ4RZ/hV1OVzJIA6mw1brZuNKRK5TV5vDT1Hn 40jeeRFc0NoSPI/805hJrHiWIRBTBr78S3hODue814g8o4yTeFpSHyfAp51V/c3WNRsT Y8oWvjg1qJ9a9+XcegZi+RJ1UrlMS2u5raxLzPz0I5U+yCmM6HXU+P3ntCT5TypDAdal VfMTzCK4601tNA1VsjHQvyxkxGRls1ewHUrMIvBTrPD9Jut6Yd5MxKdWffB10cu1SpXi BsguXidBWiIOtUMvra61ngV70nLkQbhru/Ke2mVNOyx2qj6mRTl7GiKAsBex/1tUTqVz Z0bQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=6w0l/5/OLmmiPtUBy12XvNW481vH2W+yw4PJnFK30wA=; fh=nfQSbTp1dWHt2Ier1Up8UhVNdXOqoJJLNvTrpT3jJEk=; b=vJYyODJ5QsIefqHIs+YxmqYvnJffQk6+PHANKuwCtXzYHfjaF4Adw9+zif3MSe/7g0 tZEm7G8hXUe7eSrmCri8xJrdDbGUsND09dOD0OWO5QNr3SPEiCcKZSvRNP3hihFz46sU PP72mVXCrTcloWX1FrgjeJtyw6B4Y8Umpb4hW2fAYZhZrfb7fAn+zoYUbZW51/BVsWol ByXvJ1ehc1LOktoJsK55gTzeKz6R0QLCghDxn+8Hq59gB33h6QsVX2705cUbcUMzpGTP FZCZV8+dF9i2zFBiEy2KT23F6IQ6w942JCBKQqK5ccHJdNAbS3pPrOWARsx/fWstCh4c VSkA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=tTmhSqri; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id y6-20020a170902ed4600b001bc02b730f3si6285077plb.242.2023.10.23.09.29.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Oct 2023 09:29:54 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=tTmhSqri; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 707F48072167; Mon, 23 Oct 2023 09:29:53 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233688AbjJWQ3o (ORCPT + 27 others); Mon, 23 Oct 2023 12:29:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60540 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230338AbjJWQ3f (ORCPT ); Mon, 23 Oct 2023 12:29:35 -0400 Received: from out-201.mta1.migadu.com (out-201.mta1.migadu.com [95.215.58.201]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B5572E4 for ; Mon, 23 Oct 2023 09:29:31 -0700 (PDT) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1698078243; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6w0l/5/OLmmiPtUBy12XvNW481vH2W+yw4PJnFK30wA=; b=tTmhSqriTq1xAUwIMKx9hsGiOaqTSZIq/wKJLXRg6daSa2A/wpp5RWjZ1i+KFXZWKiTvL7 FT80+nqHJsuFFL2EcMj7absCsQ1tHdiG6bPU8EWrNzLrOT2cqyLQvL1bQUd/tEodCi1gSB pSo03HvvDVMKsmPPsyN4xZhauDuuzOs= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko Cc: Andrey Konovalov , Dmitry Vyukov , Vlastimil Babka , kasan-dev@googlegroups.com, Evgenii Stepanov , Oscar Salvador , Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v3 06/19] lib/stackdepot: fix and clean-up atomic annotations Date: Mon, 23 Oct 2023 18:22:37 +0200 Message-Id: <8f649d7e5919c56bcc5d2d356c9584fdcb87800e.1698077459.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Mon, 23 Oct 2023 09:29:53 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780564459907715398 X-GMAIL-MSGID: 1780564459907715398 From: Andrey Konovalov Drop smp_load_acquire from next_pool_required in depot_init_pool, as both depot_init_pool and the all smp_store_release's to this variable are executed under the stack depot lock. Also simplify and clean up comments accompanying the use of atomic accesses in the stack depot code. Reviewed-by: Alexander Potapenko Signed-off-by: Andrey Konovalov --- This patch is not strictly required, as the atomic accesses are fully removed in one of the latter patches. However, I decided to keep the patch just in case we end up needing these atomics in the following iterations of this series. Changes v2->v3: - Keep parentheses when referring to functions in comments. - Add comment that explains why depot_init_pool reads next_pool_required non-atomically. Changes v1->v2: - Minor comment fix as suggested by Marco. - Drop READ_ONCE marking for next_pool_required. --- lib/stackdepot.c | 29 ++++++++++++++--------------- 1 file changed, 14 insertions(+), 15 deletions(-) diff --git a/lib/stackdepot.c b/lib/stackdepot.c index 128ece21afe9..60aea549429a 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -225,10 +225,10 @@ static void depot_init_pool(void **prealloc) /* * If the next pool is already initialized or the maximum number of * pools is reached, do not use the preallocated memory. - * smp_load_acquire() here pairs with smp_store_release() below and - * in depot_alloc_stack(). + * Access next_pool_required non-atomically, as there are no concurrent + * write accesses to this variable. */ - if (!smp_load_acquire(&next_pool_required)) + if (!next_pool_required) return; /* Check if the current pool is not yet allocated. */ @@ -249,8 +249,8 @@ static void depot_init_pool(void **prealloc) * At this point, either the next pool is initialized or the * maximum number of pools is reached. In either case, take * note that initializing another pool is not required. - * This smp_store_release pairs with smp_load_acquire() above - * and in stack_depot_save(). + * smp_store_release() pairs with smp_load_acquire() in + * stack_depot_save(). */ smp_store_release(&next_pool_required, 0); } @@ -273,7 +273,7 @@ depot_alloc_stack(unsigned long *entries, int size, u32 hash, void **prealloc) /* * Move on to the next pool. - * WRITE_ONCE pairs with potential concurrent read in + * WRITE_ONCE() pairs with potential concurrent read in * stack_depot_fetch(). */ WRITE_ONCE(pool_index, pool_index + 1); @@ -281,8 +281,8 @@ depot_alloc_stack(unsigned long *entries, int size, u32 hash, void **prealloc) /* * If the maximum number of pools is not reached, take note * that the next pool needs to initialized. - * smp_store_release() here pairs with smp_load_acquire() in - * stack_depot_save() and depot_init_pool(). + * smp_store_release() pairs with smp_load_acquire() in + * stack_depot_save(). */ if (pool_index + 1 < DEPOT_MAX_POOLS) smp_store_release(&next_pool_required, 1); @@ -323,7 +323,7 @@ static struct stack_record *depot_fetch_stack(depot_stack_handle_t handle) { union handle_parts parts = { .handle = handle }; /* - * READ_ONCE pairs with potential concurrent write in + * READ_ONCE() pairs with potential concurrent write in * depot_alloc_stack(). */ int pool_index_cached = READ_ONCE(pool_index); @@ -413,8 +413,7 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, /* * Fast path: look the stack trace up without locking. - * The smp_load_acquire() here pairs with smp_store_release() to - * |bucket| below. + * smp_load_acquire() pairs with smp_store_release() to |bucket| below. */ found = find_stack(smp_load_acquire(bucket), entries, nr_entries, hash); if (found) @@ -424,8 +423,8 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, * Check if another stack pool needs to be initialized. If so, allocate * the memory now - we won't be able to do that under the lock. * - * The smp_load_acquire() here pairs with smp_store_release() to - * |next_pool_inited| in depot_alloc_stack() and depot_init_pool(). + * smp_load_acquire() pairs with smp_store_release() in + * depot_alloc_stack() and depot_init_pool(). */ if (unlikely(can_alloc && smp_load_acquire(&next_pool_required))) { /* @@ -451,8 +450,8 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, if (new) { new->next = *bucket; /* - * This smp_store_release() pairs with - * smp_load_acquire() from |bucket| above. + * smp_store_release() pairs with smp_load_acquire() + * from |bucket| above. */ smp_store_release(bucket, new); found = new; From patchwork Mon Oct 23 16:22:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 156991 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:ce89:0:b0:403:3b70:6f57 with SMTP id p9csp1412749vqx; Mon, 23 Oct 2023 09:30:52 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFXbROiFrQYR+E6iXxVRDzmyvCCv/LC1yqa0J5TjHC3NL78r4pvoHtgM6HAWaHwjMxzgkh3 X-Received: by 2002:a17:902:c782:b0:1c9:d111:9b27 with SMTP id w2-20020a170902c78200b001c9d1119b27mr8762826pla.32.1698078652387; Mon, 23 Oct 2023 09:30:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1698078652; cv=none; d=google.com; s=arc-20160816; b=iPWWSwThGstUfIXiIq1n2D0/LWB8ywTVBI3kcsjL6k9CWo9PsIFBEZWWTpOgEOTlNh gN57gQMkSo4YUWyOZvlyQ+m+PZY+WXrbYZTQ3IrvFOvvFRPzUha+hTVaVnIW+TtikeMS ZPY75qYKjZLxrPfPfF/16HIgd0hmKuhpAJUwqbD5ERAtDzmPiVGD4YfMNA4DtdlU1ZsM C0bmOhfF+pGTQL7pybpr6+D8Wb67v12saYUePpSUfA1B4brXv1B39d0oMAnBuqG1HxkX gtxaUR7WGQh5uAHI9ilXjVdGqOmNHZaAMU0YNATUmHU1qqiAj/94PSyhYJgd/w0ORPx6 fuHQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=NZ9DxgJteIUWBZVu3uTAKUKrmlqiVCjIJZsHKffVHuY=; fh=nfQSbTp1dWHt2Ier1Up8UhVNdXOqoJJLNvTrpT3jJEk=; b=Cr1/LG5S+wUXl67B19T7wfCl79837l1zAEsDCakqb/c0+bUgh5us5FOOXfx47wiDDx Wc0EDOPMdC6KhVs/Z874FM35tPvn95t4ShOqLd2Nq0N2sCZom4ok2GxaqgN2pbJZCZGb W+/rGPD/MGs9bB/F4CEq6a+1kkxKZhDISxa9lxOUlLS6kf/Wu9/7LyrC4yem2KkslQXT nhHoEWp5dxcc97lUN75INTA8C1tTi653HrgETA3T8KPXnz3oHxXiGzQxEJJ51EfP9/i/ 1URTsGK9bwCuSr4d06JvrJfdj9o4UZ8wFbAJAqaBvvIGmpRjYOUz2wfzLWyjXEnYuiYa T/KA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=t+UnmZVx; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from groat.vger.email (groat.vger.email. [2620:137:e000::3:5]) by mx.google.com with ESMTPS id y23-20020a63de57000000b0059df1860d60si5501450pgi.424.2023.10.23.09.30.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Oct 2023 09:30:52 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) client-ip=2620:137:e000::3:5; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=t+UnmZVx; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by groat.vger.email (Postfix) with ESMTP id 59F7A808ABAB; Mon, 23 Oct 2023 09:30:48 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at groat.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233758AbjJWQ3u (ORCPT + 27 others); Mon, 23 Oct 2023 12:29:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60542 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229544AbjJWQ3f (ORCPT ); Mon, 23 Oct 2023 12:29:35 -0400 Received: from out-205.mta1.migadu.com (out-205.mta1.migadu.com [IPv6:2001:41d0:203:375::cd]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 02413E6 for ; Mon, 23 Oct 2023 09:29:31 -0700 (PDT) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1698078243; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NZ9DxgJteIUWBZVu3uTAKUKrmlqiVCjIJZsHKffVHuY=; b=t+UnmZVxxGN5KzWEQTBmXitTlS6kZTAGzoq0dlMyEevn1HxA0s/x7HvVUtWknOMuAkjIeo BUBxiie7hZs3gjhcBL/qrihnoFcL2dZNbSEVqgM/Rho1AztZFu7TnVgjNbIz/DOVY1yTw3 E63W+Bv8C9VotbqWpTxfMOF8IeJvWIk= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko Cc: Andrey Konovalov , Dmitry Vyukov , Vlastimil Babka , kasan-dev@googlegroups.com, Evgenii Stepanov , Oscar Salvador , Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v3 07/19] lib/stackdepot: rework helpers for depot_alloc_stack Date: Mon, 23 Oct 2023 18:22:38 +0200 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Mon, 23 Oct 2023 09:30:48 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780564521012629962 X-GMAIL-MSGID: 1780564521012629962 From: Andrey Konovalov Split code in depot_alloc_stack and depot_init_pool into 3 functions: 1. depot_keep_next_pool that keeps preallocated memory for the next pool if required. 2. depot_update_pools that moves on to the next pool if there's no space left in the current pool, uses preallocated memory for the new current pool if required, and calls depot_keep_next_pool otherwise. 3. depot_alloc_stack that calls depot_update_pools and then allocates a stack record as before. This makes it somewhat easier to follow the logic of depot_alloc_stack and also serves as a preparation for implementing the eviction of stack records from the stack depot. Reviewed-by: Alexander Potapenko Signed-off-by: Andrey Konovalov --- Changes v2->v3: - Add parentheses when referring to function calls in comments. --- lib/stackdepot.c | 86 +++++++++++++++++++++++++++--------------------- 1 file changed, 49 insertions(+), 37 deletions(-) diff --git a/lib/stackdepot.c b/lib/stackdepot.c index 60aea549429a..3f921aaae44a 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -219,11 +219,11 @@ int stack_depot_init(void) } EXPORT_SYMBOL_GPL(stack_depot_init); -/* Uses preallocated memory to initialize a new stack depot pool. */ -static void depot_init_pool(void **prealloc) +/* Keeps the preallocated memory to be used for the next stack depot pool. */ +static void depot_keep_next_pool(void **prealloc) { /* - * If the next pool is already initialized or the maximum number of + * If the next pool is already saved or the maximum number of * pools is reached, do not use the preallocated memory. * Access next_pool_required non-atomically, as there are no concurrent * write accesses to this variable. @@ -231,44 +231,34 @@ static void depot_init_pool(void **prealloc) if (!next_pool_required) return; - /* Check if the current pool is not yet allocated. */ - if (stack_pools[pool_index] == NULL) { - /* Use the preallocated memory for the current pool. */ - stack_pools[pool_index] = *prealloc; + /* + * Use the preallocated memory for the next pool + * as long as we do not exceed the maximum number of pools. + */ + if (pool_index + 1 < DEPOT_MAX_POOLS) { + stack_pools[pool_index + 1] = *prealloc; *prealloc = NULL; - } else { - /* - * Otherwise, use the preallocated memory for the next pool - * as long as we do not exceed the maximum number of pools. - */ - if (pool_index + 1 < DEPOT_MAX_POOLS) { - stack_pools[pool_index + 1] = *prealloc; - *prealloc = NULL; - } - /* - * At this point, either the next pool is initialized or the - * maximum number of pools is reached. In either case, take - * note that initializing another pool is not required. - * smp_store_release() pairs with smp_load_acquire() in - * stack_depot_save(). - */ - smp_store_release(&next_pool_required, 0); } + + /* + * At this point, either the next pool is kept or the maximum + * number of pools is reached. In either case, take note that + * keeping another pool is not required. + * smp_store_release() pairs with smp_load_acquire() in + * stack_depot_save(). + */ + smp_store_release(&next_pool_required, 0); } -/* Allocates a new stack in a stack depot pool. */ -static struct stack_record * -depot_alloc_stack(unsigned long *entries, int size, u32 hash, void **prealloc) +/* Updates refences to the current and the next stack depot pools. */ +static bool depot_update_pools(size_t required_size, void **prealloc) { - struct stack_record *stack; - size_t required_size = DEPOT_STACK_RECORD_SIZE; - /* Check if there is not enough space in the current pool. */ if (unlikely(pool_offset + required_size > DEPOT_POOL_SIZE)) { /* Bail out if we reached the pool limit. */ if (unlikely(pool_index + 1 >= DEPOT_MAX_POOLS)) { WARN_ONCE(1, "Stack depot reached limit capacity"); - return NULL; + return false; } /* @@ -278,9 +268,10 @@ depot_alloc_stack(unsigned long *entries, int size, u32 hash, void **prealloc) */ WRITE_ONCE(pool_index, pool_index + 1); pool_offset = 0; + /* * If the maximum number of pools is not reached, take note - * that the next pool needs to initialized. + * that the next pool needs to be initialized. * smp_store_release() pairs with smp_load_acquire() in * stack_depot_save(). */ @@ -288,9 +279,30 @@ depot_alloc_stack(unsigned long *entries, int size, u32 hash, void **prealloc) smp_store_release(&next_pool_required, 1); } - /* Assign the preallocated memory to a pool if required. */ + /* Check if the current pool is not yet allocated. */ + if (*prealloc && stack_pools[pool_index] == NULL) { + /* Use the preallocated memory for the current pool. */ + stack_pools[pool_index] = *prealloc; + *prealloc = NULL; + return true; + } + + /* Otherwise, try using the preallocated memory for the next pool. */ if (*prealloc) - depot_init_pool(prealloc); + depot_keep_next_pool(prealloc); + return true; +} + +/* Allocates a new stack in a stack depot pool. */ +static struct stack_record * +depot_alloc_stack(unsigned long *entries, int size, u32 hash, void **prealloc) +{ + struct stack_record *stack; + size_t required_size = DEPOT_STACK_RECORD_SIZE; + + /* Update current and next pools if required and possible. */ + if (!depot_update_pools(required_size, prealloc)) + return NULL; /* Check if we have a pool to save the stack trace. */ if (stack_pools[pool_index] == NULL) @@ -324,7 +336,7 @@ static struct stack_record *depot_fetch_stack(depot_stack_handle_t handle) union handle_parts parts = { .handle = handle }; /* * READ_ONCE() pairs with potential concurrent write in - * depot_alloc_stack(). + * depot_update_pools(). */ int pool_index_cached = READ_ONCE(pool_index); void *pool; @@ -424,7 +436,7 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, * the memory now - we won't be able to do that under the lock. * * smp_load_acquire() pairs with smp_store_release() in - * depot_alloc_stack() and depot_init_pool(). + * depot_update_pools() and depot_keep_next_pool(). */ if (unlikely(can_alloc && smp_load_acquire(&next_pool_required))) { /* @@ -461,7 +473,7 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, * Stack depot already contains this stack trace, but let's * keep the preallocated memory for the future. */ - depot_init_pool(&prealloc); + depot_keep_next_pool(&prealloc); } raw_spin_unlock_irqrestore(&pool_lock, flags); From patchwork Mon Oct 23 16:22:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 156989 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:ce89:0:b0:403:3b70:6f57 with SMTP id p9csp1412135vqx; Mon, 23 Oct 2023 09:30:00 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFOjCBb5xh0ZaxXzjOtKsxKVuwIDRQhP8HPm4ue2ngwpYQ42cqgv0KYXGlpcY6TqUDt0Xmt X-Received: by 2002:a17:902:e5cb:b0:1ca:220:ce42 with SMTP id u11-20020a170902e5cb00b001ca0220ce42mr7905473plf.37.1698078600407; Mon, 23 Oct 2023 09:30:00 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1698078600; cv=none; d=google.com; s=arc-20160816; b=xhIhN8sh6TW7T2aOuQL2O97YmqVwDpqLqXSL7JU2fKobaI6KKztKgOiZu1dAONjhG1 FtKdgkI60qt1lC30Yz8kISCY9ddddQIx3dKeXC+ZaYNcKHrI0Hzusy/aYs/lO1B/8fn1 4fBu2QJLcp9pJ9pXquQaPyllrZ891tTlswr/f/79fphgizci9+P5qaVu0QMfJEHMiUWS eqCmKfQ9zwJtkXW2i3MKoI8ufsbA9qwohSqsp9mIDWQHv+4nNXP2dwWEymyjJ+poqfjq LEg+il2rZDAms3wgRcn5gYTzrjBR60W9N6zjzgVyczL9D7l6Sn7lM3lORPJaSCrf508P Kufw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=QFz7c41J1QbS9PQ1h2ItmRgQJE/l+/U2CDCp2U3Lga0=; fh=nfQSbTp1dWHt2Ier1Up8UhVNdXOqoJJLNvTrpT3jJEk=; b=y4iQ3KvI+zjQspyUgJ1/LBqd/a0kblpAIj3tGN20rXU/nsCCgMPe8rb1voc9Y/ZZF2 q66BYao1ZXKgtro9aw7RPkYhioe1srqUGkabALriSzVhL1p1Oiihg+OnUjFuh/DzUNQx AO8xP7nrRB+w/764MeAru67FE9bqbJBmt1NNS4mqdBpj6FD+Rf+fT8E/OcW+Yc9f0jxI LnGEEO5NytIRR+yp0bXVOKguwJ6WHg8hZVaz9NpTFWvdqgMS8ruD7wrMBvRT9Fos09sP yPC+SD8CKf/w64erpdHmGQcO5Serbd7Fg6ITR38WQunBTuxaj2WvCMwXyxd56kTCARsx s4Aw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b="Ln8jn/bt"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id bf1-20020a17090b0b0100b0027d22d5aa7esi6491149pjb.46.2023.10.23.09.30.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Oct 2023 09:30:00 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b="Ln8jn/bt"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 7D8A6804E6A3; Mon, 23 Oct 2023 09:29:59 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233717AbjJWQ3r (ORCPT + 27 others); Mon, 23 Oct 2023 12:29:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60554 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230432AbjJWQ3f (ORCPT ); Mon, 23 Oct 2023 12:29:35 -0400 Received: from out-201.mta1.migadu.com (out-201.mta1.migadu.com [95.215.58.201]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8C07AFF for ; Mon, 23 Oct 2023 09:29:32 -0700 (PDT) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1698078244; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QFz7c41J1QbS9PQ1h2ItmRgQJE/l+/U2CDCp2U3Lga0=; b=Ln8jn/btUwxdS3NYz+kfGYLWvsWTvJ5u3YyVev72quQFxpQ6rAJIQeN6sfP0fJG2IQ2K/7 Ru/NFMADjWGJJEoVmdgwuQzr+Cafy+Kpdr9JMH8ZcWTuixF85WuxaVOR1wWdIJI6caGvld MaqFZ97JESE7qZ5aZRIgTe9xXo/s0x4= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko Cc: Andrey Konovalov , Dmitry Vyukov , Vlastimil Babka , kasan-dev@googlegroups.com, Evgenii Stepanov , Oscar Salvador , Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v3 08/19] lib/stackdepot: rename next_pool_required to new_pool_required Date: Mon, 23 Oct 2023 18:22:39 +0200 Message-Id: <5d8faf61763307b4d71a9b5272b88fb5c5ae7521.1698077459.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Mon, 23 Oct 2023 09:29:59 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780564466582259045 X-GMAIL-MSGID: 1780564466582259045 From: Andrey Konovalov Rename next_pool_required to new_pool_required. This a purely code readability change: the following patch will change stack depot to store the pointer to the new pool in a separate variable, and "new" seems like a more logical name. Reviewed-by: Alexander Potapenko Signed-off-by: Andrey Konovalov --- lib/stackdepot.c | 49 ++++++++++++++++++++++++------------------------ 1 file changed, 24 insertions(+), 25 deletions(-) diff --git a/lib/stackdepot.c b/lib/stackdepot.c index 3f921aaae44a..7579e20114b1 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -93,12 +93,11 @@ static size_t pool_offset; static DEFINE_RAW_SPINLOCK(pool_lock); /* * Stack depot tries to keep an extra pool allocated even before it runs out - * of space in the currently used pool. - * This flag marks that this next extra pool needs to be allocated and - * initialized. It has the value 0 when either the next pool is not yet - * initialized or the limit on the number of pools is reached. + * of space in the currently used pool. This flag marks whether this extra pool + * needs to be allocated. It has the value 0 when either an extra pool is not + * yet allocated or if the limit on the number of pools is reached. */ -static int next_pool_required = 1; +static int new_pool_required = 1; static int __init disable_stack_depot(char *str) { @@ -219,20 +218,20 @@ int stack_depot_init(void) } EXPORT_SYMBOL_GPL(stack_depot_init); -/* Keeps the preallocated memory to be used for the next stack depot pool. */ -static void depot_keep_next_pool(void **prealloc) +/* Keeps the preallocated memory to be used for a new stack depot pool. */ +static void depot_keep_new_pool(void **prealloc) { /* - * If the next pool is already saved or the maximum number of + * If a new pool is already saved or the maximum number of * pools is reached, do not use the preallocated memory. - * Access next_pool_required non-atomically, as there are no concurrent + * Access new_pool_required non-atomically, as there are no concurrent * write accesses to this variable. */ - if (!next_pool_required) + if (!new_pool_required) return; /* - * Use the preallocated memory for the next pool + * Use the preallocated memory for the new pool * as long as we do not exceed the maximum number of pools. */ if (pool_index + 1 < DEPOT_MAX_POOLS) { @@ -241,13 +240,13 @@ static void depot_keep_next_pool(void **prealloc) } /* - * At this point, either the next pool is kept or the maximum + * At this point, either a new pool is kept or the maximum * number of pools is reached. In either case, take note that * keeping another pool is not required. * smp_store_release() pairs with smp_load_acquire() in * stack_depot_save(). */ - smp_store_release(&next_pool_required, 0); + smp_store_release(&new_pool_required, 0); } /* Updates refences to the current and the next stack depot pools. */ @@ -262,7 +261,7 @@ static bool depot_update_pools(size_t required_size, void **prealloc) } /* - * Move on to the next pool. + * Move on to the new pool. * WRITE_ONCE() pairs with potential concurrent read in * stack_depot_fetch(). */ @@ -271,12 +270,12 @@ static bool depot_update_pools(size_t required_size, void **prealloc) /* * If the maximum number of pools is not reached, take note - * that the next pool needs to be initialized. + * that yet another new pool needs to be allocated. * smp_store_release() pairs with smp_load_acquire() in * stack_depot_save(). */ if (pool_index + 1 < DEPOT_MAX_POOLS) - smp_store_release(&next_pool_required, 1); + smp_store_release(&new_pool_required, 1); } /* Check if the current pool is not yet allocated. */ @@ -287,9 +286,9 @@ static bool depot_update_pools(size_t required_size, void **prealloc) return true; } - /* Otherwise, try using the preallocated memory for the next pool. */ + /* Otherwise, try using the preallocated memory for a new pool. */ if (*prealloc) - depot_keep_next_pool(prealloc); + depot_keep_new_pool(prealloc); return true; } @@ -300,7 +299,7 @@ depot_alloc_stack(unsigned long *entries, int size, u32 hash, void **prealloc) struct stack_record *stack; size_t required_size = DEPOT_STACK_RECORD_SIZE; - /* Update current and next pools if required and possible. */ + /* Update current and new pools if required and possible. */ if (!depot_update_pools(required_size, prealloc)) return NULL; @@ -432,13 +431,13 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, goto exit; /* - * Check if another stack pool needs to be initialized. If so, allocate - * the memory now - we won't be able to do that under the lock. + * Check if another stack pool needs to be allocated. If so, allocate + * the memory now: we won't be able to do that under the lock. * * smp_load_acquire() pairs with smp_store_release() in - * depot_update_pools() and depot_keep_next_pool(). + * depot_update_pools() and depot_keep_new_pool(). */ - if (unlikely(can_alloc && smp_load_acquire(&next_pool_required))) { + if (unlikely(can_alloc && smp_load_acquire(&new_pool_required))) { /* * Zero out zone modifiers, as we don't have specific zone * requirements. Keep the flags related to allocation in atomic @@ -471,9 +470,9 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, } else if (prealloc) { /* * Stack depot already contains this stack trace, but let's - * keep the preallocated memory for the future. + * keep the preallocated memory for future. */ - depot_keep_next_pool(&prealloc); + depot_keep_new_pool(&prealloc); } raw_spin_unlock_irqrestore(&pool_lock, flags); From patchwork Mon Oct 23 16:22:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 156990 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:ce89:0:b0:403:3b70:6f57 with SMTP id p9csp1412165vqx; Mon, 23 Oct 2023 09:30:03 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGNKfEwXCTXM1fYAwpoKmxA2qhUDA0+8yLAnI9CGcsqFisjLhdtJTEnkFFdptxoR/egzxt9 X-Received: by 2002:a17:902:f543:b0:1ca:77e9:3863 with SMTP id h3-20020a170902f54300b001ca77e93863mr11352692plf.31.1698078603182; Mon, 23 Oct 2023 09:30:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1698078603; cv=none; d=google.com; s=arc-20160816; b=yg2pN6ilzlP2zB3xQaqqDjiICXD05n39DJjALOc6MG3cuEqqHjcE1J3urKiFK40Lq2 oUR49/FoPRq0pCW6bSSZ2f6EgIX9URozbHuLdUimoTCr082oEZqUEmAIhJRA2WEI+NEX +1XPwg9maJyR8csFWW6FIZTsiyr+PZi8LtOe9SndxVccEJyDBlNgliDVeWhq018/vIv8 Slask3qguyxfed4CC5lvbJJJJFLV46Xvi1SVjOlqk0fJOTkDFN8PHr8lAOlqSeNK41ge JsoGF7CKC+WIUsyhNo1ESExdnsHiVxx6qmXO2CD+xbs2DpvtjJ8Cj+7KQ9X0za+nQMIr dfHA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=ApzhSzVlCGqOU4aCEVUKgk2/C6zStSgrVwNwdDBjs8U=; fh=nfQSbTp1dWHt2Ier1Up8UhVNdXOqoJJLNvTrpT3jJEk=; b=D5hr+XiNAg8NFYPWZ9wrLveFx0HSZX7PGXVQzJmA1rzFpx92DYi/iKODS/WsDIo6bD AGs+buoHx4QGT8T5Zqth0Qd8Wk5XUYYYOMcg8m/242Kqcj4M66dJ1BDyTqVep/EUZgSG +5ZrUkV7izPqw0SfPTx7f7Mom+KRaiC9eDNet2/ET/QuVBY1gzmtVGbSz4N8M0pjJSKl lMcx+UmvRma1EZbrdFfLrrQXdSaBOOyBPulePTR0O3ATMqpV6XzH1L2h85Lo2swvqkf6 95DN0KATCdpcSLODVittEcb0X8PztPbXCRe7d8j/vDYw7/I5AMCC+Xnygm6n8TiOa3NS Icag== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=NfBcevWE; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from lipwig.vger.email (lipwig.vger.email. [23.128.96.33]) by mx.google.com with ESMTPS id x1-20020a1709028ec100b001c62e465391si6489155plo.147.2023.10.23.09.30.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Oct 2023 09:30:03 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) client-ip=23.128.96.33; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=NfBcevWE; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id 887D080B26DC; Mon, 23 Oct 2023 09:29:57 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233631AbjJWQ3k (ORCPT + 27 others); Mon, 23 Oct 2023 12:29:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60536 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230255AbjJWQ3f (ORCPT ); Mon, 23 Oct 2023 12:29:35 -0400 X-Greylist: delayed 325 seconds by postgrey-1.37 at lindbergh.monkeyblade.net; Mon, 23 Oct 2023 09:29:31 PDT Received: from out-209.mta1.migadu.com (out-209.mta1.migadu.com [95.215.58.209]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B5799E5 for ; Mon, 23 Oct 2023 09:29:31 -0700 (PDT) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1698078245; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ApzhSzVlCGqOU4aCEVUKgk2/C6zStSgrVwNwdDBjs8U=; b=NfBcevWE9BlTOxp5WKU+AFsbwgvY+znjPFhBjCe3OZapZ0zGGWutAZrUfoCCoyqQvxbvaI R7EdE6nsCaw0PPJ/YMOZJqf2yEfy6oqmcAkClvCx2QV09MJBtPRaCYiYEtW9gHNizb9Lbe 030j3vL55nL81zukRlM4oSMetrRtY+M= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko Cc: Andrey Konovalov , Dmitry Vyukov , Vlastimil Babka , kasan-dev@googlegroups.com, Evgenii Stepanov , Oscar Salvador , Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v3 09/19] lib/stackdepot: store next pool pointer in new_pool Date: Mon, 23 Oct 2023 18:22:40 +0200 Message-Id: <852c5fed993f6b1e21beca9faa85e0fc2d9b84e6.1698077459.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Mon, 23 Oct 2023 09:29:57 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780564469487278366 X-GMAIL-MSGID: 1780564469487278366 From: Andrey Konovalov Instead of using the last pointer in stack_pools for storing the pointer to a new pool (which does not yet store any stack records), use a new new_pool variable. This a purely code readability change: it seems more logical to store the pointer to a pool with a special meaning in a dedicated variable. Reviewed-by: Alexander Potapenko Signed-off-by: Andrey Konovalov --- lib/stackdepot.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/lib/stackdepot.c b/lib/stackdepot.c index 7579e20114b1..5315952f26ec 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -85,6 +85,8 @@ static unsigned int stack_hash_mask; /* Array of memory regions that store stack traces. */ static void *stack_pools[DEPOT_MAX_POOLS]; +/* Newly allocated pool that is not yet added to stack_pools. */ +static void *new_pool; /* Currently used pool in stack_pools. */ static int pool_index; /* Offset to the unused space in the currently used pool. */ @@ -235,7 +237,7 @@ static void depot_keep_new_pool(void **prealloc) * as long as we do not exceed the maximum number of pools. */ if (pool_index + 1 < DEPOT_MAX_POOLS) { - stack_pools[pool_index + 1] = *prealloc; + new_pool = *prealloc; *prealloc = NULL; } @@ -266,6 +268,8 @@ static bool depot_update_pools(size_t required_size, void **prealloc) * stack_depot_fetch(). */ WRITE_ONCE(pool_index, pool_index + 1); + stack_pools[pool_index] = new_pool; + new_pool = NULL; pool_offset = 0; /* From patchwork Mon Oct 23 16:22:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 156988 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:ce89:0:b0:403:3b70:6f57 with SMTP id p9csp1412076vqx; Mon, 23 Oct 2023 09:29:56 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGT2Xq7rOuwJcNI/K0kG//jw/EjspVhG1BG8tYH/DBTYoWxLj0S2o3zoXh4+ndDlm1WCBDb X-Received: by 2002:a05:6359:b97:b0:168:e8bb:81a4 with SMTP id gf23-20020a0563590b9700b00168e8bb81a4mr1276415rwb.17.1698078596298; Mon, 23 Oct 2023 09:29:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1698078596; cv=none; d=google.com; s=arc-20160816; b=Mv8mqI5Ity7mx6juiFopp7V3d3bqLr5vX7psH6P/GYYqYeU1RUnwIxc0GCQYRcLajC CnlVjTrND/wyz/B1sX+Viz4oUgzfAA0kb9YWZSH44dsFUtmm36kqB3expuSfWLhyKpmb O0IluGyr8gGVfLpNDMuqme5HaWnWNFA1u9w/BhtlrWskA0VmB+fbIBkBIASmFKPkyTeT zwh/H8IR7pvelrFsH/CNv8TuQBG80VTkXdQ56cZLd4XC7/PRVe2MJ6xM1SrCq/Idzdb+ 5ZdCUllYqs1wbkZZOpCLjMGYyrEOPjXbimyR+yq+x2YBCFXmnsZK3iUVOPX14qwGEpPy /dXA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=hVpUL5x06hsk3ZkSjFGdljEHgMA5oAygtOmJMO9tzo8=; fh=nfQSbTp1dWHt2Ier1Up8UhVNdXOqoJJLNvTrpT3jJEk=; b=rSoh9n12Y+nAYaIeQClAOco9KSC3J+BTDt/p8o5ALXl0IDvTj4f5c/4vxJ1IhtKJ2o F/P/W+CVPGARUVl/haAAUM2+fnDvkZ1Z89ql7TtJywe8TYem+C5b1VkFpANIPycAdqZ0 Fr5unQu2JlaYMoWg3qlLjyUo0A8P7UA6mDgj/nyW2yO2ymm3QNrORnH4P7b+E3bbsU4g r6m1E0jtu7rC0VtAouqWxH4ganEMW6Jx2QXJcfs6KK/qpvd2aWaxV8+2JZPpYXWIw/EG 66GjxJ4ujE5YRp9dynZnkEJzhoIrkqJi2YdC2x5rhwv37wxk61cfQvUTCYANak8lvxGS E9XA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=iTY6JYsU; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from pete.vger.email (pete.vger.email. [23.128.96.36]) by mx.google.com with ESMTPS id 200-20020a6301d1000000b00565e39e7b80si6550284pgb.678.2023.10.23.09.29.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Oct 2023 09:29:56 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) client-ip=23.128.96.36; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=iTY6JYsU; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by pete.vger.email (Postfix) with ESMTP id 8D1B1805B20A; Mon, 23 Oct 2023 09:29:49 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at pete.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230201AbjJWQ3e (ORCPT + 27 others); Mon, 23 Oct 2023 12:29:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60506 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229544AbjJWQ3d (ORCPT ); Mon, 23 Oct 2023 12:29:33 -0400 Received: from out-201.mta1.migadu.com (out-201.mta1.migadu.com [IPv6:2001:41d0:203:375::c9]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A3319B4 for ; Mon, 23 Oct 2023 09:29:31 -0700 (PDT) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1698078245; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hVpUL5x06hsk3ZkSjFGdljEHgMA5oAygtOmJMO9tzo8=; b=iTY6JYsUY35XrAxNWjoB6AUfx0WkIdXZLDRXvdjLJBdlA2H9B3Sv8PN27k3Rj2k9vwFZ/Q kYu4cNJ28hwFR0icW3cybkceDhoOMLD3TZkJREMB1OdD+PpAjqP9FOskxFAS7mpSmwYFq0 R3sOIGWwVN6+425LalBAwW/O0YRDfck= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko Cc: Andrey Konovalov , Dmitry Vyukov , Vlastimil Babka , kasan-dev@googlegroups.com, Evgenii Stepanov , Oscar Salvador , Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v3 10/19] lib/stackdepot: store free stack records in a freelist Date: Mon, 23 Oct 2023 18:22:41 +0200 Message-Id: <0e798c0431646643ce077117933e8a79899a2403.1698077459.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Mon, 23 Oct 2023 09:29:49 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780564462103111956 X-GMAIL-MSGID: 1780564462103111956 From: Andrey Konovalov Instead of using the global pool_offset variable to find a free slot when storing a new stack record, mainlain a freelist of free slots within the allocated stack pools. A global next_stack variable is used as the head of the freelist, and the next field in the stack_record struct is reused as freelist link (when the record is not in the freelist, this field is used as a link in the hash table). This is preparatory patch for implementing the eviction of stack records from the stack depot. Reviewed-by: Alexander Potapenko Signed-off-by: Andrey Konovalov --- Changes v2->v3: - Add parentheses when referring to function calls in comments. Changes v1->v2: - Fix out-of-bounds when initializing a pool. --- lib/stackdepot.c | 131 +++++++++++++++++++++++++++++------------------ 1 file changed, 82 insertions(+), 49 deletions(-) diff --git a/lib/stackdepot.c b/lib/stackdepot.c index 5315952f26ec..1067c072a0f8 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -54,8 +54,8 @@ union handle_parts { }; struct stack_record { - struct stack_record *next; /* Link in the hash table */ - u32 hash; /* Hash in the hash table */ + struct stack_record *next; /* Link in hash table or freelist */ + u32 hash; /* Hash in hash table */ u32 size; /* Number of stored frames */ union handle_parts handle; unsigned long entries[CONFIG_STACKDEPOT_MAX_FRAMES]; /* Frames */ @@ -87,10 +87,10 @@ static unsigned int stack_hash_mask; static void *stack_pools[DEPOT_MAX_POOLS]; /* Newly allocated pool that is not yet added to stack_pools. */ static void *new_pool; -/* Currently used pool in stack_pools. */ -static int pool_index; -/* Offset to the unused space in the currently used pool. */ -static size_t pool_offset; +/* Number of pools in stack_pools. */ +static int pools_num; +/* Next stack in the freelist of stack records within stack_pools. */ +static struct stack_record *next_stack; /* Lock that protects the variables above. */ static DEFINE_RAW_SPINLOCK(pool_lock); /* @@ -220,6 +220,42 @@ int stack_depot_init(void) } EXPORT_SYMBOL_GPL(stack_depot_init); +/* Initializes a stack depol pool. */ +static void depot_init_pool(void *pool) +{ + const int records_in_pool = DEPOT_POOL_SIZE / DEPOT_STACK_RECORD_SIZE; + int i, offset; + + /* Initialize handles and link stack records to each other. */ + for (i = 0, offset = 0; + offset <= DEPOT_POOL_SIZE - DEPOT_STACK_RECORD_SIZE; + i++, offset += DEPOT_STACK_RECORD_SIZE) { + struct stack_record *stack = pool + offset; + + stack->handle.pool_index = pools_num; + stack->handle.offset = offset >> DEPOT_STACK_ALIGN; + stack->handle.extra = 0; + + if (i < records_in_pool - 1) + stack->next = (void *)stack + DEPOT_STACK_RECORD_SIZE; + else + stack->next = NULL; + } + + /* Link stack records into the freelist. */ + WARN_ON(next_stack); + next_stack = pool; + + /* Save reference to the pool to be used by depot_fetch_stack(). */ + stack_pools[pools_num] = pool; + + /* + * WRITE_ONCE() pairs with potential concurrent read in + * depot_fetch_stack(). + */ + WRITE_ONCE(pools_num, pools_num + 1); +} + /* Keeps the preallocated memory to be used for a new stack depot pool. */ static void depot_keep_new_pool(void **prealloc) { @@ -236,7 +272,7 @@ static void depot_keep_new_pool(void **prealloc) * Use the preallocated memory for the new pool * as long as we do not exceed the maximum number of pools. */ - if (pool_index + 1 < DEPOT_MAX_POOLS) { + if (pools_num < DEPOT_MAX_POOLS) { new_pool = *prealloc; *prealloc = NULL; } @@ -252,45 +288,42 @@ static void depot_keep_new_pool(void **prealloc) } /* Updates refences to the current and the next stack depot pools. */ -static bool depot_update_pools(size_t required_size, void **prealloc) +static bool depot_update_pools(void **prealloc) { - /* Check if there is not enough space in the current pool. */ - if (unlikely(pool_offset + required_size > DEPOT_POOL_SIZE)) { - /* Bail out if we reached the pool limit. */ - if (unlikely(pool_index + 1 >= DEPOT_MAX_POOLS)) { - WARN_ONCE(1, "Stack depot reached limit capacity"); - return false; - } + /* Check if we still have objects in the freelist. */ + if (next_stack) + goto out_keep_prealloc; - /* - * Move on to the new pool. - * WRITE_ONCE() pairs with potential concurrent read in - * stack_depot_fetch(). - */ - WRITE_ONCE(pool_index, pool_index + 1); - stack_pools[pool_index] = new_pool; + /* Check if we have a new pool saved and use it. */ + if (new_pool) { + depot_init_pool(new_pool); new_pool = NULL; - pool_offset = 0; - /* - * If the maximum number of pools is not reached, take note - * that yet another new pool needs to be allocated. - * smp_store_release() pairs with smp_load_acquire() in - * stack_depot_save(). - */ - if (pool_index + 1 < DEPOT_MAX_POOLS) + /* Take note that we might need a new new_pool. */ + if (pools_num < DEPOT_MAX_POOLS) smp_store_release(&new_pool_required, 1); + + /* Try keeping the preallocated memory for new_pool. */ + goto out_keep_prealloc; + } + + /* Bail out if we reached the pool limit. */ + if (unlikely(pools_num >= DEPOT_MAX_POOLS)) { + WARN_ONCE(1, "Stack depot reached limit capacity"); + return false; } - /* Check if the current pool is not yet allocated. */ - if (*prealloc && stack_pools[pool_index] == NULL) { - /* Use the preallocated memory for the current pool. */ - stack_pools[pool_index] = *prealloc; + /* Check if we have preallocated memory and use it. */ + if (*prealloc) { + depot_init_pool(*prealloc); *prealloc = NULL; return true; } - /* Otherwise, try using the preallocated memory for a new pool. */ + return false; + +out_keep_prealloc: + /* Keep the preallocated memory for a new pool if required. */ if (*prealloc) depot_keep_new_pool(prealloc); return true; @@ -301,35 +334,35 @@ static struct stack_record * depot_alloc_stack(unsigned long *entries, int size, u32 hash, void **prealloc) { struct stack_record *stack; - size_t required_size = DEPOT_STACK_RECORD_SIZE; /* Update current and new pools if required and possible. */ - if (!depot_update_pools(required_size, prealloc)) + if (!depot_update_pools(prealloc)) return NULL; - /* Check if we have a pool to save the stack trace. */ - if (stack_pools[pool_index] == NULL) + /* Check if we have a stack record to save the stack trace. */ + stack = next_stack; + if (!stack) return NULL; + /* Advance the freelist. */ + next_stack = stack->next; + /* Limit number of saved frames to CONFIG_STACKDEPOT_MAX_FRAMES. */ if (size > CONFIG_STACKDEPOT_MAX_FRAMES) size = CONFIG_STACKDEPOT_MAX_FRAMES; /* Save the stack trace. */ - stack = stack_pools[pool_index] + pool_offset; + stack->next = NULL; stack->hash = hash; stack->size = size; - stack->handle.pool_index = pool_index; - stack->handle.offset = pool_offset >> DEPOT_STACK_ALIGN; - stack->handle.extra = 0; + /* stack->handle is already filled in by depot_init_pool(). */ memcpy(stack->entries, entries, flex_array_size(stack, entries, size)); - pool_offset += required_size; /* * Let KMSAN know the stored stack record is initialized. This shall * prevent false positive reports if instrumented code accesses it. */ - kmsan_unpoison_memory(stack, required_size); + kmsan_unpoison_memory(stack, DEPOT_STACK_RECORD_SIZE); return stack; } @@ -339,16 +372,16 @@ static struct stack_record *depot_fetch_stack(depot_stack_handle_t handle) union handle_parts parts = { .handle = handle }; /* * READ_ONCE() pairs with potential concurrent write in - * depot_update_pools(). + * depot_init_pool(). */ - int pool_index_cached = READ_ONCE(pool_index); + int pools_num_cached = READ_ONCE(pools_num); void *pool; size_t offset = parts.offset << DEPOT_STACK_ALIGN; struct stack_record *stack; - if (parts.pool_index > pool_index_cached) { + if (parts.pool_index > pools_num_cached) { WARN(1, "pool index %d out of bounds (%d) for stack id %08x\n", - parts.pool_index, pool_index_cached, handle); + parts.pool_index, pools_num_cached, handle); return NULL; } From patchwork Mon Oct 23 16:22:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 156995 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:ce89:0:b0:403:3b70:6f57 with SMTP id p9csp1413411vqx; Mon, 23 Oct 2023 09:31:48 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGuBwihty1GB+mdY6Pa8dCTBnEIrHbzJ/c2AdesQNgxvMFriUBi+obaHfN5Sl47jzrQjheI X-Received: by 2002:a17:903:1112:b0:1c6:117b:7086 with SMTP id n18-20020a170903111200b001c6117b7086mr10795420plh.5.1698078708063; Mon, 23 Oct 2023 09:31:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1698078708; cv=none; d=google.com; s=arc-20160816; b=IZtGBfKjaeRgo3wH4Ntmiq5BYXHhyoQDkJR/zLucN6ZsyDoIYRYiFeIF3siRlicEJC 5n8jVbLsMDHdYtps6CNJSiYctKyB/UxI9ldtMVmlonPmoEgKCFQwr6fgdhbekdA5n/HM 44YhqFuWbwWCkUe9F+V3sgJDbXn7u09VSXRXNixMLyx+LVYLejOR5HbTVHDJq24Q+p3o d9ju5L1IlaFyPGNb5sg5mgo1RY7hoYYF+WonEUh5jQbv/TIVI+4Zj5d06isffN6asazj 8YAGMt5U2rt0fwYp6vWZ5U78xHYdvK3Kk5ozQ6WN7ECHlOSIZoC1GCeWRO8SkuT2goHt 2Rdw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=p8Hgl8kIxqRiPg7P/caKlN6EZe6EuOrMJBszDGSf5JA=; fh=nfQSbTp1dWHt2Ier1Up8UhVNdXOqoJJLNvTrpT3jJEk=; b=pBeSqBDR9T4zODWvxOwFPd3Uf0Cb/Aq9YWRr91z0gXyukNSx0gfVtnTkU+dyDsaO+1 nt65QwHVcOCMoTSKdLp9sgGAsn90YsQWZ8ZZ2ZybtJzHT8UbLQtpd+EH0056bXTU8mTr BIjC34uoifvLxieqE6kVzyi5veQ50y9QLFl6tepRba9w9vexdTJdWj9lC6iBYuegr0AR n8WD6A79wwr2ZIWrf/8UMlB49u2GE/fxI10TuSqZHEIzaLnWqi7s4gz3AFfh9q0BAdFT SIjGKDzZw2AqiF6aWbBtahL6ZKVa6/QrjVP8ElJYM+NCjcmJ7BY+t26EppJlDtclSWvF /9KQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=d8BAggel; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from howler.vger.email (howler.vger.email. [23.128.96.34]) by mx.google.com with ESMTPS id p6-20020a170902e74600b001c72476c96fsi6748610plf.143.2023.10.23.09.31.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Oct 2023 09:31:48 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) client-ip=23.128.96.34; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=d8BAggel; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id 6B80080A1E0F; Mon, 23 Oct 2023 09:30:03 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232697AbjJWQ3g (ORCPT + 27 others); Mon, 23 Oct 2023 12:29:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60522 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229690AbjJWQ3e (ORCPT ); Mon, 23 Oct 2023 12:29:34 -0400 Received: from out-201.mta1.migadu.com (out-201.mta1.migadu.com [IPv6:2001:41d0:203:375::c9]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 860E28E for ; Mon, 23 Oct 2023 09:29:31 -0700 (PDT) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1698078246; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=p8Hgl8kIxqRiPg7P/caKlN6EZe6EuOrMJBszDGSf5JA=; b=d8BAggels5u67sU3a1DsLYB3mzDLPzqPZYgCDFI/IUfM7R5EyUYByQkE3kU+k9PyxTsmM6 GNUYqd5pUyWDQggTtDoeOwneQ3CQxbmr4n6O3nWwZpqYNBDV2DNnDg12CvGqgMggwxUqrA 379T75KzzDunwidWZHcC0GAmFd/27C8= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko Cc: Andrey Konovalov , Dmitry Vyukov , Vlastimil Babka , kasan-dev@googlegroups.com, Evgenii Stepanov , Oscar Salvador , Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v3 11/19] lib/stackdepot: use read/write lock Date: Mon, 23 Oct 2023 18:22:42 +0200 Message-Id: <68b90916541c09fd64de22dd4666b53172f7e618.1698077459.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on howler.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Mon, 23 Oct 2023 09:30:03 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780564579338996788 X-GMAIL-MSGID: 1780564579338996788 From: Andrey Konovalov Currently, stack depot uses the following locking scheme: 1. Lock-free accesses when looking up a stack record, which allows to have multiple users to look up records in parallel; 2. Spinlock for protecting the stack depot pools and the hash table when adding a new record. For implementing the eviction of stack traces from stack depot, the lock-free approach is not going to work anymore, as we will need to be able to also remove records from the hash table. Convert the spinlock into a read/write lock, and drop the atomic accesses, as they are no longer required. Looking up stack traces is now protected by the read lock and adding new records - by the write lock. One of the following patches will add a new function for evicting stack records, which will be protected by the write lock as well. With this change, multiple users can still look up records in parallel. This is preparatory patch for implementing the eviction of stack records from the stack depot. Reviewed-by: Alexander Potapenko Signed-off-by: Andrey Konovalov --- Changed v2->v3: - Use lockdep_assert_held_read annotation in depot_fetch_stack. Changes v1->v2: - Add lockdep_assert annotations. --- lib/stackdepot.c | 87 +++++++++++++++++++++++++----------------------- 1 file changed, 46 insertions(+), 41 deletions(-) diff --git a/lib/stackdepot.c b/lib/stackdepot.c index 1067c072a0f8..b1ceade0acc9 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -23,6 +23,7 @@ #include #include #include +#include #include #include #include @@ -91,15 +92,15 @@ static void *new_pool; static int pools_num; /* Next stack in the freelist of stack records within stack_pools. */ static struct stack_record *next_stack; -/* Lock that protects the variables above. */ -static DEFINE_RAW_SPINLOCK(pool_lock); /* * Stack depot tries to keep an extra pool allocated even before it runs out * of space in the currently used pool. This flag marks whether this extra pool * needs to be allocated. It has the value 0 when either an extra pool is not * yet allocated or if the limit on the number of pools is reached. */ -static int new_pool_required = 1; +static bool new_pool_required = true; +/* Lock that protects the variables above. */ +static DEFINE_RWLOCK(pool_rwlock); static int __init disable_stack_depot(char *str) { @@ -226,6 +227,8 @@ static void depot_init_pool(void *pool) const int records_in_pool = DEPOT_POOL_SIZE / DEPOT_STACK_RECORD_SIZE; int i, offset; + lockdep_assert_held_write(&pool_rwlock); + /* Initialize handles and link stack records to each other. */ for (i = 0, offset = 0; offset <= DEPOT_POOL_SIZE - DEPOT_STACK_RECORD_SIZE; @@ -248,22 +251,17 @@ static void depot_init_pool(void *pool) /* Save reference to the pool to be used by depot_fetch_stack(). */ stack_pools[pools_num] = pool; - - /* - * WRITE_ONCE() pairs with potential concurrent read in - * depot_fetch_stack(). - */ - WRITE_ONCE(pools_num, pools_num + 1); + pools_num++; } /* Keeps the preallocated memory to be used for a new stack depot pool. */ static void depot_keep_new_pool(void **prealloc) { + lockdep_assert_held_write(&pool_rwlock); + /* * If a new pool is already saved or the maximum number of * pools is reached, do not use the preallocated memory. - * Access new_pool_required non-atomically, as there are no concurrent - * write accesses to this variable. */ if (!new_pool_required) return; @@ -281,15 +279,15 @@ static void depot_keep_new_pool(void **prealloc) * At this point, either a new pool is kept or the maximum * number of pools is reached. In either case, take note that * keeping another pool is not required. - * smp_store_release() pairs with smp_load_acquire() in - * stack_depot_save(). */ - smp_store_release(&new_pool_required, 0); + new_pool_required = false; } /* Updates refences to the current and the next stack depot pools. */ static bool depot_update_pools(void **prealloc) { + lockdep_assert_held_write(&pool_rwlock); + /* Check if we still have objects in the freelist. */ if (next_stack) goto out_keep_prealloc; @@ -301,7 +299,7 @@ static bool depot_update_pools(void **prealloc) /* Take note that we might need a new new_pool. */ if (pools_num < DEPOT_MAX_POOLS) - smp_store_release(&new_pool_required, 1); + new_pool_required = true; /* Try keeping the preallocated memory for new_pool. */ goto out_keep_prealloc; @@ -335,6 +333,8 @@ depot_alloc_stack(unsigned long *entries, int size, u32 hash, void **prealloc) { struct stack_record *stack; + lockdep_assert_held_write(&pool_rwlock); + /* Update current and new pools if required and possible. */ if (!depot_update_pools(prealloc)) return NULL; @@ -370,18 +370,15 @@ depot_alloc_stack(unsigned long *entries, int size, u32 hash, void **prealloc) static struct stack_record *depot_fetch_stack(depot_stack_handle_t handle) { union handle_parts parts = { .handle = handle }; - /* - * READ_ONCE() pairs with potential concurrent write in - * depot_init_pool(). - */ - int pools_num_cached = READ_ONCE(pools_num); void *pool; size_t offset = parts.offset << DEPOT_STACK_ALIGN; struct stack_record *stack; - if (parts.pool_index > pools_num_cached) { + lockdep_assert_held_read(&pool_rwlock); + + if (parts.pool_index > pools_num) { WARN(1, "pool index %d out of bounds (%d) for stack id %08x\n", - parts.pool_index, pools_num_cached, handle); + parts.pool_index, pools_num, handle); return NULL; } @@ -423,6 +420,8 @@ static inline struct stack_record *find_stack(struct stack_record *bucket, { struct stack_record *found; + lockdep_assert_held(&pool_rwlock); + for (found = bucket; found; found = found->next) { if (found->hash == hash && found->size == size && @@ -440,6 +439,7 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, depot_stack_handle_t handle = 0; struct page *page = NULL; void *prealloc = NULL; + bool need_alloc = false; unsigned long flags; u32 hash; @@ -459,22 +459,26 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, hash = hash_stack(entries, nr_entries); bucket = &stack_table[hash & stack_hash_mask]; - /* - * Fast path: look the stack trace up without locking. - * smp_load_acquire() pairs with smp_store_release() to |bucket| below. - */ - found = find_stack(smp_load_acquire(bucket), entries, nr_entries, hash); - if (found) + read_lock_irqsave(&pool_rwlock, flags); + + /* Fast path: look the stack trace up without full locking. */ + found = find_stack(*bucket, entries, nr_entries, hash); + if (found) { + read_unlock_irqrestore(&pool_rwlock, flags); goto exit; + } + + /* Take note if another stack pool needs to be allocated. */ + if (new_pool_required) + need_alloc = true; + + read_unlock_irqrestore(&pool_rwlock, flags); /* - * Check if another stack pool needs to be allocated. If so, allocate - * the memory now: we won't be able to do that under the lock. - * - * smp_load_acquire() pairs with smp_store_release() in - * depot_update_pools() and depot_keep_new_pool(). + * Allocate memory for a new pool if required now: + * we won't be able to do that under the lock. */ - if (unlikely(can_alloc && smp_load_acquire(&new_pool_required))) { + if (unlikely(can_alloc && need_alloc)) { /* * Zero out zone modifiers, as we don't have specific zone * requirements. Keep the flags related to allocation in atomic @@ -488,7 +492,7 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, prealloc = page_address(page); } - raw_spin_lock_irqsave(&pool_lock, flags); + write_lock_irqsave(&pool_rwlock, flags); found = find_stack(*bucket, entries, nr_entries, hash); if (!found) { @@ -497,11 +501,7 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, if (new) { new->next = *bucket; - /* - * smp_store_release() pairs with smp_load_acquire() - * from |bucket| above. - */ - smp_store_release(bucket, new); + *bucket = new; found = new; } } else if (prealloc) { @@ -512,7 +512,7 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, depot_keep_new_pool(&prealloc); } - raw_spin_unlock_irqrestore(&pool_lock, flags); + write_unlock_irqrestore(&pool_rwlock, flags); exit: if (prealloc) { /* Stack depot didn't use this memory, free it. */ @@ -536,6 +536,7 @@ unsigned int stack_depot_fetch(depot_stack_handle_t handle, unsigned long **entries) { struct stack_record *stack; + unsigned long flags; *entries = NULL; /* @@ -547,8 +548,12 @@ unsigned int stack_depot_fetch(depot_stack_handle_t handle, if (!handle || stack_depot_disabled) return 0; + read_lock_irqsave(&pool_rwlock, flags); + stack = depot_fetch_stack(handle); + read_unlock_irqrestore(&pool_rwlock, flags); + *entries = stack->entries; return stack->size; } From patchwork Mon Oct 23 16:22:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 156981 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:ce89:0:b0:403:3b70:6f57 with SMTP id p9csp1410000vqx; Mon, 23 Oct 2023 09:26:05 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFHyoSiWMekwtVag2q4/Zz0LlRqP5iEYcD4UIcnivd9TqFhHgMDbB7t0XCFQ5MqC7Sn1T7J X-Received: by 2002:a05:6a20:7fa4:b0:163:4288:1c3d with SMTP id d36-20020a056a207fa400b0016342881c3dmr87694pzj.43.1698078365183; Mon, 23 Oct 2023 09:26:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1698078365; cv=none; d=google.com; s=arc-20160816; b=oWqEC7cU9HhFVTw1EYnvv3qvAY0stv6LLKYM4pqFxTpwyUF0QFEDMkr4yFg6u/Gdjy gfy9wv3jZYxLvYsNbrEtz0VCIDZgr2t8XT/NVvzgau/1pVPwAizNc4ZOK16+k0t8GL5Z UNOaMSRmX2daVCkQvN4o1lBorS9y8sSnM57fpy4uxl3doH9sVSi5s7jG+irQ/kznuRko VYv4pWmzRQBMXrk3Zi3TkkX2Hf5Dvn2sXlv0IFAo9Q/ebnWMTbkkhpMWVoKyPinoxMRI WGqVXTUzkB4FhaAZuFY1qOjjnzGF7dTroJizUXx6l2Wxm/ww238zFlSDn86SnGKy26YI wgOQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=kwAPQEKRgU/QjlEBZ2MhqIx/KOtst15cg0pwz3Hyr9g=; fh=nfQSbTp1dWHt2Ier1Up8UhVNdXOqoJJLNvTrpT3jJEk=; b=CzX91/25cJCXzgAsKc07S9WLSpV5S+Di4lDABfuOQhm4u32ozszqsT7TLU7encOYW9 k5fEgIcIwh+jJmApwYOjr3qulIB79+h24hBrXWvR92XO0ZVAT5Dw1MJw2SEr30Fg3XmU uOJSKHsKuyiRm6gPw3/E8FGhajn/9OKplzreOqJm4x3S606URRMhA8UVE1KsRnmiDTZk gIrxMa2lUAPEi/tq82bgMd9nAHij8uJcOsaSxXTFF+omZ4OlD5GMF0Ri5zKC2rfUDEDj KompmyQw66GsgVWRX2IV+6hAAMpV+dvG5ljHJHsVrzzmqvEtDvwBGLeSFESxRnNVdXsI jqGw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=iGUJS+rO; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from morse.vger.email (morse.vger.email. [23.128.96.31]) by mx.google.com with ESMTPS id bv125-20020a632e83000000b005859c81f1e9si6683345pgb.229.2023.10.23.09.26.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Oct 2023 09:26:05 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) client-ip=23.128.96.31; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=iGUJS+rO; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by morse.vger.email (Postfix) with ESMTP id 0DDD98095DED; Mon, 23 Oct 2023 09:25:53 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at morse.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233581AbjJWQZQ (ORCPT + 27 others); Mon, 23 Oct 2023 12:25:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39742 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232058AbjJWQZM (ORCPT ); Mon, 23 Oct 2023 12:25:12 -0400 X-Greylist: delayed 124 seconds by postgrey-1.37 at lindbergh.monkeyblade.net; Mon, 23 Oct 2023 09:25:10 PDT Received: from out-202.mta0.migadu.com (out-202.mta0.migadu.com [IPv6:2001:41d0:1004:224b::ca]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1A324E5 for ; Mon, 23 Oct 2023 09:25:09 -0700 (PDT) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1698078308; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kwAPQEKRgU/QjlEBZ2MhqIx/KOtst15cg0pwz3Hyr9g=; b=iGUJS+rOn2dXj8f4BIY+WwdB8Mdqb1aEQqfE+6SqjcK1mxQ+u4/MVV6bams1FgB2otyMtr VknTKi9tknQ69VcwKIOlmvNy0fBatX3KvqxEFyA/LAWA33e35h42LQLvfNWq4n+5VujTpi 4WErb4Df30D2FUv9R93/ziOe2SgmPy4= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko Cc: Andrey Konovalov , Dmitry Vyukov , Vlastimil Babka , kasan-dev@googlegroups.com, Evgenii Stepanov , Oscar Salvador , Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v3 12/19] lib/stackdepot: use list_head for stack record links Date: Mon, 23 Oct 2023 18:22:43 +0200 Message-Id: <518e3873243845249c8fb019d744c5f5eac90205.1698077459.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on morse.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (morse.vger.email [0.0.0.0]); Mon, 23 Oct 2023 09:25:53 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780564219665716137 X-GMAIL-MSGID: 1780564219665716137 From: Andrey Konovalov Switch stack_record to use list_head for links in the hash table and in the freelist. This will allow removing entries from the hash table buckets. This is preparatory patch for implementing the eviction of stack records from the stack depot. Signed-off-by: Andrey Konovalov --- Changes v2->v3: - Use the proper number of entries for initializing the stack table when alloc_large_system_hash() auto-calculates the number. Changes v1->v2: - Use list_head instead of open-coding backward links. --- lib/stackdepot.c | 87 ++++++++++++++++++++++++++++-------------------- 1 file changed, 50 insertions(+), 37 deletions(-) diff --git a/lib/stackdepot.c b/lib/stackdepot.c index b1ceade0acc9..85fd40c63817 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -18,6 +18,7 @@ #include #include #include +#include #include #include #include @@ -55,7 +56,7 @@ union handle_parts { }; struct stack_record { - struct stack_record *next; /* Link in hash table or freelist */ + struct list_head list; /* Links in hash table or freelist */ u32 hash; /* Hash in hash table */ u32 size; /* Number of stored frames */ union handle_parts handle; @@ -77,21 +78,21 @@ static bool __stack_depot_early_init_passed __initdata; /* Initial seed for jhash2. */ #define STACK_HASH_SEED 0x9747b28c -/* Hash table of pointers to stored stack traces. */ -static struct stack_record **stack_table; +/* Hash table of stored stack records. */ +static struct list_head *stack_table; /* Fixed order of the number of table buckets. Used when KASAN is enabled. */ static unsigned int stack_bucket_number_order; /* Hash mask for indexing the table. */ static unsigned int stack_hash_mask; -/* Array of memory regions that store stack traces. */ +/* Array of memory regions that store stack records. */ static void *stack_pools[DEPOT_MAX_POOLS]; /* Newly allocated pool that is not yet added to stack_pools. */ static void *new_pool; /* Number of pools in stack_pools. */ static int pools_num; -/* Next stack in the freelist of stack records within stack_pools. */ -static struct stack_record *next_stack; +/* Freelist of stack records within stack_pools. */ +static LIST_HEAD(free_stacks); /* * Stack depot tries to keep an extra pool allocated even before it runs out * of space in the currently used pool. This flag marks whether this extra pool @@ -123,6 +124,15 @@ void __init stack_depot_request_early_init(void) __stack_depot_early_init_requested = true; } +/* Initialize list_head's within the hash table. */ +static void init_stack_table(unsigned long entries) +{ + unsigned long i; + + for (i = 0; i < entries; i++) + INIT_LIST_HEAD(&stack_table[i]); +} + /* Allocates a hash table via memblock. Can only be used during early boot. */ int __init stack_depot_early_init(void) { @@ -146,16 +156,16 @@ int __init stack_depot_early_init(void) /* * If stack_bucket_number_order is not set, leave entries as 0 to rely - * on the automatic calculations performed by alloc_large_system_hash. + * on the automatic calculations performed by alloc_large_system_hash(). */ if (stack_bucket_number_order) entries = 1UL << stack_bucket_number_order; pr_info("allocating hash table via alloc_large_system_hash\n"); stack_table = alloc_large_system_hash("stackdepot", - sizeof(struct stack_record *), + sizeof(struct list_head), entries, STACK_HASH_TABLE_SCALE, - HASH_EARLY | HASH_ZERO, + HASH_EARLY, NULL, &stack_hash_mask, 1UL << STACK_BUCKET_NUMBER_ORDER_MIN, @@ -165,6 +175,14 @@ int __init stack_depot_early_init(void) stack_depot_disabled = true; return -ENOMEM; } + if (!entries) { + /* + * Obtain the number of entries that was calculated by + * alloc_large_system_hash(). + */ + entries = stack_hash_mask + 1; + } + init_stack_table(entries); return 0; } @@ -205,7 +223,7 @@ int stack_depot_init(void) entries = 1UL << STACK_BUCKET_NUMBER_ORDER_MAX; pr_info("allocating hash table of %lu entries via kvcalloc\n", entries); - stack_table = kvcalloc(entries, sizeof(struct stack_record *), GFP_KERNEL); + stack_table = kvcalloc(entries, sizeof(struct list_head), GFP_KERNEL); if (!stack_table) { pr_err("hash table allocation failed, disabling\n"); stack_depot_disabled = true; @@ -213,6 +231,7 @@ int stack_depot_init(void) goto out_unlock; } stack_hash_mask = entries - 1; + init_stack_table(entries); out_unlock: mutex_unlock(&stack_depot_init_mutex); @@ -224,31 +243,24 @@ EXPORT_SYMBOL_GPL(stack_depot_init); /* Initializes a stack depol pool. */ static void depot_init_pool(void *pool) { - const int records_in_pool = DEPOT_POOL_SIZE / DEPOT_STACK_RECORD_SIZE; - int i, offset; + int offset; lockdep_assert_held_write(&pool_rwlock); - /* Initialize handles and link stack records to each other. */ - for (i = 0, offset = 0; - offset <= DEPOT_POOL_SIZE - DEPOT_STACK_RECORD_SIZE; - i++, offset += DEPOT_STACK_RECORD_SIZE) { + WARN_ON(!list_empty(&free_stacks)); + + /* Initialize handles and link stack records into the freelist. */ + for (offset = 0; offset <= DEPOT_POOL_SIZE - DEPOT_STACK_RECORD_SIZE; + offset += DEPOT_STACK_RECORD_SIZE) { struct stack_record *stack = pool + offset; stack->handle.pool_index = pools_num; stack->handle.offset = offset >> DEPOT_STACK_ALIGN; stack->handle.extra = 0; - if (i < records_in_pool - 1) - stack->next = (void *)stack + DEPOT_STACK_RECORD_SIZE; - else - stack->next = NULL; + list_add(&stack->list, &free_stacks); } - /* Link stack records into the freelist. */ - WARN_ON(next_stack); - next_stack = pool; - /* Save reference to the pool to be used by depot_fetch_stack(). */ stack_pools[pools_num] = pool; pools_num++; @@ -289,7 +301,7 @@ static bool depot_update_pools(void **prealloc) lockdep_assert_held_write(&pool_rwlock); /* Check if we still have objects in the freelist. */ - if (next_stack) + if (!list_empty(&free_stacks)) goto out_keep_prealloc; /* Check if we have a new pool saved and use it. */ @@ -340,19 +352,18 @@ depot_alloc_stack(unsigned long *entries, int size, u32 hash, void **prealloc) return NULL; /* Check if we have a stack record to save the stack trace. */ - stack = next_stack; - if (!stack) + if (list_empty(&free_stacks)) return NULL; - /* Advance the freelist. */ - next_stack = stack->next; + /* Get and unlink the first entry from the freelist. */ + stack = list_first_entry(&free_stacks, struct stack_record, list); + list_del(&stack->list); /* Limit number of saved frames to CONFIG_STACKDEPOT_MAX_FRAMES. */ if (size > CONFIG_STACKDEPOT_MAX_FRAMES) size = CONFIG_STACKDEPOT_MAX_FRAMES; /* Save the stack trace. */ - stack->next = NULL; stack->hash = hash; stack->size = size; /* stack->handle is already filled in by depot_init_pool(). */ @@ -414,15 +425,17 @@ int stackdepot_memcmp(const unsigned long *u1, const unsigned long *u2, } /* Finds a stack in a bucket of the hash table. */ -static inline struct stack_record *find_stack(struct stack_record *bucket, +static inline struct stack_record *find_stack(struct list_head *bucket, unsigned long *entries, int size, u32 hash) { + struct list_head *pos; struct stack_record *found; lockdep_assert_held(&pool_rwlock); - for (found = bucket; found; found = found->next) { + list_for_each(pos, bucket) { + found = list_entry(pos, struct stack_record, list); if (found->hash == hash && found->size == size && !stackdepot_memcmp(entries, found->entries, size)) @@ -435,7 +448,8 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, unsigned int nr_entries, gfp_t alloc_flags, bool can_alloc) { - struct stack_record *found = NULL, **bucket; + struct list_head *bucket; + struct stack_record *found = NULL; depot_stack_handle_t handle = 0; struct page *page = NULL; void *prealloc = NULL; @@ -462,7 +476,7 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, read_lock_irqsave(&pool_rwlock, flags); /* Fast path: look the stack trace up without full locking. */ - found = find_stack(*bucket, entries, nr_entries, hash); + found = find_stack(bucket, entries, nr_entries, hash); if (found) { read_unlock_irqrestore(&pool_rwlock, flags); goto exit; @@ -494,14 +508,13 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, write_lock_irqsave(&pool_rwlock, flags); - found = find_stack(*bucket, entries, nr_entries, hash); + found = find_stack(bucket, entries, nr_entries, hash); if (!found) { struct stack_record *new = depot_alloc_stack(entries, nr_entries, hash, &prealloc); if (new) { - new->next = *bucket; - *bucket = new; + list_add(&new->list, bucket); found = new; } } else if (prealloc) { From patchwork Mon Oct 23 16:22:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 156979 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:ce89:0:b0:403:3b70:6f57 with SMTP id p9csp1409922vqx; Mon, 23 Oct 2023 09:25:56 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEeseeqWaYV5c0kD8XnzrYPt4E/+cncaL2qgttOqIi1kpuoc4vXd7wrtFLKwKpQyXiWoHT4 X-Received: by 2002:a05:6a21:1cae:b0:16b:79c2:877f with SMTP id sf46-20020a056a211cae00b0016b79c2877fmr45152pzb.53.1698078356091; Mon, 23 Oct 2023 09:25:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1698078356; cv=none; d=google.com; s=arc-20160816; b=tLzjNsLBZKxiltv1Ol1SkiAqcWBxbfkli0hGIW4RWKX9AxS4zU8VKy+9+t7U7i79bK vazdv5pdqupO2TUKDyzp3FoAopjolTpxKU5GLrKFlCoyrWdGKE+Nn4yZQkZjF9wWWKNg 7ivpWhk4dAPsHwjiMgdNwbRgfFlmu52UgaoqZjA2arkr9GZHLkYx3uhSEXSuUsrM2tY/ KmeUDevjU/lD+2hdJI07C8YhlZgnk0WuchnKxOfj50YpvGqlN0ieBrVfBiE35QNgCS5F vd+49JYpb84Hw5XTh7J4uNSLh91SHX95QICP7VJMWSFY6vWn37I5gyaizho81Sn0uiPT iw+w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=EAtH6mtyDDwPJA3G1u2tEpoRGlDDlV29Wi3V3trZXF4=; fh=nfQSbTp1dWHt2Ier1Up8UhVNdXOqoJJLNvTrpT3jJEk=; b=RbYrqXM2nOf4FCPt1o+ulXtbZafZIr0VTGppV++IWfIgHfMbfPMc6WSt4bFmyjY3Qg OtZ2xPStjNrfk8X9HIa73ACx1K39d+2+BiugSf3PLpERgbkawoMLAwgy8p+QdC0bnLS/ KagyxixTB48wuyarluVBO1rr4YNA9eHp0Oz0dLjQzOjKcaEz6+rS2PbQ+M7zgyKw1ktl /6xqT028U2rbv+swgtjFolseT7F/dsuQBHFYCc3p3oun0fPD2lBWSdhipfOVvwdimlpA 6z30MJDCZDd2cMyQvva3Kxi6hy2Ewq56SeU1JpS1G0YKVU1JAxohG/SOgHSeHZ9NasC2 3+pg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=kPoNUZX0; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from agentk.vger.email (agentk.vger.email. [2620:137:e000::3:2]) by mx.google.com with ESMTPS id s194-20020a632ccb000000b00563f627f2easi6618255pgs.122.2023.10.23.09.25.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Oct 2023 09:25:56 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) client-ip=2620:137:e000::3:2; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=kPoNUZX0; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id 1F0868061176; Mon, 23 Oct 2023 09:25:36 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233519AbjJWQZO (ORCPT + 27 others); Mon, 23 Oct 2023 12:25:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39740 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229714AbjJWQZM (ORCPT ); Mon, 23 Oct 2023 12:25:12 -0400 Received: from out-201.mta0.migadu.com (out-201.mta0.migadu.com [91.218.175.201]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 63872E4 for ; Mon, 23 Oct 2023 09:25:10 -0700 (PDT) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1698078308; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EAtH6mtyDDwPJA3G1u2tEpoRGlDDlV29Wi3V3trZXF4=; b=kPoNUZX00gmVx2jTNWUBw1MnjfYZ+Ih+Xex2SmhQ0zyGsX2Ioeza+QBykBxwXLreMwqivE M2aRr8G1yeGyH9/IfvhHARI9SQY8lVb/5X48pbhNEAEo4GD626fn7sdXdC2E92CjCX+Meu xIXcm3bRhi2HV2gkZQpZtoUQzaGzoec= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko Cc: Andrey Konovalov , Dmitry Vyukov , Vlastimil Babka , kasan-dev@googlegroups.com, Evgenii Stepanov , Oscar Salvador , Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v3 13/19] kmsan: use stack_depot_save instead of __stack_depot_save Date: Mon, 23 Oct 2023 18:22:44 +0200 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Mon, 23 Oct 2023 09:25:36 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780564210702161143 X-GMAIL-MSGID: 1780564210702161143 From: Andrey Konovalov Make KMSAN use stack_depot_save instead of __stack_depot_save, as it always passes true to __stack_depot_save as the last argument. Reviewed-by: Alexander Potapenko Signed-off-by: Andrey Konovalov --- Changes v1->v2: - This is a new patch. --- mm/kmsan/core.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/mm/kmsan/core.c b/mm/kmsan/core.c index 3adb4c1d3b19..5d942f19d12a 100644 --- a/mm/kmsan/core.c +++ b/mm/kmsan/core.c @@ -76,7 +76,7 @@ depot_stack_handle_t kmsan_save_stack_with_flags(gfp_t flags, /* Don't sleep. */ flags &= ~(__GFP_DIRECT_RECLAIM | __GFP_KSWAPD_RECLAIM); - handle = __stack_depot_save(entries, nr_entries, flags, true); + handle = stack_depot_save(entries, nr_entries, flags); return stack_depot_set_extra_bits(handle, extra); } @@ -250,11 +250,10 @@ depot_stack_handle_t kmsan_internal_chain_origin(depot_stack_handle_t id) /* * @entries is a local var in non-instrumented code, so KMSAN does not * know it is initialized. Explicitly unpoison it to avoid false - * positives when __stack_depot_save() passes it to instrumented code. + * positives when stack_depot_save() passes it to instrumented code. */ kmsan_internal_unpoison_memory(entries, sizeof(entries), false); - handle = __stack_depot_save(entries, ARRAY_SIZE(entries), __GFP_HIGH, - true); + handle = stack_depot_save(entries, ARRAY_SIZE(entries), __GFP_HIGH); return stack_depot_set_extra_bits(handle, extra_bits); } From patchwork Mon Oct 23 16:22:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 156980 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:ce89:0:b0:403:3b70:6f57 with SMTP id p9csp1409932vqx; Mon, 23 Oct 2023 09:25:57 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEQIhQBHocGf2RexRCbh3AE2qRuXWXV/eoOjPU3ZswQWvBQQGf4sK70verp6zul23w0BxJz X-Received: by 2002:a05:6359:5d30:b0:168:dedc:7db4 with SMTP id ps48-20020a0563595d3000b00168dedc7db4mr2889565rwb.5.1698078357645; Mon, 23 Oct 2023 09:25:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1698078357; cv=none; d=google.com; s=arc-20160816; b=qc2xCbyJOQmXBPwuOzrXI3DTiPO4AoXItpes2BRoNRVLacdVl5WZqKpgyKHHRcYcVT BrVHy+softjSbtmUq9tZMBGmhRsQzhHREyKM0I72O2imT9fgoQsgkgX0fNAGW+fPRucV NZ3+W1ho7elZdPlY/kovo1QCRbNA/dt/YGQyMs1sTy/VfaZNqBdbKtcJj2ud4l6YizFi 5GQutrJ4mlRgVPgQNiMq+CbAlY9pzk3Pvor0QsTfFmSfGICv0Smf+lZrmgeQchjz99np PQBtK7A1anbzEYFsGbObY3aK3iqgIEU4Cfd1h+rCIQiXVEa8I/WqG5KpIFl3Jlr1V3wa 7n6w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=MAMDAy8Q02glfcWzfXnjTMXtIPTZ6ms/1w5eICwlRhw=; fh=nfQSbTp1dWHt2Ier1Up8UhVNdXOqoJJLNvTrpT3jJEk=; b=MI7u5FbbQT9ldDqLie3cFcGPCJEXUkLFqDe7nhgjOvmHCLzjHkrMgfkfzajgaBch2E XC2EhpKIWnSqfUin1yZL7sC5CnXDSQZeq2d57TWlp6vPK6MXRaP6K5K6QUmE+ZOvG4CI l28mtik80jz3v96oOhj5uMLNm7GVxSG9DXCyit0em5FfsH5v2frfhrvzNJIr/Aapbfk0 FIsXPDVI6bGZSlxrF+/EmUHkoI/H/tc7vA2sAEb2piwcjT/gASsXw2VNZ4cjySNDfMDR qmrnDLLdkuhSMNp/II06G6/PoZfv9CrTLaOTMn5JNvtdEWwdxK2R6CKqdGwuo0p94F+X yUzg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=cQKAccVp; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from agentk.vger.email (agentk.vger.email. [2620:137:e000::3:2]) by mx.google.com with ESMTPS id s140-20020a632c92000000b005aa833a5337si6616242pgs.600.2023.10.23.09.25.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Oct 2023 09:25:57 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) client-ip=2620:137:e000::3:2; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=cQKAccVp; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id 1045F8061172; Mon, 23 Oct 2023 09:25:51 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233726AbjJWQZa (ORCPT + 27 others); Mon, 23 Oct 2023 12:25:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39856 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231827AbjJWQZQ (ORCPT ); Mon, 23 Oct 2023 12:25:16 -0400 Received: from out-197.mta0.migadu.com (out-197.mta0.migadu.com [IPv6:2001:41d0:1004:224b::c5]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EB6249F for ; Mon, 23 Oct 2023 09:25:10 -0700 (PDT) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1698078309; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MAMDAy8Q02glfcWzfXnjTMXtIPTZ6ms/1w5eICwlRhw=; b=cQKAccVp4i4WGbePGAJzFG2QCILkp8GkBM1OodnLLpvC71XrWxziHhpn7+OWU7s+/rAmvr cEwzqyw5M2YJ63oA5AjW1urTtz/u0Nune13U3rAkp3m2NXgelJhraa+xC4QncjzGLUzDkQ /FKPykl7hegY40JrJhmtwgrwmInGiS4= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko Cc: Andrey Konovalov , Dmitry Vyukov , Vlastimil Babka , kasan-dev@googlegroups.com, Evgenii Stepanov , Oscar Salvador , Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v3 14/19] lib/stackdepot, kasan: add flags to __stack_depot_save and rename Date: Mon, 23 Oct 2023 18:22:45 +0200 Message-Id: <391437de83944753819a6c0b1d95bd7aa55ea106.1698077459.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Mon, 23 Oct 2023 09:25:52 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780564212120948378 X-GMAIL-MSGID: 1780564212120948378 From: Andrey Konovalov Change the bool can_alloc argument of __stack_depot_save to a u32 argument that accepts a set of flags. The following patch will add another flag to stack_depot_save_flags besides the existing STACK_DEPOT_FLAG_CAN_ALLOC. Also rename the function to stack_depot_save_flags, as __stack_depot_save is a cryptic name, Reviewed-by: Alexander Potapenko Signed-off-by: Andrey Konovalov --- Changes v2->v3: - WARN_ON invalid flags in stack_depot_save_flags. Changes v1->v2: - This is a new patch. --- include/linux/stackdepot.h | 36 +++++++++++++++++++++++++----------- lib/stackdepot.c | 16 +++++++++++----- mm/kasan/common.c | 7 ++++--- mm/kasan/generic.c | 9 +++++---- mm/kasan/kasan.h | 2 +- mm/kasan/tags.c | 3 ++- 6 files changed, 48 insertions(+), 25 deletions(-) diff --git a/include/linux/stackdepot.h b/include/linux/stackdepot.h index e58306783d8e..0b262e14144e 100644 --- a/include/linux/stackdepot.h +++ b/include/linux/stackdepot.h @@ -32,6 +32,17 @@ typedef u32 depot_stack_handle_t; */ #define STACK_DEPOT_EXTRA_BITS 5 +typedef u32 depot_flags_t; + +/* + * Flags that can be passed to stack_depot_save_flags(); see the comment next + * to its declaration for more details. + */ +#define STACK_DEPOT_FLAG_CAN_ALLOC ((depot_flags_t)0x0001) + +#define STACK_DEPOT_FLAGS_NUM 1 +#define STACK_DEPOT_FLAGS_MASK ((depot_flags_t)((1 << STACK_DEPOT_FLAGS_NUM) - 1)) + /* * Using stack depot requires its initialization, which can be done in 3 ways: * @@ -69,31 +80,34 @@ static inline int stack_depot_early_init(void) { return 0; } #endif /** - * __stack_depot_save - Save a stack trace to stack depot + * stack_depot_save_flags - Save a stack trace to stack depot * * @entries: Pointer to the stack trace * @nr_entries: Number of frames in the stack * @alloc_flags: Allocation GFP flags - * @can_alloc: Allocate stack pools (increased chance of failure if false) + * @depot_flags: Stack depot flags + * + * Saves a stack trace from @entries array of size @nr_entries. * - * Saves a stack trace from @entries array of size @nr_entries. If @can_alloc is - * %true, stack depot can replenish the stack pools in case no space is left - * (allocates using GFP flags of @alloc_flags). If @can_alloc is %false, avoids - * any allocations and fails if no space is left to store the stack trace. + * If STACK_DEPOT_FLAG_CAN_ALLOC is set in @depot_flags, stack depot can + * replenish the stack pools in case no space is left (allocates using GFP + * flags of @alloc_flags). Otherwise, stack depot avoids any allocations and + * fails if no space is left to store the stack trace. * * If the provided stack trace comes from the interrupt context, only the part * up to the interrupt entry is saved. * - * Context: Any context, but setting @can_alloc to %false is required if + * Context: Any context, but setting STACK_DEPOT_FLAG_CAN_ALLOC is required if * alloc_pages() cannot be used from the current context. Currently * this is the case for contexts where neither %GFP_ATOMIC nor * %GFP_NOWAIT can be used (NMI, raw_spin_lock). * * Return: Handle of the stack struct stored in depot, 0 on failure */ -depot_stack_handle_t __stack_depot_save(unsigned long *entries, - unsigned int nr_entries, - gfp_t gfp_flags, bool can_alloc); +depot_stack_handle_t stack_depot_save_flags(unsigned long *entries, + unsigned int nr_entries, + gfp_t gfp_flags, + depot_flags_t depot_flags); /** * stack_depot_save - Save a stack trace to stack depot @@ -103,7 +117,7 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, * @alloc_flags: Allocation GFP flags * * Context: Contexts where allocations via alloc_pages() are allowed. - * See __stack_depot_save() for more details. + * See stack_depot_save_flags() for more details. * * Return: Handle of the stack trace stored in depot, 0 on failure */ diff --git a/lib/stackdepot.c b/lib/stackdepot.c index 85fd40c63817..902d69d3ee30 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -444,19 +444,24 @@ static inline struct stack_record *find_stack(struct list_head *bucket, return NULL; } -depot_stack_handle_t __stack_depot_save(unsigned long *entries, - unsigned int nr_entries, - gfp_t alloc_flags, bool can_alloc) +depot_stack_handle_t stack_depot_save_flags(unsigned long *entries, + unsigned int nr_entries, + gfp_t alloc_flags, + depot_flags_t depot_flags) { struct list_head *bucket; struct stack_record *found = NULL; depot_stack_handle_t handle = 0; struct page *page = NULL; void *prealloc = NULL; + bool can_alloc = depot_flags & STACK_DEPOT_FLAG_CAN_ALLOC; bool need_alloc = false; unsigned long flags; u32 hash; + if (WARN_ON(depot_flags & ~STACK_DEPOT_FLAGS_MASK)) + return 0; + /* * If this stack trace is from an interrupt, including anything before * interrupt entry usually leads to unbounded stack depot growth. @@ -535,13 +540,14 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, handle = found->handle.handle; return handle; } -EXPORT_SYMBOL_GPL(__stack_depot_save); +EXPORT_SYMBOL_GPL(stack_depot_save_flags); depot_stack_handle_t stack_depot_save(unsigned long *entries, unsigned int nr_entries, gfp_t alloc_flags) { - return __stack_depot_save(entries, nr_entries, alloc_flags, true); + return stack_depot_save_flags(entries, nr_entries, alloc_flags, + STACK_DEPOT_FLAG_CAN_ALLOC); } EXPORT_SYMBOL_GPL(stack_depot_save); diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 256930da578a..825a0240ec02 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -22,6 +22,7 @@ #include #include #include +#include #include #include #include @@ -37,19 +38,19 @@ struct slab *kasan_addr_to_slab(const void *addr) return NULL; } -depot_stack_handle_t kasan_save_stack(gfp_t flags, bool can_alloc) +depot_stack_handle_t kasan_save_stack(gfp_t flags, depot_flags_t depot_flags) { unsigned long entries[KASAN_STACK_DEPTH]; unsigned int nr_entries; nr_entries = stack_trace_save(entries, ARRAY_SIZE(entries), 0); - return __stack_depot_save(entries, nr_entries, flags, can_alloc); + return stack_depot_save_flags(entries, nr_entries, flags, depot_flags); } void kasan_set_track(struct kasan_track *track, gfp_t flags) { track->pid = current->pid; - track->stack = kasan_save_stack(flags, true); + track->stack = kasan_save_stack(flags, STACK_DEPOT_FLAG_CAN_ALLOC); } #if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) diff --git a/mm/kasan/generic.c b/mm/kasan/generic.c index 4d837ab83f08..5d168c9afb32 100644 --- a/mm/kasan/generic.c +++ b/mm/kasan/generic.c @@ -25,6 +25,7 @@ #include #include #include +#include #include #include #include @@ -472,7 +473,7 @@ size_t kasan_metadata_size(struct kmem_cache *cache, bool in_object) sizeof(struct kasan_free_meta) : 0); } -static void __kasan_record_aux_stack(void *addr, bool can_alloc) +static void __kasan_record_aux_stack(void *addr, depot_flags_t depot_flags) { struct slab *slab = kasan_addr_to_slab(addr); struct kmem_cache *cache; @@ -489,17 +490,17 @@ static void __kasan_record_aux_stack(void *addr, bool can_alloc) return; alloc_meta->aux_stack[1] = alloc_meta->aux_stack[0]; - alloc_meta->aux_stack[0] = kasan_save_stack(0, can_alloc); + alloc_meta->aux_stack[0] = kasan_save_stack(0, depot_flags); } void kasan_record_aux_stack(void *addr) { - return __kasan_record_aux_stack(addr, true); + return __kasan_record_aux_stack(addr, STACK_DEPOT_FLAG_CAN_ALLOC); } void kasan_record_aux_stack_noalloc(void *addr) { - return __kasan_record_aux_stack(addr, false); + return __kasan_record_aux_stack(addr, 0); } void kasan_save_alloc_info(struct kmem_cache *cache, void *object, gfp_t flags) diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index d37831b8511c..3787266d9794 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -368,7 +368,7 @@ static inline void kasan_init_cache_meta(struct kmem_cache *cache, unsigned int static inline void kasan_init_object_meta(struct kmem_cache *cache, const void *object) { } #endif -depot_stack_handle_t kasan_save_stack(gfp_t flags, bool can_alloc); +depot_stack_handle_t kasan_save_stack(gfp_t flags, depot_flags_t depot_flags); void kasan_set_track(struct kasan_track *track, gfp_t flags); void kasan_save_alloc_info(struct kmem_cache *cache, void *object, gfp_t flags); void kasan_save_free_info(struct kmem_cache *cache, void *object); diff --git a/mm/kasan/tags.c b/mm/kasan/tags.c index 7dcfe341d48e..4fd32121b0fd 100644 --- a/mm/kasan/tags.c +++ b/mm/kasan/tags.c @@ -13,6 +13,7 @@ #include #include #include +#include #include #include #include @@ -101,7 +102,7 @@ static void save_stack_info(struct kmem_cache *cache, void *object, struct kasan_stack_ring_entry *entry; void *old_ptr; - stack = kasan_save_stack(gfp_flags, true); + stack = kasan_save_stack(gfp_flags, STACK_DEPOT_FLAG_CAN_ALLOC); /* * Prevent save_stack_info() from modifying stack ring From patchwork Mon Oct 23 16:22:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 156982 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:ce89:0:b0:403:3b70:6f57 with SMTP id p9csp1410034vqx; Mon, 23 Oct 2023 09:26:09 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHfFsEOSHdQ0cnJkZSBXzhln/1dKWZsuiUJGjTwrFCvb8xzDiLHDHGllN6s9etRu2JY+ILN X-Received: by 2002:a17:903:2304:b0:1b8:a67f:1c15 with SMTP id d4-20020a170903230400b001b8a67f1c15mr15267516plh.25.1698078368865; Mon, 23 Oct 2023 09:26:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1698078368; cv=none; d=google.com; s=arc-20160816; b=ErpaUYLZFPWJTZaKvXITBTvffxq7w1+5LChIW82zdLTyiMHxYdSNS7AfW/y3kSGYVs TrkHArN8POQULlYDUW2munFeHAAIOxOdscCbpHp02WJjFBwGhD47bXBB4EZzLNiM3DAD x/v1MtwcXaThpCgor5oZECKyqf043tCCoe4IPmmII5loeeY+5e5Djza+yWnFJx26dwEF j3sheAw2M0oEgzSgApz2Br/3nulpFr47/fsBWA4uzyQzZ7WxyAcfjW7jl7h2cWaS4zjR wWop7QAIBEtmp8XX1+b0+Rg3t0stUK4ASPuahCsRGsrjaUTEDqAyhELifFibqSsCIykX fbvA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=1heVLz37BMf9+qEekt7VobiQTsOf70yuv2uAcQVIL/s=; fh=nfQSbTp1dWHt2Ier1Up8UhVNdXOqoJJLNvTrpT3jJEk=; b=XixMF2UonNWL2QOwEMV+2FAZRaT63rAQNljobX+vnJGrOsyd4YTD6cE8DsMgYV+e6I MDBetIEPIHIjg7mMwtGyYmgiyQubtFHgVFtvmbXpR4EfYKEr6QuQzjX0SmqLQjLnKlxC TblHbtB7fMKvtjkr7ftJJE3SxMbL791wADnv1Lt47WpHcxY0hi0wcCDPXQfIXhwO4ucu AeUUvh+wyqXt4SvJ3cUCKi8+lqoosGvQMN1f95/kUuWPrGstlz/pX1SXqvM85LHI984k XxfBzsECsq8zEsJVzTWk7B5gVI9fo8Wt5GCV7H5SSMSvT7u0vUa8NDN3DjsrlZwjoT6F 7SQw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b="Kx/VCgxq"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from fry.vger.email (fry.vger.email. [2620:137:e000::3:8]) by mx.google.com with ESMTPS id x38-20020a634a26000000b00565cc12ee24si6434982pga.874.2023.10.23.09.26.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Oct 2023 09:26:08 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) client-ip=2620:137:e000::3:8; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b="Kx/VCgxq"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by fry.vger.email (Postfix) with ESMTP id 668D2807860B; Mon, 23 Oct 2023 09:26:06 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at fry.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233631AbjJWQZU (ORCPT + 27 others); Mon, 23 Oct 2023 12:25:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39822 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233529AbjJWQZO (ORCPT ); Mon, 23 Oct 2023 12:25:14 -0400 Received: from out-190.mta0.migadu.com (out-190.mta0.migadu.com [91.218.175.190]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BE3FB10E for ; Mon, 23 Oct 2023 09:25:11 -0700 (PDT) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1698078310; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1heVLz37BMf9+qEekt7VobiQTsOf70yuv2uAcQVIL/s=; b=Kx/VCgxqK8t1e20YOOctGoZgDJ6b2HmF5/SJtYpc40Yinosi40PcEw97gdh3Q2wSqoKaMv gCLt8HjJWSiEgQi1w+n4G7RMvsrZcD0tl+aQ+Iqp6FbWpwLJ3t5TLLfAK5QFLHK17pR18Y OUxp2UPDNk9ec8bWB7l49xpKBCrLcHA= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko Cc: Andrey Konovalov , Dmitry Vyukov , Vlastimil Babka , kasan-dev@googlegroups.com, Evgenii Stepanov , Oscar Salvador , Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v3 15/19] lib/stackdepot: add refcount for records Date: Mon, 23 Oct 2023 18:22:46 +0200 Message-Id: <21c7e1646cf9ae61123851c5f62bfd02f21f6bf8.1698077459.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on fry.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (fry.vger.email [0.0.0.0]); Mon, 23 Oct 2023 09:26:06 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780564223807319806 X-GMAIL-MSGID: 1780564223807319806 From: Andrey Konovalov Add a reference counter for how many times a stack records has been added to stack depot. Add a new STACK_DEPOT_FLAG_GET flag to stack_depot_save_flags that instructs the stack depot to increment the refcount. Do not yet decrement the refcount; this is implemented in one of the following patches. Do not yet enable any users to use the flag to avoid overflowing the refcount. This is preparatory patch for implementing the eviction of stack records from the stack depot. Reviewed-by: Alexander Potapenko Signed-off-by: Andrey Konovalov --- Changes v1->v2: - Add forgotten refcount_inc() under write lock. - Add STACK_DEPOT_FLAG_GET flag for stack_depot_save_flags. --- include/linux/stackdepot.h | 13 ++++++++++--- lib/stackdepot.c | 12 ++++++++++-- 2 files changed, 20 insertions(+), 5 deletions(-) diff --git a/include/linux/stackdepot.h b/include/linux/stackdepot.h index 0b262e14144e..611716702d73 100644 --- a/include/linux/stackdepot.h +++ b/include/linux/stackdepot.h @@ -39,8 +39,9 @@ typedef u32 depot_flags_t; * to its declaration for more details. */ #define STACK_DEPOT_FLAG_CAN_ALLOC ((depot_flags_t)0x0001) +#define STACK_DEPOT_FLAG_GET ((depot_flags_t)0x0002) -#define STACK_DEPOT_FLAGS_NUM 1 +#define STACK_DEPOT_FLAGS_NUM 2 #define STACK_DEPOT_FLAGS_MASK ((depot_flags_t)((1 << STACK_DEPOT_FLAGS_NUM) - 1)) /* @@ -94,6 +95,9 @@ static inline int stack_depot_early_init(void) { return 0; } * flags of @alloc_flags). Otherwise, stack depot avoids any allocations and * fails if no space is left to store the stack trace. * + * If STACK_DEPOT_FLAG_GET is set in @depot_flags, stack depot will increment + * the refcount on the saved stack trace if it already exists in stack depot. + * * If the provided stack trace comes from the interrupt context, only the part * up to the interrupt entry is saved. * @@ -116,8 +120,11 @@ depot_stack_handle_t stack_depot_save_flags(unsigned long *entries, * @nr_entries: Number of frames in the stack * @alloc_flags: Allocation GFP flags * - * Context: Contexts where allocations via alloc_pages() are allowed. - * See stack_depot_save_flags() for more details. + * Does not increment the refcount on the saved stack trace; see + * stack_depot_save_flags() for more details. + * + * Context: Contexts where allocations via alloc_pages() are allowed; + * see stack_depot_save_flags() for more details. * * Return: Handle of the stack trace stored in depot, 0 on failure */ diff --git a/lib/stackdepot.c b/lib/stackdepot.c index 902d69d3ee30..278ed646e418 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -23,6 +23,7 @@ #include #include #include +#include #include #include #include @@ -60,6 +61,7 @@ struct stack_record { u32 hash; /* Hash in hash table */ u32 size; /* Number of stored frames */ union handle_parts handle; + refcount_t count; unsigned long entries[CONFIG_STACKDEPOT_MAX_FRAMES]; /* Frames */ }; @@ -367,6 +369,7 @@ depot_alloc_stack(unsigned long *entries, int size, u32 hash, void **prealloc) stack->hash = hash; stack->size = size; /* stack->handle is already filled in by depot_init_pool(). */ + refcount_set(&stack->count, 1); memcpy(stack->entries, entries, flex_array_size(stack, entries, size)); /* @@ -483,6 +486,8 @@ depot_stack_handle_t stack_depot_save_flags(unsigned long *entries, /* Fast path: look the stack trace up without full locking. */ found = find_stack(bucket, entries, nr_entries, hash); if (found) { + if (depot_flags & STACK_DEPOT_FLAG_GET) + refcount_inc(&found->count); read_unlock_irqrestore(&pool_rwlock, flags); goto exit; } @@ -522,12 +527,15 @@ depot_stack_handle_t stack_depot_save_flags(unsigned long *entries, list_add(&new->list, bucket); found = new; } - } else if (prealloc) { + } else { + if (depot_flags & STACK_DEPOT_FLAG_GET) + refcount_inc(&found->count); /* * Stack depot already contains this stack trace, but let's * keep the preallocated memory for future. */ - depot_keep_new_pool(&prealloc); + if (prealloc) + depot_keep_new_pool(&prealloc); } write_unlock_irqrestore(&pool_rwlock, flags); From patchwork Mon Oct 23 16:22:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 156978 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:ce89:0:b0:403:3b70:6f57 with SMTP id p9csp1409784vqx; Mon, 23 Oct 2023 09:25:43 -0700 (PDT) X-Google-Smtp-Source: AGHT+IF/Si5c1RBtTuG5p3Pv/CqhX/KptugqhhhKjkjptO8ti+3cEUc5kRZzCAzY4MRV4JBoHTou X-Received: by 2002:a17:902:e841:b0:1b6:6f12:502e with SMTP id t1-20020a170902e84100b001b66f12502emr11063212plg.49.1698078343187; Mon, 23 Oct 2023 09:25:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1698078343; cv=none; d=google.com; s=arc-20160816; b=DBC+vWPCQ685wR3M5h/WRSQIpOCQdhvWl4yZDZ0eBxsnNI2xAvWWHt7vHSuxkT7JqB LwUWeJpD/4vncf3stbXt1O4jjs8TF2OH1eRGdeUXFuLaxjaW5J9fNUm5yGGLLsKVK/4R JKN3NU1ShMO2r4wQpDkwaAs9wK0mJUpkZ94iQ2ip/EZjj+ZVjkR7Et36gq3xV0ikgYpp xP87vjhMZnWIsbAtv7AID2L21fEiJP4WaWwO9j873AIS8da2HOYav0MzitucBfhRAdsV g/ldrEUrrKLv/aN3pSMcOKssloUwKzhXS47u29JyN6wLQtIIpoBtyM6UAFij/UfrDRjj 8Mkg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=KyQaV2ck0CZBTZjrz6rGoXlFfvUUwlefDCsA1d1OnDI=; fh=nfQSbTp1dWHt2Ier1Up8UhVNdXOqoJJLNvTrpT3jJEk=; b=CWr08o+Ft4IN4R1K9J9Dsg3amAoLy0kGFIwHsbWdHaidx1ivehImHRTwZAJTWfOs7y exos6eDSRGTceg+7er/YkVv4CI2XgUCIKiD5O+7CiFoYIPXDPwZvpNYGTakmaVXF4gwG 549vMPxatn+CKHGA/67pCaUy7Xk+LmwxLL4f/BjldqMPuSCr7Mnc9gCa7EIgyzqTG7qh Pa2rwrOXItJFMg6sNUCEOexm+46OD16ReibC88ZOdntd57dVF3rq6SePbjmv9Xu3IaJ5 l+RtbNtRkFf2fOMAnAvyHX5kVngss7ITUAmcZoftIA0RVtWgpQSQ34h+90ktgC6GZhKc cwTw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b="oyH/s4RT"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from fry.vger.email (fry.vger.email. [23.128.96.38]) by mx.google.com with ESMTPS id 4-20020a170902e9c400b001baff05d890si6306156plk.313.2023.10.23.09.25.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Oct 2023 09:25:43 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) client-ip=23.128.96.38; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b="oyH/s4RT"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by fry.vger.email (Postfix) with ESMTP id AE7328078611; Mon, 23 Oct 2023 09:25:33 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at fry.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233620AbjJWQZS (ORCPT + 27 others); Mon, 23 Oct 2023 12:25:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39828 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233537AbjJWQZO (ORCPT ); Mon, 23 Oct 2023 12:25:14 -0400 Received: from out-195.mta0.migadu.com (out-195.mta0.migadu.com [91.218.175.195]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B6FF0D78 for ; Mon, 23 Oct 2023 09:25:12 -0700 (PDT) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1698078310; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KyQaV2ck0CZBTZjrz6rGoXlFfvUUwlefDCsA1d1OnDI=; b=oyH/s4RTev5TTn2tx92NM3vkXg6oUk6Kpicty1qVqAfYRd+524Z47P2qCKhvUIRV4y3uiL rlsDYYtewCtLkLPB0dG2UOn/EJuLBsAiSUiToU+oaYgXFZHxJ+CthWgKW++yb5No2yIg9Y u1Z8kAH4ZHqywmyABUdVN11IBWKHboo= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko Cc: Andrey Konovalov , Dmitry Vyukov , Vlastimil Babka , kasan-dev@googlegroups.com, Evgenii Stepanov , Oscar Salvador , Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v3 16/19] lib/stackdepot: allow users to evict stack traces Date: Mon, 23 Oct 2023 18:22:47 +0200 Message-Id: <8ba201688d0e4b8f3ec56b426bb350965a30c8fd.1698077459.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on fry.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (fry.vger.email [0.0.0.0]); Mon, 23 Oct 2023 09:25:33 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780564196712737951 X-GMAIL-MSGID: 1780564196712737951 From: Andrey Konovalov Add stack_depot_put, a function that decrements the reference counter on a stack record and removes it from the stack depot once the counter reaches 0. Internally, when removing a stack record, the function unlinks it from the hash table bucket and returns to the freelist. With this change, the users of stack depot can call stack_depot_put when keeping a stack trace in the stack depot is not needed anymore. This allows avoiding polluting the stack depot with irrelevant stack traces and thus have more space to store the relevant ones before the stack depot reaches its capacity. Signed-off-by: Andrey Konovalov --- Changes v1->v2: - Comments fixes as suggested by Marco. - Add lockdep_assert annotation. - Adapt to using list_head's. - Rename stack_depot_evict to stack_depot_put. --- include/linux/stackdepot.h | 14 ++++++++++++++ lib/stackdepot.c | 35 +++++++++++++++++++++++++++++++++++ 2 files changed, 49 insertions(+) diff --git a/include/linux/stackdepot.h b/include/linux/stackdepot.h index 611716702d73..a6796f178913 100644 --- a/include/linux/stackdepot.h +++ b/include/linux/stackdepot.h @@ -97,6 +97,8 @@ static inline int stack_depot_early_init(void) { return 0; } * * If STACK_DEPOT_FLAG_GET is set in @depot_flags, stack depot will increment * the refcount on the saved stack trace if it already exists in stack depot. + * Users of this flag must also call stack_depot_put() when keeping the stack + * trace is no longer required to avoid overflowing the refcount. * * If the provided stack trace comes from the interrupt context, only the part * up to the interrupt entry is saved. @@ -162,6 +164,18 @@ void stack_depot_print(depot_stack_handle_t stack); int stack_depot_snprint(depot_stack_handle_t handle, char *buf, size_t size, int spaces); +/** + * stack_depot_put - Drop a reference to a stack trace from stack depot + * + * @handle: Stack depot handle returned from stack_depot_save() + * + * The stack trace is evicted from stack depot once all references to it have + * been dropped (once the number of stack_depot_evict() calls matches the + * number of stack_depot_save_flags() calls with STACK_DEPOT_FLAG_GET set for + * this stack trace). + */ +void stack_depot_put(depot_stack_handle_t handle); + /** * stack_depot_set_extra_bits - Set extra bits in a stack depot handle * diff --git a/lib/stackdepot.c b/lib/stackdepot.c index 278ed646e418..3a8f045696fd 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -404,6 +404,14 @@ static struct stack_record *depot_fetch_stack(depot_stack_handle_t handle) return stack; } +/* Links stack into the freelist. */ +static void depot_free_stack(struct stack_record *stack) +{ + lockdep_assert_held_write(&pool_rwlock); + + list_add(&stack->list, &free_stacks); +} + /* Calculates the hash for a stack. */ static inline u32 hash_stack(unsigned long *entries, unsigned int size) { @@ -586,6 +594,33 @@ unsigned int stack_depot_fetch(depot_stack_handle_t handle, } EXPORT_SYMBOL_GPL(stack_depot_fetch); +void stack_depot_put(depot_stack_handle_t handle) +{ + struct stack_record *stack; + unsigned long flags; + + if (!handle || stack_depot_disabled) + return; + + write_lock_irqsave(&pool_rwlock, flags); + + stack = depot_fetch_stack(handle); + if (WARN_ON(!stack)) + goto out; + + if (refcount_dec_and_test(&stack->count)) { + /* Unlink stack from the hash table. */ + list_del(&stack->list); + + /* Free stack. */ + depot_free_stack(stack); + } + +out: + write_unlock_irqrestore(&pool_rwlock, flags); +} +EXPORT_SYMBOL_GPL(stack_depot_put); + void stack_depot_print(depot_stack_handle_t stack) { unsigned long *entries; From patchwork Mon Oct 23 16:22:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 156983 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:ce89:0:b0:403:3b70:6f57 with SMTP id p9csp1410195vqx; Mon, 23 Oct 2023 09:26:29 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFoaFojhdAkiTjew/LRDhsqEOFjIyC08FBq0f40bvInGZTsaFSu++H80PvsBefhGa/e3wvE X-Received: by 2002:aa7:9619:0:b0:6b7:e577:3f7b with SMTP id q25-20020aa79619000000b006b7e5773f7bmr6758363pfg.21.1698078389216; Mon, 23 Oct 2023 09:26:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1698078389; cv=none; d=google.com; s=arc-20160816; b=fl0PlDH68Fe5o5eLFveeVV48LIxkHCImqfIR8GbZA3AGgRfL7QwnjwBKjLOnUEa2c2 zx4qfc26tJvwcYuNgvw11bf27yjOEXKRX1veBRYPLfZFgLsdo9liY77o+LiYCLtYQyMz XldXfEX953dxbqABEbgnFG20uhDEMoUFPPfbtl60UCe1WZHe1zR/N5lTc0oWfNUHdkKs L94E61ymoVEsmJ8KFeVz9/NOYOInImTscQ0Xf0HwO6too9mheBwhvYjNoFtegmnD2ZJL kr20Af48/20MlqbiVAldH04EdiSHwB14Uyftn79xXlOFl/hT6lBD4vIAbSyYjvv2BnP8 /ziw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=h9lSbnFVvjj2GZw0LrViVCVgXWgHHPxOTEIrM79fNvQ=; fh=nfQSbTp1dWHt2Ier1Up8UhVNdXOqoJJLNvTrpT3jJEk=; b=ufuE10xXSey7DGzUfeZW+QCxahSZf8Z6yVs860SnfVO3YR3PgK3uBp8Fhmr5aUF5u8 F8QoSOB+dUWtg8GT56MwPtMKWuBTpSjUoIEg2kti0ThSWtzZK5Us/odMW8IouogoZh0c pt8AR3jlZvT9YEQzD+i6eNEQnm2XA1mMdAgLJWPa/N5Rn45XtbjYM75oxiPwsSp8ogCl JnmQPXH/5/RHnQXeiuLQ1+odV9odWPlP/EPRQFZPKxkFvJhFVIETQzc8knTO77nCuMMi qAWg+JME4RgnQrWnxQQlQC9gplGM7RAUmZ6+Jh8eA3WIguSckc8cVCm+C+VVILkjmXFk a3fw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=CQBo9f5r; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from pete.vger.email (pete.vger.email. [2620:137:e000::3:6]) by mx.google.com with ESMTPS id fd35-20020a056a002ea300b00690f191430csi6929628pfb.56.2023.10.23.09.26.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Oct 2023 09:26:29 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) client-ip=2620:137:e000::3:6; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=CQBo9f5r; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by pete.vger.email (Postfix) with ESMTP id 097168081CF1; Mon, 23 Oct 2023 09:26:26 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at pete.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233759AbjJWQZd (ORCPT + 27 others); Mon, 23 Oct 2023 12:25:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54878 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233658AbjJWQZ1 (ORCPT ); Mon, 23 Oct 2023 12:25:27 -0400 Received: from out-202.mta0.migadu.com (out-202.mta0.migadu.com [91.218.175.202]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2AAC910A for ; Mon, 23 Oct 2023 09:25:13 -0700 (PDT) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1698078311; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=h9lSbnFVvjj2GZw0LrViVCVgXWgHHPxOTEIrM79fNvQ=; b=CQBo9f5r78HDYIzuMKRdcegVRv3KmK60h3RNuj3rljg/vJLY0On77Rw0SAXFAcOXknm+Yn sWYecQHmhDaiksWaRwRkX2AT+Gd9TdjhzprwlFBsP9luxtS1bbrxbl6j0iwAjY4bMTGHZc IWeq/EXSKTJEUgePln3o5gvif7vZcqM= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko Cc: Andrey Konovalov , Dmitry Vyukov , Vlastimil Babka , kasan-dev@googlegroups.com, Evgenii Stepanov , Oscar Salvador , Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v3 17/19] kasan: remove atomic accesses to stack ring entries Date: Mon, 23 Oct 2023 18:22:48 +0200 Message-Id: <81b8b7984846e0e7abfd794aad3bafee97c89c29.1698077459.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Mon, 23 Oct 2023 09:26:27 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780564245019209157 X-GMAIL-MSGID: 1780564245019209157 From: Andrey Konovalov Remove the atomic accesses to entry fields in save_stack_info and kasan_complete_mode_report_info for tag-based KASAN modes. These atomics are not required, as the read/write lock prevents the entries from being read (in kasan_complete_mode_report_info) while being written (in save_stack_info) and the try_cmpxchg prevents the same entry from being rewritten (in save_stack_info) in the unlikely case of wrapping during writing. Reviewed-by: Alexander Potapenko Signed-off-by: Andrey Konovalov --- Changes v1->v2: - This is a new patch. --- mm/kasan/report_tags.c | 25 +++++++------------------ mm/kasan/tags.c | 13 +++++-------- 2 files changed, 12 insertions(+), 26 deletions(-) diff --git a/mm/kasan/report_tags.c b/mm/kasan/report_tags.c index 8b8bfdb3cfdb..78abdcde5da9 100644 --- a/mm/kasan/report_tags.c +++ b/mm/kasan/report_tags.c @@ -31,10 +31,6 @@ void kasan_complete_mode_report_info(struct kasan_report_info *info) unsigned long flags; u64 pos; struct kasan_stack_ring_entry *entry; - void *ptr; - u32 pid; - depot_stack_handle_t stack; - bool is_free; bool alloc_found = false, free_found = false; if ((!info->cache || !info->object) && !info->bug_type) { @@ -61,18 +57,11 @@ void kasan_complete_mode_report_info(struct kasan_report_info *info) entry = &stack_ring.entries[i % stack_ring.size]; - /* Paired with smp_store_release() in save_stack_info(). */ - ptr = (void *)smp_load_acquire(&entry->ptr); - - if (kasan_reset_tag(ptr) != info->object || - get_tag(ptr) != get_tag(info->access_addr)) + if (kasan_reset_tag(entry->ptr) != info->object || + get_tag(entry->ptr) != get_tag(info->access_addr)) continue; - pid = READ_ONCE(entry->pid); - stack = READ_ONCE(entry->stack); - is_free = READ_ONCE(entry->is_free); - - if (is_free) { + if (entry->is_free) { /* * Second free of the same object. * Give up on trying to find the alloc entry. @@ -80,8 +69,8 @@ void kasan_complete_mode_report_info(struct kasan_report_info *info) if (free_found) break; - info->free_track.pid = pid; - info->free_track.stack = stack; + info->free_track.pid = entry->pid; + info->free_track.stack = entry->stack; free_found = true; /* @@ -95,8 +84,8 @@ void kasan_complete_mode_report_info(struct kasan_report_info *info) if (alloc_found) break; - info->alloc_track.pid = pid; - info->alloc_track.stack = stack; + info->alloc_track.pid = entry->pid; + info->alloc_track.stack = entry->stack; alloc_found = true; /* diff --git a/mm/kasan/tags.c b/mm/kasan/tags.c index 4fd32121b0fd..b6c017e670d8 100644 --- a/mm/kasan/tags.c +++ b/mm/kasan/tags.c @@ -121,15 +121,12 @@ static void save_stack_info(struct kmem_cache *cache, void *object, if (!try_cmpxchg(&entry->ptr, &old_ptr, STACK_RING_BUSY_PTR)) goto next; /* Busy slot. */ - WRITE_ONCE(entry->size, cache->object_size); - WRITE_ONCE(entry->pid, current->pid); - WRITE_ONCE(entry->stack, stack); - WRITE_ONCE(entry->is_free, is_free); + entry->size = cache->object_size; + entry->pid = current->pid; + entry->stack = stack; + entry->is_free = is_free; - /* - * Paired with smp_load_acquire() in kasan_complete_mode_report_info(). - */ - smp_store_release(&entry->ptr, (s64)object); + entry->ptr = object; read_unlock_irqrestore(&stack_ring.lock, flags); } From patchwork Mon Oct 23 16:22:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 156984 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:ce89:0:b0:403:3b70:6f57 with SMTP id p9csp1410305vqx; Mon, 23 Oct 2023 09:26:40 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGrH5wh98eNd4zrMW1pLe/L6GgBKvcT6gaf3b3peO/mYRgG/w8pLSUvdq8sjXB+vG8GR+Fj X-Received: by 2002:a05:6a00:c82:b0:690:41a1:9b6a with SMTP id a2-20020a056a000c8200b0069041a19b6amr11958562pfv.5.1698078400058; Mon, 23 Oct 2023 09:26:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1698078400; cv=none; d=google.com; s=arc-20160816; b=t0Hgxsm7uwgmXEsqkwwA9Ftzgocq8tBB7QHzLzepQeIuOJ1GhfOKpqLQHtFHzhoPxG A1Mfe+9sYYifAuiV3HuTp2K9K6CNLwSAxnI6BuLOwV4M9yM2FgPILZgcMDMZ7mSVB+kp /Fv4DDVmk1gpYLRRexyGI9C5EQXeFOo+na1WXwi41uOqKoQNCBuuR4Mvr2ECaXsI+Au2 SdnBeEN61BdZJ6Aw4LygIOpxitTmPXnAW5C0S1Ijb3+naFHOvUlMEp4uQJqSzwSBX5/h bU8mNbV173Kh657MTRXpIenUuyNxRKLdSO2altEx1yy/m1jbDcqMDgM1rJ45iCQmuBHq 7gSA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=uhMrVBrzgAoT1gwViC8KxLkH9OMZhix/uhvoEc70W98=; fh=nfQSbTp1dWHt2Ier1Up8UhVNdXOqoJJLNvTrpT3jJEk=; b=gSNIWKjYAckIL2InUUcuEu4zFiFe1smdvtSveP2nwrf3LZCST9zfEytdkHmWfO4wx7 2v6nUplTmD0j+2SklOCAm5N426mnlJvQEohTqblYWWRlUaxU97UwJLSaC6rOxuz7mC7i syXitUfT4lblZCuef5VWQyzXq+A9JJGrEQ00lubcNNZNeMyEdgz4dQlfNsKlZ3XXuNkO iu2mJ1rscYL8+myIhmLoFKciZQwM9zk7BjP02lhKqKsZH5BX/sZBOAirK6TD1Ui/EtfA qHvl6KSjuPBJmS03DSI61Cu1OV3bRsLRHs1ifhEzszlUulnNxaUkiieDH21PmuS2MrQj yfYA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=Ne3J1RXv; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from howler.vger.email (howler.vger.email. [23.128.96.34]) by mx.google.com with ESMTPS id j5-20020a056a00234500b0068fb6fc3ff1si6677974pfj.209.2023.10.23.09.26.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Oct 2023 09:26:40 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) client-ip=23.128.96.34; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=Ne3J1RXv; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id B500880A1A7E; Mon, 23 Oct 2023 09:26:37 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229544AbjJWQ0a (ORCPT + 27 others); Mon, 23 Oct 2023 12:26:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38450 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233582AbjJWQ0R (ORCPT ); Mon, 23 Oct 2023 12:26:17 -0400 X-Greylist: delayed 129 seconds by postgrey-1.37 at lindbergh.monkeyblade.net; Mon, 23 Oct 2023 09:26:15 PDT Received: from out-210.mta1.migadu.com (out-210.mta1.migadu.com [IPv6:2001:41d0:203:375::d2]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 18C2F10E for ; Mon, 23 Oct 2023 09:26:14 -0700 (PDT) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1698078373; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uhMrVBrzgAoT1gwViC8KxLkH9OMZhix/uhvoEc70W98=; b=Ne3J1RXv7CfIkYMjEOEAo2MOXXrLjJl0MG/nn5TuOVSCeJC7ACQ8Zc+U0+7fVaprnL1TwV YcPWlAN0bBL1/0CJWM+oWPOfzdH5fmC91XjSmronfj3U3HMmSByKdvCQOOSWPbS9RJ3nc5 z58ccvrjKa1GY9FZizpTtMWzArqCzuI= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko Cc: Andrey Konovalov , Dmitry Vyukov , Vlastimil Babka , kasan-dev@googlegroups.com, Evgenii Stepanov , Oscar Salvador , Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v3 18/19] kasan: check object_size in kasan_complete_mode_report_info Date: Mon, 23 Oct 2023 18:22:49 +0200 Message-Id: <97721c54e9ccacc494dbcfcd0cbd5f5aaab6973b.1698077459.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on howler.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Mon, 23 Oct 2023 09:26:37 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780564256883331010 X-GMAIL-MSGID: 1780564256883331010 From: Andrey Konovalov Check the object size when looking up entries in the stack ring. If the size of the object for which a report is being printed does not match the size of the object for which a stack trace has been saved in the stack ring, the saved stack trace is irrelevant. Reviewed-by: Alexander Potapenko Signed-off-by: Andrey Konovalov --- Changes v2->v3: - Added missing "../slab.h" include for accessing a kmem_cache field. Changes v1->v2: - This is a new patch. --- mm/kasan/report_tags.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/mm/kasan/report_tags.c b/mm/kasan/report_tags.c index 78abdcde5da9..55154743f915 100644 --- a/mm/kasan/report_tags.c +++ b/mm/kasan/report_tags.c @@ -7,6 +7,7 @@ #include #include "kasan.h" +#include "../slab.h" extern struct kasan_stack_ring stack_ring; @@ -58,7 +59,8 @@ void kasan_complete_mode_report_info(struct kasan_report_info *info) entry = &stack_ring.entries[i % stack_ring.size]; if (kasan_reset_tag(entry->ptr) != info->object || - get_tag(entry->ptr) != get_tag(info->access_addr)) + get_tag(entry->ptr) != get_tag(info->access_addr) || + info->cache->object_size != entry->size) continue; if (entry->is_free) { From patchwork Mon Oct 23 16:22:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 156985 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:ce89:0:b0:403:3b70:6f57 with SMTP id p9csp1410383vqx; Mon, 23 Oct 2023 09:26:49 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGo0l2deuxqdIA0OMBGJ8FwauLQixRWwjvK7QslxmwWZesc/hf72HmjxKIbWaEm4lD509Wa X-Received: by 2002:aa7:94aa:0:b0:6be:30f1:45f8 with SMTP id a10-20020aa794aa000000b006be30f145f8mr10171103pfl.20.1698078408675; Mon, 23 Oct 2023 09:26:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1698078408; cv=none; d=google.com; s=arc-20160816; b=0oNRMfNQ0ihjvnJfmhAufp2WNVB3MZGJlFgJqG/0ya4PjrGxnWzOj+maHMakSIbrq7 iWfFB1zi8yN3eDbsJd1/QiWb1gUVergm2Vgb5rmO8MZZYtFzHukcnx7zDEugsS9Y6BCY MuOfOcE7MNsBBlQ5u7ZJxjA9aQglzwP54lKpIzBWfz4aAwfQgcwW59yV3BWLQ/S83ZJX SDcMbSzpW+smFQrdTraWgea/cKJV2Vtj8njVAkmZjGaQMPVTuSt37m6sbSUxh5bfh7iN 9WLLOoCIlS4O12xpntEoibnzTlOE7r+w+7TJgAMaQeoKiiCS4JAOkYURbdbkYfy/sIjL KnHQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Qc2DDhk7eSddbt2Q3kjJLSbCNGbLZWOl9JsGZF81sG4=; fh=nfQSbTp1dWHt2Ier1Up8UhVNdXOqoJJLNvTrpT3jJEk=; b=VvyqqCkjXnQPc3izuAT1TPBfLsLK8fD0HVQquyywPpY0XyQd/nIyLAN9jtffXAiJoi OoWAgktOMcOMKyRFQ6nPWOgAzlrcFdNPfapePQpfqCwCVHNtL1FIYAq/SuzoqAlgYZBg igz/w4Odw2isfa0wXrDTkumlLHtgg3wwrxMr2nTMjt4ou8aKjDdD1+MnQcakgYIv8V64 OfVcmf0ob2lHA4+Sm5TWzc5MG80L/jVaObfNeEpv2LIRdfVeZzsW8ZvYNuJqoa7ZWMqu QVEKUPf0LsyD4iJWyDZdXvHMMsvVTY32NkTADKu7VQEpJqj+/j1DGz8J3a7Hnztg/6+w 45sA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b="Tr/G6YIY"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from howler.vger.email (howler.vger.email. [23.128.96.34]) by mx.google.com with ESMTPS id bk13-20020a056a02028d00b0056433b221b9si6473539pgb.477.2023.10.23.09.26.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Oct 2023 09:26:48 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) client-ip=23.128.96.34; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b="Tr/G6YIY"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id B87B580A1E0A; Mon, 23 Oct 2023 09:26:43 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232469AbjJWQ0d (ORCPT + 27 others); Mon, 23 Oct 2023 12:26:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38458 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233694AbjJWQ0R (ORCPT ); Mon, 23 Oct 2023 12:26:17 -0400 Received: from out-199.mta1.migadu.com (out-199.mta1.migadu.com [IPv6:2001:41d0:203:375::c7]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B80EB127 for ; Mon, 23 Oct 2023 09:26:15 -0700 (PDT) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1698078373; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Qc2DDhk7eSddbt2Q3kjJLSbCNGbLZWOl9JsGZF81sG4=; b=Tr/G6YIYPrWXP9fx3O7aG2jl6kXfEVrm47rOgoEDzFX/BZ50Z23L4IXwsR4vCJshDX5TDZ HrweRnvTS3UanFhDAX4boFO1pOlEosl/bvhOjewwp0T7l51nrmSzESgRmuTG8srBHitAMu 47CJygk1kptnKFhCWZImQgFTsPZh9NY= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko Cc: Andrey Konovalov , Dmitry Vyukov , Vlastimil Babka , kasan-dev@googlegroups.com, Evgenii Stepanov , Oscar Salvador , Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v3 19/19] kasan: use stack_depot_put for tag-based modes Date: Mon, 23 Oct 2023 18:22:50 +0200 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on howler.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Mon, 23 Oct 2023 09:26:43 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780564265097684752 X-GMAIL-MSGID: 1780564265097684752 From: Andrey Konovalov Make tag-based KASAN modes evict stack traces from the stack depot once they are evicted from the stack ring. Internally, pass STACK_DEPOT_FLAG_GET to stack_depot_save_flags (via kasan_save_stack) to increment the refcount when saving a new entry to stack ring and call stack_depot_put when removing an entry from stack ring. Reviewed-by: Alexander Potapenko Signed-off-by: Andrey Konovalov --- Changes v1->v2: - Adapt to the stack depot API change. - Drop READ_ONCE when reading entry->stack. --- mm/kasan/tags.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/mm/kasan/tags.c b/mm/kasan/tags.c index b6c017e670d8..739ae997463d 100644 --- a/mm/kasan/tags.c +++ b/mm/kasan/tags.c @@ -97,12 +97,13 @@ static void save_stack_info(struct kmem_cache *cache, void *object, gfp_t gfp_flags, bool is_free) { unsigned long flags; - depot_stack_handle_t stack; + depot_stack_handle_t stack, old_stack; u64 pos; struct kasan_stack_ring_entry *entry; void *old_ptr; - stack = kasan_save_stack(gfp_flags, STACK_DEPOT_FLAG_CAN_ALLOC); + stack = kasan_save_stack(gfp_flags, + STACK_DEPOT_FLAG_CAN_ALLOC | STACK_DEPOT_FLAG_GET); /* * Prevent save_stack_info() from modifying stack ring @@ -121,6 +122,8 @@ static void save_stack_info(struct kmem_cache *cache, void *object, if (!try_cmpxchg(&entry->ptr, &old_ptr, STACK_RING_BUSY_PTR)) goto next; /* Busy slot. */ + old_stack = entry->stack; + entry->size = cache->object_size; entry->pid = current->pid; entry->stack = stack; @@ -129,6 +132,9 @@ static void save_stack_info(struct kmem_cache *cache, void *object, entry->ptr = object; read_unlock_irqrestore(&stack_ring.lock, flags); + + if (old_stack) + stack_depot_put(old_stack); } void kasan_save_alloc_info(struct kmem_cache *cache, void *object, gfp_t flags)