From patchwork Tue Nov 7 16:51:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Matz X-Patchwork-Id: 162679 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:aa0b:0:b0:403:3b70:6f57 with SMTP id k11csp363178vqo; Tue, 7 Nov 2023 08:51:46 -0800 (PST) X-Google-Smtp-Source: AGHT+IFnjR4e+xdP4RQZPKvr8Umn8fVwHSiCXZKmd+eXj8jn4HgsMtu2HFw+pgYbUUMhcm7DoHwR X-Received: by 2002:a05:620a:1726:b0:778:8d72:91ea with SMTP id az38-20020a05620a172600b007788d7291eamr13390132qkb.25.1699375906357; Tue, 07 Nov 2023 08:51:46 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1699375906; cv=pass; d=google.com; s=arc-20160816; b=cTRgKmwchq4jI7leO5nqoUsUXVNEOYpLTaGsrMIhssHMzWNPaMulbjWwNULPZ+Nld9 WwJKLPLfaAKM6i5mYUbAo4lWS+cjiTGFa27ctEvbc7AgRoimSUdl3IdXieF755Fuctq2 Z8ISCLoMJzRKUw0+NSjMRiOLA4LIJ+oGtsyvUOh+QBh22ItoPX2yjzyxJ6TmrWo+EMl/ 13WJI75J3SuqVl8TQZ9OpInS90w7zmrW7Rynbebr9ovmMpmgKhnnCRX9zt60Omb83w/X YCuiST0bHOe2qBqbW69r8As7CJt0e2ZlY+frc+kyrn9vWangkISmby2hbz4J8ssfJeFa 7hHQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:mime-version:user-agent :message-id:subject:to:from:date:dkim-signature:dkim-signature :arc-filter:dmarc-filter:delivered-to; bh=sH6aNs8EM293wG5VfOz2Nd5HSRkfyjxJdebi/03rA8c=; fh=NLxAvL/bDfPg4AGOtxqvQlND8vazkZrNzKLY8+LAbBY=; b=mmoYrZZmx5xDw6h58XWWV6wkn9nxQcnCgZ6AhZmprCmpIzz05pu0jZ0MvunndZGlVK x1pR5Ty/7FT2qq1nmiQiJ67qn50WwSdLB3tBTqyN/tma9XeQqCysDBNix7ybUk0yXnxC 7KzlGgzjqwavtXjJWq8Qjt3YIptVxt8o6kL+oilso7f1WwIOValUY0FJGGkZupoeBTPq AXdrQOLGNnRSVVmyYrGO21yXtqOC/kXQwZyZIA0K0NzbtEgRtSjeRCIZ3X6Gknt5sQ8t gamFmrzMbKs6Q5T4Q3JVmTuqM7XXJQSz4tg+LVySgmI+kJYDpVIOdVj367oBEth6gfAm q95Q== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@suse.de header.s=susede2_rsa header.b=Kgr5V52f; dkim=neutral (no key) header.i=@suse.de; arc=pass (i=1); spf=pass (google.com: domain of binutils-bounces+ouuuleilei=gmail.com@sourceware.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="binutils-bounces+ouuuleilei=gmail.com@sourceware.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=suse.de Received: from server2.sourceware.org (server2.sourceware.org. [8.43.85.97]) by mx.google.com with ESMTPS id u14-20020a05620a084e00b007789a6caf87si72010qku.557.2023.11.07.08.51.46 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Nov 2023 08:51:46 -0800 (PST) Received-SPF: pass (google.com: domain of binutils-bounces+ouuuleilei=gmail.com@sourceware.org designates 8.43.85.97 as permitted sender) client-ip=8.43.85.97; Authentication-Results: mx.google.com; dkim=pass header.i=@suse.de header.s=susede2_rsa header.b=Kgr5V52f; dkim=neutral (no key) header.i=@suse.de; arc=pass (i=1); spf=pass (google.com: domain of binutils-bounces+ouuuleilei=gmail.com@sourceware.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="binutils-bounces+ouuuleilei=gmail.com@sourceware.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=suse.de Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 222BF38582A3 for ; Tue, 7 Nov 2023 16:51:46 +0000 (GMT) X-Original-To: binutils@sourceware.org Delivered-To: binutils@sourceware.org Received: from smtp-out2.suse.de (smtp-out2.suse.de [IPv6:2001:67c:2178:6::1d]) by sourceware.org (Postfix) with ESMTPS id DE6233858002 for ; Tue, 7 Nov 2023 16:51:40 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org DE6233858002 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=suse.de ARC-Filter: OpenARC Filter v1.0.0 sourceware.org DE6233858002 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2001:67c:2178:6::1d ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1699375902; cv=none; b=OCj8/7pJlqcl+nhjlH5hDKreiARw2p0C21LDkM4oUxoYr0kN0hLxKSr7ySBMdG2JO8g0TQ1est7Fc1ZvJkpVUw2TX8ydE9OO04sT5Qo7ClvLxZTZRlLOWf+eKlwvFGXdT6RHB7eYlAT4tZ0K/qUAW01ciOpQnXX+kEGFop2T9JM= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1699375902; c=relaxed/simple; bh=XsMyMvEsBvzbfzeeSKFHx4ICPdXs53aKgfUHLGq6ap0=; h=DKIM-Signature:DKIM-Signature:Date:From:To:Subject:Message-ID: MIME-Version; b=NDYYvs3dEzQXzlojHhK5jAbaoM2Hc2udURP4gAD+9OazrA82cdDAV7ldcDhx84ENZXasFh1Q/gpFv2on5550QyR8kHIF4EBhb3MjR38mjJBpmZfapE+ZiYRMXCpoHgKqbPeHhAmjVQNL9apgKVMrb39KQPZt+nQlpuYb2DdUFq4= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out2.suse.de (Postfix) with ESMTP id 1F9EB1F37C for ; Tue, 7 Nov 2023 16:51:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1699375900; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc: mime-version:mime-version:content-type:content-type; bh=sH6aNs8EM293wG5VfOz2Nd5HSRkfyjxJdebi/03rA8c=; b=Kgr5V52ftOsAoZZNJZnnEQ2j78VqpMN4bnP5EGgWQHgTs3CAtL3Mss4uTqfNr3+vwJmJ5q fBOYBzRgDCTw2h87HSnl7uNJsyKOkwZb/FYAdOnqp889AXQtMeYc6Rnvf2NY+ePE+OYv9O TxZFmovAj8rV595liA4iQ5vzin1otUU= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1699375900; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc: mime-version:mime-version:content-type:content-type; bh=sH6aNs8EM293wG5VfOz2Nd5HSRkfyjxJdebi/03rA8c=; b=I4R5CrijDznwKdOg9qOqGyw6pRiBNlyuAN8XSWZ+cKMaGdaCwhLXWwhJFOc6sRdNcoL/pw YwUDg+lwSZBGmZDg== Received: from wotan.suse.de (wotan.suse.de [10.160.0.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id DE6522C16D for ; Tue, 7 Nov 2023 16:51:39 +0000 (UTC) Received: by wotan.suse.de (Postfix, from userid 10510) id 1233166A4; Tue, 7 Nov 2023 16:51:40 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by wotan.suse.de (Postfix) with ESMTP id 10FAF6588 for ; Tue, 7 Nov 2023 16:51:40 +0000 (UTC) Date: Tue, 7 Nov 2023 16:51:39 +0000 (UTC) From: Michael Matz To: binutils@sourceware.org Subject: bfd: use less memory in string merging Message-ID: User-Agent: Alpine 2.20 (LSU 67 2015-01-07) MIME-Version: 1.0 X-Spam-Status: No, score=-9.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: binutils@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Binutils mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: binutils-bounces+ouuuleilei=gmail.com@sourceware.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1781924790676519780 X-GMAIL-MSGID: 1781924790676519780 the offset-to-entry mappings are allocated in blocks, which may become a bit wasteful in case there are extremely many small input files or sections. This made it so that a large project (Qt5WebEngine) didn't build anymore on x86 32bit due to address space limits. It barely fit into address space before the new string merging, and then got pushed over the limit by this. So instead of leaving the waste reallocate the maps to their final size once known. Now the link barely fits again. bfd/ * merge.c (record_section): Reallocate offset maps to their final size. --- regtested on many targets, okay for master? bfd/merge.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/bfd/merge.c b/bfd/merge.c index 4aa2f838679..ccefb707c47 100644 --- a/bfd/merge.c +++ b/bfd/merge.c @@ -767,6 +767,18 @@ record_section (struct sec_merge_info *sinfo, free (contents); contents = NULL; + + /* We allocate the ofsmap arrays in blocks of 2048 elements. + In case we have very many small input files/sections, + this might waste large amounts of memory, so reallocate these + arrays here to their true size. */ + amt = secinfo->noffsetmap + 1; + secinfo->map_ofs = bfd_realloc (secinfo->map_ofs, + amt * sizeof(secinfo->map_ofs[0])); + BFD_ASSERT (secinfo->map_ofs); + secinfo->map = bfd_realloc (secinfo->map, amt * sizeof(secinfo->map[0])); + BFD_ASSERT (secinfo->map); + /*printf ("ZZZ %s:%s %u entries\n", sec->owner->filename, sec->name, (unsigned)secinfo->noffsetmap);*/