Message ID | 20230208171756.898991570@infradead.org |
---|---|
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp3587056wrn; Wed, 8 Feb 2023 09:32:46 -0800 (PST) X-Google-Smtp-Source: AK7set+H0ZdYX4SR+Wx4NNOm5AM/cMMGTgsmhS9HvN90987WKR5vijulVS2cSh1LC7zoo8ITGOCZ X-Received: by 2002:a17:90a:4f0c:b0:230:be88:69a with SMTP id p12-20020a17090a4f0c00b00230be88069amr7206952pjh.3.1675877565969; Wed, 08 Feb 2023 09:32:45 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675877565; cv=none; d=google.com; s=arc-20160816; b=AFtejNneZVzbIPWRcZH904yym2XmJjxZWWg3yOvIf5MJ8K8amCDtLXVNWGa4eIKapO SD45bIYeraP1LeSRbuB/KBJm2tNGaSBSXt96CVXexnyN9apzPa7ktrs2YvgfY97er0ej S7kWqE2wi4jffo2AMQlCeyK1Q8PKQ1WzzyELVF8aI9/jQ0rITKwzVB8XFdYRqxqlbTbT 8Dk+479mO4PzHu6sMandsDcxc1qdr149t6FFFtuIg0GPiszprt6na8XrcKgL0ZFU5ExS WaTWNRQx8mV0HEZ+R0J9DWWUzxX0WZqY6ATtAJvGkkkQNpI9dfz33uRpW6BpMGuZqROT JtNQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:subject:cc:to:from:date:user-agent:message-id :dkim-signature; bh=IUqBstuHz31BqYAH45XQiY9e8g79CcuPVVxrpMxtUqI=; b=NJFobQoHIdKdBclDovmwZaaS5/TM3neDFe6iogV0h+dszv4Xpp9j17jhO5vMrTm60U UtSP6699Sl88m2f/bE+ZOiB3e5MZDDryLTKGiKr5y657F9UE/o776CyG602IB2wDEHnY rJUWvxvXDKlPtedz/Isya4XwQJPnJYsEazSuSkJqhczXy0jBhQwTLsYmUtMX4WWR6/2q Y2TemzDmPNfvf6KIeMZV4wLMuLdL3P5o2HYB4YCXnfGY3tQcSqmn+ihA5JPnIytlVQxA lBCEVe2VmABJQHcrCdniP+xNt+EnI11vBZehVvFJ/TUBhw4FbahuGNYrY84oh1TPRisl C+ig== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@infradead.org header.s=desiato.20200630 header.b=qEupYAAu; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id bx16-20020a17090af49000b0022c890c2d4asi2733187pjb.179.2023.02.08.09.32.33; Wed, 08 Feb 2023 09:32:45 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@infradead.org header.s=desiato.20200630 header.b=qEupYAAu; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231570AbjBHRYI (ORCPT <rfc822;ivan.orlov0322@gmail.com> + 99 others); Wed, 8 Feb 2023 12:24:08 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55388 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229879AbjBHRYG (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Wed, 8 Feb 2023 12:24:06 -0500 Received: from desiato.infradead.org (desiato.infradead.org [IPv6:2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F2A9B30282 for <linux-kernel@vger.kernel.org>; Wed, 8 Feb 2023 09:23:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Subject:Cc:To:From:Date:Message-ID: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To:References; bh=IUqBstuHz31BqYAH45XQiY9e8g79CcuPVVxrpMxtUqI=; b=qEupYAAuMu0sDh+BbYVoGnX3mv fUj+aiYhTI+PqBBRURZo/XmMFP45wNW8rQjF6Y3xDy4rncv1A7MhB7e4vshV+FItywdeCy85CWJdk ZhKMw8KEe7rwauxmqSPgZ+ZuBSop7eZTvtPNoXVgFRCPun1K4YR5SU/+i9oL4m+WBBkyQ37kc8MoY 84yUXL9u3I+hm570fG3WHTyM0pm1p9KnV8Bkb9icwOYBkrOPiFz6XNGcmWpMsg5XPRXuvnogcNcU9 1T8iQM2+ibz4deX+O4HZ8BI2HyXdK4AWpcWYaoWRDTE+/BJCnUOwKu5oPnzjsdczSmy0cSeNU3fXQ YxCBpUDg==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by desiato.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1pPoA3-007Vvc-2V; Wed, 08 Feb 2023 17:23:12 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 14F413001CB; Wed, 8 Feb 2023 18:23:50 +0100 (CET) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id F2F7A203C41EF; Wed, 8 Feb 2023 18:23:49 +0100 (CET) Message-ID: <20230208171756.898991570@infradead.org> User-Agent: quilt/0.66 Date: Wed, 08 Feb 2023 18:17:56 +0100 From: Peter Zijlstra <peterz@infradead.org> To: x86@kernel.org, jpoimboe@redhat.com, linux@weissschuh.net Cc: linux-kernel@vger.kernel.org, peterz@infradead.org Subject: [PATCH 00/10] objtool: Honey, I shrunk the instruction X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757284994938120632?= X-GMAIL-MSGID: =?utf-8?q?1757284994938120632?= |
Series |
objtool: Honey, I shrunk the instruction
|
|
Message
Peter Zijlstra
Feb. 8, 2023, 5:17 p.m. UTC
Hi, Boris complained he could no longer build allyesconfig on his 32G desktop machine without having OOM terminate either objtool or chrome. After talking about these patches on IRC, Nathan mentioned the linux-clang CI was also having trouble of recent, and these patches appear to make it happy again. In total these patches shrink an allyesconfig run by about 6G: pre: 5:58.22 real, 226.69 user, 131.22 sys, 26221520 mem post: 5:03.34 real, 210.75 user, 88.80 sys, 20241232 mem Also at: https://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git/log/?h=objtool/core
Comments
On Wed, Feb 08, 2023 at 06:17:56PM +0100, Peter Zijlstra wrote: > Hi, > > Boris complained he could no longer build allyesconfig on his 32G desktop > machine without having OOM terminate either objtool or chrome. > > After talking about these patches on IRC, Nathan mentioned the linux-clang CI > was also having trouble of recent, and these patches appear to make it happy > again. > > In total these patches shrink an allyesconfig run by about 6G: > > pre: 5:58.22 real, 226.69 user, 131.22 sys, 26221520 mem > post: 5:03.34 real, 210.75 user, 88.80 sys, 20241232 mem > > Also at: > > https://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git/log/?h=objtool/core For patches 1-9: Tested-by: Nathan Chancellor <nathan@kernel.org> # build only Prior to this series: [INFO] Memory used: 25.09GB After this series: [INFO] Memory used: 19.27GB Our builds on TuxSuite were consistenly timing out after four hours and they had no problem passing with this series (the worst time was 2.2h, which is line with the VM specs that they use I believe): https://tuxapi.tuxsuite.com/v1/groups/clangbuiltlinux/projects/nathan/plans/2LQbNuWRo3Xf62Yg3SINuA9d7cR Thanks a lot! Cheers, Nathan
On Wed, 08. Feb 18:17, Peter Zijlstra wrote: > Hi, > > Boris complained he could no longer build allyesconfig on his 32G desktop > machine without having OOM terminate either objtool or chrome. > > After talking about these patches on IRC, Nathan mentioned the linux-clang CI > was also having trouble of recent, and these patches appear to make it happy > again. > > In total these patches shrink an allyesconfig run by about 6G: > > pre: 5:58.22 real, 226.69 user, 131.22 sys, 26221520 mem > post: 5:03.34 real, 210.75 user, 88.80 sys, 20241232 mem > > Also at: > > https://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git/log/?h=objtool/core > Hello Peter, with clang-17 the build failed: In file included from weak.c:10: In file included from /home/damian/kernel/linux/tools/objtool/include/objtool/objtool.h:13: /home/damian/kernel/linux/tools/objtool/include/objtool/elf.h:88:86: error: '_Static_assert' with no message is a C2x extension [-Werror,-Wc2x-extensions] static_assert(offsetof(struct reloc, rela.r_offset) == offsetof(struct reloc, offset)); ^ , "" In file included from arch/x86/special.c:4: In file included from /home/damian/kernel/linux/tools/objtool/include/objtool/special.h:10: In file included from /home/damian/kernel/linux/tools/objtool/include/objtool/check.h:11/home/damian/kernel/linux/tools/objtool/include/objtool/elf.h: In file included from :89:86: error: '_Static_assert' with no message is a C2x extension [-Werror,-Wc2x-extensions] /home/damian/kernel/linux/tools/objtool/include/objtool/arch.h:11: In file included from static_assert(offsetof(struct reloc, rela.r_addend) == offsetof(struct reloc, addend)); ^ , "" /home/damian/kernel/linux/tools/objtool/include/objtool/objtool.h:13: /home/damian/kernel/linux/tools/objtool/include/objtool/elf.h:88:86: error: '_Static_assert' with no message is a C2x extension [-Werror,-Wc2x-extensions] static_assert(offsetof(struct reloc, rela.r_offset) == offsetof(struct reloc, offset)); ^ , "" /home/damian/kernel/linux/tools/objtool/include/objtool/elf.h:89:86: error: '_Static_assert' with no message is a C2x extension [-Werror,-Wc2x-extensions] static_assert(offsetof(struct reloc, rela.r_addend) == offsetof(struct reloc, addend)); ^ , "" 2 errors generated. In file included from builtin-check.c:10: In file included from /home/damian/kernel/linux/tools/objtool/include/objtool/objtool.h:13: /home/damian/kernel/linux/tools/objtool/include/objtool/elf.h:88:86: error: '_Static_assert' with no message is a C2x extension [-Werror,-Wc2x-extensions] static_assert(offsetof(struct reloc, rela.r_offset) == offsetof(struct reloc, offset)); ... Best regards Damian
On Wed, Feb 08, 2023 at 09:22:02PM +0100, Damian Tometzki wrote: > On Wed, 08. Feb 18:17, Peter Zijlstra wrote: > > Hi, > > > > Boris complained he could no longer build allyesconfig on his 32G desktop > > machine without having OOM terminate either objtool or chrome. > > > > After talking about these patches on IRC, Nathan mentioned the linux-clang CI > > was also having trouble of recent, and these patches appear to make it happy > > again. > > > > In total these patches shrink an allyesconfig run by about 6G: > > > > pre: 5:58.22 real, 226.69 user, 131.22 sys, 26221520 mem > > post: 5:03.34 real, 210.75 user, 88.80 sys, 20241232 mem > > > > Also at: > > > > https://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git/log/?h=objtool/core > > > Hello Peter, > > with clang-17 the build failed: > In file included from weak.c:10: > In file included from /home/damian/kernel/linux/tools/objtool/include/objtool/objtool.h:13: > /home/damian/kernel/linux/tools/objtool/include/objtool/elf.h:88:86: error: '_Static_assert' with no message is a C2x extension [-Werror,-Wc2x-extensions] > static_assert(offsetof(struct reloc, rela.r_offset) == offsetof(struct reloc, offset)); Oh urgh. Apparently the kernel wrapper went missing in this userspace project :-) include/linux/build_bug.h:#define static_assert(expr, ...) __static_assert(expr, ##__VA_ARGS__, #expr) Anyway, it's that last patch and I meant to do that in a slightly less horrid way :-)
On Thu, 09. Feb 11:22, Peter Zijlstra wrote: > On Wed, Feb 08, 2023 at 09:22:02PM +0100, Damian Tometzki wrote: > > On Wed, 08. Feb 18:17, Peter Zijlstra wrote: > > > Hi, > > > > > > Boris complained he could no longer build allyesconfig on his 32G desktop > > > machine without having OOM terminate either objtool or chrome. > > > > > > After talking about these patches on IRC, Nathan mentioned the linux-clang CI > > > was also having trouble of recent, and these patches appear to make it happy > > > again. > > > > > > In total these patches shrink an allyesconfig run by about 6G: > > > > > > pre: 5:58.22 real, 226.69 user, 131.22 sys, 26221520 mem > > > post: 5:03.34 real, 210.75 user, 88.80 sys, 20241232 mem > > > > > > Also at: > > > > > > https://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git/log/?h=objtool/core > > > > > Hello Peter, > > > > with clang-17 the build failed: > > In file included from weak.c:10: > > In file included from /home/damian/kernel/linux/tools/objtool/include/objtool/objtool.h:13: > > /home/damian/kernel/linux/tools/objtool/include/objtool/elf.h:88:86: error: '_Static_assert' with no message is a C2x extension [-Werror,-Wc2x-extensions] > > static_assert(offsetof(struct reloc, rela.r_offset) == offsetof(struct reloc, offset)); > > Oh urgh. Apparently the kernel wrapper went missing in this userspace > project :-) > > include/linux/build_bug.h:#define static_assert(expr, ...) __static_assert(expr, ##__VA_ARGS__, #expr) > > Anyway, it's that last patch and I meant to do that in a slightly less > horrid way :-) Hello Peter, here a shortipossible fix: Then the build with clang works. diff --git a/tools/objtool/Makefile b/tools/objtool/Makefile index 83b100c1e7f6..b7c8b476db95 100644 --- a/tools/objtool/Makefile +++ b/tools/objtool/Makefile @@ -31,7 +31,7 @@ INCLUDES := -I$(srctree)/tools/include \ -I$(LIBSUBCMD_OUTPUT)/include # Note, EXTRA_WARNINGS here was determined for CC and not HOSTCC, it # is passed here to match a legacy behavior. -WARNINGS := $(EXTRA_WARNINGS) -Wno-switch-default -Wno-switch-enum -Wno-packed -Wno-nested-externs +WARNINGS := $(EXTRA_WARNINGS) -Wno-switch-default -Wno-switch-enum -Wno-packed -Wno-nested-externs -Wno-c2x-extensions OBJTOOL_CFLAGS := -Werror $(WARNINGS) $(KBUILD_HOSTCFLAGS) -g $(INCLUDES) $(LIBELF_FLAGS)
On Wed, Feb 08, 2023 at 06:17:56PM +0100, Peter Zijlstra wrote: > Hi, > > Boris complained he could no longer build allyesconfig on his 32G desktop > machine without having OOM terminate either objtool or chrome. > > After talking about these patches on IRC, Nathan mentioned the linux-clang CI > was also having trouble of recent, and these patches appear to make it happy > again. > > In total these patches shrink an allyesconfig run by about 6G: > > pre: 5:58.22 real, 226.69 user, 131.22 sys, 26221520 mem > post: 5:03.34 real, 210.75 user, 88.80 sys, 20241232 mem > > Also at: > > https://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git/log/?h=objtool/core For patches 1-9: Acked-by: Josh Poimboeuf <jpoimboe@kernel.org> To make the last patch legit we could just have reloc_offset(), reloc_addend(), reloc_type() helpers to abstract access to rela/rel fields. And I'm sure there are other savings we could do to struct reloc, like single-linked lists, bitfields, etc. Let me know if you want me to do it.
On Wed, Feb 08, 2023 at 06:17:56PM +0100, Peter Zijlstra wrote: > Hi, > > Boris complained he could no longer build allyesconfig on his 32G desktop > machine without having OOM terminate either objtool or chrome. > > After talking about these patches on IRC, Nathan mentioned the linux-clang CI > was also having trouble of recent, and these patches appear to make it happy > again. > > In total these patches shrink an allyesconfig run by about 6G: > > pre: 5:58.22 real, 226.69 user, 131.22 sys, 26221520 mem > post: 5:03.34 real, 210.75 user, 88.80 sys, 20241232 mem Thanks! This makes a huge difference on my memory-starved travel machine. Seems to be back to about 6.0 levels. Tested-by: Thomas Weißschuh <linux@weissschuh.net> # compile and run > Also at: > > https://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git/log/?h=objtool/core >
On Thu, Feb 09, 2023 at 11:57:40AM -0800, Josh Poimboeuf wrote: > To make the last patch legit we could just have reloc_offset(), > reloc_addend(), reloc_type() helpers to abstract access to rela/rel > fields. > > And I'm sure there are other savings we could do to struct reloc, like > single-linked lists, bitfields, etc. > > Let me know if you want me to do it. I'll make a start today -- I've got snot pouring out of me, so thinking isn't going to happen, might as well just do the busy work :-)
On Fri, Feb 10, 2023 at 09:59:05AM +0100, Peter Zijlstra wrote: > On Thu, Feb 09, 2023 at 11:57:40AM -0800, Josh Poimboeuf wrote: > > > To make the last patch legit we could just have reloc_offset(), > > reloc_addend(), reloc_type() helpers to abstract access to rela/rel > > fields. > > > > And I'm sure there are other savings we could do to struct reloc, like > > single-linked lists, bitfields, etc. > > > > Let me know if you want me to do it. > > I'll make a start today -- I've got snot pouring out of me, so thinking > isn't going to happen, might as well just do the busy work :-) (lightly tested...) Shrinks reloc from 120 to 80 bytes: https://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git/log/?h=objtool/shrink-reloc pre: 5:03.34 real, 210.75 user, 88.80 sys, 20241232 mem post: 4:12.71 real, 188.21 user, 64.49 sys, 18144848 mem (still running on the exact same allyesconfig image) I can probably get rid of symbol::reloc_list if I rewrite that whole symtable trainwreck and I think I can also get rid of instruction::alt_group, but both of those are a bit more work.