Message ID | 20230619143725.57967-1-masahiroy@kernel.org |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp3062318vqr; Mon, 19 Jun 2023 07:59:43 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4mY31s1DyBgvWwE35kRIaG1XPtmPtlBlySH76YcMP8JGRjz3x6t9vdz2RvqL9Q3VOnjuHd X-Received: by 2002:a05:6808:1311:b0:39e:1d17:d687 with SMTP id y17-20020a056808131100b0039e1d17d687mr10777227oiv.57.1687186783694; Mon, 19 Jun 2023 07:59:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1687186783; cv=none; d=google.com; s=arc-20160816; b=vv2JMObOLHZg2fEKsFfJN0JvnlBU8W5h5LVbI8Ds5ATyJJytFGMkK6cNVIio2+I6Qc 91f85GIoS+zLA5KrggidKkjhnlKExO0z4ZhZga3x1uLDkAJj7/OJEnUXIhxX85Wa1PRr M3ml1/L+FVpg6M7o7WZm/bfrzkJ9TycyCzLv7ZQrJEeFF6gRu39+dC99EaKM103Mba59 lXXki3+TdPPFGL/ettuiSo2mffcl6tH0Zw0g5SwVo5rucEn4I6CJsUO2dTTWjtOiYVOX 27mUnz4Y++ailTRmMdbVGhQU5XBZUSDFx26dqyomf7uNCwVIV75O4oB9lYhOKxJrnLRe ek5A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=I3t8wAM2HYvRxA8dT4wipQsSG/I3Xffya/Moz8vpzjs=; b=RRLAMjT1rOc/IGD7aCBLzBRoMPrkBWQUGTywPWWg61BZMTczDfHml838YBI9TzVnUh DzouWAyGgf+zpTB93sqMI335NuBKHSJPUFHCkFFC0s82rd7anPu3eTEYVDc5p1vMBw+x Wtb7ITukpvHe5S++SvqaPflMGwjlcAxJznac3UyMPXPLd2NroijwhmUgp0ybtRFdB9K6 nRSPviy4pxQ7A+FCvurxeg762+3nv7+dVWGdgV4np3aVRjEernRQoBWqG30jMt7LuDma gbWic6f59Zr1SfVA+vhsxLidlDCoTvCCbOceuq3T0/h6U8XipQhF1TFWGs/8eHOCBWLt riAQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=QH0bwh7t; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id p9-20020a639509000000b00544538e2041si22950172pgd.305.2023.06.19.07.59.31; Mon, 19 Jun 2023 07:59:43 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=QH0bwh7t; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229765AbjFSOhh (ORCPT <rfc822;duw91626@gmail.com> + 99 others); Mon, 19 Jun 2023 10:37:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39096 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231701AbjFSOhe (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Mon, 19 Jun 2023 10:37:34 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5E376E68; Mon, 19 Jun 2023 07:37:32 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 9DC3360CA5; Mon, 19 Jun 2023 14:37:31 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 54BD1C433C0; Mon, 19 Jun 2023 14:37:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1687185451; bh=cKXZj60yRSK1IRgf3ZaAN3xJcXaOIkypoexKrxHMOhs=; h=From:To:Cc:Subject:Date:From; b=QH0bwh7t/Urer7GKMlsBQid+Lf8kVUaQTAtIDH76W4UJYv1gGjxTg2yNqLDo4zKnz ToVQqgo1qSFzOkQNm5YHLgf/0lrDc1nZdm/XZ7NNiFHck64gJC98ol21RnDSTu2I7k 9Fw3L5OVAk68dKIr1AB5nxJEB+/70lbS9KUQdz+MpacjJz0uZ1WKiHVt2vsv/p1hwF S0IM1Pbh+CgIaC0SupzbmqAHH6pz5mhSZ7B/+snZsvIdSTdSnvlKz36n31ILHDk4GS 2ZoY5kBKGu5kpL6ANlRpeJ1V48gghKpjehS27GNYhxmgp/rx3dPYqNPP1blHJjeUrM wtpYhJa6QQk8w== From: Masahiro Yamada <masahiroy@kernel.org> To: linux-arm-kernel@lists.infradead.org, linux-kbuild@vger.kernel.org, Russell King <linux@armlinux.org.uk> Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel <ardb@kernel.org>, Nick Desaulniers <ndesaulniers@google.com>, Masahiro Yamada <masahiroy@kernel.org> Subject: [PATCH] ARM: change link order of $(mmy-y) to avoid veneers Date: Mon, 19 Jun 2023 23:37:25 +0900 Message-Id: <20230619143725.57967-1-masahiroy@kernel.org> X-Mailer: git-send-email 2.39.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1769143568892226500?= X-GMAIL-MSGID: =?utf-8?q?1769143568892226500?= |
Series |
ARM: change link order of $(mmy-y) to avoid veneers
|
|
Commit Message
Masahiro Yamada
June 19, 2023, 2:37 p.m. UTC
The kernel compiled with multi_v7_defconfig + CONFIG_KASAN=y +
CONFIG_KASAN_INLINE=y does not boot.
I do not think KASAN is the direct reason of the boot failure.
CONFIG_KASAN_INLINE is just one example configuration that grows the
image size significantly and makes a big distance between function
callers and callees.
I see some veneers for __get_user_* in the bad kernel image. I am
not perfectly clear, but __get_user_* may not work with veneers for
some reasons.
If I move the link order of arch/arm/lib/getuser.S, the veneers are
gone, and the kernel gets working again.
I do not see a good reason that $(mmu-y) must be added to lib-y because
all the code in $(mmu-y) is mandatory. Add it to obj-y to move the code
to lower address.
[1] multi_v7_defconfig (works)
$ arm-linux-gnueabihf-nm vmlinux | grep __get_user_1
c072a450 T __get_user_1
c17ea033 r __kstrtab___get_user_1
c18119fe r __kstrtabns___get_user_1
c17c4878 r __ksymtab___get_user_1
[2] multi_v7_defconfig + CONFIG_KASAN_INLINE (does not work)
$ arm-linux-gnueabihf-nm vmlinux | grep __get_user_1
c341ec2c T __get_user_1
c06e3580 t ____get_user_1_veneer
c0adc6c0 t ____get_user_1_veneer
c12cf054 t ____get_user_1_veneer
c43f42cc r __kstrtab___get_user_1
c441c128 r __kstrtabns___get_user_1
c43cead8 r __ksymtab___get_user_1
[3] multi_v7_defconfig + CONFIG_KASAN_INLINE + this patch (works)
$ arm-linux-gnueabihf-nm vmlinux | grep __get_user_1
c10975b0 T __get_user_1
c43f42cc r __kstrtab___get_user_1
c441c128 r __kstrtabns___get_user_1
c43cead8 r __ksymtab___get_user_1
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
---
arch/arm/lib/Makefile | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
Comments
On Mon, Jun 19, 2023 at 11:37 PM Masahiro Yamada <masahiroy@kernel.org> wrote: > > The kernel compiled with multi_v7_defconfig + CONFIG_KASAN=y + > CONFIG_KASAN_INLINE=y does not boot. > > I do not think KASAN is the direct reason of the boot failure. > CONFIG_KASAN_INLINE is just one example configuration that grows the > image size significantly and makes a big distance between function > callers and callees. > > I see some veneers for __get_user_* in the bad kernel image. I am > not perfectly clear, but __get_user_* may not work with veneers for > some reasons. > > If I move the link order of arch/arm/lib/getuser.S, the veneers are > gone, and the kernel gets working again. > > I do not see a good reason that $(mmu-y) must be added to lib-y because > all the code in $(mmu-y) is mandatory. Add it to obj-y to move the code > to lower address. > > [1] multi_v7_defconfig (works) > > $ arm-linux-gnueabihf-nm vmlinux | grep __get_user_1 > c072a450 T __get_user_1 > c17ea033 r __kstrtab___get_user_1 > c18119fe r __kstrtabns___get_user_1 > c17c4878 r __ksymtab___get_user_1 > > [2] multi_v7_defconfig + CONFIG_KASAN_INLINE (does not work) > > $ arm-linux-gnueabihf-nm vmlinux | grep __get_user_1 > c341ec2c T __get_user_1 > c06e3580 t ____get_user_1_veneer > c0adc6c0 t ____get_user_1_veneer > c12cf054 t ____get_user_1_veneer > c43f42cc r __kstrtab___get_user_1 > c441c128 r __kstrtabns___get_user_1 > c43cead8 r __ksymtab___get_user_1 > > [3] multi_v7_defconfig + CONFIG_KASAN_INLINE + this patch (works) > > $ arm-linux-gnueabihf-nm vmlinux | grep __get_user_1 > c10975b0 T __get_user_1 > c43f42cc r __kstrtab___get_user_1 > c441c128 r __kstrtabns___get_user_1 > c43cead8 r __ksymtab___get_user_1 > > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> > --- > > arch/arm/lib/Makefile | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/arch/arm/lib/Makefile b/arch/arm/lib/Makefile > index 650404be6768..4d092ef87a1d 100644 > --- a/arch/arm/lib/Makefile > +++ b/arch/arm/lib/Makefile > @@ -28,7 +28,7 @@ endif > # using lib_ here won't override already available weak symbols > obj-$(CONFIG_UACCESS_WITH_MEMCPY) += uaccess_with_memcpy.o > > -lib-$(CONFIG_MMU) += $(mmu-y) > +obj-$(CONFIG_MMU) += $(mmu-y) > > ifeq ($(CONFIG_CPU_32v3),y) > lib-y += io-readsw-armv3.o io-writesw-armv3.o > -- > 2.39.2 > I also wonder why __get_user* is defined in arch/arm/lib/ instead of arch/arm/kernel/, but I might be a convention. x86 puts it in the lib directory, arch/x86/lib/getuser.S then other architectures copy the code structure.
Subject: [PATCH] ARM: change link order of $(mmy-y) to avoid veneers There is a typo in the subject (mmy-y -> mmu-y). Kind regards, Nicolas On Mon, Jun 19, 2023 at 11:37:25PM +0900, Masahiro Yamada wrote: > The kernel compiled with multi_v7_defconfig + CONFIG_KASAN=y + > CONFIG_KASAN_INLINE=y does not boot. > > I do not think KASAN is the direct reason of the boot failure. > CONFIG_KASAN_INLINE is just one example configuration that grows the > image size significantly and makes a big distance between function > callers and callees. > > I see some veneers for __get_user_* in the bad kernel image. I am > not perfectly clear, but __get_user_* may not work with veneers for > some reasons. > > If I move the link order of arch/arm/lib/getuser.S, the veneers are > gone, and the kernel gets working again. > > I do not see a good reason that $(mmu-y) must be added to lib-y because > all the code in $(mmu-y) is mandatory. Add it to obj-y to move the code > to lower address. > > [1] multi_v7_defconfig (works) > > $ arm-linux-gnueabihf-nm vmlinux | grep __get_user_1 > c072a450 T __get_user_1 > c17ea033 r __kstrtab___get_user_1 > c18119fe r __kstrtabns___get_user_1 > c17c4878 r __ksymtab___get_user_1 > > [2] multi_v7_defconfig + CONFIG_KASAN_INLINE (does not work) > > $ arm-linux-gnueabihf-nm vmlinux | grep __get_user_1 > c341ec2c T __get_user_1 > c06e3580 t ____get_user_1_veneer > c0adc6c0 t ____get_user_1_veneer > c12cf054 t ____get_user_1_veneer > c43f42cc r __kstrtab___get_user_1 > c441c128 r __kstrtabns___get_user_1 > c43cead8 r __ksymtab___get_user_1 > > [3] multi_v7_defconfig + CONFIG_KASAN_INLINE + this patch (works) > > $ arm-linux-gnueabihf-nm vmlinux | grep __get_user_1 > c10975b0 T __get_user_1 > c43f42cc r __kstrtab___get_user_1 > c441c128 r __kstrtabns___get_user_1 > c43cead8 r __ksymtab___get_user_1 > > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> > --- > > arch/arm/lib/Makefile | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/arch/arm/lib/Makefile b/arch/arm/lib/Makefile > index 650404be6768..4d092ef87a1d 100644 > --- a/arch/arm/lib/Makefile > +++ b/arch/arm/lib/Makefile > @@ -28,7 +28,7 @@ endif > # using lib_ here won't override already available weak symbols > obj-$(CONFIG_UACCESS_WITH_MEMCPY) += uaccess_with_memcpy.o > > -lib-$(CONFIG_MMU) += $(mmu-y) > +obj-$(CONFIG_MMU) += $(mmu-y) > > ifeq ($(CONFIG_CPU_32v3),y) > lib-y += io-readsw-armv3.o io-writesw-armv3.o > -- > 2.39.2 >
On Mon, Jun 19, 2023 at 10:37 AM Masahiro Yamada <masahiroy@kernel.org> wrote: > > The kernel compiled with multi_v7_defconfig + CONFIG_KASAN=y + > CONFIG_KASAN_INLINE=y does not boot. > > I do not think KASAN is the direct reason of the boot failure. > CONFIG_KASAN_INLINE is just one example configuration that grows the > image size significantly and makes a big distance between function > callers and callees. > > I see some veneers for __get_user_* in the bad kernel image. I am > not perfectly clear, but __get_user_* may not work with veneers for > some reasons. I'm kind of curious to know more about this. Has there been other instances in the ARCH=arm port where "there must not be a veneer from X to Y for reason Z?" I thought the linker inserted veneers were meant to be transparent here. If you disassemble ____get_user_1_veneer, do they themselves branch to different symbols, or the same symbol? (Perhaps they trampoline to each other then the final one is the "call" to the original symbol). But perhaps the symbol at the end of the chain gives us more clues. I'd bet it's the KASAN callback, though does KASAN_INLINE inline those? Perhaps the veneer is corrupting a register? Maybe inline asm in the caller is missing a clobber for that register... > > If I move the link order of arch/arm/lib/getuser.S, the veneers are > gone, and the kernel gets working again. > > I do not see a good reason that $(mmu-y) must be added to lib-y because > all the code in $(mmu-y) is mandatory. Add it to obj-y to move the code > to lower address. > > [1] multi_v7_defconfig (works) > > $ arm-linux-gnueabihf-nm vmlinux | grep __get_user_1 > c072a450 T __get_user_1 > c17ea033 r __kstrtab___get_user_1 > c18119fe r __kstrtabns___get_user_1 > c17c4878 r __ksymtab___get_user_1 > > [2] multi_v7_defconfig + CONFIG_KASAN_INLINE (does not work) > > $ arm-linux-gnueabihf-nm vmlinux | grep __get_user_1 > c341ec2c T __get_user_1 > c06e3580 t ____get_user_1_veneer > c0adc6c0 t ____get_user_1_veneer > c12cf054 t ____get_user_1_veneer > c43f42cc r __kstrtab___get_user_1 > c441c128 r __kstrtabns___get_user_1 > c43cead8 r __ksymtab___get_user_1 > > [3] multi_v7_defconfig + CONFIG_KASAN_INLINE + this patch (works) > > $ arm-linux-gnueabihf-nm vmlinux | grep __get_user_1 > c10975b0 T __get_user_1 > c43f42cc r __kstrtab___get_user_1 > c441c128 r __kstrtabns___get_user_1 > c43cead8 r __ksymtab___get_user_1 > > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> > --- > > arch/arm/lib/Makefile | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/arch/arm/lib/Makefile b/arch/arm/lib/Makefile > index 650404be6768..4d092ef87a1d 100644 > --- a/arch/arm/lib/Makefile > +++ b/arch/arm/lib/Makefile > @@ -28,7 +28,7 @@ endif > # using lib_ here won't override already available weak symbols > obj-$(CONFIG_UACCESS_WITH_MEMCPY) += uaccess_with_memcpy.o > > -lib-$(CONFIG_MMU) += $(mmu-y) > +obj-$(CONFIG_MMU) += $(mmu-y) > > ifeq ($(CONFIG_CPU_32v3),y) > lib-y += io-readsw-armv3.o io-writesw-armv3.o > -- > 2.39.2 >
diff --git a/arch/arm/lib/Makefile b/arch/arm/lib/Makefile index 650404be6768..4d092ef87a1d 100644 --- a/arch/arm/lib/Makefile +++ b/arch/arm/lib/Makefile @@ -28,7 +28,7 @@ endif # using lib_ here won't override already available weak symbols obj-$(CONFIG_UACCESS_WITH_MEMCPY) += uaccess_with_memcpy.o -lib-$(CONFIG_MMU) += $(mmu-y) +obj-$(CONFIG_MMU) += $(mmu-y) ifeq ($(CONFIG_CPU_32v3),y) lib-y += io-readsw-armv3.o io-writesw-armv3.o