| Message ID | 20221020103823.31001-1-ashimida@linux.alibaba.com |
|---|---|
| State | New |
| Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org>
Delivered-To: ouuuleilei@gmail.com
Received: by 2002:a5d:4242:0:0:0:0:0 with SMTP id s2csp26411wrr;
Thu, 20 Oct 2022 03:39:59 -0700 (PDT)
X-Google-Smtp-Source:
AMsMyM406M959QV0vJ+uzVcxrSM17QztZPBYML5N8UGJKGAu9P67NsRTF+x42IvUZFhAmVPTBynW
X-Received: by 2002:a63:513:0:b0:462:f15e:5efc with SMTP id
19-20020a630513000000b00462f15e5efcmr11586094pgf.113.1666262399068;
Thu, 20 Oct 2022 03:39:59 -0700 (PDT)
ARC-Seal: i=1; a=rsa-sha256; t=1666262399; cv=none;
d=google.com; s=arc-20160816;
b=QC22eFVCKqdE6HgH9WCgY87g0yqiluf7rRDGpmlQaoFD1ne9Rwg/7xzMjH36GW7h9o
JbMkUnzEu3T9AcG6HCX/NbwU//fUB6n2utuP5G48CKOQ2eEzVHCtKY/3wGyacHYcQsGt
pseNg18s0P+U0gEzOC57X+kCwdwhM1NlXKNzlOHg4/QL5Yg2hyR76xwS5QUWABjRe8aO
EhpwWBehZdf9OW6zUz9qPZSvOFE2lGfLBkomQ9wKBA2/SWki0G+V8gDgThPLDGHCFjTk
dRe0VgditOhNVTNF2C83vCPNO0LkxHVoUsrLE7oB9a/0//tVTMZvTQDLfu/C9Hge5UzQ
yQ5A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com;
s=arc-20160816;
h=list-id:precedence:message-id:date:subject:cc:to:from;
bh=Te4rLpHNvJyllNsKPSaW2YvWeVYj0zcpsIqOp8qcRVs=;
b=I2Qu+MhDzCY+v9u6zlNArBrQ5ZqIWyLEBoz8moaKy9Ufr/1j9oO7k38tpMPe8FrzWy
ZVwFRXu9TG9b0N3FuCdRkmlJN+0duyyuitOHA9mdD581NGJaII4EKNOo1OoOXsODz85P
YVyji5UzWkLxA28L5lKvgNwUC/yONTwyodqTAQ3NpelNUEpjojT+3GJIkkZnBdX+ddFs
ioSbiRZ88gPP6FjsIwIE/Uo0hz8DBHXTUNi3gjxueZLh2+yZpu6S16WUxpOtrfwbcZpx
vkCg7SI7LntRR5Q7u6JbmPLR/a9ZM2KVenNUCA/mK6/EnueQTP9NUcea3WvInhjIpeyV
perA==
ARC-Authentication-Results: i=1; mx.google.com;
spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org
designates 2620:137:e000::1:20 as permitted sender)
smtp.mailfrom=linux-kernel-owner@vger.kernel.org;
dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com
Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20])
by mx.google.com with ESMTP id
g63-20020a636b42000000b0046004f18c6csi21243426pgc.456.2022.10.20.03.39.45;
Thu, 20 Oct 2022 03:39:59 -0700 (PDT)
Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org
designates 2620:137:e000::1:20 as permitted sender)
client-ip=2620:137:e000::1:20;
Authentication-Results: mx.google.com;
spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org
designates 2620:137:e000::1:20 as permitted sender)
smtp.mailfrom=linux-kernel-owner@vger.kernel.org;
dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
id S229988AbiJTKjX (ORCPT <rfc822;ruipengqi7@gmail.com> + 99 others);
Thu, 20 Oct 2022 06:39:23 -0400
Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33634 "EHLO
lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
with ESMTP id S231181AbiJTKir (ORCPT
<rfc822;linux-kernel@vger.kernel.org>);
Thu, 20 Oct 2022 06:38:47 -0400
Received: from out30-131.freemail.mail.aliyun.com
(out30-131.freemail.mail.aliyun.com [115.124.30.131])
by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EA1851DEC20;
Thu, 20 Oct 2022 03:38:35 -0700 (PDT)
X-Alimail-AntiSpam:
AC=PASS;BC=-1|-1;BR=01201311R331e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045176;MF=ashimida@linux.alibaba.com;NM=1;PH=DS;RN=8;SR=0;TI=SMTPD_---0VSeueAo_1666262304;
Received: from localhost(mailfrom:ashimida@linux.alibaba.com
fp:SMTPD_---0VSeueAo_1666262304)
by smtp.aliyun-inc.com;
Thu, 20 Oct 2022 18:38:31 +0800
From: Dan Li <ashimida@linux.alibaba.com>
To: masahiroy@kernel.org, michal.lkml@markovi.net,
ndesaulniers@google.com, corbet@lwn.net
Cc: linux-kbuild@vger.kernel.org, linux-doc@vger.kernel.org,
linux-kernel@vger.kernel.org, Dan Li <ashimida@linux.alibaba.com>
Subject: [RFC] Documentation: kbuild: Add description of git for reproducible
builds
Date: Thu, 20 Oct 2022 03:38:23 -0700
Message-Id: <20221020103823.31001-1-ashimida@linux.alibaba.com>
X-Mailer: git-send-email 2.17.1
X-Spam-Status: No, score=-9.9 required=5.0 tests=BAYES_00,
ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,
UNPARSEABLE_RELAY,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no
version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?=
X-GMAIL-THRID: =?utf-8?q?1747202761649557321?=
X-GMAIL-MSGID: =?utf-8?q?1747202761649557321?=
|
| Series |
[RFC] Documentation: kbuild: Add description of git for reproducible builds
|
|
Commit Message
Dan Li
Oct. 20, 2022, 10:38 a.m. UTC
The status of git will affect the final compilation result, add it to
the documentation of reproducible builds.
Signed-off-by: Dan Li <ashimida@linux.alibaba.com>
---
Documentation/kbuild/reproducible-builds.rst | 10 ++++++++++
1 file changed, 10 insertions(+)
Comments
On Thu, Oct 20, 2022 at 03:38:23AM -0700, Dan Li wrote: > diff --git a/Documentation/kbuild/reproducible-builds.rst b/Documentation/kbuild/reproducible-builds.rst > index 071f0151a7a4..13397f38c358 100644 > --- a/Documentation/kbuild/reproducible-builds.rst > +++ b/Documentation/kbuild/reproducible-builds.rst > @@ -119,6 +119,16 @@ To avoid this, you can make the vDSO different for different > kernel versions by including an arbitrary string of "salt" in it. > This is specified by the Kconfig symbol ``CONFIG_BUILD_SALT``. > > +Git > +----------------------- > + > +Uncommitted changes or different commit ids in git can also lead > +to different compilation results. For example, after executing > +``git reset HEAD^``, even if the code is the same, the > +``include/config/kernel.release`` generated during compilation > +will be different, which will eventually lead to binary differences. > +See ``scripts/setlocalversion`` for details. > + Briefly read the script, I don't see what the correlation between git reset with LOCALVERSION thing is. Also, does the exact state of git repository required for reproducible builds? Thanks.
On 10/20/22 07:05, Bagas Sanjaya wrote: > On Thu, Oct 20, 2022 at 03:38:23AM -0700, Dan Li wrote: >> diff --git a/Documentation/kbuild/reproducible-builds.rst b/Documentation/kbuild/reproducible-builds.rst >> index 071f0151a7a4..13397f38c358 100644 >> --- a/Documentation/kbuild/reproducible-builds.rst >> +++ b/Documentation/kbuild/reproducible-builds.rst >> @@ -119,6 +119,16 @@ To avoid this, you can make the vDSO different for different >> kernel versions by including an arbitrary string of "salt" in it. >> This is specified by the Kconfig symbol ``CONFIG_BUILD_SALT``. >> >> +Git >> +----------------------- >> + >> +Uncommitted changes or different commit ids in git can also lead >> +to different compilation results. For example, after executing >> +``git reset HEAD^``, even if the code is the same, the >> +``include/config/kernel.release`` generated during compilation >> +will be different, which will eventually lead to binary differences. >> +See ``scripts/setlocalversion`` for details. >> + > > Briefly read the script, I don't see what the correlation between git > reset with LOCALVERSION thing is. Also, does the exact state of git > repository required for reproducible builds? > Hi Bagas, The Makefile has the following code: filechk_kernel.release = \ echo "$(KERNELVERSION)$$($(CONFIG_SHELL) $(srctree)/scripts/setlocalversion $(srctree))" The output of setlocalversion affects kernel.release, while the output of setlocalversion is related to the state of git when the git repository exists (see function scm_version). So changes in git state will result in changes to kernel.release, and this information will be included in the final output vmlinux/modules and in turn affect reproducible builds. For example: $ git log commit 4cd155a93eec...... $ make ... $ cat include/config/kernel.release 6.0.0-rc4-00025-g4cd155a93eec $ git reset HEAD^ $ git log commit 7b4d266b0c41...... $ make ... $ cat include/config/kernel.release 6.0.0-rc4-00024-g7b4d266b0c41-dirty AFAICT, in the presence of a git repository, we can compile a reproducible build kernel in any git state, but we need to ensure that the git state is always the same between compilations (or the same from the perspective of the scm_version function). Thanks, Dan. > Thanks. >
On Thu, Oct 20, 2022 at 06:48:20PM -0700, Dan Li wrote: > > > On 10/20/22 07:05, Bagas Sanjaya wrote: > > On Thu, Oct 20, 2022 at 03:38:23AM -0700, Dan Li wrote: > > > diff --git a/Documentation/kbuild/reproducible-builds.rst b/Documentation/kbuild/reproducible-builds.rst > > > index 071f0151a7a4..13397f38c358 100644 > > > --- a/Documentation/kbuild/reproducible-builds.rst > > > +++ b/Documentation/kbuild/reproducible-builds.rst > > > @@ -119,6 +119,16 @@ To avoid this, you can make the vDSO different for different > > > kernel versions by including an arbitrary string of "salt" in it. > > > This is specified by the Kconfig symbol ``CONFIG_BUILD_SALT``. > > > +Git > > > +----------------------- > > > + > > > +Uncommitted changes or different commit ids in git can also lead > > > +to different compilation results. For example, after executing > > > +``git reset HEAD^``, even if the code is the same, the > > > +``include/config/kernel.release`` generated during compilation > > > +will be different, which will eventually lead to binary differences. > > > +See ``scripts/setlocalversion`` for details. > > > + > > > > Briefly read the script, I don't see what the correlation between git > > reset with LOCALVERSION thing is. Also, does the exact state of git > > repository required for reproducible builds? > > > > Hi Bagas, > > The Makefile has the following code: > filechk_kernel.release = \ > echo "$(KERNELVERSION)$$($(CONFIG_SHELL) $(srctree)/scripts/setlocalversion $(srctree))" > > The output of setlocalversion affects kernel.release, while the output > of setlocalversion is related to the state of git when the git repository > exists (see function scm_version). > > So changes in git state will result in changes to kernel.release, and > this information will be included in the final output vmlinux/modules > and in turn affect reproducible builds. > > For example: > $ git log > commit 4cd155a93eec...... > $ make ... > $ cat include/config/kernel.release > 6.0.0-rc4-00025-g4cd155a93eec > > $ git reset HEAD^ > $ git log > commit 7b4d266b0c41...... > $ make ... > $ cat include/config/kernel.release > 6.0.0-rc4-00024-g7b4d266b0c41-dirty > > > AFAICT, in the presence of a git repository, we can compile a reproducible > build kernel in any git state, but we need to ensure that the git state is > always the same between compilations (or the same from the perspective of > the scm_version function). yes, that definitely true. Absence or presence of git tags can change the output of setlocalversion even more drastically. I think it is sensible to add a stanza about git in Documentation/kbuild/reproducible-builds.rst. Kind regards, Nicolas
On Fri, Oct 21, 2022 at 5:00 PM Nicolas Schier <n.schier@avm.de> wrote: > > On Thu, Oct 20, 2022 at 06:48:20PM -0700, Dan Li wrote: > > > > > > On 10/20/22 07:05, Bagas Sanjaya wrote: > > > On Thu, Oct 20, 2022 at 03:38:23AM -0700, Dan Li wrote: > > > > diff --git a/Documentation/kbuild/reproducible-builds.rst b/Documentation/kbuild/reproducible-builds.rst > > > > index 071f0151a7a4..13397f38c358 100644 > > > > --- a/Documentation/kbuild/reproducible-builds.rst > > > > +++ b/Documentation/kbuild/reproducible-builds.rst > > > > @@ -119,6 +119,16 @@ To avoid this, you can make the vDSO different for different > > > > kernel versions by including an arbitrary string of "salt" in it. > > > > This is specified by the Kconfig symbol ``CONFIG_BUILD_SALT``. > > > > +Git > > > > +----------------------- > > > > + > > > > +Uncommitted changes or different commit ids in git can also lead > > > > +to different compilation results. For example, after executing > > > > +``git reset HEAD^``, even if the code is the same, the > > > > +``include/config/kernel.release`` generated during compilation > > > > +will be different, which will eventually lead to binary differences. > > > > +See ``scripts/setlocalversion`` for details. > > > > + > > > > > > Briefly read the script, I don't see what the correlation between git > > > reset with LOCALVERSION thing is. Also, does the exact state of git > > > repository required for reproducible builds? > > > > > > > Hi Bagas, > > > > The Makefile has the following code: > > filechk_kernel.release = \ > > echo "$(KERNELVERSION)$$($(CONFIG_SHELL) $(srctree)/scripts/setlocalversion $(srctree))" > > > > The output of setlocalversion affects kernel.release, while the output > > of setlocalversion is related to the state of git when the git repository > > exists (see function scm_version). > > > > So changes in git state will result in changes to kernel.release, and > > this information will be included in the final output vmlinux/modules > > and in turn affect reproducible builds. > > > > For example: > > $ git log > > commit 4cd155a93eec...... > > $ make ... > > $ cat include/config/kernel.release > > 6.0.0-rc4-00025-g4cd155a93eec > > > > $ git reset HEAD^ > > $ git log > > commit 7b4d266b0c41...... > > $ make ... > > $ cat include/config/kernel.release > > 6.0.0-rc4-00024-g7b4d266b0c41-dirty > > > > > > AFAICT, in the presence of a git repository, we can compile a reproducible > > build kernel in any git state, but we need to ensure that the git state is > > always the same between compilations (or the same from the perspective of > > the scm_version function). > > yes, that definitely true. Absence or presence of git tags can change > the output of setlocalversion even more drastically. > > I think it is sensible to add a stanza about git in > Documentation/kbuild/reproducible-builds.rst. Make sense. Applied to linux-kbuild. Thanks.
diff --git a/Documentation/kbuild/reproducible-builds.rst b/Documentation/kbuild/reproducible-builds.rst index 071f0151a7a4..13397f38c358 100644 --- a/Documentation/kbuild/reproducible-builds.rst +++ b/Documentation/kbuild/reproducible-builds.rst @@ -119,6 +119,16 @@ To avoid this, you can make the vDSO different for different kernel versions by including an arbitrary string of "salt" in it. This is specified by the Kconfig symbol ``CONFIG_BUILD_SALT``. +Git +----------------------- + +Uncommitted changes or different commit ids in git can also lead +to different compilation results. For example, after executing +``git reset HEAD^``, even if the code is the same, the +``include/config/kernel.release`` generated during compilation +will be different, which will eventually lead to binary differences. +See ``scripts/setlocalversion`` for details. + .. _KBUILD_BUILD_TIMESTAMP: kbuild.html#kbuild-build-timestamp .. _KBUILD_BUILD_USER and KBUILD_BUILD_HOST: kbuild.html#kbuild-build-user-kbuild-build-host .. _KCFLAGS: kbuild.html#kcflags