diff mbox series

kbuild: fix package build error due to broken symlinks

Message ID	20230325141909.2512452-1-masahiroy@kernel.org
State	New
Headers	Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; From: Masahiro Yamada <masahiroy@kernel.org> To: linux-kbuild@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Masahiro Yamada <masahiroy@kernel.org>, Nathan Chancellor <nathan@kernel.org>, Nick Desaulniers <ndesaulniers@google.com>, Nicolas Schier <nicolas@fjasle.eu> Subject: [PATCH] kbuild: fix package build error due to broken symlinks Date: Sat, 25 Mar 2023 23:19:09 +0900 Message-Id: <20230325141909.2512452-1-masahiroy@kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Precedence: bulk
Series	kbuild: fix package build error due to broken symlinks \| kbuild: fix package build error due to broken symlinks

Commit Message

Masahiro Yamada March 25, 2023, 2:19 p.m. UTC

  'make deb-pkg' and 'make rpm-pkg' fail if a broken symlink exists in
a dirty source tree. Handle symlinks properly, and also, keep the
executable permission.

Fixes: 05e96e96a315 ("kbuild: use git-archive for source package creation")
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
---

 scripts/package/gen-diff-patch | 36 +++++++++++++++++++++++++---------
 1 file changed, 27 insertions(+), 9 deletions(-)

Comments

Nicolas Schier March 29, 2023, 8:02 p.m. UTC | #1

On Sat 25 Mar 2023 23:19:09 GMT, Masahiro Yamada wrote:
> 'make deb-pkg' and 'make rpm-pkg' fail if a broken symlink exists in
> a dirty source tree. Handle symlinks properly, and also, keep the
> executable permission.
> 
> Fixes: 05e96e96a315 ("kbuild: use git-archive for source package creation")
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> ---
> 
>  scripts/package/gen-diff-patch | 36 +++++++++++++++++++++++++---------
>  1 file changed, 27 insertions(+), 9 deletions(-)
> 
> diff --git a/scripts/package/gen-diff-patch b/scripts/package/gen-diff-patch
> index f842ab50a780..23551de92e1b 100755
> --- a/scripts/package/gen-diff-patch
> +++ b/scripts/package/gen-diff-patch
> @@ -23,16 +23,34 @@ fi
>  git -C ${srctree} status --porcelain --untracked-files=all |
>  while read stat path
>  do
> -	if [ "${stat}" = '??' ]; then
> -
> -		if ! diff -u /dev/null "${srctree}/${path}" > .tmp_diff &&
> -			! head -n1 .tmp_diff | grep -q "Binary files"; then
> -			{
> -				echo "--- /dev/null"
> -				echo "+++ linux/$path"
> -				cat .tmp_diff | tail -n +3
> -			} >> ${untracked_patch}
> +	if [ "${stat}" != '??' ]; then
> +		continue
> +	fi
> +
> +	if [ -L "${path}" ]; then
> +		{
> +			echo "diff --git a/${path} b/${path}"
> +			echo "new file mode 120000"
> +			echo "--- /dev/null"
> +			echo "+++ b/$path"
> +			echo "@@ -0,0 +1 @@"
> +			printf "+"; readlink ${path}

Better quote "${path}"?

> +			echo '\ No newline at end of file'
> +		} >> ${untracked_patch}

Here quoting should not be necessary as mkdebian and mkspec give only 
save filenames, but for consistency I'd quote ${untracked_patch} as 
well.

> +	elif ! diff -u /dev/null "${srctree}/${path}" > .tmp_diff &&
> +	     ! head -n1 .tmp_diff | grep -q "Binary files"; then
> +		if [ -x ${path} ]; then

Same here.

> +			mode=100755
> +		else
> +			mode=100644
>  		fi
> +		{
> +			echo "diff --git a/${path} b/${path}"
> +			echo "new file mode ${mode}"
> +			echo "--- /dev/null"
> +			echo "+++ b/$path"
> +			cat .tmp_diff | tail -n +3
> +		} >> ${untracked_patch}

And possibly here?

>  	fi
>  done
>  
> -- 
> 2.34.1

Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>

Masahiro Yamada April 1, 2023, 2:55 p.m. UTC | #2

On Thu, Mar 30, 2023 at 5:02 AM Nicolas Schier <nicolas@fjasle.eu> wrote:
>
> On Sat 25 Mar 2023 23:19:09 GMT, Masahiro Yamada wrote:
> > 'make deb-pkg' and 'make rpm-pkg' fail if a broken symlink exists in
> > a dirty source tree. Handle symlinks properly, and also, keep the
> > executable permission.
> >
> > Fixes: 05e96e96a315 ("kbuild: use git-archive for source package creation")
> > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> > ---
> >
> >  scripts/package/gen-diff-patch | 36 +++++++++++++++++++++++++---------
> >  1 file changed, 27 insertions(+), 9 deletions(-)
> >
> > diff --git a/scripts/package/gen-diff-patch b/scripts/package/gen-diff-patch
> > index f842ab50a780..23551de92e1b 100755
> > --- a/scripts/package/gen-diff-patch
> > +++ b/scripts/package/gen-diff-patch
> > @@ -23,16 +23,34 @@ fi
> >  git -C ${srctree} status --porcelain --untracked-files=all |
> >  while read stat path
> >  do
> > -     if [ "${stat}" = '??' ]; then
> > -
> > -             if ! diff -u /dev/null "${srctree}/${path}" > .tmp_diff &&
> > -                     ! head -n1 .tmp_diff | grep -q "Binary files"; then
> > -                     {
> > -                             echo "--- /dev/null"
> > -                             echo "+++ linux/$path"
> > -                             cat .tmp_diff | tail -n +3
> > -                     } >> ${untracked_patch}
> > +     if [ "${stat}" != '??' ]; then
> > +             continue
> > +     fi
> > +
> > +     if [ -L "${path}" ]; then
> > +             {
> > +                     echo "diff --git a/${path} b/${path}"
> > +                     echo "new file mode 120000"
> > +                     echo "--- /dev/null"
> > +                     echo "+++ b/$path"
> > +                     echo "@@ -0,0 +1 @@"
> > +                     printf "+"; readlink ${path}
>
> Better quote "${path}"?


Thanks for the suggestion.

Quoting variables are correct in most cases.
But, that is not enough to generate a valid
patch when a file path contains spaces.



'git format-patch' produces a patch that
is accepted by GNU patch and also by dpkg-source.

I learned a trick from GIT source code.


If you are interested, what GIT does [1].


[1] https://github.com/git/git/commit/1a9eb3b9d50367bee8fe85022684d812816fe531


I will send v2 later, where I made some more efforts
to fix several corner cases even if that is not perfect.

Nicolas Schier April 1, 2023, 5:53 p.m. UTC | #3

On Sat, Apr 01, 2023 at 11:55:44PM +0900 Masahiro Yamada wrote:
> On Thu, Mar 30, 2023 at 5:02 AM Nicolas Schier <nicolas@fjasle.eu> wrote:
> >
> > On Sat 25 Mar 2023 23:19:09 GMT, Masahiro Yamada wrote:
> > > 'make deb-pkg' and 'make rpm-pkg' fail if a broken symlink exists in
> > > a dirty source tree. Handle symlinks properly, and also, keep the
> > > executable permission.
> > >
> > > Fixes: 05e96e96a315 ("kbuild: use git-archive for source package creation")
> > > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> > > ---
> > >
> > >  scripts/package/gen-diff-patch | 36 +++++++++++++++++++++++++---------
> > >  1 file changed, 27 insertions(+), 9 deletions(-)
> > >
> > > diff --git a/scripts/package/gen-diff-patch b/scripts/package/gen-diff-patch
> > > index f842ab50a780..23551de92e1b 100755
> > > --- a/scripts/package/gen-diff-patch
> > > +++ b/scripts/package/gen-diff-patch
> > > @@ -23,16 +23,34 @@ fi
> > >  git -C ${srctree} status --porcelain --untracked-files=all |
> > >  while read stat path
> > >  do
> > > -     if [ "${stat}" = '??' ]; then
> > > -
> > > -             if ! diff -u /dev/null "${srctree}/${path}" > .tmp_diff &&
> > > -                     ! head -n1 .tmp_diff | grep -q "Binary files"; then
> > > -                     {
> > > -                             echo "--- /dev/null"
> > > -                             echo "+++ linux/$path"
> > > -                             cat .tmp_diff | tail -n +3
> > > -                     } >> ${untracked_patch}
> > > +     if [ "${stat}" != '??' ]; then
> > > +             continue
> > > +     fi
> > > +
> > > +     if [ -L "${path}" ]; then
> > > +             {
> > > +                     echo "diff --git a/${path} b/${path}"
> > > +                     echo "new file mode 120000"
> > > +                     echo "--- /dev/null"
> > > +                     echo "+++ b/$path"
> > > +                     echo "@@ -0,0 +1 @@"
> > > +                     printf "+"; readlink ${path}
> >
> > Better quote "${path}"?
> 
> 
> Thanks for the suggestion.
> 
> Quoting variables are correct in most cases.
> But, that is not enough to generate a valid
> patch when a file path contains spaces.
> 
> 
> 
> 'git format-patch' produces a patch that
> is accepted by GNU patch and also by dpkg-source.
> 
> I learned a trick from GIT source code.
> 
> 
> If you are interested, what GIT does [1].
> 
> 
> [1] https://github.com/git/git/commit/1a9eb3b9d50367bee8fe85022684d812816fe531

thanks for the pointer, that is really interesting!

Kind regards,
Nicolas


> I will send v2 later, where I made some more efforts
> to fix several corner cases even if that is not perfect.
> 
> 
> 
> 
> 
> 
> 
> 
> -- 
> Best Regards
> Masahiro Yamada

Masahiro Yamada April 2, 2023, 2:13 a.m. UTC | #4

On Sun, Apr 2, 2023 at 2:53 AM Nicolas Schier <nicolas@fjasle.eu> wrote:
>
> On Sat, Apr 01, 2023 at 11:55:44PM +0900 Masahiro Yamada wrote:
> > On Thu, Mar 30, 2023 at 5:02 AM Nicolas Schier <nicolas@fjasle.eu> wrote:
> > >
> > > On Sat 25 Mar 2023 23:19:09 GMT, Masahiro Yamada wrote:
> > > > 'make deb-pkg' and 'make rpm-pkg' fail if a broken symlink exists in
> > > > a dirty source tree. Handle symlinks properly, and also, keep the
> > > > executable permission.
> > > >
> > > > Fixes: 05e96e96a315 ("kbuild: use git-archive for source package creation")
> > > > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> > > > ---
> > > >
> > > >  scripts/package/gen-diff-patch | 36 +++++++++++++++++++++++++---------
> > > >  1 file changed, 27 insertions(+), 9 deletions(-)
> > > >
> > > > diff --git a/scripts/package/gen-diff-patch b/scripts/package/gen-diff-patch
> > > > index f842ab50a780..23551de92e1b 100755
> > > > --- a/scripts/package/gen-diff-patch
> > > > +++ b/scripts/package/gen-diff-patch
> > > > @@ -23,16 +23,34 @@ fi
> > > >  git -C ${srctree} status --porcelain --untracked-files=all |
> > > >  while read stat path
> > > >  do
> > > > -     if [ "${stat}" = '??' ]; then
> > > > -
> > > > -             if ! diff -u /dev/null "${srctree}/${path}" > .tmp_diff &&
> > > > -                     ! head -n1 .tmp_diff | grep -q "Binary files"; then
> > > > -                     {
> > > > -                             echo "--- /dev/null"
> > > > -                             echo "+++ linux/$path"
> > > > -                             cat .tmp_diff | tail -n +3
> > > > -                     } >> ${untracked_patch}
> > > > +     if [ "${stat}" != '??' ]; then
> > > > +             continue
> > > > +     fi
> > > > +
> > > > +     if [ -L "${path}" ]; then
> > > > +             {
> > > > +                     echo "diff --git a/${path} b/${path}"
> > > > +                     echo "new file mode 120000"
> > > > +                     echo "--- /dev/null"
> > > > +                     echo "+++ b/$path"
> > > > +                     echo "@@ -0,0 +1 @@"
> > > > +                     printf "+"; readlink ${path}
> > >
> > > Better quote "${path}"?
> >
> >
> > Thanks for the suggestion.
> >
> > Quoting variables are correct in most cases.
> > But, that is not enough to generate a valid
> > patch when a file path contains spaces.
> >
> >
> >
> > 'git format-patch' produces a patch that
> > is accepted by GNU patch and also by dpkg-source.
> >
> > I learned a trick from GIT source code.
> >
> >
> > If you are interested, what GIT does [1].
> >
> >
> > [1] https://github.com/git/git/commit/1a9eb3b9d50367bee8fe85022684d812816fe531
>
> thanks for the pointer, that is really interesting!
>
> Kind regards,
> Nicolas
>
>
> > I will send v2 later, where I made some more efforts
> > to fix several corner cases even if that is not perfect.


I wrote a patch (attached to this email) but I am
still thinking if this is worth the complicated scripting...

Another approach would be to stop caring untracked files.

diff mbox series

Patch

diff --git a/scripts/package/gen-diff-patch b/scripts/package/gen-diff-patch
index f842ab50a780..23551de92e1b 100755
--- a/scripts/package/gen-diff-patch
+++ b/scripts/package/gen-diff-patch
@@ -23,16 +23,34 @@  fi
 git -C ${srctree} status --porcelain --untracked-files=all |
 while read stat path
 do
-	if [ "${stat}" = '??' ]; then
-
-		if ! diff -u /dev/null "${srctree}/${path}" > .tmp_diff &&
-			! head -n1 .tmp_diff | grep -q "Binary files"; then
-			{
-				echo "--- /dev/null"
-				echo "+++ linux/$path"
-				cat .tmp_diff | tail -n +3
-			} >> ${untracked_patch}
+	if [ "${stat}" != '??' ]; then
+		continue
+	fi
+
+	if [ -L "${path}" ]; then
+		{
+			echo "diff --git a/${path} b/${path}"
+			echo "new file mode 120000"
+			echo "--- /dev/null"
+			echo "+++ b/$path"
+			echo "@@ -0,0 +1 @@"
+			printf "+"; readlink ${path}
+			echo '\ No newline at end of file'
+		} >> ${untracked_patch}
+	elif ! diff -u /dev/null "${srctree}/${path}" > .tmp_diff &&
+	     ! head -n1 .tmp_diff | grep -q "Binary files"; then
+		if [ -x ${path} ]; then
+			mode=100755
+		else
+			mode=100644
 		fi
+		{
+			echo "diff --git a/${path} b/${path}"
+			echo "new file mode ${mode}"
+			echo "--- /dev/null"
+			echo "+++ b/$path"
+			cat .tmp_diff | tail -n +3
+		} >> ${untracked_patch}
 	fi
 done