diff mbox series

[2/4] maintainer-scripts/gcc_release: create index between snapshots <-> commits

Message ID 20231102084058.1142941-2-sam@gentoo.org
State Accepted
Headers
Series [1/4] contrib: add generate_snapshot_index.py |

Checks

Context Check Description
snail/gcc-patch-check success Github commit url

Commit Message

Sam James Nov. 2, 2023, 8:39 a.m. UTC 
  Create and maintain a known_snapshots.txt index with space-separated format
BRANCH-DATE COMMIT.

For example:
8-20210107 5114ee0676e432493ada968e34071f02fb08114f
8-20210114 f9267925c648f2ccd9e4680b699e581003125bcf
...

This is helpful for bisects and quickly looking up the information from bug
reports.

maintainer-scripts/
	* gcc_release: Create known_snapshots.txt as an index between snapshots
	and commits.

Signed-off-by: Sam James <sam@gentoo.org>
---
Note that there's a few different approaches we can take here. I've gone
for the simpler one of having it still fetch from the remote site and parse
because it's obviously hard for me to test a part which runs on the remote
machine.

We can skip this patch for now if desired. I have mixed feelings about complicating
the contrib/generate_snapshot_index.py script to take an URL / path as I'd ideally
like it to still be easily usable locally.

We could have it be generated locally and then uploaded as well, as another option.

 maintainer-scripts/gcc_release | 7 +++++++
 1 file changed, 7 insertions(+)

Comments

Jonathan Wakely Nov. 2, 2023, 9:07 a.m. UTC | #1 
On 02/11/23 08:39 +0000, Sam James wrote:
>Create and maintain a known_snapshots.txt index with space-separated format
>BRANCH-DATE COMMIT.
>
>For example:
>8-20210107 5114ee0676e432493ada968e34071f02fb08114f
>8-20210114 f9267925c648f2ccd9e4680b699e581003125bcf
>...
>
>This is helpful for bisects and quickly looking up the information from bug
>reports.


Is there any reason we don't just use git tags for this?

We could run a one-off job to create all the historical tags (setting
GIT_COMMITTER_DATE and GIT_AUTHOR_DATE so the tags are backdated), and
add a tagging step to the snapshot creation.

Git tags are cheap, but I can imagine a concern about hundreds of new
tags "littering" the output of 'git tag -l'. I don't _think_ you can
put tags under an alternative ref that isn't fetched by default (as we
do with refs/users and refs/vendor). I think tags have to go under
refs/tags. But grep -v could be used to filter out snapshot tags
easily.

We could use https://git-scm.com/docs/gitnamespaces for this though,
so that git --namespace=snapshots tag -l would show the snapshot tags.
Andreas Schwab Nov. 2, 2023, 10:18 a.m. UTC | #2 
On Nov 02 2023, Jonathan Wakely wrote:

> Git tags are cheap, but I can imagine a concern about hundreds of new
> tags "littering" the output of 'git tag -l'. I don't _think_ you can
> put tags under an alternative ref that isn't fetched by default (as we
> do with refs/users and refs/vendor). I think tags have to go under
> refs/tags. But grep -v could be used to filter out snapshot tags
> easily.

There is no inherent limitation on publishing tags outside of refs/tags,
to make them invisible by git tag.  There are already existing examples
of tags residing under various refs/users and refs/vendors namespaces.
Jonathan Wakely Nov. 2, 2023, 10:25 a.m. UTC | #3 
On Thu, 2 Nov 2023 at 10:23, Andreas Schwab wrote:
>
> On Nov 02 2023, Jonathan Wakely wrote:
>
> > Git tags are cheap, but I can imagine a concern about hundreds of new
> > tags "littering" the output of 'git tag -l'. I don't _think_ you can
> > put tags under an alternative ref that isn't fetched by default (as we
> > do with refs/users and refs/vendor). I think tags have to go under
> > refs/tags. But grep -v could be used to filter out snapshot tags
> > easily.
>
> There is no inherent limitation on publishing tags outside of refs/tags,
> to make them invisible by git tag.  There are already existing examples
> of tags residing under various refs/users and refs/vendors namespaces.


Ah, good to know, thanks.

So then there's no reason that snapshots would have to clutter up the
list of default tags for anybody who isn't interested in them.
Bernhard Reutner-Fischer Nov. 2, 2023, 10:18 p.m. UTC | #4 
On 2 November 2023 11:25:47 CET, Jonathan Wakely <jwakely@redhat.com> wrote:
>On Thu, 2 Nov 2023 at 10:23, Andreas Schwab wrote:
>>
>> On Nov 02 2023, Jonathan Wakely wrote:
>>
>> > Git tags are cheap, but I can imagine a concern about hundreds of new
>> > tags "littering" the output of 'git tag -l'. I don't _think_ you can
>> > put tags under an alternative ref that isn't fetched by default (as we
>> > do with refs/users and refs/vendor). I think tags have to go under
>> > refs/tags. But grep -v could be used to filter out snapshot tags
>> > easily.
>>
>> There is no inherent limitation on publishing tags outside of refs/tags,
>> to make them invisible by git tag.  There are already existing examples
>> of tags residing under various refs/users and refs/vendors namespaces.
>
>
>Ah, good to know, thanks.
>
>So then there's no reason that snapshots would have to clutter up the
>list of default tags for anybody who isn't interested in them.
>

Thanks Andreas. Exactly. So, just to emphasise the obvious:
Let's please use refs/snapshot ?
diff mbox series

Patch

diff --git a/maintainer-scripts/gcc_release b/maintainer-scripts/gcc_release
index 962b8efe99a7..4cd1fa799660 100755
--- a/maintainer-scripts/gcc_release
+++ b/maintainer-scripts/gcc_release
@@ -448,6 +448,9 @@  announce_snapshot() {
   SNAPSHOT_INDEX=${RELEASE}/index.html
 
   changedir "${SNAPSHOTS_DIR}"
+  # Create an index if it doesn't already exist and populate it before we add
+  # the new snapshot.
+  ${PYTHON} ${SOURCE_DIRECTORY}/contrib/generate_snapshot_index.py || error "Failed to generate snapshot index"
   echo \
 "Snapshot gcc-"${RELEASE}" is now available on
   https://gcc.gnu.org/pub/gcc/snapshots/"${RELEASE}"/
@@ -514,6 +517,9 @@  Last modified "${TEXT_DATE}"
   rm -f LATEST-${BRANCH}
   ln -s ${RELEASE} LATEST-${BRANCH}
 
+  # Add the snapshot we just made to the index
+  printf "${RELEASE} ${GITREV}\n" >> known_snapshots.txt
+
   inform "Sending mail"
 
   export QMAILHOST=gcc.gnu.org
@@ -617,6 +623,7 @@  GZIP="${GZIP:-gzip --best}"
 SCP="${SCP:-scp -p}"
 SSH="${SSH:-ssh}"
 TAR="${TAR:-tar}"
+PYTHON="${PYTHON:-python3}"
 
 ########################################################################
 # Command Line Processing