Message ID | 20240228225527.1052240-1-helen.koike@collabora.com |
---|---|
Headers |
Return-Path: <linux-kernel+bounces-85809-ouuuleilei=gmail.com@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:6358:a1a:b0:17b:cd04:e0c6 with SMTP id 26csp245278rwa; Wed, 28 Feb 2024 15:03:41 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWMbXZ2uwn+RrRAcOU1GGScBAJM8RtFKoQ0e31ZuiKl+RH8uVa77YfSl+yrHgI3NBZUHZwQbilJp9CwvRDOHcs767wa0A== X-Google-Smtp-Source: AGHT+IHI/JZXzhgGLltyesTDC33i+hxx33PS5ZGM61byxMYAEe9KGE4lwWN4yvQHhApsXMzu1iwj X-Received: by 2002:a17:90a:d80c:b0:29a:2d0f:6418 with SMTP id a12-20020a17090ad80c00b0029a2d0f6418mr691185pjv.9.1709161421641; Wed, 28 Feb 2024 15:03:41 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1709161421; cv=pass; d=google.com; s=arc-20160816; b=hysCEbelSW6go/zA+53RsDDBWdZ7rvBVPhB3Sj5MD4ykBP/czHaxd0i+mbBOQWosBl haICpttXVPWMW1w0M4p24ahMKEBGYMsMSq2sdhFxYQAKQOfK2jYCC3WN+L7AXwvCW6Hg A+zIKYGZkyxTyWokG3Xgvv8AKhcSsUzzBj1M1yypY9AKG+zM2vrNuXqSw+sW9dIU+lDd T10AXIKno8ugb1kt+wmViQ9lIpspac+zRBxzlIKeZ1KYScTC5uTQWPZlG3Y6L7LTKZOT Rc629giE2eaNeLtQ8ePoosiGro1u7xC/FOEVvdV8OTQt5+/zStGkw7y1fGX/oxnPAss8 AMBg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:message-id:date:subject:to:from :dkim-signature; bh=inJzqDKMBsv/nSe1Zoytv0aLXmofJKsY6FQN8eXAukE=; fh=0m8BcSIN6jnEGrXYezAni7jiBLFEw9oBT48Gar0OTdk=; b=NrmPXgGgWuWsu/ir09us2RU0jkLl8oNAnzLBIHH5WWKVDLeEXUi8zuiTFw66pGbAKI 0GvstnWnDMAHrU4QJoxVkYHcksmbvt6rqhli0TUDHVxIgTTNTN1DHqPdiVkOL8ZlpSD7 dpnlTKR5kcgukQwxSMlxzdH8vH3PCiFpUKZrrnj9u9ty6Nat50egkffHOdnvgkWPi3pU mjzai9KctrSdqGi7sFA+mBLJsheiBcftEcxImARc6esW4GkWQzTm4bJH0VPAZUjkTppj LYhdlqpWGZK+HnN1qYYOr0vYPmo8ONjd1LSqIW7Y8dCixMWuvgNWsaUvjYuc8suYfiNG 913Q==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@collabora.com header.s=mail header.b=A87kTiHE; arc=pass (i=1 spf=pass spfdomain=collabora.com dkim=pass dkdomain=collabora.com dmarc=pass fromdomain=collabora.com); spf=pass (google.com: domain of linux-kernel+bounces-85809-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-85809-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=collabora.com Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [147.75.48.161]) by mx.google.com with ESMTPS id t9-20020a17090abc4900b002995fa56919si2147370pjv.15.2024.02.28.15.03.41 for <ouuuleilei@gmail.com> (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Feb 2024 15:03:41 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-85809-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161; Authentication-Results: mx.google.com; dkim=pass header.i=@collabora.com header.s=mail header.b=A87kTiHE; arc=pass (i=1 spf=pass spfdomain=collabora.com dkim=pass dkdomain=collabora.com dmarc=pass fromdomain=collabora.com); spf=pass (google.com: domain of linux-kernel+bounces-85809-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-85809-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=collabora.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id ECD8EB2626E for <ouuuleilei@gmail.com>; Wed, 28 Feb 2024 22:55:59 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id AE7CA7292C; Wed, 28 Feb 2024 22:55:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=collabora.com header.i=@collabora.com header.b="A87kTiHE" Received: from madrid.collaboradmins.com (madrid.collaboradmins.com [46.235.227.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0C2FD7004E; Wed, 28 Feb 2024 22:55:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=46.235.227.194 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709160941; cv=none; b=QIKdH5ez7Po7ol8rtWKdTmTFQH5Hqxg+fwADHvYEpxXUw20zvlJsovEItt4qrNXzwJmFgMt1r2Db8Q/pEBAFls7biLwdgh8Km5MoTEQmQmisvgijSWuqoeWfnr2AHoUYj9+CTc2Dtcb7H8uUf8nk15T6pmVNW8MdxiuxXv3LFfg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709160941; c=relaxed/simple; bh=6VtgkIxV7BP05QpH+CSqlVMLA5ngVi7oylPuoBSEGcU=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=mx1RUrmHi1rxB2MInkHQnjH4sPA19mak9D/xwVm6S+RwoeUf5R+xlW+KGIUyfjn+NS7kzhx5sECqts2ks6cMCIchEp5smJFJWv9KEUzMQHWETtmbIr3lIl84WX9Pq3FZH/U2A/u51FLOQ/8eJcw8icBeK/SrGW+6ykcrWNwTnf8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=collabora.com; spf=pass smtp.mailfrom=collabora.com; dkim=pass (2048-bit key) header.d=collabora.com header.i=@collabora.com header.b=A87kTiHE; arc=none smtp.client-ip=46.235.227.194 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=collabora.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=collabora.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1709160938; bh=6VtgkIxV7BP05QpH+CSqlVMLA5ngVi7oylPuoBSEGcU=; h=From:To:Subject:Date:From; b=A87kTiHEClW+SUso7bjkel8UYsTiCXSBcH8Kj4bY4dGebjnrGHM46oTUceu7kYsMJ dUKnjoqQzR4S6wUIhAypXIkDwynXGnR/khIjTFPzyNu8cWr+UG1/8Fj0LqBkW16gB1 92f9csTs5SBxNH2Dh3XKEmVlkYJaz6aSnNmb+YqHfvoKZM9r9AyKKsaZtvrYtfjCEq PCXF4hneWm6yfjFSHE+gilccufYz6HYkCyIWnpF5dK3dvqL+37/YkHDPrAHZxH3+CK e7kN3st5G5Nfuio6dxRodRU/VBZeu58ERPkEogNoptEMDEay3nYrgUcTC57iFdbfuX rYhYCzfDWw77w== Received: from localhost.localdomain (cola.collaboradmins.com [195.201.22.229]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: koike) by madrid.collaboradmins.com (Postfix) with ESMTPSA id 93F7437820CB; Wed, 28 Feb 2024 22:55:31 +0000 (UTC) From: Helen Koike <helen.koike@collabora.com> To: linuxtv-ci@linuxtv.org, dave.pigott@collabora.com, mripard@kernel.org, linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-kselftest@vger.kernel.org, gustavo.padovan@collabora.com, pawiecz@collabora.com, spbnick@gmail.com, tales.aparecida@gmail.com, workflows@vger.kernel.org, kernelci@lists.linux.dev, skhan@linuxfoundation.org, kunit-dev@googlegroups.com, nfraprado@collabora.com, davidgow@google.com, cocci@inria.fr, Julia.Lawall@inria.fr, laura.nao@collabora.com, ricardo.canuelo@collabora.com, kernel@collabora.com, torvalds@linuxfoundation.org, gregkh@linuxfoundation.org Subject: [PATCH 0/3] kci-gitlab: Introducing GitLab-CI Pipeline for Kernel Testing Date: Wed, 28 Feb 2024 19:55:24 -0300 Message-Id: <20240228225527.1052240-1-helen.koike@collabora.com> X-Mailer: git-send-email 2.40.1 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: <linux-kernel.vger.kernel.org> List-Subscribe: <mailto:linux-kernel+subscribe@vger.kernel.org> List-Unsubscribe: <mailto:linux-kernel+unsubscribe@vger.kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1792185646530891936 X-GMAIL-MSGID: 1792185646530891936 |
Series |
kci-gitlab: Introducing GitLab-CI Pipeline for Kernel Testing
|
|
Message
Helen Koike
Feb. 28, 2024, 10:55 p.m. UTC
Dear Kernel Community, This patch introduces a `.gitlab-ci` file along with a `ci/` folder, defining a basic test pipeline triggered by code pushes to a GitLab-CI instance. This initial version includes static checks (checkpatch and smatch for now) and build tests across various architectures and configurations. It leverages an integrated cache for efficient build times and introduces a flexible 'scenarios' mechanism for subsystem-specific extensions. tl;dr: check this video to see a quick demo: https://youtu.be/TWiTjhjOuzg, but don't forget to check the "Motivation for this work" below. Your feedback, whether a simple thumbs up or down, is crucial to determine if it is worthwhile to pursue this initiative. GitLab is an Open Source platform that includes integrated CI/CD. The pipeline provided in this patch is designed to work out-of-the-box with any GitLab instance, including the gitlab.com Free Tier. If you reach the limits of the Free Tier, consider using community instances like https://gitlab.freedesktop.org/. Alternatively, you can set up a local runner for more flexibility. The bootstrap-gitlab-runner.sh script included with this patch simplifies this process, enabling you to run tests on your preferred infrastructure, including your own machine. For detailed information, please refer to the documentation included in the patch, or check the rendered version here: https://koike.pages.collabora.com/-/linux/-/jobs/298498/artifacts/artifacts/Documentation-output/ci/gitlab-ci/gitlab-ci.html . Motivation for this Work ======================== We all know tests are a major topic in the community, so let's mention the specificities of this approach: 1. **Built-in User Interface:** GitLab CI/CD is growing in popularity and has an user-friendly interface. Our experience with the upstream DRM-CI in the kernel tree (see this blog post [https://www.collabora.com/news-and-blog/blog/2024/02/08/drm-ci-a-gitlab-ci-pipeline-for-linux-kernel-testing/] ) has provided insights into how such a system can benefit the wider community. 2. **Distributed Infrastructure:** The proposed GitLab-CI pipeline is designed with a distributed infrastructure model, being possible to run in any gitlab instance. 3. **Reduce regressions:** Fostering a culture where people habitually run validated tests and post their results can prevent many issues in post-merge tests. 4. **Collaborative Testing Environment:** The kernel community is already engaged in numerous testing efforts, including various GitLab-CI pipelines such as DRM-CI, which I maintain, along with other solutions like KernelCI and BPF-CI. This proposal is designed to further stimulate contributions to the evolving testing landscape. Our goal is to establish a comprehensive suite of common tools and files. 5. **Ownership of QA:** Discrepancies between kernel code and outdated tests often lead to misattributed failures, complicating regression tracking. This issue, often arising from neglected or deprioritized test updates, creates uncertainty about the source of failures. Adopting an "always green pipeline" approach, as detailed in this patch's documentation, encourages timely maintenance and validation of tests. This ensures that testing accurately reflects the current state of the kernel, thereby improving the effectiveness of our QA processes. Additionally, if we discover that this method isn't working for us, we can easily remove it from the codebase, as it is primarily contained within the ci/ folder. Future Work =========== **Expanding Static Checks:** We have the opportunity to integrate a variety of static analysis tools, including: - dtbs_checks - sparse - yamllint - dt-doc-validate - coccicheck **Adding Userspace Tests on VMs:** To further our testing, we can implement userspace tests that run on virtual machines (VMs), such as: - kselftests - kunit tests - Subsystem-specific tests, customizable in the scenarios. **Leveraging External Test Labs:** We can extend our testing to external labs, similar to what DRM-CI currently does. This includes: - Lava labs - Bare metal labs - Using KernelCI-provided labs **Other integrations** - Submit results to KCIDB **Lightweight Implementation for All Developers:** We aim to design these tests to be lightweight, ensuring developers with limited computing resources can still run essential tests. Resource-intensive tests can be set to trigger manually, rather than automatically, to accommodate diverse development environments. Chat Discussions ================ For those interested in further discussions: **Join Our Slack Channel:** We have a Slack channel, #gitlab-ci, on the KernelCI Slack instance https://kernelci.slack.com/ . Feel free to join and contribute to the conversation. The KernelCI team has weekly calls where we also discuss the GitLab-CI pipeline. **Acknowledgments:** A special thanks to Nikolai Kondrashov, Tales da Aparecida - both from Red Hat - and KernelCI community for their valuable feedback and support in this proposal. I eagerly await your thoughts and suggestions on this initiative. Also, if you want to see this initiave move faster, we are happy to discuss funding options. Best regards, Helen Koike Helen Koike (3): kci-gitlab: Introducing GitLab-CI Pipeline for Kernel Testing kci-gitlab: Add documentation kci-gitlab: docs: Add images .gitlab-ci.yml | 2 + Documentation/ci/gitlab-ci/gitlab-ci.rst | 404 ++++++++++++++++++ .../ci/gitlab-ci/images/job-matrix.png | Bin 0 -> 159752 bytes .../gitlab-ci/images/new-project-runner.png | Bin 0 -> 607737 bytes .../ci/gitlab-ci/images/pipelines-on-push.png | Bin 0 -> 532143 bytes .../ci/gitlab-ci/images/the-pipeline.png | Bin 0 -> 91675 bytes .../ci/gitlab-ci/images/variables.png | Bin 0 -> 277518 bytes Documentation/index.rst | 7 + MAINTAINERS | 9 + ci/gitlab-ci/bootstrap-gitlab-runner.sh | 55 +++ ci/gitlab-ci/ci-scripts/build-docs.sh | 35 ++ ci/gitlab-ci/ci-scripts/build-kernel.sh | 35 ++ ci/gitlab-ci/ci-scripts/ici-functions.sh | 104 +++++ ci/gitlab-ci/ci-scripts/install-smatch.sh | 13 + .../ci-scripts/parse_commit_message.sh | 27 ++ ci/gitlab-ci/ci-scripts/run-checkpatch.sh | 19 + ci/gitlab-ci/ci-scripts/run-smatch.sh | 45 ++ ci/gitlab-ci/docker-compose.yaml | 18 + ci/gitlab-ci/linux.code-workspace | 11 + ci/gitlab-ci/yml/build.yml | 43 ++ ci/gitlab-ci/yml/cache.yml | 26 ++ ci/gitlab-ci/yml/container.yml | 36 ++ ci/gitlab-ci/yml/gitlab-ci.yml | 71 +++ ci/gitlab-ci/yml/kernel-combinations.yml | 18 + ci/gitlab-ci/yml/scenarios.yml | 12 + ci/gitlab-ci/yml/scenarios/file-systems.yml | 21 + ci/gitlab-ci/yml/scenarios/media.yml | 21 + ci/gitlab-ci/yml/scenarios/network.yml | 21 + ci/gitlab-ci/yml/static-checks.yml | 21 + 29 files changed, 1074 insertions(+) create mode 100644 .gitlab-ci.yml create mode 100644 Documentation/ci/gitlab-ci/gitlab-ci.rst create mode 100644 Documentation/ci/gitlab-ci/images/job-matrix.png create mode 100644 Documentation/ci/gitlab-ci/images/new-project-runner.png create mode 100644 Documentation/ci/gitlab-ci/images/pipelines-on-push.png create mode 100644 Documentation/ci/gitlab-ci/images/the-pipeline.png create mode 100644 Documentation/ci/gitlab-ci/images/variables.png create mode 100755 ci/gitlab-ci/bootstrap-gitlab-runner.sh create mode 100755 ci/gitlab-ci/ci-scripts/build-docs.sh create mode 100755 ci/gitlab-ci/ci-scripts/build-kernel.sh create mode 100644 ci/gitlab-ci/ci-scripts/ici-functions.sh create mode 100755 ci/gitlab-ci/ci-scripts/install-smatch.sh create mode 100755 ci/gitlab-ci/ci-scripts/parse_commit_message.sh create mode 100755 ci/gitlab-ci/ci-scripts/run-checkpatch.sh create mode 100755 ci/gitlab-ci/ci-scripts/run-smatch.sh create mode 100644 ci/gitlab-ci/docker-compose.yaml create mode 100644 ci/gitlab-ci/linux.code-workspace create mode 100644 ci/gitlab-ci/yml/build.yml create mode 100644 ci/gitlab-ci/yml/cache.yml create mode 100644 ci/gitlab-ci/yml/container.yml create mode 100644 ci/gitlab-ci/yml/gitlab-ci.yml create mode 100644 ci/gitlab-ci/yml/kernel-combinations.yml create mode 100644 ci/gitlab-ci/yml/scenarios.yml create mode 100644 ci/gitlab-ci/yml/scenarios/file-systems.yml create mode 100644 ci/gitlab-ci/yml/scenarios/media.yml create mode 100644 ci/gitlab-ci/yml/scenarios/network.yml create mode 100644 ci/gitlab-ci/yml/static-checks.yml
Comments
Hi Helen, I appreciate the amount of work you've put into this, to improve testing of the kernel as a whole. I'll need more time to answer, but please see below for a quick comment already. On Wed, Feb 28, 2024 at 07:55:24PM -0300, Helen Koike wrote: > Dear Kernel Community, > > This patch introduces a `.gitlab-ci` file along with a `ci/` folder, defining a > basic test pipeline triggered by code pushes to a GitLab-CI instance. This > initial version includes static checks (checkpatch and smatch for now) and build > tests across various architectures and configurations. It leverages an > integrated cache for efficient build times and introduces a flexible 'scenarios' > mechanism for subsystem-specific extensions. > > tl;dr: check this video to see a quick demo: https://youtu.be/TWiTjhjOuzg, > but don't forget to check the "Motivation for this work" below. Your feedback, > whether a simple thumbs up or down, is crucial to determine if it is worthwhile > to pursue this initiative. > > GitLab is an Open Source platform that includes integrated CI/CD. The pipeline > provided in this patch is designed to work out-of-the-box with any GitLab > instance, including the gitlab.com Free Tier. If you reach the limits of the > Free Tier, consider using community instances like https://gitlab.freedesktop.org/. > Alternatively, you can set up a local runner for more flexibility. The > bootstrap-gitlab-runner.sh script included with this patch simplifies this > process, enabling you to run tests on your preferred infrastructure, including > your own machine. > > For detailed information, please refer to the documentation included in the > patch, or check the rendered version here: https://koike.pages.collabora.com/-/linux/-/jobs/298498/artifacts/artifacts/Documentation-output/ci/gitlab-ci/gitlab-ci.html . > > > Motivation for this Work > ======================== > > We all know tests are a major topic in the community, so let's mention the > specificities of this approach: > > 1. **Built-in User Interface:** GitLab CI/CD is growing in popularity and has an > user-friendly interface. Our experience with the upstream DRM-CI in the kernel > tree (see this blog post [https://www.collabora.com/news-and-blog/blog/2024/02/08/drm-ci-a-gitlab-ci-pipeline-for-linux-kernel-testing/] ) > has provided insights into how such a system can benefit the wider community. > > 2. **Distributed Infrastructure:** > The proposed GitLab-CI pipeline is designed with a distributed infrastructure > model, being possible to run in any gitlab instance. > > 3. **Reduce regressions:** Fostering a culture where people habitually run > validated tests and post their results can prevent many issues in post-merge > tests. > > 4. **Collaborative Testing Environment:** The kernel community is already > engaged in numerous testing efforts, including various GitLab-CI pipelines such > as DRM-CI, which I maintain, along with other solutions like KernelCI and > BPF-CI. This proposal is designed to further stimulate contributions to the > evolving testing landscape. Our goal is to establish a comprehensive suite of > common tools and files. > > 5. **Ownership of QA:** > Discrepancies between kernel code and outdated tests often lead to misattributed > failures, complicating regression tracking. This issue, often arising from > neglected or deprioritized test updates, creates uncertainty about the source of > failures. Adopting an "always green pipeline" approach, as detailed in this > patch's documentation, encourages timely maintenance and validation of tests. > This ensures that testing accurately reflects the current state of the kernel, > thereby improving the effectiveness of our QA processes. > > Additionally, if we discover that this method isn't working for us, we can > easily remove it from the codebase, as it is primarily contained within the ci/ > folder. > > > Future Work > =========== > > **Expanding Static Checks:** > We have the opportunity to integrate a variety of static analysis tools, > including: > - dtbs_checks > - sparse > - yamllint > - dt-doc-validate > - coccicheck > > **Adding Userspace Tests on VMs:** > To further our testing, we can implement userspace tests that run on virtual > machines (VMs), such as: > - kselftests > - kunit tests > - Subsystem-specific tests, customizable in the scenarios. > > **Leveraging External Test Labs:** > We can extend our testing to external labs, similar to what DRM-CI currently > does. This includes: > - Lava labs > - Bare metal labs > - Using KernelCI-provided labs > > **Other integrations** > - Submit results to KCIDB > > **Lightweight Implementation for All Developers:** > We aim to design these tests to be lightweight, ensuring developers with limited > computing resources can still run essential tests. Resource-intensive tests can > be set to trigger manually, rather than automatically, to accommodate diverse > development environments. > > > Chat Discussions > ================ > > For those interested in further discussions: > > **Join Our Slack Channel:** > We have a Slack channel, #gitlab-ci, on the KernelCI Slack instance https://kernelci.slack.com/ . > Feel free to join and contribute to the conversation. The KernelCI team has > weekly calls where we also discuss the GitLab-CI pipeline. Could we communicate using free software please ? Furthermore, it's not possible to create an account on that slack instance unless you have an e-mail address affiliated with a small number of companies (https://kernelci.slack.com/signup#/domain-signup). That's a big no-go for me. > **Acknowledgments:** > A special thanks to Nikolai Kondrashov, Tales da Aparecida - both from Red Hat - > and KernelCI community for their valuable feedback and support in this proposal. > > > I eagerly await your thoughts and suggestions on this initiative. > > Also, if you want to see this initiave move faster, we are happy to discuss > funding options. > > Best regards, > Helen Koike > > Helen Koike (3): > kci-gitlab: Introducing GitLab-CI Pipeline for Kernel Testing > kci-gitlab: Add documentation > kci-gitlab: docs: Add images > > .gitlab-ci.yml | 2 + > Documentation/ci/gitlab-ci/gitlab-ci.rst | 404 ++++++++++++++++++ > .../ci/gitlab-ci/images/job-matrix.png | Bin 0 -> 159752 bytes > .../gitlab-ci/images/new-project-runner.png | Bin 0 -> 607737 bytes > .../ci/gitlab-ci/images/pipelines-on-push.png | Bin 0 -> 532143 bytes > .../ci/gitlab-ci/images/the-pipeline.png | Bin 0 -> 91675 bytes > .../ci/gitlab-ci/images/variables.png | Bin 0 -> 277518 bytes > Documentation/index.rst | 7 + > MAINTAINERS | 9 + > ci/gitlab-ci/bootstrap-gitlab-runner.sh | 55 +++ > ci/gitlab-ci/ci-scripts/build-docs.sh | 35 ++ > ci/gitlab-ci/ci-scripts/build-kernel.sh | 35 ++ > ci/gitlab-ci/ci-scripts/ici-functions.sh | 104 +++++ > ci/gitlab-ci/ci-scripts/install-smatch.sh | 13 + > .../ci-scripts/parse_commit_message.sh | 27 ++ > ci/gitlab-ci/ci-scripts/run-checkpatch.sh | 19 + > ci/gitlab-ci/ci-scripts/run-smatch.sh | 45 ++ > ci/gitlab-ci/docker-compose.yaml | 18 + > ci/gitlab-ci/linux.code-workspace | 11 + > ci/gitlab-ci/yml/build.yml | 43 ++ > ci/gitlab-ci/yml/cache.yml | 26 ++ > ci/gitlab-ci/yml/container.yml | 36 ++ > ci/gitlab-ci/yml/gitlab-ci.yml | 71 +++ > ci/gitlab-ci/yml/kernel-combinations.yml | 18 + > ci/gitlab-ci/yml/scenarios.yml | 12 + > ci/gitlab-ci/yml/scenarios/file-systems.yml | 21 + > ci/gitlab-ci/yml/scenarios/media.yml | 21 + > ci/gitlab-ci/yml/scenarios/network.yml | 21 + > ci/gitlab-ci/yml/static-checks.yml | 21 + > 29 files changed, 1074 insertions(+) > create mode 100644 .gitlab-ci.yml > create mode 100644 Documentation/ci/gitlab-ci/gitlab-ci.rst > create mode 100644 Documentation/ci/gitlab-ci/images/job-matrix.png > create mode 100644 Documentation/ci/gitlab-ci/images/new-project-runner.png > create mode 100644 Documentation/ci/gitlab-ci/images/pipelines-on-push.png > create mode 100644 Documentation/ci/gitlab-ci/images/the-pipeline.png > create mode 100644 Documentation/ci/gitlab-ci/images/variables.png > create mode 100755 ci/gitlab-ci/bootstrap-gitlab-runner.sh > create mode 100755 ci/gitlab-ci/ci-scripts/build-docs.sh > create mode 100755 ci/gitlab-ci/ci-scripts/build-kernel.sh > create mode 100644 ci/gitlab-ci/ci-scripts/ici-functions.sh > create mode 100755 ci/gitlab-ci/ci-scripts/install-smatch.sh > create mode 100755 ci/gitlab-ci/ci-scripts/parse_commit_message.sh > create mode 100755 ci/gitlab-ci/ci-scripts/run-checkpatch.sh > create mode 100755 ci/gitlab-ci/ci-scripts/run-smatch.sh > create mode 100644 ci/gitlab-ci/docker-compose.yaml > create mode 100644 ci/gitlab-ci/linux.code-workspace > create mode 100644 ci/gitlab-ci/yml/build.yml > create mode 100644 ci/gitlab-ci/yml/cache.yml > create mode 100644 ci/gitlab-ci/yml/container.yml > create mode 100644 ci/gitlab-ci/yml/gitlab-ci.yml > create mode 100644 ci/gitlab-ci/yml/kernel-combinations.yml > create mode 100644 ci/gitlab-ci/yml/scenarios.yml > create mode 100644 ci/gitlab-ci/yml/scenarios/file-systems.yml > create mode 100644 ci/gitlab-ci/yml/scenarios/media.yml > create mode 100644 ci/gitlab-ci/yml/scenarios/network.yml > create mode 100644 ci/gitlab-ci/yml/static-checks.yml
On Thu, Feb 29, 2024 at 01:07:25AM +0200, Laurent Pinchart wrote: > > Chat Discussions > > ================ > > > > For those interested in further discussions: > > > > **Join Our Slack Channel:** > > We have a Slack channel, #gitlab-ci, on the KernelCI Slack instance https://kernelci.slack.com/ . > > Feel free to join and contribute to the conversation. The KernelCI team has > > weekly calls where we also discuss the GitLab-CI pipeline. > > Could we communicate using free software please ? Furthermore, it's not > possible to create an account on that slack instance unless you have an > e-mail address affiliated with a small number of companies > (https://kernelci.slack.com/signup#/domain-signup). That's a big no-go > for me. Yeah, and that list looks super restrictive and arbitrary. Like, microsoft is there but kernel.org isn't? I'm sure there's a reason, but we should at the very least open it to everyone. Maxime
On 2/29/24 01:07, Laurent Pinchart wrote: > On Wed, Feb 28, 2024 at 07:55:24PM -0300, Helen Koike wrote: >> **Join Our Slack Channel:** >> We have a Slack channel, #gitlab-ci, on the KernelCI Slack instance https://kernelci.slack.com/ . >> Feel free to join and contribute to the conversation. The KernelCI team has >> weekly calls where we also discuss the GitLab-CI pipeline. > > Could we communicate using free software please ? Furthermore, it's not > possible to create an account on that slack instance unless you have an > e-mail address affiliated with a small number of companies > (https://kernelci.slack.com/signup#/domain-signup). That's a big no-go > for me. Yes, it's not ideal that we use closed-source software for communication, but FWIW I'd be happy to invite you there. Perhaps if you try logging in e.g. with a Google account, I'd be able to see and approve your attempt too. Otherwise, our video meetings are generally open to everyone and run in Jitsi. Nick
On Thu, Feb 29, 2024 at 11:26:51AM +0200, Nikolai Kondrashov wrote: > On 2/29/24 01:07, Laurent Pinchart wrote: > > On Wed, Feb 28, 2024 at 07:55:24PM -0300, Helen Koike wrote: > >> **Join Our Slack Channel:** > >> We have a Slack channel, #gitlab-ci, on the KernelCI Slack instance https://kernelci.slack.com/ . > >> Feel free to join and contribute to the conversation. The KernelCI team has > >> weekly calls where we also discuss the GitLab-CI pipeline. > > > > Could we communicate using free software please ? Furthermore, it's not > > possible to create an account on that slack instance unless you have an > > e-mail address affiliated with a small number of companies > > (https://kernelci.slack.com/signup#/domain-signup). That's a big no-go > > for me. > > Yes, it's not ideal that we use closed-source software for communication, but > FWIW I'd be happy to invite you there. Perhaps if you try logging in e.g. with > a Google account, I'd be able to see and approve your attempt too. I don't use Google accounts to authenticate to third-party services, sorry. And in any case, that won't make slack free software. > Otherwise, our video meetings are generally open to everyone and run in Jitsi.
Hi Laurent, Helen, On Thu, Feb 29, 2024 at 01:07:25AM +0200, Laurent Pinchart wrote: > > **Join Our Slack Channel:** > > We have a Slack channel, #gitlab-ci, on the KernelCI Slack instance https://kernelci.slack.com/ . > > Feel free to join and contribute to the conversation. The KernelCI team has > > weekly calls where we also discuss the GitLab-CI pipeline. > > Could we communicate using free software please ? Furthermore, it's not > possible to create an account on that slack instance unless you have an > e-mail address affiliated with a small number of companies > (https://kernelci.slack.com/signup#/domain-signup). That's a big no-go > for me. I agree. There is no shortage of good alternatives either: we have IRC, Matrix and XMPP for instance. Just pick one of them.
On Thu, Feb 29, 2024 at 09:39:01AM +0000, Sakari Ailus wrote: > On Thu, Feb 29, 2024 at 01:07:25AM +0200, Laurent Pinchart wrote: > > > We have a Slack channel, #gitlab-ci, on the KernelCI Slack instance https://kernelci.slack.com/ . > > > Feel free to join and contribute to the conversation. The KernelCI team has > > > weekly calls where we also discuss the GitLab-CI pipeline. > > Could we communicate using free software please ? Furthermore, it's not > > possible to create an account on that slack instance unless you have an > > e-mail address affiliated with a small number of companies > > (https://kernelci.slack.com/signup#/domain-signup). That's a big no-go > > for me. > I agree. There is no shortage of good alternatives either: we have IRC, > Matrix and XMPP for instance. Just pick one of them. And indeed KernelCI does actually already have #kernelci on libera.chat.
On 2/29/24 11:34 AM, Laurent Pinchart wrote: > On Thu, Feb 29, 2024 at 11:26:51AM +0200, Nikolai Kondrashov wrote: >> On 2/29/24 01:07, Laurent Pinchart wrote: >>> On Wed, Feb 28, 2024 at 07:55:24PM -0300, Helen Koike wrote: >>>> **Join Our Slack Channel:** >>>> We have a Slack channel, #gitlab-ci, on the KernelCI Slack instance https://kernelci.slack.com/ . >>>> Feel free to join and contribute to the conversation. The KernelCI team has >>>> weekly calls where we also discuss the GitLab-CI pipeline. >>> >>> Could we communicate using free software please ? Furthermore, it's not >>> possible to create an account on that slack instance unless you have an >>> e-mail address affiliated with a small number of companies >>> (https://kernelci.slack.com/signup#/domain-signup). That's a big no-go >>> for me. >> >> Yes, it's not ideal that we use closed-source software for communication, but >> FWIW I'd be happy to invite you there. Perhaps if you try logging in e.g. with >> a Google account, I'd be able to see and approve your attempt too. > > I don't use Google accounts to authenticate to third-party services, > sorry. And in any case, that won't make slack free software. Of course. You're also welcome to join the #kernelci channel on libera.chat. It's much quieter there, though. Nick
On Thu, Feb 29, 2024 at 01:10:16PM +0200, Nikolai Kondrashov wrote: > On 2/29/24 11:34 AM, Laurent Pinchart wrote: > > On Thu, Feb 29, 2024 at 11:26:51AM +0200, Nikolai Kondrashov wrote: > >> On 2/29/24 01:07, Laurent Pinchart wrote: > >>> On Wed, Feb 28, 2024 at 07:55:24PM -0300, Helen Koike wrote: > >>>> **Join Our Slack Channel:** > >>>> We have a Slack channel, #gitlab-ci, on the KernelCI Slack instance https://kernelci.slack.com/ . > >>>> Feel free to join and contribute to the conversation. The KernelCI team has > >>>> weekly calls where we also discuss the GitLab-CI pipeline. > >>> > >>> Could we communicate using free software please ? Furthermore, it's not > >>> possible to create an account on that slack instance unless you have an > >>> e-mail address affiliated with a small number of companies > >>> (https://kernelci.slack.com/signup#/domain-signup). That's a big no-go > >>> for me. > >> > >> Yes, it's not ideal that we use closed-source software for communication, but > >> FWIW I'd be happy to invite you there. Perhaps if you try logging in e.g. with > >> a Google account, I'd be able to see and approve your attempt too. > > > > I don't use Google accounts to authenticate to third-party services, > > sorry. And in any case, that won't make slack free software. > > Of course. You're also welcome to join the #kernelci channel on libera.chat. Isn't that a bit pointless if it's no the main IM channel ? > It's much quieter there, though.
On 2/29/24 1:19 PM, Laurent Pinchart wrote: > On Thu, Feb 29, 2024 at 01:10:16PM +0200, Nikolai Kondrashov wrote: >> On 2/29/24 11:34 AM, Laurent Pinchart wrote: >>> On Thu, Feb 29, 2024 at 11:26:51AM +0200, Nikolai Kondrashov wrote: >>>> On 2/29/24 01:07, Laurent Pinchart wrote: >>>>> On Wed, Feb 28, 2024 at 07:55:24PM -0300, Helen Koike wrote: >>>>>> **Join Our Slack Channel:** >>>>>> We have a Slack channel, #gitlab-ci, on the KernelCI Slack instance https://kernelci.slack.com/ . >>>>>> Feel free to join and contribute to the conversation. The KernelCI team has >>>>>> weekly calls where we also discuss the GitLab-CI pipeline. >>>>> >>>>> Could we communicate using free software please ? Furthermore, it's not >>>>> possible to create an account on that slack instance unless you have an >>>>> e-mail address affiliated with a small number of companies >>>>> (https://kernelci.slack.com/signup#/domain-signup). That's a big no-go >>>>> for me. >>>> >>>> Yes, it's not ideal that we use closed-source software for communication, but >>>> FWIW I'd be happy to invite you there. Perhaps if you try logging in e.g. with >>>> a Google account, I'd be able to see and approve your attempt too. >>> >>> I don't use Google accounts to authenticate to third-party services, >>> sorry. And in any case, that won't make slack free software. >> >> Of course. You're also welcome to join the #kernelci channel on libera.chat. > > Isn't that a bit pointless if it's no the main IM channel ? Yes, it's not ideal, but if more people come there, more discussions will happen there too. Nick
On Thu, Feb 29, 2024 at 01:19:19PM +0200, Laurent Pinchart wrote: > On Thu, Feb 29, 2024 at 01:10:16PM +0200, Nikolai Kondrashov wrote: > > Of course. You're also welcome to join the #kernelci channel on libera.chat. > Isn't that a bit pointless if it's no the main IM channel ? It *was* the original channel and still gets some usage (mostly started by me admittedly since I've never joined slack for a bunch of reasons that make it hassle), IIRC the Slack was started because there were some interns who had trouble figuring out IRC and intermittent connectivity but people seem to have migrated.
On 29/02/2024 12:41, Mark Brown wrote: > On Thu, Feb 29, 2024 at 01:19:19PM +0200, Laurent Pinchart wrote: >> On Thu, Feb 29, 2024 at 01:10:16PM +0200, Nikolai Kondrashov wrote: > >>> Of course. You're also welcome to join the #kernelci channel on libera.chat. > >> Isn't that a bit pointless if it's no the main IM channel ? > > It *was* the original channel and still gets some usage (mostly started > by me admittedly since I've never joined slack for a bunch of reasons > that make it hassle), IIRC the Slack was started because there were some > interns who had trouble figuring out IRC and intermittent connectivity > but people seem to have migrated. In fact it was initially created for the members of the Linux Foundation project only, which is why registration is moderated for emails that don't have a domain linked to a member (BTW not any Google account will just work e.g. @gmail.com is moderated, only @google.com for Google employees isn't). And yes IRC is the "least common denominator" chat platform. Maybe having a bridge between the main Slack channel and IRC would help. Guillaume
On Thu, Feb 29, 2024 at 12:53:38PM +0100, Guillaume Tucker wrote: > On 29/02/2024 12:41, Mark Brown wrote: > > On Thu, Feb 29, 2024 at 01:19:19PM +0200, Laurent Pinchart wrote: > >> On Thu, Feb 29, 2024 at 01:10:16PM +0200, Nikolai Kondrashov wrote: > > > >>> Of course. You're also welcome to join the #kernelci channel on libera.chat. > > > >> Isn't that a bit pointless if it's no the main IM channel ? > > > > It *was* the original channel and still gets some usage (mostly started > > by me admittedly since I've never joined slack for a bunch of reasons > > that make it hassle), IIRC the Slack was started because there were some > > interns who had trouble figuring out IRC and intermittent connectivity > > but people seem to have migrated. > > In fact it was initially created for the members of the Linux > Foundation project only, which is why registration is moderated > for emails that don't have a domain linked to a member (BTW not > any Google account will just work e.g. @gmail.com is moderated, > only @google.com for Google employees isn't). > > And yes IRC is the "least common denominator" chat platform. > Maybe having a bridge between the main Slack channel and IRC > would help. If the gitlab CI pipeline proposal wants to be considered for inclusion in the kernel, I think it needs to switch to a free software solution for its *main* communication channels.
Hello, On 28/02/2024 23:55, Helen Koike wrote: > Dear Kernel Community, > > This patch introduces a `.gitlab-ci` file along with a `ci/` folder, defining a > basic test pipeline triggered by code pushes to a GitLab-CI instance. This > initial version includes static checks (checkpatch and smatch for now) and build > tests across various architectures and configurations. It leverages an > integrated cache for efficient build times and introduces a flexible 'scenarios' > mechanism for subsystem-specific extensions. This sounds like a nice starting point to me as an additional way to run tests upstream. I have one particular question as I see a pattern through the rest of the email, please see below. [...] > 4. **Collaborative Testing Environment:** The kernel community is already > engaged in numerous testing efforts, including various GitLab-CI pipelines such > as DRM-CI, which I maintain, along with other solutions like KernelCI and > BPF-CI. This proposal is designed to further stimulate contributions to the > evolving testing landscape. Our goal is to establish a comprehensive suite of > common tools and files. [...] > **Leveraging External Test Labs:** > We can extend our testing to external labs, similar to what DRM-CI currently > does. This includes: > - Lava labs > - Bare metal labs > - Using KernelCI-provided labs > > **Other integrations** > - Submit results to KCIDB [...] > **Join Our Slack Channel:** > We have a Slack channel, #gitlab-ci, on the KernelCI Slack instance https://kernelci.slack.com/ . > Feel free to join and contribute to the conversation. The KernelCI team has > weekly calls where we also discuss the GitLab-CI pipeline. > > **Acknowledgments:** > A special thanks to Nikolai Kondrashov, Tales da Aparecida - both from Red Hat - > and KernelCI community for their valuable feedback and support in this proposal. Where does this fit on the KernelCI roadmap? I see it mentioned a few times but it's not entirely clear whether this initiative is an independent one or in some way linked to KernelCI. Say, are you planning to use the kci tool, new API, compiler toolchains, user-space and Docker images etc? Or, are KernelCI plans evolving to follow this move? Thanks, Guillaume
On Thu, Feb 29, 2024 at 02:20:41PM +0200, Laurent Pinchart wrote: > On Thu, Feb 29, 2024 at 12:53:38PM +0100, Guillaume Tucker wrote: > > On 29/02/2024 12:41, Mark Brown wrote: > > > On Thu, Feb 29, 2024 at 01:19:19PM +0200, Laurent Pinchart wrote: > > >> On Thu, Feb 29, 2024 at 01:10:16PM +0200, Nikolai Kondrashov wrote: > > > > > >>> Of course. You're also welcome to join the #kernelci channel on libera.chat. > > > > > >> Isn't that a bit pointless if it's no the main IM channel ? > > > > > > It *was* the original channel and still gets some usage (mostly started > > > by me admittedly since I've never joined slack for a bunch of reasons > > > that make it hassle), IIRC the Slack was started because there were some > > > interns who had trouble figuring out IRC and intermittent connectivity > > > but people seem to have migrated. > > > > In fact it was initially created for the members of the Linux > > Foundation project only, which is why registration is moderated > > for emails that don't have a domain linked to a member (BTW not > > any Google account will just work e.g. @gmail.com is moderated, > > only @google.com for Google employees isn't). > > > > And yes IRC is the "least common denominator" chat platform. > > Maybe having a bridge between the main Slack channel and IRC > > would help. > > If the gitlab CI pipeline proposal wants to be considered for inclusion > in the kernel, I think it needs to switch to a free software solution > for its *main* communication channels. And to clarify, I didn't meant the kernel CI project, but only the gitlab CI pipeline for the Linux kernel project. I don't know how tightly integrated the two projects are though.
On 2/29/24 2:25 PM, Laurent Pinchart wrote: > On Thu, Feb 29, 2024 at 02:20:41PM +0200, Laurent Pinchart wrote: >> On Thu, Feb 29, 2024 at 12:53:38PM +0100, Guillaume Tucker wrote: >>> On 29/02/2024 12:41, Mark Brown wrote: >>>> On Thu, Feb 29, 2024 at 01:19:19PM +0200, Laurent Pinchart wrote: >>>>> On Thu, Feb 29, 2024 at 01:10:16PM +0200, Nikolai Kondrashov wrote: >>>> >>>>>> Of course. You're also welcome to join the #kernelci channel on libera.chat. >>>> >>>>> Isn't that a bit pointless if it's no the main IM channel ? >>>> >>>> It *was* the original channel and still gets some usage (mostly started >>>> by me admittedly since I've never joined slack for a bunch of reasons >>>> that make it hassle), IIRC the Slack was started because there were some >>>> interns who had trouble figuring out IRC and intermittent connectivity >>>> but people seem to have migrated. >>> >>> In fact it was initially created for the members of the Linux >>> Foundation project only, which is why registration is moderated >>> for emails that don't have a domain linked to a member (BTW not >>> any Google account will just work e.g. @gmail.com is moderated, >>> only @google.com for Google employees isn't). >>> >>> And yes IRC is the "least common denominator" chat platform. >>> Maybe having a bridge between the main Slack channel and IRC >>> would help. >> >> If the gitlab CI pipeline proposal wants to be considered for inclusion >> in the kernel, I think it needs to switch to a free software solution >> for its *main* communication channels. > > And to clarify, I didn't meant the kernel CI project, but only the > gitlab CI pipeline for the Linux kernel project. I don't know how > tightly integrated the two projects are though. They're not tightly integrated so far. However, we're exploring the idea of letting GitLab CI submit jobs to KernelCI and get results as a part of the pipeline. Helen already joined the #kernelci channel, and we will maintain a presence there for the GitLab CI project. Nick
On 2/29/24 2:20 PM, Guillaume Tucker wrote: > Hello, > > On 28/02/2024 23:55, Helen Koike wrote: >> Dear Kernel Community, >> >> This patch introduces a `.gitlab-ci` file along with a `ci/` folder, defining a >> basic test pipeline triggered by code pushes to a GitLab-CI instance. This >> initial version includes static checks (checkpatch and smatch for now) and build >> tests across various architectures and configurations. It leverages an >> integrated cache for efficient build times and introduces a flexible 'scenarios' >> mechanism for subsystem-specific extensions. > > This sounds like a nice starting point to me as an additional way > to run tests upstream. I have one particular question as I see a > pattern through the rest of the email, please see below. > > [...] > >> 4. **Collaborative Testing Environment:** The kernel community is already >> engaged in numerous testing efforts, including various GitLab-CI pipelines such >> as DRM-CI, which I maintain, along with other solutions like KernelCI and >> BPF-CI. This proposal is designed to further stimulate contributions to the >> evolving testing landscape. Our goal is to establish a comprehensive suite of >> common tools and files. > > [...] > >> **Leveraging External Test Labs:** >> We can extend our testing to external labs, similar to what DRM-CI currently >> does. This includes: >> - Lava labs >> - Bare metal labs >> - Using KernelCI-provided labs >> >> **Other integrations** >> - Submit results to KCIDB > > [...] > >> **Join Our Slack Channel:** >> We have a Slack channel, #gitlab-ci, on the KernelCI Slack instance https://kernelci.slack.com/ . >> Feel free to join and contribute to the conversation. The KernelCI team has >> weekly calls where we also discuss the GitLab-CI pipeline. >> >> **Acknowledgments:** >> A special thanks to Nikolai Kondrashov, Tales da Aparecida - both from Red Hat - >> and KernelCI community for their valuable feedback and support in this proposal. > > Where does this fit on the KernelCI roadmap? > > I see it mentioned a few times but it's not entirely clear > whether this initiative is an independent one or in some way > linked to KernelCI. Say, are you planning to use the kci tool, > new API, compiler toolchains, user-space and Docker images etc? > Or, are KernelCI plans evolving to follow this move? I would say this is an important part of KernelCI the project, considering its aim to improve testing and CI in the kernel. It's not a part of KernelCI the service as it is right now, although I would say it would be good to have ability to submit KernelCI jobs from GitLab CI and pull results in the same pipeline, as we discussed earlier. Nick
Hi, Le jeudi 29 février 2024 à 16:16 +0200, Nikolai Kondrashov a écrit : > On 2/29/24 2:20 PM, Guillaume Tucker wrote: > > Hello, > > > > On 28/02/2024 23:55, Helen Koike wrote: > > > Dear Kernel Community, > > > > > > This patch introduces a `.gitlab-ci` file along with a `ci/` folder, defining a > > > basic test pipeline triggered by code pushes to a GitLab-CI instance. This > > > initial version includes static checks (checkpatch and smatch for now) and build > > > tests across various architectures and configurations. It leverages an > > > integrated cache for efficient build times and introduces a flexible 'scenarios' > > > mechanism for subsystem-specific extensions. > > > > This sounds like a nice starting point to me as an additional way > > to run tests upstream. I have one particular question as I see a > > pattern through the rest of the email, please see below. > > > > [...] > > > > > 4. **Collaborative Testing Environment:** The kernel community is already > > > engaged in numerous testing efforts, including various GitLab-CI pipelines such > > > as DRM-CI, which I maintain, along with other solutions like KernelCI and > > > BPF-CI. This proposal is designed to further stimulate contributions to the > > > evolving testing landscape. Our goal is to establish a comprehensive suite of > > > common tools and files. > > > > [...] > > > > > **Leveraging External Test Labs:** > > > We can extend our testing to external labs, similar to what DRM-CI currently > > > does. This includes: > > > - Lava labs > > > - Bare metal labs > > > - Using KernelCI-provided labs > > > > > > **Other integrations** > > > - Submit results to KCIDB > > > > [...] > > > > > **Join Our Slack Channel:** > > > We have a Slack channel, #gitlab-ci, on the KernelCI Slack instance https://kernelci.slack.com/ . > > > Feel free to join and contribute to the conversation. The KernelCI team has > > > weekly calls where we also discuss the GitLab-CI pipeline. > > > > > > **Acknowledgments:** > > > A special thanks to Nikolai Kondrashov, Tales da Aparecida - both from Red Hat - > > > and KernelCI community for their valuable feedback and support in this proposal. > > > > Where does this fit on the KernelCI roadmap? > > > > I see it mentioned a few times but it's not entirely clear > > whether this initiative is an independent one or in some way > > linked to KernelCI. Say, are you planning to use the kci tool, > > new API, compiler toolchains, user-space and Docker images etc? > > Or, are KernelCI plans evolving to follow this move? > > I would say this is an important part of KernelCI the project, considering its > aim to improve testing and CI in the kernel. It's not a part of KernelCI the > service as it is right now, although I would say it would be good to have > ability to submit KernelCI jobs from GitLab CI and pull results in the same > pipeline, as we discussed earlier. I'd like to add that both CI have a different purpose in the Linux project. This CI work is a pre-merge verification. Everyone needs to run checkpatch and smatch, this is automating it (and will catch those that forgot or ran it incorrectly). But it can go further by effectively testing specific patches on real hardware (with pretty narrow filters). It will help catch submission issues earlier, and reduce kernelCI regression rate. As a side effect, kernelCI infra will endup catching the "integration" issues, which are the issue as a result of simultenous changes in different trees. They are also often more complex and benefit from the bisection capabilities. kernelCI tests are also a lot more intensive, they usually covers everything, but they bundle multiple changes per run. The pre-merge tests will be reduced to what seems meaningful for the changes. Its important to understand that pre- merge CI have a time cost, and we need to make sure CI time does not exceed the merge window period. Nicolas
On 29/02/2024 17:28, Nicolas Dufresne wrote: > Hi, > > Le jeudi 29 février 2024 à 16:16 +0200, Nikolai Kondrashov a écrit : >> On 2/29/24 2:20 PM, Guillaume Tucker wrote: >>> Hello, >>> >>> On 28/02/2024 23:55, Helen Koike wrote: >>>> Dear Kernel Community, >>>> >>>> This patch introduces a `.gitlab-ci` file along with a `ci/` folder, defining a >>>> basic test pipeline triggered by code pushes to a GitLab-CI instance. This >>>> initial version includes static checks (checkpatch and smatch for now) and build >>>> tests across various architectures and configurations. It leverages an >>>> integrated cache for efficient build times and introduces a flexible 'scenarios' >>>> mechanism for subsystem-specific extensions. >>> >>> This sounds like a nice starting point to me as an additional way >>> to run tests upstream. I have one particular question as I see a >>> pattern through the rest of the email, please see below. >>> >>> [...] >>> >>>> 4. **Collaborative Testing Environment:** The kernel community is already >>>> engaged in numerous testing efforts, including various GitLab-CI pipelines such >>>> as DRM-CI, which I maintain, along with other solutions like KernelCI and >>>> BPF-CI. This proposal is designed to further stimulate contributions to the >>>> evolving testing landscape. Our goal is to establish a comprehensive suite of >>>> common tools and files. >>> >>> [...] >>> >>>> **Leveraging External Test Labs:** >>>> We can extend our testing to external labs, similar to what DRM-CI currently >>>> does. This includes: >>>> - Lava labs >>>> - Bare metal labs >>>> - Using KernelCI-provided labs >>>> >>>> **Other integrations** >>>> - Submit results to KCIDB >>> >>> [...] >>> >>>> **Join Our Slack Channel:** >>>> We have a Slack channel, #gitlab-ci, on the KernelCI Slack instance https://kernelci.slack.com/ . >>>> Feel free to join and contribute to the conversation. The KernelCI team has >>>> weekly calls where we also discuss the GitLab-CI pipeline. >>>> >>>> **Acknowledgments:** >>>> A special thanks to Nikolai Kondrashov, Tales da Aparecida - both from Red Hat - >>>> and KernelCI community for their valuable feedback and support in this proposal. >>> >>> Where does this fit on the KernelCI roadmap? >>> >>> I see it mentioned a few times but it's not entirely clear >>> whether this initiative is an independent one or in some way >>> linked to KernelCI. Say, are you planning to use the kci tool, >>> new API, compiler toolchains, user-space and Docker images etc? >>> Or, are KernelCI plans evolving to follow this move? >> >> I would say this is an important part of KernelCI the project, considering its >> aim to improve testing and CI in the kernel. It's not a part of KernelCI the >> service as it is right now, although I would say it would be good to have >> ability to submit KernelCI jobs from GitLab CI and pull results in the same >> pipeline, as we discussed earlier. Right, I think this needs a bit of disambiguation. The legacy KernelCI system from the Linaro days several years ago is really a service on its own like the many other CIs out there. However, the new KernelCI API and related tooling (kci command line, new web dashboard, modular runtime design etc.) is not that. It's about addressing all the community requirements and that includes being able to run a same test manually in a shell, or in a VM, or automatically from GitLab CI or using a main generic pipeline hosted by KernelCI itself. With this approach, there's no distinction between "the project" and "the service", and as we discussed before there shouldn't even be a distinction with KCIDB. Just KernelCI. However I don't really see this happening, unless I'm missing a part of the story or some upcoming announcement with an updated roadmap. For some reason the old and established paradigm seems unshakeable. The new KernelCI implementation is starting to look just like a refresh of the old one with newer components - which is a huge missed opportunity to really change things IMHO. This may sound like a bit of a tangent, facilitating GitLab CI for the upstream kernel is of course significant progress in any case - no question about that. My comment is more about why it's being driven hand-in-hand with KernelCI in what seems like a diverging direction from KernelCI's announced plans. Why push for a GitLab-centered orchestration when there's a more universal solution being proposed by the project? I would find it easier to understand - and I sense I'm not the only one here reading the thread - if KernelCI wasn't mentioned that many times in the cover letter and if the scripts didn't have KCI_* in so many places, basically if this was clearly an independent initiative such as KUnit, 0-day or regzbot. > I'd like to add that both CI have a different purpose in the Linux project. This > CI work is a pre-merge verification. Everyone needs to run checkpatch and > smatch, this is automating it (and will catch those that forgot or ran it > incorrectly). But it can go further by effectively testing specific patches on > real hardware (with pretty narrow filters). It will help catch submission issues > earlier, and reduce kernelCI regression rate. As a side effect, kernelCI infra > will endup catching the "integration" issues, which are the issue as a result of > simultenous changes in different trees. They are also often more complex and > benefit from the bisection capabilities. > > kernelCI tests are also a lot more intensive, they usually covers everything, > but they bundle multiple changes per run. The pre-merge tests will be reduced to > what seems meaningful for the changes. Its important to understand that pre- > merge CI have a time cost, and we need to make sure CI time does not exceed the > merge window period. You're referring to the legacy KernelCI, to illustrate the point I made earlier. The plan with the new implementation was to be able to do pre-merge testing as well as many other things, basically to provide a platform able to cope with the diversity of workflows across the kernel subsystems and the complexity of the "system under test" itself. Well, let's see how this goes and it does look quite promising. Evolution is always a chaotic process, especially in a complex project like this. I'm not expecting to get all the answers to the questions I have but it seemed important to raise this point and seek a bit more clarity around KernelCI. Guillaume
On 02/03/2024 10:48 pm, Gustavo Padovan wrote: > On Friday, March 01, 2024 18:56 -03, Guillaume Tucker <gtucker@gtucker.io> wrote: > >> On 29/02/2024 17:28, Nicolas Dufresne wrote: >>> Hi, >>> >>> Le jeudi 29 février 2024 à 16:16 +0200, Nikolai Kondrashov a écrit : >>>> On 2/29/24 2:20 PM, Guillaume Tucker wrote: >>>>> Hello, >>>>> >>>>> On 28/02/2024 23:55, Helen Koike wrote: >>>>>> Dear Kernel Community, >>>>>> >>>>>> This patch introduces a `.gitlab-ci` file along with a `ci/` folder, defining a >>>>>> basic test pipeline triggered by code pushes to a GitLab-CI instance. This >>>>>> initial version includes static checks (checkpatch and smatch for now) and build >>>>>> tests across various architectures and configurations. It leverages an >>>>>> integrated cache for efficient build times and introduces a flexible 'scenarios' >>>>>> mechanism for subsystem-specific extensions. >>>>> >>>>> This sounds like a nice starting point to me as an additional way >>>>> to run tests upstream. I have one particular question as I see a >>>>> pattern through the rest of the email, please see below. >>>>> >>>>> [...] >>>>> >>>>>> 4. **Collaborative Testing Environment:** The kernel community is already >>>>>> engaged in numerous testing efforts, including various GitLab-CI pipelines such >>>>>> as DRM-CI, which I maintain, along with other solutions like KernelCI and >>>>>> BPF-CI. This proposal is designed to further stimulate contributions to the >>>>>> evolving testing landscape. Our goal is to establish a comprehensive suite of >>>>>> common tools and files. >>>>> >>>>> [...] >>>>> >>>>>> **Leveraging External Test Labs:** >>>>>> We can extend our testing to external labs, similar to what DRM-CI currently >>>>>> does. This includes: >>>>>> - Lava labs >>>>>> - Bare metal labs >>>>>> - Using KernelCI-provided labs >>>>>> >>>>>> **Other integrations** >>>>>> - Submit results to KCIDB >>>>> >>>>> [...] >>>>> >>>>>> **Join Our Slack Channel:** >>>>>> We have a Slack channel, #gitlab-ci, on the KernelCI Slack instance https://kernelci.slack.com/ . >>>>>> Feel free to join and contribute to the conversation. The KernelCI team has >>>>>> weekly calls where we also discuss the GitLab-CI pipeline. >>>>>> >>>>>> **Acknowledgments:** >>>>>> A special thanks to Nikolai Kondrashov, Tales da Aparecida - both from Red Hat - >>>>>> and KernelCI community for their valuable feedback and support in this proposal. >>>>> >>>>> Where does this fit on the KernelCI roadmap? >>>>> >>>>> I see it mentioned a few times but it's not entirely clear >>>>> whether this initiative is an independent one or in some way >>>>> linked to KernelCI. Say, are you planning to use the kci tool, >>>>> new API, compiler toolchains, user-space and Docker images etc? >>>>> Or, are KernelCI plans evolving to follow this move? >>>> >>>> I would say this is an important part of KernelCI the project, considering its >>>> aim to improve testing and CI in the kernel. It's not a part of KernelCI the >>>> service as it is right now, although I would say it would be good to have >>>> ability to submit KernelCI jobs from GitLab CI and pull results in the same >>>> pipeline, as we discussed earlier. >> >> Right, I think this needs a bit of disambiguation. The legacy >> KernelCI system from the Linaro days several years ago is really >> a service on its own like the many other CIs out there. However, >> the new KernelCI API and related tooling (kci command line, new >> web dashboard, modular runtime design etc.) is not that. It's >> about addressing all the community requirements and that includes >> being able to run a same test manually in a shell, or in a VM, or >> automatically from GitLab CI or using a main generic pipeline >> hosted by KernelCI itself. With this approach, there's no >> distinction between "the project" and "the service", and as we >> discussed before there shouldn't even be a distinction with >> KCIDB. Just KernelCI. >> >> However I don't really see this happening, unless I'm missing a >> part of the story or some upcoming announcement with an updated >> roadmap. For some reason the old and established paradigm seems >> unshakeable. The new KernelCI implementation is starting to look >> just like a refresh of the old one with newer components - which >> is a huge missed opportunity to really change things IMHO. > > Calling that a missed opportunity is a subjective perspective about > the latest developments in KernelCI. The system implementation is > one level less important than the actual kernel community engagement > the project can generate. If one asks people around, the lack of > community engagement with KernelCI is evident. Well I would argue that community engagement and technical development work side-by-side, not as a hierarchy. You can't run Android phones or data centers with community engagement, and you can't write the code without the people. I was enquiring about this in particluar because I'm preparing a LF webinar, so I've started another thread to avoid spamming this one as it's really a side topic: https://lore.kernel.org/all/71f59a56-aef3-4bae-867b-769a0cdd1c1b@gtucker.io/T/#u > However, after the recent leadership change in the project there is a > growing effort to bring the kernel community closer to the KernelCI > project with a renewed focus on high quality test results, clean regression > reporting, among other things. Then, with an increased number of community > members involved, we will have the necessary feedback (and funding!) to > evolve the KernelCI infrastructure and technology to new levels. In a nutshell, KernelCI started small and then joined the LF. The scope changed to encompass the whole kernel community, and as a result a number of things were done: community survey, lots of conf talks and email discussions with kernel devs etc. Then some plan was put in place with the new API and web dashboard designs, but other priorities slowed things down on this front which is why it's still not quite there two years later. That's also why community engagement has been low. But that's OK, plans are just plans and things are catching up again now I think. And once again, I think this GitLab CI move is great :) Guillaume