[Git][ghc/ghc][wip/ghcup-metadata-nightly] Incrementally update ghcup metadata in ghc/ghcup-metadata

[Matthew Pickering (@mpickering)]

Matthew Pickering pushed to branch wip/ghcup-metadata-nightly at Glasgow Haskell Compiler / GHC


Commits:
b5de89fe by Matthew Pickering at 2023-05-02T10:25:47+01:00
Incrementally update ghcup metadata in ghc/ghcup-metadata

This job paves the way for distributing nightly builds

* A new repo https://gitlab.haskell.org/ghc/ghcup-metadata stores the
  metadata on the "updates" branch.
* Each night this metadata is downloaded and the nightly builds are
  appended to the end of the metadata.
* The update job only runs on the scheduled nightly pipeline, not just
  when NIGHTLY=1.

Things which are not done yet

* Modify the retention policy for nightly jobs
* Think about building release flavour compilers to distribute nightly.

Fixes #23334

- - - - -


1 changed file:

- .gitlab-ci.yml


Changes:

=====================================
.gitlab-ci.yml
=====================================
@@ -999,7 +999,7 @@ project-version:
     - . ./version.sh
 
     # Download existing ghcup metadata
-    - nix shell --extra-experimental-features nix-command --extra-experimental-features flakes nixpkgs#wget -c wget "https://raw.githubusercontent.com/haskell/ghcup-metadata/develop/ghcup-0.0.7.yaml"
+    - nix shell --extra-experimental-features nix-command --extra-experimental-features flakes nixpkgs#wget -c wget "https://gitlab.haskell.org/ghc/ghcup-metadata/-/raw/updates/ghcup-0.0.7.yaml"
 
     - .gitlab/generate_job_metadata
 
@@ -1048,6 +1048,37 @@ ghcup-metadata-nightly:
   rules:
     - if: $NIGHTLY
 
+# Update the
+ghcup-metadata-nightly-push:
+  stage: deploy
+  image: "registry.gitlab.haskell.org/ghc/ci-images/x86_64-linux-fedora33:$DOCKER_REV"
+  dependencies: null
+  tags:
+    - x86_64-linux
+  variables:
+    BUILD_FLAVOUR: default
+    GIT_SUBMODULE_STRATEGY: "none"
+  needs:
+    - job: ghcup-metadata-nightly
+      artifacts: true
+  script:
+    - git clone https://gitlab.haskell.org/ghc/ghcup-metadata.git
+    - cp metadata_test.yaml ghcup-metadata/ghcup-0.0.7.yaml
+    - cd ghcup-metadata
+    - git config user.email "ghc-ci at gitlab-haskell.org"
+    - git config user.name "GHC GitLab CI"
+    - git remote add gitlab_origin https://oauth2:$PROJECT_PUSH_TOKEN@gitlab.haskell.org/ghc/ghcup-metadata.git
+    - git add .
+    - git commit -m "Update metadata"
+    - git push gitlab_origin HEAD:updates -o ci.skip
+  rules:
+    - if: $NIGHTLY
+    # Only run the update on scheduled nightly pipelines, ie once a day
+    - if: $CI_PIPELINE_SOURCE == "schedule"
+    # And only update the metadata for master branch
+    - if: '$CI_COMMIT_BRANCH == "master"'
+
+
 ghcup-metadata-release:
   # No explicit needs for release pipeline as we assume we need everything and everything will pass.
   extends: .ghcup-metadata



View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/b5de89fec18270c18fb60369931131bfd41d780b

-- 
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/b5de89fec18270c18fb60369931131bfd41d780b
You're receiving this email because of your account on gitlab.haskell.org.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-commits/attachments/20230502/94a86955/attachment-0001.html>