Current Page: NeXtMidas User's Guide NeXtMidas Support for Continous Integrations with GitLab
| ### NeXtMidas Support for Continuous Integrations with GitLab #### Topics covered in this page Using GitLab CI to build and publish NeXtMidas Prerequisites Define Project-specific CI/CD variables from the Gitlab UI Registering Shared Runners Versus Project-Specific Runners NeXtMidas Gitlab CI Configuration file How to build the Docker image needed by GitLab for the NeXtMidas build Prerequisites Define Project-specific CI/CD variables from the Gitlab UI Additional Files Extra steps if Standard Docker-in-Docker Images are Not Available Build the Docker-in-Docker Docker image (client version) from scratch Build the Docker-dind Docker image (backend version) from scratch Configure a Gitlab Runner to use the Docker daemon provided by Docker-dind ##### Using GitLab CI to build and publish NeXtMidas The process: Whenever a change is pushed to the NextMidas Gitlab project, a Gitlab job should be started for this project. Jobs can also be started manually. Inside Gitlab, the Docker-builder image will be pulled from the local private Docker registry to provide access to Gradle, OpenJDK, and Nexus. This project will run a Gradle build of NeXtMidas and publish all NextMidas Gradle jar files to the Nexus repo. Prerequisites Access to a Docker registry Access to the standard docker-dind Docker image (backend version) Access to the CentOS 7 (or other desired OS) Docker image Access to a Yum Repository Server Gradle (If not provided in separate Docker-builder image) Access to Nexus (NOTE: Only include gradle.properties, keystores, and truststores if not already provided in separate Docker-builder image) gradle.properties should include any necessary Nexus url's, as well as necessary keystores and truststores If using local Nexus: Add your local rootCA to the trust store referenced by the gradle.properties file Keystores and truststores Define Project-specific CI/CD variables from the Gitlab UI From the Gitlab UI, navigate to the NeXtMidas project, then click on Settings -> CI/CD, then click on the Expand button next to Variables Add three variables by clicking on Add Variable, then select type Variable and uncheck the box that says Protected DOCKER_REG_ADDR - Address at which the local Docker registry resides Example: nxm.example.com:12345 DOCKER_IMAGE_NAME - Name of the Docker image for the NeXtMidas build Example: centos7_java11_gradle6 DOCKER_IMAGE_TAG - Tag of the Docker image for the NeXtMidas build Examples: latest-builder, builder-22-05-06 Registering Shared Runners Versus Project-Specific Runners Shared Runner (NOTE: This can only be done if you have access to the Admin Area in the Gitlab UI) Gitlab UI -> Admin Area -> Runners Grab the URL and registration token provided in the top-right section of your screen Use the gitlab-runner cmd-line tool, along with the URL and token, to register your shared runner NOTE: This can be done either on your local machine or using a Docker container (please refer to steps below for container-specific instructions) Gitlab UI -> Admin Area -> Runners -> [your shared runner] Select which projects this Gitlab runner should be enabled for Project-Specific Runner (NOTE: This can only be done if you have access to the Settings in your Gitlab Project) Gitlab UI -> Projects -> [your project] -> Settings -> CI/CD, -> Runners Grab the URL and registration token provided on the left-hand side of your screen under Specific Runners Use the gitlab-runner cmd-line tool, along with the URL and token, to register your project-specific runner NOTE: This can be done either on your local machine or using a Docker container (please refer to steps above for container instructions) NeXtMidas Gitlab CI Configuration file .gitlab-ci.yml - Added to baseline in $NMROOT, but here as a reference - DOES NOT NEED TO BE MODIFIED # This is the Gitlab configuration file for NeXtMidas CI stages: - build before_script: - export COMMIT_TIME="$(date --date=${CI_COMMIT_TIMESTAMP} +%Y%m%d%H%M%S)" - export BUILD_TIME="$(date +%Y%m%d%H%M%S)" - export TAG_PREFIX=`awk 'BEGIN { printf("%s/%s:%s",tolower(ENVIRON["DOCKER_REGISTRY"]), tolower(ENVIRON["CI_PROJECT_NAME"]), tolower(ENVIRON["CI_COMMIT_BRANCH"])); }' /dev/null` - export IMAGE_TAG="${TAG_PREFIX}-${CI_COMMIT_SHORT_SHA}-${BUILD_TIME}" - export GRADLE_TARGETS="clean build" - export GRADLE_FLAGS="--refresh-dependencies -Dorg.gradle.internal.publish.checksums.insecure=true -x test" - if [[ "${CI_COMMIT_BRANCH}" = "ops" ]]; then export GRADLE_TARGETS="${GRADLE_TARGETS} publish"; fi variables: # Location of the Docker registry DOCKER_REGISTRY: ${DOCKER_REG_ADDR} build: # Grab the Docker image for the NeXtMidas build from the Docker Registry # The Docker Registry Address (DOCKER_REG_ADDR), Docker image name (DOCKER_IMAGE_NAME), # and Docker image tag (DOCKER_IMAGE_TAG) are specified as GitLab CI/CD project variables image: ${DOCKER_REG_ADDR}/${DOCKER_IMAGE_NAME}:${DOCKER_IMAGE_TAG} stage: build services: # Provides access to the Docker daemon inside the current NeXtMidas Gitlab job - ${DOCKER_REG_ADDR}/docker:dind script: # Update current Centos 7 environment - yum update -y # .repo file must be added back in after doing a yum update - rm /etc/yum.repos.d/* - cp /usr/local/centos-reposerver.repo /etc/yum.repos.d/ # install gcc-c++ - yum install -y gcc-c++ # Convert NeXtMidas to a Gradle project using softlinks - export NMROOT=$(pwd) - gradle/linkgradle.sh # Print out environment configuration for the current NeXtMidas Gitlab job - echo "==== Config ===="; echo "GITLAB_USER_NAME = ${GITLAB_USER_NAME}"; echo "CI_COMMIT_MESSAGE = ${CI_COMMIT_MESSAGE}"; echo "COMMIT_TIME = ${COMMIT_TIME}"; echo "BUILD_TIME = ${BUILD_TIME}"; echo "CI_BUILD_NAME = ${CI_BUILD_NAME}"; echo "IMAGE_TAG = ${IMAGE_TAG}"; echo "GRADLE_TARGETS = ${GRADLE_TARGETS}"; echo "GRADLE_FLAGS = ${GRADLE_FLAGS}"; # Run the NeXtMidas Gradle build using the Gradle env vars defined above - echo "==== Starting Gradle Build ====" - gradle --stop - sleep 1 - find ${HOME}/.gradle -name "*.lock" | xargs rm - gradle ${GRADLE_TARGETS} ${GRADLE_FLAGS} - gradle --stop - find -name '*.jar' | sort ##### How to build the Docker image needed by GitLab for the NeXtMidas build Prerequisites Access to a Docker registry Access to the docker-in-docker Docker image (client version) Access to the CentOS 7 (or other desired OS) Docker image Access to a Yum Repository Server Gradle Access to Nexus gradle.properties should include any necessary Nexus url's, as well as necessary keystores and truststores If using local Nexus: add your local rootCA to the trust store referenced by the gradle.properties file keystores and truststores Define Project-specific CI/CD variables from the Gitlab UI From the Gitlab UI, navigate to the docker-image project, then click on Settings -> CI/CD, then click on the Expand button next to Variables Add three variables by clicking on Add Variable, then select type Variable and uncheck the box that says Protected DOCKER_REG_ADDR - Address at which the local Docker registry resides Example: nxm.example.com:12345 DOCKER_IMAGE_NAME - Name of the Docker image for the NeXtMidas build Example: centos7_java11_gradle6 DOCKER_IMAGE_TAG - Tag of the Docker image for the NeXtMidas build Examples: latest-builder, builder-22-05-06 Additional Files: may need to install some dependencies .gitlab-ci.yml - sample below and under $NMROOT/gradle/gitlab-ci/docker-image # This is the Gitlab configuration file for building the Docker-builder Docker image which will be used by the NeXtMidas Gitlab project stages: - build before_script: - docker info - export VERSION=$(date +%y-%m-%d) - export BUILDER_TAG_NAME=${IMAGE_NAME}":builder-$VERSION" variables: # Name of the Docker-builder Docker image to be pushed to the Docker registry IMAGE_NAME: "${DOCKER_REG_ADDR}/${DOCKER_IMAGE_NAME}" build-full: # Use the docker-in-docker client image as the base image for the current java-docker-image Gitlab job image: ${DOCKER_REG_ADDR}/docker:latest stage: build script: - echo $BUILDER_TAG_NAME # Build the Docker-builder Docker image to be used by NeXtMidas - docker build --network=host -f Dockerfile-builder -t $BUILDER_TAG_NAME . # Tag and push the Docker-build Docker image to be used by NeXtMidas - docker tag $BUILDER_TAG_NAME ${IMAGE_NAME}:"latest-builder" - docker push ${IMAGE_NAME}":latest-builder" - docker push $BUILDER_TAG_NAME - echo $BUILDER_TAG_NAME only: - master Dockerfile-builder - sample below and under $NMROOT/gradle/gitlab-ci/docker-image # Start with a basic Centos 7 Docker image FROM [hostname]:[port]/centos:7 # Update repo based on the current environent and conditions/constraints RUN rm /etc/yum.repos.d/* COPY centos-reposerver.repo /usr/local/ RUN cp /usr/local/centos-reposerver.repo /etc/yum.repos.d/ # Install Openjdk 11 and unzip tools RUN yum install java-11-openjdk-devel unzip -y ENV JAVA_HOME /usr/lib/jvm/java-11-openjdk ENV JAVAHOME ${JAVA_HOME} ENV PATH $PATH:${JAVA_HOME}/bin # Working directory for this image WORKDIR /opt # Install, configure, and run the Gradle tool COPY gradle-6.3-all.zip ./ RUN jar -xf gradle-6.3-all.zip RUN rm gradle-6.3-all.zip RUN ln -sf /opt/gradle-6.3 /opt/gradle ENV GRADLE_BIN /opt/gradle/bin RUN chmod +x ${GRADLE_BIN}/gradle ENV PATH $PATH:${GRADLE_BIN} ENV JAVA_TOOL_OPTIONS -Dfile.encoding=UTF8 RUN gradle # Copy gradle.properties file into this image COPY gradle.properties /root/.gradle/ # Unzip SSL certificates to be used for Gradle COPY ssl.zip /usr/local/ssl.zip RUN unzip /usr/local/ssl.zip -d /usr/local/ RUN rm /usr/local/ssl.zip The process: Whenever a change is pushed to the java-docker-image Gitlab project, a Gitlab job should be started for this project. Inside Gitlab, the job will be using the Docker-in-Docker Docker image to run Docker. This project will build a Docker-builder image with CentOS 7, OpenJDK 11, and Gradle. This custom Docker-image will be added to a local private registry hosted by the local host environment to later be referenced by the NeXtMidas Gitlab project. ##### Extra steps if Standard Docker-in-Docker Images are Not Available Build the Docker-in-Docker Docker image (client version) from scratch Prerequisites Access to latest Alpine standard Docker image Files generally needed (As of docker:20.10) Alpine Dependencies (As of alpine:3.15, assuming no access to the Alpine public repository) ca-certificates, libc6-compat, libedit, ncurses-libs, ncurses-terminfo-base, openssh-client-common, openssh-client-default, openssh-keygen Docker-in-Docker base os tar file docker-entrypoint.sh modprobe.sh Dockerfile Commands to run if all Docker-in-Docker resources are present `sudo docker build --network=host -t docker:20.10 .` `sudo docker build --network=host -t docker:latest .` Build the Docker-dind Docker image (backend version) from scratch Prerequisites Access to latest standard Docker-in-Docker image Files generally needed (As of docker:dind-20.10) Alpine Dependencies (As of alpine:3.15, assuming no access to the Alpine public repository) btrfs-progs, e2fsprogs, e2fsprogs-extra, e2fsprogs-libs, inih, ip6tables, iptables, keyutils-libs, krb5-conf, krb5-libs, libblkid, libcom_err, libintl, libmnl, libnftnl, libtirpc, libtirpc-conf, libuuid, libverto, lzo, openssl, pigz, shadow-uidmap, xfsprogs, xz, xz-libs, zfs, zfs-libs, zstd-libs dind (shell script) docker-entrypoint.sh Dockerfile Commands to run if all Docker-dind resources are present `sudo docker build --network=host -t docker:dind .` `sudo docker build --network=host -t docker:dind-20.10 .` Configure a Gitlab Runner to use the Docker daemon provided by Docker-dind Modify the storage-driver setting for the host Docker daemon `sudo vim /etc/docker/daemon.json` { "storage-driver": "overlay2" } `sudo systemctl restart docker` Create a custom Docker network `sudo docker network create gitlab-runner-net` Instantiate a Docker-dind container to run the docker-dind backend Docker service NOTE: The Docker-dind container is added to the gitlab-runner-net custom Docker network `sudo docker run -d --name gitlab-dind --privileged --restart always --network gitlab-runner-net -v /var/lib/docker docker:20.10-dind --storage-driver=overlay2` Instantiate a Gitlab-runner container. This container will be used to create Gitlab runners connected to Gitlab. BEFORE this can be done, a Docker image with gitlab-runner must be built first using a Dockerfile and the gitlab-runner package. In this case the image is built using CentOS 7 and the gitlab-runner rpm package. NOTE: It is important that this container has access to the package-manager repository in case there are extra dependencies to be installed for gitlab-runner. `sudo docker build --network=host -t gitlab-runner:centos .` THEN, the gitlab-runner container can be instantiated NOTE: The gitlab-runner container is added to the gitlab-runner-net custon Docker network, same as that of the docker-dind container. Because all Gitlab runners are instantiated via this container, this is what will allow any runner instantiated by this container to have access to the Docker daemon provided by the Docker-dind container. NOTE: The host VM gitlab-runner configuration file is mounted inside of the gitlab-runner container. `sudo docker run -d --name gitlab-runner --restart always --network gitlab-runner-net -v /etc/gitlab-runner/config.toml:/etc/gitlab-runner/config.toml -e DOCKER_HOST=tcp://gitlab-dind:2375 gitlab-runner:centos` Now, a Gitlab-runner can be registered NOTE: Gitlab-runner url and token must be retrieved from your local Gitlab instance> NOTE: This runner should be registered as a shared runner if using one runner for all projects; otherwise, a new runner must be registered per project Use the Gitlab-runner container to register a new Gitlab runner `sudo docker run --network=host -it --rm -v /etc/gitlab-runner/config.toml:/etc/gitlab-runner/config.toml -v /etc/pki/tls/certs:/etc/pki/tls/certs -v /etc/pki/tls/private:/etc/pki/tls/private gitlab-runner:centos gitlab-runner register --executor docker --docker-image [host]:[port]/docker:20.10 --docker-volumes /var/run/docker.sock:/var/run/docker.sock` Modify the Gitlab-runner in the gitlab-runner config file to use the same network as the host `sudo vim /etc/gitlab-runner/config.toml` add 'network_mode = "host"' Enable this runner for your Gitlab projects. In this particular case, the runner would be enabled for the java-docker-image Gitlab project and the NeXtMidas Gitlab project Gitlab UI -> admin area -> runners -> [your runner] Add the runner to your Gitlab project(s)