NeXtMidas Support for Continuous Integrations with GitLab

Topics covered in this page

Using GitLab CI to build and publish NeXtMidas
Prerequisites
Define Project-specific CI/CD variables from the Gitlab UI
Registering Shared Runners Versus Project-Specific Runners
NeXtMidas Gitlab CI Configuration file
How to build the Docker image needed by GitLab for the NeXtMidas build
Prerequisites
Define Project-specific CI/CD variables from the Gitlab UI
Additional Files
Extra steps if Standard Docker-in-Docker Images are Not Available
Build the Docker-in-Docker Docker image (client version) from scratch
Build the Docker-dind Docker image (backend version) from scratch
Configure a Gitlab Runner to use the Docker daemon provided by Docker-dind


Using GitLab CI to build and publish NeXtMidas

The process: Whenever a change is pushed to the NextMidas Gitlab project, a Gitlab job should be started for this project. Jobs can also be started manually. Inside Gitlab, the Docker-builder image will be pulled from the local private Docker registry to provide access to Gradle, OpenJDK, and Nexus. This project will run a Gradle build of NeXtMidas and publish all NextMidas Gradle jar files to the Nexus repo.


  1. Prerequisites
    • Access to a Docker registry
    • Access to the standard docker-dind Docker image (backend version)
    • Access to the CentOS 7 (or other desired OS) Docker image
    • Access to a Yum Repository Server
    • Gradle (If not provided in separate Docker-builder image)
    • Access to Nexus (NOTE: Only include gradle.properties, keystores, and truststores if not already provided in separate Docker-builder image)
      • gradle.properties should include any necessary Nexus url's, as well as necessary keystores and truststores
      • If using local Nexus:
        • Add your local rootCA to the trust store referenced by the gradle.properties file
        • Keystores and truststores
  2. Define Project-specific CI/CD variables from the Gitlab UI
    • From the Gitlab UI, navigate to the NeXtMidas project, then click on Settings -> CI/CD, then click on the Expand button next to Variables
    • Add three variables by clicking on Add Variable, then select type Variable and uncheck the box that says Protected
      • DOCKER_REG_ADDR - Address at which the local Docker registry resides
        • Example: nxm.example.com:12345
      • DOCKER_IMAGE_NAME - Name of the Docker image for the NeXtMidas build
        • Example: centos7_java11_gradle6
      • DOCKER_IMAGE_TAG - Tag of the Docker image for the NeXtMidas build
        • Examples: latest-builder, builder-22-05-06
  3. Registering Shared Runners Versus Project-Specific Runners
    • Shared Runner (NOTE: This can only be done if you have access to the Admin Area in the Gitlab UI)
      1. Gitlab UI -> Admin Area -> Runners
      2. Grab the URL and registration token provided in the top-right section of your screen
      3. Use the gitlab-runner cmd-line tool, along with the URL and token, to register your shared runner
        • NOTE: This can be done either on your local machine or using a Docker container (please refer to steps below for container-specific instructions)
      4. Gitlab UI -> Admin Area -> Runners -> [your shared runner]
      5. Select which projects this Gitlab runner should be enabled for
    • Project-Specific Runner (NOTE: This can only be done if you have access to the Settings in your Gitlab Project)
      1. Gitlab UI -> Projects -> [your project] -> Settings -> CI/CD, -> Runners
      2. Grab the URL and registration token provided on the left-hand side of your screen under Specific Runners
      3. Use the gitlab-runner cmd-line tool, along with the URL and token, to register your project-specific runner
        • NOTE: This can be done either on your local machine or using a Docker container (please refer to steps above for container instructions)
  4. NeXtMidas Gitlab CI Configuration file
    • .gitlab-ci.yml - Added to baseline in $NMROOT, but here as a reference - DOES NOT NEED TO BE MODIFIED

      # This is the Gitlab configuration file for NeXtMidas CI

      stages:
       - build

      before_script:
       - export COMMIT_TIME="$(date --date=${CI_COMMIT_TIMESTAMP} +%Y%m%d%H%M%S)"
       - export BUILD_TIME="$(date +%Y%m%d%H%M%S)"
       - export TAG_PREFIX=`awk 'BEGIN { printf("%s/%s:%s",tolower(ENVIRON["DOCKER_REGISTRY"]),

       tolower(ENVIRON["CI_PROJECT_NAME"]),
       tolower(ENVIRON["CI_COMMIT_BRANCH"])); }' /dev/null`

       - export IMAGE_TAG="${TAG_PREFIX}-${CI_COMMIT_SHORT_SHA}-${BUILD_TIME}"
       - export GRADLE_TARGETS="clean build"
       - export GRADLE_FLAGS="--refresh-dependencies -Dorg.gradle.internal.publish.checksums.insecure=true -x test"
       - if [[ "${CI_COMMIT_BRANCH}" = "ops" ]]; then export GRADLE_TARGETS="${GRADLE_TARGETS} publish"; fi

      variables:
       # Location of the Docker registry
       DOCKER_REGISTRY: ${DOCKER_REG_ADDR}

      build:
       # Grab the Docker image for the NeXtMidas build from the Docker Registry
       # The Docker Registry Address (DOCKER_REG_ADDR), Docker image name (DOCKER_IMAGE_NAME),
       # and Docker image tag (DOCKER_IMAGE_TAG) are specified as GitLab CI/CD project variables
       image: ${DOCKER_REG_ADDR}/${DOCKER_IMAGE_NAME}:${DOCKER_IMAGE_TAG}
       stage: build
       services:
        # Provides access to the Docker daemon inside the current NeXtMidas Gitlab job
        - ${DOCKER_REG_ADDR}/docker:dind
       script:
        # Update current Centos 7 environment
        - yum update -y

        # .repo file must be added back in after doing a yum update
        - rm /etc/yum.repos.d/*
        - cp /usr/local/centos-reposerver.repo /etc/yum.repos.d/

        # install gcc-c++
        - yum install -y gcc-c++

        # Convert NeXtMidas to a Gradle project using softlinks
        - export NMROOT=$(pwd)
        - gradle/linkgradle.sh

        # Print out environment configuration for the current NeXtMidas Gitlab job
        - echo "==== Config ====";
          echo "GITLAB_USER_NAME = ${GITLAB_USER_NAME}";
          echo "CI_COMMIT_MESSAGE = ${CI_COMMIT_MESSAGE}";
          echo "COMMIT_TIME = ${COMMIT_TIME}";
          echo "BUILD_TIME = ${BUILD_TIME}";
          echo "CI_BUILD_NAME = ${CI_BUILD_NAME}";
          echo "IMAGE_TAG = ${IMAGE_TAG}";
          echo "GRADLE_TARGETS = ${GRADLE_TARGETS}";
          echo "GRADLE_FLAGS = ${GRADLE_FLAGS}";

        # Run the NeXtMidas Gradle build using the Gradle env vars defined above
        - echo "==== Starting Gradle Build ===="
        - gradle --stop
        - sleep 1
        - find ${HOME}/.gradle -name "*.lock" | xargs rm
        - gradle --no-daemon ${GRADLE_TARGETS} ${GRADLE_FLAGS}
        - find -name '*.jar' | sort


How to build the Docker image needed by GitLab for the NeXtMidas build

  1. Prerequisites
    • Access to a Docker registry
    • Access to the docker-in-docker Docker image (client version)
    • Access to the CentOS 7 (or other desired OS) Docker image
    • Access to a Yum Repository Server
    • Gradle
    • Access to Nexus
      • gradle.properties should include any necessary Nexus url's, as well as necessary keystores and truststores
      • If using local Nexus:
        • add your local rootCA to the trust store referenced by the gradle.properties file
        • keystores and truststores
  2. Define Project-specific CI/CD variables from the Gitlab UI
    • From the Gitlab UI, navigate to the docker-image project, then click on Settings -> CI/CD, then click on the Expand button next to Variables
    • Add three variables by clicking on Add Variable, then select type Variable and uncheck the box that says Protected
      • DOCKER_REG_ADDR - Address at which the local Docker registry resides
        • Example: nxm.example.com:12345
      • DOCKER_IMAGE_NAME - Name of the Docker image for the NeXtMidas build
        • Example: centos7_java11_gradle6
      • DOCKER_IMAGE_TAG - Tag of the Docker image for the NeXtMidas build
        • Examples: latest-builder, builder-22-05-06
  3. Additional Files:
    • may need to install some dependencies
      • .gitlab-ci.yml - sample below and under $NMROOT/gradle/gitlab-ci/docker-image

        # This is the Gitlab configuration file for building the Docker-builder Docker image which will be used by the NeXtMidas Gitlab project

        stages:
         - build

        before_script:
          - docker info
          - export VERSION=$(date +%y-%m-%d)
          - export BUILDER_TAG_NAME=${IMAGE_NAME}":builder-$VERSION"
        variables:
          # Name of the Docker-builder Docker image to be pushed to the Docker registry
          IMAGE_NAME: "${DOCKER_REG_ADDR}/${DOCKER_IMAGE_NAME}"

        build-full:
          # Use the docker-in-docker client image as the base image for the current java-docker-image Gitlab job
          image: ${DOCKER_REG_ADDR}/docker:latest
          stage: build
          script:
           - echo $BUILDER_TAG_NAME

           # Build the Docker-builder Docker image to be used by NeXtMidas
           - docker build --network=host -f Dockerfile-builder -t $BUILDER_TAG_NAME .

           # Tag and push the Docker-build Docker image to be used by NeXtMidas
           - docker tag $BUILDER_TAG_NAME ${IMAGE_NAME}:"latest-builder"
           - docker push ${IMAGE_NAME}":latest-builder"
           - docker push $BUILDER_TAG_NAME
           - echo $BUILDER_TAG_NAME
          only:
           - master

      • Dockerfile-builder - sample below and under $NMROOT/gradle/gitlab-ci/docker-image

        # Start with a basic Centos 7 Docker image
        FROM [hostname]:[port]/centos:7

        # Update repo based on the current environent and conditions/constraints
        RUN rm /etc/yum.repos.d/*
        COPY centos-reposerver.repo /usr/local/
        RUN cp /usr/local/centos-reposerver.repo /etc/yum.repos.d/

        # Install Openjdk 11 and unzip tools
        RUN yum install java-11-openjdk-devel unzip -y

        ENV JAVA_HOME /usr/lib/jvm/java-11-openjdk
        ENV JAVAHOME ${JAVA_HOME}
        ENV PATH $PATH:${JAVA_HOME}/bin

        # Working directory for this image
        WORKDIR /opt

        # Install, configure, and run the Gradle tool
        COPY gradle-6.3-all.zip ./
        RUN jar -xf gradle-6.3-all.zip
        RUN rm gradle-6.3-all.zip
        RUN ln -sf /opt/gradle-6.3 /opt/gradle
        ENV GRADLE_BIN /opt/gradle/bin
        RUN chmod +x ${GRADLE_BIN}/gradle
        ENV PATH $PATH:${GRADLE_BIN}
        ENV JAVA_TOOL_OPTIONS -Dfile.encoding=UTF8
        RUN gradle

        # Copy gradle.properties file into this image
        COPY gradle.properties /root/.gradle/

        # Unzip SSL certificates to be used for Gradle
        COPY ssl.zip /usr/local/ssl.zip
        RUN unzip /usr/local/ssl.zip -d /usr/local/
        RUN rm /usr/local/ssl.zip

      • The process: Whenever a change is pushed to the java-docker-image Gitlab project, a Gitlab job should be started for this project. Inside Gitlab, the job will be using the Docker-in-Docker Docker image to run Docker. This project will build a Docker-builder image with CentOS 7, OpenJDK 11, and Gradle. This custom Docker-image will be added to a local private registry hosted by the local host environment to later be referenced by the NeXtMidas Gitlab project.

Extra steps if Standard Docker-in-Docker Images are Not Available
  1. Build the Docker-in-Docker Docker image (client version) from scratch
    • Prerequisites
      • Access to latest Alpine standard Docker image
    • Files generally needed (As of docker:20.10)
      • Alpine Dependencies (As of alpine:3.15, assuming no access to the Alpine public repository)
        • ca-certificates, libc6-compat, libedit, ncurses-libs, ncurses-terminfo-base, openssh-client-common, openssh-client-default, openssh-keygen
      • Docker-in-Docker base os tar file
      • docker-entrypoint.sh
      • modprobe.sh
      • Dockerfile
    • Commands to run if all Docker-in-Docker resources are present
      • sudo docker build --network=host -t docker:20.10 .
      • sudo docker build --network=host -t docker:latest .
  2. Build the Docker-dind Docker image (backend version) from scratch
    • Prerequisites
      • Access to latest standard Docker-in-Docker image
    • Files generally needed (As of docker:dind-20.10)
      • Alpine Dependencies (As of alpine:3.15, assuming no access to the Alpine public repository)
        • btrfs-progs, e2fsprogs, e2fsprogs-extra, e2fsprogs-libs, inih, ip6tables, iptables, keyutils-libs, krb5-conf, krb5-libs, libblkid, libcom_err, libintl, libmnl, libnftnl, libtirpc, libtirpc-conf, libuuid, libverto, lzo, openssl, pigz, shadow-uidmap, xfsprogs, xz, xz-libs, zfs, zfs-libs, zstd-libs
      • dind (shell script)
      • docker-entrypoint.sh
      • Dockerfile
    • Commands to run if all Docker-dind resources are present
      • sudo docker build --network=host -t docker:dind .
      • sudo docker build --network=host -t docker:dind-20.10 .
  3. Configure a Gitlab Runner to use the Docker daemon provided by Docker-dind
    1. Modify the storage-driver setting for the host Docker daemon
      • sudo vim /etc/docker/daemon.json
        {
         "storage-driver": "overlay2"
        }
        sudo systemctl restart docker

    2. Create a custom Docker network
      • sudo docker network create gitlab-runner-net
    3. Instantiate a Docker-dind container to run the docker-dind backend Docker service
      • NOTE: The Docker-dind container is added to the gitlab-runner-net custom Docker network
      • sudo docker run -d --name gitlab-dind --privileged --restart always --network gitlab-runner-net -v /var/lib/docker docker:20.10-dind --storage-driver=overlay2
    4. Instantiate a Gitlab-runner container. This container will be used to create Gitlab runners connected to Gitlab.
      • BEFORE this can be done, a Docker image with gitlab-runner must be built first using a Dockerfile and the gitlab-runner package. In this case the image is built using CentOS 7 and the gitlab-runner rpm package.
      • NOTE: It is important that this container has access to the package-manager repository in case there are extra dependencies to be installed for gitlab-runner.
      • sudo docker build --network=host -t gitlab-runner:centos .
      • THEN, the gitlab-runner container can be instantiated
      • NOTE: The gitlab-runner container is added to the gitlab-runner-net custon Docker network, same as that of the docker-dind container. Because all Gitlab runners are instantiated via this container, this is what will allow any runner instantiated by this container to have access to the Docker daemon provided by the Docker-dind container.
      • NOTE: The host VM gitlab-runner configuration file is mounted inside of the gitlab-runner container.
      • sudo docker run -d --name gitlab-runner --restart always --network gitlab-runner-net -v /etc/gitlab-runner/config.toml:/etc/gitlab-runner/config.toml -e DOCKER_HOST=tcp://gitlab-dind:2375 gitlab-runner:centos
    5. Now, a Gitlab-runner can be registered
      • NOTE: Gitlab-runner url and token must be retrieved from your local Gitlab instance>
      • NOTE: This runner should be registered as a shared runner if using one runner for all projects; otherwise, a new runner must be registered per project
        1. Use the Gitlab-runner container to register a new Gitlab runner
          • sudo docker run --network=host -it --rm -v /etc/gitlab-runner/config.toml:/etc/gitlab-runner/config.toml -v /etc/pki/tls/certs:/etc/pki/tls/certs -v /etc/pki/tls/private:/etc/pki/tls/private gitlab-runner:centos gitlab-runner register --executor docker --docker-image [host]:[port]/docker:20.10 --docker-volumes /var/run/docker.sock:/var/run/docker.sock
        2. Modify the Gitlab-runner in the gitlab-runner config file to use the same network as the host
          • sudo vim /etc/gitlab-runner/config.toml
          • add 'network_mode = "host"'
    6. Enable this runner for your Gitlab projects. In this particular case, the runner would be enabled for the java-docker-image Gitlab project and the NeXtMidas Gitlab project
      1. Gitlab UI -> admin area -> runners -> [your runner]
      2. Add the runner to your Gitlab project(s)