Tags
drupal
Other

Drupal 8 and GitLab Continuous Integration (GitLab CI)

Gitlab CI and Drupal 8

First of all I will explain how our development environment looks.

We have local development/staging server where we push our code.

On development server we puts content in place.

If somebody need content for development he can use local development mysql server. You do this with local computer mysql server through the SSH tunnels.

For version control and CI we use GitLab Community Edition v9.3.6. We git all files and export config files to /config.

The site is in /public_html and the vendor is in /vendor folder.

There are at least 3 branches:

  • master branch
  • dev branch and
  • production branch.

If needed we can make few feature branches of dev branch.

If we want to deploy to staging we make marge request to master branch from dev branch.

When we deploy to live we make marge request from master to production branch.

Gitlab runners are the default bash multi runner. In CI file, code is written with bash scripts.

We tried to simplify as much as we can. With this way it is easier to debug if something go wrong in deploying job. Here is the .gitlab-ci.yml file which we use:

variables:
  #staging config
  STAGING_NAME: "STAGINGUSERNAME"
  STAGING_HOST: "127.0.0.1"
  STAGING_PORT: "22"
  #Live server config
  USER: ""
  ## !!! if SUBDOMAIN destination folder have a prefix (example: domains/subomain.domain.com/) if it is main domain live blank !!!
  FOLDER_PREFIX: ""
  PRODUCTION_HOST: "127.0.0.1"
  PRODUCTION_PORT: "22"
  PRODUCTTION_URL: "http://productiondomain.com"

stages:
  - deploy

deploy_staging:
  stage: deploy
  script:
      - rsync -a --exclude '/sites/default/files' --delete -e "ssh -p $STAGING_PORT" $CI_PROJECT_DIR/public_html/ DevAndStagingUsername@$STAGING_HOST:domains/$STAGING_NAME.DevAndStagingUsername.rs/public_html
      - rsync -a --delete -e "ssh -p $STAGING_PORT" $CI_PROJECT_DIR/config/ DevAndStagingUsername@$STAGING_HOST:domains/$STAGING_NAME.DevAndStagingUsername.rs/config
      - ssh DevAndStagingUsername@$STAGING_HOST cp domains/$STAGING_NAME.DevAndStagingUsername.dev/composer.json domains/$STAGING_NAME.DevAndStagingUsername.rs/composer.json
      - ssh DevAndStagingUsername@$STAGING_HOST cp domains/$STAGING_NAME.DevAndStagingUsername.dev/composer.lock domains/$STAGING_NAME.DevAndStagingUsername.rs/composer.lock
      - ssh DevAndStagingUsername@$STAGING_HOST chown -R DevAndStagingUsername:DevAndStagingUsername domains/$STAGING_NAME.DevAndStagingUsername.rs/
      - ssh DevAndStagingUsername@$STAGING_HOST rsync -av --delete domains/$STAGING_NAME.DevAndStagingUsername.dev/public_html/sites/default/files/ domains/$STAGING_NAME.DevAndStagingUsername.rs/public_html/sites/default/files
      - rsync -a --delete -e "ssh -p $STAGING_PORT" $CI_PROJECT_DIR/scripts/ DevAndStagingUsername@$STAGING_HOST:domains/$STAGING_NAME.DevAndStagingUsername.rs/scripts
      - ssh DevAndStagingUsername@$STAGING_HOST rsync -a /home/DevAndStagingUsername/domains/$STAGING_NAME.DevAndStagingUsername.dev/public_html/sites/default/settings.php /home/DevAndStagingUsername/domains/$STAGING_NAME.DevAndStagingUsername.rs/public_html/sites/default/settings.php
      - ssh DevAndStagingUsername@$STAGING_HOST "cd domains/$STAGING_NAME.DevAndStagingUsername.rs/; composer install --no-dev"
      - ssh DevAndStagingUsername@$STAGING_HOST "if [ -f domains/$STAGING_NAME.DevAndStagingUsername.rs/public_html/sites/default/settings.local.php ]; then rm domains/$STAGING_NAME.DevAndStagingUsername.rs/public_html/sites/default/settings.local.php; fi"
      - ssh DevAndStagingUsername@$STAGING_HOST "cd domains/$STAGING_NAME.DevAndStagingUsername.dev/public_html; drush cr"
      - ssh DevAndStagingUsername@$STAGING_HOST "cd domains/$STAGING_NAME.DevAndStagingUsername.dev/public_html; drush sql-dump > MySqlDump-$STAGING_NAME.sql"
      - ssh DevAndStagingUsername@$STAGING_HOST mv domains/$STAGING_NAME.DevAndStagingUsername.dev/public_html/MySqlDump-$STAGING_NAME.sql /home/DevAndStagingUsername/domains/$STAGING_NAME.DevAndStagingUsername.rs/public_html/MySqlDump-$STAGING_NAME.sql
      - ssh DevAndStagingUsername@$STAGING_HOST "cd domains/$STAGING_NAME.DevAndStagingUsername.rs/public_html; drush sql-cli < MySqlDump-$STAGING_NAME.sql"
      - ssh DevAndStagingUsername@$STAGING_HOST rm domains/$STAGING_NAME.DevAndStagingUsername.rs/public_html/MySqlDump-$STAGING_NAME.sql
      - ssh DevAndStagingUsername@$STAGING_HOST "cd domains/$STAGING_NAME.DevAndStagingUsername.rs/public_html; drush updb -y"
      - ssh DevAndStagingUsername@$STAGING_HOST "cd domains/$STAGING_NAME.DevAndStagingUsername.rs/public_html; drush cim -y"
      - ssh DevAndStagingUsername@$STAGING_HOST "cd domains/$STAGING_NAME.DevAndStagingUsername.rs/public_html; drush cr"
  environment:
    name: staging
    url: http://$STAGING_NAME.DevAndStagingUsername.rs
  only:
    - master

deploy_production:
  stage: deploy
  script:
      - rsync -a --exclude '/sites/default/settings.php' --exclude '/sites/default/files' --delete -e "ssh -p $PRODUCTION_PORT" $CI_PROJECT_DIR/public_html/ $USER@$PRODUCTION_HOST:$FOLDER_PREFIX'public_html'
      - rsync -a --delete -e "ssh -p $PRODUCTION_PORT" $CI_PROJECT_DIR/config/ $USER@$PRODUCTION_HOST:$FOLDER_PREFIX'config'
      - rsync -a --delete -e "ssh -p $PRODUCTION_PORT" $CI_PROJECT_DIR/vendor/ $USER@$PRODUCTION_HOST:$FOLDER_PREFIX'vendor'
      - rsync -a --delete -e "ssh -p $PRODUCTION_PORT" $CI_PROJECT_DIR/scripts/ $USER@$PRODUCTION_HOST:$FOLDER_PREFIX'scripts'
      - ssh -p $PRODUCTION_PORT $USER@$PRODUCTION_HOST "if [ -f public_html/sites/default/settings.php ]; then echo SettignsPhpExist; else printf 'NODataBase\n' > $FOLDER_PREFIX'NODataBase'; mv .ssh/settings.php $FOLDER_PREFIX'public_html/sites/default/settings.php'; fi"
      - if ssh -p $PRODUCTION_PORT $USER@$PRODUCTION_HOST "stat $FOLDER_PREFIX'NODataBase' > /dev/null 2>&1"; then rsync -a -e "ssh -p $PRODUCTION_PORT" $USER@$PRODUCTION_HOST:$FOLDER_PREFIX'NODataBase' $CI_PROJECT_DIR/NODataBase; fi
      - "if [ -f $CI_PROJECT_DIR/NODataBase ]; then scp $CI_PROJECT_DIR/NODataBase DevAndStagingUsername@$STAGING_HOST:domains/$STAGING_NAME.DevAndStagingUsername.rs/NODataBase; else echo NoNODataBaseFileInBuild; fi"
      ## files logic, copy files from staging servers
      - ssh DevAndStagingUsername@$STAGING_HOST "if [ -f domains/$STAGING_NAME.DevAndStagingUsername.rs/NODataBase ]; then cd domains/$STAGING_NAME.DevAndStagingUsername.rs/public_html/sites/default; tar -cf files.tar.gz files; else echo NoNODataBaseFileOnStaging; fi"
      - if [ -f $CI_PROJECT_DIR/NODataBase ]; then rsync -a -e ssh DevAndStagingUsername@$STAGING_HOST:domains/$STAGING_NAME.DevAndStagingUsername.rs/public_html/sites/default/files.tar.gz $CI_PROJECT_DIR/public_html/sites/default/files.tar.gz; fi
      - if [ -f $CI_PROJECT_DIR/NODataBase ]; then rsync -a -e "ssh -p $PRODUCTION_PORT" $CI_PROJECT_DIR/public_html/sites/default/files.tar.gz $USER@$PRODUCTION_HOST:$FOLDER_PREFIX'public_html/sites/default/files.tar.gz'; fi
      - ssh "-p $PRODUCTION_PORT" $USER@$PRODUCTION_HOST "if [ -f $FOLDER_PREFIX'public_html/sites/default/files.tar.gz' ]; then cd $FOLDER_PREFIX'public_html/sites/default'; tar -xvf files.tar.gz; rm files.tar.gz; fi"
      ## end of file logic
      - ssh DevAndStagingUsername@$STAGING_HOST "if [ -f domains/$STAGING_NAME.DevAndStagingUsername.rs/NODataBase ]; then cd domains/$STAGING_NAME.DevAndStagingUsername.rs/public_html; drush cr; fi"
      - ssh DevAndStagingUsername@$STAGING_HOST "if [ -f domains/$STAGING_NAME.DevAndStagingUsername.rs/NODataBase ]; then cd domains/$STAGING_NAME.DevAndStagingUsername.rs/public_html; drush sql-dump > MySqlDump-$STAGING_NAME.sql; fi"
      - if [ -f $CI_PROJECT_DIR/NODataBase ]; then rsync -a -e ssh DevAndStagingUsername@$STAGING_HOST:domains/$STAGING_NAME.DevAndStagingUsername.rs/public_html/MySqlDump-$STAGING_NAME.sql $CI_PROJECT_DIR/public_html/MySqlDump-$STAGING_NAME.sql; fi
      - if [ -f $CI_PROJECT_DIR/NODataBase ]; then rsync -a -e "ssh -p $PRODUCTION_PORT" $CI_PROJECT_DIR/public_html/MySqlDump-$STAGING_NAME.sql $USER@$PRODUCTION_HOST:$FOLDER_PREFIX'public_html/MySqlDump-'$STAGING_NAME'.sql'; fi
      - "if [ -f $CI_PROJECT_DIR/public_html/MySqlDump-$STAGING_NAME.sql ]; then rm $CI_PROJECT_DIR/public_html/MySqlDump-$STAGING_NAME.sql; fi"
      - ssh DevAndStagingUsername@$STAGING_HOST "if [ -f domains/$STAGING_NAME.DevAndStagingUsername.rs/NODataBase ]; then rm domains/$STAGING_NAME.DevAndStagingUsername.rs/NODataBase; fi"
      - ssh DevAndStagingUsername@$STAGING_HOST "if [ -f domains/$STAGING_NAME.DevAndStagingUsername.rs/public_html/MySqlDump-$STAGING_NAME.sql ]; then rm domains/$STAGING_NAME.DevAndStagingUsername.rs/public_html/MySqlDump-$STAGING_NAME.sql; fi"
      - ssh -p $PRODUCTION_PORT $USER@$PRODUCTION_HOST "if [ -f $FOLDER_PREFIX'NODataBase' ]; then echo FirstTimeRunPipelines; else cd $FOLDER_PREFIX'public_html'; drush updb -y; fi"
      - ssh -p $PRODUCTION_PORT $USER@$PRODUCTION_HOST "if [ -f $FOLDER_PREFIX'public_html/MySqlDump-$STAGING_NAME.sql' ]; then cd $FOLDER_PREFIX'public_html'; drush sql-cli < MySqlDump-$STAGING_NAME.sql; rm MySqlDump-$STAGING_NAME.sql; cd ..; rm NODataBase; else echo ExistingSiteNoDatabaseImportNeeded; fi"
      - ssh -p $PRODUCTION_PORT $USER@$PRODUCTION_HOST "cd $FOLDER_PREFIX'public_html'; drush cim -y"
      - ssh -p $PRODUCTION_PORT $USER@$PRODUCTION_HOST "cd $FOLDER_PREFIX'public_html'; drush cr"
  environment:
    name: production
    url: $PRODUCTION_URL
  only:
    - production

This .gitlab-ci.yml isn't good for everybody but maybe some part of the file can be useful for you. You can also make your .gitlab-ci.yml.

I will explain all of the lines in my next post.

Share

Share icon

Hashtags

Studio Present’s Contributions to the Drupal Community

Latest Posts

Contributors