Automatically testing graphical web applications in an continous integration environment can be quite troublesome in certain cases.

Cypress is great for creating fast and reliable end to end tests for your browser application. But it can be hard to automate it in continous integration environments like GitLab. And it is getting harder if you need to pull in the API your web application runs against. Sure you could run your tests against the real world API, but that could cause tons of other issues.

In this example we are going to setup GitLab to pull out an simple express-api, install it's dependencies, start it and then run our end to end cypress tests against it. As base docker image we are going to use the official cypress one, as it comes with node and yarn pre-installed and we enable some caching to speed up the process on multiple test runs. The test application in this example is using Vue.js, that's why we are also installing the vue.cli during the setup. But the steps are the same for any web application you want to test with cypress.io

stages:
  - test

variables:
  CYPRESS_CACHE_FOLDER: "$CI_PROJECT_DIR/cache/Cypress"

cache:
  key: ${CI_COMMIT_REF_SLUG}
  paths:
    - .yarn
    - cache/Cypress
    - node_modules

test:
  image: cypress/base:14.0.0
  stage: test

  script:
    # Remove the existing api folder (just for multi-runs)
    - rm -rf express-api
    # Clone the API repo, you can use CI_JOB_TOKEN to access private repos.
    - git clone https://gitlab-ci-token:${CI_JOB_TOKEN}@gitlab.com/bytee/sample-express.git
    # Install the dependencies and configure the API
    - cd sample-express
    - npm i
    - cp config.sample.js config.js
    # Finally start the API server
    - npm start &
    - cd ..
    # Install dependencies you may need for your app
    - apt-get update && apt-get install -y libxss-dev

    # Checkout dependencies your app uses, in this case i am testing a vue app
    - yarn global add @vue/cli
    # Configure caching and install cypress
    - yarn config set cache-folder .yarn
    - yarn install
    - $(yarn bin)/cypress cache path
    - $(yarn bin)/cypress verify
    - $(yarn bin)/cypress install

    # Just to make sure the npm server is started
    - sleep 3

    # Run the tests
    - yarn test:e2e --headless

As i am testing a Vue.js application i am using test:e2e to run the tests. In my package.json it looks like the following:

"test:e2e": "vue-cli-service test:e2e --mode development",

As the URL and port settings are different for development mode than for production, we need to set the test mode to development.

Storing artificats

Storing artifcats with GitLab is quite easy, just extend your .gitlab-ci.yml with the paths and when the artificats are supposed to expire.

Sample Gitlab artificat configuration for Cypress screenshots:

  artifacts:
    paths:
      - tests/e2e/screenshots
    expire_in: 2 days
    when: always

Screenshots are always stored in GitLab and expire after two days. They are actually only created on error or when you do one manually in cypress