Cleaning up full instance disk space

A GitLab instance can run out of disk space. This is indicated by the 500 Internal Server Error HTTP status code. In this case, you will not be able to sign in to the GitLab instance.

You can increase the instance disk space on your own.

To prevent disk space from filling up in the future:

• Regularly clean up the instance disk space manually.
• Periodically delete irrelevant pipelines.
• Configure disk space cleanup policies.

You can also set up alerts for the Data disk chart in Monitoring.

Clean up the instance disk spaceClean up the instance disk space

1. Manually review all Docker images and their tags in GitLab Container Registry.
2. Delete obsolete images and tags.

Delete obsolete pipelinesDelete obsolete pipelines

Pipelines create jobs, which in turn generate logs. The longer and more actively you use a GitLab instance, the more disk space you need to store logs. To free up space, you can delete irrelevant pipelines, thus also deleting jobs with their logs.

In the GitLab web UI, you can only delete one pipeline at a time. To bulk delete irrelevant pipelines and their logs, use the following script:

#!/bin/sh
set -e

TOKEN=<your_token>
PER_PAGE=<number_of_pipelines_to_delete>
UPDATED_BEFORE=<date_of_last_update>
GITLAB_URL=<GitLab_instance_FQDN>

for PROJECT in $(curl -sL --header "PRIVATE-TOKEN: $TOKEN" "$GITLAB_URL/api/v4/projects?per_page=$PER_PAGE" | jq '.[].id') ; do
  echo "Deleting in project $PROJECT"
  for PIPELINE in $(curl -sL --header "PRIVATE-TOKEN: $TOKEN" "$GITLAB_URL/api/v4/projects/$PROJECT/pipelines?per_page=$PER_PAGE&sort=asc&updated_before=${UPDATED_BEFORE}T00:00:00Z" | jq '.[].id') ; do
      echo "Deleting pipeline $PIPELINE"
      curl -L --header "PRIVATE-TOKEN: $TOKEN" --request "DELETE" "$GITLAB_URL/api/v4/projects/$PROJECT/pipelines/$PIPELINE"
  done
done

Where:

• TOKEN: GitLab token of the user with the Owner role in the project.
• PER_PAGE: Number of pipelines to delete per request. You can delete a maximum of 100 pipelines at a time. If you have more, you will need to run the script several times.
• UPDATED_BEFORE: Upper limit of the update date in YYYY-MM-DD format. The script will delete all pipelines updated prior to this date.
• GITLAB_URL: GitLab instance FQDN.

Configure disk space cleanup policiesConfigure disk space cleanup policies

You can use any of the following options:

• Set the expiration time for build artifacts at the instance level. By default, it is 30 days.
• Create and enable a tag cleanup policy for individual projects using GitLab Container Registry.
• If you are using the Yandex Container Registry integration, set up a Docker image cleanup policy within Yandex Container Registry.