Integration with Argo CD

Argo CD is a declarative GitOps tool for continuous delivery to Kubernetes.

This tutorial describes how to set up integration between a Yandex Managed Service for GitLab instance and a Managed Service for Kubernetes cluster running Argo CD and GitLab Runner to build Docker containers with Kaniko.

To integrate Argo CD with Managed Service for Kubernetes and Managed Service for GitLab:

1. Create a GitLab instance.
2. Configure GitLab.
3. Create a GitLab Runner.
4. Set up an application repository for deployment.
5. Deploy your application using Argo CD.

If you no longer need the resources you created, delete them.

Required paid resourcesRequired paid resources

The support cost for this solution includes:

• Fee for using the master and outgoing traffic in a Managed Service for Kubernetes cluster (see Managed Service for Kubernetes pricing).

• Fee for using computing resources, OS, and storage in cluster nodes (VMs) (see Compute Cloud pricing).

• Fee for public IP addresses assigned to cluster nodes (see Virtual Private Cloud pricing).

• Fee for a NAT gateway when used instead of public IP addresses for cluster nodes (see Virtual Private Cloud pricing).

• Fee for Container Registry storage.

• GitLab instance fee that depends on the instance creation method:

  • Managed Service for GitLab: You pay for VM resources, data and backup storage, and the amount of outgoing traffic (see Managed Service for GitLab pricing).
  • VM with the GitLab image: You pay for VM resources, GitLab image, VM public IP address (see Compute Cloud and Virtual Private Cloud pricing).

Getting startedGetting started

Set up your infrastructureSet up your infrastructure

Install additional dependenciesInstall additional dependencies

Install the following tools in the local environment:

• Yandex Cloud command line interface (YC CLI)
• jq
• Helm

• Install kubect and configure it to work with the new cluster.

Create a GitLab instanceCreate a GitLab instance

Create either a Managed Service for GitLab instance or a VM with a GitLab image in the same cloud network as the Managed Service for Kubernetes cluster.

Configure GitLabConfigure GitLab

To configure GitLab and enable Continuous Integration (CI), create a new project and enter the CI authorization parameters:

Create a GitLab RunnerCreate a GitLab Runner

To run build tasks in the Yandex Managed Service for Kubernetes cluster, create a GitLab Runner. To do this, install GitLab Runner by following this guide.

Once it is installed, you can run automated builds inside your Managed Service for Kubernetes cluster.

For more information about installing and running GitLab Runner, see the GitLab documentation.

Set up an application repository for deploymentSet up an application repository for deployment

1. Get an authorized key for the previously created service account with the container-registry.images.puller and container-registry.images.pusher roles:

   yc iam key create --service-account-name <name_of_service_account_for_nodes> -o key.json
   

2. Save the contents of the key to use it later:

   cat key.json | base64
   

3. Create GitLab environment variables:

   1. In GitLab, navigate to Settings in the left-hand panel and select CI/CD from the drop-down list.

   2. Click Expand next to Variables.

   3. Add these environment variables:

      • CI_REGISTRY: Address of the previously created registry in cr.yandex/<registry_ID> format.
      • CI_REGISTRY_USER: json_key.
      • CI_REGISTRY_PASSWORD: Output of cat key.json | base64.

      To add a variable:

      1. Click Add variable.
      2. In the window that opens, specify the variable name in the Key field and its value in the Value field.
      3. Click Add variable.

4. Set up access to the repository:

   1. Generate a new pair of SSH keys or use an existing one.
   2. Add a public part of the SSH key to the GitLab account settings.

5. Clone the repository:

   git clone git@<instance_name>.gitlab.yandexcloud.net:<admin_username>/gitlab-test.git
   

6. Clone the yc-webinar-gitops-argo-crossplane repository to your computer:

   git clone https://github.com/yandex-cloud-examples/yc-webinar-gitops-argo-crossplane.git
   

7. Copy all yc-webinar-gitops-argo-crossplane/02-argocd/app directory files, including hidden ones, to the gitlab-test directory:

   cp -rT <path_to_app_directory> <path_to_gitlab-test_directory>
   

8. Commit the changes to gitlab-test and push them to the repository:

   git add . && \
   git commit -m "Add app src and CI" && \
   git push
   

9. This will run the build script. To view its progress, in the left-hand panel in GitLab, select Build → Pipelines. Wait for both build steps to complete.

10. Open the completed build and copy the following line from the log (you will need it at the next step):

    INFO[0025] Pushing image to cr.yandex/<registry_ID>/<admin_username>/gitlab-test:main.<commit_number>
    

Deploy your application using Argo CDDeploy your application using Argo CD

Install Argo CD in the Managed Service for Kubernetes clusterInstall Argo CD in the Managed Service for Kubernetes cluster

1. Install Argo CD by following this guide.

   Warning

   Kubernetes node groups require internet access to download images and components.

   Internet access can be provided through:

   • By assigning a public IP address to each node in the group.
   • Configuring a VM as a NAT instance.
   • Setting up a NAT gateway.

2. Get the admin password from the Kubernetes secret:

   kubectl --namespace <namespace> get secret argocd-initial-admin-secret \
     --output jsonpath="{.data.password}" | base64 -d
   

3. Set up ArgoCD port forwarding to your local computer:

   kubectl port-forward service/<Argo_CD_app_name>-argocd-server \
     --namespace <namespace> 8080:443
   

4. Open the Argo CD console at https://127.0.0.1:8080 in your browser.

5. Log in to the console as admin using the password you got earlier.

Add the GitLab repository to Argo CDAdd the GitLab repository to Argo CD

1. In GitLab, navigate to Settings in the left-hand panel and select Access Tokens from the drop-down list.

2. Create a new token with the following settings:

   • Token name: argocd.
   • Select a role: Maintainer.
   • Select scopes: read_repository.

3. Click Create project access token.

4. Copy the token value.

5. In the Argo CD console, go to Settings → Repositories.

6. Click Connect Repo and select VIA HTTPS from the list.

   Note

   If you get the FATA[0000] rpc error: code = Unknown desc = error testing repository connectivity: authorization failed error when connecting a repository, enable access to GitLab via HTTP(S).

   To enable access, in the left-hand panel in GitLab, select Admin → Settings → General. Under Visibility and access controls, find the Enabled Git access protocols setting and select the option that allows HTTP(S) access from the list.

   For more information, see this GitLab guide.

7. In the form that opens, specify the following settings:

   • Repository URL: Repository URL in https://<GitLab_instance_name>.gitlab.yandexcloud.net/<admin_username>/gitlab-test.git format.
   • Username: gitlab-ci-token.
   • Password: Previously generated GitLab token.

8. Click Connect.

9. In the Argo CD console, go to Applications and click Create Application.

10. In the form that opens, specify the following settings:

    • Application Name: gitlab-test.
    • Project: default.
    • Sync policy: Automatic, then select Prune resources and Self Heal.
    • Sync options: Select Auto-Create Namespace.
    • Repository URL: Specify the repository URL in https://<GitLab_instance_name>.gitlab.yandexcloud.net/<admin_username>/gitlab-test.git format.
    • Path: .helm.
    • Cluster URL: https://kubernetes.default.svc.
    • Namespace: gitlab-test.
    • image.repository: cr.yandex/<registry_ID>/<admin_username>/gitlab-test.
    • image.tag: main.<commit_number>.

11. Click Create and wait for the synchronization to complete.

12. To check that the application is running, use the following command in the Managed Service for Kubernetes cluster:

    kubectl get all -n gitlab-test
    

    Result:

    NAME                               READY   STATUS    RESTARTS   AGE
    pod/gitlab-test-67c8d58bc4-6w4q7   1/1     Running   0          2m26s
    pod/gitlab-test-67c8d58bc4-sldpc   1/1     Running   0          2m26s
    
    NAME                  TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
    service/gitlab-test   ClusterIP   10.96.186.223   <none>        80/TCP    2m26s
    
    NAME                          READY   UP-TO-DATE   AVAILABLE   AGE
    deployment.apps/gitlab-test   2/2     2            2           2m26s
    
    NAME                                     DESIRED   CURRENT   READY   AGE
    replicaset.apps/gitlab-test-67c8d58bc4   2         2         2       2m26s
    

Test auto-syncing from the repositoryTest auto-syncing from the repository

1. Go to the directory with the cloned project and open the .helm/values.yaml file.

2. Set replicaCount to 3.

3. Save the changes and push them to the repository:

   git add . && \
   git commit -m "Increase replica count" && \
   git push
   

4. In the Argo CD console, wait for the app to synchronize.

5. Make sure the number of application pods in the Managed Service for Kubernetes cluster has increased:

   kubectl get pod -n gitlab-test
   

   Result:

   NAME                               READY   STATUS    RESTARTS   AGE
   pod/gitlab-test-67c8d58bc4-6w4q7   1/1     Running   0          15m
   pod/gitlab-test-67c8d58bc4-7hmcn   1/1     Running   0          10m
   pod/gitlab-test-67c8d58bc4-sldpc   1/1     Running   0          15m
   
   NAME                  TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
   service/gitlab-test   ClusterIP   10.96.186.223   <none>        80/TCP    15m
   
   NAME                          READY   UP-TO-DATE   AVAILABLE   AGE
   deployment.apps/gitlab-test   3/3     3            3           15m
   
   NAME                                     DESIRED   CURRENT   READY   AGE
   replicaset.apps/gitlab-test-67c8d58bc4   3         3         3       15m
   

Delete the resources you createdDelete the resources you created

Some resources are not free of charge. Delete the resources you no longer need to avoid paying for them:

1. Delete the created Docker images.

2. Delete the Managed Service for Kubernetes cluster and Container Registry:

   Manually

   Terraform

   1. Delete the Managed Service for Kubernetes cluster.
   2. Delete the Container Registry.
   3. Delete the created subnets and networks.
   4. Delete the created service accounts.

   1. In the terminal window, go to the directory containing the infrastructure plan.

      Warning

      Make sure the directory has no Terraform manifests with the resources you want to keep. Terraform deletes all resources that were created using the manifests in the current directory.

   2. Delete resources:

      1. Run this command:

         terraform destroy
         

      2. Confirm deleting the resources and wait for the operation to complete.

      All the resources described in the Terraform manifests will be deleted.

3. Delete the GitLab VM or Managed Service for GitLab instance you created.