Installing an NGINX Ingress controller with a Yandex Certificate Manager certificate

Manage the TLS certificate for the NGINX Ingress controller via Yandex Certificate Manager.

The External Secrets Operator syncs the certificate with the Kubernetes secret. This allows you to manage the deployed application's certificate through Certificate Manager by adding a self-signed certificate and updating it on your own or by issuing an automatically renewable Let's Encrypt® certificate.

Getting startedGetting started

1. If you do not have the Yandex Cloud command line interface yet, install and initialize it.

   The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name or --folder-id parameter.

2. Install the Helm package manager.

3. Install the jq utility:

   sudo apt update && sudo apt install jq
   

4. Create service accounts:

   • eso-service-account for interaction between the External Secrets Operator and Certificate Manager.
   • k8s-sa with the editor, container-registry.images.puller, and load-balancer.admin roles for the folder to create Managed Service for Kubernetes cluster resources and pull Docker images. The load-balancer.admin role is required to create a network load balancer.

5. Create an authorized key for the service account and save it to the authorized-key.json file:

   yc iam key create \
     --service-account-name eso-service-account \
     --output authorized-key.json
   

6. Create security groups for the Managed Service for Kubernetes cluster and its node groups.

   Warning

   The configuration of security groups determines the performance and availability of the cluster and the services and applications running in it.

7. Create a Managed Service for Kubernetes cluster and a node group in any suitable configuration. In the Managed Service for Kubernetes cluster settings, specify the k8s-sa service account and the security groups prepared earlier.

8. Install kubectl and configure it to work with the created cluster.

Required paid resourcesRequired paid resources

The infrastructure support cost includes:

• Fee for using the Managed Service for Kubernetes master and outgoing traffic (see Managed Service for Kubernetes pricing).
• Fee for using Managed Service for Kubernetes cluster nodes (see Yandex Compute Cloud pricing).
• Fee for using public IPs (see Yandex Virtual Private Cloud pricing).
• Fee for incoming traffic (processed by the load balancer) and using a network load balancer (see Yandex Network Load Balancer pricing).

Add a certificate to Certificate ManagerAdd a certificate to Certificate Manager

1. Issue a Let's Encrypt® certificate and add it to Certificate Manager or upload your own certificate.

2. For a Let's Encrypt® certificate, have your rights checked for the domain specified in the certificate.

3. Grant the certificate-manager.certificates.downloader role to the eso-service-account service account to enable it to read the certificate contents:

   yc cm certificate add-access-binding \
     --id <certificate_ID> \
     --service-account-name eso-service-account \
     --role certificate-manager.certificates.downloader
   

4. Check that the rights have been granted:

   yc cm certificate list-access-bindings --id <certificate_ID>
   

   Result:

   +---------------------------------------------+----------------+-------------------------------------+
   |                   ROLE ID                   |  SUBJECT TYPE  |              SUBJECT ID             |
   +---------------------------------------------+----------------+-------------------------------------+
   | certificate-manager.certificates.downloader | serviceAccount | <service_account_ID> |
   +---------------------------------------------+----------------+-------------------------------------+
   

Install the External Secrets OperatorInstall the External Secrets Operator

1. Add a Helm repository named external-secrets:

   helm repo add external-secrets https://charts.external-secrets.io
   

2. Install the External Secrets Operator in the Managed Service for Kubernetes cluster:

   helm install external-secrets \
     external-secrets/external-secrets \
     --namespace external-secrets \
     --create-namespace
   

   This command creates a new external-secrets namespace required for using the External Secrets Operator.

   Result:

   NAME: external-secrets
   LAST DEPLOYED: Sun Sep 19 11:20:58 2021
   NAMESPACE: external-secrets
   STATUS: deployed
   REVISION: 1
   TEST SUITE: None
   NOTES:
   external-secrets has been deployed successfully!
   ...
   

Configure the Managed Service for Kubernetes clusterConfigure the Managed Service for Kubernetes cluster

1. Create a namespace called ns for External Secrets Operator objects:

   kubectl create namespace ns
   

2. Create a yc-auth secret with the eso-service-account key:

   kubectl --namespace ns create secret generic yc-auth \
     --from-file=authorized-key=authorized-key.json
   

3. Create a SecretStore called secret-store containing a secret named yc-auth:

   kubectl --namespace ns apply -f - <<< '
   apiVersion: external-secrets.io/v1beta1
   kind: SecretStore
   metadata:
     name: secret-store
   spec:
     provider:
       yandexcertificatemanager:
         auth:
           authorizedKeySecretRef:
             name: yc-auth
             key: authorized-key'
   

Create an ExternalSecretCreate an ExternalSecret

1. Create an ExternalSecret object named external-secret pointing to the certificate from Certificate Manager:

   kubectl --namespace ns apply -f - <<< '
   apiVersion: external-secrets.io/v1beta1
   kind: ExternalSecret
   metadata:
     name: external-secret
   spec:
     refreshInterval: 1h
     secretStoreRef:
       name: secret-store
       kind: SecretStore
     target:
       name: k8s-secret
       template:
         type: kubernetes.io/tls
     data:
     - secretKey: tls.crt
       remoteRef:
         key: <certificate_ID>
         property: chain
     - secretKey: tls.key
       remoteRef:
         key: <certificate_ID>
         property: privateKey'
   

   Where:

   • k8s-secret: Name of the secret to be used by the External Secret Operator to contain the certificate from Certificate Manager.
   • tls.crt: Parameter of the k8s-secret secret that will contain the certificate.
   • tls.key: Parameter of the k8s-secret secret that will contain the certificate's private key.

   The following are the legal values of the property parameter:

   • chain: Retrieve a chain of certificates in PEM format.
   • privateKey: Retrieve the private key in PEM format.
   • chainAndPrivateKey or null value: Retrieve both the certificate chain and the private key.

   The External Secrets Operator will get a certificate from Certificate Manager and place it in the k8s-secret secret.

2. Make sure the certificate was placed in the k8s-secret secret:

   kubectl -n ns get secret k8s-secret -ojson \
     | jq '."data"."tls.crt"' -r \
     | base64 --decode
   

   Result example:

   -----BEGIN CERTIFICATE-----
   MIIFKTCCBBGgAwIBAgISBAlQtxTUnXa75N1TnPYRWbSLMA0GCSqGSIb3DQEBCwUA
   MDIxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQswCQYDVQQD
   EwJSMzAeFw0yMjA3MTMxNDMxNTVaFw0yMjEwMTExNDMxNTRaMB0xGzAZBgNVBAMT
   EmRkb3Mtd2ViLm5yay5tZS51azCCASIwDQYJKoZIhvcNAQEBBQADggEPADCC…
   

   To view the certificate in a user-friendly format, run the following commands:

   kubectl -n ns get secret k8s-secret -ojson | jq '."data"."tls.crt"' -r \
     | base64 --decode > cert.pem
   

   openssl x509 -in cert.pem -text
   

   Result example:

   Certificate:
       Data:
           Version: 3 (0x2)
           Serial Number:
               04:09:50:b7:14:d4:9d:76:bb:e4:dd:53:9c:f6:11:59:b4:8b
           Signature Algorithm: sha256WithRSAEncryption
           Issuer: C = US, O = Let's Encrypt, CN = R3
           Validity
               Not Before: Jul 13 14:31:55 2022 GMT
               Not After : Oct 11 14:31:54 2022 GMT
           Subject: CN = example.com
   ...
   

Install the NGINX Ingress controllerInstall the NGINX Ingress controller

For specific port forwarding at NGINX Ingress controller installation, follow this guide.

Create a web resource in your Managed Service for Kubernetes clusterCreate a web resource in your Managed Service for Kubernetes cluster

Create a Deployment object with NGINX and its corresponding service:

kubectl --namespace ns apply -f - <<< '
apiVersion: v1
kind: Service
metadata:
  name: app
spec:
  selector:
    app: app
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: app-deployment
  labels:
    app: app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: app
  template:
    metadata:
      labels:
        app: app
    spec:
      containers:
      - name: app
        image: nginx:latest
        ports:
        - containerPort: 80'

Configure a DNS record for the Ingress controllerConfigure a DNS record for the Ingress controller

1. Find out the IP address of the Ingress controller (the value in the EXTERNAL-IP column):

   kubectl get svc
   

   Result:

   NAME                      TYPE          CLUSTER-IP     EXTERNAL-IP     PORT(S)                     AGE
   ...
   ingress-nginx-controller  LoadBalancer  10.96.164.252  84.201.153.122  80:31248/TCP,443:31151/TCP  2m19s
   ...
   

2. Add an A record pointing to the Ingress controller's public IP to your DNS provider or to your own DNS server:

   <domain_name> IN A 84.201.153.122
   

Create an Ingress resourceCreate an Ingress resource

Create an Ingress resource that uses the k8s-secret certificate for HTTPS:

kubectl --namespace ns apply -f - <<< '
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-test
  namespace: ns
spec:
  tls:
    - hosts:
      - <domain_name>
      secretName: k8s-secret
  ingressClassName: nginx
  rules:
    - host: <domain_name>
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: app
              port:
                number: 80'

Where <domain_name> is the name of the domain for which the certificate is issued.

Check resource availabilityCheck resource availability

Issue a GET request to the resource via HTTPS, for example, by this command:

curl https://<your_domain> -vv

Result example:

* Trying 51.250.64.86:443...
* Connected to <domain_name> (51.250.64.86) port 443 (#0)
...
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=<domain_name>
*  start date: Jul 13 14:31:55 2022 GMT
*  expire date: Oct 11 14:31:54 2022 GMT
*  subjectAltName: host "<domain_name>" matched cert's "<domain_name>"
...
* SSL certificate verify ok.

The Let's Encrypt® certificate must update automatically after the certificate update in Certificate Manager.

You can specify a sync timeout in the refreshInterval parameter of the ExternalSecret object.

Delete the resources you createdDelete the resources you created

Some resources are not free of charge. To avoid paying for them, delete the resources you no longer need:

1. Delete the Managed Service for Kubernetes cluster.
2. Delete Network Load Balancer.
3. Delete the certificate.
4. If static public IP addresses were used for Managed Service for Kubernetes cluster and node access, release and delete them.