Installing an NGINX Ingress controller with a Yandex Certificate Manager certificate

Manage the TLS certificate for the NGINX Ingress controller via Yandex Certificate Manager.

The External Secrets Operator syncs the certificate with the Kubernetes secret. This allows you to manage the deployed application's certificate through Certificate Manager by adding a self-signed certificate and updating it on your own or by issuing an automatically renewable Let's Encrypt® certificate.

Getting startedGetting started

1. If you do not have the Yandex Cloud command line interface yet, install and initialize it.

   The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name or --folder-id parameter.

2. Install the Helm package manager.

3. Install the jq utility:

   sudo apt update && sudo apt install jq
   

4. Create service accounts:

   • eso-service-account for interaction between the External Secrets Operator and Certificate Manager.
   • k8s-sa with the editor, container-registry.images.puller, and load-balancer.admin roles for the folder to create Managed Service for Kubernetes cluster resources and pull Docker images. The load-balancer.admin role is required to create a network load balancer.

5. Create an authorized key for the service account and save it to a file named authorized-key.json:

   yc iam key create \
     --service-account-name eso-service-account \
     --output authorized-key.json
   

6. Create security groups for the Managed Service for Kubernetes cluster and its node groups.

   Warning

   The configuration of security groups determines the performance and availability of the cluster and the services and applications running in it.

7. Create a Managed Service for Kubernetes cluster and a node group in any suitable configuration. In the Managed Service for Kubernetes cluster settings, specify the k8s-sa service account and the security groups prepared earlier.

8. Install kubectl and configure it to work with the created cluster.

Required paid resourcesRequired paid resources

The infrastructure support cost includes:

• Fee for using the Managed Service for Kubernetes master and outgoing traffic (see Managed Service for Kubernetes pricing).
• Fee for using Managed Service for Kubernetes cluster nodes (see Yandex Compute Cloud pricing).
• Fee for using public IPs (see Yandex Virtual Private Cloud pricing).
• Fee for incoming traffic (processed by the load balancer) and using a network load balancer (see Yandex Network Load Balancer pricing).

Add a certificate to Certificate ManagerAdd a certificate to Certificate Manager

1. Issue a Let's Encrypt® certificate and add it to Certificate Manager or upload your own certificate.

2. For a Let's Encrypt® certificate, have your rights checked for the domain specified in the certificate.

3. Assign the certificate-manager.certificates.downloader role to the service account named eso-service-account to enable it to read the content of the certificate:

   yc cm certificate add-access-binding \
     --id <certificate_ID> \
     --service-account-name eso-service-account \
     --role certificate-manager.certificates.downloader
   

4. Check that the rights have been granted:

   yc cm certificate list-access-bindings --id <certificate_ID>
   

   Result:

   +---------------------------------------------+----------------+-------------------------------------+
   |                   ROLE ID                   |  SUBJECT TYPE  |              SUBJECT ID             |
   +---------------------------------------------+----------------+-------------------------------------+
   | certificate-manager.certificates.downloader | serviceAccount | <service_account_ID> |
   +---------------------------------------------+----------------+-------------------------------------+
   

Install the External Secrets OperatorInstall the External Secrets Operator

1. Add a Helm repository named external-secrets:

   helm repo add external-secrets https://charts.external-secrets.io
   

2. Install the External Secrets Operator in the Managed Service for Kubernetes cluster:

   helm install external-secrets \
     external-secrets/external-secrets \
     --namespace external-secrets \
     --create-namespace
   

   This command creates a new namespace named external-secrets required for the External Secrets Operator.

   Result:

   NAME: external-secrets
   LAST DEPLOYED: Sun Sep 19 11:20:58 2021
   NAMESPACE: external-secrets
   STATUS: deployed
   REVISION: 1
   TEST SUITE: None
   NOTES:
   external-secrets has been deployed successfully!
   ...
   

Configure the Managed Service for Kubernetes clusterConfigure the Managed Service for Kubernetes cluster

1. Create a namespace named ns for External Secrets Operator objects:

   kubectl create namespace ns
   

2. Create a secret named yc-auth containing the key of the service account named eso-service-account:

   kubectl --namespace ns create secret generic yc-auth \
     --from-file=authorized-key=authorized-key.json
   

3. Create a secret storage (SecretStore) named secret-store containing the yc-auth secret:

   kubectl --namespace ns apply -f - <<< '
   apiVersion: external-secrets.io/v1beta1
   kind: SecretStore
   metadata:
     name: secret-store
   spec:
     provider:
       yandexcertificatemanager:
         auth:
           authorizedKeySecretRef:
             name: yc-auth
             key: authorized-key'
   

Create an ExternalSecretCreate an ExternalSecret

1. Create an ExternalSecret object named external-secret referring to a certificate from Certificate Manager:

   kubectl --namespace ns apply -f - <<< '
   apiVersion: external-secrets.io/v1beta1
   kind: ExternalSecret
   metadata:
     name: external-secret
   spec:
     refreshInterval: 1h
     secretStoreRef:
       name: secret-store
       kind: SecretStore
     target:
       name: k8s-secret
       template:
         type: kubernetes.io/tls
     data:
     - secretKey: tls.crt
       remoteRef:
         key: <certificate_ID>
         property: chain
     - secretKey: tls.key
       remoteRef:
         key: <certificate_ID>
         property: privateKey'
   

   Where:

   • k8s-secret: Name of the secret the External Secret Operator will place the certificate it gets from Certificate Manager into.
   • tls.crt: Parameter of the k8s-secret secret to contain the certificate.
   • tls.key: Parameter of the k8s-secret secret to contain the certificate's private key.

   The following are the possible values for property:

   • chain: Get the certificate chain in PEM format.
   • privateKey: Get the private key in PEM format.
   • chainAndPrivateKey or null value: Get both the certificate chain and the private key.

   The External Secrets Operator will get a certificate from Certificate Manager and place it into k8s-secret.

2. Check that the certificate ends up in k8s-secret:

   kubectl -n ns get secret k8s-secret -ojson \
     | jq '."data"."tls.crt"' -r \
     | base64 --decode
   

   Result example:

   -----BEGIN CERTIFICATE-----
   MIIFKTCCBBGgAwIBAgISBAlQtxTUnXa75N1TnPYRWbSLMA0GCSqGSIb3DQEBCwUA
   MDIxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQswCQYDVQQD
   EwJSMzAeFw0yMjA3MTMxNDMxNTVaFw0yMjEwMTExNDMxNTRaMB0xGzAZBgNVBAMT
   EmRkb3Mtd2ViLm5yay5tZS51azCCASIwDQYJKoZIhvcNAQEBBQADggEPADCC…
   

   To view the certificate in a user-friendly format, run the following commands:

   kubectl -n ns get secret k8s-secret -ojson | jq '."data"."tls.crt"' -r \
     | base64 --decode > cert.pem
   

   openssl x509 -in cert.pem -text
   

   Result example:

   Certificate:
       Data:
           Version: 3 (0x2)
           Serial Number:
               04:09:50:b7:14:d4:9d:76:bb:e4:dd:53:9c:f6:11:59:b4:8b
           Signature Algorithm: sha256WithRSAEncryption
           Issuer: C = US, O = Let's Encrypt, CN = R3
           Validity
               Not Before: Jul 13 14:31:55 2022 GMT
               Not After : Oct 11 14:31:54 2022 GMT
           Subject: CN = example.com
   ...
   

Install the NGINX Ingress controllerInstall the NGINX Ingress controller

For specific port forwarding at NGINX Ingress controller installation, follow this guide.

Create a web resource in your Managed Service for Kubernetes clusterCreate a web resource in your Managed Service for Kubernetes cluster

Create a Deployment object with NGINX and its corresponding service:

kubectl --namespace ns apply -f - <<< '
apiVersion: v1
kind: Service
metadata:
  name: app
spec:
  selector:
    app: app
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: app-deployment
  labels:
    app: app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: app
  template:
    metadata:
      labels:
        app: app
    spec:
      containers:
      - name: app
        image: nginx:latest
        ports:
        - containerPort: 80'

Configure a DNS record for the Ingress controllerConfigure a DNS record for the Ingress controller

1. Find out the IP address of the Ingress controller (the value in the EXTERNAL-IP column):

   kubectl get svc
   

   Result:

   NAME                      TYPE          CLUSTER-IP     EXTERNAL-IP     PORT(S)                     AGE
   ...
   ingress-nginx-controller  LoadBalancer  10.96.164.252  84.201.153.122  80:31248/TCP,443:31151/TCP  2m19s
   ...
   

2. Add an A record pointing to the Ingress controller's public IP to your DNS provider or to your own DNS server:

   <domain_name> IN A 84.201.153.122
   

Create an Ingress resourceCreate an Ingress resource

Create an Ingress resource that uses the k8s-secret certificate for HTTPS:

kubectl --namespace ns apply -f - <<< '
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-test
  namespace: ns
spec:
  tls:
    - hosts:
      - <domain_name>
      secretName: k8s-secret
  ingressClassName: nginx
  rules:
    - host: <domain_name>
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: app
              port:
                number: 80'

Where <domain_name> is the name of the domain the certificate is issued for.

Check resource availabilityCheck resource availability

Issue a GET request to the resource via HTTPS, for example, by this command:

curl https://<your_domain> -vv

Result example:

* Trying 51.250.64.86:443...
* Connected to <domain_name> (51.250.64.86) port 443 (#0)
...
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=<domain_name>
*  start date: Jul 13 14:31:55 2022 GMT
*  expire date: Oct 11 14:31:54 2022 GMT
*  subjectAltName: host "<domain_name>" matched cert's "<domain_name>"
...
* SSL certificate verify ok.

The Let's Encrypt® certificate must update automatically after the certificate update in Certificate Manager.

You can specify a sync timeout in the ExternalSecret object's refreshInterval parameter.

Delete the resources you createdDelete the resources you created

Some resources are not free of charge. To avoid paying for them, delete the resources you no longer need:

1. Delete the Managed Service for Kubernetes cluster.
2. Delete Network Load Balancer.
3. Delete the certificate.
4. If static public IP addresses were used for Managed Service for Kubernetes cluster and node access, release and delete them.