Installing an NGINX Ingress controller with a Yandex Certificate Manager certificate
- Getting started
- Add a certificate to Certificate Manager
- Install the External Secrets Operator
- Configure the Managed Service for Kubernetes cluster
- Create an ExternalSecret
- Install the NGINX Ingress controller
- Create a web resource in your Managed Service for Kubernetes cluster
- Configure a DNS record for the Ingress controller
- Create an Ingress resource
- Check resource availability
- Delete the resources you created
Manage the TLS certificate for the NGINX Ingress controller via Yandex Certificate Manager.
The External Secrets Operator
Getting started
-
If you do not have the Yandex Cloud command line interface yet, install and initialize it.
The folder specified in the CLI profile is used by default. You can specify a different folder using the
--folder-name
or--folder-id
parameter. -
Install the
jq
utility:sudo apt update && sudo apt install jq
-
eso-service-account
for interaction between the External Secrets Operator and Certificate Manager.k8s-sa
with theeditor
,container-registry.images.puller
, andload-balancer.admin
roles for the folder to create Managed Service for Kubernetes cluster resources and pull Docker images. Theload-balancer.admin
role is required to create a network load balancer.
-
Create an authorized key for the service account and save it to the
authorized-key.json
file:yc iam key create \ --service-account-name eso-service-account \ --output authorized-key.json
-
Create security groups for the Managed Service for Kubernetes cluster and its node groups.
Warning
The configuration of security groups determines the performance and availability of the cluster and the services and applications running in it.
-
Create a Managed Service for Kubernetes cluster and a node group in any suitable configuration. In the Managed Service for Kubernetes cluster settings, specify the
k8s-sa
service account and the security groups prepared earlier. -
Install kubectl
and configure it to work with the created cluster.
Required paid resources
The infrastructure support cost includes:
- Fee for using the Managed Service for Kubernetes master and outgoing traffic (see Managed Service for Kubernetes pricing).
- Fee for using Managed Service for Kubernetes cluster nodes (see Yandex Compute Cloud pricing).
- Fee for using public IPs (see Yandex Virtual Private Cloud pricing).
- Fee for incoming traffic (processed by the load balancer) and using a network load balancer (see Yandex Network Load Balancer pricing).
Add a certificate to Certificate Manager
-
Issue a Let's Encrypt® certificate and add it to Certificate Manager or upload your own certificate.
-
For a Let's Encrypt® certificate, have your rights checked for the domain specified in the certificate.
-
Grant the
certificate-manager.certificates.downloader
role to theeso-service-account
service account to enable it to read the certificate contents:yc cm certificate add-access-binding \ --id <certificate_ID> \ --service-account-name eso-service-account \ --role certificate-manager.certificates.downloader
-
Check that the rights have been granted:
yc cm certificate list-access-bindings --id <certificate_ID>
Result:
+---------------------------------------------+----------------+-------------------------------------+ | ROLE ID | SUBJECT TYPE | SUBJECT ID | +---------------------------------------------+----------------+-------------------------------------+ | certificate-manager.certificates.downloader | serviceAccount | <service_account_ID> | +---------------------------------------------+----------------+-------------------------------------+
Install the External Secrets Operator
-
Add a Helm repository named
external-secrets
:helm repo add external-secrets https://charts.external-secrets.io
-
Install the External Secrets Operator in the Managed Service for Kubernetes cluster:
helm install external-secrets \ external-secrets/external-secrets \ --namespace external-secrets \ --create-namespace
This command creates a new
external-secrets
namespace required for using the External Secrets Operator.Result:
NAME: external-secrets LAST DEPLOYED: Sun Sep 19 11:20:58 2021 NAMESPACE: external-secrets STATUS: deployed REVISION: 1 TEST SUITE: None NOTES: external-secrets has been deployed successfully! ...
Configure the Managed Service for Kubernetes cluster
-
Create a namespace called
ns
for External Secrets Operator objects:kubectl create namespace ns
-
Create a
yc-auth
secret with theeso-service-account
key:kubectl --namespace ns create secret generic yc-auth \ --from-file=authorized-key=authorized-key.json
-
Create a SecretStore
calledsecret-store
containing a secret namedyc-auth
:kubectl --namespace ns apply -f - <<< ' apiVersion: external-secrets.io/v1beta1 kind: SecretStore metadata: name: secret-store spec: provider: yandexcertificatemanager: auth: authorizedKeySecretRef: name: yc-auth key: authorized-key'
Create an ExternalSecret
-
Create an ExternalSecret
object namedexternal-secret
pointing to the certificate from Certificate Manager:kubectl --namespace ns apply -f - <<< ' apiVersion: external-secrets.io/v1beta1 kind: ExternalSecret metadata: name: external-secret spec: refreshInterval: 1h secretStoreRef: name: secret-store kind: SecretStore target: name: k8s-secret template: type: kubernetes.io/tls data: - secretKey: tls.crt remoteRef: key: <certificate_ID> property: chain - secretKey: tls.key remoteRef: key: <certificate_ID> property: privateKey'
Where:
k8s-secret
: Name of the secret to be used by the External Secret Operator to contain the certificate from Certificate Manager.tls.crt
: Parameter of thek8s-secret
secret that will contain the certificate.tls.key
: Parameter of thek8s-secret
secret that will contain the certificate's private key.
The following are the legal values of the
property
parameter:chain
: Retrieve a chain of certificates in PEM format.privateKey
: Retrieve the private key in PEM format.chainAndPrivateKey
or null value: Retrieve both the certificate chain and the private key.
The External Secrets Operator will get a certificate from Certificate Manager and place it in the
k8s-secret
secret. -
Make sure the certificate was placed in the
k8s-secret
secret:kubectl -n ns get secret k8s-secret -ojson \ | jq '."data"."tls.crt"' -r \ | base64 --decode
Result example:
-----BEGIN CERTIFICATE----- MIIFKTCCBBGgAwIBAgISBAlQtxTUnXa75N1TnPYRWbSLMA0GCSqGSIb3DQEBCwUA MDIxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQswCQYDVQQD EwJSMzAeFw0yMjA3MTMxNDMxNTVaFw0yMjEwMTExNDMxNTRaMB0xGzAZBgNVBAMT EmRkb3Mtd2ViLm5yay5tZS51azCCASIwDQYJKoZIhvcNAQEBBQADggEPADCC…
To view the certificate in a user-friendly format, run the following commands:
kubectl -n ns get secret k8s-secret -ojson | jq '."data"."tls.crt"' -r \ | base64 --decode > cert.pem
openssl x509 -in cert.pem -text
Result example:
Certificate: Data: Version: 3 (0x2) Serial Number: 04:09:50:b7:14:d4:9d:76:bb:e4:dd:53:9c:f6:11:59:b4:8b Signature Algorithm: sha256WithRSAEncryption Issuer: C = US, O = Let's Encrypt, CN = R3 Validity Not Before: Jul 13 14:31:55 2022 GMT Not After : Oct 11 14:31:54 2022 GMT Subject: CN = example.com ...
Install the NGINX Ingress controller
Install the Ingress NGINX application from Cloud Marketplace using this guide.
-
Add the following to the Helm repository for NGINX:
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
Result:
"ingress-nginx" has been added to your repositories
-
Update the dataset to create an application instance in the Managed Service for Kubernetes cluster:
helm repo update
Result:
Hang tight while we grab the latest from your chart repositories... ...Successfully got an update from the "ingress-nginx" chart repository Update Complete. ⎈Happy Helming!⎈
-
Install the controller in the standard configuration. The controller will be installed with Network Load Balancer:
helm install ingress-nginx ingress-nginx/ingress-nginx
Result:
NAME: ingress-nginx LAST DEPLOYED: Sun Jul 18 22:35:37 2021 NAMESPACE: default STATUS: deployed REVISION: 1 TEST SUITE: None NOTES: The ingress-nginx controller has been installed. It may take a few minutes for the LoadBalancer IP to be available. You can watch the status by running 'kubectl --namespace default get services -o wide -w ingress-nginx-controller' ...
To set up the controller configuration yourself, follow the guidelines provided in the Helm documentation
For specific port forwarding at NGINX Ingress controller installation, follow this guide.
Create a web resource in your Managed Service for Kubernetes cluster
Create a Deployment object
kubectl --namespace ns apply -f - <<< '
apiVersion: v1
kind: Service
metadata:
name: app
spec:
selector:
app: app
ports:
- protocol: TCP
port: 80
targetPort: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: app-deployment
labels:
app: app
spec:
replicas: 1
selector:
matchLabels:
app: app
template:
metadata:
labels:
app: app
spec:
containers:
- name: app
image: nginx:latest
ports:
- containerPort: 80'
Configure a DNS record for the Ingress controller
-
Find out the IP address of the Ingress controller (the value in the
EXTERNAL-IP
column):kubectl get svc
Result:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE ... ingress-nginx-controller LoadBalancer 10.96.164.252 84.201.153.122 80:31248/TCP,443:31151/TCP 2m19s ...
-
Add an A record pointing to the Ingress controller's public IP to your DNS provider or to your own DNS server:
<domain_name> IN A 84.201.153.122
Note
Registering the Let's Encrypt® certificate and an A record may take a few minutes.
Create an Ingress resource
Create an Ingressk8s-secret
certificate for HTTPS:
kubectl --namespace ns apply -f - <<< '
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nginx-test
namespace: ns
spec:
tls:
- hosts:
- <domain_name>
secretName: k8s-secret
ingressClassName: nginx
rules:
- host: <domain_name>
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: app
port:
number: 80'
Where <domain_name>
is the name of the domain for which the certificate is issued.
Check resource availability
Issue a GET request to the resource via HTTPS, for example, by this command:
curl https://<your_domain> -vv
Result example:
* Trying 51.250.64.86:443...
* Connected to <domain_name> (51.250.64.86) port 443 (#0)
...
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
* subject: CN=<domain_name>
* start date: Jul 13 14:31:55 2022 GMT
* expire date: Oct 11 14:31:54 2022 GMT
* subjectAltName: host "<domain_name>" matched cert's "<domain_name>"
...
* SSL certificate verify ok.
Note
The Let's Encrypt® certificate must update automatically after the certificate update in Certificate Manager.
You can specify a sync timeout in the refreshInterval
parameter of the ExternalSecret object.
Delete the resources you created
Some resources are not free of charge. To avoid paying for them, delete the resources you no longer need: