Contact UsGet started
© 2024 Iron Hive LLC Belgrade

Publishing game updates using Yandex Cloud CDN

Written by
Updated at October 17, 2024

Create and configure a Cloud CDN CDN resource to host content that is expected to handle a large number of requests over a short period of time, such as game update files (patches, DLC, and so on). To prevent CDN servers from increasing the workload on the content origins during this period, files will be preloaded to the server cache once.

Let's assume a patch is a single file named ycgame-update-v1.1.exe. It will be uploaded to a Yandex Object Storage bucket.

Note

We do not recommend preloading files smaller than 200 MB or larger than 5 GB.

To create a CDN infrastructure:

  1. Get things ready.
  2. Add a certificate to Certificate Manager
  3. Create buckets in Object Storage.
  4. Enable logging for the bucket with files.
  5. Upload a file to the bucket.
  6. Create a CDN resource and enable caching.
  7. Set up DNS for your domain.
  8. Preload content to the cache of CDN servers.
  9. Test the CDN.

If you no longer need the resources you created, delete them.

Getting started

Sign up for Yandex Cloud and create a billing account:

  1. Go to the management console and log in to Yandex Cloud or create an account if you do not have one yet.
  2. On the Yandex Cloud Billing page, make sure you have a billing account linked and it has the ACTIVE or TRIAL_ACTIVE status. If you do not have a billing account, create one.

If you have an active billing account, you can go to the cloud page to create or select a folder for your infrastructure to operate in.

Learn more about clouds and folders.

Make sure you have a domain name and can access the DNS settings on the website of your DNS hosting provider. This is usually the company that registered your domain.

The cost of supporting the CDN infrastructure includes:

Add a certificate to Certificate Manager

Certificates from Yandex Certificate Manager are supported. You can issue a new Let's Encrypt® certificate or upload one of your own.

The certificate must be located in the same folder as your CDN resource.

For a Let's Encrypt® certificate, have your rights checked for the domain specified in the certificate.

Create buckets in Object Storage

Create two buckets: one will store files and the other will store request logs for the first one.

  1. In the management console, select Object Storage.
  2. Create a bucket for files:
    1. Click Create bucket.
    2. Enter a ** Name** for the bucket.
    3. In the Object read access and Object listing access fields, select Public.
    4. Click Create bucket.
  3. Create a bucket for logs:
    1. Click Create bucket.
    2. Enter a ** Name** for the bucket.
    3. Click Create bucket.

  1. Create a bucket for files:

    aws --endpoint-url=https://storage.yandexcloud.net \
  s3api create-bucket \
  --bucket <name_of_bucket_with_files> \
  --acl public-read

    Result:

    {
  "Location": "/<name_of_bucket_with_files>"
}

  2. Create a bucket for logs:

    aws --endpoint-url=https://storage.yandexcloud.net \
  s3api create-bucket \
  --bucket <name_of_bucket_with_logs>

    Result:

    {
  "Location": "/<name_of_bucket_with_logs>"
}

If you don't have Terraform, install it and configure the Yandex Cloud provider.

Before you start, obtain static access keys, i.e., a secret key and key ID used for authentication in Object Storage.

  1. In the configuration file, describe the bucket parameters:

    • access_key: Static access key ID.
    • secret_key: Secret access key value.
    • bucket: Name of the bucket you are creating.

    Here is an example of the configuration file structure:

    provider "yandex" {
  token     = "<OAuth_token>"
  cloud_id  = "<cloud_ID>"
  folder_id = "<folder_ID>"
  zone      = "ru-central1-a"
}

resource "yandex_storage_bucket" "storage" {
  access_key = "<static_key_ID>"
  secret_key = "<secret_key>"
  bucket     = "<name_of_bucket_with_files>"
  acl        = "public-read"
}

resource "yandex_storage_bucket" "logs" {
  access_key = "<static_key_ID>"
  secret_key = "<secret_key>"
  bucket     = "<name_of_bucket_with_logs>"
}

  2. Make sure that the configuration files are correct:

    1. In the command line, go to the folder where you created the configuration file.

    2. Run a check using this command:

      terraform plan

    If the configuration is described correctly, the terminal will display the parameters of the bucket being created. If the configuration contains any errors, Terraform will point them out.

  3. Deploy the bucket:

    1. If the configuration does not contain any errors, run this command:

      terraform apply

    2. Confirm that you want to create the bucket.

Use the API create method.

Enable logging for the bucket with files

Make sure that, when a user sends a request, files are downloaded from the CDN server cache rather than directly from the bucket. To do this, enable bucket logging.

  1. Create a file with logging settings in JSON format. For example:

    {
  "LoggingEnabled": {
      "TargetBucket": "<name_of_bucket_with_logs>",
      "TargetPrefix": "<key_prefix>"
  }
}

    Where:

    • TargetBucket: Name of the target bucket for the logs.
    • TargetPrefix: Prefix of the key used for objects with logs, e.g., logs/.

  2. Enable logging in the bucket:

    aws s3api put-bucket-logging \
  --bucket <name_of_bucket_with_files> \
  --endpoint-url https://storage.yandexcloud.net \
  --bucket-logging-status file://<path_to_settings_file>

    Where:

    • --bucket: Name of the source bucket to enable action logging for.
    • --bucket-logging-status: Path to the logging settings file.

Use the putBucketLogging API method for the bucket with files. HTTP request body:

<BucketLoggingStatus xmlns="http://doc.s3.amazonaws.com/2006-03-01">
  <LoggingEnabled>
    <TargetBucket>name of bucket with logs</TargetBucket>
    <TargetPrefix>key prefix</TargetPrefix>
  </LoggingEnabled>
</BucketLoggingStatus>

Where:

  • TargetBucket: Name of the bucket for logs.
  • TargetPrefix: Prefix of the key used for objects with logs, e.g., logs/.

Upload a file to the bucket

  1. In the management console, select Object Storage.
  2. Select the bucket with files.
  3. Click Upload.
  4. In the window that opens, select the ycgame-update-v1.1.exe patch file and click Open.
  5. Click Upload.

Run this command:

aws --endpoint-url=https://storage.yandexcloud.net \
  s3 cp \
  <path_to_ycgame-update-v1.1.exe> \
  s3://<name_of_bucket_with_files>/ycgame-update-v1.1.exe

Result:

upload: <path_to_ycgame-update-v1.1.exe> to s3://<name_of_bucket_with_files>/ycgame-update-v1.1.exe

  1. Add the parameters of the object to upload to the configuration file you created in the bucket creation step:

    • bucket: Name of the bucket where to add the object.
    • key: Name of the object in the bucket, ycgame-update-v1.1.exe. This is a required parameter.
    • source: Relative or absolute path to the file you upload as an object.

    Here is an example of the configuration file structure:

    ...
resource "yandex_storage_object" "patch-v1-1" {
  access_key = "<static_key_ID>"
  secret_key = "<secret_key>"
  bucket     = "<name_of_bucket_with_files>"
  key        = "ycgame-update-v1.1.exe"
  source     = "<path_to_file>/ycgame-update-v1.1.exe"
}

  2. Make sure the configuration files are correct.

    1. In the command line, go to the directory with the configuration file.

    2. Run a check using this command:

      terraform plan

    If the configuration is described correctly, the terminal will display a list of created resources and their parameters. If the configuration contains any errors, Terraform will point them out.

  3. Deploy cloud resources.

    1. If the configuration does not contain any errors, run this command:

      terraform apply

    2. Confirm that you want to create the object.

Use the API upload method.

Create a CDN resource and enable caching

  1. In the management console, select Cloud CDN.

  2. If the CDN provider is not activated yet, click Activate provider. A connection will be established automatically.

    If you do not see the Activate provider button and you can create resources and origin groups, it means that the provider is already activated. Proceed to the next step.

  3. Create a CDN resource:

    1. In the CDN resources tab, click Create resource.
    2. Set up the main parameters of the CDN resource as follows:

      • Content query: From one origin.

      • Origin type: Bucket.

      • Bucket: <name_of_bucket_with_files>.

      • Domain names for content distribution: Primary domain name you will use to publish patches, e.g., cdn.ycprojectblue.example.

        Alert

        You cannot change the primary domain name used for content distribution after creating a CDN resource.

      • Under Additional:

        • In the Origin request protocol field, select HTTPS.
        • In the Redirect clients field, select Don't use.
        • Select End-user access to content.
        • In the Certificate type field, specify Certificate from Certificate Manager and select a certificate for the cdn.ycprojectblue.example domain name.
        • In the Host header field, select Custom. In the Name field, specify the domain name of the source, Header value<name_of_bucket_with_files>.storage.yandexcloud.net`, so that the source bucket responds to CDN server requests correctly.
    3. Click Create.

  4. Enable a client redirect from HTTP to HTTPS:

    1. In the CDN resources tab, select the resource you created previously.
    2. Make sure the certificate status under Additional changes to Issued.
    3. At the top right, click Edit.
    4. Under Additional, select HTTP to HTTPS in the Redirect clients field.
    5. Click Save.

  5. Enable caching on CDN servers for the resource:

    1. In the CDN resources tab, select the resource you created previously.
    2. Go to Caching.
    3. At the top right, click Edit.
    4. Enable CDN caching.
    5. Click Save.

If you do not have the Yandex Cloud command line interface yet, install and initialize it.

The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name or --folder-id parameter.

  1. If the CDN provider is not activated yet, run this command:

    yc cdn provider activate --folder-id <folder_ID> --type gcore

  2. Create a CDN resource:

    yc cdn resource create \
  --cname cdn.ycprojectblue.example \
  --origin-bucket-source <name_of_bucket_with_files>.storage.yandexcloud.net \
  --origin-bucket-name <name_of_bucket_with_files> \
  --origin-protocol https \
  --cert-manager-ssl-cert-id <certificate_ID> \
  --host-header <name_of_bucket_with_files>.storage.yandexcloud.net

    Result:

    id: bc8e3l7s4dha********
folder_id: b1g86q4m5vej********
cname: cdn.ycprojectblue.example
...
active: true
...

    For more information about the yc cdn resource create command, see the CLI reference.

  3. Enable a client redirect for a resource:

    yc cdn resource update <resource_ID> --redirect-http-to-https

  1. Add parameters of the CDN resources to the configuration file:

    ...
resource "yandex_cdn_origin_group" "my_group" {
  name     = "updates-origin-group"
  use_next = true
  origin {
    source = "<name_of_bucket_with_files>.storage.yandexcloud.net"
  }
}

resource "yandex_cdn_resource" "my_resource" {
  cname               = "cdn.ycprojectblue.example"
  active              = true
  origin_protocol     = "https"
  origin_group_id     = yandex_cdn_origin_group.my_group.id
  options {
    custom_host_header     = "<name_of_bucket_with_files>.storage.yandexcloud.net"
  }
  ssl_certificate {
    type                   = "certificate_manager"
    certificate_manager_id = "<certificate_ID>"
  }
}

    For more information, see the description of the yandex_cdn_origin_group and yandex_cdn_resource resources in the Terraform provider documentation.

  2. Make sure the configuration files are correct.

    1. In the command line, go to the folder where you created the configuration file.

    2. Run a check using this command:

      terraform plan

    If the configuration is described correctly, the terminal will display a list of created resources and their parameters. This is a test step; no resources will be created. If the configuration contains any errors, Terraform will point them out.

  3. Apply the configuration changes:

    1. If the configuration does not contain any errors, run this command:

      terraform apply

    2. Confirm creating the resources: type yes in the terminal and press Enter.

    All the resources you need will then be created in the specified folder. You can check the new resources and their configuration using the management console.

  4. Enable client redirect for a resource. In CDN resource parameters, add this field at the top of the options section:

    ...
options {
  redirect_https_to_http = true
...

  5. Run a check using this command:

    terraform plan

    If the configuration is described correctly, the terminal will display a list of updated resources and their parameters. If the configuration contains any errors, Terraform will point them out.

  6. If there are no errors, run this command:

    terraform apply

  7. Confirm the resource update: type yes in the terminal and press Enter.

This enables a redirect for the resource.

Use the gRPC API ResourceService/Create call or the REST API create method. To enable caching on CDN servers, add the edge_cache_settings field to the request body.

Set up DNS for your domain

  1. Get a .edgecdn.ru domain name generated for the CDN resource you created:

    1. In the management console, select Cloud CDN.
    2. Select the created CDN resource (the list of resources will contain its primary domain name: cdn.ycprojectblue.example).
    3. In the Overview tab, under DNS settings, copy the generated .edgecdn.ru domain name to the clipboard.

  2. Go to your domain's DNS settings on the site of your DNS hosting provider.

  3. Edit the CNAME record for cdn so that it points to the previously copied .edgecdn.ru URL. For example:

    cdn CNAME cl-********.edgecdn.ru.

    Note

    Do not use an ANAME resource record with domain names for content distribution; otherwise, the end user will get a response from a CDN server not linked to the user's geolocation. The response will always be the same for all users.

    If you use Cloud DNS, follow this guide to configure the record:

    Configuring DNS records for Cloud DNS
    1. In the management console, select Cloud DNS.
    2. If you do not have a public DNS zone, create one:
      1. Click Create zone.
      2. Specify the zone Name: cdn-dns-a.
      3. In the Zone field, specify your domain with a period at the end: ycprojectblue.example..
      4. Select a Type of the zone: Public.
      5. Click Create.
    3. Create a record in the zone:
      1. In the list of zones, click cdn-dns-a.
      2. Click Create record.
      3. Under Name, specify cdn so that the record matches the cdn.ycprojectblue.example domain name.
      4. Select the record Type: CNAME.
      5. In the Data field, paste the .edgecdn.ru URL you copied with a dot at the end.
      6. Click Create.

    1. If you do not have a public DNS zone, create one:

      yc dns zone create --name cdn-dns-a --zone ycprojectblue.example. --public-visibility

      Where:

      • --name: Zone name
      • --zone: Domain zone (your domain with a dot at the end)
      • --public-visibility: Zone public visibility option

      Result:

      id: aetuvdw77q61********
folder_id: aoewzf73jwdl********
created_at: "2021-09-28T10:33:31.917Z"
name: cdn-zone-a
zone: ycprojectblue.example.
public_visibility: {}

    2. Create a record in the zone:

      yc dns zone add-records --name cdn-dns-a --record "cdn CNAME cl-********.edgecdn.ru."

      Where:

      • --name: Zone name
      • --record: Resource record

    3. Check that the record was created:

      yc dns zone list-records --name cdn-dns-a

      Result:

      +----------------------------+------+-------+------------------------------+
|            NAME            | TTL  | TYPE  |             DATA             |
+----------------------------+------+-------+------------------------------+
| ycprojectblue.example.     | 3600 | NS    | ns1.yandexcloud.net.         |
|                            |      |       | ns2.yandexcloud.net.         |
| ycprojectblue.example.     | 3600 | SOA   | ns1.yandexcloud.net.         |
|                            |      |       | mx.cloud.yandex.net. 1 10800 |
|                            |      |       | 900 604800 86400             |
| cdn.ycprojectblue.example. |  600 | CNAME | cl-********.edgecdn.ru.      |
+----------------------------+------+-------+------------------------------+

      The list should contain the cdn.ycprojectblue.example. record.

    1. If you do not have a public DNS zone, create one using a gRPC API call to DnsZoneService/Create or the REST API create method. To make the zone public, add the public_visibility (gRPC) or publicVisibility (REST) field to the request body.
    2. Create the cdn CNAME cl-********.edgecdn.ru. record in the zone using the DnsZoneService/UpdateRecordSets gRPC API call or the updateRecordSets REST API method.

Preload content to the cache of CDN servers

  1. In the management console, select Cloud CDN.

  2. Select the created CDN resource (the list of resources will contain its primary domain name: cdn.ycprojectblue.example).

  3. Go to the Content tab.

  4. Click → Preload content.

  5. In the File path field, specify the path to the file stored in the origin, omitting the domain name:

    /ycgame-update-v1.1.exe

  6. Click Preload content.

If you do not have the Yandex Cloud command line interface yet, install and initialize it.

The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name or --folder-id parameter.

Specify the path to the file to pre-fetch:

yc cdn cache prefetch --resource-id <resource_ID> \
  --path /ycgame-update-v1.1.exe

For more information about the yc cdn cache prefetch command, see the CLI reference.

Use the gRPC API CacheService/Prefetch call or the prefetch REST API method.

Test the CDN

  1. Wait until the DNS records are updated (this may take a few hours) and the file is prefetched to the CDN servers.

  2. Download the file at the new URL:

    https://cdn.ycprojectblue.example/ycgame-update-v1.1.exe

  3. Get the logs of requests to the bucket with files:

    1. In the management console, select Object Storage.
    2. Select the bucket with the logs.
    3. Click the name of the object matching the download time for ycgame-update-v1.1.exe.
    4. Click → Download.

    1. Get a list of objects with logs:

      aws --endpoint-url=https://storage.yandexcloud.net \
  s3 ls s3://<name_of_bucket_with_logs>

      Result:

      2021-10-01 08:37:53         10 2021-10-01-08-37-53-631E0FC3B732AEDD
2021-10-01 09:38:05         62 2021-10-01-09-38-05-436E6CDC832A20EF
2021-10-01 10:38:01         23 2021-10-01-10-38-01-7F65EF1A6366FB18
2021-10-01 11:37:57         41 2021-10-01-11-37-57-6F31613427A7DB9A
2021-10-01 12:38:02         58 2021-10-01-12-38-02-AB893E6148AFDC55
2021-10-01 13:38:02         73 2021-10-01-13-38-02-E69EAEC1C9083756

    2. In the resulting list, find the object with the log saved after downloading ycgame-update-v1.1.exe and download it:

      aws --endpoint-url=https://storage.yandexcloud.net \
  s3 cp s3://<name_of_bucket_with_logs>/2021-10-01-13-38-02-E69EAEC1C9083756 \
  2021-10-01-13-38-02-E69EAEC1C9083756

      Result:

      download: s3://<name_of_bucket_with_logs>/2021-10-01-13-38-02-E69EAEC1C9083756 to 2021-10-01-13-38-02-E69EAEC1C9083756
    1. Get a list of objects in the bucket with logs using the listObjects API method.
    2. In the resulting list, find the object whose log was saved after downloading ycgame-update-v1.1.exe and download it using the get API method.

  4. Check the logs of requests to the source bucket to make sure that the CDN servers did not download the file from the origin after your request. For more information about log contents, see Log object format of the Object Storage documentation.

How to delete the resources you created

To shut down your CDN resource and stop paying for the created resources:

  1. Disable the created resource.
  2. Delete the ycgame-update-v1.1.exe object from the bucket with files.
  3. Delete the bucket with files.
  4. Delete all objects from the bucket with logs.
  5. Delete the bucket with logs.
  6. Delete the DNS zone if you used it during DNS setup.
Previous
Configuring Cloud DNS for Managed Service for ClickHouse® cluster access from other cloud networks
Next
Overview