Contact UsGet started
© 2024 Iron Hive LLC Belgrade

Publishing game updates using Yandex Cloud CDN

Written by
Updated at October 17, 2024

Create and configure a Yandex Cloud CDN CDN resource to host content that is expected to handle a large number of requests over a short period of time, such as game update files (patches, DLC, and so on). To prevent CDN servers from increasing the workload on the content origins during this period, files will be preloaded to the server cache once.

We assume a patch is a single file named ycgame-update-v1.1.exe. It will be uploaded to a Yandex Object Storage bucket.

Note

We don't recommend preloading files smaller than 200 MB or larger than 5 GB.

To create a CDN infrastructure:

  1. Get started.
  2. Create buckets in Object Storage.
  3. Enable logging for the bucket with files.
  4. Upload a file to the bucket.
  5. Create a CDN resource and enable caching.
  6. Set up DNS for your domain.
  7. Preload content to the cache of CDN servers.
  8. Test the CDN.

If you no longer need the resources you created, delete them.

Getting started

Sign up for Yandex Cloud and create a billing account:

  1. Go to the management console and log in to Yandex Cloud or create an account if you do not have one yet.
  2. On the Yandex Cloud Billing page, make sure you have a billing account linked and it has the ACTIVE or TRIAL_ACTIVE status. If you do not have a billing account, create one.

If you have an active billing account, you can go to the cloud page to create or select a folder for your infrastructure to operate in.

Learn more about clouds and folders.

Make sure you have a domain name and can access the DNS settings on the website of your DNS hosting provider. This is usually the company that registered your domain.

The cost of supporting the CDN infrastructure includes:

Create buckets in Object Storage

Create two buckets: one will store files and the other will store request logs for the first one.

  1. In the management console, select Object Storage.

  2. Create a bucket for files:

    1. Click Create bucket.
    2. Name the bucket.
    3. In the Object read access and Object listing access fields, select Public.
    4. Click Create bucket.

  3. Create a bucket for logs:

    1. Click Create bucket.
    2. Name the bucket.
    3. Click Create bucket.

  1. Create a bucket for files:

    aws --endpoint-url=https://storage.yandexcloud.net \
  s3api create-bucket \
  --bucket <name_of_bucket_with_files> \
  --acl public-read

    Result:

    {
    "Location": "/<name_of_bucket_with_files>"
}

  2. Create a bucket for logs:

    aws --endpoint-url=https://storage.yandexcloud.net \
  s3api create-bucket \
  --bucket <name_of_bucket_with_logs>

    Result:

    {
    "Location": "/<name_of_bucket_with_logs>"
}

If you don't have Terraform, install it and configure the Yandex Cloud provider.

Before you start, retrieve the static access keys: a secret key and a key ID used for authentication in Object Storage.

  1. In the configuration file, describe the bucket parameters:

    • access_key: ID of the static access key.
    • secret_key: Value of the secret access key.
    • bucket: The name of the bucket being created.

    Example of the configuration file structure:

    provider "yandex" {
  token     = "<OAuth>"
  cloud_id  = "<cloud_ID>"
  folder_id = "<folder_ID>"
  zone      = "ru-central1-a"
}

resource "yandex_storage_bucket" "storage" {
  access_key = "<static_key_ID>"
  secret_key = "<private_key>"
  bucket     = "<name_of_bucket_with_files>"
  acl        = "public-read"
}

resource "yandex_storage_bucket" "logs" {
  access_key = "<static_key_ID>"
  secret_key = "<private_key>"
  bucket     = "<name_of_bucket_with_logs>"
}

Enable logging for the bucket with files

You need to check that, when user requests are made, files are downloaded from the CDN server cache rather than directly from the bucket. To do this, enable bucket logging.

Use the putBucketLogging API method for the bucket with files. HTTP request body:

<BucketLoggingStatus xmlns="http://doc.s3.amazonaws.com/2006-03-01">
  <LoggingEnabled>
    <TargetBucket>name_of_bucket_with_logs</TargetBucket>
  </LoggingEnabled>
</BucketLoggingStatus>

Where TargetBucket is the name of the bucket to write logs to.

Upload a file to the bucket

  1. In the management console, select Object Storage.
  2. Select the bucket with files.
  3. Click Upload.
  4. In the window that opens, select the ycgame-update-v1.1.exe patch file and click Open.
  5. Click Upload.

Run this command:

aws --endpoint-url=https://storage.yandexcloud.net \
  s3 cp \
  <path_to_ycgame-update-v1.1.exe> \
  s3://<name_of_bucket_with_files>/ycgame-update-v1.1.exe

Result:

upload: <path_to_ycgame-update-v1.1.exe> to s3://<name_of_bucket_with_files>/ycgame-update-v1.1.exe

  1. Add the parameters of the object to upload to the configuration file you created in the bucket creation step:

    • bucket: Name of the bucket to add the object to.
    • key: Name of the object in the bucket (ycgame-update-v1.1.exe). This parameter is required.
    • source: Relative or absolute path to the file that you upload as an object.

    Example of the configuration file structure:

    ...

resource "yandex_storage_object" "patch-v1-1" {
  access_key = "<static_key_ID>"
  secret_key = "<private_key>"
  bucket = "<name_of_bucket_with_files>"
  key    = "ycgame-update-v1.1.exe"
  source = "<path_to_file>/ycgame-update-v1.1.exe"
}

  2. Make sure the configuration files are valid.

    1. In the command line, go to the directory with the configuration file.

    2. Run the check using this command:

      terraform plan

    If the configuration is described correctly, the terminal will display a list of created resources and their parameters. If the configuration contains any errors, Terraform will point them out.

  3. Deploy cloud resources.

    1. If the configuration does not contain any errors, run this command:
    terraform apply
    1. Confirm that you want to create the object.

Use the API upload method.

Create a CDN resource and enable caching

  1. In the management console, select Cloud CDN.

  2. If the CDN provider hasn't been activated yet, click Activate provider.

  3. Create a CDN resource:

    1. On the CDN resources tab, click Create resource.

    2. Set up the main parameters of the CDN resource as follows:

      • Content query: From one origin.

      • Origin type: Bucket.

      • Bucket: <name_of_bucket_with_files>.

      • Domain names for content distribution: The primary domain name that you'll use to publish patches, such as cdn.ycprojectblue.example.

        Alert

        You can't change the primary domain name used for content distribution after you create a CDN resource.

      • In the Advanced section:

        • In the Origin request protocol field, select HTTPS.
        • In the Redirect clients field, select Don't use.
        • Enable End-user access to content.
        • In the Certificate type field, select Let's Encrypt® to automatically issue a certificate for the cdn.ycprojectblue.example domain name after creating your CDN resource.
        • In the Host header field, select Custom. In the Header value field, specify the origin domain name, <name_of_bucket_with_files>.storage.yandexcloud.net, so that the source bucket responds to CDN server requests correctly.

    3. Click Create.

    Wait until the Let's Encrypt® certificate is issued for the domain name. This may take up to 30 minutes.

  4. Enable a client redirect from HTTP to HTTPS:

    1. In the CDN resources tab, select the previously created resource.
    2. Make sure the certificate status under Additional changes to Issued.
    3. At the top right, click Edit.
    4. Under Additional, in the Redirect clients field, select HTTP to HTTPS.
    5. Click Save.

  5. Enable caching on CDN servers for the resource:

    1. In the CDN resources tab, select the previously created resource.
    2. Go to Caching.
    3. At the top right, click Edit.
    4. Enable CDN caching.
    5. Click Save.

If you do not have the Yandex Cloud command line interface yet, install and initialize it.

The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name or --folder-id parameter.

  1. If the CDN provider hasn't been activated yet, run the command:

    yc cdn provider activate --folder-id <folder_ID> --type gcore

  2. Create a CDN resource:

    yc cdn resource create \
  --cname cdn.ycprojectblue.example \
  --origin-bucket-source <name_of_bucket_with_files>.storage.yandexcloud.net \
  --origin-bucket-name <name_of_bucket_with_files> \
  --origin-protocol https \
  --lets-encrypt-gcore-ssl-cert \
  --host-header <name_of_bucket_with_files>.storage.yandexcloud.net \

    Result:

    id: bc8e3l7s4dha********
folder_id: b1g86q4m5vej********
cname: cdn.ycprojectblue.example
...
active: true
...

    For more information about the yc cdn resource create command, see the CLI reference.

  3. Enable a client redirect for a resource:

    yc cdn resource update <resource_ID> --redirect-http-to-https

  1. Add parameters of CDN resources to the configuration file:

    ...

resource "yandex_cdn_origin_group" "my_group" {
  name     = "updates-origin-group"
  use_next = true
  origin {
    source = "<name_of_bucket_with_files>.storage.yandexcloud.net"
  }
}

resource "yandex_cdn_resource" "my_resource" {
  cname               = "cdn.ycprojectblue.example"
  active              = true
  origin_protocol     = "https"
  origin_group_id     = yandex_cdn_origin_group.my_group.id
  options {
    custom_host_header     = "<name_of_bucket_with_files>.storage.yandexcloud.net"
  }
  ssl_certificate {
    type = "lets_encrypt_gcore"
  }
}

    For more information, see the descriptions of the yandex_cdn_origin_group and yandex_cdn_resource resources in the Terraform provider documentation.

  2. Make sure the configuration files are valid.

    1. In the command line, go to the directory where you created the configuration file.

    2. Run the check using this command:

      terraform plan

    If the configuration is described correctly, the terminal will display a list of created resources and their parameters. This is a test step. No resources are created. If the configuration contains any errors, Terraform will point them out.

  3. Apply the configuration changes:

    1. If the configuration does not contain any errors, run this command:

      terraform apply

    2. Confirm the resource creation: type yes in the terminal and press Enter.

    All the resources you need will then be created in the specified folder. You can check that the resources are there and their settings are correct using the management console.

  4. Enable client redirect for a resource. Add the following field at the beginning of the options section for a CDN resource:

    ...
options {
  redirect_https_to_http = true
...

  5. Run the check using this command:

    terraform plan

    If the configuration is described correctly, the terminal will display a list of updated resources and their parameters. If the configuration contains any errors, Terraform will point them out.

  6. If there are no errors, run this command:

    terraform apply

  7. Confirm the resource update: type yes in the terminal and press Enter.

This enables a redirect for the resource.

Use the gRPC API ResourceService/Create call or the REST API create method. To enable caching on CDN servers, add the edge_cache_settings field to the request body.

Set up DNS for your domain

  1. Get a domain name generated for the created CDN resource on the .edgecdn.ru domain:

    1. In the management console, select Cloud CDN.
    2. Select the created CDN resource (the list of resources will contain its primary domain name: cdn.ycprojectblue.example).
    3. On the Overview tab, under DNS settings, copy the name generated on the .edgecdn.ru domain to the clipboard.

  2. Go to your domain's DNS settings on the site of your DNS hosting provider.

  3. Edit the CNAME record for cdn so that it points to the previously copied URL on the .edgecdn.ru domain. For example:

    cdn CNAME cl-********.edgecdn.ru.

    If you use Cloud DNS, follow these instructions to configure the record:

    Instructions for configuring DNS records for Cloud DNS

    1. In the management console, select Cloud DNS.

    2. If you don't have a public DNS zone, create one:

      1. Click Create zone.
      2. Name the zone: cdn-dns-a.
      3. In the Zone field, specify your domain with a period at the end: ycprojectblue.example.
      4. Select the zone Type: Public.
      5. Click Create.

    3. Create a record in the zone:

      1. In the list of zones, click cdn-dns-a.
      2. Click Create record.
      3. In the Name field, specify cdn to make the record match the cdn.ycprojectblue.example domain name.
      4. Select the record Type: CNAME.
      5. In the Value field, paste the URL you copied on the .edgecdn.ru domain with a dot at the end.
      6. Click Create.

    1. If you don't have a public DNS zone, create one:

      yc dns zone create --name cdn-dns-a --zone ycprojectblue.example. --public-visibility

      Where:

      • --name: Zone name.
      • --zone: Domain zone (your domain with a dot at the end).
      • --public-visibility: Zone visibility (indicates if it's public).

      Result:

      id: aetuvdw77q61********
folder_id: aoewzf73jwdl********
created_at: "2021-09-28T10:33:31.917Z"
name: cdn-zone-a
zone: ycprojectblue.example.
public_visibility: {}

    2. Create a record in the zone:

      yc dns zone add-records --name cdn-dns-a --record "cdn CNAME cl-********.edgecdn.ru."

      Where:

      • --name: Zone name.
      • --record: Resource record.

    3. Check that the record was created:

      yc dns zone list-records --name cdn-dns-a

      Result:

      +----------------------------+------+-------+--------------------------------+
|            NAME            | TTL  | TYPE  |              DATA              |
+----------------------------+------+-------+--------------------------------+
| ycprojectblue.example.     | 3600 | NS    | ns1.yandexcloud.net.           |
|                            |      |       | ns2.yandexcloud.net.           |
| ycprojectblue.example.     | 3600 | SOA   | ns1.yandexcloud.net.           |
|                            |      |       | mx.cloud.yandex.net. 1 10800   |
|                            |      |       | 900 604800 86400               |
| cdn.ycprojectblue.example. |  600 | CNAME | cl-********.edgecdn.ru.           |
+----------------------------+------+-------+--------------------------------+

      The list should contain a record named cdn.ycprojectblue.example.

    1. If you do not have a public DNS zone, create one using a gRPC API call to DnsZoneService/Create or the REST API create method. To make the zone public, add the public_visibility (gRPC) or publicVisibility (REST) field to the request body.
    2. In the zone, create a record named cdn CNAME cl-********.edgecdn.ru. using a gRPC API DnsZoneService/UpdateRecordSets call or the REST API updateRecordSets method.

Preload content to the cache of CDN servers

  1. In the management console, select Cloud CDN.

  2. Select the created CDN resource (the list of resources will contain its primary domain name: cdn.ycprojectblue.example).

  3. Go to the Content tab.

  4. Click Preload content.

  5. In the File path field, specify the path to the file stored in the origin, omitting the domain name:

    /ycgame-update-v1.1.exe

  6. Click Preload content.

If you do not have the Yandex Cloud command line interface yet, install and initialize it.

The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name or --folder-id parameter.

Specify the path to the file to pre-fetch:

yc cdn cache prefetch --resource-id <resource_ID> \
  --path /ycgame-update-v1.1.exe

For more information about the yc cdn cache prefetch command, see the CLI reference.

Use the gRPC API CacheService/Prefetch call or the prefetch REST API method.

Test the CDN

  1. Wait until the DNS records are updated (this may take a few hours) and the file is prefetched to the CDN servers.

  2. Download the file at the new URL:

    https://cdn.ycprojectblue.example/ycgame-update-v1.1.exe

  3. Get the logs of requests to the bucket with files:

    1. In the management console, select Object Storage.
    2. Select the bucket with the logs.
    3. Click on the name of the object corresponding to the ycgame-update-v1.1.exe download time.
    4. Click Download.

    1. Get a list of objects with logs:

      aws --endpoint-url=https://storage.yandexcloud.net \
  s3 ls s3://<name_of_bucket_with_logs>

      Result:

      2021-10-01 08:37:53         10 2021-10-01-08-37-53-631E0FC3B732AEDD
2021-10-01 09:38:05         62 2021-10-01-09-38-05-436E6CDC832A20EF
2021-10-01 10:38:01         23 2021-10-01-10-38-01-7F65EF1A6366FB18
2021-10-01 11:37:57         41 2021-10-01-11-37-57-6F31613427A7DB9A
2021-10-01 12:38:02         58 2021-10-01-12-38-02-AB893E6148AFDC55
2021-10-01 13:38:02         73 2021-10-01-13-38-02-E69EAEC1C9083756

    2. In the resulting list, find the object with the log saved after downloading ycgame-update-v1.1.exe and download it:

      aws --endpoint-url=https://storage.yandexcloud.net \
  s3 cp s3://<name_of_bucket_with_logs>/2021-10-01-13-38-02-E69EAEC1C9083756 \
  2021-10-01-13-38-02-E69EAEC1C9083756

      Result:

      download: s3://<name_of_bucket_with_logs>/2021-10-01-13-38-02-E69EAEC1C9083756 to 2021-10-01-13-38-02-E69EAEC1C9083756

    1. Get a list of objects in the bucket with logs using the listObjects API method.

    2. In the resulting list, find the object whose log was saved after downloading ycgame-update-v1.1.exe and download it using the API get method.

  4. Check the logs of requests to the source bucket to make sure that the CDN servers did not download the file from the origin after your request. For more information about log contents, see the Log object format section of the Object Storage documentation.

How to delete the resources you created

To shut down your CDN resource and stop paying for the created resources:

  1. Disable the created resource.
  2. Delete the ycgame-update-v1.1.exe object from the bucket with files.
  3. Delete the bucket with files.
  4. Delete all objects from the bucket with logs.
  5. Delete the bucket with logs.
  6. Delete the DNS zone if you used it during DNS setup.
Previous
Emulating multiple IoT devices
Next
Creating an SAP program in Yandex Cloud