Discuss with expertTry it for free
© 2026 Direct Cursus Technology L.L.C.

Automatically copying objects from one Yandex Object Storage bucket to another

Written by
Updated at June 18, 2026

Configure automatic object copying from one Object Storage bucket to another. Objects will be copied using a function from Cloud Functions invoked by a trigger when a new object is added to a bucket.

To set up object copying:

  1. Get your cloud ready.
  2. Create service accounts.
  3. Create a static key.
  4. Create a Yandex Lockbox secret.
  5. Create Yandex Object Storage buckets.
  6. Create a ZIP archive with the function code.
  7. Create a function in Yandex Cloud Functions.
  8. Create a trigger.
  9. Test the function.

If you no longer need the resources you created, delete them.

Get your cloud ready

Sign up for Yandex Cloud and create a billing account:

  1. Navigate to the management console and log in to Yandex Cloud or create a new account.
  2. On the Yandex Cloud Billing page, make sure you have a billing account linked and it has the ACTIVE or TRIAL_ACTIVE status. If you do not have a billing account, create one and link a cloud to it.

If you have an active billing account, you can create or select a folder for your infrastructure on the cloud page.

Learn more about clouds and folders here.

The cost of resources includes:

Create service accounts

Create a service account named s3-copy-fn with the storage.uploader, storage.viewer, and lockbox.payloadViewer roles for the function, and a service account named s3-copy-trigger with the functions.functionInvoker role to invoke the function.

  1. In the management console, select the folder where you want to create a service account.
  2. Navigate to Identity and Access Management.
  3. Click Create service account.
  4. Name the service account: s3-copy-fn.
  5. Click Add role and select the storage.uploader, storage.viewer, and lockbox.payloadViewer roles.
  6. Click Create.
  7. Repeat the previous steps to create a service account named s3-copy-trigger with the functions.functionInvoker role. This service account will be used to invoke the function.

If you do not have the Yandex Cloud CLI yet, install and initialize it.

The folder used by default is the one specified when creating the CLI profile. To change the default folder, use the yc config set folder-id <folder_ID> command. You can also specify a different folder for any command using --folder-name or --folder-id. If you access a resource by its name, the search will be limited to the default folder. If you access a resource by its ID, the search will be global, i.e., through all folders based on access permissions.

  1. Create a service account named s3-copy-fn:

    yc iam service-account create --name s3-copy-fn

    Result:

    id: nfersamh4sjq********
folder_id: b1gc1t4cb638********
created_at: "2023-03-21T10:36:29.726397755Z"
name: s3-copy-fn

    Save the ID of the s3-copy-fn service account (id) and the ID of the folder where you created it (folder_id).

  2. Assign the storage.uploader, storage.viewer, and lockbox.payloadViewer roles to the service account:

    yc resource-manager folder add-access-binding <folder_ID> \
  --role storage.uploader \
  --subject serviceAccount:<service_account_ID>

yc resource-manager folder add-access-binding <folder_ID> \
  --role storage.viewer \
  --subject serviceAccount:<service_account_ID>

yc resource-manager folder add-access-binding <folder_ID> \
  --role lockbox.payloadViewer \
  --subject serviceAccount:<service_account_ID>

    Result:

    done (1s)

  3. Create a service account named s3-copy-trigger:

    yc iam service-account create --name s3-copy-trigger

    Save the ID of the s3-copy-trigger service account (id) and the ID of the folder where you created it (folder_id).

  4. Assign the functions.functionInvoker role for the folder to the service account:

    yc resource-manager folder add-access-binding <folder_ID> \
  --role storage.uploader \
  --subject serviceAccount:<service_account_ID>

If you do not have Terraform yet, install it and configure the Yandex Cloud provider.

To manage infrastructure using Terraform under a service account or user accounts (a Yandex account, a federated account, or a local user), authenticate using the appropriate method.

  1. In the configuration file, specify the service account properties:

    // Service account for the function
resource "yandex_iam_service_account" "s3-copy-fn" {
  name        = "s3-copy-fn"
  folder_id   = "<folder_ID>"
}

resource "yandex_resourcemanager_folder_iam_member" "uploader" {
  folder_id = "<folder_ID>"
  role      = "storage.uploader"
  member    = "serviceAccount:${yandex_iam_service_account.s3-copy-fn.id}"
}

resource "yandex_resourcemanager_folder_iam_member" "viewer" {
  folder_id = "<folder_ID>"
  role      = "storage.viewer"
  member    = "serviceAccount:${yandex_iam_service_account.s3-copy-fn.id}"
}

resource "yandex_resourcemanager_folder_iam_member" "payloadViewer" {
  folder_id = "<folder_ID>"
  role      = "lockbox.payloadViewer"
  member    = "serviceAccount:${yandex_iam_service_account.s3-copy-fn.id}"
}

// Service account to invoke the function
resource "yandex_iam_service_account" "s3-copy-trigger" {
  name        = "s3-copy-trigger"
  folder_id   = "<folder_ID>"
}

resource "yandex_resourcemanager_folder_iam_member" "functionInvoker" {
  folder_id = "<folder_ID>"
  role      = "functions.functionInvoker"
  member    = "serviceAccount:${yandex_iam_service_account.s3-copy-trigger.id}"
}

    Where:

    • name: Service account name. This is a required setting.
    • folder_id: Folder ID. This is an optional setting. It defaults to the value specified in the provider settings.
    • role: Role to assign.

    For more information about yandex_iam_service_account properties in Terraform, see this provider guide.

  2. Make sure the configuration files are correct.

    1. In the terminal, navigate to the directory where you created your configuration file.

    2. Run a check using this command:

      terraform plan

    If the configuration is correct, the terminal will display information about the service account. Otherwise, Terraform will show any detected errors.

  3. Deploy the cloud resources.

    1. If the configuration is correct, run this command:

      terraform apply

    2. Confirm creating the service accounts by typing yes in the terminal and pressing Enter.

      This will create the service accounts. You can check the new service accounts using the management console or this CLI command:

      yc iam service-account list

To create a service account, use the create REST API method for the ServiceAccount resource or the ServiceAccountService/Create gRPC API call.

To assign roles for a folder to a service account, use the setAccessBindings method for the ServiceAccount resource or the ServiceAccountService/SetAccessBindings gRPC API call.

Create a static key

Create a static access key for the s3-copy-fn service account.

  1. In the management console, select your service account folder.
  2. Navigate to Identity and Access Management.
  3. In the left-hand panel, select Service accounts and then, the s3-copy-fn service account.
  4. In the top panel, click Create new key.
  5. Select Create static access key.
  6. Enter a description for the key and click Create.
  7. Save the ID and the secret key.

  1. Run this command:

    yc iam access-key create --service-account-name s3-copy-fn

    Result:

    access_key:
  id: aje6t3vsbj8l********
  service_account_id: ajepg0mjt06s********
  created_at: "2023-03-21T14:37:51Z"
  key_id: 0n8X6WY6S24********
secret: JyTRFdqw8t1kh2-OJNz4JX5ZTz9Dj1rI********

  2. Save the ID (key_id) and the secret key (secret). This is the only time you can copy this key as it will not be shown again.

  1. In the configuration file, specify the key properties:

    resource "yandex_iam_service_account_static_access_key" "sa-static-key" {
  service_account_id = "<service_account_ID>"
}

    Where service_account_id is the s3-copy-fn service account ID.

    For more information about yandex_iam_service_account_static_access_key properties in Terraform, see this provider guide.

  2. Make sure the configuration files are correct.

    1. In the terminal, navigate to the directory where you created your configuration file.

    2. Run a check using this command:

      terraform plan

    If the configuration is correct, the terminal will display a list of the resources and their settings. Otherwise, Terraform will show any detected errors.

  3. Deploy the cloud resources.

    1. If the configuration is correct, run this command:

      terraform apply

    2. Confirm creating the static access key by typing yes in the terminal and pressing Enter.

      If there are any errors when creating the key, Terraform will report them.
      If the key is created successfully, Terraform will store it in its state without showing it to the user. The terminal will only display the ID of the created key.

      You can verify that the service account key has been created using the management console or this CLI command:

      yc iam access-key list --service-account-name=s3-copy-fn

To create an access key, use the create REST API method for the AccessKey resource or the AccessKeyService/Create gRPC API call.

Create a secret

Create a Yandex Lockbox secret to store your static access key.

  1. In the management console, select the folder where you want to create a secret.

  2. Navigate to Lockbox.

  3. Click Create secret.

  4. In the Name field, specify the secret name: s3-static-key.

  5. Under Secret data:

    1. Select the Custom secret type.

    2. Add the key ID value:

      • In the Key field, specify: key_id.
      • In the Value field, specify the key ID you got earlier.

    3. Click Add key/value.

    4. Add the secret key value:

      • In the Key field, specify: secret.
      • In the Value field, specify the secret key value you got earlier.

  6. Click Create.

To create a secret, run this command:

yc lockbox secret create --name s3-static-key \
  --payload "[{'key': 'key_id', 'text_value': '<key_ID>'},{'key': 'secret', 'text_value': '<secret_key_value>'}]"

Result:

id: e6q2ad0j9b55********
folder_id: b1gktjk2rg49********
created_at: "2021-11-08T19:23:00.383Z"
name: s3-static-key
status: ACTIVE
current_version:
  id: g6q4fn3b6okj********
  secret_id: e6e2ei4u9b55********
  created_at: "2023-03-21T19:23:00.383Z"
  status: ACTIVE
  payload_entry_keys:
    - key_id
    - secret

  1. In the configuration file, specify the secret properties:

    resource "yandex_lockbox_secret" "my_secret" {
  name = "s3-static-key"
}

resource "yandex_lockbox_secret_version" "my_version" {
  secret_id = yandex_lockbox_secret.my_secret.id
  entries {
    key        = "key_id"
    text_value = "<key_ID>"
  }
  entries {
    key        = "secret"
    text_value = "<private_key_value>"
  }
}

    Where:

    • name: Secret name.
    • key: Key name.
    • text_value: Key value.

    Note

    We recommend using yandex_lockbox_secret_version_hashed: it stores values in Terraform state in hashed format. We continue supporting yandex_lockbox_secret_version.

    For more information about yandex_lockbox_secret_version_hashed, see the relevant provider documentation.

    Learn more about the properties of Terraform resources in the relevant provider guides:

  2. Make sure the configuration files are correct.

    1. In the terminal, navigate to the directory where you created your configuration file.

    2. Run a check using this command:

      terraform plan

    If the configuration is correct, the terminal will display a list of the resources and their settings. Otherwise, Terraform will show any detected errors.

  3. Deploy the cloud resources.

    1. If the configuration is correct, run this command:

      terraform apply

    2. Confirm creating the secret creation by typing yes in the terminal and pressing Enter.

To create a secret, use the create REST API method for the Secret resource or the SecretService/Create gRPC API call.

Create Object Storage buckets

Create two buckets: a primary bucket for storing files and a backup bucket for copying files from the primary bucket.

  1. In the management console, select the folder where you want to create your buckets.

  2. Navigate to Object Storage.

  3. Create the main bucket:

    1. Click Create bucket.
    2. In the ** Name** field, enter a name for the main bucket.
    3. In the Read objects, Read object list, and Read settings fields, select With authorization.
    4. Click Create bucket.

  4. Similarly, create the backup bucket.

If you do not have the AWS CLI yet, install and configure it.

Create the main and backup buckets:

aws --endpoint-url https://storage.yandexcloud.net \
  s3 mb s3://<main_bucket_name>

aws --endpoint-url https://storage.yandexcloud.net \
  s3 mb s3://<backup_bucket_name>

Result:

make_bucket: <main_bucket_name>
make_bucket: <backup_bucket_name>

Note

If you access Object Storage via Terraform under a service account, assign to the service account the relevant role, e.g., storage.admin, for the folder you are going to create the resources in.

  1. Describe the properties for creating a service account and access key in the configuration file:

    ...
// Creating a service account
resource "yandex_iam_service_account" "sa" {
  name = "<service_account_name>"
}

// Assigning a role to a service account
resource "yandex_resourcemanager_folder_iam_member" "sa-admin" {
  folder_id = "<folder_ID>"
  role      = "storage.admin"
  member    = "serviceAccount:${yandex_iam_service_account.sa.id}"
}

// Creating a static access key
resource "yandex_iam_service_account_static_access_key" "sa-static-key" {
  service_account_id = yandex_iam_service_account.sa.id
  description        = "static access key for object storage"
}

  2. In the configuration file, specify the properties of the main and backup buckets:

    resource "yandex_storage_bucket" "main-bucket" {
  access_key = yandex_iam_service_account_static_access_key.sa-static-key.access_key
  secret_key = yandex_iam_service_account_static_access_key.sa-static-key.secret_key
  bucket     = "<main_bucket_name>"
}

resource "yandex_storage_bucket" "reserve-bucket" {
  access_key = yandex_iam_service_account_static_access_key.sa-static-key.access_key
  secret_key = yandex_iam_service_account_static_access_key.sa-static-key.secret_key
  bucket     = "<backup_bucket_name>"
}

    For more information about the yandex_storage_bucket resource, see this Terraform provider guide.

  3. Make sure the configuration files are correct.

    1. In the terminal, navigate to the directory where you created your configuration file.

    2. Run a check using this command:

      terraform plan

    If the configuration is correct, the terminal will display a list of the resources and their settings. Otherwise, Terraform will show any detected errors.

  4. Deploy the cloud resources.

    1. If the configuration is correct, run this command:

      terraform apply

    2. Confirm creating the buckets by typing yes in the terminal and pressing Enter.

To create a bucket, use the create REST API method for the Bucket resource or the BucketService/Create gRPC API call.

Create a ZIP archive with the function code

  1. Save the following code to a file named handler.sh:

    set -e
(
  cat | jq -c '.messages[]' | while read message; 
  do
    SRC_BUCKET=$(echo "$message" | jq -r .details.bucket_id)
    SRC_OBJECT=$(echo "$message" | jq -r .details.object_id)
    aws --endpoint-url="$S3_ENDPOINT" s3 cp "s3://$SRC_BUCKET/$SRC_OBJECT" "s3://$DST_BUCKET/$SRC_OBJECT"
  done;
) 1>&2

  2. Add handler.sh to the handler-sh.zip archive.

Create a function

Create a function that automatically copies new objects from the main bucket to the backup bucket.

  1. In the management console, select the folder where you want to create a function.

  2. Go to Cloud Functions.

  3. Create a function:

    1. Click Create function.
    2. Enter the function name: copy-function.
    3. Click Create.

  4. Create a function version:

    1. Select the Bash runtime, disable Add files with code examples, and click Continue.

    2. Specify the ZIP archive upload method and select the handler-sh.zip archive created in the previous step.

    3. Specify the entry point: handler.sh.

    4. Under Parameters, specify:

      • Timeout: 600.

      • Memory: 128 MB.

      • Service account: s3-copy-fn.

      • Environment variables:

        • S3_ENDPOINT: https://storage.yandexcloud.net.
        • DST_BUCKET: Name of the backup bucket to copy objects to.

      • Lockbox secrets:

        • AWS_ACCESS_KEY_ID: s3-static-key secret ID, latest version ID, key_id secret key.
        • AWS_SECRET_ACCESS_KEY: s3-static-key secret ID, latest version ID, secret key.

    5. Click Save changes.

  1. Create a function named copy-function:

    yc serverless function create --name=copy-function

    Result:

    id: b09bhaokchn9********
folder_id: <folder_ID>
created_at: "2024-10-21T20:40:03.451Z"
name: copy-function
http_invoke_url: https://functions.yandexcloud.net/b09bhaokchn9********
status: ACTIVE

  2. Create a version of the copy-function function:

    yc serverless function version create \
  --function-name copy-function \
  --memory=128m \
  --execution-timeout=600s \
  --runtime=bash \
  --entrypoint=handler.sh \
  --service-account-id=<service_account_ID> \
  --environment DST_BUCKET=<backup_bucket_name> \
  --environment S3_ENDPOINT=https://storage.yandexcloud.net \
  --secret name=s3-static-key,key=key_id,environment-variable=AWS_ACCESS_KEY_ID \
  --secret name=s3-static-key,key=secret,environment-variable=AWS_SECRET_ACCESS_KEY \
  --source-path=./handler-sh.zip

    Where:

    • --function-name: Name of the function whose version you are creating.
    • --memory: Amount of RAM.
    • --execution-timeout: Maximum function execution time before timeout.
    • --runtime: Runtime.
    • --entrypoint: Entry point.
    • --service-account-id: s3-copy-fn service account ID.
    • --environment: Environment variables.
    • --secret: Secret with parts of the static access key.
    • --source-path: Path to the handler-sh.zip archive.

    Result:

    done (1s)
id: d4e394pt4nhf********
function_id: d4efnkn79m7n********
created_at: "2024-10-21T20:41:01.345Z"
runtime: bash
entrypoint: handler.sh
resources:
  memory: "134217728"
execution_timeout: 600s
service_account_id: ajelprpohp7r********
image_size: "4096"
status: ACTIVE
tags:
  - $latest
environment:
  DST_BUCKET: <backup_bucket_name>
  S3_ENDPOINT: https://storage.yandexcloud.net
secrets:
  - id: e6qo2oprlmgn********
    version_id: e6q6i1qt0ae8********
    key: key_id
    environment_variable: AWS_ACCESS_KEY_ID
  - id: e6qo2oprlmgn********
    version_id: e6q6i1qt0ae8********
    key: secret
    environment_variable: AWS_SECRET_ACCESS_KEY
log_options:
  folder_id: b1g681qpemb4********
concurrency: "1"

  1. In the configuration file, define the function properties:

    resource "yandex_function" "copy-function" {
  name               = "copy-functionn"
  user_hash          = "first function"
  runtime            = "bash"
  entrypoint         = "handler.sh"
  memory             = "128"
  execution_timeout  = "600"
  service_account_id = "<service_account_ID>"
  environment = {
    DST_BUCKET  = "<backup_bucket_name>"
    S3_ENDPOINT = "https://storage.yandexcloud.net"
  }
  secrets {
    id                   = "<secret_ID>"
    version_id           = "<secret_version_ID>"
    key                  = "key_id"
    environment_variable = "AWS_ACCESS_KEY_ID"
  }
  secrets {
    id                   = "<secret_ID>"
    version_id           = "<secret_version_ID>"
    key                  = "secret"
    environment_variable = "AWS_SECRET_ACCESS_KEY"
  }
  content {
    zip_filename = "./handler-sh.zip"
  }
}

    Where:

    • name: Function name.
    • user_hash: Any string to identify the function version.
    • runtime: Function runtime.
    • entrypoint: Entry point.
    • memory: Amount of memory allocated for the function, in MB.
    • execution_timeout: Function execution timeout.
    • service_account_id: s3-copy-fn service account ID.
    • environment: Environment variables.
    • secrets: Secret with parts of the static access key.
    • content: Path to the handler-sh.zip archive with the function source code.

    For more information about yandex_function properties, see this provider guide.

  2. Make sure the configuration files are correct.

    1. In the terminal, navigate to the directory where you created your configuration file.

    2. Run a check using this command:

      terraform plan

    If the configuration is correct, the terminal will display a list of the resources and their settings. Otherwise, Terraform will show any detected errors.

  3. Deploy the cloud resources.

    1. If the configuration is correct, run this command:

      terraform apply

    2. To confirm the function creation, type yes in the terminal and press Enter.

To create a function, use the create REST API method for the Function resource or the FunctionService/Create gRPC API call.

To create a function version, use the createVersion REST API method for the Function resource or the FunctionService/CreateVersion gRPC API call.

Create a trigger

Create a trigger for Object Storage that invokes copy-function when you create a new object in the main bucket.

  1. In the management console, select the folder where you want to create a trigger.

  2. Go to Cloud Functions.

  3. In the left-hand panel, select Triggers.

  4. Click Create trigger.

  5. Under Basic settings:

    • Specify the trigger name: bucket-to-bucket-copying.
    • In the Type field, select Object Storage.
    • In the Launched resource field, select Function.

  6. Under Object Storage settings:

    • In the Bucket field, select the main bucket.
    • In the Event types field, select Create object.

  7. Under Function settings:

    • In the Function field, select copy-function.
    • In the Service account field, select s3-copy-trigger.

  8. Click Create trigger.

Run this command:

yc serverless trigger create object-storage \
  --name bucket-to-bucket-copying \
  --bucket-id <main_bucket_name> \
  --events 'create-object' \
  --invoke-function-name copy-function \
  --invoke-function-service-account-name s3-copy-trigger

Where:

  • --name: Trigger name.
  • --bucket-id: Name of the main bucket.
  • --events: Events that set off the trigger.
  • --invoke-function-name: Name of the function being invoked.
  • --invoke-function-service-account-name: Name of the service account to use for invoking the function.

Result:

id: a1s92agr8mpg********
folder_id: b1g88tflru0e********
created_at: "2024-10-21T21:04:01.866959640Z"
name: bucket-to-bucket-copying
rule:
  object_storage:
    event_type:
      - OBJECT_STORAGE_EVENT_TYPE_CREATE_OBJECT
    bucket_id: <main_bucket_name>
    batch_settings:
      size: "1"
      cutoff: 1s
    invoke_function:
      function_id: d4eofc7n0m03********
      function_tag: $latest
      service_account_id: aje3932acd0c********
status: ACTIVE

  1. In the configuration file, specify the trigger properties:

    resource "yandex_function_trigger" "my_trigger" {
  name        = "bucket-to-bucket-copying"
  object_storage {
      bucket_id = "<main_bucket_name>"
      create    = true
  }
  function {
    id                 = "<function_ID>"
    service_account_id = "<service_account_ID>"
  }
}

    Where:

    • name: Trigger name.
    • object_storage: Storage settings:
      • bucket_id: Name of the main bucket.
      • create: Trigger will invoke the function when a new object is created in the storage.
    • function: Settings of the function the trigger will invoke:
      • id: copy-function ID.
      • service_account_id: s3-copy-trigger service account ID.

    For more information about resource properties in Terraform, see this provider guide.

  2. Make sure the configuration files are correct.

    1. In the terminal, navigate to the directory where you created your configuration file.

    2. Run a check using this command:

      terraform plan

    If the configuration is correct, the terminal will display a list of the resources and their settings. Otherwise, Terraform will show any detected errors.

  3. Deploy the cloud resources.

    1. If the configuration is correct, run this command:

      terraform apply

    2. Confirm creating the trigger by typing yes in the terminal and pressing Enter.

To create a trigger for Object Storage, use the create method for the Trigger resource or the TriggerService/Create gRPC API call.

Test the function

  1. In the management console, select the folder containing your main bucket.
  2. Navigate to Object Storage.
  3. Click the name of the main bucket.
  4. In the top-right corner, click Upload.
  5. In the window that opens, select the files and click Open.
  6. The management console will display all objects selected for upload. Click Upload.
  7. Refresh the page.
  8. Navigate to the backup bucket and make sure it contains the files you added.

How to delete the resources you created

To stop paying for the resources you created:

  1. Delete all objects from the the buckets.
  2. Delete the buckets.
  3. Delete the bucket-to-bucket-copying trigger.
  4. Delete copy-function.
Previous
Creating an interactive serverless application using WebSocket
Next
Overview