Contact UsGet started

rclone

Written by
Updated at October 30, 2024

rclone is a tool for mounting Object Storage buckets using FUSE on Linux, macOS, and Windows platforms.

In this tutorial, you will use rclone to set up synchronization of data between an Yandex Object Storage bucket and your local desktop. The bucket will be mounted as a disk in Windows.

Note

This section describes how to connect a bucket in Windows. When connecting in other OSs, see the official rclone documentation. For rclone configuring and bucket mounting, follow the same steps used in Windows.

To mount your bucket as a disk:

  1. Prepare your cloud.
  2. Set up a work environment.
  3. Create a service account.
  4. Create a static access key.
  5. Create a bucket.
  6. Set up a connection to Object Storage.
  7. Mount the bucket.
  8. Set up the mounting service.

If you no longer need the resources you created, delete them.

Getting started

Sign up for Yandex Cloud and create a billing account:

  1. Go to the management console and log in to Yandex Cloud or create an account if you do not have one yet.
  2. On the Yandex Cloud Billing page, make sure you have a billing account linked and it has the ACTIVE or TRIAL_ACTIVE status. If you do not have a billing account, create one.

If you have an active billing account, you can go to the cloud page to create or select a folder for your infrastructure to operate in.

Learn more about clouds and folders.

The cost for bucket support includes:

Set up a working environment

  1. Download and install the winfsp distribution from the developer website.

  2. Download the archive with sysinternals suite utilities from the developer's website and unpack it to your local working folder.

  3. Download the Windows Service Wrapper (WinSW) executable file depending on your OS configuration and save it to a separate folder.

  4. Download the archive with the rclone utility from the developer's website and unpack it to your local working folder.

  5. Add the folders containing the utilities and the distribution to the PATH variable. To do this:

    1. Click Start and type Change system environment variables in the Windows search bar.
    2. Click Environment Variables... at the bottom right.
    3. In the window that opens, find the PATH parameter and click Edit.
    4. Add paths to the directories with the utilities to the list.
    5. Click OK.

Create a service account

  1. In the management console, select the folder to create a service account in.
  2. At the top of the screen, go to the Service accounts tab.
  3. At the top right, click Create service account.
  4. In the Name field, specify sa-win-disk-connect.
  5. Click Add role and select the storage.editor role.
  6. Click Create service account.

If you do not have the Yandex Cloud command line interface yet, install and initialize it.

The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name or --folder-id parameter.

Create a service account named sa-win-disk-connect:

yc iam service-account create --name sa-win-disk-connect

The name format requirements are as follows:

  • The name must be from 3 to 63 characters long.
  • It may contain lowercase Latin letters, numbers, and hyphens.
  • The first character must be a letter and the last character cannot be a hyphen.

For more information about the yc iam service-account create command, see the CLI reference.

To create the service account, use the create method for the ServiceAccount resource.

Create a static access key

  1. In the management console, navigate to the folder the service account belongs to.
  2. At the top of the screen, go to the Service accounts tab.
  3. Select the sa-win-disk-connect service account.
  4. In the top panel, click Create new key.
  5. Select Create static access key.
  6. Specify the key description and click Create.
  7. Save the ID and private key. After you close the dialog, the private key value will become unavailable.

  1. Create an access key for the sa-win-disk-connect service account:

    yc iam access-key create --service-account-name sa-win-disk-connect

    Result:

    access_key:
  id: aje6t3vsbj8l********
  service_account_id: ajepg0mjt06s********
  created_at: "2022-07-18T14:37:51Z"
  key_id: 0n8X6WY6S24N7Oj*****
secret: JyTRFdqw8t1kh2-OJNz4JX5ZTz9Dj1rI9hx*****

    For more information about the yc iam access-key create command, see the CLI reference.

  2. Save the key_id and the secret key. You will not be able to get the key value again.

To create an access key, use the create method for the AccessKey resource.

Create a bucket

  1. In the management console, select the folder you want to create a bucket in.
  2. In the list of services, select Object Storage.
  3. At the top right, click Create bucket.
  4. In the ** Name** field, enter a name for the bucket consistent with the naming conventions.
  5. In the Object read access, Object listing access, and Read access to settings fields, select Restricted.
  6. Click Create bucket.

  1. If you do not have the AWS CLI yet, install and configure it.

  2. Enter the bucket name following the naming conventions:

    aws --endpoint-url https://storage.yandexcloud.net \
  s3 mb s3://<bucket_name>

    Result:

    make_bucket: <bucket_name>

Note

Terraform uses a service account to interact with Object Storage. Assign to the service account the required role, e.g., storage.admin, for the folder where you are going to create resources.

If you don't have Terraform, install it and configure the Yandex Cloud provider.

  1. Describe the parameters for creating a service account and access key in the configuration file:

    ...
// Creating a service account
resource "yandex_iam_service_account" "sa" {
  name = "<service_account_name>"
}

// Assigning a role to a service account
resource "yandex_resourcemanager_folder_iam_member" "sa-admin" {
  folder_id = "<folder_ID>"
  role      = "storage.admin"
  member    = "serviceAccount:${yandex_iam_service_account.sa.id}"
}

// Creating a static access key
resource "yandex_iam_service_account_static_access_key" "sa-static-key" {
  service_account_id = yandex_iam_service_account.sa.id
  description        = "static access key for object storage"
}

  2. Add a section with bucket parameters to the configuration file and enter the bucket name following the naming conventions:

    resource "yandex_storage_bucket" "<bucket_name>" {
  access_key = yandex_iam_service_account_static_access_key.sa-static-key.access_key
  secret_key = yandex_iam_service_account_static_access_key.sa-static-key.secret_key
  bucket     = "<bucket_name>"
}

    For more information about the yandex_storage_bucket resource, see the Terraform provider documentation.

  3. Make sure the configuration files are correct.

    1. In the command line, go to the folder where you created the configuration file.

    2. Run a check using this command:

      terraform plan

    If the configuration is described correctly, the terminal will display a list of created resources and their parameters. If the configuration contains any errors, Terraform will point them out.

  4. Deploy cloud resources.

    1. If the configuration does not contain any errors, run this command:

      terraform apply

    2. Confirm creating the resources: type yes in the terminal and press Enter.

Use the create REST API method for the Bucket resource, the BucketService/Create gRPC API call, or the create S3 API method.

Set up a connection to Object Storage

  1. Open the command line in your working folder as an admin user and run the following command:

    .\psexec -i -s cmd.exe

  2. In the console that opens, run whoami and make sure the session has been started by the system user.

  3. Go to the folder with the rclone utility and run its configuration session:

    rclone.exe config

  4. Follow the prompts to create a new connection profile:

    1. Start creating a new profile by entering n in the terminal.
    2. Enter the connection name: s3-connect.
    3. Select the storage type by entering 4 in the terminal.
    4. Select a provider by entering 1 in the terminal.
    5. Select manual entry of credentials by entering 1 in the terminal.
    6. Enter the secret key ID in the terminal.
    7. Enter the secret key value in the terminal.
    8. Specify the region by entering ru-central1 in the terminal.
    9. Specify the endpoint by entering storage.yandexcloud.net in the terminal.
    10. You can leave all other settings at their defaults by pressing Enter to skip them.

Note

You can perform advanced connection setup if needed. To do this, type y at the Edit advanced config? step. For more information about advanced settings, see the rclone documentation.

Mount a bucket

  1. Check your connection to the bucket. In the same command line you used to configure the connection, run the following command with the bucket name specified:

    rclone.exe ls s3-connect:<bucket_name>

    If the configuration is set up correctly, the objects in the bucket will be listed in the console.

  2. Mount the bucket to the file system specifying the bucket name and an available drive letter in the file system:

    rclone.exe mount s3-connect:<bucket_name> <disk_letter>: --vfs-cache-mode full

    You will see a new disk with the objects from the bucket in Windows Explorer.

  3. To mount the bucket, press Ctrl + C.

Set up the mounting service

To mount the bucket at your desktop startup, set up mounting on behalf of the system service.

  1. In the WinSW utility folder, create a file named WinSW-x64.xml (WinSW-x86.xml if you have a 32-bit version of Windows) with the following contents:

    <service>
  <id>rclone</id>
  <name>rclone-s3-disk</name>
  <description>This service maps an S3 bucket as a system drive.</description>
  <executable>"<working_folder_location>\rclone.exe"</executable>
  <arguments>mount s3-connect:<bucket_name> <disk_letter>: --vfs-cache-mode full</arguments>
  <log mode="roll" />
  <onfailure action="restart" />
</service>

  2. In the same folder, open the command prompt as an admin user and run the following command:

    • If you have a 64-bit version of Windows:

      .\WinSW-x64.exe install .\WinSW-x64.xml

    • If you have a 32-bit version of Windows:

      .\WinSW-x86.exe install .\WinSW-x86.xml

  3. Open the Windows services panel and make sure rclone-s3-disk is listed:

    1. Press Win+R.
    2. In the window that opens, enter services.msc and click ОК.
    3. In the list of services, find rclone-s3-disk.

  4. Reboot your desktop and check that the disk is available.

Note

You can also configure the service to be run by a system user (for more information, see Service account in the WinSW utility documentation).

How to delete the resources you created

To stop paying for the resources you created:

Previous
goofys
Next
Docker container for (S)FTP(S)