Migrating data from a third-party MySQL® cluster

1. In the terminal window, go to the directory containing the infrastructure plan.

2. Delete the data-transfer-mysql-mmy.tf configuration file.

3. Make sure the Terraform configuration files are correct using this command:

   terraform validate
   

   If there are any errors in the configuration files, Terraform will point them out.

4. Confirm updating the resources.

   1. Run the command to view planned changes:

      terraform plan
      

      If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.

   2. If you are happy with the planned changes, apply them:

      1. Run the command:

         terraform apply
         

      2. Confirm the update of resources.

      3. Wait for the operation to complete.

   All the resources described in the configuration file data-transfer-mysql-mmy.tf will be deleted.

1. If you do not have Terraform yet, install it.

2. Get the authentication credentials. You can add them to environment variables or specify them later in the provider configuration file.

3. Configure and initialize a provider. There is no need to create a provider configuration file manually, you can download it.

4. Place the configuration file in a separate working directory and specify the parameter values. If you did not add the authentication credentials to environment variables, specify them in the configuration file.

5. Download the data-migration-mysql-mmy.tf configuration file to the same working directory.

   This file describes:

   • Network.
   • Subnet.
   • Security group and the rule required to connect to a cluster.
   • Managed Service for MySQL® cluster with public internet access.
   • (Optional) Virtual machine with public internet access.

6. Specify in the file data-migration-mysql-mmy.tf:

   • Target cluster parameters:

     • target_mysql_version: MySQL® version must be the same or higher than the version in the source cluster.
     • target_sql_mode: SQL mode must be the same as in the source cluster.
     • target_db_name: Database name.
     • target_user and target_password: Database owner username and password.

   • (Optional) Virtual machine parameters:

     • vm_image_id: ID of the public image with Ubuntu and no GPU, e.g., for Ubuntu 20.04 LTS.
     • vm_username and vm_public_key: Username and absolute path to a public key that will be used to access the virtual machine. By default, the specified username is ignored in the Ubuntu 20.04 LTS image. A user with the ubuntu username is created instead. Use it to connect to the instance.

7. Make sure the Terraform configuration files are correct using this command:

   terraform validate
   

   If there are any errors in the configuration files, Terraform will point them out.

8. Create the required infrastructure:

   1. Run the command to view planned changes:

      terraform plan
      

      If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.

   2. If you are happy with the planned changes, apply them:

      1. Run the command:

         terraform apply
         

      2. Confirm the update of resources.

      3. Wait for the operation to complete.

   All the required resources will be created in the specified folder. You can check resource availability and their settings in the management console.

1. Switch the database to read-only mode to avoid losing data that can appear while creating the dump.

2. Create a directory for the dump files:

   mkdir db_dump
   

3. Install the mydumper utility in the source cluster as follows (Ubuntu):

   sudo apt update && sudo apt install mydumper --yes
   

4. Create a database dump:

   mydumper \
       --triggers \
       --events \
       --routines \
       --outputdir=db_dump \
       --rows=10000000 \
       --threads=8 \
       --compress \
       --database=<DB_name> \
       --user=<username> \
       --ask-password \
       --host=<FQDN_or_IP>
   

   Where:

   • --triggers: Trigger dump.
   • --events: Event dump.
   • --routines: Stored procedure and function dump.
   • --outputdir: Dump file directory.
   • --rows: Number of rows in table fragments. The smaller the value, the more files in a dump.
   • --threads: Number of threads used. The recommended value is equal to half the server's free cores.
   • --compress: Output file compression.
   • --host: FQDN or IP of the source cluster's master host.

5. In the dump file, change the table engine names to InnoDB:

   sed -i -e 's/MyISAM/InnoDB/g' -e 's/MEMORY/InnoDB/g' `find /db_dump -name '*-schema.sql'`
   

6. Archive the dump:

   tar -cvzf db_dump.tar.gz ~/db_dump
   

This method is suitable if you created your dump using the mydumper utility and are using an intermediate virtual machine for the restore.

1. Install the myloader utility to the host that you are using to restore the dump as follows (Ubuntu):

   sudo apt update && sudo apt install mydumper --yes
   

2. Start the database restore from the dump:

   myloader \
       --host=c-<target_cluster_ID>.rw.mdb.yandexcloud.net \
       --directory=db_dump/ \
       --overwrite-tables \
       --threads=8 \
       --compress-protocol \
       --user=<username> \
       --ask-password
   

To delete the infrastructure created with Terraform:

1. In the terminal window, go to the directory containing the infrastructure plan.

2. Delete the data-migration-mysql-mmy.tf configuration file.

3. Make sure the Terraform configuration files are correct using this command:

   terraform validate
   

   If there are any errors in the configuration files, Terraform will point them out.

4. Confirm updating the resources.

   1. Run the command to view planned changes:

      terraform plan
      

      If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.

   2. If you are happy with the planned changes, apply them:

      1. Run the command:

         terraform apply
         

      2. Confirm the update of resources.

      3. Wait for the operation to complete.

   All the resources described in the configuration file data-migration-mysql-mmy.tf will be deleted.