Yandex Object Storage integration with Nextcloud

Nextcloud is an open-source software solution that allows you to easily, flexibly, and securely manage files, share them, and collaborate on documents. You can use Nextcloud create your own cloud storages as well as integrate off-the-shelf solutions, such as Yandex Object Storage.

In this tutorial, you will connect an Object Storage bucket to a Nextcloud solution deployed on a Yandex Compute Cloud VM instance with a database in a Yandex Managed Service for MySQL® cluster. To ensure fault tolerance and redundancy of your Nextcloud infrastructure, you will scale Nextcloud across an instance group and implement L7 load balancing with Yandex Application Load Balancer. In the fault-tolerant configuration, Nextcloud will be available via a domain name, for which a TLS certificate will be issued in Yandex Certificate Manager.

To deploy Nextcloud in Yandex Cloud and connect an Object Storage bucket:

1. Get your cloud ready.

2. Deploy Nextcloud in a basic configuration:

   1. Create and set up a cloud network.
   2. Create a service account and static access key.
   3. Create an Object Storage bucket.
   4. Create a VM.
   5. Create a Managed Service for MySQL® cluster.
   6. Install and configure Nextcloud on the VM.
   7. Test the solution in the basic configuration.

3. Deploy Nextcloud in a fault-tolerant configuration:

   1. Scale the Managed Service for MySQL® cluster.
   2. Complete the Nextcloud setup and create a VM disk snapshot.
   3. Add a TLS certificate to Yandex Certificate Manager.
   4. Deploy an instance group.
   5. Create an L7 load balancer.
   6. Create an A resource record in your domain’s public DNS zone.
   7. Test the solution in the fault-tolerant configuration.

If you no longer need the resources you created, delete them.

Getting startedGetting started

Sign up in Yandex Cloud and create a billing account:

1. Navigate to the management console and log in to Yandex Cloud or register a new account.
2. On the Yandex Cloud Billing page, make sure you have a billing account linked and it has the ACTIVE or TRIAL_ACTIVE status. If you do not have a billing account, create one and link a cloud to it.

If you have an active billing account, you can navigate to the cloud page to create or select a folder for your infrastructure to operate in.

Learn more about clouds and folders.

Required paid resourcesRequired paid resources

The cost of the proposed solution includes:

• Fee for disks, disk snapshots, and continuously running VMs (see Yandex Compute Cloud pricing).
• Fee for using public IP addresses and NAT gateways (see Yandex Virtual Private Cloud pricing).
• Fee for data storage in Object Storage and data operations (see Yandex Object Storage pricing).
• Fee for using a MySQL® managed DB (see Managed Service for MySQL® pricing).
• If using Yandex Cloud DNS, the fee for DNS zones and public DNS queries (see Cloud DNS pricing).
• If using a load balancer, the fee for the amount of L7 load balancer resource units (see Yandex Application Load Balancer pricing).
• If using a log group for load balancer logging, the fee for writing and storing data (see Yandex Cloud Logging pricing).

Deploy Nextcloud in a basic configurationDeploy Nextcloud in a basic configuration

You will deploy the basic Nextcloud configuration on a single VM with the Nextcloud service database created in a single-host Managed Service for MySQL® cluster or directly on the Nexcloud VM.

Set up a cloud networkSet up a cloud network

1. Create a cloud network and subnets for your new cloud infrastructure:

   Management console

   1. In the management console, select the folder where you are going to create your infrastructure.
   2. In the list of services, select Virtual Private Cloud.
   3. In the top-right corner, click Create network.
   4. In the Name field, enter a name for the network: nextcloud-network.
   5. Make sure the Create subnets option is enabled and click Create network.

2. Create a security group that allows the traffic required for your Nextcloud infrastructure:

   Management console

   1. In the management console, select the folder where you are deploying your infrastructure.

   2. In the list of services, select Virtual Private Cloud.

   3. In the left-hand panel, select Security groups.

   4. Click Create security group.

   5. In the Name field, enter nextcloud-sg for name.

   6. In the Network field, select nextcloud-network you created earlier.

   7. Under Rules, create the following traffic management rules:

      Traffic direction	Description	Port range	Protocol	Source / Destination name	CIDR blocks / Security group
      Ingress	http	80	TCP	CIDR	0.0.0.0/0
      Inbound	https	443	TCP	CIDR	0.0.0.0/0
      Inbound	ssh	22	TCP	CIDR	0.0.0.0/0
      Ingress	self	All	Any	Security group	Current
      Inbound	healthchecks	All	Any	Load balancer healthchecks	–
      Egress	any	All	Any	CIDR	0.0.0.0/0

   8. Click Create.

3. Create a NAT gateway:

   Your VMs need internet access to connect to Object Storage buckets. To avoid assigning public IP addresses to VM instances in the group, use a NAT gateway to enable internet access.

   Note

   You can also configure VM access to the bucket using service connections.

   This feature is in the Preview stage.

   Management console

   1. In the management console, select the folder where you are deploying your infrastructure.
   2. In the list of services, select Virtual Private Cloud.
   3. In the left-hand panel, select Gateways and click Create gateway.
   4. In the Name field, specify nextcloud-gateway as the gateway name.
   5. In the Type field, select Egress NAT and click Save.

4. Create a route table:

   Management console

   1. In the management console, select the folder where you are deploying your infrastructure.

   2. In the list of services, select Virtual Private Cloud.

   3. In the left-hand panel, select Routing tables and click Create routing table.

   4. In the Name field, specify nextcloud-rt-table.

   5. In the Network field, select nextcloud-network.

   6. Click Add, and in the window that opens:

      1. In the Next hop field, select Gateway.
      2. In the Gateway field, select the NAT gateway you created. The destination prefix will apply automatically.
      3. Click Add.

   7. Click Create routing table.

5. Link the route table to all subnets in nextcloud-network to route their outgoing traffic via the NAT gateway:

   Management console

   1. In the management console, select the folder where you are deploying your infrastructure.
   2. In the list of services, select Virtual Private Cloud.
   3. In the left-hand panel, select Subnets.
   4. In the subnet row, click and select Link routing table in the context menu.
   5. In the window that opens, select the route table you created in the previous step and click Link.

   Make sure the Routing table field for each listed subnet displays the name of the route table linked to it.

Create a service account and static access keyCreate a service account and static access key

To enable access from Nextcloud to the Object Storage bucket, create a service account and static access key.

1. Create a service account:

   Management console

   1. In the management console, select the folder where you are deploying your infrastructure.
   2. In the list of services, select Identity and Access Management.
   3. Click Create service account.
   4. Enter a name for the service account: nextcloud-sa.
   5. Click Add role and select the editor role.
   6. Click Create.

2. Create a static access key:

   Management console

   1. In the management console, select the folder where you are deploying your infrastructure.

   2. In the list of services, select Identity and Access Management.

   3. In the left-hand panel, select Service accounts and select the nextcloud-sa service account you created earlier.

   4. In the top panel, click Create new key and select Create static access key.

   5. Click Create.

   6. Save the ID and secret key.

      Alert

      After you close this dialog, the key value will not be shown again.

Create an Object Storage bucketCreate an Object Storage bucket

Create the Object Storage bucket you will connect to Nextcloud:

Create a VMCreate a VM

Create the VM to deploy Nextcloud on:

Create a Managed Service for MySQL® clusterCreate a Managed Service for MySQL® cluster

Nextcloud uses a MySQL® database to store service information. In this tutorial, you will deploy the Nextcloud database in a Yandex Managed Service for MySQL® cluster.

To create a Managed Service for MySQL® cluster:

It may take a few minutes to create a cluster.

Install and configure Nextcloud on the VMInstall and configure Nextcloud on the VM

1. Install Nextcloud on nextcloud-vm:

   1. Connect to nextcloud-vm over SSH. You will perform all further actions under this step in the VM terminal.

   2. Upgrade the versions of the packages installed on the VM:

      sudo apt update && sudo apt upgrade
      

   3. Install the required software packages and dependencies:

      sudo apt install \
        apache2 mariadb-server libapache2-mod-php php-gd php-mysql php-curl php-mbstring \
        php-intl php-gmp php-bcmath php-xml php-imagick php-zip php-fpm unzip
      

   4. Optionally, if you want to create the database on the same host as Nextcloud:

      If you did not create a MySQL® cluster and are not going to deploy a fault-tolerant solution:

      1. Run MySQL:

         sudo mysql
         

      2. Run these commands to create the database and user and grant the user the permissions to work with the database:

         CREATE USER 'user'@'localhost' IDENTIFIED BY '<password>';
         CREATE DATABASE IF NOT EXISTS nextcloud CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci;
         GRANT ALL PRIVILEGES ON nextcloud.* TO 'user'@'localhost';
         FLUSH PRIVILEGES;
         quit;
         

         Where <password> is the password you will use to access the database.

   5. Download the archive with the latest Nextcloud version:

      wget https://download.nextcloud.com/server/releases/latest.zip
      

   6. Unpack the archive to the /var/www directory:

      sudo unzip latest.zip -d /var/www
      

   7. Edit access permissions for the Nextcloud directory:

      sudo chown -R www-data:www-data /var/www/nextcloud
      sudo chmod -R 755 /var/www/nextcloud/
      

   8. Configure the default virtual host:

      1. Open the default virtual host's configuration file:

         sudo nano /etc/apache2/sites-available/000-default.conf
         

      2. Replace the contents of 000-default.conf with the following:

         <VirtualHost *:80>
         DocumentRoot /var/www/nextcloud/
         
         <Directory /var/www/nextcloud/>
         Require all granted
         AllowOverride All
         Options FollowSymLinks MultiViews
         
         <IfModule mod_dav.c>
         Dav off
         </IfModule>
         </Directory>
         </VirtualHost>
         

   9. Enable the required Apache web server modules:

      sudo a2enmod rewrite
      sudo a2enmod headers
      a2enmod env
      a2enmod dir
      a2enmod mime
      

   10. Increase the amount of RAM available to the PHP interpreter for processing requests to Nextcloud to 512 MB.

       1. Open the .htaccess file in the Nextcloud installation directory:

          sudo nano /var/www/nextcloud/.htaccess
          

       2. Add the following line to the end of the file:

          php_value memory_limit 512M
          

          Make sure to save your changes.

   11. Restart the web server:

       sudo systemctl restart apache2
       

2. Configure Nextcloud in the GUI:

   1. Open your local computer browser and enter the following in the address bar:

      http://<VM_public_IP_address>
      

      You can look up the VM's public IP address in the management console by checking the Network section's Public IPv4 address field on the VM information page.

   2. In the Create an administrator account form that opens:

      1. In the New administrator account name and New administrator password fields, set the Nextcloud administrator credentials you will use to log in to the system.

      2. In the Database account field, enter user, i.e., the database user name you specified when creating the MySQL® cluster or local database.

      3. In the Database password field, enter the database user password you specified when creating the MySQL® cluster or local database.

      4. In the Database name field, enter nextcloud, i.e., the database name you specified when creating the MySQL® cluster or local database.

      5. In the Database host field, specify the FQDN of the cluster’s current master host and port in this format:

         c-<cluster_ID>.rw.mdb.yandexcloud.net:3306
         

         You can look up the cluster ID in the management console by checking the ID field on the cluster information page.

         Note

         If it is not your intention to deploy a fault-tolerant solution and you did not create a MySQL® cluster, but created a database on nextcloud-vm instead, leave localhost in the Database host field.

      6. Click Install.

         This will start the deployment of the Nextcloud database in the MySQL® cluster. Wait for this process to complete.

   3. After the installation is over, click Skip in the window with recommended applications that opens. You can get back to installing the applications you need at a later time.

   4. Close the window with information on updates in the current Nextcloud version.

   5. Open the application management menu. Do it by clicking the user icon in the top-right corner and selecting Applications in the context menu that pops open.

   6. In the window that opens, select Your applications in the left-hand panel.

   7. In the list that opens, find the External storage support application and click Enable in the row with it.

      If needed, enter your Nextcloud administrator password in the pop-up window to confirm the operation.

   8. Open the main settings menu. Do it by clicking the user icon in the top-right corner and selecting Server parameters in the context menu that pops open.

   9. In the window that opens, select External storage in the left-hand panel under Server parameters and specify Object Storage integration settings under External storage in the window that opens:

      1. In the External storage section, select Amazon S3.

      2. In the Authorization method section, select Access key.

      3. Under Configuration:

         • In the Bucket field, enter a name for the bucket you created earlier, e.g., my-nextcloud-bucket.
         • In the Hostname field, specify storage.yandexcloud.net.
         • In the Port field, specify 443.
         • In the Access key field, paste the ID of the static access key you created earlier.
         • In the Secret key field, paste the secret key of the static access key you created earlier.

      4. Under Available to, enable All.

      5. On the right side of the section you are editing, click the icon to save your changes.

         Enter your Nextcloud administrator password in the pop-up window to confirm the operation.

Test the solution in the basic configurationTest the solution in the basic configuration

To test Yandex Object Storage integration with Nextcloud on a single host:

1. Open your local computer browser and enter the public IPv4 address of the Nextcloud VM in the address bar:

   http://<VM_public_IP_address>
   

2. Get authenticated in Nextcloud using the login and password created when configuring the solution in the previous step.

3. In the left part of the top menu, select Files.

4. In the left-hand menu, select External storage and then AmazonS3.

5. Click New and select Upload files to upload a file from your local computer to the storage.

6. Select a file on your local computer and upload it to the storage.

   The uploaded file will now appear in the Nextcloud storage named AmazonS3.

7. In Yandex Object Storage, make sure the file was uploaded to the bucket.

The deployment of the Nextcloud basic configuration is now complete. If you used a MySQL® cluster in the basic configuration, you can now proceed to deploy a fault-tolerant configuration.

Deploy Nextcloud in a fault-tolerant configurationDeploy Nextcloud in a fault-tolerant configuration

You will deploy a fault-tolerant Nextcloud configuration in a three-VM group of three VMs, the load on Nextcloud hosts distributed with the help of an L7 Yandex Application Load Balancer. The service database will reside in a three-host MySQL® cluster. Hosts of the instance group, load balancer, and MySQL® cluster will be evenly distributed across three availability zones. Nextcloud will be available via the domain name, for which a TLS certificate will be issued in Certificate Manager.

Scale the Managed Service for MySQL® clusterScale the Managed Service for MySQL® cluster

To add more hosts to a Managed Service for MySQL® cluster:

Complete Nextcloud setup and create a VM disk snapshotComplete Nextcloud setup and create a VM disk snapshot

Before you create a VM snapshot to proceed with the instance group deployment, add your domain to the list of trusted addresses and Nextcloud domains:

1. Connect to nextcloud-vm over SSH.

2. Add your domain to the array of trusted addresses and Nextcloud domains:

   1. In the VM terminal, open the Nextcloud configuration file:

      sudo nano /var/www/nextcloud/config/config.php
      

   2. In the trusted_domains array, replace the host IP address with your domain name.

      Here is an example:

      'trusted_domains' =>
      array (
        0 => 'example.com',
      ),
      

      The trusted_domains array allows you to restrict the range of IP addresses and/or domains you can use to access Nextcloud and ensures additional protection from unauthorized access. You can specify multiple addresses and/or domains, and you can also reduce or remove this restriction using wildcard characters:

      Example 1

      Example 2

      'trusted_domains' =>
      array (
        0 => '*.example.com',
        1 => '198.168.*.*',
      ),
      

      In this example, access is allowed from any subdomains of the example.com domain and the IP addresses of the 192.168.0.0 - 192.168.255.255 range.

      'trusted_domains' =>
      array (
        0 => '*',
      ),
      

      In this example, access is allowed from any domains and IP addresses.

   3. Delete the 'overwrite.cli.url' => 'http://<VM_IP_address>', line.

   4. Save the changes and close the nano editor.

3. Stop the nextcloud-vm virtual machine.

4. After the VM stops, create a snapshot of its disk:

   Management console

   1. In the management console, select the folder to create the infrastructure in.

   2. In the list of services, select Compute Cloud.

   3. In the left-hand panel, select Disks.

   4. In the line with the disk, click and select Create snapshot.

   5. In the Name field, specify nextcloud-vm-snapshot as the snapshot name.

   6. Click Create.

   7. Wait until the snapshot creation process is over.

      To track the snapshot status, select Snapshots in the left-hand panel.

Add a TLS certificate to Yandex Certificate ManagerAdd a TLS certificate to Yandex Certificate Manager

To enable access to Nextcloud over HTTPS, issue a TLS certificate for your domain:

1. Add a Let's Encrypt® certificate for your domain to Certificate Manager:

   Management console

   1. In the management console, select the folder where you are deploying your infrastructure.
   2. In the list of services, select Certificate Manager.
   3. Click Add certificate and select Let's Encrypt certificate.
   4. In the window that opens, specify mymanagedcert in the Name field.
   5. In the Domains field, enter a name for your domain, e.g., example.com.
   6. In the Check type field, select DNS and click Create.

   A new certificate with the Validating status will appear in the certificate list. This status means that a Let's Encrypt® certificate was requested and you need to pass a domain rights check for the request to be successfully processed.

2. For the certificate to be issued successfully, pass a domain rights check:

   Management console

   1. In the management console, select the folder where you are deploying your infrastructure.

   2. In the list of services, select Certificate Manager.

   3. From the list of certificates, select mymanagedcert.

   4. In the window that opens, under Check rights for domains, select CNAME record.

   5. Add a CNAME record required for the domain rights check to your domain’s public DNS zone. This step's further actions will depend on whether your domain is managed by Yandex Cloud DNS or a third-party DNS provider:

      Yandex Cloud DNS

      Third-party DNS provider

      Under Check rights for domains, in the CNAME record section, click Create record. In the window that opens:

      1. If the current folder contains an appropriate DNS zone, it will be automatically inserted into the Zone field. If there is no appropriate DNS zone, click Create zone and set its parameters to create a new zone.
      2. Click Create.

      1. Under Check rights for domains, in the CNAME record section, check out the record value for the domain in the Value field.

      2. Add a CNAME record to your DNS provider or to your own DNS server to delegate management privileges to the DNS zone used for the check:

         _acme-challenge.example.com CNAME <value>
         

         The <value> string is formatted as follows: <certificate_ID>.cm.yandexcloud.net.

      Note

      For a successful DNS domain rights check based on a CNAME record, make sure the _acme-challenge subdomain of the domain name you are checking has no other resource records except CNAME. For example, for the _acme-challenge.example.com. domain name, there should only be a CNAME record and no TXT record.

      The domain rights check may take from a few minutes to a few days. Wait until it is complete. As a result, the certificate will be issued and get the Issued status.

Deploy an instance groupDeploy an instance group

For a fault-tolerant Nextcloud solution, scale it across an instance group:

Create an L7 load balancerCreate an L7 load balancer

Create an application-level load balancer for traffic distribution between the hosts of the future Nextcloud instance group:

1. Create a backend group:

   Management console

   1. In the management console, select the folder where you are deploying your infrastructure.

   2. In the list of services, select Application Load Balancer.

   3. In the left-hand panel, select Backend groups and click Create backend group.

   4. In the Name field, enter a name for the group: nextcloud-bg.

   5. Enable Session affinity and select By IP address in the Affinity mode field that appears.

   6. Under Backends, click Add and do the following in the New backend form that opens:

      1. In the Name field, specify nextcloud-backend.
      2. In the Target groups field, select the nextcloud-target-group target group you created earlier.
      3. Expand the Load balancing settings section and select MAGLEV_HASH in the Balancing mode field.
      4. In the HTTP health check, click and select Delete.

   7. Click Create.

2. Create an HTTP router:

   Management console

   1. In the management console, select the folder where you are deploying your infrastructure.

   2. In the list of services, select Application Load Balancer.

   3. In the left-hand panel, select HTTP routers and click Create HTTP router.

   4. In the Name field, enter a name for the HTTP router: nextcloud-router.

   5. Under Virtual hosts, click Add virtual host and do the following in the New virtual host form that opens:

      1. In the Name field, enter nextcloud-vh for the virtual host's name and click Add route.
      2. In the New route form that opens, enter nextcloud-route as the route's name in the Name field.
      3. In the Backend group field, select the nextcloud-bg backend group created in the previous step.
      4. Leave all other parameters as they are and click Save at the bottom of the page.

3. Create an L7 load balancer:

   Management console

   1. In the management console, select the folder where you are deploying your infrastructure.

   2. In the list of services, select Application Load Balancer.

   3. Click Create L7 load balancer and select Manual.

   4. In the Name field, enter a name for the load balancer: nextcloud-alb.

   5. In the Network field, select nextcloud-network.

   6. In the Security groups field, select From list and then the nextcloud-sg security group from the list that opens.

   7. Under Allocation, make sure all availability zones are selected.

   8. If you do not want load balancer logs saved to a log group, disable Write logs.

   9. In the Listeners section, click Add listener and in the form that opens:

      1. In the Name field, enter a name for the listener: nextcloud-listener.
      2. In the Protocol field, select HTTPS.
      3. In the Certificates field, select the mymanagedcert certificate you created earlier.
      4. In the HTTP router field, select the nextcloud-router HTTP router you created earlier.

   10. Leave all other parameters as they are and click Save at the bottom of the page.

   Wait for the load balancer status to change to Active and copy its IP address from the IP addresses field.

Create an A resource record in your domain’s public DNS zoneCreate an A resource record in your domain’s public DNS zone

To route your domain's incoming requests to an L7 load balancer, in your DNS zone, create an A resource record pointing to the load balancer's IP address you got in the previous step. Further steps to follow will depend on whether your domain is managed by Yandex Cloud DNS or a third-party DNS provider.

If your domain is managed by Yandex Cloud DNS

Management console

1. In the management console, select the folder where you are deploying your infrastructure.

2. In the list of services, select Cloud DNS.

3. Select the DNS zone you need, click Create record, and do the following in the window that opens:

   1. In the Name field, select Matches zone name (@).
   2. In the Type field, select A.
   3. In the Data field, specify the load balancer IP address you saved in the previous step.
   4. Leave other parameters as they are and click Save.

If your domain is managed by an external DNS provider

Add an A resource record with the following properties to your DNS provider or your own DNS server:

• Host: @
• Type: A
• Value: <load_balancer_IP_address>

Test the solution in the fault-tolerant configurationTest the solution in the fault-tolerant configuration

To test Yandex Object Storage integration with Nextcloud in a fault-tolerant configuration:

1. Open your local computer browser and enter your domain name in the address bar, e.g.:

   https://example.com
   

2. Get authenticated in Nextcloud using the login and password created when configuring the solution.

3. In the left part of the top menu, select Files.

4. In the left-hand menu, select External storage and then AmazonS3.

5. Make sure you see the file uploaded in the previous step.

6. Download the file you uploaded earlier. Do this by clicking and selecting Download in the line with the filename.

7. Delete the file. Do this by clicking and selecting Delete file in the line with the filename.

8. In Yandex Object Storage, make sure the file was deleted from the bucket.

How to delete the resources you createdHow to delete the resources you created

1. If you had created an L7 load balancer, delete it and then delete the HTTP router, backend group, and the load balancer's target group one by one.
2. If you had created an instance group, delete it.
3. If you had created resource records, delete them in Yandex Cloud DNS or in your domain registrar’s account.
4. If you had created a DNS zone, delete it in Yandex Cloud DNS or in your domain registrar’s account.
5. If you had left the L7 load balancer logging feature on, delete the log group.
6. Delete the VM.
7. Delete the MySQL® cluster database.
8. Delete the objects you created in the bucket, then delete the bucket itself.
9. Delete the disk snapshot.
10. Delete the service account.
11. Delete the subnets, security group, route table, NAT gateway, and cloud network one by one.
12. Optionally, delete the TLS certificate if you created one.