Scheduled image and PDF recognition in an Object Storage bucket

In this tutorial, you will learn how to use Yandex Vision OCR to set up automatic recognition of new images (in supported formats) and PDFs in an bucket.

Recognition stepsRecognition steps

1. The user uploads images or documents to the input directory (prefix) in a Yandex Object Storage bucket.
2. The Yandex Cloud Functions trigger, is activated by timer and checks for new files in the input folder. Next, the files are sent for recognition to the Yandex Serverless Containers container.
3. File recognition is in progress, the operation ID is saved in the process folder in the source bucket.
4. Once the operation is successfully completed, the recognition results are saved to the result folder as JSON and TXT files. The ID of the successful operation is deleted from the process folder.

The infrastructure is built using the Yandex Cloud Terraform provider. The source code for this guide is available on GitHub.

To set up automatic image recognition via Vision OCR:

1. Get your cloud ready.
2. Create the infrastructure.
3. Upload files for recognition and check how the Vision OCR service works.

If you no longer need the resources you created, delete them.

Get your cloud readyGet your cloud ready

Sign up for Yandex Cloud and create a billing account:

1. Navigate to the management console and log in to Yandex Cloud or create a new account.
2. On the Yandex Cloud Billing page, make sure you have a billing account linked and it has the ACTIVE or TRIAL_ACTIVE status. If you do not have a billing account, create one and link a cloud to it.

If you have an active billing account, you can create or select a folder for your infrastructure on the cloud page.

Learn more about clouds and folders here.

Required paid resourcesRequired paid resources

The infrastructure support cost for scheduled image and document recognition includes:

• Fee for bucket data storage and bucket data operations (see Object Storage pricing).
• Fee for using Vision OCR (see Vision OCR pricing).
• Fee for the container invocation count, computing resources allocated for the application, and outgoing traffic (see Serverless Containers pricing).
• Fee for storing the secret and operations with it (see Yandex Lockbox pricing).

Create the infrastructureCreate the infrastructure

With Terraform, you can quickly create a cloud infrastructure in Yandex Cloud and manage it using configuration files. These files store the infrastructure description written in HashiCorp Configuration Language (HCL). If you change the configuration files, Terraform automatically detects which part of your configuration is already deployed, and what should be added or removed.

Terraform is distributed under the Business Source License. The Yandex Cloud provider for Terraform is distributed under the MPL-2.0 license.

For more information about the provider resources, see the guides on the Terraform website or its mirror.

To create your infrastructure via Terraform:

1. Install Terraform, get the authentication credentials, and specify the source for installing the Yandex Cloud provider (see Configure your provider, Step 1).

2. Prepare your infrastructure description files:

   1. Clone the repository containing the configuration files.

      git clone https://github.com/yandex-cloud-examples/yc-vision-ocr-recognizer.git
      

   2. Navigate to the terraform directory within the repository.

   3. In the variables.auto.tfvars file, specify these custom settings:

      • cloud_id: Cloud ID.
      • folder_id: Folder ID.
      • zone: Availability zone.

3. Create the resources:

   1. In the terminal, navigate to the configuration file directory.

   2. Make sure the configuration is correct using this command:

      terraform validate
      

      If the configuration is valid, you will get this message:

      Success! The configuration is valid.
      

   3. Run this command:

      terraform plan
      

      You will see a list of resources and their properties. No changes will be made at this step. Terraform will show any errors in the configuration.

   4. Apply the configuration changes:

      terraform apply
      

   5. Type yes and press Enter to confirm the changes.

The system will create a bucket named ocr-recognition-....

Upload files for recognition and check how the Vision OCR service worksUpload files for recognition and check how the Vision OCR service works

1. Upload the files for recognition to the input folder inside the bucket you created earlier.
2. Open the result folder in your bucket. The folder should contain the recognition results as TXT and JSON files.

How to delete the resources you createdHow to delete the resources you created

To stop paying for the resources you created:

1. Delete all files from the bucket.

2. Open the main.tf configuration file and delete your infrastructure description from it.

3. Apply the changes:

   1. In the terminal, navigate to the configuration file directory.

   2. Make sure the configuration is correct using this command:

      terraform validate
      

      If the configuration is valid, you will get this message:

      Success! The configuration is valid.
      

   3. Run this command:

      terraform plan
      

      You will see a list of resources and their properties. No changes will be made at this step. Terraform will show any errors in the configuration.

   4. Apply the configuration changes:

      terraform apply
      

   5. Type yes and press Enter to confirm the changes.