Table recognition

Written by

Updated at November 20, 2024

Getting started
Recognizing a table

You can recognize text in a table using the OCR API with the table recognition model. With this model, you can recognize tables in Russian, English, and Turkish.

Getting started

To use the examples, install cURL.

Get your account data for authentication:

Yandex or federated account

Service account

Get an IAM token for your Yandex account or federated account.
Get the ID of the folder for which your account has the ai.vision.user role or higher.
When accessing Vision OCR via the API, provide the received parameters in each request:
- For the Vision API and Classifier API:
  
  Specify the IAM token in the Authorization header as follows:
```
Authorization: Bearer <IAM_token>
```
  Specify the folder ID in the request body in the folderId parameter.
- For the OCR API:
  - Specify the IAM token in the Authorization header.
  - Specify the folder ID in the x-folder-id header.
```
Authorization: Bearer <IAM_token>
x-folder-id <folder_ID>
```

Vision OCR supports two authentication methods based on service accounts:

With an IAM token:
1. Get an IAM token.
2. Provide the IAM token in the Authorization header in the following format:
```
Authorization: Bearer <IAM_token>
```
With API keys.

Use API keys if requesting an IAM token automatically is not an option.
1. Get an API key.
2. Provide the API key in the Authorization header in the following format:
```
Authorization: Api-Key <API key>
```

Do not specify the folder ID in your requests, as the service uses the folder the service account was created in.

Recognizing a table

Image text recognition is implemented in the recognize OCR API method.

Prepare an image file that meets the requirements:
- The supported file formats are JPEG, PNG, and PDF. Specify the MIME type of the file in the mime_type property. The default value is image.
- The maximum file size is 10 MB.
- The image size should not exceed 20 MP (height × width).

Encode the image file as Base64:

UNIX

Windows

PowerShell

Python

Node.js

Java

base64 -i input.jpg > output.txt

C:> Base64.exe -e input.jpg > output.txt

[Convert]::ToBase64String([IO.File]::ReadAllBytes("./input.jpg")) > output.txt

# Import a library for encoding files in Base64
import base64

# Create a function that will encode a file and return results.
def encode_file(file_path):
  with open(file_path, "rb") as fid:
      file_content = fid.read()
  return base64.b64encode(file_content).decode("utf-8")

// Read the file contents to memory.
var fs = require('fs');
var file = fs.readFileSync('/path/to/file');

// Get the file contents in Base64 format.
var encoded = Buffer.from(file).toString('base64');

// Import a library for encoding files in Base64.
import org.apache.commons.codec.binary.Base64;

// Get the file contents in Base64 format.
byte[] fileData = Base64.encodeBase64(yourFile.getBytes());

import (
    "bufio"
    "encoding/base64"
    "io/ioutil"
    "os"
)

// Open the file.
f, _ := os.Open("/path/to/file")

// Read the file contents.
reader := bufio.NewReader(f)
content, _ := ioutil.ReadAll(reader)

// Get the file contents in Base64 format.
base64.StdEncoding.EncodeToString(content)

Create a file with the request body, e.g., body.json.

body.json:

{
  "mimeType": "JPEG",
  "languageCodes": ["ru","en"],
  "model": "table",
  "content": "<base64-encoded_image>"
}

In the content property, specify the image file contents encoded as Base64.

UNIX

Python

export IAM_TOKEN=<IAM_token>
curl \
  --request POST \
  --header "Content-Type: application/json" \
  --header "Authorization: Bearer ${IAM_TOKEN}" \
  --header "x-folder-id: <folder_ID>" \
  --header "x-data-logging-enabled: true" \
  --data "@body.json" \
  https://ocr.api.cloud.yandex.net/ocr/v1/recognizeText \
  --output output.json

Where:

<IAM_token>: Previously obtained IAM token.
<folder_ID>: Previously obtained folder ID.

data = {"mimeType": <mime_type>,
        "languageCodes": ["*"],
        "content": content}

url = "https://ocr.api.cloud.yandex.net/ocr/v1/recognizeText"

headers= {"Content-Type": "application/json",
          "Authorization": "Bearer {:s}".format(<IAM_token>),
          "x-folder-id": "<folder_ID>",
          "x-data-logging-enabled": "true"}
  w = requests.post(url=url, headers=headers, data=json.dumps(data))

The result will consist of recognized blocks of text, lines, and words with their position on the image.

Result:

{
  "result": {
    "textAnnotation": {
      "width": "724",
      "height": "131",
      "blocks": [
        {
          "boundingBox": {
            "vertices": [
              {
                "x": "68",
                "y": "42"
              },
              {
                "x": "68",
                "y": "60"
              },
              {
                "x": "194",
                "y": "60"
              },
              {
                "x": "194",
                "y": "42"
              }
            ]
          },
          "lines": [
            {
              "boundingBox": {
                "vertices": [
                  {
                    "x": "68",
                    "y": "42"
                  },
                  {
                    "x": "68",
                    "y": "60"
                  },
                  {
                    "x": "194",
                    "y": "60"
                  },
                  {
                    "x": "194",
                    "y": "42"
                  }
                ]
              },
              "text": "Фамилия",
              "words": [
                {
                  "boundingBox": {
                    "vertices": [
                      {
                        "x": "68",
                        "y": "40"
                      },
                      {
                        "x": "68",
                        "y": "63"
                      },
                      {
                        "x": "194",
                        "y": "63"
                      },
                      {
                        "x": "194",
                        "y": "40"
                      }
                    ]
                  },
                  "text": "Фамилия",
                  "entityIndex": "-1",
                  "textSegments": [
                    {
                      "startIndex": "0",
                      "length": "7"
                    }
                  ]
                }
              ],
              "textSegments": [
                {
                  "startIndex": "0",
                  "length": "7"
                }
              ]
            }
          ],
          "languages": [
            {
              "languageCode": "ru"
            }
          ],
          "textSegments": [
            {
              "startIndex": "0",
              "length": "7"
            }
          ]
        },
        {
          "boundingBox": {
            "vertices": [
              {
                "x": "307",
                "y": "40"
              },
              {
                "x": "307",
                "y": "61"
              },
              {
                "x": "372",
                "y": "61"
              },
              {
                "x": "372",
                "y": "40"
              }
            ]
          },
          "lines": [
            {
              "boundingBox": {
                "vertices": [
                  {
                    "x": "307",
                    "y": "40"
                  },
                  {
                    "x": "307",
                    "y": "61"
                  },
                  {
                    "x": "372",
                    "y": "61"
                  },
                  {
                    "x": "372",
                    "y": "40"
                  }
                ]
              },
              "text": "Имя",
              "words": [
                {
                  "boundingBox": {
                    "vertices": [
                      {
                        "x": "307",
                        "y": "39"
                      },
                      {
                        "x": "307",
                        "y": "63"
                      },
                      {
                        "x": "372",
                        "y": "63"
                      },
                      {
                        "x": "372",
                        "y": "39"
                      }
                    ]
                  },
                  "text": "Имя",
                  "entityIndex": "-1",
                  "textSegments": [
                    {
                      "startIndex": "8",
                      "length": "3"
                    }
                  ]
                }
              ],
              "textSegments": [
                {
                  "startIndex": "8",
                  "length": "3"
                }
              ]
            }
          ],
          "languages": [
            {
              "languageCode": "ru"
            }
          ],
          "textSegments": [
            {
              "startIndex": "8",
              "length": "3"
            }
          ]
        },
        {
          "boundingBox": {
            "vertices": [
              {
                "x": "506",
                "y": "43"
              },
              {
                "x": "506",
                "y": "59"
              },
              {
                "x": "632",
                "y": "59"
              },
              {
                "x": "632",
                "y": "43"
              }
            ]
          },
          "lines": [
            {
              "boundingBox": {
                "vertices": [
                  {
                    "x": "506",
                    "y": "43"
                  },
                  {
                    "x": "506",
                    "y": "59"
                  },
                  {
                    "x": "632",
                    "y": "59"
                  },
                  {
                    "x": "632",
                    "y": "43"
                  }
                ]
              },
              "text": "Отчество",
              "words": [
                {
                  "boundingBox": {
                    "vertices": [
                      {
                        "x": "506",
                        "y": "40"
                      },
                      {
                        "x": "506",
                        "y": "63"
                      },
                      {
                        "x": "632",
                        "y": "63"
                      },
                      {
                        "x": "632",
                        "y": "40"
                      }
                    ]
                  },
                  "text": "Отчество",
                  "entityIndex": "-1",
                  "textSegments": [
                    {
                      "startIndex": "12",
                      "length": "8"
                    }
                  ]
                }
              ],
              "textSegments": [
                {
                  "startIndex": "12",
                  "length": "8"
                }
              ]
            }
          ],
          "languages": [
            {
              "languageCode": "ru"
            }
          ],
          "textSegments": [
            {
              "startIndex": "12",
              "length": "8"
            }
          ]
        },
        {
          "boundingBox": {
            "vertices": [
              {
                "x": "80",
                "y": "81"
              },
              {
                "x": "80",
                "y": "98"
              },
              {
                "x": "179",
                "y": "98"
              },
              {
                "x": "179",
                "y": "81"
              }
            ]
          },
          "lines": [
            {
              "boundingBox": {
                "vertices": [
                  {
                    "x": "80",
                    "y": "81"
                  },
                  {
                    "x": "80",
                    "y": "98"
                  },
                  {
                    "x": "179",
                    "y": "98"
                  },
                  {
                    "x": "179",
                    "y": "81"
                  }
                ]
              },
              "text": "Иванов",
              "words": [
                {
                  "boundingBox": {
                    "vertices": [
                      {
                        "x": "80",
                        "y": "79"
                      },
                      {
                        "x": "80",
                        "y": "101"
                      },
                      {
                        "x": "179",
                        "y": "101"
                      },
                      {
                        "x": "179",
                        "y": "79"
                      }
                    ]
                  },
                  "text": "Иванов",
                  "entityIndex": "-1",
                  "textSegments": [
                    {
                      "startIndex": "21",
                      "length": "6"
                    }
                  ]
                }
              ],
              "textSegments": [
                {
                  "startIndex": "21",
                  "length": "6"
                }
              ]
            }
          ],
          "languages": [
            {
              "languageCode": "ru"
            }
          ],
          "textSegments": [
            {
              "startIndex": "21",
              "length": "6"
            }
          ]
        },
        {
          "boundingBox": {
            "vertices": [
              {
                "x": "302",
                "y": "80"
              },
              {
                "x": "302",
                "y": "98"
              },
              {
                "x": "370",
                "y": "98"
              },
              {
                "x": "370",
                "y": "80"
              }
            ]
          },
          "lines": [
            {
              "boundingBox": {
                "vertices": [
                  {
                    "x": "302",
                    "y": "80"
                  },
                  {
                    "x": "302",
                    "y": "98"
                  },
                  {
                    "x": "370",
                    "y": "98"
                  },
                  {
                    "x": "370",
                    "y": "80"
                  }
                ]
              },
              "text": "Иван",
              "words": [
                {
                  "boundingBox": {
                    "vertices": [
                      {
                        "x": "302",
                        "y": "78"
                      },
                      {
                        "x": "302",
                        "y": "101"
                      },
                      {
                        "x": "370",
                        "y": "101"
                      },
                      {
                        "x": "370",
                        "y": "78"
                      }
                    ]
                  },
                  "text": "Иван",
                  "entityIndex": "-1",
                  "textSegments": [
                    {
                      "startIndex": "28",
                      "length": "4"
                    }
                  ]
                }
              ],
              "textSegments": [
                {
                  "startIndex": "28",
                  "length": "4"
                }
              ]
            }
          ],
          "languages": [
            {
              "languageCode": "ru"
            }
          ],
          "textSegments": [
            {
              "startIndex": "28",
              "length": "4"
            }
          ]
        },
        {
          "boundingBox": {
            "vertices": [
              {
                "x": "504",
                "y": "82"
              },
              {
                "x": "504",
                "y": "98"
              },
              {
                "x": "630",
                "y": "98"
              },
              {
                "x": "630",
                "y": "82"
              }
            ]
          },
          "lines": [
            {
              "boundingBox": {
                "vertices": [
                  {
                    "x": "504",
                    "y": "82"
                  },
                  {
                    "x": "504",
                    "y": "98"
                  },
                  {
                    "x": "630",
                    "y": "98"
                  },
                  {
                    "x": "630",
                    "y": "82"
                  }
                ]
              },
              "text": "Иванович",
              "words": [
                {
                  "boundingBox": {
                    "vertices": [
                      {
                        "x": "504",
                        "y": "80"
                      },
                      {
                        "x": "504",
                        "y": "101"
                      },
                      {
                        "x": "630",
                        "y": "101"
                      },
                      {
                        "x": "630",
                        "y": "80"
                      }
                    ]
                  },
                  "text": "Иванович",
                  "entityIndex": "-1",
                  "textSegments": [
                    {
                      "startIndex": "33",
                      "length": "8"
                    }
                  ]
                }
              ],
              "textSegments": [
                {
                  "startIndex": "33",
                  "length": "8"
                }
              ]
            }
          ],
          "languages": [
            {
              "languageCode": "ru"
            }
          ],
          "textSegments": [
            {
              "startIndex": "33",
              "length": "8"
            }
          ]
        }
      ],
      "entities": [],
      "tables": [
        {
          "boundingBox": {
            "vertices": [
              {
                "x": "34",
                "y": "30"
              },
              {
                "x": "687",
                "y": "41"
              },
              {
                "x": "686",
                "y": "119"
              },
              {
                "x": "33",
                "y": "108"
              }
            ]
          },
          "rowCount": "2",
          "columnCount": "3",
          "cells": [
            {
              "boundingBox": {
                "vertices": [
                  {
                    "x": "35",
                    "y": "30"
                  },
                  {
                    "x": "242",
                    "y": "33"
                  },
                  {
                    "x": "241",
                    "y": "64"
                  },
                  {
                    "x": "34",
                    "y": "61"
                  }
                ]
              },
              "rowIndex": "0",
              "columnIndex": "0",
              "columnSpan": "1",
              "rowSpan": "1",
              "text": "Фамилия",
              "textSegments": [
                {
                  "startIndex": "0",
                  "length": "7"
                }
              ]
            },
            {
              "boundingBox": {
                "vertices": [
                  {
                    "x": "242",
                    "y": "33"
                  },
                  {
                    "x": "444",
                    "y": "37"
                  },
                  {
                    "x": "444",
                    "y": "67"
                  },
                  {
                    "x": "241",
                    "y": "64"
                  }
                ]
              },
              "rowIndex": "0",
              "columnIndex": "1",
              "columnSpan": "1",
              "rowSpan": "1",
              "text": "Имя",
              "textSegments": [
                {
                  "startIndex": "8",
                  "length": "3"
                }
              ]
            },
            {
              "boundingBox": {
                "vertices": [
                  {
                    "x": "444",
                    "y": "37"
                  },
                  {
                    "x": "686",
                    "y": "40"
                  },
                  {
                    "x": "685",
                    "y": "71"
                  },
                  {
                    "x": "444",
                    "y": "67"
                  }
                ]
              },
              "rowIndex": "0",
              "columnIndex": "2",
              "columnSpan": "1",
              "rowSpan": "1",
              "text": "Отчество",
              "textSegments": [
                {
                  "startIndex": "12",
                  "length": "8"
                }
              ]
            },
            {
              "boundingBox": {
                "vertices": [
                  {
                    "x": "34",
                    "y": "61"
                  },
                  {
                    "x": "241",
                    "y": "64"
                  },
                  {
                    "x": "241",
                    "y": "108"
                  },
                  {
                    "x": "34",
                    "y": "105"
                  }
                ]
              },
              "rowIndex": "1",
              "columnIndex": "0",
              "columnSpan": "1",
              "rowSpan": "1",
              "text": "Иванов",
              "textSegments": [
                {
                  "startIndex": "21",
                  "length": "6"
                }
              ]
            },
            {
              "boundingBox": {
                "vertices": [
                  {
                    "x": "241",
                    "y": "64"
                  },
                  {
                    "x": "444",
                    "y": "67"
                  },
                  {
                    "x": "443",
                    "y": "111"
                  },
                  {
                    "x": "241",
                    "y": "108"
                  }
                ]
              },
              "rowIndex": "1",
              "columnIndex": "1",
              "columnSpan": "1",
              "rowSpan": "1",
              "text": "Иван",
              "textSegments": [
                {
                  "startIndex": "28",
                  "length": "4"
                }
              ]
            },
            {
              "boundingBox": {
                "vertices": [
                  {
                    "x": "444",
                    "y": "67"
                  },
                  {
                    "x": "685",
                    "y": "71"
                  },
                  {
                    "x": "684",
                    "y": "115"
                  },
                  {
                    "x": "443",
                    "y": "111"
                  }
                ]
              },
              "rowIndex": "1",
              "columnIndex": "2",
              "columnSpan": "1",
              "rowSpan": "1",
              "text": "Иванович",
              "textSegments": [
                {
                  "startIndex": "33",
                  "length": "8"
                }
              ]
            }
          ]
        }
      ],
      "fullText": "Фамилия\nИмя\nОтчество\nИванов\nИван\nИванович\n"
    },
    "page": "0"
  }
}

To get all the recognized words in an image, find all the values with the text property.

Note

If the coordinates you got do not match the position of displayed elements, set up support for exif metadata in your image viewing tool or remove the Orientation attribute from the exif image section when running a transfer to the service.

Table recognition

Getting startedGetting started

Recognizing a tableRecognizing a table

Was the article helpful?

Getting started

Recognizing a table