Contact UsGet started
© 2024 Direct Cursus Technology L.L.C.

URL shortener

Written by
Updated at November 26, 2024

With this script, you will create a URL shortening service using serverless technologies available in Yandex Cloud.

The service accepts user requests via a public API gateway. The hosting service sends the user an HTML page with a field for entering the URL. The function sends the entered URL for storage in a serverless database, shortens it, and returns it to the user. When the user enters the shortened URL, the function finds the full URL in the database and redirects the user's request to it.

To configure and test the service:

  1. Prepare your cloud.
  2. Set up hosting for the URL shortener page.
  3. Create a service account.
  4. Create a database in Yandex Managed Service for YDB.
  5. Set up a function in Yandex Cloud Functions.
  6. Publish the service via Yandex API Gateway.
  7. Test the URL shortener.

If you no longer need the resources you created, delete them.

Prepare your cloud

Sign up for Yandex Cloud and create a billing account:

  1. Go to the management console and log in to Yandex Cloud or create an account if you do not have one yet.
  2. On the Yandex Cloud Billing page, make sure you have a billing account linked and it has the ACTIVE or TRIAL_ACTIVE status. If you do not have a billing account, create one.

If you have an active billing account, you can go to the cloud page to create or select a folder for your infrastructure to operate in.

Learn more about clouds and folders.

The cost of resources for the script includes:

Set up hosting for the URL shortener page

To create a bucket to place the HTML page of your service in and configure it for hosting static websites:

  1. In the management console, select your working folder.

  2. Select Object Storage.

  3. Click Create bucket.

  4. On the bucket creation page:

    1. Enter a name for the bucket.

      Warning

      Bucket names are unique throughout Object Storage, which means you cannot create two buckets with the same name, even in different folders belonging to different clouds.

    2. Set the maximum size to 1 GB.

    3. Choose Public access to read objects.

    4. Click Create bucket to complete the operation.

  5. Copy the HTML code and paste it into the index.html file:

    HTML code
    <!DOCTYPE html>
<html lang="en">

<head>
  <meta charset="UTF-8">
  <title>URL shortener</title>
  <!-- warns against sending unnecessary GET requests to /favicon.ico -->
  <link rel="icon" href="data:;base64,iVBORw0KGgo=">
</head>

<body>
  <h1>Welcome</h1>
  <form action="javascript:shorten()">
    <label for="url">Enter a link:</label><br>
    <input id="url" name="url" type="text"><br>
    <input type="submit" value="Shorten">
  </form>
  <p id="shortened"></p>
</body>

<script>
  function shorten() {
    const link = document.getElementById("url").value
    fetch("/shorten", {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json'
      },
      body: link
    })
    .then(response => response.json())
    .then(data => {
      const url = data.url
      document.getElementById("shortened").innerHTML = `<a href=${url}>${url}</a>`
    })
    .catch(error => {
      document.getElementById("shortened").innerHTML = `<p>An error (${error}) occurred. Try again.</p>`
    })
  }
</script>

</html>

  6. Click the name of the created bucket.

  7. Click Upload objects.

  8. Specify the index.html file you prepared.

  9. Click Upload.

  10. In the left-hand panel, select Settings.

  11. On the Website tab:

    • Select Hosting.
    • Specify the website's home page: index.html.

  12. Click Save.

Create a service account

To create a service account for the service components to interact:

  1. Go to your working folder.

  2. In the list of services, select Identity and Access Management.

  3. Click Create service account.

  4. Enter a name for the service account: serverless-shortener.

  5. Click Add role and select the editor role.

  6. Click Create.

  7. Click on the name of the created service account.

    Save the service account ID, you will need it in the next steps.

Create a database in Managed Service for YDB

To create a Managed Service for YDB database and configure it to store URLs:

  1. Go to your working folder.

  2. In the list of services, select Managed Service for YDB.

  3. Click Create a database.

  4. Enter the database name: for-serverless-shortener.

  5. Select the Serverless database type.

  6. Click Create a database.

  7. Wait until the database starts.

    While being created, your database will have the Provisioning status. Once it is ready for use, its status will change to Running.

  8. Click the name of the database you created.

    Save the Endpoint field value, you will need it in the next steps.

  9. In the left-hand panel, select the Navigation tab.

  10. Click New SQL query.

  11. Copy the SQL query and paste it into the Query field:

    CREATE TABLE links
(
  id Utf8,
  link Utf8,
  PRIMARY KEY (id)
);

  12. Click Run.

Set up a function in Cloud Functions

To create and set up a URL shortening function:

  1. Go to your working folder.

  2. In the list of services, select Cloud Functions.

  3. Click Create function.

  4. Enter the name: for-serverless-shortener.

  5. Click Create.

  6. In the Python drop-down list, choose the python312 runtime environment.

  7. Click Continue.

  8. Copy the function code and paste it into the index.py file under Function code.

    Function code
    import ydb

import urllib.parse
import hashlib
import base64
import json
import os


def decode(event, body):
  # The request body can be encoded.
  is_base64_encoded = event.get('isBase64Encoded')
  if is_base64_encoded:
    body = str(base64.b64decode(body), 'utf-8')
  return body

def response(statusCode, headers, isBase64Encoded, body):
  return {
    'statusCode': statusCode,
    'headers': headers,
    'isBase64Encoded': isBase64Encoded,
    'body': body,
  }

def get_config():
  endpoint = os.getenv("endpoint")
  database = os.getenv("database")
  if endpoint is None or database is None:
    raise AssertionError("You need to specify both environment variables")
  credentials = ydb.iam.MetadataUrlCredentials()
  return ydb.DriverConfig(endpoint, database, credentials=credentials)

def execute(config, query, params):
  with ydb.Driver(config) as driver:
    try:
      driver.wait(timeout=5, fail_fast=True)
    except TimeoutError:
      print("Connect failed to YDB")
      print("Last reported errors by discovery:")
      print(driver.discovery_debug_details())
      return None

    session = driver.table_client.session().create()
    prepared_query = session.prepare(query)
    return session.transaction(ydb.SerializableReadWrite()).execute(
      prepared_query,
      params,
      commit_tx=True
    )


def insert_link(id, link):
  config = get_config()
  query = """
    DECLARE $id AS Utf8;
    DECLARE $link AS Utf8;

    UPSERT INTO links (id, link) VALUES ($id, $link);
    """
  params = {'$id': id, '$link': link}
  execute(config, query, params)

def find_link(id):
  print(id)
  config = get_config()
  query = """
    DECLARE $id AS Utf8;

    SELECT link FROM links where id=$id;
    """
  params = {'$id': id}
  result_set = execute(config, query, params)

  if not result_set or not result_set[0].rows:
    return None

  return result_set[0].rows[0].link

def shorten(event):
  body = event.get('body')

  if body:
    body = decode(event, body)
    original_host = event.get('headers').get('Origin')
    link_id = hashlib.sha256(body.encode('utf8')).hexdigest()[:6]
    # The URL may contain encoded characters, e.g., %. This will interfere with API Gateway when redirecting,
    # therefore, you should get rid of these characters by invoking urllib.parse.unquote.
    insert_link(link_id, urllib.parse.unquote(body))
    return response(200, {'Content-Type': 'application/json'}, False, json.dumps({'url': f'{original_host}/r/{link_id}'}))
  return response(400, {}, False, 'The url parameter is missing in the request body')

def redirect(event):
  link_id = event.get('pathParams').get('id')
  redirect_to = find_link(link_id)

  if redirect_to:
    return response(302, {'Location': redirect_to}, False, '')

  return response(404, {}, False, 'This link does not exist')

# These checks are required because we have only one function.
# Ideally, each path in API Gateway should have a function of its own.
def get_result(url, event):
  if url == "/shorten":
    return shorten(event)
  if url.startswith("/r/"):
    return redirect(event)

  return response(404, {}, False, 'This path does not exist')

def handler(event, context):
  url = event.get('url')
  if url:
    # The URL may come from API Gateway with a question mark at the end.
    if url[-1] == '?':
      url = url[:-1]
    return get_result(url, event)

  return response(404, {}, False, 'This function should be invoked using API Gateway.')

  9. Under Function code, create a file named requirements.txt and paste the following text into it:

    ydb

  10. Specify the entry point: index.handler.

  11. Set the timeout value to 5.

  12. Select the serverless-shortener service account.

  13. Add these environment variables:

    • endpoint: Enter the first part of the previously saved Endpoint field value (preceding /?database=), e.g., grpcs://ydb.serverless.yandexcloud.net:2135.
    • database: Enter the second part of the previously saved Endpoint field value (following /?database=), e.g., /ru-central1/r1gra875baom********/g5n22e7ejfr1********.

  14. Click Save changes.

  15. Under Overview, enable the Public function option.

Save the function ID, you will need it at the next steps.

Publish the service via API Gateway

To publish the service via API Gateway:

  1. Go to your working folder.

  2. In the list of services, select API Gateway.

  3. Click Create API gateway.

  4. In the Name field, enter for-serverless-shortener.

  5. Copy and paste the following code into the Specification section:

    Specification
    openapi: 3.0.0
info:
  title: for-serverless-shortener
  version: 1.0.0
paths:
  /:
    get:
      x-yc-apigateway-integration:
        type: object_storage
        bucket: <bucket_name> # <-- bucket name
        object: index.html # <-- HTML file name
        presigned_redirect: false
        service_account: <service_account_id> # <-- service account ID
      operationId: static
  /shorten:
    post:
      x-yc-apigateway-integration:
        type: cloud_functions
        function_id: <function_id> # <-- function ID
      operationId: shorten
  /r/{id}:
    get:
      x-yc-apigateway-integration:
        type: cloud_functions
        function_id: <function_id> # <-- function ID
      operationId: redirect
      parameters:
      - description: id of the url
        explode: false
        in: path
        name: id
        required: true
        schema:
          type: string
        style: simple

    Edit the specification code:

    • Replace <service_account_id> with the ID of the previously created service account.
    • Replace <function_id> with the ID of the previously created function.
    • Replace <bucket_name> with the name of the previously created bucket.

  6. Click Create.

  7. Click the name of the API gateway you created.

  8. Copy the url value from the specification.

    Use this URL to work with the created service.

Test the URL shortener

To check that the service components interact properly:

  1. Open the copied URL in the browser.

  2. In the input field, enter the URL that you want to shorten.

  3. Click Shorten.

    You will see the shortened URL below.

  4. Follow this link. As a result, the same page should open as when using the full URL.

How to delete the resources you created

To stop paying for the resources you created:

  1. Delete the API gateway.
  2. Delete the function.
  3. Delete the database.
  4. Delete the bucket.
  5. Delete the service account.
Previous
All tutorials
Next
Entering data into storage systems