Pre-signed URLs

Object Storage incorporates multiple mechanisms for managing access to resources. To learn how these mechanisms interact, see Access management methods in Object Storage: Overview.

With pre-signed URLs, any web user can perform various operations in Object Storage, such as:

• Download an object.
• Upload an object.
• Create a bucket.

A pre-signed URL is a URL containing request authorization data in its parameters. Pre-signed URLs can be created by users with static access keys.

This section outlines the general principles for generating pre-signed URLs using AWS Signature V4.

General pre-signed URL formatGeneral pre-signed URL format

https://<bucket_name>.storage.yandexcloud.net/<object_key>?
     X-Amz-Algorithm=AWS4-HMAC-SHA256
    &X-Amz-Credential=<access_key-id>%2F<YYYYMMDD>%2Fru-central1%2Fs3%2Faws4_request
    &X-Amz-Date=<time_in_ISO_8601_format>
    &X-Amz-Expires=<URL_lifetime>
    &X-Amz-SignedHeaders=<list_of_signed_headers>
    &X-Amz-Signature=<signature>

Pre-signed URL parameters:

Creating pre-signed URLsCreating pre-signed URLs

To get a pre-signed URL:

1. Create a canonical request.
2. Compose a string to sign.
3. Generate a signing key.
4. Calculate the signature using the key.
5. Generate a pre-signed URL.

To create a pre-signed URL, you must have static access keys.

Canonical requestCanonical request

The general canonical request format is as follows:

<HTTPVerb>\n
<CanonicalURL>\n
<CanonicalQueryString>\n
<CanonicalHeaders>\n
<SignedHeaders>\n
UNSIGNED-PAYLOAD

HTTPVerbHTTPVerb

HTTPVerb stands for the HTTP method used to send a request: GET, PUT, HEAD, or DELETE.

CanonicalURLCanonicalURL

URL-encoded object key. For example, /folder/object.ext.

CanonicalQueryStringCanonicalQueryString

The canonical query string must include all query parameters of the destination URL, except X-Amz-Signature. The parameters in the string must be URL-encoded and sorted alphabetically.

Example:

X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=JK38EXAMPLEAKDID8%2F20190801%2Fru-central1%2Fs3%2Faws4_request&X-Amz-Date=20190801T000000Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host

CanonicalHeadersCanonicalHeaders

This section includes the list of the request headers and their values.

The requirements are as follows:

• Each header must be separated with the \n newline character.
• Header names must be lowercase.
• Headers must be sorted alphabetically.
• There may not be any extra spaces.
• The list must contain the host header and all x-amz-* headers used in the request.

You can also add any request header to the list. The more headers you sign, the safer your request is going to be.

Example:

host:sample-bucket.storage.yandexcloud.net
x-amz-date:20190801T000000Z

SignedHeadersSignedHeaders

This is a list of lowercase request header names, sorted alphabetically and separated by semicolons.

Example:

host;x-amz-date

Canonical request endingCanonical request ending

A canonical request must always end with an UNSIGNED-PAYLOAD string.

String to signString to sign

General format of a string to sign:

"AWS4-HMAC-SHA256" + "\n" +
<timestamp> + "\n" +
<scope> + "\n" +
Hex(Hash-SHA256(<CanonicalRequest>))

Where:

• AWS4-HMAC-SHA256: Hashing algorithm.
• timestamp: Current time in ISO 8601 format, for example, 20190801T000000Z. The specified date must match the date in scope (by value rather than by format).
• scope: <YYYYMMDD>/ru-central1/s3/aws4_request.
• CanonicalRequest: Canonical request generated earlier. The signature string contains the SHA256 hash of the canonical request in hexadecimal representation.

Signing keySigning key

To generate a signing key:

1. Use the secret key to encode the date:

   DateKey = sign("AWS4" + "SecretKey", "yyyymmdd")
   

2. Encode the region using the DateKey you got in the previous step:

   RegionKey = sign(DateKey, "ru-central1")
   

3. Encode the service using the RegionKey you got in the previous step:

   ServiceKey = sign(RegionKey, "s3")
   

4. Get the signing key:

   SigningKey = sign(ServiceKey, "aws4_request")
   

Sign a string with a keySign a string with a key

To get a string signature, use HMAC with the SHA256 hash function and convert the result to hexadecimal format.

signature = Hex(sign(SigningKey, StringToSign))

Pre-signed URLsPre-signed URLs

To compose a pre-signed URL, add the parameters required to authorize the request to the Object Storage resource URL, including the X-Amz-Signature parameter containing the calculated signature.

The other parameter values must match their respective values specified earlier in the canonical request and the signature string.

Example of composing a pre-signed URL for downloading an objectExample of composing a pre-signed URL for downloading an object

Let's put together a pre-signed URL to download the object-for-share.txt object from the bucket:

• Static key:

  access_key_id = 'JK38EXAMP********'
  secret_access_key = 'ExamP1eSecReTKeykdo********'
  

• Canonical request:

  GET
  /object-for-share.txt
  X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=YCAJEK0Iv6x********eLTAdg%2F20231208%2Fru-central1%2Fs3%2Faws4_request&X-Amz-Date=20231208T184504Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host
  host:<bucket_name>.storage.yandexcloud.net
  
  host
  UNSIGNED-PAYLOAD
  

• String to sign:

  AWS4-HMAC-SHA256
  20231208T184504Z
  20231208/ru-central1/s3/aws4_request
  e823d75aad02c1317589bd5373fe9e20d5ef44499237703ff23e5600********
  

• Signing key:

  sign(sign(sign(sign("AWS4" + "ExamP1eSecReTKeykdokKK38800","20190801"),"ru-central1"),"s3"),"aws4_request")
  

  Here, we introduce the sign function to indicate the method of key calculation that uses the HMAC algorithm with SHA256.

• Signature:

  b10c16a1997bb524bf59974512f1a6561cf2953c29dc3efbdb920790********
  

• Pre-signed URL:

  https://<bucket_name>.storage.yandexcloud.net/object-for-share.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=YCAJEK0Iv6xqy-pEQcueLTAdg%2F20231208%2Fru-central1%2Fs3%2Faws4_request&X-Amz-Date=20231208T195434Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=b10c16a1997bb524bf59974512f1a6561cf2953c29dc3efbdb920790********
  

You can work on the request and signature generation process using the AWS CLI in debug mode.

To send a signed request to the S3 API, you can use curl.

Code examples for generating pre-signed URLsCode examples for generating pre-signed URLs

The subsection provides code examples for generating pre-signed URLs.

To show the principle of forming and signing requests to Object Storage, the examples do not use AWS SDKs. For examples of using the AWS SDK and other tools, see Examples of getting a signed link in Object Storage tools.

import datetime
import hashlib
import hmac

access_key = '<static_key_ID>'
secret_key = '<static_key_contents>'
object_key = '<object_key>'
bucket = '<bucket_name>'
host = 'storage.yandexcloud.net'
now = datetime.datetime.now(datetime.UTC)
datestamp = now.strftime('%Y%m%d')
timestamp = now.strftime('%Y%m%dT%H%M%SZ')

def sign(key, msg):
    return hmac.new(key, msg.encode('utf-8'), hashlib.sha256).digest()

canonical_request = """GET
/{bucket}/{object_key}
X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential={access_key}%2F{datestamp}%2Fru-central1%2Fs3%2Faws4_request&X-Amz-Date={timestamp}&X-Amz-Expires=3600&X-Amz-SignedHeaders=host
host:{host}

host
UNSIGNED-PAYLOAD""".format(
    bucket=bucket,
    object_key=object_key,
    access_key=access_key,
    datestamp=datestamp,
    timestamp=timestamp,
    host=host)

print()
print("Canonical request:\n" + canonical_request)
print()

string_to_sign = """AWS4-HMAC-SHA256
{timestamp}
{datestamp}/ru-central1/s3/aws4_request
{request_hash}""".format(
    timestamp=timestamp,
    datestamp=datestamp,
    request_hash=hashlib.sha256(canonical_request.encode('utf-8')).hexdigest())

print()
print("String to be signed:\n" + string_to_sign)
print()

signing_key = sign(sign(sign(sign(('AWS4' + secret_key).encode('utf-8'), datestamp), 'ru-central1'), 's3'), 'aws4_request')
signature = hmac.new(signing_key, string_to_sign.encode('utf-8'), hashlib.sha256).hexdigest()

print()
print("Signature: " + signature)
print()

signed_link = "https://" + host + '/' + bucket + '/' + object_key + "?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=" + access_key + "%2F" + datestamp + "%2Fru-central1%2Fs3%2Faws4_request&X-Amz-Date=" + timestamp + "&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=" + signature + "\n"

print()
print("Signed Link:\n" + signed_link)

<?php

  $keyid = "<static_key_ID>";
  $secretkey = "<static_key_contents>";
  $path = "<object_key>";
  $objectname = "/".implode("/", array_map("rawurlencode", explode("/", $path)));
  $host = "<bucket_name>.storage.yandexcloud.net";
  $region = "ru-central1";
  $timestamp = time();
  $dater = strval(date('Ymd', $timestamp));
  $dateValue = strval(date('Ymd', $timestamp))."T".strval(date('His', $timestamp))."Z";

  // Generate the canonical request
  $canonical_request = "GET\n".$objectname."\nX-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=".$keyid."%2F".$dater."%2Fru-central1%2Fs3%2Faws4_request&X-Amz-Date=".$dateValue."&X-Amz-Expires=3600&X-Amz-SignedHeaders=host\nhost:".$host."\n\nhost\nUNSIGNED-PAYLOAD";

  echo "<b>Canonical request: </b><br>".$canonical_request."<br><br>";

  // Generate the string to be signed
  $string_to_sign = "AWS4-HMAC-SHA256\n".$dateValue."\n".$dater."/".$region."/s3/aws4_request\n".openssl_digest($canonical_request, "sha256", $binary = false);

  echo "<b>String to be signed: </b><br>".$string_to_sign."<br><br>";

  // Generate the signing key
  $signing_key = hash_hmac('sha256', 'aws4_request', hash_hmac('sha256', 's3', hash_hmac('sha256', 'ru-central1', hash_hmac('sha256', $dater, 'AWS4'.$secretkey, true), true), true), true);

  echo "<b>Signing key: </b><br>".$signing_key."<br><br>";

  // Generate the signature
  $signature = hash_hmac('sha256', $string_to_sign, $signing_key);

  echo "<b>Signature: </b><br>".$signature."<br><br>";

  // Generate the pre-signed link
  $signed_link = "https://".$host.$objectname."?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=".$keyid."%2F".$dater."%2Fru-central1%2Fs3%2Faws4_request&X-Amz-Date=".$dateValue."&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=".$signature."\n";

  echo '<b>Signed link: </b><br>'.'<a href = "'.$signed_link.'" target = "_blank">'.$signed_link.'</a>';

?>

Examples of getting pre-signed links in Object Storage toolsExamples of getting pre-signed links in Object Storage tools

This subsection provides examples of generating pre-signed URLs with the help of various Object Storage tools.

aws s3 presign s3://<bucket_name>/<object_key> \
  --expires-in <link_lifetime> \
  --endpoint-url "https://storage.yandexcloud.net/"

# coding=utf-8

import boto3
from botocore.client import Config


ENDPOINT = "https://storage.yandexcloud.net"

ACCESS_KEY = "JK38EXAMP********"
SECRET_KEY = "ExamP1eSecReTKeykdo********"

session = boto3.Session(
    aws_access_key_id=ACCESS_KEY,
    aws_secret_access_key=SECRET_KEY,
    region_name="ru-central1",
)
s3 = session.client(
    "s3", endpoint_url=ENDPOINT, config=Config(signature_version="s3v4")
)

presigned_url = s3.generate_presigned_url(
    "get_object",
    Params={"Bucket": "bucket-with-objects", "Key": "object-for-share"},
    ExpiresIn=100,
)

print(presigned_url)