Object Storage API

Available in Classic and VPC

Review the support information for the Object Storage API, which is compatible with the Amazon S3 API, and verify the common API calls and authentication methods. Requests generated using the S3 API on NAVER Cloud Platform Object Storage must be authenticated using AWS's authorization header implementation. Object Storage supports Signature Version 2 and Signature Version 4 authentication methods, and this guide is based on Version 4.

Supported API

The Object Storage API enables control over various features necessary for storage management and usage. See the Object Storage API guide for the list of supported operations.

Call API

The Object Storage API call steps are as follows:

1. Create authentication key.
2. Check calling domain
3. Authentication
4. Call

1. Create authentication key.

The method for creating an authentication key to use the Object Storage API is identical to the authentication key creation method for the Ncloud API. See 1. Create authentication key of Ncoud API.

2. Check calling domain

While it supports both HTTP and HTTPS protocols, we recommend using HTTPS for data protection. The region-specific calling domain information is as follows:

3. Authentication

The following describes how to create an authorization header.

1) Create canonical request
2) Create string to sign
3) Create signing key
4) Create signature
5) Create authorization header

1) Create canonical request

The structure of the canonical request and descriptions for each item are as follows:

<HTTPMethod>\n
<CanonicalURI>\n
<CanonicalQueryString>\n
<CanonicalHeaders>\n
<SignedHeaders>\n
<HashedPayload>

• HTTPMethod
  Declare the HTTP method to use. Example: PUT

• CanonicalURI
  Define the resource to access in a standardized (URI-encoded) format. Example: /path/object

• CanonicalQueryString
  If request parameters exist, define them in a standardized format (URI-encoded) and list them in alphabetical order.

  UriEncode("marker")+"="+UriEncode("someMarker")+"&"+
  UriEncode("max-keys")+"="+UriEncode("20") + "&" +
  UriEncode("prefix")+"="+UriEncode("somePrefix")
  

• CanonicalHeaders
  Define the header names and values included in the request in a standardized manner. Convert all header names to lowercase and remove leading and trailing whitespace from header values. Insert a newline character \n at the end of each header entry. At this point, the headers must be sorted alphabetically by header name.
  The host header and headers starting with x-amz- are mandatory. Including all headers is recommended for more accurate data consistency verification.
  All requests using Signature Version 4 authentication must include the x-amz-content-sha256 header. You must provide the hash value for the data (payload), with two options available.

  • Signed payload: Enter the hexadecimal value of the SHA256 hash of the payload. If there is no payload, enter the hash value of an empty string.
  • Unsigned payload: To not sign the payload, enter the string UNSIGNED-PAYLOAD as the value of the x-amz-content-sha256 header.

  Lowercase(<HeaderName1>)+":"+Trim(<value>)+"\n"
  Lowercase(<HeaderName2>)+":"+Trim(<value>)+"\n"
  

• SignedHeaders
  List the header names included in the request. Convert all header names to lowercase and sort them alphabetically. Enter a semicolon (";") as the separator between each header name.

  host;x-amz-content-sha256;x-amz-date
  

• HashedPayload
  Enter the hexadecimal value of the SHA256 hash of the payload. (Same as the value of the x-amz-content-sha256 header) If there is no payload, enter the hash value of an empty string. To not sign the data (payload), enter the UNSIGNED-PAYLOAD string.

  Hex(SHA256Hash(<payload>)
  or
  Hex(SHA256Hash(""))
  or
  UNSIGNED-PAYLOAD
  

2) Create string to sign

The structure of the string to sign and descriptions for each item are as follows:

AWS4-HMAC-SHA256\n
<Timestamp>\n
<Scope>\n
Hex(SHA256Hash(<CanonicalRequest>))

• Timestamp
  It must be displayed according to the ISO 8601 standard specification based on UTC (Coordinated Universal Time). Example: 20161128T152924Z

• Scope
  The scope string is structured as follows:

  <Date>/<Region>/s3/aws4_request
  

  • Date: Enter in YYYYMMDD format.
  • Region: Enter the Region name. The Region name can be found in 2. Check calling domain. While the Region name is a required input, NAVER Cloud Platform Object Storage uses the domain to distinguish Regions, so it does not verify the consistency of the Region name.

• CanonicalRequest
  Enter the hexadecimal value of the SHA256 hash of the generated canonical request.

3) Create signing key

The structure of the signing key and descriptions for each item are as follows:

kSecret     = <SecretKey>
kDate       = HMAC-SHA256("AWS4" + kSecret, <Date>)
kRegion     = HMAC-SHA256(kDate, <Region>)
kService    = HMAC-SHA256(kRegion, "s3")
kSigning    = HMAC-SHA256(kService, "aws4_request")

• SecretKey
  Enter the secret key of the API authentication key. It can be found in My Account > Manage account and security > Manage security > Manage access > API authentication key in the NAVER Cloud Platform console.

• Date
  Enter in YYYYMMDD format.

• Region
  Enter the Region name. The Region name can be found in 2. Check calling domain. While the Region name is a required input, NAVER Cloud Platform Object Storage uses the domain to distinguish Regions, so it does not verify the consistency of the Region name.

4) Create signature

The structure of the signature and descriptions for each item are as follows:

Hex(HMAC-SHA256(<SigningKey>, <StringToSign>))

• Signing Key
  Calculate the HMAC-SHA256 hash of the signing key and display it as a hexadecimal value.

• StringToSign
  Calculate the HMAC-SHA256 hash of the string to sign and display it as a hexadecimal value.

5) Create authorization header

The structure of the authorization header and descriptions for each item are as follows:

Authorization: AWS4-HMAC-SHA256 Credential=<AccessKeyID>/<Scope>, SignedHeaders=<SignedHeaders>, Signature=<Signature>

• AccessKeyID
  Enter the access key ID of the API authentication key. It can be found in My Account > Manage account and security > Manage security > Manage access > API authentication key in the NAVER Cloud Platform console.

• Scope
  The scope string is structured as follows:

  <Date>/<Region>/s3/aws4_request
  

  • Date: Enter in YYYYMMDD format.
  • Region: Enter the Region name. The Region name can be found in 2. Check calling domain. While the Region name is a required input, NAVER Cloud Platform Object Storage uses the domain to distinguish Regions, so it does not verify the consistency of the Region name.

• SignedHeaders
  List the header names included in the request. Convert all header names to lowercase and sort them alphabetically. Enter a semicolon (";") as the separator between each header name.

  host;x-amz-content-sha256;x-amz-date
  

• Signature
  Enter the signature generated in 4) Create signature.

The following describes examples of generating language-specific authorization headers.

public class ObjectStorageSample {
	private static byte[] sign(String stringData, byte[] key) throws NoSuchAlgorithmException, UnsupportedEncodingException, InvalidKeyException {
        byte[] data = stringData.getBytes(CHARSET_NAME);
        Mac e = Mac.getInstance(HMAC_ALGORITHM);
        e.init(new SecretKeySpec(key, HMAC_ALGORITHM));
        return e.doFinal(data);
    }

    private static String hash(String text) throws NoSuchAlgorithmException, UnsupportedEncodingException {
        MessageDigest e = MessageDigest.getInstance(HASH_ALGORITHM);
        e.update(text.getBytes(CHARSET_NAME));
        return Hex.encodeHexString(e.digest());
    }

    private static String getStandardizedQueryParameters(String queryString) throws UnsupportedEncodingException {
        TreeMap<String, String> sortedQueryParameters = new TreeMap<>();
        // sort by key name
        if (queryString != null && !queryString.isEmpty()) {
            String[] queryStringTokens = queryString.split("&");
            for (String field : queryStringTokens) {
                String[] fieldTokens = field.split("=");
                if (fieldTokens.length > 0) {
                    if (fieldTokens.length > 1) {
                        sortedQueryParameters.put(fieldTokens[0], fieldTokens[1]);
                    } else {
                        sortedQueryParameters.put(fieldTokens[0], "");
                    }
                }
            }
        }

        StringBuilder standardizedQueryParametersBuilder = new StringBuilder();
        int count = 0;
        for (String key : sortedQueryParameters.keySet()) {
            if (count > 0) {
                standardizedQueryParametersBuilder.append("&");
            }
            standardizedQueryParametersBuilder.append(key).append("=");

            if (sortedQueryParameters.get(key) != null && !sortedQueryParameters.get(key).isEmpty()) {
                standardizedQueryParametersBuilder.append(URLEncoder.encode(sortedQueryParameters.get(key), CHARSET_NAME));
            }

            count++;
        }
        return standardizedQueryParametersBuilder.toString();
    }

    private static TreeMap<String, String> getSortedHeaders(Header[] headers) {
        TreeMap<String, String> sortedHeaders = new TreeMap<>();
        // sort by header name
        for (Header header : headers) {
            String headerName = header.getName().toLowerCase();
            sortedHeaders.put(headerName, header.getValue());
        }

        return sortedHeaders;
    }

private static String getSignedHeaders(TreeMap<String, String> sortedHeaders) {
		StringBuilder signedHeadersBuilder = new StringBuilder();
		for (String headerName : sortedHeaders.keySet()) {
			signedHeadersBuilder.append(headerName.toLowerCase()).append(";");
		}
        String s = signedHeadersBuilder.toString();
        if (s.endsWith(";")) {
            s = s.substring(0, s.length() - 1);
        }
		return s;
	}
    
    private static String getStandardizedHeaders(TreeMap<String, String> sortedHeaders) {
        StringBuilder standardizedHeadersBuilder = new StringBuilder();
        for (String headerName : sortedHeaders.keySet()) {
            standardizedHeadersBuilder.append(headerName.toLowerCase()).append(":").append(sortedHeaders.get(headerName)).append("\n");
        }

        return standardizedHeadersBuilder.toString();
    }

    private static String getCanonicalRequest(HttpUriRequest request, String standardizedQueryParameters, String standardizedHeaders, String signedHeaders) {
        StringBuilder canonicalRequestBuilder = new StringBuilder().append(request.getMethod()).append("\n")
            .append(request.getURI().getPath()).append("\n")
            .append(standardizedQueryParameters).append("\n")
            .append(standardizedHeaders).append("\n")
            .append(signedHeaders).append("\n")
            .append(UNSIGNED_PAYLOAD);

        return canonicalRequestBuilder.toString();
    }

    private static String getScope(String datestamp, String regionName) {
        StringBuilder scopeBuilder = new StringBuilder().append(datestamp).append("/")
            .append(regionName).append("/")
            .append(SERVICE_NAME).append("/")
            .append(REQUEST_TYPE);
        return scopeBuilder.toString();
    }

    private static String getStringToSign(String timestamp, String scope, String canonicalRequest) throws NoSuchAlgorithmException, UnsupportedEncodingException {
        StringBuilder stringToSignBuilder = new StringBuilder(AWS_ALGORITHM)
            .append("\n")
            .append(timestamp).append("\n")
            .append(scope).append("\n")
            .append(hash(canonicalRequest));

        return stringToSignBuilder.toString();
    }

    private static String getSignature(String secretKey, String datestamp, String regionName, String stringToSign) throws NoSuchAlgorithmException, UnsupportedEncodingException, InvalidKeyException {
        byte[] kSecret = ("AWS4" + secretKey).getBytes(CHARSET_NAME);
        byte[] kDate = sign(datestamp, kSecret);
        byte[] kRegion = sign(regionName, kDate);
        byte[] kService = sign(SERVICE_NAME, kRegion);
        byte[] signingKey = sign(REQUEST_TYPE, kService);

        return Hex.encodeHexString(sign(stringToSign, signingKey));
    }

    private static String getAuthorization(String accessKey, String scope, String signedHeaders, String signature) {
        String signingCredentials = accessKey + "/" + scope;
        String credential = "Credential=" + signingCredentials;
        String signerHeaders = "SignedHeaders=" + signedHeaders;
        String signatureHeader = "Signature=" + signature;

        StringBuilder authHeaderBuilder = new StringBuilder().append(AWS_ALGORITHM).append(" ")
            .append(credential).append(", ")
            .append(signerHeaders).append(", ")
            .append(signatureHeader);

        return authHeaderBuilder.toString();
    }

    private static void authorization(HttpUriRequest request, String regionName, String accessKey, String secretKey) throws Exception {
        Date now = new Date();
        DATE_FORMATTER.setTimeZone(TimeZone.getTimeZone("UTC"));
        TIME_FORMATTER.setTimeZone(TimeZone.getTimeZone("UTC"));
        String datestamp = DATE_FORMATTER.format(now);
        String timestamp = TIME_FORMATTER.format(now);

        request.addHeader("X-Amz-Date", timestamp);

        request.addHeader("X-Amz-Content-Sha256", UNSIGNED_PAYLOAD);

        String standardizedQueryParameters = getStandardizedQueryParameters(request.getURI().getQuery());

        TreeMap<String, String> sortedHeaders = getSortedHeaders(request.getAllHeaders());
        String signedHeaders = getSignedHeaders(sortedHeaders);
        String standardizedHeaders = getStandardizedHeaders(sortedHeaders);

        String canonicalRequest = getCanonicalRequest(request, standardizedQueryParameters, standardizedHeaders, signedHeaders);
        System.out.println("> canonicalRequest :");
        System.out.println(canonicalRequest);

        String scope = getScope(datestamp, regionName);

        String stringToSign = getStringToSign(timestamp, scope, canonicalRequest);
        System.out.println("> stringToSign :");
        System.out.println(stringToSign);

        String signature = getSignature(secretKey, datestamp, regionName, stringToSign);

        String authorization = getAuthorization(accessKey, scope, signedHeaders, signature);
        request.addHeader("Authorization", authorization);
    }

    private static void putObject(String bucketName, String objectName, String localFilePath) throws Exception {
        HttpClient httpClient = HttpClientBuilder.create().build();

        HttpPut request = new HttpPut(ENDPOINT + "/" + bucketName + "/" + objectName);
        request.addHeader("Host", request.getURI().getHost());
        request.setEntity(new FileEntity(new File(localFilePath)));

        authorization(request, REGION_NAME, ACCESS_KEY, SECRET_KEY);

        HttpResponse response = httpClient.execute(request);
        System.out.println("Response : " + response.getStatusLine());
    }

    private static void getObject(String bucketName, String objectName, String localFilePath) throws Exception {
        HttpClient httpClient = HttpClientBuilder.create().build();
        HttpGet request = new HttpGet(ENDPOINT + "/" + bucketName + "/" + objectName);
        request.addHeader("Host", request.getURI().getHost());

        authorization(request, REGION_NAME, ACCESS_KEY, SECRET_KEY);

        HttpResponse response = httpClient.execute(request);
        System.out.println("Response : " + response.getStatusLine());

        InputStream is = response.getEntity().getContent();
        File targetFile = new File(localFilePath);
        OutputStream os = new FileOutputStream(targetFile);

        byte[] buffer = new byte[8 * 1024];
        int bytesRead;
        while ((bytesRead = is.read(buffer)) != -1) {
            os.write(buffer, 0, bytesRead);
        }

        is.close();
        os.close();
    }

    private static void listObjects(String bucketName, String queryString) throws Exception {
        HttpClient httpClient = HttpClientBuilder.create().build();
        URI uri = new URI(ENDPOINT + "/" + bucketName + "?" + queryString);
        HttpGet request = new HttpGet(uri);
        request.addHeader("Host", request.getURI().getHost());

        authorization(request, REGION_NAME, ACCESS_KEY, SECRET_KEY);

        HttpResponse response = httpClient.execute(request);
        System.out.println("> Response : " + response.getStatusLine());
        int i;
        InputStream is = response.getEntity().getContent();
        StringBuffer buffer = new StringBuffer();
        byte[] b = new byte[4096];
        while ((i = is.read(b)) != -1) {
            buffer.append(new String(b, 0, i));
        }
        System.out.println(buffer.toString());

    }

    private static final String CHARSET_NAME = "UTF-8";
    private static final String HMAC_ALGORITHM = "HmacSHA256";
    private static final String HASH_ALGORITHM = "SHA-256";
    private static final String AWS_ALGORITHM = "AWS4-HMAC-SHA256";

    private static final String SERVICE_NAME = "s3";
    private static final String REQUEST_TYPE = "aws4_request";

    private static final String UNSIGNED_PAYLOAD = "UNSIGNED-PAYLOAD";

    private static final SimpleDateFormat DATE_FORMATTER = new SimpleDateFormat("yyyyMMdd");
    private static final SimpleDateFormat TIME_FORMATTER = new SimpleDateFormat("yyyyMMdd\'T\'HHmmss\'Z\'");

    private static final String REGION_NAME = "kr-standard";
    private static final String ENDPOINT = "https://kr.object.ncloudstorage.com";
    private static final String ACCESS_KEY = "ACCESS_KEY_ID";
    private static final String SECRET_KEY = "SECRET_KEY";

    public static void main(String[] args) throws Exception {
        String bucketName = "sample-bucket";
        String objectName = "sample-object.txt";
        String sourceFilePath = "/tmp/source.txt";
        String targetFilePath = "/tmp/target.txt";

        putObject(bucketName, objectName, sourceFilePath);

        getObject(bucketName, objectName, targetFilePath);

        String queryString = "max-keys=10&delimiter=/";
        listObjects(bucketName, queryString);
    }
}

import hashlib
import hmac
import datetime
import requests
import urllib


def get_hash(key, msg):
    return hmac.new(key, msg.encode('utf-8'), hashlib.sha256).digest()


def create_signed_headers(headers):
    signed_headers = []

    for k in sorted(headers):
        signed_headers.append('%s;' % k)

    return ''.join(signed_headers)[:-1]


def create_standardized_headers(headers):
    signed_headers = []

    for k in sorted(headers):
        signed_headers.append('%s:%s\n' % (k, headers[k]))

    return ''.join(signed_headers)


def create_standardized_query_parameters(request_parameters):
    standardized_query_parameters = []

    if request_parameters:
        for k in sorted(request_parameters):
            standardized_query_parameters.append('%s=%s' % (k, urllib.quote(request_parameters[k], safe='')))

        return '&'.join(standardized_query_parameters)
    else:
        return ''


class ObjectStorageSample:
    def __init__(self):
        self.region = 'kr-standard'
        self.endpoint = 'https://kr.object.ncloudstorage.com'
        self.host = 'kr.object.ncloudstorage.com'
        self.access_key = 'ACCESS_KEY_ID'
        self.secret_key = 'SECRET_KEY'

        self.payload_hash = 'UNSIGNED-PAYLOAD'
        self.hashing_algorithm = 'AWS4-HMAC-SHA256'
        self.service_name = 's3'
        self.request_type = 'aws4_request'

        self.time_format = '%Y%m%dT%H%M%SZ'
        self.date_format = '%Y%m%d'

    def _create_credential_scope(self, date_stamp):
        return date_stamp + '/' + self.region + '/' + self.service_name + '/' + self.request_type

    def _create_canonical_request(self, http_method, request_path, request_parameters, headers):
        standardized_query_parameters = create_standardized_query_parameters(request_parameters)
        standardized_headers = create_standardized_headers(headers)
        signed_headers = create_signed_headers(headers)

        canonical_request = (http_method + '\n' +
                             request_path + '\n' +
                             standardized_query_parameters + '\n' +
                             standardized_headers + '\n' +
                             signed_headers + '\n' +
                             self.payload_hash)

        print('canonical_request:\n%s\n' % canonical_request)
        return canonical_request

    def _create_string_to_sign(self, time_stamp, credential_scope, canonical_request):
        string_to_sign = (self.hashing_algorithm + '\n' +
                          time_stamp + '\n' +
                          credential_scope + '\n' +
                          hashlib.sha256(canonical_request.encode('utf-8')).hexdigest())

        print('string_to_sign:\n%s\n' % string_to_sign)
        return string_to_sign

    def _create_signature_key(self, date_stamp):
        key_date = get_hash(('AWS4' + self.secret_key).encode('utf-8'), date_stamp)
        key_string = get_hash(key_date, self.region)
        key_service = get_hash(key_string, self.service_name)
        key_signing = get_hash(key_service, self.request_type)
        return key_signing

    def _create_authorization_header(self, headers, signature_key, string_to_sign, credential_scope):
        signed_headers = create_signed_headers(headers)
        signature = hmac.new(signature_key, string_to_sign.encode('utf-8'), hashlib.sha256).hexdigest()

        return (self.hashing_algorithm + ' ' +
                'Credential=' + self.access_key + '/' + credential_scope + ', ' +
                'SignedHeaders=' + signed_headers + ', ' +
                'Signature=' + signature)

    def _sign(self, http_method, request_path, headers, time, request_parameters=None):
        time_stamp = time.strftime(self.time_format)
        date_stamp = time.strftime(self.date_format)

        credential_scope = self._create_credential_scope(date_stamp)
        canonical_request = self._create_canonical_request(http_method, request_path, request_parameters, headers)
        string_to_sign = self._create_string_to_sign(time_stamp, credential_scope, canonical_request)
        signature_key = self._create_signature_key(date_stamp)

        headers['authorization'] = self._create_authorization_header(headers, signature_key, string_to_sign, credential_scope)

    def put_object(self, bucket_name, object_name, source_file_path, request_parameters=None):
        http_method = 'PUT'

        with open(source_file_path) as f:
            time = datetime.datetime.utcnow()
            time_stamp = time.strftime(self.time_format)

            headers = {'x-amz-date': time_stamp,
                       'x-amz-content-sha256': self.payload_hash,
                       'host': self.host}

            request_path = '/%s/%s' % (bucket_name, object_name)

            self._sign(http_method, request_path, headers, time, request_parameters)

            request_url = self.endpoint + request_path
            r = requests.put(request_url, headers=headers, params=request_parameters, data=f.read())

            print('Response code: %d' % r.status_code)

    def get_object(self, bucket_name, object_name, target_file_path, request_parameters=None):
        http_method = 'GET'

        time = datetime.datetime.utcnow()
        time_stamp = time.strftime(self.time_format)

        headers = {'x-amz-date': time_stamp,
                   'x-amz-content-sha256': self.payload_hash,
                   'host': self.host}

        request_path = '/%s/%s' % (bucket_name, object_name)

        self._sign(http_method, request_path, headers, time, request_parameters)

        request_url = self.endpoint + request_path
        r = requests.get(request_url, headers=headers, params=request_parameters, stream=True)

        print('Response code: %d' % r.status_code)

        if r.status_code == 200:
            with open(target_file_path, 'wb') as f:
                f.write(r.content)

    def list_objects(self, bucket_name, request_parameters=None):
        http_method = 'GET'

        time = datetime.datetime.utcnow()
        time_stamp = time.strftime(self.time_format)

        headers = {'x-amz-date': time_stamp,
                   'x-amz-content-sha256': self.payload_hash,
                   'host': self.host}

        request_path = '/%s' % bucket_name

        self._sign(http_method, request_path, headers, time, request_parameters)

        request_url = self.endpoint + request_path
        r = requests.get(request_url, headers=headers, params=request_parameters)

        print('Response code: %d' % r.status_code)
        print('Response content:\n%s' % r.content)


if __name__ == '__main__':
    sample = ObjectStorageSample()
    sample.put_object('sample-bucket', 'sample-object.txt', '/tmp/source.txt')
    sample.get_object('sample-bucket', 'sample-object.txt', '/tmp/target.txt')
    sample.list_objects('sample-bucket', request_parameters={'max-keys': '10', 'delimiter': '/'})

4. Call

Call the API based on the content in steps 1 through 3. The result of the API call can be categorized as either success or failure. If the response is successful, check the returned result. If the response fails, an error code is returned. Check the returned error code and retry the call.

Succeeded

For information on handling responses to API calls, see the Object Storage API guide.

Failure

Error codes returned upon call failure include common error codes and operation-specific error codes. For error codes, see the Object Storage API guide.