Token calculator (chat)

Available in Classic and VPC

Calculate the number of tokens in a sentence entered in the HCX model.

Request

This section describes the request format. The method and URI are as follows:

Method	URI
POST	/v1/api-tools/chat-tokenize/{modelName}

Request headers

For information about the headers common to all CLOVA Studio APIs, see CLOVA Studio request headers.

Request path parameters

You can use the following path parameters with your request:

Field	Type	Required	Description
`modelName`	String	Required	Model name Example: HCX-003

Request body

You can include the following data in the body of your request:

Field	Type	Required	Description
`messages`	Array	Required	List of conversation messages to count tokens for: messages

`messages`

The following describes messages.

Field	Type	Required	Description
`role`	Enum	Required	Role of conversation messages `system` \| `user` \| `assistant` `system`: directives that define roles `user`: user utterances/questions `assistant`: answers of the model
`content`	String	Required	Content of conversation messages

Request example

The request example is as follows:

curl --location --request POST 'https://clovastudio.stream.ntruss.com/v1/api-tools/chat-tokenize/{modelName}' \
--header 'Authorization: Bearer {API Key}' \
--header 'X-NCP-CLOVASTUDIO-REQUEST-ID: {Request ID}' \
--header 'Content-Type: application/json' \
--data '{
        "messages": [
        {
            "role": "system",
            "content": "- HyperCLOVA X is NAVER's hyperscale AI."
        },
        {
            "role": "user",
            "content": "Hello, what is your name?"
        },
        {
            "role": "assistant",
            "content": "I'm HyperCLOVA X."
        }
    ]
}'

Response

This section describes the response format.

Response body

The response body includes the following data:

Field	Type	Required	Description
`messages`	Array	-	List of request conversation messages with counted tokens: messages

`messages`

The following describes messages.

Field	Type	Required	Description
`role`	Enum	-	Role of conversation messages `system` \| `user` \| `assistant` `system`: directives that define roles `user`: user utterances/questions `assistant`: answers of the model
`content`	String	-	Content of conversation messages
`count`	Integer	-	Result of calculating the number of tokens in the conversation message

Response example

The response example is as follows:

Succeeded

The following is a sample response upon a successful call.

{
    "status": {
      "code": "20000",
      "message": "OK"
    },
    "result": {
      "messages": [
        {
          "role": "system",
          "content": "- HyperCLOVA X is NAVER's hyperscale AI.",
          "count": 15
        },
        {
          "role": "user",
          "content": "Hello, what is your name?",
          "count": 10
        },
        {
          "role": "assistant",
          "content": "I'm HyperCLOVA X.",
          "count": 9
        }
      ]
    }
  }

Failure

The following is a sample response upon a failed call.