Live streaming recognition

Available in Classic and VPC

Recognize and convert real-time speech data in PCM format (headerless WAV files) at 16 kHz, 1 channel, 16 bits per sample into text. Access is available only via the gRPC protocol.

Request

This section describes the request format. The method and URI are as follows:

Request order

To request recognition via gRPC:

1. Preparation
2. Config JSON
3. Recognize

1. Install and prepare protoc compiler

In preparation for using the API, install the protoc compiler by referring to the gRPC site. After installation, select your preferred language (Python or Java), then Call compiler using the 'nest.proto' file where the API interface is defined.
To install the protoc compiler and generate gRPC code:

1. Connect remotely to the server where you want to install the protoc compiler.

2. Install the packages and plugins for using gRPC.

   • Rocky Linux: Python

     # Check the latest status
     sudo dnf update
     
     # Install Python: Install Python on the Linux server.
     sudo dnf install python3
     
     # Install and upgrade pip: pip is a package installer for Python.
     sudo dnf install python3-pip
     pip3 install --upgrade pip
     
     # Install grpcio-tools: Install "grpcio-tools" using pip.
     pip3 install grpcio-tools
     
     # Create nest.proto file
     touch nest.proto
     
     # Compile nest.proto file with protoc compiler
     python3 -m grpc_tools.protoc -I=. --python_out=. --grpc_python_out=. nest.proto
     

   • Rocky Linux: Java

     # Download protoc-gen-grpc-java plugin (check https://github.com/grpc/grpc-java/releases for version number)
     curl -OL https://repo1.maven.org/maven2/io/grpc/protoc-gen-grpc-java/1.66.0/protoc-gen-grpc-java-1.66.0-linux-x86_64.exe
     
     # Add to path
     mv protoc-gen-grpc-java-1.66.0-linux-x86_64.exe /usr/local/bin/protoc-gen-grpc-java
     
     # Change to execute permission
     chmod +x /usr/local/bin/protoc-gen-grpc-java
     
     # Confirm installation
     protoc-gen-grpc-java --version
     
     # Create nest.proto file
     touch nest.proto
     
     # Compile nest.proto file with protoc compiler
     protoc --proto_path=. --java_out=output/directory --grpc-java_out=output/directory nest.proto
     

3. Open the 'nest.proto' file, enter the following code, and generate the gRPC code.

   syntax = "proto3";
   option java_multiple_files = true;
   package com.nbp.cdncp.nest.grpc.proto.v1;
   
   enum RequestType {
     CONFIG = 0;
     DATA = 1;
   }
   
   message NestConfig {
     string config = 1;
   }
   
   message NestData {
     bytes chunk = 1;
     string extra_contents = 2;
   }
   message NestRequest {
     RequestType type = 1;
     oneof part {
       NestConfig config = 2;
       NestData data = 3;
     }
   }
   
   message NestResponse {
     string contents = 1;
   }
   service NestService {
     rpc recognize(stream NestRequest) returns (stream NestResponse){};
   }
   

2. Authorization

After completing gRPC code generation via the protoc compiler, proceed with authorization. Authorization involves including a bearer token in the Authorization header during API calls to verify the client's integrity with the server. Note the following when performing authorization.

• Set up the gRPC channel and generate the stub, which is the client-side proxy for nest_grpc_pb2.
• After generating the stub, execute the desired function by including metadata containing the authentication key in the recognize method.
  • The real-time streaming recognition API is not supported on the Free plan; it is only available on the Basic long sentence recognition plan.

Authentication header

The authorization header is as follows:

Authorization order

The authorization method is as follows:

Python

To authorize using Python in a Rocky Linux environment:

1. Create a Python file. Here, the file name is specified as main.py.

   touch main.py
   

2. Add the following content to main.py.

   import grpc
   import json
   
   import nest_pb2
   import nest_pb2_grpc
   
   channel = grpc.secure_channel(
           'clovaspeech-gw.ncloud.com:50051',
           grpc.ssl_channel_credentials()
   )
   client = NestServiceStub(channel)
   metadata = (("authorization", f"Bearer {secretKey}"),) # Lowercase authorization required, secretkey is verified in the long sentence recognition domain
   call = client.YourMethod(YourRequest(), metadata=metadata)
   

Java

To authorize using Java in a Rocky Linux environment:

1. Create a Java file. Here, the file name is specified as main.java.

   touch main.java
   

2. Add the following content to main.java.

   ManagedChannel channel = NettyChannelBuilder
               .forTarget("clovaspeech-gw.ncloud.com:50051")
               .useTransportSecurity()
               .build();
   NestServiceGrpc.NestServiceStub client = NestServiceGrpc.newStub(channel);
   Metadata metadata = new Metadata();
   metadata.put(Metadata.Key.of("Authorization", Metadata.ASCII_STRING_MARSHALLER),
                "Bearer ${secretKey}");
   client = MetadataUtils.attachHeaders(client, metadata);
   

3. Config JSON

This section describes the config JSON sent to the streaming endpoint via the NestRequest object generated by nest_pb2 in protoc. The config JSON must be sent during the first call to the real-time streaming recognition API.
The config JSON provides the following fields.

• transcription: Set speech recognition language
• keywordBoosting: Set to boost the recognition rate for entered words
• forbidden: Set banned words
• semanticEpd: Set criteria for generating speech recognition results
• translationEpd: Set target language, response reception method, etc. during translation

Request body

The following describes the request body for the config JSON.

Transcription

The following describes Transcription.

Keyword Boosting

The following describes the Keyword Boosting field.

boostings

The following describes boostings.

Forbidden

The following describes the Forbidden field.

SemanticEPD

The following describes the Semantic EPD field.

Translation

The following describes the Translation field.

Request example

The following is a sample request for the config JSON.

#Semantic EPD    
 {
  "semanticEpd": {
    "skipEmptyText": false,           
    "useWordEpd": false,              
    "usePeriodEpd": true,             
    "gapThreshold": 2000,             
    "durationThreshold": 20000,       
    "syllableThreshold": 0            
  }
}
#Translation Info / EPD
 {
  "translation": {
    "targets": ["en"],         
    "mergedResult": False,
    "gapThreshold": 2000,      
    "durationThreshold": 20000,
    "honorific": False,      
    "glossaryKey": string
            }
         }    
#KeywordBoosting
  {
  "keywordBoosting": {                  
    "boostings": [
      {
        "words": "test,test1,test2",
        "weight": 1
      },
      {
        "words": "Test, test 1, test 2",
        "weight": 0.5
      }
    ],
  },
#Forbidden
    "forbidden": {
    "forbiddens":  "Banned word 1, banned word 2",
  }
}

Response body

The following describes the response body for the config JSON.

Error message

The following describes the error messages displayed when a request fails.

Response example

The following is a sample response for the config JSON.

Succeeded

The following is a sample response upon a successful call.

{
  "uid": "{uid}",
  "responseType": [ "config" ],
  "config": {
    "status": "Success",
    "keywordBoosting": {
      "status": "Success"
    },
    "forbidden" : {
      "status": "Success"
    }
}

Failure

The following is a sample response when the call fails upon entering hobidden in the request.

{
  "uid": "{uid}",
  "responseType": [ "config" ]
  "config": {
    "status": "Unknown key: hobidden"
  }
}

4. Recognize

After configuring the desired settings via the config JSON, call the speech recognition API using recognize to process and recognize speech data in real time. The NestRequest and authorization metadata in the code generated by the protoc will call the speech recognition API through the recognize method of the stub.

Request body

The following describes the request body for Recognize.

Response body

The following describes the response body for Recognize.

alignInfos

The following describes alignInfos.

Error message

The following describes the error messages displayed when a Recognize request fails.

Response example

The response example is as follows:

Succeeded

The following is a sample response upon a successful call.

• When the response is successful and "responseType": [ "transcription" ]

{
  "uid": "{uid}"
  "responseType": [ "transcription" ]
  "transcription": {
    "text": "This is text.",
    "position": 0,
    "periodPositions": [3],
    "periodAlignIndices": [3],
    "epFlag": false,
    "seqId": 0,
    "epdType": "durationThreshold",
    "startTimestamp": 190,
    "endTimestamp": 840,
    "confidence": 0.997389124199423,
    "alignInfos": [
      {"word":"This","start":190,"end":340,"confidence":0.9988637124943075},
      {"word":"is","start":341,"end":447,"confidence":0.9990018488549978},
      {"word":"text","start":448,"end":580,"confidence":0.9912501264550316},
      {"word":".","start":581,"end":700,"confidence":0.9994397226648595},
      {"word":" ","start":701,"end":840,"confidence":0.9984142043105126}
    ]
  }
}

Failure

The following is a sample response upon a failed call.

{
  "uid": string,                     # required
  "responseType": [ "recognize" ],    # required
  "recognize": {                     # required
    "status": string,                # required
    "epFlag": {                      # optional
      "status": string
    },
    "seqId": {                       # optional
      "status": string
    },
    "audio": {                       # op
      "status": 
    }
  }
}

5. Other API

Get the number of currently active stub calls and the maximum allowed stub count for the domain.

curl --location 'https://clovaspeech-gw.ncloud.com:50051/api/v1/${domainId}/active-calls' \
--header 'Authorization: Bearer ${API_KEY}'

Response

This section describes the response format.

Response body

The response body includes the following data:

Response example

{
    "activeCalls": 3,
    "maxCalls": 15,
    "timestamp": "2025-04-25T12:09:27.382+09:00"
}