I am trying to use Amazon's new streaming transcribe API from Go 1.11. Currently Amazon provides Java SDK only so I am trying the low-level way.
The only relevant piece of documentation is here but it does not show the endpoint. I have found it in a Java example that it is https://transcribestreaming.<region>.amazonaws.com
and I am trying the Ireland region i.e. https://transcribestreaming.eu-west-1.amazonaws.com
. Here is my code to open an HTTP/2 bi-directional stream:
import (
"crypto/tls"
"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/aws/external"
"github.com/aws/aws-sdk-go-v2/aws/signer/v4"
"golang.org/x/net/http2"
"io"
"io/ioutil"
"log"
"net/http"
"os"
"time"
)
const (
HeaderKeyLanguageCode = "x-amzn-transcribe-language-code" // en-US
HeaderKeyMediaEncoding = "x-amzn-transcribe-media-encoding" // pcm only
HeaderKeySampleRate = "x-amzn-transcribe-sample-rate" // 8000, 16000 ... 48000
HeaderKeySessionId = "x-amzn-transcribe-session-id" // For retrying a session. Pattern: [a-fA-F0-9]{8}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{12}
HeaderKeyVocabularyName = "x-amzn-transcribe-vocabulary-name"
HeaderKeyRequestId = "x-amzn-request-id"
)
...
region := "eu-west-1"
cfg, err := external.LoadDefaultAWSConfig(aws.Config{
Region: region,
})
if err != nil {
log.Printf("could not load default AWS config: %v", err)
return
}
signer := v4.NewSigner(cfg.Credentials)
transport := &http2.Transport{
TLSClientConfig: &tls.Config{
// allow insecure just for debugging
InsecureSkipVerify: true,
},
}
client := &http.Client{
Transport: transport,
}
signTime := time.Now()
header := http.Header{}
header.Set(HeaderKeyLanguageCode, "en-US")
header.Set(HeaderKeyMediaEncoding, "pcm")
header.Set(HeaderKeySampleRate, "16000")
header.Set("Content-type", "application/json")
// Bi-directional streaming via a pipe.
pr, pw := io.Pipe()
req, err := http.NewRequest(http.MethodPost, "https://transcribestreaming.eu-west-1.amazonaws.com/stream-transcription", ioutil.NopCloser(pr))
if err != nil {
log.Printf("err: %+v", err)
return
}
req.Header = header
_, err = signer.Sign(req, nil, "transcribe", region, signTime)
if err != nil {
log.Printf("problem signing headers: %+v", err)
return
}
// This freezes and ends after 5 minutes with "unexpected EOF".
res, err := client.Do(req)
...
Problem is that executing the request (client.Do(req)
) freezes for five minutes and then ends with the "unexpected EOF" error.
Any ideas what I am doing wrong? Did someone successfully use the new streaming transcribe API without the Java SDK?
EDIT (March 11, 2019):
I tested this again and now it does not time out but immediately returns 200 OK
response. There is an "exception" in the response body though: {"Output":{"__type":"com.amazon.coral.service#SerializationException"},"Version":"1.0"}
I tried opening the HTTP2 stream with io.Pipe
(like the code above) and also with a JSON body described in the documentation:
{
"AudioStream": {
"AudioEvent": {
"AudioChunk": ""
}
}
}
The result was the same.
EDIT (March 13, 2019):
As mentioned by @gpeng, removing the content-type
from headers will fix the SerializationException
. But then there is an IAM exception and it is needed to add the transcription:StartStreamTranscription
permission to your IAM user. That is though nowhere in the AWS IAM console and must be added manually as a custom JSON permission :/
There is also a new/another documentation document here which shows incorrect host
and a new content-type
(do not use that content-type
, the request will return 404 with it).
After removing the content-type
, and adding the new permission, now I am getting an exception {"Message":"A complete signal was sent without the preceding empty frame."}
. Also writing to the pipe blocks forever, so I am stuck again. The messages described in the new documentation are different than in the old one, now finally binary, but I do not understand them. Any ideas how to send such HTTP2 messages in Go?
EDIT (Match 15, 2019):*
If you get HTTP 403 error about signature mismatch, then do not set the transfer-encoding
and x-amz-content-sha256
HTTP headers. When I set them, sign the request with AWS SDK's V4 signer, then I receive HTTP 403 The request signature we calculated does not match the signature you provided.