Audio processing

This section contains audio processing proto files.

Notes:

To download Gatekeeper proto files, click download Gatekeeper proto files.
Default values are determined by the gRPC specification. For more information, go to protocol buffers language guide .

audio_processing_service.proto

AudioProcessor

This service offers methods to ensure audio is genuine and valid. Available operations are:

Antispoofing - Detects if the audio is a playback or contains synthetic speech.
Text Validation - Checks if the speaker in the supplied audio says the expected text.
Transcription and Redaction - Transcribes supplied spoken audio into text, and optionally remove sensitive data.
Liveness Detection - Compares audio samples to determine if the Media Segment is a recording.

Normally, these operations are executed after a successful authentication to confirm that the audio is trustworthy.

gRPC method definitions

gRPC method definitions
Method Name	Request Type	Response Type	Description
DetectAudioSpoofing	DetectAudioSpoofing Request	DetectAudioSpoofing Response	Detects playback or synthesized audio attacks. Supported playback detection includes: Channel playback detection — Detects the presence of signal artifacts introduced by the recording and playback process. Footprint playback detection — Detects if a fraudster attempts to play surreptitiously recorded audio. The operation determines if two audio buffers correspond to the same utterance by comparing the current audio with a footprint of a previously collected verification passphrase. If a match is detected, the current audio is marked as a recording (only applicable for active authentication). In addition, this method can detect speech that is electronically modified to sound like a legitimate speaker, such as synthetic speech.
ValidateText	ValidateText Request	ValidateText Response	Checks that the speaker in the supplied audio is saying the required text.
DetectLiveness	DetectLiveness Request	DetectLiveness Response	Detects intra-session voice variation using a random prompt in order to confirm that a real live person is seeking access. Liveness detection is performed after successful text-dependent voice biometrics authentication. See Verify. During liveness detection, the audio is compared with the text-dependent utterance to make sure that the same person is speaking in both recordings. The randomness of the prompts ensures that fraudsters cannot use the pre-recorded audio of the authentic speaker. Text validation ensures that the speaker is saying the expected random phrase. The input is a reference audio for active authentication and one or more recordings of random phrases. The system can assist with selecting a random phrase using the GetLivenessPrompt method.
GetLivenessPrompt	GetLivenessPrompt Request	GetLivenessPrompt Response	Randomly selects a prompt from a large list of pre-configured prompts. The semi-random prompt can be used for liveness detection.
Transcribe	Transcribe Request	Transcribe Response	Provides a transcription of an audio segment and returns details about each recognized word. When specified via input parameters or configuration, the method also redacts all credit card information and returns locations of the redacted text.
Redact	Transcribe Request	Transcribe Response	Provides a transcription of an audio segment after redacting all credit card information. If the audio was already transcribed, unless the input parameters or configuration require transcribing again, the method performs redaction on the transcribed audio.

Methods with HTTP bindings

Notes:

REST API responses use the camel case naming convention, such as gkPersonId, but the REST API requests can either use the camel case or snake case naming convention, such as gk_person_id.
The following example displays root prefix and endpoint to build a complete URL, such as https://gatekeeper.api.nuance.com/v1/audios/upload-url:
- Root prefix for API: https://gatekeeper.api.nuance.com
- Endpoint: /v1/audios/upload-url
To know more about root prefixes for API that can be used for different geographies, go to Sites.

DetectAudioSpoofing

POST /v1/audios/spoofing

Detects playback or synthesized audio attacks. Supported playback detection includes:

Channel playback detection — Detects the presence of signal artifacts introduced by the recording and playback process.
Footprint playback detection — Detects if a fraudster attempts to play surreptitiously recorded audio. The operation determines if two audio buffers correspond to the same utterance by comparing the current audio with a footprint of a previously collected verification passphrase.

If a match is detected, the current audio is marked as a recording (only applicable for active authentication). In addition, this method can detect speech that is electronically modified to sound like a legitimate speaker, such as synthetic speech.

For more information, see DetectAudioSpoofing.

Endpoint

/v1/audios/spoofing

Request

{
    "context": {
        "gk_engagement_id": {
            "value": "45891471-3023-4654-bcc1-f50c5fb5ebbb"
        },
        "gk_session_id": {
            "value": "c0204252-c53f-4319-ba52-0180a7a96ee2"
        },
        "gk_scope_id": {
            "value": "3809e857-f8f6-418b-8954-2de39bd7b0f4"
        },
        "configset_id": "AutoSourceConfigset"
    },
    "gk_processed_audio_id": {
        "value": "af87c3bd-0606-41ac-b748-e1096a63e909"
    },
    "gk_person_id": {
        "value": "f7d2f69d-2854-434c-aa84-029e9ac745a9"
    },
    "syntheticSpeechDetectionParameters": {},
    "channelPlaybackDetectionParameters": {},
    "footprintPlaybackDetectionParameters": {
        "gk_voiceprint_profile_id": {
            "value": "4c9fa036-4cc3-46bc-8bdc-f4a082e63ec2"
        }
    },
    "channel_selector": {
        "channel": 1
    }
}

Response

{
    "status": {
        "statusCode": "OK",
        "statusSubCode": 0,
        "httpTransCode": 0,
        "fieldViolations": [],
        "statusDetails": [],
        "diagnosticDetails": []
    },
    "audioDetails": {
        "gkMediaSegmentId": {
            "value": "urn:mm:v2:62b900e7-d699-4764-a4e0-f9c20cede863:5b2e5765-8ff1-418e-b5a6-a6bdddca6ffe:1eeffdc9-197f-3080-919c-8558ea813cf2"
        },
        "gkProcessedAudioId": {
            "value": "8a4c64db-1520-4ef4-996f-b58fe57ee47a"
        },
        "grossAudio": "8.650s",
        "netAudio": "3.010s",
        "snrDb": 24.7154732,
        "speechLevel": 6.0422368,
        "saturationLevel": 0,
        "mediaNumberOfChannels": 1,
        "mediaSamplingRateHz": 8000,
        "processedAudioNumberOfChannels": 1,
        "processedAudioSamplingRateHz": 8000
    },
    "validity": "REASON_AUDIO_OK",
    "syntheticSpeechDetectionResult": {
        "gkMediaSegmentId": {
            "value": "urn:mm:v2:62b900e7-d699-4764-a4e0-f9c20cede863:5b2e5765-8ff1-418e-b5a6-a6bdddca6ffe:1eeffdc9-197f-3080-919c-8558ea813cf2"
        },
        "gkProcessedAudioId": {
            "value": "8a4c64db-1520-4ef4-996f-b58fe57ee47a"
        },
        "decision": "DECISION_NO_RISK_DETECTED",
        "reason": "DECISION_REASON_AUDIO_OK",
        "result": {
            "risk": 4,
            "reliability": 99
        },
        "score": -3.42693901
    },
    "channelPlaybackDetectionResult": {
        "gkMediaSegmentId": {
            "value": "urn:mm:v2:62b900e7-d699-4764-a4e0-f9c20cede863:5b2e5765-8ff1-418e-b5a6-a6bdddca6ffe:1eeffdc9-197f-3080-919c-8558ea813cf2"
        },
        "gkProcessedAudioId": {
            "value": "8a4c64db-1520-4ef4-996f-b58fe57ee47a"
        },
        "decision": "DECISION_UNCERTAIN",
        "reason": "DECISION_REASON_LOW_RELIABILITY",
        "result": {
            "risk": 669,
            "reliability": 0
        },
        "gkPlaybackCalibrationModelRevisionId": {
            "value": "41f87500-53df-4ad5-8b69-212a9aaac723"
        },
        "score": 0.634593368,
        "playbackVersion": 4
    },
    "footprintPlaybackDetectionResult": {
        "gkMediaSegmentId": {
            "value": "urn:mm:v2:62b900e7-d699-4764-a4e0-f9c20cede863:5b2e5765-8ff1-418e-b5a6-a6bdddca6ffe:1eeffdc9-197f-3080-919c-8558ea813cf2"
        },
        "gkProcessedAudioId": {
            "value": "8a4c64db-1520-4ef4-996f-b58fe57ee47a"
        },
        "decision": "DECISION_UNCERTAIN",
        "reason": "DECISION_REASON_LOW_RELIABILITY",
        "result": {
            "risk": 666,
            "reliability": 0
        },
        "score": 100
    }
}

ValidateText

POST /v1/audios/text-validation

DetectLiveness

POST /v1/audios/liveness

Detects intra-session voice variation using a random prompt in order to confirm that a real live person is seeking access. Liveness detection is performed after successful text-dependent voice biometrics authentication. See Verify.
During liveness detection, the audio is compared with the text-dependent utterance to make sure that the same person is speaking in both recordings. The randomness of the prompts ensures that fraudsters cannot use the pre-recorded audio of the authentic speaker. Text validation ensures that the speaker is saying the expected random phrase.
The input is a reference audio for active authentication and one or more recordings of random phrases. The system can assist with selecting a random phrase using the GetLivenessPrompt method.

For more information, see DetectLiveness.

Endpoint

/v1/audios/liveness

Request

{
    "context": {
        "gk_engagement_id": {
            "value": "45891471-3023-4654-bcc1-f50c5fb5ebbb"
        },
        "gk_session_id": {
            "value": "c0204252-c53f-4319-ba52-0180a7a96ee2"
        },
        "gk_scope_id": {
            "value": "3809e857-f8f6-418b-8954-2de39bd7b0f4"
        },
        "configset_id": "AutoSourceConfigset"
    },
    "gk_processed_audio_id": {
        "value": "2b18b214-86cc-4695-a51c-c1e0994d7e9c"
    },
    "gk_person_id": {
        "value": "f7d2f69d-2854-434c-aa84-029e9ac745a9"
    },
    "random_audio_set": [
        {
            "gk_processed_audio_id": {
                "value": "22d3ef64-4d46-4639-8710-ceee7ba2c8d2"
            },
            "expected_text": "0 1 2 3 4 5 6 7 8 9",
            "channel_selector": {
                "channel": 1
            }
        },
        {
            "gk_processed_audio_id": {
                "value": "e2388872-6f9f-44ef-bcbd-bd240310984a"
            },
            "expected_text": "0 1 2 3 4 5 6 7 8 9",
            "channel_selector": {
                "channel": 1
            }
        }
    ],
    "channel_selector": {
        "channel": {}
    }
}

Response

{
    "status": {
        "statusCode": "OK",
        "statusSubCode": 0,
        "httpTransCode": 0,
        "fieldViolations": [],
        "statusDetails": [],
        "diagnosticDetails": []
    },
    "result": {
        "riskEngineResult": {
            "risk": 666,
            "reliability": 0
        },
        "decision": "DECISION_UNCERTAIN",
        "reason": "DECISION_REASON_LOW_RELIABILITY",
        "details": [
            {
                "gkMediaSegmentId": {
                    "value": "urn:mm:v2:265061d1-27cc-48c4-a92e-0959cd0139b5:90385863-1f37-402c-85b0-6282cc5e16cc:1eeffdc9-197f-3080-919c-8558ea813cf2"
                },
                "gkProcessedAudioId": {
                    "value": "22d3ef64-4d46-4639-8710-ceee7ba2c8d2"
                },
                "validity": "REASON_AUDIO_OK",
                "biometricScore": 6.62396479,
                "textValidationScore": 0.941
            },
            {
                "gkMediaSegmentId": {
                    "value": "urn:mm:v2:32faf69e-8282-4720-b73a-c2066c39f1a7:e88fd8ce-56cf-41bc-9828-b59bc2f00fb6:1eeffdc9-197f-3080-919c-8558ea813cf2"
                },
                "gkProcessedAudioId": {
                    "value": "e2388872-6f9f-44ef-bcbd-bd240310984a"
                },
                "validity": "REASON_AUDIO_OK",
                "biometricScore": 6.28422356,
                "textValidationScore": 0.921
            }
        ],
        "gkReferenceProcessedAudioId": {
            "value": "2b18b214-86cc-4695-a51c-c1e0994d7e9c"
        },
        "livenessDetectionScore": 6.45409393
    }
}

GetLivenessPrompt

GET /v1/prompts/liveness

Transcribe

POST /v1/audios/transcription

Provides a transcription of an audio segment and returns details about each recognized word. When specified via input parameters or configuration, the method also redacts all credit card information and returns locations of the redacted text.

For more information, see Transcribe.

Endpoint

/v1/audios/transcription

Request

{
    "context": {
        "gk_engagement_id": {
            "value": "45891471-3023-4654-bcc1-f50c5fb5ebbb"
        },
        "gk_session_id": {
            "value": "c0204252-c53f-4319-ba52-0180a7a96ee2"
        },
        "gk_scope_id": {
            "value": "3809e857-f8f6-418b-8954-2de39bd7b0f4"
        },
        "configset_id": "AutoSourceConfigset"
    },
    "gk_processed_audio_id": {
        "value": "af87c3bd-0606-41ac-b748-e1096a63e909"
    },
    "gk_person_id": {
        "value": "f7d2f69d-2854-434c-aa84-029e9ac745a9"
    },
    "language": "eng-USA",
    "channel_selector": {
        "channel": {
            "value": 1
        }
    }
}

Response

{
    "status": {
        "statusCode": "OK",
        "statusSubCode": 0,
        "httpTransCode": 0,
        "fieldViolations": [],
        "statusDetails": [],
        "diagnosticDetails": []
    },
    "result": {
        "language": "eng-USA",
        "text": "012-3456789",
        "transcribed": true,
        "gkProcessedAudioId": {
            "value": "5d38d16d-e18e-4fab-a5b9-385e42928fbe"
        },
        "gkTextId": {
            "value": "54ef587f-72e1-4f13-bef6-1b311206f617"
        },
        "transcribeAudioDetails": {
            "gkMediaSegmentId": {
                "value": "urn:mm:v2:62b900e7-d699-4764-a4e0-f9c20cede863:5b2e5765-8ff1-418e-b5a6-a6bdddca6ffe:1eeffdc9-197f-3080-919c-8558ea813cf2"
            },
            "gkProcessedAudioId": {
                "value": "5d38d16d-e18e-4fab-a5b9-385e42928fbe"
            },
            "grossAudio": "8.650s",
            "netAudio": "7.960s",
            "absStartMs": 0,
            "absEndMs": 7960,
            "initialSilence": "0.600s",
            "mediaNumberOfChannels": 1,
            "mediaSamplingRateHz": 8000,
            "processedAudioNumberOfChannels": 1,
            "processedAudioSamplingRateHz": 8000
        },
        "textDetails": {
            "gkTextId": {
                "value": "54ef587f-72e1-4f13-bef6-1b311206f617"
            },
            "dataPack": {
                "language": "eng-USA",
                "topic": "GEN",
                "version": "4.12.1",
                "id": "GMT20231026205000"
            },
            "wordDetails": [
                {
                    "redactionStatus": "REDACTION_STATUS_PLAIN",
                    "text": "012-3456789",
                    "startMs": 790,
                    "endMs": 7370,
                    "silenceAfterWord": "0s"
                }
            ],
            "redactionStatus": "REDACTION_STATUS_PLAIN",
            "formattedText": "012-3456789",
            "minimallyFormattedText": " zero one two three four five six seven eight nine"
        }
    }
}

Redact

POST /v1/audios/redaction

audio_processing_messages.proto

ChannelPlaybackDetectionParameters

Enables Channel Playback Detection. Included in DetectAudioSpoofingRequest.

ChannelPlaybackDetectionResult

Contains the Channel Playback Detection scores and decision. Included in DetectAudioSpoofing.

ChannelPlaybackDetectionResult
Field	Type	Description
gk_media_segment_id	nuance.biosec.v1.UniqueId	Unique ID of the media segment used for detection (if only a subset of audio was used, the value may be different from the input).
gk_processed_audio_id	nuance.biosec.v1.UniqueId	Unique ID of the processed audio.
decision	nuance.biosec.v1.Decision	Indicates whether the given audio was identified as playback.
reason	DecisionReason	The reason of the decision.
result	nuance.biosec.v1.RiskEngineResult	Level of risk and its reliability, computed by the risk engine.
score	google.protobuf.FloatValue	Raw Channel Playback Detection score.
playback_version	google.protobuf.Int32Value	Playback algorithm version that the system used to compute the score.
gk_playback_calibration_model_revision_id	nuance.biosec.v1.UniqueId	Unique ID of the calibration model that the system used to compute the score.

DataPack

Contains details about the data pack that the system used to perform transcription.

DataPack
Field	Type	Description
language	google.protobuf.StringValue	Optional. Data pack language. List of available languages can be found here: https://docs.nuance.com/mix/overview/mix-geographies/ .
topic	google.protobuf.StringValue	Optional. Data pack topic. Range and semantic is the same values as returned by Nuance Recognizer API .
version	google.protobuf.StringValue	Optional. Data pack version. Range and semantic is the same values as returned by Nuance Recognizer API .
id	google.protobuf.StringValue	Optional. Data pack identifier string, including nightly build information if applicable. Range and semantic is the same values as returned by Nuance Recognizer API .

DetectAudioSpoofingRequest

Input message that defines parameters for DetectAudioSpoofing.

Uses the context field as follows:

gk_engagement_id: Mandatory
gk_session_id: Mandatory
gk_scope_id: Mandatory
configset_id: Mandatory

DetectAudioSpoofingRequest
Field	Type	Description
context	nuance.biosec.v1.Context	Mandatory. General request context.
One of input:		Mandatory. Specified the audio for processing.
gk_media_segment_id	nuance.biosec.v1.UniqueId	Unique ID of the new media segment.
gk_processed_audio_id	nuance.biosec.v1.UniqueId	Unique ID of the processed audio.
gk_person_id	nuance.biosec.v1.UniqueId	Optional, owner of the audio. When set, the system handles the audio as personal information.
syntheticSpeechDetectionParameters	SyntheticSpeechDetectionParameters	Optional. Setting this value enables Synthetic Speech Detection.
channelPlaybackDetectionParameters	ChannelPlaybackDetectionParameters	Optional. Setting this value enables Channel Playback Detection.
footprintPlaybackDetectionParameters	FootprintPlaybackDetectionParameters	Optional. Setting this value enables Footprint Playback Detection.
channel_selector	nuance.biosec.v1.AudioChannelSelector	Optional. Specifies the processed channel when processing media with multiple channels. This parameter is ignored if the media has a single channel or if the input is gk_processed_audio_id.

DetectAudioSpoofingResponse

Output message that defines parameters returned by DetectAudioSpoofing.

DetectAudioSpoofingResponse
Field	Type	Description
status	nuance.rpc.Status	General operation status.
audio_details	nuance.biosec.v1.AudioDetails	General audio metrics, such as gross audio, net audio, SNR, and so on.
validity	nuance.biosec.v1.AudioValidity	Audio validity status. For invalid audio, indicates the invalidity reason.
synthetic_speech_detection_result	SyntheticSpeechDetectionResult	Result of Synthetic Speech Detection.
channel_playback_detection_result	ChannelPlaybackDetectionResult	Result of Channel Playback Detection.
footprint_playback_detection_result	FootprintPlaybackDetectionResult	Result of Footprint Playback Detection.

DetectLivenessRequest

Input message that defines parameters for DetectLiveness.

Uses the context field as follows:

gk_engagement_id: Mandatory
gk_session_id: Mandatory
gk_scope_id: Mandatory
configset_id: Mandatory

DetectLivenessRequest
Field	Type	Description
context	nuance.biosec.v1.Context	Mandatory. General request context.
One of audio:		Mandatory. Audio that the system uses as reference when comparing it with the random audio.
gk_media_segment_id	nuance.biosec.v1.UniqueId	Unique ID of the new media segment.
gk_processed_audio_id	nuance.biosec.v1.UniqueId	Unique ID of the processed audio.
random_audio_set	repeated RandomAudio	Mandatory. One or more audio samples of the person responding to a liveness prompt.
gk_person_id	nuance.biosec.v1.UniqueId	Optional, owner of the audio. When set, the system handles the audio as personal information.
channel_selector	nuance.biosec.v1.AudioChannelSelector	Optional. Specifies the processed channel when processing media with multiple channels. This parameter is ignored, if the media has a single channel or if the input is gk_processed_audio_id.

DetectLivenessResponse

Output message that defines parameters returned by DetectLiveness.

DetectLivenessResponse
Field	Type	Description
status	nuance.rpc.Status	General operation status.
result	DetectLivenessResult	Liveness Detection result.

DetectLivenessResult

Contains the liveness Detection scores and decision. Included in DetectLivenessResponse.

DetectLivenessResult
Field	Type	Description
risk_engine_result	nuance.biosec.v1.RiskEngineResult	Level of risk and its reliability, computed by the risk engine.
liveness_detection_score	google.protobuf.FloatValue	Raw liveness detection score.
decision	nuance.biosec.v1.Decision	Indicates if the media segment is a recording.
reason	DecisionReason	The reason of the decision.
details	repeated RandomAudioDetails	Detailed information for each random phrase recording.
gk_reference_processed_audio_id	nuance.biosec.v1.UniqueId	Unique ID of the processed audio that the system uses as reference when comparing it with the random audio.

FootprintPlaybackDetectionParameters

Enables Footprint Playback Detection. Included in DetectAudioSpoofingRequest.

FootprintPlaybackDetectionParameters
Field	Type	Description
gk_voiceprint_profile_id	nuance.biosec.v1.UniqueId	Mandatory. Unique ID of the voiceprint that the method uses to retrieve previous footprints.

FootprintPlaybackDetectionResult

Contains the Footprint Playback Detection scores and decision. Included in DetectAudioSpoofing.

FootprintPlaybackDetectionResult
Field	Type	Description
gk_media_segment_id	nuance.biosec.v1.UniqueId	Unique ID of the media segment used for detection (if only a subset of audio was used, the value may be different from the input).
gk_processed_audio_id	nuance.biosec.v1.UniqueId	Unique ID of the processed audio.
decision	nuance.biosec.v1.Decision	Indicates whether the given audio was detected as playback.
reason	DecisionReason	The reason of the decision.
result	nuance.biosec.v1.RiskEngineResult	Level of risk and its reliability, computed by the risk engine.
score	google.protobuf.FloatValue	Raw Footprint Playback Detection score.

GetLivenessPromptRequest

Input message that defines parameters for GetLivenessPrompt.

Uses the context field as follows:

gk_engagement_id: Mandatory
gk_session_id: Mandatory
gk_scope_id: Mandatory
configset_id: Mandatory

GetLivenessPromptRequest
Field	Type	Description
context	nuance.biosec.v1.Context	Mandatory. General request context.

GetLivenessPromptResponse

Output message that defines parameters returned by GetLivenessPrompt.

GetLivenessPromptResponse
Field	Type	Description
status	nuance.rpc.Status	General operation status.
expected_text	string	The expected text.
ivr_info	string	Additional information meant to assist with prompting the caller to say the expected phrase.

RandomAudio

Contains text and an audio recording of a random phrase. Included in DetectLivenessRequest.

RandomAudio
Field	Type	Description
One of audio:		Mandatory. Audio for processing.
gk_media_segment_id	nuance.biosec.v1.UniqueId	Unique ID of the new media segment.
gk_processed_audio_id	nuance.biosec.v1.UniqueId	Unique ID of the processed audio.
expected_text	string	Mandatory. Expected text.
channel_selector	nuance.biosec.v1.AudioChannelSelector	Optional. Specifies the processed channel when processing media with multiple channels. This parameter is ignored if the media has a single channel or if the input is gk_processed_audio_id.

RandomAudioDetails

Contains detailed scores and decision for each random phrase recording. Included in DetectLivenessResult.

RandomAudioDetails
Field	Type	Description
gk_media_segment_id	nuance.biosec.v1.UniqueId	Unique ID of the media segment used (if only a subset of audio was used, the value may be different from the input).
gk_processed_audio_id	nuance.biosec.v1.UniqueId	Unique ID of the processed audio.
validity	nuance.biosec.v1.AudioValidity	Audio validity status. In case audio is not valid, the field contains the invalidity reason.
biometric_score	google.protobuf.FloatValue	Raw score from comparing this audio with the fixed text phrase used in Verify.
text_validation_score	google.protobuf.FloatValue	Raw text validation score for this media segment.

SyntheticSpeechDetectionParameters

Enables Synthetic Speech Detection. Included in DetectAudioSpoofingRequest.

SyntheticSpeechDetectionResult

Contains the Synthetic Speech Detection scores and decision. Included in DetectAudioSpoofingResponse.

SyntheticSpeechDetectionResult
Field	Type	Description
gk_media_segment_id	nuance.biosec.v1.UniqueId	Unique ID of the media segment used for detection (if only a subset of audio was used, the value may be different from the input).
gk_processed_audio_id	nuance.biosec.v1.UniqueId	Unique ID of the processed audio.
decision	nuance.biosec.v1.Decision	Indicates whether audio was classified as synthetic.
reason	DecisionReason	The reason of the decision.
result	nuance.biosec.v1.RiskEngineResult	Level of risk and reliability, computed by the risk engine.
score	google.protobuf.FloatValue	Raw Synthetic Speech Detection score.

TextDetails

Contains details about the transcribed text.

TextDetails
Field	Type	Description
gk_text_id	nuance.biosec.v1.UniqueId	Optional. Unique ID of the transcribed text.
formatted_text	google.protobuf.StringValue	Optional. Formatted text of the result, for example, $500. Word details are referred to this field.
minimally_formatted_text	google.protobuf.StringValue	Optional. Slightly formatted text of the result, for example, five hundred dollars.
data_pack	DataPack	Optional. Data pack information.
word_details	repeated WordDetails	Optional. Repeated. One or more recognized words in the result.
redaction_status	RedactionStatus	Optional. Indicates if the text is plain, masked, or redacted.
quality_check	google.protobuf.FloatValue	—————— Convoprints-Only —————— Optional. A score between 0 and 1 that represents the quality of the text. Relevant only for Convoprint APIs.
raw_quality	google.protobuf.FloatValue	Optional. A uncapped score that represents the quality of the text. Relevant only for Convoprint APIs.
gk_calibration_model_revision_id	nuance.biosec.v1.UniqueId	Optional. Unique ID of the calibration model that the system used to create the text model.
text_model_created	google.protobuf.BoolValue	Optional. Indicates if a text model has been created.

TranscribeAudioDetails

Information about the audio.

TranscribeAudioDetails
Field	Type	Description
gk_media_segment_id	nuance.biosec.v1.UniqueId	Optional. Unique ID of the media segment.
gk_processed_audio_id	nuance.biosec.v1.UniqueId	Optional. Unique ID of the audio returned by a processing method.
gross_audio	google.protobuf.Duration	Optional. Amount of gross audio including speech, noise, and silence. Range and semantic is the same values as returned by Nuance Recognizer API .
net_audio	google.protobuf.Duration	Optional. Amount of speech is equal to the utterance duration. Range and semantic is the same values as returned by Nuance Recognizer API .
abs_start_ms	google.protobuf.UInt32Value	Optional. Audio stream start time. Range and semantic is the same values as returned by Nuance Recognizer API .
abs_end_ms	google.protobuf.UInt32Value	Optional. Audio stream end time. Range and semantic is the same values as returned by Nuance Recognizer API .
initial_silence	google.protobuf.Duration	Optional. Silence observed before start of utterance. Range and semantic is the same values as returned by Nuance Recognizer API .
media_number_of_channels	google.protobuf.UInt32Value	Optional. Number of audio input channels. Values are 1 for mono and 2 for stereo.
media_sampling_rate_hz	google.protobuf.UInt32Value	Optional. Sampling rate of the input audio in Hertz. It can be different from the one used for processing.
selected_channel	nuance.biosec.v1.AudioChannelSelector	Optional. Specifies the processed channel when processing media with multiple channels. This parameter is not set if the media has a single channel.
processed_audio_number_of_channels	google.protobuf.UInt32Value	Optional. Number of audio channels in the processed audio. Values are 1 for mono and 2 for stereo.
processed_audio_sampling_rate_hz	google.protobuf.UInt32Value	Optional. Sampling rate used for processing the audio. Value is in Hertz and can differ from the value of the input audio.

TranscribeRequest

Input message that defines parameters for Transcribe.

Uses the context field as follows:

gk_engagement_id: Mandatory
gk_session_id: Mandatory
gk_scope_id: Mandatory
configset_id: Mandatory

TranscribeRequest
Field	Type	Description
context	nuance.biosec.v1.Context	Mandatory. General request context.
One of audio:		Mandatory. Audio for processing.
gk_media_segment_id	nuance.biosec.v1.UniqueId	Unique ID of the new media segment.
gk_processed_audio_id	nuance.biosec.v1.UniqueId	Unique ID of the processed audio.
language	string	Optional. The language spoken in the audio. If not specified, the method uses the actual_language processed audio attribute or the configuration parameter. At least one of them must be valid. List of available languages can be found here: https://docs.nuance.com/mix/overview/mix-geographies/ Contact Nuance account manager to get the list of available languages in your region.
gk_person_id	nuance.biosec.v1.UniqueId	Optional, owner of the audio. When set, the system handles the audio as personal information.
channel_selector	nuance.biosec.v1.AudioChannelSelector	Optional. Specifies the processed channel when processing media with multiple channels. This parameter is ignored if the media has a single channel or if the input is gk_processed_audio_id.

TranscribeResponse

Output message that defines parameters returned by Transcribe.

TranscribeResponse
Field	Type	Description
status	nuance.rpc.Status	Mandatory. General operation status.
result	TranscribeResult	Optional. Text validation result.

TranscribeResult

Contains the text validation score and decision. Included in TranscribeResponse.

TranscribeResult
Field	Type	Description
language	string	Optional. Spoken language. List of available languages can be found here: https://docs.nuance.com/mix/overview/mix-geographies/ .
text	string	Optional. Spoken text.
transcribed	bool	Indicates if a transcription has been triggered.
gk_processed_audio_id	nuance.biosec.v1.UniqueId	Optional. Unique ID of the processed audio.
gk_text_id	nuance.biosec.v1.UniqueId	Optional. Unique ID of the transcribed text.
transcribe_audio_details	TranscribeAudioDetails	Optional. Transcription audio metrics, such as gross audio, net audio, SNR, and so on.
text_details	TextDetails	Optional. Text details, such as Confidence, formatted text, word details, and so on.

ValidateTextRequest

Input message that defines parameters for ValidateText.

Uses the context field as follows:

gk_engagement_id: Mandatory
gk_session_id: Mandatory
gk_scope_id: Mandatory
configset_id: Mandatory

ValidateTextRequest
Field	Type	Description
context	nuance.biosec.v1.Context	Mandatory. General request context.
One of audio:		Mandatory. Audio for processing.
gk_media_segment_id	nuance.biosec.v1.UniqueId	Unique ID of the new media segment.
gk_processed_audio_id	nuance.biosec.v1.UniqueId	Unique ID of the processed audio.
expected_text	string	Mandatory, the expected text.
gk_person_id	nuance.biosec.v1.UniqueId	Optional, owner of the audio. When set, the system handles the audio as personal information.
channel_selector	nuance.biosec.v1.AudioChannelSelector	Optional. Specifies the processed channel when processing media with multiple channels. This parameter is ignored, if the media has a single channel or if the input is gk_processed_audio_id.

ValidateTextResponse

Output message that defines parameters returned by ValidateText.

ValidateTextResponse
Field	Type	Description
status	nuance.rpc.Status	General operation status.
result	ValidateTextResult	Text validation result.

ValidateTextResult

Contains the text validation score and decision. Included in ValidateTextResponse.

ValidateTextResult
Field	Type	Description
result	nuance.biosec.v1.RiskEngineResult	Level of risk and its reliability, computed by the risk engine.
score	google.protobuf.FloatValue	Raw text validation score.
decision	nuance.biosec.v1.Decision	Indicates if the speaker said the required text.
reason	DecisionReason	Reason of the decision.
detected_text	string	Spoken text.
gk_processed_audio_id	nuance.biosec.v1.UniqueId	Unique ID of the processed audio.

WordDetails

Contains details about the transcribed words.

WordDetails
Field	Type	Description
text	google.protobuf.StringValue	Optional. The recognized word.
confidence	google.protobuf.FloatValue	Optional. The confidence score of the recognized word, 0 to 1. Range and semantic is the same values as returned by Nuance Recognizer API .
start_ms	google.protobuf.UInt32Value	Optional. Word start offset in the audio stream. Range and semantic is the same values as returned by Nuance Recognizer API .
end_ms	google.protobuf.UInt32Value	Optional. Word end offset in the audio stream. Range and semantic is the same values as returned by Nuance Recognizer API .
silence_after_word	google.protobuf.Duration	Optional. The amount of silence, in ms, detected after the word. Range and semantic is the same values as returned by Nuance Recognizer API .
redaction_status	RedactionStatus	Optional. Indicates if the word is plain, masked, or redacted.

audio_processing_common_types.proto

DecisionReason

Audio processing and spoofing detection decision reason.

Name	Number	Description
DECISION_REASON_UNSPECIFIED	0	Decision reason is not specified.
DECISION_REASON_AUDIO_OK	1	Audio is valid.
DECISION_REASON_NOT_ENOUGH_SEGMENTS	2	Processing requires more audio segments than provided.
DECISION_REASON_INVALID_AUDIO	3	Audio format is invalid or not supported.
DECISION_REASON_AUDIO_TOO_SHORT	4	Audio does not contain the amount of speech required to complete the process.
DECISION_REASON_AUDIO_TOO_SOFT	5	Audio speech level is lower than the level required to complete the process.
DECISION_REASON_AUDIO_TOO_LOUD	6	Audio is too loud and causes saturation.
DECISION_REASON_AUDIO_TOO_NOISY	7	Audio SNR (Signal to Noise Ratio) is too low.
DECISION_REASON_SYNTHETIC_SPEECH_DETECTED	8	Audio is likely to be generated by a synthesizer.
DECISION_REASON_PLAYBACK_INDICATION	9	Audio is likely to be a playback and not a live recording.
DECISION_REASON_LOW_RELIABILITY	10	The system could not reach a reliable decision.
DECISION_REASON_INTERNAL_ERROR	999	An error occurred while making the decision.

RedactionStatus

Redaction status of word or full text.

Name	Number	Description
REDACTION_STATUS_UNSPECIFIED	0	Unspecified value.
REDACTION_STATUS_PLAIN	1	The text is written in plain.
REDACTION_STATUS_MASKED	2	The text is written in plain but it is marked as masked.
REDACTION_STATUS_REDACTED	3	The text is redacted and cannot be read in any way.