Customize the Advanced Subtitling Workflow

Using Automatic Speech Recognition

To instruct the workflow to generate the transcript for you, the generateSubtitleFromASR property can be used.

For example:

{
    "language": "en",
    "subtitlePresetId": "default",
    "generateSubtitleFromASR": true,
    "transcriptionLanguage": "en"
}

When generateSubtitleFromASR is set, a transcription will be generated automatically using the transcriptionLanguage property, but only if the transcript does not exist yet. If a transcript was already present in that language; either by being generated automatically earlier, or created manually; that transcript will be used in the workflow.

If a transcript needs to be generated automatically, this will come at an additional cost.

To learn more about automatic transcription, see this page.

Using Automatic Speech Alignment

Besides automatic transcription, automatic alignment can be used as well.

To instruct the workflow to generate the transcript for you using a given alignment file, the generateSubtitleFromASA property can be used, along with a reference to the alignment file resource.

For example:

{
    "language": "en",
    "subtitlePresetId": "default",
    "generateSubtitleFromASA": true,
    "speechAlignmentFileResource": { "id": 123456789 }
}

First, the alignment file must be uploaded as a file resource. Instructions on how to do this can be found at file resource upload.

Using Import

Instead of generating a transcript, a subtitle file can be imported and used directly.

First, the subtitle file must be uploaded as a file resource. Instructions on how to do this can be found at file resource upload.

Next, the workflow can be started.

{
    "language": "en",
    "subtitlePresetId": "default",
    "generateSubtitleFromASA": true,
    "importFileResource": { "id": 123456789 }
}

Additional properties can be set to control the import process:

{
    "language": "en",
    "subtitlePresetId": "default",
    "generateSubtitleFromASA": true,
    "importFileResource": { "id": 123456789 },

    "removeBlankSubtitles": true
}

Post Processing

The Advanced Subtitle Workflow is capable of doing post-processing on the import/generated subtitles as well.

Automatic Coloring based on Speaker Recognition

Subtitle coloring can be achieved by using the speaker recognition from the transcription. If no existing transcription is present, one will be created automatically.

There are two modes in which the coloring algorithm can be steered:

  • Consistent color per speaker (default)

  • Maximize color transitions

These settings can be adjusted in the subtitle preset settings: Automatic Speaker Colouring Mode

Consistent color per speaker

Colors are assigned to each unique speaker and will be used throughout the entire subtitle.

When choosing a color for a subtitle part, that speaker color will be used unless multiple speakers are assigned the same color within a single subtitle. If multiple speakers within a subtitle are assigned the same color, a different color will be used, but the color assigment will remain unchanged.

Maximize color transitions

Colors are initially assigned to each unique speaker. However, these can change on two conditions:

  1. When the Current Speaker’s Color Matches the previous speaker’s color, but their id’s differ:

    • If the color of the current speaker matches that of the previous speaker, yet their IDs are different, indicating a transition between speakers, the color is updated.

  2. When multiple speakers are assigned the same color within a single subtitle:

    • If multiple speakers within a subtitle are assigned the same color, it suggests a need for color differentiation, prompting an update in color assignments.

When a color assignment changes, that color will be used from that point on till the end of the subtitle or when a new color assignment is needed.

{
    "language": "en",
    "subtitlePresetId": "default",
    "applySpeakerIds": true,
    "transcriptionLanguage": "en"
}
If a transcript needs to be generated automatically, this will come at an additional cost.

Subtitle Timing Corrections

When importing subtitles, the timings of the subtitles might be off and can be corrected.

The workflow input parameter which controls this is called subtitleTimingCorrections of which an example is given below.

The corrections are a series of timing adjustments performed on each subtitle.

For example, the frame rate of the source subtitle might not match the frame rate of the current media file.

Second, the timings of the source subtitle might include the start time code of the source media file. Since subtitles in Limecraft Flow operate on media playback time, i.e., the offset in seconds/frames since the beginning of the media, that time code needs to be subtracted from the timings of the subtitles.

All changes will be stored under the enhancements property of the SubtitleObject.

{
  "language": "en",
  "subtitlePresetId": "default",
  "subtitleTimingCorrections": [
    { "type": "OffsetNormalize" },
    { "type": "Spacing", "minSpacing": "00:00:00:01", "alignGapThreshold": "00:00:02:00" },
    { "type": "Offset", "offset": "00:00:30:00", "offsetDirection": "add" },
    {
        "type": "FrameRate",
        "sourceRate": { "numerator": 3000, "denumerator": 1001 },
        "targetRate": { "numerator": 25, "denumerator": 1 }
    }
  ]
}

We currently support the following types of corrections:

  • FrameRate

  • Offset

  • OffsetNormalize

  • Spacing

Frame Rate

The timings of the subtitle can be converted from that source frame rate to the frame rate of the media file.

{
  "type": "FrameRate",
  "sourceRate": { "numerator": 3000, "denumerator": 1001 },
  "targetRate": { "numerator": 25, "denumerator": 1 }
}

Offset

This will add or subtract an offset to the timings of the subtitles.

For this, two parameters need to be specified:

  • offset

    • This can be in the 10:00:00.000 (media) o 10:00:00:00 (SMPTE) notation

  • offsetDirection: add or subtract

{
  "type": "Offset",
  "offset": "00:00:30:00",
  "offsetDirection": "add"
}

The offset will be converted into seconds and depending on the direction added or subtracted from each timestamp (converted from frames) of each subtitle and then converted back into frame timestamps.

OffsetNormalize

OffsetNormalize will adjust the timings of the subtitles to fit into the first hour.

For this, we check the timestamp of the first subtitle. The aim is to get this timestamp below one hour and above zero.

This is achieved by adding/subtracting whole hours to the timestamp until it falls within that range.

This correction is then applied to all the following subtitles.

{
  "type": "OffsetNormalize"
}

Spacing

This correction will adjust the spacing between consecutive subtitles to match the spotting rules.

This requires two parameters:

  • minSpacing

    • Force a timing gap between consecutive subtitles

  • alignGapThreshold

    • The maximum time interval, between two consecutive subtitles in which they are adjusted to align, with taking minSpacing into account.

{
  "type": "Spacing",
  "minSpacing": "00:00:00:01",
  "alignGapThreshold": "00:00:02:00"
}

Shot Boundary Alignment

Shot Boundary alignment will adjust the timings of the subtitles to match shot boundaries.

{
    "language": "en",
    "subtitlePresetId": "default",
    "applyShotAlignment": true
}

The adjustment is controlled by the Shot Snap Threshold which can be adjusted in the subtitle editor settings page.

When generating subtitles from a transcript, this is done automatically.