Lower Third Recognition

Module Description

Lower Third Recognition reads the names displayed on screen text-inserts and associates them to the corresponding person. Create an individual and unique dataset of your own repository of personalities.

For historical reason the Lower Third Recognition module can be used to extract training data from videos. Please use Face Dataset Creation to create your training data from videos with lower thirds.

Module ID: lower_third_recognition

Module Parameters

Name	Type	Default	Description
detect_single_name	boolean	false	Lower Third Recognition is optimized on detecting full names. Enable this to detect single names as well.
apply_generic	boolean	true	Use the generic method to find names
name_dictionary_list	List of strings	null	You can provide a custom names dictionary to detect your custom names. Only names from this list will be detected. Example: ["Angela Merkel", "Markus Söder"]
min_face_size	integer	112	Minimum size of the smallest side of the face in pixels
sharpness_threshold	integer	40	Minimum quality of the face (lower number is more blurry)
only_faces	boolean	true	Only take lower-thirds into account if a face is visible at the same time. You can explicitly set this to false if you also want to detect name inserts even if no faces are visible.
secondary_dictionaries	List of strings	null	You can provide a list of Dictionary object ID's. These dictionaries will be used to recognice custom name entities. Example: ["70e9f19a-e084-4f93-bae9-85949ba7bfb8", "da05faf8-abdd-411d-b46c-e3d6ad8766ae", ...]. See Dictionary Resource.
dictionaries	list of Dictionary Specification objects	[ ]	List of dictionaries to detect your own keywords/entities in the resulting transcription.
name_heuristic	string	v1	Method used to detect names in the lower thirds. Possible values are: v1 (classic method), v2 (line-based detection with higher accuracy; supports hyphenated and multi-part names, academic titles and party affiliations after the name, and ignores headlines, credits and text in the scene).
allow_unknown_surnames	boolean	false	Only with name_heuristic v2: also detect names whose surname is not contained in the name dictionaries, if the layout of the lower third clearly identifies the name. Detects more names, but may add false detections.
max_text_tilt	number	2.0	Only with name_heuristic v2: maximum tilt of the name text in degrees (range from 0 to 90). More tilted text is treated as text in the scene (e.g. name plates or signs) and ignored. Increase the value if your lower thirds are slightly rotated or animated.
min_text_height	number	0	Only with name_heuristic v2: minimum height of the name text in percent of the frame height (range from 0 to 100). Smaller text, e.g. captions or text in the scene, is ignored. 0 disables this filter.
extract_subline	boolean	false	Add the text directly below the detected name, usually the role or function of the person (e.g. Spokesperson of the Federal Government), to the result as field subtitle. Works with both values of name_heuristic.
multi_line_subline	boolean	false	Only with extract_subtitle: also include further lines below the first subtitle line, separated by commas. Can include unrelated text directly below the lower third, such as channel logos.

Example

Send the following JSON as request body via POST to the /jobs/ endpoint:

{
  "sources": [
    "{url-to-your-image}"
  ],
  "modules": {
    "lower_third_recognition": {}
  }
}

In order to get the results you can request the /jobs/{JOB_ID}/detailed-results/ endpoint. For the job below with "name_heuristic": "v2" and "extract_subtitle": true, the response looks like this:

{
  "limit": 100,
  "next": null,
  "offset": 0,
  "prev": null,
  "total": 2,
  "data": [
    {
      "detections": [
        {
          "data": {
            "x": 0.412,
            "y": 0.181,
            "w": 0.142,
            "h": 0.297
          },
          "frame_index": 1050,
          "type": "face_bbox"
        },
        {
          "data": {
            "x": 0.415,
            "y": 0.183,
            "w": 0.141,
            "h": 0.295
          },
          "frame_index": 1100,
          "type": "face_bbox"
        },
        {
          "data": {
            "x": 0.413,
            "y": 0.18,
            "w": 0.143,
            "h": 0.298
          },
          "frame_index": 1150,
          "type": "face_bbox"
        }
      ],
      "frame_end": 1175,
      "frame_start": 1050,
      "id": "3f6c2a1e-5b7d-4c1a-9e2f-8a4b6d0c1e23",
      "media_type": "video",
      "meta": {
        "from_dictionary": false,
        "label": "Max Mustermann",
        "subtitle": "Mayor of Sampletown"
      },
      "module": "lower_third_recognition",
      "source": "storage://VSY8ihIGhR5qehVjWEWC",
      "tc_end": "00:00:47:00",
      "tc_start": "00:00:42:00",
      "thumbnail": null,
      "time_end": 47.0,
      "time_start": 42.0
    },
    {
      "detections": [
        {
          "data": {
            "x": 0.538,
            "y": 0.204,
            "w": 0.121,
            "h": 0.268
          },
          "frame_index": 3400,
          "type": "face_bbox"
        },
        {
          "data": {
            "x": 0.54,
            "y": 0.206,
            "w": 0.12,
            "h": 0.266
          },
          "frame_index": 3450,
          "type": "face_bbox"
        }
      ],
      "frame_end": 3525,
      "frame_start": 3400,
      "id": "9b1e4d7a-2c3f-4e8b-a6d5-1f0c7e9a3b42",
      "media_type": "video",
      "meta": {
        "from_dictionary": false,
        "label": "Erika Musterfrau",
        "subline": "Reporter"
      },
      "module": "lower_third_recognition",
      "source": "storage://VSY8ihIGhR5qehVjWEWC",
      "tc_end": "00:02:21:00",
      "tc_start": "00:02:16:00",
      "thumbnail": null,
      "time_end": 141.0,
      "time_start": 136.0
    }
  ]
}