KR20190069695A - Video content creation device with subtitle - Google Patents

Abstract

The present invention relates to a video content creating device with a subtitle. According to the present invention, a subtitle character is inserted into a speech bubble part to be displayed, thereby clearly confirming a relationship between a subtitle and a talker. This is even more helpful in a place where a speaker or the like cannot be utilized. In addition, an important part on a screen can be prevented from being hidden by enabling the speech bubble to be provided in an even tone part of an image or by changing a position of the speech bubble according to an instruction of a user. Furthermore, a form of the speech bubble can be varied according to a comment of the talker or characteristics of lines.

Description

Title: Video content creation device with subtitle}

BACKGROUND OF THE INVENTION 1. Field of the Invention [0002] The present invention relates to a video content creating apparatus, and more particularly, to a technique for creating video content having a caption attached thereto.

Conventionally, in order to understand the movie contents of a foreign language, the characters of the characters were displayed on the part of the video screen in the native language. Through this, the viewer could understand the conversation contents through the mother tongue characters even if the character talks in a foreign language.

Recently, even if the characters are conversing in their own language, a broadcast program for displaying the contents of the conversation as subtitles is becoming prevalent for reasons of presentation. Of course, in addition to the contents of the characters, subtitles are being used for the explanation of the video contents.

Typically, the viewer recognizes who is speaking through the voice output along with the screen. Therefore, when only the subtitle output is used as in the conventional art, it is impossible to know who is speaking when watching the screen in the mute state.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a technique for easily knowing a relationship between a speaker and a caption.

According to another aspect of the present invention, there is provided an apparatus for generating a video content having a caption according to the present invention, the apparatus comprising: a content creator for creating data necessary for providing caption- A speech bubble region determining means for determining a speech bubble region suitable for display of a speech balloon in an image of a time extracted by the speech balloon time extracting means; A balloon image determining means for determining a balloon image to be synthesized in the balloon region determined by the region determining means; a caption character determining means for determining a caption character to be synthesized with the balloon image determined by the balloon image determining means; Information on time to be performed, And bubble data creating means for creating bubble data by converting at least one piece of information among the information on the region, the information on the region, the information on the region, the information on the region, the information on the balloon image, and the information on the caption character, Can be reproduced together with the image content data, thereby providing image contents with captions by the speech balloon.

According to the present invention, the relationship between the caption and the speaker can be clearly confirmed by inserting and displaying the caption character in the speech bubble portion. This is even more helpful in places where you can not use speakers, for example.

Also, it is possible to prevent the important part on the screen from being hidden by allowing the speech balloon to be provided in the flat part of the image in the image or by changing the position of the speech balloon according to the instruction of the user. Also, depending on the utterance of the speaker or the nature of the dialogue, the form of speech balloon can be variously changed.

BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a diagram for explaining an apparatus for producing image contents with captions according to an embodiment of the present invention; Fig.

Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. However, some configurations which are not related to the gist of the present invention may be omitted or compressed, but the configurations omitted are not necessarily those not necessary in the present invention, and they may be combined by a person having ordinary skill in the art to which the present invention belongs. .

1 is a diagram for explaining an apparatus for generating an image content having a caption according to an embodiment of the present invention. 1, the broadcasting system includes a content creating apparatus 1, a content transmitting apparatus 2, a broadcasting apparatus 3, a content reproducing apparatus 4, and a content display apparatus 5. [

The content creating apparatus 1 is provided for composing the caption using the speech balloon to the content data stored in advance to create speech balloon data.

The content transmitting apparatus 2 acquires the content data and speech balloon data from the content creating apparatus 1 and multiplexes them and transmits the multiplexed data to the broadcasting apparatus 3 via a network such as an in-premise line, a public network, the Internet, do.

The broadcast apparatus 3 receives the multiplexed data sent from the content transmitting apparatus 2 and broadcasts the multiplexed data through the antenna.

The content reproduction apparatus 4 receives and interprets the multiplexed data transmitted from the broadcasting apparatus 3 and displays an image having the caption using the speech balloon on the content display apparatus 5. [ The content display apparatus 5 displays an image with a caption using speech balloon according to a signal sent from the content reproduction apparatus 4. [

The content creating apparatus 1 includes a data creation control unit 11, an input unit 12, a display output unit 13, a time count unit 14, and a storage unit 15.

The input unit 12 is an input device such as a mouse, a keyboard, a touch panel, or a joystick, and can input operation information input by a user to the data creation control unit 11 by operating these devices.

The storage unit 15 is a recording device such as a hard disk. The storage unit 15 stores content data, caption list data, and shape data.

The content data may be stream data in which video and audio are encoded by a method such as MPEG (Moving Picture Experts Group).

The caption list data may include information on a caption character and a time for displaying it. For example, the caption start time, the caption continuation time, and the caption character may be registered in the caption list data. Here, the caption start time is information indicating how long a caption character is displayed at the beginning of the content. The caption continuation time is information on the time when the corresponding caption character is continuously displayed.

The balloon shape data is data defining the shape of the balloon. For example, in the speech ball shape data, the name of the speech ball shape, data for output, and the like may be stored.

4 is a diagram showing an example of the data structure of speech balloon data. As shown in Fig. 4, the output data includes, for example, a caption start time, a caption continuation time, a caption character expansion speed, a caption character attribute, a speech bubble range start point, a speech bubble start point, . The caption start time and the caption continuation time are information on the time to display a speech balloon. The subtitle character development speed, the subtitle character attribute, and the subtitle character are information on the subtitle character. The speech balloon range and the speech balloon start point are information about the speech balloon region in the image suitable for display of the speech balloon. The speech balloon shape is information about speech balloons to be synthesized in the speech balloon region. The speech balloon data may be data in which at least one piece of information among the information on the time to display the speech balloon, information on the speech balloon region, information on the speech balloon image, and information on the caption character is data. For example, speech balloon data is described as meta-language. Here, the caption start time, caption continuation time, and caption character are the same as those of the caption list data.

The caption character development speed is information indicating the speed at which caption characters are sequentially displayed at the beginning of the caption within the caption continuation time. The subtitle character attribute is information indicating the font type of the subtitle character, the color of the subtitle character, the background and transparency of the subtitle character, and the frame type of the subtitle character. The speech balloon range is information indicating a position on the screen for synthesizing speech balloons. The speech balloon start point is information indicating the position on the screen where the speech balloon starts. The speech ball shape is information indicating the name of the speech ball registered in the speech ball data.

The time counting section 14 measures the time. The display output unit 13 displays an image for creating a video image and a speech balloon in accordance with a signal from the data creation control unit 11 and outputs audio.

The data creation control unit 11 detects the start time and the fall time of the voice by reproducing the content data, and recognizes the caption start time and the caption continuation time. The data creation control unit 11 creates the caption list data in association with the recognized caption start time and the caption continuation time and the caption character input by the user through the input unit 12 and stores the created caption list data in the storage unit 15. [ The data creation control unit 11 detects the start time of the voice with reference to the caption list data, and displays and outputs the video and audio of the display time on the display output unit 13. [ The data creation control unit 11 composites the shape of the displayed image and synthesizes the caption characters in the balloon shape. When the final determination by the user is made as a result of the composition, the data creation control unit 11 creates speech balloon data of the subtitle start time. The data creation control unit 11 synthesizes the speech ball data created at each caption start time to generate final speech balloon data. The data creation control unit 11 stores the final speech balloon data generated in the storage unit 15. [

The contents transmitting apparatus 2 includes a multiplexing control section 21, an operation section 22, an error correction code adding section 23, a digital modulation section 24 and a transmission section 25.

The content reproduction apparatus 4 includes a reproduction control section 41, an operation section 42, a synthesis transfer section 43, a time count section 44, a speech balloon shape storage section 45, a reception section 46, And an error correction unit 48.

The content display device 5 includes a display output device unit 51 and a drive circuit unit 52. [

Hereinafter, the operation of the content creating apparatus 1 will be described.

The data creation control unit 11 of the content creation apparatus 1 reads desired content data stored in the storage unit 15 in accordance with an instruction input by the user through the input unit 12, ) And outputs a voice.

Thereafter, the data creation control unit 11 determines whether or not the voice start timing comes through voice recognition. When the start timing of the voice is reached, the data creation control section 11 sets the start timing of the voice as the caption start time and the elapsed time until the time of the voice gradually decreases as the caption continuation time, To the user. The data creation control unit 11 stores the caption start time, the caption continuation time and the caption character in the storage unit 15 as part of the caption list data.

The data creation control unit 11 determines whether or not the reproduction of the content data is completely finished. The data creation control unit 11 arranges the closed caption list data in one time and finally creates the caption list data for the content and stores it in the storage unit 15. [

The data creation control unit 11 recognizes the caption start time and the caption continuation time in order from the head with reference to the caption list data. Subsequently, the data creation control section (11) refers to the content data and reproduces the video and audio from the recognized caption start time to the caption continuation time on the display output section (13).

Subsequently, the data creation control unit 11 calculates the flatness of the color in the video from the caption start time to the caption continuation time, and extracts a portion having the same color tone (hereinafter, referred to as a flat portion). Then, the data creation control unit 11 recognizes a rectangle having a size enough to fit into the extracted flat portion. Subsequently, the data creation control unit 11 displays the synthesized image on the display output unit 13 with the image at the caption start time, so that the recognized rectangle is represented by a dotted line frame (hereinafter referred to as quadrangle frame). At this time, the data creation control unit 11 adds specific markings to the four corners of the quadrangular frame.

Then, the data creation control unit 11 causes the display output unit 13 to display the quadrangular frame on the display output unit 13 so as to ask the user whether it is good as a range for displaying speech bubbles. If there is a correction instruction from the user, Display range. At this time, the data creation control unit 11 temporarily stores four corner coordinates of the quadrangular frame in the memory. In addition, modification by the user is performed by the input unit 12. For example, the user can modify the size and / or position of the quadrangle by positioning and dragging the mouse pointer at four or four corners of the quadrangle.

Then, the data creation control unit 11 recognizes the face portion of the person in the image. As a recognition method at this time, the face portion of the person can be recognized based on the color of the skin or the shape of the face.

Subsequently, the data creation control unit 11 obtains the area of the recognized face portion and determines whether the area exceeds a predetermined threshold value. The data creation control unit 11 detects the mouth portion and displays a reference line toward the diagonal intersection of the quadrangular frame at the mouth (hereinafter, referred to as quadrangular frame center) on the display output unit 13, Display the candidate point of the speech balloon on the reference line.

On the other hand, if not exceeding, the data creation control unit 11 recognizes the center portion of the face, displays the reference line on the display output unit 13 from the center portion toward the center of the quadrangle frame, Mark candidates.

The data creation control unit 11 corrects the speech balloon starting point according to an instruction from the user through the input unit 12 and stores the coordinates of the final speech balloon starting point in the memory. In addition, in the case where there is no modification by the user, the data creation control unit 11 stores coordinates that are candidates for the balloon start point.

The data creation control unit 11 reads the data relating to the balloon shape that is preset in the standard balloon shape in the storage unit 15 and sets the size of the balloon shape so as to be within the determined four- And causes the display unit 13 to display the speech balloon image after the resizing in the four corners.

Subsequently, the data creation control unit 11 corrects the speech balloon image in accordance with an instruction from the user. Specifically, the shape, size, and direction of speech balloon are modified. This modification is performed by allowing the user to select a desired shape in a dialog box representing the shape of the balloon. The size modification may also be performed by dragging the speech balloon in the display.

If the modification by the user is not ended or corrected, the data creation control unit 11 determines the final speech balloon image. At this time, the data creation control unit 11 temporarily stores the name indicating the shape of the speech balloon image in the memory. When the size of the speech balloon image has been changed, the data creation control unit 11 sets the minimum size of the four-corner frame surrounding the speech balloon in which the size change has been made, and the four corner coordinates as a range for displaying speech balloon, .

Subsequently, the data creation control unit 11 reads the caption character of the caption start time from the caption list data, and inserts the caption character into the determined speech balloon. At this time, the data creation control unit 11 instructs the display output unit 13 to display the caption characters at the beginning of each frame, from the caption start time until the caption continuation time elapses. At this time, the data creation control section 11 determines the subtitle character development speed. The subtitle speed is defined by stepping through the display of the subtitle of a character in a frame. For example, a normal speed is defined as displaying six new characters per frame. The data creation control unit 11 also temporarily stores the subtitle character development speed in the memory.

Subsequently, the data creation control unit 11 corrects the caption character according to an instruction from the user. Here, as the caption character attribute, the type of the character, the color of the character, the background of the caption, the transmittance of the caption, the type of the caption frame, and the emphasis of the caption character can be modified. The data creation control section 11 temporarily stores these subtitle character attributes in the memory. In addition, the content creating apparatus 1 may include a volume determining unit for determining the volume of audio during reproduction of the video content data. At this time, the content creating apparatus 1 can change the attribute of the caption character according to the volume determined by the volume determination unit. For example, when the volume is large, the content creating apparatus 1 may enlarge or discolor the caption characters.

Subsequently, the data creation control unit 11 reads the information temporarily stored in the memory, and reads the information of the subtitle duration time, the subtitle character development speed, the subtitle character attribute, the speech balloon range (four corner coordinates of the quadrangular frame) The coordinates of the speech balloon start point, the speech balloon shape, and the caption character are stored in the storage unit 15.

Subsequently, the data creation control unit 11 determines whether or not the content has been completely terminated. The data creation control unit 11 creates the final speech balloon data corresponding to the desired content data by integrating the speech balloon data for each caption start time created so far into one and stores it in the storage unit 15. [

According to the apparatus for creating a picture content with a speech balloon caption according to the present invention, the relationship between a speaking person and a caption can be easily recognized and a whole screen can be easily viewed.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, And additions should be considered as falling within the scope of the claims of the present invention.

1: Content creation device
2.2a: Content transmitting device
2b: Package mediaization device
3: Broadcasting device
3a: Internet
3b: Distribution system
4a: Content playback device
4b: Package media playback device
5: Content display device
11: Data creation control section
12:
13:
14,44: Time count section
15:
21: multiplexing control section
22,
23: Error correcting code adding unit
24: Digital Modulation section
25:
41:
43:
45: speech ball shape memory unit
46:
47: Demodulator
48:
51: Display output device section
52:

Claims (1)

A content creating apparatus for creating data necessary for providing image content with caption by a balloon, comprising: a speech balloon time extracting means for extracting a time to display a speech balloon in an image based on image content data to be original data; A bubble area determining means for determining a bubble area suitable for display of a speech balloon in an image of the time extracted by the speech balloon time extracting means; a speech balloon image determining means for determining a speech balloon image to be synthesized in the speech balloon region determined by the speech balloon region determining means; A caption character determination means for determining a caption character to be synthesized with the speech balloon image determined by the speech balloon image determination means, information on a time to display the speech balloon, information on the speech balloon region, information on the speech balloon image And information on the caption character The speech balloon data created by the speech balloon data creation means is reproduced together with the image content data, so that the speech balloon data generated by the speech balloon data creation means is reproduced together with the image content data, The content providing apparatus comprising:

Images