JP2013247611A - Closed caption broadcast output device, method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To securely avoid overlapping of a closed caption with an important area in an image.SOLUTION: A closed caption broadcast output device comprises: image storage means for storing inputted image data; closed caption storage means for storing closed caption data for the image data; image analysis means for reading the image data from the image storage means to calculate a predetermined important area and a moving area of the area from the image data within a period when a closed caption having the same content is displayed, to calculate a display position at which the closed caption can be displayed in an area other than these areas; closed caption data regeneration means for using the display position of the closed caption to regenerate the closed caption data; and image/closed caption combination means for outputting image data formed by combining a closed caption based on the regenerated closed caption data with the image data.

Description

  The present invention relates to a technique for outputting a caption broadcast.

  There are two types of TV subtitle broadcasting: open captions and closed captions. The open caption is character information such as a telop transmitted in a state included in the video signal in advance, and the display side cannot be selected or controlled on the receiver side. On the other hand, closed captions are character information that is transmitted synchronously separately from the video signal, and can be controlled by a digital television as a receiver. Can be hidden.

  The closed caption display method is determined based on character display information (character code, font size, character color, display position, etc.) in the caption signal (closed caption data) included in the broadcast wave. In particular, the display position is designated not to overlap the open caption when the video includes an open caption. For example, in the case of a video in which an open caption is included in the lower part of the screen, a position coordinate that displays a closed caption in the center of the screen is designated (see Non-Patent Document 1).

  In addition, when captions are generated and transmitted in real time during live broadcasting, these two captions may overlap due to an inappropriate designation of position coordinates. In such a case, at the timing when the subtitle signal included in the broadcast wave is received on the receiver side, OCR processing is performed on the image information at that time, and the display position of the closed caption is automatically changed, thereby overlapping the overlap. Can be avoided (see Patent Document 1)

JP 2004-208014 A

Takao Kadoma, 3 others, “Development of Automatic Formatting Technology for Offline Subtitle Production Practical System”, 2003 Video Media Society Winter Conference

  However, by avoiding overlapping of these two types of captions, closed captions overlap important areas in the video (such as the face of the subject), which makes viewers feel more annoying. was there.

  For such a problem, it is conceivable to consider an important area when displaying a closed caption. However, closed captions with the same content are displayed continuously for a certain period of time (for example, the content spoken in the video or the sound effect that was uttered while the speaker is speaking or the sound effect continues) Therefore, it is possible to avoid the superimposition on the important area in the first frame of the video data displayed within that time, but as shown in FIG. When the important region R moves (Ra → Rb → Rc), the closed caption CC may overlap the important region R.

  The present invention has been made in view of the above, and it is an object of the present invention to reliably avoid the overlap between the closed caption and the important area in the video.

  The caption broadcast output device according to claim 1, wherein video storage means for storing input video data, closed caption storage means for storing closed caption data for the video data, and video data is read from the video storage means. Video analysis means for calculating a predetermined important area and its movement area from video data within a time period when closed captions having the same content are displayed, and calculating a display position where the closed caption can be displayed in other areas; A closed caption data regenerating unit that regenerates the closed caption data using the display position of the closed caption, and a video that outputs video data obtained by synthesizing the closed caption based on the regenerated closed caption data with the video data ·closed And having a Yapushon synthesizing means.

  The caption broadcast output device according to claim 2 is the caption broadcast output device according to claim 1, wherein the video analysis means includes target frame calculation means for determining a plurality of processing target frames from the video data within the time, and An important area calculating means for calculating each of the important areas from a plurality of processing target frames; a closed caption non-display area calculating means for adding all the calculated important areas as a closed caption non-display area; and the closed caption non-display. Closed caption new position information calculating means for determining a display position where the closed caption can be displayed in an area other than the display area.

  The subtitle broadcast output method according to claim 3 is a video storage step of storing input video data in video storage means by a computer, and closed caption storage for storing the closed caption data for the video data in closed caption storage means. Reading out video data from the video storage means, calculating a predetermined important area and its moving area from video data within a time when the closed caption of the same content is displayed, and applying the closed caption to other areas A video analysis step for calculating a display position that can be displayed, a closed caption data regeneration step for regenerating the closed caption data using the display position of the closed caption, and a caption based on the closed caption data after the regeneration. And having video-closed caption synthesizing step of outputting the video data to over Zudo caption synthesized to the video data.

  The subtitle broadcast output method according to claim 4 is the subtitle broadcast output method according to claim 3, wherein the video analysis step includes a target frame calculation step of determining a plurality of processing target frames from the video data within the time, and An important area calculating step for calculating the important areas from a plurality of processing target frames; a closed caption non-display area calculating step for adding all the calculated important areas to form a closed caption non-display area; and A closed caption new position information calculating step for determining a display position where the closed caption can be displayed in an area other than the display area.

  A subtitle broadcast output program according to claim 5 causes a computer to execute the subtitle broadcast output method according to claim 3 or 4.

  As described above, according to the present invention, a predetermined important area and its moving area are calculated from video data within the time when the closed caption of the same content is displayed, and the closed caption is output to other areas. And an important area in the video can be surely avoided.

  According to the present invention, it is possible to reliably avoid the overlap between the closed caption and the important area in the video.

It is a figure which shows the functional block structure of a caption broadcast output apparatus. It is a figure which shows the functional block structure of a video analysis part. It is a reference figure at the time of explaining operation | movement of an image | video analysis part. It is a subtitle broadcast output example (one closed caption). It is an example of subtitle broadcast output (multiple closed captions). It is a conventional subtitle broadcast output example.

  The present invention uses the video data and the closed caption data once stored in the storage means, calculates a region where the importance of the video flowing within the closed caption display time and its moving region are calculated, and avoids the region by using the closed caption. Is the main feature.

  Hereinafter, an embodiment for carrying out the present invention will be described with reference to the drawings. However, the present invention can be implemented in many different modes and should not be construed as being limited to the description of the present embodiment.

  FIG. 1 is a diagram illustrating a functional block configuration of a caption broadcast output apparatus 100 according to the present embodiment. The caption broadcast output apparatus 100 includes a video storage unit 11 that stores input video data, a closed caption storage unit 12 that stores closed caption data for the video data, and a time during which closed captions having the same content are displayed. A video analysis unit 13 that calculates a predetermined important area and its movement area from the video data in the image and calculates a display position where the closed caption can be displayed in the other screen areas, and the calculated display position of the closed caption are used. A closed caption data regeneration unit 14 that regenerates closed caption data, a video / closed caption synthesis unit 15 that outputs video data obtained by synthesizing closed caption data based on the closed caption data after regeneration to video data, Mainly with

  Note that the caption broadcast output apparatus 100 can be implemented on a digital television, for example, and can be realized by a computer including a storage unit such as a memory, a calculation / control unit such as a CPU, and a screen display unit such as a monitor. Moreover, the process of each said function part 11-15 which comprises this apparatus 100 is performed by a program.

  Next, the operation of the caption broadcast output device 100 will be described.

  First, the video data input to the subtitle broadcast output device 100 is stored by the video storage unit 11, and the closed caption data for the video data input together with the input of the video data is stored by the closed caption storage unit 12. (Step S101).

  Next, the video analysis unit 13 reads the video data and the closed caption data from the video storage unit 11 and the closed caption storage unit 12 respectively, and displays the same content of closed captions every time (hereinafter referred to as CC section). An important area is calculated from the video data in each CC section, and the display position of the closed caption in each CC section is determined based on the calculation result (step S102).

  Here, the processing of this step will be described in detail with reference to FIG. 2 and FIG. FIG. 2 is a diagram illustrating a functional block configuration of the video analysis unit 13. FIG. 3 is a reference diagram for explaining the operation of the video analysis unit 13.

  First, the frame corresponding to each CC section is determined from the video data based on the CC section information output from the closed caption storage unit 12 by the target frame calculation unit 13a, and a plurality of frames to be processed from among the frames. Is determined (step S102-1). The processing target frame may be all frames in the CC section, or may be selected for each frame every 1 second or 0.5 seconds, for example.

  Next, the important area calculation unit 13b calculates areas with high importance of the video from a plurality of frames to be processed (step S102-2). Although this important area is arbitrary, for example, a face area of a subject or an open caption display area is taken as an example.

  As a method for calculating the face area of the subject, for example, a feature amount corresponding to the orientation of the face is extracted, the similarity is calculated with the region in the input image using the feature amount, and based on the calculation result There is a method for recognizing a face area in an input image (see JP 2009-157766 A).

  Further, as a method of calculating the display area of the open caption, for example, a method of calculating character-like characteristics from individual character display scenes in video data and recognizing characters using a parameter set corresponding to the character-like characteristics (See JP 2007-164253 A). Normally, the display position of the open caption does not change during video display. For example, the display area may be within a certain width inside the screen frame.

  Note that these important area calculation methods are examples, and any method can be used as long as a specific image pattern displayed in the video can be detected.

  Next, the closed caption non-display area calculation unit 13c combines all the important areas calculated for each processing target frame to form a closed caption non-display area (step S102-3).

  Next, the closed caption new position information calculation unit 13d determines the position coordinates where the closed caption can be displayed from the image area other than the closed caption non-display area, and the closed caption data together with the CC section information as the closed caption new position information. It is transmitted to the regeneration unit 14 (step S102-4).

  The method for determining the new position coordinate is not particularly limited as long as it can be displayed so as to avoid the closed caption non-display area. For example, as shown in FIGS. 4 and 5 to be described later, a method of determining above or below a predetermined position is conceivable.

  Next, returning to FIG. 1, the closed caption data regenerating unit 14 uses the closed caption display position output from the video analysis unit 13 and the closed caption data output from the closed caption storage unit 12. Caption data is regenerated (step S103).

  Specifically, for example, since the closed caption data includes character display information (character code, font size, character color, display position, etc.), the display position in the character display information is received from the video analysis unit 13. Updated to display position.

  Finally, the closed caption displayed based on the regenerated closed caption data is synthesized by the video / closed caption synthesizing unit 15 with the video data output from the video storage unit 11, and the synthesized new video data. Is output to the screen (step S104).

  The processing of the video analysis unit 13 described above may be started after the video data and the closed caption data are completely stored from the beginning to the end in the caption broadcast output device 100, or a part of each data is stored. May be started sequentially at a predetermined timing.

  Particularly in the latter case, first, a storage area capable of recording video data for T seconds is provided in the video storage unit 11. If the start point of the i-th CC section is Si seconds and the end point is Ei seconds, if the value of T is set so that Ei−Si ≦ T, the video data from Si to Ei at time t = Ei Is recorded 11 in the video storage unit.

  Then, when the time t becomes Ei based on the i-th CC section information received from the closed caption storage unit 12, the video analysis unit 13 starts the above-described processing for the frames in Si to Ei. .

  When this latter method is applied to, for example, digital television broadcasting, the closed caption display time is normally about 10 seconds at the longest, and therefore, for example, T = 15 seconds can be determined. In this case, the user can wait for the end of the target program without delaying the target program, and a region where the importance of the video is high, such as a telop or a face part of the subject included in the video with a delay of about 15 seconds from the actual broadcast time The target program video can be viewed with closed caption display that avoids the above.

  According to the processing operation described above, as shown in FIGS. 4 and 5, even if the important region R moves from Ra → Rb → Rc in the CC section, the closed caption CC is superimposed on the important region R. Can be avoided.

  That is, according to the present embodiment, a predetermined important area and its moving area are calculated from video data in a CC section in which closed captions having the same content are displayed, and closed captions are output to the screen in other areas. Therefore, it is possible to reliably avoid the overlap between the closed caption and an important area (an area having movement such as a human face) in the video.

DESCRIPTION OF SYMBOLS 100 ... Subtitle broadcast output device 11 ... Video storage part 12 ... Closed caption storage part 13 ... Video analysis part 13a ... Target frame calculation part 13b ... Important area calculation part 13c ... Closed caption non-display area calculation part 13d ... Closed caption new position information Calculation unit 14 ... Closed caption data regeneration unit 15 ... Video / closed caption composition unit S101 to S104, S102-1 to S102-4 ... Step

Claims (5)

Video storage means for storing input video data;
Closed caption storage means for storing the closed caption data for the video data;
The video data is read from the video storage means, a predetermined important area and its moving area are calculated from the video data within the time when the closed caption of the same content is displayed, and the closed caption can be displayed in other areas Video analysis means for calculating the display position;
Closed caption data regenerating means for regenerating the closed caption data using the display position of the closed caption;
Video / closed caption synthesis means for outputting video data obtained by synthesizing the closed caption based on the closed caption data after regeneration with the video data;
A subtitle broadcast output device comprising: The video analysis means includes
Target frame calculation means for determining a plurality of processing target frames from the video data within the time;
Important area calculation means for calculating the important areas from the plurality of processing target frames, respectively;
A closed caption non-display area calculating means for adding all the calculated important areas and making it a closed caption non-display area;
Closed caption new position information calculating means for determining a display position where the closed caption can be displayed in an area other than the closed caption non-display area;
The subtitle broadcast output device according to claim 1, further comprising: By computer
A video storage step for storing the input video data in the video storage means;
A closed caption storage step of storing the closed caption data for the video data in a closed caption storage means;
The video data is read from the video storage means, a predetermined important area and its moving area are calculated from the video data within the time when the closed caption of the same content is displayed, and the closed caption can be displayed in other areas A video analysis step for calculating the display position;
A closed caption data regeneration step for regenerating the closed caption data using a display position of the closed caption;
A video / closed caption synthesis step for outputting video data obtained by synthesizing the closed caption based on the closed caption data after regeneration with the video data;
A subtitle broadcast output method characterized by comprising: The video analysis step includes
A target frame calculation step of determining a plurality of processing target frames from the video data within the time; and
An important area calculation step of calculating the important areas from the plurality of processing target frames, respectively;
A closed caption non-display area calculating step for adding all the calculated important areas and setting it as a closed caption non-display area;
A closed caption new position information calculation step for determining a display position where the closed caption can be displayed in an area other than the closed caption non-display area;
The subtitle broadcast output method according to claim 3, further comprising:   5. A closed caption broadcast output program for causing a computer to execute the closed caption broadcast output method according to claim 3.

Images