KR101462249B1 - Apparatus and method for detecting output error of audiovisual information of video contents - Google Patents

Abstract

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus and a method for detecting an error in which audiovisual information of video contents is abnormally output, and more particularly, to an apparatus and a method for detecting an error in subtitles and audio output from video contents.
An apparatus for detecting a caption display error of video content includes a voice extracting unit for checking start and end times of a voice signal using frequency information from an audio signal of the video content, Detects a change in the character string of the N-th video frame and the subtitle area from the (N + 1) -th video frame, detects a character string in the subtitle area, A subtitle detection unit for setting a termination flag in a video frame not detected for the first time; and a subtitle detection unit for comparing the start time and the start flag, comparing the end time and the end flag, And a subtitle synchronization determination unit .

Description

[0001] APPARATUS AND METHOD FOR DETECTING OUTPUT ERROR OF AUDIOVISUAL INFORMATION OF VIDEO CONTENTS [0002]

The present invention relates to an apparatus and a method for detecting an error in which audiovisual information of a video content is abnormally output, and more particularly, to an apparatus and a method for detecting an inconsistency in timing And more particularly,

Information delivery media is becoming more diverse and more abundant in the rapidly changing modern society. In the past, information delivery media used text - based presentation methods, but nowadays, video media that simultaneously stimulate the user 's sight and hearing are used as the most popular information delivery means.

Especially, as TV, computer, internet, and other technologies become popular, it becomes very easy to access information provided in video form. In recent years, with the development of the encoding technology of video contents, it is possible to easily access high-quality images of several tens to several hundreds of megabytes of video contents files.

In addition, video content shooting equipment capable of producing such moving images has been developed, and cell phone cameras, digital cameras, and camcorders are used by many, even if not experts. As a result, various video contents such as UCC and movie are constantly produced, and this trend is expected to accelerate further. In addition, data storage media such as a hard disk drive, a memory card, and the like are becoming cheaper and becoming cheaper.

In addition, media playback terminals capable of viewing video content have also become diverse, including mobile phones, smart phones, PMPs, mobile DMB terminals, notebooks, and car navigation systems. In the past, information transmission of video contents was only terrestrial TV broadcasting, but now there are various methods such as IPTV, satellite broadcasting, and digital cable broadcasting.

On the other hand, it is necessary to investigate the defects of the video contents themselves before the produced video contents are served through a medium such as IPTV. This is because, when there is a caption or lip sync error in the video content, the complaint of the viewer is directed to the subject of the content service, not the content creator.

FIG. 1 is a view for explaining a method of detecting an audiovisual output information error of conventional moving picture contents.

As shown in FIG. 1, in general, the video contents are inspected depending on human eyes in order to detect errors of subtitle information and audio information, which are audio-visual output information. However, it is practically impossible for a company providing video contents to detect all errors of video contents circulating in the hundreds of times a day.

In addition, even if the video content is quickly reproduced and the error is investigated, the conventional art requires a great deal of time and manpower and is costly. In addition, since the method is a method of manually detecting an error, there is a problem that the error is largely deviated according to the ability of the person detecting the error and the error is not completely eliminated even after the visual inspection is finished.

In order to solve the problems of the related art, some embodiments of the present invention provide an apparatus and method for detecting the presence or absence of a caption in an area where a caption of video content is located and detecting a caption display error by extracting a voice.

In addition, some embodiments of the present invention provide an apparatus and method for analyzing a mouth shape of a face of a video content using a facial recognition technique and extracting a voice to detect a voice output error.

The objects of the present invention are not limited to the above-mentioned objects, and other objects not mentioned can be clearly understood from the following description.

As a technical means for achieving the above technical object, a first aspect of the present invention is a speech recognition apparatus comprising: a speech extracting unit for checking start and end times of a speech signal using frequency information from an audio signal of the video content; Detecting whether a character string exists in a predetermined caption area, setting a start flag in an Nth video frame in which the character string is detected for the first time, determining whether a character string of the Nth video frame and the caption area is changed A subtitle detection unit for detecting a character string in the subtitle area and setting an end flag in a video frame in which a character string is not detected for the first time in the caption area, and a subtitle detection unit for comparing the start time and the start flag, A person who judges whether the subtitle error setting time is exceeded And a caption display error detection device including a membrane synchronization determination unit.

According to a second aspect of the present invention, there is provided a video signal processing apparatus including a voice extracting unit for checking a start time and an end time of a voice signal using frequency information in an audio signal of the video content, a mouth shape of a talker appearing in the video content, And the start time of the first video frame checked by the video recognition unit is compared with the start time of the audio signal extracted by the audio extraction unit And a voice synchronization determination unit for determining whether the first voice error setting time exceeds a predetermined first voice error setting time.

In order to attain the above object, a third aspect of the present invention is a method for generating a video signal, comprising the steps of: (a) extracting a voice signal from an audio signal of the video content using frequency information; (b) (C) determining whether a character string exists in a predetermined caption area of an Nth video frame, (d) if a character string exists in the caption area, And setting an end flag in an N + Mth video frame in which a character string disappears in the caption area, and (e) comparing a start time and an end time of the audio signal with a start flag and an end flag, And a step of checking the error.

According to a fourth aspect of the present invention, there is provided a method for extracting a speech signal, comprising the steps of: (a) extracting a speech signal using frequency information from an audio signal of the video content; (b) And checking the start time of the first video frame in which the mouth shape occurs, and (c) comparing the start time of the audio signal and the start time of the first video frame checked to check a voice sync error And a voice sync error detection method.

BRIEF DESCRIPTION OF THE DRAWINGS The above and other objects, features and advantages of the present invention will be more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which: FIG.

The present invention may, however, be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. It is provided to fully inform the owner of the scope of the invention.

According to one embodiment of the present invention, the subtitle display error and the voice sync error of the video contents can be detected.

According to an embodiment of the present invention, a subtitle display error and a voice sync error of a video content do not require a separate error checking task, and the cost can be reduced by reducing the time required.

In addition, according to the present invention, it is possible to more accurately detect the subtitle display error and the voice sync error of the video content, and thus it can be utilized for the integrity verification and the quality verification of the contents.

In addition, according to the present invention, the video content can be widely used in a video content encoding process, a transcoding process, and a format conversion process performed by a video content service provider. Therefore, have.

FIG. 1 is a view for explaining a method of detecting an audiovisual output information error of conventional moving picture contents.
FIG. 2 (a) is a block diagram for explaining a caption display error detection apparatus according to an embodiment of the present invention, and FIG. 2 (b) is a block diagram for explaining a subtitle display error detection apparatus according to another embodiment of the present invention. Block diagram.
3 is a block diagram for explaining an audiovisual output information error detection apparatus according to another embodiment of the present invention.
4A and 4B are diagrams for explaining a region in which a caption of video content is located according to an embodiment of the present invention.
5 is a diagram for explaining an example in which a video frame itself has a caption according to an embodiment of the present invention.
6 is a diagram for explaining an example in which a subtitle is added to a video frame according to an embodiment of the present invention.
7 is a diagram for explaining a method of detecting an error of a caption according to an embodiment of the present invention.
8 is a view for explaining a mouth shape recognition method using a face recognition technique according to another embodiment of the present invention.
9 is a diagram for explaining a method of detecting errors of video and audio of video contents according to another embodiment of the present invention.
10 and 11 are flowcharts for explaining a subtitle display error detection method according to an embodiment of the present invention.
12 is a flowchart illustrating a method of detecting a voice sync error according to another embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, which will be readily apparent to those skilled in the art. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and similar parts are denoted by like reference characters throughout the specification.

Throughout the specification, when a part is referred to as being "connected" to another part, it includes not only "directly connected" but also "electrically connected" with another part in between . Also, when an element is referred to as "comprising ", it means that it can include other elements as well, without departing from the other elements unless specifically stated otherwise.

Also, when a part is referred to as "including " an element, it does not exclude other elements unless specifically stated otherwise.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings.

FIG. 2 (a) is a block diagram for explaining a caption display error detection apparatus according to an embodiment of the present invention, and FIG. 2 (b) is a block diagram for explaining a subtitle display error detection apparatus according to another embodiment of the present invention. Block diagram.

2, the caption display error detection apparatus 200 according to an exemplary embodiment of the present invention includes a video content input unit 210, a voice extraction unit 220, a caption detection unit 230, (240).

In the present invention, "subtitle display error" means that a subtitle corresponding to a specific voice is not displayed on the screen at an appropriate moment. In other words, this may be the case where the subtitles in the national language or foreign language of some dialogue that the talker spoken during the output of the video content video unintentionally fall into this category.

For example, if the subtitles are not displayed at the time when the mouth of the talker in the moving picture is displayed, if the subtitles are not displayed at the time of outputting the conversant's voice in the moving picture, If the viewer disappears quickly enough to be unreadable, the viewer of the general reading level includes the case where the subtitle does not disappear even though it is determined that the subtitle has been read. In addition, the video content includes a case in which the caption is not accelerated or slowed down but the caption is not input itself, that is, the caption is not even played. In this case, the caption is not corresponded to the caption display error, You may.

The content input unit 210 receives video content data from a device connected to the caption display error detection device 200 through a wired or wireless connection. The input video content may be stored in a database (not shown) to implement the present invention.

Here, the video content refers to general moving image data in which various types of scenes are composed in a complex manner. In the video contents, a dialogue that speaks a language such as a foreign language user or a Korean user can appear. In addition, subtitles may be provided in the video content for a specific purpose, and for example, Korean subtitles may be provided for each scene so that the hearing impaired may use the content of the content, Hangul subtitles may be provided for easy understanding of the dialect, or appropriately summarized Hangul subtitles may be provided if the conversant speaks professional or difficult content.

The audio extracting unit 220 can extract the audio signal using the frequency information of the video content inputted by the content input unit 210 and check the start time and end time of each frame of the video content. The extracted voice data is necessary to confirm whether the subtitles appear at the appropriate timing or at a time difference in the portion where the voice data is outputted.

If the voice is output and there is no caption, it can be classified as no caption error and classified as no caption and managed separately. The subtitle display error detection process of the present invention can be applied after the subtitle insertion operation is separately managed.

Here, the audio may be audio data extracted from the video content data and may be human audio data. In addition, audio data such as animal documentary, animation, and SF movie can be extracted from the object of any object as long as it is a video content in which a subject capable of producing sounds such as animals, aliens, and robots can appear. Here, the voice data is extracted from a person.

In addition, the audio data is extracted by using a point different in frequency from the other data. For example, the speech frequency band from the vocal cords of a normal speaker is between about 300 Hz and about 3.5 kHz. Therefore, by extracting a similar signal in this frequency band, it can be used as audio data required in the present invention. The voice extraction method is not limited to a specific method, and various methods can be used. In addition, it is not necessary to analyze the contents of the extracted voice data, and only the presence or absence of the voice and the time of the voice may be checked.

The caption detection unit 230 detects whether a character string is present in a predetermined caption area from the video content data, sets a start flag in a first video frame in which a character string is detected first, And can set an end flag in a video frame in which no string exists in the caption area.

Here, the caption area may be the bottom, top, right, or left side of the screen where the caption is displayed, but the caption recognition rate can be increased by setting the area where the caption appears. In this embodiment, it is described that various video media such as general video contents such as a motion picture film, a TV, a Blu-ray, a DVD, a CD, a VHS tape and the like are located in a lower quarter of an image something to do.

4A and 4B are diagrams for explaining a region in which a caption of video content is located according to an embodiment of the present invention.

FIG. 4A is an example of a standard-definition (SD) screen 410 used in a video graphic display device such as a digital standard television. Since the SD level picture has a resolution of 720 x 480, a region of about 120 pixels from the lower end of the picture, which is a quarter of this size vertically, can be set as the first caption area 415.

4B is an example of an HD (High-Definition) class screen 420 used in a video graphic display device such as a high-definition television. Since the HD-class screen has a resolution of 1920 x 1080, an area of about 270 pixels from the lower end of the screen, which is one-quarter of this size, can be set as the second caption area 425.

5 is a diagram for explaining an example in which a video frame itself has a caption according to an embodiment of the present invention.

Figure 5 shows the video frame itself "Arriving in Uljin, Gyeongbuk" (510) is inserted. Unlike the case where a caption is inserted into a caption area of an image as described above, the video frame itself is an example of a caption of another image. Accordingly, in such a case, the caption detector 230 may determine that there is no caption when checking whether there is a caption in the caption area, and may not detect the caption. In addition, the text-like figure 510 represents the voice of the talker in the caption area, and is different from the caption inserted into the large-size text as the size of the picture 510. [ Another example of such a picture may be a phrase that is easy to understand and interesting and summarizes a person's voice content or situation.

In order to determine whether a caption is present in each of the caption areas of FIGS. 4A and 4B, the caption detector 230 determines whether a caption is a character in the video content itself or a caption Can be distinguished. There are a variety of methods for distinguishing subtitles embedded in an image (hereinafter referred to as an inserted subtitle) or pictures of a glyph (hereinafter referred to as a subtitle).

First, it is possible to judge whether a caption or a caption of a picture is a caption or not by viewing the position where the text on the screen appears. For example, it is possible to check whether a caption exists only in the lower part of the screen, and judge it as a caption if it is in the middle or upper part of the screen, not the bottom part. In addition, even if the subtitles are present at the bottom, it is possible to judge more accurately by further checking whether the text is arranged in an aligned state or irregularly randomly arranged.

In addition, it is possible to judge whether it is an inserted caption or a picture caption by the size of the text displayed on the screen. For example, in the case of the SD class, it can be viewed as an insertion subtitle if it is within 25 x 25 pixels per character and within 50 x 50 pixels for HD class. However, if the text on the screen is too small or the size is not constant, it can be regarded as not an insertion subtitle.

In addition, it is possible to determine whether the subtitle is an embedded subtitle or a subtitle subtitle depending on whether the displayed text has a temporal similarity or a correlation. That is, the characters displayed in the caption area on the screen can not be moved for at least one second (30 frames) in time and can be judged as the inserted caption when they are located at the same position.

In addition, it is possible to determine whether the text on the screen is a horizontal insertion subtitle or a picture subtitle which is inclined in the horizontal direction. Generally, since the inserted subtitles appear repeatedly in the pre-designated subtitle area, they are arranged in a horizontal direction and at a right angle so that viewers can easily view them.

The above-mentioned various criteria may be changed according to the characteristics of the video contents, and may be changed according to the case since the video contents can not be uniformly applied to all the video contents.

Also, in the present invention, various techniques and techniques for detecting the caption area can be used, such as edge map usage, stroke filtering, and color and luminance value, and are not limited to a specific caption area detection technique.

6 is a diagram for explaining an example in which a subtitle is added to a video frame according to an embodiment of the present invention.

As shown in FIG. 6, the normal insertion subtitle is displayed at the position of the caption area 425 at the lower end of the screen. And the size of the inserted subtitles is usually 25 to 50 pixels in width and length, respectively.

The caption synchronization determination unit 240 compares the start time and the end time of the voice signal extracted by the voice extraction unit 220 with the start flag and end flag of the string of the caption area detected by the caption detection unit 230, It is possible to judge whether the first subtitle error setting time is exceeded.

First, in order to detect the caption error of the video content, character detection is performed in a predetermined portion of the lower end of the video, for example, a quarter of the vertical size, and compared with the voice extracted from the audio data, May be used. Fig. 7 is a diagram specifically showing such a method.

7 is a diagram for explaining a method of detecting an error of a caption according to an embodiment of the present invention.

7 shows a method of comparing the voice signal extracted by the voice extraction unit 220 with the start and end of a caption detected by the caption detection unit 230. FIG. A start flag may be set in the first frame in which the caption area is detected, and a caption area may be detected in the next frame using the difference from the previous image. And an end flag is set at the end of the caption area so that an algorithm for comparing start and end flags of the audio signal extracted from the audio extracting unit 220 is used.

The caption displayed between the caption start flag 710 and the caption end flag 720 is the same as the time at which the voice is output, and thus shows an example of the section A, that is, the section in which the caption is normally displayed. That is, it is a section indicating that the beginning and end of the voice are almost coincident with the beginning of the subtitles normally. In order to distinguish the normal subtitles from the error subtitles, the error limit of the time and the end time of the audio file can be set in seconds. For example, a difference of about 0.5 seconds (15 frames) can be set.

Here, the time at which the start and end points of the audio signal can be detected can be known by using the PTS (Presentation Time Stamp) of the audio file. The method of calculating the start and end times of a speech signal that can be applied in the present invention is not limited to a specific method, and various methods can be used.

B section and C section represent an example of a subtitle error section in which an error subtitle appears.

In section B, a voice signal is outputted but no caption is displayed. Accordingly, the caption synchronization determination unit 240 may determine that a caption error is detected if the time during which the caption is not displayed continues to exceed a predetermined threshold value despite the presence of a voice signal. At this time, the threshold value of the time without the caption display, that is, the threshold time can be set to, for example, about 2 seconds, which can be set to a different time depending on the type of the video content. That is, an algorithm may be used to determine that a subtitle display error is present when there is a predetermined time difference (for example, 2 seconds) between the start of the extracted audio signal and the start flag of the subtitle area, but there is no audio signal but no subtitle display.

C section is an example of a case where there is no voice signal and the subtitles are displayed as opposed to the section B. The subtitle start flag 730 and the subtitle end flag 740 indicate that the audio signal is not output even if the subtitles are displayed. This case can also be treated as a caption error. Such a caption error occurs with a probability of less than about 1% in one video content, and corresponds to one or two measures of the caption in which the error occurs in terms of the caption amount. Therefore, it may be efficient to use an algorithm in which the proportion of subtitles in the B section having a higher occurrence frequency is larger. This is because it is possible to detect more subtitle errors than actual subtitle errors. Accordingly, in the present invention, an algorithm for detecting a caption error in a section B may be included, and a caption error in a section C may be determined according to a situation.

Next, a speech sync error detection apparatus 250 according to an embodiment of the present invention will be described with reference to FIG. 2 (b). The constituent elements already described in Fig. 2 (a) are denoted by the same reference numerals, and redundant explanations are omitted.

2B, the apparatus for detecting a sound sync error 250 according to another embodiment of the present invention includes a video content input unit 210, a sound extraction unit 220, an image recognition unit 235, And a determination unit 245.

In the present invention, a voice sync error means that a voice is not output at a proper moment, and a lip synchronization error corresponds to the voice sync error. For example, if a voice is not output at the same time as the mouth of a talker in the video is not output, if the voice is output before or after the mouth of the talker in the video is output, A case in which a voice is outputted even though the mouth of the talker in the video does not move, and the like.

If a sync error occurs, the video and audio are not synchronized in the video content, so if sync is lost in one part, it will be affected by the video and audio that appears later. That is, it continues from the moment of synchronization to the end of video content. Generally speaking, although it is not easy to detect a voice sync error, it is possible to detect an error if the timing at which the voice starts to be output (the moment of conversation start) can be grasped accurately.

The content input unit 210 may receive video content data from a device connected to the voice sync error detection device 250 through a wired or wireless connection and the voice extraction unit 220 may receive the frequency of the video content received by the content input unit 210 Information can be used to extract a voice signal, and start and end times of each frame of the video content can be checked.

The image recognizing unit 235 can recognize the mouth shape of the talker using the face recognition technology from the video content data and check the start time and the end time of the video frame in which the mouth shape first occurs. Therefore, in order to determine whether or not the audio is properly output from the video data, a process of analyzing the face of the talker of the video data is required. 8 will be described with reference to FIG.

8 is a view for explaining a mouth shape recognition method using a face recognition technique according to another embodiment of the present invention.

As shown in FIG. 8, the image recognizing unit 235 recognizes the face of the talker from the background or the body part in the entire image, and recognizes the shape of the mouth part of the face by recognizing it. The way to distinguish the face and the mouth from other images can be judged by comparing the graphic in the image and the human face template (for example, circle, eye, nose, mouth shape) and human normal skin color. Since such techniques are widely used in digital cameras, CCTV, facial recognition software, and the like, various methods can be used in the present invention without being limited to the specific facial recognition technology.

For example, the face of the talker 810 is recognized and the mouth of the talker 820 is recognized through the face recognition technology. The mouth of the talker is first closed in one shape, and the moment when the mouth is opened can be grasped as the moment when the horse begins. It is possible to compare the timings at which the speech signals extracted from the audio data corresponding thereto are considered as the dialog start timing from the frame in which the mouth is opened. A method of indicating how to compare the timings of each other is shown in Fig.

9 is a diagram for explaining a method of detecting an output error between video and audio of video contents according to another embodiment of the present invention.

The voice synchronization determination unit 245 compares the start time and end time of the voice signal extracted by the voice extraction unit 220 with the start time and end time of the video frame detected by the image recognition unit 230, 1 voice error setting time is exceeded.

As described above, the face and mouth are respectively recognized in the image, and the time at which the mouth starts to open when the mouth is in one character is detected, and the start time of the extracted voice signal is detected and the predetermined reference time (for example, 165 msec or 5 Frame), it is possible to make an algorithm that judges as a voice sync error.

9 shows the error between the start time of the extracted voice signal and the talk start time of the talker. If this error is ahead of or behind the predetermined reference time, i.e., the threshold time, it can be regarded as a voice sync error. Here, the threshold time can be, for example, plus or minus 165 msec (5 frames).

The interval D 910 is a case where the difference between the start time of the extracted voice and the dialog start time exceeds the threshold time mentioned above and is less than a predetermined time, for example, 2 seconds (60 frames). In this case, the voice synchronization determination unit 245 determines that it is a voice sync error. On the other hand, in the case of the interval E (920), the difference between the start time of the extracted voice and the conversation start time is 2 seconds (60 frames) or more. Therefore, in this case, the voice synchronization determination unit 245 can determine the extracted voice signal as a voice of another person other than the conversation partner, rather than the probability of the voice synchronization error.

In addition, since there is a probability that the voice synchronization determination unit 245 recognizes the conversation start time of the talker incorrectly, it may not be included in the voice sync error when the difference is more than 2 seconds. That is, if the first time when the mouth shape starts to move, that is, the start time of the conversation and the start time of the extracted voice signal differ by more than a predetermined reference time, for example, 2 seconds or more, It can also be implemented.

 In addition, the facial recognition technique described above has a limited recognition rate because the angle at which facial recognition is possible is limited, and especially when the conversation person starts conversation, the recognition rate is further reduced. However, a voice sync error is not a one-off error; once it occurs, it affects all voice signals until the playback of the video content ends, making all voice sync unsuitable. Therefore, even if the recognition rate of the moment when the conversation starts is low, the problem can be solved by recognizing the face and mouth shape in the face image which is easy to recognize at any time from the time of the voice sync error.

3 is a block diagram for explaining an audiovisual output information error detection apparatus 300 according to another embodiment of the present invention.

Combining the subtitle display error detecting apparatus 200 and the speech sync error detecting apparatus 250 described above makes the audiovisual output error detecting apparatus 300 as shown in FIG. The video content received by the content input unit 210 is divided into voice data, caption data, and video data and is analyzed by the voice extraction unit 220, the caption detection unit 230, and the image recognition unit 235, respectively. The subtitle synchronization determination unit 240 determines whether the subtitle display is displayed according to the audio using the data extracted by the audio extraction unit 220 and the subtitle detection unit 230. The audio synchronization determination unit 245 determines whether the subtitle display 220 and the image recognizing unit 235 to determine whether the voice sync appears according to the image. The subtitle synchronization determination unit 240 and the voice synchronization determination unit 245 may output the corrected video content capable of outputting the subtitle and the video at appropriate timing based on the audio signal commonly used. A detailed description of each component is the same as that described above, so that a duplicate description will be omitted.

10 and 11 are flowcharts for explaining a subtitle display error detection method according to an embodiment of the present invention.

First, in step S1010, the caption display error detection apparatus 200 receives video content and extracts a voice signal from the video content input in step S1020.

In operation S 1020, the subtitle display error detection apparatus 200 starts playback from the N-th frame of the video content. At this time, the subtitle display error detection apparatus 200 determines whether subtitles are detected in the subtitle region of the corresponding frame. If no caption is detected, the process moves to the (N + 1) th frame of the next frame (S1045). If a caption is detected, a flag indicating that the caption starts is set to 1 (S1050).

In step S1060, the frame in which the start of subtitles is set is compared with the frame of the sequence incremented by one in the frame. If the subtitle is not changed in the caption area, the frame is moved to the next frame, and the frame is compared with the frame of the sequence incremented by one from the frame. The subtitle can be detected by repeating this comparison method. Subtitles can be sustained for a few seconds because they are usually conversations, and 30 frames per second for normal video content. Once detected, the caption is usually kept about 30 to 150 frames.

The method of detecting the presence or absence of the caption compared with the above-described continuous frames can detect the difference between the n-th caption region and the (N + 1) -th caption region, and then detect using the various algorithms.

In step S1080, the subtitle display error detection apparatus 200 sets the flag to 1 to indicate that the subtitle is over after detecting that the subtitle has been terminated. The ending flag can be set to be the end of the N + Mth video frame in which the character string disappears in the caption area through the above-described repetition process. Here, the flags indicating the start of the subtitles and the end of the subtitles are necessary to compare the start and end times of the audio extracted from the audio.

In step S1090, the subtitle display error detection apparatus 200 determines whether the frame is the last frame. If it is the last frame, the video content playback ends in step S1100. Otherwise, the subtitle display error detection apparatus 200 moves to the next frame in step S1095. It is determined whether a caption is detected in the caption area of the caption area.

The audio signal extracted in step S1110 and the flagged video content are simultaneously reproduced. Then, in step S1120, it is checked whether a flag indicating that the caption exists is set within a predetermined time after the reproduction of the audio signal is started. If there is no setting, the synchronization error is checked at the corresponding position (S1135), and the presence or absence of the setting of the caption flag is checked within a predetermined time after the reproduction of the audio signal starts.

In step S1130, when the caption flag is set within a predetermined time after the reproduction of the signal is started, the caption display error detection apparatus 200 determines whether there is an audio start signal within a predetermined time after detecting the caption flag . If the audio start signal is within the corresponding time, the synchronization error is checked at the corresponding position (S1135). If the audio start signal is not within the new time, the video content is determined to be completely played back (S1140). If it can not be determined, the reproduction is continued and the process goes back to the step S1120, and if it can be determined, the reproduction is ended.

12 is a flowchart illustrating a method of detecting a voice sync error according to another embodiment of the present invention.

In step S1210, the voice sync error detection apparatus 250 receives the video content and extracts a voice signal from the video content (S1220). Then, the extracted audio signal and video content are simultaneously reproduced (S1230), and the dialogue start time is checked (S1240).

In step S1250, it is determined whether the difference between the dialog start time and the start time of the voice signal exceeds a predetermined first voice error setting time. If the first voice error setting time is exceeded, it is checked again whether the difference between the dialog start time and the start time of the voice signal is within a predetermined second voice error setting time (S1260).

In step S1270, if the difference between the conversation start time and the start time of the voice signal occurs within the second voice error setting time, the voice sync error detection device 250 checks the synchronization error at the corresponding position. Then, the reproduction of the video content is stopped (S1280).

In step S1290, the audio-sync error detection apparatus 250 determines whether to complete the video content playback when the difference between the dialog start time and the start time of the audio signal exceeds a predetermined first audio error setting time. If the reproduction of the video contents is not completed, the reproduction is continuously performed and the operation is resumed from the step S1240. If the reproduction is completed, the execution is terminated.

Although not shown in the drawings, the audiovisual output information error detection method that performs both the subtitle display error detection method and the audio sync error detection method is derived by combining the subtitle display error detection method and the audio sync error detection method The method will be obvious to those skilled in the art and a description thereof will be omitted.

Each of the embodiments of the present invention can also be embodied in the form of a recording medium including instructions executable by a computer such as a program module executed by a computer. Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. In addition, the computer-readable medium may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media typically includes any information delivery media, including computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other transport mechanism.

While the methods and apparatus of the present invention have been described with reference to particular embodiments, some or all of those elements or operations may be implemented using a computer system having a general purpose hardware architecture.

It will be understood by those skilled in the art that the foregoing description of the present invention is for illustrative purposes only and that those of ordinary skill in the art can readily understand that various changes and modifications may be made without departing from the spirit or essential characteristics of the present invention. will be. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive. For example, each component described as a single entity may be distributed and implemented, and components described as being distributed may also be implemented in a combined form.

The scope of the present invention is defined by the appended claims rather than the detailed description and all changes or modifications derived from the meaning and scope of the claims and their equivalents are to be construed as being included within the scope of the present invention do.

200: Caption display error detection device
210:
220:
230: caption detection unit 235:
240: caption synchronization determination unit 245: voice synchronization determination unit
250: Voice sync error detection device
300: Audio-visual output error detection device

Claims (18)

An apparatus for detecting a subtitle display error of video content,
A voice extracting unit for checking the start time and the end time of the voice signal using the frequency information from the audio signal of the video content;
The method comprising: detecting whether a character string is present in a predetermined caption area in the video content, setting a start flag in an Nth video frame in which the character string is detected for the first time, A caption detection unit detecting whether a character string is changed and setting an end flag in a video frame in which a character string is not detected first in the caption area; And
Compares the start time and the start flag, and compares the end time and the end flag to determine whether the start time and the end time exceed a predetermined first caption error set time,
Lt; / RTI >
Wherein the caption detection unit distinguishes the caption region by a temporal similarity correlation in which the character string is displayed.
The method according to claim 1,
Wherein the caption detection unit distinguishes the caption area by a position at which the character string is displayed.
The method according to claim 1,
Wherein the caption detection section (20) is configured to classify the caption area (20) according to the size of the character string.
delete The method according to claim 1,
Wherein the caption detecting unit distinguishes the caption area by an angle that is inclined with respect to the horizontal direction of the character string.
The method according to claim 1,
The caption synchronization determination unit determines that the caption error is a caption error when there is a voice signal but no string when the time difference between the start time of the audio signal and the start flag of the caption region exceeds the first caption error set time Caption display error detection device.
delete delete A method for detecting a subtitle display error of video content,
Extracting a speech signal from the audio signal of the video content using frequency information,
Extracting a start time and an end time of a voice signal extracted from the video content,
Confirming whether a character string exists in a predetermined caption area of an Nth video frame,
Setting a start flag in the Nth video frame when a character string exists in the caption area, setting an end flag in an N + Mth video frame in which the character string disappears in the caption area, and
Comparing the start and end times of the audio signal with the start flag and the end flag to check a subtitle display error
Lt; / RTI >
Wherein the caption area is divided by a temporal similarity in which the character string is displayed.
10. The method of claim 9,
The step of confirming the subtitle display error includes:
Checking a synchronization error at a position where the start flag does not exist within a predetermined first subtitle error set time after the start time of the audio signal
Wherein the subtitle display error detection method comprises:
10. The method of claim 9,
The step of confirming the subtitle display error includes:
If the start flag exists within a predetermined first caption error set time after the start time of the audio signal and there is a voice signal after the predetermined second caption error set time from the frame in which the end flag is set, Checking a synchronization error at a position corresponding to a predetermined second subtitle error set time from a set frame
Wherein the subtitle display error detection method comprises:
10. The method of claim 9,
Wherein the caption area is divided by a position at which the character string is displayed.
10. The method of claim 9,
Wherein the caption area is divided by the size of the character string.
delete 10. The method of claim 9,
Wherein the caption area is divided by an angle at which the character string is inclined with respect to a horizontal direction. delete delete delete

Images