KR20190132074A - Method and apparatus for monitoring quality of sign language broadcasting - Google Patents

Abstract

The present invention relates to a method for monitoring quality of a sign language broadcast, and an apparatus thereof. According to one embodiment of the present invention, the method for monitoring quality of a sign language broadcast comprises the following steps: identifying at least one captured image that captures broadcast content at every predetermined first time unit; generating a different image regarding the at least one captured image and detecting a sign language candidate region by accumulating the difference images regarding a unit of the broadcast content; determining a sign language region by analyzing the at least one captured image corresponding to the sign language candidate region; and determining quality of a sign language broadcast of the broadcast content based on at least one among broadcast programming information, signaling information, a unit object for monitoring, or a detailed unit object for monitoring associated with the captured image by using an image included in the sign language region of the broadcast content.

Description

Sign Language Broadcast Quality Monitoring Method and Apparatus {METHOD AND APPARATUS FOR MONITORING QUALITY OF SIGN LANGUAGE BROADCASTING}

The present disclosure relates to a sign language broadcast, and more particularly, to a method and apparatus for monitoring the quality of a sign language broadcast.

Recently, support for broadcast supplementary services for the disabled has increased. For example, broadcast supplementary services for the hearing impaired, such as subtitles, broadcast commentary information, or a sign language interpreter screen are provided for broadcast content. The Korea Communications Commission also set a goal to increase the ratio of captions, screen commentaries, and sign language interpretation.

In order to monitor whether or not such a sign language broadcast is provided, the non-disabled person stays at the level of manually examining some samples while watching the broadcast. That is, there is no way to automatically monitor the quality of sign language broadcasting.

An object of the present disclosure is to provide a method and apparatus for detecting a sign language area in a sign language broadcast.

Another technical problem of the present disclosure is to provide a method and apparatus for measuring a sign language time in a sign language broadcast.

Another technical problem of the present disclosure is to provide a method and apparatus for determining the quality of a sign language broadcast.

Technical problems to be achieved in the present disclosure are not limited to the above-mentioned technical problems, and other technical problems not mentioned above will be clearly understood by those skilled in the art from the following description. Could be.

According to an aspect of the present disclosure, a method of monitoring a sign language broadcast quality includes: identifying at least one captured image of capturing broadcast content every predetermined first time unit; Generating a difference image for the at least one captured image and accumulating the difference image for the broadcast content unit to detect a sign language candidate region; Determining a sign language region by analyzing the at least one captured image corresponding to the sign language candidate region; And a sign language broadcast of the broadcast content based on one or more of broadcast schedule information, signaling information, a monitoring unit object, or a monitoring subunit object related to the captured image, and using an image included in the sign language area of the broadcast content. Determining quality.

An apparatus for monitoring sign language broadcast quality according to another aspect of the present disclosure may include receiving at least one captured image of capturing the broadcast content every predetermined first time unit from at least one collection device providing the broadcast content to a user. Generating a difference image for the at least one captured image, accumulating the difference image for the broadcast content unit, detecting a sign language candidate region, and detecting the at least one sign language candidate region; Analyze a captured image to determine a sign language area, and based on one or more of broadcast programming information, signaling information, a monitoring unit object, or a monitoring sub-unit object related to the captured image, and included in the sign language area of the broadcast content. Using a video to determine the sign language broadcast quality of the broadcast content. May include a data analyzer.

The features briefly summarized above with respect to the present disclosure are merely exemplary aspects of the detailed description of the present disclosure described below, and do not limit the scope of the present disclosure.

According to the present disclosure, a method and apparatus for detecting a sign language area in a sign language broadcast may be provided.

According to the present disclosure, a method and apparatus for measuring a sign language time in a sign language broadcast may be provided.

According to the present disclosure, a method and apparatus for determining the quality of a sign language broadcast may be provided.

The effects obtainable in the present disclosure are not limited to the above-mentioned effects, and other effects not mentioned above may be clearly understood by those skilled in the art from the following description. will be.

1 and 2 are diagrams illustrating an exemplary configuration of a sign language broadcast quality monitoring apparatus according to the present disclosure.
3A to 3H are diagrams illustrating broadcast content including various sign language regions.
4A and 4B illustrate a sign language detection method according to an exemplary embodiment of the present disclosure.
5 and 6 are views for explaining examples of a sign language detection method according to the present disclosure.
7 is a flowchart illustrating the operation of the collecting device according to the present disclosure.
8 is a flowchart illustrating an operation of an analysis server according to the present disclosure.

Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the present disclosure. As those skilled in the art would realize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention.

In describing the embodiments of the present disclosure, when it is determined that a detailed description of a known structure or function may obscure the gist of the present disclosure, a detailed description thereof will be omitted. In the drawings, parts irrelevant to the description of the present disclosure are omitted, and like reference numerals designate like parts.

In the present disclosure, when a component is "connected", "coupled" or "connected" with another component, it is not only a direct connection, but also an indirect connection in which another component exists in the middle. It may also include. In addition, when a component "includes" or "having" another component, it means that it may further include another component, without excluding the other component unless otherwise stated. .

In the present disclosure, terms such as first and second are used only for the purpose of distinguishing one component from other components, and do not limit the order or importance between the components unless specifically mentioned. Accordingly, within the scope of the present disclosure, a first component in one embodiment may be referred to as a second component in another embodiment, and likewise, a second component in one embodiment may be referred to as a first component in another embodiment. It may also be called.

In the present disclosure, components that are distinguished from each other are for clearly describing each feature, and do not necessarily mean that the components are separated. That is, a plurality of components may be integrated into one hardware or software unit, or one component may be distributed and formed into a plurality of hardware or software units. Therefore, even if not mentioned otherwise, such integrated or distributed embodiments are included in the scope of the present disclosure.

In the present disclosure, components described in various embodiments are not necessarily required components, and some may be optional components. Therefore, an embodiment composed of a subset of components described in an embodiment is also included in the scope of the present disclosure. In addition, embodiments including other components in addition to the components described in the various embodiments are included in the scope of the present disclosure.

Definitions of terms used in the present disclosure are as follows.

Sign language broadcast: Broadcast content including an image including a sign language area or the transmission of such broadcast content.

Sign language image: An image including a sign language region.

-Sign language area: the area where the called party is displayed in the sign language image.

-Recipient: An object representing a sign language operation in a sign language image. The listener is usually a human but may be replaced by a graphic similar to a human.

-Sign language background: An area other than the part recognized as a sign language in a sign language area.

-Sign Language Time: The time at which sign language is provided in a sign language broadcast.

-Non-hydration time: The remaining time except the sign language time among sign broadcasts.

-Sign language broadcast quality: A value indicating the degree to which a valid sign language service is provided based on the presence of a sign language area, a sign language operation state, a sign language weight ratio, and a sign language provision frequency.

The definitions of the above terms are merely exemplary, and the scope of the present invention is not limited by these definitions. For example, the definition of the above terms does not exclude the basic meaning easily understood by those skilled in the art for the terms.

Hereinafter, various examples of the present disclosure for a method and apparatus for monitoring the quality of a sign language broadcast according to the present disclosure will be described.

The Korea Communications Commission is considering the provision of various additional services for the disabled by aiming to improve broadcasting services for the disabled. For example, a mandatory operator was selected to provide broadcasting for the hearing impaired, and the selected operator was required to achieve a predetermined target for programming ratios such as subtitles, screen commentary, and sign language interpretation. However, the Korea Communications Commission has reduced the selection criteria for mandatory broadcasters for the disabled and delayed the completion of the targets for the ratio of captions, screen descriptions, and sign language interpretation for two to three years. The main contents of this amendment are to reduce the target and scope of broadcast channel users and to delay the preparation of recommendations for 2-3 years in consideration of the poor real broadcast environment. In other words, terrestrial broadcasters such as KBS, MBS, and SBS had to achieve 100% of subtitles, 5% of sign language interpreters, and 10% of screen commentary by 2014, but this goal was delayed until 2016. In addition, paid broadcasting such as satellite broadcasting and Jongpyeon had to arrange 70 ~ 100% of subtitles, 5 ~ 7% of screen commentary, and 3 ~ 5% of sign language interpretation by 2016, but this time was also revised to 2018.

The Korea Communications Commission said, "We will postpone the time to achieve the target ratio for the disabled for a certain period in consideration of the financial situation of broadcasters who are suffering from poor broadcasting production infrastructure environment and difficulties in the broadcast advertising recession."

One of the reasons why the application of the mandatory broadcasting standard for persons with disabilities is delayed is because there is no plan for automatically evaluating the quality of sign language broadcasting. That is, until now, passively inspecting broadcasts by handicapped non-disabled persons, such as subtitles, screen commentary, and sign language interpretation, are conducted for all broadcasting times of all broadcasting channels. Since it is impossible to do so, it is still being investigated to some sampling criteria.

In order to solve this problem, the present disclosure provides a method for automating the monitoring of a sign language broadcasting quality, which is one of the handicapped broadcasting services, to provide a foundation for fully investigating the quality of a sign language broadcasting over all broadcasting times for all broadcasting channels. Can be prepared.

1 and 2 are diagrams illustrating an exemplary configuration of a sign language broadcast quality monitoring apparatus according to the present disclosure.

According to the current broadcast standard, the broadcast transmission signal itself does not include information indicating whether the broadcast includes a sign language broadcast. In addition, the signaling information of the broadcast signal may include information on the presence and quality of picture description audio and / or closed caption, but the broadcast signaling information for the sign language broadcast is not included on the transport stream (TS). Does not.

Therefore, broadcast programming information (eg, channel, program name, broadcast date, program start time and end time, etc.) and signaling information (eg, a complex transport stream (TS) belonging to a multiplexed program) provided by a broadcasting company are provided. It is necessary to determine the presence or absence of a sign language broadcast by analyzing a sign language broadcast image based on meta data expressing program information required in a table format) and checking whether a sign language region is included in the image.

Accordingly, the sign language broadcast quality monitoring apparatus according to the present disclosure may include a collection device 100 and an analysis server 200. Based on the one or more images captured by each of the collection apparatuses 100 in units of broadcast programs, the analysis server 200 performs a sign language broadcast quality (eg, whether a sign language area exists, a sign language operation state, or a sign language time) of the corresponding broadcast program. Specific gravity, frequency of hydration provision, etc.).

1 and 2, the sign language broadcast quality monitoring apparatus may include N collection apparatuses 100_1, 100_2,..., 100_N and an analysis server 200.

Referring to FIG. 1, in the N collecting apparatuses 100_1, 100_2,..., 100_N, N may be one or more natural numbers. In addition, each of the N collection devices 100_1, 100_2,..., 100_N may correspond to one monitoring unit object. That is, the N collection apparatuses 100_1, 100_2,..., 100_N may simultaneously collect information about N monitoring unit targets.

For example, the monitoring unit object may be defined by region, channel, time, program, and type. For example, N may correspond to the number of broadcast areas to be monitored, the number of broadcast channels, the number of broadcast times, the number of broadcast programs, and the number of broadcast types (eg, news, entertainment, culture, etc.).

Each collection device 100 may include M demux / tuner modules 110, storage modules 120, and collection management module 130.

In M demux / tuner modules 110, M may be one or more natural numbers. In addition, each of the M demux / tuner modules 110 may correspond to one monitoring subunit object. That is, the M demux / tuner modules 110_1, 110_2,..., 110_M may simultaneously collect information on M monitoring subunit targets.

If the monitoring unit target is determined as one of region, channel, time, program, and type, the monitoring detailed unit target may be determined as the other. Accordingly, monitoring of any combination of region, channel, time, program, or type is possible. For example, any combination may be defined to allow full inspection of all monitored subjects organized or organized by a sign language broadcast.

For example, when the monitoring unit object is a region, the monitoring detailed unit object may be defined as a channel, a time, a program, or a type.

For example, when the monitoring unit object is a channel, the monitoring detailed unit object may be defined as a region, time, program, or type.

For example, when the monitoring unit object is time, the monitoring detailed unit object may be defined as a region, a channel, a program, or a type.

For example, when the monitoring unit object is a program, the monitoring detailed unit object may be defined as a region, a channel, a time, or a type.

For example, when the monitoring unit object is a type, the monitoring detailed unit object may be defined as a region, a channel, a time, or a program.

In addition, the number of demux / tuner modules 110 included in each of the plurality of collection devices 100 may be the same as or different from each other. For example, the number of demux / tuner modules 110 included in the first collecting device 100_1 and the number of demux / tuner modules 110 included in the second collecting device 100_2 may be the same. It can be different or different.

Each demux / tuner module 110 may capture a video corresponding to the broadcast content at a predetermined period from the received broadcast signal and transfer the captured image to the storage module 120. In addition, each demux / tuner module 120 analyzes a transport stream (TS) of a received broadcast signal, and collects program and system information protocol (PSIP) and / or program specific information (PSI) information. 130).

The storage module 120 may store and manage the captured images of the sign language broadcast content obtained from the demux / tuner module 110 for each monitoring unit object and a monitoring subunit object. For example, the storage module 120 may store and manage captured images of the broadcast content for any combination of region, channel, time, program, or type.

In addition, the storage module 120 may store broadcast programming information (PSIP) and signaling information (PSI) together with the captured image.

The collection management module 130 may manage broadcast program organization information and program signaling information for each monitoring unit object and monitoring subunit object based on the PSIP and / or PSI information obtained from the demux / tuner module 110. . For example, the collection management module 130 may manage organization information and program signaling information of the corresponding broadcast program for any combination of region, channel, time, program, or type.

In addition, the collection management module 130 may obtain a PSIP from web EPG (Electronic Program Guide) information.

In addition, the collection management module 130 may control the demux / tuner module 110 to analyze signaling information (PSI) for each monitoring unit object and the monitoring detailed unit object.

In addition, the collection management module 130 may transmit the analysis result of analyzing the broadcast scheduling information (PSIP) and the signaling information (PSI) stored in the storage module 120 to the analysis server 200 together with the broadcast content captured image.

2, the analysis server 200 may include an analysis management module 210, a statistics module 220, a data analysis module 230, and a database 240.

The analysis management module 210 may receive a monitoring result performed according to a predetermined period from each of the N collection devices 100_1, 100_2,..., 100_N. Here, the monitoring result may include a sign language broadcast content capture image and related information. Here, the related information may include one or more of a monitoring unit object, a monitoring detailed unit object, a broadcast scheduling information (PSIP), or signaling information (PSI) including the corresponding captured image.

In addition, the analysis management module 210 may store the received monitoring result in the database 240.

In addition, the analysis management module 210 may periodically check whether there is an error of the collecting device 100 by analyzing the received monitoring result.

The statistical module 220 may aggregate and manage the analysis result of the data analysis module 230 for each monitoring unit target and / or monitoring detailed unit target.

The data analysis module 230 may analyze the monitoring result for each monitoring unit object and / or the monitoring detailed unit object in real time or non-real time, and store the analysis result in the database 240.

In addition, the data analysis module 230 may analyze the monitoring result to determine whether there is a sign language region, a sign language operation state, a sign language weight ratio, a sign language provision frequency, and the like.

3A to 3H illustrate broadcast content including various sign language regions.

Referring to FIGS. 3A and 3B, the first broadcast content 310 and the second broadcast content 320 illustrate a plurality of sign language regions. For example, the first broadcast content 310 and the second broadcast content 320 may be broadcast content in which a plurality of users appear, and each of the plurality of users may output different voices. Correspondingly, the first broadcast content 310 and the second broadcast content 320 may each include a sign language area corresponding to each of the plurality of users.

In addition, referring to FIG. 3C, the third broadcast content 330 may be broadcast content dedicated to the hearing impaired, and may include a sign language area 332 that is relatively larger than the broadcast video area 331. In addition, the sign language area 332 may be provided in a relatively central area within the third broadcast content 330.

3D to 3G, the fourth to seventh broadcast contents 340, 350, 360, and 370 may include predetermined caption regions 341, 351, 361, and 371, respectively. Regions 341, 351, 361, and 371 may be provided in various sizes and positions. Accordingly, the fourth to seventh broadcast contents 340, 350, 360, and 370 consider the sizes and positions of the subtitle areas 341, 351, 361, and 371, and thus, the sign language areas 342, 352, 362, and 372. ) Can be changed in various ways.

As another example, referring to FIG. 3H, the sign language area 381 included in the eighth broadcast content 380 may be provided to include only the receiver rather than the background area.

As described above, with respect to the sign language area included in the broadcast content, since there is no special regulation or regulation, various types of sign language areas may be provided in the broadcast content. Therefore, it is not easy to automatically detect a sign language area included in broadcast content and determine whether the broadcast content is performing a sign language broadcast. In one embodiment of the present disclosure, a method for accurately detecting a sign language area included in broadcast content is provided.

4A and 4B illustrate a sign language detection method according to an exemplary embodiment of the present disclosure.

In the sign language detection method described with reference to FIGS. 4A and 4B, even when a broadcast signal (for example, PSIP, PSI, etc.) or other information (for example, EPG information) is referenced, the corresponding broadcast content is a sign language. It may be applied when it is not known whether a region is included or when the sign language region is located in the corresponding broadcast content. However, the scope of the present disclosure is not limited thereto, and even when the sign language region may be determined by a broadcast signal or separate information, examples described with reference to FIGS. 4A and 4B may be applied for verification.

Although various and complex image analysis algorithms defined in the field of image processing may be used to analyze a sign language region, it is not necessary to use a complex or high-specific algorithm that uses hardware or software resources in analyzing a sign language. Therefore, in the present disclosure, examples of simply and efficiently analyzing a sign language image will be described.

In particular, the present disclosure describes examples of specifying or detecting a sign language area in a captured image.

First, referring to FIG. 4A, in a sign language detection method according to an exemplary embodiment of the present disclosure, a motion image is based on a feature that a motion of a sign language region does not change much compared to another region, that is, a non-signal region. The hydration area can be detected.

First, with respect to the plurality of images 410 periodically captured for a time period between the start of the broadcast program and the end, a permutation is generated for all difference images regardless of the captured time sequence. For example, a difference image may be generated by pairing two captured images, and a permutation of the difference image generated from the pair of images may be generated.

The difference image may be generated as it is from the captured images 410 as the original image. Preferably, after converting the pixels included in the captured images 410 into luminance values, the difference images of the converted images Can be generated. As another example, after converting pixels included in the plurality of captured images 410 based on a predetermined color value (for example, red), a difference image of the converted image may be generated.

In addition, the generated n difference images (diff (n)) 420 have a lot of noise, and thus, noise pixels having a small size are removed.

Thereafter, only coordinates having a small pixel value difference are extracted from the n difference images (diff (n)) 420, and the coordinate position is accumulated to generate the difference image histogram 430.

Furthermore, the final difference image histogram may be generated by removing a small amount of noise from the difference image histogram 430 generated as described above.

Next, based on a predetermined threshold value, an area in which a value of the difference image histogram is less may be detected. Here, the region where the value of the difference image histogram is small means that there is a high probability that the same object exists at the same position in the image. Accordingly, the detected region may be determined to be a sign language candidate region.

In addition, the predetermined threshold may be set differently according to a broadcaster, a broadcast program, a broadcast type, and the like. For example, in the case of the entertainment program, since it can be expected that the amount of change in most areas is larger than that of the news program, the threshold for the entertainment program may be set lower than the threshold for the news program.

For each of the sign language candidate regions detected in the captured image through the difference image histogram, face detection may be attempted. Accordingly, a region where one face is detected among the sign language candidate regions may be finally determined as a sign language region.

It is assumed that the first, second, and third sign language candidate regions 451, 452, and 453 having a change amount below a predetermined threshold are detected in the captured image 400. More specifically, the first, second, and third sign language candidate regions 451, 452, and 453 are regions in which the difference image histogram values are low over a plurality of captured images, that is, the same object is fixedly positioned at the same position. It may correspond to an area.

Next, face detection may be attempted for each of the first, second, and third sign language candidate regions 451, 452, and 453. Since one face is detected in the first sign language candidate region 451, it may be finally determined as a sign language region. Since face recognition fails in the second and third sign language candidate areas 452 and 453, it may be determined that the sign language is not a sign language area.

In addition, instead of face recognition, a sign language region pattern matching test may be performed. Specifically, the sign language candidate region detected as having a small difference image histogram value may be checked to match a predetermined sign language region pattern. For example, a predetermined hydration area pattern may be set as a background of a shape (eg, oval or circular). In the example of FIG. 4B, the first sign language candidate region 451 is finally determined to be a sign language region because an elliptical background portion is matched to a predetermined ratio (for example, 50% or more) compared to the predetermined sign language region pattern. Can be. Since the second sign language candidate region 452 has a low ratio matching the predetermined sign language region pattern, the second sign language candidate region 452 may be determined not to be a sign language region.

Here, examples of the predetermined hydration region pattern are not limited to ellipses. For example, the degree of matching with another pattern (eg, a black rectangle) may be checked according to a broadcaster, a broadcast program, a broadcast type, and the like. That is, the scope of the present disclosure is not limited to the example of a predetermined sign language region pattern, but includes various methods of detecting a sign language region using a degree of matching with a predetermined sign language region pattern.

5 and 6 are views for explaining examples of a sign language detection method according to the present disclosure.

In order to detect the hydration time, a method of excluding the non-hydration time from the time where the hydration region exists may be applied. Examples for detecting the dehydration time are described below for this purpose.

5 illustrates a method for measuring a sign language time or a non-sign language time based on hand position recognition or hand gesture recognition. For example, the hand of the called party may be recognized and the position of the recognized hand may be analyzed at predetermined intervals.

In addition, the viewpoint of recognizing the hand of the receiver may be included in the viewpoint of recognition of the face of the receiver in the sign language area. That is, the time when the face of the caller is not recognized may be determined as the non-hydration time even if the hand position is not recognized, and whether or not the time of the sign language or the sign language may be determined based on the hand position recognition at the time when the face of the caller is recognized. .

In the example of FIG. 5, since the position of the recognized hand is at the left and right sides outside the lower center of the sign language area, it may be determined that the recognized hand belongs to the sign language time.

Next, at the time points t and t + 1, since the recognized hand position is the center of the lower end of the sign language area, it may be determined that it belongs to the non-hydration time.

Next, at the time t + 2, since the recognized position of the hand is the middle of the break out of the center of the lower end of the sign language region, it may be determined that it belongs to the sign language time.

That is, the time at which the position of the object recognized by the hand of the receiver is located at the bottom center of the sign language area is determined as the non-hydration time, and the rest of the entire program providing time except for the non-hydration time during the time in which the sign language area exists. Can be determined by time.

The example of FIG. 6 relates to a method of measuring a sign language time or a non-sign language time based on color recognition. For example, by recognizing the color of a specific subregion of the sign language area, it is possible to analyze whether or not the recognized color matches the color of the hand of the receiver at every predetermined period.

In addition, the viewpoint of recognizing a color of a specific subregion of the sign language region may be included in the viewpoint at which the face of the receiver is recognized in the sign language region. That is, the time when the face of the receiver is not recognized may be determined as the non-hydration time even when the color of the specific sub-area is not recognized, and the non-hydration time or the sign language is based on the color recognition of the specific sub-area at the time when the face of the receiver is recognized. You can decide whether to time.

In the example of FIG. 6, since the specific gravity of the color value of the hand of the receiver is low in a specific lower region of the lower center of the sign language region, it may be determined to belong to a sign language time.

Next, at the time of t and t + 1, since the specific gravity of the color value of the hand of the receiver is high in a specific lower region in the lower center of the sign language area, it can be determined that the time belongs to the non-hydration time.

Next, at the time t + 2, since the specific gravity of the color value of the hand of the receiver is low in the specific lower region in the lower center of the sign language area, it can be determined that the time belongs to the sign language time.

That is, by analyzing the color of a specific subregion (eg, bottom center) in the sign language area, if the time occupied by a color similar to the color value of the hand of the receiver and / or the size of the area is higher than a predetermined threshold, the sign language is not. You can decide that it is time. Accordingly, the rest of the program providing time except for the non-hydration time may be determined as the hydration time while the hydration region is present.

In order to apply the example shown in FIG. 6, the receiver may have a dress of a color that is clearly distinguished from the background (eg, blue) of the sign language area and clearly distinguished from the color of the face and hands of the receiver.

As an additional example, the hydration time may be measured based on the motion vector.

The motion vector represents a displacement in which a specific position in the image of the first viewpoint is moved in the image of the second viewpoint. In the present disclosure, a motion vector based on a displacement of each of a plurality of feature points (for example, pixels) in a sign language area of a captured image at a first point in time is measured, and a plurality of feature points are measured. It is possible to determine whether or not the value of the scalar sum of the motion vectors for these fields is equal to or less than a predetermined threshold.

If the sum of the motion vectors is less than or equal to the threshold, it may be regarded as a time without a sign language operation of the receiver, and thus, the time between the first time point and the second time may be determined as the non-hydration time. Accordingly, the rest of the program providing time except for the non-hydration time may be determined as the hydration time while the hydration region is present.

If the sum of the motion vectors exceeds the threshold, it can be regarded as the time during which the sign language operation is performed by the receiver, and thus, the time between the first time point and the second time may be determined as the sign time.

7 is a flowchart illustrating the operation of the collecting device according to the present disclosure.

In operation S710 of FIG. 7, the collection apparatus 100 may obtain broadcast programming information (eg, channel information, program name, broadcast date, program start time, program end time, etc.). For example, such broadcast scheduling information (or PSIP) may be directly obtained by the collection management module 130 from the web EPG, and the PSIP information obtained by receiving and analyzing a broadcast signal by the demux / tuner module 110 may be obtained. It may be delivered to the collection management module 130.

In operation S720, the collecting device 100 may display signaling information (eg, program information required for a complex transport stream (TS) belonging to the multiplexed program) of the monitoring unit object and / or the monitoring subunit object in a table format. ) Can be analyzed. For example, the collection management module 130 may control the demux / tuner module 110 to analyze signaling information (or PSI) for any combination of region, channel, time, program, or type.

In operation S730, the collection device 100 may periodically capture a broadcast program screen. For example, the demux / tuner module 110 may generate a captured image at predetermined intervals during a broadcast program providing time.

In operation S740, the collection apparatus 100 may store related information of the corresponding captured image while storing the captured image. For example, the storage module 120 may include, as the related information, one or more of a monitoring unit object, a monitoring detailed unit object, a broadcasting program information (PSIP), or signaling information (PSI) including the captured image. Can be stored.

In operation S750, the collection apparatus 100 may transmit the stored captured image and the related information to the analysis server 200. In addition, the collection apparatus 100 may transmit the analysis result of the related information of the corresponding captured image to the analysis server 200. For example, the collection management module 130 may include a captured image stored in the storage module 120, related information (eg, one or more of a monitoring unit object, a monitoring detailed unit object, a PSIP, or a PSI) and The analysis result may be transmitted to the analysis management module 210 of the analysis server 200.

The analysis result may include error information about the related information. For example, the analysis result may include information indicating whether there is an error with signaling information, and if there is an error, what error occurred.

In the example of FIG. 7, the collecting device may repeat the operations of steps S730 to S750 every predetermined capture period.

Alternatively, steps S730 and S740 may be performed for each predetermined capture period, and in step S750, a plurality of captured images, related information, and analysis results corresponding to a plurality of cumulatively stored capture periods may be transmitted to the analysis server 200 at one time. have.

8 is a flowchart illustrating an operation of an analysis server according to the present disclosure.

In operation S810 of FIG. 8, the analysis server 200 may receive a captured image, related information, and an analysis result from the collection apparatus 100. For example, the analysis management module 210 may receive a captured image, related information, and an analysis result transmitted from the collection management module 130 of the collection device 100.

In operation S820, the analysis server 200 may store the received data. For example, the analysis management module 210 may store the received data (eg, captured image, related information, and analysis result) in the database 240.

In operation S830, the analysis server 200 may analyze the stored data. For example, the data analysis module 230 may analyze the stored data (eg, captured image, related information, and analysis result).

In addition, the data analysis module 230 may determine whether a sign language area exists in the captured image of the stored data. For example, the data analysis module 230 may determine whether a sign language region exists in the captured image based on the broadcasting schedule information and / or signaling information. For example, the data analysis module 230 may determine whether a sign language region exists in the captured image based on a histogram analysis, face recognition, pattern matching, or the like for the difference image.

In detail, the data analysis module 230 generates a permutation of all difference images, regardless of the captured time sequence, for the plurality of images 410 periodically captured for a time period between the start of the broadcast program and the end of the broadcast program. . For example, the data analysis module 230 may generate a difference image by pairing two captured images and generate a permutation of the difference images generated from the pair of images.

The difference image may be generated as the original image from the captured images 410, but preferably, the data analysis module 230 converts the pixels included in the captured images 410 into luminance values. A difference image of the converted image may be generated. As another example, the data analysis module 230 converts pixels included in the plurality of captured images 410 based on a predetermined color value (for example, red), and then the difference of the converted images. You can also create an image.

Furthermore, the data analysis module 230 may remove noise pixels having a small size from the generated n difference images (diff (n)) 420.

Thereafter, the data analysis module 230 extracts only coordinates having small pixel value differences from the n difference images (diff (n)) 420, and accumulates corresponding coordinate positions to generate the difference image histogram 430.

Furthermore, the data analysis module 230 may generate a final difference image histogram by removing a small amount of noise from the difference image histogram 430 generated as described above.

Next, the data analysis module 230 may detect an area where the value of the difference image histogram is less, based on a predetermined threshold value. Here, the region where the value of the difference image histogram is small means that there is a high probability that the same object exists at the same position in the image. Accordingly, the data analysis module 230 may determine that the detected area is a sign language candidate area.

In addition, the predetermined threshold may be set differently according to a broadcaster, a broadcast program, a broadcast type, and the like. For example, in the case of the entertainment program, since it can be expected that the amount of change in most areas is larger than that of the news program, the threshold for the entertainment program may be set lower than the threshold for the news program.

Thereafter, the data analysis module 230 may attempt to detect a face for each of the sign language candidate regions detected in the captured image through the difference image histogram. Accordingly, a region where one face is detected among the sign language candidate regions may be finally determined as a sign language region.

In addition, the data analysis module 230 may determine whether the captured image of the stored data belongs to a hydration time or a non-hydration time. For example, the data analysis module 230 may determine whether the captured image belongs to a sign language time or a non-sign language time based on the broadcast scheduling information and / or signaling information. For example, the data analysis module 230 may determine whether the captured image belongs to a hydration time or a non-hydration time based on a method such as hand position recognition, color recognition, motion vector analysis, or the like in the captured image.

In addition, the data analysis module 230 may store the captured image analysis result (for example, whether a sign language broadcast or a sign language broadcast quality) of the stored data is related information (for example, a monitoring unit object, a monitoring detailed unit object, a PSIP, or Along with one or more of the PSI).

In addition, the data analysis module 230 may delete the captured image from the stored data when the captured image is not a sign language broadcast.

In operation S840, the analysis server 200 may aggregate and manage the analysis result for each monitoring unit target and / or monitoring detailed unit target.

For example, the statistics module 220 may aggregate the analysis result by any combination of region, channel, time, program, or type, and output the analysis result on the screen according to a user's request. For example, the sign language broadcast quality may be represented by a value of a sign language provision frequency, and the sign language provision frequency may be represented by a numerical value such as a specific gravity of the sign language time, a distribution of sign language time, and the like.

Operations of steps S810 to S840 of FIG. 7 may be periodically performed, and information corresponding to a plurality of capture periods may be received at one time, and storage, analysis, and statistical calculation thereof may be performed at once. Alternatively, steps S810 to S830 of FIG. 7 may be repeatedly performed for each capture period, and step S840 may be performed at once based on accumulated and stored data after completion of a plurality of capture periods.

According to the present disclosure, a new image analysis method for a sign language broadcast is provided, and thus, whether or not a sign language broadcast is organized objectively and automatically, and a quality of a sign language broadcast can be verified. Therefore, by installing and operating a sign language broadcasting quality monitoring apparatus nationwide in accordance with the present disclosure, it can be utilized to verify whether the disabled broadcast notification mandatory operators and achieve the target.

Exemplary methods of the present disclosure are represented as a series of operations for clarity of description, but are not intended to limit the order in which the steps are performed, and each step may be performed simultaneously or in a different order as necessary. In order to implement the method according to the present disclosure, the illustrated step may further include other steps, may include other steps except some, or may include additional other steps except some.

The various embodiments of the present disclosure are not an exhaustive list of all possible combinations and are intended to describe representative aspects of the present disclosure, and the matters described in the various embodiments may be applied independently or in combination of two or more.

In addition, various embodiments of the present disclosure may be implemented by hardware, firmware, software, or a combination thereof. For hardware implementations, one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), General Purpose It may be implemented by a general processor, a controller, a microcontroller, a microprocessor, and the like.

It is intended that the scope of the disclosure include software or machine-executable instructions (eg, an operating system, an application, firmware, a program, etc.) to cause an operation in accordance with various embodiments of the method to be executed on an apparatus or a computer, and such software or Instructions, and the like, including non-transitory computer-readable media that are stored and executable on a device or computer.

Claims (20)

In the method for monitoring the sign language broadcast quality,
Identifying at least one captured image capturing broadcast content every predetermined first time unit;
Generating a difference image for the at least one captured image and accumulating the difference image for the broadcast content unit to detect a sign language candidate region;
Determining a sign language region by analyzing the at least one captured image corresponding to the sign language candidate region; And
A sign language broadcast quality of the broadcast content based on one or more of broadcast programming information, signaling information, a monitoring unit object, or a monitoring subunit object related to the captured image, and using an image included in the sign language area of the broadcast content. Determining the sign language broadcast quality monitoring method.
The method of claim 1,
Detecting the sign language candidate region,
Removing noise from the difference image of the at least one captured image;
Extracting coordinates of a region having a small difference between pixel values from the difference image from which the noise is removed;
And accumulating the extracted coordinates to generate a difference image histogram.
The method of claim 2,
Detecting the sign language candidate region,
And removing noise from the difference image histogram.
The method of claim 2,
Detecting the sign language candidate region,
And detecting a region having a value relatively larger than a predetermined threshold value in the difference image histogram as the sign language candidate region.
The method of claim 4, wherein
The predetermined threshold is set based on at least one of a broadcaster, a broadcast program, and a broadcast type.
The method of claim 1,
Determining the hydration region,
Confirming the presence of a face region from the region corresponding to the sign language candidate region in the at least one captured image;
And determining the sign language candidate area in which the face area exists as the sign language area.
The method of claim 1,
Determining the hydration region,
Determining whether a region corresponding to the sign language candidate region in the at least one captured image matches a predetermined sign language region pattern;
And determining the sign language candidate area corresponding to the predetermined sign language area pattern as the sign language area.
The method of claim 1,
Determining the sign language broadcast quality of the broadcast content,
And determining whether a non-hydration time or a hydration time is determined based on the image information of the sign language area in the at least one captured image.
The method of claim 8,
Determining whether the non-hydration time or hydration time,
Recognizing a hand position from the image information of the sign language area;
And determining the non-hydration time or the hydration time based on whether the position of the hand is present at a predetermined position within the sign language area and in consideration of the first predetermined time unit. How to monitor broadcast quality.
The method of claim 8,
Determining whether the non-hydration time or hydration time,
Checking whether a color corresponding to a hand is present in the first predetermined area within the sign language area;
And determining a non-hydration time or a hydration time based on whether a color corresponding to a hand exists in the first predetermined area and considering the first predetermined time unit. Quality monitoring method. An apparatus for monitoring sign language broadcast quality,
An analysis management unit for receiving and storing at least one captured image of capturing the broadcast content at a predetermined first time unit from at least one collection device providing the broadcast content to a user;
Generating a difference image for the at least one captured image, accumulating the difference image for the broadcast content unit, detecting a sign language candidate region, and analyzing the at least one captured image corresponding to the sign language candidate region, Determining a sign language area, and based on one or more of broadcast programming information, signaling information, a monitoring unit object, or a monitoring sub-unit object related to the captured image, by using an image included in the sign language area of the broadcast content; Sign language broadcast quality monitoring device comprising a data analysis unit for determining the sign language broadcast quality of the broadcast content.
The method of claim 11,
The data analysis unit,
Removing noise from the difference image for the at least one captured image,
Extracting coordinates of a region having a small difference in pixel value from the difference image from which the noise is removed,
Sign language broadcast quality monitoring device, characterized in that for generating the difference image histogram by accumulating the extracted coordinates.
The method of claim 12,
The data analysis unit,
Sign language broadcast quality monitoring device, characterized in that to remove noise from the difference image histogram.
The method of claim 12,
The data analysis unit,
And a sign language broadcasting apparatus for detecting a region having a value relatively larger than a predetermined threshold value in the difference image histogram as the sign language candidate region.
The method of claim 11,
The data analysis unit,
In the at least one captured image, the presence or absence of a face area is determined from an area corresponding to the sign language candidate area.
The sign language broadcast quality monitoring apparatus, wherein the sign language candidate area in which the face area exists is determined as the sign language area.
The method of claim 11,
The data analysis unit,
Determining whether a region corresponding to the sign language candidate region in the at least one captured image matches a predetermined sign language region pattern,
The sign language broadcast quality monitoring apparatus, wherein the sign language candidate area corresponding to the predetermined sign language area pattern is determined as the sign language area.
The method of claim 11,
The data analysis unit,
The sign language broadcast quality monitoring device, characterized in that for determining the non-hydration time or the sign language time, based on the image information of the sign language area in the at least one captured image.
The method of claim 17,
The data analysis unit,
Recognize the position of the hand from the image information of the sign language area,
A sign language broadcast quality monitoring apparatus, wherein the non-hydration time or the sign language time is determined based on whether the position of the hand exists at a predetermined position within the sign language area and in consideration of the first predetermined time unit. .
The method of claim 17,
The data analysis unit,
Check whether a color corresponding to a hand exists in the first predetermined area within the sign language area,
The sign language broadcast quality monitoring apparatus of claim 1, wherein the non-hydration time or the sign language time is determined based on whether a color corresponding to a hand exists in the first predetermined area and considering the first predetermined time unit.
The method of claim 11,
The analysis management unit,
A sign language broadcast quality monitoring device further comprising receiving and storing at least one of a monitoring unit object corresponding to the at least one captured image, a monitoring detailed unit object, broadcast scheduling information (PSIP), and signaling information (PSI). .

Images