JP2014110531A - Device, method and program for telop detection - Google Patents

Abstract

PROBLEM TO BE SOLVED: To detect a telop other than a stationary displayed telop accurately at high speed.SOLUTION: A telop detection method includes: dividing a frame image group constituting video data into plurality along a time axis; from frame images in each divided frame image group, detecting a stationary displayed telop area displayed over all video images; applying a mask image to mask the position of the stationary displayed telop area to the frame images in each frame image group; and from the frame images after the mask image is applied thereto, detecting each telop area in parallel on the basis of each frame image group.

Description

  The present invention relates to a technique for detecting a telop area displayed in an image.

  In order to support efficient video viewing such as listing of video contents and access to a specific scene, there is a technique for detecting a telop area in which text and other telops are displayed from frame images in the video (Patent Document 1, Patent Document 1). 2).

  In order to shorten the detection time of the telop area, there is a technique in which video data is divided into a plurality along the time axis, and telop detection in each processing section is performed in parallel by a plurality of CPUs or processes (non-patent document). Reference 1).

  If the video data is arbitrarily divided, there is a possibility that the telop area cannot be accurately detected due to the overlap between the detection section of the telop area and the processing section. However, in Non-Patent Document 1, the telop area is the last frame of the previous processing section. Is being detected, the previous processing section is dynamically extended to continue detecting the telop area.

Japanese Patent Laid-Open No. 10-320557 JP-A-11-205679

Goto and 2 others, "Acceleration of video event detection by parallel distribution", IPSJ Research Report, Vol.2012-AVM-76 No.2, February 23, 2012, p.1-6

  However, in addition to normal telops that are displayed according to the scene, there are regular display telops that are continuously displayed over the entire video, such as time and logo marks. If the speed-up technique of Patent Document 1 is simply applied, each processing section is significantly extended due to continuous display of the regular display telop, and the detection time of the telop area increases (see FIG. 11).

  Further, since the steady display telop does not become index information that characterizes a specific section in the video, it is inappropriate to use it for video viewing support such as cuing of a specific scene (see FIG. 12).

  The present invention has been made in view of the above circumstances, and an object thereof is to detect only a telop other than a steady display telop at high speed and accurately.

  2. The telop detection device according to claim 1, wherein the video data dividing means for dividing the frame image group constituting the video data into a plurality along the time axis, and the video data from the divided frame images of the frame image group. A stationary display telop detection means for detecting a steady display telop area displayed over the entire video, and a mask image for reading out the video data from the storage means and masking the position of the steady display telop area for each frame image group And a telop detection means for detecting a telop area in parallel for each frame image group from the frame image after applying the mask image.

  According to the present invention, the frame image group constituting the video data is divided into a plurality along the time axis, and the steady display telop area displayed over the entire video from the frame image of each divided frame image group Is applied to the frame image of each frame image group, and the telop area is detected in parallel for each frame image group from the frame image after application of the mask image. Only telops other than the steady display telop can be detected at high speed and accurately.

  2. The telop detection device according to claim 2, wherein the telop detection means is configured to extract, for each frame image group, a telop area including the steady display telop area from a frame image before application of a mask image. A plurality of telop detection means for detecting in parallel, wherein the steady display telop detection means calculates the number of telop display times each pixel in the frame image is included in the telop area, and is detected by each telop detection means. If the pixels at the same position are included in each telop area, the telop display number calculation means for adding the number of detected telop detection means to the number of telop display times for the pixel, and the number of telop display times for each pixel. A regular table for generating a histogram and determining pixels having the number of telop display times greater than or equal to a threshold as the regular display telop area A telop determining means, and summarized in that with the mask image generation means for generating a mask image with the constant display telop area and telop detected outside the determined pixels.

  The telop detection device according to claim 3 is the telop detection device according to claim 1 or 2, wherein the telop detection means detects a telop region including the steady display telop region from a frame image before application of a mask image, and The steady display telop detection means determines a threshold based on the number of telop display times for each pixel in the frame image and calculates the number of telop display times included in the telop area, and determines a threshold value based on the number of telop display times for each pixel. A normal display telop determination unit that determines pixels having the number of telop display times as the normal display telop region, and a mask that generates a mask image in which all pixels are subject to telop detection when there is no pixel determined as the normal display telop region. And an image generation means.

  5. The telop detection method according to claim 4, wherein a computer divides the frame image group constituting the video data into a plurality of pieces along the time axis, and the video data is obtained from the divided frame images of each frame image group. Detecting a steady display telop area displayed over the entire video, and reading the video data from the storage means, and a mask image for masking the position of the steady display telop area as a frame image of each frame image group And a step of detecting a telop area from each frame image group in parallel from the frame image after applying the mask image.

  The gist of the telop detection program according to a fifth aspect is to cause a computer to function as the telop detection device according to any one of the first to third aspects.

  According to the present invention, only telops other than the steady display telop can be detected at high speed and accurately.

It is a figure which shows the functional block structure of a telop detection apparatus. It is a figure which shows the whole processing flow of a telop detection apparatus. It is a figure which shows the production | generation processing flow of process area data. It is a figure which shows the example of process area data. It is a figure which shows the example of allocation of a process area. It is a figure which shows the detection processing flow of a telop area. It is a figure which shows the detection processing flow of a regular display telop area | region. It is a figure which shows the calculation image of the number of times of telop display. It is a figure which shows the application image of a mask image. It is a figure which shows the example of mounting of a telop detection apparatus. It is a reference figure at the time of the subject description of a prior art. It is a reference figure at the time of the subject description of a prior art.

  The present invention applies a mask image masking the position of a steady display telop area displayed over the entire video of video data to the frame image of each processing section, and processes each telop area from the frame image after applying the mask image It is characterized by detecting in parallel for each section.

  As a result, the telop area detection process is executed in a state where the steady display telop area is masked, so that the continuation of the steady display telop is continued even when a high-speed technique such as Non-Patent Document 1 is used. Since the processing section is not extended based on the display, it is possible to accurately detect only useful telops (hereinafter referred to as unsteady display telops) that can be used for video viewing support.

  In addition, since the detection of the telop area is performed in parallel for each processing section without causing unnecessary extension of the processing section, it takes a processing time as detailed in Patent Documents 1 and 2. Since the telop area detection processing is distributed in parallel, it is possible to detect a non-steady display telop at high speed.

  Hereinafter, an embodiment for carrying out the present invention will be described with reference to the drawings. However, the present invention can be implemented in many different modes and should not be construed as being limited to the description of the present embodiment.

[Configuration of telop detection device]
FIG. 1 is a diagram showing a functional block configuration of a telop detection device 1 according to the present embodiment. The telop detection device 1 includes a video storage unit 11, a video data division unit 12, a telop detection unit 13, a steady display telop detection unit 14, an intermediate result storage unit 15, a detection result integration unit 16, and an integration result storage. It is mainly comprised by the part 17.

  The video storage unit 11 has a function of storing video data input from an external data input device and the number of processing workers using a communication protocol such as HTTP (HyperText Transfer Protocol) or SMB (Server Message Block). .

  The video data dividing unit 12 has a function of dividing the frame image group constituting the video data into a plurality of pieces along the time axis.

  Specifically, the processing section calculation unit 121 that calculates the time stamps of the start position and end position of each processing section obtained by dividing the video data by the number of input processing workers, and associates each time stamp with the processing section ID. A processing section storage unit 122 that stores the processing section data, and an analysis processing management unit 123 that controls the detection processing of the telop area by assigning each time stamp of each processing section to each processing worker unit 132 of the telop detection unit 13. It consists of.

  The telop detection unit 13 reads video data from the video storage unit 11, detects a telop area from the frame images of the divided frame image groups of each processing section, and determines the position of the steady display telop area included in the telop area. A mask image to be masked is applied to the frame image of each frame image group, and a telop area is detected in parallel for each frame image group from the frame image after the mask image application.

  Specifically, a video acquisition unit 131 that reads out a frame image group of each processing section from the video storage unit 11 based on the processing section data and assigns it to each processing worker unit 132, and a frame of the frame image group of each processing section A plurality of processing worker units 132 that independently detect a telop area from an image in a time parallel manner. The number of processing worker units 132 corresponds to the input number of processing workers.

  The steady display telop detection unit 14 has a function of detecting a steady display telop area from the frame images of the frame image group of each divided processing section.

  Specifically, a telop display frequency calculation unit 141 that calculates the number of times each pixel in the frame image is included in the telop area as the telop display frequency of the pixel, and a frequency histogram of the telop display frequency of each pixel are generated, A normal display telop determination unit 142 that determines whether or not a normal display telop area exists in the telop area, and a frame mask generation that generates a mask image that excludes pixels determined to be the normal display telop area from the telop detection target Part 143.

  The intermediate result storage unit 15 stores the telop area detection results (for example, the coordinate position of the telop area, the start time stamp and the end time stamp of the telop area) detected by each processing worker unit 132 as intermediate results. have.

  The detection result integration unit 16 has a function of integrating all intermediate results stored in the intermediate result storage unit 15 into one, and storing the integrated telop detection result in the integration result storage unit 17.

  The integration result storage unit 17 has a function of associating the start time stamp and end time stamp of the video section in which the telop area is continuously displayed with the coordinate position of the telop area and storing it as a telop detection result of the video data. ing.

  Note that the telop detection unit 13 dynamically extends the previous processing section when the telop area is being detected in the final frame image of the previous processing section, as in the high-speed technique disclosed in Non-Patent Document 1. Thus, detection of the telop area is continued.

  The telop detection device 1 including the functional units 11 to 17 described above can be realized using a computer including a memory and a CPU. Moreover, each processing operation of each function part 11-17 is performed by a program. Furthermore, it is also possible to construct a telop detection program that causes a computer to function as the telop detection device 1 by mounting a program for executing the functional units 11 to 17 on the computer.

[About the processing operation of the telop detection device 1]
Next, the processing operation of the telop detection device 1 will be described. FIG. 2 is a diagram showing an overall processing flow of the telop detection device 1.

  First, the telop detection device 1 receives video data to be processed input by a data input device by a user operation, and stores it in the video storage unit 11 in association with the number of processing workers input at the same time (step S1). ).

  Next, the processing section calculation unit 121 divides the number of video data by the input number of processing workers, thereby calculating a plurality of processing sections constituted by each frame image group divided thereby (step S2). ).

  Here, the process of step S2 will be described in detail with reference to FIG. First, the processing section calculation unit 121 acquires the video length L and the number of processing workers W of video data to be processed from the video storage unit 11 (step S201).

Then, calculate the W number of processing sections P _i obtained by dividing the image length L obtained in the process worker number W. For example, the variable i (i is a natural number including 0) is initialized (step S202), and the value of the variable i is compared with the value of the processing worker number W (step S203).

If the relationship i ≧ W does not hold, the start time stamp S _i of the processing section P _i of the variable i is calculated from the equation (1), and the end time stamp E _i is calculated from the equation (2) (step S204). ), and their calculation result stored in the processing section storage unit 122 as the processing section _{P i} (step S205). The time stamp means the time (ms) from the beginning when the frame image is displayed.

S _i = i × (L / W) (1)
E _i = S _{i + 1} Expression (2)
Thereafter, after incrementing the variable i (step S206), the process returns to step S203, and steps S204 to S206 are repeated until the relationship of i ≧ W is established.

Through the above processing, the variable i is by being incremented from i = 0 to i = W-1, processing block data composed of W number of processing sections P _i are calculated as FIG.

Finally, the analysis processing management unit 123 reads the obtained processing section data from the processing section storage unit 122, and obtains the start time stamp S _i and end time stamp E _i of each processing section P _i as shown in FIG. Each processing worker unit 132 is assigned (step S207).

Subsequently, returning to FIG. 2, the processing worker unit 132 sequentially obtains a frame image for each predetermined time interval from the frame images of each of the processing sections P _i assigned to each, the acquired frame images From this, a telop area is detected (step S3).

  For example, as described in Patent Documents 1 and 2, paying attention to the fact that the shading level of a character is convex or concave on the scanning line, the number of pairs of adjacent edges having convex or concave shading changes is calculated. Then, an area where the total number of edge pairs in the horizontal direction and the vertical direction of the frame image falls within a predetermined setting range is detected as a telop area. Other detection methods may be used.

  Thereafter, the steady display telop determination unit 142 uses the telop display count of the pixel calculated by the telop display count calculation unit 141 to determine whether or not the steady display telop area is included in the telop area detected in step S3. Determine (step S4).

  Here, when there is a steady display telop area in the telop area, the frame mask generation unit 143 generates a mask image that masks the position of the steady display telop area (step S5).

  On the other hand, when the steady display telop area does not exist in the telop area, the frame mask generation unit 143 generates a mask image in which all pixels of the frame image are telop detection targets (step S6).

  Thereafter, each processing worker unit 132 detects a telop area from the frame image to which the mask image generated in step S5 or step S6 is applied (step S7).

  In this step S7, since the telop area is detected in a state where the position of the steady display telop area is masked, the processing section is not extended based on the continuous display of the steady display telop, so that it can be used for video viewing support. Only unsteady display telops can be accurately detected.

  In addition, since the telop detection processing of step S3 and step S7 is executed in parallel for each processing section, the detection processing of heavy telop areas involving processing such as edge detection is distributed in parallel. It can be detected at high speed.

  Thereafter, the detection result integration unit 16 integrates the intermediate results of the telop area detection processing performed by the processing worker units 132 in step S7 (step S8), and the integrated result storage unit as the telop detection result of the video data. 17 (step S9).

  Below, the processing of step S3 to step S9 will be described in detail. First, the telop detection process performed by the telop detection unit 13 will be described in detail with reference to FIG.

<Telop detection processing>
First, before detecting a telop area, the telop detection unit 13 sets a measurement period T of the number of telop display of pixels at a time interval equal to or greater than the telop detection processing period F, and counts the number of telop detection processes. c is initialized (step S301).

  For example, when the telop area detection process is executed every 500 ms, the measurement cycle T of the number of telop displays is set to 500 ms or 1000 ms. As a result, when the telop area detection process is executed once, the telop display count of the pixel is measured once or twice.

Next, each processing worker unit 132 seeks to the start point frame of each processing section P _i , acquires a frame image group of the processing section P _i assigned to each processing section P _i (step S302), and the frame of the start point frame Using the image, the telop area detection process is executed in parallel for each processing section (step S303).

  Next, only when the remainder value when the telop detection processing period F is multiplied by the processing number counter c and divided by the measurement period T is 0 and the telop area is detected, the coordinate position of the telop area is detected. Is transmitted to the telop display count calculator 141 (step S304).

  At this time, the telop display number calculation unit 141 counts the telop display number of the pixel corresponding to the coordinate position, and the frame mask generation unit 143 generates a mask image based on the count result.

  Subsequently, each processing worker unit 132 increments the processing number counter c every time the telop detection processing is executed once (step S305), and whether the currently processed frame image is the frame image of the end point frame of each processing section. It is determined whether or not (step S306).

  If it is not the frame image of the end point frame of each processing section, the next frame image in each assigned processing section is acquired (step S307), and the mask image generated by the frame mask generation unit 143 is applied to the frame image. (Step S308).

  Then, after step S308, the process returns to step S303, and each processing worker unit 132 detects a telop area from the frame image after application of the mask image (step S303 '). Thereafter, step S303 to step S308 are repeated while sequentially obtaining frame images until reaching the end point frame in each processing section.

  As will be described later, the gray level (for example, the luminance value or the saturation value) of the pixel masked by the mask image is 0, so that it is not processed in the telop area detection process in step S303 '. As a result, unnecessary extension of the processing section based on the steady display telop area is eliminated, and only useful unsteady display telops can be accurately detected.

  On the other hand, if the result of determination in step S306 is that the currently processed frame image is a frame image of the end point frame of each processing section, it is determined whether or not a telop area is currently being detected (step S309). .

  If a telop area is being detected, a frame image of the next processing section is acquired (step S310), and step S308 described above is executed. Thereafter, the processing section is extended until no detected telop appears, and steps S303 to S306, step S309, and step S310 are repeated.

  As a result of the determination in step S309, if the telop area is not detected, there is no need to continue the telop area detection process in each processing section, so that each detection obtained by each telop detection process The result is stored in the intermediate result storage unit 15 as an intermediate result (step S311), and the telop area detection process is terminated.

<Normal display telop detection processing>
Next, the steady display telop detection process performed by the steady display telop detection unit 14 will be described in detail with reference to FIG.

  First, the telop display count calculation unit 141 receives the coordinate position of the telop area transmitted from each processing worker unit 132, and counts the telop display count of each pixel (step S401).

  As a method of counting the number of telop displays, for example, 0 may be given as the initial value for the number of telop displays, and the number of telop displays may be incremented according to the number of times included in the telop area. As a result, the number of telop display times for pixels that have not been included in the telop area once becomes zero, and the number of telop display times for pixels included in the telop area increases in accordance with the number of times they are included.

  In addition, for example, when the three processing worker units 132 each detect a telop area, and each telop area includes a pixel at the same position, the process in which the telop area is detected by the number of telop display times of the pixel. The number of worker units 132 may be further added (FIG. 8).

  Next, the steady display telop determination unit 142 acquires the number of telop display for each pixel from the telop display number calculation unit 141, and generates a frequency histogram (step S402).

  Subsequently, the steady display telop determination unit 142 determines whether or not there is a pixel having a telop display count equal to or greater than a threshold from the frequency histogram (step S403).

  If there is a pixel having a telop display count equal to or greater than the threshold, the pixel group is set as a steady display telop area (step S404). If no such pixel exists, the pixel group is an area other than the regular display telop area. (Step S405).

  This threshold is a value used to separate the pixels in the steady display telop area from the other pixels, and may be a fixed value set in advance before processing, or may be generated from the number of telop display times of each pixel. It may be determined dynamically during processing based on the frequency histogram. As a calculation method for dynamically determining the threshold value during the processing, for example, an existing automatic binarization algorithm such as a discrimination / identification method, a least square method, or a mode method can be used.

  As described with reference to FIG. 8, the number of telop display for each pixel may be proportional to the number of processing worker units 132. Therefore, when a fixed value is used as the threshold value, the following equation (3) is used. Thus, the threshold Th may be determined by calculating how many of the processing worker units 132 have detected the telop area. α is a ratio included in the upper n percent (n is a natural number), and W is the number of processing worker parts.

Th = α × W Formula (3)
Next, the frame mask generation unit 143 acquires the determination result from the steady display telop determination unit 142, and acquires the size of the frame image from the analysis processing management unit 123 (step S406).

  Then, a mask image is generated that has the same size as the frame image of the video data to be processed and sets the pixel value of the coordinate position included in the steady display telop area to 0, and sets the pixel value of other coordinate positions to 1. Step S407).

  Through the above processing, a mask image is generated in which 0 is set only for the pixel at the position of the steady display telop area and 1 is set for the other pixels. In particular, if there is no pixel determined to be a steady display telop area as a result of the determination in step S403, a mask image for setting all pixels to telop detection is generated by setting all pixel values to 1. .

  As described in step S <b> 303 ′, each processing worker unit 132 executes the telop area detection process by applying the mask image generated here to the frame image.

  Specifically, each processing worker unit 132 integrates each pixel value p (x, y) of the original original frame image and each pixel value q (x, y) of the mask image to obtain each pixel value p. A frame image after application of the mask image consisting of '(x, y) is generated (FIG. 9), and a telop area detection process is executed.

  When the pixel value q (x, y) of the coordinate (x, y) of the mask image is 0, the pixel value p ′ of the coordinate (x, y) of the frame image after application of the mask image is obtained from Equation (4). (X, y) is 0.

  On the other hand, when the pixel value q (x, y) of the coordinate (x, y) of the mask image is 1, the pixel value of the coordinate (x, y) of the frame image after application of the mask image is obtained from Equation (5). p ′ (x, y) is the pixel value p (x, y) of the coordinates (x, y) of the original frame image.

p ′ (x, y) = 0 (q (x, y) = 0) (4)
p ′ (x, y) = p (x, y) (q (x, y) = 1) (5)
Through all the processing operations described above, the integrated result storage unit 17 stores the start time stamp (start frame position information) and end time stamp (end frame position information) of the video section in which the telop area is continuously displayed as the telop detection result after integration. (End frame position information) and the coordinate position of the telop area are stored in association with each other.

  By outputting such an integrated telop detection result to a screen or the like using a data output device (FIG. 1) or reading it out as a text file, the user can use it for efficient viewing support such as access to a specific scene. it can.

  Finally, an implementation example of the telop detection device 1 is shown in FIG. FIG. 10A illustrates an implementation example in which one core 1 of the multi-core CPU functions as the analysis processing management unit 123, and a plurality of cores 2 to 4 connected thereto function as a plurality of processing worker units 132. Is shown.

  FIG. 10B illustrates an implementation example in which one server functions as the analysis processing management unit 123 and a plurality of analysis servers 1 to 3 connected to the server function as a plurality of processing worker units 132. Is shown.

  In addition to the above, telop area detection processing may be performed in parallel by a plurality of servers having multi-core CPUs. As long as each processing worker unit 132 can be operated independently, the construction form of each processing worker unit 132 is not limited.

  As described above, according to the present embodiment, the frame image group constituting the video data is divided into a plurality along the time axis, and the divided frame image groups are displayed over the entire video. A mask image that masks the position of the steady display telop area is applied to the frame image of each frame image group, and the telop area is applied in parallel to each frame image group from the frame image after the mask image is applied. Therefore, only telops other than the regular display telop such as time and logo mark can be detected at high speed and accurately.

DESCRIPTION OF SYMBOLS 1 ... Telop detection apparatus 11 ... Video | video memory | storage part 12 ... Video data division | segmentation part (video data division | segmentation means)
121 ... processing section calculation unit 122 ... processing section storage unit 123 ... analysis processing management unit 13 ... telop detection unit (telop detection means)
131 ... Video acquisition unit 132 ... Processing worker unit 14 ... Steady display telop detection unit (steady display telop detection means)
141... Telop display count calculation unit (telop display count calculation means)
142 ... steady display telop determination unit (steady display telop determination means)
143 ... Frame mask generation unit (mask image generation means)
15 ... Intermediate result storage unit 16 ... Detection result integration unit 17 ... Integration result storage unit S1 to S9, S201 to S207, S301 to S311, S401 to S407 ... step

Claims (5)

Video data dividing means for dividing the frame image group constituting the video data into a plurality along the time axis;
Steady display telop detection means for detecting a steady display telop area displayed over the entire video of the video data from the divided frame image of each frame image group;
The video data is read from the storage means, a mask image for masking the position of the steady display telop area is applied to the frame image of each frame image group, and the telop area is applied to each frame image group from the frame image after applying the mask image. Telop detection means for detecting in parallel,
A telop detection device comprising: The telop detection means includes
A plurality of telop detection means for detecting, in parallel for each frame image group, a telop area including the steady display telop area from a frame image before application of a mask image;
The steady display telop detection means includes:
The number of telop display times that each pixel in the frame image is included in the telop area is calculated, and if each telop area detected by each telop detection means includes a pixel at the same position, the telop of the pixel Telop display number calculation means for adding the number of detected telop detection means to the display number;
A normal display telop determining unit that generates a histogram of the number of telop display times of each pixel, and determines a pixel having the telop display number equal to or greater than a threshold value as the normal display telop area;
Mask image generation means for generating a mask image that excludes pixels determined to be the normal display telop area as telop detection targets;
The telop detection device according to claim 1, comprising: The telop detection means includes
Detecting a telop area including the stationary display telop area from a frame image before application of a mask image;
The steady display telop detection means includes:
A telop display number calculating means for calculating the number of telop display times each pixel in the frame image is included in the telop area;
A normal display telop determination means for determining a threshold based on the number of telop display times of each pixel, and determining a pixel having the telop display frequency equal to or greater than the threshold as the normal display telop area;
When there is no pixel determined to be the steady display telop area, a mask image generation unit that generates a mask image in which all pixels are telop detection targets;
The telop detection device according to claim 1, further comprising: By computer
Dividing the frame image group constituting the video data into a plurality along the time axis;
Detecting a steady display telop area displayed over the entire video of the video data from the frame image of each divided frame image group;
The video data is read from the storage means, a mask image for masking the position of the steady display telop area is applied to the frame image of each frame image group, and the telop area is applied to each frame image group from the frame image after applying the mask image. Detecting in parallel with,
A telop detection method characterized by comprising:   A telop detection program for causing a computer to function as the telop detection device according to any one of claims 1 to 3.

Images