JP4913801B2 - Shielding object image identification apparatus and method - Google Patents

Description

  The present invention relates to a shielding object video identification apparatus and method for identifying a shielding object in an image.

  Video cameras are widely used by general users, and many people enjoy taking pictures at various events and trips. By the way, as for what to do with the captured content afterwards, there are many examples in which it is left as it is depending on whether it is viewed once or not. One of the main reasons is that shooting is still “material” and is not suitable for viewing as it is. The reason why it is not suitable for viewing is that there are many unsightly scenes in which a person crosses in front of the video camera during shooting or a person's head enters the lower part of the screen. Editing work is absolutely necessary to create attractive content suitable for viewing. However, because it takes time and effort, it gives up editing and ends up being shot.

  In the case of editing, it is necessary to first check all the images shot first, and to select a scene to be left and an unnecessary scene to be deleted. Thereafter, correction or processing is performed if necessary, and finally a telop is added or a transition effect is added to the joint. Inserting the last telop and adding a transition effect are both creative and fun, but the first scene selection needs to see all of the video shot for editing, which is very time consuming and not interesting. For this reason, there are many cases where people dislike editing at this stage and give up editing.

  In order to easily select an unnecessary scene image from an image during editing, it is necessary to identify an obstruction in the image, that is, an object that obstructs the subject. As a conventional method for identifying a moving body as such a shielding object, an image memory for storing an image signal obtained by imaging with a camera is provided, and an image motion vector amount (motion vector) between image frames is set. Measure and detect a moving area with motion, convert the coordinates of the detected moving area on the camera image into real space coordinates, and convert at least the size, aspect ratio or moving speed of the object converted into the real space coordinates There is a method in which one is collated with a table in which features of a plurality of objects are registered in advance to identify the type of the corresponding moving object (Patent Document 1).

  Also, a method is known in which a motion vector for a moving object in a video is extracted, an object block is extracted based on a motion index, its statistic is obtained, and a target moving body is determined by comparison with a reference value. (See Patent Document 2).

Furthermore, as a method of identifying the blurred area of the captured image, the area is detected for an area where the in-focus state is a predetermined level or less according to the contrast ratio, and if the detected area is equal to or larger than the predetermined area, the area is blurred. A method for determining an area is known (see Patent Document 3).
Japanese Patent Laid-Open No. 2002-027449 JP 09-252467 A JP 2005-136727 A

  However, such conventional methods for identifying moving objects and blurred parts require camera information and feature point extraction / tracking, and identify a shielding object with sufficient accuracy from a general video signal. There was a drawback that it was not possible.

  The problems to be solved by the present invention include the above-mentioned drawbacks as an example, and a shielding object image identification apparatus and method, and a shielding object image identification method capable of accurately identifying an obstruction in an image from an image signal. It is an object of the present invention to provide a computer readable program for executing

A shielding object video identification apparatus according to a first aspect of the present invention is a shielding object video identification apparatus for identifying a shielding object in a video, and calculates a motion level of a video signal in units of pixels or blocks for each frame. A calculation unit, a binarization unit that binarizes the calculated value of the motion level, and a continuous area adjacent between pixels or blocks in which the output value of the binarization unit takes the same value for each frame. Two-dimensional continuous region detecting means for detecting, large region extracting means for extracting a region having a predetermined area or more as a large region among the regions detected by the two-dimensional continuous region detecting means, and a circumscribed rectangular area of the large region An area ratio determination unit that determines that the area ratio determination condition is satisfied when the ratio of the area to the area is a value in a first predetermined range, and a ratio between the horizontal width and the vertical width of the large area is in a second predetermined range If it is a value, A shape determination unit that determines that the position is determined; a position determination unit that determines that a position determination condition is satisfied when a vertical coordinate of the center of gravity of the large region is in a predetermined lower portion of the entire screen; and the area ratio determination condition, And occlusion determination means for determining the large area as an occlusion portion when at least one of a shape determination condition and the position determination condition is satisfied .

According to a seventh aspect of the present invention, there is provided a shielding object video identification device for identifying a shielding object in an image , wherein a frequency analysis is performed on a video signal in units of pixels or blocks each including a plurality of pixels. And a ratio between the low frequency component power and the high frequency component power other than that in pixel units or block units according to the frequency analysis result by the frequency analysis unit, and a calculated value of the blur level of the video A blur level calculating means, a binarizing means for binarizing the calculated value of the blur level, and a continuous adjacent pixel or block where the output values of the binarizing means have the same value for each frame. Two-dimensional continuous area detecting means for detecting a specific area, and a large area extracting means for extracting an area having a predetermined area or more as a large area among the areas detected by the two-dimensional continuous area detecting means The area ratio in the large region extracted by the large region extracting means, and determining means for determining the large region based on at least the shape and position as a shield portion, comprising the.

A shielding object video identification device according to a thirteenth aspect of the present invention is a shielding object image identification apparatus that identifies a shielding object in an image, and calculates a motion level of a video signal for each frame in units of pixels or blocks. A calculation unit, a first binarization unit that binarizes the calculated value of the motion level, and an adjacent pixel or block that has the same output value of the first binarization unit for each frame A first two-dimensional continuous region detecting means for detecting a continuous region, and a region having a predetermined area or more extracted as a large region among the regions detected by the first two-dimensional continuous region detecting means. 1 of a large area extracting means, the first area ratio in the large region extracted by the large area extracting means, first determination means for determining the large region based at least on one of the shape and position as a shield portion And single pixel Alternatively, the blur level calculation means for calculating the blur level of the video in block units, the second binarization means for binarizing the calculated blur level, and the output of the second binarization means for each frame A second two-dimensional continuous region detecting means for detecting a continuous region adjacent between pixels or blocks having the same value, and a region detected by the second two-dimensional continuous region detecting means; a second large area extracting means for extracting a predetermined area or more area as a large area, the large the area ratio in the large area extracted by the second large area extracting means, based on at least the shape and position a second determination means for determining an area as the shield portion, wherein the blur level calculation means, frequency analysis of frequency analysis in blocks consisting of pixel units or a plurality of pixels a video signal for each frame Has a step, the calculated value of the ratio the blur level of the frequency component power of the high-frequency except it with the frequency component power in low frequency in pixel units or block units in accordance with the frequency analysis result of said frequency analysis means It is characterized by that.

The shielding image identifying method of the invention according to claim 18 is a shielding image identifying method for identifying a shielding in an image, wherein the motion level is calculated for each frame of a video signal in units of pixels or blocks. A calculation step, a binarization step for binarizing the calculated value of the motion level, and a continuous adjacent pixel or block between the binarized values in the binarization step for each frame A two-dimensional continuous region detecting step for detecting a specific region, a large region extracting step for extracting a region having a predetermined area or more as a large region among the regions detected in the two-dimensional continuous region detecting step, and the large region An area ratio determination step for determining that the area ratio determination condition is satisfied when a ratio of a circumscribed rectangular area to a region area is a value in a first predetermined range; and a horizontal width and a vertical width of the large region A shape determination step that determines that the shape determination condition is satisfied when is a second predetermined range value, and a position determination condition is satisfied when the vertical coordinate of the center of gravity of the large area is in a predetermined lower portion of the entire screen. A position determination step for determining, and a shielding object determination step for determining the large region as a shielding object part when at least one of the area ratio determination condition, the shape determination condition, and the position determination condition is satisfied. It is characterized by that.

According to a nineteenth aspect of the present invention, there is provided a shielding object video identification method for identifying a shielding object in an image , wherein a frequency analysis is performed on a video signal by a pixel unit or a block unit including a plurality of pixels for each frame. The ratio of the frequency component power of the low frequency to the frequency component power of the high frequency other than that in the pixel unit or block unit according to the frequency analysis result of the frequency analysis step to be calculated value of the blur level of the video A blur level calculating step, a binarizing step for binarizing the calculated value of the blur level, and between pixels or blocks in which the values binarized in the binarizing step for each frame have the same value A two-dimensional continuous region detecting step for detecting adjacent continuous regions, and a predetermined area or less of the regions detected in the two-dimensional continuous region detecting step. A large area extracting step of extracting a region as a large region, the determination step of determining the area ratio in the large region extracted in the large area extracting step, the large region based at least on one of the shape and position as a shield portion It is characterized by having.

A program according to a twentieth aspect of the present invention is a computer-readable program for executing a shielding image identifying method for identifying a shielding object in an image, wherein a motion level of a video signal is determined in units of pixels or blocks for each frame. A motion level calculation step for calculating, a binarization step for binarizing the calculated value of the motion level, and between pixels or blocks in which the values binarized in the binarization step for each frame have the same value A two-dimensional continuous region detecting step for detecting continuous regions adjacent to each other, and a large region extracting step for extracting a region having a predetermined area or more as a large region among the regions detected in the two-dimensional continuous region detecting step; , the large area circumscribed rectangle area and the ratio of the region area is an area ratio determined stearyl determines to satisfy the area ratio determination condition when a value of the first predetermined range of And a shape determination step that determines that a shape determination condition is satisfied when the ratio of the horizontal width to the vertical width of the large area is a value in a second predetermined range, and the vertical coordinate of the center of gravity of the large area is the entire screen A position determination step for determining that the position determination condition is satisfied when the position is in a predetermined lower portion of the region, and when at least one of the area ratio determination condition, the shape determination condition, and the position determination condition is satisfied, And a shielding object judging step for judging as a shielding part .

A program according to a twenty-first aspect of the present invention is a computer-readable program for executing a shielding image identifying method for identifying a shielding object in an image, wherein the image signal is a block composed of a pixel unit or a plurality of pixels for each frame. The frequency analysis step for analyzing the frequency in units and the ratio between the low frequency component power and the high frequency component power in the pixel unit or block unit according to the frequency analysis result in the frequency analysis step is blurred. A blur level calculation step for calculating a level, a binarization step for binarizing the blur level calculation value, and a value binarized in the binarization step for each frame have the same value. A two-dimensional continuous region detecting step for detecting adjacent continuous regions between pixels or blocks, and the two-dimensional continuous region detecting step; As the large area extracting step of extracting a predetermined area or more area of the areas that are issued as a large area, the area ratio in the large region extracted in the large region extraction step, based on at least the shape and position And a determination step of determining a large area as a shielding part.

It is a block diagram which shows the Example of this invention. It is a figure for demonstrating calculation of a motion level. It is a figure for demonstrating calculation of a blur level. It is a figure which shows the state of each part of the apparatus in the case of determination operation | movement about the image | video in which a motion obstruction exists. It is a figure which shows the state of each part of the apparatus in the case of determination operation | movement about the image | video at the time of camera panning. It is a figure which shows the state of each part of the apparatus in the case of determination operation | movement about the image | video which has a blur obstruction. It is a block diagram which shows the other Example of this invention. It is a block diagram which shows the obstruction image identification device provided only with the structure of the movement obstruction determination system as an Example of this invention. It is a block diagram which shows the shielding object video identification device provided only with the structure of the blur shielding object determination system as an Example of this invention. It is a block diagram which shows the other Example of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 2D-DCT circuit 2 Motion shielding object determination system 3 Blur shielding object determination system 4 Determination result integration part 5 Detection frame expansion part

According to the first aspect of the present invention, the motion level is calculated by the motion level calculation means for each pixel or block unit of the video signal for each frame, and after the motion level calculation value is binarized, the two-dimensional continuous region detection is performed. The two-dimensional continuous region is detected by the means, and a large region is extracted from the region, and the ratio of the circumscribed rectangular area and the region area of the large region is a value in the first predetermined range by the area ratio determination unit. When it is determined that the area ratio determination condition is satisfied, the shape determination unit determines that the shape determination condition is satisfied when the ratio of the horizontal width to the vertical width of the large region is a value in the second predetermined range, and the position determination unit It is determined that the position determination condition is satisfied when the vertical coordinate of the center of gravity of the region is in a predetermined lower part of the entire screen, and at least one of the area ratio determination condition, the shape determination condition, and the position determination condition is determined by the shielding object determination unit. 1 it is performed determines its large area as a shield portion when satisfied. According to the invention of claim 18 , the motion level is calculated for each frame of the video signal in units of pixels or blocks, and the calculated value of the motion level is binarized, and then a two-dimensional continuous region is detected, Further, a large area is extracted from the area, and it is determined that the area ratio determination condition is satisfied when the ratio of the circumscribed rectangular area and the area of the large area is a value in the first predetermined range. When the ratio between the height and the vertical width is a value in the second predetermined range, it is determined that the shape determination condition is satisfied, and the position determination condition is satisfied when the vertical coordinate of the center of gravity of the large area is in a predetermined lower part of the entire screen Then, when at least one of the area ratio determination condition, the shape determination condition, and the position determination condition is satisfied, the large area is determined as the shielding part . Furthermore, shield image identification method similar of the invention according to claim 18, according to the invention of claim 20 is executed by a computer.

Therefore, according to the inventions according to claims 1, 18 and 20 , it is possible to accurately identify a shielding object having a motion in the video from the video signal.

According to the frequency analysis result of the frequency analysis means, the motion level calculation means according to claim 1 calculates the ratio of the power of the highest frequency component in the vertical direction and the sum of the powers of all components excluding the DC component as the value of the motion level. To do. The frequency analysis means performs DCT (two-dimensional discrete cosine transform) or DFT (discrete Fourier transform) in units of blocks composed of a plurality of pixels. The two-dimensional continuous area detecting means according to claim 1 performs a labeling process on the output value of the binarizing means for each frame.

According to the seventh aspect of the present invention, the blur level is calculated by the blur level calculation means for each pixel or block unit of the video signal for each frame, and the calculated value of the blur level is binarized. The two-dimensional continuous area is detected by the means, and the large area is extracted, and the large area is determined as the shielding part based on at least one of the area ratio, shape and position in the large area by the determining means. Is done. According to the invention of claim 19 , the blur level is calculated for each frame of the video signal in units of pixels or blocks, and after calculating the blur level, the two-dimensional continuous region is detected. Further, a large area is extracted from the areas, and the large area is determined as the shielding portion based on at least one of the area ratio, shape, and position in the large area. Further, according to the invention of claim 21 , the same method as the shielding image identification method of the invention of claim 19 is executed by a computer.

Therefore, according to the inventions according to claims 7, 19 and 21 , the out-of-focus shield in the video can be accurately identified from the video signal.

The blur level calculation means according to claim 7 sets the ratio of the frequency component power in the low frequency band to the frequency component power in the high frequency band other than the frequency analysis power and the blur frequency level according to the frequency analysis result. The frequency analysis means performs DCT or DFT in units of blocks composed of a plurality of pixels. The two-dimensional continuous area detecting means according to claim 7 performs a labeling process on the output value of the binarizing means for each frame.

The determination means according to claim 7 includes an area ratio determination unit that determines that the area ratio determination condition is satisfied when the ratio of the circumscribed rectangular area of the large region to the region area is a value in a first predetermined range, and the width of the large region A shape determination unit that determines that the shape determination condition is satisfied when the ratio between the vertical width and the vertical width is a value in the second predetermined range, and a position when the vertical coordinate of the center of gravity of the large area is in a predetermined lower part of the entire screen A position determination unit that determines that the determination condition is satisfied.

According to the invention of claim 13 , the video signal is frequency-analyzed in units of pixels or in units of blocks composed of a plurality of pixels for each frame, and the motion level is calculated in units of pixels or units according to the frequency analysis results, After binarizing the calculated value of the motion level, a two-dimensional continuous region is detected, and a large region is further extracted, and the large region is extracted based on at least one of the area ratio, shape, and position in the large region. Is determined as a shielding part, the blur level is calculated in pixel units or block units according to the frequency analysis result, and the calculated value of the blur level is binarized, and then a two-dimensional continuous area is detected. The large area is extracted, and the large area is determined as the shielding part based on at least one of the area ratio, shape, and position in the large area. Therefore, it is possible to accurately identify both a shield with movement and a non-focused shield from the video signal.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 shows a shield image identification apparatus according to the present invention. The shielding object image identification device includes a 2D-DCT (two-dimensional discrete cosine transform) circuit 1, a motion shielding object determination system 2, a blur shielding object determination system 3, a determination result integration unit 4, and a detection frame enlargement unit 5. .

  An input video signal is supplied to the 2D-DCT circuit 1. It is assumed that the input video signal indicates a video with a resolution of 720 × 480 pixels. The 2D-DCT circuit 1 performs DCT conversion of one frame of the input video signal with a block of 16 × 16 pixels. When the input video signal is an interlace video signal, two fields are set as one frame. In the case of a progressive video signal, the odd field of one frame and the even field of the next frame are combined into one frame. The combination of the odd and even fields may be reversed, or may be combined over several frames instead of from consecutive frames. If the time difference between the fields to be combined increases, it will react even to slow moving objects, but if the moving distance within that time exceeds the size of the processing block, it will not be able to be detected, so usually within a few frames is desirable . Further, the odd and even fields of adjacent frames may be combined regardless of the progressive interlace of the input signal. This makes it possible to detect whether the input signal is progressive or interlaced. Furthermore, odd fields and even fields may be combined.

  A motion shielding object determination system 2 and a blur shielding object determination system 3 are connected to the output of the 2D-DCT circuit 1. The movement shielding object determination system 2 includes a motion level calculation unit 21, a binarization unit 22, a labeling unit 23, a large region extraction unit 24, an area ratio determination unit 25, a shape determination unit 26, a position determination unit 27, and a label number determination unit. 28 and a general attribute determination unit 29.

The motion level calculator 21 calculates a motion level indicating the magnitude of motion in order to detect a motion video block according to the DCT coefficient of the DCT result. As shown in FIG. 2A, when a block having m pixels in the horizontal direction and n pixels in the vertical direction is DCTed, the spatial frequency in the horizontal direction u and the vertical direction are obtained as shown in FIG. The motion level is calculated by the following equation in the transformation space coordinate composed of the spatial frequency of v. The DCT coefficients are X _{0,0 to} X _{N−1, N−1} , and N = 16.

  The binarization unit 22 is connected to the output of the motion level calculation unit 21 and binarizes the motion level calculated by the motion level calculation unit 21 with a predetermined threshold.

  The labeling unit 23 is connected to the output of the binarizing unit 22 and performs a labeling process on the binarized motion level. In the labeling process, blocks that are adjacent in one of the four directions (upper, lower, left, and right) and the oblique four directions with the same binarized motion level are defined as one area (or group), and the same label is assigned to the blocks in the same area. Is to add and classify. For example, if the binarized motion levels are adjacent to each other, the block becomes one area (group), and the same label is added to each block in the area. The labeling unit 23 outputs the region information to the large region extraction unit 24 and outputs the number of labels added, that is, the number of labels to the label number determination unit 28.

  As a result of the labeling process, the large area extraction unit 24 extracts a large area whose area is greater than or equal to a predetermined area. The predetermined area is, for example, 1/30 of the area of one frame. The large area extraction unit 24 may extract the area of the maximum area as a large area. An area ratio determination unit 25, a shape determination unit 26, and a position determination unit 27 are connected to the output of the large region extraction unit 24.

  The area ratio determination unit 25 detects a ratio of a circumscribed rectangular area and a region area of a large area having an area equal to or larger than a predetermined area in one frame for each frame = circumscribed rectangular area / region area, and the ratio is a first predetermined value. When the value is within a range (for example, 0.5 or more and less than 1.0), it is determined that the area ratio determination condition of the frame is satisfied. The shape determination unit 26 determines whether or not a frame shape determination condition is satisfied according to the shape of a large region having an area equal to or larger than a predetermined area in one frame. It is determined that the shape determination condition is satisfied when the horizontal width / vertical width of the large region is a value in a second predetermined range (for example, 0.7 to 7). The position determination unit 27 determines whether or not the frame position determination condition is satisfied according to the position of the large area in one frame. It is determined that the position determination condition is satisfied when the vertical coordinate of the center of gravity of the large area is in the lower 2/3 portion of the entire screen.

  The label number determination unit 28 determines whether or not the frame is a shielding object detection target according to the number of labels added by the labeling unit 23. When the number of labels is equal to or less than the predetermined number (10), it is determined that the frame is a shielding object detection target.

  The general attribute determination unit 29 is connected to the outputs of the area ratio determination unit 25, the shape determination unit 26, the position determination unit 27, and the label number determination unit 28, and the area ratio determination result by the area ratio determination unit 25 for each frame, The presence of the shielding object is determined according to the shape determination result by the shape determination unit 26, the position determination result by the position determination unit 27, and the label number determination result by the label number determination unit 28. That is, if the label number determination unit 28 determines that the object is to be detected and the area ratio determination condition, the shape determination condition, and the position determination condition are satisfied, the general attribute determination unit 29 indicates that there is a shield. Determined.

  The blur shielding object determination system 3 includes a blur level calculation unit 31, a binarization unit 32, a labeling unit 33, a large region extraction unit 34, an area ratio determination unit 35, a shape determination unit 36, a position determination unit 37, and a label number determination unit. 38 and a general attribute determination unit 39.

  The blur level calculation unit 31 calculates a blur level indicating the degree of out-of-focus in order to detect an out-of-focus block according to the DCT coefficient of the DCT conversion result.

  As shown in FIG. 3A, when a frame having the number of pixels in the horizontal direction is m and the number of pixels in the vertical direction is n is subjected to DCT conversion, as shown in FIG. The DCT coefficient power in the low frequency is expressed in the transformation space coordinate consisting of the spatial frequency in the direction v.

Then, the blur level is calculated by the following equation.

  As can be seen from this calculation formula, the blur level is represented by the ratio of the low-frequency DCT coefficient power to the high-frequency DCT coefficient power other than the low-frequency DCT coefficient power. Therefore, the blur level becomes larger as the block is out of focus.

  The binarization unit 32, the labeling unit 33, the large region extraction unit 34, the area ratio determination unit 35, the shape determination unit 36, the position determination unit 37, the label number determination unit 38, and the comprehensive attribute determination unit 39 A binarization unit 22, a labeling unit 23, a large region extraction unit 24, an area ratio determination unit 25, a shape determination unit 26, a position determination unit 27, a label number determination unit 28, and a general attribute determination unit 29 of the object determination system 2 Are the same.

The binarization unit 32 is connected to the output of the blur level calculation unit 31 and binarizes the blur level calculated by the blur level calculation unit 31 with a predetermined threshold. The labeling unit 33 is connected to the output of the binarization unit 32 and performs a labeling process on the binarized blur level. As a result of the labeling process, the large area extraction unit 34 extracts a large area whose area is greater than or equal to a predetermined area. The area ratio determination unit 35 determines whether or not the area ratio determination condition is satisfied for each frame. The shape determination unit 36 determines whether or not the shape determination condition is satisfied for each frame. The position determination unit 37 determines whether or not the position determination condition is satisfied for each frame. The label number determination unit 38 determines whether or not the frame is a shielding object detection target according to the number of labels added by the labeling unit 33. The comprehensive attribute determination unit 39 determines the presence of the shielding object according to the determination results of the area ratio determination unit 35 , the shape determination unit 36 , the position determination unit 37 , and the label number determination unit 38 .

  The determination result integration unit 4 is connected to the outputs of the general attribute determination units 29 and 39. The determination result integration unit 4 determines that there is a shielding object in one frame when one of the general attribute determination units 29 and 39 determines the presence of the shielding object for one frame.

  The detection frame enlarging unit 5 determines the frames before and after the frame determined to have a shield as the shield presence frame.

  In the shield image identifying apparatus having such a configuration, first, a case where a person enters the source image of one frame of the input image signal from the right side so as to block the front of the subject as shown in FIG. 4A will be described. . For such a frame of video, the motion level calculation unit 21 calculates the motion level for each block. FIG. 4B shows the calculated motion level binarized by the binarizer 22. The black part is the moving part. The binarized video is labeled by the labeling unit 23. In the labeling process, blocks adjacent to each other in any of the four vertical and horizontal four directions and the four oblique directions among the black blocks in FIG. 4B are defined as one region. FIG. 4C shows the result of the labeling process. In this example, the number of labels is 5 (Labels # 1 to # 5). When the labeling unit 23 detects a plurality of regions, a large region is extracted from the plurality of regions by the large region extraction unit 24. As shown in FIG. 4 (d) with respect to the result of the labeling process in FIG. 4 (c), the region corresponding to the moving person on the right side in FIG. 4 (a) is extracted as a large region. As shown in FIG. 4 (e), if the extracted area satisfies the determination conditions of the area ratio determination unit 25, the shape determination unit 26, the position determination unit 27, and the label number determination unit 28, In addition, it is detected as a moving obstacle. That is, the current frame is determined as a frame including a moving obstacle. Therefore, it is possible to identify an image including a person who crosses the front of the subject.

  When a person crosses in front of the camera as in this example, it suddenly comes in, so it moves at a relatively fast speed and is detected as a two-dimensional continuous region with movement. Since the object and the camera are crossed, the area of the two-dimensional continuous area is naturally large. Therefore, the “person crossing the front of the camera” is detected by the above processing. When passing behind the subject, the moving region is divided by the subject, so that it does not become a large continuous region. The same applies to objects that move far away. In addition, to prevent false detection, it is detected by “area attribute determination” when the area has the center of gravity at the top of the screen, extremely long area (= not human), and the center of gravity is at the center of the screen (= high possibility of subject) By setting a condition such as “no”, it is possible to accurately detect “when a person crosses the camera”. “Region attribute determination” can also include the dominant color and brightness of the region as a condition.

  When the camera moves, there may be a motion area across the entire screen, but this is also a “region attribute determination”, and if almost the entire screen is detected as one area, the threshold value is the number of areas. By setting a condition such as not detecting when there are the above (= small areas are scattered over the entire screen), it is possible to distinguish the movement by the camera work and the movement by the shielding object. The distinction from camera work may be performed by separately detecting camera work and referring to the result.

  When the video camera is panned, as shown in FIG. 5A, when the source video includes a large number of moving persons and objects, the calculated value of the motion level increases on the entire screen. The motion level binarized by the digitizer 22 is as shown in FIG. Further, FIG. 5C shows the result of the labeling process, and it can be seen that the number of detected regions (= number of labels, Label # 1 to # 50) is large. Even if the area of the maximum area is extracted from the large number of areas as shown in FIG. 5 (d), the number of labels is larger than the predetermined number, so the label number determination unit 28 determines that the large area is an object to be shielded. do not do. Therefore, the video when the video camera is panned is not determined as a frame including a moving obstacle as shown in FIG.

  On the other hand, as shown in FIG. 6 (a) in the source video of one frame of the input video signal, when the head of an out-of-focus person exists in the lower left part, the blur level calculation unit 31 sets the blur level for each block. Calculated. FIG. 6B shows a calculated value of the blur level binarized by the binarizer 32. The black part is the blurred part. The binarized video is labeled by the labeling unit 33. In the labeling process, blocks adjacent to each other in any of the four vertical and horizontal directions and the oblique four directions among the black blocks in FIG. 6B are defined as one region. FIG. 6C shows the result of the labeling process. In this example, the number of labels is 8 (Labels # 1 to # 8). A large area is extracted from the plurality of areas by the large area extracting unit 24 as shown in FIG. As shown in FIG. 6 (e), if the extracted region satisfies the determination conditions of the area ratio determination unit 35, the shape determination unit 36, the position determination unit 37, and the label number determination unit 38, In addition, it is detected as a blur shielding object. That is, the current frame is determined to be a frame including a blur shielding object. Therefore, it is possible to identify an image in which a person's head blocks the subject.

  When the head enters the lower part of the screen as in this example, the “head” in that case is a relatively large two-dimensional continuous area that is out of focus and is thus detected by the above processing. . When blur level detection is performed by frequency analysis, flat areas such as the sky may be erroneously detected as blur areas, but areas with a center of gravity at the top of the screen in `` area attribute determination '', extremely horizontally long areas By setting conditions such as not detecting (= not human), it is possible to accurately detect “when the head enters”. “Region attribute determination” can also include the dominant color and brightness of the region as a condition.

  Also, when the camera moves, the entire screen may be blurred, but this is also “area attribute determination”, and when almost the entire screen is detected as one area, the number of areas exceeds a certain threshold (= By setting a condition such that detection is not performed when small areas are scattered over the entire screen, blur due to camera work and blur due to an obstacle can be distinguished. The distinction from camera work may be performed by separately detecting camera work and referring to the result.

  When the frame in which the shield is present is determined in this way, the determined frame and the frames before and after the determined frame are determined by the detection frame enlargement unit 5 as the shield presence frame.

  In the above embodiment, the video signal is frequency-analyzed by the 2D-DCT circuit 1 and the motion level and blur level are calculated using the result of the frequency analysis. However, as shown in FIG. The signal may be directly supplied to the motion level calculation unit 21 and the blur level calculation unit 31, and the motion level and the blur level may be calculated by different methods according to the video signal. That is, the motion level calculation method by the motion level calculation unit 21 and the blur level calculation method by the blur level calculation unit 31 are not limited to the above-described embodiments.

  In the above-described embodiment, the motion shielding object determination system 2 and the blur shielding object determination system 3 are provided, and the determination result integration unit 4 takes the logical sum of the determination results. And only one of the blur shielding object determination system 3 may be provided. That is, as shown in FIG. 8, it is good also as an obstruction | occlusion image | video identification apparatus provided only with the structure of said motion shielding object determination system 2, or only the structure of said blur shielding object determination system 3 as shown in FIG. It is good also as a shield image identification device provided.

  Further, the blur shielding object determination system is configured such that the output signal of the detection frame enlargement unit 5 is input to the general attribute determination unit 39 as shown in FIG. The blur shielding object determination may be continued only when it is detected only within the detected peripheral block or within a predetermined time after the shielding object is detected by the motion shielding object determination system. By doing so, it is possible to accurately recognize a shielding object that has crossed in front of the subject and stayed there, and to avoid misrecognizing another blurred region as a shielding object.

  In the above-described embodiment, if all the determination conditions of the area ratio determination unit 25 (35), the shape determination unit 26 (36), and the position determination unit 27 (37) are not satisfied, the determination as a shielding object is determined. However, the large area may be determined as the shielding part if at least one of the determination conditions is satisfied. Further, the label number determination unit 28 (38) may not determine the number of labels.

  In the above-described embodiment, the video signal is DCTed by the 2D-DCT circuit 1. However, instead of DCT, DFT (Discrete Fourier Transform) may be used as the conversion method.

  Furthermore, in the above-described embodiment, the video signal is processed in units of blocks for each frame in order to determine the shielding object, but the video signal may be processed in units of pixels for each frame. Further, the block is not limited to 16 pixels × 16 pixels, and the number of pixels such as 8 pixels × 8 pixels may be used.

  Further, the motion level calculation method and the blur level calculation method are not limited to the above-described embodiments. The motion level may be calculated from a motion vector by block matching as used in MPEG encoding, or may be obtained from an optical flow. The blur level may be obtained from the contrast of the block unit.

  The motion shielding object determination system 2 and the blur shielding object determination system 3 described above may be configured individually or by a single computer. That is, a computer-readable program for executing a process of each part of the motion shielding object determination system 2 and a computer-readable program for executing a process of each part of the blur shielding object determination system 3 are stored in a memory. When executed by the computer, the function of each part of the motion shielding object judgment system 2 and the blur shielding object judgment system 3 is achieved.

  As described above, according to the present invention, it is possible to accurately identify both a moving object and a non-focused object in the image from the image signal. Can be easily selected.

Claims (21)

A shield image identification device for identifying a shield in a video,
A motion level calculating means for calculating a motion level for each frame of a video signal in units of pixels or blocks;
Binarization means for binarizing the calculated value of the movement level;
Two-dimensional continuous area detecting means for detecting adjacent areas adjacent to each other between pixels or blocks in which the output value of the binarizing means takes the same value for each frame;
A large area extracting means for extracting, as a large area, an area having a predetermined area or more among the areas detected by the two-dimensional continuous area detecting means;
An area ratio determination unit that determines that an area ratio determination condition is satisfied when a ratio between a circumscribed rectangular area of the large region and a region area is a value in a first predetermined range;
A shape determination unit that determines that a shape determination condition is satisfied when a ratio of a horizontal width to a vertical width of the large region is a value in a second predetermined range;
A position determination unit that determines that the position determination condition is satisfied when the vertical coordinate of the center of gravity of the large area is in a predetermined lower part of the entire screen;
Shielding means comprising: shielding object determination means for determining the large area as a shielding object part when at least one of the area ratio determination condition, the shape determination condition, and the position determination condition is satisfied Object image identification device. The motion level calculation means has frequency analysis means for analyzing the frequency of the video signal for each frame in units of pixels or in units of blocks including a plurality of pixels . 2. The shielding object image identification apparatus according to claim 1, wherein a ratio between the power of the highest frequency component and the sum of the powers of all components excluding the DC component is set as the value of the motion level. 3. The shielding object image identification apparatus according to claim 2 , wherein the frequency analysis means performs DCT (two-dimensional discrete cosine transform) or DFT (discrete Fourier transform) on a block unit basis of a plurality of pixels.   The shielding image identifying apparatus according to claim 1, wherein the two-dimensional continuous area detecting unit performs a labeling process on the output value of the binarizing unit for each frame. Before SL includes a count determination unit determines whether the number of regions in the 1 frame detected is less than a predetermined number by two-dimensional continuous region detecting means,
2. The shielding object judging means, wherein when the number judging unit judges that the number of areas in one frame is equal to or less than a predetermined number, the large area is judged as a shielding part. The shielding object image identification device described.   And a detection frame enlarging unit that determines a frame in which the large region is determined to be a shield part by the determination unit and a predetermined frame section before and after the frame as a section where the shield part exists. The shielding object image identification device according to claim 1. A shield image identification device for identifying a shield in a video,
A frequency analysis means for analyzing the frequency of the video signal in units of pixels or in units of blocks composed of a plurality of pixels for each frame;
A blur level calculation unit that uses a ratio between a low frequency component power and a high frequency component power other than that in a pixel unit or a block unit according to a frequency analysis result by the frequency analysis unit as a calculated value of a blur level of an image. When,
Binarization means for binarizing the calculated value of the blur level;
Two-dimensional continuous area detecting means for detecting adjacent areas adjacent to each other between pixels or blocks in which the output value of the binarizing means takes the same value for each frame;
A large area extracting means for extracting, as a large area, an area having a predetermined area or more among the areas detected by the two-dimensional continuous area detecting means;
The area ratio in the large region extracted by the large area extracting means, the shape and determining means and, shield image identification, characterized in that it comprises a its large area as a shield portion based at least on the first position apparatus. The shielding image identifying apparatus according to claim 7 , wherein the frequency analysis unit performs DCT or DFT in units of blocks including a plurality of pixels. The shielding image identifying apparatus according to claim 7, wherein the two-dimensional continuous area detecting unit performs a labeling process on the output value of the binarizing unit for each frame. The determination means includes an area ratio determination unit that determines that an area ratio determination condition is satisfied when a ratio of a circumscribed rectangular area of the large region to a region area is a value in a first predetermined range; and a width of the large region A shape determination unit that determines that a shape determination condition is satisfied when the ratio to the vertical width is a value in a second predetermined range, and a position when the vertical coordinate of the center of gravity of the large area is in a predetermined lower part of the entire screen A shielding object image identification apparatus according to claim 7 , further comprising a position determination unit that determines that the determination condition is satisfied. Before SL includes a count determination unit determines whether the number of regions in the 1 frame detected is less than a predetermined number by two-dimensional continuous region detecting means,
The determination means according to claim 7, wherein said large area determining the shield portion when the number of areas in one frame by the number determining unit determines not more than a predetermined number Shield image identification device. And a detection frame enlarging unit that determines a frame in which the large region is determined to be a shield part by the determination unit and a predetermined frame section before and after the frame as a section where the shield part exists. The shielding object image identification device according to claim 7 . A shield image identification device for identifying a shield in a video,
A motion level calculating means for calculating a motion level for each frame of a video signal in units of pixels or blocks;
First binarization means for binarizing the calculated value of the movement level;
First two-dimensional continuous area detection means for detecting continuous areas adjacent between pixels or blocks in which the output value of the first binarization means has the same value for each frame;
First large area extraction means for extracting an area having a predetermined area or more as a large area among areas detected by the first two-dimensional continuous area detection means;
A first determination means for determining the area ratio in the large regions extracted, and the large area based on at least the shape and position as a shield portions by the first large-area extracting means,
A blur level calculation means for calculating a blur level of a video in pixel units or block units;
Second binarizing means for binarizing the calculated value of the blur level;
Second two-dimensional continuous area detecting means for detecting continuous areas adjacent between pixels or blocks in which the output value of the second binarizing means has the same value for each frame;
A second large area extracting means for extracting an area having a predetermined area or more as a large area among the areas detected by the second two-dimensional continuous area detecting means;
Second determining means for determining the large area as a shielding part based on at least one of the area ratio, shape and position in the large area extracted by the second large area extracting means ;
The blur level calculation means includes frequency analysis means for analyzing the frequency of the video signal in units of pixels or in blocks of a plurality of pixels for each frame, and in units of pixels or blocks according to the frequency analysis result by the frequency analysis means And a ratio of a low frequency component power to a high frequency component power excluding the low frequency component power is used as the calculated blur level . 14. The shielding object image identification apparatus according to claim 13 , further comprising a determination result integration unit that obtains a final determination result based on the determination result of the first determination unit and the determination result of the second determination unit. 15. The shielding object image identification apparatus according to claim 14 , wherein the determination result integration unit takes a logical sum of the determination result of the first determination unit and the determination result of the second determination unit. And a detection frame enlarging unit that determines that the frame determined to be the shielding part by the determination result integrating unit and the predetermined frame sections before and after the frame are sections where the shielding part exists. The shielding object image identification device according to claim 14. The second determination means, the detection frame expanding means only expanded detection frame interval, the shield image identification apparatus according to claim 16, wherein the determining a shield portion. A method of identifying a shielding object image that identifies a shielding object in an image,
A motion level calculating step for calculating a motion level for each frame of a video signal in units of pixels or blocks;
A binarization step for binarizing the calculated value of the movement level;
A two-dimensional continuous region detection step for detecting a continuous region adjacent between pixels or blocks in which the values binarized in the binarization step have the same value for each frame;
A large region extracting step of extracting a region having a predetermined area or more as a large region among the regions detected in the two-dimensional continuous region detecting step;
An area ratio determining step for determining that an area ratio determination condition is satisfied when a ratio of a circumscribed rectangular area of the large region and a region area is a value in a first predetermined range;
A shape determination step for determining that a shape determination condition is satisfied when the ratio of the horizontal width to the vertical width of the large region is a value in a second predetermined range;
A position determination step for determining that the position determination condition is satisfied when the vertical coordinate of the center of gravity of the large area is in a predetermined lower portion of the entire screen;
A shielding object determining step of determining the large region as a shielding object part when at least one of the area ratio determination condition, the shape determination condition, and the position determination condition is satisfied. Object image identification method. A method of identifying a shielding object image that identifies a shielding object in an image,
A frequency analysis step for analyzing the frequency of the video signal in units of pixels or in units of blocks including a plurality of pixels for each frame;
A blur level calculation step in which a ratio of a low frequency component power to a high frequency component power other than that in a pixel unit or a block unit according to a frequency analysis result in the frequency analysis step is used as a calculation value of a blur level of an image. When,
A binarization step for binarizing the calculated value of the blur level;
A two-dimensional continuous region detection step for detecting a continuous region adjacent between pixels or blocks in which the values binarized in the binarization step have the same value for each frame;
A large region extracting step of extracting a region having a predetermined area or more as a large region among the regions detected in the two-dimensional continuous region detecting step;
The area ratio in the large region extracted in large area extracting step, the shape and determining steps and, shield image identification, characterized in that it comprises a its large area as a shield portion based at least on the first position Method. A computer-readable program for executing a shielding image identification method for identifying a shielding object in an image,
A motion level calculating step for calculating a motion level for each frame of a video signal in units of pixels or blocks;
A binarization step for binarizing the calculated value of the movement level;
A two-dimensional continuous region detection step for detecting a continuous region adjacent between pixels or blocks in which the values binarized in the binarization step have the same value for each frame;
A large region extracting step of extracting a region having a predetermined area or more as a large region among the regions extracted in the two-dimensional continuous region detecting step;
An area ratio determining step for determining that an area ratio determination condition is satisfied when a ratio of a circumscribed rectangular area of the large region and a region area is a value in a first predetermined range;
A shape determination step for determining that a shape determination condition is satisfied when the ratio of the horizontal width to the vertical width of the large region is a value in a second predetermined range;
A position determination step for determining that the position determination condition is satisfied when the vertical coordinate of the center of gravity of the large area is in a predetermined lower portion of the entire screen;
And a shielding object determining step for determining the large area as a shielding object part when at least one of the area ratio determination condition, the shape determination condition, and the position determination condition is satisfied. . A computer-readable program for executing a shielding image identification method for identifying a shielding object in an image,
A frequency analysis step for analyzing the frequency of the video signal in units of pixels or in units of blocks including a plurality of pixels for each frame;
A blur level calculation step in which a ratio of a low frequency component power to a high frequency component power other than that in a pixel unit or a block unit according to a frequency analysis result in the frequency analysis step is used as a calculation value of a blur level of an image. When,
A binarization step for binarizing the calculated value of the blur level;
A two-dimensional continuous region detection step for detecting a continuous region adjacent between pixels or blocks in which the values binarized in the binarization step have the same value for each frame;
A large region extracting step of extracting a region having a predetermined area or more as a large region among the regions detected in the two-dimensional continuous region detecting step;
The large region extraction area ratio in the large region extracted in step, the shape and position program characterized by comprising a determining step the large area as a shield portion based at least 1, of.

Images