JPH07129777A - Traveling body detector - Google Patents

Abstract

PURPOSE:To accurately detect a traveling body on a screen. CONSTITUTION:A device 14 computes a correlation value from the image (a) at a prescribed time and the image (b) at the time unit prior to a prescribed time point, and devices 16, 18 detect the distribution of a motion vector on a screen and that of reliability for the motion vector, respectively. A device 22 extracts the motion vector with reliability higher than a first threshold value set in advance based on the distribution of reliability as an initial motion vector, and a device 20 outputs the motion vector with reliability higher than a second threshold value (< first threshold value) by eliminating the motion vector with reliability lower than the second threshold value set in advance based on the distribution of reliability. A device 24 extracts the motion vector equal or similar to the initial motion vector from the motion vectors outputted from the device 20 by retrieving an area in the periphery of a position equipped with the initial motion vector, and finds the area by integrating the position equipped with an extracted motion vector. A device 26 detects and outputs a traveling body area from a found area.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving object detecting device, and more particularly to a moving object detecting device capable of accurately detecting a plurality of moving objects on an image.

[0002]

2. Description of the Related Art In an environment recognition technique using images, there is a method of using a sequence of moving images (images using temporally continuous frames) to detect moving objects in moving images. The method of detection is to calculate the difference between consecutive frames, the speed of the entire image (motion vector)
There is a method to obtain the distribution.

As a technique using motion vectors, a motion vector distribution of the entire screen is obtained, and when motion vectors having similar directions and sizes are spatially continuous on an image, they are integrated. A technique is known that attempts to recognize the environment as one moving object.

This technique uses two images when there is a movement as shown by an arrow in FIG. 3C between two images which are temporally continuous as shown in FIGS. 3A and 3B. It is intended to detect the area of a moving object by integrating similar motion vectors between the images.

As an apparatus using the technique of integrating motion vectors distributed on an image, as shown in FIG. 1, a frame memory 1-1, a frame memory 1-2, a motion vector detecting device 2, and a region. There is a device composed of the integrated device 4, and this device is disclosed in, for example, JP-A-2-118884, JP-A-3-24675, and JP-A-4-1482.
No. 83 publication and the like.

However, in general, when a motion vector is calculated on an actual image, a correct motion vector may not be detected depending on the image density pattern at each position, as shown in FIG. You cannot get the correct motion vector with. Therefore, there is a problem in that the moving object region cannot be correctly extracted due to the influence of an incorrect motion vector simply by integrating the obtained motion vectors. Note that, in FIG. 3D, an arrow indicates a motion vector, and · indicates a motion vector of size 0.

On the other hand, as an apparatus using a method of calculating the motion vector distribution of the entire screen and the reliability of the motion vector at the same time, as shown in FIG.
There is a device including a frame memory 1-2, a motion vector detection device 2, and a reliability detection device 3. This device is disclosed in, for example, Japanese Patent Laid-Open No. 3-48372 and Japanese Patent Laid-Open No. 4-262.
No. 83 publication and the like.

This reliability represents an index of the correctness of each motion vector, but since the prior art does not consider the positional relationship between adjacent motion vectors, it is necessary to understand the entire image. It is inappropriate.

Therefore, in the area integration of the image by the integration of the motion vectors, the reliability of the obtained motion vector is compared with a certain threshold value by using the reliability, and the reliability exceeds the threshold value. A method is conceivable in which area integration is performed using only motion vectors. However, since the reliability distributions of the correct motion vector and the incorrect motion vector are not necessarily separated by one threshold value, if the threshold value is set high so that the incorrect motion vector is hardly used for the region integration, As shown in FIG. 4A, only a small number of motion vectors exceed the threshold value so that the adjacency relationship cannot be examined. On the other hand, when the threshold value is set low so that the correct motion vector is not truncated, as shown in FIG. 4B, the number of erroneous motion vectors also exceeds the threshold value, and the region integration result is shown in FIG. ), A large number of erroneous regions are detected, and the effect of introducing reliability cannot be obtained.

The present invention has been made in order to solve the above problems, and is based on a motion vector distribution having a large number of motion vectors obtained on the screen and having a high reliability and a high reliability to some extent. An object of the present invention is to provide a moving object detection device capable of accurately detecting a moving object on an image in consideration of the positional relationship between motion vectors.

[0011]

In order to achieve the above object, the invention of claim 1 is, as shown in FIG. 5, a motion vector on a screen from an image at a predetermined time and an image a unit time before the predetermined time. And a motion vector / reliability detecting means A for detecting a distribution of reliability for the motion vector, and a motion vector whose reliability is higher than a first threshold value set in advance based on the detected distribution of reliability. Initial motion vector extracting means B for extracting as an initial motion vector, and removing a motion vector whose reliability is lower than a second threshold value set in advance to a value smaller than the first threshold value based on the detected reliability distribution. And a vector removing unit C that outputs a motion vector whose reliability is equal to or higher than a second threshold value, and a region around the position having the initial motion vector are searched to find the same as the initial motion vector. The similar to the motion vector extracted from the motion vector output from the vector removal means, extracting a region integrating unit D to determine the area by integrating the position with the motion vector has, which is constituted contains.

[0012]

The operation of the invention of claim 1 will be described. The detected motion vector on the screen usually contains noise, error, etc., but it is possible to detect the reliability indicating the reliability of this motion vector. Therefore, in the motion vector / reliability detecting means A of the invention of claim 1, the distribution of the motion vector on the screen and the distribution of the reliability for this motion vector are calculated from the image at the predetermined time and the image unit time before the predetermined time. To detect.

As a method of detecting the distribution of motion vectors, there is a method of dividing an image into a large number of small areas and calculating a correspondence between a plurality of images by a correlation calculation for each small area. In this method, in the process of calculating the correspondence, the maximum correlation value, the slope of the correlation value distribution near the maximum correlation value, the peak difference between the maximum correlation value and the correlation value having the second highest value, or the ratio, etc. The reliability of this motion vector can be evaluated by calculation.

The initial motion vector extraction means B extracts a motion vector whose reliability is higher than a preset first threshold value based on the reliability distribution detected by the motion vector / reliability detection means A as an initial motion vector. To do. The vector removing means C is a motion vector whose reliability is lower than a second threshold value set in advance to a value smaller than the first threshold value based on the distribution of reliability detected by the motion vector / reliability detecting means A, that is, a region. Unreliable motion vectors that are inappropriate for use in integration are removed, and motion vectors whose reliability is equal to or higher than the second threshold value are output.

The area integrating means D extracts a motion vector identical or similar to the initial motion vector from the motion vectors output by the vector removing means by searching the area around the position having the initial motion vector. Then, the positions provided with the extracted motion vectors are integrated to obtain a region. In the area integration, the area where the object exists in the image exists as a lump area, and the motion vectors detected inside one object are at least their directions, that is, the direction , Or based on the knowledge that the directions and sizes are the same or similar. That is, if there is a motion vector that is the same as or similar to a highly reliable motion vector around a highly reliable motion vector, then it is determined that a motion vector that is determined to be somewhat unreliable is correct. They are integrated as belonging to the same area. Whether the motion vectors are the same or similar can be determined by determining the direction of the motion vector, or the direction and magnitude.

As described above, according to the first aspect of the present invention, the periphery of the highly reliable initial motion vector is searched, and the highly reliable initial motion vector distributed near the highly reliable initial motion vector is obtained. Since region integration is performed by including motion vectors that are the same or similar and have relatively low reliability, a moving object region that contains highly reliable motion vectors is selected and detected from motion vector distributions that include noise and errors. be able to. On the other hand, in the process of detecting the motion vector and the reliability, it is possible to add a motion vector having a moderate reliability, which is considered to represent a correct vector to the integration, although the reliability is not sufficiently high. Therefore, it is less likely that the shape of the moving object region is largely missing or a large number of erroneous regions are detected, and the region in which the moving object exists can be correctly estimated as compared with the conventional method.

[0017]

Other Description of the Invention According to the invention of claim 1, if the camera for detecting an image is stationary, it is relatively easy to separate the stationary object region and the moving object region on the screen, but the camera itself. When the object is moving, a motion vector due to the camera motion is generated in the area occupied by the stationary object, so that it becomes difficult to separate it.

Therefore, a first invention of another invention is, as shown in FIG. 6, a translation speed detecting means E for detecting a translation speed of the camera according to the invention of claim 1, and a camera based on the detected translation speed. Of the motion vector detected by the motion vector / reliability detection unit A and the motion vector generated by the translational motion of the camera are compared to determine the movement state. And a moving state determining unit G for making a determination, and using the detection result of the moving state determining unit G in the region integrating unit D, a moving object region that is moving independently of the translational motion of the camera is a stationary object region. It is to separate and integrate areas.

According to the first invention, when the camera is moving in translation, the stationary object on the screen is in a direction corresponding to the direction of the translation speed of the camera, and a motion vector of a magnitude corresponding to the magnitude of the translation speed. Is used to detect a motion vector generated when a stationary object exists, that is, a motion vector generated by the translational motion of the camera, and this motion vector and the motion vector / reliability detecting means A detect the motion vector. A moving object area that is moving independently of the translational motion of the camera by detecting the motion vector on the object that is originally stationary by determining the moving state by comparing it with the motion vector Is separated from the area of the stationary object constituted by the motion vector generated by the translational motion of the camera.

As a result, in addition to the case of the first aspect of the present invention, even when the camera is moving in translation, a moving object moving independently of the translation of the camera is moved by the translation of the camera as a motion vector. Can be detected separately from the area where

In the first invention, the translational velocity of the camera may be detected by detecting the translational motion component from the motion vector detected by the motion vector / reliability detecting means A. It may be detected by the sensor. Further, the moving object area may be integrated by the remaining motion vector obtained by removing the motion vector generated by the translational motion of the camera from the motion vector detected by the motion vector / reliability detection means A. By labeling the motion vector generated by the translational motion of the camera in the motion vector detected by the means A, the motion vector of the stationary object and the motion vector of the moving object are distinguished from each other, and the area of the stationary object and the movement of the stationary object are moved. The object area may be separated and integrated.

As shown in FIG. 7, a second aspect of the present invention is the invention of claim 1 in which the rotational speed detecting means H for detecting the rotational speed of the camera and the rotational movement of the camera from the detected rotational speed. The motion vector detecting means I for detecting the motion vector and the motion vector detected by the motion vector / reliability detecting means A are used so as to eliminate the component due to the rotary motion of the camera by using the motion vector generated by the rotary motion of the camera. A correction means J for correction is further provided, and the initial motion vector extraction means B, the vector removal means C, and the area integration means D perform area integration using the corrected motion vectors.

According to the present invention, with the above configuration, it is detected that a constant rotational motion component is added to the motion vector of the entire image, and if a constant rotational motion component is detected, it is corrected. There is.

As a result, in addition to the case of the first aspect of the invention, even when the camera makes a rotational motion, the rotational motion component of the camera added to each motion vector is corrected thereby, and the area of the moving object can be correctly detected.

Note that the rotation speed of the camera is the motion vector
The rotational motion component may be detected from the vertical component and the horizontal component of the motion vector detected by the reliability detection unit A, or may be detected by another sensor attached to the camera or the like. Further, the correction means J is provided in the subsequent stage of the initial motion vector extraction means B and the vector removal means C,
Each of the extracted initial motion vector and the motion vector output from the vector removing means C may be corrected so as to eliminate the component due to the rotational motion of the camera. In this case, since it is not necessary to correct the motion vector with low reliability removed by the vector removing means C, the process becomes quick.

A third invention is obtained by adding the first invention and the second invention to the invention of claim 1, and as shown in FIG. 8, the translation speed for detecting the translation speed of the camera. A detection means E, a motion vector detection means F for detecting a motion vector generated by the translational motion of the camera from the detected translational speed, and a motion vector detected by the motion vector / reliability detection means A and the translational motion of the camera. Moving state determination means G for detecting the motion vector of the stationary object from the generated motion vector, rotation speed detection means H for detecting the rotation speed of the camera, and motion vector generated by the rotational movement of the camera from the detected rotation speed. Vector detecting means I for detecting
And a correction means J for correcting the motion vector detected by the motion vector / reliability detection means A so that the component caused by the rotation motion of the camera is eliminated by using the motion vector generated by the rotation motion of the camera. , Initial motion vector extraction means B, vector removal means C and area integration means D
In the above method, the moving object area that is moving independently of the translational movement of the camera is separated from the stationary object area by using the motion vector in which the effect of the rotational movement of the camera is corrected, and the areas are integrated.

According to the third aspect of the present invention, in addition to the case of the first aspect of the invention, even if the camera is in translation, the movement is independent of the translation of the camera. The object can be detected, and further, even when the camera makes a rotational movement, the area of the moving object can be correctly detected, and thus the moving object area can be detected more accurately.
The effect is obtained.

[0028]

Embodiments of the present invention will now be described in detail with reference to the drawings. The first embodiment is an application of the invention of claim 1.

As shown in FIG. 9, the moving object detecting apparatus according to the first embodiment has a frame memory 10 for storing data of an original image, that is, an image a at a certain time, and a unit time before the image a (one time). The frame memory 12 for storing the data of the image b of the previous) is provided. Frame memory 1
Reference numerals 0 and 12 denote a correlation value detecting device 1 for calculating a correlation value.
4 is connected. The correlation value detection device 14 uses the image a,
motion vector detecting device 1 for detecting motion vector between b
6 and the reliability detecting device 18 for detecting a value (reliability) indicating the reliability of each motion vector detected by the motion vector detecting device 16.

The motion vector detecting device 16 and the reliability detecting device 18 respectively compare the reliability with a preset first threshold value and have a high reliability among the motion vectors detected by the motion vector detecting device 16. A highly reliable motion vector determination device 22 that detects and outputs a motion vector (corresponding to the initial motion vector) having Among the motion vectors detected by the motion vector detection device 16, a motion vector having a very low reliability is determined, and a motion vector that is not a target of the region integration is removed, so that the motion vector having a high reliability and the reliability It is connected to the low reliability motion vector determination device 20 which outputs an intermediate motion vector. The low-reliability motion vector determination device 20 and the high-reliability motion vector determination device 22 include a motion vector output from the low-reliability motion vector determination device 20 and a high-reliability motion vector determination device 22.
To a moving object area detection device 26 that detects a moving object area from the area group detected by the area integration device 24 via the area integration device 24 that performs area integration based on the information of the high-reliability motion vector detected by It is connected.

The operation of the moving object detecting apparatus of the first embodiment constructed as above will be described. The frame memory 10 and the frame memory 12 are input with two pieces of image data in which an image a at a certain time and an image b one time before are temporally continuous. The correlation value detecting device 14 calculates the correlation value using the area-based correlation method, and the motion vector detecting device 16 and the reliability detecting device 18 calculate the distribution of the motion vector from the correlation value calculated by the correlation value detecting device 14 and its distribution value. The distribution of the reliability of the motion vector is detected.

That is, as shown in FIG. 10 (b), the screen of the original image shown in FIG. 10 (a) is divided into a grid, and each grid section is a unit for detecting a motion vector and reliability. A small region is set, and each small region is set as a region of interest, and a correlation calculation is performed with respect to the image b one hour before, with the same position of the image a one hour later as the center, and its surroundings. Figure 1
R in the image b one hour before in 1 (a) indicates a small area in which a motion vector is detected, that is, a target area, and S is a search area for searching a small area having the highest correlation with the small area R. The size of each region can be selected to be, for example, 16 pixels × 16 pixels for S and 8 pixels × 8 pixels for R.

In this correlation calculation, the region of interest on the image a is displaced by (i, j) (where imin <i <imax, jmin <j <jmax) in the x and y directions of the image, respectively. The absolute value sum of the differences between the image gray value in the search area on the previous image b and the image gray value in the search area of the image a after displacement is detected as a correlation value. This correlation value C (i, j) is expressed by the following equation (1).

[0034]

[Equation 1]

However, (xc-ωx / 2) <x <(xc
+ Ωx / 2), (yc-ωy / 2) <y <(yc + ωy
/ 2). Where It (x, y), Is (x,
y) is a gray value at coordinates (x, y) of image a and image b, Is (x + i, y + j) is a gray value after displacement by (i, j) in the x direction and y direction, xc,
yc is the center coordinate of the attention area in which the motion vector is detected, and ωx and ωy are the sizes (pixels) of the attention area.

In the motion vector detecting device 16, the correlation value C
The displacement amount (i, j) that gives the minimum value of (i, j) is calculated, and the displacement amount (i, j) is used as the motion vector of the small region.

In the reliability detecting device 18, as shown in the following equation (2), the minimum value C (i, j) min of the correlation value and the correlation value C (i ', j') which is the next smallest value of the correlation value. ) Is calculated, and this difference is defined as the reliability R (i, j) of this motion vector.

R (i, j) = C (i ′, j ′) − C (i, j) min (2) The small area s in FIG. 11B has the highest correlation with the small area R. The small region, that is, the position where the expression (1) has the minimum value is shown.

Next, the high-reliability motion vector determination device 22 compares the reliability of each motion vector with a preset first threshold value, and determines a motion vector with a very high reliability,
That is, when performing region integration, a motion vector with extremely high reliability that serves as the core of each region is detected, and the detected motion vector is output.

In the low-reliability motion vector determination device 20, the reliability of each motion vector is compared with a preset second threshold value, and the reliability is very low, which is not suitable as a motion vector used for region integration. Area integration device 2
4 is removed from the input motion vector group and the remaining motion vectors are output.

FIG. 12C shows a small area of a motion vector having a very high reliability, a small area of a motion vector having a very low reliability, and a small area of a motion vector having an intermediate reliability.

In the area unifying device 24, the periphery of the motion vector having the extremely high reliability determined by the high reliability motion vector determining device 22 is continuously set with the motion vector having the extremely high reliability serving as the core of the region as a starting point. 4 small areas are searched for, and the motion vectors that are determined to have the same direction and size as or similar to the core motion vector are integrated to form the same region as the core motion vector. As a result, from the viewpoint of a motion vector with extremely high reliability, it is said that the motion vector is the same as that of the motion vector with extremely high reliability, although the reliability existing in the range continuously connected by four neighboring connections is not very high. Possible areas are merged into the same area.

The similarity of the direction and the magnitude of the motion vector is determined, for example, by taking the difference between the angle (direction) of the reference motion vector and the angle of the motion vector to be compared, and the difference is given in advance. When it is smaller than the threshold value of the difference, and regarding the magnitude, the magnitude of the motion that three-dimensionally represents the magnitude of the motion vector is taken into consideration in consideration of the presence or absence of camera motion and the position of the small area on the image. Then, the difference between the magnitude of the motion of the reference motion vector and the magnitude of the motion of the motion vector to be compared is calculated. When this difference is smaller than the threshold value of the difference given in advance, it is determined that they are similar to each other. Note that, for example, when a motion vector is calculated with an accuracy of 1 pixel for an image of a size of 512 × 436 by a 2/3 inch CCD camera having a focal length of 8 mm, the threshold value of the angle is 10 °.
The threshold value of the degree of movement can be set to 10 m. These thresholds can be reduced by improving the resolution and calculation accuracy of the image.

The manner of integration by the area integration device 24 will be described with reference to FIGS. 12, 13 and 17.

If the actual movement of the moving object is as shown by the arrow in FIG. 12A, the motion vector detecting device 16
As shown in FIG. 12 (b), the motion vector detected in step includes noise and error. The distribution of the reliability detected by the reliability detecting device 18 is as shown in FIG. 12C, and the motion vector having a very low reliability is removed by the low reliability motion vector determining device 20 and the reliability is Very high motion vector is highly reliable motion vector determination device 2
The small areas detected by 2 and having the motion vector with extremely high reliability are sparsely obtained as shown in FIG.

A small area in the vicinity of four small areas (small area of 1 shown in FIG. 17A) having the highly reliable motion vector thus obtained is searched (FIG. 17A), If there is a motion vector whose direction and magnitude are the same as or similar to those of the central motion vector, they are integrated as belonging to the same region (FIGS. 17B and 17C), and the integrated motion vectors are The motion vectors of the four small areas in the vicinity of (2 small areas shown in FIG. 17C) have the same direction and size as the highly reliable motion vector of the 1 small area shown in FIG. 17A. Or, if they are similar, they are integrated to form the same region (FIGS. 17D and 17E). This procedure is integrated into 4 neighborhoods of all motion vectors, and the 1 in FIG.
Repeat until there is no small area in.

When the small areas are integrated according to the above procedure, the areas are integrated as shown in FIG.
Finally, the areas are integrated as shown in FIG.
Since the motion vector with extremely low reliability is removed by the low-reliability motion vector determination means 20, the small area having the motion vector with extremely low reliability among the four neighborhoods is determined from the beginning. No need.

In the moving object area detecting device 26, as shown in FIG.
As shown in (c), the size of the motion vector of each integrated area is determined, and the area whose size is not 0 is output as the moving object area. At this time, if the direction of the motion vector is also determined at the same time, information about the moving direction of the moving object area can be output.

As described above, according to the present embodiment, even when there are a plurality of moving objects on the image, those candidate areas can be correctly divided as shown in FIG. 13 (c).

In the present embodiment, the difference between the minimum value of the correlation value and the next smallest value of the minimum value is used as the reliability, but in addition, the ratio of the minimum value of the correlation value to the next smallest value of the minimum value is used. Alternatively, the degree of protrusion from the average value of the correlation value distribution of the minimum correlation value may be used.

Further, in FIG. 10, the small areas for obtaining the motion vector are set so as not to overlap each other.
It is also possible to move the pixel by pixel and calculate the motion vector with high density. Although the direction and magnitude of the motion vector are used in the present embodiment as a criterion for determining whether two motion vectors are similar, only the direction of the motion vector may be used, or a component added by the motion of the camera. May be predicted or measured, and the difference with the component may be added to the comparison standard.

Next, a moving object detecting apparatus according to the second embodiment of the present invention will be described with reference to the block diagram shown in FIG. In FIG. 14, parts corresponding to those in FIG. 9 are designated by the same reference numerals, and description thereof will be omitted.

In this embodiment, a high reliability area detection device 28 is provided instead of the high reliability motion vector determination device 22 of the first embodiment.

In FIG. 14, the low-reliability motion vector determination device 20 compares the reliability of each motion vector with a preset second threshold, as in the first embodiment, and the reliability is very low. , A motion vector that is not suitable as a motion vector used for region integration is determined, removed from the motion vector group that is the input of the region integration device 24, and the remaining motion vectors are output together with the reliability. The area integration device 24 integrates the areas by scanning the array of motion vectors output from the low-reliability motion vector determination device 20, excluding the motion vectors with extremely low reliability. That is, scanning is performed from the left to the right of the screen and further from the top to the bottom, and the motion vector of the attention area is the same as or similar to the motion vector of the small area located on the left side (that is, one before) of the attention area. Whether or not the motion vector is the same or similar to the motion vector of the small region located above the attention region (that is, one scanning line before), and the small regions having the same or similar motion vector are the same region. The area is divided into the entire screen by performing area integration so that the area is included in. At this time, the reliability of each motion vector is also stored in association with each small area according to the label of the area belonging to that small area.

Further, when performing area integration, a chain structure list is added every time a new area is found, in addition to the array-shaped memory for storing area labels as shown in FIG. Dynamically add, merge with the area already formed in the above line, and delete the newly created area from the list after merging. .

For example, as shown in FIG. 18A, when v is determined to belong to the region 4 (state 1) and it is determined that the motion vector belonging to the region 3 is also similar (state 2), The area 4 is merged with the area 3 (state 3), and the area 4 is deleted from the list (state).

At this time, in the area label storage area as shown in FIG. 18A, the label at the past position may be rewritten at the time when the areas are merged or deleted.
Alternatively, the relationship of area 4 = area 3 may be stored and corrected at the end of area integration.

On the other hand, as shown in FIG. 18 (b), a list of areas up to the present time at which dynamic addition, merging, and deletion of motion vectors is performed simultaneously with area integration is created and updated appropriately.

The scanning direction is not limited to this, and may be left-right reversed or up-down reversed.

In this embodiment, since the area division is completed when one scan is performed on the small area array, the time required for integration can be shortened and the time can be predicted.

The high reliability area detecting device 28 selects only a highly reliable area as a moving object area from the area group integrated by the area integrating device 24. That is, the distribution state of the reliability of the motion vector forming each area group of the integrated area list integrated by the area integration device 24 is checked,
The reliability of each area is determined and a high reliability area is detected. For each area integrated by the area integration device 24, the average value or the maximum value of the reliability of the motion vector in the area is stored, and the high reliability area detection device 28 stores the average value of the reliability, etc. The size may be determined to detect the high reliability area.

As described above, the moving object detecting apparatus of the first and second embodiments integrates motion vectors detected from two temporally consecutive images and integrates similar motion vectors. In this case, a motion vector with extremely low reliability is excluded, and a motion vector with extremely high reliability and a motion vector that is in the vicinity of it and is the same as or similar to the motion vector are considered to have high reliability. Since region integration is performed using only vectors, it is possible to accurately detect a moving object on an image by considering the reliability of each motion vector and the positional relationship between motion vectors with a certain value or more. A third embodiment to which the third invention is applied will be described.

FIG. 15 is a block diagram of a moving object detecting apparatus according to the third embodiment of the present invention. In FIG. 15, parts corresponding to those in FIG. 9 are assigned the same reference numerals and explanations thereof will be omitted.

As shown in the figure, the moving object detecting apparatus of this embodiment detects the translational motion component of the camera from the entire motion vector detected by the motion vector detecting apparatus 16 and detects the translational motion velocity. From the translational velocity of the camera detected by the detection device 30 and the translational velocity detection device 30,
A motion vector detection device 32 for detecting a motion vector of a stationary object on the screen caused by the translational motion, that is, a motion vector due to the translational motion of the camera is provided. The motion vector detection device 32 uses the motion vector of the stationary object as the movement state information from the motion vector of the translational motion of the camera detected by the motion vector detection device 32 of the translational motion and the motion vector detected by the motion vector detection device 16. It is connected to the moving state determination device 34 for detecting.

Further, there is provided a rotary motion detecting device 36 for detecting the rotary motion component of the camera and the rotary motion speed from the whole motion vector detected by the motion vector detecting device 16. Is connected to a motion vector detection device 38 that detects a motion vector on the screen generated by the rotation motion, that is, a motion vector caused by the rotation motion of the camera, from the rotation speed of the camera obtained from the rotation motion detection device 36. . The motion vector detecting device 38 for detecting the motion vector due to the rotary motion uses the motion vector due to the rotary motion of the camera detected by the motion vector detecting device 38 to detect the rotary motion component of the motion vector detected by the motion vector detecting device 16. Is connected to the camera rotation motion correction device 40 that corrects so as to cancel.

In the moving object detecting apparatus according to the third embodiment having the above-mentioned structure, the translation velocity detecting apparatus 30 detects the camera motion from the distribution of the directional components of the whole moving vector detected by the moving vector detecting apparatus 16. The presence or absence of horizontal movement, vertical movement, forward movement or backward movement, that is, the presence or absence of translational movement is estimated, and the translation speed of the camera is detected. The translation speed may be detected by a sensor attached to a camera or the like. The translation vector motion vector detecting means 32 calculates the direction of the motion vector obtained when it is assumed that all objects on the screen are stationary. In the moving state determination device 34, the motion vector detecting device 16 compares the direction of the motion vector calculated by the motion vector detecting means 32 based on the translation velocity with the direction of the motion vector detected by the motion vector detecting device 16. It is determined whether the obtained motion vector is caused by the translational movement of the camera or an object moving regardless of the translational movement of the camera, and the movement state information is output.

In the area unifying device 24, the moving state is discriminated using the moving state information with respect to the motion vectors forming the integrated area group, and each of the obtained areas is a stationary object area or a moving object area. Then, a moving object that is moving independently of the translational movement of the camera is detected separately from the region caused by the camera movement.

Further, in the rotary motion detecting device 36, the distribution of the directional component of the motion vector detected by the motion vector detecting device 16 is obtained, and the rotary motion component of the camera constituted by the components shown in FIG. When it is constantly added to the motion vector of, it is detected. The rotation speed may be obtained from the camera rotation angle detected by a sensor attached to the camera or the like. The camera rotation motion correction device 40 corrects the motion vector so as to cancel the component added to the motion vector of the entire screen, using the motion vector of the rotation motion obtained by the motion vector detection device 38 of the rotation motion.

For example, when a constant vertical component, that is, the tilt component in FIG. 16 is added to the entire screen due to camera vibration or the like, a small vertical component appears in the entire area that should be stationary. Become. Considering the temporal continuity of positions where highly reliable regions exist and the restraint of positions, when an unnatural component is constantly present on the entire screen, the region that should be seen still, that is, the size is 0 When a motion vector having a constant magnitude and direction is observed at the position where the motion vector should be observed, the correction is applied to the motion vector of the entire screen to remove the vibration component.

[0070]

As described above, according to the present invention, based on the motion vector distribution having many motion vectors including the error obtained on the screen, the reliability of each motion vector is greater than or equal to a predetermined value. Since the regions are integrated in consideration of the positional relationship, there is an effect that a moving object on the image can be accurately detected.

[Brief description of drawings]

FIG. 1 is a block diagram of a conventional device that performs region integration using motion vectors.

FIG. 2 is a block diagram of a conventional apparatus that performs area integration using a motion vector and reliability.

3 (a), (b), (c), (d) of prior art 1
It is a diagram showing an image before time, an image at a predetermined time, actual movement, and distribution of motion vectors.

4 (a), (b), and (c) are diagrams showing a state of a prior art area integration state.

FIG. 5 is a block diagram of the invention of claim 1.

FIG. 6 is a block diagram of a first invention in another invention.

FIG. 7 is a block diagram of a second invention in another invention.

FIG. 8 is a block diagram of a third invention in another invention.

FIG. 9 is a block diagram of a first embodiment of the present invention.

10A is a diagram showing an original image, and FIG. 10B is a diagram showing a state in which the original image is divided into unit small areas.

FIG. 11A is a diagram showing a target region and a peripheral region of an image one hour before, and FIG. 11B is a diagram showing a region in which an image has a minimum correlation value at a predetermined time.

12A, 12B, and 12C are diagrams showing distributions of actual motions, motion vectors, and reliability.

13 (a), (b), and (c) are diagrams showing a state of a region integration state.

FIG. 14 is a block diagram of a second embodiment of the present invention.

FIG. 15 is a block diagram of a third embodiment of the present invention.

FIG. 16 is a diagram showing a rotational movement component of a camera.

17 (a), (b), (c) and (d) are diagrams showing a state of a region integration process.

FIG. 18A is a diagram showing an array of region labels, and FIG. 18B is a diagram showing updating of a region list.

[Explanation of symbols]

 16 motion vector detection device 18 reliability detection device 20 low reliability motion vector determination device 22 high reliability motion vector determination device 24 area integration device

Claims (1)

[Claims] 1. A motion vector / reliability detection means for detecting a distribution of a motion vector on a screen and a reliability distribution for the motion vector from an image at a predetermined time and an image unit time before the predetermined time, and the detection. Initial motion vector extraction means for extracting, as an initial motion vector, a motion vector whose reliability is higher than a preset first threshold value based on the distribution of the calculated reliability, and reliability based on the distribution of the detected reliability. A vector removing means for removing a motion vector lower than a second threshold value set in advance to a value smaller than the first threshold value and outputting a motion vector having a reliability of a second threshold value or more; and the initial motion vector. By extracting a region around the specified position, a motion vector that is the same as or similar to the initial motion vector is extracted from the motion vectors output from the vector removing means, and extracted. A moving object detecting device including: a region integrating unit that obtains a region by integrating a position including the motion vector.