JP3432937B2 - Moving object detecting device and moving object detecting method - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving object detecting device and a moving object detecting method used in the fields of autonomous mobile vehicles, intelligent work robots, image monitoring and the like.

[0002]

2. Description of the Related Art In recent years, with the advancement of integration technology, image sensors such as TV cameras have become smaller and have higher functions, and a processor for processing an image obtained by the image sensor has become smaller and more sophisticated. Research on autonomous mobile vehicles that use TV cameras as sensors, intelligent work robots, advanced image monitoring, etc. is being actively conducted toward practical application.

That is, when constructing a system such as an autonomous mobile vehicle, an intelligent work robot, or an image monitoring device, an image that changes from moment to moment is captured, this image is processed, and a moving object in the image is detected. By doing so, it is necessary to obtain information on a moving object, and a moving object detection device that detects such a moving object at high speed and stably is indispensable.

By the way, as a conventional moving body detection apparatus, an image successively obtained from a stationary observation system such as a fixed camera over time is used, and image difference processing between images and threshold processing are performed to intrude. The method of detecting the area of the object was common. However, in this method, when considering to use by installing it in an object such as an autonomous mobile vehicle where the observation system moves, the obtained image moves along with the movement of the observation system, and therefore the entire background Therefore, it becomes difficult to stably separate and detect the moving object region from the background.

[0005]

As a conventional moving body detection apparatus, images obtained sequentially from a stationary observation system such as a fixed camera over time are used to perform image difference processing between images and threshold processing. Therefore, the method of detecting the area of the invading object was general, but in this method, when considering to install and use it in an object such as an autonomous mobile vehicle where the observation system moves, The image obtained will move as the observation system moves, and
Since the entire background also changes, there is a problem that it is difficult to stably separate and detect the moving object area from the background.

However, there is a technique for directly detecting motion information even in such an image. For example, it is an optical flow in image information processing technology. That is, the optical flow is a movement in units of local areas in an image, and in the field of image information processing technology, a subject called optical flow extraction has been widely studied as a technique for directly detecting motion information from the image.

This optical flow is a motion in a three-dimensional space projected on an image plane and is obtained as a motion vector for each local region in the image. Therefore, in contrast to the above change detection method, a moving object / background is detected. There is an advantage that the separation can be performed stably.

However, since the optical flow detection process is originally performed on a local area in an image, the estimation accuracy is lowered in an environment where there are flatness, fluctuations, and noises in the brightness change. There is a problem.

Therefore, when a device that may be used in an outdoor environment such as an autonomous mobile vehicle is targeted, it is necessary to newly take the following measures. That is, it is necessary to provide a processing system that works stably even if there is an error in a part of each optical flow.

In order to stably extract motion information in an outdoor environment, it is indispensable to provide a processing system that works stably even if there is an error in a part of each optical flow.

Further, when the observation system moves, this movement also causes the background area to move, so that the movements of both are mixed and it is not possible to easily detect the appearance of a moving object. There was a problem.

Therefore, when the observation system moves, even if the background region also moves due to the movement of the observation system, it is possible to easily detect the moving object if it appears, and further, to detect the optical flow. There is a strong demand for realization of a moving object detection device that enables highly accurate detection of a moving object without having to provide a processing system that works stably even if there is an error in a part.

Therefore, an object of the present invention is to provide a moving object detecting device and a moving object detecting method capable of detecting a moving object at high speed and stably even when the observation system moves.

[0014]

In order to achieve the above object, the present invention is configured as follows. That is, firstly, by observing an observation means for acquiring and outputting an image of an observation target with the passage of time, and analyzing a temporal change of the image from this observation means, Optical flow extraction means for extracting optical flow information that is motion, and based on the optical flow information obtained by this optical flow extraction means, divides the image into partial areas centered on the portion in the image where this optical flow has occurred Then, a moving object is detected by analyzing the moving state of the partial area in the predicted image created by this predicted image creating means and the predicted image creating means for creating the predicted image at the next point of time by this partial area unit. And a moving object detecting means.

Secondly, by observing means for acquiring and outputting an image of an observation object with the passage of time, and analyzing the temporal change of the image from this observing means, the local in the image is analyzed. Optical flow extraction means for extracting optical flow information, which is a movement in a region unit, and the image is divided into units of a predetermined size to set a partial region for prediction, and among the set partial regions, ,, based on the optical flow information obtained by the optical flow extraction means, predict the image position of the next time point of the partial area centered on the part in the image in which this optical flow has occurred, the next time point in this partial area unit Predicted image creating means for creating a predicted image at the next point in time from the predicted position at, and the predicted image created by the predicted image creating means. By analyzing the moving state of the partial region, more configuration and moving object detection means for detecting a moving object.

Thirdly, by observing means for acquiring and outputting an image of an observation object with the passage of time, and analyzing the temporal change of the image from this observing means, a local area unit in the image is analyzed. An optical flow extraction unit that extracts optical flow information that is a motion, and a partial area for prediction is set by dividing an image into units of a predetermined size. Of the set partial areas, the optical flow is set. Based on the optical flow information obtained by the extraction means, predict the image position at the next time point of the partial area centered on the part in the image where the optical flow has occurred, and predict at the next time point in this partial area unit A predicted image creating unit that creates a predicted image at the next point in time from the position and an analysis of the moving state created by the predicted image creating unit. Detects the presence of the overlapping portion of the partial region in the prediction image, more configuration and moving object detection means for presence detection of the moving object than this.

Fourthly, by observing means for acquiring and outputting an image of an observation object with the passage of time, and analyzing the temporal change of the image from this observing means, a unit of local area in the image is obtained. Optical flow extraction means for extracting optical flow information, which is the motion of the image, and a partial area for prediction is set by dividing the image into units of a predetermined size. Based on the optical flow information obtained by the flow extraction means to predict the image position of the next time point of the partial area centered on the portion in the image in which this optical flow has occurred, at the next time point in this partial area unit A predicted image creating unit that creates a predicted image at the next point in time from the predicted position and a predicted image created by this predicted image creating unit are analyzed to obtain a predicted image. And a moving object detecting means for detecting a moving object by determining that a moving object is present when the overlapping portion of the partial area is measured, the size of the overlapping portion is measured, and the size is equal to or larger than a predetermined value. .

[0018]

In the case of the first configuration, the observing means acquires every image of the observation target, and the optical flow extracting means analyzes the temporal change of the acquired image to determine the local area unit in the image. Extract optical flow information, which is a motion. Then, the predictive image creating means divides the image into partial regions centered on the part in the image where the optical flow occurs, based on the optical flow information obtained by the optical flow extracting means, and the predictive image creating means The position of the partial area unit at the time point is predicted and a predicted image at the next time point is created. Then, the moving object detecting means detects the presence of the moving object by analyzing the moving state of the partial area in the predicted image created by the predicted image creating means.

In the case of the second configuration, the observing means acquires every image of the observation target, and the optical flow extracting means analyzes the temporal change of the obtained image to obtain a local region in the image. Optical flow information, which is a unit movement, is extracted. Then, the predicted image creating means sets a partial area for performing prediction by dividing the image into units of a predetermined size, and among the set partial areas, an optical flow obtained by the optical flow extracting means is set. Based on the flow information, the image position at the next time point of the partial area centered on the part in the image where this optical flow has occurred is predicted, and at the next time point from the predicted position at the next time point in units of this partial area. Create a prediction image of. Then, the moving object detecting means detects the presence of the moving object by analyzing the moving state of the partial area in the predicted image created by the predicted image creating means.

In the case of the third configuration, the observing means acquires every image of the observation target, and the optical flow extracting means analyzes the temporal change of the obtained image to obtain a local region in the image. Optical flow information, which is a unit movement, is extracted. Then, the predicted image creating means sets a partial area for performing prediction by dividing the image into units of a predetermined size, and among the set partial areas, an optical flow obtained by the optical flow extracting means is set. Based on the flow information, the image position at the next time point of the partial area centered on the part in the image for which this optical flow information was obtained is predicted, and the next position is predicted from the predicted position at the next time point in this partial area unit. Create a predicted image at the point in time. Then, the moving object detection means checks the presence or absence of the overlapping portion of the partial areas in the predicted image created by the predicted image creating means, and detects the presence of the moving object from the existence of the overlapping portion.

In the case of the fourth configuration, the observing means acquires every image of the observation target, and the optical flow extracting means analyzes the temporal change of the obtained image to thereby analyze the movement of each local area unit. The optical flow information is extracted. Then, the predicted image creating means sets a partial area for performing prediction by dividing the image into units of a predetermined size, and among the set partial areas,
Based on the optical flow information obtained by the optical flow extraction means, predict the image position at the next time point of the partial area centered on the part in the image where this optical flow occurred, and the next time point in this partial area unit A predicted image at the next time is created from the predicted position in. Then, the moving object detection means examines the presence or absence of an overlapping portion of the partial areas in the predicted image created by the predicted image creating means, and if there is an overlapping portion, measures the size of the overlapping portion to obtain a predetermined value or more. When it has a size of, the presence of a moving object is detected by determining that a moving object exists.

As described above, according to the present invention, an image of the observation target is obtained from the observing means, and the optical flow extracting means analyzes the temporal change of the image.
The optical flow information, which is the movement of each local area in the image, is extracted. For example, if the observing means makes a linear motion, in this case, the optical flow in the background area stationary in the environment exists on a straight line boiling from one point (FOE). Then, in the predicted image creating means, the image is divided into partial regions, among these, the position prediction at the next time point is performed in partial region units centered on the point where the optical flow occurs, and a predicted image is generated, and The moving object detection means extracts a portion where a plurality of areas in the predicted image overlap each other, and if the area in which the background area is shielded by the moving object is detected, the moving object is determined to exist.

Further, by measuring the size of the extracted "portion where a plurality of regions overlap" and comparing it with a threshold value to eliminate those caused by optical flow detection error, accuracy is maintained. I chose

As a result, according to the present invention, when the observation system moves, it becomes possible to easily and highly accurately detect a moving object included in an image captured by the observation system. An optimum moving body detection device can be obtained for use in fields such as mobile vehicles, intelligent work robots, and image monitoring.

[0025]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing the outline of the system configuration of the device of the present invention. In the figure, 1 is an observation system, 10 is an optical flow extraction unit, 20 is a prediction image creation unit, and 30 is
Is a moving object detection unit. The observation system 1 is composed of a TV camera or the like, captures an image of an observation target, and outputs image information moment by moment.

The optical flow extraction unit 10 obtains a time-series image from the TV camera of the observation system 1 and analyzes the temporal change of the image to extract an optical flow which is a movement in a local region unit. To do. In this embodiment, it is assumed that the observation system makes a linear motion. In this case, the optical flow of the background area stationary in the environment is one point (FO
It exists on the straight line boiled from E).

In the predictive image creating unit 20, the image is divided into a plurality of partial areas in advance in a predetermined size unit, and among these partial areas, a partial area unit having a point at which the optical flow is obtained as a center is used. Then, the position prediction at the next time point is performed, and an image reflecting the position prediction result is generated from the position prediction result as a predicted image.

The moving object detection unit 30 detects a region in which the background region is occluded by the moving object by extracting a portion in which the plurality of regions overlap in the predicted image generated by the predicted image generation unit 20, and moves. Detects the presence of an object. That is, the moving object detection unit 30 has a function of detecting the presence of a moving object by knowing that there is an area in which the background area in the predicted image is occluded by the moving object.

FIG. 2 shows a flow of processing corresponding to the flow of information between the above constituent elements. The operation of this system will be described with reference to FIG. That is, the image of the observation target is picked up from the observation system 1 and the image information is output every moment,
Since it is input to the optical flow extraction unit 10, the optical flow extraction unit 10 obtains image information from the observation system 1 in time series. Then, by analyzing the temporal change of the images obtained in time series, the optical flow that is the movement of each local region in the image is extracted (optical flow extraction process (S11)).

In this embodiment, the observation system 1 is assumed to make a linear motion. In this case, the optical flow of the background area stationary in the environment exists on the straight line boiling from one point (FOE).

The predictive image creating section 20 divides the image into a partial area centered on the point where the optical flow is obtained (prediction area setting process (S12)), and the next time point of the partial area in units of the divided partial areas. Position prediction in
A predicted image is generated from the result (region position prediction process at the next time (S13)).

The moving object detection unit 30 extracts a portion where a plurality of areas in this predicted image overlap (area interference portion detection processing (S14)), and detects an area in which the background area is shielded by the moving object (step S14). Moving object boundary candidate extraction processing (S15)). Then, if there is an area in which the background area is shielded by the moving object, the fact that the moving object exists is known from this (moving object detection processing (S16)).

In this way, the present apparatus can detect the moving object when the moving object is captured by the observation system by the optical flow from the images from the observation system obtained every moment.

Next, the details of each component in the apparatus of the present invention will be described. FIG. 3 shows a specific configuration example of the optical flow extraction unit 10. In this example, an implementation example of flow detection by local correlation of feature points in an image will be described.

Optical flow extraction unit 10 shown here
Is composed of a candidate point extraction unit 11, a local correlation calculation unit 12, a maximum correlation point position detection unit 13, an optical flow calculation unit 14, and an optical flow memory 15.

Of these, the candidate point detection unit 11 has a function of detecting a candidate point group for obtaining an optical flow in the reference image. Specifically, K pixels x K in the image
A process of setting a local region of pixels and calculating a density variance value in this region is performed over the entire region in which a moving object may appear.

The area size K is the type of the object to be detected,
It is a constant that is set according to the size. Points with a small variance value are included in the density-wise flat partial area, and the accuracy of optical flow extraction is greatly reduced, so they are excluded from candidate points and only those points with a variance value equal to or greater than a predetermined threshold value are excluded. A process of sequentially selecting the candidate points and transmitting the positions of the candidate points to the local correlation value detection unit 12 is performed.

The local correlation value detecting section 12 has M pixels centered on the candidate point (X _o , Y _o ) in the reference image.
The correlation value between the local region of M pixels and the local region of M pixels × M pixels around each point inside the candidate point neighboring region of the image at the next time is sequentially calculated. The size of the neighborhood area is set according to the speed of the moving object. The size M of the correlation window is a constant set according to the type, size, and speed of the target to be detected.

The correlation value is calculated by the following equation. Here, I (x, y) and J (x, y) are the reference image, the pixel density value of the point (x, y) of the image at the next time point, and I and J are the reference image and the image at the next time point, respectively. Is the average density in the area near the candidate point.

[0040]

[Equation 1]

Further, the maximum correlation point position detecting section 13 detects the position (X _s , Y) of the point on the next time point image where this correlation value becomes maximum.
_s ) is calculated. In the optical flow calculation unit 14, the difference (vx, vy) = (X _s −X) between them.
_o , Y _s −Y _o ) is calculated, and is calculated as a point (X
_(o , Y _o ) has a function of performing processing such as writing to the optical flow memory 15 as an optical flow vector.

FIG. 4 shows the optical flow extraction unit 1 described above.
The processing flow corresponding to each component of 0 is shown. this is,
3 shows a specific method of the processing (optical flow extraction processing) in S11 in the processing flow shown in FIG.

That is, the candidate point detection unit 11 detects and extracts a candidate point group for obtaining an optical flow in the reference image (candidate point extraction processing (S11a)). As described above, the process of setting a local region of K pixels × K pixels in the image and calculating the density variance value in this region is performed over the entire region in which a moving object may appear. Realize by doing.

The area size K is the type of the object to be detected,
It is a constant that is set according to the size. Points with a small variance value are included in the density-wise flat partial area, and the accuracy of optical flow extraction is greatly reduced, so they are excluded from candidate points and only those points with a variance value equal to or greater than a predetermined threshold value are excluded. It selects sequentially as a candidate point. In this way, in S11a, the candidate points are sequentially selected and extracted, and the information on the positions of the sequentially selected candidate points is sent to the local correlation value detection unit 12.

In the local correlation value detection unit 12, M pixels × M centered on the candidate point (X _o , Y _o ) in the reference image.
The correlation value between the local region of pixels and the local region of M pixels × M pixels around each point inside the candidate point neighboring region of the image at the next time is sequentially calculated (local correlation calculation process (S11
b)). The size of the neighborhood area is set according to the speed of the moving object. The size M of the correlation window is a constant set according to the type, size, and speed of the target to be detected.
The correlation value is calculated by the formula shown in [Equation 1] above.

In this way, if the correlation value is obtained in S11b, the maximum correlation point position detection unit 13 carries out a process for obtaining the position (X _s , Y _s ) of the point on the next time point image having the maximum correlation value. (Position extraction processing of the point having the maximum correlation value (S11c)).

Then, in the optical flow calculation unit 14, the difference (vx, vy) = (X _s −X _o , Y _s ) between them.
-Y _o ) is calculated, and this is calculated as the point (X _o , Y _o ) of the reference image.
Is written in the optical flow memory 15 as the optical flow vector in (1) (optical flow calculation, memory writing process (S11d)).

Next, a specific configuration example of the predicted image creating section 20 will be described with reference to FIG. As shown in the figure, the predicted image creation unit 20 includes a predicted partial area setting unit 21, a next time position prediction unit 22, and a predicted image creation unit 23.

Of these, the prediction partial area setting unit 21
Divides the image in advance into a size of a square small region of N pixels × N pixels, and the start point position (X _s of the flow vector written in the optical flow memory 15 which is a constituent element of the optical flow extraction unit 10). , Y _s ) are sequentially read out and this point (X _s , Y _s ) in the small area is read.
It has a function of performing processing such as selecting a small area including the position of as the predicted partial area. The size N of the small area is
It is a constant determined according to the minimum size of the object to be detected.

The next-time position predicting unit 22 estimates the positions of these predicted partial areas in the image at the next time using the size information of the corresponding flow vector written in the optical flow memory 15. It has a function of performing processing.

When the time interval of the time series image is small, the center position of the square small area is (X _c , Y _c ), and the corresponding flow vector is (vx, vy). _{c + vx} , Y _{c + vy} ), and it can be considered that the size of the area does not change in a square area of N pixels × N pixels. The unit 23 performs a process of obtaining the position of each small area at the next time point by parallel movement of the square small area based on the flow vector, and generates an image in which the position of each small area at the next time point is reflected as a predicted image. Then, processing such as storing in a storage means (not shown) is performed.

FIG. 6 shows the flow of processing corresponding to each of the constituent elements 21 to 23 of the predictive image creating section 20 having such a configuration. This is S1 in the process flow shown in FIG.
2 shows a specific method of the processing of S13.

The predictive partial area setting section 21 of the predictive image creating section 20 divides the image beforehand into square small areas of N pixels × N pixels. Then, the predictive partial area setting unit 21 sequentially reads out the start point positions (X _s , Y _s ) of the flow vector written in the optical flow memory 15 (optical flow vector read processing (S12a)),
Among the small areas, the starting point position (X _s, Y _s) of the read-out flow vectors in each square small region of the small region (above N pixels × N pixels including, include (X _s, Y _s) The square small area) is selected as the prediction partial area (prediction partial area selection processing (S12b)). As described above, the size N of the small area is a constant determined according to the minimum size of the object to be detected.

When the predicted partial area selection processing in S12b is completed, the predicted position calculation processing for each partial area is performed by the next time position prediction section 22. In the next time point position prediction unit 22, S12
The process of estimating the position in the image at the next point of time of these predicted partial regions obtained in the predicted partial region selection processing of b using the information of the magnitude of the corresponding flow vector written in the optical flow memory 15. Do.

When the time interval of the time series image is small, the center position of the square small area is (X _c , Y _c ) and the corresponding flow vector is (vx, vy), the center position at the next time point is ( X _{c + vx} , Y _{c + vy} ) and the area size is N
It can be considered that there is no change in the square region of pixels × N pixels. Therefore, the predicted image creation unit 23 obtains the position of each small area at the next time point by the parallel movement of the square small area based on the flow vector (predicted position calculation process of each partial area (S13a)). Then, the image obtained as a result of the parallel movement of the square small region is stored as a predicted image (predicted image creating process (S13a)).

As described above, the predicted image can be created. FIG. 7 shows a specific configuration example of the moving object detection unit 30.
The moving object detection unit 30 includes an overlapping portion detection unit 31, a moving object boundary candidate area detection unit 32, and a moving object detection unit 33.

In the predicted image P, when the observation system 1 for obtaining the original image is linearly moved, each small area BA included in the background moves as shown in FIG. 9 (a). It moves in the direction in which it boiled from one point F0E, which is the projection of the direction on the image.

When the observation system 1 is rotated,
The background area (each small area BA included in the background) translates in the direction opposite to the rotation direction. That is, it is as shown in FIG. Further, in these images P in which no moving object appears, the small areas BA in the predicted image P do not overlap each other and move to independent parts.

On the other hand, when a moving object appears,
The small area MA existing thereabove moves to a position that shields the background area BA. That is, as shown in FIG. 9C, both areas MA and BA move in a direction of approaching each other, and an overlapping portion OLA of the both areas is generated.

Actually, when there is a moving object in front of the background, in the image at the next time, this background area portion is covered and only the moving object is observed. Generally, this shielding area is a vertically long partial area.

Considering such a factual relationship, if the moving object detects a boundary portion that shields the background area by detecting and analyzing the overlapping portion OLA of the small area in the predicted image P, the appearance of the moving object is detected. You can see that it can be easily detected.

This system utilizes this fact. That is, in the overlapping portion extracting section 31 which is a constituent element of the moving object detecting section 30, a portion in which the small areas are overlapped with each other in the predicted image created by the predicted image creating section 23 which is a constituent element of the predicted image creating section 20 is detected. It has a function of extracting and passing this to the moving object boundary candidate area detection unit 32.

The moving object boundary candidate area detecting unit 32 which receives this has a function of detecting the connected areas in the overlapping portion as moving object candidate areas. By the way, the detected candidate areas include those due to optical flow detection error. However, while this detection error appears locally, the area covered by the moving object is at the boundary of the moving object.
Appears with a certain size or more.

Therefore, the moving object detecting section 33 has a function of measuring the size and width of these candidate areas detected by the moving object boundary candidate area detecting section 32.
It has a function of determining whether the size is equal to or larger than the threshold and the width is equal to or smaller than the threshold, and determining that a moving object has appeared if there is a candidate area satisfying this condition. Then, as a result of this measurement and determination by the moving object detection unit 33, the appearance of the moving object can be detected.

FIG. 8 shows a flow of processing corresponding to each component of the moving object detecting section 30. This shows a specific method of the S15 and S16 processes in the flow of the process shown in FIG.

In other words, in the predicted image, when the observation system 1 for obtaining the original image moves linearly, each small area included in the background has a movement direction as shown in FIG. 9A. Move in the direction of boiling from one point, which is the projection on the image. When the observation system 1 makes a rotational movement, the background area moves in parallel in the direction opposite to the rotational direction. That is, it is as shown in FIG. Further, in these images in which the moving object does not appear, the small areas in the predicted image do not overlap each other and move to independent parts.

On the other hand, when a moving object appears,
The small area existing above this moves to a place that shields the background area. That is, it is as shown in FIG. actually,
When there is a moving object in front of the background, this background area portion is blocked in the image at the next time point, and only the moving object is observed. Generally, this shielding area is a vertically long partial area.

Therefore, the appearance of the moving object can be easily detected by detecting and analyzing the overlapping portion of the small areas in the predicted image to detect the boundary portion where the moving object shields the background area.

The overlapping portion extracting unit 31 extracts a portion in which the small areas overlap each other in the predicted image created by the predicted image creating unit 23, which is a constituent element of the predicted image creating unit 20, and extracts this as a moving object boundary. It is passed to the candidate area detection unit 32 (small area overlapping portion extraction processing in the predicted image (S15
a)).

Receiving this, the moving object boundary candidate area detection unit 32 detects the connected areas in the overlapping portion as moving object candidate areas. By the way, the detected candidate areas include those due to optical flow detection error. However, while this detection error appears locally, the occluded region by the moving object appears at the boundary of the moving object with a certain size or more. Therefore, by determining the connectivity of the overlapping portion, it is determined whether or not this candidate is a candidate obtained by a detection error of the optical flow (connectivity determination of the overlapping portion (S15b)). As a result of this determination processing, only correct candidates can be selected.

The moving object detection unit 33 measures the size and width of these candidate regions selected and selected by the moving object boundary candidate area detection unit 32 (moving object boundary candidate area detection processing (S16a)). Then, it is determined whether the size is greater than or equal to the threshold and the width is less than or equal to the threshold,
If there is a candidate area that meets this condition, it is determined that a moving object has appeared (moving object detection by size determination (S16
b)). As a result of this measurement and determination by the moving object detection unit 33, the appearance of a moving object can be detected.

As described above, this apparatus is a TV provided in the observation system 1.
After obtaining the camera image, the optical flow extraction unit 10
By analyzing the temporal change of the image, the optical flow, which is the movement of the local area unit, is extracted. For example, if the observation system 1 moves linearly, in this case, the optical flow in the background area stationary in the environment is one point (FO).
It exists on the straight line boiled from E). Then, the predicted image creation unit 20 divides the image into partial regions, and performs position prediction at the next time point in units of partial regions centered on the point at which the optical flow is obtained, of these, and the predicted image P is obtained.
To generate. Then, the moving object detection unit 30 extracts a portion OLA in which a plurality of areas in the predicted image P overlap each other. If a region in which the background area is shielded by the moving object is detected, it is determined that the moving object exists. .

The accuracy is maintained by measuring the size of the extracted "portion OLA where a plurality of regions overlap" and comparing it with a threshold value so as to eliminate the error caused by the optical flow detection error. I did it.

As described above, according to the present invention, when the observation system moves, it becomes possible to easily and highly accurately detect a moving object included in an image captured by the observation system. An optimum moving object detecting device can be obtained for use in fields such as mobile vehicles, intelligent work robots, and image monitoring. Although the embodiments of the present invention have been described above, the present invention can be variously modified and implemented without departing from the spirit thereof.

[0075]

As described above in detail, according to the present invention, the predicted image at the next time point is generated by using the optical flow obtained by processing the time-series image of the observation target obtained from the observation system, and the predicted image is generated. It is possible to detect the appearance of a moving object at high speed and stably by analyzing the background area and finding the part where the background area is covered by the moving object. Since the original optical flow and the error due to other errors are discriminated according to the size, it is possible to detect the appearance of a moving object with high reliability.

[Brief description of drawings]

FIG. 1 is a diagram for explaining an embodiment of the present invention,
The block diagram which shows the whole structure of one Example of this invention.

FIG. 2 is a diagram for explaining an embodiment of the present invention,
3 is a flowchart for explaining the operation of the present apparatus having the configuration of FIG. 1.

FIG. 3 is a diagram for explaining an embodiment of the present invention,
FIG. 3 is a block diagram showing a specific configuration example of an optical flow extraction unit 10 in the configuration of FIG. 1.

FIG. 4 is a diagram for explaining an embodiment of the present invention,
FIG. 4 is a diagram showing a flow of processing corresponding to each component of the optical flow extraction unit 10 shown in FIG. 3.

FIG. 5 is a diagram for explaining an embodiment of the present invention,
FIG. 2 is a block diagram showing a specific configuration example of a predicted image creation unit 20 in the configuration of FIG. 1.

FIG. 6 is a diagram for explaining an embodiment of the present invention,
FIG. 6 is a diagram showing a flow of processing corresponding to each component of the predicted image creating unit 20 shown in FIG. 5.

FIG. 7 is a diagram for explaining an embodiment of the present invention,
FIG. 3 is a block diagram showing a specific configuration example of a moving object detection unit 30 in the configuration of FIG. 1.

FIG. 8 is a diagram for explaining an embodiment of the present invention,
FIG. 8 is a diagram showing a flow of processing corresponding to each component of the moving object detection unit 30 shown in FIG. 7.

FIG. 9 is a view for explaining the embodiment of the present invention,
The figure explaining various examples of the movement of the small area | region in a prediction image when the observation system 1 moves.

[Explanation of symbols]

1 ... Observation system 10 ... Optical flow extraction unit 11 ... Candidate point extraction unit 12 ... Local correlation calculator 13 ... Maximum correlation point position detection unit 14 ... Optical flow calculator 15 ... Optical flow memory 20 ... Prediction image creation unit 21 ... Prediction partial area setting section 22 ... Next time position prediction unit 23 ... Prediction image creation unit 30 ... Moving object detector 31 ... Overlap detection unit 32 ... Moving object boundary candidate area detection unit 33 ... Moving object detector P ... Predicted image BA ... small area OLA ... A part where multiple areas overlap

Continued Front Page (56) References Toshihide Watanabe 1 person, Motion Detection Neural Network, NHK Giken R & D, Japan Broadcasting Corporation, August 15, 1993, No. 25, pp. 42-43 (58) Fields investigated (Int.Cl. ⁷ , DB name) G06T ^7/ 00-7/60 H04N 7/18

Claims (8)

(57) [Claims] 1. An observing means for acquiring and outputting an image of an observation target with the passage of time, and analyzing a temporal change of each of the images from the observing means to determine a local area unit in the image. Optical flow extraction means for extracting optical flow information that is motion, and based on the optical flow information obtained by this optical flow extraction means, divides the image into partial areas centered on the portion in the image where this optical flow has occurred Then, a moving object is detected by analyzing the moving state of the partial area in the predicted image created by this predicted image creating means and the predicted image creating means for creating the predicted image at the next point of time by this partial area unit. And a moving object detecting means for performing the moving object detecting device. 2. An observing means for acquiring and outputting an image of an observation target with the passage of time, and analyzing a temporal change of each of the images from the observing means to obtain a local region unit in the image. An optical flow extraction unit that extracts optical flow information that is a motion, and sets a partial region for prediction by dividing the image into units of a predetermined size. Of the set partial regions, an optical region is set. Based on the optical flow information obtained by the flow extraction means to predict the image position of the next time point of the partial area centered on the portion in the image in which this optical flow has occurred, at the next time point in this partial area unit Predictive image creating means for creating a predicted image at the next time point from the predicted position, and the part in the predicted image created by the predicted image creating means A moving object detecting device, comprising: a moving object detecting unit that detects a moving object by analyzing a moving state of a minute area. 3. An observing unit that acquires and outputs an image of an observation target with the passage of time, and by analyzing a temporal change of each image from the observing unit, a local region unit in the image is analyzed. An optical flow extraction unit that extracts optical flow information that is a motion, and a partial area for prediction is set by dividing the image into units of a predetermined size. Of the set partial areas, the optical flow is set. Based on the optical flow information obtained by the extraction means, predict the image position at the next time point of the partial area centered on the part in the image where the optical flow has occurred, and predict at the next time point in this partial area unit A predictive image creating means for creating a predicted image at the next time point from the position and an analysis of the movement state created by the predictive image creating means. A moving object detecting device, comprising: a moving object detecting unit that detects the presence or absence of an overlapping portion of the partial areas in the predicted image, and detects the presence of a moving object from the overlapping portion. 4. An observing unit that acquires and outputs an image of an observation target with the passage of time, and by analyzing a temporal change of each image from the observing unit, a local region unit in the image is analyzed. An optical flow extraction unit that extracts optical flow information that is a motion, and a partial area for prediction is set by dividing the image into units of a predetermined size. Of the set partial areas, the optical flow is set. Based on the optical flow information obtained by the extraction means, predict the image position at the next time point of the partial area centered on the part in the image where the optical flow has occurred, and predict at the next time point in this partial area unit A predicted image creating unit that creates a predicted image at the next point in time from a position, and a predicted image created by this predicted image creating unit is analyzed to obtain a predicted image. A moving object detecting unit that detects the moving object by determining that a moving object exists, by extracting the overlapping portion of the partial area in, and measuring the size of the overlapping portion and having a size of a predetermined value or more, A moving object detection device comprising. 5. An image acquisition step of acquiring every minute image of an observation target with the passage of time, and analyzing the temporal change of the acquired image to show the movement in units of local regions in the image. Optical flow extraction step to extract the optical flow information, and based on the optical flow information obtained by the optical flow extraction step, the image is divided into partial areas centered on the portion in the image where the optical flow occurs, this A moving object detecting a moving object by analyzing the moving state of the partial area in the predicted image created by this predicted image creating step A moving object detection method comprising a detection step. 6. An image acquisition step of acquiring an image of an observation target with the passage of time, and by analyzing a temporal change of the image obtained by this image acquisition step, the movement in units of local regions in the image is detected. An optical flow extraction step for extracting certain optical flow information and a partial area for prediction is set by dividing the image into units of a predetermined size, and the optical flow extraction step among the set partial areas is set. Based on the optical flow information obtained by predicting the image position of the next time point of the partial area centered on the portion in the image where this optical flow occurred, from the predicted position at the next time point in this partial area unit A prediction image creation step of creating a prediction image at the next point in time, and a prediction created by this prediction image creation step And a moving object detecting step of detecting a moving object by analyzing a moving state of the partial area in the image. 7. An image acquisition step of acquiring an image of an observation target with the passage of time, and by analyzing a temporal change of the image obtained by this image acquisition step, it is a movement of an image in a local region unit. An optical flow extraction step for extracting optical flow information and a partial area for prediction is set by dividing the image into units of a predetermined size. Among the set partial areas, the optical flow extraction step is performed. Based on the obtained optical flow information to predict the image position of the next time point of the partial area centered on the portion in the image in which this optical flow has occurred, from the predicted position at the next time point in this partial area unit Prediction image creation step for creating the prediction image at the next time point and the prediction image created by this prediction image creation step A moving object detection method comprising: a moving object detection step of checking the presence or absence of an overlapping portion of the partial areas in an image and detecting the presence of a moving object from the presence of the overlapping portion. 8. An image acquisition step for acquiring an image of an observation target with the passage of time, and by analyzing a temporal change of the image obtained by this image acquisition step, an optical flow which is a movement in a local region unit is analyzed. An optical flow extraction step that extracts information and a partial area for prediction is set by dividing the image into units of a predetermined size, and the partial area that is set is obtained by the optical flow extraction step. Based on the optical flow information, the image position at the next time point of the partial area centered on the part in the image where this optical flow has occurred is predicted, and the next time point is calculated from the predicted position at the next time point in this partial area unit. The predicted image creation step for creating the predicted image in Check the presence or absence of overlapping parts of the partial area, and if there is an overlapping part, measure the size of the overlapping part and if it has a size of a predetermined value or more, determine that a moving object exists and A moving object detection method comprising: a moving object detection step of detecting the presence.