JPH07239998A - Periphery monitoring device for vehicle - Google Patents

Abstract

PURPOSE:To surely detect the behavior of a moving body in the periphery of vehicle in a short time by detecting the presence of the moving body in the periphery from a moving vector in a block extracted as a moving body direction pattern. CONSTITUTION:A front photographing means 201 photographs the part in front of its own vehicle and a last image Gf' and a current image Gf which are different in time series are obtained through a last image storage means 202. Then, a front moving vector arithmetic means 203 calculates a front moving vector Vf on the basis of the images Gf' and Gf. Then, a front moving vector direction detecting means 204 detects the direction Df of a front moving vector larger than a specific value in each front arithmetic area among front moving vectors Vf in each front block. Lastly, a precedent vehicle start detecting means 208 detects a precedent starting vehicle on the basis of front moving vectors in a precedent start direction pattern FP and a front approaching vehicle detecting means 209 detects a front approaching vehicle on the basis of front moving vectors in the front vehicle direction pattern FD.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a vehicle surroundings monitoring apparatus for monitoring the behavior of a moving body around a vehicle using a photographing means mounted on an automobile, and more particularly to a vehicle surroundings monitoring apparatus for photographing an optical system such as a CCD image sensor. The present invention relates to a vehicle surroundings monitoring device capable of reliably detecting an approaching vehicle around the own vehicle (front, side, or rear) by processing an image with a simple algorithm in a short time.

[0002]

2. Description of the Related Art Conventionally, in order to prevent a running vehicle from colliding with another short-distance vehicle or inadvertently changing lanes, an image of the surrounding area taken by an in-vehicle image sensor or the like is used to identify the vehicle. Various vehicle surroundings monitoring devices for recognizing the surrounding environment have been proposed. An image processing method used in this type of vehicle periphery monitoring device is, for example, Japanese Patent Laid-Open No. 4-1
There are a binarization method as disclosed in Japanese Patent No. 37016, and an edge extraction method as disclosed in Japanese Patent Laid-Open No. 4-151343.

However, when an object recognition method based on binarization is applied to a captured image from a moving vehicle, for example, the contrast of the image changes drastically due to changes in illuminance, shadows, etc. It is difficult to set the threshold in. Therefore, a conventional example using the edge extraction method will be described below.

FIG. 14 is a block diagram showing a schematic configuration of a conventional vehicle periphery monitoring device (obstacle recognition device) using an edge extraction method, for example. In the figure, 101 is a photographing means such as an image sensor, 102 is an edge constituent point detecting means for detecting an edge constituent point E from an image G photographed by the photographing means 101, and 103 is a horizontal continuous edge Eh from the edge constituent point E. Horizontal direction continuous edge detecting means for detecting, 104 is a vertical direction continuous edge E from the edge constituent point E.
A vertical continuous edge detecting means for detecting v, and 105 is an obstacle recognizing means for recognizing an obstacle based on the horizontal continuous edge Eh and the vertical continuous edge Ev.

FIG. 15 is an explanatory view showing an image state processed in the apparatus of FIG. 14, 110 is a processed image obtained by the edge constituent point detecting means 102, that is, edge constituent points E and 111 are horizontal continuous edges. Processed image obtained by the detection means 103, that is, horizontal continuous edges Eh, 1
Reference numeral 12 denotes a processed image obtained by the vertical continuous edge detecting means 104, that is, a vertical continuous edge Ev.

Reference numerals 113 and 114 denote obstacle recognizing means 10.
5 is an image processed by No. 5, 113 is an extraction region in which both horizontal continuous edges Eh and vertical continuous edges Ev exist, and 114 is an obstacle recognized from the extraction region 113.

Next, the operation of the conventional vehicle periphery monitoring device shown in FIG. 14 will be described with reference to FIG. First, the edge composing point detecting means 102 detects the edge composing point E from the image G and extracts the processed image 110.

Next, the horizontal continuous edge detecting means 103
The vertical continuous edge detecting means 104 detects edge constituent points which are continuous in the horizontal direction and the vertical direction for a predetermined length or longer as the horizontal continuous edge Eh and the vertical continuous edge Ev, and process images 111 and 1 respectively.
12 is extracted.

The horizontal continuous edges E thus extracted
h and the vertical continuous edge Ev are the obstacle recognition means 1
Then, the obstacle recognition means 105 extracts the area 113 in which both the horizontal continuous edge Eh and the vertical continuous edge Ev exist, and the obstacle 114 (hatched portion in FIG. 15) is extracted from the extracted area 113. Recognize.

The above is an example of the vehicle surroundings monitoring device for recognizing an obstacle by the edge extraction method. However, when the background is complicated, many edges having a length similar to the obstacle are detected, and the obstacle is correctly detected from the background. Is very difficult to recognize separately, and the algorithm is complicated. Also,
It is not possible to specifically analyze whether the obstacle is stationary or approaching.

Therefore, in order to detect the movement of surrounding vehicles,
For example, as disclosed in Japanese Patent Application Laid-Open No. 4-40599, a vehicle surroundings monitoring device has been proposed which monitors the relative positions of the own vehicle and the preceding vehicle from an image when the own vehicle is stopped.

This relative position monitoring device sets a predetermined area (window) on the photographed image, stores the front vehicle image in the window when the own vehicle is stopped, and the stored image and the photographed after that. By detecting the difference from the image in the window, the relative position change with respect to the preceding vehicle while the own vehicle is stopped is detected.

In this case, however, it is possible to detect the presence or absence of movement of the vehicle in front in the image, but it is not possible to detect the direction of movement such as shaking of the image. Therefore, if a person inside the vehicle moves, such as when the person inside the vehicle moves while parked or parked, the image of the vehicle in front in the image captured by the image sensor attached to the vehicle will also move, and There is a large difference between the image of the front vehicle and the image in the window after the shake, and it may be erroneously determined that the front vehicle has moved and the relative position has changed even if the front vehicle does not move at all.

Therefore, further, Japanese Patent Laid-Open No. 213973/1992.
As described in Japanese Patent Publication, an image using a method in which a captured image is divided into a plurality of blocks and a motion vector indicating the motion of the image in each block is calculated from two images that are different in time series. A shake correction device has also been proposed.

This image shake correction apparatus divides the image into a plurality of blocks, obtains the motion vector of the image in each block, and specifies the background block among the blocks that often includes the background on the image. Motion vector of
The motion vector of the moving body block other than the background block is compared. Then, from the motion vector detected in each block, the motion vector of the background block due to the image shake is detected separately from the motion vector of the moving body block, and the image shake is corrected based on the detected image shake vector. Is.

As described above, the shake of the entire screen can be extracted by the method of dividing the image into a plurality of blocks and obtaining the motion vector of the image for each block. However, in this case, the photographing means mounted on the stopped vehicle is taken into consideration.

Therefore, the above-mentioned correction method is applied to an image taken from a moving body such as a moving vehicle, and a vehicle that makes a specific motion with respect to the background is detected based on the direction pattern of the detected motion vector. Is not suggested. If the above correction method is applied directly to an image taken from a moving vehicle, the processing will take time, and a motion vector will be erroneously calculated for a background image such as a sky with little contrast, resulting in an erroneous vehicle. It will lead to detection.

[0018]

As described above, the conventional vehicle surroundings monitoring apparatus applies the object recognition method by binarization to the captured image from the moving body such as a vehicle (Japanese Patent Laid-Open No. 4-42). JP-A-137016) has a problem that it is difficult to set a threshold value in binarization due to a sharp change in image contrast due to a change in illuminance, a shadow, and the like.

When the object recognition method based on edge extraction is applied (see Japanese Patent Laid-Open No. 4-151343),
Due to the complicated background, many edges with a length similar to the obstacle were detected, and it was very difficult to correctly separate and recognize the obstacle from the background, and the algorithm became complicated. .

Further, in the case where the relative change with respect to the front vehicle is detected by the change in the difference between the image of the front vehicle in the window and the image in the subsequent window when the vehicle is parked or parked (Japanese Patent Laid-Open No. 40599/1992). It is very difficult to determine whether the surrounding vehicle actually moved or caused by the shake when a shake etc. occurred in the captured image,
There is a problem that the movement of surrounding vehicles may be erroneously detected.

Further, when the entire image is divided into a plurality of blocks and the motion vector of the image in each block is obtained (see Japanese Patent Laid-Open No. 213973/1992), the shaking during traveling is not taken into consideration, and therefore, it is left as it is. When applied to an image taken from a running vehicle, it takes a lot of processing time, and a motion vector is erroneously calculated for a background image with little contrast, resulting in erroneous detection of the vehicle.

The present invention has been made in order to solve the above problems, and suppresses erroneous detection of a motion vector in an image due to shaking of the own vehicle or a complicated background even while traveling, An object of the present invention is to provide a vehicle surroundings monitoring device capable of reliably detecting the behavior of a moving body around the vehicle in a short time.

[0023]

According to a first aspect of the present invention, there is provided a vehicle surroundings monitoring device, a photographing means for photographing the surroundings of an own vehicle and generating a plurality of images different in time series, and a plurality of images. A motion vector is set in each of the above predetermined positions, the motion region is divided into a plurality of blocks, and a motion vector calculation means for calculating the image motion in each block as a motion vector; And a motion vector direction detecting means for outputting the direction of the motion vector having a magnitude larger than a predetermined value, and a motion vector monitoring region is further set in the calculation region, and the motion vector direction in the block is set in the motion vector monitoring region. A movement that monitors and extracts a combination of blocks of motion vectors having the same direction as the image motion direction when a moving body exists around the vehicle as a moving body direction pattern. And body direction pattern extraction means, in which a moving object detecting means for detecting the presence of a moving object near the motion vector in a block extracted by the mobile direction pattern extraction means.

Further, a vehicle periphery monitoring device according to a second aspect of the present invention includes a front photographing means for photographing the front of the running vehicle and generating a plurality of front images different in time series.
A forward motion vector that sets forward calculation areas at predetermined positions on multiple forward images, divides the forward calculation area into multiple forward blocks, and calculates the forward image motion in each forward block as a forward motion vector. Computing means,
Out of the forward motion vectors, a forward motion vector direction detecting means for outputting the direction of the forward motion vector having a size larger than a predetermined value in the forward calculation region, and a forward motion vector monitoring region are set in the forward calculation region. , The forward motion vector direction in the front block is monitored in the forward motion vector monitoring area, and the combination of front motion vector front blocks having the same direction as the forward image motion direction when a front approaching vehicle is present is used as the front vehicle. Front vehicle direction pattern extracting means for extracting as a direction pattern, and front approaching vehicle detecting means for detecting the presence of a front approaching vehicle from the forward motion vector in the front block extracted by the front vehicle direction pattern extracting means Is.

A vehicle surroundings monitoring device according to a third aspect of the present invention is the vehicle surroundings monitoring device according to the second aspect, wherein the own vehicle stop determining means determines the stop state of the own vehicle based on the forward motion vector direction. Front vehicle starting direction pattern extracting means for extracting the front vehicle starting direction pattern in response to the stop state, and front vehicle starting detecting means for detecting the front vehicle starting from the forward motion vector in the front vehicle starting direction pattern, The vehicle starting direction pattern extracting means sets the front vehicle starting vector monitoring area setting means for setting the front vehicle starting vector monitoring area of the front starting vehicle in the front calculation area when the host vehicle stops, and the front vehicle starting vector monitoring The direction of the front motion vector in the front block is monitored in the area, and the combination of the front blocks having the front motion vector in the same direction as the direction of the front image motion by the front start vehicle is combined with the front vehicle start direction. It is intended to include the vehicle in front starting direction pattern detection means for generating a turn.

According to a fourth aspect of the present invention, in the vehicle periphery monitoring apparatus according to the second aspect or the third aspect, the rear side of the running vehicle is photographed and a plurality of rear images different in time series are taken. The backward image capturing means to be generated and the backward calculation area are set at predetermined positions on the plurality of rear images, respectively, and the backward calculation area is divided into a plurality of rear blocks, and the rear image movement in each rear block is moved backward. A backward motion vector calculating means for calculating as a vector, a backward motion vector direction detecting means for outputting the direction of a backward motion vector having a magnitude of a predetermined value or more in the backward calculation vector, and a backward calculation vector in the backward calculation region In addition to setting the backward motion vector monitoring area, the direction of the backward motion vector is monitored in the backward motion vector monitoring area, and the backward image motion assumed when a vehicle approaching backward is present. A backward approaching vehicle direction pattern extracting means for extracting a combination of backward blocks of backward motion vectors having the same direction as the backward approaching vehicle direction pattern, and the presence of the backward approaching vehicle from the backward moving vector in the backward approaching vehicle direction pattern. And a rear approaching vehicle detecting means for detecting.

According to a fifth aspect of the present invention, there is provided a vehicle surroundings monitoring apparatus according to any one of the second to fourth aspects, in which a side view of a running vehicle is photographed and different in time series. A side photographing means for generating a plurality of side images and a side calculation area is set at a predetermined position on each of the plurality of side images, and the side calculation area is divided into a plurality of side blocks. Lateral motion vector calculating means for calculating the lateral image motion in each lateral block as a lateral motion vector, and lateral motion having a magnitude of a predetermined value or more within the lateral calculation area of the lateral motion vectors. A lateral motion vector direction detecting means for outputting the direction of the vector, and a lateral motion vector monitoring area is set in the lateral calculation area, and the direction of the lateral motion vector is monitored in the lateral motion vector monitoring area. Assumed if there is a lateral overtaking vehicle A side-passing vehicle direction pattern extraction means for extracting a combination of side-side blocks of a side-direction motion vector having a direction similar to the direction of the side-direction moving image as a side-passing vehicle direction pattern, and a side-passing vehicle direction pattern And a side-passing vehicle detecting means for detecting the presence of a side-passing vehicle from the side-moving vector.

According to a sixth aspect of the present invention, in the vehicle periphery monitoring apparatus according to the fifth aspect, a small motion vector monitoring area is set in the side calculation area, and the small motion vector monitoring area is set in the lateral motion area. A small motion vector direction extracting means for extracting a direction of a side motion vector having a magnitude smaller than a predetermined value among the motion vectors as a small motion vector direction, and a small motion having a direction similar to the background motion from the small motion vector direction. A background vector deleting means for eliminating the vector direction, and a side-by-side running vehicle detecting means for detecting the presence of a side vehicle with a small relative speed from the position and size of the side motion vector from which the background vector is deleted are provided. It is a thing.

[0029]

According to the first aspect of the present invention, the calculation area for calculating the motion vector in the captured image is limited, the motion vector monitoring area is further limited in the calculation area, and the predetermined value is determined from the block in the motion vector monitoring area. By detecting only the above motion vector and eliminating the part that seems to be the background from the beginning, it is possible to suppress the image shake caused by the bounce due to the unevenness of the road and the false detection of the motion vector caused by the complicated background, and The behavior of the surrounding moving body can be detected reliably and in a short time.

Further, according to a second aspect of the present invention, the front calculation area for calculating the front motion vector is limited in the front image of the running vehicle, and the front motion vector monitoring area is further limited in the front calculation area. However, by detecting only the forward motion vector of a predetermined value or more in the moving direction of the approaching vehicle from the block in the forward motion vector monitoring area, and eliminating the part that seems to be the background from the beginning, it is possible for the vehicle to shake or become complicated. An erroneous detection of a motion vector due to the background is suppressed, and a vehicle approaching ahead of the own vehicle is reliably detected in a short time.

Further, according to a third aspect of the present invention, the apparent magnitude and direction of the motion vector are detected from the block when the vehicle is stopped in the forward motion vector monitoring area according to the second aspect, and the forward starting vehicle is detected. Only forward motion vectors that exceed a certain value in the direction of motion are extracted as candidates for starting the preceding vehicle, and parts that are likely to be the background are excluded from the beginning to prevent erroneous detection of motion vectors due to the shaking of the vehicle or a complicated background. Then, the forward starting vehicle is detected reliably and in a short time from the relative position change between the own vehicle and the preceding vehicle.

Further, according to a fourth aspect of the present invention, the backward calculation area for calculating the backward motion vector is limited in the backward image of the vehicle being driven, and the backward motion vector monitoring area is further limited in the backward calculation area. However, by detecting only the backward motion vector of a predetermined value or more in the moving direction of the vehicle approaching backward from the block in the backward motion vector monitoring area, and eliminating the part that seems to be the background from the beginning, The erroneous detection of a motion vector due to a complicated background is suppressed, and a vehicle approaching to the rear of the host vehicle is detected reliably and in a short time.

Further, according to a fifth aspect of the present invention, the lateral operation area for calculating the lateral motion vector is limited in the lateral image of the own vehicle in motion, and the lateral motion area is further included in the lateral operation area. Limit the vector monitoring area, detect only the lateral motion vector of a certain value or more in the direction of movement of the lateral passing vehicle from the blocks in the lateral motion vector monitoring area, and eliminate the part that seems to be the background from the beginning. As a result, erroneous detection of a motion vector due to a sway of the host vehicle or a complicated background is suppressed, and a lateral overtaking vehicle of the host vehicle is detected reliably and in a short time.

According to a sixth aspect of the present invention, a small motion vector monitoring area is set in the lateral calculation area in the fifth aspect, and the moving direction of the lateral parallel running vehicle is set from the block in the small motion vector monitoring area. In addition to detecting only small motion vectors below a predetermined value in, the background vector that has the same direction pattern as the background image motion is deleted to suppress the false detection of motion vectors due to the shaking of the own vehicle and the complicated background. The side-by-side running vehicle with little movement above can be detected reliably and in a short time.

[0035]

【Example】

Example 1. Embodiment 1 (corresponding to claims 1 to 6) of the present invention will be described below with reference to the drawings. 1 to 6 are functional block diagrams showing the configuration of the first embodiment of the present invention, FIG. 1 is a functional block diagram showing the configuration of a forward monitoring apparatus in the first embodiment of the present invention, and FIG. FIG. 1 is a functional block diagram showing a specific configuration of the forward motion vector computing means.
It is a functional block diagram which shows the concrete structure of the front vehicle starting direction pattern extraction means in the inside.

4 and 5 are functional block diagrams showing respective configurations of the rear monitoring device and the side monitoring device in the first embodiment of the present invention, and FIG. 6 is a small motion vector direction extracting means in FIG. 3 is a functional block diagram showing a specific configuration of FIG.

In FIG. 1, reference numeral 200 denotes a front monitoring device, which is composed of the following means 201 to 209. Two
Reference numeral 01 denotes a front photographing means that is composed of an image sensor such as a CCD camera and generates a front current image Gf, and 202 denotes a front current image Gf that has been photographed and is stored until the next photographing timing to generate a front image Gf '. This is the previous image storage means.
The front image storage unit 202 may be included in the front image capturing unit 201.

Reference numeral 203 denotes a forward motion vector calculating means for calculating the forward motion vector Vf based on the previous image Gf 'and the current image Gf which are different in time series, and means 20 shown in FIG.
3a to 203c.

In FIG. 2, reference numeral 203a denotes a front calculation area setting means for setting front calculation areas Rf 'and Rf for each image, which are located at predetermined positions on the front image Gf' and the current image Gf (possible presence of a front vehicle). (Higher area), one or more forward calculation areas Rf 'and Rf are set.

Reference numeral 203b denotes a front block dividing means for dividing each front calculation area into front blocks Bf 'and Bf, and each of the front calculation areas Rf' and Rf is divided into a plurality of front blocks Bf 'and Bf (motion vectors of the front target). Area that is the minimum unit for detecting). Two
Reference numeral 03c is a front motion vector detecting means for detecting the front motion vector Vf based on the images in the front blocks Bf 'and Bf.

Returning to FIG. 1, 204 is the forward motion vector Vf.
Is a forward motion vector direction detecting means for detecting the forward motion vector direction Df based on the above, and is equal to or larger than a predetermined value (corresponding to a size indicating a behavior such as approaching) in the front calculation regions Rf ′ and Rf of the front motion vector Vf. The direction Df of the forward motion vector Vf having the magnitude of (Yes) is output.

Reference numeral 205 is a vehicle stop determination means for outputting a determination result P or D indicating the stopped state or running state of the vehicle based on the forward motion vector direction Df. For example, the divergence direction pattern of the background image does not exist. In this case, the determination result P of the own vehicle stop state is output.

Reference numeral 206 denotes a front vehicle starting direction pattern extracting means for extracting the front starting direction pattern FP in response to the determination result P indicating the stopped state of the own vehicle, and means 20 shown in FIG.
6a to 206c.

In FIG. 3, reference numeral 206a designates a front vehicle start vector monitoring area setting means, which further includes one or more front vehicle start vectors corresponding to the front start vehicle and its surroundings in the front calculation areas Rf 'and Rf. The monitoring areas RWf 'and RWf (areas in which there is a high possibility that a forward vehicle is present) are set.

Reference numeral 206b denotes a front vehicle start vector direction monitoring means, which refers to the front motion vector Vf and the front motion vector direction Df in each of the front vehicle start vector monitoring areas RWf 'and RWf to start a plurality of front vehicles. Vector direction DV
f (the direction of image movement that converges because the vehicle is moving away from the vehicle) is output.

Reference numeral 206c is a front vehicle starting direction pattern detecting means for detecting the front vehicle starting direction pattern FP in response to the judgment result P indicating the stopped state, and refers to the front vehicle starting vector direction DVf when the own vehicle is stopped, Each image G of the forward starting vehicle
A combination of the front blocks Bf ′ and Bf having the same direction pattern as the movement pattern on f ′ and Gf is extracted as the front vehicle starting direction pattern FP.

Returning to FIG. 1, 207 is a forward vehicle direction pattern extracting means having the same functional configuration as that of FIG. 3 (front vehicle starting direction pattern extracting means 206), and responds to the determination result D indicating the running state, Forward motion vector V while the vehicle is running
Refer to f, etc., and approach forward vehicles (including forward interruption vehicles, etc.)
A combination of front blocks having the same direction pattern as the image movement pattern of is extracted as the front vehicle direction pattern FD. At this time, the background pattern or the like that diverges at high speed corresponding to the traveling speed is deleted.

In this case, the image moving direction of the vehicle approaching in front is
As it approaches the host vehicle, it will diverge. In addition, the image movement direction of the front interrupt vehicle is a direction in which the image is diverging with movement to the left or right.

Reference numeral 208 denotes a front vehicle start detecting means for detecting a front start vehicle on the basis of the front vehicle start direction pattern FP while the host vehicle is stopped. The front motion vector Vf in the front block included in the front vehicle start direction pattern FP. The front vehicle start is detected from. 209 is a forward vehicle direction pattern FD while the vehicle is traveling
It is a front approaching vehicle detection means for detecting a front approaching vehicle based on the.

On the other hand, in FIG. 4, reference numeral 210 is a rear monitoring device, and each means 211 to 214, 217 and 219.
It consists of The respective means constituting the rear monitoring device 210 are respectively the respective means 20 in the front monitoring device 200.
1 to 204, 207 and 209.

That is, 211 is a rear photographing means for photographing the rear, and 212 is a front image storage means for storing the rear current image Gr as a front image Gr '. Reference numeral 213 denotes a backward motion vector calculating means having the same functional configuration as that of FIG. 2, and calculates a backward motion vector Vr based on two backward images that are different in time series, that is, the previous image Gr ′ and the current image Gr. Reference numeral 214 is a backward motion vector direction detecting means for detecting the backward motion vector direction Dr based on the backward motion vector Vr.

Reference numeral 217 is a rear approaching vehicle direction pattern extracting means for extracting the rearward approaching vehicle direction pattern RP based on the rearward moving vector direction Dr, and has the same functional configuration as that of FIG. 3, that is, in the rearward images Gr 'and Gr. Rear motion vector monitoring area setting means for setting one or more rear motion vector monitoring areas (areas in which a rear approaching vehicle is likely to exist) on the rear calculation area, and the rear motion vector monitoring area within the rear motion vector monitoring area. Backward motion vector monitoring means for monitoring the direction,
A rear approaching vehicle direction pattern detecting means for detecting a combination of rearward blocks having the same direction pattern as the image movement of the rear approaching vehicle (a direction diverging to approach the own vehicle) as a rear approaching vehicle direction pattern RP.

Reference numeral 219 denotes a rear approaching vehicle detecting means for detecting a rear approaching vehicle on the basis of the rear approaching vehicle direction pattern RP. The rear approaching vehicle is detected from the rearward moving vector in the rear block obtained by the rearward approaching vehicle direction pattern extracting means 217. Detect a car.

Further, in FIG. 5, reference numeral 220 denotes a side monitoring device, which is composed of each means 221 to 229,
These respective means correspond to the respective means 201 to 209 in the front monitoring device 200.

That is, 221 is a side image capturing means for capturing a side image, and 222 is a front image storing means for storing the side current image Gs as a front image Gs'. Reference numeral 223 denotes a lateral motion vector calculation means having the same functional configuration as in FIG. 2, and calculates the lateral motion vector Vs based on the previous image Gs ′ and the current image Gs which are different in time series. Reference numeral 224 is a lateral motion vector direction detecting means for detecting the lateral motion vector direction Ds based on the lateral motion vector Vs.

225 is a small motion vector direction extracting means for extracting the direction of the side motion vector Vs having a magnitude smaller than a certain fixed value (corresponding to the motion of the side-by-side running vehicle), that is, the small motion vector direction Dss. , 225a and 225b shown in FIG.

In FIG. 6, 225a is a side image Gs'.
And a small motion vector monitoring area for setting one or more small motion vector monitoring areas Rss 'and Rss (areas in which a side parallel running vehicle is likely to exist) on the lateral calculation areas Rs' and Rs in Gs. The setting means 225b sets the direction of the lateral motion vector Vs detected in the small motion vector monitoring areas Rss' and Rss to a small value or less to the small motion vector direction Dss.
Is a small motion vector detecting means for outputting as.

Returning to FIG. 5, 226 is the small motion vector direction D.
It is a background vector deleting means that eliminates the lateral motion vector Vs having the same direction as the background motion based on ss. In this case, the movement of the lateral parallel vehicle on the image is not characteristic compared to other behaviors (divergence, convergence, etc.), and therefore it is particularly desirable to delete the background vector. Reference numeral 227 is a lateral overtaking vehicle direction pattern extraction means having the same functional configuration as in FIG. 3, and extracts the lateral overtaking vehicle direction pattern SP based on the lateral motion vector direction Ds.

That is, the side-passing vehicle direction pattern extraction means 227 is arranged so that one or more side-moving-vector monitoring areas (side-passing vehicles can exist) on the side calculation areas in the side images Gs' and Gs. A lateral motion vector monitoring area setting means for setting a lateral motion vector monitoring area, a lateral motion vector monitoring means for monitoring the direction Ds of the lateral motion vector Vs in the lateral motion vector monitoring area, and a lateral motion vector direction. And a side-passing vehicle direction pattern detecting means for detecting a combination of side-blocks having the same direction pattern as the movement on the image of the side-passing vehicle based on Ds as a side-passing vehicle direction pattern SP.

Reference numeral 228 denotes a side-by-side vehicle detection means for detecting the presence of a side-by-side traveling vehicle. From the position and size of the side-by-side motion vector Vs' from which the background vector is deleted by the background vector deletion means 226, The presence of a lateral parallel vehicle with a small relative speed is determined and detected. Reference numeral 229 denotes a side-passing vehicle detecting means for detecting a side-passing vehicle, which detects the side-passing vehicle from the side-movement vector in the side-block included in the side-passing pattern SP.

As shown in FIGS. 1 to 6, the vehicle periphery monitoring device according to the first embodiment of the present invention comprises front, rear and side monitoring devices 200, 210 and 220, and each monitoring device is Imaging means 201, 211 and 221
And means 203, 21 for calculating the motion vector in each image
3 and 223, and means 204, 214 and 224 for detecting the direction of a motion vector of a predetermined value or more in each calculation area
And means 20 for extracting, as a pattern, a block having a motion vector of the same pattern as the motion direction of each detection target.
7, 217 and 227, and means 209, 219 and 229 for detecting each detection target vehicle based on each extracted pattern.

Further, the front monitoring device 200 includes the own vehicle stop judging means 205 for judging the parking / stopping state based on the front moving vector direction Df, and the front moving vector having the same pattern as the moving direction of the front starting vehicle when the own vehicle is stopped. Front vehicle start direction pattern extraction means 206 for extracting a front block having Vf as a front vehicle start direction pattern FP, and front vehicle start detection means 2 for detecting a front start vehicle based on the front vehicle start direction pattern.
08 and.

Further, in the lateral monitoring device 220, a small motion vector direction detecting means for detecting the direction of the lateral motion vector of the lateral motion vector which is less than a certain value in the small motion vector monitoring area as the small motion vector direction. 225, a background vector deleting means 226 for deleting a small motion vector having a direction pattern similar to the background image motion, and a lateral direction for detecting a lateral parallel vehicle from the lateral motion vector Vs' with the background vector deleted. It has parallel running vehicle detection means 228.

Although not shown here, the detection results of the detection means 208, 209, 219, 228 and 229 are used to drive an arbitrary display device or alarm device to call the driver's attention. May be. For example, if a forward-starting vehicle is detected while the vehicle is stopped due to a signal waiting, a caution is issued to promptly start and follow the vehicle, and a forward or backward approaching vehicle is detected while traveling. It can alert you to dangerous situations.

Further, the front monitoring device 200 detects not only a front approaching vehicle but also a front starting vehicle, and the side monitoring device 2
Although not only the side-passing vehicle but also the side-by-side traveling vehicle are detected with respect to 20, only the front approaching vehicle or the side-passing vehicle having a high degree of risk is detected as in the case of the rear monitoring device 210. You may

Further, the case where all of the monitoring devices 200, 210 and 220 for the front, the rear and the side are provided is shown, but it is possible to provide only one monitoring device or a combination of any two monitoring devices. Even in
The objective effect corresponding to the monitoring device can be achieved.

Needless to say, the same monitoring device as described above can be provided in any direction around the host vehicle, and the objective effect corresponding to the monitoring device can be achieved. Further, although the case where the detection target in the vicinity of the own vehicle is the vehicle is shown, any detection target can be used as long as it is a moving body on the traveling lane.

Next, referring to the flow chart of FIG. 7, regarding the specific vehicle behavior detection processing operation according to the first embodiment of the present invention, the case of the front monitoring device 200 (see FIGS. 1 to 3) is taken as an example. explain. 7, steps S1 to S9 correspond to the processing operations of the respective means 201 to 209 in FIG.

First, the front photographing means 201 photographs the front of the vehicle (step S1), and the front image storing means 202
The previous image Gf ′ and the current image G that are different in time series via
f is acquired (step S2).

Next, the forward motion vector calculation means 203
A forward motion vector Vf based on each image Gf ′ and Gf
Is calculated (step S3). That is, the front calculation area setting means 203a sets the front calculation areas Rf 'and Rf at predetermined positions in the images Gf' and Gf, and the front block dividing means 203b further divides each front calculation area into a plurality of front blocks Bf. ′ And Bf, the front motion vector detecting means 203c detects the front motion vector Vf of the image in each front block (step S3).

Subsequently, the forward motion vector direction detecting means 20
4 is the front motion vector Vf in each front block,
The direction Df of the forward motion vector having a magnitude equal to or larger than a predetermined value is detected in each of the forward calculation areas (step S4), and the host vehicle stop determination means 205 determines whether the host vehicle is stopped (step S4). S5).

If it is determined in step S5 that the host vehicle is stopped (that is, YES), the front vehicle starting direction pattern extraction means 206 responds to the determination result P indicating the stopped state, and the front vehicle starting direction pattern is obtained. FP is extracted (step S6).

That is, the front vehicle start vector monitoring area setting means 206a sets the front vehicle start vector monitoring areas RWf 'and RWf at the front vehicle and its peripheral positions in each front calculation area, and the front vehicle start vector is set. Direction monitoring means 2
06b monitors the forward motion vector direction Df in the front vehicle start motion vector monitoring area, and detects the front vehicle start motion vector direction DV.
f is generated, and the front vehicle starting direction pattern detection means 206c
Detects a front block including a motion vector having a direction pattern similar to the image motion of the front vehicle as a front vehicle start direction pattern FP based on the front vehicle start vector direction DVf.

On the other hand, when it is determined in step S5 that the host vehicle is traveling (that is, NO), the forward vehicle direction pattern extraction means 207 determines the determination result D indicating the traveling state.
In response to, the forward vehicle direction pattern FD is extracted (step S7).

That is, 1 is set at a predetermined position in the front calculation area.
One or more forward motion vector monitoring areas are set, and each forward motion vector direction Df is monitored within the forward motion vector monitoring area,
A front block including a front motion vector having a direction pattern similar to the image motion direction pattern assumed when a vehicle approaching in front or an interrupting vehicle exists is extracted as a front vehicle direction pattern FD.

Finally, the front vehicle start detection means 208 detects the front start vehicle based on the front motion vector in the front vehicle start direction pattern FP (step S8), and the front approaching vehicle detection means 209 detects the front vehicle direction. A front approaching vehicle is detected based on the front motion vector in the pattern FD (step S
9). In this way, the behavior (starting or approaching) of the vehicle ahead can be detected.

Next, the specific positions of the front calculation area Rf, the front block Bf and the like set for the front image Gf will be described with reference to the explanatory view of FIG. Figure 8
, V and H are the numbers of pixels in the vertical and horizontal directions of the image Gf, RWf1 is the front motion vector monitoring area for the left front interrupt vehicle, RWf2 is the front motion vector monitoring area for the right front interrupt vehicle, and RWf3 is It is a forward motion vector monitoring area (or a forward starting motion vector monitoring area) for a vehicle approaching in front (or a vehicle starting ahead).

In this case, 3 in one forward calculation area Rf.
It shows a case where one forward motion vector monitoring area RWf1 to RWf3 is set. The front calculation region Rf occupies a region from the bottom to the 3/4 of the image vertical direction V, and the front motion vector monitoring regions RWf1 and RWf2 for the interrupting vehicle are from the position 1/6 to the bottom of the image vertical direction V. To ¼, the forward motion vector monitoring area RWf3 for approaching vehicles occupies the central half area in the horizontal direction H of the image in the forward calculation area RWf.

The front block Bf limited to calculate the front motion vector has a size of 10 × 10 pixels, and divides the front calculation region Rf in the front image Gf into a plurality of parts. A forward motion vector is calculated within these forward blocks Bf. Further, in the detection of the forward motion vector Vf in the forward block Bf (step S3) described above, a known all-point matching method is used.

The forward image G will be described below with reference to the explanatory diagram of FIG. 9 showing the processing operation of the forward motion vector computing means 203.
The all-point matching method when the person image in f is the detection target will be described. In FIG. 9, (a) shows time t = t.
1, the previous image Gf ′ serving as the standard image in FIG. 1, (b) is the current image Gf serving as the reference image at time t = t1 + Δt,
(C) is an image of the extracted forward motion vector.

Further, 20 'and 20 are human images to be detected in the images Gf' and Gf, and Q '(i, j) and Q (p, q) are front images including the human images 20' and 20. An upper left point representing the positions of blocks Bf 'and Bf,
Reference numeral 21 denotes a search range having a size of ± u × ± v pixels with respect to the position Q ′ (i, j) of the front block Bf ′, R (i−
u, j-v) is the upper left point indicating the position of the search range 21. Here, the sizes of the front blocks Bf ′ and Bf are generally WH × WV pixels.

Now, as shown in FIG. 9A, the previous image Gf '
At the upper point Q '(i, j), a front block Bf' for front motion vector calculation is set so as to surround the human image 20 ', and at the next photographing timing, as shown in FIG. 9 (b), It is assumed that the person image 20 moves on the current image Gf.

At this time, the search range 21 is set on the current image Gf with respect to the position Q ′ (i, j) of the front block in the previous image, and the search range 21 is set to the front of the size of WH × WV pixels. While moving the block Bf, the sum of the absolute values of the differences between the image signal of each pixel in the front block Bf ′ on the previous image and the image signal of each pixel in the front block Bf on the current image is calculated.

For example, the point S (x, on the previous image Gf '
The image signal of y) is Sxy, and the point T (x,
The image signal of y) is set as Txy, and each image signal Sxy and Tx
Letting Akm be the sum of the absolute values of the differences with y, the following equation is obtained.

Akm = ΣΣ | Sxy-Tx + k, y + m |

However, in each summation term in the above equation, x = i, i + 1, ..., i + WH, y = j, j +, respectively.
It is assumed that the calculation is performed on 1, ..., j + WV. Further, k is a value within the range of −u <k <u, and m is −v <
It is a value within the range of m <v.

In the above equation, Akm is obtained by sequentially changing the values of k and m, and the point Q at which Akm becomes the minimum is obtained.
Let (p, q) be the point that most closely matches the image in the front block Bf '. Therefore, as shown in FIG. 9C, the point Q '
The movement vector from (i, j) to the point Q (p, q) can be detected as the front movement vector Vf of the image movement of the person image 20 in the front blocks Bf ′ and Bf.

Next, a specific example of the processing operation of the forward motion vector direction detecting means 204 will be described with reference to the explanatory view of FIG. For example, the forward motion vector direction detecting means 204 sets a certain threshold value (predetermined value) for the forward motion vector Vf obtained as shown in FIG. 9C, and only for the forward motion vector Vf equal to or more than the threshold value, The forward motion vector direction Df is quantized and output, and the magnitudes of the front motion vectors equal to or less than the threshold value are set to 0.

In FIG. 10A, Df0 to Df7 are eight directions indicating the forward motion vector direction Df. Further, in FIG. 10B, 30 is a forward vehicle on the current image Gf. 50 to 70 are backgrounds on the front calculation area Rf in the current image Gf, 50 is an adjacent driving lane, 55 is a sign arranged along the adjacent driving lane 50, 60 is a line on the lane, and 70 is a guardrail. Is.

In this case, the forward motion vector direction of the forward vehicle 30 is Df3, the forward motion vector direction of the adjacent lane 50 and the sign 55 is Df5, the calculated motion vector direction of the line 60 is Df6, and the forward motion vector direction of the guardrail 70 is It is Df7.

Here, by setting the threshold value of the front motion vector Vf detected in the front calculation area Rf to, for example, √2 times the size of the pixel, the image sensor characteristics of the front photographing means 201 and the image sensor. The front motion vector Vf corresponding to the subtle shaking of the front image Gf caused by the slight shaking of the vehicle equipped with is regarded as background noise and becomes “0”. Forward motion vector direction D thus obtained
The relationship between f and the current image Gf is as shown in FIG.

As shown in FIG. 10B, the approaching vehicle (interrupting vehicle) is detected by the forward motion vector direction detecting means 204.
Direction of the front vehicle 30 such as the front vehicle 30, the background 55 having a high contrast such as the sign 55 and the line 60, etc.
3, Df5 to Df7 only are detected and output.

Next, referring to the explanatory view of FIG. 11 together with FIGS. 8 to 10, the forward vehicle direction pattern extraction means 2
A specific example of the processing operation of 07 will be described. The processing operation of the forward vehicle direction pattern extraction unit 207 is performed by the type of movement of the forward vehicle 30 to be detected, that is, the direction pattern of the forward motion vector Vf extracted by the approaching vehicle, the front interrupt vehicle, or the start vehicle. Is different,
Here, first, the operation of extracting the direction pattern of the interrupting vehicle will be described.

The forward vehicle direction pattern extracting means 207 is
Forward motion vector Vf obtained by the forward motion vector calculation means 203 and the forward motion vector direction detection means 204
On the other hand, first, to the left and right, the forward motion vector monitoring area RWf1
And RWf2 (see FIG. 8) are set. Then, within the front motion vector monitoring areas RWf1 and RWf2, the front motion vector V having the same direction as the motion of the front interrupt vehicle
Extract f.

For example, an image Gf as shown in FIG.
On the other hand, in the forward motion vector monitoring area RWf1 on the left side, the forward motion vector Vf having the direction of the forward motion vector direction Df3 or Df4 (see FIG. 10A) in which the forward interrupt vehicle is assumed to move is interrupted in the forward direction. Extract as a vehicle direction pattern. Further, in the forward motion vector monitoring area RWf2 on the right side, the forward motion vector Vf having the direction of the forward motion vector direction Df0 or Df1 in which the forward motion vehicle is assumed to move is extracted as the forward motion vector direction pattern.

FIG. 1 shows a two-dimensional image of the front block Bf including the front motion vector Vf thus extracted.
It is the image Gf11 in 1. As shown in FIG. 11, in the front motion vector monitoring area RWf1 on the left side, the front motion vector Vf31 around the interrupting vehicle (for example, the front vehicle 30 in FIG. 10B) is extracted.

Next, referring to FIGS. 8 to 10 and the explanatory diagrams of FIGS. 12 and 13, the front vehicle direction pattern extracting means 207 extracts the direction pattern of the front approaching vehicle, and the front vehicle starting direction pattern. Extraction means 206
A specific example of the direction pattern extraction operation of the forward starting vehicle according to will be described. Here, the operation of extracting the front vehicle starting direction pattern FP by the front vehicle starting direction pattern extracting means 206 will be representatively described.

In FIG. 12, reference numeral 32 denotes a forward vehicle detected in the forward motion vector monitoring area RWf3 (see FIG. 8). Further, in FIG. 13, RWf31 to RWf34
Is a divided area for extracting the vertical motion vector Vf set in the forward motion vector monitoring area RWf3 in the vertical and horizontal directions, and RVF is the extraction of the forward motion vector Vf equal to or greater than the threshold value detected in the forward motion vector monitoring area RWf3. Area.

First, the front calculation area Rf in the front image Gf
A forward motion vector monitoring area RWf3 is set above, and in this forward motion vector monitoring area RWf3, the forward motion vector calculation means 203 and the forward motion vector direction detection means 204.
Based on the forward motion vector Vf extracted by, a direction similar to the direction of motion of the forward starting vehicle (or forward approaching vehicle) is extracted. That is, the forward motion vector Vf having all the directions Df0 to Df7 indicating the convergent (or divergent) direction in FIG. 10A is extracted.

Here, when the forward vehicle 32 is a forward starting vehicle, the movement on the forward image Gf moves away from the own vehicle, and therefore becomes inward (converging) as shown by the solid arrow in FIG. In the case of an approaching vehicle, since the vehicle approaches the own vehicle, the direction is an outward (divergent) direction as indicated by a dashed arrow.

Therefore, as shown in FIG. 13A, four divided regions R are provided in the forward motion vector direction monitoring region RWf3.
Wf31 to 34 are set, and a forward motion vector V equal to or larger than a threshold value in the same direction as the motion of a vehicle starting from the front (or a vehicle approaching in the front) is set.
Extract f.

For example, in the case of a vehicle starting ahead, the divided region R
The forward motion vector in the downward direction is extracted in Wf31, and the forward motion vector in the upward direction is extracted in the divided region RWf32.
The forward motion vector in the right direction is extracted in the divided region RWf33, and the forward motion vector in the left direction is extracted in the divided region RWf34. The forward motion vector Vf thus extracted becomes the extraction region RVF in FIG.

In this case, the extraction region RVF of the front motion vector Vf becomes the front vehicle starting direction pattern FP, and from this front vehicle starting direction pattern FP, for example, only images of three consecutive pixels or more are extracted to perform noise removal processing. When applied and imaged, it becomes as shown in FIG.

In this way, the front vehicle starting vehicle can be detected as follows based on the direction pattern FP of the front vehicle motion vector Vf extracted by the front vehicle starting direction pattern extracting means 206. Further, in the same manner, the direction pattern FD of the front approaching vehicle including the front interrupting vehicle is extracted, and the front approaching vehicle or the like can be detected.

That is, in the forward motion vector monitoring areas RWf1 and RWf2 for the interrupting vehicle and in the forward motion vector monitoring area RWf3 for the approaching vehicle (or the starting vehicle) (see FIG. 8), almost the same as the vehicle to be detected. A vehicle detection window having a size is set, and this window is moved within the forward motion vector direction monitoring areas RWf1 to RWf3.

Then, a vehicle ahead (or an interruption vehicle)
The area included in the window of the direction pattern FD of the front motion vector Vf extracted by the direction pattern extraction unit 207 (or the direction pattern FP of the front motion vector Vf extracted by the front vehicle start direction pattern extraction unit 206) is obtained. A position at which this area is equal to or larger than a certain threshold value is specified as a region in which a front vehicle (front interrupt vehicle, front approach vehicle, or front start vehicle) exists.

Similarly to the front monitoring device 200 described above, in the rear monitoring device 210 and the side monitoring device 220,
The respective motion vectors Vr and V obtained by the backward motion vector calculation means 213 and the side motion vector calculation means 223.
Based on s, the same image movement direction and size of the detection target vehicle are extracted. Then, based on the position, area, etc. of the extracted motion vector, by judging whether or not it is the extraction target vehicle, even in a complicated background, a rear approaching vehicle, a side overtaking vehicle, and a side-by-side vehicle are detected. The running vehicle can be detected and specified more accurately.

As described above, in each of the monitoring devices 200, 210 and 220 according to the first embodiment of the present invention, the photographed images Gf from the photographing means 201, 211 and 221 are detected by the motion vector calculating means 203, 213 and 223. , G
The calculation areas Rf, Rr, and Rs are limited to r and Gs, and blocks Bf, Br, and Bs are further defined in the calculation area.
And the motion vectors Vf, Vr and V within the block.
Calculate s.

As a result, it is possible to eliminate the erroneous detection by eliminating the portion that seems to be the background from the beginning, suppress the erroneous detection of the motion vector due to the image shake and the complicated background, and shorten the processing time. In addition, even in an image in which the background also moves, such as an image taken from a moving body called a traveling vehicle, it is possible to easily separate and extract only an object (vehicle) moving in a certain direction from the image. You can

Further, each direction pattern extracting means 207, 2
Reference numerals 17 and 227 denote one or more motion vector monitoring areas RWf, R at predetermined positions where the detection target vehicle appears.
Wr and RWs are set, and a motion vector having a direction pattern similar to the image motion direction of each detection target vehicle and having a predetermined value or more is extracted from the motion vector detected in the motion vector direction monitoring area. Thereby, the moving body (vehicle to be detected) to be extracted can be separated from the background.

Therefore, a high-intensity motion vector caused by the shaking of the image caused by the bounce due to the unevenness of the road and the complicated background is removed, and a front approaching vehicle (interrupting vehicle), a rear approaching vehicle, or a side overtaking vehicle is removed. It can be detected easily and accurately.

Further, in the front monitoring device 200, the front motion vector Vf indicating the magnitude and direction of the apparent motion of the block image in the front calculation area Rf is obtained, and the front vehicle starting direction pattern extraction means 206 determines the own vehicle. When the vehicle is parked or stopped, one or more front vehicle start (forward) motion vector monitoring area RWf is set at a predetermined position in the front vehicle and its surroundings. A forward motion vector Vf candidate having a direction pattern similar to the starting image motion direction is extracted.

As a result, it is possible to distinguish the image movement due to the shaking from the image movement due to the start, and to accurately detect the image movement due to the start of the front vehicle (change in relative position between the front vehicle and the host vehicle). That is, in the related art, the relative position detecting device with respect to the vehicle ahead uses the difference between the images, and only knows the presence or absence of the motion on the image, whereas the first embodiment of the present invention uses the motion vector to detect the image. Since the apparent magnitude and direction of the movement of the vehicle can be known, the front vehicle can be started more accurately by distinguishing the shaking of the image due to the shaking of the stopped vehicle and the preceding vehicle and the relative position change caused by the starting of the preceding vehicle. You can

Further, in the side monitoring device 220, one or more small motion vector monitoring areas Rss are located at a predetermined position where a vehicle having a specific motion (side parallel running vehicle) appears on the side image Gs. And the background vector deleting means 226
Deletes the lateral motion vector having the same directional pattern as the motion on the background image in the small motion vector monitoring region Rss from the small motion vector Vss detected by the small motion vector direction extraction means 225. Accordingly, a lateral parallel vehicle that does not move much on the complex background image can be separated from the background and accurately and easily detected.

[0115]

As described above, according to claim 1 of the present invention, the photographing means for photographing the surroundings of the own vehicle to generate a plurality of images which are different in time series, and the predetermined positions on the plurality of images. And a motion vector calculating means for setting a calculation region for each of the calculation regions, dividing the calculation region into a plurality of blocks, and calculating an image motion in each block as a motion vector; And a motion vector direction detecting means for outputting the direction of the motion vector having a magnitude of, and a motion vector monitoring region is further set in the calculation region, and the motion vector direction in the block is monitored in the motion vector monitoring region, Moving body direction pattern extraction that extracts a combination of blocks of motion vectors having the same direction as the direction of image movement when a moving body exists around the vehicle as a moving body direction pattern Means and a moving body detecting means for detecting the presence of a moving body in the vicinity of the own vehicle from the moving vector in the block extracted by the moving body direction pattern extracting means, and further the moving vector in the moving vector calculation area in the captured image. By limiting the monitoring area and detecting only the motion vector of a predetermined value or more from the blocks in the motion vector monitoring area, the part that seems to be the background is excluded from the beginning.
Even when the host vehicle is traveling, it suppresses image shake caused by bounces due to road surface irregularities and erroneous detection of motion vectors caused by a complicated background peculiar to the driving environment, and ensures reliable and short-term behavior of moving objects around the host vehicle. Therefore, there is an effect that a vehicle surroundings monitoring device that can be detected can be obtained.

Further, according to claim 2 of the present invention, a front photographing means for photographing the front of the running vehicle to generate a plurality of front images different in time series, and a predetermined front image on the plurality of front images. And a front motion vector calculating means for dividing the front calculation region into a plurality of front blocks and calculating the front image motion in each front block as a front motion vector, and the front motion vector. Among these, a forward motion vector direction detecting means for outputting a direction of a forward motion vector having a magnitude larger than a predetermined value in the forward calculation region, and a forward motion vector monitoring region further set in the forward calculation region, and a forward motion vector monitoring region is set. The direction of the forward motion vector in the forward block is monitored in the vector monitoring area, and the direction of the forward motion vector that has the same direction as the direction of the forward image motion when there is a vehicle approaching ahead is detected. Front vehicle direction pattern extraction means for extracting a combination of the front blocks as a front vehicle direction pattern, and front approach vehicle detection for detecting the presence of a front approach vehicle from the front motion vector in the front block extracted by the front vehicle direction pattern extraction means. With the output means, only the forward motion vector of a predetermined value or more is detected in the moving direction of the approaching vehicle from the block in the forward motion vector monitoring area, and the part that seems to be the background is excluded from the beginning.
There is an effect that a vehicle surroundings monitoring device can be obtained which can suppress erroneous detection of a motion vector due to a sway of the host vehicle or a complicated background and can reliably detect a vehicle approaching in front of the host vehicle in a short time.

According to claim 3 of the present invention, in claim 2, the own vehicle stop judging means for judging the stopped state of the own vehicle based on the direction of the forward motion vector, and a response to the stopped state of the own vehicle. Front vehicle starting direction pattern extracting means for extracting the front vehicle starting direction pattern and front vehicle starting direction detecting means for detecting the front vehicle starting from the forward motion vector in the front vehicle starting direction pattern, and front vehicle starting direction pattern extraction The means is a front vehicle start vector monitoring area setting means for setting a front vehicle start vector monitoring area of the front start vehicle in the front calculation area when the host vehicle is stopped, and a front block in the front vehicle start vector monitoring area. The direction of the forward motion vector inside the vehicle is monitored, and a combination of front blocks that have a forward motion vector in the same direction as the direction of the forward image movement by the forward starting vehicle is generated as the front vehicle starting direction pattern. Front vehicle starting direction pattern detection means, which detects the apparent magnitude and direction of the motion vector from the block when the vehicle is stopped in the front motion vector monitoring area, and a predetermined value or more in the moving direction of the front starting vehicle. Only the forward motion vector of the vehicle is extracted as a starting vehicle start candidate, and the part that seems to be the background is excluded from the beginning.Therefore, false detection of the motion vector due to the shaking of the own vehicle or the complicated background is suppressed, and There is an effect that a vehicle surroundings monitoring device can be obtained which can reliably detect a forward starting vehicle in a short time based on a change in relative position between the vehicle and the preceding vehicle.

Further, according to claim 4 of the present invention, in claim 2 or 3, the rear photographing means for photographing the rear of the running vehicle and generating a plurality of rear images which are different in time series. And a backward calculation area is set at a predetermined position on each of the plurality of rear images, the rear calculation area is divided into a plurality of rear blocks, and the rear image motion in each rear block is calculated as a rear motion vector. A motion vector calculating means, a backward motion vector direction detecting means for outputting a direction of a backward motion vector having a magnitude larger than a predetermined value in the backward calculation area among the backward motion vectors, and a backward motion vector monitoring in the backward calculation area. While setting the area, monitor the direction of the backward motion vector in the backward motion vector monitoring area,
Rear approaching vehicle direction pattern extraction means for extracting as a rearward approaching vehicle direction pattern a combination of rearward blocks of a rearward motion vector having a direction similar to the direction of the rearward image movement assumed when a rearward approaching vehicle is present; A rear approaching vehicle detecting means for detecting the presence of a rear approaching vehicle from a rearward moving vector in the vehicle direction pattern, and only a rearward moving vector having a predetermined value or more in the moving direction of the rearward approaching vehicle from the block in the rearward moving vector monitoring area. Since it was designed to eliminate the part that seems to be the background from the beginning, erroneous detection of the motion vector due to the shaking of the own vehicle or the complicated background is suppressed, and the vehicle approaching to the rear of the own vehicle can be reliably and There is an effect that a vehicle periphery monitoring device that can detect in a short time can be obtained.

According to a fifth aspect of the present invention, in any one of the second to fourth aspects, a side image of the own vehicle in motion is photographed and a plurality of side images different in time series. And a side calculation area for generating a side calculation area, and a side calculation area is set at a predetermined position on each of the plurality of side images, and the side calculation area is divided into a plurality of side blocks. Side motion vector calculating means for calculating the side image motion of the side motion vector as a side motion vector, and outputting the direction of the side motion vector having a magnitude larger than a predetermined value within the side calculation region among the side motion vectors. The side motion vector direction detecting means and the side motion vector monitoring region are set in the side calculation region, and the direction of the side motion vector is monitored in the side motion vector monitoring region to detect the side overtaking vehicle. Of the lateral image motion that is assumed if present From the lateral overtaking vehicle direction pattern extraction means for extracting the combination of the lateral blocks of the lateral motion vector having the same direction as the lateral direction as the lateral overtaking vehicle direction pattern, and the lateral motion vector in the lateral overtaking vehicle direction pattern It is equipped with a lateral overtaking vehicle detecting means for detecting the presence of a lateral overtaking vehicle, and detects only a lateral motion vector of a predetermined value or more in the moving direction of the lateral overtaking vehicle from a block in the lateral motion vector monitoring area. Since the part that seems to be the background is eliminated from the beginning, erroneous detection of the motion vector due to the shaking of the own vehicle or the complicated background is suppressed, and the vehicle that laterally overtakes the own vehicle can be reliably and quickly. There is an effect that a vehicle surroundings monitoring device capable of detecting is obtained.

According to claim 6 of the present invention, in claim 5, a small motion vector monitoring area is set in the lateral calculation area, and a predetermined one of the lateral motion vectors is set in the small motion vector monitoring area. Small motion vector direction extraction means for extracting a direction of a side motion vector having a size equal to or smaller than a value as a small motion vector direction, and a small motion vector direction having a direction similar to the background motion from the small motion vector direction is excluded. The background vector deleting means and the side-by-side running vehicle detection means for detecting the presence of a side vehicle having a small relative speed from the position and size of the side motion vector from which the background vector is deleted are provided, and the small motion vector monitoring is provided. Since only the small motion vector of a predetermined value or less is detected in the moving direction of the lateral parallel departure from the block in the area, the background vector having the same direction pattern as the background image motion is deleted. A vehicle periphery monitoring device capable of suppressing erroneous detection of a motion vector due to vehicle sway or a complicated background and reliably detecting a side-by-side running vehicle with little motion on an image in a short time is obtained. effective.

[Brief description of drawings]

FIG. 1 is a functional block diagram showing a configuration of a front monitoring device according to a first embodiment of the present invention.

FIG. 2 is a functional block diagram showing a specific configuration example of a forward motion vector calculation means in FIG.

FIG. 3 is a functional block diagram showing a specific configuration example of a forward starting direction pattern extraction means in FIG.

FIG. 4 is a functional block diagram showing a configuration of a rear monitoring device according to the first embodiment of the present invention.

FIG. 5 is a functional block diagram showing a configuration of a side monitoring device according to the first embodiment of the present invention.

FIG. 6 is a functional block diagram showing a specific configuration example of the small motion vector extraction means in FIG.

FIG. 7 is a flowchart showing an example of the operation of the front monitoring device according to the first embodiment of the present invention.

FIG. 8 is an explanatory diagram showing a front motion vector calculation region, a front motion vector monitoring region, and a front block set by the front monitoring device according to the first embodiment of the present invention.

FIG. 9 is an explanatory diagram showing the operation of the forward motion vector calculating means in the forward monitoring apparatus in Embodiment 1 of the present invention.

FIG. 10 is an explanatory diagram showing a forward motion vector detecting operation by the forward motion vector direction detecting means in the forward monitoring device in Embodiment 1 of the present invention, (a) is an explanatory view showing a forward motion vector direction, (b) ) Is an explanatory view showing an example of a front image.

FIG. 11 is an explanatory diagram showing an example of a direction pattern extraction result of a front interruption vehicle by the front monitoring device according to the first embodiment of the present invention.

FIG. 12 is an explanatory diagram showing a direction monitoring region of a front approaching vehicle or a front starting vehicle and a movement direction on an image by the front monitoring device according to the first embodiment of the present invention.

FIG. 13 is an explanatory diagram showing an example of a direction pattern extracting operation of a front approaching vehicle or a front starting vehicle by the front monitoring device according to the first embodiment of the present invention.

FIG. 14 is a functional block diagram showing an obstacle recognition device as a conventional vehicle surroundings monitoring device.

FIG. 15 is an explanatory diagram showing an image state in a processing process by each unit in FIG.

[Explanation of symbols]

200 front monitoring device 201 front photographing means 203 front motion vector calculation means 204 front motion vector direction detection means 205 own vehicle stop determination means 206 front vehicle start direction pattern extraction means 206a front vehicle start movement vector monitoring area setting means 206b front vehicle start Vector direction monitoring means 206c Front vehicle starting direction pattern detecting means 207 Front vehicle direction pattern extracting means 208 Front vehicle starting detecting means 209 Front approaching vehicle detecting means 210 Rear monitoring device 211 Rear photographing means 213 Rear moving vector computing means 214 Rear moving vector direction Detecting means 217 Rear approaching vehicle direction pattern extracting means 219 Rear approaching vehicle detecting means 220 Side monitoring device 221 Side photographing means 223 Side moving vector calculating means 224 Side moving vector direction detecting means 225 Small moving vector direction extracting means 22 5a Small motion vector monitoring area setting means 225b Small motion vector direction detecting means 226 Background vector deleting means 227 Side overtaking vehicle direction pattern extracting means 228 Side parallel running vehicle detecting means 229 Side overtaking vehicle detecting means Bf ', Bf Front block Df Forward motion vector direction Dr Rear motion vector direction Ds Side motion vector direction Dss Small motion vector direction FD Front vehicle direction pattern FP Front vehicle start direction pattern Gf ', Gf Front image Gr', Gr Rear image Gs', Gs Sideways Image P Stopping state determination result Rf ', Rf Forward calculation area RP Rear approaching vehicle direction pattern Rs'Rs Side calculation area Rss', Rss Small motion vector monitoring area RWf', RWf Forward motion vector monitoring area (front vehicle start vector Surveillance area) SP Lateral overtaking vehicle direction pattern Vf Forward Vector Vr Rearward motion vector Vs Sideward motion vector Vs' Sideward motion vector with background vector deleted S1 Forward shooting step S3 Forward motion vector calculation step S4 Forward motion vector direction detection step S5 Own vehicle stop determination step S6 Front vehicle start direction Pattern extraction step S7 Forward vehicle direction pattern extraction step S8 Front vehicle start detection step S9 Forward approaching vehicle detection step

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Internal reference number FI Technical display location G06T 1/00 7/20 G08G 1/04 C 7531-3H H04N 7/18 J

Claims (6)

[Claims] 1. A photographing means for photographing the surroundings of a host vehicle to generate a plurality of images that are different in time series, and a calculation area is set at a predetermined position on each of the plurality of images. A motion vector calculating unit that divides each block into image motions in each block as a motion vector; and a direction of the motion vector having a magnitude of a predetermined value or more in the calculation region among the motion vectors. A moving vector direction detecting means for outputting, and a moving vector monitoring area is further set in the calculation area, and a moving vector direction in the block is monitored in the moving vector monitoring area, and a moving body is provided around the own vehicle. Moving body direction pattern extraction for extracting, as a moving body direction pattern, a combination of blocks of motion vectors having the same direction as the image motion direction when A vehicle surroundings monitoring device comprising: means and a moving body detecting means for detecting the presence of the moving body from the motion vector in the block extracted by the moving body direction pattern extracting means. 2. A front photographing means for photographing a front of a running vehicle to generate a plurality of front images which are different in time series, and a front calculation area is set at a predetermined position on each of the plurality of front images. In addition, the front calculation area is divided into a plurality of front blocks, and front motion vector calculation means for calculating a front image motion in each front block as a front motion vector; A forward motion vector direction detecting means for outputting a direction of a forward motion vector having a size equal to or larger than a predetermined value in the region; and a forward motion vector monitoring region set in the forward calculation region, and the forward motion vector monitoring region. The forward motion vector direction in the front block is monitored, and the forward motion vector has the same direction as the direction of the forward image motion when a front approaching vehicle is present. The presence of the front approaching vehicle is detected from the front vehicle direction pattern extracting means for extracting a combination of the front blocks of the vehicle as a front vehicle direction pattern and the front motion vector in the front block extracted by the front vehicle direction pattern extracting means. A vehicle surroundings monitoring device including a front approaching vehicle detecting unit. 3. A host vehicle stop determination means for determining a stop state of the host vehicle based on the forward motion vector direction, and a front vehicle start for extracting a front vehicle start direction pattern in response to the stop state of the host vehicle. Directional pattern extraction means, comprising a front vehicle start detection means for detecting the front vehicle start from the forward motion vector in the front vehicle start direction pattern, the front vehicle start direction pattern extraction means, when the vehicle is stopped, Front vehicle start vector monitoring area setting means for setting a front vehicle start vector monitoring area of a front start vehicle in the front calculation area, and a front motion vector direction in the front block in the front vehicle start vector monitoring area And a front vehicle that generates a combination of front blocks having a front motion vector in a direction similar to the direction of the forward image movement by the front starting vehicle as a front vehicle starting direction pattern. Advancing direction pattern detecting means and the vehicle periphery monitoring apparatus according to claim 2, characterized in that it comprises a. 4. A rear photographing means for photographing the rear of a moving vehicle to generate a plurality of rear images which are different in time series, and a rear calculation area is set at a predetermined position on each of the plurality of rear images. In addition, the backward calculation area is divided into a plurality of backward blocks, and backward motion vector calculation means for calculating a backward image motion in each of the backward blocks as a backward motion vector; A backward motion vector direction detecting means for outputting a direction of a backward motion vector having a size equal to or larger than a predetermined value in the area, and a backward motion vector monitoring area set in the backward calculation area, and in the backward motion vector monitoring area The direction of the rearward motion vector is monitored with, and the rearward direction of the rearward motion vector having the same direction as the direction of the rearward image movement assumed when a rear approaching vehicle exists is detected. Rear approaching vehicle direction pattern extracting means for extracting a combination of locks as a rear approaching vehicle direction pattern, and rear approaching vehicle detecting means for detecting the presence of the rear approaching vehicle from a rear motion vector in the rear approaching vehicle direction pattern. The vehicle periphery monitoring device according to claim 2 or 3, characterized in that. 5. A side photographing means for photographing a side of a running vehicle to generate a plurality of side images which are different in time series, and side photographing means at respective predetermined positions on the plurality of side images. Side motion vector calculation means for setting a side calculation region, dividing the side calculation region into a plurality of side blocks, and calculating a side image motion in each side block as a side motion vector; Among the lateral motion vectors, a lateral motion vector direction detecting means for outputting a direction of the lateral motion vector having a magnitude equal to or larger than a predetermined value in the lateral calculation area, and a side in the lateral calculation area. The direction of the lateral motion vector is monitored in the lateral motion vector monitoring area while setting the lateral motion vector monitoring area, and is the same as the direction of the lateral image motion assumed when a lateral overtaking vehicle exists. Of the lateral motion vector with the direction of Lateral overtaking vehicle direction pattern extraction means for extracting a combination of locks as a lateral overtaking vehicle direction pattern, and lateral overtaking for detecting the presence of the lateral overtaking vehicle from the lateral motion vector in the lateral overtaking vehicle direction pattern The vehicle surroundings monitoring device according to any one of claims 2 to 4, further comprising a vehicle detection unit. 6. A small motion vector monitoring area is set in the lateral calculation area, and in the small motion vector monitoring area, a lateral motion vector having a magnitude of a predetermined value or less among the lateral motion vectors is set. A small motion vector direction extracting means for extracting a direction as a small motion vector direction; a background vector deleting means for excluding a small motion vector direction having a direction similar to the motion of the background from the small motion vector direction; and the background vector being deleted. 6. The vehicle surroundings monitoring device according to claim 5, further comprising: a side-by-side vehicle detecting unit that detects the presence of a side vehicle having a small relative speed from the position and magnitude of the generated side motion vector. .